mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 06:10:44 +01:00
DottaandClaude Opus 5.5
988a4b6749
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A control-plane restart leaves in-progress records owned by dead processes, and about 31 per-subsystem proofs decide when to release them > - The durable ownership plan (section C) asks for one registry of ownable record kinds and one reconciler > - Phase 1 must measure first: how many in-progress records exist, who owns them, and which invariants they break, with no behavior change > - This pull request registers 46 kinds from the Phase 0 inventory's section C and adds a report-only pass to the existing native finalization reconciler > - The benefit is per-kind invariant-violation counts from chaos waves and staging. That data unlocks Phase 2 (moving kinds onto one settle path) and the removal of the company-wide admission gate ## Linked Issues or Issue Description Refs #19831 (plan, section C and Phase 1). Refs #19846 (Phase 0 inventory, section C). Stacked on #19853 (the prover) and #19851 (one process identity). **Problem** No single place lists the control plane's in-progress records or checks the plan's invariants. Each subsystem releases its own records by its own rules, so nobody can count how many records break "owned", "settled" or "visible next action". **Expected behavior** One report-only pass lists every registered kind, classifies each owner as gone, alive or unprovable, and records each invariant violation once. It changes no record and stays quiet when nothing is in progress. **Impact** Phase 2 moves kinds onto the shared settle path in the order these counts show. Phase 3 removes the admission gate once the I3 data exists. ## What Changed - New `server/src/services/recovery/ownership-registry.ts`: 46 `OwnableKind` entries (`id`, `obligation`, `storage`, `listInProgress`, `settle: null`, `invariantChecks`). Each record names its owner as a process stamp, a fence token, a run, an operator, or none. The skill-cache lock is registered with a named coverage gap because its lock files live in per-skill roots. - New `server/src/services/recovery/ownership-invariant-report.ts`: one pass classifies every owner with `proveOwnerGone` (or the owning run's status), then runs: - I1 owned: an in-progress record with no owner; - I2 settled: an owner proven gone on two consecutive passes while the record is still in progress; - I3 admission independence: terminal runs whose accounting outlives the 10 min closure, which hold company admission; - I4 exactly-once intents: wake requests ended `skipped`/`failed` on a temporary refusal (heuristic until section E); - I7 visible next action: an agent-assigned `in_progress` issue with no live run, no pending wake and no open recovery action for 5 min. - I5, I6 and I8 are reported as named coverage gaps; I8 comes in the speed-budget PR. - Violations are written once per process to the activity log as `recovery.invariant_violation` (`actorId: ownership-reconciler`, `details.mode: report_only`), insert-only with no live-event or plugin fan-out. Nothing goes to Telemetry (AGENTS.md §5.7). - `native-finalization-reconciler.ts`: `registerOwnershipReconciliationWork` runs the pass in the shared delivery-work scheduler (new queue `ownership-invariant-report`). There is no second loop. - `index.ts` registers it after residual recovery and does not await its first pass, so readiness is not delayed. ### Sleep-aware trigger (rules/sleep-aware-background-work.mdc) - **Trigger:** one pass at registration (startup reconstruction from durable truth). After that, a settled `active-execution-recovery` signal, which run lifecycle, controller renewal and process-evidence writes already emit. Signals are coalesced to at most one pass per 30 s. No producer gained a write. - **Pending predicate and next deadline:** a 30 s follow-up only while the last pass saw a changing in-progress set, or an owner proven gone that needs the second I2 pass. - **Quiet:** an empty set or the same stuck set twice sets no deadline. The test asserts `nextWakeAt() === null` after an idle pass. - **Turn path:** the pass never runs before `turn.start`. Stamps read only cached identity. ## Verification - `npx vitest run src/__tests__/ownership-reconciliation-report.test.ts` (embedded Postgres, in `server/`): 4 tests pass: - every kind is registered once with no settle path; - verdicts per class: alive (this process's claim), gone (lease of a failed run), unprovable (no owner; another PID space); - each rule fires on its fixture, and I2 fires only on the second pass; - every touched table is byte-identical before and after (report-only); - activity rows are written once per violation, and none are written with `record: false`; - the worker is quiet when idle and schedules a follow-up only while records change. - `npx vitest run src/__tests__/delivery-work-coordinator.test.ts src/services/owner-proof.test.ts src/__tests__/server-startup-feedback-export.test.ts`: pass. - `npx tsc --noEmit -p server`: no errors. ## Risks - No record changes, so no settle risk. The risk is load: one pass runs about 46 small indexed or `limit 201` queries, at most once per 30 s while runs are active, and none when idle. - I4 matches refusal text with a heuristic. It can over- or under-count until section E gives wake requests closed terminal reasons. It is labelled as such in the code. - Violations dedupe per process, so a restart writes a still-open violation again (once per boot). Chaos reports count distinct `(rule, entityId)`. - Previous-release story (`rules/upgrade-safe-run-state.mdc`): no persisted run state changes. The pass reads rows of any age, and a missing field classifies as unprovable. - `heartbeat_runs` and `native_run_finalizations` record no host or PID space, so foreign-boot owners there classify as unprovable (`missing_identity`). This is the inventory's finding 2, and the report makes it countable. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
171 lines
8.0 KiB
TypeScript
171 lines
8.0 KiB
TypeScript
import { signalDatabaseWork, subscribeDatabaseWork } from "@paperclipai/db/work-signals";
|
|
|
|
/** Topics for existing durable queues; register intent before their writes. */
|
|
export const DELIVERY_QUEUES = {
|
|
mcpClientMirror: "mcp-client-mirror",
|
|
mcpIngressHandoffs: "mcp-ingress-handoffs",
|
|
mcpRevocationIndex: "mcp-revocation-index",
|
|
nativeRuntimeResponses: "native-runtime-responses",
|
|
nativeCancellation: "native-cancellation",
|
|
deploymentNotifications: "deployment-notifications",
|
|
liveInvalidations: "live-invalidations",
|
|
nativeProcessSessions: "native-process-sessions",
|
|
nativeQuestionCancellations: "native-question-cancellations",
|
|
activeExecutionRecovery: "active-execution-recovery",
|
|
ownershipInvariantReport: "ownership-invariant-report",
|
|
settledNativeProcesses: "settled-native-processes",
|
|
orphanActiveLeases: "orphan-active-leases",
|
|
legacyShutdownSettlement: "legacy-shutdown-settlement",
|
|
strandedAgentStatus: "stranded-agent-status",
|
|
activeRunSilence: "active-run-silence",
|
|
deferredExecutionReceipts: "deferred-execution-receipts",
|
|
taskRetryReceiptSettlement: "task-retry-receipt-settlement",
|
|
nativeStatusWakeIntents: "native-status-wake-intents",
|
|
orphanBlockers: "orphan-blockers",
|
|
activeRecoveryActions: "active-recovery-actions",
|
|
strandedAssignments: "stranded-assignments",
|
|
resolvedDependencies: "resolved-dependencies",
|
|
taskWatchdogs: "task-watchdogs",
|
|
staleIssueLocks: "stale-issue-locks",
|
|
scheduledRetryDispatch: "scheduled-retry-dispatch",
|
|
queuedRunDispatch: "queued-run-dispatch",
|
|
statusCardEvaluation: "status-card-evaluation",
|
|
runEvidenceMigration: "run-evidence-migration",
|
|
providerTraceRetention: "provider-trace-retention",
|
|
nativeRunnerEvidenceRetention: "native-runner-evidence-retention",
|
|
skillVersionRetention: "skill-version-retention",
|
|
externalObjectRefresh: "external-object-refresh",
|
|
databaseBackups: "database-backups",
|
|
importSpoolCleanup: "import-spool-cleanup",
|
|
externalConversationSettlement: "external-conversation-settlement",
|
|
nativeRecoveryNotices: "native-recovery-notices",
|
|
nativeEnvironmentSettlement: "native-environment-settlement",
|
|
nativeRunProjection: "native-run-projection",
|
|
nativeResultCommit: "native-result-commit",
|
|
nativeWorkspaceFinalization: "native-workspace-finalization",
|
|
retainedNativeSessionCleanup: "retained-native-session-cleanup",
|
|
nativeStatusReassessment: "native-status-reassessment",
|
|
nativeReviewResponse: "native-review-response",
|
|
nativeResponsePresentation: "native-response-presentation",
|
|
nativePendingEffects: "native-pending-effects",
|
|
unsafeWorkspaceExportRecovery: "unsafe-workspace-export-recovery",
|
|
automaticCompletionReviews: "automatic-completion-reviews",
|
|
obsoletePolicyReviews: "obsolete-policy-reviews",
|
|
cancelledWorkspacePreservation: "cancelled-workspace-preservation",
|
|
instructionCollection: "instruction-collection",
|
|
directoryCleanup: "directory-cleanup",
|
|
instructionCommit: "instruction-commit",
|
|
scheduledRetryCancellation: "scheduled-retry-cancellation",
|
|
executionFinalization: "execution-finalization",
|
|
workspaceRecovery: "workspace-recovery",
|
|
executionReplacement: "execution-replacement",
|
|
executionContinuation: "execution-continuation",
|
|
executionStatus: "execution-status",
|
|
executionDisposition: "execution-disposition",
|
|
environmentCleanup: "environment-cleanup",
|
|
approvalQueueSeeding: "approval-queue-seeding",
|
|
attentionRetention: "attention-retention",
|
|
approvalRetention: "approval-retention",
|
|
joinRequestRetention: "join-request-retention",
|
|
decisionTargets: "decision-targets",
|
|
decisionEffectRecovery: "decision-effect-recovery",
|
|
decisionExpiry: "decision-expiry",
|
|
decisionContinuation: "decision-continuation",
|
|
|
|
decisionArchiveDelivery: "decision-archive-delivery",
|
|
toolConnectionHealth: "tool-connection-health",
|
|
toolReviewProjection: "tool-review-projection",
|
|
toolReviewInterruption: "tool-review-interruption",
|
|
toolReviewRecovery: "tool-review-recovery",
|
|
toolReviewExpiry: "tool-review-expiry",
|
|
gatewayCredentialExpiry: "gateway-credential-expiry",
|
|
secretProposalExpiry: "secret-proposal-expiry",
|
|
customImageExpiry: "custom-image-expiry",
|
|
historicalCleanup: "historical-cleanup",
|
|
adapterLoginCleanup: "adapter-login-cleanup",
|
|
setupTokenCleanup: "setup-token-cleanup",
|
|
localLoginCleanup: "local-login-cleanup",
|
|
budgetEnforcement: "budget-enforcement",
|
|
accounting: "cost-accounting",
|
|
taskDrainExpiry: "task-drain-expiry",
|
|
agentLifecycle: "agent-lifecycle",
|
|
chatBoardMessages: "chat-board-messages",
|
|
chatTaskStarts: "chat-task-starts",
|
|
chatReceiptReactions: "chat-receipt-reactions",
|
|
chatSessionStops: "chat-session-stops",
|
|
chatProviderEffects: "chat-provider-effects",
|
|
chatGitHubWebhookRecovery: "chat-github-webhook-recovery",
|
|
chatGitHubIngress: "chat-github-ingress",
|
|
chatGitHubPublications: "chat-github-publications",
|
|
chatGitHubReviewProjection: "chat-github-review-projection",
|
|
chatGitHubReviewChecks: "chat-github-review-checks",
|
|
chatGitHubTerminalResponses: "chat-github-terminal-responses",
|
|
chatGitHubAcknowledgements: "chat-github-acknowledgements",
|
|
chatFailedRunRetries: "chat-failed-run-retries",
|
|
chatTelegramMaintenance: "chat-telegram-maintenance",
|
|
chatReceivers: "chat-receivers",
|
|
voiceSessionRecovery: "voice-session-recovery",
|
|
voiceReports: "voice-call-reports",
|
|
voiceInboundRetention: "voice-inbound-retention",
|
|
voiceInboundCleanup: "voice-inbound-cleanup",
|
|
voiceReplies: "voice-reply-delivery",
|
|
chatTeamsConsentExpiry: "chat-teams-consent-expiry",
|
|
chatTeamsAttemptRecovery: "chat-teams-attempt-recovery",
|
|
chatCommandRegistration: "chat-command-registration",
|
|
chatTelegramSubscriptions: "chat-telegram-subscriptions",
|
|
chatSessionSyncs: "chat-session-syncs",
|
|
chatVerification: "chat-verification",
|
|
chatInbound: "chat-inbound",
|
|
chatPublications: "chat-publications",
|
|
chatNativeProgress: "chat-native-progress",
|
|
chatInboundWakeNotices: "chat-inbound-wake-notices",
|
|
chatInteractionContinuations: "chat-interaction-continuations",
|
|
chatTerminalConfirmations: "chat-terminal-confirmations",
|
|
chatFailedRetryNotices: "chat-failed-retry-notices",
|
|
chatRunMilestones: "chat-run-milestones",
|
|
chatReceipts: "chat-slack-receipts",
|
|
browser: "browser-use",
|
|
email: "email-channels",
|
|
museMaintenance: "muse-maintenance",
|
|
mergedPullRequestConfirmations: "merged-pull-request-confirmations",
|
|
mcpEvents: "mcp-events",
|
|
dotEvents: "dot-events",
|
|
grokDispatch: "grok-dispatch",
|
|
skillSourceSync: "skill-source-sync",
|
|
skillPackages: "skill-packages",
|
|
runtimeAssetRetention: "runtime-asset-retention",
|
|
fastResponse: "fast-responses",
|
|
feedback: "feedback-exports",
|
|
chatCompletion: "chat-completions",
|
|
connection: "connection-continuations",
|
|
connectionEvents: "connection-events",
|
|
connectionContinuity: "connection-continuity",
|
|
gitRestorePoints: "git-restore-points",
|
|
remoteWorkspaceSnapshots: "remote-workspace-snapshots",
|
|
nativeWorkspaceSyncRetention: "native-workspace-sync-retention",
|
|
question: "question-responses",
|
|
pluginSchedules: "plugin-schedules",
|
|
pluginLifecycle: "plugin-resource-lifecycle",
|
|
toolAction: "tool-action-receipts",
|
|
terminalWorkspaces: "terminal-workspaces",
|
|
routineExecutionStatus: "routine-execution-status",
|
|
routineSchedules: "routine-schedules",
|
|
issueMonitors: "issue-monitors",
|
|
agentTimers: "agent-timers",
|
|
caseHandoffs: "case-handoffs",
|
|
nativeHarnessBackupExpiration: "native-harness-backup-expiration",
|
|
nativeCheckpointExpiration: "native-checkpoint-expiration",
|
|
} as const;
|
|
export type DeliveryQueue = typeof DELIVERY_QUEUES[keyof typeof DELIVERY_QUEUES];
|
|
|
|
export async function notifyDeliveryWork(transaction: object, queue: DeliveryQueue, hint?: string): Promise<void> {
|
|
await signalDatabaseWork(transaction, queue, hint);
|
|
}
|
|
// Public delivery notification observers only see settled work. The coordinator
|
|
// uses the lower-level lifecycle subscription to also fence in-flight writes.
|
|
export function subscribeDeliveryWork(owner: object, queue: DeliveryQueue, wake: () => void): () => void {
|
|
return subscribeDatabaseWork(owner, queue, event => {
|
|
if (event === "settled") wake();
|
|
});
|
|
}
|