## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Incoming voice calls need approval, expiry, and safe provider cleanup. > - The remaining voice maintenance tick scans admissions every second, including idle endpoints. > - Admission status does not prove that the provider call has ended. > - Existing admission rows can retain the cleanup owner, confirmation, and retry deadline. > - This pull request replaces the cleanup scan with committed changes and actual deadlines. > - Idle cleanup stops querying while unfinished calls keep a recoverable path to termination. ## Linked Issues or Issue Description **What existing behavior does this improve?** Inbound voice admission cleanup. Its reply-delivery parent #17578 has landed; this PR targets master. Live-session recovery is in #17576. It preserves separate ownership for those services. **Current behavior** The voice maintenance tick scans expired admissions even when no cleanup is due. A session can commit before the separate approved-admission write. Expiry can then terminate that authorized call. A rejected guest call can be labeled ended before provider termination succeeds. Seven-day retention can delete that unfinished cleanup. **Proposed behavior** Wake cleanup from committed admission and session changes. Schedule the original expiry or saved retry. Claim current state under the admission lock before provider I/O. Commit approval with session reservation. Keep unconfirmed termination durable and transfer bound calls to live-session recovery. **Reason and benefit** Configured idle endpoints and confirmed history cause no recurring admission cleanup SQL or provider requests. Approval, retries, crash recovery, and cleanup remain supported. History retention and call reports stay separate migration items. ## What Changed - Recover terminal and rejected legacy admissions whose session binding was lost. Copy known termination or bind a proven matching live session for cleanup without granting approval. - Register inbound cleanup with the shared coordinator and scoped deadline worker. Yield after each 100-row scope page and reset cursors on registration. Remove only cleanup from periodic voice maintenance. Keep complete manual reconciliation. - Add commit-aware notifications at admission, approval, rejection, guest binding, pending-tool expiry, lifecycle confirmation, and session termination writes. - Commit authorized session binding and admission approval together. Recheck cleanup ownership under the same admission lock. Recover older interrupted bindings only from matching session evidence. - Add nullable cleanup confirmation, next-attempt, and provider-account fields to the existing admission table. No new queue table is added. Generated migration 0396 backfills only provable historical termination in committed 1,000-row keyset batches and adds a concurrent partial index. - Save the retry before provider contact. An unconfirmed end receipt retains cleanup. Bound calls remain owned by live-session recovery. - Verify the signed provider account before using a rotated key for unbound cleanup. A different account cannot inspect or end the saved call. Legacy admissions require their original credential fingerprint. - Prevent history pruning and company deletion from erasing unfinished cleanup. Company deletion locks endpoints before the voice owner's check, matching ingress serialization. - Document producer coverage, ownership, deadlines, verification, and remaining sleep limits. ## Verification Published head: `a4e7ccf976fc5793a8914e4e4861aa080d43a4da`. The reply parent #17578 has landed. A normal merge integrates actual master `e7ea2e2961863171956e70fb05d3d1f3790f7909`, preserves its unrelated changes, and regenerates this slice’s migration after landed migration 0395. Fresh exact-head reviews and CI are required. - The previous Ponytail 3/5 finding is repaired: legacy terminal admissions no longer enter approval-only repair, and rejected admissions with valid matching live sessions retain their rejection and persist the binding needed for session-scoped cleanup confirmation. Five regressions failed with ownership conflicts before the repair. Five conflicting-session regressions prevent an unrelated leg from being claimed. No cleanup operation grants approval or starts a provider call. - **24 complete cleanup lifecycle cases and 122 complete voice-store integration cases pass** on the repair. After actual-master integration and migration regeneration, the complete 24-case cleanup suite passes again on a new empty PostgreSQL 17 database, and the database migration integration test passes separately. The cleanup owner, voice-store test and admission schema are byte-identical to that tested repair; the cleanup fixture only changes the migration filename. Migration numbering/safety, database build, production server typecheck and build pass. - Migration `0396_tan_paladin.sql` was generated with `pnpm db:generate`. Every landed SQL migration and the complete parent journal prefix are unchanged. Generated snapshots retain exactly 0392–0396, pruning 0391. The reviewed cleanup SQL, concurrent idempotent index and committed 1,000-row historical backfill are retained byte for byte under the new generated filename, registered for top-level execution. The real application migrator boots a fresh database, repairs an interrupted concurrent index, serializes replays and verifies eligible-history backfill across actual transactions. - The preceding published head `48d8356e73308579470c182e759eead6e83173d2` received Greptile 5/5 but failed CI server shard 9/12: one existing connected MCP rate-limit test expected two real-clock calls to share a fixed 60-second bucket. A controlled window rollover reproduced its exact failure. That existing fixture now freezes only `Date` around the same-window 429 assertion, verifies acceptance after advancing one window, and restores the real clock in `finally`. All 92 gateway acceptance cases pass. Production rate limiting, timeouts and assertions are unchanged. This adds one existing CI fixture to the slice; the original failed run remains recorded. - The first regression invocation skipped the suite because it used the wrong database variable; that is not a pass. The corrected invocation reproduced five ownership conflicts. The first fixed cleanup run passed 22 and failed two new low-level deletion test calls on a retained-session foreign key; the regressions now check the cleanup deletion guard at its owning boundary, with production deletion ordering unchanged. The final full suite passes all 24. - The first complete voice-store invocation passed 121 and timed out on one transcript case at the unchanged 15-second limit. That case passed in isolation on a fresh database. One subsequent complete invocation, with builds finished, passed all 122 without source, assertion or timeout changes. The underlying timeout cause is unproven; the original failure remains recorded. A new PostgreSQL startup also failed from exhausted host semaphore capacity; fresh databases on the already-owned server were used without removing shared IPC resources. - Prior source `b8b12199b9` passed 266 distinct cases and carried three guarded mutation proofs; those older results are not added to this repair’s case count. Its CI run 40641168596 failed E2E shard 3 when legacy cancellation hit a one-second PostgreSQL lock timeout. GitHub initially rejected a retry while the run was active, then accepted one targeted retry after completion; attempt 2 passed on that same old head. The lock holder and underlying cause remain unproven. The unchanged Node policy checker’s older failure on ignored generated Copilot manifests and passing tracked-input mirror are also retained in the audit; no workflow, checker, vendor manifest or ignore rule was modified. - [Current repair and failure record](https://github.com/paperclipai/paperclip/blob/a4e7ccf976fc5793a8914e4e4861aa080d43a4da/doc/plans/2026-10-25-voice-inbound-cleanup-sleep-audit.md#legacy-terminal-admissions-missing-their-session-binding--october-28). Current-head CI and Greptile 5/5 gate merge. A completed failing Ponytail verdict must be fixed; missing or interrupted reviews do not block, and active pending review waits are capped at 30 minutes under the maintainer’s explicit instruction. No new full local repository suite, real provider, Cloud alarm, host/database suspension or savings claim is made. ## Risks - The migration distinguishes admission status from physical termination. Legacy ended guest rejections without proof remain cleanup obligations. Missing original credentials can keep those old obligations unresolved. - Future writers must use the existing notification boundary. Independent SQL writers and multiple processes need their own wake contract. - Pending inbound cleanup now blocks company deletion instead of allowing cascade deletion to lose the provider obligation. Existing voice-history deletion restrictions remain. This deliberately includes an active anonymous inbound ring and an old call whose termination cannot be verified. An operator must stop ingress and settle the call before deletion; this slice adds no abandonment override and never treats missing credentials as proof of termination. This is the retained cleanup ownership policy, not a claim that every deletion attempt can complete immediately. - History retention and report processing still run in the remaining maintenance lane. This PR does not extend Cloud wake coverage or claim measured host/database suspension. ## Model Used OpenAI Codex (GPT-6), with reasoning and code execution. The exact serving model ID and context window were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
6.7 KiB
Inbound voice cleanup without idle scans
Inbound admission cleanup runs from committed changes and saved deadlines. It uses the shared delivery coordinator and scoped deadline worker. The recurring voice maintenance lane now owns only history retention and reports, which remain separate migrations. Manual reconciliation remains available for diagnostics.
Ownership and durable state
An unbound admission owns the existing provider call until approval or guest intake binds a voice session. The live-session recovery service then owns physical termination. Admission cleanup never also ends that bound call. It can request ending through the session row and copy the owner's confirmed terminal result.
Approval now commits the admission binding with the authorized session insert or guest promotion. Expiry cannot see an old approving admission after that session commits. Cleanup also locks and rechecks the admission before claiming it; a candidate from an earlier scan is not permission to terminate a call. Both paths lock admission before session. Provider requests run outside SQL transactions. Older interrupted handoffs are repaired only when the existing session matches the approval actor, authority, request key, task, token, generation and credential identity. Conflicting ownership retains an error/retry instead of terminating a call or inventing authority.
The migration adds three nullable fields to chat_voice_inbound_calls, not a new
queue table:
cleanupConfirmedAtseparates admission status from physical termination. An ended or denied admission can still have an unconfirmed provider leg.cleanupNextAttemptAtsurvives crashes and restart. The claim saves the retry before contacting the provider.providerAccountIdrecords the account authenticated by the signed pre-call. Cleanup can use a rotated key after verifying its account. A key for another account cannot inspect or terminate this call. Older admissions lack that evidence and require their original credential fingerprint.
The migration is replayable. It backfills only historical ended rows whose old writer or signed lifecycle audit proves termination. It leaves old guest-denial rows uncertain when their ended label did not prove physical cleanup. Those rows receive a real check rather than disappearing as completed history. The partial index covers unconfirmed admission identities. Startup reads at most 100 rows per scope refresh and reconstructs deadlines across pages. The historical backfill commits keyset batches of at most 1,000 rows through the migration runner. Migration 0395 preserves the landed checkpoint lineage through 0394.
Deadlines and quiet state
Rejection returns after its decision transaction commits. Its notification wakes the registered cleanup worker; the HTTP request does not wait for provider account verification, inspection, or hangup. The saved denial immediately rejects further unauthorized call input while physical termination remains tracked separately.
Pending unbound admissions retain their original two-minute approval or ten-minute guest-intake expiry. Rejection is immediately due. A cleanup claim saves a 30-second retry before provider I/O. An unconfirmed end receipt keeps that retry; inspection, a confirmed end receipt, or a signed terminal callback clears it. Unexpected failures also receive the shared worker's bounded failure backoff. Neither a timeout nor an end request by itself proves termination.
Bound live calls leave no admission cleanup deadline. Their existing live-session worker owns expiry, authorization checks and provider cleanup. Its terminal write signals admission settlement. Healthy passive provider inspections do not wake this admission worker. Confirmed history and configured-but-idle endpoints leave no recurring admission SQL or provider requests.
Feature disable does not discard an outstanding call. Cleanup continues with its saved identity. Missing credentials or account mismatches retain debt; they do not permit use of a replacement account. No recovery path creates or redials a call.
Producer audit
| Producer | Commit boundary and wake |
|---|---|
| Signed pre-call admission | Endpoint-scoped intent before insert in the quota transaction |
| Approval or rejection | Admission-scoped intent before the locked state write |
| Authorized session insert or guest promotion | Admission binding and intent inside the reservation transaction |
| Guest intake task/session binding | Session-scoped voice change before insert; admission binding shares the transaction |
| Pending tool detects expiry | Admission intent before the expired-state write |
| Signed terminal lifecycle/report callback | Session and admission intents before confirmed termination writes |
| Live-session termination, inspection or no-resource expiry | Existing voice-change boundary also signals inbound settlement |
| Reply provider confirms the call ended | Inbound session intent before the terminal session write |
| Configuration, authority or connection changes | Existing scoped voice-change composition |
| Cleanup claim, legacy handoff repair and confirmation | Admission intent in the owning transaction |
Notifications are process-local hints. Existing database rows remain truth. The shared transaction layer delays hints until outer commit and retains uncertainty. Other replicas and independent SQL writers require an explicit integration.
History retention cannot delete unconfirmed cleanup. Company deletion locks its endpoints, matching ingress, then asks the voice owner to prove inbound cleanup is settled. It rejects deletion while a provider obligation remains. This prevents cascade deletion from erasing the evidence needed to finish cleanup. Existing voice-history foreign-key restrictions otherwise remain unchanged.
Verification and limits
Real PostgreSQL tests cover confirmed-history quiet, exact expiry, restart, transaction rollback/outer commit, stale discovery, old interrupted handoffs, bound-session ownership, unconfirmed requests, failures, fair paged backlog, drain/resume, shutdown, retention protection, and deletion protection. Production service tests exercise signed ingress, rejection, account mismatch and rotation, feature disable, and atomic ordinary/guest approval. See the dated inbound cleanup audit for commands, counts and preserved failures. The current-parent acceptance records 266 distinct passing cases and the guarded batching, restart and approval regressions. Earlier counts in that audit belong to their stated revisions.
These changes do not migrate history retention or call reports, extend Cloud's external wake coverage, relax sleep eligibility for live calls, or prove actual host/database suspension. A registered in-process deadline alone cannot wake stopped compute.