Files
PaperClipAI/doc/idle-voice-inbound-cleanup.md
T
DottaandPaperclip 71b448b9cb fix(server): schedule inbound voice cleanup from durable obligations (#17580)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Incoming voice calls need approval, expiry, and safe provider
cleanup.
> - The remaining voice maintenance tick scans admissions every second,
including idle endpoints.
> - Admission status does not prove that the provider call has ended.
> - Existing admission rows can retain the cleanup owner, confirmation,
and retry deadline.
> - This pull request replaces the cleanup scan with committed changes
and actual deadlines.
> - Idle cleanup stops querying while unfinished calls keep a
recoverable path to termination.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Inbound voice admission cleanup. Its reply-delivery parent #17578 has
landed; this PR targets master. Live-session recovery is in #17576. It
preserves separate ownership for those services.

**Current behavior**

The voice maintenance tick scans expired admissions even when no cleanup
is due. A session can commit before the separate approved-admission
write. Expiry can then terminate that authorized call. A rejected guest
call can be labeled ended before provider termination succeeds.
Seven-day retention can delete that unfinished cleanup.

**Proposed behavior**

Wake cleanup from committed admission and session changes. Schedule the
original expiry or saved retry. Claim current state under the admission
lock before provider I/O. Commit approval with session reservation. Keep
unconfirmed termination durable and transfer bound calls to live-session
recovery.

**Reason and benefit**

Configured idle endpoints and confirmed history cause no recurring
admission cleanup SQL or provider requests. Approval, retries, crash
recovery, and cleanup remain supported. History retention and call
reports stay separate migration items.

## What Changed

- Recover terminal and rejected legacy admissions whose session binding
was lost. Copy known termination or bind a proven matching live session
for cleanup without granting approval.
- Register inbound cleanup with the shared coordinator and scoped
deadline worker. Yield after each 100-row scope page and reset cursors
on registration. Remove only cleanup from periodic voice maintenance.
Keep complete manual reconciliation.
- Add commit-aware notifications at admission, approval, rejection,
guest binding, pending-tool expiry, lifecycle confirmation, and session
termination writes.
- Commit authorized session binding and admission approval together.
Recheck cleanup ownership under the same admission lock. Recover older
interrupted bindings only from matching session evidence.
- Add nullable cleanup confirmation, next-attempt, and provider-account
fields to the existing admission table. No new queue table is added.
Generated migration 0396 backfills only provable historical termination
in committed 1,000-row keyset batches and adds a concurrent partial
index.
- Save the retry before provider contact. An unconfirmed end receipt
retains cleanup. Bound calls remain owned by live-session recovery.
- Verify the signed provider account before using a rotated key for
unbound cleanup. A different account cannot inspect or end the saved
call. Legacy admissions require their original credential fingerprint.
- Prevent history pruning and company deletion from erasing unfinished
cleanup. Company deletion locks endpoints before the voice owner's
check, matching ingress serialization.
- Document producer coverage, ownership, deadlines, verification, and
remaining sleep limits.

## Verification

Published head: `a4e7ccf976fc5793a8914e4e4861aa080d43a4da`. The reply
parent #17578 has landed. A normal merge integrates actual master
`e7ea2e2961863171956e70fb05d3d1f3790f7909`, preserves its unrelated
changes, and regenerates this slice’s migration after landed migration
0395. Fresh exact-head reviews and CI are required.

- The previous Ponytail 3/5 finding is repaired: legacy terminal
admissions no longer enter approval-only repair, and rejected admissions
with valid matching live sessions retain their rejection and persist the
binding needed for session-scoped cleanup confirmation. Five regressions
failed with ownership conflicts before the repair. Five
conflicting-session regressions prevent an unrelated leg from being
claimed. No cleanup operation grants approval or starts a provider call.
- **24 complete cleanup lifecycle cases and 122 complete voice-store
integration cases pass** on the repair. After actual-master integration
and migration regeneration, the complete 24-case cleanup suite passes
again on a new empty PostgreSQL 17 database, and the database migration
integration test passes separately. The cleanup owner, voice-store test
and admission schema are byte-identical to that tested repair; the
cleanup fixture only changes the migration filename. Migration
numbering/safety, database build, production server typecheck and build
pass.
- Migration `0396_tan_paladin.sql` was generated with `pnpm
db:generate`. Every landed SQL migration and the complete parent journal
prefix are unchanged. Generated snapshots retain exactly 0392–0396,
pruning 0391. The reviewed cleanup SQL, concurrent idempotent index and
committed 1,000-row historical backfill are retained byte for byte under
the new generated filename, registered for top-level execution. The real
application migrator boots a fresh database, repairs an interrupted
concurrent index, serializes replays and verifies eligible-history
backfill across actual transactions.
- The preceding published head
`48d8356e73308579470c182e759eead6e83173d2` received Greptile 5/5 but
failed CI server shard 9/12: one existing connected MCP rate-limit test
expected two real-clock calls to share a fixed 60-second bucket. A
controlled window rollover reproduced its exact failure. That existing
fixture now freezes only `Date` around the same-window 429 assertion,
verifies acceptance after advancing one window, and restores the real
clock in `finally`. All 92 gateway acceptance cases pass. Production
rate limiting, timeouts and assertions are unchanged. This adds one
existing CI fixture to the slice; the original failed run remains
recorded.
- The first regression invocation skipped the suite because it used the
wrong database variable; that is not a pass. The corrected invocation
reproduced five ownership conflicts. The first fixed cleanup run passed
22 and failed two new low-level deletion test calls on a
retained-session foreign key; the regressions now check the cleanup
deletion guard at its owning boundary, with production deletion ordering
unchanged. The final full suite passes all 24.
- The first complete voice-store invocation passed 121 and timed out on
one transcript case at the unchanged 15-second limit. That case passed
in isolation on a fresh database. One subsequent complete invocation,
with builds finished, passed all 122 without source, assertion or
timeout changes. The underlying timeout cause is unproven; the original
failure remains recorded. A new PostgreSQL startup also failed from
exhausted host semaphore capacity; fresh databases on the already-owned
server were used without removing shared IPC resources.
- Prior source `b8b12199b9` passed 266 distinct cases and carried three
guarded mutation proofs; those older results are not added to this
repair’s case count. Its CI run 40641168596 failed E2E shard 3 when
legacy cancellation hit a one-second PostgreSQL lock timeout. GitHub
initially rejected a retry while the run was active, then accepted one
targeted retry after completion; attempt 2 passed on that same old head.
The lock holder and underlying cause remain unproven. The unchanged Node
policy checker’s older failure on ignored generated Copilot manifests
and passing tracked-input mirror are also retained in the audit; no
workflow, checker, vendor manifest or ignore rule was modified.
- [Current repair and failure
record](https://github.com/paperclipai/paperclip/blob/a4e7ccf976fc5793a8914e4e4861aa080d43a4da/doc/plans/2026-10-25-voice-inbound-cleanup-sleep-audit.md#legacy-terminal-admissions-missing-their-session-binding--october-28).
Current-head CI and Greptile 5/5 gate merge. A completed failing
Ponytail verdict must be fixed; missing or interrupted reviews do not
block, and active pending review waits are capped at 30 minutes under
the maintainer’s explicit instruction. No new full local repository
suite, real provider, Cloud alarm, host/database suspension or savings
claim is made.

## Risks

- The migration distinguishes admission status from physical
termination. Legacy ended guest rejections without proof remain cleanup
obligations. Missing original credentials can keep those old obligations
unresolved.
- Future writers must use the existing notification boundary.
Independent SQL writers and multiple processes need their own wake
contract.
- Pending inbound cleanup now blocks company deletion instead of
allowing cascade deletion to lose the provider obligation. Existing
voice-history deletion restrictions remain. This deliberately includes
an active anonymous inbound ring and an old call whose termination
cannot be verified. An operator must stop ingress and settle the call
before deletion; this slice adds no abandonment override and never
treats missing credentials as proof of termination. This is the retained
cleanup ownership policy, not a claim that every deletion attempt can
complete immediately.
- History retention and report processing still run in the remaining
maintenance lane. This PR does not extend Cloud wake coverage or claim
measured host/database suspension.

## Model Used

OpenAI Codex (GPT-6), with reasoning and code execution. The exact
serving model ID and context window were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 08:30:15 -05:00

6.7 KiB

Inbound voice cleanup without idle scans

Inbound admission cleanup runs from committed changes and saved deadlines. It uses the shared delivery coordinator and scoped deadline worker. The recurring voice maintenance lane now owns only history retention and reports, which remain separate migrations. Manual reconciliation remains available for diagnostics.

Ownership and durable state

An unbound admission owns the existing provider call until approval or guest intake binds a voice session. The live-session recovery service then owns physical termination. Admission cleanup never also ends that bound call. It can request ending through the session row and copy the owner's confirmed terminal result.

Approval now commits the admission binding with the authorized session insert or guest promotion. Expiry cannot see an old approving admission after that session commits. Cleanup also locks and rechecks the admission before claiming it; a candidate from an earlier scan is not permission to terminate a call. Both paths lock admission before session. Provider requests run outside SQL transactions. Older interrupted handoffs are repaired only when the existing session matches the approval actor, authority, request key, task, token, generation and credential identity. Conflicting ownership retains an error/retry instead of terminating a call or inventing authority.

The migration adds three nullable fields to chat_voice_inbound_calls, not a new queue table:

  • cleanupConfirmedAt separates admission status from physical termination. An ended or denied admission can still have an unconfirmed provider leg.
  • cleanupNextAttemptAt survives crashes and restart. The claim saves the retry before contacting the provider.
  • providerAccountId records the account authenticated by the signed pre-call. Cleanup can use a rotated key after verifying its account. A key for another account cannot inspect or terminate this call. Older admissions lack that evidence and require their original credential fingerprint.

The migration is replayable. It backfills only historical ended rows whose old writer or signed lifecycle audit proves termination. It leaves old guest-denial rows uncertain when their ended label did not prove physical cleanup. Those rows receive a real check rather than disappearing as completed history. The partial index covers unconfirmed admission identities. Startup reads at most 100 rows per scope refresh and reconstructs deadlines across pages. The historical backfill commits keyset batches of at most 1,000 rows through the migration runner. Migration 0395 preserves the landed checkpoint lineage through 0394.

Deadlines and quiet state

Rejection returns after its decision transaction commits. Its notification wakes the registered cleanup worker; the HTTP request does not wait for provider account verification, inspection, or hangup. The saved denial immediately rejects further unauthorized call input while physical termination remains tracked separately.

Pending unbound admissions retain their original two-minute approval or ten-minute guest-intake expiry. Rejection is immediately due. A cleanup claim saves a 30-second retry before provider I/O. An unconfirmed end receipt keeps that retry; inspection, a confirmed end receipt, or a signed terminal callback clears it. Unexpected failures also receive the shared worker's bounded failure backoff. Neither a timeout nor an end request by itself proves termination.

Bound live calls leave no admission cleanup deadline. Their existing live-session worker owns expiry, authorization checks and provider cleanup. Its terminal write signals admission settlement. Healthy passive provider inspections do not wake this admission worker. Confirmed history and configured-but-idle endpoints leave no recurring admission SQL or provider requests.

Feature disable does not discard an outstanding call. Cleanup continues with its saved identity. Missing credentials or account mismatches retain debt; they do not permit use of a replacement account. No recovery path creates or redials a call.

Producer audit

Producer Commit boundary and wake
Signed pre-call admission Endpoint-scoped intent before insert in the quota transaction
Approval or rejection Admission-scoped intent before the locked state write
Authorized session insert or guest promotion Admission binding and intent inside the reservation transaction
Guest intake task/session binding Session-scoped voice change before insert; admission binding shares the transaction
Pending tool detects expiry Admission intent before the expired-state write
Signed terminal lifecycle/report callback Session and admission intents before confirmed termination writes
Live-session termination, inspection or no-resource expiry Existing voice-change boundary also signals inbound settlement
Reply provider confirms the call ended Inbound session intent before the terminal session write
Configuration, authority or connection changes Existing scoped voice-change composition
Cleanup claim, legacy handoff repair and confirmation Admission intent in the owning transaction

Notifications are process-local hints. Existing database rows remain truth. The shared transaction layer delays hints until outer commit and retains uncertainty. Other replicas and independent SQL writers require an explicit integration.

History retention cannot delete unconfirmed cleanup. Company deletion locks its endpoints, matching ingress, then asks the voice owner to prove inbound cleanup is settled. It rejects deletion while a provider obligation remains. This prevents cascade deletion from erasing the evidence needed to finish cleanup. Existing voice-history foreign-key restrictions otherwise remain unchanged.

Verification and limits

Real PostgreSQL tests cover confirmed-history quiet, exact expiry, restart, transaction rollback/outer commit, stale discovery, old interrupted handoffs, bound-session ownership, unconfirmed requests, failures, fair paged backlog, drain/resume, shutdown, retention protection, and deletion protection. Production service tests exercise signed ingress, rejection, account mismatch and rotation, feature disable, and atomic ordinary/guest approval. See the dated inbound cleanup audit for commands, counts and preserved failures. The current-parent acceptance records 266 distinct passing cases and the guarded batching, restart and approval regressions. Earlier counts in that audit belong to their stated revisions.

These changes do not migrate history retention or call reports, extend Cloud's external wake coverage, relax sleep eligibility for live calls, or prove actual host/database suspension. A registered in-process deadline alone cannot wake stopped compute.