mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-02 16:58:52 +01:00
DottaandPaperclip
e93732c3d4
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Phone calls receive approved answers from the existing chat publication queue. > - The chat maintenance timer checks for voice replies every second, including when no reply is ready. > - A reply can be saved before its publication commits. An early wake alone can miss that final transition. > - Existing session, reply, publication, and action rows already record delivery and recovery work. > - This pull request uses committed changes and saved deadlines to drive phone reply delivery. > - Empty reply queues become quiet while authorization, ordering, and uncertain-send protection remain in place. ## Linked Issues or Issue Description Stacked on #17576. Related delivery infrastructure: #15625. Related voice connection work: #11418. **What existing behavior does this improve?** Background phone reply delivery and recovery of abandoned dispatch intents. **Current behavior** Chat maintenance calls voice reply delivery every second. It also scans old dispatch intents. Publication transport starts a direct push before the final publication commit. The timer masks that timing gap. **Proposed behavior** Committed publication, reply, session, and configuration changes wake the shared coordinator. Ready replies drain in bounded passes. Saved rate-limit and dispatch-expiry deadlines retain recovery. An empty queue or a permanent delivery barrier has no recurring reply scan. **Reason and benefit** Remove recurring SQL from idle phone reply delivery without dropping replies, replaying uncertain sends, or adding a queue table. **Breaking changes** No API or schema change. Explicit manual recovery remains available. Inbound admission cleanup, history retention, and reports remain on their current paths for separate migrations. ## What Changed - Extract ordered reply state and multipart policy within the voice owner. Retain current authorization and atomic dispatch claims. - Register a scoped reply worker with startup reconstruction, fair work batches, failure backoff, drain/resume handling and shutdown ownership. - Extend the existing shared dispatcher with normalized partial-page results to preserve the bounded parent recovery contract. Existing complete-array consumers retain their calls. - Signal before relevant writes in their existing transactions. Both normal publication completion and operator resolution wake the actual conversation after commit. - Bound reply inventory to 100 total headers per refresh. Preserve partial scans, concurrent changes and newer denied-call restoration obligations across reconstruction. - Track abandoned dispatch actions independently of live calls. Quarantine expired intents without resending them. - Use existing active-call authorization checks to reactivate replies that were previously denied. Healthy call inspections do not rescan empty reply queues. - Remove the periodic reply scan and the early direct publication push. Keep provider requests outside database transactions. ## Verification Published head: `1435cab10be3f95c38ab6672d2a890ae49e9d300`. Voice recovery #17576 has landed at master `e1ba77f06771a27114b3999004b0ca5e3c172dd4`. A normal merge reconciles squash ancestry while preserving the complete reviewed 31-file reply patch byte for byte and all unrelated master changes. The PR now targets master; fresh exact-head CI and Greptile are required. The prior 449-case/typecheck/build results below cover the identical owned source, not a new full-suite run. - **449 distinct targeted cases pass**: 111 complete voice-store/reply; 19 complete parent recovery; and 319 complete tool-gateway, secret-proposal, shared dispatcher/coordinator/batching/startup/hostname cases. Production server typecheck and build pass. The policy checker and its Node tests pass. Later edits record evidence only. - Two real normal-publication/operator-resolution tests seed 105 unrelated calls and 105 future dispatches. Five publications selected **1,065/1,060 inventory rows before repair**; both paths now pass the same **125-row total budget** and deliver every answer once. This counts selected SQL results, not physical tuple visits or host savings. - Publication commits now signal their actual conversation. The voice reader returns at most 100 total session/action headers per refresh, preserves phase/cursor state and retries, and resets on worker registration. Current-state checks and dispatch claims still own provider effects. The 105-action case measures the page cap, restarts the same owner during reconstruction, waits at saved future deadlines and quarantines every abandoned action without resending. - A real 403 fixture proves that an older broad scan cannot erase a denied call first observed behind its cursor. Its owner callback still emits the committed restoration wake after scan completion. The existing actual-service authority-restoration and final-publication timing tests remain in the full suite. They are not replaced by direct helper calls. - Guarded removal of conversation scoping, cursor reset or the denial revision fence fails the intended budget, 100-versus-5 restart or missing committed-wake assertion; source is restored. An initial shadowed fixture name and an inadequate due-action reset probe are retained as failed attempts in the [current audit](https://github.com/paperclipai/paperclip/blob/d2c0a7de8c3e9cac8382b1268f3f5d883a2067b3/doc/plans/2026-10-25-voice-reply-delivery-sleep-audit.md#publication-inventory-and-paging-acceptance-october-28). - Parent recovery still passes paging, restart, behind-cursor insertion, feature changes, actual disablement, quiet health observations, current authority, exact-scope failure fences and wildcard-failure isolation through the shared worker. Four existing array-based worker consumers retain their API and pass their full suites. - The two voice fixtures self-migrate separate empty databases. Legacy tool-review/secrets suites use separate migrated clones. Earlier full-local attempts remain limited by embedded PostgreSQL setup and are not passing full-matrix claims. Older counts apply only to their stated revisions. Fresh exact-head Greptile, Ponytail if present, security and all CI remain merge gates. No live-provider, multi-writer, external-wake, deployed-sleep or savings claim is made. ## Risks - Missing a supported producer can strand work. Tests cover real enqueue, retrieval, normal publication, and operator publication paths. Startup reconstructs existing records. - Signals cover the process-local database owner. Independent SQL writers or other replicas need their own wake contract. - Authority restoration relies on the existing live-call recovery check. This dependency must be preserved if that owner changes later. - A provider acceptance receipt does not prove speech. Unknown outcomes remain blocked from automatic resend. - This change does not extend Cloud alarms, allow sleep during live calls, or prove deployed suspension or cost savings. ## Model Used OpenAI GPT-6 via Codex, with repository inspection, reasoning, code execution and test tools. The runtime does not expose a more specific deployment identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with the available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket identifier - [x] I have run the selected tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
157 lines
9.7 KiB
Markdown
157 lines
9.7 KiB
Markdown
# Voice reply delivery without idle polling
|
|
|
|
Phone replies wake from committed publication, reply, session, and configuration
|
|
changes. The application registers the worker with the shared delivery coordinator.
|
|
The one-second chat maintenance lane no longer calls `pushReplies`. Inbound voice
|
|
admission, report reconciliation, and other chat duties remain separate migrations.
|
|
|
|
## Durable truth and delivery boundaries
|
|
|
|
The existing voice sessions, voice replies, chat publications, and chat actions
|
|
remain authoritative. No table or migration is added. The worker reconstructs
|
|
active phone sessions and outstanding dispatch intents at startup. Notifications
|
|
are scoped hints, never permission to replay a provider action.
|
|
|
|
A reply inserted while its publication is streaming is not yet deliverable.
|
|
Both normal publication completion and operator resolution signal the endpoint
|
|
before committing its published state. Reply insertion and authorized question
|
|
retrieval signal their session before writing. The existing transaction boundary
|
|
holds notifications until outer commit and reconciles uncertain outcomes.
|
|
|
|
The dispatcher retains current authorization, session locking, ordered reply and
|
|
multipart barriers, and its atomic action claim. Provider calls remain outside
|
|
SQL transactions. An accepted receipt is distinct from spoken playback. Unknown
|
|
outcomes never authorize resending the answer or appending its remaining parts.
|
|
|
|
## Deadlines and disarming
|
|
|
|
- A ready ordered reply is immediate work; a bounded pass drains up to eight parts.
|
|
- A definitive rate limit retains its saved retry deadline.
|
|
- A dispatch without a durable receipt retains its existing strict 30-second
|
|
uncertainty deadline. These actions are tracked independently of live sessions,
|
|
so ending a call cannot strand its abandoned dispatch intent.
|
|
- Database or provider-path failures retain per-entry backoff. Scoped inventory
|
|
failures retry their scope; healthy companies retain fair processing access.
|
|
- No ready reply, a presented question awaiting retrieval, accepted history, or an
|
|
unknown/failed outcome leaves no recurring reply scan.
|
|
|
|
Live-call recovery already checks current authority before provider inspection.
|
|
Reply delivery remembers sessions denied by its authorization check. A successful
|
|
subsequent live-call authorization signals only those blocked sessions. This
|
|
preserves restoration after membership/task authority changes without adding
|
|
notifications to every authorization table or scanning healthy empty reply queues
|
|
on every live-call inspection. Startup reconstructs this in-memory state from
|
|
existing sessions. This dependency must remain explicit if active-call recovery
|
|
is changed later; a notification alone never bypasses reauthorization.
|
|
|
|
## Lifecycle and limits
|
|
|
|
The coordinator coalesces changes, serializes work, preserves dirty scopes across
|
|
failure and drain, and stops work on shutdown. Startup loads identities in pages;
|
|
processing uses fair bounded batches. Manual `pushReplies` and one-shot `reconcile`
|
|
remain available for explicit diagnostics/tests and must not become periodic lanes.
|
|
|
|
Signals cover the supported process-local database owner. Independent SQL writers
|
|
or other processes require their own notification/reconciliation contract. These
|
|
changes do not extend Cloud wake coverage, permit sleep during a live call, or
|
|
prove deployed instance/database suspension.
|
|
|
|
|
|
Both live-session recovery and reply delivery use `scoped-deadline-work.ts` for
|
|
coalescing, fair batching, scope reconstruction, retries and drain retention.
|
|
Their adapters keep domain state and provider operations in the voice owner.
|
|
The production-wiring regression covers authority returning before live recovery
|
|
observes the denial; recovery still ends calls when their authority is absent.
|
|
|
|
|
|
## Current-parent paging contract acceptance, October 27
|
|
|
|
This slice normally integrates bounded voice recovery #17576 at `712f3862b5`,
|
|
including receiver `6e88ca0201` and the landed foundations. The original 30-file
|
|
reply scope adds one shared contract path, `scoped-deadline-work.ts`. No database
|
|
file changes. The existing helper could represent only a complete inventory;
|
|
using that contract for the repaired parent would erase its bounded-pass guarantee.
|
|
|
|
The real 205-call parent regression fails before the integration repair: startup
|
|
executes three inventory queries (100, 100, 5 rows) instead of one 100-row query.
|
|
The shared helper now accepts either its existing complete array or a normalized
|
|
page result. It retains page generations and seen identities, keeps existing
|
|
entries until reconstruction completes, limits dirty scopes per pass, and replays
|
|
writes received during reconstruction. It owns no SQL or voice/provider policy.
|
|
Recovery delegates to its owner refresh and resets partial owner state on worker
|
|
registration. Existing complete-array consumers keep their public calls.
|
|
Connection observations retain the parent's separate receiver/command notification
|
|
path; actual configuration wakes both voice owners.
|
|
|
|
All 445 distinct targeted cases pass: 18 complete parent recovery lifecycle
|
|
cases, 108 complete voice-store/reply integrations, 224 complete tool-gateway and
|
|
secret-proposal cases, and 95 shared dispatcher/coordinator/batching/startup/
|
|
hostname cases. The helper's single existing test also ran in the initial
|
|
19-case green group; it is counted only in the 95-case group. The 224 cases
|
|
exercise the four other production consumers of the shared helper, including
|
|
complete-array empty/expiry/recovery/projection, transaction and drain behavior.
|
|
Server production typecheck and server build pass. The voice fixtures self-migrate
|
|
separate initially empty databases. The two other database suites use separate
|
|
migrated clones. Expected injected provider/wakeup failures and listener warnings
|
|
remain in logs. Only documentation edits follow these passing production checks.
|
|
|
|
This acceptance preserves the parent's startup paging, restart, behind-cursor
|
|
insert, concurrent feature change, health-only quiet, true disablement, current
|
|
authority, failure, drain and shutdown contracts. Existing real publication,
|
|
operator resolution, uncertain send, multipart/question ordering, authority
|
|
restoration and 105-abandoned-dispatch tests remain in the 108-case suite.
|
|
Fresh exact-head reviews and all CI remain required. No full local matrix,
|
|
live provider acceptance, Cloud wake, deployed suspension or savings is claimed.
|
|
|
|
|
|
## Publication inventory and paging acceptance, October 28
|
|
|
|
The security review correctly found an endpoint-wide read per publication.
|
|
Two real publication-path regressions seed 105 unrelated active phone calls and
|
|
105 future dispatch intents on the same endpoint. Five normal publications select
|
|
1,065 inventory rows; five operator resolutions select 1,060. Both exceed the
|
|
125-row total budget. The corrected paths pass that same budget and deliver each
|
|
answer once. These measurements count selected row results from executed queries,
|
|
not physical PostgreSQL tuple visits or a deployment cost benchmark.
|
|
|
|
Normal publication completion and operator resolution now notify their actual
|
|
conversation inside the existing transaction. The voice owner handles that
|
|
normalized conversation scope. Its reader returns at most 100 total session and
|
|
action headers per refresh, with a saved phase/cursor for the next pass. Startup,
|
|
configuration changes and abandoned actions retain finite reconstruction. The
|
|
shared dispatcher retains generations and observed IDs across pages; it also
|
|
preserves the parent wildcard-failure isolation. SQL and provider decisions stay
|
|
inside the voice owner. No schema, table, provider-specific shared branch or
|
|
per-route recovery wrapper is introduced.
|
|
|
|
The abandoned-dispatch case now proves a 100-row startup cap, restart of the same
|
|
owner before its scan completes, restoration of all 105 saved deadlines, no SQL
|
|
before the deadline, and quarantine without resending. A separate owner-level
|
|
real-database case reproduces a new denied call behind an older scan cursor:
|
|
completion of the old scan must not erase the later restoration obligation.
|
|
Denial revisions fence that cleanup. The case confirms a real 403, then observes
|
|
the committed wake. The existing actual-service authority-restoration case still
|
|
verifies the complete production callback path; the new cache test is not a
|
|
replacement for that integration coverage.
|
|
|
|
Guarded mutations broaden publication hints back to endpoints, omit owner reset,
|
|
and omit the denial revision fence. All fail the corresponding row-budget,
|
|
100-versus-5 restart, or missing committed-wake assertion, with source restored.
|
|
An earlier reset probe using already-due actions did not detect the missing reset
|
|
because its own action writes rediscovered the backlog. That incomplete probe is
|
|
retained as such; future deadlines make the strengthened regression deterministic.
|
|
The first amplification fixture also used a shadowed publication function name;
|
|
that fixture failure preceded the actual 1,065/1,060-row baseline.
|
|
|
|
The final branch normally integrates voice parent `ad79c7e805`. Its original
|
|
30-file scope plus the existing shared dispatcher remains 31 files, with no DB
|
|
changes. All 449 distinct targeted cases pass: 111 complete voice-store/reply,
|
|
19 complete parent recovery, and 319 complete shared/legacy-consumer/startup cases.
|
|
Both voice fixtures self-migrate separate initially empty databases; the two
|
|
legacy DB suites use separate migrated clones. Production server typecheck and
|
|
build pass. The current policy checker and its Node tests pass. Only evidence
|
|
edits follow these checks. Older 445/786-case totals belong to their stated source
|
|
revisions and are not added here. Current exact-head reviews and all CI remain
|
|
required. No full local matrix, live provider, external wake, deployed suspension,
|
|
physical query-cost or savings result is claimed.
|