mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 23:43:16 +01:00
DottaandPaperclip
b889ac39b7
## Thinking Path > - Paperclip manages agents and their work. > - A delegated task reports its result to the source conversation through a durable completion outbox. > - Native execution can succeed before the presentation worker saves the accepted final reply. > - The outbox treated that short interval as a missing answer and cleared the target run. > - The presentation worker then rejected the valid answer because its delivery no longer belonged to that run. > - This change retains the existing target for one fixed presentation grace while accepted presentation is pending. > - The existing presentation owner can save the reply without another model run or more budget. ## Linked Issues or Issue Description **What happened?** A completion-delivery retry can run immediately after a native callback succeeds. It clears the callback's delivery target before the presentation worker saves the accepted answer. The reply then fails with `chat_completion_superseded`. A daily run limit correctly blocks the unnecessary replacement run, leaving the conversation active without its saved answer. **Expected behavior** The outbox must retain its current target and attempt while that exact native run has accepted presentation work. The existing presentation worker must save the answer once, acknowledge the delivery, and settle the conversation. Existing actor, session, result, digest, budget and retry checks must remain enforced. **Steps to reproduce** 1. Accept a native DONE result and current-turn final message for a completion callback. 2. Run its due completion-outbox handler after terminal projection but before reply presentation. 3. Run the existing presentation worker. Before this fix it rejects the reply because the outbox has cleared the target. **Paperclip version or commit** Reproduced on `b7f1c3769c00a107ffe4128b48b0ef8cc452cf3f` through the real PostgreSQL outbox and presentation services. The reported failure remains preserved. A new QA run is required after deployment; local tests do not qualify the deployed fix. Related: #17696 schedules accepted-response recovery. #18118 preserves canonical completion receipts. #18364 handles provider output and response-stage contracts. Those changes do not protect the completion-outbox target during this interval. ## What Changed - Defer a new execution attempt when the existing selector finds pending presentation for the same company, conversation, run and agent. - Retain the queued delivery target and attempt with an atomic compare-and-set. Await the existing scoped worker notification before that write in the same transaction. - Serialize retry with reply insertion under the existing child-task-first lock. Recheck locked task Done state and the delivery's stable completion-event version, status, target, attempt and exact lease. A delivered, retargeted or newer-claim winner prevents another wake. Admit the next attempt only after the winning transaction commits and releases its locks. - Limit presentation grace to five existing one-minute delivery leases, measured from the original run `finishedAt`. Clamp both the delivery lease and the native retry deadline to that immutable limit. Repeated sweeps cannot extend it. After expiry, resume the existing retry, exhaustion and daily-limit checks. - Keep publication authority in the existing materializer. The candidate query only defers a retry. Read the freshly reloaded delivery status, target and saved receipt after acquiring its child-task lock. - Fence both hold and retry writes with the exact claimed event version, attempt and lease. A stalled old hold cannot overwrite a newer lease. - Keep completion-event identity separate from mutable task ownership versions. Ordinary Done reassignment preserves its existing completion event; a real reopen retains normal supersession. - Add real PostgreSQL coverage for the observed ordering, outer commit and rollback, one saved reply, conversation settlement, idle SQL, retry boundaries and superseded deliveries. - Document the completion-outbox and presentation contract. ## Verification - Before the fix, the real PostgreSQL regression fails with `chat_completion_superseded` in `existingChatCompletionReply` when the outbox handles the terminal run first. After the fix, the same test passes without a new model wake. - Focused command: `vitest run --project @paperclipai/server server/src/services/native-runtime/agent-chat-response-recovery.test.ts server/src/__tests__/chat-completion-delivery.test.ts` — **95 pass** on `45f53a8c32fcca0bbdacf717286d37e30c1ba308`, using a disposable database on an owned local PostgreSQL 17 container. - Controls preserve retries for failed, cancelled, missing-final, missing-accepted-result, old-generation, foreign-company and foreign-agent targets. Reset or superseded task versions retain supersession. Digest and external-context declines never authorize a reply, never extend the fixed window, and leave it at exact expiry. An external native deadline cannot extend the limit. A valid reply at the last millisecond saves once without another model run. - The first reviewed head `b6f9c484b9567bc60af6d684c04e00058c1048ed` received Greptile 5/5 but an independent 3/5 review correctly found an unbounded hold. Three new real PostgreSQL controls failed on that source. This head adds the fixed bound and makes them pass. The earlier review, controls and CI remain retained; its review score does not qualify the new head. - The bounded predecessor `7300d2a978a5c4b3c13d85899bed3e98ae3c49fa` received an independent 4/5 review that correctly found an expiry-versus-in-flight-reply race. A real PostgreSQL advisory gate reproduced it after reply authority checked the delivery but before insertion. The retry lock and exact claim CAS now make that control pass. Both orderings, outer rollback, delivered/retargeted/superseded winners and competing lease claims pass. The next-wake test can acquire both parent and child locks, proving dispatch occurs after release. The valid 4/5 review and failing control remain retained. - At `39dd86e8f51d937cf382b2757736e0477201d21d`, both reviews found valid remaining defects: the reply helper still used its pre-lock delivery snapshot, and ordinary Done reassignment could stall a delivery by changing the task version. Four real PostgreSQL controls fail on that source and pass here: concurrent retry-first, a newly saved receipt after waiting, and reassignment within grace or at expiry. A real reopen through the issue service still supersedes the old event. - A later coordinator review found the hold CAS still lacked the attempt and lease fences already present on retry. A real PostgreSQL producer barrier reproduced an old hold shortening a newer claim from 120 seconds to 60 seconds. The same exact claim fences now prevent that overwrite. This failed control is retained; no old run was blindly retried. - The real worker saves one exact run-owned reply and acknowledges the delivery only after outer commit. An outer rollback publishes no reply. After settlement, advancing the scheduler clock by 60 seconds causes no further SQL. - Module boundaries, map structure, `git diff --check` and the canonical server build (including TypeScript and Runner vendor preparation) pass. The first local build stopped in the host package-manager setup; the installed pinned pnpm completed the build. Full repository typecheck, test and build gates pass in current-head CI; they have not all been run locally. - Final local checks also caught a test cleanup type error (TS2677) and one 94/95 readiness failure: the worker saves its comment before post-commit conversation settlement finishes. The test-only follow-up removes an invalid cleanup type predicate and waits on the existing bounded worker-quiet check before reading final state. All assertions and the same five-second bound remain. Source `45f53a8c32fcca0bbdacf717286d37e30c1ba308` passes all 95 cases and the canonical server build. The earlier failures are retained. - The final policy cleanup removes only 15 newly added narrative comment lines (seven service, eight test), as required by CONTRIBUTING.md. Both TypeScript AST signatures and `removeComments=true` emitted JavaScript are identical to tested source `45f53a8c32`. This was checked with the TypeScript 6.0.3 API and the installed native TypeScript 7.0.2 compiler. Assertions, timeouts and behavior are unchanged; the 95 tests were not repeated for identical compiled code. - The superseded `45f53a8c32` CI run recorded two 15-second timeouts in attention-retention cases. That failure remains retained. The test and its ten inspected owner or support files are identical on the tested master revisions. The canonical base run passes all 53 attention cases, and the comparison PR passes the same server lane, including those two cases in 9.3 and 9.8 seconds. This does not establish the timeout cause. Current-head CI is a new run after the required comment cleanup, not a retry of the failed source. - The optional Paperclip review bot ended neutral without a completed assessment because its execution paused. That result is not an approval. A separate [independent read-only audit](https://github.com/paperclipai/paperclip/pull/18502#issuecomment-6550216227) reports 5/5 confidence on exact head `9cfd3f74c69cc7a2da7b4ee5d1fca80f15d4af0c`, with no findings. Its manual assessment does not replace or relabel the incomplete bot result. - Current head `9cfd3f74c69cc7a2da7b4ee5d1fca80f15d4af0c` has a fresh Greptile 5/5 review with no open threads. [CI run 41249007430](https://github.com/paperclipai/paperclip/actions/runs/41249007430) passes all primary jobs and both required gates (`ci / verify` and `ci / e2e`). All 59 PR contexts are terminal, with no failed contexts. The actual CI checkout is `18db9e2750d6a799f26b5b8ba31bcccbea254227`, which combines master `681f7b77262a9df6c1b4aad55291bf960061e491` with this exact head. Fresh QA verification on the deployed application remains required before claiming the live failure is fixed. ## Risks A candidate can retain a delivery whose stricter presentation checks still prevent publication, but only within the fixed five-minute window. The existing worker keeps publication authority. If a valid reply remains unavailable at expiry, the normal callback retry may require another admitted model run; the original result and completed child work remain preserved. Existing retry, exhaustion, model-run and spending limits still apply. Failed or cancelled execution and genuinely missing accepted output retain the existing retry path. ## Model Used OpenAI GPT-6 through Codex, with repository inspection, tool use and code execution. The runtime does not expose the exact hosted revision or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>