mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A native (Paperclip Runner) run claims its `native_run_finalizations` coordinator before it starts any provider work. The claim is one transaction. It locks the coordinator row, then the run row. > - A writer that holds the run row and then waits for the coordinator row closes a lock cycle. PostgreSQL then aborts the claim with a deadlock (`40P01`), because the claim waited first. > - The aborted claim rolls back whole and starts no provider work. But the raw database error failed the run before its first turn. > - The failure path then preserved the prepared workspace. That stamped the unclaimed coordinator `native_continuation_requires_reconciliation`, and `failRun` used it as the run's error code. The task waited for reconciliation. > - A transient database fault must never end a run like this. The target rate of runs that need Retry is 0%. > - This pull request retries the claim transaction on transient lock contention with the shared bounded helper. > - The benefit is that a fresh run that loses a lock race at its claim continues to its provider turn by itself. ## Linked Issues or Issue Description **What happened?** On a hosted QA stack, a fresh native assignment run (not a retry) failed after about 50 s. The run status was `failed`. The error code was `native_continuation_requires_reconciliation`. The error text was `Database query failed (40P01)`. The stderr was empty. The task needed reconciliation. **Steps to reproduce** 1. Prepare a native run with an unclaimed coordinator (`observed`, attempt 0), as runtime selection leaves it. 2. In another transaction, lock the run row. 3. Start the native session. Its claim locks the coordinator and waits for the run row. 4. In the other transaction, lock the coordinator row. 5. PostgreSQL aborts the claim with `40P01`, and `executePaperclipNativeSession` rejects with that error. **Expected behavior** The claim retries after the competing transaction commits. The run reaches its provider turn. **Paperclip version or commit** `master` at958eda6f16. ## What Changed - `native-session-executor.ts`: the coordinator claim transaction runs inside `retryTransientLockContention` (`40P01`, `40001`, `55P03`; 5 attempts, 100 ms × attempt backoff). Each attempt re-reads and re-proves the coordinator, the run, cancellation and lineage, so a retry admits nothing that the first attempt would refuse. - A startup context that a rolled-back attempt opened is finished, and the startup owner's previous context is restored, before the next attempt. - New PostgreSQL test `native-coordinator-claim-deadlock.integration.test.ts`: `retries a claim that lost a deadlock and reaches the provider`. ## Verification - RED on `master` (958eda6f16): the new test fails. `executePaperclipNativeSession` rejects with a `40P01` cause, and the provider is not called. - GREEN with this change: `pnpm --filter @paperclipai/server exec vitest run src/services/native-runtime/native-coordinator-claim-deadlock.integration.test.ts` passes. The provider is called once and the coordinator is at attempt 1. - `tsc --noEmit -p server/tsconfig.json` passes. - `pnpm verify` for the affected files: see the PR comments. ## Risks - Low risk. The normal path has no contention and does not retry. - Under contention, the claim waits at most about 1 s more (backoff 100–400 ms) before provider work. This is before `turn.start`, so time to first token grows only in the contended case. - The lock order (coordinator, then run) is unchanged. It matches admission and the runner checkpoint owner lock. The writer that took the run row first on the QA stack is not identified, because no server logs or SQL were available. This change makes the claim survive any such writer. - Out of scope: a transient fault in other pre-dispatch statements of a run with a prepared remote workspace. There, workspace preservation still stamps the pre-claim code before `failRun`. The pre-claim fresh-continuation fallback then continues the task. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, embedded PostgreSQL tests). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Refs #20561 (pre-dispatch transient retry), #20682 (provider-pack admission lock wait). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>