mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-04 19:28:46 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - A user can click Retry on a task that is connected to an external
chat conversation.
> - That Retry prepares the chat retry inside one admission transaction.
The chat step takes a per-principal authorization lock with
`pg_try_advisory_xact_lock`.
> - When another chat transaction holds that lock for a short time, the
chat step throws `NativeChatReviewPresentationContentionError`. This
error is not an `HttpError`. The route returns HTTP 500, and the error
handler reports a server crash.
> - The contention is routine and short. The admission transaction does
not contact a provider, and it rolls back completely. The owning service
can safely try it again.
> - This pull request retries the admission a bounded number of times.
If the lock stays busy, the service returns a retryable 409 instead of a
500.
> - The benefit: the user's click usually succeeds without an error, and
Paperclip stops reporting expected lock contention as a crash.
## Linked Issues or Issue Description
**What happened?**
A hosted staging server reported
`NativeChatReviewPresentationContentionError: Chat response
authorization is temporarily busy; no provider delivery was attempted`
from `POST /api/issues/:id/retry`. The stack is `requestTaskRetry` →
`writeExecutionControl` → `prepareFailedChatRunRetry` →
`failedChatRetrySource`. The error came from the
`chat-identity:<company>:<principal>` advisory try-lock. The route
returned HTTP 500 for each occurrence.
**Expected behavior**
Short lock contention must not fail a deliberate Retry click. The
service must wait it out. If the lock stays busy, the service must
return a clear, retryable client error and must not report a server
crash.
**Steps to reproduce**
1. Bind a task to a restricted chat endpoint and conversation. Make the
latest task run a failed run.
2. In a different database transaction, hold
`pg_try_advisory_xact_lock(hashtextextended('chat-identity:<company>:<principal>',
0))`.
3. Call `POST /api/issues/:id/retry`. Before this change, the chat retry
step throws `NativeChatReviewPresentationContentionError` and the route
returns 500.
**Paperclip version or commit**
Reproduced on master `8771606fd87f69ed4057779e3dce3a9bca8088e6`. The
server release that reported it was
`d59791828482f213713ca55000c40e6c9c6c03e5`.
**Deployment mode**
Hosted staging, Node 24, PostgreSQL.
Related: #18845 and #18621 changed nearby Retry admission. They do not
handle this contention.
## What Changed
- `server/src/services/task-retry.ts`: `requestTaskRetry` now calls the
unchanged admission transaction (`admitTaskRetry`) in a bounded loop: 10
attempts, 100 ms apart. It retries only
`NativeChatReviewPresentationContentionError` and lock-not-available
contention (`isExternalChatWaitAuthorizationContention`, which matches
`55P03`). Every other error, including 403/404/409 decisions, propagates
unchanged on the first attempt.
- If the lock stays busy for all attempts, the service throws
`conflict(..., { code: "task_retry_busy", retryable: true })`. The UI's
existing 4xx handling resets the Retry button, and the error handler
does not report a 4xx as a crash.
- `server/src/services/task-retry.test.ts`: three PostgreSQL cases:
- Real advisory-lock contention. A second transaction holds the same
`chat-identity` lock that production checks, then releases it. Retry
succeeds and persists exactly one wake request. A replay with the same
idempotency key returns the same request.
- Persistent contention returns 409 `task_retry_busy` after exactly 10
attempts and persists no wake request.
- A 403 from the chat step does not retry.
The transaction body is unchanged. It was moved into `admitTaskRetry`
without edits.
## Verification
- RED: with the old `task-retry.ts` and the new tests, the contention
case fails with the exact production error
(`NativeChatReviewPresentationContentionError: Chat response
authorization is temporarily busy…`). The persistent case fails because
the error is not an `HttpError`.
- GREEN: `pnpm exec vitest run src/services/task-retry.test.ts` passes
all 39 cases against embedded PostgreSQL.
- `pnpm --filter @paperclipai/server typecheck` passes (all layers,
including tests).
- `src/services/heartbeat/workspace-preparation.test.ts` (another
`requestTaskRetry` caller): 165 of 167 cases pass. The 2 failures
(`remote_workspace_migration_source_changed` in "resumes an imported
path/file-url Git source…") also fail on unchanged master in the same
environment, so this change does not cause them.
- Runtime assumption (rule `runtime-dependency-boundaries`): the retry
is safe only if a failed attempt leaves no persisted state. The
contention test uses a real PostgreSQL advisory lock held by a separate
connection and checks that only one wake row exists after the failed
attempts. Each attempt is a separate transaction, so the loop holds no
row lock or advisory lock while it waits.
- The full repository `pnpm test:run` and `pnpm build` were not run
locally. I rely on CI for those.
## Risks
- Low risk. The worst case adds about 1 second of request time when the
lock stays busy, and then returns 409 instead of 500.
- An attempt can fail after an earlier step in the same transaction (for
example, workspace preservation registration). The whole transaction
rolls back, so the next attempt repeats those steps from a clean state.
Those steps run in the same transaction and do not contact a provider.
- If `requestTaskRetry` is called inside an outer transaction, each
attempt rolls back only to its savepoint. Locks that the outer
transaction holds stay held. In that case the loop ends with the same
409 after its bounded budget.
- No schema, API shape, or UI change.
## Model Used
Claude Opus 5.5 (`claude-opus-5-5`, Anthropic), through Claude Code with
tool use (shell, file editing, test execution) and extended reasoning.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused suites; see
Verification for the two pre-existing failures)
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes (no
behavior or command documentation changes are needed)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>