From 7eaf38130beb8460e40e8a8120da3311e1ef0461 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Thu, 19 Nov 2026 02:43:48 -0600 Subject: [PATCH] fix(tasks): keep a due automatic retry working while it is dispatched (#22306) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - When a run fails for a reason that Paperclip retries by itself, Paperclip saves a new run in the `scheduled_retry` state. The task shows "retry scheduled" and does not show a Retry button. > - When the retry time comes, the run stays `scheduled_retry` until the dispatcher promotes it to `queued`. This takes a few seconds. > - In that gap, the task view does not see a future retry time and does not see a queued run. So it falls through to "failed, Retry available". > - In a restart test wave, two tasks showed a Retry banner for about 4 seconds this way. Both then continued without a click. A person can click Retry in that window and start a second, competing retry. > - This pull request treats a due automatic retry the same way as a due native resume: the task shows "working" while the dispatcher promotes it, for the same 2 minute grace. > - The benefit is that an automatic retry never shows a Retry button. A retry that never starts still shows one Retry after the grace. ## Linked Issues or Issue Description **What happened?** After an automatic retry became due, the task showed "failed" with a Retry button for one poll (about 4 seconds). Then the retry started by itself. Two tasks in a 197-task restart test wave showed this. One retry came from an environment provider timeout (`environment_provider_unavailable`). The other came from a `native_environment_blocker` retry. **Expected behavior** A task whose automatic retry is due and not yet dispatched shows that Paperclip is working. It does not offer Retry. **Steps to reproduce** 1. Make a run fail with a cause that Paperclip retries automatically (for example a transient failure). 2. Watch the task while the retry time passes. 3. Before the dispatcher promotes the retry run to `queued`, the task interruption shows `state: "failed"` and `canRetry: true`. **Paperclip version or commit** `master` at 1c67687998. ## What Changed - `server/src/services/task-interruption-presentation.ts`: a `scheduled_retry` run whose retry time has passed shows `working` (no Retry) for `NATIVE_RESUME_DISPATCH_GRACE_MS` (2 minutes), the same rule that a due native resume already uses. After the grace, a retry that never started still falls through to one Retry. - `server/src/services/task-interruption-presentation.test.ts`: adds a regression test for a just-due `transient_failure` and `native_environment_blocker` retry (0 s to 119 s overdue). One existing assertion now uses a retry that is 10 minutes overdue, so it still proves that a stuck retry offers Retry. ## Verification - Red: with the old source, the 2 new tests fail (`state: "failed"`, `canRetry: true`). - Green: `cd server && npx vitest run src/services/task-interruption-presentation.test.ts` passes (75 tests). - `pnpm verify` for the affected tests (result in the PR comments below if it changes). - Evidence from the wave: for task t118 the samples show `retry_scheduled` from 06:27:40 to 06:28:03, then `failed`/`canRetry: true` at 06:28:07 while the retry run was still `scheduled_retry`, then `working` at 06:28:11 when it was `queued`. Task t181 shows the same sequence at 06:45:22. ## Risks - Low risk. This changes only presentation. It does not change dispatch or retry state. - A retry that is due but cannot start now shows "working" for up to 2 minutes before Retry appears. This is the same grace that a native resume already has. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edit). Extended reasoning on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 --- .../task-interruption-presentation.test.ts | 14 ++++++++++++-- .../src/services/task-interruption-presentation.ts | 3 ++- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/server/src/services/task-interruption-presentation.test.ts b/server/src/services/task-interruption-presentation.test.ts index 579cc7c9ae4..10970b0cfcf 100644 --- a/server/src/services/task-interruption-presentation.test.ts +++ b/server/src/services/task-interruption-presentation.test.ts @@ -436,8 +436,9 @@ describe("scheduled resource wait presentation", () => { }); it("keeps failure retries, user gates and failed Retry preparation authoritative", () => { - expect(projectTaskInterruption(issue, { run: { ...waiting, scheduledRetryReason: "transient_failure" }, - agent, paused: false, approval: false }, now)).toMatchObject({ state: "failed", canRetry: true }); + expect(projectTaskInterruption(issue, { run: { ...waiting, scheduledRetryReason: "transient_failure", + scheduledRetryAt: new Date(now - 10 * 60_000) }, agent, paused: false, approval: false }, now)) + .toMatchObject({ state: "failed", canRetry: true }); expect(projectTaskInterruption(issue, { run: waiting, agent, paused: true, approval: false }, now)) .toMatchObject({ state: "waiting_for_user", userAction: "resume" }); expect(projectTaskInterruption(issue, { run: waiting, agent, paused: false, approval: false, @@ -570,6 +571,15 @@ describe("automatic scheduled retry", () => { .toMatchObject({ state: "retry_scheduled", canRetry: false, nextRetryAt: new Date(now + 60_000).toISOString() }); }); + it.each(["transient_failure", "native_environment_blocker"])( + "keeps a just-due %s retry working while the dispatcher promotes it", scheduledRetryReason => { + for (const overdueMs of [0, 1_000, 30_000, 119_000]) { + expect(projectTaskInterruption({ ...issue, status: "in_progress" }, { run: { ...scheduled, runtimeMode: "native", + scheduledRetryReason, scheduledRetryAt: new Date(now - overdueMs) }, agent, paused: false, approval: false }, now)) + .toMatchObject({ state: "working", canRetry: false }); + } + }); + it("offers Retry once the scheduled retry is past due with nothing else pending", () => { expect(projectTaskInterruption(issue, { run: { ...scheduled, scheduledRetryAt: new Date(now - 10 * 60_000) }, agent, paused: false, approval: false }, now)).toMatchObject({ state: "failed", canRetry: true }); diff --git a/server/src/services/task-interruption-presentation.ts b/server/src/services/task-interruption-presentation.ts index 6debf54ce92..7222d614954 100644 --- a/server/src/services/task-interruption-presentation.ts +++ b/server/src/services/task-interruption-presentation.ts @@ -374,7 +374,8 @@ function projectTaskInterruptionState(issue: ProjectionIssue, input: ProjectionS if (retryAt && retryAt > now) return { ...base, state: "retry_scheduled", nextRetryAt: new Date(retryAt).toISOString(), canRetry: !preparationPending && !nativeResumeAt && run.status !== "scheduled_retry", ...(system?.nextAttemptAt && recoveryStep ? { recoveryStep } : {}) }; - if (nativeResumeAt && !system && now - nativeResumeAt < NATIVE_RESUME_DISPATCH_GRACE_MS) + const dispatchDueAt = nativeResumeAt ?? (run.status === "scheduled_retry" ? run.scheduledRetryAt?.getTime() : undefined); + if (dispatchDueAt && !system && now - dispatchDueAt < NATIVE_RESUME_DISPATCH_GRACE_MS) return { ...base, state: "working" }; if (recoveryRunning) return { ...base, state: "working", ...(recoveryStep ? { recoveryStep } : {}) }; const providerRecoveryAt = run.providerRecoveryAt?.getTime();