From 9ceaec6f97ec988d898c042d2a37377bbe6bc48a Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Thu, 5 Nov 2026 19:08:27 -0600 Subject: [PATCH] fix(issues): stop showing resource waits as failed attempts (#19369) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The task page shows one interruption banner. The server builds it from the task's latest run. > - An auto-retry that waits for a busy workspace, AI connection or runner update is not a failure. The dispatcher re-checks the resource, so the run can stay `scheduled_retry` after its retry time. > - The projection did not know about these waits. After the retry time passed, it fell through to "Couldn't finish this attempt" with a Retry button. > - Retry cannot free the resource. The banner cleared for the 10-second grace and then came back. Users had no way to fix it. > - This pull request projects a scheduled resource wait as `working`, with no Retry. > - The benefit is that the failed-attempt banner shows only for real failures. ## Linked Issues or Issue Description **What happened?** During onboarding on Paperclip Cloud staging (Claude subscription and Codex subscription paths), the onboarding task showed "Couldn't finish this attempt. Your message is saved." with a Retry button. The banner did not name an attempt. Retry cleared it briefly, then it came back. **Steps to reproduce** 1. Start a task run while another run holds the same workspace. 2. The run is cancelled with `workspace_busy`, and a `scheduled_retry` run is created with `scheduledRetryReason: "workspace_busy"`. 3. Keep the workspace busy after `scheduledRetryAt`. On the staging company, one retry stayed scheduled for 16 minutes past its retry time. 4. Open the task. The failed-attempt banner shows. Click Retry. The banner comes back. **Expected behavior** No failed-attempt banner while the system waits for a shared resource. The run status already says "Waiting for workspace" (`execution-projection.ts`). ## What Changed - `server/src/services/task-interruption-presentation.ts`: a `scheduled_retry` run whose `scheduledRetryReason` is `workspace_busy`, `ai_connection_busy`, `ai_connection_pool_wait` or `runner_revision_wait` (and that has no system recovery) projects as `state: "working"`, `canRetry: false`. User gates and failed Retry preparations stay authoritative because they are checked first. - The batched query now selects `scheduledRetryReason`. - Tests: a Postgres integration test (`task-interruption-query.test.ts`) that seeds the staging run chain for each reason, and unit tests for the guard cases (failure-lane retries, paused agent, failed Retry preparation). ## Verification Red on `master` (fix not applied), `cd server && npx vitest run src/__tests__/task-interruption-query.test.ts -t "overdue"`: ``` × does not present an overdue workspace_busy wait as a failed attempt 213ms × does not present an overdue ai_connection_busy wait as a failed attempt 25ms × does not present an overdue ai_connection_pool_wait wait as a failed attempt 13ms × does not present an overdue runner_revision_wait wait as a failed attempt 11ms - Expected + Received - "canRetry": false, + "canRetry": true, - "state": "working", + "state": "failed", Tests 4 failed | 11 skipped (15) ``` Green with the fix, same tests plus the new unit tests: ``` ✓ task-interruption-query.test.ts > does not present an overdue workspace_busy wait as a failed attempt ✓ task-interruption-query.test.ts > does not present an overdue ai_connection_busy wait as a failed attempt ✓ task-interruption-query.test.ts > does not present an overdue ai_connection_pool_wait wait as a failed attempt ✓ task-interruption-query.test.ts > does not present an overdue runner_revision_wait wait as a failed attempt ✓ task-interruption-presentation.test.ts > scheduled resource wait presentation > keeps an overdue workspace wait out of the failed-attempt UI ✓ task-interruption-presentation.test.ts > scheduled resource wait presentation > keeps failure retries, user gates and failed Retry preparation authoritative ``` Neighbouring suites: `task-interruption-presentation`, `task-interruption-query`, `workspace-recovery`, `agent-project-access`, `heartbeat-dependency-scheduling`: 5 files, 458 tests passed. Server `tsc --noEmit` reports no errors in changed files. The only errors locally come from the unbuilt `@paperclipai/paperclip-runner` vendor package, which this change does not touch. No UI change, so `pnpm check:token-gates` is not applicable. Evidence: the staging onboarding task's run history shows `workspace_busy` cancellations at 17:46, 17:52 and 18:10, each followed by a `scheduled_retry`. One had `scheduledRetryAt` 17:53:57 and started at 18:10:20. The next run succeeded at 18:20. ## Risks - Low risk. Presentation only; dispatch and admission do not change. - If a resource wait never clears, the task shows no failed-attempt banner. The run status still shows "Waiting for workspace", and the existing retry-attempt limits still end the wait. - Not in scope: genuine failures without a provider or workspace explanation still show the generic text, and an overdue failure-lane retry still shows as failed. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, browser for read-only staging evidence). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 --- .../__tests__/task-interruption-query.test.ts | 20 ++++++++++++++++++ .../task-interruption-presentation.test.ts | 21 +++++++++++++++++++ .../task-interruption-presentation.ts | 14 ++++++++++++- 3 files changed, 54 insertions(+), 1 deletion(-) diff --git a/server/src/__tests__/task-interruption-query.test.ts b/server/src/__tests__/task-interruption-query.test.ts index a179045cb39..626c59d52cc 100644 --- a/server/src/__tests__/task-interruption-query.test.ts +++ b/server/src/__tests__/task-interruption-query.test.ts @@ -222,4 +222,24 @@ describe.skipIf(!support.supported)("task interruption latest run query", () => taskId: reassigned!.id, agentAdapter: "process", })); }); + // N-03: ONB-1 on staging waited 16 minutes on an overdue workspace_busy + // retry. Nothing failed, yet the banner offered a Retry that cannot help. + it.each(["workspace_busy", "ai_connection_busy", "ai_connection_pool_wait", "runner_revision_wait"])( + "does not present an overdue %s wait as a failed attempt", async reason => { + const companyId = randomUUID(), agentId = randomUUID(), parentRunId = randomUUID(); + await db.insert(companies).values({ id: companyId, name: "Waiting task", issuePrefix: `W${companyId.slice(0, 7)}` }); + await db.insert(agents).values({ id: agentId, companyId, name: "Waiting agent", adapterType: "process" }); + const [issue] = await db.insert(issues).values({ companyId, title: "Paperclip onboarding", + status: "in_progress", assigneeAgentId: agentId }).returning(); + const now = Date.now(); + await db.insert(heartbeatRuns).values({ id: parentRunId, companyId, agentId, issueId: issue!.id, scopeKind: "issue", + status: "cancelled", errorCode: reason, createdAt: new Date(now - 10 * 60_000), finishedAt: new Date(now - 9 * 60_000) }); + await db.insert(heartbeatRuns).values({ companyId, agentId, issueId: issue!.id, scopeKind: "issue", + status: "scheduled_retry", retryOfRunId: parentRunId, scheduledRetryReason: reason, scheduledRetryAttempt: 2, + scheduledRetryAt: new Date(now - 5 * 60_000), createdAt: new Date(now - 9 * 60_000) }); + captureTaskInterruptionFailure.mockClear(); + const presentation = (await taskInterruptionPresentations(db, companyId, [issue!])).get(issue!.id); + expect(presentation).toMatchObject({ responsibility: "system", state: "working", canRetry: false }); + expect(captureTaskInterruptionFailure).not.toHaveBeenCalled(); + }); }); diff --git a/server/src/services/task-interruption-presentation.test.ts b/server/src/services/task-interruption-presentation.test.ts index 9be38f14077..749e952a807 100644 --- a/server/src/services/task-interruption-presentation.test.ts +++ b/server/src/services/task-interruption-presentation.test.ts @@ -259,3 +259,24 @@ describe("saved-work recovery step", () => { paused: false, approval: false }, now)).not.toHaveProperty("recoveryStep"); }); }); + +describe("scheduled resource wait presentation", () => { + const now = Date.parse("2026-01-01T12:00:00Z"); + const waiting = { ...run, status: "scheduled_retry", errorCode: null, resultJson: null, + scheduledRetryReason: "workspace_busy", scheduledRetryAt: new Date(now - 60_000) }; + + it("keeps an overdue workspace wait out of the failed-attempt UI", () => { + expect(projectTaskInterruption(issue, { run: waiting, agent, paused: false, approval: false }, now)) + .toMatchObject({ state: "working", canRetry: false }); + }); + + it("keeps failure retries, user gates and failed Retry preparation authoritative", () => { + expect(projectTaskInterruption(issue, { run: { ...waiting, scheduledRetryReason: "transient_failure" }, + agent, paused: false, approval: false }, now)).toMatchObject({ state: "failed", canRetry: true }); + expect(projectTaskInterruption(issue, { run: waiting, agent, paused: true, approval: false }, now)) + .toMatchObject({ state: "waiting_for_user", userAction: "resume" }); + expect(projectTaskInterruption(issue, { run: waiting, agent, paused: false, approval: false, + retryWait: { id: "retry", sourceRunId: run.id, requestedAt: new Date(now), preparationStatus: "failed" } }, now)) + .toMatchObject({ state: "failed", canRetry: true }); + }); +}); diff --git a/server/src/services/task-interruption-presentation.ts b/server/src/services/task-interruption-presentation.ts index 761102fa847..ea8f1402b25 100644 --- a/server/src/services/task-interruption-presentation.ts +++ b/server/src/services/task-interruption-presentation.ts @@ -17,11 +17,20 @@ import { logger } from "../middleware/logger.js"; import { canonicalExecutionUuid } from "./execution-identity.js"; import { chatActionDeadline } from "./chat-action-work.js"; import { isQueuedDependencyWait } from "./run-cancellation.js"; +import { AI_CONNECTION_BUSY_RETRY_REASON, AI_CONNECTION_POOL_WAIT_RETRY_REASON, RUNNER_REVISION_WAIT_RETRY_REASON, + WORKSPACE_BUSY_RETRY_REASON } from "../modules/run-dispatch/index.js"; + +// These scheduled runs wait for a shared resource; none records a failed +// attempt. Their dispatcher re-checks the resource, so the run can stay +// scheduled past its retry time. Retry cannot free the resource either. +const resourceWaitRetryReasons = new Set([WORKSPACE_BUSY_RETRY_REASON, AI_CONNECTION_BUSY_RETRY_REASON, + AI_CONNECTION_POOL_WAIT_RETRY_REASON, RUNNER_REVISION_WAIT_RETRY_REASON]); type ProjectionIssue = Pick; type ProjectionRun = Pick & - Partial>; + Partial>; type ProjectionWait = { id: string; requestedAt: Date; sourceRunId?: string | null; executionWaitReason?: string | null; preparationStatus?: string | null; preparationRetryAt?: number | string | null }; type ProjectionState = { @@ -132,6 +141,8 @@ export function projectTaskInterruption(issue: ProjectionIssue, input: Projectio return { ...base, state: "retry_scheduled", nextRetryAt: new Date(deadline).toISOString() }; if (!pending) return { ...base, state: "failed", canRetry: retryWait.preparationStatus !== "cancelled" }; } + if (run.status === "scheduled_retry" && resourceWaitRetryReasons.has(run.scheduledRetryReason ?? "") && !system) + return { ...base, state: "working" }; const retryAt = system?.nextAttemptAt ?? run.scheduledRetryAt?.getTime(); if (retryAt && retryAt > now) return { ...base, state: "retry_scheduled", nextRetryAt: new Date(retryAt).toISOString(), canRetry: !preparationPending, @@ -184,6 +195,7 @@ export async function taskInterruptionPresentations(db: Db, companyId: string, i and ${heartbeatRuns.error} = 'Project is outside this low-trust boundary.' or ${heartbeatRuns.errorCode} = 'workspace_validation_failed' then ${heartbeatRuns.error} else null end`, errorCode: heartbeatRuns.errorCode, livenessState: heartbeatRuns.livenessState, scheduledRetryAt: heartbeatRuns.scheduledRetryAt, + scheduledRetryReason: heartbeatRuns.scheduledRetryReason, resultJson: sql>`jsonb_build_object('configurationIncomplete', ${heartbeatRuns.resultJson}->'configurationIncomplete', 'stopReason', ${boundedJsonString(sql`${heartbeatRuns.resultJson}->'stopReason'`, 80)}, 'timeoutSource', ${boundedJsonString(sql`${heartbeatRuns.resultJson}->'timeoutSource'`, 80)},