Do not show Retry on a parent with open subtasks while its queued message is settled (#21320)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The task view shows a "Retry" notice when a task needs a person to
start it again.
> - The product rule is that a task that continues by itself never shows
this notice, not even for a few seconds.
> - A parent task that waits for its subtasks showed the notice for 3.4
seconds after its run succeeded.
> - The cause is the queued-message grace in the task interruption
presentation: it does not apply to `blocked` tasks.
> - #21200 kept `blocked` out of that grace on purpose, so this pull
request adds only the exact observed shape: a parent with open subtasks
and a queued task message.
> - The benefit is that a parent that waits for its subtasks no longer
shows a false failure, and every other blocked task keeps its Retry.

## Linked Issues or Issue Description

Refs #21264, #21200 (added the queued-message grace and kept `blocked`
out of it)

**What happened?**
A Codex parent task ran across a two-server release swap. Its run waited
for two subtasks and then succeeded with the disposition `blocked`. The
task status became `blocked`. A user comment (a deferred wake, status
`deferred_issue_execution`) had been queued 13 minutes earlier, while
the run was active. For 3.4 seconds after the run finished, the task
presentation was `state: "failed"`, `canRetry: true`. Then the release
drain cancelled the wake as "Queued messages were already delivered to a
completed run", and the presentation changed to `resolved`. The subtasks
finished later. Their `issue_children_completed` wake continued the
parent, and it completed without help.

The cause is in `projectTaskInterruptionState` in
`server/src/services/task-interruption-presentation.ts`. The
`queuedDuringRun` check shows a succeeded run with user input queued
before it finished as `working` for up to 2 minutes. It applies only to
`todo`, `in_progress` and `in_review`. For a `blocked` task the
presentation fell through to the last line: `failed` with `canRetry:
true`.

The workspace finalization deferral
(`native_workspace_finalization_busy`) in the same window does not
change the presentation. The presentation does not read it.

**How this differs from what #21200 narrowed**
#21200 first applied the grace to every status. Its commit `a3fae9bb67`
removed `blocked` because input queued on a blocked task is not promoted
by itself, and because a recovery that gives up hands its hold to the
board and must end with Retry. This change does not add `blocked` back.
It adds one narrow case, and all of these must be true:
- The latest run **succeeded**. A failed, interrupted or given-up run is
not affected.
- The task is `blocked` **and has open subtasks of its own** (not `done`
or `cancelled`). Their completion wakes the parent again
(`issue_children_completed`).
- The pending item is an ordinary **task comment** wake (`commentId` or
`wakeCommentIds` in its payload), not a Retry request
(`taskRetry.sourceRunId`).
- The comment was queued before the run finished, the run finished less
than 2 minutes ago, and saved-work recovery is not exhausted. These are
the existing bounds.

A blocked task that waits for a person or a board decision has no open
subtasks, so it keeps Retry as before.

**Expected behavior**
After the run succeeds, the parent shows `working` with no Retry while
its queued message is settled. Then it shows `resolved` (message already
delivered) or `working` (message promoted to a new run). If nothing
settles the message in 2 minutes, Retry shows.

**Steps to reproduce**
1. Assign a parent task that creates subtasks and waits for them.
2. Post a comment on the parent while its run is active.
3. Let the run succeed with the disposition `blocked`.
4. Read the task presentation in the first second after the run
finishes. It shows `failed` with Retry.

**Paperclip version or commit**
master `bbafa78fe2`. The wave that found it ran on `62d673cc3f`.

## What Changed

- `server/src/services/task-interruption-presentation.ts`:
- The batched wait read also reads whether the wake carries a task
comment (`queuedComment`).
- One company-scoped read finds which `blocked` tasks with a succeeded
latest run and a queued comment have open subtasks
(`waitingOnOpenSubtasks`). It runs only for those tasks.
- The `queuedDuringRun` grace also applies to such a task. The `todo`,
`in_progress` and `in_review` behavior does not change.
- `server/src/__tests__/task-interruption-automatic-path.test.ts`, new
embedded-Postgres tests:
- The exact observed state: a `blocked` parent with two open subtasks, a
run that succeeded 1 second ago, a user comment wake queued 13 minutes
earlier. Shows `working`, no Retry.
- The same parent after the wake is cancelled as already delivered:
`resolved`.
- The same parent after the wake is promoted to a new run: `working` on
the new run.
- Guard: the same parent after the 2-minute grace: `failed` with Retry.
- Guard: a `blocked` task with a queued comment and no subtasks, or only
`done` or `cancelled` subtasks (waiting for a person or a board
decision): `failed` with Retry, unchanged.
- Guard: a parent with open subtasks whose queued wake carries no
comment: `failed` with Retry, unchanged.
- Guard: a parent with open subtasks and a queued comment whose run
failed and whose recovery hold went to the board: `failed` with Retry.
- Guard: a parent with open subtasks and a queued comment whose
saved-work recovery is exhausted: `failed` with Retry.
- A parent whose run succeeded with no queued message stays `resolved`
when its subtasks are open, blocked or cancelled.
- The existing #21200 assertion (a `blocked` task with queued input and
no subtasks shows Retry) is kept unchanged.

## Verification

- `cd server && npx vitest run
src/__tests__/task-interruption-automatic-path.test.ts`
- Red on master `bbafa78fe2`: "shows a parent whose run just succeeded
waiting on its subtasks as working while its queued message is settled"
returns `state: "failed"`, `canRetry: true`. This is the value that the
wave recorded. The other 15 tests pass on master.
  - Green with this change: 16 of 16.
- A plain `blocked` inclusion fails 3 of the guards: the #21200
assertion, "still offers Retry on a blocked task with a queued message
and no open subtasks" and "still offers Retry on a parent with open
subtasks when its queued wake carries no message". This shows that the
narrow condition is necessary.
- `task-interruption-presentation.test.ts`,
`task-interruption-query.test.ts`: green (116 tests with the file
above).
- `cd server && npx tsc --noEmit -p .`: 0 errors.
- Not verified: no new chaos wave. The flash was seen once in a loaded
release-swap wave. The server test reproduces its saved state.

## Risks

- Low risk. The change is presentation only. It does not change wake
promotion, dispatch or Retry.
- A parent with open subtasks and a stranded queued comment now shows
`working` for up to 2 minutes after its run succeeded, and then Retry.
Before, it showed Retry at once.
- One extra indexed read (`issues_company_parent_idx`) runs only for
blocked tasks with a succeeded latest run and a queued comment.

## Model Used

- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, in Claude
Code with tool use (shell, file edits, test execution). Extended
reasoning was available.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-13 02:27:22 -06:00
1 parent d9a4bb23fe
commit 4fe34df748
2 files changed
+117 -4

No files matched your search

@@ -4,6 +4,7 @@ import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
import { agents, agentWakeupRequests, applyPendingMigrations, companies, createDb, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS,
heartbeatRuns, issueRecoveryActions, issues, nativeRunFinalizations } from "@paperclipai/db";
import { taskInterruptionPresentation } from "../services/task-interruption-presentation.js";
import { createSystemRecovery } from "../services/system-recovery-state.js";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
import { resolveExternalTestDatabaseUrl } from "./helpers/external-test-database.js";
@@ -96,6 +97,106 @@ describe.skipIf(!support.supported)("Retry is offered only when no automatic pat
expect(await taskInterruptionPresentation(db, issue)).toMatchObject({ canRetry: true });
});
async function seedParentWaitingOnSubtasks(childStatus = "in_progress") {
const { companyId, agentId, issue } = await seedTask("blocked");
for (const title of ["Subtask 1", "Subtask 2"])
await db.insert(issues).values({ companyId, title, status: childStatus, assigneeAgentId: agentId, parentId: issue.id });
const runId = randomUUID(), finishedAt = new Date(Date.now() - 1_000);
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, nativeIssueId: issue.id, scopeKind: "issue",
status: "succeeded", runtimeMode: "native", startedAt: new Date(Date.now() - 25 * 60_000), finishedAt });
return { companyId, agentId, issue, runId };
}
async function seedCommentQueuedDuringRun(companyId: string, agentId: string, issueId: string) {
const commentId = randomUUID();
const [wake] = await db.insert(agentWakeupRequests).values({ companyId, agentId, source: "automation",
reason: "issue_execution_deferred", triggerDetail: "system", status: "deferred_issue_execution",
requestedByActorType: "user", requestedByActorId: "local-board", requestedAt: new Date(Date.now() - 13 * 60_000),
payload: { issueId, mutation: "comment", commentId,
_paperclipWakeContext: { source: "issue.comment", issueId, wakeReason: "issue_commented", wakeCommentIds: [commentId] } },
}).returning();
return wake!;
}
it("shows a parent whose run just succeeded waiting on its subtasks as working while its queued message is settled", async () => {
const { companyId, agentId, issue, runId } = await seedParentWaitingOnSubtasks();
await seedCommentQueuedDuringRun(companyId, agentId, issue.id);
expect(await taskInterruptionPresentation(db, issue)).toMatchObject({ responsibility: "system", state: "working",
canRetry: false, observedRunId: runId });
});
it("shows the parent as resolved once its queued message is cancelled as already delivered", async () => {
const { companyId, agentId, issue } = await seedParentWaitingOnSubtasks();
const wake = await seedCommentQueuedDuringRun(companyId, agentId, issue.id);
await db.update(agentWakeupRequests).set({ status: "cancelled", finishedAt: new Date(),
error: "Queued messages were already delivered to a completed run" }).where(eq(agentWakeupRequests.id, wake.id));
expect(await taskInterruptionPresentation(db, issue)).toMatchObject({ state: "resolved", canRetry: false });
});
it("shows the parent as working once its queued message is promoted to a new run", async () => {
const { companyId, agentId, issue } = await seedParentWaitingOnSubtasks();
const wake = await seedCommentQueuedDuringRun(companyId, agentId, issue.id);
const nextRunId = randomUUID();
await db.insert(heartbeatRuns).values({ id: nextRunId, companyId, agentId, nativeIssueId: issue.id, scopeKind: "issue",
status: "queued", runtimeMode: "native", wakeupRequestId: wake.id });
await db.update(agentWakeupRequests).set({ status: "queued", runId: nextRunId }).where(eq(agentWakeupRequests.id, wake.id));
expect(await taskInterruptionPresentation(db, issue)).toMatchObject({ state: "working", canRetry: false, observedRunId: nextRunId });
});
it("offers Retry on the parent once its queued message is not settled within the grace", async () => {
const { companyId, agentId, issue, runId } = await seedParentWaitingOnSubtasks();
await seedCommentQueuedDuringRun(companyId, agentId, issue.id);
await db.update(heartbeatRuns).set({ finishedAt: new Date(Date.now() - 3 * 60_000) }).where(eq(heartbeatRuns.id, runId));
expect(await taskInterruptionPresentation(db, issue)).toMatchObject({ state: "failed", canRetry: true });
});
it("still offers Retry on a blocked task with a queued message and no open subtasks", async () => {
for (const childStatus of [null, "done", "cancelled"]) {
const { companyId, agentId, issue } = await seedTask("blocked");
if (childStatus) await db.insert(issues).values({ companyId, title: "Finished subtask", status: childStatus,
assigneeAgentId: agentId, parentId: issue.id });
await db.insert(heartbeatRuns).values({ companyId, agentId, nativeIssueId: issue.id, scopeKind: "issue",
status: "succeeded", runtimeMode: "native", finishedAt: new Date(Date.now() - 1_000) });
await seedCommentQueuedDuringRun(companyId, agentId, issue.id);
expect(await taskInterruptionPresentation(db, issue)).toMatchObject({ state: "failed", canRetry: true });
}
});
it("still offers Retry on a parent with open subtasks when its queued wake carries no message", async () => {
const { companyId, agentId, issue } = await seedParentWaitingOnSubtasks();
await db.insert(agentWakeupRequests).values({ companyId, agentId, source: "on_demand", reason: "issue_assigned",
status: "deferred_issue_execution", requestedByActorType: "user", requestedAt: new Date(Date.now() - 13 * 60_000),
payload: { issueId: issue.id } });
expect(await taskInterruptionPresentation(db, issue)).toMatchObject({ state: "failed", canRetry: true });
});
it("still offers Retry on a parent with open subtasks and a queued message when its run failed and recovery gave up", async () => {
const { companyId, agentId, issue, runId, hold } = await seedSwapEndedRun(new Date(Date.now() - 5_000));
await db.update(issues).set({ status: "blocked" }).where(eq(issues.id, issue.id));
await db.insert(issues).values({ companyId, title: "Open subtask", status: "in_progress", assigneeAgentId: agentId, parentId: issue.id });
await seedCommentQueuedDuringRun(companyId, agentId, issue.id);
await db.update(issueRecoveryActions).set({ ownerType: "board" }).where(eq(issueRecoveryActions.id, hold.id));
const blocked = { ...issue, status: "blocked" };
expect(await taskInterruptionPresentation(db, blocked)).toMatchObject({ state: "failed", canRetry: true, observedRunId: runId });
});
it("still offers Retry on a parent with open subtasks and a queued message when saved-work recovery is exhausted", async () => {
const { companyId, agentId, issue, runId } = await seedParentWaitingOnSubtasks();
await seedCommentQueuedDuringRun(companyId, agentId, issue.id);
await db.insert(issueRecoveryActions).values({ companyId, sourceIssueId: issue.id, kind: "active_run_watchdog",
status: "escalated", ownerType: "system", returnOwnerAgentId: agentId, cause: "native_restart_recovery_blocked",
fingerprint: `exhausted:${runId}`, nextAction: "Retry to continue.", maxAttempts: 3,
evidence: { runId, systemRecovery: { ...createSystemRecovery(runId), state: "exhausted", attempts: 3, retriesUsed: 3 } } });
expect(await taskInterruptionPresentation(db, issue)).toMatchObject({ state: "failed", canRetry: true });
});
it("does not show a failed attempt on a parent whose run succeeded with no queued message", async () => {
for (const childStatus of ["in_progress", "blocked", "cancelled"]) {
const { issue } = await seedParentWaitingOnSubtasks(childStatus);
expect(await taskInterruptionPresentation(db, issue)).toMatchObject({ state: "resolved", canRetry: false });
}
});
it("does not offer Retry on a parent whose run succeeded and that waits for its subtasks", async () => {
const { companyId, agentId, issue } = await seedTask();
await db.insert(issues).values({ companyId, title: "Subtask", status: "in_progress", assigneeAgentId: agentId, parentId: issue.id });
@@ -1,7 +1,7 @@
import { STORAGE_UNAVAILABLE_WAIT_RETRY_REASON } from "../storage/storage-unavailable.js";
import { AGENT_DIRECTORY_WAIT_RETRY_REASON } from "./execution-recovery-attempt.js";
import { boundSetupCommandFailure, readSetupCommandFailure } from "./workspace-setup-command-failure.js";
import { and, desc, eq, inArray, isNull, not, sql, type SQL } from "drizzle-orm";
import { and, desc, eq, inArray, isNull, not, notInArray, sql, type SQL } from "drizzle-orm";
import { unidentifiedLocalLaunchRefusals } from "./native-local-process-stop.js";
import { nativeSessionCleanupBlock } from "../vendor/paperclip-runner/index.js";
import { agents, agentWakeupRequests, approvals, chatActions, companies, environmentLeases, heartbeatRuns, issueApprovals, issueRecoveryActions,
@@ -65,7 +65,8 @@ type ProjectionRun = Pick<typeof heartbeatRuns.$inferSelect,
type ProjectionWait = { id: string; requestedAt: Date; sourceRunId?: string | null; executionWaitReason?: string | null;
preparationStatus?: string | null; preparationRetryAt?: number | string | null; preparationUpdatedAt?: Date | null;
/** When the user pressed Interrupt to deliver this queued message now. */
queuedInterruptRequestedAt?: string | null };
queuedInterruptRequestedAt?: string | null;
queuedComment?: boolean };
type ProjectionState = {
run?: ProjectionRun;
action?: typeof issueRecoveryActions.$inferSelect;
@@ -89,6 +90,7 @@ type ProjectionState = {
retryDependencies?: RetryDependencies;
/** The task waits on unresolved blocker tasks, a wait that does not depend on its last run. */
waitingOnBlockers?: boolean;
waitingOnOpenSubtasks?: boolean;
/** A live owner is settling this run's session cleanup gate: an unreleased
* remote environment, or a scheduled close in this process. Without one,
* nothing settles the gate by itself, so Retry is the way forward. */
@@ -345,7 +347,8 @@ function projectTaskInterruptionState(issue: ProjectionIssue, input: ProjectionS
canRetry: heldRetryOffersRetry(reason, input.cleanupSettling), retryWait: { reason,
message: retryWaitMessages[reason] ?? "Retry is waiting for execution checks. Your message is saved." } };
}
const queuedDuringRun = Boolean(wait && run.status === "succeeded" && ["todo", "in_progress", "in_review"].includes(issue.status) && run.finishedAt && wait.requestedAt.getTime() <= run.finishedAt.getTime()
const queuedDuringRun = Boolean(wait && run.status === "succeeded" && (["todo", "in_progress", "in_review"].includes(issue.status)
|| (issue.status === "blocked" && input.waitingOnOpenSubtasks && !wait.sourceRunId && wait.queuedComment)) && run.finishedAt && wait.requestedAt.getTime() <= run.finishedAt.getTime()
&& now - run.finishedAt.getTime() < QUEUED_INPUT_DISPATCH_GRACE_MS);
if (wait && (wait.requestedAt.getTime() > now - 10_000 || queuedDuringRun) && system?.state !== "exhausted")
return { ...base, state: "working" };
@@ -496,7 +499,10 @@ export async function taskInterruptionPresentations(db: Db, companyId: string, i
executionWaitReason: boundedJsonString(sql`to_jsonb(${unpricedUsageWait(sql`${agentWakeupRequests.payload}#>>'{executionWait,reason}'`,
sql`${agentWakeupRequests.payload}#>>'{executionWait,message}'`)})`, 80),
preparationStatus: chatActions.status, preparationRetryAt: chatActionDeadline("failed_run_retry"),
preparationUpdatedAt: chatActions.updatedAt })
preparationUpdatedAt: chatActions.updatedAt,
queuedComment: sql<boolean>`coalesce(jsonb_typeof(${agentWakeupRequests.payload}->'commentId') = 'string'
or jsonb_array_length(case when jsonb_typeof(${agentWakeupRequests.payload}#>'{_paperclipWakeContext,wakeCommentIds}') = 'array'
then ${agentWakeupRequests.payload}#>'{_paperclipWakeContext,wakeCommentIds}' end) > 0, false)` })
.from(agentWakeupRequests).leftJoin(chatActions, and(eq(chatActions.companyId, agentWakeupRequests.companyId),
eq(chatActions.id, canonicalExecutionUuid(sql`${agentWakeupRequests.payload}#>>'{taskRetry,chatActionId}'`)),
eq(chatActions.kind, "failed_run_retry"), sql`${chatActions.payload}->>'taskRetryIntentId' = ${agentWakeupRequests.id}::text`,
@@ -670,6 +676,11 @@ export async function taskInterruptionPresentations(db: Db, companyId: string, i
for (const [waitingIssueId, row] of readiness)
if (row.unresolvedBlockerIssueIds.length) waitingOnBlockers.add(waitingIssueId);
}
const parentIds = selected.filter(issue => issue.status === "blocked" && runMap.get(issue.id)?.status === "succeeded"
&& currentWaits.some(wait => wait.issueId === issue.id && wait.queuedComment)).map(issue => issue.id);
const waitingOnOpenSubtasks = new Set(parentIds.length ? (await db.selectDistinct({ parentId: issues.parentId }).from(issues)
.where(and(eq(issues.companyId, companyId), inArray(issues.parentId, parentIds), notInArray(issues.status, ["done", "cancelled"]))))
.flatMap(row => row.parentId ? [row.parentId] : []) : []);
for (const wait of currentWaits) {
const current = waitMap.get(wait.issueId);
if (!current || wait.requestedAt > current.requestedAt ||
@@ -736,6 +747,7 @@ export async function taskInterruptionPresentations(db: Db, companyId: string, i
admissionWait: admissionWaitMap.get(issue.id),
retryDependencies: retryDependencyMap.get(issue.id),
waitingOnBlockers: waitingOnBlockers.has(issue.id),
waitingOnOpenSubtasks: waitingOnOpenSubtasks.has(issue.id),
cleanupSettling: cleanupSettling.has(issue.id),
}, now);
result.set(issue.id, presentation);