Files
PaperClipAI/ui/src/components/ExecutionBlockerNotice.test.tsx
T
DottaandClaude Opus 5.5 08f7ac03dc fix(recovery): keep a never-started task waiting through a temporary accounting hold (#19702)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Admission checks every wake. A short company accounting hold can
refuse new work after a control-plane restart.
> - When the hold refused a new task's assignment wake, stranded-work
recovery found the task with no run while the hold was still active. It
moved the task to `blocked` with a board recovery comment.
> - A `blocked` task leaves the stranded-assignment sweep, so nothing
woke it after the hold cleared. The task stayed `blocked` with no run,
no interruption and no Retry.
> - The hold was temporary. Tasks created a few seconds later (after the
sweep) started normally once accounting settled.
> - This pull request keeps a never-started task `todo` while a
temporary budget hold lasts. The existing assigned-todo liveness
dispatch then admits it once the hold clears. The task shows the hold it
waits for.
> - The benefit is that a restart no longer silently drops new tasks
created during the accounting settle window.

## Linked Issues or Issue Description

**What happened?**
In a restart-chaos storm on master `06bbcea12f`, three new tasks were
created while the company accounting hold was active. Each assignment
wake got 409 "Company cannot start work while completed runs await
accounting." (or "...native run recovers its unfinished accounting.").
One task rested `blocked` with 0 runs, interruption `null` and
`canRetry=false`. The other two ran later.

**Root cause**
`reconcileStrandedAssignedIssues`
(`server/src/services/recovery/service.ts`, the
`readInvocationBudgetBlock` branch near line 4826) escalates any
assigned task to `blocked` when a budget block exists. It does this
before the assigned-todo no-run branch, which already skips a
budget-blocked task and dispatches it later. So a task that never
started was treated as failed recovery work. The refused wake itself is
stored as a `skipped` `budget.blocked` diagnostic, which is correct; the
escalation removed the only path that re-admits it.

**Expected behavior**
A wake refused by a temporary admission hold waits and is admitted once
the hold clears, with one run. A real hard stop is still a board
decision. Removed assignees and cancelled tasks still do not wake.

**Steps to reproduce**
Create a company with a hard-stop budget and a completed run that has
`cost_accounting_pending = true`. Assign a new `todo` task. The wake
gets 409. Run the stranded-assignment sweep. Before this change the task
becomes `blocked` and never runs.

Related: #19700 bounds the accounting hold itself. This PR is
independent: it handles what happens to a task whose first wake the hold
refused.

## What Changed

- Stranded-work recovery skips a `todo` task with no run while a
temporary budget hold blocks admission. It does not escalate it to
`blocked`. The existing assigned-todo liveness dispatch admits the task
once the hold clears.
- A hard stop (spend at the limit, or blocked unpriced usage) keeps the
existing board escalation. Recovery checks this with the read-only
`policyBlocks` for the policies of the blocking scope. It does not
change budget hold semantics.
- The task interruption projection now covers a `todo` task with no run
whose latest wake was refused with `budget.blocked`. It shows "Waiting
for the budget to allow new work. The task starts automatically when it
does." with no Retry. A budget pause shows the existing "budget" user
action.
- The no-run projection applies the same person-owned gates as the
run-backed projection: a budget pause shows the "budget" action, and an
ordinary agent, project, company or task pause shows "resume". Only an
accounting hold shows the automatic-start text.
- `ExecutionBlockerNotice` shows that wait text for a `working`
presentation that carries a wait reason.
- `doc/execution-semantics.md` records the rule.

## Verification

- New embedded-Postgres regression
`server/src/__tests__/heartbeat-refused-assignment-wake.test.ts`:
- hold active, new task assigned, wake refused (409, `skipped`
`budget.blocked`), two recovery sweeps: the task stays `todo` with no
run and shows the wait reason. Before the fix this failed with `expected
'blocked' to be 'todo'` (the CHA-9 shape).
- hold cleared, two sweeps: exactly one provider turn starts, from
`issue.assigned_todo_liveness_dispatch`, and the task finishes. A later
sweep does not start another turn.
- counter-tests: a terminated assignee and a cancelled task get no run
after the hold clears.
- Existing `creates a board recovery action for budget-blocked assigned
work` (real hard stop) still passes unchanged.
- `pnpm exec vitest run
src/__tests__/heartbeat-refused-assignment-wake.test.ts
src/__tests__/heartbeat-process-recovery.test.ts
src/services/task-interruption-presentation.test.ts
src/__tests__/heartbeat-issue-liveness-escalation.test.ts` (server) and
`src/components/ExecutionBlockerNotice.test.tsx` (ui).
- Server typecheck and `pnpm check:token-gates`.
- Live restart-chaos storm (local, codex/claude/opencode, 6 in flight,
K2 every 3-4 minutes for 20 minutes, no RC-06 mask), revision
e38bb5c8f7: 17 tasks, 10 pass.
- Two new tasks (CHA-16, CHA-17) were created while the company
accounting hold was active. Their assignment wakes got 409 "Company
cannot start work while completed runs await accounting." Both stayed
`todo` (not `blocked`) and showed `state: working`, `canRetry: false`,
`retryWait.reason: budget.blocked`, "Waiting for the budget to allow new
work. The task starts automatically when it does." No task rested
`blocked` with 0 runs.
- The hold itself never cleared in this wave (`pendingRunCount: 1` at
wave end), so these two tasks did not start before their deadline. That
stuck hold is the separate accounting-closure defect that #19700 bounds.
The other 5 failures (`R3:retry_no_new_run:B5`) are Retries refused by
the same stuck hold. The re-admission after the hold clears is proved by
the embedded-Postgres regression above.

## Risks

- Low. The change narrows one escalation to tasks that never started,
under a budget block that is not a hard stop. Those tasks now stay
`todo` and are re-checked by the existing 30-second stranded-assignment
sweep, which already re-checks every admission gate.
- The hard-stop check reads the blocking scope's active policies once
per budget-blocked never-started task in a sweep. It runs only while a
hold is active.
- The interruption projection adds one indexed read (company + payload
issue id) for `todo` tasks in the batch.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 21:12:00 -06:00

122 lines
7.9 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
// @vitest-environment jsdom
import { act } from "react";
import { createRoot, type Root } from "react-dom/client";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import type { TaskInterruptionPresentation } from "@paperclipai/shared";
import { ExecutionBlockerNotice } from "./ExecutionBlockerNotice";
const failed: TaskInterruptionPresentation = {
intentId: "intent", responsibility: "system", state: "failed", nextRetryAt: null,
canRetry: true, userAction: null, sourceRunIds: ["source"], observedRunId: "source",
};
describe("ordinary task interruption", () => {
let root: Root;
let container: HTMLDivElement;
const retry = vi.fn();
beforeEach(() => {
retry.mockReset(); container = document.createElement("div"); document.body.append(container);
root = createRoot(container);
});
afterEach(async () => { await act(async () => root.unmount()); container.remove(); vi.useRealTimers(); });
const render = async (presentation = failed, options: { canRetry?: boolean; optimistic?: boolean; takingLonger?: boolean; error?: Error } = {}) => {
await act(async () => root.render(<ExecutionBlockerNotice presentation={presentation} canRetry onRetry={retry} {...options} />));
};
it("offers one Retry without delegating infrastructure chores", async () => {
await render();
expect(container.textContent).toContain("Couldn’t finish this attempt. Your message is saved.");
expect(container.querySelectorAll("button")).toHaveLength(1);
await act(async () => container.querySelector<HTMLButtonElement>("button")!.click());
expect(retry).toHaveBeenCalledOnce();
expect(container.querySelector("a, details, input, textarea")).toBeNull();
expect(container.textContent).not.toMatch(/service attention|process|receipt|workspace/i);
});
it("shows persisted backoff with a usable Retry", async () => {
vi.useFakeTimers(); vi.setSystemTime(new Date("2026-10-20T00:00:00Z"));
await render({ ...failed, state: "retry_scheduled", nextRetryAt: "2026-10-20T00:00:30Z" });
expect(container.textContent).toContain("Trying again in 30s");
await act(async () => { await vi.advanceTimersByTimeAsync(5_000); });
expect(container.textContent).toContain("Trying again in 25s");
expect(container.querySelector("button")?.textContent).toBe("Retry");
});
it.each(["working", "resolved"] as const)("does not duplicate the thread indicator for %s", async state => {
await render({ ...failed, state }); expect(container.textContent).toBe("");
});
it("names the admission hold a task that has not started waits for", async () => {
await render({ ...failed, state: "working", canRetry: false, sourceRunIds: [], observedRunId: null,
retryWait: { reason: "budget.blocked", message: "Waiting for the budget to allow new work. The task starts automatically when it does." } });
expect(container.textContent).toBe("Waiting for the budget to allow new work. The task starts automatically when it does.");
expect(container.querySelector("button")).toBeNull();
});
it("names the saved-work recovery step instead of a bare queue", async () => {
await render({ ...failed, state: "working", canRetry: false, recoveryStep: "saving_previous_files" });
expect(container.textContent).toBe("Saving files from the previous attempt…");
expect(container.querySelector("button")).toBeNull();
});
it("names the step a scheduled recovery will retry", async () => {
vi.useFakeTimers(); vi.setSystemTime(new Date("2026-10-20T00:00:00Z"));
await render({ ...failed, state: "retry_scheduled", nextRetryAt: "2026-10-20T00:00:30Z", recoveryStep: "checking_previous_run" });
expect(container.textContent).toContain("Checking that the previous attempt stopped. Trying again in 30s…");
expect(container.querySelector("button")?.textContent).toBe("Retry");
});
it("names the recovery step inside a provider failure notice", async () => {
vi.useFakeTimers(); vi.setSystemTime(new Date("2026-10-20T00:00:00Z"));
await render({ ...failed, state: "retry_scheduled", nextRetryAt: "2026-10-20T00:00:30Z", recoveryStep: "checking_previous_run",
providerFailure: { schema: "paperclip.provider-failure.v1", provider: "openai", phase: "startup", category: "overloaded",
retryability: "unknown", message: "Try later." } });
expect(container.textContent).toContain("Checking that the previous attempt stopped. Trying again in 30s…");
expect(container.textContent).not.toContain("Paperclip will retry automatically.");
});
it("leaves genuine user decisions to their specific action surface", async () => {
await render({ ...failed, responsibility: "user", state: "waiting_for_user", userAction: "approval", canRetry: false });
expect(container.textContent).toBe("");
});
it("does not give read-only users a mutation", async () => {
await render(failed, { canRetry: false }); expect(container.querySelector("button")).toBeNull();
});
it("removes the stale failure immediately while admission is pending", async () => {
await render(failed, { optimistic: true }); expect(container.textContent).toBe("");
await render(failed, { optimistic: true, takingLonger: true });
expect(container.textContent).toContain("Taking longer than usual…");
});
it("shows a definitive admission rejection instead of silently dropping Retry", async () => {
await render(failed, { error: new Error("Your task access changed. Refresh the task.") });
expect(container.querySelector('[role="alert"]')?.textContent).toContain("Your task access changed.");
expect(container.querySelectorAll("button")).toHaveLength(1);
});
it("shows workspace preparation failures as plain text while keeping retry permissions", async () => {
const workspaceFailure = { message: 'Could not prepare the project folder <img src=x onerror=alert(1)>.' };
await render({ ...failed, workspaceFailure }, { canRetry: false });
expect(container.textContent).toContain("Couldn’t prepare the task folders.");
expect(container.textContent).toContain(workspaceFailure.message);
expect(container.querySelector("img, button")).toBeNull();
await render({ ...failed, workspaceFailure });
await act(async () => container.querySelector<HTMLButtonElement>("button")!.click());
expect(retry).toHaveBeenCalledOnce();
await render({ ...failed, workspaceFailure, state: "working" });
expect(container.textContent).toBe("");
});
it("shows why an accepted Retry is waiting and keeps another attempt available", async () => {
const retryWait = { reason: "remote_cleanup", message: "Retry is waiting for the previous environment to stop. Your message is saved." };
await render({ ...failed, retryWait });
expect(container.textContent).toContain(retryWait.message);
expect(container.textContent).not.toContain("Couldn’t finish this attempt.");
await act(async () => container.querySelector<HTMLButtonElement>("button")!.click());
expect(retry).toHaveBeenCalledOnce();
await render({ ...failed, retryWait }, { canRetry: false });
expect(container.textContent).toContain(retryWait.message);
expect(container.querySelector("button")).toBeNull();
});
it("shows an unknown provider's original explanation as text and preserves read-only access", async () => {
const providerFailure = { schema: "paperclip.provider-failure.v1" as const, provider: "new-provider", phase: "startup" as const,
category: "unknown" as const, retryability: "unknown" as const, httpStatus: 429,
message: "Upstream failure <img src=x onerror=alert(1)>" };
await render({ ...failed, providerFailure }, { canRetry: false });
expect(container.textContent).toContain(providerFailure.message);
expect(container.textContent).toContain("Failed before work started");
expect(container.textContent).toContain("HTTP 429");
expect(container.querySelector("img, button")).toBeNull();
await render({ ...failed, providerFailure, state: "resolved" });
expect(container.textContent).toBe("");
});
});