Files
PaperClipAI/server/src/services/task-interruption-presentation.test.ts
T
DottaandClaude Opus 5.5 4938a930d6 fix(budgets): stop pending or restart-lost accounting from hard-stopping companies (#19497)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Budget admission and run accounting decide whether any task in a
company may start
> - A control-plane restart that kills a provider turn mid-flight can
lose that turn's final usage receipt. ACPX Claude reports usage only at
turn end, so the receipt never becomes complete
> - Accounting kept such a run "awaiting receipt" forever. Every later
admission in the company then refused with "completed runs await
accounting", and the budget evaluator opened a hard-stop incident that
paused the company. No product action could clear it: raising the budget
refuses while runs await accounting, accounting retry finds no new data,
and allowing unpriced usage does not apply to pending runs
> - Restart chaos waves showed this as a dead task Retry, a new task
blocked as "over budget" at 22 of 500 cents, and a hard stop that
cancelled queued Retries and an unrelated running task
> - This pull request closes the debt from evidence: once native
recovery reaches terminal failure and the provider stop is verified, the
known part settles as an unpriced charge. Until then the run holds new
work as recovering accounting, without a hard stop
> - The benefit is that a restart no longer bricks a company with a hard
budget. The operator's unpriced-usage policy decides admission, and
unknown spend still never shows as zero cost

## Linked Issues or Issue Description

Refs #19399, #19426, #19431 (earlier restart Retry fixes; this is the
budget layer under the same journey).

**What happened?**
After a control-plane SIGKILL during an ACPX Claude turn, the killed run
ended `native_provider_terminal_failed` with
`usage_json.accountingReceiptReady = false` and `cost_accounting_pending
= true`. It stayed that way for good. The task Retry was accepted but
parked with `budget.blocked` ("Company cannot start work while completed
runs await accounting"), which the UI showed as "Retry is waiting for
execution checks". A new Codex task was escalated to `blocked` with "the
original recovery target is over budget" while the company had spent 22
of 500 cents. After the next restart the budget evaluator opened a
hard-stop incident (`amount_observed` 22.08 < 500), paused the company
and cancelled both queued Retries.

**What did you expect?**
A restart should not stop the company. Unknown usage from a stopped
provider should become an honest, operator-governed state, and Retry
should start a run.

**Steps to reproduce**
1. Give a company a hard-stop budget.
2. Start an ACPX Claude task and SIGKILL the control plane mid-turn, so
the resumed turn fails.
3. Observe the run stay `cost_accounting_pending` with an incomplete
receipt, and every new admission refuse.

## What Changed

- `budgets.ts`: completed runs that await accounting now hold new work
only. They no longer open a hard-stop incident, pause the company, place
an agent budget hold, or deliver Stop to running work. Only spend at the
limit, or unpriced usage under a blocking policy, does that. Chaos wave
A and a pre-restart lifecycle cell showed the wedge: a run that had just
succeeded stayed terminal for about 100 ms before its charge committed,
the evaluator paused the company in that gap, and the Stop it delivered
left a cancelled run whose receipt never closed.
- `run-cost-accounting.ts`: a failed native run is a closed receipt
source only with positive stop evidence: its finalization is
`terminal_failure` with no result (not a finalization-deadline failure),
it no longer records a process identity, and its latest server-authored
process event is `native.local_process_stopped`. One SQL predicate
serves the reconciler scan and settlement. Accounting now settles its
known part as an `unpriced` charge instead of waiting forever. The
background reconciler selects these runs. The reservation settles. Other
incomplete receipts stay pending as before. A cancelled run keeps its
existing accounting drain, so Stop recovery can still deliver its final
receipt.
- `native-local-process-stop.ts` and its transactional callers:
releasing a verified local or remote process stop wakes the accounting
worker, so the closed debt settles promptly.
- `usage-receipts.ts`: a charge settled without its receipt is marked
`accountingReceiptUnavailable`. A later complete receipt that carries
its own price (including a rate-card price) corrects it. A receipt with
no own price and unchanged usage does not reopen the hold.
- `budget-exposure.ts`: the blocking unpriced-usage policy no longer
counts a charge settled from a lost receipt with no observed usage. That
is unknown, not "usage with no reliable price". Complete receipts that
report no tokens (for example Cursor Cloud) still count. A live wave
under the default policy hard-stopped its company at 0 of 500 cents on
one such charge.
- `budget-exposure.ts`: a terminal-failure native run with an
incomplete, not capture-failed receipt counts as recovering accounting.
Admission holds new work ("New work must wait while a native run
recovers its unfinished accounting") without a hard-stop incident, a
company pause or cancellation of running work.
- `recovery/service.ts`: the stranded-work comment names the actual
budget gate instead of always saying "over budget".
- `task-interruption-presentation.ts`: a Retry parked by
`budget.blocked` says it waits for the budget, not for "execution
checks".
- `doc/SPEC-implementation.md`: documents the closed-receipt contract
next to the existing incomplete-receipt rule.

## Verification

- New embedded-Postgres suite
`server/src/__tests__/closed-native-receipt-accounting.test.ts`, built
from the persisted shape of the killed run:
- Red before the fix: the run stayed pending, `reconcileRunCosts`
accounted 0, and admission refused with "completed runs await
accounting".
- Debt stays pending with no verified stop event, with a
finalization-deadline failure, with a recorded process identity, and
while the coordinator can still produce a result.
- Under a blocking policy, a lost receipt with no usage does not
hard-stop; one with tokens still blocks. A run with a still-recorded
process identity paused the company with a hard-stop incident.
- Green after: the stopped run settles as one `unpriced` charge with the
reservation settled; admission is clear under an `allow` policy and
names "no reliable price" under `block`; a run with a recorded process
identity or a non-terminal coordinator stays pending; the closing window
holds admission without an incident or pause; a final receipt that
arrives later still enters late-receipt reconciliation.
- Pending accounting from a run that just finished holds admission with
no incident, no pause, no enforcement version bump, and an unrelated
running run stays running. Red before the `budgets.ts` change: the
company was paused.
- A later priced receipt corrects the unavailable charge to its price;
an unpriced one leaves it settled.
- `task-interruption-presentation.test.ts`: the `budget.blocked` Retry
wait message.
- `budget-enforcement-scoping.test.ts`: the accounting-hold release
cases now assert that pending accounting places no budget hold and that
admission clears after settlement.
- Existing suites pass locally: all budget suites (control publication,
enforcement scoping, enforcement worker, evaluation work, live attempts,
history, service, live usage, budget cancellation),
native-stop-accounting, explicit-native-continuation,
native-case-handoff, cost-data-quality, heartbeat-cost-accounting,
run-cost-accounting, native-usage-delivery. Server typecheck passes.
- Runner e2e cell
`lifecycle-baseline.runner-codex.local.lifecycle-dependency-restart-neutral`,
which failed on master with a false agent budget hold before any
restart, passes on this branch.
- Live restart-chaos wave (local runner, K1 SIGKILL, codex + 2× ACPX
Claude + OpenCode, 4 restarts, company policy `unpricedUsagePolicy:
allow`) with the accounting fix: every killed run settled (no
`cost_accounting_pending` left), the company stayed active, the OpenCode
Retry completed the task on the first click and the Codex Retry finished
its task. That wave also exposed a 1-second false hard stop in the
closing window (observed 0 of 500 cents) that cancelled the running
OpenCode turn; the `budget-exposure.ts` change fixes it and has its own
red/green test.
- Confirming live wave with both commits, same plan (2× ACPX Claude,
OpenCode, Codex; 4 K1 restarts): both Claude tasks passed R3 (one Retry
click started a run that completed the task), OpenCode passed R1
(continued by itself), and the Codex Retry completed its task. Zero
budget incidents, zero cancelled runs, zero runs left pending
accounting, company active; three killed runs recorded as `unpriced`
charges. The only open signature is an unrelated workspace-durability
check (`R2:pre_fault_files_lost` on the Codex long-bench task).

## Risks

- Behavior change: a stopped native run's lost receipt now becomes an
`unpriced` ledger charge with its known usage. If that usage has tokens,
the default `block` policy still stops new work with "recorded usage has
no reliable price". If it has none, the charge is visible in Costs but
does not stop the company. Other unpriced charges keep their existing
effect. Unknown spend is never recorded as a priced receipt. Budget
enforcement stays best effort, as the spec says; killed OpenCode and
Codex requests already lose their in-flight usage.
- Remote runs and runs without a verified local stop keep their debt
pending (an admission hold only, no hard stop).
- Contract change: pending accounting no longer pauses a scope or stops
running work. New work still waits with "completed runs await
accounting". A scope's real hard stop (spend at the limit, or blocked
unpriced usage) is unchanged.
- If a provider stop can never be verified, the run stays "recovering
accounting": new work waits without a hard-stop incident. Before, the
same state paused the company. The hold is visible in admission errors
and Costs.
- A receipt that arrives after settlement posts a late correction. The
original unpriced charge stays in the ledger.
- No schema or migration change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file editing, embedded Postgres tests, live
restart-chaos harness).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 14:27:51 -06:00

316 lines
21 KiB
TypeScript

import { describe, expect, it } from "vitest";
import { projectTaskInterruption } from "./task-interruption-presentation.js";
const issue = { id: "task", companyId: "11111111-1111-4111-8111-111111111111", status: "blocked", assigneeAgentId: "agent", projectId: "22222222-2222-4222-8222-222222222222" };
const run = { id: "run", agentId: "agent", status: "failed", scheduledRetryAt: null,
errorCode: "project_access_denied", error: null, livenessState: null,
resultJson: { projectAccessDenied: { projectId: "22222222-2222-4222-8222-222222222222" } } };
const agent = { status: "idle", pauseReason: null };
describe("queued dependency wait presentation", () => {
const waitingRun = { ...run, status: "cancelled", runtimeMode: "legacy" as const, startedAt: null,
errorCode: "issue_dependencies_blocked", resultJson: {
stopReason: "issue_dependencies_blocked", timeoutSource: "dependency_gate",
cancellation: { source: "unknown", expected: false },
} };
it("keeps a dependency-gated wake out of the failed-attempt UI, including retained input", () => {
for (const wait of [undefined, { id: "saved-input", requestedAt: new Date(0) }]) {
expect(projectTaskInterruption(issue, { run: waitingRun, agent, wait, paused: false, approval: false }))
.toMatchObject({ state: "resolved", canRetry: false });
}
});
it.each(["legacy", "native"])("keeps an unstarted %s dispatcher dependency wait neutral", runtimeMode => {
const dispatchedWait = { ...waitingRun, runtimeMode,
resultJson: { ...waitingRun.resultJson, timeoutSource: "stale_queued_run_gate" } };
expect(projectTaskInterruption(issue, { run: dispatchedWait, agent, paused: false, approval: false }))
.toMatchObject({ state: "resolved", canRetry: false });
expect(projectTaskInterruption(issue, { run: { ...dispatchedWait, startedAt: new Date() }, agent, paused: false, approval: false }))
.toMatchObject({ state: "failed", canRetry: true });
});
it("keeps genuine failures and current user gates visible", () => {
for (const changed of [{ status: "failed" }, { runtimeMode: "native" }, { startedAt: new Date() },
{ errorCode: "setup_failed" }, { resultJson: {} }, { resultJson: { ...waitingRun.resultJson, timeoutFired: true } }]) {
expect(projectTaskInterruption(issue, { run: { ...waitingRun, ...changed }, agent, paused: false, approval: false }))
.toMatchObject({ state: "failed", canRetry: true });
}
expect(projectTaskInterruption(issue, { run: waitingRun, agent, paused: true, approval: false }))
.toMatchObject({ state: "waiting_for_user", userAction: "resume" });
expect(projectTaskInterruption(issue, { run: waitingRun, agent, paused: false, approval: true }))
.toMatchObject({ state: "waiting_for_user", userAction: "approval" });
expect(projectTaskInterruption(issue, { run: waitingRun, agent, paused: false, approval: false,
action: { evidence: {}, status: "active" } as never })).toMatchObject({ state: "failed", canRetry: true });
expect(projectTaskInterruption(issue, { run: waitingRun, agent, paused: false, approval: false,
retryWait: { id: "retry", sourceRunId: waitingRun.id, requestedAt: new Date(), preparationStatus: "failed" } }))
.toMatchObject({ state: "failed", canRetry: true });
});
});
describe("unadmitted chat Retry presentation", () => {
const now = Date.parse("2026-01-01T12:00:00Z");
const failed = { ...run, errorCode: "native_provider_terminal_failed", resultJson: null };
const wait = { id: "retry", requestedAt: new Date(now), sourceRunId: run.id,
preparationStatus: "issued", preparationRetryAt: now + 250 };
const input = { run: failed, agent, wait, paused: false, approval: false };
it.each(["issued", "processing"])("uses the persisted %s outbox deadline without implying provider work", status => {
for (const offset of [0, 20_000]) expect(projectTaskInterruption(issue, {
...input, wait: { ...wait, preparationStatus: status },
}, now + offset)).toMatchObject({ state: "retry_scheduled", canRetry: false,
nextRetryAt: new Date(wait.preparationRetryAt).toISOString() });
});
it.each(["failed", "cancelled"])("keeps terminal %s preparation out of the working state", status => {
expect(projectTaskInterruption(issue, { ...input, wait: { ...wait, preparationStatus: status,
preparationRetryAt: null } }, now)).toMatchObject({ state: "failed", nextRetryAt: null });
});
it("keeps current gates and admitted execution authoritative", () => {
expect(projectTaskInterruption(issue, { ...input, paused: true }, now))
.toMatchObject({ state: "waiting_for_user", userAction: "resume" });
expect(projectTaskInterruption(issue, { ...input, approval: true }, now))
.toMatchObject({ state: "waiting_for_user", userAction: "approval" });
for (const status of ["queued", "running"]) expect(projectTaskInterruption(issue, {
...input, run: { ...failed, id: "successor", status },
}, now)).toMatchObject({ state: "working", nextRetryAt: null });
expect(projectTaskInterruption(issue, { ...input, wait: { ...wait, sourceRunId: "old-run" } }, now))
.toMatchObject({ state: "working", nextRetryAt: null });
});
});
// The native coordinator resumes the same run itself after an attach failure.
// A manual Retry offered meanwhile would race that automatic resume.
it("shows a pending native same-run resume instead of a competing Retry", () => {
const now = Date.parse("2026-11-06T15:25:50.000Z");
const failed = { ...run, runtimeMode: "native" as const, errorCode: "native_session_interrupted", resultJson: null };
const resumeAt = new Date("2026-11-06T15:26:18.687Z");
expect(projectTaskInterruption(issue, { run: { ...failed, nativeResumeAt: resumeAt }, agent, paused: false, approval: false }, now))
.toMatchObject({ state: "retry_scheduled", canRetry: false, nextRetryAt: resumeAt.toISOString() });
// A due resume is being dispatched; it is still not the user's turn.
expect(projectTaskInterruption(issue, { run: { ...failed, nativeResumeAt: new Date(now - 30_000) }, agent, paused: false, approval: false }, now))
.toMatchObject({ state: "working", canRetry: false });
// A resume that never ran, or no resume at all, leaves exactly one Retry.
expect(projectTaskInterruption(issue, { run: { ...failed, nativeResumeAt: new Date(now - 10 * 60_000) }, agent, paused: false, approval: false }, now))
.toMatchObject({ state: "failed", canRetry: true });
expect(projectTaskInterruption(issue, { run: failed, agent, paused: false, approval: false }, now))
.toMatchObject({ state: "failed", canRetry: true });
});
it("explains a persisted Retry wait without exposing the saved payload", () => {
const failed = { ...run, errorCode: "native_provider_terminal_failed", resultJson: null };
const wait = { id: "retry", requestedAt: new Date(), sourceRunId: failed.id,
executionWaitReason: "workspace_repair_required" };
const presentation = projectTaskInterruption(issue, { run: failed, agent, wait, paused: false, approval: false });
expect(presentation).toMatchObject({ state: "failed", canRetry: true,
retryWait: { reason: "workspace_repair_required", message: "Retry is waiting for Paperclip to recover the saved files. Your message is saved." } });
expect(projectTaskInterruption(issue, { run: failed, agent, wait: { ...wait, sourceRunId: "another-run" }, paused: false, approval: false }))
.not.toHaveProperty("retryWait");
expect(projectTaskInterruption(issue, { run: failed, agent, wait: { ...wait, executionWaitReason: "secret-payload" }, paused: false, approval: false }))
.toMatchObject({ retryWait: { reason: "execution_recovery", message: "Retry is waiting for execution checks. Your message is saved." } });
expect(projectTaskInterruption(issue, { run: failed, agent, wait, paused: true, approval: false }))
.toMatchObject({ state: "waiting_for_user", userAction: "resume" });
});
it("names a budget gate instead of generic execution checks for a refused Retry", () => {
const failed = { ...run, errorCode: "native_provider_terminal_failed", resultJson: null };
expect(projectTaskInterruption(issue, { run: failed, agent, paused: false, approval: false,
wait: { id: "retry", requestedAt: new Date(), sourceRunId: failed.id, executionWaitReason: "budget.blocked" } }))
.toMatchObject({ state: "failed", canRetry: true, retryWait: { reason: "budget.blocked",
message: "Retry is waiting for the budget to allow new work. Check Costs for unrecorded or unpriced usage." } });
});
it.each(["cleanup_quarantined", "cancelled_by_operator", "retry_source_changed"])("does not offer Retry for terminal Retry waits: %s", reason => {
const failed = { ...run, errorCode: "native_provider_terminal_failed", resultJson: null };
expect(projectTaskInterruption(issue, { run: failed, agent, paused: false, approval: false,
wait: { id: "retry", requestedAt: new Date(), sourceRunId: failed.id, executionWaitReason: reason } }))
.toMatchObject({ state: "failed", canRetry: false, retryWait: { reason } });
});
it("keeps Retry for a Retry declined for its requester's access, since a new click replaces it", () => {
const failed = { ...run, errorCode: "native_provider_terminal_failed", resultJson: null };
expect(projectTaskInterruption(issue, { run: failed, agent, paused: false, approval: false,
wait: { id: "retry", requestedAt: new Date(), sourceRunId: failed.id, executionWaitReason: "retry_authority_denied" } }))
.toMatchObject({ state: "failed", canRetry: true, retryWait: { reason: "retry_authority_denied" } });
});
it("preserves the actual workspace preparation error without changing retry authority", () => {
const failedRun = { ...run, errorCode: "workspace_validation_failed",
error: "The selected project has no repository source for a remote workspace.", resultJson: null };
expect(projectTaskInterruption(issue, { run: failedRun, agent, paused: false, approval: false }))
.toMatchObject({ state: "failed", canRetry: true, workspaceFailure: { message: failedRun.error } });
for (const status of ["done", "cancelled"]) expect(projectTaskInterruption({ ...issue, status },
{ run: failedRun, agent, paused: false, approval: false })).not.toHaveProperty("workspaceFailure");
expect(projectTaskInterruption(issue, { run: { ...failedRun, errorCode: "unrelated_failure" }, agent, paused: false, approval: false }))
.not.toHaveProperty("workspaceFailure");
});
function operatorStoppedRun() {
return { ...run, status: "cancelled", runtimeMode: "native" as const, companyId: issue.companyId,
nativeIssueId: issue.id, errorCode: "cancelled", resultJson: {
cancelledByActorType: "user", cancelledByUserId: "board",
executionCancellation: { state: "requested" },
nativeCancellation: { schema: "paperclip.native-cancellation.v1", runId: run.id,
companyId: issue.companyId, issueId: issue.id, scope: "run",
reasonCode: "cancellation_run_only", dispatched: true, dispatchState: "acknowledged",
intentAuditId: "intent", acknowledgementAuditId: "ack" },
} };
}
describe("intentional native Stop presentation", () => {
const workingIssue = { ...issue, status: "in_progress" };
it("does not offer a failed-attempt Retry for an audited operator Stop that keeps its task in progress", () => {
expect(projectTaskInterruption(workingIssue, { run: operatorStoppedRun(), agent, paused: false, approval: false }))
.toMatchObject({ state: "resolved", canRetry: false, userAction: null });
expect(workingIssue.status).toBe("in_progress");
});
it.each([{ runId: "foreign" }, { companyId: "foreign" }, { issueId: "foreign" },
{ dispatchState: "pending" }, { dispatched: false }, { acknowledgementAuditId: undefined },
{ intentAuditId: 123 }, { acknowledgementAuditId: {} },
{ scope: "subtree" }])("retains failure presentation without the matching operator receipt %j", change => {
const stopped = operatorStoppedRun();
expect(projectTaskInterruption(workingIssue, { run: { ...stopped, resultJson: { ...stopped.resultJson,
nativeCancellation: { ...stopped.resultJson.nativeCancellation, ...change } } }, agent, paused: false, approval: false }))
.toMatchObject({ state: "failed", canRetry: true });
});
it("keeps pause, approval, non-user cancellation and new input authoritative", () => {
const stopped = operatorStoppedRun();
expect(projectTaskInterruption(workingIssue, { run: stopped, agent, paused: true, approval: false }))
.toMatchObject({ state: "waiting_for_user", userAction: "resume" });
expect(projectTaskInterruption(workingIssue, { run: stopped, agent, paused: false, approval: true }))
.toMatchObject({ state: "waiting_for_user", userAction: "approval" });
expect(projectTaskInterruption(workingIssue, { run: { ...stopped, resultJson: {
...stopped.resultJson, cancelledByActorType: "system" } }, agent, paused: false, approval: false }))
.toMatchObject({ state: "failed", canRetry: true });
expect(projectTaskInterruption(workingIssue, { run: stopped, agent, paused: false, approval: false,
wait: { id: "new-input", requestedAt: new Date() } }))
.toMatchObject({ state: "working", canRetry: false });
});
});
it("preserves a provider explanation on a failed task without changing retry authority", () => {
const providerFailure = { schema: "paperclip.provider-failure.v1", provider: "anthropic", phase: "startup",
category: "rate_limit", message: "API error: 429 This request would exceed your account's rate limit.",
httpStatus: 429, retryability: "unknown" };
const failedRun = { ...run, errorCode: "native_session_interrupted", resultJson: { providerFailure } };
const presentation = projectTaskInterruption(issue, { run: failedRun, agent, paused: false, approval: false });
expect(presentation).toMatchObject({ state: "failed", canRetry: true, providerFailure });
expect(projectTaskInterruption({ ...issue, status: "done" }, { run: failedRun, agent, paused: false, approval: false }))
.not.toHaveProperty("providerFailure");
});
describe("task project-access interruption", () => {
it("explains a project denial without offering a futile retry", () => {
expect(projectTaskInterruption(issue, { run, agent, paused: false, approval: false })).toMatchObject({
state: "waiting_for_user", responsibility: "user", userAction: "project_access", canRetry: false,
projectAccess: { agentId: "agent", projectId: "22222222-2222-4222-8222-222222222222", ready: false },
});
});
it("projects the historical setup denial without requiring another failed run", () => {
expect(projectTaskInterruption(issue, { run: { ...run, errorCode: "setup_failed",
error: "Project is outside this low-trust boundary.", resultJson: null }, agent, paused: false, approval: false }))
.toMatchObject({ userAction: "project_access", canRetry: false });
});
it("offers an explicit retry after the project is added, without changing the failed run", () => {
expect(projectTaskInterruption(issue, { run, agent, projectAccessAllowed: true, paused: false, approval: false }))
.toMatchObject({ state: "failed", canRetry: true, projectAccess: { ready: true } });
expect(run.status).toBe("failed");
});
it("shows an accepted retry as working after access is repaired", () => {
const input = { run, agent, paused: false, approval: false, wait: { id: "request", requestedAt: new Date() } };
expect(projectTaskInterruption(issue, { ...input, projectAccessAllowed: true }))
.toMatchObject({ state: "working", canRetry: false });
expect(projectTaskInterruption(issue, { ...input, projectAccessAllowed: true })).not.toHaveProperty("projectAccess");
expect(projectTaskInterruption(issue, { ...input, projectAccessAllowed: false }))
.toMatchObject({ state: "waiting_for_user", userAction: "project_access", canRetry: false });
});
it("does not reinterpret unrelated failures or an old assignee's denial", () => {
for (const otherRun of [{ ...run, errorCode: "setup_failed", error: "Disk full", resultJson: null },
{ ...run, agentId: "previous-agent" }]) {
expect(projectTaskInterruption(issue, { run: otherRun, agent, paused: false, approval: false }))
.not.toHaveProperty("projectAccess");
}
});
it("keeps approval, pause, running and completed states authoritative", () => {
expect(projectTaskInterruption(issue, { run, agent, paused: false, approval: true })).toMatchObject({ userAction: "approval" });
expect(projectTaskInterruption(issue, { run, agent, paused: true, approval: false })).toMatchObject({ userAction: "resume" });
expect(projectTaskInterruption(issue, { run: { ...run, status: "running" }, agent, paused: false, approval: false }))
.toMatchObject({ state: "working" });
expect(projectTaskInterruption({ ...issue, status: "done" }, { run, agent, paused: false, approval: false }))
.toMatchObject({ state: "resolved" });
});
});
describe("saved-work recovery step", () => {
const now = Date.parse("2026-01-01T12:00:00Z");
const failed = { ...run, errorCode: "setup_failed", resultJson: null };
const system = (phase: string, state: string, nextAttemptAt: number | null) => ({
schema: "paperclip.system-recovery.v1", policyVersion: 1, sourceRunId: "33333333-3333-4333-8333-333333333333", revision: 3,
generationId: "44444444-4444-4444-8444-444444444444", intentId: null, phase, state, attemptId: null, attempts: 2,
retriesUsed: 1, firstFailureAt: now - 60_000, nextAttemptAt, deadlineAt: now + 600_000, providerCooldownAt: null,
failure: null, preservationOperationIds: [], freshAttemptRunId: null });
const action = (evidence: Record<string, unknown>) => ({ evidence }) as never;
it("names the step a scheduled recovery will retry", () => {
expect(projectTaskInterruption(issue, { run: failed, agent, paused: false, approval: false,
action: action({ systemRecovery: system("verify_execution", "scheduled", now + 30_000) }) }, now))
.toMatchObject({ state: "retry_scheduled", recoveryStep: "checking_previous_run" });
});
it("names the step a running recovery is on", () => {
expect(projectTaskInterruption(issue, { run: failed, agent, paused: false, approval: false,
action: action({ systemRecovery: system("preserve_files", "running", null), systemRecoveryActiveAt: now - 1_000 }) }, now))
.toMatchObject({ state: "working", recoveryStep: "saving_previous_files" });
});
it("names recovery over a pending chat preparation and keeps failed preparation authoritative", () => {
const wait = { id: "retry", requestedAt: new Date(now), sourceRunId: failed.id, preparationStatus: "issued", preparationRetryAt: now + 250 };
const running = action({ systemRecovery: system("preserve_files", "running", null), systemRecoveryActiveAt: now - 1_000 });
expect(projectTaskInterruption(issue, { run: failed, agent, wait, paused: false, approval: false, action: running }, now))
.toMatchObject({ state: "working", recoveryStep: "saving_previous_files" });
const scheduled = action({ systemRecovery: system("verify_execution", "scheduled", now + 30_000) });
expect(projectTaskInterruption(issue, { run: failed, agent, wait, paused: false, approval: false, action: scheduled }, now))
.toMatchObject({ state: "retry_scheduled", recoveryStep: "checking_previous_run", canRetry: false });
expect(projectTaskInterruption(issue, { run: failed, agent, paused: false, approval: false, action: scheduled }, now))
.toMatchObject({ state: "retry_scheduled", canRetry: true });
expect(projectTaskInterruption(issue, { run: failed, agent, wait: { ...wait, preparationStatus: "failed" },
paused: false, approval: false, action: running }, now)).toMatchObject({ state: "failed" });
expect(projectTaskInterruption(issue, { run: failed, agent, wait, paused: false, approval: false }, now))
.toMatchObject({ state: "retry_scheduled", nextRetryAt: new Date(now + 250).toISOString() });
});
it("does not name a step for an ordinary scheduled retry", () => {
expect(projectTaskInterruption(issue, { run: { ...failed, scheduledRetryAt: new Date(now + 30_000) }, agent,
paused: false, approval: false }, now)).not.toHaveProperty("recoveryStep");
});
});
describe("scheduled resource wait presentation", () => {
const now = Date.parse("2026-01-01T12:00:00Z");
const waiting = { ...run, status: "scheduled_retry", errorCode: null, resultJson: null,
scheduledRetryReason: "workspace_busy", scheduledRetryAt: new Date(now - 60_000) };
it("keeps an overdue workspace wait out of the failed-attempt UI", () => {
expect(projectTaskInterruption(issue, { run: waiting, agent, paused: false, approval: false }, now))
.toMatchObject({ state: "working", canRetry: false });
});
it("keeps failure retries, user gates and failed Retry preparation authoritative", () => {
expect(projectTaskInterruption(issue, { run: { ...waiting, scheduledRetryReason: "transient_failure" },
agent, paused: false, approval: false }, now)).toMatchObject({ state: "failed", canRetry: true });
expect(projectTaskInterruption(issue, { run: waiting, agent, paused: true, approval: false }, now))
.toMatchObject({ state: "waiting_for_user", userAction: "resume" });
expect(projectTaskInterruption(issue, { run: waiting, agent, paused: false, approval: false,
retryWait: { id: "retry", sourceRunId: run.id, requestedAt: new Date(now), preparationStatus: "failed" } }, now))
.toMatchObject({ state: "failed", canRetry: true });
});
});