mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-13 07:12:00 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Heartbeat run settlement records why a run failed, and the board shows that record. > - Runs that fail without a named code settle as `adapter_failed` (or `setup_failed`) with no cause. > - A user, an operator, and the restart chaos harness cannot tell what went wrong from such a run. > - Some run errors also show raw SQL statement text to users. > - This pull request records a bounded `failureCause` code for generic failures, shows it on the board, and keeps SQL text out of run errors. > - The benefit is that a generic failure now names its cause, for example "adapter failed (40001)", without leaking messages, SQL, or secrets. ## Linked Issues or Issue Description Refs #20385 (the nested-cause extraction this change reuses). Related: #20421 (restart chaos universal checks U1-U4). **What happened?** The restart chaos audit of 79 saved waves found 30 failed runs with `errorCode: "adapter_failed"` and a null or generic error. Every U3 ("diagnosable") warning was `adapter_failed`. Nobody could tell what failed. Some board run errors also showed raw SQL statement text from wrapped database errors (Drizzle `Failed query: ... params: ...`). **Expected behavior** A run that fails with a generic code names a short, safe cause. The board shows the cause next to the stop reason. Run error text never shows SQL statements or their parameters. **Steps to reproduce** Make an adapter throw a wrapped database error (an `Error` whose nested `cause` has `code: "40001"` and whose message has a Drizzle `Failed query:` statement). Before this change, the run settles as `adapter_failed` with no cause, and `heartbeat_runs.error` contains the SQL statement and parameters. The new tests in `server/src/services/heartbeat/run-completion.test.ts` reproduce this. ## What Changed - Runner: move the #20385 event-commit cause extraction into `safeErrorCauseCode` (`packages/paperclip-runner/src/control-plane/safe-error-cause.ts`) and export it. The Runner's own diagnostics use the same function, unchanged in behavior. - Shared: add the `failureCause` contract (`packages/shared/src/run-failure-cause.ts`): one `[A-Za-z0-9_.:-]` token of at most 80 characters, `normalizeRunFailureCause`, `readRunFailureCause`, and `withoutSqlStatementText`. - Server: add `server/src/services/heartbeat/run-failure-cause.ts`. Settlement records `resultJson.failureCause` only when the code is generic (`adapter_failed`, `setup_failed`). `errorCode` stays the same. - Thrown errors (`failRun`, `failRunSetup`): `provider_http_<status>`, else `provider_<category>`, else the first safe code in the error and up to three nested causes (`40001`, `ECONNRESET`), else the error class name, else `unknown_error`. - Adapter results without a code (`completeRun`): the error family, `signal_<name>`, `process_cancellation_failed`, `exit_code_<n>`, `adapter_reported_error`, or `unknown_error`. - The failure log lines (`heartbeat execution failed`, `heartbeat execution setup failed`, and a new `heartbeat run failed` warning for generic adapter results) include `failureCause` and `errorCode`. - Server: replace SQL statement text in run error text with "Database query failed (<cause>)" at all three settlement paths, including the cancelled branch. The execution blocker's `runError`, the attention inbox (failed-run excerpt and metadata), and continuation summaries apply the same replacement on read, so older rows also stop showing SQL there. - Server: Retry admission and failed-preparation replay accept only a closed set of `resultJson` keys. `failureCause` is added to each of those lists (`task-retry.ts`, `chat-channels/failed-run-retry.ts`, `cancelled-native-startup.ts`, `fresh-workspace-continuation.ts`). It is a diagnostic code with no execution or replay authority, so a failed-preparation run stays retryable. - Server: `runsForIssue` returns a validated top-level `failureCause`. It is null for older rows, non-string JSON values, and malformed values. - UI: `RunForIssue.failureCause`; `IssueRunLedger` shows "adapter failed (40001)". Older runs keep "adapter failed". The Storybook run-ledger fixture has a failed run with a cause. - Restart chaos harness: run samples keep `failureCause`. U3 treats a generic code with a specific `failureCause` as diagnosable. A bare `Error`, `unknown_error`, or the no-code fallback `adapter_reported_error` still counts as vague. U3 stays a warning. - Docs: `doc/run-log-events.md` and the harness README describe the field and the SQL replacement. ## Verification - Red before the fix: the four new `run-completion.test.ts` cases failed against `origin/master`'s `run-completion.ts` (no `failureCause`, SQL text in `error`). - Green: - `server/src/services/heartbeat/run-completion.test.ts`: all tests passed (real embedded PostgreSQL). New cases cover a nested `40001` three levels deep for execution and setup failures, SQL and a secret parameter absent from `error` and `resultJson`, the bound (an 81-character code and a SQL "code" fall back to the class name), `exit_code_137` for a code-less adapter result, a cancelled result with SQL text, no cause for a named code, and the log line. On this loaded host the suite's `afterAll` database teardown hit its 30 s hook limit after every test passed (the known local teardown limit, also seen in #20385). CI runs the suite. - `server/src/services/heartbeat/run-failure-cause.test.ts` (4), `server/src/__tests__/activity-service.test.ts` (recorded, legacy, malformed, overlong, boolean, and number cause projections), `server/src/__tests__/issue-continuation-summary.test.ts` (legacy SQL row). - `packages/shared/src/run-failure-cause.test.ts` (6), `ui/src/components/IssueRunLedger.test.tsx` (21), restart chaos `universal.test.ts` and `observer.test.ts`. - `packages/paperclip-runner/src/control-plane/durable-prp-control-plane.test.ts`: 207 passed (the stall-diagnostic cases still read `40001` and `ECONNRESET`). - Typecheck: shared, runner, server (all layers), UI, restart chaos harness. `pnpm check:token-gates`: all gates clean. - `pnpm verify` returned exit 2 (global doc input changed); required CI supplies broad coverage. - CI on `6295f8d` found that the new key made failed-preparation runs ineligible for Retry (`chat-channels.integration.test.ts` saved board-message Retry got 409; `workspace-recovery-preparation-dispatch.test.ts` dispatched 0 Retries). Both reproduced locally and pass after the allowlist fix, with the other 459 Retry cases in `chat-channels.integration.test.ts` and all 25 in `workspace-recovery-preparation-dispatch.test.ts`. - No chaos waves were run. Storybook was not opened in a browser; the fixture typechecks and the label is covered by the UI test. ## Risks - Low risk. `errorCode` values do not change, so retry, recovery, and comparison logic is unaffected. - No schema change and no migration. `failureCause` is a new optional `resultJson` key. Rows saved by the previous release have no key and render as before; a new row read by an old server is ignored. - The SQL replacement only acts on a Drizzle `Failed query:` marker, or on `$n` placeholder text with a statement shape. Ordinary error text does not change. When it acts, it also drops the text after the statement. - Code that checks `resultJson` against a closed key list must allow `failureCause`. I found and updated the four lists on the Retry path; a list I missed would refuse a Retry for a run with a generic failure. - This is the run log / instance database path. It adds no Telemetry event. - The server failure log line still logs the raw error message (unchanged behavior); only the board-facing text is changed. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through Claude Code with tool use (shell, file editing) and code execution for tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude <noreply@anthropic.com>
682 lines
24 KiB
TypeScript
682 lines
24 KiB
TypeScript
// @vitest-environment jsdom
|
|
|
|
import { act } from "react";
|
|
import type { ComponentProps, ReactNode } from "react";
|
|
import { createRoot, type Root } from "react-dom/client";
|
|
import type { ActivityEvent, Issue, RunLivenessState } from "@paperclipai/shared";
|
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
|
import type { RunForIssue } from "../api/activity";
|
|
import type { ActiveRunForIssue } from "../api/heartbeats";
|
|
import { IssueRunLedgerContent } from "./IssueRunLedger";
|
|
|
|
vi.mock("@/lib/router", () => ({
|
|
Link: ({ children, to, ...props }: { children: ReactNode; to: string } & ComponentProps<"a">) => (
|
|
<a href={to} {...props}>{children}</a>
|
|
),
|
|
}));
|
|
|
|
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
|
(globalThis as any).IS_REACT_ACT_ENVIRONMENT = true;
|
|
|
|
let container: HTMLDivElement;
|
|
let root: Root;
|
|
|
|
beforeEach(() => {
|
|
vi.useFakeTimers();
|
|
vi.setSystemTime(new Date("2026-04-18T20:00:00.000Z"));
|
|
container = document.createElement("div");
|
|
document.body.appendChild(container);
|
|
root = createRoot(container);
|
|
});
|
|
|
|
afterEach(() => {
|
|
act(() => root.unmount());
|
|
container.remove();
|
|
vi.useRealTimers();
|
|
});
|
|
|
|
function render(ui: ReactNode) {
|
|
act(() => {
|
|
root.render(ui);
|
|
});
|
|
}
|
|
|
|
function createRun(overrides: Partial<RunForIssue> = {}): RunForIssue {
|
|
return {
|
|
runId: "run-00000000",
|
|
status: "succeeded",
|
|
agentId: "agent-1",
|
|
adapterType: "codex_local",
|
|
startedAt: "2026-04-18T19:58:00.000Z",
|
|
finishedAt: "2026-04-18T19:59:00.000Z",
|
|
createdAt: "2026-04-18T19:58:00.000Z",
|
|
invocationSource: "assignment",
|
|
usageJson: null,
|
|
resultJson: null,
|
|
livenessState: "advanced",
|
|
livenessReason: "Run produced concrete action evidence: 2 activity event(s)",
|
|
continuationAttempt: 0,
|
|
lastUsefulActionAt: "2026-04-18T19:59:00.000Z",
|
|
nextAction: null,
|
|
...overrides,
|
|
};
|
|
}
|
|
|
|
function createActivity(overrides: Partial<ActivityEvent> = {}): ActivityEvent {
|
|
return {
|
|
id: "activity-1",
|
|
companyId: "company-1",
|
|
actorType: "system",
|
|
actorId: "system",
|
|
action: "issue.updated",
|
|
entityType: "issue",
|
|
entityId: "issue-1",
|
|
agentId: null,
|
|
runId: null,
|
|
details: null,
|
|
createdAt: new Date("2026-04-18T19:57:00.000Z"),
|
|
...overrides,
|
|
};
|
|
}
|
|
|
|
function createIssue(overrides: Partial<Issue> = {}): Issue {
|
|
return {
|
|
id: "issue-1",
|
|
companyId: "company-1",
|
|
projectId: null,
|
|
projectWorkspaceId: null,
|
|
goalId: null,
|
|
parentId: null,
|
|
title: "Child issue",
|
|
description: null,
|
|
status: "todo",
|
|
priority: "medium",
|
|
reviewPolicy: null,
|
|
assigneeAgentId: null,
|
|
assigneeUserId: null,
|
|
responsibleUserId: null,
|
|
checkoutRunId: null,
|
|
executionRunId: null,
|
|
executionAgentNameKey: null,
|
|
executionLockedAt: null,
|
|
createdByAgentId: null,
|
|
createdByUserId: null,
|
|
issueNumber: null,
|
|
identifier: "PAP-1",
|
|
requestDepth: 0,
|
|
billingCode: null,
|
|
assigneeAdapterOverrides: null,
|
|
executionWorkspaceId: null,
|
|
executionWorkspacePreference: null,
|
|
executionWorkspaceSettings: null,
|
|
startedAt: null,
|
|
completedAt: null,
|
|
cancelledAt: null,
|
|
hiddenAt: null,
|
|
createdAt: new Date("2026-04-18T19:00:00.000Z"),
|
|
updatedAt: new Date("2026-04-18T19:00:00.000Z"),
|
|
...overrides,
|
|
workMode: overrides.workMode ?? "standard",
|
|
};
|
|
}
|
|
|
|
function createActiveRun(overrides: Partial<ActiveRunForIssue> = {}): ActiveRunForIssue {
|
|
return {
|
|
id: "run-live-1",
|
|
status: "running",
|
|
invocationSource: "assignment",
|
|
triggerDetail: null,
|
|
startedAt: "2026-04-18T19:58:00.000Z",
|
|
finishedAt: null,
|
|
createdAt: "2026-04-18T19:58:00.000Z",
|
|
agentId: "agent-1",
|
|
agentName: "CodexCoder",
|
|
adapterType: "codex_local",
|
|
outputSilence: {
|
|
lastOutputAt: "2026-04-18T19:00:00.000Z",
|
|
lastOutputSeq: 4,
|
|
lastOutputStream: "stdout",
|
|
silenceStartedAt: "2026-04-18T19:30:00.000Z",
|
|
silenceAgeMs: 45 * 60 * 1000,
|
|
level: "critical",
|
|
suspicionThresholdMs: 10 * 60 * 1000,
|
|
criticalThresholdMs: 30 * 60 * 1000,
|
|
snoozedUntil: null,
|
|
evaluationIssueId: "issue-eval-1",
|
|
evaluationIssueIdentifier: "PAP-404",
|
|
evaluationIssueAssigneeAgentId: "agent-owner",
|
|
},
|
|
...overrides,
|
|
};
|
|
}
|
|
|
|
function renderLedger(props: Partial<ComponentProps<typeof IssueRunLedgerContent>> = {}) {
|
|
render(
|
|
<IssueRunLedgerContent
|
|
runs={props.runs ?? []}
|
|
liveRuns={props.liveRuns}
|
|
activeRun={props.activeRun}
|
|
issueStatus={props.issueStatus ?? "in_progress"}
|
|
childIssues={props.childIssues ?? []}
|
|
agentMap={props.agentMap ?? new Map([["agent-1", { name: "CodexCoder" }]])}
|
|
activityEvents={props.activityEvents}
|
|
renderActivityEvent={props.renderActivityEvent}
|
|
resolveUserLabel={props.resolveUserLabel}
|
|
pendingWatchdogDecision={props.pendingWatchdogDecision}
|
|
canRecordWatchdogDecisions={props.canRecordWatchdogDecisions}
|
|
watchdogDecisionError={props.watchdogDecisionError}
|
|
onWatchdogDecision={props.onWatchdogDecision}
|
|
/>,
|
|
);
|
|
}
|
|
|
|
describe("IssueRunLedger", () => {
|
|
it("renders every liveness state with exhausted continuation context", () => {
|
|
const states: RunLivenessState[] = [
|
|
"advanced",
|
|
"plan_only",
|
|
"empty_response",
|
|
"blocked",
|
|
"failed",
|
|
"completed",
|
|
"needs_followup",
|
|
];
|
|
|
|
renderLedger({
|
|
runs: states.map((state, index) =>
|
|
createRun({
|
|
runId: `run-${index}0000000`,
|
|
createdAt: `2026-04-18T19:5${index}:00.000Z`,
|
|
livenessState: state,
|
|
livenessReason: state === "needs_followup"
|
|
? "Run produced useful output but no concrete action evidence; continuation attempts exhausted"
|
|
: `state ${state}`,
|
|
continuationAttempt: state === "needs_followup" ? 3 : 0,
|
|
}),
|
|
),
|
|
});
|
|
|
|
expect(container.textContent).toContain("Advanced");
|
|
expect(container.textContent).toContain("Plan only");
|
|
expect(container.textContent).toContain("Empty response");
|
|
expect(container.textContent).toContain("Blocked");
|
|
expect(container.textContent).toContain("Failed");
|
|
expect(container.textContent).toContain("Completed");
|
|
expect(container.textContent).toContain("Needs follow-up");
|
|
expect(container.textContent).toContain("Exhausted");
|
|
expect(container.textContent).toContain("Continuation attempt 3");
|
|
});
|
|
|
|
it("renders historical runs without liveness metadata as unavailable", () => {
|
|
renderLedger({
|
|
runs: [
|
|
createRun({
|
|
livenessState: null,
|
|
livenessReason: null,
|
|
continuationAttempt: undefined,
|
|
lastUsefulActionAt: null,
|
|
nextAction: null,
|
|
resultJson: null,
|
|
}),
|
|
],
|
|
});
|
|
|
|
expect(container.textContent).toContain("No liveness data");
|
|
expect(container.textContent).toContain("Stop Unavailable");
|
|
expect(container.textContent).toContain("Last useful action Unavailable");
|
|
});
|
|
|
|
it("interleaves run rows and activity rows by timestamp", () => {
|
|
renderLedger({
|
|
runs: [
|
|
createRun({
|
|
runId: "run-oldest",
|
|
startedAt: "2026-04-18T19:55:00.000Z",
|
|
createdAt: "2026-04-18T19:55:00.000Z",
|
|
}),
|
|
createRun({
|
|
runId: "run-newest",
|
|
startedAt: "2026-04-18T19:59:00.000Z",
|
|
createdAt: "2026-04-18T19:59:00.000Z",
|
|
}),
|
|
],
|
|
activityEvents: [
|
|
createActivity({
|
|
id: "activity-middle",
|
|
action: "activity-middle",
|
|
createdAt: new Date("2026-04-18T19:57:00.000Z"),
|
|
}),
|
|
],
|
|
renderActivityEvent: (event) => (
|
|
<div data-testid={`activity-${event.id}`}>{event.action}</div>
|
|
),
|
|
});
|
|
|
|
const text = container.textContent ?? "";
|
|
const newestIndex = text.indexOf("run-newe");
|
|
const activityIndex = text.indexOf("activity-middle");
|
|
const oldestIndex = text.indexOf("run-olde");
|
|
|
|
expect(newestIndex).toBeGreaterThanOrEqual(0);
|
|
expect(activityIndex).toBeGreaterThan(newestIndex);
|
|
expect(oldestIndex).toBeGreaterThan(activityIndex);
|
|
});
|
|
|
|
it("shows live runs as pending final checks without missing-data language", () => {
|
|
renderLedger({
|
|
runs: [
|
|
createRun({
|
|
status: "running",
|
|
finishedAt: null,
|
|
livenessState: null,
|
|
livenessReason: null,
|
|
continuationAttempt: 0,
|
|
lastUsefulActionAt: null,
|
|
nextAction: null,
|
|
resultJson: null,
|
|
}),
|
|
],
|
|
});
|
|
|
|
expect(container.textContent).toContain("Running now by CodexCoder");
|
|
expect(container.textContent).toContain("Checks after finish");
|
|
expect(container.textContent).toContain("Last useful action No action recorded yet");
|
|
expect(container.textContent).toContain("Stop Still running");
|
|
expect(container.textContent).not.toContain("Liveness pending");
|
|
expect(container.textContent).not.toContain("initial attempt");
|
|
});
|
|
|
|
it("surfaces scheduled retry timing and exhaustion state without opening logs", () => {
|
|
renderLedger({
|
|
runs: [
|
|
createRun({
|
|
runId: "run-scheduled",
|
|
status: "scheduled_retry",
|
|
finishedAt: null,
|
|
livenessState: null,
|
|
livenessReason: null,
|
|
retryOfRunId: "run-root",
|
|
scheduledRetryAt: "2026-04-18T20:15:00.000Z",
|
|
scheduledRetryAttempt: 2,
|
|
scheduledRetryReason: "transient_failure",
|
|
}),
|
|
createRun({
|
|
runId: "run-exhausted",
|
|
status: "failed",
|
|
createdAt: "2026-04-18T19:57:00.000Z",
|
|
retryOfRunId: "run-root",
|
|
scheduledRetryAttempt: 4,
|
|
scheduledRetryReason: "transient_failure",
|
|
retryExhaustedReason: "Bounded retry exhausted after 4 scheduled attempts; no further automatic retry will be queued",
|
|
}),
|
|
],
|
|
});
|
|
|
|
expect(container.textContent).toContain("Retry scheduled");
|
|
expect(container.textContent).toContain("Attempt 2");
|
|
expect(container.textContent).toContain("Transient failure");
|
|
expect(container.textContent).toContain("Next retry");
|
|
expect(container.textContent).toContain("Retry exhausted");
|
|
expect(container.textContent).toContain("no further automatic retry will be queued");
|
|
expect(container.textContent).toContain("Manual intervention required");
|
|
});
|
|
|
|
it("labels max-turn stops and continuation retries without confusing them with per-run turns", () => {
|
|
renderLedger({
|
|
runs: [
|
|
createRun({
|
|
runId: "run-scheduled-continuation",
|
|
status: "scheduled_retry",
|
|
finishedAt: null,
|
|
livenessState: null,
|
|
livenessReason: null,
|
|
retryOfRunId: "run-max-turns",
|
|
scheduledRetryAt: "2026-04-18T20:15:00.000Z",
|
|
scheduledRetryAttempt: 1,
|
|
scheduledRetryReason: "max_turns_continuation",
|
|
}),
|
|
createRun({
|
|
runId: "run-max-turns",
|
|
resultJson: { stopReason: "max_turns_exhausted" },
|
|
createdAt: "2026-04-18T19:57:00.000Z",
|
|
}),
|
|
createRun({
|
|
runId: "run-continuation-exhausted",
|
|
status: "failed",
|
|
createdAt: "2026-04-18T19:56:00.000Z",
|
|
retryOfRunId: "run-max-turns",
|
|
scheduledRetryAttempt: 3,
|
|
scheduledRetryReason: "max_turns_continuation",
|
|
retryExhaustedReason: "Bounded retry exhausted after 3 scheduled attempts; no further automatic retry will be queued",
|
|
}),
|
|
],
|
|
});
|
|
|
|
expect(container.textContent).toContain("Continuation scheduled");
|
|
expect(container.textContent).toContain("Max-turn continuation");
|
|
expect(container.textContent).toContain("Next continuation");
|
|
expect(container.textContent).toContain("Stop max turns exhausted");
|
|
expect(container.textContent).toContain("Continuation exhausted");
|
|
});
|
|
|
|
it("shows timeout, cancel, and budget stop reasons without raw logs", () => {
|
|
renderLedger({
|
|
runs: [
|
|
createRun({
|
|
runId: "run-timeout",
|
|
resultJson: { stopReason: "timeout", timeoutFired: true, effectiveTimeoutSec: 30 },
|
|
}),
|
|
createRun({
|
|
runId: "run-cancel",
|
|
resultJson: { stopReason: "cancelled" },
|
|
createdAt: "2026-04-18T19:57:00.000Z",
|
|
}),
|
|
createRun({
|
|
runId: "run-budget",
|
|
resultJson: { stopReason: "budget_paused" },
|
|
createdAt: "2026-04-18T19:56:00.000Z",
|
|
}),
|
|
createRun({
|
|
runId: "run-background-task",
|
|
resultJson: { stopReason: "unmanaged_background_task_stopped" },
|
|
createdAt: "2026-04-18T19:55:30.000Z",
|
|
}),
|
|
createRun({
|
|
runId: "run-paused",
|
|
resultJson: { stopReason: "paused" },
|
|
createdAt: "2026-04-18T19:55:00.000Z",
|
|
}),
|
|
],
|
|
});
|
|
|
|
expect(container.textContent).toContain("timeout (30s timeout)");
|
|
expect(container.textContent).toContain("cancelled");
|
|
expect(container.textContent).toContain("budget paused");
|
|
expect(container.textContent).toContain("unmanaged background task stopped");
|
|
expect(container.textContent).toContain("paused by board");
|
|
});
|
|
|
|
it("names the recorded cause of an adapter failure and keeps the plain label for older runs", () => {
|
|
renderLedger({
|
|
runs: [
|
|
createRun({
|
|
runId: "run-adapter-cause",
|
|
status: "failed",
|
|
errorCode: "adapter_failed",
|
|
failureCause: "40001",
|
|
resultJson: { stopReason: "adapter_failed" },
|
|
}),
|
|
createRun({
|
|
runId: "run-adapter-legacy",
|
|
status: "failed",
|
|
errorCode: "adapter_failed",
|
|
resultJson: { stopReason: "adapter_failed" },
|
|
createdAt: "2026-04-18T19:57:00.000Z",
|
|
}),
|
|
createRun({
|
|
runId: "run-adapter-malformed",
|
|
status: "failed",
|
|
errorCode: "adapter_failed",
|
|
failureCause: "select * from secrets",
|
|
resultJson: { stopReason: "adapter_failed" },
|
|
createdAt: "2026-04-18T19:56:00.000Z",
|
|
}),
|
|
],
|
|
});
|
|
|
|
expect(container.textContent).toContain("adapter failed (40001)");
|
|
expect(container.textContent?.match(/adapter failed(?! \()/g)).toHaveLength(2);
|
|
expect(container.textContent).not.toContain("secrets");
|
|
});
|
|
|
|
it("surfaces active and completed child issue summaries", () => {
|
|
renderLedger({
|
|
childIssues: [
|
|
createIssue({ id: "child-1", identifier: "PAP-2", title: "Implement worker handoff", status: "in_progress" }),
|
|
createIssue({ id: "child-2", identifier: "PAP-3", title: "Verify final report", status: "done" }),
|
|
createIssue({ id: "child-3", identifier: "PAP-4", title: "Cancelled experiment", status: "cancelled" }),
|
|
],
|
|
});
|
|
|
|
expect(container.textContent).toContain("Child work");
|
|
expect(container.textContent).toContain("1 active, 1 done, 1 cancelled");
|
|
expect(container.textContent).toContain("PAP-2");
|
|
expect(container.textContent).toContain("Implement worker handoff");
|
|
|
|
renderLedger({
|
|
childIssues: [
|
|
createIssue({ id: "child-2", identifier: "PAP-3", title: "Verify final report", status: "done" }),
|
|
createIssue({ id: "child-3", identifier: "PAP-4", title: "Cancelled experiment", status: "cancelled" }),
|
|
],
|
|
});
|
|
|
|
expect(container.textContent).toContain("all 2 terminal (1 done, 1 cancelled)");
|
|
});
|
|
|
|
it("shows the current execution instruction and clears stale instructions after a recorded successor", () => {
|
|
const execution = {
|
|
phase: "recovery_needed" as const, label: "Stopped", cause: "native_continuation_requires_reconciliation",
|
|
lastConfirmedActivityAt: null, retryAt: null, attempt: 1, maxAttempts: 3,
|
|
recoveryOwner: null, nextAction: "Saved work recovered. Send a message to continue.",
|
|
permittedActions: ["inspect_run"] as const, predecessorRunId: null, successorRunId: null,
|
|
};
|
|
const stopped = createRun({ status: "failed", nextAction: "Repair the retained sandbox.",
|
|
execution: { ...execution, permittedActions: ["inspect_run"] } });
|
|
renderLedger({ runs: [stopped] });
|
|
expect(container.textContent).toContain(execution.nextAction);
|
|
expect(container.textContent).not.toContain("Repair the retained sandbox.");
|
|
renderLedger({ runs: [{ ...stopped, execution: { ...stopped.execution!, phase: "completed",
|
|
label: "Continued in another run", nextAction: null, successorRunId: "successor" } }] });
|
|
expect(container.textContent).not.toContain("Next action:");
|
|
});
|
|
|
|
it("uses wrapping-friendly markup for long next action text", () => {
|
|
renderLedger({
|
|
runs: [
|
|
createRun({
|
|
nextAction: "Continue investigating this intentionally-long-next-action-token-that-needs-to-wrap-cleanly-on-mobile-and-desktop-without-overlapping-controls.",
|
|
}),
|
|
],
|
|
});
|
|
|
|
const nextAction = [...container.querySelectorAll("span")]
|
|
.find((node) => node.textContent?.includes("intentionally-long-next-action-token"));
|
|
expect(nextAction?.className).toContain("break-words");
|
|
expect(container.textContent).toContain("Next action:");
|
|
});
|
|
|
|
it("shows when older runs are clipped from the ledger", () => {
|
|
renderLedger({
|
|
runs: Array.from({ length: 22 }, (_, index) =>
|
|
createRun({
|
|
runId: `run-${index.toString().padStart(8, "0")}`,
|
|
createdAt: `2026-04-18T19:${String(index).padStart(2, "0")}:00.000Z`,
|
|
}),
|
|
),
|
|
});
|
|
|
|
expect(container.textContent).toContain("2 older items not shown");
|
|
});
|
|
|
|
it("renders legacy evaluation context with watchdog actions and a silence badge", () => {
|
|
const onWatchdogDecision = vi.fn();
|
|
renderLedger({
|
|
runs: [createRun({ runId: "run-live-1", status: "running", finishedAt: null })],
|
|
activeRun: createActiveRun(),
|
|
onWatchdogDecision,
|
|
});
|
|
|
|
expect(container.textContent).toContain("Critical output silence");
|
|
expect(container.textContent).toContain("PAP-404");
|
|
expect(container.textContent).toContain("Critical silence");
|
|
expect(container.textContent).toContain("Paperclip did not create new delegated recovery work");
|
|
const watchdogBanner = Array.from(container.querySelectorAll("p"))
|
|
.find((node) => node.textContent?.includes("Critical output silence"))
|
|
?.closest("div");
|
|
expect(watchdogBanner?.className).toContain("border-red-500/30");
|
|
expect(watchdogBanner?.className).toContain("bg-red-500/10");
|
|
|
|
const continueButton = Array.from(container.querySelectorAll("button")).find(
|
|
(button) => button.textContent?.includes("Continue monitoring"),
|
|
);
|
|
expect(continueButton).not.toBeUndefined();
|
|
act(() => {
|
|
continueButton?.dispatchEvent(new MouseEvent("click", { bubbles: true }));
|
|
});
|
|
expect(onWatchdogDecision).toHaveBeenCalledWith({
|
|
runId: "run-live-1",
|
|
decision: "continue",
|
|
evaluationIssueId: "issue-eval-1",
|
|
});
|
|
});
|
|
|
|
it.each([
|
|
{
|
|
level: "suspicious" as const,
|
|
heading: "Output silence watchdog warning",
|
|
badge: "Output silence",
|
|
},
|
|
{
|
|
level: "critical" as const,
|
|
heading: "Critical output silence",
|
|
badge: "Critical silence",
|
|
},
|
|
])("renders a $level UI-only signal without an evaluation-task link", ({ level, heading, badge }) => {
|
|
const onWatchdogDecision = vi.fn();
|
|
const activeRun = createActiveRun();
|
|
renderLedger({
|
|
runs: [createRun({ runId: activeRun.id, status: "running", finishedAt: null })],
|
|
activeRun: createActiveRun({
|
|
outputSilence: {
|
|
...activeRun.outputSilence!,
|
|
level,
|
|
evaluationIssueId: null,
|
|
evaluationIssueIdentifier: null,
|
|
evaluationIssueAssigneeAgentId: null,
|
|
},
|
|
}),
|
|
onWatchdogDecision,
|
|
});
|
|
|
|
expect(container.textContent).toContain(heading);
|
|
expect(container.textContent).toContain(badge);
|
|
expect(container.textContent).toContain("Paperclip did not create or assign a recovery task");
|
|
expect(container.textContent).not.toContain("PAP-404");
|
|
expect(container.querySelector('a[href^="/issues/"]')).toBeNull();
|
|
expect(container.textContent).toContain("Continue monitoring");
|
|
expect(container.textContent).toContain("Snooze 1h");
|
|
expect(container.textContent).toContain("Mark false positive");
|
|
|
|
const continueButton = Array.from(container.querySelectorAll("button")).find(
|
|
(button) => button.textContent?.includes("Continue monitoring"),
|
|
);
|
|
act(() => {
|
|
continueButton?.dispatchEvent(new MouseEvent("click", { bubbles: true }));
|
|
});
|
|
expect(onWatchdogDecision).toHaveBeenCalledWith({
|
|
runId: "run-live-1",
|
|
decision: "continue",
|
|
evaluationIssueId: null,
|
|
});
|
|
});
|
|
|
|
it("hides watchdog decision actions for known non-owner viewers", () => {
|
|
const onWatchdogDecision = vi.fn();
|
|
renderLedger({
|
|
runs: [createRun({ runId: "run-live-1", status: "running", finishedAt: null })],
|
|
activeRun: createActiveRun(),
|
|
canRecordWatchdogDecisions: false,
|
|
onWatchdogDecision,
|
|
});
|
|
|
|
expect(container.textContent).toContain("Critical output silence");
|
|
expect(container.textContent).toContain("PAP-404");
|
|
expect(container.textContent).not.toContain("Continue monitoring");
|
|
expect(container.textContent).not.toContain("Snooze 1h");
|
|
expect(container.textContent).not.toContain("Mark false positive");
|
|
expect(
|
|
Array.from(container.querySelectorAll("button")).filter((button) =>
|
|
/continue|snooze|false positive/i.test(button.textContent ?? ""),
|
|
),
|
|
).toHaveLength(0);
|
|
expect(onWatchdogDecision).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("surfaces the responsible user a run acts on behalf of", () => {
|
|
renderLedger({
|
|
runs: [createRun({ runId: "run-obo-1", responsibleUserId: "user-2" })],
|
|
resolveUserLabel: (userId) => (userId === "user-2" ? "Ada Lovelace" : null),
|
|
});
|
|
|
|
const chip = container.querySelector('[data-testid="run-on-behalf-of"]');
|
|
expect(chip).not.toBeNull();
|
|
expect(chip?.textContent).toContain("on behalf of");
|
|
expect(chip?.textContent).toContain("Ada Lovelace");
|
|
});
|
|
|
|
it("omits the on-behalf-of chip when the run has no responsible user", () => {
|
|
renderLedger({
|
|
runs: [createRun({ runId: "run-obo-2", responsibleUserId: null })],
|
|
resolveUserLabel: () => "Ada Lovelace",
|
|
});
|
|
|
|
expect(container.querySelector('[data-testid="run-on-behalf-of"]')).toBeNull();
|
|
});
|
|
|
|
it("renders actionable copy for a responsible-user-unauthorized run failure", () => {
|
|
renderLedger({
|
|
runs: [
|
|
createRun({
|
|
runId: "run-denied-1",
|
|
status: "failed",
|
|
livenessState: "failed",
|
|
responsibleUserId: "user-2",
|
|
errorCode: "RESPONSIBLE_USER_UNAUTHORIZED",
|
|
}),
|
|
],
|
|
resolveUserLabel: () => "Ada Lovelace",
|
|
});
|
|
|
|
const notice = container.querySelector('[data-testid="responsible-user-denial-notice"]');
|
|
expect(notice).not.toBeNull();
|
|
expect(notice?.getAttribute("data-denial-tone")).toBe("unauthorized");
|
|
expect(notice?.textContent).toContain("Ada Lovelace");
|
|
expect(notice?.textContent).toContain("Responsible user not authorized");
|
|
});
|
|
|
|
it("steers the responsible-user-unavailable failure toward marking work blocked", () => {
|
|
renderLedger({
|
|
runs: [
|
|
createRun({
|
|
runId: "run-denied-2",
|
|
status: "failed",
|
|
livenessState: "failed",
|
|
responsibleUserId: "user-3",
|
|
errorCode: "RESPONSIBLE_USER_UNAVAILABLE",
|
|
}),
|
|
],
|
|
resolveUserLabel: () => "Grace Hopper",
|
|
});
|
|
|
|
const notice = container.querySelector('[data-testid="responsible-user-denial-notice"]');
|
|
expect(notice).not.toBeNull();
|
|
expect(notice?.getAttribute("data-denial-tone")).toBe("unavailable");
|
|
expect(notice?.textContent?.toLowerCase()).toContain("blocked");
|
|
});
|
|
|
|
it("does not render a denial notice for a generic agent failure", () => {
|
|
renderLedger({
|
|
runs: [
|
|
createRun({
|
|
runId: "run-denied-3",
|
|
status: "failed",
|
|
livenessState: "failed",
|
|
errorCode: "budget_blocked",
|
|
}),
|
|
],
|
|
});
|
|
|
|
expect(container.querySelector('[data-testid="responsible-user-denial-notice"]')).toBeNull();
|
|
});
|
|
});
|