Files
PaperClipAI/ui/src/components/IssueRunLedger.test.tsx
T
DottaandPaperclip f554d67377 fix(server): add explicit review verdict policies (#10931)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Issues use `in_review` to request a final decision from an
authorized writer
> - The server rejected an assignee agent that tried to close its own
review, even when the issue had no independent-review rule
> - This rejection stopped the default agent workflow and did not
represent the configured execution-stage rules
> - Paperclip needs an open default and explicit issue-level constraints
for teams that require an independent or human verdict
> - This pull request removes the unconditional rejection and adds
`anyone`, `not_creator`, and `human_only` review policies
> - The benefit is a working default path with opt-in, authenticated
verdict controls

## Linked Issues or Issue Description

Refs #10635, #4429, and #10671.

The related public work covers execution-stage independence,
self-approval fallback behavior, and durable review paths. This change
is distinct. It controls who can resolve an issue review verdict. It
keeps configured execution stages active.

## What Changed

- Added a nullable `review_policy` issue column. Null has the same
meaning as `anyone`. The migration does not backfill existing issues.
- Added shared create, update, response, and compact issue contracts for
`anyone`, `not_creator`, and `human_only`.
- Removed the unconditional agent self-approval rejection for
`in_review` issues.
- Added one reusable verdict-actor check for terminal status changes and
pending interaction accept or reject actions.
- Used the authenticated principal type for `human_only`. Agent keys and
run tokens remain agent principals.
- Used the latest transition into `in_review` to identify the requester
for `not_creator`.
- Added actionable 403 responses that name the policy, the allowed
actor, and the next step.
- Kept the configured execution-stage transition and signoff behavior.
- Added focused contract, helper, status-route, interaction-route, and
execution-stage regression tests.
- Updated the implementation specification for the new issue field.

## Verification

- `pnpm exec vitest run packages/shared/src/validators/issue.test.ts
server/src/__tests__/issue-review-policy.test.ts
server/src/__tests__/issue-stalled-review-decision-routes.test.ts
--reporter=dot` passed: 42 tests.
- `pnpm --filter @paperclipai/shared typecheck` passed.
- `pnpm --filter @paperclipai/db typecheck` passed, including migration
numbering and safety checks.
- `pnpm --filter @paperclipai/server typecheck` passed.
- `pnpm run typecheck:build-gaps` passed across server, CLI, plugin
SDK/examples, plugin wiki, and UI.
- `git diff --check origin/master...HEAD` passed.
- SecurityEngineer review approved the authenticated-principal checks
and accepted policy-relaxation tradeoff with no required changes.
- Greptile reviewed the latest head at 5/5 with zero inline comments or
follow-ups.
- The latest-head GitHub rollup passed build, typecheck,
server/workspace tests, serialized suites, canary, e2e, and external
security checks.

## Risks

- The migration adds one nullable text column. It has no default and no
backfill.
- `not_creator` reads the latest recorded transition into `in_review`.
It denies the verdict when it cannot identify the requester.
- Agents can change or relax `reviewPolicy` when they have issue write
access. This is intentional for this issue-level control.
- Null and `anyone` do not add a database query to the verdict path.
- Configured execution-stage checks still run after the issue-level
policy check.

> This work aligns with the completed "Agent Reviews and Approvals" and
"Enforced Outcomes" roadmap items. It does not add a new roadmap
capability.

## Model Used

- OpenAI Codex, GPT-5. The exact deployment ID and context-window size
are not exposed to the agent. The run used reasoning, repository tools,
code execution, and GitHub CLI access.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-05 23:12:41 -05:00

612 lines
21 KiB
TypeScript

// @vitest-environment jsdom
import { act } from "react";
import type { ComponentProps, ReactNode } from "react";
import { createRoot, type Root } from "react-dom/client";
import type { ActivityEvent, Issue, RunLivenessState } from "@paperclipai/shared";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import type { RunForIssue } from "../api/activity";
import type { ActiveRunForIssue } from "../api/heartbeats";
import { IssueRunLedgerContent } from "./IssueRunLedger";
vi.mock("@/lib/router", () => ({
Link: ({ children, to, ...props }: { children: ReactNode; to: string } & ComponentProps<"a">) => (
<a href={to} {...props}>{children}</a>
),
}));
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).IS_REACT_ACT_ENVIRONMENT = true;
let container: HTMLDivElement;
let root: Root;
beforeEach(() => {
vi.useFakeTimers();
vi.setSystemTime(new Date("2026-04-18T20:00:00.000Z"));
container = document.createElement("div");
document.body.appendChild(container);
root = createRoot(container);
});
afterEach(() => {
act(() => root.unmount());
container.remove();
vi.useRealTimers();
});
function render(ui: ReactNode) {
act(() => {
root.render(ui);
});
}
function createRun(overrides: Partial<RunForIssue> = {}): RunForIssue {
return {
runId: "run-00000000",
status: "succeeded",
agentId: "agent-1",
adapterType: "codex_local",
startedAt: "2026-04-18T19:58:00.000Z",
finishedAt: "2026-04-18T19:59:00.000Z",
createdAt: "2026-04-18T19:58:00.000Z",
invocationSource: "assignment",
usageJson: null,
resultJson: null,
livenessState: "advanced",
livenessReason: "Run produced concrete action evidence: 2 activity event(s)",
continuationAttempt: 0,
lastUsefulActionAt: "2026-04-18T19:59:00.000Z",
nextAction: null,
...overrides,
};
}
function createActivity(overrides: Partial<ActivityEvent> = {}): ActivityEvent {
return {
id: "activity-1",
companyId: "company-1",
actorType: "system",
actorId: "system",
action: "issue.updated",
entityType: "issue",
entityId: "issue-1",
agentId: null,
runId: null,
details: null,
createdAt: new Date("2026-04-18T19:57:00.000Z"),
...overrides,
};
}
function createIssue(overrides: Partial<Issue> = {}): Issue {
return {
id: "issue-1",
companyId: "company-1",
projectId: null,
projectWorkspaceId: null,
goalId: null,
parentId: null,
title: "Child issue",
description: null,
status: "todo",
priority: "medium",
reviewPolicy: null,
assigneeAgentId: null,
assigneeUserId: null,
responsibleUserId: null,
checkoutRunId: null,
executionRunId: null,
executionAgentNameKey: null,
executionLockedAt: null,
createdByAgentId: null,
createdByUserId: null,
issueNumber: null,
identifier: "PAP-1",
requestDepth: 0,
billingCode: null,
assigneeAdapterOverrides: null,
executionWorkspaceId: null,
executionWorkspacePreference: null,
executionWorkspaceSettings: null,
startedAt: null,
completedAt: null,
cancelledAt: null,
hiddenAt: null,
createdAt: new Date("2026-04-18T19:00:00.000Z"),
updatedAt: new Date("2026-04-18T19:00:00.000Z"),
...overrides,
workMode: overrides.workMode ?? "standard",
};
}
function createActiveRun(overrides: Partial<ActiveRunForIssue> = {}): ActiveRunForIssue {
return {
id: "run-live-1",
status: "running",
invocationSource: "assignment",
triggerDetail: null,
startedAt: "2026-04-18T19:58:00.000Z",
finishedAt: null,
createdAt: "2026-04-18T19:58:00.000Z",
agentId: "agent-1",
agentName: "CodexCoder",
adapterType: "codex_local",
outputSilence: {
lastOutputAt: "2026-04-18T19:00:00.000Z",
lastOutputSeq: 4,
lastOutputStream: "stdout",
silenceStartedAt: "2026-04-18T19:30:00.000Z",
silenceAgeMs: 45 * 60 * 1000,
level: "critical",
suspicionThresholdMs: 10 * 60 * 1000,
criticalThresholdMs: 30 * 60 * 1000,
snoozedUntil: null,
evaluationIssueId: "issue-eval-1",
evaluationIssueIdentifier: "PAP-404",
evaluationIssueAssigneeAgentId: "agent-owner",
},
...overrides,
};
}
function renderLedger(props: Partial<ComponentProps<typeof IssueRunLedgerContent>> = {}) {
render(
<IssueRunLedgerContent
runs={props.runs ?? []}
liveRuns={props.liveRuns}
activeRun={props.activeRun}
issueStatus={props.issueStatus ?? "in_progress"}
childIssues={props.childIssues ?? []}
agentMap={props.agentMap ?? new Map([["agent-1", { name: "CodexCoder" }]])}
activityEvents={props.activityEvents}
renderActivityEvent={props.renderActivityEvent}
resolveUserLabel={props.resolveUserLabel}
pendingWatchdogDecision={props.pendingWatchdogDecision}
canRecordWatchdogDecisions={props.canRecordWatchdogDecisions}
watchdogDecisionError={props.watchdogDecisionError}
onWatchdogDecision={props.onWatchdogDecision}
/>,
);
}
describe("IssueRunLedger", () => {
it("renders every liveness state with exhausted continuation context", () => {
const states: RunLivenessState[] = [
"advanced",
"plan_only",
"empty_response",
"blocked",
"failed",
"completed",
"needs_followup",
];
renderLedger({
runs: states.map((state, index) =>
createRun({
runId: `run-${index}0000000`,
createdAt: `2026-04-18T19:5${index}:00.000Z`,
livenessState: state,
livenessReason: state === "needs_followup"
? "Run produced useful output but no concrete action evidence; continuation attempts exhausted"
: `state ${state}`,
continuationAttempt: state === "needs_followup" ? 3 : 0,
}),
),
});
expect(container.textContent).toContain("Advanced");
expect(container.textContent).toContain("Plan only");
expect(container.textContent).toContain("Empty response");
expect(container.textContent).toContain("Blocked");
expect(container.textContent).toContain("Failed");
expect(container.textContent).toContain("Completed");
expect(container.textContent).toContain("Needs follow-up");
expect(container.textContent).toContain("Exhausted");
expect(container.textContent).toContain("Continuation attempt 3");
});
it("renders historical runs without liveness metadata as unavailable", () => {
renderLedger({
runs: [
createRun({
livenessState: null,
livenessReason: null,
continuationAttempt: undefined,
lastUsefulActionAt: null,
nextAction: null,
resultJson: null,
}),
],
});
expect(container.textContent).toContain("No liveness data");
expect(container.textContent).toContain("Stop Unavailable");
expect(container.textContent).toContain("Last useful action Unavailable");
});
it("interleaves run rows and activity rows by timestamp", () => {
renderLedger({
runs: [
createRun({
runId: "run-oldest",
startedAt: "2026-04-18T19:55:00.000Z",
createdAt: "2026-04-18T19:55:00.000Z",
}),
createRun({
runId: "run-newest",
startedAt: "2026-04-18T19:59:00.000Z",
createdAt: "2026-04-18T19:59:00.000Z",
}),
],
activityEvents: [
createActivity({
id: "activity-middle",
action: "activity-middle",
createdAt: new Date("2026-04-18T19:57:00.000Z"),
}),
],
renderActivityEvent: (event) => (
<div data-testid={`activity-${event.id}`}>{event.action}</div>
),
});
const text = container.textContent ?? "";
const newestIndex = text.indexOf("run-newe");
const activityIndex = text.indexOf("activity-middle");
const oldestIndex = text.indexOf("run-olde");
expect(newestIndex).toBeGreaterThanOrEqual(0);
expect(activityIndex).toBeGreaterThan(newestIndex);
expect(oldestIndex).toBeGreaterThan(activityIndex);
});
it("shows live runs as pending final checks without missing-data language", () => {
renderLedger({
runs: [
createRun({
status: "running",
finishedAt: null,
livenessState: null,
livenessReason: null,
continuationAttempt: 0,
lastUsefulActionAt: null,
nextAction: null,
resultJson: null,
}),
],
});
expect(container.textContent).toContain("Running now by CodexCoder");
expect(container.textContent).toContain("Checks after finish");
expect(container.textContent).toContain("Last useful action No action recorded yet");
expect(container.textContent).toContain("Stop Still running");
expect(container.textContent).not.toContain("Liveness pending");
expect(container.textContent).not.toContain("initial attempt");
});
it("surfaces scheduled retry timing and exhaustion state without opening logs", () => {
renderLedger({
runs: [
createRun({
runId: "run-scheduled",
status: "scheduled_retry",
finishedAt: null,
livenessState: null,
livenessReason: null,
retryOfRunId: "run-root",
scheduledRetryAt: "2026-04-18T20:15:00.000Z",
scheduledRetryAttempt: 2,
scheduledRetryReason: "transient_failure",
}),
createRun({
runId: "run-exhausted",
status: "failed",
createdAt: "2026-04-18T19:57:00.000Z",
retryOfRunId: "run-root",
scheduledRetryAttempt: 4,
scheduledRetryReason: "transient_failure",
retryExhaustedReason: "Bounded retry exhausted after 4 scheduled attempts; no further automatic retry will be queued",
}),
],
});
expect(container.textContent).toContain("Retry scheduled");
expect(container.textContent).toContain("Attempt 2");
expect(container.textContent).toContain("Transient failure");
expect(container.textContent).toContain("Next retry");
expect(container.textContent).toContain("Retry exhausted");
expect(container.textContent).toContain("no further automatic retry will be queued");
expect(container.textContent).toContain("Manual intervention required");
});
it("labels max-turn stops and continuation retries without confusing them with per-run turns", () => {
renderLedger({
runs: [
createRun({
runId: "run-scheduled-continuation",
status: "scheduled_retry",
finishedAt: null,
livenessState: null,
livenessReason: null,
retryOfRunId: "run-max-turns",
scheduledRetryAt: "2026-04-18T20:15:00.000Z",
scheduledRetryAttempt: 1,
scheduledRetryReason: "max_turns_continuation",
}),
createRun({
runId: "run-max-turns",
resultJson: { stopReason: "max_turns_exhausted" },
createdAt: "2026-04-18T19:57:00.000Z",
}),
createRun({
runId: "run-continuation-exhausted",
status: "failed",
createdAt: "2026-04-18T19:56:00.000Z",
retryOfRunId: "run-max-turns",
scheduledRetryAttempt: 3,
scheduledRetryReason: "max_turns_continuation",
retryExhaustedReason: "Bounded retry exhausted after 3 scheduled attempts; no further automatic retry will be queued",
}),
],
});
expect(container.textContent).toContain("Continuation scheduled");
expect(container.textContent).toContain("Max-turn continuation");
expect(container.textContent).toContain("Next continuation");
expect(container.textContent).toContain("Stop max turns exhausted");
expect(container.textContent).toContain("Continuation exhausted");
});
it("shows timeout, cancel, and budget stop reasons without raw logs", () => {
renderLedger({
runs: [
createRun({
runId: "run-timeout",
resultJson: { stopReason: "timeout", timeoutFired: true, effectiveTimeoutSec: 30 },
}),
createRun({
runId: "run-cancel",
resultJson: { stopReason: "cancelled" },
createdAt: "2026-04-18T19:57:00.000Z",
}),
createRun({
runId: "run-budget",
resultJson: { stopReason: "budget_paused" },
createdAt: "2026-04-18T19:56:00.000Z",
}),
createRun({
runId: "run-background-task",
resultJson: { stopReason: "unmanaged_background_task_stopped" },
createdAt: "2026-04-18T19:55:30.000Z",
}),
createRun({
runId: "run-paused",
resultJson: { stopReason: "paused" },
createdAt: "2026-04-18T19:55:00.000Z",
}),
],
});
expect(container.textContent).toContain("timeout (30s timeout)");
expect(container.textContent).toContain("cancelled");
expect(container.textContent).toContain("budget paused");
expect(container.textContent).toContain("unmanaged background task stopped");
expect(container.textContent).toContain("paused by board");
});
it("surfaces active and completed child issue summaries", () => {
renderLedger({
childIssues: [
createIssue({ id: "child-1", identifier: "PAP-2", title: "Implement worker handoff", status: "in_progress" }),
createIssue({ id: "child-2", identifier: "PAP-3", title: "Verify final report", status: "done" }),
createIssue({ id: "child-3", identifier: "PAP-4", title: "Cancelled experiment", status: "cancelled" }),
],
});
expect(container.textContent).toContain("Child work");
expect(container.textContent).toContain("1 active, 1 done, 1 cancelled");
expect(container.textContent).toContain("PAP-2");
expect(container.textContent).toContain("Implement worker handoff");
renderLedger({
childIssues: [
createIssue({ id: "child-2", identifier: "PAP-3", title: "Verify final report", status: "done" }),
createIssue({ id: "child-3", identifier: "PAP-4", title: "Cancelled experiment", status: "cancelled" }),
],
});
expect(container.textContent).toContain("all 2 terminal (1 done, 1 cancelled)");
});
it("uses wrapping-friendly markup for long next action text", () => {
renderLedger({
runs: [
createRun({
nextAction: "Continue investigating this intentionally-long-next-action-token-that-needs-to-wrap-cleanly-on-mobile-and-desktop-without-overlapping-controls.",
}),
],
});
const nextAction = [...container.querySelectorAll("span")]
.find((node) => node.textContent?.includes("intentionally-long-next-action-token"));
expect(nextAction?.className).toContain("break-words");
expect(container.textContent).toContain("Next action:");
});
it("shows when older runs are clipped from the ledger", () => {
renderLedger({
runs: Array.from({ length: 22 }, (_, index) =>
createRun({
runId: `run-${index.toString().padStart(8, "0")}`,
createdAt: `2026-04-18T19:${String(index).padStart(2, "0")}:00.000Z`,
}),
),
});
expect(container.textContent).toContain("2 older items not shown");
});
it("renders stale-run banner, watchdog actions, and silence badge for live runs", () => {
const onWatchdogDecision = vi.fn();
renderLedger({
runs: [createRun({ runId: "run-live-1", status: "running", finishedAt: null })],
activeRun: createActiveRun(),
onWatchdogDecision,
});
expect(container.textContent).toContain("Stale-run watchdog alert");
expect(container.textContent).toContain("PAP-404");
expect(container.textContent).toContain("Stale run");
const watchdogBanner = Array.from(container.querySelectorAll("p"))
.find((node) => node.textContent?.includes("Stale-run watchdog alert"))
?.closest("div");
expect(watchdogBanner?.className).toContain("border-red-500/30");
expect(watchdogBanner?.className).toContain("bg-red-500/10");
const continueButton = Array.from(container.querySelectorAll("button")).find(
(button) => button.textContent?.includes("Continue monitoring"),
);
expect(continueButton).not.toBeUndefined();
act(() => {
continueButton?.dispatchEvent(new MouseEvent("click", { bubbles: true }));
});
expect(onWatchdogDecision).toHaveBeenCalledWith({
runId: "run-live-1",
decision: "continue",
evaluationIssueId: "issue-eval-1",
});
});
it("renders requested/applied model profile and surfaces fallback reasons", () => {
renderLedger({
runs: [
createRun({
runId: "run-cheap-applied",
resultJson: {
modelProfile: {
requested: "cheap",
applied: "cheap",
configSource: "agent_runtime",
fallbackReason: null,
},
},
}),
createRun({
runId: "run-cheap-fallback",
createdAt: "2026-04-18T19:50:00.000Z",
resultJson: {
modelProfile: {
requested: "cheap",
applied: null,
configSource: null,
fallbackReason: "agent_runtime_profile_disabled",
},
},
}),
],
});
expect(container.textContent).toContain("Profile: cheap");
expect(container.textContent).toContain("Profile: cheap (unavailable)");
expect(container.textContent).toContain("Cheap profile fell back to primary");
expect(container.textContent).toContain("agent_runtime_profile_disabled");
});
it("hides watchdog decision actions for known non-owner viewers", () => {
const onWatchdogDecision = vi.fn();
renderLedger({
runs: [createRun({ runId: "run-live-1", status: "running", finishedAt: null })],
activeRun: createActiveRun(),
canRecordWatchdogDecisions: false,
onWatchdogDecision,
});
expect(container.textContent).toContain("Stale-run watchdog alert");
expect(container.textContent).toContain("PAP-404");
expect(container.textContent).not.toContain("Continue monitoring");
expect(container.textContent).not.toContain("Snooze 1h");
expect(container.textContent).not.toContain("Mark false positive");
expect(container.querySelectorAll("button")).toHaveLength(0);
expect(onWatchdogDecision).not.toHaveBeenCalled();
});
it("surfaces the responsible user a run acts on behalf of", () => {
renderLedger({
runs: [createRun({ runId: "run-obo-1", responsibleUserId: "user-2" })],
resolveUserLabel: (userId) => (userId === "user-2" ? "Ada Lovelace" : null),
});
const chip = container.querySelector('[data-testid="run-on-behalf-of"]');
expect(chip).not.toBeNull();
expect(chip?.textContent).toContain("on behalf of");
expect(chip?.textContent).toContain("Ada Lovelace");
});
it("omits the on-behalf-of chip when the run has no responsible user", () => {
renderLedger({
runs: [createRun({ runId: "run-obo-2", responsibleUserId: null })],
resolveUserLabel: () => "Ada Lovelace",
});
expect(container.querySelector('[data-testid="run-on-behalf-of"]')).toBeNull();
});
it("renders actionable copy for a responsible-user-unauthorized run failure", () => {
renderLedger({
runs: [
createRun({
runId: "run-denied-1",
status: "failed",
livenessState: "failed",
responsibleUserId: "user-2",
errorCode: "RESPONSIBLE_USER_UNAUTHORIZED",
}),
],
resolveUserLabel: () => "Ada Lovelace",
});
const notice = container.querySelector('[data-testid="responsible-user-denial-notice"]');
expect(notice).not.toBeNull();
expect(notice?.getAttribute("data-denial-tone")).toBe("unauthorized");
expect(notice?.textContent).toContain("Ada Lovelace");
expect(notice?.textContent).toContain("Responsible user not authorized");
});
it("steers the responsible-user-unavailable failure toward marking work blocked", () => {
renderLedger({
runs: [
createRun({
runId: "run-denied-2",
status: "failed",
livenessState: "failed",
responsibleUserId: "user-3",
errorCode: "RESPONSIBLE_USER_UNAVAILABLE",
}),
],
resolveUserLabel: () => "Grace Hopper",
});
const notice = container.querySelector('[data-testid="responsible-user-denial-notice"]');
expect(notice).not.toBeNull();
expect(notice?.getAttribute("data-denial-tone")).toBe("unavailable");
expect(notice?.textContent?.toLowerCase()).toContain("blocked");
});
it("does not render a denial notice for a generic agent failure", () => {
renderLedger({
runs: [
createRun({
runId: "run-denied-3",
status: "failed",
livenessState: "failed",
errorCode: "budget_blocked",
}),
],
});
expect(container.querySelector('[data-testid="responsible-user-denial-notice"]')).toBeNull();
});
});