Files
PaperClipAI/server/src/__tests__/activity-service.test.ts
T
DottaandClaude Opus 5.5 8a97037bd2 fix(runs): say who started a retry run in the issue run read (#21486)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The chat agent answers questions about failed runs by reading the
task's run list (`GET /api/issues/{id}/runs`)
> - For a retry run, that read returned only raw invocation fields
(`on_demand`, `manual`) and an owner id. It did not say who started the
retry
> - In chat-eval cohort 13, the agent told the person "Paperclip retried
automatically" about a run that the board started with Retry
> - This pull request adds a plain-words `retriedBy` field to each retry
run, and tells the chat agent to use it
> - The benefit is that people get a true account of who retried their
work, with a name and no raw ids

## Linked Issues or Issue Description

**What happened?**
Chat-eval cohort 13 ran OpenCode with
`openrouter/deepseek/deepseek-v4.1-flash-1017` on master `df1b12ad8e`.
In `runs-adapters.inspect-failed-run`, the reply said "Paperclip retried
automatically on a supported model". The successful run was a person's
manual Retry: `invocationSource: on_demand`, `triggerDetail: manual`,
wake reason `retry_failed_run`, and the responsible user was the board.
The agent's own reasoning hedged ("probably automatic"), because the run
read did not say who started the retry.

**Expected behavior**
The run reads that the agent uses say who started each retry: a person,
by display name, or Paperclip's automatic recovery, in plain words and
with no raw ids.

**Steps to reproduce**
1. Let a run fail on a task.
2. Press Retry on the failed run as a board user.
3. Read `GET /api/issues/{id}/runs`. Before this change, nothing in the
retry row says that a person pressed Retry.

## What Changed

- `runsForIssue` (`server/src/services/activity.ts`) reads the wake
request of each retry run. It adds `retriedBy` to each row:
- `{ kind: "person", personName, summary }` when a user wake with reason
`retry_failed_run` or `task_retry` started the run. The name comes from
the user record, or "Board" for the local board, with the same fallback
as activity actor names.
  - `{ kind: "automatic", summary }` for an `automation` retry run.
- `null` for a run that is not a retry, or a retry whose starter is not
recorded. The read does not guess. It does not use `responsibleUserId`,
because automatic retries keep the original responsible user.
- The OpenAPI description of `GET /api/issues/{id}/runs` and the chat
agent directive tell the agent to use `retriedBy` and not to call a
retry automatic unless `retriedBy` says so.
- The UI `RunForIssue` type has the new optional field.

## Verification

- `cd server && npx vitest run src/__tests__/activity-service.test.ts
src/__tests__/agent-conversations.test.ts
src/__tests__/activity-routes.test.ts
src/__tests__/openapi-routes.test.ts` passes.
- Red/green: the new test "says who started each retry run" fails
without the service change (`expected undefined to be null`) and passes
with it. It covers a board Retry, a named member's Retry, an automatic
retry that keeps the board as responsible user, a retry with no recorded
wake, and no raw ids in the output.
- `npx tsc --noEmit -p server` passes.
- A narrow paid run of `runs-adapters.inspect-failed-run` (with the two
sibling cohort-13 PRs) ran on a combined branch. The results are in a PR
comment.

## Risks

- Low risk. The change adds one read-only field and one bounded query by
primary key on the wake requests of retry runs in the page.
- No persisted state changes.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), used through Claude
Code with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 14:24:04 -06:00

887 lines
36 KiB
TypeScript

import { randomUUID } from "node:crypto";
import { eq } from "drizzle-orm";
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
import {
activityLog,
agentWakeupRequests,
agents,
authUsers,
companies,
createDb,
documentRevisions,
documents,
heartbeatRuns,
issueComments,
issueDocuments,
issues,
} from "@paperclipai/db";
import { ISSUE_CONTINUATION_SUMMARY_DOCUMENT_KEY } from "@paperclipai/shared";
import {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
} from "./helpers/embedded-postgres.js";
import { activityService } from "../services/activity.ts";
import { issueReadSqlCondition } from "../services/authorization.ts";
import { issueService } from "../services/issues.ts";
import { documentService } from "../services/documents.ts";
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
type ActivityService = ReturnType<typeof activityService>;
type IssueRun = Awaited<ReturnType<ActivityService["runsForIssue"]>>[number];
if (!embeddedPostgresSupport.supported) {
console.warn(
`Skipping embedded Postgres activity service tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
);
}
async function waitForIssueRun(
service: ActivityService,
companyId: string,
issueId: string,
predicate: (run: IssueRun) => boolean,
) {
const deadline = Date.now() + 2_000;
let latestRuns: IssueRun[] = [];
while (Date.now() < deadline) {
latestRuns = await service.runsForIssue(companyId, issueId);
const run = latestRuns.find(predicate);
if (run) return { run, runs: latestRuns };
await new Promise((resolve) => setTimeout(resolve, 25));
}
throw new Error(`Timed out waiting for issue run. Latest run count: ${latestRuns.length}`);
}
describeEmbeddedPostgres("activity service", () => {
let db!: ReturnType<typeof createDb>;
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
beforeAll(async () => {
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-activity-service-");
db = createDb(tempDb.connectionString);
}, 20_000);
afterEach(async () => {
await db.delete(activityLog);
await db.delete(issueComments);
await db.delete(issueDocuments);
await db.delete(documentRevisions);
await db.delete(documents);
await db.delete(issues);
await db.delete(heartbeatRuns);
await db.delete(agentWakeupRequests);
await db.delete(agents);
await db.delete(companies);
});
afterAll(async () => {
await tempDb?.cleanup();
});
it.each(["standard", "planning", "ask"] as const)("LCA-05 persists explicit %s mode across wording edits and uses it in ledger backfill", async (workMode) => {
const companyId = randomUUID();
const agentId = randomUUID();
await db.insert(companies).values({ id: companyId, name: "Mode fixture", issuePrefix: `M${companyId.slice(0, 6)}` });
await db.insert(agents).values({ id: agentId, companyId, name: "Writer", role: "engineer", status: "idle", adapterType: "codex_local" });
const service = issueService(db);
const task = await service.create(companyId, { title: "Making a plan", description: "Create a report and research proposal.", status: "in_progress", assigneeAgentId: agentId, ...(workMode === "standard" ? {} : { workMode }) });
expect(task.workMode).toBe(workMode);
for (const prose of [
{ title: "Implement exporter", description: "Change the code now." },
{ title: "Making a plan", description: "Create a plan for the report exporter." },
]) {
await service.update(task.id, prose);
expect((await service.getById(task.id))?.workMode).toBe(workMode);
}
const runId = randomUUID();
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, status: "succeeded", contextSnapshot: { issueId: task.id }, resultJson: { summary: "I will inspect the repository next." } });
const expected = workMode === "planning" ? "advanced" : "plan_only";
await waitForIssueRun(activityService(db), companyId, task.id, run => run.runId === runId && run.livenessState === expected);
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId)))[0].livenessState).toBe(expected);
if (workMode === "standard") {
await documentService(db).upsertIssueDocument({ issueId: task.id, key: "plan", title: "Requested plan", format: "markdown", body: "1. Inspect files.\n2. Implement the exporter.", createdByAgentId: agentId });
expect((await service.getById(task.id))?.workMode).toBe("standard");
expect(await documentService(db).getIssueDocumentByKey(task.id, "plan")).toBeTruthy();
}
const nextMode = workMode === "planning" ? "standard" : "planning";
await service.update(task.id, { workMode: nextMode });
expect((await service.getById(task.id))?.workMode).toBe(nextMode);
});
it("names each activity actor for the person instead of showing a raw id such as local-board", async () => {
const companyId = randomUUID(), agentId = randomUUID(), userId = `member-${companyId}`;
await db.insert(companies).values({ id: companyId, name: "Names", issuePrefix: `N${companyId.slice(0, 6)}` });
await db.insert(agents).values({ id: agentId, companyId, name: "Morgan", role: "engineer", status: "idle", adapterType: "codex_local" });
await db.insert(authUsers).values({ id: userId, name: "Sam Rivera", email: `${userId}@example.test`, createdAt: new Date(), updatedAt: new Date() });
const base = { companyId, action: "issue.updated", entityType: "company", entityId: companyId };
await db.insert(activityLog).values([
{ ...base, actorType: "user", actorId: "local-board" },
{ ...base, actorType: "user", actorId: userId },
{ ...base, actorType: "agent", actorId: agentId },
{ ...base, actorType: "system", actorId: "system" },
{ ...base, actorType: "agent", actorId: "external-agent" },
]);
const named = await activityService(db).withActorNames(await activityService(db).list({ companyId }));
expect(Object.fromEntries(named.map(row => [row.actorId, row.actorName]))).toEqual({
"local-board": "Board", [userId]: "Sam Rivera", [agentId]: "Morgan", system: "Paperclip", "external-agent": "An agent",
});
await db.delete(authUsers).where(eq(authUsers.id, userId));
});
// Recorded in chat-eval cohort 13 (runs-adapters.inspect-failed-run): the
// agent told the person "Paperclip retried automatically" about a run the
// board started with Retry, because the run read gave only raw invocation
// fields (on_demand, manual) and an owner id.
it("says who started each retry run: a named person's Retry or automatic recovery", async () => {
const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID(), userId = `member-${companyId}`;
const failedRunId = randomUUID(), boardRetryId = randomUUID(), memberRetryId = randomUUID(), autoRetryId = randomUUID(), unknownRetryId = randomUUID();
await db.insert(companies).values({ id: companyId, name: "Retries", issuePrefix: `R${companyId.slice(0, 6)}` });
await db.insert(agents).values({ id: agentId, companyId, name: "Morgan", adapterType: "process" });
await db.insert(authUsers).values({ id: userId, name: "Sam Rivera", email: `${userId}@example.test`, createdAt: new Date(), updatedAt: new Date() });
await db.insert(issues).values({ id: issueId, companyId, title: "Website", assigneeAgentId: agentId });
const wake = async (reason: string, requestedByActorType: string | null, requestedByActorId: string | null, source = "on_demand") => {
const [row] = await db.insert(agentWakeupRequests).values({ companyId, agentId, source, triggerDetail: source === "automation" ? "system" : "manual",
reason, requestedByActorType, requestedByActorId, status: "completed", payload: { issueId } }).returning();
return row!.id;
};
const base = { companyId, agentId, issueId, scopeKind: "issue", contextSnapshot: { issueId } } as const;
const memberTaskRetryId = await wake("task_retry", "user", userId), inheritedRetryId = randomUUID();
const at = (seconds: number) => new Date(Date.UTC(2026, 10, 13, 18, 0, seconds));
await db.insert(heartbeatRuns).values([
{ ...base, id: failedRunId, status: "failed", errorCode: "native_provider_model_rejected", invocationSource: "assignment", createdAt: at(0) },
{ ...base, id: boardRetryId, status: "succeeded", invocationSource: "on_demand", triggerDetail: "manual", responsibleUserId: "local-board",
retryOfRunId: failedRunId, wakeupRequestId: await wake("retry_failed_run", "user", "local-board"), createdAt: at(1) },
// Task Retry, as the queue saves it: the person's request is linked from
// the run context with the source run as previousRunId, and retryOfRunId stays empty.
{ ...base, id: memberRetryId, status: "succeeded", invocationSource: "on_demand", triggerDetail: "manual",
wakeupRequestId: await wake("issue_assigned", "user", userId),
contextSnapshot: { issueId, previousRunId: failedRunId, forceFreshSession: true, taskRetryRequestId: memberTaskRetryId }, createdAt: at(2) },
// Automatic recovery keeps the original responsible user; that is not who started it.
{ ...base, id: autoRetryId, status: "succeeded", invocationSource: "automation", triggerDetail: "system", responsibleUserId: "local-board",
retryOfRunId: failedRunId, wakeupRequestId: await wake("transient_failure_retry", null, null, "automation"), createdAt: at(3) },
// Automatic recovery after that Task Retry inherits the person's request in its context.
{ ...base, id: inheritedRetryId, status: "succeeded", invocationSource: "automation", triggerDetail: "system", retryOfRunId: memberRetryId,
wakeupRequestId: await wake("transient_failure_retry", null, null, "automation"),
contextSnapshot: { issueId, previousRunId: failedRunId, taskRetryRequestId: memberTaskRetryId }, createdAt: at(5) },
{ ...base, id: unknownRetryId, status: "succeeded", invocationSource: "on_demand", triggerDetail: "manual", retryOfRunId: failedRunId, createdAt: at(4) },
]);
const rows = await activityService(db).runsForIssue(companyId, issueId);
const byId = new Map(rows.map(row => [row.runId, row.retriedBy]));
expect(byId.get(failedRunId)).toBeNull();
expect(byId.get(boardRetryId)).toEqual({ kind: "person", personName: "Board",
summary: "Board pressed Retry on the failed run. Paperclip did not retry it automatically." });
expect(byId.get(memberRetryId)).toEqual({ kind: "person", personName: "Sam Rivera",
summary: "Sam Rivera pressed Retry on the failed run. Paperclip did not retry it automatically." });
expect(byId.get(autoRetryId)).toEqual({ kind: "automatic",
summary: "Paperclip retried the failed run automatically; no person pressed Retry." });
expect(byId.get(inheritedRetryId)).toMatchObject({ kind: "automatic" });
expect(byId.get(unknownRetryId)).toBeNull();
expect(JSON.stringify([...byId.values()])).not.toMatch(/local-board|member-/);
await db.delete(authUsers).where(eq(authUsers.id, userId));
});
it("limits company activity lists", async () => {
const companyId = randomUUID();
await db.insert(companies).values({
id: companyId,
name: "Paperclip",
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
await db.insert(activityLog).values([
{
companyId,
actorType: "system",
actorId: "system",
action: "test.oldest",
entityType: "company",
entityId: companyId,
createdAt: new Date("2026-04-21T10:00:00.000Z"),
},
{
companyId,
actorType: "system",
actorId: "system",
action: "test.middle",
entityType: "company",
entityId: companyId,
createdAt: new Date("2026-04-21T11:00:00.000Z"),
},
{
companyId,
actorType: "system",
actorId: "system",
action: "test.newest",
entityType: "company",
entityId: companyId,
createdAt: new Date("2026-04-21T12:00:00.000Z"),
},
]);
const result = await activityService(db).list({ companyId, limit: 2 });
expect(result.map((event) => event.action)).toEqual(["test.newest", "test.middle"]);
});
it("filters company activity rows whose entity is a private issue", async () => {
const companyId = randomUUID();
const privateIssueId = randomUUID();
const openIssueId = randomUUID();
await db.insert(companies).values({
id: companyId,
name: "Private activity",
issuePrefix: `A${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
});
await db.insert(issues).values([
{
id: privateIssueId,
companyId,
title: "Confidential activity",
status: "todo",
priority: "medium",
visibility: "private",
privacyRootIssueId: privateIssueId,
responsibleUserId: "private-owner",
},
{
id: openIssueId,
companyId,
title: "Open activity",
status: "todo",
priority: "medium",
},
]);
await db.insert(activityLog).values([
{ companyId, actorType: "system", actorId: "system", action: "private.changed", entityType: "issue", entityId: privateIssueId },
{ companyId, actorType: "system", actorId: "system", action: "open.changed", entityType: "issue", entityId: openIssueId },
]);
const actor = {
type: "board" as const,
userId: "non-member",
companyIds: [companyId],
source: "session" as const,
isInstanceAdmin: false,
};
const rows = await activityService(db).list({
companyId,
readCondition: await issueReadSqlCondition(db, actor),
});
expect(rows.map((row) => row.action)).toContain("open.changed");
expect(rows.map((row) => row.action)).not.toContain("private.changed");
});
it.each([null, "native_provider_model_rejected", "adapter_failed"])("returns compact issue runs with bounded model rejection details: %s", async (errorCode) => {
const companyId = randomUUID();
const agentId = randomUUID();
const issueId = randomUUID();
const runId = randomUUID();
const wakeupRequestId = randomUUID();
await db.insert(companies).values({
id: companyId,
name: "Paperclip",
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
await db.insert(agents).values({
id: agentId,
companyId,
name: "CodexCoder",
role: "engineer",
status: "running",
adapterType: "codex_local",
adapterConfig: {},
runtimeConfig: {},
permissions: {},
});
await db.insert(issues).values({
id: issueId,
companyId,
title: "Summarize a completed run",
status: "done",
priority: "medium",
assigneeAgentId: agentId,
});
await db.insert(agentWakeupRequests).values({
id: wakeupRequestId, companyId, agentId, source: "issue", status: "claimed",
});
await db.insert(heartbeatRuns).values({
id: runId,
wakeupRequestId,
companyId,
agentId,
scopeKind: "issue",
issueId,
invocationSource: "assignment",
status: errorCode ? "failed" : "succeeded",
errorCode,
error: "provider-error".repeat(200),
contextSnapshot: { issueId, queuedCommentSourceWakeIds: ["coalesced-queue"] },
usageJson: {
inputTokens: 11,
output_tokens: 7,
cache_read_input_tokens: 3,
billingType: "metered",
costUsd: 0.42,
enormousBlob: "x".repeat(256_000),
},
resultJson: {
conversationReset: true,
configurationIncomplete: { reason: "workspace_base_ref_unresolved", requestedRef: "main", fetchError: "private diagnostic" },
billing_type: "metered",
total_cost_usd: 0.42,
stopReason: "timeout",
effectiveTimeoutSec: 30,
timeoutFired: true,
summary: "done",
nestedHuge: { payload: "y".repeat(256_000) },
},
livenessState: "advanced",
livenessReason: "Run produced concrete action evidence: 1 issue comment(s)",
continuationAttempt: 2,
lastUsefulActionAt: new Date("2026-04-18T19:59:00.000Z"),
nextAction: "Review the completed output.",
});
const runs = await activityService(db).runsForIssue(companyId, issueId);
expect(runs).toHaveLength(1);
expect(runs[0]).toMatchObject({
runId,
wakeupRequestId,
agentId,
queuedCommentSourceWakeIds: ["coalesced-queue"],
invocationSource: "assignment",
contextIssueId: issueId,
error: errorCode === "native_provider_model_rejected" ? "provider-error".repeat(200).slice(0, 2000) : null,
});
expect(runs[0]?.usageJson).toEqual({
inputTokens: 11,
input_tokens: 11,
outputTokens: 7,
output_tokens: 7,
cachedInputTokens: 3,
cached_input_tokens: 3,
cache_read_input_tokens: 3,
billingType: "metered",
billing_type: "metered",
costUsd: 0.42,
cost_usd: 0.42,
total_cost_usd: 0.42,
});
expect(runs[0]?.resultJson).toEqual({
conversationReset: true,
configurationIncomplete: { reason: "workspace_base_ref_unresolved" },
billingType: "metered",
billing_type: "metered",
costUsd: 0.42,
cost_usd: 0.42,
total_cost_usd: 0.42,
stopReason: "timeout",
effectiveTimeoutSec: 30,
timeoutFired: true,
});
expect(runs[0]).toMatchObject({
livenessState: "advanced",
livenessReason: "Run produced concrete action evidence: 1 issue comment(s)",
continuationAttempt: 2,
lastUsefulActionAt: new Date("2026-04-18T19:59:00.000Z"),
nextAction: "Review the completed output.",
});
expect(runs[0]).not.toHaveProperty("contextSnapshot");
});
it.each(["requested", "acknowledged", "pending_native", "missing_native", "wrong_run", "wrong_company",
"malformed_execution", "null_execution", "malformed_native"] as const)(
"projects only bounded, owner-matched Stop evidence in issue history: %s", async proof => {
const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID(), runId = randomUUID();
await db.insert(companies).values({ id: companyId, name: "Stop history", issuePrefix: `S${companyId.slice(0, 6)}` });
await db.insert(agents).values({ id: agentId, companyId, name: "Stopped runner", adapterType: "process" });
await db.insert(issues).values({ id: issueId, companyId, title: "Stop the current run", assigneeAgentId: agentId });
const native = { schema: "paperclip.native-cancellation.v1", runId, companyId,
dispatchState: proof === "pending_native" ? "pending" : "acknowledged",
intentAuditId: "private-stop-proof", effects: ["private-stop-proof"], secret: "private-stop-proof" };
const result: Record<string, unknown> = {
executionCancellation: { state: proof === "requested" ? "requested" : "acknowledged",
proof: "private-stop-proof", secret: "private-stop-proof" },
nativeCancellation: native, unrelated: "private-stop-proof",
};
if (proof === "missing_native") delete result.nativeCancellation;
if (proof === "wrong_run") native.runId = randomUUID();
if (proof === "wrong_company") native.companyId = randomUUID();
if (proof === "malformed_native") result.nativeCancellation = { ...native, dispatchState: { secret: "private-stop-proof" } };
if (proof === "malformed_execution") result.executionCancellation = { state: { secret: "private-stop-proof" } };
if (proof === "null_execution") result.executionCancellation = null;
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, issueId, scopeKind: "issue",
nativeIssueId: issueId, runtimeMode: "native", status: "cancelled", finishedAt: new Date(),
contextSnapshot: { issueId }, livenessState: "advanced", resultJson: result });
const rows = await activityService(db).runsForIssue(companyId, issueId);
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({ runId, runtimeMode: "native", status: "cancelled" });
expect(rows[0]!.resultJson).toEqual({
executionCancellation: { state: proof === "requested" ? "requested"
: proof === "malformed_execution" || proof === "null_execution" ? "unknown" : "acknowledged" },
...(["missing_native", "wrong_run", "wrong_company", "malformed_native"].includes(proof) ? {} : {
nativeCancellation: { schema: "paperclip.native-cancellation.v1", runId, companyId,
dispatchState: proof === "pending_native" ? "pending" : "acknowledged" },
}),
});
expect(JSON.stringify(rows)).not.toContain("private-stop-proof");
},
);
it.each([
["recorded", "40001", "40001"],
["legacy row without the field", undefined, null],
["malformed", 'Failed query: select "secret" from x', null],
["overlong", "X".repeat(81), null],
["boolean", true, null],
["number", 40001, null],
] as const)("projects only a closed-shape failure cause in issue history: %s", async (_case, stored, expected) => {
const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID(), runId = randomUUID();
await db.insert(companies).values({ id: companyId, name: "Failure cause", issuePrefix: `F${companyId.slice(0, 6)}` });
await db.insert(agents).values({ id: agentId, companyId, name: "Failing runner", adapterType: "process" });
await db.insert(issues).values({ id: issueId, companyId, title: "Fail with a cause", assigneeAgentId: agentId });
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, issueId, scopeKind: "issue", status: "failed",
errorCode: "adapter_failed", finishedAt: new Date(), contextSnapshot: { issueId }, livenessState: "failed",
resultJson: { stopReason: "adapter_failed", ...(stored === undefined ? {} : { failureCause: stored }) } });
const [row] = await activityService(db).runsForIssue(companyId, issueId);
expect(row).toMatchObject({ runId, errorCode: "adapter_failed", failureCause: expected });
expect(row!.resultJson).toEqual({ stopReason: "adapter_failed" });
expect(JSON.stringify(row)).not.toContain("secret");
});
it("backfills missing liveness for completed issue runs before returning the ledger", async () => {
const companyId = randomUUID();
const agentId = randomUUID();
const issueId = randomUUID();
const runId = randomUUID();
const completedAt = new Date("2026-04-18T20:04:00.000Z");
await db.insert(companies).values({
id: companyId,
name: "Paperclip",
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
await db.insert(agents).values({
id: agentId,
companyId,
name: "CodexCoder",
role: "engineer",
status: "idle",
adapterType: "codex_local",
adapterConfig: {},
runtimeConfig: {},
permissions: {},
});
await db.insert(issues).values({
id: issueId,
companyId,
title: "Fix run ledger",
description: "Make the run ledger answer whether a run advanced.",
status: "done",
priority: "medium",
assigneeAgentId: agentId,
completedAt,
});
await db.insert(heartbeatRuns).values({
id: runId,
companyId,
agentId,
scopeKind: "issue",
issueId,
invocationSource: "assignment",
status: "succeeded",
startedAt: new Date("2026-04-18T20:00:00.000Z"),
finishedAt: completedAt,
contextSnapshot: { issueId },
resultJson: {
summary: "Finished the implementation.",
},
livenessState: null,
livenessReason: null,
lastUsefulActionAt: null,
nextAction: null,
});
await db.insert(issueComments).values({
companyId,
issueId,
authorAgentId: agentId,
createdByRunId: runId,
body: "Done",
createdAt: completedAt,
});
const service = activityService(db);
const { run, runs } = await waitForIssueRun(
service,
companyId,
issueId,
(entry) => entry.runId === runId && entry.livenessState === "completed",
);
expect(runs).toHaveLength(1);
expect(run).toMatchObject({
runId,
livenessState: "completed",
livenessReason: "Issue is done",
continuationAttempt: 0,
lastUsefulActionAt: completedAt,
});
const [persisted] = await db.select().from(heartbeatRuns);
expect(persisted).toMatchObject({
id: runId,
livenessState: "completed",
livenessReason: "Issue is done",
continuationAttempt: 0,
lastUsefulActionAt: completedAt,
});
});
it("does not backfill document evidence from a different run", async () => {
const companyId = randomUUID();
const agentId = randomUUID();
const issueId = randomUUID();
const runId = randomUUID();
const otherRunId = randomUUID();
const documentId = randomUUID();
const revisionId = randomUUID();
const createdAt = new Date("2026-04-18T20:08:00.000Z");
await db.insert(companies).values({
id: companyId,
name: "Paperclip",
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
await db.insert(agents).values({
id: agentId,
companyId,
name: "CodexCoder",
role: "engineer",
status: "idle",
adapterType: "codex_local",
adapterConfig: {},
runtimeConfig: {},
permissions: {},
});
await db.insert(issues).values({
id: issueId,
companyId,
title: "Fix run ledger",
description: "Make the run ledger answer whether a run advanced.",
status: "in_progress",
priority: "medium",
assigneeAgentId: agentId,
});
await db.insert(heartbeatRuns).values([
{
id: runId,
companyId,
agentId,
scopeKind: "issue",
issueId,
invocationSource: "assignment",
status: "succeeded",
startedAt: new Date("2026-04-18T20:00:00.000Z"),
finishedAt: new Date("2026-04-18T20:02:00.000Z"),
contextSnapshot: { issueId },
resultJson: {
summary: "Next steps:\n- inspect files",
},
livenessState: null,
livenessReason: null,
},
{
id: otherRunId,
companyId,
agentId,
scopeKind: "issue",
issueId,
invocationSource: "assignment",
status: "succeeded",
startedAt: new Date("2026-04-18T20:05:00.000Z"),
finishedAt: createdAt,
contextSnapshot: { issueId },
resultJson: {
summary: "Updated the plan document.",
},
livenessState: "advanced",
livenessReason: "Run produced concrete action evidence: 1 document revision(s)",
},
]);
await db.insert(documents).values({
id: documentId,
companyId,
title: "Plan",
format: "markdown",
latestBody: "# Plan\n\n- Inspect files",
latestRevisionId: revisionId,
latestRevisionNumber: 1,
createdByAgentId: agentId,
updatedByAgentId: agentId,
createdAt,
updatedAt: createdAt,
});
await db.insert(documentRevisions).values({
id: revisionId,
companyId,
documentId,
revisionNumber: 1,
title: "Plan",
format: "markdown",
body: "# Plan\n\n- Inspect files",
createdByAgentId: agentId,
createdByRunId: otherRunId,
createdAt,
});
await db.insert(issueDocuments).values({
companyId,
issueId,
documentId,
key: "plan",
createdAt,
updatedAt: createdAt,
});
const service = activityService(db);
const { run: backfilledRun } = await waitForIssueRun(
service,
companyId,
issueId,
(entry) => entry.runId === runId && entry.livenessState === "plan_only",
);
expect(backfilledRun).toMatchObject({
runId,
livenessState: "plan_only",
livenessReason: "Run described runnable future work without concrete action evidence",
lastUsefulActionAt: null,
});
});
it("does not treat continuation summary revisions as concrete backfill evidence", async () => {
const companyId = randomUUID();
const agentId = randomUUID();
const issueId = randomUUID();
const runId = randomUUID();
const documentId = randomUUID();
const revisionId = randomUUID();
const createdAt = new Date("2026-04-18T20:12:00.000Z");
await db.insert(companies).values({
id: companyId,
name: "Paperclip",
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
await db.insert(agents).values({
id: agentId,
companyId,
name: "CodexCoder",
role: "engineer",
status: "idle",
adapterType: "codex_local",
adapterConfig: {},
runtimeConfig: {},
permissions: {},
});
await db.insert(issues).values({
id: issueId,
companyId,
title: "Fix run ledger",
description: "Make the run ledger answer whether a run advanced.",
status: "in_progress",
priority: "medium",
assigneeAgentId: agentId,
});
await db.insert(heartbeatRuns).values({
id: runId,
companyId,
agentId,
scopeKind: "issue",
issueId,
invocationSource: "assignment",
status: "succeeded",
startedAt: new Date("2026-04-18T20:10:00.000Z"),
finishedAt: createdAt,
contextSnapshot: { issueId },
resultJson: {
summary: "Next steps:\n- inspect files",
},
livenessState: null,
livenessReason: null,
});
await db.insert(documents).values({
id: documentId,
companyId,
title: "Continuation Summary",
format: "markdown",
latestBody: "# Continuation Summary",
latestRevisionId: revisionId,
latestRevisionNumber: 1,
createdByAgentId: agentId,
updatedByAgentId: agentId,
createdAt,
updatedAt: createdAt,
});
await db.insert(documentRevisions).values({
id: revisionId,
companyId,
documentId,
revisionNumber: 1,
title: "Continuation Summary",
format: "markdown",
body: "# Continuation Summary",
createdByAgentId: agentId,
createdByRunId: runId,
createdAt,
});
await db.insert(issueDocuments).values({
companyId,
issueId,
documentId,
key: ISSUE_CONTINUATION_SUMMARY_DOCUMENT_KEY,
createdAt,
updatedAt: createdAt,
});
const service = activityService(db);
const { run: backfilledRun } = await waitForIssueRun(
service,
companyId,
issueId,
(entry) => entry.runId === runId && entry.livenessState === "plan_only",
);
expect(backfilledRun).toMatchObject({
runId,
livenessState: "plan_only",
livenessReason: "Run described runnable future work without concrete action evidence",
lastUsefulActionAt: null,
});
});
describe("run liveness backfill", () => {
async function seedIssue() {
const companyId = randomUUID();
const agentId = randomUUID();
const issueId = randomUUID();
await db.insert(companies).values({ id: companyId, name: "Backfill", issuePrefix: `B${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}` });
await db.insert(agents).values({ id: agentId, companyId, name: "Coder", role: "engineer", status: "idle", adapterType: "codex_local" });
await db.insert(issues).values({ id: issueId, companyId, title: "Backfill target", status: "done", assigneeAgentId: agentId });
return { companyId, agentId, issueId };
}
async function insertMissingRun(companyId: string, agentId: string, contextSnapshot: Record<string, unknown>, status = "succeeded") {
const runId = randomUUID();
await db.insert(heartbeatRuns).values({ id: runId, companyId, agentId, status, contextSnapshot, livenessState: null });
return runId;
}
async function livenessOf(runIds: string[]) {
const rows = await db.select({ id: heartbeatRuns.id, livenessState: heartbeatRuns.livenessState }).from(heartbeatRuns);
return runIds.map((id) => rows.find((row) => row.id === id)?.livenessState ?? null);
}
async function waitForLiveness(service: ActivityService, companyId: string, issueId: string, runIds: string[]) {
const deadline = performance.now() + 3_000;
while (performance.now() < deadline) {
await service.runsForIssue(companyId, issueId);
if ((await livenessOf(runIds)).every((state) => state !== null)) {
// Let the pass finish its bookkeeping after the last update.
await new Promise((resolve) => setTimeout(resolve, 50));
return;
}
await new Promise((resolve) => setTimeout(resolve, 25));
}
throw new Error(`Timed out waiting for liveness: ${JSON.stringify(await livenessOf(runIds))}`);
}
it("backfills only this issue's runs, found by context or by issue activity", async () => {
const { companyId, agentId, issueId } = await seedIssue();
const viaContext = await insertMissingRun(companyId, agentId, { issueId });
const viaActivity = await insertMissingRun(companyId, agentId, {}, "failed");
await db.insert(activityLog).values({ companyId, actorType: "agent", actorId: agentId, action: "issue.updated",
entityType: "issue", entityId: issueId, runId: viaActivity });
const running = await insertMissingRun(companyId, agentId, { issueId }, "running");
const otherIssueRun = await insertMissingRun(companyId, agentId, { issueId: randomUUID() });
const other = await seedIssue();
const otherCompanyRun = await insertMissingRun(other.companyId, other.agentId, { issueId });
await waitForLiveness(activityService(db), companyId, issueId, [viaContext, viaActivity]);
expect(await livenessOf([running, otherIssueRun, otherCompanyRun])).toEqual([null, null, null]);
});
it("does not repeat the lookup for a clean issue until the skip window ends", async () => {
const { companyId, agentId, issueId } = await seedIssue();
const first = await insertMissingRun(companyId, agentId, { issueId });
const service = activityService(db);
await waitForLiveness(service, companyId, issueId, [first]);
const late = await insertMissingRun(companyId, agentId, { issueId });
for (let read = 0; read < 3; read++) {
await service.runsForIssue(companyId, issueId);
await new Promise((resolve) => setTimeout(resolve, 50));
}
expect(await livenessOf([late])).toEqual([null]);
const now = Date.now();
const clock = vi.spyOn(Date, "now").mockReturnValue(now + 11 * 60_000);
try {
await waitForLiveness(service, companyId, issueId, [late]);
} finally {
clock.mockRestore();
}
});
it("keeps backfilling on later reads while an issue has more than one batch", async () => {
const { companyId, agentId, issueId } = await seedIssue();
const runIds: string[] = [];
for (let index = 0; index < 25; index++) runIds.push(await insertMissingRun(companyId, agentId, { issueId }));
await waitForLiveness(activityService(db), companyId, issueId, runIds);
});
it("leaves the backfill to the task page when a read skips it", async () => {
const { companyId, agentId, issueId } = await seedIssue();
const runId = await insertMissingRun(companyId, agentId, { issueId });
const service = activityService(db);
await service.runsForIssue(companyId, issueId, null, 2, undefined, { skipLivenessBackfill: true });
await new Promise((resolve) => setTimeout(resolve, 100));
expect(await livenessOf([runId])).toEqual([null]);
await waitForLiveness(service, companyId, issueId, [runId]);
});
});
});