test(restart-chaos): add a result trigger and recovered-run comment-wake guards (#19689)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The restart chaos harness kills the control plane at chosen moments
and grades whether each task still finishes correctly
> - One wave lost a user's follow-up comment after a kill that landed
after the run accepted its result but before the run settled
> - No existing trigger aims a fault at that moment, so the case was
hard to reproduce on purpose
> - The recovery path that settles such a run had no regression test
that covers the saved comment
> - This pull request adds a `result` trigger to the harness and a
regression test for the recovery path
> - The benefit is that the harness can aim faults at the
result-accepted window, and a change that drops, repeats, or re-wakes a
saved comment fails a test

## Linked Issues or Issue Description

**What happened?**
In a restart-chaos storm wave, a user posted a comment while a native
run was active. A hard kill (K1) landed after the run accepted its
result and before the run settled. The run settled `succeeded` and the
task stayed `done`, but no follow-up run read the comment.

**Steps to reproduce**
The original failure did not reproduce. Three targeted waves ran 12
tasks with K1 at comment time or in the new result window. Every
follow-up comment was consumed exactly once. The new regression test
also passes on current `master`. The test and the trigger guard the path
and do not claim a fix.

**Expected behavior**
A comment posted during a run is consumed exactly once after the run
ends, including when restart recovery settles the run.

## What Changed

- `tests/restart-chaos`: add `--triggers result` (C1 only). K1 fires 0–3
s after the run that received the follow-up proposes or accepts its
result (`run.result.proposed` or `run.result.accepted`). The fault fires
only while that run is still `running`: the observer checks the latest
sample and rereads the runs right before dispatch. A run that settled
during the delay, or a follow-up posted at rest, records a `result`
miss. Add poll-driven unit tests for these cases and a README row.
- `server/src/__tests__/native-run-projection-work.test.ts`: add
embedded Postgres tests in which background recovery, not the executor,
settles the run (projection, environment settlement, and
deferred-receipt workers), with a deferred user comment and an active
local lease left by the killed executor. The tests check:
- The comment is promoted into exactly one follow-up run, and a second
restart adds none.
- A comment the run already received as a same-turn steer is not woken
again. The other saved comment still is.
- A queue whose only comment was steered stays cancelled, and no
follow-up starts.
- A Retry click does not consume the comment a second time. A click that
observed the source is stale. A click that observed the follow-up joins
it.

## Verification

- `cd tests/restart-chaos && npx vitest run --config vitest.config.ts`
(96 tests pass)
- `cd server && npx vitest run
src/__tests__/native-run-projection-work.test.ts` (54 tests pass)
- Live, local, ACPX Claude, Codex and OpenCode: `node
cli/node_modules/tsx/dist/cli.mjs tests/restart-chaos/main.ts --env
local --workloads C1 --triggers comment --faults K1 --adapters
codex,claude,opencode --tasks 4 --concurrency 2 --rc06-masked` passed 4
of 4. Two waves with `--triggers result` consumed each follow-up exactly
once. One task in those waves found a separate defect: a kill 1.4 s
after the result commit left the run `running` behind the dead
controller's 20-minute finalization lease. A separate pull request fixes
that defect.

## Risks

- Low risk. These are test and harness changes only. No production code
changes.
- The `result` trigger depends on the observer's poll interval (about 3
s), so the exact kill point within the window varies.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, run as an agent in
Claude Code with tool use (shell, file editing, local test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-06 19:53:58 -06:00
1 parent 5e4d24b6c8
commit d26372dc8a
6 files changed
+244 -7

No files matched your search

@@ -11,7 +11,7 @@ import { join } from "node:path";
import { setTimeout as delay } from "node:timers/promises";
import { eq, sql } from "drizzle-orm";
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { agentWakeupRequests, agents, companies, completionContracts, createDb, heartbeatRuns, issues, nativeRunFinalizations, nativeRunResults, workspaceOperations, type Db } from "@paperclipai/db";
import { agentWakeupRequests, agents, authUsers, companies, companyMemberships, completionContracts, createDb, environmentLeases, environments, heartbeatRuns, issueComments, issues, nativeRunFinalizations, nativeRunResults, workspaceOperations, type Db } from "@paperclipai/db";
import { createDeliveryWorkCoordinator } from "../services/delivery-work-coordinator.js";
import { nativeRunProjectionWork } from "../services/native-runtime/native-run-projection-work.js";
import { projectCommittedRun } from "../services/native-runtime/native-run-projection.js";
@@ -22,6 +22,9 @@ import * as statusCommitter from "../services/native-runtime/status-decision-com
import { PaperclipControlPlanePort } from "../services/native-runtime/paperclip-control-plane-port.js";
import { NativeRunCoordinatorStore } from "../services/native-runtime/native-run-coordinator-store.js";
import { workspaceOperationService } from "../services/workspace-operations.js";
import { environmentService } from "../services/environments.js";
import { withQueuedCommentIdsInWakePayload } from "../services/issue-queued-comment-queue.js";
import { requestTaskRetry } from "../services/task-retry.js";
import { idleWorkSnapshot } from "../services/task-admission.js";
import { CONTROL_PLANE_CONFORMANCE_RESULT, CONTROL_PLANE_CONFORMANCE_TERMINAL } from "../vendor/paperclip-runner/testing.js";
import { startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
@@ -652,4 +655,147 @@ describe("committed native run projection lifecycle", () => {
skipNativeResponsePresentation: true, skipNativeReviewResponse: true, onWorkspaceSettled: settled });
expect(settled).toHaveBeenCalledTimes(1);
});
describe("a comment deferred behind a run whose projection is left to restart recovery", () => {
const commenter = "projection-comment-user";
let busyRunId: string | undefined;
afterEach(async () => {
if (busyRunId) await db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date() }).where(eq(heartbeatRuns.id, busyRunId));
await db.update(agents).set({ runtimeConfig: {} }).where(eq(agents.id, agentId));
busyRunId = undefined;
});
async function deferredComment(binding: Awaited<ReturnType<typeof seed>>, bodies = ["Also include the FOLLOWUP line."]) {
await db.insert(authUsers).values({ id: commenter, name: "Commenter", email: `${commenter}@example.test`,
createdAt: new Date(), updatedAt: new Date() }).onConflictDoNothing();
await db.insert(companyMemberships).values({ companyId, principalType: "user", principalId: commenter,
status: "active", membershipRole: "operator" }).onConflictDoNothing();
// One provider slot is held by another task, so a promoted follow-up stays
// queued here instead of starting a provider.
await db.update(agents).set({ runtimeConfig: { heartbeat: { maxConcurrentRuns: 1 } } }).where(eq(agents.id, agentId));
[{ id: busyRunId }] = await db.insert(heartbeatRuns).values({ companyId, agentId, status: "running",
contextSnapshot: { issueId: randomUUID() } }).returning({ id: heartbeatRuns.id });
const comments = await db.insert(issueComments).values(bodies.map(body => ({ companyId, issueId: binding.issueId,
authorUserId: commenter, body }))).returning();
const ids = comments.map(comment => comment.id), latest = ids[ids.length - 1];
// The durable receipt the comment route writes while the run is active.
const [wake] = await db.insert(agentWakeupRequests).values({ companyId, agentId, source: "automation",
triggerDetail: "system", reason: "issue_commented", status: "deferred_issue_execution",
requestedByActorType: "user", requestedByActorId: commenter,
payload: { issueId: binding.issueId, commentId: latest, _paperclipWakeContext: {
issueId: binding.issueId, taskId: binding.issueId, commentId: latest, wakeCommentId: latest,
wakeCommentIds: ids, wakeReason: "issue_commented", source: "issue.comment" } },
}).returning();
// The killed executor never reached its cleanup, so its local lease is still active.
const environmentId = (await db.select({ id: environments.id }).from(environments).where(eq(environments.driver, "local")))[0]?.id
?? (await db.insert(environments).values({ name: "Local", driver: "local" }).returning())[0].id;
const lease = await environmentService(db).acquireLease({ companyId, environmentId, heartbeatRunId: binding.runId,
issueId: binding.issueId, leasePolicy: "ephemeral", provider: "local" });
return { comment: comments[0], comments, wake, lease };
}
// The durable writes of an acknowledged same-turn steer (the queued-comment
// steer route): the message leaves the saved queue and the live run records
// that its turn received it.
async function steer(binding: Awaited<ReturnType<typeof seed>>, wakeId: string, commentId: string) {
const wake = await readWake(wakeId);
const remaining = (wake.payload as { _paperclipWakeContext: { wakeCommentIds: string[] } })
._paperclipWakeContext.wakeCommentIds.filter(id => id !== commentId);
await db.update(agentWakeupRequests).set({ payload: withQueuedCommentIdsInWakePayload(wake.payload, remaining) })
.where(eq(agentWakeupRequests.id, wakeId));
const run = await read(binding.runId);
await db.update(heartbeatRuns).set({ resultJson: { ...(run.resultJson ?? {}), queuedSteeringAcknowledgements: {
[commentId]: { status: "acknowledged", queueId: wakeId, turnId: "steered-turn", acknowledgedAt: new Date().toISOString() } } } })
.where(eq(heartbeatRuns.id, binding.runId));
}
async function startRecovered() {
coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => admitted, canReconcile: () => admitted,
onError: error => errors.push(error) });
const heartbeat = heartbeatService(db);
worker = heartbeat.registerNativeRunProjectionWork(coordinator, () => admitted);
const receipts = heartbeat.registerDeferredReceiptWork(coordinator, () => admitted);
const settlement = heartbeat.registerNativeEnvironmentSettlementWork(coordinator, () => admitted);
await Promise.all([worker.ready, receipts.ready, settlement.ready]);
}
const followUps = async (issueId: string, sourceRunId: string) => (await db.select().from(heartbeatRuns)
.where(sql`${heartbeatRuns.contextSnapshot}->>'issueId' = ${issueId}`)).filter(run => run.id !== sourceRunId);
it("promotes the comment exactly once after the recovered run is projected terminal", async () => {
const binding = await seed(true);
const { comment, wake, lease } = await deferredComment(binding);
await complete(binding, "control_plane"); await barrier(binding);
// Startup recovery runs while the killed run is still `running`: the
// comment correctly stays deferred behind it.
await startRecovered(); await flush();
expect(await readWake(wake.id)).toMatchObject({ status: "deferred_issue_execution", runId: null });
// The accepted result commits, but its executor died before projecting the
// run. The background projection worker settles the run instead.
await finalizeNativeRun({ db, runId: binding.runId, workspaceFinalizeStatus: "succeeded" });
await flush();
expect((await read(binding.runId)).status).toBe("succeeded");
expect((await db.select().from(environmentLeases).where(eq(environmentLeases.id, lease.id)))[0].status).not.toBe("active");
const promoted = await readWake(wake.id);
expect(promoted).toMatchObject({ status: "queued", runId: expect.any(String) });
const [followUp, ...extra] = await followUps(binding.issueId, binding.runId);
expect(extra).toEqual([]);
expect(followUp).toMatchObject({ id: promoted.runId, status: "queued" });
expect(followUp.contextSnapshot?.wakeCommentIds).toEqual([comment.id]);
// Another restart re-derives nothing new: the comment is not consumed twice.
await coordinator!.stop(); await startRecovered(); await flush();
await heartbeatService(db).resumeQueuedRuns();
expect((await followUps(binding.issueId, binding.runId)).map(run => run.id)).toEqual([followUp.id]);
});
it("does not wake again for a comment the recovered run already received as a steer", async () => {
const binding = await seed(true);
const { comments: [steered, saved], wake } = await deferredComment(binding,
["Steer this into the live turn.", "Also include the FOLLOWUP line."]);
await steer(binding, wake.id, steered.id);
await complete(binding, "control_plane"); await barrier(binding);
await startRecovered(); await flush();
await finalizeNativeRun({ db, runId: binding.runId, workspaceFinalizeStatus: "succeeded" });
await flush();
const [followUp, ...extra] = await followUps(binding.issueId, binding.runId);
expect(extra).toEqual([]);
expect(followUp.contextSnapshot?.wakeCommentIds).toEqual([saved.id]);
expect((await read(binding.runId)).resultJson?.queuedSteeringAcknowledgements).toMatchObject({
[steered.id]: { status: "acknowledged" } });
});
it("cancels the saved queue and starts no follow-up when its only comment was steered", async () => {
const binding = await seed(true);
const { comment, wake } = await deferredComment(binding);
// The steer route cancels a queue whose final message was delivered.
await steer(binding, wake.id, comment.id);
await db.update(agentWakeupRequests).set({ status: "cancelled", finishedAt: new Date() })
.where(eq(agentWakeupRequests.id, wake.id));
await complete(binding, "control_plane"); await barrier(binding);
await startRecovered(); await flush();
await finalizeNativeRun({ db, runId: binding.runId, workspaceFinalizeStatus: "succeeded" });
await flush(); await heartbeatService(db).resumeQueuedRuns();
expect((await read(binding.runId)).status).toBe("succeeded");
expect(await followUps(binding.issueId, binding.runId)).toEqual([]);
expect(await readWake(wake.id)).toMatchObject({ status: "cancelled", runId: null });
});
it("joins a Retry click to the promoted follow-up instead of consuming the comment twice", async () => {
const binding = await seed(true);
const { comment, wake } = await deferredComment(binding);
await complete(binding, "control_plane"); await barrier(binding);
await startRecovered(); await flush();
await finalizeNativeRun({ db, runId: binding.runId, workspaceFinalizeStatus: "succeeded" });
await flush();
const [followUp] = await followUps(binding.issueId, binding.runId);
expect(followUp).toMatchObject({ id: (await readWake(wake.id)).runId, status: "queued" });
// A click that observed the recovered source is stale; one that observed
// the follow-up joins it. Neither admits another run with the comment.
expect(await requestTaskRetry({ db, companyId, issueId: binding.issueId, actorId: commenter,
idempotencyKey: randomUUID(), observedRunId: binding.runId })).toMatchObject({ accepted: false, stale: true });
expect(await requestTaskRetry({ db, companyId, issueId: binding.issueId, actorId: commenter,
idempotencyKey: randomUUID(), observedRunId: followUp.id })).toMatchObject({ accepted: true });
await flush(); await heartbeatService(db).resumeQueuedRuns();
const runs = await followUps(binding.issueId, binding.runId);
expect(runs.map(run => run.id)).toEqual([followUp.id]);
expect(runs.filter(run => (run.contextSnapshot?.wakeCommentIds as string[] | undefined)?.includes(comment.id)))
.toHaveLength(1);
});
});
});
+2 -1
View File
@@ -21,7 +21,7 @@ pnpm exec vitest run --config tests/restart-chaos/vitest.config.ts # credential
node scripts/typecheck-test-projects.mjs tests/restart-chaos # typecheck
```
Options: `--triggers time,comment,retry,queued,tool` (fault timing, below),
Options: `--triggers time,comment,retry,queued,tool,result` (fault timing, below),
`--workloads A1,B1,B5,C1`, `--budget-cents` (company hard cap,
default 3000), `--poll-ms` (2000–5000), `--keep` (keep the temporary instance).
@@ -137,6 +137,7 @@ the observer, after a seeded delay:
| Trigger | Fires | Plan |
| --- | --- | --- |
| `comment` | 0–5 s after the C1 follow-up was posted to a live run (C1 only) | C1 + K1, B8 |
| `result` | 0–3 s after the run that received the C1 follow-up proposed or accepted its result, only while that run is still `running` (C1 only) | C1 + K1, B8 |
| `retry` | 0–2 s after an accepted Retry click, as an extra K1 after the planned fault | K8 |
| `queued` | on the first poll whose run is still `queued` with no start | B1 |
| `tool` | 15–60 s after `bench.sh` progress first appears (B5 only) | B6 |
+29
View File
@@ -39,6 +39,8 @@ function stillQueued(runs: readonly RunSample[]): boolean {
return runs.some(run => run.status === "queued" && !run.startedAt) && !runs.some(run => run.startedAt || run.status === "running");
}
export const TRIGGER_LATE_MS = 10_000;
/** The run settles soon after acceptance; arming on the proposal keeps the window reachable. */
const RESULT_TRIGGER_EVENTS = new Set(["run.result.proposed", "run.result.accepted"]);
/** Rest must hold for this long before the observer acts on it. */
export const REST_CONFIRM_MS = 12_000;
@@ -160,6 +162,7 @@ export class TaskObserver {
private triggerWindowOpen(fault: ScheduledFault, now: number, dueAt: number): boolean {
if (now - dueAt > TRIGGER_LATE_MS) return false;
if (fault.trigger === "result") return this.resultRunStillRunning(this.latestReachableRuns());
if (fault.trigger !== "queued") return true;
const latest = [...this.record.samples].reverse().find(sample => sample.reachable);
return stillQueued(latest?.runs ?? []);
@@ -169,6 +172,11 @@ export class TaskObserver {
if (!fault.trigger || fault.trigger === "time") return true;
const armedAt = this.triggerArmedAt(fault);
let open = typeof armedAt === "number" && now - (armedAt + (fault.delayMs ?? 0)) <= TRIGGER_LATE_MS;
if (open && fault.trigger === "result") {
const runs = await this.api.get<Json[]>(`/api/issues/${this.record.issueId}/runs`, 5_000)
.then(rows => (rows ?? []).map(toRunSample)).catch(() => null);
open = runs !== null && this.resultRunStillRunning(runs);
}
if (open && fault.trigger === "queued") {
const runs = await this.api.get<Json[]>(`/api/issues/${this.record.issueId}/runs`, 5_000)
.then(rows => (rows ?? []).map(toRunSample)).catch(() => null);
@@ -178,6 +186,15 @@ export class TaskObserver {
return open;
}
private latestReachableRuns(): RunSample[] {
return [...this.record.samples].reverse().find(sample => sample.reachable)?.runs ?? [];
}
/** A `result` fault outside the accepted-but-unsettled window would test a different boundary. */
private resultRunStillRunning(runs: readonly RunSample[]): boolean {
return runs.some(run => run.id === this.record.resultRunId && run.status === "running");
}
private noteTriggerMissed(fault: ScheduledFault) {
const trigger = fault.trigger ?? "time";
(this.record.triggerMissed ??= []).push(trigger);
@@ -206,6 +223,14 @@ export class TaskObserver {
return queued && !started ? first.at : "missed";
}
case "tool": return this.record.benchFirstSeenAt ?? null;
case "result": {
// B8: the run that received the follow-up has accepted its result but
// has not settled yet. A follow-up posted at rest cannot cross it.
const followUp = this.record.followUp;
if (!followUp) return null;
if (!followUp.commentId || !followUp.duringRun) return "missed";
return this.record.resultAcceptedAt ?? null;
}
}
}
@@ -294,6 +319,10 @@ export class TaskObserver {
if (!events?.length) break;
for (const event of events) {
if (typeof event.seq === "number" && event.seq > cursor.seq) cursor.seq = event.seq;
if (RESULT_TRIGGER_EVENTS.has(String(event.eventType)) && this.record.followUp?.duringRun && !this.record.resultAcceptedAt) {
this.record.resultAcceptedAt = Date.now();
this.record.resultRunId = run.id;
}
cursor.lastAt = latestIso([cursor.lastAt, event.createdAt]);
}
if (events.length < RUN_EVENT_PAGE) break;
+3 -2
View File
@@ -34,10 +34,11 @@ export const TRIGGER_DELAY_RANGE_MS: Record<Exclude<TriggerId, "time">, readonly
retry: [0, 2_000],
queued: [0, 0],
tool: [15_000, 60_000],
result: [0, 3_000],
};
export function triggerFitsWorkload(trigger: TriggerId, workload: WorkloadId): boolean {
if (trigger === "comment") return workload === "C1";
if (trigger === "comment" || trigger === "result") return workload === "C1";
if (trigger === "tool") return workload === "B5";
return true;
}
@@ -54,7 +55,7 @@ export function planTasks(options: Pick<WaveOptions, "seed" | "tasks" | "workloa
if (options.stormMinutes && triggers.some(trigger => trigger !== "time")) throw new Error("A K13 storm fires on its own clock; use --triggers time");
for (const trigger of triggers) {
if (!options.workloads.some(workload => triggerFitsWorkload(trigger, workload))) {
throw new Error(`Trigger "${trigger}" needs workload ${trigger === "comment" ? "C1" : "B5"} in --workloads`);
throw new Error(`Trigger "${trigger}" needs workload ${trigger === "comment" || trigger === "result" ? "C1" : "B5"} in --workloads`);
}
}
const cells: Array<{ workload: WorkloadId; adapter: AdapterId; fault: TaskFaultKind; trigger: TriggerId }> = [];
+58 -2
View File
@@ -25,9 +25,15 @@ const options: WaveOptions = { env: "local", adapters: ["codex", "claude", "open
triggers: ["time", "comment", "retry", "queued", "tool"] };
type Json = Record<string, unknown>;
function scriptedApi(state: { runs: Json[]; issue: Json; retry?: Json }) {
function scriptedApi(state: { runs: Json[]; issue: Json; retry?: Json; events?: Json[] }) {
const api = {
async get(route: string) { return route.endsWith("/runs") ? state.runs : route.endsWith("/comments") ? [] : state.issue; },
async get(route: string) {
if (route.includes("/events")) {
const after = Number(new URL(route, "http://x").searchParams.get("afterSeq") ?? 0);
return (state.events ?? []).filter(event => Number(event.seq) > after);
}
return route.endsWith("/runs") ? state.runs : route.endsWith("/comments") ? [] : state.issue;
},
async post(route: string) {
if (route.endsWith("/issues")) return { id: "i1", identifier: "CHA-1" };
if (route.endsWith("/retry")) return state.retry ?? { accepted: true };
@@ -198,6 +204,56 @@ describe("trigger firing", () => {
expect(observer.faultDue(start + 50_000)).toMatchObject({ trigger: "tool" });
});
it("result: arms from the target run's result event and fires only while that run is running", async () => {
const running = { runId: "r1", status: "running", startedAt: new Date().toISOString() };
const state = { runs: [running] as Json[], issue: { id: "i1", status: "in_progress" },
events: [{ seq: 1, eventType: "turn.started", createdAt: new Date().toISOString() }] as Json[] };
const observer = new TaskObserver({ ...base, trigger: "result", triggerDelayMs: 2_000 }, scriptedApi(state), lab, new Timeline(null), ready);
await observer.launch();
const start = observer.record.createdAt!;
observer.record.followUp = { at: start, postedAt: start, commentId: "c1", error: null, duringRun: true };
await observer.poll(start + 1_000);
// An unrelated event does not arm the trigger.
expect(observer.record.resultAcceptedAt).toBeUndefined();
state.events.push({ seq: 2, eventType: "run.result.accepted", createdAt: new Date().toISOString() });
await observer.poll(start + 2_000);
const armedAt = observer.record.resultAcceptedAt!;
expect(observer.record.resultRunId).toBe("r1");
expect(observer.faultDue(armedAt + 1_999)).toBeNull();
const fault = observer.faultDue(armedAt + 2_000);
expect(fault).toMatchObject({ trigger: "result" });
expect(await observer.confirmTrigger(fault!, armedAt + 2_000)).toBe(true);
// The run settled before dispatch: the fault would land outside the window.
state.runs = [{ ...running, status: "succeeded", finishedAt: new Date().toISOString() }];
expect(await observer.confirmTrigger(fault!, armedAt + 2_000)).toBe(false);
expect(observer.record.triggerMissed).toEqual(["result"]);
});
it("result: records a miss when the target run settled during the delay", async () => {
const state = { runs: [{ runId: "r1", status: "running" }] as Json[], issue: { id: "i1", status: "in_progress" },
events: [{ seq: 1, eventType: "run.result.proposed", createdAt: new Date().toISOString() }] as Json[] };
const observer = new TaskObserver({ ...base, trigger: "result", triggerDelayMs: 3_000 }, scriptedApi(state), lab, new Timeline(null), ready);
await observer.launch();
const start = observer.record.createdAt!;
observer.record.followUp = { at: start, postedAt: start, commentId: "c1", error: null, duringRun: true };
await observer.poll(start + 1_000);
const armedAt = observer.record.resultAcceptedAt!;
state.runs = [{ runId: "r1", status: "succeeded" }];
await observer.poll(armedAt + 1_000);
expect(observer.faultDue(armedAt + 3_000)).toBeNull();
expect(observer.record.triggerMissed).toEqual(["result"]);
});
it("result: misses when the follow-up was posted at rest", async () => {
const observer = new TaskObserver({ ...base, trigger: "result", triggerDelayMs: 0 },
scriptedApi({ runs: [], issue: { id: "i1", status: "done" } }), lab, new Timeline(null), ready);
await observer.launch();
const start = observer.record.createdAt!;
observer.record.followUp = { at: start, postedAt: start, commentId: "c1", error: null, duringRun: false };
expect(observer.faultDue(start + 1)).toBeNull();
expect(observer.record.triggerMissed).toEqual(["result"]);
});
it("records a miss instead of firing a triggered fault after its window has passed", async () => {
const observer = new TaskObserver({ ...base, workload: "B5", trigger: "tool", triggerDelayMs: 1_000 },
scriptedApi({ runs: [], issue: { id: "i1", status: "in_progress" } }), lab, new Timeline(null), ready);
+5 -1
View File
@@ -11,7 +11,7 @@ export type WorkloadId = (typeof WORKLOAD_IDS)[number];
export const ADAPTER_IDS = ["codex", "claude", "opencode"] as const;
export type AdapterId = (typeof ADAPTER_IDS)[number];
export const TRIGGER_IDS = ["time", "comment", "retry", "queued", "tool"] as const;
export const TRIGGER_IDS = ["time", "comment", "retry", "queued", "tool", "result"] as const;
export type TriggerId = (typeof TRIGGER_IDS)[number];
export const ENVIRONMENT_KINDS = ["local", "daytona"] as const;
@@ -227,6 +227,10 @@ export interface TaskRecord {
/** B5: distinct bench progress values the observer read during polling. */
benchProgressSeen?: string[];
benchFirstSeenAt?: number | null;
/** C1: first time the observer saw the run that received the follow-up propose or accept its result. */
resultAcceptedAt?: number | null;
/** The run whose result armed the `result` trigger; the fault fires only while it is still running. */
resultRunId?: string | null;
triggerMissed?: TriggerId[];
}