mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-18 14:35:55 +01:00
fix(restart-chaos): exempt a continuation of an uncommitted follow-up run from completion_once (#22137)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart-chaos harness grades each task after deploy-like restarts, and the `completion_once` rule finds tasks that an agent completed twice > - A follow-up comment can arrive while a task runs. The run that answers it can complete the task again, and the grader exempts that run > - The grader did not exempt a recovery continuation of that answering run when the answering run failed before it committed a completion > - So the grader reported a duplicate completion for correct product behavior, and the wave headline counted a false duplicated side effect and an extra turn > - This pull request makes a continuation (a run with `retryOfRunId`) of an answering run that never committed a completion count as the answering run > - The benefit is that wave results show real duplicates only, and a real second completion after a committed one still fails ## Linked Issues or Issue Description **What happened?** In a restart-chaos staging-mix wave, task t128 (opencode, fault K13) graded `completion_once` as failed with the detail "duplicate completion: runs 38a8772f, 833314ed each reported completion". The wake evidence shows this sequence: - Run 38a8772f completed the task (accepted disposition `done`) and posted the DONE comment. - The follow-up comment arrived before that completion. Its first two runs (5707e93a and 2c1d364e) failed before a provider turn, so they did not receive it. - Run c5f9a5a4 received the follow-up and started a provider turn. It then failed with `native_session_retry_exhausted` and did not commit a completion. - Execution recovery started run 833314ed with `retryOfRunId` = c5f9a5a4 (wake reason `issue_recovery_action_restored`). That run applied the follow-up and committed `done`. The grader marked only c5f9a5a4 as the answering run. Run 833314ed was a continuation of it, but the grader counted it as a second completion and as an extra turn. **Expected behavior** A continuation of the answering run must count as the answering run when the answering run did not commit a completion. A continuation that completes again after the answering run committed its completion must still fail `completion_once`. **Steps to reproduce** 1. Use the t128 wake evidence shape: a first run that completes, two follow-up runs that fail before a provider turn, an answering run that fails without a completion, and a recovery run with `retryOfRunId` set to the answering run that completes. 2. Grade it with `gradeStagingMix` through `evidenceCompletion` or `completionEvidence`. 3. Before this change, `completion_once` fails with "duplicate completion: runs 38a8772f, 833314ed each reported completion". ## What Changed - `tests/restart-chaos/follow-up-delivery.ts`: add `continuedRunId` (reads `retryOfRunId` from the run context or the wake payload) and `followUpAnsweringRuns`. The new function returns the runs that answered a follow-up, plus each continuation of an answering run that did not commit a `done` result. - `tests/restart-chaos/staging-mix-retry.ts`: `evidenceCompletion` and `runsAfterDone` use `followUpAnsweringRuns` to set `answersFollowUp` and to separate answered runs from extra turns. `followUpsAnsweredLater` and the wait statistics are not changed, so one follow-up still counts as one answer. - `tests/restart-chaos/wake-evidence.ts`: `completionEvidence` (the record that the wave grade uses) uses the same function. - `tests/restart-chaos/staging-mix-retry.test.ts`: add a t128 test group from the real wave evidence. One test shows the continuation passes. One test shows a continuation after a committed answering completion still fails. ## Verification - Red before the fix: `passes when the continuation of an answering run that never committed completes the follow-up` failed (`completionRunIds` returned both 38a8772f and 833314ed). - Green after the fix: both new tests pass, and all existing truth-table and follow-up tests pass. - `npx vitest run --config tests/restart-chaos/vitest.config.ts`: 71 files, 844 tests passed. - `tsc -p tests/restart-chaos/tsconfig.json --noEmit`: passed. - I regraded the real t128 wake evidence and comments with the fixed functions. The result is one completer (38a8772f), no extra turns, and answered runs c5f9a5a4 and 833314ed. - `pnpm verify` exited 2 because `follow-up-delivery.ts` is a global CI input. Required CI supplies the broad coverage. ## Risks - Low risk. The change is in the restart-chaos test harness only. It does not change product code. - The exemption needs an explicit `retryOfRunId` link to the answering run. A recovery run with no link (for example the t005 case) still fails, as before. - If the answering run committed `done`, its continuation is not exempt, so a real second completion still fails. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used in Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
9e1a85fb46
commit
069ec8a74e
4 files changed
+192
-8
No files matched your search
@@ -6,6 +6,7 @@ export interface DeliveryRun {
|
||||
startedAt?: number | null;
|
||||
wakeupRequestId: string | null;
|
||||
contextSnapshot?: unknown;
|
||||
retryOfRunId?: string | null;
|
||||
providerTurnStarted?: boolean | null;
|
||||
}
|
||||
|
||||
@@ -71,6 +72,44 @@ export function undeliveredFollowUpRuns(evidence: DeliveryEvidence): Set<string>
|
||||
return followUpDeliveries(evidence).undelivered;
|
||||
}
|
||||
|
||||
export function continuedRunIds(run: Pick<DeliveryRun, "contextSnapshot" | "retryOfRunId">, wake: DeliveryWake | undefined): string[] {
|
||||
const context = object(run.contextSnapshot);
|
||||
const payload = object(wake?.payload);
|
||||
return [...new Set([run.retryOfRunId, context.retryOfRunId, context.previousRunId, object(context.executionContinuation).interruptedRunId,
|
||||
object(payload._paperclipWakeContext).retryOfRunId, payload.retryOfRunId]
|
||||
.filter((value): value is string => typeof value === "string" && value.length > 0))];
|
||||
}
|
||||
|
||||
export function committedCompletion(run: { runId: string | null; acceptedDisposition?: string | null } | undefined, rejected: ReadonlySet<string>): boolean {
|
||||
return run?.acceptedDisposition === "done" && !(run.runId && rejected.has(run.runId));
|
||||
}
|
||||
|
||||
export function followUpAnsweringRuns(evidence: { runs?: ReadonlyArray<DeliveryRun & { acceptedDisposition?: string | null }>;
|
||||
wakes?: readonly DeliveryWake[] } | null, rejected: ReadonlySet<string>): Set<string> {
|
||||
const answering = new Set(laterAnswers(evidence).keys());
|
||||
if (!evidence) return answering;
|
||||
const wakes = new Map((evidence.wakes ?? []).map(wake => [wake.id, wake]));
|
||||
const runs = (evidence.runs ?? []).filter((run): run is typeof run & { runId: string } => Boolean(run.runId));
|
||||
const byId = new Map(runs.map(run => [run.runId, run]));
|
||||
const lineage = new Map(runs.map(run => [run.runId, continuedRunIds(run, run.wakeupRequestId ? wakes.get(run.wakeupRequestId) : undefined)]));
|
||||
const committedAncestor = (runId: string): boolean => {
|
||||
const visited = new Set<string>([runId]);
|
||||
const pending = [...(lineage.get(runId) ?? [])];
|
||||
while (pending.length) {
|
||||
const id = pending.pop()!;
|
||||
if (visited.has(id)) continue;
|
||||
visited.add(id);
|
||||
if (committedCompletion(byId.get(id), rejected)) return true;
|
||||
pending.push(...(lineage.get(id) ?? []));
|
||||
}
|
||||
return false;
|
||||
};
|
||||
for (const run of [...runs].sort((a, b) => (a.createdAt ?? 0) - (b.createdAt ?? 0))) {
|
||||
if ((lineage.get(run.runId) ?? []).some(id => answering.has(id)) && !committedAncestor(run.runId)) answering.add(run.runId);
|
||||
}
|
||||
return answering;
|
||||
}
|
||||
|
||||
export const COMPLETION_INCOMPLETE_KEY = "native-completion-incomplete";
|
||||
|
||||
export function rejectedDoneRuns(evidence: { runs?: ReadonlyArray<{ runId: string | null; acceptedDisposition?: string | null; acceptedAt?: number | null }>;
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
} from "./staging-mix-retry.js";
|
||||
import type { OracleArtifacts, TaskPlan, TaskRecord, TaskSample } from "./types.js";
|
||||
import { completionEvidence } from "./wake-evidence.js";
|
||||
import { COMPLETION_INCOMPLETE_KEY } from "./follow-up-delivery.js";
|
||||
|
||||
const wave = (argv: string[]) => {
|
||||
const command = parseCommand(argv);
|
||||
@@ -554,3 +555,147 @@ describe("a follow-up whose run failed before any provider turn (w-20261114T2314
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("a recovery continuation of a follow-up run that never committed (w-20261117T170216-s7100 t128)", () => {
|
||||
const nonce = "ed987e41";
|
||||
const comment = "956f5926-1621-4868-9d03-ca9a307b36e9";
|
||||
const first = "38a8772f-33f6-4291-bdd6-1f16494aa6ed", promoted = "5707e93a-05dc-4b7f-a7d3-ec5c25a17367", leaseRetry = "2c1d364e-6c46-423e-84f3-8a80e89a7cb5";
|
||||
const answering = "c5f9a5a4-bf76-42d3-a26d-771fe3a3d176", continuation = "833314ed-bd6c-4e41-9992-78629d065816";
|
||||
const t128 = (answeringDisposition: string | null = null): WakeEvidenceFile => ({
|
||||
completedAt: 1_794_946_774_445,
|
||||
runs: [{ runId: first, createdAt: 1_794_941_952_144, startedAt: 1_794_942_615_951, invocationSource: "assignment", triggerDetail: "system",
|
||||
wakeupRequestId: "55a6afa9", acceptedDisposition: "done", acceptedAt: 1_794_943_908_538, finishedAt: 1_794_943_909_767, status: "succeeded", providerTurnStarted: true,
|
||||
contextSnapshot: { wakeReason: "issue_assigned" } },
|
||||
{ runId: promoted, createdAt: 1_794_943_909_966, startedAt: 1_794_944_484_183, invocationSource: "automation", triggerDetail: "system",
|
||||
wakeupRequestId: "be67ffb7", acceptedDisposition: null, acceptedAt: null, finishedAt: 1_794_944_484_330, status: "failed", providerTurnStarted: false,
|
||||
contextSnapshot: { wakeReason: "issue_commented", wakeCommentIds: [comment], wakeCommentId: comment } },
|
||||
{ runId: leaseRetry, createdAt: 1_794_944_484_384, startedAt: 1_794_944_511_264, invocationSource: "automation", triggerDetail: "system",
|
||||
wakeupRequestId: "549cf01c", acceptedDisposition: null, acceptedAt: null, finishedAt: 1_794_944_511_437, status: "failed", providerTurnStarted: false,
|
||||
contextSnapshot: { wakeReason: "transient_failure_retry", wakeCommentIds: [comment], wakeCommentId: comment, retryOfRunId: promoted } },
|
||||
{ runId: answering, createdAt: 1_794_944_511_476, startedAt: 1_794_944_580_608, invocationSource: "automation", triggerDetail: "system",
|
||||
wakeupRequestId: "08e16978", acceptedDisposition: answeringDisposition, acceptedAt: answeringDisposition ? 1_794_945_390_000 : null, finishedAt: 1_794_945_394_924,
|
||||
status: answeringDisposition ? "succeeded" : "failed", providerTurnStarted: true,
|
||||
contextSnapshot: { wakeReason: "transient_failure_retry", wakeCommentIds: [comment], wakeCommentId: comment, retryOfRunId: leaseRetry } },
|
||||
{ runId: continuation, createdAt: 1_794_945_559_524, startedAt: 1_794_946_114_761, invocationSource: "automation", triggerDetail: "system",
|
||||
wakeupRequestId: "88d1a3e0", acceptedDisposition: "done", acceptedAt: 1_794_946_772_351, finishedAt: 1_794_946_774_505, status: "succeeded", providerTurnStarted: true,
|
||||
contextSnapshot: { wakeReason: "issue_recovery_action_restored", retryOfRunId: answering } }],
|
||||
wakes: [{ id: "55a6afa9", reason: "issue_assigned", source: "assignment", triggerDetail: "system", requestedByActorType: "user", requestedByActorId: "local-board",
|
||||
createdAt: 1_794_941_952_144 },
|
||||
{ id: "be67ffb7", reason: "issue_execution_promoted", source: "automation", triggerDetail: "system", requestedByActorType: "user", requestedByActorId: "local-board",
|
||||
createdAt: 1_794_943_909_966, payload: { commentId: comment, _paperclipWakeContext: { wakeCommentIds: [comment] } } },
|
||||
{ id: "549cf01c", reason: "transient_failure_retry", source: "automation", triggerDetail: "system", requestedByActorType: "system", requestedByActorId: null,
|
||||
createdAt: 1_794_944_484_384, payload: { retryReason: "transient_failure", retryOfRunId: promoted } },
|
||||
{ id: "08e16978", reason: "transient_failure_retry", source: "automation", triggerDetail: "system", requestedByActorType: "system", requestedByActorId: null,
|
||||
createdAt: 1_794_944_511_476, payload: { retryReason: "transient_failure", retryOfRunId: leaseRetry } },
|
||||
{ id: "88d1a3e0", reason: "issue_recovery_action_restored", source: "automation", triggerDetail: "system", requestedByActorType: "system",
|
||||
requestedByActorId: "execution-recovery", createdAt: 1_794_945_559_524,
|
||||
payload: { recoveryActionId: "8d383ed2", _paperclipWakeContext: { wakeReason: "issue_recovery_action_restored", retryOfRunId: answering } } }],
|
||||
});
|
||||
const posted = commentEvidence([
|
||||
{ body: `The ledger document already contains all 26 rounds plus the \`FOLLOWUP-${nonce}\` line on its own line (revision 2). The \`${doneMarker(nonce)}\` comment was previously posted.`,
|
||||
authorAgentId: "a", createdByRunId: continuation, createdAt: "2026-11-17T20:19:34.669Z" },
|
||||
{ body: `Task complete. All 26 rounds ran and were recorded, the ledger document was saved, and the \`${doneMarker(nonce)}\` comment was posted.`,
|
||||
authorAgentId: "a", createdByRunId: first, createdAt: "2026-11-17T19:31:49.960Z" },
|
||||
{ body: doneMarker(nonce), authorAgentId: "a", createdByRunId: first, createdAt: "2026-11-17T19:31:41.204Z" }]);
|
||||
const completionOnce = (completion: CompletionEvidence) => gradeStagingMix(nonce,
|
||||
{ turnMinutes: 39, rounds: 4, subtasks: 0, followUpAfterMs: 1_102_604, arriveAtMs: 0, holdMs: 600_000 },
|
||||
{ issueStatus: "done", documents: { ledger: `${ledger(nonce, 4)}\nFOLLOWUP-${nonce}` }, agentComments: [], files: {}, commands: {}, workspacePath: null,
|
||||
stagingMix: { followUp: { at: 1_794_943_656_111, commentId: comment, duringRun: true, inOverlap: false, error: null }, followUpErrors: [], retries: [], comments: posted,
|
||||
completion } }).checks.find(check => check.name === "completion_once")!;
|
||||
const both = (evidence: WakeEvidenceFile) => [evidenceCompletion(evidence)!, completionEvidence({ completedAt: evidence.completedAt!, wakes: evidence.wakes,
|
||||
runs: evidence.runs!.map(run => ({ ...run, acceptedDisposition: run.acceptedDisposition ?? null, acceptedAt: run.acceptedAt ?? null })) })];
|
||||
|
||||
it("passes when the continuation of an answering run that never committed completes the follow-up", () => {
|
||||
const evidence = t128();
|
||||
for (const completion of both(evidence)) {
|
||||
const anchor = completedAtAnchor(completion, posted, nonce);
|
||||
expect(anchor).toEqual({ at: 1_794_943_908_538, source: "accepted_result" });
|
||||
expect(completionRunIds(posted, nonce, completion)).toEqual([first]);
|
||||
const after = runsAfterDone("t128", evidence, anchor);
|
||||
expect(after.started).toEqual([]);
|
||||
expect(after.answeredFollowUp.map(turn => turn.runId)).toEqual([answering, continuation]);
|
||||
expect(completionOnce(completion)).toMatchObject({ ok: true, detail: "runs=1" });
|
||||
}
|
||||
});
|
||||
|
||||
const withLineage = (lineage: { context?: Record<string, unknown>; retryOfRunId?: string }): WakeEvidenceFile => {
|
||||
const evidence = t128();
|
||||
evidence.runs = evidence.runs!.map(run => run.runId === continuation
|
||||
? { ...run, retryOfRunId: lineage.retryOfRunId ?? null, contextSnapshot: { wakeReason: "issue_recovery_action_restored", ...lineage.context } } : run);
|
||||
evidence.wakes = evidence.wakes!.map(wake => wake.id === "88d1a3e0" ? { ...wake, payload: { recoveryActionId: "8d383ed2" } } : wake);
|
||||
return evidence;
|
||||
};
|
||||
|
||||
it.each([
|
||||
{ name: "only previousRunId", lineage: { context: { previousRunId: answering } } },
|
||||
{ name: "only executionContinuation.interruptedRunId", lineage: { context: { executionContinuation: { interruptedRunId: answering } } } },
|
||||
{ name: "only the retry_of_run_id column", lineage: { retryOfRunId: answering } },
|
||||
{ name: "a retry_of_run_id column that names a sibling and a previousRunId that names the answering run",
|
||||
lineage: { retryOfRunId: leaseRetry, context: { previousRunId: answering } } },
|
||||
])("passes when the continuation carries $name", ({ lineage }) => {
|
||||
const evidence = withLineage(lineage);
|
||||
for (const completion of both(evidence)) {
|
||||
const anchor = completedAtAnchor(completion, posted, nonce);
|
||||
expect(runsAfterDone("t128", evidence, anchor).started).toEqual([]);
|
||||
expect(completionOnce(completion)).toMatchObject({ ok: true, detail: "runs=1" });
|
||||
}
|
||||
});
|
||||
|
||||
it("still fails a retry of a committed continuation that keeps a stale previousRunId to the answering run", () => {
|
||||
const retry = "9c9c9c9c-retry";
|
||||
const evidence = t128();
|
||||
evidence.runs!.push({ runId: retry, createdAt: 1_794_946_800_000, startedAt: 1_794_946_810_000, invocationSource: "automation", triggerDetail: "system",
|
||||
wakeupRequestId: "w-retry", retryOfRunId: continuation, acceptedDisposition: "done", acceptedAt: 1_794_946_900_000, status: "succeeded", providerTurnStarted: true,
|
||||
contextSnapshot: { wakeReason: "transient_failure_retry", previousRunId: answering, retryOfRunId: continuation } });
|
||||
evidence.wakes!.push({ id: "w-retry", reason: "transient_failure_retry", source: "automation", triggerDetail: "system", requestedByActorType: "system",
|
||||
requestedByActorId: null, createdAt: 1_794_946_800_000, payload: { retryOfRunId: continuation } });
|
||||
for (const completion of both(evidence)) {
|
||||
const anchor = completedAtAnchor(completion, posted, nonce);
|
||||
expect(runsAfterDone("t128", evidence, anchor).started.map(turn => turn.runId)).toEqual([retry]);
|
||||
expect(completionOnce(completion)).toMatchObject({ ok: false, detail: "duplicate completion: runs 38a8772f, 9c9c9c9c each reported completion" });
|
||||
}
|
||||
});
|
||||
|
||||
it("still fails a later continuation whose committed ancestor is reachable only through its lineage chain", () => {
|
||||
const failedRetry = "7c7c7c7c-failed", finalRetry = "9d9d9d9d-final";
|
||||
const evidence = withLineage({ context: { previousRunId: answering } });
|
||||
evidence.runs!.push({ runId: failedRetry, createdAt: 1_794_946_800_000, startedAt: 1_794_946_810_000, invocationSource: "automation", triggerDetail: "system",
|
||||
wakeupRequestId: null, retryOfRunId: continuation, acceptedDisposition: null, acceptedAt: null, status: "failed", providerTurnStarted: true,
|
||||
contextSnapshot: { wakeReason: "transient_failure_retry", retryOfRunId: continuation } },
|
||||
{ runId: finalRetry, createdAt: 1_794_946_900_000, startedAt: 1_794_946_910_000, invocationSource: "automation", triggerDetail: "system",
|
||||
wakeupRequestId: null, retryOfRunId: failedRetry, acceptedDisposition: "done", acceptedAt: 1_794_947_000_000, status: "succeeded", providerTurnStarted: true,
|
||||
contextSnapshot: { wakeReason: "transient_failure_retry", retryOfRunId: failedRetry, previousRunId: answering } });
|
||||
for (const completion of both(evidence)) {
|
||||
const anchor = completedAtAnchor(completion, posted, nonce);
|
||||
expect(runsAfterDone("t128", evidence, anchor).started.map(turn => turn.runId)).toEqual([failedRetry, finalRetry]);
|
||||
expect(completionOnce(completion)).toMatchObject({ ok: false, detail: "duplicate completion: runs 38a8772f, 9d9d9d9d each reported completion" });
|
||||
}
|
||||
});
|
||||
|
||||
it("still fails a continuation with no lineage to the answering run", () => {
|
||||
const evidence = withLineage({});
|
||||
for (const completion of both(evidence)) {
|
||||
expect(completionOnce(completion)).toMatchObject({ ok: false, detail: "duplicate completion: runs 38a8772f, 833314ed each reported completion" });
|
||||
}
|
||||
});
|
||||
|
||||
it("passes when the answering run's done result was rejected as incomplete", () => {
|
||||
const evidence = t128("done");
|
||||
evidence.wakes!.push({ id: "incomplete-wake", reason: "issue_continuation_needed", source: "automation", triggerDetail: "system", requestedByActorType: "system",
|
||||
requestedByActorId: null, createdAt: 1_794_945_400_000, payload: { continuationIdempotencyKey: COMPLETION_INCOMPLETE_KEY } });
|
||||
for (const completion of both(evidence)) {
|
||||
const anchor = completedAtAnchor(completion, posted, nonce);
|
||||
expect(runsAfterDone("t128", evidence, anchor).started).toEqual([]);
|
||||
expect(completionOnce(completion)).toMatchObject({ ok: true, detail: "runs=1" });
|
||||
}
|
||||
});
|
||||
|
||||
it("still fails a continuation that completes again after the answering run committed its completion", () => {
|
||||
const evidence = t128("done");
|
||||
for (const completion of both(evidence)) {
|
||||
const anchor = completedAtAnchor(completion, posted, nonce);
|
||||
expect(runsAfterDone("t128", evidence, anchor).started.map(turn => turn.runId)).toEqual([continuation]);
|
||||
expect(completionOnce(completion)).toMatchObject({ ok: false, detail: "duplicate completion: runs 38a8772f, 833314ed each reported completion" });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -6,7 +6,7 @@ import {
|
||||
bannerShownMs, completedAtAnchor, doneCommentsBeforeCompletion, DUPLICATE_CHECKS, firstDoneCommentAt, noteRetryBanner, type CompletionEvidence,
|
||||
} from "./staging-mix.js";
|
||||
import type { InterruptionSample, OracleResult, TaskPlan, TaskRecord, TaskSample } from "./types.js";
|
||||
import { laterAnswers, laterAnswerWaits, rejectedDoneRuns, undeliveredFollowUpRuns, type LaterAnswer } from "./follow-up-delivery.js";
|
||||
import { followUpAnsweringRuns, laterAnswers, laterAnswerWaits, rejectedDoneRuns, undeliveredFollowUpRuns, type LaterAnswer } from "./follow-up-delivery.js";
|
||||
|
||||
type Json = Record<string, any>;
|
||||
|
||||
@@ -192,7 +192,7 @@ export interface ExtraTurn {
|
||||
export interface WakeEvidenceFile {
|
||||
completedAt?: number | null;
|
||||
runs?: Array<{ runId: string | null; createdAt: number | null; startedAt?: number | null; invocationSource: string | null; triggerDetail: string | null;
|
||||
wakeupRequestId: string | null; acceptedDisposition?: string | null; acceptedAt?: number | null; finishedAt?: number | null; contextSnapshot?: unknown; status?: string | null;
|
||||
wakeupRequestId: string | null; retryOfRunId?: string | null; acceptedDisposition?: string | null; acceptedAt?: number | null; finishedAt?: number | null; contextSnapshot?: unknown; status?: string | null;
|
||||
providerTurnStarted?: boolean | null }>;
|
||||
wakes?: Array<{ id: string | null; reason: string | null; source: string | null; triggerDetail: string | null; requestedByActorType: string | null; requestedByActorId: string | null;
|
||||
createdAt?: number | null; payload?: unknown }>;
|
||||
@@ -212,8 +212,8 @@ export function commentEvidence(rows: readonly CommentRow[] | null): Array<{ bod
|
||||
export function evidenceCompletion(evidence: WakeEvidenceFile | null, issueCompletedAt?: unknown): CompletionEvidence | null {
|
||||
const fromIssue = typeof issueCompletedAt === "string" && Number.isFinite(Date.parse(issueCompletedAt)) ? Date.parse(issueCompletedAt) : null;
|
||||
if (!evidence && fromIssue === null) return null;
|
||||
const answers = laterAnswers(evidence);
|
||||
const rejected = rejectedDoneRuns(evidence);
|
||||
const answers = followUpAnsweringRuns(evidence, rejected);
|
||||
return { completedAt: evidence?.completedAt ?? fromIssue, runs: (evidence?.runs ?? []).map(run => ({ runId: run.runId, startedAt: run.startedAt ?? null,
|
||||
acceptedDisposition: run.acceptedDisposition ?? null, acceptedAt: run.acceptedAt ?? null, answersFollowUp: Boolean(run.runId && answers.has(run.runId)),
|
||||
doneRejected: Boolean(run.runId && rejected.has(run.runId)) })) };
|
||||
@@ -223,7 +223,7 @@ export function runsAfterDone(taskId: string, evidence: WakeEvidenceFile | null,
|
||||
{ started: ExtraTurn[]; neverStarted: ExtraTurn[]; answeredFollowUp: ExtraTurn[] } {
|
||||
const doneAt = anchor.at;
|
||||
if (!evidence || doneAt === null) return { started: [], neverStarted: [], answeredFollowUp: [] };
|
||||
const answers = laterAnswers(evidence);
|
||||
const answers = followUpAnsweringRuns(evidence, rejectedDoneRuns(evidence));
|
||||
const wakes = new Map((evidence.wakes ?? []).map(wake => [wake.id, wake]));
|
||||
const turn = (run: NonNullable<WakeEvidenceFile["runs"]>[number]): ExtraTurn => {
|
||||
const wake = run.wakeupRequestId ? wakes.get(run.wakeupRequestId) : undefined;
|
||||
|
||||
@@ -4,7 +4,7 @@ import type { ServerProcessWindow } from "./multi-server.js";
|
||||
import { unshiftJson, unshiftMs, type AgingShift } from "./time-faults.js";
|
||||
import { ISSUE_STATUSES } from "../../packages/shared/src/constants.js";
|
||||
import type { CompletionEvidence } from "./staging-mix.js";
|
||||
import { laterAnswers, rejectedDoneRuns, type DeliveryRun, type DeliveryWake } from "./follow-up-delivery.js";
|
||||
import { followUpAnsweringRuns, rejectedDoneRuns, type DeliveryRun, type DeliveryWake } from "./follow-up-delivery.js";
|
||||
import { AGENT_SLOTS_FILE, agentSlotsFromRows } from "./agent-slots.js";
|
||||
import { agents, createDb, type Db } from "../../packages/db/src/index.js";
|
||||
import { maxConcurrentRunsSql } from "../../server/src/services/heartbeat/timer-policy.js";
|
||||
@@ -78,7 +78,7 @@ export function wakeEvidenceForIssue(issueId: string, input: WakeEvidenceInput)
|
||||
runs: runs.map(run => {
|
||||
const owner = controllerByBoot.get(String(run.controllerBootId ?? ""));
|
||||
return { runId: text(run.id), status: text(run.status), invocationSource: text(run.invocationSource), triggerDetail: text(run.triggerDetail),
|
||||
wakeupRequestId: text(run.wakeupRequestId), createdAt: ms(run.createdAt), startedAt: ms(run.startedAt), finishedAt: ms(run.finishedAt),
|
||||
wakeupRequestId: text(run.wakeupRequestId), retryOfRunId: text(run.retryOfRunId), createdAt: ms(run.createdAt), startedAt: ms(run.startedAt), finishedAt: ms(run.finishedAt),
|
||||
acceptedDisposition: text(run.acceptedDisposition), acceptedAt: ms(run.acceptedAt), contextSnapshot: run.contextSnapshot ?? null,
|
||||
providerTurnStarted: run.providerTurnStarted == null ? null : Boolean(run.providerTurnStarted), owner: owner ? { pid: owner.pid, bootId: owner.bootId, epoch: owner.epoch, processSpace: owner.processSpace } : null,
|
||||
servingAtCreate: servingAt(ms(run.createdAt), controllers, input.windows) };
|
||||
@@ -97,7 +97,7 @@ async function readRows(db: Pick<Db, "$client" | "select">, companyId: string, s
|
||||
const client = db.$client;
|
||||
const runs = await client.unsafe(
|
||||
`select r.id::text as id, r.agent_id::text as "agentId", r.issue_id::text as "issueId", r.status, r.invocation_source as "invocationSource", r.trigger_detail as "triggerDetail",
|
||||
r.wakeup_request_id::text as "wakeupRequestId", r.context_snapshot as "contextSnapshot", r.controller_boot_id::text as "controllerBootId",
|
||||
r.wakeup_request_id::text as "wakeupRequestId", r.retry_of_run_id::text as "retryOfRunId", r.context_snapshot as "contextSnapshot", r.controller_boot_id::text as "controllerBootId",
|
||||
floor(extract(epoch from r.created_at) * 1000) as "createdAt", floor(extract(epoch from r.started_at) * 1000) as "startedAt",
|
||||
floor(extract(epoch from r.finished_at) * 1000) as "finishedAt", accepted.disposition as "acceptedDisposition",
|
||||
floor(extract(epoch from accepted.accepted_at) * 1000) as "acceptedAt",
|
||||
@@ -170,8 +170,8 @@ export interface EvidenceObserver {
|
||||
|
||||
export function completionEvidence(evidence: { completedAt: number | null; wakes?: readonly DeliveryWake[];
|
||||
runs: ReadonlyArray<DeliveryRun & { acceptedDisposition: string | null; acceptedAt: number | null }> }): CompletionEvidence {
|
||||
const answers = laterAnswers(evidence);
|
||||
const rejected = rejectedDoneRuns(evidence);
|
||||
const answers = followUpAnsweringRuns(evidence, rejected);
|
||||
return { completedAt: evidence.completedAt, runs: evidence.runs.map(run => ({ runId: run.runId, startedAt: run.startedAt ?? null,
|
||||
acceptedDisposition: run.acceptedDisposition, acceptedAt: run.acceptedAt, answersFollowUp: Boolean(run.runId && answers.has(run.runId)),
|
||||
doneRejected: Boolean(run.runId && rejected.has(run.runId)) })) };
|
||||
|
||||
Reference in new issue
Block a user