mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-22 03:29:18 +01:00
fix: preserve warm Codex turns with incremental managed file checkpoints (#14735)
## Thinking Path > - Paperclip manages AI agents and keeps their instructions and files durable. > - Native Codex runners can keep a process alive between compatible turns. > - Managed file collection stopped that process after each turn, which defeated warm reuse. > - Agent folders can contain large images and other files, so full copies on every turn are expensive. > - This change keeps one managed directory for the live session and saves only file changes after each turn. > - Ownership, authorization, instruction changes, and process retirement still control when reuse is safe. ## Linked Issues or Issue Description Related: #13710 introduced native warm session reuse. This fixes managed file collection that still forced those sessions to stop. No duplicate open PR or issue was found. **What happened?** With managed instructions and warm native Codex enabled, consecutive turns reused a Daytona sandbox but started a new runner process each time. The managed directory collector required process termination before saving files. **Expected behavior** Compatible turns keep the same process and managed `AGENT_HOME`. Each completed turn saves added, changed, and deleted files before the next turn starts. Unchanged large files do not transfer again. **Steps to reproduce** 1. Use a native Codex agent with managed instructions and a reusable Daytona environment. 2. Enable warm session reuse and run three turns on the same task. 3. Write a large binary on the first turn, edit a small note on each turn, and delete a file on the second turn. 4. Compare process identity across turns and read the canonical files through the public agent-files API. **Paperclip version or commit** Reproduced on `d30b03bd8c17604cdab1533eeeeb087aba30e8b1`. **Deployment mode** Local server with remote Daytona execution; cloud native runner uses the same path. ## What Changed - Retain the managed directory only for the verified owner of a live native Codex session. - Checkpoint each completed turn before releasing the session for reuse. Retry unstable captures, then stop and collect when a warm checkpoint cannot be validated. - Compare metadata and cached hashes, stream only changed file payloads, record deletions, and validate path, content, quota, and authorization before saving. - Rotate sessions when canonical files, loaded instructions, credentials, or launch policy change. Fence stale collection and cleanup callbacks from later owners. - Keep cleanup and recovery aware of the current session owner. Recheck canonical files under the writer lock at handoff, attach the successor collector before fallible bookkeeping, and emit one final save receipt on checkpoint fallback. Preserve storage warnings across unchanged checkpoints. - Add regression coverage and a three-turn Daytona test with independent public API file checks, an unchanged 8 MiB binary, deletion checks, and strict process identity checks. - Document checkpoint consistency, lifecycle behavior, and local run-log counters. - Replace a timing assumption in the Daytona teardown test with explicit transfer-arrival gates after CI exposed an unset release callback. ## Verification - Full local `pnpm -r typecheck` and `pnpm build` passed. Server checks were repeated after the final storage-warning fix. - Runner E2E typecheck and 749 runner E2E unit tests passed. - Focused file checkpoint, directory ownership, instruction collection, native session, and merge tests passed. After review fixes, the managed-directory and native-session suites passed 550 tests, including intervening canonical edits, same-run fresh restore, failed handoff collection, and one-call fallback collection. Server typecheck passed again. The Daytona plugin suite passed 218 tests. The quota-warning regression failed before the fix and passed afterward. - Three real Daytona campaigns passed before the final handoff review fixes. The latest kept PID 547 across all three turns. The first checkpoint copied 8,388,635 bytes; the next two copied 36 and 54 bytes. Public API reads verified the binary, note contents, and deletion after every turn. Test cleanup deleted the sandbox. - The final head was also deployed to an isolated cloud staging instance and passed three UI-triggered native Codex turns with managed instructions. All three retained the same process ID/start time, native session, provider session, runner instance, and Daytona sandbox. Checkpoints copied 8,388,643 bytes on turn 1, then only 52 and 78 bytes on turns 2 and 3; those warm captures also hashed only 52 and 78 bytes. Independent canonical API reads verified every byte of the unchanged 8 MiB binary and the exact note contents after every turn; the deleted file returned 404 after turns 2 and 3. After restoring the original lifecycle and agent-auth configuration, removing the temporary secret, pausing the test agent, and deleting both test sandboxes, independent canonical API reads still verified the entire binary, the final 78-byte three-line note, and the deletion. The native runner flag remained enabled and the final serving revision remained the PR head. - Two earlier staging attempts are preserved as failures and are excluded from the acceptance result: a saved ChatGPT login failed with a provider routing 401, and its subsequent stopped-sandbox retry failed before provider startup with a closed-lease admission error. The successful campaign used a fresh sandbox and a temporary encrypted API-key binding. The stopped-lease retry remains unexplained; this campaign does not establish recovery of that failed sandbox. - All [Paperclip CI gates](https://github.com/paperclipai/paperclip/actions/runs/36750397355) pass on `26ef2ef56a389259246809805c0b34a4747eb86b`, including full test partitions, build, typecheck, runner verification, E2E shards, and the Canary clean public-npm install. Greptile reviewed that exact head at 5/5 with no unresolved review threads or outstanding findings. - Full local repository coverage used the existing CI partitions, but the 40,000-file Git streaming stress test timed out and its local retry was interrupted by macOS thermal emergency sleep; this is not a green full local suite claim. The exact stress test passed on the final head in [CI server shard 2/12](https://github.com/paperclipai/paperclip/actions/runs/36750397355/job/110008294290), in 111.9 seconds. - Repeat the live test with configured credentials and a Linux runner artifact: `pnpm test:e2e:runner -- --id daytona-warm-continuity.runner-codex.daytona.warm-three-turn`. ## Risks - This is a file-level checkpoint, not an atomic snapshot of the whole folder. Background writes after a capture are saved by the next checkpoint or final stopped collection. - Metadata scans still visit all paths. Modified files transfer in full; unchanged files do not rehash or transfer. - Incorrect ownership or reuse could collect the wrong directory. Run ownership fences, current authorization, stable capture validation, and stopped collection fallbacks are covered by tests. - Warm reuse remains opt-in. No database migration or fleet default changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code editing, tool use, and test execution. The exact serving model ID and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
d432dc7fa3
commit
b54b2dc35c
23 files changed
+1195
-92
No files matched your search
@@ -108,7 +108,13 @@ canonical Plan revision, capture its pending UI, approve in the browser, and
|
||||
prove exactly two successful runs. `warm_three_turn` provides exactly two
|
||||
browser follow-up messages, preserves one project/execution-workspace scope,
|
||||
verifies host file contents after every turn, and finishes within three
|
||||
ten-minute turn deadlines. Native turns 1 and 2 include an actionable human review in the completion report's `attentionRequests`. Paperclip creates the review gate from that report. An explicit question-tool wait yields the turn and suppresses its final prose, so it is not interchangeable with this completion-review fixture. Turn 3 reports Done without another review.
|
||||
ten-minute turn deadlines. The ordinary warm fixture uses managed instructions,
|
||||
updates AGENT_HOME each turn, and verifies memory, an unchanged 8 MiB binary and
|
||||
a deletion through public file APIs. Native turns 2 and 3 must copy/hash only the
|
||||
changed memory file, with a saved receipt and the same provider PID. Journal and
|
||||
Git stress fixtures retain fixed external bundles as controls. Keep the stable-PID
|
||||
oracle strict; `instruction-persistence` also covers cold restarts and quota handling.
|
||||
Native turns 1 and 2 include an actionable human review in the completion report's `attentionRequests`. Paperclip creates the review gate from that report. An explicit question-tool wait yields the turn and suppresses its final prose, so it is not interchangeable with this completion-review fixture. Turn 3 reports Done without another review.
|
||||
|
||||
Every selected case runs in its own isolated Paperclip process, and independent
|
||||
cases may run concurrently. Follow-up turns inside one case retain their shared
|
||||
|
||||
@@ -274,9 +274,8 @@ a warning while full and clear it after cleanup. Rejected bytes must not replace
|
||||
the saved file. This adds at most one 256 MiB saved fixture per isolated agent.
|
||||
The deadline is twenty minutes per cell, with six expected provider runs;
|
||||
normal instance/Daytona cleanup, screenshots, evidence, and billing apply. Run with
|
||||
`pnpm test:e2e:runner -- --suite instruction-persistence`. Managed agent directories
|
||||
checkpoint and close the provider before collection while retaining conversation
|
||||
state. The separate `daytona-warm-continuity` suite covers warm runtime behavior.
|
||||
`pnpm test:e2e:runner -- --suite instruction-persistence`. Managed directories in per-turn sessions collect after provider stop. The separate
|
||||
`daytona-warm-continuity` suite covers incremental saves while retaining a live native process.
|
||||
|
||||
`daytona-warm-continuity` (**Daytona Warm Continuity**) is exactly two paid
|
||||
cells: legacy Codex and Runner Codex against one reusable warm Daytona
|
||||
@@ -286,7 +285,12 @@ three browser-driven turns on one issue. Every turn reads and extends the same
|
||||
nonce file, verifies host copy-back, records scheduler/run/end-to-end timing,
|
||||
and asserts `created`, `resumed`, `resumed` lease acquisition on one sandbox.
|
||||
Runner Codex additionally proves stable native session, provider session,
|
||||
runner instance, PID, and process-start identity. Each turn is bounded to ten
|
||||
runner instance, PID, and process-start identity. The ordinary warm cell uses
|
||||
managed instructions and edits AGENT_HOME on every turn: a growing memory file,
|
||||
an unchanged 8 MiB binary, and a deletion. Public API reads independently verify
|
||||
the canonical bytes after every turn. Native checkpoint receipts must show only
|
||||
the changed memory file transferred on turns 2 and 3; the PID oracle remains strict.
|
||||
The journal stress cell retains fixed external instructions as a control. Each warm turn is bounded to ten
|
||||
minutes, the cell to thirty minutes, and cleanup explicitly deletes the
|
||||
sandbox rather than waiting for Daytona's idle timeout.
|
||||
|
||||
@@ -310,10 +314,9 @@ over the agent setting. Large copyback plus the next preparation
|
||||
can exceed the normal five-minute idle window; the PID and process-fingerprint
|
||||
continuity checks remain strict. The ordinary warm-continuity cell keeps its
|
||||
existing five-minute policy.
|
||||
This cell uses a fixed external instruction bundle. Managed agent folders
|
||||
intentionally checkpoint and stop the provider after every turn for file
|
||||
collection; external instructions allow this cell to test retained-process
|
||||
continuity without changing that collection policy.
|
||||
This Git stress cell uses a fixed external instruction bundle to isolate workspace
|
||||
transfer from managed agent-file persistence. The ordinary warm cell separately
|
||||
requires incremental managed-file checkpoints and the same strict process continuity.
|
||||
It seeds an empty local Git project, creates 60,000 small untracked files through
|
||||
the real provider, then performs the same three browser-driven review turns.
|
||||
Each later turn updates all 60,000 generated files to distinct turn-specific
|
||||
|
||||
@@ -32,6 +32,29 @@ import {
|
||||
} from "./selectors.js";
|
||||
|
||||
describe("runner E2E catalog", () => {
|
||||
it.each(["daytona-journal-continuity"])(
|
||||
"%s retains native processes with fixed external instructions",
|
||||
(suiteId) => {
|
||||
const suite = runnerSuites.find(suite => suite.id === suiteId)!;
|
||||
const input = {
|
||||
executionId: "warm-instructions", environmentId: "env-1",
|
||||
environmentFixtureId: "daytona" as const, workspacePath: "/workspace",
|
||||
secretRefs: { OPENAI_API_KEY: { type: "secret_ref" as const, secretId: "22222222-2222-4222-8222-222222222222", version: "latest" as const } },
|
||||
};
|
||||
const profile = suite.profiles.find(profile => profile.id === "runner-codex")!;
|
||||
const config = profile.buildAgent(input).adapterConfig as Record<string, unknown>;
|
||||
expect(config).toMatchObject({
|
||||
instructionsBundleMode: "external", instructionsEntryFile: "AGENTS.md", idleTimeoutMs: 300_000,
|
||||
});
|
||||
expect(readFileSync(String(config.instructionsFilePath), "utf8")).toContain("Preserve the workspace files across turns");
|
||||
expect(suite.definitionMetadata).toMatchObject({ nativeInstructions: "fixed-external" });
|
||||
expect(runnerProfiles.find(profile => profile.id === "runner-codex")!.buildAgent(input).adapterConfig)
|
||||
.not.toHaveProperty("instructionsBundleMode", "external");
|
||||
const legacy = suite.profiles.find(profile => profile.id === "legacy-codex");
|
||||
if (legacy) expect(legacy.buildAgent(input).adapterConfig).not.toHaveProperty("instructionsBundleMode", "external");
|
||||
},
|
||||
);
|
||||
|
||||
it("keeps the large Git filename workload explicit-only with three real turns", () => {
|
||||
const suite = runnerSuites.find(suite => suite.id === "daytona-git-streaming")!;
|
||||
expect(suite.manualOnly).toBe(true);
|
||||
@@ -235,7 +258,9 @@ describe("runner E2E catalog", () => {
|
||||
expect(suite.profiles.map((profile) => profile.id)).toEqual(["runner-codex"]);
|
||||
expect(daytonaLargeJournalTask.flow).toBe("warm_three_turn");
|
||||
expect(daytonaLargeJournalTask.buildPrompt("nonce")).toContain("240 separate execution-tool calls");
|
||||
expect(daytonaLargeJournalTask.buildFollowupMessages!("nonce")).toEqual(daytonaWarmContinuityTask.buildFollowupMessages!("nonce"));
|
||||
expect(daytonaLargeJournalTask.buildFollowupMessages!("nonce")).toHaveLength(2);
|
||||
expect(daytonaLargeJournalTask.buildFollowupMessages!("nonce").join("\n")).not.toContain("notes/warm-memory.txt");
|
||||
expect(daytonaWarmContinuityTask.buildPrompt("nonce")).toContain("notes/warm-memory.txt");
|
||||
expect(selectRunnerExecutions(parseRunnerSelectors(["--all"]))
|
||||
.some((cell) => cell.suite.id === suite.id)).toBe(false);
|
||||
});
|
||||
|
||||
@@ -935,6 +935,16 @@ function warmTurnInstructions(turn: 1 | 2 | 3, nonce: string) {
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
function managedWarmTurnInstructions(turn: 1 | 2 | 3, nonce: string) {
|
||||
return [warmTurnInstructions(turn, nonce),
|
||||
"Also update your personal AGENT_HOME with ordinary filesystem tools. Do not edit the loaded AGENTS.md instructions and do not use an API to save these files.",
|
||||
turn === 1
|
||||
? `Create notes/warm-memory.txt containing exactly T1-${nonce} followed by a newline. Create notes/unchanged.bin with exactly 8388608 bytes, each byte equal to 93. Create notes/delete-me.txt containing temporary.`
|
||||
: `Read notes/warm-memory.txt under AGENT_HOME and verify it contains exactly the prior turn lines ${Array.from({ length: turn - 1 }, (_, i) => `T${i + 1}-${nonce}`).join(" | ")}, each followed by a newline. Append exactly T${turn}-${nonce} and a newline. Verify notes/unchanged.bin still has 8388608 bytes, each equal to 93, and leave it unchanged. ${turn === 2 ? "Delete notes/delete-me.txt." : "Verify notes/delete-me.txt is absent."}`,
|
||||
"Perform these personal-file edits and verification before calling paperclip_finish. Paperclip saves them at the turn boundary.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
export const daytonaWarmContinuityTask: RunnerTaskFixture = {
|
||||
id: "warm-three-turn",
|
||||
label: "Warm three-turn workspace continuity",
|
||||
@@ -947,10 +957,10 @@ export const daytonaWarmContinuityTask: RunnerTaskFixture = {
|
||||
expectedTerminalState: { issue: "done", run: "succeeded" },
|
||||
buildTitle: (nonce) => `Runner E2E warm Daytona continuity ${nonce}`,
|
||||
buildVisibleMarker: (nonce) => warmTurnMarker(3, nonce),
|
||||
buildPrompt: (nonce) => warmTurnInstructions(1, nonce),
|
||||
buildPrompt: (nonce) => managedWarmTurnInstructions(1, nonce),
|
||||
buildFollowupMessages: (nonce) => [
|
||||
warmTurnInstructions(2, nonce),
|
||||
warmTurnInstructions(3, nonce),
|
||||
managedWarmTurnInstructions(2, nonce),
|
||||
managedWarmTurnInstructions(3, nonce),
|
||||
],
|
||||
buildMatchers(nonce, execution) {
|
||||
// Workspace persistence is the oracle for this story. Exact response text
|
||||
@@ -980,6 +990,7 @@ export const daytonaLargeJournalTask: RunnerTaskFixture = {
|
||||
...daytonaWarmContinuityTask,
|
||||
id: "large-journal-three-turn",
|
||||
label: "Large journal three-turn workspace continuity",
|
||||
buildFollowupMessages: nonce => [warmTurnInstructions(2, nonce), warmTurnInstructions(3, nonce)],
|
||||
buildTitle: (nonce) => `Runner E2E large journal continuity ${nonce}`,
|
||||
buildPrompt: (nonce) => [
|
||||
"First exercise ordinary execution history with 240 separate execution-tool calls. In each call, run the Python command below exactly once. Issue the calls one by one. Do not combine them into a shell loop, script, parallel wrapper, or a single tool call: each command must be a separate ordinary execution-tool invocation. Keep a count from 1 through 240. The output is synthetic fixture data and needs no analysis.",
|
||||
@@ -988,12 +999,30 @@ export const daytonaLargeJournalTask: RunnerTaskFixture = {
|
||||
warmTurnInstructions(1, nonce),
|
||||
].join("\n"),
|
||||
};
|
||||
export const daytonaGitStreamingTask = createGitStreamingTask(daytonaWarmContinuityTask);
|
||||
export const daytonaGitStreamingTask = createGitStreamingTask({ ...daytonaWarmContinuityTask, buildPrompt: nonce => warmTurnInstructions(1, nonce), buildFollowupMessages: nonce => [warmTurnInstructions(2, nonce), warmTurnInstructions(3, nonce)] });
|
||||
|
||||
const codexContinuityProfiles = runnerProfiles.filter((profile) =>
|
||||
["legacy-codex", "runner-codex"].includes(profile.id),
|
||||
);
|
||||
|
||||
// The journal stress fixture keeps a fixed external bundle as a control.
|
||||
// Ordinary warm continuity exercises managed files and incremental checkpoints.
|
||||
const warmCodexContinuityProfiles = codexContinuityProfiles.map((profile) =>
|
||||
profile.generation !== "native" ? profile : {
|
||||
...profile,
|
||||
buildAgent(input: AgentFixtureBuildInput) {
|
||||
const agent = profile.buildAgent(input);
|
||||
return { ...agent, adapterConfig: {
|
||||
...(agent.adapterConfig as Record<string, unknown>),
|
||||
instructionsBundleMode: "external",
|
||||
instructionsRootPath: fileURLToPath(new URL("./fixtures/warm-continuity/", import.meta.url)),
|
||||
instructionsEntryFile: "AGENTS.md",
|
||||
instructionsFilePath: fileURLToPath(new URL("./fixtures/warm-continuity/AGENTS.md", import.meta.url)),
|
||||
} };
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
export const connectionReviewSuite: RunnerSuiteFixture = {
|
||||
id: "connection-reviews",
|
||||
label: "Governed Connection Reviews",
|
||||
@@ -1292,6 +1321,7 @@ export const runnerSuites: readonly RunnerSuiteFixture[] = [
|
||||
environments: [daytonaWarmEnvironment],
|
||||
tasks: [daytonaWarmContinuityTask],
|
||||
expectedMatrixSize: 2,
|
||||
definitionMetadata: { version: 3, nativeInstructions: "managed-incremental", managedFileBytes: 8 * 1024 * 1024 },
|
||||
},
|
||||
{
|
||||
id: "daytona-journal-continuity",
|
||||
@@ -1299,11 +1329,11 @@ export const runnerSuites: readonly RunnerSuiteFixture[] = [
|
||||
manualOnly: true,
|
||||
description: "Continue the same native session after separate ordinary tool invocations and their output grow its durable journal beyond 2 MiB.",
|
||||
groups: ["daytona", "warm"],
|
||||
profiles: codexContinuityProfiles.filter((profile) => profile.id === "runner-codex"),
|
||||
profiles: warmCodexContinuityProfiles.filter((profile) => profile.id === "runner-codex"),
|
||||
environments: [daytonaWarmEnvironment],
|
||||
tasks: [daytonaLargeJournalTask],
|
||||
expectedMatrixSize: 1,
|
||||
definitionMetadata: { version: 4, journalMinimumBytes: 2 * 1024 * 1024, toolInvocations: 240, outputBytesPerInvocation: 65020, scheduling: "explicit-only" },
|
||||
definitionMetadata: { version: 5, nativeInstructions: "fixed-external", journalMinimumBytes: 2 * 1024 * 1024, toolInvocations: 240, outputBytesPerInvocation: 65020, scheduling: "explicit-only" },
|
||||
},
|
||||
{
|
||||
id: "daytona-git-streaming",
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Warm continuity fixture agent
|
||||
|
||||
Complete the current task and its authorized review revisions using the supplied
|
||||
task context. Preserve the workspace files across turns and follow the task's
|
||||
verification and completion instructions. These external instructions are fixed;
|
||||
do not edit this instruction file or create a personal agent folder.
|
||||
@@ -1,3 +1,4 @@
|
||||
import { warmManagedFileEvidence } from "./warm-managed-files.js";
|
||||
import { gitFinalizationEvidence, gitStreamingEvidence, setupGitStreamingWorkspace } from "./daytona-git-streaming.js";
|
||||
import { runsCompletionUpdateProbe, completionQualityControls, completionQualityStatus, judgeCompletionQuality, reserveCompletionQuality, type CompletionQualityRecord } from "./completion-quality.js";
|
||||
import { completionDelivery, type CompletionObservation } from "./completion-updates.js";
|
||||
@@ -1541,6 +1542,11 @@ for (const execution of executions) {
|
||||
await writeSanitizedJson(snapshotsDir, `git-finalization-turn-${completedTurn}.json`, finalization, secrets);
|
||||
expect(finalization.passed, finalization.failures.join("; ")).toBe(true);
|
||||
}
|
||||
if (execution.suite.id === "daytona-warm-continuity") {
|
||||
const evidence = await warmManagedFileEvidence(api, fixtures.agent.id, sortRunsChronologically(waitingState.taskRuns).at(-1)!.id, completedTurn, nonce, execution.profile.generation === "native");
|
||||
await writeSanitizedJson(snapshotsDir, `managed-warm-turn-${completedTurn}.json`, evidence, secrets);
|
||||
expect(evidence.passed, JSON.stringify(evidence.checks)).toBe(true);
|
||||
}
|
||||
const expectedPrefix = `${Array.from(
|
||||
{ length: completedTurn },
|
||||
(_, index) => `T${index + 1}-${nonce}`,
|
||||
@@ -1823,6 +1829,11 @@ for (const execution of executions) {
|
||||
};
|
||||
});
|
||||
}
|
||||
if (execution.suite.id === "daytona-warm-continuity") {
|
||||
const evidence = await warmManagedFileEvidence(api, fixtures.agent.id, selectedRuns.at(-1)!.id, 3, nonce, execution.profile.generation === "native");
|
||||
await writeSanitizedJson(snapshotsDir, "managed-warm-turn-3.json", evidence, secrets);
|
||||
expect(evidence.passed, JSON.stringify(evidence.checks)).toBe(true);
|
||||
}
|
||||
const finalRun = selectedRuns.at(-1)!;
|
||||
const run =
|
||||
selectedRuns.find(
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { gradeWarmManagedFiles } from "./warm-managed-files.js";
|
||||
describe("managed warm file oracle", () => {
|
||||
const good = { turn: 2, nonce: "test", content: "T1-test\nT2-test\n", binary: Buffer.alloc(8388608, 93), deletedStatus: 404, native: true,
|
||||
save: { state: "saved", contract: "agent_files", checkpointStats: { copiedBytes: 16, copiedFiles: 1, hashedBytes: 16 } } };
|
||||
it("accepts independently saved bytes with an incremental receipt", () => expect(gradeWarmManagedFiles(good).passed).toBe(true));
|
||||
it.each([
|
||||
{ content: "T1-test\n" }, { binary: Buffer.from("claimed image") }, { deletedStatus: 200 }, { save: undefined },
|
||||
{ save: { ...good.save, checkpointStats: { copiedBytes: 8388624, copiedFiles: 2, hashedBytes: 8388624 } } },
|
||||
])("rejects missing, stale, deleted, or fully recopied data", wrong => expect(gradeWarmManagedFiles({ ...good, ...wrong }).passed).toBe(false));
|
||||
});
|
||||
@@ -0,0 +1,32 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import type { RunnerApi } from "./api.js";
|
||||
|
||||
const record = (v: unknown): Record<string, unknown> => v && typeof v === "object" && !Array.isArray(v) ? v as Record<string, unknown> : {};
|
||||
export function gradeWarmManagedFiles(input: {
|
||||
turn: number; nonce: string; content: unknown; binary: Buffer; deletedStatus: number; save: unknown; native: boolean;
|
||||
}) {
|
||||
const expected = `${Array.from({ length: input.turn }, (_, i) => `T${i + 1}-${input.nonce}`).join("\n")}\n`;
|
||||
const expectedBinary = Buffer.alloc(8 * 1024 * 1024, 93);
|
||||
const save = record(input.save), stats = record(save.checkpointStats);
|
||||
const checks = [
|
||||
{ id: "managed-memory-exact", passed: input.content === expected },
|
||||
{ id: "managed-binary-exact", passed: input.binary.equals(expectedBinary) },
|
||||
{ id: "managed-deletion", passed: input.deletedStatus === (input.turn === 1 ? 200 : 404) },
|
||||
{ id: "managed-save-receipt", passed: save.state === "saved" && save.contract === "agent_files" && !save.errorCode },
|
||||
...(input.native ? [{ id: "incremental-payload", passed: typeof stats.copiedFiles === "number" &&
|
||||
(input.turn === 1 ? stats.copiedFiles === 3 && stats.copiedBytes === expectedBinary.length + Buffer.byteLength(expected) + 9
|
||||
: stats.copiedFiles === 1 && stats.copiedBytes === Buffer.byteLength(expected) && stats.hashedBytes === Buffer.byteLength(expected)) }] : []),
|
||||
];
|
||||
return { passed: checks.every(c => c.passed), checks, expected, binaryBytes: input.binary.length,
|
||||
binarySha256: createHash("sha256").update(input.binary).digest("hex"), save };
|
||||
}
|
||||
export async function warmManagedFileEvidence(api: RunnerApi, agentId: string, runId: string, turn: number, nonce: string, native: boolean) {
|
||||
const base = `/api/agents/${agentId}/instructions-bundle/file?path=`;
|
||||
const note = await api.get<{ content: string }>(`${base}notes%2Fwarm-memory.txt`);
|
||||
const binary = await api.request.get(`${base}notes%2Funchanged.bin&download=true`);
|
||||
if (!binary.ok()) throw new Error(`Managed binary download failed: ${binary.status()}`);
|
||||
const deleted = await api.request.get(`${base}notes%2Fdelete-me.txt&download=true`);
|
||||
const run = await api.get<{ resultJson?: unknown }>(`/api/heartbeat-runs/${runId}`);
|
||||
return gradeWarmManagedFiles({ turn, nonce, content: note.content, binary: await binary.body(), deletedStatus: deleted.status(),
|
||||
save: record(run.resultJson).instructionSave, native });
|
||||
}
|
||||
Reference in new issue
Block a user