Files
PaperClipAI/tests/runner-e2e/warm-managed-files.test.ts
T
DottaandPaperclip 53f480d1b3 fix(native): preserve warm continuity and deferred review replies (#18631)
## Thinking Path

> - Paperclip manages AI agents and their saved work.
> - Product E2E tests must check saved personal files across separate
turns.
> - The server records file collection in the durable `instruction_save`
run-log event.
> - The warm test read optional terminal result metadata instead and
rejected a recorded successful save.
> - This pull request reads the complete event stream for the current
run and uses its latest save receipt.
> - The prompt now specifies the exact nine bytes required by the
temporary-file transfer count.
> - Human review actions select the actual interaction card, which can
share its interaction ID with a navigation button.
> - A later real three-turn run exposed a missing reply after workspace
finalization moved to its durable recovery owner.
> - Reply recovery handled completed tasks but omitted ordinary tasks
that awaited review.
> - This pull request also publishes their recorded final reply once
while preserving the pending review.

## Linked Issues or Issue Description

**What happened?**

The [retained live continuity
attempt](https://github.com/paperclipai/paperclip/actions/runs/41477302876/job/125420001074)
saved the correct memory file and unchanged 8 MiB binary. Its durable
run log contained a successful `instruction_save` receipt. The oracle
read `resultJson.instructionSave`, which was absent, and failed before
turn two. The temporary file also had a trailing LF. Its prompt allowed
that result, while the byte-count check required nine bytes.

**Expected behavior**

The oracle must use the documented authoritative save event. A missing
receipt, a later failed save, incorrect transfer counts, or another
run's events must fail. The prompt must specify the bytes that its
independent check requires.

**Steps to reproduce**

1. Run `daytona-warm-continuity.runner-codex.daytona.warm-three-turn`
with the configured protected credentials and immutable image.
2. Complete a native turn whose terminal result has no `instructionSave`
field.
3. Compare the oracle output with the same run's successful
`instruction_save` event.

A subsequent [three-turn
attempt](https://github.com/paperclipai/paperclip/actions/runs/41495337334/job/125451304553)
passed every personal-file assertion and reused the same warm process.
Its first final reply was recorded as a completed agent-message event
but was absent from comments. The log shows
`native_workspace_finalization_busy`; the durable recovery path excluded
`needs_review` replies. The final message matcher correctly failed. This
failure remains recorded.

The observed application revision was
`aab87ce70744660a4a21c77278b0ede267c04720`. Related work: #14695 and
#18494. The retained failed result keeps its original grade and
definition.

## What Changed

- Read all run-event pages through the existing bounded collector.
- Require events to belong to the requested run and select the latest
`instruction_save` event.
- Retain strict byte counts and reject missing, failed, pending or
superseded receipts.
- Specify nine ASCII bytes and no trailing newline for the temporary
file.
- Scope human review actions to the interaction card and its exact
interaction ID.
- Advance the warm suite definition to version 5 and document its
evidence source.
- Recover recorded final replies for ordinary native tasks with an
accepted `needs_review` result and terminal state.
- Preserve pending reviews, existing reply selection,
company/turn/digest checks and external governed-response authorization.
- Update the task feature and agent access maps.

## Verification

- Real PostgreSQL regression: deferred review reply discovery failed
before the production repair (expected one candidate; found zero). The
final-message event and pending review were present.
- Recovery cases cover one publication, no repeated provider run,
unchanged review, and rejection of absent/wrong-turn final messages, bad
digest and suppressed comments.

- Red: seven new boundary regressions failed against the old receipt
lookup.
- Green: 169 focused oracle, event pagination and catalog checks passed.
Product E2E typecheck passed after rebuilding local package
declarations.
- Retained-evidence replay confirms that the new lookup finds the real
successful save event. The original attempt remains failed because its
temporary file included an LF.
- Recursive typecheck passed before the final recovery change; server
typecheck passed afterward. The recovery and control-plane suites passed
all 112 tests. Full build and map checks passed.
- The broad local test run reached one unrelated 30-second timeout in
`attention-retention-work.test.ts` (maintained-review/confirmation
batching). The same unchanged case then passed on one targeted retry
(39.9 seconds including module load). The original failure remains
retained. The broad run also reported one unchanged workspace-conflict
case, which passed on its targeted retry (14.85 seconds including module
load). Both original failures remain retained. The redundant broad local
run was stopped after current-head CI completed all required gates: 55
successful checks and 3 selection skips. No all-green local broad-run
claim is made.
- The separate artifact-transfer cell already passed on the recorded
application revision: independent source and delivered bytes matched at
407,485 bytes, with a completed task, succeeded native run and cleared
execution locks. This does not establish a three-turn continuity pass.

- [Real native three-turn
acceptance](https://github.com/paperclipai/paperclip/actions/runs/41499102994/job/125465795210)
**passed** at `3b207a42943e9eeb371d4f2d5623de72093eb6d5`, suite
definition 5. Turn one exercised `committed_task_response_recovered`.
All three completed final replies were persisted once and in order. All
three strict personal-file receipts passed. The unchanged 8 MiB binary
survived; later transfers copied only 36 and 54 bytes. The lease was
created, resumed, and resumed with the same runner PID/start time,
runner instance, native session and provider session. Final task was
`done`, all three runs succeeded, and checkout/execution locks were
null.
- The aggregate startup summary has no separate provider-process PID
proof. The raw run records independently prove the stable runner process
and provider session stated above. This single native cell does not
qualify the legacy cell or a full provider matrix.
- The passing campaign reused the signed immutable runner image. It
required no disk expansion. Earlier campaign image builds used the
recorded temporary 120 GiB mitigation; default 60 GiB build capacity
remains unqualified. All scoped completed probes were removed and
independently returned 404; org CPU/disk usage returned to zero. This
does not resolve unknown historical resources under a different
credential identity.
- Current-head Greptile is 5/5; review-thread pagination is complete
with zero threads. All 55 current-head checks are successful; 3
selection checks are skipped. The paid campaign, result enforcement and
report publication all completed successfully.

## Risks

- Publishing a reply must not approve the work. Recovery only copies the
accepted turn’s completed final message. It preserves the pending review
and does not start an executor.
- Conversation and external governed-response paths retain their
existing separate authorization.

The latest save event can record failure after an earlier success. The
oracle rejects that case. Event pagination, missing receipts and
cross-run evidence also fail closed. The exact temporary-file byte count
remains required. The review selector still requires the intended
interaction ID and enabled action on its actual card. The production
change is limited to ordinary internal-task reply recovery. It still
requires an accepted result, a matching successful terminal disposition,
the completed final message, and existing publication authority.

## Model Used

OpenAI Codex, based on GPT-6, with shell and code tools. The exact
deployment variant and context-window size are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-02 14:19:48 -06:00

68 lines
3.7 KiB
TypeScript

import { describe, expect, it } from "vitest";
import type { RunnerApi } from "./api.js";
import { gradeWarmManagedFiles, warmManagedFileEvidence } from "./warm-managed-files.js";
describe("managed warm file oracle", () => {
const good = { turn: 2, nonce: "test", content: "T1-test\nT2-test\n", binary: Buffer.alloc(8388608, 93), deletedStatus: 404, native: true,
save: { state: "saved", contract: "agent_files", checkpointStats: { copiedBytes: 16, copiedFiles: 1, hashedBytes: 16 } } };
it("accepts independently saved bytes with an incremental receipt", () => expect(gradeWarmManagedFiles(good).passed).toBe(true));
it.each([
{ content: "T1-test\n" }, { content: "T1-test\\nT2-test\\n" },
{ binary: Buffer.from("claimed image") }, { deletedStatus: 200 }, { save: undefined },
{ save: { ...good.save, checkpointStats: { copiedBytes: 8388624, copiedFiles: 2, hashedBytes: 8388624 } } },
])("rejects missing, stale, deleted, or fully recopied data", wrong => expect(gradeWarmManagedFiles({ ...good, ...wrong }).passed).toBe(false));
});
describe("managed warm file receipt evidence", () => {
const saved = { state: "saved", contract: "agent_files", errorCode: null,
checkpointStats: { copiedFiles: 3, copiedBytes: 8388625, hashedBytes: 8389685 } };
type Event = { seq: number; runId: string; eventType: string; payload?: unknown };
const receipt = (seq: number, payload: unknown = saved): Event => ({ seq, runId: "warm-run", eventType: "instruction_save", payload });
function fixtureApi(events: Event[], summary: unknown = { instructionSave: saved }) {
const requestedPages: number[] = [];
const api = {
async get(path: string) {
if (path.includes("instructions-bundle")) return { content: "T1-test\n" };
if (path.includes("/events?")) {
const after = Number(new URL(path, "http://fixture.invalid").searchParams.get("afterSeq"));
requestedPages.push(after);
return events.filter(event => event.seq > after).slice(0, 1000);
}
return { resultJson: summary };
},
request: { async get(path: string) {
return { ok: () => true, status: () => 200,
body: async () => path.includes("unchanged.bin") ? Buffer.alloc(8388608, 93) : Buffer.from("temporary") };
} },
} as unknown as RunnerApi;
return { api, requestedPages };
}
const capture = (api: RunnerApi) => warmManagedFileEvidence(api, "agent", "warm-run", 1, "test", true);
it("reads the durable save receipt when terminal result metadata omits it", async () => {
const { api } = fixtureApi([receipt(297)], {});
expect((await capture(api)).passed).toBe(true);
});
it("reads all event pages before selecting the latest receipt", async () => {
const filler = Array.from({ length: 1000 }, (_, index) => ({ seq: index + 1, runId: "warm-run", eventType: "item.delta" }));
const { api, requestedPages } = fixtureApi([...filler, receipt(1001)], {});
expect((await capture(api)).passed).toBe(true);
expect(requestedPages).toEqual([0, 1000]);
});
it.each([
[],
[receipt(1), receipt(2, { ...saved, state: "pending_collection" })],
[receipt(1), receipt(2, { ...saved, errorCode: "AGENT_FILES_CHECKPOINT_UNSTABLE" })],
[receipt(1), receipt(2, { ...saved, checkpointStats: { copiedFiles: 1, copiedBytes: 8 } })],
])("rejects missing or superseded evidence even when the run summary claims success", async (...events) => {
const { api } = fixtureApi(events);
expect((await capture(api)).passed).toBe(false);
});
it("rejects a receipt returned for a different run", async () => {
const { api } = fixtureApi([{ ...receipt(1), runId: "another-run" }]);
await expect(capture(api)).rejects.toThrow("different run");
});
});