Measure compact native continuation alongside full task delivery

Co-Authored-By: Paperclip <noreply@paperclip.ing>

doc/plans/2026-10-03-native-completion-consolidation.md

@@ -16,7 +16,7 @@ Both variants start from master `2a8a99e4a5f69aa803b3f10b982f583e75a87042`. The
 
 ## Provider-free measurement
 
-`src/backends/native-instruction-measurement.test.ts` in the Runner package captures actual `thread/start`, `thread/resume` and `turn/start` parameters from the real runnerd facade with a scripted transport. It captures native Codex, ACPX Claude and OpenCode for input v4 and v5: twelve receipts per variant. Every receipt records exact UTF-8 bytes, character counts and hashes of the instruction string, fixture tool schemas, turn input and their full projection. It checks finish/block descriptions, document/file rules, approval mode and identical start/resume tools and instructions. The fixture uses the actual production AGENTS file and a fixed projection of the core semantic catalog plus the reserved completion tools.
+`src/backends/native-instruction-measurement.test.ts` in the Runner package captures actual `thread/start`, `thread/resume` and `turn/start` parameters from the real runnerd facade with a scripted transport. It captures native Codex, ACPX Claude and OpenCode for input v4 and v5: eighteen receipts per variant. Every receipt records exact UTF-8 bytes, character counts and hashes of the instruction string, fixture tool schemas, turn input and their full projection. It checks finish/block descriptions, document/file rules, approval mode and identical start/resume tools and instructions. The fixture uses the actual production AGENTS file and a fixed projection of the core semantic catalog plus the reserved completion tools.
 
 Run from `packages/paperclip-runner`:
 
@@ -24,7 +24,7 @@ Run from `packages/paperclip-runner`:
 PAPERCLIP_NATIVE_INSTRUCTION_REPORT=/absolute/path/measurement.json pnpm exec vitest run src/backends/native-instruction-measurement.test.ts
 ```
 
-The receipt declares its source SHA, source file hashes, fixture hash and dirty state. A dirty overlay can aid diagnosis but is rejected for live admission. Temporary workspace paths are normalized to a fixed path for comparison. This is the complete declared Paperclip projection at that boundary, not the production server's full granted catalog or a capture of provider-owned stock prompts. Token counts, billing, provider preprocessing and cognitive consumption remain unknown. Scripted resume capture does not qualify live resume behavior.
+The receipt declares its source SHA, source file hashes, fixture hash and dirty state. A dirty overlay can aid diagnosis but is rejected for live admission. Temporary workspace paths are normalized to a fixed path for comparison. This is the complete declared Paperclip projection at that boundary, not the production server's full granted catalog or a capture of provider-owned stock prompts. Token counts, billing, provider preprocessing and cognitive consumption remain unknown. The capture distinguishes a resumed full task from the compact user-follow-up continuation envelope. Input v5 sends only new events and contract references for that continuation; its projection must remain identical between variants. Input v4 still wraps the compact message in a task envelope. Scripted resume and continuation capture do not qualify live resume behavior.
 
 ## Bounded live comparison
 
@@ -44,7 +44,7 @@ Each source variant declares these original tasks on the same local native profi
 
 Six cells per variant means twelve actual runs for the pair, each with one attempt. Use exact IDs with parallelism one and no automatic retry. Each cell uses the original task deadline and 1,000-cent company and agent hard stops. These budgets bound recorded Paperclip spend; they do not guarantee final provider invoice amounts. Do not run providers until the human authorizes the live scope. Stop to inspect a usable behavior failure; do not reroll it to obtain a pass.
 
-The new source gate requires clean committed sources, the declared base ancestry, one complete known source variant, no unrelated paths, the current fixture digest and all twelve provider-free capture receipts. It rejects missing, duplicated, dirty, stale or paid capture evidence. It verifies the requested immutable source SHA when supplied. Before live execution also verify the selected compiled Runner/source identity through the existing launcher build path. Hosted shallow sources are not admitted by this local gate; do not substitute a relaxed ancestry check.
+The new source gate requires clean committed sources, the declared base ancestry, one complete known source variant, no unrelated paths, the current fixture digest and all eighteen provider-free capture receipts. It rejects missing, duplicated, dirty, stale or paid capture evidence. It verifies the requested immutable source SHA when supplied. Before live execution also verify the selected compiled Runner/source identity through the existing launcher build path. Hosted shallow sources are not admitted by this local gate; do not substitute a relaxed ancestry check.
 
 The suite uses the original assigned-skill durable-document oracle and strict native completion grading. It retains named call-ID joins, acceptance/termination, final-response order, provider-final persistence, exact blocker owner/action/scope, workspace and issue/agent side effects, default instructions and budget evidence. Unknown or absent evidence cannot pass. No relaxed source-read matcher or provider-call join is introduced.
 

packages/paperclip-runner/src/backends/native-instruction-measurement.test.ts

@@ -68,9 +68,9 @@ function measure(value: unknown) {
 
 describe("native instruction payload measurement", () => {
   for (const provider of providers) for (const schema of ["v4", "v5"] as const) {
-    it(`captures ${provider.kind} ${schema} instructions and tools on start and resume`, async () => {
+    it(`captures ${provider.kind} ${schema} instructions and tools on start, resume and continuation`, async () => {
       const input = execution(provider, schema);
-      const fresh = transport(provider), resumed = transport(provider);
+      const fresh = transport(provider), resumed = transport(provider), continued = transport(provider);
       const backend = createRunnerdNativeSessionBackend(input, {
         transportFactory: () => fresh, dynamicTools: tools,
         environment: { PAPERCLIP_WORKSPACE_CWD: WORKSPACE },
@@ -92,7 +92,20 @@ describe("native instruction payload measurement", () => {
       await recovery.session!.startTurn({ message: { role: "user", text: JSON.stringify(envelope) } });
       await recovery.session!.close({ reason: "measurement" });
 
-      for (const [phase, captured, method] of [["start", fresh, "thread/start"], ["resume", resumed, "thread/resume"]] as const) {
+      continued.readResponse = resumed.readResponse;
+      const continuationInput = parseNativeExecutionInput({ ...input, continuationPrompt: "User update: retain the existing document and finish." });
+      const continuationBackend = createRunnerdNativeSessionBackend(continuationInput, {
+        transportFactory: () => continued, dynamicTools: tools,
+        environment: { PAPERCLIP_WORKSPACE_CWD: WORKSPACE },
+      });
+      const continuationRecovery = await continuationBackend.recoverSession!(checkpoint, { signal: new AbortController().signal });
+      expect(continuationRecovery.recovered).toBe(true);
+      const continuationEnvelope = buildNativeModelEnvelope(continuationInput, { resumedSession: true });
+      expect(continuationEnvelope.schema).toBe("paperclip.native-continuation.v1");
+      await continuationRecovery.session!.startTurn({ message: { role: "user", text: JSON.stringify(continuationEnvelope) } });
+      await continuationRecovery.session!.close({ reason: "measurement" });
+
+      for (const [phase, captured, method] of [["start", fresh, "thread/start"], ["resume", resumed, "thread/resume"], ["continuation", continued, "thread/resume"]] as const) {
         const setup = captured.calls.find(call => call.method === method)!.params;
         const turn = captured.calls.find(call => call.method === "turn/start")!.params;
         const instructions = setup.developerInstructions ?? setup.baseInstructions;
@@ -106,10 +119,25 @@ describe("native instruction payload measurement", () => {
           expect.objectContaining({ description: PRP_BLOCK_TOOL_DESCRIPTION }),
         ]);
         expect(deliveredTools).toHaveLength(tools.length + 2);
-        expect(JSON.stringify(turn.input)).toContain("Obtain one accepted result");
-        expect(JSON.stringify(turn.input)).toContain("Document saved");
-        expect(JSON.stringify(turn.input)).toContain("write_document");
-        expect(JSON.stringify(turn.input)).toContain("register_deliverable");
+        if (phase === "continuation") {
+          expect(JSON.stringify(turn.input)).toContain("User update: retain the existing document and finish.");
+          const delivered = JSON.parse((turn.input as Array<{ text: string }>)[0]!.text);
+          expect(schema === "v4" ? JSON.parse(delivered.message) : delivered).toEqual(continuationEnvelope);
+          if (schema === "v5") {
+            expect(JSON.stringify(turn.input)).not.toContain("constraints");
+            expect(JSON.stringify(turn.input)).not.toContain("Document saved");
+          } else {
+            // v4's task-mode driver still wraps the compact message in its task envelope.
+            expect(JSON.stringify(turn.input)).toContain("Obtain one accepted result");
+            expect(JSON.stringify(turn.input)).toContain("Document saved");
+          }
+          expect(JSON.stringify(turn.input)).toContain("objective");
+        } else {
+          expect(JSON.stringify(turn.input)).toContain("Obtain one accepted result");
+          expect(JSON.stringify(turn.input)).toContain("Document saved");
+          expect(JSON.stringify(turn.input)).toContain("write_document");
+          expect(JSON.stringify(turn.input)).toContain("register_deliverable");
+        }
         expect(setup.approvalPolicy).toBe("never");
         receipts.push({ provider: provider.kind, schema, phase,
           instructions: measure(instructions), tools: measure(setup.dynamicTools), input: measure(turn.input),
@@ -128,7 +156,7 @@ describe("native instruction payload measurement", () => {
 afterAll(() => {
   const output = process.env.PAPERCLIP_NATIVE_INSTRUCTION_REPORT;
   if (!output) return;
-  if (receipts.length !== 12) throw new Error("Incomplete native instruction measurement; refusing a partial receipt");
+  if (receipts.length !== 18) throw new Error("Incomplete native instruction measurement; refusing a partial receipt");
   const sourcePaths = ["runtime-context.ts", "codex-native-backend.ts", "opencode-native-backend.ts"];
   writeFileSync(output, `${JSON.stringify({
     schema: "paperclip.native-instruction-measurement.v1",

tests/runner-e2e/README.md

@@ -424,7 +424,7 @@ pnpm test:e2e:runner -- --id agent-chat-hardening.runner-codex.local.stop-startu
 
 The independent, explicit-only `native-completion` suite qualifies native finish/block descriptions on unchanged master defaults. It preserves the original assigned-skill document journey and pairs it with whole-task blocking across three native profiles, with enforced single attempts. See [NATIVE-COMPLETION.md](NATIVE-COMPLETION.md) for admission, provenance and limits.
 
-The separate, explicit-only `native-instruction-consolidation` suite reuses those original tasks and strict graders to compare completion constraints on the production defaults at `2a8a99e4a5f69aa803b3f10b982f583e75a87042`. It declares six local cells: document completion and whole-task blocking on native Codex, ACPX Claude, and OpenCode. Each cell allows one attempt and applies a 1,000-cent company and agent budget hard stop. Its source gate rejects dirty, mixed, unknown, or unrelated source changes before credentials load. A provider-free fixture captures the complete Paperclip instruction/tool/message projection at the scripted runnerd RPC boundary on start and resume for native input v4 and v5. That capture measures bytes; it does not measure vendor-owned prompts, tokens, billing, or model behavior. See the [comparison plan](../../doc/plans/2026-10-03-native-completion-consolidation.md) for exact scope and live qualification limits. Existing `native-completion` results do not qualify this new reduction.
+The separate, explicit-only `native-instruction-consolidation` suite reuses those original tasks and strict graders to compare completion constraints on the production defaults at `2a8a99e4a5f69aa803b3f10b982f583e75a87042`. It declares six local cells: document completion and whole-task blocking on native Codex, ACPX Claude, and OpenCode. Each cell allows one attempt and applies a 1,000-cent company and agent budget hard stop. Its source gate rejects dirty, mixed, unknown, or unrelated source changes before credentials load. A provider-free fixture captures the complete Paperclip instruction/tool/message projection at the scripted runnerd RPC boundary on start, full-task resume and compact user-follow-up continuation for native input v4 and v5. That capture measures bytes; it does not measure vendor-owned prompts, tokens, billing, or model behavior. See the [comparison plan](../../doc/plans/2026-10-03-native-completion-consolidation.md) for exact scope and live qualification limits. Existing `native-completion` results do not qualify this new reduction.
 
 `context-integrity` is an explicit-only local suite with two bounded cases across
 ten listed legacy/native profiles (20 cells). Six cells are pending-prerequisite

tests/runner-e2e/native-instruction-consolidation.test.ts

@@ -54,7 +54,7 @@ describe("native instruction comparison admission", () => {
       schema: "paperclip.native-instruction-measurement.v1", sourceSha: "frozen", sourceDirty: false, providerCalls: 0,
       fixtureSha256: createHash("sha256").update(readFileSync(new URL("../../packages/paperclip-runner/src/backends/native-instruction-measurement.test.ts", import.meta.url))).digest("hex"),
       sourceHashes: Object.fromEntries(Object.entries(NATIVE_INSTRUCTION_VARIANTS.baseline).map(([file, digest]) => [file.split('/').at(-1)!, digest])),
-      receipts: ['codex', 'acpx', 'opencode'].flatMap(provider => ['v4', 'v5'].flatMap(schema => ['start', 'resume'].map(phase => ({ provider, schema, phase })))),
+      receipts: ['codex', 'acpx', 'opencode'].flatMap(provider => ['v4', 'v5'].flatMap(schema => ['start', 'resume', 'continuation'].map(phase => ({ provider, schema, phase })))),
     };
     const source = { sourceSha: "frozen", variant: "baseline" };
     expect(() => validateNativeInstructionMeasurement(measurement, source)).not.toThrow();

tests/runner-e2e/native-instruction-consolidation.ts

@@ -88,7 +88,7 @@ export function validateNativeInstructionMeasurement(measurement: {
   sourceHashes: Record<string, string>; receipts: Array<{ provider: string; schema: string; phase: string }>;
 }, source: { sourceSha: string; variant: string }) {
   const expected = ['codex', 'acpx', 'opencode'].flatMap(provider => ['v4', 'v5'].flatMap(schema =>
-    ['start', 'resume'].map(phase => `${provider}/${schema}/${phase}`))).sort();
+    ['start', 'resume', 'continuation'].map(phase => `${provider}/${schema}/${phase}`))).sort();
   const actual = measurement.receipts.map(value => `${value.provider}/${value.schema}/${value.phase}`).sort();
   const files = NATIVE_INSTRUCTION_VARIANTS[source.variant as keyof typeof NATIVE_INSTRUCTION_VARIANTS];
   if (measurement.schema !== "paperclip.native-instruction-measurement.v1"