diff --git a/doc/plans/2026-10-02-pi-production-readiness.md b/doc/plans/2026-10-02-pi-production-readiness.md index 75614c81fc1..4ae2e9b881b 100644 --- a/doc/plans/2026-10-02-pi-production-readiness.md +++ b/doc/plans/2026-10-02-pi-production-readiness.md @@ -202,15 +202,48 @@ prices are estimates. Unpriced usage must stay unpriced in the ledger and UI. the verifier gives that launch's private runtime snapshots the same bounded scratch capacity. This is not counted as a passing Linux receipt yet. +## Resumed goal — 2026-10-02 21:50 CDT + +- Current-head `6cfc79b50` CI completes with two failures: Linux Canary + admission and a 15-second issue-document route test timeout. Runner, build, + typecheck and the browser shards pass. The prerequisite stack remains open. +- The Linux admission failure is reproduced through normal public packages. + Pi setup verifies profile 14, then admission times out in 37.3 seconds. + A credential-free direct native RPC response arrives in 2.4 seconds. + The actual Docker scratch mount reports `noexec`; executing the verified + snapshot fails with `EACCES`. The offline runtime probe now uses executable + scratch. Lifecycle and download probes explicitly retain `noexec`. + The same installed Linux packages then pass the unchanged public admission + assertions in 17.2 seconds, with clean Runner exit and zero prompts. + This receipt uses historical shipping source `dc2b053f3` and native inputs + from `3728bb45f`. It diagnoses and validates the sandbox correction; it does + not qualify the new native source or the final Daytona image. +- A provider-free regression proves that receipt-limit deadline settlement + attempted to restart the provider when polling terminal evidence. The run + now closes permanently at that deadline. Existing startup admission fences + remain intact. All 333 native core tests and the original Mac interruption + test pass after the correction. The regression checks both the existing + controller and a fresh controller with unacknowledged terminal evidence. +- Runner progress evidence contains four successful reads and continued model + output before the 120-second cutoff, including unrelated fixture notes. + Bounded direct-eval instructions now ask for the minimum context needed, + the requested action, then turn completion. Case assertions and timeout + stay unchanged. All 35 Runner session-contract tests and TypeScript + typecheck pass. The retained paid failure remains unchanged. Fresh exact + source packaging and profile-14 eval definitions must precede a paid retry. +- No paid call, key reset, fallback credential, workflow change, lockfile + commit, merge or release occurs in this resumed diagnosis. + ## Remaining work in order 1. Diagnose Linux closed admission from its real startup and cleanup evidence. - Resolve `session_handshake_timeout` and classify the observed Mac ownership - contention. Produce the normal Linux public-install and companion receipts, + Recheck the corrected executable-scratch probe at the final source and + classify the observed Mac ownership contention. Produce the normal Linux + public-install and companion receipts, then import an immutable exact-source Daytona image without an override. Do not accept a timeout as an installation pass or widen the admission limit. 2. Resolve Runner `context-before-action` with a concrete behavioral correction. - It currently repeats context calls without the required progress mutation. + Bounded harness instructions are corrected; the retained attempt has no progress mutation. Do not run an unchanged paid retry. Then execute the remaining explicit Runner cases and the full 26-cell Product matrix on the frozen candidate. 3. Close the prerequisite review stack and final CI gates. Retain the local diff --git a/packages/paperclip-runner/docs/runner-protocol-live-evals.md b/packages/paperclip-runner/docs/runner-protocol-live-evals.md index bbaaf1aee5e..0dc7e2b8de4 100644 --- a/packages/paperclip-runner/docs/runner-protocol-live-evals.md +++ b/packages/paperclip-runner/docs/runner-protocol-live-evals.md @@ -196,6 +196,9 @@ replacement objective or proof that a newly requested action is already done. Finishing the provider turn does not authorize an unrequested mock task-state change or completion comment. These harness instructions keep single-operation cases bounded while leaving their operation and state-effect assertions intact. +For a bounded request, the harness also asks for only the context needed to act. +A brief progress request does not require an investigation of unrelated history +or documents. Its grader still requires the actual successful mutation. ACPX accounting uses the qualified server's billable token semantics: Claude and Codex already include reasoning in output, and Codex has no cache-write diff --git a/packages/paperclip-runner/runner/crates/runner-core/src/provider_backend.rs b/packages/paperclip-runner/runner/crates/runner-core/src/provider_backend.rs index aea835f7244..27c0ab72015 100644 --- a/packages/paperclip-runner/runner/crates/runner-core/src/provider_backend.rs +++ b/packages/paperclip-runner/runner/crates/runner-core/src/provider_backend.rs @@ -3580,7 +3580,9 @@ impl CodexCommandExecutor { state.receipt_limit_interrupt_accepted = false; state.receipt_limit_interrupt_attempts = 0; state.receipt_limit_interrupt_deadline_unix_ms = None; - state.lifecycle = "provider_exited".to_owned(); + // Deadline settlement retires this run permanently. Polling its terminal + // outbox must not reopen a provider that still reports the stopped turn. + state.lifecycle = "closed".to_owned(); let terminal_event_type = if interrupt_accepted { "turn.interrupted" } else { @@ -6260,7 +6262,7 @@ mod tests { provider: "codex".to_owned(), driver: "codex_app_server".to_owned(), provider_version: "test".to_owned(), - command: PathBuf::from("codex"), + command: PathBuf::from("/definitely-missing-receipt-deadline-provider"), args: vec!["app-server".to_owned()], cwd: std::env::current_dir() .unwrap() @@ -6307,7 +6309,7 @@ mod tests { executor.maintain_backpressured_provider().unwrap(); let state = executor.state.as_ref().unwrap(); - assert_eq!(state.lifecycle, "provider_exited"); + assert_eq!(state.lifecycle, "closed"); assert!(state.active_provider_turn_id.is_none()); assert!(!state.receipt_limit_interrupt_pending); assert!(state.pending_events.iter().any(|event| { @@ -6325,6 +6327,13 @@ mod tests { settled ); assert!(executor.provider.is_none()); + // Reading terminal evidence must not restart work whose interruption + // exhausted its deadline, even after a controller reconnects. + assert!(!executor.poll_events().unwrap().is_empty()); + assert!(executor.provider.is_none()); + let mut restarted = CodexCommandExecutor::new(&directory); + assert!(!restarted.poll_events().unwrap().is_empty()); + assert!(restarted.provider.is_none()); fs::remove_dir_all(directory).unwrap(); } } diff --git a/packages/paperclip-runner/src/cli/eval-session-contract.test.ts b/packages/paperclip-runner/src/cli/eval-session-contract.test.ts index 66bb2707afe..83864926b5a 100644 --- a/packages/paperclip-runner/src/cli/eval-session-contract.test.ts +++ b/packages/paperclip-runner/src/cli/eval-session-contract.test.ts @@ -158,6 +158,7 @@ describe("eval-session request contract", () => { expect(systemInstructions).toContain("Paperclip direct live evaluation"); expect(systemInstructions).toContain("Task-state changes in this mock control plane use finish_task and block_task"); expect(systemInstructions).toContain("The current user request defines the work for this turn"); + expect(systemInstructions).toContain("read only the context needed for that request, perform the requested action, and end the turn"); expect(systemInstructions).toContain("Do not finish or block the mock task unless the current request asks for that state change"); expect(systemInstructions).toContain( `Read-only instruction sibling root: ${context.instructions.bundle.rootPath}`, diff --git a/packages/paperclip-runner/src/cli/eval-session.ts b/packages/paperclip-runner/src/cli/eval-session.ts index c9dd23991a6..af22a154aa1 100644 --- a/packages/paperclip-runner/src/cli/eval-session.ts +++ b/packages/paperclip-runner/src/cli/eval-session.ts @@ -103,6 +103,7 @@ const EVAL_RUNTIME_INSTRUCTIONS = [ "Use the provided Paperclip semantic tools to inspect and act on the assigned task.", "Treat the seeded control-plane state as authoritative and keep every action within the requested scope.", "The current user request defines the work for this turn. Seeded task descriptions, notes, and past interaction results are background context; they do not supersede that request or establish that a newly requested action has already been performed.", + "For a bounded request, read only the context needed for that request, perform the requested action, and end the turn. A request to record a brief progress update does not require investigating unrelated history or documents.", "Task-state changes in this mock control plane use finish_task and block_task. Native paperclip_finish and paperclip_block report the provider run result but do not update the mock task. When asked to finish or block the assigned task, use its task-state semantic operation before reporting the run result.", "Do not finish or block the mock task unless the current request asks for that state change. Ending the provider turn after another requested action does not authorize additional task-state changes or completion comments.", "", diff --git a/scripts/__tests__/grok-public-install-sandbox.test.mjs b/scripts/__tests__/grok-public-install-sandbox.test.mjs index db64e36a462..f4446ec1afa 100644 --- a/scripts/__tests__/grok-public-install-sandbox.test.mjs +++ b/scripts/__tests__/grok-public-install-sandbox.test.mjs @@ -40,7 +40,7 @@ test('only the scripts-disabled dependency download gets network access', () => test('Pi assembly gets bounded scratch capacity while preserving sandbox restrictions', () => { const args = grokConsumerDockerArgs({ ...paths, download: true, temporarySizeMiB: 2048, command: ['node', '/consumer/node_modules/paperclipai/dist/index.js', 'runtime', 'setup', 'pi'] }); - assert.deepEqual(values(args, '--tmpfs'), ['/tmp:rw,nosuid,nodev,size=2048m,mode=1777']); + assert.deepEqual(values(args, '--tmpfs'), ['/tmp:rw,nosuid,nodev,noexec,size=2048m,mode=1777']); assert.deepEqual(values(args, '--memory'), ['3g']); assert.ok(args.includes('--read-only')); assert.deepEqual(values(args, '--cap-drop'), ['ALL']); @@ -50,6 +50,21 @@ test('Pi assembly gets bounded scratch capacity while preserving sandbox restric } }); +test('only an offline runtime probe can execute its verified scratch snapshot', () => { + const command = ['node', '/packages/pi-public-install-probe.mjs', '/consumer/node_modules/@paperclipai/server']; + const args = grokConsumerDockerArgs({ ...paths, command, temporarySizeMiB: 2048, temporaryExecutable: true }); + assert.deepEqual(values(args, '--tmpfs'), ['/tmp:rw,nosuid,nodev,exec,size=2048m,mode=1777']); + assert.deepEqual(values(args, '--network'), ['none']); + assert.deepEqual(values(args, '--user'), ['1001:1001']); + assert.ok(args.includes('--read-only')); + assert.deepEqual(values(args, '--cap-drop'), ['ALL']); + assert.deepEqual(values(args, '--security-opt'), ['no-new-privileges']); + assert.throws(() => grokConsumerDockerArgs({ ...paths, command, download: true, temporaryExecutable: true }), /offline/); + assert.throws(() => grokConsumerDockerArgs({ ...paths, command, temporaryExecutable: 'true' }), /offline/); + const source = readFileSync(new URL('../verify-grok-npm-install.mjs', import.meta.url), 'utf8'); + assert.ok(source.includes("temporarySizeMiB: 2048, temporaryExecutable: true")); +}); + test('the separately provisioned executable is exposed read-only to the offline probe', () => { const prerequisite = '/private/staging/native/grok'; const args = grokConsumerDockerArgs({ ...paths, prerequisite, command: ['node', '/packages/probe.mjs', 'present'] }); diff --git a/scripts/grok-public-install-sandbox.mjs b/scripts/grok-public-install-sandbox.mjs index d8ca643bfb1..12b0c75d9e9 100644 --- a/scripts/grok-public-install-sandbox.mjs +++ b/scripts/grok-public-install-sandbox.mjs @@ -7,20 +7,23 @@ export const GROK_PUBLIC_INSTALL_LIFECYCLE = [ 'npm', 'rebuild', '--offline', '--ignore-scripts=false', '--dangerously-allow-all-scripts', ]; -export function grokConsumerDockerArgs({ assets, consumer, cache, command, uid, gid, download = false, prerequisite, temporarySizeMiB = 256 }) { +export function grokConsumerDockerArgs({ assets, consumer, cache, command, uid, gid, download = false, prerequisite, temporarySizeMiB = 256, temporaryExecutable = false }) { if (!Number.isSafeInteger(uid) || uid <= 0 || !Number.isSafeInteger(gid) || gid <= 0) { throw new Error('Public-install verification requires an unprivileged host user'); } if (!Number.isSafeInteger(temporarySizeMiB) || temporarySizeMiB < 256 || temporarySizeMiB > 2048) { throw new Error('Public-install temporary storage must be bounded between 256 and 2048 MiB'); } + if (typeof temporaryExecutable !== 'boolean' || (temporaryExecutable && download)) { + throw new Error('Executable runtime snapshots require an offline verification sandbox'); + } return [ 'run', '--rm', '--platform', 'linux/amd64', '--user', `${uid}:${gid}`, '--read-only', '--cap-drop', 'ALL', '--security-opt', 'no-new-privileges', '--pids-limit', '256', '--memory', '3g', '--network', download ? 'bridge' : 'none', - '--tmpfs', `/tmp:rw,nosuid,nodev,size=${temporarySizeMiB}m,mode=1777`, + '--tmpfs', `/tmp:rw,nosuid,nodev,${temporaryExecutable ? 'exec' : 'noexec'},size=${temporarySizeMiB}m,mode=1777`, '--env', 'HOME=/tmp', '--env', 'npm_config_cache=/cache', '--env', 'npm_config_nodedir=/usr/local', '--env', 'npm_config_audit=false', '--env', 'npm_config_fund=false', diff --git a/scripts/verify-grok-npm-install.mjs b/scripts/verify-grok-npm-install.mjs index 5f44d39425c..b3226102ae1 100644 --- a/scripts/verify-grok-npm-install.mjs +++ b/scripts/verify-grok-npm-install.mjs @@ -130,7 +130,9 @@ try { assert.equal(receipt.target, 'linux-x64'); assert.equal(readFileSync(join(consumer, 'package-lock.json'), 'utf8'), consumerLock, 'Pi setup must preserve the consumer dependency graph'); // Verified launch leases also materialize the runtime in private scratch. - console.log(isolated(['node', '/packages/pi-public-install-probe.mjs', '/consumer/node_modules/@paperclipai/server'], { temporarySizeMiB: 2048 }).toString().trim()); + // Docker defaults tmpfs to noexec. The offline admission probe must execute + // its verified private snapshot while keeping lifecycle/download scratch noexec. + console.log(isolated(['node', '/packages/pi-public-install-probe.mjs', '/consumer/node_modules/@paperclipai/server'], { temporarySizeMiB: 2048, temporaryExecutable: true }).toString().trim()); } finally { rmSync(root, { recursive: true, force: true }); }