2 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	b76e36d6cf	Add durable PRP transport and recovery (#12100) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The package-local runner can supervise a local process, but it cannot yet survive a broken controller connection. > - A production transport must authenticate both peers without putting the bootstrap secret on the wire. > - Commands and events must remain bounded, ordered, and recoverable across reconnects and crashes. > - Retrying an uncertain side effect is unsafe, so indeterminate outcomes must fail closed instead of running twice. > - This pull request adds those transport and recovery guarantees inside the runner package only. > - The benefit is a durable PRP boundary that can be reviewed before any provider or server integration exists. ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting. This pull request extends private transport infrastructure in `packages/paperclip-runner`. **Problem or motivation** The local runner introduced by #12095 has no authenticated network handshake, durable outbox, reconnect lease, cumulative acknowledgement, or crash-safe command journal. A dropped connection could otherwise lose an event or tempt a controller to repeat a side effect whose outcome is unknown. **Proposed solution** Add an authenticated PRP v1 WebSocket transport, encrypted frames, lease-based reconnects, a bounded durable event outbox, cumulative acknowledgements, and an idempotent command journal. Preserve pending commands before execution and classify the crash window as indeterminate so an uncertain side effect is never repeated automatically. **Alternatives considered** The combined runner branch implements transport together with Codex, semantic tools, and server coordination. That change is too large for one review unit. Keeping transport in memory would make reconnect and crash recovery unverifiable. Re-running a pending command after restart would weaken the at-most-once side-effect boundary. **Roadmap alignment** This work supports the governed tools and self-healing run direction in `ROADMAP.md`. It does not add a production provider, server endpoint, adapter, feature flag, or user-facing behavior. **Additional context** Refs #12095 and #11962. Pull request #12095 was squash-merged first. This branch has been rebased onto the resulting `master` commit, and its current delta is 13 files. ## What Changed - Added a loopback-only WebSocket connection policy with one-time DNS resolution and pinned reconnect addresses. - Added an HMAC mutual-authentication handshake that never sends the bootstrap ticket over the socket. - Added AES-256-GCM secure frames with per-direction keys, monotonic counters, and session-bound authenticated data. - Added one-use bootstrap-ticket handling and lease-based reconnect validation with expiry, revocation, and epoch checks. - Added a private, symlink-resistant state directory with atomic, synchronized state replacement. - Added a bounded durable event outbox, priority-zero reserve, cumulative acknowledgements, and reconnect replay of only the unacknowledged suffix. - Added a bounded command journal with contiguous sequence enforcement, persistent results, and deterministic duplicate responses. Duplicate replay requires a SHA-256 match over the complete canonical command. - Persisted commands before their effects. A crash after persistence but before result storage returns an indeterminate terminal result and does not execute the command again. - Migrated pre-fingerprint command journals by compacting through their persisted controller cursor. Legacy redelivery fails closed instead of reconstructing an incomplete identity or repeating an uncertain effect. - Added strict limits and validation for frames, state, results, outbox entries, command history, and redacted diagnostics. - Added a transport-only `paperclip-runnerd --connect-url` mode. It handles lifecycle commands and rejects provider commands because no provider is present in this pull request. - Added a full disconnect-before-ack fault test that reconnects with the lease, replays identical command and event state, and proves the effect ran once. - Kept provider transports, semantic tools, server integration, and production runtime selection out of this pull request. ## Verification - `pnpm --filter @paperclipai/paperclip-runner check:all` passes. - TypeScript contract tests pass: 8 Node tests and 44 Vitest tests. - Rust tests pass: 33 unit tests, 3 public durable-recovery integration tests, plus the existing 2 local-runner and 3 process-supervisor tests. - The disconnect-before-ack, lease reconnect, duplicate command, malformed state, unknown command, bounds, and crash-window tests pass. - Rust conformance and replay parity checks pass against the shared PRP fixtures. - `cargo clippy --workspace --all-targets -- -A clippy::filter-map-bool-then -D warnings` passes. The narrow allow covers an unchanged replay implementation from the preceding contract pull request. - `pnpm -r typecheck` passes. - `pnpm build` passes. - `pnpm check:token-gates` passes. - `git diff --check` passes. - The delta against `master` is 13 files. The package lockfile is unchanged. - `pnpm test:run` completed locally with 4,686 passing tests, 19 skipped tests, and 30 failures in 8 unchanged server test files. The failures reproduce the established local macOS path-alias, listener, port-range, and workspace-runtime baseline. No changed-file test failed; Linux CI remains the repository handoff authority. - Storybook visual regression is not applicable because this pull request changes no UI or story files. ## Risks Production behavior is unchanged because no server code starts or connects to this transport. The main risks are secret disclosure, forged or replayed frames, state corruption, unbounded disk growth, duplicated side effects, and incorrect recovery. Mutual authentication, encrypted counter-bound frames, private atomic state, explicit bounds, cumulative acknowledgements, a durable command journal, fail-closed indeterminate recovery, and fault-injection tests cover these risks. I checked `ROADMAP.md`. This change is private transport infrastructure for planned control-plane work. It does not duplicate a shipped or public product surface. ## Model Used OpenAI Codex with GPT-5 was used. The exact serving model ID and context size were not exposed. The model used high reasoning, repository tools, GitHub tools, and local code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-24 12:55:08 -05:00
DottaandPaperclip	6b20cc97cc	Add local fake runner supervision (#12095) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip Runner needs a small local process model before it can connect to a production provider or server. > - The TypeScript PRP contracts now define the expected replay behavior. > - A second language implementation must produce the same result from the same fixtures. > - Local child processes also need bounded input, bounded output, and complete descendant cleanup. > - This pull request adds a package-local Rust runner, a scripted fake harness, and deterministic parity checks. > - The benefit is a testable process boundary with no production Paperclip behavior change. ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting. This pull request adds private test infrastructure to `packages/paperclip-runner`. **Problem or motivation** The PRP contracts have no second implementation on `master`. There is also no small harness that can prove process cleanup, command idempotency, terminal reconciliation, or bounded JSONL handling without a production provider. **Proposed solution** Add a minimal Rust workspace. Add a local runner process, a scripted fake harness, a bounded process supervisor, and Rust conformance and replay checks. Keep all binaries package-local. Do not connect them to the Paperclip server. **Alternatives considered** The combined runner branch includes provider transports, durable networking, SDKs, labs, and server behavior. That change is too large for this review unit. A TypeScript-only harness would not test cross-language contract parity. **Roadmap alignment** This work supports the governed tools and self-healing run direction in `ROADMAP.md`. It does not add a user-facing runtime, adapter, endpoint, or rollout flag. **Additional context** Refs #12091 and #11962. Pull request #12091 was merged before this branch opened. This branch is based on the current `master`. Its delta is 25 files. ## What Changed - Added a minimal locked Rust workspace with only `serde` and `serde_json` dependencies. - Added a package-local `paperclip-runnerd` local mode and a scripted fake harness. - Added bounded controller input, harness input, subprocess output queues, line sizes, log retention, script sizes, script steps, and command history. - Added contiguous controller and harness sequence checks and equivalent-command replay handling. - Added process-group supervision that cleans up child processes and remaining descendants after forced or natural harness exit. - Added runner-owned terminal reconciliation for success, failure, interruption, cancellation, controller closure, and protocol failure. - Added Rust conformance output and deterministic replay summaries for the shared PRP fixtures. - Added fake scripts for success, failure, interruption, interaction, duplicate terminal output, process cleanup, and oversized output. - Added package scripts and documentation for the Rust and cross-language checks. - Kept provider transport, server integration, semantic tools, and production runtime selection out of this pull request. ## Verification - `pnpm --filter @paperclipai/paperclip-runner check:all` passes. - TypeScript contract tests pass: 8 Node tests and 44 Vitest tests. - Rust tests pass: 20 unit tests, 2 local-runner tests, and 3 process-supervisor tests. - The Rust conformance and replay parity checks pass against the shared fixtures. - The natural-exit and forced-exit tests confirm that the harness and its worker process are stopped. - The oversized-frame test confirms that a harness frame above the configured limit is rejected. - `pnpm -r typecheck` passes after the final rebase to `master`. - `pnpm build` passes after the final rebase to `master`. - `pnpm check:token-gates` passes. - `git diff --check` passes. - The delta against `master` is 25 files. The package lockfile is unchanged. - `pnpm test:run` completed locally with 4,686 passing tests, 19 skipped tests, and 30 failures in 8 unchanged server test files. The failures are local macOS path-alias, listener, port-range, and workspace-runtime baseline failures. No changed-file test failed, and every applicable Linux CI shard passes. - Storybook visual regression skipped intentionally because this pull request changes no UI or story files. ## Risks Low production risk. No server code invokes the new binaries. The package remains private. The main risks are process leaks, unbounded local input, and cross-language drift. Bounded queues and sizes, process-group cleanup tests, fixture manifests, and parity checks cover these risks. I checked `ROADMAP.md`. This change is private test infrastructure for planned control-plane work. It does not duplicate a shipped or public product surface. ## Model Used OpenAI Codex with GPT-5 was used. The exact serving model ID and context size were not exposed. The model used high reasoning, repository tools, GitHub tools, and local code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-24 12:16:48 -05:00