mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-11 13:24:55 +01:00
324e1331f8701ac57334f8404d3c613e99ddb4d9
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dotta
|
9964b034bb |
feat(runner): add hidden server PRP coordinator (#12176)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Paperclip Runner needs a narrow server trust boundary before an adapter can start it. > - The package has durable runner transport, but the server does not host or authorize that transport. > - Native persistence exists, but no writer connects PRP events to those records. > - A direct adapter must not enter this path by accident. > - This pull request adds a hidden, run-bound PRP server coordinator. > - The benefit is a recoverable server boundary that remains unavailable to normal execution. ## Linked Issues or Issue Description Refs #11962 Refs #12129 Refs #12169 **Subsystem affected** Cross-cutting. The change affects the runner package and server orchestration. **Problem or motivation** The server cannot authenticate runnerd, commit PRP events before ACK, authorize semantic tools, or enter native finalization from a durable runner result. The application must have this hidden boundary before a guarded adapter can use the runner. **Proposed solution** Add an authenticated PRP WebSocket authority and register it only for one exact persisted native Codex run. Bind each connection and event to the company, issue, agent, run, runner, session, turn, item, and verified runner identity. Commit each event before its cumulative ACK. Project only authorized same-task read tools. Rebuild the accepted result and finalization record from durable result and terminal events. **Alternatives considered** The server could expose a broad runner API key or route semantic calls through existing adapter endpoints. Those options grant too much authority and weaken replay recovery. The server could also add the user-facing adapter in this pull request. That option would mix rollout selection with the transport trust boundary and make legacy compatibility harder to review. **Roadmap alignment** This work supports the shipped enforced-outcomes, governed-tool, and self-healing-run milestones. It does not add a new roadmap surface. ## What Changed - Add the durable PRP server authority with one-use bootstrap tickets, reconnect leases, encrypted frames, bounded state, cumulative ACKs, and idempotent commands. - Add `/api/runner/v1/connect/:runId`. Derive its `ws://` or `wss://` URL from the configured Paperclip API URL. - Register one authority only after the coordinator verifies the complete native Codex run binding. - Commit validated PRP events to `heartbeat_run_events` before ACK. Reject source gaps and conflicting replays. - Rebuild accepted results and finalization records from durable result and terminal events. Enforce finalization owner leases and retry times. - Project five same-task read operations. Recheck run, agent, task, and company authority for each call. - Keep the route hidden. No adapter selects this coordinator, and no code starts runnerd. - Vendor the compiled runner TypeScript runtime into the server package while keeping the workspace package development-only for the server. - Document the package, database writer, run-log payload, and credential exclusions. ## Verification - Run `pnpm --filter @paperclipai/paperclip-runner check:all`. All TypeScript protocol checks and 69 Vitest tests pass, including commit-before-ACK crash recovery. All 43 Rust unit tests and 13 Rust integration tests pass. Conformance and replay parity pass. - Run the focused server WebSocket, coordinator, package-build, and startup-wiring suites. All 26 tests pass, including a clean-checkout reproduction with the runner `dist` directory absent. - Run `pnpm -r typecheck`. - Run `pnpm test:run`. - Run `pnpm build`. - Confirm that the diff contains 19 files. Confirm that it contains no workflow or `pnpm-lock.yaml` change. ## Risks - The server installs the WebSocket route at startup. An unregistered or malformed run path fails closed and creates no native record. - Bootstrap tickets are one use. The private state directory uses mode `0700`, and the state file uses mode `0600`. The file stores derived authentication verifiers and never stores raw tickets or lease tokens. - The journal has explicit frame, command, event-window, and file-size bounds. A bound violation closes the runner connection or rejects the command. - A runner event reaches the database before its ACK. A crash between event commit and ACK causes a byte-equivalent replay, not a second logical effect. - The coordinator accepts only an existing queued or running native Codex row with exact company, task, agent, runner, session, and completion-contract ownership. - Existing direct adapters do not call this service. They keep their current execution, transcript, result, and finalization paths. - The server has no production dependency on the private runner package. Its build copies the compiled runtime into `server/dist`; the workspace link is development-only. This adds no external package and does not change the lockfile. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with GPT-5. The exact deployment ID and context-window size are not exposed. The model used agentic reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
ffff1fe6e3 |
feat(runner): define package API and verification boundary (#12129)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The runner package now has protocol, transport, provider, catalog, and authorization foundations. > - Its first upstream package boundary should expose only the implemented runtime and test-helper surfaces. > - Rust correctness belongs in the repository existing build verification, without introducing a parallel release process. > - Direct package creation must build the files declared by the package manifest. > - This pull request defines the minimal package API and verifies the optimized runner binaries in the existing PR and release Build jobs. > - The benefit is a production-ready runner package boundary with minimal build-process change. ## Linked Issues or Issue Description Refs #11962 This pull request replaces one bounded part of the archived large runner change. It follows the package-local authorization change in #12126. ## What Changed - Export only `@paperclipai/paperclip-runner` and `@paperclipai/paperclip-runner/testing`. - Keep Node-only fixture loading and semantic conformance helpers out of the runtime root. - Add a provider-neutral semantic conformance kit with stable JSON comparison and fail-closed input checks. - Keep deferred SDK, eval, browser, React, lab, and command surfaces private. - Pin the runner Rust toolchain to 1.97.1 with the minimal profile and `rustfmt`. - Run the Rust workspace tests in release mode. - Launch the optimized `paperclip-runnerd` and fake-harness binaries in process-level integration coverage. - Add one `pnpm --filter @paperclipai/paperclip-runner check:all` step to each existing PR and release Build job. - Make the existing server `prepack` lifecycle run its existing build after it prepares UI assets. - Document that no production adapter starts runnerd yet. This revision adds no standalone GitHub Actions job. It adds no server runner dependency or runner vendoring. It adds no Docker bootstrap or clean-consumer harness. It does not change `pnpm-lock.yaml`. ## Verification - `pnpm --filter @paperclipai/paperclip-runner check:all` - 66 TypeScript tests - 8 protocol contract tests - 56 Rust unit and integration tests - Release-mode integration coverage launches the optimized runnerd and fake-harness binaries. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/server-package-build-script.test.ts` (2 tests) - Clean `pnpm pack` from `server/` rebuilt the server and produced both `package/dist/index.js` and `package/dist/index.d.ts`. - `node --test scripts/__tests__/release-verify-workflow.test.mjs` (8 tests) - `pnpm -r typecheck` - `pnpm build` - `pnpm check:token-gates` - `git diff --check` - No `pnpm-lock.yaml` diff. - The diff changes 12 files. ## Risks The runner adds Rust work to the existing Build jobs. These jobs can take longer on a cold cache. The pinned toolchain makes contributor and CI behavior reproducible. Cargo tests use `--release` to verify optimized executables. The server prepack lifecycle now performs the build that its published entry points require. This can make direct server packing slower. This pull request does not wire runnerd into the server. It does not select runnerd for any adapter. Existing application execution and finalization paths remain unchanged. ## Model Used OpenAI Codex with GPT-5. Agentic coding mode used repository tools, code execution, and automated tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
42b8f7ab2f |
feat(runner): authorize semantic tool dispatch (#12126)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The runner package defines a provider-neutral protocol and semantic action catalog. > - Catalog membership alone must not grant access to an action. > - Each run needs current company, actor, task, claim, mode, and application-binding authority. > - Mutating actions also need safe retry behavior and durable receipts. > - This pull request adds a package-local authority and dispatch layer. > - The benefit is a small and testable trust boundary before server integration lands. ## Linked Issues or Issue Description Refs #11962 This pull request replaces one bounded part of the archived large runner change. ## What Changed - Add run-scoped tool projection and optional tool discovery. - Require an explicit application binding before an action is visible. - Intersect actor claims with claims delegated to the run. - Recheck company, actor, task, mode, state, role, claim, and policy authority before each call. - Validate action input and output with the canonical catalog schemas. - Redact protected values and keep raw tool content out of semantic receipts. - Require atomic idempotency claims for mutating actions. - Replay exact completed retries and reject changed or concurrent retries. - Recover a durable completed receipt if the primary receipt commit fails, without re-executing the mutation. - Add bounded authorization records and PRP semantic input and result receipts. - Document that this change adds no server binding or production tool installation. ## Verification - `pnpm --filter @paperclipai/paperclip-runner check:all` - `pnpm -r typecheck` - `pnpm check:token-gates` - `pnpm build` - 60 package TypeScript tests pass. - 56 Rust unit and integration tests pass. - Protocol, replay, and cross-language conformance checks pass. - `pnpm test:run` completed with 4,684 passing and 19 skipped tests. It reproduced 32 local baseline failures across 9 unchanged server files; all corresponding hosted test shards pass. - Every applicable GitHub Actions gate passes. The Storybook job skipped because this PR has no UI changes. - Socket and Snyk pass with no findings. Superagent completed neutral with zero annotations because its external sandbox did not start within 120 seconds. - Greptile is 5/5 with no unresolved actionable comments. - The diff changes 11 files. ## Risks The main risk is an authorization or idempotency error at the tool boundary. The dispatcher fails closed for malformed authority, unavailable receipt storage, stale authority, unauthorized actions, protected input, invalid binding output, and unrecoverable receipt completion. The receipt store must recover a completed mutation outcome idempotently if its primary commit fails; otherwise the claim remains reserved for operator recovery rather than allowing automated re-execution. Unbound actions are absent. No server or provider installs these tools in this change. Existing adapters and application behavior do not change. ## Model Used OpenAI Codex with GPT-5. Agentic coding mode used repository tools, code execution, and automated tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run the affected tests locally and they pass; full-suite baseline exceptions are documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
23048f1219 |
Add canonical semantic action catalog to Paperclip Runner (#12121)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip Runner now has a durable PRP transport and a Codex provider bridge. > - Codex must use stable, provider-neutral action contracts before Paperclip can grant run-scoped tool access. > - A catalog must describe actions without granting permission to discover or invoke them. > - Generated inventory must stay synchronized with its TypeScript source. > - This pull request adds the canonical Codex-spine semantic action catalog inside the runner package. > - The benefit is a small review unit for schemas and inventory before authorization and dispatch land. ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting. This pull request extends private runner infrastructure in `packages/paperclip-runner`. **Problem or motivation** The Codex provider bridge has no canonical description of the Paperclip actions that a later authorization layer can project into a run. Independent operation lists can drift in names, claims, task modes, effects, and input bounds. **Proposed solution** Add one immutable v1 catalog for the first 27 Codex-spine actions. Give each action a stable identifier, placement, effect, required claims, supported task modes, and JSON Schema input and output contracts. Generate a deterministic JSON inventory from that source and fail package checks on drift. **Alternatives considered** The combined runner branch contains larger live and scenario catalogs with authorization, bindings, labs, and other providers. That change is too large for this review unit. A generic API escape hatch would also bypass the operation-level boundary, so this catalog excludes it. **Roadmap alignment** This work supports the governed tool access direction in `ROADMAP.md`. It does not add a tool gateway, application binding, server endpoint, or production authorization decision. **Additional context** Refs #12111 and #11962. Pull request #12111 was squash-merged first. This branch starts at the resulting `master` commit. Its delta is 10 files. ## What Changed - Added 27 versioned, provider-neutral semantic action declarations for the Codex spine. - Added bounded JSON Schema input contracts and normalized operation receipt output contracts. - Added placement, effect, claim, mode, and role metadata. - Added a deeply frozen public catalog and an operation lookup helper. - Added a deterministic checked-in JSON inventory and generation commands. - Added a byte-for-byte drift gate to the package build. - Added AJV schema compilation, mutation-bound, forged-field, immutability, inventory, and non-executable-boundary tests. - Exported only the catalog types and declarations from the existing package root. - Documented that catalog membership does not grant discovery, authorization, dispatch, or application binding. - Kept server code, UI code, other providers, scenario-only actions, labs, generic API access, authorization, dispatch, and receipts processing out of this pull request. ## Verification - `pnpm --filter @paperclipai/paperclip-runner check:all` passes. - TypeScript protocol tests pass: 8 Node tests and 49 Vitest tests. - All package Rust tests and conformance and replay parity checks pass. - `pnpm --filter @paperclipai/paperclip-runner check:semantic-action-catalog` passes. - `pnpm -r typecheck` passes. - `pnpm build` passes. - `pnpm check:token-gates` passes. - Prettier and `git diff --check` pass for the changed source and documentation files. - The generated catalog matches its source byte for byte. - The secret scan is clean. - The delta against `master` is 10 files. `pnpm-lock.yaml` is unchanged. - `pnpm test:run` completed locally with 4,692 passing tests, 19 skipped tests, and 24 failures in 8 unchanged server test files. The failures reproduce the established local macOS path-alias, listener, and workspace-runtime baseline. No changed-file test failed. Linux CI remains the repository handoff authority. - The full Linux PR workflow passes, including the aggregate `verify` gate. - Snyk, Socket, Superagent security, and supply-chain checks pass. - Greptile is 5/5 with no actionable comments, recommendations, or follow-ups. - Storybook visual regression skipped by design because this pull request changes no UI file. - Browser and migration tests are not applicable because this pull request changes no server, UI, database, or migration file. ## Risks Production behavior is unchanged because no consumer projects this catalog into a provider run. The main risks are contract drift, unbounded mutation input, forged scope fields, accidental executable authority, and generated inventory drift. Closed input schemas, explicit bounds, a frozen catalog, tests, and the byte drift gate cover these risks. The later authorization layer must still bind every action to the active run and company before discovery or invocation. I checked `ROADMAP.md`. This change is private contract infrastructure for the governed tool access direction. It does not duplicate a shipped or public product surface. ## Model Used OpenAI Codex with GPT-5 was used. The exact serving model ID and context size were not exposed. The model used high reasoning, repository tools, GitHub tools, and local code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
b2d1673b9e |
Add TypeScript PRP replay contracts (#12091)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip Runner needs one typed interpretation of the language-neutral PRP contract. > - The JSON Schemas and fixtures now exist, but TypeScript consumers cannot validate or replay them yet. > - A deterministic reducer must define how duplicate delivery and source gaps affect the projected session. > - Result and question contracts must also validate untrusted provider and user input before later runtime code uses it. > - This pull request adds those TypeScript contracts and replay oracles without adding a process, provider, endpoint, or production behavior. > - The benefit is a reviewable and testable TypeScript foundation for the local runner and transport pull requests. ## Linked Issues or Issue Description **Subsystem affected** This change affects the private `@paperclipai/paperclip-runner` package. It does not change an existing server or adapter execution path. **Problem or motivation** The PRP v1 schemas do not yet provide TypeScript types, runtime validators, normalized result handling, or a deterministic session projection. Later Rust, transport, provider, and server work needs one tested TypeScript oracle instead of separate interpretations. **Proposed solution** Generate a checked-in TypeScript schema bundle from the PRP v1 sources. Add derived types, AJV validation, result and question validation, deterministic replay, a reducer, and generated golden snapshots. Export only these implemented root-package surfaces. **Alternatives considered** The combined runner branch adds the TypeScript contracts together with Rust, providers, semantic authorization, SDKs, labs, and server behavior. That delta is too large for normal review. Handwritten duplicate protocol types would also create a drift risk. **Roadmap alignment** This work supports the governed tool and control-plane direction in `ROADMAP.md`. It does not enable a new production adapter or endpoint. **Additional context** Refs #12087 and #11962. This pull request was prepared on #12087, then rebased onto its squash merge before opening. The current delta against `master` is 37 files. ## What Changed - Added JSON-Schema-derived PRP v1 types and AJV runtime validation. - Added fail-closed required-version checks and cross-envelope binding checks. - Added provider-neutral completion-result and structured-question contracts. - Added normalization for accepted legacy provider result aliases before strict validation. - Added a deterministic session reducer for replay, duplicate delivery, source gaps, requests, items, results, and terminal state. - Added generated replay snapshots and compact parity summaries for six accepted fixtures. - Added schema-bundle, manifest, and replay-golden drift gates. - Added only the root package export. Deferred testing, SDK, evaluation, lab, provider, and browser entry points remain unavailable. ## Verification - `pnpm --filter @paperclipai/paperclip-runner test` passed with 8 protocol tests and 44 TypeScript tests. - `pnpm --filter @paperclipai/paperclip-runner typecheck` passed. - `pnpm --filter @paperclipai/paperclip-runner check:replay-goldens` passed. - `pnpm -r typecheck` passed. - `pnpm build` passed. - `pnpm check:token-gates` passed. - `git diff --check` passed. - The delta against its declared base is 37 files. - `pnpm test:run` was executed locally. The package tests pass, while the macOS repository run retains the unchanged local-environment failures documented on #12087. The complete Linux CI matrix must pass on this commit. - A scoped scan found no secret-like values, internal references, or deferred-provider file names. - Greptile found an unbounded sequence-gap allocation. Commit `4a405c17` caps detailed missing IDs at 256, records the full missing count and truncation state, and rejects sequence values above the exact JavaScript integer range. The focused tests, workspace typecheck, build, and token gates pass after this fix. ## Risks Low production risk. The package remains private. This change adds no process, network endpoint, provider bridge, server integration, database change, or execution selection. The main risk is protocol interpretation drift. Generated schema and replay gates detect that drift. Browser and CSP-specific validator packaging remains deferred to its later package boundary. I checked `ROADMAP.md`. This change defines contracts for planned control-plane work and does not add overlapping product behavior. ## Model Used OpenAI Codex with GPT-5 was used. The exact serving model ID and context size were not exposed. The model used high reasoning, repository tools, GitHub tools, and local code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |