4 Commits

Include renames

Author	SHA1	Message	Date
Dotta	397de98193	feat(runner): add flagged Codex execution adapter (#12188) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Paperclip Runner now has protocol, provider, tool, package, persistence, and hidden server boundaries. > - The server still cannot select that path for a real agent heartbeat. > - A new runtime must not change any existing direct adapter. > - An experimental runtime must fail closed when its rollout flag is off. > - This pull request adds one guarded Codex vertical slice through runnerd. > - The benefit is a production-built runner path that users cannot start by default. ## Linked Issues or Issue Description Refs #11962 Refs #12111 Refs #12169 Refs #12176 **Subsystem affected** Cross-cutting. The change affects the runner package, server orchestration, shared settings, and adapter configuration UI. **Problem or motivation** The hidden PRP coordinator cannot execute a real heartbeat. The application also needs an explicit rollout boundary before it can expose the experimental runner. Existing direct adapters must keep their current execution and finalization behavior. **Proposed solution** Add `paperclip_runner` as a Codex-only adapter behind the default-off `enableNativeRunner` instance flag. Select the native runtime only for that adapter. Persist the run binding before runnerd starts. Wait for the durable PRP result and terminal event. Resume the real Codex provider thread on later heartbeats. Keep persisted native runs readable and recoverable after the flag changes. **Alternatives considered** The server could route `codex_local` through runnerd. That option would change an existing adapter and weaken rollback safety. The server could expose all providers now. That option would add unreviewed provider behavior. The build could depend on a prebuilt runner binary. That option would make source builds architecture-dependent and difficult to verify. **Roadmap alignment** This work supports the shipped enforced-outcomes, governed-tool, and self-healing-run milestones. It does not add a new roadmap surface. It is the guarded execution step after the merged hidden runner boundaries. **Additional context** This is the next replacement for the closed large runner pull request. Task-thread presentation remains a separate follow-up so this change can preserve the current direct-adapter UI. ## What Changed - Add `paperclip_runner` as an explicit Codex-only adapter. - Add the default-off `enableNativeRunner` instance flag. - Reject fresh create, hire, import, switch, and execution requests while the flag is off. - Allow edits to persisted runner agents while the flag is off. - Recover an already persisted native run even after the flag is disabled. - Keep every built-in direct adapter on its existing runtime path. - Persist an immutable native run binding and revisioned completion contract before runnerd starts. - Execute server to PRP to runnerd to Codex to server through the hidden coordinator. - Validate the durable result against the terminal event and exact completion criteria before finalization. - Preserve the Codex provider thread ID and use `thread/resume` on the next heartbeat. - Strip unsupported Codex configuration fields from the experimental adapter. - Build a target-native release runner binary from source and vendor it into the server distribution. - Install Rust only in the Docker build stage. Do not add a workflow or lockfile change. - Stop the runner process group on completion, cancellation, and forced shutdown. ## Verification - Run `pnpm --filter @paperclipai/paperclip-runner check:all`. All 69 TypeScript tests and 58 Rust tests pass. Protocol, conformance, replay, formatting, and generated-file checks pass. - Run the 12 focused adapter, settings, runtime-selection, coordinator, direct-isolation, and real Codex integration test files. All 186 tests pass. - The real integration test uses PostgreSQL, HTTP, WebSocket, runnerd, and a fake Codex app server. It proves one `thread/start` followed by one `thread/resume`. - Run `pnpm -r typecheck`. - Run `pnpm build`. - Run `pnpm check:token-gates`. - Build the Docker `build` target from a clean context. Confirm that the server distribution contains an executable `paperclip-runnerd` built with Debian Rust 1.85. - Start the server through the source-mode tsx entry point with the package `dist` directory absent. Confirm the vendor shim resolves source exports and the server boots. - Run `pnpm test:run` twice. On this macOS host, 405 files pass and 1 file skips. Eight untouched workspace and loopback tests fail because macOS resolves `/tmp` and `/var` through `/private` and because PID-derived test ports exceed 65535. Linux CI must pass the full suite. - Confirm that the diff contains 52 files. Confirm that it contains no `.github` or `pnpm-lock.yaml` change. ## Risks - The feature flag is off by default. A fresh native start fails with a stable error while the flag is off. - A persisted native run remains recoverable after the flag changes. This prevents rollout changes from corrupting recorded work. - Only local Codex execution is accepted. Other providers and remote work modes fail closed. - Existing direct adapters do not start runnerd, create native rows, use native status arbitration, or enter native finalization. - The runner receives its one-use bootstrap ticket through the child environment. The server does not put the ticket in command arguments or logs. - The server validates the company, task, agent, run, runner, session, completion contract, result, and terminal binding before it accepts completion. - The build compiles a target-native Rust binary. Cross-platform release packaging remains a later concern. Source builds and Docker builds compile for their current target. - Docker needs enough build memory for the existing server TypeScript compile. The Docker build stage sets a 4 GB V8 heap limit. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with GPT-5. The exact deployment ID and context-window size are not exposed. The model used agentic reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and applicable tests pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-25 16:03:41 -05:00
Dotta	9964b034bb	feat(runner): add hidden server PRP coordinator (#12176) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Paperclip Runner needs a narrow server trust boundary before an adapter can start it. > - The package has durable runner transport, but the server does not host or authorize that transport. > - Native persistence exists, but no writer connects PRP events to those records. > - A direct adapter must not enter this path by accident. > - This pull request adds a hidden, run-bound PRP server coordinator. > - The benefit is a recoverable server boundary that remains unavailable to normal execution. ## Linked Issues or Issue Description Refs #11962 Refs #12129 Refs #12169 **Subsystem affected** Cross-cutting. The change affects the runner package and server orchestration. **Problem or motivation** The server cannot authenticate runnerd, commit PRP events before ACK, authorize semantic tools, or enter native finalization from a durable runner result. The application must have this hidden boundary before a guarded adapter can use the runner. **Proposed solution** Add an authenticated PRP WebSocket authority and register it only for one exact persisted native Codex run. Bind each connection and event to the company, issue, agent, run, runner, session, turn, item, and verified runner identity. Commit each event before its cumulative ACK. Project only authorized same-task read tools. Rebuild the accepted result and finalization record from durable result and terminal events. **Alternatives considered** The server could expose a broad runner API key or route semantic calls through existing adapter endpoints. Those options grant too much authority and weaken replay recovery. The server could also add the user-facing adapter in this pull request. That option would mix rollout selection with the transport trust boundary and make legacy compatibility harder to review. **Roadmap alignment** This work supports the shipped enforced-outcomes, governed-tool, and self-healing-run milestones. It does not add a new roadmap surface. ## What Changed - Add the durable PRP server authority with one-use bootstrap tickets, reconnect leases, encrypted frames, bounded state, cumulative ACKs, and idempotent commands. - Add `/api/runner/v1/connect/:runId`. Derive its `ws://` or `wss://` URL from the configured Paperclip API URL. - Register one authority only after the coordinator verifies the complete native Codex run binding. - Commit validated PRP events to `heartbeat_run_events` before ACK. Reject source gaps and conflicting replays. - Rebuild accepted results and finalization records from durable result and terminal events. Enforce finalization owner leases and retry times. - Project five same-task read operations. Recheck run, agent, task, and company authority for each call. - Keep the route hidden. No adapter selects this coordinator, and no code starts runnerd. - Vendor the compiled runner TypeScript runtime into the server package while keeping the workspace package development-only for the server. - Document the package, database writer, run-log payload, and credential exclusions. ## Verification - Run `pnpm --filter @paperclipai/paperclip-runner check:all`. All TypeScript protocol checks and 69 Vitest tests pass, including commit-before-ACK crash recovery. All 43 Rust unit tests and 13 Rust integration tests pass. Conformance and replay parity pass. - Run the focused server WebSocket, coordinator, package-build, and startup-wiring suites. All 26 tests pass, including a clean-checkout reproduction with the runner `dist` directory absent. - Run `pnpm -r typecheck`. - Run `pnpm test:run`. - Run `pnpm build`. - Confirm that the diff contains 19 files. Confirm that it contains no workflow or `pnpm-lock.yaml` change. ## Risks - The server installs the WebSocket route at startup. An unregistered or malformed run path fails closed and creates no native record. - Bootstrap tickets are one use. The private state directory uses mode `0700`, and the state file uses mode `0600`. The file stores derived authentication verifiers and never stores raw tickets or lease tokens. - The journal has explicit frame, command, event-window, and file-size bounds. A bound violation closes the runner connection or rejects the command. - A runner event reaches the database before its ACK. A crash between event commit and ACK causes a byte-equivalent replay, not a second logical effect. - The coordinator accepts only an existing queued or running native Codex row with exact company, task, agent, runner, session, and completion-contract ownership. - Existing direct adapters do not call this service. They keep their current execution, transcript, result, and finalization paths. - The server has no production dependency on the private runner package. Its build copies the compiled runtime into `server/dist`; the workspace link is development-only. This adds no external package and does not change the lockfile. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with GPT-5. The exact deployment ID and context-window size are not exposed. The model used agentic reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-25 14:17:14 -05:00
DottaandPaperclip	ffff1fe6e3	feat(runner): define package API and verification boundary (#12129) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The runner package now has protocol, transport, provider, catalog, and authorization foundations. > - Its first upstream package boundary should expose only the implemented runtime and test-helper surfaces. > - Rust correctness belongs in the repository existing build verification, without introducing a parallel release process. > - Direct package creation must build the files declared by the package manifest. > - This pull request defines the minimal package API and verifies the optimized runner binaries in the existing PR and release Build jobs. > - The benefit is a production-ready runner package boundary with minimal build-process change. ## Linked Issues or Issue Description Refs #11962 This pull request replaces one bounded part of the archived large runner change. It follows the package-local authorization change in #12126. ## What Changed - Export only `@paperclipai/paperclip-runner` and `@paperclipai/paperclip-runner/testing`. - Keep Node-only fixture loading and semantic conformance helpers out of the runtime root. - Add a provider-neutral semantic conformance kit with stable JSON comparison and fail-closed input checks. - Keep deferred SDK, eval, browser, React, lab, and command surfaces private. - Pin the runner Rust toolchain to 1.97.1 with the minimal profile and `rustfmt`. - Run the Rust workspace tests in release mode. - Launch the optimized `paperclip-runnerd` and fake-harness binaries in process-level integration coverage. - Add one `pnpm --filter @paperclipai/paperclip-runner check:all` step to each existing PR and release Build job. - Make the existing server `prepack` lifecycle run its existing build after it prepares UI assets. - Document that no production adapter starts runnerd yet. This revision adds no standalone GitHub Actions job. It adds no server runner dependency or runner vendoring. It adds no Docker bootstrap or clean-consumer harness. It does not change `pnpm-lock.yaml`. ## Verification - `pnpm --filter @paperclipai/paperclip-runner check:all` - 66 TypeScript tests - 8 protocol contract tests - 56 Rust unit and integration tests - Release-mode integration coverage launches the optimized runnerd and fake-harness binaries. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/server-package-build-script.test.ts` (2 tests) - Clean `pnpm pack` from `server/` rebuilt the server and produced both `package/dist/index.js` and `package/dist/index.d.ts`. - `node --test scripts/__tests__/release-verify-workflow.test.mjs` (8 tests) - `pnpm -r typecheck` - `pnpm build` - `pnpm check:token-gates` - `git diff --check` - No `pnpm-lock.yaml` diff. - The diff changes 12 files. ## Risks The runner adds Rust work to the existing Build jobs. These jobs can take longer on a cold cache. The pinned toolchain makes contributor and CI behavior reproducible. Cargo tests use `--release` to verify optimized executables. The server prepack lifecycle now performs the build that its published entry points require. This can make direct server packing slower. This pull request does not wire runnerd into the server. It does not select runnerd for any adapter. Existing application execution and finalization paths remain unchanged. ## Model Used OpenAI Codex with GPT-5. Agentic coding mode used repository tools, code execution, and automated tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-25 09:31:48 -05:00
DottaandPaperclip	d166069bc4	Preserve built-in agent assets in server builds (#9339) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server package ships compiled runtime code plus static runtime assets. > - Built-in agent definitions live under `server/src/built-ins` and runtime code resolves them relative to compiled server files. > - The server build already copied onboarding assets into `dist`, but it did not copy built-in agent assets alongside the compiled code. > - Packaged server builds could therefore miss built-in agent definitions even though source-based development runs worked. > - This pull request extends the server build copy step to preserve built-in agent assets in `dist/built-ins`. > - The benefit is packaged server builds keep the same built-in agent runtime assets available as source-based development runs. ## Linked Issues or Issue Description No public GitHub issue found. This PR describes the underlying bug inline using the bug report template fields. ### What happened? `@paperclipai/server` build output copied `server/src/onboarding-assets` into `server/dist/onboarding-assets`, but did not copy `server/src/built-ins` into `server/dist/built-ins`. Runtime code for built-in agents resolves those assets relative to the compiled server files, so packaged builds could omit built-in agent markdown assets that are present during source-based development. ### Expected behavior Packaged server builds should include built-in agent assets under `server/dist/built-ins`, matching the runtime location expected by the compiled server code. ### Steps to reproduce 1. Check out current `master` before this PR. 2. Run `pnpm --filter @paperclipai/server build`. 3. Check for `server/dist/built-ins/agents/reflection-coach/AGENTS.md`. 4. Observe that the built-in agent asset is missing from the server build output. ### Paperclip version or commit Reproduces on current `master` before this PR. The fix is verified on commit `2b89984ccb7857f06359bf65c48222f110c7aeff`. ### Deployment mode Build/package artifact behavior. This can affect any deployment mode that runs from the built server package rather than directly from source. ### Installation method Built from source with `pnpm --filter @paperclipai/server build`. ### Agent adapter(s) involved Not adapter-specific. This is a core server packaging bug for built-in agent assets. ### Database mode Not database-related. ### Access context Not applicable. This happens during package build output generation. Related search: - Searched public PRs/issues for `built-ins build copy repo:paperclipai/paperclip`. - Found no directly related open issue. One old closed Hermes adapter PR was not directly related. ## What Changed - Updated the `@paperclipai/server` build script to create `dist/built-ins`. - Added the copy step from `server/src/built-ins` into `server/dist/built-ins` alongside the existing onboarding asset copy. - Added a focused server package build-script test that asserts both onboarding and built-in static runtime asset directories are copied into `dist`. ## Verification - `pnpm exec vitest run server/src/__tests__/server-package-build-script.test.ts` - `pnpm --filter @paperclipai/server build` - `test -f server/dist/built-ins/agents/reflection-coach/AGENTS.md` ## Risks Low risk. This changes only the package build asset copy step and adds focused test coverage. The main risk is build-script portability, but it follows the existing `mkdir -p` and `cp -R` pattern already used for onboarding assets. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5-based coding agent in a tool-enabled Paperclip heartbeat. Exact model ID and context-window size are not exposed in this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-10 07:44:08 -05:00