mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-10 01:50:13 +01:00
c66b4a55efdc5675b5a5146197078b94de6407da
14
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DottaandPaperclip
|
c6573962f8 |
fix(workspaces): recover full sandboxes through read-only export (#19465)
## Thinking Path > - Paperclip preserves agent work across failed runs and server restarts. > - A remote workspace must have a durable save before its source can be released. > - Portable capture normally writes a Git mirror and a full archive inside the sandbox. > - A full sandbox can prevent capture before the provider's streaming download starts. > - Retrying that write cannot create the required free space. > - This change transfers a checked standalone source and seals its archive on the controller. > - The source remains retained unless the existing durable save completes. ## Linked Issues or Issue Description Related: #18644 reduced temporary capture space. #18172 added the provider's disk-free directory transfer. This repair connects portable capture to that transfer when the archive itself cannot be created. **What happened?** Workspace capture returns `remote_workspace_sandbox_disk_full` after provider work has finished. File recovery repeats the same sandbox write. **Expected behavior** Recover a stable standalone workspace without requiring another full archive on the source disk. Preserve staged changes, untracked work, confined links and the original source. **Steps to reproduce** 1. Create a standalone Git repository with a commit, staged changes and an untracked result. 2. Use the existing real-command transport fixture. Inject ENOSPC into the sandbox helper write or archive write. 3. Run portable capture. Before this change it rejects the helper failure. 4. With the change, restore the saved archive after removing the fixture source. Check the commit, staged content, untracked result and Git connectivity. ## What Changed - Attempt one controller-side capture after an exact sandbox disk-full failure. - Read a source inventory without installing a helper. Use the existing provider bulk transfer with confined links and runtime exclusions. - Compare the transferred inventory with source fingerprints before and after sealing. - Reject copied Git pointers, alternates and administrative symlinks. Parse config as data without includes; allow only portable keys before controller Git normalization. - Preserve socket exclusions. Retain the source and remove incomplete controller staging after failure. - Update execution semantics and focused regression coverage. No database migration or new execution authority. ## Verification - TDD red on base `2f35eb1080`: the new full-sandbox test failed with `remote_workspace_sandbox_disk_full` before production edits. - Green: 202 transport and archive tests passed before the additional Git configuration guards. At head `8541086cb5`, all 116 transport tests and the PostgreSQL durability acceptance pass. The final `e24e173ec9` exclusion guard passes those 116 cases plus its new Git-history regression across the full run and focused rerun. Environment: macOS arm64, Node 26.4.0; real Node/Git commands and filesystem transfer. - Review regressions were also red before their fixes: an internal Git config symlink bypassed the check, and `core.attributesFile` could select a controller path. Git administrative symlinks and nonportable config now fail before normalization. The focused safety selection passes 11 cases. - Cases cover helper and archive ENOSPC, commit/staged/untracked continuity after source removal, confined links, sockets, changed source, corrupt or extra copied files, unsafe Git metadata, retained diagnostic text and failed download. - Real PostgreSQL acceptance passes: normal workspace save recovers one sandbox ENOSPC, commits the encrypted snapshot, clears its writer, loses the original allocation, then independently reloads the saved result twice. The first acceptance attempt used an incorrect expected status (`available`); the documented state is `committed`. No production change was needed for that correction. - Review regressions for untracked Python/Node environments and excessive Git roots failed before their fixes. Dependency exclusions now match normal capture. Controller parsing starts only after whole-tree/repository budgets and layout checks; refspec/settings budgets and a shared two-minute verification/normalization deadline bound further work. The excessive-root test verifies zero Git subprocesses. - Real provider-boundary red/green: Daytona direct tar and capacity-fallback PTY transfers first dropped `._result.txt`. The new optional mapping contract preserves all user names and excludes only current transfer scratch. Kubernetes implements the same contract. Explicit caller exclusions and archive validation remain in force. - Standalone Linux qualification on Node 24.22.2/Debian bookworm: fresh shared/SDK build; Daytona and Kubernetes typecheck and build; Daytona file-sync/capacity/diagnostics 68/68 pass; Kubernetes file-sync 31/31 pass, including the actual `/proc` case. Qualification was refreshed again on the final `be6f7feacc` source tree: current shared/SDK rebuilt; all 815 staged source/config/manifest hashes match the delivered tree before and after testing. An initial container run as root bypassed a read-only file test; the same source passes as uid 1000. Installing zstd enabled eight initially skipped cases. - A further review regression proved `.git/pyvenv.cfg` could exclude Git history as a dependency. The inventory now handles `.git` before dependency filtering, matching normal capture. The full capture/restore test verifies the original commit and saved work. The first green attempt incorrectly compared command-result object identity; correcting the test to compare stdout passed with no production change. - Further red/green regressions cover read-only private-copy cleanup, a regular file replaced by a FIFO before open, case aliases of Git administrative metadata, and permission preservation under restrictive umask. Nonblocking no-follow file handles verify identity before and after reading. Cleanup changes only private controller directories. Provider extraction preserves ordinary permissions, refuses special mode bits before writing, and does not adopt source ownership, ACLs, attributes or flags. Final macOS Daytona tests pass 65/65; the final Linux qualification above was repeated on these exact provider sources. - Focused strict TypeScript compilation of the inventory and archive modules passes. - The first sandboxed archive run failed three Unix socket fixtures with EPERM. The permitted run outside that filesystem sandbox passed all tests. - Initial full local server typecheck could not resolve the unbuilt Runner package and produced cascading errors. Exact-head `e24e173ec9` CI typecheck, build, Runner and browser checks pass. That CI attempt failed two server shards: the unchanged retained-cleanup backlog test timed out at 15 seconds and passes locally unchanged; ten cache-reclaim fixture oracles failed because directory fallback consumed their sealed-archive counters/faults. Final commit `caed4e4f83` isolates the intended unavailable-directory boundary while preserving archive counts, recovery status, integrity and no-second-reclamation assertions. Those ten failures reproduced locally before the fixture edit. The full affected file passes 394/395; its sole other HTTP socket-hang-up case passes unchanged in an isolated rerun. This is combined coverage of all 395 cases, not a single complete green run. The final branch integrates master `53645a0104` at `98ea07225592930cbb0d18bcf250bc64f6bb13b4`, preserving upstream workspace phase timings. After integration: 117 transport cases, the PostgreSQL durability acceptance, the existing object-storage/timing regression, and 12 affected cache cases pass. The expanded recovery file now has 421 cases. Final-head `be6f7feacc45c1e32a26a2417e973f1e3752fd8d` CI passes all 55 applicable checks (three skipped, one neutral review), including the expanded recovery suite, typecheck, builds and Runner coverage. [CI result](https://github.com/paperclipai/paperclip/actions/runs/42251345554). - Provider exclusion regressions were red through four real Linux transfer paths: Daytona direct archive, direct safe-link repair, capacity PTY repair, and Kubernetes streaming. A root socket and nested regular file share a basename; literal glob-character files/subtrees also survive. Preserve mode now anchors tar exclusions as literal root-relative paths and escapes `find -path` pruning. Default callers keep glob semantics. Receipt-count regressions were also red before correcting preserved-output counting. All 31 Kubernetes and 68 Daytona tests, builds and typechecks pass on the final source with 815 matching inputs. - The final semantics documentation explicitly records special mode bits, the stricter config allowlist (including innocuous unlisted keys), and provider qualification limited to Daytona/Kubernetes; these address the optional current-head review observations without broadening controller Git authority. - No Daytona credential is available in this session. Provider transfer is a local filesystem boundary in these tests. Hosted task recovery and rollout remain unverified. ## Risks - Recovery reads the source repeatedly and transfers the standalone tree. It can take longer than normal capture. - Linked worktrees, initialized submodules, Git administrative symlinks, nonportable Git configuration, special mode bits and alternate object stores remain retained if the normal capture cannot finish. This fallback does not follow their pointers onto the controller. - A provider without usable directory export, a changed source, or insufficient controller space cannot produce a receipt. The original source stays retained. - The server, SDK and affected provider plugins must be built from this revision together. An older provider can fail inventory verification on names it omits; it cannot publish an incomplete save. No merge, publication or deployment is included. ## Model Used OpenAI GPT-6 family through Codex, with reasoning, code editing, shell tools and automated tests. The exact serving model ID and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandClaude Opus 5.5
|
83c6832ccb |
fix(build): typecheck package, Playwright, and e2e test files in CI (#18839)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Runner, plugin, and Playwright tests verify how those agents run, connect, and appear in the board > - Most package tsconfigs exclude tests, and Vitest strips types, so a type error in those tests fails nothing > - The tests drifted to 803 real type errors, and some of those errors hid assertions that checked nothing > - paperclipai/paperclip#18739 and paperclipai/paperclip#18738 check the server test tree and four test projects, but not these files > - This pull request adds a required check for these test files, fixes every error, and fails CI if a test file is in no program > - The benefit is that a type error in any test file outside `server/src` now fails CI ## Linked Issues or Issue Description Refs #18739 Refs #18738 **What happened?** 517 test files were in no program that required CI checks. A type error in those files did not fail CI. **Expected behavior** CI fails on a type error in any test file. **Steps to reproduce** Add `export const value: number = "text";` to a test file under `packages/paperclip-runner/src` on master. CI stays green, because `tsconfig.json` excludes test files and Vitest strips types. **Measured gap** Each row is a program with its tests included, on one thread. File and error counts are for head `7d4b5e907a`. Memory peaks were measured before the two latest merges of master. | Group | Test files | Real errors | Peak | | --- | ---: | ---: | ---: | | `paperclip-runner` Node source | 290 | 644 | 1.09 GiB | | `paperclip-runner` surfaces | 9 | 12 | 0.42 GiB | | `paperclip-runner` devtools and examples | 14 | 0 | 0.68 GiB | | `sandbox-providers/kubernetes` | 21 | 65 | 0.44 GiB | | 8 other sandbox providers and `bridge-template` | 32 | 4 | 0.34 GiB | | `plugins/sdk` | 22 | 15 | 0.41 GiB | | `plugin-workspace-diff` | 5 | 0 | 0.37 GiB | | `plugin-orchestration-smoke-example` | 1 | 1 | 0.15 GiB | | `kv-demo-mcp-server` | 3 | 0 | 0.13 GiB | | `packages/db` scripts | 1 | 0 | 1.01 GiB | | 8 Playwright suites under `tests/` | 52 | 25 | 1.72 GiB | | `tests/runner-acceptance` | 3 | 0 | 0.58 GiB | | `tests/e2e`, through layer declarations | 63 | 37 | at most 1.59 GiB | | `ui` `AttentionInteractionResolver.test.tsx` | 1 | 0 | n/a | | **Total** | **517** | **803** | | 27 of the 803 errors came from master while this pull request was open. Each merge of master brought new test code that no check had read. Errors that were config errors, not real errors: - One TS2688 for each provider, because the root install does not install provider dependencies. - 9 in the Runner Node program (TS7016 and TS6059). - 2 in `tsconfig.browser.json`. TypeScript 7 removed `baseUrl`, so that config was invalid. - 213 in the Playwright suites (`@/*` aliases and Vite types). - About 60 in `tests/e2e` (import extensions and aliases). `packages/plugins/create-paperclip-plugin` has no gap. Its tsconfig excludes its test, but `cli/tsconfig.json` reaches that test through an import. `server/test-baselines` also has no gap, because `tests/lifecycle-baseline` reads it. ## What Changed - Added `scripts/typecheck-test-programs.mjs` and its tests. The `Typecheck + Release Registry` job runs it. It checks each program in `PROGRAMS` on one thread. - The script fails if a tracked `*.test.ts(x)` or `*.spec.ts(x)` file outside `server/src` is in no program. The compiler file list decides this, because `include` drops `foo.test.tsx` when `foo.test.ts` exists. - Added a `tsconfig.test.json` to each uncovered package, and `tests/tsconfig.playwright.json`. - Added `tests/e2e` to `PROJECTS` in `scripts/typecheck-test-projects.mjs`, with a `tsconfig.json` and a `tsconfig.layers.json`. - Added `scripts/typecheck-sandbox-providers`. It is a small workspace that installs provider SDKs from one lockfile. I did not add a lockfile inside each provider, because `scripts/build-standalone-public-packages.mjs` uses a frozen install when one exists, and that changes what a release resolves. - Corrected `packages/paperclip-runner/tsconfig.browser.json` for TypeScript 7. - Set `rootDir` to `..` in `packages/paperclip-runner/tsconfig.test.json`, because one Runner test imports from `packages/adapter-utils/src`. The program does not emit files, so this changes no build output. - Moved the Kubernetes provider from `vitest ^3.2.4` to `^4.1.8`, the version the other providers use. Vitest 3 requires `tinypool ^1.1.1`, which has two critical advisories (GHSA-85c8-ppgw-ccpr and GHSA-5gmw-xhrv-c9v3). The first patched version is 2.1.2. Its tests have the same result on both versions: 189 passed, and the same 12 `file-sync` tests failed. - Moved the Kubernetes provider from `zod ^3.24.2` to `^4.6.6`, and gave its two `z.record` calls a key schema. This is the only production change, and it stays in this pull request. See "Production change for review". - Renamed `AttentionInteractionResolver.test.ts` to `AttentionInteractionResolver.replace.test.ts`, so TypeScript reads the `.test.tsx` beside it. - Fixed all 803 errors, with changes to test files and test support only. The fix for `contracts/native-execution.test.ts` is the same as the fix in paperclipai/paperclip#18739, so those two changes merge cleanly. - Updated `doc/DEVELOPING.md`. This pull request does not edit `server/scripts/typecheck-layers.mjs`. paperclipai/paperclip#18739 and paperclipai/paperclip#18738 have both merged, and this branch includes them. ### Production change for review The new lockfile made dependency review flag `zod@3.25.76` as a high-severity advisory (GHSA-mrhx-c7xc-m3w5). The first patched version is 4.6.6, so 3.x has no fix. The published Kubernetes plugin uses zod to validate operator config. The zod 3 source does not compile against zod 4, so a typecheck-only override cannot fix this. The existing tests do not prove this change, because they also passed with the invalid one-argument `z.record`. So I parsed 45 inputs with the same schema on zod 3.25.76 and zod 4.6.7: | Result | Inputs | Detail | | --- | ---: | --- | | Identical | 23 | Includes defaults, valid config, and a `__proto__` key | | Same outcome, new error wording | 21 | For example, `Invalid url` is now `Invalid URL`. `validateConfig` shows this text to operators. | | Different outcome | 1 | `jobTtlSecondsAfterFinished: 2 ** 60` was valid and is now rejected. The new limit is `Number.MAX_SAFE_INTEGER`. | I did not test this against a real cluster. The decision is to keep this change in this pull request. Since then, paperclipai/paperclip#17364 has moved the rest of the repository to zod 4.6.x, so the Kubernetes plugin now matches it. ### Behavior change for review `reducedMotion` under `use` is not a Playwright option. It had no effect in 11 configs and 2 `test.use` calls, so those suites ran with `no-preference`. The options are now under `contextOptions`, so reduced motion now takes effect. `app-shell-design.spec.ts` asserts a real transition, so it now asks for `no-preference` itself. The baseline manifest has `snapshotCount: 0`, so this invalidates no published baseline. A frozen trial reference PNG can differ. ### Findings These are product questions. I did not change them, and I did not hide them with casts: 1. **`PrpEvent` and its schema disagree.** The TypeScript type in `src/protocol/replay-contract.ts` requires `normalizedSessionId`. The `required` list in `protocol/schemas/event.schema.json` does not include it. `validatePrpEvent` accepts events that the type forbids. 2. **`requestCheckboxConfirmation` is stricter than its runtime.** It uses the output type, so plugin authors must pass `continuationPolicy` and `allowDeclineReason`, although the schema gives them defaults. `z.input` would fix this. 3. **`RetainedRunnerdMaintenanceInput.backend.kind` is a free string.** The quarantine domain comes from `descriptor.kind`, but the release uses `proof.backend.kind`. If they differ, `completeRetainedNativeSessionCleanup` returns 0 and the quarantine stays. The production caller passes `"runner"` today, so this is not a live bug. 4. **`CodexHarnessSession.close(input?)` is optional, but the contract requires `{ reason }`.** These are tests that verified nothing. I corrected them, and they pass: - `backends/native-instruction-measurement.test.ts` filtered on `placement === "core"`, which is not a valid placement. It measured an empty tool list. It now measures 14 tools. - `drivers/acpx/codex-acpx-driver.test.ts` checked for `usage.updated`, which is not an event type. - The `hostFixture()` helper in `drivers/acpx/runtime-host.test.ts` ignored two overrides, so `expect(prepareSandbox).not.toHaveBeenCalled()` could not fail. - `live/live-session.test.ts` set `agent: "pi"` before a spread that overwrote it. - `native-distribution-integrity.test.ts` passed an AbortSignal where `openCommand` expects `{ signal }`. These are gaps that this pull request does not close: - **The root lockfile still has some old zod versions.** paperclipai/paperclip#17364 moved the direct zod dependencies to 4.6.x. `pnpm-lock.yaml` still resolves `zod@3.25.76`, `4.1.11`, and `4.4.3`, which are below the patched 4.6.6. `packages/paperclip-runner/scripts/pi-distribution/package.json` pins `zod` to `3.25.76`, and some agent SDKs resolve `4.4.3`. Dependency review does not flag them, because this pull request does not add them. - **Dead GitHub coverage.** `chat-adapters-ui-providers.spec.ts` returned early for GitHub, so four later `provider === "github"` branches never ran. I removed them. They included the private key file and paste match, and the reconnect flow. Nothing asserts those behaviors in that spec now. - **Provider tests do not run in CI.** The root `vitest.config.ts` lists only Daytona. The other 8 providers, `bridge-template` included, are typechecked now, but CI does not run their tests. - **One `packages/db` test never runs.** `scripts/clean-poisoned-claude-sessions.test.ts` is outside the `include` of `packages/db/vitest.config.ts`. - **`plugin-orchestration-smoke-example` does not collect.** `@paperclipai/shared` is not linked in its `node_modules`. It passes 3 of 3 with an alias. ## Verification Run these commands: ```sh node scripts/typecheck-test-programs.mjs --low-memory pnpm --filter @paperclipai/server typecheck node scripts/typecheck-test-projects.mjs --low-memory --reuse-declarations node --test scripts/typecheck-test-programs.test.mjs ``` Results on macOS (arm64), Node 26.12. CI uses Node 24, and its `Typecheck + Release Registry` job also passes. At head `7d4b5e907a`, the test programs, `tests/e2e`, and the script tests pass again with 0 errors. The times and peaks below come from the run before the two latest merges of master: | Check | Result | | --- | --- | | Test programs, `--low-memory` | Exit 0, 0 errors, 42 seconds, peak 1.78 GiB | | Server layers, low memory | Exit 0, 0 errors, 38 seconds, peak 3.41 GiB | | Test projects, including `tests/e2e` | Exit 0, 0 errors, 13 seconds, peak 1.59 GiB | | Script tests | 12 passed | | Injected type error in each of 23 places | Each one failed its program with TS2322 | | Coverage check | 0 unchecked files, 10 seconds | I did not run this in a 4 GiB container. The peaks above are `maximum resident set size` for the largest single process. **Runtime check of the fixes.** Types do not prove that a test still works, so I ran the suites before the two latest merges of master: | Suite | Result | | --- | --- | | Runner, full suite | 4,885 passed, 3 failed, 13 skipped | | Sandbox providers, 10 packages | 939 passed, 12 failed, 36 skipped | | Plugin SDK | 200 passed | | `kv-demo-mcp-server` | 12 passed | | Renamed UI test and its sibling | 4 passed | All 15 failures exist without these fixes or come from the local machine: - **3 Cursor Cloud timeouts.** `artifact-delivery.test.ts` has 1 and `cursor-cloud-driver.test.ts` has 2. They time out in a full local run, and all 133 tests in those files with `codex-security-config.test.ts` pass when run alone. They fail in the same way with those two files at their unedited versions. - **12 Kubernetes `file-sync.test.ts` failures.** They fail in the same way before and after the fixes. The first failure is that `/proc/self/fd` does not exist on macOS. After each of the two latest merges of master, I ran only the Runner test files that I changed: | Merge | Result | | --- | --- | | `885c249cb9`, 8 files | 534 passed | | `7d4b5e907a`, 7 files | 672 passed, 16 failed | The 16 failures are in the two `source-reconciliation` test files. They fail in the same way with master's unedited versions of those files. 15 of them fail with `ACPX sidecar exited at initialize`, so I think my local build of the Rust and sidecar binaries is out of date. I did not rebuild them to check. CI builds them from scratch, and all Runner jobs pass in CI at `7d4b5e907a`. **Not run:** - `tests/e2e` and the 8 Playwright suites need a server, so most of them have only compile checks. `playwright test --list` loads 308 tests from the main e2e config. - `packages/plugins/sandbox-providers/kubernetes/test/integration` needs a cluster, so it has only compile checks. - I did not run the full `pnpm test:run` or root `pnpm build`. Required CI runs them. **Reduced motion.** I ran 6 motion-sensitive Storybook specs in system Chrome with a temporary config, once with `reduce` and once with `no-preference`. Both runs had 45 passed and the same 5 failed. So reduced motion did not change those results. The 5 failures do not depend on motion. At least one comes from the browser, because system Chrome prints `oklch(55.6% 0 0)` where the test expects `oklch(0.556 0 0)`. I did not run the other Storybook specs. The `storybook-visual` label runs that workflow. ## Risks - **A new lockfile to maintain.** A change to a provider `package.json` now fails the typecheck job until someone refreshes `scripts/typecheck-sandbox-providers/pnpm-lock.yaml`. The failure message gives the command. - **The job has a new install.** The step downloads provider SDKs, so it depends on the npm registry. It uses `--frozen-lockfile` and `--ignore-scripts`. - **The local install replaces provider `node_modules`.** A contributor with a standalone provider install must reinstall that provider after running the script. - **Reduced motion can change Storybook results.** See "Behavior change for review". That workflow is not part of required CI. - **Kubernetes plugin validation changed.** A release of this plugin ships zod 4. Error wording changes, and very large integers are now rejected. See "Production change for review". - **Master moves fast.** Each merge of master brought new test code with type errors, 27 so far. After this merges, the check reports those errors in the pull request that adds them. ## Model Used - Provider: Anthropic Claude, through Claude Code - Model: Claude Opus 5.5 (`claude-opus-5-5`) - Capabilities: tool use, code execution, and subagents for the type fixes in separate files ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details. The app generated the branch name. - [ ] I have run tests locally and they pass. See the failures explained in Verification. - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DottaandPaperclip
|
8f822bfcf7 |
fix(cloud): persist configuration and load release-image packages (#18335)
Land the approved Cloud configuration and release-image package stack.
Includes the separately reviewed adapter settings (#18335), external adapters (#18333), bundled plugins (#18336), startup configuration (#18339), and migration/package checks (#18341).
Includes the final image-owned loader and linked-dependency inventory corrections.
Verified stack head:
|
||
|
yzxcj797andAndrew Aymeloglu
|
81dcc4f6bf |
docs(kubernetes): fix agent-sandbox install manifest URL (#11410)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Kubernetes sandbox provider supports the Sandbox CR backend. > - Its migration guide tells users to install the agent-sandbox controller. > - The command names install.yaml, which is not a release asset. > - This pull request changes the asset name to sandbox.yaml. > - Users can then download the base controller and CRD manifest. ## Linked Issues or Issue Description **Issue type** Incorrect information. **Where is the issue?** packages/plugins/sandbox-providers/kubernetes/README.md, under “Migrating from job to sandbox-cr”. **What's wrong?** The installation command uses releases/latest/download/install.yaml. The upstream agent-sandbox release does not publish that asset. **Suggested fix** Use releases/latest/download/sandbox.yaml, the base installation manifest. ## What Changed - Replace install.yaml with sandbox.yaml in the controller installation command. ## Verification - Checked upstream release v1.0.8. Its assets are sandbox.yaml, extensions.yaml, and sandbox-with-extensions.yaml. - Reviewed the one-line documentation diff. - No Kubernetes installation or local tests were run. ## Risks - Low risk. This changes one documentation URL. - The URL follows the latest upstream release, so future upstream asset changes can affect it. ## Model Used Original author model use was not reported. Adoption and description update assisted by OpenAI GPT-6 (Codex), with repository and GitHub tool access. A more specific runtime model ID is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [ ] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [ ] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com> |
||
|
Nicky LeachandPaperclip
|
38d8f37172 |
fix(build): enforce Node 24 across Paperclip (#11792)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip runs across the CLI, server, adapters, plugins, CI, and container images. > - These surfaces declared different Node.js versions from 20 through 24. > - A newer `@types/node` major can expose APIs that the supported runtime does not provide. > - Node.js 20 is no longer a suitable project baseline, and Node.js 24 is the current LTS line. > - This pull request sets Node.js 24.11.0 as one repository-wide baseline, adds a drift check, and gives users actionable startup guidance when their runtime is too old. > - The benefit is one clear runtime contract for development, release, installation, and published packages. ## Linked Issues or Issue Description Refs #2734 Refs #11727 Refs #739 ## What Changed - Require Node.js 24.11.0 or newer in all 42 package manifests and runtime checks. - Use Node.js 24 in GitHub Actions, Docker images, smoke images, sandbox setup, portable installs, and esbuild targets. - Align every direct `@types/node` declaration on `^24.0.0`. - Prevent Dependabot from opening major `@types/node` upgrades without a matching runtime decision. - Add `.nvmrc` and a CI policy check for Node version drift. - Update ACP version gates, tests, and user documentation for the new minimum. - Print a non-blocking warning on CLI and server startup when Node is unsupported, with remediation through a version manager or the documented downloaded `install.sh` workflow. - Deduplicate that warning when `paperclipai run` boots the CLI and server in the same process. ## Verification - `node scripts/check-node-version-policy.mjs` - `node --check scripts/check-node-version-policy.mjs` - `node --check cli/esbuild.config.mjs` - `node --check scripts/generate-npm-package-json.mjs` - `bash -n scripts/install.sh scripts/test-install-sh-docker.sh scripts/e2e-install-lifecycle.sh` - Parsed all 42 package manifests and confirmed `engines.node` is `>=24.11.0`. - `git diff --check` - `vitest run packages/adapter-utils/src/sandbox-install-command.test.ts` passed with 3 tests. - `vitest run cli/src/node-version.test.ts` passed with 4 tests. - Directly exercised the shared warning helper for unsupported-version messaging and same-process deduplication. - The focused exe.dev suite could not resolve the locally unbuilt plugin SDK from this isolated worktree. A full offline workspace install was also blocked because the package-manager signature verifier requires registry access. The full suite was not run locally; draft CI performs a clean install and evaluates the wider impact. ## Risks - This is a breaking runtime change for users, plugins, and deployments that still use Node.js 20 or 22. - Published workspace packages will now produce an engine warning or failure in strict package managers on older Node.js releases. - Node.js 24 can reveal dependency, native module, Playwright, or agent CLI compatibility issues in CI. - The bootstrap installer now installs Node.js 24 when the current runtime is older than 24.11.0. - The portable sandbox fallback is pinned to Node.js 24.11.0 and depends on that upstream tarball remaining available. - Unsupported runtimes continue booting after a warning, so a later incompatibility can still fail at its point of use. - The CLI and server share the warning policy through the published `@paperclipai/shared` package; packaging checks must keep that subpath export available. - This PR does not commit `pnpm-lock.yaml` because repository policy assigns lockfile generation to CI. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex based on GPT-5. The exact deployment ID and context window are not exposed in this session. Reasoning, repository tools, shell execution, and GitHub tools were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
fdb9a4880d |
fix(security): route paperclipai CLI guidance through safe npx form (CWE-78) (#11400)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip provides CLI commands and guidance for operators and agents > - The `pnpm paperclipai` script can pass argument values through a shell > - Shell re-parsing can execute command substitutions inside quoted values > - This pull request routes guidance through inert-argv `npx paperclipai` commands and adds regression coverage > - The benefit is safer operator guidance across documentation and runtime hints ## Linked Issues or Issue Description This pull request fixes a command-injection-class defect in Paperclip CLI guidance. **What happened?** The `pnpm paperclipai <sub> --flag "$VALUE"` form can re-parse argument values through a shell. A command substitution inside a quoted value can execute on the host. **Expected behavior** Paperclip guidance must pass CLI values as inert argument values. Host-derived values must not appear in copyable commands. **Steps to reproduce** 1. Run a Paperclip guidance command that uses the `pnpm paperclipai` script. 2. Provide a quoted value that contains a command substitution. 3. Observe that the shell can evaluate the substitution before the CLI starts. 4. Compare the result with the `npx paperclipai` form. **Paperclip version or commit** `5670984b75d109950c968542a0111ebb6967f4da` **Deployment mode** All deployment modes that show or use the affected CLI guidance. **Installation method** Built from source and installed CLI guidance. **Agent adapter(s) involved** Not adapter-specific (core bug). **Database mode** Not database-related. **Access context** Both. **Additional context** The earlier merged PR [#11343](https://github.com/paperclipai/paperclip/pull/11343) used the unsafe `pnpm exec paperclipai` form. This fresh PR replaces that guidance with the safe `npx paperclipai` form. ## What Changed - Standardize documentation and runtime hints on `npx paperclipai`. - Remove the broken `pnpm exec paperclipai` guidance. - Use a static `<host>` placeholder in private-hostname guidance. - Add regression tests for unsafe forms, continued lines, static hosts, and offline guidance. ## Verification - `git diff --check origin/master...origin/fix/paperclipai-cli-npx-safe-invocation` passes. - The branch adds `server/src/__tests__/cli-invocation-safety.test.ts` and updates private-hostname tests. - CI must run the new tests, typecheck, lint, and build checks. - Local Vitest execution was not available because this worktree has no installed Vitest binary. ## Risks - The change affects operator and agent documentation text. - The runtime hints now show `<host>` instead of a request-derived host value. - No database schema or migration changes exist. - CI will detect any missed unsafe invocation or type error. ## Model Used OpenAI GPT-5, exact model ID `gpt-5`, with tool use and code-review assistance. The model used repository inspection, Git operations, and PR preparation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] CI ran the test suites and they pass; local test execution was unavailable in this worktree - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I addressed all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
c57c0f7498 |
fix(sandbox-providers): accept bsdtar listings in the syncOut tarball confinement check (#11289)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox providers (Daytona, Kubernetes) sync run results back to the host with a sandbox-authored tarball > - Before extraction, a confinement check parses the host `tar -tvf` listing and fails closed on unparseable lines > - The parser only understands the GNU tar listing dialect; macOS ships bsdtar, whose ls-style listing never matches > - Every sandbox syncOut on a macOS host therefore aborts with "refusing tarball with an unparseable entry listing", and the run fails at copy-back > - This pull request teaches the parser both dialects while keeping the fail-closed and traversal guarantees > - The benefit is that Daytona and Kubernetes sandbox runs work on macOS hosts, with no behavior change on Linux ## Linked Issues or Issue Description No existing issue. Bug description: **What happened?** On a macOS host, every Daytona sandbox run fails at syncOut. The adapter reports: `Daytona syncOut refusing tarball with an unparseable entry listing: -rw-r--r-- 0 daytona daytona 7560 Aug 11 21:43 AGENTS.md`. The Kubernetes provider has the same parser and fails the same way. **Expected behavior** The confinement check accepts a well-formed listing from the host tar, whichever dialect the host tar emits. It still rejects members that escape the extraction directory, and it still fails closed on lines it cannot parse. **Steps to reproduce** 1. Run Paperclip on macOS (system tar is bsdtar). 2. Configure an agent with the Daytona sandbox provider. 3. Trigger any run that syncs files back from the sandbox. 4. The run fails at syncOut with the unparseable-entry-listing error, because bsdtar prints `<perms> <links> <user> <group> <size> <Mon> <day> <time|year> <name>` while the parser expects the GNU `<perms> <owner>/<group> <size> <date> <time> <name>` shape. **Operating system** macOS (bsdtar 3.5.3). Linux hosts with GNU tar are unaffected. ## What Changed - Extracted the listing-line parse in both providers' `file-sync.ts` into an exported `parseTarVerboseListingLine` that accepts the GNU/busybox dialect and the bsdtar (libarchive) dialect. - The GNU shape now requires the slash-joined `<owner>/<group>` field. This keeps the two shapes mutually exclusive. Without it, a bsdtar line with numeric uid/gid satisfies the loose GNU pattern shifted by one field, which would hide a leading `../` from the traversal check. - Unparseable lines still fail closed. This includes device-node entries, whose size column is `major,minor` in both dialects. - Made the path-traversal fixture in the Daytona suite portable: GNU spells member renaming `--transform`, bsdtar spells it `-s`. - Added a Daytona test that refuses a sandbox-authored tarball carrying a symlink whose target escapes the extraction dir. - Added parser unit tests for both dialects (file, dir, symlink, hardlink, numeric owner, year-form dates, fail-closed lines) to both providers' suites. ## Verification - `pnpm test` in `packages/plugins/sandbox-providers/daytona`: 136/136 pass on a macOS host. On unpatched `master` the round-trip test fails there with the unparseable-entry-listing error. - `pnpm test` in `packages/plugins/sandbox-providers/kubernetes`: the new parser tests pass; no new failures against the `master` baseline on the same host. - `pnpm typecheck` passes in both packages. - CI runs the same suites on Linux/GNU tar and proves the GNU path is unchanged. ## Risks - Low risk. The GNU pattern is one token stricter (`<owner>/<group>` must contain `/`). GNU and busybox tar always print the slash-joined owner field, so accepted GNU listings are unchanged. - The bsdtar branch only widens acceptance on hosts that were failing 100% of syncOuts before, so no working deployment changes behavior. - The check still fails closed on anything neither pattern matches. ## Model Used Claude Fable 5 (`claude-fable-5`, Claude Code CLI, extended thinking + tool use). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Nicky LeachandPaperclip
|
0ccba45e4d |
feat(sandbox-providers): native providers honor postUploadCommands (Daytona executes; Kubernetes executes) (#10347)
## Thinking Path > - Paperclip is the control plane for autonomous AI companies, so runtime handoffs have to preserve the exact behavior an agent asked for. > - The sandbox-provider layer is where uploaded files and follow-up commands become a real in-sandbox operation. > - The provider-delegable sync-in seam already exists from the prior PR; without this follow-up, native providers can still accept command-bearing uploads and silently drop the commands. > - That is a fail-open gap for native sandbox providers, because the file transfer succeeds while the intended post-upload work never runs. > - This pull request teaches Daytona and Kubernetes to execute `postUploadCommands` in order, inside the sandbox, after the files land. > - The benefit is consistent and safer sync semantics: native providers either run the commands as requested or fail fast instead of pretending the operation completed fully. ## Linked Issues or Issue Description This PR builds on the previously merged provider-delegable sync-in seam and closes the remaining gap for native providers that still dropped `postUploadCommands`. Problem: - A sync operation could include ordered `postUploadCommands`, but a native provider could finish the file upload and skip the commands entirely. - That creates fail-open behavior for command-bearing uploads, especially when the caller relies on the provider to execute the follow-up action in the sandbox. Proposed fix: - Execute `postUploadCommands` inside the sandbox after file placement. - Preserve the provided command order. - Fail fast on the first non-zero exit or timeout. - Keep command execution verbatim and confine any provided `cwd` under the workspace root. Related public PR: - Refs: #10340 ## What Changed - Daytona `performSyncIn` now executes ordered `postUploadCommands` through the existing `executeCommand` seam. - Kubernetes `performSyncIn` now executes ordered `postUploadCommands` through its streaming pod exec path. - Added workspace confinement for provided `cwd` values and defaulted missing `cwd` to the remote root. - Added tests covering the new post-upload command execution behavior in both provider packages. ## Verification - Latest validation recorded on the handoff branch: `@paperclipai/plugin-sdk` and `@paperclipai/plugin-kubernetes` typechecks passed. - Daytona Vitest: `63/63` passing in `plugin.test.ts`. - Kubernetes Vitest: `195/195` passing, including `file-sync.test.ts`. - `git log --oneline origin/master..HEAD` showed a single expected commit on the branch. ## Risks - Command execution semantics are stricter now, so malformed commands or a bad `cwd` will fail the sync instead of being ignored. - The change makes provider behavior more explicit, which can surface previously hidden failures in callers that assumed commands were optional. - Timeout behavior may differ slightly between providers, so the failure mode is intentionally fail-fast. ## Model Used OpenAI Codex, GPT-5-based coding agent with tool use enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
7f766526a6 |
feat(sandbox): add task-scoped egress grants (#10155)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Confinement providers protect agent runs with default-deny network policies > - Kubernetes environments currently apply only provider-level, namespace-wide egress allowances > - Tasks that legitimately need GitHub or package registries therefore cannot request narrow access, while network failures do not explain the governing policy or how to request a grant > - This pull request adds issue-scoped egress grants that become workload-owned, run-label-selected policies and carries the effective grant through lease audit metadata > - The benefit is that internet-dependent work can run without enabling broad egress for every concurrent task, and denied requests point operators to the exact grant path ## Linked Issues or Issue Description No public issue exists. Related but distinct: Refs #9944, which adds a provider-wide open-internet posture; this PR keeps provider defaults narrow and adds per-task grants. **Problem / motivation** Kubernetes sandbox egress is configured at the provider/tenant level. A task that needs to clone from GitHub or install from PyPI cannot request those destinations without changing the policy for every run in the tenant namespace. DNS/connectivity failures also surface as generic tool errors with no policy name or remediation path. **Proposed solution** Accept `executionWorkspaceSettings.networkEgress.allowFqdns` and `allowCidrs`, forward the setting through heartbeat environment acquisition, and create a workload-owned NetworkPolicy or CiliumNetworkPolicy selected by `paperclip.io/run-id`. Record the effective grant in lease activity/metadata, expose policy context through `PAPERCLIP_NETWORK_EGRESS_*`, and append the grant path to likely policy-related stderr failures. **Alternatives considered** A provider-wide open-internet switch is broader than required and is already covered by #9944. Mutating the existing namespace policy would leak each task's destinations to other concurrent runs. Standard Kubernetes NetworkPolicy cannot enforce FQDNs exactly, so standard mode uses the existing hardened public-IPv4 TCP 80/443 fallback only for the selected run; Cilium mode remains exact. **Roadmap alignment** This extends the existing cloud/sandbox agent roadmap capability with task-level control-plane policy and does not duplicate a planned roadmap item. ## What Changed - Added validated `networkEgress` grants to issue execution workspace settings and forwarded them through environment lease acquisition. - Added workload-owned, run-label-scoped NetworkPolicy/CiliumNetworkPolicy resources for task FQDN/CIDR grants. - Added lease audit metadata, sandbox policy environment variables, and actionable network-denial stderr guidance. - Added focused parser, manifest, policy creation, and denial-message tests plus Kubernetes provider documentation. ## Verification - `pnpm -C packages/shared exec vitest run src/validators/issue.test.ts` — 27 passed. - `pnpm -C packages/plugins/sandbox-providers/kubernetes test -- --run test/unit/network-policy.test.ts test/unit/cilium-network-policy.test.ts test/unit/scoped-network-egress.test.ts` — 21 passed. - `pnpm -C server exec vitest run src/__tests__/execution-workspace-policy.test.ts` — 15 passed. - `pnpm exec vitest run server/src/__tests__/heartbeat-plugin-environment.test.ts server/src/__tests__/environment-runtime.test.ts` — 26 passed. - `pnpm --dir packages/db build && pnpm --dir packages/shared build && pnpm --dir packages/plugins/sdk build` — passed, including migration safety checks. - `pnpm --dir packages/plugins/sandbox-providers/kubernetes typecheck && pnpm --dir server typecheck` — passed after refreshing the worktree's frozen offline dependencies. - End-to-end cluster validation of the `build-cython-ext` benchmark remains for CI/maintainer Kubernetes infrastructure; the focused tests assert `github.com` and `pypi.org` produce a policy selected only by the granted run. ## Risks - Standard NetworkPolicy cannot express FQDNs, so an FQDN grant allows hardened public IPv4 TCP 80/443 for that run; use Cilium mode for exact hostname enforcement. - The new field is additive and absent by default, so existing runs keep the current provider-level policy. - Workload owner references garbage-collect scoped policies with the Job/Sandbox; a cluster/controller that ignores owner references could temporarily strand a policy that still selects no future run ID. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, exact model ID `gpt-5.6-sol`, high reasoning mode, tool use and code execution. The runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
9edde68373 |
feat(kubernetes): native file-sync lifecycle hooks over pod exec (#10053)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - AI agents run in sandboxed execution environments (Kubernetes pods, Daytona workspaces, etc.) and need to sync files between the host and those environments — for workspace setup, asset delivery, and output retrieval > - The existing sync path for Kubernetes uses a base64-over-exec chunk loop: each ~4 MB chunk requires its own `execInPod` round-trip, so large syncs balloon into many exec calls with corresponding overhead > - `execInPod` supports piped stdin/stdout, meaning the full transfer can be done as a single exec that streams a raw `tar` archive over the data channel — one round-trip regardless of file size, with nothing base64-encoded and nothing buffered whole in memory on either side > - PR-1 (#10013, merged) added the `onEnvironmentSyncIn`/`onEnvironmentSyncOut` opt-in hook API to the sandbox provider interface and documented the protocol; PR-2 (#10028, merged) implemented these hooks for the Daytona provider > - This pull request implements the same two lifecycle hooks in the Kubernetes sandbox provider, so workspace/asset file sync streams through one `execInPod` per operation instead of the chunk loop > - The benefit is significantly fewer exec round-trips for large syncs and flat memory use on both host and pod, with security properties preserved: atomic replace, secret-mode enforcement, path confinement, TOCTOU-safe snapshot, and member-confinement on host-assembled archives from sandbox-authored tar output ## Linked Issues or Issue Description This is the third and final PR in a sequential series: - Refs #10013 — PR-1: opt-in sync hook API + provider docs (merged) - Refs #10028 — PR-2: native file-sync lifecycle hooks for Daytona provider (merged) **Feature:** Native single-exec file-sync lifecycle hooks for the Kubernetes sandbox provider. *Motivation:* The existing Kubernetes sync path encodes files as base64 and loops over `execInPod` one chunk at a time (~4 MB per exec). For large workspaces or asset sets this is slow and resource-intensive. The Kubernetes `execInPod` API supports piped stdin/stdout, enabling a raw-`tar` streaming transfer that needs only one exec regardless of file count or size and never buffers the whole payload in memory. *Proposed solution:* Implement `onEnvironmentSyncIn` and `onEnvironmentSyncOut` in the Kubernetes provider using a streaming `execInPod` with a tar pipeline — for syncIn the host builds the archive on disk and streams its raw bytes into the pod's stdin (`head -c <exact-size> | tar -x`, no base64); for syncOut in-pod `tar` writes to the exec's stdout and the host streams those bytes straight to a file. Path confinement, atomic replace, secret-mode enforcement, TOCTOU protection, and a streamed-bytes fail-closed guard are all enforced. ## What Changed - **New `src/file-sync.ts`** in `packages/plugins/sandbox-providers/kubernetes/` — `performSyncIn` and `performSyncOut` over an injected pod-exec closure, keeping transfer logic hermetically unit-testable - **New `execInPodStreaming` in `src/pod-exec.ts`** — a streaming exec primitive that binds a caller-supplied stdin readable and a stdout writable to the exec WebSocket data channel, added alongside the existing `execInPod` (which is unchanged). This lets a transfer stream raw bytes to/from disk instead of buffering the payload as a single string - **Updated `src/plugin.ts`** — registers `onEnvironmentSyncIn`/`onEnvironmentSyncOut`; resolves the `sandbox-cr` pod exactly like `onEnvironmentExecute` and delegates; `job` backend rejects file-sync calls explicitly (out of scope) - **syncIn path:** host builds the tarball to a temp file → streams its raw bytes over exec stdin, bounded in-pod by `head -c <exact-archive-size> | tar -x` (no base64 anywhere) → extract into a `/proc/self/fd`-pinned reserved `0700` staging dir → `chmod`-before-`mv -f` atomic replace per file (directory mappings use `followSymlinks`→`-h`) - **syncOut path:** in-pod validate + realpath-snapshot each source (closes the validation→copy TOCTOU window) → single-exec `tar -c` streamed over exec stdout → host streams that stdout straight to a temp file through a byte-counting transform → member-confined extraction of the sandbox-authored archive - **Security properties:** secret files land at requested mode with no widened window; every interpolated path is shell-quoted and confined lexically plus via in-pod `realpath`; the outbound stream is bounded by a **streamed-bytes disk guard** (`MAX_SYNC_OUTPUT_BYTES`, 8 GiB default, per-call overridable) that fails the transfer closed — writing no target file — if an untrusted pod emits more bytes than allowed. Neither host nor pod buffers the whole payload, so there is no in-memory size cap on the transfer - **No changes** to `execInPod`, `wrapCommandWithEnv`, or `FastUploadInterceptor` (the `environmentExecute` path is untouched) - **No dependency or lockfile changes** - **New tests** in `test/unit/file-sync.test.ts` (atomic-replace, `0600` secret mode, symlink preserve/deref, dir-mapping, exclude, path-confinement rejection, streamed-output guard fail-closed) and `test/unit/pod-exec.test.ts` (streaming stdin/stdout, caller-sink error fail-closed), plus extended `test/unit/plugin.test.ts` ## Follow-up: Legacy Job-Lease Base64 Fallback Fix Addresses the Greptile 4/5 blocking finding ("Handle existing job leases", `server/src/services/environment-runtime.ts`). Job leases provisioned before the `nativeFileSyncUnsupported` metadata flag existed carry `backend: "job"` but no flag, so `supportsSync()` treated them as native-capable and routed their sync to the pod-exec hook — which the job backend rejects (it has no exec channel) instead of using the byte-identical base64 fallback. The fix adds a belt-and-suspenders gate on the persisted `backend === "job"` field alongside the existing `nativeFileSyncUnsupported` flag check, so pre-existing job leases continue syncing via the base64 fallback after deployment. No behaviour change for `sandbox-cr` leases. ## Verification - `pnpm --filter @paperclipai/sandbox-provider-kubernetes test` — 19 files / 182 tests green, including the existing `upload-interceptor` and `pod-exec` suites - `tsc --noEmit` in the kubernetes package — 0 errors - The sync hooks are opt-in; existing `environmentExecute` behaviour is unaffected and tested by the unchanged existing suites ## Risks - **Opt-in only:** `onEnvironmentSyncIn`/`onEnvironmentSyncOut` are registered conditionally; providers that do not register them fall back to the existing chunk loop. No regression risk on the existing path. - **Shell-injection surface:** all path interpolation uses shell-quoting; paths are additionally confined lexically and via in-pod `realpath` before use. - **TOCTOU on syncOut:** the in-pod snapshot validates and records file metadata before the tar call, closing the window between validation and copy. - **Archive member confinement:** host-side reassembly rejects any tar member whose resolved path escapes the target directory, preventing a malicious in-pod tar from writing outside the intended destination. - **Untrusted-output volume:** an over-large outbound stream trips the streamed-bytes disk guard and fails closed (no target written and the temp sink is swept) rather than filling host disk or memory; the guard bounds disk unconditionally and bounds memory insofar as WebSocket write-backpressure holds. ## Model Used Anthropic Claude Sonnet 4.6 (`claude-sonnet-4-6`) — produced by a Claude-based AI agent using agentic tool use and multi-step code generation. 200K context window, extended reasoning, code execution and verification capabilities. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
eedc7ddef2 |
Make ACP the default engine for local adapters (#9238)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Adapter packages are the bridge between the control plane and local agent harnesses such as Claude Code, Codex, and Gemini CLI. > - ACP support was concentrated in a separate `acpx_local` adapter, which made ACP feel like a separate agent choice instead of an execution capability of the harness adapters. > - Claude, Codex, and Gemini now have ACP-capable harnesses, so the native adapter should own ACP selection, fallback, config, transcript parsing, and environment diagnostics. > - The standalone ACPX adapter still needs a compatibility path for existing rows, but it should not be offered as an active adapter for new agents. > - This pull request moves the shared ACP runtime into `@paperclipai/acpx-engine`, wires Claude/Codex/Gemini local adapters to prefer ACP when prerequisites are available, and retires `acpx_local` to a tombstone. > - The benefit is one adapter per harness, richer ACP transcripts by default where possible, and a migration path for existing Claude/Codex ACPX agents. ## Linked Issues or Issue Description Closes #5932 — the broken default `acpx_local` Claude path is replaced by native `claude_local` ACP support, existing Claude/Codex ACPX rows migrate to native adapters, and new agents no longer choose the standalone ACPX adapter. Refs #4893 — original merged ACPX local adapter runtime that this PR replaces with native per-harness ACP engines. Refs #6590 — prior ACPX-Claude seamlessness work folded into the new native Claude ACP path. Refs #197 — related open generic ACP/Kiro adapter work; this PR does not close it because Kiro/custom generic ACP remains a separate adapter decision. Refs #7018 — related Kimi-specific `acpx_local` shell failure; this PR retires the built-in standalone adapter but does not add a native Kimi adapter. Refs #8864 — related ACPX prompt/API guidance PR; this PR moves runtime guidance into the shared/native ACP engine path instead of the old standalone adapter. Refs #8881 — related `acpx_local` POSIX shell failure from the old `acpx` pin; this PR updates ACP dependencies but does not claim custom/OMP ACP support as a first-class native adapter. Refs #8964 — related open `acpx_local` stderr cleanup PR; this PR makes the old runtime path obsolete for new agents but keeps it as a non-closing reference. Problem description: - The standalone `acpx_local` adapter duplicates Claude/Codex agent choices that already have first-class local adapters. - ACP should be an execution engine capability of each harness adapter when the underlying harness supports ACP. - Existing `acpx_local` agents should either migrate to native harness adapters or fail with an explicit retirement message instead of silently falling back to the process adapter. ## What Changed - Added `@paperclipai/acpx-engine` as the shared ACP execution, session-codec, CLI formatter, and UI parser package. - Wired `claude_local`, `codex_local`, and `gemini_local` to auto-select ACP by default when prerequisites pass, with `engine=cli` opt-out and `engine=acp` strict mode. - Added ACP config schema/UI fields, environment checks, session-codec preservation, transcript parsing, and adapter capability metadata for the native adapters. - Retired `acpx_local` to a server tombstone, removed its UI/package/runtime image surface, and added a migration for existing Claude/Codex ACPX agents. - Updated package manifests, lockfile, release tooling, docs, Kubernetes sandbox defaults, and tests. ## Verification - `corepack pnpm --filter @paperclipai/acpx-engine typecheck` - `corepack pnpm --filter @paperclipai/adapter-claude-local typecheck` - `corepack pnpm --filter @paperclipai/adapter-codex-local typecheck` - `corepack pnpm --filter @paperclipai/adapter-gemini-local typecheck` - `corepack pnpm --filter @paperclipai/acpx-engine exec vitest run` - `corepack pnpm --filter @paperclipai/adapter-claude-local exec vitest run src/server/acp.test.ts src/server/execute.acp-fallback.test.ts src/ui/build-config.test.ts` - `corepack pnpm --filter @paperclipai/adapter-codex-local exec vitest run src/server/acp.test.ts src/ui/build-config.test.ts` - `corepack pnpm --filter @paperclipai/adapter-gemini-local exec vitest run src/server/acp.test.ts src/ui/build-config.test.ts src/ui/parse-stdout.test.ts` - `corepack pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && corepack pnpm --filter @paperclipai/server exec tsc --noEmit` - `corepack pnpm --filter @paperclipai/server exec vitest run src/__tests__/adapter-routes.test.ts src/__tests__/adapter-session-codecs.test.ts src/__tests__/adapter-models.test.ts` - `corepack pnpm --filter @paperclipai/ui typecheck` - `corepack pnpm --filter @paperclipai/ui exec vitest run src/adapters/metadata.test.ts src/adapters/adapter-display-registry.test.ts src/components/AgentConfigForm.test.ts src/components/AgentConfigForm.render.test.tsx src/components/transcript/RunTranscriptView.test.tsx` - `node --test scripts/bootstrap-npm-package.test.mjs scripts/release-package-map.test.mjs scripts/verify-release-registry-state.test.mjs` Note: the server typecheck script calls `pnpm` internally; this dev shell exposes pnpm through Corepack only, so I ran the two script steps manually with `corepack pnpm`. ## Risks - Migration changes existing `acpx_local` Claude/Codex agents to native adapter types and clears old ACPX task sessions/runtime state. - Custom ACP commands remain on the retired tombstone and will need a separate future adapter/plugin path. - ACP auto-selection depends on local Node and ACP server command prerequisites; remote and unsupported environments fall back to CLI unless `engine=acp` is explicit. - `@paperclipai/acpx-engine` is a new public package and needs npm trusted-publishing bootstrap before release automation can publish it. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex coding agent. Exact hosted model build and context-window size are not exposed in this runtime. Tool use included shell execution, repository editing, GitHub CLI operations, and local test/typecheck execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
93291df5c8 |
fix(plugins): move dev SDK linking out of plugin postinstall scripts (#8255)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox-provider plugins (cloudflare, daytona, e2b, exe-dev, kubernetes, modal, novita) and `plugin-workspace-diff` are published as standalone npm packages, but during local dev they need the in-repo `@paperclipai/plugin-sdk` symlinked in > - Each of these plugins shipped a `postinstall` lifecycle script that traversed *out* of its own package directory (`node ../../../../scripts/link-plugin-dev-sdk.mjs`) to do that linking > - The publishable manifest is built by a `prepack` whitelist that drops the `scripts` field, so npm consumers don't see the postinstall today — but that safety property depends entirely on `prepack` running on every publish. A publish that skips lifecycle scripts would ship a tarball whose postinstall escapes its package directory at consumer install time > - This pull request removes the escape-the-package-dir lifecycle script from every plugin source manifest and moves the dev linking to a single root-level postinstall that iterates the excluded plugin directories itself > - The benefit is that plugin tarballs can no longer carry an install-time script that reaches outside their own directory, regardless of whether `prepack` runs ## Linked Issues or Issue Description This is a follow-up hardening change flagged during review of the Novita sandbox provider PR (#7595). **Problem (security):** Excluded plugin packages each carried `"postinstall": "node ../../../../scripts/link-plugin-dev-sdk.mjs"`. The relative path traverses outside the package root. Today the published manifest is sanitized by a `prepack` whitelist that drops `scripts`, so consumers are unaffected in the normal publish path. The risk is that this is a defense-in-depth gap: if a publish ever skips lifecycle scripts (e.g. `npm publish --ignore-scripts` is *not* used, or a tool publishes the raw manifest), the tarball would ship a postinstall that runs out-of-tree code at the consumer's install time. ## What Changed - Added a single root `package.json` `postinstall`: `node scripts/link-plugin-dev-sdk.mjs`. - Rewrote `scripts/link-plugin-dev-sdk.mjs` to iterate the excluded plugin directories itself (`packages/plugins/sandbox-providers/*` + the orchestration smoke example) instead of relying on each plugin to invoke it from its own cwd. Preserves both prior behaviors: leave a real installed SDK dir alone, and skip when already correctly symlinked (idempotent). - Removed `scripts.postinstall` from all 7 sandbox-provider plugins (cloudflare, daytona, e2b, exe-dev, kubernetes, modal, novita). - Removed `scripts.postinstall` from `plugin-workspace-diff` (a pnpm workspace member — pnpm already links the SDK, so the script was a no-op there). ## Verification - `node scripts/link-plugin-dev-sdk.mjs` from repo root: links the SDK into the excluded plugins and reports skipped (already-linked) dirs; re-running is idempotent. - `grep -r "link-plugin-dev-sdk" packages/plugins/*/package.json packages/plugins/sandbox-providers/*/package.json` returns no matches — no plugin source manifest references the linker any longer. - All affected `package.json` files re-validated as parseable JSON. ## Risks Low risk. Dev-only tooling: the linker only runs at the repo root during local install and only touches `node_modules/@paperclipai/plugin-sdk` symlinks inside excluded plugin dirs. No change to published plugin behavior or runtime code. Worst case if the root postinstall failed to run, local dev of an excluded plugin would not find the SDK symlink — easily re-run manually. ## Model Used Claude Opus (claude-opus-4-8), extended reasoning, with tool use / code execution in an agentic coding harness. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable (added `scripts/link-plugin-dev-sdk.test.js`, wired into `test:release-registry`) - [ ] If this change affects the UI, I have included before/after screenshots (N/A — no UI change) - [ ] I have updated relevant documentation to reflect my changes (N/A) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Jannes Stubbemann
|
482f64e343 |
fix(plugin-kubernetes): resolve sandbox pod by exact name (controller labels pods with sandbox-name-hash, not sandbox-name) (#7982)
## Thinking Path Production e2e on the merged #5790 plugin failed on every fresh lease with "Failed to install the adapter runtime command" for a harness that was present in the runtime image. Tracing the lease showed the first exec resolved no pod: the exact-label fallback added during the #5790 review queries `agents.x-k8s.io/sandbox-name=<name>`, but the kubernetes-sigs agent-sandbox controller labels pods only with `agents.x-k8s.io/sandbox-name-hash` (see `sandboxLabel` in its `controllers/sandbox_controller.go`) and NAMES the backing pod exactly after the Sandbox CR. The selector matches nothing, `findPodForSandbox` returns null, execute returns "podName could not be resolved", and adapter-utils misreports it as a missing runtime command. ## What Changed Between the `status.podName` read and the label fallback, try an exact-name pod GET (`readNamespacedPod({namespace, name})`). This is collision-free, so the original review concern (name-prefix matching execing into a concurrent sandbox's pod) stays honored. A 404 falls through to the existing full-name label selector for controller versions that do set such a label. Non-404 errors propagate unchanged. ## Verification - New unit test pins the controller reality: pod named exactly like the sandbox, only a `sandbox-name-hash` label, no full-name label; fails before the fix, passes after. - Review-feedback round: the primary-path test now asserts the exact-name GET is never called, and a new test covers non-404 error propagation (403 rejects, no fallback). 153/153 plugin tests green, tsc clean. - Production-verified on our deployment: agent runs were broken on every fresh lease before this patch and complete end-to-end after it (gVisor sandbox pool, agent-sandbox controller v0.4.6; verified run with cost event and agent reply on a fresh tenant). ## Risks Low: one additional pod GET per first-exec on a fresh lease, only when `status.podName` is unset. Non-404 errors from the GET propagate unchanged (now test-pinned). ## Issue No existing issue; the defect is described in full under Thinking Path (introduced by the review-round fallback change in #5790, first hit in production e2e on 2026-06-11). ## Model Used Claude Fable 5 (claude-fable-5, Claude Code CLI, extended reasoning, tool use) ## Duplicate search Searched open and closed PRs for `findPodForSandbox`, `sandbox-name-hash`, and pod-resolution fixes; no duplicate found. Related parent: #5790 (introduced the fallback this PR repairs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (no UI change) - [x] I have updated relevant documentation to reflect my changes (code comments; no doc surface affected) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Jannes StubbemannandClaude Opus 4.8
|
05ab45225a |
feat(plugin-kubernetes): self-hostable Kubernetes sandbox provider (stage 1/3: plugin package) (#5790)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox providers are the seam that lets agent runs execute in isolated environments; today the only first-party remote provider is Daytona, a hosted third-party service > - Self-hosters running Paperclip on their own infrastructure (often Kubernetes already) have no first-party way to run agent sandboxes on a cluster they control > - That gap matters for teams with data-residency, sovereignty, or cost constraints who cannot or will not send workloads to a hosted sandbox service > - This pull request adds a Kubernetes sandbox-provider plugin as a standalone, workspace-excluded package: it implements every SandboxProvider hook the Daytona provider does, on infrastructure the operator owns > - The benefit is that any Paperclip deployment with a Kubernetes cluster gets multi-tenant, network-isolated, quota-bounded agent sandboxes with zero new external dependencies ## Linked Issues or Issue Description No existing issue. Following the feature template: - **Problem:** Paperclip's remote sandbox execution requires a hosted third-party provider. Self-hosters cannot run agent sandboxes on their own Kubernetes clusters with a first-party provider. - **Proposed solution:** A `@paperclipai/plugin-kubernetes` sandbox-provider plugin with two backends: long-lived sandboxes via the [kubernetes-sigs/agent-sandbox](https://github.com/kubernetes-sigs/agent-sandbox) CRD (multi-command exec, adapter-install pattern) and one-shot `batch/v1` Jobs (stable APIs only, no extra controllers). - **Alternatives considered:** Driving kubectl from a generic shell provider (no lifecycle/lease semantics), or requiring a hosted provider (exactly the constraint this removes). ## What Changed This is **stage 1 of 3** of a staged contribution (direction agreed with maintainers): the plugin package alone. Stage 2 (server integration: lease params, provider registration) and stage 3 (agent runtime images + CI) are companion PRs that will be cross-linked from a comment here. - New package `packages/plugins/sandbox-providers/kubernetes` (workspace-excluded, like the path already carved out in `pnpm-workspace.yaml`): src, unit + kind integration tests, operator prerequisite manifests, README, smoke-test guide - Implements the full SandboxProvider hook surface the Daytona provider implements: `validateConfig`, `probe`, `acquireLease`, `resumeLease`, `releaseLease`, `destroyLease`, `realizeWorkspace`, `execute` - Two backends: `sandbox-cr` (default; long-lived pod via the agent-sandbox `Sandbox` CR, supports multi-command exec) and `job` (one-shot `batch/v1` Job; nothing beyond k8s 1.27+ required) - Per-run adapter resolution: one environment serves mixed harnesses; the per-run `adapterType` hint is read through a local optional type extension, so the plugin typechecks and builds against the current plugin SDK and simply falls back to the environment's configured default adapter until stage 2 lands - Exec-env wrapping: the Kubernetes exec API carries no environment, so commands are wrapped to receive the run's env - Fast-upload interception for workspace realization, scoped per lease - Per-tenant isolation: derived namespace per company, RBAC, ResourceQuota, restricted-PSS pod security (runAsNonRoot, drop ALL, seccomp RuntimeDefault, no SA token automount) - Network egress policy in two flavors: native `NetworkPolicy` and `CiliumNetworkPolicy` (FQDN allowlists) - Image allowlist with glob matching, registry override, and per-run image override validation - Per-run Kubernetes Secrets carrying agent credentials, ownerRef'd to the Job or Sandbox CR for cascade GC ## Verification - Standalone build, exactly as the README documents: ```bash cd packages/plugins/sandbox-providers/kubernetes pnpm install --ignore-workspace pnpm test # 147 unit tests, 17 files, all green pnpm typecheck # clean against the in-repo plugin SDK on master pnpm build # dist/ emitted, manifest + worker entrypoints present ``` - A kind-cluster end-to-end integration test is included (`RUN_K8S_INTEGRATION_TESTS=1 pnpm test test/integration/end-to-end-run.test.ts`) - Beyond CI: this provider has been verified in a production multi-tenant deployment against five harnesses (opencode, pi, codex, gemini, claude code) with real billed runs ## Risks - **Zero behavior change for any existing deployment.** The package is workspace-excluded; nothing in the server imports or loads it until stage 2's integration lands. No existing code paths are touched. - The default `sandbox-cr` backend depends on an alpha CRD (`agents.x-k8s.io/v1alpha1`); the README flags this and the `job` backend uses only stable APIs as a fallback. - Risk surface is confined to deployments that explicitly install and configure the plugin. - The default runtime images (`ghcr.io/paperclipai/agent-runtime-*`) are published by the stage 3 companion PR (#7934); until that lands, deployments must point `runtimeImage` at their own images. ## Model Used Claude Opus 4.8 (1M context), extended thinking, with tool use (Claude Code). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots (no UI changes) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (pending this push) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |