mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 21:05:21 +02:00
c8a136fb02394e18eaa3161dc6e46a482cfbc39b
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Zannis Kalampoukisandzannis
|
5db8ce3c44 |
fix(docker): make tini PID 1 in the server image so adopted orphans are reaped (#12137)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent runs execute inside the server container, and they spawn many
short-lived descendants: git, the adapter CLI, esbuild, sh
> - The server image sets `ENTRYPOINT ["docker-entrypoint.sh"]`, and
that entrypoint ends in `exec`, so node becomes PID 1
> - Node reaps only the children it spawned itself. It installs no
`SIGCHLD`/`waitpid` handler for orphans that the kernel re-parents onto
PID 1, so those orphans stay as zombies forever
> - Zombies accumulate monotonically. When the cgroup pid limit is
reached, every `fork()` in the container fails and the instance is dead
> - This pull request installs `tini` and makes it PID 1 in front of the
existing entrypoint, adds a behavioural test that proves reaping, and
adds a `pids_limit` backstop to both compose files
> - The benefit is that a long-running container no longer degrades into
total fork failure, and a future regression is caught by CI instead of
by an outage
Depends-on: none — this change is self-contained in the image build and
its tests, and it touches no other in-flight branch
## Linked Issues or Issue Description
No public GitHub issue exists for this defect. It was found on a live
long-running instance. Description follows the bug report template.
**What happened?**
The server container ran for 22 hours and reached 2039 of 2048 pids in
its cgroup. Of 1760 processes, 1731 were zombies, and all 1731 had PID 1
as their parent. PID 1 was `node --import
./server/node_modules/tsx/dist/loader.mjs server/dist/index.js`. Zombies
accrued at about 79 per hour and were never reaped. The oldest zombie
was 20.8 hours old against a container uptime of 22.0 hours, so nothing
had been reaped since boot. Once the pid limit was reached, `git` and
`gh` failed with `pthread_create failed: Resource temporarily
unavailable`.
**Expected behavior**
PID 1 reaps orphaned processes that the kernel re-parents onto it. The
pid count of a long-running container stays flat instead of growing
without bound.
**Steps to reproduce**
1. Start the server image without `docker run --init` and without `init:
true`.
2. Run agent work that spawns descendants which outlive their immediate
parent.
3. Read `/sys/fs/cgroup/pids.current` and count processes in `Z` state
over several hours.
4. The zombie count grows monotonically and every zombie has PPID 1.
**Relevant logs or output**
```
cgroup pids.current / pids.max : 2039 / 2048
total processes : 1760
zombies : 1731 (98.4%)
parent of every zombie : PID 1 (1731/1731)
PID 1 cmdline : node --import .../tsx/dist/loader.mjs server/dist/index.js
container uptime : 22.0 h
oldest zombie : 20.8 h median: 14.4 h
zombie names : git 717, claude 280, MainThread 167, sleep 141,
esbuild 138, postgres 76, sh 65, sccache 50
```
**Additional context**
The fix pattern is already in this repository.
`docker/agent-runtime/Dockerfile.base` installs `tini` and sets
`ENTRYPOINT ["/usr/bin/tini", "--"]`. It was never applied to the server
image.
## What Changed
- `Dockerfile`: install `tini` in the `base` stage and set `ENTRYPOINT
["/usr/bin/tini", "--", "docker-entrypoint.sh"]`. The entrypoint stays
in the exec chain, so UID/GID remapping, `gosu`, and graceful shutdown
are unchanged.
- `scripts/assert-orphan-reaping.sh` (new): a behavioural probe. It
spawns a leader that forks a grandchild, exits the leader, and asserts
that the orphaned grandchild leaves `Z` state instead of persisting. It
fails closed if the grandchild is not re-parented onto PID 1, so a pass
cannot mean the check ran too early.
- `.github/workflows/docker.yml`: run that probe against the pushed
image after the publish step. The publish step is multi-arch with `push:
true`, so nothing is loaded into the runner daemon and the pushed tag is
the only thing to test. The cloud variant is `FROM production` and
inherits the same `ENTRYPOINT`.
- `scripts/docker-build-test.sh`: run the same probe against a local
build.
- `docker/docker-compose.yml` and
`docker/docker-compose.quickstart.yml`: add `pids_limit: 2048` as a
backstop, so a future leak dies visibly at its own ceiling instead of
starving the host of pids.
- `server/src/__tests__/container-init-reaping.test.ts` (new): 13
assertions that guard the configuration the probe depends on.
No per-orchestrator init lever was added. The image owning PID 1 covers
compose, plain `docker run`, the quadlet units, and the ECS task
definition in one place. Adding `init: true` in compose or
`initProcessEnabled` on the ECS task would nest a second init around
`tini`, and `tini` then warns on every boot that it is not PID 1. The
new test asserts the absence of both levers across all three manifests,
so the decision survives the next edit.
## Verification
| Check | Result |
|---|---|
| `scripts/assert-orphan-reaping.sh` against a real init | Grandchild
re-parented to PPID 1, then reaped. Exit 0. |
| Same probe forced against a genuine zombie | Reports `Z` and fails.
The failure branch is not vacuous. |
| Config guard against the pre-fix files | Exactly the 3 relevant
assertions turn red. |
| Config guard with `tini` removed from `apt-get` but the comments kept
| Red. It checks the install, not a mention of the name. |
| `cd server && npx vitest run
src/__tests__/container-init-reaping.test.ts` | 13 passed |
| `npx tsc --noEmit -p server` | Clean |
| `node scripts/check-docker-deps-stage.mjs` | PASS |
| `node --test scripts/release-verify-workflow.test.mjs` | 8 passed |
Not verified locally: no container runtime is available in the authoring
environment, so the probe has not run against a build of this image. The
new `docker.yml` step runs it against the pushed image on this PR.
## Risks
Low risk, but it is an image and entrypoint change, so it affects
deployments.
- `tini` adds one small package to the `base` stage.
`docker/agent-runtime/Dockerfile.base` already installs it from the same
Debian archive.
- Signal handling changes shape: `tini` receives `SIGTERM` and forwards
it to the entrypoint, which `exec`s node. `tini` forwards signals to its
direct child by default, and the exec chain keeps node as that child, so
graceful shutdown is preserved. A reviewer should confirm this on a real
stop.
- `pids_limit: 2048` is new for compose users. A deployment that
legitimately needs more than 2048 processes would now hit the ceiling.
The measured steady state on a busy instance was under 400.
- If a deployment already passes `--init` or `init: true`, `tini` runs
under another init and prints a warning that it is not PID 1. Reaping
still works because the outer init handles it. The compose files in this
repository do not set `init: true`.
## Model Used
Claude Opus 5 (`claude-opus-5`), extended thinking, with tool use and
code execution in an agent harness.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local issues or links
- [x] My branch name describes the change and contains no internal
ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: zannis <1011451+zannis@users.noreply.github.com>
|
||
|
Nicky LeachandPaperclip
|
38d8f37172 |
fix(build): enforce Node 24 across Paperclip (#11792)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip runs across the CLI, server, adapters, plugins, CI, and container images. > - These surfaces declared different Node.js versions from 20 through 24. > - A newer `@types/node` major can expose APIs that the supported runtime does not provide. > - Node.js 20 is no longer a suitable project baseline, and Node.js 24 is the current LTS line. > - This pull request sets Node.js 24.11.0 as one repository-wide baseline, adds a drift check, and gives users actionable startup guidance when their runtime is too old. > - The benefit is one clear runtime contract for development, release, installation, and published packages. ## Linked Issues or Issue Description Refs #2734 Refs #11727 Refs #739 ## What Changed - Require Node.js 24.11.0 or newer in all 42 package manifests and runtime checks. - Use Node.js 24 in GitHub Actions, Docker images, smoke images, sandbox setup, portable installs, and esbuild targets. - Align every direct `@types/node` declaration on `^24.0.0`. - Prevent Dependabot from opening major `@types/node` upgrades without a matching runtime decision. - Add `.nvmrc` and a CI policy check for Node version drift. - Update ACP version gates, tests, and user documentation for the new minimum. - Print a non-blocking warning on CLI and server startup when Node is unsupported, with remediation through a version manager or the documented downloaded `install.sh` workflow. - Deduplicate that warning when `paperclipai run` boots the CLI and server in the same process. ## Verification - `node scripts/check-node-version-policy.mjs` - `node --check scripts/check-node-version-policy.mjs` - `node --check cli/esbuild.config.mjs` - `node --check scripts/generate-npm-package-json.mjs` - `bash -n scripts/install.sh scripts/test-install-sh-docker.sh scripts/e2e-install-lifecycle.sh` - Parsed all 42 package manifests and confirmed `engines.node` is `>=24.11.0`. - `git diff --check` - `vitest run packages/adapter-utils/src/sandbox-install-command.test.ts` passed with 3 tests. - `vitest run cli/src/node-version.test.ts` passed with 4 tests. - Directly exercised the shared warning helper for unsupported-version messaging and same-process deduplication. - The focused exe.dev suite could not resolve the locally unbuilt plugin SDK from this isolated worktree. A full offline workspace install was also blocked because the package-manager signature verifier requires registry access. The full suite was not run locally; draft CI performs a clean install and evaluates the wider impact. ## Risks - This is a breaking runtime change for users, plugins, and deployments that still use Node.js 20 or 22. - Published workspace packages will now produce an engine warning or failure in strict package managers on older Node.js releases. - Node.js 24 can reveal dependency, native module, Playwright, or agent CLI compatibility issues in CI. - The bootstrap installer now installs Node.js 24 when the current runtime is older than 24.11.0. - The portable sandbox fallback is pinned to Node.js 24.11.0 and depends on that upstream tarball remaining available. - Unsupported runtimes continue booting after a warning, so a later incompatibility can still fail at its point of use. - The CLI and server share the warning policy through the published `@paperclipai/shared` package; packaging checks must keep that subpath export available. - This PR does not commit `pnpm-lock.yaml` because repository policy assigns lockfile generation to CI. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex based on GPT-5. The exact deployment ID and context window are not exposed in this session. Reasoning, repository tools, shell execution, and GitHub tools were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
f94f6003c6 |
fix(release-smoke): pin the smoke container to the lan bind preset (#11189)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The release subsystem's nightly lane gates every nightly on the release smoke suite, which boots the published artifact in a Docker container and drives real onboarding > - The gate kept failing even after the readiness budget fix (#11187), and the new container-log dump revealed the server was healthy but listening on 127.0.0.1 inside the container, unreachable through Docker's port mapping > - `onboard --yes` without an explicit `--bind` prefers trusted-local quickstart defaults: it writes a loopback bind into the instance config and ignores the deployment env vars the harness passes, and that config outranks `HOST` at runtime > - This pull request pins the smoke container to the `lan` bind preset and adds a wiring test for it > - The benefit is a working nightly gate, verified end to end against a real published canary ## Linked Issues or Issue Description **Subsystem affected** Release smoke testing: `docker/Dockerfile.onboard-smoke`, `scripts/__tests__/release-verify-workflow.test.mjs`. **Problem or motivation** Nightly run 31428558684 failed in smoke with the server unreachable at the mapped port for the full 420 second budget. The container logs (captured thanks to #11187) show a fully booted server with `Bind loopback (127.0.0.1)`. The harness sets `HOST=0.0.0.0` and the deployment env vars, but `onboard --yes` without `--bind` deliberately prefers trusted-local defaults, writes `bind: loopback` into the instance config, and the config outranks `HOST` at runtime. A loopback listener inside a container is invisible to the port mapping, so the health check can never pass. This behavior predates the current stable, so the harness was silently broken against every recent version — it only surfaced now because the nightly lane is the suite's first CI consumer. **Proposed solution** Pass `--bind lan` in the smoke container command (the flag is supported by `latest` and canary alike; it selects the all-interfaces preset and keeps the env-driven authenticated deployment), and pin the flag with a wiring test so it cannot regress silently. ## What Changed - `docker/Dockerfile.onboard-smoke`: the onboard command is now `onboard --yes --bind lan --data-dir ...`, with a comment explaining why the flag is load-bearing - `scripts/__tests__/release-verify-workflow.test.mjs`: wiring test asserting the smoke Dockerfile pins a non-loopback bind preset ## Verification - Full local harness run against the real nightly candidate `2026.810.0-canary.1`: container healthy, bind banner shows `lan (0.0.0.0)`, authenticated bootstrap completed (admin created, bootstrap invite accepted, board session verified), `/api/health` returns `bootstrapStatus: ready` - `node --test scripts/__tests__/release-verify-workflow.test.mjs`: 4 pass - After merge: dispatch `release.yml` with `channel: nightly` to run the gate end to end in CI ## Risks - Low. The change only affects the smoke container. `--bind lan` inside a container exposes the port to the container network only; reachability from outside still goes through Docker's explicit port mapping ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) in Claude Code, with extended thinking and full tool use (CI log forensics, upstream source tracing, local Docker reproduction and verification). All changes model-authored under human direction. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (pending — will confirm before merge) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (pending — will confirm before merge) - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Nicky LeachandPaperclip
|
7083c275c8 |
refactor(sandbox): retire the dead noProfile flag from the exec path (#10461)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The sandbox exec path starts agent commands and passes runtime options to the server and plugin layers > - This path kept a noProfile flag after the exec wrappers stopped sourcing a login profile > - The flag no longer changed behavior, so it left dead API surface in the protocol and runtime helpers > - This pull request removes that dead flag from the plugin protocol, the server drivers, and the managed-runtime helpers > - It also updates the tests and points the agent runtime README at the sandbox requirements file > - The benefit is a smaller and clearer exec-path contract with no behavior change ## Linked Issues or Issue Description - No public GitHub issue exists. ### What happened? The sandbox exec path kept a `noProfile` field after the exec wrappers stopped sourcing a login profile. ### Expected behavior The plugin protocol, server drivers, and managed-runtime helpers should not expose or forward a dead field. ### Steps to reproduce 1. Run a managed-runtime command through the sandbox exec path. 2. Inspect the protocol payload and runtime helper inputs. 3. Observe that `noProfile` is present even though it no longer changes behavior. ### Paperclip version or commit `60c7da86fc7a6c1dbf37bbcd86e25ecaaff01607` ### Deployment mode Built from source (pnpm dev / pnpm build) ### Additional context This pull request removes the dead field, updates the affected tests, and updates the README note for the sandbox profile path. ## What Changed - Removed noProfile from the plugin protocol and the server exec-path call sites. - Updated the managed-runtime helpers to use the narrower exec-path contract. - Updated the affected tests and added the README pointer to SANDBOX-REQUIREMENTS.md. ## Verification - `git grep -n "noProfile" -- packages/ server/` returns zero matches. - `tsc --noEmit` passed for `@paperclipai/adapter-utils`, `@paperclipai/plugin-sdk`, and `@paperclipai/server`. - `command-managed-runtime.test.ts` passed: 22/22. - `environment-runtime.test.ts` passed: 24/24. ## Risks - Low risk. The flag was already a no-op. - A hidden external caller may still send the removed field. ## Model Used - OpenAI GPT-5, tool-enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Evyatar Bluzer
|
b287281940 |
docs: add inline comments to docker quickstart compose file (#2431)
## Problem The quickstart docker-compose file was recently moved to \`docker/docker-compose.quickstart.yml\` during the Docker reorganization but still have zero inline comments. When new user copy this file for self-hosting, they see variables like: - \`BETTER_AUTH_SECRET\` - what is this? How to generate? - \`PAPERCLIP_DEPLOYMENT_MODE: "authenticated"\` - what other modes available? - \`PAPERCLIP_DEPLOYMENT_EXPOSURE: "private"\` - what does private vs public mean? - \`OPENAI_API_KEY\` and \`ANTHROPIC_API_KEY\` - both required? Or just one? They have to go read DOCKER.md or other docs to understand each variable. ## What I changed Added inline YAML comments directly in the file: - Header block with step-by-step quickstart commands (cd docker, export, docker compose up) - Section headers grouping LLM keys, deployment settings, and secrets - Comment explaining each non-obvious variable with valid values - Note about \`BETTER_AUTH_SECRET\` with openssl generation command - Comment on the volume explaining what data it persist No functional change - only YAML comments added. |
||
|
Jannes StubbemannandClaude Fable 5
|
4f539625f7 |
build(agent-runtime): ship ripgrep in the base image (#8976)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents execute inside sandboxed runtime containers built from `docker/agent-runtime/Dockerfile.base` > - OpenCode's skill tooling shells out to ripgrep; when `rg` is not on PATH it tries to download a pinned build from `github.com/BurntSushi/ripgrep/releases` at run time > - In a sandbox with locked-down egress that download hangs ~127s and then fails, burning run budget on every agent run before the agent reaches its actual work > - The root repo `Dockerfile` already installs ripgrep; the agent-runtime base image drifted without it > - This pull request adds `ripgrep` to the base image's apt install so OpenCode uses the system binary and never reaches for the network > - The benefit is that every sandboxed agent run stops wasting ~2 minutes on a doomed download and spends its budget on real work ## Linked Issues or Issue Description No existing issue — problem described here per the bug report template: - **What happened:** Sandboxed agent runs using OpenCode stall for ~127 seconds at startup, then log `Transport error ... BurntSushi/ripgrep/releases/download/...` before continuing degraded. - **Expected behavior:** The agent starts working immediately; skill tooling finds `rg` on PATH. - **Root cause:** The agent-runtime base image (`docker/agent-runtime/Dockerfile.base`) does not ship ripgrep, so OpenCode falls back to downloading a pinned build at run time, which egress-restricted sandboxes block. - **Reproduction:** Run any OpenCode-backed agent in a sandbox with locked-down egress using the current agent-runtime image; observe the startup hang and transport error. Supersedes #8859. ## What Changed - Added `ripgrep` to the existing `apt-get install --no-install-recommends` list in `docker/agent-runtime/Dockerfile.base` - Added an explanatory comment documenting why ripgrep must be present (run-time download fallback + egress-restricted sandboxes), restoring parity with the root repo `Dockerfile` ## Verification - `docker build -f docker/agent-runtime/Dockerfile.base .` then `docker run --rm <image> rg --version` — prints the ripgrep version from the system package - Run an OpenCode-backed agent in an egress-restricted sandbox on the new image: no `BurntSushi/ripgrep` download attempt, no ~127s startup stall ## Risks - Low risk: no behavior change beyond shipping one additional apt package in the base image; slightly larger image size > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude (Anthropic), model ID `claude-fable-5` (Fable 5), agentic coding via Claude Code with tool use; original change authored with Claude Opus 4.8 (1M context) and re-based/re-verified with Fable 5 ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
eedc7ddef2 |
Make ACP the default engine for local adapters (#9238)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Adapter packages are the bridge between the control plane and local agent harnesses such as Claude Code, Codex, and Gemini CLI. > - ACP support was concentrated in a separate `acpx_local` adapter, which made ACP feel like a separate agent choice instead of an execution capability of the harness adapters. > - Claude, Codex, and Gemini now have ACP-capable harnesses, so the native adapter should own ACP selection, fallback, config, transcript parsing, and environment diagnostics. > - The standalone ACPX adapter still needs a compatibility path for existing rows, but it should not be offered as an active adapter for new agents. > - This pull request moves the shared ACP runtime into `@paperclipai/acpx-engine`, wires Claude/Codex/Gemini local adapters to prefer ACP when prerequisites are available, and retires `acpx_local` to a tombstone. > - The benefit is one adapter per harness, richer ACP transcripts by default where possible, and a migration path for existing Claude/Codex ACPX agents. ## Linked Issues or Issue Description Closes #5932 — the broken default `acpx_local` Claude path is replaced by native `claude_local` ACP support, existing Claude/Codex ACPX rows migrate to native adapters, and new agents no longer choose the standalone ACPX adapter. Refs #4893 — original merged ACPX local adapter runtime that this PR replaces with native per-harness ACP engines. Refs #6590 — prior ACPX-Claude seamlessness work folded into the new native Claude ACP path. Refs #197 — related open generic ACP/Kiro adapter work; this PR does not close it because Kiro/custom generic ACP remains a separate adapter decision. Refs #7018 — related Kimi-specific `acpx_local` shell failure; this PR retires the built-in standalone adapter but does not add a native Kimi adapter. Refs #8864 — related ACPX prompt/API guidance PR; this PR moves runtime guidance into the shared/native ACP engine path instead of the old standalone adapter. Refs #8881 — related `acpx_local` POSIX shell failure from the old `acpx` pin; this PR updates ACP dependencies but does not claim custom/OMP ACP support as a first-class native adapter. Refs #8964 — related open `acpx_local` stderr cleanup PR; this PR makes the old runtime path obsolete for new agents but keeps it as a non-closing reference. Problem description: - The standalone `acpx_local` adapter duplicates Claude/Codex agent choices that already have first-class local adapters. - ACP should be an execution engine capability of each harness adapter when the underlying harness supports ACP. - Existing `acpx_local` agents should either migrate to native harness adapters or fail with an explicit retirement message instead of silently falling back to the process adapter. ## What Changed - Added `@paperclipai/acpx-engine` as the shared ACP execution, session-codec, CLI formatter, and UI parser package. - Wired `claude_local`, `codex_local`, and `gemini_local` to auto-select ACP by default when prerequisites pass, with `engine=cli` opt-out and `engine=acp` strict mode. - Added ACP config schema/UI fields, environment checks, session-codec preservation, transcript parsing, and adapter capability metadata for the native adapters. - Retired `acpx_local` to a server tombstone, removed its UI/package/runtime image surface, and added a migration for existing Claude/Codex ACPX agents. - Updated package manifests, lockfile, release tooling, docs, Kubernetes sandbox defaults, and tests. ## Verification - `corepack pnpm --filter @paperclipai/acpx-engine typecheck` - `corepack pnpm --filter @paperclipai/adapter-claude-local typecheck` - `corepack pnpm --filter @paperclipai/adapter-codex-local typecheck` - `corepack pnpm --filter @paperclipai/adapter-gemini-local typecheck` - `corepack pnpm --filter @paperclipai/acpx-engine exec vitest run` - `corepack pnpm --filter @paperclipai/adapter-claude-local exec vitest run src/server/acp.test.ts src/server/execute.acp-fallback.test.ts src/ui/build-config.test.ts` - `corepack pnpm --filter @paperclipai/adapter-codex-local exec vitest run src/server/acp.test.ts src/ui/build-config.test.ts` - `corepack pnpm --filter @paperclipai/adapter-gemini-local exec vitest run src/server/acp.test.ts src/ui/build-config.test.ts src/ui/parse-stdout.test.ts` - `corepack pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && corepack pnpm --filter @paperclipai/server exec tsc --noEmit` - `corepack pnpm --filter @paperclipai/server exec vitest run src/__tests__/adapter-routes.test.ts src/__tests__/adapter-session-codecs.test.ts src/__tests__/adapter-models.test.ts` - `corepack pnpm --filter @paperclipai/ui typecheck` - `corepack pnpm --filter @paperclipai/ui exec vitest run src/adapters/metadata.test.ts src/adapters/adapter-display-registry.test.ts src/components/AgentConfigForm.test.ts src/components/AgentConfigForm.render.test.tsx src/components/transcript/RunTranscriptView.test.tsx` - `node --test scripts/bootstrap-npm-package.test.mjs scripts/release-package-map.test.mjs scripts/verify-release-registry-state.test.mjs` Note: the server typecheck script calls `pnpm` internally; this dev shell exposes pnpm through Corepack only, so I ran the two script steps manually with `corepack pnpm`. ## Risks - Migration changes existing `acpx_local` Claude/Codex agents to native adapter types and clears old ACPX task sessions/runtime state. - Custom ACP commands remain on the retired tombstone and will need a separate future adapter/plugin path. - ACP auto-selection depends on local Node and ACP server command prerequisites; remote and unsupported environments fall back to CLI unless `engine=acp` is explicit. - `@paperclipai/acpx-engine` is a new public package and needs npm trusted-publishing bootstrap before release automation can publish it. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex coding agent. Exact hosted model build and context-window size are not exposed in this runtime. Tool use included shell execution, repository editing, GitHub CLI operations, and local test/typecheck execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
fd2f82ac5b |
[codex] Add built-in Hermes adapters (#8543)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent adapters are the boundary between the control plane and the runtimes that actually do work. > - Hermes support needs to be available as first-class local and gateway adapters while still preserving the adapter-manager override path for external packages. > - The adapter work touches runtime execution, UI adapter metadata, onboarding prompts, scoped credentials, release packaging, and smoke coverage, so the handoff needs concrete verification rather than only unit tests. > - This pull request adds built-in Hermes local and Hermes gateway support, keeps external adapter overrides compatible, and documents/tests the gateway flow end to end. > - The benefit is that operators can hire Hermes-backed agents without a manual plugin install, while self-hosted installs can still override/shadow the built-ins through Adapter manager packages. ## Linked Issues or Issue Description No public GitHub issue exists for this exact Hermes built-in adapter, gateway onboarding, and release-source work. Problem description: - Hermes local and gateway adapters need a public, reviewable source path in the monorepo so package artifacts and built-in adapter behavior match the application source. - Operators need built-in `hermes_local` and `hermes_gateway` adapter choices without losing the ability to install external Hermes packages as overrides. - Gateway onboarding needs secure defaults for API server URLs, API keys, and generated agent setup text. - Hermes-originated task bridge credentials need narrower API-key scope configuration. - Related public PRs found during duplicate search include #3027, #2363, #7544, #7950, #8095, and #8543. ## What Changed - Added the unified Hermes adapter package with local and gateway server/UI/CLI exports, config schemas, transcript parsing, model detection, and package metadata. - Registered `hermes_local` and `hermes_gateway` as built-in adapters across shared constants, server registries, CLI packaging, and UI adapter registries. - Kept the external adapter override path compatible so installed Hermes packages can shadow built-ins and restore the built-in parser when disabled. - Added Hermes gateway onboarding docs, board-operator docs, Docker smoke assets, and shell smoke harnesses for join/e2e validation. - Added scoped task-bridge API-key support, authorization checks, issue-origin handling, and tests for Hermes-created Paperclip tasks. - Hardened gateway transport and redaction behavior for API keys, headers, session data, and smoke diagnostics. - Updated release packaging/bootstrap checks for the Hermes packages while leaving `pnpm-lock.yaml` out of the PR per repository policy. ## Verification Targeted local verification recorded before PR handoff: - `pnpm --filter @paperclipai/hermes-paperclip-adapter exec vitest run src/gateway/server/execute.test.ts` — 14/14 passed. - `pnpm test:hermes-gateway-smoke` — 6/6 passed. - Hermes package typecheck/build checks passed. - Focused server/UI adapter tests passed — 31/31. - Release helper Node tests passed — 18/18. - `git diff --check origin/master..HEAD` passed. Fresh Docker E2E smoke evidence: - Ran `pnpm smoke:hermes-gateway-e2e` on 2026-06-26 with a fresh state directory and fresh Docker container against a live Paperclip dev server. - Hermes direct execution reached `completed`. - Hermes stop/cancel path reached `cancelled`. - Hermes gateway created a Paperclip task, Paperclip ran the Hermes agent, and the task reached `done` with the expected marker response. - Temporary board auth keys, token files, smoke state, and Docker containers were cleaned up after the run. PR checks on head `b5eae40ce`: - GitHub Actions passed: `policy`, `review`, `Typecheck + Release Registry`, all general test shards, all serialized server shards, `Build`, `Canary Dry Run`, `e2e`, and aggregate `verify`. - External checks passed: Snyk and Socket Project Report. - External Socket Pull Request Alerts remained pending after the first-party CI matrix completed. ## Risks - Medium risk: this spans adapter registration, package publishing, gateway execution, onboarding docs, API-key scoping, and UI adapter metadata. - Migration risk is low: the scope-config migration adds a nullable column and does not rewrite existing keys. - Gateway execution depends on operator-provided Hermes API configuration; the smoke covers the Docker gateway path but real deployments may differ by network/auth setup. - Direct Greptile review on the latest expanded diff is file-count limited, although the commitperclip review gate passed. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5 coding agent, tool use enabled in a local repository workspace. Context window size is not exposed in this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Commitperclip review gate is green; direct Greptile review is file-count limited on the latest expanded diff - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Jannes StubbemannandClaude Opus 4.8
|
398d746093 |
build(agent-runtime): harness runtime images for sandboxed execution (stage 3/3) (#7934)
> [!NOTE] > This is **stage 3 of 3** of the staged Kubernetes contribution: stage 1 is the kubernetes sandbox-provider plugin (#5790), stage 2 is the provider backend/hardening refresh filed separately, and this stage ships the runtime images those sandboxes run. ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandboxed agent execution (Refs #248) runs each agent turn in an isolated environment; the kubernetes sandbox provider (stage 1, #5790) schedules those runs as hardened pods > - A sandbox pod needs a runtime image with the harness CLI preinstalled: installing CLIs at run start is slow, flaky, and needs network egress the sandbox should not have > - There is no first-party image family for this, so every deployer would have to hand-roll Ubuntu + Node + CLI images per harness and solve signal handling, non-root, and image chaining themselves > - This PR ships the agent-runtime image family: a hardened base (non-root uid 1000, tini, git, the agent shim) plus one derived image per harness, a buildx bake file that chains them, and a publish workflow with cosign keyless signing > - The benefit is that any sandbox infrastructure, the kubernetes provider or otherwise, gets ready-made, signed, security-hardened per-harness runtime images that are verified in production across five harnesses ## Linked Issues or Issue Description Refs #248 (sandboxed agent execution proposal) and #5790 (the kubernetes sandbox provider, stage 1 of this contribution, which consumes these images as per-run runtime images via its adapter defaults). No issue covers the image gap itself, described in-PR: sandbox providers reference `ghcr.io/paperclipai/agent-runtime-*` images, but the repository contains neither the Dockerfiles nor the workflow that builds and publishes them. Without this, self-deployers cannot reproduce or audit the images their agent runs execute in. ## What Changed - `docker/agent-runtime/Dockerfile.base`: foundation image. Ubuntu 22.04 + Node 22 + git + tini (PID 1, signal propagation) + non-root `paperclip` user (uid/gid 1000) + the agent shim compiled in a Go build stage. `WORKDIR /workspace`, entrypoint `tini -- paperclip-agent-shim`. - One derived Dockerfile per harness: `opencode` (opencode-ai), `pi` (@mariozechner/pi-coding-agent), `codex` (@openai/codex), `gemini` (@google/gemini-cli, plus headless auth-mode settings), `claude` (@anthropic-ai/claude-code, symlinked as `claude-code`). Each installs the CLI as root, returns to uid 1000, and asserts the binary is on PATH at build time. - `acpx` and `hermes` Dockerfiles are included in the bake group but are not in the default publish scope (hermes is a stub until a CLI package exists). - `docker/agent-runtime/buildx-bake.hcl`: builds the whole family in one pass. Derived targets chain off the `base` target through bake `contexts` (the literal registry in each `FROM` is overridden to `target:base` at build time, so no intermediate push is needed). `REGISTRY` (default `ghcr.io/paperclipai`) and `VERSION` are overridable variables. - `tools/agent-shim/`: a small Go shim that runs as the container command. It reads `/run/paperclip/runtime-command.json` (`{ "command", "args" }`), resolves the harness CLI on PATH, and `syscall.Exec`s it so SIGTERM from the kubelet reaches the harness directly. Harness-agnostic, with unit tests. - `.github/workflows/agent-runtime-images.yml`: builds and pushes the default scope (base, opencode, pi, codex, gemini, claude) for linux/amd64 on `workflow_dispatch` (explicit version tag) or pushes to `master` touching these paths, then signs every digest with cosign keyless OIDC. Uses only `GITHUB_TOKEN`; no extra secrets. - `docker/agent-runtime/README.md`: image lineup, base contents, local build instructions, the runtime-command contract, and the security model. Additive only: nothing in the product loads these images. Deployments opt in via their sandbox provider configuration (for example the kubernetes plugin's image settings). ## Verification - `cd tools/agent-shim && go build ./... && go test ./... && go vet ./...`: all passing. - `docker buildx bake -f docker/agent-runtime/buildx-bake.hcl --print base opencode pi codex gemini claude`: resolves cleanly; every tag and build context lands on `ghcr.io/paperclipai/agent-runtime-*` and derived targets map the base ref to `target:base`. - Workflow YAML validated (parses, single job, no org-specific secrets). - This exact image family (built from these Dockerfiles, bake file, and workflow) is what runs agent execution in production on paperclip.inc, verified end-to-end across five harnesses (opencode, pi, codex, gemini, claude): each as a full loop from assigned issue to per-run runtime image in a sandboxed pod to completed run. ## Risks - Low risk: purely additive, nothing in paperclip-server or the UI references these files. The workflow only triggers on its own paths. - Derived images install harness CLIs `@latest` at build time; a broken upstream CLI release would surface at image build, not at run time, and the PATH assertion fails the build rather than shipping a broken image. - The hermes image is an explicit stub (documented in its Dockerfile) until a hermes CLI package exists; it is outside the default publish scope. - cosign signing is keyless OIDC with the workflow identity; no long-lived signing keys are introduced. ## Model Used Claude Opus 4.8 (claude-opus-4-8, 1M context, extended thinking, tool use via Claude Code). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (no UI changes) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Neeraj Kumar Singh BandClaude Opus 4.6
|
f0f9460d1d |
docs: AWS ECS Fargate deployment runbook (#3897)
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies and ships
a
> "local-first, cloud-ready" deployment model
> - The deploy docs currently cover local/Docker but not a production
> cloud target, so teams asking "how do I put this behind a real domain"
> have no canonical path
> - We already support Docker images, RDS-compatible Postgres, and an
EFS
> storage profile, so AWS ECS Fargate is a natural fit
> - Without a runbook, each team reinvents VPC, security groups, TLS,
and
> secrets wiring and usually gets at least one step wrong
> - This pull request adds `docs/deploy/aws-ecs.md`, an ECS
task-definition
> template, and an `.env.aws.example`, cross-linked from the deploy
overview
> - The benefit is a single, reproducible ~$110/mo path to a production
> deployment, plus a full teardown for throwaway environments
## What Changed
- New `docs/deploy/aws-ecs.md` — an 11-step ECS Fargate runbook covering
ECR,
VPC, RDS, EFS, Secrets Manager, IAM, ALB, and ECS service with the
deployment circuit breaker enabled
- New `docker/ecs-task-definition.json` — Fargate-ready task definition
with
`<ACCOUNT_ID>`, `<REGION>`, `<EFS_ID>`, `<DOMAIN>` placeholder tokens
- New `docker/.env.aws.example` — documents every non-secret env var the
ECS deployment needs
- `docs/deploy/overview.md` — one-line cross-reference to the new guide
- Greptile feedback addressed in follow-up commits:
- `containerName` in the service-create call now matches
`paperclip-server` in the task definition
- HTTP :80 listener added that 301-redirects to :443
- Dedicated RDS DB subnet group created before `create-db-instance`
- EFS teardown polls on mount-target deletion instead of `sleep 30`
## Verification
- Walked every step of the runbook against the task definition to
confirm
variable names (`$ALB_SG`, `$ECS_SG`, `$RDS_SG`, `$EFS_SG`, `$TG_ARN`,
`$LISTENER_ARN`, `$HTTP_LISTENER_ARN`, `$EFS_ID`, `$RDS_ENDPOINT`, etc.)
are
defined before they are referenced
- Confirmed the `containerName` in Step 10 (`paperclip-server`) matches
`docker/ecs-task-definition.json` line 11
- Confirmed the `sed` placeholder substitution in Step 8 matches the
tokens
in the task definition template
- Teardown order was checked in reverse-dependency order: ECS service →
listeners → target group → ALB → RDS (waits for deletion) → DB subnet
group → EFS mount targets (polled) → EFS → secrets → SGs → ECR → IAM →
log group
## Risks
- **Low risk for the repo.** Docs-only change plus two template files
under
`docker/`; no runtime code paths are touched and nothing is imported by
the build.
- **Risk for users who follow the runbook:** AWS bills accrue
immediately
once RDS/ALB/EFS exist. The runbook calls this out and includes a full
teardown procedure. Placeholder tokens (`<ACCOUNT_ID>`, `<REGION>`,
`<EFS_ID>`, `<DOMAIN>`) are documented so nothing is silently
hard-coded.
## Model Used
- Claude (Anthropic), model `claude-opus-4-6`, ~200K context window,
extended thinking mode on, used with tool access (file edit, shell) via
Claude Code. The Greptile follow-up commits were authored the same way.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have run tests locally and they pass — N/A for docs/config
templates; validated by reading
- [x] I have added or updated tests where applicable — N/A for docs
- [x] If this change affects the UI, I have included before/after
screenshots — N/A, no UI
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|
Cody (Radius Red)andPaperclip
|
420cd4fd8d |
chore(docker): improve base image and organize docker files
- Add wget, ripgrep, python3, and GitHub CLI (gh) to base image - Add OPENCODE_ALLOW_ALL_MODELS=true to production ENV - Move compose files, onboard-smoke Dockerfile to docker/ - Move entrypoint script to scripts/docker-entrypoint.sh - Add Podman Quadlet unit files (pod, app, db containers) - Add docker/README.md with build, compose, and quadlet docs - Add scripts/docker-build-test.sh for local build validation - Update all doc references for new file locations - Keep main Dockerfile at project root (no .dockerignore changes needed) Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandClaude Opus 4.6
|
6f931b8405 |
Add Docker setup for untrusted PR review in isolated containers
Adds a dedicated Docker environment for reviewing untrusted pull requests with codex/claude, keeping CLI auth state in volumes and using a separate scratch workspace for PR checkouts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Dotta
|
be50daba42 | Add OpenClaw onboarding text endpoint and join smoke harness |