Files
PaperClipAI/docs/adapters/claude-local.md
T
ff7e7b4963 fix(claude-local): make sandbox tool commands the first OOM victims (#16844)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Hosted `claude_local` runs execute in a sandbox. The sandbox has a
fixed memory limit (4 GiB in the observed sandboxes).
> - In that sandbox, every process has the same `oom_score_adj`. When a
Bash tool command uses most of the memory, the kernel kills the largest
process. That can be the Claude Code process.
> - When Claude Code is killed, `claude-agent-acp` reports "The
connection to Claude was lost." and the whole run fails. Server logs
show `Claude Agent process died: Claude Code process terminated by
signal SIGKILL` for each failure.
> - A failed tool command is recoverable. A dead agent is not.
> - This pull request makes Claude's tool commands the first
out-of-memory victims in managed sandboxes.
> - The benefit is that a memory-heavy command fails with exit 137, and
the agent continues the run.

## Linked Issues or Issue Description

Refs #16366 (diagnostic PR for the same failure). Related: #14509
protects the host control plane from local agent processes. That PR does
not cover a sandboxed agent and its own tool commands.

**What happened?**

Hosted `claude_local` runs fail with `acpx_turn_failed` and "The
connection to Claude was lost." while one `execute` (Bash) tool call is
pending. The pending commands are memory-heavy, for example `tsc
--noEmit` with a 3.3 GB heap, a monorepo `pnpm install`, or a type check
plus `vitest`. At the same second, the server log shows `Claude Agent
process died: Claude Code process terminated by signal SIGKILL`. Over 10
days, all 12 such lines on one staging stack were `SIGKILL`, and they
match the failed runs. The sandbox memory cgroup has `memory.max` 4 GiB
and `memory.oom.group` 0, so the kernel kills one process. Every process
has `oom_score_adj` 0, so the kernel picks the largest one.

**Expected behavior**

Under memory pressure, the kernel kills the tool command. Claude gets a
failed command (exit 137) and continues the run.

**Steps to reproduce**

1. Start a `claude_local` run on a sandbox target with a 4 GiB memory
limit.
2. Ask the agent to run a memory-heavy command in a large repository,
for example a full `tsc --noEmit` and a test run together.
3. The kernel kills the Claude Code process, and the run fails with "The
connection to Claude was lost."

**Agent adapter(s) involved**

Claude Code (`claude_local`, ACP lane and CLI lane, managed sandbox
targets).

## What Changed

- `claude-config.ts`: add `CLAUDE_TOOL_SHELL_PREFIX_SCRIPT`. Claude Code
calls a `CLAUDE_CODE_SHELL_PREFIX` with one argument, the full command
string. The script raises its own `oom_score_adj` to 1000 (no privilege
is necessary to raise it). Then it runs the command string in the shell
that Claude Code selects: `CLAUDE_CODE_SHELL`, then `SHELL`, when either
is an executable bash or zsh, else zsh or bash from `PATH` (bash first
when `SHELL` names bash).
- `buildRemoteClaudeConfigMaterializationCommand` can install the script
(umask 077, write to `.partial`, `chmod 700`, `mv`). The install runs
last in the existing materialization command, so it adds no sandbox
round trip. It prints a marker only when the file is in place.
- `materializeRemoteClaudeConfig` returns `toolShellPrefixInstalled`. It
is true only when the command exits 0 and prints the marker.
- ACP lane (`acp.ts`) and CLI lane (`execute.ts`): set
`CLAUDE_CODE_SHELL_PREFIX` to the installed script only when the install
is confirmed. A prefix path that does not exist would fail every Bash
tool call, so a failed install keeps the previous behavior. An operator
`CLAUDE_CODE_SHELL_PREFIX` in the adapter env is kept, and no script is
installed.
- `docs/adapters/claude-local.md`: add "Sandbox memory limits".

## Verification

- `npx vitest run src/server/claude-config.test.ts` in
`packages/adapters/claude-local`: 10 pass, 6 new. The new tests install
the script with the real materialization command through `/bin/sh` and
check:
- the file content, mode 0700, the marker, and that no `.partial` file
remains;
- the command runs with `oom_score_adj` 1000 while the caller keeps its
own score (Linux only);
- shell selection: a valid bash `SHELL`, an invalid `CLAUDE_CODE_SHELL`
that falls back to `SHELL`, and a `SHELL` of `/bin/sh` that falls back
to bash from `PATH`;
  - quoting and the exit status of the command are kept;
  - an operator prefix wins, and no marker means no install.
- `acp.test.ts`: the managed-config test now checks that
`CLAUDE_CODE_SHELL_PREFIX` names an executable 0700 file next to the
config dir. A new test checks that an operator prefix is kept and no
script is installed. Both use the local sandbox runner, which runs the
setup commands for real.
- `tsc --noEmit -p packages/adapters/claude-local` is clean.
- Two tests fail in my sandbox both with and without this change: `uses
the same default model in ACP startup and session identity` and
`reselects the full assignment and bootstrap guidance after a failed
resume`. They also fail on clean `master` in the same environment, so I
think they depend on that environment. CI covers them.
- Real Claude Code check (Claude Code 2.1.286): with
`CLAUDE_CODE_SHELL_PREFIX` set to the script, a Bash tool call printed
`oom_score_adj` 1000 and ran in zsh (the selected shell), and its parent
was the `claude` process at score 0. Without a prefix file that exists,
every Bash tool call fails with exit 127. This is why the env is set
only after a confirmed install.
- Real OOM check in a 4 GiB Daytona sandbox (`memory.oom.group` 0). A
stand-in "agent" process held 1.2 GB at score 0, and a tool command
started 14 processes of 250 MB each:
- without the prefix, the kernel killed the stand-in agent (SIGKILL,
137; `oom_kill` +1);
- through the installed prefix, the stand-in agent stayed alive, and the
kernel killed 5 tool processes (`oom_kill` +5).
- Assumption that could invalidate the design: Claude Code passes one
argument (the command string) to the prefix and honors the variable
through `claude-agent-acp`. I checked the argument shape with a logging
prefix, and `claude-agent-acp` 0.73.0 spawns the CLI with `{
...process.env }`.

## Risks

- Every Bash tool command and hook in a managed sandbox now runs through
one more `sh` process and an `exec`. The cost is a few milliseconds for
each command.
- If a future Claude Code release changes how it calls
`CLAUDE_CODE_SHELL_PREFIX`, Bash tool calls can fail. The tests above
and the documented variable limit this risk. Removing the variable
restores the old behavior.
- The inner shell is not a login shell. Claude Code sources its own
shell snapshot inside the command string, so the environment stays the
same. If Claude Code runs without a snapshot, it uses a login shell.
Only exported variables then pass through to the inner shell.
- If the selected shell differs from the one that Claude Code chose (for
example, a custom `CLAUDE_CODE_SHELL` that is not bash or zsh), the
script uses the same fallback order as Claude Code.
- This does not help when the Claude Code process itself uses most of
the memory. The local (non-sandbox) lanes do not change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with tool use
and code execution. It read the repository and the Claude Code CLI
behavior, ran the OOM experiment, edited the code, and ran the tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass (two tests fail on clean
`master` in my sandbox too; see Verification)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-04 12:09:54 -08:00

175 lines
9.7 KiB
Markdown

---
title: Claude Code
summary: Claude Code local adapter setup and configuration
---
The `claude_local` adapter runs Anthropic's Claude Code CLI locally. It supports session persistence, skills injection, and structured output parsing.
## Quota waits
Claude ACP runs that end with a typed provider-quota error retain the quota
classification and any parsed reset time. Recovery waits until that time, or
uses its existing one-hour quota backoff when no reset time is available.
This includes the Claude bridge's typed “The Claude account has no available
quota.” fallback, which carries no reset timestamp.
The adapter inspects the terminal provider message in memory; the run result
and run log retain only the generic failure message, recovery labels, and reset
timestamp. Context, turn, rate, and configured budget limits are not treated as
subscription quota exhaustion merely because ACP labels them `limit`.
## Prerequisites
- Claude Code CLI installed (`claude` command available)
- Either `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN` in adapter or
environment env (or host env), or a Claude Code subscription login
available to the execution target
## Configuration Fields
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `cwd` | string | Yes | Working directory for the agent process (absolute path; created automatically if missing when permissions allow) |
| `model` | string | No | Claude model to use (default: `claude-opus-5`) |
| `promptTemplate` | string | No | Prompt used for all runs |
| `env` | object | No | Environment variables (supports secret refs) |
| `timeoutSec` | number | No | Process timeout (0 = no timeout) |
| `graceSec` | number | No | Grace period before force-kill |
| `maxTurnsPerRun` | number | No | Max agentic turns per heartbeat (defaults to `300`) |
| `dangerouslySkipPermissions` | boolean | No | Skip permission prompts (default: `true`); required for headless runs where interactive approval is impossible |
On the CLI engine, version and `--effort` capability probes have a bounded
20-second budget when `timeoutSec` is zero or unset. A positive run timeout below
20 seconds also limits these probes, with a minimum probe budget of one second.
## Default model
An omitted, empty, or whitespace-only `model` uses Claude Opus 5
(`claude-opus-5`) on both the CLI and ACP engines. This also applies to existing
agents with an unset model, including agents created through the API and agents
running in sandboxes. No database migration is needed. The editor shows the
Paperclip default and leaves the setting unset until you select a model.
An explicit `model` takes precedence over `ANTHROPIC_MODEL`. When only
`ANTHROPIC_MODEL` is configured, the adapter keeps that override. Bedrock and
Vertex configurations without an explicit model keep their provider-specific
default because those providers use different model IDs. Host environment
settings apply only to local targets when resolving the model.
The default does not change explicitly configured agent models or the separate
Paperclip Runner's qualified provider profiles.
## Prompt Templates
Templates support `{{variable}}` substitution:
| Variable | Value |
|----------|-------|
| `{{agentId}}` | Agent's ID |
| `{{companyId}}` | Company ID |
| `{{runId}}` | Current run ID |
| `{{agent.name}}` | Agent's name |
| `{{company.name}}` | Company name |
## Session Persistence
The adapter persists Claude Code session IDs between heartbeats. On the next wake, it resumes the existing conversation so the agent retains full context.
Session resume is cwd-aware: if the agent's working directory changed since the last run, a fresh session starts instead.
If resume fails with an unknown session error, the adapter automatically retries with a fresh session.
### Poisoned `previous_message_id` (recovery)
Symptom in logs / issue thread:
```
API Error: 400 diagnostics.previous_message_id: must be the `id` from a prior /v1/messages response (starts with `msg_`)
```
What it means: the on-disk Claude Code transcript JSONL for that session contains a malformed (non-`msg_`-prefixed) `previous_message_id`. Anthropic's `/v1/messages` rejects every resume attempt against that transcript with a deterministic 400. Without guards, Paperclip would re-persist the same poisoned session id and the issue is stranded permanently — see [RED-976](../../../) / [RED-978](../../../).
What the adapter does automatically:
1. **Auto-rotate on resume.** If a `--resume` attempt returns this 400, the adapter retries once with a fresh session, deletes the poisoned `<session>.jsonl` from the local Claude config dir (best effort), and uses the fresh session id going forward.
2. **Validate-before-persist.** A result that carries this 400 never gets its `session_id` written back to the task session store, even if Claude Code emits one in the result event. The adapter returns `sessionId: null`, `sessionParams: null`, and `errorCode: "claude_poisoned_previous_message_id"`.
3. **Clear-on-error.** The adapter sets `clearSession: true` on the result, which causes the heartbeat service to drop any persisted session row for that issue (`clearTaskSessions`). The next continuation starts from a clean slate.
On-call checklist if you see this in production:
- Confirm `errorCode` is `claude_poisoned_previous_message_id` in the run row — that means the guards fired correctly and the issue auto-recovers on the next heartbeat.
- If the same issue still loops after one heartbeat, check that `agentTaskSessions` for that `(agentId, taskKey)` was cleared. If not, the adapter return value was lost (e.g. a malformed run finalization) — escalate; do **not** manually edit the row, file a child issue with the run id.
- For remote execution targets (sandbox/SSH), the poisoned JSONL is on the remote and the adapter only logs the cleanup intent. The fresh-session retry still succeeds because it uses a new session id, and the server-side `clearSession: true` is authoritative regardless of remote disk state.
## Skills Injection
The adapter creates a temporary directory with symlinks to Paperclip skills and passes it via `--add-dir`. This makes skills discoverable without polluting the agent's working directory.
## Remote credential ownership
When no API key or `CLAUDE_CODE_OAUTH_TOKEN` is configured,
`claude_local` uses a snapshot-owns-auth topology for managed sandbox execution
targets. When the run uses a sandbox execution target and no explicit
`CLAUDE_CONFIG_DIR` is configured, Paperclip creates a remote
`CLAUDE_CONFIG_DIR` under the run's Claude runtime directory. It uploads
sanitized host-side settings such as `settings.json` and `CLAUDE.md`, but the
managed seed does not upload host Claude credential files.
After the seed is copied, the remote materialization command checks the
execution target's own `$HOME/.claude` directory. For each missing credential
file, it copies `.credentials.json` or `credentials.json` from that remote home
into the managed `CLAUDE_CONFIG_DIR`. That means credentials baked into the
sandbox image win for managed remote Claude runs.
Worked example: a sandbox image contains `$HOME/.claude/.credentials.json` from
its own Claude Code login. Paperclip starts a managed remote `claude_local` run,
uploads only the sanitized config seed, and sets `CLAUDE_CONFIG_DIR` to the
remote runtime config path. Because the managed config has no credential file,
the adapter copies the sandbox image's `$HOME/.claude/.credentials.json` into
that path before invoking Claude. The sandbox snapshot owns the credential for
the run.
This differs from [`codex_local`](/adapters/codex-local), where a
Paperclip-managed sandbox run uploads a host-owned `CODEX_HOME/auth.json` and
therefore shadows any Codex login already present inside the sandbox image.
## Sandbox memory limits
A sandbox has a fixed memory limit. When a tool command (for example a type
check or a dependency install) uses most of that memory, the kernel kills one
process. Without help it can kill the Claude Code process, and the run fails
with "The connection to Claude was lost."
For managed sandbox runs, the same materialization step installs a
`tool-shell-prefix.sh` script in the run's Claude runtime directory and sets
`CLAUDE_CODE_SHELL_PREFIX` to it. Claude Code runs each Bash tool command and
hook through that script. The script raises the command's `oom_score_adj` to
1000 and then runs the command in the shell that Claude Code selects. The
kernel then kills the tool command first, and Claude sees a failed command
(exit 137) that it can recover from. Paperclip sets the variable only after the
sandbox confirms the install. An operator `CLAUDE_CODE_SHELL_PREFIX` in the
adapter env is kept, and no script is installed.
For manual local CLI usage outside heartbeat runs (for example running as `claudecoder` directly), use:
```sh
npx paperclipai agent local-cli claudecoder --company-id <company-id>
```
This installs Paperclip skills in `~/.claude/skills`, creates an agent API key, and prints shell exports to run as that agent.
## Environment Test
Use the "Test Environment" button in the UI to validate the adapter config. It checks:
- Claude CLI is installed and accessible
- Working directory is absolute and available (auto-created if missing and permitted)
- API key/auth mode hints (`ANTHROPIC_API_KEY` vs `CLAUDE_CODE_OAUTH_TOKEN` vs subscription login)
- A live hello probe (`claude --print - --output-format stream-json --verbose` with prompt `Respond with hello.`) to verify CLI readiness
The probe sees the same layered env as a real run: when an environment is
selected, its environment variables (secret refs included) are resolved and
merged under the adapter config's `env`, so environment-level auth is
reflected in the test result. A secret binding that is missing surfaces as
an `environment_env_binding_missing` failure instead of a silently passing
probe.