Files
PaperClipAI/docs/adapters/claude-local.md
T
ff7e7b4963 fix(claude-local): make sandbox tool commands the first OOM victims (#16844)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Hosted `claude_local` runs execute in a sandbox. The sandbox has a
fixed memory limit (4 GiB in the observed sandboxes).
> - In that sandbox, every process has the same `oom_score_adj`. When a
Bash tool command uses most of the memory, the kernel kills the largest
process. That can be the Claude Code process.
> - When Claude Code is killed, `claude-agent-acp` reports "The
connection to Claude was lost." and the whole run fails. Server logs
show `Claude Agent process died: Claude Code process terminated by
signal SIGKILL` for each failure.
> - A failed tool command is recoverable. A dead agent is not.
> - This pull request makes Claude's tool commands the first
out-of-memory victims in managed sandboxes.
> - The benefit is that a memory-heavy command fails with exit 137, and
the agent continues the run.

## Linked Issues or Issue Description

Refs #16366 (diagnostic PR for the same failure). Related: #14509
protects the host control plane from local agent processes. That PR does
not cover a sandboxed agent and its own tool commands.

**What happened?**

Hosted `claude_local` runs fail with `acpx_turn_failed` and "The
connection to Claude was lost." while one `execute` (Bash) tool call is
pending. The pending commands are memory-heavy, for example `tsc
--noEmit` with a 3.3 GB heap, a monorepo `pnpm install`, or a type check
plus `vitest`. At the same second, the server log shows `Claude Agent
process died: Claude Code process terminated by signal SIGKILL`. Over 10
days, all 12 such lines on one staging stack were `SIGKILL`, and they
match the failed runs. The sandbox memory cgroup has `memory.max` 4 GiB
and `memory.oom.group` 0, so the kernel kills one process. Every process
has `oom_score_adj` 0, so the kernel picks the largest one.

**Expected behavior**

Under memory pressure, the kernel kills the tool command. Claude gets a
failed command (exit 137) and continues the run.

**Steps to reproduce**

1. Start a `claude_local` run on a sandbox target with a 4 GiB memory
limit.
2. Ask the agent to run a memory-heavy command in a large repository,
for example a full `tsc --noEmit` and a test run together.
3. The kernel kills the Claude Code process, and the run fails with "The
connection to Claude was lost."

**Agent adapter(s) involved**

Claude Code (`claude_local`, ACP lane and CLI lane, managed sandbox
targets).

## What Changed

- `claude-config.ts`: add `CLAUDE_TOOL_SHELL_PREFIX_SCRIPT`. Claude Code
calls a `CLAUDE_CODE_SHELL_PREFIX` with one argument, the full command
string. The script raises its own `oom_score_adj` to 1000 (no privilege
is necessary to raise it). Then it runs the command string in the shell
that Claude Code selects: `CLAUDE_CODE_SHELL`, then `SHELL`, when either
is an executable bash or zsh, else zsh or bash from `PATH` (bash first
when `SHELL` names bash).
- `buildRemoteClaudeConfigMaterializationCommand` can install the script
(umask 077, write to `.partial`, `chmod 700`, `mv`). The install runs
last in the existing materialization command, so it adds no sandbox
round trip. It prints a marker only when the file is in place.
- `materializeRemoteClaudeConfig` returns `toolShellPrefixInstalled`. It
is true only when the command exits 0 and prints the marker.
- ACP lane (`acp.ts`) and CLI lane (`execute.ts`): set
`CLAUDE_CODE_SHELL_PREFIX` to the installed script only when the install
is confirmed. A prefix path that does not exist would fail every Bash
tool call, so a failed install keeps the previous behavior. An operator
`CLAUDE_CODE_SHELL_PREFIX` in the adapter env is kept, and no script is
installed.
- `docs/adapters/claude-local.md`: add "Sandbox memory limits".

## Verification

- `npx vitest run src/server/claude-config.test.ts` in
`packages/adapters/claude-local`: 10 pass, 6 new. The new tests install
the script with the real materialization command through `/bin/sh` and
check:
- the file content, mode 0700, the marker, and that no `.partial` file
remains;
- the command runs with `oom_score_adj` 1000 while the caller keeps its
own score (Linux only);
- shell selection: a valid bash `SHELL`, an invalid `CLAUDE_CODE_SHELL`
that falls back to `SHELL`, and a `SHELL` of `/bin/sh` that falls back
to bash from `PATH`;
  - quoting and the exit status of the command are kept;
  - an operator prefix wins, and no marker means no install.
- `acp.test.ts`: the managed-config test now checks that
`CLAUDE_CODE_SHELL_PREFIX` names an executable 0700 file next to the
config dir. A new test checks that an operator prefix is kept and no
script is installed. Both use the local sandbox runner, which runs the
setup commands for real.
- `tsc --noEmit -p packages/adapters/claude-local` is clean.
- Two tests fail in my sandbox both with and without this change: `uses
the same default model in ACP startup and session identity` and
`reselects the full assignment and bootstrap guidance after a failed
resume`. They also fail on clean `master` in the same environment, so I
think they depend on that environment. CI covers them.
- Real Claude Code check (Claude Code 2.1.286): with
`CLAUDE_CODE_SHELL_PREFIX` set to the script, a Bash tool call printed
`oom_score_adj` 1000 and ran in zsh (the selected shell), and its parent
was the `claude` process at score 0. Without a prefix file that exists,
every Bash tool call fails with exit 127. This is why the env is set
only after a confirmed install.
- Real OOM check in a 4 GiB Daytona sandbox (`memory.oom.group` 0). A
stand-in "agent" process held 1.2 GB at score 0, and a tool command
started 14 processes of 250 MB each:
- without the prefix, the kernel killed the stand-in agent (SIGKILL,
137; `oom_kill` +1);
- through the installed prefix, the stand-in agent stayed alive, and the
kernel killed 5 tool processes (`oom_kill` +5).
- Assumption that could invalidate the design: Claude Code passes one
argument (the command string) to the prefix and honors the variable
through `claude-agent-acp`. I checked the argument shape with a logging
prefix, and `claude-agent-acp` 0.73.0 spawns the CLI with `{
...process.env }`.

## Risks

- Every Bash tool command and hook in a managed sandbox now runs through
one more `sh` process and an `exec`. The cost is a few milliseconds for
each command.
- If a future Claude Code release changes how it calls
`CLAUDE_CODE_SHELL_PREFIX`, Bash tool calls can fail. The tests above
and the documented variable limit this risk. Removing the variable
restores the old behavior.
- The inner shell is not a login shell. Claude Code sources its own
shell snapshot inside the command string, so the environment stays the
same. If Claude Code runs without a snapshot, it uses a login shell.
Only exported variables then pass through to the inner shell.
- If the selected shell differs from the one that Claude Code chose (for
example, a custom `CLAUDE_CODE_SHELL` that is not bash or zsh), the
script uses the same fallback order as Claude Code.
- This does not help when the Claude Code process itself uses most of
the memory. The local (non-sandbox) lanes do not change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with tool use
and code execution. It read the repository and the Claude Code CLI
behavior, ran the OOM experiment, edited the code, and ran the tests.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass (two tests fail on clean
`master` in my sandbox too; see Verification)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-04 12:09:54 -08:00

9.7 KiB

title, summary
title summary
Claude Code Claude Code local adapter setup and configuration

The claude_local adapter runs Anthropic's Claude Code CLI locally. It supports session persistence, skills injection, and structured output parsing.

Quota waits

Claude ACP runs that end with a typed provider-quota error retain the quota classification and any parsed reset time. Recovery waits until that time, or uses its existing one-hour quota backoff when no reset time is available. This includes the Claude bridge's typed “The Claude account has no available quota.” fallback, which carries no reset timestamp. The adapter inspects the terminal provider message in memory; the run result and run log retain only the generic failure message, recovery labels, and reset timestamp. Context, turn, rate, and configured budget limits are not treated as subscription quota exhaustion merely because ACP labels them limit.

Prerequisites

  • Claude Code CLI installed (claude command available)
  • Either ANTHROPIC_API_KEY or CLAUDE_CODE_OAUTH_TOKEN in adapter or environment env (or host env), or a Claude Code subscription login available to the execution target

Configuration Fields

Field Type Required Description
cwd string Yes Working directory for the agent process (absolute path; created automatically if missing when permissions allow)
model string No Claude model to use (default: claude-opus-5)
promptTemplate string No Prompt used for all runs
env object No Environment variables (supports secret refs)
timeoutSec number No Process timeout (0 = no timeout)
graceSec number No Grace period before force-kill
maxTurnsPerRun number No Max agentic turns per heartbeat (defaults to 300)
dangerouslySkipPermissions boolean No Skip permission prompts (default: true); required for headless runs where interactive approval is impossible

On the CLI engine, version and --effort capability probes have a bounded 20-second budget when timeoutSec is zero or unset. A positive run timeout below 20 seconds also limits these probes, with a minimum probe budget of one second.

Default model

An omitted, empty, or whitespace-only model uses Claude Opus 5 (claude-opus-5) on both the CLI and ACP engines. This also applies to existing agents with an unset model, including agents created through the API and agents running in sandboxes. No database migration is needed. The editor shows the Paperclip default and leaves the setting unset until you select a model.

An explicit model takes precedence over ANTHROPIC_MODEL. When only ANTHROPIC_MODEL is configured, the adapter keeps that override. Bedrock and Vertex configurations without an explicit model keep their provider-specific default because those providers use different model IDs. Host environment settings apply only to local targets when resolving the model.

The default does not change explicitly configured agent models or the separate Paperclip Runner's qualified provider profiles.

Prompt Templates

Templates support {{variable}} substitution:

Variable Value
{{agentId}} Agent's ID
{{companyId}} Company ID
{{runId}} Current run ID
{{agent.name}} Agent's name
{{company.name}} Company name

Session Persistence

The adapter persists Claude Code session IDs between heartbeats. On the next wake, it resumes the existing conversation so the agent retains full context.

Session resume is cwd-aware: if the agent's working directory changed since the last run, a fresh session starts instead.

If resume fails with an unknown session error, the adapter automatically retries with a fresh session.

Poisoned previous_message_id (recovery)

Symptom in logs / issue thread:

API Error: 400 diagnostics.previous_message_id: must be the `id` from a prior /v1/messages response (starts with `msg_`)

What it means: the on-disk Claude Code transcript JSONL for that session contains a malformed (non-msg_-prefixed) previous_message_id. Anthropic's /v1/messages rejects every resume attempt against that transcript with a deterministic 400. Without guards, Paperclip would re-persist the same poisoned session id and the issue is stranded permanently — see RED-976 / RED-978.

What the adapter does automatically:

  1. Auto-rotate on resume. If a --resume attempt returns this 400, the adapter retries once with a fresh session, deletes the poisoned <session>.jsonl from the local Claude config dir (best effort), and uses the fresh session id going forward.
  2. Validate-before-persist. A result that carries this 400 never gets its session_id written back to the task session store, even if Claude Code emits one in the result event. The adapter returns sessionId: null, sessionParams: null, and errorCode: "claude_poisoned_previous_message_id".
  3. Clear-on-error. The adapter sets clearSession: true on the result, which causes the heartbeat service to drop any persisted session row for that issue (clearTaskSessions). The next continuation starts from a clean slate.

On-call checklist if you see this in production:

  • Confirm errorCode is claude_poisoned_previous_message_id in the run row — that means the guards fired correctly and the issue auto-recovers on the next heartbeat.
  • If the same issue still loops after one heartbeat, check that agentTaskSessions for that (agentId, taskKey) was cleared. If not, the adapter return value was lost (e.g. a malformed run finalization) — escalate; do not manually edit the row, file a child issue with the run id.
  • For remote execution targets (sandbox/SSH), the poisoned JSONL is on the remote and the adapter only logs the cleanup intent. The fresh-session retry still succeeds because it uses a new session id, and the server-side clearSession: true is authoritative regardless of remote disk state.

Skills Injection

The adapter creates a temporary directory with symlinks to Paperclip skills and passes it via --add-dir. This makes skills discoverable without polluting the agent's working directory.

Remote credential ownership

When no API key or CLAUDE_CODE_OAUTH_TOKEN is configured, claude_local uses a snapshot-owns-auth topology for managed sandbox execution targets. When the run uses a sandbox execution target and no explicit CLAUDE_CONFIG_DIR is configured, Paperclip creates a remote CLAUDE_CONFIG_DIR under the run's Claude runtime directory. It uploads sanitized host-side settings such as settings.json and CLAUDE.md, but the managed seed does not upload host Claude credential files.

After the seed is copied, the remote materialization command checks the execution target's own $HOME/.claude directory. For each missing credential file, it copies .credentials.json or credentials.json from that remote home into the managed CLAUDE_CONFIG_DIR. That means credentials baked into the sandbox image win for managed remote Claude runs.

Worked example: a sandbox image contains $HOME/.claude/.credentials.json from its own Claude Code login. Paperclip starts a managed remote claude_local run, uploads only the sanitized config seed, and sets CLAUDE_CONFIG_DIR to the remote runtime config path. Because the managed config has no credential file, the adapter copies the sandbox image's $HOME/.claude/.credentials.json into that path before invoking Claude. The sandbox snapshot owns the credential for the run.

This differs from codex_local, where a Paperclip-managed sandbox run uploads a host-owned CODEX_HOME/auth.json and therefore shadows any Codex login already present inside the sandbox image.

Sandbox memory limits

A sandbox has a fixed memory limit. When a tool command (for example a type check or a dependency install) uses most of that memory, the kernel kills one process. Without help it can kill the Claude Code process, and the run fails with "The connection to Claude was lost."

For managed sandbox runs, the same materialization step installs a tool-shell-prefix.sh script in the run's Claude runtime directory and sets CLAUDE_CODE_SHELL_PREFIX to it. Claude Code runs each Bash tool command and hook through that script. The script raises the command's oom_score_adj to 1000 and then runs the command in the shell that Claude Code selects. The kernel then kills the tool command first, and Claude sees a failed command (exit 137) that it can recover from. Paperclip sets the variable only after the sandbox confirms the install. An operator CLAUDE_CODE_SHELL_PREFIX in the adapter env is kept, and no script is installed.

For manual local CLI usage outside heartbeat runs (for example running as claudecoder directly), use:

npx paperclipai agent local-cli claudecoder --company-id <company-id>

This installs Paperclip skills in ~/.claude/skills, creates an agent API key, and prints shell exports to run as that agent.

Environment Test

Use the "Test Environment" button in the UI to validate the adapter config. It checks:

  • Claude CLI is installed and accessible
  • Working directory is absolute and available (auto-created if missing and permitted)
  • API key/auth mode hints (ANTHROPIC_API_KEY vs CLAUDE_CODE_OAUTH_TOKEN vs subscription login)
  • A live hello probe (claude --print - --output-format stream-json --verbose with prompt Respond with hello.) to verify CLI readiness

The probe sees the same layered env as a real run: when an environment is selected, its environment variables (secret refs included) are resolved and merged under the adapter config's env, so environment-level auth is reflected in the test result. A secret binding that is missing surfaces as an environment_env_binding_missing failure instead of a silently passing probe.