Files
PaperClipAI/packages/adapter-utils
Priya RamanandPaperclip 0e3a7ed740 fix(adapter-utils): stream the file gateway body and stop the 15x copy amplification
The file gateway charged reserveBodyRead(declaredBytes * 15) and held the
whole request as four copies at once: the raw buffer, the decoded string,
the JSON.stringify string, and the UTF-8 write buffer. At the production
body cap, that charge alone (about 150 MiB) exceeded the whole 64 MiB
inflight bound, so a body at the cap always got a 503 on an idle gateway.
An unknown-length request (no Content-Length header, for example a chunked
request) hit the same charge, because declaredBodyBytes fell back to the
body cap for it.

Stream the JSON request file to disk instead of building it in memory
first. Write the small envelope fields with JSON.stringify, then stream the
decoded body through a bounded escape-and-write loop, one fixed-size slice
at a time, then the closing fields. This removes the whole-escaped-string
copy and the whole write-buffer copy, so the reserve factor drops from 15
to 4 and covers only the raw buffer and the decoded string. A separate
fixed term, FILE_GATEWAY_BODY_RESERVE_EXTRA_BYTES, covers the one bounded
escape slice the write loop holds at a time.

Grow the body read for an unknown-length request instead of preallocating
the body cap: accumulate chunks in an array and join them once at the end,
capped at the declared body cap, so a 200-byte chunked request no longer
allocates 10 MiB. readBody now takes the declared length as null instead of
a sentinel value, so a caller can tell a real declared length from an
unknown one.

The duplex gateway keeps its existing charge (the declared length plus one
frame's worth of headroom); only the null-based sentinel changed, not the
charge value.

Verified: a body of 10,485,760 bytes and a request with no Content-Length
header both now reach the host worker instead of getting a 503, at the
production body cap and inflight bound. Both new tests fail with 503
against the prior commit and pass against this change. All 43 tests in
sandbox-callback-bridge.test.ts, 19 in duplex-bridge-local-e2e.test.ts, 136
in execution-target-sandbox.test.ts, and 2 in execution-target-ssh-buffer.test.ts
pass. npx tsc --noEmit on packages/adapter-utils is clean.

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-23 20:58:27 +00:00
..

@paperclipai/adapter-utils

Shared utilities for Paperclip adapters: process spawning, environment injection, sandbox/SSH transport, workspace sync, and the round-trip helpers that move code between the local execution-workspace cwd and wherever the agent actually runs.

For the adapter-author guide see docs/adapters/creating-an-adapter.md and the in-repo notes at packages/adapters/AUTHORING.md.

No-remote-git contract

The local execution-workspace cwd is the only persistence boundary across runs. No adapter may depend on a git remote for cross-run state.

Adapters that run the agent on a different host should use the SSH round-trip helpers in src/ssh.ts:

  • prepareWorkspaceForSshExecution({ spec, localDir, remoteDir }) — bundles the local cwd (tracked files, dirty edits, untracked additions, and the git history needed to reconstruct it) to remoteDir before the run starts. Runs with no git remote configured.
  • restoreWorkspaceFromSshExecution({ spec, localDir, remoteDir, ... }) — syncs the remote cwd back into localDir after the run, including any new commits the agent created. Also runs with no git remote configured.

prepareRemoteManagedRuntime in src/remote-managed-runtime.ts wraps both calls for adapters that want a per-run remote workspace and an automatic restoreWorkspace() finally hook.

The invariant is pinned by the no-remote-git contract case in src/ssh-fixture.test.ts, which asserts that a remote-only commit propagates to the local worktree through the prepare → restore round-trip with no git remote configured at any point. Do not regress that test.