mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-06 10:48:12 +02:00
feat(adapter-utils): carry binary bodies and attachment routes over the HTTP/2 sandbox bridge (#12923)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip runs agents in local and remote sandboxes through adapter utilities > - The HTTP/2 sandbox bridge decoded every body as UTF-8 text and rejected non-JSON content > - This stopped agents from uploading or downloading issue attachments through that bridge > - This pull request carries raw bytes, permits the two attachment routes, and enforces a shared body limit > - The benefit is correct attachment transfer with a process-wide memory guard ## Linked Issues or Issue Description **What existing behavior does this improve?** The HTTP/2 sandbox bridge forwards request bodies between an agent sandbox and the Paperclip host. It now supports binary bodies and the issue attachment routes. **Current behavior** The bridge decodes each body as UTF-8 text. It returns HTTP 415 for content types outside the JSON route list. An agent cannot upload or download an issue attachment through this transport. **Proposed behavior** The bridge carries raw bytes through the forward path. It permits the attachment upload and content routes. The queue transport and file gateway keep their existing route behavior. A shared 10 MiB body limit and process-wide byte reservation protect memory use. **Reason and benefit** Attachment clients need byte-preserving transfer. The shared limit keeps the gateway and host aligned. The reservation prevents concurrent streams from exceeding the accepted process memory ceiling. **Breaking changes** The HTTP/2 bridge accepts two attachment routes and permits binary content. The queue transport and file gateway keep their previous route lists and HTTP 415 behavior. No schema or external endpoint changes. ## What Changed - Carry request and response bodies as raw bytes through the HTTP/2 bridge. - Permit attachment upload and attachment content routes on the HTTP/2 bridge only. - Raise the resolved per-body limit to 10 MiB and share it between the gateway and host. - Reserve body bytes before allocation and release each stream reservation on every terminal path. - Document the body limit, process ceiling, and reservation behavior. ## Verification - Run `pnpm exec vitest run packages/adapter-utils/src/http2-bridge-server.test.ts packages/adapter-utils/src/execution-target-sandbox.test.ts packages/adapter-utils/src/sandbox-callback-bridge.test.ts`; 226 tests pass. - Run `pnpm --filter @paperclipai/adapter-utils typecheck`; it passes. - Run the direct server TypeScript check with `tsc --noEmit` in `server/`; it passes with zero errors. - Verify multipart upload and binary download round trips over HTTP/2 without corruption. - Verify the queue transport and file gateway return HTTP 415 for the same routes. - Verify the host rejects bodies over the resolved limit. - Verify a denied reservation returns HTTP 503 and allocates no copy. - Verify stream cleanup releases reservations after completion, error, abort, timeout, and close. ## Risks The bridge now accepts larger bodies and binary content. The process-wide reservation limits total live body bytes to 1 GiB. Route behavior changes only for the HTTP/2 bridge. The security review found no blocking issue for this commit range. ## Model Used OpenAI Codex, GPT-5. The runtime used tool calls and code execution. The runtime did not expose the context window size. No model-generated code changes were made for this pull request. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
e095b84dab
commit
6019e2bd6e
8 files changed
+2696
-137
No files matched your search
+42
-11
@@ -754,18 +754,49 @@ To add a name or an enum value, extend the literal constant in
|
||||
|
||||
### Known behavior: aggregate retained body bytes
|
||||
|
||||
The HTTP/2 bridge bounds retained body bytes for one route only. Each route
|
||||
holds up to 8,388,608 bytes (8 MiB) at its own peak (see
|
||||
`HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS` in `http2-bridge-server.ts`). The host
|
||||
process admits up to 128 concurrent routes (see
|
||||
`DEFAULT_MAX_CONCURRENT_DUPLEX_ROUTES` in `plugin-worker-manager.ts`). The
|
||||
process can therefore retain up to 1,073,741,824 bytes (1 GiB) of body data
|
||||
across every route at the same time.
|
||||
Each HTTP/2 bridge route holds up to 168,820,736 bytes (161 MiB) at its own
|
||||
peak (see `HTTP2_BRIDGE_MAX_CONCURRENT_STREAMS` in `http2-bridge-server.ts`).
|
||||
The host process admits up to 128 concurrent routes (see
|
||||
`DEFAULT_MAX_CONCURRENT_DUPLEX_ROUTES` in `plugin-worker-manager.ts`). Those
|
||||
two figures alone would let the process retain up to 21,609,054,208 bytes
|
||||
(about 20.1 GiB) of body data across every route at the same time.
|
||||
|
||||
The process does not reach that figure, on two levels.
|
||||
`HTTP2_BRIDGE_MAX_PROCESS_BODY_BYTES` (`http2-bridge-server.ts`) enforces a
|
||||
real, live ledger: 1,073,741,824 bytes (1 GiB) across every route, not merely
|
||||
an accepted paper ceiling. Every HTTP/2 stream creates one `BridgeBodyReservation` owner over
|
||||
its lifetime, and every source-level full-body buffer that stream retains —
|
||||
its request-body chunk array, the concatenated request body, the
|
||||
response-body chunk array, and the concatenated response body — reserves
|
||||
against that one owner before it allocates. A reservation that would pass the
|
||||
process total is denied before it copies anything, and the host answers 503
|
||||
instead of accepting the body. The reservation stays live for the response
|
||||
body until the HTTP/2 write actually finishes flowing to the peer or the
|
||||
stream closes, not merely until the write call returns, so a slow or
|
||||
backpressured peer cannot hold response bytes in memory the ledger no longer
|
||||
counts.
|
||||
|
||||
`HTTP2_BRIDGE_MAX_ROUTE_BODY_BYTES` adds a second, per-route ledger on top of
|
||||
that process-wide one: each route's own reservations also check a ceiling
|
||||
scoped to that one route (its own 168,820,736-byte peak from above), so one
|
||||
busy or malicious route can pass its own ceiling and get denied with a 503,
|
||||
but it can never spend the whole process-wide total and deny every sibling
|
||||
route admission. This accounting covers source-level full-body buffers only:
|
||||
internal Node.js and Undici copies (socket buffers, HTTP/2 frame buffers,
|
||||
decompression buffers) stay outside it.
|
||||
|
||||
The generated gateway process inside the sandbox (`getSandboxCallbackBridgeServerSource`
|
||||
in `sandbox-callback-bridge.ts`) enforces its own separate ledger, independent
|
||||
of the two host-side ledgers above: each side bounds only the memory in its
|
||||
own process. `readBodyBytes` reserves a request body's chunk bytes as they
|
||||
arrive, then reserves the concatenated buffer's own byte count before
|
||||
`Buffer.concat` allocates it, against a ceiling of `maxBodyBytes * 8` (4
|
||||
concurrent bodies, each counted twice for its two live copies). A denied
|
||||
reservation answers 503 with no forward call. Each request handler releases
|
||||
its own reservation once the whole request settles: a completed response, a
|
||||
thrown error, a client abort, or a deadline timeout all reach the same
|
||||
release call.
|
||||
|
||||
This is accepted, known behavior. The process tracks no aggregate byte
|
||||
ledger across routes: a per-route bound stops one busy route from starving
|
||||
another route's own budget, but the host enforces no smaller ceiling on the
|
||||
sum across every route.
|
||||
Keep every dimension low-cardinality and free of user content.
|
||||
|
||||
### Shared skill preparation
|
||||
|
||||
Reference in new issue
Block a user