Files
PaperClipAI/packages/adapter-utils/src/duplex-frame-vectors.json
T
Nicky LeachandPaperclip 16b59c9315 feat(adapter-utils): stream duplex bridge bodies as sequenced chunks with receive-side spill (#12006)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent adapters use a duplex bridge to send requests and responses
across an isolated boundary
> - The bridge held each request body and response body in memory on
both ends
> - Large bodies can exhaust memory and reduce the safe size of adapter
traffic
> - This pull request sends receive-side bodies as sequenced chunks and
spills large bodies to disk
> - The benefit is bounded memory use with strict size, order, and
cleanup checks

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The adapter-utils duplex bridge transports request and response bodies
across the sandbox boundary.

**Current behavior**

The bridge stores each complete body in memory on both ends of the
duplex channel.

**Proposed behavior**

The bridge sends body chunks with sequence checks. The receive side
keeps bodies up to 1 MiB in memory and spills larger bodies to a
temporary file.

**Reason and benefit**

This change reduces memory pressure and keeps malformed or oversized
input on a terminal error path.

**Breaking changes**

The duplex frame version changes to version 2. The request and response
envelopes now carry bodyByteCount, and body_chunk frames carry the body
data.

## What Changed

- Add version 2 body_chunk frames with 256 KiB raw slices encoded as
canonical base64 text.
- Add receive-side memory and spill reassembly with per-channel disk and
file limits.
- Reject malformed, reordered, oversized, truncated, and non-canonical
body chunks.
- Stream reassembled request bodies to the host forward handler with a
web stream and half-duplex request.
- Remove spill files on success, failure, channel death, and startup
cleanup.

## Verification

- Run the adapter-utils type-check.
- Run the adapter-utils duplex test suite.
- Run all pull request checks.
- Run the Greptile review and confirm a 5/5 score with no open findings.

## Risks

The wire format changes from version 1 to version 2. Older bridge peers
cannot use this protocol. The receive path adds temporary file
operations and cleanup paths. The implementation fails closed when a
body violates size or sequence rules.

## Model Used

OpenAI GPT-5 Codex, tool-enabled coding agent. The exact context-window
size and reasoning mode are not exposed by the runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with Fixes: # / Closes #
/ Refs # OR (b) described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub #NNN / github.com/paperclipai/paperclip URLs)
- [x] My branch name describes the change (e.g. docs/..., fix/...) and
contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-08-23 08:52:31 -07:00

632 lines
20 KiB
JSON

{
"frameVersion": 2,
"defaultMaxFrameBytes": 1000000,
"description": "Shared wire-compatibility vectors for the duplex frame codec. Every codec copy decodes the same bytes. bytes is a UTF-8 byte stream; splitByteOffsets cuts it into chunks; expected lists the ordered decode results (frame or protocol-error code).",
"vectors": [
{
"name": "valid-request",
"category": "valid",
"bytes": "{\"version\":2,\"type\":\"request\",\"id\":\"req-1\",\"method\":\"GET\",\"path\":\"/api/issues/PAP-1\",\"query\":\"expand=comments\",\"headers\":{\"accept\":\"application/json\",\"content-type\":\"application/json\"},\"bodyByteCount\":0}\n",
"roundTrip": true,
"expected": [
{
"frame": {
"version": 2,
"type": "request",
"id": "req-1",
"method": "GET",
"path": "/api/issues/PAP-1",
"query": "expand=comments",
"headers": {
"accept": "application/json",
"content-type": "application/json"
},
"bodyByteCount": 0
}
}
]
},
{
"name": "valid-request-with-body-count",
"category": "valid",
"bytes": "{\"version\":2,\"type\":\"request\",\"id\":\"req-body\",\"method\":\"GET\",\"path\":\"/api/issues/PAP-1\",\"query\":\"expand=comments\",\"headers\":{\"accept\":\"application/json\",\"content-type\":\"application/json\"},\"bodyByteCount\":1048576}\n",
"roundTrip": true,
"expected": [
{
"frame": {
"version": 2,
"type": "request",
"id": "req-body",
"method": "GET",
"path": "/api/issues/PAP-1",
"query": "expand=comments",
"headers": {
"accept": "application/json",
"content-type": "application/json"
},
"bodyByteCount": 1048576
}
}
]
},
{
"name": "valid-response-completed",
"category": "valid",
"bytes": "{\"version\":2,\"type\":\"response\",\"id\":\"req-1\",\"status\":200,\"headers\":{\"content-type\":\"application/json\"},\"bodyByteCount\":11,\"outcome\":\"completed\"}\n",
"roundTrip": true,
"expected": [
{
"frame": {
"version": 2,
"type": "response",
"id": "req-1",
"status": 200,
"headers": {
"content-type": "application/json"
},
"bodyByteCount": 11,
"outcome": "completed"
}
}
]
},
{
"name": "valid-response-indeterminate",
"category": "valid",
"bytes": "{\"version\":2,\"type\":\"response\",\"id\":\"req-2\",\"status\":409,\"headers\":{\"content-type\":\"application/json\"},\"bodyByteCount\":33,\"outcome\":\"indeterminate\"}\n",
"roundTrip": true,
"expected": [
{
"frame": {
"version": 2,
"type": "response",
"id": "req-2",
"status": 409,
"headers": {
"content-type": "application/json"
},
"bodyByteCount": 33,
"outcome": "indeterminate"
}
}
]
},
{
"name": "valid-response-unavailable",
"category": "valid",
"bytes": "{\"version\":2,\"type\":\"response\",\"id\":\"req-3\",\"status\":503,\"headers\":{\"content-type\":\"application/json\"},\"bodyByteCount\":30,\"outcome\":\"unavailable\"}\n",
"roundTrip": true,
"expected": [
{
"frame": {
"version": 2,
"type": "response",
"id": "req-3",
"status": 503,
"headers": {
"content-type": "application/json"
},
"bodyByteCount": 30,
"outcome": "unavailable"
}
}
]
},
{
"name": "valid-body-chunk",
"category": "valid",
"bytes": "{\"version\":2,\"type\":\"body_chunk\",\"id\":\"req-1\",\"seq\":0,\"data\":\"aGVsbG8gYm9keSBjaHVuaw==\"}\n",
"roundTrip": true,
"expected": [
{
"frame": {
"version": 2,
"type": "body_chunk",
"id": "req-1",
"seq": 0,
"data": "aGVsbG8gYm9keSBjaHVuaw=="
}
}
]
},
{
"name": "valid-body-chunk-final",
"category": "valid",
"bytes": "{\"version\":2,\"type\":\"body_chunk\",\"id\":\"req-1\",\"seq\":7,\"data\":\"dGFpbA==\"}\n",
"roundTrip": true,
"expected": [
{
"frame": {
"version": 2,
"type": "body_chunk",
"id": "req-1",
"seq": 7,
"data": "dGFpbA=="
}
}
]
},
{
"name": "valid-ready",
"category": "valid",
"bytes": "{\"version\":2,\"type\":\"ready\",\"nonce\":\"9f8e7d6c5b4a39281706f5e4d3c2b1a0\"}\n",
"roundTrip": true,
"expected": [
{
"frame": {
"version": 2,
"type": "ready",
"nonce": "9f8e7d6c5b4a39281706f5e4d3c2b1a0"
}
}
]
},
{
"name": "valid-heartbeat",
"category": "valid",
"bytes": "{\"version\":2,\"type\":\"heartbeat\"}\n",
"roundTrip": true,
"expected": [
{
"frame": {
"version": 2,
"type": "heartbeat"
}
}
]
},
{
"name": "valid-close",
"category": "valid",
"bytes": "{\"version\":2,\"type\":\"close\"}\n",
"roundTrip": true,
"expected": [
{
"frame": {
"version": 2,
"type": "close"
}
}
]
},
{
"name": "valid-error",
"category": "valid",
"bytes": "{\"version\":2,\"type\":\"error\",\"code\":\"malformed_frame\",\"message\":\"peer reported a malformed frame\"}\n",
"roundTrip": true,
"expected": [
{
"frame": {
"version": 2,
"type": "error",
"code": "malformed_frame",
"message": "peer reported a malformed frame"
}
}
]
},
{
"name": "invalid-ready-missing-nonce",
"category": "invalid",
"bytes": "{\"version\":2,\"type\":\"ready\"}\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-ready-with-address",
"category": "invalid",
"bytes": "{\"version\":2,\"type\":\"ready\",\"nonce\":\"9f8e7d6c5b4a39281706f5e4d3c2b1a0\",\"address\":\"http://127.0.0.1:47215\"}\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-ready-with-port",
"category": "invalid",
"bytes": "{\"version\":2,\"type\":\"ready\",\"nonce\":\"9f8e7d6c5b4a39281706f5e4d3c2b1a0\",\"port\":47215}\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-not-json",
"category": "invalid",
"bytes": "{not valid json\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-json-array",
"category": "invalid",
"bytes": "[1,2,3]\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-json-number",
"category": "invalid",
"bytes": "42\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-unknown-type",
"category": "invalid",
"bytes": "{\"version\":2,\"type\":\"teleport\"}\n",
"expected": [
{
"error": "unknown_type"
}
]
},
{
"name": "invalid-missing-id",
"category": "invalid",
"bytes": "{\"version\":2,\"type\":\"request\",\"method\":\"GET\",\"path\":\"/\",\"query\":\"\",\"headers\":{},\"bodyByteCount\":0}\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-request-body-count-string",
"category": "invalid",
"bytes": "{\"version\":2,\"type\":\"request\",\"id\":\"req-1\",\"method\":\"GET\",\"path\":\"/\",\"query\":\"\",\"headers\":{},\"bodyByteCount\":\"10\"}\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-request-body-count-negative",
"category": "invalid",
"bytes": "{\"version\":2,\"type\":\"request\",\"id\":\"req-1\",\"method\":\"GET\",\"path\":\"/\",\"query\":\"\",\"headers\":{},\"bodyByteCount\":-1}\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-request-body-count-fractional",
"category": "invalid",
"bytes": "{\"version\":2,\"type\":\"request\",\"id\":\"req-1\",\"method\":\"GET\",\"path\":\"/\",\"query\":\"\",\"headers\":{},\"bodyByteCount\":1.5}\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-bad-headers",
"category": "invalid",
"bytes": "{\"version\":2,\"type\":\"response\",\"id\":\"req-1\",\"status\":200,\"headers\":{\"accept\":7},\"bodyByteCount\":10,\"outcome\":\"completed\"}\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-body-chunk-negative-seq",
"category": "invalid",
"bytes": "{\"version\":2,\"type\":\"body_chunk\",\"id\":\"req-1\",\"seq\":-1,\"data\":\"eA==\"}\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-body-chunk-fractional-seq",
"category": "invalid",
"bytes": "{\"version\":2,\"type\":\"body_chunk\",\"id\":\"req-1\",\"seq\":2.5,\"data\":\"eA==\"}\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-body-chunk-missing-data",
"category": "invalid",
"bytes": "{\"version\":2,\"type\":\"body_chunk\",\"id\":\"req-1\",\"seq\":0}\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "invalid-body-chunk-nonstring-data",
"category": "invalid",
"bytes": "{\"version\":2,\"type\":\"body_chunk\",\"id\":\"req-1\",\"seq\":0,\"data\":42}\n",
"expected": [
{
"error": "malformed_frame"
}
]
},
{
"name": "partial-request-split",
"category": "partial",
"bytes": "{\"version\":2,\"type\":\"request\",\"id\":\"req-1\",\"method\":\"GET\",\"path\":\"/api/issues/PAP-1\",\"query\":\"expand=comments\",\"headers\":{\"accept\":\"application/json\",\"content-type\":\"application/json\"},\"bodyByteCount\":0}\n",
"splitByteOffsets": [
102
],
"expected": [
{
"frame": {
"version": 2,
"type": "request",
"id": "req-1",
"method": "GET",
"path": "/api/issues/PAP-1",
"query": "expand=comments",
"headers": {
"accept": "application/json",
"content-type": "application/json"
},
"bodyByteCount": 0
}
}
]
},
{
"name": "partial-utf8-split",
"category": "partial",
"bytes": "{\"version\":2,\"type\":\"request\",\"id\":\"req-utf8\",\"method\":\"POST\",\"path\":\"/api/issues/PAP-1\",\"query\":\"expand=comments\",\"headers\":{\"content-type\":\"application/json\",\"x-note\":\"héllo 😀 wörld\"},\"bodyByteCount\":0}\n",
"splitByteOffsets": [
172
],
"expected": [
{
"frame": {
"version": 2,
"type": "request",
"id": "req-utf8",
"method": "POST",
"path": "/api/issues/PAP-1",
"query": "expand=comments",
"headers": {
"content-type": "application/json",
"x-note": "héllo 😀 wörld"
},
"bodyByteCount": 0
}
}
]
},
{
"name": "partial-two-frames",
"category": "partial",
"bytes": "{\"version\":2,\"type\":\"request\",\"id\":\"req-1\",\"method\":\"GET\",\"path\":\"/api/issues/PAP-1\",\"query\":\"expand=comments\",\"headers\":{\"accept\":\"application/json\",\"content-type\":\"application/json\"},\"bodyByteCount\":0}\n{\"version\":2,\"type\":\"response\",\"id\":\"req-1\",\"status\":200,\"headers\":{\"content-type\":\"application/json\"},\"bodyByteCount\":11,\"outcome\":\"completed\"}\n",
"splitByteOffsets": [
7,
174,
344
],
"expected": [
{
"frame": {
"version": 2,
"type": "request",
"id": "req-1",
"method": "GET",
"path": "/api/issues/PAP-1",
"query": "expand=comments",
"headers": {
"accept": "application/json",
"content-type": "application/json"
},
"bodyByteCount": 0
}
},
{
"frame": {
"version": 2,
"type": "response",
"id": "req-1",
"status": 200,
"headers": {
"content-type": "application/json"
},
"bodyByteCount": 11,
"outcome": "completed"
}
}
]
},
{
"name": "partial-no-trailing-newline",
"category": "partial",
"bytes": "{\"version\":2,\"type\":\"heartbeat\"}",
"expected": []
},
{
"name": "oversized-complete-line",
"category": "oversized",
"maxFrameBytes": 64,
"bytes": "{\"version\":2,\"type\":\"request\",\"id\":\"req-1\",\"method\":\"GET\",\"path\":\"/api/issues/PAP-1\",\"query\":\"q=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\",\"headers\":{\"accept\":\"application/json\",\"content-type\":\"application/json\"},\"bodyByteCount\":0}\n",
"expected": [
{
"error": "frame_too_large"
}
]
},
{
"name": "oversized-flood-then-resync",
"category": "oversized",
"maxFrameBytes": 64,
"splitByteOffsets": [
40
],
"bytes": "yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy\n{\"version\":2,\"type\":\"heartbeat\"}\n",
"expected": [
{
"error": "frame_too_large"
},
{
"frame": {
"version": 2,
"type": "heartbeat"
}
}
]
},
{
"name": "version-mismatch-lower",
"category": "versionMismatch",
"bytes": "{\"version\":1,\"type\":\"request\",\"id\":\"req-1\",\"method\":\"GET\",\"path\":\"/api/issues/PAP-1\",\"query\":\"expand=comments\",\"headers\":{\"accept\":\"application/json\",\"content-type\":\"application/json\"},\"bodyByteCount\":0}\n",
"expected": [
{
"error": "version_mismatch"
}
]
},
{
"name": "version-mismatch-higher",
"category": "versionMismatch",
"bytes": "{\"version\":3,\"type\":\"request\",\"id\":\"req-1\",\"method\":\"GET\",\"path\":\"/api/issues/PAP-1\",\"query\":\"expand=comments\",\"headers\":{\"accept\":\"application/json\",\"content-type\":\"application/json\"},\"bodyByteCount\":0}\n",
"expected": [
{
"error": "version_mismatch"
}
]
},
{
"name": "version-mismatch-missing",
"category": "versionMismatch",
"bytes": "{\"type\":\"heartbeat\"}\n",
"expected": [
{
"error": "version_mismatch"
}
]
},
{
"name": "request-id-at-max-bytes",
"category": "idBoundValid",
"bytes": "{\"version\":2,\"type\":\"request\",\"id\":\"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\",\"method\":\"GET\",\"path\":\"/\",\"query\":\"\",\"headers\":{},\"bodyByteCount\":0}\n",
"roundTrip": true,
"expected": [
{
"frame": {
"version": 2,
"type": "request",
"id": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"method": "GET",
"path": "/",
"query": "",
"headers": {},
"bodyByteCount": 0
}
}
]
},
{
"name": "request-id-over-max-bytes",
"category": "idBoundInvalid",
"bytes": "{\"version\":2,\"type\":\"request\",\"id\":\"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\",\"method\":\"GET\",\"path\":\"/\",\"query\":\"\",\"headers\":{},\"bodyByteCount\":0}\n",
"expected": [
{
"error": "id_too_large"
}
]
},
{
"name": "body-chunk-id-over-max-bytes",
"category": "idBoundInvalid",
"bytes": "{\"version\":2,\"type\":\"body_chunk\",\"id\":\"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\",\"seq\":0,\"data\":\"eA==\"}\n",
"expected": [
{
"error": "id_too_large"
}
]
}
],
"encodeDescription": "Encode-bound vectors. Each codec copy runs the size-checked encode against the frame with the maxFrameBytes limit, then asserts the same result. An ok result must decode back to the same frame. This proves both copies enforce the same bound on encode.",
"encodeVectors": [
{
"name": "encode-within-bound",
"maxFrameBytes": 200,
"frame": {
"version": 2,
"type": "request",
"id": "req-enc-1",
"method": "GET",
"path": "/",
"query": "",
"headers": {},
"bodyByteCount": 5
},
"expected": {
"ok": true
}
},
{
"name": "encode-body-chunk-within-bound",
"maxFrameBytes": 200,
"frame": {
"version": 2,
"type": "body_chunk",
"id": "req-enc-3",
"seq": 0,
"data": "Y2h1bmsgcGF5bG9hZA=="
},
"expected": {
"ok": true
}
},
{
"name": "encode-control-frame-never-rejected",
"maxFrameBytes": 200,
"frame": {
"version": 2,
"type": "heartbeat"
},
"expected": {
"ok": true
}
},
{
"name": "encode-over-bound",
"maxFrameBytes": 200,
"frame": {
"version": 2,
"type": "response",
"id": "req-enc-2",
"status": 200,
"headers": {
"content-type": "application/json",
"x-pad": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
},
"bodyByteCount": 0,
"outcome": "completed"
},
"expected": {
"ok": false,
"error": "frame_too_large"
}
}
]
}