mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-10 12:07:09 +02:00
6ce62cac75da162eef112f2fe8a378faaff2eaa6
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nicky LeachandPaperclip
|
802f2af154 |
refactor(adapter-utils): delete the dead duplex body-chunk protocol code (#12186)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The adapter utilities package provides transport code for sandbox agents > - The retired `duplex_v1` broker no longer produces or consumes body-chunk frames > - Dead protocol code remains in the host codec, gateway copy, bridge options, and tests > - This pull request removes that dead code and keeps the READY handshake unchanged > - The benefit is a smaller transport surface with fewer unused paths to maintain ## Linked Issues or Issue Description **What existing behavior does this improve?** This change improves the adapter utilities code that supports sandbox duplex readiness and frame handling. **Subsystem affected** `packages/adapter-utils/` — sandbox transport codecs, execution targets, and callback bridge tests. **Current behavior** The repository keeps body-chunk frame types, validators, a body spool, decoder limits, and tests after the `duplex_v1` broker removal. No live producer or consumer uses this code. **Proposed behavior** Remove the unused body-chunk protocol code and retain the READY handshake, its strict checks, and its size limits. **Reason and benefit** The removal reduces dead code and keeps the host and embedded gateway paths easier to inspect. It adds no new behavior. **Breaking changes** The removed frame types now decode as `unknown_type`. The live readiness gate already ignores those frames. The READY handshake stays byte-for-byte compatible. **Additional context** This cleanup follows [PR #12171](https://github.com/paperclipai/paperclip/pull/12171), which removed the duplex broker. ## What Changed - Remove `duplex-body-spool.ts` and its test. - Remove unused body-chunk frame types, validators, decoder code, vectors, and limits. - Remove the unused `reassembledBody` option and decoder limit environment entry. - Remove the embedded gateway decoder copy and the unused frame type map. - Keep the READY handshake and its existing boundary tests unchanged in behavior. ## Verification - `pnpm -F @paperclip/adapter-utils typecheck` passes. - The duplex frame codec test passes with 30 tests. - The sandbox execution-target test passes with 136 tests. - The sandbox callback bridge test passes with 46 tests. - CI must confirm all required checks after it starts. ## Risks Low risk. The change removes code only. The READY handshake, HTTP/2 body path, and byte-ledger path remain unchanged. ## Model Used OpenAI Codex, GPT-5, tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
16b59c9315 |
feat(adapter-utils): stream duplex bridge bodies as sequenced chunks with receive-side spill (#12006)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent adapters use a duplex bridge to send requests and responses across an isolated boundary > - The bridge held each request body and response body in memory on both ends > - Large bodies can exhaust memory and reduce the safe size of adapter traffic > - This pull request sends receive-side bodies as sequenced chunks and spills large bodies to disk > - The benefit is bounded memory use with strict size, order, and cleanup checks ## Linked Issues or Issue Description **What existing behavior does this improve?** The adapter-utils duplex bridge transports request and response bodies across the sandbox boundary. **Current behavior** The bridge stores each complete body in memory on both ends of the duplex channel. **Proposed behavior** The bridge sends body chunks with sequence checks. The receive side keeps bodies up to 1 MiB in memory and spills larger bodies to a temporary file. **Reason and benefit** This change reduces memory pressure and keeps malformed or oversized input on a terminal error path. **Breaking changes** The duplex frame version changes to version 2. The request and response envelopes now carry bodyByteCount, and body_chunk frames carry the body data. ## What Changed - Add version 2 body_chunk frames with 256 KiB raw slices encoded as canonical base64 text. - Add receive-side memory and spill reassembly with per-channel disk and file limits. - Reject malformed, reordered, oversized, truncated, and non-canonical body chunks. - Stream reassembled request bodies to the host forward handler with a web stream and half-duplex request. - Remove spill files on success, failure, channel death, and startup cleanup. ## Verification - Run the adapter-utils type-check. - Run the adapter-utils duplex test suite. - Run all pull request checks. - Run the Greptile review and confirm a 5/5 score with no open findings. ## Risks The wire format changes from version 1 to version 2. Older bridge peers cannot use this protocol. The receive path adds temporary file operations and cleanup paths. The implementation fails closed when a body violates size or sequence rules. ## Model Used OpenAI GPT-5 Codex, tool-enabled coding agent. The exact context-window size and reasoning mode are not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with Fixes: # / Closes # / Refs # OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub #NNN / github.com/paperclipai/paperclip URLs) - [x] My branch name describes the change (e.g. docs/..., fix/...) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
05b35d4669 |
feat(duplex): bound aggregate duplex route resource consumption with a process-owned byte ledger (#12003)
## Thinking Path > - Paperclip runs AI agents through adapters and sandboxed execution targets. > - Duplex routes retain bytes across route data, broker messages, decoder buffers, and readiness replay. > - Per-route limits bound each route but do not bound the total retained bytes across many routes. > - A process-owned ledger must charge each retained buffer before allocation and release the charge during cleanup. > - This pull request adds the aggregate ledger, connects it to host and sandbox duplex paths, and adds route coverage. > - The benefit is a fail-closed process-wide byte limit that keeps concurrent duplex work within a safe resource budget. ## Linked Issues or Issue Description **Subsystem affected** This change affects packages/adapter-utils and server duplex orchestration. **Problem or motivation** Many routes can each stay below their per-route limits while their combined retained bytes exceed a safe process budget. **Proposed solution** Add a process-owned aggregate byte ledger. Charge route data, broker bytes, decoder buffers, and readiness replay bytes before allocation. Release each charge during cleanup. Use a separate sandbox_process decoder cap for the in-sandbox path. **Alternatives considered** Keep only per-route limits. This does not bound the combined process use. Set a fixed limit at one call site. This misses retained bytes in other duplex paths. **Roadmap alignment** This is a tightly scoped reliability and resource-safety improvement. It does not duplicate a roadmap feature. **Additional context** The aggregate ceiling uses a safe 256 MiB default. An invalid override falls back to that default and reports the rejected value. ## What Changed - Add a process-owned aggregate byte ledger for duplex route resource use. - Charge and release route data, broker forward and response bytes, decoder buffers, and readiness replay bytes. - Bound host-to-worker pending writes and standard input transport bytes. - Add a separate decoder cap for the sandbox_process path. - Make invalid aggregate-ceiling overrides fall back to the safe default without host startup failure. - Add adapter-utils and server tests for charging, release, rejection, cleanup, and many-route aggregate limits. ## Verification - pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit - pnpm --filter @paperclipai/server exec tsc --noEmit - Run the focused adapter-utils duplex ledger and execution-target tests. - Run the server aggregate-ledger route test. - Confirm all required pull request checks pass on this branch. ## Risks The ledger touches several duplex buffer paths. A missed release could reduce later capacity until process restart. The tests cover charge, release, rejection, cleanup, and route aggregation. The change uses a safe default when configuration input is invalid. ## Model Used OpenAI GPT-5 Codex. The runtime model ID and context window are not exposed to this task. The model used tool calls, shell commands, and code review workflow support. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with Fixes/Closes/Refs OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub issue references) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
141b815294 |
fix(adapter-utils): enforce the duplex frame size bound on encode in both codec copies (#11983)
## Thinking Path > - Paperclip uses adapter utilities to move bounded messages between agent processes. > - The duplex frame codec encodes and decodes these messages. > - The decoder rejects frames above the documented byte limit. > - The encoder did not apply the same limit before it sent a frame. > - This mismatch let a sender write a frame that the peer rejected after transmission. > - This pull request applies the same byte limit to both codec copies and keeps the broker channel open. > - The benefit is a local error with stable request telemetry instead of a channel loss. ## Linked Issues or Issue Description No public GitHub issue exists for this change. The problem follows the bug report fields below. **What happened?** The duplex encoder could write a frame larger than `DEFAULT_MAX_DUPLEX_FRAME_BYTES`. The peer decoder then rejected the frame after transmission. In the WebSocket 1009 case, this closed the channel and reported a process exit. **Expected behavior** The encoder should reject an oversized frame before it writes bytes. The gateway should return HTTP 413. The broker should return a bounded terminal response and keep other requests active. **Steps to reproduce** 1. Encode a duplex frame above `DEFAULT_MAX_DUPLEX_FRAME_BYTES`. 2. Send the frame through the gateway or broker. 3. Observe that the old path writes the frame or drops the channel after peer rejection. **Paperclip version or commit** Reproduced from the `master` development line before this change. **Deployment mode** Local dev (`pnpm dev`). ## What Changed - Add `encodeDuplexFrameChecked` to the host and embedded gateway codecs. - Measure encoded JSON bytes without the trailing newline. - Return a typed `frame_too_large` result without throwing. - Return HTTP 413 for oversized gateway requests without writing a frame. - Share one frame bound between broker decode and encode checks. - Return a bounded, non-retryable terminal response for oversized broker responses. - Add encode vectors to the shared wire-compatibility fixture. ## Verification - Run `pnpm --filter @paperclipai/adapter-utils typecheck`. - Run `npx vitest run packages/adapter-utils/src/duplex-frame-codec.test.ts packages/adapter-utils/src/duplex-bridge-broker.test.ts packages/adapter-utils/src/execution-target-sandbox.test.ts`. - Confirm the oversized-response broker test keeps the channel open and serves the other in-flight request. - Confirm the gateway test returns HTTP 413 and keeps the channel open. ## Risks The encoder now rejects oversized frames before transmission. This changes an unsafe write into a typed local error. The broker and gateway keep existing frame limits and affect only oversized frames. ## Model Used OpenAI Codex, GPT-5, tool use and code execution. The model assisted with review and repository operations. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
10d2781a29 |
feat(sandbox): add the duplex bridge broker, gated transport selection, and fixed observability (#11769)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandbox adapters provide controlled execution for untrusted provider environments. > - The sandbox channel needs one persistent duplex transport with strict host control. > - The transport must remain off unless the instance setting and provider capability both allow it. > - The host must detect loss, bound resource use, and expose only safe telemetry. > - This pull request adds the broker, gated selection, kill-switch wiring, fixed observability, and real-process proof. > - The benefit is safer sandbox execution with bounded failure behavior and inspectable transport results. ## Linked Issues or Issue Description No public issue exists for this change. The related pull requests are #11738 and #11750. **Problem or motivation** The sandbox duplex channel needs a host-controlled broker, strict transport gates, bounded provider input, and safe loss telemetry. Without these controls, a provider can cause replay, resource growth, unsafe endpoint selection, or data exposure through telemetry. **Proposed solution** Add a host broker with nested time limits, request limits, one-shot loss, and per-id deduplication. Select duplex transport only when the instance setting and provider capability both equal true. Assign the endpoint and nonce on the host. Reject invalid readiness data and use the file bridge on failure. Add fixed redacted telemetry and a real-process end-to-end test harness. **Alternatives considered** Keep the file bridge as the only transport. This avoids new channel behavior but does not provide persistent duplex operation for supported sandbox providers. **Roadmap alignment** This change supports the Cloud / Sandbox agents section in ROADMAP.md. ## What Changed - Add the duplex bridge broker with bounded forward, response, and gateway wait budgets. - Bound concurrent requests, lifetime requests, and request-id bytes before retention or forwarding. - Select duplex transport only when both required gates are true. - Assign the loopback port and nonce on the host and enforce a liveness-only READY frame. - Fall back to the file bridge after invalid readiness, contamination, bind failure, or timeout. - Carry the kill switch through the server, acpx engine, and six local adapters. - Add fixed, redacted duplex telemetry with a provider allowlist. - Add a real-process end-to-end harness for readiness, round trips, loss, and teardown. - Add regression coverage for limits, loss, UTF-8 splits, concurrency, and telemetry dimensions. ## Verification - Adapter-utils, server, and Daytona typechecks pass locally. - Adapter-utils tests pass, including the codec, broker, execution-target sandbox, and real-process harness. - Server kill-switch tests pass. - Live Daytona tests pass with the required provider key and skip without that key. - The root pnpm-lock.yaml file has no diff. - The branch contains ten commits after origin/master. ## Risks - Duplex transport remains disabled unless both gates equal true. - A provider remains an untrusted boundary and needs least-privilege credentials and quotas. - The server telemetry recorder stays deferred; the default recorder does nothing. - A provider that pre-binds the host port causes a fail-closed fallback to the file bridge. - The change adds no database migration and changes no root lockfile. ## Model Used OpenAI GPT-5, exact model family GPT-5, large context window, reasoning, and tool use. The model assisted with Git handoff validation and PR preparation. The implementation commits came from the engineering worktree. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with Fixes: # / Closes # / Refs # OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub #NNN / github.com/paperclipai/paperclip URLs) - [x] My branch name describes the change (e.g. docs/... or fix/...) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge - [x] I searched the GitHub PR list for similar PRs and confirmed this is not a duplicate --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
8161244284 |
feat(sandbox): add opt-in duplex command-stream foundation (capability, protocol, bounded host route, frame codec) (#11738)
## Thinking Path > - Paperclip provides a control plane for companies that run AI agents. > - Sandboxed agents need a safe execution path for persistent command streams. > - The existing callback transport does not provide a bounded, generic duplex route. > - The host must control capability access, route identity, protocol limits, and close behavior. > - This pull request adds an opt-in duplex command-stream foundation across the sandbox layers. > - The feature stays inert because no current provider declares the capability. ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting (multiple of the above) **Problem or motivation** Sandbox command execution needs a persistent host-to-sandbox stream. The current callback bridge uses a file transport and does not provide this generic route. **Proposed solution** Add a fail-closed provider capability, generic worker protocol messages, a host-owned bounded route, cross-layer service mediation, and a versioned newline-delimited frame codec. **Alternatives considered** Keep the file transport and add feature-specific commands. This does not provide one reusable duplex contract or host-owned route bounds. **Roadmap alignment** This work supports the completed Cloud / Sandbox agents roadmap area and the safe autonomy goal in the product definition. **Additional context** The change passed a two-stage security review. The final code review verdict was approve after fixes for active-stream bounds and service-layer capability mediation. ## What Changed - Add the opt-in `duplexCommandStream` provider capability with fail-closed narrowing. - Add duplex open, write, stop, and close requests and data and exit notifications to the plugin worker protocol. - Add a host-owned route with bounds for chunk size, cumulative bytes, lifetime, protocol errors, pending requests, and pre-bind buffering. - Add close acknowledgement handling with worker retirement when the close remains unconfirmed. - Wire `openDuplexChannel` through the execution target, runtime service, and plugin worker. - Add a versioned frame codec with shared wire-compatibility vectors and split UTF-8 handling. ## Verification - `server/src/__tests__/plugin-worker-manager-duplex.test.ts` passes 18 tests. - `server/src/__tests__/environment-execution-target-duplex.test.ts` passes 11 tests. - `packages/adapter-utils/src/duplex-frame-codec.test.ts` passes 38 tests. - `server/src/__tests__/sandbox-capability-contract.test.ts` passes 15 tests. - Setup-token pseudo-terminal regression tests pass 47 tests. - Server TypeScript check passes. - Continuous integration will run the full required test, typecheck, build, and policy checks. ## Risks - Providers that opt into the capability must implement the complete worker protocol. - Route limit defaults can close a stream when a workload exceeds the configured bounds. - The capability remains disabled for current providers, so current production behavior does not change. ## Model Used OpenAI GPT-5 (`gpt-5`), with tool use and code execution. The model reviewed and prepared this pull request from the supplied implementation and verification record. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |