mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-25 17:45:53 +01:00
fix/postgres-null-socket-write-guard
22
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nicky LeachandPaperclip
|
29d12045f4 |
perf(gateway): remove the whole-body string round trip on the HTTP/2 send path (#12189)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandbox agents send callback requests through a generated gateway. > - The HTTP/2 callback path converts each full request body to a UTF-8 string. > - The same path converts that string back to a buffer before it sends the request. > - This pull request sends the body buffer directly to the HTTP/2 forwarder. > - The benefit is less copying and unchanged queue payload behavior. ## Linked Issues or Issue Description **What existing behavior does this improve?** The sandbox callback gateway HTTP/2 send path copies the full request body through a UTF-8 string before it sends the body. **Subsystem affected** The affected subsystem is `packages/adapter-utils`, which contains sandbox gateway and HTTP/2 adapter utilities. **Current behavior** The gateway reads the body into a buffer, converts the complete body to a UTF-8 string, and converts that string back to a buffer for the HTTP/2 path. The queue path stores the string payload. **Proposed behavior** The gateway keeps a byte reader for the HTTP/2 path. A thin text wrapper keeps the queue path behavior. The HTTP/2 path sends the original body buffer. **Reason and benefit** The extra conversions add work and memory use without changing the HTTP/2 body bytes. Direct buffer forwarding removes that work and preserves the size limit and reject behavior. **Breaking changes** None. The queue payload remains a string. The body size limit, content type check, and reject point remain unchanged. **Additional context** This change has no public issue link. The repository roadmap search found no duplicate planned work. The implementation also adds tests for non-ASCII JSON, malformed UTF-8, size limits, and queue payload shape. ## What Changed - Add `readBodyBytes(req)` for byte-preserving body reads. - Keep `readBody(req)` as a string wrapper for the queue path. - Send the byte buffer directly on the HTTP/2 path. - Add tests for byte identity, malformed UTF-8, size limits, and queue payload shape. ## Verification - `pnpm --filter @paperclip/adapter-utils test src/sandbox-callback-bridge.test.ts` passed with 51 tests. - `pnpm --filter @paperclip/adapter-utils exec tsc --noEmit` passed. - The tests spawn the generated gateway and the real host HTTP/2 bridge. - The tests verify byte identity, malformed UTF-8, pre-forward size rejection, queue file protection, and string queue payloads. - Full repository CI must pass before merge. ## Risks Low risk. The HTTP/2 path changes its internal body conversion only. The queue path keeps the prior string payload. The size limit and reject point stay unchanged. ## Model Used OpenAI Codex, GPT-5, tool use and code review assistance. The exact runtime context window is managed by the Codex service. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
802f2af154 |
refactor(adapter-utils): delete the dead duplex body-chunk protocol code (#12186)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The adapter utilities package provides transport code for sandbox agents > - The retired `duplex_v1` broker no longer produces or consumes body-chunk frames > - Dead protocol code remains in the host codec, gateway copy, bridge options, and tests > - This pull request removes that dead code and keeps the READY handshake unchanged > - The benefit is a smaller transport surface with fewer unused paths to maintain ## Linked Issues or Issue Description **What existing behavior does this improve?** This change improves the adapter utilities code that supports sandbox duplex readiness and frame handling. **Subsystem affected** `packages/adapter-utils/` — sandbox transport codecs, execution targets, and callback bridge tests. **Current behavior** The repository keeps body-chunk frame types, validators, a body spool, decoder limits, and tests after the `duplex_v1` broker removal. No live producer or consumer uses this code. **Proposed behavior** Remove the unused body-chunk protocol code and retain the READY handshake, its strict checks, and its size limits. **Reason and benefit** The removal reduces dead code and keeps the host and embedded gateway paths easier to inspect. It adds no new behavior. **Breaking changes** The removed frame types now decode as `unknown_type`. The live readiness gate already ignores those frames. The READY handshake stays byte-for-byte compatible. **Additional context** This cleanup follows [PR #12171](https://github.com/paperclipai/paperclip/pull/12171), which removed the duplex broker. ## What Changed - Remove `duplex-body-spool.ts` and its test. - Remove unused body-chunk frame types, validators, decoder code, vectors, and limits. - Remove the unused `reassembledBody` option and decoder limit environment entry. - Remove the embedded gateway decoder copy and the unused frame type map. - Keep the READY handshake and its existing boundary tests unchanged in behavior. ## Verification - `pnpm -F @paperclip/adapter-utils typecheck` passes. - The duplex frame codec test passes with 30 tests. - The sandbox execution-target test passes with 136 tests. - The sandbox callback bridge test passes with 46 tests. - CI must confirm all required checks after it starts. ## Risks Low risk. The change removes code only. The READY handshake, HTTP/2 body path, and byte-ledger path remain unchanged. ## Model Used OpenAI Codex, GPT-5, tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
2862e18484 |
refactor(adapter-utils): remove the retired duplex_v1 sandbox bridge transport (#12171)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The adapter utilities provide sandbox transport paths for agent execution > - The retired `duplex_v1` path remains in host, gateway, and test code after `http2_v1` replaced it > - Retired transport code adds maintenance cost and leaves an unsafe fallback for unknown gateway modes > - This pull request removes the retired path, moves shared `http2_v1` contracts to a leaf module, and closes mode dispatch to a fixed allowlist > - The benefit is a smaller transport surface and explicit failure for unsupported modes ## Linked Issues or Issue Description Refs #12120 The `http2_v1` transport replaced `duplex_v1`, but the retired broker, gateway, constants, and tests remain in the adapter utilities. An unknown bridge mode can also fall through to the queue gateway when a queue directory exists. This change removes the retired code and rejects unsupported modes before gateway selection. ## What Changed - Delete the host `duplex_v1` broker and its transport-only tests. - Delete the in-sandbox duplex gateway and retired mode constants. - Move shared `http2_v1` symbols into `bridge-transport-contract.ts`. - Update the remaining importers and repair their focused tests. - Validate bridge modes against `http2_v1` and `queue_v1` before queue lookup. - Keep `queue_v1`, `duplex-frame-codec.ts`, and duplex telemetry dimensions unchanged. ## Verification - [x] `npx tsc --noEmit -p packages/adapter-utils` passes. - [x] `npx vitest run packages/adapter-utils/src` passes: 48 files and 968 tests pass, with 4 pre-existing platform skips. - [x] Full CI is green on this pull request. - [x] Greptile review is complete and every finding is resolved. ## Risks The change removes an internal transport that no host path selects. The main risk is an overlooked import or test dependency. Targeted typecheck and tests cover the adapter utility package. Full CI must confirm workspace-wide compatibility. ## Model Used Anthropic Claude Sonnet 5 assisted with the implementation, as recorded in the commit. The commit does not record a context-window size or reasoning mode. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
445547c989 |
feat(duplex): run the Daytona sandbox callback bridge over Node HTTP/2 (#12120)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox providers carry agent work through controlled execution channels > - The Daytona callback bridge uses a bespoke line-framed protocol over its duplex channel > - The bespoke protocol adds framing work and does not use the Node transport that already supports multiplexed streams > - This pull request carries raw bytes across the channel, adds a Node HTTP/2 bridge, and selects it for Daytona > - The benefit is one authenticated, multiplexed callback session with queue_v1 as the bounded fallback ## Linked Issues or Issue Description **Subsystem affected** The packages/plugins Daytona provider and the shared duplex execution path. **Problem or motivation** The Daytona callback bridge uses a bespoke line-framed protocol over the provider duplex channel. This adds protocol work and limits stream handling. **Proposed solution** Carry raw bytes through the cross-layer channel. Add an authenticated Node HTTP/2 host server and sandbox client gateway. Select http2_v1 for Daytona and retain queue_v1 as the fallback. **Alternatives considered** Keep the current duplex_v1 protocol. This keeps the bespoke framing path and does not provide one HTTP/2 session for callback streams. **Roadmap alignment** ROADMAP.md lists Daytona under cloud and sandbox agents. This change improves the shipped Daytona provider path. **Additional context** The branch adds no dependency. Node 24 provides the http2 module. The host token check and canonical path parser remain the single dispatch path. ## What Changed - Carry raw Uint8Array chunks through the adapter, plugin, worker, runtime, and Daytona layers. - Encode bytes as base64 only across the JSON-RPC hop, because JSON has no binary type. - Add the bounded host HTTP/2 server and the in-sandbox HTTP/2 client gateway. - Authenticate every stream with the per-run bridge token before route work. - Parse the path once and reuse the canonical result for route and forwarding work. - Select http2_v1 for Daytona and fall back once to queue_v1 when the client preface is absent. - Add transport, session, stream, and fallback telemetry. - Mark HTTP/2 as the preferred transport and queue_v1 as the soft-deprecated fallback. ## Verification - `npx vitest run packages/adapter-utils/src` — 990 passed and 4 skipped. - `npx vitest run server/src/__tests__/plugin-worker-manager-duplex.test.ts` — 32 passed. - `npx vitest run --config packages/plugins/sandbox-providers/daytona/vitest.config.ts` — 220 passed and 6 skipped. - `npx tsc --noEmit` in `packages/adapter-utils`, `packages/shared`, `packages/plugins/sdk`, and `server` — clean. - No `package.json` or `pnpm-lock.yaml` file changed. - The live Daytona test skips when `DAYTONA_API_KEY` is absent. - The root `npx tsc --noEmit` command has a pre-existing missing `packages/adapters/droid-local` reference on this branch and on `master`. ## Risks - The transport change affects several duplex layers and could expose byte-boundary errors. - A missing HTTP/2 client preface falls back once to queue_v1 and records `preface_missing`. - The host token check and canonical path parser must remain on the shared dispatch path. - The live Daytona test needs `DAYTONA_API_KEY` and does not run in this agent sandbox. ## Model Used OpenAI GPT-5, tool-enabled coding agent with repository inspection, GitHub CLI, and shell execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
87d68f476b |
fix: harden the sandbox bridge gateway against crashes and queue wedge (#12060)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents on remote sandbox targets reach the Paperclip API through the sandbox callback bridge: a loopback HTTP gateway inside the sandbox queues request files for a host-side worker > - The gateway process has no supervisor: nothing inside the sandbox respawns it, so a crash leaves a dead loopback port for the rest of the run > - The gateway also never cleaned up request files whose responses never arrived, so a stalled host wedged the queue at its depth cap and every later request got an immediate 503 > - #12052 made the host-side worker survive transient faults; this pull request hardens the other half of the relay > - The benefit is that a gateway fault degrades one request instead of severing the agent from the control plane until run end ## Linked Issues or Issue Description Refs #12052 (host-side worker half of the same relay). Refs #9904 and #8977 (adjacent bridge behavior). No public issue exists for this defect. The description below follows the bug report template. **What happened?** During a staging run, an agent's API calls to the bridge's loopback port began failing at the connection level (curl reported HTTP 000) partway through the run. A dead gateway process is the only mechanism that produces connection-level failures on that port, and nothing restarts it. Separately, request files for timed-out requests stayed in the queue; after 64 accumulated, the gateway answered every request with `503 Bridge request queue is full.` until the run ended. **Expected behavior** An uncaught fault in the gateway must not kill the loopback listener. A request that times out must not leave its file counting toward the queue-depth cap. A queue full of orphaned files must recover instead of rejecting until run end. **Steps to reproduce** 1. Start a remote-sandbox run and stop the host-side bridge worker. 2. Send requests to the gateway until they time out; the request files stay in `requests/`. 3. After 64 such files, every request gets an immediate 503, even after the host recovers. 4. Independently, raise any uncaught exception in the gateway process; the loopback port dies for the rest of the run. ## What Changed - The generated gateway source installs global `uncaughtException` / `unhandledRejection` handlers that log to stderr (already redirected to `logs/bridge.log`) and keep serving. The relay holds no state a fault can corrupt beyond the one request it interrupted. - Survival is gated on readiness: before the gateway has written its readiness file (file mode) or sent its READY frame (duplex mode), the same handlers exit(1) instead. A startup fault (failed bind, failed readiness write) means the process can never serve, and surviving there would only leave an un-ready zombie while the host waits out its readiness poll. - The file gateway attaches an explicit `error` listener to its server and pins the event loop with a keepalive until the bind settles. Newer Node runtimes do not reliably surface a failed bind through `uncaughtException` in this shape: the process can drain and exit 0 before the error event is delivered (reproduced on Node 24/25; Node 22 delivered it). The duplex gateway already had an explicit listener. - A request that times out waiting for the host now deletes its own request file. The host's response write is guarded on that file, so the removal also signals that no caller waits anymore. - At the queue-depth cap, the gateway sweeps request files older than the response deadline (orphans from killed callers or a previous gateway process) before rejecting with 503. - Host-side, `processRequestFile` treats a request file that vanished before the read as the benign caller-gave-up race and skips it quietly instead of escalating into the recovery pass. ## Verification - `npx vitest run packages/adapter-utils/src/sandbox-callback-bridge.test.ts` — 43 passed, verified on both Node 22 and Node 25. - New end-to-end test: with no worker running, a request times out (502), its file is cleaned, and the same gateway then serves a 200 once a worker starts — no wedge, no dead port. - New end-to-end test: with `maxQueueDepth: 1` and a backdated orphan file at the cap, the gateway sweeps the orphan and admits the request instead of answering 503. - New worker test: a request file that vanishes before the read is skipped without a handler call, a response write, or a run-level error. - New generated-source test: spawned directly against an already-occupied port, the gateway exits 1 promptly with the `EADDRINUSE` fault on stderr instead of lingering un-ready (or exiting 0 silently, the pre-existing behavior on Node 24/25). - A pin keeps the crash handlers, the readiness gate, and the sweep in the generated source. - `pnpm --filter @paperclipai/adapter-utils typecheck`. ## Risks - Keeping a Node process alive after `uncaughtException` is normally suspect; here the alternative is a dead loopback port for the rest of the run, and the gateway is a stateless per-request relay. The fault is logged with its stack to `bridge.log`, and survival applies only after readiness — startup faults still fail fast. - Deleting a timed-out request file could race a host that is mid-processing. The host's response write is already guarded on request-file existence, and the new host-side skip treats the vanished file as a no-op, so no duplicate mutation path is introduced. - The stale sweep runs only at the depth cap and only removes files older than the response deadline plus a 2 s grace, so a live caller's file is never swept. - Orphaned response files (host responded after the caller gave up) still linger; that pre-existing minor leak is unchanged here. ## Model Used - Claude Fable 5 (Anthropic), model id `claude-fable-5`, extended thinking enabled, agentic tool use via Claude Code (CLI harness), 200k context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Devin Foley
|
c7f4bc1300 |
fix: survive transient sandbox exec failures in the callback bridge worker (#12052)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents on remote sandbox targets reach the Paperclip API through the sandbox callback bridge: a loopback gateway inside the sandbox writes request files, and a host-side worker polls them over the provider's exec channel and forwards them to the server > - The worker's poll loop had one terminal catch: a single reset or slow exec ended the relay for the rest of the run > - The in-sandbox gateway kept queueing requests against the dead worker, so every later API call from the agent stranded, including its final status write > - A relay that dies on one transient fault turns a routine provider hiccup into a lost issue disposition > - This pull request restructures the loop so transient faults back off and retry, while the watchdog remains the escalation path for sustained outages > - The benefit is that one flaky exec no longer severs an agent from the control plane mid-run ## Linked Issues or Issue Description Refs #9904. Refs #8977. Both touch adjacent bridge behavior (curl shim, header forwarding); neither addresses worker-loop lifetime. No public issue exists for this defect. The description below follows the bug report template. **What happened?** During a staging run, the host-side bridge worker hit one failed sandbox exec while relaying requests. The poll loop's only catch is terminal: it failed the pending requests and set the worker to settled, with no restart. The agent's later API calls saw connection-level failures or bridge errors until the run ended. Its final `PATCH status: done` was lost, and the missing-disposition recovery had to repair the issue in a corrective run. **Expected behavior** One transient exec failure must not end the relay for the rest of the run. The worker must back off and retry. A sustained outage must still fail queued requests fast through the watchdog. In-flight request semantics (abort plus 504 backstop, retry-safe 503) must not change. **Steps to reproduce** 1. Start a remote-sandbox run and let the provider exec channel reject or stall one call while the bridge worker polls. 2. The worker hits one `listJsonFiles` failure or one request-attempt timeout, and the loop exits through its terminal catch. 3. Every later bridge request strands. The loopback gateway keeps accepting requests that never complete, and after 64 queued files it answers every request with 503 until run end. ## What Changed - `startSandboxCallbackBridgeWorker`'s poll loop now separates three failure domains: - A failed poll backs off exponentially (capped at 5 s) and retries instead of dying. The first failure of a streak still lands on the run trace through the workerFailed span; later repeats only warn. - A failed or hung request attempt runs the same recovery pass the loop previously died on — abort the in-flight handler (its 504 backstop keeps the caller from stranding) and 503 the unclaimed queued requests — and the loop then continues and serves the caller's retry. - A listing where every file already has an in-flight attempt sleeps one poll interval, like an empty listing. The previous immediate re-list was a hot spin: an exec storm against a real channel, and a pure-microtask loop that starved every timer in the process when the queue client resolves synchronously. - The watchdog, the claim/finalize fences, and the stop/drain semantics are unchanged. ## Verification - `npx vitest run packages/adapter-utils/src/sandbox-callback-bridge.test.ts` — 38 passed. - New regression test: a request queued behind three consecutive poll failures is still delivered. - Updated tests: the stalled-poll test now expects recovery (the handler's real 200) instead of a terminal 503; the sustained-outage 503 path remains proven by the dedicated watchdog test; the recovery-503-write-retry test triggers the recovery pass through a hung request read, because a hung poll no longer runs that pass. - `pnpm --filter @paperclipai/adapter-utils typecheck`. ## Risks - Behavior change: a transiently failing poll no longer mass-fails queued requests on the first error. Callers wait through the backoff window, bounded by the existing in-sandbox 30 s response deadline, or the watchdog fails them after 20 s of no successful iteration. This trades fast-but-terminal degradation for recovery. - A hard-down channel now retries every ≤5 s for the rest of the run instead of stopping. Each retry is one exec attempt against a channel that already fails. - In-flight mutation safety is unchanged: the guard map and the claim protocol still prevent a double-applied host mutation, and a guarded file is always finalized by its own attempt or by its 504 backstop. ## Model Used - Claude Fable 5 (Anthropic), model id `claude-fable-5`, extended thinking enabled, agentic tool use via Claude Code (CLI harness), 200k context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Jannes StubbemannandClaude Fable 5
|
d77eeb8914 |
fix(sandbox-bridge): allow the agent-hire skill's routes through the callback bridge (#8978)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Managed agents run inside a sandbox and reach the Paperclip server only through the sandbox callback bridge, which forwards a fixed route allowlist (`DEFAULT_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST`) > - The `paperclip-create-agent` skill instructs an agent to call adapter/icon discovery endpoints, compare existing agent configurations, submit a hire request, and link the resulting approval to its source issue > - None of those routes were on the bridge allowlist, so a sandboxed agent following the skill correctly hit `Route not allowed` on every call — including the hire `POST` itself — making hiring impossible from inside a sandbox > - This pull request adds the six routes the skill uses to the bridge allowlist, while keeping direct agent creation (`POST /api/companies/:id/agents`) denied > - The benefit is that hiring works end-to-end for sandboxed agents through the approval-gated `agent-hires` path, without widening the bridge beyond what the skill needs ## Linked Issues or Issue Description No public issue exists; describing the bug in-PR (bug template fields): - **What happened:** A managed agent running in a sandbox followed the `paperclip-create-agent` skill and got `Route not allowed` from the callback bridge on every endpoint the skill documents — adapter discovery (`/llms/agent-configuration.txt`, `/llms/agent-configuration/:adapterType.txt`, `/llms/agent-icons.txt`), config comparison (`GET /api/companies/:id/agent-configurations`), the hire submission (`POST /api/companies/:id/agent-hires`), and approval linking (`POST /api/issues/:id/approvals`). - **Expected behavior:** An agent with hiring permission can complete the hire flow from inside a sandbox; the bridge forwards the skill's routes and the server enforces authorization (`canCreateAgents`). - **Impact:** Hiring by sandboxed agents was fully broken — the failure is in the transport allowlist, not permissions, so no configuration could work around it. Related: #8981 (companion fix making the `paperclip-create-agent` skill available to agents that can hire; supersedes #8823). The two changes serve the same end-to-end hire flow but are independently mergeable — this PR is purely the bridge transport allowlist. Supersedes #8853. ## What Changed - `packages/adapter-utils/src/sandbox-callback-bridge.ts`: add six routes used by the `paperclip-create-agent` skill to `DEFAULT_SANDBOX_CALLBACK_BRIDGE_ROUTE_ALLOWLIST` (three `GET /llms/...` discovery routes, `GET .../agent-configurations`, `POST .../agent-hires`, `POST /api/issues/:id/approvals`), with a comment documenting why direct agent creation stays denied - `packages/adapter-utils/src/sandbox-callback-bridge.test.ts`: assert the six routes are allowed, and add negative cases proving the regexes do not over-match (no `POST .../agents`, no non-`.txt` or arbitrary `/llms` files, no `agent-hires` sub-resources) ## Verification - `npx vitest run packages/adapter-utils/src/sandbox-callback-bridge.test.ts` — 13/13 tests pass locally - `npx tsc --noEmit -p packages/adapter-utils` — clean - Manual: run a managed agent in a sandbox, invoke the `paperclip-create-agent` skill, and confirm the discovery calls, hire `POST`, and approval linking all pass through the bridge; `POST /api/companies/:id/agents` still returns `Route not allowed` ## Risks - Low risk: additive allowlist entries only; anchored regexes with `[^/]+` segments prevent over-matching (covered by tests) - The bridge allowlist bounds surface area but does not replace server-side authorization — the hire `POST` remains approval-gated and permission-checked (`canCreateAgents`) on the server > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude (Anthropic) — Fable 5 (`claude-fable-5`), extended thinking, agentic tool use via Claude Code; original diff authored with Claude Opus 4.8 (1M context) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Nicky LeachandPaperclip
|
6f26f2a450 |
fix(adapter-utils): add per-iteration timeout and watchdog to the sandbox callback bridge poll loop (#11341)
## Thinking Path > - Paperclip runs AI agents through adapters and sandboxed execution paths > - The sandbox callback bridge carries file requests between the host and a sandbox > - The poll loop waited forever when a sandbox call stopped responding > - A permanent wait stranded queued requests and hid the run failure > - This pull request adds bounded timeouts, abort handling, recovery backstops, and trace reporting > - The benefit is prompt request failure, safe mutation outcomes, run-level error reporting, and trace visibility ## Linked Issues or Issue Description **What happened?** The sandbox callback bridge could wait forever when a client call stopped responding without a rejection. **Expected behavior** The bridge should fail queued requests and report a run-level error when the sandbox channel stops responding. **Steps to reproduce** 1. Start a sandbox callback bridge. 2. Queue a request. 3. Make the sandbox call stop responding. 4. Observe that the request does not receive a failure response. **Paperclip version or commit** Commit `edc4f71b460c600f97cf44cb486d5cac72ca2db9`. **Deployment mode** Built from source. **Installation method** Built from source with pnpm. **Agent adapter(s) involved** Custom or external sandbox callback bridge. **Database mode** Not database-related. ## What Changed - Add a per-iteration timeout for `listJsonFiles` and `processRequestFile`. - Add a watchdog that fails pending requests when the loop makes no progress. - Abort a hung handler and use a non-retryable 504 backstop when its outcome can be indeterminate. - Retry recovery writes and keep queued requests when a recovery write fails. - Forward the indeterminate-outcome header through the execution target. - Record worker failures through the `sandbox.callbackBridge.workerFailed` trace span. - Add tests for timeout, watchdog, recovery, mutation safety, header forwarding, and fast-request behavior. ## Verification - Run `pnpm exec vitest run packages/adapter-utils/src/sandbox-callback-bridge.test.ts packages/adapter-utils/src/execution-target-sandbox.test.ts`. - Confirm that the PR test, typecheck, build, end-to-end, serialized test, and security checks pass. - Confirm that the current PR head is `edc4f71b460c600f97cf44cb486d5cac72ca2db9`. - Confirm that the PR changes four files: the callback bridge, its tests, the execution target, and its tests. ## Risks The default timeout can fail a slow but valid sandbox call. The defaults remain configurable, and the iteration timeout stays below the sandbox response deadline. A mutation that may have committed returns a non-retryable 504 outcome so the caller does not apply it twice. ## Model Used OpenAI Codex, GPT-5 current runtime, with extended reasoning and tool use. The exact context window is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
9ace548fd2 |
feat(observability): rename sandbox provider spans and add run-time wrapper spans (#10999)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip uses adapter and sandbox code to start agents and run sandbox work > - The current sandbox spans use mixed names and do not group related run-time work > - Mixed names make traces harder to read and compare across providers > - This pull request renames provider spans, adds run-time wrapper spans, and keeps the host allowlist closed > - The benefit is clearer traces with the same sandbox behavior and trust boundary ## Linked Issues or Issue Description **What existing behavior does this improve?** This improves OpenTelemetry span names and grouping for sandbox startup, execution, callback relay, and agent session work. **Subsystem affected** Cross-cutting (multiple of the above): adapter utilities, sandbox providers, shared telemetry documentation, and server instrumentation. **Current behavior** Sandbox provider spans use mixed names. Related run-time operations expose inner `sandbox.exec` spans without a named wrapper span. The host mapper uses a closed allowlist for provider span names. **Proposed behavior** Use descriptive provider-scoped span names. Add wrapper spans for agent session input, agent session output polling, and callback relay. Keep the host mapper allowlist closed and map unknown names to `other`. **Reason and benefit** Clear names make traces easier to read and reduce ambiguity during sandbox operation analysis. Wrapper spans show the full operation while preserving the inner execution spans. **Breaking changes** None. This change updates telemetry span names and grouping only. It does not change sandbox behavior, endpoint behavior, or the host trust boundary. **Additional context** Related prior work: [#10758](https://github.com/paperclipai/paperclip/pull/10758). ## What Changed - Rename Daytona provider sync and session spans with descriptive provider-scoped names. - Add three run-time wrapper spans for agent session input, output polling, and callback relay. - Add a shared span runner that preserves no-op behavior without a real tracer. - Keep the host mapper allowlist closed and map unknown names to `other`. - Update telemetry documentation and span-name tests. ## Verification - Focused adapter-utils span tests pass for startup timing, callback relay, and sandbox execution. - Focused Daytona plugin span tests pass for renamed leaf spans and session open or close spans. - Focused server tests pass for host mapping and instrumentation. - The stacked diff contains one commit on top of `feat/daytona-persistent-session-model`. ## Risks - Span names change for existing telemetry consumers. - The wrapper spans add trace structure but do not change sandbox execution. - The host mapper keeps the existing closed allowlist and `other` bucket. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 (Codex agent); exact deployment revision and context window are not exposed in this run; tool use and code execution enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
c647b8cc2e |
feat(acpx-engine): give sandbox.exec spans real parents (#10852)
## Thinking Path > - Paperclip uses spans and traces to show how work moves through agents and tools > - sandbox.exec spans need a real parent so the trace tree matches the work tree > - Wrong parent links make execution history hard to read and hard to debug > - This pull request adds a single task.run root span and re-parents live work to the nearest active span > - The change keeps detached work under the closest live span instead of the HTTP root > - The benefit is a clear trace tree for sandbox.exec work and better execution diagnosis ## Linked Issues or Issue Description **What happened?** sandbox.exec spans attached to the wrong parent or to no live parent in some paths. **Expected behavior** Each sandbox.exec span should attach to the nearest live span. **Steps to reproduce** 1. Run work that creates sandbox.exec spans during startup and callback bridge paths. 2. Inspect the trace tree. 3. Observe an orphaned span or a span with the wrong parent. **Paperclip version or commit** `672e9de9c8b004aebc1f08e24b612ab067735ad1` **Deployment mode** Local dev. **Additional context** The branch adds the task.run root span, parents sandbox.startup to it, and re-parents detached bridge work to the nearest live span. ## What Changed - Added a task.run root span for the run tree. - Re-parented sandbox.startup, agent.turn, and detached bridge work to the nearest live span. - Added end-to-end trace-tree assertions for the full parent chain. - Added negative coverage so sandbox.exec does not parent to the HTTP root. ## Verification - Focused Vitest suite passed: `packages/adapter-utils/src/acpx-engine/execute.test.ts`, `packages/adapter-utils/src/acpx-engine/startup-timing.test.ts`, `packages/adapter-utils/src/execution-target-sandbox.test.ts`, `packages/adapter-utils/src/sandbox-callback-bridge.test.ts`, and `server/src/__tests__/environment-execution-target.test.ts`. - Result: 5 files passed, 204 tests passed. - The submitted branch also reported `adapter-utils` checks, `server` seam checks, and `tsc` exit 0 in the handoff state. ## Risks - This change can alter trace tree shape in tools that read parent spans. - A missed bridge path could still point to the wrong live span. - Low risk for runtime behavior, because the change only changes span parent attribution. ## Model Used OpenAI GPT-5, tool-use capable. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
ded813ad6f |
feat(interactions): add governed agent addressees (#10252)
## Thinking Path > - Paperclip is the control plane that lets humans govern companies of AI agents. > - Issue-thread interactions are the structured handoff point for confirmations, questions, suggested tasks, and other governed decisions. > - Those interactions previously assumed that only board users could resolve them, preventing one agent from explicitly addressing another agent for a response. > - Agent resolution needs company-level governance, auditable resolver identity, safe terminal-state handling, and attention routing so authorization is enforced server-side rather than inferred from UI behavior. > - This pull request adds governed agent resolution, withdrawal and terminal expiry semantics, explicit agent addressees, lifecycle reconciliation, and attention-feed filtering. > - The benefit is that agents can participate in structured decisions without weakening board control, company isolation, wake behavior, or audit invariants. ## Linked Issues or Issue Description ### Subsystem affected Issue-thread interactions across database, shared contracts, server authorization/services, adapter callbacks, agent skill guidance, API docs, and UI governance surfaces. ### Problem or motivation Structured interactions were board-only, had no explicit agent addressee, and lacked durable withdrawal/terminal-expiry semantics. That made peer-agent decisions impossible to authorize and audit safely. ### Proposed solution Persist requested/effective resolver policy and addressee identity, enforce company governance and eligible agent resolution, reconcile addressee lifecycle changes, expose withdrawal and terminal expiry, and route attention to the intended active agent with board fallback. ### Alternatives considered Implicitly authorizing the issue assignee or mentioned agents was rejected as ambiguous and difficult to audit. Using comments alone was rejected because it loses structured outcomes and continuation behavior. ### Roadmap alignment Supports the ROADMAP direction for lightweight leadership-agent communication that still resolves into governed decisions and work objects. ### Additional context Public GitHub issue/PR search found no duplicate implementation; open PR search for interaction resolver governance and agent addressees only returned this PR. ## What Changed - Add company-scoped interaction resolver governance contracts and persistence. - Add requested/effective resolver policy, resolver identity, withdrawal, and terminal-expiry behavior. - Add explicit `addresseeAgentId` validation, authorization, persistence, lifecycle reconciliation, API documentation, and skill guidance. - Route pending addressed interactions to the intended invokable agent and fall back to board attention when that agent becomes ineligible or is deleted. - Preserve sandbox callback identity fields required by governed resolution paths. - Add migrations `0193` and `0194` plus route, service, attention, adapter, CLI, and UI coverage. - Add governance state and company settings UI, including responsive mobile behavior and distinct withdrawn/expired audit presentation. ## Verification - `pnpm check:token-gates` — passed. - `pnpm -r typecheck` — passed, including migration numbering and safety checks. - `pnpm test:run` — feature/server and UI workspace suites passed; one unrelated CLI AWS doctor test observed injected static AWS credentials and warned instead of passing. - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY pnpm exec vitest run cli/src/__tests__/secrets.test.ts --project paperclipai` — 8 tests passed, confirming the failure was environment-sensitive. - `pnpm build` — passed. - Latest rebased head `e24cece6be9f1877bdbac7691bcb44fd583c0161` completed all GitHub CI jobs successfully. ## Risks - Migrations add interaction and company-governance fields; numbering is conflict-free on current `master`, additive statements are idempotent, and migration safety checks pass. - Agent authorization behavior expands beyond board-only resolution, but defaults remain board-only and coverage exercises company boundaries, resolver eligibility, lifecycle invalidation, wake behavior, withdrawal, expiry, and attention fallback. - Attention routing depends on current agent invokability; reconciliation and read-time filtering prevent stale addressees from retaining visibility or resolution authority. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex using `gpt-5.6-sol` with reasoning, terminal tool use, code execution, Git/GitHub integration, and Paperclip control-plane tools. Context-window metadata was not reported by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Nicky LeachandPaperclip
|
2ab797dcbe |
fix(sandbox): reset step store for long-lived bridge work (#10813)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Bridge workers keep startup setup separate from long-lived runtime
work
> - The callback bridge must keep queue-directory setup inside the
startup step
> - The long-lived poll loop must run with no active startup step
> - This pull request keeps that boundary in the right place
> - The benefit is correct span parents and correct runtime exec
metadata
## Linked Issues or Issue Description
**Bug**
**What happened?**
Long-lived bridge continuations kept a stale startup step store during
the queue-directory setup path.
**Expected behavior**
Runtime exec spans should start with no active startup step.
**Steps to reproduce**
1. Start a bridge lane.
2. Let the startup step end.
3. Run later runtime exec work on the same lane.
**Paperclip version or commit**
|
||
|
Nicky LeachandPaperclip
|
674d71548a |
perf(sandbox): fold dead sandbox start round trips (bridge dirs + handle-cache seed) (#10485)
## Thinking Path > - Paperclip is the control plane for AI agents. > - Sandbox startup uses bridge directories and a Daytona workspace handle. > - The cold path made repeat directory creation calls and one avoidable handle fetch. > - Those calls add delay but do not change state. > - This pull request folds the bridge directory setup into one exec, removes redundant process-session setup, and seeds the Daytona handle cache at acquire. > - The benefit is fewer deterministic host-to-sandbox round trips and faster cold starts. ## Linked Issues or Issue Description - Problem: Cold sandbox start does extra directory creation work and re-fetches a handle it already has. - Expected result: The startup path should create each directory once and reuse the fresh handle. - Related PRs I found on GitHub: #9280, #9293. ## What Changed - Added `makeDirs` to the bridge queue client and used one `mkdir -p` exec for the callback bridge directories. - Removed the two upfront `mkdir` execs for the process-session bridge stdin and events directories. - Seeded the Daytona sandbox handle cache at acquire so realize can reuse the fresh handle. - Reset the process-scoped cache in the compatibility test so the second sync run sees the expected exec count. ## Verification - `pnpm --filter @paperclipai/adapter-utils exec vitest run` - 351 passed, 4 skipped. - `pnpm --filter @paperclipai/sandbox-provider-daytona exec vitest run` - 91 passed. - `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit` - clean. - I checked `ROADMAP.md` for sandbox round-trip work. I found no duplicate planned core work for this change. ## Risks - Low risk. The change removes redundant calls and adds cache seeding. - A wrong cache scope would hide the handle. The seed now checks the lease scope and fails loudly. - The daytona package `tsc --noEmit` still depends on SDK types that are not installed in this isolated workspace. CI covers that path. ## Model Used - OpenAI GPT-5, tool-using, with code execution in the current workspace. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used with capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked public issues or described the issue in-PR - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
9ff7e1caf3 |
perf(adapter-utils): content-hash-skip process-session remote script write + lock inbound round-trip count (#10377)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The sandbox runtime depends on efficiently staging files into remote environments > - The process-session bridge was rewriting a static remote script on every start, even when the remote copy already matched > - That caused unnecessary round trips and made the startup path noisier than it needed to be > - This pull request adds a hash-skip path for the process-session script write and keeps the existing bridge-entrypoint hash gate on the same helper > - It also locks in the reduced inbound round-trip count so the collapse from the prior PR cannot silently regress > - The benefit is fewer remote execs on warm starts and a stronger guard against performance regressions ## Linked Issues or Issue Description - Refs: https://github.com/paperclipai/paperclip/pull/10354 ### Feature request-style description **Subsystem affected** - Cross-cutting (packages/adapter-utils and sandbox staging helpers) **Problem or motivation** - The process-session bootstrap path was rewriting a static remote script on every start even when the remote file already matched the host content. - That added avoidable remote execs and latency to warm starts. - The inbound staging collapse from the previous PR also needed a regression lock so it could not silently drift back to extra round trips. **Proposed solution** - Route the process-session remote script upload through the existing hash-skip helper used by the sandbox callback bridge entrypoint. - Keep the bridge-entrypoint behavior unchanged by delegating it to the same helper. - Add a regression test that asserts the inbound staging path still collapses to the expected round-trip count. **Alternatives considered** - Keep the existing unconditional write path and accept the extra execs on warm starts. Rejected because it preserves avoidable overhead. - Add a second specialized helper just for process-session scripts. Rejected because the bridge-entrypoint logic already solved the same problem and should stay aligned. **Roadmap alignment** - This fits the roadmap items around cloud/sandbox agents and enforced outcomes by reducing bootstrap waste and preventing performance regressions. - It does not introduce new product surface area, telemetry, or user-facing workflow changes. **Additional context** - The implementation preserves the existing upload/verify/rename behavior when the remote content differs and only skips the write when the hash already matches. - PR #10354 collapsed the inbound staging path; this PR preserves that win. ## What Changed - Added a shared hash-skip helper for remote text-file synchronization so unchanged content skips the write path after a single remote hash check. - Switched the process-session remote script upload to use that helper, preserving the existing write/verify/rename behavior when the remote content differs. - Refactored the sandbox callback bridge entrypoint sync to use the same helper without changing its observable behavior. - Added a mocked-native-runner regression test that locks the inbound staging path to one `client.syncIn` round-trip per step and zero direct write/run execs. ## Verification - `tsc --noEmit` - `pnpm --filter @paperclipai/adapter-utils exec vitest run` - Full adapter-utils vitest sweep: 338 passed / 4 skipped ## Risks - Low risk: the helper changes when a write occurs, not the script contents or the remote execution surface. - A hash-check failure now fails loudly instead of silently rewriting, which is safer but could surface provider-side issues earlier than before. - The regression test is intentionally specific to the current collapsed staging path, so future architectural changes will need test updates. ## Model Used OpenAI Codex (GPT-5, tool-using coding agent) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
9cde4e128c |
feat: run ACP sessions in sandbox execution targets (#9390)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent adapters (Claude, Codex, Gemini) default to the ACP engine lane, which needs a live bidirectional stdio session with the agent process > - Sandbox execution targets only exposed one-shot command execution, so every ACP-capable adapter refused remote targets and fell back to the CLI lane with a "supports only the local Paperclip host" warning > - Running agents in sandboxes is a core deployment mode, and losing ACP there means losing streaming updates, structured events, and default-lane parity with local runs > - This pull request adds a provider-agnostic process-session bridge that relays the ACP stdio session into the sandbox over the existing sandbox runner contract, and updates the adapters to use it > - The benefit is that the default ACP lane now behaves the same on the local host and in any sandbox provider, with CLI fallback reserved for targets that genuinely cannot host a bidirectional session ## Linked Issues or Issue Description No existing public issue covers this; inline description following the feature request template: **Problem or motivation** Configuring an ACP-capable adapter (e.g. Claude) with a sandbox environment made every run fall back to the CLI lane with the warning "Claude ACP currently supports only the local Paperclip host, but this run targets a remote environment." The ACP engine only knew how to spawn a local subprocess, while sandbox providers only expose one-shot command execution — so there was no way to hold the bidirectional stdio session ACP requires. **Proposed solution** Add a process-session bridge in `adapter-utils`: a local ACPX-spawnable proxy script connects to a token-authenticated loopback TCP server, which relays JSON-framed stdin/stdout/stderr events to and from a small relay script executed inside the sandbox via the provider's ordinary runner. Claude/Codex/Gemini adapters now treat sandbox targets with a runner as ACP-capable, resolve agent commands against the remote target, and fall back to CLI only when the sandbox exposes no bidirectional path. The sandbox callback bridge injects a run-scoped API endpoint and bridge token so the agent inside the sandbox can reach Paperclip (including work-product handoffs) without ever receiving the host run JWT. **Alternatives considered** A provider-specific lane was prototyped first: Daytona minting SSH access metadata at lease time, converted into an SSH execution target. It was dropped because it only worked for providers able to advertise SSH, added per-provider surface area, and left every other sandbox provider on the CLI fallback. The merged design rides the one-shot runner contract all providers already implement; a regression test pins that sandbox targets stay on the bridge lane even when lease metadata advertises SSH access. **Roadmap alignment** Directly advances the "Cloud / Sandbox agents" roadmap item — agents running in remote and sandboxed environments keep the same control-plane behavior as local ones. No overlap with other planned core work. ## What Changed - `packages/adapter-utils/src/execution-target.ts`: new `startAdapterExecutionTargetProcessSessionBridge()` plus helpers — writes a token-authenticated local proxy script (spawnable by ACPX) and a remote relay script synced into the sandbox, with a loopback TCP server streaming JSON-framed stdio between them; events emitted before the ACP client attaches are buffered so none are lost. - `packages/adapter-utils/src/acpx-engine/execute.ts`: the ACP engine can execute against remote sandbox targets through the bridge instead of requiring a local subprocess, including remote cwd/env shaping. - `packages/adapter-utils/src/sandbox-callback-bridge.ts`: sandbox-scoped API bridging extended to allow work-product handoffs; the sandbox payload env carries a bridge token, never the host run JWT. - `packages/adapters/claude-local`, `codex-local`, `gemini-local` (`src/server/acp.ts`): default-lane selection no longer rejects all remote targets; command resolution is remote-aware (`ensureAdapterExecutionTargetCommandResolvable`, `resolveAdapterExecutionTargetCwd`); the fallback reason is now scoped to sandboxes that expose only one-shot execution. - `server/src/__tests__/environment-execution-target.test.ts`: pins that sandbox targets resolve to the bridge lane, including when lease metadata advertises SSH access. - Non-sandbox remote targets (e.g. SSH) keep the CLI lane: the ACP engine's remote transport is sandbox-only, so default-lane selection falls back for those targets across all three adapters, and tests covering CLI-specific remote behavior pin `engine: "cli"` explicitly. - The bridge authenticates loopback connections before they can own the session or receive buffered output (token required, idle unauthenticated peers dropped), and remote event writes are serialized so the exit event always lands after stdout/stderr have drained. - Daytona plugin: formatting-only residue from the earlier iteration; no functional change. ## Verification - `vitest run` over the touched suites — `packages/adapter-utils/src/acpx-engine/execute.test.ts`, `packages/adapter-utils/src/execution-target-sandbox.test.ts`, `packages/adapter-utils/src/sandbox-callback-bridge.test.ts`, the three adapter `acp.test.ts` files, and `server/src/__tests__/environment-execution-target.test.ts` — 102 tests pass. - End to end: with a Claude agent configured on a Daytona sandbox environment, the primary-model test now selects the default ACP lane (no fallback warning), and the full round trip (wake → sandbox execution → API bridge → comment post) was exercised twice from inside a live sandbox. ## Risks - Behavioral shift: adapters that previously always fell back to CLI on sandbox targets now default to ACP there; `engine=cli` still pins the CLI lane explicitly. - The bridge relays stdio as JSON lines over loopback TCP guarded by a per-session random token; the remote relay runs inside the sandbox under the provider's runner. Providers with slow one-shot execution will see higher session startup latency — the CLI fallback remains for genuinely incapable targets. - No schema or migration changes. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic) — extended thinking enabled, agentic tool use via the Claude Agent SDK harness; implementation iterated with local Vitest verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no shipped docs describe the old local-only ACP limitation) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Cody <noreply@paperclip.ing> Co-authored-by: Cody <cody@paperclip.local> |
||
|
Devin FoleyandPaperclip
|
486fb88a15 |
Add Cloudflare sandbox provider plugin (#5687)
> _Stacked on top of #5685 → #5686. Diff against master includes commits from earlier PRs in the stack — review focuses on the two new commits (`Extend sandbox callback bridge for Worker-hosted plugins` + `Add Cloudflare sandbox provider plugin`)._ ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Each agent runs in a sandbox environment, and operators choose which provider backs that sandbox — today E2B and Daytona are bundled with the platform > - Cloudflare Workers + Durable Objects + the Sandbox SDK offer a credible new option: globally distributed, cheap idle, and operator-deployable as a single Worker > - To plug it in, Paperclip needs (a) a provider plugin that speaks the `PaperclipPluginManifestV1` lifecycle and (b) a small operator-deployed Worker — the **bridge** — that adapts Paperclip's runtime RPCs to the Cloudflare Sandbox SDK > - The plugin extends the existing sandbox-callback-bridge with a `bridge.transport: "worker"` discriminator so the platform routes runtime RPCs through the Worker bridge instead of the in-process runner > - This pull request adds the plugin, the bridge Worker template, and the supporting adapter-utils + server hooks the new transport needs > - The benefit is that operators can run sandboxes on Cloudflare's edge with no new platform code beyond installing the plugin and deploying the Worker ## What Changed **Shared support (`Extend sandbox callback bridge for Worker-hosted plugins`):** - `packages/adapter-utils/src/sandbox-callback-bridge.{ts,test.ts}`: expose `expectedHostHeader` so plugin-side bridge clients can verify the canonical request envelope before forwarding. - `packages/adapter-utils/src/command-managed-runtime.{ts,test.ts}`: relax the always-fresh runner construction so callers can re-use a runner across exec calls (Worker-hosted bridges hold the runner inside a Durable Object). - `server/src/services/environment-runtime.ts` + `environment-runtime.test.ts`: route Worker-hosted bridges through the same env-shaping path as E2B and pin the `requestEnv` contract. - `server/src/services/plugin-environment-driver.ts`: thread an optional `issueId` through the runtime descriptor so bridges can scope leases to the originating issue (used by Cloudflare to map a sandbox to the issue/workflow for billing and audit). - `packages/plugins/sdk/src/protocol.ts`: add `issueId?` to `PluginEnvironmentDriverBaseParams` and the new `bridge.transport: "worker"` discriminator that the new plugin declares. - `server/__tests__/heartbeat-plugin-environment.test.ts`: pin the heartbeat path against the new runtime descriptor. **The Cloudflare plugin itself (`Add Cloudflare sandbox provider plugin`):** - `packages/plugins/sandbox-providers/cloudflare/`: plugin entry, manifest, plugin runtime (lifecycle + bridge client), config parsing, and Vitest coverage. Manifest declares `bridge.transport: "worker"` so the platform routes runtime RPCs through the bridge client. - `bridge-template/`: a Worker template the operator deploys with `wrangler`. Owns Durable Object-backed sessions (`sessions.ts`), exec/stream routes (`exec.ts`, `routes.ts`), and an HMAC auth layer (`auth.ts`) that pins the `Host` header surface. Includes the SDK-contract-correct exec implementation, lease recovery, and chunked stdout/stderr streaming. - Tests cover lease/session handoff (`bridge-template/src/exec.test.ts`, `routes.test.ts`), bridge client request shaping (`src/bridge-client.test.ts`), and end-to-end plugin behavior (`src/plugin.test.ts`) including streamed exec output. 27 tests in total. - `README.md` walks the operator through deploying the bridge Worker, registering the plugin, and configuring the runtime. ## Verification - `pnpm typecheck` - `pnpm exec vitest run --no-coverage packages/adapter-utils/src/sandbox-callback-bridge.test.ts packages/adapter-utils/src/command-managed-runtime.test.ts server/src/__tests__/environment-runtime.test.ts server/src/__tests__/heartbeat-plugin-environment.test.ts` - `(cd packages/plugins/sandbox-providers/cloudflare && pnpm test)` — 27 passing For an operator-side smoke test: 1. Deploy the bridge: `cd packages/plugins/sandbox-providers/cloudflare/bridge-template && wrangler deploy` 2. Register the plugin in your Paperclip instance, point its bridge URL at the deployed Worker, set the HMAC shared secret. 3. Create a sandbox environment whose provider is `cloudflare`, then run a Codex or Claude job against it. ## Risks - Adds a new `bridge.transport: "worker"` code path, but the existing E2B / Daytona transports go through the same shaped helpers and have explicit test coverage that pins their behavior unchanged. - The Worker bridge stores session state in a Durable Object; operator instances must be aware of the corresponding Cloudflare costs (DO requests, storage). Documented in the README. - The `issueId` plumbing is optional throughout — existing plugins that don't supply it continue to work. ## Model Used - Provider: Anthropic - Model: Claude Opus 4.7 (1M context) - Capabilities used: extended reasoning, tool use (Read/Edit/Bash/Grep) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots — N/A, no UI change - [x] I have updated relevant documentation to reflect my changes (plugin README, bridge-template README) - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
778e775c35 |
Add secrets provider vaults and remote import (#5429)
## Thinking Path > - Paperclip orchestrates AI-agent companies and needs secrets handling to work across local development, hosted operators, and governed agent execution. > - The affected subsystem is the company-scoped secrets control plane: database schema, server services/routes, CLI workflows, and the Secrets settings UI. > - The gap was that secrets were local-only and operators could not manage provider vaults or import existing remote references without exposing plaintext. > - This branch adds provider vault configuration plus an AWS Secrets Manager remote-import path while preserving company boundaries, binding context, and audit trails. > - I kept the PR to a single branch PR, removed unrelated lockfile/package drift, rebased the full branch onto the current `public-gh/master`, and addressed fresh Greptile findings. > - The benefit is a reviewable implementation of provider-backed secrets with focused tests covering provider selection, import conflicts, deleted secret reuse, rotation guards, and AWS signing behavior. ## What Changed - Added provider vault support for company secrets, including provider config storage, default vault handling, health checks, binding usage, access events, and remote import preview/commit. - Added an AWS Secrets Manager provider using SigV4 request signing, bounded request timeouts, namespace guardrails, cached runtime credential resolution, and external-reference linking without plaintext reads. - Added Secrets UI surfaces for vault management and remote import, plus CLI/API documentation for setup and operations. - Stabilized routine webhook secret binding paths and SSH environment-driver fixture bindings discovered during verification. - Addressed Greptile and CI findings: no lockfile/package drift, monotonic migration metadata, disabled-vault default races, soft-deleted secret hiding/recreate behavior, remove behavior with disabled vaults, soft-deleted external-reference re-import, non-active rotation guards, managed-secret soft deletion through PATCH, and per-call AWS SDK credential client churn. - Rebased this branch onto `public-gh/master` at `0e1a5828` and force-pushed with lease to keep this as the single PR for the branch. ## Verification - `git fetch public-gh master` - `git rebase public-gh/master` - `git diff --name-only public-gh/master...HEAD | grep '^pnpm-lock\.yaml$' || true` confirmed `pnpm-lock.yaml` is not in the PR diff. - Confirmed migration ordering: master ends at `0081_optimal_dormammu`; this PR adds `0082_dry_vision` and `0083_company_secret_provider_configs`. - Inspected migrations for repeat safety: new tables/indexes use `IF NOT EXISTS`; foreign keys are guarded by `DO $$ ... IF NOT EXISTS`; column additions use `ADD COLUMN IF NOT EXISTS`. - `pnpm -r typecheck` passed before the Greptile follow-up commits. - `pnpm test:run` ran the full stable Vitest path before the Greptile follow-up commits; it completed with 3 timing-related failures under parallel load: `codex-local-execute.test.ts`, `cursor-local-execute.test.ts`, and `environment-service.test.ts`. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/codex-local-execute.test.ts src/__tests__/cursor-local-execute.test.ts src/__tests__/environment-service.test.ts` passed on targeted rerun (`24/24`). - `pnpm build` passed before the Greptile follow-up commits. Vite reported existing chunk-size/dynamic-import warnings. - After Greptile follow-up commits: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/secrets-service.test.ts` passed (`26/26`). - After Greptile follow-up commits: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/aws-secrets-manager-provider.test.ts src/__tests__/secrets-service.test.ts` passed (`39/39`). - After Greptile follow-up commits: `pnpm --filter @paperclipai/server typecheck` passed. - Captured Storybook screenshots from `ui/storybook-static` for visual review. - Latest PR checks on `5ca3a5cf`: `policy`, serialized server suites 1/4-4/4, `Canary Dry Run`, `e2e`, `security/snyk`, and `Greptile Review` pass; aggregate `verify` is still registering the completed child checks. - Greptile review loop continued through the latest requested pass; all Greptile review threads are resolved and the latest `Greptile Review` check on `5ca3a5cf` passed with 0 comments added. ## Screenshots Before: the provider-vault and remote-import surfaces did not exist on `master`; these are after-state screenshots from the Storybook fixtures.    ## Risks - Migration risk: this adds new secret provider tables and extends existing secret rows. The migrations were checked for monotonic ordering and idempotent guards, but reviewers should still inspect upgrade behavior carefully. - Provider risk: AWS support uses direct SigV4 requests. Automated tests cover signing, request timeouts, vault-config selection, namespace guardrails, pending-version archival, sanitized provider errors, and service-level cleanup paths. A real-vault AWS smoke test remains deployment validation for an operator with AWS credentials rather than an unverified merge blocker in this local branch. - UI risk: the Secrets page and import dialog are large new surfaces; screenshots are included above for reviewer inspection. - Verification risk: the full local stable test command hit parallel-load timing failures, although the exact failed files passed when rerun directly. - Operational risk: remote import intentionally avoids plaintext reads; operators must understand that imported external references resolve at runtime and may fail if AWS permissions change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent with local shell/tool use in the Paperclip worktree. Exact context-window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Devin Foley
|
fe3904f434 |
Stabilize runtime probes and Codex env tests (#5445)
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies
> - Adapters expose a Test action that probes the configured runtime —
install, resolvability, hello — to give operators a fast yes/no on
whether an environment is healthy
> - The Codex test path was running its hello probe directly without
going through the managed-runtime preparation that production runs use,
so a healthy production setup could still report a probe failure
> - The plugin worker manager wasn't surfacing terminated workers
cleanly, leaving the runtime probe waiting on a dead worker until the
request timed out
> - This pull request routes the Codex test probe through
`prepareAdapterExecutionTargetRuntime` (so it sees the same managed
Codex home production sees), exposes `commandCwd` on
`createCommandManagedRuntimeClient` so callers can target a per-probe
directory without leaking the workspace `remoteCwd`, and propagates
plugin-worker termination as a usable error instead of a hang
> - The benefit is the Codex Test action mirrors production behavior
end-to-end, and probes against a terminated plugin worker fail fast
instead of timing out
## What Changed
- `packages/adapter-utils/src/command-managed-runtime.ts`: rename the
`remoteCwd` knob to `commandCwd` so callers can target a per-probe
directory without inheriting the workspace cwd; matching test coverage
in `command-managed-runtime.test.ts`
- `packages/adapter-utils/src/sandbox-callback-bridge.{ts,test.ts}`:
small fixes to keep callback bridge stop semantics deterministic
- `packages/adapters/codex-local/src/server/test.ts`: thread the Codex
hello probe through `prepareAdapterExecutionTargetRuntime` +
`prepareManagedCodexHome` so the probe sees the same managed home
production sees; new `test.remote.test.ts` covers the remote probe path
- `packages/adapters/cursor-local/src/server/execute.ts`: small
probe-side cleanup that aligns with the new commandCwd contract
- `server/src/services/plugin-worker-manager.ts`: surface plugin-worker
termination as a structured error so callers fail fast; new
`plugin-worker-terminated.cjs` fixture and
`plugin-worker-manager.test.ts` cases pin the behavior
## Verification
- `pnpm vitest run --no-coverage --project @paperclipai/adapter-utils
--project @paperclipai/adapter-codex-local --project
@paperclipai/adapter-cursor-local --project @paperclipai/server` —
1749/1750 passing (1 unrelated skip)
- `pnpm typecheck` clean
## Risks
Low–medium. The `remoteCwd → commandCwd` rename is a parameter renaming
on an internal helper used only by adapter test/execute paths in this
repo. The plugin-worker-terminated path was previously a hang; failing
fast may surface latent timeouts as explicit termination errors in
callers that already expected them.
## Model Used
Claude Opus 4.7 (1M context)
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable — new tests cover
commandCwd, plugin-worker termination, and Codex remote test path
- [x] If this change affects the UI, I have included before/after
screenshots — N/A (no UI)
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
---
> **Stacked PR.** Sits on top of #5444 which adds the per-run runtime
API surface this PR builds on. Cumulative diff against `master` includes
that PR's content; the files touched by *this* PR's commit are listed
under "What Changed" above. Will rebase onto `master` and force-push
once #5444 merges.
|
||
|
Devin Foley
|
50db8c01d2 |
Serialize sandbox callback bridge against concurrent heartbeats (#5326)
> **Stacked PR.** This PR's branch carries cumulative content from #5324 (bridge allowlist expand) and #5325 (env sanitization) — the mutex/sha256 logic in this PR sits on top of both. Reviewers should focus on the files this PR's commit touches: `packages/adapter-utils/src/sandbox-callback-bridge.{ts,test.ts}`, `packages/adapter-utils/src/ssh.ts`, and `packages/adapter-utils/src/ssh-fixture.test.ts`. Will rebase onto `master` and force-push once both prerequisite PRs are merged. ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Each agent that runs in a sandbox or via SSH talks back to the Paperclip server through a per-lease callback bridge whose entrypoint script is uploaded to the remote > - When two heartbeats target the same agent on the same machine concurrently, both upload the bridge entrypoint and both write to the same response files — producing torn-write races: `SyntaxError: Identifier 'randomUUID' has already been declared` from a concatenated upload, `mv: cannot stat …` from colliding `.json.tmp` writes, and 0-byte commits from a truncated stdin > - This pull request serializes those operations with a POSIX `mkdir`-mutex (PID liveness check + atomic rename) at the bridge entrypoint upload, applies the same lock to the bridge response writer, forwards stdin into remote ssh commands so the entrypoint payload arrives intact, and verifies a sha256 of the upload before promoting it > - The benefit is concurrent heartbeats no longer corrupt each other's bridge state ## What Changed - `packages/adapter-utils/src/sandbox-callback-bridge.ts`: serialize entrypoint upload and response writes via POSIX `mkdir`-mutex with PID liveness; sha256 the upload before promoting via `mv`; content-skip when the existing entrypoint already matches - `packages/adapter-utils/src/ssh.ts`: forward stdin into remote ssh commands through the SSH managed runtime so `cat > "$remote_upload"` actually receives the base64-encoded entrypoint - `packages/adapter-utils/src/ssh-fixture.test.ts`: cover the stdin-forwarded SSH path - `packages/adapter-utils/src/sandbox-callback-bridge.test.ts`: cover the mutex, content-skip, sha256-verify, and atomic-rename paths ## Verification - `pnpm vitest run --no-coverage --project @paperclipai/adapter-utils` - `pnpm typecheck` clean - Manual: two parallel heartbeats targeting the same SSH agent no longer race on the bridge entrypoint or response files ## Risks Medium. Serializing previously-parallel operations adds latency on the contended path (one heartbeat waits on another), bounded by the entrypoint upload time. The mutex includes PID liveness so a crashed heartbeat doesn't deadlock subsequent ones. Sha256-verify gives a clear "torn upload" failure mode instead of silent 0-byte commits. ## Model Used Claude Opus 4.7 (1M context) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable — tests cover mutex + sha256-verify + stdin-forwarded ssh - [x] If this change affects the UI, I have included before/after screenshots — N/A (no UI) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Devin Foley
|
36eaf9778f |
Expand sandbox callback bridge allowlist to cover the documented heartbeat surface (#5324)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - When an agent runs in an e2b sandbox or other non-managed environment, it talks back to the Paperclip server through a per-lease callback bridge that proxies HTTP requests > - The bridge has an allowlist of method/path patterns it will forward; anything outside the list is rejected to keep the bridge tight > - The allowlist had drifted behind what the heartbeat documentation describes as the supported callback surface — several documented endpoints (issue updates, agent-side log emit, work-status writes) were being rejected at the bridge > - This pull request expands the allowlist to cover the documented heartbeat surface and adds tests that pin every newly-allowed pattern, so the doc and the bridge stay in sync > - The benefit is sandboxed runs no longer hit "method not allowed" / "path not allowed" rejections on the documented set of callbacks ## What Changed - `packages/adapter-utils/src/sandbox-callback-bridge.ts`: expand the method/path allowlist to match the documented heartbeat callback surface - `packages/adapter-utils/src/sandbox-callback-bridge.test.ts`: add coverage for every newly-allowed pattern, plus negative cases for patterns that should still be rejected ## Verification - `pnpm vitest run --no-coverage --project @paperclipai/adapter-utils` - `pnpm typecheck` clean - Manual: previously-rejected callbacks from sandboxed runs now succeed end-to-end ## Risks Low. The allowlist only grows; nothing previously allowed is now blocked. Tests pin both the new allowed patterns and that out-of-doc patterns stay rejected. ## Model Used Claude Opus 4.7 (1M context) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable — new tests cover added patterns + still-rejected negatives - [x] If this change affects the UI, I have included before/after screenshots — N/A (no UI) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Devin Foley
|
a7b45938b7 |
Let sandbox providers declare shell defaults (#5114)
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies
> - Agents execute in sandboxed remote environments served by pluggable
sandbox
> providers (E2B today, more later)
> - Today every sandbox command runs under `sh -lc` regardless of what
the
> provider's container actually ships
> - That misses bash-only shell init on E2B (which ships bash) and
prevents
> future providers from declaring a different default — there's no way
for a
> provider to say "I have bash, use it"
> - This PR adds a `shellCommand` field to sandbox execution targets so
providers
> can declare their preferred shell ("bash" for E2B), threads it through
the
> sandbox-managed-runtime client, callback bridge, and execution-target
shell
> helper, and validates the value at the lease-metadata boundary
> - The benefit is that sandbox commands run under the right shell on
the right
> provider, and adding new sandbox providers only needs to declare a
shell
> preference
## What Changed
- Added `packages/adapter-utils/src/sandbox-shell.ts` exporting
`preferredShellForSandbox(shellCommand)` (returns `"bash"` if input is
`"bash"`,
else `"sh"`)
- Added `shellCommand?: "bash" | "sh" | null` to
`AdapterSandboxExecutionTarget`
and `CommandManagedRuntimeSpec`; threaded it through
`runAdapterExecutionTargetShellCommand`,
`prepareAdapterExecutionTargetRuntime`,
and `startAdapterExecutionTargetPaperclipBridge`
- `createCommandManagedRuntimeClient`, `prepareCommandManagedRuntime`,
and
`createCommandManagedSandboxCallbackBridgeQueueClient` now take an
optional
`shellCommand` and use `preferredShellForSandbox` to pick the shell
- `startSandboxCallbackBridgeServer` accepts a `shellCommand` for its
server
startup, readiness probe, and stop hook
- E2B sandbox plugin declares `shellCommand: "bash"` in `leaseMetadata`
- `resolveEnvironmentExecutionTarget` reads `shellCommand` from lease
metadata
(validating against `"bash" | "sh" | null`)
- `environment-runtime.ts` adds `"shellCommand"` to
`INTERNAL_PLUGIN_SANDBOX_CONFIG_KEYS`
so the field round-trips through internal plugin config without leaking
to
external plugin metadata
- Updated tests in `command-managed-runtime.test.ts`,
`execution-target-sandbox.test.ts`, `sandbox-callback-bridge.test.ts`,
`environment-execution-target.test.ts`
## Verification
- `pnpm --filter @paperclipai/adapter-utils test`
- `pnpm --filter @paperclipai/server test --
environment-execution-target`
- `pnpm --filter @paperclipai/sandbox-providers-e2b test`
- Manual QA: boot a Paperclip instance, create an E2B-backed
environment, run a
claude_local agent against it, and confirm the run completes (verifies
bash
shell semantics flow through the callback bridge end-to-end)
## Risks
- E2B sandbox commands now run under `bash -lc` instead of `sh -lc`.
Bash is a
strict superset for the commands we issue (no busybox-only flags in our
shell
scripts), so risk is low. The shellCommand field is opt-in via lease
metadata —
providers that don't declare it stay on `sh`.
- New optional field on `CommandManagedRuntimeSpec` and
`AdapterSandboxExecutionTarget`.
Consumers ignoring the field retain previous behaviour (sh).
- Lease metadata now carries an additional field. Existing leases
without
`shellCommand` resolve to `null` and fall back to sh — backwards
compatible.
## Model Used
- OpenAI GPT-5.4 (reasoning effort: high) via Codex CLI
- Provider: OpenAI
- Used to author the code changes in this PR
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots — N/A (no UI changes)
- [ ] I have updated relevant documentation to reflect my changes — N/A
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
|
||
|
Devin Foley
|
a4ac6ff133 |
Add sandbox callback bridge for remote environment API access (#4801)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Agents can run inside sandboxed environments like E2B, which are isolated from the host network > - Sandboxed agents need to call back to the Paperclip API to report progress, post comments, and update issue status > - But sandbox environments cannot reach the Paperclip server directly because they run in isolated network namespaces > - This PR adds a callback bridge that proxies API requests from the sandbox to the Paperclip server, running as a local HTTP server on the host that forwards authenticated requests > - The bridge is started automatically when an adapter launches a sandbox execution, and torn down when the run completes > - The benefit is sandboxed agents can interact with the Paperclip API without requiring network-level access to the host, enabling E2B and similar providers to work end-to-end ## What Changed - Added `sandbox-callback-bridge.ts` in `packages/adapter-utils/` — a lightweight HTTP bridge server that accepts requests from sandbox environments and proxies them to the Paperclip API with authentication - Added request validation and security policy: the bridge only forwards requests to the configured API URL, validates content types, enforces size limits, and rejects non-API paths - Wired the bridge into all remote adapter execute paths (claude, codex, cursor, gemini, pi) — the bridge starts before the agent process and the bridge URL is passed via environment variables - Updated `environment-execution-target.ts` to prefer the explicit API URL from environment lease metadata for sandbox callback routing - Fixed Claude sandbox runtime setup to work with the bridge configuration - Added comprehensive test coverage for bridge request handling, policy enforcement, and sandbox execution integration - Fixed browser bundling — the bridge module is excluded from the frontend bundle via the adapter-utils index export ## Verification - `pnpm test` — all existing and new tests pass, including bridge unit tests and sandbox execution integration tests - `pnpm typecheck` — clean - Manual: configure an E2B environment, run an agent task, verify the agent can post comments and update issue status through the bridge ## Risks - Medium. This is a new network-facing component (HTTP server on localhost). The security policy restricts forwarding to the configured API URL only and validates all requests, but any proxy introduces attack surface. The bridge binds to localhost only and is scoped to the lifetime of a single agent run. ## Model Used Codex GPT 5.4 high via Paperclip. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
