384 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	2568bdecc4	fix(adapter-utils): keep sandbox proxy sockets within Linux path limit (#10221) ... ## Thinking Path > - Paperclip is the open source control plane people use to manage AI agents for work > - Local process adapters can enforce network allowlists through a Unix-socket proxy inside the Linux sandbox > - The proxy socket was created under `os.tmpdir()`, which can be a deeply nested run-specific directory > - Linux Unix-domain socket paths are limited to 107 usable bytes, so deep temporary paths can be silently truncated during bind > - A truncated proxy socket resets every confined outbound connection, including the model provider API, before the agent can do work > - This pull request creates proxy sockets under the short `/tmp` path when possible and validates the path length before bind > - The benefit is reliable sandboxed egress under deep `TMPDIR` values and an explicit error instead of an opaque connection-reset storm ## Linked Issues or Issue Description ### Pre-submission checklist - [x] I searched existing open and closed issues and this is not a duplicate. - [x] I can reproduce this on upstream `master`. - [x] I confirmed the error originates in Paperclip's local-process sandbox proxy. ### What happened? Sandboxed local-process agents using `networkScope: "allowlist"` could lose all proxied egress when `TMPDIR` was deeply nested because `proxy.sock` exceeded Linux's Unix socket path limit. The truncated bind surfaced as repeated connection resets, including for provider API traffic. ### Expected behavior Proxy creation should use a Linux-safe short path and fail explicitly if no safe socket path can be created. ### Steps to reproduce 1. Set `TMPDIR` to a path long enough that `<TMPDIR>/paperclip-network-sandbox-XXXXXX/proxy.sock` exceeds 107 bytes. 2. Build or run a local-process sandbox with `networkScope: "allowlist"`. 3. Make an HTTP or HTTPS request through the generated sandbox proxy. 4. Observe connection resets from the overlong Unix socket path on affected code. ### Paperclip version or commit Upstream `master` before this change. ### Deployment mode Other — Linux local-process adapters using Bubblewrap network allowlisting. ### Installation method Built from source. ### Agent adapter(s) involved Codex and any other local-process adapter using the shared sandbox utility. ### Operating system Linux. ### Additional context GitHub search found no duplicate public issue or pull request for this defect. ## What Changed - Add a Linux Unix-socket byte-length guard that reports the unsafe path before binding. - Create network proxy temporary directories under `/tmp` first, with `os.tmpdir()` as a validated fallback. - Use the safe temporary-directory helper at both allowlist proxy creation sites while preserving trusted URL handling. - Add regression coverage that sets a deliberately deep `TMPDIR` and verifies the working socket remains within 107 bytes under `/tmp`. ## Verification - `node /srv/paperclip/home/paperclipai/paperclip/node_modules/vitest/vitest.mjs run packages/adapter-utils/src/local-process-sandbox.test.ts` — 8 passed, 4 environment-gated tests skipped. - `git diff --check origin/master...HEAD` — passed. - Greptile Review — passed after reviewing 2 files with 0 comments and no unresolved threads; this repository integration did not emit a separate numeric confidence score. - Full GitHub CI matrix — passed after one rerun of an unrelated ACPX `ENOTEMPTY` cleanup flake. - Package typecheck was attempted, but the existing shared install cannot resolve `acpx/runtime`; the failure is outside the changed files and is expected to be covered by CI's clean dependency install. ## Risks - Low risk: the change is limited to Linux sandbox proxy temporary-directory selection and validation. - Systems without a writable `/tmp` fall back to `os.tmpdir()` only when the resulting socket path is safe; otherwise startup now fails loudly instead of producing connection resets. - No schema, API, UI, migration, or documentation behavior changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, `gpt-5.3-codex`, tool-enabled coding agent with managed reasoning and code execution; context-window size is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-26 10:52:16 -05:00
DottaandPaperclip	4e00818574	fix(runtime): support in-place workspace realization (#10230) ... ## Thinking Path > - Paperclip is the control plane people use to coordinate AI agents and their execution environments. > - Environment realization decides where an agent runs and which filesystem and toolchain are authoritative. > - Copy-based realization is unsafe for container-anchored tasks because absolute paths such as `/app` can point outside the synchronized tree and task-specific binaries may be absent. > - That mismatch can let an agent successfully verify work in a phantom writable path while sync-back silently discards the result. > - Existing task environments already provide the authoritative filesystem and toolchain, so they should be executed in place rather than copied. > - Copy mode still needs explicit confinement rules so aliases target the synchronized workspace and unsynchronized writable paths fail visibly. > - This pull request adds typed realization metadata, propagates the authoritative root through orchestration, and teaches Codex to honor it. > - The benefit is that container-anchored tasks operate on verifier-visible state with the intended tools, while copy mode remains safe and backward compatible. ## Linked Issues or Issue Description No public GitHub issue exists for this defect. GitHub duplicate searches for in-place execution, workspace realization, and authoritative workspace roots found no related pull request to link. ### What happened? Environment-backed agent runs were always realized through a copied workspace. Tasks anchored to absolute container paths could therefore write outside the synchronized tree, and task-provided toolchains were unavailable in the copy. A run could report success even though sync-back discarded its output. ### Expected behavior Existing task environments should run against their real authoritative root and toolchain. Copy-mode runs should map declared absolute aliases into the synchronized tree and reject writable paths that cannot be restored. ### Steps to reproduce 1. Run a Codex task environment whose required files live under `/app` or `/workspace` and whose required binary exists only in the task container. 2. Observe that copy realization changes the effective filesystem/toolchain or permits writes outside the synchronized root. 3. Complete and verify the task inside the agent sandbox. 4. Observe that the verifier cannot see out-of-tree artifacts or that task-specific commands were unavailable. ### Reproduction context - Paperclip commit: `3a16b91217483d2c233926de5b7f7bc3a1077924` - Deployment: built from source in a task-container execution environment - Adapter: Codex local - Database: not database-related - Access context: agent execution ## What Changed - Added typed `copy | in_place` workspace-realization metadata, authoritative roots, confined aliases, and outbound restore paths to shared execution-target contracts. - Selected in-place realization for existing task environments and skipped archive prepare/restore when the authoritative environment is used directly. - Propagated the authoritative root into adapter context so Codex uses it for cwd and `PAPERCLIP_WORKSPACE_*` semantics, including ACP execution. - Bound copy-mode aliases such as `/app` to the synchronized workspace and rejected writable out-of-tree paths without explicit restore mappings. - Added focused regression coverage while preserving existing copy-mode archive restore behavior. ## Verification - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm exec vitest run packages/adapter-utils/src/local-process-sandbox.test.ts packages/adapters/codex-local/src/server/acp.test.ts packages/adapters/codex-local/src/server/execute.remote.test.ts server/src/__tests__/environment-run-orchestrator.test.ts` — 48 passed, 4 skipped. - `pnpm -r typecheck` — passed. - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY -u AWS_SESSION_TOKEN pnpm test:run` — passed across all general and serialized Vitest shards. - `pnpm build` — passed. - Codex `k=1` acceptance run completed July 24, 2026 at 23:54:30 UTC with 4 completed, 0 exceptions, and mean reward 1.0: `build-cython-ext`, `openssl-selfsigned-cert`, `prove-plus-comm`, and `sqlite-db-truncate` each received terminal grade 1.0 against real task-environment paths and toolchains. ## Risks - In-place mode deliberately exposes the authoritative task root to the adapter; incorrect environment metadata could point execution at the wrong root. Typed metadata and focused orchestration tests cover selection and propagation. - Copy-mode writable-path validation is stricter and may reject previously accepted unsafe configurations. The rejection is intentional and produces a visible error instead of silently losing output. - The acceptance run is focused on four Codex task-environment workloads, not a broad cross-adapter benchmark. Existing copy-mode archive tests and the full repository suite remain green. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex CLI coding agent; exact model ID and context-window size were not exposed to this runtime. Capabilities used: extended reasoning, repository editing, shell execution, test/build execution, Git, GitHub CLI, and Paperclip API tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-25 08:29:26 -05:00
DottaandClaude Fable 5	b996b71a38	Deduplicate wake-payload issue descriptions and compact resume deltas (#10216) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Heartbeat wake payloads and the task-context markdown are the two channels that deliver an issue's brief into an agent's prompt > - #10151 fixed wake-prompt-only adapter lanes waking without the issue description by adding it to the structured wake payload > - That left the description delivered twice per prompt on lanes that also inject the task-context markdown, and re-delivered in full on every resume wake, permanently bloating persistent-session context > - This pull request makes the task markdown the single description carrier on lanes that use it, and omits the description from non-assignment resume deltas on all lanes while keeping it for assignment-shaped and recovery wakes > - The benefit is that every lane receives the brief exactly once when it needs it, and long-lived sessions stop re-paying the full brief in tokens on every wake ## Linked Issues or Issue Description Refs #10151 Related prior work: #2883, #8402 (earlier description-delivery attempts referenced by #10151). I searched the PR list for open work on wake-payload description handling and found none besides the merged #10151. **Bug:** After #10151, adapters that inject the `Paperclip task context` markdown (ACPX engine lanes, claude-local CLI, hermes server and gateway) receive the issue description twice in a single prompt — once in the wake prompt's `Issue description:` block and once in the task markdown. Separately, resume deltas re-send the full description (up to 12k characters) on every wake even though the persistent session already received it. **Expected behavior:** The description appears exactly once per prompt on every lane, and resume deltas only carry it when the resuming session may not have seen the brief (assignment-shaped or recovery wakes), leaving an explicit fetch breadcrumb otherwise. **Reproduction:** Wake a claude-local or ACPX agent on an issue with a description and inspect the assembled prompt: the description text appears in both the wake-payload block and the task-context block. Wake the same session again via a comment: the full description is present again in the resume delta. **Affected version:** Current `master` (with #10151 merged). **Deployment mode:** Adapter-backed heartbeat execution, local and sandboxed lanes. ## What Changed - `renderPaperclipWakePrompt` accepts `suppressIssueDescription`; the four task-markdown lanes pass it so the task markdown stays the single, uncapped description carrier there. - Non-assignment resume deltas omit the description and emit `- issue description: omitted from this resume delta; fetch the issue if you need the latest brief`. Assignment-shaped reasons (`issue_assigned`, `issue_reopened_via_comment`, `issue_recovery_action_restored`, `issue_tree_restored`) and recovery wakes still deliver the full brief. - `buildPaperclipTaskMarkdown` gains `includeDescription`; the server now also publishes `context.paperclipTaskMarkdownCompact` (description stripped, directives and wake comment kept), and the new `selectPaperclipTaskMarkdown` helper picks the right variant under the same resume rules, falling back to the full markdown when no compact variant exists (version skew safety). - The wake prompt's description block now carries the same user-authored trust framing the task markdown already had. ## Verification - `npx vitest run packages/adapter-utils/src/server-utils.test.ts packages/adapters/claude-local/src/server/acp.test.ts packages/adapters/codex-local/src/server/acp.test.ts server/src/__tests__/heartbeat-context-summary.test.ts` — 137 tests passed, including new coverage for suppression, resume omission plus breadcrumb, assignment-shaped resume inclusion, compact-variant building, variant selection, and an end-to-end ACPX prompt-assembly test asserting the description appears exactly once on fresh wakes and not at all on comment resumes. - `npx vitest run` in `packages/adapters/hermes` — 59 tests passed, including a gateway execute-level test asserting the brief is sent exactly once on fresh runs and not re-sent on stable-session resumes. - `tsc --noEmit` in `packages/adapter-utils`, `packages/adapters/claude-local`, `packages/adapters/hermes` — clean; `server` matches the `master` baseline exactly (pre-existing plugin-sdk resolution errors only, none in touched files). - Pre-existing failures confirmed identical on clean `master`: claude-local `execute.remote.test.ts` / `test.probe.test.ts`, adapter-utils `mcp-isolation.integration.test.ts` (requires a newer local Claude CLI). ## Risks - Behavioral shift, prompt-only: a resumed session woken by a comment on an issue it never handled (rare — assignment wakes normally precede comment wakes) would not get the inline description; the breadcrumb plus the standard issue-fetch path covers it. - Additive context key (`paperclipTaskMarkdownCompact`); older adapters ignore it and newer adapters fall back to the full markdown when it is absent, so mixed-version deployments degrade to current behavior. - No schema, migration, or API changes; the structured wake-payload JSON shape is unchanged. - Known follow-up deliberately out of scope: openclaw embeds the raw wake-payload JSON (which still contains the description) in prompt text for machine parsing. The hermes-gateway lane is handled: it detects stable-session resumes (issue/agent session-key strategy plus a stored prior session id), compacts the task markdown, and omits the description from its prompt-embedded JSON copy. > This is a focused correctness/efficiency fix to existing wake plumbing and does not overlap with planned roadmap feature work. ## Model Used - Anthropic Claude Fable 5 (`claude-fable-5`), extended thinking enabled, with repository tool use, shell execution, and local test execution via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details (execution-workspace branch, same convention as merged #10202) - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (code-level docs; no user-facing docs affected) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-24 16:35:07 -05:00
DottaandPaperclip	d3c004d1b8	Fix issue descriptions in structured wake payloads (#10151) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Heartbeat wake payloads provide the scoped task context an agent needs before it can act safely > - Assignment wakes already loaded the issue description for task markdown, but the structured wake-payload builder dropped it > - Agents reading `PAPERCLIP_WAKE_PAYLOAD_JSON` could therefore see a missing brief while also being told no fallback fetch was needed > - Long descriptions also need a bounded representation so wake environments and prompts remain safe > - This pull request carries the description through the server and adapter contract, and marks truncated descriptions as requiring fallback fetch > - The benefit is that agents receive the actual brief instead of inventing requirements from the title ## Linked Issues or Issue Description Fixes: #5844 Fixes: #2882 Related prior attempts: #2883 and #8402. This change adds focused regression coverage and enforces the missing long-description fallback invariant. **Bug:** Issue-assignment wake payloads omitted the issue description from the structured payload even when the issue had a populated description. **Expected behavior:** The structured wake payload includes the issue description. If the description must be truncated for payload size, `fallbackFetchNeeded` is `true`. **Reproduction:** Assign an issue with a description to an agent and inspect `PAPERCLIP_WAKE_PAYLOAD_JSON`; before this change, `issue.description` was absent while `fallbackFetchNeeded` could remain `false`. **Affected version:** Reproduced on current `master` before this patch. **Deployment mode:** Adapter-backed heartbeat execution, including local Codex agents. ## What Changed - Include `issues.description` in the server wake-payload query and supplied issue summaries. - Bound inline descriptions at 12,000 characters and force fallback fetch when truncation occurs. - Preserve and render description metadata through shared adapter normalization and prompt rendering. - Add focused tests for long-description fallback and exact brief-string rendering. ## Verification - `pnpm exec vitest run server/src/__tests__/heartbeat-agent-session-message.test.ts packages/adapter-utils/src/server-utils.test.ts` — 81 tests passed. - `pnpm --filter @paperclipai/adapter-utils typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `git diff --check` — passed. ## Risks - Low risk: the payload shape is additive. - Very long descriptions are truncated at 12,000 characters; the payload explicitly requests a fallback fetch for the full brief. - Prompt size increases by the issue-description length for scoped wakes, bounded by the same limit. > This is a focused correctness fix and does not overlap with planned roadmap feature work. ## Model Used - OpenAI GPT-5.4 via Codex CLI, with reasoning, repository tool use, shell execution, and test execution. The runtime did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-24 15:40:39 -05:00
DottaandPaperclip	e4662b1f9d	fix: preserve control plane access in Codex sandbox (#10152) ... ## Thinking Path > - Paperclip is the open source control plane people use to manage AI agents for work > - Local coding adapters can be confined with filesystem and network sandbox policies > - Codex confinement proxied only user-allowlisted hosts, so it denied Paperclip's own run API and managed MCP endpoints > - The same proxy returned untyped plaintext denials, which MCP clients could treat as a fatal unexpected content type > - Host-only `PAPERCLIPAI_CMD` values could also leak into agents even when the referenced checkout module did not exist > - This pull request gives the sandbox an explicit trusted-URL channel for Paperclip endpoints, returns structured JSON errors, and removes the inherited host CLI pointer > - The benefit is confined Codex agents retain control-plane access without broadening the operator's external network allowlist or crashing on policy denials ## Linked Issues or Issue Description Refs #3802 Related foundation: #9504 ### What happened? A `codex_local` agent configured with `networkScope: "allowlist"` and an external-only allowlist could not reach its own Paperclip API or Paperclip-managed MCP endpoints. The sandbox proxy returned a `403` plaintext response without `Content-Type`, and an inherited `PAPERCLIPAI_CMD` could point at a missing checkout-local CLI module. ### Expected behavior Paperclip's run-scoped API and managed MCP endpoints remain reachable regardless of the user external allowlist. Policy denials are valid structured JSON responses with an explicit media type, and host-only CLI pointers are not inherited by agent processes. ### Steps to reproduce 1. Configure a `codex_local` agent with `networkScope: "allowlist"` and `networkAllowlist: ["api.openai.com"]`. 2. Run the agent and request its Paperclip issue API or a Paperclip-managed MCP endpoint. 3. Observe the sandbox proxy deny the request with an untyped plaintext `403` response. ### Environment - Paperclip commit: `f49a3f99` originally exhibited the defect; fix is based on current `master`. - Deployment: local source build on Linux. - Adapter: Codex. - Database: not database-related. - Access context: agent bearer/run-scoped credentials. ## What Changed - Added internal trusted URL rules to the local network allowlist proxy and supplied the Codex run API plus managed MCP endpoints. - Returned JSON error envelopes with `Content-Type` and `Content-Length` for HTTP and CONNECT policy denials. - Removed inherited `PAPERCLIPAI_CMD` from child process environments while preserving explicitly constructed runtime variables. - Added focused proxy and environment sanitizer regression tests. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/local-process-sandbox.test.ts packages/adapter-utils/src/server-utils-env.test.ts --reporter=verbose` - 2 test files passed; 8 tests passed; 4 platform-dependent tests skipped. - `git diff --check` - Package typecheck was attempted; it reaches unrelated current-`master` type drift in untouched files (`spawnCwd` in `adapter-utils`, and staged-runtime ACP types in `codex-local`). ## Risks - Low risk: trusted access is restricted to exact HTTP(S) hostname and port pairs derived from Paperclip-provided URLs. - Invalid or non-HTTP trusted URL values are ignored rather than broadening access. - Denial response bodies change from plaintext to structured JSON; status codes remain unchanged. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent (exact model ID and context window are not exposed to this runtime), reasoning and terminal/tool execution enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-24 15:25:59 -05:00
3 people	762ce5b4ef	feat(acpx): per-step timing observability for sandbox run-startup (#10204) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox-backed runs need observable startup behavior so operators can see where time is spent before an adapter is invoked > - The current startup path only surfaced aggregate timing, which makes it hard to identify the slow boundary in the bring-up sequence > - That gap matters because sandbox startup latency is often dominated by one specific step, and aggregate timing hides the bottleneck > - This pull request adds per-step startup timing events for the named sandbox bring-up boundaries > - The benefit is more precise observability with no control-flow change and no schema migration ## Linked Issues or Issue Description ### Subsystem affected Cross-cutting (multiple of the above) ### Problem or motivation Sandbox run startup only exposed aggregate timing. That makes it hard to identify which bring-up boundary is responsible for slow starts, especially in remote or sandboxed execution where the bottleneck can move between workspace setup, skill reconciliation, bridge setup, and adapter handshake. ### Proposed solution Emit a structured timing event for each named startup boundary before the adapter is invoked, so the existing run-event stream carries per-step duration data. This keeps the event path additive and lets operators see which step dominates startup latency without changing control flow or introducing a schema migration. ### Alternatives considered - Keep only the aggregate startup duration: simpler, but it hides the bottleneck and makes regression analysis much harder. - Add a new telemetry sink or schema field: rejected because the existing run-event payload already carries structured event data and does not need a new storage path. - Log unstructured text for each step: rejected because it is harder to query and aggregate than a structured `step` + `durationMs` event. ### Roadmap alignment This fits the roadmap direction around cloud / sandbox agents and enforced outcomes by improving observability for sandboxed execution without changing the control plane model. The roadmap section is broad, but it does not call out this specific startup-timing work as a planned duplicate. ### Additional context This PR is intentionally additive. It records timing for the named startup boundaries in the existing event stream and leaves the bridge, database shape, and adapter invocation order unchanged. ## What Changed - Added a `measureStartupStep` helper that times a startup step, emits one structured `run.startup.step` event, and rethrows failures after recording duration - Wrapped the seven sandbox bring-up boundaries in `execute.ts` so the structured timing covers each named step before adapter invocation - Added unit coverage for the helper and integration coverage for the startup-step events in the adapter-utils execute path - Kept the event path additive, with no bridge change and no database migration ## Verification - `tsc --noEmit` for `@paperclip/adapter-utils` - `pnpm test` in `packages/adapter-utils` equivalent suite coverage: 292 passed, 4 skipped - Adjacent server event/log-store suites: `run-log-store.test.ts` and `heartbeat-run-log.test.ts` passed (11 total) - Git validation: fetched `origin/feat/sandbox-startup-step-timing`, confirmed it matches the authorized submit SHA, and confirmed `origin/master..origin/feat/sandbox-startup-step-timing` contains the expected single commit - Searched GitHub for duplicate or related open PRs/issues and found no overlapping open items - Checked `ROADMAP.md`; the roadmap covers sandboxed environments generally, but does not call out this specific startup-timing observability work as a planned duplicate ## Risks - Low risk: the change is additive and only emits additional structured events - If downstream consumers assume startup events are aggregate-only, they may need to ignore or account for the new `run.startup.step` entries - Timing is measured via the injected clock and event emission happens in a `finally`, so failures still report duration before rethrowing ## Model Used OpenAI GPT-5, tool-using coding agent ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-24 12:50:37 -07:00
Michael NguyenandPaperclip	caae2778f0	fix: deliver plugin agent session turns and replies (#10137) ... ## Thinking Path > - Paperclip manages agent execution through heartbeat runs and adapter-specific sessions > - Plugins can open an agent session and send a conversational message through the host service > - The host previously stored that message only in opaque wake payload metadata, so local adapters never saw it in their CLI prompt > - The host also forwarded run log chunks but did not expose the persisted final assistant text as the session reply > - This pull request defines both sides of the session contract in the shared wake renderer and terminal run event > - The benefit is that local adapters receive the actual conversational turn and plugins receive one canonical final reply ## Linked Issues or Issue Description Related context: Refs #629 and Refs #2880 describe adjacent `claude_local` final-text visibility failures. They concern issue comments rather than plugin agent sessions, but exercise the same need for a canonical persisted run summary. Companion consumer change: paperclipai/paperclip-gateway#3. Bug description: - **Observed:** calling the plugin host's `agents.sessions.sendMessage()` with `prompt: "hello"` woke a `claude_local` agent, but the generated CLI prompt omitted `hello`. On completion, the session emitted log chunks and a generic `Run completed` done event, so callers could not reliably recover the assistant reply. - **Expected:** the prompt becomes the user-supplied conversational turn for that agent session, and the successful terminal event carries the run's canonical final user-facing assistant text. - **Reproduction:** create a plugin agent session for a local adapter, call `sendMessage()` with a non-empty prompt, inspect the adapter prompt and terminal session event. - **Affected baseline:** `b517b887a` on `master`, local trusted deployment with plugin host services and `claude_local`; `codex_local` shared the wake-rendering gap because both use the common Paperclip wake prompt renderer. ## What Changed - Added a typed `agentMessage` wake payload rendered by the shared adapter prompt path used by `claude_local`, `codex_local`, and other local adapters. - Labeled session content as user-supplied and explicitly non-authoritative: it cannot expand authorization, permissions, task scope, or company boundaries. - Preserved ordinary heartbeat behavior by omitting the section when no agent-session message exists. - Added canonical `finalText` to terminal heartbeat status events from the already-persisted run summary/result/message. - Defined successful `AgentSessionEvent.message` as the canonical final user-facing reply (or `null`) and forwarded it on the terminal `done` event. - Added host, wake-renderer, normal-heartbeat, and terminal-reply regression coverage. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/server-utils.test.ts server/src/__tests__/heartbeat-agent-session-message.test.ts server/src/__tests__/heartbeat-run-status-payload.test.ts server/src/__tests__/plugin-agent-sessions.test.ts server/src/__tests__/heartbeat-run-summary.test.ts` — 87 passed. - `pnpm -r typecheck` — passed across all 31 workspaces. - `pnpm build` — passed. - `pnpm test:run` — 2,860 passed, 1 skipped, 3 unrelated failures: two existing macOS temp-path alias assertions (`/tmp` vs `/private/tmp`) in workspace branch-containment tests and one reproducible auto-port runtime-service adoption failure. The same three failures reproduce when the two files run alone; none touch this change. - Live Slack verification intentionally remains operator-gated because it requires rebuilding/restarting the host. ## Risks - User-controlled chat text now reaches the model prompt, which is an intentional prompt-injection surface. The renderer labels it as untrusted conversational content, while the existing plugin/session company checks and caller authorization remain unchanged. - `finalText` is added to company-scoped heartbeat status events. It is derived from the same persisted summary/result/message already used for run comments; no raw stdout or secrets are added. - Consumers that ignore the new field remain compatible, and successful runs without usable final text still emit `message: null`. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex (GPT-5), agentic reasoning with repository/tool use and code execution; context-window size is not surfaced in this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-23 20:52:42 -07:00
Nicky Leach	b517b887ad	fix(acpx): decouple host proxy spawn cwd from in-sandbox remoteCwd (#10122)	2026-07-23 16:48:52 -07:00
Nicky LeachandPaperclip	d36ea13e08	feat(acpx): stage once per remote ACP session - compatible resume reuses staged runtime, no cross-session credential reuse (#10089) ... ## Thinking Path > - Paperclip coordinates autonomous agent work across isolated company-scoped sessions > - The ACP remote lane stages workspaces and managed home state inside the sandbox so sessions can resume safely > - If a compatible resume re-staged everything every time, it would waste work and risk inconsistent session reuse behavior > - If an incompatible resume reused the wrong staged runtime, it could cross session boundaries or leak credentials > - This pull request keeps the session fingerprint as the scoping key and adds a staged-runtime cache keyed to that fingerprint > - Compatible resumes now reuse the already staged runtime while incompatible fingerprints stage fresh > - The benefit is faster safe resumes without weakening session isolation or credential separation ## Linked Issues or Issue Description ### Problem or motivation The ACP remote lane currently needs to preserve safe resume behavior without repeatedly re-staging work that is already valid for the same session. The failure mode to avoid is letting one session reuse another session's staged workspace or credentials. ### Proposed solution Keep the session fingerprint as the scoping key and add a staged-runtime cache keyed to that fingerprint. When the fingerprint matches, reuse the already staged workspace and managed home. When the fingerprint changes, stage fresh. ### Alternatives considered - Always restage on resume: safest mechanically, but wastes work and breaks the compatible-resume optimization. - Reuse without fingerprint scoping: too risky because it could cross session boundaries. ### Roadmap alignment This is a narrow implementation change for the ACP remote lane and does not duplicate any broader roadmap item I could find in `ROADMAP.md`. ### Additional context The change preserves the existing session fingerprint contents and codex auth copy-back cadence while adding tests for compatible reuse, incompatible fresh staging, no cross-session credential reuse, and failed-turn eviction. ## What Changed - Added a staged-runtime cache in the ACP remote execution path keyed by the session fingerprint. - Reused the existing staged workspace and managed home for compatible resumes. - Kept incompatible fingerprints on the fresh staging path. - Preserved the existing session fingerprint contents and codex auth copy-back cadence. - Added tests for compatible reuse, incompatible fresh staging, no cross-session credential reuse, and failed-turn eviction. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts` (74/74 pass, including the active-turn lease regression) - `pnpm exec tsc -p packages/adapter-utils/tsconfig.json --noEmit` - Verified the PR touches only `packages/adapter-utils/src/acpx-engine/execute.ts` and `packages/adapter-utils/src/acpx-engine/execute.test.ts` ## Risks - A cache eviction bug could cause an unavailable or partially staged runtime to be reused. - If the fingerprint scoping regressed, a session could incorrectly reuse another session's state. - The change is isolated to the ACP remote lane, but it still affects resume behavior for that path. ## Model Used OpenAI Codex, GPT-5, reasoning-capable coding agent, tool-enabled session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-23 10:13:44 -07:00
3 people	e1cc63328a	feat(acpx): per-adapter managed-home seed (x3) + Codex auth copy-back for remote ACP lane (#10073) ... ## Thinking Path > - Paperclip is a control plane that orchestrates AI agents and adapter execution for human operators. > - Agents run across local and remote execution contexts, and reliability in remote sessions depends on consistent adapter bootstrapping. > - The ACP path must prepare per-adapter runtime homes so managed credentials and config are available in sandboxed runner environments. > - Before this change, the new remote ACP lane did not yet consistently stage managed-home paths for all affected adapters or restore Codex auth state on teardown. > - We added a shared per-adapter seam in the ACP engine, then wired Codex, Claude, and Gemini remote lanes to seed managed homes and remap to in-sandbox locations. > - Codex additionally reuses the existing atomic auth restore flow to copy auth back on teardown, matching CLI behavior. > - This improves remote runner parity with existing CLI behavior and avoids credential drift in shared-code-path executions. ## Linked Issues or Issue Description - This change continues the ACP remote managed-home work by completing per-adapter remote bootstrapping and Codex auth restore behavior for the remote ACP lane. - It specifically covers: `acpx-engine`, `codex-local`, `claude-local`, and `gemini-local`. - Related prior work in this repo: PR #10070. ## What Changed - Add a per-adapter remote managed-home seam (`prepareRemoteManagedHome`) in `acpx-engine` and thread it through ACP execution options. - Implement Codex ACP preparation to: - stage `CODEX_HOME` (auth/config/skills) into a sandboxed remote home, - repoint `CODEX_HOME` to the in-sandbox path, - and wire teardown copy-back through the existing managed-auth restore path. - Implement Claude ACP preparation to seed a sanitized `config-seed` into `CLAUDE_CONFIG_DIR` and remap that directory to the sandbox root. - Implement Gemini ACP preparation to seed `~/.gemini/skills`, set `HOME` to managed runtime root, and preselect API-key auth in `settings.json`. - Keep local and runner-less ACP→CLI behavior unchanged by only invoking the remote managed-home seam when running in remote ACP mode. - Preserve existing authorization and activity boundaries in the shared engine and adapter layers. ## Verification - `git log --oneline origin/master..origin/feat/acp-remote-managed-home-seed` confirms only the expected 4 commits. - `tsc --noEmit` is clean in `adapter-utils`, `codex-local`, `claude-local`, and `gemini-local`. - Vitest selection used during validation passed (120 tests across the ACP-related suites). ## Risks - If remote sandbox teardown occurs after token rotation but before restore timing, Codex credentials can become stale and require re-auth on next startup. - Partial provisioning of managed-home assets would cause adapter bootstrap failures in runner-backed ACP sessions. - This change is scoped to execution-path behavior; it should not affect CLI behavior. ## Model Used None — human-authored. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-23 06:42:52 -07:00
3 people	8a3cc86531	feat(acpx): stage workspace + route in-sandbox cwd for remote ACP lane (#10070) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The shared ACP engine (`packages/adapter-utils/src/acpx-engine/execute.ts`) is responsible for launching local and remote agent processes via the ACP protocol > - On runner-backed remote sandbox (Daytona) targets, `buildRuntime` never crossed the CLI's staging seam: it never called `prepareAdapterExecutionTargetRuntime`, left `runtimeRootDir: null` in both the paperclip and process-session bridges, and handed the agent the **HOST filesystem path** as the `session/new` cwd — meaning Claude/Gemini silently operated on a path that does not exist inside the sandbox (Codex additionally crashes on its HOST home path, addressed in a follow-up PR) > - The fix must cross the staging seam for remote sandboxes, thread the real `runtimeRootDir` through both bridges, and bind the in-sandbox workspace path as the session cwd — without touching local ACP runs or the runner-less ACP→CLI fallback > - This pull request introduces a `stageAcpRemoteRuntime` helper that calls `prepareAdapterExecutionTargetRuntime` for runner-backed remote runs, captures `{ workspaceRemoteDir, runtimeRootDir, assetDirs, restoreWorkspace }`, and reuses the in-sandbox `workspaceRemoteDir` as the single `sessionCwd` across `session/new`, the fingerprint, compatibility check, persistence, `ensureSession`, the process-session bridge cwd, and the error path > - The benefit is that remote ACP runs now operate in the correct in-sandbox cwd and receive a non-null `runtimeRootDir` in both bridges — fixing silent wrong-cwd degradation for Claude/Gemini on Daytona targets; this is PR 1 of 3 and seeds no credential material ## Linked Issues or Issue Description No public GitHub issue exists for this change. Inline description follows the feature request template: ### Subsystem affected packages/adapters — agent adapter implementations ### Problem or motivation The shared ACP engine (`packages/adapter-utils/src/acpx-engine/execute.ts`) never crossed the CLI's staging seam on runner-backed remote sandbox targets. It never called `prepareAdapterExecutionTargetRuntime`, always passed `runtimeRootDir: null` to both the paperclip and process-session bridges, and handed the agent the **HOST filesystem path** as the ACP `session/new` cwd. As a result, Claude and Gemini silently operated on a cwd that does not exist inside the sandbox; Codex crashed with a fatal error on the HOST `CODEX_HOME` path (that crash is in a follow-up PR). ### Proposed solution Gate on `usesRunnerBackedSandbox` (`kind === "remote" && transport === "sandbox" && runner`). For runs that pass the gate, call `prepareAdapterExecutionTargetRuntime` via a new `stageAcpRemoteRuntime` helper that ships the workspace into the sandbox and captures `{ workspaceRemoteDir, runtimeRootDir, assetDirs, restoreWorkspace }`. Thread the real `runtimeRootDir` into both bridges. Bind a single `sessionCwd` (the in-sandbox `workspaceRemoteDir`) and use it at every cwd-keyed session site (`session/new`, fingerprint, compat, persist, `ensureSession`, process-session bridge, error path) so a warm/resumable session is reused rather than invalidated. For local runs and the runner-less ACP→CLI fallback, `sessionCwd` resolves to the HOST cwd — byte-identical to the previous behavior. ### Alternatives considered Patching each per-adapter bridge individually — rejected because the bug is in the shared engine layer and the fix belongs there so all three adapters (Codex, Claude, Gemini) benefit without per-adapter duplication. ### Roadmap alignment Internal correctness fix enabling remote ACP to work as designed; no new user-facing features. This is PR 1 of 3 in a sequential chain: PR 1 (this PR) stages the workspace and routes the cwd; PR 2 adds per-adapter home seeding and copy-back; PR 3 wires session-lifecycle restore. ## What Changed - **`packages/adapter-utils/src/acpx-engine/execute.ts`** — added `stageAcpRemoteRuntime` helper that calls `prepareAdapterExecutionTargetRuntime` for runner-backed remote sandboxes and returns `{ sessionCwd, runtimeRootDir, stagedRuntime }`; `buildRuntime` now uses this helper to derive `sessionCwd` (in-sandbox `workspaceRemoteDir` for remote; HOST cwd unchanged for local/runner-less) and threads the real `runtimeRootDir` to both the paperclip bridge and process-session bridge; `stagedRuntime` is stashed for the follow-up credential PR - **`packages/adapter-utils/src/acpx-engine/execute.test.ts`** — new engine-level unit tests: staging seam crossed with no credential asset, non-null `runtimeRootDir` in both bridges, in-sandbox `session/new` cwd, warm-handle reuse after the cwd change, local-unchanged; 60 tests green - **`packages/adapters/codex-local/src/server/acp.test.ts`** — new per-adapter test: runner-backed remote asserts `ensureSession` cwd == `workspaceRemoteDir`; runner-less sandbox falls back to CLI - **`packages/adapters/claude-local/src/server/acp.test.ts`** — same per-adapter coverage for Claude - **`packages/adapters/gemini-local/src/server/acp.test.ts`** — same per-adapter coverage for Gemini ## Verification - `adapter-utils` typecheck clean; `codex/claude/gemini-local` typecheck clean - Engine units (`acpx-engine/execute.test.ts`): 60/60 green — staging seam crossed with no credential asset, non-null `runtimeRootDir` to both bridges, in-sandbox `session/new` cwd, warm-handle reuse after cwd change, local unchanged - Per-adapter ACP test suites (`codex/claude/gemini-local` `acp.test.ts`): 103 tests green — runner-backed remote asserts `ensureSession` cwd == `workspaceRemoteDir`; runner-less sandbox falls back to CLI - CI green on PR (in progress) ## Risks This is PR 1 of 3 in a strictly sequential chain; it seeds **no credential material** (no `assets`, no `installCommand`). The per-adapter home seeding is deferred to PR 2, which consumes the `stagedRuntime` object stashed here. The `restoreWorkspace` callback is carried on `stagedRuntime` for PR 3's session-lifecycle wiring (see the `stageAcpRemoteRuntime` function comment). Local ACP runs and the runner-less ACP→CLI fallback are untouched — `sessionCwd` resolves to the HOST cwd for those paths, preserving existing behavior. The `stageAcpRemoteRuntime` helper is gated on `usesRunnerBackedSandbox`, so there is no regression risk for local or CLI-lane runs. ## Model Used Claude Sonnet 4.6 (`claude-sonnet-4-6`) via Paperclip ACPX engine — extended context, tool use enabled, co-authored with Paperclip agent orchestration. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`, `feat/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-22 22:24:30 -07:00
3 people	b247bf7150	feat(runtime): opt-in sandbox file-sync lifecycle hooks (API + provider docs) (#10013) ... ## Thinking Path > - Paperclip is an open-source AI-agent management platform; agents run tasks inside sandboxed environments (Daytona, Kubernetes, E2B, etc.) > - The control-plane ↔ sandbox file-transfer path flows through the `environmentExecute` seam in `protocol.ts` — the only verb available to plugins — which forces a base64-over-exec chunked loop for every file move: workspace files, assets, Codex home sync > - This transport is correct and safe, but it bypasses provider-native bulk/streaming APIs (Daytona `uploadFiles`, K8s `FastUploadInterceptor` / volume mounts), leaving significant throughput on the table for large workspaces > - The right fix is an opt-in seam extension: providers with faster native transfer declare two optional verbs; providers that do not opt in stay on the existing fallback with zero code or behavior change required > - This PR adds the first layer of that extension — two optional verbs (`environmentSyncIn` / `environmentSyncOut`) in the plugin SDK, the runtime plumbing to prefer the native path for the two clean destroy-then-replace cases, and a doc for the contract > - The core correctness invariant is byte-identical fallback: if no provider opts in, execution is exactly what ships today; `assertSyncOperationsConfined` enforces host-side path confinement for providers that do opt in > - No provider advertises the verbs yet → zero production behavior change; future PRs wire up Daytona and K8s providers against this contract ## Linked Issues or Issue Description No public GitHub issue exists for this feature. Description follows the `feature_request` issue template: **Subsystem affected:** packages/plugins — plugin system; packages/adapter-utils — adapter runtime; server/ — EnvironmentRuntimeService **Problem or motivation:** Sandbox file transfers currently always use a base64-over-exec chunked loop regardless of what the underlying provider supports. For workspaces larger than a few MB this becomes the dominant wall-clock cost of every sandbox run, and it bypasses bulk/stream APIs that providers like Daytona already expose natively. **Proposed solution:** Add two optional, opt-in plugin hooks — `onEnvironmentSyncIn` / `onEnvironmentSyncOut` — to the plugin SDK. When a provider defines both hooks and both are advertised via the existing `supportedMethods` negotiation, the runtime prefers the native path for the two clean destroy-then-replace transfer cases; all other cases fall back to the existing byte-identical base64 transport. **Alternatives considered:** An unconditional verb would require every provider to implement or stub the verb. The opt-in / `METHOD_NOT_IMPLEMENTED` pattern (already used by `environmentExecute`) preserves backward compatibility with zero provider changes required. **Roadmap alignment:** Consistent with the ✅ "Cloud / Sandbox agents" and ✅ "Plugin system" milestones; extends the plugin seam rather than adding control-plane-level logic. **Additional context:** Searched open pull requests and issues for duplicate sandbox file-sync / native-transfer work; none found. ## What Changed - **`packages/plugins/sdk`** - `protocol.ts`: two new optional `HostToWorkerMethods` — `environmentSyncIn` / `environmentSyncOut` — plus generic `SyncOperation`, `SyncFileMapping`, and `SyncOutcome` types - `define-plugin.ts`: optional `onEnvironmentSyncIn` / `onEnvironmentSyncOut` fields on `PluginDefinition`; worker advertises each verb only when its hook is defined (else `METHOD_NOT_IMPLEMENTED`, mirroring `environmentExecute`) - `worker-rpc-host.ts`: route new verbs to plugin hooks - `index.ts`: re-export new public types - **`packages/adapter-utils`** - `command-managed-runtime.ts`: expose optional `syncIn` / `syncOut` on `CommandManagedRuntimeRunner` (available only when both verbs are advertised); add `assertSyncOperationsConfined` host-side path-confinement guard - `sandbox-managed-runtime.ts`: `SandboxManagedRuntimeClient` gains optional `syncIn` / `syncOut`; orchestrator prefers native path for default-provision asset inbound and workspace-download-into-fresh-dir outbound; all other paths keep the existing base64 fallback - `sandbox-file-sync.test.ts` (new): 234-line characterization suite — native-opt-in branch, fallback branch, `assertSyncOperationsConfined` escape-path rejection, `followSymlinks` → tar `-h` - `command-managed-runtime.test.ts`: negotiation + native-sync + confinement tests - **`server/src/services/environment-runtime.ts`**: `EnvironmentRuntimeService` delegates to `syncIn` / `syncOut`, gated on advertised support - **`server/src/services/environment-execution-target.ts`**: minor typing fix alongside the new verbs - **`doc/plugins/SANDBOX_FILE_SYNC_HOOKS.md`** (new): documents the full contract — opt-in / no-op guarantee, operation ordering, provider-may-tar, atomicity, `followSymlinks`, secret modes (0600, no window), path confinement, `operationId` opacity, resource bounds, shell-quoting ## Verification ```bash # SDK suite pnpm --filter packages/plugins/sdk test # Adapter-utils suite (includes new sandbox-file-sync characterization tests) pnpm --filter packages/adapter-utils test # Expected: 255 pass / 4 skip # Type-check across affected packages pnpm --filter packages/plugins/sdk typecheck pnpm --filter packages/adapter-utils typecheck # Server changed-file spot check: cd server && npx tsc --noEmit --skipLibCheck 2>&1 | grep -E "environment-(runtime|execution-target)" | head -20 ``` Key behavioral invariant to spot-check: with no provider opting in (the current state), run any sandbox task and confirm file-transfer behavior is byte-for-byte identical to what the pre-PR code produces. The characterization tests assert this at the unit level. ## Risks - **Zero production risk today**: no provider advertises `environmentSyncIn` / `environmentSyncOut`, so the new code paths are unreachable in production; all real traffic stays on the existing base64 fallback - **Path confinement**: `assertSyncOperationsConfined` rejects any `targetPath` that escapes the declared root — this is the primary security boundary for future providers. The test suite covers escape-path rejection - **Atomicity**: the contract delegates atomicity to providers; the doc explicitly calls out that directory-level ops are not guaranteed atomic - **Secret transport**: credential assets (e.g., Codex `auth.json`, directory mappings) continue to use the existing tar path — they do not go through the new verbs in any current provider > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Provider: Anthropic Model: `claude-sonnet-4-6` (Claude Sonnet 4.6) Context window: 200 K tokens Capabilities: extended tool use, multi-file code generation, agentic reasoning via the Paperclip agent framework ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-22 10:08:03 -07:00
DottaandPaperclip	d31a28828b	fix(acpx): support Windows agent spawning (#9980) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Local Claude, Codex, Gemini, and custom ACP adapters run through the shared embedded ACPX engine > - That engine wrapped every local agent command in a generated Bash script to inject environment variables and filter child stderr > - Windows cannot directly spawn that Bash wrapper, and npm/pnpm ACP binaries are exposed through `.cmd` shims there > - ACPX 0.12 already supports per-session child environment variables, so the wrapper is unnecessary > - This pull request registers agent commands directly, injects env through ACPX session options, captures child stderr in-process, and adds a real Node ACP spawn smoke on Ubuntu and Windows > - The benefit is one cross-platform spawn path with a reusable smoke test instead of parallel shell-wrapper implementations ## Linked Issues or Issue Description Fixes #9941. Refs #9428 and #9771. **What happened** ACPX-backed local agents failed to start on Windows because Paperclip registered a generated POSIX `.sh` wrapper as the agent command. Windows also needs the `.cmd` npm/pnpm shim when resolving built-in ACP binaries, and symlink creation can fail with `EPERM` for seeded auth/skill files. **Expected behavior** The same ACPX engine path should spawn a real ACP agent on Windows and Linux, forward Paperclip/runtime env without mutating `process.env`, preserve filtered/unfiltered child stderr behavior, and fall back to copies where Windows symlinks are unavailable. **Steps to reproduce** Run a local ACPX adapter on Windows with the prior wrapper path. ACPX attempts to spawn the generated `.sh` file and the agent never initializes. **Deployment mode** Local Paperclip adapters using `packages/adapter-utils/src/acpx-engine/`. ## What Changed - Removed generated Bash agent/env wrappers and registered local commands directly with ACPX. - Passed the resolved child environment through ACPX `sessionOptions.env`, including resume retry paths. - Added a minimal `acpx@0.12.0` package patch exposing child stderr callbacks and allowing documented uppercase env-map keys in persisted session options. - Moved stderr tee/filter behavior in-process: raw stderr remains in the per-run file while benign `nes/close` noise is omitted from live stderr. - Preferred `.cmd` ancestor binaries on Windows and added `EPERM` copy fallbacks for Codex auth seeding and Gemini skill materialization. - Added a real Node ACP echo-agent spawn smoke that can run directly on any supported platform. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts packages/adapter-utils/src/acpx-engine/spawn-smoke.test.ts` — 57 passed. - `pnpm --filter @paperclipai/adapter-utils typecheck` — passed. - `node --test scripts/acpx-patch-packaging.test.mjs scripts/release-lib.test.mjs` — 10 passed. - Full canary release dry run under Node 24.18.0 / npm 11.16.0 — passed in an isolated scratch clone. - `git diff --check` — passed during implementation verification. - One-time GitHub Actions proof: [Ubuntu ACPX spawn smoke](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774579), [Windows ACPX spawn smoke](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774558), and [Canary Dry Run](https://github.com/paperclipai/paperclip/actions/runs/29924348927/job/88937774497) passed on head `f345ac69f2`; the dedicated smoke jobs are intentionally not retained in the recurring PR workflow. ## Risks - The ACPX stderr callback and env persistence exemption are carried as a pnpm dependency patch until ACPX exposes/fixes those behaviors upstream. - Child stderr is synchronously appended to preserve ordering and failure diagnostics; unusually high-volume agent stderr could briefly block the Node event loop. - The Windows-specific `.cmd` resolution and symlink `EPERM` branches are proven by the standalone smoke test and the linked one-time `windows-latest` run rather than a permanent CI gate. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-5.4 via Codex CLI, medium reasoning, repository/tool execution enabled; context-window size is not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-22 10:22:29 -05:00
3 people	5ed0b74b34	fix(runtime): scope PAPERCLIP_ env-binding strip to reserved keys (#9974) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent runs get their environment from user/adapter/project/routine env bindings resolved by the server heartbeat, plus `PAPERCLIP_*` runtime vars (identity, wake, workspace, API access) injected by the harness > - The heartbeat stripped **every** `PAPERCLIP_`-prefixed binding before resolution, so legitimately user-named keys (e.g. cloud provider token bindings like `PAPERCLIP_CLOUD_PROD_PROVIDER_RAILWAY_*`) were silently dropped and never reached the run env > - At the same time, several adapters honored an explicitly configured `PAPERCLIP_API_KEY` over the harness-minted run token, which is exactly the one key config must never control > - This pull request replaces the blanket prefix strip with a precise three-rule policy: never accept `PAPERCLIP_API_KEY` from config, always let harness-assigned runtime vars win, and let every other `PAPERCLIP_*`-named user binding flow through > - The benefit is that user secrets with a `PAPERCLIP_`-style name work like any other binding, while runtime identity and API credentials stay fully harness-controlled ## Linked Issues or Issue Description **Bug description** (no public issue exists): - **What happened:** Env bindings whose key starts with `PAPERCLIP_` (e.g. a cloud provider token a user deliberately named `PAPERCLIP_CLOUD_PROD_PROVIDER_RAILWAY_TOKEN`) were silently stripped by the server before secret resolution, so the spawned agent never received them. No error, no access event — the variable just never appeared. - **Expected behavior:** A user-named `PAPERCLIP_*` binding should reach the run env unless the harness itself uses that key. Only `PAPERCLIP_API_KEY` should be categorically rejected, and harness-assigned runtime vars (`PAPERCLIP_RUN_ID`, `PAPERCLIP_AGENT_ID`, wake/workspace vars, …) should always win over config. - **Steps to reproduce:** Configure an agent/project env binding named `PAPERCLIP_<ANYTHING>` (plain or secret_ref), run a heartbeat, and inspect the spawned process env — the key is absent. - **Deployment mode:** local server, any local adapter. Related prior PRs (different, save-time/API-layer blanket-ban approach; this PR supersedes that direction with a runtime allow-except-reserved policy): Refs #8239, Refs #8439. ## What Changed - `server/src/services/heartbeat.ts`: the pre-resolution strip now removes only `PAPERCLIP_API_KEY` (hard denylist) instead of every `PAPERCLIP_`-prefixed binding; other `PAPERCLIP_*` keys flow into binding resolution. Low-trust inline-sensitive-env checks now also cover those keys. - `packages/adapter-utils/src/server-utils.ts`: new `isForbiddenConfigEnvKey()` helper; the shared `refreshPaperclipWorkspaceEnvForExecution` merge drops `PAPERCLIP_API_KEY` from config and keeps harness-assigned `PAPERCLIP_*` keys authoritative. - `packages/adapter-utils/src/acpx-engine/execute.ts`: removed the explicit-`PAPERCLIP_API_KEY`-from-config allowance; the run token (`authToken`) is now always applied; config `PAPERCLIP_API_KEY` is ignored. - All local adapters (`claude-local`, `codex-local`, `cursor-local`, `gemini-local`, `grok-local`, `opencode-local`, `pi-local`) plus `cursor-cloud`, `hermes`, and the server `process` adapter: removed `hasExplicitApiKey`-style allowances so the harness token always wins, and guarded the remaining unguarded env-merge loops (claude-local inline loop, process adapter) with the same policy. - Tests updated/added: heartbeat binding-strip test now asserts the three-rule policy; adapter-utils merge tests assert the `PAPERCLIP_API_KEY` ban and `PAPERCLIP_*` pass-through; acpx engine tests moved credential fixtures to `authToken` and assert config `PAPERCLIP_API_KEY` is ignored while other `PAPERCLIP_*` config keys forward and still bust the session fingerprint on rotation. ## Verification - `pnpm vitest run packages/adapter-utils/src/server-utils.test.ts packages/adapter-utils/src/acpx-engine/execute.test.ts` — 127 passed - `pnpm vitest run server/src/__tests__/heartbeat-project-env.test.ts server/src/__tests__/heartbeat-local-environment.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/codex-local-execute.test.ts server/src/__tests__/cursor-local-execute.test.ts server/src/__tests__/gemini-local-execute.test.ts` — 68 passed - Adapter package execute suites and the server tests touching API-key fixtures (`heartbeat-run-log`, `redaction`, `effective-run-config-fingerprints`, `agent-permissions-routes`) — green. Three pre-existing sandbox/SSH fixture failures reproduce identically on clean `master` on this host and are unrelated. - `pnpm --filter <pkg> typecheck` for server, adapter-utils, and all nine touched adapter packages — all pass. ## Risks - Behavioral change: a deployment that relied on configuring a static `PAPERCLIP_API_KEY` in adapter config env loses that override — by design; the harness-minted run token is now the only source. When no run token exists, no API key is injected at all. - `PAPERCLIP_*`-named user bindings now reach binding resolution and run envs; a key that collides with a harness runtime var is still discarded at merge time, so runtime identity/wake/workspace vars cannot be spoofed. - Low risk otherwise: no migrations, no API surface changes. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic Claude 5 family, Mythos-class tier), extended thinking enabled, agentic tool use (file edits, shell, test runner) via Claude Agent SDK. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-22 07:01:57 -05:00
3 people	7ffeafad1f	refactor(codex-local): relocate Codex auth-merge scripts + decision predicate into the adapter (#9785) ... ## Thinking Path > - Paperclip is an open source platform for managing AI agent teams, with adapter packages providing concrete runtime environments (e.g. `codex-local` runs a local OpenAI Codex sandbox) > - `adapter-utils` is the shared utilities package — it should hold only generic, adapter-agnostic primitives (sandbox lifecycle helpers, merge logic, type definitions) usable by every adapter > - Three files lived in `adapter-utils/src/` that are entirely Codex-specific: `codex-auth-merge-extract.sh` (shell script that extracts auth tokens), `codex-auth-merge-decision.cjs` (CJS decision helper), and `codex-auth-merge-scripts.ts` (TypeScript factory that wires them into the inbound provision seam added in PR #9778) > - Their presence in a "generic" package violates the adapter isolation principle and requires a cross-package build step to copy `.sh`/`.cjs` files into `codex-local/dist/server/` at build time > - This PR completes Phase 2 of the inbound-seam refactor: move all three files to `packages/adapters/codex-local/src/server/`, rebase the TypeScript imports, update `execute.ts` to import locally, move the Codex-auth tests into a new `codex-local` test file, and fix packaging so `codex-local` copies its own scripts to `dist/server/` > - Script bytes are identical after the move; no change to inbound auth-merge behavior or to which bytes cross the sandbox boundary > - The benefit is a clean ownership boundary: `adapter-utils` retains only generic runtime code, and the structural "Codex-free core" test in the adapter-utils suite validates this invariant going forward ## Linked Issues or Issue Description No pre-existing public GitHub issue. Describing the underlying problem inline: **Problem:** `packages/adapter-utils/src/` contains three files (`codex-auth-merge-extract.sh`, `codex-auth-merge-decision.cjs`, `codex-auth-merge-scripts.ts`) consumed exclusively by the `codex-local` adapter. Their presence in a generic utilities package violates adapter isolation and requires a cross-package build step (copy `.sh`/`.cjs` into `codex-local/dist/server/`). The inbound provision seam landed in PR #9778 routed these through `adapter-utils`; this PR finishes the relocation. **Related:** Refs #9778 (Phase 1 — generic asset-lifecycle-seam, now merged). ## What Changed - Moved `codex-auth-merge-extract.sh`, `codex-auth-merge-decision.cjs`, and `codex-auth-merge-scripts.ts` from `packages/adapter-utils/src/` → `packages/adapters/codex-local/src/server/` (script bytes are unchanged; `codex-auth-merge-scripts.ts` imports rebased to `adapter-utils` subpaths for `shellQuote` and `SandboxManagedRuntimeAssetProvision`) - `packages/adapters/codex-local/src/server/execute.ts`: updated import of `buildCodexAuthInboundProvision` from `adapter-utils` → local `./codex-auth-merge-scripts` - New `packages/adapters/codex-local/src/server/codex-auth-merge.test.ts` — 5 Codex-auth test rows extracted from `adapter-utils/src/workspace-restore-merge.test.ts`; the generic test file stays and no longer references Codex - `packages/adapters/codex-local/package.json`: build now copies `.sh`/`.cjs` from `src/server/` into `dist/server/` directly - `packages/adapter-utils/package.json`: removed the now-obsolete cross-package copy step for those files ## Verification ```sh # Type-check both affected packages pnpm --filter @paperclipai/adapter-utils --filter @paperclipai/adapter-codex-local typecheck # Codex auth-merge suite (moved tests — 5/5 pass) pnpm --filter @paperclipai/adapter-codex-local test -- --reporter=verbose codex-auth-merge # Full adapter-utils suite including the structural "core free of Codex literals" test (243 passed, 4 pre-existing skips) pnpm --filter @paperclipai/adapter-utils test # Confirm build copies scripts to dist/server pnpm --filter @paperclipai/adapter-codex-local build ls packages/adapters/codex-local/dist/server/*.sh packages/adapters/codex-local/dist/server/*.cjs ``` All of the above were run locally and passed before this PR was opened. ## Risks Low risk — pure file relocation: - No change to `.sh` or `.cjs` script bytes; the same content reaches the sandbox boundary as before - No behavioral change to inbound auth-merge logic from the sandbox's perspective - `adapter-utils`' structural "core free of Codex literals" test now validates that the relocation is complete and will catch any regression - Only `codex-local` consumed these files from `adapter-utils`; no other package in the monorepo imported them from there ## Model Used Claude Sonnet 4.6 (`claude-sonnet-4-6`) — Anthropic, 200k context window, tool use, agentic reasoning. Used to author the refactor. `Co-authored-by: Paperclip <noreply@paperclip.ing>` trailer present on all commits. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-17 14:45:36 -05:00
3 people	cf5ba4bbea	feat(adapter-utils): generic per-asset lifecycle-contribution seam (#9778) ... ## Thinking Path > - Paperclip's sandbox managed runtime is responsible for provisioning the agent's execution environment — it extracts a home directory asset into the sandbox before the adapter runs. > - The sandbox runtime core was directly branching on the adapter key (`codex`) to decide which merge scripts to stage and which merge-extract command to run, coupling generic infrastructure to a specific adapter's credential-merge protocol. > - This makes it harder to add, remove, or modify per-adapter asset provisioning without touching the runtime core; it also prevents other adapters from contributing staged files or a custom extract command at all. > - The fix is to move the adapter-specific knowledge into the adapter itself: the asset descriptor gains optional `provision` (stageFiles + extractCommand) and `restore` contribution fields that any adapter can populate, and the runtime core consumes them generically. > - This pull request introduces those contribution fields, wires the Codex adapter's inbound credential-merge as a `provision` contribution, and removes the adapter-specific branching from the runtime core. > - The benefit is a clean seam: the runtime core is now adapter-agnostic for asset provisioning, the inbound behavior is unchanged (same merge matrix, same scripts), and other adapters can attach custom staged files or extract commands without modifying shared infrastructure. ## Linked Issues or Issue Description No pre-existing public GitHub issue. Describing the problem inline per the feature template: **Problem or motivation** The sandbox managed-runtime asset provisioning in `sandbox-managed-runtime.ts` branched directly on the adapter key (`codex`) to decide which merge scripts to stage and which shell command to use during asset extraction. This tight coupling prevents other adapters from customizing their provisioning without modifying the runtime core, and it means the runtime core must import and know about adapter-specific merge scripts. **Proposed solution** Add an optional `provision` contribution (array of `stageFiles` entries + an `extractCommand` string) and an optional `restore` contribution to the asset descriptor returned by adapters. The runtime core now consumes these generically — if a `provision` contribution is present, it stages those files and uses the supplied command; otherwise it falls back to the default `tar -xf` extraction. The Codex adapter populates the `provision` contribution where it previously depended on core branching. **Alternatives considered** Keeping the adapter-specific logic in the core as a documented exception; rejected because it makes the seam inextensible. **Roadmap alignment** Decoupling — removes a latent coupling between the runtime core and a specific adapter. ## What Changed - Added `provision` contribution field (`stageFiles: Array<{src, dest}>` + `extractCommand: string`) to the `SandboxManagedRuntimeAsset` descriptor type in `adapter-utils`. - Added `restore` contribution field (hook for post-restore logic, populated in a later phase) to the descriptor. - Removed adapter-key branching (`if adapterKey === 'codex'`) from the runtime core in `sandbox-managed-runtime.ts`; the core now reads `provision.stageFiles` and `provision.extractCommand` generically. - Extracted Codex-specific merge-script paths and the merge-extract command into `codex-auth-merge-scripts.ts` in `adapter-utils`; the Codex adapter's `execute.ts` now attaches them as a `provision` contribution when it builds its managed-home asset descriptor. - Updated `execution-target.ts` to pass the extended asset type through to the adapter call site so the new fields are load-bearing end-to-end. - Added seam-proving unit tests in `sandbox-managed-runtime.test.ts`: contribution-less asset uses the default path; a non-adapter asset round-trips the generic provision+restore seam; a structural assertion verifies the runtime core carries no Codex-specific string literals. - Added one test in `workspace-restore-merge.test.ts` confirming the inbound merge matrix is unaffected. ## Verification ```bash # Unit tests (20 pass): npx vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts packages/adapter-utils/src/workspace-restore-merge.test.ts # Type-check both affected packages: cd packages/adapter-utils && npx tsc --noEmit cd packages/adapters/codex-local && npx tsc --noEmit # Structural: runtime core carries no adapter string literals grep -n 'codex\|auth\.json' packages/adapter-utils/src/sandbox-managed-runtime.ts # Expected: zero matches ``` ## Risks **Low risk.** This is a behavior-preserving refactor: the inbound provisioning output (which files get staged, which command runs) is identical to before, now driven by the adapter-supplied contribution instead of core branching. The existing inbound merge matrix tests are the regression guard. No change to which bytes cross the sandbox boundary. The SSH transport is untouched. ## Model Used - **Provider:** Anthropic - **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`) - **Context window:** 200 K tokens - **Capabilities used:** tool use (file read/edit, bash execution, Paperclip API), extended reasoning over multi-file TypeScript refactor - **Mode:** agentic (Paperclip ACPX platform) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Harold Kim <harold@paperclip.ing>	2026-07-17 14:18:01 -05:00
DottaandPaperclip	5a5c918705	fix(acpx): configure Codex models at startup (#9700) ... Move Codex ACPX model, reasoning effort, and fast-mode settings into CODEX_CONFIG startup config so arbitrary model IDs avoid ACP session picker validation. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-07-16 11:41:48 -05:00
DottaandPaperclip	d32ed88443	fix(recovery): route recovery by failure cause (#9634) ... ## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI-agent companies and their work. > - Its recovery subsystem detects stranded issue execution and decides whether to retry, escalate, or request operator intervention. > - The existing recovery path used a mostly generic owner ladder and generic execution contract, so transient failures could wake a manager who then performed the deliverable instead of repairing and returning the task. > - Provider quota failures also entered the same takeover path even when the correct action was to wait for capacity and retry the original assignee. > - Recovery actions already retain the source owner and evidence needed to choose a cause-specific route, render a scoped contract, and measure whether work was handed back. > - This pull request adds a cause-keyed recovery playbook, propagates its contract through every built-in adapter, and makes resolved recovery actions return work to the original owner by default. > - The benefit is bounded self-recovery that preserves task ownership, avoids needless management takeover, and makes recovery outcomes observable. ## Linked Issues or Issue Description No matching public GitHub issue was found. Related recovery work was reviewed but is not duplicated here: #9630 restores bounded recovery continuations, #8807 changes one assignee-ranking case, and #9404 records runtime-failure transition evidence. This change instead introduces cause-specific routing and recovery contracts across the recovery lifecycle. ### What happened? When an issue became stranded, recovery generally selected an owner through the same fallback ladder and rendered the normal execution contract. That made the recovery wake look like ordinary deliverable work, even when the correct action was to retry the original agent, repair its runtime, or wait for a provider quota reset. ### Expected behavior Recovery should select a response by failure cause, tell the recipient to recover rather than complete the deliverable, suppress takeover wakes for provider quota waits, and return repaired work to its original assignee unless the recovery owner explicitly completes it. ### Actual behavior Recovery could escalate transient failures to management, omit the cause-specific next action from the wake, and leave the recovery owner assigned after the runtime problem was resolved. ### Impact The generic path creates avoidable management work, ownership churn, and budget consumption while obscuring whether recovery successfully returned work to the responsible agent. ## What Changed - Added cause-keyed routing for process loss, missing disposition, provider quota limits, Codex output inactivity, workspace validation failures, and fallback recovery causes. - Added recovery-scoped wake rendering that replaces the generic execution contract with the failure summary, original assignee, attempt count, next action, and cause-specific playbook instruction. - Propagated the structured recovery contract through all built-in adapter execution paths, including Hermes local and gateway adapters. - Added provider-quota wait monitoring so capacity failures schedule the original assignee instead of enqueueing a takeover wake. - Added hand-back behavior and `handed_back` / `owner_completed` outcome accounting when recovery actions are resolved. - Added focused routing, renderer, quota-monitor, and hand-back regression coverage plus implementation-spec documentation. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/server-utils.test.ts server/src/__tests__/heartbeat-process-recovery.test.ts server/src/__tests__/heartbeat-workspace-branch-containment.test.ts server/src/__tests__/issue-recovery-actions.test.ts` - 4 test files passed; 194 tests passed. - Targeted `pnpm --filter ... typecheck` across `@paperclipai/adapter-utils`, `@paperclipai/shared`, `@paperclipai/server`, `@paperclipai/ui`, and all nine changed adapter packages. - 13 affected workspace packages passed typecheck. - `pnpm check:token-gates` - All UI token gates passed. ## Risks - Recovery routing behavior changes for stranded work, so an incorrectly classified cause could select a different recipient than before; fallback causes retain the existing management ladder. - Provider quota detection depends on structured failure evidence and conservative text matching; unmatched failures continue through fallback recovery. - Adapter prompt plumbing changes across built-ins, covered by shared renderer tests and compile-time call signatures. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with exact model ID `gpt-5.6-sol`, using reasoning, tool use, and code execution. The runtime does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-15 20:04:42 -05:00
3 people	3ae2c30f2f	feat(skills): import skills from projects (#9620) ... ## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies. > - Company skills make reusable agent behavior discoverable and editable from one place. > - Projects already contain skill directories, but operators had to import each skill path manually. > - Copying those skills would break the desired write-through workflow between Skill Studio and the source project. > - The server therefore needs a safe preview/select/import contract that only accepts rediscovered, workspace-contained candidates. > - The UI needs a guided project picker that explains reference semantics, handles conflicts, and remains usable on mobile. > - This pull request adds that end-to-end project skill import flow with authorization, tenant-scope, traversal, and symlink regression coverage. > - The benefit is faster bulk onboarding while keeping project files as the single source of truth. ## Linked Issues or Issue Description **Feature request** **Problem:** Importing several skills already stored in a Paperclip project requires operators to discover and submit each local path individually. This is slow, hides which well-known directories were searched, and makes conflict/already-imported states difficult to evaluate before mutation. **Proposed solution:** Add an “Import skills from project” flow that previews skills from well-known directories, lets operators selectively import eligible candidates, and stores local-path references so Skill Studio edits write through to the project files. **Alternatives considered:** Copying files into company-managed skill storage was rejected because it creates divergent copies. Trusting client-supplied paths was rejected because imports must be constrained to server-rediscovered, workspace-contained candidates. **Additional context:** GitHub duplicate search found no existing issue or PR for this exact workflow. Refs #3799 for related skill-import inventory behavior; this PR does not claim to close that issue. ## What Changed - Extend `scan-projects` with backward-compatible preview and selective-import modes, typed validation, candidate statuses, and OpenAPI coverage. - Discover project skills under `skills`, `.agents/skills`, `.claude/skills`, `.codex/skills`, `.cursor/skills`, `.opencode/skills`, and `.gemini/skills`. - Re-discover selections server-side, enforce company/project/workspace scope, and reject traversal or symlink escapes before creating `local_path` references. - Add the Skills-page menu entry and responsive project import dialog with project selection, grouped candidates, select all/deselect all, conflicts, empty/error/403 states, and import results. - Add route, service, and component regressions for preview authorization, cross-tenant selections, traversal/symlink safety, selection counts, grouping, and result semantics. ### Screenshots **Choose a project**  **Review discovered skills**  **Mobile selection footer**  **Import result**  ## Verification - `pnpm exec vitest run server/src/__tests__/company-skills-service.test.ts server/src/__tests__/company-skills-routes.test.ts ui/src/pages/skills/ImportSkillsFromProjectDialog.test.tsx` — 3 files, 81 tests passed. - `pnpm check:token-gates` — all token gates clean. - `pnpm --filter @paperclipai/shared typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - Security review passed after adding tenant-scope and unauthorized-preview regressions; UX re-review approved desktop/mobile surfaces; QA passed all seven acceptance areas including write-through editing, deduplication, conflicts, empty state, and permission denial. ## Risks - Files remain referenced in project workspaces, so moving or deleting a source directory can make an imported skill unavailable; the UI explicitly communicates the reference behavior. - New well-known directory scans may discover more candidates than older versions, but preview mode prevents mutation until the operator confirms a selection. - The endpoint remains backward compatible: omitting `mode` preserves the prior full-import behavior. - No schema migration or telemetry event changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Anthropic Claude Opus 4.8 with tool use/code execution assisted with the UI implementation and UX polish. OpenAI Codex CLI with tool use/code execution assisted with server implementation, security fixes, regression coverage, integration, and PR preparation; the runtime did not expose Codex's exact backing model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-15 18:01:44 -05:00
Nicky Leach	3a727bf780	fix(codex): warn when sandbox auth is shadowed (#9259)	2026-07-15 10:02:28 -07:00
DottaandPaperclip	0ecae2cd7e	fix(adapters): forward resolved adapter env to local agents, keep runtime env authoritative (#9617) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Local coding adapters (Claude, Codex) run each agent heartbeat in a spawned child process; Paperclip resolves adapter-configured env — including `secret_ref` bindings — into the env for that process > - The resolved adapter env was not being forwarded reliably: config env could overwrite Paperclip's own runtime vars, and a warm/resumable ACP session could keep serving stale env because its fingerprint ignored the resolved env > - This mattered because a configured API key or other secret could be silently absent from the agent shell, and a resumed session would never pick up an updated value — while a config binding could also override runtime identity/wake vars > - This pull request keeps Paperclip-managed `PAPERCLIP_*` runtime env authoritative over config, and folds a stable hash of the applied adapter env into the session fingerprint so an env change forces a fresh launch > - The benefit is that adapter-configured env (plain values and resolved secrets) reliably reaches the agent process, updates are picked up on the next launch, and runtime identity can never be clobbered by config ## Linked Issues or Issue Description No public GitHub issue exists, so the underlying bug is described inline following the bug-report template. **What happened** Env keys configured on a local adapter (plain values and `secret_ref` bindings, resolved server-side into plain strings) did not reliably reach the spawned agent process. Two distinct gaps: (1) when merging config env into the process env, a config key in the reserved `PAPERCLIP_*` namespace could overwrite a Paperclip-managed runtime variable (identity, wake, workspace, API access); (2) a warm-handle / resumable ACP session computed its reuse fingerprint from `secretManifestHash` only, which misses plain-value edits and same-version secret rotations — so a resumed session kept serving stale env and never re-launched with updated values. **Expected behavior** Non-`PAPERCLIP_*` adapter env (plain + resolved secret values) is forwarded to the agent process; a change to any applied forwarded value invalidates a warm/resumable session so the next launch sources the latest env; configured `PAPERCLIP_*` entries can never override Paperclip runtime env, while an explicitly configured `PAPERCLIP_API_KEY` (stable per-run config) is still honored and its rotation also busts the session. **Steps to reproduce** Configure an adapter with an env key (e.g. a `secret_ref` API key) and a resumable ACP session. On resume, the updated env value is not sourced; separately, a `PAPERCLIP_*` config key overrides the runtime value. **Deployment mode** Local adapters (Claude / Codex) via the shared adapter-utils execution path. ## What Changed - `packages/adapter-utils/src/server-utils.ts`: add `isPaperclipRuntimeEnvKey` and, in `refreshPaperclipWorkspaceEnvForExecution` (used by all local adapters), skip a `PAPERCLIP_*` config key when Paperclip has already assigned it this run; all other keys still forward. - `packages/adapter-utils/src/acpx-engine/execute.ts`: apply the same `PAPERCLIP_*` non-override rule (via the shared helper) when merging config env, capture the applied config env in `resolvedAdapterEnv`, and fold a stable `adapterEnvHash` of it into the session fingerprint so an env change forces a fresh launch. Per-wake `PAPERCLIP_*` runtime vars are assigned earlier and never enter that map, so they stay out of the hash; stable configured `PAPERCLIP_*` values (e.g. an explicit `PAPERCLIP_API_KEY`) are included so rotating one busts the session. - Added unit/integration tests for plain + secret forwarding, `PAPERCLIP_*` non-override, the explicit-API-key path, fingerprint refresh-on-env-change vs. stable-across-wakes, and rotation of a configured `PAPERCLIP_API_KEY`. ## Verification - `npx vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts packages/adapter-utils/src/server-utils.test.ts` → 2 files, 111 tests passing (includes the new cases). - Tests assert: forwarded plain/secret values appear in the spawned wrapper `.env`; a `PAPERCLIP_*` config key does not override the runtime value; changing an applied forwarded env value (including a rotated `PAPERCLIP_API_KEY`) changes `configFingerprint`, while a new wake with the same config env keeps it stable. ## Risks Low risk. Behavior change is limited to (a) config env no longer overriding `PAPERCLIP_*` runtime vars — a security-positive tightening — and (b) a resumable session re-launching when its applied config env changes, which is the intended fix. Per-wake `PAPERCLIP_*` churn is deliberately excluded from the fingerprint so normal sessions still resume across heartbeats. Existing sessions get a new fingerprint once on first deploy (the added `adapterEnvHash` field), which is expected. No secret values are logged (key-name redaction in the adapter plus manifest-driven redaction of `meta.env`). ## Model Used Claude Opus 4.8 (Anthropic), model id `claude-opus-4-8`, extended reasoning with tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-15 11:26:47 -05:00
Nicky LeachandPaperclip	7947308276	fix(codex): classify refresh auth failures (#9598) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip supports the Codex local adapter, which runs OpenAI Codex CLI sessions on behalf of agents > - Codex uses OAuth refresh tokens to maintain long-running authenticated sessions > - When a refresh fails, the failure has distinct root causes: a refresh token was already reused in a parallel request, the token expired by TTL, or the token was invalidated/revoked by the provider > - Without classifying these failure modes, all refresh auth errors surface identically — operators cannot distinguish retryable transient collisions from permanent invalidations, and run logs carry no actionable diagnosis > - This pull request adds structured classification (`refresh_token_reused`, `refresh_token_expired`, `refresh_token_invalidated`) of Codex refresh-token auth failures across the CLI quota-probe, ACP auth path, and execute path > - The benefit is that these distinct failure modes can be surfaced in run logs and acted on appropriately — transient reuse can be retried; true invalidations require re-auth ## Linked Issues or Issue Description <!-- Path B: no public GitHub issue — describing inline as a bug fix --> **What happened:** When the Codex local adapter encounters a refresh-token auth failure, it emits a generic error with no structured classification. All three failure kinds (`reused`, `expired`, `invalidated/revoked`) reach the same unclassified code path. **Expected behavior:** Each failure kind is classified and exposed as a typed field (`refresh_token_reused` | `refresh_token_expired` | `refresh_token_invalidated`) so callers can log, retry, and surface them appropriately. **Steps to reproduce:** 1. Run a Codex agent session with a reused or expired OAuth refresh token. 2. Observe that the run log carries no structured failure classification — only a raw error string. **Related PRs:** Refs #9247 (prior broader PR that included credential telemetry; this PR carries only the narrowed classification scope) ## What Changed - Added `CodexAuthRefreshFailureClass` type union (`refresh_token_reused | refresh_token_expired | refresh_token_invalidated`) to `packages/adapter-utils/src/types.ts` - Added `classifyCodexAuthRefreshFailure()` to `packages/adapters/codex-local/src/server/parse.ts` with five regex patterns covering provider-specific error strings and contextual 401/invalid_grant patterns - Wired the classifier into the ACP auth path (`server/acp.ts`), execute path (`server/execute.ts`), and CLI quota-probe (`cli/quota-probe.ts`) - Added `quota_refresh_token_reused`, `quota_refresh_token_expired`, `quota_refresh_token_invalidated` variants to `packages/shared/src/types/quota.ts` - Added classification unit tests (`parse.test.ts`, `quota-spawn-error.test.ts`, `acp.test.ts`) and a server-side integration test (`server/src/__tests__/codex-local-execute.test.ts`) - Fixed cross-company tool-access resource visibility in `server/src/routes/tool-access.ts` - Stabilized `heartbeat-retry-scheduling.test.ts` (CASCADE cleanup), `heartbeat-run-log.test.ts`, and `quota-windows.test.ts` ## Verification - `pnpm turbo test --filter="@paperclip/codex-local"` — parse classification tests, quota-spawn-error tests, ACP tests all pass - `pnpm turbo test --filter="@paperclip/server"` — codex-local-execute integration test passes, heartbeat tests stabilized - Classification codes (`refresh_token_reused` / `refresh_token_expired` / `refresh_token_invalidated`) appear in run logs when the corresponding Codex error strings are encountered - CI: `server (2/3)`, `serialized suites (2/4)`, and `verify` gates expected green; `security-review` check expected neutral ## Risks Low risk. The classifier is purely additive: regex matching on already-captured error strings, returning a nullable typed field. Callers that do not inspect the classification field are unaffected. No execution paths, retry logic, or existing error surfaces changed. ## Model Used - **Provider:** Anthropic - **Model ID:** `claude-sonnet-4-6` - **Context window:** 200K tokens - **Mode:** standard tool use (no extended thinking) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-14 22:40:38 -07:00
4 people	b79f744a8d	Fix Codex auth merge host-unusable fail closed (#9276) ... ## Thinking Path > - Paperclip is the open source platform people use to manage AI agents for work > - The Codex adapter runs agent tasks in isolated sandbox environments on the user's machine > - When a Codex sandbox is reused across agent runs, its home directory (including `~/.codex/auth.json`) is restored from a prior snapshot > - Both the host machine and the sandbox independently maintain `auth.json` credentials; on sandbox reuse, these can diverge > - The previous merge code had fail-open edge cases: if host auth was in an unusable state, if the auth JSON object shapes differed between host and sandbox, or if the subscription account identities didn't match, the merge would proceed silently with whatever data was available > - This PR adds fail-closed behavior: if host Codex auth is unusable, if auth parser shapes differ, or if subscription account identities don't match, the merge fails explicitly rather than silently continuing with stale or incorrect credentials > - The benefit is that Codex agents on reused sandboxes now fail fast and loudly when auth is in a broken state, instead of silently running with wrong credentials and producing confusing downstream failures ## Linked Issues or Issue Description No pre-existing public GitHub issue. This is a targeted security hardening fix for the Codex reused-sandbox auth merge path. **Problem:** When a Codex sandbox is reused, the merge logic that reconciles host and sandbox `auth.json` credentials failed open in several cases: - Host `auth.json` present but in an unusable state (missing required keys, empty token material, malformed JSON) → merge would proceed with whatever the sandbox had - Host and sandbox auth payloads had different shapes (e.g., one uses `OPENAI_API_KEY`, the other uses a `tokens` object) → parser-differential case not detected - Subscription account identities (`tokens.account_id`) differed between host and sandbox → stale sandbox identity would be used silently **Fix:** All three cases now fail closed. The merge returns an explicit error rather than proceeding with potentially stale or mismatched credentials. Related PRs: - Refs #9262 — sandbox Codex auth shadow warning (adjacent auth area) - Refs #9259 — auth precedence exports (adjacent auth area) ## What Changed - `packages/adapters/codex-local/src/server/codex-home.ts` — New file with `hasUsableAuthPayload()`, `codexHomeHasUsableAuth()`, and full Codex home setup/teardown. Includes fail-closed auth merge guards: rejects unusable host auth, detects parser shape differentials, and checks subscription account identity match before merging - `packages/adapter-utils/src/workspace-restore-merge.ts` — New file with directory snapshot diffing and restore-merge logic; the merge operation fails closed when auth validation fails - `packages/adapters/codex-local/src/server/codex-home.test.ts` — Unit tests covering auth usability checks, symlink management, and fail-closed merge paths - `packages/adapter-utils/src/workspace-restore-merge.test.ts` — Unit tests for snapshot/restore-merge behavior including fail-closed cases - `packages/adapter-utils/src/sandbox-managed-runtime.ts` — Updated to invoke the fail-closed auth merge during sandbox restore ## Verification Tests run and passing: ```sh corepack pnpm exec vitest run packages/adapter-utils/src/workspace-restore-merge.test.ts packages/adapters/codex-local/src/server/codex-home.test.ts corepack pnpm exec vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts corepack pnpm --filter @paperclipai/adapter-utils typecheck corepack pnpm --filter @paperclipai/adapter-codex-local typecheck git diff --check origin/master HEAD ``` All passed locally before push. ## Risks - **Intentional behavioral change (breaking for previously-silent failures):** Reused sandboxes that previously completed auth merge with unusable host auth, parser-differential auth shapes, or mismatched account identities will now fail with an explicit error. This is the correct behavior — the prior silent-proceed path was the bug. Users affected will see a clear error message rather than a confusing downstream auth failure. - **Auth.json symlink migration:** `ensureSymlink()` detects stale copied `auth.json` files (written by older Paperclip versions) and replaces them with symlinks on first run. This is safe: the target is always under the Paperclip-managed company home, never the user's real `~/.codex`. Directories at the symlink path are left untouched (EISDIR is not silently swallowed). - **Low risk for non-reuse paths:** The fail-closed logic only activates during sandbox restore/reuse. Fresh sandbox allocations are unaffected. ## Model Used - **Provider:** Anthropic - **Model ID:** claude-sonnet-4-6 (Claude Sonnet 4.6) - **Context window:** 200k tokens - **Mode:** Agentic coding with tool use; extended thinking not used - **Role:** Code author (Priya Raman, BackendEngineer) with Harold Kim (Git Expert) handling push and PR operations ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Priya Raman <priya.raman@paperclip.local> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Harold Kim <harold@paperclip.ing>	2026-07-14 15:49:36 -07:00
DottaandPaperclip	931eec3fbf	feat(mcp) [split 4/8]: wire gateway runtime and Smoke Lab (#9559) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Governed MCP access spans contracts, runtime enforcement, adapters, UI surfaces, and operator verification > - The parity reference PR #9534 is too large for effective automated or human review > - The feature therefore needs a linear stack whose individual diffs stay below the 100-file review limit > - This pull request is split 4/8 and focuses on gateway runtime, Smoke Lab, plugins, and server wiring > - The benefit is a standalone, testable review boundary while preserving byte-for-byte parity at the top of the stack ## Linked Issues or Issue Description - Related parity reference: #9534 - Problem: The policy core needs runtime execution, endpoint guards, route registration, heartbeat integration, and adapter MCP injection to become operational. - Proposed solution: Adds the remaining server routes/wiring/consumers, runtime tests, adapter-utils MCP contracts, and Claude/Codex injection implementations required by the server layer. - Alternatives considered: keeping #9534 as one 403-file review, or rewriting the feature to manufacture seams; both were rejected in favor of path extraction plus compile-driven boundary moves. - Roadmap alignment: this advances the existing governed MCP/tool-access work already represented by #9534; it does not introduce a separate roadmap initiative. - Stack position: base branch is `pap10341-split/03-server-tool-access`. - Merge policy: merge bottom-up, in order, only after the complete eight-PR stack has been reviewed and the top-of-stack parity gate remains empty. - Requested review: SecurityEngineer for gateway, endpoint guard, token issuance, and runtime wiring; Greptile on every PR. ## What Changed - Adds the remaining server routes/wiring/consumers, runtime tests, adapter-utils MCP contracts, and Claude/Codex injection implementations required by the server layer. - Keeps this PR below 100 changed files and independently typecheckable. - Preserves the final tree from #9534 when combined with the other seven stack levels. ## Verification - `pnpm typecheck` - Changed server test set — 26 files, 382 tests passed - Affected server adapter tests — 38 tests passed after concrete adapter boundary move - Adapter-utils and Codex focused tests — 76 tests passed ## Risks - Remote endpoint validation, token handling, and runtime supervision are security-sensitive and can fail closed or deny legitimate access if misconfigured. - Stack risk: merging out of order can expose incomplete layers; mitigate by following the documented bottom-up merge policy. - Parity risk: later edits to an intermediate branch can drift from #9534; mitigate by re-running the empty top-of-stack diff before merge. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context window; medium reasoning with repository, shell, Git, GitHub CLI, and code-execution tools enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] Internal references are omitted except the execution-plan link explicitly required for this coordinated split stack - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge ## Stack Coordination - Internal execution plan: [PAP-13874](/PAP/issues/PAP-13874#document-plan) - Parity reference: #9534 - Stack: #9556 → #9557 → #9558 → #9559 → #9560 → #9561 → #9562 → #9563 - Merge bottom-up only after full-stack review and an empty parity diff at #9563. --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-14 15:07:30 -05:00
DottaandPaperclip	98d9360658	feat(adapters): confine local coding processes (#9504) ... ## Thinking Path > - Paperclip is the open source control plane people use to manage AI agents for work. > - Local coding adapters currently spawn their CLI processes directly on the Paperclip host. > - CLI-native approval and sandbox flags do not provide a reliable host filesystem or network boundary. > - An agent can therefore inspect unrelated host files or fetch external material when an operator needs stronger isolation. > - The confinement must stay opt-in so existing local adapter behavior does not change unexpectedly. > - This pull request adds a shared Linux Bubblewrap spawn layer for workspace filesystem and deny/allowlist network scopes. > - The benefit is enforceable defense in depth around Codex and Claude local runs while preserving explicit provider connectivity. ## Linked Issues or Issue Description ### What happened? `codex_local` and `claude_local` processes could read arbitrary host paths and make unrestricted outbound network requests because Paperclip did not impose a spawn-level boundary. ### Expected behavior Operators can opt into a workspace-only filesystem view and either deny network egress or allow exact provider/API hosts, independently of CLI approval flags. ### Steps to reproduce 1. Run current `master` on Linux and configure a Codex or Claude local adapter. 2. Ask the agent to read a canary file outside its active workspace. 3. Ask the agent to `curl` a public host. 4. Observe that both operations succeed without a Paperclip-level confinement option. ### Environment - Paperclip commit: `c36f1a4af` / current `master` base. - Deployment mode: Linux local dev or self-hosted server. - Installation: built from source. - Adapters: Codex and Claude Code. - Database: not related. ## What Changed - Added a shared Bubblewrap process wrapper with opt-in `filesystemScope: "workspace"`, managed/extra path mounts, private `/tmp`, and Linux-only validation. - Added `networkScope: "deny" | "allowlist"`; both use a private network namespace, while allowlist mode exposes an exact-host HTTP(S) proxy over a Unix-socket bridge. - Wired Codex and Claude local CLI execution through the wrapper and forced scoped auto runs onto the CLI lane because ACP processes are not covered. - Added unit and gated Bubblewrap canaries for outside-file denial, workspace writes, direct network denial, allowlisted forwarding, and rejected destinations. - Documented both scopes, provider allowlist examples, Bubblewrap requirements, and default-off behavior. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/local-process-sandbox.test.ts packages/adapters/codex-local/src/server/acp.test.ts packages/adapters/claude-local/src/server/acp.test.ts` — 34 passed, 4 gated Bubblewrap tests skipped by default. - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/adapter-codex-local typecheck` - `pnpm --filter @paperclipai/adapter-claude-local typecheck` - `pnpm --filter @paperclipai/adapter-utils build` - `pnpm --filter @paperclipai/adapter-codex-local build` - `pnpm --filter @paperclipai/adapter-claude-local build` - Attempted the gated tests with a vendored Bubblewrap binary; this container blocks unprivileged namespace setup (`setting up uid map: Permission denied` / loopback `RTM_NEWADDR: Operation not permitted`), so kernel-level execution remains for CI or a namespace-enabled Linux host. ## Risks - Bubblewrap must be installed and unprivileged user/mount/network namespaces must be enabled on the host; scoped runs fail clearly if the prerequisite is missing. - Allowlist mode depends on the coding CLI honoring standard `HTTP_PROXY` / `HTTPS_PROXY` variables; custom providers must list every required exact hostname and port. - Exact-host allowlists intentionally reject wildcards, which is safer but may require operators to enumerate multi-host provider setups. - No behavior changes unless an operator enables `filesystemScope` or `networkScope`. > This aligns with the ROADMAP direction toward safer remote and sandboxed agent environments and does not duplicate an open PR or issue found in the repository search. ## Model Used - OpenAI GPT-5.5 (`gpt-5.5`) via Codex CLI, with reasoning, repository tool use, shell execution, code editing, and test execution. The serving context-window size is not exposed to the agent. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-14 11:01:19 -05:00
DottaandPaperclip	c8253e3641	fix(adapters): inject execution contract once per fresh heartbeat (#9469) ... ## Thinking Path > - Paperclip coordinates AI-agent work through repeated heartbeat runs. > - Adapter prompts combine a default heartbeat template with scoped wake context. > - Fresh heartbeats received the same execution contract from both layers, wasting prompt tokens and obscuring which layer owns the contract. > - Resume deltas and template-less adapters do not share that composition path, so removing the wake-payload copy unconditionally would drop required guidance. > - Empty comment batches also emitted instructions and metadata that only matter when comments exist. > - This pull request makes execution-contract inclusion explicit by prompt path, preserves OpenClaw gateway behavior, and suppresses no-op comment boilerplate. > - The benefit is one contract per heartbeat path and roughly 300 fewer prompt tokens on a fresh zero-comment wake. ## Linked Issues or Issue Description - Fixes #9221 - Refs #9200 - Refs #7634 ## What Changed - Stop emitting the execution-contract paragraph from fresh scoped wake payloads because the default heartbeat template already contains the full contract. - Keep the contract in resume deltas, and add `includeExecutionContract` for adapters that do not render the default heartbeat template. - Opt `openclaw-gateway` into wake-payload contract rendering so template-less gateway runs retain the guidance. - Omit comment-batch acknowledgement/fetch guidance and empty `pending comments` / `latest comment id` metadata when a fresh wake has no pending comments. - Add regression and acceptance coverage proving composed fresh prompts contain `Execution contract` exactly once while resume and template-less paths retain it. Measured effect: the fresh zero-comment wake block drops from 1,840 to 855 characters (about 300 tokens saved per fresh heartbeat; about 220 on comment wakes), and the composed fresh prompt contains `Execution contract` once instead of twice. ## Verification - `npx vitest run packages/adapter-utils/src/server-utils.test.ts` — 63 passed - `npx vitest run server/src/__tests__/codex-local-execute.test.ts` — 13 passed - `npx vitest run server/src/__tests__/heartbeat-comment-wake-batching.test.ts server/src/__tests__/openclaw-gateway-adapter.test.ts server/src/__tests__/low-trust-red-team-routes.test.ts` — 27 passed - `pnpm --filter @paperclipai/adapter-utils typecheck` — passed - `pnpm --filter @paperclipai/adapter-openclaw-gateway typecheck` — passed ## Risks - Low risk: prompt text and adapter composition only; no database or API migration. - The main compatibility risk is a template-less adapter losing the contract. The explicit option and OpenClaw gateway regression coverage protect the known template-less path. - External adapters that call `renderPaperclipWakePrompt` directly can opt into `includeExecutionContract: true` when they do not render the default template. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5.4, reasoning mode with tool use and code execution; context-window size is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-12 20:41:52 -05:00
3 people	9e7e84e3fe	fix(adapters): propagate ACP-lane usage and cost into spend telemetry (#9471) ... ## Thinking Path > - Paperclip is the open source control plane for running and governing AI-agent companies. > - Adapter executions feed token usage, billing identity, and run cost into the control plane's spend telemetry. > - The default ACP execution lane for local Claude and Codex adapters did not propagate per-turn usage or cost, so paid runs could be recorded with zero spend and no tokens. > - Claude CLI result events could also undercount output tokens by reading only the main-loop usage block instead of the complete per-model ledger. > - The shared executor needs to distinguish per-run usage from session-cumulative usage so the server does not apply the wrong delta heuristic. > - This pull request captures ACP usage and cumulative-cost deltas, resolves adapter billing identity, uses Claude's complete model-usage ledger, and preserves per-run usage in server normalization. > - The benefit is accurate token and cost accounting across the default paid Claude and Codex execution paths. ## Linked Issues or Issue Description ### What happened? Paid `claude_local` and `codex_local` runs using the default ACP engine can complete successfully while the control plane records zero or null cost and missing token usage. Claude CLI result parsing can additionally undercount output tokens when subagent or sidechain usage is present. ### Steps to reproduce 1. Run a paid Claude or Codex local adapter through the ACP engine. 2. Complete a turn that reports usage and cumulative cost through ACP status/events. 3. Inspect the execution result and normalized run telemetry. ### Expected behavior The execution result contains per-turn token usage, a per-run USD cost delta, and the correct billing identity. Server normalization records those per-run values without applying a session-cumulative delta a second time. ### Actual behavior before this change ACP execution results returned no usage and `costUsd: null` with unknown billing. The server therefore recorded zero spend and no tokens for paid runs. Claude CLI parsing could use an incomplete usage block. ## What Changed - Capture ACP usage from runtime status and `usage_update` events, reporting it as `usageBasis: per_run`. - Convert agent-reported cumulative ACP cost into a per-turn delta, including counter-reset and no-report safeguards. - Add a shared billing-identity resolver and map Claude and Codex authentication/provider modes to control-plane billing types. - Prefer Claude result-event `modelUsage` totals so subagent and sidechain tokens are included. - Skip the server's session-cumulative usage delta when an adapter explicitly reports per-run usage. - Add regression coverage for usage capture, event fallback, cost resets, stale reports, billing identities, model-usage totals, and server spend normalization. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts packages/adapters/claude-local/src/server/parse.test.ts packages/adapters/claude-local/src/server/acp.test.ts packages/adapters/codex-local/src/server/acp.test.ts server/src/__tests__/costs-service.test.ts server/src/__tests__/monthly-spend-service.test.ts` — 6 files, 126 tests passed. - `pnpm --filter @paperclipai/adapter-utils typecheck` — passed. - `pnpm --filter @paperclipai/adapter-claude-local typecheck` — passed. - `pnpm --filter @paperclipai/adapter-codex-local typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - A broader Claude-local suite has a pre-existing rate-limit classification failure in `test.probe.test.ts`; it also fails on clean `master` and is unrelated to this change. ## Risks - Cost reporting depends on the agent's cumulative counter semantics; reset handling falls back to the post-turn amount and is covered by regression tests. - Incorrect billing-mode inference could misclassify spend; provider/auth mappings mirror each adapter's existing CLI behavior and have focused tests. - The new `usageBasis` contract changes server normalization only when adapters explicitly opt into `per_run`; existing adapters retain prior behavior. - No database migration, workflow, lockfile, or UI changes are included. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Implementation commit: Anthropic Claude Fable 5, tool-enabled coding workflow (exact context window and runtime configuration were not recorded in the commit metadata). - PR preparation and verification: OpenAI Codex, tool-enabled coding agent (runtime model ID and context window are not exposed to this session). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-12 20:37:22 -05:00
3 people	9cde4e128c	feat: run ACP sessions in sandbox execution targets (#9390) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent adapters (Claude, Codex, Gemini) default to the ACP engine lane, which needs a live bidirectional stdio session with the agent process > - Sandbox execution targets only exposed one-shot command execution, so every ACP-capable adapter refused remote targets and fell back to the CLI lane with a "supports only the local Paperclip host" warning > - Running agents in sandboxes is a core deployment mode, and losing ACP there means losing streaming updates, structured events, and default-lane parity with local runs > - This pull request adds a provider-agnostic process-session bridge that relays the ACP stdio session into the sandbox over the existing sandbox runner contract, and updates the adapters to use it > - The benefit is that the default ACP lane now behaves the same on the local host and in any sandbox provider, with CLI fallback reserved for targets that genuinely cannot host a bidirectional session ## Linked Issues or Issue Description No existing public issue covers this; inline description following the feature request template: **Problem or motivation** Configuring an ACP-capable adapter (e.g. Claude) with a sandbox environment made every run fall back to the CLI lane with the warning "Claude ACP currently supports only the local Paperclip host, but this run targets a remote environment." The ACP engine only knew how to spawn a local subprocess, while sandbox providers only expose one-shot command execution — so there was no way to hold the bidirectional stdio session ACP requires. **Proposed solution** Add a process-session bridge in `adapter-utils`: a local ACPX-spawnable proxy script connects to a token-authenticated loopback TCP server, which relays JSON-framed stdin/stdout/stderr events to and from a small relay script executed inside the sandbox via the provider's ordinary runner. Claude/Codex/Gemini adapters now treat sandbox targets with a runner as ACP-capable, resolve agent commands against the remote target, and fall back to CLI only when the sandbox exposes no bidirectional path. The sandbox callback bridge injects a run-scoped API endpoint and bridge token so the agent inside the sandbox can reach Paperclip (including work-product handoffs) without ever receiving the host run JWT. **Alternatives considered** A provider-specific lane was prototyped first: Daytona minting SSH access metadata at lease time, converted into an SSH execution target. It was dropped because it only worked for providers able to advertise SSH, added per-provider surface area, and left every other sandbox provider on the CLI fallback. The merged design rides the one-shot runner contract all providers already implement; a regression test pins that sandbox targets stay on the bridge lane even when lease metadata advertises SSH access. **Roadmap alignment** Directly advances the "Cloud / Sandbox agents" roadmap item — agents running in remote and sandboxed environments keep the same control-plane behavior as local ones. No overlap with other planned core work. ## What Changed - `packages/adapter-utils/src/execution-target.ts`: new `startAdapterExecutionTargetProcessSessionBridge()` plus helpers — writes a token-authenticated local proxy script (spawnable by ACPX) and a remote relay script synced into the sandbox, with a loopback TCP server streaming JSON-framed stdio between them; events emitted before the ACP client attaches are buffered so none are lost. - `packages/adapter-utils/src/acpx-engine/execute.ts`: the ACP engine can execute against remote sandbox targets through the bridge instead of requiring a local subprocess, including remote cwd/env shaping. - `packages/adapter-utils/src/sandbox-callback-bridge.ts`: sandbox-scoped API bridging extended to allow work-product handoffs; the sandbox payload env carries a bridge token, never the host run JWT. - `packages/adapters/claude-local`, `codex-local`, `gemini-local` (`src/server/acp.ts`): default-lane selection no longer rejects all remote targets; command resolution is remote-aware (`ensureAdapterExecutionTargetCommandResolvable`, `resolveAdapterExecutionTargetCwd`); the fallback reason is now scoped to sandboxes that expose only one-shot execution. - `server/src/__tests__/environment-execution-target.test.ts`: pins that sandbox targets resolve to the bridge lane, including when lease metadata advertises SSH access. - Non-sandbox remote targets (e.g. SSH) keep the CLI lane: the ACP engine's remote transport is sandbox-only, so default-lane selection falls back for those targets across all three adapters, and tests covering CLI-specific remote behavior pin `engine: "cli"` explicitly. - The bridge authenticates loopback connections before they can own the session or receive buffered output (token required, idle unauthenticated peers dropped), and remote event writes are serialized so the exit event always lands after stdout/stderr have drained. - Daytona plugin: formatting-only residue from the earlier iteration; no functional change. ## Verification - `vitest run` over the touched suites — `packages/adapter-utils/src/acpx-engine/execute.test.ts`, `packages/adapter-utils/src/execution-target-sandbox.test.ts`, `packages/adapter-utils/src/sandbox-callback-bridge.test.ts`, the three adapter `acp.test.ts` files, and `server/src/__tests__/environment-execution-target.test.ts` — 102 tests pass. - End to end: with a Claude agent configured on a Daytona sandbox environment, the primary-model test now selects the default ACP lane (no fallback warning), and the full round trip (wake → sandbox execution → API bridge → comment post) was exercised twice from inside a live sandbox. ## Risks - Behavioral shift: adapters that previously always fell back to CLI on sandbox targets now default to ACP there; `engine=cli` still pins the CLI lane explicitly. - The bridge relays stdio as JSON lines over loopback TCP guarded by a per-session random token; the remote relay runs inside the sandbox under the provider's runner. Providers with slow one-shot execution will see higher session startup latency — the CLI fallback remains for genuinely incapable targets. - No schema or migration changes. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic) — extended thinking enabled, agentic tool use via the Claude Agent SDK harness; implementation iterated with local Vitest verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no shipped docs describe the old local-only ACP limitation) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Cody <noreply@paperclip.ing> Co-authored-by: Cody <cody@paperclip.local>	2026-07-10 17:13:53 -07:00
Justin ToddandAndrew Aymeloglu	1e8ede4e1e	fix(adapter-utils): runChildProcess escalates to SIGKILL on liveness, not child.killed (#8598) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Those agents run as child processes spawned by `@paperclipai/adapter-utils`'s `runChildProcess`, which arms a parent-side wall-clock timer at `timeoutSec` to bound a hung run > - At the deadline `runChildProcess` sends SIGTERM, then after a grace window escalates to SIGKILL — the SIGKILL backstop is what turns a wedged child into a dead PID so the scheduler can reclaim and retry it > - On the **direct-child fallback** path (`signalRunningProcess`, used on win32 and whenever process-group signaling is unavailable or throws) the escalation was gated on `!child.killed` > - But Node sets `ChildProcess.killed` to `true` the instant a signal is *successfully sent*, not when the process exits — so once the earlier SIGTERM has been sent, `child.killed` is already `true`, the `!child.killed` guard is `false`, and the SIGKILL escalation never runs > - A child that ignores SIGTERM (e.g. a graceful-shutdown handler wedged on a socket) is therefore never force-killed, outlives its deadline, and for an unattended scheduler sits running forever with no terminal state > - This PR gates the fallback escalation on real liveness (`exitCode === null && signalCode === null`), so SIGKILL fires precisely while the child is still alive > - The benefit is the hard timeout actually guarantees termination (except true uninterruptible D-state) on every platform/configuration, not just where the process-group path is available ## Linked Issues or Issue Description No existing public issue — describing the bug inline, following `.github/ISSUE_TEMPLATE/bug_report.yml`: ### What happened? When `@paperclipai/adapter-utils`'s `runChildProcess` reaches `timeoutSec` and the spawned child ignores SIGTERM, the SIGKILL escalation on the **direct-child fallback** path (`signalRunningProcess`, taken on win32 or whenever `process.kill(-pgid, …)` is unavailable or throws) never fires, so the child outlives its deadline indefinitely. Root cause: the escalation is gated on `!running.child.killed`, and `ChildProcess.killed` reflects only that a signal was *successfully sent* (per the Node docs it "does not indicate that the child process has been terminated"). After the deadline SIGTERM, `child.killed` is already `true`, so `!child.killed` is `false` and the follow-up SIGKILL is suppressed. ### Expected behavior After the grace window, a child that is still alive is force-killed with SIGKILL regardless of whether SIGTERM was already sent — the hard timeout should guarantee termination (except true uninterruptible D-state) on every platform/configuration. ### Steps to reproduce 1. Spawn a child that installs a no-op `SIGTERM` handler and never exits (e.g. `process.on('SIGTERM', () => {}); setInterval(() => {}, 1000)`). 2. Drive it through the direct-child fallback, i.e. `signalRunningProcess({ child, processGroupId: null }, …)` (the path used on win32 / when group signaling is unavailable). 3. Send SIGTERM (the child swallows it; `child.killed` becomes `true`), then send SIGKILL. 4. On the pre-fix `!child.killed` guard the SIGKILL call is a no-op and the PID survives past its deadline. Covered by the new regression test in this PR. ### Paperclip version or commit Reproduces on `master` (the `signalRunningProcess` fallback). Also present in published `@paperclipai/adapter-utils` (e.g. `2026.325.0`), where the same `!child.killed` guard sits on the single direct-child escalation path. _Searched the open PR list for duplicates/related work on `runChildProcess` / `signalRunningProcess` / SIGKILL escalation; found none._ ## What Changed - `packages/adapter-utils/src/server-utils.ts`: in `signalRunningProcess`, replace the direct-child fallback guard `!running.child.killed` with `running.child.exitCode === null && running.child.signalCode === null` (real liveness). The process-group path is unchanged. - `packages/adapter-utils/src/server-utils.ts`: `export` `signalRunningProcess` so the fallback branch can be unit-tested directly. - `packages/adapter-utils/src/server-utils.test.ts`: add a companion regression test (POSIX-only, like the sibling timeout tests) that forces the fallback (`processGroupId: null`) — sends SIGTERM (child swallows it, `child.killed` becomes `true`), asserts the child is still alive, then sends SIGKILL and asserts the PID dies. Also keeps the end-to-end `runChildProcess` SIGTERM-ignoring test. ## Verification ``` npx vitest run packages/adapter-utils/src/server-utils.test.ts # 52 passed npx tsc --noEmit # clean ``` - **Regression proof:** reverting the guard to `!running.child.killed` makes the new fallback test fail (`waitForPidExit` → false; the child survives); the liveness guard makes it pass. This addresses the prior review note that the existing test only exercised the process-group path (which already escalated correctly on POSIX) and never reached the changed branch. ## Risks Low. A one-line guard change scoped to the direct-child fallback; the process-group path is untouched. SIGKILL is only sent when `exitCode`/`signalCode` are both still `null`, i.e. the process is provably alive, so the change cannot signal an already-reaped/recycled PID. New tests are POSIX-only and `skipIf(win32)`, consistent with the sibling timeout tests in this file. ## Model Used Anthropic **Claude Opus 4.8**, driven via the Cursor agent (extended reasoning + tool use, large context). Diff, tests, and the regression proof above were produced and run by the agent; reviewed by a human before pushing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com>	2026-07-10 14:03:22 -05:00
3 people	1fe89eb8f8	Enforce durable external-wait liveness (#9373) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The heartbeat/recovery subsystem decides whether an agent run has a durable continuation path after the process stops. > - External waits need stricter semantics than local background watchers: a killed local process is not durable, while a first-class blocker/monitor/scheduled wake is. > - Without that distinction, recovery can repeatedly treat adapter-failed continuations as live work and obscure the real reason a task stopped. > - This pull request adds explicit durable external-wait liveness handling and documents the expected execution semantics. > - It also improves operator-visible recovery evidence so invalid external-wait paths explain why they were rejected. > - The benefit is clearer recovery behavior, fewer duplicate continuation recoveries, and a safer contract for monitor-backed external waits. ## Linked Issues or Issue Description - Refs #5978 - Related PRs: #4988, #7495, #8502 ## What Changed - Added durable external-wait liveness classification so local/background watchers are not accepted as durable live paths after the owning process exits. - Preserved first-class blocker/monitor/scheduled wake paths as valid external-wait continuations. - Added backend regression coverage for killed watcher failure, monitor-backed durable wait resumption, normal completion, blocker behavior, and no duplicate recovery. - Added adapter utility coverage for terminal cleanup behavior used by local process adapters. - Surfaced invalid external-wait recovery evidence in the recovery action card and run ledger. - Updated execution semantics documentation and the V1 implementation contract. ## Verification - `pnpm check:token-gates` passed. - `pnpm -r typecheck` passed. - `node scripts/run-vitest-stable.mjs --mode general --group general-server` equivalent lane passed in CI-clean env: 238 files, 2164 tests passed, 1 skipped. - `node scripts/run-vitest-stable.mjs --mode general --group general-workspaces-a` passed in fully Paperclip-env-clean env: UI 305 files / 2430 tests; CLI 43 files / 230 tests. - `node scripts/run-vitest-stable.mjs --mode general --group general-workspaces-b` passed in fully Paperclip-env-clean env: shared/db/adapters/plugin packages all green. - `node scripts/run-vitest-stable.mjs --mode serialized` passed in fully Paperclip-env-clean env: 107 serialized server suites green, including 84/84 heartbeat-process-recovery tests. - `pnpm build` passed in fully Paperclip-env-clean env. Notes: running `pnpm test:run` directly inside the Paperclip heartbeat environment exposed local harness env contamination in existing tests (`PAPERCLIP_CONFIG`, `PAPERCLIP_DB_BACKUP_DIR`, and `PAPERCLIP_WORKTREE_START_POINT`). Re-running the same lanes with inherited `PAPERCLIP_*` and port env removed produced the CI-equivalent green results above. ## Risks - Medium behavioral risk: this changes recovery classification for stopped local external-wait processes, so adapters relying on unmanaged background watchers must use blockers, monitors, scheduled wakes, or explicit durable handoff instead. - Low UI risk: recovery-card copy changes are covered by component tests and Storybook screenshot QA. - No database migration is included. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5-based coding agent, tool-enabled terminal/code execution. Exact context-window metadata was not exposed in the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-10 13:21:12 -05:00
DottaandPaperclip	3369c0dab7	fix(prompt): render exact branch name with backtick-safe fence in wake branch guard (#9326) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents run heartbeats inside execution workspaces pinned to a specific git branch; the wake prompt now carries a one-time "stay on this branch" guard (#9319) > - Greptile's final review round on #9319 landed after the PR merged: the guard sanitized the branch name by stripping backticks, which mutates the ref, so the prompt could tell the agent to stay on a branch name that does not exist > - A guard that names the wrong branch defeats its purpose and still leaves the workspace contract breakable > - This pull request keeps the pinned ref name exact and instead escapes it at render time with a backtick fence longer than any backtick run inside the name (standard Markdown inline-code escaping) > - The benefit is the guard always names the real branch while a hostile ref name still cannot close the code span or inject prompt text ## Linked Issues or Issue Description Refs #9319 — follow-up addressing the final Greptile review round that arrived after that PR merged. ## What Changed - `normalizePaperclipWakeExecutionWorkspace` no longer strips backticks from the branch name; it removes only control characters (illegal in git ref names, and the newline route into the prompt), trims, and caps length. - Added a `markdownInlineCode` helper that wraps a value in an inline-code span whose backtick fence is one longer than the longest backtick run in the value, and used it when rendering the branch guard line. - Updated the hostile-branch-name test to assert the exact ref is preserved and fenced, and control characters are removed. ## Verification - `pnpm vitest run packages/adapter-utils/src/server-utils.test.ts` — 58 tests pass, including the updated hostile-branch-name case. - `npx tsc --noEmit` in `packages/adapter-utils` — clean. - Manual: render a wake payload with `branchName: "evil` + backtick + `name"` and confirm the guard line reads ``` `` evil`name `` ``` and the span does not break. ## Risks - Low risk: prompt-rendering-only change; the normalized payload shape is unchanged. Branch names containing backticks (extremely rare) now render exactly instead of mutated. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic) with extended thinking and tool use, via Claude Code / Paperclip agent harness. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-09 18:23:54 -05:00
3 people	4d898aa7af	feat(prompt): one-time execution-workspace branch guard in wake prompt (PAP-13326) (#9319) ... ## Summary When a task runs in a branch-pinned execution workspace, agents sometimes switch or rename the workspace branch, which breaks the worktree contract. This adds a short, one-time prompt hint telling the agent to stay on the pinned branch. - **heartbeat.ts**: after the execution workspace is resolved, attach `executionWorkspace: { branchName }` to the wake payload (only when a branch pin exists — agent-home runs without a branch are untouched). - **server-utils.ts (adapter-utils)**: normalize the new payload field and render one bullet in `renderPaperclipWakePrompt`: > `- execution workspace branch: you are running in an execution workspace on branch \`<name>\`. Do not switch, rename, or re-point this branch; keep all commits on it.` - The hint renders **only on non-resumed sessions** — resume-delta prompts skip it, so it appears the first time an issue's session starts, not on every turn, and it never pollutes the issue thread. One renderer change covers every adapter (claude, codex, cursor, gemini, grok, opencode, pi, hermes, acpx engine) with zero per-adapter edits. ## Tests - `server-utils.test.ts`: branch guard renders on first prompt, absent on resumed-session prompts, absent when no branch is pinned; payload round-trips through `stringifyPaperclipWakePayload`. - `heartbeat-workspace-branch-containment.test.ts`: the finalize-path adapter mock now asserts the wake payload the adapter receives carries the branch pin matching `context.paperclipWorkspace.branchName` (end-to-end heartbeat wiring, embedded postgres). All 6 pass. - Full `server-utils` (56) and acpx-engine execute (34) suites green; adapter-utils typechecks clean; no new server tsc errors. PAP-13326 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-09 17:53:00 -05:00
Devin FoleyandPaperclip	eedc7ddef2	Make ACP the default engine for local adapters (#9238) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Adapter packages are the bridge between the control plane and local agent harnesses such as Claude Code, Codex, and Gemini CLI. > - ACP support was concentrated in a separate `acpx_local` adapter, which made ACP feel like a separate agent choice instead of an execution capability of the harness adapters. > - Claude, Codex, and Gemini now have ACP-capable harnesses, so the native adapter should own ACP selection, fallback, config, transcript parsing, and environment diagnostics. > - The standalone ACPX adapter still needs a compatibility path for existing rows, but it should not be offered as an active adapter for new agents. > - This pull request moves the shared ACP runtime into `@paperclipai/acpx-engine`, wires Claude/Codex/Gemini local adapters to prefer ACP when prerequisites are available, and retires `acpx_local` to a tombstone. > - The benefit is one adapter per harness, richer ACP transcripts by default where possible, and a migration path for existing Claude/Codex ACPX agents. ## Linked Issues or Issue Description Closes #5932 — the broken default `acpx_local` Claude path is replaced by native `claude_local` ACP support, existing Claude/Codex ACPX rows migrate to native adapters, and new agents no longer choose the standalone ACPX adapter. Refs #4893 — original merged ACPX local adapter runtime that this PR replaces with native per-harness ACP engines. Refs #6590 — prior ACPX-Claude seamlessness work folded into the new native Claude ACP path. Refs #197 — related open generic ACP/Kiro adapter work; this PR does not close it because Kiro/custom generic ACP remains a separate adapter decision. Refs #7018 — related Kimi-specific `acpx_local` shell failure; this PR retires the built-in standalone adapter but does not add a native Kimi adapter. Refs #8864 — related ACPX prompt/API guidance PR; this PR moves runtime guidance into the shared/native ACP engine path instead of the old standalone adapter. Refs #8881 — related `acpx_local` POSIX shell failure from the old `acpx` pin; this PR updates ACP dependencies but does not claim custom/OMP ACP support as a first-class native adapter. Refs #8964 — related open `acpx_local` stderr cleanup PR; this PR makes the old runtime path obsolete for new agents but keeps it as a non-closing reference. Problem description: - The standalone `acpx_local` adapter duplicates Claude/Codex agent choices that already have first-class local adapters. - ACP should be an execution engine capability of each harness adapter when the underlying harness supports ACP. - Existing `acpx_local` agents should either migrate to native harness adapters or fail with an explicit retirement message instead of silently falling back to the process adapter. ## What Changed - Added `@paperclipai/acpx-engine` as the shared ACP execution, session-codec, CLI formatter, and UI parser package. - Wired `claude_local`, `codex_local`, and `gemini_local` to auto-select ACP by default when prerequisites pass, with `engine=cli` opt-out and `engine=acp` strict mode. - Added ACP config schema/UI fields, environment checks, session-codec preservation, transcript parsing, and adapter capability metadata for the native adapters. - Retired `acpx_local` to a server tombstone, removed its UI/package/runtime image surface, and added a migration for existing Claude/Codex ACPX agents. - Updated package manifests, lockfile, release tooling, docs, Kubernetes sandbox defaults, and tests. ## Verification - `corepack pnpm --filter @paperclipai/acpx-engine typecheck` - `corepack pnpm --filter @paperclipai/adapter-claude-local typecheck` - `corepack pnpm --filter @paperclipai/adapter-codex-local typecheck` - `corepack pnpm --filter @paperclipai/adapter-gemini-local typecheck` - `corepack pnpm --filter @paperclipai/acpx-engine exec vitest run` - `corepack pnpm --filter @paperclipai/adapter-claude-local exec vitest run src/server/acp.test.ts src/server/execute.acp-fallback.test.ts src/ui/build-config.test.ts` - `corepack pnpm --filter @paperclipai/adapter-codex-local exec vitest run src/server/acp.test.ts src/ui/build-config.test.ts` - `corepack pnpm --filter @paperclipai/adapter-gemini-local exec vitest run src/server/acp.test.ts src/ui/build-config.test.ts src/ui/parse-stdout.test.ts` - `corepack pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && corepack pnpm --filter @paperclipai/server exec tsc --noEmit` - `corepack pnpm --filter @paperclipai/server exec vitest run src/__tests__/adapter-routes.test.ts src/__tests__/adapter-session-codecs.test.ts src/__tests__/adapter-models.test.ts` - `corepack pnpm --filter @paperclipai/ui typecheck` - `corepack pnpm --filter @paperclipai/ui exec vitest run src/adapters/metadata.test.ts src/adapters/adapter-display-registry.test.ts src/components/AgentConfigForm.test.ts src/components/AgentConfigForm.render.test.tsx src/components/transcript/RunTranscriptView.test.tsx` - `node --test scripts/bootstrap-npm-package.test.mjs scripts/release-package-map.test.mjs scripts/verify-release-registry-state.test.mjs` Note: the server typecheck script calls `pnpm` internally; this dev shell exposes pnpm through Corepack only, so I ran the two script steps manually with `corepack pnpm`. ## Risks - Migration changes existing `acpx_local` Claude/Codex agents to native adapter types and clears old ACPX task sessions/runtime state. - Custom ACP commands remain on the retired tombstone and will need a separate future adapter/plugin path. - ACP auto-selection depends on local Node and ACP server command prerequisites; remote and unsupported environments fall back to CLI unless `engine=acp` is explicit. - `@paperclipai/acpx-engine` is a new public package and needs npm trusted-publishing bootstrap before release automation can publish it. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex coding agent. Exact hosted model build and context-window size are not exposed in this runtime. Tool use included shell execution, repository editing, GitHub CLI operations, and local test/typecheck execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-08 19:05:03 -07:00
Devin FoleyandPaperclip	cc17f29e7e	fix(timeouts): raise sandbox wall-clock backstop to 4h and make acpx_local timeouts self-describing (#9232) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent runs execute through adapters (e.g. `acpx_local`), which can run locally, over SSH, or inside sandbox execution targets, each with a wall-clock execution timeout > - Sandbox-backed runs defaulted to a 30-minute wall-clock backstop (`DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC = 1800`), which kills healthy long agent runs that are still making progress — long before the recovery watchdog's 4h critical threshold would even consider them stuck > - On top of that, `acpx_local` resolved its timeout directly from `adapterConfig.timeoutSec` instead of the shared execution-target resolver, and its timeout failures surfaced as a bare `Timed out after Ns` — giving operators no clue which timer fired or which knob raises it > - This pull request raises the sandbox backstop to 4h (aligned with the recovery watchdog), routes `acpx_local` through the shared timeout resolver, logs the effective timeout and its source at run start, and makes every timeout error message self-describing > - The benefit is that long-running sandbox agent runs no longer die at 30 minutes, and when a wall-clock timeout does fire, the run log states exactly which timer fired and how to configure it ## Linked Issues or Issue Description Refs #4535 (related: wall-clock execution timeouts killing agent runs that are still making progress — that issue covers a different hardcoded 600s timer, but the operator pain is the same). No exact public issue exists for this one, so describing it in-PR: **Bug:** A long sandbox-backed `acpx_local` agent run was killed with a bare `Timed out after 1800s` even though the agent was actively working. - **What happened:** The run hit the 30-minute `DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC` backstop. `acpx_local` never consulted the shared execution-target timeout resolution (it read `adapterConfig.timeoutSec` directly, default 0), so on sandbox targets the sandbox-provider default applied with no adapter-level say. The resulting error named neither the timer that fired nor the knob that controls it. - **Expected:** Healthy long runs should not be killed by a 30-minute wall-clock backstop when the recovery watchdog only treats runs as critically stuck after 4h of output silence; and any timeout error should say which timeout fired and how to raise it. - **Impact:** Long, legitimate agent runs in sandboxes fail mid-work; operators waste time reverse-engineering which of several timers produced "Timed out after Ns". ## What Changed - `packages/adapter-utils/src/execution-target.ts` - `DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC` raised from `1_800` to `14_400` (4h), with a comment explaining it intentionally matches the recovery watchdog's `ACTIVE_RUN_OUTPUT_CRITICAL_THRESHOLD_MS` (4h) so the adapter backstop never fires before the watchdog path. Output-inactivity monitors remain the primary hang detectors. - New `resolveAdapterExecutionTargetTimeout(target, configuredTimeoutSec)` returns `{ timeoutSec, source }` where `source` is `configured` / `sandbox_default` / `unlimited`. The existing `resolveAdapterExecutionTargetTimeoutSec` is preserved as a thin wrapper, so current callers are unaffected. - New `formatAdapterExecutionTimeoutErrorMessage(resolution)` and `formatAdapterExecutionTimeoutStartLogLine(resolution)` produce self-describing messages that name the timer that fired and the `adapterConfig.timeoutSec` knob that controls it. - `packages/adapters/acpx-local/src/server/execute.ts` - `buildRuntime` now resolves the wall-clock timeout through the shared resolver: sandbox targets default to the 4h backstop, local/SSH keep the historical "0 = no adapter timeout", and a configured `adapterConfig.timeoutSec` always wins. - The executor logs the effective timeout and its source at run start (`[paperclip] Adapter execution timeout: …`), so a later timeout is diagnosable from the run log alone. - All three bare timeout messages (timer cancel reason, turn result `errorMessage`, catch-path `messageOverride`) now use the self-describing format. - `packages/adapters/acpx-local/src/index.ts` — the adapter configuration doc for `timeoutSec` states the sandbox default and that the output-inactivity monitor remains the primary hang detector. - Tests: `packages/adapter-utils/src/execution-target-sandbox.test.ts` and `packages/adapters/acpx-local/src/server/execute.test.ts` (see Verification). ## Verification - `pnpm --filter @paperclipai/adapter-utils typecheck` — passes - `pnpm --filter @paperclipai/adapter-acpx-local typecheck` — passes - `npx vitest run packages/adapter-utils/src/execution-target-sandbox.test.ts packages/adapters/acpx-local/src/server/execute.test.ts` — 2 files, 40 tests, all pass - New/updated test coverage: - sandbox default resolves to 4h (and the constant is asserted to be `4 * 60 * 60`) - `resolveAdapterExecutionTargetTimeout` reports `configured` / `sandbox_default` / `unlimited` sources with the correct precedence (configured > sandbox default; local/SSH stay unlimited) - exact wording of the self-describing error message and the start-of-run log line - `acpx_local` runtime picks up the sandbox default into `timeoutMs`, keeps the unlimited local default, honors configured-over-default precedence, emits the start-of-run log line, and surfaces the self-describing `errorMessage`/cancel reason when the wall-clock timer kills a turn ## Risks - **Behavioral shift:** sandbox-backed adapter runs that previously hit the 30-minute backstop now run up to 4h before the adapter kills them. Genuinely hung runs are still caught much earlier by the adapters' output-inactivity monitors and by the recovery watchdog; the wall-clock timer is a last-resort kill switch. Operators who relied on the 30-minute default can restore it explicitly via `adapterConfig.timeoutSec`. - **Error-message consumers:** any tooling that pattern-matched the exact `Timed out after Ns` string from `acpx_local` will see the new self-describing message instead. - No API or schema changes; `resolveAdapterExecutionTargetTimeoutSec` keeps its exact signature and behavior (modulo the raised sandbox default). > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude (Anthropic) via Claude Code CLI — model ID `claude-fable-5`, extended thinking enabled, agentic tool use (file edits, shell, test execution) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-08 13:50:00 -07:00
DottaandPaperclip	f616b6746c	fix(server): clean heartbeat run scratch directories (#9234) ... Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-07-08 13:38:36 -05:00
DottaandPaperclip	555391fed7	fix: run restart recovery, workspace self-heal, quota-aware retries, failed-run metrics (#9183) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents run in heartbeat runs orchestrated by the server; run lifecycle, retry scheduling, and the dashboard's run-activity metrics are the subsystems involved > - A spike in "failed" tasks traced to three causes: server restarts killing in-flight runs and mislabeling them as failures, deterministic workspace-validation loops when a worktree's branch diverged, and provider quota/usage-limit errors being classified as generic transient failures (putting agents into error state and polluting metrics) > - Killed-then-recovered runs and quota waits are not product failures, so both the runtime behavior and the reporting needed to distinguish them > - This pull request drains runs gracefully on shutdown with idempotent restart retries, self-heals workspace branch mismatches, adds a quota-aware failure class with reset-time retry, separates recovered restart kills from true failures on the dashboard, and documents restart hygiene for operators > - The benefit is fewer spurious failures, automatic recovery instead of manual repair, and dashboard metrics that reflect real failure rates ## Linked Issues or Issue Description No public GitHub issue exists; describing the bug inline per the bug-report template: **What happened?** In-flight heartbeat runs are marked `failed` when the server restarts, even though a retry later succeeds. Worktrees whose checked-out branch diverges from the issue branch fail workspace validation on every subsequent run with no recovery path. Provider quota/usage-limit responses are treated as generic transient upstream errors, putting agents into an error state and retrying before the quota window resets. The dashboard counts all of these as true failures, inflating failure metrics. **Expected behavior** Graceful shutdown should interrupt (not fail) running runs and chain exactly one recovery retry. Workspace validation should repair recoverable branch mismatches automatically. Quota errors should get their own error class with the retry scheduled at the provider reset time and the agent left idle. The dashboard should report recovered restart kills separately from true failures. **Steps to reproduce** 1. Start a heartbeat run, then restart the server (SIGTERM) while it is in flight — the run lands as `failed` with a process-loss error code even when its retry succeeds 2. Check out an issue whose worktree branch has diverged (e.g. after a force-moved branch) — every subsequent run fails `workspace_validation_failed` deterministically 3. Drive an agent into a provider usage-limit window — the run fails as a generic transient upstream error and the agent enters an error state instead of idling until the reset time **Paperclip version or commit** master (base c07e650cd) **Deployment mode** Self-hosted dev plane (Linux, node server + embedded Postgres) ## What Changed - Graceful shutdown (SIGTERM hook) now marks in-flight runs `interrupted` instead of `failed` and enqueues an idempotent process-loss retry (pre-insert existence check on `retryOfRunId` prevents duplicates; bursts chain exactly one retry per interrupted run) - Run-liveness classification routes `interrupted` to `needs_followup` rather than `failed` - Workspace validation self-heals branch mismatch / missing-branch states instead of failing deterministically on every run - New `provider_quota` error class: session/usage-limit responses schedule the retry at the provider reset time and leave the agent idle (not errored); fixes a case where a quota-terminated run with subtype `success` was misclassified as failed; HTTP 529 remains transient - Dashboard run-activity query separates recovered restart kills from true failures via a recursive CTE over `retry_of_run_id` (ancestors of a succeeded retry count as recovered), adds a per-day failed-by-error-code breakdown, and binds the window start as a timestamptz string - Activity charts UI: amber "Recovered" segment with legend and per-day error-code tooltip; success-rate chart counts recovered runs as successes - New ops runbook: `docs/deploy/dev-plane-restart-hygiene.md` ## Verification - Greptile follow-up fixes on `992705edd`: `pnpm exec vitest run server/src/__tests__/server-startup-feedback-export.test.ts server/src/__tests__/heartbeat-workspace-branch-containment.test.ts server/src/__tests__/heartbeat-workspace-session.test.ts server/src/__tests__/heartbeat-process-recovery.test.ts server/src/__tests__/heartbeat-workspace-finalize-branch.test.ts` (209 tests); `pnpm --filter @paperclipai/server typecheck`; `git diff --check` - Post-rebase CI fixes: `pnpm exec vitest run server/src/__tests__/heartbeat-retry-scheduling.test.ts`; `pnpm exec vitest run server/src/__tests__/heartbeat-retry-scheduling.test.ts server/src/__tests__/workspace-runtime.test.ts server/src/__tests__/heartbeat-workspace-branch-containment.test.ts server/src/__tests__/heartbeat-workspace-finalize-branch.test.ts server/src/__tests__/heartbeat-workspace-session.test.ts` (227 tests); `pnpm --filter @paperclipai/server typecheck` - `npm test` server suites covering the changes: `heartbeat-process-recovery`, `heartbeat-stop-metadata`, `heartbeat-retry-scheduling` (95 tests), quota parse + execute/retry-scheduling suites (100 tests), workspace self-heal suites (200 tests), dashboard run-activity tests (3 tests) — all green, typecheck exit 0 - Dashboard CTE cross-checked against a real development database: two restart-burst days moved from 17 to 8 and 19 to 6 true failures once recovered kills were separated, matching manual retry-chain inspection - Screenshot verification of the real ActivityCharts component (recovered segment + tooltip) during QA ## Risks - Behavioral shift: runs killed by a restart no longer surface as `failed`; anyone consuming raw run statuses will see `interrupted` (new status value) — dashboards/queries in this repo were updated accordingly - Retry chaining on repeated restarts is bounded (one chained retry per interruption) but a pathological restart loop still delays work rather than failing it; the runbook covers operator hygiene for that case - Dashboard query adds a recursive CTE; cost is bounded by the day-window row count and was verified against production-sized data - No schema migrations; low migration risk ## Model Used - Claude (Anthropic) — claude-fable-5 via Claude Code / Paperclip agent harness, extended thinking with tool use; implementation commits also produced with Codex CLI (GPT-5 class) agents under the same harness ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-07 20:30:09 -05:00
Devin FoleyandPaperclip	c48feee190	Improve live agent feedback during sandboxed runs (#8915) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A core part of that experience is watching active agent runs without dropping into raw logs first > - Local and sandbox-backed adapters already record useful run output, progress, and tool activity > - But active issue threads could sit visually stale while the agent was syncing workspaces, tailing sandbox output, or emitting incremental tool-call updates > - Operators need timely, human-readable progress while preserving the raw transcript underneath > - This pull request streams sandbox run-log progress into runtime status, keeps visible issue threads refreshed, and folds repeated ACPX tool updates into stable transcript cards > - The benefit is that long-running agent work becomes easier to supervise without changing the task/comment control-plane model ## Linked Issues or Issue Description No public GitHub issue exists for this exact change. Problem/motivation: - During long-running sandboxed agent work, the issue UI can appear idle even though the agent is actively syncing, running tools, or producing incremental output. - Operators need realtime feedback at the issue-thread layer, not only after opening raw logs or waiting for the final heartbeat result. - Related public context: #1808 previously added live-run status dots to Projects; #4362 touches heartbeat wakeup behavior but is not a duplicate of this runtime/UI feedback change. ## What Changed - Added sandbox run-log streaming support and defaulted sandbox-capable local adapters into the richer live-feedback path. - Surfaced environment/sandbox sync progress through heartbeat runtime status with bounded, redacted snippets. - Added live issue-thread cache patching so visible active runs update as progress events arrive. - Folded repeated ACPX `tool_call` updates into one transcript card instead of stacking duplicate cards. - Updated adapter docs and added focused regression coverage for sandbox log streaming, runtime status, ACPX parsing, live updates, transcript rendering, and issue chat messages. ## Verification - `pnpm install --frozen-lockfile` - `pnpm exec vitest run ui/src/context/LiveUpdatesProvider.test.ts` - `pnpm exec vitest run server/src/services/heartbeat-run-runtime-status.test.ts server/src/__tests__/heartbeat-runtime-state.test.ts ui/src/context/LiveUpdatesProvider.test.ts` - `pnpm exec vitest run packages/adapter-utils/src/execution-target-sandbox.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts server/src/services/heartbeat-run-runtime-status.test.ts server/src/__tests__/agent-live-run-routes.test.ts server/src/__tests__/heartbeat-runtime-state.test.ts packages/adapters/acpx-local/src/ui/parse-stdout.test.ts ui/src/context/LiveUpdatesProvider.test.ts ui/src/components/transcript/RunTranscriptView.test.tsx ui/src/lib/issue-chat-messages.test.ts ui/src/components/IssueChatThread.test.tsx` - GitHub PR workflow on head `8397953e7b41ccd42e5d9457ee7e4dfb996e4ec5`: `verify`, build, typecheck/release-registry, e2e, general shards, serialized server shards, and canary dry run passed. - Greptile Review on head `8397953e7b41ccd42e5d9457ee7e4dfb996e4ec5`: Confidence Score 5/5, no unresolved review threads. ## Risks - Live issue-thread cache patching could miss an edge case for a route shape not covered by tests. - Surfacing active-run snippets needs continued care around redaction; this PR keeps snippets bounded and adds redaction-focused coverage. - More frequent active-run UI refreshes could expose performance issues on very large issue threads, though updates are scoped to visible run/query caches. ## Model Used OpenAI GPT-5 via Codex, operating as a tool-enabled coding agent with shell, git, and repository-editing capabilities. Context window size is not exposed in this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-02 22:21:56 -07:00
Devin FoleyandPaperclip	bac7307ec4	fix(adapter-utils): improve sandbox restore failure diagnostics (#8903) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent adapters share managed-runtime helpers from `@paperclipai/adapter-utils` so local and sandboxed runs can prepare, execute, and restore workspaces consistently. > - The sandbox managed runtime syncs workspaces in both directions by creating tar archives on the local host and inside the remote sandbox. > - A prior fix avoided archiving `.` during upload because tar self-entries can force chmod/utime on a directory the command user does not own. > - The restore/download path still archived the remote workspace as `.`, and failed managed commands only surfaced stderr. > - This pull request applies entry-based tar creation to the sandbox restore path and includes stdout in managed command failure diagnostics. > - The benefit is that restore failures become visible to operators and sandbox workspace restore avoids the same directory-metadata failure class already fixed for upload. ## Linked Issues or Issue Description No existing public issue found; describing in-PR (bug). - **What happens:** sandbox workspace restore can fail while creating `workspace-download.tar` from the remote workspace when the tar command includes a `.` self-entry and the command user cannot update metadata on the workspace directory. If the failing command writes its diagnostic to stdout, the managed runtime error can collapse to a generic failed shell command without the useful tar message. - **Expected behavior:** restore should archive the workspace entries without a `.` self-entry, and failed managed runtime commands should include useful stdout/stderr diagnostics. - **Where:** `packages/adapter-utils/src/sandbox-managed-runtime.ts` restore/download path and `packages/adapter-utils/src/command-managed-runtime.ts` command error formatting. - **Related public context:** #7836 fixed the upload side of the same tar self-entry failure class. ## What Changed - Added stdout-aware failed-command formatting in the command managed runtime, keeping diagnostics bounded to the tail of stdout/stderr. - Added remote workspace tarball creation that names top-level entries explicitly instead of archiving `.` during sandbox restore. - Preserved empty-workspace restore support by creating a valid empty tarball when the remote workspace has no entries. - Added regression coverage for stdout diagnostics, restore tar members, and empty workspace restore tarballs. ## Verification - `pnpm vitest run packages/adapter-utils/src/command-managed-runtime.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — 2 files, 17 tests passed. - `pnpm --filter @paperclipai/adapter-utils typecheck` — passed. - `git diff --check origin/master..HEAD` — passed. - Local public-hygiene/PII scan of the committed diff checked for internal ticket refs, local/private URLs, common token/key patterns, private-key blocks, and email-like values — passed. - GitHub duplicate/related search found no public issue or PR for `workspace-download.tar Permission denied` or `sandbox restore tar permission denied`; #7836 is linked as related prior work. - PR CI on commit `3ad4c8d` — all GitHub Actions lanes, security scans, and aggregate `verify` passed. - Greptile Review on commit `3ad4c8d` — Confidence Score 5/5; the prior P2 thread is resolved with no open P2s, recommendations, or follow-ups. ## Risks Low. This is limited to shared adapter runtime error formatting and sandbox restore archive construction. Archive contents should remain equivalent apart from the removed `.` self-entry, and the new diagnostics are bounded to avoid dumping unbounded command output. ## Model Used OpenAI Codex, GPT-5, tool-enabled coding agent with shell and GitHub CLI access. Context window size was not reported by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (n/a; internal adapter-runtime behavior only) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-02 17:06:34 -07:00
DottaandPaperclip	246e1b38bf	[codex] Include checkbox selections in continuation wakes (#8893) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Issue-thread interactions are the subsystem that lets board users answer structured prompts and resume agent work > - Checkbox confirmations capture a selected subset of known options, then wake the assignee through continuation context > - The wake context previously carried generic interaction metadata, but not the accepted checkbox option ids or option labels > - That meant the resumed agent could be woken after a checkbox confirmation without seeing the board's selected options in the turn context > - This pull request carries accepted checkbox selections through the interaction continuation wake snapshot and renders them into the adapter wake prompt > - The benefit is that agents can act on checkbox-confirmation selections without refetching or guessing the user's choices ## Linked Issues or Issue Description No public GitHub issue exists for this bug. Searched public issues and PRs for checkbox confirmation / continuation selection duplicates and found no matching issue or PR. ### Pre-submission checklist - [x] I have searched existing open and closed issues and this is not a duplicate. - [x] I am on the latest released version of Paperclip or can reproduce on `master`. - [x] I have confirmed the error originates in Paperclip itself, not in my agent adapter, API provider, or local configuration. ### What happened? When a board user accepted a `request_checkbox_confirmation` interaction, the assignee continuation wake included generic interaction metadata but did not include the accepted checkbox selections. The resumed agent turn therefore had no in-prompt access to the selected option ids or option labels/descriptions. ### Expected behavior When a `request_checkbox_confirmation` interaction is accepted, the resumed agent wake should include the checkbox prompt, selected option ids, and selected option labels/descriptions so the agent can act on the selected subset directly. ### Steps to reproduce 1. Create an issue-thread `request_checkbox_confirmation` interaction with multiple options and `continuationPolicy: "wake_assignee"`. 2. Accept the interaction with one or more selected options. 3. Inspect the continuation wake payload/prompt received by the assignee. 4. Observe that the selected checkbox options are missing from the wake context before this fix. ### Paperclip version or commit Reproduced against the pre-fix code path on `master`; this PR head is `9d17e70bce373e4850117f30c015c973c4b61789`. ### Deployment mode Local dev (pnpm dev) / built from source. ### Installation method Built from source (pnpm dev / pnpm build). ### Agent adapter(s) involved Not adapter-specific (core bug). The Codex/local adapter path exposed the missing wake context, but the missing field was in core interaction continuation payload construction. ### Database mode Embedded PGlite or external Postgres; the bug is not database-mode specific. ### Access context Both. Board users resolve the checkbox interaction, and agent bearer-key wakes consume the continuation context. ### Node.js version `v22.22.2` ### Operating system Linux workspace. ### Relevant logs or output No runtime exception is required to reproduce this. The failure mode is missing `checkboxSelection` data in the resolved interaction continuation wake payload. ### Relevant config Not config-related. ### Additional context Root cause: accepted checkbox interaction results were not extracted into the continuation wake context, and adapter wake payload normalization/rendering had no typed `checkboxSelection` field. ### Privacy checklist - [x] I have reviewed all pasted output for PII (usernames, file paths, API keys, tokens, company names) and redacted where necessary. ## What Changed - Added checkbox selection extraction for accepted `request_checkbox_confirmation` interactions and stored it in interaction continuation wake context. - Included checkbox selection context in heartbeat wake payload construction. - Added adapter-utils normalization and wake prompt rendering for checkbox prompt, selected ids, and selected option details. - Added regression coverage for route continuation context, heartbeat payload summaries, and adapter wake prompt rendering. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/server-utils.test.ts server/src/__tests__/heartbeat-context-summary.test.ts server/src/__tests__/issue-thread-interaction-routes.test.ts` - `git diff --check origin/master...HEAD` - `rg -n "checkbox|confirmation|interaction|wake|continuation" ROADMAP.md` - `gh pr list --state all --search "checkbox continuation selection repo:paperclipai/paperclip" --json number,title,state,url,headRefName --limit 20` - `gh issue list --state all --search "checkbox confirmation options repo:paperclipai/paperclip" --json number,title,state,url --limit 20` ## Risks Low risk. The new payload field is additive, only populated for accepted checkbox confirmations, and existing continuation fields are preserved. The main compatibility risk is downstream code assuming an exact wake payload shape; adapter normalization treats the new field as optional. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex coding agent based on GPT-5, with shell/tool execution in this workspace. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-02 18:39:16 -05:00
Devin FoleyandPaperclip	2eba718bef	Fix sandbox bridge credentials and stalled review recovery (#8844) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The local adapter and heartbeat recovery systems decide whether an agent has a real control-plane mutation path. > - Sandboxed local adapters split execution between the trusted host process and the sandbox shell/tool surface. > - A host-side adapter can still reach Paperclip while the sandbox shell surface cannot, which leaves agents thinking no endpoint or credentials are configured even though the host can still post comments. > - Execution-policy review stages can also remain pending after a reviewer run finishes without recording a decision. > - This pull request makes the sandbox bridge available to the actual shell mutation surface and adds bounded recovery for terminal-but-still-pending review participants. > - The benefit is that agents get a real reachable Paperclip API path where they need it, and stalled review stages become visible recovery work instead of silently drifting. ## Linked Issues or Issue Description No exact public GitHub issue matched this combined failure. I searched for exact and related terms including `cannot reach the Paperclip control plane`, `execution_review_participant_recovery`, `sandbox callback bridge`, `review participant in_review`, and `control plane sandbox`. Related public issues: - Refs #8482 for `in_review` liveness invariant recovery. - Refs #863 for prior agent API-key reachability confusion. - Refs #248 for the broader sandboxed agent execution model. Bug summary: - What happened: a sandboxed local-adapter run could have host-side Paperclip access while the sandbox Bash/tool surface lacked a reachable API endpoint or usable run credentials. Separately, a reviewer run could finish while its execution-review stage remained pending, leaving the source issue in `in_review` with no decision and no live participant run. - Expected behavior: the mutation surface that agents actually use should receive a run-scoped Paperclip bridge, and pending review participants should get one bounded normal-model recovery wake before moving to explicit blocked/source-scoped recovery. - Steps to reproduce: run a sandbox-backed local adapter that needs Bash/curl/tooling to call Paperclip from inside the sandbox, or finish an execution-policy reviewer run without submitting the pending review decision. - Deployment mode: local/authenticated private development instance with sandbox-backed local adapters. ## What Changed - Changed sandbox callback bridge startup so bridge credentials are passed through the sandbox runner environment instead of embedded in the visible `nohup env ...` command string. - Added adapter-utils coverage proving the sandbox shell can call Paperclip through the bridge, forwards the host run JWT with `X-Paperclip-Run-Id`, and does not leak host or bridge tokens into stdout/stderr, runner command text, or runtime files. - Added one bounded execution-review participant recovery path for terminal reviewer runs whose `executionState` remains pending. - Escalated exhausted or non-invokable review participant recovery to blocked/source-scoped recovery with dedicated evidence, activity, and next-action text. - Documented the mutation-surface reachability contract in `doc/execution-semantics.md` and updated the Paperclip skill authentication guidance for sandbox bridge env vars. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/execution-target-sandbox.test.ts` - `pnpm exec vitest run server/src/__tests__/heartbeat-process-recovery.test.ts --no-file-parallelism --maxWorkers=1` - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/server typecheck` - `git diff --check` - `curl -fsS $PAPERCLIP_API_URL/api/health` returned `status: ok` on the local instance. ## Risks - Medium behavioral risk: more `in_review` issues with terminal-but-pending reviewer runs will now be retried once and then blocked explicitly instead of remaining quiet. - Low sandbox bridge risk: credential delivery moved from command text to the runner environment, which is less leaky but depends on sandbox providers honoring the env payload for startup commands. - No database migration is included. - Full repo build and CI were not run locally before opening the PR; targeted server/adapter tests and typechecks passed. ## Model Used OpenAI GPT-5 via the Codex local agent, with repository tool use and shell-based code execution. The runtime did not expose a precise context-window value to the agent. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-01 21:38:01 -07:00
DottaandPaperclip	5e3d6e3627	[codex] Preserve plan review context in agent wakes (#8649) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Planning work relies on issue documents, request-confirmation interactions, and inline plan annotations > - Agents can be woken after a plan comment, annotation, or confirmation decision > - The wake payload needs enough plan-review context for the agent to act on the specific feedback instead of losing the thread and falling back to broad refetches > - This pull request adds bounded plan-review context to wake payloads and heartbeat context > - It also teaches the adapter wake prompt renderer to surface those open plan annotations and interaction results directly > - The benefit is that agents can continue plan review and plan acceptance flows with the relevant comments in hand while keeping wake payloads bounded and company-scoped ## Linked Issues or Issue Description No matching public GitHub issue was found. ### Subsystem affected Cross-cutting: `server/`, `packages/shared`, and `packages/adapter-utils`. ### Problem or motivation Plan-review continuations can wake an agent after a plan comment, inline annotation, or request-confirmation decision without enough inline context about the open plan annotations or accepted/rejected confirmation target. That makes scoped wakes less reliable because the agent may need to refetch broad issue history before it can tell what feedback should be incorporated. ### Proposed solution Include bounded, company-scoped plan review context in wake payloads and heartbeat context. The context includes open `plan` annotation threads, recent annotation comments, truncation metadata, and plan-confirmation interaction target/result details. Render that information in the adapter wake prompt so agents see the relevant plan-review feedback immediately. ### Alternatives considered Relying on agents to fetch the full issue thread after every plan-review wake was rejected because it is slower, harder to audit, and easier to mishandle when the wake is meant to be scoped to a specific comment, annotation, or interaction result. ### Roadmap alignment This supports the roadmap areas for Agent Reviews and Approvals, Deep Planning, and Enforced Outcomes by making plan approval continuations explicit and actionable. ### Additional context The implementation keeps payload size bounded with per-thread, per-comment, and total-body limits. Resolved annotation threads are intentionally omitted so the wake focuses on feedback still needing action. ## What Changed - Added shared `PlanReviewContext` types for plan annotation threads, comments, interaction targets, and continuation results. - Added server-side plan review context assembly for open `plan` annotation threads with bounded thread/comment/body limits. - Included plan review context in heartbeat context and scoped wake payloads for planning, annotation, comment, and plan-confirmation interaction wakes. - Rendered plan annotation deltas, open plan comments, interaction results, and accepted target revisions in adapter wake prompts. - Added focused regression coverage for scoped plan review context, wake prompt rendering, annotation filtering, and safe standard-mode annotation wakes. - Addressed Greptile feedback by bounding the plan-comment DB fetch and removing unused plan review context input fields. ## Verification - `pnpm run preflight:workspace-links` - `pnpm exec vitest run --project @paperclipai/adapter-utils packages/adapter-utils/src/server-utils.test.ts` - `pnpm exec vitest run --project @paperclipai/server --no-file-parallelism --maxWorkers=1 server/src/__tests__/document-annotations-service.test.ts server/src/__tests__/issue-thread-interaction-routes.test.ts server/src/__tests__/issues-goal-context-routes.test.ts` - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/server typecheck` - `git diff --check public-gh/master...HEAD` - GitHub PR checks are green on `36d0ac6a5dce27b9d62e201bf6d9829170c5974e`n- Rebased onto current `paperclipai/paperclip:master` and confirmed GitHub reports the PR as mergeable - Greptile Review completed successfully after 2 comments were addressed and resolved; 0 unresolved review threads remain ## Risks - Medium: wake payloads now include additional plan-review data, so limits and truncation behavior need to stay conservative as annotation volume grows. - Low migration risk: no database schema or migration changes. - Low repository hygiene risk: this PR does not touch `pnpm-lock.yaml`, `.github/workflows`, or media assets. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex using `gpt-5` as a coding agent with shell/tool execution. Reasoning mode and exact context window were not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-06-29 13:26:26 -05:00
Devin FoleyandPaperclip	721541c41d	Fix sandbox restore index drift (#8595) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandboxed agents can do useful work in a remote workspace, then export that result back to a host review checkout. > - A restore can leave the host checkout's index stale even when the sandbox content and committed state are correct. > - That drift makes reviewers see misleading missing-file or dirty-state results after sandbox execution. > - This pull request repairs the host index refresh path and surfaces a clean/dirty restore signal during finalization. > - The benefit is more reliable review checkouts and clearer sandbox finalization status. ## Linked Issues or Issue Description Refs #248 No exact public GitHub issue was found for this restore/index consistency fix. The underlying bug is that a sandbox restore can update host working-tree content without leaving the host git index aligned, so local review tooling may report stale deletions or misleading dirtiness. This PR keeps the restore/export path consistent and reports the resulting clean/dirty state. GitHub search performed for related or duplicate work: `sandbox runtime status`, `sandbox restore index`, and `runtime progress`. No direct duplicate PR was found. ## What Changed - Refreshed git workspace sync/index handling after sandbox restore/export operations. - Added finalization status that reports whether the restored host checkout is clean or dirty. - Preserved sandbox-managed runtime progress callbacks around restore and finalization. - Added adapter-utils tests covering the host index drift repair and clean/dirty finalization signal. ## Verification - Local PII scan before push: high-confidence secret patterns, internal issue links, local user paths, and private URL patterns checked across all three split diffs; no real secrets or internal links found. - `git diff --check feat/sandbox-runtime-status..fix/sandbox-restore-index-sync` - `pnpm exec vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — 1 file, 10 tests passed. - `pnpm run typecheck` passed on `fix/sandbox-restore-index-sync`. - `pnpm run build` passed on `fix/sandbox-restore-index-sync`; Vite reported existing CSS `::highlight` and chunk-size warnings. - `pnpm run test:run` was attempted on `fix/sandbox-restore-index-sync`; it failed in two unrelated broad-suite tests. One depends on this host's Git default branch behavior, and one depends on local Claude model-discovery environment. The changed focused suite above passes. ## Risks - Git index refresh behavior needs to remain conservative so it does not hide real uncommitted user changes. - The clean/dirty finalization signal is diagnostic; it should not by itself mark a sandbox run successful or failed. - This PR is stacked on the runtime-status branch so finalization progress can reuse the same callback path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex coding agent, with shell/tool execution in a local worktree. Exact context-window metadata is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-06-24 23:21:13 -07:00
3 people	a27e5ad002	Add ephemeral sandbox runtime status plumbing (#8593) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandboxed agent runs can spend meaningful time preparing a remote workspace before the agent transcript shows useful output. > - Operators need short, current progress text for those setup phases, but that text should not become durable run history. > - The existing live-run websocket path already carries run updates to the UI, so the backend can reuse that channel instead of adding polling. > - This pull request adds an ephemeral runtime-progress contract, a process-local status store, and heartbeat integration for sandbox-managed runs. > - The benefit is a clearer active-run experience without database migrations or persistent progress rows. ## Linked Issues or Issue Description Refs #248 No exact public GitHub issue was found for this status-message plumbing. The underlying problem is that active sandboxed runs currently have setup phases, such as workspace sync and restore, where the operator cannot see concise current progress through the live run state. This PR addresses that gap for the backend/runtime layer while keeping progress messages ephemeral. GitHub search performed for related or duplicate work: `sandbox runtime status`, `sandbox restore index`, and `runtime progress`. No direct duplicate PR was found. ## What Changed - Added shared runtime-progress types and the `heartbeat.run.progress` live event type. - Added a process-local heartbeat run runtime-status store with TTL, bounded/redacted messages, and terminal cleanup. - Threaded runtime progress callbacks through heartbeat execution and active/live run serialization. - Emitted sandbox-managed runtime phase updates for sync, adapter startup, restore/export, and finalization paths. - Added backend and adapter-utils tests for ephemeral status behavior, terminal cleanup, live serialization, and sandbox progress callbacks. ## Verification - `pnpm install --frozen-lockfile` - Local PII scan before push: high-confidence secret patterns, internal issue links, local user paths, and private URL patterns checked across all three split diffs; no real secrets or internal links found. The only secret-like text is an intentional fake test fixture (`sk-test-secret`). - `git diff --check origin/master..feat/sandbox-runtime-status` - `pnpm exec vitest run server/src/services/heartbeat-run-runtime-status.test.ts server/src/__tests__/heartbeat-runtime-state.test.ts server/src/__tests__/agent-live-run-routes.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — 4 files, 23 tests passed. - `pnpm run typecheck` passed on both top stacks that include this branch: `feat/sandbox-status-ui` and `fix/sandbox-restore-index-sync`. - `pnpm run build` passed on both top stacks that include this branch; Vite reported existing CSS `::highlight` and chunk-size warnings. - `pnpm run test:run` was attempted on `fix/sandbox-restore-index-sync`; it failed in two unrelated broad-suite tests. One depends on this host's Git default branch behavior, and one depends on local Claude model-discovery environment. The changed focused suites above pass. ## Risks - Runtime progress is process-local by design, so status disappears after TTL, terminal cleanup, or server restart. - Clients that do not consume `heartbeat.run.progress` simply keep existing behavior. - Message redaction is intentionally generic; overly specific phase details should stay out of runtime-progress payloads. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex coding agent, with shell/tool execution in a local worktree. Exact context-window metadata is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip CTO <cto@paperclip.local> Co-authored-by: Paperclip CTO <noreply@paperclip.ing>	2026-06-24 23:17:39 -07:00
Devin FoleyandPaperclip	51ffbb380f	Exclude transient Codex home dirs from sandbox sync (#8581) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Local adapters can run agents against sandboxed execution targets by syncing the workspace and selected runtime assets into the sandbox. > - The Codex local adapter includes a managed Codex home asset so sandboxed Codex runs can use the expected auth, config, skills, and session state. > - That asset follows symlinks, which is useful for real Codex home content but unsafe for transient launcher directories. > - Transient `tmp` and `.tmp` directories can contain symlinks to large host binaries, so the sandbox archive can inline large executable targets instead of just the small home directory content. > - This pull request excludes transient Codex home directories from the sandbox home asset while preserving the required Codex home files. > - The benefit is a much smaller and more predictable sandbox setup upload without changing the runtime files Codex actually needs. ## Linked Issues or Issue Description No public issue was found for this exact sandbox archive-size bug. Bug description: - What happened: sandboxed `codex_local` runs sync the managed Codex home as a `home` asset with `followSymlinks` enabled. If transient Codex home dirs such as `tmp` or `.tmp` contain symlinks to a large host binary, the archive can inline that binary and make `Syncing home to sandbox` much larger than the managed home directory itself. - Expected behavior: sandbox setup should include the Codex home files needed for auth, config, skills, and session continuity, but should not archive transient launcher scratch directories. - Reproduction shape: create a managed Codex home with normal auth/config/skills files and a `tmp/arg0` or `.tmp` symlink to a large host executable, then start a sandboxed `codex_local` run. The home asset archive grows by the symlink target size. - Version/commit: observed on local `master` before this change. - Related public context: #5028 covers a different managed Codex home reliability issue around stale auth files; this PR addresses sandbox archive bloat from transient symlink targets. ## What Changed - Excluded `tmp` and `.tmp` from the Codex `home` asset that is uploaded for sandboxed runs. - Added regression coverage proving transient symlinked home dirs are excluded from the tar while required auth/config/skills files remain included. - Kept `followSymlinks` behavior for the rest of the Codex home asset so existing non-transient symlink behavior is preserved. ## Verification - `git diff --check` - Local PII/secret pattern scan over the committed diff - `pnpm exec vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts` - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/adapter-codex-local typecheck` ## Risks Low risk. The exclusion is limited to transient Codex home scratch directories, and the regression test verifies the files needed in the sandbox are still archived. The main compatibility risk is if a user intentionally placed required persistent Codex state under `tmp` or `.tmp`; those paths are treated as volatile scratch space by this change. ## Model Used OpenAI Codex coding agent based on GPT-5, with shell, git, and GitHub CLI tool use. Exact hosted model build and context-window size were not exposed in the local adapter runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-06-24 07:31:17 -07:00
Devin FoleyandPaperclip	2c98c8e1e5	Fix sandbox git publishing and large workspace uploads (#8422) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox and SSH runtimes need to preserve agent work across isolated execution environments > - Git-backed workspaces were being copied mostly as filesystem archives, which breaks when `.git` points outside the mounted workspace and makes sandbox agents unable to publish their own branches > - Large ignored dependency trees could also be swept into the sandbox overlay, causing multi-GB transfers and max-string failures in some sandbox clients > - This pull request makes sandbox runtime setup use a git-backed HEAD sync plus a small dirty/untracked overlay, and bounds sandbox file transfers so large archives do not need one huge string > - The benefit is that sandbox agents can commit and push from a usable git checkout without uploading dependency trees such as `node_modules` ## Linked Issues or Issue Description Refs #8395 No public duplicate issue or PR was found after searches for `sandbox git workspace`, `git push sandbox`, and `node_modules sandbox upload`. Bug report: - What happened: sandbox-backed agent workspaces could receive a `.git` file that pointed at host-only git state, leaving the sandbox unable to run normal git workflows. The sandbox overlay upload could also include ignored dependency directories, creating very large transfers. - Expected behavior: sandbox and remote runtimes should prepare a usable git-backed workspace, copy only the necessary workspace overlay, and restore git history plus file changes without depending on a host-only `.git` path. - Steps to reproduce: 1. Run an agent in a sandbox-backed workspace whose local git checkout is a worktree. 2. Ask the agent to complete a GitHub workflow that requires commit/push access. 3. Observe that git operations can fail inside the sandbox, and ignored dependency trees can be uploaded as part of the workspace overlay. - Paperclip version or commit: reproduced against `master` before this PR, base `7aa212296eb1`. - Deployment mode: local dev / sandbox-backed runtime. - Installation method: built from source. - Agent adapters involved: local adapters using shared adapter-utils runtime preparation. - Database mode: not database-related. - Access context: agent runtime. - Local verification environment: Node.js v25.6.1, pnpm 9.15.4, macOS arm64. - Privacy checklist: all pasted output was reviewed for secrets, private hostnames, local usernames, and internal instance links. ## What Changed - Added a GitHub workflow push preflight so agent runs can detect missing push credentials when a workflow explicitly needs GitHub publishing. - Added shared git workspace sync helpers for shallow HEAD import/export and dirty/untracked overlay tracking. - Updated sandbox managed runtime setup to use git history plus a selected overlay instead of uploading the full local workspace for git-backed workspaces. - Bounded sandbox archive upload/download paths so large payloads stream or chunk instead of materializing one oversized string. - Excluded `.git` and ignored dependency trees from sandbox upload, download, and restore baselines while preserving local ignored directories during sync-back. - Added focused tests for git workspace sync, sandbox overlay selection, transfer chunking, and heartbeat push-preflight behavior. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/git-workspace-sync.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts packages/adapter-utils/src/command-managed-runtime.test.ts server/src/__tests__/heartbeat-project-env.test.ts server/src/__tests__/heartbeat-workspace-session.test.ts` - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm build` - Public-hygiene scan of the PR diff and commit messages for internal issue ids, local paths, private hostnames, and obvious token patterns. ## Risks - Medium risk: this changes sandbox runtime synchronization semantics for git-backed workspaces, especially around dirty tracked files, untracked files, deleted paths, and ignored files. - The main mitigation is focused test coverage for upload contents, restore exclusions, and git round-trip behavior. - The SSH runtime keeps the current bundle-based implementation from `master`; this PR only aligns shared excludes and sandbox behavior with that model. ## Model Used OpenAI Codex, GPT-5-based coding agent, tool-enabled shell/git/GitHub workflow, with code execution and repository inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-06-20 22:03:55 -07:00
Devin FoleyandPaperclip	33353ce62b	feat(skills): remove bundled paperclip-dev skill and retire required skill attribute (#7029) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Local adapters (Claude, Codex, Cursor, Gemini, Grok, OpenCode, Pi, ACPX) ship bundled "skills" — opinionated Markdown prompt bundles materialized into the agent's runtime > - One of those bundled skills, `paperclip-dev`, existed to let agents develop Paperclip itself; it has now moved to its own external repo and no longer belongs in the core tree > - The adapter skill model also carried a `required` / `requiredReason` attribute plus a `paperclip_required` `AdapterSkillOrigin` variant, all of which only existed to mark bundled skills as non-optional in the UI and adapter sync logic > - With `paperclip-dev` gone, no bundled skill is "required" anymore, and the type / runtime surface for `required` is dead weight — but it is computed at request time and never persisted, so a clean removal is safe (no compatibility shim needed) > - This pull request deletes `skills/paperclip-dev/` and removes every trace of the `required` / `requiredReason` field and the `paperclip_required` origin across shared types, validators, adapter-utils, all eight local adapters, server routes, the company-skills service, the UI, the storybook fixtures, and the test suite > - The benefit is a smaller, simpler adapter-skill surface: one origin (`company_managed`) for managed bundled skills, `resolvePaperclipDesiredSkillNames` collapses to "just the configured desired set", and the AgentDetail skills tab no longer renders a "Required by Paperclip" section that no longer applies ## Linked Issues or Issue Description <!-- No existing public GitHub issue; describing the underlying work inline (feature_request template fields). --> **Summary** Remove the bundled `paperclip-dev` skill (now maintained in its own external repo) and retire the `required` / `requiredReason` skill attribute and the `paperclip_required` skill origin, which only existed to support it. **Problem or motivation** `paperclip-dev` is the only bundled skill that was ever marked "required". Now that it lives in a separate repository, shipping it inside the core tree is wrong, and the entire `required` surface (a type field, a validator field, a synthesized `paperclip_required` origin, UI "Required by Paperclip" section, and required-skill merging in the desired-skills calculation) becomes dead weight. The `required` value is computed at request time and never persisted, so it can be removed cleanly without a migration or compatibility shim. **Proposed solution** Delete `skills/paperclip-dev/`, drop the `required` / `requiredReason` fields and `paperclip_required` origin everywhere they are produced or consumed, collapse managed-skill origin to a single `company_managed` value, and simplify `resolvePaperclipDesiredSkillNames` to return only the configured desired set. **Alternatives considered** Keeping the `required` attribute as a no-op for forward compatibility — rejected because it is request-time only (nothing persists it), so leaving it in place is pure dead surface area with no callers. **Roadmap alignment** Internal cleanup / dead-code removal that simplifies the adapter-skill surface; it does not introduce or duplicate any planned core feature in ROADMAP.md. ## What Changed - Deleted bundled `skills/paperclip-dev/` (moved to a separate repo). - Dropped `required`, `requiredReason`, and the `paperclip_required` origin from `packages/shared/src/types/adapter-skills.ts`, `packages/shared/src/validators/adapter-skills.ts`, and `packages/adapter-utils/src/types.ts`. - In `packages/adapter-utils/src/server-utils.ts`: removed `readSkillRequired()`; dropped `required`/`requiredReason` from `listPaperclipSkillEntries()`, `normalizeConfiguredPaperclipRuntimeSkills()`, `buildPersistentSkillSnapshot()`, and `PaperclipSkillEntry`; collapsed `buildManagedSkillOrigin()` to always return `company_managed`; simplified `resolvePaperclipDesiredSkillNames()` to return only the configured desired set (signature preserved so adapter call sites are untouched). - Walked all eight local adapters (`acpx-local`, `claude-local`, `codex-local`, `cursor-local`, `gemini-local`, `grok-local`, `opencode-local`, `pi-local`) and removed every remaining `requiredReason` / `paperclip_required` reference. - `server/src/services/company-skills.ts`: dropped the `required = sourceKind === "paperclip_bundled"` synthesis when listing runtime skill entries. - `server/src/routes/agents.ts`: removed required-skill merging from the desired-skills calculation in the persist-config path and the unsupported-snapshot path (keeping the current version-aware `desiredSkillEntries` structure). - `ui/src/pages/AgentDetail.tsx`: dropped required-based filters, the required tooltip, and the entire "Required by Paperclip" section from the agent skills tab; storybook fixtures in `ui/storybook/stories/acpx-local.stories.tsx` cleaned up to match. - Tests: deleted the `required: false` case in `paperclip-skill-utils.test.ts` and the "keeps required bundled skills installed" case in every `*-local-skill-sync.test.ts`; `acpx-local-execute.test.ts`, `cursor-local-execute.test.ts`, `cursor-local-skill-sync.test.ts`, `agent-skills-routes.test.ts`, and `packages/adapter-utils/src/server-utils.test.ts` were updated to drop removed fields and map `origin: "paperclip_required"` → `"company_managed"`. - `server/src/adapters/registry.ts`: two `as unknown as ServerAdapterModule["..."]` casts on `hermesListSkills` / `hermesSyncSkills` (matching the existing `executeHermesLocal` pattern). `hermes-paperclip-adapter@0.2.0` still depends on the published `@paperclipai/adapter-utils` which keeps the retired `paperclip_required` variant; the cast bridges the workspace-vs-published type mismatch at the registry seam and can drop once hermes upgrades. ## Verification Run from the workspace root: ```sh grep -rn "skills/paperclip-dev" . grep -rn "paperclip_required" --include="*.ts" --include="*.tsx" . grep -rn "requiredReason" --include="*.ts" --include="*.tsx" . pnpm -w typecheck pnpm --filter @paperclipai/server exec vitest run paperclip-skill-utils pnpm --filter @paperclipai/server exec vitest run skill-sync ``` The first three greps return only the explanatory comment in `server/src/adapters/registry.ts` (no live `paperclip_required` / `requiredReason` usage) and zero `skills/paperclip-dev` source hits. Locally: - `pnpm -w typecheck` → all packages this PR touches pass (adapter-utils, shared, server, ui, cli, and the cursor/gemini/opencode/pi adapters). - Affected vitest suites pass: `paperclip-skill-utils`, `server-utils`, all eight `*-local-skill-sync`, `agent-skills-routes`, and the `acpx`/`cursor`/`pi` execute suites. ## Risks - Behavioral shift in the agent skills UI: the "Required by Paperclip" section disappears. No bundled skill is required anymore, so this only affects environments that previously surfaced `paperclip-dev` as a forced-on row; those installs will see the skill move into the regular "company-managed" list (and be uninstalled on next sync unless explicitly listed as desired). - Existing agents may still have the string `"paperclip-dev"` in their persisted `desiredSkills`. That entry is inert (no source for it to install from); a one-time DB cleanup is out of scope. Low risk. - Hermes adapter type bridge: two casts in `registry.ts` paper over a type-only divergence between the workspace `@paperclipai/adapter-utils` and the published version still pinned by `hermes-paperclip-adapter@0.2.0`. Runtime behavior is unaffected because the retired `paperclip_required` value is no longer produced by anything in this tree. The casts can be removed once hermes upgrades its dependency. ## Model Used - Provider: Anthropic - Model: Claude Opus 4.7 (`claude-opus-4-7`) - Capability: agent tool use via Paperclip's `claude_local` adapter ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-06-20 21:58:44 -07:00
Devin FoleyandPaperclip	07e98d2b2c	feat(adapter-utils): add observable sandbox sync progress (#8395) ... ## Thinking Path > - Paperclip is the control plane for running AI-agent companies, so long-running remote work needs to stay observable to human operators. > - Cloud / sandbox agents are an active roadmap area, and their workspace sync path is part of the runtime substrate every remote coding run depends on. > - In the sandbox and SSH execution-target flows, Paperclip logged that sync had started, then often went silent for the full transfer window. > - That made large remote syncs feel stalled and also hid a real performance problem in the command-managed sandbox upload path. > - The first part of this pull request threads a throttled progress-reporting surface through the adapter execution-target stack so sync and restore work can emit meaningful updates. > - The second part fixes the command-managed sandbox transport itself: it removes the old serial 32KB append bottleneck, but also falls back away from the single-stream path when a provider-backed sandbox runner cannot surface mid-flight stdin progress. > - The result is that sandbox and SSH transfers are both faster and more observable, including the live Daytona-style sandbox case that previously only emitted `0%` and `100%`. ## Linked Issues or Issue Description No public GitHub issue exists for this bug, so it is described inline below following the bug report template. ### What happened - Remote sandbox and SSH workspace syncs could spend a long time transferring data while only logging a start line (`Syncing workspace and runtime assets to sandbox environment`) and, at best, a terminal line. - In the command-managed sandbox path, the original upload implementation also paid a large performance cost by appending base64 data in many small sequential remote writes (thousands of serial 32KB round-trips on a large workspace). - After the initial transport rewrite, live provider-backed sandbox runs still only emitted `0%` and `100%` because the single-stream stdin RPC buffered progress until completion. ### Expected behavior - Long-running sandbox and SSH syncs should periodically report how much of the transfer is complete (a percentage and/or MB transferred) so an operator can tell the run is healthy and making progress rather than stuck. - The main sandbox upload path should not be artificially slow. - A transfer that fails partway should leave an explicit failure marker in the log rather than a dangling intermediate percentage. ### Steps to reproduce 1. Run an agent against a sandbox (command-managed) or SSH (remote-managed) execution target with a non-trivial workspace. 2. Watch the run log during the workspace/runtime asset sync phase. 3. Observe that the log shows the sync start line and then stays silent for the full transfer (live provider-backed sandbox runs only show `0%` then `100%`). ### Paperclip version or commit - Branch `PAPA-825-provide-status-updates-when-syncing-sandboxes` off `master`. ### Deployment mode - Self-hosted / local instance using sandbox (command-managed) and SSH (remote-managed) execution targets, including provider-backed sandbox runners. ## What Changed - Added shared throttled runtime progress reporting and threaded `onProgress` through the adapter execution-target surface and adapter `execute.ts` entrypoints. - Added sync and restore progress reporting for the command-managed sandbox path and the SSH/remote-managed path, including git import/export progress where totals are known. - Reworked command-managed sandbox transfer behavior so uploads use the faster single-stream path when appropriate, but fall back to chunked progress-emitting writes when the runner cannot expose mid-stream stdin progress. - Marked provider-backed environment sandbox runners as not supporting single-stream stdin progress so live sandbox runs emit meaningful intermediate updates instead of only `0%` and `100%`. - Emit an explicit terminal failure marker (`failed at NN% (x/y MB)`) when an SSH/tar transfer rejects, so a failed sync no longer leaves a dangling intermediate percentage in the log. - Run the SSH sync/restore size estimate (local directory walk / remote `du` probe) concurrently with the transfer instead of awaiting it before opening the pipe, so progress instrumentation no longer adds startup latency proportional to workspace file count. - Added and extended focused regression coverage for runtime progress throttling and the new failure marker, command-managed sandbox transfers, sandbox orchestration, SSH transfer progress, and environment execution-target wiring. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/runtime-progress.test.ts packages/adapter-utils/src/ssh-fixture.test.ts packages/adapter-utils/src/command-managed-runtime.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts` - `pnpm exec vitest run packages/adapter-utils/src/command-managed-runtime.test.ts server/src/__tests__/environment-execution-target.test.ts` - `npx tsc --noEmit` for `packages/adapter-utils` ## Risks - The provider-backed sandbox fallback now prefers chunked command-managed writes when progress hooks are active, so small-to-medium uploads may trade some raw throughput for observable intermediate progress on runtimes that cannot surface true mid-stream stdin progress. - Progress percentages on tar-based transfers still depend on estimates in some cases, so operators may briefly see MB-only lines before the estimate resolves, then near-final clamping before the terminal `100%` line. - This PR changes shared execution-target behavior used by multiple adapters, so regressions would most likely appear in remote runtime setup/teardown flows rather than in a single adapter. ## Model Used - Initial implementation: OpenAI GPT-5.4 via Codex local agent (`codex_local`), high reasoning mode. - Observability follow-ups (failure marker, concurrent size estimate, added tests): Claude Opus 4.8 via Claude Code (`claude_local`). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-06-20 13:03:42 -07:00
uky333andFull-Stack Engineer	eb68198c42	feat(adapter-claude-local): emit errorCode claude_refusal on stop_reason refusal (#8314) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents run through adapters; `@paperclipai/adapter-claude-local` shells out to the Claude CLI and maps its result JSON into Paperclip's run outcome (`errorCode` / `errorFamily`) > - A Fable 5 policy refusal exits the CLI cleanly (`exitCode=0`, `is_error=false`) with `stop_reason: "refusal"`, which the adapter mapped to `errorCode: null` > - Paperclip therefore recorded the run as a silent success, so the agent's heartbeat stalled with no signal for operators and no hook to retry or alert > - This pull request adds a refusal detector to the adapter so a refusal resolves to a distinct `errorCode: "claude_refusal"` (`errorFamily: "model_refusal"`) > - The benefit is that policy refusals become observable: the server persists the code, operators can see it on the run, and downstream retry/alert policy can key off it ## Linked Issues or Issue Description No public GitHub issue exists. Describing the underlying bug in-PR, following `.github/ISSUE_TEMPLATE/bug_report.yml`: **What happened?** When the Claude CLI returns `stop_reason: "refusal"` (a model policy refusal), it still exits cleanly (`exitCode=0`, `is_error=false`). `@paperclipai/adapter-claude-local` keyed refusal detection off the failure flag during error-code resolution, so it returned `errorCode: null` — a refused run was indistinguishable from a successful one, recorded by Paperclip as a silent success with no signal to retry or alert. **Expected behavior** A refusal should resolve to a distinct, non-null error code (`errorCode: "claude_refusal"`, `errorFamily: "model_refusal"`) so the server can surface it to operators and downstream policy can react. **Steps to reproduce** 1. Run any agent on the `claude-local` adapter with a prompt the model refuses on policy grounds. 2. The Claude CLI exits `0` with `is_error=false` and `stop_reason: "refusal"`. 3. Observe the run resolves `errorCode: null` (pre-fix) instead of a refusal-specific code. **Paperclip version or commit** `adapter-claude-local` v2026.609.0 (`dist/server/execute.js` error-code resolution block); reproduced on `master`. **Deployment mode** Local / self-hosted. **Agent adapter(s) involved** `@paperclipai/adapter-claude-local` (Claude Code, local). ## What Changed - **`packages/adapters/claude-local/src/server/parse.ts`** — new `isClaudeRefusalResult()` helper, parallel to `isClaudeMaxTurnsResult()`. Detects `stop_reason` / `stopReason` / `error_code` / `errorCode` == `refusal` and `subtype` == `model_refusal`, case/whitespace tolerant. - **`packages/adapters/claude-local/src/server/execute.ts`** — compute the refusal flag independent of the `failed` flag (a refusal exits cleanly, so keying off `failed` would miss it); resolve `errorCode: "claude_refusal"` and `errorFamily: "model_refusal"`; surface `stopReason: "refusal"` in `resultJson`. - **`packages/adapter-utils/src/types.ts`** — widen `AdapterExecutionErrorFamily` with `"model_refusal"`. - **`packages/adapters/claude-local/src/server/index.ts`** — export the new helper. - **`packages/adapters/claude-local/src/server/parse.test.ts`** — 7 new unit tests. ## Verification ``` npx vitest run packages/adapters/claude-local # 35 passed (7 new) npx tsc --noEmit -p packages/adapter-utils # clean npx tsc --noEmit -p packages/adapters/claude-local # clean ``` Manual: a result JSON with `stop_reason: "refusal"` on a clean exit now resolves `errorCode: "claude_refusal"` and `errorFamily: "model_refusal"`; non-refusal results are unaffected. ## Risks Low risk. Additive only — it introduces a new error code on a path that previously returned `null`; no existing error code or success path changes. `claude_refusal` is deliberately **not** added to the transient-upstream retry set: a refusal is deterministic, so retrying the same prompt yields the same refusal. It surfaces as a distinct non-transient code for operators; downstream retry/alert policy can key off `claude_refusal` / `errorFamily: model_refusal` later. ## Model Used Claude Opus 4.8 (`claude-opus-4-8`), extended-thinking mode, run via Claude Code with tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above (no related PRs found) - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (N/A — no UI changes) - [x] I have updated relevant documentation to reflect my changes (N/A — internal adapter error-code addition; no user-facing docs) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (this push re-runs the gates; will confirm once green) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (prior run was 4/5 on a PR-description note now addressed; will confirm on re-run) - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Full-Stack Engineer <noreply@paperclip.ing>	2026-06-19 23:24:06 -07:00
3 people	a71c4b6782	[codex] feat(watchdog): add task watchdog control plane (#8339) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The task lifecycle and recovery subsystems decide when agent work is still productive, stalled, or ready for review. > - Existing recovery paths can observe stopped or incomplete work, but there was no first-class per-task watchdog model with scoped review permissions. > - Watchdog follow-ups also need strict boundaries so recovery/status-only runs cannot mutate approvals or perform deliverable work. > - This pull request adds the task watchdog data model, API/service layer, scheduler/review flow, adapter wake context, UI configuration surfaces, and docs. > - The branch has been rebased onto current `paperclipai/paperclip` `master`; the watchdog migration is now ordered after master's latest migrations as `0104_issue_watchdogs`. > - The benefit is a more explicit task-review loop that preserves Paperclip's single-assignee and governance invariants while making stalled work easier to route. ## Linked Issues or Issue Description No linked GitHub issue. Paperclip task: [PAP-11275](/PAP/issues/PAP-11275). ## Problem or motivation Task recovery needs a first-class watchdog path that can inspect stopped work and create scoped follow-ups without bypassing normal task ownership. Board/UI users need a way to configure watchdogs on tasks and see watchdog-related live work. Recovery/status-only runs must remain limited to status reporting and must not create approvals, link approvals, or submit approval comments. ## Proposed solution Add a task-watchdog data model, scheduler/classifier, scoped mutation guard, adapter wake context, API/UI configuration surfaces, and documentation so watchdog agents can review stopped task subtrees under explicit boundaries. ## Alternatives considered Reuse the existing recovery-action flow only. That would keep stopped-work detection implicit, make per-task watchdog assignment harder to expose in the UI, and would not provide a durable scoped-review issue for stalled task trees. ## Roadmap alignment This is Paperclip control-plane lifecycle infrastructure for task execution and recovery. I checked `ROADMAP.md`; this PR does not duplicate an existing planned core item. ## What Changed - Added issue watchdog schema, migration, shared contracts, validators, CRUD API, and service support. - Added task watchdog scheduler/classifier behavior, scoped mutation enforcement, adapter wake context, and default watchdog mandate guidance. - Added UI surfaces for configuring watchdogs on new/existing tasks, viewing watchdog activity, and exposing the experimental setting. - Added docs for the user-facing task watchdog workflow and implementation semantics. - Gated new-task watchdog setup behind `enableTaskWatchdogs` and blocked cheap status-only recovery runs from approval mutations. - Rebased onto current `master` and renumbered the idempotent watchdog migration from the branch-local `0102_issue_watchdogs` slot to `0104_issue_watchdogs`. - Addressed Greptile feedback by loading watchdog classifier input with a recursive subtree query and centralizing the watchdog origin-kind constant. - Added and updated focused server/UI tests for watchdog routes, scheduler/classifier behavior, scope boundaries, live task visibility, settings, and new issue dialog behavior. ## Verification - `pnpm vitest run server/src/__tests__/task-watchdogs-scheduler.test.ts server/src/__tests__/task-watchdogs-classifier.test.ts` - `pnpm vitest run server/src/__tests__/approval-routes-idempotency.test.ts server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts` - `pnpm vitest run ui/src/components/NewIssueDialog.test.tsx` - `pnpm --filter @paperclipai/server typecheck` - `git diff --check` - Verified the PR diff does not include `pnpm-lock.yaml` or `.github/workflows`. ## Risks - Medium risk: this introduces a new task lifecycle surface touching DB schema, server routes/services, adapter wake context, and UI task configuration. - Watchdog scheduling behavior depends on the new experimental setting and runtime context checks behaving consistently across local and production agents. - The watchdog migration is idempotent (`IF NOT EXISTS` / duplicate-object guards) so users who tried the previous branch-local migration number should not get duplicate-object failures. - CI and the second Greptile pass are pending after the latest review-fix push. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5-class coding agent in the Paperclip workspace. Exact runtime model id and context window were not exposed to the agent; tool use and local command execution were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots — N/A per Paperclip task instruction: do not add screenshots/images to this PR unless they are specifically part of the work. - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-19 15:38:52 -05:00
dependabot[bot]	6f142a60ce	build(deps-dev): bump @types/node from 22.19.11 to 22.19.21 (#7748) ... Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 22.19.11 to 22.19.21. <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-06-18 15:46:06 -07:00