mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-22 16:51:28 +02:00
4e8cd757eb696b2d61a0dce75f0b7c0312bfdf3a
159
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
5ed0b74b34 |
fix(runtime): scope PAPERCLIP_ env-binding strip to reserved keys (#9974)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent runs get their environment from user/adapter/project/routine env bindings resolved by the server heartbeat, plus `PAPERCLIP_*` runtime vars (identity, wake, workspace, API access) injected by the harness > - The heartbeat stripped **every** `PAPERCLIP_`-prefixed binding before resolution, so legitimately user-named keys (e.g. cloud provider token bindings like `PAPERCLIP_CLOUD_PROD_PROVIDER_RAILWAY_*`) were silently dropped and never reached the run env > - At the same time, several adapters honored an explicitly configured `PAPERCLIP_API_KEY` over the harness-minted run token, which is exactly the one key config must never control > - This pull request replaces the blanket prefix strip with a precise three-rule policy: never accept `PAPERCLIP_API_KEY` from config, always let harness-assigned runtime vars win, and let every other `PAPERCLIP_*`-named user binding flow through > - The benefit is that user secrets with a `PAPERCLIP_`-style name work like any other binding, while runtime identity and API credentials stay fully harness-controlled ## Linked Issues or Issue Description **Bug description** (no public issue exists): - **What happened:** Env bindings whose key starts with `PAPERCLIP_` (e.g. a cloud provider token a user deliberately named `PAPERCLIP_CLOUD_PROD_PROVIDER_RAILWAY_TOKEN`) were silently stripped by the server before secret resolution, so the spawned agent never received them. No error, no access event — the variable just never appeared. - **Expected behavior:** A user-named `PAPERCLIP_*` binding should reach the run env unless the harness itself uses that key. Only `PAPERCLIP_API_KEY` should be categorically rejected, and harness-assigned runtime vars (`PAPERCLIP_RUN_ID`, `PAPERCLIP_AGENT_ID`, wake/workspace vars, …) should always win over config. - **Steps to reproduce:** Configure an agent/project env binding named `PAPERCLIP_<ANYTHING>` (plain or secret_ref), run a heartbeat, and inspect the spawned process env — the key is absent. - **Deployment mode:** local server, any local adapter. Related prior PRs (different, save-time/API-layer blanket-ban approach; this PR supersedes that direction with a runtime allow-except-reserved policy): Refs #8239, Refs #8439. ## What Changed - `server/src/services/heartbeat.ts`: the pre-resolution strip now removes only `PAPERCLIP_API_KEY` (hard denylist) instead of every `PAPERCLIP_`-prefixed binding; other `PAPERCLIP_*` keys flow into binding resolution. Low-trust inline-sensitive-env checks now also cover those keys. - `packages/adapter-utils/src/server-utils.ts`: new `isForbiddenConfigEnvKey()` helper; the shared `refreshPaperclipWorkspaceEnvForExecution` merge drops `PAPERCLIP_API_KEY` from config and keeps harness-assigned `PAPERCLIP_*` keys authoritative. - `packages/adapter-utils/src/acpx-engine/execute.ts`: removed the explicit-`PAPERCLIP_API_KEY`-from-config allowance; the run token (`authToken`) is now always applied; config `PAPERCLIP_API_KEY` is ignored. - All local adapters (`claude-local`, `codex-local`, `cursor-local`, `gemini-local`, `grok-local`, `opencode-local`, `pi-local`) plus `cursor-cloud`, `hermes`, and the server `process` adapter: removed `hasExplicitApiKey`-style allowances so the harness token always wins, and guarded the remaining unguarded env-merge loops (claude-local inline loop, process adapter) with the same policy. - Tests updated/added: heartbeat binding-strip test now asserts the three-rule policy; adapter-utils merge tests assert the `PAPERCLIP_API_KEY` ban and `PAPERCLIP_*` pass-through; acpx engine tests moved credential fixtures to `authToken` and assert config `PAPERCLIP_API_KEY` is ignored while other `PAPERCLIP_*` config keys forward and still bust the session fingerprint on rotation. ## Verification - `pnpm vitest run packages/adapter-utils/src/server-utils.test.ts packages/adapter-utils/src/acpx-engine/execute.test.ts` — 127 passed - `pnpm vitest run server/src/__tests__/heartbeat-project-env.test.ts server/src/__tests__/heartbeat-local-environment.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/codex-local-execute.test.ts server/src/__tests__/cursor-local-execute.test.ts server/src/__tests__/gemini-local-execute.test.ts` — 68 passed - Adapter package execute suites and the server tests touching API-key fixtures (`heartbeat-run-log`, `redaction`, `effective-run-config-fingerprints`, `agent-permissions-routes`) — green. Three pre-existing sandbox/SSH fixture failures reproduce identically on clean `master` on this host and are unrelated. - `pnpm --filter <pkg> typecheck` for server, adapter-utils, and all nine touched adapter packages — all pass. ## Risks - Behavioral change: a deployment that relied on configuring a static `PAPERCLIP_API_KEY` in adapter config env loses that override — by design; the harness-minted run token is now the only source. When no run token exists, no API key is injected at all. - `PAPERCLIP_*`-named user bindings now reach binding resolution and run envs; a key that collides with a harness runtime var is still discarded at merge time, so runtime identity/wake/workspace vars cannot be spoofed. - Low risk otherwise: no migrations, no API surface changes. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic Claude 5 family, Mythos-class tier), extended thinking enabled, agentic tool use (file edits, shell, test runner) via Claude Agent SDK. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
7985c1e000 |
refactor(codex-local): stage an allowlist for sandbox CODEX_HOME sync (#9972)
## Thinking Path
> - Paperclip manages AI agents for work; agents can run in remote
sandboxes via the codex-local adapter
> - When a remote sandbox run starts, the codex-local adapter syncs the
host CODEX_HOME directory into the sandbox so Codex can authenticate and
load its configuration
> - The old approach staged the entire CODEX_HOME and relied on a 4-name
denylist (`tmp`, `.tmp`, `sessions`, `shell_snapshots`) to exclude
unnecessary state
> - The denylist was too narrow: large host-local runtime files
(`logs_2.sqlite`, `plugins/`, etc.) still uploaded on every remote run —
state the sandbox never uses
> - Additionally, `config.toml` (which embeds the managed MCP
`Authorization: Bearer …` header) was staged at 0644, relying solely on
the staging directory's 0700 permission for protection — a
credential-bearing file without per-file least-privilege
> - This pull request replaces the denylist with an explicit allowlist:
only `{auth.json, config.toml, config.json, instructions.md, skills/}`
are staged into a private `0700` temp dir; all staged regular files are
written `0600` (fail-closed on any staging error)
> - The fix is adapter-local to `packages/adapters/codex-local` with
zero changes to the shared `packages/adapter-utils` seam; inbound
auth-merge and outbound copy-back are preserved unchanged
## Linked Issues or Issue Description
No public GitHub issue for this specific bug. Describing the problem
directly:
**Bug:** The codex-local remote sandbox sync uploads more CODEX_HOME
content than the sandbox uses, and stages a credential-bearing file with
an over-permissive mode.
- **What happened:** The codex-local adapter synced the entire
CODEX_HOME directory minus four directory names (`tmp`, `.tmp`,
`sessions`, `shell_snapshots`). Large host-local runtime artifacts — a
SQLite write-ahead log that can reach 420 MB, plugin directories, crash
logs — uploaded on every remote run. `config.toml` (which embeds the
managed MCP `Authorization: Bearer …` header) was staged at `0644`; only
the staging temp dir's `0700` mode prevented broader exposure.
- **Expected behavior:** Only the files Codex needs inside the sandbox
should sync: `auth.json`, `config.toml`, `config.json`,
`instructions.md`, and `skills/`. All staged regular files should use
mode `0600` (single-layer credential protection is insufficient for
secret-bearing files).
- **Steps to reproduce:** Run a remote codex sandbox from a host with a
mature CODEX_HOME. Observe the sync payload includes `sessions/`,
`*.sqlite`, `plugins/`, etc.
Related work: Refs #9621 (outbound auth sync-back in the same
codex-local adapter area; this PR covers the inbound staging direction)
## What Changed
- New `stageCodexHomeForSync(codexHomeDir)` helper in
`packages/adapters/codex-local/src/server/codex-home.ts` stages only
allowlisted files into a fresh `fs.mkdtemp`-created `0700` temp dir,
then registers that dir as the `home` asset (dropping the old `exclude`
field entirely)
- All staged regular files written with mode `0600` — auth.json holds
single-use OAuth tokens; config.toml embeds MCP bearer headers; least
privilege applies throughout
- Symlinks dereferenced to resolved file bytes (`auth.json` single-use
token, each `skills/` entry land as real files)
- Missing-but-optional allowlist entries skipped silently (keyring-only
mode with no `auth.json`; absent `config.json`)
- Staged temp dir removed on teardown AND on error; staging is
fail-closed — a staging error aborts the run rather than shipping a
partial CODEX_HOME
- `execute.ts` updated to call `stageCodexHomeForSync` instead of the
old whole-dir + denylist path; the `exclude` field is removed from the
`home` asset
- Adapter-local fix: zero diff to `packages/adapter-utils` (shared
managed-runtime seam untouched); inbound auth-merge (`provision`) and
outbound copy-back (`restore`) preserved unchanged
## Verification
```sh
# Full codex-local test suite
npx vitest run packages/adapters/codex-local
# 174/174 pass
# Shared seam regression — must be zero diff to packages/adapter-utils
npx vitest run packages/adapter-utils
# 14/14 pass; zero changes to packages/adapter-utils source
# TypeScript (both packages)
npx tsc --noEmit -p packages/adapters/codex-local/tsconfig.json
npx tsc --noEmit -p packages/adapter-utils/tsconfig.json
# Both clean
```
Key new test coverage
(`packages/adapters/codex-local/src/server/codex-home.test.ts`):
- Stager copies exactly the allowlisted entries and nothing else
- Staged `auth.json` dereferenced from symlink and written mode `0600`
- Staged temp dir created with mode `0700`
- ALL staged regular files mode `0600`, including `config.toml` carrying
an MCP bearer token (regression test fails against the pre-fix `0644`
path)
- Fail-closed: I/O error during staging removes the temp dir and
re-throws
- Missing-but-optional entries (`config.json`) skipped without error
- Dangling symlink entries skipped without error
Execute-path integration test (`execute.remote.test.ts`): home asset
uses staged dir; staged config preserves provider block; skills survive;
staged temp dir removed after run teardown.
## Risks
- **Allowlist narrower than denylist:** Any CODEX_HOME file not in
`{auth.json, config.toml, config.json, instructions.md, skills/}` is now
excluded. If a codex-local configuration stores additional files in
CODEX_HOME that the sandbox requires, those runs will miss that state.
This tradeoff is intentional — only known-necessary files sync.
- **Permission change:** `config.toml` previously staged at `0644`, now
staged at `0600`. Strictly more restrictive; no known behavioral impact
(the sandbox reads as the file owner). This was the security gap that
motivated the fix.
- **Fail-closed staging:** A staging error (disk full, temp dir creation
failed) now aborts the run rather than falling back to the raw
CODEX_HOME. This is the safer behavior, but it changes the failure mode
from "run with potentially wrong state" to "run aborted cleanly."
## Model Used
Claude Opus 4.8 (claude-opus-4-8), Anthropic. Reasoning/agentic tool-use
mode (extended thinking, multi-step tool calls).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Harold Kim <harold@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
||
|
Nicky Leach
|
14da75dfc7 | feat(codex-local): outbound auth copy-back as home-asset restore contribution (#9788) | ||
|
|
936215693c |
feat(codex-local): add outbound direction to codex-auth-merge decision predicate (#9787)
## Thinking Path > - Paperclip runs Codex as a sandboxed agent with its own CODEX_HOME; at the start of each run Paperclip merges the sandbox's auth.json with the host's so the agent can authenticate to the Codex API > - The merge uses `codex-auth-merge-decision.cjs`, a small CJS script that reads two `auth.json` snapshots and exits `10` (use source copy) or `20` (keep destination copy) > - Until now the script only handled the **inbound** direction (host → sandbox) with hard-coded sandbox/host semantics > - Sandbox-to-host copy-back (**outbound**) requires the same freshness guard but with swapped roles — and exposing both directions via a flag requires hard-coding knowledge about which role is "sandbox" and which is "host" into the predicate itself > - A cleaner design: the predicate is **direction-agnostic** — it accepts a `source` path and a `destination` path; the caller decides direction by which argument it passes first. The script answers one question: "should destination be replaced by source?" Exit `10` = yes (use source); exit `20` = no (keep destination) > - This PR refactors `codex-auth-merge-decision.cjs` to this source/destination model, removes the `--direction` flag, and extends coverage with a 15-row source/destination predicate matrix plus updated extract integration tests ## Linked Issues or Issue Description No public GitHub issue exists. The underlying problem: **Context** — Paperclip merges `auth.json` files between sandbox and host to keep Codex credentials in sync across agent runs. The existing decision predicate (`codex-auth-merge-decision.cjs`) handled the inbound direction (host→sandbox) with hard-coded sandbox/host naming. A copy-back (outbound) direction was needed, but rather than add a `--direction` flag that hard-codes role knowledge into the predicate, the design was simplified: the predicate is now **direction-agnostic**, taking `source` and `destination` paths and answering "replace destination with source?" The freshness guard is unified: exit `10` only when source and destination share the same `account_id`, same subscription-kind, both carry a parseable `last_refresh`, and `source.last_refresh > destination.last_refresh` (strict). Ties and null freshness keep the destination copy so a spent single-use refresh token (openai/codex#15502) is never written back over a valid one. Related PRs: - Refs #9785 — Phase 2: relocated the decision predicate from `adapter-utils` into `codex-local`; this PR builds on that merged change - Refs #9621 — community PR targeting Codex sandbox auth sync-back via a different approach; operates on different files (runtime layer vs. decision script), non-conflicting ## What Changed - `packages/adapters/codex-local/src/server/codex-auth-merge-decision.cjs` — refactored from direction-parameterized (sandbox/host with `--direction`) to **direction-agnostic source/destination** semantics. Args: first = source path, second = destination path. Exit `10` (use source) only when: same `account_id`, same subscription-kind, both carry a parseable `last_refresh`, and `source.last_refresh > destination.last_refresh` (strict). Every other case exits `20` (keep destination). Unknown/extra flags are still rejected with exit `1`. No token bytes are emitted to stdout/stderr. - `packages/adapters/codex-local/src/server/codex-auth-merge.test.ts` — rewritten to source/destination API: 15-row predicate matrix (strictly-newer→`10`; tie/older/identity-mismatch/null-refresh/apikey/kind-mismatch/unusable→`20`) plus token-bytes non-emission guard. The inbound extract integration suite updated to match. - Runtime callers (`codex-auth-merge-extract.sh`) already invoke `node decision <sandbox> <host>` = `(source, destination)` and branch on exit `10` → keep sandbox, so **no caller changes are needed**. **Behavioral note:** unifying the two directions under a single "use source only when strictly fresher" rule means a tie (exact equal `last_refresh`) now keeps the **destination** for both the inbound and outbound frames. For inbound restore, this means an exact timestamp tie keeps the host copy. On a tie, both credentials are the same generation — preferring the canonical destination (host) copy is more conservative. This is an intentional consequence of the unified guard. ## Verification ```sh # From the codex-local package root: pnpm vitest run src/server/codex-auth-merge.test.ts # Expected: all predicate rows green (15-row source/destination matrix + token-leak guard) # Plus: codex-auth-merge-extract integration rows green ``` The test suite drives the `.cjs` as a child process over temporary `auth.json` fixtures: - Source/destination predicate matrix (15 rows): strictly-newer→`10`; tie→`20`; older→`20`; identity-mismatch→`20`; source last_refresh unparseable→`20`; destination last_refresh unparseable→`20`; apikey either side→`20`; kind-mismatch→`20`; unusable either side→`20` - Token-bytes non-emission guard: asserts no auth token bytes appear in `stdout`/`stderr` - Inbound extract integration: `codex-auth-merge-extract.sh` end-to-end rows including tie/identity-mismatch/sandbox-newer cases ## Risks - **Tie behavior change**: exact `last_refresh` ties now always keep the destination copy (was: keep-sandbox for inbound). On a tie both credentials are the same generation; keeping the canonical destination (host) copy is the more conservative choice. Covered by test assertions. - **Token non-emission**: `stdout`/`stderr` of the `.cjs` must not echo credential bytes; covered by the token-bytes non-emission guard test. - **Unknown/extra flags**: handled with `process.exit(1)` rather than a silent default, so misconfigured callers surface errors immediately. - **Caller compatibility**: the only runtime caller, `codex-auth-merge-extract.sh`, already calls `node decision <sandbox> <host>` with no flags — compatible with the new source/destination API with no changes required. - **Low risk overall**: the script is exercised end-to-end via the extract integration tests; all paths covered. ## Model Used Claude claude-sonnet-4-6 (Anthropic, `claude-sonnet-4-6`), 200 k context window, tool use and agentic code execution enabled. Automated Paperclip multi-agent system. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
7ffeafad1f |
refactor(codex-local): relocate Codex auth-merge scripts + decision predicate into the adapter (#9785)
## Thinking Path > - Paperclip is an open source platform for managing AI agent teams, with adapter packages providing concrete runtime environments (e.g. `codex-local` runs a local OpenAI Codex sandbox) > - `adapter-utils` is the shared utilities package — it should hold only generic, adapter-agnostic primitives (sandbox lifecycle helpers, merge logic, type definitions) usable by every adapter > - Three files lived in `adapter-utils/src/` that are entirely Codex-specific: `codex-auth-merge-extract.sh` (shell script that extracts auth tokens), `codex-auth-merge-decision.cjs` (CJS decision helper), and `codex-auth-merge-scripts.ts` (TypeScript factory that wires them into the inbound provision seam added in PR #9778) > - Their presence in a "generic" package violates the adapter isolation principle and requires a cross-package build step to copy `.sh`/`.cjs` files into `codex-local/dist/server/` at build time > - This PR completes Phase 2 of the inbound-seam refactor: move all three files to `packages/adapters/codex-local/src/server/`, rebase the TypeScript imports, update `execute.ts` to import locally, move the Codex-auth tests into a new `codex-local` test file, and fix packaging so `codex-local` copies its own scripts to `dist/server/` > - Script bytes are identical after the move; no change to inbound auth-merge behavior or to which bytes cross the sandbox boundary > - The benefit is a clean ownership boundary: `adapter-utils` retains only generic runtime code, and the structural "Codex-free core" test in the adapter-utils suite validates this invariant going forward ## Linked Issues or Issue Description No pre-existing public GitHub issue. Describing the underlying problem inline: **Problem:** `packages/adapter-utils/src/` contains three files (`codex-auth-merge-extract.sh`, `codex-auth-merge-decision.cjs`, `codex-auth-merge-scripts.ts`) consumed exclusively by the `codex-local` adapter. Their presence in a generic utilities package violates adapter isolation and requires a cross-package build step (copy `.sh`/`.cjs` into `codex-local/dist/server/`). The inbound provision seam landed in PR #9778 routed these through `adapter-utils`; this PR finishes the relocation. **Related:** Refs #9778 (Phase 1 — generic asset-lifecycle-seam, now merged). ## What Changed - Moved `codex-auth-merge-extract.sh`, `codex-auth-merge-decision.cjs`, and `codex-auth-merge-scripts.ts` from `packages/adapter-utils/src/` → `packages/adapters/codex-local/src/server/` (script bytes are unchanged; `codex-auth-merge-scripts.ts` imports rebased to `adapter-utils` subpaths for `shellQuote` and `SandboxManagedRuntimeAssetProvision`) - `packages/adapters/codex-local/src/server/execute.ts`: updated import of `buildCodexAuthInboundProvision` from `adapter-utils` → local `./codex-auth-merge-scripts` - New `packages/adapters/codex-local/src/server/codex-auth-merge.test.ts` — 5 Codex-auth test rows extracted from `adapter-utils/src/workspace-restore-merge.test.ts`; the generic test file stays and no longer references Codex - `packages/adapters/codex-local/package.json`: build now copies `.sh`/`.cjs` from `src/server/` into `dist/server/` directly - `packages/adapter-utils/package.json`: removed the now-obsolete cross-package copy step for those files ## Verification ```sh # Type-check both affected packages pnpm --filter @paperclipai/adapter-utils --filter @paperclipai/adapter-codex-local typecheck # Codex auth-merge suite (moved tests — 5/5 pass) pnpm --filter @paperclipai/adapter-codex-local test -- --reporter=verbose codex-auth-merge # Full adapter-utils suite including the structural "core free of Codex literals" test (243 passed, 4 pre-existing skips) pnpm --filter @paperclipai/adapter-utils test # Confirm build copies scripts to dist/server pnpm --filter @paperclipai/adapter-codex-local build ls packages/adapters/codex-local/dist/server/*.sh packages/adapters/codex-local/dist/server/*.cjs ``` All of the above were run locally and passed before this PR was opened. ## Risks Low risk — pure file relocation: - No change to `.sh` or `.cjs` script bytes; the same content reaches the sandbox boundary as before - No behavioral change to inbound auth-merge logic from the sandbox's perspective - `adapter-utils`' structural "core free of Codex literals" test now validates that the relocation is complete and will catch any regression - Only `codex-local` consumed these files from `adapter-utils`; no other package in the monorepo imported them from there ## Model Used Claude Sonnet 4.6 (`claude-sonnet-4-6`) — Anthropic, 200k context window, tool use, agentic reasoning. Used to author the refactor. `Co-authored-by: Paperclip <noreply@paperclip.ing>` trailer present on all commits. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Harold Kim <harold@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
cf5ba4bbea |
feat(adapter-utils): generic per-asset lifecycle-contribution seam (#9778)
## Thinking Path
> - Paperclip's sandbox managed runtime is responsible for provisioning
the agent's execution environment — it extracts a home directory asset
into the sandbox before the adapter runs.
> - The sandbox runtime core was directly branching on the adapter key
(`codex`) to decide which merge scripts to stage and which merge-extract
command to run, coupling generic infrastructure to a specific adapter's
credential-merge protocol.
> - This makes it harder to add, remove, or modify per-adapter asset
provisioning without touching the runtime core; it also prevents other
adapters from contributing staged files or a custom extract command at
all.
> - The fix is to move the adapter-specific knowledge into the adapter
itself: the asset descriptor gains optional `provision` (stageFiles +
extractCommand) and `restore` contribution fields that any adapter can
populate, and the runtime core consumes them generically.
> - This pull request introduces those contribution fields, wires the
Codex adapter's inbound credential-merge as a `provision` contribution,
and removes the adapter-specific branching from the runtime core.
> - The benefit is a clean seam: the runtime core is now
adapter-agnostic for asset provisioning, the inbound behavior is
unchanged (same merge matrix, same scripts), and other adapters can
attach custom staged files or extract commands without modifying shared
infrastructure.
## Linked Issues or Issue Description
No pre-existing public GitHub issue. Describing the problem inline per
the feature template:
**Problem or motivation**
The sandbox managed-runtime asset provisioning in
`sandbox-managed-runtime.ts` branched directly on the adapter key
(`codex`) to decide which merge scripts to stage and which shell command
to use during asset extraction. This tight coupling prevents other
adapters from customizing their provisioning without modifying the
runtime core, and it means the runtime core must import and know about
adapter-specific merge scripts.
**Proposed solution**
Add an optional `provision` contribution (array of `stageFiles` entries
+ an `extractCommand` string) and an optional `restore` contribution to
the asset descriptor returned by adapters. The runtime core now consumes
these generically — if a `provision` contribution is present, it stages
those files and uses the supplied command; otherwise it falls back to
the default `tar -xf` extraction. The Codex adapter populates the
`provision` contribution where it previously depended on core branching.
**Alternatives considered**
Keeping the adapter-specific logic in the core as a documented
exception; rejected because it makes the seam inextensible.
**Roadmap alignment**
Decoupling — removes a latent coupling between the runtime core and a
specific adapter.
## What Changed
- Added `provision` contribution field (`stageFiles: Array<{src, dest}>`
+ `extractCommand: string`) to the `SandboxManagedRuntimeAsset`
descriptor type in `adapter-utils`.
- Added `restore` contribution field (hook for post-restore logic,
populated in a later phase) to the descriptor.
- Removed adapter-key branching (`if adapterKey === 'codex'`) from the
runtime core in `sandbox-managed-runtime.ts`; the core now reads
`provision.stageFiles` and `provision.extractCommand` generically.
- Extracted Codex-specific merge-script paths and the merge-extract
command into `codex-auth-merge-scripts.ts` in `adapter-utils`; the Codex
adapter's `execute.ts` now attaches them as a `provision` contribution
when it builds its managed-home asset descriptor.
- Updated `execution-target.ts` to pass the extended asset type through
to the adapter call site so the new fields are load-bearing end-to-end.
- Added seam-proving unit tests in `sandbox-managed-runtime.test.ts`:
contribution-less asset uses the default path; a non-adapter asset
round-trips the generic provision+restore seam; a structural assertion
verifies the runtime core carries no Codex-specific string literals.
- Added one test in `workspace-restore-merge.test.ts` confirming the
inbound merge matrix is unaffected.
## Verification
```bash
# Unit tests (20 pass):
npx vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts packages/adapter-utils/src/workspace-restore-merge.test.ts
# Type-check both affected packages:
cd packages/adapter-utils && npx tsc --noEmit
cd packages/adapters/codex-local && npx tsc --noEmit
# Structural: runtime core carries no adapter string literals
grep -n 'codex\|auth\.json' packages/adapter-utils/src/sandbox-managed-runtime.ts
# Expected: zero matches
```
## Risks
**Low risk.** This is a behavior-preserving refactor: the inbound
provisioning output (which files get staged, which command runs) is
identical to before, now driven by the adapter-supplied contribution
instead of core branching. The existing inbound merge matrix tests are
the regression guard. No change to which bytes cross the sandbox
boundary. The SSH transport is untouched.
## Model Used
- **Provider:** Anthropic
- **Model:** Claude Sonnet 4.6 (`claude-sonnet-4-6`)
- **Context window:** 200 K tokens
- **Capabilities used:** tool use (file read/edit, bash execution,
Paperclip API), extended reasoning over multi-file TypeScript refactor
- **Mode:** agentic (Paperclip ACPX platform)
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Harold Kim <harold@paperclip.ing>
|
||
|
DottaandPaperclip
|
5a5c918705 |
fix(acpx): configure Codex models at startup (#9700)
Move Codex ACPX model, reasoning effort, and fast-mode settings into CODEX_CONFIG startup config so arbitrary model IDs avoid ACP session picker validation. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9a92124c63 |
fix(codex-local): raise default output-inactivity timeout to 30m (#9699)
## Thinking Path > - Paperclip is the open source control plane for managing AI-agent companies. > - Agent adapters are responsible for launching, observing, and terminating provider processes safely. > - The local Codex adapter uses an output-inactivity monitor to stop genuinely hung child processes. > - The existing seven-minute default also stopped legitimate long-running tasks that emitted no output while tests or remote checks were still running. > - Explicit per-agent timeout overrides already provide configuration flexibility, so the smallest safe correction is to increase only the default window. > - This pull request raises the default to 30 minutes, retains the existing termination behavior, and adds a regression assertion for the new value. > - The benefit is fewer unnecessary Codex restarts while still bounding genuinely silent processes well below the platform safety limit. ## Linked Issues or Issue Description **Pre-submission checklist** - Searched open and closed GitHub issues and pull requests; no duplicate fix exists. The original inactivity monitor was introduced in #5017. - Reproduces on the current `master` implementation. - The termination originates in Paperclip's Codex adapter inactivity monitor rather than the model provider or local configuration. **What happened?** Long-running `codex_local` tasks were terminated after seven minutes without stdout or stderr, even when the child process was still performing legitimate work such as a quiet test suite or waiting for remote checks. **Expected behavior** The default inactivity window should tolerate common long-running quiet tasks while continuing to terminate processes that remain silent for an extended period. **Steps to reproduce** 1. Start a `codex_local` run with the default `outputInactivityTimeoutMs` configuration. 2. Have the child process perform legitimate work without emitting stdout or stderr for more than seven minutes. 3. Observe the adapter terminate the process at the old default threshold. **Version / deployment** Current `master`, built from source in local/self-hosted deployments, using the Codex adapter. This behavior is not database- or access-context-specific. ## What Changed - Raise `DEFAULT_CODEX_OUTPUT_INACTIVITY_TIMEOUT_MS` from seven minutes to 30 minutes. - Update the adapter configuration documentation to state the new default. - Add a focused regression assertion that pins the default to 30 minutes. ## Verification - `pnpm vitest run packages/adapters/codex-local/src/server/output-inactivity-monitor.test.ts` — 15 tests passed. - `pnpm --filter @paperclipai/adapter-codex-local typecheck` — passed. ## Risks - Low risk: only the fallback default changes; explicit positive timeout values and `null` disablement retain their existing behavior. - A genuinely silent Codex process now remains alive up to 23 minutes longer before the same SIGTERM/SIGKILL cleanup path runs. - No schema, API, migration, UI, or workflow changes are included. > This is a focused bug fix and does not overlap with planned core feature work in `ROADMAP.md`. ## Model Used - Anthropic Claude Fable 5 via the `claude_local` adapter produced the initial investigation and implementation with repository/tool access. - OpenAI Codex via the Codex CLI prepared the PR, added the focused regression assertion, and ran verification; the runtime did not expose a more specific model ID or context-window value. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] I have used a public-friendly branch name without internal tracker identifiers (execution-workspace exception: this branch is runtime-managed and cannot be renamed) - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge The source branch is execution-workspace managed and cannot be renamed during this run; the PR title and body intentionally contain no internal tracker references. Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
df2404d443 |
fix(codex): count raw child output as activity (#9632)
## Thinking Path > - Paperclip is the open source control plane teams use to manage AI agents and their work > - The Codex local adapter supervises CLI child processes and terminates genuinely silent runs > - The existing inactivity timer only recognized parsed JSONL stdout events as activity > - Long verification commands can emit ordinary stdout or stderr while producing no JSONL, so healthy children could be killed > - This pull request makes the watchdog observe raw child-process output before filtering or parsing > - The benefit is that long typecheck, build, and test phases survive while truly silent children remain bounded ## Linked Issues or Issue Description - **Bug:** The Codex output-inactivity watchdog could terminate healthy runs during long verification phases because non-JSON stdout and stderr did not reset its timer. - **Expected:** Any bytes emitted by the child process count as output activity; only a child with no stdout or stderr for the configured interval is terminated. - **Reproduction:** Configure a short `outputInactivityTimeoutMs`, run a Codex child that periodically emits plain-text verification progress without JSONL events, and observe the old monitor firing despite continued output. - **Deployment mode:** Local `codex_local` adapter execution. - Related implementation: #5017 - Related recovery behavior: #8680 ## What Changed - Reset the Codex inactivity monitor on every non-empty stdout or stderr chunk before stderr noise filtering. - Track raw output chunk and byte counts in monitor diagnostics while retaining parsed JSONL event counts. - Add regression coverage for more than 21 simulated minutes of non-JSON verification output and retain silent-child termination coverage. - Document that `outputInactivityTimeoutMs` observes raw child output and that `null` still disables the monitor. ## Verification - `/srv/paperclip/home/paperclipai/paperclip/node_modules/.bin/vitest run packages/adapters/codex-local/src/server/output-inactivity-monitor.test.ts packages/adapters/codex-local/src/server/output-inactivity-monitor.integration.test.ts` - `/srv/paperclip/home/paperclipai/paperclip/node_modules/.bin/tsc -p packages/adapters/codex-local/tsconfig.json --noEmit` ## Risks - Low risk: the monitor becomes more conservative and may allow a noisy-but-stuck child to run longer, but the configured hard timeout and platform silent-run safety net remain unchanged. - No schema, API, migration, or UI changes. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex CLI coding agent; runtime model ID and context-window size were not exposed to the agent. Used repository/tool access, code execution, and focused test verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d32ed88443 |
fix(recovery): route recovery by failure cause (#9634)
## Thinking Path > - Paperclip is the open source control plane people use to coordinate AI-agent companies and their work. > - Its recovery subsystem detects stranded issue execution and decides whether to retry, escalate, or request operator intervention. > - The existing recovery path used a mostly generic owner ladder and generic execution contract, so transient failures could wake a manager who then performed the deliverable instead of repairing and returning the task. > - Provider quota failures also entered the same takeover path even when the correct action was to wait for capacity and retry the original assignee. > - Recovery actions already retain the source owner and evidence needed to choose a cause-specific route, render a scoped contract, and measure whether work was handed back. > - This pull request adds a cause-keyed recovery playbook, propagates its contract through every built-in adapter, and makes resolved recovery actions return work to the original owner by default. > - The benefit is bounded self-recovery that preserves task ownership, avoids needless management takeover, and makes recovery outcomes observable. ## Linked Issues or Issue Description No matching public GitHub issue was found. Related recovery work was reviewed but is not duplicated here: #9630 restores bounded recovery continuations, #8807 changes one assignee-ranking case, and #9404 records runtime-failure transition evidence. This change instead introduces cause-specific routing and recovery contracts across the recovery lifecycle. ### What happened? When an issue became stranded, recovery generally selected an owner through the same fallback ladder and rendered the normal execution contract. That made the recovery wake look like ordinary deliverable work, even when the correct action was to retry the original agent, repair its runtime, or wait for a provider quota reset. ### Expected behavior Recovery should select a response by failure cause, tell the recipient to recover rather than complete the deliverable, suppress takeover wakes for provider quota waits, and return repaired work to its original assignee unless the recovery owner explicitly completes it. ### Actual behavior Recovery could escalate transient failures to management, omit the cause-specific next action from the wake, and leave the recovery owner assigned after the runtime problem was resolved. ### Impact The generic path creates avoidable management work, ownership churn, and budget consumption while obscuring whether recovery successfully returned work to the responsible agent. ## What Changed - Added cause-keyed routing for process loss, missing disposition, provider quota limits, Codex output inactivity, workspace validation failures, and fallback recovery causes. - Added recovery-scoped wake rendering that replaces the generic execution contract with the failure summary, original assignee, attempt count, next action, and cause-specific playbook instruction. - Propagated the structured recovery contract through all built-in adapter execution paths, including Hermes local and gateway adapters. - Added provider-quota wait monitoring so capacity failures schedule the original assignee instead of enqueueing a takeover wake. - Added hand-back behavior and `handed_back` / `owner_completed` outcome accounting when recovery actions are resolved. - Added focused routing, renderer, quota-monitor, and hand-back regression coverage plus implementation-spec documentation. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/server-utils.test.ts server/src/__tests__/heartbeat-process-recovery.test.ts server/src/__tests__/heartbeat-workspace-branch-containment.test.ts server/src/__tests__/issue-recovery-actions.test.ts` - 4 test files passed; 194 tests passed. - Targeted `pnpm --filter ... typecheck` across `@paperclipai/adapter-utils`, `@paperclipai/shared`, `@paperclipai/server`, `@paperclipai/ui`, and all nine changed adapter packages. - 13 affected workspace packages passed typecheck. - `pnpm check:token-gates` - All UI token gates passed. ## Risks - Recovery routing behavior changes for stranded work, so an incorrectly classified cause could select a different recipient than before; fallback causes retain the existing management ladder. - Provider quota detection depends on structured failure evidence and conservative text matching; unmatched failures continue through fallback recovery. - Adapter prompt plumbing changes across built-ins, covered by shared renderer tests and compile-time call signatures. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex with exact model ID `gpt-5.6-sol`, using reasoning, tool use, and code execution. The runtime does not expose its configured context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
3a727bf780 | fix(codex): warn when sandbox auth is shadowed (#9259) | ||
|
Nicky LeachandPaperclip
|
7947308276 |
fix(codex): classify refresh auth failures (#9598)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip supports the Codex local adapter, which runs OpenAI Codex CLI sessions on behalf of agents > - Codex uses OAuth refresh tokens to maintain long-running authenticated sessions > - When a refresh fails, the failure has distinct root causes: a refresh token was already reused in a parallel request, the token expired by TTL, or the token was invalidated/revoked by the provider > - Without classifying these failure modes, all refresh auth errors surface identically — operators cannot distinguish retryable transient collisions from permanent invalidations, and run logs carry no actionable diagnosis > - This pull request adds structured classification (`refresh_token_reused`, `refresh_token_expired`, `refresh_token_invalidated`) of Codex refresh-token auth failures across the CLI quota-probe, ACP auth path, and execute path > - The benefit is that these distinct failure modes can be surfaced in run logs and acted on appropriately — transient reuse can be retried; true invalidations require re-auth ## Linked Issues or Issue Description <!-- Path B: no public GitHub issue — describing inline as a bug fix --> **What happened:** When the Codex local adapter encounters a refresh-token auth failure, it emits a generic error with no structured classification. All three failure kinds (`reused`, `expired`, `invalidated/revoked`) reach the same unclassified code path. **Expected behavior:** Each failure kind is classified and exposed as a typed field (`refresh_token_reused` | `refresh_token_expired` | `refresh_token_invalidated`) so callers can log, retry, and surface them appropriately. **Steps to reproduce:** 1. Run a Codex agent session with a reused or expired OAuth refresh token. 2. Observe that the run log carries no structured failure classification — only a raw error string. **Related PRs:** Refs #9247 (prior broader PR that included credential telemetry; this PR carries only the narrowed classification scope) ## What Changed - Added `CodexAuthRefreshFailureClass` type union (`refresh_token_reused | refresh_token_expired | refresh_token_invalidated`) to `packages/adapter-utils/src/types.ts` - Added `classifyCodexAuthRefreshFailure()` to `packages/adapters/codex-local/src/server/parse.ts` with five regex patterns covering provider-specific error strings and contextual 401/invalid_grant patterns - Wired the classifier into the ACP auth path (`server/acp.ts`), execute path (`server/execute.ts`), and CLI quota-probe (`cli/quota-probe.ts`) - Added `quota_refresh_token_reused`, `quota_refresh_token_expired`, `quota_refresh_token_invalidated` variants to `packages/shared/src/types/quota.ts` - Added classification unit tests (`parse.test.ts`, `quota-spawn-error.test.ts`, `acp.test.ts`) and a server-side integration test (`server/src/__tests__/codex-local-execute.test.ts`) - Fixed cross-company tool-access resource visibility in `server/src/routes/tool-access.ts` - Stabilized `heartbeat-retry-scheduling.test.ts` (CASCADE cleanup), `heartbeat-run-log.test.ts`, and `quota-windows.test.ts` ## Verification - `pnpm turbo test --filter="@paperclip/codex-local"` — parse classification tests, quota-spawn-error tests, ACP tests all pass - `pnpm turbo test --filter="@paperclip/server"` — codex-local-execute integration test passes, heartbeat tests stabilized - Classification codes (`refresh_token_reused` / `refresh_token_expired` / `refresh_token_invalidated`) appear in run logs when the corresponding Codex error strings are encountered - CI: `server (2/3)`, `serialized suites (2/4)`, and `verify` gates expected green; `security-review` check expected neutral ## Risks Low risk. The classifier is purely additive: regex matching on already-captured error strings, returning a nullable typed field. Callers that do not inspect the classification field are unaffected. No execution paths, retry logic, or existing error surfaces changed. ## Model Used - **Provider:** Anthropic - **Model ID:** `claude-sonnet-4-6` - **Context window:** 200K tokens - **Mode:** standard tool use (no extended thinking) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
b79f744a8d |
Fix Codex auth merge host-unusable fail closed (#9276)
## Thinking Path > - Paperclip is the open source platform people use to manage AI agents for work > - The Codex adapter runs agent tasks in isolated sandbox environments on the user's machine > - When a Codex sandbox is reused across agent runs, its home directory (including `~/.codex/auth.json`) is restored from a prior snapshot > - Both the host machine and the sandbox independently maintain `auth.json` credentials; on sandbox reuse, these can diverge > - The previous merge code had fail-open edge cases: if host auth was in an unusable state, if the auth JSON object shapes differed between host and sandbox, or if the subscription account identities didn't match, the merge would proceed silently with whatever data was available > - This PR adds fail-closed behavior: if host Codex auth is unusable, if auth parser shapes differ, or if subscription account identities don't match, the merge fails explicitly rather than silently continuing with stale or incorrect credentials > - The benefit is that Codex agents on reused sandboxes now fail fast and loudly when auth is in a broken state, instead of silently running with wrong credentials and producing confusing downstream failures ## Linked Issues or Issue Description No pre-existing public GitHub issue. This is a targeted security hardening fix for the Codex reused-sandbox auth merge path. **Problem:** When a Codex sandbox is reused, the merge logic that reconciles host and sandbox `auth.json` credentials failed open in several cases: - Host `auth.json` present but in an unusable state (missing required keys, empty token material, malformed JSON) → merge would proceed with whatever the sandbox had - Host and sandbox auth payloads had different shapes (e.g., one uses `OPENAI_API_KEY`, the other uses a `tokens` object) → parser-differential case not detected - Subscription account identities (`tokens.account_id`) differed between host and sandbox → stale sandbox identity would be used silently **Fix:** All three cases now fail closed. The merge returns an explicit error rather than proceeding with potentially stale or mismatched credentials. Related PRs: - Refs #9262 — sandbox Codex auth shadow warning (adjacent auth area) - Refs #9259 — auth precedence exports (adjacent auth area) ## What Changed - `packages/adapters/codex-local/src/server/codex-home.ts` — New file with `hasUsableAuthPayload()`, `codexHomeHasUsableAuth()`, and full Codex home setup/teardown. Includes fail-closed auth merge guards: rejects unusable host auth, detects parser shape differentials, and checks subscription account identity match before merging - `packages/adapter-utils/src/workspace-restore-merge.ts` — New file with directory snapshot diffing and restore-merge logic; the merge operation fails closed when auth validation fails - `packages/adapters/codex-local/src/server/codex-home.test.ts` — Unit tests covering auth usability checks, symlink management, and fail-closed merge paths - `packages/adapter-utils/src/workspace-restore-merge.test.ts` — Unit tests for snapshot/restore-merge behavior including fail-closed cases - `packages/adapter-utils/src/sandbox-managed-runtime.ts` — Updated to invoke the fail-closed auth merge during sandbox restore ## Verification Tests run and passing: ```sh corepack pnpm exec vitest run packages/adapter-utils/src/workspace-restore-merge.test.ts packages/adapters/codex-local/src/server/codex-home.test.ts corepack pnpm exec vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts corepack pnpm --filter @paperclipai/adapter-utils typecheck corepack pnpm --filter @paperclipai/adapter-codex-local typecheck git diff --check origin/master HEAD ``` All passed locally before push. ## Risks - **Intentional behavioral change (breaking for previously-silent failures):** Reused sandboxes that previously completed auth merge with unusable host auth, parser-differential auth shapes, or mismatched account identities will now fail with an explicit error. This is the correct behavior — the prior silent-proceed path was the bug. Users affected will see a clear error message rather than a confusing downstream auth failure. - **Auth.json symlink migration:** `ensureSymlink()` detects stale copied `auth.json` files (written by older Paperclip versions) and replaces them with symlinks on first run. This is safe: the target is always under the Paperclip-managed company home, never the user's real `~/.codex`. Directories at the symlink path are left untouched (EISDIR is not silently swallowed). - **Low risk for non-reuse paths:** The fail-closed logic only activates during sandbox restore/reuse. Fresh sandbox allocations are unaffected. ## Model Used - **Provider:** Anthropic - **Model ID:** claude-sonnet-4-6 (Claude Sonnet 4.6) - **Context window:** 200k tokens - **Mode:** Agentic coding with tool use; extended thinking not used - **Role:** Code author (Priya Raman, BackendEngineer) with Harold Kim (Git Expert) handling push and PR operations ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Priya Raman <priya.raman@paperclip.local> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Harold Kim <harold@paperclip.ing> |
||
|
DottaandPaperclip
|
931eec3fbf |
feat(mcp) [split 4/8]: wire gateway runtime and Smoke Lab (#9559)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Governed MCP access spans contracts, runtime enforcement, adapters, UI surfaces, and operator verification > - The parity reference PR #9534 is too large for effective automated or human review > - The feature therefore needs a linear stack whose individual diffs stay below the 100-file review limit > - This pull request is split 4/8 and focuses on gateway runtime, Smoke Lab, plugins, and server wiring > - The benefit is a standalone, testable review boundary while preserving byte-for-byte parity at the top of the stack ## Linked Issues or Issue Description - Related parity reference: #9534 - Problem: The policy core needs runtime execution, endpoint guards, route registration, heartbeat integration, and adapter MCP injection to become operational. - Proposed solution: Adds the remaining server routes/wiring/consumers, runtime tests, adapter-utils MCP contracts, and Claude/Codex injection implementations required by the server layer. - Alternatives considered: keeping #9534 as one 403-file review, or rewriting the feature to manufacture seams; both were rejected in favor of path extraction plus compile-driven boundary moves. - Roadmap alignment: this advances the existing governed MCP/tool-access work already represented by #9534; it does not introduce a separate roadmap initiative. - Stack position: base branch is `pap10341-split/03-server-tool-access`. - Merge policy: merge bottom-up, in order, only after the complete eight-PR stack has been reviewed and the top-of-stack parity gate remains empty. - Requested review: SecurityEngineer for gateway, endpoint guard, token issuance, and runtime wiring; Greptile on every PR. ## What Changed - Adds the remaining server routes/wiring/consumers, runtime tests, adapter-utils MCP contracts, and Claude/Codex injection implementations required by the server layer. - Keeps this PR below 100 changed files and independently typecheckable. - Preserves the final tree from #9534 when combined with the other seven stack levels. ## Verification - `pnpm typecheck` - Changed server test set — 26 files, 382 tests passed - Affected server adapter tests — 38 tests passed after concrete adapter boundary move - Adapter-utils and Codex focused tests — 76 tests passed ## Risks - Remote endpoint validation, token handling, and runtime supervision are security-sensitive and can fail closed or deny legitimate access if misconfigured. - Stack risk: merging out of order can expose incomplete layers; mitigate by following the documented bottom-up merge policy. - Parity risk: later edits to an intermediate branch can drift from #9534; mitigate by re-running the empty top-of-stack diff before merge. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, exact model ID `gpt-5.4`; runtime-managed context window; medium reasoning with repository, shell, Git, GitHub CLI, and code-execution tools enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] Internal references are omitted except the execution-plan link explicitly required for this coordinated split stack - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge ## Stack Coordination - Internal execution plan: [PAP-13874](/PAP/issues/PAP-13874#document-plan) - Parity reference: #9534 - Stack: #9556 → #9557 → #9558 → #9559 → #9560 → #9561 → #9562 → #9563 - Merge bottom-up only after full-stack review and an empty parity diff at #9563. --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
98d9360658 |
feat(adapters): confine local coding processes (#9504)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI agents for work. > - Local coding adapters currently spawn their CLI processes directly on the Paperclip host. > - CLI-native approval and sandbox flags do not provide a reliable host filesystem or network boundary. > - An agent can therefore inspect unrelated host files or fetch external material when an operator needs stronger isolation. > - The confinement must stay opt-in so existing local adapter behavior does not change unexpectedly. > - This pull request adds a shared Linux Bubblewrap spawn layer for workspace filesystem and deny/allowlist network scopes. > - The benefit is enforceable defense in depth around Codex and Claude local runs while preserving explicit provider connectivity. ## Linked Issues or Issue Description ### What happened? `codex_local` and `claude_local` processes could read arbitrary host paths and make unrestricted outbound network requests because Paperclip did not impose a spawn-level boundary. ### Expected behavior Operators can opt into a workspace-only filesystem view and either deny network egress or allow exact provider/API hosts, independently of CLI approval flags. ### Steps to reproduce 1. Run current `master` on Linux and configure a Codex or Claude local adapter. 2. Ask the agent to read a canary file outside its active workspace. 3. Ask the agent to `curl` a public host. 4. Observe that both operations succeed without a Paperclip-level confinement option. ### Environment - Paperclip commit: `c36f1a4af` / current `master` base. - Deployment mode: Linux local dev or self-hosted server. - Installation: built from source. - Adapters: Codex and Claude Code. - Database: not related. ## What Changed - Added a shared Bubblewrap process wrapper with opt-in `filesystemScope: "workspace"`, managed/extra path mounts, private `/tmp`, and Linux-only validation. - Added `networkScope: "deny" | "allowlist"`; both use a private network namespace, while allowlist mode exposes an exact-host HTTP(S) proxy over a Unix-socket bridge. - Wired Codex and Claude local CLI execution through the wrapper and forced scoped auto runs onto the CLI lane because ACP processes are not covered. - Added unit and gated Bubblewrap canaries for outside-file denial, workspace writes, direct network denial, allowlisted forwarding, and rejected destinations. - Documented both scopes, provider allowlist examples, Bubblewrap requirements, and default-off behavior. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/local-process-sandbox.test.ts packages/adapters/codex-local/src/server/acp.test.ts packages/adapters/claude-local/src/server/acp.test.ts` — 34 passed, 4 gated Bubblewrap tests skipped by default. - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/adapter-codex-local typecheck` - `pnpm --filter @paperclipai/adapter-claude-local typecheck` - `pnpm --filter @paperclipai/adapter-utils build` - `pnpm --filter @paperclipai/adapter-codex-local build` - `pnpm --filter @paperclipai/adapter-claude-local build` - Attempted the gated tests with a vendored Bubblewrap binary; this container blocks unprivileged namespace setup (`setting up uid map: Permission denied` / loopback `RTM_NEWADDR: Operation not permitted`), so kernel-level execution remains for CI or a namespace-enabled Linux host. ## Risks - Bubblewrap must be installed and unprivileged user/mount/network namespaces must be enabled on the host; scoped runs fail clearly if the prerequisite is missing. - Allowlist mode depends on the coding CLI honoring standard `HTTP_PROXY` / `HTTPS_PROXY` variables; custom providers must list every required exact hostname and port. - Exact-host allowlists intentionally reject wildcards, which is safer but may require operators to enumerate multi-host provider setups. - No behavior changes unless an operator enables `filesystemScope` or `networkScope`. > This aligns with the ROADMAP direction toward safer remote and sandboxed agent environments and does not duplicate an open PR or issue found in the repository search. ## Model Used - OpenAI GPT-5.5 (`gpt-5.5`) via Codex CLI, with reasoning, repository tool use, shell execution, code editing, and test execution. The serving context-window size is not exposed to the agent. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
efcce9cc8e |
fix(adapters): record unpriced CLI usage (#9505)
## Thinking Path > - Paperclip is the open source control plane people use to manage AI-agent companies > - Budgets and spend telemetry are control-plane safety features, not just reporting > - Local Codex and Claude adapters can execute through either ACP or their native CLI engines > - The ACP lane records usage and reported cost, but CLI JSON output often reports tokens without a price > - The CLI lane was either losing per-run usage semantics or coercing missing cost to zero, making real usage indistinguishable from a genuinely free run > - This pull request preserves CLI usage as per-run totals and records token-bearing runs without a reported price as explicitly unpriced ledger events > - The benefit is accurate usage accounting and a visible pricing gap instead of silently misleading zero-cost telemetry ## Linked Issues or Issue Description Refs #9471 Refs #9230 **Bug description** A `codex_local` run using the CLI engine can emit a final `turn.completed` event with millions of input tokens and tens of thousands of output tokens while the agent's spend ledger remains indistinguishable from a true zero-usage, zero-cost run. Claude CLI output has the same missing-price edge case. **Expected behavior** Token-bearing CLI runs should persist their usage. If the adapter reports a price, the ledger should record it as reported; if the CLI reports usage but no price, the ledger should explicitly mark the event as unpriced rather than silently treating missing price data as a reported `$0` cost. **Reproduction shape** 1. Configure `codex_local` with `engine: cli`. 2. Run a task that produces a `turn.completed` usage payload. 3. Observe token usage in the run stream. 4. Before this change, missing price data is represented as ordinary zero-cost spend and the CLI usage basis is not consistently propagated. ## What Changed - Mark Codex and Claude native CLI usage totals as `per_run` and propagate that basis through success and failure results. - Stop coercing missing Claude CLI cost to `0`. - Add `cost_status` to cost events with `reported` and `unpriced` values, including an idempotent migration and shared validation/types. - Persist token-bearing runs without a reported price as `unpriced` ledger events while retaining zero cents until an authoritative price exists. - Add parser, execute-path, heartbeat-accounting, and cost-service regression coverage for both local CLI adapters. - Document the cost-status invariant and CLI accounting behavior. ## Verification - `pnpm exec vitest run packages/adapters/codex-local/src/server/parse.test.ts packages/adapters/claude-local/src/server/parse.test.ts server/src/__tests__/codex-local-execute.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/heartbeat-cost-accounting.test.ts server/src/__tests__/costs-service.test.ts` — 6 files / 102 tests passed. - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/db typecheck` — includes migration numbering and safety checks. - `pnpm --filter @paperclipai/adapter-codex-local typecheck` - `pnpm --filter @paperclipai/adapter-claude-local typecheck` - `pnpm --filter @paperclipai/server typecheck` ## Risks - Existing cost rows default to `reported`, preserving current interpretation; only new token-bearing events with absent cost are marked `unpriced`. - This change does not invent model pricing. Budget hard stops still cannot charge an unknown amount, but operators and evals can now distinguish missing pricing from a genuinely reported zero cost. - Consumers that enumerate cost-event fields should tolerate the additive `costStatus` field. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, exact model `gpt-5.3-codex`, with repository tool use and code execution; default reasoning mode. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
9e7e84e3fe |
fix(adapters): propagate ACP-lane usage and cost into spend telemetry (#9471)
## Thinking Path > - Paperclip is the open source control plane for running and governing AI-agent companies. > - Adapter executions feed token usage, billing identity, and run cost into the control plane's spend telemetry. > - The default ACP execution lane for local Claude and Codex adapters did not propagate per-turn usage or cost, so paid runs could be recorded with zero spend and no tokens. > - Claude CLI result events could also undercount output tokens by reading only the main-loop usage block instead of the complete per-model ledger. > - The shared executor needs to distinguish per-run usage from session-cumulative usage so the server does not apply the wrong delta heuristic. > - This pull request captures ACP usage and cumulative-cost deltas, resolves adapter billing identity, uses Claude's complete model-usage ledger, and preserves per-run usage in server normalization. > - The benefit is accurate token and cost accounting across the default paid Claude and Codex execution paths. ## Linked Issues or Issue Description ### What happened? Paid `claude_local` and `codex_local` runs using the default ACP engine can complete successfully while the control plane records zero or null cost and missing token usage. Claude CLI result parsing can additionally undercount output tokens when subagent or sidechain usage is present. ### Steps to reproduce 1. Run a paid Claude or Codex local adapter through the ACP engine. 2. Complete a turn that reports usage and cumulative cost through ACP status/events. 3. Inspect the execution result and normalized run telemetry. ### Expected behavior The execution result contains per-turn token usage, a per-run USD cost delta, and the correct billing identity. Server normalization records those per-run values without applying a session-cumulative delta a second time. ### Actual behavior before this change ACP execution results returned no usage and `costUsd: null` with unknown billing. The server therefore recorded zero spend and no tokens for paid runs. Claude CLI parsing could use an incomplete usage block. ## What Changed - Capture ACP usage from runtime status and `usage_update` events, reporting it as `usageBasis: per_run`. - Convert agent-reported cumulative ACP cost into a per-turn delta, including counter-reset and no-report safeguards. - Add a shared billing-identity resolver and map Claude and Codex authentication/provider modes to control-plane billing types. - Prefer Claude result-event `modelUsage` totals so subagent and sidechain tokens are included. - Skip the server's session-cumulative usage delta when an adapter explicitly reports per-run usage. - Add regression coverage for usage capture, event fallback, cost resets, stale reports, billing identities, model-usage totals, and server spend normalization. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts packages/adapters/claude-local/src/server/parse.test.ts packages/adapters/claude-local/src/server/acp.test.ts packages/adapters/codex-local/src/server/acp.test.ts server/src/__tests__/costs-service.test.ts server/src/__tests__/monthly-spend-service.test.ts` — 6 files, 126 tests passed. - `pnpm --filter @paperclipai/adapter-utils typecheck` — passed. - `pnpm --filter @paperclipai/adapter-claude-local typecheck` — passed. - `pnpm --filter @paperclipai/adapter-codex-local typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - A broader Claude-local suite has a pre-existing rate-limit classification failure in `test.probe.test.ts`; it also fails on clean `master` and is unrelated to this change. ## Risks - Cost reporting depends on the agent's cumulative counter semantics; reset handling falls back to the post-turn amount and is covered by regression tests. - Incorrect billing-mode inference could misclassify spend; provider/auth mappings mirror each adapter's existing CLI behavior and have focused tests. - The new `usageBasis` contract changes server normalization only when adapters explicitly opt into `per_run`; existing adapters retain prior behavior. - No database migration, workflow, lockfile, or UI changes are included. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Implementation commit: Anthropic Claude Fable 5, tool-enabled coding workflow (exact context window and runtime configuration were not recorded in the commit metadata). - PR preparation and verification: OpenAI Codex, tool-enabled coding agent (runtime model ID and context window are not exposed to this session). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
9cde4e128c |
feat: run ACP sessions in sandbox execution targets (#9390)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent adapters (Claude, Codex, Gemini) default to the ACP engine lane, which needs a live bidirectional stdio session with the agent process > - Sandbox execution targets only exposed one-shot command execution, so every ACP-capable adapter refused remote targets and fell back to the CLI lane with a "supports only the local Paperclip host" warning > - Running agents in sandboxes is a core deployment mode, and losing ACP there means losing streaming updates, structured events, and default-lane parity with local runs > - This pull request adds a provider-agnostic process-session bridge that relays the ACP stdio session into the sandbox over the existing sandbox runner contract, and updates the adapters to use it > - The benefit is that the default ACP lane now behaves the same on the local host and in any sandbox provider, with CLI fallback reserved for targets that genuinely cannot host a bidirectional session ## Linked Issues or Issue Description No existing public issue covers this; inline description following the feature request template: **Problem or motivation** Configuring an ACP-capable adapter (e.g. Claude) with a sandbox environment made every run fall back to the CLI lane with the warning "Claude ACP currently supports only the local Paperclip host, but this run targets a remote environment." The ACP engine only knew how to spawn a local subprocess, while sandbox providers only expose one-shot command execution — so there was no way to hold the bidirectional stdio session ACP requires. **Proposed solution** Add a process-session bridge in `adapter-utils`: a local ACPX-spawnable proxy script connects to a token-authenticated loopback TCP server, which relays JSON-framed stdin/stdout/stderr events to and from a small relay script executed inside the sandbox via the provider's ordinary runner. Claude/Codex/Gemini adapters now treat sandbox targets with a runner as ACP-capable, resolve agent commands against the remote target, and fall back to CLI only when the sandbox exposes no bidirectional path. The sandbox callback bridge injects a run-scoped API endpoint and bridge token so the agent inside the sandbox can reach Paperclip (including work-product handoffs) without ever receiving the host run JWT. **Alternatives considered** A provider-specific lane was prototyped first: Daytona minting SSH access metadata at lease time, converted into an SSH execution target. It was dropped because it only worked for providers able to advertise SSH, added per-provider surface area, and left every other sandbox provider on the CLI fallback. The merged design rides the one-shot runner contract all providers already implement; a regression test pins that sandbox targets stay on the bridge lane even when lease metadata advertises SSH access. **Roadmap alignment** Directly advances the "Cloud / Sandbox agents" roadmap item — agents running in remote and sandboxed environments keep the same control-plane behavior as local ones. No overlap with other planned core work. ## What Changed - `packages/adapter-utils/src/execution-target.ts`: new `startAdapterExecutionTargetProcessSessionBridge()` plus helpers — writes a token-authenticated local proxy script (spawnable by ACPX) and a remote relay script synced into the sandbox, with a loopback TCP server streaming JSON-framed stdio between them; events emitted before the ACP client attaches are buffered so none are lost. - `packages/adapter-utils/src/acpx-engine/execute.ts`: the ACP engine can execute against remote sandbox targets through the bridge instead of requiring a local subprocess, including remote cwd/env shaping. - `packages/adapter-utils/src/sandbox-callback-bridge.ts`: sandbox-scoped API bridging extended to allow work-product handoffs; the sandbox payload env carries a bridge token, never the host run JWT. - `packages/adapters/claude-local`, `codex-local`, `gemini-local` (`src/server/acp.ts`): default-lane selection no longer rejects all remote targets; command resolution is remote-aware (`ensureAdapterExecutionTargetCommandResolvable`, `resolveAdapterExecutionTargetCwd`); the fallback reason is now scoped to sandboxes that expose only one-shot execution. - `server/src/__tests__/environment-execution-target.test.ts`: pins that sandbox targets resolve to the bridge lane, including when lease metadata advertises SSH access. - Non-sandbox remote targets (e.g. SSH) keep the CLI lane: the ACP engine's remote transport is sandbox-only, so default-lane selection falls back for those targets across all three adapters, and tests covering CLI-specific remote behavior pin `engine: "cli"` explicitly. - The bridge authenticates loopback connections before they can own the session or receive buffered output (token required, idle unauthenticated peers dropped), and remote event writes are serialized so the exit event always lands after stdout/stderr have drained. - Daytona plugin: formatting-only residue from the earlier iteration; no functional change. ## Verification - `vitest run` over the touched suites — `packages/adapter-utils/src/acpx-engine/execute.test.ts`, `packages/adapter-utils/src/execution-target-sandbox.test.ts`, `packages/adapter-utils/src/sandbox-callback-bridge.test.ts`, the three adapter `acp.test.ts` files, and `server/src/__tests__/environment-execution-target.test.ts` — 102 tests pass. - End to end: with a Claude agent configured on a Daytona sandbox environment, the primary-model test now selects the default ACP lane (no fallback warning), and the full round trip (wake → sandbox execution → API bridge → comment post) was exercised twice from inside a live sandbox. ## Risks - Behavioral shift: adapters that previously always fell back to CLI on sandbox targets now default to ACP there; `engine=cli` still pins the CLI lane explicitly. - The bridge relays stdio as JSON lines over loopback TCP guarded by a per-session random token; the remote relay runs inside the sandbox under the provider's runner. Providers with slow one-shot execution will see higher session startup latency — the CLI fallback remains for genuinely incapable targets. - No schema or migration changes. ## Model Used - Claude Fable 5 (`claude-fable-5`, Anthropic) — extended thinking enabled, agentic tool use via the Claude Agent SDK harness; implementation iterated with local Vitest verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no shipped docs describe the old local-only ACP limitation) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Cody <noreply@paperclip.ing> Co-authored-by: Cody <cody@paperclip.local> |
||
|
DottaandPaperclip
|
a02fe8d575 |
Update Codex adapter GPT-5.6 defaults (#9352)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Codex local is the adapter subsystem that exposes OpenAI Codex CLI model choices to agents and issue overrides. > - OpenAI has GPT-5.6 Codex-capable models that should appear in Paperclip's built-in Codex model list and refresh behavior. > - Paperclip's server model listing falls back to the adapter metadata and merges OpenAI refresh results with known Codex defaults. > - This pull request updates the Codex default model metadata to include GPT-5.6 options and adds regression coverage for fallback and refresh paths. > - The benefit is that operators can select the new Codex models without relying on manual model IDs, and refresh behavior keeps known GPT-5.6 options visible. ## Linked Issues or Issue Description Refs #9322. Refs #9342. Refs #9346. ### Agent or provider Codex CLI (OpenAI). ### Why this adapter is useful OpenAI's GPT-5.6 Codex-capable models should be available in Paperclip's Codex adapter defaults and model refresh path. ### How the agent is invoked `codex` ## What Changed - Changed the `codex_local` default model metadata from `gpt-5.5` to `gpt-5.6`. - Added `gpt-5.6-sol`, `gpt-5.6-terra`, and `gpt-5.6-luna` to the built-in Codex adapter model list. - Updated adapter and server model-listing tests to cover GPT-5.6 fallback and refresh behavior. - Aligned Codex Fast mode support and helper text with the new `gpt-5.6` default, while preserving GPT-5.5, GPT-5.4, and manual model ID support. ## Verification - `git diff --check origin/master...HEAD` - `pnpm exec vitest run packages/adapters/codex-local/src/index.test.ts packages/adapters/codex-local/src/server/codex-args.test.ts server/src/__tests__/adapter-models.test.ts server/src/__tests__/adapter-model-refresh-routes.test.ts` - `pnpm check:token-gates` - `pnpm --filter @paperclipai/adapter-codex-local typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck` ## Risks Medium risk because changing `DEFAULT_CODEX_LOCAL_MODEL` from `gpt-5.5` to `gpt-5.6` changes the adapter's default model selection for new blank configurations. The model-list additions are otherwise low risk and covered by adapter/server metadata tests. This PR intentionally overlaps related PRs #9342 and #9346, so reviewers may prefer to close or fold it into one of those branches. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex coding agent based on GPT-5, with shell, git, GitHub CLI, and repository editing tool use. Exact served model ID and context window were not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
3571b6c38b |
feat(models): add gpt-5.4-mini to Codex and OpenCode selection (and openai/gpt-5.5 to OpenCode) (#4357)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agent runtimes are selected through adapters, and each adapter exposes the model IDs an operator can pick in the UI > - The `codex_local` and `opencode_local` adapters hardcode those lists, so a newly released model stays unreachable until it is added > - `gpt-5.4-mini` is available via both the Codex CLI and the OpenCode CLI, but neither adapter lists it; `openai/gpt-5.5` is likewise missing from `opencode_local` > - This pull request adds those entries, and deliberately keeps `gpt-5.4-mini` out of the Codex Fast mode allowlist because the model does not support Fast mode > - The benefit is that operators can select these models from the UI instead of falling back to a manual model ID, and Fast mode fails closed rather than sending unsupported overrides to the CLI ## Linked Issues or Issue Description No existing issue covers this. Describing it inline, following the feature request template: ### Problem or motivation `gpt-5.4-mini` is absent from the `models` list of both the `codex_local` and `opencode_local` adapters, and `openai/gpt-5.5` is absent from `opencode_local`. (`codex_local` already ships `gpt-5.5` — it is the default model on master.) Operators who want these models must type a manual model ID. For `codex_local` that has a real side effect. `isCodexLocalFastModeSupported` treats any *unknown* model as Fast-mode-capable and passes `service_tier="fast"` and `features.fast_mode=true` through to the CLI. Because `gpt-5.4-mini` does not support Codex Fast mode, an agent configured with a manual `gpt-5.4-mini` model ID and `fastMode` enabled silently sends overrides the CLI cannot honor. ### Proposed solution Add the three missing entries to the two `models` lists, and leave `CODEX_LOCAL_FAST_MODE_SUPPORTED_MODELS` untouched. Listing `gpt-5.4-mini` in `models` is precisely what makes it a *known* model, so `isCodexLocalFastModeSupported` returns `false`, `buildCodexExecArgs` omits the Fast mode overrides, and `fastModeIgnoredReason` is surfaced to the operator. ### Alternatives considered Adding `gpt-5.4-mini` to `CODEX_LOCAL_FAST_MODE_SUPPORTED_MODELS` as well — rejected, because the model does not support Fast mode and the overrides would be rejected at run time. Leaving the models unlisted so operators keep using manual IDs — rejected, because that is the path that silently enables Fast mode for a model that cannot use it. ### Roadmap alignment Not core roadmap work. `ROADMAP.md` does not plan adapter model-list maintenance; this is routine upkeep as upstream CLIs ship new models. ## What Changed - Add `gpt-5.4-mini` to the `codex_local` adapter's `models` list, positioned after `gpt-5.4` (newest-first ordering). - Add `openai/gpt-5.5` and `openai/gpt-5.4-mini` to the `opencode_local` adapter's `models` list. - Add a `buildCodexExecArgs` test asserting Fast mode is ignored for `gpt-5.4-mini`. `CODEX_LOCAL_FAST_MODE_SUPPORTED_MODELS` is intentionally unchanged. No behavior changes to existing models or adapter logic. ## Verification ``` pnpm --filter @paperclipai/adapter-codex-local --filter @paperclipai/adapter-opencode-local typecheck npx vitest run packages/adapters/codex-local packages/adapters/opencode-local ``` Both pass: typecheck clean on both packages, and 22 test files / 133 tests green, including the new `ignores fast mode for gpt-5.4-mini` case. ## Risks Low risk. The change is additive: three entries appended to two model-selection lists, plus one test. No default model changes, no adapter logic changes, no migrations. One behavioral shift is intended. An operator who had `gpt-5.4-mini` configured as a *manual* model ID with `fastMode` enabled was getting Fast mode overrides passed through to the Codex CLI. After this change `gpt-5.4-mini` is a known model, so those overrides are dropped and `fastModeIgnoredReason` explains why. ## Model Used - OpenAI Codex CLI with GPT-5 / GPT-5.5-assisted code editing (the original commits on this branch). - Anthropic Claude Opus 4.8 (`claude-opus-4-8`, 1M context, extended thinking, tool use) for the master merge, conflict resolution, and the scope reduction in the latest commit. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched the GitHub PR list for similar or duplicate PRs and confirmed this one is not a duplicate - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (N/A; the adapter docs describe Fast mode support, which is unchanged) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Andrew Aymeloglu <aaymeloglu@gmail.com> |
||
|
Devin FoleyandPaperclip
|
eedc7ddef2 |
Make ACP the default engine for local adapters (#9238)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Adapter packages are the bridge between the control plane and local agent harnesses such as Claude Code, Codex, and Gemini CLI. > - ACP support was concentrated in a separate `acpx_local` adapter, which made ACP feel like a separate agent choice instead of an execution capability of the harness adapters. > - Claude, Codex, and Gemini now have ACP-capable harnesses, so the native adapter should own ACP selection, fallback, config, transcript parsing, and environment diagnostics. > - The standalone ACPX adapter still needs a compatibility path for existing rows, but it should not be offered as an active adapter for new agents. > - This pull request moves the shared ACP runtime into `@paperclipai/acpx-engine`, wires Claude/Codex/Gemini local adapters to prefer ACP when prerequisites are available, and retires `acpx_local` to a tombstone. > - The benefit is one adapter per harness, richer ACP transcripts by default where possible, and a migration path for existing Claude/Codex ACPX agents. ## Linked Issues or Issue Description Closes #5932 — the broken default `acpx_local` Claude path is replaced by native `claude_local` ACP support, existing Claude/Codex ACPX rows migrate to native adapters, and new agents no longer choose the standalone ACPX adapter. Refs #4893 — original merged ACPX local adapter runtime that this PR replaces with native per-harness ACP engines. Refs #6590 — prior ACPX-Claude seamlessness work folded into the new native Claude ACP path. Refs #197 — related open generic ACP/Kiro adapter work; this PR does not close it because Kiro/custom generic ACP remains a separate adapter decision. Refs #7018 — related Kimi-specific `acpx_local` shell failure; this PR retires the built-in standalone adapter but does not add a native Kimi adapter. Refs #8864 — related ACPX prompt/API guidance PR; this PR moves runtime guidance into the shared/native ACP engine path instead of the old standalone adapter. Refs #8881 — related `acpx_local` POSIX shell failure from the old `acpx` pin; this PR updates ACP dependencies but does not claim custom/OMP ACP support as a first-class native adapter. Refs #8964 — related open `acpx_local` stderr cleanup PR; this PR makes the old runtime path obsolete for new agents but keeps it as a non-closing reference. Problem description: - The standalone `acpx_local` adapter duplicates Claude/Codex agent choices that already have first-class local adapters. - ACP should be an execution engine capability of each harness adapter when the underlying harness supports ACP. - Existing `acpx_local` agents should either migrate to native harness adapters or fail with an explicit retirement message instead of silently falling back to the process adapter. ## What Changed - Added `@paperclipai/acpx-engine` as the shared ACP execution, session-codec, CLI formatter, and UI parser package. - Wired `claude_local`, `codex_local`, and `gemini_local` to auto-select ACP by default when prerequisites pass, with `engine=cli` opt-out and `engine=acp` strict mode. - Added ACP config schema/UI fields, environment checks, session-codec preservation, transcript parsing, and adapter capability metadata for the native adapters. - Retired `acpx_local` to a server tombstone, removed its UI/package/runtime image surface, and added a migration for existing Claude/Codex ACPX agents. - Updated package manifests, lockfile, release tooling, docs, Kubernetes sandbox defaults, and tests. ## Verification - `corepack pnpm --filter @paperclipai/acpx-engine typecheck` - `corepack pnpm --filter @paperclipai/adapter-claude-local typecheck` - `corepack pnpm --filter @paperclipai/adapter-codex-local typecheck` - `corepack pnpm --filter @paperclipai/adapter-gemini-local typecheck` - `corepack pnpm --filter @paperclipai/acpx-engine exec vitest run` - `corepack pnpm --filter @paperclipai/adapter-claude-local exec vitest run src/server/acp.test.ts src/server/execute.acp-fallback.test.ts src/ui/build-config.test.ts` - `corepack pnpm --filter @paperclipai/adapter-codex-local exec vitest run src/server/acp.test.ts src/ui/build-config.test.ts` - `corepack pnpm --filter @paperclipai/adapter-gemini-local exec vitest run src/server/acp.test.ts src/ui/build-config.test.ts src/ui/parse-stdout.test.ts` - `corepack pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && corepack pnpm --filter @paperclipai/server exec tsc --noEmit` - `corepack pnpm --filter @paperclipai/server exec vitest run src/__tests__/adapter-routes.test.ts src/__tests__/adapter-session-codecs.test.ts src/__tests__/adapter-models.test.ts` - `corepack pnpm --filter @paperclipai/ui typecheck` - `corepack pnpm --filter @paperclipai/ui exec vitest run src/adapters/metadata.test.ts src/adapters/adapter-display-registry.test.ts src/components/AgentConfigForm.test.ts src/components/AgentConfigForm.render.test.tsx src/components/transcript/RunTranscriptView.test.tsx` - `node --test scripts/bootstrap-npm-package.test.mjs scripts/release-package-map.test.mjs scripts/verify-release-registry-state.test.mjs` Note: the server typecheck script calls `pnpm` internally; this dev shell exposes pnpm through Corepack only, so I ran the two script steps manually with `corepack pnpm`. ## Risks - Migration changes existing `acpx_local` Claude/Codex agents to native adapter types and clears old ACPX task sessions/runtime state. - Custom ACP commands remain on the retired tombstone and will need a separate future adapter/plugin path. - ACP auto-selection depends on local Node and ACP server command prerequisites; remote and unsupported environments fall back to CLI unless `engine=acp` is explicit. - `@paperclipai/acpx-engine` is a new public package and needs npm trusted-publishing bootstrap before release automation can publish it. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex coding agent. Exact hosted model build and context-window size are not exposed in this runtime. Tool use included shell execution, repository editing, GitHub CLI operations, and local test/typecheck execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
555391fed7 |
fix: run restart recovery, workspace self-heal, quota-aware retries, failed-run metrics (#9183)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents run in heartbeat runs orchestrated by the server; run
lifecycle, retry scheduling, and the dashboard's run-activity metrics
are the subsystems involved
> - A spike in "failed" tasks traced to three causes: server restarts
killing in-flight runs and mislabeling them as failures, deterministic
workspace-validation loops when a worktree's branch diverged, and
provider quota/usage-limit errors being classified as generic transient
failures (putting agents into error state and polluting metrics)
> - Killed-then-recovered runs and quota waits are not product failures,
so both the runtime behavior and the reporting needed to distinguish
them
> - This pull request drains runs gracefully on shutdown with idempotent
restart retries, self-heals workspace branch mismatches, adds a
quota-aware failure class with reset-time retry, separates recovered
restart kills from true failures on the dashboard, and documents restart
hygiene for operators
> - The benefit is fewer spurious failures, automatic recovery instead
of manual repair, and dashboard metrics that reflect real failure rates
## Linked Issues or Issue Description
No public GitHub issue exists; describing the bug inline per the
bug-report template:
**What happened?**
In-flight heartbeat runs are marked `failed` when the server restarts,
even though a retry later succeeds. Worktrees whose checked-out branch
diverges from the issue branch fail workspace validation on every
subsequent run with no recovery path. Provider quota/usage-limit
responses are treated as generic transient upstream errors, putting
agents into an error state and retrying before the quota window resets.
The dashboard counts all of these as true failures, inflating failure
metrics.
**Expected behavior**
Graceful shutdown should interrupt (not fail) running runs and chain
exactly one recovery retry. Workspace validation should repair
recoverable branch mismatches automatically. Quota errors should get
their own error class with the retry scheduled at the provider reset
time and the agent left idle. The dashboard should report recovered
restart kills separately from true failures.
**Steps to reproduce**
1. Start a heartbeat run, then restart the server (SIGTERM) while it is
in flight — the run lands as `failed` with a process-loss error code
even when its retry succeeds
2. Check out an issue whose worktree branch has diverged (e.g. after a
force-moved branch) — every subsequent run fails
`workspace_validation_failed` deterministically
3. Drive an agent into a provider usage-limit window — the run fails as
a generic transient upstream error and the agent enters an error state
instead of idling until the reset time
**Paperclip version or commit**
master (base
|
||
|
|
ad961227f5 |
feat(secrets): add user-specific runtime secrets (#8825)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent runs often need provider credentials, API tokens, and other environment-bound secrets. > - Company-level secrets work for shared credentials, but they do not model values that should differ by human operator. > - Without a user-scoped model, a run can dispatch without knowing whether the responsible human has supplied the needed value. > - Paperclip also needs run attribution to make those user-scoped runtime checks deterministic and auditable. > - This pull request adds user-specific secret definitions, per-user values, environment bindings, responsible-user attribution, and runtime resolution gates. > - The benefit is that teams can define the secret once, let each user provide their own value, and block runs before dispatch when required user secrets or active definitions are unavailable. ## Linked Issues or Issue Description Refs #224 Refs #6057 This PR implements user-specific secret support as a core secret-management capability rather than a one-off adapter setting. It is related to existing public work on company secrets UI and runtime secret refs, but is distinct because the value is owned by the responsible user and resolved at run dispatch time. Related PR search before opening found existing secrets work such as #1550, #8256, #8614, #8634, and #8647; none of those add the full user-secret definition/value/runtime gate covered here. ## What Changed - Added user-secret definitions and per-user "My secrets" values, keeping stored values out of access metadata. - Added `user_secret_ref` environment bindings and UI affordances to pick them alongside existing secret refs. - Added responsible-user runtime resolution so user-secret refs resolve against the human responsible for the run. - Added pre-dispatch missing-secret gates so runs fail before adapter dispatch when required user values are absent or definitions are inactive. - Added low-trust allowlist hardening for user-secret runtime access. - Added issue, routine, run, and agent API key responsible-user attribution and fail-closed dispatch behavior when attribution cannot be resolved. - Added denial-copy mapping so responsible-user authorization failures surface as actionable run outcomes instead of opaque setup failures. - Added OpenAPI documentation for the user-secret routes. - Rebases cleanly on current `master`; migrations were renumbered incrementally as `0128_user_specific_secrets`, `0129_agent_api_key_responsible_user`, and `0130_run_responsible_user_invariant` after upstream `0126`/`0127` migrations. - Removed previously committed local design screenshots so the PR contains code/docs/tests only. ## Verification - PASS: PR head `2527febd106bcf3ca264ca0da7fca491084192d6` is based on `paperclipai/paperclip:master`. - PASS: `git diff --check` - PASS: `git diff --name-only public/master...HEAD | rg '^(pnpm-lock\\.yaml|\\.github/workflows/|screenshots/)' || true` produced no files. - PASS: migration journal audit confirmed unique indexes through `130` with tail entries `0126_issue_comment_derived_attribution`, `0127_environment_custom_images_instance_scoped`, `0128_user_specific_secrets`, `0129_agent_api_key_responsible_user`, and `0130_run_responsible_user_invariant`. - PASS: `pnpm --filter @paperclipai/ui typecheck` - PASS: `pnpm --filter @paperclipai/server typecheck` - PASS: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-responsible-user-invariant.test.ts` - PASS: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-active-run-output-watchdog.test.ts src/__tests__/heartbeat-stale-queue-invalidation.test.ts src/__tests__/heartbeat-workspace-finalize-branch.test.ts src/__tests__/issue-monitor-scheduler.test.ts` - PASS: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-comment-wake-batching.test.ts src/__tests__/heartbeat-retry-scheduling.test.ts src/__tests__/heartbeat-accepted-plan-workspace-refresh.test.ts src/__tests__/heartbeat-plugin-environment.test.ts` - PASS: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/low-trust-red-team-routes.test.ts` - PASS: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/secrets-service.test.ts` (55 tests) - PASS: `pnpm vitest run server/src/__tests__/secrets-routes.test.ts server/src/__tests__/secrets-service.test.ts` (89 tests after final Greptile cleanup fixes) - PASS: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/heartbeat-issue-liveness-escalation.test.ts` (17 tests after the final rebase CI fix) - PASS: focused server Vitest batches covering heartbeat recovery, project env, plugin env, routines, low-trust, pipelines, monitors, watchdog, and stale queue paths. - PASS: GitHub checks are green on `2527febd106bcf3ca264ca0da7fca491084192d6`, including Typecheck + Release Registry, Build, General tests, serialized server suites, e2e, Canary Dry Run, verify, security checks, and Greptile Review. - PASS: Greptile Review completed successfully on `2527febd106bcf3ca264ca0da7fca491084192d6` with Confidence Score 5/5, and GraphQL review-thread audit returned zero unresolved non-outdated threads. ## Risks - Runtime behavior now depends on a run having a correct responsible user; missing or incorrect responsibility assignment can block runs before adapter dispatch. - `user_secret_ref` bindings intentionally expose metadata without values, but UI/API callers may need to handle the new binding kind explicitly. - External secret providers and IAM policies are not automatically provisioned by this PR; operators still need to configure provider-side access for non-local vaults. - The PR is broad across db/shared/server/UI/runtime paths, so release validation should include both API and UI secret workflows before merge. - The migration renumbering is intentionally incremental after upstream migrations; the branch migrations use guarded column/table/index/constraint creation so users who tested the older draft numbering should not hit duplicate DDL for the existing objects. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, GPT-5-based coding agent (`gpt-5`), Codex local adapter with shell/tool use and code execution. Context window and internal reasoning mode are not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
c48feee190 |
Improve live agent feedback during sandboxed runs (#8915)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A core part of that experience is watching active agent runs without dropping into raw logs first > - Local and sandbox-backed adapters already record useful run output, progress, and tool activity > - But active issue threads could sit visually stale while the agent was syncing workspaces, tailing sandbox output, or emitting incremental tool-call updates > - Operators need timely, human-readable progress while preserving the raw transcript underneath > - This pull request streams sandbox run-log progress into runtime status, keeps visible issue threads refreshed, and folds repeated ACPX tool updates into stable transcript cards > - The benefit is that long-running agent work becomes easier to supervise without changing the task/comment control-plane model ## Linked Issues or Issue Description No public GitHub issue exists for this exact change. Problem/motivation: - During long-running sandboxed agent work, the issue UI can appear idle even though the agent is actively syncing, running tools, or producing incremental output. - Operators need realtime feedback at the issue-thread layer, not only after opening raw logs or waiting for the final heartbeat result. - Related public context: #1808 previously added live-run status dots to Projects; #4362 touches heartbeat wakeup behavior but is not a duplicate of this runtime/UI feedback change. ## What Changed - Added sandbox run-log streaming support and defaulted sandbox-capable local adapters into the richer live-feedback path. - Surfaced environment/sandbox sync progress through heartbeat runtime status with bounded, redacted snippets. - Added live issue-thread cache patching so visible active runs update as progress events arrive. - Folded repeated ACPX `tool_call` updates into one transcript card instead of stacking duplicate cards. - Updated adapter docs and added focused regression coverage for sandbox log streaming, runtime status, ACPX parsing, live updates, transcript rendering, and issue chat messages. ## Verification - `pnpm install --frozen-lockfile` - `pnpm exec vitest run ui/src/context/LiveUpdatesProvider.test.ts` - `pnpm exec vitest run server/src/services/heartbeat-run-runtime-status.test.ts server/src/__tests__/heartbeat-runtime-state.test.ts ui/src/context/LiveUpdatesProvider.test.ts` - `pnpm exec vitest run packages/adapter-utils/src/execution-target-sandbox.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts server/src/services/heartbeat-run-runtime-status.test.ts server/src/__tests__/agent-live-run-routes.test.ts server/src/__tests__/heartbeat-runtime-state.test.ts packages/adapters/acpx-local/src/ui/parse-stdout.test.ts ui/src/context/LiveUpdatesProvider.test.ts ui/src/components/transcript/RunTranscriptView.test.tsx ui/src/lib/issue-chat-messages.test.ts ui/src/components/IssueChatThread.test.tsx` - GitHub PR workflow on head `8397953e7b41ccd42e5d9457ee7e4dfb996e4ec5`: `verify`, build, typecheck/release-registry, e2e, general shards, serialized server shards, and canary dry run passed. - Greptile Review on head `8397953e7b41ccd42e5d9457ee7e4dfb996e4ec5`: Confidence Score 5/5, no unresolved review threads. ## Risks - Live issue-thread cache patching could miss an edge case for a route shape not covered by tests. - Surfacing active-run snippets needs continued care around redaction; this PR keeps snippets bounded and adds redaction-focused coverage. - More frequent active-run UI refreshes could expose performance issues on very large issue threads, though updates are scoped to visible run/query caches. ## Model Used OpenAI GPT-5 via Codex, operating as a tool-enabled coding agent with shell, git, and repository-editing capabilities. Context window size is not exposed in this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
bf982c8c83 |
Normalize adapter display labels (#8913)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Adapter names are part of the board-facing agent setup and management experience. > - The product now treats adapters as harnesses, while execution environments are modeled separately. > - Several built-in adapter labels still carried legacy local wording from the older harness-by-environment model. > - That wording makes the UI noisier and implies a distinction users no longer need to reason about. > - This pull request normalizes adapter display labels while keeping persisted adapter type identifiers unchanged. > - The benefit is clearer adapter selection and management copy without a database migration. ## Linked Issues or Issue Description No public GitHub issue was found for this exact cleanup. Related public PRs: - Supersedes #8910, an earlier branch for the same cleanup that did not include the later docs/gateway/Cursor alignment. - Refs #8819, which is related display-registry work for external multi-segment adapter labels, but not a duplicate of this built-in label cleanup. Feature request details: - Subsystem affected: Cross-cutting (`ui/`, `packages/adapters`, and docs). - Problem or motivation: user-facing adapter names include legacy local qualifiers even though adapters map to harnesses and environments are first-class elsewhere. - Proposed solution: remove the legacy local wording from built-in display labels, keep machine-readable adapter type ids unchanged, and keep gateway disambiguation where it is useful. - Alternatives considered: changing persisted adapter type ids was ruled out because it would create migration and compatibility risk; one-off UI replacements were ruled out because the display registry is already the correct central label boundary. - Roadmap alignment: this is small adapter UX polish, not a new roadmap-level core feature. ## What Changed - Updated the adapter display registry so known adapter labels are final and no built-in local adapter renders a legacy local suffix. - Preserved clean derived labels for unknown plugin local types while keeping gateway disambiguation for unknown gateway types. - Updated `AdapterManager` to prefer registry labels when the server reports raw adapter type ids for built-ins. - Removed legacy local wording from built-in adapter metadata labels in UI and adapter packages. - Aligned Cursor adapter metadata with the central display registry label. - Updated adapter docs and Storybook fixtures to match the new display names. - Added focused registry coverage for built-in labels and unknown plugin suffix behavior. ## Verification - `pnpm check:tokens` - `git diff --check origin/master...fix/adapter-display-labels` - Patch-addition scan for added secrets, private paths, and internal links: no matches. - GitHub duplicate search for open adapter-label/local-suffix issues and PRs; #8910 was identified as the older superseded public PR. - `pnpm exec vitest run ui/src/adapters/adapter-display-registry.test.ts` - `pnpm --filter @paperclipai/ui typecheck` - Stale-label scan found no remaining user-facing display-label suffixes; remaining local wording is operational/test terminology such as adapter ids, docs about running locally, and test descriptions. ## Risks Low risk. The change is display-label and documentation focused, and adapter type ids remain unchanged. The main risk is ambiguous gateway naming, mitigated by keeping explicit gateway labels where variants need disambiguation. ## Model Used OpenAI GPT-5 via Codex, tool-enabled coding agent in a local repository workspace. Context window size is not exposed by this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
8d9f9fd240 |
Add reusable sandbox custom images (#8794)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A growing part of that work runs in sandboxed environments rather than on the operator's local machine. > - Today sandbox providers can start fresh workspaces and run probes, but they do not have a shared contract for capturing and reusing prepared sandbox state. > - Operators need a way to set up tools, credentials, and project dependencies once, then reuse that prepared image for later agent runs. > - This pull request adds reusable sandbox custom images across the provider contract, server runtime, and board UI. > - It also keeps probes and sandbox copy flows aligned with pre-authenticated/custom-image environments. > - The benefit is faster, more reliable sandbox runs without repeatedly rebuilding the same environment setup. ## Linked Issues or Issue Description No public GitHub issue was found for this change. Inline feature request follows. ### Problem or motivation Sandboxed agents need reusable prepared runtime state so repeated runs do not require manual setup every time. Operators often need system packages, CLIs, SDKs, dependency caches, credentials, and project tooling available before an agent can work productively. ### Proposed solution Add a provider-level custom-image capability, server-side setup/capture lifecycle, Daytona/fake provider support, and board UI controls for creating, testing, selecting, and deleting custom images. ### Alternatives considered Leaving this as provider-specific setup outside Paperclip would keep the control plane blind to image state and would not give agents consistent environment metadata. Re-running setup commands for every lease is simpler, but slower and less reliable for interactive or credentialed setup. ### Roadmap alignment Checked `ROADMAP.md`; this aligns with the Cloud / Sandbox agents roadmap area and does not duplicate any related public issue or PR found by search. Additional context: - Subsystem affected: cross-cutting (`packages/db`, `packages/shared`, `packages/plugins`, `server`, `ui`). - Duplicate search: searched GitHub for `sandbox custom image` and `sandbox template environment`; no related public issues or PRs were found. ## What Changed - Added custom-image shared types, validators, constants, API paths, and database schema/migration. - Added server services/routes for custom-image templates and setup sessions, including runtime cleanup and provider metadata handling. - Extended plugin/sandbox provider capabilities for interactive setup, template capture, and template deletion. - Implemented custom-image support in the fake sandbox provider and Daytona provider. - Updated environment runtime/config handling so active custom images flow into leases, probes, and agent execution. - Added board UI controls and API client support for custom-image setup, capture, selection, status, and error states. - Hardened sandbox copy/probe behavior for insecure clipboard contexts and pre-authenticated sandbox images. - Added targeted coverage across shared validators, DB schema, server routes/services, provider plugins, adapter probes, and UI flows. ## Verification - `pnpm install --frozen-lockfile --ignore-scripts` - `pnpm vitest run packages/adapters/claude-local/src/server/test.probe.test.ts packages/adapters/claude-local/src/server/test.ts packages/adapters/codex-local/src/server/test.remote.test.ts packages/adapters/codex-local/src/server/test.ts` - `pnpm --filter @paperclipai/adapter-claude-local typecheck` - `pnpm --filter @paperclipai/adapter-codex-local typecheck` - `pnpm vitest run packages/db/src/environment-custom-images-schema.test.ts packages/shared/src/environment-custom-images.test.ts packages/shared/src/validators/plugin.test.ts server/src/__tests__/environment-custom-images-service.test.ts server/src/__tests__/workspace-runtime.test.ts packages/plugins/sandbox-providers/daytona/src/plugin.test.ts ui/src/pages/CompanyEnvironments.test.tsx ui/src/pages/CompanySettings.test.tsx` - `pnpm -r typecheck` - `pnpm build` - `rm -rf packages/db/dist && pnpm test:run` - Public-safety scan of the final diff found no internal Paperclip issue links, private instance URLs, or real secret patterns. ## Risks - Adds a database migration and new environment runtime tables, so migration ordering and rollback need care. - Provider implementations may differ in how reliably they can capture/delete images; unsupported providers surface capability-gated UI states. - Custom-image state can contain operator-prepared tooling and credentials inside the provider image, so providers must enforce their own access controls and cleanup semantics. - Broad surface area across shared contracts, server runtime, plugins, adapters, and UI means CI and Greptile review should be watched closely. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 (`gpt-5`) via Codex CLI with tool use and code execution. Assisted with branch cleanup, conflict resolution, local verification, and PR preparation. Earlier branch implementation work was assisted by Paperclip-managed Claude/Codex agents. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
0c2ec7deb4 |
Fix sandboxed Claude and Codex probe behavior (#8775)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Local adapters are the bridge between Paperclip's control plane and provider CLIs such as Claude Code and Codex. > - Those adapters can run either on the host machine or inside a remote/sandbox execution target. > - Sandbox probes need to validate the same auth/config path that real sandbox execution will use. > - The previous probe paths could surface misleading Claude errors, rely on host-only Codex state, or upload far more Codex home state than the probe needed. > - This pull request fixes the Claude and Codex sandbox probe/runtime behavior together while keeping provider-specific sandbox image work out of scope. > - The benefit is faster, clearer adapter health checks that better match real sandbox execution. ## Linked Issues or Issue Description No public GitHub issue was found for this exact bug during duplicate search. Bug report: **What happened?** Sandboxed Claude/Codex adapter tests could diverge from real runtime auth/config behavior. Claude sandbox probes could show the leading stream init line instead of the real final error, and Codex sandbox probes could upload full managed home state or mask a sandbox-local login with an empty uploaded `CODEX_HOME`. **Expected behavior** Sandbox probes should exercise the remote runtime contract, preserve useful sandbox credentials, avoid relying on unrelated host state, and report actionable probe failures. **Steps to reproduce** 1. Configure a remote/sandbox execution target for `claude_local` or `codex_local`. 2. Run the environment Test/probe path where host credentials differ from the sandbox's runtime credentials or the managed Codex home contains session history. 3. Observe that probe behavior can differ from the actual sandbox runtime path or surface an unhelpful Claude stream initialization line. **Paperclip version or commit** Current `master` before this PR, based on `4a2447da3`. **Deployment mode** Local development/control-plane deployment with remote sandbox execution targets. Related search performed: - Public issues: `Claude sandbox probe`, `Codex CODEX_HOME sandbox` returned no matches. - Public PRs: `Claude Codex sandbox probe`, `codex home sandbox`, `claude auth sandbox` returned no matches. ## What Changed - Made Claude sandbox Test probes materialize the same Paperclip-managed Claude config seed path used by sandbox execution. - Preserved sandbox-local Claude credentials when materializing remote Claude config and expanded auth-required detection for `/login` API-key failures. - Improved Claude hello-probe diagnostics so the final result/error is surfaced instead of the unhelpful stream init event, with transient upstream failures downgraded to warnings. - Changed Codex probe behavior to upload only minimal auth/config files instead of the full managed `CODEX_HOME`. - Let Codex sandbox probes leave `CODEX_HOME` unset when the host has no credentials, so pre-authenticated sandbox images can be tested directly. - Excluded bulky host-local Codex session/shell state from sandbox runtime home uploads. - Switched the Codex local default model away from the ChatGPT-unsupported `gpt-5.3-codex` option. - Added regression coverage for Claude parsing/probe paths, Codex adapter metadata/argument/probe behavior, and server-level Claude sandbox environment behavior. ## Verification Passed locally: - `pnpm install --frozen-lockfile` - `pnpm vitest run packages/adapters/claude-local/src/server/parse.test.ts packages/adapters/claude-local/src/server/test.probe.test.ts server/src/__tests__/claude-local-adapter-environment.test.ts` - `pnpm vitest run packages/adapters/codex-local/src/index.test.ts packages/adapters/codex-local/src/server/codex-args.test.ts packages/adapters/codex-local/src/server/test.remote.test.ts` - `pnpm --filter @paperclipai/adapter-claude-local typecheck` - `pnpm --filter @paperclipai/adapter-codex-local typecheck` - `pnpm --filter @paperclipai/server typecheck` - `git diff --check` ## Risks - Adapter configuration behavior is sensitive to local vs sandboxed execution mode, so review should focus on environment detection, argument construction, and any state written during probe/test runs. - The Codex default-model change may affect newly created agents that rely on the adapter default instead of an explicit model. - Excluding Codex session/shell state from sandbox uploads should be safe for fresh sandbox runs, but reviewers should confirm no runtime resume path depends on that host-local state. - Provider-specific setup/capture behavior is intentionally left to separate work. ## Model Used OpenAI GPT-5 Codex via Paperclip `codex_local`; tool-enabled local coding session with terminal access. Context window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
a27e5ad002 |
Add ephemeral sandbox runtime status plumbing (#8593)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandboxed agent runs can spend meaningful time preparing a remote workspace before the agent transcript shows useful output. > - Operators need short, current progress text for those setup phases, but that text should not become durable run history. > - The existing live-run websocket path already carries run updates to the UI, so the backend can reuse that channel instead of adding polling. > - This pull request adds an ephemeral runtime-progress contract, a process-local status store, and heartbeat integration for sandbox-managed runs. > - The benefit is a clearer active-run experience without database migrations or persistent progress rows. ## Linked Issues or Issue Description Refs #248 No exact public GitHub issue was found for this status-message plumbing. The underlying problem is that active sandboxed runs currently have setup phases, such as workspace sync and restore, where the operator cannot see concise current progress through the live run state. This PR addresses that gap for the backend/runtime layer while keeping progress messages ephemeral. GitHub search performed for related or duplicate work: `sandbox runtime status`, `sandbox restore index`, and `runtime progress`. No direct duplicate PR was found. ## What Changed - Added shared runtime-progress types and the `heartbeat.run.progress` live event type. - Added a process-local heartbeat run runtime-status store with TTL, bounded/redacted messages, and terminal cleanup. - Threaded runtime progress callbacks through heartbeat execution and active/live run serialization. - Emitted sandbox-managed runtime phase updates for sync, adapter startup, restore/export, and finalization paths. - Added backend and adapter-utils tests for ephemeral status behavior, terminal cleanup, live serialization, and sandbox progress callbacks. ## Verification - `pnpm install --frozen-lockfile` - Local PII scan before push: high-confidence secret patterns, internal issue links, local user paths, and private URL patterns checked across all three split diffs; no real secrets or internal links found. The only secret-like text is an intentional fake test fixture (`sk-test-secret`). - `git diff --check origin/master..feat/sandbox-runtime-status` - `pnpm exec vitest run server/src/services/heartbeat-run-runtime-status.test.ts server/src/__tests__/heartbeat-runtime-state.test.ts server/src/__tests__/agent-live-run-routes.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — 4 files, 23 tests passed. - `pnpm run typecheck` passed on both top stacks that include this branch: `feat/sandbox-status-ui` and `fix/sandbox-restore-index-sync`. - `pnpm run build` passed on both top stacks that include this branch; Vite reported existing CSS `::highlight` and chunk-size warnings. - `pnpm run test:run` was attempted on `fix/sandbox-restore-index-sync`; it failed in two unrelated broad-suite tests. One depends on this host's Git default branch behavior, and one depends on local Claude model-discovery environment. The changed focused suites above pass. ## Risks - Runtime progress is process-local by design, so status disappears after TTL, terminal cleanup, or server restart. - Clients that do not consume `heartbeat.run.progress` simply keep existing behavior. - Message redaction is intentionally generic; overly specific phase details should stay out of runtime-progress payloads. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex coding agent, with shell/tool execution in a local worktree. Exact context-window metadata is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip CTO <cto@paperclip.local> Co-authored-by: Paperclip CTO <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
51ffbb380f |
Exclude transient Codex home dirs from sandbox sync (#8581)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Local adapters can run agents against sandboxed execution targets by syncing the workspace and selected runtime assets into the sandbox. > - The Codex local adapter includes a managed Codex home asset so sandboxed Codex runs can use the expected auth, config, skills, and session state. > - That asset follows symlinks, which is useful for real Codex home content but unsafe for transient launcher directories. > - Transient `tmp` and `.tmp` directories can contain symlinks to large host binaries, so the sandbox archive can inline large executable targets instead of just the small home directory content. > - This pull request excludes transient Codex home directories from the sandbox home asset while preserving the required Codex home files. > - The benefit is a much smaller and more predictable sandbox setup upload without changing the runtime files Codex actually needs. ## Linked Issues or Issue Description No public issue was found for this exact sandbox archive-size bug. Bug description: - What happened: sandboxed `codex_local` runs sync the managed Codex home as a `home` asset with `followSymlinks` enabled. If transient Codex home dirs such as `tmp` or `.tmp` contain symlinks to a large host binary, the archive can inline that binary and make `Syncing home to sandbox` much larger than the managed home directory itself. - Expected behavior: sandbox setup should include the Codex home files needed for auth, config, skills, and session continuity, but should not archive transient launcher scratch directories. - Reproduction shape: create a managed Codex home with normal auth/config/skills files and a `tmp/arg0` or `.tmp` symlink to a large host executable, then start a sandboxed `codex_local` run. The home asset archive grows by the symlink target size. - Version/commit: observed on local `master` before this change. - Related public context: #5028 covers a different managed Codex home reliability issue around stale auth files; this PR addresses sandbox archive bloat from transient symlink targets. ## What Changed - Excluded `tmp` and `.tmp` from the Codex `home` asset that is uploaded for sandboxed runs. - Added regression coverage proving transient symlinked home dirs are excluded from the tar while required auth/config/skills files remain included. - Kept `followSymlinks` behavior for the rest of the Codex home asset so existing non-transient symlink behavior is preserved. ## Verification - `git diff --check` - Local PII/secret pattern scan over the committed diff - `pnpm exec vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts` - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/adapter-codex-local typecheck` ## Risks Low risk. The exclusion is limited to transient Codex home scratch directories, and the regression test verifies the files needed in the sandbox are still archived. The main compatibility risk is if a user intentionally placed required persistent Codex state under `tmp` or `.tmp`; those paths are treated as volatile scratch space by this change. ## Model Used OpenAI Codex coding agent based on GPT-5, with shell, git, and GitHub CLI tool use. Exact hosted model build and context-window size were not exposed in the local adapter runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
7aa212296e |
codex_local adapter output inactivity monitor (#5017)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - The `codex_local` adapter runs `codex exec` as a child process and streams JSONL events from its stdout > - NEE-79 caught a real-world `codex_local` orphan: codex sat in `read()` on stdin for 1h+, no JSON events emitted past startup, no LLM call in flight. The adapter had no inactivity timer; the only safety net was the platform-level 1h silent-run detector > - This is precisely the failure shape NEE-80 scoped: an adapter-detected fault that should be killed *by the adapter* and surfaced as `failed`, not waited out for an hour > - This pull request adds an output-inactivity watchdog inside the codex-local adapter that resets on every parsed JSONL event from stdout, kills the child via SIGTERM → 5s grace → SIGKILL when it fires, and resolves the run with a structured watchdog failure > - The benefit is that NEE-79-class hangs shorter than 1h stop reaching the platform-level safety net — they fail fast and visibly at the adapter layer, with diagnostic logs that don't require host shell access ## Linked Issues or Issue Description No existing GitHub issue covers this; describing the underlying bug in-PR per the bug report template (`.github/ISSUE_TEMPLATE/bug_report.yml`): - **What happened?** A `codex_local` run sat in `read()` on stdin for over an hour: codex emitted its startup JSONL events, then nothing — no further events, no LLM call in flight, no exit. The adapter kept the child alive indefinitely; the run was only reaped by the platform-level 1h silent-run safety net, an hour after it had effectively died. - **Expected behavior:** The adapter should detect that its child has stopped producing output long before the platform-level safety net, kill it, and surface the run as `failed` with a diagnostic that explains what happened. - **Steps to reproduce:** Run any `codex_local` issue where `codex exec` hangs after startup (e.g. codex blocks reading stdin and never emits another JSONL event). Observe the run stays alive until the 1h platform safety net fires. - **Paperclip version or commit:** reproduced on master prior to this branch. - **Agent adapter(s) involved:** codex_local. Related PRs found while searching for duplicates (none implement an event-aware inactivity watchdog inside `codex_local`): - #4004 — generic idle + wall watchdogs for `runChildProcess` in adapter-utils; complementary, operates below the JSONL parse layer and is not codex-event-aware - #4742 — fail-fast on codex-local *startup* hang; this PR covers the post-startup hang class - #6861 — zombie-run termination for codex-local; reaping after exit, not inactivity detection - #7811 — the analogous output-idle timeout for grok-local ## What Changed - `packages/adapters/codex-local/src/server/watchdog.ts` *(new)* — pure watchdog primitive with injectable timers/clock: `resolveCodexInactivityTimeout`, `createCodexInactivityWatchdog`, `formatWatchdogErrorMessage`. Default `7 * 60_000` ms; honors `null` as the disabled escape hatch - `packages/adapters/codex-local/src/server/execute.ts` — `runAttempt` now wraps `onSpawn` to capture `pid`/`processGroupId`, feeds stdout chunks through `noteStdoutChunk`, and on watchdog fire sends SIGTERM to the process group, schedules SIGKILL after 5s, and returns an `AdapterExecutionResult` with `exitCode: null`, `signal: SIGTERM|SIGKILL`, `errorMessage: "watchdog: no codex output for {N}m {S}s"`, `errorCode: "codex_output_inactivity_watchdog"`. With `errorMessage` set and `timedOut: false`, `heartbeat.ts:5860` maps the run to `outcome === "failed"` (not `cancelled`) - `packages/adapters/codex-local/src/index.ts` — `agentConfigurationDoc` documents `outputInactivityTimeoutMs` (number ms; `null` disables; non-positive falls back to default with a warning log at spawn) - `packages/adapters/codex-local/src/server/watchdog.test.ts` *(new)* — 13 tests covering acceptance criteria 2 and 3 plus supporting cases: fires after silence, no-fire across 12× (threshold − 1s) cycles, multi-event chunks, non-JSON ignoring, single-fire idempotency, formatter shape, full resolution table, `null` → disabled - `packages/adapters/codex-local/src/server/watchdog.integration.test.ts` *(new)* — real Node subprocess that prints one JSONL event then sleeps; `runChildProcess` reaps it within `threshold + 6s`; signal is SIGTERM or SIGKILL; `parsedEventCount === 1` (acceptance criteria 1 and 4) ## Verification ``` $ pnpm --filter @paperclipai/adapter-codex-local typecheck > tsc --noEmit # clean $ pnpm --filter @paperclipai/adapter-codex-local exec vitest run ✓ src/server/quota-spawn-error.test.ts (1 test) ✓ src/server/codex-home.test.ts (3 tests) ✓ src/server/codex-args.test.ts (3 tests) ✓ src/server/watchdog.test.ts (13 tests) ✓ src/server/parse.test.ts (9 tests) ✓ src/server/execute.remote.test.ts (4 tests) ✓ src/ui/parse-stdout.test.ts (3 tests) ✓ src/ui/build-config.test.ts (1 test) ✓ src/server/watchdog.integration.test.ts (1 test) Test Files 9 passed (9) Tests 38 passed (38) ``` Acceptance criteria check: 1. ✅ Simulated child emits one event then sleeps → killed at threshold; result `errorMessage` matches `watchdog: no codex output for {N}m {S}s` (`watchdog.integration.test.ts`) 2. ✅ Child emits events every (threshold − 1s) → not killed (`watchdog.test.ts: "does not fire when events arrive every (threshold - 1s)"`) 3. ✅ `outputInactivityTimeoutMs: null` disables the watchdog (`resolveCodexInactivityTimeout` returns `disabled`; `execute.ts` skips watchdog construction and logs a startup warning) 4. ✅ Real subprocess reaped well within `threshold + 6s` — 250 ms threshold, 290 ms wall clock in CI 5. ✅ Adapter-level fault → `outcome === "failed"` per `heartbeat.ts:5860`. NEE-79-class hangs <1h get caught at the adapter, not the platform-level safety net Post-rebase verification (head `2a8703fab`, rebased onto master `69a368ed5`): ``` $ pnpm --filter @paperclipai/adapter-codex-local typecheck # clean $ pnpm --filter @paperclipai/adapter-codex-local exec vitest run Test Files 11 passed (11) Tests 64 passed (64) ``` ## Risks - Low risk. Behind a default-on watchdog that only fires after 7m of zero parsed JSON events. Operators can disable it with `outputInactivityTimeoutMs: null` for known-slow tasks - The kill path reuses the same `process.kill(-pgid, signal)` pattern that `runChildProcess` already uses for its terminal-result cleanup, so signal semantics match the existing code path - `timedOut: false` is preserved on watchdog fire — the platform-level timeout outcome is unchanged, only the `failed`-vs-success classification flips. No behavioral shift for already-failing runs - Sandbox/SSH execution targets: the watchdog fires and emits the structured log, but the kill is best-effort because remote pids aren't owned by this process. The platform-level 1h safety net still applies. Out of scope for NEE-81 by design ## Model Used - Provider: Anthropic Claude - Model: claude-opus-4-7 - Mode: Claude Code (extended thinking, tool use) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots (N/A — no UI changes) - [x] I have updated relevant documentation to reflect my changes (`agentConfigurationDoc` for `outputInactivityTimeoutMs`) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (re-running on the rebased head) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (P2 addressed; review threads resolved) - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Neeraj Kumar Singh <b.nirajkumarsingh@hotmail.com> Co-authored-by: Devin Foley <devin@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
67ca8287e8 |
fix(codex-local): seed managed auth into isolated homes (#8403)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The `codex_local` adapter isolates local Codex runs by assigning managed `CODEX_HOME` state per company and per agent > - PR #8272 tightened that isolation, but it left a gap: once `CODEX_HOME` became explicit, the adapter treated it like a user-managed override and skipped auth seeding > - That meant newly isolated agents could launch with no usable `auth.json`, hit OpenAI unauthenticated, and fail with `401 Missing bearer` > - Users who had already persisted one of those broken managed homes could remain stranded even after config changes unless Paperclip repaired the home itself > - This pull request teaches Paperclip to seed managed homes correctly, backfill already-stranded managed homes on startup, and reject credential-less managed homes before they reach the provider > - The benefit is that affected managed `codex_local` agents recover automatically after upgrade and restart, without manual `CODEX_HOME` surgery ## Linked Issues or Issue Description - Fixes #497 - Refs #5028 - Related PR: #8272 - Related PR: #8399 ## What Changed - Distinguished Paperclip-managed `CODEX_HOME` paths from genuine external overrides and always seeded auth into managed homes, even when `CODEX_HOME` is explicit in config. - Wrote API-key-backed `auth.json` files for managed homes when `OPENAI_API_KEY` is configured, otherwise symlinked the shared Codex auth for subscription/OAuth flows. - Added a startup reconciliation pass that backfills already-isolated managed homes created by the broken release so upgrade plus restart repairs stranded agents automatically. - Preserved previously resolved API-key auth when the stored `OPENAI_API_KEY` binding is secret-backed and startup cannot resolve the secret value directly. - Hardened the managed-home preflight to require a credential-bearing `auth.json`, not just file presence, and documented the recovery behavior. - Added regression tests covering managed-home seeding, fail-fast behavior, and server-side startup reconciliation. ## Verification ```bash pnpm exec vitest run packages/adapters/codex-local/src/server/codex-home.test.ts packages/adapters/codex-local/src/server/execute.auth.test.ts server/src/__tests__/codex-auth-reconciliation.test.ts server/src/__tests__/codex-local-execute.test.ts server/src/__tests__/server-startup-feedback-export.test.ts pnpm --filter @paperclipai/adapter-codex-local typecheck pnpm --filter @paperclipai/server typecheck pnpm check:tokens pnpm exec vitest run server/src/__tests__/server-startup-feedback-export.test.ts ``` GitHub Actions `PR` workflow is green on latest head `e5b1e08d4`, including policy, typecheck, test shards, build, e2e, serialized server suites, and canary dry run. Greptile Review is green on latest head with 0 comments added. No UI changes. ## Risks - Startup reconciliation now mutates persisted managed Codex homes at boot. Risk is low because it only touches Paperclip-managed company/agent home paths and no-ops when a home already has usable auth. - Genuine external `CODEX_HOME` overrides remain intentionally self-managed, so those users still own repair steps inside their custom home. - Hosts with neither shared Codex auth nor an explicit per-agent API key now fail earlier with a clearer adapter error instead of surfacing a downstream `401`, which changes timing but not capability. ## Model Used - OpenAI Codex, GPT-5-based coding agent in a local Codex session; exact served model ID/context window were not exposed to the session. Tool use and code execution were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
07e98d2b2c |
feat(adapter-utils): add observable sandbox sync progress (#8395)
## Thinking Path > - Paperclip is the control plane for running AI-agent companies, so long-running remote work needs to stay observable to human operators. > - Cloud / sandbox agents are an active roadmap area, and their workspace sync path is part of the runtime substrate every remote coding run depends on. > - In the sandbox and SSH execution-target flows, Paperclip logged that sync had started, then often went silent for the full transfer window. > - That made large remote syncs feel stalled and also hid a real performance problem in the command-managed sandbox upload path. > - The first part of this pull request threads a throttled progress-reporting surface through the adapter execution-target stack so sync and restore work can emit meaningful updates. > - The second part fixes the command-managed sandbox transport itself: it removes the old serial 32KB append bottleneck, but also falls back away from the single-stream path when a provider-backed sandbox runner cannot surface mid-flight stdin progress. > - The result is that sandbox and SSH transfers are both faster and more observable, including the live Daytona-style sandbox case that previously only emitted `0%` and `100%`. ## Linked Issues or Issue Description No public GitHub issue exists for this bug, so it is described inline below following the bug report template. ### What happened - Remote sandbox and SSH workspace syncs could spend a long time transferring data while only logging a start line (`Syncing workspace and runtime assets to sandbox environment`) and, at best, a terminal line. - In the command-managed sandbox path, the original upload implementation also paid a large performance cost by appending base64 data in many small sequential remote writes (thousands of serial 32KB round-trips on a large workspace). - After the initial transport rewrite, live provider-backed sandbox runs still only emitted `0%` and `100%` because the single-stream stdin RPC buffered progress until completion. ### Expected behavior - Long-running sandbox and SSH syncs should periodically report how much of the transfer is complete (a percentage and/or MB transferred) so an operator can tell the run is healthy and making progress rather than stuck. - The main sandbox upload path should not be artificially slow. - A transfer that fails partway should leave an explicit failure marker in the log rather than a dangling intermediate percentage. ### Steps to reproduce 1. Run an agent against a sandbox (command-managed) or SSH (remote-managed) execution target with a non-trivial workspace. 2. Watch the run log during the workspace/runtime asset sync phase. 3. Observe that the log shows the sync start line and then stays silent for the full transfer (live provider-backed sandbox runs only show `0%` then `100%`). ### Paperclip version or commit - Branch `PAPA-825-provide-status-updates-when-syncing-sandboxes` off `master`. ### Deployment mode - Self-hosted / local instance using sandbox (command-managed) and SSH (remote-managed) execution targets, including provider-backed sandbox runners. ## What Changed - Added shared throttled runtime progress reporting and threaded `onProgress` through the adapter execution-target surface and adapter `execute.ts` entrypoints. - Added sync and restore progress reporting for the command-managed sandbox path and the SSH/remote-managed path, including git import/export progress where totals are known. - Reworked command-managed sandbox transfer behavior so uploads use the faster single-stream path when appropriate, but fall back to chunked progress-emitting writes when the runner cannot expose mid-stream stdin progress. - Marked provider-backed environment sandbox runners as not supporting single-stream stdin progress so live sandbox runs emit meaningful intermediate updates instead of only `0%` and `100%`. - Emit an explicit terminal failure marker (`failed at NN% (x/y MB)`) when an SSH/tar transfer rejects, so a failed sync no longer leaves a dangling intermediate percentage in the log. - Run the SSH sync/restore size estimate (local directory walk / remote `du` probe) concurrently with the transfer instead of awaiting it before opening the pipe, so progress instrumentation no longer adds startup latency proportional to workspace file count. - Added and extended focused regression coverage for runtime progress throttling and the new failure marker, command-managed sandbox transfers, sandbox orchestration, SSH transfer progress, and environment execution-target wiring. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/runtime-progress.test.ts packages/adapter-utils/src/ssh-fixture.test.ts packages/adapter-utils/src/command-managed-runtime.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts` - `pnpm exec vitest run packages/adapter-utils/src/command-managed-runtime.test.ts server/src/__tests__/environment-execution-target.test.ts` - `npx tsc --noEmit` for `packages/adapter-utils` ## Risks - The provider-backed sandbox fallback now prefers chunked command-managed writes when progress hooks are active, so small-to-medium uploads may trade some raw throughput for observable intermediate progress on runtimes that cannot surface true mid-stream stdin progress. - Progress percentages on tar-based transfers still depend on estimates in some cases, so operators may briefly see MB-only lines before the estimate resolves, then near-final clamping before the terminal `100%` line. - This PR changes shared execution-target behavior used by multiple adapters, so regressions would most likely appear in remote runtime setup/teardown flows rather than in a single adapter. ## Model Used - Initial implementation: OpenAI GPT-5.4 via Codex local agent (`codex_local`), high reasoning mode. - Observability follow-ups (failure marker, concurrent size estimate, added tests): Claude Opus 4.8 via Claude Code (`claude_local`). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
dependabot[bot]
|
6f142a60ce |
build(deps-dev): bump @types/node from 22.19.11 to 22.19.21 (#7748)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 22.19.11 to 22.19.21. <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Svetlana ZolotenkovaandPaperclip
|
5320a44088 |
Guard codex_local agents from shared OpenAI key (#8272)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - The `codex_local` adapter runs local Codex CLI processes and builds their environment from persisted agent config plus host process env. > - A host-level `OPENAI_API_KEY` or shared Codex auth home can silently make new agents spend through shared credentials. > - Existing agents can be repaired manually, but new and updated agents need a persistent guard at the agent configuration boundary. > - This pull request isolates new and updated `codex_local` agents with per-agent `CODEX_HOME` and an empty `OPENAI_API_KEY` override. > - The benefit is that future agent creation or adapter updates cannot silently fall back to shared OpenAI credentials. ## Linked Issues or Issue Description Paperclip work item: [ZOL-5477](/ZOL/issues/ZOL-5477). No matching GitHub issue exists, so the bug is described inline following `.github/ISSUE_TEMPLATE/bug_report.yml`. **Pre-submission checklist** - [x] I have searched existing open and closed issues and this is not a duplicate. - [x] I am on the latest `master` commit for this PR branch. - [x] I have confirmed the error originates in Paperclip's `codex_local` adapter configuration boundary, not in a provider outage. **What happened?** New or updated `codex_local` agents could inherit a host-level `OPENAI_API_KEY` or use a shared Codex home when their adapter config did not explicitly isolate those values. That made it possible for future agents or manual adapter edits to silently fall back to shared OpenAI credentials. **Expected behavior** Creating, hiring, or updating a `codex_local` agent should either persist isolated per-agent configuration or reject unsafe shared Codex home configuration with a clear 422 response. The guard must not print secret values. **Steps to reproduce** 1. Create or update a `codex_local` agent without an explicit `adapterConfig.env.OPENAI_API_KEY` override. 2. Run it on a host where the Paperclip server process has `OPENAI_API_KEY` set. 3. Observe that the adapter process can inherit the host key unless Paperclip persists a blocking empty override. 4. Set `adapterConfig.env.CODEX_HOME` to a shared path such as `~/.codex` or the company-level `codex-home`. 5. Observe that the old code allowed the shared auth home instead of returning a validation error. **Paperclip version or commit** - Reproduced by inspection against `master` before this PR. **Deployment mode** - Local dev / self-hosted server with `codex_local` agents. **Installation method** - Built from source. **Agent adapter(s) involved** - Codex. **Database mode** - Not database-related. **Access context** - Board and agent configuration paths. **Relevant logs or output** - No secret-bearing logs included. **Relevant config** - Unsafe shape: missing `adapterConfig.env.OPENAI_API_KEY`, or shared `adapterConfig.env.CODEX_HOME`. - Fixed shape: per-agent `CODEX_HOME` plus empty `OPENAI_API_KEY` override. **Additional context** Related PR search for `codex_local OPENAI_API_KEY CODEX_HOME` found: - #3681 `fix: preserve managed Codex auth and repo-root env loading` - #5621 `fix: copy worktree codex auth locally` Those are adjacent auth-handling changes, but they do not add the agent create/update guard implemented here. **Privacy checklist** - [x] I have reviewed all pasted output for PII, usernames, file paths, API keys, tokens, company names, and redacted where necessary. ## What Changed - Added a `codex_local` config guard in agent create, hire, and update routes. - The guard assigns `adapterConfig.env.CODEX_HOME` to `companies/<companyId>/agents/<agentId>/codex-home` when missing. - The guard persists `adapterConfig.env.OPENAI_API_KEY = ""` when missing, preventing host env inheritance. - Shared `CODEX_HOME` values for the company codex-home, host `$CODEX_HOME`, or `~/.codex` now fail with a 422 error. - Added route tests for create, hire, update, and rejected shared host Codex home. - Updated `codex_local` and development docs to describe the per-agent home contract. ## Verification - `pnpm exec vitest run server/src/__tests__/agent-adapter-validation-routes.test.ts` - `pnpm exec vitest run server/src/__tests__/agent-skills-routes.test.ts` - `pnpm typecheck` - `git diff --check upstream/master...HEAD` - `gh pr list --repo paperclipai/paperclip --state all --search "codex_local OPENAI_API_KEY CODEX_HOME" --limit 20 --json number,title,state,url` - `rg -n "codex|OPENAI_API_KEY|CODEX_HOME|adapter" ROADMAP.md` returned no roadmap overlap. ## Risks - Existing legacy `codex_local` agents with shared `CODEX_HOME` will get a clear 422 when their adapter config is updated until the shared path is replaced. This is intentional because silent fallback is the bug being guarded. - Low migration risk: no database migration and no secret values are printed or persisted beyond the empty override. ## Model Used - OpenAI GPT-5.5 Codex, Codex coding-agent session with repository tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge ## Paperclip - Issue: [ZOL-5477](/ZOL/issues/ZOL-5477) - Owner: Разработчик (`6625498c-66c9-429f-b578-4463ddc3ba16`) - Status: waiting reviewer - Next action: merge after approval and green CI --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Jannes StubbemannandClaude Opus 4.8
|
9e750d3e92 |
feat(codex-local): env-driven gateway routing via PAPERCLIP_CODEX_PROVIDERS config.toml (#7919)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The `codex-local` adapter runs the OpenAI Codex CLI; Paperclip already maintains a managed `CODEX_HOME` per company and ships it to remote/sandboxed execution targets > - Deployments increasingly put an OpenAI-compatible LLM gateway between the harness and the model for cost, governance, or data-residency reasons: LiteLLM, OpenRouter, Portkey, Kong, a corporate proxy, self-hosted models (vLLM/Ollama), or region-pinned/sovereign endpoints. But Codex has no CLI flag or env var for a custom endpoint: its only mechanism is `[model_providers.<id>]` tables (with `base_url`, `env_key`, `wire_api`) in `$CODEX_HOME/config.toml`, selected by a root-level `model_provider` key > - Today there is no supported way to get such provider config into the managed `CODEX_HOME`, so gateway routing requires hand-editing files the adapter owns and regenerates > - This pull request adds the codex analogue of #7837's opencode mechanism: a `PAPERCLIP_CODEX_PROVIDERS` JSON env var whose shape maps 1:1 onto codex's TOML schema, merged into the managed `config.toml` so the existing asset-shipping + `env.CODEX_HOME` mechanics deliver it to local and sandboxed runs alike; nothing here is specific to one hosting setup > - The benefit is Codex works behind any OpenAI-compatible gateway with config only; with no env set, behavior is unchanged ## Linked Issues or Issue Description No existing issue; describing in-PR (feature / adapter enhancement). - **Gap:** there is no supported way to register a custom/gateway `[model_providers.*]` endpoint for `codex-local`. Codex's only custom-endpoint mechanism is `config.toml` (`base_url` + `env_key` + `wire_api`, selected via the root `model_provider` key), and the adapter owns/regenerates the managed `CODEX_HOME`, so operators cannot durably hand-edit it. - Related: #7837 (the opencode-local analogue of this change, same env-driven gateway-routing pattern). Searched for duplicate/related PRs: no existing codex-local gateway/provider-routing PR found. > Note on ROADMAP: this is adapter-level, opt-in config (defaults unchanged) that *enables* gateway routing for one harness; it is not the core "Cloud / Sandbox agents" platform work itself. ## What Changed - New `prepareCodexRuntimeConfig()` (`packages/adapters/codex-local/src/server/runtime-config.ts`): reads `PAPERCLIP_CODEX_PROVIDERS` (run env first, then `process.env`), shaped as `{"providers": {"<id>": {base_url, env_key, wire_api, ...}}, "model_provider": "<id>"}`, and merges it into the managed `CODEX_HOME`'s `config.toml`. No-op when unset or empty. - A malformed value (invalid JSON, not a JSON object, no `providers` object, no usable provider entries, or individual entries with empty names or non-object values, which are skipped by name) is never silently dropped: each case surfaces a distinct, user-visible note (via the prepare notes, which flow into command notes + `onLog`) and unusable input leaves `config.toml` untouched. - Merge is marker-delimited and TOML-correct: existing `config.toml` content is preserved between two managed blocks. Root keys (e.g. `model_provider`) are prepended **before the first table header** (TOML root-region rule), `[model_providers.*]` tables are appended. Pre-existing same-name provider sections and root `model_provider` keys are excised so the managed definitions win without duplicate-table parse errors. - `{env:VAR}` placeholders are expanded server-side for literal-credential fields; `env_key` indirection remains the preferred path. - Crash-safe restore: prepare writes a pre-run backup (`config.toml.paperclip-backup`) before the merged file; `cleanup()` restores the original in the execute `finally` and removes the backup. If a run never reaches `cleanup()` (a throw during the setup between prepare and execution, or SIGKILL), the next prepare restores the original from the backup with full fidelity, including user `[model_providers.*]` sections the merge excised (review feedback, P2); plain block-stripping remains the fallback for pre-backup state. - An explicit adapter-config `env.CODEX_HOME` override is treated as user-managed: no merge, surfaced as a command note. - Dependency-free hand-emitted TOML (strings/numbers/booleans, arrays of scalars, plain objects as inline tables); basic strings escape U+0000-U+001F and U+007F per TOML 1.0 (review feedback, P2). Merged output was additionally validated locally with python tomllib during development; the committed tests assert the structural invariants. - `execute.ts` wiring: `prepareCodexRuntimeConfig` runs after `prepareManagedCodexHome` (before the home ships to the remote target), notes surface via `onLog` + command notes, and the `finally` calls `cleanup()`. **Note for reviewers:** current codex removed `wire_api = "chat"` (openai/codex#10157, Feb 2026), so gateway provider configs must use `wire_api = "responses"`, i.e. the gateway must speak `/v1/responses`. The adapter passes the value through verbatim; this is a codex-side constraint worth knowing when configuring it. ## Verification - `pnpm --filter @paperclipai/adapter-codex-local build` and `typecheck`: tsc clean against current `master` - `pnpm exec vitest run packages/adapters/codex-local`: 45 passing (incl. 17 `runtime-config` tests: fresh-merge + cleanup restore, root-region placement, same-name provider override, inline tables/arrays, DEL escaping, `{env:}` expansion from run env + `process.env`, per-case malformed-input notes with `config.toml` untouched, skipped-entry notes alongside a successful merge, silent no-op when unset/empty, explicit-`CODEX_HOME` skip note, backup restore of excised user sections after an interrupted run, backup removal on cleanup, stale-block self-heal, re-run replacement) - Verified end-to-end: a codex agent in a hardened Kubernetes (gVisor) sandbox completed a real task routed through an OpenAI-compatible gateway's `/v1/responses`, with a billed usage row recorded on the gateway. That deployment supplies the verification evidence; the mechanism is gateway-agnostic. ## Risks Low. Entirely env-driven and opt-in; with `PAPERCLIP_CODEX_PROVIDERS` unset the adapter never touches `config.toml` and behavior is byte-identical to before. The merge preserves user content, restores the original file on cleanup, and survives interrupted runs via the pre-run backup; malformed input surfaces a visible note and is ignored without touching `config.toml`. No migration/UI impact. ## Model Used Claude Opus 4.8 (`claude-opus-4-8`, 1M context), extended thinking + tool use, via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md (adapter-level opt-in config enabling gateway routing; not the core sandbox-platform work, noted above) - [x] I have searched GitHub for duplicate or related PRs and linked them above (#7837 is the opencode analogue; no codex-local duplicate found) - [x] I have either (a) linked existing issues OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots (n/a, no UI) - [ ] I have updated relevant documentation to reflect my changes (env var documented inline; no central doc references the adapter env yet) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (green on the previous head; re-running on the final note-copy polish commit) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (both review P2s are fixed at head: the interrupted-run restore via the pre-run backup and the U+007F escaping; a re-review is requested for the note-copy polish) - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Devin FoleyandPaperclip
|
c139d6c025 |
fix(codex-local): omit default model so codex CLI picks per auth mode (#7971)
## Thinking Path > - Paperclip orchestrates AI agents through pluggable local adapters; codex_local wraps OpenAI's `codex` CLI. > - The codex_local adapter declares a hard-coded `DEFAULT_CODEX_LOCAL_MODEL = "gpt-5.3-codex"` and multiple Paperclip consumers (UI build-config, server route, OnboardingWizard, NewAgent form, AgentConfigForm) fall back to it when the operator doesn't pick a model. > - That model — and every `*-codex` model plus the older `gpt-5/5.1/5.2` lines — is API-key-only. Codex CLI rejects them on ChatGPT subscription auth with "The 'gpt-5.3-codex' model is not supported when using Codex with a ChatGPT account." > - Every codex_local agent created through the default onboarding path inherits this pin and breaks on its first heartbeat for any user authed via `codex login` (ChatGPT). > - claude_local already takes the right shape: its build-config only sets `adapterConfig.model` when the operator actually picked one, and falls through to whatever default `claude` CLI uses. > - Codex CLI's own default is auth-mode-aware. ChatGPT-subscription accounts get `gpt-5.5`; API-key accounts get the codex-tuned default. A Paperclip-side pin masks this and downgrades whichever group it wasn't built for. > - This PR makes codex_local match claude_local's shape: omit `adapterConfig.model` when the user picks "default," and let the CLI choose. Subscription users stop breaking; API-key users stop getting downgraded. > - The benefit is auth-mode-correct defaults with no Paperclip-side hard pin, plus future-proofing: when OpenAI bumps the CLI default we inherit it for free. ## What Changed - `packages/adapters/codex-local/src/ui/build-config.ts` — only set `adapterConfig.model` when the operator picked one (parity with `packages/adapters/claude-local/src/ui/build-config.ts`). - `server/src/routes/agents.ts` — drop the codex_local-specific `next.model = DEFAULT_CODEX_LOCAL_MODEL` fallback in `applyCreateDefaultsByAdapterType`. Bypass-sandbox default is left in place (security posture, not a model choice). - `ui/src/pages/NewAgent.tsx`, `ui/src/components/AgentConfigForm.tsx`, `ui/src/components/OnboardingWizard.tsx` — stop pre-populating the model field with `DEFAULT_CODEX_LOCAL_MODEL` when the user selects the Codex adapter. Other adapters' defaults (gemini_local, cursor, opencode_local) are unchanged. - `DEFAULT_CODEX_LOCAL_MODEL` is preserved as an exported constant for downstream consumers / plugin authors who want to opt in to a pin; we just stop forcing it on operators who didn't ask for one. - Test: assert `buildCodexLocalConfig` omits `model` when input is blank. ## Verification - `pnpm exec vitest run packages/adapters/codex-local/src/ui/build-config.test.ts packages/adapters/codex-local/src/server/codex-args.test.ts server/src/__tests__/adapter-registry.test.ts server/src/__tests__/heartbeat-model-profile.test.ts server/src/__tests__/agent-permissions-routes.test.ts` → 74/74 passing - `pnpm exec vitest run ui/src/lib/duplicate-agent-payload.test.ts ui/src/lib/acpx-model-filter.test.ts` → passing - `pnpm tsc --noEmit -p .` → clean - Live: I separately verified live during initial investigation that on ChatGPT-subscription auth, `gpt-5.3-codex` is rejected and `gpt-5.5` is what Codex CLI picks by default. Omitting model lets the CLI handle that. ## Risks - Telemetry: any sink that reads `adapterConfig.model` for cost attribution will now see the empty/omitted case more often. The CLI emits the actually-used model in its event stream; downstream telemetry should already read from there for accuracy, but worth a check. - Operator UX: "default" now means "whatever the CLI picks" instead of a Paperclip-known model. The selectable catalog still includes `gpt-5.5`, `gpt-5.4`, `gpt-5.3-codex`, etc. for operators who want to pin explicitly. - Existing agents are unaffected — their `adapterConfig.model` is already set; this only changes the *new-agent* default flow. ## Related work - Depends on: an open catalog-add PR adding `gpt-5.5` to the selectable model list and to `CODEX_LOCAL_FAST_MODE_SUPPORTED_MODELS`. Operators who want to switch to `gpt-5.5` explicitly need that PR merged first; this PR is the structural change that makes "default" mean "let the CLI choose." - Closes #5371 — codex_local default model selection persists `gpt-5.3-codex` instead of adapter default (this PR is the exact fix #5371 proposes). - Related: #5132 (opencode-local: hire-time default model fails on ChatGPT-OAuth accounts) — same problem shape on a sibling adapter; not fixed here but worth tracking for a parallel. - Related: #5939 (codex_local adapter hardcodes `gpt-5.3-codex-spark` validation, fails on ChatGPT OAuth accounts regardless of configured model) — separate validation-path bug; not fixed here. ## Model Used Claude (Sonnet-class), running inside Paperclip as a claude_local executor. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
9a48d92104 |
Add GPT-5.5 to Codex local model options (#5575)
## Related work This PR is the cleanest "add `gpt-5.5` to the codex-local catalog" change open against master. Several other PRs propose the same catalog/fast-mode update; they should close as duplicates once this lands: - #4646 — Add Codex gpt-5.5 model option - #6044 — feat(codex-local): add gpt-5.5 to model catalog, default reasoning to medium, cheap profile xhigh - #6045 — feat(codex-local): add gpt-5.5 to model catalog, default medium reasoning, xhigh cheap profile - #6595 — feat(adapters): add new Codex models (gpt-5.5, gpt-5.4-mini, gpt-5.3-codex, gpt-5.2) Related issues this enables (catalog-level surface area): - #5371 — codex_local default model selection persists `gpt-5.3-codex` instead of adapter default. This PR makes `gpt-5.5` selectable in the dropdown; a separate follow-up changes the *default* behavior so users who don't pick a model are subscription-compatible. - #5132 — opencode-local: hire-time default model fails on ChatGPT-OAuth accounts. Sibling adapter, same problem shape; not fixed here but worth tracking as a parallel for the opencode side. --- ## Thinking Path > - Paperclip orchestrates AI agents through adapter-backed local and remote runtimes. > - The `codex_local` adapter declares built-in model options that feed the server model list and, in turn, the agent configuration UI dropdown. > - GPT-5.5 is available in newer Codex environments but was missing from Paperclip's fallback `codex_local` model list. > - Operators could still type a manual model ID, but the default dropdown made the supported path look unavailable. > - Codex fast mode support is declared separately, so adding GPT-5.5 to the visible list should also include it in the supported fast-mode set. > - This pull request adds GPT-5.5 to the built-in Codex local model options and updates focused tests around argument generation and adapter model listing. > - The benefit is a clearer default setup path for agents using GPT-5.5 without changing existing defaults or migrations. ## What Changed - Added `gpt-5.5` to the `codex_local` fallback model list. - Added `gpt-5.5` to `CODEX_LOCAL_FAST_MODE_SUPPORTED_MODELS`. - Updated Codex argument tests to cover GPT-5.5 fast mode and preserve manual-model fast mode behavior. - Updated adapter model listing tests to assert the Codex fallback list includes GPT-5.5. ## Verification - `pnpm exec vitest run packages/adapters/codex-local/src/server/codex-args.test.ts server/src/__tests__/adapter-models.test.ts` - `git diff --check` - UI note: this is a dropdown data-source change rather than a layout/component change; the adapter model listing test covers the list consumed by the UI. ## Risks - Low risk. This only extends a static fallback model list and fast-mode allowlist. - Existing defaults remain unchanged (`gpt-5.3-codex`). - If a local Codex CLI does not support `gpt-5.5`, selecting it will still fail at execution time the same way any unavailable manual model would. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex desktop coding agent, GPT-5-family model. The exact backing model ID was not exposed by the local runtime; the session used shell, Git, test execution, and GitHub CLI tool access. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: apple <apple@appledeMacBook-Pro.local> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Devin Foley <devin@paperclip.ing> |
||
|
|
c32193c85e |
test(codex-local): cover EEXIST race rejection with mismatched symlink (#5269)
## Thinking Path > - The `codex-local` adapter sets up a per-company Codex home with an auth symlink. Between `lstat` and `symlink` there is a race where two concurrent setups can both try to create the same symlink, surfacing `EEXIST`. > - Master already handles this at runtime via `createExpectedSymlink`, which accepts `EEXIST` only when the raced-in entry resolves to the expected source, and ships a regression test for the tolerated-race path (symlink already points at the right place). > - The symmetric path — `EEXIST` raised by a symlink pointing somewhere else — must stay strictly rejected so a future refactor cannot silently weaken the guard. > - This PR locks that in with a single additive test. No production code change. ## What Changed - Added one regression test in `packages/adapters/codex-local/src/server/codex-home.test.ts` that injects an `EEXIST` whose raced-in symlink target points at a different file, and asserts: - `prepareManagedCodexHome` rejects with `code: "EEXIST"`. - The mismatched symlink is left on disk (we do not blindly overwrite the raced-in entry). Complements the existing "treats a concurrently-created expected auth symlink as success" test already on master. Refs #5240 (Stack B — codex-home adapter session/auth handling). ## Verification - `pnpm --filter @paperclipai/adapter-codex-local exec vitest run src/server/codex-home.test.ts` — passes. - `pnpm --filter @paperclipai/adapter-codex-local typecheck` — clean. ## Risks - Test-only change. No production code is modified. ## Model Used - Provider: Anthropic - Model: Claude (Opus 4.7) - Mode/capabilities: tool-using coding agent with shell execution and test verification ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge - [x] I searched the GitHub PR list for similar PRs and confirmed this is not a duplicate Co-authored-by: Devin Foley <devin@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
c297ba2a80 |
fix(codex-local): replace stale auth.json copy with symlink on prepare (#5028) (#5240)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - codex_local runs Codex CLI under a per-company "managed home" so multiple companies don't trample on each other's session state > - For `auth.json` specifically, the managed home keeps a SYMLINK to the user's real `~/.codex/auth.json` rather than a copy — Codex refresh tokens rotate and are single-use, so any copy goes stale the moment the source rotates and every subsequent run dies with `401 refresh_token_reused` > - Older Paperclip versions copied `auth.json` instead. After upgrading, `ensureSymlink()` saw a regular file at the target, hit `if (!existing.isSymbolicLink()) return;`, and silently kept the stale copy > - This pull request makes the upgrade path self-healing inside `ensureSymlink()` itself: when the target is a regular file, unlink it and create the symlink, since the target lives under the Paperclip-managed home and is safe to delete. Directories are skipped to avoid `EISDIR` on Unix (and inconsistent behavior on Windows) > - The benefit is operators who upgraded from a copy-based version stop getting refresh-token-reused failures without having to manually purge `companies/<id>/codex-home/auth.json`, and the healing is defense-in-depth even outside the `prepareManagedCodexHome` cleanup path ## What Changed - `packages/adapters/codex-local/src/server/codex-home.ts` — `ensureSymlink()` previously bailed out of the `!existing.isSymbolicLink()` branch, leaving any pre-existing regular file untouched. Now unlinks and recreates the symlink in that branch via the existing `createExpectedSymlink()` helper (preserves the EEXIST race-tolerance behavior added in #5119). A guard skips directories so the call never throws `EISDIR` and aborts `prepareManagedCodexHome`. Inline comment explains the safety: target is always under the company-scoped managed home (`<paperclipHome>/instances/<id>/companies/<companyId>/codex-home/`), never the user's real `~/.codex`. - `packages/adapters/codex-local/src/server/codex-home.test.ts` — adds a regression test for #5028: pre-seed a stale copy at the target, run `prepareManagedCodexHome`, assert the target is now a symlink and reads through to the fresh source. The existing concurrent-symlink test is preserved. ## Verification ``` pnpm --filter @paperclipai/adapter-codex-local exec vitest run # Test Files 8 passed (8) # Tests 26 passed (26) pnpm --filter @paperclipai/adapter-codex-local exec tsc --noEmit # clean ``` Manual repro flow that the regression test mirrors: 1. Create a stale copy: `echo '{"token":"old"}' > <managedHome>/auth.json`. 2. Rotate source: `echo '{"token":"new"}' > ~/.codex/auth.json`. 3. Trigger any codex_local run — `prepareManagedCodexHome` is called from the execute path, the managed file is now a symlink to the source, and the CLI sees the fresh token. ## Risks - **Low risk.** The new branch only fires when the target file is a regular file (the upgrade path) — a pure copy that Codex couldn't have written, since Codex never writes into the managed home. Operators in steady-state on the symlink-based version are unaffected. - The `fs.unlink` only runs against the per-company managed-home path, never the user's real `~/.codex`. Inline comment makes this guarantee explicit. - A directory at the auth.json path is left in place (no silent `EISDIR` crash) — this requires operator inspection rather than autonomous deletion. - The healing uses `createExpectedSymlink()` so it remains tolerant of EEXIST races with concurrent prepare calls (the concurrent-symlink test still passes). - No DB / migration / schema impact. ## Model Used - Anthropic Claude Opus 4.7 (claude-opus-4-7), via Claude Code CLI with extended tool use (Read / Edit / Bash / Grep). No extended-thinking budget consumed beyond default. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots — N/A, adapter-only - [x] I have updated relevant documentation to reflect my changes — inline comment explains the why and the safety of the unlink - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge - [x] I searched the GitHub PR list for similar PRs and confirmed this is not a duplicate Fixes #5028. --------- Co-authored-by: Devin Foley <devin@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
9eac727cf1 |
[codex] Add skills CLI and catalog management (#6782)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies through company-scoped control-plane workflows. > - Agents need reusable, inspectable skills that can be installed, reset, audited, exported, and assigned without bespoke local setup. > - The existing skill truth model needed cleanup so bundled skills, optional catalog skills, runtime skills, and adapter-provided skills have clear provenance. > - Operators also need a practical CLI and board UI for discovering and managing company skills. > - This pull request adds the skills CLI, packaged skills catalog, company skills APIs, and catalog-aware board UI. > - The benefit is a more reusable Paperclip company setup where skills are portable, auditable, and easier for operators and agents to manage. ## What Changed - Added `paperclipai skills` CLI commands and coverage for catalog listing, installing, resetting, and inspecting company skills. - Added a packaged `@paperclipai/skills-catalog` workspace with bundled and optional skill content plus validation/build tests. - Added shared company-skill types and validators used across CLI, server, and UI contracts. - Added server catalog APIs/services for company skill catalog operations, reset semantics, audit behavior, and portability provenance. - Updated adapter skill handling so runtime/catalog provenance remains explicit across local adapters. - Added board UI support for browsing and managing catalog-backed company skills. - Updated docs for the skills CLI/catalog flow and the company skills Paperclip skill reference. - Rebased the branch onto current `paperclipai/paperclip:master`; no `pnpm-lock.yaml`, `.github/workflows`, or migration files are included in the final PR diff. ## Verification - Passed: `pnpm run preflight:workspace-links && pnpm exec vitest run cli/src/__tests__/skills.test.ts packages/skills-catalog/src/catalog-builder.test.ts packages/skills-catalog/src/shipped-catalog.test.ts packages/shared/src/validators/company-skill.test.ts packages/adapter-utils/src/server-utils.test.ts packages/plugins/create-paperclip-plugin/src/entrypoints.test.ts server/src/__tests__/company-skills-catalog-service.test.ts server/src/__tests__/company-skills-routes.test.ts server/src/__tests__/company-portability.test.ts`. - Passed: `pnpm exec vitest run server/src/__tests__/workspace-runtime.test.ts -t "default branch|origin/master|symbolic-ref"`. - Attempted: full `server/src/__tests__/workspace-runtime.test.ts`. Four provisioning tests failed while seeding an isolated worktree database from the local Paperclip instance because the local plugin schema dump contains a duplicate-column foreign key (`plugin_content_machine_18a7bc327b.content_case_signals`). The default-branch tests touched by the rebase conflict passed in the focused run above. - Checked final diff: no `pnpm-lock.yaml`, no `.github/workflows`, and no migration-file changes relative to `master`. ## Risks - Medium: this is a broad skills/catalog change touching CLI, server APIs, shared contracts, adapter skill sync, and UI. - Catalog validation and reset semantics need careful reviewer attention because they affect reusable company setup and portability. - No database migrations are included in this PR, so there is no migration ordering/idempotency risk in the final diff. - No lockfile is included by design; dependency resolution will be handled by the repository lockfile workflow. ## Model Used - OpenAI Codex coding agent based on GPT-5, running in Paperclip via the `codex_local` adapter with shell, git, GitHub CLI, and code-editing tool access. Exact hosted model build/context-window metadata is not exposed in this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run targeted tests locally and documented the local workspace-runtime seed failure above - [x] I have added or updated tests where applicable - [x] If this change affects the UI, screenshots were intentionally omitted per PAP-10124 instructions; UI behavior is covered by tests and reviewer inspection - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
38c185fb8b |
[codex] Add agent permissions and controls plan (#6386)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies by keeping task ownership, approvals, and operator control inside one control plane. > - Agent permissions and plugin-hosted company settings sit on the boundary between autonomy and governance. > - V1 needs scoped task assignment rules, plugin extension points, and clearer company access surfaces without weakening company boundaries. > - The branch builds the core authorization service, plugin SDK/host APIs, and UI simplifications needed to support those controls. > - Paperclip EE plugin surfaces were intentionally moved out of this core PR per review direction, so this PR now carries only the public core/plugin infrastructure work. > - The latest updates preserve the PAP-9937 branch changes that belong in this PR, remove the `design/` artifacts, and exclude the experimental `plugin-briefs` package. > - Greptile feedback was applied through the authorization/audit paths and the final cleanup commit was re-reviewed at 5/5 with no unresolved Greptile threads. > - The benefit is safer assignment control with extension hooks for richer permission products while preserving simple defaults for normal operators. ## What Changed - Added scoped task-assignment authorization decisions and routed issue/agent assignment mutations through the authorization service. - Added plugin SDK and host APIs for company settings slots, authorization policy/grant management, assignment previews, and bridge invocation scope propagation. - Simplified core company access UI and moved advanced controls behind plugin-provided settings surfaces. - Added retry-now affordances for blocked issue next-step notices. - Added protected-assignment enforcement for persisted agent/project/issue policies, including explicit-grant fallback behavior. - Added incremental principal-access compatibility backfill for active agent memberships and role-default human permission grants. - Added the Markdown code block wrap action fix from the latest branch changes. - Removed `design/` artifacts from the PR and removed `packages/plugins/plugin-briefs` from the final diff. - Addressed Greptile feedback for plugin actor sanitization, legacy membership handling, audit pagination, unknown grant-scope metadata, and startup test mocks. ## Verification - `pnpm exec vitest run server/src/__tests__/access-service.test.ts server/src/__tests__/company-portability.test.ts` -> 2 files passed, 54 tests passed. - `pnpm exec vitest run server/src/__tests__/server-startup-feedback-export.test.ts server/src/__tests__/access-service.test.ts server/src/__tests__/company-portability.test.ts` -> 3 files passed, 62 tests passed. - `pnpm exec vitest run server/src/__tests__/authorization-service.test.ts server/src/__tests__/plugin-access-authorization-host-services.test.ts server/src/__tests__/server-startup-feedback-export.test.ts` -> 3 files passed, 28 tests passed. - `pnpm --filter @paperclipai/server typecheck` -> passed. - `git diff --check` -> passed. - `node ./scripts/check-docker-deps-stage.mjs` -> passed. - `CI=true pnpm install --frozen-lockfile --ignore-scripts` -> passed with no lockfile update. - `pnpm exec vitest run ui/src/components/MarkdownBody.interaction.test.tsx` -> 1 test passed. - `git ls-files design packages/plugins/plugin-briefs | wc -l` -> 0. - GitHub CI on `40cd83b53` -> all checks passed, merge state `CLEAN`. - Greptile on `40cd83b53` -> 5/5, 102 files reviewed, 0 comments/annotations added, 0 unresolved review threads. - Confirmed the PR diff contains no `design/`, `packages/plugins/plugin-briefs`, `pnpm-lock.yaml`, or `.github/workflows` changes. ## Risks - Medium: task assignment authorization paths are behaviorally stricter for protected/private policy data, so existing plugin-authored policies may block assignment until explicit grants or approval flows are configured. - Medium: plugin-host authorization APIs expand the surface area available to trusted plugins and need careful review for company scoping. - Low: startup now performs a principal-access compatibility backfill, but the migration and runtime backfill use conflict-tolerant inserts. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent, tool-enabled workflow with shell, git, and GitHub CLI access. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
d734bd43d1 |
[codex] Roll up May 17 branch changes (#6210)
## Thinking Path > - Paperclip is the control plane for autonomous AI companies, so agent work needs visible ownership, recovery, and operator controls. > - This local branch had accumulated several related control-plane reliability and operator-experience fixes across recovery actions, watchdog folding, model-profile defaults, mentions, markdown editing, plugin launchers, and small UI polish. > - The branch needed to be converted into a PR against the current `origin/master` without losing dirty work or including lockfile/workflow churn. > - The safest standalone shape is a single rollup PR because the recovery/server/UI files overlap heavily across the local commits and splitting would create avoidable conflicts. > - This pull request replays the local branch onto latest `origin/master`, preserves the uncommitted work as logical commits, and adds a Zod 4 validator compatibility fix found during verification. > - The benefit is that the May 17 local branch can be reviewed and merged as one coherent, conflict-free branch under the 100-file Greptile limit. ## What Changed - Rebased the local May 17 branch work onto current `origin/master` in a dedicated worktree. - Preserved and committed previously dirty changes for recovery retry handling, plugin/sidebar launcher polish, and `.herenow` ignores. - Added recovery-action behavior for returning source issues to `todo` when retrying source-scoped recovery. - Included the existing local recovery/liveness/watchdog fold, Codex cheap-profile, markdown/mention, duplicate-agent, and UI polish commits from the branch. - Normalized shared validator `z.record(...)` schemas to explicit string-key records for Zod 4 compatibility. - Confirmed the PR has no `pnpm-lock.yaml` or `.github/workflows/*` changes and stays below the 100-file Greptile limit. ## Verification - `pnpm install --frozen-lockfile --ignore-scripts` - `npm run install` in `node_modules/.pnpm/sqlite3@5.1.7/node_modules/sqlite3` to build the local native sqlite3 binding after installing with scripts disabled - `pnpm exec vitest run packages/shared/src/validators/issue.test.ts packages/shared/src/project-mentions.test.ts packages/adapter-utils/src/server-utils.test.ts server/src/__tests__/heartbeat-model-profile.test.ts server/src/__tests__/issue-recovery-actions.test.ts server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts server/src/__tests__/heartbeat-active-run-output-watchdog.test.ts server/src/__tests__/plugin-local-folders.test.ts ui/src/components/IssueRecoveryActionCard.test.tsx ui/src/components/Sidebar.test.tsx ui/src/components/SidebarAccountMenu.test.tsx ui/src/components/IssueProperties.test.tsx ui/src/components/MarkdownEditor.test.tsx ui/src/components/MarkdownBody.test.tsx ui/src/lib/duplicate-agent-payload.test.ts ui/src/pages/Routines.test.tsx` - First pass: 13 files passed with 201 passing tests; 3 server files failed before sqlite3 native binding was built. - After rebuilding sqlite3: `server/src/__tests__/heartbeat-model-profile.test.ts`, `server/src/__tests__/issue-recovery-actions.test.ts`, and `server/src/__tests__/heartbeat-active-run-output-watchdog.test.ts` passed/loaded; embedded Postgres tests were skipped by the local host guard. - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck` ## Risks - Medium risk: this is a broad rollup PR across recovery semantics, server tests, shared validators, and UI surfaces. - Some embedded Postgres tests skipped locally due the host guard, so CI should provide the stronger database-backed signal. - UI changes were covered by component tests, but no browser screenshot was captured in this PR creation pass. - This branch may overlap with existing recovery/liveness PR work; merge this PR independently or restack/close overlapping branches rather than merging duplicate implementations together. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5-based coding agent, tool-enabled local repository and GitHub workflow, medium reasoning effort. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
b24c6909e8 |
Harden remote sandbox runtime probes, timeouts, and installs (#5685)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Each agent runs inside a sandbox environment so its CLI is isolated from the host > - Sandbox-backed adapter runs go through a small set of shared helpers — `ensureAdapterExecutionTargetCommandResolvable`, the sandbox callback bridge runner, and per-adapter `SANDBOX_INSTALL_COMMAND` strings > - When standing up new sandbox provider plugins, the existing helpers timed out, missed install fallbacks, or leaned on assumptions that only held for E2B > - Local adapters (`claude-local`, `codex-local`, `gemini-local`, `opencode-local`) needed slightly hardened probes so they could install themselves and validate inside *any* remote sandbox transport, not just E2B > - This pull request bundles those runtime fixes so future sandbox provider plugins inherit a working baseline > - The benefit is that adding a new sandbox provider plugin no longer requires touching adapter-utils or each local-adapter probe — the supporting infra is already correct ## What Changed - `packages/adapter-utils/src/execution-target.ts`: introduce `DEFAULT_REMOTE_SANDBOX_ADAPTER_TIMEOUT_SEC = 1800` and `resolveAdapterExecutionTargetTimeoutSec(...)`. Local and SSH adapters keep the historical "0 means no adapter timeout" behavior; sandbox-backed runs without an explicit `timeoutSec` get an explicit 30-minute default so remote installs and warm-up don't time out at the per-RPC default. Plumbed `timeoutSec` through `ensureAdapterExecutionTargetCommandResolvable` so install probes inside a sandbox honor adapter-level overrides instead of the bridge's 5-minute default. - `packages/adapters/opencode-local/src/index.ts`: switch `SANDBOX_INSTALL_COMMAND` from `npm install -g opencode-ai` to `curl -fsSL https://opencode.ai/install | bash`. The npm package reifies four large prebuilt-binary subpackages in parallel even though only one matches the host arch; on bandwidth-constrained sandboxes that blew through the 240s install budget. The official installer fetches one arch-specific binary and adds `$HOME/.opencode/bin` to PATH via `~/.bashrc`, which the sandbox-callback-bridge login-shell script already sources. - `packages/adapters/{claude,codex,gemini,opencode}-local/`: harden remote-target probes — pass `--skip-git-repo-check` for Codex when probing outside a repo, normalize permission flags for Claude, and add `*.remote.test.ts` coverage that exercises the remote-sandbox path explicitly for each adapter. - `packages/adapter-utils/src/sandbox-install-command.{ts,test.ts}` (new): add `buildSandboxNpmInstallCommand` helper. `server/src/adapters/registry.ts` + new `server/src/__tests__/adapter-registry.test.ts`: wire adapter install commands so they fall back to a writable `$HOME/.local` prefix when global install isn't available. - `server/src/__tests__/plugin-worker-manager.test.ts` + new `server/src/__tests__/fixtures/plugin-worker-delayed.cjs`: pin per-call timeout overrides so plugin worker exec calls honor the caller's timeout instead of the worker's default. ## Verification - `pnpm typecheck` - `pnpm exec vitest run --no-coverage packages/adapter-utils/src/execution-target-sandbox.test.ts packages/adapter-utils/src/sandbox-install-command.test.ts` - `pnpm exec vitest run --no-coverage server/src/__tests__/plugin-worker-manager.test.ts server/src/__tests__/adapter-registry.test.ts server/src/__tests__/claude-local-adapter-environment.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/gemini-local-adapter-environment.test.ts` - `pnpm exec vitest run --no-coverage packages/adapters/codex-local/src/server/test.remote.test.ts packages/adapters/opencode-local/src/server/test.remote.test.ts packages/adapters/codex-local/src/server/codex-args.test.ts packages/adapters/codex-local/src/server/execute.remote.test.ts packages/adapters/gemini-local/src/server/execute.remote.test.ts` All passing locally. ## Risks - Touches shared `adapter-utils` and several `*-local` adapters. The 30-minute default applies only when both (a) the target is `remote+sandbox` and (b) no `timeoutSec` is configured — local + SSH paths are unchanged. New test coverage was added alongside each behavior change to pin the contracts. - Switching OpenCode's install command to the official installer is a behavior change for any operator running OpenCode inside a remote sandbox. Local installs are unaffected (the `SANDBOX_INSTALL_COMMAND` only runs when an adapter is being installed inside a sandbox). - Low risk overall — no migrations, no API surface change. ## Model Used - Provider: Anthropic - Model: Claude Opus 4.7 (1M context) - Capabilities used: extended reasoning, tool use (Read/Edit/Bash/Grep), no code execution beyond local repo commands ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots — N/A, no UI change - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
DottaandPaperclip
|
0096b56a1c |
[codex] Add LLM Wiki plugin host support (#5597)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - The plugin system needs host contracts and runtime support before large plugins can integrate cleanly. > - The source branch mixed the LLM Wiki package with supporting host/runtime work, managed plugin skills, root-level storage spaces, and a bookmarks reference plugin. > - [PAP-9173](/PAP/issues/PAP-9173) asked for the current branch to be split by file boundary: plugin package separately from everything else. > - [PAP-9188](/PAP/issues/PAP-9188) clarified that LLM Wiki may have plugin-local spaces, but Paperclip core should not reorganize top-level local storage into spaces. > - Follow-up review clarified that the bookmarks example should not ship in this PR either. > - This pull request contains the non-`packages/plugins/plugin-llm-wiki/` host/runtime work, keeps runtime state under the selected Paperclip instance root, and no longer includes the bookmarks example. ## What Changed - Added/updated plugin host contracts, SDK types, worker RPC plumbing, managed plugin skill support, and related server tests. - Removed the bookmarks example plugin package and its bundled-example/workspace references. - Removed the root-level local spaces CLI/migration surface and restored instance-root runtime defaults for config, db, logs, storage, secrets, workspaces, projects, and adapter homes. - Replaced shared root `space-paths` helpers with `home-paths` helpers for core runtime storage. - Tightened stranded recovery unique-conflict detection so concurrent recovery scans reuse the raced recovery issue when Postgres errors are wrapped. - Kept `packages/plugins/plugin-llm-wiki/` out of this PR diff; plugin-local spaces remain in the stacked plugin-only PR. ## Verification - `pnpm exec vitest run cli/src/__tests__/data-dir.test.ts cli/src/__tests__/home-paths.test.ts cli/src/__tests__/onboard.test.ts packages/shared/src/home-paths.test.ts packages/db/src/runtime-config.test.ts server/src/__tests__/agent-instructions-service.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/codex-local-execute.test.ts` - `pnpm exec vitest run packages/db/src/runtime-config.test.ts` - `pnpm exec vitest run server/src/__tests__/plugin-routes-authz.test.ts` - `pnpm --filter @paperclipai/server typecheck` - `pnpm exec vitest run server/src/__tests__/heartbeat-process-recovery.test.ts -t "reuses the raced stranded recovery issue"` skipped locally because embedded Postgres did not initialize on this macOS temp host; the code path was typechecked and is covered by Linux CI. - Boundary check: no core references remain for `PAPERCLIP_SPACE_ID`, `spaces migrate-default`, `@paperclipai/shared/space-paths`, `registerSpacesCommands`, or the removed bookmarks example. - Previous PR head `4f23e034` had green GitHub checks: `verify`, all four serialized server shards, `e2e`, `Canary Dry Run`, `policy`, Snyk, and `Greptile Review`. Current head `582f466d` is re-running checks after the bookmarks deletion. ## Risks - Plugin host changes touch shared runtime paths, so regressions would most likely appear in adapter startup, plugin loading, or local dev path defaults. - Removing the bookmarks example also removes one demonstration of plugin database namespaces plus local-folder persistence; remaining plugin examples still cover bundled example discovery and plugin host flows. - The plugin package itself is intentionally deferred to the stacked plugin-only PR, where LLM Wiki plugin-local spaces live. - Existing installs that tested the transient root-level spaces CLI should stop using it; this PR intentionally removes that unsupported migration surface before merge. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-5 Codex via Codex CLI, tool use and local code execution enabled; context window not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass, except where noted above for host-specific embedded Postgres initialization - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge Stacked follow-up: PR #5592 contains only `packages/plugins/plugin-llm-wiki/` and targets this branch. --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
fe3904f434 |
Stabilize runtime probes and Codex env tests (#5445)
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies
> - Adapters expose a Test action that probes the configured runtime —
install, resolvability, hello — to give operators a fast yes/no on
whether an environment is healthy
> - The Codex test path was running its hello probe directly without
going through the managed-runtime preparation that production runs use,
so a healthy production setup could still report a probe failure
> - The plugin worker manager wasn't surfacing terminated workers
cleanly, leaving the runtime probe waiting on a dead worker until the
request timed out
> - This pull request routes the Codex test probe through
`prepareAdapterExecutionTargetRuntime` (so it sees the same managed
Codex home production sees), exposes `commandCwd` on
`createCommandManagedRuntimeClient` so callers can target a per-probe
directory without leaking the workspace `remoteCwd`, and propagates
plugin-worker termination as a usable error instead of a hang
> - The benefit is the Codex Test action mirrors production behavior
end-to-end, and probes against a terminated plugin worker fail fast
instead of timing out
## What Changed
- `packages/adapter-utils/src/command-managed-runtime.ts`: rename the
`remoteCwd` knob to `commandCwd` so callers can target a per-probe
directory without inheriting the workspace cwd; matching test coverage
in `command-managed-runtime.test.ts`
- `packages/adapter-utils/src/sandbox-callback-bridge.{ts,test.ts}`:
small fixes to keep callback bridge stop semantics deterministic
- `packages/adapters/codex-local/src/server/test.ts`: thread the Codex
hello probe through `prepareAdapterExecutionTargetRuntime` +
`prepareManagedCodexHome` so the probe sees the same managed home
production sees; new `test.remote.test.ts` covers the remote probe path
- `packages/adapters/cursor-local/src/server/execute.ts`: small
probe-side cleanup that aligns with the new commandCwd contract
- `server/src/services/plugin-worker-manager.ts`: surface plugin-worker
termination as a structured error so callers fail fast; new
`plugin-worker-terminated.cjs` fixture and
`plugin-worker-manager.test.ts` cases pin the behavior
## Verification
- `pnpm vitest run --no-coverage --project @paperclipai/adapter-utils
--project @paperclipai/adapter-codex-local --project
@paperclipai/adapter-cursor-local --project @paperclipai/server` —
1749/1750 passing (1 unrelated skip)
- `pnpm typecheck` clean
## Risks
Low–medium. The `remoteCwd → commandCwd` rename is a parameter renaming
on an internal helper used only by adapter test/execute paths in this
repo. The plugin-worker-terminated path was previously a hang; failing
fast may surface latent timeouts as explicit termination errors in
callers that already expected them.
## Model Used
Claude Opus 4.7 (1M context)
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable — new tests cover
commandCwd, plugin-worker termination, and Codex remote test path
- [x] If this change affects the UI, I have included before/after
screenshots — N/A (no UI)
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
---
> **Stacked PR.** Sits on top of #5444 which adds the per-run runtime
API surface this PR builds on. Cumulative diff against `master` includes
that PR's content; the files touched by *this* PR's commit are listed
under "What Changed" above. Will rebase onto `master` and force-push
once #5444 merges.
|
||
|
Devin Foley
|
12cb7b40fd |
Harden remote workspace sync and restore flows (#5444)
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies
> - When an agent runs against a remote target, Paperclip syncs the
workspace out to the remote at run start and restores changes back to
the local workspace at run end
> - The previous restore flow naïvely overwrote local files with
whatever the remote returned, so files that the remote run never touched
but had timestamp/mode drift could be needlessly rewritten — and a
single static `refs/paperclip/ssh-sync/imported` ref made concurrent SSH
workspace exports race on the same git ref
> - This pull request adds a `workspace-restore-merge` module that diffs
a pre-run snapshot against the post-run remote state and only writes
back files the remote actually changed; SSH workspace exports now use a
per-import unique ref so concurrent runs can't trample each other
> - Every adapter's execute path threads the snapshot through
`prepareAdapterExecutionTargetRuntime` so the merge has the baseline it
needs
> - The benefit is workspace restores no longer churn untouched files,
and concurrent SSH runs no longer collide on the import ref
## What Changed
- `packages/adapter-utils/src/workspace-restore-merge.{ts,test.ts}`: new
module — directory snapshot (kind/mode/sha256/symlink target) plus
snapshot-aware merge that writes only the files the remote changed
- `packages/adapter-utils/src/ssh.ts`: SSH workspace export uses a
per-import unique ref (`refs/paperclip/ssh-sync/imported/<uuid>`);
restore goes through the new merge helper; `ssh-fixture.test.ts` covers
the unique-ref + merge paths
- `packages/adapter-utils/src/sandbox-managed-runtime.ts` +
`remote-managed-runtime.ts`: thread the snapshot/merge through the
sandbox and SSH paths
- `packages/adapter-utils/src/server-utils.{ts,test.ts}` +
`execution-target.ts`: helpers for capturing the pre-run snapshot;
`prepareAdapterExecutionTargetRuntime` gains required `runId` and
optional `workspaceRemoteDir`, and returns the realized
`workspaceRemoteDir`
- Each adapter's `execute.ts` (acpx, claude, codex, cursor, gemini,
opencode, pi) takes the snapshot at run start and passes it through to
the runtime restore
- Remote execute test mocks updated to match the new
`prepareWorkspaceForSshExecution` return shape and the per-run
`${managedRemoteWorkspace}` cwd subdirectory
## Verification
- `pnpm vitest run --no-coverage --project @paperclipai/adapter-utils
--project @paperclipai/adapter-acpx-local --project
@paperclipai/adapter-claude-local --project
@paperclipai/adapter-codex-local --project
@paperclipai/adapter-cursor-local --project
@paperclipai/adapter-gemini-local --project
@paperclipai/adapter-opencode-local --project
@paperclipai/adapter-pi-local` — 196/196 passing
- `pnpm typecheck` clean across the workspace
## Risks
Medium. The restore path now writes a strict subset of what it
previously did — files the remote did not touch are no longer rewritten.
If any flow was relying on a touch-without-content-change being copied
back (timestamp or permission propagation only), that behavior is now
skipped. Snapshot capture adds an O(N-files-in-workspace) hash pass at
run start; the cost is bounded by the existing exclude list. The `runId`
parameter on `prepareAdapterExecutionTargetRuntime` is now required —
every in-tree caller is updated; out-of-tree adapter authors need to
pass it.
## Model Used
Claude Opus 4.7 (1M context)
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable — new module +
every adapter execute path covered
- [x] If this change affects the UI, I have included before/after
screenshots — N/A (no UI)
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
|
||
|
DottaandPaperclip
|
a1b30c9f35 |
Add planning mode for issue work (#5353)
## Thinking Path > - Paperclip is a control plane for autonomous AI companies. > - Issues are the core unit of work, and issue comments are how board users and agents coordinate execution. > - Some issue conversations need to produce plans and approvals instead of immediate implementation work. > - The existing issue contract did not distinguish standard execution comments from planning-oriented issue work. > - This pull request adds an issue work-mode contract and board UI affordances for standard vs planning mode. > - The benefit is that planning-mode issues can be created, displayed, discussed, and carried through agent heartbeat context without losing the normal issue workflow. ## What Changed - Added `standard` / `planning` issue work-mode contracts across DB, shared validators/types, server issue flows, plugin protocol, and adapter heartbeat payloads. - Added an idempotent `0081_optimal_dormammu` migration for `issues.work_mode`, ordered after current `public-gh/master` migrations. - Updated heartbeat/context summaries and issue-thread interaction behavior so planning work mode is preserved when creating suggested follow-up issues. - Added UI support for planning-mode issue creation, issue rows, detail composer styling, and composer work-mode toggles. - Added focused server/shared/UI tests plus a Playwright visual verification spec for planning-mode surfaces. - Rebased the branch onto current `public-gh/master` and added durable planning-mode screenshots under `doc/assets/pap-3368/`. ## Verification - `pnpm --filter @paperclipai/db run check:migrations` - `pnpm exec vitest run --project @paperclipai/shared packages/shared/src/validators/issue.test.ts` - `pnpm exec vitest run --project @paperclipai/server server/src/__tests__/heartbeat-context-summary.test.ts server/src/__tests__/issue-thread-interactions-service.test.ts server/src/__tests__/issues-goal-context-routes.test.ts --pool=forks --poolOptions.forks.isolate=true` - `pnpm exec vitest run --project @paperclipai/ui ui/src/components/IssueChatThread.test.tsx ui/src/components/NewIssueDialog.test.tsx ui/src/components/IssueRow.test.tsx ui/src/pages/IssueDetail.test.tsx` - `pnpm exec vitest run --project @paperclipai/adapter-utils packages/adapter-utils/src/server-utils.test.ts` - `PAPERCLIP_E2E_SKIP_LLM=true npx playwright test --config tests/e2e/playwright.config.ts tests/e2e/planning-mode-visual-verification.spec.ts` ## Screenshots Desktop planning detail:  Desktop planning row:  Desktop staged standard toggle:  Mobile planning detail:  Mobile planning row:  ## Risks - Medium migration risk: this adds a non-null issue column. The migration uses `ADD COLUMN IF NOT EXISTS` so installations that applied an older branch-local migration number can still apply the final numbered migration safely. - Medium contract risk: issue payloads, plugin payloads, and adapter heartbeat payloads now include work mode; compatibility is handled by defaulting missing values to `standard`. - UI risk is moderate because composer controls changed; focused component tests and visual e2e coverage exercise standard vs planning display and toggle behavior. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent in a local Paperclip worktree, with shell/tool use. Exact context-window size is not exposed in this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
9578dc3da7 |
Wire per-adapter sandbox install commands through test and execute paths (#5280)
> **Stacked PR.** Sits on top of the e2b sandbox chain — #5278 (stdin staging) and #5279 (honest-resolvability + login-profiles). The cumulative diff against `master` includes both of those PRs' content; the files touched by *this* PR's commit are the new `maybeRunSandboxInstallCommand` helper in `packages/adapter-utils/src/execution-target.ts` and the per-adapter `index.ts`/`server/test.ts`/`server/execute.ts` wiring under `packages/adapters/{claude,codex,cursor,gemini,opencode,pi}-local/`. The honest resolvability check from #5279 is what gives this PR's install command a meaningful "did it actually land on PATH" follow-up. ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Sandbox execution targets are ephemeral — each fresh lease starts from a template image that may or may not have the agent CLIs preinstalled > - When a CLI isn't preinstalled, the resolvability probe fails at `command -v` and the hello probe never runs > - There's no shared mechanism for "before you probe or provision, install the CLI on this sandbox" > - This pull request adds a `SANDBOX_INSTALL_COMMAND` constant per adapter and a `maybeRunSandboxInstallCommand` helper that runs it via the existing sandbox login shell, captures structured output, and never throws (so the resolvability + hello probe still run after); each adapter's `test()` and `execute()` share the constant so the two callsites can't drift > - The benefit is a fresh sandbox lease without a preinstalled CLI now installs it once via `sh -lc` before the resolvability probe and before managed-runtime provisioning, with a uniform `<adapter>_install_command_run` check on the test report ## What Changed - `packages/adapter-utils/src/execution-target.ts`: add `AdapterSandboxInstallCommandCheck` and `maybeRunSandboxInstallCommand` (runs the install via existing sandbox shell, captures exit/stdout/stderr, returns a structured info/warn check, never throws) - Add `SANDBOX_INSTALL_COMMAND` to each adapter's `index.ts` so `test()` and `execute()` share a single source of truth - Wire each of the 6 affected adapter `testEnvironment()`s to call `maybeRunSandboxInstallCommand` before `ensureAdapterExecutionTargetCommandResolvable` - Pass `installCommand: SANDBOX_INSTALL_COMMAND` through `prepareAdapterExecutionTargetRuntime` in each adapter's `execute()` - Per-adapter install commands use npm globals where possible so binaries land on a PATH segment the template already exports: - claude → `npm install -g @anthropic-ai/claude-code` - codex → `npm install -g @openai/codex` - cursor → `curl https://cursor.com/install -fsS | bash` - gemini → `npm install -g @google/gemini-cli` - opencode → `npm install -g opencode-ai` - pi → `npm install -g @mariozechner/pi-coding-agent` SSH and local targets ignore `installCommand` (SSH runtime takes no such param; local short-circuits before runtime prep), so this is a no-op for non-sandbox environments. ## Verification - `pnpm typecheck` clean - `pnpm vitest run --no-coverage --project @paperclipai/adapter-utils` and per-adapter projects pass - Manual sandbox matrix (claude, codex, cursor, gemini, opencode, pi) — each goes `install_command_run → resolvable → hello_probe_passed` (Codex and Pi land on `hello_probe_auth_required`, which is the configured-credentials problem, not an install issue) - SSH no-regression: SSH Claude still passes; the helper short-circuits on non-sandbox targets ## Risks Medium — adds a network/CPU cost (npm install / curl) on every fresh sandbox lease. Cost is bounded (one-time per lease, typically tens of seconds for npm globals), and the helper never throws so a failing install still lets the report run resolvability and hello probes. If a sandbox image already has the CLI, the install is an idempotent reinstall. ## Model Used Claude Opus 4.7 (1M context) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots — N/A (no UI) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Devin Foley
|
9042b8d042 |
Write apikey-mode auth.json so Codex CLI 0.122+ can authenticate via OPENAI_API_KEY (#5276)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - The Codex adapter spawns the OpenAI Codex CLI to drive the model > - Codex CLI 0.122 changed how it reads credentials: it ignores `OPENAI_API_KEY` from the environment and reads only `$CODEX_HOME/auth.json` > - Without auth.json, Codex 0.122+ returns 401 "Missing bearer or basic authentication" on `/v1/responses` even when `OPENAI_API_KEY` is forwarded into the sandbox or remote shell > - This pull request materializes an apikey-mode `auth.json` in the managed Codex home (or per-run for the test probe) when an `OPENAI_API_KEY` is configured > - The benefit is configured Codex API keys authenticate correctly with current Codex CLI versions across local, SSH, and sandbox targets ## What Changed - `codex-home.ts`: add `writeApiKeyAuthJson()` and let `prepareManagedCodexHome` accept an `apiKey` override that replaces the symlinked host auth.json with an apikey-mode file - `execute.ts`: pass `envConfig.OPENAI_API_KEY` into `prepareManagedCodexHome` so the managed (and synced-to-remote) Codex home authenticates via the configured key - `test.ts`: when `OPENAI_API_KEY` is available, wrap the hello probe with a small shell that materializes a per-run `$CODEX_HOME/auth.json` before exec'ing codex; key content rides through env to avoid leaking into process listings - Update the `codex_hello_probe_auth_required` hint to explain Codex CLI does not read `OPENAI_API_KEY` from env ## Verification - `pnpm vitest run --no-coverage --project @paperclipai/adapter-codex-local` - `pnpm typecheck` clean - Manual: Codex 0.122.0 with empty `CODEX_HOME` returns 401 with env-only auth; with this change it authenticates cleanly ## Risks Low risk — when no API key is configured, behavior is unchanged (no auth.json written, existing chatgpt-mode flow preserved). Apikey-mode `auth.json` is the upstream-supported format. ## Model Used Claude Opus 4.7 (1M context) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots — N/A (no UI) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Devin Foley
|
90631b09b3 |
Let adapters declare runtime command spec for remote provisioning (#5141)
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies, running
adapter
> commands like `claude`, `codex`, `pi` either locally or on remote
runtimes
> (SSH hosts, sandboxes, etc.)
> - On a fresh remote runtime — particularly an ephemeral sandbox — the
> adapter's CLI may not be installed yet. Today operators handle this
via
> external configuration (e.g. a project-level `provisionCommand` shell
> script) that has to know about every adapter the operator might want
to use
> - This means every adapter has its own well-known npm package, but
operators
> end up writing duplicate provision shell scripts that paste together
> `npm install -g @anthropic-ai/claude-code`, `npm install -g
@openai/codex`,
> etc. — knowledge the adapter itself already has
> - This PR moves that knowledge into the adapter modules: each adapter
declares
> how its runtime command should be detected and (if applicable)
installed
> via `getRuntimeCommandSpec(config)`. The execution path runs the
adapter's
> own install command on remote sandbox targets before launching, so a
fresh
> sandbox bootstraps itself instead of requiring a hand-written
provision script
> - The benefit is fewer footguns for operators provisioning remote
runtimes,
> and a clean place for new adapters to plug in their install recipe
## What Changed
- New types in `packages/adapter-utils/src/types.ts`:
- `AdapterRuntimeCommandSpec` describing `command`, optional
`detectCommand`, and optional `installCommand`
- Optional `getRuntimeCommandSpec(config)` on `ServerAdapterModule`
- Optional `runtimeCommandSpec` on `AdapterExecutionContext` so adapters
receive the resolved spec at execute time
- New helper `ensureAdapterExecutionTargetRuntimeCommandInstalled(...)`
in
`packages/adapter-utils/src/execution-target.ts` that runs the install
command
on remote targets when `transport === "sandbox"`. SSH and local targets
are
no-ops. Throws on timeout or non-zero exit so failures surface early.
- Each of `claude-local`, `codex-local`, `cursor-local`, `gemini-local`,
`opencode-local`, `pi-local`'s `execute.ts` now reads
`ctx.runtimeCommandSpec?.installCommand` and calls the helper before
launching
the adapter command.
- `server/src/adapters/registry.ts` declares `getRuntimeCommandSpec` for
each
adapter:
- claude/codex/gemini/opencode/pi-local: `npm install -g <package>`
recipe via
a shared `buildNpmRuntimeCommandSpec` helper, with a defensive guard
that
only auto-installs when the configured `command` matches the well-known
fallback (custom binaries are left alone).
- cursor-local: declares `command` only; no auto-install (no public npm
package), preserving the existing manual setup.
- `server/src/services/heartbeat.ts` resolves the spec via
`adapter.getRuntimeCommandSpec?.(runtimeConfig)` and passes it through
to
`AdapterExecutionContext`.
- Tests added in `execution-target.test.ts` (~75 lines), e2b
`plugin.test.ts` (~32 lines), and `environment-run-orchestrator.test.ts`
(~76 lines).
## Verification
- `pnpm --filter @paperclipai/adapter-utils test`
- `pnpm --filter @paperclipai/server test --
environment-run-orchestrator`
- `pnpm --filter @paperclipai/sandbox-providers-e2b test`
- Manual QA: run an adapter (claude/codex/etc.) against a fresh
sandbox-backed
environment that does NOT have the adapter CLI pre-installed. Confirm
the
install runs once at the start of the agent run and the adapter then
launches
successfully. Re-run on the same sandbox; confirm the install command is
idempotent and the second run starts faster.
- Confirm SSH and local execution paths are unaffected (gated by
`transport === "sandbox"`).
## Risks
- Behavioural shift on sandbox runs: a new install step now runs at the
start
of every sandbox agent run for adapters with `installCommand` set. The
install commands are idempotent (`if ! command -v X >/dev/null 2>&1;
then
npm install -g <pkg>; fi`), so this is fast on warm sandboxes. On a cold
sandbox, the first run takes longer.
- Operators who used the legacy project-level `provisionCommand` to
install
adapter CLIs can drop that part of their script; the adapter handles it
now.
Existing scripts continue to work — installs are idempotent.
- The cursor-local adapter has no auto-install (no public npm package).
Behaviour for cursor-local on sandboxes is unchanged.
- New optional surface on `ServerAdapterModule`. Plugins that don't
implement
`getRuntimeCommandSpec` retain previous behaviour (no auto-install).
## Model Used
- OpenAI GPT-5.4 (reasoning effort: high) via Codex CLI
- Provider: OpenAI
- Used to author the code changes in this PR
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots — N/A
- [ ] I have updated relevant documentation to reflect my changes — N/A
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
|
