PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-04 19:28:46 +01:00

Author	SHA1	Message	Date
Devin FoleyandPaperclip	c4c4f7eecb	perf(environments): reuse provider schemas after content validation (#17581 ) Add a separate schema-only provider lookup with current plugin identity and SHA-256 content validation in one statement. Retain bounded schema definitions, return independent objects, and preserve current full driver and secret access paths. Build, repository typecheck, 485 affected tests, the supported-host database tests, and the complete required CI matrix passed. Greptile scored this exact head 5/5 at medium impact with no unresolved findings. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-25 03:20:45 -07:00
DottaandPaperclip	dabb8c5daf	test: remove repeated setup and strengthen regression proof (#16022 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its tests protect company boundaries, approvals, durable tasks, credentials and the board UI. > - Several owner suites repeat large database fixtures and render frames. > - Some tests repeat stronger proof or pass when the behavior they describe is broken. > - This pull request shares exact setup, removes proven duplicate cases and repairs weak observations. > - It preserves distinct security, lifecycle, concurrency and user-facing contracts. > - The benefit is 5,027 fewer test lines and stronger regression checks. ## Linked Issues or Issue Description Refs #15978. This continues the merged test-audit cleanup in [PR #15978](https://github.com/paperclipai/paperclip/pull/15978), which removed 11,976 test/support lines. The savings in this pull request are additional. Related open work: [PR #13460](https://github.com/paperclipai/paperclip/pull/13460) changes legacy OAuth parent/version lock ordering. This pull request retains those production paths and the existing lock-order test. Its secrets changes compact fixtures and repair the two AWS wrapper exclusion observations. Scoped title searches found no open duplicate audit or fixture-consolidation PR and no matching audit issue. The problem is described here using the enhancement template. What existing behavior does this improve? The maintenance and regression value of the existing server, UI, CLI, shared and wiki-plugin test suites. Subsystem affected Cross-cutting test code. Two UI modules also lose unused test inspection exports. Current behavior Owner tests repeat database rows, service inputs, HTTP setup and render frames. Some cases repeat stronger assertions. Two AWS lock tests record only cleanup events, so an AWS provider write can happen inside the held lock without failing those observations. Several other negative tests accept or do not observe the wrong result. Worktree fixtures also use a temporary path alias that conflicts with the existing canonical-path checks on macOS. Proposed behavior Use small local helpers that preserve exact inputs, fresh allocations, callbacks, query order and await positions. Remove only cases with named stronger keepers. Tighten negative assertions and observe actual AWS provider entry while a confirmed lock holder remains parked. Resolve temporary fixture roots to their canonical paths. Reason and benefit Remove 5,027 net test lines and 15 redundant registered executions. Remove 11 lines of unused UI inspection exports. Keep every distinct company, actor, approval, credential, persistence, stale-write and concurrency contract. No wall-time speedup is claimed for setup compaction. Breaking changes None for application behavior, APIs or stored data. The removed exports expose private UI test state and have no remaining callers. ## What Changed - Consolidate repeated runtime, workspace, portability, route, tool-access, policy, gateway and state-service fixtures. - Share repeated UI render frames and CLI/wiki-plugin setup while retaining the original actions and expected results. - Remove 15 repeated case executions only where the audit records an exact stronger keeper. - Repair weak HTTP/UI negative assertions and two AWS account-home lock observations. Keep the existing durable results and cleanup behavior. - Remove the unused `__inflightGetCount` and `__secondTickInternals` inspection exports. Keep observable request coalescing and ticker lifecycle checks. - Canonicalize temporary worktree fixture paths without weakening production path checks. - Record the owner decisions, savings, independent preservation reviews and validation in [the dated audit report](https://github.com/paperclipai/paperclip/blob/58c5c4aa241c85f4665a3159c192888bbb2a5cfa/doc/plans/2026-10-12-test-audit-rounds.md) and [the compact control evidence](https://github.com/paperclipai/paperclip/blob/58c5c4aa241c85f4665a3159c192888bbb2a5cfa/doc/plans/2026-10-12-test-audit-controls.json). ## Verification - Complete owner and retained-sibling baselines passed before each audit cutover. Independent reviewers checked the full original tests, actual helper bodies, exact inputs, receipt provenance, async order, fresh allocations and case registrations. - All 14 UI/route/wiki mutations failed at their intended assertions: ten UI/API/ticker controls, one route control and three wiki persistence/receipt controls. Each mutated production source was restored to its exact original bytes. - Both old AWS lock tests passed with their respective AWS-only lock bypass. Both repaired tests failed at the intended assertions with the same bypasses. These give 16 intended failures and two passing weak controls across 18 executions. Each source was restored exactly. - Final complete state-owner/sibling run: 329/329 passed at code revision `64370831ebe191e5ce092f85f79b6fb661df7e16`. - Final `pnpm -r typecheck`, `pnpm check:token-gates` and `pnpm build`: passed at the same code revision. - Delivery head `58c5c4aa241c85f4665a3159c192888bbb2a5cfa` includes the evidence commit and master `4b766b872993045a2e787404f0475503683e5ded`. All 27 audited test files and both application files match the locally verified code revision byte-for-byte. Current-head CI validates the integrated branch, including the incoming master changes. - All eight production-owner SHA guards for the 16 deliberate failing mutation controls still match the integrated delivery checkout. The mutation receipts therefore apply to those current owners. - Full default `pnpm test:run`: failed with exit 1 at frozen code revision `64370831ebe191e5ce092f85f79b6fb661df7e16`; HEAD and tracked source bytes remained unchanged. The first general-server group reported 878 passing files, four failing files and six skipped files; 17,846 passing cases, 24 failing cases and 105 skipped cases. Later groups did not run. Failures were `http-log-redaction` (socket hangup), `public-mcp` (initial 15-second timeout followed by hosted token “Refresh before subscribing” failures), `remove-test-home` (macOS `ENOTEMPTY` where permission errors were expected, plus cleanup failure), and `workspace-git-snapshot-streaming` (300-second timeout). All four failing test files and the relevant cleanup/MCP production owners are unchanged by this audit. Full local success is not claimed. - Focused complete-owner diagnosis at the same frozen revision: 149/152 passed, three failed, zero skipped. `public-mcp` passed 82/82 and `http-log-redaction` passed 66/66 unchanged. The two `remove-test-home` cases reproduced macOS `ENOTEMPTY` against an expected `EACCES`/`EPERM` contract; the 32 MiB workspace snapshot case reproduced its existing 300,000 ms timeout. HEAD and tracked files stayed clean. Exact blob comparison confirmed all four test files and the relevant cleanup/MCP owners match both inspected master `4b766b872993045a2e787404f0475503683e5ded` and the frozen local revision. The three remaining local failures are unresolved; no timeout, retry or assertion was weakened. - Current-head CI: [workflow 38299262149](https://github.com/paperclipai/paperclip/actions/runs/38299262149) has passed runner selection and created all downstream test/typecheck/build gates; those gates are queued, with no failures reported. Greptile completed successfully at `58c5c4aa241c85f4665a3159c192888bbb2a5cfa` with [5/5 and zero comments](https://github.com/paperclipai/paperclip/pull/16022#issuecomment-6131457592). All review threads are resolved or absent. This pull request will remain open after verification. Reviewers can reproduce the required workspace checks: ```sh pnpm -r typecheck pnpm check:token-gates pnpm test:run pnpm build ``` Browser suites were not run. Their sources and user journeys are unchanged by this audit. ## Risks - A fixture helper can hide a specialized input or change object freshness. Full-program expansion checks and independent reviews preserve the exact old inputs and execution order. - Removing duplicate proof can remove a distinct contract by mistake. The audit records the keeper for each deletion; all distinct security and lifecycle cases remain. - Concurrency checks can miss progress during a fixed delay. The AWS repairs record actual provider entry, confirm holder entry, use measured entry windows, require exact final event order and drain both operations in `finally`. - The full local default run failed. Focused diagnosis reproduced two macOS cleanup failures and one existing snapshot timeout in unchanged owners; these remain unresolved. Current-head CI is pending. No test deadline, retry policy, skip or CI shard was weakened to obtain a pass. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex with GPT-6, reasoning, repository inspection, code execution and delegated audit/review. The runtime does not expose the exact served model identifier or context-window size; those values are unavailable and are not inferred from the model catalog. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-12 22:23:11 +00:00
Dotta	0c1e7504c0	fix(runner): persist warm Daytona workspaces across turns (#12904 ) ## Thinking Path > - Daytona preserves a stopped sandbox filesystem, but deleting or replacing a sandbox removes its only remote copy. > - Warm reuse therefore improves latency but cannot be Paperclip's durability boundary. > - The host execution workspace must remain authoritative after every successful turn, while same-run recovery must avoid overwriting unexported remote work. > - Result proposal, workspace export/merge, and terminal completion need a durable, replayable ordering so a crash never starts a duplicate provider turn. > - A paid browser acceptance suite must exercise both legacy Codex and Runner Codex for three real turns on one continuously warm Daytona sandbox. ## Linked Issues or Issue Description Refs #12901. Runner Codex did not previously export successful Daytona workspace changes back to the authoritative host workspace. That made warm reuse depend on Daytona's remote filesystem and left deleted/replacement sandboxes without a reliable reconstruction path. The existing paid fixture also lacked a focused three-turn continuity case for both Codex adapters. ## What Changed - Persist versioned, atomic native workspace-sync descriptors and durable seeds in `PAPERCLIP_HOME`, without credentials or a database migration. - Classify fresh, warm, replacement, and same-run-recovery workspace preparation explicitly; ambiguous lease/root/digest evidence fails closed. - Finalize native workspace export/merge after semantic result proposal and before run completion, with idempotent replay that never submits a second provider turn. - Surface legacy Codex workspace restoration failures instead of masking them, while preserving an earlier provider error when both fail. - Keep healthy reusable Daytona leases warm for legacy and native adapters, stamp finalized workspace generations, and retain existing cleanup behavior for per-turn or unhealthy leases. - Preserve Runner Codex's provider process/session across warm turns, including bounded post-terminal tail draining and exact authority rotation. - Add the exact paid `daytona-warm-continuity` matrix: - `legacy-codex × daytona × warm-three-turn` - `runner-codex × daytona × warm-three-turn` - Drive all three turns through the browser, verify ordered file continuity and stable lease/workspace/runtime identities, capture per-turn timings, and delete the sandbox immediately after assertions. - Document `pnpm test:e2e:runner -- --suite daytona-warm-continuity`; no package script was added. ## Verification - `pnpm typecheck` — passed, including migration safety (no migration added) - Focused server/runner Vitest coverage — 144 passed - `pnpm test:e2e:runner:unit` — 114 passed - `pnpm test:e2e:runner:typecheck` — passed - `pnpm --filter @paperclipai/paperclip-runner test:codex` — 66 passed, 1 helper ignored - `native-session-executor.test.ts` — 139 passed, including safe fail-closed cleanup after remote runner identity capture failure - Paid local browser acceptance, exact post-rebase Linux/amd64 runner binary: - Runner Codex — passed in 1.7m; 3 runs; lease outcomes `created, resumed, resumed`; 10/10 matchers; cleanup passed - Legacy Codex — passed in 2.7m; 3 runs; lease outcomes `created, resumed, resumed`; 10/10 matchers; cleanup passed - [Protected paid GitHub Actions campaign](https://github.com/paperclipai/paperclip/actions/runs/34026735033) against `7da42a91b95fa7fb2df126668ef7e37afb3b2b9d` — passed 2/2: - Runner Codex — 3 runs; lease outcomes `created, resumed, resumed`; evidence and cleanup passed - Legacy Codex — 3 runs; lease outcomes `created, resumed, resumed`; evidence and cleanup passed - Merge/enforcement, S3 history, and Pages publication jobs passed - Paid result artifacts were scanned for both provider credentials; neither secret was present. - Current PR checks — 31 passed, 1 expected Storybook skip; Greptile 5/5; Superagent security scan passed - `git diff --check origin/master...HEAD` — passed - Confirmed no `package.json`, lockfile, migration, or SQL changes. ## Risks - Workspace synchronization now sits on the terminal-success path, so a remote export failure deliberately prevents false success. Retryable state retains its lease/seed; loss of the only unexported remote copy fails closed. - Warm provider reuse has strict identity and quiescence checks. Mismatched or ambiguous evidence blocks reuse rather than risking concurrent provider work. - The paid suite incurs Daytona and Codex cost only in the existing protected scheduled/manual workflow and explicitly destroys its sandbox after each cell. ## Model Used OpenAI Codex with GPT-5 agentic reasoning, repository inspection, real browser E2E execution, Rust/TypeScript test execution, and GitHub Actions diagnostics. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked an existing issue or described the issue in-PR - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have documented the dedicated suite invocation without adding a package script - [x] I have considered and documented risks above - [x] All Paperclip CI gates are green on the current revision - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups on the current revision - [x] I will address all reviewer comments before requesting merge	2026-09-06 08:16:12 -05:00
Devin Foley	c62bb4b16b	feat: environment delete with agent reassignment and consented sandbox destroy (#12053 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environments define where agent runs execute: local, SSH, or provider sandboxes > - Operators can create and edit environments, but the UI has no way to delete one > - The server already exposes `DELETE /environments/:id` and a delete-blast-radius preflight, but no UI consumes them, and a delete blocked by reusable sandbox leases gives the operator no path forward > - This pull request adds the delete flow to the environment configuration page: a preflight-driven modal that reassigns dependent agents, names the workspaces that hold blocking sandbox leases, and can destroy those sandboxes with explicit consent > - The benefit is that operators can retire stale environments from the UI without database surgery, and dependent agents move to a chosen replacement instead of silently falling back ## Linked Issues or Issue Description Refs #8554 Refs #11124 Subsystem affected Environments (server routes, environment runtime service, and the environment settings UI). Problem or motivation The environment configuration page has no delete control. The server delete endpoint exists, but nothing in the UI calls it. When reusable sandbox leases block a delete, the 409 error names no owner, so the operator cannot find the blocking workspace. Agents that use the environment as their default lose it silently through the FK `on delete set null`. Proposed solution Add a delete button with a confirmation modal on the environment edit page. The modal reads the delete-blast-radius preflight. It offers a dropdown to reassign dependent agents to another environment before the delete. It lists each workspace that holds a blocking reusable sandbox lease, with a link. When those leases are the only blocker, the confirm button destroys the sandboxes inline (`?destroyReusableSandboxLeases=true`) and then deletes. A failed teardown falls back to `pending_cleanup` for the sweep, so no sandbox is orphaned. ## What Changed - `ui/src/pages/CompanyEnvironments.tsx`: delete button on the edit page header, confirmation modal with agent reassignment select, lease-holder list, impact notes, and a consent-labeled destroy-and-delete action - `ui/src/api/environments.ts`: `deleteBlastRadius` and `remove` client methods; `remove` takes an optional `destroyReusableSandboxLeases` flag - `server/src/routes/environments.ts`: `DELETE /environments/:id` accepts `?destroyReusableSandboxLeases=true`; it destroys the environment's reusable sandbox leases first, but only when those leases are the sole delete blocker, then re-checks the blast radius before it deletes - `server/src/services/environment-runtime.ts`: new `destroyReusableSandboxLeasesForEnvironment` — destroys every reusable sandbox lease an environment still owns while the environment config (provider credentials) is still available - `server/src/services/environments.ts`: the delete blast radius now returns `reusableSandboxLeaseHolders` (lease id, workspace, issue) so clients can name what blocks a delete - `packages/shared/src/types/environment.ts`: `EnvironmentDeleteReusableLeaseHolder` type on the blast radius - Tests: route gating for the consent flag (destroy runs, mixed-blocker rejection, surviving-lease rejection), runtime destroy scoped to an environment, blast-radius holder join, and UI tests for the reassignment flow, holder links, and the consent button ## Verification - `npx vitest run server/src/__tests__/environment-routes.test.ts server/src/__tests__/environment-service.test.ts server/src/__tests__/environment-runtime.test.ts ui/src/pages/CompanyEnvironments.test.tsx` - Manual: open Settings → Environments → edit an environment. The trash icon opens the modal. With agents on the environment, pick a reassignment target and confirm; agents move and the environment deletes. With reusable sandbox leases, the modal names the holding workspaces and the confirm button reads "Destroy N sandboxes and delete". ## Risks - The consented path destroys provider sandboxes. It runs only when reusable leases are the sole blocker, so a delete that would still be rejected never destroys anything. A failed teardown routes to `pending_cleanup` and the delete stays blocked until the sweep resolves it. - Agent reassignment issues one PATCH per agent from the client. A mid-sequence failure leaves some agents reassigned; the reassignments are valid on their own and the UI refreshes to the actual state. - Hard blockers (managed local, instance default, pending cleanup) keep the existing 409 behavior and disable the confirm button. ## Model Used - Claude (Anthropic) — Fable 5, model id `claude-fable-5`, extended thinking, agentic tool use via Claude Code CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-23 16:53:17 -07:00
Nicky LeachandPaperclip	c1c46f1e4e	feat: Claude login on the new-agent page before agent creation (#11347 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The Claude local adapter supports subscription login through a sandbox > - The new-agent page must show login before the user creates an agent > - Test results must not expose raw sandbox diagnostics or secret values > - This pull request adds the login UI to both Test lanes and closes the diagnostic boundary > - The branch also adds durable cleanup recovery for failed sandbox teardown > - Reusable sandboxes must retain both their recorded teardown configuration and a valid lifecycle path until destruction succeeds > - The benefit is a usable login flow with fixed public checks, redacted server logs, and recoverable sandbox cleanup ## Linked Issues or Issue Description Related public work: [#9488](https://github.com/paperclipai/paperclip/pull/9488) adds first-class recognition for `CLAUDE_CODE_OAUTH_TOKEN` in headless and remote runs. Related public issue: [#2681](https://github.com/paperclipai/paperclip/issues/2681) requests Claude Code subscription support. This pull request adds the login transport and new-agent UI flow that those changes do not provide. Subsystem affected: Claude local adapter, server login probes, sandbox provider setup, cleanup recovery, and the new-agent UI. Problem or motivation: The Test lanes did not show the sandbox login panel in all supported cases. Test results also exposed raw probe diagnostics, and JSON escapes could end secret redaction early. Proposed solution: Surface the login capability through the bundled provider manifest. Prepare the same probe runtime in the ACP lane. Send diagnostics only to redacted server logs. Keep Test checks on fixed public messages. Normalize login URL hints to allowlisted HTTPS Claude and Anthropic hosts. Consume JSON escapes during redaction. Preserve failed sandbox cleanup state across retries and restarts, and prevent deletion from severing the lifecycle context of a live reusable sandbox. Alternatives considered: Keep raw diagnostics in Test checks or trust login URL text from the sandbox. Both choices increase information exposure. Keep separate probe behavior in the ACP lane. That choice would leave the two Test lanes inconsistent. ## What Changed - Surface the sandbox login panel on both Test lanes. - Reconcile the bundled Daytona plugin manifest so `supportsSetupTokenLogin` reaches the UI capability gate. - Prepare the ACP Test lane with the same probe runtime as the CLI Test lane. - Add the `claude_acp_login_probe_unavailable` warning when the ACP probe cannot run. - Send raw sandbox diagnostics only to redacted server logs. - Keep Test checks on fixed public messages in the ACP, managed-config, and CLI paths. - Normalize login URL hints to allowlisted HTTPS Claude and Anthropic hosts. - Redact JSON and escaped-JSON secret values, including escaped quotes and backslashes. - Preserve orphan cleanup records across provider failures, restarts, and unavailable plugins. - Atomically block environment deletion while a live reusable sandbox lease still depends on it. - Verify pending cleanup destroys plugin sandboxes with the provider configuration recorded on the lease, even after the current environment configuration changes. ## Verification - Head under review: `506b7fa2d83c36bfa5fd722ee9d95b0c7431c241`. - Focused environment route/service/runtime coverage passes: 196 tests across 3 files. - `pnpm -r typecheck` passes. - `pnpm build` passes. - The full Vitest run completed with 4,754 passing and 28 failing tests. All 23 source-test failures reproduce unchanged on parent head `58cfe61a33191ce03d965d65085d26064b4888ba`; the other 5 are duplicate executions from stale `server/dist` output. The failures are unrelated macOS path/listener and scheduler-fixture failures, so there is no new bad commit for bisect to localize. - All required CI checks pass for the current head, including build, typecheck/release registry, all server and workspace shards, serialized server suites, canary, and e2e. - A fresh Greptile review for `506b7fa2d83c36bfa5fd722ee9d95b0c7431c241` reports 5/5, “safe to merge,” with no blocking failure remaining. ## Risks - A probe or redaction change could hide useful server diagnostics. - An allowlist change could reject a valid Claude login URL. - Cleanup recovery changes could affect provider teardown ordering. - An environment with a live reusable sandbox can no longer be deleted until the owning issue or execution workspace completes teardown. - The implementation keeps public Test messages fixed and sends detail to redacted server logs. ## Model Used OpenAI GPT-5 via Codex — exact model ID: GPT-5; tool use and code execution enabled; extended reasoning enabled. The implementation author used AI-assisted development. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and documented the result - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation or confirmed no separate documentation change is needed - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-17 13:42:51 -07:00
Nicky LeachandPaperclip	3061ce6901	feat(sandbox): stream session output by capability, drop three operator flags (#11557 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandboxed agents use provider capabilities to select safe execution paths > - Session output still depends on three operator flags that duplicate capability data > - Duplicate flags can drift from the verified sandbox capability snapshot > - This pull request makes the capability snapshot the only streaming decision and removes the obsolete flags > - The benefit is default streaming with a poll fallback when a capability or stream fails ## Linked Issues or Issue Description What existing behavior does this improve? ACP sandbox session-output streaming and sandbox execution configuration. Subsystem affected Cross-cutting (multiple of the above): server/, packages/shared/, packages/adapter-utils/, and packages/plugins/. Current behavior Session-output streaming requires operator flags in the server and Daytona plugin configuration. Saved configurations can retain a removed key. Proposed behavior The verified capability snapshot selects streaming. The Daytona plugin uses persistent sessions by default, keeps bypass commands one-shot, and falls back from the log stream to polling. Removed configuration keys become inert. Reason and benefit One capability source prevents configuration drift. The fallback keeps output available when capability resolution or log streaming fails. Breaking changes The three operator flags no longer control session-output streaming. Existing saved keys load but have no effect. ## What Changed - Remove `useSessions` and `useLogStream` from the Daytona plugin configuration and manifest. - Remove `streamAgentSessionOutput` from server configuration, shared types, and execution-target plumbing. - Select streaming from `persistentProcessSessions` and `independentControlCommands`. - Keep poll fallback on capability resolution failure and stream failure. - Strip removed keys from strict fake-sandbox and catchall plugin configuration. - Update the sandbox capability documentation and focused tests. ## Verification - `tsc --noEmit` passed in `packages/shared`, `packages/adapter-utils`, `server`, and the Daytona plugin. - Daytona `plugin.test.ts` passed 139 tests. - Server capability, configuration, route, and runtime suites passed 160 tests. - `packages/adapter-utils` `execution-target-sandbox.test.ts` passed 44 tests. - The capability matrix covers stream, poll, and resolution-failure paths. - Removed-key tests cover strict fake-sandbox and catchall plugin schemas. ## Risks - A capability snapshot that lacks either required session capability uses polling. - A log stream failure uses polling and can increase request count. - Existing removed configuration keys no longer change behavior. - The isolated-worktree Daytona Vitest run has a pre-existing missing `packages/adapters/droid-local` reference. CI and standard checkouts use the committed configuration. ## Model Used OpenAI Codex, GPT-5, tool use and code review assistance. The exact runtime context window is managed by the Codex platform. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-17 13:25:30 -07:00
Nicky LeachandPaperclip	e71ce9a9d3	feat: sandbox provider capability contract with fail-closed effective resolution (#11463 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip runs work through adapters and sandbox providers > - Providers need a clear contract so the server can use only verified capabilities > - A declared capability must not grant a method that the live worker did not verify > - This pull request adds manifest declarations and fail-closed effective capability resolution > - The benefit is safe provider reuse across execution targets and run lifecycles ## Linked Issues or Issue Description Subsystem affected Cross-cutting (multiple of the above) Problem or motivation Sandbox providers expose different runtime methods. The server needs one safe capability contract that accounts for provider declarations, worker verification, and narrowing configuration. Proposed solution Add strict manifest validation for five sandbox capabilities. Resolve effective capabilities as the subset of verified, declared, and narrowed values. Store the result as a frozen execution-target snapshot. Alternatives considered Trusting the manifest alone could grant methods that the worker does not support. Trusting only a fixed built-in list would reject valid third-party providers. The intersection rule keeps the verified runtime ceiling and supports both provider types. Roadmap alignment This change supports the ACP run lifecycle track and the sandbox provider contract work in the current roadmap. Additional context The legacy `supportsReusableLeases` field remains supported. The nested capability validator rejects unknown keys. Missing or unavailable verification resolves all capabilities to `false`. ## What Changed - Add strict `sandboxCapabilities` manifest validation with legacy reusable-lease compatibility. - Carry declarations through the ready-driver projection. - Add fail-closed effective resolution from verified, declared, and narrowed capabilities. - Add narrowing for provider configuration, Kubernetes Job leases, and Daytona sessions. - Add a frozen read-only capability snapshot to execution targets. - Add focused tests and keep existing characterization baselines covered. - Add and update sandbox provider capability documentation. ## Verification - `npx vitest run packages/shared/src/validators/plugin.test.ts` - `npx vitest run server/src/__tests__/plugin-environment-driver-sandbox-capabilities.test.ts` - `npx vitest run server/src/__tests__/sandbox-capability-contract.test.ts` - `npx vitest run server/src/__tests__/environment-execution-target-capabilities.test.ts` - `npx vitest run packages/adapter-utils/src/acpx-engine/startup-characterization.test.ts packages/adapter-utils/src/acpx-engine/turn-characterization.test.ts packages/adapter-utils/src/acpx-engine/settlement-characterization.test.ts packages/adapter-utils/src/acpx-engine/composed-run-characterization.test.ts` - Package typechecks for shared, server, and adapter-utils pass. - Stage-2 security review suites pass with 28 tests. ## Risks The resolver fails closed when verification is absent or unavailable. Providers that rely on undeclared capabilities may see narrower behavior until they expose verified worker methods. The change does not alter the existing native-sync guard. ## Model Used OpenAI Codex, GPT-5, exact runtime model ID `gpt-5`, tool use and code execution. The implementation author used this model to assist with the change. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-17 10:55:40 -07:00
Nicky LeachandPaperclip	e52b8a343f	fix: ACP run lifecycle corrections — failure settlement, workspace sync-back, lease cleanup (#11454 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent adapters run ACP sessions and manage runtime, workspace, and lease resources. > - Several failure paths left runtime bridges, staged workspaces, or environment leases active after an error. > - These leaks reduce run reliability and can leave later runs without clean resources. > - This pull request closes the failure paths, applies one teardown policy, and adds regression tests. > - The benefit is consistent failure settlement and safer reuse of agent workspaces and leases. ## Linked Issues or Issue Description What happened? ACP runs could leave runtime bridges, staged workspaces, or environment leases active after failures. Claude and Gemini ACP runs did not restore the sandbox workspace on teardown. Lease release stopped when one lease returned an error. Expected behavior Each ACP failure must return an error result and settle its resources. Teardown must run each step, release leases independently, and restore the host workspace when the sandbox ends. Pending cleanup leases must receive bounded retry attempts. Steps to reproduce 1. Run an ACP session that fails after runtime creation or during turn preparation. 2. Run an ACP session that fails during a warm hit or staged runtime handoff. 3. Run lease cleanup with more than one lease when the first release returns an error. 4. Inspect the result phase, teardown calls, workspace state, and lease metadata. 5. Run the regression suites listed in the Verification section. ## What Changed - Settle every ACP failure after runtime creation with an error result and one sandbox.startup span closure. - Close the ACP runtime and remove warm entries after every pre-turn failure. - Run all teardown steps, record teardown errors, release staging leases in finally, and prevent duplicate teardown. - Dispose staged runtimes after seam failures and remove borrowed staged entries with identity guards. - Add fail-open workspace sync-back teardown for Claude and Gemini ACP adapters. - Isolate lease release errors and add bounded retry sweeps for stranded pending_cleanup leases. - Atomically claim pending_cleanup retries and clamp attempt readers to keep the five-attempt bound. - Default absent provider reusableLeases values to false and align the fake provider with its runtime declaration. - Add regression tests for engine, adapter, server, and shared environment behavior. ## Verification - [x] `npx vitest run packages/adapter-utils/src/acpx-engine/execute.test.ts` — 124 tests passed. - [x] `npx vitest run packages/adapters/codex-local/src/server/acp.test.ts packages/adapters/claude-local/src/server/acp.test.ts packages/adapters/gemini-local/src/server/acp.test.ts` — 61 tests passed. - [x] `npx vitest run server/src/__tests__/environment-runtime.test.ts server/src/__tests__/heartbeat-pending-cleanup-sweep.test.ts server/src/__tests__/reusable-leases-default.test.ts server/src/__tests__/environment-routes.test.ts packages/shared/src/environment-support.test.ts` — passed. - [x] All listed suites ran from the repository root. - [x] GitHub CI completed successfully for `cfc349c9f232711433897915112a1c52c0e462ca`. - [x] Greptile completed with a 5/5 confidence score and no blocking finding. ## Risks The engine changes affect failure settlement and teardown order across ACP runs. The server changes add retry state to existing lease metadata without a schema migration. The adapter changes restore workspaces after sandbox execution. Regression tests cover the changed paths. GitHub CI and Greptile passed for the current head. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. This change fixes runtime reliability and does not duplicate a roadmap feature. ## Model Used OpenAI GPT-5 Codex. The model used tool-based repository inspection, GitHub operations, and code review support. The runtime does not expose a context-window value. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (for example, `docs/...` or `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-15 18:52:08 -07:00
Devin Foley	ff5fd62d07	Resolve the environment secret companyId context on first save (#11291 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environments give agent runs an execution target, and their env vars can carry secrets, so writes need a company scope for secret bindings > - `PATCH /environments/:id` resolves that scope from a query param, existing bindings, or a single-membership actor — and fails closed otherwise > - A fresh environment has no bindings yet, so an admin whose memberships cannot pin one company gets a 422 on the very first env var save and can never create the first binding > - #11200 made this reachable from the UI: it opened the envVars-only edit on the managed sandbox row, but the UI never sends the company it already knows > - This pull request sends the page's selected company on environment updates and adds a server fallback to the instance's only company > - The benefit is that the first env var save works on fresh environments, while multi-company ambiguity still fails closed ## Linked Issues or Issue Description Refs #11200 What happened? On a fresh platform-managed sandbox environment, the first "Save environment variables" in the UI fails with HTTP 422: "Environment secret management requires a companyId context during the instance-scoped transition." The same failure hits any environment update that touches `envVars` or `config` when the environment has no secret bindings yet and the actor's memberships do not name exactly one company (for example an instance admin provisioned without a membership row, or a user in two companies). Expected behavior The save succeeds. The client tells the server which company scopes the secret bindings, and on a single-company instance the server can resolve the only possible scope by itself. Steps to reproduce 1. Provision a platform-managed sandbox environment (managed-config `environments` entry) so the row exists with no secret bindings. 2. Sign in as an instance admin whose memberships do not resolve to exactly one company. 3. Open Environments, edit the managed row, add an env var, and save. 4. The PATCH returns 422 with the companyId-context error. ## What Changed - `ui/src/api/environments.ts`: `environmentsApi.update` accepts an optional `companyId` and sends it as the `companyId` query param the server already reads. Renamed the `customImageCompanyQuery` helper to `companyIdQuery` since it now serves plain updates and probes too. - `ui/src/pages/CompanyEnvironments.tsx`: both the managed envVars-only save and the general environment save pass `selectedCompanyId`. - `server/src/routes/environments.ts`: `resolveEnvironmentSecretContextCompanyId` falls back to the instance's only company when exactly one exists, mirroring the existing fallback in `resolveCustomImageCompanyId`. Multi-company instances still fail closed. - Tests: three new route tests (explicit query context for a multi-company actor, single-company-instance fallback for a membership-less admin, fail-closed regression on a multi-company instance), and updated the SSH-probe and probe-ambiguity tests for the new fallback. ## Verification - `cd server && pnpm vitest run src/__tests__/environment-routes.test.ts` — 78 pass. - `cd ui && pnpm vitest run src/pages/CompanyEnvironments.test.tsx` — 22 pass. - Full `server` and `ui` vitest suites and `tsc --noEmit` on both packages pass locally. - Manual: on a managed instance, edit the managed sandbox environment, add an env var, save. Before: 422. After: the save persists and the PATCH carries `?companyId=`. ## Risks - Low risk. The explicit query param path already existed on the server; the UI now uses it. - Behavioral shift: on single-company instances, environment secret-context resolution (including `POST /environments/:id/probe`) now resolves the only company instead of returning no context. That lets probes resolve secret-backed config where they previously returned a 422 asking for an explicit companyId. Multi-company instances keep the fail-closed behavior, covered by a regression test. - Self-hosted single-company instances gain the same first-save fix; no schema or config changes. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-12 11:32:15 -07:00
Devin Foley	0044fa8904	Let tenants edit env vars on managed sandbox environments; add managed-sandbox-only mode (#11200 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environments give each agent run an execution target: the local host, SSH, or a sandbox provider > - A managed deployment can provision one platform-managed sandbox environment through the `PAPERCLIP_MANAGED_CONFIG` `environments` section > - That row is fully locked today. A tenant cannot add environment variables for their agents. There is also no way to hide local execution — run selection falls back to the local row > - A platform that manages the sandbox for its tenants needs both: the tenant adds env vars (and nothing else), and local execution is neither visible nor reachable > - This pull request opens exactly one tenant edit (env vars) on the managed sandbox row, and adds an `enableManagedSandboxOnly` mode that hides local and makes run selection fail closed > - The benefit is a complete managed-sandbox experience with no change for self-hosted instances ## Linked Issues or Issue Description Subsystem affected Environments (managed sandbox provisioning, environment routes, run environment selection) and the environments UI. Problem or motivation Platform-provisioned sandbox environments (`metadata.managedByPaperclip`) reject every write on cloud-managed instances. Agents often need environment variables inside their sandbox. The tenant has no way to set them on the managed row. Separately, an operator cannot remove local execution: the environment list always shows the local row, and run selection falls back to it when no default is set. Proposed solution Allow an envVars-only PATCH on the managed sandbox row, and echo those env vars back for editing. Add a managed-tier feature (`enableManagedSandboxOnly`) that hides the local environment from all read surfaces and redirects local-landing run selection to the managed sandbox environment, failing closed when it is unavailable. Alternatives considered UI-only hiding of the local row. This was rejected: it does not stop a run from resolving to local, so it is presentation without enforcement. Full unlock of the managed row was also rejected: name, driver, and config stay platform-owned so boot reconciliation cannot fight tenant edits. ## What Changed - `server/src/routes/environments.ts`: the platform-provisioned write floor admits an envVars-only PATCH on the generalized managed sandbox row (sandbox driver, `managedByPaperclip`, not legacy kubernetes-marker rows). Name, driver, config, status, metadata, and DELETE stay rejected. The read floor stops blanking env vars on that row; credential-shaped config keys stay redacted for every actor. Legacy kubernetes-marker rows keep the full floor. - Same file: under `enableManagedSandboxOnly`, the environments list and the by-id read omit the local row for every actor, including instance admins. - `server/src/services/execution-workspace-policy.ts`: `resolveExecutionWorkspaceEnvironmentId` gains the managed-sandbox-only inputs. A selection that lands on the local environment is redirected to the managed sandbox environment. With no active managed row it throws `ManagedSandboxUnavailableError` — never local. Non-local selections (ssh, user-created sandboxes) are untouched. - `server/src/services/heartbeat.ts`: the run path reads the flag, looks up the managed row (`findManagedSandboxEnvironment`, new read-only finder in `environments.ts`), and passes both to the resolver. Mirrors the forced-kubernetes precedent, which keeps precedence when both regimes are on. - `server/src/services/managed-environments.ts`: after a successful reconcile, the instance default environment moves to the managed sandbox row when the current default is unset, local, or dangling. A tenant-chosen custom environment is never overridden. - `packages/shared`: new `enableManagedSandboxOnly` key (schema default false, catalog tier `managed`, cloudDefault false, selfHostedDefault false) and the matching interface field. - UI: managed rows show a "Managed by Paperclip" lock badge; editing one opens a dedicated env-vars-only editor that sends the one PATCH shape the server admits (the old full form failed with a 403 on save). New `ui/src/lib/managed-sandbox-environment.ts` mirrors the local filter for cached lists (applied in the project picker; the agent picker already excluded local). The experimental settings page gains the toggle at its alphabetical card position. `environmentsApi.update` now declares the `envVars` field it already sent. - Tests: environment route floor coverage (envVars-only accepted, mixed bodies rejected, legacy rows still blanked and locked, local hidden and 404 under the flag, self-hosted unchanged), an embedded-postgres service test pinning that boot reconciliation never touches tenant env vars, resolver redirect/fail-closed cases, managed-environments default-stamping cases, and UI lib/settings tests. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/environment-routes.test.ts src/__tests__/environment-service.test.ts src/__tests__/execution-workspace-policy.test.ts src/services/managed-environments.test.ts` — all pass. - `pnpm --filter @paperclipai/shared exec vitest run` — 425 pass (catalog/schema default parity is pinned by an existing test). - `pnpm --filter @paperclipai/ui exec tsc --noEmit` and the affected UI suites (CompanyEnvironments, InstanceExperimentalSettings incl. card-order test, new lib test) — all pass. - Full workspace `pnpm test`: 3,414 passed. 17 files report failures on this machine; the identical 17 fail on a clean `origin/master` worktree in the same environment (git-worktree/skills/embedded-postgres environment dependencies and plugin-SDK zero-test collections). One additional file (`issue-monitor-scheduler.test.ts`) failed one timing-sensitive test in one of two full-suite runs and passes 7/7 in isolation on this branch — a flake in a domain this diff does not touch. The branch introduces no new failures. - Self-hosted zero-delta: every new behavior is gated on the cloud-managed instance check or the new flag, which defaults to false in schema and catalog; pinned by the "does not floor platform-marked rows on self-hosted instances" and flag-off tests. ## Risks - Behavior is opt-in twice over: the write-floor exception applies only to rows the managed-config provisioner stamps, and the hiding/forcing applies only when `enableManagedSandboxOnly` is on (default false everywhere). Self-hosted instances see no change. - The env-vars echo is scoped to the generalized managed sandbox row; legacy kubernetes-marker rows keep the blanket floor because pre-generalization builds may have written platform values there. - Fail-closed run selection means a managed instance with the flag on and an archived managed row (provider plugin down) refuses runs with a precise error instead of running locally. That is the intended posture. ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use via Claude Code CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-08-11 11:40:57 -07:00
Nicky LeachandPaperclip	6b7e0814a0	feat(acp): stream Daytona sandbox agent output and remove the host output poll (#11049 ) ## Thinking Path > - Paperclip is the open source app that manages AI agents for work > - Sandbox providers let agents run in remote and isolated environments > - Daytona session commands need a path that sends agent output to the host without host polling > - Host polling adds delay and repeats provider output work > - This pull request adds typed execute.log notifications and a log sink for incremental output > - This pull request adds an optional ACP session stream with final-result replay protection > - The benefit is lower output delay while the default flags keep current behavior unchanged ## Linked Issues or Issue Description Subsystem affected Cross-cutting. The change spans the plugin SDK, Daytona provider, adapter utilities, and server execution services. Problem or motivation The Daytona ACP bridge polls a host output file while an agent command runs. This adds delay and can repeat work. The host also needs a safe route for provider output chunks. Proposed solution Add a typed `execute.log` notification with host-issued invocation correlation. Add an ordered log sink to the environment execute path. Add an optional ACP session-log path that parses newline-delimited JSON frames and removes the host output poll for that path. Alternatives considered Keep the output-file poll as the only path. This keeps the current behavior but does not provide timely output. The new path stays behind flags, so the existing path remains the default fallback. Roadmap alignment This change supports the shipped Cloud / Sandbox agents milestone in `ROADMAP.md`, including Daytona support. ## What Changed - Add the typed `execute.log` worker-to-host notification and company-scoped host route. - Add ordered `stdout` and `stderr` chunk delivery before the final execute result. - Add the Daytona session log sink and the optional ACP streamed session path. - Add monotonic frame handling so live and final output reach the host once. - Keep `useLogStream` and `streamAgentSessionOutput` off by default. - Add unit and integration coverage for the notification, execution target, runtime, and Daytona paths. ## Verification - Run adapter-utils tests: 445 tests pass locally. - Run server environment tests: 73 tests pass locally. - Run Daytona plugin tests: 131 tests pass locally. - Run TypeScript checks for shared, adapter-utils, and server. - Review the pull request checks after GitHub completes them. - All required GitHub checks pass on the current head. ## Risks The new paths change output delivery only when a feature flag enables them. The final execute result remains available for parsing and fallback. The main risk is a provider stream or frame-order error; the final-result parser limits that risk. ## Model Used OpenAI Codex, GPT-5, tool use and code execution. The runtime did not supply a context-window value. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes No operator documentation change applies because both new flags remain disabled by default. - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-07 14:29:58 -07:00
Devin Foley	90ead239a8	feat(ui/server): name cross-company environment secret refs instead of calling them missing (#10577 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environment configs (sandbox providers, SSH) can bind stored company secrets through `format: "secret-ref"` fields, picked in the environment editor's secret picker > - Environments are instance-scoped and shared by every company on an instance, but the picker lists only the current company's secrets, so a ref pointing at another company's secret renders as "Missing secret (…)" in destructive styling > - That state is indistinguishable from a genuinely deleted secret, so operators "fix" a healthy binding by creating a duplicate secret in their own company — the exact sequence that used to corrupt bindings before #10576 > - This pull request adds an instance-gated metadata endpoint for an environment's secret refs and teaches the picker to name a cross-company secret and its owner honestly > - The benefit is that operators can tell a healthy cross-company binding from a broken one, and stop creating duplicate secrets ## Linked Issues or Issue Description Is your feature request related to a problem? Please describe. In the environment editor, a secret-ref field that points at a secret owned by a different company shows "Missing secret (22095402…)" in red, with "The previously selected secret is no longer available. Pick another or remove the binding." The binding is actually healthy — the current company's picker just cannot list the other company's secrets. Operators react by creating a duplicate secret and re-pointing the field. Describe the solution you'd like The editor should know the referenced secret's name, status, and owning company (metadata only, never the value) and present a cross-company ref neutrally, a deleted secret as deleted, and only an unknown id as missing. Related: #10576 (fixes the binding corruption this UI state used to trigger). ## What Changed - New `GET /environments/:id/secret-refs` returns `{ refs: [{ configPath, secretId, name, status, companyId, companyName }] }` for the environment's config-derived secret refs. Values are never returned. The route sits behind `assertCanAccessInstanceEnvironments`, the same gate as environment editing. - New `secretService.describeSecretRefs` loads that metadata across companies; unknown ids are omitted. - `SecretBindingPicker` reads an optional `SecretRefHintsContext` (keyed by secret id). With a hint, a ref the company list cannot show renders as `NAME — Owning Company` with neutral styling and the note "Owned by the … company. The binding keeps working; selecting a secret from this list re-points it here." A hint with `status: "deleted"` reports the secret as deleted. Without hints, behavior is byte-identical to before — agent editors and other picker users are unaffected. - `CompanyEnvironments` fetches descriptors for the environment being edited and provides them through the context. ## Verification - `cd server && pnpm vitest run src/__tests__/environment-routes.test.ts src/__tests__/secrets-service.test.ts` — new endpoint happy path, agent 403 (descriptors never computed), and embedded-Postgres coverage proving cross-company names resolve and unknown ids drop out. - `cd ui && pnpm vitest run src/components/SecretBindingPicker.test.tsx src/components/JsonSchemaForm.test.tsx src/pages/CompanyEnvironments.test.tsx` — hinted cross-company rendering, hinted deleted secret, and unchanged no-hint fallback. - `pnpm run typecheck` in `server` and `ui`. - Manual: edit an environment whose secret-ref field references another company's secret; the field names the secret and its owning company instead of "Missing secret". ## Risks - The endpoint exposes secret names and company names across companies to instance-level environment editors. Those actors already manage instance-shared environments (and instance admins are implicit members of every company), so this reveals no secret material and no new reach; the service method documents that callers must sit behind an instance-level gate. - UI change is additive and context-gated; pickers without a provider render exactly as before. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use (file edits, vitest/tsc runs). No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-31 16:46:26 -07:00
Devin Foley	f51cba33fa	fix(server): keep environment secret bindings consistent when re-pointing config secrets (#10576 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents can run inside environments (SSH boxes, sandbox providers); a sandbox environment's config can reference stored company secrets (for example a provider API key) through `format: "secret-ref"` fields > - Environments are instance-scoped and shared by every company on an instance, but `company_secret_bindings` rows are company-scoped, and the environment routes synced config-derived bindings under one guessed "context company" resolved from the environment's existing bindings > - When a save re-pointed a secret-ref field at a secret owned by a different company, the binding sync threw after the config row had already been persisted: the config referenced the new secret, the binding still pointed at the old one, every later lease acquisition failed with `Secret is not bound to environment:<id> at apiKey`, and the stale cross-company binding made every later save fail with a company-context conflict — with no route-level way to recover > - This pull request makes config-derived bindings follow the company that owns each referenced secret, and makes the environment write and its binding syncs atomic > - The benefit is that environment saves can no longer strand an environment in a half-updated state that breaks all of its runs ## Linked Issues or Issue Description Refs #10577 (companion UX change: the editor state that nudges operators into this sequence). What happened? Saving an environment whose secret-ref config field points at a secret owned by a different company than the environment's existing binding partially applied: the config row updated, the binding sync failed server-side, and the environment was left referencing a secret it has no binding for. Every run that leased the environment then failed with `lease_acquire_failed: ... Secret is not bound to environment:<id> at apiKey`, and every later save of the environment returned 409 `Environment secret bindings already use a different company context.` — with no route-level way to recover. Steps to reproduce 1. On an instance with two companies, create a sandbox environment from company A with a picker-bound API-key secret owned by A (the binding lands in A). 2. From company B, create a new secret and re-point the environment's API-key field at it, then save. 3. The save persists the config but the binding sync throws, so no binding for B's secret exists. 4. Run any agent that uses the environment, or try to save the environment again. Expected behavior The save either fully applies (config and bindings consistent) or fully fails. Re-pointing a config secret ref to a secret owned by another company moves the binding with the secret. Paperclip version Reproduced on current `master` (also present on recent release images). Deployment mode Multi-company server deployment (any mode with more than one company on the instance). ## What Changed - New `secretService.replaceSecretRefsForInstanceTarget`: writes each config-derived binding under the company that owns the referenced secret, replaces all non-`env.` bindings of the target across every company, and validates every ref (secret exists, not deleted, config-path and projection-class rules) before any row is written. `env.` env-var bindings stay company-scoped and untouched. - The environment create and update routes now run the environment write and its binding syncs inside one `db.transaction`, threading the transaction through new optional executor seams on `environmentService.create/update` and the existing `SecretBindingDb` seam pattern, so an invalid ref rolls the whole save back instead of leaving a half-updated environment. - `resolveEnvironmentSecretContextCompanyId` no longer lets existing bindings veto the caller's context (the 409s above); it now only picks where new raw-pasted secrets are created and how env-var bindings and probes resolve: explicit route/query company first, then the single company the bindings live in, then the actor's company. ## Verification - `cd server && pnpm vitest run src/__tests__/environment-routes.test.ts src/__tests__/environment-instance-routes.test.ts src/__tests__/secrets-service.test.ts src/__tests__/environment-custom-image-routes.test.ts` (165 tests, includes new coverage below) - New embedded-Postgres tests prove: a re-point moves the binding to the new secret's company and deletes the stale row; refs across several companies each bind under their own secret's company; an unknown secret ref rejects without touching existing bindings; `env.*` rows survive config-ref replacement. - New route tests prove: a cross-company re-point that previously 409'd now saves, with the update and binding replacement on the same transaction executor; a failing ref surfaces as 422. - `cd server && pnpm run typecheck` ## Risks - Behavioral shift: environment saves no longer 409 on a company-context mismatch between the caller and existing bindings; bindings follow the referenced secret's company instead. Environment routes are instance-admin gated, and instance admins already had access to every company's secrets by passing the company explicitly, so this removes an ordering trap rather than widening access. - Runtime lease resolution is unchanged: a run still resolves environment secrets under the run's own company, so an environment referencing company B's secret still only leases for company B runs (fail-closed as before). - The delete route's per-company binding cleanup is unchanged. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use (file edits, vitest/tsc runs). No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-31 16:19:25 -07:00
Devin Foley	9f5af4ea5d	fix(server): accept secret_ref binding objects in sandbox provider environment config (#10355 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents execute in environments; sandbox provider plugins (Daytona, Modal, e2b, …) declare their config via a JSON schema, with credentials marked `format: "secret-ref"` > - The environments UI renders those fields with a secret picker that submits `{ type: "secret_ref", secretId, version }` binding objects, while the server-side environment config paths only understood raw string values and bare secret-id strings > - The binding object reached the plugin worker's `environmentValidateConfig` untouched; plugins parse non-string config values as absent, so saving or testing an environment with a picker-bound secret always failed validation (e.g. "Daytona sandbox environments require an API key in config or DAYTONA_API_KEY.", "Modal sandbox environments require tokenId and tokenSecret.") > - Worse, an environment first saved with raw pasted values becomes uneditable: the stored value is a secret reference, the edit form re-submits it as a binding object, and every subsequent save fails the same way > - This pull request canonicalizes binding objects to the bare secret id before plugin validation, and teaches the persistence/runtime/probe secret-ref resolvers to accept the object shape defensively > - The benefit is that picker-bound secrets work for every schema-driven sandbox provider — create, edit, and Test — with no plugin changes required ## Linked Issues or Issue Description Fixes #10105 The same failure reproduces with the Daytona provider: Settings → Instance settings → Environments → New, driver sandbox, provider daytona, bind Api Key to an existing secret via the picker → Save fails with "Daytona sandbox environments require an API key in config or DAYTONA_API_KEY." ## What Changed - `server/src/services/json-schema-secret-refs.ts`: new `parseSecretRefBindingObject()` that recognizes the `{ type: "secret_ref", secretId, version? }` shape the secret picker submits (version defaults to `"latest"`; malformed objects return null). - `server/src/services/plugin-environment-driver.ts`: `validatePluginSandboxProviderConfig()` now canonicalizes binding objects at the driver schema's `format: "secret-ref"` paths to the bare secret id (the persisted shape) before invoking the plugin worker's `environmentValidateConfig`. Pinned numeric versions are rejected with a clear 422, since sandbox provider references always resolve the latest version — silently resolving a different version would be worse. - `server/src/services/environment-config.ts`: the persistence, runtime, and probe secret-ref resolvers plus `collectEnvironmentSecretRefs()` accept the binding-object shape defensively, so any previously persisted object-shaped refs (from providers whose validation tolerated them) resolve instead of being silently skipped; the missing-companyId runtime guard also now fails closed for object-shaped refs. ## Verification - `npx vitest run server/src/__tests__/json-schema-secret-refs.test.ts server/src/__tests__/plugin-sandbox-provider-config-validation.test.ts server/src/__tests__/environment-routes.test.ts server/src/__tests__/environment-config.test.ts` — 82 tests pass, including new coverage: binding-object canonicalization before plugin validation, pinned-version rejection, raw-string pass-through, and a route-level create with a picker-submitted binding object persisting the bare secret id without minting a duplicate secret. - `npx vitest run server/src/__tests__/environment-runtime.test.ts` — 24 tests pass against embedded Postgres, including a new test that persists an object-shaped ref and verifies runtime resolution produces the plaintext credential for the plugin worker. - `pnpm typecheck` in `server/` — clean. ## Risks - Low. The canonical persisted shape (bare secret-id string) is unchanged, so existing saved environments and lease-resume fingerprints are unaffected; raw pasted values and bare-id strings take exactly the same code path as before. - New behavior only triggers where a save/probe previously failed 422 (binding objects at secret-ref paths) or where an object-shaped ref was previously skipped silently at runtime (now resolved, or failed closed without a companyId). - Pinned binding versions at sandbox-provider paths are now an explicit 422 instead of an accidental validation failure; no UI submits pinned versions today (`allowVersionSelector={false}`). ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) with extended thinking and agentic tool use (Claude Code harness): source diagnosis, fix, and tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no doc surface changed) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-28 10:50:13 -07:00
Devin FoleyandPaperclip	c274f10abc	feat(server): computed owner instance-admin elevation for cloud-managed instances, behind platform floors (#10343 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Cloud-managed instances authenticate tenant users through a trusted-header path (`resolveCloudTenantActor`) that deliberately never grants `instance_admin`, so every tenant user is company-scoped > - On a dedicated (single-owner) managed instance that leaves the paying owner unable to administer their own instance: instance settings, the environments admin surface, and the custom sandbox image flow are all instance-admin gated (the environments UI can't even show the provider/image of the platform sandbox because the restricted read view blanks `config` entirely) > - Re-granting the old blanket `instance_user_roles` row would repeat the mistake the shared-pool hardening fixed: DB rows go stale, resurrect via restores, and elevate through every auth path > - This pull request elevates only the stack `owner`, computed per request at the trusted-header boundary behind a new managed-tier feature flag, and ships that elevation together with code floors on the platform-owned surfaces an instance admin must not control on a managed instance > - The benefit is that dedicated-stack owners can administer their own instance while platform credentials, execution policy, backups, and runtime code-install stay platform-owned, and self-hosted behavior is unchanged ## Linked Issues or Issue Description No public GitHub issue exists for this change; the underlying issue is described here following the feature-request template. ### Problem or motivation - On cloud-managed instances, tenant users resolved from trusted headers are always company-scoped. For dedicated instances with a single paying owner, the owner cannot reach any instance-admin surface of their own instance (instance settings, environments administration, custom image setup), and the restricted environment read view hides even structural fields like the sandbox provider and image. - The previous hardening intentionally removed blanket elevation (and purges stale `instance_user_roles` rows on every trusted-header authentication). That protection must not regress for shared multi-tenant pools. ### Proposed solution - Owner-only, computed, flag-gated elevation plus code floors on platform-owned surfaces, in one PR so the elevation can never ship without the floors. ### Alternatives considered - Re-inserting an `instance_user_roles` row for owners (the pre-hardening model): rejected — DB rows go stale, survive restores, and elevate through every auth path; #7525 removed exactly this. - Widening only the environments read view without any elevation: rejected — it fixes one screen but still leaves a dedicated-instance owner unable to administer instance settings or custom images. - Elevating additional stack roles (`member`/`admin`/`support`): rejected — only the owner has an ownership claim over the whole instance; other roles stay company-scoped. ### Roadmap alignment - Extends the shipped "Cloud deployments" roadmap work (multi-tenant isolation, company-scoped cloud tenants, managed-instance bootstrap) without overlapping planned core items, and leaves self-hosted behavior unchanged. ## What Changed - New feature key `enableOwnerInstanceAdmin` (`packages/shared`): boolean flag in `instanceExperimentalSettingsSchema`, catalog tier `managed`, `cloudDefault: true`, `selfHostedDefault: false`. Inert on self-hosted instances — the elevation path only exists behind the cloud tenant trust token. - Computed elevation (`server/src/middleware/auth.ts`): `resolveCloudTenantActor` now returns `isInstanceAdmin: true` only when the trusted-header stack role is `owner` and the flag is enabled. The flag is resolved through the instance-settings service so the managed-config overlay applies (the control plane can disable elevation fleet-wide without touching tenant databases; a DB row edit or restore cannot resurrect it). Resolution fails closed on settings read errors. The `instance_user_roles` never-insert and the per-request stale-row purge are byte-identical. `member`/`admin`/`support` stack roles stay company-scoped. - Authorization guard split (`server/src/services/authorization.ts`): the blanket-allow now trusts the actor's computed `isInstanceAdmin` flag (only the attested resolver can set it for `cloud_tenant` actors) while keeping the `instance_user_roles` DB lookup excluded for `cloud_tenant` — a stale or hand-inserted role row still elevates nothing. - Floor F1 — platform environment credentials (`server/src/routes/environments.ts`): on cloud-managed instances, platform-provisioned environment rows (`managedByPaperclip` marker, plus the legacy managed-Kubernetes marker) use a single floored view for every reader on all environment routes (list, get, create, update, delete responses): `envVars` are never echoed and credential-shaped `config` keys (reusing the managed-config `SECRET_LIKE_CONFIG_KEY_PATTERN`) are dropped — for all actors including instance admins — while structural config (provider, image, template, region, …) and the managed markers stay visible. This also fixes the environments UI for managed sandboxes, which previously lost the provider/image entirely in the restricted view. The floor also covers writes: `PATCH /environments/:id` and `DELETE /environments/:id` on a platform-provisioned row are rejected (403, `environment_platform_managed`) for every actor including instance admins, and the guard binds to the persisted row's markers so a patch cannot strip the managed marker to lift the floor. The one recovery path is a metadata-only PATCH that solely clears the marker keys (null/false), for rows stamped through the old unrestricted API before the markers became reserved — and it never applies to a row whose slot markers are live platform state: the single local row (`environments_local_driver_idx`), which `ensureLocalEnvironment` adopts and stamps on cloud-managed instances from every caller (company creation, the heartbeat, run orchestration), and the single marked sandbox row (`environments_managed_sandbox_idx`) while a managed-sandbox bootstrap path is configured (managed-config `environments` section or `PAPERCLIP_EXECUTION_MODE=kubernetes`) and the provisioner therefore adopts and refreshes it on every boot. Clearing a live slot row's markers would let the next write reclassify it as tenant-managed and bypass the floor; conversely, when no sandbox provisioning path is configured the platform holds no claim on any sandbox row, so a platform marker there is stale by definition and the recovery patch applies. Every marker outside a live slot is clearable, so no legacy row is ever locked permanently. Custom-image setup and probes on the platform sandbox stay available to instance admins — those are the owner-facing flows this elevation exists for. The marker keys themselves are reserved: client create/update payloads that set `managedByPaperclip` or `managedKubernetesSandbox` are rejected (422, `environment_platform_marker_reserved`) on cloud-managed instances, so a tenant row can never be stamped platform-provisioned through the API and self-locked behind the write floor (the provisioner writes markers at the service layer, not through these routes). Tenant-created environments are otherwise unaffected. - Floor F2 — executionMode (`server/src/routes/instance-settings.ts`): on cloud-managed instances, `PATCH /instance/settings/general` rejects writes that would change `executionMode` (403, `execution_mode_platform_managed`). Same-value echoes pass so settings forms that submit the full general-settings object keep working. The boot-time execution-policy bootstrap path is untouched (it calls the service directly). - Floor F3 — manual database backups (`server/src/routes/instance-database-backups.ts`): floored off on cloud-managed instances (403, `database_backups_platform_managed`); backups are platform-owned there, and the result would also echo a server-side filesystem path. - Floor F4 — adapter code install (`server/src/routes/adapters.ts`): `POST /adapters/install` and `POST /adapters/:type/reinstall` are floored off on cloud-managed instances (403, `adapter_install_platform_managed`). Adapter packages execute in the server process, so a runtime install would let an instance admin read the platform trust anchors out of the process environment. This mirrors the existing bundled-only plugin install floor; adapter code on managed instances comes bundled with the platform image. ## Instance-admin surface audit Before widening who can hold `isInstanceAdmin`, every instance-admin-gated surface in `server/src` was enumerated and reviewed for whether its response or side effects could echo process environment values or platform credentials (tenant trust token, JWT signing keys, database connection strings, provider API keys): 29 distinct gate definitions covering ~90+ call sites, in four groups — sole instance-admin gates (12), instance-admin-or-company-permission gates (10), response-shaping/scope-widening sites (6), and the central `allow_instance_admin` short-circuit in the authorization service (58 `decide()` call sites). Findings and dispositions: - Environment read/write responses exposed platform sandbox `envVars`/credential-shaped config to instance admins → closed by floor F1. - Manual backup trigger echoed a server filesystem path and triggers a platform-owned operation → closed by floor F3. - Adapter install/reinstall loads externally fetched code into the server process (indirect, complete env exposure) → closed by floor F4. The sibling plugin-install path already had a bundled-only floor on managed instances and needed no change. - Token-minting surfaces (gateway tokens, custom-image terminal/connection tokens) mint credentials scoped to the instance's own resources, not platform trust anchors → acceptable for an owner-admin of a dedicated instance; unchanged. - All remaining gated surfaces return ordinary instance-scoped business data; none echo `process.env` or platform secrets directly. OAuth client secrets are referenced by env-var name only; SSH private keys are stored as secret refs before persistence and are not echoed. Operational note for managed platforms: this model assumes the process environment of a managed instance holds only that instance's own credentials. Platform operators should keep provider credentials per-instance (never fleet-shared) since an instance admin ultimately controls in-process code on their own instance. ## Verification - `pnpm vitest run server/src/middleware/cloud-tenant-actor.test.ts` — resolver matrix: owner × flag on/off, flag via managed overlay (on-over-DB-off and off-over-DB-on), member/admin/support × flag on, no-token self-hosted, fail-closed settings read, purge still runs and no role row is ever inserted (14 tests). - `pnpm vitest run server/src/__tests__/authorization-service.test.ts` — computed flag elevates a `cloud_tenant` actor; a stale `instance_user_roles` row still never does; `session` actors unchanged (full suite, embedded Postgres). - `pnpm vitest run server/src/__tests__/environment-routes.test.ts` — F1: no secret echo to admins on get/list, structural config visible to restricted readers, platform-row PATCH/DELETE rejected for admins (including a marker-stripping patch), marker-clear recovery allowed for stale legacy rows and for a marked sandbox row when no provisioning path is configured, but refused on the managed local row and on the sandbox slot row under a managed-config `environments` entry or the forced kubernetes execution mode, client marker-stamping creates/patches rejected, tenant rows still readable and writable, self-hosted read+write regression (60 tests). - `pnpm vitest run server/src/__tests__/environment-service.test.ts` — `ensureLocalEnvironment` adopts a pre-existing local row on cloud-managed instances (marker stamped, other metadata preserved, idempotent — no rewrite on re-ensure) and leaves self-hosted rows untouched (22 tests, embedded Postgres). - `pnpm vitest run server/src/__tests__/instance-settings-routes.test.ts server/src/__tests__/instance-database-backups-routes.test.ts` — F2 change-vs-echo matrix incl. self-hosted regression; F3 floor for both admin shapes (32 tests). - `pnpm vitest run server/src/__tests__/adapter-routes-authz.test.ts` — F4 floor; self-hosted install/reinstall behavior unchanged (existing cases). - `pnpm vitest run server/src/__tests__/first-admin-claim.test.ts server/src/__tests__/bootstrap-claim-routes.test.ts server/src/__tests__/managed-config.test.ts server/src/__tests__/health.test.ts server/src/__tests__/instance-settings-managed-overlay.test.ts server/src/services/managed-environments.test.ts server/src/services/execution-policy-bootstrap.test.ts` — first-admin bootstrap gate and managed-config behavior unchanged (91 tests). - `pnpm vitest run packages/shared/src/feature-catalog.test.ts` — catalog/schema sync tests cover the new key (selfHostedDefault must equal the schema default). - `pnpm run typecheck` — all 31 workspace projects clean. ## Risks - Self-hosted behavior is unchanged: every floor binds to `isCloudManagedInstance()` (tenant trust token present), the new flag defaults off with no elevation path, and regression tests pin the self-hosted branches. - The elevation is fail-closed and stateless: turning the flag off (managed overlay or DB) de-elevates on the next request; there is no role row to clean up and restores cannot resurrect elevation. - On a cloud-managed instance a pre-existing unmarked local row is adopted (stamped `managedByPaperclip`) by the next ensure and becomes platform-owned — the intended managed-product semantic: the platform owns the single local slot. Self-hosted instances are untouched. - F1 widens restricted readers' view of platform-provisioned rows from fully blanked `config`/`metadata` to structural-only `config` plus markers. Platform-delivered config is guaranteed secret-free by the managed-config contract (secret-shaped keys fail startup), and the floor re-drops secret-shaped keys defensively. - One extra instance-settings read per trusted-header request for owner-role actors (the resolver already performs several queries per request). ## Model Used Claude Fable 5 (Anthropic) — model id `claude-fable-5`, extended thinking enabled, agentic tool use via Claude Code; read-only explore subagents on the same model were used for the surface audit sweep. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-27 18:58:31 -07:00
Devin Foley	17dde9d3f2	fix(sandbox): keep custom-image snapshots applied to config tests, probes, and saves (#9385 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox environments can capture reusable custom images (provider snapshots) so agents boot with pre-installed tools and CLI logins > - The custom-image runtime fingerprint check included provider secret-ref paths (e.g. the Daytona `apiKey`), while capture-time fingerprinting excluded them, so any config carrying a credential never matched its captured snapshot > - As a result, agent config tests and environment probes silently booted the provider base image instead of the snapshot, test sandboxes were deleted before operators could inspect them, and any environment save orphaned the snapshot without warning > - The UI compounded the confusion by displaying an internal template id that matches nothing in the provider dashboard > - This pull request aligns runtime fingerprints with capture-time exclusions, re-stamps fingerprints on saves that cannot affect the snapshot (warning when they can), archives test/probe sandboxes instead of deleting them, and surfaces the provider snapshot ref in the UI > - The benefit is that custom images actually apply to config tests and probes, survive unrelated config edits, and are debuggable against the provider dashboard ## Linked Issues or Issue Description No public GitHub issue exists for this; describing it in-PR per the bug template. Related: Refs #9329 (saved-environment probe company context — this branch carries an equivalent fix), Refs #8794 (introduced reusable sandbox custom images). What happened? With a Daytona environment whose provider config stores the API key as a secret reference and an active captured custom-image snapshot: - Agent config tests and environment probes booted the provider base image (`daytonaio/sandbox:0.8.0`) instead of the captured snapshot, so CLI upgrades/logins baked into the snapshot were missing and the probe reported "login required" and an outdated CLI. - The environment card showed an internal template id (e.g. `b5be03e1-ca5…`) that does not correspond to any snapshot name in the provider dashboard, making the active image impossible to correlate. - Test/probe sandboxes were deleted immediately after the run, so the sandbox a test used could not be inspected afterwards. - Saving the environment config (even fields unrelated to the image) changed the stored fingerprint, silently detaching the snapshot with no warning. Expected behavior Config tests and probes boot the captured snapshot when one is active; the UI shows the provider-facing snapshot/template ref; test sandboxes stay inspectable for a short window; unrelated config edits keep the snapshot linked, and edits that genuinely invalidate it produce an explicit warning. Steps to reproduce 1. Configure a sandbox environment on Daytona with the API key stored as a company secret reference. 2. Capture a custom image snapshot from the environment page and mark it active (e.g. after installing/logging into a CLI in the setup sandbox). 3. Run the agent config test or an environment probe: the sandbox boots the base image, not the snapshot, and the sandbox is deleted immediately after the test. 4. Save the environment config with an unrelated field change: the snapshot silently stops applying. Paperclip version or commit `master` at the merge-base of this branch. Deployment mode Self-hosted local instance (macOS, pnpm dev server) with the Daytona sandbox provider plugin. ## What Changed - Runtime custom-image fingerprint checks now exclude provider secret-ref paths, matching capture-time exclusions, so configs carrying credentials match their captured snapshots (`environment-custom-image-runtime.ts`). - Agent config tests and saved-environment probes force fresh, non-reused sandboxes and pass company context so lease-backed probes can resolve company secrets and boot the real snapshot (`environment-probe.ts`, `routes/agents.ts`, `routes/environments.ts`). - Test/probe sandboxes are released by archiving (stop + 60-minute provider-side auto-delete) instead of immediate deletion, so operators can inspect the exact sandbox a test used (Daytona plugin). - On environment PATCH save, changes that cannot affect the captured snapshot re-stamp the template's source fingerprint so the snapshot stays linked; boot-source or provider-identity changes (new manifest field `templateIdentityPaths`) mark the template detached and the save response reports it (`environment-custom-images.ts`, shared plugin types/validators). - The custom-image overview exposes `activeTemplateMatchesConfig`; the environments UI shows the provider snapshot/template ref (internal id moved to a tooltip), warns via toast when a save detaches the snapshot, and shows a persistent "Not in use" warning when the active template no longer matches the saved config (`CompanyEnvironments.tsx`, `api/environments.ts`). ## Verification - `pnpm vitest run server/src/__tests__/environment-custom-images-service.test.ts server/src/__tests__/environment-probe.test.ts server/src/__tests__/environment-routes.test.ts server/src/__tests__/agent-test-environment-routes.test.ts` — server coverage for fingerprint exclusions, re-stamp/detach on save, probe company context, and fresh-sandbox test behavior. - `pnpm vitest run packages/plugins/sandbox-providers/daytona/src/plugin.test.ts` — archive-on-release and snapshot ref handling. - `pnpm vitest run ui/src/pages/CompanyEnvironments.test.tsx` — snapshot ref display, detach toast, and "Not in use" warning. - Manually verified end-to-end on a live self-hosted instance against real Daytona: config test boots the captured snapshot (CLI login and version persist), the test sandbox remains visible in the provider dashboard as archived, and saving unrelated fields keeps the snapshot applied. ## Risks - Fingerprint exclusion widening: a provider credential rotation alone no longer detaches a captured snapshot; that is the intended behavior (the snapshot content does not depend on the credential), and provider-identity fields (e.g. Daytona `apiUrl`) still detach via `templateIdentityPaths`. - Archived test sandboxes consume provider-side resources for up to their auto-delete window instead of being freed immediately; bounded (60 minutes) and only for test/probe sandboxes. - New optional manifest field `templateIdentityPaths` is backward-compatible; providers that omit it keep current matching behavior. ## Model Used - Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use via Claude Code / Claude Agent SDK. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-07-10 14:32:47 -07:00
Nicky Leach Priya Raman Paperclip Harold Kim	719da5f9b5	Harden environment deletion and expose delete blast radius (#9250 ) ## Thinking Path > - Paperclip manages AI agents that each have an associated execution environment (local, Kubernetes, etc.) > - Instance administrators can create and delete environments; currently the DELETE endpoint has no protection against deleting managed or in-use environments > - Deleting the managed local environment or the instance-default environment would break all agents using those environments with no path to recovery > - The endpoint also suffered a TOCTOU race: a check-then-delete pattern allowed the managed-local or default guard to pass if the environment's role changed between the read and the delete > - This pull request adds a blast-radius read endpoint so admins can preview impact, hard-blocks the dangerous deletes atomically, cleans up all dependent references after a valid delete, and fixes a concurrent creation race in ensureLocalEnvironment ## Linked Issues or Issue Description Fixes #9251 ## What Changed - New endpoint `GET /api/environments/:id/delete-blast-radius` (instance-admin gated): returns reference counts (agent defaults, workspace selections, issue selections, project selections, secret bindings, active leases, active setup sessions) and blocking reasons — no config, env-var values, or secret data returned. - Atomic delete guard `environmentService.removeIfDeletable(id)`: performs the DELETE with an inline `WHERE driver != 'local' AND NOT EXISTS (instanceSettings where defaultEnvironmentId = id)` predicate, eliminating the TOCTOU race between the app-level check and the DB write. - Route hardening: `DELETE /environments/:id` now calls `getDeleteBlastRadius` first (app-level check + logging), then calls `removeIfDeletable` (atomic guard). If the atomic guard returns null the route fetches a fresh blast-radius snapshot and rejects with a 409 Conflict carrying `deleteBlockedReasons`. - Reference cleanup on valid delete: after a successful delete, the route clears environment selections on all company execution workspaces, issues, and projects; syncs env-var secret bindings to `{}` (removing bindings for the deleted environment); syncs config secret refs to `[]` for the environment target; and removes the SSH private-key secret if one was stored. - Race fix in `ensureLocalEnvironment`: the insert-or-nothing path now catches a `environments_name_idx` unique-constraint violation and falls through to the existing SELECT, treating the name conflict as idempotent. - Shared types: `EnvironmentDeleteBlastRadius` and `EnvironmentDeleteBlockedReason` exported from `@paperclipai/shared`. - OpenAPI: registers the new blast-radius endpoint; updates the delete-environment response schema to document 403/404/409. - Tests: 56 existing environment-route and service tests continue to pass; new service-level regression tests assert the atomic guard rejects `local`-driver environments and instance-default environments and succeeds for deletable ones. ## Verification ``` corepack pnpm exec vitest run \ server/src/__tests__/environment-routes.test.ts \ server/src/__tests__/environment-service.test.ts # 56 tests, all passing corepack pnpm --filter @paperclipai/shared typecheck node scripts/ensure-plugin-build-deps.mjs cd server && ../node_modules/.bin/tsc --noEmit ``` ## Risks - Blast-radius endpoint auth: guarded by `assertCanAccessInstanceEnvironments`, the same gate as the existing environment-list and delete routes. Non-admin callers receive 401/403 before any data is returned. - Atomic guard may reject a delete that the app-level check passed: this is intentional — it means the environment became protected between the read and the write. The caller receives a fresh blast-radius snapshot explaining why. - Secret cleanup ordering: cleanup runs after the atomic DELETE succeeds, in parallel across companies. If cleanup partially fails the environment row is already gone; partial-cleanup state is recoverable by re-running the sync operations. Risk: low — these are idempotent upsert/sync operations. - ensureLocalEnvironment race fix: swapping a unique-constraint error for an idempotent SELECT adds one extra query on the conflict path. This path is rare (only fires during concurrent boot) and is significantly safer than the previous behavior. - No migration: all changes are application-level; no schema changes required. ## Model Used - Provider: Anthropic - Model: claude-sonnet-4-6 (Claude Sonnet 4.6) - Context window: 200k tokens - Mode: agentic tool use via Paperclip agent system (Claude Code) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Priya Raman <priya.raman@paperclip.ing> Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Harold Kim <harold.kim@paperclip.ing>	2026-07-08 22:04:22 -07:00
Devin FoleyandPaperclip	b4815bf964	Scope environment custom images to instance environments (#8850 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environments are now managed as instance-level runtime resources rather than per-company rows > - The custom environment image setup tables were introduced with their own `company_id` columns and route query parameters > - That split made one saved environment image state depend on an extra company context even though the environment itself is the durable owner > - It also made saved-environment probes harder because applying the active custom image template could require a company context when no secret-backed config needed one > - This pull request scopes custom image templates and setup sessions directly to the saved environment > - The benefit is that reusable environment images follow the same instance-scoped model as environments while secret resolution still uses company context only when secrets require it ## Linked Issues or Issue Description No matching public GitHub issue was found. Bug report: ### What happened? saved environment custom-image routes and persistence required a `companyId` even though environments are instance-scoped, and saved sandbox probes did not opt into active custom-image template application unless a company context was present. ### Expected behavior custom-image templates and setup sessions should be owned by the saved environment, and saved sandbox probes should apply the active template while still requiring a company context only for secret-backed runtime config. ### Steps to reproduce 1. Configure an instance-scoped sandbox environment with custom-image setup support. 2. Start or inspect a custom-image session or template for that saved environment. 3. Probe the saved environment without a custom-image-specific `companyId` query parameter. ### Paperclip version or commit current `master` after the environment custom-image template migration. ### Deployment mode Local dev (pnpm dev) or authenticated local Paperclip instance. ### Installation method Built from source (pnpm dev / pnpm build). ### Agent adapter(s) involved Not adapter-specific (core bug). ### Database mode Embedded PGlite/Postgres dev database. ### Access context Board human operator. ### Privacy checklist No logs, secrets, tokens, private URLs, or local machine paths are included. Duplicate search performed: - `gh search prs "environment custom image companyId repo:paperclipai/paperclip" --state open --limit 20` - `gh search prs "custom image environment scoped repo:paperclipai/paperclip" --state open --limit 20` - `gh search issues "environment custom image repo:paperclipai/paperclip" --state open --limit 20` The returned results were unrelated adapter, Docker, auth, or stale-workspace items. ## What Changed - Removed redundant `company_id` columns from environment custom-image templates and setup sessions. - Added migration `0127_environment_custom_images_instance_scoped` to collapse duplicate active rows per environment before dropping the old company-scoped indexes/columns. - Updated custom-image services, route handlers, shared validators, and UI API/query keys to use environment-scoped custom-image state. - Kept runtime secret resolution company-aware only when secret refs or bindings require a company context. - Made saved sandbox environment probes opt into active custom-image template application. - Updated DB, shared, server, and UI tests for the new environment-scoped contract. ## Verification - `pnpm --filter @paperclipai/db run check:migrations` - `pnpm exec vitest run packages/db/src/environment-custom-images-schema.test.ts packages/shared/src/environment-custom-images.test.ts server/src/__tests__/environment-custom-image-routes.test.ts server/src/__tests__/environment-custom-images-service.test.ts server/src/__tests__/environment-routes.test.ts ui/src/pages/CompanyEnvironments.test.tsx` - `pnpm -r typecheck` - `pnpm test:run` before rebasing onto latest `master`; after the rebase only the migration number changed, and the migration check plus focused suite, typecheck, and build were rerun. - `pnpm build` ## Risks - Migration safety: the migration supersedes duplicate active templates per environment and fails duplicate active setup sessions before adding environment-only unique indexes. Operators with duplicate historical active rows should review which active template is kept. - Behavior shift: plugin custom-image setup calls now receive `companyId: "instance"` when no secret binding determines a concrete company context. - Secret-backed configs still require an explicit or uniquely inferable company context; environments with secret bindings spread across multiple companies continue to fail fast. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex via the `codex_local` adapter, GPT-5-based coding model with tool-enabled repository inspection, editing, testing, git, and GitHub CLI access. Exact context-window metadata was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-07-01 23:57:31 -07:00
Devin FoleyandPaperclip	8d9f9fd240	Add reusable sandbox custom images (#8794 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A growing part of that work runs in sandboxed environments rather than on the operator's local machine. > - Today sandbox providers can start fresh workspaces and run probes, but they do not have a shared contract for capturing and reusing prepared sandbox state. > - Operators need a way to set up tools, credentials, and project dependencies once, then reuse that prepared image for later agent runs. > - This pull request adds reusable sandbox custom images across the provider contract, server runtime, and board UI. > - It also keeps probes and sandbox copy flows aligned with pre-authenticated/custom-image environments. > - The benefit is faster, more reliable sandbox runs without repeatedly rebuilding the same environment setup. ## Linked Issues or Issue Description No public GitHub issue was found for this change. Inline feature request follows. ### Problem or motivation Sandboxed agents need reusable prepared runtime state so repeated runs do not require manual setup every time. Operators often need system packages, CLIs, SDKs, dependency caches, credentials, and project tooling available before an agent can work productively. ### Proposed solution Add a provider-level custom-image capability, server-side setup/capture lifecycle, Daytona/fake provider support, and board UI controls for creating, testing, selecting, and deleting custom images. ### Alternatives considered Leaving this as provider-specific setup outside Paperclip would keep the control plane blind to image state and would not give agents consistent environment metadata. Re-running setup commands for every lease is simpler, but slower and less reliable for interactive or credentialed setup. ### Roadmap alignment Checked `ROADMAP.md`; this aligns with the Cloud / Sandbox agents roadmap area and does not duplicate any related public issue or PR found by search. Additional context: - Subsystem affected: cross-cutting (`packages/db`, `packages/shared`, `packages/plugins`, `server`, `ui`). - Duplicate search: searched GitHub for `sandbox custom image` and `sandbox template environment`; no related public issues or PRs were found. ## What Changed - Added custom-image shared types, validators, constants, API paths, and database schema/migration. - Added server services/routes for custom-image templates and setup sessions, including runtime cleanup and provider metadata handling. - Extended plugin/sandbox provider capabilities for interactive setup, template capture, and template deletion. - Implemented custom-image support in the fake sandbox provider and Daytona provider. - Updated environment runtime/config handling so active custom images flow into leases, probes, and agent execution. - Added board UI controls and API client support for custom-image setup, capture, selection, status, and error states. - Hardened sandbox copy/probe behavior for insecure clipboard contexts and pre-authenticated sandbox images. - Added targeted coverage across shared validators, DB schema, server routes/services, provider plugins, adapter probes, and UI flows. ## Verification - `pnpm install --frozen-lockfile --ignore-scripts` - `pnpm vitest run packages/adapters/claude-local/src/server/test.probe.test.ts packages/adapters/claude-local/src/server/test.ts packages/adapters/codex-local/src/server/test.remote.test.ts packages/adapters/codex-local/src/server/test.ts` - `pnpm --filter @paperclipai/adapter-claude-local typecheck` - `pnpm --filter @paperclipai/adapter-codex-local typecheck` - `pnpm vitest run packages/db/src/environment-custom-images-schema.test.ts packages/shared/src/environment-custom-images.test.ts packages/shared/src/validators/plugin.test.ts server/src/__tests__/environment-custom-images-service.test.ts server/src/__tests__/workspace-runtime.test.ts packages/plugins/sandbox-providers/daytona/src/plugin.test.ts ui/src/pages/CompanyEnvironments.test.tsx ui/src/pages/CompanySettings.test.tsx` - `pnpm -r typecheck` - `pnpm build` - `rm -rf packages/db/dist && pnpm test:run` - Public-safety scan of the final diff found no internal Paperclip issue links, private instance URLs, or real secret patterns. ## Risks - Adds a database migration and new environment runtime tables, so migration ordering and rollback need care. - Provider implementations may differ in how reliably they can capture/delete images; unsupported providers surface capability-gated UI states. - Custom-image state can contain operator-prepared tooling and credentials inside the provider image, so providers must enforce their own access controls and cleanup semantics. - Broad surface area across shared contracts, server runtime, plugins, adapters, and UI means CI and Greptile review should be watched closely. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 (`gpt-5`) via Codex CLI with tool use and code execution. Assisted with branch cleanup, conflict resolution, local verification, and PR preparation. Earlier branch implementation work was assisted by Paperclip-managed Claude/Codex agents. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-06-30 09:57:49 -07:00
Devin Foley	547463d3a2	refactor(environments): make execution environments instance-scoped (#8375 ) ## Thinking Path > - Paperclip is the control plane for AI-agent companies, so execution environment selection has to stay inspectable and predictable across companies, agents, and runs. > - The environment subsystem decides where an agent heartbeat actually runs and how remote sandbox state is realized and restored. > - That subsystem previously mixed company-scoped environment catalogs with issue-level environment stamping, so a reassigned issue could keep executing in the previous assignee's sandbox. > - That behavior breaks the control-plane contract: changing the assignee should change the executing agent/environment path unless there is an explicit current override. > - Fixing it cleanly required more than a narrow patch; the environment model had to move to instance scope with a single inherited default and per-agent override semantics. > - This pull request rewires the schema, server/API surface, runtime resolution, and UI around that model, then adds regression coverage for cross-company inheritance and per-agent isolation. > - The benefit is that environment choice now follows the approved instance/agent configuration path instead of stale issue state, while shared environments only need to be configured once per instance. ## Linked Issues or Issue Description - No directly matching public GitHub issue or PR was found while searching for this refactor. ### What happened? Reassigning work between agents with different execution environments could keep running in the previous sandbox because environment choice was stamped onto the issue and outranked the current assignee. The same subsystem also forced environment catalogs to be duplicated per company even though the underlying execution environments were instance-wide resources. ### Expected behavior Execution should resolve through the current instance and agent configuration path, with one instance-scoped environment catalog, one instance default, optional per-agent override, and no stale issue-level environment authority surviving reassignment. ### Steps to reproduce 1. Configure two agents to use different execution environments. 2. Assign an issue to the first agent so the issue records execution state in that environment. 3. Reassign the same issue to the second agent and run another heartbeat. 4. Observe that the pre-fix runtime can still sync or execute in the original sandbox instead of the second agent's environment. ### Paperclip version or commit Current `master` before this PR. ### Deployment mode Self-hosted server. ### Installation method Built from source (`pnpm dev` / `pnpm build`). ### Agent adapter(s) involved - Claude Code - Not adapter-specific (core bug in environment authority / resolution) ### Database mode External Postgres. ### Access context Both board reassignment and agent heartbeats were involved. ## What Changed - Moved environments and their default selection contract to instance scope in DB/shared types, including the migration that dedupes legacy per-company environments and seeds the instance local default. - Reworked environment CRUD/auth flows to use instance-scoped APIs and added route/service coverage for instance-level environment management. - Changed runtime resolution to prefer `agent default -> instance default -> built-in local`, removed issue-level environment stamping from the active execution path, and isolated sandbox/plugin leases by `(executionWorkspaceId, agentId)`. - Added environment env-var runtime precedence so environment-provided values act as the baseline for agent execution. - Moved the environment UI into instance settings and updated agent configuration surfaces to reflect inherit/override behavior. - Added regression coverage for instance-default inheritance across companies and for the new runtime resolution behavior. - Fixed a rebase-only duplicate `enableTaskWatchdogs` flag regression in instance settings types/validators/services so the branch typechecks cleanly on current `master`. - Updated stale server tests so CI matches the shipped instance-scoped environment contract. ## Verification - `git diff --check` - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/db typecheck` - `pnpm exec vitest run server/src/__tests__/environment-runtime-driver-contract.test.ts server/src/__tests__/agent-permissions-routes.test.ts server/src/__tests__/environment-routes.test.ts server/src/__tests__/environment-instance-routes.test.ts server/src/__tests__/execution-workspace-policy.test.ts server/src/__tests__/heartbeat-plugin-environment.test.ts server/src/__tests__/instance-settings-routes.test.ts` ## Risks - The migration changes environment scope and dedupes existing rows, so installs with unusual legacy environment combinations should be reviewed carefully during upgrade. - Remote execution behavior now depends on instance-default inheritance semantics instead of issue-level stamping, so any remaining code paths that still assume issue-scoped environment authority would surface as follow-up bugs. - This PR includes both server/runtime behavior and UI relocation, so reviewers should watch for authorization edge cases around instance settings and environment management. > I checked [`ROADMAP.md`](ROADMAP.md). This work fits the existing Cloud / Sandbox agents direction as a bug-fix/refactor to current behavior, not a new parallel product surface. ## Model Used - OpenAI Codex coding agent in this Paperclip/Codex session; GPT-5-class tool-using model with code execution and shell access. The exact backend model ID is not exposed to the session runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-06-20 09:42:53 -07:00
Devin FoleyandPaperclip	04173b341d	fix: resolve secret refs before sandbox draft probes (#8256 ) ## Thinking Path > - Paperclip is the control plane operators use to manage agent execution environments, including plugin-declared sandbox providers. > - The failing user path here was `Test draft` for an unsaved sandbox environment using a schema field marked `format: "secret-ref"`. > - Saved environments already resolve secret refs before provider use, but the unsaved probe path was forwarding the selected secret UUID directly to the provider, which made Novita draft probes fail. > - Fixing that safely required a probe-only secret resolution path with explicit actor authorization and audit context, because an unsaved draft has no persisted environment binding to authorize against. > - Once that was fixed, CI and review surfaced follow-up hardening work: preserve actor source through the draft-probe path, prevent late heartbeat finalization from overwriting already-terminal runs, avoid duplicate successful-run handoff wakes for comment-driven runs, make SSH git ref updates tolerate concurrent managed-runtime restores, and keep the skills catalog build from failing on transient GitHub errors for pinned references. > - The result is that Novita draft probes now behave like saved environments, the new secret access path is constrained and audited, and the PR is green end-to-end with Greptile at 5/5. ## Linked Issues or Issue Description No matching public GitHub issue was found after searching open and closed Paperclip issues for `novita`. Related PR search found [#8255](https://github.com/paperclipai/paperclip/pull/8255), but it addresses Novita/dev-SDK linking rather than this draft probe bug. Bug summary: - What happened? When a board user configured a sandbox environment backed by a schema-driven plugin provider such as Novita, selecting an existing company secret for `apiKey` and clicking `Test draft` failed because the probe received the secret UUID instead of the resolved secret value. - Expected behavior `Test draft` should resolve secret-ref fields before calling the provider probe, just like the saved runtime path does. - Steps to reproduce 1. Open `Company Settings -> Environments`. 2. Create or edit a `Sandbox` environment using a provider with a `format: "secret-ref"` field such as `Novita Agent Sandbox`. 3. Select an existing company secret for `apiKey`. 4. Click `Test draft`. 5. Observe the probe failure before this patch. - Paperclip version or commit Reproduced on a local `master` dev checkout; fixed and verified on branch commit `ed982d0c0`. - Deployment mode Local dev (`pnpm dev`). - Installation method Built from source (`pnpm dev` / `pnpm build`). - Agent adapter(s) involved Not adapter-specific in the core bug path; affects schema-driven sandbox provider plugins such as Novita. - Database mode Not database-related. - Access context Board (human operator). - Node.js version `v25.6.1`. - Operating system `macOS 15.7.4`. - Relevant logs or output The user-visible failure was `Novita sandbox probe failed` during `Test draft`. ## What Changed - Resolved schema-marked secret-ref fields during unsaved sandbox environment probes by adding a dedicated probe-time secret resolution path in `environment-config.ts`. - Passed `companyId` plus the full authenticated actor context into the draft probe normalization route so secret resolution stays company-scoped, authorized, and auditable. - Hardened ephemeral secret resolution so unsaved probes require `secrets:read`, preserve the original actor source (`local_implicit`, `agent_jwt`, etc.), and emit usable audit metadata. - Added a conditional heartbeat run-status update so late adapter completions cannot overwrite runs that were already cancelled or otherwise terminal. - Skipped successful-run handoff synthesis for comment-driven wakes, which removes the extra wake/run that was breaking `heartbeat-comment-wake-batching`. - Retried managed-runtime SSH git ref updates on concurrent ref-lock races instead of failing the restore path. - Reused the previous skills-catalog manifest entry when a pinned GitHub reference fails with a recoverable transient error during CI catalog generation. - Added focused regression coverage for the draft probe, ephemeral secret access, heartbeat handoff behavior, SSH ref-lock races, and catalog fallback behavior. ## Verification - `pnpm vitest run server/src/__tests__/environment-routes.test.ts` - `pnpm vitest run server/src/__tests__/secrets-service.test.ts` - `pnpm vitest run server/src/__tests__/heartbeat-comment-wake-batching.test.ts` - `pnpm vitest run server/src/services/recovery/successful-run-handoff.test.ts` - `pnpm vitest run server/src/__tests__/openclaw-gateway-adapter.test.ts` - `pnpm exec vitest run packages/adapter-utils/src/ssh-fixture.test.ts -t "merges concurrent remote commits through the managed runtime restore path"` - `pnpm exec vitest run packages/skills-catalog/src/catalog-builder.test.ts` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/skills-catalog build` - `pnpm --filter @paperclipai/adapter-utils build` - `gh pr checks 8256` - Manual/live validation: the same fix was cherry-picked into the running local dev checkout and the user re-tested the Novita `Test draft` flow successfully after the server restart. ## Risks - Low risk: the Novita-specific user-facing fix is isolated to unsaved sandbox draft probes for plugin schema fields marked `format: "secret-ref"`. - The new ephemeral secret resolution path is intentionally stricter than the original broken behavior; regressions would most likely show up as denied draft probes rather than accidental secret exposure. - The heartbeat, SSH, and catalog changes are all defensive; if they regress, they should affect test/CI orchestration paths rather than persisted company data. ## Model Used - OpenAI Codex Local (`codex_local` in Paperclip). The runtime does not expose the exact backend model ID in agent metadata. GPT-5-class coding model with shell/tool use, repository editing, test execution, GitHub review handling, and issue-thread coordination. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-06-18 07:48:09 -07:00
Aron PrinsandDevin Foley	70b1a9109d	Improve CLI API parity coverage (#6626 ) ## Thinking Path > - Paperclip is a control plane for AI-agent companies, with the CLI acting as a scriptable operator and agent interface to that control plane. > - The REST API surface has grown across companies, agents, issues, routines, plugins, auth, workspaces, secrets, and operational inspection commands. > - The CLI had drifted from that API surface: some commands were missing, some command shapes differed from docs/reference material, and several edge cases only failed during end-to-end local-source testing. > - The local development runbook requires these tests to be disposable and isolated from a real `~/.paperclip`, `~/.codex`, or `~/.claude` installation. > - This pull request adds broad CLI/API parity coverage, fixes the actionable bugs found during that pass, and records the reproducible test log under `doc/logs`. > - The benefit is a more complete, scriptable CLI surface with regression coverage for the command families exercised by the parity run. ## What Changed - Added or expanded CLI command coverage for access/auth, companies, agents, projects, goals, issues and subresources, routines, plugins, workspaces, activity/run/cost/dashboard inspection, assets, skills, secrets, tokens, prompt/wake flows, and local setup helpers. - Fixed CLI/API parity bugs found during the run, including context profile patching, issue interaction optional payloads, malformed tree-hold errors, environment duplicate handling, configure invalid-section exit codes, worktree pnpm invocation, token agent ID resolution, plugin tool worker lookup, and routine webhook secret cleanup. - Added missing CLI wrappers and route coverage for health/access, invite resolution URL forwarding, join status normalization, secret lifecycle commands, LLM docs routes, available-skill isolation, positive board-claim coverage, and interactive `connect` prompt-flow tests. - Added a schema-backed `/api/openapi.json` route sufficient for CLI parity and `paperclipai openapi --json` smoke coverage. - Added `doc/logs/2026-05-24-cli-api-parity-e2e-log.md` with the detailed living test/bug log and renamed the log directory from `doc/bugs` to `doc/logs`. - Added `doc/plans/2026-05-23-cli-api-parity.md` and the OpenAPI parity reference used during the pass. OpenAPI note: this PR intentionally does not try to subsume `feature/openapi-spec`. The OpenAPI implementation here is schema-backed and better than the earlier route-inventory stub, but `feature/openapi-spec` is the fuller/better OpenAPI branch because it includes exact mounted-route coverage tests and additional current route coverage. That branch should stay as its own PR and can supersede this OpenAPI route implementation. ## Verification Targeted automated checks run: - `pnpm exec vitest run server/src/__tests__/openapi-routes.test.ts` - `pnpm exec vitest run server/src/__tests__/board-claim.test.ts` - `pnpm exec vitest run cli/src/__tests__/connect.test.ts` - `pnpm exec vitest run cli/src/__tests__/agent-lifecycle.test.ts` - `pnpm exec vitest run server/src/__tests__/plugin-database.test.ts` - `pnpm exec vitest run server/src/__tests__/routines-service.test.ts` - `pnpm --dir cli typecheck` - `pnpm --dir server typecheck` Manual/local E2E verification: - Ran the full disposable local-source CLI/API parity pass with isolated `PAPERCLIP_HOME`, `PAPERCLIP_CONFIG`, `PAPERCLIP_CONTEXT`, `PAPERCLIP_AUTH_STORE`, `CODEX_HOME`, and `CLAUDE_HOME` under `tmp/cli-api-parity`. - Verified `DATABASE_URL` and `DATABASE_MIGRATION_URL` stayed unset for the scratch server. - Verified live health and schema-backed OpenAPI responses on non-default port `3197`. - Revoked created board/agent tokens and cleaned up temporary plugins, secrets, non-default environments, and project workspaces. - See `doc/logs/2026-05-24-cli-api-parity-e2e-log.md` for the full command-by-command reproduction log. Not run: - Full `pnpm test`, `pnpm test:run`, or `pnpm build` were not run after the entire branch because the branch is broad and the parity pass used focused test/typecheck verification plus live isolated CLI reruns. ## Risks - This is a broad PR and touches many CLI command modules, so review surface is high. The changes are grouped around one theme, but a split may be easier if maintainers prefer narrower PRs. - The OpenAPI route in this PR is not the final/best OpenAPI implementation. `feature/openapi-spec` has stronger exact-route coverage and should remain the source for the dedicated OpenAPI PR. - The living log is intentionally detailed and large. It is useful for reproducibility but adds documentation weight. - No UI changes are intended; screenshots are not applicable. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5-based coding agent in Codex desktop. Exact served model/context-window identifier was not exposed in the local app. Work used shell/Git/GitHub CLI tooling, local source inspection, targeted test execution, and live isolated Paperclip CLI/API smoke testing. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Devin Foley <devin@devinfoley.com>	2026-06-02 17:13:29 -07:00
Dotta Paperclip Claude Sonnet 4.6	778e775c35	Add secrets provider vaults and remote import (#5429 ) ## Thinking Path > - Paperclip orchestrates AI-agent companies and needs secrets handling to work across local development, hosted operators, and governed agent execution. > - The affected subsystem is the company-scoped secrets control plane: database schema, server services/routes, CLI workflows, and the Secrets settings UI. > - The gap was that secrets were local-only and operators could not manage provider vaults or import existing remote references without exposing plaintext. > - This branch adds provider vault configuration plus an AWS Secrets Manager remote-import path while preserving company boundaries, binding context, and audit trails. > - I kept the PR to a single branch PR, removed unrelated lockfile/package drift, rebased the full branch onto the current `public-gh/master`, and addressed fresh Greptile findings. > - The benefit is a reviewable implementation of provider-backed secrets with focused tests covering provider selection, import conflicts, deleted secret reuse, rotation guards, and AWS signing behavior. ## What Changed - Added provider vault support for company secrets, including provider config storage, default vault handling, health checks, binding usage, access events, and remote import preview/commit. - Added an AWS Secrets Manager provider using SigV4 request signing, bounded request timeouts, namespace guardrails, cached runtime credential resolution, and external-reference linking without plaintext reads. - Added Secrets UI surfaces for vault management and remote import, plus CLI/API documentation for setup and operations. - Stabilized routine webhook secret binding paths and SSH environment-driver fixture bindings discovered during verification. - Addressed Greptile and CI findings: no lockfile/package drift, monotonic migration metadata, disabled-vault default races, soft-deleted secret hiding/recreate behavior, remove behavior with disabled vaults, soft-deleted external-reference re-import, non-active rotation guards, managed-secret soft deletion through PATCH, and per-call AWS SDK credential client churn. - Rebased this branch onto `public-gh/master` at `0e1a5828` and force-pushed with lease to keep this as the single PR for the branch. ## Verification - `git fetch public-gh master` - `git rebase public-gh/master` - `git diff --name-only public-gh/master...HEAD \| grep '^pnpm-lock\.yaml$' \|\| true` confirmed `pnpm-lock.yaml` is not in the PR diff. - Confirmed migration ordering: master ends at `0081_optimal_dormammu`; this PR adds `0082_dry_vision` and `0083_company_secret_provider_configs`. - Inspected migrations for repeat safety: new tables/indexes use `IF NOT EXISTS`; foreign keys are guarded by `DO $$ ... IF NOT EXISTS`; column additions use `ADD COLUMN IF NOT EXISTS`. - `pnpm -r typecheck` passed before the Greptile follow-up commits. - `pnpm test:run` ran the full stable Vitest path before the Greptile follow-up commits; it completed with 3 timing-related failures under parallel load: `codex-local-execute.test.ts`, `cursor-local-execute.test.ts`, and `environment-service.test.ts`. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/codex-local-execute.test.ts src/__tests__/cursor-local-execute.test.ts src/__tests__/environment-service.test.ts` passed on targeted rerun (`24/24`). - `pnpm build` passed before the Greptile follow-up commits. Vite reported existing chunk-size/dynamic-import warnings. - After Greptile follow-up commits: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/secrets-service.test.ts` passed (`26/26`). - After Greptile follow-up commits: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/aws-secrets-manager-provider.test.ts src/__tests__/secrets-service.test.ts` passed (`39/39`). - After Greptile follow-up commits: `pnpm --filter @paperclipai/server typecheck` passed. - Captured Storybook screenshots from `ui/storybook-static` for visual review. - Latest PR checks on `5ca3a5cf`: `policy`, serialized server suites 1/4-4/4, `Canary Dry Run`, `e2e`, `security/snyk`, and `Greptile Review` pass; aggregate `verify` is still registering the completed child checks. - Greptile review loop continued through the latest requested pass; all Greptile review threads are resolved and the latest `Greptile Review` check on `5ca3a5cf` passed with 0 comments added. ## Screenshots Before: the provider-vault and remote-import surfaces did not exist on `master`; these are after-state screenshots from the Storybook fixtures. ![Secrets inventory](https://raw.githubusercontent.com/paperclipai/paperclip/PAP-2339-secrets-make-a-plan/doc/pr/5429/secrets-inventory.png) ![Secret binding picker](https://raw.githubusercontent.com/paperclipai/paperclip/PAP-2339-secrets-make-a-plan/doc/pr/5429/secret-binding-picker.png) ![Environment editor with secrets](https://raw.githubusercontent.com/paperclipai/paperclip/PAP-2339-secrets-make-a-plan/doc/pr/5429/env-editor-with-secrets.png) ## Risks - Migration risk: this adds new secret provider tables and extends existing secret rows. The migrations were checked for monotonic ordering and idempotent guards, but reviewers should still inspect upgrade behavior carefully. - Provider risk: AWS support uses direct SigV4 requests. Automated tests cover signing, request timeouts, vault-config selection, namespace guardrails, pending-version archival, sanitized provider errors, and service-level cleanup paths. A real-vault AWS smoke test remains deployment validation for an operator with AWS credentials rather than an unverified merge blocker in this local branch. - UI risk: the Secrets page and import dialog are large new surfaces; screenshots are included above for reviewer inspection. - Verification risk: the full local stable test command hit parallel-load timing failures, although the exact failed files passed when rerun directly. - Operational risk: remote import intentionally avoids plaintext reads; operators must understand that imported external references resolve at runtime and may fail if AWS permissions change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent with local shell/tool use in the Paperclip worktree. Exact context-window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-09 18:22:17 -05:00
Devin Foley	5bd0f578fd	Generalize sandbox provider core for plugin-only providers (#4449 ) ## Thinking Path > - Paperclip is a control plane, so optional execution providers should sit at the plugin edge instead of hardcoding provider-specific behavior into core shared/server/ui layers. > - Sandbox environments are already first-class, and the fake provider proves the built-in path; the remaining gap was that real providers still leaked provider-specific config and runtime assumptions into core. > - That coupling showed up in config normalization, secret persistence, capabilities reporting, lease reconstruction, and the board UI form fields. > - As long as core knew about those provider-shaped details, shipping a provider as a pure third-party plugin meant every new provider would still require host changes. > - This pull request generalizes the sandbox provider seam around schema-driven plugin metadata and generic secret-ref handling. > - The runtime and UI now consume provider metadata generically, so core only special-cases the built-in fake provider while third-party providers can live entirely in plugins. ## What Changed - Added generic sandbox-provider capability metadata so plugin-backed providers can expose `configSchema` through shared environment support and the environments capabilities API. - Reworked sandbox config normalization/persistence/runtime resolution to handle schema-declared secret-ref fields generically, storing them as Paperclip secrets and resolving them for probe/execute/release flows. - Generalized plugin sandbox runtime handling so provider validation, reusable-lease matching, lease reconstruction, and plugin worker calls all operate on provider-agnostic config instead of provider-shaped branches. - Replaced hardcoded sandbox provider form fields in Company Settings with schema-driven rendering and blocked agent environment selection from the built-in fake provider. - Added regression coverage for the generic seam across shared support helpers plus environment config, probe, routes, runtime, and sandbox-provider runtime tests. ## Verification - `pnpm vitest --run packages/shared/src/environment-support.test.ts server/src/__tests__/environment-config.test.ts server/src/__tests__/environment-probe.test.ts server/src/__tests__/environment-routes.test.ts server/src/__tests__/environment-runtime.test.ts server/src/__tests__/sandbox-provider-runtime.test.ts` - `pnpm -r typecheck` ## Risks - Plugin sandbox providers now depend more heavily on accurate `configSchema` declarations; incorrect schemas can misclassify secret-bearing fields or omit required config. - Reusable lease matching is now metadata-driven for plugin-backed providers, so providers that fail to persist stable metadata may reprovision instead of resuming an existing lease. - The UI form is now fully schema-driven for plugin-backed sandbox providers; provider manifests without good defaults or descriptions may produce a rougher operator experience. ## Model Used - OpenAI Codex via `codex_local` - Model ID: `gpt-5.4` - Reasoning effort: `high` - Context window observed in runtime session metadata: `258400` tokens - Capabilities used: terminal tool execution, git, and local code/test inspection ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge	2026-04-24 18:03:41 -07:00
Devin Foley	70679a3321	Add sandbox environment support (#4415 ) ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - The environment/runtime layer decides where agent work executes and how the control plane reaches those runtimes. > - Today Paperclip can run locally and over SSH, but sandboxed execution needs a first-class environment model instead of one-off adapter behavior. > - We also want sandbox providers to be pluggable so the core does not hardcode every provider implementation. > - This branch adds the Sandbox environment path, the provider contract, and a deterministic fake provider plugin. > - That required synchronized changes across shared contracts, plugin SDK surfaces, server runtime orchestration, and the UI environment/workspace flows. > - The result is that sandbox execution becomes a core control-plane capability while keeping provider implementations extensible and testable. ## What Changed - Added sandbox runtime support to the environment execution path, including runtime URL discovery, sandbox execution targeting, orchestration, and heartbeat integration. - Added plugin-provider support for sandbox environments so providers can be supplied via plugins instead of hardcoded server logic. - Added the fake sandbox provider plugin with deterministic behavior suitable for local and automated testing. - Updated shared types, validators, plugin protocol definitions, and SDK helpers to carry sandbox provider and workspace-runtime contracts across package boundaries. - Updated server routes and services so companies can create sandbox environments, select them for work, and execute work through the sandbox runtime path. - Updated the UI environment and workspace surfaces to expose sandbox environment configuration and selection. - Added test coverage for sandbox runtime behavior, provider seams, environment route guards, orchestration, and the fake provider plugin. ## Verification - Ran locally before the final fixture-only scrub: - `pnpm -r typecheck` - `pnpm test:run` - `pnpm build` - Ran locally after the final scrub amend: - `pnpm vitest run server/src/__tests__/runtime-api.test.ts` - Reviewer spot checks: - create a sandbox environment backed by the fake provider plugin - run work through that environment - confirm sandbox provider execution does not inherit host secrets implicitly ## Risks - This touches shared contracts, plugin SDK plumbing, server runtime orchestration, and UI environment/workspace flows, so regressions would likely show up as cross-layer mismatches rather than isolated type errors. - Runtime URL discovery and sandbox callback selection are sensitive to host/bind configuration; if that logic is wrong, sandbox-backed callbacks may fail even when execution succeeds. - The fake provider plugin is intentionally deterministic and test-oriented; future providers may expose capability gaps that this branch does not yet cover. ## Model Used - OpenAI Codex coding agent on a GPT-5-class backend in the Paperclip/Codex harness. Exact backend model ID is not exposed in-session. Tool-assisted workflow with shell execution, file editing, git history inspection, and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge	2026-04-24 12:15:53 -07:00
Devin Foley	e4995bbb1c	Add SSH environment support (#4358 ) ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - The environments subsystem already models execution environments, but before this branch there was no end-to-end SSH-backed runtime path for agents to actually run work against a remote box > - That meant agents could be configured around environment concepts without a reliable way to execute adapter sessions remotely, sync workspace state, and preserve run context across supported adapters > - We also need environment selection to participate in normal Paperclip control-plane behavior: agent defaults, project/issue selection, route validation, and environment probing > - Because this capability is still experimental, the UI surface should be easy to hide and easy to remove later without undoing the underlying implementation > - This pull request adds SSH environment execution support across the runtime, adapters, routes, schema, and tests, then puts the visible environment-management UI behind an experimental flag > - The benefit is that we can validate real SSH-backed agent execution now while keeping the user-facing controls safely gated until the feature is ready to come out of experimentation ## What Changed - Added SSH-backed execution target support in the shared adapter runtime, including remote workspace preparation, skill/runtime asset sync, remote session handling, and workspace restore behavior after runs. - Added SSH execution coverage for supported local adapters, plus remote execution tests across Claude, Codex, Cursor, Gemini, OpenCode, and Pi. - Added environment selection and environment-management backend support needed for SSH execution, including route/service work, validation, probing, and agent default environment persistence. - Added CLI support for SSH environment lab verification and updated related docs/tests. - Added the `enableEnvironments` experimental flag and gated the environment UI behind it on company settings, agent configuration, and project configuration surfaces. ## Verification - `pnpm exec vitest run packages/adapters/claude-local/src/server/execute.remote.test.ts packages/adapters/cursor-local/src/server/execute.remote.test.ts packages/adapters/gemini-local/src/server/execute.remote.test.ts packages/adapters/opencode-local/src/server/execute.remote.test.ts packages/adapters/pi-local/src/server/execute.remote.test.ts` - `pnpm exec vitest run server/src/__tests__/environment-routes.test.ts` - `pnpm exec vitest run server/src/__tests__/instance-settings-routes.test.ts` - `pnpm exec vitest run ui/src/lib/new-agent-hire-payload.test.ts ui/src/lib/new-agent-runtime-config.test.ts` - `pnpm -r typecheck` - `pnpm build` - Manual verification on a branch-local dev server: - enabled the experimental flag - created an SSH environment - created a Linux Claude agent using that environment - confirmed a run executed on the Linux box and synced workspace changes back ## Risks - Medium: this touches runtime execution flow across multiple adapters, so regressions would likely show up in remote session setup, workspace sync, or environment selection precedence. - The UI flag reduces exposure, but the underlying runtime and route changes are still substantial and rely on migration correctness. - The change set is broad across adapters, control-plane services, migrations, and UI gating, so review should pay close attention to environment-selection precedence and remote workspace lifecycle behavior. ## Model Used - OpenAI Codex via Paperclip's local Codex adapter, GPT-5-class coding model with tool use and code execution in the local repo workspace. The local adapter does not surface a more specific public model version string in this branch workflow. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge	2026-04-23 19:15:22 -07:00