mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-28 04:02:18 +01:00
8a1490b15384ceae2fc7a30cfd381fc5cc4fedc8
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Devin FoleyandPaperclip
|
c48feee190 |
Improve live agent feedback during sandboxed runs (#8915)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A core part of that experience is watching active agent runs without dropping into raw logs first > - Local and sandbox-backed adapters already record useful run output, progress, and tool activity > - But active issue threads could sit visually stale while the agent was syncing workspaces, tailing sandbox output, or emitting incremental tool-call updates > - Operators need timely, human-readable progress while preserving the raw transcript underneath > - This pull request streams sandbox run-log progress into runtime status, keeps visible issue threads refreshed, and folds repeated ACPX tool updates into stable transcript cards > - The benefit is that long-running agent work becomes easier to supervise without changing the task/comment control-plane model ## Linked Issues or Issue Description No public GitHub issue exists for this exact change. Problem/motivation: - During long-running sandboxed agent work, the issue UI can appear idle even though the agent is actively syncing, running tools, or producing incremental output. - Operators need realtime feedback at the issue-thread layer, not only after opening raw logs or waiting for the final heartbeat result. - Related public context: #1808 previously added live-run status dots to Projects; #4362 touches heartbeat wakeup behavior but is not a duplicate of this runtime/UI feedback change. ## What Changed - Added sandbox run-log streaming support and defaulted sandbox-capable local adapters into the richer live-feedback path. - Surfaced environment/sandbox sync progress through heartbeat runtime status with bounded, redacted snippets. - Added live issue-thread cache patching so visible active runs update as progress events arrive. - Folded repeated ACPX `tool_call` updates into one transcript card instead of stacking duplicate cards. - Updated adapter docs and added focused regression coverage for sandbox log streaming, runtime status, ACPX parsing, live updates, transcript rendering, and issue chat messages. ## Verification - `pnpm install --frozen-lockfile` - `pnpm exec vitest run ui/src/context/LiveUpdatesProvider.test.ts` - `pnpm exec vitest run server/src/services/heartbeat-run-runtime-status.test.ts server/src/__tests__/heartbeat-runtime-state.test.ts ui/src/context/LiveUpdatesProvider.test.ts` - `pnpm exec vitest run packages/adapter-utils/src/execution-target-sandbox.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts server/src/services/heartbeat-run-runtime-status.test.ts server/src/__tests__/agent-live-run-routes.test.ts server/src/__tests__/heartbeat-runtime-state.test.ts packages/adapters/acpx-local/src/ui/parse-stdout.test.ts ui/src/context/LiveUpdatesProvider.test.ts ui/src/components/transcript/RunTranscriptView.test.tsx ui/src/lib/issue-chat-messages.test.ts ui/src/components/IssueChatThread.test.tsx` - GitHub PR workflow on head `8397953e7b41ccd42e5d9457ee7e4dfb996e4ec5`: `verify`, build, typecheck/release-registry, e2e, general shards, serialized server shards, and canary dry run passed. - Greptile Review on head `8397953e7b41ccd42e5d9457ee7e4dfb996e4ec5`: Confidence Score 5/5, no unresolved review threads. ## Risks - Live issue-thread cache patching could miss an edge case for a route shape not covered by tests. - Surfacing active-run snippets needs continued care around redaction; this PR keeps snippets bounded and adds redaction-focused coverage. - More frequent active-run UI refreshes could expose performance issues on very large issue threads, though updates are scoped to visible run/query caches. ## Model Used OpenAI GPT-5 via Codex, operating as a tool-enabled coding agent with shell, git, and repository-editing capabilities. Context window size is not exposed in this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
bac7307ec4 |
fix(adapter-utils): improve sandbox restore failure diagnostics (#8903)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent adapters share managed-runtime helpers from `@paperclipai/adapter-utils` so local and sandboxed runs can prepare, execute, and restore workspaces consistently. > - The sandbox managed runtime syncs workspaces in both directions by creating tar archives on the local host and inside the remote sandbox. > - A prior fix avoided archiving `.` during upload because tar self-entries can force chmod/utime on a directory the command user does not own. > - The restore/download path still archived the remote workspace as `.`, and failed managed commands only surfaced stderr. > - This pull request applies entry-based tar creation to the sandbox restore path and includes stdout in managed command failure diagnostics. > - The benefit is that restore failures become visible to operators and sandbox workspace restore avoids the same directory-metadata failure class already fixed for upload. ## Linked Issues or Issue Description No existing public issue found; describing in-PR (bug). - **What happens:** sandbox workspace restore can fail while creating `workspace-download.tar` from the remote workspace when the tar command includes a `.` self-entry and the command user cannot update metadata on the workspace directory. If the failing command writes its diagnostic to stdout, the managed runtime error can collapse to a generic failed shell command without the useful tar message. - **Expected behavior:** restore should archive the workspace entries without a `.` self-entry, and failed managed runtime commands should include useful stdout/stderr diagnostics. - **Where:** `packages/adapter-utils/src/sandbox-managed-runtime.ts` restore/download path and `packages/adapter-utils/src/command-managed-runtime.ts` command error formatting. - **Related public context:** #7836 fixed the upload side of the same tar self-entry failure class. ## What Changed - Added stdout-aware failed-command formatting in the command managed runtime, keeping diagnostics bounded to the tail of stdout/stderr. - Added remote workspace tarball creation that names top-level entries explicitly instead of archiving `.` during sandbox restore. - Preserved empty-workspace restore support by creating a valid empty tarball when the remote workspace has no entries. - Added regression coverage for stdout diagnostics, restore tar members, and empty workspace restore tarballs. ## Verification - `pnpm vitest run packages/adapter-utils/src/command-managed-runtime.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — 2 files, 17 tests passed. - `pnpm --filter @paperclipai/adapter-utils typecheck` — passed. - `git diff --check origin/master..HEAD` — passed. - Local public-hygiene/PII scan of the committed diff checked for internal ticket refs, local/private URLs, common token/key patterns, private-key blocks, and email-like values — passed. - GitHub duplicate/related search found no public issue or PR for `workspace-download.tar Permission denied` or `sandbox restore tar permission denied`; #7836 is linked as related prior work. - PR CI on commit `3ad4c8d` — all GitHub Actions lanes, security scans, and aggregate `verify` passed. - Greptile Review on commit `3ad4c8d` — Confidence Score 5/5; the prior P2 thread is resolved with no open P2s, recommendations, or follow-ups. ## Risks Low. This is limited to shared adapter runtime error formatting and sandbox restore archive construction. Archive contents should remain equivalent apart from the removed `.` self-entry, and the new diagnostics are bounded to avoid dumping unbounded command output. ## Model Used OpenAI Codex, GPT-5, tool-enabled coding agent with shell and GitHub CLI access. Context window size was not reported by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (n/a; internal adapter-runtime behavior only) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
721541c41d |
Fix sandbox restore index drift (#8595)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandboxed agents can do useful work in a remote workspace, then export that result back to a host review checkout. > - A restore can leave the host checkout's index stale even when the sandbox content and committed state are correct. > - That drift makes reviewers see misleading missing-file or dirty-state results after sandbox execution. > - This pull request repairs the host index refresh path and surfaces a clean/dirty restore signal during finalization. > - The benefit is more reliable review checkouts and clearer sandbox finalization status. ## Linked Issues or Issue Description Refs #248 No exact public GitHub issue was found for this restore/index consistency fix. The underlying bug is that a sandbox restore can update host working-tree content without leaving the host git index aligned, so local review tooling may report stale deletions or misleading dirtiness. This PR keeps the restore/export path consistent and reports the resulting clean/dirty state. GitHub search performed for related or duplicate work: `sandbox runtime status`, `sandbox restore index`, and `runtime progress`. No direct duplicate PR was found. ## What Changed - Refreshed git workspace sync/index handling after sandbox restore/export operations. - Added finalization status that reports whether the restored host checkout is clean or dirty. - Preserved sandbox-managed runtime progress callbacks around restore and finalization. - Added adapter-utils tests covering the host index drift repair and clean/dirty finalization signal. ## Verification - Local PII scan before push: high-confidence secret patterns, internal issue links, local user paths, and private URL patterns checked across all three split diffs; no real secrets or internal links found. - `git diff --check feat/sandbox-runtime-status..fix/sandbox-restore-index-sync` - `pnpm exec vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — 1 file, 10 tests passed. - `pnpm run typecheck` passed on `fix/sandbox-restore-index-sync`. - `pnpm run build` passed on `fix/sandbox-restore-index-sync`; Vite reported existing CSS `::highlight` and chunk-size warnings. - `pnpm run test:run` was attempted on `fix/sandbox-restore-index-sync`; it failed in two unrelated broad-suite tests. One depends on this host's Git default branch behavior, and one depends on local Claude model-discovery environment. The changed focused suite above passes. ## Risks - Git index refresh behavior needs to remain conservative so it does not hide real uncommitted user changes. - The clean/dirty finalization signal is diagnostic; it should not by itself mark a sandbox run successful or failed. - This PR is stacked on the runtime-status branch so finalization progress can reuse the same callback path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex coding agent, with shell/tool execution in a local worktree. Exact context-window metadata is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
|
a27e5ad002 |
Add ephemeral sandbox runtime status plumbing (#8593)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandboxed agent runs can spend meaningful time preparing a remote workspace before the agent transcript shows useful output. > - Operators need short, current progress text for those setup phases, but that text should not become durable run history. > - The existing live-run websocket path already carries run updates to the UI, so the backend can reuse that channel instead of adding polling. > - This pull request adds an ephemeral runtime-progress contract, a process-local status store, and heartbeat integration for sandbox-managed runs. > - The benefit is a clearer active-run experience without database migrations or persistent progress rows. ## Linked Issues or Issue Description Refs #248 No exact public GitHub issue was found for this status-message plumbing. The underlying problem is that active sandboxed runs currently have setup phases, such as workspace sync and restore, where the operator cannot see concise current progress through the live run state. This PR addresses that gap for the backend/runtime layer while keeping progress messages ephemeral. GitHub search performed for related or duplicate work: `sandbox runtime status`, `sandbox restore index`, and `runtime progress`. No direct duplicate PR was found. ## What Changed - Added shared runtime-progress types and the `heartbeat.run.progress` live event type. - Added a process-local heartbeat run runtime-status store with TTL, bounded/redacted messages, and terminal cleanup. - Threaded runtime progress callbacks through heartbeat execution and active/live run serialization. - Emitted sandbox-managed runtime phase updates for sync, adapter startup, restore/export, and finalization paths. - Added backend and adapter-utils tests for ephemeral status behavior, terminal cleanup, live serialization, and sandbox progress callbacks. ## Verification - `pnpm install --frozen-lockfile` - Local PII scan before push: high-confidence secret patterns, internal issue links, local user paths, and private URL patterns checked across all three split diffs; no real secrets or internal links found. The only secret-like text is an intentional fake test fixture (`sk-test-secret`). - `git diff --check origin/master..feat/sandbox-runtime-status` - `pnpm exec vitest run server/src/services/heartbeat-run-runtime-status.test.ts server/src/__tests__/heartbeat-runtime-state.test.ts server/src/__tests__/agent-live-run-routes.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — 4 files, 23 tests passed. - `pnpm run typecheck` passed on both top stacks that include this branch: `feat/sandbox-status-ui` and `fix/sandbox-restore-index-sync`. - `pnpm run build` passed on both top stacks that include this branch; Vite reported existing CSS `::highlight` and chunk-size warnings. - `pnpm run test:run` was attempted on `fix/sandbox-restore-index-sync`; it failed in two unrelated broad-suite tests. One depends on this host's Git default branch behavior, and one depends on local Claude model-discovery environment. The changed focused suites above pass. ## Risks - Runtime progress is process-local by design, so status disappears after TTL, terminal cleanup, or server restart. - Clients that do not consume `heartbeat.run.progress` simply keep existing behavior. - Message redaction is intentionally generic; overly specific phase details should stay out of runtime-progress payloads. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex coding agent, with shell/tool execution in a local worktree. Exact context-window metadata is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip CTO <cto@paperclip.local> Co-authored-by: Paperclip CTO <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
51ffbb380f |
Exclude transient Codex home dirs from sandbox sync (#8581)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Local adapters can run agents against sandboxed execution targets by syncing the workspace and selected runtime assets into the sandbox. > - The Codex local adapter includes a managed Codex home asset so sandboxed Codex runs can use the expected auth, config, skills, and session state. > - That asset follows symlinks, which is useful for real Codex home content but unsafe for transient launcher directories. > - Transient `tmp` and `.tmp` directories can contain symlinks to large host binaries, so the sandbox archive can inline large executable targets instead of just the small home directory content. > - This pull request excludes transient Codex home directories from the sandbox home asset while preserving the required Codex home files. > - The benefit is a much smaller and more predictable sandbox setup upload without changing the runtime files Codex actually needs. ## Linked Issues or Issue Description No public issue was found for this exact sandbox archive-size bug. Bug description: - What happened: sandboxed `codex_local` runs sync the managed Codex home as a `home` asset with `followSymlinks` enabled. If transient Codex home dirs such as `tmp` or `.tmp` contain symlinks to a large host binary, the archive can inline that binary and make `Syncing home to sandbox` much larger than the managed home directory itself. - Expected behavior: sandbox setup should include the Codex home files needed for auth, config, skills, and session continuity, but should not archive transient launcher scratch directories. - Reproduction shape: create a managed Codex home with normal auth/config/skills files and a `tmp/arg0` or `.tmp` symlink to a large host executable, then start a sandboxed `codex_local` run. The home asset archive grows by the symlink target size. - Version/commit: observed on local `master` before this change. - Related public context: #5028 covers a different managed Codex home reliability issue around stale auth files; this PR addresses sandbox archive bloat from transient symlink targets. ## What Changed - Excluded `tmp` and `.tmp` from the Codex `home` asset that is uploaded for sandboxed runs. - Added regression coverage proving transient symlinked home dirs are excluded from the tar while required auth/config/skills files remain included. - Kept `followSymlinks` behavior for the rest of the Codex home asset so existing non-transient symlink behavior is preserved. ## Verification - `git diff --check` - Local PII/secret pattern scan over the committed diff - `pnpm exec vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts` - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/adapter-codex-local typecheck` ## Risks Low risk. The exclusion is limited to transient Codex home scratch directories, and the regression test verifies the files needed in the sandbox are still archived. The main compatibility risk is if a user intentionally placed required persistent Codex state under `tmp` or `.tmp`; those paths are treated as volatile scratch space by this change. ## Model Used OpenAI Codex coding agent based on GPT-5, with shell, git, and GitHub CLI tool use. Exact hosted model build and context-window size were not exposed in the local adapter runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
2c98c8e1e5 |
Fix sandbox git publishing and large workspace uploads (#8422)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox and SSH runtimes need to preserve agent work across isolated execution environments > - Git-backed workspaces were being copied mostly as filesystem archives, which breaks when `.git` points outside the mounted workspace and makes sandbox agents unable to publish their own branches > - Large ignored dependency trees could also be swept into the sandbox overlay, causing multi-GB transfers and max-string failures in some sandbox clients > - This pull request makes sandbox runtime setup use a git-backed HEAD sync plus a small dirty/untracked overlay, and bounds sandbox file transfers so large archives do not need one huge string > - The benefit is that sandbox agents can commit and push from a usable git checkout without uploading dependency trees such as `node_modules` ## Linked Issues or Issue Description Refs #8395 No public duplicate issue or PR was found after searches for `sandbox git workspace`, `git push sandbox`, and `node_modules sandbox upload`. Bug report: - What happened: sandbox-backed agent workspaces could receive a `.git` file that pointed at host-only git state, leaving the sandbox unable to run normal git workflows. The sandbox overlay upload could also include ignored dependency directories, creating very large transfers. - Expected behavior: sandbox and remote runtimes should prepare a usable git-backed workspace, copy only the necessary workspace overlay, and restore git history plus file changes without depending on a host-only `.git` path. - Steps to reproduce: 1. Run an agent in a sandbox-backed workspace whose local git checkout is a worktree. 2. Ask the agent to complete a GitHub workflow that requires commit/push access. 3. Observe that git operations can fail inside the sandbox, and ignored dependency trees can be uploaded as part of the workspace overlay. - Paperclip version or commit: reproduced against `master` before this PR, base `7aa212296eb1`. - Deployment mode: local dev / sandbox-backed runtime. - Installation method: built from source. - Agent adapters involved: local adapters using shared adapter-utils runtime preparation. - Database mode: not database-related. - Access context: agent runtime. - Local verification environment: Node.js v25.6.1, pnpm 9.15.4, macOS arm64. - Privacy checklist: all pasted output was reviewed for secrets, private hostnames, local usernames, and internal instance links. ## What Changed - Added a GitHub workflow push preflight so agent runs can detect missing push credentials when a workflow explicitly needs GitHub publishing. - Added shared git workspace sync helpers for shallow HEAD import/export and dirty/untracked overlay tracking. - Updated sandbox managed runtime setup to use git history plus a selected overlay instead of uploading the full local workspace for git-backed workspaces. - Bounded sandbox archive upload/download paths so large payloads stream or chunk instead of materializing one oversized string. - Excluded `.git` and ignored dependency trees from sandbox upload, download, and restore baselines while preserving local ignored directories during sync-back. - Added focused tests for git workspace sync, sandbox overlay selection, transfer chunking, and heartbeat push-preflight behavior. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/git-workspace-sync.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts packages/adapter-utils/src/command-managed-runtime.test.ts server/src/__tests__/heartbeat-project-env.test.ts server/src/__tests__/heartbeat-workspace-session.test.ts` - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm build` - Public-hygiene scan of the PR diff and commit messages for internal issue ids, local paths, private hostnames, and obvious token patterns. ## Risks - Medium risk: this changes sandbox runtime synchronization semantics for git-backed workspaces, especially around dirty tracked files, untracked files, deleted paths, and ignored files. - The main mitigation is focused test coverage for upload contents, restore exclusions, and git round-trip behavior. - The SSH runtime keeps the current bundle-based implementation from `master`; this PR only aligns shared excludes and sandbox behavior with that model. ## Model Used OpenAI Codex, GPT-5-based coding agent, tool-enabled shell/git/GitHub workflow, with code execution and repository inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
07e98d2b2c |
feat(adapter-utils): add observable sandbox sync progress (#8395)
## Thinking Path > - Paperclip is the control plane for running AI-agent companies, so long-running remote work needs to stay observable to human operators. > - Cloud / sandbox agents are an active roadmap area, and their workspace sync path is part of the runtime substrate every remote coding run depends on. > - In the sandbox and SSH execution-target flows, Paperclip logged that sync had started, then often went silent for the full transfer window. > - That made large remote syncs feel stalled and also hid a real performance problem in the command-managed sandbox upload path. > - The first part of this pull request threads a throttled progress-reporting surface through the adapter execution-target stack so sync and restore work can emit meaningful updates. > - The second part fixes the command-managed sandbox transport itself: it removes the old serial 32KB append bottleneck, but also falls back away from the single-stream path when a provider-backed sandbox runner cannot surface mid-flight stdin progress. > - The result is that sandbox and SSH transfers are both faster and more observable, including the live Daytona-style sandbox case that previously only emitted `0%` and `100%`. ## Linked Issues or Issue Description No public GitHub issue exists for this bug, so it is described inline below following the bug report template. ### What happened - Remote sandbox and SSH workspace syncs could spend a long time transferring data while only logging a start line (`Syncing workspace and runtime assets to sandbox environment`) and, at best, a terminal line. - In the command-managed sandbox path, the original upload implementation also paid a large performance cost by appending base64 data in many small sequential remote writes (thousands of serial 32KB round-trips on a large workspace). - After the initial transport rewrite, live provider-backed sandbox runs still only emitted `0%` and `100%` because the single-stream stdin RPC buffered progress until completion. ### Expected behavior - Long-running sandbox and SSH syncs should periodically report how much of the transfer is complete (a percentage and/or MB transferred) so an operator can tell the run is healthy and making progress rather than stuck. - The main sandbox upload path should not be artificially slow. - A transfer that fails partway should leave an explicit failure marker in the log rather than a dangling intermediate percentage. ### Steps to reproduce 1. Run an agent against a sandbox (command-managed) or SSH (remote-managed) execution target with a non-trivial workspace. 2. Watch the run log during the workspace/runtime asset sync phase. 3. Observe that the log shows the sync start line and then stays silent for the full transfer (live provider-backed sandbox runs only show `0%` then `100%`). ### Paperclip version or commit - Branch `PAPA-825-provide-status-updates-when-syncing-sandboxes` off `master`. ### Deployment mode - Self-hosted / local instance using sandbox (command-managed) and SSH (remote-managed) execution targets, including provider-backed sandbox runners. ## What Changed - Added shared throttled runtime progress reporting and threaded `onProgress` through the adapter execution-target surface and adapter `execute.ts` entrypoints. - Added sync and restore progress reporting for the command-managed sandbox path and the SSH/remote-managed path, including git import/export progress where totals are known. - Reworked command-managed sandbox transfer behavior so uploads use the faster single-stream path when appropriate, but fall back to chunked progress-emitting writes when the runner cannot expose mid-stream stdin progress. - Marked provider-backed environment sandbox runners as not supporting single-stream stdin progress so live sandbox runs emit meaningful intermediate updates instead of only `0%` and `100%`. - Emit an explicit terminal failure marker (`failed at NN% (x/y MB)`) when an SSH/tar transfer rejects, so a failed sync no longer leaves a dangling intermediate percentage in the log. - Run the SSH sync/restore size estimate (local directory walk / remote `du` probe) concurrently with the transfer instead of awaiting it before opening the pipe, so progress instrumentation no longer adds startup latency proportional to workspace file count. - Added and extended focused regression coverage for runtime progress throttling and the new failure marker, command-managed sandbox transfers, sandbox orchestration, SSH transfer progress, and environment execution-target wiring. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/runtime-progress.test.ts packages/adapter-utils/src/ssh-fixture.test.ts packages/adapter-utils/src/command-managed-runtime.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts` - `pnpm exec vitest run packages/adapter-utils/src/command-managed-runtime.test.ts server/src/__tests__/environment-execution-target.test.ts` - `npx tsc --noEmit` for `packages/adapter-utils` ## Risks - The provider-backed sandbox fallback now prefers chunked command-managed writes when progress hooks are active, so small-to-medium uploads may trade some raw throughput for observable intermediate progress on runtimes that cannot surface true mid-stream stdin progress. - Progress percentages on tar-based transfers still depend on estimates in some cases, so operators may briefly see MB-only lines before the estimate resolves, then near-final clamping before the terminal `100%` line. - This PR changes shared execution-target behavior used by multiple adapters, so regressions would most likely appear in remote runtime setup/teardown flows rather than in a single adapter. ## Model Used - Initial implementation: OpenAI GPT-5.4 via Codex local agent (`codex_local`), high reasoning mode. - Observability follow-ups (failure marker, concurrent size estimate, added tests): Claude Opus 4.8 via Claude Code (`claude_local`). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Jannes StubbemannandClaude Opus 4.8
|
93cdc5c1ce |
fix(adapter-utils): tar sandbox workspace by entry, not '.', to avoid EPERM on unowned target dir (#7836)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents can run in remote/sandboxed environments via the shared sandbox managed-runtime in `@paperclipai/adapter-utils` (used by SSH/E2B/Daytona and other sandbox providers), which syncs the workspace into the sandbox by tarring it up and extracting it inside the pod/host > - When the sandbox runs the harness as a non-root user whose home/workspace dir it does not own (for example a hardened, non-root, gVisor pod with an `emptyDir`-mounted workspace), the workspace upload aborts before the agent can start > - Root cause: `createTarballFromDirectory` archives `.`, embedding a `./` self-entry whose mode/mtime tar then tries to restore onto the **extraction target directory**; `chmod`/`utime` of `.` fails with `Operation not permitted` for a non-owner > - This is not specific to any one deployment: the `.` self-entry EPERM can bite every sandbox provider built on the shared managed runtime as soon as the extracting user does not own the target directory, which is the norm for hardened non-root sandboxes > - This pull request archives the directory's top-level entries by name instead of `.`, so there is no `./` self-entry and extraction never touches the target dir's metadata > - The benefit is that workspace sync works in any sandbox where the target dir is non-root or not owned by the extracting user, without GNU-only tar flags ## Linked Issues or Issue Description No existing issue; describing in-PR (bug). - **What happens:** managed sandbox runs that sync the workspace fail at upload with `tar: .: Cannot utime: Operation not permitted` / `tar: .: Cannot change mode to ... : Operation not permitted`, aborting the run before the harness starts. - **Where:** `packages/adapter-utils/src/sandbox-managed-runtime.ts`, in `createTarballFromDirectory` (archives `.`). - **When:** the extraction target directory is not owned by the (non-root) user extracting the tar inside the sandbox. - Closely related (different root cause): #6560 (E2B workspace upload + lease idle failures). ## What Changed - `createTarballFromDirectory` enumerates the directory's top-level entries with `fs.readdir` and passes them by name after `--` (guards flag-like filenames) instead of archiving `.`, eliminating the `./` self-entry that triggers the EPERM. - Empty workspaces (legitimate for blank-workspace runs) write a valid 1024-byte all-zero EOF tar instead of invoking `tar` with no paths. - `--exclude` patterns continue to apply (to nested matches and any named entry). ## Verification - `pnpm --filter @paperclipai/adapter-utils build` (tsc clean) - `pnpm exec vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts` runs green - New tests: uploaded workspace/asset tarballs contain no `.`/`./` member yet still extract correctly; empty workspace produces a valid (no-op) tarball. Existing managed-runtime sync test unchanged. - Manually verified in a hardened (non-root, gVisor) sandbox pod: with the fix, the workspace upload that previously aborted with the EPERM now succeeds. That deployment is the reproduction and verification environment; the fix itself is provider-agnostic. ## Risks Low. Behavior is unchanged for owned/root targets; the archive contents are the same minus the `./` self-entry (which tar recreates implicitly on extract). Portable across GNU/BSD/busybox tar (no GNU-only `--no-overwrite-dir`). No API/migration/UI impact. ## Model Used Claude Opus 4.8 (`claude-opus-4-8`, 1M context), extended thinking + tool use, via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work (bug fix in shared sandbox utils, not core feature work) - [x] I have searched GitHub for duplicate or related PRs and linked them above (#6560) - [x] I have either (a) linked existing issues OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots (n/a, no UI) - [ ] I have updated relevant documentation to reflect my changes (n/a, internal behavior, no docs reference this) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (the only finding was the description-template P2, resolved by this description; the latest review covers the current head with no code findings and all CI gates are green) - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
778e775c35 |
Add secrets provider vaults and remote import (#5429)
## Thinking Path > - Paperclip orchestrates AI-agent companies and needs secrets handling to work across local development, hosted operators, and governed agent execution. > - The affected subsystem is the company-scoped secrets control plane: database schema, server services/routes, CLI workflows, and the Secrets settings UI. > - The gap was that secrets were local-only and operators could not manage provider vaults or import existing remote references without exposing plaintext. > - This branch adds provider vault configuration plus an AWS Secrets Manager remote-import path while preserving company boundaries, binding context, and audit trails. > - I kept the PR to a single branch PR, removed unrelated lockfile/package drift, rebased the full branch onto the current `public-gh/master`, and addressed fresh Greptile findings. > - The benefit is a reviewable implementation of provider-backed secrets with focused tests covering provider selection, import conflicts, deleted secret reuse, rotation guards, and AWS signing behavior. ## What Changed - Added provider vault support for company secrets, including provider config storage, default vault handling, health checks, binding usage, access events, and remote import preview/commit. - Added an AWS Secrets Manager provider using SigV4 request signing, bounded request timeouts, namespace guardrails, cached runtime credential resolution, and external-reference linking without plaintext reads. - Added Secrets UI surfaces for vault management and remote import, plus CLI/API documentation for setup and operations. - Stabilized routine webhook secret binding paths and SSH environment-driver fixture bindings discovered during verification. - Addressed Greptile and CI findings: no lockfile/package drift, monotonic migration metadata, disabled-vault default races, soft-deleted secret hiding/recreate behavior, remove behavior with disabled vaults, soft-deleted external-reference re-import, non-active rotation guards, managed-secret soft deletion through PATCH, and per-call AWS SDK credential client churn. - Rebased this branch onto `public-gh/master` at `0e1a5828` and force-pushed with lease to keep this as the single PR for the branch. ## Verification - `git fetch public-gh master` - `git rebase public-gh/master` - `git diff --name-only public-gh/master...HEAD | grep '^pnpm-lock\.yaml$' || true` confirmed `pnpm-lock.yaml` is not in the PR diff. - Confirmed migration ordering: master ends at `0081_optimal_dormammu`; this PR adds `0082_dry_vision` and `0083_company_secret_provider_configs`. - Inspected migrations for repeat safety: new tables/indexes use `IF NOT EXISTS`; foreign keys are guarded by `DO $$ ... IF NOT EXISTS`; column additions use `ADD COLUMN IF NOT EXISTS`. - `pnpm -r typecheck` passed before the Greptile follow-up commits. - `pnpm test:run` ran the full stable Vitest path before the Greptile follow-up commits; it completed with 3 timing-related failures under parallel load: `codex-local-execute.test.ts`, `cursor-local-execute.test.ts`, and `environment-service.test.ts`. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/codex-local-execute.test.ts src/__tests__/cursor-local-execute.test.ts src/__tests__/environment-service.test.ts` passed on targeted rerun (`24/24`). - `pnpm build` passed before the Greptile follow-up commits. Vite reported existing chunk-size/dynamic-import warnings. - After Greptile follow-up commits: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/secrets-service.test.ts` passed (`26/26`). - After Greptile follow-up commits: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/aws-secrets-manager-provider.test.ts src/__tests__/secrets-service.test.ts` passed (`39/39`). - After Greptile follow-up commits: `pnpm --filter @paperclipai/server typecheck` passed. - Captured Storybook screenshots from `ui/storybook-static` for visual review. - Latest PR checks on `5ca3a5cf`: `policy`, serialized server suites 1/4-4/4, `Canary Dry Run`, `e2e`, `security/snyk`, and `Greptile Review` pass; aggregate `verify` is still registering the completed child checks. - Greptile review loop continued through the latest requested pass; all Greptile review threads are resolved and the latest `Greptile Review` check on `5ca3a5cf` passed with 0 comments added. ## Screenshots Before: the provider-vault and remote-import surfaces did not exist on `master`; these are after-state screenshots from the Storybook fixtures.    ## Risks - Migration risk: this adds new secret provider tables and extends existing secret rows. The migrations were checked for monotonic ordering and idempotent guards, but reviewers should still inspect upgrade behavior carefully. - Provider risk: AWS support uses direct SigV4 requests. Automated tests cover signing, request timeouts, vault-config selection, namespace guardrails, pending-version archival, sanitized provider errors, and service-level cleanup paths. A real-vault AWS smoke test remains deployment validation for an operator with AWS credentials rather than an unverified merge blocker in this local branch. - UI risk: the Secrets page and import dialog are large new surfaces; screenshots are included above for reviewer inspection. - Verification risk: the full local stable test command hit parallel-load timing failures, although the exact failed files passed when rerun directly. - Operational risk: remote import intentionally avoids plaintext reads; operators must understand that imported external references resolve at runtime and may fail if AWS permissions change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent with local shell/tool use in the Paperclip worktree. Exact context-window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Devin Foley
|
12cb7b40fd |
Harden remote workspace sync and restore flows (#5444)
## Thinking Path
> - Paperclip orchestrates AI agents for zero-human companies
> - When an agent runs against a remote target, Paperclip syncs the
workspace out to the remote at run start and restores changes back to
the local workspace at run end
> - The previous restore flow naïvely overwrote local files with
whatever the remote returned, so files that the remote run never touched
but had timestamp/mode drift could be needlessly rewritten — and a
single static `refs/paperclip/ssh-sync/imported` ref made concurrent SSH
workspace exports race on the same git ref
> - This pull request adds a `workspace-restore-merge` module that diffs
a pre-run snapshot against the post-run remote state and only writes
back files the remote actually changed; SSH workspace exports now use a
per-import unique ref so concurrent runs can't trample each other
> - Every adapter's execute path threads the snapshot through
`prepareAdapterExecutionTargetRuntime` so the merge has the baseline it
needs
> - The benefit is workspace restores no longer churn untouched files,
and concurrent SSH runs no longer collide on the import ref
## What Changed
- `packages/adapter-utils/src/workspace-restore-merge.{ts,test.ts}`: new
module — directory snapshot (kind/mode/sha256/symlink target) plus
snapshot-aware merge that writes only the files the remote changed
- `packages/adapter-utils/src/ssh.ts`: SSH workspace export uses a
per-import unique ref (`refs/paperclip/ssh-sync/imported/<uuid>`);
restore goes through the new merge helper; `ssh-fixture.test.ts` covers
the unique-ref + merge paths
- `packages/adapter-utils/src/sandbox-managed-runtime.ts` +
`remote-managed-runtime.ts`: thread the snapshot/merge through the
sandbox and SSH paths
- `packages/adapter-utils/src/server-utils.{ts,test.ts}` +
`execution-target.ts`: helpers for capturing the pre-run snapshot;
`prepareAdapterExecutionTargetRuntime` gains required `runId` and
optional `workspaceRemoteDir`, and returns the realized
`workspaceRemoteDir`
- Each adapter's `execute.ts` (acpx, claude, codex, cursor, gemini,
opencode, pi) takes the snapshot at run start and passes it through to
the runtime restore
- Remote execute test mocks updated to match the new
`prepareWorkspaceForSshExecution` return shape and the per-run
`${managedRemoteWorkspace}` cwd subdirectory
## Verification
- `pnpm vitest run --no-coverage --project @paperclipai/adapter-utils
--project @paperclipai/adapter-acpx-local --project
@paperclipai/adapter-claude-local --project
@paperclipai/adapter-codex-local --project
@paperclipai/adapter-cursor-local --project
@paperclipai/adapter-gemini-local --project
@paperclipai/adapter-opencode-local --project
@paperclipai/adapter-pi-local` — 196/196 passing
- `pnpm typecheck` clean across the workspace
## Risks
Medium. The restore path now writes a strict subset of what it
previously did — files the remote did not touch are no longer rewritten.
If any flow was relying on a touch-without-content-change being copied
back (timestamp or permission propagation only), that behavior is now
skipped. Snapshot capture adds an O(N-files-in-workspace) hash pass at
run start; the cost is bounded by the existing exclude list. The `runId`
parameter on `prepareAdapterExecutionTargetRuntime` is now required —
every in-tree caller is updated; out-of-tree adapter authors need to
pass it.
## Model Used
Claude Opus 4.7 (1M context)
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable — new module +
every adapter execute path covered
- [x] If this change affects the UI, I have included before/after
screenshots — N/A (no UI)
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
|
||
|
Devin Foley
|
a4ac6ff133 |
Add sandbox callback bridge for remote environment API access (#4801)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Agents can run inside sandboxed environments like E2B, which are isolated from the host network > - Sandboxed agents need to call back to the Paperclip API to report progress, post comments, and update issue status > - But sandbox environments cannot reach the Paperclip server directly because they run in isolated network namespaces > - This PR adds a callback bridge that proxies API requests from the sandbox to the Paperclip server, running as a local HTTP server on the host that forwards authenticated requests > - The bridge is started automatically when an adapter launches a sandbox execution, and torn down when the run completes > - The benefit is sandboxed agents can interact with the Paperclip API without requiring network-level access to the host, enabling E2B and similar providers to work end-to-end ## What Changed - Added `sandbox-callback-bridge.ts` in `packages/adapter-utils/` — a lightweight HTTP bridge server that accepts requests from sandbox environments and proxies them to the Paperclip API with authentication - Added request validation and security policy: the bridge only forwards requests to the configured API URL, validates content types, enforces size limits, and rejects non-API paths - Wired the bridge into all remote adapter execute paths (claude, codex, cursor, gemini, pi) — the bridge starts before the agent process and the bridge URL is passed via environment variables - Updated `environment-execution-target.ts` to prefer the explicit API URL from environment lease metadata for sandbox callback routing - Fixed Claude sandbox runtime setup to work with the bridge configuration - Added comprehensive test coverage for bridge request handling, policy enforcement, and sandbox execution integration - Fixed browser bundling — the bridge module is excluded from the frontend bundle via the adapter-utils index export ## Verification - `pnpm test` — all existing and new tests pass, including bridge unit tests and sandbox execution integration tests - `pnpm typecheck` — clean - Manual: configure an E2B environment, run an agent task, verify the agent can post comments and update issue status through the bridge ## Risks - Medium. This is a new network-facing component (HTTP server on localhost). The security policy restricts forwarding to the configured API URL only and validates all requests, but any proxy introduces attack surface. The bridge binds to localhost only and is scoped to the lifetime of a single agent run. ## Model Used Codex GPT 5.4 high via Paperclip. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Devin Foley
|
70679a3321 |
Add sandbox environment support (#4415)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - The environment/runtime layer decides where agent work executes and how the control plane reaches those runtimes. > - Today Paperclip can run locally and over SSH, but sandboxed execution needs a first-class environment model instead of one-off adapter behavior. > - We also want sandbox providers to be pluggable so the core does not hardcode every provider implementation. > - This branch adds the Sandbox environment path, the provider contract, and a deterministic fake provider plugin. > - That required synchronized changes across shared contracts, plugin SDK surfaces, server runtime orchestration, and the UI environment/workspace flows. > - The result is that sandbox execution becomes a core control-plane capability while keeping provider implementations extensible and testable. ## What Changed - Added sandbox runtime support to the environment execution path, including runtime URL discovery, sandbox execution targeting, orchestration, and heartbeat integration. - Added plugin-provider support for sandbox environments so providers can be supplied via plugins instead of hardcoded server logic. - Added the fake sandbox provider plugin with deterministic behavior suitable for local and automated testing. - Updated shared types, validators, plugin protocol definitions, and SDK helpers to carry sandbox provider and workspace-runtime contracts across package boundaries. - Updated server routes and services so companies can create sandbox environments, select them for work, and execute work through the sandbox runtime path. - Updated the UI environment and workspace surfaces to expose sandbox environment configuration and selection. - Added test coverage for sandbox runtime behavior, provider seams, environment route guards, orchestration, and the fake provider plugin. ## Verification - Ran locally before the final fixture-only scrub: - `pnpm -r typecheck` - `pnpm test:run` - `pnpm build` - Ran locally after the final scrub amend: - `pnpm vitest run server/src/__tests__/runtime-api.test.ts` - Reviewer spot checks: - create a sandbox environment backed by the fake provider plugin - run work through that environment - confirm sandbox provider execution does not inherit host secrets implicitly ## Risks - This touches shared contracts, plugin SDK plumbing, server runtime orchestration, and UI environment/workspace flows, so regressions would likely show up as cross-layer mismatches rather than isolated type errors. - Runtime URL discovery and sandbox callback selection are sensitive to host/bind configuration; if that logic is wrong, sandbox-backed callbacks may fail even when execution succeeds. - The fake provider plugin is intentionally deterministic and test-oriented; future providers may expose capability gaps that this branch does not yet cover. ## Model Used - OpenAI Codex coding agent on a GPT-5-class backend in the Paperclip/Codex harness. Exact backend model ID is not exposed in-session. Tool-assisted workflow with shell execution, file editing, git history inspection, and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
