mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-24 18:19:05 +02:00
bd059a073dc6cbf2ac2f0295ecc5a0b8aab60e5a
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Devin Foley
|
393da0f67c |
fix(adapter-utils): graft unrelated imported histories instead of failing the run (#11638)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - At run finalize, the host imports the sandbox git history and reconciles it with the local worktree in `integrateImportedGitHead` > - Transported workspaces are depth-1 shallow clones, so the boundary commit reads as parentless inside the sandbox > - A `git commit --amend` there rewrites the boundary commit into a root commit, and the re-imported history no longer connects to the host history > - `git merge-tree` has no common base to merge against, so the sync throws "Failed to merge concurrent remote git histories" and the run fails with its work stranded in the sandbox > - This pull request grafts the imported tree onto the current head as a single commit instead of failing > - The benefit is that a history rewrite inside the sandbox can no longer lose a run's work ## Linked Issues or Issue Description No existing issue found. I searched issues and PRs for "unrelated histories", "Failed to merge concurrent", and "shallow". Depends on #11637 (merged; the graft commit reuses its identity constant). This PR is now rebased onto `master`. **What happened?** An agent run amended a commit inside its sandbox workspace to address review feedback. The sandbox clone is depth-1 shallow, so git treated the boundary commit as parentless and the amend produced a root commit. At finalize, the host-side sync failed with `Failed to merge concurrent remote git histories for <sha>` and the run was marked failed. A follow-up run had to repair the branch by hand: fetch the true parent from origin and rebuild the commit with `git commit-tree`. **Expected behavior** The sync must never strand completed work. When the imported history shares no ancestor with the local one, the imported tree should still land on the current head, with the imported message preserved and the graft recorded. **Steps to reproduce** 1. Start a run whose workspace transport uses the shallow clone path (`withShallowGitWorkspaceClone`, depth 1). 2. Inside the sandbox workspace, run `git commit --amend` on the boundary commit. The result is a parentless root commit. 3. Finish the run. The host-side `integrateImportedGitHead` finds no merge base, `merge-tree` fails, and the run fails. ## What Changed - `git-workspace-sync.ts`: new exported `createUnrelatedHistoryGraftCommit` helper. It reads the imported head's tree and message, and creates one commit on top of the current head with the deterministic sync identity and a trailer that records the graft and both shas. - `integrateImportedGitHead` (both the remote-git-sync version and the SSH copy in `ssh.ts`): when `merge-base` reports no common ancestor, graft instead of throwing. The ref update keeps the same compare-and-swap and concurrent-retry semantics as the merge path. - The graft is gated on `git merge-base` exiting with status 1 — the no-ancestor signal. Operational failures (timeout, missing object, repository error) keep the loud merge failure instead of rewriting the tip. - New regression tests: one builds the exact shallow-amend shape (a root commit rebuilt from the base tree) and asserts the graft lands on the current head with the imported tree, subject, and graft trailer; one integrates a well-formed sha the repository does not hold and asserts the integration still throws with the branch tip unchanged. ## Verification - `pnpm vitest run packages/adapter-utils/src/git-workspace-sync.test.ts` — 19/19 pass (includes the new graft test and the merge-base failure-discrimination test). - `pnpm --filter @paperclipai/adapter-utils typecheck` — clean. - Full `pnpm vitest run packages/adapter-utils`: every file passes except `local-process-sandbox.test.ts`, which fails identically on an untouched `master` checkout on macOS (bubblewrap-dependent, pre-existing, unrelated). ## Risks - Behavioral shift: unrelated imported histories previously failed the integration; now they land as a squash-graft. In this degenerate case there is no base to merge against, so the imported tree is taken wholesale and concurrent local-only tree changes are superseded at the tip. The local commits keep their place in the graft's ancestry, and the trailer records both shas, so nothing is unrecoverable. The old behavior lost the imported work instead, which is the worse failure for an autonomous run. - The graft reuses the imported head's commit message, so branch history still reads naturally after a sandbox rewrite. ## Model Used - Claude Fable 5 (`claude-fable-5`), extended thinking, via Claude Code CLI (tool use for code exploration, test runs, and verification). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no doc surface describes this internal sync path) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Devin Foley
|
9ea8143c87 |
fix(adapter-utils): give sync-created merge commits a deterministic git identity (#11637)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Runs execute in transported workspaces; at finalize, the host syncs the sandbox git history back into the local worktree > - When both sides advanced, `integrateImportedGitHead` reconciles them with `git merge-tree` plus `git commit-tree` on the host > - Execution hosts are often containers with no git config and no resolvable hostname, so `commit-tree` fails with "Author identity unknown" > - That one local command failure marks the whole run as failed, even though the run's work succeeded > - This pull request gives sync-created merge commits an explicit, deterministic identity at the call site > - The benefit is that workspace finalize no longer depends on ambient host git configuration ## Linked Issues or Issue Description No existing issue found. I searched issues and PRs for "Author identity unknown", "unrelated histories", and "commit-tree identity". **What happened?** A run finished its work, but workspace finalize failed. The host-side sync ran `git commit-tree <tree> -p <localHead> -p <importedHead> -m "Paperclip remote git sync merge <sha>"`. Git exited with `Author identity unknown ... fatal: unable to auto-detect email address (got 'node@<container-id>.(none)')`. The adapter recorded the whole run as failed, and the host worktree kept the stale head. Any container deployment without a global gitconfig reproduces this; I observed it on a Paperclip Cloud stack. **Expected behavior** Commits that the sync machinery itself creates must not depend on ambient host git configuration. The merge commit is machine-authored, so it should carry a deterministic Paperclip identity. **Steps to reproduce** 1. Run the Paperclip server in a container with no `user.name`/`user.email` git config and a hostname git cannot turn into an email. 2. Let a run's sandbox branch diverge from the host worktree, so both sides advance. 3. Workspace finalize calls `integrateImportedGitHead`. The `git commit-tree` step fails with "Author identity unknown" and the run fails. ## What Changed - `git-workspace-sync.ts`: new exported `GIT_SYNC_COMMIT_IDENTITY_ARGS` (`-c user.name=Paperclip -c user.email=noreply@paperclip.ing`), applied to the `commit-tree` call in `integrateImportedGitHead`. - `ssh.ts`: the SSH-sync copy of `integrateImportedGitHead` applies the same identity args to its `commit-tree` call. - New regression test: builds divergent histories in a repo with no configured identity and asserts the sync merge commit is created with the deterministic identity, correct parents, and merged tree. ## Verification - `pnpm vitest run packages/adapter-utils/src/git-workspace-sync.test.ts` — 18/18 pass. - `pnpm --filter @paperclipai/adapter-utils typecheck` — clean. - Negative proof: with the source fix stashed, the new test fails on the identity assertion. - Full `pnpm vitest run packages/adapter-utils`: every file passes except `local-process-sandbox.test.ts`, which fails identically on an untouched `master` checkout on macOS (bubblewrap-dependent, pre-existing, unrelated). ## Risks Low risk. The change only adds `-c` identity flags to two machine-generated commit invocations. `GIT_AUTHOR_*` / `GIT_COMMITTER_*` environment variables still take precedence over `-c` when an operator sets them, so existing deployments that configure an identity keep their behavior. ## Model Used - Claude Fable 5 (`claude-fable-5`), extended thinking, via Claude Code CLI (tool use for code exploration, test runs, and verification). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (no doc surface describes this internal sync path) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
8087661bb8 |
fix: bound workspace Git scans (#11572)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Workspaces let users and agents inspect files that belong to an issue > - Changed-file views use full-tree Git status scans > - Many issue views could start those scans at the same time and make the server unresponsive > - Route-level limits did not protect the process or coalesce work for one repository > - This pull request adds one bounded scheduler for every expensive workspace Git scan > - It also starts browser scans only when the file panel is open and visible > - The benefit is bounded child-process use and responsive health checks during request storms ## Linked Issues or Issue Description **What happened?** Many changed-file requests could start full `git status --porcelain=v1 -z --untracked-files=all` scans at the same time. One production incident produced about 270 direct Git child processes. The Node process stayed alive but stopped answering health requests in time. **Expected behavior** Paperclip must bound expensive Git work across all companies, actors, issues, repositories, and browser tabs. Duplicate requests for one worktree must share work. Excess requests must fail fast with a retryable response. Hidden or closed file panels must not start scans. **Steps to reproduce** 1. Open changed-file views for many issue and actor keys. 2. Send requests for two large workspace roots at the same time. 3. Observe that route-level limiter keys allow many full Git scans to run together. 4. Observe delayed health responses and accumulated Git children. **Paperclip version or commit** Reproduced on master before commit `43ab441f0f`. **Deployment mode** Self-hosted server with local workspace repositories. ## What Changed - Add a process-wide scheduler with configurable concurrency, queue capacity, timeout, and cache TTL. - Add fair admission, a bounded queue, canonical worktree keys, single-flight joins, and bounded result caching. - Add subprocess timeouts, TERM-to-KILL escalation, bounded output, waiter cancellation, and slot cleanup. - Route full-tree status work from file resources, workspace runtime, execution workspaces, and adapter overlay sync through the scheduler. - Return stable retryable `503` and `504` error codes for saturation and timeout. - Add structured logs with safe workspace hashes, durations, queue state, cache use, joins, and terminal outcomes. - Gate UI queries on panel and document visibility. Cancel queries on close, hide, unmount, and workspace change. - Disable focus and reconnect bursts. Keep one explicit refresh action and a retryable unavailable state. - Document the 10-second default freshness tradeoff and all configuration variables. - Add unit, route, UI, adapter, and deterministic 500-request load coverage. ## Verification - `pnpm -r typecheck` - `pnpm build` - `pnpm check:token-gates` - `pnpm --filter @paperclipai/server exec vitest run src/services/workspace-git-operation-scheduler.test.ts src/__tests__/file-resources-git-scan-load.test.ts --reporter=dot` — 16 tests passed. - `pnpm --filter @paperclipai/ui exec vitest run src/components/WorkspaceFileBrowser.test.tsx src/lib/page-visibility.test.ts --reporter=dot` — 38 tests passed. - `pnpm --filter @paperclipai/adapter-utils exec vitest run src/git-workspace-sync.test.ts --reporter=dot` — 16 tests passed. - Existing file-resource, workspace-runtime, and execution-workspace regression selections passed. - Two cleanup safety regressions prove failed scans preserve the worktree before archive and at the final deletion fence. - Before: the incident produced about 270 Git children and health requests timed out. - After: 500 concurrent requests across 500 issue keys, 73 actors, and two roots started two underlying scans. Peak scan concurrency was 2. All 500 requests succeeded. Health p99 was 4.94 ms. The harness found zero unreaped children. - The full local Vitest run passed 4,267 tests. Ten existing fixed-port HTTPS exposure tests could not run because this host already owns Tailnet listeners on ports 42000 and 52000. Clean GitHub CI is the final full-suite result. - Latest-head GitHub CI passed all required test, typecheck, build, canary, e2e, policy, and security gates. - Greptile completed at 5/5 with zero unresolved comments, recommendations, or follow-ups. ## Risks - Changed-file results can be up to 10 seconds old by default. Explicit refresh remains available. - A full queue returns a retryable `503` instead of waiting without a bound. - A scan that exceeds the default 8-second deadline returns a retryable `504` and terminates its process group. - Operators can tune all limits with documented environment variables. Safe defaults protect local and shared servers. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 family. The runtime does not expose the exact deployment ID or context-window size. High reasoning, tool use, and code execution were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
ffd62a4cbb |
fix(adapter-utils): carry the workspace origin remote into transported git workspaces (#10873)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - When an agent runs on a different host (sandbox or SSH), the adapter transport copies the local git execution workspace to that host and syncs changes back after the run > - The transport materializes the remote copy with `git init` plus a depth-1 or bundle fetch, so the copy has no `origin` remote and its head reads as a parentless snapshot commit > - An agent asked to publish its branch (push it, open a pull request) sees "no remote, root snapshot" and must hand the publish step back to a human operator, even when the branch base is a commit the upstream remote already holds > - This pull request carries the workspace's `origin` URL (credential-scrubbed) onto the transported copy as metadata > - The benefit is that branches produced in transported workspaces stay publishable by any actor with credentials, while the transport itself still never fetches or pushes ## Linked Issues or Issue Description No public issue exists. Description follows the enhancement template: **What existing behavior does this improve?** The workspace transport in `@paperclipai/adapter-utils` already copies a git workspace to the execution host and back. This change improves the fidelity of that copy: the transported repo keeps the workspace's `origin` remote instead of losing it. **Subsystem affected** Adapter utilities — the sandbox transport (`withShallowGitWorkspaceClone` in `packages/adapter-utils/src/git-workspace-sync.ts`) and the SSH transport (`importGitWorkspaceToSsh` in `packages/adapter-utils/src/ssh.ts`). **Current behavior** The transported copy is built with `git init` plus a depth-1 (sandbox) or bundle (SSH) fetch. It has no remotes. `git remote -v` is empty and the head commit reads as a root snapshot with no visible ancestry. Agents and operators inside the execution host cannot fetch real ancestry or push a branch, even when the branch base is a commit the upstream remote already holds. **Proposed behavior** The transport reads the source workspace's `origin` URL, scrubs credentials from it, and configures it on the transported copy. The sandbox path adds the remote to the fresh clone. The SSH path sets or adds the remote in the remote setup script, which also covers reused workspace directories. A workspace with no `origin` transports exactly as before. **Reason and benefit** A branch committed in a transported workspace becomes publishable in place: the shallow boundary commit already exists on the remote, so a push pack closes without full local ancestry (a new test locks in this property). Fetching real ancestry also becomes possible for whoever holds credentials. Without this, agents must describe their change in a handoff document and a human must reconstruct the branch by hand. **Breaking changes** None. The URL copy is best-effort and metadata-only. The transport never fetches from or pushes to the remote. The no-remote-git contract holds: sync-back through the local cwd stays the only cross-run persistence path, and `packages/adapters/AUTHORING.md` gains a paragraph that makes the carried-remote nuance explicit. ## What Changed - `packages/adapter-utils/src/git-workspace-sync.ts`: new `sanitizeGitRemoteUrl` (strips http(s) userinfo, where tokens can be embedded; scp-like/ssh forms and filesystem paths pass through) and `readSanitizedOriginRemoteUrl`; `withShallowGitWorkspaceClone` configures the scrubbed `origin` on the fresh clone, best-effort. - `packages/adapter-utils/src/ssh.ts`: `importGitWorkspaceToSsh` sets or adds the scrubbed `origin` in the remote setup script, non-fatal under `set -e`. - `packages/adapter-utils/src/git-workspace-sync.test.ts`: four new integration cases (remote copied, credentials scrubbed, no-origin unchanged, push from the shallow clone to an origin that holds the base commit) plus `sanitizeGitRemoteUrl` unit tests. - `packages/adapters/AUTHORING.md`: documents that a transported copy may carry a credential-scrubbed `origin` as metadata, and why this does not weaken the no-remote-git contract. ## Verification - `npx vitest run packages/adapter-utils/src/git-workspace-sync.test.ts` — 12/12 pass (4 new integration cases + sanitizer unit tests). - `npx vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — 24/24 pass. - `npx vitest run packages/adapter-utils/src/ssh-fixture.test.ts` — 16/16 pass, including the `no-remote-git contract` case (a workspace without `origin` still round-trips with no remote introduced at any point). - `node scripts/check-no-git-push.mjs` — passes; this change adds no push or fetch to adapter/runtime code. - `pnpm typecheck` in `packages/adapter-utils` — clean. ## Risks - Low risk. The change is additive metadata on the transported copy only; failure to record the remote never fails the transport. - Credential exposure is the real hazard and is handled: http(s) userinfo is stripped before the URL leaves the host. Non-http forms (scp-like, `ssh://`) carry no secret in the URL and pass through. - A reused SSH workspace whose project `origin` changed now gets the current URL via `set-url` instead of keeping a stale one. ## Model Used Claude Fable 5 (`claude-fable-5`), Anthropic — extended thinking, agentic tool use via Claude Code CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Devin Foley
|
6401f4f78c |
fix(sandbox): bundle git copy-back against the merge-base so diverged/reset workspaces still import (#10601)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - An agent that runs in a sandbox has its workspace copied back to the host when the run ends, so its work persists — the copy-back ships a git bundle of the sandbox's commits > - The bundle is created as a thin delta, `git bundle create HEAD --not <baseSha>`, which records `baseSha` (the host workspace HEAD captured at export) as a prerequisite the host must already hold > - That assumption breaks when the sandbox HEAD has diverged from `baseSha`, or when a shared host workspace no longer holds `baseSha` at import time — then `git fetch` on the host hard-fails and the entire run is lost even though the agent finished its work > - This pull request bundles against the merge-base of `baseSha` and the sandbox HEAD (with a full-bundle fallback), which the host can satisfy in those cases > - The benefit is that copy-back no longer discards a completed run's work over a base the host can't reconcile ## Linked Issues or Issue Description **What happened?** A sandbox agent run completed its work, then failed during workspace finalize: ``` git -C <host workspace> fetch --force <git-delta.bundle> refs/…/export:refs/…/imported error: Could not read <baseSha> fatal: revision walk setup failed error: git-delta.bundle did not send all necessary objects ``` The run is reported as `adapter_failed` even though the agent produced output. The copy-back bundle names the host workspace's recorded HEAD (`baseSha`) as a prerequisite, but the host cannot satisfy it. **Steps to reproduce** Two independent triggers, both reproduced in tests: 1. The sandbox's HEAD has diverged from `baseSha` — e.g. the sandbox carries a local-only branch that forked from an older commit than the host's current HEAD. 2. The shared host workspace no longer holds `baseSha` at import time (it was reset / re-realized between export and import). In either case `git fetch` of the thin bundle fails with a missing prerequisite. **Expected behavior** Copy-back imports the sandbox's work as long as the host holds any common ancestor, instead of hard-failing and discarding the run. **Paperclip version** Current `master`. **Deployment mode** Any deployment running agents in sandbox environments with workspace sync (notably shared-workspace clones and custom images that carry a local-ahead branch). ## What Changed - `buildRemoteGitDeltaBundleScript` now computes `bundle_base = git merge-base <baseSha> HEAD` and bundles `HEAD --not <bundle_base>`. The merge-base is an ancestor of `baseSha`, so any host that holds `baseSha` (or an ancestor of it — e.g. after a reset) can satisfy the prerequisite, and the bundle stays a delta rather than a full-history transfer. - When `baseSha` is absent from the sandbox, or no merge-base exists, it falls back to a full, self-contained bundle (no prerequisites) so the import can always complete. - The existing empty-bundle no-op (no new commits) and the ordinary fast-forward path are unchanged; the `cat-file` base check no longer aborts the script under `set -e`. ## Verification - `pnpm vitest run packages/adapter-utils/src/git-workspace-sync.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — new cases: a diverged sandbox HEAD imports when the host holds only the merge-base (not `baseSha`), and the full-bundle fallback imports into a host that shares no history; existing thin-delta and empty-bundle cases still pass. - `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`. - Standalone shell repro confirmed the old thin bundle fails with "Repository lacks these prerequisite commits" in both trigger cases, and the merge-base bundle imports successfully. ## Risks - Low. For the common case (sandbox HEAD descends from `baseSha`) the merge-base is `baseSha`, so the bundle is byte-for-byte the same delta as before. The change only alters behavior when the old code would have hard-failed. - This makes the copy-back import succeed on a diverged base; the subsequent reconciliation of divergent histories (`integrateImportedGitHead`) is unchanged and still owns how the imported head is merged into the host branch. Where a workspace's history has genuinely diverged (e.g. a stale custom image carrying a local-only branch), a clean re-clone/re-capture is still the right operational fix — this change prevents work loss, it does not reconcile intentional divergence. ## Model Used Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended thinking, agentic tool use (file edits, shell repro, vitest/tsc runs). No other models involved. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Devin FoleyandPaperclip
|
721541c41d |
Fix sandbox restore index drift (#8595)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandboxed agents can do useful work in a remote workspace, then export that result back to a host review checkout. > - A restore can leave the host checkout's index stale even when the sandbox content and committed state are correct. > - That drift makes reviewers see misleading missing-file or dirty-state results after sandbox execution. > - This pull request repairs the host index refresh path and surfaces a clean/dirty restore signal during finalization. > - The benefit is more reliable review checkouts and clearer sandbox finalization status. ## Linked Issues or Issue Description Refs #248 No exact public GitHub issue was found for this restore/index consistency fix. The underlying bug is that a sandbox restore can update host working-tree content without leaving the host git index aligned, so local review tooling may report stale deletions or misleading dirtiness. This PR keeps the restore/export path consistent and reports the resulting clean/dirty state. GitHub search performed for related or duplicate work: `sandbox runtime status`, `sandbox restore index`, and `runtime progress`. No direct duplicate PR was found. ## What Changed - Refreshed git workspace sync/index handling after sandbox restore/export operations. - Added finalization status that reports whether the restored host checkout is clean or dirty. - Preserved sandbox-managed runtime progress callbacks around restore and finalization. - Added adapter-utils tests covering the host index drift repair and clean/dirty finalization signal. ## Verification - Local PII scan before push: high-confidence secret patterns, internal issue links, local user paths, and private URL patterns checked across all three split diffs; no real secrets or internal links found. - `git diff --check feat/sandbox-runtime-status..fix/sandbox-restore-index-sync` - `pnpm exec vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — 1 file, 10 tests passed. - `pnpm run typecheck` passed on `fix/sandbox-restore-index-sync`. - `pnpm run build` passed on `fix/sandbox-restore-index-sync`; Vite reported existing CSS `::highlight` and chunk-size warnings. - `pnpm run test:run` was attempted on `fix/sandbox-restore-index-sync`; it failed in two unrelated broad-suite tests. One depends on this host's Git default branch behavior, and one depends on local Claude model-discovery environment. The changed focused suite above passes. ## Risks - Git index refresh behavior needs to remain conservative so it does not hide real uncommitted user changes. - The clean/dirty finalization signal is diagnostic; it should not by itself mark a sandbox run successful or failed. - This PR is stacked on the runtime-status branch so finalization progress can reuse the same callback path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 via Codex coding agent, with shell/tool execution in a local worktree. Exact context-window metadata is not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin FoleyandPaperclip
|
2c98c8e1e5 |
Fix sandbox git publishing and large workspace uploads (#8422)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox and SSH runtimes need to preserve agent work across isolated execution environments > - Git-backed workspaces were being copied mostly as filesystem archives, which breaks when `.git` points outside the mounted workspace and makes sandbox agents unable to publish their own branches > - Large ignored dependency trees could also be swept into the sandbox overlay, causing multi-GB transfers and max-string failures in some sandbox clients > - This pull request makes sandbox runtime setup use a git-backed HEAD sync plus a small dirty/untracked overlay, and bounds sandbox file transfers so large archives do not need one huge string > - The benefit is that sandbox agents can commit and push from a usable git checkout without uploading dependency trees such as `node_modules` ## Linked Issues or Issue Description Refs #8395 No public duplicate issue or PR was found after searches for `sandbox git workspace`, `git push sandbox`, and `node_modules sandbox upload`. Bug report: - What happened: sandbox-backed agent workspaces could receive a `.git` file that pointed at host-only git state, leaving the sandbox unable to run normal git workflows. The sandbox overlay upload could also include ignored dependency directories, creating very large transfers. - Expected behavior: sandbox and remote runtimes should prepare a usable git-backed workspace, copy only the necessary workspace overlay, and restore git history plus file changes without depending on a host-only `.git` path. - Steps to reproduce: 1. Run an agent in a sandbox-backed workspace whose local git checkout is a worktree. 2. Ask the agent to complete a GitHub workflow that requires commit/push access. 3. Observe that git operations can fail inside the sandbox, and ignored dependency trees can be uploaded as part of the workspace overlay. - Paperclip version or commit: reproduced against `master` before this PR, base `7aa212296eb1`. - Deployment mode: local dev / sandbox-backed runtime. - Installation method: built from source. - Agent adapters involved: local adapters using shared adapter-utils runtime preparation. - Database mode: not database-related. - Access context: agent runtime. - Local verification environment: Node.js v25.6.1, pnpm 9.15.4, macOS arm64. - Privacy checklist: all pasted output was reviewed for secrets, private hostnames, local usernames, and internal instance links. ## What Changed - Added a GitHub workflow push preflight so agent runs can detect missing push credentials when a workflow explicitly needs GitHub publishing. - Added shared git workspace sync helpers for shallow HEAD import/export and dirty/untracked overlay tracking. - Updated sandbox managed runtime setup to use git history plus a selected overlay instead of uploading the full local workspace for git-backed workspaces. - Bounded sandbox archive upload/download paths so large payloads stream or chunk instead of materializing one oversized string. - Excluded `.git` and ignored dependency trees from sandbox upload, download, and restore baselines while preserving local ignored directories during sync-back. - Added focused tests for git workspace sync, sandbox overlay selection, transfer chunking, and heartbeat push-preflight behavior. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/git-workspace-sync.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts packages/adapter-utils/src/command-managed-runtime.test.ts server/src/__tests__/heartbeat-project-env.test.ts server/src/__tests__/heartbeat-workspace-session.test.ts` - `pnpm --filter @paperclipai/adapter-utils typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm build` - Public-hygiene scan of the PR diff and commit messages for internal issue ids, local paths, private hostnames, and obvious token patterns. ## Risks - Medium risk: this changes sandbox runtime synchronization semantics for git-backed workspaces, especially around dirty tracked files, untracked files, deleted paths, and ignored files. - The main mitigation is focused test coverage for upload contents, restore exclusions, and git round-trip behavior. - The SSH runtime keeps the current bundle-based implementation from `master`; this PR only aligns shared excludes and sandbox behavior with that model. ## Model Used OpenAI Codex, GPT-5-based coding agent, tool-enabled shell/git/GitHub workflow, with code execution and repository inspection. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |