Jannik/PaperClipAI
1
0 Fork 0
mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-10 12:37:55 +01:00
Code Issues Packages Projects Releases Wiki Activity
3,630 Commits 1,705 Branches 2,405 Tags
8cb0ce0de597542d1fec971d83a9a6725d7e195d
Commit Graph
3 Commits
Author SHA1 Message Date
Devin Foley ffd62a4cbb fix(adapter-utils): carry the workspace origin remote into transported git workspaces (#10873) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - When an agent runs on a different host (sandbox or SSH), the adapter
transport copies the local git execution workspace to that host and
syncs changes back after the run
> - The transport materializes the remote copy with `git init` plus a
depth-1 or bundle fetch, so the copy has no `origin` remote and its head
reads as a parentless snapshot commit
> - An agent asked to publish its branch (push it, open a pull request)
sees "no remote, root snapshot" and must hand the publish step back to a
human operator, even when the branch base is a commit the upstream
remote already holds
> - This pull request carries the workspace's `origin` URL
(credential-scrubbed) onto the transported copy as metadata
> - The benefit is that branches produced in transported workspaces stay
publishable by any actor with credentials, while the transport itself
still never fetches or pushes

## Linked Issues or Issue Description

No public issue exists. Description follows the enhancement template:

**What existing behavior does this improve?**

The workspace transport in `@paperclipai/adapter-utils` already copies a
git workspace to the execution host and back. This change improves the
fidelity of that copy: the transported repo keeps the workspace's
`origin` remote instead of losing it.

**Subsystem affected**

Adapter utilities — the sandbox transport
(`withShallowGitWorkspaceClone` in
`packages/adapter-utils/src/git-workspace-sync.ts`) and the SSH
transport (`importGitWorkspaceToSsh` in
`packages/adapter-utils/src/ssh.ts`).

**Current behavior**

The transported copy is built with `git init` plus a depth-1 (sandbox)
or bundle (SSH) fetch. It has no remotes. `git remote -v` is empty and
the head commit reads as a root snapshot with no visible ancestry.
Agents and operators inside the execution host cannot fetch real
ancestry or push a branch, even when the branch base is a commit the
upstream remote already holds.

**Proposed behavior**

The transport reads the source workspace's `origin` URL, scrubs
credentials from it, and configures it on the transported copy. The
sandbox path adds the remote to the fresh clone. The SSH path sets or
adds the remote in the remote setup script, which also covers reused
workspace directories. A workspace with no `origin` transports exactly
as before.

**Reason and benefit**

A branch committed in a transported workspace becomes publishable in
place: the shallow boundary commit already exists on the remote, so a
push pack closes without full local ancestry (a new test locks in this
property). Fetching real ancestry also becomes possible for whoever
holds credentials. Without this, agents must describe their change in a
handoff document and a human must reconstruct the branch by hand.

**Breaking changes**

None. The URL copy is best-effort and metadata-only. The transport never
fetches from or pushes to the remote. The no-remote-git contract holds:
sync-back through the local cwd stays the only cross-run persistence
path, and `packages/adapters/AUTHORING.md` gains a paragraph that makes
the carried-remote nuance explicit.

## What Changed

- `packages/adapter-utils/src/git-workspace-sync.ts`: new
`sanitizeGitRemoteUrl` (strips http(s) userinfo, where tokens can be
embedded; scp-like/ssh forms and filesystem paths pass through) and
`readSanitizedOriginRemoteUrl`; `withShallowGitWorkspaceClone`
configures the scrubbed `origin` on the fresh clone, best-effort.
- `packages/adapter-utils/src/ssh.ts`: `importGitWorkspaceToSsh` sets or
adds the scrubbed `origin` in the remote setup script, non-fatal under
`set -e`.
- `packages/adapter-utils/src/git-workspace-sync.test.ts`: four new
integration cases (remote copied, credentials scrubbed, no-origin
unchanged, push from the shallow clone to an origin that holds the base
commit) plus `sanitizeGitRemoteUrl` unit tests.
- `packages/adapters/AUTHORING.md`: documents that a transported copy
may carry a credential-scrubbed `origin` as metadata, and why this does
not weaken the no-remote-git contract.

## Verification

- `npx vitest run packages/adapter-utils/src/git-workspace-sync.test.ts`
— 12/12 pass (4 new integration cases + sanitizer unit tests).
- `npx vitest run
packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — 24/24
pass.
- `npx vitest run packages/adapter-utils/src/ssh-fixture.test.ts` —
16/16 pass, including the `no-remote-git contract` case (a workspace
without `origin` still round-trips with no remote introduced at any
point).
- `node scripts/check-no-git-push.mjs` — passes; this change adds no
push or fetch to adapter/runtime code.
- `pnpm typecheck` in `packages/adapter-utils` — clean.

## Risks

- Low risk. The change is additive metadata on the transported copy
only; failure to record the remote never fails the transport.
- Credential exposure is the real hazard and is handled: http(s)
userinfo is stripped before the URL leaves the host. Non-http forms
(scp-like, `ssh://`) carry no secret in the URL and pass through.
- A reused SSH workspace whose project `origin` changed now gets the
current URL via `set-url` instead of keeping a stale one.

## Model Used

Claude Fable 5 (`claude-fable-5`), Anthropic — extended thinking,
agentic tool use via Claude Code CLI.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
 2026-08-05 10:24:01 -07:00
Devin Foley 6401f4f78c fix(sandbox): bundle git copy-back against the merge-base so diverged/reset workspaces still import (#10601) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - An agent that runs in a sandbox has its workspace copied back to the
host when the run ends, so its work persists — the copy-back ships a git
bundle of the sandbox's commits
> - The bundle is created as a thin delta, `git bundle create HEAD --not
<baseSha>`, which records `baseSha` (the host workspace HEAD captured at
export) as a prerequisite the host must already hold
> - That assumption breaks when the sandbox HEAD has diverged from
`baseSha`, or when a shared host workspace no longer holds `baseSha` at
import time — then `git fetch` on the host hard-fails and the entire run
is lost even though the agent finished its work
> - This pull request bundles against the merge-base of `baseSha` and
the sandbox HEAD (with a full-bundle fallback), which the host can
satisfy in those cases
> - The benefit is that copy-back no longer discards a completed run's
work over a base the host can't reconcile

## Linked Issues or Issue Description

**What happened?**

A sandbox agent run completed its work, then failed during workspace
finalize:

```
git -C <host workspace> fetch --force <git-delta.bundle> refs/…/export:refs/…/imported
error: Could not read <baseSha>
fatal: revision walk setup failed
error: git-delta.bundle did not send all necessary objects
```

The run is reported as `adapter_failed` even though the agent produced
output. The copy-back bundle names the host workspace's recorded HEAD
(`baseSha`) as a prerequisite, but the host cannot satisfy it.

**Steps to reproduce**

Two independent triggers, both reproduced in tests:
1. The sandbox's HEAD has diverged from `baseSha` — e.g. the sandbox
carries a local-only branch that forked from an older commit than the
host's current HEAD.
2. The shared host workspace no longer holds `baseSha` at import time
(it was reset / re-realized between export and import).

In either case `git fetch` of the thin bundle fails with a missing
prerequisite.

**Expected behavior**

Copy-back imports the sandbox's work as long as the host holds any
common ancestor, instead of hard-failing and discarding the run.

**Paperclip version**

Current `master`.

**Deployment mode**

Any deployment running agents in sandbox environments with workspace
sync (notably shared-workspace clones and custom images that carry a
local-ahead branch).

## What Changed

- `buildRemoteGitDeltaBundleScript` now computes `bundle_base = git
merge-base <baseSha> HEAD` and bundles `HEAD --not <bundle_base>`. The
merge-base is an ancestor of `baseSha`, so any host that holds `baseSha`
(or an ancestor of it — e.g. after a reset) can satisfy the
prerequisite, and the bundle stays a delta rather than a full-history
transfer.
- When `baseSha` is absent from the sandbox, or no merge-base exists, it
falls back to a full, self-contained bundle (no prerequisites) so the
import can always complete.
- The existing empty-bundle no-op (no new commits) and the ordinary
fast-forward path are unchanged; the `cat-file` base check no longer
aborts the script under `set -e`.

## Verification

- `pnpm vitest run packages/adapter-utils/src/git-workspace-sync.test.ts
packages/adapter-utils/src/sandbox-managed-runtime.test.ts` — new cases:
a diverged sandbox HEAD imports when the host holds only the merge-base
(not `baseSha`), and the full-bundle fallback imports into a host that
shares no history; existing thin-delta and empty-bundle cases still
pass.
- `pnpm --filter @paperclipai/adapter-utils exec tsc --noEmit`.
- Standalone shell repro confirmed the old thin bundle fails with
"Repository lacks these prerequisite commits" in both trigger cases, and
the merge-base bundle imports successfully.

## Risks

- Low. For the common case (sandbox HEAD descends from `baseSha`) the
merge-base is `baseSha`, so the bundle is byte-for-byte the same delta
as before. The change only alters behavior when the old code would have
hard-failed.
- This makes the copy-back import succeed on a diverged base; the
subsequent reconciliation of divergent histories
(`integrateImportedGitHead`) is unchanged and still owns how the
imported head is merged into the host branch. Where a workspace's
history has genuinely diverged (e.g. a stale custom image carrying a
local-only branch), a clean re-clone/re-capture is still the right
operational fix — this change prevents work loss, it does not reconcile
intentional divergence.

## Model Used

Claude Fable 5 (`claude-fable-5`, Anthropic) via Claude Code — extended
thinking, agentic tool use (file edits, shell repro, vitest/tsc runs).
No other models involved.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
 2026-07-31 23:41:42 -07:00
Devin FoleyandPaperclip 2c98c8e1e5 Fix sandbox git publishing and large workspace uploads (#8422) 
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Sandbox and SSH runtimes need to preserve agent work across isolated
execution environments
> - Git-backed workspaces were being copied mostly as filesystem
archives, which breaks when `.git` points outside the mounted workspace
and makes sandbox agents unable to publish their own branches
> - Large ignored dependency trees could also be swept into the sandbox
overlay, causing multi-GB transfers and max-string failures in some
sandbox clients
> - This pull request makes sandbox runtime setup use a git-backed HEAD
sync plus a small dirty/untracked overlay, and bounds sandbox file
transfers so large archives do not need one huge string
> - The benefit is that sandbox agents can commit and push from a usable
git checkout without uploading dependency trees such as `node_modules`

## Linked Issues or Issue Description

Refs #8395

No public duplicate issue or PR was found after searches for `sandbox
git workspace`, `git push sandbox`, and `node_modules sandbox upload`.

Bug report:

- What happened: sandbox-backed agent workspaces could receive a `.git`
file that pointed at host-only git state, leaving the sandbox unable to
run normal git workflows. The sandbox overlay upload could also include
ignored dependency directories, creating very large transfers.
- Expected behavior: sandbox and remote runtimes should prepare a usable
git-backed workspace, copy only the necessary workspace overlay, and
restore git history plus file changes without depending on a host-only
`.git` path.
- Steps to reproduce:
1. Run an agent in a sandbox-backed workspace whose local git checkout
is a worktree.
2. Ask the agent to complete a GitHub workflow that requires commit/push
access.
3. Observe that git operations can fail inside the sandbox, and ignored
dependency trees can be uploaded as part of the workspace overlay.
- Paperclip version or commit: reproduced against `master` before this
PR, base `7aa212296eb1`.
- Deployment mode: local dev / sandbox-backed runtime.
- Installation method: built from source.
- Agent adapters involved: local adapters using shared adapter-utils
runtime preparation.
- Database mode: not database-related.
- Access context: agent runtime.
- Local verification environment: Node.js v25.6.1, pnpm 9.15.4, macOS
arm64.
- Privacy checklist: all pasted output was reviewed for secrets, private
hostnames, local usernames, and internal instance links.

## What Changed

- Added a GitHub workflow push preflight so agent runs can detect
missing push credentials when a workflow explicitly needs GitHub
publishing.
- Added shared git workspace sync helpers for shallow HEAD import/export
and dirty/untracked overlay tracking.
- Updated sandbox managed runtime setup to use git history plus a
selected overlay instead of uploading the full local workspace for
git-backed workspaces.
- Bounded sandbox archive upload/download paths so large payloads stream
or chunk instead of materializing one oversized string.
- Excluded `.git` and ignored dependency trees from sandbox upload,
download, and restore baselines while preserving local ignored
directories during sync-back.
- Added focused tests for git workspace sync, sandbox overlay selection,
transfer chunking, and heartbeat push-preflight behavior.

## Verification

- `pnpm exec vitest run
packages/adapter-utils/src/git-workspace-sync.test.ts
packages/adapter-utils/src/sandbox-managed-runtime.test.ts
packages/adapter-utils/src/command-managed-runtime.test.ts
server/src/__tests__/heartbeat-project-env.test.ts
server/src/__tests__/heartbeat-workspace-session.test.ts`
- `pnpm --filter @paperclipai/adapter-utils typecheck`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm build`
- Public-hygiene scan of the PR diff and commit messages for internal
issue ids, local paths, private hostnames, and obvious token patterns.

## Risks

- Medium risk: this changes sandbox runtime synchronization semantics
for git-backed workspaces, especially around dirty tracked files,
untracked files, deleted paths, and ignored files.
- The main mitigation is focused test coverage for upload contents,
restore exclusions, and git round-trip behavior.
- The SSH runtime keeps the current bundle-based implementation from
`master`; this PR only aligns shared excludes and sandbox behavior with
that model.

## Model Used

OpenAI Codex, GPT-5-based coding agent, tool-enabled shell/git/GitHub
workflow, with code execution and repository inspection.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
 2026-06-20 22:03:55 -07:00