mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 06:10:44 +01:00
DottaandClaude Opus 5.5
4d2a77cf5a
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Remote (sandbox) workspaces are saved to durable storage as encrypted snapshots, and restored before a later turn > - Saves and restores are a reported source of slowness on staging, so #20280 added run-log timing for each step > - A Daytona storage wave with a Paperclip-sized workspace (R1, ~750 MB, 19.5k entries) showed each save taking 58–68 s. Storage was only 2–3 s of that; capture inside the sandbox was 44–47 s > - The first save runs while the first run prepares its workspace, so it delays the first token > - A local profile of the same capture found one cause: `git fsck --full` re-hashes the whole history on one thread on every save > - This pull request keeps the same object checks but verifies packs on several threads > - The benefit is about 10 s less per save on a Paperclip-sized repository, with the same corruption protection ## Linked Issues or Issue Description Refs #18990, #20280 **What happened?** Saving a remote workspace with a large Git history is slow. On Daytona (3 vCPU), a Paperclip clone's save took 58–68 s; capture was 44–47 s of it. The first save is on the first run's critical path: `heartbeat.prepare.workspace` took 90–110 s before the first turn. **Steps to reproduce** 1. Clone Paperclip (`git clone https://github.com/paperclipai/paperclip.git`, ~750 MB, one 459 MiB pack). 2. Run the generated archive helper: `node archive.cjs capture <clone> <out.pcws>` (from `remoteWorkspaceArchiveScript()`). 3. It takes ~30 s. A wrapper on `git` shows 22.5 s in one `git fsck --full --no-dangling` on the normalized copy. **Expected behavior** Capture verifies saved Git objects as #18990 requires (a missing or corrupt object blocks the save), without spending most of the save re-hashing unchanged history on one thread. ## What Changed - `remote-workspace-snapshot-archive.ts`: both `git fsck --full --no-dangling` calls (the offline source check and the normalized-copy check) now call a new `verifyGitObjects`. It runs: - `git fsck --connectivity-only --no-dangling`: every object reachable from refs, the index and saved Git state is present. - A direct hash of each loose object (inflate, then SHA-1 or SHA-256 per the repository's object format) against its file name. A mismatch or unreadable object fails with the existing `invalid_git_repository` reason. - `git index-pack --verify --threads=N` for every pack (N = available CPUs, at most 8). This re-hashes every packed object, on several threads. - `fsck --no-full` was not used: in a shallow clone it reports reflog entries that point at packed objects as invalid. - Tests (`remote-workspace-snapshot-archive.test.ts`): - New: capture refuses a repository missing a staged object, or a committed object. - New: capture refuses a repository with a corrupt byte inside a pack. - The existing #18990 test `rejects corrupt available objects in full/shallow/partial Git before publishing a receipt` (valid zlib, wrong hash) still passes. It failed with connectivity-only fsck alone, which is why the loose-object hash is there. ## Verification - `npx vitest run` on `remote-workspace-snapshot-archive`, `remote-workspace-transport`, `remote-workspace-migration`, `remote-workspace-online-source` and `remote-workspace-readonly-capture-budget`: 526 passed. - Local benchmark: a fresh clone of `https://github.com/paperclipai/paperclip.git` (753,506,675-byte archive, 19,506 entries), alternating runs on one host: | Capture | Run 1 | Run 2 | |---|---|---| | master helper | 32.04 s | 30.05 s | | this branch | 21.07 s | 20.09 s | Both produce the same archive: equal byte size, manifest digest (`e5facf033409…`) and sha256 (`a4eb843bff1f…`). - Component timings on the same clone: `fsck --full` 20.5 s; `index-pack --verify` 19.5 s on 1 thread, 10.7 s on 3 threads; `fsck --connectivity-only` 0.9 s. - Daytona evidence for the problem (wave `w-20261109T170632-s9921`, R1, 3 vCPU, MinIO): save `captureMs` 44–47 s, `transferMs` 11–20 s, storage `putMs` 1.3–1.8 s, `encryptMs` 0.4–0.7 s. I did not rerun Daytona on this branch: its image must match the checkout's runner content. The local helper run is the same code the sandbox executes. ## Risks - Same guarantees as before for missing and corrupt objects. `fsck --full` also applies its object syntax checks to every packed object; `index-pack --verify` checks hashes and pack integrity. Commits and trees are still parsed by the connectivity walk, so malformed ones still fail. - More CPU threads during capture, for a shorter time. Capture already runs inside the sandbox, off the control plane. - A follow-up could skip re-verifying a pack that an earlier save of the same workspace already verified. Most saves keep the same clone pack, so that would remove the remaining ~10 s on repeat saves. Not in this PR. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`, Anthropic), 1M context window, tool use and code execution, via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>