mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 03:31:44 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Remote agents work in sandboxes, and the server saves each sandbox workspace as an encrypted snapshot in object storage > - One Node main thread on the server controls every run, so per-agent byte work on that thread limits how many agents one server can hold > - The save path read every downloaded archive twice: a main-thread SHA-256 re-hash after the download, then a full read in the off-thread encrypt pass that already computes the same digest > - This pull request removes the main-thread re-hash and checks the encrypt pass's digest and size against the sandbox receipt instead > - The benefit is one full archive read per save instead of two, and no archive hashing on the main thread, with the same integrity guarantee ## Linked Issues or Issue Description Refs the 1,000-concurrent-agents plan in `doc/plans/2026-11-13-thousand-concurrent-agents.md` (Phase 1, "Workspace snapshot" row). **What existing behavior does this improve?** Saving a remote workspace snapshot after a run. **Subsystem affected** Server: remote workspace snapshot save (`remote-workspace-transport.ts`, `remote-workspace-snapshot-storage.ts`, `remote-workspaces.ts`). **Current behavior** `captureSandboxWorkspaceArchive` streams the whole downloaded archive through `createHash` on the main thread to compare it with the sandbox receipt. Then `uploadWorkspaceSnapshot` reads the whole archive again on a worker thread to encrypt it, and computes the same plaintext digest. **Proposed behavior** After the download, the server checks only the file size against the receipt. The encrypt pass compares its plaintext digest and size with the receipt. A mismatch fails with the same `remote_workspace_capture_transfer_mismatch` error before the upload attempt is recorded and before any object is stored. **Reason and benefit** One full read and one SHA-256 of each archive leave the server's main thread (about 0.36 ms of main-thread time per MB on a loaded dev machine; 37 ms for a 100 MB archive, at event-loop utilization 1.0 for that time). ## What Changed - `captureSandboxWorkspaceArchive` no longer re-hashes the downloaded archive. It checks with `lstat` that the download is a file with the receipt's byte size, and fails with `remote_workspace_capture_transfer_mismatch` when it is not. - `uploadWorkspaceSnapshot` accepts an optional `expectedArchive` (`sha256`, `byteSize`). After the encrypt pass, it compares the sealed plaintext digest and size with it. A mismatch throws `remote_workspace_capture_transfer_mismatch` before the attempt enters `metadata.uploads` and before `putFile`. The scratch ciphertext is removed as before. - `saveRemoteWorkspace` passes the sandbox receipt as `expectedArchive` when it captured an archive. - The save-level `verifyMs` timing field is removed, because that step no longer exists. The storage-level `verifyMs` stays. - `doc/run-log-events.md` says that the encrypt step is the only full read and where a mismatch fails. Callers that relied on the early hash keep their behavior: - Unchanged-snapshot reuse (`findUnchangedWorkspaceSnapshot`) compares the sandbox receipt, not the local hash. When it reuses the parent, the local copy is used only to list directories, and `readRemoteWorkspaceArchiveDirectories` verifies the manifest digest. - Reuse of an earlier upload attempt or an already uploaded snapshot does not read the archive. Those objects were verified when they were stored. - The read-only disk-full fallback builds its archive on the server, so its receipt always matches. Its behavior does not change. - The error message and the failure path do not change, so the save failure code, retry and fallback stay the same. ## Verification - Red/green: `remote-workspace-transport.test.ts` "leaves the downloaded archive's single full read to the encrypting upload" counts main-thread read streams of the downloaded archive during capture. On master it is 1 (fails). With this change it is 0 (passes). The encrypt pass is the one remaining read. - New: digest mismatch and size mismatch in `uploadWorkspaceSnapshot` reject with the transfer-mismatch error, do not call `putFile`, and leave the row `pending` with no recorded attempt. A retry with the right receipt saves. - New: an end-to-end `saveRemoteWorkspace` with a transfer that flips one byte (same size) fails with the transfer-mismatch error, stores nothing, keeps the writer and the previous snapshot, and a retry saves and restores the right content. - New: a download whose size differs from the receipt fails in capture. - `cd server && npx vitest run src/services/remote-workspace-snapshot-storage.test.ts src/services/remote-workspace-transport.test.ts src/services/remote-workspaces.integration.test.ts src/services/remote-workspace-transfer-timeout.test.ts src/services/remote-workspace-readonly-capture-budget.test.ts src/services/sandbox-disk-full-fault.test.ts` passes: 6 files, 360 tests. `src/services/remote-workspace-snapshot-reuse.test.ts` passes: 32 tests. - `npx tsc --noEmit -p server/tsconfig.json` passes. ## Risks - Low risk. A same-size corrupt download is now detected in the encrypt pass instead of right after the download. It is still detected before any attempt is recorded or any object is stored. - When the sandbox reports an unchanged workspace, a corrupt local copy no longer fails the save. The save reuses the verified parent snapshot, which matches the sandbox's content. The local copy is used only for its manifest, and the manifest digest is verified. - The save-level `saveTiming.verifyMs` field is gone. No code reads it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>