Files
PaperClipAI/server/src
DottaandClaude Opus 5.5 28761f9c25 perf(workspaces): read a downloaded snapshot archive once, in the encrypt pass (#21683)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Remote agents work in sandboxes, and the server saves each sandbox
workspace as an encrypted snapshot in object storage
> - One Node main thread on the server controls every run, so per-agent
byte work on that thread limits how many agents one server can hold
> - The save path read every downloaded archive twice: a main-thread
SHA-256 re-hash after the download, then a full read in the off-thread
encrypt pass that already computes the same digest
> - This pull request removes the main-thread re-hash and checks the
encrypt pass's digest and size against the sandbox receipt instead
> - The benefit is one full archive read per save instead of two, and no
archive hashing on the main thread, with the same integrity guarantee

## Linked Issues or Issue Description

Refs the 1,000-concurrent-agents plan in
`doc/plans/2026-11-13-thousand-concurrent-agents.md` (Phase 1,
"Workspace snapshot" row).

**What existing behavior does this improve?**
Saving a remote workspace snapshot after a run.

**Subsystem affected**
Server: remote workspace snapshot save (`remote-workspace-transport.ts`,
`remote-workspace-snapshot-storage.ts`, `remote-workspaces.ts`).

**Current behavior**
`captureSandboxWorkspaceArchive` streams the whole downloaded archive
through `createHash` on the main thread to compare it with the sandbox
receipt. Then `uploadWorkspaceSnapshot` reads the whole archive again on
a worker thread to encrypt it, and computes the same plaintext digest.

**Proposed behavior**
After the download, the server checks only the file size against the
receipt. The encrypt pass compares its plaintext digest and size with
the receipt. A mismatch fails with the same
`remote_workspace_capture_transfer_mismatch` error before the upload
attempt is recorded and before any object is stored.

**Reason and benefit**
One full read and one SHA-256 of each archive leave the server's main
thread (about 0.36 ms of main-thread time per MB on a loaded dev
machine; 37 ms for a 100 MB archive, at event-loop utilization 1.0 for
that time).

## What Changed

- `captureSandboxWorkspaceArchive` no longer re-hashes the downloaded
archive. It checks with `lstat` that the download is a file with the
receipt's byte size, and fails with
`remote_workspace_capture_transfer_mismatch` when it is not.
- `uploadWorkspaceSnapshot` accepts an optional `expectedArchive`
(`sha256`, `byteSize`). After the encrypt pass, it compares the sealed
plaintext digest and size with it. A mismatch throws
`remote_workspace_capture_transfer_mismatch` before the attempt enters
`metadata.uploads` and before `putFile`. The scratch ciphertext is
removed as before.
- `saveRemoteWorkspace` passes the sandbox receipt as `expectedArchive`
when it captured an archive.
- The save-level `verifyMs` timing field is removed, because that step
no longer exists. The storage-level `verifyMs` stays.
- `doc/run-log-events.md` says that the encrypt step is the only full
read and where a mismatch fails.

Callers that relied on the early hash keep their behavior:
- Unchanged-snapshot reuse (`findUnchangedWorkspaceSnapshot`) compares
the sandbox receipt, not the local hash. When it reuses the parent, the
local copy is used only to list directories, and
`readRemoteWorkspaceArchiveDirectories` verifies the manifest digest.
- Reuse of an earlier upload attempt or an already uploaded snapshot
does not read the archive. Those objects were verified when they were
stored.
- The read-only disk-full fallback builds its archive on the server, so
its receipt always matches. Its behavior does not change.
- The error message and the failure path do not change, so the save
failure code, retry and fallback stay the same.

## Verification

- Red/green: `remote-workspace-transport.test.ts` "leaves the downloaded
archive's single full read to the encrypting upload" counts main-thread
read streams of the downloaded archive during capture. On master it is 1
(fails). With this change it is 0 (passes). The encrypt pass is the one
remaining read.
- New: digest mismatch and size mismatch in `uploadWorkspaceSnapshot`
reject with the transfer-mismatch error, do not call `putFile`, and
leave the row `pending` with no recorded attempt. A retry with the right
receipt saves.
- New: an end-to-end `saveRemoteWorkspace` with a transfer that flips
one byte (same size) fails with the transfer-mismatch error, stores
nothing, keeps the writer and the previous snapshot, and a retry saves
and restores the right content.
- New: a download whose size differs from the receipt fails in capture.
- `cd server && npx vitest run
src/services/remote-workspace-snapshot-storage.test.ts
src/services/remote-workspace-transport.test.ts
src/services/remote-workspaces.integration.test.ts
src/services/remote-workspace-transfer-timeout.test.ts
src/services/remote-workspace-readonly-capture-budget.test.ts
src/services/sandbox-disk-full-fault.test.ts` passes: 6 files, 360
tests. `src/services/remote-workspace-snapshot-reuse.test.ts` passes: 32
tests.
- `npx tsc --noEmit -p server/tsconfig.json` passes.

## Risks

- Low risk. A same-size corrupt download is now detected in the encrypt
pass instead of right after the download. It is still detected before
any attempt is recorded or any object is stored.
- When the sandbox reports an unchanged workspace, a corrupt local copy
no longer fails the save. The save reuses the verified parent snapshot,
which matches the sandbox's content. The local copy is used only for its
manifest, and the manifest digest is verified.
- The save-level `saveTiming.verifyMs` field is gone. No code reads it.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 18:41:29 -06:00
..
…
…
…
…
…
…