mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
DottaandPaperclip
c92bc9e5ec
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Workspace import saves local work to an encrypted immutable snapshot before it changes storage authority. > - The snapshot uploader reads the whole object back to verify its ciphertext. > - The importer then downloads that same object again for authenticated decryption and restore. > - Both reads are required by the current call sequence, but they need not transfer the object twice. > - This PR lets that caller retain the authenticated upload readback in private scratch. > - The importer still restores the full archive, recaptures the source, checks ownership, and publishes through the existing CAS. ## Linked Issues or Issue Description Refs #19642, #19805, and #19810 for the existing workspace import and preservation owners. **What happened?** A fresh workspace import performs a PUT and two full GETs of the same encrypted snapshot. The first GET verifies and discards the object. The next GET decrypts it for restore. Already-uploaded and lost-reply recovery paths also repeat the GET. **Expected behavior** Use one authenticated readback for the exact final winning reference. Preserve the default upload behavior and returned reference. Keep every restore, source, recovery, ownership, and publication check. **Steps to reproduce** 1. Capture a local source through the real workspace archive codec. 2. Upload it through the real snapshot ledger and filesystem object backend. 3. Load and restore its archive. Count the object GETs. 4. The three baseline cases each make two GETs. The opt-in path now makes one and restores the same bytes after the original source is removed. **Paperclip version or commit** Baseline `585c885168`. The first PR head was `5b18121609`; the follow-up repairs the review findings without changing the storage optimization. ## What Changed - Add optional `verifiedArchivePath` to the existing upload owner. The default path and `WorkspaceSnapshotRef` return contract stay unchanged. - Authenticate each readback with the existing ciphertext, GCM, key-binding, and plaintext checks. - Bind retained files to the full winning reference. Recheck private file ownership and plaintext SHA/size before atomic publication to a new caller-owned path. - Clean all invocation scratch on success or failure. Do not overwrite caller files or reuse state across calls. - Use this path in POSIX workspace import. Keep full extraction, the second source capture, closed alias/source/F/writer/task checks, and CAS publication. Windows keeps the existing download path and does not bypass private-file checks. - Cover real cold restore, fresh/replay/lost-reply GET counts, late winners, private-path rejection, local tampering, and GCM failure. ## Verification - Genuine baseline RED: three real PostgreSQL/crypto cases each report two GETs where the new contract requires one. - GREEN: all 30 snapshot storage cases, one actual online migration case, and seven focused migration controls pass; 38 total. The other 232 cases in those files were unselected locally. - Review RED: four obsolete second-GET fault triggers and one uploaded-journal retry case failed before repair. The triggers now fire on the retained readback and assert that they fired. The uploaded-journal case interrupts after its real ledger commit and proves recovery and cold Git restore through the same journal. - A focused platform-selection control proves Windows retains the original two-GET path. This is not native Windows acceptance. - Strict server TypeScript, module boundaries, and `git diff --check` pass. - Local command from `server/`: `node ../cli/node_modules/tsx/dist/cli.mjs ../packages/db/src/test-postgres-command.ts -- node ../node_modules/vitest/vitest.mjs run src/services/remote-workspace-snapshot-storage.test.ts src/services/remote-workspace-online-source.test.ts src/services/remote-workspace-migration.test.ts --testNamePattern 'remote workspace snapshot storage|imports an alias with unrelated|removes plaintext verification files|rechecks a service that becomes|rechecks a writer that appears|preserves a local edit|keeps the existing download path|missing-base-uploaded-retry' --hookTimeout 90000 --maxWorkers 1 --no-file-parallelism --fsModuleCache`. - The codec timing control used an exact earlier owner and a synthetic 72.9 MB archive. Discarding final-capture output saved only 3%; this PR keeps that capture unchanged. - This removes one full object GET. It does not claim a measured hosted wall-clock improvement. - Fresh Greptile review is 5/5 on `0933a0a29a`, with no open findings. Exact-head CI is complete and green: 56 successful checks, four intended skips, and one neutral result. The second review returned Ship (4/5); its optional test-maintainability suggestion is deferred. No source change or CI restart was made for that suggestion. ## Risks - A late upload can win with a different reference. The retained file must match all fields of that final reference; otherwise the owner authenticates the winner. - A private file can change before relay. Ownership, no-follow reads, stable file identity, plaintext SHA/size, and no-overwrite publication protect that boundary. - Authentication may overlap the encrypted upload scratch. Peak local disk use can grow by up to one ciphertext file. Data remains streaming; no whole-object memory buffer is added. - Opt-in callers must supply a canonical private directory owned by the process. Existing callers that omit the option retain their behavior. - No archive format, database schema, provider lifecycle, writer protocol, admission, or background polling changes. ## Model Used OpenAI `gpt-6.1-sol`, with reasoning, tool use, and local code execution. The context window size is not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>