mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-07 07:23:08 +02:00
Nicky LeachandPaperclip
24d639abad
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox providers transfer files between the host and an agent sandbox > - The Daytona file-mapping upload path sends staged tar files without compression > - Large raw uploads use more transfer bandwidth and storage > - This pull request adds transparent zstd-3 compression for eligible inbound file mappings > - The benefit is lower transfer size without changes for callers or post-upload commands ## Linked Issues or Issue Description **Subsystem affected** The change affects `packages/plugins`, in the Daytona sandbox provider. **Problem or motivation** The Daytona file-mapping upload path sends eligible staged tar files without compression. This increases transfer size and storage use. **Proposed solution** Compress eligible files on the host with zstd level 3. Upload the compressed bytes to a confined remote scratch path. Decompress them during the existing promote command, then promote the raw file. Use the raw upload path when compression cannot run or does not reduce size enough. **Alternatives considered** Keep the raw path for all uploads. This avoids compression work but does not reduce transfer size. Add a new sandbox round trip for decompression. This adds latency, so the change uses the existing promote round trip. **Roadmap alignment** `ROADMAP.md` lists Daytona under cloud and sandbox agents. This focused plugin change does not duplicate a planned core feature. **Additional context** The directory-mapping flow stays on the raw path. Callers and post-upload commands keep the same behavior. ## What Changed - Add transparent zstd-3 compression to `syncInFileMappings`. - Upload compressed artifacts to confined remote scratch paths and decompress them during promotion. - Keep the raw upload fallback when zstd is absent, compression fails, or the compressed result does not reduce size enough. - Create the raw scratch file once with atomic exclusive no-clobber open and write through the retained descriptor. - Stage compressed host artifacts in private `0700` directories with `0600` files. - Remove temporary directories on success and failure. - Add regression tests for compressed uploads, fallbacks, decompression failures, and cleanup. ## Verification - `pnpm --dir packages/plugins/sandbox-providers/daytona test` passes with 20 tests. - The compressed success path produces a byte-identical remote file. - A target without zstd uses the raw upload path. - A decompression failure does not promote a partial raw file. - Temporary files and directories do not remain after success or failure. - The newest cleanup regression test fails when the production cleanup fix is reverted and passes with the fix. ## Risks - Compression adds host CPU work for eligible file mappings. - The raw path remains available when compression is unavailable or ineffective. - Decompression runs during the existing promote command and can fail before promotion. - The change does not alter the directory-mapping flow or caller interface. ## Model Used OpenAI Codex, GPT-5, tool use and code execution enabled. The model assisted with repository review and pull request preparation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>