Files
PaperClipAI/doc/plans
DottaandPaperclip 1d348bbef0 fix(server): restore durable agent folders with bounded concurrency (#18531)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A replacement web server restores agent files from shared storage.
> - The old restore reads each object in sequence.
> - Modest network latency makes a 1,024-file folder exceed its
deadline.
> - This pull request uses at most eight concurrent restore streams.
> - Every file must pass its size and hash checks before the folder is
published.

## Linked Issues or Issue Description

Refs #18518 and #18526. Local deployment qualification found that a 64
MiB folder could not be restored after the old server filesystem was
removed.

## What Changed

- Restore at most eight files concurrently within the existing file
authority service.
- Cancel sibling transfers on failure. Await all workers before removing
the temporary folder.
- Keep the existing deadline, file-transfer database pool, authorization
and publication rules.
- Add regression coverage for bounded concurrency, corruption, settled
cancellation, retained previous bytes and retry.
- Document the behavior and update the deployment plan with local
qualification evidence.

## Verification

- Real PostgreSQL and S3-compatible storage: 1,024 files of 64 KiB each,
plus instructions, with 25 ms added to each object GET. Delete the
entire predecessor home and disable local adoption before reading
through the successor.
- Before: the read failed after 30.065 seconds. After: the same case
passed in 4.180 seconds and fetched 1,025 objects. All owned test
resources were cleaned.
- The durable-file suite passed eight tests against PostgreSQL. Server
TypeScript compilation passed. All 56 latest-head CI checks are complete
and green or intentionally skipped, including full typecheck, build,
unit and browser suites. Greptile is 5/5 with no review findings or open
threads.
- The plan separately records a released-container overlap proof: 130
reads, 26 saves, zero failures, two deleted predecessor containers and a
retained authenticated session. That image predates this restore change.

## Risks

- A cold folder can use eight simultaneous streams instead of one. Two
file-transfer database connections bound the simultaneous restore
callers. Streams verify bytes without loading entire objects into
memory.
- The 30-second deadline is unchanged. The measured size and latency do
not qualify every folder at the maximum storage limits.
- No schema, API or UI changes. Paid-model and hosted deployment
acceptance remain separate work.

## Model Used

OpenAI Codex, GPT-6. Tool use, code execution and repository review. The
runtime does not expose a more specific model ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-01 04:09:47 -06:00
..