mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-22 11:54:35 +01:00
ac02de7732e8aad2d44304792db8cec4a5a0ebc0
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
b853cc28ae |
perf(server): reach readiness without waiting for backlog-sized recovery passes (#21014)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server process starts, recovers saved work, and then reports ready on `/api/ready`. The deploy health gate waits for that signal. > - On a busy staging workspace, one boot took 85 s to bind the listener and 4.4 minutes to report ready. Small, idle stacks take 20–90 s. > - Two parts of that time do not need to block readiness. The server published several hundred UI assets one at a time before it bound the listener. After the listener bound, it waited for the first pass of each periodic recovery queue, and those passes grow with the backlog. > - The volume-free data gate, which can refuse a start, ran last. By then the server had already dispatched queued runs that a refusal would abandon. > - This pull request moves the refusal gate to the start, publishes the assets 8 at a time with replayable bodies, and runs the backlog-sized first passes after readiness, in their original order. > - The benefit is a faster time from container start to ready, and no work admitted before a gate that can refuse to start. ## Linked Issues or Issue Description No public issue exists. This is the problem description (enhancement template). **Is your enhancement related to a problem? Please describe.** Server boot time grows with the saved backlog. These timestamps come from the log of one staging boot (release |
||
|
|
82e0393598 |
fix(ui-assets): retry a throttled asset upload and never fail startup on a storage outage (#21010)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A hosted Paperclip server can run volume-free. Before it serves, it publishes its UI build assets to shared object storage, so an older browser tab can still load its chunks after the old container is gone > - A build has several hundred assets. The server uploads them one after another at startup > - Object storage throttles a burst with `SlowDown` (HTTP 503). The upload body is a stream, so the storage client cannot replay the request, and the error leaves `startServer` > - The process exits, the host restarts it, and the next start uploads the same assets again. A workspace stayed down while four starts in a row failed this way > - This pull request retries a throttled asset and lets the server start when storage stays unavailable > - The benefit is that a storage slowdown cannot keep a server from starting ## Linked Issues or Issue Description Refs #20990, #20952 (other startup failures found on the same hosted workspace). **What happened?** A hosted volume-free workspace received a new release. Each start logged many `An error was encountered in a non-retryable streaming request.` lines and then `Paperclip server failed to start` with `SlowDown: Please reduce your request rate.` for a key under `ui-assets/v1/`. The four starts failed after 44, 42, 79 and 90 seconds, each on a later asset name. **Expected behavior** The server waits and repeats a throttled upload. If shared storage stays unavailable, the server starts and serves its build from local files. **Steps to reproduce** 1. Run a server with `PAPERCLIP_SHARED_UI_ASSETS=true` and S3-compatible storage. 2. Make the storage answer `SlowDown` (HTTP 503) to one `PutObject` during startup. 3. The server exits with `Paperclip server failed to start`. **Paperclip version or commit** `52cdf76bf3e0cdf29037b63bf0f5e99aa464eab7` ## What Changed - `retainedUiAssets.publish`: each asset is published in a retry loop. A throttled or server-side storage failure waits (0.5 s to 8 s, seven waits at most) and repeats the asset with a fresh stream. - `isTransientStorageError`: recognizes `SlowDown`, throttling, request timeout and HTTP 429/500/502/503/504 answers. - `app.ts`: when publication still fails with a transient storage error, the server logs an error and continues. Other failures, such as a name collision with different bytes, are still fatal. - Tests for the retry, for the give-up path, and for the error classification. ## Verification - `npx vitest run src/__tests__/retained-ui-assets.test.ts` in `server/`: 4 tests passed. The two new tests fail on `master` because `publish` rejects on the first `SlowDown` and takes no `sleep` option. - Not run locally: the full suite. CI supplies that coverage. - Not verified: a live start against throttling storage. ## Risks - A start can take up to about 31 seconds longer for each throttled asset before it gives up. - If storage stays unavailable, the server starts without its assets in shared storage. An older browser tab can then fail to load a chunk after the old container is gone. The server logs this at error level. - Collision detection is unchanged. ## Model Used Claude Fable 5.2 (`claude-fable-5-2`), through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com> |
||
|
|
006d5691a6 |
feat(server): support volume-free web server handoff (#18518)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its web server also owns controller admission and user commands. > - A rolling deployment starts a replacement before the old server exits. > - Shared database state alone does not preserve local agent files, browser chunks, or in-process commands. > - This pull request adds an opt-in ephemeral server contract and a local overlap test. > - The benefit is a web server that can relinquish traffic and local files without losing acknowledged work. ## Linked Issues or Issue Description Refs #18403, #18327, #18335, #18341. **Current behavior** The control-plane web server still has local agent-file authority and local command targets. A new container can miss saved bytes or send a command to the wrong process. An existing browser tab also needs chunks from its original build. **Proposed behavior** Eligible servers share only PostgreSQL, object storage and stable secrets. A replacement waits for readiness. The old server drains admitted requests, keeps owner commands available through handoff, and releases controller leases. Explicit maintenance adopts existing files before the volume is removed. **Reason and benefit** This is the application prerequisite for rolling app-only deployments. Schema migrations and permanent multi-replica operation are outside this change. ## What Changed - Store agent-file manifests in PostgreSQL and immutable bytes in S3. Treat local roots as disposable projections. Preserve existing permission checks and edit conflicts. - Add an explicit migration, eligibility and object-retention command to the production runtime bundle. - Add `/api/ready`, admitted-request drain, browser restart signaling and retained immutable UI chunks. - Carry transaction-aware work hints across controllers during handoff and active execution. Keep listeners closed when idle. - Deliver durable Stop and runtime responses to the execution owner. Recheck request lifecycle and access before response dispatch. - Reject local-only configuration in ephemeral mode. Keep the feature disabled for existing installations by default. - Add an isolated PostgreSQL/S3/proxy/browser deployment harness with independent server homes and retained failure evidence. ## Verification - All CI checks pass on `ce17b95019d3cf1cabd357d19b6d37f96782cc5c`: workspace typecheck, build, Runner checks, server/general tests and browser shards. Greptile reviewed the runtime changes at 5/5 with all findings resolved; the final commit updates maps and the plan. - Local targeted checks passed: coordinator 27; Stop/recovery 33; PostgreSQL command delivery 6; route/workspace regressions 8; bootstrap Stop 1; restart recovery 75. PostgreSQL command tests simulate the provider edge. - Local browser proof passed 91 reads and 19 saves; reconnect took 1,013ms. The tab opened Audit from S3 after deletion of A's whole home/build. This uses compiled workspace code and local-trusted auth, not released containers or live agents. - Local workspace typecheck/build passed; current server and runtime bundle passed. The earlier broad local suite was stopped after three hours and is not a passing receipt. Its retained failures and isolated reruns are documented in the plan. - Stacked #18526 adds stricter graceful-exit proof, an unready replacement control, live Runner qualification and a Stop handoff correction. Hosted enablement remains gated on those results and Cloud #1096. No hosted stack has been enabled. ## Risks - Apply migration 0422 before enabling the feature. Existing data must pass each owner's migration and eligibility checks. Do not discard a volume based on a single migration page. - Durable manifests remain authoritative after the opt-in is disabled. Rolling back to a binary that ignores them is unsupported. - Cold directory restoration and complete shutdown timing still need qualification before enabling a hosted stack. - Retained UI assets must cover the supported browser and rollback window. Cleanup is explicit; no new idle maintenance polling is introduced. - This change does not claim production zero downtime or authorize a fleet-wide enablement. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window limit are not exposed in this session. Used reasoning, code editing, shell execution and browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |