mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-22 11:54:35 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The restart-chaos harness (`tests/restart-chaos`) proves that agent work survives a control-plane release swap > - Local waves grade each task with rules R1–R6. R1 is the time budget, with credit for a provably fair queue wait (#22446, #22452) > - Hosted mode (`hosted.ts`) runs the same proof on a real test staging instance that takes automatic releases > - Hosted mode passed a run when every task finished hands-off. It did not apply R1, and banners, failed health probes, duplicates and the swap shape were only reported > - The next acceptance gate needs 10 consecutive hosted releases with 100% hands-off, 0 Retry banners and 0 failed health probes, so the exit code must enforce these rules > - This pull request adds hosted R1 and a strict "Gate D" verdict to the hosted grade > - The benefit is that one hosted run gives a single pass or fail with the same R1 rule as the local waves ## Linked Issues or Issue Description Refs #22446 Refs #22452 Refs #21153 **What happened?** A hosted run (`hosted.ts --hosted-releases N`) exited 0 when every started task finished without a person. The run did not grade R1. A Retry banner that cleared by itself, a failed `/api/health` probe, a duplicated agent comment, a stop-then-start release or a release with no live run did not fail the run. These were only lines in `SUMMARY.md`. **Expected behavior** A hosted run passes only when every task finishes hands-off and inside the R1 budget (with the same fair queue-wait credit as the local waves), no Retry banner shows, no health probe fails, no work is duplicated, no release is stop-then-start, and each release carries at least one live run. **Steps to reproduce** 1. Run `hosted.ts --hosted-releases 1` against a test staging instance whose deploy is stop-then-start. 2. Let all tasks finish by themselves after the outage. 3. See exit 0, although health probes failed during the release. **Paperclip version or commit** `95ba0a7de6` (master) ## What Changed - `hosted-gate.ts` (new): `hostedR1` builds the R1 input from API rows and calls `gradeR1` from `grader.ts`. It uses the staging-mix budget (`stagingMixBudgetMs`), outage time only for a release whose health probes failed before the new build answered, and the fair queue-wait proof (`queueWaitFairnessFrom`) from the company run list and each agent's `maxConcurrentRuns`. A follow-up run's wait is not credited, because the API has no wake evidence. `hostedGate` lists seven checks and passes only when all hold. - `hosted-readers.ts`: run rows keep `invocationSource`, `retryOfRunId` and the wake comment ids. - `hosted-grade.ts`: grades every started task with R1, adds the `Gate D` block to `SUMMARY.md`, adds `gate` and `r1` to `hosted-release-swap.json`, and returns the gate as the pass. - `hosted-main.ts`: passes the task records to the grade. - `README.md`: documents the gate and the API proxies for per-release hand-off and reach. ## Verification - `npx vitest run --config tests/restart-chaos/vitest.config.ts tests/restart-chaos/hosted` passes (8 files, 56 tests). The new `hosted-gate.test.ts` covers R1 pass and fail, a fair and an unfair queue wait, outage credit, Retry and open tasks, and each gate check. - `npx tsc -p tests/restart-chaos/tsconfig.json --noEmit` is clean. - No hosted run was started for this change. It changes only how a finished run is graded. ## Risks - Low risk. The change is in the test harness only. - Hosted runs that passed before can now fail on R1, a banner, a failed probe, a duplicate, a stop-then-start release or an unloaded release. This is intended. - A follow-up run's queue wait gets no credit on hosted runs, so R1 can be stricter than in local waves when follow-ups queue for a long time. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>