Files
PaperClipAI/tests
DottaandClaude Opus 5.5 e5f0e1508b test(restart-chaos): drain a max-hours hosted run from the end of arrivals, skip slot-queued tasks, count per-run loop-guard notices as information (#22533)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The restart-chaos harness has a hosted mode. It runs real agent
tasks on a hosted test stack and grades the stack's own deploys with
Gate D.
> - A 16 h hosted run saw only 2 releases. Both were clean. The run then
stopped at `--hosted-max-hours` and Gate D failed on 7 open tasks and 2
duplicated comments.
> - The artefacts show that the harness caused the failures, not the
product. The drain ended after 0.9 s. The grade aged run activity
against a late clock. The duplicate check counted a once-per-run product
notice as a duplicate.
> - A gate that fails on harness artefacts hides real product failures
and wastes long runs.
> - This pull request fixes the three harness artefacts and keeps the
gate strict for real stalls.
> - The benefit is that a max-hours run grades its last tasks after they
finish, and Gate D fails only on real problems.

## Linked Issues or Issue Description

**What happened?**
A hosted Gate D run stopped at the 16 h cap, 15 h after its last
release. `drainHosted` computed the drain cap from the last release
(14:12Z), so the cap was already in the past. The drain ended 0.9 s
after arrivals ended, with 5 runs still progressing. Tasks launched 2 to
10 min earlier were graded as open. 4 of the 7 finished hands-off within
8 min (05:38Z to 05:43Z). Cleanup cancelled the other 3 while they were
running. The grade also took its clock after about 533 task view reads,
so it aged run activity by several extra minutes. Two healthy running
tasks became "nothing pending". Two subtasks were queued behind the
agent's full 5-slot cap and were graded as stalled. The two "duplicated
agent comments" were the product's loop-guard notice, posted once by
each of two different runs on the same task.

**Expected behavior**
A run that stops at the max-hours cap drains for the bounded
`--hosted-drain-minutes` after arrivals end. Run activity is aged from
the moment the runs were read. A task whose only live runs are provably
queued behind a full agent slot cap is not graded. A loop-guard notice
that each run posts once is information, not a duplicate.

**Steps to reproduce**
1. Run `tests/restart-chaos/hosted.ts --hosted-releases 10
--hosted-max-hours 16` against a stack that deploys fewer than 10 times.
2. Let the run stop at the hours cap.
3. See `hosted_settling` with a `capAt` in the past, and
`hosted_drained` with reason `cap` less than 1 s later.

**Paperclip version or commit**
master at 324ec1f537.

## What Changed

- `drainHosted` counts the drain cap from the later of the last release
and the start of the drain. It records `drainFrom` on `hosted_settling`.
The drain is still bounded by `--hosted-drain-minutes`.
- `gradeHosted` takes the grade clock immediately before it reads the
runs. Activity age now matches the run snapshot.
- New `slotQueuedRuns` and `queuedBehindSlotCap` in `hosted-gate.ts`. A
never-started run is slot-queued only when its agent's running runs fill
`maxConcurrentRuns` and `queueWaitFairnessFrom` finds no idle slot and
no newer assignment run started ahead of it. An open task whose every
live run is slot-queued is not graded. `SUMMARY.md` names it on a `Not
graded` line, and `hosted-release-swap.json` lists it under `notGraded`.
The runs that hold the slots are still graded.
- New `isNoticeRepeat` in `release-swap-grade.ts`. The loop-guard notice
is information when each copy has a different `createdByRunId`. Two
copies from one run, or a copy without a run, are still a duplicate. The
hosted reader and the local RS1 grader both use it.
- README and SCENARIOS describe the new drain start, the `Not graded`
line and the loop-guard rule.

## Verification

- `npx vitest run --config tests/restart-chaos/vitest.config.ts
tests/restart-chaos`: 73 files, 918 tests pass.
- `npx tsc -p tests/restart-chaos/tsconfig.json`: clean.
- Red/green: the new `hosted-cutoff.test.ts` fails 6 of 8 tests with
master's `hosted-grade.ts` and `hosted-readers.ts`, and passes with this
change.
- New tests: the drain runs past a release from 15 h earlier until it is
quiet; the drain still stops at its cap; activity is aged from the run
read; queued subtasks behind a full cap are not graded and the gate
still fails on the slot holder; a queued subtask with a free slot is
still graded as "nothing pending"; loop-guard notices from different
runs are information and from one run are a duplicate, for the hosted
and the local grader.

## Risks

- Low risk. The change is in the test harness only.
- A max-hours run now takes up to `--hosted-drain-minutes` longer
(default about 57 min) before it grades.
- The not-graded rule could hide a stall only if the slot proof is
wrong. The rule needs the agent cap, a full set of running runs, and a
fair queue wait. The gate still grades and fails the runs that hold the
slots.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, read-only API reads of the test
stack).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-22 00:33:43 -06:00
..