Files
PaperClipAI/tests
DottaandClaude Opus 5.5 069ec8a74e fix(restart-chaos): exempt a continuation of an uncommitted follow-up run from completion_once (#22137)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The restart-chaos harness grades each task after deploy-like
restarts, and the `completion_once` rule finds tasks that an agent
completed twice
> - A follow-up comment can arrive while a task runs. The run that
answers it can complete the task again, and the grader exempts that run
> - The grader did not exempt a recovery continuation of that answering
run when the answering run failed before it committed a completion
> - So the grader reported a duplicate completion for correct product
behavior, and the wave headline counted a false duplicated side effect
and an extra turn
> - This pull request makes a continuation (a run with `retryOfRunId`)
of an answering run that never committed a completion count as the
answering run
> - The benefit is that wave results show real duplicates only, and a
real second completion after a committed one still fails

## Linked Issues or Issue Description

**What happened?**

In a restart-chaos staging-mix wave, task t128 (opencode, fault K13)
graded `completion_once` as failed with the detail "duplicate
completion: runs 38a8772f, 833314ed each reported completion". The wake
evidence shows this sequence:

- Run 38a8772f completed the task (accepted disposition `done`) and
posted the DONE comment.
- The follow-up comment arrived before that completion. Its first two
runs (5707e93a and 2c1d364e) failed before a provider turn, so they did
not receive it.
- Run c5f9a5a4 received the follow-up and started a provider turn. It
then failed with `native_session_retry_exhausted` and did not commit a
completion.
- Execution recovery started run 833314ed with `retryOfRunId` = c5f9a5a4
(wake reason `issue_recovery_action_restored`). That run applied the
follow-up and committed `done`.

The grader marked only c5f9a5a4 as the answering run. Run 833314ed was a
continuation of it, but the grader counted it as a second completion and
as an extra turn.

**Expected behavior**

A continuation of the answering run must count as the answering run when
the answering run did not commit a completion. A continuation that
completes again after the answering run committed its completion must
still fail `completion_once`.

**Steps to reproduce**

1. Use the t128 wake evidence shape: a first run that completes, two
follow-up runs that fail before a provider turn, an answering run that
fails without a completion, and a recovery run with `retryOfRunId` set
to the answering run that completes.
2. Grade it with `gradeStagingMix` through `evidenceCompletion` or
`completionEvidence`.
3. Before this change, `completion_once` fails with "duplicate
completion: runs 38a8772f, 833314ed each reported completion".

## What Changed

- `tests/restart-chaos/follow-up-delivery.ts`: add `continuedRunId`
(reads `retryOfRunId` from the run context or the wake payload) and
`followUpAnsweringRuns`. The new function returns the runs that answered
a follow-up, plus each continuation of an answering run that did not
commit a `done` result.
- `tests/restart-chaos/staging-mix-retry.ts`: `evidenceCompletion` and
`runsAfterDone` use `followUpAnsweringRuns` to set `answersFollowUp` and
to separate answered runs from extra turns. `followUpsAnsweredLater` and
the wait statistics are not changed, so one follow-up still counts as
one answer.
- `tests/restart-chaos/wake-evidence.ts`: `completionEvidence` (the
record that the wave grade uses) uses the same function.
- `tests/restart-chaos/staging-mix-retry.test.ts`: add a t128 test group
from the real wave evidence. One test shows the continuation passes. One
test shows a continuation after a committed answering completion still
fails.

## Verification

- Red before the fix: `passes when the continuation of an answering run
that never committed completes the follow-up` failed (`completionRunIds`
returned both 38a8772f and 833314ed).
- Green after the fix: both new tests pass, and all existing truth-table
and follow-up tests pass.
- `npx vitest run --config tests/restart-chaos/vitest.config.ts`: 71
files, 844 tests passed.
- `tsc -p tests/restart-chaos/tsconfig.json --noEmit`: passed.
- I regraded the real t128 wake evidence and comments with the fixed
functions. The result is one completer (38a8772f), no extra turns, and
answered runs c5f9a5a4 and 833314ed.
- `pnpm verify` exited 2 because `follow-up-delivery.ts` is a global CI
input. Required CI supplies the broad coverage.

## Risks

- Low risk. The change is in the restart-chaos test harness only. It
does not change product code.
- The exemption needs an explicit `retryOfRunId` link to the answering
run. A recovery run with no link (for example the t005 case) still
fails, as before.
- If the answering run committed `done`, its continuation is not exempt,
so a real second completion still fails.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used in
Claude Code with tool use (shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 01:12:27 -06:00
..