Files
PaperClipAI/server/src/__tests__
DottaandClaude Opus 5.5 d26372dc8a test(restart-chaos): add a result trigger and recovered-run comment-wake guards (#19689)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The restart chaos harness kills the control plane at chosen moments
and grades whether each task still finishes correctly
> - One wave lost a user's follow-up comment after a kill that landed
after the run accepted its result but before the run settled
> - No existing trigger aims a fault at that moment, so the case was
hard to reproduce on purpose
> - The recovery path that settles such a run had no regression test
that covers the saved comment
> - This pull request adds a `result` trigger to the harness and a
regression test for the recovery path
> - The benefit is that the harness can aim faults at the
result-accepted window, and a change that drops, repeats, or re-wakes a
saved comment fails a test

## Linked Issues or Issue Description

**What happened?**
In a restart-chaos storm wave, a user posted a comment while a native
run was active. A hard kill (K1) landed after the run accepted its
result and before the run settled. The run settled `succeeded` and the
task stayed `done`, but no follow-up run read the comment.

**Steps to reproduce**
The original failure did not reproduce. Three targeted waves ran 12
tasks with K1 at comment time or in the new result window. Every
follow-up comment was consumed exactly once. The new regression test
also passes on current `master`. The test and the trigger guard the path
and do not claim a fix.

**Expected behavior**
A comment posted during a run is consumed exactly once after the run
ends, including when restart recovery settles the run.

## What Changed

- `tests/restart-chaos`: add `--triggers result` (C1 only). K1 fires 0–3
s after the run that received the follow-up proposes or accepts its
result (`run.result.proposed` or `run.result.accepted`). The fault fires
only while that run is still `running`: the observer checks the latest
sample and rereads the runs right before dispatch. A run that settled
during the delay, or a follow-up posted at rest, records a `result`
miss. Add poll-driven unit tests for these cases and a README row.
- `server/src/__tests__/native-run-projection-work.test.ts`: add
embedded Postgres tests in which background recovery, not the executor,
settles the run (projection, environment settlement, and
deferred-receipt workers), with a deferred user comment and an active
local lease left by the killed executor. The tests check:
- The comment is promoted into exactly one follow-up run, and a second
restart adds none.
- A comment the run already received as a same-turn steer is not woken
again. The other saved comment still is.
- A queue whose only comment was steered stays cancelled, and no
follow-up starts.
- A Retry click does not consume the comment a second time. A click that
observed the source is stale. A click that observed the follow-up joins
it.

## Verification

- `cd tests/restart-chaos && npx vitest run --config vitest.config.ts`
(96 tests pass)
- `cd server && npx vitest run
src/__tests__/native-run-projection-work.test.ts` (54 tests pass)
- Live, local, ACPX Claude, Codex and OpenCode: `node
cli/node_modules/tsx/dist/cli.mjs tests/restart-chaos/main.ts --env
local --workloads C1 --triggers comment --faults K1 --adapters
codex,claude,opencode --tasks 4 --concurrency 2 --rc06-masked` passed 4
of 4. Two waves with `--triggers result` consumed each follow-up exactly
once. One task in those waves found a separate defect: a kill 1.4 s
after the result commit left the run `running` behind the dead
controller's 20-minute finalization lease. A separate pull request fixes
that defect.

## Risks

- Low risk. These are test and harness changes only. No production code
changes.
- The `result` trigger depends on the observer's poll interval (about 3
s), so the exact kill point within the window varies.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, run as an agent in
Claude Code with tool use (shell, file editing, local test execution).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 19:53:58 -06:00
..
…

Server Tests

Server tests that need a real PostgreSQL process must use ./helpers/embedded-postgres.ts instead of constructing embedded-postgres directly.

The shared helper creates a throwaway data directory and a reserved-safe loopback port for each test database. This protects the live Paperclip control-plane Postgres from server vitest runs; see PAP-2033 for the incident that introduced this guard.