Files
PaperClipAI/tests
DottaandClaude Opus 5.5 179694de7d test(chaos): provider message faults for Claude (ACPX) and across a K1 restart (#20454)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The restart chaos harness (`tests/restart-chaos/`) proves that agent
work survives control-plane restarts and provider faults
> - PR #20432 added provider message faults (PV1 delay, PV2 drop, PV3
duplicate), but only for Codex, and only in waves without restarts
> - Claude runs through the ACPX sidecar, whose provider pack and
sidecar are digest-verified, so the Codex executable shim cannot reach
it
> - A late or missing terminal can also arrive across a control-plane
restart, which no wave could exercise
> - This pull request adds a Claude (ACPX) seam with no product change,
and a K1 restart that fires after a provider fault applies
> - The benefit is that governed waits, Stop and approvals are now
tested under provider message faults for both main adapters, also across
a kill -9

## Linked Issues or Issue Description

Refs #20432, #20330, #20336

**What is the problem?**
Provider message faults covered only Codex. They could not run with
restart faults. PV3, the `interrupt_ack` target, and the S1 and G1
workloads under PV1/PV2 never ran live.

**What is the expected outcome?**
The harness can delay, drop or duplicate one targeted provider message
for Claude and Codex. A PV1/PV2 fault can straddle a K1 restart, and the
grade states what a pass means.

**Why is this the right layer?**
This is test-harness work. The product is not changed. The seam replaces
only a file the runner already hashes and verifies at launch, inside a
per-wave stand-in package.

## What Changed

- `acpx-provider-faults.ts`: builds a per-wave stand-in runner package
under `<repo>/node_modules/.restart-chaos-<wave>/` (symlinks to the real
package, except a generated `dist/cli/acpx-runtime-sidecar.cjs`). It
also parses `--provider-fault-restart`.
- `provider-shim/acpx-sidecar-hook.mjs`: a server-only resolve hook that
loads `runnerd-codex-transport` (source `src/live/*.ts` or built
`dist/live/*.js`) from the stand-in package. Product code then hashes
the stand-in sidecar, and the runner verifies and launches it. The
module's own imports resolve to the real package.
- `provider-shim/acpx-fault-tap.cjs`: the stand-in installs this tap and
then runs the real sidecar bundle in the same process, as the runner's
verified CommonJS loader does. For Claude, `terminal` is the provider's
ACP `session/prompt` response after a `session/cancel`. `interrupt_ack`
is the sidecar's `turn.cancel` response, because ACP's cancel has no
acknowledgement. The provider process is still spawned and admitted by
product code.
- `--provider-fault-restart K1 [--provider-fault-restart-after-seconds
n]`: each PV1/PV2 task also kills -9 the control plane n seconds
(default 3) after its fault applied, through a new internal `provider`
trigger. PV3 and planned K cells stay rejected.
- `withProviderFaultCheck`: with a restart, the check passes only if the
fault applied once, the K1 started after it, and for PV1 no release came
first. The ordinary R1–R6 grade and the workload T-oracle still decide
whether the task recovered.
- `config.ts`: provider faults accept `--adapters codex,claude`. G1 with
a provider fault needs `--provider-fault-restart K1`, because its oracle
grades the board decision after a restart.
- README: the Claude seam, the target table per adapter, what a pass
means across a restart, and the Claude PV1 delay window.
- Unit tests: the ACP and runner-pipe taps, the generated stand-in with
a fixture sidecar, the hook against a fake runner package, planning,
config validation, and the restart check.

## Verification

- `pnpm exec vitest run --config tests/restart-chaos/vitest.config.ts`
(all pass after rebase on master)
- `node scripts/typecheck-test-projects.mjs tests/restart-chaos`
- Live waves (local, one at a time through the wave-slot protocol):

| Wave | Scenario | Result |
| --- | --- | --- |
| w-20261110T040202-s202 | Claude Q1, PV1 terminal 18 s | Task passed
its Q1 oracle; `provider_fault_applied` failed: the runner settled the
cancelled turn ~3.6 s after the cancel and closed the sidecar before the
release, so 18 s is a drop for Claude (documented) |
| w-20261110T041224-s203 | Claude Q1, PV1 terminal 1 s and 3 s | 2/2
pass (held and released) |
| w-20261110T041355-s204 | Claude Q1 and S1, PV2 terminal | 2/2 pass |
| w-20261110T043354-s205 | Codex Q1, PV3 terminal (duplicate) | 1/1 pass
|
| w-20261110T050407-s206 | Codex S1 and G1, PV1/PV2 terminal | 0/4: S1
terminal never sent (Codex exits after Stop, so `terminal` cannot apply;
documented, use `interrupt_ack`); G1 needs a restart by design (now
rejected without `--provider-fault-restart K1`). No product failure. |
| queued | Codex+Claude Q1/S1 PV1/PV2 `interrupt_ack`; Codex+Claude
Q1/G1 PV1/PV2 + K1 | Runs from master after merge (queue is ~90 min
deep) |


## Risks

- Low product risk: no product code changes.
- The seam depends on the runner computing the local sidecar path from
the transport module URL and on the ACPX provider bootstrap marker
string. If either changes, the tap logs nothing and every Claude PV task
fails `provider_fault_applied`. It does not pass silently.
- The stand-in package is written under the workspace `node_modules`
(gitignored) and removed at wave end.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Anthropic Claude, model ID `claude-opus-5-5` (Claude Code agent with
tool use and code execution)

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 01:06:25 -06:00
..