mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-13 07:12:00 +01:00
DottaandClaude Opus 5.5
c9226de30b
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The heartbeat service runs each agent task. A graceful shutdown
(SIGTERM) drains running native runs and hands them to the next server.
> - The drain detaches the native runner session and releases the run's
lease with the recovery state `awaiting_runner_reattach`.
> - An in-process execution can still be in run preparation at that
moment, for example while the run-log journal claim waits on a slow or
unavailable object store.
> - When that execution then lost writer authority
(`run_log_writer_authority_lost`), the generic failure path settled the
run as `adapter_failed`. The next server then saw
`native_continuation_requires_reconciliation`, and replay stayed
blocked.
> - This pull request makes a preparing execution that a restart
detached stop quietly, the same way a detaching native session already
does.
> - The benefit is that a slow preparation step plus a graceful restart
no longer turns into a failed run that needs a manual retry.
## Linked Issues or Issue Description
**What happened?**
In restart-chaos wave `w-20261110T092411-s701` (local, A1/C1, K1/K2, ST7
read-after-write lag), three of four tasks failed with R3
`retry_needed`. For run `218a0425`, the K2 SIGTERM drain wrote
`native.recovery.transition` with `awaiting_runner_reattach` and
released the lease. About 3 s later, the old server's in-process
execution, which was still in `runOutput.begin`, got
`run_log_writer_authority_lost`. `runCompletion.failRun` then settled
the run `adapter_failed` with a durable settlement. The next server
found `native_continuation_requires_reconciliation` and did not replay
the run.
**Expected behavior**
The drain hands the run to the next server. The old execution must not
settle it. A real authority loss outside a drain must still fail the
run.
**Steps to reproduce**
1. Start a native run recovery, and hold it in preparation (slow store,
ST1 outage wait, or ST7 lag).
2. Send SIGTERM so `drainRunningRunsForShutdown` suspends the run.
3. Let preparation continue and check writer authority. The run settles
`adapter_failed`.
**Paperclip version or commit**
`master` at 914e3ae14f.
## What Changed
- `native-session-executor.ts`: add `watchNativeRestartDetach(runId)`.
It records the run's restart-detach marker when the execution starts. It
reports `true` only when a later restart detach for that run happens. A
marker left by an earlier detach does not count.
- `heartbeat.ts`: the execution creates the watcher at start. In the
inner failure path (before `runCompletion.failRun`) and the setup
failure path (before `runCompletion.failRunSetup`), if a restart
detached the run and no provider dispatch started, the execution logs
the handoff and stops without settling. It uses the same teardown flag
(`nativeSessionResumeScheduled`) as
`NativeControllerDetachedForRestartError`.
- `heartbeat-native-restart-suspension.test.ts`: add a
red-before/green-after regression. A recovery execution is drained
during preparation and then checks the real run-log writer authority.
With the drain, the run stays `running` with `awaiting_runner_reattach`,
and the next controller resumes it. Without the drain (the lease only
expired), the run still fails `adapter_failed`.
## Verification
- `cd server && npx vitest run
src/__tests__/heartbeat-native-restart-suspension.test.ts`: 98 passed.
Before the fix, the new drained case failed with `status: failed,
errorCode: adapter_failed`.
- `cd server && pnpm typecheck`: passed.
- Live restart-chaos wave: see the comment below.
## Risks
- Low risk. The new branch applies only when this process's own restart
detach for this run happened after the execution started, and before any
provider dispatch. Typed native outcomes (startup cleanup, ownership
unverified, cancellation pending, resume scheduled) keep their earlier
handling because they are checked first.
- Previous-release story (upgrade-safe run state): no persisted shape,
launch identity or pack identity changes. The marker is process-local.
State saved by the previous release carries forward unchanged. The
deploy-lands-mid-run case is the regression test itself: a drain lands
during recovery preparation and the next controller resumes the same
run.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>