mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 06:10:44 +01:00
DottaandClaude Opus 5.5
c4ba3974a7
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A graceful restart (K2) drains the control plane: it closes HTTP admission, suspends running runs, and the next boot resumes them > - Run admission stayed open while the HTTP listener closed. Heartbeat's shutdown flag was set only later, inside hot-restart preparation > - A run that finished in that window released its issue and promoted the queued comment wake. That started a new run that had no callback listener > - A remote warm continuation then failed with `local_runtime_api_listener_unavailable`, and the task needed a Retry (restart chaos t09) > - This pull request closes run admission before the listener closes, so the promoted wake stays queued for the next boot > - The benefit is that a comment sent during a deploy continues on its own after the restart ## Linked Issues or Issue Description Refs #19596 **What happened?** In restart-chaos wave w-20261108T192459-s9501 (Daytona warm, Codex C1, K2), a K2 SIGTERM started a graceful drain at 19:42:52 UTC, and the HTTP listener closed first. At 19:43:03 the running turn finished and released its issue. That promoted the queued comment wake, and `startNextQueuedRunForAgent` started a new run about 11 s into the drain. Backend setup resolved the local runtime API URL, found the listener closed, and threw `local_runtime_api_listener_unavailable`. The remote target is excluded from the closed-listener handoff, so the run failed, and the task waited for a manual Retry. **Expected behavior** No queued run starts once a graceful shutdown has begun. A wake promoted during the drain stays queued, and the next boot starts it with no Retry. **Steps to reproduce** 1. Run a remote warm task and post a comment while its turn runs, so the comment wake queues behind it. 2. Send SIGTERM to the control plane just before the turn finishes. 3. Before this change, a new run starts during the drain and fails with `local_runtime_api_listener_unavailable`. **Paperclip version or commit** master atd007afbf86(the wave ran onc5a172a183). ## What Changed - `heartbeat/run-admission.ts` (new): a process-wide run admission gate (`closeProcessRunAdmission`, `isProcessRunAdmissionClosed`). HTTP routes build their own heartbeat instances, so the gate cannot live in one instance. - `heartbeat/queue.ts`: a new optional dependency, `isRunAdmissionClosed`, which defaults to the process-wide gate. `startNextQueuedRunForAgent`, `resumeQueuedRuns` and `resumeQueuedRunsAfterDrain` start nothing (and read nothing) once it returns true. Wakes are still recorded and promoted; only the start is held. - `heartbeat.ts`: also honors its own `shutdownInProgress` flag. - `shutdown.ts`: `closeHttpListenerForShutdown` takes `closeRunAdmission` and calls it first, before it checks or closes the listener. `index.ts` passes `closeProcessRunAdmission`. ## Verification - Red/green, new tests: - `queue.test.ts`: "starts no run once shutdown closed run admission" (for `startNextQueuedRunForAgent` and `resumeQueuedRuns`), and "honors the process-wide run admission gate in every heartbeat instance". - `shutdown.test.ts`: "closes run admission before it stops accepting requests". - `pnpm exec vitest run src/services/heartbeat/queue.test.ts src/services/heartbeat/recovery.test.ts src/__tests__/heartbeat-comment-wake-batching.test.ts src/services/native-runtime/native-startup-retry.integration.test.ts src/shutdown.test.ts` (server): 5 files, 277 tests passed. - `PAPERCLIP_TYPECHECK_LOW_MEMORY=1 pnpm typecheck` (server): passed. ## Risks - Low risk. Only queued-run starts are held during a shutdown that has already begun; wakes, drains, suspension and finalizers are unchanged. The next boot's queued-run dispatch starts held runs. - Known remaining window: a run admitted just before shutdown can still reach backend setup after the listener closed. A local runner hands off to restart recovery there. A remote warm continuation still fails, because the remote target is excluded from that handoff (#19596) and the bootstrap retry does not cover warm continuations. This change removes the t09 sequence, where the run started after shutdown had begun; it does not remove that narrower window. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use (shell, file edits, test execution, a read-only research subagent). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>