mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-18 23:08:26 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A native run ends in two steps: the live heartbeat exports the workspace, then it commits the accepted result and records the run's exit code, log summary and usage > - Two recovery workers can also do these steps: the workspace finalization worker and the result commit worker > - The runner posts its accepted result before its process returns. That post wakes both workers at once > - The workspace finalization worker then takes the export lock while the heartbeat still waits for the runner. The heartbeat gets "busy", stops, and leaves the run to the workers > - On staging, 45 of 135 recent succeeded native runs had no exit code and no log size because of this > - This pull request makes the workers leave a run alone while its live heartbeat in the same process still runs > - The benefit is that the heartbeat finishes its own run again, with exit code, logs and usage. The workers still recover a run that the heartbeat leaves unfinished ## Linked Issues or Issue Description Refs #22043 **What happened?** About a third of succeeded native runs had a null `exit_code` and a null `log_bytes`. Their traces show a `task.workspace_finalization` span (only the workspace finalization worker makes it) and no `execution.native_finalize` or `completion.*` spans. The live heartbeat lost the workspace export lock to the worker, threw `NativeWorkspaceFinalizationBusyError` and stopped without completing the run. **Expected behavior** The live heartbeat exports the workspace, commits the result, and records the exit code, log summary and usage. The workers act only when the heartbeat cannot finish the run. **Steps to reproduce** 1. Start a native run that syncs its workspace. 2. Let the runner post its accepted result before its process returns. The result post wakes the workspace finalization worker at once. 3. The worker exports the workspace first. The heartbeat gets `acquired: false` and stops. The result commit worker then commits the run without the heartbeat's completion data. **Relevant logs or output** Staging run `cb1cfc23-79eb-4528-bcdc-dcf639430d31`: the worker restored the workspace in about 51s. The worker then committed the status decision and projected the run as succeeded. The heartbeat wrote no completion data. ## What Changed - Add `server/src/services/native-runtime/live-native-finalization.ts`. It keeps an in-process reservation for each run whose live heartbeat is still running. - The heartbeat (`runtime-execution.ts`) reserves a native run before it dispatches. It releases the reservation when its execution ends. The release wakes both worker queues for that run. - `withNativeWorkspaceFinalizationOwnership` returns `owner_active` to every caller except the run's own heartbeat while the run is reserved. The workspace finalization worker already waits for a release notification on `owner_active`. - The result commit worker skips a reserved run. The release notification wakes it again. ## Verification - New regression tests, red before the fix and green after: - `native-workspace-finalization-work.test.ts`: "leaves a result posted mid-run to the live heartbeat, then resumes after its release". Without the fix the worker exported the workspace while the heartbeat held the run. - `native-result-commit-work.test.ts`: "leaves a live heartbeat's run to it and commits only what the heartbeat leaves behind". Without the fix the worker committed the run while the heartbeat held it. - Both tests also prove that the workers still finish the run after the heartbeat releases it without finishing. - `pnpm verify` (affected tests) passes. Server typecheck passes. - After deploy: measure the share of succeeded native runs on staging with a null `exit_code`. It should go down toward 0%. ## Risks - The reservation is in memory. A worker in another server process (for example during a deploy overlap) can still race the heartbeat. That case behaves as before this change. - If the heartbeat stops in an unusual way, its `finally` block releases the reservation and wakes the workers. If the process crashes, the reservation goes away, and startup discovery recovers the run as before. - While a heartbeat is still running after the runner posted its result, the workers wait. Before, they could start the export at once. The heartbeat does the same work, so the result reaches the user at about the same time. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>