mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 13:01:05 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs store every relayed runner event in `heartbeat_run_events`. Each stored event also locks and updates the run's `heartbeat_runs` row. > - Text deltas already commit in batches. Every other event, including each `tool.execution.progressed` event, commits in its own transaction. > - Tool receipts and usage accounting lock the same run row. Under parallel tool calls, accounting failed with "other updates to the same run kept conflicting", and a provider-pack receipt save hit a lock timeout. > - #21520 made events that arrive while a batch commits join the next batch. This pull request adds tool progress events to the pipelined types, so they use those batches too. > - The benefit is fewer run-row transactions in tool-heavy turns, without changing event order. ## Linked Issues or Issue Description **What happened?** A routine run started several subagents that made about 190 tool calls in about 10 minutes. Three tool calls failed with "could not save this run's budget or usage records because other updates to the same run kept conflicting", and a provider-pack admission receipt failed with a lock timeout (55P03). Every tool progress event of the run was a separate transaction that updated the same run row. **Expected behavior** Observational events such as tool progress commit in batches, so they hold the run row lock fewer times. **Steps to reproduce** 1. Run a turn that makes many tool calls in parallel. 2. Each `tool.execution.progressed` event commits in its own transaction and updates `heartbeat_runs`. **Relevant logs or output** ``` could not save this run's budget or usage records because other updates to the same run kept conflicting native provider-pack admission receipt could not be saved (cause=55P03) ``` Related: #21531 tried letting the runner relay events faster, which would make this contention matter more. It is closed. ## What Changed - `paperclip-control-plane-port.ts`: - `pipelinedEventTypes` names `item.delta` and `tool.execution.progressed`. Both go through the batched append path and its group commit from #21520. - The text-delta boundary is now tracked when an event is appended: a pipelined event that is not a delta ends it, and a delta with text starts it. The first text delta after any other event still commits at once. - `contracts/control-plane-port.ts`: optional `pipelinedEventTypes`, and the `streaming` option documents that it applies to those types. - `native-session-runtime.ts`: the runtime pipelines the port's `pipelinedEventTypes`, and only `item.delta` when the port does not name them. - Tests: - `paperclip-control-plane-port.test.ts`: three progress events appended while a batch commits share one transaction, and a following event commits after them. - `native-session-runtime.test.ts`: progress appends start without waiting only when the port pipelines them. ## Verification - After rebasing onto current master: `pnpm exec vitest run src/services/native-runtime/paperclip-control-plane-port.test.ts` (in `server/`): 108 passed. - `npx vitest run src/native-session-runtime.test.ts` (in `packages/paperclip-runner`): 298 passed, 1 failed. `settles a governed wait through real Runner shutdown after a missing Codex terminal` needs the built Rust test binaries (`fake-codex-app-server`), which my checkout does not have. CI builds them. - `pnpm exec tsc --noEmit -p .` in `server/` (after building its workspace dependencies) and in `packages/paperclip-runner` pass. - `npx vitest run src/__tests__/chat-channels.integration.test.ts src/__tests__/heartbeat-process-recovery.test.ts src/services/native-runtime/native-plan-wait.test.ts src/services/native-runtime/paperclip-control-plane-port.test.ts` (in `server/`): 2,586 passed, 2 failed. `binds Slack tools to the admitted linked request...` reads a skills checkout that is not next to my worktree. `materializes direct external-chat finals once...` failed in the full file run but passes alone, with and without this change. ## Risks - Progress events now commit up to one transaction later than before. Their receipts and live UI updates follow their batch. - A batch that contains progress events still writes rows one by one inside its transaction, because progress events register chat-progress wakes. The gain is one transaction and one lock wait per batch, not one row update per batch. - If a batch fails, every event in it fails, and the runner resends them. That is the same as for text deltas today. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>