From 04082a8fd64ea37d5121dc69cf8c2702155675b6 Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Thu, 12 Nov 2026 11:45:02 -0600 Subject: [PATCH] test(heartbeat): check preparation span parents only in the attempt trace (#21149) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The heartbeat service prepares each run in named phases and traces each phase as an OpenTelemetry span under the execution attempt span > - The test `heartbeat-preparation-gap-spans.test.ts` checks that each named preparation span has the execution attempt span as its parent > - The test failed in CI 5 times in 48 hours on unrelated PR branches, always for `heartbeat.prepare.chat_retry_admission` > - The cause is in the test: the native run trace also replays each recorded preparation phase of 50 ms or longer as a historical span with the same name, under `task.prepare` in a separate trace. When the machine is slow, `chat_retry_admission` takes more than 50 ms, and the test also checks the parent of the replayed copy > - This pull request limits the span checks to the trace of the live execution attempt > - The benefit is that the test is stable and still proves that every live preparation span is a child of the execution attempt ## Linked Issues or Issue Description **What happened?** `server/src/__tests__/heartbeat-preparation-gap-spans.test.ts` failed intermittently in CI with `AssertionError: heartbeat.prepare.chat_retry_admission: expected '' to be ''`. It failed 5 times in 48 hours on 5 unrelated PR branches. **Expected behavior** The test passes on every run. It checks only the live preparation spans of the execution attempt. **Steps to reproduce** Run the test 4 times in parallel on a loaded machine: `cd server && npx vitest run src/__tests__/heartbeat-preparation-gap-spans.test.ts`. A span dump of a failed run shows two `heartbeat.prepare.chat_retry_admission` spans. One is the live span under `task.execution_attempt` (correct). The other is the historical replay under `task.prepare` in the native run trace, made because the phase took 72 ms (the record limit is 50 ms). **Paperclip version or commit** master at f2e449f8f6. ## What Changed - The test now finds the single `task.execution_attempt` span first. Then it reads only the spans in that span's trace. - The name checks and the parent checks use only those spans. The `native.controller.*` spans are in the same trace, so their checks do not change. - No product code changes. The replay of recorded phases under `task.prepare` is intended behavior (`drainRunPreparationPhases` in `heartbeat/runtime-execution.ts` feeds the native run trace). ## Verification - Before the change, 4 parallel loops of 5 runs each: 8 pass, 12 fail. All 12 failures were the `chat_retry_admission` parent assertion. - After the change, the same loops: 36 pass, 8 fail out of 44 runs. All 8 failures were `Timed out waiting for the run to settle` in one window when the machine load average was about 34 from other work. There were 0 parent assertion failures. A second batch of 20 runs passed 20 of 20. - A diagnostic dump of a failed run confirmed the replayed span is in a different trace (`task.provider_session` > `task.prepare`) and has the native trace schema attributes. ## Risks - Low risk. This is a test-only change. - The test no longer checks spans in other traces. The live preparation spans are still checked for name and parent. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits) and code execution for the test loops. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes (not applicable: test-only change) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 --- .../src/__tests__/heartbeat-preparation-gap-spans.test.ts | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/server/src/__tests__/heartbeat-preparation-gap-spans.test.ts b/server/src/__tests__/heartbeat-preparation-gap-spans.test.ts index e9e63f45ee1..8413b682c55 100644 --- a/server/src/__tests__/heartbeat-preparation-gap-spans.test.ts +++ b/server/src/__tests__/heartbeat-preparation-gap-spans.test.ts @@ -100,7 +100,10 @@ describe.skipIf(!support.supported)("heartbeat preparation gap spans", () => { .toMatchObject({ status: "succeeded" }); await provider.forceFlush(); - const spans = exporter.getFinishedSpans(); + const attempts = exporter.getFinishedSpans().filter(span => span.name === "task.execution_attempt"); + expect(attempts).toHaveLength(1); + const attemptTrace = attempts[0]!.spanContext().traceId; + const spans = exporter.getFinishedSpans().filter(span => span.spanContext().traceId === attemptTrace); const names = new Set(spans.map(span => span.name)); const preparation = [ "heartbeat.prepare.chat_retry_admission", @@ -120,8 +123,6 @@ describe.skipIf(!support.supported)("heartbeat preparation gap spans", () => { "native.controller.state_migration", ]; for (const name of [...preparation, ...controller]) expect(names, name).toContain(name); - const attempts = spans.filter(span => span.name === "task.execution_attempt"); - expect(attempts).toHaveLength(1); for (const span of spans.filter(span => preparation.includes(span.name))) { expect(span.parentSpanContext?.spanId, span.name).toBe(attempts[0]!.spanContext().spanId); }