mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-22 11:54:35 +01:00
test(runner): wait for the background task notification turn before the next Claude turn (#22522)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Paperclip Runner converts raw Claude ACP session updates the same way ACPX does. One test proves this against the real Claude ACP bridge. > - The test runs four turns. The third turn starts a background command (`sleep 1`). When that command ends, Claude starts its own task-notification turn. > - The test starts to record raw updates before ACPX sends the next prompt. If the notification turn arrives in that window, the test records it. ACPX correctly does not give it to the prompt. The two lists then differ. > - This race stopped the Release fora722935dbcin "Verify Paperclip Runner (protocol)". > - This pull request makes the test wait until the notification turn ends before it starts the next turn. > - The benefit is a deterministic test and a Release lane that does not fail on runner speed. ## Linked Issues or Issue Description **What happened?** The Cloud readiness run 44970524141 (Release fora722935dbc) failed in `verify / Verify Paperclip Runner (protocol)`. The failed test is `acpx-runtime-adapter.claude-session-update-events.test.ts > converts the real Claude ACP stream exactly as ACPX does for every update kind it emits`. The assertion on the `turn-plan` turn received `msg_10` with `_claude/origin.kind: "task-notification"`. ACPX gave `msg_12` with `kind: "human"`. The log line "Copilot metadata verification failed … shutdown-incomplete" is the expected stderr of a different test that passed. It is not related. **Expected behavior** The test compares only the updates of the prompt that ACPX runs. A notification turn from an earlier background command must not get into the recorded list of the next prompt. **Steps to reproduce** 1. On master, add `await new Promise(resolve => setTimeout(resolve, 3_000));` in `runTurn` after `const yielded = []` for `turnId === "turn-plan"`. This holds the window between "start to record" and "ACPX sends the prompt" open while the background command ends. 2. Run `pnpm --filter @paperclipai/paperclip-runner exec vitest run src/drivers/acpx/acpx-runtime-adapter.claude-session-update-events.test.ts`. 3. The test fails with the same diff as CI (`msg_10` / `task-notification` against `msg_12` / `human`). **Paperclip version or commit** master ata722935dbc(the race exists since #21420). Classification: timing flake in the test design. It is not a product regression and not a dependency change. The task-notification turn is known product behavior (see `claude-background-tasks.ts`). The test passed in every earlier Cloud readiness run on master. ## What Changed - The test counts task-notification turn-end markers (`usage_update` with `_claude/origin.kind === "task-notification"`) in its stdout reader. - After the background turn, the test waits (up to 30 s) until that notification turn ends. Then it starts the plan turn. - No Runner source, ACPX profile, digest, pin or image changes. ## Verification - Red: the original test with the 3 s hold in the window fails with the exact CI diff (3 of 3 runs). - Green: the fixed test with the same 3 s hold passes. The fixed test without the hold passes 5 of 5 local runs. - `pnpm --filter @paperclipai/paperclip-runner run check:acpx-profiles` and `check:protocol-manifest` pass. This is a test-only change, so no new runner image or attestation pin is necessary. ## Risks - Low risk. Test-only change. If a future Claude Code release stops sending a task-notification turn, the wait fails after 30 s with a clear assertion instead of a silent pass. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edit, GitHub CLI) for investigation, the fix and local verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
a722935dbc
commit
28c11084ad
1 file changed
+7
-1
+7
-1
@@ -112,6 +112,7 @@ it("converts the real Claude ACP stream exactly as ACPX does for every update ki
|
||||
const mcpPort = (mcp.address() as { port: number }).port;
|
||||
let capture: Raw[] | null = null;
|
||||
let promptAnswered = false;
|
||||
let taskNotificationTurnsEnded = 0;
|
||||
const entry = createRequire(import.meta.url).resolve("@agentclientprotocol/claude-agent-acp/dist/index.js");
|
||||
const command: VerifiedAcpxCommandLease = {
|
||||
spawn: (args, input) => {
|
||||
@@ -125,7 +126,11 @@ it("converts the real Claude ACP stream exactly as ACPX does for every update ki
|
||||
let message: Record<string, unknown>;
|
||||
try { message = JSON.parse(line) as Record<string, unknown>; } catch { continue; }
|
||||
if (message.method === undefined && (message.result as { stopReason?: unknown } | undefined)?.stopReason !== undefined) promptAnswered = true;
|
||||
if (message.method === "session/update") capture?.push({ message, afterPromptResponse: promptAnswered });
|
||||
if (message.method !== "session/update") continue;
|
||||
capture?.push({ message, afterPromptResponse: promptAnswered });
|
||||
const update = (message.params as { update: Record<string, unknown> }).update;
|
||||
const origin = (update._meta as { "_claude/origin"?: { kind?: unknown } } | undefined)?.["_claude/origin"];
|
||||
if (update.sessionUpdate === "usage_update" && origin?.kind === "task-notification") taskNotificationTurnsEnded += 1;
|
||||
}
|
||||
});
|
||||
return child;
|
||||
@@ -185,6 +190,7 @@ it("converts the real Claude ACP stream exactly as ACPX does for every update ki
|
||||
await runTurn("turn-mcp", "KINDS-MCP: think, say something and call the probe.");
|
||||
await runTurn("turn-foreground", "KINDS-FOREGROUND: run a foreground command.");
|
||||
await runTurn("turn-background", "KINDS-BACKGROUND: run a background command.");
|
||||
await vi.waitFor(() => expect(taskNotificationTurnsEnded).toBe(1), { timeout: 30_000, interval: 25 });
|
||||
await runTurn("turn-plan", "KINDS-PLAN: write a plan.");
|
||||
for (const required of ["tool_call", "tool_call_update", "agent_message_chunk", "agent_thought_chunk", "usage_update", "available_commands_update"]) {
|
||||
expect(seen.get(required)?.acpx ?? 0, required).toBeGreaterThan(0);
|
||||
|
||||
Reference in new issue
Block a user