From f92f91f0fbd063b679a39ba718641fa6bfe4bfef Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Thu, 12 Nov 2026 15:19:32 -0600 Subject: [PATCH] fix(native): finish a parent claim when its child results arrived before the run started (#21189) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A parent task gets a wake when one of its child tasks completes. The wake carries the child results. > - When the parent already has a run, the child result wake waits (`deferred_issue_execution`) behind that run. > - At finalization, a parent completion claim stays open (`native_child_completion_pending`) while such a wake is still queued, claimed or deferred. This guard protects child results that arrive while the parent works. > - The guard did not check when the child result arrived. A child result saved before the parent run started was already visible to that run, but it still blocked the claim. > - So the task stayed `in_progress`, the deferred wake started a second run, and the agent repeated its completion summary. > - This pull request counts a child result as undelivered only when it was saved after the finishing run started. > - The benefit is that a parent that already read its child results finishes in one turn, and child results that arrive during the run still get a fresh turn. ## Linked Issues or Issue Description Refs #15372 (added the pending child-completion guard). Refs #20935 (same family: a self-wake after a settled run). **What happened?** Restart-chaos wave `w-20261112T181948-s7101`, task t001 (OpenCode, native runner, staging-mix parent with two subtasks). Both subtasks were done before parent run `8006a205` started at 18:45:29. The run read them, posted its final comment and claimed `done` at 18:46:15.860. A child-completion wake for the parent had been deferred since about 18:42 behind that run. The finalizer found that wake and committed `native_child_completion_pending`, so the task stayed `in_progress`. 63 ms later the release promoted the deferred wake into run `9282de34` (`invocationSource: automation`, `wakeCommentId: null`). Its `paperclip.native-continuation.v1` turn carried the subtask results that the first run had already used, plus the agent's own two comments (resume delta since the base run). That run posted a second "complete" summary and only then moved the task to `done`. Root cause: `hasPendingNativeChildCompletion()` (`server/src/services/native-runtime/native-child-completion-delivery.ts`) treated every queued, claimed or deferred child-completion wake for the issue as unread, except a wake bound to the finishing run itself. It did not compare the wake's save time with the run's start. **Expected behavior** A child result that was saved before the finishing run started is not new input for that run. The completion claim completes the task. The deferred wake is cancelled at release as a wake on a terminal task. A child result saved during the run still keeps the task open and gets a fresh parent turn. **Steps to reproduce** 1. Create a parent task with a done child. Save a deferred `issue_children_completed` wake for the parent. 2. Start a native parent run after that wake was saved. Let it claim `done`. 3. Finalize the run. On `master` the decision is `native_child_completion_pending` and the task stays `in_progress`; the deferred wake then starts a second run. ## What Changed - `native-child-completion-delivery.ts`: `hasPendingNativeChildCompletion()` counts a child-completion wake only when its `createdAt` is at or after the finishing run's `startedAt`. A run without a recorded start keeps the old behavior: every such wake stays pending. The finalizer, the status-commit recheck and status reassessment all use this function. - `heartbeat-process-recovery.test.ts`: a regression test finalizes a real native done claim with a deferred child-completion wake saved before (task becomes `done`) or after (task stays `in_progress` as `native_child_completion_pending`) the run started. It then releases the issue execution through the wake queue. In the `before` case the deferred wake is cancelled and there is one run total. In the `after` case the wake is promoted into a fresh run that carries the child results. - `doc/architecture/native-status-arbitration.md`: describes when a child result counts as undelivered. ## Verification - RED on `master` (`ef6d347f79`): `heartbeat-process-recovery.test.ts` "treats a child result saved before the parent run started…" fails with decision `reasonCode: "native_child_completion_pending"`, `toStatus: "in_progress"`. This is the same decision as the wave. The "after" case passes on `master`. - GREEN: both cases pass with the fix. - Neighbouring suites pass: `native-status-arbiter-corpus` (includes the queued / claimed / deferred / consumed child-delivery cases from #15372), `heartbeat-dependency-scheduling` (revised child completion delivered after a running parent), `heartbeat-process-recovery`, `heartbeat-issue-rewake-throttle`, and `src/services/native-runtime`. - `pnpm --filter @paperclipai/server typecheck` passes. - The regression test runs finalization and release end to end. The cancel-at-release step itself is existing behavior, also covered by `modules/wake-queue/application/use-cases.test.ts` ("cancels stale assignee continuations before claiming promotion on %s tasks", `done` case). ## Risks - A child result saved before the run started no longer blocks the claim. The child status was already `done` when the run started, so the run could read it. If the clocks of two server instances differ, a child result saved during the run could look older than the run start; the window is the clock skew. - A re-dispatched native status wake gets a new row with a later `createdAt`. Such a row stays pending, which is the old behavior. - No migration. Saved wakes from older builds work unchanged, because the check reads only existing columns. - Scope: this change does not alter the comment-wake promotion path or the resume-delta builder. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 --- doc/architecture/native-status-arbitration.md | 8 ++ .../heartbeat-process-recovery.test.ts | 86 +++++++++++++++++++ .../native-child-completion-delivery.ts | 4 +- 3 files changed, 97 insertions(+), 1 deletion(-) diff --git a/doc/architecture/native-status-arbitration.md b/doc/architecture/native-status-arbitration.md index 91cd4d16621..93ba0a32f1e 100644 --- a/doc/architecture/native-status-arbitration.md +++ b/doc/architecture/native-status-arbitration.md @@ -191,6 +191,14 @@ progress or new input. The "Waiting for a reply" execution label shows only while the parked decision is still the issue's latest decision and the issue is `in_progress`. +A completion claim stays `in_progress` as `native_child_completion_pending` +while an undelivered child-completion wake for the same issue and agent is +queued, claimed or deferred. Only a child result saved after the finishing run +started counts as undelivered. A result saved before that run started was +already visible to it, so the claim completes the task and the deferred wake +is cancelled at release instead of starting another turn. A run with no +recorded start keeps every such wake pending. + ## Transactional decision commit `commitNativeStatusDecision()` applies the decision against authoritative issue diff --git a/server/src/__tests__/heartbeat-process-recovery.test.ts b/server/src/__tests__/heartbeat-process-recovery.test.ts index c90676328e5..ecb8058cfb0 100644 --- a/server/src/__tests__/heartbeat-process-recovery.test.ts +++ b/server/src/__tests__/heartbeat-process-recovery.test.ts @@ -266,6 +266,7 @@ import { import { claimNativeSessionResumptions } from "../services/native-runtime/native-finalization-reconciler.ts"; import { PaperclipControlPlanePort } from "../services/native-runtime/paperclip-control-plane-port.js"; import { finalizeNativeRun } from "../services/native-runtime/native-run-finalizer.js"; +import { createWakeQueue } from "../modules/wake-queue/index.js"; import { recordNativeAttentionAssessment } from "../services/native-runtime/work-assessments.js"; import { routeNativeAttention } from "../services/native-runtime/native-interaction-bridge.js"; import * as paperclipRunner from "../vendor/paperclip-runner/index.js"; @@ -15477,6 +15478,91 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => { .toMatchObject({ phase: "completed", label: "Completed" }); }); + it.each([ + ["before", "done", "completion_claim_policy_accepted"], + ["after", "in_progress", "native_child_completion_pending"], + ] as const)("treats a child result saved %s the parent run started as read when the run finishes the task", async ( + savedRelativeToStart, expectedStatus, expectedReason, + ) => { + const fixture = await seedStrandedIssueFixture({ status: "in_progress", runStatus: "succeeded", livenessState: "advanced" }); + const { companyId, agentId, issueId, runId } = fixture; + const childId = randomUUID(); + const contractId = randomUUID(); + const runnerInstanceId = randomUUID(); + const runStartedAt = new Date(Date.now() - 60_000); + const childResultSavedAt = new Date(runStartedAt.getTime() + (savedRelativeToStart === "before" ? -30_000 : 30_000)); + await db.insert(issues).values({ id: childId, companyId, parentId: issueId, title: "Child ledger", status: "done", + priority: "medium", responsibleUserId: "responsible-user" }); + const [childResult] = await db.insert(agentWakeupRequests).values({ + companyId, agentId, source: "automation", triggerDetail: "system", reason: "issue_execution_deferred", + requestedByActorType: "system", requestedByActorId: `native-status-wake-dispatch:${randomUUID()}`, + status: "deferred_issue_execution", createdAt: childResultSavedAt, updatedAt: childResultSavedAt, + payload: { issueId, _paperclipWakeContext: { + issueId, wakeReason: "issue_children_completed", source: "native_status_decision", + nativeChildCompletionDecisionId: randomUUID(), completedChildIssueId: childId, childIssueIds: [childId], + childIssueSummaries: [{ id: childId, status: "done", summary: "Ledger saved." }], + } }, + }).returning(); + await db.insert(completionContracts).values({ + id: contractId, companyId, issueId, revision: 1, schemaVersion: "paperclip.completion-contract.v1", + policyVersion: "phase6-v4", risk: "low", completionAuthority: "agent_claim_policy", + incompleteCriteriaPolicy: "preserve_non_terminal", + contractJson: { revision: "1", objective: "Merge the child ledgers.", criteria: [{ id: "objective", requirement: "Merge the child ledgers." }] }, + canonicalSha256: `child-result-contract-${runId}`, createdByActorType: "system", createdByActorId: "test", + }); + await db.update(heartbeatRuns).set({ + runtimeMode: "native", nativeIssueId: issueId, nativeSessionId: runId, completionContractId: contractId, + completionContractSha256: `child-result-contract-${runId}`, runnerInstanceId, + startedAt: runStartedAt, finishedAt: new Date(), + }).where(eq(heartbeatRuns.id, runId)); + await db.update(issues).set({ executionRunId: runId, executionLockedAt: runStartedAt }).where(eq(issues.id, issueId)); + await db.insert(issueComments).values({ companyId, issueId, authorType: "agent", authorAgentId: agentId, + createdByRunId: runId, body: "Both children are done; merged their ledgers. Task complete." }); + await new PaperclipControlPlanePort(db, { + companyId, issueId, runId, agentId, sessionId: runId, completionContractId: contractId, + completionContractSha256: `child-result-contract-${runId}`, sourceInstanceId: runnerInstanceId, + controlPlaneSourceInstanceId: `child-result-control-${runId}`, + }).completeRun({ + result: { + schema: "paperclip.run_result.v1", reportedWorkDisposition: "done", + summary: "Both children are done; merged their ledgers. Task complete.", + completionClaim: { contractRevision: "1", objectiveSatisfied: true, + criteria: [{ criterionId: "objective", status: "satisfied", evidenceRefs: [] }], remainingWork: [] }, + evidence: [], verification: [], attentionRequests: [], artifacts: [], + }, + terminal: { schema: "paperclip.prp.terminal.v1", runTerminalState: "succeeded", + turnTerminalState: "completed", reportedWorkDisposition: "done" }, + }); + + await finalizeNativeRun({ db, runId, workspaceFinalizeStatus: "succeeded", projectRunStatus: true }); + await createWakeQueue(db, { + resolveResponsibleUserId: async () => "responsible-user", + getRoutineEnv: async () => ({ routineId: null, env: null, responsibleUserId: null }), + resolveSessionBeforeForWakeup: async () => null, + wakeAdmissionHelpers: {} as Parameters[1]["wakeAdmissionHelpers"], + recovery: { escalateStrandedAssignedIssue: async () => {}, escalateStrandedRecoveryIssueInPlace: async () => {} }, + }).releaseIssueExecution({ companyId, runId, now: new Date() }); + + expect(await db.select().from(statusDecisions).where(eq(statusDecisions.runId, runId))).toEqual([ + expect.objectContaining({ reasonCode: expectedReason, toStatus: expectedStatus, applicationState: "applied" }), + ]); + expect(await db.select().from(issues).where(eq(issues.id, issueId))) + .toEqual([expect.objectContaining({ status: expectedStatus })]); + const runs = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, companyId)); + const [wake] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, childResult!.id)); + if (savedRelativeToStart === "before") { + expect(wake).toMatchObject({ status: "cancelled", runId: null }); + expect(runs).toHaveLength(1); + } else { + const successor = runs.find((run) => run.id !== runId); + expect(runs).toHaveLength(2); + expect(wake).toMatchObject({ runId: successor!.id }); + expect(successor!.contextSnapshot).toMatchObject({ wakeReason: "issue_children_completed", + childIssueSummaries: [{ id: childId, status: "done", summary: "Ledger saved." }] }); + } + expect(mockAdapterExecute).not.toHaveBeenCalled(); + }); + it("delays a no-progress native continuation until the rewake cooldown instead of dropping it", async () => { const fixture = await seedNativePassiveBoardResponse("same_agent", { assignmentWake: true }); const finishedAt = new Date(Date.now() - 30_000); diff --git a/server/src/services/native-runtime/native-child-completion-delivery.ts b/server/src/services/native-runtime/native-child-completion-delivery.ts index 8c16296798e..2d0b79f1160 100644 --- a/server/src/services/native-runtime/native-child-completion-delivery.ts +++ b/server/src/services/native-runtime/native-child-completion-delivery.ts @@ -1,5 +1,5 @@ import { and, eq, inArray, isNull, ne, or, sql } from "drizzle-orm"; -import { agentWakeupRequests, type Db } from "@paperclipai/db"; +import { agentWakeupRequests, heartbeatRuns, type Db } from "@paperclipai/db"; export interface NativeChildCompletionRecipient { companyId: string; @@ -24,6 +24,8 @@ export async function hasPendingNativeChildCompletion(db: Db, input: NativeChild sql`${agentWakeupRequests.payload}->>'issueId' = ${input.issueId}`, sql`nullif(${agentWakeupRequests.payload}->'_paperclipWakeContext'->>'nativeChildCompletionDecisionId', '') is not null`, or(isNull(agentWakeupRequests.runId), ne(agentWakeupRequests.runId, input.runId)), + sql`${agentWakeupRequests.createdAt} >= coalesce((select ${heartbeatRuns.startedAt} from ${heartbeatRuns} + where ${heartbeatRuns.id} = ${input.runId} and ${heartbeatRuns.companyId} = ${input.companyId}), '-infinity'::timestamptz)`, input.sourceIntentId ? ne(agentWakeupRequests.id, input.sourceIntentId) : undefined, )).limit(1); return Boolean(pending);