mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 21:34:48 +01:00
fix(issues): release a done blocker whose ended run already finalized (#21948)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A parent task can wait for its subtasks through "blocks" relations. When the last blocker is done, the parent wakes with `issue_children_completed` or `issue_blockers_resolved` > - A done blocker with workspace work also passes a workspace-finalize barrier. The barrier holds dependents until the blocker's run saves its workspace > - The barrier reads the blocker's latest workspace operation. A save that starts inside the run's finalize and ends after it, or any operation that the same run records after its finalize, makes the barrier look unsaved > - That run has ended, so it records no further finalize. Nothing clears the barrier, every wake for the parent becomes a skipped `issue_dependencies_blocked` wait, and the periodic backstop also sees the blocker as unresolved > - This pull request treats a settled operation as saved when its run has ended and that run recorded a successful finalize for the blocker issue > - The benefit is that the parent gets its children-completed run, and parents that are stuck now are released by the existing dependency backstop with no manual step ## Linked Issues or Issue Description Refs #19219, #17206, #18665. #19219 releases the barrier when a terminal run recorded no finalize at all. This change is narrower: the run must have recorded a successful finalize for the blocker issue. A failed finalize with no later success still holds the gate (#17206 owns that recovery). **What happened?** A parent task created two subtasks. Subtask 1 finished and its run saved the workspace. Then the parent run bound both subtasks as blockers and moved the parent to `blocked` (`durable_dependency_blocker_bound`). Subtask 2 finished 30 seconds later. The `issue_children_completed` wake for the parent was skipped and recorded as `issue_dependencies_blocked` with `unresolvedBlockerIssueIds` set to subtask 1, which had been done for 2.5 minutes. The parent showed `pendingFinalizeBlockerIssueIds: [subtask 1]` and `canRetry: false`. No later wake came. The parent stayed blocked until a person changed it by hand. **Expected behavior** When every blocker is done and each blocker's run has finished with a successful workspace finalize, the parent gets one children-completed run. **Steps to reproduce** 1. Create a parent with two subtasks on separate execution workspaces. 2. Finish subtask 1. Its run records a successful `workspace_finalize`, and a `remote_workspace_snapshot` of the same run ends after the finalize ends. The incident run had a failed copy-back that was retried during a restart overlap. 3. Bind both subtasks as blockers of the parent, then block the parent. 4. Finish subtask 2 and dispatch the native `issue_children_completed` intent. 5. Before this change, the wake is skipped as `issue_dependencies_blocked` and no run starts. **Paperclip version or commit** master `6205b24d54`. ## What Changed - `pendingFinalizeBlockerCondition` (`server/src/services/issues.ts`) has one more release rule. The latest blocker operation is settled (`succeeded`, `failed` or `skipped`). Its heartbeat run is terminal. That run recorded a successful `workspace_finalize` for the blocker issue on the same workspace (or with no workspace id). Readiness, the queue dependency gate, blocker attention and the periodic backstop all read this condition. - Live runs keep the old behavior. A save that ends after its finalize, or an operation that starts after it, still holds dependents while the run is running. - Foreign-run operations, unfinished operations and failed finalizes with no success still hold dependents. ## Verification - New regression test `heartbeat-dependency-scheduling.test.ts` › "runs the parent's children-completed wake when a child it bound after completion saved after its finalize". It replays the incident order: child 1 done and finalized, its snapshot ends after the finalize, the parent binds both children and blocks, child 2 finishes, the native children-completed intent dispatches. Red before the fix (no run starts; the wake becomes `issue_dependencies_blocked`). Green after the fix (a run starts with the intent's context, and no dependency wait is recorded). - `issues-service.test.ts` nested-snapshot matrix: `late-finish` and `later-operation` now run with a live owner run (still pending) and with a terminal owner run (released). `covered`, `foreign-run`, `unfinished` and `failed-finalize` stay as before. - `cd server && npx vitest run src/__tests__/heartbeat-dependency-scheduling.test.ts src/__tests__/issues-service.test.ts` passes (207 tests). - `pnpm verify` and `tsc --noEmit` for `server` pass locally. ## Risks - Low risk. The change only releases a barrier for a run that has already ended after a successful finalize for the issue. That run cannot write another finalize, so the old rule held dependents forever with no path to release them. - An operation that a terminal run recorded after its finalize is no longer a reason to hold dependents. Such work can never be saved by that run. Holding the parent forever does not protect it. - No schema, API or UI contract change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, test execution) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
784bc06276
commit
b2b2c1e60b
3 files changed
+118
-6
No files matched your search
@@ -277,6 +277,91 @@ describeEmbeddedPostgres("heartbeat dependency-aware queued run selection", () =
|
||||
}
|
||||
}, 45_000);
|
||||
|
||||
it("runs the parent's children-completed wake when a child it bound after completion saved after its finalize", async () => {
|
||||
const companyId = randomUUID();
|
||||
const agentId = randomUUID();
|
||||
const parentId = randomUUID();
|
||||
const firstChildId = randomUUID();
|
||||
const secondChildId = randomUUID();
|
||||
const firstWorkspaceId = randomUUID();
|
||||
const secondWorkspaceId = randomUUID();
|
||||
const firstRunId = randomUUID();
|
||||
const secondRunId = randomUUID();
|
||||
const decisionId = randomUUID();
|
||||
await db.insert(companies).values({ id: companyId, name: "Child completion",
|
||||
issuePrefix: `C${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
|
||||
requireBoardApprovalForNewAgents: false, defaultResponsibleUserId: "responsible-user" });
|
||||
await db.insert(companyMemberships).values({ companyId, principalType: "user", principalId: "responsible-user", status: "active", membershipRole: "member" });
|
||||
await db.insert(agents).values({ id: agentId, companyId, name: "Parent", role: "engineer",
|
||||
status: "active", adapterType: "codex_local", adapterConfig: {}, permissions: {},
|
||||
runtimeConfig: { heartbeat: { wakeOnDemand: true, maxConcurrentRuns: 1 } } });
|
||||
await db.insert(executionWorkspaces).values([firstWorkspaceId, secondWorkspaceId].map((id) => ({
|
||||
id, companyId, mode: "shared_workspace", strategyType: "project_primary", name: "Child workspace", status: "active",
|
||||
})));
|
||||
await db.insert(issues).values([
|
||||
{ id: parentId, companyId, title: "Parent", status: "in_progress",
|
||||
assigneeAgentId: agentId, responsibleUserId: "responsible-user" },
|
||||
{ id: firstChildId, companyId, parentId, title: "First child", status: "done",
|
||||
assigneeAgentId: agentId, executionWorkspaceId: firstWorkspaceId, responsibleUserId: "responsible-user",
|
||||
completedAt: new Date("2026-11-16T23:33:00.000Z") },
|
||||
{ id: secondChildId, companyId, parentId, title: "Second child", status: "in_progress",
|
||||
assigneeAgentId: agentId, executionWorkspaceId: secondWorkspaceId, responsibleUserId: "responsible-user" },
|
||||
]);
|
||||
await db.insert(heartbeatRuns).values([
|
||||
{ id: firstRunId, companyId, agentId, issueId: firstChildId, status: "succeeded",
|
||||
startedAt: new Date("2026-11-16T23:28:28.000Z"), finishedAt: new Date("2026-11-16T23:32:59.430Z") },
|
||||
{ id: secondRunId, companyId, agentId, issueId: secondChildId, status: "running",
|
||||
startedAt: new Date("2026-11-16T23:31:17.000Z") },
|
||||
]);
|
||||
await db.insert(workspaceOperations).values([
|
||||
{ companyId, executionWorkspaceId: firstWorkspaceId, issueId: firstChildId, heartbeatRunId: firstRunId,
|
||||
phase: "workspace_finalize", status: "succeeded",
|
||||
startedAt: new Date("2026-11-16T23:32:57.000Z"), finishedAt: new Date("2026-11-16T23:32:59.000Z") },
|
||||
{ companyId, executionWorkspaceId: firstWorkspaceId, issueId: firstChildId, heartbeatRunId: firstRunId,
|
||||
phase: "remote_workspace_snapshot", status: "succeeded",
|
||||
startedAt: new Date("2026-11-16T23:32:58.000Z"), finishedAt: new Date("2026-11-16T23:32:59.200Z") },
|
||||
]);
|
||||
await db.insert(issueRelations).values([firstChildId, secondChildId].map((blockerId) => ({
|
||||
companyId, issueId: blockerId, relatedIssueId: parentId, type: "blocks" as const, createdByAgentId: agentId,
|
||||
})));
|
||||
await db.update(issues).set({ status: "blocked", blockedTransitionAt: new Date("2026-11-16T23:34:52.374Z") })
|
||||
.where(eq(issues.id, parentId));
|
||||
await db.insert(workspaceOperations).values({ companyId, executionWorkspaceId: secondWorkspaceId,
|
||||
issueId: secondChildId, heartbeatRunId: secondRunId, phase: "workspace_finalize", status: "succeeded",
|
||||
startedAt: new Date("2026-11-16T23:35:20.000Z"), finishedAt: new Date("2026-11-16T23:35:22.000Z") });
|
||||
await db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date("2026-11-16T23:35:23.250Z") })
|
||||
.where(eq(heartbeatRuns.id, secondRunId));
|
||||
await db.update(issues).set({ status: "done", completedAt: new Date("2026-11-16T23:35:23.218Z") })
|
||||
.where(eq(issues.id, secondChildId));
|
||||
const [intent] = await db.insert(agentWakeupRequests).values({ companyId, agentId,
|
||||
source: "automation", triggerDetail: "system", reason: "issue_children_completed",
|
||||
requestedByActorType: "system", requestedByActorId: "native-status-committer",
|
||||
idempotencyKey: `issue_children_completed:${parentId}:${secondChildId}:${decisionId}`,
|
||||
payload: { issueId: parentId, taskId: parentId, _paperclipWakeContext: {
|
||||
issueId: parentId, taskId: parentId, source: "native_status_decision",
|
||||
nativeChildCompletionDecisionId: decisionId, completedChildIssueId: secondChildId,
|
||||
childIssueIds: [firstChildId, secondChildId],
|
||||
} },
|
||||
}).returning();
|
||||
|
||||
await heartbeat.dispatchPendingNativeStatusWakeups({ companyId });
|
||||
expect(await waitForCondition(async () => mockAdapterExecute.mock.calls.length === 1, 10_000)).toBe(true);
|
||||
await heartbeat.drainActiveRunExecutions();
|
||||
|
||||
const dependencyWaits = await db.select().from(agentWakeupRequests).where(and(
|
||||
eq(agentWakeupRequests.companyId, companyId), eq(agentWakeupRequests.reason, "issue_dependencies_blocked"),
|
||||
));
|
||||
expect(dependencyWaits).toEqual([]);
|
||||
const parentRuns = await db.select().from(heartbeatRuns).where(and(
|
||||
eq(heartbeatRuns.companyId, companyId), sql`${heartbeatRuns.contextSnapshot}->>'issueId' = ${parentId}`,
|
||||
));
|
||||
const childrenCompletedRun = parentRuns.find((run) => run.contextSnapshot?.nativeStatusWakeIntentId === intent!.id);
|
||||
expect(childrenCompletedRun?.contextSnapshot).toMatchObject({
|
||||
wakeReason: "issue_children_completed", nativeStatusWakeIntentId: intent!.id,
|
||||
completedChildIssueId: secondChildId,
|
||||
});
|
||||
}, 45_000);
|
||||
|
||||
it("keeps a saved native continuation behind its active predecessor without retagging that run", async () => {
|
||||
const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID();
|
||||
let finishPredecessor!: () => void;
|
||||
|
||||
@@ -4773,13 +4773,23 @@ const dependencyTestDatabaseUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_ISSU
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["covered", "foreign-run", "unfinished", "failed-finalize", "late-finish", "later-operation"])(
|
||||
"requires a successful owning finalize to cover a nested snapshot: %s", async kind => {
|
||||
it.each([
|
||||
["covered", "succeeded", true],
|
||||
["foreign-run", "succeeded", false],
|
||||
["unfinished", "succeeded", false],
|
||||
["failed-finalize", "succeeded", false],
|
||||
["late-finish", "running", false],
|
||||
["later-operation", "running", false],
|
||||
["late-finish", "succeeded", true],
|
||||
["later-operation", "succeeded", true],
|
||||
] as const)(
|
||||
"requires a successful owning finalize to cover a nested snapshot: %s (run %s)", async (kind, runStatus, ready) => {
|
||||
const f = await seedSharedWorkspaceDependency();
|
||||
const runId = randomUUID(), foreignRunId = randomUUID();
|
||||
await db.insert(heartbeatRuns).values([runId, foreignRunId].map(id => ({
|
||||
id, companyId: f.companyId, agentId: f.assigneeAgentId, status: "succeeded",
|
||||
})));
|
||||
await db.insert(heartbeatRuns).values([
|
||||
{ id: runId, companyId: f.companyId, agentId: f.assigneeAgentId, status: runStatus },
|
||||
{ id: foreignRunId, companyId: f.companyId, agentId: f.assigneeAgentId, status: "succeeded" },
|
||||
]);
|
||||
const started = new Date("2026-05-23T22:00:00.000Z");
|
||||
const finished = new Date("2026-05-23T22:00:02.000Z");
|
||||
await db.insert(workspaceOperations).values([
|
||||
@@ -4797,7 +4807,6 @@ const dependencyTestDatabaseUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_ISSU
|
||||
startedAt: new Date(finished.getTime() + 100), finishedAt: new Date(finished.getTime() + 200),
|
||||
}] : []),
|
||||
]);
|
||||
const ready = kind === "covered";
|
||||
expect((await svc.listWakeableBlockedDependents(f.blockerId)).map(issue => issue.id))
|
||||
.toEqual(ready ? [f.dependentId] : []);
|
||||
await expect(svc.getDependencyReadiness(f.dependentId)).resolves.toMatchObject({
|
||||
|
||||
@@ -2625,6 +2625,24 @@ function pendingFinalizeBlockerCondition(companyId: string, issueId: SQL, worksp
|
||||
and recovery_finalize.started_at >= coalesce(latest_blocker_operation.finished_at, latest_blocker_operation.started_at)
|
||||
)
|
||||
)
|
||||
and not (
|
||||
latest_blocker_operation.status in ('succeeded', 'failed', 'skipped')
|
||||
and exists (
|
||||
select 1 from heartbeat_runs settled_owner_run
|
||||
where settled_owner_run.company_id = ${companyId}::uuid
|
||||
and settled_owner_run.id = latest_blocker_operation.heartbeat_run_id
|
||||
and settled_owner_run.status in (${sql.join([...TERMINAL_HEARTBEAT_RUN_STATUSES].map(status => sql`${status}`), sql`, `)})
|
||||
)
|
||||
and exists (
|
||||
select 1 from workspace_operations settled_run_finalize
|
||||
where settled_run_finalize.company_id = ${companyId}::uuid
|
||||
and settled_run_finalize.issue_id = ${issueId}
|
||||
and settled_run_finalize.heartbeat_run_id = latest_blocker_operation.heartbeat_run_id
|
||||
and settled_run_finalize.phase = 'workspace_finalize' and settled_run_finalize.status = 'succeeded'
|
||||
and (settled_run_finalize.execution_workspace_id is null
|
||||
or settled_run_finalize.execution_workspace_id = ${workspaceId})
|
||||
)
|
||||
)
|
||||
)`;
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user