fix(issues): release a done blocker whose ended run already finalized (#21948)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A parent task can wait for its subtasks through "blocks" relations.
When the last blocker is done, the parent wakes with
`issue_children_completed` or `issue_blockers_resolved`
> - A done blocker with workspace work also passes a workspace-finalize
barrier. The barrier holds dependents until the blocker's run saves its
workspace
> - The barrier reads the blocker's latest workspace operation. A save
that starts inside the run's finalize and ends after it, or any
operation that the same run records after its finalize, makes the
barrier look unsaved
> - That run has ended, so it records no further finalize. Nothing
clears the barrier, every wake for the parent becomes a skipped
`issue_dependencies_blocked` wait, and the periodic backstop also sees
the blocker as unresolved
> - This pull request treats a settled operation as saved when its run
has ended and that run recorded a successful finalize for the blocker
issue
> - The benefit is that the parent gets its children-completed run, and
parents that are stuck now are released by the existing dependency
backstop with no manual step

## Linked Issues or Issue Description

Refs #19219, #17206, #18665. #19219 releases the barrier when a terminal
run recorded no finalize at all. This change is narrower: the run must
have recorded a successful finalize for the blocker issue. A failed
finalize with no later success still holds the gate (#17206 owns that
recovery).

**What happened?**

A parent task created two subtasks. Subtask 1 finished and its run saved
the workspace. Then the parent run bound both subtasks as blockers and
moved the parent to `blocked` (`durable_dependency_blocker_bound`).
Subtask 2 finished 30 seconds later. The `issue_children_completed` wake
for the parent was skipped and recorded as `issue_dependencies_blocked`
with `unresolvedBlockerIssueIds` set to subtask 1, which had been done
for 2.5 minutes. The parent showed `pendingFinalizeBlockerIssueIds:
[subtask 1]` and `canRetry: false`. No later wake came. The parent
stayed blocked until a person changed it by hand.

**Expected behavior**

When every blocker is done and each blocker's run has finished with a
successful workspace finalize, the parent gets one children-completed
run.

**Steps to reproduce**

1. Create a parent with two subtasks on separate execution workspaces.
2. Finish subtask 1. Its run records a successful `workspace_finalize`,
and a `remote_workspace_snapshot` of the same run ends after the
finalize ends. The incident run had a failed copy-back that was retried
during a restart overlap.
3. Bind both subtasks as blockers of the parent, then block the parent.
4. Finish subtask 2 and dispatch the native `issue_children_completed`
intent.
5. Before this change, the wake is skipped as
`issue_dependencies_blocked` and no run starts.

**Paperclip version or commit**

master `6205b24d54`.

## What Changed

- `pendingFinalizeBlockerCondition` (`server/src/services/issues.ts`)
has one more release rule. The latest blocker operation is settled
(`succeeded`, `failed` or `skipped`). Its heartbeat run is terminal.
That run recorded a successful `workspace_finalize` for the blocker
issue on the same workspace (or with no workspace id). Readiness, the
queue dependency gate, blocker attention and the periodic backstop all
read this condition.
- Live runs keep the old behavior. A save that ends after its finalize,
or an operation that starts after it, still holds dependents while the
run is running.
- Foreign-run operations, unfinished operations and failed finalizes
with no success still hold dependents.

## Verification

- New regression test `heartbeat-dependency-scheduling.test.ts` › "runs
the parent's children-completed wake when a child it bound after
completion saved after its finalize". It replays the incident order:
child 1 done and finalized, its snapshot ends after the finalize, the
parent binds both children and blocks, child 2 finishes, the native
children-completed intent dispatches. Red before the fix (no run starts;
the wake becomes `issue_dependencies_blocked`). Green after the fix (a
run starts with the intent's context, and no dependency wait is
recorded).
- `issues-service.test.ts` nested-snapshot matrix: `late-finish` and
`later-operation` now run with a live owner run (still pending) and with
a terminal owner run (released). `covered`, `foreign-run`, `unfinished`
and `failed-finalize` stay as before.
- `cd server && npx vitest run
src/__tests__/heartbeat-dependency-scheduling.test.ts
src/__tests__/issues-service.test.ts` passes (207 tests).
- `pnpm verify` and `tsc --noEmit` for `server` pass locally.

## Risks

- Low risk. The change only releases a barrier for a run that has
already ended after a successful finalize for the issue. That run cannot
write another finalize, so the old rule held dependents forever with no
path to release them.
- An operation that a terminal run recorded after its finalize is no
longer a reason to hold dependents. Such work can never be saved by that
run. Holding the parent forever does not protect it.
- No schema, API or UI contract change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, test execution) and extended
reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-16 19:59:48 -06:00
1 parent 784bc06276
commit b2b2c1e60b
3 files changed
+118 -6

No files matched your search

@@ -277,6 +277,91 @@ describeEmbeddedPostgres("heartbeat dependency-aware queued run selection", () =
}
}, 45_000);
it("runs the parent's children-completed wake when a child it bound after completion saved after its finalize", async () => {
const companyId = randomUUID();
const agentId = randomUUID();
const parentId = randomUUID();
const firstChildId = randomUUID();
const secondChildId = randomUUID();
const firstWorkspaceId = randomUUID();
const secondWorkspaceId = randomUUID();
const firstRunId = randomUUID();
const secondRunId = randomUUID();
const decisionId = randomUUID();
await db.insert(companies).values({ id: companyId, name: "Child completion",
issuePrefix: `C${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false, defaultResponsibleUserId: "responsible-user" });
await db.insert(companyMemberships).values({ companyId, principalType: "user", principalId: "responsible-user", status: "active", membershipRole: "member" });
await db.insert(agents).values({ id: agentId, companyId, name: "Parent", role: "engineer",
status: "active", adapterType: "codex_local", adapterConfig: {}, permissions: {},
runtimeConfig: { heartbeat: { wakeOnDemand: true, maxConcurrentRuns: 1 } } });
await db.insert(executionWorkspaces).values([firstWorkspaceId, secondWorkspaceId].map((id) => ({
id, companyId, mode: "shared_workspace", strategyType: "project_primary", name: "Child workspace", status: "active",
})));
await db.insert(issues).values([
{ id: parentId, companyId, title: "Parent", status: "in_progress",
assigneeAgentId: agentId, responsibleUserId: "responsible-user" },
{ id: firstChildId, companyId, parentId, title: "First child", status: "done",
assigneeAgentId: agentId, executionWorkspaceId: firstWorkspaceId, responsibleUserId: "responsible-user",
completedAt: new Date("2026-11-16T23:33:00.000Z") },
{ id: secondChildId, companyId, parentId, title: "Second child", status: "in_progress",
assigneeAgentId: agentId, executionWorkspaceId: secondWorkspaceId, responsibleUserId: "responsible-user" },
]);
await db.insert(heartbeatRuns).values([
{ id: firstRunId, companyId, agentId, issueId: firstChildId, status: "succeeded",
startedAt: new Date("2026-11-16T23:28:28.000Z"), finishedAt: new Date("2026-11-16T23:32:59.430Z") },
{ id: secondRunId, companyId, agentId, issueId: secondChildId, status: "running",
startedAt: new Date("2026-11-16T23:31:17.000Z") },
]);
await db.insert(workspaceOperations).values([
{ companyId, executionWorkspaceId: firstWorkspaceId, issueId: firstChildId, heartbeatRunId: firstRunId,
phase: "workspace_finalize", status: "succeeded",
startedAt: new Date("2026-11-16T23:32:57.000Z"), finishedAt: new Date("2026-11-16T23:32:59.000Z") },
{ companyId, executionWorkspaceId: firstWorkspaceId, issueId: firstChildId, heartbeatRunId: firstRunId,
phase: "remote_workspace_snapshot", status: "succeeded",
startedAt: new Date("2026-11-16T23:32:58.000Z"), finishedAt: new Date("2026-11-16T23:32:59.200Z") },
]);
await db.insert(issueRelations).values([firstChildId, secondChildId].map((blockerId) => ({
companyId, issueId: blockerId, relatedIssueId: parentId, type: "blocks" as const, createdByAgentId: agentId,
})));
await db.update(issues).set({ status: "blocked", blockedTransitionAt: new Date("2026-11-16T23:34:52.374Z") })
.where(eq(issues.id, parentId));
await db.insert(workspaceOperations).values({ companyId, executionWorkspaceId: secondWorkspaceId,
issueId: secondChildId, heartbeatRunId: secondRunId, phase: "workspace_finalize", status: "succeeded",
startedAt: new Date("2026-11-16T23:35:20.000Z"), finishedAt: new Date("2026-11-16T23:35:22.000Z") });
await db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date("2026-11-16T23:35:23.250Z") })
.where(eq(heartbeatRuns.id, secondRunId));
await db.update(issues).set({ status: "done", completedAt: new Date("2026-11-16T23:35:23.218Z") })
.where(eq(issues.id, secondChildId));
const [intent] = await db.insert(agentWakeupRequests).values({ companyId, agentId,
source: "automation", triggerDetail: "system", reason: "issue_children_completed",
requestedByActorType: "system", requestedByActorId: "native-status-committer",
idempotencyKey: `issue_children_completed:${parentId}:${secondChildId}:${decisionId}`,
payload: { issueId: parentId, taskId: parentId, _paperclipWakeContext: {
issueId: parentId, taskId: parentId, source: "native_status_decision",
nativeChildCompletionDecisionId: decisionId, completedChildIssueId: secondChildId,
childIssueIds: [firstChildId, secondChildId],
} },
}).returning();
await heartbeat.dispatchPendingNativeStatusWakeups({ companyId });
expect(await waitForCondition(async () => mockAdapterExecute.mock.calls.length === 1, 10_000)).toBe(true);
await heartbeat.drainActiveRunExecutions();
const dependencyWaits = await db.select().from(agentWakeupRequests).where(and(
eq(agentWakeupRequests.companyId, companyId), eq(agentWakeupRequests.reason, "issue_dependencies_blocked"),
));
expect(dependencyWaits).toEqual([]);
const parentRuns = await db.select().from(heartbeatRuns).where(and(
eq(heartbeatRuns.companyId, companyId), sql`${heartbeatRuns.contextSnapshot}->>'issueId' = ${parentId}`,
));
const childrenCompletedRun = parentRuns.find((run) => run.contextSnapshot?.nativeStatusWakeIntentId === intent!.id);
expect(childrenCompletedRun?.contextSnapshot).toMatchObject({
wakeReason: "issue_children_completed", nativeStatusWakeIntentId: intent!.id,
completedChildIssueId: secondChildId,
});
}, 45_000);
it("keeps a saved native continuation behind its active predecessor without retagging that run", async () => {
const companyId = randomUUID(), agentId = randomUUID(), issueId = randomUUID();
let finishPredecessor!: () => void;
+15 -6
View File
@@ -4773,13 +4773,23 @@ const dependencyTestDatabaseUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_ISSU
});
});
it.each(["covered", "foreign-run", "unfinished", "failed-finalize", "late-finish", "later-operation"])(
"requires a successful owning finalize to cover a nested snapshot: %s", async kind => {
it.each([
["covered", "succeeded", true],
["foreign-run", "succeeded", false],
["unfinished", "succeeded", false],
["failed-finalize", "succeeded", false],
["late-finish", "running", false],
["later-operation", "running", false],
["late-finish", "succeeded", true],
["later-operation", "succeeded", true],
] as const)(
"requires a successful owning finalize to cover a nested snapshot: %s (run %s)", async (kind, runStatus, ready) => {
const f = await seedSharedWorkspaceDependency();
const runId = randomUUID(), foreignRunId = randomUUID();
await db.insert(heartbeatRuns).values([runId, foreignRunId].map(id => ({
id, companyId: f.companyId, agentId: f.assigneeAgentId, status: "succeeded",
})));
await db.insert(heartbeatRuns).values([
{ id: runId, companyId: f.companyId, agentId: f.assigneeAgentId, status: runStatus },
{ id: foreignRunId, companyId: f.companyId, agentId: f.assigneeAgentId, status: "succeeded" },
]);
const started = new Date("2026-05-23T22:00:00.000Z");
const finished = new Date("2026-05-23T22:00:02.000Z");
await db.insert(workspaceOperations).values([
@@ -4797,7 +4807,6 @@ const dependencyTestDatabaseUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_ISSU
startedAt: new Date(finished.getTime() + 100), finishedAt: new Date(finished.getTime() + 200),
}] : []),
]);
const ready = kind === "covered";
expect((await svc.listWakeableBlockedDependents(f.blockerId)).map(issue => issue.id))
.toEqual(ready ? [f.dependentId] : []);
await expect(svc.getDependencyReadiness(f.dependentId)).resolves.toMatchObject({
+18
View File
@@ -2625,6 +2625,24 @@ function pendingFinalizeBlockerCondition(companyId: string, issueId: SQL, worksp
and recovery_finalize.started_at >= coalesce(latest_blocker_operation.finished_at, latest_blocker_operation.started_at)
)
)
and not (
latest_blocker_operation.status in ('succeeded', 'failed', 'skipped')
and exists (
select 1 from heartbeat_runs settled_owner_run
where settled_owner_run.company_id = ${companyId}::uuid
and settled_owner_run.id = latest_blocker_operation.heartbeat_run_id
and settled_owner_run.status in (${sql.join([...TERMINAL_HEARTBEAT_RUN_STATUSES].map(status => sql`${status}`), sql`, `)})
)
and exists (
select 1 from workspace_operations settled_run_finalize
where settled_run_finalize.company_id = ${companyId}::uuid
and settled_run_finalize.issue_id = ${issueId}
and settled_run_finalize.heartbeat_run_id = latest_blocker_operation.heartbeat_run_id
and settled_run_finalize.phase = 'workspace_finalize' and settled_run_finalize.status = 'succeeded'
and (settled_run_finalize.execution_workspace_id is null
or settled_run_finalize.execution_workspace_id = ${workspaceId})
)
)
)`;
}