perf(recovery): keep a finished task's settled attention settled and halve graph hint fan-out (#21676)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Issue recovery records "recovery actions" on tasks. A sweep resolves
the active action of a task that is already done or cancelled.
> - On an idle hosted instance, a debug-log run (from #21640) named a
ping-pong. A native worker re-surfaces a board attention for 13 finished
tasks. Recovery resolves it as `source_terminal`. The unique "one active
action" index then lets the worker insert it again, about every 26
seconds per task. That built 45,008 rows since 2026-10-21. Each insert
and each resolve also signals run dispatch and the task's recovery
graph, which wakes both sides again.
> - Separately, every task-graph recovery hint is published twice per
task queue: once as the graph hint and once as a `{companyId, backstop:
true}` twin. That is half of all idle signals, for a backstop that is
time-gated anyway.
> - This pull request breaks the ping-pong from the recovery side and
removes the backstop twin.
> - The benefit is that a finished task's settled attention stays
settled and wakes nothing. Each graph hint costs half the signals.

## Linked Issues or Issue Description

No issue exists. This describes the problem with the fields of
`.github/ISSUE_TEMPLATE/bug_report.yml`.

**What happened?**

With zero runs active, recovery and native workers kept re-creating and
re-resolving the same board attention on 13 done or cancelled tasks. The
rows are `active_run_watchdog`, owner `board`, cause
`native_restart_recovery_blocked`. The loop added about one row per task
every ~26 s and about 2.6 work-signal commits per second.

**Expected behavior**

Once recovery settles a finished task's attention, the same attention
identity is not surfaced again, and settling it wakes nothing that a
finished task cannot use.

**Steps to reproduce**

1. Create a cancelled task.
2. Surface the native "controller process space unverified" attention
for it (same cause and fingerprint each time), then run
`reconcileActiveRecoveryActions`.
3. Repeat step 2. On master each cycle inserts a new active row and
resolves it, and each resolve signals the task queues and run dispatch.

**Paperclip version or commit**

master at 36101ebcb9.

## What Changed

- `server/src/services/issue-recovery-actions.ts`:
- **No-match skip.** `resolveActiveForIssue` first reads whether any
action matches. When none does, it returns `null` with no transaction
and no signal. Before, `writeExecutionControl` always published
run-dispatch, graph-recovery and execution hints first.
- **Settled-attention guard.** `upsertSourceScoped` returns the existing
resolved action instead of inserting when ALL of these hold:
    - the task is done or cancelled;
    - it has no active action;
- an action with the same cause and fingerprint was already resolved
with `source_terminal`.
The guard writes no row and sends no signal, and the caller gets that
resolved action. It covers only the exact identity that recovery itself
settled because the task finished. A new fingerprint (renewed controller
authority), a different cause, or a reopened task still creates an
action.
- **Quiet terminal resolve.** A new `sourceTerminal` resolve mode. It
updates the row only while the task is still done or cancelled. It still
notifies native recovery notices, unsafe-workspace export recovery, and
a preserved native finalization waiter. It does NOT signal run dispatch,
the task's recovery graph, or the execution disposition/continuation
queues: a finished task has no run to dispatch and no assignment to
recover.
- `server/src/services/recovery/service.ts`: the `source_terminal`
branch of `reconcileActiveRecoveryActions` uses `sourceTerminal: true`.
- `server/src/services/issue-recovery-notifications.ts`:
`notifyIssueGraphRecovery` publishes one graph hint per task queue and
no longer adds a `{companyId, backstop: true}` hint per queue. A new
`issueRecoveryBackstopHint(companyId)` builds that hint string.
- `server/src/services/recovery/work.ts`: when a task queue loads a
graph scope, it arms its own backstop in process
(`worker.invalidate(issueRecoveryBackstopHint(companyId))`). The #21552
guarantee stays: any graph activity arms the company backstop, which
runs at most once per backstop interval after the last full pass. It no
longer costs one signal per queue per hint.
- `server/src/__tests__/issue-recovery-actions.test.ts`: the fake-DB
unit tests accept the guard's extra read.
- `server/src/__tests__/issue-recovery-signal-fanout.test.ts` (new, real
PostgreSQL), six tests:
  - one graph hint per task queue and no backstop hint;
  - a resolve with no matching action returns null and signals nothing;
- an escalated action on a finished task is still resolved and still
signals;
- the ping-pong cycle: surface then reconcile, five times, leaves one
row and no stranded or active-recovery signals;
- settling a finished task's attention sends no task-queue, run-dispatch
or active-recovery signals;
- a renewed authority and a reopened task still get a new attention,
which stays active on a reopened task.

## Verification

- New test file:
  - master: 4 of 6 fail (red).
    - 8 task-queue hints per graph hint instead of 4.
    - A no-match resolve publishes 18 settled signals.
- Cycle 2 inserts a new active row; master builds a new row every cycle.
- Settling a finished task's attention signals the task queues and run
dispatch.
  - This branch: 6 of 6 pass (green).
- Regression on real PostgreSQL:
- heartbeat-native-restart-suspension, native-recovery-notice-work,
issue-recovery-deadlines (including the #21552 backstop tests),
assigned-work-invariant, execution-control-work, issue-recovery-actions,
issue-recovery-work-lifecycle, issue-recovery-assignments,
execution-recovery-resolution, automatic-run-brake-recovery-passes: all
pass (381 tests).
- Four unit tests in issue-recovery-actions use a hand-written fake DB.
They needed `innerJoin` on the fake query and an expected count of two
selects, because the guard adds one read.
  - recovery-deadline-acceptance: passes.
- `tsc --noEmit -p server/tsconfig.json` passes.

## Risks

- The guard returns a resolved action to a caller that asked to surface
one. I checked the callers: the one that loops
(`surfaceNativeControllerNamespaceAttention`) ignores the return value.
The others only reach the guard for a finished task whose identical
attention recovery already settled, where re-surfacing it would be
resolved again within one sweep.
- The terminal resolve no longer wakes run dispatch or the task's graph.
Completing or cancelling a task already sends those signals itself.
- This does not stop the native worker from trying to surface attention
for finished tasks. That fix belongs in the native owner-release code,
which another change owns.
- The ~45k historical resolved rows on the hosted instance stay as they
are. A cleanup would need a migration or an operator script that deletes
resolved `native_restart_recovery_blocked` rows whose source is done or
cancelled, keeping the newest per (task, fingerprint) so the guard still
matches. That is not part of this change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, local PostgreSQL test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Related: #21552, #21621, #21640.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-14 18:06:38 -06:00
1 parent 97a167093d
commit dcba9628c7
6 files changed
+195 -16

No files matched your search

@@ -95,7 +95,7 @@ describe("issueRecoveryActionService", () => {
const error: Error & { cause?: unknown } = new Error("Wrapped database failure");
error.cause = kind === "unrelated unique constraint" ? { code: "23505", constraint_name: "other_unique_constraint" }
: kind === "other SQLSTATE" ? { code: "23503", constraint_name: constraint } : error;
const query = { from() { return this; }, where() { return this; }, orderBy() { return this; },
const query = { from() { return this; }, innerJoin() { return this; }, where() { return this; }, orderBy() { return this; },
limit() { return Promise.resolve([]); } };
const fakeDb = {
async transaction<T>(fn: (tx: unknown) => Promise<T>): Promise<T> { return fn(fakeDb); },
@@ -106,7 +106,7 @@ describe("issueRecoveryActionService", () => {
await expect(issueRecoveryActionService(fakeDb as never).upsertSourceScoped({ companyId: "company", sourceIssueId: "issue",
kind: "missing_disposition", cause: "saved-debt", fingerprint: "same-source", nextAction: "Restore existing work." }))
.rejects.toBe(error);
expect(fakeDb.select).toHaveBeenCalledOnce();
expect(fakeDb.select).toHaveBeenCalledTimes(2);
expect(fakeDb.insert).toHaveBeenCalledOnce();
});
it("does not reactivate an action resolved between the active read and update", async () => {
@@ -118,6 +118,9 @@ describe("issueRecoveryActionService", () => {
from() {
return this;
},
innerJoin() {
return this;
},
where() {
return this;
},
@@ -0,0 +1,133 @@
import { and, eq } from "drizzle-orm";
import { expect, it } from "vitest";
import { issueRecoveryActions, issues } from "@paperclipai/db";
import { subscribeDatabaseWork } from "@paperclipai/db/work-signals";
import { DELIVERY_QUEUES } from "../services/delivery-work-notifications.js";
import { issueRecoveryActionService } from "../services/issue-recovery-actions.js";
import { notifyIssueGraphRecovery } from "../services/issue-recovery-notifications.js";
import { recoveryService } from "../services/recovery/service.js";
import { describeIssueRecovery, issueRecoveryFixture } from "./issue-recovery.test-fixture.js";
const TASK_QUEUES = [DELIVERY_QUEUES.strandedAssignments, DELIVERY_QUEUES.assignedWorkInvariant,
DELIVERY_QUEUES.resolvedDependencies, DELIVERY_QUEUES.staleIssueLocks];
const WATCHED = [...TASK_QUEUES, DELIVERY_QUEUES.orphanBlockers, DELIVERY_QUEUES.activeRecoveryActions,
DELIVERY_QUEUES.taskWatchdogs, DELIVERY_QUEUES.queuedRunDispatch, DELIVERY_QUEUES.executionDisposition];
describeIssueRecovery("issue recovery signal fan-out", () => {
const state = issueRecoveryFixture();
function record(owner: object) {
const settled: Array<[string, string[]]> = [];
const stops = WATCHED.map(queue => subscribeDatabaseWork(owner, queue, (event, hints) => {
if (event === "settled") settled.push([queue, [...hints ?? []]]);
}));
return { settled, stop: () => { for (const stop of stops) stop(); } };
}
it("publishes one graph hint per task queue and no per-queue backstop twin", async () => {
const f = await state.fixture();
const [task] = await state.db.insert(issues).values({ companyId: f.companyId, title: "Task", status: "todo" }).returning();
const recorder = record(state.db);
try {
await state.db.transaction(tx => notifyIssueGraphRecovery(tx, { companyId: f.companyId, issueId: task.id }));
} finally { recorder.stop(); }
const taskHints = recorder.settled.filter(([queue]) => TASK_QUEUES.includes(queue as never)).flatMap(([, hints]) => hints);
expect(taskHints).toHaveLength(TASK_QUEUES.length);
expect(taskHints.every(hint => JSON.parse(hint).graphIssueId === task.id)).toBe(true);
expect(recorder.settled.flatMap(([, hints]) => hints).some(hint => hint.includes("\"backstop\""))).toBe(false);
});
it("resolves nothing and signals nothing when the issue has no matching active recovery action", async () => {
const f = await state.fixture();
const [task] = await state.db.insert(issues).values({ companyId: f.companyId, title: "Done", status: "done" }).returning();
const [resolved] = await state.db.insert(issueRecoveryActions).values({ companyId: f.companyId, sourceIssueId: task.id,
kind: "stranded_assigned_issue", status: "resolved", ownerType: "agent", ownerAgentId: f.agentId,
cause: "stranded_assigned_issue", fingerprint: "already-resolved", nextAction: "Done", outcome: "restored",
resolvedAt: new Date(), evidence: {} }).returning();
const recorder = record(state.db);
const actions = issueRecoveryActionService(state.db);
try {
expect(await actions.resolveActiveForIssue({ companyId: f.companyId, sourceIssueId: task.id, actionId: resolved.id,
status: "resolved", outcome: "restored", resolutionNote: "source_terminal" })).toBeNull();
expect(await actions.resolveActiveForIssue({ companyId: f.companyId, sourceIssueId: task.id,
status: "cancelled", outcome: "cancelled", resolutionNote: "source_state_changed" })).toBeNull();
} finally { recorder.stop(); }
expect(recorder.settled).toEqual([]);
});
it("still resolves an active action and wakes its task graph", async () => {
const f = await state.fixture();
const [task] = await state.db.insert(issues).values({ companyId: f.companyId, title: "Done", status: "done" }).returning();
const [active] = await state.db.insert(issueRecoveryActions).values({ companyId: f.companyId, sourceIssueId: task.id,
kind: "stranded_assigned_issue", status: "escalated", ownerType: "agent", ownerAgentId: f.agentId,
cause: "stranded_assigned_issue", fingerprint: "escalated", nextAction: "Restore", evidence: {} }).returning();
const recorder = record(state.db);
let result;
try {
result = await issueRecoveryActionService(state.db).resolveActiveForIssue({ companyId: f.companyId, sourceIssueId: task.id,
actionId: active.id, status: "resolved", outcome: "restored", resolutionNote: "source_terminal" });
} finally { recorder.stop(); }
expect(result).toMatchObject({ id: active.id, status: "resolved", outcome: "restored" });
expect(recorder.settled.filter(([queue]) => queue === DELIVERY_QUEUES.strandedAssignments).flatMap(([, hints]) => hints)
.map(hint => JSON.parse(hint))).toContainEqual({ companyId: f.companyId, graphIssueId: task.id });
});
async function blockedAttention(companyId: string, sourceIssueId: string, agentId: string, fingerprint = "native-controller-space:run:digest") {
return issueRecoveryActionService(state.db).upsertSourceScoped({ companyId, sourceIssueId, kind: "active_run_watchdog",
ownerType: "board", returnOwnerAgentId: agentId, cause: "native_restart_recovery_blocked", fingerprint,
evidence: { runId: "run", reason: "controller_process_space_unverified", executorStopEvidence: "unverified" },
nextAction: "Wait for the previous controller to stop" });
}
it("does not re-open attention that recovery settled because its task finished", async () => {
const f = await state.fixture();
const [task] = await state.db.insert(issues).values({ companyId: f.companyId, title: "Cancelled", status: "cancelled" }).returning();
const recovery = recoveryService(state.db, { enqueueWakeup: async () => null });
await blockedAttention(f.companyId, task.id, f.agentId);
await recovery.reconcileActiveRecoveryActions({ companyId: f.companyId });
const recorder = record(state.db);
try {
for (let cycle = 0; cycle < 5; cycle++) {
const surfaced = await blockedAttention(f.companyId, task.id, f.agentId);
expect(surfaced.status).toBe("resolved");
await recovery.reconcileActiveRecoveryActions({ companyId: f.companyId });
}
} finally { recorder.stop(); }
const rows = await state.db.select().from(issueRecoveryActions).where(and(
eq(issueRecoveryActions.companyId, f.companyId), eq(issueRecoveryActions.sourceIssueId, task.id)));
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({ status: "resolved", outcome: "restored", resolutionNote: "source_terminal" });
expect(recorder.settled.filter(([queue]) => queue === DELIVERY_QUEUES.strandedAssignments
|| queue === DELIVERY_QUEUES.activeRecoveryActions)).toEqual([]);
});
it("settles a finished task's attention without waking run dispatch or its task graph", async () => {
const f = await state.fixture();
const [task] = await state.db.insert(issues).values({ companyId: f.companyId, title: "Done", status: "done" }).returning();
await blockedAttention(f.companyId, task.id, f.agentId);
const recorder = record(state.db);
try {
expect(await recoveryService(state.db, { enqueueWakeup: async () => null }).reconcileActiveRecoveryActions({ companyId: f.companyId }))
.toMatchObject({ resolved: 1 });
} finally { recorder.stop(); }
expect(recorder.settled.filter(([queue]) => queue !== DELIVERY_QUEUES.executionDisposition)
.filter(([queue]) => [...TASK_QUEUES, DELIVERY_QUEUES.queuedRunDispatch, DELIVERY_QUEUES.activeRecoveryActions].includes(queue as never)))
.toEqual([]);
});
it("surfaces attention again once the task is reopened, and for a new controller authority", async () => {
const f = await state.fixture();
const [task] = await state.db.insert(issues).values({ companyId: f.companyId, title: "Cancelled", status: "cancelled" }).returning();
const recovery = recoveryService(state.db, { enqueueWakeup: async () => null });
await blockedAttention(f.companyId, task.id, f.agentId);
await recovery.reconcileActiveRecoveryActions({ companyId: f.companyId });
expect((await blockedAttention(f.companyId, task.id, f.agentId, "native-controller-space:run:renewed")).status).toBe("active");
await recovery.reconcileActiveRecoveryActions({ companyId: f.companyId });
await state.db.update(issues).set({ status: "todo" }).where(eq(issues.id, task.id));
const reopened = await blockedAttention(f.companyId, task.id, f.agentId);
expect(reopened.status).toBe("active");
await recovery.reconcileActiveRecoveryActions({ companyId: f.companyId });
expect((await state.db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, reopened.id)))[0].status)
.toBe("active");
});
});
+47 -7
View File
@@ -6,7 +6,7 @@ import { writeExecutionControl } from "./execution-control-notifications.js";
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
import { and, desc, eq, inArray, isNull, isNotNull, or, sql } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import { heartbeatRuns, issueRecoveryActions, nativeRunFinalizations, workspaceOperations } from "@paperclipai/db";
import { heartbeatRuns, issueRecoveryActions, issues, nativeRunFinalizations, workspaceOperations } from "@paperclipai/db";
import { isUuidLike } from "@paperclipai/shared";
import type {
IssueRecoveryAction,
@@ -86,8 +86,11 @@ export type ResolveIssueRecoveryActionInput = {
status: Extract<IssueRecoveryActionStatus, "resolved" | "cancelled">;
outcome: IssueRecoveryActionOutcome;
resolutionNote?: string | null;
sourceTerminal?: boolean;
};
export const SOURCE_TERMINAL_RESOLUTION = "source_terminal";
/** A spent saved-work recovery is described from its current state. Text saved
* by an older release could ask a person to inspect the run's environment. */
function readModelNextAction(row: IssueRecoveryActionRow): string {
@@ -390,6 +393,21 @@ export function issueRecoveryActionService(db: DbOrTransaction) {
const existing = await getActiveForIssue(input.companyId, input.sourceIssueId);
const now = new Date();
if (existing && input.createOnly) return existing;
if (!existing) {
const [source] = await db.select({ status: issues.status }).from(issues)
.where(and(eq(issues.companyId, input.companyId), eq(issues.id, input.sourceIssueId))).limit(1);
if (source && (source.status === "done" || source.status === "cancelled")) {
const [settled] = await db.select().from(issueRecoveryActions).where(and(
eq(issueRecoveryActions.companyId, input.companyId),
eq(issueRecoveryActions.sourceIssueId, input.sourceIssueId),
eq(issueRecoveryActions.status, "resolved"),
eq(issueRecoveryActions.cause, input.cause),
eq(issueRecoveryActions.fingerprint, input.fingerprint),
eq(issueRecoveryActions.resolutionNote, SOURCE_TERMINAL_RESOLUTION),
)).limit(1);
if (settled) return toReadModel(settled);
}
}
const ownerType = input.ownerType ?? (input.ownerAgentId ? "agent" : "board");
if (existing) {
// A distinct failure identity must not overwrite the active action of a
@@ -619,12 +637,11 @@ export function issueRecoveryActionService(db: DbOrTransaction) {
predicates.push(eq(issueRecoveryActions.fingerprint, input.fingerprint));
}
const [updated] = await writeExecutionControl(dbOrTx, [DELIVERY_QUEUES.executionDisposition, DELIVERY_QUEUES.executionContinuation], async tx => {
const pending = await tx.select({ id: issueRecoveryActions.id }).from(issueRecoveryActions)
.where(and(...predicates)).limit(1);
if (!pending.length) return [];
// Resolving another attention releases this issue's preserved Native
// waiter to one normal inspection. No recurring monitor or new turn.
const [candidate] = await dbOrTx.select({ id: issueRecoveryActions.id }).from(issueRecoveryActions)
.where(and(...predicates)).limit(1);
if (!candidate) return null;
const releaseWaiter = async (tx: DbOrTransaction) => {
const [waiter] = await tx.select({ id: nativeRunFinalizations.runId }).from(nativeRunFinalizations)
.innerJoin(heartbeatRuns, eq(heartbeatRuns.id, nativeRunFinalizations.runId)).where(and(
eq(nativeRunFinalizations.companyId, input.companyId), eq(nativeRunFinalizations.issueId, input.sourceIssueId),
@@ -632,6 +649,29 @@ export function issueRecoveryActionService(db: DbOrTransaction) {
isNotNull(nativeRunFinalizations.leaseOwner), inArray(heartbeatRuns.status, ["failed", "timed_out", "interrupted"]),
)).limit(1);
if (waiter) await notifyDeliveryWork(tx, DELIVERY_QUEUES.executionFinalization);
};
if (input.sourceTerminal) {
const [settled] = await dbOrTx.transaction(async tx => {
await releaseWaiter(tx as unknown as DbOrTransaction);
await notifyNativeRecoveryNotices(tx, { companyId: input.companyId, issueId: input.sourceIssueId });
await notifyUnsafeWorkspaceExportRecovery(tx, { companyId: input.companyId, issueId: input.sourceIssueId });
return tx.update(issueRecoveryActions)
.set({ status: input.status, outcome: input.outcome, resolutionNote: input.resolutionNote ?? null, resolvedAt: now, updatedAt: now })
.where(and(...predicates, sql`exists (select 1 from ${issues} where ${issues.id} = ${issueRecoveryActions.sourceIssueId}
and ${issues.companyId} = ${issueRecoveryActions.companyId} and ${issues.status} in ('done', 'cancelled'))`))
.returning();
});
return settled ? toReadModel(settled) : null;
}
const [updated] = await writeExecutionControl(dbOrTx, [DELIVERY_QUEUES.executionDisposition, DELIVERY_QUEUES.executionContinuation], async tx => {
const pending = await tx.select({ id: issueRecoveryActions.id }).from(issueRecoveryActions)
.where(and(...predicates)).limit(1);
if (!pending.length) return [];
// Resolving another attention releases this issue's preserved Native
// waiter to one normal inspection. No recurring monitor or new turn.
await releaseWaiter(tx as unknown as DbOrTransaction);
await notifyIssueGraphRecovery(tx, { companyId: input.companyId, issueId: input.sourceIssueId });
await notifyNativeRecoveryNotices(tx, { companyId: input.companyId, issueId: input.sourceIssueId });
await notifyUnsafeWorkspaceExportRecovery(tx, { companyId: input.companyId, issueId: input.sourceIssueId });
@@ -26,6 +26,10 @@ export async function notifyIssueRecovery(transaction: object, scope?: IssueReco
}
}
export function issueRecoveryBackstopHint(companyId: string) {
return JSON.stringify({ companyId, backstop: true });
}
export function decodeIssueRecoveryHint(hint: string): IssueRecoveryScope | undefined {
return hint === "*" ? undefined : JSON.parse(hint) as IssueRecoveryScope;
}
@@ -44,10 +48,7 @@ export async function notifyIssueGraphRecovery(transaction: object,
const companyHint = JSON.stringify({ companyId: scope.companyId });
for (const queue of COMPANY_QUEUES) await notifyDeliveryWork(transaction, queue, companyHint);
const graphHint = JSON.stringify({ companyId: scope.companyId, graphRunId: scope.runId ?? undefined, graphIssueId: scope.issueId ?? undefined });
for (const queue of TASK_QUEUES) {
await notifyDeliveryWork(transaction, queue, graphHint);
await notifyDeliveryWork(transaction, queue, JSON.stringify({ companyId: scope.companyId, backstop: true }));
}
for (const queue of TASK_QUEUES) await notifyDeliveryWork(transaction, queue, graphHint);
}
export async function resolveIssueGraph(db: Db, scope: { companyId: string; runId?: string; issueId?: string }): Promise<string[] | null> {
+3 -2
View File
@@ -127,7 +127,7 @@ import { emitAgentTaskRun } from "../agent-task-run-telemetry.js";
import { budgetService } from "../budgets.js";
import { isBoardOwnedBudgetBlock } from "../budget-block-ownership.js";
import { unadmittedChatWakeupCondition } from "../durable-chat-wakeup.js";
import { issueRecoveryActionService } from "../issue-recovery-actions.js";
import { issueRecoveryActionService, SOURCE_TERMINAL_RESOLUTION } from "../issue-recovery-actions.js";
import { PROVIDER_LIMIT_DEFAULT_COOLDOWN_MS } from "../system-recovery-state.js";
import { parseProviderQuotaClockReset } from "../provider-quota-reset.js";
import { hasUnverifiedNativeWorkspaceFinalizationOwner } from "../native-runtime/native-workspace-finalization-recovery-state.js";
@@ -3670,7 +3670,8 @@ export function recoveryService(
actionId: action.id,
status: "resolved",
outcome: "restored",
resolutionNote: "source_terminal",
resolutionNote: SOURCE_TERMINAL_RESOLUTION,
sourceTerminal: true,
});
if (resolved) {
result.resolved += 1;
+2 -1
View File
@@ -3,7 +3,7 @@ import { and, asc, eq, gt, inArray, isNotNull, isNull, or, sql, type SQL } from
import { issueRecoveryActions, issueRelations, issues, type Db } from "@paperclipai/db";
import type { createDeliveryWorkCoordinator } from "../delivery-work-coordinator.js";
import { DELIVERY_QUEUES, type DeliveryQueue } from "../delivery-work-notifications.js";
import { decodeIssueRecoveryHint, ISSUE_RECOVERY_BACKSTOP_MS, issueRecoveryScopeMatches, resolveIssueGraph, type IssueRecoveryScope }
import { decodeIssueRecoveryHint, ISSUE_RECOVERY_BACKSTOP_MS, issueRecoveryBackstopHint, issueRecoveryScopeMatches, resolveIssueGraph, type IssueRecoveryScope }
from "../issue-recovery-notifications.js";
import { registerScopedDeadlineWork } from "../scoped-deadline-work.js";
import { workSignalDebug } from "@paperclipai/db/work-signal-debug";
@@ -71,6 +71,7 @@ export function registerIssueRecoveryWork(
scope.wholeCompany = ids === null;
}
async function toCompany(scope: IssueRecoveryScope | undefined, worker: { invalidate(hint?: string): void }) {
if (scope && isGraph(scope)) worker.invalidate(issueRecoveryBackstopHint(scope.companyId));
await resolveGraph(scope);
if (!scope?.wholeCompany) return false;
worker.invalidate(JSON.stringify({ companyId: scope.companyId }));