Files
PaperClipAI/server/src/modules/wake-queue/adapters/queued-comment-postgres.ts
T
DottaandPaperclip cc67d4e1d8 fix: preserve steering and recover stopped task conversations (#15015)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - A task conversation must let a user guide a running agent and resume
stopped work.
> - The active run owns its input protocol, even when the user changes
the next model or effort.
> - Queue delivery waits for a provider receipt, which must be able to
persist during the request.
> - A stopped startup also needs a clear user action that passes normal
task admission.
> - This pull request fixes steering delivery, makes queue actions
immediate, and restores explicit continuation.
> - The benefit is a responsive conversation that can recover without
losing saved input.

## Linked Issues or Issue Description

**What happened?**

A queued message could change from Steer to Interrupt while a native run
prepared. A steer request could wait on its own database lock and fail
to deliver. A stopped startup could then leave the conversation without
a working Retry or message continuation. Interrupt also waited for the
server and showed a toast.

**Expected behavior**

The active run keeps its input protocol. Steer delivers input to that
run. Steer and Interrupt clear the submitted queue rows and show the
input in the conversation immediately. Failed delivery restores the
latest queue with an inline error. An eligible stopped run offers Retry,
and authenticated user input can start a fresh turn through normal task
admission.

**Steps to reproduce**

1. Start a task with a native Paperclip Runner.
2. Change the selected model or effort while that run prepares.
3. Queue a message and press Steer.
4. Observe the provider receipt and queue state during the request.
5. Stop a startup before its provider process begins, then try Retry or
send a new message.
6. Repeat queued delivery with a legacy runner and press Interrupt.

**Paperclip version or commit**

Reproduced on the parent of this branch, `59c07ede7`.

**Deployment mode**

Authenticated private deployment. The fixes also cover local task
conversations.

Related work: Refs #12834, Refs #13354, Refs #13275. The open refactor
in #13160 moves the same queue route; it does not fix the receipt lock
or stopped-run continuation addressed here.

## What Changed

- Select queue behavior from the active run's immutable dispatch and
runtime resolution.
- Leave the run row unlocked during provider acknowledgement, then lock
and read it before merging the receipt.
- Retain queued input if the target run stops during that wait. Keep
inline delivery errors visible after empty queue updates.
- Permit exact Retry and authenticated continuation after verified
native startup cancellation. Preserve pause, approval, budget,
ownership, and process-stop gates.
- Carry undelivered native queue input into a fresh turn once the old
execution is confirmed stopped.
- Show Steer and Interrupt input in the conversation and clear submitted
composer rows immediately. Restore the latest queue inline on failure.
Remove delivery toasts.
- Keep optimistic delivery stable across stale polls, empty queues, and
paginated history. Preserve classic Interrupt error handling.
- Document recovery and optimistic delivery behavior. Add regression
tests across server, shared queue projection, and UI boundaries.

## Verification

- Red-green regression tests reproduced the queue protocol, receipt
lock, stopped-startup continuation, and optimistic delivery failures.
- The focused server route, continuation, queue, and runner boundary
suites passed during implementation.
- The queue-route suite passes with 78 tests. The three complete
conversation UI suites pass with 347 tests.
- UI typecheck, production build, and `pnpm check:token-gates` pass.
- Workspace `pnpm -r typecheck` and `pnpm build` pass. The local
monolithic `pnpm test:run` is still running; remote CI verifies the
complete suite on the latest commit.
- All CI gates pass on `bd9031ad56abfcde13d13a13488c1b9217c2fd3a`,
including the full test shards, runner verification, browser E2E,
typecheck, release registry, and canary dry run.
- Greptile reports 5/5 for that commit. Both review threads are
resolved.

## Risks

This changes queue display and explicit continuation admission. The UI
must restore rejected delivery without losing other-session edits. The
server must preserve concurrent provider result updates and must not
resume a process whose stop is uncertain. Focused tests cover these
boundaries. This change has no database migration.

## Model Used

OpenAI Codex, an agent based on GPT-6. The exact runtime model ID and
context window are not exposed in this session. Used reasoning,
repository tools, code execution, and browser inspection.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-02 22:07:14 -05:00

314 lines
13 KiB
TypeScript

import { and, eq, inArray } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import { agentWakeupRequests, agents, heartbeatRuns, issueComments, issues } from "@paperclipai/db";
import type { IssueComment, IssueQueuedCommentQueue } from "@paperclipai/shared";
import {
buildQueuedCommentQueueSnapshot,
decideQueuedCommentQueueSteering,
queuedCommentIdsFromWakePayload,
withQueuedCommentIdsInRunContext,
withQueuedCommentIdsInWakePayload,
} from "../../../services/issue-queued-comment-queue.js";
import { logActivity as persistActivityLogRow, type ActivityPublication } from "../../../services/activity-log.js";
import { decideQueuedCommentWakeLookup } from "../domain/policy.js";
import { parseObject, readNonEmptyString } from "../domain/values.js";
import { QueuedCommentMutationError } from "../application/queued-comment-use-cases.js";
import type {
LockedQueuedCommentState,
QueuedCommentActivityLogInput,
QueuedCommentActivityPublication,
QueuedCommentIssueLockWriter,
QueuedCommentQueueTransaction,
QueuedCommentRunRow,
QueuedCommentWakeRow,
} from "../application/queued-comment-ports.js";
type WakeRow = typeof agentWakeupRequests.$inferSelect;
type RunRow = typeof heartbeatRuns.$inferSelect;
function toWakeRow(row: WakeRow): QueuedCommentWakeRow {
return { id: row.id, agentId: row.agentId, status: row.status, runId: row.runId, payload: parseObject(row.payload) };
}
function toRunRow(row: RunRow): QueuedCommentRunRow {
return { id: row.id, status: row.status, runtimeMode: row.runtimeMode,
runtimeModeResolvedAt: row.runtimeModeResolvedAt, runnerProfileJson: row.runnerProfileJson,
contextSnapshot: parseObject(row.contextSnapshot) };
}
export type QueuedCommentQueuePostgresAdapterDeps = {
/** `issueReferenceService(db).syncComment`; runs on the module's own transaction. */
syncCommentReferences(commentId: string, tx: Db): Promise<void>;
/** `issueReferenceService(db).deleteCommentSource`; runs on the module's own transaction. */
deleteCommentReferenceSource(commentId: string, tx: Db): Promise<void>;
/** `externalObjectService(db, opts).syncCommentSafely`; runs on the module's own transaction. */
syncCommentExternalObjectsSafely(commentId: string, tx: Db): Promise<void>;
};
function buildTransaction(tx: Db, companyId: string, deps: QueuedCommentQueuePostgresAdapterDeps): QueuedCommentQueueTransaction {
return {
async updateCommentBody({ issueId, commentId, body, updatedAt }) {
const updated = await tx
.update(issueComments)
.set({ body, updatedAt })
.where(and(eq(issueComments.id, commentId), eq(issueComments.issueId, issueId), eq(issueComments.companyId, companyId)))
.returning({ id: issueComments.id })
.then((rows) => rows[0] ?? null);
return updated !== null;
},
async touchIssueUpdatedAt({ issueId, updatedAt }) {
await tx.update(issues).set({ updatedAt }).where(and(eq(issues.id, issueId), eq(issues.companyId, companyId)));
},
async updateWakeQueuedCommentIds({ wakeId, payload, ids, updatedAt }) {
const row = await tx
.update(agentWakeupRequests)
.set({ payload: withQueuedCommentIdsInWakePayload(payload, ids), updatedAt })
.where(and(eq(agentWakeupRequests.id, wakeId), eq(agentWakeupRequests.companyId, companyId)))
.returning()
.then((rows) => rows[0]);
return toWakeRow(row);
},
async updateQueueRunCommentIds({ queueRunId, contextSnapshot, ids, updatedAt }) {
const row = await tx
.update(heartbeatRuns)
.set({ contextSnapshot: withQueuedCommentIdsInRunContext(contextSnapshot, ids), updatedAt })
.where(and(eq(heartbeatRuns.id, queueRunId), eq(heartbeatRuns.companyId, companyId), eq(heartbeatRuns.status, "queued")))
.returning()
.then((rows) => rows[0] ?? null);
return row ? toRunRow(row) : null;
},
async deleteComment({ issueId, commentId }) {
const row = await tx
.delete(issueComments)
.where(and(eq(issueComments.id, commentId), eq(issueComments.issueId, issueId), eq(issueComments.companyId, companyId)))
.returning()
.then((rows) => rows[0] ?? null);
return row ? (row as IssueComment) : null;
},
async cancelWake({ wakeId, reason, now }) {
await tx
.update(agentWakeupRequests)
.set({ status: "cancelled", finishedAt: now, error: reason, updatedAt: now })
.where(and(eq(agentWakeupRequests.id, wakeId), eq(agentWakeupRequests.companyId, companyId)));
},
async cancelQueueRun({ queueRunId, reason, now }) {
const row = await tx
.update(heartbeatRuns)
.set({ status: "cancelled", finishedAt: now, error: reason, errorCode: "queued_comment_discarded", updatedAt: now })
.where(and(eq(heartbeatRuns.id, queueRunId), eq(heartbeatRuns.companyId, companyId), eq(heartbeatRuns.status, "queued")))
.returning({ id: heartbeatRuns.id })
.then((rows) => rows[0] ?? null);
return row ? { id: row.id } : null;
},
async clearExecutionLockAndTouchIssue({ issueId, executionRunId, updatedAt }) {
await tx
.update(issues)
.set({ executionRunId: null, executionAgentNameKey: null, executionLockedAt: null, updatedAt })
.where(and(eq(issues.id, issueId), eq(issues.companyId, companyId), eq(issues.executionRunId, executionRunId)));
},
async buildQueueSnapshot({ issue, actor, wake, state, queueRun, activeRun }): Promise<IssueQueuedCommentQueue> {
const commentIds = queuedCommentIdsFromWakePayload(wake?.payload ?? null);
const rows =
commentIds.length > 0
? await tx
.select()
.from(issueComments)
.where(
and(
eq(issueComments.companyId, companyId),
eq(issueComments.issueId, issue.id),
inArray(issueComments.id, commentIds),
),
)
: [];
const byId = new Map(rows.map((row) => [row.id, row]));
const comments = commentIds.flatMap((id) => {
const row = byId.get(id);
return row && !row.deletedAt ? [row] : [];
});
const assignedAgent = issue.assigneeAgentId
? await tx
.select({ adapterType: agents.adapterType })
.from(agents)
.where(and(eq(agents.id, issue.assigneeAgentId), eq(agents.companyId, companyId)))
.limit(1)
.then((agentRows) => agentRows[0] ?? null)
: null;
// A queue mutation never delivers same-turn steering itself, so this
// adapter never probes the live runner: it answers
// "temporarily_unavailable" wherever the shared rule says a caller
// may probe. Only the read path probes the live provider.
const steering = decideQueuedCommentQueueSteering({
state,
queueRunRuntimeMode: queueRun?.runtimeMode ?? null,
activeRun,
assignedAgentAdapterType: assignedAgent?.adapterType ?? null,
queuedCommentCount: comments.length,
});
const steeringDisposition: IssueQueuedCommentQueue["steeringDisposition"] =
steering.kind === "probe" ? "temporarily_unavailable" : steering.kind;
return buildQueuedCommentQueueSnapshot({
issueId: issue.id,
queueId: wake?.id ?? null,
state,
activeRunId: activeRun?.id ?? null,
protocol: steering.protocol,
steeringDisposition,
comments,
actorType: actor.actorType,
actorId: actor.actorId,
});
},
async syncCommentReferences(commentId) {
await deps.syncCommentReferences(commentId, tx);
},
async deleteCommentReferenceSource(commentId) {
await deps.deleteCommentReferenceSource(commentId, tx);
},
async syncCommentExternalObjectsSafely(commentId) {
await deps.syncCommentExternalObjectsSafely(commentId, tx);
},
async logActivity(input: QueuedCommentActivityLogInput): Promise<QueuedCommentActivityPublication> {
const publications: ActivityPublication[] = [];
await persistActivityLogRow(
tx,
{
companyId,
actorType: input.actorType,
actorId: input.actorId,
agentId: input.agentId,
runId: input.runId,
agentApiKeyId: input.agentApiKeyId,
action: input.action,
entityType: "issue",
entityId: input.entityId,
details: input.details,
},
publications,
);
return publications[0];
},
};
}
export function createQueuedCommentIssueLockWriter(db: Db, deps: QueuedCommentQueuePostgresAdapterDeps): QueuedCommentIssueLockWriter {
return {
async withLockedQueue(input, fn) {
return db.transaction(async (rawTx) => {
const tx = rawTx as unknown as Db;
const companyId = input.issue.companyId;
await tx
.select({ id: issues.id })
.from(issues)
.where(and(eq(issues.id, input.issue.id), eq(issues.companyId, companyId)))
.for("update");
const wakeRow = await tx
.select()
.from(agentWakeupRequests)
.where(
and(
eq(agentWakeupRequests.id, input.queueId),
eq(agentWakeupRequests.companyId, companyId),
input.issue.assigneeAgentId ? eq(agentWakeupRequests.agentId, input.issue.assigneeAgentId) : undefined,
),
)
.for("update")
.limit(1)
.then((rows) => rows[0] ?? null);
const wakePayload = parseObject(wakeRow?.payload);
const lookup = decideQueuedCommentWakeLookup({
wakePresent: wakeRow !== null,
wakeIssueIdMatches: readNonEmptyString(wakePayload.issueId) === input.issue.id,
hasQueuedCommentIds: queuedCommentIdsFromWakePayload(wakeRow?.payload ?? null).length > 0,
wakeStatus: wakeRow?.status ?? null,
wakeHasRunId: Boolean(wakeRow?.runId),
});
if (lookup.kind === "not_pending") {
throw new QueuedCommentMutationError("queued_comment_not_pending", "The queued message is no longer pending");
}
if (lookup.kind === "already_dispatching") {
throw new QueuedCommentMutationError("queued_comment_already_dispatching", "The queued message is already being dispatched");
}
// Unreachable: `decideQueuedCommentWakeLookup` only returns "deferred" or "check_queue_run" when the wake row is present.
if (!wakeRow) throw new Error("wake-queue: queued-comment lookup resolved without a wake row");
let state: "deferred" | "queued";
let queueRunRow: RunRow | null = null;
if (lookup.kind === "deferred") {
state = "deferred";
} else {
// check_queue_run: `wakeHasRunId` was true for this branch to have been reached.
queueRunRow = await tx
.select()
.from(heartbeatRuns)
.where(
and(
eq(heartbeatRuns.id, wakeRow.runId!),
eq(heartbeatRuns.companyId, companyId),
eq(heartbeatRuns.agentId, wakeRow.agentId),
eq(heartbeatRuns.wakeupRequestId, wakeRow.id),
),
)
.for("update")
.limit(1)
.then((rows) => rows[0] ?? null);
if (!queueRunRow || queueRunRow.status !== "queued") {
throw new QueuedCommentMutationError(
"queued_comment_already_dispatching",
"The queued message is already being dispatched",
);
}
state = "queued";
}
const activeRunId = state === "deferred" ? input.issue.executionRunId : null;
const activeRunRow = activeRunId
? await tx
.select()
.from(heartbeatRuns)
.where(and(eq(heartbeatRuns.id, activeRunId), eq(heartbeatRuns.companyId, companyId), eq(heartbeatRuns.status, "running")))
.for("update")
.limit(1)
.then((rows) => rows[0] ?? null)
: null;
const transaction = buildTransaction(tx, companyId, deps);
const wake = toWakeRow(wakeRow);
const queueRun = queueRunRow ? toRunRow(queueRunRow) : null;
const activeRun = activeRunRow ? toRunRow(activeRunRow) : null;
const queue = await transaction.buildQueueSnapshot({
issue: input.issue,
actor: input.actor,
wake,
state,
queueRun,
activeRun,
});
const locked: LockedQueuedCommentState = { wake, state, queueRun, activeRun, queue };
return fn(locked, transaction);
});
},
};
}