Files
PaperClipAI/server/src/__tests__/issue-queued-comments-routes.test.ts
T
1477d1ecea test: remove the no-op sequential describe modifier (#15286)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip runs the server and runner test suites with Vitest.
> - Vitest 5 removes the deprecated `describe.sequential` property, so
the pending Vitest 5 upgrade fails the type-check and test jobs.
> - `describe.sequential` only changes behaviour inside a
`describe.concurrent` suite, or when `sequence.concurrent` is on.
> - This repository has neither, so the modifier changed nothing at run
time.
> - The benefit is that the Vitest 5 upgrade can land, and the test
files lose a modifier that did no work.

## Linked Issues or Issue Description

Refs: #12969

## What Changed

- Replace every `describe.sequential` use with a plain `describe` call.
- Drop the `{ concurrent: false }` suite option from the two runner test
files.
- Add a comment to `server/vitest.config.ts` that records why these
suites must run one test at a time.
- Leave the package manifests and the lockfile unchanged.

## Why the modifier did nothing

The Vitest documentation states that `describe.sequential` is useful to
run tests in sequence inside a `describe.concurrent` suite, or with the
`--sequence.concurrent` option. `sequence.concurrent` defaults to
`false`.

This repository satisfies neither condition:

- No test file uses `describe.concurrent`, `it.concurrent`, or
`test.concurrent`.
- `server/vitest.config.ts` sets `sequence.concurrent: false`, with
`maxWorkers: 1`, `maxConcurrency: 1`, and `isolate: true`.
- `packages/paperclip-runner/vitest.config.ts` sets no `sequence` block,
so the `false` default applies.

`packages/db` and `cli` already run the same embedded-Postgres suites
with a plain `describe`, and those jobs are green. The server package
was the only outlier.

The modifier did carry one real piece of knowledge: these suites need
their tests to run one at a time. The new comment in
`server/vitest.config.ts` records that reason next to the setting that
enforces it.

## Verification

- `git grep` for `describe.sequential` returns nothing outside
`node_modules`.
- The author ran the changed server test files under the installed
Vitest 4, and the results match the results without this change.
- Two very large embedded-Postgres test files exceeded the author's
local memory limit, so the CI test jobs cover those two.
- The two changed runner test files have pre-existing local failures
caused by a missing Rust toolchain and a missing global `pnpm` binary.
The failures are identical with and without this change.
- The author type-checked the changed files and found no new error.
- CI must pass the typecheck, build, server test, and runner verify
jobs.

## Risks

- Low risk. Suite execution stays serial, because the Vitest config
enforces it.
- The change adds no dependency and changes no package manifest or
lockfile.
- A future change that turns `sequence.concurrent` on would break these
suites. The new config comment warns against it.

## Model Used

- Claude Sonnet 5 — code edits and local verification.
- OpenAI Codex, GPT-5 — the earlier revision of this branch.

## Test plan

- [x] Every CI check reaches a terminal green state. A pending or queued
check is not a pass.
- [x] The `Typecheck + Release Registry` job passes. This change must
not introduce a type error.
- [x] The `Build` job passes.
- [x] The server test jobs and the runner verify jobs pass.
- [x] Greptile re-reviews this commit set and posts a passing verdict.
The dependabot waiver does not apply to this pull request.
- [x] `mergeable` reads `MERGEABLE` as a terminal value.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have linked the related public issue with `Refs: #12969`
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-Authored-By: Priya Raman <priya.raman@paperclip.ing>

---------

Co-authored-by: Priya Raman <priya.raman@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: nickyleach <331803+nickyleach@users.noreply.github.com>
2026-10-06 07:19:15 -07:00

1748 lines
92 KiB
TypeScript

import { spawn, type ChildProcess } from "node:child_process";
import { runningProcesses } from "../adapters/index.js";
import { randomUUID } from "node:crypto";
import express from "express";
import request from "supertest";
import { eq, sql } from "drizzle-orm";
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
import {
agentRuntimeState,
agentWakeupRequests,
agents,
activityLog,
companies,
companyMemberships,
companySkills,
createDb,
environmentLeases,
heartbeatRuns,
heartbeatRunEvents,
issueComments,
issueThreadInteractions,
issueRecoveryActions,
issues,
runIdentityContexts,
} from "@paperclipai/db";
import { errorHandler } from "../middleware/index.js";
import { issueRoutes } from "../routes/issues.js";
import { heartbeatService } from "../services/heartbeat.js";
import { issueService } from "../services/issues.js";
import { issueThreadInteractionService } from "../services/issue-thread-interactions.js";
import { remoteTerminationReceipt } from "../services/remote-execution-termination.js";
import { initializeRunIdentity, reconcileSteeredIdentity } from "../services/run-identity.js";
import { appendHeartbeatRunEvent } from "../services/heartbeat-run-events.js";
import {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
} from "./helpers/embedded-postgres.js";
const steerNativeSessionMock = vi.hoisted(() => vi.fn());
vi.mock("../services/native-runtime/native-session-executor.js", async (importOriginal) => {
const actual = await importOriginal<typeof import("../services/native-runtime/native-session-executor.js")>();
steerNativeSessionMock.mockImplementation(actual.steerNativeSession);
return { ...actual, steerNativeSession: steerNativeSessionMock };
});
const { NativeSessionSteeringError } = await import("../services/native-runtime/native-session-executor.js");
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
if (!embeddedPostgresSupport.supported) {
console.warn(
`Skipping queued-comment route tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
);
}
describeEmbeddedPostgres("issue queued-comment routes", () => {
let db!: ReturnType<typeof createDb>;
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
beforeAll(async () => {
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-queued-comments-");
db = createDb(tempDb.connectionString);
}, 30_000);
const testProcesses = new Map<string, ChildProcess>();
afterEach(async () => {
for (const [id, child] of testProcesses) {
runningProcesses.delete(id);
child.kill();
}
testProcesses.clear();
// Each case owns the entire disposable database. Clear the full company
// graph, including attribution rows and constraints added by migrations.
await db.execute(sql`TRUNCATE TABLE companies CASCADE`);
});
afterAll(async () => {
await tempDb?.cleanup();
});
function app(companyId: string, userId = "queue-owner", agentActor?: { agentId: string; runId: string }) {
const testApp = express();
testApp.use(express.json());
testApp.use((req, _res, next) => {
(req as any).actor = {
type: agentActor ? "agent" : "board",
...(agentActor ? { ...agentActor, companyId } : {}),
source: "session",
userId,
companyIds: [companyId],
memberships: [{ companyId, status: "active", membershipRole: "operator" }],
isInstanceAdmin: false,
};
next();
});
testApp.use("/api", issueRoutes(db, {} as any, {}));
testApp.use(errorHandler);
return testApp;
}
async function seedQueue() {
const companyId = randomUUID();
const agentId = randomUUID();
const issueId = randomUUID();
const runId = randomUUID();
const wakeId = randomUUID();
const commentIds = [randomUUID(), randomUUID()];
await db.insert(companies).values({
id: companyId,
name: "Queue Test Company",
issuePrefix: "QUE",
requireBoardApprovalForNewAgents: false,
});
await db.insert(agents).values({
id: agentId,
companyId,
name: "Paperclip Runner",
role: "engineer",
status: "idle",
adapterType: "paperclip_runner",
adapterConfig: {},
runtimeConfig: {},
permissions: {},
});
await db.insert(companyMemberships).values([
{
companyId,
principalType: "user",
principalId: "queue-owner",
status: "active",
membershipRole: "operator",
},
{
companyId,
principalType: "user",
principalId: "other-operator",
status: "active",
membershipRole: "operator",
},
]);
await db.insert(agentWakeupRequests).values({
id: wakeId,
companyId,
agentId,
source: "issue_comment",
reason: "Follow-up comments arrived during the active run",
status: "deferred_issue_execution",
requestedByActorType: "user",
requestedByActorId: "queue-owner",
payload: {
issueId,
commentId: commentIds[1],
_paperclipWakeContext: {
commentId: commentIds[1],
wakeCommentId: commentIds[1],
wakeCommentIds: commentIds,
},
},
});
await db.insert(heartbeatRuns).values({
id: runId,
companyId,
agentId,
invocationSource: "assignment",
triggerDetail: "queue route test",
status: "running",
runtimeMode: "native",
startedAt: new Date("2026-08-22T15:00:00.000Z"),
contextSnapshot: { issueId },
});
await db.insert(issues).values({
id: issueId,
companyId,
identifier: "QUE-1",
title: "Queued steering",
status: "in_progress",
priority: "medium",
assigneeAgentId: agentId,
executionRunId: runId,
});
await db.insert(issueComments).values(commentIds.map((id, index) => ({
id,
companyId,
issueId,
authorType: "user" as const,
authorUserId: "queue-owner",
body: index === 0 ? "First queued message" : "Second queued message",
createdAt: new Date(`2026-08-22T15:0${index + 1}:00.000Z`),
updatedAt: new Date(`2026-08-22T15:0${index + 1}:00.000Z`),
})));
return { companyId, agentId, issueId, runId, wakeId, commentIds };
}
it.each(["cancelled", "running"])("rejects a late Done from an interrupted %s task run at the write boundary", async status => {
const seeded = await seedQueue();
await db.update(heartbeatRuns).set({ status, resultJson: { executionCancellation: { state: "requested" } } })
.where(eq(heartbeatRuns.id, seeded.runId));
// Model a request that passed middleware before cancellation and reached
// the service after the run lost its authority. Recovery may have released
// checkout and changed the task to Blocked in the meantime.
await db.update(issues).set({ status: "blocked", executionRunId: null }).where(eq(issues.id, seeded.issueId));
await expect(issueService(db).update(seeded.issueId, { status: "done",
actorAgentId: seeded.agentId, actorRunId: seeded.runId,
})).rejects.toMatchObject({ status: 403, details: { code: "agent_run_cancelled" } });
expect((await db.select().from(issues).where(eq(issues.id, seeded.issueId)))[0].status).toBe("blocked");
// A board disposition remains authoritative.
expect(await issueService(db).update(seeded.issueId, { status: "done", actorUserId: "queue-owner" }))
.toMatchObject({ status: "done" });
});
it("delivers a saved Grok interrupt exactly once after remote cleanup, including concurrent restart sweeps", async () => {
const seeded = await seedQueue();
await db.update(agents).set({ adapterType: "grok_local", runtimeConfig: { heartbeat: { maxConcurrentRuns: 1 } } })
.where(eq(agents.id, seeded.agentId));
await db.update(heartbeatRuns).set({ runtimeMode: "legacy", status: "cancelled", errorCode: "operator_interrupted",
finishedAt: new Date("2026-08-22T15:03:00.000Z"),
contextSnapshot: { issueId: seeded.issueId, paperclipWorkspace: { remoteExecution: { transport: "sandbox" } } },
}).where(eq(heartbeatRuns.id, seeded.runId));
await db.update(issues).set({ status: "blocked", executionRunId: null }).where(eq(issues.id, seeded.issueId));
await db.insert(issueRecoveryActions).values({ companyId: seeded.companyId, sourceIssueId: seeded.issueId,
kind: "active_run_watchdog", cause: "legacy_execution_requires_reconciliation", fingerprint: seeded.runId,
status: "resolved", outcome: "blocked", nextAction: "Automatic recovery stopped.",
evidence: { runId: seeded.runId, automaticRecovery: { replay: "blocked", actionOutcome: "unknown" } },
});
const [lease] = await db.insert(environmentLeases).values({ companyId: seeded.companyId,
issueId: seeded.issueId, heartbeatRunId: seeded.runId, provider: "daytona", providerLeaseId: randomUUID(),
status: "active", leasePolicy: "ephemeral",
}).returning();
// Occupy the agent's slot on another task so this test never invokes a model.
await db.insert(heartbeatRuns).values({ companyId: seeded.companyId, agentId: seeded.agentId,
status: "running", contextSnapshot: { issueId: randomUUID() },
});
const client = app(seeded.companyId);
const queue = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
await request(client).post(`/api/issues/${seeded.issueId}/queued-comments/interrupt`).send({
queueId: seeded.wakeId, revision: queue.body.revision, targetRunId: null,
}).expect(200);
expect((await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId)))[0].status)
.toBe("deferred_issue_execution");
// Cleanup timestamps alone are not proof. A mismatched receipt is not proof either.
await db.update(environmentLeases).set({ status: "expired", releasedAt: new Date(), cleanupStatus: "success",
metadata: { remoteExecutionTermination: { ...remoteTerminationReceipt(lease, {
providerLeaseId: lease.providerLeaseId, state: "destroyed",
}), runId: randomUUID() } },
}).where(eq(environmentLeases.id, lease.id));
await heartbeatService(db).resumeQueuedCommentInterrupt(seeded.companyId, seeded.wakeId);
expect((await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId)))[0].status)
.toBe("deferred_issue_execution");
await db.update(environmentLeases).set({ metadata: { remoteExecutionTermination: remoteTerminationReceipt(lease, {
providerLeaseId: lease.providerLeaseId, state: "destroyed",
}) } }).where(eq(environmentLeases.id, lease.id));
await db.update(agentWakeupRequests).set({ updatedAt: new Date(0) }).where(eq(agentWakeupRequests.id, seeded.wakeId));
await Promise.all([heartbeatService(db).resumeQueuedRuns(), heartbeatService(db).resumeQueuedRuns()]);
const [delivered] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(delivered.status).toBe("coalesced");
const [successor] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, delivered.runId!));
expect(successor.contextSnapshot).toMatchObject({ wakeCommentIds: seeded.commentIds, previousRunId: seeded.runId,
forceFreshSession: true });
await heartbeatService(db).resumeQueuedCommentInterrupt(seeded.companyId, seeded.wakeId);
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, seeded.companyId))).toHaveLength(3);
});
it("allows only the exact task mutation that stopped its own run to commit", async () => {
const seeded = await seedQueue();
const stopId = randomUUID();
await db.update(heartbeatRuns).set({ status: "cancelled", errorCode: "issue_reassigned",
resultJson: { reassignmentStopConfirmed: true, issueMutationStopId: stopId },
}).where(eq(heartbeatRuns.id, seeded.runId));
const data = { status: "todo", actorAgentId: seeded.agentId, actorRunId: seeded.runId };
await expect(issueService(db).update(seeded.issueId, { ...data, actorRunStopId: randomUUID() }))
.rejects.toMatchObject({ status: 403, details: { code: "agent_run_cancelled" } });
expect(await issueService(db).update(seeded.issueId, { ...data, actorRunStopId: stopId }))
.toMatchObject({ status: "todo" });
await expect(issueService(db).update(seeded.issueId, { ...data, status: "done" }))
.rejects.toMatchObject({ status: 403, details: { code: "agent_run_cancelled" } });
});
it.each(["questions", "verdicts"] as const)("rejects admitted %s responses after Stop revokes the run", async kind => {
const seeded = await seedQueue();
const service = issueThreadInteractionService(db);
const issue = { id: seeded.issueId, companyId: seeded.companyId };
const interaction = await service.create(issue, kind === "questions" ? {
kind: "ask_user_questions", resolverPolicy: "board_or_agents", payload: { version: 1,
questions: [{ id: "scope", prompt: "Choose scope", selectionMode: "single", options: [{ id: "first", label: "First" }] }],
},
} : {
kind: "request_item_verdicts", resolverPolicy: "board_or_agents", payload: { version: 1,
prompt: "Review the work",
items: [{ id: "work", label: "Work" }],
},
}, { userId: "queue-owner" });
await db.update(heartbeatRuns).set({ resultJson: { executionCancellation: { state: "requested" } } })
.where(eq(heartbeatRuns.id, seeded.runId));
const actor = { agentId: seeded.agentId, runId: seeded.runId };
const responding = kind === "questions"
? service.answerQuestions(issue, interaction.id, { answers: [{ questionId: "scope", optionIds: ["first"] }] }, actor)
: service.submitItemVerdicts(issue, interaction.id, { verdicts: [{ id: "work", verdict: "approve" }] }, actor);
await expect(responding).rejects.toMatchObject({ status: 403, details: { code: "agent_run_cancelled" } });
expect((await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, interaction.id)))[0].status)
.toBe("pending");
});
it.each((["request_confirmation", "request_checkbox_confirmation", "ask_user_questions"] as const)
.flatMap(kind => (["legacy", "native"] as const).map(runtime => ({ kind, runtime }))))(
"queues $kind resolution while its $runtime source run is still running",
async ({ kind, runtime }) => {
const seeded = await seedQueue();
await db.delete(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
if (runtime === "legacy") {
await db.update(agents).set({ adapterType: "codex_local" }).where(eq(agents.id, seeded.agentId));
await db.update(heartbeatRuns).set({ runtimeMode: "legacy" }).where(eq(heartbeatRuns.id, seeded.runId));
}
const child = spawn(process.execPath, ["-e", "setInterval(() => {}, 1000)"], { stdio: "ignore" });
testProcesses.set(seeded.runId, child);
runningProcesses.set(seeded.runId, { child, graceSec: 1, processGroupId: null });
const interactionId = randomUUID();
await db.insert(issueThreadInteractions).values({
id: interactionId, companyId: seeded.companyId, issueId: seeded.issueId,
kind, status: "pending", createdByAgentId: seeded.agentId, sourceRunId: seeded.runId,
continuationPolicy: "wake_assignee", requestedResolverPolicy: "human_only",
effectiveResolverPolicy: "human_only", resolverPolicyProvenance: "explicit",
payload: kind === "ask_user_questions"
? { version: 1, questions: [{ id: "runtime", prompt: "Which runtime?", selectionMode: "single", required: true,
options: [{ id: "node", label: "Node.js" }, { id: "python", label: "Python" }] }] }
: kind === "request_checkbox_confirmation"
? { version: 1, prompt: "Accept scope", options: [{ id: "scope", label: "Build the app" }] }
: { version: 1, prompt: "Build the app?", detailsMarkdown: "Create exactly one child task." },
});
const client = app(seeded.companyId);
await request(client).post(`/api/issues/${seeded.issueId}/interactions/${interactionId}/${kind === "ask_user_questions" ? "respond" : "accept"}`)
.send(kind === "ask_user_questions" ? { answers: [{ questionId: "runtime", optionIds: ["node"] }] }
: kind === "request_checkbox_confirmation" ? { selectedOptionIds: ["scope"] } : {}).expect(200);
await vi.waitFor(async () => {
const receipts = await db.select().from(agentWakeupRequests);
expect(receipts.some(row => row.status === "deferred_issue_execution")).toBe(true);
});
const queue = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
expect(queue.body.entries).toHaveLength(1);
expect(queue.body.entries[0]).toMatchObject({
source: { kind: "interaction", interactionId, interactionKind: kind },
canEdit: false, canDiscard: false,
});
expect(queue.body.targetRunId).toBe(seeded.runId);
expect(queue.body.entries[0].comment.body).toContain(kind === "ask_user_questions" ? "Node.js" : "Accepted");
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seeded.runId)))[0]!.status).toBe("running");
const wakes = await db.select().from(agentWakeupRequests);
expect(wakes).toHaveLength(1);
expect(wakes[0]).toMatchObject({ status: "deferred_issue_execution", runId: null });
},
);
async function seedResponseQueue(native = false) {
const seeded = await seedQueue();
const interactionId = randomUUID();
await db.insert(issueThreadInteractions).values({
id: interactionId, companyId: seeded.companyId, issueId: seeded.issueId,
kind: "request_confirmation", status: "accepted", createdByAgentId: seeded.agentId,
sourceRunId: seeded.runId, resolvedByUserId: "queue-owner", resolvedAt: new Date(),
continuationPolicy: "wake_assignee", requestedResolverPolicy: "human_only",
effectiveResolverPolicy: "human_only", resolverPolicyProvenance: "explicit",
title: "Build the app", payload: { version: 1, prompt: "Build the app?" },
result: { decision: "accepted" },
});
const context = { issueId: seeded.issueId, interactionId, interactionKind: "request_confirmation",
interactionStatus: "accepted", planReviewInteraction: { id: interactionId,
acceptedTargetRevision: { documentId: "plan-1", revisionId: "revision-1" } } };
await db.update(agentWakeupRequests).set({ payload: { issueId: seeded.issueId,
mutation: "interaction", interactionId, interactionStatus: "accepted", _paperclipWakeContext: context },
}).where(eq(agentWakeupRequests.id, seeded.wakeId));
if (!native) {
await db.update(agents).set({ adapterType: "claude_local",
runtimeConfig: { heartbeat: { maxConcurrentRuns: 1 } },
}).where(eq(agents.id, seeded.agentId));
await db.update(heartbeatRuns).set({ runtimeMode: "legacy" }).where(eq(heartbeatRuns.id, seeded.runId));
}
return { ...seeded, interactionId, context };
}
it("keeps messages and approvals in separate durable queues in either arrival order", async () => {
const seeded = await seedResponseQueue();
const child = spawn(process.execPath, ["-e", "setInterval(() => {}, 1000)"], { stdio: "ignore" });
testProcesses.set(seeded.runId, child);
runningProcesses.set(seeded.runId, { child, graceSec: 1, processGroupId: null });
const client = app(seeded.companyId);
await request(client).post(`/api/issues/${seeded.issueId}/comments`)
.send({ body: "Keep this message too" }).expect(201);
await vi.waitFor(async () => expect(await db.select().from(agentWakeupRequests)).toHaveLength(2));
let wakes = await db.select().from(agentWakeupRequests);
expect(wakes.find(w => w.id === seeded.wakeId)?.payload).not.toHaveProperty("commentId");
// A second resolved card must not overwrite either existing receipt.
const nextId = randomUUID();
await db.insert(issueThreadInteractions).values({
id: nextId, companyId: seeded.companyId, issueId: seeded.issueId,
kind: "request_confirmation", status: "pending", createdByAgentId: seeded.agentId,
sourceRunId: seeded.runId, continuationPolicy: "wake_assignee",
requestedResolverPolicy: "human_only", effectiveResolverPolicy: "human_only",
resolverPolicyProvenance: "explicit", payload: { version: 1, prompt: "Second approval?" },
});
await request(client).post(`/api/issues/${seeded.issueId}/interactions/${nextId}/accept`).send({}).expect(200);
wakes = await db.select().from(agentWakeupRequests);
expect(wakes).toHaveLength(3);
expect(wakes.filter(w => w.payload?.mutation === "interaction").map(w => w.payload?.interactionId).sort())
.toEqual([seeded.interactionId, nextId].sort());
expect(wakes.find(w => w.payload?.commentId)?.payload?._paperclipWakeContext).not.toHaveProperty("interactionId");
});
it("recovers an approval acknowledged before the steering transaction failed without redelivery", async () => {
const seeded = await seedResponseQueue(true);
await seedDispatchIdentity(seeded);
// Reconstruct the durable state after acknowledgement and HTTP rollback.
const [pending] = await db.insert(runIdentityContexts).values({
companyId: seeded.companyId, runId: seeded.runId, revision: 2,
cause: "steering", correlationId: `interaction:${seeded.interactionId}`,
messageId: seeded.interactionId, responsibleUserId: "queue-owner", status: "pending",
}).returning();
await db.insert(heartbeatRunEvents).values({
companyId: seeded.companyId, runId: seeded.runId, agentId: seeded.agentId,
seq: 1, eventType: "item.completed", sourceEventId: randomUUID(),
payload: { prpEvent: { turnId: "approval-ack", itemId: `approval-ack:steer:${seeded.interactionId}`,
payload: { kind: "steering_acknowledgement" } } },
});
const client = app(seeded.companyId);
const queue = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
const body = { queueId: seeded.wakeId, targetRunId: seeded.runId, revision: queue.body.revision };
const endpoint = `/api/issues/${seeded.issueId}/queued-comments/${seeded.interactionId}/steer`;
const count = steerNativeSessionMock.mock.calls.length;
await request(client).post(endpoint).send(body).expect(200);
expect(steerNativeSessionMock.mock.calls).toHaveLength(count);
expect((await db.select().from(runIdentityContexts).where(eq(runIdentityContexts.id, pending.id)))[0].status).toBe("accepted");
});
it("interrupts an active legacy run and delivers the exact approval once", async () => {
const seeded = await seedResponseQueue();
const child = spawn(process.execPath, ["-e", "setInterval(() => {}, 1000)"], { stdio: "ignore" });
testProcesses.set(seeded.runId, child);
runningProcesses.set(seeded.runId, { child, graceSec: 1, processGroupId: null });
await db.update(heartbeatRuns).set({ processPid: child.pid }).where(eq(heartbeatRuns.id, seeded.runId));
// Occupy another task to inspect admission without invoking a paid provider.
await db.insert(heartbeatRuns).values({ companyId: seeded.companyId, agentId: seeded.agentId,
status: "running", contextSnapshot: { issueId: randomUUID() } });
const client = app(seeded.companyId);
const queue = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
await request(client).post(`/api/issues/${seeded.issueId}/queued-comments/interrupt`)
.send({ queueId: seeded.wakeId, targetRunId: seeded.runId, revision: queue.body.revision }).expect(200);
runningProcesses.delete(seeded.runId);
await heartbeatService(db).resumeQueuedCommentInterrupt(seeded.companyId, seeded.wakeId);
const [wake] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(["queued", "coalesced"]).toContain(wake.status);
const [successor] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, wake.runId!));
expect(successor.status).toBe("queued");
expect(successor.contextSnapshot).toMatchObject(seeded.context);
await heartbeatService(db).resumeQueuedCommentInterrupt(seeded.companyId, seeded.wakeId);
expect(await db.select().from(heartbeatRuns)).toHaveLength(3);
});
it("steers a saved approval only on click and acknowledges retries once", async () => {
const seeded = await seedResponseQueue(true);
await seedDispatchIdentity(seeded);
steerNativeSessionMock.mockResolvedValueOnce({ turnId: "approval-turn" });
const client = app(seeded.companyId);
const queue = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
const body = { queueId: seeded.wakeId, targetRunId: seeded.runId, revision: queue.body.revision };
const endpoint = `/api/issues/${seeded.issueId}/queued-comments/${seeded.interactionId}/steer`;
await request(client).post(endpoint).send(body).expect(200);
const delivered = steerNativeSessionMock.mock.calls.at(-1)![0];
expect(delivered.message).toContain(seeded.interactionId);
expect(delivered.message).toContain('"status": "accepted"');
expect((await db.select().from(runIdentityContexts).where(eq(runIdentityContexts.messageId, seeded.interactionId)))[0])
.toMatchObject({ status: "accepted", responsibleUserId: "queue-owner", cause: "steering" });
const callCount = steerNativeSessionMock.mock.calls.length;
await request(client).post(endpoint).send(body).expect(200);
expect(steerNativeSessionMock.mock.calls).toHaveLength(callCount);
expect((await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId)))[0].status).toBe("cancelled");
expect(await db.select().from(issueComments)).toHaveLength(2); // no forged approval comment
});
it.each([false, true])("lets the source run finish review after acceptance, before wake persistence: %s", async (beforeWake) => {
const seeded = await seedResponseQueue();
if (beforeWake) await db.delete(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
const client = app(seeded.companyId, "queue-owner", { agentId: seeded.agentId, runId: seeded.runId });
const result = await request(client).patch(`/api/issues/${seeded.issueId}`).send({ status: "in_review", reviewInteractionId: seeded.interactionId });
expect(result.status, JSON.stringify(result.body)).toBe(200);
const [task] = await db.select().from(issues).where(eq(issues.id, seeded.issueId));
expect(task.assigneeAgentId).toBe(seeded.agentId);
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seeded.runId)))[0].status).toBe("running");
});
it.each(["legacy", "native"] as const)("resumes a queued plan approval across a stopped %s run only after cleanup", async (runtime) => {
const seeded = await seedResponseQueue(runtime === "native");
await db.update(agents).set({ runtimeConfig: { heartbeat: { maxConcurrentRuns: 1 } } }).where(eq(agents.id, seeded.agentId));
const [receipt] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
await db.update(agentWakeupRequests).set({ payload: { ...receipt.payload,
_paperclipWakeContext: { ...seeded.context, forceFreshSession: true } },
}).where(eq(agentWakeupRequests.id, seeded.wakeId));
await db.update(heartbeatRuns).set({ status: "failed", finishedAt: new Date(), errorCode: "process_lost", processPid: process.pid })
.where(eq(heartbeatRuns.id, seeded.runId));
await db.update(issues).set({ executionRunId: null }).where(eq(issues.id, seeded.issueId));
await db.insert(issueRecoveryActions).values({ companyId: seeded.companyId, sourceIssueId: seeded.issueId,
kind: "active_run_watchdog", cause: runtime === "legacy" ? "legacy_execution_requires_reconciliation" : "native_continuation_requires_reconciliation",
fingerprint: seeded.runId, status: "resolved", outcome: "blocked", nextAction: "Inspect stopped execution",
evidence: { runId: seeded.runId, automaticRecovery: { replay: "blocked", actionOutcome: "unknown" } } });
await db.insert(heartbeatRuns).values({ companyId: seeded.companyId, agentId: seeded.agentId,
status: "running", contextSnapshot: { issueId: randomUUID() } });
const client = app(seeded.companyId);
const queue = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
await request(client).post(`/api/issues/${seeded.issueId}/queued-comments/interrupt`)
.send({ queueId: seeded.wakeId, targetRunId: null, revision: queue.body.revision }).expect(200);
const [waiting] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(waiting.status).toBe("deferred_issue_execution");
expect(waiting.payload?.executionWait).toMatchObject({ reason: "process_running" });
await db.update(heartbeatRuns).set({ processPid: 999999999 }).where(eq(heartbeatRuns.id, seeded.runId));
await heartbeatService(db).resumeQueuedCommentInterrupt(seeded.companyId, seeded.wakeId);
const [wake] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(wake.status).toBe("coalesced");
const [successor] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, wake.runId!));
expect(successor.contextSnapshot).toMatchObject({ ...seeded.context, forceFreshSession: true, previousRunId: seeded.runId });
await heartbeatService(db).resumeQueuedCommentInterrupt(seeded.companyId, seeded.wakeId);
expect(await db.select().from(heartbeatRuns)).toHaveLength(3);
});
it("keeps queued approval content immutable through comment mutation endpoints", async () => {
const seeded = await seedResponseQueue();
const client = app(seeded.companyId);
const queue = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
const body = { queueId: seeded.wakeId, revision: queue.body.revision };
await request(client).patch(`/api/issues/${seeded.issueId}/queued-comments/${seeded.interactionId}`)
.send({ ...body, body: "Reject instead" }).expect(409);
await request(client).delete(`/api/issues/${seeded.issueId}/queued-comments/${seeded.interactionId}`).send(body).expect(409);
await request(client).put(`/api/issues/${seeded.issueId}/queued-comments/order`)
.send({ ...body, orderedCommentIds: [seeded.interactionId] }).expect(409);
expect((await db.select().from(issueThreadInteractions).where(eq(issueThreadInteractions.id, seeded.interactionId)))[0].status).toBe("accepted");
expect((await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId)))[0].status).toBe("deferred_issue_execution");
});
it("does not let a viewer steer someone else's approval", async () => {
const seeded = await seedResponseQueue(true);
const client = app(seeded.companyId);
const queue = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
await db.update(companyMemberships).set({ membershipRole: "viewer" }).where(eq(companyMemberships.principalId, "queue-owner"));
const calls = steerNativeSessionMock.mock.calls.length;
await request(client).post(`/api/issues/${seeded.issueId}/queued-comments/${seeded.interactionId}/steer`)
.send({ queueId: seeded.wakeId, targetRunId: seeded.runId, revision: queue.body.revision }).expect(403);
expect(steerNativeSessionMock.mock.calls).toHaveLength(calls);
});
it("does not steer a plan approval that requires a fresh execution session", async () => {
const seeded = await seedResponseQueue(true);
const [wake] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
await db.update(agentWakeupRequests).set({ payload: { ...wake.payload,
_paperclipWakeContext: { ...seeded.context, forceFreshSession: true } } }).where(eq(agentWakeupRequests.id, seeded.wakeId));
const client = app(seeded.companyId);
const queue = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
expect(queue.body.entries[0].source.requiresFreshSession).toBe(true);
const calls = steerNativeSessionMock.mock.calls.length;
const result = await request(client).post(`/api/issues/${seeded.issueId}/queued-comments/${seeded.interactionId}/steer`)
.send({ queueId: seeded.wakeId, targetRunId: seeded.runId, revision: queue.body.revision }).expect(409);
expect(result.body.details.code).toBe("queued_response_requires_fresh_session");
expect(steerNativeSessionMock.mock.calls).toHaveLength(calls);
});
it("rejects stale agent handback without cancelling the accepted proposal's run", async () => {
const seeded = await seedResponseQueue();
await db.update(issues).set({ createdByUserId: "queue-owner" }).where(eq(issues.id, seeded.issueId));
const client = app(seeded.companyId, "queue-owner", { agentId: seeded.agentId, runId: seeded.runId });
const result = await request(client).patch(`/api/issues/${seeded.issueId}`)
.send({ status: "in_review", assigneeAgentId: null, assigneeUserId: "queue-owner" }).expect(409);
expect(result.body.details.code).toBe("interaction_response_queued");
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seeded.runId)))[0].status).toBe("running");
});
it("promotes an approval after normal completion without a click", async () => {
const seeded = await seedResponseQueue();
await db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date() }).where(eq(heartbeatRuns.id, seeded.runId));
await db.update(issues).set({ executionRunId: null }).where(eq(issues.id, seeded.issueId));
await db.insert(heartbeatRuns).values({ companyId: seeded.companyId, agentId: seeded.agentId,
status: "running", contextSnapshot: { issueId: randomUUID() } });
await heartbeatService(db).resumeQueuedRuns();
const [wake] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(wake.status).toBe("queued");
const [successor] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, wake.runId!));
expect(successor.contextSnapshot).toMatchObject(seeded.context);
await heartbeatService(db).resumeQueuedRuns();
expect(await db.select().from(heartbeatRuns)).toHaveLength(3);
});
it.each(["stale revision", "native run", "different issue"] as const)(
"rejects queued interruption for a %s without stopping the run",
async (scenario) => {
const seeded = await seedQueue();
if (scenario !== "native run") {
await db.update(agents).set({ adapterType: "codex_local" }).where(eq(agents.id, seeded.agentId));
await db.update(heartbeatRuns).set({ runtimeMode: "legacy" }).where(eq(heartbeatRuns.id, seeded.runId));
}
const client = app(seeded.companyId);
const queue = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
if (scenario === "different issue") {
await db.update(heartbeatRuns).set({ contextSnapshot: { issueId: randomUUID() } }).where(eq(heartbeatRuns.id, seeded.runId));
}
await request(client).post(`/api/issues/${seeded.issueId}/queued-comments/interrupt`).send({
queueId: seeded.wakeId, targetRunId: seeded.runId,
revision: scenario === "stale revision" ? "stale" : queue.body.revision,
}).expect(409);
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seeded.runId)))[0]!.status).toBe("running");
},
);
it("does not accept interruption authority from an agent wake payload", async () => {
const seeded = await seedQueue();
await db.update(agents).set({ adapterType: "claude_local",
runtimeConfig: { heartbeat: { maxConcurrentRuns: 1 } },
}).where(eq(agents.id, seeded.agentId));
await db.update(heartbeatRuns).set({ runtimeMode: "legacy" }).where(eq(heartbeatRuns.id, seeded.runId));
await heartbeatService(db).wakeup(seeded.agentId, {
source: "on_demand", reason: "issue_commented",
requestedByActorType: "agent", requestedByActorId: seeded.agentId,
payload: { issueId: seeded.issueId, commentId: seeded.commentIds[1],
queuedCommentInterrupt: { actorId: "other-operator", requestedAt: new Date().toISOString() } },
contextSnapshot: { issueId: seeded.issueId, wakeCommentId: seeded.commentIds[1] },
});
const wakes = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.companyId, seeded.companyId));
expect(wakes.length).toBeGreaterThan(0);
expect(wakes.every(wake => !wake.payload?.queuedCommentInterrupt)).toBe(true);
const runs = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, seeded.companyId));
expect(runs.find(run => run.id === seeded.runId)?.status).toBe("running");
expect(runs.every(run => !run.contextSnapshot?.explicitUserContinuation)).toBe(true);
});
it("denies a viewer's interrupt before persisting intent or cancelling a run", async () => {
const seeded = await seedQueue();
await db.update(companyMemberships).set({ membershipRole: "viewer" })
.where(eq(companyMemberships.principalId, "other-operator"));
const client = app(seeded.companyId, "other-operator");
const queue = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
await request(client).post(`/api/issues/${seeded.issueId}/queued-comments/interrupt`).send({
queueId: seeded.wakeId, revision: queue.body.revision, targetRunId: seeded.runId,
}).expect(403);
const [wake] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(wake.payload?.queuedCommentInterrupt).toBeUndefined();
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seeded.runId)))[0].status).toBe("running");
});
it.each([null, "stopped-target", "system-receipt"])("sends a stopped legacy queue once with target %s", async (target) => {
const seeded = await seedQueue();
if (target === "system-receipt") await db.update(agentWakeupRequests).set({
requestedByActorType: "system", requestedByActorId: "heartbeat",
}).where(eq(agentWakeupRequests.id, seeded.wakeId));
await db.update(agents).set({ adapterType: "claude_local",
runtimeConfig: { heartbeat: { maxConcurrentRuns: 1 } },
}).where(eq(agents.id, seeded.agentId));
await db.update(heartbeatRuns).set({ runtimeMode: "legacy", status: "succeeded",
finishedAt: new Date("2026-08-22T15:03:00.000Z"),
}).where(eq(heartbeatRuns.id, seeded.runId));
await db.update(issues).set({ executionRunId: null }).where(eq(issues.id, seeded.issueId));
// Occupy this agent on a different task so the actual successor remains
// queued and the test never launches a provider.
await db.insert(heartbeatRuns).values({ companyId: seeded.companyId, agentId: seeded.agentId,
status: "running", contextSnapshot: { issueId: randomUUID() },
});
const client = app(seeded.companyId, "other-operator");
const queue = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
expect(queue.body.targetRunId).toBeNull();
const body = { queueId: seeded.wakeId, revision: queue.body.revision,
targetRunId: target === "stopped-target" ? seeded.runId : null };
await request(client).post(`/api/issues/${seeded.issueId}/queued-comments/interrupt`).send(body).expect(200);
const [wake] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(wake.status).toBe("coalesced");
const [successor] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, wake.runId!));
expect(successor.status).toBe("queued");
expect(successor.responsibleUserId).toBe("other-operator");
const identity = await initializeRunIdentity(db, {
companyId: seeded.companyId, issueId: seeded.issueId,
runId: successor.id, messageIds: seeded.commentIds, responsibleUserId: "queue-owner", cause: "dispatch",
});
expect(identity.responsibleUserId).toBe("other-operator");
expect(identity.cause).toBe("queued_comment_interrupt");
expect(successor.contextSnapshot?.wakeCommentIds).toEqual(seeded.commentIds);
await heartbeatService(db).resumeQueuedCommentInterrupt(seeded.companyId, seeded.wakeId);
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, seeded.companyId))).toHaveLength(3);
await request(client).post(`/api/issues/${seeded.issueId}/queued-comments/interrupt`).send(body).expect(409);
});
it.each([
["other", "running"], ["other", "queued"], ["other", "scheduled_retry"],
["same", "running"], ["same", "queued"], ["same", "scheduled_retry"],
] as const)("scopes interrupted queue successors to its agent: %s agent %s", async (owner, status) => {
const seeded = await seedQueue();
await db.update(agents).set({ adapterType: "claude_local",
runtimeConfig: { heartbeat: { maxConcurrentRuns: 1 } },
}).where(eq(agents.id, seeded.agentId));
await db.update(heartbeatRuns).set({ runtimeMode: "legacy", status: "succeeded",
finishedAt: new Date("2026-08-22T15:03:00.000Z"),
}).where(eq(heartbeatRuns.id, seeded.runId));
await db.update(issues).set({ executionRunId: null }).where(eq(issues.id, seeded.issueId));
const [wake] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
await db.update(agentWakeupRequests).set({ payload: { ...wake.payload,
queuedCommentInterrupt: { actorId: "other-operator", requestedAt: new Date().toISOString() },
} }).where(eq(agentWakeupRequests.id, seeded.wakeId));
// Keep this agent at capacity so successful delivery queues a successor
// without launching a provider. The independent run shares only the task.
await db.insert(heartbeatRuns).values({ companyId: seeded.companyId, agentId: seeded.agentId,
status: "running", contextSnapshot: { issueId: randomUUID() },
});
const successorAgentId = owner === "same" ? seeded.agentId : randomUUID();
if (owner === "other") await db.insert(agents).values({ id: successorAgentId,
companyId: seeded.companyId, name: "Independent agent", role: "engineer",
status: "idle", adapterType: "claude_local",
});
const [existingRun] = await db.insert(heartbeatRuns).values({ companyId: seeded.companyId, agentId: successorAgentId,
status, contextSnapshot: { issueId: seeded.issueId },
}).returning();
await heartbeatService(db).resumeQueuedCommentInterrupt(seeded.companyId, seeded.wakeId, { retryCleanup: true });
if (owner === "other" && status !== "scheduled_retry") {
// Another agent is not this queue's successor, but ordinary admission
// must still preserve the task execution lock until its work stops.
const [waiting] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
const [task] = await db.select().from(issues).where(eq(issues.id, seeded.issueId));
expect(waiting.status).toBe("deferred_issue_execution");
expect(task.executionRunId).toBe(existingRun.id);
expect(waiting.payload?.queuedCommentInterrupt).toMatchObject({ actorId: "other-operator" });
await db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date() })
.where(eq(heartbeatRuns.id, existingRun.id));
await heartbeatService(db).resumeQueuedCommentInterrupt(seeded.companyId, seeded.wakeId, { retryCleanup: true });
}
const [receipt] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(receipt.status).toBe(owner === "same" ? "deferred_issue_execution" : "coalesced");
if (owner === "other") {
const [successor] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, receipt.runId!));
expect(successor).toMatchObject({ agentId: seeded.agentId, status: "queued", responsibleUserId: "other-operator" });
expect(successor.contextSnapshot?.wakeCommentIds).toEqual(seeded.commentIds);
}
await heartbeatService(db).resumeQueuedCommentInterrupt(seeded.companyId, seeded.wakeId, { retryCleanup: true });
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, seeded.companyId)))
.toHaveLength(owner === "same" ? 3 : 4);
});
it.each(["user", "system"])("keeps stopped-run interruption intent on a %s receipt across restart until the process stops, then delivers once", async (actorType) => {
const seeded = await seedQueue();
await db.update(agentWakeupRequests).set({ requestedByActorType: actorType })
.where(eq(agentWakeupRequests.id, seeded.wakeId));
await db.update(agents).set({ adapterType: "claude_local",
runtimeConfig: { heartbeat: { maxConcurrentRuns: 1 } },
}).where(eq(agents.id, seeded.agentId));
await db.update(heartbeatRuns).set({ runtimeMode: "legacy", status: "failed",
processPid: process.pid, errorCode: "process_lost",
finishedAt: new Date("2026-08-22T15:03:00.000Z"),
}).where(eq(heartbeatRuns.id, seeded.runId));
await db.update(issues).set({ executionRunId: null }).where(eq(issues.id, seeded.issueId));
await db.insert(issueRecoveryActions).values({ companyId: seeded.companyId, sourceIssueId: seeded.issueId,
kind: "active_run_watchdog", cause: "legacy_execution_requires_reconciliation", fingerprint: seeded.runId,
status: "resolved", outcome: "blocked", nextAction: "Automatic recovery stopped.",
evidence: { runId: seeded.runId, automaticRecovery: { replay: "blocked", actionOutcome: "unknown" } },
});
await db.insert(heartbeatRuns).values({ companyId: seeded.companyId, agentId: seeded.agentId,
status: "running", contextSnapshot: { issueId: randomUUID() },
});
const client = app(seeded.companyId, "other-operator");
const queue = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
await request(client).post(`/api/issues/${seeded.issueId}/queued-comments/interrupt`).send({
queueId: seeded.wakeId, revision: queue.body.revision, targetRunId: null,
}).expect(200);
const [waiting] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(waiting.status).toBe("deferred_issue_execution");
expect(waiting.payload?.queuedCommentInterrupt).toMatchObject({ actorId: "other-operator" });
expect(waiting.payload?.executionWait).toMatchObject({ reason: "process_running" });
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, seeded.companyId))).toHaveLength(2);
await db.update(heartbeatRuns).set({ processPid: 999999999 }).where(eq(heartbeatRuns.id, seeded.runId));
await db.update(agentWakeupRequests).set({ updatedAt: new Date(0) }).where(eq(agentWakeupRequests.id, seeded.wakeId));
// New service instances have no memory of the HTTP request. Concurrent
// periodic workers must consume its durable receipt exactly once.
await Promise.all([heartbeatService(db).resumeQueuedRuns(), heartbeatService(db).resumeQueuedRuns()]);
const [delivered] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(delivered.status).toBe("coalesced");
const [successor] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, delivered.runId!));
expect(successor.contextSnapshot).toMatchObject({ wakeCommentIds: seeded.commentIds,
previousRunId: seeded.runId, forceFreshSession: true });
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, seeded.companyId))).toHaveLength(3);
});
it.each(["user", "system", "manual_receipt_other_actor", "rejected_admission", "consumed_first", "consumed_last", "deleted_first", "agent_first", "cancelled_queued"])("delivers saved user messages on a %s queue after automatic recovery stopped, without another click", async (actorType) => {
const seeded = await seedQueue();
await db.update(agentWakeupRequests).set({ requestedByActorType: actorType === "user" ? "user" : "system" })
.where(eq(agentWakeupRequests.id, seeded.wakeId));
if (actorType === "manual_receipt_other_actor") {
const [saved] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
await db.update(agentWakeupRequests).set({ requestedByActorType: "user", requestedByActorId: "other-operator",
payload: { ...saved.payload, manualUserWake: true },
}).where(eq(agentWakeupRequests.id, seeded.wakeId));
}
await db.update(agents).set({ adapterType: "claude_local",
runtimeConfig: { heartbeat: { maxConcurrentRuns: 1 } },
}).where(eq(agents.id, seeded.agentId));
await db.update(heartbeatRuns).set({ runtimeMode: "legacy", status: "failed",
processPid: process.pid, errorCode: "process_lost", finishedAt: new Date("2026-08-22T15:03:00.000Z"),
}).where(eq(heartbeatRuns.id, seeded.runId));
await db.update(issues).set({ executionRunId: null }).where(eq(issues.id, seeded.issueId));
await db.insert(issueRecoveryActions).values({ companyId: seeded.companyId, sourceIssueId: seeded.issueId,
kind: "active_run_watchdog", cause: "legacy_execution_requires_reconciliation", fingerprint: seeded.runId,
status: "resolved", outcome: "blocked", nextAction: "Automatic recovery stopped.",
evidence: { runId: seeded.runId, automaticRecovery: { replay: "blocked", actionOutcome: "unknown" } },
});
// Leave the agent's only slot occupied on another task so dispatch stays queued.
await db.insert(heartbeatRuns).values({ companyId: seeded.companyId, agentId: seeded.agentId,
status: "running", contextSnapshot: { issueId: randomUUID() },
});
await heartbeatService(db).resumeQueuedRuns();
expect((await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId)))[0].status)
.toBe("deferred_issue_execution");
await db.update(heartbeatRuns).set({ processPid: 999999999 }).where(eq(heartbeatRuns.id, seeded.runId));
const excludedIndex = actorType === "consumed_last" ? 1 : 0;
const filtersInput = ["consumed_first", "consumed_last", "deleted_first", "agent_first"].includes(actorType);
if (actorType.startsWith("consumed_")) await db.update(heartbeatRuns).set({
contextSnapshot: { issueId: seeded.issueId, wakeCommentIds: [seeded.commentIds[excludedIndex]] },
}).where(eq(heartbeatRuns.id, seeded.runId));
if (actorType === "deleted_first") await db.update(issueComments).set({ deletedAt: new Date() })
.where(eq(issueComments.id, seeded.commentIds[0]));
if (actorType === "agent_first") await db.update(issueComments).set({
authorType: "agent", authorUserId: null, authorAgentId: seeded.agentId,
}).where(eq(issueComments.id, seeded.commentIds[0]));
const expectedIds = seeded.commentIds.filter((_, index) => !filtersInput || index !== excludedIndex);
if (actorType === "cancelled_queued") await db.insert(heartbeatRuns).values({
companyId: seeded.companyId, agentId: seeded.agentId, status: "cancelled", runtimeMode: "legacy",
errorCode: "agent_paused", createdAt: new Date(0), finishedAt: new Date(1),
contextSnapshot: { issueId: seeded.issueId, wakeCommentIds: [seeded.commentIds[0]] },
});
if (actorType === "rejected_admission") {
await db.insert(heartbeatRuns).values({ companyId: seeded.companyId, agentId: seeded.agentId,
status: "cancelled", runtimeMode: "legacy", errorCode: "execution_reconciliation_required",
contextSnapshot: { issueId: seeded.issueId }, finishedAt: new Date(),
});
}
await Promise.all([heartbeatService(db).resumeQueuedRuns(), heartbeatService(db).resumeQueuedRuns()]);
const [delivered] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(delivered.status).toBe("coalesced");
const [successor] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, delivered.runId!));
expect(successor.contextSnapshot).toMatchObject({ wakeCommentIds: expectedIds,
previousRunId: seeded.runId, forceFreshSession: true });
if (actorType === "manual_receipt_other_actor") {
const [dispatch] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, successor.wakeupRequestId!));
expect(dispatch.requestedByActorId).toBe("queue-owner");
expect(dispatch.payload?.manualUserWake).toBeUndefined();
expect(successor.responsibleUserId).toBe("queue-owner");
}
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, seeded.companyId))).toHaveLength(["rejected_admission", "cancelled_queued"].includes(actorType) ? 4 : 3);
const [recovery] = await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.sourceIssueId, seeded.issueId));
expect(recovery.evidence.automaticRecovery).toMatchObject({ actionOutcome: "unknown" });
});
it("recovers a message deferred after legacy finalization released the task lock", async () => {
const seeded = await seedQueue();
await db.update(agents).set({ adapterType: "claude_local",
runtimeConfig: { heartbeat: { maxConcurrentRuns: 1 } },
}).where(eq(agents.id, seeded.agentId));
await db.update(heartbeatRuns).set({ runtimeMode: "legacy", status: "succeeded",
finishedAt: new Date("2026-08-22T15:03:00.000Z"),
}).where(eq(heartbeatRuns.id, seeded.runId));
await db.update(issues).set({ executionRunId: null }).where(eq(issues.id, seeded.issueId));
await db.insert(heartbeatRuns).values({ companyId: seeded.companyId, agentId: seeded.agentId,
status: "running", contextSnapshot: { issueId: randomUUID() },
});
await heartbeatService(db).resumeQueuedRuns();
const [wake] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(wake.status).toBe("queued");
const [successor] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, wake.runId!));
expect(successor.contextSnapshot?.wakeCommentIds).toEqual(seeded.commentIds);
await heartbeatService(db).resumeQueuedRuns();
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.companyId, seeded.companyId))).toHaveLength(3);
});
async function promoteQueue(seeded: Awaited<ReturnType<typeof seedQueue>>) {
const queueRunId = randomUUID();
const wake = await db
.select({ payload: agentWakeupRequests.payload })
.from(agentWakeupRequests)
.where(eq(agentWakeupRequests.id, seeded.wakeId))
.then((rows) => rows[0]);
const wakeContext = (wake?.payload as any)?._paperclipWakeContext ?? {};
await db
.update(heartbeatRuns)
.set({ status: "succeeded", finishedAt: new Date("2026-08-22T15:05:00.000Z") })
.where(eq(heartbeatRuns.id, seeded.runId));
await db.insert(heartbeatRuns).values({
id: queueRunId,
companyId: seeded.companyId,
agentId: seeded.agentId,
invocationSource: "issue_comment",
triggerDetail: "queue route promotion test",
status: "queued",
runtimeMode: "native",
wakeupRequestId: seeded.wakeId,
contextSnapshot: {
issueId: seeded.issueId,
wakeReason: "issue_reopened_via_comment",
...wakeContext,
},
});
await db
.update(agentWakeupRequests)
.set({ status: "queued", runId: queueRunId })
.where(eq(agentWakeupRequests.id, seeded.wakeId));
await db
.update(issues)
.set({ executionRunId: queueRunId })
.where(eq(issues.id, seeded.issueId));
return queueRunId;
}
it("projects the recovery wait reason only while the message is deferred", async () => {
const seeded = await seedQueue();
const executionWait = { reason: "remote_cleanup", message: "Waiting for the previous environment to stop." };
await db.update(agentWakeupRequests).set({
payload: sql`coalesce(${agentWakeupRequests.payload}, '{}'::jsonb) || ${JSON.stringify({ executionWait })}::jsonb`,
}).where(eq(agentWakeupRequests.id, seeded.wakeId));
const waiting = await request(app(seeded.companyId)).get(`/api/issues/${seeded.issueId}/queued-comments`);
expect(waiting.status).toBe(200);
expect(waiting.body.executionWait).toEqual(executionWait);
await promoteQueue(seeded);
const admitted = await request(app(seeded.companyId)).get(`/api/issues/${seeded.issueId}/queued-comments`);
expect(admitted.status).toBe(200);
expect(admitted.body.executionWait).toBeUndefined();
});
it("returns the authoritative order, preserves full Markdown edits, and rejects stale revisions", async () => {
const seeded = await seedQueue();
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
expect(initial.status, JSON.stringify(initial.body)).toBe(200);
expect(initial.body).toMatchObject({
issueId: seeded.issueId,
queueId: seeded.wakeId,
state: "deferred",
targetRunId: seeded.runId,
protocol: "paperclip_runner_v1",
entries: [
{ position: 0, canEdit: true, canDiscard: true, comment: { id: seeded.commentIds[0] } },
{ position: 1, canEdit: true, canDiscard: true, comment: { id: seeded.commentIds[1] } },
],
});
const markdown = " Keep **all** Markdown. \n";
const edited = await request(app(seeded.companyId))
.patch(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: initial.body.revision, body: markdown });
expect(edited.status, JSON.stringify(edited.body)).toBe(200);
expect(edited.body.entries[0].comment.body).toBe(markdown);
expect(edited.body.revision).not.toBe(initial.body.revision);
const stored = await db.select({ body: issueComments.body })
.from(issueComments)
.where(eq(issueComments.id, seeded.commentIds[0]))
.then((rows) => rows[0]);
expect(stored?.body).toBe(markdown);
const stale = await request(app(seeded.companyId))
.patch(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: initial.body.revision, body: "stale" });
expect(stale.status).toBe(409);
expect(stale.body.details?.code).toBe("queued_comment_revision_conflict");
});
it("writes one activity log row for each successful queue mutation, and none for a rejected one", async () => {
const seeded = await seedQueue();
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
const edited = await request(app(seeded.companyId))
.patch(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: initial.body.revision, body: "edited body" });
expect(edited.status, JSON.stringify(edited.body)).toBe(200);
const staleEdit = await request(app(seeded.companyId))
.patch(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: initial.body.revision, body: "stale" });
expect(staleEdit.status).toBe(409);
const editRows = await db
.select({ details: activityLog.details })
.from(activityLog)
.where(eq(activityLog.action, "issue.queued_comment_edited"));
expect(editRows).toHaveLength(1);
expect(editRows[0]?.details).toMatchObject({
commentId: seeded.commentIds[0],
queueId: seeded.wakeId,
revision: edited.body.revision,
});
const reordered = await request(app(seeded.companyId))
.put(`/api/issues/${seeded.issueId}/queued-comments/order`)
.send({
queueId: seeded.wakeId,
revision: edited.body.revision,
orderedCommentIds: [...seeded.commentIds].reverse(),
});
expect(reordered.status, JSON.stringify(reordered.body)).toBe(200);
const reorderRow = await db
.select({ details: activityLog.details })
.from(activityLog)
.where(eq(activityLog.action, "issue.queued_comments_reordered"))
.then((rows) => rows[0]);
expect(reorderRow?.details).toMatchObject({
queueId: seeded.wakeId,
revision: reordered.body.revision,
orderedCommentIds: [...seeded.commentIds].reverse(),
});
const discarded = await request(app(seeded.companyId))
.delete(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[1]}`)
.send({ queueId: seeded.wakeId, revision: reordered.body.revision });
expect(discarded.status, JSON.stringify(discarded.body)).toBe(200);
const discardRow = await db
.select({ details: activityLog.details })
.from(activityLog)
.where(eq(activityLog.action, "issue.queued_comment_discarded"))
.then((rows) => rows[0]);
expect(discardRow?.details).toMatchObject({
commentId: seeded.commentIds[1],
queueId: seeded.wakeId,
revision: discarded.body.revision,
cancelledRunId: null,
});
});
it("preserves reordered messages across promotion and cancels the queued run after final trash", async () => {
const seeded = await seedQueue();
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
const reordered = await request(app(seeded.companyId))
.put(`/api/issues/${seeded.issueId}/queued-comments/order`)
.send({
queueId: seeded.wakeId,
revision: initial.body.revision,
orderedCommentIds: [...seeded.commentIds].reverse(),
});
expect(reordered.status, JSON.stringify(reordered.body)).toBe(200);
expect(reordered.body.entries.map((entry: any) => entry.comment.id)).toEqual([...seeded.commentIds].reverse());
const queueRunId = await promoteQueue(seeded);
const promoted = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
expect(promoted.body).toMatchObject({
queueId: seeded.wakeId,
state: "queued",
targetRunId: null,
revision: reordered.body.revision,
});
const afterFirstTrash = await request(app(seeded.companyId))
.delete(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[1]}`)
.send({ queueId: seeded.wakeId, revision: promoted.body.revision });
expect(afterFirstTrash.status, JSON.stringify(afterFirstTrash.body)).toBe(200);
expect(afterFirstTrash.body.entries.map((entry: any) => entry.comment.id)).toEqual([seeded.commentIds[0]]);
const queuedRunAfterFirstTrash = await db
.select({ contextSnapshot: heartbeatRuns.contextSnapshot })
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, queueRunId))
.then((rows) => rows[0]);
expect((queuedRunAfterFirstTrash?.contextSnapshot as any)?.wakeCommentIds).toEqual([
seeded.commentIds[0],
]);
const emptied = await request(app(seeded.companyId))
.delete(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: afterFirstTrash.body.revision });
expect(emptied.status, JSON.stringify(emptied.body)).toBe(200);
expect(emptied.body.entries).toEqual([]);
const wake = await db.select({ status: agentWakeupRequests.status })
.from(agentWakeupRequests)
.where(eq(agentWakeupRequests.id, seeded.wakeId))
.then((rows) => rows[0]);
expect(wake?.status).toBe("cancelled");
const [queueRun, storedIssue] = await Promise.all([
db.select({ status: heartbeatRuns.status })
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, queueRunId))
.then((rows) => rows[0]),
db.select({ executionRunId: issues.executionRunId })
.from(issues)
.where(eq(issues.id, seeded.issueId))
.then((rows) => rows[0]),
]);
expect(queueRun?.status).toBe("cancelled");
expect(storedIssue?.executionRunId).toBeNull();
// Two discards happen in this scenario (the first trash, then the final
// one that empties the queue), so match the row by its own commentId
// instead of assuming insertion order.
const discardRows = await db
.select({ details: activityLog.details })
.from(activityLog)
.where(eq(activityLog.action, "issue.queued_comment_discarded"));
const finalDiscardRow = discardRows.find(
(row) => (row.details as { commentId?: string } | null)?.commentId === seeded.commentIds[0],
);
expect(finalDiscardRow?.details).toMatchObject({
commentId: seeded.commentIds[0],
cancelledRunId: queueRunId,
});
});
it("keeps a mutation response's steering disposition in step with a fresh GET after promotion", async () => {
const seeded = await seedQueue();
await promoteQueue(seeded);
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
expect(initial.body.steeringDisposition).toBe("temporarily_unavailable");
const edited = await request(app(seeded.companyId))
.patch(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: initial.body.revision, body: "edited during promotion" });
expect(edited.status, JSON.stringify(edited.body)).toBe(200);
const afterEdit = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
expect(edited.body.steeringDisposition).toBe(afterEdit.body.steeringDisposition);
const reordered = await request(app(seeded.companyId))
.put(`/api/issues/${seeded.issueId}/queued-comments/order`)
.send({
queueId: seeded.wakeId,
revision: edited.body.revision,
orderedCommentIds: [...seeded.commentIds].reverse(),
});
expect(reordered.status, JSON.stringify(reordered.body)).toBe(200);
const afterReorder = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
expect(reordered.body.steeringDisposition).toBe(afterReorder.body.steeringDisposition);
});
it("returns entry objects with the same keys as the GET endpoint", async () => {
const seeded = await seedQueue();
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
const edited = await request(app(seeded.companyId))
.patch(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: initial.body.revision, body: "edited body" });
expect(edited.status, JSON.stringify(edited.body)).toBe(200);
expect(Object.keys(edited.body.entries[0]).sort()).toEqual(Object.keys(initial.body.entries[0]).sort());
});
it("cancels the deferred wake when the final message is discarded before promotion", async () => {
const seeded = await seedQueue();
await db.delete(issueComments).where(eq(issueComments.id, seeded.commentIds[1]));
await db
.update(agentWakeupRequests)
.set({
payload: {
issueId: seeded.issueId,
commentId: seeded.commentIds[0],
_paperclipWakeContext: {
commentId: seeded.commentIds[0],
wakeCommentId: seeded.commentIds[0],
wakeCommentIds: [seeded.commentIds[0]],
},
},
})
.where(eq(agentWakeupRequests.id, seeded.wakeId));
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
const discarded = await request(app(seeded.companyId))
.delete(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: initial.body.revision });
expect(discarded.status, JSON.stringify(discarded.body)).toBe(200);
const [wake, runs] = await Promise.all([
db.select({ status: agentWakeupRequests.status })
.from(agentWakeupRequests)
.where(eq(agentWakeupRequests.id, seeded.wakeId))
.then((rows) => rows[0]),
db.select({ id: heartbeatRuns.id }).from(heartbeatRuns),
]);
expect(wake?.status).toBe("cancelled");
expect(runs.map((run) => run.id)).toEqual([seeded.runId]);
});
it("routes legacy queued-comment cancellation through the promoted queue", async () => {
const seeded = await seedQueue();
const queueRunId = await promoteQueue(seeded);
const cancelled = await request(app(seeded.companyId))
.delete(`/api/issues/${seeded.issueId}/comments/${seeded.commentIds[0]}?mode=cancel`);
expect(cancelled.status, JSON.stringify(cancelled.body)).toBe(200);
expect(cancelled.body.id).toBe(seeded.commentIds[0]);
const [wake, queueRun] = await Promise.all([
db.select({ payload: agentWakeupRequests.payload })
.from(agentWakeupRequests)
.where(eq(agentWakeupRequests.id, seeded.wakeId))
.then((rows) => rows[0]),
db.select({ contextSnapshot: heartbeatRuns.contextSnapshot })
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, queueRunId))
.then((rows) => rows[0]),
]);
expect((wake?.payload as any)?._paperclipWakeContext?.wakeCommentIds).toEqual([
seeded.commentIds[1],
]);
expect((queueRun?.contextSnapshot as any)?.wakeCommentIds).toEqual([
seeded.commentIds[1],
]);
});
it("reports an explicit conflict after queued-run dispatch has begun", async () => {
const seeded = await seedQueue();
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
const queueRunId = await promoteQueue(seeded);
await db
.update(agentWakeupRequests)
.set({ status: "claimed", claimedAt: new Date() })
.where(eq(agentWakeupRequests.id, seeded.wakeId));
await db
.update(heartbeatRuns)
.set({ status: "running", startedAt: new Date() })
.where(eq(heartbeatRuns.id, queueRunId));
const discard = await request(app(seeded.companyId))
.delete(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: initial.body.revision });
expect(discard.status).toBe(409);
expect(discard.body.details?.code).toBe("queued_comment_already_dispatching");
const comment = await db
.select({ id: issueComments.id })
.from(issueComments)
.where(eq(issueComments.id, seeded.commentIds[0]))
.then((rows) => rows[0]);
expect(comment?.id).toBe(seeded.commentIds[0]);
});
it("limits edit and trash to the comment owner", async () => {
const seeded = await seedQueue();
const initial = await request(app(seeded.companyId, "other-operator"))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
expect(initial.status).toBe(200);
expect(initial.body.entries[0]).toMatchObject({ canEdit: false, canDiscard: false });
const edit = await request(app(seeded.companyId, "other-operator"))
.patch(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: initial.body.revision, body: "not mine" });
expect(edit.status).toBe(403);
const discard = await request(app(seeded.companyId, "other-operator"))
.delete(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: initial.body.revision });
expect(discard.status).toBe(403);
});
it("leaves the selected row queued when no native steering session is attached", async () => {
const seeded = await seedQueue();
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
expect(initial.body.steeringDisposition).toBe("temporarily_unavailable");
const steered = await request(app(seeded.companyId))
.post(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}/steer`)
.send({ queueId: seeded.wakeId, targetRunId: seeded.runId, revision: initial.body.revision });
expect(steered.status).toBe(409);
expect(steered.body.details).toMatchObject({
code: "steering_temporarily_unavailable",
retryable: true,
});
const queueAfterFailure = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
expect(queueAfterFailure.body.entries.map((entry: any) => entry.comment.id)).toEqual(seeded.commentIds);
});
it("returns the persisted acknowledgement when the final steering response is retried", async () => {
const seeded = await seedQueue();
await db.delete(issueComments).where(eq(issueComments.id, seeded.commentIds[1]));
await db
.update(agentWakeupRequests)
.set({
status: "cancelled",
finishedAt: new Date("2026-08-22T15:04:00.000Z"),
payload: {
issueId: seeded.issueId,
commentId: seeded.commentIds[0],
_paperclipWakeContext: {
commentId: seeded.commentIds[0],
wakeCommentId: seeded.commentIds[0],
wakeCommentIds: [seeded.commentIds[0]],
},
},
})
.where(eq(agentWakeupRequests.id, seeded.wakeId));
await db
.update(heartbeatRuns)
.set({
resultJson: {
queuedSteeringAcknowledgements: {
[seeded.commentIds[0]]: {
status: "acknowledged",
queueId: seeded.wakeId,
turnId: "turn-acknowledged",
acknowledgedAt: "2026-08-22T15:04:00.000Z",
},
},
},
})
.where(eq(heartbeatRuns.id, seeded.runId));
const retried = await request(app(seeded.companyId))
.post(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}/steer`)
.send({
queueId: seeded.wakeId,
targetRunId: seeded.runId,
revision: "response-was-lost-before-the-client-stored-the-revision",
});
expect(retried.status, JSON.stringify(retried.body)).toBe(200);
expect(retried.body).toMatchObject({
issueId: seeded.issueId,
queueId: null,
state: null,
entries: [],
});
const activity = await db
.select({ details: activityLog.details })
.from(activityLog)
.where(eq(activityLog.action, "issue.queued_comment_steered"))
.then((rows) => rows[0]);
expect(activity?.details).toMatchObject({
commentId: seeded.commentIds[0],
targetRunId: seeded.runId,
turnId: "turn-acknowledged",
duplicate: true,
});
});
it("does not reuse an acknowledgement from a different queue", async () => {
const seeded = await seedQueue();
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
await db
.update(heartbeatRuns)
.set({
resultJson: {
queuedSteeringAcknowledgements: {
[seeded.commentIds[0]]: {
status: "acknowledged",
queueId: randomUUID(),
turnId: "turn-from-another-queue",
acknowledgedAt: "2026-08-22T15:04:00.000Z",
},
},
},
})
.where(eq(heartbeatRuns.id, seeded.runId));
const steered = await request(app(seeded.companyId))
.post(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}/steer`)
.send({
queueId: seeded.wakeId,
targetRunId: seeded.runId,
revision: initial.body.revision,
});
expect(steered.status).toBe(409);
expect(steered.body.details).toMatchObject({
code: "steering_temporarily_unavailable",
retryable: true,
});
const queueAfterFailure = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
expect(queueAfterFailure.body.entries.map((entry: any) => entry.comment.id))
.toEqual(seeded.commentIds);
});
it("keeps queue edits available during handoff but rejects stale same-turn steering", async () => {
const seeded = await seedQueue();
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
await db
.update(heartbeatRuns)
.set({ status: "succeeded", finishedAt: new Date("2026-08-22T15:05:00.000Z") })
.where(eq(heartbeatRuns.id, seeded.runId));
const edit = await request(app(seeded.companyId))
.patch(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({
queueId: seeded.wakeId,
revision: initial.body.revision,
body: "edited during handoff",
});
expect(edit.status, JSON.stringify(edit.body)).toBe(200);
const stored = await db
.select({ body: issueComments.body })
.from(issueComments)
.where(eq(issueComments.id, seeded.commentIds[0]))
.then((rows) => rows[0]);
expect(stored?.body).toBe("edited during handoff");
const steer = await request(app(seeded.companyId))
.post(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}/steer`)
.send({
queueId: seeded.wakeId,
targetRunId: seeded.runId,
revision: edit.body.revision,
});
expect(steer.status).toBe(409);
expect(steer.body.details?.code).toBe("queued_comment_stale_target");
const wake = await db
.select({ status: agentWakeupRequests.status, payload: agentWakeupRequests.payload })
.from(agentWakeupRequests)
.where(eq(agentWakeupRequests.id, seeded.wakeId))
.then((rows) => rows[0]);
expect(wake?.status).toBe("deferred_issue_execution");
expect((wake?.payload as any)?._paperclipWakeContext?.wakeCommentIds).toEqual(seeded.commentIds);
});
it("cancels a queued continuation whose comments disappeared before claim", async () => {
const seeded = await seedQueue();
const queueRunId = await promoteQueue(seeded);
await db.delete(issueComments).where(eq(issueComments.issueId, seeded.issueId));
const heartbeat = heartbeatService(db, { runtimeEnv: {} });
await heartbeat.resumeQueuedRuns();
const [queueRun, wake] = await Promise.all([
db.select({ status: heartbeatRuns.status, errorCode: heartbeatRuns.errorCode })
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, queueRunId))
.then((rows) => rows[0]),
db.select({ status: agentWakeupRequests.status })
.from(agentWakeupRequests)
.where(eq(agentWakeupRequests.id, seeded.wakeId))
.then((rows) => rows[0]),
]);
expect(queueRun).toMatchObject({
status: "cancelled",
errorCode: "queued_comment_discarded",
});
expect(wake?.status).toBe("cancelled");
});
it("keeps a persisted legacy queue on the legacy protocol after the agent changes adapters", async () => {
const seeded = await seedQueue();
const queueRunId = await promoteQueue(seeded);
await db
.update(heartbeatRuns)
.set({ runtimeMode: "legacy" })
.where(eq(heartbeatRuns.id, queueRunId));
const queued = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
expect(queued.status, JSON.stringify(queued.body)).toBe(200);
expect(queued.body).toMatchObject({
queueId: seeded.wakeId,
state: "queued",
targetRunId: null,
protocol: "legacy",
steeringDisposition: "unsupported",
});
});
it.each(["paperclip_runner", "codex_local"])("keeps Steer during native preparation even after agent settings change to %s", async adapterType => {
const seeded = await seedQueue();
const steeringCalls = steerNativeSessionMock.mock.calls.length;
await db.update(heartbeatRuns).set({ runtimeMode: "legacy", runtimeModeResolvedAt: null,
runnerProfileJson: { adapterDispatch: { adapterType: "paperclip_runner" } },
}).where(eq(heartbeatRuns.id, seeded.runId));
await db.update(agents).set({ adapterType }).where(eq(agents.id, seeded.agentId));
const client = app(seeded.companyId);
const initial = await request(client).get(`/api/issues/${seeded.issueId}/queued-comments`).expect(200);
expect(initial.body).toMatchObject({ protocol: "paperclip_runner_v1", steeringDisposition: "temporarily_unavailable" });
const edited = await request(client).patch(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: initial.body.revision, body: "Updated direction" }).expect(200);
expect(edited.body).toMatchObject({ protocol: "paperclip_runner_v1", steeringDisposition: "temporarily_unavailable" });
expect(steerNativeSessionMock.mock.calls).toHaveLength(steeringCalls);
});
it("serializes discard against queued-run claim", async () => {
const seeded = await seedQueue();
await db.delete(issueComments).where(eq(issueComments.id, seeded.commentIds[1]));
await db
.update(agentWakeupRequests)
.set({
payload: {
issueId: seeded.issueId,
commentId: seeded.commentIds[0],
_paperclipWakeContext: {
commentId: seeded.commentIds[0],
wakeCommentId: seeded.commentIds[0],
wakeCommentIds: [seeded.commentIds[0]],
},
},
})
.where(eq(agentWakeupRequests.id, seeded.wakeId));
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
const queueRunId = await promoteQueue(seeded);
await db
.update(agents)
.set({
adapterType: "process",
adapterConfig: {
command: process.execPath,
args: ["-e", "process.exit(0)"],
},
})
.where(eq(agents.id, seeded.agentId));
await db
.update(heartbeatRuns)
.set({ runtimeMode: "legacy" })
.where(eq(heartbeatRuns.id, queueRunId));
const heartbeat = heartbeatService(db, { runtimeEnv: {} });
const [discard] = await Promise.all([
request(app(seeded.companyId))
.delete(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}`)
.send({ queueId: seeded.wakeId, revision: initial.body.revision }),
heartbeat.resumeQueuedRuns(),
]);
const queueRun = await db
.select({ status: heartbeatRuns.status })
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, queueRunId))
.then((rows) => rows[0]);
if (discard.status === 200) {
expect(queueRun?.status).toBe("cancelled");
} else {
expect(discard.status, JSON.stringify(discard.body)).toBe(409);
expect(discard.body.details?.code).toBe("queued_comment_already_dispatching");
expect(queueRun?.status).not.toBe("queued");
}
await heartbeat.drainActiveRunExecutions();
}, 30_000);
async function seedDispatchIdentity(seeded: Awaited<ReturnType<typeof seedQueue>>) {
const [identity] = await db.insert(runIdentityContexts).values({
companyId: seeded.companyId,
runId: seeded.runId,
revision: 1,
cause: "dispatch",
correlationId: "dispatch",
status: "accepted",
acceptedAt: new Date("2026-08-22T15:00:00.000Z"),
}).returning();
await db.update(heartbeatRuns)
.set({ activeIdentityContextId: identity!.id })
.where(eq(heartbeatRuns.id, seeded.runId));
return identity!;
}
it("writes the identity acceptance, the wake payload, the run acknowledgement, and the activity row on a successful steering transaction", async () => {
const seeded = await seedQueue();
await seedDispatchIdentity(seeded);
steerNativeSessionMock.mockResolvedValueOnce({ turnId: "turn-1" });
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
const steered = await request(app(seeded.companyId))
.post(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}/steer`)
.send({ queueId: seeded.wakeId, targetRunId: seeded.runId, revision: initial.body.revision });
expect(steered.status, JSON.stringify(steered.body)).toBe(200);
const steeringIdentity = await db
.select()
.from(runIdentityContexts)
.where(eq(runIdentityContexts.messageId, seeded.commentIds[0]))
.then((rows) => rows[0]);
expect(steeringIdentity).toMatchObject({
status: "accepted",
responsibleUserId: "queue-owner",
cause: "steering",
});
const wake = await db
.select({ payload: agentWakeupRequests.payload })
.from(agentWakeupRequests)
.where(eq(agentWakeupRequests.id, seeded.wakeId))
.then((rows) => rows[0]);
expect((wake?.payload as any)?._paperclipWakeContext?.wakeCommentIds).toEqual([seeded.commentIds[1]]);
const run = await db
.select({ resultJson: heartbeatRuns.resultJson, activeIdentityContextId: heartbeatRuns.activeIdentityContextId })
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, seeded.runId))
.then((rows) => rows[0]);
expect((run?.resultJson as any)?.queuedSteeringAcknowledgements?.[seeded.commentIds[0]]).toMatchObject({
status: "acknowledged",
queueId: seeded.wakeId,
turnId: "turn-1",
});
expect(run?.activeIdentityContextId).toBe(steeringIdentity!.id);
const activity = await db
.select({ details: activityLog.details })
.from(activityLog)
.where(eq(activityLog.action, "issue.queued_comment_steered"))
.then((rows) => rows[0]);
expect(activity?.details).toMatchObject({
commentId: seeded.commentIds[0],
targetRunId: seeded.runId,
turnId: "turn-1",
duplicate: false,
});
});
it("allows the runner to persist its acknowledgement while steering, preserving concurrent run updates", async () => {
const seeded = await seedQueue();
await seedDispatchIdentity(seeded);
const originalProfile = { adapterDispatch: { adapterType: "paperclip_runner" },
nativeExecutionInput: { provider: { kind: "codex", model: "gpt-5.6-sol", reasoningEffort: "medium" } } };
await db.update(heartbeatRuns).set({ runnerProfileJson: originalProfile }).where(eq(heartbeatRuns.id, seeded.runId));
await db.update(issues).set({ assigneeAdapterOverrides: { model: "gpt-6-sol", reasoningEffort: "xhigh" } })
.where(eq(issues.id, seeded.issueId));
steerNativeSessionMock.mockImplementationOnce(async () => {
await db.transaction(async tx => {
await tx.execute(sql`set local lock_timeout = '1000ms'`);
await appendHeartbeatRunEvent(tx as unknown as typeof db, {
companyId: seeded.companyId, agentId: seeded.agentId, runId: seeded.runId,
eventType: "item.completed", payload: { kind: "steering_acknowledgement" },
});
await tx.update(heartbeatRuns).set({ resultJson: { concurrentProviderReceipt: "preserve" } })
.where(eq(heartbeatRuns.id, seeded.runId));
});
return { turnId: "turn-before-model-change" };
});
const initial = await request(app(seeded.companyId)).get(`/api/issues/${seeded.issueId}/queued-comments`);
const steered = await request(app(seeded.companyId))
.post(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}/steer`)
.send({ queueId: seeded.wakeId, targetRunId: seeded.runId, revision: initial.body.revision });
expect(steered.status, JSON.stringify(steered.body)).toBe(200);
const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seeded.runId));
expect(run.status).toBe("running");
expect(run.runnerProfileJson).toEqual(originalProfile);
expect(steerNativeSessionMock).toHaveBeenCalledWith(expect.objectContaining({ runId: seeded.runId,
message: "First queued message", correlationId: seeded.commentIds[0] }));
expect(run.resultJson).toMatchObject({ concurrentProviderReceipt: "preserve", queuedSteeringAcknowledgements: {
[seeded.commentIds[0]]: { status: "acknowledged", turnId: "turn-before-model-change" },
} });
expect(steered.body.entries.map((entry: any) => entry.comment.id)).toEqual([seeded.commentIds[1]]);
});
it.each(["cancelled", "failed", "succeeded"] as const)("preserves queued input when the run stops during acknowledgement (%s)", async status => {
const seeded = await seedQueue();
await seedDispatchIdentity(seeded);
steerNativeSessionMock.mockImplementationOnce(async () => {
await db.update(heartbeatRuns).set({ status, finishedAt: new Date(), resultJson: { terminalReceipt: "preserve" } })
.where(eq(heartbeatRuns.id, seeded.runId));
return { turnId: "turn-that-stopped" };
});
const initial = await request(app(seeded.companyId)).get(`/api/issues/${seeded.issueId}/queued-comments`);
const response = await request(app(seeded.companyId))
.post(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}/steer`)
.send({ queueId: seeded.wakeId, targetRunId: seeded.runId, revision: initial.body.revision });
expect(response.status).toBe(409);
expect(response.body.details).toMatchObject({ code: "queued_comment_stale_target" });
const [wake] = await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.id, seeded.wakeId));
expect(wake.status).toBe("deferred_issue_execution");
expect((wake.payload as any)._paperclipWakeContext.wakeCommentIds).toEqual(seeded.commentIds);
const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, seeded.runId));
expect(run.resultJson).toEqual({ terminalReceipt: "preserve" });
const activity = await db.select().from(activityLog).where(eq(activityLog.action, "issue.queued_comment_steered"));
expect(activity).toHaveLength(0);
});
it("keeps the identity pending after a steering timeout, then reconciles it on a later acknowledgement", async () => {
const seeded = await seedQueue();
await seedDispatchIdentity(seeded);
steerNativeSessionMock.mockRejectedValueOnce(
new NativeSessionSteeringError("steering_timeout", "The provider did not acknowledge steering in time."),
);
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
const steered = await request(app(seeded.companyId))
.post(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}/steer`)
.send({ queueId: seeded.wakeId, targetRunId: seeded.runId, revision: initial.body.revision });
expect(steered.status).toBe(409);
expect(steered.body.details).toMatchObject({ code: "steering_timeout", retryable: true });
const pending = await db
.select()
.from(runIdentityContexts)
.where(eq(runIdentityContexts.messageId, seeded.commentIds[0]))
.then((rows) => rows[0]);
expect(pending?.status).toBe("pending");
await reconcileSteeredIdentity(db, pending!);
const reconciled = await db
.select({ status: runIdentityContexts.status })
.from(runIdentityContexts)
.where(eq(runIdentityContexts.id, pending!.id))
.then((rows) => rows[0]);
expect(reconciled?.status).toBe("accepted");
const run = await db
.select({ activeIdentityContextId: heartbeatRuns.activeIdentityContextId })
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, seeded.runId))
.then((rows) => rows[0]);
expect(run?.activeIdentityContextId).toBe(pending!.id);
});
it("sets the identity to rejected on a definite steering rejection", async () => {
const seeded = await seedQueue();
await seedDispatchIdentity(seeded);
steerNativeSessionMock.mockRejectedValueOnce(
new NativeSessionSteeringError("steering_rejected", "The provider rejected the steering message."),
);
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
const steered = await request(app(seeded.companyId))
.post(`/api/issues/${seeded.issueId}/queued-comments/${seeded.commentIds[0]}/steer`)
.send({ queueId: seeded.wakeId, targetRunId: seeded.runId, revision: initial.body.revision });
expect(steered.status).toBe(409);
expect(steered.body.details).toMatchObject({ code: "steering_rejected", retryable: true });
const rejected = await db
.select({ status: runIdentityContexts.status })
.from(runIdentityContexts)
.where(eq(runIdentityContexts.messageId, seeded.commentIds[0]))
.then((rows) => rows[0]);
expect(rejected?.status).toBe("rejected");
});
it("throws queued_comment_order_mismatch and changes no row for an invalid reorder set", async () => {
const seeded = await seedQueue();
const initial = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
const invalid = await request(app(seeded.companyId))
.put(`/api/issues/${seeded.issueId}/queued-comments/order`)
.send({
queueId: seeded.wakeId,
revision: initial.body.revision,
orderedCommentIds: [seeded.commentIds[0], randomUUID()],
});
expect(invalid.status, JSON.stringify(invalid.body)).toBe(409);
expect(invalid.body.details?.code).toBe("queued_comment_order_mismatch");
const wake = await db
.select({ payload: agentWakeupRequests.payload })
.from(agentWakeupRequests)
.where(eq(agentWakeupRequests.id, seeded.wakeId))
.then((rows) => rows[0]);
expect((wake?.payload as any)?._paperclipWakeContext?.wakeCommentIds).toEqual(seeded.commentIds);
const after = await request(app(seeded.companyId))
.get(`/api/issues/${seeded.issueId}/queued-comments`);
expect(after.body.entries.map((entry: any) => entry.comment.id)).toEqual(seeded.commentIds);
});
});