refactor(server): move the admission half of the deferred wake state machine into the wake-queue module (#13136)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The heartbeat service admits wake requests while an issue has an
active execution run.
> - That admission branch mixes wake policy, database reads, and
database writes in one service.
> - This structure makes the wake-queue boundary hard to test and
extend.
> - This pull request moves the admission policy and its database
adapter into the wake-queue module.
> - The result keeps heartbeat orchestration small and makes the
admission behavior testable in isolation.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

The heartbeat service now delegates deferred wake admission to the
wake-queue module. The module keeps the existing merge, defer, and
ordinary-wake outcomes.

**Subsystem affected**

server/ — REST API and orchestration services.

**Current behavior**

The heartbeat service contains a 146-line branch that reads wake state,
chooses an outcome, and writes the result.

**Proposed behavior**

The wake-queue module owns the pure admission decision and the adapter
reads and writes. The heartbeat service calls one module method.

**Reason and benefit**

This boundary reduces service coupling and lets module tests cover the
admission policy. The change keeps the existing reason strings and
outcomes.

**Breaking changes**

None. The change preserves the current behavior and public API.

**Additional context**

This pull request follows [PR
#13132](https://github.com/paperclipai/paperclip/pull/13132), which
merged the first slice of this refactor. I searched GitHub for duplicate
and related pull requests before opening this pull request.

## What Changed

- Move deferred wake admission policy into
`server/src/modules/wake-queue`.
- Add module ports and a PostgreSQL adapter for the admission reads and
writes.
- Keep the existing wake outcomes and stored reason strings.
- Extend the module boundary check to reject service imports from the
application layer.
- Add unit and adapter tests for the moved behavior.

## Verification

- `node --test scripts/check-module-boundaries.test.mjs` passes.
- The `server/src/modules/wake-queue` suite passes 58 tests.
- The eight pinned heartbeat and queued-comment tests remain unchanged
and require CI verification.
- Every continuous-integration check must reach a terminal green state
before merge.

## Risks

The refactor changes the location of wake admission logic. A missed
adapter condition could change deferred wake behavior. The tests cover
the policy outcomes and the adapter writes. The residual tenant-scope
risk remains documented in the review record.

## Model Used

OpenAI Codex, GPT-5, tool use and code execution. The implementation
author ran the tests and prepared the commit set.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` /
`Closes: #` / `Refs: #` OR (b) described the issue in-PR following the
relevant issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Nicky LeachandPaperclip authored and GitHub committed 2026-09-10 00:55:48 -07:00
1 parent 0bff1d5cb6
commit ae0c1fbbd5
11 files changed
+888 -136

No files matched your search

@@ -244,3 +244,160 @@ export interface RecoveryEscalationPort {
}
export type { PostCommitEffect, ReleaseOutcome };
/**
* Temporary port: `heartbeat.ts` still opens and owns the transaction that
* admits a wake behind an active issue execution; the module does not own
* that transaction yet. A later change will decompose `enqueueWakeup` so
* the module owns the transaction itself. That change removes this handle
* and replaces it with a transaction the module opens on its own.
*
* Only this module builds a scope. `createWakeQueue` exposes a method that
* builds one for a caller outside the module. That caller receives an
* opaque handle back and never touches this class directly.
*/
export class TransactionScope {
private constructor(
private readonly companyId: string,
private readonly rawTx: unknown,
) {}
static create(companyId: string, rawTx: unknown): TransactionScope {
return new TransactionScope(companyId, rawTx);
}
/** Returns the bound transaction only when `companyId` matches the scope's own company. */
requireTx(companyId: string): unknown {
if (companyId !== this.companyId) {
throw new Error(
"wake-queue: the transaction scope belongs to a different company than the requested write",
);
}
return this.rawTx;
}
}
/** Reads a scope's bound transaction. Rejects a missing scope with the same clear error as a mismatched one; never falls back to any other executor. */
export function requireTransactionScopeTx(
scope: TransactionScope | null | undefined,
companyId: string,
): unknown {
if (!scope) {
throw new Error("wake-queue: this call carries no transaction scope");
}
return scope.requireTx(companyId);
}
/** The active execution run a new wake arrives behind. */
export type WakeAdmissionActiveExecutionRun = {
id: string;
agentId: string;
status: string;
contextSnapshot: unknown;
};
export type ExistingDeferredWake = {
id: string;
payload: Record<string, unknown>;
/** `payload._paperclipWakeContext`, already parsed to a plain object. */
deferredContext: Record<string, unknown>;
coalescedCount: number | null;
};
/**
* The four wake-admission decision helpers that stay in `heartbeat.ts`
* today. The application layer receives them through this port so it never
* imports the service it is extracted from.
*/
export type WakeAdmissionHeartbeatHelpers = {
/** `filterZombieCoalesceTarget` in `heartbeat.ts`. */
filterZombieCoalesceTarget(
target: WakeAdmissionActiveExecutionRun | null,
liveRunExecutions: { has(id: string): boolean },
): WakeAdmissionActiveExecutionRun | null;
/** `mergeCoalescedContextSnapshot` in `heartbeat.ts`. */
mergeCoalescedContextSnapshot(
existingRaw: unknown,
incoming: Record<string, unknown>,
options?: { preserveExistingInteractionContinuation?: boolean },
): Record<string, unknown>;
/** `shouldDeferFollowupWakeForSameIssue` in `heartbeat.ts`. */
shouldDeferFollowupWakeForSameIssue(input: {
activeRunStatus: string | null | undefined;
isSameExecutionAgent: boolean;
wakeCommentId: string | null | undefined;
forceFreshSession: boolean;
}): boolean;
/** `shouldQueueFollowupForRunningIssueWake` in `heartbeat.ts`. */
shouldQueueFollowupForRunningIssueWake(input: {
contextSnapshot: Record<string, unknown> | null | undefined;
wakeCommentId: string | null;
}): boolean;
};
export type AdmitWakeBehindIssueExecutionResult =
| { kind: "proceed" }
| { kind: "coalesced"; run: Record<string, unknown> }
| { kind: "deferred" };
/** Read-only lookups the admission use case needs, each scoped to a company. */
export interface WakeAdmissionReader {
/** True when the active execution run's agent and this wake's own agent share an execution-agent-name key. */
isSameExecutionAgent(
scope: TransactionScope,
input: {
companyId: string;
activeExecutionRunAgentId: string;
issueExecutionAgentNameKey: string | null;
agentNameKey: string | null;
},
): Promise<boolean>;
findExistingDeferredWake(
scope: TransactionScope,
input: { companyId: string; agentId: string; issueId: string },
): Promise<ExistingDeferredWake | null>;
}
/** The transaction-scoped write operations that admit a wake behind an active issue execution. */
export interface WakeAdmissionWriter {
/** Merges the wake's context into the active execution run and records the wake as coalesced. Returns the updated run row. */
coalesceIntoActiveExecutionRun(
scope: TransactionScope,
input: {
companyId: string;
activeExecutionRunId: string;
mergedContextSnapshot: Record<string, unknown>;
agentId: string;
source: string;
triggerDetail: string | null;
payload: Record<string, unknown> | null;
requestedByActorType: string | null;
requestedByActorId: string | null;
idempotencyKey: string | null;
},
): Promise<Record<string, unknown>>;
/** Merges the wake's context into an already-queued deferred wake, guarded by its current status. */
mergeIntoExistingDeferredWake(
scope: TransactionScope,
input: {
companyId: string;
existingDeferredWakeId: string;
mergedPayload: Record<string, unknown>;
nextCoalescedCount: number;
},
): Promise<void>;
/** Queues a new deferred wake behind the active execution run. */
insertNewDeferredWake(
scope: TransactionScope,
input: {
companyId: string;
agentId: string;
source: string;
triggerDetail: string | null;
payload: Record<string, unknown>;
requestedByActorType: string | null;
requestedByActorId: string | null;
idempotencyKey: string | null;
},
): Promise<void>;
}
@@ -1,5 +1,6 @@
import { describe, expect, it, vi } from "vitest";
import { createReleaseIssueExecution } from "./use-cases.js";
import { createAdmitWakeBehindIssueExecution, createReleaseIssueExecution } from "./use-cases.js";
import type { AdmitWakeBehindIssueExecutionInput } from "./use-cases.js";
import { WakeQueueApplicationError } from "./types.js";
import type {
DeferredWakeCandidate,
@@ -10,6 +11,11 @@ import type {
RecoveryEscalationPort,
RunSnapshot,
RunSummary,
TransactionScope,
WakeAdmissionActiveExecutionRun,
WakeAdmissionHeartbeatHelpers,
WakeAdmissionReader,
WakeAdmissionWriter,
WakeQueueHost,
WakeQueueTransaction,
} from "./ports.js";
@@ -405,3 +411,161 @@ describe("releaseIssueExecution", () => {
expect(recovery.escalateStrandedAssignedIssue).toHaveBeenCalledTimes(1);
});
});
const ACTIVE_EXECUTION_RUN: WakeAdmissionActiveExecutionRun = {
id: "active-run-1",
agentId: "execution-agent",
status: "running",
contextSnapshot: { taskKey: "issue-1" },
};
// The scope is opaque to the use case; the fakes below never inspect it.
const SCOPE = {} as TransactionScope;
function admissionInput(
overrides: Partial<AdmitWakeBehindIssueExecutionInput> = {},
): AdmitWakeBehindIssueExecutionInput {
return {
companyId: "company-1",
issueId: "issue-1",
agentId: "wake-agent",
agentNameKey: "codexcoder",
issueExecutionAgentNameKey: null,
activeExecutionRun: ACTIVE_EXECUTION_RUN,
liveRunExecutions: { has: () => true },
wakeCommentId: null,
forceFreshSession: false,
contextSnapshot: { wakeReason: "issue_commented" },
source: "on_demand",
triggerDetail: null,
payload: { issueId: "issue-1" },
requestedByActorType: "user",
requestedByActorId: "user-1",
idempotencyKey: null,
...overrides,
};
}
function createFakeAdmissionReader(overrides: Partial<WakeAdmissionReader> = {}): WakeAdmissionReader {
return {
isSameExecutionAgent: vi.fn(async () => true),
findExistingDeferredWake: vi.fn(async () => null),
...overrides,
};
}
function createFakeAdmissionWriter(overrides: Partial<WakeAdmissionWriter> = {}): WakeAdmissionWriter {
return {
coalesceIntoActiveExecutionRun: vi.fn(async () => ({ id: "merged-run-1" })),
mergeIntoExistingDeferredWake: vi.fn(async () => {}),
insertNewDeferredWake: vi.fn(async () => {}),
...overrides,
};
}
// Test doubles for the four heartbeat.ts decision helpers the module
// receives as a port. The defaults mirror the real helpers' behaviour for
// the plain wake in `admissionInput()`: no comment id, no forced fresh
// session, and a live coalesce target when `liveRunExecutions.has` says so.
function createFakeAdmissionHelpers(
overrides: Partial<WakeAdmissionHeartbeatHelpers> = {},
): WakeAdmissionHeartbeatHelpers {
return {
filterZombieCoalesceTarget: vi.fn((target, liveRunExecutions) =>
target && liveRunExecutions.has(target.id) ? target : null,
),
mergeCoalescedContextSnapshot: vi.fn((existingRaw, incoming) => ({
...(existingRaw && typeof existingRaw === "object" ? (existingRaw as Record<string, unknown>) : {}),
...incoming,
})),
shouldDeferFollowupWakeForSameIssue: vi.fn(() => false),
shouldQueueFollowupForRunningIssueWake: vi.fn(() => false),
...overrides,
};
}
describe("admitWakeBehindIssueExecution", () => {
it("returns the coalesce outcome and calls the writer one time when the same agent's run absorbs the wake", async () => {
const writer = createFakeAdmissionWriter();
const reader = createFakeAdmissionReader();
const helpers = createFakeAdmissionHelpers();
const admit = createAdmitWakeBehindIssueExecution({ reader, writer, helpers });
const result = await admit(SCOPE, admissionInput());
expect(result).toEqual({ kind: "coalesced", run: { id: "merged-run-1" } });
expect(writer.coalesceIntoActiveExecutionRun).toHaveBeenCalledTimes(1);
expect(writer.mergeIntoExistingDeferredWake).not.toHaveBeenCalled();
expect(writer.insertNewDeferredWake).not.toHaveBeenCalled();
});
it("never reads for an existing deferred wake on the coalesce path", async () => {
const writer = createFakeAdmissionWriter();
const reader = createFakeAdmissionReader();
const helpers = createFakeAdmissionHelpers();
const admit = createAdmitWakeBehindIssueExecution({ reader, writer, helpers });
const result = await admit(SCOPE, admissionInput());
expect(result.kind).toBe("coalesced");
expect(reader.findExistingDeferredWake).not.toHaveBeenCalled();
});
it("merges into the existing deferred wake when the policy returns a merge target", async () => {
const mergeIntoExistingDeferredWake = vi.fn(
async (_scope: TransactionScope, _input: Parameters<WakeAdmissionWriter["mergeIntoExistingDeferredWake"]>[1]) => {},
);
const writer = createFakeAdmissionWriter({ mergeIntoExistingDeferredWake });
const reader = createFakeAdmissionReader({
isSameExecutionAgent: vi.fn(async () => false),
findExistingDeferredWake: vi.fn(async () => ({
id: "deferred-1",
payload: { issueId: "issue-1", foo: "bar" },
deferredContext: { wakeReason: "issue_commented" },
coalescedCount: 2,
})),
});
const helpers = createFakeAdmissionHelpers();
const admit = createAdmitWakeBehindIssueExecution({ reader, writer, helpers });
const result = await admit(SCOPE, admissionInput());
expect(result).toEqual({ kind: "deferred" });
expect(mergeIntoExistingDeferredWake).toHaveBeenCalledTimes(1);
const call = mergeIntoExistingDeferredWake.mock.calls[0]![1];
expect(call.existingDeferredWakeId).toBe("deferred-1");
expect(call.nextCoalescedCount).toBe(3);
expect(call.mergedPayload.foo).toBe("bar");
expect(writer.coalesceIntoActiveExecutionRun).not.toHaveBeenCalled();
expect(writer.insertNewDeferredWake).not.toHaveBeenCalled();
});
it("inserts a new deferred wake when a different agent holds the lock and none is queued yet", async () => {
const writer = createFakeAdmissionWriter();
const reader = createFakeAdmissionReader({ isSameExecutionAgent: vi.fn(async () => false) });
const helpers = createFakeAdmissionHelpers();
const admit = createAdmitWakeBehindIssueExecution({ reader, writer, helpers });
const result = await admit(SCOPE, admissionInput());
expect(result).toEqual({ kind: "deferred" });
expect(writer.insertNewDeferredWake).toHaveBeenCalledTimes(1);
expect(writer.coalesceIntoActiveExecutionRun).not.toHaveBeenCalled();
expect(writer.mergeIntoExistingDeferredWake).not.toHaveBeenCalled();
});
it("proceeds, and never reads for an existing deferred wake, when the zombie-run filter leaves no live coalesce target", async () => {
const writer = createFakeAdmissionWriter();
const reader = createFakeAdmissionReader();
const helpers = createFakeAdmissionHelpers();
const admit = createAdmitWakeBehindIssueExecution({ reader, writer, helpers });
const result = await admit(SCOPE, admissionInput({ liveRunExecutions: { has: () => false } }));
expect(result).toEqual({ kind: "proceed" });
expect(reader.findExistingDeferredWake).not.toHaveBeenCalled();
expect(writer.coalesceIntoActiveExecutionRun).not.toHaveBeenCalled();
expect(writer.mergeIntoExistingDeferredWake).not.toHaveBeenCalled();
expect(writer.insertNewDeferredWake).not.toHaveBeenCalled();
});
});
@@ -2,6 +2,7 @@ import { enrichPromotedWakeContext } from "../domain/context.js";
import {
decideQueuedCommentAction,
decideReleaseRecovery,
decideWakeAdmission,
decideWakeOutcome,
deriveImmediateRecoveryContextLabels,
} from "../domain/policy.js";
@@ -13,6 +14,7 @@ import {
} from "../domain/values.js";
import { withRecoveryContext } from "../../../services/recovery/status-only-context.js";
import type {
AdmitWakeBehindIssueExecutionResult,
DeferredWakeCandidate,
InvokableAgentSnapshot,
IssueLockWriter,
@@ -21,12 +23,19 @@ import type {
RecoveryEscalationPort,
ReleaseTransactionResult,
RunSnapshot,
TransactionScope,
WakeAdmissionActiveExecutionRun,
WakeAdmissionHeartbeatHelpers,
WakeAdmissionReader,
WakeAdmissionWriter,
WakeQueueHost,
WakeQueueTransaction,
} from "./ports.js";
import type { PostCommitEffect, ReleaseOutcome } from "./types.js";
import { WakeQueueApplicationError } from "./types.js";
const DEFERRED_WAKE_CONTEXT_KEY = "_paperclipWakeContext";
const ISSUE_DISPOSITION_REPAIR_RETRY_REASON = "issue_disposition_repair";
const EXECUTION_REVIEW_PARTICIPANT_RECOVERY_WAKE_REASONS = new Set([
"execution_review_requested",
@@ -553,6 +562,144 @@ function statusForBlock(issue: IssueSnapshot): "todo" | "in_progress" | "in_revi
return issue.status === "todo" || issue.status === "in_review" ? issue.status : "in_progress";
}
export type AdmitWakeBehindIssueExecutionInput = {
companyId: string;
issueId: string;
agentId: string;
agentNameKey: string | null;
issueExecutionAgentNameKey: string | null;
activeExecutionRun: WakeAdmissionActiveExecutionRun;
/** Tracks which runs are still live in this process, for the zombie-run filter. */
liveRunExecutions: { has(id: string): boolean };
wakeCommentId: string | null;
forceFreshSession: boolean;
contextSnapshot: Record<string, unknown>;
source: string;
triggerDetail: string | null;
payload: Record<string, unknown> | null;
requestedByActorType: string | null;
requestedByActorId: string | null;
idempotencyKey: string | null;
};
export type { AdmitWakeBehindIssueExecutionResult };
/**
* Decides and applies the admission outcome for a wake that arrives while
* an active execution run already holds the issue's execution lock: merge
* it into that run (coalesce), hold it behind the run (defer), or leave it
* for the caller to queue as an ordinary wake (proceed).
*/
export function createAdmitWakeBehindIssueExecution(deps: {
reader: WakeAdmissionReader;
writer: WakeAdmissionWriter;
helpers: WakeAdmissionHeartbeatHelpers;
}) {
return async function admitWakeBehindIssueExecution(
scope: TransactionScope,
input: AdmitWakeBehindIssueExecutionInput,
): Promise<AdmitWakeBehindIssueExecutionResult> {
const isSameExecutionAgent = await deps.reader.isSameExecutionAgent(scope, {
companyId: input.companyId,
activeExecutionRunAgentId: input.activeExecutionRun.agentId,
issueExecutionAgentNameKey: input.issueExecutionAgentNameKey,
agentNameKey: input.agentNameKey,
});
const shouldDeferFollowupWake = deps.helpers.shouldDeferFollowupWakeForSameIssue({
activeRunStatus: input.activeExecutionRun.status,
isSameExecutionAgent,
wakeCommentId: input.wakeCommentId,
forceFreshSession: input.forceFreshSession,
});
const shouldQueueFollowupForRunningWake =
deps.helpers.shouldQueueFollowupForRunningIssueWake({
contextSnapshot: input.contextSnapshot,
wakeCommentId: input.wakeCommentId,
}) &&
input.activeExecutionRun.status === "running" &&
isSameExecutionAgent;
const availableActiveExecutionRun = isSameExecutionAgent
? deps.helpers.filterZombieCoalesceTarget(input.activeExecutionRun, input.liveRunExecutions)
: input.activeExecutionRun;
const decision = decideWakeAdmission({
isSameExecutionAgent,
shouldDeferFollowupWake,
shouldQueueFollowupForRunningWake,
availableActiveExecutionRunPresent: availableActiveExecutionRun !== null,
});
if (decision.kind === "proceed") return { kind: "proceed" };
if (decision.kind === "coalesce") {
const target = availableActiveExecutionRun!;
const mergedContextSnapshot = deps.helpers.mergeCoalescedContextSnapshot(target.contextSnapshot, input.contextSnapshot, {
preserveExistingInteractionContinuation:
target.status === "queued" || target.status === "scheduled_retry",
});
const run = await deps.writer.coalesceIntoActiveExecutionRun(scope, {
companyId: input.companyId,
activeExecutionRunId: target.id,
mergedContextSnapshot,
agentId: input.agentId,
source: input.source,
triggerDetail: input.triggerDetail,
payload: input.payload,
requestedByActorType: input.requestedByActorType,
requestedByActorId: input.requestedByActorId,
idempotencyKey: input.idempotencyKey,
});
return { kind: "coalesced", run };
}
// decision.kind === "defer": only now does the module read for an
// existing deferred wake, so the coalesce path (the common path) never
// pays for this query.
const existingDeferred = await deps.reader.findExistingDeferredWake(scope, {
companyId: input.companyId,
agentId: input.agentId,
issueId: input.issueId,
});
if (existingDeferred) {
const mergedDeferredContext = deps.helpers.mergeCoalescedContextSnapshot(existingDeferred.deferredContext, input.contextSnapshot, {
preserveExistingInteractionContinuation: true,
});
const mergedPayload = {
...existingDeferred.payload,
...(input.payload ?? {}),
issueId: input.issueId,
[DEFERRED_WAKE_CONTEXT_KEY]: mergedDeferredContext,
};
await deps.writer.mergeIntoExistingDeferredWake(scope, {
companyId: input.companyId,
existingDeferredWakeId: existingDeferred.id,
mergedPayload,
nextCoalescedCount: (existingDeferred.coalescedCount ?? 0) + 1,
});
return { kind: "deferred" };
}
const deferredPayload = {
...(input.payload ?? {}),
issueId: input.issueId,
[DEFERRED_WAKE_CONTEXT_KEY]: input.contextSnapshot,
};
await deps.writer.insertNewDeferredWake(scope, {
companyId: input.companyId,
agentId: input.agentId,
source: input.source,
triggerDetail: input.triggerDetail,
payload: deferredPayload,
requestedByActorType: input.requestedByActorType,
requestedByActorId: input.requestedByActorId,
idempotencyKey: input.idempotencyKey,
});
return { kind: "deferred" };
};
}
export function createReleaseIssueExecution(deps: {
issueLock: IssueLockWriter;
recovery: RecoveryEscalationPort;