From 503268b80669a142d9993e937f91060bd37168cb Mon Sep 17 00:00:00 2001 From: Nicky Leach Date: Thu, 19 Nov 2026 07:23:02 -0800 Subject: [PATCH] perf(wake-queue): preflight task admission without loading execution context (#22289) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip uses task admission to turn assignments and comments into queued work. > - Wake-queue owns the issue-admission transaction. Heartbeat previously loaded full execution context to decide whether a conversation wake could proceed. > - After the conversation preflight, task state can change before admission obtains its lock. > - Wake-queue now exposes the preflight decision and obtains only the fields it needs. It provides current task attribution from the locked row. > - Heartbeat uses the existing responsible-user and routine rules when it creates a run. ## Linked Issues or Issue Description **What existing behavior does this improve?** Admitting board assignments and saved task comments. **Subsystem affected** Server task admission. **Current behavior** Conversation preflight and fresh-run attribution load full task execution context from heartbeat, including chat joins. **Proposed behavior** Give wake-queue ownership of conversation preflight and its PostgreSQL read. Use the issue row already locked by wake-queue for fresh-run attribution. **Additional context** Builds on #22280 and #22282. #17396 covers a separate workspace inspection on task context reads. ## What Changed - Give heartbeat a preflight decision. Keep storage and the five-field query inside wake-queue. - Reuse the conversation and retry predicates for all callers. Keep the locked recheck after preflight, including durable chat authorization. - Supply the task's responsible user, parent and routine origin from the current locked row. A company-only admission supplies no issue. ## Verification On `b89bed954a`, all 307 tests passed across the wake-queue preflight and release application suites, heartbeat queue and run-preparation suites, and Slack conversation lifecycle suite. The integration suites used embedded PostgreSQL. Coverage includes the owner 403, a conversation settling before admission locks the issue, saved-comment attribution after reassignment, and routine ownership written inside the admission transaction. The layered server typecheck and module dependency check passed on the same revision. In a repeated local comparison of base `90d985b8b9` and PR head `b89bed954a`, we measured 34 samples per revision, scenario, and recovery-history state, alternating the two revisions in ABBA order. Server-side admission medians were: | Recovery history | Assignment | One comment | Two serial comments | | --- | ---: | ---: | ---: | | Empty | 23.45 → 21.80 ms | 23.95 → 23.00 ms | 47.30 → 45.40 ms | | Resolved | 23.35 → 21.30 ms | 23.70 → 22.05 ms | 46.70 → 45.40 ms | Typical SQL statements fell from 36/37/74 to 35/36/73 for the three scenarios. Lock statement counts were 2/2/6 for both revisions. Throughput varied under local contention. These are local measurements. Standard Greptile scored the exact head 5/5 with no review threads. [Greptile review](https://github.com/paperclipai/paperclip/pull/22289#issuecomment-6999578088) GitHub CI passed on the exact head after one targeted retry of the serialized server shard. The initial status-card maintenance case timed out waiting for idle, passed when run locally on the same revision, and passed in the retried shard. [CI workflow attempt 2](https://github.com/paperclipai/paperclip/actions/runs/44458613437/attempts/2) ## Risks An early decision can go stale, and the caller must preserve authenticated actor identity. The locked path retains its own eligibility checks. A regression exercises the gap between the two decisions. An attribution test inserts its evidence inside the transaction to detect a read through a different connection. ## Model Used OpenAI GPT-6.2 Sol (gpt-6.2-sol) implemented the change. OpenAI Codex reviewed and qualified it. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --- doc/heartbeat-internals.md | 8 +- .../src/__tests__/helpers/heartbeat-queue.ts | 1 + server/src/lib/conversation-wake-policy.ts | 36 ++++++ .../modules/wake-queue/adapters/postgres.ts | 18 +++ .../application/issue-wake-admission.test.ts | 51 ++++++++- .../application/issue-wake-admission.ts | 46 ++++++++ .../modules/wake-queue/application/ports.ts | 5 +- .../modules/wake-queue/application/types.ts | 2 + server/src/modules/wake-queue/index.ts | 13 ++- server/src/services/agent-conversations.ts | 34 +----- server/src/services/heartbeat/queue.test.ts | 47 ++++++-- server/src/services/heartbeat/queue.ts | 106 ++++++++---------- server/src/services/task-retry.ts | 4 +- 13 files changed, 262 insertions(+), 109 deletions(-) create mode 100644 server/src/lib/conversation-wake-policy.ts diff --git a/doc/heartbeat-internals.md b/doc/heartbeat-internals.md index 257261c4005..b4adc245c4e 100644 --- a/doc/heartbeat-internals.md +++ b/doc/heartbeat-internals.md @@ -337,9 +337,11 @@ Heartbeat coordinates wakeup dispatch, queued-run claims, daily heartbeat caps, concurrency and priority checks, timer admission, and native status wake intents in `server/src/services/heartbeat/queue.ts`. The wake-queue module owns the company and issue lock, the admission transaction, and the handoff after commit -for both fresh task runs and wakes arriving behind an active execution. Heartbeat -uses the locked transaction for its specialized eligibility gates and continuation -writes. Wake-queue selects eligible saved comment receipts in request order and +for both fresh task runs and wakes arriving behind an active execution. Wake-queue +checks conversation eligibility before locking the task. The locked task supplies +the attribution fields used to select the responsible user for a fresh run. +Heartbeat uses the transaction for its specialized eligibility gates and +continuation writes. Wake-queue selects eligible saved comment receipts in request order and binds them to the new run. Chat authorization and its admission callback run in the same transaction. Authorization contention defers the wake and rolls back without publishing buffered effects. diff --git a/server/src/__tests__/helpers/heartbeat-queue.ts b/server/src/__tests__/helpers/heartbeat-queue.ts index 641a643c609..c7eb911d5a9 100644 --- a/server/src/__tests__/helpers/heartbeat-queue.ts +++ b/server/src/__tests__/helpers/heartbeat-queue.ts @@ -62,6 +62,7 @@ export function createHeartbeatQueueTestDependencies(db: Db): QueueTestDependenc instanceSettings: { getExperimental: vi.fn() }, runDispatch: createRunDispatch(db), wakeQueue: { + preflightIssueWake: vi.fn(admission.preflightIssueWake), admitIssueWake: vi.fn(admission.admitIssueWake), }, activeRunExecutions: new Set(), diff --git a/server/src/lib/conversation-wake-policy.ts b/server/src/lib/conversation-wake-policy.ts new file mode 100644 index 00000000000..6599f0aca4a --- /dev/null +++ b/server/src/lib/conversation-wake-policy.ts @@ -0,0 +1,36 @@ +export type ConversationIdentity = { + conversationAgentId?: string | null; + conversationUserId?: string | null; + conversationState?: string | null; + status?: string; +}; +export function isConversation( + issue: ConversationIdentity | null | undefined, +): boolean { + return Boolean(issue?.conversationAgentId && issue.conversationUserId); +} +export function isWaitingConversation( + issue: ConversationIdentity | null | undefined, +): boolean { + return ( + isConversation(issue) && + issue?.conversationState === "waiting" && + issue.status === "in_review" + ); +} + +export function isConversationExecutionWake( + issue: ConversationIdentity | null | undefined, + reason: string | null | undefined, +): boolean { + return isConversation(issue) && ( + reason === "issue_blockers_resolved" || + reason === "issue_children_completed" || + reason === "issue_blocker_stalled" || + reason === "issue_unblock_requested" + ); +} + +export function canActorRetryConversation(issue: ConversationIdentity, actorId: string): boolean { + return !issue.conversationAgentId || issue.conversationUserId === actorId; +} diff --git a/server/src/modules/wake-queue/adapters/postgres.ts b/server/src/modules/wake-queue/adapters/postgres.ts index ec2dcf9ada0..efc3ac254de 100644 --- a/server/src/modules/wake-queue/adapters/postgres.ts +++ b/server/src/modules/wake-queue/adapters/postgres.ts @@ -1276,6 +1276,19 @@ export function createWakeAdmissionWriter(rawTx: IssueAdmissionDbTransaction, ad }; } +export function createIssueWakePreflightReader(db: Db) { + return async (companyId: string, issueId: string) => { + const [issue] = await db.select({ + conversationAgentId: issues.conversationAgentId, + conversationUserId: issues.conversationUserId, + conversationState: issues.conversationState, + status: issues.status, + originKind: issues.originKind, + }).from(issues).where(and(eq(issues.id, issueId), eq(issues.companyId, companyId))); + return issue ?? null; + }; +} + const issueAdmissionFields = { id: issues.id, companyId: issues.companyId, @@ -1298,6 +1311,11 @@ const issueAdmissionFields = { checkoutRunId: issues.checkoutRunId, executionRunId: issues.executionRunId, executionAgentNameKey: issues.executionAgentNameKey, + responsibleUserId: issues.responsibleUserId, + parentId: issues.parentId, + originKind: issues.originKind, + originId: issues.originId, + originRunId: issues.originRunId, createdAt: issues.createdAt, }; diff --git a/server/src/modules/wake-queue/application/issue-wake-admission.test.ts b/server/src/modules/wake-queue/application/issue-wake-admission.test.ts index 4ae47b68427..945e4e8d9b6 100644 --- a/server/src/modules/wake-queue/application/issue-wake-admission.test.ts +++ b/server/src/modules/wake-queue/application/issue-wake-admission.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "vitest"; -import { admitIssueWake, type IssueWakeAdmissionEffects, type IssueWakeAdmissionOutcome } from "./issue-wake-admission.js"; +import { admitIssueWake, createIssueWakePreflight, type IssueWakeAdmissionEffects, type IssueWakeAdmissionOutcome } from "./issue-wake-admission.js"; const queuedRun = { id: "run", companyId: "company", agentId: "agent", status: "queued", invocationSource: "assignment", triggerDetail: null, wakeupRequestId: "wake" }; @@ -23,6 +23,55 @@ function recordEffects(trace: string[]): IssueWakeAdmissionEffects { }; } +describe("issue wake preflight", () => { + const input = { + companyId: "company", + issueId: "issue", + agentId: "agent", + reason: null, + wakeReason: null, + taskRetryActorId: null, + hasIndependentConversationWakeCause: false, + }; + const activeConversation = { + conversationAgentId: "agent", + conversationUserId: "owner", + conversationState: "active", + status: "in_progress", + originKind: null, + }; + const onboardingTask = { + ...activeConversation, + conversationAgentId: null, + conversationUserId: null, + conversationState: null, + originKind: "onboarding_first_task", + }; + + it.each([ + { label: "onboarding completion", issue: onboardingTask, reasons: { reason: "issue_children_completed" }, + result: { kind: "eligible", onboardingCompletion: true } }, + { label: "onboarding context fallback", issue: onboardingTask, reasons: { wakeReason: "issue_children_completed" }, + result: { kind: "eligible", onboardingCompletion: false } }, + { label: "execution context fallback", issue: activeConversation, reasons: { wakeReason: "issue_children_completed" }, + result: { kind: "skipped" } }, + { label: "explicit reason over execution context", issue: activeConversation, + reasons: { reason: "issue_commented", wakeReason: "issue_children_completed" }, + result: { kind: "eligible", onboardingCompletion: false } }, + ])("handles $label", async ({ issue, reasons, result }) => { + const preflight = createIssueWakePreflight(async () => issue); + await expect(preflight({ ...input, ...reasons })).resolves.toEqual(result); + }); + + it("requires the owner of a conversation for a manual start or retry", async () => { + const preflight = createIssueWakePreflight(async () => activeConversation); + await expect(preflight({ ...input, manualUserWake: true, requestedByActorId: "someone-else" })) + .resolves.toEqual({ kind: "conversation_owner_required", action: "start" }); + await expect(preflight({ ...input, taskRetryActorId: "someone-else", reason: "issue_children_completed" })) + .resolves.toEqual({ kind: "conversation_owner_required", action: "retry" }); + }); +}); + describe("issue wake admission handoff", () => { it("waits for commit before publishing or dispatching and leaves telemetry in the background", async () => { const trace: string[] = []; diff --git a/server/src/modules/wake-queue/application/issue-wake-admission.ts b/server/src/modules/wake-queue/application/issue-wake-admission.ts index fac3a4e424f..6f57462ee0c 100644 --- a/server/src/modules/wake-queue/application/issue-wake-admission.ts +++ b/server/src/modules/wake-queue/application/issue-wake-admission.ts @@ -1,5 +1,51 @@ +import { canActorRetryConversation, isConversation, isConversationExecutionWake, isWaitingConversation } from "../../../lib/conversation-wake-policy.js"; import type { RunSummary } from "./types.js"; +type ConversationWakeIssue = { + conversationAgentId: string | null; + conversationUserId: string | null; + conversationState: string | null; + status: string; + originKind: string | null; +}; + +export function createIssueWakePreflight(readIssue: (companyId: string, issueId: string) => Promise) { + return async function preflightIssueWake(input: { + companyId: string; + issueId: string; + agentId: string; + reason: string | null; + wakeReason: string | null; + requestedByActorId?: string | null; + manualUserWake?: boolean; + taskRetryActorId: string | null; + hasIndependentConversationWakeCause: boolean; + }): Promise< + | { kind: "eligible"; onboardingCompletion: boolean } + | { kind: "skipped" } + | { kind: "conversation_owner_required"; action: "start" | "retry" } + > { + const issue = await readIssue(input.companyId, input.issueId); + if (input.taskRetryActorId && issue && !canActorRetryConversation(issue, input.taskRetryActorId)) { + return { kind: "conversation_owner_required", action: "retry" }; + } + const onboardingCompletion = input.reason === "issue_children_completed" && issue?.originKind === "onboarding_first_task"; + if (issue && isConversation(issue)) { + if (input.reason === "onboarding_chat_greeting" && (issue.originKind !== "onboarding_chat" + || issue.conversationUserId !== input.requestedByActorId)) return { kind: "skipped" }; + if (input.manualUserWake && issue.conversationUserId !== input.requestedByActorId) { + return { kind: "conversation_owner_required", action: "start" }; + } + if (isConversationExecutionWake(issue, input.reason ?? input.wakeReason) + || input.agentId !== issue.conversationAgentId + || (isWaitingConversation(issue) && !input.hasIndependentConversationWakeCause)) { + return { kind: "skipped" }; + } + } + return { kind: "eligible", onboardingCompletion }; + }; +} + export type IssueWakeAdmissionOutcome = | { kind: "queued" | "coalesced" | "replayed"; run: Run } | { kind: "durable"; receipt: { runId: string | null } } diff --git a/server/src/modules/wake-queue/application/ports.ts b/server/src/modules/wake-queue/application/ports.ts index f6061ea37a0..7ceade63544 100644 --- a/server/src/modules/wake-queue/application/ports.ts +++ b/server/src/modules/wake-queue/application/ports.ts @@ -1,6 +1,7 @@ import type { ReleaseRecoveryBlockedNoticeKind } from "../domain/policy.js"; import type { InvokableAgentSnapshot, + IssueAttributionSnapshot, IssueSnapshot, PostCommitEffect, ReleaseOutcome, @@ -44,7 +45,7 @@ export interface WakeQueueHost { resolveResponsibleUserId(input: { companyId: string; contextSnapshot: Record; - issue: IssueSnapshot; + issue: IssueAttributionSnapshot; /** From a prior `getRoutineEnv` call against the same issue; pass `{ routineId: null, env: null, responsibleUserId: null }` when the issue is not a routine execution. */ routineEnvContext: { routineId: string | null; env: unknown; responsibleUserId: string | null }; requestedByActorType: "user" | "agent" | "system" | null; @@ -60,7 +61,7 @@ export interface WakeQueueHost { */ getRoutineEnv(input: { companyId: string; - issue: IssueSnapshot; + issue: IssueAttributionSnapshot; }): Promise<{ routineId: string | null; env: unknown; responsibleUserId: string | null }>; resolveSessionBeforeForWakeup(input: { companyId: string; diff --git a/server/src/modules/wake-queue/application/types.ts b/server/src/modules/wake-queue/application/types.ts index 2f496d5c0b3..73e27c5a35d 100644 --- a/server/src/modules/wake-queue/application/types.ts +++ b/server/src/modules/wake-queue/application/types.ts @@ -55,6 +55,8 @@ export type IssueSnapshot = { originRunId: string | null; }; +export type IssueAttributionSnapshot = Pick; + export type InvokableAgentSnapshot = { id: string; companyId: string; diff --git a/server/src/modules/wake-queue/index.ts b/server/src/modules/wake-queue/index.ts index 96dec6b04e8..405d9d82e7f 100644 --- a/server/src/modules/wake-queue/index.ts +++ b/server/src/modules/wake-queue/index.ts @@ -1,5 +1,7 @@ import type { Db, heartbeatRuns } from "@paperclipai/db"; +import { forbidden } from "../../errors.js"; import { + createIssueWakePreflightReader, createPostgresIssueWakeAdmission, createPostgresWakeQueueAdapter, type IssueAdmissionDbTransaction, @@ -10,7 +12,7 @@ import { createQueuedCommentIssueLockWriter } from "./adapters/queued-comment-po import { createQueuedCommentSteeringAdapter, lockBoardQueuedComments, readBoardQueuedComments } from "./adapters/queued-comment-board-postgres.js"; import type { QueuedCommentQueuePostgresAdapterDeps } from "./adapters/queued-comment-postgres.js"; import { createFreshIssueRunAdmission } from "./application/fresh-issue-run.js"; -import { admitIssueWake, type IssueWakeAdmissionEffects, type IssueWakeAdmissionOutcome } from "./application/issue-wake-admission.js"; +import { admitIssueWake, createIssueWakePreflight, type IssueWakeAdmissionEffects, type IssueWakeAdmissionOutcome } from "./application/issue-wake-admission.js"; import { createAdmitWakeBehindIssueExecution, createReleaseIssueExecution } from "./application/use-cases.js"; import { createDiscardQueuedComment, @@ -27,6 +29,7 @@ import type { } from "./application/ports.js"; export type { + IssueAttributionSnapshot, PostCommitEffect, ReleaseOutcome, RunSummary, @@ -97,8 +100,16 @@ export function createWakeQueue(db: Db, deps: WakeQueueDeps) { resolveSessionBeforeForWakeup: deps.resolveSessionBeforeForWakeup, }); const withIssueAdmission = createPostgresIssueWakeAdmission(db); + const preflight = createIssueWakePreflight(createIssueWakePreflightReader(db)); return { + async preflightIssueWake(input: Parameters[0]) { + const outcome = await preflight(input); + if (outcome.kind === "conversation_owner_required") { + throw forbidden(`Only the conversation owner can ${outcome.action} a chat run`); + } + return outcome; + }, releaseIssueExecution: createReleaseIssueExecution({ issueLock, recovery: deps.recovery }), admitIssueWake( { companyId, issueId, durableChatRequest }: IssueAdmissionTarget, diff --git a/server/src/services/agent-conversations.ts b/server/src/services/agent-conversations.ts index af9470932c3..e26f1289439 100644 --- a/server/src/services/agent-conversations.ts +++ b/server/src/services/agent-conversations.ts @@ -32,26 +32,8 @@ import { isAgentAwaitingSetup } from "../modules/agent-lifecycle/index.js"; import { HttpError } from "../errors.js"; import { ONBOARDING_CHAT_GREETING_REASON } from "./onboarding-chat-prompt.js"; -export type ConversationIdentity = { - conversationAgentId?: string | null; - conversationUserId?: string | null; - conversationState?: string | null; - status?: string; -}; -export function isConversation( - issue: ConversationIdentity | null | undefined, -): boolean { - return Boolean(issue?.conversationAgentId && issue.conversationUserId); -} -export function isWaitingConversation( - issue: ConversationIdentity | null | undefined, -): boolean { - return ( - isConversation(issue) && - issue?.conversationState === "waiting" && - issue.status === "in_review" - ); -} +import { isConversation, isWaitingConversation, type ConversationIdentity } from "../lib/conversation-wake-policy.js"; +export { isConversation, isWaitingConversation, isConversationExecutionWake, type ConversationIdentity } from "../lib/conversation-wake-policy.js"; /** Recovery for an older turn must not replace a reset or an answered chat. */ export function isSupersededConversationRun( @@ -76,18 +58,6 @@ export function isSupersededConversationRun( * Apply before enqueue, including while a reply is still running: waiting until * finalization is too late to prevent a deferred dependency follow-up. */ -export function isConversationExecutionWake( - issue: ConversationIdentity | null | undefined, - reason: string | null | undefined, -): boolean { - return isConversation(issue) && ( - reason === "issue_blockers_resolved" || - reason === "issue_children_completed" || - reason === "issue_blocker_stalled" || - reason === "issue_unblock_requested" - ); -} - export function isConversationReset(body: string): boolean { return body.trim() === "/new"; } diff --git a/server/src/services/heartbeat/queue.test.ts b/server/src/services/heartbeat/queue.test.ts index e29dddafa32..e03f3ba14e4 100644 --- a/server/src/services/heartbeat/queue.test.ts +++ b/server/src/services/heartbeat/queue.test.ts @@ -2,7 +2,7 @@ import { randomUUID } from "node:crypto"; import { contextCompactionService } from "../context-compaction.js"; import { claimNativeSessionResumptions } from "../native-runtime/native-finalization-reconciler.js"; import type { Db } from "@paperclipai/db"; -import { authUsers, agents, agentTaskSessions, agentWakeupRequests, budgetIncidents, budgetPolicies, caseHandoffs, cases, chatCompletionDeliveries, companies, companyMemberships, costEvents, createDb, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, environmentLeases, heartbeatRuns, issueComments, issueThreadInteractions, issues, nativeRunFinalizations } from "@paperclipai/db"; +import { authUsers, agents, agentTaskSessions, agentWakeupRequests, budgetIncidents, budgetPolicies, caseHandoffs, cases, chatCompletionDeliveries, companies, companyMemberships, costEvents, createDb, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, environmentLeases, heartbeatRuns, issueComments, issueThreadInteractions, issues, nativeRunFinalizations, routines, routineRuns } from "@paperclipai/db"; import { eq, inArray, sql } from "drizzle-orm"; import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest"; import { resolveExternalTestDatabaseUrl } from "../../__tests__/helpers/external-test-database.js"; @@ -184,13 +184,7 @@ describe.skipIf(!support.supported)("heartbeat queue database wiring", () => { // Admit through the normal transaction, but hold dispatch so no adapter runs. deps.getSchedulingSuppression.mockResolvedValue({ suppressed: true, reason: "task_drain" }); deps.getSchedulingSuppression.mockResolvedValueOnce({ suppressed: false, reason: null }); - const getIssueExecutionContext = deps.getIssueExecutionContext; - const outsideStates: Array<{ status: string; conversationState: string | null }> = []; - deps.getIssueExecutionContext = async (...args) => { - const current = await getIssueExecutionContext(...args); - if (current) outsideStates.push({ status: current.status, conversationState: current.conversationState }); - return current; - }; + const preflight = deps.wakeQueue.preflightIssueWake; let reached!: () => void; let release!: () => void; const atBarrier = new Promise(resolve => { reached = resolve; }); @@ -213,7 +207,8 @@ describe.skipIf(!support.supported)("heartbeat queue database wiring", () => { }); try { await atBarrier; - expect(outsideStates).toEqual([{ status: "in_progress", conversationState: "active" }]); + expect(preflight).toHaveBeenCalledOnce(); + await expect(preflight.mock.results[0]?.value).resolves.toMatchObject({ kind: "eligible" }); expect(await settleConversationTurn(db, completed)).toBe(true); expect((await db.select().from(issues).where(eq(issues.id, issue.id)))[0]) .toMatchObject({ status: "in_review", conversationState: "waiting" }); @@ -452,7 +447,7 @@ describe.skipIf(!support.supported)("heartbeat queue database wiring", () => { { actor: "foreign owner", userId: "another-conversation-owner", message: "Only the conversation owner can start a chat run" }, ])("refuses waiting-conversation compaction from a $actor", async ({ userId, message }) => { const f = await waitingCompactionFixture(); - await expect(f.service.request({ companyId: f.company.id, issue: f.issue, userId })).rejects.toThrow(message); + await expect(f.service.request({ companyId: f.company.id, issue: f.issue, userId })).rejects.toMatchObject({ status: 403, message }); expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.agentId, f.agent.id))).toHaveLength(0); expect(await db.select().from(agentWakeupRequests).where(eq(agentWakeupRequests.agentId, f.agent.id))).toHaveLength(0); expect(f.deps.executeRun).not.toHaveBeenCalled(); @@ -577,7 +572,7 @@ describe.skipIf(!support.supported)("heartbeat queue database wiring", () => { } it.each([true, false])("adopts only eligible saved comments from the exact reassignment source (%s)", async (matchingIssue) => { - const { user, company, agent, issue, deps, queue } = await waitingConversationFixture(); + const { user, company, agent, issue, deps } = await waitingConversationFixture(); await db.update(issues).set({ conversationAgentId: null, conversationUserId: null, conversationState: null, status: "todo" }).where(eq(issues.id, issue.id)); const [previousAgent] = await db.insert(agents).values({ companyId: company.id, @@ -599,6 +594,8 @@ describe.skipIf(!support.supported)("heartbeat queue database wiring", () => { payload: withQueuedCommentIdsInWakePayload({ issueId: issue.id, mutation }, [commentId]), }))).returning(); + const loadExecutionContext = vi.spyOn(deps, "getIssueExecutionContext"); + const queue = createHeartbeatQueue(db, deps); const admit = deps.wakeQueue.admitIssueWake.getMockImplementation()!; deps.wakeQueue.admitIssueWake.mockImplementation((target, evaluate, effects) => admit(target, async (tx, admission) => { await tx.insert(agentTaskSessions).values({ companyId: company.id, agentId: agent.id, @@ -610,6 +607,7 @@ describe.skipIf(!support.supported)("heartbeat queue database wiring", () => { payload: { issueId: issue.id }, contextSnapshot: { issueId: issue.id, interruptedRunId: interruptedRun.id }, }); + expect(loadExecutionContext).not.toHaveBeenCalled(); const commentIds = matchingIssue ? [earlier.id, later.id] : [later.id]; expect(run).toMatchObject({ status: "queued", responsibleUserId: user.id, invocationSource: "assignment", sessionIdBefore: "session-available-in-admission", @@ -626,6 +624,33 @@ describe.skipIf(!support.supported)("heartbeat queue database wiring", () => { expect(deps.executeRun).not.toHaveBeenCalled(); }); + it("uses the routine run attribution visible in the admission transaction", async () => { + const { company, agent, issue, deps, queue } = await waitingConversationFixture(); + const [routine] = await db.insert(routines).values({ + companyId: company.id, title: "Routine", assigneeAgentId: agent.id, responsibleUserId: "current-routine-owner", + }).returning(); + const [routineRun] = await db.insert(routineRuns).values({ + companyId: company.id, routineId: routine.id, source: "schedule", responsibleUserId: "original-routine-owner", + }).returning(); + await db.update(issues).set({ + conversationAgentId: null, conversationUserId: null, conversationState: null, status: "todo", + originKind: "routine_execution", originId: routine.id, + }).where(eq(issues.id, issue.id)); + + const admit = deps.wakeQueue.admitIssueWake.getMockImplementation()!; + deps.wakeQueue.admitIssueWake.mockImplementation((target, evaluate, effects) => admit(target, async (tx, admission) => { + await tx.update(issues).set({ originRunId: routineRun.id }).where(eq(issues.id, issue.id)); + return evaluate(tx, { ...admission, lockedIssue: await admission.lockIssue() }); + }, effects)); + const run = await queue.enqueueWakeup(agent.id, { + source: "assignment", triggerDetail: "system", reason: "issue_assigned", requestedByActorType: "system", + payload: { issueId: issue.id }, + }); + + expect(run).toMatchObject({ status: "queued", responsibleUserId: "original-routine-owner" }); + expect(deps.executeRun).not.toHaveBeenCalled(); + }); + it("only promotes a deferred chat receipt for its agent and binds receipts within its company", async () => { const local = await fixture("queued"); const [foreignCompany] = await db.insert(companies).values({ name: "Other company", issuePrefix: "OC" }).returning(); diff --git a/server/src/services/heartbeat/queue.ts b/server/src/services/heartbeat/queue.ts index f7995b8771a..4d63a52832d 100644 --- a/server/src/services/heartbeat/queue.ts +++ b/server/src/services/heartbeat/queue.ts @@ -57,7 +57,6 @@ import { isVerifiedDependencyInteractionResponse, readQueuedInteractionResponse import { isConversation, isConversationReset, - isConversationExecutionWake, isWaitingConversation, } from "../agent-conversations.js"; import { onboardingGreetingNeeded } from "../onboarding-chat.js"; @@ -219,7 +218,7 @@ import type { budgetService } from "../budgets.js"; import { isBoardOwnedBudgetBlock } from "../budget-block-ownership.js"; import type { instanceSettingsService } from "../instance-settings.js"; import type { createRunDispatch } from "../../modules/run-dispatch/index.js"; -import { hasRetainedExternalChatRetry, type PostCommitEffect as WakeQueuePostCommitEffect, type createWakeQueue } from "../../modules/wake-queue/index.js"; +import { hasRetainedExternalChatRetry, type IssueAttributionSnapshot, type PostCommitEffect as WakeQueuePostCommitEffect, type createWakeQueue } from "../../modules/wake-queue/index.js"; import { crashPoint } from "@paperclipai/shared/crash-points"; import { emitSystemTaskBlockedAfterCommit } from "../product-telemetry.js"; @@ -248,7 +247,7 @@ export interface HeartbeatQueueDependencies extends Pick, "getInvocationBlock">; instanceSettings: Pick, "getExperimental">; runDispatch: Pick, "cancelStaleQueuedRun">; - wakeQueue: Pick, "admitIssueWake">; + wakeQueue: Pick, "preflightIssueWake" | "admitIssueWake">; activeRunExecutions: Set; activeRunExecutionPromises: Set>; activeWakeupPromises: Set>; @@ -2877,19 +2876,19 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende // a run or deferred request; assignment and review have their own wakes. if (reason === "issue_comment_mentioned" || enrichedContextSnapshot.wakeReason === "issue_comment_mentioned") return null; if (issueId) { - const conversation = await getIssueExecutionContext(agent.companyId, issueId); - if (taskRetryActorId && conversation) assertTaskRetryConversationAllowed(conversation, taskRetryActorId); - if (reason === "issue_children_completed" && conversation?.originKind === "onboarding_first_task") enrichedContextSnapshot.onboardingCompletion = true; - if (isConversation(conversation)) { - if (reason === "onboarding_chat_greeting" && (conversation?.originKind !== "onboarding_chat" - || conversation.conversationUserId !== opts.requestedByActorId)) return null; - if (opts.manualUserWake && conversation!.conversationUserId !== opts.requestedByActorId) { - throw new HttpError(403, "Only the conversation owner can start a chat run"); - } - if (isConversationExecutionWake(conversation, reason ?? readNonEmptyString(enrichedContextSnapshot.wakeReason))) return null; - if (agent.id !== conversation!.conversationAgentId) return null; - if (isWaitingConversation(conversation) && !hasIndependentConversationWakeCause()) return null; - } + const preflight = await wakeQueue.preflightIssueWake({ + companyId: agent.companyId, + issueId, + agentId: agent.id, + reason, + wakeReason: readNonEmptyString(enrichedContextSnapshot.wakeReason), + requestedByActorId: opts.requestedByActorId, + manualUserWake: opts.manualUserWake, + taskRetryActorId, + hasIndependentConversationWakeCause: hasIndependentConversationWakeCause(), + }); + if (preflight.kind === "skipped") return null; + if (preflight.onboardingCompletion) enrichedContextSnapshot.onboardingCompletion = true; } if (agent.adapterType === "paperclip_runner") { const oldConfig = parseObject(agent.adapterConfig); @@ -3350,48 +3349,41 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende ? (await instanceSettings.getExperimental()).enableIsolatedWorkspaces : false; let operatorResponsibleUserId: string | null = opts.manualUserWake ? opts.requestedByActorId! : null; - let queuedResponsibleUserIdPromise: Promise | null = null; - const resolveQueuedResponsibleUserId = () => { - if (operatorResponsibleUserId) return Promise.resolve(operatorResponsibleUserId); - queuedResponsibleUserIdPromise ??= (async () => { - const queuedIssueContext = issueId - ? await getIssueExecutionContext(agent.companyId, issueId) - : null; - const queuedRoutineEnvContext = await getRoutineEnvForExecutionIssue( - agent.companyId, - queuedIssueContext, - ); - const queuedResponsibleUserId = - await resolveResponsibleUserIdForRunSeed({ + const resolveQueuedResponsibleUserId = async (issueContext: IssueAttributionSnapshot | null) => { + if (operatorResponsibleUserId) return operatorResponsibleUserId; + const queuedRoutineEnvContext = await getRoutineEnvForExecutionIssue( + agent.companyId, + issueContext, + ); + const queuedResponsibleUserId = + await resolveResponsibleUserIdForRunSeed({ + companyId: agent.companyId, + contextSnapshot: enrichedContextSnapshot, + issueContext, + routineEnvContext: queuedRoutineEnvContext, + requestedByActorType: opts.requestedByActorType ?? null, + requestedByActorId: opts.requestedByActorId ?? null, + source, + triggerDetail, + }); + if (!queuedResponsibleUserId) { + throw new HttpError( + 422, + "Unable to resolve responsible user for heartbeat run dispatch", + { + code: "responsible_user_unresolved", + agentId, companyId: agent.companyId, - contextSnapshot: enrichedContextSnapshot, - issueContext: queuedIssueContext, - routineEnvContext: queuedRoutineEnvContext, - requestedByActorType: opts.requestedByActorType ?? null, - requestedByActorId: opts.requestedByActorId ?? null, + issueId: issueId ?? null, source, triggerDetail, - }); - if (!queuedResponsibleUserId) { - throw new HttpError( - 422, - "Unable to resolve responsible user for heartbeat run dispatch", - { - code: "responsible_user_unresolved", - agentId, - companyId: agent.companyId, - issueId: issueId ?? null, - source, - triggerDetail, - wakeReason: readNonEmptyString( - enrichedContextSnapshot.wakeReason, - ), - }, - ); - } - return queuedResponsibleUserId; - })(); - return queuedResponsibleUserIdPromise; + wakeReason: readNonEmptyString( + enrichedContextSnapshot.wakeReason, + ), + }, + ); + } + return queuedResponsibleUserId; }; const budgetBlock = await budgets.getInvocationBlock( @@ -5068,7 +5060,7 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende continuationAttempt, reconciledSourceRunId, reconciledRestoreRetryCount, - resolveResponsibleUserId: resolveQueuedResponsibleUserId, + resolveResponsibleUserId: () => resolveQueuedResponsibleUserId(issue), async recordContinuationDelivery(runId) { if (ownsReconstructionContinuation) { await tx.update(issueRecoveryActions).set({ evidence: sql`${issueRecoveryActions.evidence} || ${JSON.stringify({ @@ -5296,7 +5288,7 @@ export function createHeartbeatQueue(db: Db, dependencies: HeartbeatQueueDepende invocationSource: source, triggerDetail, status: "queued", - responsibleUserId: await resolveQueuedResponsibleUserId(), + responsibleUserId: await resolveQueuedResponsibleUserId(null), wakeupRequestId: wakeupRequest.id, contextSnapshot: enrichedContextSnapshot, sessionIdBefore: await resolveSessionBefore(tx), diff --git a/server/src/services/task-retry.ts b/server/src/services/task-retry.ts index 248362ee749..a060fa14ea8 100644 --- a/server/src/services/task-retry.ts +++ b/server/src/services/task-retry.ts @@ -1,3 +1,4 @@ +import { canActorRetryConversation } from "../lib/conversation-wake-policy.js"; import { notifyStrandedAgentStatus } from "./live-run-recovery-notifications.js"; import { notifyExternalConversationSettlement } from "./external-conversation-settlement-notifications.js"; import { writeExecutionControl } from "./execution-control-notifications.js"; @@ -46,8 +47,7 @@ export function assertTaskRetryConversationAllowed( issue: { conversationAgentId?: string | null; conversationUserId?: string | null }, actorId: string, ) { - if (!issue.conversationAgentId) return; - if (issue.conversationUserId !== actorId) throw forbidden("Only the conversation owner can retry a chat run"); + if (!canActorRetryConversation(issue, actorId)) throw forbidden("Only the conversation owner can retry a chat run"); } const taskRetryIntentSchema = z.object({