From 25273aa69bf9a580f68fd8287e1e37dfb64ba51e Mon Sep 17 00:00:00 2001 From: Devin Foley Date: Sat, 7 Nov 2026 11:29:23 -0800 Subject: [PATCH] fix(onboarding): start the agent when the opening question is answered (#19886) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A new company starts with an onboarding first task. The task opens with a greeting and an opening question card that the assignee agent posts. > - Onboarding does not wake the agent when it creates the task. The user's answer must start the agent. > - When the user clicks an answer, run dispatch cancels the agent's run 0.4 seconds later with `issue_not_in_progress`. Answer continuations only start on `in_progress` or `in_review` tasks, and the onboarding task is still `todo`. > - The user sees nothing until an unrelated bounded retry starts the agent about two minutes later. This is the first thing every new user does. > - This pull request lets an answer start a `todo` task when the task has never started and no run asked the question. > - The benefit is that the agent starts immediately after the first answer. The protection for parked tasks does not change. ## Linked Issues or Issue Description **What happened?** I created a new company and clicked an option on the onboarding opening question ("What would you like to do?"). The agent's run was cancelled immediately with `issue_not_in_progress`: "Cancelled because resolved-interaction continuation issue is no longer in_progress (current status: todo) before the queued run could start". The task stayed in `todo`. About two minutes later, a `transient_failure` bounded retry of the cancelled run started the agent through the normal checkout path. **Expected behavior** The agent starts as soon as the user answers the opening question. **Steps to reproduce** 1. Create a new company and hire the first agent through onboarding. 2. Open the onboarding first task. 3. Click an option on the opening question card and submit. 4. Look at the agent's runs. The first run is `cancelled` with `issue_not_in_progress`. **Paperclip version or commit** `master` at `86bc3db6938`. The same code is on `00ad9503fe3`. **Relevant logs or output** ``` heartbeat_runs: status=cancelled error_code=issue_not_in_progress "Cancelled because resolved-interaction continuation issue is no longer in_progress (current status: todo) before the queued run could start" two minutes later: "Scheduled bounded retry 1/2" (retryReason transient_failure) ``` Related: #13068 added the onboarding opening question. #13318 made answer continuations require `in_progress` or `in_review`. ## What Changed - New `server/src/services/unstarted-task-question-answer.ts`. `isUnstartedTaskQuestionAnswer` returns true only when all of these are true: - the interaction is an answered question (`ask_user_questions`, status `answered`) on this task, and it has no `sourceRunId` (no run asked it); - the task is `todo`; - the task's `startedAt` is null. It reads only server-owned rows. `startedAt` is never cleared, so a task that started and was later moved back to `todo` does not match. - `run-dispatch/adapters/postgres.ts`: the staleness facts include `isUnstartedTaskQuestionAnswer`. The adapter reads it under the issue lock, and only for a resolved-interaction continuation on a `todo` task. - `run-dispatch/domain/policy.ts`: `decideQueuedRunStaleness` lets such a continuation start from `todo`. The other answer continuations still require `in_progress` or `in_review`. - `services/heartbeat.ts`: the final continuation checkout adds `todo` to the expected statuses only for the same case. The task then moves to `in_progress` in the same guarded update. ## Verification - `heartbeat-stale-queue-invalidation.test.ts` (embedded Postgres): - New test: an answer to a question that no run asked, on a never-started `todo` task, starts the assignee. The task moves to `in_progress` under the run's execution lock. With the fix reverted, this test fails with the reported error (`cancelled`, `issue_not_in_progress`). - New test: three regression cases stay cancelled with `issue_not_in_progress`: - an answer on a task that started and was moved back to `todo`; - an answer to a question that an agent run asked; - a rejected confirmation that no run asked. Without the kind and status check, the confirmation case fails. - `policy.test.ts`: the new fact allows `todo`. The same fact on a `backlog` task is still cancelled. - `pnpm --filter @paperclipai/server typecheck`: exit 0. - `npx vitest run` on these suites: 8 files, 497 tests passed. - stale-queue invalidation - run-dispatch policy - onboarding first-task routes - onboarding first-task admission - question-response delivery - issue-thread interaction routes - heartbeat retry scheduling - legacy continuation authority ## Risks - Low risk. The change only affects answer continuations on `todo` tasks that have never started, where the question has no source run. All other continuations behave as before. - The exception is not limited to the onboarding card. Any answered question that no run asked, on a never-started `todo` task, can now start the assignee. This matches what a plain comment wake does on a `todo` task. Confirmations and other interaction kinds keep the existing rule. For example, rejecting a confirmation that no run asked still does not start the task. - The final checkout does one extra indexed read. It only runs for resolved-interaction continuations on `todo` tasks. - I did not find which component scheduled the two-minute `transient_failure` retry. After this change, onboarding does not depend on that retry. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use (shell, file edits, test runs). Investigation, fix, and tests were AI-assisted, with human direction. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] I have updated relevant documentation to reflect my changes (no docs describe this behavior) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- ...heartbeat-stale-queue-invalidation.test.ts | 119 ++++++++++++++++++ .../modules/run-dispatch/adapters/postgres.ts | 3 + .../run-dispatch/domain/policy.test.ts | 18 +++ .../src/modules/run-dispatch/domain/policy.ts | 6 +- server/src/services/heartbeat.ts | 8 +- .../unstarted-task-question-answer.ts | 41 ++++++ 6 files changed, 192 insertions(+), 3 deletions(-) create mode 100644 server/src/services/unstarted-task-question-answer.ts diff --git a/server/src/__tests__/heartbeat-stale-queue-invalidation.test.ts b/server/src/__tests__/heartbeat-stale-queue-invalidation.test.ts index fe0d043aefa..a23dff7c4f9 100644 --- a/server/src/__tests__/heartbeat-stale-queue-invalidation.test.ts +++ b/server/src/__tests__/heartbeat-stale-queue-invalidation.test.ts @@ -1866,4 +1866,123 @@ describeEmbeddedPostgres("heartbeat stale queued-run invalidation", () => { expect(countExecuteCallsForRun(runId)).toBe(1); expect(claimedIssue).toEqual({ status: "in_progress", executionRunId: runId }); }); + + async function seedAnsweredQuestion(input: { + companyId: string; + issueId: string; + agentId: string; + sourceRunId: string | null; + rejectedConfirmation?: boolean; + }) { + const interactionId = randomUUID(); + const resolved = { + id: interactionId, + companyId: input.companyId, + issueId: input.issueId, + continuationPolicy: "wake_assignee", + createdByAgentId: input.agentId, + sourceRunId: input.sourceRunId, + resolvedByUserId: "responsible-user", + resolvedAt: new Date(), + }; + await db.insert(issueThreadInteractions).values(input.rejectedConfirmation + ? { + ...resolved, + kind: "request_confirmation", + status: "rejected", + payload: { version: 1, prompt: "Approve the plan?" }, + result: { version: 1, outcome: "rejected", reason: "Needs more work" }, + } + : { + ...resolved, + kind: "ask_user_questions", + status: "answered", + payload: { version: 1, questions: [{ id: "first-task-opening", prompt: "What would you like to do?", selectionMode: "single", + required: true, options: [{ id: "interview", label: "Interview me" }] }] }, + result: { version: 1, answers: [{ questionId: "first-task-opening", optionIds: ["interview"] }] }, + }); + // Mirrors the context question-response delivery queues for an answer. + return { + interactionId, + interactionKind: input.rejectedConfirmation ? "request_confirmation" : "ask_user_questions", + interactionStatus: input.rejectedConfirmation ? "rejected" : "answered", + mutation: "interaction", + sourceRunId: input.sourceRunId, + externalChatContinuation: false, + source: "issue.interaction.respond", + }; + } + + it("starts the assignee when a question no run asked is answered on a todo task that has never started", async () => { + const { companyId, agentId } = await seedCompanyAndAgent(); + const issueId = randomUUID(); + let answeredRunId: string | undefined; + let claimedIssue: { status: string; executionRunId: string | null } | null = null; + afterContinuationDispatchCheck = async ({ runId: checkedRunId, issueId: checkedIssueId }) => { + if (checkedIssueId !== issueId || checkedRunId !== answeredRunId) return; + claimedIssue = await db + .select({ status: issues.status, executionRunId: issues.executionRunId }) + .from(issues) + .where(eq(issues.id, issueId)) + .then((rows) => rows[0] ?? null); + }; + await db.insert(issues).values({ id: issueId, companyId, title: "Onboarding first task", status: "todo", priority: "medium", assigneeAgentId: agentId }); + const contextExtras = await seedAnsweredQuestion({ companyId, issueId, agentId, sourceRunId: null }); + const { runId } = await seedQueuedRun({ companyId, agentId, issueId, wakeReason: "issue_commented", + invocationSource: "automation", contextExtras }); + answeredRunId = runId; + + await heartbeat.resumeQueuedRuns(); + await vi.waitFor(async () => { + const current = await heartbeat.getRun(runId); + expect(countExecuteCallsForRun(runId), JSON.stringify({ + status: current?.status, errorCode: current?.errorCode, + })).toBe(1); + }, { timeout: 5_000 }); + await heartbeat.waitForRunExecutionDrain(runId); + + expect((await heartbeat.getRun(runId))?.status).toBe("succeeded"); + expect(claimedIssue).toEqual({ status: "in_progress", executionRunId: runId }); + expect((await db.select({ startedAt: issues.startedAt }).from(issues).where(eq(issues.id, issueId)))[0]?.startedAt) + .toBeInstanceOf(Date); + }); + + it.each([ + { name: "a started task parked back in todo", startedAt: new Date("2026-01-01T00:00:00.000Z"), askedByRun: false, rejectedConfirmation: false }, + { name: "a question an agent run asked", startedAt: null, askedByRun: true, rejectedConfirmation: false }, + { name: "a rejected confirmation no run asked", startedAt: null, askedByRun: false, rejectedConfirmation: true }, + ])("keeps cancelling an answer continuation for $name", async ({ startedAt, askedByRun, rejectedConfirmation }) => { + const { companyId, agentId } = await seedCompanyAndAgent(); + const issueId = randomUUID(); + await db.insert(issues).values({ id: issueId, companyId, title: "Parked or agent-asked", status: "todo", priority: "medium", + assigneeAgentId: agentId, startedAt }); + const [sourceRun] = askedByRun + ? await db.insert(heartbeatRuns).values({ companyId, agentId, invocationSource: "assignment", status: "succeeded", + startedAt: new Date(), finishedAt: new Date(), contextSnapshot: { issueId } }).returning() + : [null]; + const contextExtras = await seedAnsweredQuestion({ companyId, issueId, agentId, sourceRunId: sourceRun?.id ?? null, + rejectedConfirmation }); + const { runId, wakeupRequestId } = await seedQueuedRun({ companyId, agentId, issueId, wakeReason: "issue_commented", + invocationSource: "automation", contextExtras }); + + await heartbeat.resumeQueuedRuns(); + await heartbeat.drainActiveRunExecutions(); + + const [run, wakeup, issue] = await Promise.all([ + db.select({ status: heartbeatRuns.status, errorCode: heartbeatRuns.errorCode }) + .from(heartbeatRuns) + .where(eq(heartbeatRuns.id, runId)) + .then((rows) => rows[0] ?? null), + db.select({ status: agentWakeupRequests.status, error: agentWakeupRequests.error }) + .from(agentWakeupRequests) + .where(eq(agentWakeupRequests.id, wakeupRequestId)) + .then((rows) => rows[0] ?? null), + db.select({ status: issues.status }).from(issues).where(eq(issues.id, issueId)) + .then((rows) => rows[0] ?? null), + ]); + expect(run).toMatchObject({ status: "cancelled", errorCode: "issue_not_in_progress" }); + expect(wakeup).toMatchObject({ status: "skipped", error: expect.stringContaining("no longer in_progress") }); + expect(issue?.status).toBe("todo"); + expect(countExecuteCallsForRun(runId)).toBe(0); + }); }); diff --git a/server/src/modules/run-dispatch/adapters/postgres.ts b/server/src/modules/run-dispatch/adapters/postgres.ts index 86aee36ca77..0dda710dadf 100644 --- a/server/src/modules/run-dispatch/adapters/postgres.ts +++ b/server/src/modules/run-dispatch/adapters/postgres.ts @@ -41,6 +41,7 @@ import { evaluateAgentInvokabilityFromDb } from "../../../services/agent-invokab import { budgetService, budgetServiceInTransaction, recheckAdmittedRunBudgetReadOnly } from "../../../services/budgets.js"; import { lockRunBudgetRecoveryOwnerInTransaction, recheckRunBudgetReservationInTransaction } from "../../../services/budget-reservations.js"; import { isCompletedOnboardingHandoffWake } from "../../../services/chat-completion-delivery.js"; +import { isUnstartedTaskQuestionAnswer } from "../../../services/unstarted-task-question-answer.js"; import { isHeartbeatWakeOnDemandEnabled } from "../../../services/heartbeat-policy.js"; import { notifyChatRunMilestones } from "../../../services/chat-run-milestone-notifications.js"; import { DELIVERY_QUEUES, notifyDeliveryWork } from "../../../services/delivery-work-notifications.js"; @@ -666,6 +667,8 @@ export function createPostgresRunDispatchAdapter( issueExecutionRunId: issue?.executionRunId ?? null, issueCheckoutRunId: issue?.checkoutRunId ?? null, isResolvedInteractionContinuation, + isUnstartedTaskQuestionAnswer: Boolean(isResolvedInteractionContinuation && issue?.status === "todo" && + await isUnstartedTaskQuestionAnswer(dbOrTx, { companyId: input.companyId, issueId, interactionId: context.interactionId })), isConnectionContinuation: (isResolvedInteractionContinuation && context.interactionKind === "connection_intent") || context.source === "connection_tools.refreshed", isInteractionWake, diff --git a/server/src/modules/run-dispatch/domain/policy.test.ts b/server/src/modules/run-dispatch/domain/policy.test.ts index 051c676d2c5..8fc730f76b4 100644 --- a/server/src/modules/run-dispatch/domain/policy.test.ts +++ b/server/src/modules/run-dispatch/domain/policy.test.ts @@ -302,6 +302,15 @@ describe("decideQueuedRunStaleness", () => { } satisfies Partial, errorCode: "issue_not_in_progress", }, + { + name: "issue_not_in_progress (resolved-interaction continuation, unstarted answer on a non-todo task)", + overrides: { + isResolvedInteractionContinuation: true, + isUnstartedTaskQuestionAnswer: true, + issueStatus: "backlog", + } satisfies Partial, + errorCode: "issue_not_in_progress", + }, { name: "issue_not_in_progress (max-turn continuation)", overrides: { @@ -399,6 +408,15 @@ describe("decideQueuedRunStaleness", () => { }, NOW)).toEqual({ stale: false }); }); + it("allows an answer to a question no run asked to start a todo task that has never started", () => { + expect(decideQueuedRunStaleness({ + ...baseStalenessFacts(), + isResolvedInteractionContinuation: true, + isUnstartedTaskQuestionAnswer: true, + issueStatus: "todo", + }, NOW)).toEqual({ stale: false }); + }); + it("does not cancel a parked continuation summary when the classifier says it does not park the executor", () => { const facts: QueuedRunFacts = { ...baseStalenessFacts(), diff --git a/server/src/modules/run-dispatch/domain/policy.ts b/server/src/modules/run-dispatch/domain/policy.ts index 9c26cba3eb1..59d955278cd 100644 --- a/server/src/modules/run-dispatch/domain/policy.ts +++ b/server/src/modules/run-dispatch/domain/policy.ts @@ -145,6 +145,8 @@ export type QueuedRunFacts = { issueCheckoutRunId?: string | null; isResolvedInteractionContinuation: boolean; + /** Verified under the issue lock: an answer to a question no run asked, on a todo task that has never started. */ + isUnstartedTaskQuestionAnswer?: boolean; /** A connection resolution or tool refresh can resume an agent waiting in review. */ isConnectionContinuation?: boolean; isInteractionWake: boolean; @@ -510,9 +512,11 @@ export function decideQueuedRunStaleness( } if (facts.isResolvedInteractionContinuation || facts.isConnectionContinuation) { + // A todo task that has never started has no parked execution to protect. + const startsUnstartedTask = facts.issueStatus === "todo" && facts.isUnstartedTaskQuestionAnswer === true; const earlyStatus = decideIssueStatus({ status: facts.issueStatus, - requiresInProgress: facts.issueStatus !== "in_review", + requiresInProgress: facts.issueStatus !== "in_review" && !startsUnstartedTask, terminalBypass: true, }); if (earlyStatus === "not_in_progress") { diff --git a/server/src/services/heartbeat.ts b/server/src/services/heartbeat.ts index 9c85c7515c5..43496f50eb8 100644 --- a/server/src/services/heartbeat.ts +++ b/server/src/services/heartbeat.ts @@ -1,5 +1,6 @@ import { resolveProviderApiPlacement } from "./native-runtime/provider-api-placement.js"; import { isContextCompactionRun } from "./context-compaction.js"; +import { isUnstartedTaskQuestionAnswer } from "./unstarted-task-question-answer.js"; import { registerDeferredReceiptWork } from "./heartbeat/deferred-receipt-work.js"; import { reconcileTaskChatRetryReceipts } from "./task-retry.js"; import { registerIssueRecoveryWork } from "./recovery/work.js"; @@ -2216,11 +2217,14 @@ export function heartbeatService( // Claim the issue under the same active-status predicate used by the // queued-run staleness gate. This is the final atomic guard before // dispatch: an operator parking the issue after claim but before this - // checkout must not be overwritten by the continuation. + // checkout must not be overwritten by the continuation. A todo task + // that has never started has nothing parked, so it may start here. + const startsUnstartedTask = issueContext.status === "todo" && + await isUnstartedTaskQuestionAnswer(db, { companyId: agent.companyId, issueId, interactionId: context.interactionId }); await issuesSvc.checkout( issueId, agent.id, - [...resolvedInteractionCheckoutExpectedStatuses()], + [...resolvedInteractionCheckoutExpectedStatuses(), ...(startsUnstartedTask ? ["todo"] : [])], run.id, ); context[PAPERCLIP_HARNESS_CHECKOUT_KEY] = true; diff --git a/server/src/services/unstarted-task-question-answer.ts b/server/src/services/unstarted-task-question-answer.ts new file mode 100644 index 00000000000..15be9115ab0 --- /dev/null +++ b/server/src/services/unstarted-task-question-answer.ts @@ -0,0 +1,41 @@ +import { and, eq, isNull } from "drizzle-orm"; +import { issueThreadInteractions, issues, type Db } from "@paperclipai/db"; +import { isUuidLike } from "@paperclipai/shared"; + +/** + * A question that no run asked, such as the onboarding opening card, can be + * answered before its task has ever started. Answer continuations normally + * require in_progress or in_review so an operator who parked the task wins. + * A todo task that has never started has no parked execution to protect, so + * its answer may check the task out from todo. + * + * Reads only server-owned rows: the interaction must be an answered question + * on this task with no source run, and the task must still be todo with no + * startedAt. Confirmations and other interaction kinds keep the stricter rule. + */ +export async function isUnstartedTaskQuestionAnswer( + db: Db, + input: { companyId: string; issueId: string; interactionId: unknown }, +): Promise { + const interactionId = typeof input.interactionId === "string" ? input.interactionId : null; + if (!interactionId || !isUuidLike(interactionId)) return false; + const rows = await db + .select({ id: issueThreadInteractions.id }) + .from(issueThreadInteractions) + .innerJoin(issues, and( + eq(issues.id, issueThreadInteractions.issueId), + eq(issues.companyId, issueThreadInteractions.companyId), + )) + .where(and( + eq(issueThreadInteractions.id, interactionId), + eq(issueThreadInteractions.companyId, input.companyId), + eq(issueThreadInteractions.issueId, input.issueId), + eq(issueThreadInteractions.kind, "ask_user_questions"), + eq(issueThreadInteractions.status, "answered"), + isNull(issueThreadInteractions.sourceRunId), + eq(issues.status, "todo"), + isNull(issues.startedAt), + )) + .limit(1); + return rows.length > 0; +}