mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-10 12:37:55 +01:00
fix(onboarding): start the agent when the opening question is answered (#19886)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - A new company starts with an onboarding first task. The task opens
with a greeting and an opening question card that the assignee agent
posts.
> - Onboarding does not wake the agent when it creates the task. The
user's answer must start the agent.
> - When the user clicks an answer, run dispatch cancels the agent's run
0.4 seconds later with `issue_not_in_progress`. Answer continuations
only start on `in_progress` or `in_review` tasks, and the onboarding
task is still `todo`.
> - The user sees nothing until an unrelated bounded retry starts the
agent about two minutes later. This is the first thing every new user
does.
> - This pull request lets an answer start a `todo` task when the task
has never started and no run asked the question.
> - The benefit is that the agent starts immediately after the first
answer. The protection for parked tasks does not change.
## Linked Issues or Issue Description
**What happened?**
I created a new company and clicked an option on the onboarding opening
question ("What would you like to do?"). The agent's run was cancelled
immediately with `issue_not_in_progress`: "Cancelled because
resolved-interaction continuation issue is no longer in_progress
(current status: todo) before the queued run could start". The task
stayed in `todo`. About two minutes later, a `transient_failure` bounded
retry of the cancelled run started the agent through the normal checkout
path.
**Expected behavior**
The agent starts as soon as the user answers the opening question.
**Steps to reproduce**
1. Create a new company and hire the first agent through onboarding.
2. Open the onboarding first task.
3. Click an option on the opening question card and submit.
4. Look at the agent's runs. The first run is `cancelled` with
`issue_not_in_progress`.
**Paperclip version or commit**
`master` at `86bc3db6938`. The same code is on `00ad9503fe3`.
**Relevant logs or output**
```
heartbeat_runs: status=cancelled error_code=issue_not_in_progress
"Cancelled because resolved-interaction continuation issue is no longer in_progress (current status: todo) before the queued run could start"
two minutes later: "Scheduled bounded retry 1/2" (retryReason transient_failure)
```
Related: #13068 added the onboarding opening question. #13318 made
answer continuations require `in_progress` or `in_review`.
## What Changed
- New `server/src/services/unstarted-task-question-answer.ts`.
`isUnstartedTaskQuestionAnswer` returns true only when all of these are
true:
- the interaction is an answered question (`ask_user_questions`, status
`answered`) on this task, and it has no `sourceRunId` (no run asked it);
- the task is `todo`;
- the task's `startedAt` is null.
It reads only server-owned rows. `startedAt` is never cleared, so a task
that started and was later moved back to `todo` does not match.
- `run-dispatch/adapters/postgres.ts`: the staleness facts include
`isUnstartedTaskQuestionAnswer`. The adapter reads it under the issue
lock, and only for a resolved-interaction continuation on a `todo` task.
- `run-dispatch/domain/policy.ts`: `decideQueuedRunStaleness` lets such
a continuation start from `todo`. The other answer continuations still
require `in_progress` or `in_review`.
- `services/heartbeat.ts`: the final continuation checkout adds `todo`
to the expected statuses only for the same case. The task then moves to
`in_progress` in the same guarded update.
## Verification
- `heartbeat-stale-queue-invalidation.test.ts` (embedded Postgres):
- New test: an answer to a question that no run asked, on a
never-started `todo` task, starts the assignee. The task moves to
`in_progress` under the run's execution lock. With the fix reverted,
this test fails with the reported error (`cancelled`,
`issue_not_in_progress`).
- New test: three regression cases stay cancelled with
`issue_not_in_progress`:
- an answer on a task that started and was moved back to `todo`;
- an answer to a question that an agent run asked;
- a rejected confirmation that no run asked.
Without the kind and status check, the confirmation case fails.
- `policy.test.ts`: the new fact allows `todo`. The same fact on a
`backlog` task is still cancelled.
- `pnpm --filter @paperclipai/server typecheck`: exit 0.
- `npx vitest run` on these suites: 8 files, 497 tests passed.
- stale-queue invalidation
- run-dispatch policy
- onboarding first-task routes
- onboarding first-task admission
- question-response delivery
- issue-thread interaction routes
- heartbeat retry scheduling
- legacy continuation authority
## Risks
- Low risk. The change only affects answer continuations on `todo` tasks
that have never started, where the question has no source run. All other
continuations behave as before.
- The exception is not limited to the onboarding card. Any answered
question that no run asked, on a never-started `todo` task, can now
start the assignee. This matches what a plain comment wake does on a
`todo` task. Confirmations and other interaction kinds keep the existing
rule. For example, rejecting a confirmation that no run asked still does
not start the task.
- The final checkout does one extra indexed read. It only runs for
resolved-interaction continuations on `todo` tasks.
- I did not find which component scheduled the two-minute
`transient_failure` retry. After this change, onboarding does not depend
on that retry.
> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, test runs). Investigation, fix, and tests were
AI-assisted, with human direction.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] I have updated relevant documentation to reflect my changes (no
docs describe this behavior)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
f439150cb3
commit
25273aa69b
6 files changed
+192
-3
No files matched your search
@@ -1866,4 +1866,123 @@ describeEmbeddedPostgres("heartbeat stale queued-run invalidation", () => {
|
||||
expect(countExecuteCallsForRun(runId)).toBe(1);
|
||||
expect(claimedIssue).toEqual({ status: "in_progress", executionRunId: runId });
|
||||
});
|
||||
|
||||
async function seedAnsweredQuestion(input: {
|
||||
companyId: string;
|
||||
issueId: string;
|
||||
agentId: string;
|
||||
sourceRunId: string | null;
|
||||
rejectedConfirmation?: boolean;
|
||||
}) {
|
||||
const interactionId = randomUUID();
|
||||
const resolved = {
|
||||
id: interactionId,
|
||||
companyId: input.companyId,
|
||||
issueId: input.issueId,
|
||||
continuationPolicy: "wake_assignee",
|
||||
createdByAgentId: input.agentId,
|
||||
sourceRunId: input.sourceRunId,
|
||||
resolvedByUserId: "responsible-user",
|
||||
resolvedAt: new Date(),
|
||||
};
|
||||
await db.insert(issueThreadInteractions).values(input.rejectedConfirmation
|
||||
? {
|
||||
...resolved,
|
||||
kind: "request_confirmation",
|
||||
status: "rejected",
|
||||
payload: { version: 1, prompt: "Approve the plan?" },
|
||||
result: { version: 1, outcome: "rejected", reason: "Needs more work" },
|
||||
}
|
||||
: {
|
||||
...resolved,
|
||||
kind: "ask_user_questions",
|
||||
status: "answered",
|
||||
payload: { version: 1, questions: [{ id: "first-task-opening", prompt: "What would you like to do?", selectionMode: "single",
|
||||
required: true, options: [{ id: "interview", label: "Interview me" }] }] },
|
||||
result: { version: 1, answers: [{ questionId: "first-task-opening", optionIds: ["interview"] }] },
|
||||
});
|
||||
// Mirrors the context question-response delivery queues for an answer.
|
||||
return {
|
||||
interactionId,
|
||||
interactionKind: input.rejectedConfirmation ? "request_confirmation" : "ask_user_questions",
|
||||
interactionStatus: input.rejectedConfirmation ? "rejected" : "answered",
|
||||
mutation: "interaction",
|
||||
sourceRunId: input.sourceRunId,
|
||||
externalChatContinuation: false,
|
||||
source: "issue.interaction.respond",
|
||||
};
|
||||
}
|
||||
|
||||
it("starts the assignee when a question no run asked is answered on a todo task that has never started", async () => {
|
||||
const { companyId, agentId } = await seedCompanyAndAgent();
|
||||
const issueId = randomUUID();
|
||||
let answeredRunId: string | undefined;
|
||||
let claimedIssue: { status: string; executionRunId: string | null } | null = null;
|
||||
afterContinuationDispatchCheck = async ({ runId: checkedRunId, issueId: checkedIssueId }) => {
|
||||
if (checkedIssueId !== issueId || checkedRunId !== answeredRunId) return;
|
||||
claimedIssue = await db
|
||||
.select({ status: issues.status, executionRunId: issues.executionRunId })
|
||||
.from(issues)
|
||||
.where(eq(issues.id, issueId))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
};
|
||||
await db.insert(issues).values({ id: issueId, companyId, title: "Onboarding first task", status: "todo", priority: "medium", assigneeAgentId: agentId });
|
||||
const contextExtras = await seedAnsweredQuestion({ companyId, issueId, agentId, sourceRunId: null });
|
||||
const { runId } = await seedQueuedRun({ companyId, agentId, issueId, wakeReason: "issue_commented",
|
||||
invocationSource: "automation", contextExtras });
|
||||
answeredRunId = runId;
|
||||
|
||||
await heartbeat.resumeQueuedRuns();
|
||||
await vi.waitFor(async () => {
|
||||
const current = await heartbeat.getRun(runId);
|
||||
expect(countExecuteCallsForRun(runId), JSON.stringify({
|
||||
status: current?.status, errorCode: current?.errorCode,
|
||||
})).toBe(1);
|
||||
}, { timeout: 5_000 });
|
||||
await heartbeat.waitForRunExecutionDrain(runId);
|
||||
|
||||
expect((await heartbeat.getRun(runId))?.status).toBe("succeeded");
|
||||
expect(claimedIssue).toEqual({ status: "in_progress", executionRunId: runId });
|
||||
expect((await db.select({ startedAt: issues.startedAt }).from(issues).where(eq(issues.id, issueId)))[0]?.startedAt)
|
||||
.toBeInstanceOf(Date);
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ name: "a started task parked back in todo", startedAt: new Date("2026-01-01T00:00:00.000Z"), askedByRun: false, rejectedConfirmation: false },
|
||||
{ name: "a question an agent run asked", startedAt: null, askedByRun: true, rejectedConfirmation: false },
|
||||
{ name: "a rejected confirmation no run asked", startedAt: null, askedByRun: false, rejectedConfirmation: true },
|
||||
])("keeps cancelling an answer continuation for $name", async ({ startedAt, askedByRun, rejectedConfirmation }) => {
|
||||
const { companyId, agentId } = await seedCompanyAndAgent();
|
||||
const issueId = randomUUID();
|
||||
await db.insert(issues).values({ id: issueId, companyId, title: "Parked or agent-asked", status: "todo", priority: "medium",
|
||||
assigneeAgentId: agentId, startedAt });
|
||||
const [sourceRun] = askedByRun
|
||||
? await db.insert(heartbeatRuns).values({ companyId, agentId, invocationSource: "assignment", status: "succeeded",
|
||||
startedAt: new Date(), finishedAt: new Date(), contextSnapshot: { issueId } }).returning()
|
||||
: [null];
|
||||
const contextExtras = await seedAnsweredQuestion({ companyId, issueId, agentId, sourceRunId: sourceRun?.id ?? null,
|
||||
rejectedConfirmation });
|
||||
const { runId, wakeupRequestId } = await seedQueuedRun({ companyId, agentId, issueId, wakeReason: "issue_commented",
|
||||
invocationSource: "automation", contextExtras });
|
||||
|
||||
await heartbeat.resumeQueuedRuns();
|
||||
await heartbeat.drainActiveRunExecutions();
|
||||
|
||||
const [run, wakeup, issue] = await Promise.all([
|
||||
db.select({ status: heartbeatRuns.status, errorCode: heartbeatRuns.errorCode })
|
||||
.from(heartbeatRuns)
|
||||
.where(eq(heartbeatRuns.id, runId))
|
||||
.then((rows) => rows[0] ?? null),
|
||||
db.select({ status: agentWakeupRequests.status, error: agentWakeupRequests.error })
|
||||
.from(agentWakeupRequests)
|
||||
.where(eq(agentWakeupRequests.id, wakeupRequestId))
|
||||
.then((rows) => rows[0] ?? null),
|
||||
db.select({ status: issues.status }).from(issues).where(eq(issues.id, issueId))
|
||||
.then((rows) => rows[0] ?? null),
|
||||
]);
|
||||
expect(run).toMatchObject({ status: "cancelled", errorCode: "issue_not_in_progress" });
|
||||
expect(wakeup).toMatchObject({ status: "skipped", error: expect.stringContaining("no longer in_progress") });
|
||||
expect(issue?.status).toBe("todo");
|
||||
expect(countExecuteCallsForRun(runId)).toBe(0);
|
||||
});
|
||||
});
|
||||
@@ -41,6 +41,7 @@ import { evaluateAgentInvokabilityFromDb } from "../../../services/agent-invokab
|
||||
import { budgetService, budgetServiceInTransaction, recheckAdmittedRunBudgetReadOnly } from "../../../services/budgets.js";
|
||||
import { lockRunBudgetRecoveryOwnerInTransaction, recheckRunBudgetReservationInTransaction } from "../../../services/budget-reservations.js";
|
||||
import { isCompletedOnboardingHandoffWake } from "../../../services/chat-completion-delivery.js";
|
||||
import { isUnstartedTaskQuestionAnswer } from "../../../services/unstarted-task-question-answer.js";
|
||||
import { isHeartbeatWakeOnDemandEnabled } from "../../../services/heartbeat-policy.js";
|
||||
import { notifyChatRunMilestones } from "../../../services/chat-run-milestone-notifications.js";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "../../../services/delivery-work-notifications.js";
|
||||
@@ -666,6 +667,8 @@ export function createPostgresRunDispatchAdapter(
|
||||
issueExecutionRunId: issue?.executionRunId ?? null,
|
||||
issueCheckoutRunId: issue?.checkoutRunId ?? null,
|
||||
isResolvedInteractionContinuation,
|
||||
isUnstartedTaskQuestionAnswer: Boolean(isResolvedInteractionContinuation && issue?.status === "todo" &&
|
||||
await isUnstartedTaskQuestionAnswer(dbOrTx, { companyId: input.companyId, issueId, interactionId: context.interactionId })),
|
||||
isConnectionContinuation: (isResolvedInteractionContinuation && context.interactionKind === "connection_intent")
|
||||
|| context.source === "connection_tools.refreshed",
|
||||
isInteractionWake,
|
||||
|
||||
@@ -302,6 +302,15 @@ describe("decideQueuedRunStaleness", () => {
|
||||
} satisfies Partial<QueuedRunFacts>,
|
||||
errorCode: "issue_not_in_progress",
|
||||
},
|
||||
{
|
||||
name: "issue_not_in_progress (resolved-interaction continuation, unstarted answer on a non-todo task)",
|
||||
overrides: {
|
||||
isResolvedInteractionContinuation: true,
|
||||
isUnstartedTaskQuestionAnswer: true,
|
||||
issueStatus: "backlog",
|
||||
} satisfies Partial<QueuedRunFacts>,
|
||||
errorCode: "issue_not_in_progress",
|
||||
},
|
||||
{
|
||||
name: "issue_not_in_progress (max-turn continuation)",
|
||||
overrides: {
|
||||
@@ -399,6 +408,15 @@ describe("decideQueuedRunStaleness", () => {
|
||||
}, NOW)).toEqual({ stale: false });
|
||||
});
|
||||
|
||||
it("allows an answer to a question no run asked to start a todo task that has never started", () => {
|
||||
expect(decideQueuedRunStaleness({
|
||||
...baseStalenessFacts(),
|
||||
isResolvedInteractionContinuation: true,
|
||||
isUnstartedTaskQuestionAnswer: true,
|
||||
issueStatus: "todo",
|
||||
}, NOW)).toEqual({ stale: false });
|
||||
});
|
||||
|
||||
it("does not cancel a parked continuation summary when the classifier says it does not park the executor", () => {
|
||||
const facts: QueuedRunFacts = {
|
||||
...baseStalenessFacts(),
|
||||
|
||||
@@ -145,6 +145,8 @@ export type QueuedRunFacts = {
|
||||
issueCheckoutRunId?: string | null;
|
||||
|
||||
isResolvedInteractionContinuation: boolean;
|
||||
/** Verified under the issue lock: an answer to a question no run asked, on a todo task that has never started. */
|
||||
isUnstartedTaskQuestionAnswer?: boolean;
|
||||
/** A connection resolution or tool refresh can resume an agent waiting in review. */
|
||||
isConnectionContinuation?: boolean;
|
||||
isInteractionWake: boolean;
|
||||
@@ -510,9 +512,11 @@ export function decideQueuedRunStaleness(
|
||||
}
|
||||
|
||||
if (facts.isResolvedInteractionContinuation || facts.isConnectionContinuation) {
|
||||
// A todo task that has never started has no parked execution to protect.
|
||||
const startsUnstartedTask = facts.issueStatus === "todo" && facts.isUnstartedTaskQuestionAnswer === true;
|
||||
const earlyStatus = decideIssueStatus({
|
||||
status: facts.issueStatus,
|
||||
requiresInProgress: facts.issueStatus !== "in_review",
|
||||
requiresInProgress: facts.issueStatus !== "in_review" && !startsUnstartedTask,
|
||||
terminalBypass: true,
|
||||
});
|
||||
if (earlyStatus === "not_in_progress") {
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { resolveProviderApiPlacement } from "./native-runtime/provider-api-placement.js";
|
||||
import { isContextCompactionRun } from "./context-compaction.js";
|
||||
import { isUnstartedTaskQuestionAnswer } from "./unstarted-task-question-answer.js";
|
||||
import { registerDeferredReceiptWork } from "./heartbeat/deferred-receipt-work.js";
|
||||
import { reconcileTaskChatRetryReceipts } from "./task-retry.js";
|
||||
import { registerIssueRecoveryWork } from "./recovery/work.js";
|
||||
@@ -2216,11 +2217,14 @@ export function heartbeatService(
|
||||
// Claim the issue under the same active-status predicate used by the
|
||||
// queued-run staleness gate. This is the final atomic guard before
|
||||
// dispatch: an operator parking the issue after claim but before this
|
||||
// checkout must not be overwritten by the continuation.
|
||||
// checkout must not be overwritten by the continuation. A todo task
|
||||
// that has never started has nothing parked, so it may start here.
|
||||
const startsUnstartedTask = issueContext.status === "todo" &&
|
||||
await isUnstartedTaskQuestionAnswer(db, { companyId: agent.companyId, issueId, interactionId: context.interactionId });
|
||||
await issuesSvc.checkout(
|
||||
issueId,
|
||||
agent.id,
|
||||
[...resolvedInteractionCheckoutExpectedStatuses()],
|
||||
[...resolvedInteractionCheckoutExpectedStatuses(), ...(startsUnstartedTask ? ["todo"] : [])],
|
||||
run.id,
|
||||
);
|
||||
context[PAPERCLIP_HARNESS_CHECKOUT_KEY] = true;
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
import { and, eq, isNull } from "drizzle-orm";
|
||||
import { issueThreadInteractions, issues, type Db } from "@paperclipai/db";
|
||||
import { isUuidLike } from "@paperclipai/shared";
|
||||
|
||||
/**
|
||||
* A question that no run asked, such as the onboarding opening card, can be
|
||||
* answered before its task has ever started. Answer continuations normally
|
||||
* require in_progress or in_review so an operator who parked the task wins.
|
||||
* A todo task that has never started has no parked execution to protect, so
|
||||
* its answer may check the task out from todo.
|
||||
*
|
||||
* Reads only server-owned rows: the interaction must be an answered question
|
||||
* on this task with no source run, and the task must still be todo with no
|
||||
* startedAt. Confirmations and other interaction kinds keep the stricter rule.
|
||||
*/
|
||||
export async function isUnstartedTaskQuestionAnswer(
|
||||
db: Db,
|
||||
input: { companyId: string; issueId: string; interactionId: unknown },
|
||||
): Promise<boolean> {
|
||||
const interactionId = typeof input.interactionId === "string" ? input.interactionId : null;
|
||||
if (!interactionId || !isUuidLike(interactionId)) return false;
|
||||
const rows = await db
|
||||
.select({ id: issueThreadInteractions.id })
|
||||
.from(issueThreadInteractions)
|
||||
.innerJoin(issues, and(
|
||||
eq(issues.id, issueThreadInteractions.issueId),
|
||||
eq(issues.companyId, issueThreadInteractions.companyId),
|
||||
))
|
||||
.where(and(
|
||||
eq(issueThreadInteractions.id, interactionId),
|
||||
eq(issueThreadInteractions.companyId, input.companyId),
|
||||
eq(issueThreadInteractions.issueId, input.issueId),
|
||||
eq(issueThreadInteractions.kind, "ask_user_questions"),
|
||||
eq(issueThreadInteractions.status, "answered"),
|
||||
isNull(issueThreadInteractions.sourceRunId),
|
||||
eq(issues.status, "todo"),
|
||||
isNull(issues.startedAt),
|
||||
))
|
||||
.limit(1);
|
||||
return rows.length > 0;
|
||||
}
|
||||
Reference in new issue
Block a user