Files
PaperClipAI/server/src/services/native-runtime/native-board-response-wait.ts
T
DottaandPaperclip 1778075155 fix(server): continue unfinished tasks after status replies (#14626)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native runs report their task outcome through a structured finish
result.
> - A Board comment can permit a passive wait for the next response.
> - That exception accepted reports that also admitted blocking
unfinished work.
> - The task then stayed In Progress without a runner, and recovery
treated the wait as healthy.
> - This pull request rejects that contradiction and uses the existing
bounded continuation path.
> - Real wait conditions and protection against obsolete requests remain
in force.

## Linked Issues or Issue Description

**What happened?**

An agent answered a status inquiry with `yielded` and `response_wake`.
The same result listed blocking remaining work. The server accepted an
indefinite wait without a question, approval, dependency, or pause. No
further run was queued.

**Expected behavior**

Unfinished ordinary tasks must continue or have a recorded reason to
wait. A status reply alone must not suspend the work.

**Steps to reproduce**

1. Add a Board status inquiry to an unfinished assigned task.
2. Submit a successful native result with `yielded`, `response_wake`,
and `remainingWork[].blocksCompletion: true`.
3. Leave the task without any real wait condition.
4. Observe that the old policy preserves In Progress with no
continuation and suppresses recovery.

**Paperclip version or commit**

Reproduced in the native status policy at `da887ea3e`. The branch is
based on current master.

**Deployment mode**

Authenticated server with the native Paperclip Runner.

Related work: Refs #13338 (native response waits and recovery). Refs
#12071 (separate legacy recovery and retry-state work). This change
fixes the native unfinished-response-wait exception.

## What Changed

- Reject contradictory finish reports while the provider can still
correct them.
- Route accepted unfinished response waits through the existing
one-follow-up continuation budget. Repeated incomplete results create a
visible recovery action.
- Preserve questions, approvals, dependencies, pauses, conversation
lifecycles, and superseded Board requests.
- Recheck the current Board source in the decision transaction before
queuing repair.
- Let normal recovery reconsider old committed waits that report
blocking work and still have a current source.
- Add policy and database regression tests. Document the rule.

## Verification

- `pnpm exec vitest run
server/src/services/native-runtime/status-arbiter.test.ts
server/src/__tests__/heartbeat-process-recovery.test.ts
--no-file-parallelism`: 346 tests passed, no skips.
- `pnpm -r typecheck`: passed.
- `pnpm build`: passed.
- `git diff --check origin/master...HEAD`: passed.
- Final head `8f0824d9bb4d631f2347f8afffe5fff6d574cb69`: all CI checks
green (54 passed, 2 intentionally skipped), including all general tests,
serialized server suites, runner tests, eight E2E shards, and the canary
dry run.
- Greptile: 5/5 on the final head, with no unresolved review threads.
- The broad local `pnpm test:run` encountered an unrelated timing
failure in `workspace-runtime.test.ts` (waiting for managed process-tree
listeners). That test passed on an isolated rerun. The duplicate broad
run was stopped; the complete CI suite passed on the final commit.
- The transaction-race regression reproduced an obsolete continuation
before the fix. All 12 source-change cases now pass, covering passive
waits, corrective continuations, and exhausted-repair decisions.

## Risks

- Agents that previously parked unfinished ordinary tasks must now
continue or record an actual wait condition.
- Old contradictory waits become eligible for normal recovery. Existing
ownership, budget, pause, and supersession checks still apply.
- The rule uses the structured blocking-work flag. It does not infer
omitted work from prose.
- No schema, dependency, or UI change.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository inspection, code
editing, and test execution. The session does not expose a more specific
deployment identifier or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-29 16:51:50 -05:00

460 lines
15 KiB
TypeScript

import { and, eq, inArray, isNull, sql } from "drizzle-orm";
import {
agentWakeupRequests,
approvals,
completionContracts,
heartbeatRuns,
issueApprovals,
issueComments,
issues,
issueThreadInteractions,
nativeRunFinalizations,
nativeRunResults,
statusDecisions,
type Db,
} from "@paperclipai/db";
import { nativeSha256 } from "./canonical.js";
import { isConversation } from "../agent-conversations.js";
type Binding = {
companyId: string;
issueId: string;
runId: string;
agentId: string;
};
function record(value: unknown): Record<string, unknown> {
return value && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: {};
}
/** A server-issued decision receipt, never a caller/context-supplied grant. */
export interface NativeBoardResponseWaitSource extends Binding {
schema: "paperclip.native_board_response_wait.v1";
wakeupRequestId: string;
sourceCommentId: string;
sourceUserId: string;
sourceSha256: string;
resultId: string;
resultSha256: string;
contractId: string;
contractSha256: string;
}
export type NativeBoardResponseWaitOrigin = Omit<
NativeBoardResponseWaitSource,
"schema" | "sourceSha256"
> & {
schema: "paperclip.native_board_response_wait_origin.v1";
};
/**
* Denial-only provenance: an accepted response to a direct user comment asks
* for the next response, not an automatic replay. It remains passive if its
* source was edited/deleted/superseded. This never authorizes presentation or
* a new run; those still need current source/assignment checks.
*/
export async function readNativeBoardResponseWaitOrigin(
db: Db,
binding: Binding,
locked = false,
): Promise<NativeBoardResponseWaitOrigin | null> {
const query = db
.select({
run: heartbeatRuns,
wake: agentWakeupRequests,
result: nativeRunResults,
contract: completionContracts,
})
.from(heartbeatRuns)
.innerJoin(
agentWakeupRequests,
and(
eq(agentWakeupRequests.id, heartbeatRuns.wakeupRequestId),
eq(agentWakeupRequests.companyId, binding.companyId),
eq(agentWakeupRequests.agentId, binding.agentId),
eq(agentWakeupRequests.runId, binding.runId),
eq(agentWakeupRequests.source, "automation"),
inArray(agentWakeupRequests.reason, ["issue_commented", "issue_reopened_via_comment"]),
eq(agentWakeupRequests.requestedByActorType, "user"),
),
)
.innerJoin(
nativeRunFinalizations,
and(
eq(nativeRunFinalizations.runId, binding.runId),
eq(nativeRunFinalizations.companyId, binding.companyId),
eq(nativeRunFinalizations.issueId, binding.issueId),
),
)
.innerJoin(
nativeRunResults,
and(
eq(nativeRunResults.id, nativeRunFinalizations.resultId),
eq(nativeRunResults.runId, binding.runId),
eq(nativeRunResults.companyId, binding.companyId),
eq(nativeRunResults.issueId, binding.issueId),
eq(
nativeRunResults.completionContractId,
heartbeatRuns.completionContractId,
),
eq(nativeRunResults.schemaStatus, "accepted"),
),
)
.innerJoin(
completionContracts,
and(
eq(completionContracts.id, heartbeatRuns.completionContractId),
eq(completionContracts.companyId, binding.companyId),
eq(completionContracts.issueId, binding.issueId),
),
)
.where(
and(
eq(heartbeatRuns.id, binding.runId),
eq(heartbeatRuns.companyId, binding.companyId),
eq(heartbeatRuns.agentId, binding.agentId),
eq(heartbeatRuns.nativeIssueId, binding.issueId),
eq(heartbeatRuns.runtimeMode, "native"),
),
)
.limit(1);
const [row] = await (locked ? query.for("share", { noWait: true }) : query);
if (
!row ||
!row.wake.requestedByActorId ||
row.contract.canonicalSha256 !== row.run.completionContractSha256
)
return null;
const payload = record(row.wake.payload);
if (
payload.issueId !== binding.issueId ||
typeof payload.commentId !== "string" ||
!/^[a-f0-9-]{36}$/i.test(payload.commentId)
)
return null;
const result = record(record(row.result.resultJson).result);
const terminal = record(record(row.result.resultJson).terminal);
const continuation = record(result.continuation);
if (
result.schema !== "paperclip.run_result.v1" ||
result.reportedWorkDisposition !== "yielded" ||
continuation.kind !== "response_wake" ||
typeof continuation.idempotencyKey !== "string" ||
!continuation.idempotencyKey.trim() ||
terminal.runTerminalState !== "succeeded" ||
terminal.turnTerminalState !== "completed" ||
terminal.reportedWorkDisposition !== "yielded" ||
!Array.isArray(result.attentionRequests) ||
result.attentionRequests.length !== 0
)
return null;
return {
schema: "paperclip.native_board_response_wait_origin.v1",
companyId: binding.companyId,
issueId: binding.issueId,
runId: binding.runId,
agentId: binding.agentId,
wakeupRequestId: row.wake.id,
sourceCommentId: payload.commentId,
sourceUserId: row.wake.requestedByActorId,
resultId: row.result.id,
resultSha256: row.result.canonicalSha256,
contractId: row.contract.id,
contractSha256: row.contract.canonicalSha256,
};
}
/**
* Only an unchanged, directly admitted Board comment can make response_wake a
* passive wait. Context snapshots, model prose and automatic retry ancestry do
* not grant this exception. In the committer the issue is already locked; use
* NOWAIT for child rows to avoid reversing a concurrent wake/comment lock order.
*/
export async function readNativeBoardResponseWaitSource(
db: Db,
binding: Binding,
locked = false,
): Promise<{ source: NativeBoardResponseWaitSource; summary: string } | null> {
const runQuery = db
.select()
.from(heartbeatRuns)
.where(
and(
eq(heartbeatRuns.id, binding.runId),
eq(heartbeatRuns.companyId, binding.companyId),
eq(heartbeatRuns.agentId, binding.agentId),
eq(heartbeatRuns.nativeIssueId, binding.issueId),
eq(heartbeatRuns.runtimeMode, "native"),
),
)
.limit(1);
const [run] = await (locked
? runQuery.for("share", { noWait: true })
: runQuery);
if (!run?.wakeupRequestId || !run.completionContractId || !run.startedAt)
return null;
const wakeQuery = db
.select()
.from(agentWakeupRequests)
.where(
and(
eq(agentWakeupRequests.id, run.wakeupRequestId),
eq(agentWakeupRequests.companyId, binding.companyId),
eq(agentWakeupRequests.agentId, binding.agentId),
eq(agentWakeupRequests.runId, binding.runId),
eq(agentWakeupRequests.source, "automation"),
inArray(agentWakeupRequests.reason, ["issue_commented", "issue_reopened_via_comment"]),
eq(agentWakeupRequests.requestedByActorType, "user"),
),
)
.limit(1);
const [wake] = await (locked
? wakeQuery.for("share", { noWait: true })
: wakeQuery);
const payload = record(wake?.payload);
if (
!wake?.requestedByActorId ||
payload.issueId !== binding.issueId ||
typeof payload.commentId !== "string" ||
!/^[a-f0-9-]{36}$/i.test(payload.commentId)
)
return null;
const [issue] = await db
.select()
.from(issues)
.where(
and(
eq(issues.id, binding.issueId),
eq(issues.companyId, binding.companyId),
eq(issues.assigneeAgentId, binding.agentId),
eq(issues.status, "in_progress"),
isNull(issues.hiddenAt),
),
)
.limit(1);
if (!issue || record(issue.executionState).status === "pending") return null;
const commentQuery = db
.select()
.from(issueComments)
.where(
and(
eq(issueComments.id, payload.commentId),
eq(issueComments.companyId, binding.companyId),
eq(issueComments.issueId, binding.issueId),
eq(issueComments.authorType, "user"),
eq(issueComments.authorUserId, wake.requestedByActorId),
isNull(issueComments.authorAgentId),
isNull(issueComments.createdByRunId),
isNull(issueComments.deletedAt),
isNull(issueComments.sourceTrust),
// Compare in PostgreSQL, preserving timestamp precision across JS reads.
sql`${issueComments.updatedAt} = ${issueComments.createdAt}`,
sql`${issueComments.createdAt} <= (select started_at from heartbeat_runs where id = ${run.id})`,
),
)
.limit(1);
const [comment] = await (locked
? commentQuery.for("share", { noWait: true })
: commentQuery);
if (!comment?.body.trim()) return null;
const [newer, interaction, approval] = await Promise.all([
db
.select({ id: issueComments.id })
.from(issueComments)
.where(
and(
eq(issueComments.companyId, binding.companyId),
eq(issueComments.issueId, binding.issueId),
eq(issueComments.authorType, "user"),
isNull(issueComments.createdByRunId),
isNull(issueComments.deletedAt),
sql`(${issueComments.createdAt}, ${issueComments.id}) > (select created_at, id from issue_comments where id = ${comment.id})`,
),
)
.limit(1),
db
.select({ id: issueThreadInteractions.id })
.from(issueThreadInteractions)
.where(
and(
eq(issueThreadInteractions.companyId, binding.companyId),
eq(issueThreadInteractions.issueId, binding.issueId),
eq(issueThreadInteractions.status, "pending"),
),
)
.limit(1),
db
.select({ id: approvals.id })
.from(issueApprovals)
.innerJoin(approvals, eq(approvals.id, issueApprovals.approvalId))
.where(
and(
eq(issueApprovals.companyId, binding.companyId),
eq(issueApprovals.issueId, binding.issueId),
eq(approvals.companyId, binding.companyId),
inArray(approvals.status, ["pending", "revision_requested"]),
),
)
.limit(1),
]);
if (newer.length || interaction.length || approval.length) return null;
const [accepted] = await db
.select({ result: nativeRunResults, contract: completionContracts })
.from(nativeRunFinalizations)
.innerJoin(
nativeRunResults,
and(
eq(nativeRunResults.id, nativeRunFinalizations.resultId),
eq(nativeRunResults.companyId, binding.companyId),
eq(nativeRunResults.issueId, binding.issueId),
eq(nativeRunResults.runId, binding.runId),
eq(nativeRunResults.completionContractId, run.completionContractId),
eq(nativeRunResults.schemaStatus, "accepted"),
),
)
.innerJoin(
completionContracts,
and(
eq(completionContracts.id, run.completionContractId),
eq(completionContracts.companyId, binding.companyId),
eq(completionContracts.issueId, binding.issueId),
),
)
.where(
and(
eq(nativeRunFinalizations.runId, run.id),
eq(nativeRunFinalizations.companyId, binding.companyId),
eq(nativeRunFinalizations.issueId, binding.issueId),
),
)
.limit(1);
if (
!accepted ||
accepted.contract.canonicalSha256 !== run.completionContractSha256
)
return null;
const envelope = record(accepted.result.resultJson);
const result = record(envelope.result);
const terminal = record(envelope.terminal);
const continuation = record(result.continuation);
if (
result.schema !== "paperclip.run_result.v1" ||
result.reportedWorkDisposition !== "yielded" ||
continuation.kind !== "response_wake" ||
typeof continuation.idempotencyKey !== "string" ||
!continuation.idempotencyKey.trim() ||
terminal.runTerminalState !== "succeeded" ||
terminal.turnTerminalState !== "completed" ||
terminal.reportedWorkDisposition !== "yielded" ||
!Array.isArray(result.attentionRequests) ||
result.attentionRequests.length !== 0 ||
typeof result.summary !== "string" ||
!result.summary.trim()
)
return null;
return {
source: {
schema: "paperclip.native_board_response_wait.v1",
companyId: binding.companyId,
issueId: binding.issueId,
runId: binding.runId,
agentId: binding.agentId,
wakeupRequestId: wake.id,
sourceCommentId: comment.id,
sourceUserId: wake.requestedByActorId,
sourceSha256: nativeSha256({
id: comment.id,
userId: comment.authorUserId,
body: comment.body,
}),
resultId: accepted.result.id,
resultSha256: accepted.result.canonicalSha256,
contractId: accepted.contract.id,
contractSha256: accepted.contract.canonicalSha256,
},
summary: result.summary.trim(),
};
}
/** Recovery suppresses only this exact applied passive wait, never another run. */
export async function hasCommittedNativeBoardResponseWait(
db: Db,
binding: Binding,
): Promise<boolean> {
const [decision] = await db
.select({
decision: statusDecisions,
resultJson: nativeRunResults.resultJson,
conversationAgentId: issues.conversationAgentId,
conversationUserId: issues.conversationUserId,
})
.from(nativeRunFinalizations)
.innerJoin(
nativeRunResults,
and(
eq(nativeRunResults.id, nativeRunFinalizations.resultId),
eq(nativeRunResults.companyId, binding.companyId),
eq(nativeRunResults.issueId, binding.issueId),
eq(nativeRunResults.runId, binding.runId),
eq(nativeRunResults.schemaStatus, "accepted"),
),
)
.innerJoin(
statusDecisions,
and(
eq(statusDecisions.id, nativeRunFinalizations.decisionId),
eq(statusDecisions.assessmentId, nativeRunFinalizations.assessmentId),
eq(statusDecisions.companyId, binding.companyId),
eq(statusDecisions.issueId, binding.issueId),
eq(statusDecisions.runId, binding.runId),
eq(statusDecisions.applicationState, "applied"),
inArray(statusDecisions.reasonCode, [
"board_response_waiting",
"board_response_wait_superseded",
]),
),
)
.innerJoin(
issues,
and(
eq(issues.id, binding.issueId),
eq(issues.companyId, binding.companyId),
eq(issues.assigneeAgentId, binding.agentId),
eq(issues.status, "in_progress"),
),
)
.where(
and(
eq(nativeRunFinalizations.companyId, binding.companyId),
eq(nativeRunFinalizations.issueId, binding.issueId),
eq(nativeRunFinalizations.runId, binding.runId),
eq(nativeRunFinalizations.phase, "committed"),
),
)
.limit(1);
if (!decision) return false;
const origin = await readNativeBoardResponseWaitOrigin(db, binding);
// A new comment cannot grant a replay of the old response. New independent
// wakes are admitted normally and their new run ID is not this receipt.
const bound =
origin !== null &&
nativeSha256(
record(decision.decision.decisionJson).boardResponseWaitOrigin,
) === nativeSha256(origin);
if (!bound) return false;
const claim = record(record(decision.resultJson).result).completionClaim;
const remainingWork = record(claim).remainingWork;
if (
!isConversation(decision) &&
decision.decision.reasonCode === "board_response_waiting" &&
Array.isArray(remainingWork) &&
remainingWork.some((entry) => record(entry).blocksCompletion === true)
) {
// Old policy accepted contradictory waits. Let normal recovery handle the
// still-current work, but never replay a response to an obsolete comment.
return (await readNativeBoardResponseWaitSource(db, binding)) === null;
}
return true;
}