Files
PaperClipAI/server/src/services/native-runtime/automatic-completion-reviews.ts
T
DottaandPaperclip c9021c6721 fix: require explicit native completion reviews (#13314)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native runs report their outcome through paperclip_finish.
> - The server previously turned incomplete reports into human approval
requests.
> - Those requests could block a later successful run, even when no
person had requested review.
> - This pull request creates review cards only for explicit attention
requests and withdraws proven old fallback cards.
> - Agents receive useful completion feedback, while explicit approval
gates and task state protections remain in force.

## Linked Issues or Issue Description

Related: #13266 removed reviews caused by policy upgrades. This change
removes the separate completion fallback.

**What happened?**

An agent reported needs_review while waiting for checks without
requesting a human decision. Paperclip created a generic Native
completion review. A later successful report could not complete the task
because that old card remained pending.

**Expected behavior**

Ordinary low-risk work completes after a valid done report, a successful
run, and workspace finalization. Incomplete work stays with the agent.
Explicit approval requests remain visible and must be resolved.

**Steps to reproduce**

1. Complete a native run with needs_review and no attention requests.
2. Continue the task and submit a successful done report.
3. Observe that the old implementation leaves the task in review behind
a generic confirmation card.

## What Changed

- Require explicit attention requests to create native review cards.
Route each request independently and preserve pending or declined
decisions.
- Withdraw only pending system cards with matching old decision,
assessment, effect, contract, and prompt provenance. Preserve history
and explicit or answered requests.
- Reassess an affected current result without overwriting later task
edits, runs, contracts, or workspace failures.
- Return pending approval links and required actions through the
completion tool. Reject contradictory done reports and empty review
requests before accepting a result.
- Allow one corrective continuation for incomplete results, then expose
a recovery action.
- Update status fixtures, database regressions, runner tests, and the
completion contract documentation.

## Verification

- `pnpm -r typecheck` passed after merging current master.
- `pnpm build` passed after merging current master.
- The combined branch passed 89 completion and Agent Chat tests. Other
targeted tests passed: 170 external-chat and reconciliation tests; 50
runner-resume and control-plane tests; 13 arbiter tests; 7 chat delivery
tests; 21 runner completion and runtime-context tests.
- The full local test attempt exposed old review fixtures and a missing
fake-provider binary. The fixtures are fixed and the helper is built.
All affected suites pass in fresh reruns. The timing-sensitive Discord
test also passed on rerun.
- All latest-head CI checks passed, including build, typecheck, general
and serialized tests, runner verification, browser tests, and canary dry
run. Greptile is 5/5 with zero unresolved comments.

## Risks

- Cleanup changes existing pending cards. It requires exact system
provenance and only applies to low-risk agent-claim contracts. It does
not delete history or dismiss explicit requests.
- Status still commits after the turn and workspace finalization.
Completion feedback reports current constraints and does not claim an
early status commit.
- Incomplete reports now request a bounded corrective run instead of an
automatic approval. Repeated failures expose recovery.

## Model Used

OpenAI Codex, GPT-6, with repository inspection, code execution, and
test tools. The exact deployment identifier and context window size are
not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-12 13:00:04 -05:00

214 lines
7.6 KiB
TypeScript

import { and, eq, inArray, isNull, sql } from "drizzle-orm";
import {
completionContracts,
issueThreadInteractions,
issues,
nativeRunFinalizations,
statusDecisionEffects,
statusDecisions,
workAssessments,
type Db,
} from "@paperclipai/db";
import {
persistActivity,
publishActivity,
type ActivityPublication,
} from "../activity-log.js";
import { enqueueTerminalIssueInteractionChatPublications } from "../chat-interaction-publications.js";
import { issueThreadInteractionService } from "../issue-thread-interactions.js";
import { logger } from "../../middleware/logger.js";
const withdrawalReason = "automatic_completion_review_removed";
const automaticPrompt =
"Review the persisted native-run evidence and confirm whether this issue may be completed.";
/** Identify only proven system fallback cards; this lookup never changes state. */
export async function findAutomaticCompletionReviews(db: Db, issueId?: string) {
return db
.select({ interaction: issueThreadInteractions, decision: statusDecisions })
.from(issueThreadInteractions)
.innerJoin(
statusDecisionEffects,
and(
eq(statusDecisionEffects.companyId, issueThreadInteractions.companyId),
eq(statusDecisionEffects.issueId, issueThreadInteractions.issueId),
sql`${statusDecisionEffects.targetId} = ${issueThreadInteractions.id}::text`,
eq(statusDecisionEffects.targetType, "issue_thread_interaction"),
eq(statusDecisionEffects.effectKind, "bind_reviewer"),
),
)
.innerJoin(
statusDecisions,
and(
eq(statusDecisions.id, statusDecisionEffects.decisionId),
eq(statusDecisions.companyId, issueThreadInteractions.companyId),
eq(statusDecisions.issueId, issueThreadInteractions.issueId),
eq(statusDecisions.runId, issueThreadInteractions.sourceRunId),
),
)
.innerJoin(
workAssessments,
and(
eq(workAssessments.id, statusDecisions.assessmentId),
eq(workAssessments.companyId, statusDecisions.companyId),
eq(workAssessments.issueId, statusDecisions.issueId),
),
)
.innerJoin(
completionContracts,
and(
eq(completionContracts.id, workAssessments.contractId),
eq(completionContracts.companyId, workAssessments.companyId),
eq(completionContracts.issueId, workAssessments.issueId),
),
)
.where(
and(
eq(issueThreadInteractions.status, "pending"),
eq(issueThreadInteractions.kind, "request_confirmation"),
isNull(issueThreadInteractions.createdByAgentId),
isNull(issueThreadInteractions.createdByUserId),
eq(statusDecisions.applicationState, "applied"),
eq(statusDecisions.toStatus, "in_review"),
inArray(statusDecisions.reasonCode, [
"completion_claim_incomplete",
"completion_claim_conflict",
"external_verification_required",
]),
eq(completionContracts.risk, "low"),
eq(completionContracts.completionAuthority, "agent_claim_policy"),
sql`${workAssessments.assessmentJson}->'attentionRequests' = '[]'::jsonb`,
sql`${issueThreadInteractions.idempotencyKey} = 'native-review:' || ${statusDecisions.id}::text`,
sql`${issueThreadInteractions.payload}->'target'->>'key' = 'native_completion_review'`,
sql`${issueThreadInteractions.payload}->'target'->>'revisionId' = ${statusDecisions.id}::text`,
sql`split_part(${issueThreadInteractions.payload}->>'prompt', E'\n', 1) = ${automaticPrompt}`,
...(issueId ? [eq(issueThreadInteractions.issueId, issueId)] : []),
),
)
.limit(100)
.catch((err) => {
logger.warn(
{ err },
"Automatic completion review lookup failed; will retry",
);
return [];
});
}
/** Narrow, replay-safe retirement. Explicit requests and answered cards are immutable here. */
export async function dismissAutomaticCompletionReviews(
db: Db,
issueId?: string,
) {
const candidates = await findAutomaticCompletionReviews(db, issueId);
for (const { interaction, decision } of candidates) {
const publications: ActivityPublication[] = [];
try {
await db.transaction(async (tx) => {
await tx
.select()
.from(nativeRunFinalizations)
.where(
and(
eq(nativeRunFinalizations.runId, decision.runId),
eq(nativeRunFinalizations.companyId, decision.companyId),
),
)
.for("update");
const [issue] = await tx
.select()
.from(issues)
.where(
and(
eq(issues.id, decision.issueId),
eq(issues.companyId, decision.companyId),
),
)
.for("update");
if (!issue) return;
const now = new Date();
const [retired] = await tx
.update(issueThreadInteractions)
.set({
status: "cancelled",
result: {
version: 1,
outcome: "withdrawn",
reason: withdrawalReason,
},
resolvedAt: now,
updatedAt: now,
})
.where(
and(
eq(issueThreadInteractions.id, interaction.id),
eq(issueThreadInteractions.companyId, issue.companyId),
eq(issueThreadInteractions.status, "pending"),
eq(issueThreadInteractions.payload, interaction.payload),
),
)
.returning();
if (!retired) return;
const projected = await issueThreadInteractionService(
tx as unknown as Db,
).getById(retired.id);
if (projected)
await enqueueTerminalIssueInteractionChatPublications(
tx as unknown as Db,
projected,
);
const { publication } = await persistActivity(tx as unknown as Db, {
companyId: issue.companyId,
actorType: "system",
actorId: "native-completion-review-cleanup",
action: "issue.interaction_cancelled",
entityType: "issue",
entityId: issue.id,
issueId: issue.id,
runId: decision.runId,
details: {
source: withdrawalReason,
interactionId: retired.id,
decisionId: decision.id,
},
});
publications.push(publication);
});
for (const publication of publications) publishActivity(publication);
} catch (err) {
logger.warn(
{ err, interactionId: interaction.id },
"Automatic completion review cleanup failed; will retry",
);
}
}
}
/** A durable trigger survives a restart between withdrawing a card and reassessment. */
export async function decisionHasRetiredAutomaticReview(
db: Db,
decision: typeof statusDecisions.$inferSelect,
) {
const effects = decision.decisionJson.effects as
Array<{ kind: string; gate?: { kind: string; id: string } }> | undefined;
const ids =
effects?.flatMap((effect) =>
effect.gate?.kind === "interaction" ? [effect.gate.id] : [],
) ?? [];
const rows = await db
.select({ id: issueThreadInteractions.id })
.from(issueThreadInteractions)
.where(
and(
eq(issueThreadInteractions.companyId, decision.companyId),
eq(issueThreadInteractions.issueId, decision.issueId),
eq(issueThreadInteractions.status, "cancelled"),
sql`${issueThreadInteractions.result}->>'reason' = ${withdrawalReason}`,
sql`(${issueThreadInteractions.payload}->'target'->>'revisionId' = ${decision.id}::text
or ${ids.length ? inArray(issueThreadInteractions.id, ids) : sql`false`})`,
),
)
.limit(1);
return rows.length > 0;
}