feat(native-runtime): end the wait for the provider's own turn when a person comments (#21455)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Core runs Claude Code through Paperclip Runner. Claude can start a
shell command in the background and end its turn. When the command
finishes, Claude Code starts a cycle of its own to continue.
> - Today the run ends when the turn ends. The command stops, and the
run fails with `native_finalization_missing`.
> - The Runner part of this work is complete in #21316 and the pull
requests below it: when a controller sends `providerStartedTurns` on
`turn.start`, the Runner keeps Claude while a background shell is
running, and gives the cycle of Claude to the controller as a turn that
the provider started.
> - Part 2 gave the consumer a wait with a limit and two inputs: a hook
that ends the wait early, and a start time.
> - This pull request gives both from the server for a native ACPX
Claude run. A new comment from a person ends the wait. The start time
comes from an event that is already saved, so a restart of Core does not
make the wait longer.
> - The benefit is that a person who comments does not wait up to 10
minutes for the agent to see it.

## Linked Issues or Issue Description

No public issue exists. The issue is described here.

This pull request is one of four parts on the Core side. They are
stacked on #21316:

1. The driver admits a turn that the provider started.
2. The consumer waits for that turn, for a limited time.
3. The server ends the wait when a person comments, and keeps the clock
across a restart.
4. The transport sends the opt-in. Only this part changes what a
production run does.

This is part 3. Its base branch is `feat/core-provider-turn-consumer`.
Refs #19912 (design).

**What existing behavior does this improve?**

Nothing visible yet. What Core gives to the consumer of a native ACPX
Claude run.

**Subsystem affected**

Server: `server/src/services/native-runtime`.

**Current behavior**

- A comment from a person during a run is queued as a deferred wake. The
next run delivers it. This does not change.
- Core has no wait for a provider-started turn.

**Proposed behavior**

- For a native ACPX Claude run, Core gives `executeNativeSession` the
option `providerTurnWait`.
- One delivery worker does the person-comment check for all runs that
wait. The server registers it at startup with the shared delivery work
coordinator, as it registers the cost accounting worker. The worker
listens to the existing notification for deferred wakes. On each notice
it runs one query for each run that waits: a comment from a person on
this issue, made after the start of the run, that a deferred wake of
this agent names. A comment that the query finds ends the wait of that
run. The run then takes the path of today, and the next wake delivers
the comment as today.
- If a database read fails, that run stays owed a check, and the
coordinator runs the worker again after 5 seconds, as for each other
delivery worker. A read that fails for one run does not stop the check
of the other runs. When no run is owed a check, the worker has no
deadline and does no query.
- The start time of the wait is the time of the first saved
`turn.completed` event of the run that has `awaitingProviderTurn`. With
no such event it is empty.
- Each query has the company id in its condition.

**Reason and benefit**

- The comment window starts at the start of the run, not at the start of
the wait. A comment made during the prompt turn must not wait for the
limit.

## What Changed

- `server/src/services/native-runtime/native-provider-turn-wait.ts`
(new, 109 lines) and 4 lines in `native-session-executor.ts`.
- `server/src/index.ts` (4 lines): the registration of the worker, its
stop at server close, and its place in the list of delivery workers that
must be ready. `delivery-work-notifications.ts` (1 line): the queue key
of the worker. The key needs no database change, because each topic uses
the one shared channel.
- `server/src/vendor/paperclip-runner/index.ts`: one type export.
- Tests: `native-provider-turn-wait.test.ts` (new, 252 lines) and one
case in `native-usage-accounting.test.ts` (25 lines).

## State saved by the previous release

- No new persisted state and no schema change. The start time is read
from `heartbeat_run_events`.
- **A run that the previous release started:** it has no waiting
terminal, so the start time is empty.
- **Core restarts during a wait:** the new process reads the start time,
so the limit is the original one. The consumer then continues the wait
(#21454), and the worker checks the comments again from the start of the
run, so a comment that was made before the restart still ends the wait.
- **Rollback of Core during a wait:** the previous Core stops at the
waiting terminal and finalizes the run as `native_finalization_missing`,
as today.

## Verification

At the head of this branch:

- `npx tsc --noEmit -p tsconfig.json`, `npx tsc --noEmit -p
tsconfig.test.json` and `pnpm run build:typescript` in
`packages/paperclip-runner`; `node
scripts/verify-runner-vendor-dependencies.mjs` and `pnpm typecheck` in
`server`; `node --test
scripts/__tests__/runner-release-crate-cache.test.mjs`: all passed.
- `native-provider-turn-wait.test.ts`, 10 tests on embedded Postgres
with the real coordinator: a comment from a person in a deferred wake
ends the wait; a comment from an agent, a comment before the run, and a
comment of a different company do not; the start time comes from the
first waiting terminal, and a run of the previous release has none.
- The 6 new tests: a read that fails one time is tried again after 5
seconds and finds a comment that is already saved, with no new notice; a
notice during a read that fails gives a new check; a read that fails for
one run does not hold back a second run, and each run ends only for its
own comment; a run that starts to wait during a check is checked; a wait
that is aborted is removed, nothing checks it again and no deadline
stays; with no run owed a check there is no deadline and no query in one
hour of test time.
- The test helper now registers the wake intent before it writes the
deferred wake, in the same transaction. This is the order of each
production writer (`modules/wake-queue/adapters/postgres.ts` and
`services/heartbeat/queue.ts`). Production code did not change for this.
- Mutation checks: a read that fails clears the owed check (3 tests
fail); an aborted wait is not removed (the abort test fails).
- Also run, unchanged and green: `delivery-work-coordinator.test.ts`,
`budget-enforcement-worker.test.ts`,
`server-startup-feedback-export.test.ts`, `startup-timeline.test.ts`.
The 7 files together: 1,699 passed, 7 skipped.
- `native-usage-accounting.test.ts`: a prompt turn and the
provider-started turn after it are each counted one time, with the
events as the executor receives them. The cost of the provider-started
turn is the session total in the cumulative channel, and it is not added
two times. The accounting code is not changed.

Not run: a run through the executor with a database; the full suite. CI
supplies the broad coverage.

## Risks

- **One more query at each start of a native ACPX Claude run,** on
`heartbeat_run_events` by company and run.
- **The worker is registered in each server process, and it is idle with
no wait.** A notice with no run that waits does no query. No wait starts
before part 4.
- **After an abort during a retry, the 5 second retry of the coordinator
runs one more time.** It finds nothing owed and does no query. The
coordinator has no call to cancel the retry of one worker.
- **A process that does not register the worker does no comment check.**
The server registers it with no condition at startup, and native runs
execute only in that process. Only a unit test can be in this state.
- **A comment ends the wait but does not interrupt a turn that is
running.** That is the rule of part 2.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code planned the
change and the split, reviewed the diff, and ran a part of the
verification again.
- Claude Opus 5.5 (`claude-opus-5-5`), as a Claude Code subagent with
tool use (shell, file edits, local test runs), wrote the change and the
tests and cut this slice from the full change.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Nicky LeachandClaude Opus 5.5 authored and GitHub committed 2026-11-13 22:55:11 -08:00
1 parent 2102492806
commit e977c3b922
7 files changed
+396 -1

No files matched your search

+4 -1
View File
@@ -4,6 +4,7 @@ import { closeProcessRunAdmission, isProcessRunAdmissionClosed } from "./service
import { registerReattachPhaseWork } from "./services/heartbeat/reattach-phase.js";
import { createRunStartHandover, registerRunStartHandover } from "./services/run-start-handover.js";
import { registerOwnershipReconciliationWork } from "./services/native-runtime/native-finalization-reconciler.js";
import { registerPersonCommentWaitWork } from "./services/native-runtime/native-provider-turn-wait.js";
import { bindIdleDeadlineOwner } from "./services/idle-deadline-owner.js";
import { readIdleRetrySchedule } from "./services/heartbeat/idle-retry-schedule.js";
import { readIdleQuotaSchedule } from "./services/heartbeat/idle-quota-schedule.js";
@@ -1500,6 +1501,8 @@ async function startServerWithDatabaseTeardown(
server.on("close", () => { void budgetEnforcementWorker.stop(); });
const accountingWorker = registerCostAccountingWork(db, deliveryWork, accountingHeartbeat.reconcileCostAccounting);
server.on("close", accountingWorker.stop);
const personCommentWaitWorker = registerPersonCommentWaitWork(db, deliveryWork);
server.on("close", personCommentWaitWorker.stop);
skillSourceSync.start(deliveryWork);
idleDeadlineOwners.skills = bindIdleDeadlineOwner(skillSourceSync.reconciliationSnapshot, readIdleSkillSourceSchedule,
() => ({ coverage: { kind: "skill_sources" } }));
@@ -1625,7 +1628,7 @@ async function startServerWithDatabaseTeardown(
() => !isIdleTaskDrainActive() && !heartbeatSchedulerStopped && !isWarmStandby()).ready)
.catch((err: unknown) => logger.error({ err }, "startup provider task recovery failed"))));
trackHeartbeatSchedulerWork(startupTimeline.background("delivery_workers", () => timeStartupRecovery("delivery_workers",
() => Promise.all([budgetEnforcementWorker, accountingWorker, fastResponseWorker, ...deliveryWorkers].map(worker => worker.ready)))));
() => Promise.all([budgetEnforcementWorker, accountingWorker, personCommentWaitWorker, fastResponseWorker, ...deliveryWorkers].map(worker => worker.ready)))));
trackHeartbeatSchedulerWork(startupTimeline.background("github_connections",
() => Promise.all([githubEventWorker.ready, githubContinuityWorker.ready])));
@@ -19,6 +19,7 @@ export const DELIVERY_QUEUES = {
strandedAgentStatus: "stranded-agent-status",
activeRunSilence: "active-run-silence",
deferredExecutionReceipts: "deferred-execution-receipts",
nativeProviderTurnWaits: "native-provider-turn-waits",
taskRetryReceiptSettlement: "task-retry-receipt-settlement",
nativeStatusWakeIntents: "native-status-wake-intents",
orphanBlockers: "orphan-blockers",
@@ -0,0 +1,252 @@
import { randomUUID } from "node:crypto";
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import {
agents, agentWakeupRequests, companies, createDb, type Db, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, heartbeatRuns, issueComments, issues,
} from "@paperclipai/db";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
import { appendHeartbeatRunEvent } from "../heartbeat-run-events.js";
import { notifyDeferredExecutionWork } from "../deferred-execution-notifications.js";
import { createDeliveryWorkCoordinator } from "../delivery-work-coordinator.js";
import { idleWorkSnapshot } from "../task-admission.js";
import {
awaitPersonCommentDuringNativeRun, nativeProviderTurnWaitOptions, nativeProviderTurnWaitStartedAt, personCommentedDuringNativeRun,
registerPersonCommentWaitWork,
} from "./native-provider-turn-wait.js";
const support = await getEmbeddedPostgresTestSupport();
const USER = "provider-turn-user";
describe.skipIf(!support.supported)("native provider-turn wait control-plane hooks", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: ReturnType<typeof createDb>;
let coordinator: ReturnType<typeof createDeliveryWorkCoordinator> | undefined;
let worker: ReturnType<typeof registerPersonCommentWaitWork> | undefined;
const errors: unknown[] = [];
const controllers: AbortController[] = [];
beforeAll(async () => {
database = await startEmbeddedPostgresTestDatabase("paperclip-provider-turn-wait-");
db = createDb(database.connectionString, { idleTimeoutSeconds: 0, connectTimeoutSeconds: 0, maxLifetimeSeconds: 0 });
}, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS);
beforeEach(() => {
errors.length = 0;
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"], shouldAdvanceTime: true });
});
afterEach(async () => {
for (const controller of controllers.splice(0)) controller.abort(new Error("test ended"));
worker?.stop(); await coordinator?.stop(); worker = undefined; coordinator = undefined;
vi.useRealTimers();
});
afterAll(async () => { await db?.$client.end(); await database?.cleanup(); });
type Scope = Parameters<typeof personCommentedDuringNativeRun>[1];
async function start(read: (database: Db, scope: Scope) => Promise<boolean> = personCommentedDuringNativeRun) {
coordinator = createDeliveryWorkCoordinator({ owner: db, canRun: () => true, canReconcile: () => true,
onError: error => errors.push(error) });
worker = registerPersonCommentWaitWork(db, coordinator, read);
await worker.ready;
}
const settle = () => vi.waitFor(() => expect(idleWorkSnapshot().active).toBe(0));
const unavailable = () => new Error("database unavailable");
function wait(scope: Scope) {
const abort = new AbortController();
controllers.push(abort);
const state = { ended: false, abort, promise: Promise.resolve() };
state.promise = awaitPersonCommentDuringNativeRun(db, scope, abort.signal).then(() => { state.ended = true; });
state.promise.catch(() => {});
return state;
}
async function seed(startedAt = new Date()) {
const [company] = await db.insert(companies).values({ name: "Provider turn", issuePrefix: `P${randomUUID().slice(0, 4)}` }).returning();
const [agent] = await db.insert(agents).values({ companyId: company!.id, name: "Claude", adapterType: "paperclip_runner" }).returning();
const issueId = randomUUID();
const [run] = await db.insert(heartbeatRuns).values({ companyId: company!.id, agentId: agent!.id, status: "running", runtimeMode: "native",
runtimeModeResolvedAt: new Date(), nativeIssueId: issueId, responsibleUserId: USER, startedAt, contextSnapshot: { issueId } }).returning();
await db.insert(issues).values({ id: issueId, companyId: company!.id, title: "Install and test", status: "in_progress",
assigneeAgentId: agent!.id, executionRunId: run!.id });
return { companyId: company!.id, agentId: agent!.id, issueId, runId: run!.id };
}
async function comment(f: Awaited<ReturnType<typeof seed>>, author: "user" | "agent", queued: boolean, createdAt?: Date) {
const [saved] = await db.insert(issueComments).values({ companyId: f.companyId, issueId: f.issueId,
...(author === "user" ? { authorType: "user", authorUserId: USER } : { authorType: "agent", authorAgentId: f.agentId }),
body: "The install finished, please continue.", ...(createdAt ? { createdAt } : {}) }).returning();
if (queued) {
await db.transaction(async tx => {
await notifyDeferredExecutionWork(tx, { companyId: f.companyId });
await tx.insert(agentWakeupRequests).values({ companyId: f.companyId, agentId: f.agentId, source: "automation",
triggerDetail: "system", reason: "issue_commented", status: "deferred_issue_execution",
requestedByActorType: "user", requestedByActorId: USER,
payload: { issueId: f.issueId, commentId: saved!.id, _paperclipWakeContext: { issueId: f.issueId, taskId: f.issueId,
wakeReason: "issue_commented", source: "issue.comment", wakeCommentId: saved!.id, wakeCommentIds: [saved!.id] } } });
});
}
return saved!.id;
}
const prpTerminal = (f: Awaited<ReturnType<typeof seed>>, awaiting: boolean) => appendHeartbeatRunEvent(db, {
companyId: f.companyId, runId: f.runId, agentId: f.agentId, eventType: "turn.completed", stream: "system", level: "info",
message: "turn completed", payload: { prpEvent: { eventType: "turn.completed", turnId: "turn-1",
payload: { status: "completed", error: null, ...(awaiting ? { awaitingProviderTurn: true } : {}) } } } });
it("restores the wait clock from the first awaited terminal and has none for a previous release's run", async () => {
const f = await seed();
await prpTerminal(f, false);
expect(await nativeProviderTurnWaitStartedAt(db, f)).toBeNull();
await prpTerminal(f, true);
const first = await nativeProviderTurnWaitStartedAt(db, f);
expect(first).toEqual(expect.any(Number));
await new Promise(resolve => setTimeout(resolve, 20));
await prpTerminal(f, true);
expect(await nativeProviderTurnWaitStartedAt(db, f)).toBe(first);
expect((await nativeProviderTurnWaitOptions(db, f)).startedAtMs).toBe(first);
});
it("counts only a person's queued comment made during this run", async () => {
const f = await seed(new Date(Date.now() - 60_000));
await comment(f, "user", true, new Date(Date.now() - 120_000));
await comment(f, "agent", true);
await comment(f, "user", false);
expect(await personCommentedDuringNativeRun(db, f)).toBe(false);
await comment(f, "user", true);
expect(await personCommentedDuringNativeRun(db, f)).toBe(true);
});
it("ends the wait when a person's comment is queued, through the deferred-wake notification", async () => {
await start();
const f = await seed();
const abort = new AbortController();
let ended = false;
const waiting = awaitPersonCommentDuringNativeRun(db, f, abort.signal).then(() => { ended = true; });
await new Promise(resolve => setTimeout(resolve, 100));
expect(ended).toBe(false);
await comment(f, "user", true);
await waiting;
expect(ended).toBe(true);
});
it("stops listening when the wait ends for another reason", async () => {
await start();
const f = await seed();
const abort = new AbortController();
const waiting = awaitPersonCommentDuringNativeRun(db, f, abort.signal);
abort.abort(new Error("wait ended"));
await expect(waiting).rejects.toThrow("wait ended");
});
it("retries a failed read and finds a comment already queued, with no further notification", async () => {
const f = await seed();
await comment(f, "user", true);
let failures = 1;
const read = vi.fn(async (database: Db, scope: Scope) => {
if (failures-- > 0) throw unavailable();
return personCommentedDuringNativeRun(database, scope);
});
await start(read);
const waiting = wait(f);
await vi.waitFor(() => expect(errors).toHaveLength(1));
expect(waiting.ended).toBe(false);
expect(coordinator!.nextWakeAt()! - Date.now()).toBeGreaterThan(4_000);
await vi.advanceTimersByTimeAsync(5_000);
await vi.waitFor(() => expect(waiting.ended).toBe(true), { timeout: 2_000 });
expect(read).toHaveBeenCalledTimes(2);
});
it("checks again after a wake that arrives during a failing read", async () => {
const f = await seed();
let release!: () => void;
const gate = new Promise<void>(resolve => { release = resolve; });
const read = vi.fn(async (database: Db, scope: Scope) => {
if (read.mock.calls.length === 1) { await gate; throw unavailable(); }
return personCommentedDuringNativeRun(database, scope);
});
await start(read);
const waiting = wait(f);
await vi.waitFor(() => expect(read).toHaveBeenCalledTimes(1));
await comment(f, "user", true);
release();
await vi.waitFor(() => expect(waiting.ended).toBe(true), { timeout: 2_000 });
expect(read).toHaveBeenCalledTimes(2);
});
it("does not let one waiter's failing read hold back another, and resolves each only for its own comment", async () => {
const a = await seed();
const b = await seed();
let broken = true;
let checkedA = 0;
const read = vi.fn(async (database: Db, scope: Scope) => {
if (scope.runId === a.runId && broken) throw unavailable();
const found = await personCommentedDuringNativeRun(database, scope);
if (scope.runId === a.runId) checkedA += 1;
return found;
});
await start(read);
const waitingA = wait(a);
const waitingB = wait(b);
await vi.waitFor(() => expect(errors.length).toBeGreaterThan(0));
await comment(b, "user", true);
await vi.waitFor(() => expect(waitingB.ended).toBe(true), { timeout: 2_000 });
expect(waitingA.ended).toBe(false);
broken = false;
await vi.advanceTimersByTimeAsync(5_000);
await vi.waitFor(() => expect(checkedA).toBeGreaterThan(0), { timeout: 2_000 });
expect(waitingA.ended).toBe(false);
await comment(a, "user", true);
await vi.waitFor(() => expect(waitingA.ended).toBe(true), { timeout: 2_000 });
});
it("checks a waiter that is added while a check is running", async () => {
const first = await seed();
const second = await seed();
await comment(second, "user", true);
let release!: () => void;
const gate = new Promise<void>(resolve => { release = resolve; });
const read = vi.fn(async (database: Db, scope: Scope) => {
if (scope.runId === first.runId) await gate;
return personCommentedDuringNativeRun(database, scope);
});
await start(read);
const waitingFirst = wait(first);
await vi.waitFor(() => expect(read).toHaveBeenCalledTimes(1));
const waitingSecond = wait(second);
release();
await vi.waitFor(() => expect(waitingSecond.ended).toBe(true), { timeout: 2_000 });
expect(read.mock.calls.map(([, scope]) => scope.runId)).toEqual([first.runId, second.runId]);
expect(waitingFirst.ended).toBe(false);
});
it("removes an aborted waiter so nothing checks its scope again and no deadline remains", async () => {
const f = await seed();
const read = vi.fn(async (_database: Db, _scope: Scope) => { throw unavailable(); });
await start(read);
const abort = new AbortController();
const waiting = awaitPersonCommentDuringNativeRun(db, f, abort.signal);
await vi.waitFor(() => expect(errors).toHaveLength(1));
expect(coordinator!.nextWakeAt()).not.toBeNull();
const reason = new Error("wait ended");
abort.abort(reason);
await expect(waiting).rejects.toBe(reason);
await comment(f, "user", true);
await vi.advanceTimersByTimeAsync(5_000);
await settle();
await vi.advanceTimersByTimeAsync(3_600_000);
await settle();
expect(read).toHaveBeenCalledTimes(1);
expect(coordinator!.nextWakeAt()).toBeNull();
});
it("holds no deadline and runs no check while no waiter is owed a check", async () => {
const f = await seed();
const read = vi.fn(personCommentedDuringNativeRun);
await start(read);
expect(coordinator!.nextWakeAt()).toBeNull();
wait(f);
await vi.waitFor(() => expect(read).toHaveBeenCalledTimes(1));
await settle();
expect(coordinator!.nextWakeAt()).toBeNull();
await vi.advanceTimersByTimeAsync(3_600_000);
await settle();
expect(read).toHaveBeenCalledTimes(1);
expect(coordinator!.nextWakeAt()).toBeNull();
});
});
@@ -0,0 +1,109 @@
import { and, asc, eq, gte, isNotNull, isNull, sql } from "drizzle-orm";
import { agentWakeupRequests, heartbeatRunEvents, heartbeatRuns, issueComments, type Db } from "@paperclipai/db";
import type { createDeliveryWorkCoordinator } from "../delivery-work-coordinator.js";
import { DELIVERY_QUEUES, subscribeDeliveryWork } from "../delivery-work-notifications.js";
import type { ProviderTurnWaitOptions } from "../../vendor/paperclip-runner/index.js";
type NativeRunScope = { companyId: string; runId: string; issueId: string; agentId: string };
export async function nativeProviderTurnWaitStartedAt(db: Db, scope: Pick<NativeRunScope, "companyId" | "runId">) {
const [first] = await db.select({ createdAt: heartbeatRunEvents.createdAt }).from(heartbeatRunEvents).where(and(
eq(heartbeatRunEvents.companyId, scope.companyId), eq(heartbeatRunEvents.runId, scope.runId),
eq(heartbeatRunEvents.eventType, "turn.completed"),
sql`${heartbeatRunEvents.payload} -> 'prpEvent' -> 'payload' ->> 'awaitingProviderTurn' = 'true'`,
)).orderBy(asc(heartbeatRunEvents.seq)).limit(1);
return first?.createdAt instanceof Date ? first.createdAt.getTime() : null;
}
export async function personCommentedDuringNativeRun(db: Db, scope: NativeRunScope) {
const [run] = await db.select({ startedAt: heartbeatRuns.startedAt, createdAt: heartbeatRuns.createdAt }).from(heartbeatRuns)
.where(and(eq(heartbeatRuns.companyId, scope.companyId), eq(heartbeatRuns.id, scope.runId))).limit(1);
if (!run) return false;
const [comment] = await db.select({ id: issueComments.id }).from(issueComments).where(and(
eq(issueComments.companyId, scope.companyId), eq(issueComments.issueId, scope.issueId),
isNotNull(issueComments.authorUserId), isNull(issueComments.authorAgentId), isNull(issueComments.createdByRunId),
isNull(issueComments.deletedAt), gte(issueComments.createdAt, run.startedAt ?? run.createdAt),
sql`exists (select 1 from ${agentWakeupRequests} where ${agentWakeupRequests.companyId} = ${scope.companyId}
and ${agentWakeupRequests.agentId} = ${scope.agentId} and ${agentWakeupRequests.status} = 'deferred_issue_execution'
and ${agentWakeupRequests.payload} ->> 'issueId' = ${scope.issueId}
and (${agentWakeupRequests.payload} -> '_paperclipWakeContext' -> 'wakeCommentIds') @> jsonb_build_array((${issueComments.id})::text))`,
)).limit(1);
return comment !== undefined;
}
type PersonCommentWaiter = { scope: NativeRunScope; owed: boolean; resolve: () => void };
type Coordinator = Pick<ReturnType<typeof createDeliveryWorkCoordinator>, "register">;
const personCommentWaiters = new WeakMap<Db, Set<PersonCommentWaiter>>();
const personCommentWakes = new WeakMap<Db, () => void>();
function waitersFor(db: Db) {
let waiters = personCommentWaiters.get(db);
if (!waiters) personCommentWaiters.set(db, waiters = new Set());
return waiters;
}
export function registerPersonCommentWaitWork(db: Db, coordinator: Coordinator, read = personCommentedDuringNativeRun) {
const waiters = waitersFor(db);
const worker = coordinator.register(DELIVERY_QUEUES.nativeProviderTurnWaits, {
retryMs: 5_000,
runDuringDrain: true,
async run(signal) {
const failures: unknown[] = [];
for (const waiter of [...waiters]) {
if (signal.aborted) return;
if (!waiter.owed) continue;
waiter.owed = false;
try {
if (await read(db, waiter.scope) && waiters.delete(waiter)) waiter.resolve();
} catch (error) {
waiter.owed = true;
failures.push(error);
}
}
if (failures.length) throw failures[0];
},
async hasPending() {
return [...waiters].some(waiter => waiter.owed);
},
});
personCommentWakes.set(db, worker.wake);
const unsubscribe = subscribeDeliveryWork(db, DELIVERY_QUEUES.deferredExecutionReceipts, () => {
if (!waiters.size) return;
for (const waiter of waiters) waiter.owed = true;
worker.wake();
});
const stop = () => {
unsubscribe();
if (personCommentWakes.get(db) === worker.wake) personCommentWakes.delete(db);
};
return { ...worker, stop };
}
export function awaitPersonCommentDuringNativeRun(db: Db, scope: NativeRunScope, signal: AbortSignal): Promise<void> {
return new Promise<void>((resolve, reject) => {
if (signal.aborted) {
reject(signal.reason);
return;
}
const waiters = waitersFor(db);
const abort = () => {
waiters.delete(waiter);
reject(signal.reason);
};
const waiter: PersonCommentWaiter = { scope, owed: true, resolve: () => {
signal.removeEventListener("abort", abort);
resolve();
} };
signal.addEventListener("abort", abort, { once: true });
waiters.add(waiter);
personCommentWakes.get(db)?.();
});
}
export async function nativeProviderTurnWaitOptions(db: Db, scope: NativeRunScope): Promise<ProviderTurnWaitOptions> {
return {
startedAtMs: await nativeProviderTurnWaitStartedAt(db, scope),
onWait: ({ signal }) => awaitPersonCommentDuringNativeRun(db, scope, signal),
};
}
@@ -31,6 +31,7 @@ import { prepareRemoteOpenCodeArtifacts, type OpenCodeLaunchArtifacts } from "./
import { prepareRuntimeContextSource, rehydratePinnedInstructionAsset } from "./runtime-context.js";
import { createNativeUsageDelivery } from "./native-usage-delivery.js";
import { createNativeRecoveryUsageDelivery } from "./native-recovery-usage-delivery.js";
import { nativeProviderTurnWaitOptions } from "./native-provider-turn-wait.js";
import { createNativeTurnAccounting, NATIVE_LOST_TURN_USAGE_EVIDENCE, nativeUsageCostUsd, nativeUsageBiller, normalizeNativeUsage, usesNativeUsageDelivery, isNativeUsageAccountingEvent, isNativeUsageDeliveryEvent } from "./native-usage-accounting.js";
import { crashPoint } from "@paperclipai/shared/crash-points";
export { nativeUsageCostUsd, nativeUsageBiller, normalizeNativeUsage } from "./native-usage-accounting.js";
@@ -12377,10 +12378,13 @@ async function executePaperclipNativeSessionAttempt(
startedAtMs: nativeSessionExecuteStartedAtMs,
});
trace.activate(runnerSessionStartupScope);
const providerTurnWait = input.execution.provider.kind === "acpx" && input.execution.provider.agent === "claude"
? await nativeProviderTurnWaitOptions(input.db, input.execution.binding) : undefined;
const runExecution = () =>
executeNativeSession({
resumeInterruptedTurn: input.restartRecovery?.kind === "resume_dead_runner"
|| input.restartRecovery?.kind === "reattach_remote_runner" || scheduledResumeContinuation,
providerTurnWait,
getFreshSessionHandoff: input.getFreshSessionHandoff,
// A failed seal commits nothing: the messages stay queued for the
// turn boundary instead of failing this run before its turn.
@@ -126,6 +126,31 @@ describe("native turn accounting", () => {
expect(accounting.snapshot().usage).not.toHaveProperty("accountingUsageLost");
});
it("counts a prompt turn and the provider-started turn after it once each, as the executor receives them", () => {
const acpx = { kind: "acpx" as const, agent: "claude", model: "claude-sonnet-5" } as never;
const accounting = createNativeTurnAccounting(acpx);
const report = (inputTokens: number, outputTokens: number, sessionCost: number) => ({ kind: "usage", usage: {
total: { inputTokens: 0, outputTokens: 0, providerCostUsd: sessionCost },
runDelta: { inputTokens, outputTokens, cacheReadTokens: 0, cacheWriteTokens: 0, providerCostUsd: 0 }, runDeltaComplete: true } });
const history = [
event(1, "turn.started", "turn_lab_prompt", { status: "inProgress" }),
event(2, "item.completed", "turn_lab_prompt", report(6, 4, 0.2)),
event(3, "turn.completed", "turn_lab_prompt", { status: "completed", error: null, awaitingProviderTurn: true }),
event(4, "turn.started", "provider-1", { status: "inProgress", origin: "provider" }),
event(5, "item.completed", "provider-1", report(30, 20, 0.465)),
event(6, "turn.completed", "provider-1", { status: "completed", error: null }),
];
history.forEach(value => accounting.observe(value));
history.forEach(value => accounting.observe(value));
const snapshot = accounting.finish("provider-1");
expect(snapshot.complete).toBe(true);
expect(snapshot.turnId).toBe("provider-1");
expect(normalizeNativeUsage(snapshot.usage)).toEqual({ inputTokens: 36, outputTokens: 24, cachedInputTokens: 0, cacheWriteTokens: 0 });
expect(nativeUsageCostUsd(snapshot.usage, acpx)).toBeUndefined();
expect(snapshot.usage).not.toHaveProperty("accountingCostUsdExact");
expect(JSON.stringify(snapshot.usage)).not.toMatch(/0\.665|0\.465|0\.2[^0-9]/);
});
it("keeps an incomplete provider report pending instead of trusting placeholder zeroes", () => {
const accounting = createNativeTurnAccounting(provider);
accounting.observe(event(1, "turn.started"));
+1
View File
@@ -67,6 +67,7 @@ export type {
PrpTerminalState,
PrpVerificationReasonCode,
PrpWireConnection,
ProviderTurnWaitOptions,
ReplayControlPlaneEventsInput,
RunnerExecutionDriver,
RunnerLaunchPreparer,