PaperClipAI/doc/idle-decision-continuations.md

# Decision continuation delivery while idle

A terminal decision can owe an agent wake. Its existing row stores that debt:
`continuationPolicy = wake_origin_agent`, `metadata.continuationPending = true`,
and either expired/cancelled status or a decided status with succeeded/partial/
failed execution. Open decisions and running effects are not ready to deliver.
This migration adds no queue table and changes no wake payload.

## Ownership and producers

`decisions.ts` owns decision creation, authorization, signatures, effects, and
state changes. Its ten update sites now use a narrow transaction-aware write
helper before the SQL statement. These cover the running claim; ordinary effect
completion; archive-proposal signature, manifest, and execution failures; archive
success; TTL rejection on use; cancel; direct dismissal; and background expiry.
The helper acquires intent in the existing outer transaction or its savepoint.
It does not include effect execution, provider calls, activity publication, or
agent wake callbacks in a new database transaction.

Creation and bundle creation insert open decisions, so they do not create ready
continuation debt. Acknowledgement only clears debt and belongs to the delivery
owner; it does not signal itself. The production decision services are created
by server startup and routes with the application's DB handle. Independent SQL
writers and additional active processes require a separate notification and
ownership contract. This change does not make wake-producing service methods
safe to call inside an uncommitted caller transaction; only their SQL mutation
boundary has the existing outer-commit guarantee.

`decision-continuations.ts` owns the readiness predicate, delivery, and recovery.
Each worker pass attempts at most 100 candidates. A saved in-memory cursor
advances past failed candidates between passes and wraps after the last page.
Remaining durable work retains another pass. This avoids starving later rows behind a failing first page.
A partial ID index contains ready pending continuations only. The predicate uses
fixed SQL literals so a prepared generic plan can still use that index.

The shared DB package exposes an opaque work-scope identity. Raw DB object
identity is insufficient because dedicated and liveness child pools already
share their owner's signals. The key is shared by those children and nested
transactions; independent roots have different keys. No connection credentials,
private transaction state, or decision policy are exposed. Existing signal
consumers and transactions that do not register work keep their behavior.

## Foreground, recovery, and failure

Foreground delivery remains immediate. All service instances sharing the DB
owner coalesce a simultaneous wake for the same decision into one promise.
Delivery re-reads durable readiness after claiming that local flight, so a late
foreground call cannot resend a continuation already acknowledged by recovery.
The promise is removed on success or failure. Effect execution and its final
state write also share one local flight per decision. This prevents concurrent
archive callers from turning a successful archive into a stale-manifest failure. This is process-local coalescing,
not a new durable claim or an exactly-once guarantee.

Effect completion requires the row to remain running. A competing completion
cannot restore pending metadata after another caller has completed and delivered
it. Effects retain their own existing durable claims and authorization checks.
The delivery worker never executes effects.

A wake failure leaves pending metadata intact. A failed acknowledgement also
leaves debt, so a later attempt can send the wake again. A crash after accepting
the wake but before acknowledgement retains the existing at-least-once contract.
An uncertain producer transaction remains held until the shared work-signal
protocol establishes settlement; one empty scan cannot clear that uncertainty.

## Lifecycle and scope

Startup reconstructs pending delivery. Committed changes wake the worker through
the existing coordinator. Failures or remaining ready debt retain a 30-second
retry. Open/running decisions, no decisions, and delivered history have no
recurring continuation scan or deadline. Changes during a pass retain another
request. Drain/standby defer dispatch without consuming that request; shutdown
waits for the current admitted delivery and stops before starting another.
The shared deadline adapter forwards its existing AbortSignal to task execution;
existing callbacks that do not consume the signal keep their behavior. A future process-local retry is not a Cloud alarm.

Enabled-heartbeat startup registers this worker before the combined decision
sweep. The main tick skips that sweep's independent continuation scan. Its public
API retains combined behavior by default. Expiry or effect recovery can still
deliver their newly produced continuations immediately. Disabled-heartbeat
startup keeps its existing behavior and does not dispatch this new worker.

Decision expiry, invalidated-target scanning, interrupted effect recovery, and
retention remain separate migrations. Existing Cloud safety blockers are not
relaxed. This establishes worker quiet under the named conditions, not deployed
host/database suspension or measured fleet savings.

## Verification

The selected run passes 93 tests: 50 real-PostgreSQL decision cases, 39 startup
cases, and four database work-signal unit cases. Fifteen new continuation cases
cover real decide/cancel/dismiss/expiry producers, running-to-terminal completion,
foreground contention, dedicated child pools, stale caller re-read, restart,
failed acknowledgement, outer rollback/commit, first write during drain, failed
retry through drain, changes during delivery, shutdown, and paging past a failed
candidate. Quiet checks observe no SQL calls across an hour of controlled time.
Nine pre-existing embedded-PostgreSQL signal tests are skipped locally because
the embedded runtime is unavailable; CI must run that supported environment.
Workspace typecheck and build pass. The final literal-predicate adjustment also
passes all 49 decision tests before the additional archive-race regression.

Migration 0382 applies and replays without error. A rolled-back synthetic
40,000-row history with one pending continuation uses the partial index under
`force_generic_plan`, reads two shared buffers, and executes in 0.015 ms on the
local PostgreSQL server. This is query evidence, not a production latency claim.

The first test selection had 90 passes and one failure: the injected PostgreSQL
acknowledgement error is wrapped by Drizzle, but the assertion expected its
message at the top level. The assertion now checks the original cause. No
production retry, hold, or timeout was weakened. A recursive unit-fixture type
initially prevented migration generation; an explicit fixture type fixed it.
Full CI and fresh review remain required before PR readiness.


Review found two real regressions. Before repair, a controlled concurrent archive
reported failed after archiving successfully, and shutdown sent a second wake
after stop was requested. Both regressions failed before the fix. Shared local
effect ownership fixes archive completion; bounded passes and abort propagation
fix backlog shutdown. The 93-case selection passes after repair. The paging test
also proves a failed first candidate cannot hide the 101st candidate across
bounded passes. A result-type alias initially made the known decision ID optional;
the alias now matches the existing outcome contract.

The existing tool-review and archive-delivery consumers also pass 124 real-DB
tests after AbortSignal propagation. Together with the decision/startup/signal
selection, 217 selected tests pass; the same nine older embedded-only cases
remain locally skipped. Workspace typecheck passes after the outcome type fix.