fix(wake-queue): save temporarily refused work intents and promote them through one path (#19956)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Users send work to agents as assignments, comments and Retries. The
wake queue delivers each one as a run
> - When an admission check refused a wake for a temporary reason, the
wake often ended as a terminal skip, so the work was lost
> - Restart-chaos waves found dropped comments and Retries (RC-37,
s7532) and double delivery (RC-53). Each fix covered one hold type
> - The durable ownership plan (section E) asks for one outbox rule: a
closed set of terminal outcomes, a saved deferral for every temporary
refusal, and one promotion path
> - This pull request adds that rule. It saves messages refused by a
pause, promotes deferred intents from every hold-release event through
one function, and adds a reconciler backstop
> - The benefit is that an accepted message is delivered exactly once
after a pause lifts, and the reconciler reports any intent that is stuck
or dropped

## Linked Issues or Issue Description

Refs #19831 (plan, section E), #19838 (inventory, wake-queue rows and
section 1b), #19927 (Phase 2 data: the I4 false positive), #19702,
#19741, #19829, #19826. Builds on #19944 (section D, merged).

**What happened?**
- A user commented while the agent was paused, or while the company was
paused. Admission refused the wake. Nothing saved the message as
pending, so when the pause lifted, nothing delivered it.
- Hold-release events woke different subsets of workers. A saved intent
then waited out a 30-second refusal backoff after its hold cleared.
- The I4 reconciler check matched free text with a regular expression.
Phase 2 found a false positive: a delivered retry whose error said
"retrying once".

**Expected behavior**
An accepted work intent ends only as delivered, consumed by a later run,
task cancelled, or hard stop. Every temporary refusal saves the intent
as `deferred` with its reason and its release trigger. Every
hold-release event promotes deferred intents through one function
(invariant 4).

**Steps to reproduce**
Pause an agent. Post a user comment on its task. Resume the agent. On
`master` the comment is never delivered as a wake.

## What Changed

- New `server/src/services/work-intent-outbox.ts`:
- `WORK_INTENT_TERMINAL_OUTCOMES`: the closed set of terminal outcomes.
- `TEMPORARY_REFUSAL_TRIGGERS`: maps each temporary refusal code to its
release trigger.
- `promoteDeferredWorkIntents`: the one promotion function. It removes
the 30-second refusal backoff from the matching deferred intents. It
then wakes run dispatch, the saved-receipt worker and stranded-work
recovery, and hints each saved intent of the issue.
- `heartbeat/queue.ts`: the #19829 save-as-deferred block is now a
shared helper. A user's message refused by a paused agent, a paused
company, or a budget pause that is not a reached hard stop is saved with
`executionWait: { reason, trigger, message }`. The caller still gets a
409. The message is saved only when the task's latest run is terminal,
because the release drain promotes a saved message behind that run. A
task that never ran keeps the message in the task, and stranded-work
recovery starts the task once the pause lifts. A `pending_approval`
agent and a task-tree hold are not changed here: the release drain still
fails a promoted wake for a pending approval, and tree holds have their
own resume path.
- Hold-release events call `promoteDeferredWorkIntents`:
  - accounting settled (`run-cost-accounting.ts`);
  - environment lease released (`environments.ts`, two sites);
  - native run projected terminal (`native-run-projection.ts`);
  - budget pause lifted (`budgets.ts`);
  - company reactivated (`companies.ts`);
  - task-tree hold released (`issue-tree-control.ts`);
  - agent lifecycle ready (`agent-lifecycle-driver.ts`).
- Reconciler (`recovery/ownership-registry.ts`,
`ownership-invariant-report.ts`, `native-finalization-reconciler.ts`):
- registers three new kinds: `wake_request_deferred`, `queued_run` and
`scheduled_retry_run`;
- I4 now reads the closed refusal code (the wake `reason`) of accepted
intents, not free text;
- a new I4 check reports a deferred pause refusal whose pause has lifted
for more than 5 minutes. The worker then promotes it as a backstop;
- a lifted pause inside its 5-minute grace window returns its deadline
through `nextDueAt`, and the worker also wakes on agent-lifecycle
signals (company and agent pause changes). So a missed release event is
still checked.
- `doc/SPEC-implementation.md`: adds the rule "Accepted work intents are
delivered exactly once".

## Verification

Each focused file ran alone in one vitest process (`--maxWorkers=1`) on
embedded Postgres. Results at `2875c85fd2`:

| Question | Loop | Elapsed | Result |
|---|---|---|---|
| Is a comment refused by a pause saved, then delivered once after the
pause lifts? | `heartbeat-held-comment-wake.test.ts -t "pause lifts"` |
8 s | 2 pass. With `queue.ts` from `master`, both fail: no intent is
saved |
| Does I4 use the closed code, and does it catch a stuck deferral? |
`ownership-reconciliation-report.test.ts` | 45 s | 20 pass. This
includes the Phase 2 "retrying once" false positive and the case
"paused, then lifted" |
| No regression on the paths that changed | held-comment (15),
archived-company guard (8), undispatched execution wait (32), deferred
receipt work (45), recovery pause durability (16), native run projection
work (55), tree-control routes and services (19, 1, 7) | 8–334 s each |
All pass |

- Exactly-once: each pause test runs two promotions (a release event,
then the reconciler backstop) and many dispatch passes. It asserts one
delivery. The existing consumption tests (RC-53) still pass.
- Server package typecheck passes.
- Speed: nothing new runs before `turn.start`. The new admission work
happens only on a refusal. Promotion runs in the release transaction and
in the background reconciler.
- Not run here: chaos waves. The coordinating session schedules them on
the shared harness.

## Risks

- `promoteDeferredWorkIntents` moves `updated_at` back for deferred
intents that were refused in the last 30 seconds. A busy company with
many release events can retry refused intents more often. Each retry
re-runs normal admission, and the refusal resets the backoff.
- More release sites now wake run dispatch and stranded-work recovery.
These notifications are coalesced per company.
- Lifting a pause is now the operator's explicit authorization to send
what the pause refused, as Send is after a Stop. When the pause stopped
the task's run, the lifted pause issues the task's next explicit wake,
which adopts the saved message once. Other messages queued behind an
acknowledged Stop keep the existing rule. If Stop recovery still holds
the task, the message waits for that recovery.
- An agent pause's stop sweep keeps the messages that the pause deferred
(`preservePauseDeferred`). Terminate and cleanup stops still end them.
- The backstop check reads only pause refusals (`company.inactive`,
`agent.not_invokable`), because those holds can be checked exactly.
Other deferred kinds keep their existing promotion paths.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

server/src/modules/wake-queue/adapters/postgres.ts

@@ -18,6 +18,7 @@ import { nativeManualAdmissionHoldsExecution } from "../../../services/native-ru
 import { and, asc, eq, inArray, isNull, notInArray, or, sql } from "drizzle-orm";
 import { readReleaseRun, type ReleaseRun } from "./release-run-read.js";
 import type { Db } from "@paperclipai/db";
+import { deliveredQueuedCommentIds } from "../../../services/queued-comment-delivery.js";
 import { extractIssueReferenceIdentifiers, isUuidLike } from "@paperclipai/shared";
 import {
   activityLog,
@@ -318,31 +319,10 @@ function buildTransaction(tx: Db, deps: WakeQueuePostgresAdapterDeps, db: Db, ru
         const row = rows.find((candidate) => candidate.id === commentId);
         return Boolean(row && !row.deletedAt && (!targetsFinishingRunAgent || row.createdByRunId !== finishingRunId));
       });
-      // A succeeded run of the wake's agent that started after the comment's
-      // latest revision consumed it when it received the comment as its own
-      // wake input, or when it was a retry of an interrupted turn whose task
-      // history included the comment (the retry re-reads the whole thread).
-      // Promoting the saved wake again would deliver it twice and can reopen
-      // finished work. Task history alone does not consume a message saved
-      // for its own turn, and a run that was already running never counts:
-      // its context can be refreshed after the fact, and an edit made after
-      // it started is still unread.
-      const delivered = new Set(liveAndUnauthored.length === 0 ? [] : (await tx.select({ id: issueComments.id })
-        .from(issueComments).where(and(
-          eq(issueComments.companyId, companyId), eq(issueComments.issueId, issueId),
-          inArray(issueComments.id, liveAndUnauthored),
-          sql`exists (select 1 from ${heartbeatRuns} delivered_run
-            where delivered_run.company_id = ${companyId} and delivered_run.agent_id = ${wakeAgentId}
-              and delivered_run.status = 'succeeded'
-              and (delivered_run.native_issue_id = ${issueId} or delivered_run.context_snapshot->>'issueId' = ${issueId})
-              and delivered_run.started_at > ${issueComments.createdAt}
-              and delivered_run.started_at >= ${issueComments.updatedAt}
-              and (coalesce(delivered_run.context_snapshot->'wakeCommentIds', '[]'::jsonb) ? ${issueComments.id}::text
-                or (delivered_run.retry_of_run_id is not null and jsonb_path_exists(delivered_run.context_snapshot, '$.executionContinuation.messages[*] ? (@.id == $id)',
-                  jsonb_build_object('id', ${issueComments.id}::text))
-                or jsonb_path_exists(delivered_run.context_snapshot, '$.paperclipWake.messages[*] ? (@.id == $id)',
-                  jsonb_build_object('id', ${issueComments.id}::text)))))`,
-        ))).map(row => row.id));
+      // Promoting a saved wake whose comments a completed run already
+      // received would deliver them twice and can reopen finished work.
+      const delivered = await deliveredQueuedCommentIds(tx, { companyId, issueId, agentId: wakeAgentId,
+        commentIds: liveAndUnauthored });
       const liveNonSelfCommentIds = liveAndUnauthored.filter(commentId => !delivered.has(commentId));
       const containedSelfAuthoredComment = rows.some(
         (row) => targetsFinishingRunAgent && !row.deletedAt && row.createdByRunId === finishingRunId,