Deliver durable resource lifecycle events to plugins without polling (#16203)

## Thinking Path

> - Paperclip manages AI agents and their work.
> - Plugins keep external resources aligned with Paperclip projects and
agents.
> - The durable resource journal stores changes, but its plugin
interface requires polling.
> - Project deletion has no journal entry, so consumers must scan for
missing projects.
> - This PR delivers committed journal events through a native plugin
callback and records deletion.
> - Plugins can recover unfinished work without periodic change scans.

## Linked Issues or Issue Description

Related: #15343 introduced the durable inbox. #15631 added required
agent lifecycle participation. This change keeps both contracts and adds
automatic delivery for the resource inbox.

**What happened?**

Plugins must schedule reads of the durable inbox. A deleted project has
no durable deletion event. Consumers must scan saved bindings to detect
its removal.

**Expected behavior**

Core should deliver committed resource events and replay unfinished
work. Empty queues should have no polling timer. Project removal should
leave a durable event.

**Steps to reproduce**

1. Install a plugin that consumes project lifecycle events.
2. Create or update a project. The plugin must call `listLifecycle` to
discover it.
3. Delete the project. The journal has no deletion entry.

## What Changed

- Add `resourceLifecycle` to the manifest and `onResourceLifecycle` to
the SDK. Require `events.subscribe` and a configured, enabled company
scope.
- Reuse transaction-aware work signals and the delivery coordinator.
Wake after commit, on startup and plugin activation, and after
configuration changes.
- Acknowledge only a matching successful result. Retry unfinished or
interrupted work. Retain deferred events without an idle retry timer.
- Wake again after successful worker configuration application, and
retain failed scans through idle drain (both Apex findings fixed).
- Keep ordering per resource and deliver past failed or deferred
resources. Keep the manual inbox methods for existing consumers.
- Record project deletion in the removal transaction. Extend the journal
constraint with a generated migration.

## Verification

- Apex follow-up: 129 focused worker, delivery, coordinator, and
application tests passed; server typecheck passed. Regression coverage
verifies configuration-application races and failed scans across idle
drain.

- Focused lifecycle, transaction coordinator, manifest, and SDK dispatch
suites: 105 tests passed, including single-connection saves, redacted
failure diagnostics, wakeups retained during idle drain, and per-event
retry deadlines.
- Complete SDK suite plus delivery database tests: 128 tests passed.
- Shared, database, and SDK typechecks passed. Direct server `tsc
--noEmit` passed against rebuilt SDK declarations.
- Full local `pnpm test:run` finished with 19,491 passed, 148 skipped,
and 6 failed tests across 5 files (plus two suite setup errors). This
run overlapped branch updates and is not final-head evidence: failures
included missing Cargo, local Git default-branch assumptions, and stale
in-process lifecycle modules. Final Apex-fix commit `75d028788` passed
all CI gates, including the full test shards, and received Apex 5/5 with
no unresolved comments.
- `pnpm -r typecheck` and `pnpm build` were attempted. Both stopped
because Cargo is unavailable locally. CI supplies the Rust toolchain.
- Local installation used the repository's documented lockfile fallback
because master has dependencies missing from its lockfile. No dependency
changes are included here.

## Risks

- Delivery is at least once. Plugins must re-read current state and make
external effects idempotent.
- Signals are process-local, like the other delivery queues. Startup
recovers committed rows. Independent database writers or multiple active
server processes require an explicit ownership and wake protocol.
- Apply migration `0340_open_warlock.sql` before running code that
records project deletion. Earlier deletions cannot be reconstructed.
- Automatic delivery selects plugins with company configuration rows.
`deferred` waits for a later startup, configuration change, or other
work notification.
- This does not replace required agent lifecycle participation or
establish that external mounts are unused.

## Model Used

OpenAI Codex (GPT-6), with tool-assisted code editing, code inspection,
and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

doc/plugins/PLUGIN_AUTHORING_GUIDE.md

@@ -114,48 +114,71 @@ agents. The durable event inbox below is separate: acknowledging an event does
 not complete required lifecycle work. See [Agent lifecycle](../AGENT-LIFECYCLE.md)
 for phase ordering, recovery, and existing-installation limits.
 
-## Durable resource lifecycle inbox
+## Durable resource lifecycle delivery
 
-Plugins with `events.subscribe` can read durable, company-scoped resource hooks
-through `ctx.events.listLifecycle(companyId, limit?, afterId?)` and acknowledge successful
-work with `ctx.events.acknowledgeLifecycle(companyId, eventId)`. Use an existing
-plugin job to poll each configured company; the host checks invocation scope and whether
-the plugin is ready and enabled for that company. These methods are separate
-from the fire-and-forget `ctx.events.on()` bus.
-Proactive jobs and timers may read only companies authorized by the plugin's
-company configuration. Calls inside a host-issued invocation must match its company.
+Declare `resourceLifecycle: ["project"]` (or `["agent", "project"]`) in the
+manifest, with `events.subscribe`, and implement `onResourceLifecycle(event)`:
 
-An event has `id`, `companyId`, `resourceType`, `resourceId`, `action`, and
+```ts
+async onResourceLifecycle(event) {
+  const config = await ctx.config.get(event.companyId);
+  if (!config.provider) return { eventId: event.id, status: "deferred" };
+  const finished = await reconcileCurrentResource(event);
+  return { eventId: event.id, status: finished ? "acknowledged" : "retry" };
+}
+```
+
+Core delivers events to ready plugins with a configuration row for that company,
+unless the plugin is disabled there. Each invocation has the event's company
+scope. The dispatcher wakes after the mutation's outer transaction settles,
+on host startup, plugin activation/worker start, and company configuration or
+enablement changes. Successful worker `onConfigChanged` completion wakes delivery
+again, so a callback that deferred while configuration was still applying is
+revisited. Failed scans retain pending work through idle drain. Empty and deferred
+queues have no periodic scans. `retry`,
+exceptions, invalid results, and unavailable workers leave the event pending and
+schedule another attempt after 30 seconds. Other acknowledgments or mutations
+do not bypass that event's delay; a host restart may replay it immediately.
+`deferred` retains the event without
+a retry deadline, useful while waiting for operator configuration. A later wake
+revisits deferred events. Acknowledgment immediately exposes subsequent work.
+
+Events contain `id`, `companyId`, `resourceType`, `resourceId`, `action`, and
 `createdAt`. Agent actions are `create`, `pause`, `resume`, and `terminate`;
-project actions are `create`, `update`, and `archive`. Pending hires produce creation
-after approval. Project updates include repository/workspace mutations and restoring
-an archived project. Provider cleanup and retention policy belong to the plugin.
+project actions are `create`, `update`, `archive`, and `delete`. Pending hires
+produce creation after approval. Project updates include repository/workspace
+mutations and restoring an archived project. Deletion is recorded in the same
+transaction as removal and survives the project row. Earlier deletions cannot
+be reconstructed. Provider cleanup and retention policy belong to the plugin.
 
-Reads return at most one pending event per resource (default 50, maximum 100).
 Creation is delivered before other events for that resource, even when its
-backfilled ID is newer. Remaining events follow ID order. After acknowledging
-an event, a later read exposes its successor. A failed
-resource remains pending without blocking other resources; process the rest of
-the batch independently. Progress is stored per plugin, survives worker restarts,
-and is not a global sequence cursor. The delivery migration seeds existing hired
-agents and all projects once, including archived projects. Pending hires require
-approval; paused and terminated agents retain their current status intents.
-This baseline represents current desired state, not reconstructed history.
-The baseline includes archive for existing archived projects and update for restored projects whose
-journal still ends at archive. Archiving does not authorize provider cleanup.
-After that migration, capture stays forward-only; no later journal backfill runs.
+backfilled ID is newer. Remaining events follow ID order. Failed or deferred
+resources do not block independent resources. Acknowledgments are stored per
+plugin, survive worker/host restarts, and are not a global sequence cursor.
+The initial journal migration seeds existing hired agents and projects,
+including archived projects; this is a desired-state baseline, not history.
+Agent lifecycle participation (`onAgentLifecycle`) remains a separate contract:
+acknowledging a resource event does not advance an agent lifecycle phase.
 
-Delivery is at least once: concurrent reads or a crash after a provider operation
-can repeat an event. Serialize polling and use stable company/event idempotency
-keys, then acknowledge only after successful completion. Load current authorized
-agent/project/workspace data before acting; the journal contains no configuration
-snapshots, repository credentials, or deletion authority. For offline plugin tests,
-seed `lifecycleEvents` with `createTestHarness().seed()`.
+Delivery is at least once. Re-read current authorized state and make provider
+operations idempotent; a worker can finish an external effect and exit before
+Core commits its acknowledgment. Content-free events contain no repository
+credentials or configuration snapshots. Deleted projects may return `null`.
+Use stable resource identifiers and fence delayed provider requests when removing
+resources. Neither the callback nor an acknowledgment proves mounts are unused.
 
-Lifecycle polls can page past failed resources using the last returned event id as
-`afterId`. Reset `afterId` at the start of every polling sweep: it is a page
-cursor, never a persisted high-water mark. This retries failures and includes
-transactions that commit later with lower ids.
+Delivery uses the same process-local transaction signals as Core's other durable
+queues. Startup replays committed work. Independent database writers and separate
+active server processes need an explicit ownership/wake protocol; see
+[database work signals](../DATABASE.md#transaction-aware-delivery-work-signals).
+
+The existing `ctx.events.listLifecycle(companyId, limit?, afterId?)` and
+`acknowledgeLifecycle(companyId, eventId)` remain available for manual consumers.
+Do not mix automatic and manual acknowledgment for the same resources. Reads
+return one pending event per resource (default 50, maximum 100). `afterId` is a
+page cursor, never a durable high-water mark: restart pagination to revisit
+failures and transactions that commit later with lower IDs. Offline SDK tests
+can seed `lifecycleEvents` with `createTestHarness().seed()`.
 
 ## External object reference providers