Deliver durable resource lifecycle events to plugins without polling (#16203)

## Thinking Path

> - Paperclip manages AI agents and their work.
> - Plugins keep external resources aligned with Paperclip projects and
agents.
> - The durable resource journal stores changes, but its plugin
interface requires polling.
> - Project deletion has no journal entry, so consumers must scan for
missing projects.
> - This PR delivers committed journal events through a native plugin
callback and records deletion.
> - Plugins can recover unfinished work without periodic change scans.

## Linked Issues or Issue Description

Related: #15343 introduced the durable inbox. #15631 added required
agent lifecycle participation. This change keeps both contracts and adds
automatic delivery for the resource inbox.

**What happened?**

Plugins must schedule reads of the durable inbox. A deleted project has
no durable deletion event. Consumers must scan saved bindings to detect
its removal.

**Expected behavior**

Core should deliver committed resource events and replay unfinished
work. Empty queues should have no polling timer. Project removal should
leave a durable event.

**Steps to reproduce**

1. Install a plugin that consumes project lifecycle events.
2. Create or update a project. The plugin must call `listLifecycle` to
discover it.
3. Delete the project. The journal has no deletion entry.

## What Changed

- Add `resourceLifecycle` to the manifest and `onResourceLifecycle` to
the SDK. Require `events.subscribe` and a configured, enabled company
scope.
- Reuse transaction-aware work signals and the delivery coordinator.
Wake after commit, on startup and plugin activation, and after
configuration changes.
- Acknowledge only a matching successful result. Retry unfinished or
interrupted work. Retain deferred events without an idle retry timer.
- Wake again after successful worker configuration application, and
retain failed scans through idle drain (both Apex findings fixed).
- Keep ordering per resource and deliver past failed or deferred
resources. Keep the manual inbox methods for existing consumers.
- Record project deletion in the removal transaction. Extend the journal
constraint with a generated migration.

## Verification

- Apex follow-up: 129 focused worker, delivery, coordinator, and
application tests passed; server typecheck passed. Regression coverage
verifies configuration-application races and failed scans across idle
drain.

- Focused lifecycle, transaction coordinator, manifest, and SDK dispatch
suites: 105 tests passed, including single-connection saves, redacted
failure diagnostics, wakeups retained during idle drain, and per-event
retry deadlines.
- Complete SDK suite plus delivery database tests: 128 tests passed.
- Shared, database, and SDK typechecks passed. Direct server `tsc
--noEmit` passed against rebuilt SDK declarations.
- Full local `pnpm test:run` finished with 19,491 passed, 148 skipped,
and 6 failed tests across 5 files (plus two suite setup errors). This
run overlapped branch updates and is not final-head evidence: failures
included missing Cargo, local Git default-branch assumptions, and stale
in-process lifecycle modules. Final Apex-fix commit `75d028788` passed
all CI gates, including the full test shards, and received Apex 5/5 with
no unresolved comments.
- `pnpm -r typecheck` and `pnpm build` were attempted. Both stopped
because Cargo is unavailable locally. CI supplies the Rust toolchain.
- Local installation used the repository's documented lockfile fallback
because master has dependencies missing from its lockfile. No dependency
changes are included here.

## Risks

- Delivery is at least once. Plugins must re-read current state and make
external effects idempotent.
- Signals are process-local, like the other delivery queues. Startup
recovers committed rows. Independent database writers or multiple active
server processes require an explicit ownership and wake protocol.
- Apply migration `0340_open_warlock.sql` before running code that
records project deletion. Earlier deletions cannot be reconstructed.
- Automatic delivery selects plugins with company configuration rows.
`deferred` waits for a later startup, configuration change, or other
work notification.
- This does not replace required agent lifecycle participation or
establish that external mounts are unused.

## Model Used

OpenAI Codex (GPT-6), with tool-assisted code editing, code inspection,
and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

packages/plugins/sdk/src/define-plugin.ts

@@ -51,7 +51,7 @@ import type { AiConnectionRouterRequest, AiConnectionRouterResult } from "@paper
  * ```
  */
 
-import type { PluginContext } from "./types.js";
+import type { PluginContext, ResourceLifecycleEvent, ResourceLifecycleResult } from "./types.js";
 import type {
   PluginEnvironmentAcquireLeaseParams,
   PluginEnvironmentDestroyLeaseParams,
@@ -360,6 +360,14 @@ export interface PluginDefinition {
    */
   onAgentLifecycle?(params: AgentLifecycleRequest): Promise<AgentLifecycleResult>;
 
+  /**
+   * Reconcile a committed resource event. Requires manifest resourceLifecycle and events.subscribe.
+   * The host acknowledges successful results and replays interrupted/unfinished work.
+   * Calls are at least once, in order per resource. Re-read current state and use
+   * idempotent external operations; deletion events remain available after the row is gone.
+   */
+  onResourceLifecycle?(event: ResourceLifecycleEvent): Promise<ResourceLifecycleResult>;
+
   /** Propose a member from host-authorized candidates. Requires ai.connections.route. */
   onRouteAiConnection?(params: AiConnectionRouterRequest): Promise<AiConnectionRouterResult>;
 

packages/plugins/sdk/src/index.ts

@@ -239,6 +239,7 @@ export type {
   PluginLocalFoldersClient,
   PluginEventsClient,
   ResourceLifecycleEvent,
+  ResourceLifecycleResult,
   PluginJobsClient,
   PluginLaunchersClient,
   PluginHttpClient,

packages/plugins/sdk/src/protocol.ts

@@ -56,6 +56,7 @@ export type { PluginLauncherRenderContextSnapshot } from "@paperclipai/shared";
 import type {
   PluginEvent,
   ResourceLifecycleEvent,
+  ResourceLifecycleResult,
   PluginIssueCheckoutOwnership,
   PluginIssueOrchestrationSummary,
   PluginIssueRelationSummary,
@@ -1351,6 +1352,7 @@ export interface HostToWorkerMethods {
   handleApiRequest: [params: PluginApiRequestInput, result: PluginApiResponse];
   /** @see PLUGIN_SPEC.md §13.8 */
   agentLifecycle: [params: AgentLifecycleRequest, result: AgentLifecycleResult];
+  resourceLifecycle: [params: ResourceLifecycleEvent, result: ResourceLifecycleResult];
   getData: [params: GetDataParams, result: unknown];
   /** @see PLUGIN_SPEC.md §13.9 */
   performAction: [params: PerformActionParams, result: unknown];
@@ -1488,6 +1490,7 @@ export const HOST_TO_WORKER_OPTIONAL_METHODS: readonly HostToWorkerMethodName[]
   "handleWebhook",
   "handleApiRequest",
   "agentLifecycle",
+  "resourceLifecycle",
   "getData",
   "performAction",
   "executeTool",

packages/plugins/sdk/src/types.ts

@@ -547,7 +547,8 @@ export interface PluginLocalFoldersClient {
 export interface PluginEventsClient {
   /** Read durable resource hooks. Requires events.subscribe and a company scope.
    * Creation is delivered first, then the remaining events in id order.
-   * Page using afterId; reset it each polling sweep to retry failures and late commits.
+   * Prefer manifest resourceLifecycle + onResourceLifecycle for automatic delivery.
+   * Manual consumers page using afterId and restart pagination to revisit unacknowledged events.
    * Events repeat until acknowledged; use a company-scoped provider idempotency key.
    */
   listLifecycle(companyId: string, limit?: number, afterId?: string): Promise<ResourceLifecycleEvent[]>;
@@ -597,9 +598,18 @@ export type ResourceLifecycleEvent = {
   createdAt: string;
 } & (
   | { resourceType: "agent"; action: "create" | "pause" | "resume" | "terminate" }
-  | { resourceType: "project"; action: "create" | "update" | "archive" }
+  | { resourceType: "project"; action: "create" | "update" | "archive" | "delete" }
 );
 
+/** Return acknowledged only after durable effects finish. Retry unfinished work;
+ * defer when configuration is missing (replayed on configuration changes or startup).
+ * Both retry and deferred leave the event unacknowledged.
+ */
+export interface ResourceLifecycleResult {
+  eventId: string;
+  status: "acknowledged" | "retry" | "deferred";
+}
+
 /**
  * `ctx.jobs` — register handlers for scheduled jobs declared in the manifest.
  *

packages/plugins/sdk/src/worker-rpc-host.ts

@@ -39,6 +39,7 @@ import { createPluginIdleDrain } from "./idle-drain.js";
  * @see PLUGIN_SPEC.md §14 — SDK Surface
  */
 
+import type { ResourceLifecycleEvent } from "./types.js";
 import fs from "node:fs";
 import { AsyncLocalStorage } from "node:async_hooks";
 import path from "node:path";
@@ -1660,6 +1661,9 @@ export function startWorkerRpcHost(options: WorkerRpcHostOptions): WorkerRpcHost
         return handleExecuteTool(params as ExecuteToolParams);
       case "detectExternalObjects":
         return handleDetectExternalObjects(params as DetectExternalObjectsParams);
+      case "resourceLifecycle":
+        if (!plugin.definition.onResourceLifecycle) throw methodNotImplemented("resourceLifecycle");
+        return plugin.definition.onResourceLifecycle(params as ResourceLifecycleEvent);
       case "agentLifecycle":
         if (!plugin.definition.onAgentLifecycle) throw methodNotImplemented("agentLifecycle");
         return plugin.definition.onAgentLifecycle(params as AgentLifecycleRequest);
@@ -1783,6 +1787,7 @@ export function startWorkerRpcHost(options: WorkerRpcHostOptions): WorkerRpcHost
     if (plugin.definition.onIdleDrain) supportedMethods.push("prepareIdleSleep", "releaseIdleSleep");
     if (plugin.definition.onApiRequest) supportedMethods.push("handleApiRequest");
     if (plugin.definition.onDetectExternalObjects) supportedMethods.push("detectExternalObjects");
+    if (plugin.definition.onResourceLifecycle) supportedMethods.push("resourceLifecycle");
     if (plugin.definition.onAgentLifecycle) supportedMethods.push("agentLifecycle");
     if (plugin.definition.onRouteAiConnection) supportedMethods.push("routeAiConnection");
     if (plugin.definition.onResolveExternalObject) supportedMethods.push("resolveExternalObject");

packages/plugins/sdk/tests/worker-rpc-host.test.ts

@@ -73,6 +73,38 @@ describe("isWorkerEntrypoint", () => {
   });
 });
 
+describe("worker resource lifecycle delivery", () => {
+  it("advertises and dispatches the native lifecycle callback", async () => {
+    const stdin = new PassThrough(), stdout = new PassThrough();
+    const lines = createInterface({ input: stdout });
+    const plugin = definePlugin({
+      async setup() {},
+      async onResourceLifecycle(event) { return { eventId: event.id, status: "retry" }; },
+    });
+    const worker = startWorkerRpcHost({ plugin, stdin, stdout });
+    async function call(method: string, params: unknown) {
+      const response = new Promise<JsonRpcResponse>(resolve => {
+        const handler = (line: string) => {
+          const message = parseMessage(line);
+          if (isJsonRpcResponse(message)) { lines.off("line", handler); resolve(message); }
+        };
+        lines.on("line", handler);
+      });
+      stdin.write(serializeMessage(createRequest(method, params, method)));
+      return response;
+    }
+    try {
+      const initialized = await call("initialize", {
+        manifest: { id: "test.lifecycle", apiVersion: 1, version: "1.0.0", displayName: "Lifecycle", description: "Test", author: "Tests", categories: ["automation"], capabilities: ["events.subscribe"], resourceLifecycle: ["project"], entrypoints: { worker: "worker.js" } },
+        config: {}, databaseNamespace: null,
+      });
+      expect(initialized).toMatchObject({ result: { supportedMethods: expect.arrayContaining(["resourceLifecycle"]) } });
+      expect(await call("resourceLifecycle", { id: "17", companyId: "company", resourceType: "project", resourceId: "project", action: "delete", createdAt: "2020-01-01T00:00:00Z" }))
+        .toMatchObject({ result: { eventId: "17", status: "retry" } });
+    } finally { worker.stop(); lines.close(); stdin.destroy(); stdout.destroy(); }
+  });
+});
+
 describe("worker performAction context", () => {
   it("does not derive context companyId from caller params without host actor context", async () => {
     const hostToWorker = new PassThrough();