Deliver durable resource lifecycle events to plugins without polling (#16203)

## Thinking Path

> - Paperclip manages AI agents and their work.
> - Plugins keep external resources aligned with Paperclip projects and
agents.
> - The durable resource journal stores changes, but its plugin
interface requires polling.
> - Project deletion has no journal entry, so consumers must scan for
missing projects.
> - This PR delivers committed journal events through a native plugin
callback and records deletion.
> - Plugins can recover unfinished work without periodic change scans.

## Linked Issues or Issue Description

Related: #15343 introduced the durable inbox. #15631 added required
agent lifecycle participation. This change keeps both contracts and adds
automatic delivery for the resource inbox.

**What happened?**

Plugins must schedule reads of the durable inbox. A deleted project has
no durable deletion event. Consumers must scan saved bindings to detect
its removal.

**Expected behavior**

Core should deliver committed resource events and replay unfinished
work. Empty queues should have no polling timer. Project removal should
leave a durable event.

**Steps to reproduce**

1. Install a plugin that consumes project lifecycle events.
2. Create or update a project. The plugin must call `listLifecycle` to
discover it.
3. Delete the project. The journal has no deletion entry.

## What Changed

- Add `resourceLifecycle` to the manifest and `onResourceLifecycle` to
the SDK. Require `events.subscribe` and a configured, enabled company
scope.
- Reuse transaction-aware work signals and the delivery coordinator.
Wake after commit, on startup and plugin activation, and after
configuration changes.
- Acknowledge only a matching successful result. Retry unfinished or
interrupted work. Retain deferred events without an idle retry timer.
- Wake again after successful worker configuration application, and
retain failed scans through idle drain (both Apex findings fixed).
- Keep ordering per resource and deliver past failed or deferred
resources. Keep the manual inbox methods for existing consumers.
- Record project deletion in the removal transaction. Extend the journal
constraint with a generated migration.

## Verification

- Apex follow-up: 129 focused worker, delivery, coordinator, and
application tests passed; server typecheck passed. Regression coverage
verifies configuration-application races and failed scans across idle
drain.

- Focused lifecycle, transaction coordinator, manifest, and SDK dispatch
suites: 105 tests passed, including single-connection saves, redacted
failure diagnostics, wakeups retained during idle drain, and per-event
retry deadlines.
- Complete SDK suite plus delivery database tests: 128 tests passed.
- Shared, database, and SDK typechecks passed. Direct server `tsc
--noEmit` passed against rebuilt SDK declarations.
- Full local `pnpm test:run` finished with 19,491 passed, 148 skipped,
and 6 failed tests across 5 files (plus two suite setup errors). This
run overlapped branch updates and is not final-head evidence: failures
included missing Cargo, local Git default-branch assumptions, and stale
in-process lifecycle modules. Final Apex-fix commit `75d028788` passed
all CI gates, including the full test shards, and received Apex 5/5 with
no unresolved comments.
- `pnpm -r typecheck` and `pnpm build` were attempted. Both stopped
because Cargo is unavailable locally. CI supplies the Rust toolchain.
- Local installation used the repository's documented lockfile fallback
because master has dependencies missing from its lockfile. No dependency
changes are included here.

## Risks

- Delivery is at least once. Plugins must re-read current state and make
external effects idempotent.
- Signals are process-local, like the other delivery queues. Startup
recovers committed rows. Independent database writers or multiple active
server processes require an explicit ownership and wake protocol.
- Apply migration `0340_open_warlock.sql` before running code that
records project deletion. Earlier deletions cannot be reconstructed.
- Automatic delivery selects plugins with company configuration rows.
`deferred` waits for a later startup, configuration change, or other
work notification.
- This does not replace required agent lifecycle participation or
establish that external mounts are unused.

## Model Used

OpenAI Codex (GPT-6), with tool-assisted code editing, code inspection,
and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Nicky LeachandPaperclip authored and GitHub committed 2026-10-13 12:00:30 -07:00
1 parent 5a6d0d08a7
commit db65a0863e
27 files changed
+1252 -145

No files matched your search

+2
View File
@@ -689,6 +689,8 @@ export interface PaperclipPluginManifestV1 {
capabilities: PluginCapability[];
/** Required participant in agent preparation, stop, resume, and cleanup. */
agentLifecycle?: true;
/** Durable, acknowledged resource events delivered to onResourceLifecycle. Requires events.subscribe. */
resourceLifecycle?: Array<"agent" | "project">;
/** Opt into the native pooled-connection catalog, setup, and management UI. */
aiConnectionRouter?: { name: string; description: string };
/** Entrypoint paths relative to the package root. */
@@ -39,6 +39,15 @@ describe("plugin manifest validators", () => {
expect(pluginManifestV1Schema.parse({ ...manifest, capabilities: ["agents.lifecycle.manage"] }).agentLifecycle).toBe(true);
});
it("requires subscription authority and known resource types for durable delivery", () => {
const manifest = { id: "example.lifecycle", apiVersion: 1, version: "1.0.0", displayName: "Lifecycle", description: "Lifecycle tests", author: "Tests", categories: ["automation"], entrypoints: { worker: "worker.js" }, resourceLifecycle: ["project"] };
expect(pluginManifestV1Schema.safeParse({ ...manifest, capabilities: ["projects.read"] }).success).toBe(false);
expect(pluginManifestV1Schema.parse({ ...manifest, capabilities: ["events.subscribe"] }).resourceLifecycle).toEqual(["project"]);
for (const resourceLifecycle of [[], ["issue"]]) {
expect(pluginManifestV1Schema.safeParse({ ...manifest, resourceLifecycle, capabilities: ["events.subscribe"] }).success).toBe(false);
}
});
it("requires routing authority for native pooled connector declarations without a custom UI bundle", () => {
const manifest = { id: "example.pool", apiVersion: 1, version: "0.1.0", displayName: "Pool", description: "Pool", author: "Tests", categories: ["connector"], entrypoints: { worker: "worker.js" }, aiConnectionRouter: { name: "AI connection pool", description: "Use saved connections" } };
expect(pluginManifestV1Schema.safeParse({ ...manifest, capabilities: ["ui.page.register"] }).success).toBe(false);
+5
View File
@@ -783,6 +783,7 @@ export const pluginManifestV1Schema = z.object({
).optional(),
capabilities: z.array(z.enum(PLUGIN_CAPABILITIES)).min(1),
agentLifecycle: z.literal(true).optional(),
resourceLifecycle: z.array(z.enum(["agent", "project"])).min(1).max(2).optional(),
aiConnectionRouter: z.object({ name: z.string().trim().min(1).max(100), description: z.string().trim().min(1).max(500) }).strict().optional(),
entrypoints: z.object({
worker: z.string().min(1),
@@ -811,6 +812,10 @@ export const pluginManifestV1Schema = z.object({
ctx.addIssue({ code: z.ZodIssueCode.custom, message: "Agent lifecycle participation requires agents.lifecycle.manage", path: ["capabilities"] });
}
if (manifest.resourceLifecycle && !manifest.capabilities.includes("events.subscribe")) {
ctx.addIssue({ code: z.ZodIssueCode.custom, message: "Resource lifecycle delivery requires events.subscribe", path: ["capabilities"] });
}
// ── Entrypoint ↔ UI slot consistency ──────────────────────────────────
// Plugins that declare UI slots must also declare a UI entrypoint so the
// host knows where to load the bundle from (PLUGIN_SPEC.md §10.1).