Deliver durable resource lifecycle events to plugins without polling (#16203)

## Thinking Path

> - Paperclip manages AI agents and their work.
> - Plugins keep external resources aligned with Paperclip projects and
agents.
> - The durable resource journal stores changes, but its plugin
interface requires polling.
> - Project deletion has no journal entry, so consumers must scan for
missing projects.
> - This PR delivers committed journal events through a native plugin
callback and records deletion.
> - Plugins can recover unfinished work without periodic change scans.

## Linked Issues or Issue Description

Related: #15343 introduced the durable inbox. #15631 added required
agent lifecycle participation. This change keeps both contracts and adds
automatic delivery for the resource inbox.

**What happened?**

Plugins must schedule reads of the durable inbox. A deleted project has
no durable deletion event. Consumers must scan saved bindings to detect
its removal.

**Expected behavior**

Core should deliver committed resource events and replay unfinished
work. Empty queues should have no polling timer. Project removal should
leave a durable event.

**Steps to reproduce**

1. Install a plugin that consumes project lifecycle events.
2. Create or update a project. The plugin must call `listLifecycle` to
discover it.
3. Delete the project. The journal has no deletion entry.

## What Changed

- Add `resourceLifecycle` to the manifest and `onResourceLifecycle` to
the SDK. Require `events.subscribe` and a configured, enabled company
scope.
- Reuse transaction-aware work signals and the delivery coordinator.
Wake after commit, on startup and plugin activation, and after
configuration changes.
- Acknowledge only a matching successful result. Retry unfinished or
interrupted work. Retain deferred events without an idle retry timer.
- Wake again after successful worker configuration application, and
retain failed scans through idle drain (both Apex findings fixed).
- Keep ordering per resource and deliver past failed or deferred
resources. Keep the manual inbox methods for existing consumers.
- Record project deletion in the removal transaction. Extend the journal
constraint with a generated migration.

## Verification

- Apex follow-up: 129 focused worker, delivery, coordinator, and
application tests passed; server typecheck passed. Regression coverage
verifies configuration-application races and failed scans across idle
drain.

- Focused lifecycle, transaction coordinator, manifest, and SDK dispatch
suites: 105 tests passed, including single-connection saves, redacted
failure diagnostics, wakeups retained during idle drain, and per-event
retry deadlines.
- Complete SDK suite plus delivery database tests: 128 tests passed.
- Shared, database, and SDK typechecks passed. Direct server `tsc
--noEmit` passed against rebuilt SDK declarations.
- Full local `pnpm test:run` finished with 19,491 passed, 148 skipped,
and 6 failed tests across 5 files (plus two suite setup errors). This
run overlapped branch updates and is not final-head evidence: failures
included missing Cargo, local Git default-branch assumptions, and stale
in-process lifecycle modules. Final Apex-fix commit `75d028788` passed
all CI gates, including the full test shards, and received Apex 5/5 with
no unresolved comments.
- `pnpm -r typecheck` and `pnpm build` were attempted. Both stopped
because Cargo is unavailable locally. CI supplies the Rust toolchain.
- Local installation used the repository's documented lockfile fallback
because master has dependencies missing from its lockfile. No dependency
changes are included here.

## Risks

- Delivery is at least once. Plugins must re-read current state and make
external effects idempotent.
- Signals are process-local, like the other delivery queues. Startup
recovers committed rows. Independent database writers or multiple active
server processes require an explicit ownership and wake protocol.
- Apply migration `0340_open_warlock.sql` before running code that
records project deletion. Earlier deletions cannot be reconstructed.
- Automatic delivery selects plugins with company configuration rows.
`deferred` waits for a later startup, configuration change, or other
work notification.
- This does not replace required agent lifecycle participation or
establish that external mounts are unused.

## Model Used

OpenAI Codex (GPT-6), with tool-assisted code editing, code inspection,
and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Nicky LeachandPaperclip authored and GitHub committed 2026-10-13 12:00:30 -07:00
1 parent 5a6d0d08a7
commit db65a0863e
27 files changed
+1252 -145

No files matched your search

+15 -1
View File
@@ -796,6 +796,8 @@ export interface WorkerDiagnostics {
* for starting/stopping all workers and routing RPC calls.
*/
export interface PluginWorkerManager {
/** Subscribe after a worker successfully applies live company configuration. */
onConfigApplied(listener: () => void): () => void;
inspectIdleSleep?(hold: { ownerId: string; expiresAt: number }): Promise<{
backgroundWork: "none" | "present" | "unknown"; pluginIds: string[];
}>;
@@ -3726,6 +3728,7 @@ export function createPluginWorkerManager(
): PluginWorkerManager {
const log = logger.child({ service: "plugin-worker-manager" });
const workers = new Map<string, PluginWorkerHandle>();
const configApplied = new Set<() => void>();
/** Per-plugin startup locks to prevent concurrent spawn races. */
const startupLocks = new Map<string, Promise<PluginWorkerHandle>>();
// The one shared, process-scoped aggregate route-slot controller. The manager
@@ -3736,6 +3739,10 @@ export function createPluginWorkerManager(
);
return {
onConfigApplied(listener) {
configApplied.add(listener);
return () => { configApplied.delete(listener); };
},
async inspectIdleSleep(hold) {
if (startupLocks.size) return { backgroundWork: "unknown", pluginIds: [] };
const snapshot = [...workers.entries()];
@@ -3872,7 +3879,14 @@ export function createPluginWorkerManager(
new Error(`No worker registered for plugin "${pluginId}"`),
);
}
return handle.call(method, params, timeoutMs, executeLogSink);
const result = handle.call(method, params, timeoutMs, executeLogSink);
if (method === "configChanged") void result.then(() => {
for (const listener of configApplied) {
try { listener(); }
catch { log.warn({ pluginId }, "Plugin configuration listener failed"); }
}
}, () => {});
return result;
},
openLoginPtySession(pluginId: string, input: LoginPtyOpenInput) {