Deliver durable resource lifecycle events to plugins without polling (#16203)

## Thinking Path

> - Paperclip manages AI agents and their work.
> - Plugins keep external resources aligned with Paperclip projects and
agents.
> - The durable resource journal stores changes, but its plugin
interface requires polling.
> - Project deletion has no journal entry, so consumers must scan for
missing projects.
> - This PR delivers committed journal events through a native plugin
callback and records deletion.
> - Plugins can recover unfinished work without periodic change scans.

## Linked Issues or Issue Description

Related: #15343 introduced the durable inbox. #15631 added required
agent lifecycle participation. This change keeps both contracts and adds
automatic delivery for the resource inbox.

**What happened?**

Plugins must schedule reads of the durable inbox. A deleted project has
no durable deletion event. Consumers must scan saved bindings to detect
its removal.

**Expected behavior**

Core should deliver committed resource events and replay unfinished
work. Empty queues should have no polling timer. Project removal should
leave a durable event.

**Steps to reproduce**

1. Install a plugin that consumes project lifecycle events.
2. Create or update a project. The plugin must call `listLifecycle` to
discover it.
3. Delete the project. The journal has no deletion entry.

## What Changed

- Add `resourceLifecycle` to the manifest and `onResourceLifecycle` to
the SDK. Require `events.subscribe` and a configured, enabled company
scope.
- Reuse transaction-aware work signals and the delivery coordinator.
Wake after commit, on startup and plugin activation, and after
configuration changes.
- Acknowledge only a matching successful result. Retry unfinished or
interrupted work. Retain deferred events without an idle retry timer.
- Wake again after successful worker configuration application, and
retain failed scans through idle drain (both Apex findings fixed).
- Keep ordering per resource and deliver past failed or deferred
resources. Keep the manual inbox methods for existing consumers.
- Record project deletion in the removal transaction. Extend the journal
constraint with a generated migration.

## Verification

- Apex follow-up: 129 focused worker, delivery, coordinator, and
application tests passed; server typecheck passed. Regression coverage
verifies configuration-application races and failed scans across idle
drain.

- Focused lifecycle, transaction coordinator, manifest, and SDK dispatch
suites: 105 tests passed, including single-connection saves, redacted
failure diagnostics, wakeups retained during idle drain, and per-event
retry deadlines.
- Complete SDK suite plus delivery database tests: 128 tests passed.
- Shared, database, and SDK typechecks passed. Direct server `tsc
--noEmit` passed against rebuilt SDK declarations.
- Full local `pnpm test:run` finished with 19,491 passed, 148 skipped,
and 6 failed tests across 5 files (plus two suite setup errors). This
run overlapped branch updates and is not final-head evidence: failures
included missing Cargo, local Git default-branch assumptions, and stale
in-process lifecycle modules. Final Apex-fix commit `75d028788` passed
all CI gates, including the full test shards, and received Apex 5/5 with
no unresolved comments.
- `pnpm -r typecheck` and `pnpm build` were attempted. Both stopped
because Cargo is unavailable locally. CI supplies the Rust toolchain.
- Local installation used the repository's documented lockfile fallback
because master has dependencies missing from its lockfile. No dependency
changes are included here.

## Risks

- Delivery is at least once. Plugins must re-read current state and make
external effects idempotent.
- Signals are process-local, like the other delivery queues. Startup
recovers committed rows. Independent database writers or multiple active
server processes require an explicit ownership and wake protocol.
- Apply migration `0340_open_warlock.sql` before running code that
records project deletion. Earlier deletions cannot be reconstructed.
- Automatic delivery selects plugins with company configuration rows.
`deferred` waits for a later startup, configuration change, or other
work notification.
- This does not replace required agent lifecycle participation or
establish that external mounts are unused.

## Model Used

OpenAI Codex (GPT-6), with tool-assisted code editing, code inspection,
and test execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Nicky LeachandPaperclip authored and GitHub committed 2026-10-13 12:00:30 -07:00
1 parent 5a6d0d08a7
commit db65a0863e
27 files changed
+1252 -145

No files matched your search

@@ -31,6 +31,7 @@ import {
appendStderrExcerpt,
createDuplexRouteSlotController,
createPluginWorkerHandle,
createPluginWorkerManager,
formatWorkerFailureMessage,
resolveRpcCallTimeoutMs,
} from "../services/plugin-worker-manager.js";
@@ -56,6 +57,28 @@ const TEST_MANIFEST: PaperclipPluginManifestV1 = {
entrypoints: { worker: "dist/worker.js" },
};
describe("worker configuration completion", () => {
it("notifies only after a successful config callback and releases subscriptions", async () => {
const manager = createPluginWorkerManager();
const applied = vi.fn(), unsubscribe = manager.onConfigApplied(applied);
try {
await manager.startWorker("test.plugin", {
entrypointPath: DELAYED_WORKER_ENTRYPOINT, manifest: TEST_MANIFEST, config: {},
instanceInfo: { instanceId: "instance-1", hostVersion: "1.0.0" }, apiVersion: 1, hostHandlers: {},
});
expect(applied).not.toHaveBeenCalled();
const updating = manager.call("test.plugin", "configChanged", { companyId: "company-1", config: { delayMs: 20 } });
expect(applied).not.toHaveBeenCalled();
await updating; expect(applied).toHaveBeenCalledTimes(1);
await expect(manager.call("test.plugin", "configChanged", { companyId: "company-1", config: { fail: true } })).rejects.toThrow("configuration rejected");
expect(applied).toHaveBeenCalledTimes(1);
unsubscribe();
await manager.call("test.plugin", "configChanged", { companyId: "company-1", config: {} });
expect(applied).toHaveBeenCalledTimes(1);
} finally { unsubscribe(); await manager.stopAll(); }
});
});
describe("resolveRpcCallTimeoutMs", () => {
const MAX_RPC_TIMEOUT_MS = 15 * 60 * 1_000;
const MAX_NODE_TIMER_TIMEOUT_MS = 2_147_483_647;