Files
PaperClipAI/server/src/services/plugin-lifecycle-inbox.ts
T
Nicky LeachandPaperclip 1cebdd4c4f Add plugin lifecycle delivery and an initial resource baseline (#15343)
## Thinking Path

> - Paperclip manages agents and projects for work.
> - Their lifecycle changes already commit to a durable journal.
> - Plugins need to read these records and track completed work.
> - Existing resources need a one-time baseline before delivery is
enabled.
> - Worker crashes must leave unfinished records available for retry.
> - Each plugin needs its own progress and company access checks.
> - This PR adds a pull inbox through the existing plugin SDK and job
system.

## Linked Issues or Issue Description

**Problem or motivation**

Plugins cannot consume the durable resource lifecycle journal.
In-process notifications can disappear during a restart and cannot
record successful completion. Resource plugins need retries, company
boundaries, and ordered transitions.

**Proposed solution**

Add `ctx.events.listLifecycle(companyId, limit?, afterId?)` and
`ctx.events.acknowledgeLifecycle(companyId, eventId)` under
`events.subscribe`. Seed a one-time baseline from current resource
state. Deliver creation first, then the remaining transitions in ID
order. Store acknowledgments for each plugin. Use existing plugin jobs
to poll configured companies.

**Alternatives considered**

A global sequence cursor can skip lower IDs that commit later.
Fire-and-forget subscriptions cannot record completion. A new dispatcher
is unnecessary because plugin jobs support polling. Consumers serialize
polling and use provider idempotency keys.

**Roadmap alignment**

This extends the existing plugin system and builds on #15280 and #15306.
Searches found no duplicate resource inbox work. Related #13306 exposes
decision events on the in-process bus. This PR includes the initial
journal baseline. Provider provisioning remains separate work.

## What Changed

- Add company-scoped lifecycle reads and acknowledgments to the SDK and
worker RPC host.
- Gate both methods by capability, invocation or proactive company
scope, plugin readiness, and company enablement.
- Seed hired agents and all projects once. Preserve paused/terminated
agent state and archived project state. Keep pending hires behind
approval.
- Preserve existing history and deliver backfilled creation before
partial transition histories.
- Deliver project archive events from #15371 through the SDK. Seed
archive intents for archived projects and update intents for active
projects with a partial archive history.
- Store acknowledgments per plugin and reject acknowledgments that skip
earlier resource events.
- Page past failed resources while retaining their pending records.
Reset the page cursor each sweep to include late commits.
- Add acknowledgment storage, an index for resource ordering, tests, and
authoring guidance.
- Preserve native identity definitions and sequence progress in
JavaScript backups. Repair journal ID generators lost by older backups
before seeding the baseline, without changing existing IDs.

## Verification

- `pnpm -r typecheck` passed after the final origin/master rebase.
- `pnpm build` passed before the final metadata rebase. The final CI
build also passed.
- All 46 focused lifecycle, SDK harness/RPC, migration snapshot, and
legacy restore checks passed again with migration 0309.
- Checks cover retries, per-plugin progress, resource order,
capability/company boundaries, baseline idempotency, approval gates,
archive delivery, restored projects with partial histories, and atomic
migration rollback.
- Full CI passed on final commit `0e674b87fc`. One unrelated Cursor
fixture hit a 10-second timeout; it passed locally in under one second,
and the failed server shard passed on retry.
- Greptile scored the final commit 5/5 with no unresolved review
threads.
- The branch is rebased onto origin/master and is conflict-free.
- Earlier full local runs were stopped as scope changed. CI ran the
complete repository suite.
- `git diff --check` and a local secret/PII scan passed.

## Risks


- Apply migration `0309_loving_the_hood.sql` before starting the new
server. It creates acknowledgment storage and an index, then seeds the
baseline in the same transaction. Agent/project writes wait for the
migration to commit. Keep these writes quiesced through the migration
and activation of the new capture-capable server; do not resume an older
runtime that lacks capture after the baseline.
- The baseline records current desired state, not historical
transitions. It includes archived projects and terminated-agent cleanup
intents. It runs once with the delivery migration; no later or runtime
journal backfill is planned.
- Delivery is at least once. Concurrent reads can repeat an event.
Consumers must serialize polling, use stable company/event idempotency
keys, and acknowledge successful operations only.
- Reset `afterId` at the start of every polling sweep. It is a page
cursor, not a persisted high-water mark.
- Deleting a plugin removes its acknowledgment records. A new
installation may replay existing journal events.
- Records contain identity and action. Consumers must load current
authorized data before acting. Cleanup and retention policy belong to
the provider plugin.
- Provider calls, VM/volume provisioning, and journal retention are
outside this PR.

## Model Used

OpenAI Codex, based on GPT-6, with reasoning, repository inspection,
code execution, and tool use. The exact deployment model ID and context
window are not exposed in this session.

## Checklist


- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-06 12:27:09 -07:00

59 lines
3.9 KiB
TypeScript

import { pluginCompanySettings, pluginLifecycleAcknowledgments, plugins, resourceLifecycleEvents, type Db } from "@paperclipai/db";
import type { ResourceLifecycleEvent } from "@paperclipai/plugin-sdk";
import { z } from "zod";
import { and, asc, eq, gt, sql } from "drizzle-orm";
import { badRequest, conflict, forbidden, notFound } from "../errors.js";
/** Durable pull delivery. Provider work stays outside tenant transactions. */
export function pluginLifecycleInbox(db: Db, pluginId: string) {
const unacknowledged = sql`NOT EXISTS (
SELECT 1 FROM plugin_lifecycle_acknowledgments ack
WHERE ack.plugin_id = ${pluginId} AND ack.event_id = ${resourceLifecycleEvents.id}
)`;
const firstForResource = sql`NOT EXISTS (
SELECT 1 FROM resource_lifecycle_events older
WHERE older.company_id = ${resourceLifecycleEvents.companyId}
AND older.resource_type = ${resourceLifecycleEvents.resourceType}
AND older.resource_id = ${resourceLifecycleEvents.resourceId}
-- Backfilled creation must precede transitions captured before this baseline.
AND ${resourceLifecycleEvents.action} <> 'create'
AND (older.action = 'create' OR older.id < ${resourceLifecycleEvents.id})
AND NOT EXISTS (SELECT 1 FROM plugin_lifecycle_acknowledgments ack
WHERE ack.plugin_id = ${pluginId} AND ack.event_id = older.id)
)`;
async function assertAvailable(database: Db, companyId: string) {
if (!z.string().uuid().safeParse(companyId).success) throw badRequest("companyId must be a UUID");
const [plugin] = await database.select({ status: plugins.status }).from(plugins).where(eq(plugins.id, pluginId));
const [settings] = await database.select({ enabled: pluginCompanySettings.enabled }).from(pluginCompanySettings)
.where(and(eq(pluginCompanySettings.pluginId, pluginId), eq(pluginCompanySettings.companyId, companyId)));
if (plugin?.status !== "ready" || settings?.enabled === false) throw forbidden("Plugin is not enabled for this company");
}
return {
async list(companyId: string, limit = 50, afterId?: string): Promise<ResourceLifecycleEvent[]> {
if (!Number.isInteger(limit) || limit < 1 || limit > 100) throw badRequest("limit must be an integer from 1 to 100");
if (afterId !== undefined && (typeof afterId !== "string" || !/^[1-9]\d*$/.test(afterId) || !Number.isSafeInteger(Number(afterId)))) throw badRequest("Invalid lifecycle page id");
await assertAvailable(db, companyId);
const rows = await db.select().from(resourceLifecycleEvents)
.where(and(eq(resourceLifecycleEvents.companyId, companyId), unacknowledged, firstForResource, afterId === undefined ? undefined : gt(resourceLifecycleEvents.id, Number(afterId))))
.orderBy(asc(resourceLifecycleEvents.id)).limit(limit);
return rows.map(row => ({ ...row, id: String(row.id), createdAt: row.createdAt.toISOString() })) as ResourceLifecycleEvent[];
},
async acknowledge(companyId: string, eventId: string): Promise<void> {
if (typeof eventId !== "string" || !/^[1-9]\d*$/.test(eventId) || !Number.isSafeInteger(Number(eventId))) throw badRequest("Invalid lifecycle event id");
await db.transaction(async tx => {
const txDb = tx as unknown as Db;
await assertAvailable(txDb, companyId);
const [event] = await tx.select().from(resourceLifecycleEvents)
.where(and(eq(resourceLifecycleEvents.companyId, companyId), eq(resourceLifecycleEvents.id, Number(eventId)))).for("update");
if (!event) throw notFound("Lifecycle event not found");
const [eligible] = await tx.select({ id: resourceLifecycleEvents.id }).from(resourceLifecycleEvents)
.where(and(eq(resourceLifecycleEvents.id, event.id), firstForResource));
if (!eligible) throw conflict("Acknowledge earlier lifecycle events for this resource first");
await tx.insert(pluginLifecycleAcknowledgments).values({ pluginId, eventId: event.id }).onConflictDoNothing();
});
},
};
}