mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 22:56:58 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Plugins get agent, project and agent runtime changes from the `resource_lifecycle_events` journal. Each plugin acknowledges events one resource at a time. > - The inbox page query checked each unacknowledged event against all older events of the same resource. Each check walked the acknowledged history of that resource. > - Agent runtime streams get a new event for each configuration change, so one resource can have thousands of acknowledged events. The cost grew as pending events times acknowledged history. > - On a hosted instance, three copies of this read ran for 601 seconds each, and one earlier copy ran for 19 minutes. The delivery worker runs it again after each acknowledgment. > - This pull request selects the head of each resource in one pass and keeps the same result. > - The benefit is that the page read is linear in the company journal size. In local tests it is 250x to 660x faster, and the result rows are equal. ## Linked Issues or Issue Description **What happened?** The plugin lifecycle inbox page query (`select "id", "company_id", "resource_type", "resource_id", "action", "created_at" from "resource_lifecycle_events" where ...`) ran for 601 seconds on a hosted database. Three copies were active at the same time. **Expected behavior** A lifecycle page read must stay fast when a resource has a long acknowledged history and a pending backlog. **Steps to reproduce** 1. Give a company 40 agents with 1,500 `agent_runtime` events each. 2. Acknowledge the first 1,200 events of each agent for one plugin. 3. Call `pluginLifecycleInbox(db, pluginId).list(companyId, 100, undefined, ["agent", "project", "agent_runtime"])`. 4. The read takes about 10 seconds for 60,000 events of that company. At 200,000 events it takes about 6 minutes. **Paperclip version or commit** `02bfe71a6b` (master). **Deployment mode** Hosted (PostgreSQL). Local repro uses embedded PostgreSQL 18. ## What Changed - `server/src/services/plugin-lifecycle-inbox.ts`: `list` now selects one head per resource with `DISTINCT ON (resource_type, resource_id)`. The head is the unacknowledged creation, else the oldest unacknowledged event. Then the query applies `afterId`, id order and the limit. This is the same set that the old `firstForResource` predicate admits. `acknowledge` keeps the old predicate for its single-event check. - `plugin-lifecycle-inbox.ts`: new `hasPending(companyId, types)`. It returns whether any unacknowledged event exists. Each resource with an unacknowledged event has a head, so this gives the same answer as `list(companyId, 1).length > 0`. - `server/src/services/plugin-lifecycle-delivery.ts`: the idle-sleep journal check uses `hasPending`. This keeps its early exit. A one-row `list` page would read the full journal. - `server/src/__tests__/resource-lifecycle-events.test.ts`: new database test for heads across acknowledged history, a creation recorded after transitions, acknowledged gaps, shared ids across resource types, per-plugin progress, another company, `afterId` pages, limits, `hasPending`, and the next head after an acknowledgment. The test also passes on the old query, so it locks the contract. - `scripts/benchmark-plugin-lifecycle-inbox.ts`: new benchmark. It loads the baseline service from git, runs both on the same fixture, alternates order, asserts equal rows, and prints plans. - `doc/DATABASE.md`: describes the read cost and the benchmark command. - No schema change, index or migration. ## Verification Thesis: the old read cost is (unacknowledged candidates) x (acknowledged older events per resource), because each candidate walks the acknowledged prefix of its resource. One pass that keeps the per-resource head removes that product and returns equal rows. Baseline plan at 220,080 events: 40,020 candidates each walked about 3,999 older rows, which gave 160,039,980 acknowledgment probes and 562,140,833 buffer hits. The treatment plan reads the journal once with a hash anti join and sorts the 40,020 unacknowledged rows (3,925 buffer hits). Local SQL timings (embedded PostgreSQL 18, Apple Silicon; event counts include a 20,000-event second company, synthetic data, warm cache, one warmup, order alternates): | Fixture (events / acknowledged) | Case | Before median | After median | Samples (after) | Speedup | | --- | --- | --- | --- | --- | --- | | 40,060 / 16,040 | page of 100, all types | 3,263 ms | 13.1 ms | 3 (12.0-14.1) | 250x | | 80,080 / 48,060 | page of 100, all types | 10,137 ms | 23.9 ms | 5 (22.7-32.6) | 424x | | 220,080 / 160,060 | page of 100, all types | 355,718 ms | 538 ms | 3 (72-785) | 662x | | 1,020,080 / 960,060 | page of 100, all types | not run (est. > 30 min) | 1,682 ms | 7 (1,203-2,384) | - | | 80,080 / 48,060 | page of 100, agent and project only | 1.5 ms | 1.6 ms | 5 | neutral | | 80,080 / 48,060 | idle-sleep check | 14.5 ms | 13.8 ms | 5 | neutral | | 1,020,080 / 960,060 | idle-sleep check | not run | 406 ms | 7 | - | - Result rows are equal (`assert.deepEqual` on all mapped rows, in order) for each case where both versions ran. - The agent runtime revision read in `plugin-agent-runtime-state.ts` and `agent-file-store.ts` (`select id ... order by id desc limit 1`) uses a backward index-only scan and reads 5 buffers at 1M events. It is not changed. Commands: ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-plugin-lifecycle-inbox.ts --baseline-ref=02bfe71a6b --agents=40 --per-agent=1500 --acked=1200 --samples=5 pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-plugin-lifecycle-inbox.ts --baseline-ref=02bfe71a6b --agents=40 --per-agent=25000 --acked=24000 --samples=7 --treatment-only cd server && npx vitest run src/__tests__/resource-lifecycle-events.test.ts src/__tests__/plugin-lifecycle-delivery.test.ts src/__tests__/agent-runtime-hooks.test.ts ``` - `resource-lifecycle-events.test.ts`: 28 passed. `plugin-lifecycle-delivery.test.ts`: 40 passed (includes idle-sleep "none" and "present"). `agent-runtime-hooks.test.ts`: 22 passed. - `tsc --noEmit -p server`: pass. ## Risks - The page read is still linear in the company journal size, because acknowledgments are per plugin and the journal has no retention yet. At 1M events a page takes about 1.7 s locally. Each acknowledgment wakes another full pass. A per-plugin cursor or retention can remove this later. - The planner can choose a different plan on the hosted database. The local plans use a hash anti join and one sort. The hosted improvement is not yet measured. - `acknowledge` still checks one event against older events of its resource. That check is linear in that resource's history per call, as before. - Low behavior risk: the new test passes on both the old and new queries. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, local PostgreSQL). Extended reasoning on. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
87 lines
6.5 KiB
TypeScript
87 lines
6.5 KiB
TypeScript
import { withAccountHomeSecretMutationLock } from "@paperclipai/adapter-codex-local/server";
|
|
import { projectInstanceSecretService, type ProjectSecretSnapshot } from "./project-instance-secrets.js";
|
|
import { publishActivity, type ActivityPublication } from "./activity-log.js";
|
|
import { pluginCompanySettings, pluginLifecycleAcknowledgments, plugins, resourceLifecycleEvents, type Db } from "@paperclipai/db";
|
|
import type { ResourceLifecycleEvent } from "@paperclipai/plugin-sdk";
|
|
import { z } from "zod";
|
|
import { and, asc, eq, gt, inArray, sql } from "drizzle-orm";
|
|
import { badRequest, conflict, forbidden, notFound } from "../errors.js";
|
|
|
|
/** Durable delivery inbox. Provider work stays outside tenant transactions. */
|
|
export function pluginLifecycleInbox(db: Db, pluginId: string) {
|
|
const unacknowledged = sql`NOT EXISTS (
|
|
SELECT 1 FROM plugin_lifecycle_acknowledgments ack
|
|
WHERE ack.plugin_id = ${pluginId} AND ack.event_id = ${resourceLifecycleEvents.id}
|
|
)`;
|
|
const firstForResource = sql`NOT EXISTS (
|
|
SELECT 1 FROM resource_lifecycle_events older
|
|
WHERE older.company_id = ${resourceLifecycleEvents.companyId}
|
|
AND older.resource_type = ${resourceLifecycleEvents.resourceType}
|
|
AND older.resource_id = ${resourceLifecycleEvents.resourceId}
|
|
-- Backfilled creation must precede transitions captured before this baseline.
|
|
AND ${resourceLifecycleEvents.action} <> 'create'
|
|
AND (older.action = 'create' OR older.id < ${resourceLifecycleEvents.id})
|
|
AND NOT EXISTS (SELECT 1 FROM plugin_lifecycle_acknowledgments ack
|
|
WHERE ack.plugin_id = ${pluginId} AND ack.event_id = older.id)
|
|
)`;
|
|
|
|
async function assertAvailable(database: Db, companyId: string) {
|
|
if (!z.string().uuid().safeParse(companyId).success) throw badRequest("companyId must be a UUID");
|
|
const [plugin] = await database.select({ status: plugins.status }).from(plugins).where(eq(plugins.id, pluginId));
|
|
const [settings] = await database.select({ enabled: pluginCompanySettings.enabled }).from(pluginCompanySettings)
|
|
.where(and(eq(pluginCompanySettings.pluginId, pluginId), eq(pluginCompanySettings.companyId, companyId)));
|
|
if (plugin?.status !== "ready" || settings?.enabled === false) throw forbidden("Plugin is not enabled for this company");
|
|
}
|
|
|
|
return {
|
|
async list(companyId: string, limit = 50, afterId?: string, resourceTypes?: ResourceLifecycleEvent["resourceType"][]): Promise<ResourceLifecycleEvent[]> {
|
|
if (!Number.isInteger(limit) || limit < 1 || limit > 100) throw badRequest("limit must be an integer from 1 to 100");
|
|
if (afterId !== undefined && (typeof afterId !== "string" || !/^[1-9]\d*$/.test(afterId) || !Number.isSafeInteger(Number(afterId)))) throw badRequest("Invalid lifecycle page id");
|
|
await assertAvailable(db, companyId);
|
|
// One pass picks each resource's deliverable head: its unacknowledged creation, else its
|
|
// oldest unacknowledged event. This is the set `firstForResource` admits, without probing
|
|
// every acknowledged older event once per pending candidate. The page filter stays outside.
|
|
const heads = db.selectDistinctOn([resourceLifecycleEvents.resourceType, resourceLifecycleEvents.resourceId]).from(resourceLifecycleEvents)
|
|
.where(and(eq(resourceLifecycleEvents.companyId, companyId), inArray(resourceLifecycleEvents.resourceType, resourceTypes ?? ["agent", "project"]), unacknowledged))
|
|
.orderBy(resourceLifecycleEvents.resourceType, resourceLifecycleEvents.resourceId, sql`${resourceLifecycleEvents.action} <> 'create'`, resourceLifecycleEvents.id)
|
|
.as("heads");
|
|
const rows = await db.select().from(heads)
|
|
.where(afterId === undefined ? undefined : gt(heads.id, Number(afterId)))
|
|
.orderBy(asc(heads.id)).limit(limit);
|
|
return rows.map(row => ({ ...row, id: String(row.id), createdAt: row.createdAt.toISOString() })) as ResourceLifecycleEvent[];
|
|
},
|
|
/** Whether `list` would return any event. Every resource with an unacknowledged event has a head. */
|
|
async hasPending(companyId: string, resourceTypes?: ResourceLifecycleEvent["resourceType"][]): Promise<boolean> {
|
|
await assertAvailable(db, companyId);
|
|
const [row] = await db.select({ id: resourceLifecycleEvents.id }).from(resourceLifecycleEvents)
|
|
.where(and(eq(resourceLifecycleEvents.companyId, companyId), inArray(resourceLifecycleEvents.resourceType, resourceTypes ?? ["agent", "project"]), unacknowledged))
|
|
.orderBy(asc(resourceLifecycleEvents.id)).limit(1);
|
|
return row !== undefined;
|
|
},
|
|
async acknowledge(companyId: string, eventId: string, credentials?: { snapshot: ProjectSecretSnapshot; changes: unknown }): Promise<void> {
|
|
if (typeof eventId !== "string" || !/^[1-9]\d*$/.test(eventId) || !Number.isSafeInteger(Number(eventId))) throw badRequest("Invalid lifecycle event id");
|
|
const commit = () => db.transaction(async tx => {
|
|
const publications: ActivityPublication[] = [];
|
|
const txDb = tx as unknown as Db;
|
|
await assertAvailable(txDb, companyId);
|
|
const [event] = await tx.select().from(resourceLifecycleEvents)
|
|
.where(and(eq(resourceLifecycleEvents.companyId, companyId), eq(resourceLifecycleEvents.id, Number(eventId)))).for("update");
|
|
if (!event) throw notFound("Lifecycle event not found");
|
|
const [acked] = await tx.select().from(pluginLifecycleAcknowledgments).where(and(eq(pluginLifecycleAcknowledgments.pluginId, pluginId), eq(pluginLifecycleAcknowledgments.eventId, event.id)));
|
|
if (acked) return publications;
|
|
const [eligible] = await tx.select({ id: resourceLifecycleEvents.id }).from(resourceLifecycleEvents)
|
|
.where(and(eq(resourceLifecycleEvents.id, event.id), firstForResource));
|
|
if (!eligible) throw conflict("Acknowledge earlier lifecycle events for this resource first");
|
|
if (credentials) {
|
|
if (event.resourceType !== "project") throw forbidden("Project instance secret event required");
|
|
publications.push(...await projectInstanceSecretService(txDb, pluginId).apply({ ...event, id: String(event.id), createdAt: event.createdAt.toISOString() } as ResourceLifecycleEvent, credentials.snapshot, credentials.changes));
|
|
}
|
|
await tx.insert(pluginLifecycleAcknowledgments).values({ pluginId, eventId: event.id }).onConflictDoNothing();
|
|
return publications;
|
|
});
|
|
const publications = credentials ? await withAccountHomeSecretMutationLock(undefined, companyId, commit) : await commit();
|
|
for (const publication of publications) publishActivity(publication);
|
|
},
|
|
};
|
|
}
|