Files
PaperClipAI/server/src/__tests__/resource-lifecycle-events.test.ts
T
DottaandClaude Opus 5.5 3bf7af6c77 perf(plugins): read the lifecycle inbox from per-resource cursors (#20576)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Plugins get agent, project and agent runtime changes from the
`resource_lifecycle_events` journal. Each plugin acknowledges events one
resource at a time.
> - #20405 made the inbox page read linear in the company journal size.
The journal has no retention, so each page still reads every event of
the company to find the unacknowledged ones: about 1.7 s per 1M events.
> - The delivery worker reads a page again after each acknowledgment.
The idle-sleep check also reads the whole journal when all events are
acknowledged. `acknowledge` checks every older event of the resource.
> - On a hosted instance, the page read ran 181 times in one window at
about 1 s each.
> - This pull request adds a per-plugin, per-resource cursor. Reads
start at the cursor, so they do not read acknowledged history again.
> - The benefit is that page, idle-check and acknowledgment cost follow
the number of resources and pending events, not the journal size.
Delivery order and results stay the same.

## Linked Issues or Issue Description

Refs #20405

**What happened?**
The plugin lifecycle inbox page query (`select ... from
"resource_lifecycle_events" ... DISTINCT ON ...`) ran 181 times at about
1 s each on a hosted database in one window. Each acknowledgment causes
another full read of the company journal. The idle-sleep check
(`hasPending`) reads the full journal when all events are acknowledged.
`acknowledge` reads the full history of the resource.

**Expected behavior**
A page read, the idle-sleep check and an acknowledgment must not read
acknowledged history again. Their cost must not grow with the journal
size.

**Steps to reproduce**
1. Give a company 400 agents with 2,500 `agent_runtime` events each (1M
events).
2. Acknowledge the first 2,000 events of each agent for one plugin.
3. Call `pluginLifecycleInbox(db, pluginId).list(companyId, 100,
undefined, ["agent", "project", "agent_runtime"])`.
4. The read takes about 1.7 s. `hasPending` takes about 240 ms.

**Paperclip version or commit**
`8d2bf78323` (master).

**Deployment mode**
Hosted (PostgreSQL). Local repro uses embedded PostgreSQL 18.

## What Changed

- `packages/db/src/schema/plugin_lifecycle_resource_cursors.ts`: new
table `plugin_lifecycle_resource_cursors`. It has one row per plugin,
company, resource type and resource id. `acknowledged_through_id` means:
every event of that resource at or below this id is acknowledged by the
plugin. Plugin and company deletion cascade.
- Migration `0462_plugin_lifecycle_resource_cursors`: creates the table
and carries existing acknowledgments forward (see Risks for the
previous-release story).
- `server/src/services/plugin-lifecycle-inbox.ts`:
- `list` and `hasPending` use a backward loose index scan on
`resource_lifecycle_events_resource_order_idx`. It visits each resource
stream once and gets its latest event id. A stream whose latest event is
at or below its cursor is settled and is not read again. For other
streams, the read gets one head: the unacknowledged creation, else the
first unacknowledged transition above the cursor. Then it applies
`afterId`, id order and the limit, as before.
- `acknowledge` reads only the events between the cursor and the
acknowledged event. It still writes the per-event
`plugin_lifecycle_acknowledgments` row. It moves the cursor only over a
contiguous acknowledged prefix. A creation recorded after a pending
transition does not move the cursor.
- An event counts as acknowledged when it is at or below the cursor, or
when it has an acknowledgment row. Thus rows written by an earlier
release, and gaps in old history, keep their current behavior.
- `server/src/__tests__/resource-lifecycle-events.test.ts`: new database
tests for the migration carry-forward, gaps, late creations, cursor
reads, the rollback rows and the plugin-delete cascade. All tests from
#20405 still pass without change.
- `scripts/benchmark-plugin-lifecycle-inbox.ts`: runs the migration
backfill on the fixture and times it. Adds an acknowledgment case. Runs
`VACUUM ANALYZE`, because hosted tables are autovacuumed.
- `doc/DATABASE.md`: describes the cursor, its read cost and the
migration.
- The delivery worker (`plugin-lifecycle-delivery.ts`) is not changed.

## Verification

Thesis: the cost is a read of the whole company journal (page, idle
check) or the whole resource history (acknowledgment). If a per-resource
cursor marks the acknowledged prefix, each read can skip that prefix.
Then the cost follows the number of resource streams and pending events.
The rows must stay equal.

Local SQL timings: embedded PostgreSQL 18.1 on Apple Silicon, synthetic
data, warm cache, one warmup, 7 samples, baseline and treatment order
alternate per sample. The machine was shared with other jobs (load
average about 35), so spread is wide. Values are median (min to max) in
ms. Rows are equal (`assert.deepEqual`) for every list and probe case.

Fixture A: 400 agents x 2,500 runtime events, 2,000 acknowledged per
agent. 1,020,440 events, 800,420 acknowledgments.

| Case | Before | After | Speedup |
| --- | --- | --- | --- |
| page of 100, all types | 1,700 (1,633-2,438) | 11.5 (9.5-12.2) | 148x
|
| page of 100, agent and project | 3.3 (2.5-5.4) | 4.2 (3.6-6.7) | 0.8x
(slower) |
| idle-sleep check | 238 (174-367) | 2.7 (2.1-3.4) | 88x |
| acknowledge one head | 15.3 (9.4-30.1) | 20.3 (12.9-29.9) | neutral
(noise) |
| migration backfill | - | 9.3 s | - |

Fixture B: 40 agents x 25,000 runtime events, 24,000 acknowledged per
agent (long history). 1,020,080 events.

| Case | Before | After | Speedup |
| --- | --- | --- | --- |
| page of 100, all types | 1,943 (1,642-4,979) | 4.4 (2.9-5.6) | 444x |
| page of 100, agent and project | 3.5 (2.1-5.7) | 4.6 (2.8-12.0) | 0.8x
(slower) |
| idle-sleep check | 234 (220-243) | 2.8 (1.9-6.5) | 85x |
| acknowledge one head | 64.6 (48.5-71.6) | 5.0 (2.6-7.1) | 13x |
| migration backfill | - | 4.6 s | - |

Fixture C: 5,000 agents x 200 runtime events, 190 acknowledged per agent
(many resources). 1,025,040 events.

| Case | Before | After | Speedup |
| --- | --- | --- | --- |
| page of 100, all types | 1,867 (1,649-1,975) | 76.9 (66.2-497) | 24x |
| page of 100, agent and project | 10.3 (8.7-12.4) | 21.6 (20.2-23.2) |
0.5x (slower) |
| idle-sleep check | 220 (201-298) | 2.8 (2.0-5.3) | 78x |
| acknowledge one head | 4.9 (3.4-8.6) | 6.0 (5.2-7.6) | neutral |
| migration backfill | - | 15.6 s | - |

`EXPLAIN (ANALYZE, BUFFERS)` for the all-types page on fixture A: before
2,488 ms, 13,597 shared hits plus temp files; after 7.0 ms, 7,926 shared
hits.

Commands:

```sh
pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-plugin-lifecycle-inbox.ts --baseline-ref=8d2bf78323 --agents=400 --per-agent=2500 --acked=2000 --samples=7
pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-plugin-lifecycle-inbox.ts --baseline-ref=8d2bf78323 --agents=40 --per-agent=25000 --acked=24000 --samples=7
pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-plugin-lifecycle-inbox.ts --baseline-ref=8d2bf78323 --agents=5000 --per-agent=200 --acked=190 --samples=7
cd server && npx vitest run src/__tests__/resource-lifecycle-events.test.ts src/__tests__/plugin-lifecycle-delivery.test.ts src/__tests__/project-instance-secrets.test.ts src/__tests__/agent-runtime-hooks.test.ts
pnpm --filter @paperclipai/db check:migrations
```

- `resource-lifecycle-events.test.ts`,
`plugin-lifecycle-delivery.test.ts` and
`project-instance-secrets.test.ts`: 86 passed.
`agent-runtime-hooks.test.ts`: 22 passed.
- `tsc --noEmit -p server`: pass. `check:migrations`: pass.

## Risks

- **Schema change.** Migration `0462` adds one table and two foreign
keys to `plugins` and `companies`. It adds no index on an existing
table. The backfill reads the acknowledgment table once. It took 4.6 s
to 15.6 s on 1M-event fixtures with 0.8M to 1M acknowledgments.
- **Migration order.** The number `0462` is reserved after open PRs that
use `0459` to `0461` (#20571, #20568, #20573). Merge this PR after them.
Before merge, rebase onto master and chain the snapshot after the latest
migration.
- **Previous-release story.**
- Upgrade: the backfill stores, per plugin and resource, the end of the
contiguous acknowledged prefix. A cursor never passes an unacknowledged
event, so no event is lost. Acknowledged events above a gap keep their
rows, so no event is delivered again. A test runs the migration
statement on rows that only an earlier release wrote and checks that
`list` returns the same rows before and after.
- Rolling deploy: an earlier release that still runs writes only
acknowledgment rows. The new reads honor those rows.
- Rollback: the new code still writes every acknowledgment row, so an
earlier release sees the same progress.
- **Ordering assumption.** The cursor is safe because writers record a
resource's events while they hold that resource's row lock
(`paperclip_record_agent_runtime` locks the agent row; agent status and
project writers document the lock). Thus a later commit for the same
resource always has a higher id. A writer that breaks this contract
could commit an event below the cursor, and that event would not be
delivered. A lower id from a different resource is still delivered
(existing test).
- **Slower path.** A page that asks only for `agent` and `project`
streams is 1 to 11 ms slower. These streams are short, so the old full
read was cheap, and the new read visits each stream once. The cost is
bounded by the number of resources.
- **Many resources.** The page cost grows with the number of resource
streams of the requested types. At 5,000 agents a page is about 77 ms,
compared with about 1.9 s before.
- The hosted planner can choose a different plan. The hosted improvement
is not yet measured.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, local PostgreSQL benchmarks).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 12:58:32 -06:00

743 lines
51 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { createAgentLifecycle, configureAgentLifecycle } from "../services/agent-lifecycle.js";
import { randomUUID } from "node:crypto";
import { readFileSync } from "node:fs";
import { and, eq, inArray, ne, sql } from "drizzle-orm";
import { agentApiKeys, agents, budgetPolicies, companies, costEvents, createDb, pluginCompanySettings, pluginLifecycleAcknowledgments, pluginLifecycleResourceCursors, plugins, projects, resourceLifecycleEvents, type Db } from "@paperclipai/db";
import { createHostClientHandlers } from "@paperclipai/plugin-sdk";
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
import { agentService } from "../services/agents.js";
import { approvalService } from "../services/approvals.js";
import { budgetService, flushBudgetEvaluationWork } from "../services/budgets.js";
import { projectService } from "../services/projects.js";
import { recordResourceCreationEvent } from "../services/resource-lifecycle-events.js";
import { pluginLifecycleInbox } from "../services/plugin-lifecycle-inbox.js";
import { buildHostServices } from "../services/plugin-host-services.js";
import { createPluginEventBus } from "../services/plugin-event-bus.js";
const support = await getEmbeddedPostgresTestSupport();
const describePostgres = support.supported ? describe : describe.skip;
if (!support.supported) console.warn(`Skipping lifecycle event database tests: ${support.reason}`);
describePostgres("Resource lifecycle events", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: Db;
let companyId: string;
let worker: ReturnType<typeof configureAgentLifecycle>;
beforeAll(async () => {
database = await startEmbeddedPostgresTestDatabase("paperclip-resource-events-");
db = createDb(database.connectionString);
}, 90_000);
afterAll(async () => { await worker?.stop(); await database?.cleanup(); });
beforeEach(async () => {
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", undefined);
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", undefined);
await worker?.stop();
worker = configureAgentLifecycle(db, { requiredPluginIds: async () => [], runPlugin: async () => "complete", runHost: async () => "complete" });
companyId = randomUUID();
await db.insert(companies).values({ id: companyId, name: "Lifecycle fixture", issuePrefix: `L${companyId.replaceAll("-", "").slice(0, 6)}` });
});
afterEach(() => { vi.unstubAllEnvs(); });
const events = () => db.select().from(resourceLifecycleEvents).where(and(eq(resourceLifecycleEvents.companyId, companyId), ne(resourceLifecycleEvents.resourceType, "agent_runtime")));
const createAgent = async (status: "idle" | "pending_approval" | "terminated" = "idle", database: Db = db) => {
const agent = await createAgentLifecycle(database).requestHire(companyId, { name: "Lifecycle agent", adapterType: "process", adapterConfig: {}, status });
await worker.process(agent.id);
return (await agentService(db).getById(agent.id))!;
};
const createPlugin = async () => {
const [plugin] = await db.insert(plugins).values({ pluginKey: randomUUID(), packageName: "lifecycle-fixture", version: "1.0.0", status: "ready", manifestJson: {} as never }).returning();
return plugin;
};
it("records direct and approval-free hires once, including concurrent duplicate submissions", async () => {
const agent = await createAgent();
await Promise.all(Array.from({ length: 5 }, () => db.transaction(tx => recordResourceCreationEvent(tx as unknown as Db, companyId, "agent", agent.id))));
expect(await events()).toEqual([expect.objectContaining({ resourceType: "agent", resourceId: agent.id, action: "create" })]);
});
it("records pending hires only after approval and ignores repeated approval", async () => {
const agent = await createAgent("pending_approval");
expect(await events()).toEqual([]);
const approval = await approvalService(db).create(companyId, { type: "hire_agent", status: "pending", payload: { agentId: agent.id } });
await approvalService(db).approve(approval.id, "fixture-board");
await approvalService(db).approve(approval.id, "fixture-board");
await createAgentLifecycle(db).approveHire(agent.id);
expect(await events()).toEqual([expect.objectContaining({ resourceType: "agent", resourceId: agent.id })]);
});
it("records legacy approvals that create the agent at approval time", async () => {
const approval = await approvalService(db).create(companyId, { type: "hire_agent", status: "pending", payload: { name: "Legacy hire", adapterType: "process" } });
await approvalService(db).approve(approval.id, "fixture-board");
const [intent] = await events();
if (intent) await worker.process(intent.resourceId);
expect(intent?.resourceType).toBe("agent");
expect(await agentService(db).getById(intent!.resourceId)).toMatchObject({ status: "idle", companyId });
});
it("does not allocate for rejected or terminated hires", async () => {
const agent = await createAgent("pending_approval");
const approval = await approvalService(db).create(companyId, { type: "hire_agent", status: "pending", payload: { agentId: agent.id } });
await approvalService(db).reject(approval.id, "fixture-board");
await createAgent("terminated");
expect(await events()).toEqual([expect.objectContaining({ action: "terminate", resourceId: agent.id })]);
});
it("rolls back hire rejection if termination cannot record its hook, allowing a retry", async () => {
const agent = await createAgent("pending_approval");
const service = approvalService(db);
const approval = await service.create(companyId, { type: "hire_agent", status: "pending", payload: { agentId: agent.id } });
await db.execute(sql`ALTER TABLE resource_lifecycle_events ADD CONSTRAINT fixture_reject_hook CHECK (false) NOT VALID`);
try {
await expect(service.reject(approval.id, "fixture-board")).rejects.toThrow();
expect(await service.getById(approval.id)).toMatchObject({ status: "pending" });
expect(await agentService(db).getById(agent.id)).toMatchObject({ status: "pending_approval" });
expect(await events()).toEqual([]);
} finally {
await db.execute(sql`ALTER TABLE resource_lifecycle_events DROP CONSTRAINT fixture_reject_hook`);
}
expect(await service.reject(approval.id, "fixture-board")).toMatchObject({ applied: true, approval: { status: "rejected" } });
expect(await agentService(db).getById(agent.id)).toMatchObject({ status: "terminated" });
expect(await service.reject(approval.id, "fixture-board")).toMatchObject({ applied: false });
expect(await events()).toEqual([expect.objectContaining({ action: "terminate", resourceId: agent.id })]);
});
it("records projects with zero or multiple repository workspaces without storing repository data", async () => {
const empty = await projectService(db).create(companyId, { name: "Empty project" });
const project = await projectService(db).createWithRepositories(companyId, { name: "Repository project" }, [
{ id: "1", fullName: "fixture/one", url: "https://github.com/fixture/one", connections: [] },
{ id: "2", fullName: "fixture/two", url: "https://github.com/fixture/two", connections: [] },
]);
expect(project.workspaces).toHaveLength(2);
const rows = await events();
expect(rows.map(row => row.resourceId).sort()).toEqual([empty.id, project.id].sort());
expect(rows.every(row => row.resourceType === "project")).toBe(true);
expect(Object.keys(rows[0]).sort()).toEqual(["action", "companyId", "createdAt", "id", "resourceId", "resourceType"]);
});
it("records every pause/resume cycle and termination in resource order, without duplicate hooks", async () => {
const agent = await createAgent();
const service = agentService(db);
const serviceLifecycle = createAgentLifecycle(db);
await db.insert(agentApiKeys).values({ agentId: agent.id, companyId, name: "Lifecycle key", keyHash: "fixture-hash" });
await Promise.all(Array.from({ length: 4 }, () => serviceLifecycle.pauseAgent(agent.id)));
await worker.process(agent.id);
await Promise.all(Array.from({ length: 4 }, () => serviceLifecycle.resumeAgent(agent.id)));
await worker.process(agent.id);
await serviceLifecycle.pauseAgent(agent.id);
await worker.process(agent.id);
await worker.process(agent.id);
await serviceLifecycle.resumeAgent(agent.id);
await worker.process(agent.id);
await worker.process(agent.id);
await Promise.all(Array.from({ length: 4 }, () => serviceLifecycle.terminateAgent(agent.id)));
const rows = (await events()).sort((a, b) => a.id - b.id);
expect(rows.map(row => row.action)).toEqual(["create", "pause", "resume", "pause", "resume", "terminate"]);
expect(rows.every(row => row.resourceId === agent.id)).toBe(true);
const [key] = await db.select().from(agentApiKeys).where(eq(agentApiKeys.agentId, agent.id));
expect(key.revokedAt).not.toBeNull();
await expect(serviceLifecycle.pauseAgent(agent.id)).rejects.toMatchObject({ name: "AgentLifecycleConflict" });
await expect(serviceLifecycle.resumeAgent(agent.id)).rejects.toMatchObject({ name: "AgentLifecycleConflict" });
});
it("does not bypass pending hire approval with pause or resume", async () => {
const agent = await createAgent("pending_approval");
await expect(createAgentLifecycle(db).pauseAgent(agent.id)).rejects.toMatchObject({ name: "AgentLifecycleConflict" });
await expect(createAgentLifecycle(db).resumeAgent(agent.id)).rejects.toMatchObject({ name: "AgentLifecycleConflict" });
expect(await events()).toEqual([]);
});
it("records budget pause and resume hooks without replaying repeated budget evaluation", async () => {
const agent = await createAgent();
const service = budgetService(db);
await db.insert(budgetPolicies).values({ companyId, scopeType: "agent", scopeId: agent.id, metric: "billed_cents", windowKind: "calendar_month_utc", amount: 100, notifyEnabled: false });
const [event] = await db.insert(costEvents).values({ companyId, agentId: agent.id, provider: "fixture", model: "fixture", costCents: 150, occurredAt: new Date() }).returning();
await service.evaluateCostEvent(event);
await flushBudgetEvaluationWork(db, { companyId });
await service.evaluateCostEvent(event);
await flushBudgetEvaluationWork(db, { companyId });
expect(await agentService(db).getById(agent.id)).toMatchObject({ status: "paused", pauseReason: "budget" });
await service.upsertPolicy(companyId, { scopeType: "agent", scopeId: agent.id, amount: 200 }, "fixture-board");
await flushBudgetEvaluationWork(db, { companyId });
await worker.process(agent.id);
expect(await agentService(db).getById(agent.id)).toMatchObject({ status: "idle", pauseReason: null });
expect((await events()).sort((a, b) => a.id - b.id).map(row => row.action)).toEqual(["create", "pause", "resume"]);
});
it("rolls back pause, resume, termination, and key revocation if a hook write fails", async () => {
const agent = await createAgent();
const paused = await createAgent();
await createAgentLifecycle(db).pauseAgent(paused.id);
await worker.process(paused.id);
await db.insert(agentApiKeys).values({ agentId: agent.id, companyId, name: "Retained key", keyHash: "fixture-retained-hash" });
const before = await events();
await db.execute(sql`ALTER TABLE resource_lifecycle_events ADD CONSTRAINT fixture_reject_hook CHECK (false) NOT VALID`);
try {
await expect(createAgentLifecycle(db).pauseAgent(agent.id)).rejects.toThrow();
await expect(createAgentLifecycle(db).resumeAgent(paused.id)).rejects.toThrow();
await expect(createAgentLifecycle(db).terminateAgent(agent.id)).rejects.toThrow();
expect(await agentService(db).getById(agent.id)).toMatchObject({ status: "idle" });
expect(await agentService(db).getById(paused.id)).toMatchObject({ status: "paused" });
const [key] = await db.select().from(agentApiKeys).where(eq(agentApiKeys.agentId, agent.id));
expect(key.revokedAt).toBeNull();
expect(await events()).toEqual(before);
} finally {
await db.execute(sql`ALTER TABLE resource_lifecycle_events DROP CONSTRAINT fixture_reject_hook`);
}
});
it("rolls back agent and project events with the outer creation transaction", async () => {
const agentId = randomUUID();
const projectId = randomUUID();
await expect(db.transaction(async tx => {
const txDb = tx as unknown as Db;
await tx.insert(agents).values({ id: agentId, companyId, name: "Rolled back agent" });
await recordResourceCreationEvent(txDb, companyId, "agent", agentId);
await projectService(txDb).createWithRepositories(companyId, { id: projectId, name: "Rolled back project" }, []);
throw new Error("rollback fixture");
})).rejects.toThrow("rollback fixture");
expect(await events()).toEqual([]);
expect(await agentService(db).getById(agentId)).toBeNull();
expect(await projectService(db).getById(projectId)).toBeNull();
});
it("fails creation and activation atomically if the intent cannot be persisted", async () => {
const pending = await createAgent("pending_approval");
const approval = await approvalService(db).create(companyId, { type: "hire_agent", status: "pending", payload: { agentId: pending.id } });
await db.execute(sql`ALTER TABLE resource_lifecycle_events ADD CONSTRAINT fixture_reject_intent CHECK (false) NOT VALID`);
try {
await expect(createAgent()).rejects.toThrow();
await expect(projectService(db).create(companyId, { name: "Rejected project" })).rejects.toThrow();
await expect(approvalService(db).approve(approval.id, "fixture-board")).rejects.toThrow();
expect(await approvalService(db).getById(approval.id)).toMatchObject({ status: "pending" });
expect(await db.select().from(agents).where(eq(agents.companyId, companyId))).toEqual([expect.objectContaining({ id: pending.id, status: "pending_approval" })]);
expect(await db.select().from(projects).where(eq(projects.companyId, companyId))).toEqual([]);
expect(await events()).toEqual([]);
} finally {
await db.execute(sql`ALTER TABLE resource_lifecycle_events DROP CONSTRAINT fixture_reject_intent`);
}
});
it("captures self-hosted lifecycle events without backfilling older resources", async () => {
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", undefined);
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", undefined);
const oldAgentId = randomUUID();
const oldProjectId = randomUUID();
await db.insert(agents).values({ id: oldAgentId, companyId, name: "Existing agent" });
await db.insert(projects).values({ id: oldProjectId, companyId, name: "Existing project" });
await agentService(db).update(oldAgentId, { name: "Renamed agent" });
await projectService(db).update(oldProjectId, { name: "Renamed project" });
expect(await events()).toEqual([expect.objectContaining({ action: "update", resourceId: oldProjectId })]);
const agent = await createAgent();
const project = await projectService(db).create(companyId, { name: "New project" });
expect((await events()).filter(row => row.action === "create").map(row => row.resourceId).sort()).toEqual([agent.id, project.id].sort());
});
it("scopes resource identities by company and type", async () => {
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", undefined);
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", "fixture-token");
const id = randomUUID();
await db.transaction(tx => recordResourceCreationEvent(tx as unknown as Db, companyId, "agent", id));
await db.transaction(tx => recordResourceCreationEvent(tx as unknown as Db, companyId, "project", id));
const otherCompanyId = randomUUID();
await db.insert(companies).values({ id: otherCompanyId, name: "Other company", issuePrefix: "OTHER" });
await db.transaction(tx => recordResourceCreationEvent(tx as unknown as Db, otherCompanyId, "agent", id));
expect(await events()).toHaveLength(2);
expect(await db.select().from(resourceLifecycleEvents).where(eq(resourceLifecycleEvents.companyId, otherCompanyId))).toHaveLength(1);
await db.delete(companies).where(eq(companies.id, otherCompanyId));
expect(await db.select().from(resourceLifecycleEvents).where(eq(resourceLifecycleEvents.companyId, otherCompanyId))).toEqual([]);
});
it("captures project and workspace updates and batches repository replacement", async () => {
const service = projectService(db);
const project = await service.createWithRepositories(companyId, { name: "Mutable project" }, [
{ id: "1", fullName: "fixture/one", url: "https://github.com/fixture/one", connections: [] },
]);
expect((await events()).map(row => row.action)).toEqual(["create"]);
await service.update(project.id, { name: "Updated project" });
const workspace = await service.createWorkspace(project.id, { repoUrl: "https://github.com/fixture/two" });
await service.updateWorkspace(project.id, workspace!.id, { repoRef: "main" });
await service.removeWorkspace(project.id, workspace!.id);
const beforeReplace = (await events()).length;
await service.replaceRepositories(project.id, [
{ id: "3", fullName: "fixture/three", url: "https://github.com/fixture/three", connections: [] },
{ id: "4", fullName: "fixture/four", url: "https://github.com/fixture/four", connections: [] },
]);
expect(await events()).toHaveLength(beforeReplace + 1);
const beforeArchive = await events();
expect((await service.getById(project.id))?.workspaces).toHaveLength(2);
expect(beforeArchive.sort((a, b) => a.id - b.id).map(row => row.action)).toEqual(["create", "update", "update", "update", "update", "update"]);
});
it("records deletion atomically and retains the event after removing the project", async () => {
const service = projectService(db);
const project = await service.create(companyId, { name: "Delete fixture" });
await db.execute(sql`ALTER TABLE resource_lifecycle_events ADD CONSTRAINT fixture_reject_delete CHECK (action <> 'delete') NOT VALID`);
try {
await expect(service.remove(project.id)).rejects.toThrow();
expect(await service.getById(project.id)).not.toBeNull();
expect((await events()).map(row => row.action)).toEqual(["create"]);
} finally {
await db.execute(sql`ALTER TABLE resource_lifecycle_events DROP CONSTRAINT fixture_reject_delete`);
}
await service.remove(project.id);
await service.remove(project.id);
expect(await service.getById(project.id)).toBeNull();
expect((await events()).map(row => row.action)).toEqual(["create", "delete"]);
});
it("records archive once per transition and update on restore, retaining repositories", async () => {
const service = projectService(db);
const project = await service.createWithRepositories(companyId, { name: "Archive fixture" }, [
{ id: "1", fullName: "fixture/one", url: "https://github.com/fixture/one", connections: [] },
]);
await Promise.all([
service.update(project.id, { archivedAt: new Date(), name: undefined }),
service.update(project.id, { archivedAt: new Date() }),
]);
expect((await events()).map(row => row.action)).toEqual(["create", "archive"]);
expect((await service.getById(project.id))?.workspaces).toHaveLength(1);
await service.update(project.id, { archivedAt: null });
await service.update(project.id, { archivedAt: null });
await service.update(project.id, { archivedAt: new Date(), name: "Edited and archived" });
expect((await events()).sort((a, b) => a.id - b.id).map(row => row.action))
.toEqual(["create", "archive", "update", "update", "archive"]);
});
it("rolls back archive and combined edits when the archive record fails", async () => {
const service = projectService(db);
const project = await service.create(companyId, { name: "Retained project" });
const before = await events();
await db.execute(sql`ALTER TABLE resource_lifecycle_events ADD CONSTRAINT fixture_reject_archive CHECK (action <> 'archive') NOT VALID`);
try {
await expect(service.update(project.id, { archivedAt: new Date(), name: "Rejected edit" })).rejects.toThrow();
expect(await service.getById(project.id)).toMatchObject({ name: "Retained project", archivedAt: null });
expect(await events()).toEqual(before);
} finally {
await db.execute(sql`ALTER TABLE resource_lifecycle_events DROP CONSTRAINT fixture_reject_archive`);
}
});
it("rolls back project and repository mutations when their update record fails", async () => {
const service = projectService(db);
const project = await service.create(companyId, { name: "Retained project" });
const workspace = await service.createWorkspace(project.id, { repoUrl: "https://github.com/fixture/retained" });
const before = await events();
await db.execute(sql`ALTER TABLE resource_lifecycle_events ADD CONSTRAINT fixture_reject_update CHECK (action <> 'update') NOT VALID`);
try {
await expect(service.update(project.id, { name: "Rejected update" })).rejects.toThrow();
await expect(service.createWorkspace(project.id, { repoUrl: "https://github.com/fixture/rejected" })).rejects.toThrow();
await expect(service.updateWorkspace(project.id, workspace!.id, { repoRef: "rejected" })).rejects.toThrow();
await expect(service.removeWorkspace(project.id, workspace!.id)).rejects.toThrow();
await expect(service.replaceRepositories(project.id, [])).rejects.toThrow();
expect(await service.getById(project.id)).toMatchObject({ name: "Retained project", workspaces: [expect.objectContaining({ id: workspace!.id, repoRef: null })] });
expect(await events()).toEqual(before);
} finally {
await db.execute(sql`ALTER TABLE resource_lifecycle_events DROP CONSTRAINT fixture_reject_update`);
}
});
const backfillBaseline = () => db.transaction(async tx => {
const migration = readFileSync(new URL("../../../packages/db/src/migrations/0309_loving_the_hood.sql", import.meta.url), "utf8");
const lock = migration.split("--> statement-breakpoint")[0];
const baseline = migration.slice(migration.indexOf("-- Seed a one-time current-state baseline"));
for (const statement of [lock, ...baseline.split("--> statement-breakpoint")]) {
if (statement.trim()) await tx.execute(sql.raw(statement));
}
});
it("backfills a current-state baseline once without bypassing approval or reordering partial histories", async () => {
const [live, paused, terminated, pending, partial, resumed] = await db.insert(agents).values([
{ companyId, name: "Legacy live", status: "idle" },
{ companyId, name: "Legacy paused", status: "paused" },
{ companyId, name: "Legacy terminated", status: "terminated" },
{ companyId, name: "Legacy pending", status: "pending_approval" },
{ companyId, name: "Partial pause", status: "paused" },
{ companyId, name: "Partial resume", status: "idle" },
]).returning();
const [archived] = await db.insert(projects).values({ companyId, name: "Archived baseline", archivedAt: new Date(0) }).returning();
const [restored] = await db.insert(projects).values({ companyId, name: "Restored baseline" }).returning();
await db.insert(resourceLifecycleEvents).values({ companyId, resourceType: "project", resourceId: restored.id, action: "archive" });
const existing = await createAgent();
const [earlyPause] = await db.insert(resourceLifecycleEvents).values([
{ companyId, resourceType: "agent", resourceId: partial.id, action: "pause" },
{ companyId, resourceType: "agent", resourceId: resumed.id, action: "pause" },
]).returning();
const deletedId = randomUUID();
const [deleted] = await db.insert(resourceLifecycleEvents).values({ companyId, resourceType: "agent", resourceId: deletedId, action: "terminate" }).returning();
await backfillBaseline();
const baseline = (await events()).sort((a, b) => a.id - b.id);
const actions = (id: string) => baseline.filter(e => e.resourceId === id).map(e => e.action);
expect(actions(live.id)).toEqual(["create"]);
expect(actions(paused.id)).toEqual(["create", "pause"]);
expect(actions(terminated.id)).toEqual(["terminate"]);
expect(actions(pending.id)).toEqual([]);
expect(actions(partial.id)).toEqual(["pause", "create"]);
expect(actions(resumed.id)).toEqual(["pause", "create", "resume"]);
expect(actions(archived.id)).toEqual(["create", "archive"]);
expect(actions(restored.id)).toEqual(["archive", "create", "update"]);
expect(actions(existing.id)).toEqual(["create"]);
expect(baseline.find(e => e.id === deleted.id)).toEqual(deleted);
const plugin = await createPlugin();
const inbox = pluginLifecycleInbox(db, plugin.id);
const creation = (await inbox.list(companyId)).find(e => e.resourceId === partial.id)!;
expect(creation.action).toBe("create");
expect(Number(creation.id)).toBeGreaterThan(earlyPause.id);
await expect(inbox.acknowledge(companyId, String(earlyPause.id))).rejects.toMatchObject({ status: 409 });
await inbox.acknowledge(companyId, creation.id);
expect((await inbox.list(companyId)).find(e => e.resourceId === partial.id)).toMatchObject({ id: String(earlyPause.id), action: "pause" });
const projectCreate = (await inbox.list(companyId)).find(e => e.resourceId === archived.id)!;
await inbox.acknowledge(companyId, projectCreate.id);
const archive = (await inbox.list(companyId)).find(e => e.resourceId === archived.id)!;
expect(archive.action).toBe("archive");
await inbox.acknowledge(companyId, archive.id);
expect((await inbox.list(companyId)).find(e => e.resourceId === archived.id)).toBeUndefined();
await backfillBaseline();
expect((await events()).sort((a, b) => a.id - b.id)).toEqual(baseline);
});
it("keeps delivery unavailable when the baseline migration fails", async () => {
await db.insert(agents).values({ companyId, name: "Legacy paused", status: "paused" });
await db.execute(sql`ALTER TABLE resource_lifecycle_events ALTER COLUMN id DROP IDENTITY`);
await db.execute(sql`DROP TABLE plugin_lifecycle_acknowledgments`);
await db.execute(sql`DROP INDEX resource_lifecycle_events_resource_order_idx`);
const applyDeliveryMigration = () => db.transaction(async tx => {
const migration = readFileSync(new URL("../../../packages/db/src/migrations/0309_loving_the_hood.sql", import.meta.url), "utf8");
for (const statement of migration.split("--> statement-breakpoint")) {
if (statement.trim()) await tx.execute(sql.raw(statement));
}
});
await db.execute(sql`ALTER TABLE resource_lifecycle_events ADD CONSTRAINT fixture_reject_baseline CHECK (action <> 'pause') NOT VALID`);
try {
await expect(applyDeliveryMigration()).rejects.toMatchObject({ cause: { constraint_name: "fixture_reject_baseline" } });
expect(await events()).toEqual([]);
expect((await db.execute(sql`SELECT is_identity FROM information_schema.columns WHERE table_schema = 'public' AND table_name = 'resource_lifecycle_events' AND column_name = 'id'`))[0]).toMatchObject({ is_identity: "NO" });
expect((await db.execute(sql`SELECT to_regclass('public.plugin_lifecycle_acknowledgments') AS relation`))[0]).toMatchObject({ relation: null });
} finally {
await db.execute(sql`ALTER TABLE resource_lifecycle_events DROP CONSTRAINT fixture_reject_baseline`);
await applyDeliveryMigration();
expect((await db.execute(sql`SELECT is_identity FROM information_schema.columns WHERE table_schema = 'public' AND table_name = 'resource_lifecycle_events' AND column_name = 'id'`))[0]).toMatchObject({ is_identity: "YES" });
}
});
it("retries until acknowledgment, preserves resource order, and isolates progress per plugin", async () => {
const first = await createPlugin();
const second = await createPlugin();
const agent = await createAgent();
await createAgentLifecycle(db).pauseAgent(agent.id);
await worker.process(agent.id);
await createAgentLifecycle(db).resumeAgent(agent.id);
await worker.process(agent.id);
const journal = (await events()).sort((a, b) => a.id - b.id);
const inbox = pluginLifecycleInbox(db, first.id);
const [event] = await inbox.list(companyId);
expect(event).toMatchObject({ id: String(journal[0].id), resourceId: agent.id, action: "create" });
expect(await pluginLifecycleInbox(db, first.id).list(companyId)).toEqual([event]);
await expect(inbox.acknowledge(companyId, String(journal[1].id))).rejects.toMatchObject({ status: 409 });
await Promise.all([inbox.acknowledge(companyId, event.id), inbox.acknowledge(companyId, event.id)]);
expect(await inbox.list(companyId)).toEqual([expect.objectContaining({ action: "pause" })]);
expect(await pluginLifecycleInbox(db, second.id).list(companyId)).toEqual([event]);
await inbox.acknowledge(companyId, String(journal[1].id));
await inbox.acknowledge(companyId, String(journal[2].id));
expect(await inbox.list(companyId)).toEqual([]);
await db.delete(plugins).where(eq(plugins.id, first.id));
expect(await db.select().from(pluginLifecycleAcknowledgments).where(eq(pluginLifecycleAcknowledgments.pluginId, first.id))).toEqual([]);
});
it("pages past 100 failing resources without skipping their later events", async () => {
const plugin = await createPlugin();
const inbox = pluginLifecycleInbox(db, plugin.id);
await db.insert(resourceLifecycleEvents).values(Array.from({ length: 101 }, () => ({
companyId, resourceType: "agent" as const, resourceId: randomUUID(), action: "create" as const,
})));
const first = await inbox.list(companyId, 100);
await db.insert(resourceLifecycleEvents).values({ companyId, resourceType: "agent", resourceId: first[0].resourceId, action: "pause" });
const next = await inbox.list(companyId, 100, first.at(-1)!.id);
expect(next).toHaveLength(1);
expect(next[0].resourceId).not.toBe(first[0].resourceId);
await inbox.acknowledge(companyId, next[0].id);
expect(await inbox.list(companyId, 100, first.at(-1)!.id)).toEqual([]);
expect(await inbox.list(companyId, 100)).toEqual(first);
await expect(inbox.list(companyId, 100, "invalid")).rejects.toMatchObject({ status: 400 });
});
it("lists a resource's head without rechecking its acknowledged backlog for every later event", async () => {
// A long-lived resource accumulates an acknowledged prefix and then a
// backlog behind a deferred head. Checking each backlog event against
// every older event costs acknowledged × backlog probes; staging ran this
// statement for over 20 minutes per call.
const plugin = await createPlugin();
const inbox = pluginLifecycleInbox(db, plugin.id);
const resourceId = randomUUID();
const acknowledged = 2_000;
const backlog = 2_000;
const rows = await db.insert(resourceLifecycleEvents).values([
{ companyId, resourceType: "agent" as const, resourceId, action: "create" as const },
...Array.from({ length: acknowledged + backlog }, (_, index) => ({
companyId, resourceType: "agent" as const, resourceId, action: index % 2 ? "resume" as const : "pause" as const,
})),
]).returning({ id: resourceLifecycleEvents.id });
await db.insert(pluginLifecycleAcknowledgments)
.values(rows.slice(0, acknowledged + 1).map(row => ({ pluginId: plugin.id, eventId: row.id })));
const other = randomUUID();
await db.insert(resourceLifecycleEvents).values({ companyId, resourceType: "agent", resourceId: other, action: "create" });
const started = performance.now();
const listed = await inbox.list(companyId, 100);
const elapsedMs = performance.now() - started;
expect(listed.map(event => [event.resourceId, event.id])).toEqual([
[resourceId, String(rows[acknowledged + 1].id)],
[other, expect.any(String)],
]);
// Before: 3.5 s locally. After: under 30 ms. The bound leaves room for a slow runner.
expect(elapsedMs).toBeLessThan(2_000);
// Acknowledgement applies the same head rule that the list used.
await expect(inbox.acknowledge(companyId, String(rows[acknowledged + 2].id))).rejects.toMatchObject({ status: 409 });
await inbox.acknowledge(companyId, listed[0].id);
expect((await inbox.list(companyId, 100))[0]).toMatchObject({ resourceId, id: String(rows[acknowledged + 2].id) });
}, 120_000);
it("lists exactly one head per resource across acknowledged history, late creations, gaps, and pages", async () => {
const plugin = await createPlugin();
const other = await createPlugin();
const inbox = pluginLifecycleInbox(db, plugin.id);
const decoyCompanyId = randomUUID();
await db.insert(companies).values({ id: decoyCompanyId, name: "Lifecycle decoy", issuePrefix: `D${decoyCompanyId.replaceAll("-", "").slice(0, 6)}` });
const agentId = randomUUID(), lateCreateId = randomUUID(), gapId = randomUUID(), doneId = randomUUID(), projectId = randomUUID();
type Row = typeof resourceLifecycleEvents.$inferInsert;
const insert = async (rows: Row[]) => db.insert(resourceLifecycleEvents).values(rows).returning();
const ack = async (ids: number[], pluginId = plugin.id) => {
if (ids.length) await db.insert(pluginLifecycleAcknowledgments).values(ids.map(eventId => ({ pluginId, eventId })));
};
const [agentCreate] = await insert([{ companyId, resourceType: "agent", resourceId: agentId, action: "create" }]);
// The same id as a different resource type is an independent stream.
const runtime = await insert(Array.from({ length: 12 }, () => ({ companyId, resourceType: "agent_runtime" as const, resourceId: agentId, action: "update" as const })));
const [latePause] = await insert([{ companyId, resourceType: "agent", resourceId: lateCreateId, action: "pause" }]);
const gap = await insert([0, 1, 2].map(() => ({ companyId, resourceType: "project" as const, resourceId: gapId, action: "update" as const })));
const done = await insert([{ companyId, resourceType: "agent", resourceId: doneId, action: "create" }, { companyId, resourceType: "agent", resourceId: doneId, action: "pause" }]);
const [lateCreate] = await insert([{ companyId, resourceType: "agent", resourceId: lateCreateId, action: "create" }]);
const [projectCreate, projectArchive] = await insert([
{ companyId, resourceType: "project", resourceId: projectId, action: "create" }, { companyId, resourceType: "project", resourceId: projectId, action: "archive" }]);
await insert([{ companyId: decoyCompanyId, resourceType: "agent", resourceId: randomUUID(), action: "create" }]);
await ack([agentCreate.id, ...runtime.slice(0, 9).map(e => e.id), gap[1].id, ...done.map(e => e.id), projectCreate.id]);
await ack(runtime.map(e => e.id), other.id);
const all = ["agent", "project", "agent_runtime"] as const;
expect((await inbox.list(companyId, 100, undefined, [...all])).map(e => Number(e.id))).toEqual(
[runtime[9].id, gap[0].id, lateCreate.id, projectArchive.id].sort((a, b) => a - b));
expect((await inbox.list(companyId)).map(e => [e.resourceId, e.action])).toEqual(
[[gapId, "update"], [lateCreateId, "create"], [projectId, "archive"]]);
expect(await inbox.list(companyId, 2, undefined, [...all])).toEqual((await inbox.list(companyId, 100, undefined, [...all])).slice(0, 2));
expect((await inbox.list(companyId, 100, String(gap[0].id), [...all])).map(e => Number(e.id))).toEqual([lateCreate.id, projectArchive.id]);
expect(await pluginLifecycleInbox(db, other.id).list(companyId, 100, undefined, ["agent_runtime"])).toEqual([]);
expect(await pluginLifecycleInbox(db, other.id).hasPending(companyId, ["agent_runtime"])).toBe(false);
expect(await inbox.hasPending(companyId, ["agent_runtime"])).toBe(true);
expect(await inbox.hasPending(companyId)).toBe(true);
expect(await inbox.list(decoyCompanyId, 100, undefined, [...all])).toEqual([expect.objectContaining({ companyId: decoyCompanyId })]);
const [head] = await inbox.list(companyId, 1, undefined, ["project"]);
expect(head).toEqual({ id: String(gap[0].id), companyId, resourceType: "project", resourceId: gapId, action: "update", createdAt: gap[0].createdAt.toISOString() });
// Acknowledging a head exposes that resource's next unacknowledged event, skipping acknowledged gaps.
await inbox.acknowledge(companyId, String(gap[0].id));
await inbox.acknowledge(companyId, String(lateCreate.id));
expect((await inbox.list(companyId)).map(e => Number(e.id))).toEqual([latePause.id, gap[2].id, projectArchive.id].sort((a, b) => a - b));
await expect(inbox.acknowledge(companyId, String(runtime[10].id))).rejects.toMatchObject({ status: 409 });
});
it("carries acknowledgments from an earlier release into resource cursors without redelivery or loss", async () => {
const plugin = await createPlugin();
const other = await createPlugin();
const inbox = pluginLifecycleInbox(db, plugin.id);
const all = ["agent", "project", "agent_runtime"] as const;
const agentId = randomUUID(), gapId = randomUUID(), runtimeId = randomUUID(), createdOnlyId = randomUUID();
type Row = typeof resourceLifecycleEvents.$inferInsert;
const insert = async (rows: Row[]) => db.insert(resourceLifecycleEvents).values(rows).returning();
// An earlier release wrote only per-event acknowledgment rows.
const ack = async (ids: number[], pluginId = plugin.id) => {
if (ids.length) await db.insert(pluginLifecycleAcknowledgments).values(ids.map(eventId => ({ pluginId, eventId })));
};
const [agentCreate, ...agentTransitions] = await insert([
{ companyId, resourceType: "agent", resourceId: agentId, action: "create" },
...Array.from({ length: 6 }, (_, index) => ({ companyId, resourceType: "agent" as const, resourceId: agentId, action: index % 2 ? "resume" as const : "pause" as const })),
]);
const gap = await insert([0, 1, 2].map(() => ({ companyId, resourceType: "project" as const, resourceId: gapId, action: "update" as const })));
const runtime = await insert(Array.from({ length: 5 }, () => ({ companyId, resourceType: "agent_runtime" as const, resourceId: runtimeId, action: "update" as const })));
const [createdOnly] = await insert([{ companyId, resourceType: "agent", resourceId: createdOnlyId, action: "create" }]);
await ack([agentCreate.id, ...agentTransitions.slice(0, 3).map(e => e.id), gap[1].id, ...runtime.map(e => e.id), createdOnly.id]);
await ack([gap[0].id], other.id);
const before = await inbox.list(companyId, 100, undefined, [...all]);
const otherBefore = await pluginLifecycleInbox(db, other.id).list(companyId, 100, undefined, [...all]);
expect(before.map(e => Number(e.id))).toEqual([agentTransitions[3].id, gap[0].id]);
const migration = readFileSync(new URL("../../../packages/db/src/migrations/0462_plugin_lifecycle_resource_cursors.sql", import.meta.url), "utf8");
const backfill = migration.split("--> statement-breakpoint").filter(statement => statement.includes('INSERT INTO "plugin_lifecycle_resource_cursors"'));
expect(backfill).toHaveLength(1);
await db.execute(sql.raw(backfill[0]));
await db.execute(sql.raw(backfill[0]));
const cursors = await db.select().from(pluginLifecycleResourceCursors).where(eq(pluginLifecycleResourceCursors.companyId, companyId));
expect(cursors.map(c => [c.pluginId, c.resourceType, c.resourceId, c.acknowledgedThroughId]).sort()).toEqual([
[plugin.id, "agent", agentId, agentTransitions[2].id],
// The cursor stops before the unacknowledged gap, so the gap is still delivered.
[plugin.id, "project", gapId, gap[0].id - 1],
[plugin.id, "agent_runtime", runtimeId, runtime[4].id],
[plugin.id, "agent", createdOnlyId, createdOnly.id],
[other.id, "project", gapId, gap[0].id],
].sort());
expect(await inbox.list(companyId, 100, undefined, [...all])).toEqual(before);
expect(await pluginLifecycleInbox(db, other.id).list(companyId, 100, undefined, [...all])).toEqual(otherBefore);
expect(await inbox.hasPending(companyId, ["agent_runtime"])).toBe(false);
// Reads use the cursor: history at or below it stays acknowledged without its rows.
await db.delete(pluginLifecycleAcknowledgments).where(and(eq(pluginLifecycleAcknowledgments.pluginId, plugin.id), inArray(pluginLifecycleAcknowledgments.eventId, runtime.map(e => e.id))));
expect(await inbox.hasPending(companyId, ["agent_runtime"])).toBe(false);
await inbox.acknowledge(companyId, String(runtime[2].id));
expect(await inbox.list(companyId, 100, undefined, ["agent_runtime"])).toEqual([]);
// Acknowledging the gap moves the cursor and skips the event that the earlier release acknowledged.
await expect(inbox.acknowledge(companyId, String(gap[2].id))).rejects.toMatchObject({ status: 409 });
await inbox.acknowledge(companyId, String(gap[0].id));
expect((await inbox.list(companyId, 100, undefined, ["project"])).map(e => Number(e.id))).toEqual([gap[2].id]);
await inbox.acknowledge(companyId, String(gap[2].id));
expect(await inbox.list(companyId, 100, undefined, ["project"])).toEqual([]);
const [gapCursor] = await db.select().from(pluginLifecycleResourceCursors)
.where(and(eq(pluginLifecycleResourceCursors.pluginId, plugin.id), eq(pluginLifecycleResourceCursors.resourceId, gapId)));
expect(gapCursor.acknowledgedThroughId).toBe(gap[2].id);
// An earlier release that reads only acknowledgment rows sees the same progress after a rollback.
const rows = await db.select({ eventId: pluginLifecycleAcknowledgments.eventId }).from(pluginLifecycleAcknowledgments)
.where(and(eq(pluginLifecycleAcknowledgments.pluginId, plugin.id), inArray(pluginLifecycleAcknowledgments.eventId, gap.map(e => e.id))));
expect(rows.map(r => r.eventId).sort((a, b) => a - b)).toEqual(gap.map(e => e.id));
// A new transition after the cursor is delivered, and a later one waits behind it.
const [late, later] = await insert([{ companyId, resourceType: "agent_runtime", resourceId: runtimeId, action: "update" }, { companyId, resourceType: "agent_runtime", resourceId: runtimeId, action: "delete" }]);
expect((await inbox.list(companyId, 100, undefined, ["agent_runtime"])).map(e => Number(e.id))).toEqual([late.id]);
await expect(inbox.acknowledge(companyId, String(later.id))).rejects.toMatchObject({ status: 409 });
await inbox.acknowledge(companyId, String(late.id));
expect((await inbox.list(companyId, 100, undefined, ["agent_runtime"])).map(e => Number(e.id))).toEqual([later.id]);
await db.delete(plugins).where(eq(plugins.id, plugin.id));
expect(await db.select().from(pluginLifecycleResourceCursors).where(eq(pluginLifecycleResourceCursors.pluginId, plugin.id))).toEqual([]);
});
it("returns a full page when a manifest repeats a resource type", async () => {
const plugin = await createPlugin();
const inbox = pluginLifecycleInbox(db, plugin.id);
await db.insert(resourceLifecycleEvents).values(Array.from({ length: 101 }, () => ({
companyId, resourceType: "agent" as const, resourceId: randomUUID(), action: "create" as const,
})));
const page = await inbox.list(companyId, 100, undefined, ["agent", "agent"]);
expect(page).toHaveLength(100);
expect(page).toEqual(await inbox.list(companyId, 100, undefined, ["agent"]));
expect(await inbox.list(companyId, 100, page.at(-1)!.id, ["agent", "agent"])).toHaveLength(1);
});
it("keeps the cursor behind a pending transition when a late creation is acknowledged first", async () => {
const plugin = await createPlugin();
const inbox = pluginLifecycleInbox(db, plugin.id);
const lateId = randomUUID(), freshId = randomUUID();
const through = async (resourceId: string) => (await db.select().from(pluginLifecycleResourceCursors)
.where(and(eq(pluginLifecycleResourceCursors.pluginId, plugin.id), eq(pluginLifecycleResourceCursors.resourceId, resourceId))))[0]?.acknowledgedThroughId;
const [pause] = await db.insert(resourceLifecycleEvents).values({ companyId, resourceType: "agent", resourceId: lateId, action: "pause" }).returning();
const [create] = await db.insert(resourceLifecycleEvents).values({ companyId, resourceType: "agent", resourceId: lateId, action: "create" }).returning();
const [freshCreate] = await db.insert(resourceLifecycleEvents).values({ companyId, resourceType: "agent", resourceId: freshId, action: "create" }).returning();
expect((await inbox.list(companyId)).map(e => Number(e.id))).toEqual([create.id, freshCreate.id]);
await inbox.acknowledge(companyId, String(create.id));
await inbox.acknowledge(companyId, String(freshCreate.id));
// The earlier pause is still pending, so the creation does not move the cursor past it.
expect(await through(lateId)).toBeUndefined();
expect(await through(freshId)).toBe(freshCreate.id);
expect((await inbox.list(companyId)).map(e => Number(e.id))).toEqual([pause.id]);
await inbox.acknowledge(companyId, String(pause.id));
expect(await through(lateId)).toBe(pause.id);
// The creation above the cursor stays acknowledged through its row.
expect(await inbox.list(companyId)).toEqual([]);
expect(await inbox.hasPending(companyId)).toBe(false);
const [resume] = await db.insert(resourceLifecycleEvents).values({ companyId, resourceType: "agent", resourceId: lateId, action: "resume" }).returning();
expect(await inbox.hasPending(companyId)).toBe(true);
expect((await inbox.list(companyId)).map(e => Number(e.id))).toEqual([resume.id]);
await inbox.acknowledge(companyId, String(resume.id));
expect(await through(lateId)).toBe(resume.id);
expect(await inbox.list(companyId)).toEqual([]);
});
it("acknowledges a head without reading the resource's acknowledged history", async () => {
// Before the cursor, eligibility read every older event of the resource. Remove the
// acknowledgment rows below the cursor: a history scan would now see them as pending.
const plugin = await createPlugin();
const inbox = pluginLifecycleInbox(db, plugin.id);
const resourceId = randomUUID();
const rows = await db.insert(resourceLifecycleEvents).values(Array.from({ length: 50 }, () => ({
companyId, resourceType: "agent_runtime" as const, resourceId, action: "update" as const,
}))).returning({ id: resourceLifecycleEvents.id });
for (const row of rows.slice(0, 40)) await inbox.acknowledge(companyId, String(row.id));
await db.delete(pluginLifecycleAcknowledgments).where(eq(pluginLifecycleAcknowledgments.pluginId, plugin.id));
const all = ["agent_runtime"] as const;
expect((await inbox.list(companyId, 100, undefined, [...all])).map(e => Number(e.id))).toEqual([rows[40].id]);
await inbox.acknowledge(companyId, String(rows[40].id));
expect((await inbox.list(companyId, 100, undefined, [...all])).map(e => Number(e.id))).toEqual([rows[41].id]);
});
it("never skips a lower event id that commits after a higher id is acknowledged", async () => {
const plugin = await createPlugin();
const inbox = pluginLifecycleInbox(db, plugin.id);
await db.transaction(async tx => {
const [low] = await tx.insert(agents).values({ companyId, name: "Delayed commit" }).returning();
await recordResourceCreationEvent(tx as unknown as Db, companyId, "agent", low.id);
const high = await createAgent();
const [visible] = await inbox.list(companyId);
expect(visible.resourceId).toBe(high.id);
await inbox.acknowledge(companyId, visible.id);
expect(await inbox.list(companyId)).toEqual([]);
expect(low.id).not.toBe(high.id);
});
expect(await inbox.list(companyId)).toEqual([expect.objectContaining({ action: "create" })]);
});
it.each([{ resourceLifecycle: ["project"] }, { resourceLifecycle: ["agent_runtime"] }] as const)("preserves legacy manual streams with automatic subscription %j", async ({ resourceLifecycle }) => {
const plugin = await createPlugin();
await createAgent();
await db.insert(resourceLifecycleEvents).values({ companyId, resourceType: "project", resourceId: randomUUID(), action: "create" });
const services = buildHostServices(db, plugin.id, plugin.pluginKey, createPluginEventBus(), undefined, { manifest: { resourceLifecycle } as never });
const handlers = createHostClientHandlers({ pluginId: plugin.id, capabilities: ["events.subscribe"], services });
try {
const rows = await handlers["events.listLifecycle"]({ companyId }, { invocationScope: { companyId } });
expect(new Set(rows.map(row => row.resourceType))).toEqual(new Set(resourceLifecycle[0] === "agent_runtime" ? ["agent", "project", "agent_runtime"] : ["agent", "project"]));
} finally { services.dispose(); }
});
it("enforces capability, invocation company, plugin availability, and company boundaries through host RPC", async () => {
const plugin = await createPlugin();
await createAgent();
const services = buildHostServices(db, plugin.id, plugin.pluginKey, createPluginEventBus());
const handlers = createHostClientHandlers({ pluginId: plugin.id, capabilities: ["events.subscribe"], services });
const scope = { invocationScope: { companyId } };
try {
await expect(handlers["events.listLifecycle"]({ companyId })).rejects.toThrow("company context");
const other = randomUUID();
await expect(handlers["events.listLifecycle"]({ companyId: other }, scope)).rejects.toThrow("requested company");
const denied = createHostClientHandlers({ pluginId: plugin.id, capabilities: [], services });
await expect(denied["events.listLifecycle"]({ companyId }, scope)).rejects.toThrow("events.subscribe");
await expect(denied["events.acknowledgeLifecycle"]({ companyId, eventId: "1" }, scope)).rejects.toThrow("events.subscribe");
const [event] = await handlers["events.listLifecycle"]({ companyId }, scope);
await expect(handlers["events.acknowledgeLifecycle"]({ companyId: other, eventId: event.id }, scope)).rejects.toThrow("requested company");
await expect(pluginLifecycleInbox(db, plugin.id).acknowledge(other, event.id)).rejects.toMatchObject({ status: 404 });
await db.insert(pluginCompanySettings).values({ pluginId: plugin.id, companyId, enabled: false });
await expect(handlers["events.listLifecycle"]({ companyId }, scope)).rejects.toMatchObject({ status: 403 });
await expect(handlers["events.acknowledgeLifecycle"]({ companyId, eventId: event.id }, scope)).rejects.toMatchObject({ status: 403 });
await db.update(pluginCompanySettings).set({ enabled: true }).where(eq(pluginCompanySettings.pluginId, plugin.id));
await expect(handlers["events.listLifecycle"]({ companyId, limit: 0 }, scope)).rejects.toMatchObject({ status: 400 });
await expect(handlers["events.acknowledgeLifecycle"]({ companyId, eventId: "1e2" }, scope)).rejects.toMatchObject({ status: 400 });
await handlers["events.acknowledgeLifecycle"]({ companyId, eventId: event.id }, scope);
expect(await handlers["events.listLifecycle"]({ companyId }, scope)).toEqual([]);
await db.update(plugins).set({ status: "disabled" }).where(eq(plugins.id, plugin.id));
await expect(handlers["events.listLifecycle"]({ companyId }, scope)).rejects.toMatchObject({ status: 403 });
} finally {
services.dispose();
}
});
});