mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-13 07:12:00 +01:00
DottaandClaude Opus 5.5
3bf7af6c77
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Plugins get agent, project and agent runtime changes from the `resource_lifecycle_events` journal. Each plugin acknowledges events one resource at a time. > - #20405 made the inbox page read linear in the company journal size. The journal has no retention, so each page still reads every event of the company to find the unacknowledged ones: about 1.7 s per 1M events. > - The delivery worker reads a page again after each acknowledgment. The idle-sleep check also reads the whole journal when all events are acknowledged. `acknowledge` checks every older event of the resource. > - On a hosted instance, the page read ran 181 times in one window at about 1 s each. > - This pull request adds a per-plugin, per-resource cursor. Reads start at the cursor, so they do not read acknowledged history again. > - The benefit is that page, idle-check and acknowledgment cost follow the number of resources and pending events, not the journal size. Delivery order and results stay the same. ## Linked Issues or Issue Description Refs #20405 **What happened?** The plugin lifecycle inbox page query (`select ... from "resource_lifecycle_events" ... DISTINCT ON ...`) ran 181 times at about 1 s each on a hosted database in one window. Each acknowledgment causes another full read of the company journal. The idle-sleep check (`hasPending`) reads the full journal when all events are acknowledged. `acknowledge` reads the full history of the resource. **Expected behavior** A page read, the idle-sleep check and an acknowledgment must not read acknowledged history again. Their cost must not grow with the journal size. **Steps to reproduce** 1. Give a company 400 agents with 2,500 `agent_runtime` events each (1M events). 2. Acknowledge the first 2,000 events of each agent for one plugin. 3. Call `pluginLifecycleInbox(db, pluginId).list(companyId, 100, undefined, ["agent", "project", "agent_runtime"])`. 4. The read takes about 1.7 s. `hasPending` takes about 240 ms. **Paperclip version or commit** `8d2bf78323` (master). **Deployment mode** Hosted (PostgreSQL). Local repro uses embedded PostgreSQL 18. ## What Changed - `packages/db/src/schema/plugin_lifecycle_resource_cursors.ts`: new table `plugin_lifecycle_resource_cursors`. It has one row per plugin, company, resource type and resource id. `acknowledged_through_id` means: every event of that resource at or below this id is acknowledged by the plugin. Plugin and company deletion cascade. - Migration `0462_plugin_lifecycle_resource_cursors`: creates the table and carries existing acknowledgments forward (see Risks for the previous-release story). - `server/src/services/plugin-lifecycle-inbox.ts`: - `list` and `hasPending` use a backward loose index scan on `resource_lifecycle_events_resource_order_idx`. It visits each resource stream once and gets its latest event id. A stream whose latest event is at or below its cursor is settled and is not read again. For other streams, the read gets one head: the unacknowledged creation, else the first unacknowledged transition above the cursor. Then it applies `afterId`, id order and the limit, as before. - `acknowledge` reads only the events between the cursor and the acknowledged event. It still writes the per-event `plugin_lifecycle_acknowledgments` row. It moves the cursor only over a contiguous acknowledged prefix. A creation recorded after a pending transition does not move the cursor. - An event counts as acknowledged when it is at or below the cursor, or when it has an acknowledgment row. Thus rows written by an earlier release, and gaps in old history, keep their current behavior. - `server/src/__tests__/resource-lifecycle-events.test.ts`: new database tests for the migration carry-forward, gaps, late creations, cursor reads, the rollback rows and the plugin-delete cascade. All tests from #20405 still pass without change. - `scripts/benchmark-plugin-lifecycle-inbox.ts`: runs the migration backfill on the fixture and times it. Adds an acknowledgment case. Runs `VACUUM ANALYZE`, because hosted tables are autovacuumed. - `doc/DATABASE.md`: describes the cursor, its read cost and the migration. - The delivery worker (`plugin-lifecycle-delivery.ts`) is not changed. ## Verification Thesis: the cost is a read of the whole company journal (page, idle check) or the whole resource history (acknowledgment). If a per-resource cursor marks the acknowledged prefix, each read can skip that prefix. Then the cost follows the number of resource streams and pending events. The rows must stay equal. Local SQL timings: embedded PostgreSQL 18.1 on Apple Silicon, synthetic data, warm cache, one warmup, 7 samples, baseline and treatment order alternate per sample. The machine was shared with other jobs (load average about 35), so spread is wide. Values are median (min to max) in ms. Rows are equal (`assert.deepEqual`) for every list and probe case. Fixture A: 400 agents x 2,500 runtime events, 2,000 acknowledged per agent. 1,020,440 events, 800,420 acknowledgments. | Case | Before | After | Speedup | | --- | --- | --- | --- | | page of 100, all types | 1,700 (1,633-2,438) | 11.5 (9.5-12.2) | 148x | | page of 100, agent and project | 3.3 (2.5-5.4) | 4.2 (3.6-6.7) | 0.8x (slower) | | idle-sleep check | 238 (174-367) | 2.7 (2.1-3.4) | 88x | | acknowledge one head | 15.3 (9.4-30.1) | 20.3 (12.9-29.9) | neutral (noise) | | migration backfill | - | 9.3 s | - | Fixture B: 40 agents x 25,000 runtime events, 24,000 acknowledged per agent (long history). 1,020,080 events. | Case | Before | After | Speedup | | --- | --- | --- | --- | | page of 100, all types | 1,943 (1,642-4,979) | 4.4 (2.9-5.6) | 444x | | page of 100, agent and project | 3.5 (2.1-5.7) | 4.6 (2.8-12.0) | 0.8x (slower) | | idle-sleep check | 234 (220-243) | 2.8 (1.9-6.5) | 85x | | acknowledge one head | 64.6 (48.5-71.6) | 5.0 (2.6-7.1) | 13x | | migration backfill | - | 4.6 s | - | Fixture C: 5,000 agents x 200 runtime events, 190 acknowledged per agent (many resources). 1,025,040 events. | Case | Before | After | Speedup | | --- | --- | --- | --- | | page of 100, all types | 1,867 (1,649-1,975) | 76.9 (66.2-497) | 24x | | page of 100, agent and project | 10.3 (8.7-12.4) | 21.6 (20.2-23.2) | 0.5x (slower) | | idle-sleep check | 220 (201-298) | 2.8 (2.0-5.3) | 78x | | acknowledge one head | 4.9 (3.4-8.6) | 6.0 (5.2-7.6) | neutral | | migration backfill | - | 15.6 s | - | `EXPLAIN (ANALYZE, BUFFERS)` for the all-types page on fixture A: before 2,488 ms, 13,597 shared hits plus temp files; after 7.0 ms, 7,926 shared hits. Commands: ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-plugin-lifecycle-inbox.ts --baseline-ref=8d2bf78323 --agents=400 --per-agent=2500 --acked=2000 --samples=7 pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-plugin-lifecycle-inbox.ts --baseline-ref=8d2bf78323 --agents=40 --per-agent=25000 --acked=24000 --samples=7 pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-plugin-lifecycle-inbox.ts --baseline-ref=8d2bf78323 --agents=5000 --per-agent=200 --acked=190 --samples=7 cd server && npx vitest run src/__tests__/resource-lifecycle-events.test.ts src/__tests__/plugin-lifecycle-delivery.test.ts src/__tests__/project-instance-secrets.test.ts src/__tests__/agent-runtime-hooks.test.ts pnpm --filter @paperclipai/db check:migrations ``` - `resource-lifecycle-events.test.ts`, `plugin-lifecycle-delivery.test.ts` and `project-instance-secrets.test.ts`: 86 passed. `agent-runtime-hooks.test.ts`: 22 passed. - `tsc --noEmit -p server`: pass. `check:migrations`: pass. ## Risks - **Schema change.** Migration `0462` adds one table and two foreign keys to `plugins` and `companies`. It adds no index on an existing table. The backfill reads the acknowledgment table once. It took 4.6 s to 15.6 s on 1M-event fixtures with 0.8M to 1M acknowledgments. - **Migration order.** The number `0462` is reserved after open PRs that use `0459` to `0461` (#20571, #20568, #20573). Merge this PR after them. Before merge, rebase onto master and chain the snapshot after the latest migration. - **Previous-release story.** - Upgrade: the backfill stores, per plugin and resource, the end of the contiguous acknowledged prefix. A cursor never passes an unacknowledged event, so no event is lost. Acknowledged events above a gap keep their rows, so no event is delivered again. A test runs the migration statement on rows that only an earlier release wrote and checks that `list` returns the same rows before and after. - Rolling deploy: an earlier release that still runs writes only acknowledgment rows. The new reads honor those rows. - Rollback: the new code still writes every acknowledgment row, so an earlier release sees the same progress. - **Ordering assumption.** The cursor is safe because writers record a resource's events while they hold that resource's row lock (`paperclip_record_agent_runtime` locks the agent row; agent status and project writers document the lock). Thus a later commit for the same resource always has a higher id. A writer that breaks this contract could commit an event below the cursor, and that event would not be delivered. A lower id from a different resource is still delivered (existing test). - **Slower path.** A page that asks only for `agent` and `project` streams is 1 to 11 ms slower. These streams are short, so the old full read was cheap, and the new read visits each stream once. The cost is bounded by the number of resources. - **Many resources.** The page cost grows with the number of resource streams of the requested types. At 5,000 agents a page is about 77 ms, compared with about 1.9 s before. - The hosted planner can choose a different plan. The hosted improvement is not yet measured. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, local PostgreSQL benchmarks). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>