mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 11:59:26 +01:00
<!-- Write all pull request text in Simplified Technical English (ASD-STE100): short sentences, one instruction per sentence, simple approved vocabulary, and the active voice. --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A task in review needs a path to its next action: a reviewer, a pending interaction or approval, a monitor, a run, a wake, an external reply or a recovery. A review without a path is stalled, and the attention feed tells a person about it. > - The server decides this for each task in review (`listIssueReviewAttentionMap` in `server/src/services/issues.ts`, with the classification in `server/src/services/recovery/issue-graph-liveness.ts`). The task list, the blocked inbox and the attention feed call it. `GET /attention/summary`, which the board sidebar reads on most pages, runs it for every task in review of the company. > - The classification did the company-wide work again for each task. Each check of an agent copied the agent list, indexed it again and walked the reporting chain. Each task scanned every run, wake, interaction, approval, external wait and recovery of the request. Each stalled review also filtered and sorted every agent again, with one such check for each agent. > - Thus the work grew with reviews × agents for covered reviews and with stalled reviews × agents² for stalled reviews. It also allocated a new list and a new index for each check, and the garbage collector of the server paid for that. > - On a hosted instance, an allocation sample of the server put 4.7% of all allocated bytes under this function, and garbage collection used 25% of the main thread. The two slowest board requests there are the attention summary and the task list, which both call it. > - This pull request builds one index for one classification: the agents by id, the eligibility of each agent, the invokable agents of each company in id order, and the paths of each task. Each task then costs its own paths. > - The benefit is that the review classification of a summary request falls from 62 ms to 5 ms for 900 reviews and 60 agents, and from 123 ms to 8 ms for 300 reviews and 180 agents. The request allocates 57% to 91% fewer bytes. Every finding, every review path and every count is the same. ## Linked Issues or Issue Description Refs #20851 and #20858 (they batched the review-wait provenance check and the task read decision of the same requests; this change is the next cost on that path). Refs #19194 and #13729 (open; they change the failed-run lookup of the attention feed, which is a different statement and a different mechanism; this pull request does not touch it). **What existing behavior does this improve?** The time and the memory that the server uses to decide the review attention of the tasks in review, for `GET /api/companies/:companyId/attention/summary`, `GET /api/companies/:companyId/attention`, `GET /api/companies/:companyId/issues` and the blocked inbox. **Subsystem affected** Server: task-graph liveness classification (`server/src/services/recovery/issue-graph-liveness.ts`) and the review-attention map in `server/src/services/issues.ts`. **Current behavior** - `classifyIssueReviewPaths(input, issue)` built `new Map(input.agents…)` for each task and scanned six path lists of the request for each task. - `isInvokableAgent` copied the agent list for each check. `getAgentOrgChainHealth` then indexed that copy again. - `ownerCandidatesForRecoveryIssue` ran for each stalled review. It filtered and sorted all agents, and it checked each agent again in two more loops. Each check did the copy and the index above. - `hasActiveExecutionPath` built a joined list of the runs and the wakes for each blocker it looked at. - `listIssueReviewAttentionMap` searched the review list for each open recovery task (`reviewIds.includes`) and for each review (`issues.find`). Evidence from a hosted instance (sanitized): - Server main thread at 93% of one core. Garbage collection: 25% of the main thread (387 pauses in 32 s, the longest 224 ms). - Allocation sample, 30 s: 77.7 MB/s in total. `listIssueReviewAttentionMap` and its callees: 4.7% of all bytes (`classifyIssueReviewPaths` 4.4%, `classifyIssueGraphLiveness` 2.3%, `reviewFinding` 2.3%). - `GET /attention/summary` took 16.3 s of server time and `GET /issues` up to 15.3 s. Most of that time is waiting behind other work on the same thread (see "Not verified"). **Proposed behavior** One classification indexes its input one time. The cost of one task is the number of its own paths. The cost of the agents is one eligibility decision for each agent. The results do not change. **Reason and benefit** The summary request runs on most board pages. Its CPU time and its allocation also delay every other request, because the server has one main thread and one garbage collector. **Breaking changes** None. `classifyIssueGraphLiveness` and `classifyIssueReviewPaths` keep their signatures and results. `packages/shared` does not change. ## What Changed - `server/src/services/recovery/issue-graph-liveness.ts` - New internal index for one input (`createIssueGraphLivenessIndex`): agents by id, a memo of `getAgentWorkEligibility` for each agent record, the invokable agents of each company in id order, and the paths of each task for the six path lists. The path lists are indexed on first use and keep the order of the input. The company check of each path stays. - `classifyIssueGraphLiveness` builds the index one time and uses it for the review paths, the waiting paths of blockers, the owner candidates and the assignee eligibility of blockers. - New export `createIssueReviewPathClassifier(input)`. It returns the review paths of many tasks from one index. `classifyIssueReviewPaths(input, issue)` stays for single tasks. - The memo uses the agent record as its key, not the agent id. Thus an input that lists one id two times gets the same answers as before. - The eligibility decision itself is still `getAgentWorkEligibility` of `@paperclipai/shared`, with the same agent list. The index calls it one time for each agent, not one time for each check. - `server/src/services/issues.ts` (`listIssueReviewAttentionMap` only): one classifier for all reviews of the request, a set for the review ids, and the review by position instead of a search. No query changes. - Tests and the reproduction (see Verification): - `server/src/__tests__/helpers/issue-graph-liveness-reference.ts`: the classification as it was before this change. It is the oracle of the equivalence tests. - `server/src/__tests__/issue-graph-liveness-equivalence.test.ts`, `server/src/__tests__/review-attention-classification-equivalence.test.ts`, `server/src/__tests__/helpers/review-attention-fixture.ts`. - `server/scripts/benchmark-review-attention.ts`: the reproduction through the real routes. ## Verification Reproduction and before/after measurement. The script starts a local PostgreSQL, seeds a synthetic company, mounts the real `attentionRoutes` and `issueRoutes` with a board user, and sends the two requests over HTTP. With `--baseline-ref` it runs the pre-fix modules in a second process against the same database. It fails unless both versions return the same summary, the same full attention feed, the same task list and the same review attention for every task in review. ```sh pnpm --filter @paperclipai/server exec tsx scripts/benchmark-review-attention.ts --baseline-ref=ef2b53b675feebb1bfbd6701627f6c302f8c6916 --tasks=3000,9000 --agents=60 pnpm --filter @paperclipai/server exec tsx scripts/benchmark-review-attention.ts --baseline-ref=ef2b53b675feebb1bfbd6701627f6c302f8c6916 --tasks=3000 --agents=180 ``` Fixture: one company with tasks of every status (10% in review, 10% blocked), four of five tasks are children, a reporting tree of agents with paused, terminated and pending agents. The reviews cover every kind of path and three ways to stall (31% stalled). The blocked tasks wait behind chains, cycles, cancelled, done, backlog and unassigned blockers. Open recovery tasks (5% of the tasks) name reviews, blocked tasks and done tasks. 6% of the reviews wait for a GitHub reply. A second company with the same shapes is in the same database. Embedded PostgreSQL 18.1, Node v26.4.0, `VACUUM ANALYZE` after seeding. Each version ran in a new process, three rounds in alternating order, 2 warm-up requests and 5 measured requests in each round. A CPU profile of 3 more requests gives the last two columns. | Tasks / agents / reviews | Request | Version | Statements | Server ms, median (min) | CPU ms, median (min) | Main thread busy ms, median (min) | Allocated MB | Review classification ms | GC ms | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | 3000 / 60 / 300 (94 stalled) | attention/summary | before | 69 | 121.4 (112.7) | 131.5 (114.4) | 80.2 (73.2) | 170.4 | 16.8 | 5.0 | | 3000 / 60 / 300 (94 stalled) | attention/summary | after | 69 | 105.9 (94.7) | 113.3 (96.3) | 67.3 (58.6) | 73.3 | 2.4 | 3.4 | | 9000 / 60 / 900 (283 stalled) | attention/summary | before | 85 | 280.9 (263.9) | 281.4 (253.4) | 203.7 (186.8) | 509.8 | 62.1 | 30.0 | | 9000 / 60 / 900 (283 stalled) | attention/summary | after | 85 | 224.8 (213.4) | 220.1 (205.8) | 149.9 (142.3) | 219.1 | 4.8 | 6.6 | | 3000 / 180 / 300 (94 stalled) | attention/summary | before | 69 | 241.2 (203.9) | 248.7 (227.4) | 197.1 (171.1) | 950.2 | 123.2 | 21.2 | | 3000 / 180 / 300 (94 stalled) | attention/summary | after | 69 | 127.8 (108.5) | 128.5 (102.0) | 76.7 (64.8) | 87.3 | 7.8 | 5.1 | | 3000 / 60 / 300 | task list (500 rows, 48 reviews) | before | 59 | 102.7 (96.7) | 92.4 (83.6) | 54.7 (48.9) | 73.2 | 2.6 | 2.4 | | 3000 / 60 / 300 | task list (500 rows, 48 reviews) | after | 59 | 106.2 (91.2) | 89.9 (78.8) | 55.2 (44.8) | 63.4 | 0.9 | 1.6 | | 9000 / 60 / 900 | task list (500 rows, 50 reviews) | before | 42 | 194.3 (186.0) | 77.5 (69.1) | 43.3 (38.4) | 57.3 | 1.0 | 1.8 | | 9000 / 60 / 900 | task list (500 rows, 50 reviews) | after | 42 | 189.6 (182.3) | 72.1 (69.5) | 41.9 (39.4) | 57.8 | 1.2 | 2.5 | | 3000 / 180 / 300 | task list (500 rows, 48 reviews) | before | 59 | 124.6 (95.7) | 110.2 (99.4) | 67.8 (58.1) | 165.4 | 15.4 | 3.8 | | 3000 / 180 / 300 | task list (500 rows, 48 reviews) | after | 59 | 104.1 (91.9) | 90.7 (79.9) | 56.2 (47.1) | 78.2 | 2.9 | 5.3 | How to read the table: - "Review classification ms" is the time in `listIssueReviewAttentionMap` and everything it calls on the main thread, for one request, from the CPU profile. It is the direct measure of this change. Before the fix it was the largest block of application code in the summary handler: 22% of the busy time at 3000 tasks, 20% at 9000 tasks, and 61% with 180 agents. `getAgentOrgChainHealth` under `isInvokableAgent` under `ownerCandidatesForRecoveryIssue` had most of it. After the fix it is 4% to 9%. - The scaling: three times the tasks gave 3.7 times the classification time before and 2.0 times after. Three times the agents gave 7.3 times before and 3.3 times after. - "Allocated MB" is all bytes that the server process allocated for one request (`total_allocated_bytes`), including the HTTP client of the script. - Other agents used the same machine during the measurement (load average 8 to 16 on 18 CPUs), so the medians of server time have noise. The profile columns and the allocation column are the stable ones. - The task list changes little with 60 agents, because one page has about 50 reviews. With 180 agents its classification falls from 15 ms to 3 ms and its allocation by half. - Statement counts do not change: this change removes no statement. Red/green regression test for the mechanism (`server/src/__tests__/issue-graph-liveness-equivalence.test.ts`): - `reads each agent and each path a number of times that does not grow with the reviews`. The test counts the reads of the agent status and of the path task id for 20 and for 400 stalled reviews with 40 agents and 360 paths. On the code before this change: `expected { agentStatus: 200800, pathIssueId: 144000 } to deeply equal { agentStatus: 10040, pathIssueId: 7200 }`. After: 316 agent reads and 360 path reads for both sizes (one read of each path). The test also runs the oracle and asserts that its reads grow, so it shows what it catches. - `classifies the review paths of a page from one index`. The same count for `createIssueReviewPathClassifier`. Equivalence tests (old code and new code on the same input): - `returns the findings and review paths of the pre-fix classification for random task graphs`. 400 seeded random graphs of two companies. Each graph has reviews with and without paths, blocked chains and cycles, terminal blockers, relations and paths that name the other company or a missing task, reporting chains with cycles, missing managers and managers of the other company, and ids that the input lists two times. The test compares the findings of the graph and the review paths of every task, for both `classifyIssueReviewPaths` and the new classifier. It asserts that the graphs reach all six finding states and all nine kinds of path, each at least 20 times. - `returns the pre-fix findings for blocked chains, cycles and terminal blockers`. One named graph with the expected state of each finding, so the comparison cannot pass on two wrong answers. - `server/src/__tests__/review-attention-classification-equivalence.test.ts`, on real PostgreSQL through the real routes. The test wraps the classification module: every classification that a request runs is also run by the oracle on the same input, and a difference fails the test. - `gives every review the paths the fixture was built with, and the pre-fix classification of every request`. `GET /issues?status=in_review`, `GET /issues` and `GET /issues?attention=blocked` (the blocked inbox, whose graph has the chains, cycles and terminal blockers). Each of the 64 reviews has the path kinds and the stalled reason that the fixture states for it, including reviews that wait for an external reply and reviews that open recovery tasks cover. - `counts the reviews that need a person, leaves out the ones a person dismissed, and reads one company`. `GET /attention/summary` and `GET /attention`: the review count and the review cards are the reviews that the fixture built to need a person. The second company has its own counts and none of the first company's tasks. Three dismissed cards leave the count and stay stalled in the task list. - The benchmark above compares the complete responses of both versions through the routes at each size (summary, full feed, task list, review attention of every review). Commands run locally: - `pnpm --filter @paperclipai/server exec vitest run` for `issue-graph-liveness-equivalence`, `review-attention-classification-equivalence`, `issue-liveness`, `recovery-classifiers`, `issue-review-attention`, `issue-blocker-attention`, `blocked-inbox-review-wait-scope`, `heartbeat-issue-liveness-escalation`, `attention-retention-work` and `attention-service`: 210 tests passed. In that run the `afterAll` hook of `attention-retention-work` and of `attention-service` (the stop of the test database) timed out at 30 s while other agents loaded the machine. No assertion failed. CI runs both files. - `tsc --noEmit` for the server package: no errors. The new test files and the script are outside the server `tsconfig`; a focused `tsc` program for them reports no error in these files. - `pnpm check:module-boundaries`, `node scripts/check-no-git-push.mjs` and `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs`: pass. - The first CI run of this pull request failed `ci / Runner support suite`. An earlier revision also let the eligibility functions of `packages/shared` accept a prebuilt index, and the source of `packages/shared` is pinned for the runner SDK source proof (`tests/runner-e2e/source-sdk-public-inputs.json`). The fix does not need that change, so the current revision leaves `packages/shared` as it is on `master`: no pin moves, and the UI, the CLI and the runner use the same eligibility code as before. `source-sdk-public-inputs.test.mjs`, `source-sdk-evidence.test.ts` and `source-sdk-public-type.test.ts` pass locally on this revision (58 tests), and the drift guard reports no stale pin. - `pnpm verify` returns exit 2 (broader verification required) because the change adds a file under `server/scripts/`. CI supplies the broad coverage. The repo-wide typecheck, build and test suite were not run locally (shared machine). Not verified: - The effect on the hosted instance. This pull request does not deploy. The local requests take 0.1 s to 0.4 s, and the hosted requests took 15 s. Thus most of the hosted time is not this code: the hosted main thread is saturated by other work, and each of the 69 to 85 statements of a summary request waits for its turn on that thread. This change removes CPU time and allocation from that thread. It does not remove a statement. - The number of agents and of stalled reviews of the hosted company. The hosted allocation sample has most of the bytes in `classifyIssueReviewPaths`, which is the cost for each review (reviews × agents). The local fixture has more stalled reviews, where the owner candidates cost more. - A remaining cost inside the classification: each eligibility decision of `@paperclipai/shared` indexes the agent list again. One classification now does this one time for each agent (agents² map entries for one request, 3 ms with 180 agents), not one time for each check. The helper can accept a prebuilt index in a later change; that change moves the SDK source pins, so it needs its own review. - A second, independent cost that this pull request does not change: `GET /attention/summary` returns counts only, but it builds the full feed. After this change, the largest application blocks in its profile at 9000 tasks are the full task list of the 900 blocked tasks (15 ms on the main thread, of which 9 ms decode description previews that the summary never returns), the blocker attention of those tasks (9 ms), and the text of its 85 statements. A later change can give the summary a count-only path. ## Risks - The attention feed decides what a person is told needs them. The risk is a classification that differs from the previous one. Mitigation: the rules did not move. Only the lookups changed (a map lookup instead of a scan, a stored eligibility instead of a new one). The oracle tests compare the findings and the paths for random graphs, for a named graph and for every classification of the route tests. The benchmark compares the full responses. - The index is a snapshot of the input. `createIssueReviewPathClassifier` documents that the input must not change while the classifier is in use. The two callers build the input and use it in the same synchronous block. - When the input has no `now`, one classification now reads the clock one time. Before, `classifyIssueGraphLiveness` read it one time for the blocker checks and one more time for each review. All callers in the server that classify many tasks pass `now` or run in one synchronous call, so the values differ by less than the run time of the call. - The oracle file is a copy of the previous classification (about 640 lines, test-only). A later change of a classification rule must change the oracle in the same way. The header of the file says so. - The repository rule in `CONTRIBUTING.md` says that a pull request has no code comments. This pull request has comments where the code has a constraint that the names do not show (the memo key, the snapshot rule, the purpose of the oracle, the usage of the script), as the merged changes around this code do. - No migration. No query change. No change to company scope or to actor permissions: the change is after the reads, on rows that the same queries return. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`), Anthropic, used through Claude Code with tool use (code reading, editing, shell, tests, CPU profiles). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>