mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 06:10:44 +01:00
DottaandClaude Opus 5.5
326df52673
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - The Mine inbox reads `GET /api/companies/:companyId/inbox/mine`. Its
main query is `issueService.list` with "touched by me", "not archived by
me", six statuses including `done`, and `LIMIT 100` by canonical last
activity.
> - That query evaluates `touchedByUserCondition` (a five-way OR of
EXISTS) on every status-matching issue in the company. Because `done` is
included, the scan grows with company history, not with the user's work.
> - The unread badge already solved the same problem with a UNION of
indexed participation sources (#17973). The list path did not.
> - This pull request makes the Mine list rank page ids from that same
participation UNION, then load full rows (with the expensive select-list
subqueries) only for the 100 selected ids.
> - The benefit is that the main Mine query scales with the user's
participation instead of the company's issue count. Results stay
identical, and a new test proves it against the original scan.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
The latency of the Mine inbox snapshot, `GET
/api/companies/:companyId/inbox/mine`.
**Subsystem affected**
Server: issue list service (`server/src/services/issues.ts`) and the
Mine route.
**Current behavior**
Measured on 2026-11-09: 280–570 ms server time on a dev instance and
1.9–3.2 s on staging. The list query scans every status-matching issue
(28k of 30k in the fixture below) to test participation, then sorts the
touched ones.
**Proposed behavior**
The list query reads only the user's participation set and computes the
row projection only for the page.
**Reason and benefit**
Inbox opens are frequent. The old cost grows with every completed task
in the company.
Refs #17973 (same UNION for the unread badge).
## What Changed
- Extracted the participation UNION from `unreadInboxIssueIds` into
`mineParticipationIssueIds()`. The unread badge uses it unchanged.
- New `IssueFilters.touchedCandidates: "participation"`. When it is set
with `touchedByUserId` (and there is no search and no
`oneChildPerParent`), `list`:
1. replaces `touchedByUserCondition` with `issues.id IN (<participation
UNION>)`. Each UNION member is one OR branch of the old condition, so
the set is the same;
2. runs a narrow `candidates` query (`SELECT id … WHERE <all other
filters> ORDER BY <same order> LIMIT/OFFSET`);
3. loads the full projection for only those ids (re-checking company,
visibility and the read condition), and restores the phase 1 order.
- Only the Mine route sets the flag. Every other `svc.list` caller keeps
the old SQL.
- A new Server-Timing phase, `inbox_issues_candidates`.
- `server/src/__tests__/inbox-mine-candidates.test.ts`: a
result-equivalence test.
- `scripts/benchmark-inbox-mine.ts`: a local benchmark that seeds a
large fixture, compares full payloads, and records `EXPLAIN (ANALYZE,
BUFFERS)` per query.
### Not changed, with evidence
- **Recomputed comment/activity maxima** (`lastActivityStatsForIssues`):
0.9 + 0.2 ms per call in the plans. Not worth a second code path.
- **Touched condition inside `userCommentStatsForIssues`**: 1.4–2.1 ms.
Not changed.
- **`liveDescendantCountMapForIssues`**: 12 ms at 30k issues and 40 ms
at 90k. It now costs about as much as the list query. It scans all live
runs and hashes all company issues. It is company-wide work, outside the
candidate selection, so I left it for a separate change.
## Verification
### Correctness
`inbox-mine-candidates.test.ts` runs the old scan and the new path on
the same data. It asserts equal ordered ids and equal full JSON
payloads. Fixture: 170+ touched issues (more than one page, with tied
timestamps) plus these edge cases:
- included: created by me, assigned to me only, user activity, my direct
comment, `done`, pending request addressed to me, archived then
re-activated by a newer outside comment, read state with a later agent
comment, a live child (live descendant count);
- excluded: archived with no newer activity, mention-only, read audit
only (`issue.read_marked`), uppercase entity id, channel-sourced
comment, agent-created with my user id, `cancelled`, hidden, harness,
plugin operation, private to another user (`issueReadSqlCondition`),
answered request, untouched issues, another company.
It also compares other shapes: no archive filter, `status=done` with the
full projection, `offset`/`limit`, ascending order, unrestricted read
condition, a second user, and a user with no participation. Mutation
check: removing the comment source from the UNION makes the test fail.
The benchmark also asserts identical full payloads for every sample of
both arms.
Focused suites (local): `inbox-mine-candidates`, `inbox-mine-routes`,
`authorization-privacy-query`, `issues-unread-inbox`,
`inbox-archive-routes`, `issues-assignee-unread`,
`issues-creation-visibility` — all pass.
### Performance (local SQL measurements)
Command: `pnpm --filter @paperclipai/server exec tsx
../scripts/benchmark-inbox-mine.ts --baseline-ref=18c3f4ef7b
--issues=<N> [--touch-density=<d>]`. Baseline = 18c3f4ef7b, the #20408
head with the same timing hooks. Embedded PostgreSQL 18.1,
`shared_buffers` 128MB, `work_mem` 4MB, JIT on (not triggered), pool of
8. Alternating order, 2 warmups. Fixture per run: 12 activity rows and 6
comments per issue, 55% `done`, archives, read states, pending requests,
150 live runs.
| Fixture | Arm | `svc.list` median (min–max) | List query phase | Main
query plan |
| --- | --- | --- | --- | --- |
| 30k issues, user touched ~1.1k | before | 47.8 ms (41–66) | 30.0 ms |
28.1 ms exec, 33.7k buffers |
| | after | 39.4 ms (36–42) | 19.4 + 4.2 ms | 16.0 ms exec, 22.1k
buffers (+1.8 ms rows) |
| 90k issues, user touched ~3.4k | before | 124.4 ms (113–180) | 71.1 ms
| 70.8 ms exec, 103.9k buffers |
| | after | 102.9 ms (88–118) | 43.1 + 3.8 ms | 41.7 ms exec, 72.7k
buffers |
| 30k issues, user touched ~11k (dense) | before | 135.0 ms | 119.2 ms |
112.6 ms exec, 128k buffers |
| | after | 136.6 ms | 108.7 + 4.9 ms | 101.9 ms exec, 144k buffers |
- The old plan scanned 27,993 status-matching issues to test
participation. The new plan reads the participation UNION (1,155 ids at
30k; 3,448 at 90k) and probes `issues_pkey`.
- **Neutral case:** when a user has touched most issues (dense row), the
UNION does not shrink the set. The per-candidate privacy check, archive
visibility and activity maxima dominate. I report this as neutral, not
as a gain.
- On a loaded machine (a parallel test run), the same 30k fixture took
129 ms before and 55 ms after. Planning time of the old query rose to 27
ms there. I report the quiet-machine numbers above as the result.
- I did not get query evidence from staging's database. This session had
no Neon or CloudOps access (no `neonctl`, no stack credentials), so all
numbers are local. #20408 adds the per-phase header, which can confirm
the effect on staging after deploy.
### Pre-PR checks
- `tsc --noEmit -p server`: no errors in the changed files.
- `pnpm verify --explain` selects 2,417 of 3,204 test files for this
change, because `services/issues.ts` is imported almost everywhere. I
did not repeat that full suite locally (AGENTS.md §7); required CI
supplies it.
## Risks
- **Equivalence depends on the UNION matching
`touchedByUserCondition`.** Each branch has a UNION member:
human-created by me, assigned to me, pending request addressed to me, my
participation activity (canonical lowercase UUID ids only, the same as
the old `= issues.id::text` join), and my direct human comments. The
equivalence test pins these. A future change to `touchedByUserCondition`
must also change the UNION. Both live next to each other, and the unread
badge already depends on the same equality.
- **Two reads instead of one.** A row can change between the candidate
query and the row query. The row query re-checks company, visibility and
read access, so a row that became unreadable is dropped. The page can
then have fewer than 100 rows for that one response.
- Scope: only the Mine route sets the flag. No migration.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, via Claude Code with
tool use (shell, file edits, embedded PostgreSQL benchmarks). Standard
context window; extended thinking not used.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>