PaperClipAI

Jannik/PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-13 07:12:00 +01:00

Commit Graph

Author	SHA1	Message	Date
DottaandClaude Opus 5.5	81c52e3666	perf(chat): find terminal GitHub responses by issue run indexes (#20671 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The GitHub chat channel posts a terminal response when an agent run for a GitHub request stops. A background sweep finds acknowledged response comments (`processed`, not final) whose issue is blocked or whose latest run is terminal > - For each candidate, the sweep reads the status of the latest run of the assigned agent for the issue. It matches runs with `coalesce(native_issue_id::text, context_snapshot->>'issueId')` > - No index has that expression. So PostgreSQL reads every run of the agent for each candidate, and it decompresses the large legacy `context_snapshot` of each run > - On a hosted test database, this statement ran 13 times at a mean of 11 s in a 38-minute window (149 s in total), with single runs of 16 s and 41 s > - This pull request splits the run match into its two disjoint branches. Each branch uses an existing issue index > - The benefit is that each candidate reads a few index entries instead of the agent's full run history. The returned rows do not change, and no migration is necessary ## Linked Issues or Issue Description What existing behavior does this improve? The discovery page query in `registerGitHubTerminalResponseWork` in `server/src/services/chat-channels/github/terminal-response-work.ts`. The query runs when the sweep reconstructs its candidates, and once or twice for each candidate check. Subsystem affected GitHub chat channel terminal-response recovery. Current behavior The `WHERE` clause has `issues.status = 'blocked' or (select r.status from heartbeat_runs r where r.company_id = ... and r.agent_id = ... and coalesce(r.native_issue_id::text, r.context_snapshot->>'issueId') = <issue id> order by r.created_at desc, r.id desc limit 1) in (<terminal statuses>)`. The planner can only use `heartbeat_runs_company_agent_started_idx`. It reads all runs of the agent, then evaluates the `coalesce` on each row. That decompresses the context of each legacy run. The cost is the number of candidates times the number of agent runs. Proposed behavior The subquery takes the latest run from two branches and then the latest of the two: - runs with `native_issue_id = <issue id>`, through `heartbeat_runs_company_native_issue_contract_id_uq` - runs with `native_issue_id is null and context_snapshot->>'issueId' = <issue id>`, through `heartbeat_runs_company_ctx_issue_created_idx` The two branches are disjoint, and together they are exactly the rows where the `coalesce` is equal to the issue ID. Both branches and the outer query use the same order (`created_at desc, id desc`). So the selected run is the same, including ties. Reason and benefit This statement was the largest single-statement cost in the hosted window after the earlier database fixes. A local reproduction is about 490x faster on the default fixture, and the gain grows with the agent's run history. Breaking changes None. The API, the candidate rules and the results do not change. ## What Changed - Rewrite the latest-run status subquery in the GitHub terminal-response discovery page as two index-backed branches (native binding, and legacy context without a native binding). - Add two integration tests for the sweep. In the first, newer runs of the same agent that do not bind the issue (one with a native binding to another issue, but a legacy context that names this issue) must not hide the stopped run. In the second, a newer active run with a native binding to the issue keeps the request waiting. - Add `scripts/benchmark-github-terminal-response-sweep.ts`. It captures the SQL that the real worker generates at a baseline commit and at `HEAD`, compares all rows, and measures both. ## Verification Cause and thesis. The cost comes from the `coalesce` run match, which has no index and makes each candidate scan the agent's run history. Thesis: if each precedence branch uses its own issue index, the cost of the page no longer depends on the agent's run count, and the rows stay identical. The benchmark confirms the thesis. Local SQL benchmark: ```sh pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-github-terminal-response-sweep.ts --baseline-ref=7b33384446 ``` Fixture (synthetic): two companies, each with one GitHub endpoint and one assigned agent. Each issue has one GitHub response comment. The first N comments are acknowledged and not final; the others are final. Every tenth issue is blocked. Runs alternate between native binding and legacy context, and each run has a 4 KiB incompressible context. The fixture also has these edge cases: newer active runs with a native binding to another issue but a legacy context that names this issue, active runs that tie on `created_at` with the latest run (between native and legacy runs), and newer active runs of another agent. The benchmark compares every column of every candidate row (the page limit is lifted for this check). Embedded PostgreSQL 18.1 on an Apple laptop. The hosted database is Neon with different compute, so absolute times are local SQL measurements only. Each size has 2 warmups and then alternates the baseline and the treatment. \| Size \| Runs (both companies) \| Candidate rows equal \| Baseline median (range) \| Treatment median (range) \| Speedup \| \| --- \| --- \| --- \| --- \| --- \| --- \| \| Small: 200 issues, 3 runs each, 20 candidates, 7 samples \| 1,372 \| 26 of 26 \| 18.7 ms (17.7–20.8) \| 3.8 ms (3.3–4.3) \| 4.9x \| \| Default: 1,500 issues, 8 runs each, 150 candidates, 7 samples \| 25,300 (156 MB) \| 200 of 200 \| 3,697 ms (2,875–4,032) \| 7.6 ms (6.6–9.6) \| 490x \| \| Large: 3,000 issues, 10 runs each, 400 candidates, 5 samples \| 62,600 (382 MB) \| 538 of 538 \| 57,055 ms (48,343–66,079) \| 31 ms (16–79) \| 1,800x \| Sizes are set with `BENCH_ISSUES`, `BENCH_RUNS_PER_ISSUE`, `BENCH_CANDIDATES` and `BENCH_ITERATIONS`. The timed query is the real first page (limit 100). Plan (default size): the baseline runs `SubPlan` 270 times. Each run is a `Bitmap Heap Scan` on `heartbeat_runs_company_agent_started_idx` that removes about 12,000 rows by filter, for about 8.2 million buffer reads in total. The treatment runs an `Append` of two `Limit` index scans (`heartbeat_runs_company_native_issue_contract_id_uq` and `heartbeat_runs_company_ctx_issue_created_idx`), for about 3,800 buffer reads in total. Tests: - `cd server && npx vitest run src/__tests__/chat-channels.integration.test.ts -t "terminal GitHub"`: 13 passed, including the 2 new tests. - Mutation check: when the legacy branch does not exclude runs with a native binding, the first new test fails. It passes with the fix. - `cd server && npx tsc --noEmit`: passed. ## Risks - Low risk. The change is one SQL subquery. The match is the same set of runs: `native_issue_id::text = <uuid text>` is equal to `native_issue_id = <uuid>`, and a null `native_issue_id` makes the `coalesce` use the legacy context. - The candidate set itself is not bounded by this change. Acknowledged responses that never become final stay in the candidate set. Each candidate is now cheap, but the count still grows with that history. - `readGitHubResponseRun` in `response-run-state.ts` uses the same `coalesce` match. The sweep calls it once for each candidate row, and other GitHub paths also call it. Each call is a separate, smaller statement. This pull request does not change it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Used through Claude Code with tool use (shell, file edits, and local PostgreSQL benchmarks). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-10 18:25:59 -06:00

Author

SHA1

Message

Date

DottaandClaude Opus 5.5

81c52e3666

perf(chat): find terminal GitHub responses by issue run indexes (#20671 )

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The GitHub chat channel posts a terminal response when an agent run
for a GitHub request stops. A background sweep finds acknowledged
response comments (`processed`, not final) whose issue is blocked or
whose latest run is terminal
> - For each candidate, the sweep reads the status of the latest run of
the assigned agent for the issue. It matches runs with
`coalesce(native_issue_id::text, context_snapshot->>'issueId')`
> - No index has that expression. So PostgreSQL reads every run of the
agent for each candidate, and it decompresses the large legacy
`context_snapshot` of each run
> - On a hosted test database, this statement ran 13 times at a mean of
11 s in a 38-minute window (149 s in total), with single runs of 16 s
and 41 s
> - This pull request splits the run match into its two disjoint
branches. Each branch uses an existing issue index
> - The benefit is that each candidate reads a few index entries instead
of the agent's full run history. The returned rows do not change, and no
migration is necessary

## Linked Issues or Issue Description

**What existing behavior does this improve?**
The discovery page query in `registerGitHubTerminalResponseWork` in
`server/src/services/chat-channels/github/terminal-response-work.ts`.
The query runs when the sweep reconstructs its candidates, and once or
twice for each candidate check.

**Subsystem affected**
GitHub chat channel terminal-response recovery.

**Current behavior**
The `WHERE` clause has `issues.status = 'blocked' or (select r.status
from heartbeat_runs r where r.company_id = ... and r.agent_id = ... and
coalesce(r.native_issue_id::text, r.context_snapshot->>'issueId') =
<issue id> order by r.created_at desc, r.id desc limit 1) in (<terminal
statuses>)`. The planner can only use
`heartbeat_runs_company_agent_started_idx`. It reads all runs of the
agent, then evaluates the `coalesce` on each row. That decompresses the
context of each legacy run. The cost is the number of candidates times
the number of agent runs.

**Proposed behavior**
The subquery takes the latest run from two branches and then the latest
of the two:

- runs with `native_issue_id = <issue id>`, through
`heartbeat_runs_company_native_issue_contract_id_uq`
- runs with `native_issue_id is null and context_snapshot->>'issueId' =
<issue id>`, through `heartbeat_runs_company_ctx_issue_created_idx`

The two branches are disjoint, and together they are exactly the rows
where the `coalesce` is equal to the issue ID. Both branches and the
outer query use the same order (`created_at desc, id desc`). So the
selected run is the same, including ties.

**Reason and benefit**
This statement was the largest single-statement cost in the hosted
window after the earlier database fixes. A local reproduction is about
490x faster on the default fixture, and the gain grows with the agent's
run history.

**Breaking changes**
None. The API, the candidate rules and the results do not change.

## What Changed

- Rewrite the latest-run status subquery in the GitHub terminal-response
discovery page as two index-backed branches (native binding, and legacy
context without a native binding).
- Add two integration tests for the sweep. In the first, newer runs of
the same agent that do not bind the issue (one with a native binding to
another issue, but a legacy context that names this issue) must not hide
the stopped run. In the second, a newer active run with a native binding
to the issue keeps the request waiting.
- Add `scripts/benchmark-github-terminal-response-sweep.ts`. It captures
the SQL that the real worker generates at a baseline commit and at
`HEAD`, compares all rows, and measures both.

## Verification

Cause and thesis. The cost comes from the `coalesce` run match, which
has no index and makes each candidate scan the agent's run history.
Thesis: if each precedence branch uses its own issue index, the cost of
the page no longer depends on the agent's run count, and the rows stay
identical. The benchmark confirms the thesis.

Local SQL benchmark:

```sh
pnpm --filter @paperclipai/server exec tsx ../scripts/benchmark-github-terminal-response-sweep.ts --baseline-ref=7b33384446
```

Fixture (synthetic): two companies, each with one GitHub endpoint and
one assigned agent. Each issue has one GitHub response comment. The
first N comments are acknowledged and not final; the others are final.
Every tenth issue is blocked. Runs alternate between native binding and
legacy context, and each run has a 4 KiB incompressible context. The
fixture also has these edge cases: newer active runs with a native
binding to another issue but a legacy context that names this issue,
active runs that tie on `created_at` with the latest run (between native
and legacy runs), and newer active runs of another agent. The benchmark
compares every column of every candidate row (the page limit is lifted
for this check). Embedded PostgreSQL 18.1 on an Apple laptop. The hosted
database is Neon with different compute, so absolute times are local SQL
measurements only. Each size has 2 warmups and then alternates the
baseline and the treatment.

| Size | Runs (both companies) | Candidate rows equal | Baseline median
(range) | Treatment median (range) | Speedup |
| --- | --- | --- | --- | --- | --- |
| Small: 200 issues, 3 runs each, 20 candidates, 7 samples | 1,372 | 26
of 26 | 18.7 ms (17.7–20.8) | 3.8 ms (3.3–4.3) | 4.9x |
| Default: 1,500 issues, 8 runs each, 150 candidates, 7 samples | 25,300
(156 MB) | 200 of 200 | 3,697 ms (2,875–4,032) | 7.6 ms (6.6–9.6) | 490x
|
| Large: 3,000 issues, 10 runs each, 400 candidates, 5 samples | 62,600
(382 MB) | 538 of 538 | 57,055 ms (48,343–66,079) | 31 ms (16–79) |
1,800x |

Sizes are set with `BENCH_ISSUES`, `BENCH_RUNS_PER_ISSUE`,
`BENCH_CANDIDATES` and `BENCH_ITERATIONS`. The timed query is the real
first page (limit 100).

Plan (default size): the baseline runs `SubPlan` 270 times. Each run is
a `Bitmap Heap Scan` on `heartbeat_runs_company_agent_started_idx` that
removes about 12,000 rows by filter, for about 8.2 million buffer reads
in total. The treatment runs an `Append` of two `Limit` index scans
(`heartbeat_runs_company_native_issue_contract_id_uq` and
`heartbeat_runs_company_ctx_issue_created_idx`), for about 3,800 buffer
reads in total.

Tests:

- `cd server && npx vitest run
src/__tests__/chat-channels.integration.test.ts -t "terminal GitHub"`:
13 passed, including the 2 new tests.
- Mutation check: when the legacy branch does not exclude runs with a
native binding, the first new test fails. It passes with the fix.
- `cd server && npx tsc --noEmit`: passed.

## Risks

- Low risk. The change is one SQL subquery. The match is the same set of
runs: `native_issue_id::text = <uuid text>` is equal to `native_issue_id
= <uuid>`, and a null `native_issue_id` makes the `coalesce` use the
legacy context.
- The candidate set itself is not bounded by this change. Acknowledged
responses that never become final stay in the candidate set. Each
candidate is now cheap, but the count still grows with that history.
- `readGitHubResponseRun` in `response-run-state.ts` uses the same
`coalesce` match. The sweep calls it once for each candidate row, and
other GitHub paths also call it. Each call is a separate, smaller
statement. This pull request does not change it.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Used through Claude
Code with tool use (shell, file edits, and local PostgreSQL benchmarks).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

2026-11-10 18:25:59 -06:00