Files
PaperClipAI/server
DottaandClaude Opus 5.5 a1c37b5cc4 perf(chat): rebuild GitHub response sweeps for company-wide hints at most every 30 seconds (#21580)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The GitHub chat channel has two background sweeps for request
response comments: acknowledgements, and terminal responses. Each sweep
rebuilds its candidate set when a hint arrives, and it reads the latest
run of each candidate request (`readGitHubResponseRun`) on each rebuild
> - A run milestone for a run with no task sends a company-wide hint to
both sweeps. Examples: each status change of a run without a task
(`setRunStatus`), and a comment merged into an active run in the wake
queue. In a busy company these hints arrive more than once a second
> - So both sweeps rebuild the whole company about once a second (their
minimum pass interval). Each rebuild reads every parked request again,
although nothing about those requests changed. On a hosted test
database, `readGitHubResponseRun` ran 6.7 times a second
> - This pull request rebuilds a GitHub response sweep for a
company-wide hint at most once every 30 seconds. Issue, request,
conversation, endpoint and lease hints still rebuild immediately.
`external-objects.ts` uses the same rule for its refresh sweep
> - The benefit is that a steady stream of company-wide hints no longer
makes each sweep read every parked request every second

## Linked Issues or Issue Description

Refs #21577 (makes each `readGitHubResponseRun` call cheap; this pull
request reduces how often the sweeps call it). Refs #18384 (the same
reconstruction interval for company-wide external-object hints).

**What existing behavior does this improve?**
The candidate rebuild of `registerGitHubAcknowledgementWork`
(`acknowledgement-work.ts`) and `registerGitHubTerminalResponseWork`
(`terminal-response-work.ts`) for company-wide hints.

**Subsystem affected**
GitHub chat channel: request acknowledgement and terminal-response
recovery sweeps.

**Current behavior**
`notifyChatRunMilestones` calls `notifyChatResponseAvailability` with
its scope. When the scope has no issue (only a company, or a company and
a run), the hint for both GitHub response queues is company-wide
(`{"companyId": ...}`), because the response scope has no run field.
Each company-wide hint makes each sweep load every candidate of the
company and call `readGitHubResponseRun` for each one. Passes are
limited to one a second, so with a steady hint stream the cost is about
(parked candidates) reads a second.

**Proposed behavior**
Both sweeps set `reconstructionIntervalMs`. A company-wide scope
(company ID only) waits 30 seconds after its last rebuild before it
rebuilds again. All narrower scopes, the startup scope (`*`), and saved
deadlines are not changed. Entries that are already loaded stay
available for dispatch while a company-wide hint waits, and each check
still reads the request again from the database.

**Reason and benefit**
On the hosted test database, `readGitHubResponseRun` ran 6.7 times a
second for no change in state. In a local reproduction, the call rate
drops from about one call for each parked request each second to one
call for each parked request each 30 seconds.

**Breaking changes**
None.

## What Changed

- Add `response-sweep-scope.ts` with the shared scope type of the two
GitHub response sweeps and `githubResponseReconstructionIntervalMs`: 30
seconds for a company-wide scope, 0 for all other scopes.
- Use it as `reconstructionIntervalMs` in the acknowledgement sweep and
the terminal-response sweep. Both files now import the shared scope type
instead of declaring it.
- Add `response-sweep-scope.test.ts`. It registers both real sweeps on a
real database with 6 parked requests (3 for each sweep) and a controlled
clock. It sends company-wide hints and issue hints, and counts
`readGitHubResponseRun` statements on each pass.

## Verification

Cause, from a local reproduction with the real delivery coordinator and
both real sweeps on embedded PostgreSQL 18.1, with 8 parked requests (4
for each sweep), each with a succeeded native run that waits for its
presentation:

| Hints | Hints a second | `readGitHubResponseRun` calls a second |
| --- | --- | --- |
| None | 0 | 0 |
| Company-wide run milestone (`{ companyId, runId }`) | 9.7 | 8.75 |
| Company-wide run milestone | 1.0 | 7.98 |
| Run milestone for an unrelated issue | 9.7 | 0 |

So the call rate is about one call for each parked request each second,
from company-wide hints only. The hosted rate (6.7 a second) is
consistent with about 7 parked GitHub requests in one company.

Red/green regression test:

- `cd server && npx vitest run
src/services/chat-channels/github/response-sweep-scope.test.ts`
- Red (without the two `reconstructionIntervalMs` lines): `expected [ 6,
6, 6, 6, 6 ] to deeply equal [ 6, +0, +0, +0, +0 ]`. Each of 5
company-wide hints rebuilt all 6 parked requests.
- Green: 2 of 2 pass. The first company-wide hint rebuilds once (6
reads). The next 4 hints read nothing. After 30 seconds, the waiting
hint rebuilds (6 reads). Issue hints still read exactly the one request
of that issue at once, and a hint for another company reads nothing. No
acknowledgement is sent for a parked request.

Other checks:

- `cd server && npx vitest run
src/__tests__/chat-channels.integration.test.ts -t ithub`: 104 passed
(all GitHub chat-channel integration tests, including the
acknowledgement and terminal-response sweeps).
- `cd server && npx tsc --noEmit`: passed.
- After the rebase onto #21577: both new test files pass (4 of 4), and
the server typecheck passes.
- `pnpm verify` selected most of the server suite, because the sweep
modules are imported widely. I stopped that broad local run. Required CI
supplies that coverage.

## Risks

- A change that only a company-wide hint reports can take up to 30
seconds to rebuild a GitHub response sweep. I checked the callers.
Changes to one request use issue, delivery, conversation or lease hints:
run status changes with a task, native presentation commits
(`status-decision-committer.ts`, `native-run-finalizer.ts`,
`native-chat-review-presentation.ts`), publications, fast responses, and
the publication lease. Those stay immediate. Company-wide hints come
from runs with no task, from a comment merged into an active run (the
wake queue sends only the run ID), and from company authority changes. A
run with no task cannot change the latest run of a GitHub request's
task. For a merged comment, the active run's next status change sends an
issue hint, and staging the request's response sends a delivery hint;
both rebuild at once. Only a mid-run ownership update can wait up to 30
seconds. For an authority change, each check reads the request and its
authority again before it acts.
- The deadline reconciliation snapshot reports these two sweeps as not
reconciled while a company-wide hint waits (up to 30 seconds). This is
the same as the external-object refresh sweep.
- The startup rebuild (`*`) and failure retries do not change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Used through Claude
Code with tool use (shell, file edits, and local PostgreSQL
reproductions).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 02:05:45 -06:00
..