mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-17 04:21:24 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The GitHub chat channel has two background sweeps for request response comments: acknowledgements, and terminal responses. Each sweep rebuilds its candidate set when a hint arrives, and it reads the latest run of each candidate request (`readGitHubResponseRun`) on each rebuild > - A run milestone for a run with no task sends a company-wide hint to both sweeps. Examples: each status change of a run without a task (`setRunStatus`), and a comment merged into an active run in the wake queue. In a busy company these hints arrive more than once a second > - So both sweeps rebuild the whole company about once a second (their minimum pass interval). Each rebuild reads every parked request again, although nothing about those requests changed. On a hosted test database, `readGitHubResponseRun` ran 6.7 times a second > - This pull request rebuilds a GitHub response sweep for a company-wide hint at most once every 30 seconds. Issue, request, conversation, endpoint and lease hints still rebuild immediately. `external-objects.ts` uses the same rule for its refresh sweep > - The benefit is that a steady stream of company-wide hints no longer makes each sweep read every parked request every second ## Linked Issues or Issue Description Refs #21577 (makes each `readGitHubResponseRun` call cheap; this pull request reduces how often the sweeps call it). Refs #18384 (the same reconstruction interval for company-wide external-object hints). **What existing behavior does this improve?** The candidate rebuild of `registerGitHubAcknowledgementWork` (`acknowledgement-work.ts`) and `registerGitHubTerminalResponseWork` (`terminal-response-work.ts`) for company-wide hints. **Subsystem affected** GitHub chat channel: request acknowledgement and terminal-response recovery sweeps. **Current behavior** `notifyChatRunMilestones` calls `notifyChatResponseAvailability` with its scope. When the scope has no issue (only a company, or a company and a run), the hint for both GitHub response queues is company-wide (`{"companyId": ...}`), because the response scope has no run field. Each company-wide hint makes each sweep load every candidate of the company and call `readGitHubResponseRun` for each one. Passes are limited to one a second, so with a steady hint stream the cost is about (parked candidates) reads a second. **Proposed behavior** Both sweeps set `reconstructionIntervalMs`. A company-wide scope (company ID only) waits 30 seconds after its last rebuild before it rebuilds again. All narrower scopes, the startup scope (`*`), and saved deadlines are not changed. Entries that are already loaded stay available for dispatch while a company-wide hint waits, and each check still reads the request again from the database. **Reason and benefit** On the hosted test database, `readGitHubResponseRun` ran 6.7 times a second for no change in state. In a local reproduction, the call rate drops from about one call for each parked request each second to one call for each parked request each 30 seconds. **Breaking changes** None. ## What Changed - Add `response-sweep-scope.ts` with the shared scope type of the two GitHub response sweeps and `githubResponseReconstructionIntervalMs`: 30 seconds for a company-wide scope, 0 for all other scopes. - Use it as `reconstructionIntervalMs` in the acknowledgement sweep and the terminal-response sweep. Both files now import the shared scope type instead of declaring it. - Add `response-sweep-scope.test.ts`. It registers both real sweeps on a real database with 6 parked requests (3 for each sweep) and a controlled clock. It sends company-wide hints and issue hints, and counts `readGitHubResponseRun` statements on each pass. ## Verification Cause, from a local reproduction with the real delivery coordinator and both real sweeps on embedded PostgreSQL 18.1, with 8 parked requests (4 for each sweep), each with a succeeded native run that waits for its presentation: | Hints | Hints a second | `readGitHubResponseRun` calls a second | | --- | --- | --- | | None | 0 | 0 | | Company-wide run milestone (`{ companyId, runId }`) | 9.7 | 8.75 | | Company-wide run milestone | 1.0 | 7.98 | | Run milestone for an unrelated issue | 9.7 | 0 | So the call rate is about one call for each parked request each second, from company-wide hints only. The hosted rate (6.7 a second) is consistent with about 7 parked GitHub requests in one company. Red/green regression test: - `cd server && npx vitest run src/services/chat-channels/github/response-sweep-scope.test.ts` - Red (without the two `reconstructionIntervalMs` lines): `expected [ 6, 6, 6, 6, 6 ] to deeply equal [ 6, +0, +0, +0, +0 ]`. Each of 5 company-wide hints rebuilt all 6 parked requests. - Green: 2 of 2 pass. The first company-wide hint rebuilds once (6 reads). The next 4 hints read nothing. After 30 seconds, the waiting hint rebuilds (6 reads). Issue hints still read exactly the one request of that issue at once, and a hint for another company reads nothing. No acknowledgement is sent for a parked request. Other checks: - `cd server && npx vitest run src/__tests__/chat-channels.integration.test.ts -t ithub`: 104 passed (all GitHub chat-channel integration tests, including the acknowledgement and terminal-response sweeps). - `cd server && npx tsc --noEmit`: passed. - After the rebase onto #21577: both new test files pass (4 of 4), and the server typecheck passes. - `pnpm verify` selected most of the server suite, because the sweep modules are imported widely. I stopped that broad local run. Required CI supplies that coverage. ## Risks - A change that only a company-wide hint reports can take up to 30 seconds to rebuild a GitHub response sweep. I checked the callers. Changes to one request use issue, delivery, conversation or lease hints: run status changes with a task, native presentation commits (`status-decision-committer.ts`, `native-run-finalizer.ts`, `native-chat-review-presentation.ts`), publications, fast responses, and the publication lease. Those stay immediate. Company-wide hints come from runs with no task, from a comment merged into an active run (the wake queue sends only the run ID), and from company authority changes. A run with no task cannot change the latest run of a GitHub request's task. For a merged comment, the active run's next status change sends an issue hint, and staging the request's response sends a delivery hint; both rebuild at once. Only a mid-run ownership update can wait up to 30 seconds. For an authority change, each check reads the request and its authority again before it acts. - The deadline reconciliation snapshot reports these two sweeps as not reconciled while a company-wide hint waits (up to 30 seconds). This is the same as the external-object refresh sweep. - The startup rebuild (`*`) and failure retries do not change. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Used through Claude Code with tool use (shell, file edits, and local PostgreSQL reproductions). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>