Commit Graph
8 Commits
Author SHA1 Message Date
DottaandClaude Opus 5.5 163d9f323c perf(github): reuse repository installation tokens until shortly before expiry (#21641)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The GitHub chat channel lets an agent review pull requests and
answer comments through a GitHub App
> - Each GitHub call gets a repository-scoped installation token from
`githubBotRepositoryToken`
> - That function minted a new token on every call. To mint, it resolved
every App secret through the secret service, and each resolution wrote
one audit row
> - The review-check head poll calls it about every 5 minutes for each
waiting check. On staging this made the secret audit insert one of the
busiest statements in the database
> - This pull request keeps a minted token in memory until shortly
before it expires, and checks the connection's current state on every
call
> - The benefit is far fewer database writes and reads, fewer GitHub
token mints, and an audit log that records real secret use

## Linked Issues or Issue Description

**What happened?**
On paperclip.staging (commit a18479bbba, measured at 19:05Z on
2026-11-14), `secret_access_events` got about 285 rows in 60 s. These
were 44 resolutions each of the GitHub App connection's
`credentials.appId`, `privateKey`, `installationId`, `webhookSecret`,
`clientId` and `clientSecret` (+21 `slug`), with `consumer_type`
`tool_connection` and actor `system`. That is about 4.7 audit inserts/s.
The matching `secret_versions` and binding reads ran at about 3.6/s each
in Neon. These were among the busiest statements on the database.

**Expected behavior**
A GitHub installation token is valid for one hour. Paperclip should
reuse it and resolve the App secrets only when it mints a new token.

**Steps to reproduce**
1. Connect a GitHub App chat endpoint and get pull request reviews into
the `received` review-check state (staging had 84 `github_review_check`
actions).
2. Let the review-check worker poll the pull request head (23 polls/min
measured on staging).
3. Count `secret_access_events` rows per minute. Each poll adds one row
per App secret and mints a new token at GitHub.

**Paperclip version or commit**
master at a18479bbba (the commit serving paperclip.staging at the time
of measurement).

**Where in the code**
`githubBotRepositoryToken` in
`server/src/services/chat-github-client.ts`. Callers:
`chat-github-checks.ts` (review-check head poll),
`chat-github-response-comments.ts` and `chat-github-reviews.ts`.

## What Changed

- `githubBotRepositoryToken` keeps a minted token in a per-process,
in-memory cache (at most 512 entries, least recently used first out). It
never persists or logs tokens.
- The cache uses the token until 5 minutes before GitHub's `expires_at`
(capped at one hour). If GitHub does not give a usable expiry, the token
is not kept.
- The cache key holds the company, endpoint, repository id, the exact
permission set, and an authority revision. The revision holds the
connection id, the endpoint's runtime generation (pause, revoke and
availability changes advance it), and for each mint credential its
secret ref, version selector, secret status, current version,
`updatedAt`, and binding. The installation id is a secret, so its secret
ref and version identify it in the key.
- Each call reads the current authority from the database in two narrow
statements: one row from the endpoint and connection join (status,
enabled, refs, runtime generation), and the three mint secrets with
their bindings (status and version columns only, no secret values). The
endpoint and connection status checks run on every call, as before.
- If a status check fails, all cached tokens for that endpoint are
dropped. A token cannot survive a revoke, pause, disable or credential
change.
- Secrets are resolved only when a token is minted, and only the three
that minting uses (`appId`, `privateKey`, `installationId`). So the
audit log gets three rows per mint instead of one row per App secret per
call.
- New `githubBotRepositoryClient` sends repository requests. If GitHub
returns 401 for a cached token, it drops that token, mints a new one,
and sends the request once more. A 401 for a newly minted token is not
retried. The review-check, response-comment and review paths use this
client.
- The token policy stays in the GitHub chat client module
(`rules/module-boundaries.mdc`). The callers only change how they get a
request function.

## Verification

- New test file
`server/src/__tests__/github-repository-token-cache.test.ts` (embedded
Postgres, real secret service, mocked GitHub fetch):
- `mints once and resolves the App's secrets only when it mints`: red on
master (second call minted `installation-token-2`), green now. It also
checks that only the three mint secrets are audited.
- `refreshes a token shortly before GitHub's expiry`: reused at +54 min,
minted again at +56 min.
  - `never keeps a token GitHub issues for less than the safety margin`
- `mints again after a credential rotation and uses the new credential`:
rotates the installation id secret and checks that the new mint uses it.
  - `mints again after the connection's credential refs change`
- `never serves a cached token across a pause, revoke or disabled
connection`: includes a pause and resume with no call in between.
  - `mints a separate token for each repository`
- `replaces a cached token GitHub rejects and retries the request once`:
also checks that a 401 for a new token is not retried.
- On master, 5 of the 8 tests fail. All 8 pass with this change.
- Focused suites pass locally: every server test file that matches
`github`, plus `chat-sdk-runtime.adapters`, `chat-provider-inventory`,
`idle-completed-history` and `connection-management.integration` (41 of
42 files, 955 of 956 tests). The one failure,
`github-connection-enrollment-work.test.ts`, also fails on unchanged
master (`expected +0 to be 1`), so this change did not cause it.
- `pnpm --filter @paperclipai/server typecheck` passes.
- CI supplies the full typecheck, build and test coverage for the final
revision.

## Risks

- Low risk. A cached token is used only while the endpoint, connection
and mint secrets are unchanged, and the status checks still read current
database state on every call.
- A GitHub-side revoke of a token (for example, an App uninstall) makes
GitHub return 401. The client then mints again once. If the installation
is gone, the mint fails as before.
- The cache is per process. Each server process mints its own token, at
most once per hour per repository and permission set.
- Two concurrent first calls for the same key can both mint. This is the
same as before and is rare.
- No schema, migration or API change.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edit, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 13:42:06 -06:00
DottaandClaude Opus 5.5 a7453eba60 perf(chat): back off GitHub review check head polls while the check cannot change (#21608)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The GitHub chat channel publishes a "Paperclip Review" check for
each admitted pull request head. A background queue
(`chat-github-review-checks`) keeps each unresolved check in step with
its task and run
> - While a check is queued or running, the queue polls GitHub for the
pull request head every 5 seconds. This finds a push whose webhook was
lost
> - The poll never stops while the check is unresolved. A check whose
run never comes, or whose run the check cannot find, stays queued
forever. It re-reads its whole state and calls GitHub every 5 seconds,
also when nothing changes
> - On a hosted test database with zero active runs, the five state
statements of this check ran 10.3 times a second each
> - This pull request doubles the head poll interval for each poll that
finds the same state, up to 5 minutes. A state change resets it to 5
seconds
> - The benefit is that an idle unresolved check costs about one poll
each 5 minutes, not one each 5 seconds. Local changes still publish at
once

## Linked Issues or Issue Description

Refs #17662 (added the 5-second head reconciliation for unresolved
checks). Refs #21580 (the same kind of fix for the GitHub response
sweeps).

**What happened?**

On a hosted test database (Neon, 488-second window, zero running or
queued runs, 2 runs created in 30 minutes), these five statements ran at
exactly 10.3 a second each: the `chat_endpoints` source read, the
`chat_github_reviews` read, the `chat_message_links` comment read, the
`heartbeat_runs` `wakeCommentId` lookup and the `chat_deliveries` state
read. All five come from `githubReviewCheckState`
(`server/src/services/chat-channels/github/review-check-state.ts`).

**Expected behavior**

A check whose state cannot change makes no recurring database reads or
GitHub calls until something that can change it happens: a local hint, a
lease expiry, or a saved deadline. The missed-webhook head check stays,
at a bounded cost.

**Steps to reproduce**

1. Admit a GitHub review check. Bind a task review to it, but give it no
run that the check can find (for example, the run was never created, or
its wake comment is not the delivery's comment).
2. Start the review check queue and leave the instance idle.
3. Count statements. The check is due again 5 seconds after each poll,
for ever. Each poll evaluates the check state about 3 to 4 times (before
dispatch, in dispatch, after dispatch, and in the rebuild that the
publication lease release starts). Each poll also asks GitHub for an
installation token and the pull request.

**Cause**

`dispatch` (`server/src/services/chat-github-checks.ts`) saved `retryAt
= now + 5s` after each poll. `entry()` in `review-check-work.ts` uses
that deadline when the state is unchanged. So an unresolved check was
due every 5 seconds without end.

## What Changed

- `chat-github-checks.ts`: save `unchangedPolls` in the check result. A
poll that finds the same state as the saved state adds one (up to 6). A
changed state, a first publication and a recovered failure set it to 0.
The next head poll is due after `min(5 minutes, 5 seconds ×
2^unchangedPolls)`.
- Export `GITHUB_REVIEW_CHECK_HEAD_POLL_MS` (5 seconds) and
`GITHUB_REVIEW_CHECK_MAX_HEAD_POLL_MS` (5 minutes).
- `chat-channels.integration.test.ts`: two regression tests. They use
the real review check queue, the real `dispatch`, embedded PostgreSQL, a
controlled clock and one pass each second. The fixture has the hosted
state: a review with no run, a delivery with a linked comment, and no
run for that comment.

## Verification

Reproduction with the real delivery coordinator, the real queue and
embedded PostgreSQL. One queued check with no findable run, 30 seconds
of real time, before this change:

| Company-wide hints | Statements a second | Check state reads a second
| GitHub pull reads a second |
| --- | --- | --- | --- |
| None | 8.4 | 0.70 | 0.17 |
| One each second (`notifyChatRunMilestones`) | 15.2 | 1.67 | 0.17 |

So one stuck check costs about 0.7 state reads a second from the head
poll alone. The hosted rate (10.3 a second) is consistent with about 15
such checks, or fewer checks with frequent company-wide hints.
Company-wide hints are a separate cause and a separate pull request.

Red/green regression test, 600 simulated seconds with one pass each
second:

- `cd server && npx vitest run
src/__tests__/chat-channels.integration.test.ts -t "unresolved GitHub
review check cannot change|publishes a local change at once"`
- Red (without the fix): `expected 120 to be less than or equal to 7`.
Over 600 seconds: 5,160 statements, 360 check state reads, 120 GitHub
pull reads.
- Green: 258 statements, 18 check state reads, 6 GitHub pull reads (8.6
→ 0.43 statements a second, a 95% drop). The saved interval stops at 5
minutes.
- Correctness (green): after the back-off, a restarted queue makes no
early GitHub call. A run bound to the review publishes `in_progress` at
the next pass and resets the interval. A pushed head is still found
within 5 minutes, the check is cancelled and the review is superseded.

Other checks:

- `cd server && npx vitest run
src/__tests__/chat-channels.integration.test.ts -t ithub`: 104 passed,
including all 12 review check queue tests.
- `cd server && npx vitest run
src/services/chat-github-review-repair.test.ts
src/__tests__/idle-completed-history.test.ts
src/__tests__/chat-admission-condition.test.ts
src/__tests__/app-private-hostname-gate.test.ts`: 109 passed.
- `cd server && node ../scripts/run-tsc.mjs` and `node
scripts/typecheck-layers.mjs` (with test layers): passed.
- `pnpm verify --explain` retains 2,718 test files conservatively, so I
did not run it locally. Required CI supplies that coverage.

## Risks

- A push whose webhook is lost is found later: after up to 5 minutes,
not 5 seconds, when the check has not changed for a while. The new
head's own webhook still creates its new check at once. Only the old
head's check stays queued or running on GitHub a little longer before it
is cancelled.
- Local changes are not delayed. `entry()` still makes a check due at
once when its state differs from the saved state, and run, review,
enqueue and lease hints still rebuild it.
- Saved actions from an older release have no `unchangedPolls`. They
start at 0. An invalid saved value also starts at 0. No migration.
- Restart recovery is unchanged: the deadline is saved on the action,
and startup rebuilds all unfinished checks.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Used through Claude
Code with tool use (shell, file edits, and local embedded PostgreSQL
reproductions).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 05:22:35 -06:00
DottaandPaperclip 8bc051245f fix(server): wake stale GitHub review repair from committed changes (#17664)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/215/changes/paperclip/215
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - GitHub checks and task admission can finish in either order.
> - A check can reject an old head before admission creates its review
row.
> - The chat maintenance loop repeatedly scans for that late review.
> - Existing cancellation and delivery receipts already identify the
repair.
> - This pull request wakes repair from committed changes and leaves
settled history quiet.

## Linked Issues or Issue Description

**What happened?**

GitHub chat maintenance repeatedly scans all review projections. Empty
instances and retained history incur database work even when no repair
is owed.

**What did you expect?**

A committed stale-head cancellation or admitted review should wake
repair. Only the exact delivery's unassessed queued/running review
should become superseded. When no repair remains, this duty should stop
making database calls.

**Steps to reproduce**

Configure a GitHub chat endpoint and leave it idle after all reviews
settle. The previous maintenance callback continues to call
repairSupersededReviews. A rapid push can also cancel a check before
admission inserts its review, which explains why recovery must remain.

Refs #17662, the merged review-check publication parent. Related #17661
and #17656 migrate terminal responses and acknowledgements. This slice
owns only late stale-head projection repair.

## What Changed

- Move the repair predicate and execution into the GitHub provider
owner. Match the original admitted delivery and cancellation receipt.
Recheck state and assessment in the update.
- Register a scoped worker with the existing coordinator. Use keyset
pages and fair bounded dispatch. Keep retries only for unresolved repair
or failed reconstruction.
- Signal before cancellation, automatic receipt insertion,
automatic-review binding, and check-driven review-state writes. Preserve
existing transaction ownership and outer commit behavior.
- Remove the periodic provider wrapper and publication-maintenance
callback. Preserve explicit manual recovery and independent
missed-webhook recovery.
- Add two concurrent, idempotent partial indexes. Use a guarded delivery
UUID lookup and a bounded exact-receipt lookup. Add no tables.
- Add ten real-database lifecycle and producer cases. Document the
producer audit, failure history, and remaining limits.

- Keep YAML browser reads scoped to the editor after focus changes, with
the original journey assertions and deadlines.

## Verification

Current source: `0886a0229b1c4f68d2f582da86aa1f248e9124f3`.

- At `dd6bca0485c3444d6e3d0ee10ec4aa651e801392`, **287 distinct cases
pass**: 274 complete GitHub workflow cases and 13 startup cases. The
GitHub command filters out 1,485 unrelated cases. The workflow suite
uses a new, pre-migrated PostgreSQL database.
- Coverage includes both orders of cancellation and admission, outer
commit and rollback, restart, drain, standby, failed-update retry,
concurrent assessment under an observed row lock, shutdown during that
wait, more than one candidate page, manual history, and settled-history
quiet.
- Database build, server typecheck/build, module boundaries, and target
migration order pass.
- Canonical migration `0403_natural_cerise.sql` preserves both original
index definitions and concurrent idempotent creation. All 401 parent
migration receipts remain unchanged during upgrade. One new receipt is
added. Replay adds none. Both indexes are ready and valid on fresh and
upgraded databases. The five newest snapshots remain.
- The normal parent merge preserved the prepared tree. The later browser
fixture repair changes only its central read helper and verification
note. Entire server, package, and UI trees remain identical to the
accepted source.
- [The owning
audit](https://github.com/paperclipai/paperclip/blob/0886a0229b1c4f68d2f582da86aa1f248e9124f3/doc/plans/2026-10-25-github-review-projection-sleep-audit.md)
records the exact evidence and earlier failures. Earlier query-plan
measurements retain their historical revision scope.
- Fresh current-head CI and Greptile review remain required. No deployed
sleep, full repository-suite, or fleet-savings claim is made.

Failure accounting: browser shard 7 failed twice on
`1453a4447c0d4d33f713ae3e00734c7950f95426` because the YAML helper read
whole-page clipboard text while main was focused. Three isolated
unchanged local passes did not resolve that failure. A controlled native
focus-loss probe reproduced the same error before the repair and passed
after it. The helper now clears stale clipboard data, focuses the
editor, and waits within the existing expectation budget for a fresh
YAML copy. All six complete YAML journeys pass with zero retries in 50.9
seconds. Journey assertions and deadlines are unchanged. The temporary
fault is removed. The exact CI focus-change trigger remains unknown.
[Both CI
attempts](https://github.com/paperclipai/paperclip/actions/runs/40777754306)
remain part of the verification record. Fresh new-head checks are
required.

Ponytail follow-up: the 4/5 review of `b3c81ba` was conditional on
fixing a stale startup-test slice marker, so the merge was held.
Boundary assertions reproduced the invalid -1 end index. The test now
uses the existing timer declaration and checks both bounds; all 13
startup/hostname cases pass. The optional no-op coordinator cleanup is
already in #17666. The six YAML journeys and all production code remain
unchanged. Fresh review and CI are required for this corrected head.

## Risks

- Missing a future producer signal could delay discovery until restart.
Supported writers share the current database owner's process-local
notification scope. Independent SQL writers and other server processes
need their own integration.
- The migration builds indexes concurrently. The existing migration
runner handles interruption and invalid index recovery.
- Direct GitHub missed-webhook discovery remains periodic. This change
does not make the whole chat service idle.
- Existing Cloud wake coverage and sleep blockers remain unchanged.
These tests do not prove deployed host or database suspension.
- Each remaining sleep slice needs its own review and CI checks before
merge.

## Model Used

OpenAI Codex (GPT-6), with reasoning and code execution. The exact
serving model ID and context-window size were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 18:42:54 -05:00
DottaandPaperclip 65fa1cd617 fix(server): schedule GitHub review checks from committed changes (#17662)
<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/213/changes/paperclip/213
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - GitHub review checks report the state of admitted agent work.
> - The chat maintenance loop searches for check work even when none
remains.
> - Existing action rows already record unfinished publication and retry
deadlines.
> - Committed changes can wake those rows through the shared
coordinator.
> - An unresolved check must still detect a remote head change when a
webhook is missed.
> - This pull request schedules only outstanding checks and leaves
settled history quiet.

## Linked Issues or Issue Description

**What happened?**

The GitHub chat maintenance callback repeatedly queries review-check
actions. Empty queues and completed history still incur database reads.
Local state changes wait for the maintenance callback.

**Expected behavior**

Committed check creation, review binding, run changes, and
publication-lease release wake the worker. Existing retry and
remote-head deadlines apply only to unfinished checks. A settled worker
makes no recurring database or provider calls.

**Steps to reproduce**

Configure a GitHub bot with no outstanding checks, or finish all of its
checks. Leave the instance idle. The old maintenance callback continues
to call the check service.

Refs #17661, the merged terminal-response parent. Related #17656
migrates response acknowledgements. Related #16385 changes branch-policy
admission; this PR preserves that separate policy boundary. Late
stale-head projection repair remains a separate follow-up slice.

## What Changed

- Move the existing check-state reader into the GitHub provider owner.
Preserve current authority, run selection, assessment ownership, and
check replacement behavior.
- Register a scoped worker with the existing coordinator and scheduler.
Existing received and retryable-failed actions remain durable truth. No
new table is added.
- Signal check enqueue, task review receipts, manual review
insertion/binding, run lifecycle changes, and publication-lease release
through existing transaction ownership.
- Preserve the saved failure backoff, eight-attempt policy, and
competing lease expiry. Keep five-second remote-head reconciliation only
while a received check remains unresolved.
- Check admission before each HTTP request. Shutdown leaves durable work
recoverable without spending a provider failure attempt.
- Remove check dispatch from the periodic GitHub callback. Keep the
separate late-projection repair and the manual combined recovery method
compatible.
- Add an idempotent concurrent partial index for unfinished check
selection. Retain the newest five generated snapshots.
- Add eleven real-database worker cases and startup wiring coverage.
Document producer coverage and the external reconciliation limit.

## Verification

Current reviewed source: `2bba08403b195bf646eda48992b46d6f5cebf756`.

- Normal merges preserve the complete review-check patch, the landed
terminal-response parent, and current master changes. No landed SQL
migration or journal entry changes.
- At `6995f84b240a14a7b8660e1cf095b3aa1e1e3168`, **331 distinct tests
pass**: 264 complete GitHub workflow cases, 30 completion cases, eight
issue-tree producer cases, 16 login-cleanup cases, and 13 startup cases.
The GitHub command filters out 1,485 unrelated cases. Login cleanup uses
its real embedded PostgreSQL fixture.
- Combined-master acceptance: **277 distinct tests pass** at
`9c538aed1251fd7f7cecb829a0ecbce6110fb1ed`: 264 complete GitHub workflow
cases and 13 startup cases. The GitHub command filters out 1,485
unrelated cases. A new PostgreSQL database receives all migrations
through 0402..
- Migration `0402_ancient_malice.sql` preserves the original partial
index definition and concurrent idempotent creation. All 400 prior
migration receipts stay unchanged during upgrade. One new receipt is
added. Replay adds no receipt. The index is ready and valid on fresh and
upgraded PostgreSQL databases. The five newest snapshots are retained.
- A rollback-only query probe adds 10,000 processed actions and one
pending action. The actual parameterized candidate predicate selects the
one pending row through the partial index, with two shared buffers and
0.007 ms execution time. This measures candidate discovery, not all
lease or provenance reads.
- Database build, server typecheck and build, module boundaries,
migration safety and migration order pass. Audit: [GitHub review-check
scheduling](https://github.com/paperclipai/paperclip/blob/2bba08403b195bf646eda48992b46d6f5cebf756/doc/plans/2026-10-25-github-review-check-sleep-audit.md).
- Fresh checks and Greptile review are required for this head. Prior
scores do not count. These checks do not prove deployed host suspension
or fleet savings.

## Risks

- Unresolved queued/running checks still make remote-head requests at
the existing five-second deadline. GitHub webhook receipt does not prove
future pushes cannot be missed. This PR does not claim active reviews
become network-idle.
- Run and configuration hints rebuild outstanding checks for the
affected company. Enqueue, review, and lease hints narrow to the
affected endpoint/repository/PR. Startup and scope reconstruction read
existing pending rows; they are not constant-cost operations.
- Delivery admission failures remain discoverable through the unresolved
check's external reconciliation deadline. Not every shared delivery
mutation gains a new immediate signal.
- Notifications use the existing process-local database ownership
contract. Independent SQL writers and other replicas need their own
supported integration.
- Provider markers, publication leases, and saved actions preserve
recovery after uncertainty. This change does not expand Cloud alarm
coverage or prove deployed host/database suspension.
- Each remaining sleep slice needs its own review and CI checks before
merge.

## Model Used

OpenAI Codex (GPT-6), with reasoning and code execution. The exact
serving model ID and context-window size were not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-28 17:08:27 -05:00
Devin FoleyandPaperclip a8e63f0e2a perf(server): avoid repeated delivery scans in GitHub review repair (#17669)
Resolve the exact delivery through its primary key before looking up stale-head cancellations. Backport the predicate from #17664 while preserving the current worker lifecycle and receipt checks.

Verified 372 selected tests, workspace typechecking, the server build, green CI, and an Apex 5/5 review.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-25 17:42:54 -07:00
Devin FoleyandPaperclip 35bd7f6059 Reduce encrypted event retries and GitHub run-query transfer (#17251)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Its Cloud connector delivers encrypted GitHub events to an instance.
> - One failed inner event currently rejects the whole authenticated
batch.
> - Healthy events then remain unacknowledged and are downloaded again.
> - GitHub response and check reconciliation also reads full run records
when it needs only ownership and status.
> - This pull request isolates inner failures and narrows those run
projections.
> - Valid events can finish while failed events remain available for
recovery.

## Linked Issues or Issue Description

**What happened?**

An inner App event encrypted with an unavailable key causes
`leaseEvents` to reject the complete batch. No sibling can finish or
acknowledge. Repeated retries download the same encrypted payloads.
Separately, GitHub response and check reads load prompts, runtime
profiles, results, and log excerpts that their callers do not use.

**Expected behavior**

Authenticate the outer batch and lease before handling individual
events. Process valid siblings independently. Retain failed events
without acknowledging them. Read only the current run fields needed for
response ownership and status checks.

**Steps to reproduce**

1. Seal one event for an old instance key and another for the current
key.
2. Put both events in a correctly authenticated outer lease.
3. Observe that the old client rejects the whole lease.
4. With this change, the valid event processes, the failed ID is
reported locally, and only completed events acknowledge.

No public issue exists. The change does not add a provider integration
or duplicate a roadmap feature.

## What Changed

- Keep outer authentication, environment checks, and lease binding
strict.
- Isolate inner decryption and route-binding failures. Return their
authenticated IDs only to the local poller and count them as failures.
- Leave failed events unacknowledged in the broker. Continue processing
and acknowledging healthy and duplicate siblings.
- Read only run identity, status, and required request context for
GitHub responses. Read only run ID and status for check reconciliation.
- Cover stale keys, swapped routes, invalid outer bindings, all-failed
batches, large unused run payloads, and cross-company/task refusals.

## Verification

- Connector cryptography suite: 47 tests passed.
- Final connector suite: 50 tests passed, including cryptography and
three real PostgreSQL connection-event tests. These cover acknowledgment
isolation, all-failed batches, and the exact safe warning fields.
- GitHub workflow regression suite after the master refresh: 307 tests
passed, including the large-payload/company/task test (1,084 unrelated
tests skipped by the selection).
- Server typecheck passed after the final source changes.
- `pnpm -r typecheck` and `pnpm build` passed after the latest
conflict-free master refresh.
- The broad local `pnpm test:run` was stopped when the branch was
refreshed; it has no complete passing result. All CI gates passed on the
final commit, including general and serialized server suites, runner
verification, build, typecheck, and end-to-end tests. Apex is 5/5 with
all review comments resolved. The last change removes source comments
and expands docs; both edited TypeScript files emit identical JavaScript
before and after that change.
- A read-only sample compared full run records with the new field sets:
response data decreased by about 96%, and check data by over 99%. This
is serialized query output, not a claim of realized billing savings or
CPU reduction.
- An initial database test setup failed because the local dependency
install omitted native-library symlinks. The package's standard setup
script repaired them. The unchanged test timeout then passed.
- Added lines checked for credentials, customer data, private links, and
internal issue references.

## Risks

Failed inner events remain pending and retry under the broker's existing
lease and retention policy. This change does not recover a lost original
private key or introduce a dead-letter deletion path. Outer
authentication failures still reject the complete batch.

Run projections must retain request provenance and task binding fields.
Existing predicates, ordering, identity validation, and publication
authority remain unchanged. The change has no schema migration and adds
no background timer or new provider call.

## Model Used

OpenAI GPT-6 through Codex, with reasoning and shell/GitHub CLI tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used with version and capability
details
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs
- [x] I have described the issue in-PR following the bug report template
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket ID
- [x] Focused local regressions and all CI gates pass
- [x] I have added and updated tests
- [x] I have updated relevant documentation
- [x] I have documented risks
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-22 17:40:29 -07:00
DottaandPaperclip a8b652b085 fix(github): preserve review delivery through reconnect (#16259)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - GitHub review bots turn authorized repository events into agent
tasks and publish the result on the pull request.
> - Cloud Apps send callbacks through a relay, but the direct callback
recovery worker reported that URL as broken.
> - Reconnecting the same App then prevented a new authorized
publication from updating its existing working comment.
> - Report-only runs could also produce an action-required check when no
complete assessment was available.
> - This pull request preserves comment identity across reconnects,
verifies fresh signed delivery after repair, and keeps report-only
results neutral.
> - Users can receive the review without changing repository access or
making an advisory bot block their pull request.

## Linked Issues or Issue Description

**What happened?**

A review could be saved in Paperclip while GitHub kept showing “Working
on this…”. A same-App reconnect changed the runtime generation and the
existing response receipt then failed with “GitHub response binding
changed”. Cloud callback recovery also compared the relay URL with a
direct tenant URL. An incomplete or failed report-only run could produce
an action-required check.

**Expected behavior**

A fresh authorized attempt updates the same App-owned comment after
reconnect. Stale attempts and revoked people remain denied. Cloud relay
delivery does not produce a direct-callback mismatch. Report-only checks
remain neutral and explain incomplete work honestly.

**Steps to reproduce**

1. Accept a GitHub request and publish its working comment.
2. Reconnect the same App before the agent publishes its result.
3. Submit the result and inspect the working comment and check.
4. Separately, let a report-only run end without an assessment and
inspect its check conclusion.

Related: #16138 addresses the separate classification of local
authorization errors as provider credential failures. This change fixes
the response binding and report-only paths themselves.

## What Changed

- Bind durable response comments to the App and agent. Keep per-attempt
runtime generation checks and provider comment ownership checks,
including legacy receipts.
- Exclude Cloud-managed Apps from the direct webhook recovery scan and
its stale callback-mismatch warning. The Cloud inbox owns relay retries.
- Let a fresh authenticated event from the bound installation verify
webhook delivery after repair. Keep signature, runtime epoch,
installation, and credential lease checks.
- Keep report-only assessment and terminal-run checks neutral. Preserve
incomplete/error text and gated-review behavior.
- Add integration regressions for reconnect races, legacy comments,
failed runs, Cloud recovery, and foreign or forged webhook traffic.
Update connector documentation.
- Correct the assigned-tool search guidance to name the actual input
fields (`offset` and `schemaOffset`) and show valid arguments. The
failed automatic review had an input-schema rejection before tool
execution; this clarifies recovery without expanding tool access.

## Verification

- `pnpm -r typecheck` passed.
- GitHub policy and integration suite: 291 tests passed (`vitest run
server/src/services/chat-github-review-policy.test.ts
server/src/__tests__/chat-channels.integration.test.ts -t
'GitHub|github'`).
- The report-only policy regression failed before the fix and passed
after it.
- `pnpm build` passed.
- Assigned tool discovery: 24 tests passed. Its guidance now maps
returned pagination offsets to the correct input fields and tells the
model to omit unused optional fields.
- Full `pnpm test:run` is running. It encountered a 15-second timeout in
the unrelated authenticated Workspace calendar-write test. That test
passed on one isolated rerun.
- After the review-requested module extraction, server typecheck and
both direct/Cloud recovery activity integration checks passed.
- All 55 non-skipped CI checks passed for
`32e690279d847e44a6946b6c7cfb110fb11aff79`. Greptile scored that commit
5/5 with no unresolved findings. Live staging publication verification
follows deployment.

## Risks

- A durable response can now survive a same-App reconnect. Fresh
authorization, runtime generation checks for each publication attempt,
App ownership, and the publication lease still fence writes.
- Report-only failures intentionally become neutral. The check still
reports the failure; it does not claim a complete assessment.
- Cloud-managed callbacks rely on their existing durable relay inbox.
The direct callback recovery worker remains unchanged for direct
installations.
- No schema migration or permission expansion.

## Model Used

OpenAI Codex (GPT-6), with code inspection, editing, test execution, and
browser verification. The exact deployment model ID and context window
are not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-13 22:24:41 -05:00
DottaandPaperclip 8813a50105 feat: run GitHub review bots through Paperclip agents (#13717)
## Thinking Path

> - Paperclip manages agent work as tasks and runs.
> - GitHub chat brings repository conversations into those tasks.
> - A review bot needs the assigned agent, its authority, and governed
provider tools.
> - The existing channel connection did not supply that review workflow
or a complete setup journey.
> - This pull request adds GitHub App setup, account access, event
prompts, task-bound review tools, and exact-commit checks.
> - Operators can inspect each review through the same task, run, and
activity systems.

## Linked Issues or Issue Description

**Subsystem affected**

GitHub chat, governed connection tools, task execution, shared/database
contracts, and connector setup UI.

**Problem or motivation**

Operators need a GitHub review bot that runs their assigned Paperclip
agent. Mentions and PR events must preserve task ownership and requester
authority. Provider publication must use the bot App identity and
enforce the configured permissions.

**Proposed solution**

Extend the existing GitHub chat connector with resumable App onboarding,
linked-member and sponsored-guest access, editable event prompts, and
governed review operations. Validate structured assessments on the
server and compute a stable Paperclip Review check for the exact head
commit.

**Alternatives considered**

A separate review scheduler would duplicate Paperclip execution and
permissions. Reusing personal GitHub credentials would change the bot
identity and credential boundary.

**Roadmap alignment**

This extends the existing Connected Apps and governed-tool
infrastructure. The project owner requested and approved this design.
Related PR #8645 imports external Codex review feedback; this change
runs an assigned Paperclip agent and publishes its results through the
existing chat connector.

## What Changed

- Include the current Paperclip instance origin in the copied setup
prompt. Storybook uses its configured Paperclip origin; callback
parameters and URL credentials are excluded.

- Add a Claude/Codex copy button in the real setup and Storybook opening
step. Its detailed prompt asks four setup questions and guides
embedded-browser setup, verification, and optional required checks.
Clipboard failure exposes selectable instructions.
- Add a tutorial that explains why App installation, review scheduling,
and required checks are separate choices.

- Add manifest registration, an existing-App path, separate installation
and repository selection, repository refresh, and explicit account
confirmation.
- Add low-trust agent guidance, effective capability verification,
member selection, and explicit restricted guests with a sponsor.
- Add configurable PR events, prompts, repository overrides, rating
thresholds, and separate formal-review permissions.
- Give the assigned agent governed App tools to read PRs, comment, begin
an assessment, submit findings, and optionally submit a formal review.
- Bind review history, root PR events, and inline replies to ordinary
tasks. Deduplicate deliveries/findings and reject stale publication.
- Link check Details to the underlying task on the current trusted
hostname, or to Reviews before task creation.
- Add schema migration 0283, API contracts, production UI, and 49
interactive Storybook states.
- Repair local lease recovery. Keep the Cloud Dockerfile identical to
master; no provider-pack layer or runtime-default environment variable
is added.
- Retry only rolled-back wake-admission transactions after transient
endpoint-lock contention. A deterministic held-lock regression proves
one accepted wake.

## Verification

- Current head: `7ba761fe007bb798400d3e62346fa964f607f0f8`, rebased on
master `d9b3a5653e41f2ee5a1345b97c86a238f7a5c8e9`. Dockerfile has zero
diff against master. Final workspace typecheck and build passed. The new
PostgreSQL migration regression passed and preserves existing relation
and constraint identities after replay.
- Greptile reviewed this exact head at 5/5. There are zero unresolved
review threads and no merge conflicts.
- All current-head checks are green: 54 passed and two conditional
Storybook jobs skipped. This includes complete server/workspace test
suites, build, typechecks, policy checks, Runner suites, browser suites,
and security status. One timing-sensitive callback-ordering test passed
in isolation and its CI shard passed one retry. The duplicate local
full-suite run was stopped after CI completed; it is not counted as a
local full-suite pass.
- Before the final Slack rebase and migration renumbering, 186 focused
GitHub tests, 14 native bootstrap cases, token gates, and Storybook
build passed. The final rebase retained the new Slack communication
guidance.
- The embedded-browser setup test copied the full detailed prompt,
including the configured Paperclip instance URL. Desktop and narrow
layouts were checked. Component tests cover successful copying and
clipboard failure with selectable text and retry.
- Live local and hosted GitHub acceptance evidence refers to application
revision `cb703ac959876a07ebf3d7a295847f9f351eb6fc`. Real agent tasks
exercised issue mentions, automatic PR reviews, inline findings,
repeated mentions, task continuation, and failing-to-passing checks
after a push. The Storybook agent generated, built, and browser-rendered
pages; missing acceptance text failed, matching text passed, and broken
JSX produced an incomplete result.
- Live cases also covered independently disabled push events, prompt
injection, duplicate signed deliveries, rapid pushes, stale-result
rejection, finding deduplication, and restart recovery. Formal reviews
were denied while disabled and published only after explicit enablement.
Check Details links pointed to the underlying task on the trusted
hostname.
- Those hosted native Claude runs used the provider-pack layer now
removed from this PR. They do not prove native Claude works on the
standard Cloud image. A replacement hosted native Codex run is not yet
verified: the disposable QA tenant has only an Anthropic AI connection.
No new staging or production deployment was made for the packaging
removal.
- Required-check merge enforcement could not be tested because the
private disposable repository's GitHub plan rejected the rules
configuration. Published success/failure/incomplete check states were
verified directly.

## Risks

- Latest master allocated migration 0282 to Slack. The GitHub migration
is regenerated as 0283 with replay-safe table/index/constraint creation;
a PostgreSQL regression verifies existing relations and constraints are
preserved. Existing preview tenants remain subject to the fleet
migration-history compatibility preflight; no bypass is introduced.

- Migration 0283 adds company-scoped configuration, registration,
review, and publication records. Existing connections retain their
behavior until reviews/tools are enabled.
- Signed webhooks and expiring registration state remain required.
Hosted installations also need the companion narrow Cloud gateway
exemptions.
- Agent assessments can be incomplete or wrong. The server enforces
coverage/result structure, current-head publication, rating policy, and
separate formal-review permission; it does not replace code-review
judgment.
- No Cloud image packaging changes are included. Remote native
ACPX/Claude and OpenCode retain their existing operator-supplied
provider-pack prerequisite. Native Codex and Codex with managed MCP
tools do not require that pack. Earlier staging deployment evidence
refers to its stated revision, not this packaging-removal head.
Production rollout and merging remain outside this change.

## Model Used

OpenAI GPT-6 through Codex, with repository, code execution, API, and
embedded-browser tools. The exact serving model ID and context-window
size were not exposed by the environment.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-09-21 14:41:19 -05:00