mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 14:19:00 +01:00
DottaandClaude Opus 5.5
2e7c66aac9
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Its GitHub Actions CI shares the `ubuntu-latest` runner pool across all workflows > - #19480 added the `PR merge review notification` workflow. It triggers on `workflow_run`, on every completed `check_run`, and on every commit `status` > - One PR push completes about 60 checks, so it queues about 60 notifier jobs. Each job takes a runner > - The Actions queue now holds about 340 queued runs. About 260 of them are this notifier. Other `ubuntu-latest` jobs wait more than one hour (for example, the `authorize` job of Runner Full-Stack E2E image builds) > - This pull request skips events that cannot complete a green PR state before a runner is allocated, and it collapses the remaining events to the newest one per PR head > - The benefit is that this unjams the Actions queue and keeps the advisory review notification ## Linked Issues or Issue Description Refs #19480 **What happened?** The `PR merge review notification` workflow from #19480 schedules one runner job for each `workflow_run`, `check_run`, and `status` event. One PR push makes about 60+ jobs. The repository Actions queue has about 340 queued runs (about 260 from this notifier at the time of writing). Other `ubuntu-latest` jobs wait more than one hour. **What did you expect?** The notifier requests one advisory merge review when PR verification is complete. It must not use a runner for each intermediate check. **Steps to reproduce** 1. Push to any PR. 2. Run `gh run list --workflow pr-merge-review.yml -L 200 --json event,status`. 3. See tens of queued `check_run`, `status`, and `workflow_run` runs for the same push. ## What Changed - `.github/workflows/pr-merge-review.yml`: the job `if:` now schedules the job only for events that can complete a green PR state: - `workflow_run` from `pull_request` or `pull_request_target` with conclusion `success`, `skipped`, or `neutral`. - `check_run` from a non-Actions app (app id is not `15368`) with conclusion `success`, `skipped`, or `neutral`. GitHub Actions check runs are already covered by the `workflow_run` list. - `status` with state `success`. - GitHub cannot filter `check_run` or `status` at the trigger level. A skipped job makes a run record but does not take a runner. - The job has a concurrency group `pr-merge-review-<PR head SHA>` with `cancel-in-progress: true`. The notifier reads the current PR state, so the newest event subsumes older ones. The group is on the job, not the workflow, so a skipped event cannot cancel a scheduled notifier. - The environment, permissions, trusted default-branch checkout, and secret handling do not change. - `.github/scripts/tests/notify-pr-merge-review.test.mjs`: a new test pins the trigger kinds, the job condition, and the job-level concurrency group. - `doc/PR-MERGE-REVIEW-WEBHOOK.md`: describes the event filter and the concurrency group. **Expected runner jobs per PR push** (typical PR: `PR`, `Storybook Visual`, `commitperclip PR Review` workflows; Greptile, Paperclip Review, Socket (2), Superagent (2) external checks; one Snyk status): | | Before | After | | --- | --- | --- | | Eligible events that take a runner | about 60–70 (every check run + 2 statuses + 3 workflow runs) | at most about 10 (3 workflow runs + 6 external checks + 1 status, only when green) | | Jobs that run to completion | about 60–70 | usually 1–4: events that arrive while a job is pending or running cancel it, so a burst collapses to the newest event. In a backed-up queue, all events for one head collapse to one job | ## Verification - `node --test .github/scripts/tests/notify-pr-merge-review.test.mjs`: 22 tests pass. - The new workflow test fails on the old workflow (red) and passes on the new workflow (green). - `actionlint .github/workflows/pr-merge-review.yml`: no findings. - I confirmed with `gh run list` that `workflow_run.head_sha` for the `pull_request_target` workflow (`commitperclip PR Review`) is the PR head SHA. So one SHA key groups all event kinds for one PR head. - I confirmed that all workflows that make PR checks are in the `workflow_run` list (`CI routing` and `Trusted PR CI` are reusable workflows called by `PR`). - After merge: check that `gh run list --workflow pr-merge-review.yml` shows mostly `skipped` runs for each push, and a small number of runs that execute. ## Risks - A check provider that reports a final green state through a GitHub Actions check run from a workflow that is not in the `workflow_run` list does not trigger the notifier. All current PR workflows are in the list. The doc already tells authors to add new PR workflows to the list. - A failed or cancelled check that later succeeds on re-run still triggers the notifier, because the re-run completes with `success`. - `cancel-in-progress` can cancel a notifier during delivery. The newer job evaluates the same or a more complete state, and the Paperclip idempotency key deduplicates deliveries for the same PR, head, and base. - Each event still makes a workflow run record. Skipped runs do not take a runner. - Low risk otherwise: the notifier is advisory and `continue-on-error`. It does not affect PR verification results. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, GitHub CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
95 lines
5.4 KiB
Markdown
95 lines
5.4 KiB
Markdown
# Advisory PR review notifications
|
|
|
|
The optional `PR merge review notification` workflow sends a private webhook
|
|
after a current, open, non-draft PR targeting `master` passes verification.
|
|
It does not approve or merge a PR. Webhook failures do not change PR CI results.
|
|
|
|
## Private configuration
|
|
|
|
Create a GitHub Actions environment named `pr-merge-review`. Restrict its
|
|
deployment branch policy to the `master` branch only, with no tag policy.
|
|
Keep these two values in **environment secrets**, not repository variables,
|
|
repository secrets, committed files, task comments, or workflow outputs:
|
|
|
|
- `PAPERCLIP_PR_REVIEW_WEBHOOK_URL`: a Paperclip routine webhook URL.
|
|
- `PAPERCLIP_PR_REVIEW_WEBHOOK_TOKEN`: its separate bearer signing secret.
|
|
|
|
Use a webhook trigger with `signingMode: bearer`. The routine should use
|
|
`always_enqueue` concurrency so separate PRs cannot be coalesced into one task.
|
|
Define required text variables `pr_url`, `pr_number`, `head_sha`, and `base_sha`.
|
|
The notifier sends only those identifiers. The receiving agent must treat them
|
|
as untrusted data and verify the repository, PR identity, and current revision.
|
|
|
|
Give the routine instructions to run `pr-merge-review` and publish its MERGE,
|
|
HOLD, or DO NOT MERGE recommendation on the routine execution's Paperclip task.
|
|
Require a fresh check of the head, base, reviews, required checks, and task gates.
|
|
Keep GitHub approval, merge, auto-merge, and public-comment actions disabled in
|
|
the instructions for this advisory phase. Configure the routine in Paperclip;
|
|
do not put instance-specific URLs or credentials in this repository.
|
|
|
|
## Delivery and verification
|
|
|
|
The notification workflow runs trusted default-branch code. It never checks out
|
|
PR code or downloads PR artifacts or caches. It reacts to the repository's PR
|
|
workflow completions, external check completions, and legacy commit statuses.
|
|
When adding another PR verification workflow, add its name to `workflow_run`.
|
|
|
|
GitHub cannot filter `check_run` or `status` triggers, so the job condition
|
|
skips events that cannot complete a green state before a runner is allocated.
|
|
Only successful, skipped, or neutral PR workflow runs and non-Actions check runs,
|
|
and successful commit statuses, schedule the job. GitHub Actions check runs are
|
|
skipped because their workflow completions already arrive through `workflow_run`.
|
|
A job-level concurrency group keyed by the PR head SHA cancels older pending or
|
|
running notifiers, because the newest event sees the most complete PR state.
|
|
The group is on the job so skipped events cannot cancel a scheduled notifier.
|
|
|
|
The separate `PR review API access` workflow checks notifier changes using
|
|
the real read-only Actions token. It has no private environment or webhook
|
|
credentials and sends no webhook. Both workflows explicitly grant
|
|
`actions`, `checks`, `contents`, `pull-requests`, and `statuses` read access.
|
|
The GraphQL status rollup needs both check and commit-status permissions.
|
|
|
|
The notifier reads the current PR's status rollup and branch rules. Both
|
|
`ci / verify` and `ci / e2e`, plus any additional required checks, must succeed.
|
|
Other checks must succeed or finish as skipped or neutral. Optional skipped or
|
|
neutral checks permit an assessment; they do not prove successful verification.
|
|
For example, a neutral scanner can mean coverage is unavailable. The advisory
|
|
routine evaluates these evidence gaps and reviews before recommending merge.
|
|
Workflow names do not exclude checks. The notifier and CI profile exporter run
|
|
on the default branch, so their own checks do not belong to the PR revision.
|
|
|
|
Check names alone are not trusted. Both core aggregates must belong to the same
|
|
GitHub Actions suite and a successful run of the repository's actual
|
|
`.github/workflows/pr.yml`. Its run metadata must reference
|
|
`pr-trusted.yml@master`. The complete PR file list must prove that the caller
|
|
`pr.yml` was not changed or renamed. A missing or truncated list stops delivery.
|
|
PRs that change this caller need a manual advisory review until their caller
|
|
change reaches the trusted branch. Other workflow changes can still be reviewed.
|
|
|
|
A final refresh rejects head or base changes during evaluation. Paperclip's
|
|
idempotency receipt deduplicates events for the same PR, head, and base. A new
|
|
base can produce a new review. The routine must refresh again before its final
|
|
recommendation. Event delivery and host check visibility can race; notifications
|
|
are best effort, with no polling or delivery guarantee.
|
|
|
|
The HTTP request uses TLS, a bearer header, no redirects, and a 15 second timeout.
|
|
Logs contain fixed delivery states only. Request and response bodies, URLs,
|
|
tokens, and raw exceptions are never logged. Missing secrets skip delivery.
|
|
|
|
Run the focused regression checks with:
|
|
|
|
```sh
|
|
node --test .github/scripts/tests/notify-pr-merge-review.test.mjs
|
|
```
|
|
|
|
For setup verification, use Paperclip's webhook `setupPending` test mode before
|
|
activation. This verifies authentication and ingress without starting a review.
|
|
After the workflow is merged to `master`, inspect a real successful workflow
|
|
delivery and its linked routine task to prove end-to-end operation. A saved
|
|
configuration or a setup test alone does not prove that journey.
|
|
|
|
To pause reviews, pause the routine in Paperclip or disable its trigger. To stop
|
|
notifications, disable this workflow or remove the two environment secrets.
|
|
Rotate the trigger signing secret in Paperclip and replace the token environment
|
|
secret together. Keep the URL secret even though it has no embedded bearer token.
|