DottaandClaude Opus 5.5
2e7c66aac9
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Its GitHub Actions CI shares the `ubuntu-latest` runner pool across all workflows > - #19480 added the `PR merge review notification` workflow. It triggers on `workflow_run`, on every completed `check_run`, and on every commit `status` > - One PR push completes about 60 checks, so it queues about 60 notifier jobs. Each job takes a runner > - The Actions queue now holds about 340 queued runs. About 260 of them are this notifier. Other `ubuntu-latest` jobs wait more than one hour (for example, the `authorize` job of Runner Full-Stack E2E image builds) > - This pull request skips events that cannot complete a green PR state before a runner is allocated, and it collapses the remaining events to the newest one per PR head > - The benefit is that this unjams the Actions queue and keeps the advisory review notification ## Linked Issues or Issue Description Refs #19480 **What happened?** The `PR merge review notification` workflow from #19480 schedules one runner job for each `workflow_run`, `check_run`, and `status` event. One PR push makes about 60+ jobs. The repository Actions queue has about 340 queued runs (about 260 from this notifier at the time of writing). Other `ubuntu-latest` jobs wait more than one hour. **What did you expect?** The notifier requests one advisory merge review when PR verification is complete. It must not use a runner for each intermediate check. **Steps to reproduce** 1. Push to any PR. 2. Run `gh run list --workflow pr-merge-review.yml -L 200 --json event,status`. 3. See tens of queued `check_run`, `status`, and `workflow_run` runs for the same push. ## What Changed - `.github/workflows/pr-merge-review.yml`: the job `if:` now schedules the job only for events that can complete a green PR state: - `workflow_run` from `pull_request` or `pull_request_target` with conclusion `success`, `skipped`, or `neutral`. - `check_run` from a non-Actions app (app id is not `15368`) with conclusion `success`, `skipped`, or `neutral`. GitHub Actions check runs are already covered by the `workflow_run` list. - `status` with state `success`. - GitHub cannot filter `check_run` or `status` at the trigger level. A skipped job makes a run record but does not take a runner. - The job has a concurrency group `pr-merge-review-<PR head SHA>` with `cancel-in-progress: true`. The notifier reads the current PR state, so the newest event subsumes older ones. The group is on the job, not the workflow, so a skipped event cannot cancel a scheduled notifier. - The environment, permissions, trusted default-branch checkout, and secret handling do not change. - `.github/scripts/tests/notify-pr-merge-review.test.mjs`: a new test pins the trigger kinds, the job condition, and the job-level concurrency group. - `doc/PR-MERGE-REVIEW-WEBHOOK.md`: describes the event filter and the concurrency group. **Expected runner jobs per PR push** (typical PR: `PR`, `Storybook Visual`, `commitperclip PR Review` workflows; Greptile, Paperclip Review, Socket (2), Superagent (2) external checks; one Snyk status): | | Before | After | | --- | --- | --- | | Eligible events that take a runner | about 60–70 (every check run + 2 statuses + 3 workflow runs) | at most about 10 (3 workflow runs + 6 external checks + 1 status, only when green) | | Jobs that run to completion | about 60–70 | usually 1–4: events that arrive while a job is pending or running cancel it, so a burst collapses to the newest event. In a backed-up queue, all events for one head collapse to one job | ## Verification - `node --test .github/scripts/tests/notify-pr-merge-review.test.mjs`: 22 tests pass. - The new workflow test fails on the old workflow (red) and passes on the new workflow (green). - `actionlint .github/workflows/pr-merge-review.yml`: no findings. - I confirmed with `gh run list` that `workflow_run.head_sha` for the `pull_request_target` workflow (`commitperclip PR Review`) is the PR head SHA. So one SHA key groups all event kinds for one PR head. - I confirmed that all workflows that make PR checks are in the `workflow_run` list (`CI routing` and `Trusted PR CI` are reusable workflows called by `PR`). - After merge: check that `gh run list --workflow pr-merge-review.yml` shows mostly `skipped` runs for each push, and a small number of runs that execute. ## Risks - A check provider that reports a final green state through a GitHub Actions check run from a workflow that is not in the `workflow_run` list does not trigger the notifier. All current PR workflows are in the list. The doc already tells authors to add new PR workflows to the list. - A failed or cancelled check that later succeeds on re-run still triggers the notifier, because the re-run completes with `success`. - `cancel-in-progress` can cancel a notifier during delivery. The newer job evaluates the same or a more complete state, and the Paperclip idempotency key deduplicates deliveries for the same PR, head, and base. - Each event still makes a workflow run record. Skipped runs do not take a runner. - Low risk otherwise: the notifier is advisory and `continue-on-error`. It does not affect PR verification results. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude Code with tool use (shell, file edits, GitHub CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
5.4 KiB
Advisory PR review notifications
The optional PR merge review notification workflow sends a private webhook
after a current, open, non-draft PR targeting master passes verification.
It does not approve or merge a PR. Webhook failures do not change PR CI results.
Private configuration
Create a GitHub Actions environment named pr-merge-review. Restrict its
deployment branch policy to the master branch only, with no tag policy.
Keep these two values in environment secrets, not repository variables,
repository secrets, committed files, task comments, or workflow outputs:
PAPERCLIP_PR_REVIEW_WEBHOOK_URL: a Paperclip routine webhook URL.PAPERCLIP_PR_REVIEW_WEBHOOK_TOKEN: its separate bearer signing secret.
Use a webhook trigger with signingMode: bearer. The routine should use
always_enqueue concurrency so separate PRs cannot be coalesced into one task.
Define required text variables pr_url, pr_number, head_sha, and base_sha.
The notifier sends only those identifiers. The receiving agent must treat them
as untrusted data and verify the repository, PR identity, and current revision.
Give the routine instructions to run pr-merge-review and publish its MERGE,
HOLD, or DO NOT MERGE recommendation on the routine execution's Paperclip task.
Require a fresh check of the head, base, reviews, required checks, and task gates.
Keep GitHub approval, merge, auto-merge, and public-comment actions disabled in
the instructions for this advisory phase. Configure the routine in Paperclip;
do not put instance-specific URLs or credentials in this repository.
Delivery and verification
The notification workflow runs trusted default-branch code. It never checks out
PR code or downloads PR artifacts or caches. It reacts to the repository's PR
workflow completions, external check completions, and legacy commit statuses.
When adding another PR verification workflow, add its name to workflow_run.
GitHub cannot filter check_run or status triggers, so the job condition
skips events that cannot complete a green state before a runner is allocated.
Only successful, skipped, or neutral PR workflow runs and non-Actions check runs,
and successful commit statuses, schedule the job. GitHub Actions check runs are
skipped because their workflow completions already arrive through workflow_run.
A job-level concurrency group keyed by the PR head SHA cancels older pending or
running notifiers, because the newest event sees the most complete PR state.
The group is on the job so skipped events cannot cancel a scheduled notifier.
The separate PR review API access workflow checks notifier changes using
the real read-only Actions token. It has no private environment or webhook
credentials and sends no webhook. Both workflows explicitly grant
actions, checks, contents, pull-requests, and statuses read access.
The GraphQL status rollup needs both check and commit-status permissions.
The notifier reads the current PR's status rollup and branch rules. Both
ci / verify and ci / e2e, plus any additional required checks, must succeed.
Other checks must succeed or finish as skipped or neutral. Optional skipped or
neutral checks permit an assessment; they do not prove successful verification.
For example, a neutral scanner can mean coverage is unavailable. The advisory
routine evaluates these evidence gaps and reviews before recommending merge.
Workflow names do not exclude checks. The notifier and CI profile exporter run
on the default branch, so their own checks do not belong to the PR revision.
Check names alone are not trusted. Both core aggregates must belong to the same
GitHub Actions suite and a successful run of the repository's actual
.github/workflows/pr.yml. Its run metadata must reference
pr-trusted.yml@master. The complete PR file list must prove that the caller
pr.yml was not changed or renamed. A missing or truncated list stops delivery.
PRs that change this caller need a manual advisory review until their caller
change reaches the trusted branch. Other workflow changes can still be reviewed.
A final refresh rejects head or base changes during evaluation. Paperclip's idempotency receipt deduplicates events for the same PR, head, and base. A new base can produce a new review. The routine must refresh again before its final recommendation. Event delivery and host check visibility can race; notifications are best effort, with no polling or delivery guarantee.
The HTTP request uses TLS, a bearer header, no redirects, and a 15 second timeout. Logs contain fixed delivery states only. Request and response bodies, URLs, tokens, and raw exceptions are never logged. Missing secrets skip delivery.
Run the focused regression checks with:
node --test .github/scripts/tests/notify-pr-merge-review.test.mjs
For setup verification, use Paperclip's webhook setupPending test mode before
activation. This verifies authentication and ingress without starting a review.
After the workflow is merged to master, inspect a real successful workflow
delivery and its linked routine task to prove end-to-end operation. A saved
configuration or a setup test alone does not prove that journey.
To pause reviews, pause the routine in Paperclip or disable its trigger. To stop notifications, disable this workflow or remove the two environment secrets. Rotate the trigger signing secret in Paperclip and replace the token environment secret together. Keep the URL secret even though it has no embedded bearer token.