mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 20:12:20 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Its pull request CI runs on two pools: an AWS fleet for CODEOWNERS, and GitHub-hosted runners for every other contributor > - Some short jobs for CODEOWNERS pull requests also run on GitHub-hosted runners: the commitperclip review and the merge review notifier > - GitHub starts about 550 hosted jobs each hour for this repository, and all hosted jobs wait in one first-come queue > - A community pull request run puts 52 jobs in that queue at one time, so the short CODEOWNERS jobs wait behind them > - This pull request limits how many jobs one hosted run can queue, and lets the short CODEOWNERS jobs use the reserved AWS metadata fleet > - The benefit is that community pull requests do not delay CODEOWNERS work ## Linked Issues or Issue Description **What happened?** On 2026-11-12 the repository had about 348 queued workflow runs. Jobs on the AWS fleet started in a median of 29 seconds. Jobs on `ubuntu-latest` did not: | Job | Median wait | Job length | | --- | --- | --- | | PR merge review notification | 42 min | 9 s | | commitperclip PR Review | 36 min | 15 s | | Community PR test jobs | 57 min (p90 76 min) | 7 min | Between 12:00Z and 18:00Z, hosted job starts stayed between 492 and 567 each hour while the number of running hosted jobs changed from 63 to 134. The start rate is the limit. Community PR runs supplied 3,462 of the 3,744 hosted jobs in that window (92%). **Expected behavior** A short job for a CODEOWNERS pull request starts in about one minute, whatever the community pull request load is. **Steps to reproduce** 1. Open several pull requests from forks in a short time. 2. Open a pull request as a CODEOWNER. 3. Look at the start time of `commitperclip PR Review` and `PR merge review notification` for the CODEOWNER pull request. ## What Changed - `pr-trusted.yml`: each of the four test matrices has `max-parallel: 4` when the run uses `ubuntu-latest`. The run has the same jobs. The other jobs wait in the run, not in the shared queue. AWS runs have no limit (256). - `commitperclip-review.yml` and `pr-merge-review.yml`: the job uses the reserved `paperclip-ci-routing-x64` fleet when `AWS_CI_SHORT_JOBS_ENABLED` is `true`, the pull request branch is in this repository, and the author and the sender (or the run actor and the triggering actor) are in the AWS admission list. The admission variables are the same as in `ci-routing.yml`. All other events use `ubuntu-latest`. - Check-run and commit-status notifications stay on `ubuntu-latest`. Those events do not identify the author. - `ci-routing.test.mjs`: 24 placement cases for the two jobs, and a contract test for the matrix limit. - `doc/DEVELOPING.md`: the two settings and the runner group step. ## Verification - `node --test .github/scripts/tests/ci-routing.test.mjs .github/scripts/tests/notify-pr-merge-review.test.mjs .github/scripts/tests/ci-plan.test.mjs .github/scripts/tests/ci-profile.test.mjs .github/scripts/tests/run-quality-gates.test.mjs`: 200 tests pass. - `actionlint` accepts the three workflows, including the `max-parallel` expression. - Not verified before merge: `pr.yml` calls `pr-trusted.yml@master`, and the two short workflows run from master. The effect on queue time can only be measured after merge. ## Risks - A community pull request run takes longer when the hosted queue is empty: about 45 minutes for the server matrix, compared with about 13 minutes. Today these runs take 150 to 200 minutes because of the queue. - The AWS routing does nothing until a maintainer sets `AWS_CI_SHORT_JOBS_ENABLED`. Before that, the `paperclip-ci-routing` runner group must allow the two workflow references in `doc/DEVELOPING.md`. If the variable is set first, the two jobs for CODEOWNERS pull requests stay queued. - The two jobs run only scripts from master and no pull request code. The secrets they read do not change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
67 lines
2.9 KiB
YAML
67 lines
2.9 KiB
YAML
name: commitperclip PR Review
|
|
|
|
on:
|
|
pull_request_target:
|
|
types: [opened, synchronize, reopened]
|
|
|
|
# Always runs from base branch context — never executes PR code.
|
|
# pull_request_target gives access to secrets for untrusted fork PRs.
|
|
|
|
permissions:
|
|
pull-requests: write
|
|
checks: write
|
|
contents: read
|
|
|
|
jobs:
|
|
review:
|
|
# Community PR CI shares the GitHub-hosted queue. A CODEOWNER's own PR is
|
|
# reviewed from the reserved AWS metadata fleet so it never waits there.
|
|
# Same admission as ci-routing.yml; every other PR stays GitHub-hosted.
|
|
runs-on: ${{ vars.AWS_CI_SHORT_JOBS_ENABLED == 'true' && vars.AWS_CI_ENABLED == 'true' && vars.AWS_CI_ROUTING_USER_IDS_SOURCE != '' && vars.AWS_CI_TRUSTED_USER_IDS == vars.AWS_CI_ROUTING_USER_IDS_SOURCE && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && github.event_name == 'pull_request_target' && github.event.pull_request.head.repo.id == 1170821064 && contains(vars.AWS_CI_ROUTING_USER_IDS, format('|{0}|', github.event.pull_request.user.id)) && contains(vars.AWS_CI_ROUTING_USER_IDS, format('|{0}|', github.event.sender.id)) && 'runs-on/fleet=paperclip-ci-routing-x64/env=public-ci' || 'ubuntu-latest' }}
|
|
timeout-minutes: 5
|
|
steps:
|
|
- name: Checkout base branch (never PR code)
|
|
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
|
|
with:
|
|
ref: master
|
|
|
|
- name: Dependency Review
|
|
uses: actions/dependency-review-action@a1d282b36b6f3519aa1f3fc636f609c47dddb294 # v5.0.0
|
|
with:
|
|
base-ref: ${{ github.event.pull_request.base.sha }}
|
|
head-ref: ${{ github.event.pull_request.head.sha }}
|
|
|
|
- name: Set up Node
|
|
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
|
with:
|
|
node-version: '24'
|
|
|
|
- name: Generate commitperclip token
|
|
id: token
|
|
run: |
|
|
TOKEN=$(node .github/scripts/get-bot-token.mjs)
|
|
echo "::add-mask::$TOKEN"
|
|
echo "value=$TOKEN" >> $GITHUB_OUTPUT
|
|
env:
|
|
COMMITPERCLIP_KEY: ${{ secrets.COMMITPERCLIP_KEY }}
|
|
|
|
- name: Run quality gates
|
|
id: quality
|
|
if: github.event.pull_request.user.login != 'dependabot[bot]'
|
|
run: node .github/scripts/run-quality-gates.mjs
|
|
continue-on-error: true
|
|
env:
|
|
GH_TOKEN: ${{ steps.token.outputs.value }}
|
|
GH_REPO: ${{ github.repository }}
|
|
PR_NUMBER: ${{ github.event.pull_request.number }}
|
|
PR_AUTHOR: ${{ github.event.pull_request.user.login }}
|
|
PR_BRANCH: ${{ github.event.pull_request.head.ref }}
|
|
|
|
- name: Fail if quality gates failed
|
|
if: >-
|
|
github.event.pull_request.user.login != 'dependabot[bot]' &&
|
|
steps.quality.outcome == 'failure'
|
|
run: |
|
|
echo "One or more quality gates failed. See commitperclip comment on the PR for details."
|
|
exit 1
|