Files
PaperClipAI/.github/scripts/tests
DottaandClaude Fable 5.2 8990e94005 fix(ci): keep community PR runs from delaying CODEOWNERS jobs in the hosted queue (#21166)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Its pull request CI runs on two pools: an AWS fleet for CODEOWNERS,
and GitHub-hosted runners for every other contributor
> - Some short jobs for CODEOWNERS pull requests also run on
GitHub-hosted runners: the commitperclip review and the merge review
notifier
> - GitHub starts about 550 hosted jobs each hour for this repository,
and all hosted jobs wait in one first-come queue
> - A community pull request run puts 52 jobs in that queue at one time,
so the short CODEOWNERS jobs wait behind them
> - This pull request limits how many jobs one hosted run can queue, and
lets the short CODEOWNERS jobs use the reserved AWS metadata fleet
> - The benefit is that community pull requests do not delay CODEOWNERS
work

## Linked Issues or Issue Description

**What happened?**

On 2026-11-12 the repository had about 348 queued workflow runs. Jobs on
the AWS fleet started in a median of 29 seconds. Jobs on `ubuntu-latest`
did not:

| Job | Median wait | Job length |
| --- | --- | --- |
| PR merge review notification | 42 min | 9 s |
| commitperclip PR Review | 36 min | 15 s |
| Community PR test jobs | 57 min (p90 76 min) | 7 min |

Between 12:00Z and 18:00Z, hosted job starts stayed between 492 and 567
each hour while the number of running hosted jobs changed from 63 to
134. The start rate is the limit. Community PR runs supplied 3,462 of
the 3,744 hosted jobs in that window (92%).

**Expected behavior**

A short job for a CODEOWNERS pull request starts in about one minute,
whatever the community pull request load is.

**Steps to reproduce**

1. Open several pull requests from forks in a short time.
2. Open a pull request as a CODEOWNER.
3. Look at the start time of `commitperclip PR Review` and `PR merge
review notification` for the CODEOWNER pull request.

## What Changed

- `pr-trusted.yml`: each of the four test matrices has `max-parallel: 4`
when the run uses `ubuntu-latest`. The run has the same jobs. The other
jobs wait in the run, not in the shared queue. AWS runs have no limit
(256).
- `commitperclip-review.yml` and `pr-merge-review.yml`: the job uses the
reserved `paperclip-ci-routing-x64` fleet when
`AWS_CI_SHORT_JOBS_ENABLED` is `true`, the pull request branch is in
this repository, and the author and the sender (or the run actor and the
triggering actor) are in the AWS admission list. The admission variables
are the same as in `ci-routing.yml`. All other events use
`ubuntu-latest`.
- Check-run and commit-status notifications stay on `ubuntu-latest`.
Those events do not identify the author.
- `ci-routing.test.mjs`: 24 placement cases for the two jobs, and a
contract test for the matrix limit.
- `doc/DEVELOPING.md`: the two settings and the runner group step.

## Verification

- `node --test .github/scripts/tests/ci-routing.test.mjs
.github/scripts/tests/notify-pr-merge-review.test.mjs
.github/scripts/tests/ci-plan.test.mjs
.github/scripts/tests/ci-profile.test.mjs
.github/scripts/tests/run-quality-gates.test.mjs`: 200 tests pass.
- `actionlint` accepts the three workflows, including the `max-parallel`
expression.
- Not verified before merge: `pr.yml` calls `pr-trusted.yml@master`, and
the two short workflows run from master. The effect on queue time can
only be measured after merge.

## Risks

- A community pull request run takes longer when the hosted queue is
empty: about 45 minutes for the server matrix, compared with about 13
minutes. Today these runs take 150 to 200 minutes because of the queue.
- The AWS routing does nothing until a maintainer sets
`AWS_CI_SHORT_JOBS_ENABLED`. Before that, the `paperclip-ci-routing`
runner group must allow the two workflow references in
`doc/DEVELOPING.md`. If the variable is set first, the two jobs for
CODEOWNERS pull requests stay queued.
- The two jobs run only scripts from master and no pull request code.
The secrets they read do not change.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Fable 5.2 (`claude-fable-5-2`) in Claude Code, with tool use
and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 13:48:25 -06:00
..