mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-16 05:17:41 +02:00
## Thinking Path
> - Paperclip uses CI to verify changes before release.
> - Trusted PR tests can run on isolated AWS workers.
> - Their runner-selection job currently waits for GitHub-hosted
capacity.
> - A large hosted test queue therefore blocks admission to AWS.
> - A dedicated routing workflow lets us reserve capacity for metadata
validation.
> - This PR moves the routing job to reserved AWS capacity after the
bootstrap workflow and pool are available.
## Linked Issues or Issue Description
**Issue type**
CI reliability enhancement.
**Where is the issue?**
The Select trusted runner job in PR CI.
**What's wrong?**
The routing job competes with a large hosted test queue. Its sampled
median wait was about 20 minutes. AWS tests cannot begin before this job
finishes.
**Suggested fix**
Add a protected reusable routing workflow. Deploy its dedicated AWS
pool, then switch the PR caller in a follow-up PR. No overlapping open
PR or issue was found.
## What Changed
- Call ci-routing.yml on protected master from pr-trusted.yml.
- Check the canonical repository and approved PR author/sender before
allocating reserved AWS metadata capacity. Unapproved PRs continue to
use hosted routing.
- Permit approved manual routing canaries on protected master.
- Use a validated roster snapshot for placement. Invalid JSON, missing
settings, stale snapshots, and unapproved identities select hosted
routing without a placement-time JSON parse.
- Preserve the full in-job identity, rerun actor, repository, head,
merge, and stack checks.
- Update stack-scope and placement regression tests and document the
pool and rollback.
## Verification
- The routing scripts and placement expression tests passed (44 tests),
routing-plus-shard tests passed (56 tests), and the existing
.github/scripts suite passed (381 tests).
- Existing routing allow/deny scenarios passed with the extracted script
(16 cases).
- actionlint passed with the dedicated custom runner label declared in
its local configuration.
- git diff --check passed.
- Full pnpm typecheck/test/build cannot run in this worktree because
dependencies are not installed. The change is limited to workflow YAML
and Node tests.
- The dedicated pool is deployed with two warm On-Demand workers and a
four-worker ceiling. The routing qualification passed: it started about
five seconds after workflow creation, selected AWS, and finished while
the hosted gate in the same PR remained queued.
- Bootstrap https://github.com/paperclipai/paperclip/pull/16039 is
merged. The deployed group now authorizes only ci-routing.yml on
protected master, and its master canary passed.
## Risks
- The maintainer explicitly approved a queued-CI admin bypass after
current-head Greptile 5/5 and focused verification.
- Activation requires the deployed routing Fleet and its group to
authorize the routing workflow on protected master. Keep the bootstrap
dependency explicit.
- The group must authorize only ci-routing.yml on protected master and
canonical repository ID 1170821064.
- The routing workflow runs no checkout or PR code and has read-only
permissions. The test admission checks are unchanged.
## Model Used
- OpenAI GPT-6 via Codex, with reasoning, shell tools, and file editing.
The exact runtime model ID and context window are not exposed in this
session.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (focused routing and
workflow tests; full pnpm commands are blocked by missing worktree
dependencies)
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] Required CI status checked; the maintainer explicitly approved
admin bypass for queued CI after clean current-head review and focused
verification
- [x] Greptile is 5/5 on 993fbe695a with
no open findings or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
253 lines
12 KiB
YAML
253 lines
12 KiB
YAML
name: CI routing
|
|
|
|
on:
|
|
workflow_dispatch:
|
|
workflow_call:
|
|
outputs:
|
|
runner:
|
|
value: ${{ jobs.gate.outputs.runner }}
|
|
full_ci:
|
|
value: ${{ jobs.gate.outputs.full_ci }}
|
|
|
|
permissions:
|
|
actions: read
|
|
contents: read
|
|
pull-requests: read
|
|
|
|
jobs:
|
|
gate:
|
|
name: Select trusted runner
|
|
runs-on: ${{ vars.AWS_CI_ENABLED == 'true' && vars.AWS_CI_ROUTING_USER_IDS_SOURCE != '' && vars.AWS_CI_TRUSTED_USER_IDS == vars.AWS_CI_ROUTING_USER_IDS_SOURCE && github.repository == 'paperclipai/paperclip' && github.repository_id == '1170821064' && ((github.event_name == 'pull_request' && contains(vars.AWS_CI_ROUTING_USER_IDS, format('|{0}|', github.event.pull_request.user.id)) && contains(vars.AWS_CI_ROUTING_USER_IDS, format('|{0}|', github.event.sender.id))) || (github.event_name == 'workflow_dispatch' && github.ref == 'refs/heads/master' && contains(vars.AWS_CI_ROUTING_USER_IDS, format('|{0}|', github.actor_id)))) && 'runs-on/fleet=paperclip-ci-routing-x64/env=public-ci' || 'ubuntu-latest' }}
|
|
timeout-minutes: 5
|
|
outputs:
|
|
runner: ${{ steps.route.outputs.runner }}
|
|
full_ci: ${{ steps.scope.outputs.full_ci }}
|
|
|
|
steps:
|
|
- name: Validate PR identity and select runner
|
|
id: route
|
|
shell: bash
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
AWS_CI_ENABLED: ${{ vars.AWS_CI_ENABLED }}
|
|
TRUSTED_USER_IDS: ${{ vars.AWS_CI_TRUSTED_USER_IDS }}
|
|
EVENT_NAME: ${{ github.event_name }}
|
|
EVENT_REPOSITORY: ${{ github.repository }}
|
|
EVENT_REPOSITORY_ID: ${{ github.repository_id }}
|
|
EVENT_ACTION: ${{ github.event.action }}
|
|
EVENT_PR_NUMBER: ${{ github.event.pull_request.number }}
|
|
EVENT_PR_AUTHOR_ID: ${{ github.event.pull_request.user.id }}
|
|
EVENT_SENDER_ID: ${{ github.event.sender.id }}
|
|
EVENT_BASE_REPOSITORY_ID: ${{ github.event.pull_request.base.repo.id }}
|
|
EVENT_BASE_REF: ${{ github.event.pull_request.base.ref }}
|
|
EVENT_BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
|
EVENT_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
|
EVENT_MERGE_SHA: ${{ github.sha }}
|
|
RUN_ID: ${{ github.run_id }}
|
|
run: |
|
|
set -u
|
|
|
|
github_runner='ubuntu-latest'
|
|
aws_runner='runs-on/fleet=paperclip-public-pr-x64/env=public-ci'
|
|
|
|
fail_closed() {
|
|
echo "runner=$github_runner" >> "$GITHUB_OUTPUT"
|
|
echo "::notice title=AWS CI routing::Using GitHub-hosted runner: $1"
|
|
exit 0
|
|
}
|
|
|
|
is_positive_integer() {
|
|
[[ "$1" =~ ^[1-9][0-9]*$ ]]
|
|
}
|
|
|
|
is_commit_sha() {
|
|
[[ "$1" =~ ^[0-9a-f]{40}$ ]]
|
|
}
|
|
|
|
is_allowed() {
|
|
local user_id="$1"
|
|
jq -e --argjson user_id "$user_id" 'index($user_id) != null' \
|
|
<<< "$TRUSTED_USER_IDS" >/dev/null
|
|
}
|
|
|
|
[[ "$AWS_CI_ENABLED" == 'true' ]] || fail_closed 'AWS_CI_ENABLED is not true'
|
|
[[ "$EVENT_NAME" == 'pull_request' ]] || fail_closed 'event is not pull_request'
|
|
[[ "$EVENT_REPOSITORY" == 'paperclipai/paperclip' ]] || fail_closed 'unexpected repository'
|
|
[[ "$EVENT_REPOSITORY_ID" == '1170821064' ]] || fail_closed 'unexpected repository ID'
|
|
[[ "$EVENT_BASE_REPOSITORY_ID" == '1170821064' ]] || fail_closed 'unexpected base repository ID'
|
|
[[ -n "$EVENT_BASE_REF" ]] || fail_closed 'missing base branch'
|
|
[[ "$EVENT_ACTION" =~ ^(opened|reopened|synchronize)$ ]] || fail_closed 'unsupported pull_request action'
|
|
|
|
jq -e '
|
|
type == "array" and
|
|
length > 0 and
|
|
all(.[]; type == "number" and . > 0 and floor == .)
|
|
' <<< "$TRUSTED_USER_IDS" >/dev/null 2>&1 || fail_closed 'trusted user ID list is malformed'
|
|
|
|
for user_id in "$EVENT_PR_AUTHOR_ID" "$EVENT_SENDER_ID"; do
|
|
is_positive_integer "$user_id" || fail_closed 'event contains a malformed user ID'
|
|
is_allowed "$user_id" || fail_closed "GitHub user ID $user_id is not allowlisted"
|
|
done
|
|
|
|
for commit_sha in "$EVENT_BASE_SHA" "$EVENT_HEAD_SHA" "$EVENT_MERGE_SHA"; do
|
|
is_commit_sha "$commit_sha" || fail_closed 'event contains a malformed commit SHA'
|
|
done
|
|
|
|
pr_json="$(gh api \
|
|
-H 'Accept: application/vnd.github+json' \
|
|
-H 'X-GitHub-Api-Version: 2022-11-28' \
|
|
"/repos/paperclipai/paperclip/pulls/$EVENT_PR_NUMBER" 2>/dev/null)" \
|
|
|| fail_closed 'could not refresh pull request state'
|
|
|
|
jq -e \
|
|
--argjson repository_id 1170821064 \
|
|
--argjson author_id "$EVENT_PR_AUTHOR_ID" \
|
|
--arg base_ref "$EVENT_BASE_REF" \
|
|
--arg head_sha "$EVENT_HEAD_SHA" \
|
|
'
|
|
.state == "open" and
|
|
.user.id == $author_id and
|
|
.base.repo.id == $repository_id and
|
|
.base.ref == $base_ref and
|
|
.head.sha == $head_sha
|
|
' <<< "$pr_json" >/dev/null 2>&1 \
|
|
|| fail_closed 'current pull request state does not match the triggering event'
|
|
|
|
live_merge_sha="$(jq -r '.merge_commit_sha // empty' <<< "$pr_json")"
|
|
is_commit_sha "$live_merge_sha" || fail_closed 'current pull request has no valid merge SHA'
|
|
|
|
base_ref_json="$(gh api \
|
|
-H 'Accept: application/vnd.github+json' \
|
|
-H 'X-GitHub-Api-Version: 2022-11-28' \
|
|
"/repos/paperclipai/paperclip/git/ref/heads/$EVENT_BASE_REF" 2>/dev/null)" \
|
|
|| fail_closed 'could not inspect the current base branch'
|
|
live_base_ref_sha="$(jq -r '.object.sha // empty' <<< "$base_ref_json")"
|
|
is_commit_sha "$live_base_ref_sha" || fail_closed 'current base branch has no valid commit SHA'
|
|
|
|
base_comparison="$(gh api \
|
|
-H 'Accept: application/vnd.github+json' \
|
|
-H 'X-GitHub-Api-Version: 2022-11-28' \
|
|
"/repos/paperclipai/paperclip/compare/$EVENT_BASE_SHA...$live_base_ref_sha" 2>/dev/null)" \
|
|
|| fail_closed 'could not compare the triggering and current base branches'
|
|
jq -e \
|
|
--arg event_base_sha "$EVENT_BASE_SHA" '
|
|
.merge_base_commit.sha == $event_base_sha and
|
|
(.status == "ahead" or .status == "identical")
|
|
' <<< "$base_comparison" >/dev/null 2>&1 \
|
|
|| fail_closed 'current base branch does not descend from the triggering base snapshot'
|
|
|
|
event_merge_json="$(gh api \
|
|
-H 'Accept: application/vnd.github+json' \
|
|
-H 'X-GitHub-Api-Version: 2022-11-28' \
|
|
"/repos/paperclipai/paperclip/git/commits/$EVENT_MERGE_SHA" 2>/dev/null)" \
|
|
|| fail_closed 'could not inspect the event merge commit'
|
|
|
|
live_merge_json="$(gh api \
|
|
-H 'Accept: application/vnd.github+json' \
|
|
-H 'X-GitHub-Api-Version: 2022-11-28' \
|
|
"/repos/paperclipai/paperclip/git/commits/$live_merge_sha" 2>/dev/null)" \
|
|
|| fail_closed 'could not inspect the current merge commit'
|
|
|
|
validate_merge_commit() {
|
|
local merge_json="$1"
|
|
local expected_sha="$2"
|
|
jq -e \
|
|
--arg expected_sha "$expected_sha" \
|
|
--arg head_sha "$EVENT_HEAD_SHA" '
|
|
.sha == $expected_sha and
|
|
(.parents | length) == 2 and
|
|
.parents[1].sha == $head_sha and
|
|
(.parents[0].sha | test("^[0-9a-f]{40}$")) and
|
|
(.tree.sha | test("^[0-9a-f]{40}$"))
|
|
' <<< "$merge_json" >/dev/null 2>&1
|
|
}
|
|
|
|
validate_merge_commit "$event_merge_json" "$EVENT_MERGE_SHA" \
|
|
|| fail_closed 'event merge commit does not match the current base and head'
|
|
validate_merge_commit "$live_merge_json" "$live_merge_sha" \
|
|
|| fail_closed 'current merge commit does not match the triggering base and head'
|
|
|
|
event_merge_parent="$(jq -r '.parents[0].sha' <<< "$event_merge_json")"
|
|
live_merge_parent="$(jq -r '.parents[0].sha' <<< "$live_merge_json")"
|
|
[[ "$event_merge_parent" == "$live_merge_parent" ]] \
|
|
|| fail_closed 'current merge commit uses a different base merge parent'
|
|
|
|
if [[ "$event_merge_parent" != "$live_base_ref_sha" ]]; then
|
|
base_merge_json="$(gh api \
|
|
-H 'Accept: application/vnd.github+json' \
|
|
-H 'X-GitHub-Api-Version: 2022-11-28' \
|
|
"/repos/paperclipai/paperclip/git/commits/$event_merge_parent" 2>/dev/null)" \
|
|
|| fail_closed 'could not inspect the stacked base merge commit'
|
|
jq -e \
|
|
--arg expected_sha "$event_merge_parent" \
|
|
--arg live_base_ref_sha "$live_base_ref_sha" '
|
|
.sha == $expected_sha and
|
|
(.parents | length) == 2 and
|
|
any(.parents[]; .sha == $live_base_ref_sha) and
|
|
(.tree.sha | test("^[0-9a-f]{40}$"))
|
|
' <<< "$base_merge_json" >/dev/null 2>&1 \
|
|
|| fail_closed 'stacked base merge commit does not contain the current base branch'
|
|
fi
|
|
|
|
event_merge_tree="$(jq -r '.tree.sha' <<< "$event_merge_json")"
|
|
live_merge_tree="$(jq -r '.tree.sha' <<< "$live_merge_json")"
|
|
[[ "$event_merge_tree" == "$live_merge_tree" ]] \
|
|
|| fail_closed 'current merge tree differs from the triggering merge tree'
|
|
|
|
run_json="$(gh api \
|
|
-H 'Accept: application/vnd.github+json' \
|
|
-H 'X-GitHub-Api-Version: 2022-11-28' \
|
|
"/repos/paperclipai/paperclip/actions/runs/$RUN_ID" 2>/dev/null)" \
|
|
|| fail_closed 'could not refresh workflow run state'
|
|
|
|
triggering_actor_id="$(jq -r '.triggering_actor.id // empty' <<< "$run_json")"
|
|
is_positive_integer "$triggering_actor_id" || fail_closed 'workflow run has no valid triggering actor ID'
|
|
is_allowed "$triggering_actor_id" || fail_closed "triggering GitHub user ID $triggering_actor_id is not allowlisted"
|
|
|
|
jq -e \
|
|
--argjson repository_id 1170821064 \
|
|
--arg event_name "$EVENT_NAME" '
|
|
.repository.id == $repository_id and
|
|
.event == $event_name
|
|
' <<< "$run_json" >/dev/null 2>&1 \
|
|
|| fail_closed 'current workflow run does not match the expected repository event'
|
|
|
|
echo "runner=$aws_runner" >> "$GITHUB_OUTPUT"
|
|
echo '::notice title=AWS CI routing::Using an ephemeral RunsOn Fleet runner'
|
|
|
|
- name: Select stacked PR CI scope
|
|
id: scope
|
|
shell: bash
|
|
env:
|
|
STACK_JSON: ${{ toJSON(github.event.pull_request.stack) }}
|
|
PR_BASE_REF: ${{ github.event.pull_request.base.ref }}
|
|
run: |
|
|
set -euo pipefail
|
|
|
|
full_ci='true'
|
|
reason='ordinary pull request'
|
|
|
|
if jq -e 'type == "object"' <<< "$STACK_JSON" >/dev/null 2>&1; then
|
|
stack_position="$(jq -r '.position // empty' <<< "$STACK_JSON")"
|
|
stack_size="$(jq -r '.size // empty' <<< "$STACK_JSON")"
|
|
stack_base_ref="$(jq -r '.base.ref // empty' <<< "$STACK_JSON")"
|
|
|
|
if [[ ! "$stack_position" =~ ^[1-9][0-9]*$ ]] ||
|
|
[[ ! "$stack_size" =~ ^[1-9][0-9]*$ ]] ||
|
|
(( stack_position > stack_size )) ||
|
|
[[ -z "$stack_base_ref" ]]; then
|
|
reason='malformed stack metadata; defaulting to full CI'
|
|
elif (( stack_position == stack_size )); then
|
|
reason='top pull request in stack'
|
|
elif [[ "$stack_base_ref" == "$PR_BASE_REF" ]]; then
|
|
reason='lowest unmerged pull request in stack'
|
|
else
|
|
full_ci='false'
|
|
reason='middle pull request in stack'
|
|
fi
|
|
fi
|
|
|
|
echo "full_ci=$full_ci" >> "$GITHUB_OUTPUT"
|
|
echo "::notice title=Stacked PR CI scope::$reason; full_ci=$full_ci"
|
|
|