ci(runner): route paid matrix to AWS fleet (#12765)

## Thinking Path

> - Paperclip manages AI agents that perform work.
> - The paid runner matrix verifies complete runner behavior with real
providers.
> - Each matrix job currently repeats work on GitHub-hosted runners.
> - Paperclip has an ephemeral AWS runner fleet for trusted workflows.
> - The paid workflow needs a reviewed and fail-closed route to that
fleet.
> - This pull request adds that route and keeps the existing hosted
runner as the disabled-state fallback.
> - The benefit is faster paid campaigns with the same actor,
environment, and secret boundaries.

## Linked Issues or Issue Description

**What happened?**

The Runner Full-Stack E2E workflow always uses `ubuntu-latest-m`. It
limits the matrix to 57 parallel jobs. The repository AWS fleet can run
100 ephemeral jobs, but the paid workflow cannot select it.

**Expected behavior**

An explicit repository flag must select the reviewed AWS fleet label. A
missing or invalid flag must keep the existing hosted runner. The
workflow must authorize the stable actor identity before it routes any
paid job.

**Steps to reproduce**

1. Dispatch the Runner Full-Stack E2E workflow from `master`.
2. Inspect a paid matrix job.
3. Observe that the job requests `ubuntu-latest-m` even when the AWS
fleet should be used.

**Paperclip version or commit**

`da0947d3582ac7779d6bf11851c9938eca6c5c8c`

**Deployment mode**

GitHub Actions paid runner campaign.

## What Changed

- Add a fail-closed `RUNNER_E2E_AWS_ENABLED` switch.
- Select only the reviewed AWS fleet label or the existing hosted label.
- Permit up to 100 parallel jobs in AWS mode.
- Keep the hosted-runner limit at 57.
- Reauthorize paid execution before checkout and provider access.
- Stop paid checkouts from storing GitHub credentials.
- Cancel superseded validation-ref campaigns while preserving `master`
audit runs.
- Add workflow policy checks and operator documentation.

## Verification

- `git diff --check`
- `actionlint -ignore SC2129
.github/workflows/runner-full-stack-e2e.yml`
- The organization runner group permits this workflow only from
`refs/heads/master`.
- The repository AWS switch remains disabled until this pull request is
merged and a one-cell probe succeeds.

## Risks

- A wrong fleet policy can leave jobs queued. The disabled state keeps
the existing hosted runner.
- The AWS fleet uses paid compute. The workflow validates a configured
maximum of 100 jobs.
- The runner group, actor allowlist, and paid environment remain
separate enforcement layers.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex based on GPT-5 with agentic reasoning, repository
inspection, code editing, Git, GitHub API coordination, and static
workflow analysis. The exact deployed model identifier and
context-window size are not exposed to this task.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
This commit is contained in:
Dotta authored and GitHub committed 2026-09-03 08:50:36 -05:00
1 parent e4afd163bf
commit 1b74561fea
4 files changed
+129 -33

No files matched your search

+46 -6
View File
@@ -39,7 +39,9 @@ permissions:
concurrency:
group: runner-full-stack-e2e-${{ github.ref }}
cancel-in-progress: false
# Development-only validation refs supersede older runs on the same ref.
# Preserve every protected default-branch campaign for its paid audit trail.
cancel-in-progress: ${{ github.ref != format('refs/heads/{0}', github.event.repository.default_branch) }}
jobs:
authorize:
@@ -49,6 +51,10 @@ jobs:
timeout-minutes: 5
permissions:
contents: read
outputs:
test_runner: ${{ steps.runner.outputs.runner }}
max_parallel_default: ${{ steps.runner.outputs.max_parallel_default }}
max_parallel_limit: ${{ steps.runner.outputs.max_parallel_limit }}
steps:
- name: Require default branch and allowlisted numeric actor IDs
env:
@@ -83,6 +89,31 @@ jobs:
fi
done
- name: Select paid test runner
id: runner
env:
AWS_PAID_RUNNER_ENABLED: ${{ vars.RUNNER_E2E_AWS_ENABLED }}
run: |
set -euo pipefail
github_runner='ubuntu-latest-m'
aws_runner='runs-on/fleet=paperclip-public-pr-x64/env=public-ci'
if [ "$AWS_PAID_RUNNER_ENABLED" = true ]; then
{
echo "runner=$aws_runner"
echo "max_parallel_default=100"
echo "max_parallel_limit=100"
} >> "$GITHUB_OUTPUT"
echo '::notice title=Paid runner routing::Using an ephemeral RunsOn Fleet runner'
else
{
echo "runner=$github_runner"
echo "max_parallel_default=32"
echo "max_parallel_limit=57"
} >> "$GITHUB_OUTPUT"
echo '::notice title=Paid runner routing::RUNNER_E2E_AWS_ENABLED is not true; using the existing paid runner'
fi
catalog:
name: Validate catalog and select cells
needs: authorize
@@ -128,7 +159,8 @@ jobs:
SELECT_ENVIRONMENT: ${{ inputs.environment }}
SELECT_CASE: ${{ inputs.case }}
SELECT_ID: ${{ inputs.id }}
MAX_PARALLEL: ${{ vars.RUNNER_E2E_MAX_PARALLEL || '32' }}
MAX_PARALLEL: ${{ vars.RUNNER_E2E_MAX_PARALLEL || needs.authorize.outputs.max_parallel_default }}
MAX_PARALLEL_LIMIT: ${{ needs.authorize.outputs.max_parallel_limit }}
run: |
set -euo pipefail
args=(--matrix-json)
@@ -176,8 +208,12 @@ jobs:
echo "matrix=$(jq -c '{include: .include}' <<< "$catalog_json")" >> "$GITHUB_OUTPUT"
echo "needs_daytona=$(jq -r '.needsDaytona' <<< "$catalog_json")" >> "$GITHUB_OUTPUT"
echo "execution_ids=$(jq -c '.executionIds' <<< "$catalog_json")" >> "$GITHUB_OUTPUT"
if ! [[ "$MAX_PARALLEL" =~ ^[1-9][0-9]*$ ]] || [ "$MAX_PARALLEL" -gt 57 ]; then
echo "RUNNER_E2E_MAX_PARALLEL must be an integer from 1 through 57." >&2
if ! [[ "$MAX_PARALLEL_LIMIT" =~ ^[1-9][0-9]*$ ]] || [ "$MAX_PARALLEL_LIMIT" -gt 100 ]; then
echo "Runner selection emitted an invalid max-parallel limit." >&2
exit 1
fi
if ! [[ "$MAX_PARALLEL" =~ ^[1-9][0-9]*$ ]] || [ "$MAX_PARALLEL" -gt "$MAX_PARALLEL_LIMIT" ]; then
echo "RUNNER_E2E_MAX_PARALLEL must be an integer from 1 through $MAX_PARALLEL_LIMIT for the selected runner." >&2
exit 1
fi
echo "max_parallel=$MAX_PARALLEL" >> "$GITHUB_OUTPUT"
@@ -286,8 +322,10 @@ jobs:
test:
name: ${{ matrix.executionId }}
needs: [catalog, daytona_image]
runs-on: ubuntu-latest-m
needs: [authorize, catalog, daytona_image]
# The authorize job selects only one of two literal, reviewed runner labels;
# no dispatch input or repository variable can inject an arbitrary label.
runs-on: ${{ needs.authorize.outputs.test_runner }}
timeout-minutes: ${{ matrix.timeoutMinutes }}
permissions:
contents: read
@@ -315,6 +353,8 @@ jobs:
jq -e --argjson candidate "$ACTOR_ID" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
persist-credentials: false
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
with: