mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
DottaandClaude Opus 5.5
744056ecc8
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The runner full-stack E2E lane runs real Paperclip work against real agents, and TestOps launches its standing plans > - PR #19455 added seven staging-soak reliability cases, but no TestOps plan or workflow lane selects them, so they run only by hand with personal keys > - These cases catch restart, interrupt, queue and lost-runner regressions, so they must run from TestOps before a release with the standard reports > - This pull request adds a sibling "Reliability — staging soak" TestOps plan on the existing plan-2 lane, with native OpenCode and DeepSeek 1017 through OpenRouter as the plan profile for six of seven cases > - The benefit is a cheap, repeatable pre-release reliability launch with the same Allure/TestOps results, retained report and dashboard as plan 2 ## Linked Issues or Issue Description Refs #19455 **What existing behavior does this improve?** The staging-soak reliability cases from #19455 run only by a manual local command. **Subsystem affected** Runner full-stack E2E (`tests/runner-e2e/`) and its TestOps lane in `.github/workflows/runner-full-stack-e2e.yml`. **Current behavior** No TestOps plan includes the cases. The workflow cannot select them from TestOps. Every cell uses native Codex (`gpt-5.6-sol`) or an ACPX facade on Daytona, which is expensive. **Proposed behavior** A TestOps launch of the Reliability plan runs seven cells (six DeepSeek 1017, one native Codex for steering) through the standing plan-2 path and reports them like plan 2. **Reason and benefit** The cases become a first-class, low-cost, pre-release check with retained evidence. Plan 2 keeps its 20-case qualification semantics. **Breaking changes** None. Plan 2 selection, reporting and history identities stay the same. ## What Changed - **Plan choice: a sibling plan, not an extension of plan 2.** Plan 2 is bounded at 20 standing IDs. Its `full-run` / `test-plan-2` tag needs a complete 20-case attempt, and subset runs do not qualify it. Adding seven cases would change that denominator and the qualification history. The sibling plan shares the same TestOps GitHub job, selection, admission, projection, upload and verification code. A launch selects cases from exactly one plan (`plan_mixed` fails), and each plan has its own full-plan size (20 or 7). - `tests/runner-e2e/reliability-testops.mjs` (new): the mapping module. It has seven cases, full names `paperclip.Product E2E.everyday-workflows.<case>`, execution IDs, soak signatures and an empty `reliabilityRegistration`. A partial, duplicate, plan-2 or malformed registration is rejected. An empty registration makes selection fail closed before any spend (`product_regression_testops_reliability_unregistered` for a bootstrap, `case_mismatch` for a TestOps launch). No TestOps IDs are invented. - `product-regression-testops.mjs` / `-cli.mjs`: select a plan by its case identities or by the new bootstrap input. The projection binds each cell's own environment (`local` or `daytona`) instead of a fixed `local`. It adds a `soak-<signature>` tag and reports `plan` and a per-plan `fullPlan`. Plan-2 history IDs, parameters and results do not change. - Catalog: `runner-opencode-deepseek-1017` is now one shared fixture with `supportedEnvironments: ["local", "daytona"]`. The OpenCode server driver already runs on Daytona for `runner-opencode` (task-workspaces, core-compatibility, rolling deployment); only the model differs. The chat-feature suite stays local, and its definition hash does not change. `everyday-workflows` gains exactly six DeepSeek plan cells (definition version 12, 52 + 13 + 3 cells). - Cell matrix (plan), DeepSeek 1017 on native OpenCode: `interrupt-redirect`, `reassign-midrun` and `tree-pause-release` run on `local`. `three-turn-revise` runs on Daytona with the per-turn lease (S49). `long-silent-command` (S52/S52b) and `queued-across-restart` (S55/U9) run on the Daytona warm lease. - `queue-steer` is the one exception. It must steer an active turn, and the OpenCode server driver declares `steering: false`, so the route answers `steering_unsupported` and the case cannot pass. The plan uses the existing steering-capable `everyday-workflows.runner-codex.daytona.queue-steer` cell for it (Greptile finding). No DeepSeek `queue-steer` cell exists. The other native Codex Daytona cells and the ACPX facades stay selectable through `--group reliability`, but they are outside the plan. - `runner-full-stack-e2e.yml`: adds the `testops_plan` bootstrap input (`product-regression` by default), sets the plan-specific bootstrap launch name and tags, and renames two jobs to plan-neutral names. The OpenCode steps now match `startsWith(matrix.profileId, 'runner-opencode')`, so the DeepSeek profile gets the pinned OpenCode binary, runner TypeScript, the local Node check and the remote provider pack. Before this change, the DeepSeek profile could not run in this workflow. `OPENROUTER_API_KEY`, `OPENAI_API_KEY` (only for `queue-steer`) and `DAYTONA_API_KEY` come from the same `matrix.credentialName` and `matrix.environmentId` gates that other cells use. - `runner-e2e-testops-registration.yml` + `reliability-testops-registration.mjs` (new): a trusted, master-only, allowlisted-actor, idempotent `workflow_dispatch`. `inspect` only reads. `register` reuses cases with the same full name, creates missing cases and the plan, and assigns the plan to plan 2's GitHub job with plan 2's job parameters. It writes a receipt with a ready-to-commit `reliabilityRegistration` snippet. Only this job gets `ALLURE_TOKEN`, and it never runs a provider cell. - **No schedule and not per PR.** The plan runs only from a TestOps launch or a manual dispatch. It is not in the daily or weekly cron (`scheduledSelection` excludes it). A weekly run would add a paid 50-minute Daytona cell every week, but these signatures move with release-risky changes, not with time. - Docs: `EVERYDAY-WORKFLOWS.md` (matrix, environment reasons, local coverage limits, registration steps), `README.md`, `doc/evals.md`. ### Registration still needed (requires TestOps credentials) 1. Dispatch **Runner E2E TestOps registration** from master with `operation=inspect`. 2. Dispatch it with `operation=register`. 3. Commit the `reliabilityRegistration` snippet from the run summary into `tests/runner-e2e/reliability-testops.mjs` in one reviewed PR. 4. Launch the plan from TestOps. Tag only a complete seven-case attempt `full-run`. ### What local plan cells cannot reproduce The local cells do not reproduce the Daytona-only variants: the remote `workspace_busy` save after Stop (S47), remote stop settlement during reassignment (S48), and a lost or replaced remote runner during a tree hold (S50). S51 (lost runner) has no case yet. The explicit-only native Codex Daytona extras still cover S47, S48 and S50 on a sandbox. ## Verification - `node --test tests/runner-e2e/product-regression-testops.test.mjs tests/runner-e2e/reliability-testops.test.mjs`: 26 pass. These tests cover fail-closed selection, partial or duplicate registration, plan mixing, environment binding, per-plan denominators, preserved failure classes, the CLI select → project → verify flow for a seven-case launch, registration idempotency and job checks, and the workflow inputs and credential gates. - `vitest` for `reliability-testops-catalog`, `product-regression-testops-catalog`, `everyday-reliability` and `catalog.test`: 98 pass. These tests bind the mapping to real catalog cells (profile, model, environment, credentials, single attempt). They also check that the cells stay out of `--all`, the suite, profile and environment selectors and the daily/weekly cron. Real grader output must survive projection. - `pnpm test:e2e:runner:typecheck`: pass. - `pnpm test:e2e:runner -- --list --suite everyday-workflows --group reliability` lists 16 cells: 6 DeepSeek plan cells, 7 native Codex cells (one is the `queue-steer` plan cell) and 3 ACPX extras. `--matrix-json` for the DeepSeek plan IDs emits `needsDaytona=true` and three Daytona OpenCode cells with `credentialName=OPENROUTER_API_KEY`. - Full `pnpm test:e2e:runner:unit`: the remaining failures happen the same way on an unchanged master checkout. They are in `workflow-security` (3), `controller-outage`, `operating-receipts`, `pi-*`, `private-framework-report` and `remote-native-bootstrap`. #19529 tracks that drift. This PR adds no failures. - Not run: paid cells, a TestOps registration and a TestOps launch. They need provider and TestOps credentials, which exist only in protected CI environments. Catalog and projection tests are not live qualification. ## Risks - The registration script calls TestOps APIs (`POST /api/testcase`, `/api/testplan`, `/assign`, `/jobparameter`) through the same endpoints as the chat catalog reconcile. Tests use fakes, not the live API. `inspect` comes first, and `register` is idempotent. - DeepSeek 1017 on Daytona has not run before. The first launch is the acceptance step for those three cells. - If a later case needs another capability that OpenCode lacks, it must use a capable profile in the same way as `queue-steer`. The catalog test pins the single exception. - The OpenCode workflow conditions now match any `runner-opencode*` profile. Today this covers `runner-opencode` and `runner-opencode-deepseek-1017` only. - If someone commits a bad `reliabilityRegistration`, module load fails for both plans. A unit test rejects any committed registration that is not empty or complete. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with tool use (shell, file edits, local test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2216 lines
111 KiB
YAML
2216 lines
111 KiB
YAML
name: Runner Full-Stack E2E
|
|
run-name: Runner Full-Stack E2E / ${{ github.event.schedule || inputs.cadence || 'manual' }}
|
|
|
|
on:
|
|
schedule:
|
|
- cron: "47 8 * * *"
|
|
- cron: "47 9 * * 0"
|
|
workflow_dispatch:
|
|
inputs:
|
|
cadence:
|
|
description: "Bounded daily ($20) or weekly ($100) selection; manual keeps explicit selectors"
|
|
type: choice
|
|
options: [manual, daily, weekly]
|
|
default: manual
|
|
target_branch:
|
|
description: "Branch in paperclipai/paperclip to test; the trusted workflow still runs from master"
|
|
type: string
|
|
required: false
|
|
max_parallel:
|
|
description: "Optional lower concurrency for this campaign (cannot exceed the configured limit)"
|
|
type: string
|
|
required: false
|
|
all:
|
|
description: "Run the complete paid matrix when no narrower selector is supplied"
|
|
type: boolean
|
|
default: true
|
|
group:
|
|
description: "Comma-separated groups (AND semantics: legacy,native,local,daytona,warm,core,breadth)"
|
|
type: string
|
|
required: false
|
|
suite:
|
|
description: "Comma-separated suite IDs"
|
|
type: string
|
|
required: false
|
|
profile:
|
|
description: "Comma-separated runner profile fixture IDs"
|
|
type: string
|
|
required: false
|
|
environment:
|
|
description: "Comma-separated environment fixture IDs"
|
|
type: string
|
|
required: false
|
|
case:
|
|
description: "Comma-separated task case fixture IDs"
|
|
type: string
|
|
required: false
|
|
id:
|
|
description: "Comma-separated full suite.profile.environment.case IDs; exclusive with other selectors"
|
|
type: string
|
|
required: false
|
|
testops_case:
|
|
description: "TestOps plan bootstrap: all, case name or case ID; leave empty for normal campaigns"
|
|
type: string
|
|
required: false
|
|
testops_plan:
|
|
description: "TestOps plan for a testops_case bootstrap; a TestOps launch names its own plan"
|
|
type: choice
|
|
options: [product-regression, reliability]
|
|
default: product-regression
|
|
ALLURE_JOB_RUN_ID:
|
|
description: "Allure TestOps service input; its requested selection takes precedence"
|
|
type: string
|
|
required: false
|
|
ALLURE_USERNAME:
|
|
description: "Allure TestOps service input"
|
|
type: string
|
|
required: false
|
|
image_only:
|
|
description: "Only build, sign and verify the Daytona image for the selected cells; skip every campaign job and paid gate"
|
|
type: boolean
|
|
default: false
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
concurrency:
|
|
group: runner-full-stack-e2e-${{ github.event_name == 'workflow_dispatch' && inputs.target_branch != '' && inputs.target_branch != github.event.repository.default_branch && format('development-{0}', inputs.target_branch) || format('protected-{0}', github.run_id) }}
|
|
# Development branch campaigns supersede older runs for the same target.
|
|
# Give protected/default-branch campaigns unique groups because GitHub also
|
|
# replaces pending runs when cancel-in-progress is false.
|
|
cancel-in-progress: ${{ github.event_name == 'workflow_dispatch' && inputs.target_branch != '' && inputs.target_branch != github.event.repository.default_branch }}
|
|
|
|
jobs:
|
|
authorize:
|
|
name: Authorize paid campaign
|
|
if: github.event_name != 'schedule' || (vars.RUNNER_FULL_STACK_E2E_NIGHTLY_ENABLED == 'true' && vars.RUNNER_FULL_STACK_E2E_OPERATIONS_ENABLED == 'true')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
permissions:
|
|
contents: read
|
|
outputs:
|
|
test_runner: ${{ steps.runner.outputs.runner }}
|
|
max_parallel_default: ${{ steps.runner.outputs.max_parallel_default }}
|
|
max_parallel_limit: ${{ steps.runner.outputs.max_parallel_limit }}
|
|
playwright_channel: ${{ steps.runner.outputs.playwright_channel }}
|
|
target_sha: ${{ steps.target.outputs.sha }}
|
|
target_ref: ${{ steps.target.outputs.ref }}
|
|
evidence_retention: ${{ steps.evidence_route.outputs.evidence_retention }}
|
|
manual_private_ids: ${{ steps.evidence_route.outputs.manual_private_ids }}
|
|
steps:
|
|
- name: Require default branch and allowlisted numeric actor IDs
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
REPOSITORY: ${{ github.repository }}
|
|
REF: ${{ github.ref }}
|
|
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
|
ACTOR: ${{ github.actor }}
|
|
ACTOR_ID: ${{ github.actor_id }}
|
|
TRIGGERING_ACTOR: ${{ github.triggering_actor }}
|
|
ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
|
|
run: |
|
|
set -euo pipefail
|
|
if [ "$REF" != "refs/heads/$DEFAULT_BRANCH" ]; then
|
|
echo "Paid runner E2E campaigns may run only from the default branch." >&2
|
|
exit 1
|
|
fi
|
|
if ! jq -e 'type == "array" and length > 0 and all(.[]; type == "number" and . > 0 and floor == .)' <<< "${ALLOWED_ACTOR_IDS:-}" >/dev/null; then
|
|
echo "RUNNER_E2E_ALLOWED_ACTOR_IDS must be a non-empty JSON array of numeric GitHub user IDs." >&2
|
|
exit 1
|
|
fi
|
|
# Retry transient transport/server failures, but never proceed without
|
|
# a successfully resolved identity and the allowlist checks below.
|
|
for attempt in 1 2 3; do
|
|
if triggering_actor_id="$(timeout 30s gh api "users/$TRIGGERING_ACTOR" --jq .id)"; then
|
|
break
|
|
fi
|
|
if [ "$attempt" = 3 ]; then exit 1; fi
|
|
sleep "$((attempt * 2))"
|
|
done
|
|
if [ "$triggering_actor_id" != "$ACTOR_ID" ] && [ "$TRIGGERING_ACTOR" = "$ACTOR" ]; then
|
|
echo "GitHub actor identity contexts disagree; refusing the paid run." >&2
|
|
exit 1
|
|
fi
|
|
candidates=("$triggering_actor_id" "$ACTOR_ID")
|
|
for candidate in "${candidates[@]}"; do
|
|
if ! jq -e --argjson candidate "$candidate" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null; then
|
|
echo "The initiating GitHub account is not authorized to run paid runner E2E campaigns." >&2
|
|
exit 1
|
|
fi
|
|
done
|
|
|
|
- name: Resolve requested repository branch to an immutable commit
|
|
id: target
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
REPOSITORY: ${{ github.repository }}
|
|
TARGET_BRANCH: ${{ inputs.target_branch || github.event.repository.default_branch }}
|
|
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
|
EVENT_NAME: ${{ github.event_name }}
|
|
EVAL_CADENCE: ${{ inputs.cadence || 'manual' }}
|
|
IMAGE_ONLY: ${{ inputs.image_only == true }}
|
|
TESTOPS_REQUESTED: ${{ inputs.ALLURE_JOB_RUN_ID != '' || inputs.testops_case != '' }}
|
|
run: |
|
|
set -euo pipefail
|
|
if [ "$IMAGE_ONLY" = true ] && [ "$TESTOPS_REQUESTED" = true ]; then
|
|
echo "image_only builds publish no results; it cannot bind a TestOps selection." >&2
|
|
exit 1
|
|
fi
|
|
if [ -z "$TARGET_BRANCH" ] || [[ "$TARGET_BRANCH" == refs/* ]]; then
|
|
echo "target_branch must name a branch in this repository without a refs/ prefix." >&2
|
|
exit 1
|
|
fi
|
|
if [ "$EVENT_NAME" = schedule ] || [ "$EVAL_CADENCE" != manual ]; then
|
|
if [ "$TARGET_BRANCH" != "$DEFAULT_BRANCH" ]; then
|
|
echo "Bounded campaigns require the workflow's default-branch commit." >&2
|
|
exit 1
|
|
fi
|
|
target_sha="$GITHUB_SHA"
|
|
else
|
|
encoded_branch="$(jq -rn --arg branch "$TARGET_BRANCH" '$branch | @uri')"
|
|
target_sha="$(gh api -X GET "repos/$REPOSITORY/branches/$encoded_branch" --jq .commit.sha)"
|
|
fi
|
|
if ! [[ "$target_sha" =~ ^[0-9a-f]{40}$ ]]; then
|
|
echo "The requested repository branch did not resolve to a commit." >&2
|
|
exit 1
|
|
fi
|
|
echo "sha=$target_sha" >> "$GITHUB_OUTPUT"
|
|
echo "ref=refs/heads/$TARGET_BRANCH" >> "$GITHUB_OUTPUT"
|
|
echo "Resolved the requested repository branch to $target_sha."
|
|
|
|
- name: Select paid test runner
|
|
id: runner
|
|
env:
|
|
AWS_PAID_RUNNER_ENABLED: ${{ vars.RUNNER_E2E_AWS_ENABLED }}
|
|
EVENT_NAME: ${{ github.event_name }}
|
|
EVAL_CADENCE: ${{ inputs.cadence || 'manual' }}
|
|
REPOSITORY_PRIVATE: ${{ github.event.repository.private }}
|
|
run: |
|
|
set -euo pipefail
|
|
github_runner='ubuntu-latest'
|
|
aws_runner='runs-on/fleet=paperclip-public-pr-x64/env=public-ci'
|
|
bounded=false
|
|
if [ "$EVENT_NAME" = schedule ] || [ "$EVAL_CADENCE" != manual ]; then
|
|
if [ "$REPOSITORY_PRIVATE" != false ]; then
|
|
echo 'Bounded campaigns require a public repository and standard GitHub-hosted compute.' >&2
|
|
exit 1
|
|
fi
|
|
bounded=true
|
|
fi
|
|
|
|
if [ "$AWS_PAID_RUNNER_ENABLED" = true ] && [ "$bounded" = false ]; then
|
|
{
|
|
echo "runner=$aws_runner"
|
|
echo "max_parallel_default=100"
|
|
echo "max_parallel_limit=100"
|
|
echo "playwright_channel=chrome"
|
|
} >> "$GITHUB_OUTPUT"
|
|
echo '::notice title=Paid runner routing::Using an ephemeral RunsOn Fleet runner'
|
|
else
|
|
{
|
|
echo "runner=$github_runner"
|
|
echo "max_parallel_default=32"
|
|
echo "max_parallel_limit=57"
|
|
echo "playwright_channel="
|
|
} >> "$GITHUB_OUTPUT"
|
|
echo '::notice title=Paid runner routing::Using a standard GitHub-hosted runner'
|
|
fi
|
|
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
ref: ${{ github.sha }}
|
|
persist-credentials: false
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
|
|
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
|
|
env:
|
|
NPM_CONFIG_AUDIT: "false"
|
|
NPM_CONFIG_FUND: "false"
|
|
NPM_CONFIG_UPDATE_NOTIFIER: "false"
|
|
with:
|
|
version: 9.15.4
|
|
run_install: false
|
|
|
|
- name: Preflight reviewed report dependencies before paid admission
|
|
run: node tests/runner-e2e/prepare-report-lock.mjs
|
|
|
|
- name: Validate report dependency manifests offline
|
|
run: pnpm install --lockfile-only --frozen-lockfile --ignore-scripts --offline
|
|
|
|
- name: Classify manual evidence before any protected credential job
|
|
id: evidence_route
|
|
env:
|
|
EVENT_NAME: ${{ github.event_name }}
|
|
EVAL_CADENCE: ${{ inputs.cadence || 'manual' }}
|
|
SELECT_ID: ${{ inputs.id }}
|
|
SELECT_SUITE: ${{ inputs.suite }}
|
|
SELECT_GROUP: ${{ inputs.group }}
|
|
SELECT_PROFILE: ${{ inputs.profile }}
|
|
SELECT_ENVIRONMENT: ${{ inputs.environment }}
|
|
SELECT_CASE: ${{ inputs.case }}
|
|
TEST_RUNNER: ${{ steps.runner.outputs.runner }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
run: node tests/runner-e2e/manual-private-policy.mjs
|
|
|
|
target_lock:
|
|
name: Resolve target pnpm lockfile
|
|
needs: authorize
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
permissions:
|
|
contents: read
|
|
outputs:
|
|
artifact_id: ${{ steps.upload.outputs.artifact-id }}
|
|
lock_sha256: ${{ steps.lock.outputs.sha256 }}
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
ref: ${{ needs.authorize.outputs.target_sha }}
|
|
persist-credentials: false
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
|
|
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
|
|
env:
|
|
NPM_CONFIG_AUDIT: "false"
|
|
NPM_CONFIG_FUND: "false"
|
|
NPM_CONFIG_UPDATE_NOTIFIER: "false"
|
|
with:
|
|
version: 9.15.4
|
|
|
|
- name: Resolve target lockfile without lifecycle scripts
|
|
id: lock
|
|
run: |
|
|
set -euo pipefail
|
|
pnpm install --ignore-scripts --no-frozen-lockfile --lockfile-only
|
|
test -s pnpm-lock.yaml
|
|
unexpected="$(git status --short | awk '$2 != "pnpm-lock.yaml" { print }')"
|
|
if [ -n "$unexpected" ]; then
|
|
echo "Lockfile resolution changed files other than pnpm-lock.yaml:" >&2
|
|
echo "$unexpected" >&2
|
|
exit 1
|
|
fi
|
|
echo "sha256=$(sha256sum pnpm-lock.yaml | cut -d ' ' -f 1)" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Bound scheduled lockfile upload
|
|
if: github.event_name == 'schedule' || (inputs.cadence != '' && inputs.cadence != 'manual')
|
|
run: node tests/runner-e2e/artifact-budget.mjs check lock pnpm-lock.yaml
|
|
|
|
- name: Upload resolved target lockfile
|
|
id: upload
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: runner-e2e-target-pnpm-lock-${{ github.run_id }}-${{ github.run_attempt }}
|
|
path: pnpm-lock.yaml
|
|
retention-days: 30
|
|
if-no-files-found: error
|
|
|
|
testops_selection:
|
|
name: Bind TestOps plan selection
|
|
needs: authorize
|
|
if: github.event_name == 'workflow_dispatch' && (inputs.ALLURE_JOB_RUN_ID != '' || inputs.testops_case != '')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
environment: runner-e2e-testops-reporting
|
|
permissions:
|
|
contents: read
|
|
actions: read
|
|
outputs:
|
|
ids: ${{ steps.selection.outputs.ids }}
|
|
job_run_started: ${{ steps.selection.outputs.job_run_started }}
|
|
job_run_id: ${{ steps.selection.outputs.job_run_id }}
|
|
selection_sha256: ${{ steps.selection.outputs.selection_sha256 }}
|
|
launch_url: ${{ steps.selection.outputs.launch_url }}
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
ref: ${{ github.sha }}
|
|
persist-credentials: false
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
- uses: allure-framework/setup-allurectl@2d89ee0b7ba2f59dbe9849b6df5763cab78e14db # v1
|
|
with:
|
|
allurectl-version: '2.23.1'
|
|
allure-endpoint: https://paperclip.testops.cloud
|
|
allure-project-id: '1'
|
|
allure-token: ${{ secrets.ALLURE_TOKEN }}
|
|
- name: Resolve only admitted plan cases before provider jobs
|
|
id: selection
|
|
env:
|
|
SOURCE_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
EVAL_CADENCE: ${{ inputs.cadence || 'manual' }}
|
|
SELECT_ID: ${{ inputs.id }}
|
|
SELECT_SUITE: ${{ inputs.suite }}
|
|
SELECT_GROUP: ${{ inputs.group }}
|
|
SELECT_PROFILE: ${{ inputs.profile }}
|
|
SELECT_ENVIRONMENT: ${{ inputs.environment }}
|
|
SELECT_CASE: ${{ inputs.case }}
|
|
TESTOPS_CASE: ${{ inputs.testops_case }}
|
|
TESTOPS_PLAN: ${{ inputs.testops_plan || 'product-regression' }}
|
|
ALLURE_JOB_RUN_ID: ${{ inputs.ALLURE_JOB_RUN_ID }}
|
|
ALLURE_USERNAME: ${{ inputs.ALLURE_USERNAME }}
|
|
# Names only a bootstrap launch; a TestOps launch already has its plan's name.
|
|
ALLURE_LAUNCH_NAME: ${{ inputs.testops_plan == 'reliability' && 'Reliability' || 'Product regression' }} · ${{ github.run_number }}
|
|
ALLURE_LAUNCH_TAGS: ${{ inputs.testops_plan == 'reliability' && 'reliability' || 'product-regression' }},github-actions
|
|
run: node tests/runner-e2e/product-regression-testops-cli.mjs select product-testops/selection.json
|
|
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: product-testops-selection-${{ github.run_id }}-${{ github.run_attempt }}
|
|
path: product-testops/selection.json
|
|
if-no-files-found: error
|
|
retention-days: 30
|
|
|
|
catalog:
|
|
name: Validate catalog and select cells
|
|
needs: [authorize, target_lock, testops_selection]
|
|
if: >-
|
|
always() && needs.authorize.result == 'success' && needs.target_lock.result == 'success' &&
|
|
(needs.testops_selection.result == 'success' || needs.testops_selection.result == 'skipped') &&
|
|
(github.event_name != 'schedule' || (vars.RUNNER_FULL_STACK_E2E_NIGHTLY_ENABLED == 'true' && vars.RUNNER_FULL_STACK_E2E_OPERATIONS_ENABLED == 'true'))
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
permissions:
|
|
contents: read
|
|
outputs:
|
|
matrix: ${{ steps.catalog.outputs.matrix }}
|
|
operations_cadence: ${{ steps.catalog.outputs.operations_cadence }}
|
|
operating_overhead: ${{ steps.catalog.outputs.operating_overhead }}
|
|
needs_daytona: ${{ steps.catalog.outputs.needs_daytona }}
|
|
needs_runner_typescript: ${{ steps.catalog.outputs.needs_runner_typescript }}
|
|
needs_native_binaries: ${{ steps.catalog.outputs.needs_native_binaries }}
|
|
needs_remote_provider_pack: ${{ steps.catalog.outputs.needs_remote_provider_pack }}
|
|
execution_ids: ${{ steps.catalog.outputs.execution_ids }}
|
|
max_parallel: ${{ steps.catalog.outputs.max_parallel }}
|
|
daytona_image_content_id: ${{ steps.daytona_image_content.outputs.content_id }}
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
ref: ${{ needs.authorize.outputs.target_sha }}
|
|
persist-credentials: false
|
|
|
|
- name: Download resolved target lockfile
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
artifact-ids: ${{ needs.target_lock.outputs.artifact_id }}
|
|
path: ${{ runner.temp }}/runner-e2e-target-lock
|
|
|
|
- name: Restore resolved target lockfile
|
|
env:
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
EXPECTED_LOCK_SHA256: ${{ needs.target_lock.outputs.lock_sha256 }}
|
|
run: |
|
|
set -euo pipefail
|
|
test "$(git rev-parse HEAD)" = "$TARGET_SHA"
|
|
lock="$RUNNER_TEMP/runner-e2e-target-lock/pnpm-lock.yaml"
|
|
test -f "$lock"
|
|
test "$(find "$(dirname "$lock")" -type f | wc -l | tr -d ' ')" = 1
|
|
test "$(sha256sum "$lock" | cut -d ' ' -f 1)" = "$EXPECTED_LOCK_SHA256"
|
|
cp "$lock" pnpm-lock.yaml
|
|
test "$(sha256sum pnpm-lock.yaml | cut -d ' ' -f 1)" = "$EXPECTED_LOCK_SHA256"
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
|
|
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
|
|
env:
|
|
NPM_CONFIG_AUDIT: "false"
|
|
NPM_CONFIG_FUND: "false"
|
|
NPM_CONFIG_UPDATE_NOTIFIER: "false"
|
|
with:
|
|
version: 9.15.4
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
cache: pnpm
|
|
|
|
- run: pnpm install --frozen-lockfile
|
|
|
|
# The v2 contract fails closed unless every Docker FROM is digest-pinned,
|
|
# and hashes those exact base references into the immutable image tag.
|
|
- name: Compute Daytona image content ID with pinned bases
|
|
id: daytona_image_content
|
|
run: echo "content_id=$(pnpm --silent test:e2e:runner:image-id)" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Validate selectors and emit matrix
|
|
id: catalog
|
|
env:
|
|
EVENT_NAME: ${{ github.event_name }}
|
|
EVAL_EVENT: ${{ github.event_name }}
|
|
EVAL_SCHEDULE: ${{ github.event.schedule }}
|
|
EVAL_CADENCE: ${{ inputs.cadence || 'manual' }}
|
|
EVAL_OPERATIONS_ENABLED: ${{ vars.RUNNER_FULL_STACK_E2E_OPERATIONS_ENABLED }}
|
|
EVAL_OPERATING_OVERHEAD_JSON: ${{ vars.RUNNER_E2E_OPERATING_OVERHEAD_JSON }}
|
|
EVAL_TEST_RUNNER: ${{ needs.authorize.outputs.test_runner }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
SELECT_ALL: ${{ inputs.all }}
|
|
SELECT_SUITE: ${{ inputs.suite }}
|
|
SELECT_GROUP: ${{ inputs.group }}
|
|
SELECT_PROFILE: ${{ inputs.profile }}
|
|
SELECT_ENVIRONMENT: ${{ inputs.environment }}
|
|
SELECT_CASE: ${{ inputs.case }}
|
|
SELECT_ID: ${{ needs.testops_selection.outputs.ids || inputs.id }}
|
|
MAX_PARALLEL: ${{ vars.RUNNER_E2E_MAX_PARALLEL || needs.authorize.outputs.max_parallel_default }}
|
|
MAX_PARALLEL_LIMIT: ${{ needs.authorize.outputs.max_parallel_limit }}
|
|
REQUESTED_MAX_PARALLEL: ${{ inputs.max_parallel }}
|
|
run: |
|
|
set -euo pipefail
|
|
if [ "$EVAL_EVENT" = schedule ] || [ "$EVAL_CADENCE" != manual ]; then
|
|
test "$TARGET_SHA" = "$GITHUB_SHA"
|
|
test -z "${SELECT_SUITE:-}${SELECT_GROUP:-}${SELECT_PROFILE:-}${SELECT_ENVIRONMENT:-}${SELECT_CASE:-}${SELECT_ID:-}"
|
|
catalog_json="$(node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/scheduled-operations.ts matrix)"
|
|
cadence="$EVAL_CADENCE"
|
|
if [ "$EVAL_EVENT" = schedule ]; then
|
|
case "$EVAL_SCHEDULE" in
|
|
'47 8 * * *') cadence=daily ;;
|
|
'47 9 * * 0') cadence=weekly ;;
|
|
*) exit 1 ;;
|
|
esac
|
|
fi
|
|
{
|
|
echo "operations_cadence=$cadence"
|
|
echo "operating_overhead=$(jq -c '.overhead' <<< "$catalog_json")"
|
|
echo "matrix=$(jq -c '{include: .include}' <<< "$catalog_json")"
|
|
echo "execution_ids=$(jq -c '.executionIds' <<< "$catalog_json")"
|
|
echo "needs_daytona=false"
|
|
echo "needs_runner_typescript=true"
|
|
echo "needs_native_binaries=true"
|
|
echo "needs_remote_provider_pack=false"
|
|
echo "max_parallel=1"
|
|
} >> "$GITHUB_OUTPUT"
|
|
exit 0
|
|
fi
|
|
args=(--matrix-json)
|
|
add_values() {
|
|
local flag="$1"
|
|
local values="$2"
|
|
local value
|
|
IFS=',' read -ra entries <<< "$values"
|
|
for value in "${entries[@]}"; do
|
|
value="${value#"${value%%[![:space:]]*}"}"
|
|
value="${value%"${value##*[![:space:]]}"}"
|
|
if [ -n "$value" ]; then
|
|
args+=("$flag" "$value")
|
|
fi
|
|
done
|
|
}
|
|
explicit=false
|
|
if [ -n "${SELECT_ID:-}" ]; then
|
|
if [ -n "${SELECT_SUITE:-}${SELECT_GROUP:-}${SELECT_PROFILE:-}${SELECT_ENVIRONMENT:-}${SELECT_CASE:-}" ]; then
|
|
echo "The id selector is exclusive with suite/group/profile/environment/case" >&2
|
|
exit 1
|
|
fi
|
|
add_values --id "$SELECT_ID"
|
|
explicit=true
|
|
else
|
|
for pair in \
|
|
"--suite:${SELECT_SUITE:-}" \
|
|
"--group:${SELECT_GROUP:-}" \
|
|
"--profile:${SELECT_PROFILE:-}" \
|
|
"--environment:${SELECT_ENVIRONMENT:-}" \
|
|
"--case:${SELECT_CASE:-}"
|
|
do
|
|
flag="${pair%%:*}"
|
|
values="${pair#*:}"
|
|
if [ -n "$values" ]; then
|
|
add_values "$flag" "$values"
|
|
explicit=true
|
|
fi
|
|
done
|
|
fi
|
|
if [ "$explicit" = false ] && { [ "$EVENT_NAME" = schedule ] || [ "${SELECT_ALL:-false}" = true ]; }; then
|
|
args+=(--all)
|
|
fi
|
|
catalog_json="$(pnpm --silent test:e2e:runner -- "${args[@]}")"
|
|
{
|
|
echo "matrix=$(jq -c '{include: .include}' <<< "$catalog_json")"
|
|
echo "needs_daytona=$(jq -r '.needsDaytona' <<< "$catalog_json")"
|
|
echo "needs_runner_typescript=$(jq -r '[.include[] | select((.profileId | startswith("runner-opencode")) or (.profileId | startswith("runner-acpx-")) or (.suiteId == "openrouter-model-breadth"))] | length > 0' <<< "$catalog_json")"
|
|
echo "needs_native_binaries=$(jq -r '[.include[] | select((.profileId | startswith("runner-")) or (.suiteId == "openrouter-model-breadth"))] | length > 0' <<< "$catalog_json")"
|
|
echo "needs_remote_provider_pack=$(jq -r '[.include[] | select((.environmentId == "daytona") and ((.profileId | startswith("runner-opencode")) or (.profileId | startswith("runner-acpx-"))))] | length > 0' <<< "$catalog_json")"
|
|
echo "execution_ids=$(jq -c '.executionIds' <<< "$catalog_json")"
|
|
} >> "$GITHUB_OUTPUT"
|
|
if ! [[ "$MAX_PARALLEL_LIMIT" =~ ^[1-9][0-9]*$ ]] || [ "$MAX_PARALLEL_LIMIT" -gt 100 ]; then
|
|
echo "Runner selection emitted an invalid max-parallel limit." >&2
|
|
exit 1
|
|
fi
|
|
if ! [[ "$MAX_PARALLEL" =~ ^[1-9][0-9]*$ ]] || [ "$MAX_PARALLEL" -gt "$MAX_PARALLEL_LIMIT" ]; then
|
|
echo "RUNNER_E2E_MAX_PARALLEL must be an integer from 1 through $MAX_PARALLEL_LIMIT for the selected runner." >&2
|
|
exit 1
|
|
fi
|
|
if [ -n "${REQUESTED_MAX_PARALLEL:-}" ]; then
|
|
if ! [[ "$REQUESTED_MAX_PARALLEL" =~ ^[1-9][0-9]{0,2}$ ]] || [ "$REQUESTED_MAX_PARALLEL" -gt "$MAX_PARALLEL" ]; then
|
|
echo "max_parallel must be an integer from 1 through the configured campaign limit." >&2
|
|
exit 1
|
|
fi
|
|
MAX_PARALLEL="$REQUESTED_MAX_PARALLEL"
|
|
fi
|
|
echo "max_parallel=$MAX_PARALLEL" >> "$GITHUB_OUTPUT"
|
|
|
|
daytona_image:
|
|
name: Publish verified Daytona image
|
|
needs: [authorize, target_lock, catalog]
|
|
# testops_selection is skipped outside TestOps campaigns. GitHub's implicit
|
|
# success() also checks skipped ancestors, so jobs below catalog state
|
|
# their real dependency results explicitly.
|
|
if: ${{ !cancelled() && needs.catalog.result == 'success' }}
|
|
# Keep image builds on the same authorized fleet as campaign execution.
|
|
runs-on: ${{ needs.authorize.outputs.test_runner }}
|
|
timeout-minutes: 45
|
|
permissions:
|
|
contents: read
|
|
packages: write
|
|
id-token: write
|
|
outputs:
|
|
image: ${{ steps.image.outputs.image }}
|
|
source_revision: ${{ steps.image.outputs.source_revision }}
|
|
content_id: ${{ steps.image.outputs.content_id }}
|
|
steps:
|
|
- if: needs.catalog.outputs.needs_daytona == 'true'
|
|
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
ref: ${{ needs.authorize.outputs.target_sha }}
|
|
persist-credentials: false
|
|
|
|
- name: Download resolved target lockfile
|
|
if: needs.catalog.outputs.needs_daytona == 'true'
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
artifact-ids: ${{ needs.target_lock.outputs.artifact_id }}
|
|
path: ${{ runner.temp }}/runner-e2e-target-lock
|
|
|
|
- name: Restore resolved target lockfile
|
|
if: needs.catalog.outputs.needs_daytona == 'true'
|
|
env:
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
EXPECTED_LOCK_SHA256: ${{ needs.target_lock.outputs.lock_sha256 }}
|
|
run: |
|
|
set -euo pipefail
|
|
test "$(git rev-parse HEAD)" = "$TARGET_SHA"
|
|
lock="$RUNNER_TEMP/runner-e2e-target-lock/pnpm-lock.yaml"
|
|
test -f "$lock"
|
|
test "$(find "$(dirname "$lock")" -type f | wc -l | tr -d ' ')" = 1
|
|
test "$(sha256sum "$lock" | cut -d ' ' -f 1)" = "$EXPECTED_LOCK_SHA256"
|
|
cp "$lock" pnpm-lock.yaml
|
|
test "$(sha256sum pnpm-lock.yaml | cut -d ' ' -f 1)" = "$EXPECTED_LOCK_SHA256"
|
|
|
|
- name: No Daytona image needed
|
|
id: local_only
|
|
if: needs.catalog.outputs.needs_daytona != 'true'
|
|
run: echo "image=" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Set up Docker Buildx
|
|
if: needs.catalog.outputs.needs_daytona == 'true'
|
|
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
|
|
|
|
- name: Log into GHCR
|
|
if: needs.catalog.outputs.needs_daytona == 'true'
|
|
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4
|
|
with:
|
|
registry: ghcr.io
|
|
username: ${{ github.actor }}
|
|
password: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
- name: Install cosign
|
|
if: needs.catalog.outputs.needs_daytona == 'true'
|
|
uses: sigstore/cosign-installer@398d4b0eeef1380460a10c8013a76f728fb906ac # v3
|
|
|
|
- name: Reuse or publish immutable image
|
|
id: image
|
|
env:
|
|
NEEDS_DAYTONA: ${{ needs.catalog.outputs.needs_daytona }}
|
|
IMAGE_CONTENT_ID: ${{ needs.catalog.outputs.daytona_image_content_id }}
|
|
IMAGE_TAG: ghcr.io/paperclipai/paperclip-daytona-runner:e2e-content-${{ needs.catalog.outputs.daytona_image_content_id }}
|
|
IMAGE_CACHE: ghcr.io/paperclipai/paperclip-daytona-runner:e2e-buildcache-amd64
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
TARGET_LOCK_SHA256: ${{ needs.target_lock.outputs.lock_sha256 }}
|
|
TARGET_REF: ${{ needs.authorize.outputs.target_ref }}
|
|
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
|
run: |
|
|
set -euo pipefail
|
|
if [ "$NEEDS_DAYTONA" != true ]; then
|
|
{
|
|
echo "image="
|
|
echo "source_revision="
|
|
echo "content_id="
|
|
} >> "$GITHUB_OUTPUT"
|
|
exit 0
|
|
fi
|
|
[[ "$IMAGE_CONTENT_ID" =~ ^[0-9a-f]{64}$ ]]
|
|
[[ "$TARGET_LOCK_SHA256" =~ ^[0-9a-f]{64}$ ]]
|
|
identity="^https://github.com/${GITHUB_REPOSITORY}/.github/workflows/runner-full-stack-e2e.yml@"
|
|
if docker buildx imagetools inspect "$IMAGE_TAG" >/dev/null 2>&1; then
|
|
digest="$(docker buildx imagetools inspect "$IMAGE_TAG" --format '{{json .Manifest.Digest}}' | tr -d '"')"
|
|
else
|
|
cache_args=(
|
|
--cache-from "type=registry,ref=${IMAGE_CACHE}"
|
|
)
|
|
if [ "$TARGET_REF" = "refs/heads/$DEFAULT_BRANCH" ]; then
|
|
cache_args+=(
|
|
--cache-to "type=registry,ref=${IMAGE_CACHE},mode=max"
|
|
)
|
|
echo '::notice title=Daytona image cache::Publishing cache from the trusted default-branch target'
|
|
else
|
|
echo '::notice title=Daytona image cache::Using the default-branch cache without publishing development-branch layers'
|
|
fi
|
|
docker buildx build \
|
|
--platform linux/amd64 \
|
|
--build-arg "PAPERCLIP_RUNNER_CONTENT_ID=${IMAGE_CONTENT_ID}" \
|
|
--build-arg "PAPERCLIP_RUNNER_SOURCE_REVISION=${TARGET_SHA}" \
|
|
--build-arg "PAPERCLIP_RUNNER_LOCK_SHA256=${TARGET_LOCK_SHA256}" \
|
|
--file docker/daytona-runner/Dockerfile \
|
|
--tag "$IMAGE_TAG" \
|
|
"${cache_args[@]}" \
|
|
--push \
|
|
.
|
|
digest="$(docker buildx imagetools inspect "$IMAGE_TAG" --format '{{json .Manifest.Digest}}' | tr -d '"')"
|
|
cosign sign --yes "$IMAGE_TAG@$digest"
|
|
fi
|
|
cosign verify \
|
|
--certificate-identity-regexp "$identity" \
|
|
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
|
|
"$IMAGE_TAG@$digest" >/dev/null
|
|
immutable="${IMAGE_TAG%:*}@$digest"
|
|
# The Dockerfile's final two RUN steps execute the runner metadata,
|
|
# transport-mode, provider-pack JSON, and pinned ACP binary checks as
|
|
# root and as the unprivileged Daytona user. Buildx reads the signed
|
|
# digest's OCI config directly from GHCR, so verification does not
|
|
# download the image's large filesystem layers. Logging out first
|
|
# preserves the proof that Daytona can retrieve this public image
|
|
# without the workflow's package credentials.
|
|
docker logout ghcr.io >/dev/null
|
|
image_config="$(docker buildx imagetools inspect "$immutable" \
|
|
--format '{{json .Image}}')"
|
|
published_content_id="$(jq -r '.config.Labels["io.paperclip.runner.content-id"] // empty' <<< "$image_config")"
|
|
source_revision="$(jq -r '.config.Labels["org.opencontainers.image.revision"] // empty' <<< "$image_config")"
|
|
test "$published_content_id" = "$IMAGE_CONTENT_ID"
|
|
[[ "$source_revision" =~ ^[0-9a-f]{40}$ ]]
|
|
jq -e \
|
|
'.architecture == "amd64" and
|
|
.os == "linux" and
|
|
.config.User == "daytona" and
|
|
(.config.Env | any(startswith("PAPERCLIP_RUNNER_PROVIDER_PACK_ROOT=")))' \
|
|
<<< "$image_config" >/dev/null
|
|
{
|
|
echo "image=$immutable"
|
|
echo "source_revision=$source_revision"
|
|
echo "content_id=$published_content_id"
|
|
} >> "$GITHUB_OUTPUT"
|
|
|
|
build_runner_artifacts:
|
|
name: Build reusable runner campaign artifacts
|
|
needs: [authorize, target_lock, catalog]
|
|
if: >-
|
|
!cancelled() && needs.catalog.result == 'success' && inputs.image_only != true &&
|
|
(github.event_name != 'schedule' || (vars.RUNNER_FULL_STACK_E2E_NIGHTLY_ENABLED == 'true' && vars.RUNNER_FULL_STACK_E2E_OPERATIONS_ENABLED == 'true'))
|
|
# Compile native binaries on the same reviewed image used to execute them,
|
|
# avoiding libc/architecture drift between GitHub-hosted and AWS lanes.
|
|
runs-on: ${{ needs.authorize.outputs.test_runner }}
|
|
timeout-minutes: 20
|
|
permissions:
|
|
contents: read
|
|
outputs:
|
|
build_artifact_name: ${{ steps.build_artifact_name.outputs.name }}
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
ref: ${{ needs.authorize.outputs.target_sha }}
|
|
persist-credentials: false
|
|
|
|
- name: Download resolved target lockfile
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
artifact-ids: ${{ needs.target_lock.outputs.artifact_id }}
|
|
path: ${{ runner.temp }}/runner-e2e-target-lock
|
|
|
|
- name: Restore resolved target lockfile
|
|
env:
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
EXPECTED_LOCK_SHA256: ${{ needs.target_lock.outputs.lock_sha256 }}
|
|
run: |
|
|
set -euo pipefail
|
|
test "$(git rev-parse HEAD)" = "$TARGET_SHA"
|
|
lock="$RUNNER_TEMP/runner-e2e-target-lock/pnpm-lock.yaml"
|
|
test -f "$lock"
|
|
test "$(find "$(dirname "$lock")" -type f | wc -l | tr -d ' ')" = 1
|
|
test "$(sha256sum "$lock" | cut -d ' ' -f 1)" = "$EXPECTED_LOCK_SHA256"
|
|
cp "$lock" pnpm-lock.yaml
|
|
test "$(sha256sum pnpm-lock.yaml | cut -d ' ' -f 1)" = "$EXPECTED_LOCK_SHA256"
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
|
|
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
|
|
env:
|
|
NPM_CONFIG_AUDIT: "false"
|
|
NPM_CONFIG_FUND: "false"
|
|
NPM_CONFIG_UPDATE_NOTIFIER: "false"
|
|
with:
|
|
version: 9.15.4
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
cache: pnpm
|
|
|
|
- run: pnpm install --frozen-lockfile --ignore-scripts
|
|
|
|
# build:typescript also builds the eval-kernel dependency, so the two
|
|
# TypeScript trees are compiled at most once in this campaign.
|
|
- name: Build shared TypeScript and native runner outputs
|
|
env:
|
|
NEEDS_RUNNER_TYPESCRIPT: ${{ needs.catalog.outputs.needs_runner_typescript }}
|
|
NEEDS_NATIVE_BINARIES: ${{ needs.catalog.outputs.needs_native_binaries }}
|
|
run: |
|
|
set -euo pipefail
|
|
if [ "$NEEDS_RUNNER_TYPESCRIPT" = true ]; then
|
|
pnpm --filter @paperclipai/paperclip-runner build:typescript
|
|
else
|
|
pnpm --filter @paperclipai/paperclip-eval-kernel build
|
|
fi
|
|
if [ "$NEEDS_NATIVE_BINARIES" = true ]; then
|
|
pnpm --filter @paperclipai/paperclip-runner build:runner-binaries
|
|
fi
|
|
|
|
- name: Package immutable campaign outputs
|
|
env:
|
|
NEEDS_RUNNER_TYPESCRIPT: ${{ needs.catalog.outputs.needs_runner_typescript }}
|
|
NEEDS_NATIVE_BINARIES: ${{ needs.catalog.outputs.needs_native_binaries }}
|
|
run: |
|
|
set -euo pipefail
|
|
binary_root="packages/paperclip-runner/runner/target/debug"
|
|
binaries=(
|
|
conformance-tracer
|
|
paperclip-runnerd
|
|
fake-harness
|
|
fake-codex-app-server
|
|
fake-acpx-sidecar
|
|
)
|
|
archive_paths=(
|
|
packages/paperclip-eval-kernel/dist
|
|
)
|
|
if [ "$NEEDS_RUNNER_TYPESCRIPT" = true ]; then
|
|
test -d packages/paperclip-runner/dist
|
|
archive_paths+=(packages/paperclip-runner/dist)
|
|
fi
|
|
if [ "$NEEDS_NATIVE_BINARIES" = true ]; then
|
|
for binary in "${binaries[@]}"; do
|
|
test -x "$binary_root/$binary"
|
|
archive_paths+=("$binary_root/$binary")
|
|
done
|
|
fi
|
|
tar --create --gzip \
|
|
--file runner-e2e-build-bundle.tar.gz \
|
|
"${archive_paths[@]}"
|
|
sha256sum runner-e2e-build-bundle.tar.gz > runner-e2e-build-bundle.tar.gz.sha256
|
|
|
|
- name: Name immutable shared campaign outputs
|
|
id: build_artifact_name
|
|
env:
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
run: echo "name=runner-e2e-build-${TARGET_SHA}-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Bound scheduled build upload
|
|
if: needs.catalog.outputs.operations_cadence != ''
|
|
run: node tests/runner-e2e/artifact-budget.mjs check build runner-e2e-build-bundle.tar.gz runner-e2e-build-bundle.tar.gz.sha256
|
|
|
|
- name: Upload immutable shared campaign outputs
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: ${{ steps.build_artifact_name.outputs.name }}
|
|
path: |
|
|
runner-e2e-build-bundle.tar.gz
|
|
runner-e2e-build-bundle.tar.gz.sha256
|
|
retention-days: 1
|
|
compression-level: 0
|
|
if-no-files-found: error
|
|
|
|
build_remote_provider_pack:
|
|
name: Build reusable remote provider pack
|
|
needs:
|
|
[authorize, target_lock, catalog, daytona_image, build_runner_artifacts]
|
|
if: >-
|
|
!cancelled() && needs.catalog.result == 'success' && needs.daytona_image.result == 'success' &&
|
|
needs.build_runner_artifacts.result == 'success' && inputs.image_only != true &&
|
|
(github.event_name != 'schedule' || (vars.RUNNER_FULL_STACK_E2E_NIGHTLY_ENABLED == 'true' && vars.RUNNER_FULL_STACK_E2E_OPERATIONS_ENABLED == 'true'))
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 15
|
|
permissions:
|
|
contents: read
|
|
outputs:
|
|
provider_pack_artifact_name: ${{ steps.provider_pack_artifact_name.outputs.name }}
|
|
steps:
|
|
- name: No remote provider pack needed
|
|
if: needs.catalog.outputs.needs_remote_provider_pack != 'true'
|
|
run: echo "Selected cells do not require a remote provider pack."
|
|
|
|
- if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
ref: ${{ needs.authorize.outputs.target_sha }}
|
|
persist-credentials: false
|
|
|
|
- name: Download resolved target lockfile
|
|
if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
artifact-ids: ${{ needs.target_lock.outputs.artifact_id }}
|
|
path: ${{ runner.temp }}/runner-e2e-target-lock
|
|
|
|
- name: Restore resolved target lockfile
|
|
if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
env:
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
EXPECTED_LOCK_SHA256: ${{ needs.target_lock.outputs.lock_sha256 }}
|
|
run: |
|
|
set -euo pipefail
|
|
test "$(git rev-parse HEAD)" = "$TARGET_SHA"
|
|
lock="$RUNNER_TEMP/runner-e2e-target-lock/pnpm-lock.yaml"
|
|
test -f "$lock"
|
|
test "$(find "$(dirname "$lock")" -type f | wc -l | tr -d ' ')" = 1
|
|
test "$(sha256sum "$lock" | cut -d ' ' -f 1)" = "$EXPECTED_LOCK_SHA256"
|
|
cp "$lock" pnpm-lock.yaml
|
|
test "$(sha256sum pnpm-lock.yaml | cut -d ' ' -f 1)" = "$EXPECTED_LOCK_SHA256"
|
|
|
|
- if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
|
|
- if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
|
|
env:
|
|
NPM_CONFIG_AUDIT: "false"
|
|
NPM_CONFIG_FUND: "false"
|
|
NPM_CONFIG_UPDATE_NOTIFIER: "false"
|
|
with:
|
|
version: 9.15.4
|
|
|
|
- if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
cache: pnpm
|
|
|
|
- if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
run: pnpm install --frozen-lockfile --ignore-scripts
|
|
|
|
- name: Materialize verified pinned OpenCode executable
|
|
if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
run: node packages/paperclip-runner/scripts/materialize-opencode-binary.mjs
|
|
|
|
- name: Download immutable shared campaign outputs
|
|
if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
name: ${{ needs.build_runner_artifacts.outputs.build_artifact_name }}
|
|
path: runner-e2e-build
|
|
|
|
- name: Verify and restore shared TypeScript outputs
|
|
if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
run: |
|
|
set -euo pipefail
|
|
(
|
|
cd runner-e2e-build
|
|
sha256sum --check runner-e2e-build-bundle.tar.gz.sha256
|
|
)
|
|
tar --extract --gzip \
|
|
--file runner-e2e-build/runner-e2e-build-bundle.tar.gz \
|
|
--directory "$GITHUB_WORKSPACE"
|
|
test -d packages/paperclip-eval-kernel/dist
|
|
test -d packages/paperclip-runner/dist
|
|
|
|
- name: Assemble native remote provider pack
|
|
if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
env:
|
|
# A reused image can have an older source revision with the same
|
|
# content ID. Matching that revision lets remote execution reuse the
|
|
# verified pack already installed in the immutable image.
|
|
PAPERCLIP_RUNNER_SOURCE_REVISION: ${{ needs.daytona_image.outputs.source_revision }}
|
|
run: node packages/paperclip-runner/scripts/build-provider-pack.mjs packages/paperclip-runner/provider-pack
|
|
|
|
- name: Package verified remote provider pack
|
|
if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
env:
|
|
IMAGE_SOURCE_REVISION: ${{ needs.daytona_image.outputs.source_revision }}
|
|
run: |
|
|
set -euo pipefail
|
|
test -f packages/paperclip-runner/provider-pack/provider-pack.json
|
|
jq -e \
|
|
--arg revision "$IMAGE_SOURCE_REVISION" \
|
|
'.schema == "paperclip-runner/remote-provider-pack/v1" and
|
|
.payload.runnerSourceRevision == $revision and
|
|
(.digest | test("^sha256:[0-9a-f]{64}$"))' \
|
|
packages/paperclip-runner/provider-pack/provider-pack.json >/dev/null
|
|
tar --create --gzip \
|
|
--file runner-e2e-provider-pack.tar.gz \
|
|
packages/paperclip-runner/provider-pack
|
|
sha256sum runner-e2e-provider-pack.tar.gz > runner-e2e-provider-pack.tar.gz.sha256
|
|
|
|
- name: Name immutable remote provider pack
|
|
id: provider_pack_artifact_name
|
|
if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
env:
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
run: echo "name=runner-e2e-provider-pack-${TARGET_SHA}-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Upload immutable remote provider pack
|
|
if: needs.catalog.outputs.needs_remote_provider_pack == 'true'
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: ${{ steps.provider_pack_artifact_name.outputs.name }}
|
|
path: |
|
|
runner-e2e-provider-pack.tar.gz
|
|
runner-e2e-provider-pack.tar.gz.sha256
|
|
retention-days: 1
|
|
compression-level: 0
|
|
if-no-files-found: error
|
|
|
|
test:
|
|
name: ${{ matrix.executionId }}
|
|
if: >-
|
|
!cancelled() && needs.catalog.result == 'success' && needs.daytona_image.result == 'success' &&
|
|
needs.build_runner_artifacts.result == 'success' && needs.build_remote_provider_pack.result == 'success' &&
|
|
needs.catalog.outputs.operations_cadence == '' && needs.authorize.outputs.evidence_retention == 'actions' &&
|
|
inputs.image_only != true
|
|
needs:
|
|
[
|
|
authorize,
|
|
target_lock,
|
|
catalog,
|
|
daytona_image,
|
|
build_runner_artifacts,
|
|
build_remote_provider_pack,
|
|
]
|
|
# The authorize job selects only one of two literal, reviewed runner labels;
|
|
# no dispatch input or repository variable can inject an arbitrary label.
|
|
runs-on: ${{ needs.authorize.outputs.test_runner }}
|
|
timeout-minutes: ${{ matrix.timeoutMinutes }}
|
|
permissions:
|
|
contents: read
|
|
environment:
|
|
name: runner-e2e-paid
|
|
strategy:
|
|
fail-fast: false
|
|
max-parallel: ${{ fromJSON(needs.catalog.outputs.max_parallel) }}
|
|
matrix: ${{ fromJSON(needs.catalog.outputs.matrix) }}
|
|
steps: &paid_test_steps
|
|
- name: Reauthorize paid execution before provider access
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
REF: ${{ github.ref }}
|
|
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
|
ACTOR_ID: ${{ github.actor_id }}
|
|
TRIGGERING_ACTOR: ${{ github.triggering_actor }}
|
|
ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
|
|
run: |
|
|
set -euo pipefail
|
|
test "$REF" = "refs/heads/$DEFAULT_BRANCH"
|
|
jq -e 'type == "array" and length > 0 and all(.[]; type == "number" and . > 0 and floor == .)' <<< "${ALLOWED_ACTOR_IDS:-}" >/dev/null
|
|
# Retry transient transport/server failures, but never proceed without
|
|
# a successfully resolved identity and the allowlist checks below.
|
|
for attempt in 1 2 3; do
|
|
if triggering_actor_id="$(timeout 30s gh api "users/$TRIGGERING_ACTOR" --jq .id)"; then
|
|
break
|
|
fi
|
|
if [ "$attempt" = 3 ]; then exit 1; fi
|
|
sleep "$((attempt * 2))"
|
|
done
|
|
jq -e --argjson candidate "$triggering_actor_id" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
|
|
jq -e --argjson candidate "$ACTOR_ID" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
|
|
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
ref: ${{ needs.authorize.outputs.target_sha }}
|
|
persist-credentials: false
|
|
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
|
|
if: needs.catalog.outputs.operations_cadence == ''
|
|
with:
|
|
ref: ${{ github.sha }}
|
|
path: .manual-private-trusted
|
|
persist-credentials: false
|
|
|
|
- name: Download resolved target lockfile
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
artifact-ids: ${{ needs.target_lock.outputs.artifact_id }}
|
|
path: ${{ runner.temp }}/runner-e2e-target-lock
|
|
|
|
- name: Restore resolved target lockfile
|
|
env:
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
EXPECTED_LOCK_SHA256: ${{ needs.target_lock.outputs.lock_sha256 }}
|
|
run: |
|
|
set -euo pipefail
|
|
test "$(git rev-parse HEAD)" = "$TARGET_SHA"
|
|
lock="$RUNNER_TEMP/runner-e2e-target-lock/pnpm-lock.yaml"
|
|
test -f "$lock"
|
|
test "$(find "$(dirname "$lock")" -type f | wc -l | tr -d ' ')" = 1
|
|
test "$(sha256sum "$lock" | cut -d ' ' -f 1)" = "$EXPECTED_LOCK_SHA256"
|
|
cp "$lock" pnpm-lock.yaml
|
|
test "$(sha256sum pnpm-lock.yaml | cut -d ' ' -f 1)" = "$EXPECTED_LOCK_SHA256"
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
|
|
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
|
|
env:
|
|
NPM_CONFIG_AUDIT: "false"
|
|
NPM_CONFIG_FUND: "false"
|
|
NPM_CONFIG_UPDATE_NOTIFIER: "false"
|
|
with:
|
|
version: 9.15.4
|
|
|
|
- name: Validate actual manual cell against trusted evidence routing
|
|
id: manual_cell_route
|
|
if: needs.catalog.outputs.operations_cadence == ''
|
|
env:
|
|
EVIDENCE_RETENTION: ${{ needs.authorize.outputs.evidence_retention }}
|
|
SELECTED_EXECUTION_ID: ${{ matrix.executionId }}
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
run: node "$GITHUB_WORKSPACE/.manual-private-trusted/tests/runner-e2e/manual-private-policy.mjs" cell
|
|
|
|
- name: Prepare workflow-pinned private transfer runtime before archive credentials
|
|
if: steps.manual_cell_route.outputs.route == 'private'
|
|
working-directory: .manual-private-trusted
|
|
env:
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
run: |
|
|
set -euo pipefail
|
|
test "$(git rev-parse HEAD)" = "$GITHUB_SHA"
|
|
node tests/runner-e2e/prepare-report-lock.mjs
|
|
pnpm install --frozen-lockfile --ignore-scripts --store-dir "$RUNNER_TEMP/manual-private-publication-store"
|
|
|
|
- name: Bind protected original run and compatible target evidence protocol
|
|
id: manual_private
|
|
if: steps.manual_cell_route.outputs.route == 'private'
|
|
working-directory: .manual-private-trusted
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
TARGET_ROOT: ${{ github.workspace }}
|
|
SELECTED_EXECUTION_ID: ${{ matrix.executionId }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
run: node "$GITHUB_WORKSPACE/.manual-private-trusted/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/.manual-private-trusted/tests/runner-e2e/manual-private-transfer.ts" prepare
|
|
|
|
- name: Obtain exact-key manual archive session
|
|
id: manual_private_credentials
|
|
if: steps.manual_private.outcome == 'success'
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
inline-session-policy: ${{ steps.manual_private.outputs.session_policy }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
|
|
- name: Verify restricted complete evidence storage before model or fixture secrets
|
|
id: manual_private_verified
|
|
if: steps.manual_private.outcome == 'success'
|
|
working-directory: .manual-private-trusted
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
TARGET_ROOT: ${{ github.workspace }}
|
|
SELECTED_EXECUTION_ID: ${{ matrix.executionId }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
PRIVATE_EVIDENCE_ROOT: ${{ steps.manual_private.outputs.root }}
|
|
AWS_ACCESS_KEY_ID: ${{ steps.manual_private_credentials.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.manual_private_credentials.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.manual_private_credentials.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
run: node "$GITHUB_WORKSPACE/.manual-private-trusted/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/.manual-private-trusted/tests/runner-e2e/manual-private-transfer.ts" verify
|
|
|
|
# This job receives provider credentials only in the final paid-test
|
|
# step. Keep target-selected dependency lifecycle code from running in
|
|
# the protected environment during setup.
|
|
- run: pnpm install --frozen-lockfile --ignore-scripts
|
|
|
|
# Sandbox-provider plugins are intentionally excluded from the root
|
|
# workspace. The ordinary root postinstall links the in-repo plugin SDK,
|
|
# but that lifecycle hook is deliberately disabled above. Prepare the
|
|
# one host plugin needed by Daytona explicitly, before this job receives
|
|
# provider credentials, and keep dependency lifecycle scripts disabled.
|
|
- name: Prepare bundled Daytona plugin without dependency lifecycle scripts
|
|
if: matrix.environmentId == 'daytona'
|
|
run: |
|
|
set -euo pipefail
|
|
daytona_root="packages/plugins/sandbox-providers/daytona"
|
|
sdk_root="packages/plugins/sdk"
|
|
test -d "$daytona_root"
|
|
test -d "$sdk_root"
|
|
test ! -L "$daytona_root"
|
|
test ! -L "$sdk_root"
|
|
test -f "$daytona_root/pnpm-lock.yaml"
|
|
test "$(jq -r .name "$daytona_root/package.json")" = "@paperclipai/plugin-daytona"
|
|
test "$(jq -r .name "$sdk_root/package.json")" = "@paperclipai/plugin-sdk"
|
|
(
|
|
cd "$daytona_root"
|
|
pnpm install --ignore-workspace --frozen-lockfile --ignore-scripts
|
|
)
|
|
node scripts/link-plugin-dev-sdk.mjs
|
|
test "$(realpath "$daytona_root/node_modules/@paperclipai/plugin-sdk")" = "$(realpath "$sdk_root")"
|
|
pnpm --dir "$daytona_root" build
|
|
test -f "$daytona_root/dist/manifest.js"
|
|
test -f "$daytona_root/dist/worker.js"
|
|
test -e "$daytona_root/node_modules/@daytonaio/sdk"
|
|
|
|
- name: Materialize verified pinned OpenCode executable
|
|
if: matrix.environmentId == 'local' && (matrix.profileId == 'legacy-opencode' || startsWith(matrix.profileId, 'runner-opencode') || matrix.suiteId == 'openrouter-model-breadth')
|
|
run: node packages/paperclip-runner/scripts/materialize-opencode-binary.mjs
|
|
|
|
- name: Install checksum-verified Grok executable
|
|
if: matrix.environmentId == 'local' && (matrix.profileId == 'runner-acpx-grok' || matrix.profileId == 'runner-acpx-grok-subscription')
|
|
run: sudo node packages/paperclip-runner/scripts/provision-grok.mjs /opt/paperclip/providers/grok/1.0.13/grok
|
|
|
|
- name: Download immutable campaign outputs
|
|
if: startsWith(matrix.profileId, 'runner-') || matrix.suiteId == 'openrouter-model-breadth'
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
name: ${{ needs.build_runner_artifacts.outputs.build_artifact_name }}
|
|
path: runner-e2e-build
|
|
|
|
- name: Download immutable remote provider pack
|
|
if: matrix.environmentId == 'daytona' && (startsWith(matrix.profileId, 'runner-opencode') || startsWith(matrix.profileId, 'runner-acpx-'))
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
name: ${{ needs.build_remote_provider_pack.outputs.provider_pack_artifact_name }}
|
|
path: runner-e2e-provider-pack
|
|
|
|
- name: Verify and restore campaign outputs
|
|
if: startsWith(matrix.profileId, 'runner-') || matrix.suiteId == 'openrouter-model-breadth'
|
|
env:
|
|
NEEDS_RUNNER_TYPESCRIPT: ${{ startsWith(matrix.profileId, 'runner-opencode') || startsWith(matrix.profileId, 'runner-acpx-') || matrix.suiteId == 'openrouter-model-breadth' }}
|
|
NEEDS_NATIVE_BINARY: ${{ startsWith(matrix.profileId, 'runner-') || matrix.suiteId == 'openrouter-model-breadth' }}
|
|
run: |
|
|
set -euo pipefail
|
|
(
|
|
cd runner-e2e-build
|
|
sha256sum --check runner-e2e-build-bundle.tar.gz.sha256
|
|
)
|
|
tar --extract --gzip \
|
|
--file runner-e2e-build/runner-e2e-build-bundle.tar.gz \
|
|
--directory "$GITHUB_WORKSPACE"
|
|
test -d packages/paperclip-eval-kernel/dist
|
|
if [ "$NEEDS_RUNNER_TYPESCRIPT" = true ]; then
|
|
test -d packages/paperclip-runner/dist
|
|
fi
|
|
if [ "$NEEDS_NATIVE_BINARY" = true ]; then
|
|
test -x packages/paperclip-runner/runner/target/debug/paperclip-runnerd
|
|
fi
|
|
|
|
- name: Verify and restore remote provider pack
|
|
if: matrix.environmentId == 'daytona' && (startsWith(matrix.profileId, 'runner-opencode') || startsWith(matrix.profileId, 'runner-acpx-'))
|
|
env:
|
|
IMAGE_SOURCE_REVISION: ${{ needs.daytona_image.outputs.source_revision }}
|
|
run: |
|
|
set -euo pipefail
|
|
(
|
|
cd runner-e2e-provider-pack
|
|
sha256sum --check runner-e2e-provider-pack.tar.gz.sha256
|
|
)
|
|
tar --extract --gzip \
|
|
--file runner-e2e-provider-pack/runner-e2e-provider-pack.tar.gz \
|
|
--directory "$GITHUB_WORKSPACE"
|
|
jq -e \
|
|
--arg revision "$IMAGE_SOURCE_REVISION" \
|
|
'.schema == "paperclip-runner/remote-provider-pack/v1" and
|
|
.payload.runnerSourceRevision == $revision and
|
|
(.digest | test("^sha256:[0-9a-f]{64}$"))' \
|
|
packages/paperclip-runner/provider-pack/provider-pack.json >/dev/null
|
|
|
|
- name: Qualify local provider Node interpreter
|
|
if: matrix.environmentId == 'local' && (startsWith(matrix.profileId, 'runner-opencode') || startsWith(matrix.profileId, 'runner-acpx-') || matrix.suiteId == 'openrouter-model-breadth')
|
|
run: |
|
|
node <<'NODE'
|
|
const fs = require("node:fs");
|
|
const mode = fs.statSync(process.execPath).mode & 0o777;
|
|
fs.chmodSync(process.execPath, mode & ~0o022);
|
|
if ((fs.statSync(process.execPath).mode & 0o022) !== 0) {
|
|
throw new Error("provider Node interpreter remains group- or world-writable");
|
|
}
|
|
NODE
|
|
|
|
- name: Install pinned Claude CLI for setup verification
|
|
if: matrix.profileId == 'legacy-claude' || matrix.profileId == 'runner-acpx-claude'
|
|
run: |
|
|
npm install --global --omit=dev --ignore-scripts @anthropic-ai/claude-code@2.1.277
|
|
launcher="$(npm root --global)/@anthropic-ai/claude-code/cli-wrapper.cjs"
|
|
chmod +x "$launcher"
|
|
ln -sf "$launcher" "$(npm prefix --global)/bin/claude"
|
|
claude --version
|
|
|
|
- name: Qualify preinstalled Chrome
|
|
if: needs.authorize.outputs.playwright_channel == 'chrome'
|
|
run: |
|
|
set -euo pipefail
|
|
chrome_path="$(command -v google-chrome)"
|
|
test -x "$chrome_path"
|
|
google-chrome --version
|
|
|
|
- name: Install Playwright FFmpeg on AWS runner
|
|
if: needs.authorize.outputs.playwright_channel == 'chrome'
|
|
run: |
|
|
set -euo pipefail
|
|
for attempt in 1 2 3; do
|
|
if pnpm exec playwright install ffmpeg; then
|
|
exit 0
|
|
fi
|
|
if [ "$attempt" -eq 3 ]; then
|
|
echo "Playwright FFmpeg installation failed after $attempt attempts." >&2
|
|
exit 1
|
|
fi
|
|
sleep "$((attempt * 10))"
|
|
done
|
|
|
|
- name: Install Chromium headless shell on GitHub-hosted fallback
|
|
if: needs.authorize.outputs.playwright_channel != 'chrome'
|
|
run: |
|
|
set -euo pipefail
|
|
for attempt in 1 2 3; do
|
|
if pnpm exec playwright install --with-deps --only-shell chromium; then
|
|
exit 0
|
|
fi
|
|
if [ "$attempt" -eq 3 ]; then
|
|
echo "Chromium headless shell installation failed after $attempt attempts." >&2
|
|
exit 1
|
|
fi
|
|
sleep "$((attempt * 10))"
|
|
done
|
|
|
|
# This definition executes only from the authorized default-branch workflow.
|
|
# Provision host policy before credentials reach target-controlled tests.
|
|
- name: Provision Codex sandbox on the disposable trusted runner
|
|
if: matrix.environmentId == 'local' && (needs.catalog.outputs.operations_cadence == 'weekly' || matrix.profileId == 'runner-codex' || matrix.profileId == 'runner-codex-mini' || matrix.profileId == 'runner-acpx-codex' || matrix.profileId == 'runner-codex-subscription')
|
|
run: |
|
|
node --input-type=module <<'NODE'
|
|
import { execFileSync } from "node:child_process";
|
|
import { createHash } from "node:crypto";
|
|
import { readFileSync, realpathSync, writeFileSync } from "node:fs";
|
|
import { createRequire } from "node:module";
|
|
import path from "node:path";
|
|
if (process.platform !== "linux") process.exit(0);
|
|
let restricted = "0";
|
|
try { restricted = readFileSync("/proc/sys/kernel/apparmor_restrict_unprivileged_userns", "utf8").trim(); } catch {}
|
|
if (restricted !== "1") process.exit(0);
|
|
const root = realpathSync(process.env.GITHUB_WORKSPACE);
|
|
const runnerRequire = createRequire(path.join(root, "packages/paperclip-runner/package.json"));
|
|
const acpRequire = createRequire(runnerRequire.resolve("@agentclientprotocol/codex-acp/package.json"));
|
|
const codexRequire = createRequire(acpRequire.resolve("@openai/codex/package.json"));
|
|
const arch = process.arch === "x64" ? "x64" : process.arch === "arm64" ? "arm64" : null;
|
|
if (!arch) throw new Error("Unsupported Codex CI architecture");
|
|
const platformPackage = codexRequire.resolve(`@openai/codex-linux-${arch}/package.json`);
|
|
const triple = arch === "x64" ? "x86_64-unknown-linux-musl" : "aarch64-unknown-linux-musl";
|
|
const suffix = `/vendor/${triple}/bin/codex`;
|
|
const binary = realpathSync(path.join(path.dirname(platformPackage), suffix));
|
|
if (!binary.startsWith(root + "/node_modules/.pnpm/") || !binary.endsWith(suffix) || !/^[/A-Za-z0-9_.@+\-]+$/.test(binary)) {
|
|
throw new Error("Codex executable is outside the resolved dependency tree");
|
|
}
|
|
const name = `paperclip-e2e-codex-${createHash("sha256").update(binary).digest("hex").slice(0,16)}`;
|
|
const profilePath = path.join(process.env.RUNNER_TEMP, "paperclip-codex-userns.apparmor");
|
|
writeFileSync(profilePath, `abi <abi/4.0>,\ninclude <tunables/global>\nprofile ${name} "${binary}" flags=(unconfined) {\n userns,\n}\n`, {mode:0o600, flag:"wx"});
|
|
execFileSync("sudo", ["-n", "apparmor_parser", "-r", profilePath], {timeout:15000, stdio:"pipe"});
|
|
NODE
|
|
|
|
- name: Prepare pinned Python artifact oracle image
|
|
if: matrix.caseId == 'build-revise' || matrix.caseId == 'delegate-feedback' || matrix.caseId == 'agent-review-handoff' || matrix.caseId == 'hire-reuse' || matrix.caseId == 'recover-controller' || matrix.caseId == 'stop-redirect'
|
|
run: |
|
|
set -euo pipefail
|
|
oracle_image='python@sha256:9d2e5553305c7c7b0097999bb17187c69b921ccd6bc9d40e4bb5ebe652c00285'
|
|
timeout 30s docker version --format '{{.Server.Version}}'
|
|
timeout 120s docker pull "$oracle_image"
|
|
timeout 30s docker image inspect "$oracle_image" --format '{{.Id}}'
|
|
|
|
- name: Obtain scoped private history credentials
|
|
id: private_history_credentials
|
|
if: needs.catalog.outputs.operations_cadence != ''
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd # v6
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
|
|
- name: Verify private evidence retention before paid execution
|
|
if: needs.catalog.outputs.operations_cadence != ''
|
|
env:
|
|
AWS_ACCESS_KEY_ID: ${{ steps.private_history_credentials.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.private_history_credentials.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.private_history_credentials.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
run: node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/private-history.ts verify evidence "$GITHUB_RUN_ID" bounded-runner-e2e-evidence
|
|
|
|
- name: Prepare bounded campaign before credentials
|
|
if: needs.catalog.outputs.operations_cadence != ''
|
|
env:
|
|
EVAL_EVENT: ${{ github.event_name }}
|
|
EVAL_SCHEDULE: ${{ github.event.schedule }}
|
|
EVAL_CADENCE: ${{ needs.catalog.outputs.operations_cadence }}
|
|
EVAL_OPERATIONS_ENABLED: ${{ vars.RUNNER_FULL_STACK_E2E_OPERATIONS_ENABLED }}
|
|
EVAL_OPERATING_OVERHEAD_JSON: ${{ needs.catalog.outputs.operating_overhead }}
|
|
EVAL_TEST_RUNNER: ${{ needs.authorize.outputs.test_runner }}
|
|
PAPERCLIP_E2E_BRAINTRUST_PROJECT_ID: ${{ vars.RUNNER_E2E_BRAINTRUST_PROJECT_ID }}
|
|
run: node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/scheduled-operations.ts prepare
|
|
|
|
- name: Run bounded campaign and retain feedback review
|
|
if: needs.catalog.outputs.operations_cadence != ''
|
|
env:
|
|
BRAINTRUST_API_KEY: ${{ secrets.BRAINTRUST_API_KEY }}
|
|
PAPERCLIP_E2E_BRAINTRUST_PROJECT_ID: ${{ vars.RUNNER_E2E_BRAINTRUST_PROJECT_ID }}
|
|
PAPERCLIP_RUNNER_E2E_SOURCE_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
PAPERCLIP_RUNNER_E2E_SOURCE_REF: ${{ needs.authorize.outputs.target_ref }}
|
|
PAPERCLIP_PLAYWRIGHT_CHANNEL: ${{ needs.authorize.outputs.playwright_channel }}
|
|
run: |
|
|
set +e
|
|
args=(--max-parallel 1 --max-automatic-retries 0)
|
|
while IFS= read -r id; do args+=(--id "$id"); done < <(jq -r '.executionIds[]' "$PAPERCLIP_E2E_OPERATIONS_MANIFEST")
|
|
pnpm test:e2e:runner -- "${args[@]}"
|
|
run_status=$?
|
|
node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/scheduled-finalization.ts
|
|
finalization_status=$?
|
|
test "$run_status" = 0 && test "$finalization_status" = 0
|
|
|
|
- name: Run paid cell
|
|
if: needs.catalog.outputs.operations_cadence == '' && (steps.manual_cell_route.outputs.route == 'actions' || steps.manual_private_verified.outcome == 'success')
|
|
env:
|
|
PAPERCLIP_RUNNER_E2E_PRIVATE_PREFLIGHT: ${{ steps.manual_private_verified.outcome == 'success' && format('{0}/preflight.json', steps.manual_private.outputs.root) || '' }}
|
|
PRIVATE_EVIDENCE_ROOT: ${{ steps.manual_private.outputs.root }}
|
|
OPENAI_API_KEY: ${{ matrix.credentialName == 'OPENAI_API_KEY' && secrets.OPENAI_API_KEY || '' }}
|
|
ANTHROPIC_API_KEY: ${{ matrix.credentialName == 'ANTHROPIC_API_KEY' && secrets.ANTHROPIC_API_KEY || '' }}
|
|
OPENROUTER_API_KEY: ${{ matrix.credentialName == 'OPENROUTER_API_KEY' && secrets.OPENROUTER_API_KEY || '' }}
|
|
XAI_API_KEY: ${{ matrix.credentialName == 'XAI_API_KEY' && secrets.XAI_API_KEY || '' }}
|
|
GROK_AUTH_JSON: ${{ matrix.credentialName == 'GROK_AUTH_JSON' && secrets.GROK_AUTH_JSON || '' }}
|
|
COPILOT_GITHUB_TOKEN: ${{ matrix.credentialName == 'COPILOT_GITHUB_TOKEN' && secrets.COPILOT_GITHUB_TOKEN || '' }}
|
|
DAYTONA_API_KEY: ${{ matrix.environmentId == 'daytona' && secrets.DAYTONA_API_KEY || '' }}
|
|
PAPERCLIP_E2E_DAYTONA_IMAGE: ${{ needs.daytona_image.outputs.image }}
|
|
PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH: ${{ github.workspace }}/packages/paperclip-runner/provider-pack
|
|
PAPERCLIP_E2E_CAMPAIGN_ID: gha-${{ github.run_id }}-${{ github.run_attempt }}-${{ matrix.executionId }}
|
|
PAPERCLIP_E2E_ACCOUNTING_CAMPAIGN_ID: gha-${{ github.run_id }}-${{ github.run_attempt }}
|
|
PAPERCLIP_RUNNER_E2E_SOURCE_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
PAPERCLIP_RUNNER_E2E_SOURCE_REF: ${{ needs.authorize.outputs.target_ref }}
|
|
PAPERCLIP_PLAYWRIGHT_CHANNEL: ${{ needs.authorize.outputs.playwright_channel }}
|
|
run: |
|
|
if [ -n "$PAPERCLIP_RUNNER_E2E_PRIVATE_PREFLIGHT" ]; then
|
|
pnpm test:e2e:runner -- --id "${{ matrix.executionId }}" > "$PRIVATE_EVIDENCE_ROOT/launcher-output.log" 2>&1
|
|
else
|
|
pnpm test:e2e:runner -- --id "${{ matrix.executionId }}"
|
|
fi
|
|
|
|
- name: Stage bounded campaign evidence
|
|
id: stage_evidence
|
|
if: always() && needs.catalog.outputs.operations_cadence != ''
|
|
run: node tests/runner-e2e/artifact-budget.mjs stage evidence tests/runner-e2e/results bounded-runner-e2e-evidence
|
|
|
|
- name: Upload nonprivate packaged cell evidence
|
|
if: always() && needs.catalog.outputs.operations_cadence == '' && needs.authorize.outputs.evidence_retention == 'actions' && steps.manual_cell_route.outputs.route == 'actions'
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: runner-e2e-${{ github.run_id }}-${{ github.run_attempt }}-${{ matrix.executionId }}
|
|
path: tests/runner-e2e/results/
|
|
retention-days: 30
|
|
if-no-files-found: error
|
|
|
|
- name: Retain bounded campaign evidence in private AWS storage
|
|
if: always() && needs.catalog.outputs.operations_cadence != '' && steps.private_history_credentials.outcome == 'success'
|
|
env:
|
|
AWS_ACCESS_KEY_ID: ${{ steps.private_history_credentials.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.private_history_credentials.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.private_history_credentials.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
run: node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/private-history.ts upload evidence "$GITHUB_RUN_ID" bounded-runner-e2e-evidence
|
|
|
|
- name: Enforce complete bounded evidence retention
|
|
if: always() && needs.catalog.outputs.operations_cadence != '' && steps.stage_evidence.outputs.complete != 'true'
|
|
run: exit 1
|
|
|
|
- name: Retain complete approved manual originals with no public fallback
|
|
if: always() && steps.manual_private_verified.outcome == 'success'
|
|
working-directory: .manual-private-trusted
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
TARGET_ROOT: ${{ github.workspace }}
|
|
SELECTED_EXECUTION_ID: ${{ matrix.executionId }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
PRIVATE_EVIDENCE_ROOT: ${{ steps.manual_private.outputs.root }}
|
|
AWS_ACCESS_KEY_ID: ${{ steps.manual_private_credentials.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.manual_private_credentials.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.manual_private_credentials.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
run: node "$GITHUB_WORKSPACE/.manual-private-trusted/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/.manual-private-trusted/tests/runner-e2e/manual-private-transfer.ts" upload-cell
|
|
|
|
test_recurring:
|
|
name: ${{ matrix.executionId }}
|
|
if: >-
|
|
!cancelled() && needs.catalog.result == 'success' && needs.daytona_image.result == 'success' &&
|
|
needs.build_runner_artifacts.result == 'success' && needs.build_remote_provider_pack.result == 'success' &&
|
|
needs.catalog.outputs.operations_cadence != '' && needs.authorize.outputs.target_sha == github.sha && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) &&
|
|
inputs.image_only != true
|
|
needs: [authorize, target_lock, catalog, daytona_image, build_runner_artifacts, build_remote_provider_pack]
|
|
runs-on: ${{ needs.authorize.outputs.test_runner }}
|
|
timeout-minutes: ${{ matrix.timeoutMinutes }}
|
|
permissions:
|
|
contents: read
|
|
id-token: write
|
|
environment:
|
|
name: runner-e2e-private-paid
|
|
strategy:
|
|
fail-fast: false
|
|
max-parallel: ${{ fromJSON(needs.catalog.outputs.max_parallel) }}
|
|
matrix: ${{ fromJSON(needs.catalog.outputs.matrix) }}
|
|
steps: *paid_test_steps
|
|
|
|
test_manual_private:
|
|
name: ${{ matrix.executionId }}
|
|
if: >-
|
|
!cancelled() && needs.catalog.result == 'success' && needs.daytona_image.result == 'success' &&
|
|
needs.build_runner_artifacts.result == 'success' && needs.build_remote_provider_pack.result == 'success' &&
|
|
needs.catalog.outputs.operations_cadence == '' && needs.authorize.outputs.evidence_retention == 'private' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) &&
|
|
inputs.image_only != true
|
|
needs: [authorize, target_lock, catalog, daytona_image, build_runner_artifacts, build_remote_provider_pack]
|
|
runs-on: ${{ needs.authorize.outputs.test_runner }}
|
|
timeout-minutes: ${{ matrix.timeoutMinutes }}
|
|
permissions:
|
|
contents: read
|
|
id-token: write
|
|
environment:
|
|
name: runner-e2e-private-paid
|
|
strategy:
|
|
fail-fast: false
|
|
max-parallel: ${{ fromJSON(needs.catalog.outputs.max_parallel) }}
|
|
matrix: ${{ fromJSON(needs.catalog.outputs.matrix) }}
|
|
steps: *paid_test_steps
|
|
|
|
report:
|
|
name: Merge and enforce campaign result
|
|
if: always() && !cancelled() && needs.catalog.result == 'success' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && inputs.image_only != true
|
|
needs: [authorize, catalog, daytona_image, test, test_recurring, test_manual_private]
|
|
outputs:
|
|
history_source_ready: ${{ steps.history_source_ready.outputs.ready }}
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 15
|
|
permissions:
|
|
actions: read
|
|
contents: read
|
|
id-token: write
|
|
environment:
|
|
name: runner-e2e-private-history
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
# Reporting and sanitization are part of the trusted workflow boundary.
|
|
ref: ${{ github.sha }}
|
|
persist-credentials: false
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
|
|
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
|
|
env:
|
|
NPM_CONFIG_AUDIT: "false"
|
|
NPM_CONFIG_FUND: "false"
|
|
NPM_CONFIG_UPDATE_NOTIFIER: "false"
|
|
with:
|
|
version: 9.15.4
|
|
|
|
- name: Apply reviewed reporting lock corrections
|
|
run: node tests/runner-e2e/prepare-report-lock.mjs
|
|
|
|
- name: Install pinned reporting dependencies
|
|
run: pnpm install --frozen-lockfile --ignore-scripts --store-dir "${{ runner.temp }}/eval-report-store"
|
|
|
|
- name: Prepare exact manual report storage identity
|
|
id: manual_report
|
|
if: needs.authorize.outputs.evidence_retention == 'private'
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
run: node "$GITHUB_WORKSPACE/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/tests/runner-e2e/manual-private-transfer.ts" report-session
|
|
|
|
- name: Obtain exact current manual report archive session
|
|
id: manual_report_credentials
|
|
if: steps.manual_report.outcome == 'success'
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
inline-session-policy: ${{ steps.manual_report.outputs.session_policy }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
|
|
- name: Obtain exact private cell 0 read session
|
|
id: manual_read_credentials_0
|
|
if: steps.manual_report.outcome == 'success' && fromJSON(steps.manual_report.outputs.cell_count) > 0
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
inline-session-policy: ${{ steps.manual_report.outputs.read_policy_0 }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
|
|
- name: Download restricted manual cell 0
|
|
if: steps.manual_read_credentials_0.outcome == 'success'
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
PRIVATE_REPORT_ROOT: ${{ github.workspace }}
|
|
SELECTED_CELL_INDEX: '0'
|
|
AWS_ACCESS_KEY_ID: ${{ steps.manual_read_credentials_0.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.manual_read_credentials_0.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.manual_read_credentials_0.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
run: node "$GITHUB_WORKSPACE/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/tests/runner-e2e/manual-private-transfer.ts" download-cell
|
|
|
|
- name: Obtain exact private cell 1 read session
|
|
id: manual_read_credentials_1
|
|
if: steps.manual_report.outcome == 'success' && fromJSON(steps.manual_report.outputs.cell_count) > 1
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
inline-session-policy: ${{ steps.manual_report.outputs.read_policy_1 }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
|
|
- name: Download restricted manual cell 1
|
|
if: steps.manual_read_credentials_1.outcome == 'success'
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
PRIVATE_REPORT_ROOT: ${{ github.workspace }}
|
|
SELECTED_CELL_INDEX: '1'
|
|
AWS_ACCESS_KEY_ID: ${{ steps.manual_read_credentials_1.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.manual_read_credentials_1.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.manual_read_credentials_1.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
run: node "$GITHUB_WORKSPACE/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/tests/runner-e2e/manual-private-transfer.ts" download-cell
|
|
|
|
- name: Obtain exact private cell 2 read session
|
|
id: manual_read_credentials_2
|
|
if: steps.manual_report.outcome == 'success' && fromJSON(steps.manual_report.outputs.cell_count) > 2
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
inline-session-policy: ${{ steps.manual_report.outputs.read_policy_2 }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
|
|
- name: Download restricted manual cell 2
|
|
if: steps.manual_read_credentials_2.outcome == 'success'
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
PRIVATE_REPORT_ROOT: ${{ github.workspace }}
|
|
SELECTED_CELL_INDEX: '2'
|
|
AWS_ACCESS_KEY_ID: ${{ steps.manual_read_credentials_2.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.manual_read_credentials_2.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.manual_read_credentials_2.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
run: node "$GITHUB_WORKSPACE/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/tests/runner-e2e/manual-private-transfer.ts" download-cell
|
|
|
|
- name: Obtain exact private cell 3 read session
|
|
id: manual_read_credentials_3
|
|
if: steps.manual_report.outcome == 'success' && fromJSON(steps.manual_report.outputs.cell_count) > 3
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
inline-session-policy: ${{ steps.manual_report.outputs.read_policy_3 }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
|
|
- name: Download restricted manual cell 3
|
|
if: steps.manual_read_credentials_3.outcome == 'success'
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
PRIVATE_REPORT_ROOT: ${{ github.workspace }}
|
|
SELECTED_CELL_INDEX: '3'
|
|
AWS_ACCESS_KEY_ID: ${{ steps.manual_read_credentials_3.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.manual_read_credentials_3.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.manual_read_credentials_3.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
run: node "$GITHUB_WORKSPACE/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/tests/runner-e2e/manual-private-transfer.ts" download-cell
|
|
|
|
- name: Obtain exact private cell 4 read session
|
|
id: manual_read_credentials_4
|
|
if: steps.manual_report.outcome == 'success' && fromJSON(steps.manual_report.outputs.cell_count) > 4
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
inline-session-policy: ${{ steps.manual_report.outputs.read_policy_4 }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
|
|
- name: Download restricted manual cell 4
|
|
if: steps.manual_read_credentials_4.outcome == 'success'
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
PRIVATE_REPORT_ROOT: ${{ github.workspace }}
|
|
SELECTED_CELL_INDEX: '4'
|
|
AWS_ACCESS_KEY_ID: ${{ steps.manual_read_credentials_4.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.manual_read_credentials_4.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.manual_read_credentials_4.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
run: node "$GITHUB_WORKSPACE/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/tests/runner-e2e/manual-private-transfer.ts" download-cell
|
|
|
|
- name: Obtain exact private cell 5 read session
|
|
id: manual_read_credentials_5
|
|
if: steps.manual_report.outcome == 'success' && fromJSON(steps.manual_report.outputs.cell_count) > 5
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
inline-session-policy: ${{ steps.manual_report.outputs.read_policy_5 }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
|
|
- name: Download restricted manual cell 5
|
|
if: steps.manual_read_credentials_5.outcome == 'success'
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
PRIVATE_REPORT_ROOT: ${{ github.workspace }}
|
|
SELECTED_CELL_INDEX: '5'
|
|
AWS_ACCESS_KEY_ID: ${{ steps.manual_read_credentials_5.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.manual_read_credentials_5.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.manual_read_credentials_5.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
run: node "$GITHUB_WORKSPACE/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/tests/runner-e2e/manual-private-transfer.ts" download-cell
|
|
|
|
- name: Obtain exact private cell 6 read session
|
|
id: manual_read_credentials_6
|
|
if: steps.manual_report.outcome == 'success' && fromJSON(steps.manual_report.outputs.cell_count) > 6
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
inline-session-policy: ${{ steps.manual_report.outputs.read_policy_6 }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
|
|
- name: Download restricted manual cell 6
|
|
if: steps.manual_read_credentials_6.outcome == 'success'
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
PRIVATE_REPORT_ROOT: ${{ github.workspace }}
|
|
SELECTED_CELL_INDEX: '6'
|
|
AWS_ACCESS_KEY_ID: ${{ steps.manual_read_credentials_6.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.manual_read_credentials_6.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.manual_read_credentials_6.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
run: node "$GITHUB_WORKSPACE/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/tests/runner-e2e/manual-private-transfer.ts" download-cell
|
|
|
|
- name: Obtain exact private cell 7 read session
|
|
id: manual_read_credentials_7
|
|
if: steps.manual_report.outcome == 'success' && fromJSON(steps.manual_report.outputs.cell_count) > 7
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
inline-session-policy: ${{ steps.manual_report.outputs.read_policy_7 }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
|
|
- name: Download restricted manual cell 7
|
|
if: steps.manual_read_credentials_7.outcome == 'success'
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
PRIVATE_REPORT_ROOT: ${{ github.workspace }}
|
|
SELECTED_CELL_INDEX: '7'
|
|
AWS_ACCESS_KEY_ID: ${{ steps.manual_read_credentials_7.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.manual_read_credentials_7.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.manual_read_credentials_7.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
run: node "$GITHUB_WORKSPACE/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/tests/runner-e2e/manual-private-transfer.ts" download-cell
|
|
|
|
- name: Verify all approved private cell transfers before merging
|
|
id: manual_cells_downloaded
|
|
if: steps.manual_report.outcome == 'success'
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
PRIVATE_REPORT_ROOT: ${{ github.workspace }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
run: node "$GITHUB_WORKSPACE/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/tests/runner-e2e/manual-private-transfer.ts" verify-downloads
|
|
|
|
- name: Resolve workflow job attempts
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
REPOSITORY: ${{ github.repository }}
|
|
RUN_ID: ${{ github.run_id }}
|
|
run: |
|
|
set -euo pipefail
|
|
gh api --paginate --slurp \
|
|
"repos/$REPOSITORY/actions/runs/$RUN_ID/jobs?filter=all&per_page=100" \
|
|
> runner-e2e-job-pages.json
|
|
for attempt in $(seq 1 "${{ github.run_attempt }}"); do
|
|
gh api "repos/$REPOSITORY/actions/runs/$RUN_ID/attempts/$attempt" \
|
|
--jq '{run_attempt, run_started_at}'
|
|
done > runner-e2e-attempts.jsonl
|
|
jq -s '.' runner-e2e-attempts.jsonl > runner-e2e-attempts.json
|
|
jq --slurpfile attempts runner-e2e-attempts.json \
|
|
'{jobs: [.[].jobs[]], attempts: $attempts[0]}' \
|
|
runner-e2e-job-pages.json > runner-e2e-jobs.json
|
|
|
|
- name: Download cell evidence
|
|
id: download_evidence
|
|
if: needs.catalog.outputs.operations_cadence == '' && needs.authorize.outputs.evidence_retention == 'actions'
|
|
continue-on-error: true
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
pattern: runner-e2e-${{ github.run_id }}-*-*
|
|
path: downloaded-runner-e2e
|
|
merge-multiple: false
|
|
|
|
- name: Retry cell evidence download after transport failure
|
|
if: needs.catalog.outputs.operations_cadence == '' && needs.authorize.outputs.evidence_retention == 'actions' && steps.download_evidence.outcome == 'failure'
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
pattern: runner-e2e-${{ github.run_id }}-*-*
|
|
path: downloaded-runner-e2e
|
|
merge-multiple: false
|
|
|
|
- name: Obtain scoped private history credentials
|
|
id: private_history_credentials
|
|
if: needs.catalog.outputs.operations_cadence != ''
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd # v6
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
|
|
- name: Download private bounded campaign evidence
|
|
if: needs.catalog.outputs.operations_cadence != ''
|
|
env:
|
|
AWS_ACCESS_KEY_ID: ${{ steps.private_history_credentials.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.private_history_credentials.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.private_history_credentials.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
run: node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/private-history.ts download evidence "$GITHUB_RUN_ID" "downloaded-runner-e2e/runner-e2e-$GITHUB_RUN_ID-1-scheduled-campaign"
|
|
|
|
- name: Select latest workflow attempt per cell
|
|
if: always() && needs.catalog.outputs.operations_cadence == '' && needs.authorize.outputs.evidence_retention == 'actions'
|
|
env:
|
|
PAPERCLIP_RUNNER_E2E_ARTIFACT_ROOT: ${{ github.workspace }}/downloaded-runner-e2e
|
|
PAPERCLIP_RUNNER_E2E_SELECTED_ROOT: ${{ github.workspace }}/selected-runner-e2e
|
|
PAPERCLIP_RUNNER_E2E_JOBS_JSON: ${{ github.workspace }}/runner-e2e-jobs.json
|
|
PAPERCLIP_RUNNER_E2E_EXPECTED_IDS: ${{ needs.catalog.outputs.execution_ids }}
|
|
PAPERCLIP_RUNNER_E2E_SOURCE_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
PAPERCLIP_RUNNER_E2E_SOURCE_REF: ${{ needs.authorize.outputs.target_ref }}
|
|
run: node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/select-rerun-artifacts.ts
|
|
|
|
- name: Preserve bounded campaign evidence and journal
|
|
if: always() && needs.catalog.outputs.operations_cadence != ''
|
|
env:
|
|
RUN_ID: ${{ github.run_id }}
|
|
run: |
|
|
set -euo pipefail
|
|
artifact="downloaded-runner-e2e/runner-e2e-$RUN_ID-1-scheduled-campaign"
|
|
directory="$GITHUB_WORKSPACE/selected-runner-e2e/runner-e2e-$RUN_ID-1-scheduled-campaign/scheduled"
|
|
echo "PAPERCLIP_E2E_OPERATIONS_LEDGER=$directory/ledger.jsonl" >> "$GITHUB_ENV"
|
|
test -d "$artifact"
|
|
mkdir -p selected-runner-e2e
|
|
cp -R "$artifact" selected-runner-e2e/
|
|
if [ -f "$directory/feedback-review.json" ]; then
|
|
echo "PAPERCLIP_E2E_FEEDBACK_REVIEW=$directory/feedback-review.json" >> "$GITHUB_ENV"
|
|
fi
|
|
if [ -f "$directory/experiment-receipt.json" ]; then
|
|
echo "PAPERCLIP_E2E_BRAINTRUST_EXPERIMENT=$directory/experiment-receipt.json" >> "$GITHUB_ENV"
|
|
fi
|
|
|
|
- name: Collect blob reports
|
|
run: |
|
|
set -euo pipefail
|
|
mkdir -p merged-blob-reports
|
|
while IFS= read -r -d '' report; do
|
|
digest="$(sha256sum "$report" | cut -d ' ' -f 1)"
|
|
target="merged-blob-reports/report-${digest}.zip"
|
|
if [ ! -e "$target" ]; then
|
|
cp "$report" "$target"
|
|
fi
|
|
done < <(find selected-runner-e2e -path '*/blob-report/*.zip' -print0)
|
|
|
|
- name: Merge Playwright HTML and JUnit
|
|
if: always()
|
|
env:
|
|
PAPERCLIP_RUNNER_E2E_MERGED_REPORT_DIR: ${{ github.workspace }}/runner-e2e-merged-report
|
|
run: |
|
|
if [ "${{ needs.authorize.outputs.evidence_retention }}" = private ]; then
|
|
mkdir -p runner-e2e-merged-report
|
|
pnpm exec playwright merge-reports --config tests/runner-e2e/merge.config.ts merged-blob-reports > runner-e2e-merged-report/merge.log 2>&1
|
|
else
|
|
pnpm exec playwright merge-reports --config tests/runner-e2e/merge.config.ts merged-blob-reports
|
|
fi
|
|
|
|
- name: Aggregate normalized campaign results
|
|
if: always()
|
|
env:
|
|
PAPERCLIP_RUNNER_E2E_REPORT_ROOT: ${{ github.workspace }}/selected-runner-e2e
|
|
PAPERCLIP_RUNNER_E2E_REPORT_OUT: ${{ github.workspace }}/runner-e2e-merged-report/normalized
|
|
PAPERCLIP_RUNNER_E2E_EXPECTED_IDS: ${{ needs.catalog.outputs.execution_ids }}
|
|
PAPERCLIP_E2E_CAMPAIGN_ID: ${{ needs.catalog.outputs.operations_cadence != '' && format('gha-{0}-1-scheduled-campaign', github.run_id) || format('gha-{0}-{1}', github.run_id, github.run_attempt) }}
|
|
PAPERCLIP_RUNNER_E2E_SOURCE_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
PAPERCLIP_RUNNER_E2E_SOURCE_REF: ${{ needs.authorize.outputs.target_ref }}
|
|
PAPERCLIP_RUNNER_E2E_HISTORY_PUBLIC_BASE_URL: ${{ needs.catalog.outputs.operations_cadence == '' && needs.authorize.outputs.evidence_retention == 'actions' && vars.RUNNER_E2E_HISTORY_PUBLIC_BASE_URL || '' }}
|
|
PAPERCLIP_RUNNER_E2E_HISTORY_PREFIX: ${{ vars.RUNNER_E2E_HISTORY_PREFIX || 'runner-e2e' }}
|
|
EVIDENCE_RETENTION: ${{ needs.authorize.outputs.evidence_retention }}
|
|
EVAL_OPERATIONS_CADENCE: ${{ needs.catalog.outputs.operations_cadence }}
|
|
run: |
|
|
set +e
|
|
if [ -n "$EVAL_OPERATIONS_CADENCE" ] || [ "$EVIDENCE_RETENTION" = private ]; then
|
|
mkdir -p runner-e2e-merged-report
|
|
pnpm test:e2e:runner:report > runner-e2e-merged-report/report.log 2>&1
|
|
else
|
|
pnpm test:e2e:runner:report
|
|
fi
|
|
report_status=$?
|
|
set -e
|
|
if [ -z "$EVAL_OPERATIONS_CADENCE" ] && [ "$EVIDENCE_RETENTION" = actions ]; then
|
|
cat runner-e2e-merged-report/normalized/summary.md >> "$GITHUB_STEP_SUMMARY"
|
|
fi
|
|
exit "$report_status"
|
|
|
|
- name: Stage bounded merged report
|
|
id: stage_report
|
|
if: always() && needs.catalog.outputs.operations_cadence != ''
|
|
run: node tests/runner-e2e/artifact-budget.mjs stage report runner-e2e-merged-report bounded-runner-e2e-report
|
|
|
|
- name: Upload nonprivate merged report
|
|
if: always() && needs.catalog.outputs.operations_cadence == '' && needs.authorize.outputs.evidence_retention == 'actions'
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: runner-e2e-report-${{ github.run_id }}-${{ github.run_attempt }}
|
|
path: runner-e2e-merged-report/
|
|
retention-days: 30
|
|
if-no-files-found: error
|
|
|
|
- name: Retain bounded report in private AWS storage
|
|
id: private_report
|
|
if: always() && needs.catalog.outputs.operations_cadence != '' && steps.private_history_credentials.outcome == 'success'
|
|
env:
|
|
AWS_ACCESS_KEY_ID: ${{ steps.private_history_credentials.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.private_history_credentials.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.private_history_credentials.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
run: |
|
|
node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/private-history.ts upload report "$GITHUB_RUN_ID" bounded-runner-e2e-report
|
|
echo '{"version":1,"privateReportRetained":true}' > private-report-status.json
|
|
echo 'The recurring campaign report is retained in private AWS storage. See the eval operations runbook for authenticated retrieval.' >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
- name: Retain report availability for the schedule monitor
|
|
if: always() && needs.catalog.outputs.operations_cadence != '' && steps.private_report.outcome == 'success'
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: runner-e2e-report-${{ github.run_id }}-${{ github.run_attempt }}
|
|
path: private-report-status.json
|
|
retention-days: 30
|
|
if-no-files-found: error
|
|
|
|
- name: Enforce complete bounded report retention
|
|
if: always() && needs.catalog.outputs.operations_cadence != '' && steps.stage_report.outputs.complete != 'true'
|
|
run: exit 1
|
|
|
|
- name: Retain complete manual merged evidence only in restricted storage
|
|
id: manual_private_report
|
|
if: always() && steps.manual_report_credentials.outcome == 'success' && steps.manual_cells_downloaded.outcome == 'success'
|
|
env:
|
|
MANUAL_PRIVATE_IDS: ${{ needs.authorize.outputs.manual_private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ vars.RUNNER_E2E_MANUAL_PRIVATE_APPROVALS_JSON }}
|
|
TARGET_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
PRIVATE_REPORT_ROOT: ${{ github.workspace }}
|
|
PRIVATE_REPORT_SOURCE: ${{ github.workspace }}/runner-e2e-merged-report
|
|
AWS_ACCESS_KEY_ID: ${{ steps.manual_report_credentials.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.manual_report_credentials.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.manual_report_credentials.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
run: |
|
|
node "$GITHUB_WORKSPACE/cli/node_modules/tsx/dist/cli.mjs" "$GITHUB_WORKSPACE/tests/runner-e2e/manual-private-transfer.ts" upload-report
|
|
echo 'The private report archive is retained. Transfer does not certify feature PASS or required evidence completeness; see its typed receipt. No raw Actions artifact or public page was uploaded.' >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
- name: Verify normalized history source report
|
|
id: history_source_ready
|
|
if: always() && needs.authorize.outputs.evidence_retention == 'actions'
|
|
run: |
|
|
set -euo pipefail
|
|
dashboard_root="runner-e2e-merged-report/normalized"
|
|
if [ -f "$dashboard_root/index.html" ] && [ -f "$dashboard_root/normalized-results.json" ]; then
|
|
echo "ready=true" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "ready=false" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
publish_testops:
|
|
name: Publish bound TestOps plan results
|
|
needs: [authorize, testops_selection, catalog, report]
|
|
if: always() && needs.testops_selection.outputs.job_run_started == 'true' && inputs.image_only != true
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
environment: runner-e2e-testops-reporting
|
|
permissions:
|
|
contents: read
|
|
actions: read
|
|
env:
|
|
SOURCE_SHA: ${{ needs.authorize.outputs.target_sha }}
|
|
SELECTION_SHA256: ${{ needs.testops_selection.outputs.selection_sha256 }}
|
|
EVAL_CADENCE: manual
|
|
ALLURE_JOB_RUN_ID: ${{ needs.testops_selection.outputs.job_run_id || inputs.ALLURE_JOB_RUN_ID }}
|
|
ALLURE_USERNAME: ${{ inputs.ALLURE_USERNAME }}
|
|
ALLURE_RESULTS: ${{ github.workspace }}/product-regression-allure
|
|
REPORT_DIR: ${{ github.workspace }}/product-report
|
|
EXPORT_SUMMARY: ${{ github.workspace }}/product-regression-export.json
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
# Only trusted workflow code sees the reporting token. Candidate
|
|
# reports are parsed as data and never loaded as executable modules.
|
|
ref: ${{ github.sha }}
|
|
persist-credentials: false
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
if: needs.testops_selection.result == 'success' && !cancelled()
|
|
id: download_selection
|
|
with:
|
|
name: product-testops-selection-${{ github.run_id }}-${{ github.run_attempt }}
|
|
path: product-testops
|
|
- name: Download retained full-stack report
|
|
if: needs.testops_selection.result == 'success' && !cancelled()
|
|
continue-on-error: true
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
name: runner-e2e-report-${{ github.run_id }}-${{ github.run_attempt }}
|
|
path: product-report
|
|
- uses: allure-framework/setup-allurectl@2d89ee0b7ba2f59dbe9849b6df5763cab78e14db # v1
|
|
if: always()
|
|
with:
|
|
allurectl-version: '2.23.1'
|
|
allure-endpoint: https://paperclip.testops.cloud
|
|
allure-project-id: '1'
|
|
allure-token: ${{ secrets.ALLURE_TOKEN }}
|
|
- name: Project original verdicts and explicitly report missing results
|
|
if: steps.download_selection.outcome == 'success' && !cancelled()
|
|
id: project
|
|
run: node tests/runner-e2e/product-regression-testops-cli.mjs project product-testops/selection.json
|
|
- name: Upload only the bound result projection
|
|
if: steps.project.outcome == 'success'
|
|
run: allurectl upload --validate-json "$ALLURE_RESULTS"
|
|
- name: Verify server-side case identities, completeness and verdicts
|
|
if: steps.project.outcome == 'success'
|
|
run: node tests/runner-e2e/product-regression-testops-cli.mjs verify product-testops/selection.json
|
|
- name: Finish the original TestOps job run
|
|
if: always()
|
|
run: allurectl job-run stop
|
|
|
|
publish_history:
|
|
name: Publish S3 history and Pages bundle with declared screenshots
|
|
needs: [authorize, catalog, report]
|
|
if: always() && needs.catalog.result == 'success' && needs.catalog.outputs.operations_cadence == '' && needs.authorize.outputs.evidence_retention == 'actions' && needs.report.outputs.history_source_ready == 'true' && inputs.image_only != true
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 15
|
|
outputs:
|
|
pages_artifact_name: ${{ steps.pages_artifact_name.outputs.name }}
|
|
concurrency:
|
|
group: runner-e2e-history-publish
|
|
cancel-in-progress: false
|
|
permissions:
|
|
contents: read
|
|
id-token: write
|
|
environment:
|
|
name: runner-e2e-history
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
# Never execute target-controlled publication code with AWS credentials.
|
|
ref: ${{ github.sha }}
|
|
persist-credentials: false
|
|
|
|
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
|
|
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
|
|
env:
|
|
NPM_CONFIG_AUDIT: "false"
|
|
NPM_CONFIG_FUND: "false"
|
|
NPM_CONFIG_UPDATE_NOTIFIER: "false"
|
|
with:
|
|
version: 9.15.4
|
|
|
|
# Use a job-local store and disable lifecycle scripts throughout install.
|
|
# OIDC is available to every step, even before the AWS credential exchange.
|
|
- name: Apply reviewed publication lock corrections
|
|
run: node tests/runner-e2e/prepare-report-lock.mjs
|
|
|
|
- name: Install pinned publication dependencies
|
|
run: pnpm install --frozen-lockfile --ignore-scripts --store-dir "${{ runner.temp }}/eval-publisher-store"
|
|
|
|
- name: Install publisher-only Chromium
|
|
run: pnpm exec playwright install --with-deps --only-shell chromium
|
|
|
|
- name: Download access-controlled normalized campaign
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
name: runner-e2e-report-${{ github.run_id }}-${{ github.run_attempt }}
|
|
path: runner-e2e-merged-report
|
|
|
|
- name: Exchange GitHub OIDC identity for scoped AWS credentials
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd # v6
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_HISTORY_AWS_REGION }}
|
|
|
|
- name: Publish trusted summary and declared screenshots to public bundles
|
|
env:
|
|
PAPERCLIP_RUNNER_E2E_REPORT_DIR: ${{ github.workspace }}/runner-e2e-merged-report/normalized
|
|
RUNNER_E2E_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_HISTORY_S3_BUCKET }}
|
|
RUNNER_E2E_HISTORY_PREFIX: ${{ vars.RUNNER_E2E_HISTORY_PREFIX || 'runner-e2e' }}
|
|
RUNNER_E2E_HISTORY_PUBLIC_BASE_URL: ${{ vars.RUNNER_E2E_HISTORY_PUBLIC_BASE_URL }}
|
|
run: pnpm test:e2e:runner:history:publish
|
|
|
|
- name: Resolve Pages artifact name
|
|
id: pages_artifact_name
|
|
if: vars.RUNNER_FULL_STACK_E2E_PUBLISH_PAGES == 'true'
|
|
run: echo "name=github-pages-${{ github.run_id }}-${{ github.run_attempt }}" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Package pruned dashboard with declared screenshots for GitHub Pages
|
|
if: vars.RUNNER_FULL_STACK_E2E_PUBLISH_PAGES == 'true'
|
|
uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0
|
|
with:
|
|
name: ${{ steps.pages_artifact_name.outputs.name }}
|
|
path: runner-e2e-merged-report/pages
|
|
|
|
pages:
|
|
name: Publish latest dashboard with declared screenshots
|
|
needs: [report, publish_history]
|
|
if: always() && needs.report.outputs.history_source_ready == 'true' && needs.publish_history.result == 'success' && vars.RUNNER_FULL_STACK_E2E_PUBLISH_PAGES == 'true' && inputs.image_only != true
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
pages: write
|
|
id-token: write
|
|
environment:
|
|
name: github-pages
|
|
url: ${{ steps.deployment.outputs.page_url }}
|
|
steps:
|
|
- name: Deploy to GitHub Pages
|
|
id: deployment
|
|
uses: actions/deploy-pages@368f82528645a54fb793d4d04e342629a3f51346 # v5.0.1
|
|
with:
|
|
# If only this failed job is rerun, GitHub retains the successful
|
|
# publisher job's output from the earlier workflow attempt.
|
|
artifact_name: ${{ needs.publish_history.outputs.pages_artifact_name }}
|