test(runner): add full-stack acceptance and eval gates (#12700)

## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The runner subsystem executes agent work across local and managed provider backends. > - The lower pull requests restore the task runtime, provider backends, and managed-provider control plane. > - The restored system needs repeatable full-stack checks before it can ship safely. > - Paid live checks also need clear access, cost, and secret controls. > - This pull request adds acceptance, live evaluation, chaos, and release gates for the restored runner stack. > - The benefit is measurable runner parity with safer release decisions. ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting. This change covers runner tests, release workflows, server contracts, and evaluation tools. **Problem or motivation** The runner stack did not have one complete acceptance surface for native Codex, ACPX, Claude Managed, and AWS AgentCore. Release checks could miss provider drift, task-view regressions, cost-policy errors, and destructive cleanup errors. **Proposed solution** Add a 57-cell full-stack catalog, a Daytona image, and opt-in paid workflows. Add live evaluation, chaos, cost-limit, redaction, and release contract checks. Add AWS AgentCore infrastructure and guarded provisioning tools. Keep the native runner experimental flag off by default. **Alternatives considered** We considered manual smoke tests only. They do not give repeatable evidence and they do not protect release branches. We also considered one large pull request. The stacked pull requests keep each review below the Greptile file limit. **Roadmap alignment** This work supports the shipped Cloud / Sandbox agents milestone and the shipped Agent evals & feedback milestone in `ROADMAP.md`. Related stack: - #12699 adds managed provider backends and lifecycle support. - #12691 adds qualified OpenCode and ACPX provider backends. - #12685 restores task runtime rendering and steering. ## What Changed - Add the runner full-stack harness with 57 catalog cells and 60 unit tests. - Add a Daytona runner image with digest-pinned base images and base-aware image-content checks. - Add guarded live evaluation and chaos workflows with a fixed 40-execution matrix; live and full-stack paid schedules now run only on Sundays or by manual dispatch. - Add in-flight reported-usage cost stops, post-turn cost caps, exact-threshold failure classification, secret redaction, retry classification, and actor authorization. - Reattach stream and hard-budget listeners before restart-recovery continuations so restored paid sessions cannot bypass in-flight interruption. - Preserve OpenCode usage and cost across tool-loop messages and turns while exposing an explicit current-run delta to durable accounting. - Keep PNG/WebM evidence in access-controlled artifacts only, reject SVG, and publish only pruned inert structured per-attempt evidence. - Add AWS AgentCore infrastructure, provisioning checks, and smoke tools; reject unsafe model identifiers, require exact stack ownership markers, and make failed-stack replacement explicit. - Add evaluation-session contracts and capability reports. - Add release workflow checks for immutable action pins, frozen dependency installs, exact weekly cron shape, paid-run guards, provider-secret isolation, and chaos test paths. - Reauthorize the original and triggering numeric actor IDs as the first step of every provider-secret job, including partial reruns, before checkout or provider access. - Give each full-stack matrix cell only its matching provider credential, expose Daytona only to Daytona cells, and disable shared dependency caches anywhere paid credentials or OIDC write access are present. - Protect the legacy manual E2E workflow with the same default-branch, allowlist, environment, and per-job authorization boundary. - Rotate live-eval candidates by week and retain 120 days of compatible history so the seven-week trend window remains viable. - Restore the root runner-acceptance commands and reconcile reported snapshots, raw receipts, and terminal usage without double counting or losing late usage. - Mark ACPX token deltas exact only when every budget field is present, keep cumulative cost/request authority separate, reject non-USD cost labeling, and include thought tokens in output-token budgets. - Keep `enableNativeRunner` off by default. The acceptance harness enables it only in its isolated test instance. ## Verification Passed locally: - `pnpm --filter @paperclipai/paperclip-runner typecheck` - `pnpm test:runner-acceptance:typecheck` - `pnpm test:runner-acceptance` (19 tests) - focused OpenCode proxy, driver, runnerd transport, live-session, and turn-stream tests (106 tests) - `pnpm --filter @paperclipai/paperclip-runner exec vitest run src/live/clean-room-server.test.ts` (22 tests) - `pnpm test:e2e:runner:typecheck` - `pnpm test:e2e:runner:unit` (62 tests) - `node --test scripts/__tests__/release-verify-workflow.test.mjs` - `pnpm --filter @paperclipai/paperclip-runner test:runner-workflow-evals` (22 tests) - `pnpm -r typecheck` - `pnpm build` - `node --test packages/paperclip-runner/scripts/aws-agentcore-provisioning.test.mjs` (6 tests) - `git diff --check` - `cargo test --manifest-path packages/paperclip-runner/runner/Cargo.toml -p paperclip-runner-core --lib --locked` (161 tests) - focused ACPX provider-event tests (10 tests) - The rebased PR changes 92 files. `pnpm-lock.yaml` is unchanged. I did not run paid live provider jobs or provision AWS resources. Those checks need credentials and can create cost. ## Risks The paid workflows can create provider cost. They require an allowlisted original and triggering actor, the protected `runner-e2e-paid` environment, explicit opt-in variables, and cost limits. The four provider credentials exist only in that master-only environment, which requires allowlisted reviewer approval and disables administrator bypass; repository and organization Actions scopes contain no copies. Provider usage arrives after a billable request, so the live guard cannot prevent one request from crossing a threshold. It interrupts immediately on the first reported threshold hit and permits no continuation. Visual evidence can contain secrets rendered as pixels. PNG/WebM remain only in access-controlled workflow artifacts; SVG and per-attempt XML are excluded, and S3/Pages receive a pruned structured dashboard. The AWS scripts can create cloud resources. They use explicit commands, least-privilege roles, KMS encryption, saved nonsecret metadata, and explicit teardown. This pull request does not enable the experimental native runner for existing instances. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex with GPT-5. The model used extended reasoning, tool use, code execution, and parallel subagents. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge
2026-10-06 21:05:21 +02:00 · 2026-09-02 08:55:08 -05:00
1 parent 109d81db4f
commit 5716fe907e
@@ -9,23 +9,65 @@ on:
        default: true

 jobs:
+  authorize:
+    name: Authorize optional paid E2E
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    permissions:
+      contents: read
+    steps:
+      - name: Require default branch and allowlisted numeric actor IDs
+        env:
+          GH_TOKEN: ${{ github.token }}
+          REF: ${{ github.ref }}
+          DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+          ACTOR_ID: ${{ github.actor_id }}
+          TRIGGERING_ACTOR: ${{ github.triggering_actor }}
+          ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
+        run: |
+          set -euo pipefail
+          test "$REF" = "refs/heads/$DEFAULT_BRANCH"
+          jq -e 'type == "array" and length > 0 and all(.[]; type == "number" and . > 0 and floor == .)' <<< "${ALLOWED_ACTOR_IDS:-}" >/dev/null
+          triggering_actor_id="$(gh api "users/$TRIGGERING_ACTOR" --jq .id)"
+          jq -e --argjson candidate "$triggering_actor_id" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
+          jq -e --argjson candidate "$ACTOR_ID" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
+
  e2e:
+    needs: authorize
    runs-on: ubuntu-latest
    timeout-minutes: 30
+    permissions:
+      contents: read
+    environment:
+      name: runner-e2e-paid
    env:
      PAPERCLIP_E2E_SKIP_LLM: ${{ inputs.skip_llm && 'true' || 'false' }}
-      ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
    steps:
-      - uses: actions/checkout@v7
+      - name: Reauthorize execution before optional provider access
+        env:
+          GH_TOKEN: ${{ github.token }}
+          REF: ${{ github.ref }}
+          DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+          ACTOR_ID: ${{ github.actor_id }}
+          TRIGGERING_ACTOR: ${{ github.triggering_actor }}
+          ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
+        run: |
+          set -euo pipefail
+          test "$REF" = "refs/heads/$DEFAULT_BRANCH"
+          jq -e 'type == "array" and length > 0 and all(.[]; type == "number" and . > 0 and floor == .)' <<< "${ALLOWED_ACTOR_IDS:-}" >/dev/null
+          triggering_actor_id="$(gh api "users/$TRIGGERING_ACTOR" --jq .id)"
+          jq -e --argjson candidate "$triggering_actor_id" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
+          jq -e --argjson candidate "$ACTOR_ID" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null

-      - uses: pnpm/action-setup@v6
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+
+      - uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
        with:
          version: 9

-      - uses: actions/setup-node@v7
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
        with:
          node-version: 24
-          cache: pnpm

      - run: pnpm install --frozen-lockfile
      - run: pnpm build
@@ -34,9 +76,10 @@ jobs:
      - name: Run e2e tests
        env:
          PAPERCLIP_PLAYWRIGHT_CHANNEL: "chrome"
+          ANTHROPIC_API_KEY: ${{ !inputs.skip_llm && secrets.ANTHROPIC_API_KEY || '' }}
        run: pnpm run test:e2e

-      - uses: actions/upload-artifact@v7
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
        if: always()
        with:
          name: playwright-report
@@ -9,6 +9,12 @@ on:
        type: string

 jobs:
+  runner_chaos_evals:
+    name: Pre-release Runner chaos evals
+    uses: ./.github/workflows/runner-chaos-evals.yml
+    with:
+      ref: ${{ inputs.ref }}
+
  typecheck:
    name: Typecheck
    runs-on: ubuntu-latest
@@ -18,17 +24,17 @@ jobs:

    steps:
      - name: Checkout repository
-        uses: actions/checkout@v7
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
        with:
          ref: ${{ inputs.ref }}

      - name: Setup pnpm
-        uses: pnpm/action-setup@v6
+        uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
        with:
          version: 9.15.4

      - name: Setup Node.js
-        uses: actions/setup-node@v7
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
        with:
          node-version: 24
          cache: pnpm
@@ -79,17 +85,17 @@ jobs:

    steps:
      - name: Checkout repository
-        uses: actions/checkout@v7
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
        with:
          ref: ${{ inputs.ref }}

      - name: Setup pnpm
-        uses: pnpm/action-setup@v6
+        uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
        with:
          version: 9.15.4

      - name: Setup Node.js
-        uses: actions/setup-node@v7
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
        with:
          node-version: 24
          cache: pnpm
@@ -134,17 +140,17 @@ jobs:

    steps:
      - name: Checkout repository
-        uses: actions/checkout@v7
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
        with:
          ref: ${{ inputs.ref }}

      - name: Setup pnpm
-        uses: pnpm/action-setup@v6
+        uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
        with:
          version: 9.15.4

      - name: Setup Node.js
-        uses: actions/setup-node@v7
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
        with:
          node-version: 24
          cache: pnpm
@@ -155,6 +161,36 @@ jobs:
      - name: Run serialized server test shard
        run: pnpm test:run:serialized -- --shard-index ${{ matrix.shard_index }} --shard-count ${{ matrix.shard_count }}

+  runner_workflow_evals:
+    name: Runner workflow eval scorer contract
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    permissions:
+      contents: read
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          ref: ${{ inputs.ref }}
+
+      - name: Setup pnpm
+        uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
+        with:
+          version: 9.15.4
+
+      - name: Setup Node.js
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+        with:
+          node-version: 24
+          cache: pnpm
+
+      - name: Install dependencies
+        run: pnpm install --frozen-lockfile
+
+      - name: Run deterministic Runner workflow scorer tests
+        run: pnpm test:runner-workflow-evals
+
  build:
    name: Build
    runs-on: ubuntu-latest
@@ -164,17 +200,17 @@ jobs:

    steps:
      - name: Checkout repository
-        uses: actions/checkout@v7
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
        with:
          ref: ${{ inputs.ref }}

      - name: Setup pnpm
-        uses: pnpm/action-setup@v6
+        uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
        with:
          version: 9.15.4

      - name: Setup Node.js
-        uses: actions/setup-node@v7
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
        with:
          node-version: 24
          cache: pnpm
@@ -0,0 +1,83 @@
+name: Runner Chaos Evals
+
+on:
+  schedule:
+    - cron: "43 7 * * 0"
+  workflow_dispatch:
+  workflow_call:
+    inputs:
+      ref:
+        description: Commit SHA, branch, or tag to verify before release
+        required: false
+        type: string
+
+concurrency:
+  group: runner-chaos-evals-${{ inputs.ref || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  chaos_and_recovery:
+    name: Restart, replay, trace, and recovery faults
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
+    permissions:
+      contents: read
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          ref: ${{ inputs.ref || github.sha }}
+
+      - name: Setup pnpm
+        uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
+        with:
+          version: 9.15.4
+
+      - name: Setup Node.js
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+        with:
+          node-version: 24
+          cache: pnpm
+
+      - name: Install dependencies
+        run: pnpm install --frozen-lockfile
+
+      - name: Build eval and Runner contracts
+        run: |
+          pnpm --filter @paperclipai/paperclip-eval-kernel build
+          pnpm --filter @paperclipai/paperclip-runner report:runner-chaos-evals
+
+      - name: Run Runner fault and replay suites
+        run: |
+          pnpm --filter @paperclipai/paperclip-runner exec vitest run \
+            src/eval/workflow-evals.test.ts \
+            src/native-session-runtime.test.ts \
+            src/live/live-session.test.ts \
+            src/live/turn-stream.test.ts \
+            src/protocol/replay-contract.test.ts \
+            src/drivers/opencode/mcp-bridge.test.ts \
+            src/drivers/acpx/runtime-host.test.ts
+
+      - name: Build server test dependencies
+        run: pnpm --filter @paperclipai/plugin-sdk ensure-build-deps
+
+      - name: Run server finalization and recovery suites
+        run: |
+          pnpm --filter @paperclipai/server exec vitest run \
+            src/__tests__/native-finalization-recovery.test.ts \
+            src/__tests__/heartbeat-process-recovery.test.ts \
+            src/__tests__/heartbeat-comment-wake-batching.test.ts \
+            src/__tests__/heartbeat-dependency-scheduling.test.ts \
+            src/__tests__/provider-trace-store.test.ts \
+            src/services/issue-thread-interaction-resolution.test.ts \
+            src/services/recovery/successful-run-handoff.test.ts
+
+      - name: Upload chaos eval bundle
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
+        with:
+          name: runner-chaos-evals-${{ github.run_id }}
+          path: packages/paperclip-runner/.paperclip-local/evals/workflows/
+          retention-days: 30
+          if-no-files-found: error
@@ -0,0 +1,535 @@
+name: Runner Full-Stack E2E
+
+on:
+  schedule:
+    - cron: "47 8 * * 0"
+  workflow_dispatch:
+    inputs:
+      all:
+        description: "Run the complete paid matrix when no narrower selector is supplied"
+        type: boolean
+        default: true
+      group:
+        description: "Comma-separated groups (AND semantics: legacy,native,local,daytona,core,breadth)"
+        type: string
+        required: false
+      suite:
+        description: "Comma-separated suite IDs"
+        type: string
+        required: false
+      profile:
+        description: "Comma-separated runner profile fixture IDs"
+        type: string
+        required: false
+      environment:
+        description: "Comma-separated environment fixture IDs"
+        type: string
+        required: false
+      case:
+        description: "Comma-separated task case fixture IDs"
+        type: string
+        required: false
+      id:
+        description: "Comma-separated full suite.profile.environment.case IDs; exclusive with other selectors"
+        type: string
+        required: false
+
+permissions:
+  contents: read
+
+concurrency:
+  group: runner-full-stack-e2e-${{ github.ref }}
+  cancel-in-progress: false
+
+jobs:
+  authorize:
+    name: Authorize paid campaign
+    if: github.event_name != 'schedule' || vars.RUNNER_FULL_STACK_E2E_NIGHTLY_ENABLED == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    permissions:
+      contents: read
+    steps:
+      - name: Require default branch and allowlisted numeric actor IDs
+        env:
+          GH_TOKEN: ${{ github.token }}
+          REPOSITORY: ${{ github.repository }}
+          REF: ${{ github.ref }}
+          DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+          ACTOR: ${{ github.actor }}
+          ACTOR_ID: ${{ github.actor_id }}
+          TRIGGERING_ACTOR: ${{ github.triggering_actor }}
+          ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
+        run: |
+          set -euo pipefail
+          if [ "$REF" != "refs/heads/$DEFAULT_BRANCH" ]; then
+            echo "Paid runner E2E campaigns may run only from the default branch." >&2
+            exit 1
+          fi
+          if ! jq -e 'type == "array" and length > 0 and all(.[]; type == "number" and . > 0 and floor == .)' <<< "${ALLOWED_ACTOR_IDS:-}" >/dev/null; then
+            echo "RUNNER_E2E_ALLOWED_ACTOR_IDS must be a non-empty JSON array of numeric GitHub user IDs." >&2
+            exit 1
+          fi
+          triggering_actor_id="$(gh api "users/$TRIGGERING_ACTOR" --jq .id)"
+          if [ "$triggering_actor_id" != "$ACTOR_ID" ] && [ "$TRIGGERING_ACTOR" = "$ACTOR" ]; then
+            echo "GitHub actor identity contexts disagree; refusing the paid run." >&2
+            exit 1
+          fi
+          candidates=("$triggering_actor_id" "$ACTOR_ID")
+          for candidate in "${candidates[@]}"; do
+            if ! jq -e --argjson candidate "$candidate" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null; then
+              echo "The initiating GitHub account is not authorized to run paid runner E2E campaigns." >&2
+              exit 1
+            fi
+          done
+
+  catalog:
+    name: Validate catalog and select cells
+    needs: authorize
+    if: github.event_name != 'schedule' || vars.RUNNER_FULL_STACK_E2E_NIGHTLY_ENABLED == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    permissions:
+      contents: read
+    outputs:
+      matrix: ${{ steps.catalog.outputs.matrix }}
+      needs_daytona: ${{ steps.catalog.outputs.needs_daytona }}
+      execution_ids: ${{ steps.catalog.outputs.execution_ids }}
+      max_parallel: ${{ steps.catalog.outputs.max_parallel }}
+      daytona_image_content_id: ${{ steps.daytona_image_content.outputs.content_id }}
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+
+      - uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
+        with:
+          version: 9.15.4
+
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+        with:
+          node-version: 24
+          cache: pnpm
+
+      - run: pnpm install --frozen-lockfile
+
+      # The v2 contract fails closed unless every Docker FROM is digest-pinned,
+      # and hashes those exact base references into the immutable image tag.
+      - name: Compute Daytona image content ID with pinned bases
+        id: daytona_image_content
+        run: echo "content_id=$(pnpm --silent test:e2e:runner:image-id)" >> "$GITHUB_OUTPUT"
+
+      - name: Validate selectors and emit matrix
+        id: catalog
+        env:
+          EVENT_NAME: ${{ github.event_name }}
+          SELECT_ALL: ${{ inputs.all }}
+          SELECT_SUITE: ${{ inputs.suite }}
+          SELECT_GROUP: ${{ inputs.group }}
+          SELECT_PROFILE: ${{ inputs.profile }}
+          SELECT_ENVIRONMENT: ${{ inputs.environment }}
+          SELECT_CASE: ${{ inputs.case }}
+          SELECT_ID: ${{ inputs.id }}
+          MAX_PARALLEL: ${{ vars.RUNNER_E2E_MAX_PARALLEL || '32' }}
+        run: |
+          set -euo pipefail
+          args=(--matrix-json)
+          add_values() {
+            local flag="$1"
+            local values="$2"
+            local value
+            IFS=',' read -ra entries <<< "$values"
+            for value in "${entries[@]}"; do
+              value="${value#"${value%%[![:space:]]*}"}"
+              value="${value%"${value##*[![:space:]]}"}"
+              if [ -n "$value" ]; then
+                args+=("$flag" "$value")
+              fi
+            done
+          }
+          explicit=false
+          if [ -n "${SELECT_ID:-}" ]; then
+            if [ -n "${SELECT_SUITE:-}${SELECT_GROUP:-}${SELECT_PROFILE:-}${SELECT_ENVIRONMENT:-}${SELECT_CASE:-}" ]; then
+              echo "The id selector is exclusive with suite/group/profile/environment/case" >&2
+              exit 1
+            fi
+            add_values --id "$SELECT_ID"
+            explicit=true
+          else
+            for pair in \
+              "--suite:${SELECT_SUITE:-}" \
+              "--group:${SELECT_GROUP:-}" \
+              "--profile:${SELECT_PROFILE:-}" \
+              "--environment:${SELECT_ENVIRONMENT:-}" \
+              "--case:${SELECT_CASE:-}"
+            do
+              flag="${pair%%:*}"
+              values="${pair#*:}"
+              if [ -n "$values" ]; then
+                add_values "$flag" "$values"
+                explicit=true
+              fi
+            done
+          fi
+          if [ "$explicit" = false ] && { [ "$EVENT_NAME" = schedule ] || [ "${SELECT_ALL:-false}" = true ]; }; then
+            args+=(--all)
+          fi
+          catalog_json="$(pnpm --silent test:e2e:runner -- "${args[@]}")"
+          echo "matrix=$(jq -c '{include: .include}' <<< "$catalog_json")" >> "$GITHUB_OUTPUT"
+          echo "needs_daytona=$(jq -r '.needsDaytona' <<< "$catalog_json")" >> "$GITHUB_OUTPUT"
+          echo "execution_ids=$(jq -c '.executionIds' <<< "$catalog_json")" >> "$GITHUB_OUTPUT"
+          if ! [[ "$MAX_PARALLEL" =~ ^[1-9][0-9]*$ ]] || [ "$MAX_PARALLEL" -gt 57 ]; then
+            echo "RUNNER_E2E_MAX_PARALLEL must be an integer from 1 through 57." >&2
+            exit 1
+          fi
+          echo "max_parallel=$MAX_PARALLEL" >> "$GITHUB_OUTPUT"
+
+  daytona_image:
+    name: Publish verified Daytona image
+    needs: [authorize, catalog]
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
+    permissions:
+      contents: read
+      packages: write
+      id-token: write
+    outputs:
+      image: ${{ steps.image.outputs.image }}
+      source_revision: ${{ steps.image.outputs.source_revision }}
+      content_id: ${{ steps.image.outputs.content_id }}
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+
+      - name: No Daytona image needed
+        id: local_only
+        if: needs.catalog.outputs.needs_daytona != 'true'
+        run: echo "image=" >> "$GITHUB_OUTPUT"
+
+      - name: Set up Docker Buildx
+        if: needs.catalog.outputs.needs_daytona == 'true'
+        uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
+
+      - name: Log into GHCR
+        if: needs.catalog.outputs.needs_daytona == 'true'
+        uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Install cosign
+        if: needs.catalog.outputs.needs_daytona == 'true'
+        uses: sigstore/cosign-installer@398d4b0eeef1380460a10c8013a76f728fb906ac # v3
+
+      - name: Reuse or publish immutable image
+        id: image
+        env:
+          NEEDS_DAYTONA: ${{ needs.catalog.outputs.needs_daytona }}
+          IMAGE_CONTENT_ID: ${{ needs.catalog.outputs.daytona_image_content_id }}
+          IMAGE_TAG: ghcr.io/paperclipai/paperclip-daytona-runner:e2e-content-${{ needs.catalog.outputs.daytona_image_content_id }}
+        run: |
+          set -euo pipefail
+          if [ "$NEEDS_DAYTONA" != true ]; then
+            echo "image=" >> "$GITHUB_OUTPUT"
+            echo "source_revision=" >> "$GITHUB_OUTPUT"
+            echo "content_id=" >> "$GITHUB_OUTPUT"
+            exit 0
+          fi
+          [[ "$IMAGE_CONTENT_ID" =~ ^[0-9a-f]{64}$ ]]
+          identity="^https://github.com/${GITHUB_REPOSITORY}/.github/workflows/runner-full-stack-e2e.yml@"
+          if docker buildx imagetools inspect "$IMAGE_TAG" >/dev/null 2>&1; then
+            digest="$(docker buildx imagetools inspect "$IMAGE_TAG" --format '{{json .Manifest.Digest}}' | tr -d '"')"
+          else
+            docker buildx build \
+              --platform linux/amd64 \
+              --build-arg "PAPERCLIP_RUNNER_CONTENT_ID=${IMAGE_CONTENT_ID}" \
+              --build-arg "PAPERCLIP_RUNNER_SOURCE_REVISION=${GITHUB_SHA}" \
+              --file docker/daytona-runner/Dockerfile \
+              --tag "$IMAGE_TAG" \
+              --push \
+              .
+            digest="$(docker buildx imagetools inspect "$IMAGE_TAG" --format '{{json .Manifest.Digest}}' | tr -d '"')"
+            cosign sign --yes "$IMAGE_TAG@$digest"
+          fi
+          cosign verify \
+            --certificate-identity-regexp "$identity" \
+            --certificate-oidc-issuer https://token.actions.githubusercontent.com \
+            "$IMAGE_TAG@$digest" >/dev/null
+          immutable="${IMAGE_TAG%:*}@$digest"
+          # The Daytona base image is large. The build cache plus a second full
+          # anonymous pull can exhaust a standard GitHub-hosted runner before
+          # Docker creates the tiny metadata-probe container. The pushed digest
+          # is already immutable, so release the local builder/cache first.
+          docker buildx prune --all --force >/dev/null
+          docker system prune --all --force >/dev/null
+          anonymous_config="$(mktemp -d)"
+          docker --config "$anonymous_config" pull "$immutable"
+          # The Dockerfile's final two RUN steps execute the runner metadata,
+          # transport-mode, provider-pack JSON, and pinned ACP binary checks as
+          # root and as the unprivileged Daytona user. Starting another
+          # container after this full pull can exhaust the hosted runner's thin
+          # writable layer even after pruning, so assert the published image
+          # configuration here without creating a redundant container.
+          image_config="$(docker image inspect "$immutable" \
+            --format '{{json .}}')"
+          published_content_id="$(jq -r '.Config.Labels["io.paperclip.runner.content-id"] // empty' <<< "$image_config")"
+          source_revision="$(jq -r '.Config.Labels["org.opencontainers.image.revision"] // empty' <<< "$image_config")"
+          test "$published_content_id" = "$IMAGE_CONTENT_ID"
+          [[ "$source_revision" =~ ^[0-9a-f]{40}$ ]]
+          jq -e \
+            '.Architecture == "amd64" and
+             .Os == "linux" and
+             .Config.User == "daytona" and
+             (.Config.Env | any(startswith("PAPERCLIP_RUNNER_PROVIDER_PACK_ROOT=")))' \
+            <<< "$image_config" >/dev/null
+          echo "image=$immutable" >> "$GITHUB_OUTPUT"
+          echo "source_revision=$source_revision" >> "$GITHUB_OUTPUT"
+          echo "content_id=$published_content_id" >> "$GITHUB_OUTPUT"
+
+  test:
+    name: ${{ matrix.executionId }}
+    needs: [catalog, daytona_image]
+    runs-on: ubuntu-latest-m
+    timeout-minutes: ${{ matrix.timeoutMinutes }}
+    permissions:
+      contents: read
+    environment:
+      name: runner-e2e-paid
+    strategy:
+      fail-fast: false
+      max-parallel: ${{ fromJSON(needs.catalog.outputs.max_parallel) }}
+      matrix: ${{ fromJSON(needs.catalog.outputs.matrix) }}
+    steps:
+      - name: Reauthorize paid execution before provider access
+        env:
+          GH_TOKEN: ${{ github.token }}
+          REF: ${{ github.ref }}
+          DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+          ACTOR_ID: ${{ github.actor_id }}
+          TRIGGERING_ACTOR: ${{ github.triggering_actor }}
+          ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
+        run: |
+          set -euo pipefail
+          test "$REF" = "refs/heads/$DEFAULT_BRANCH"
+          jq -e 'type == "array" and length > 0 and all(.[]; type == "number" and . > 0 and floor == .)' <<< "${ALLOWED_ACTOR_IDS:-}" >/dev/null
+          triggering_actor_id="$(gh api "users/$TRIGGERING_ACTOR" --jq .id)"
+          jq -e --argjson candidate "$triggering_actor_id" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
+          jq -e --argjson candidate "$ACTOR_ID" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
+
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+
+      - uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
+        with:
+          version: 9.15.4
+
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+        with:
+          node-version: 24
+
+      - run: pnpm install --frozen-lockfile
+
+      - name: Build runner TypeScript prerequisites
+        run: pnpm --filter @paperclipai/paperclip-eval-kernel build
+
+      - name: Build native remote provider pack
+        if: matrix.environmentId == 'daytona' && (matrix.profileId == 'runner-opencode' || startsWith(matrix.profileId, 'runner-acpx-'))
+        env:
+          # Reused images retain the source revision that was embedded in their
+          # provider pack. Matching it here lets the server reuse that exact
+          # preinstalled pack instead of uploading a duplicate to the lease.
+          PAPERCLIP_RUNNER_SOURCE_REVISION: ${{ needs.daytona_image.outputs.source_revision }}
+        run: pnpm --filter @paperclipai/paperclip-runner build:provider-pack
+
+      - name: Install pinned legacy Claude CLI
+        if: matrix.profileId == 'legacy-claude'
+        run: npm install --global --omit=dev @anthropic-ai/claude-code@2.1.19
+
+      - name: Build native runner binaries
+        if: startsWith(matrix.profileId, 'runner-') || matrix.suiteId == 'openrouter-model-breadth'
+        run: pnpm --filter @paperclipai/paperclip-runner build:runner-binaries
+
+      - name: Install Chromium
+        run: pnpm exec playwright install --with-deps chromium
+
+      - name: Run paid cell
+        env:
+          OPENAI_API_KEY: ${{ matrix.credentialName == 'OPENAI_API_KEY' && secrets.OPENAI_API_KEY || '' }}
+          ANTHROPIC_API_KEY: ${{ matrix.credentialName == 'ANTHROPIC_API_KEY' && secrets.ANTHROPIC_API_KEY || '' }}
+          OPENROUTER_API_KEY: ${{ matrix.credentialName == 'OPENROUTER_API_KEY' && secrets.OPENROUTER_API_KEY || '' }}
+          DAYTONA_API_KEY: ${{ matrix.environmentId == 'daytona' && secrets.DAYTONA_API_KEY || '' }}
+          PAPERCLIP_E2E_DAYTONA_IMAGE: ${{ needs.daytona_image.outputs.image }}
+          PAPERCLIP_RUNNER_REMOTE_PROVIDER_PACK_PATH: ${{ github.workspace }}/packages/paperclip-runner/provider-pack
+          PAPERCLIP_E2E_CAMPAIGN_ID: gha-${{ github.run_id }}-${{ github.run_attempt }}-${{ matrix.executionId }}
+        run: pnpm test:e2e:runner -- --id "${{ matrix.executionId }}"
+
+      - name: Upload access-controlled packaged cell evidence
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
+        with:
+          name: runner-e2e-${{ github.run_id }}-${{ github.run_attempt }}-${{ matrix.executionId }}
+          path: tests/runner-e2e/results/
+          retention-days: 30
+          if-no-files-found: error
+
+  report:
+    name: Merge and enforce campaign result
+    if: always() && needs.catalog.result == 'success'
+    needs: [catalog, daytona_image, test]
+    outputs:
+      history_source_ready: ${{ steps.history_source_ready.outputs.ready }}
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+
+      - uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
+        with:
+          version: 9.15.4
+
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+        with:
+          node-version: 24
+          cache: pnpm
+
+      - run: pnpm install --frozen-lockfile
+
+      - name: Download cell evidence
+        id: download_evidence
+        continue-on-error: true
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
+        with:
+          pattern: runner-e2e-${{ github.run_id }}-${{ github.run_attempt }}-*
+          path: downloaded-runner-e2e
+          merge-multiple: true
+
+      - name: Retry cell evidence download after transport failure
+        if: steps.download_evidence.outcome == 'failure'
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
+        with:
+          pattern: runner-e2e-${{ github.run_id }}-${{ github.run_attempt }}-*
+          path: downloaded-runner-e2e
+          merge-multiple: true
+
+      - name: Collect blob reports
+        run: |
+          set -euo pipefail
+          mkdir -p merged-blob-reports
+          while IFS= read -r -d '' report; do
+            digest="$(sha256sum "$report" | cut -d ' ' -f 1)"
+            target="merged-blob-reports/report-${digest}.zip"
+            if [ ! -e "$target" ]; then
+              cp "$report" "$target"
+            fi
+          done < <(find downloaded-runner-e2e -path '*/blob-report/*.zip' -print0)
+
+      - name: Merge Playwright HTML and JUnit
+        if: always()
+        env:
+          PAPERCLIP_RUNNER_E2E_MERGED_REPORT_DIR: ${{ github.workspace }}/runner-e2e-merged-report
+        run: pnpm exec playwright merge-reports --config tests/runner-e2e/merge.config.ts merged-blob-reports
+
+      - name: Aggregate normalized campaign results
+        if: always()
+        env:
+          PAPERCLIP_RUNNER_E2E_REPORT_ROOT: ${{ github.workspace }}/downloaded-runner-e2e
+          PAPERCLIP_RUNNER_E2E_REPORT_OUT: ${{ github.workspace }}/runner-e2e-merged-report/normalized
+          PAPERCLIP_RUNNER_E2E_EXPECTED_IDS: ${{ needs.catalog.outputs.execution_ids }}
+          PAPERCLIP_E2E_CAMPAIGN_ID: gha-${{ github.run_id }}-${{ github.run_attempt }}
+        run: |
+          set +e
+          pnpm test:e2e:runner:report
+          report_status=$?
+          set -e
+          cat runner-e2e-merged-report/normalized/summary.md >> "$GITHUB_STEP_SUMMARY"
+          exit "$report_status"
+
+      - name: Upload access-controlled merged report
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
+        with:
+          name: runner-e2e-report-${{ github.run_id }}-${{ github.run_attempt }}
+          path: runner-e2e-merged-report/
+          retention-days: 30
+          if-no-files-found: error
+
+      - name: Verify history source report and private screenshot evidence
+        id: history_source_ready
+        if: always()
+        run: |
+          set -euo pipefail
+          dashboard_root="runner-e2e-merged-report/normalized"
+          private_screenshot="$(find "$dashboard_root" -type f -name '*.png' -print -quit 2>/dev/null || true)"
+          if [ -f "$dashboard_root/index.html" ] && [ -n "$private_screenshot" ]; then
+            echo "ready=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "ready=false" >> "$GITHUB_OUTPUT"
+          fi
+
+  publish_history:
+    name: Publish pruned immutable history and landing site
+    needs: [catalog, report]
+    if: always() && needs.catalog.result == 'success' && needs.report.outputs.history_source_ready == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    concurrency:
+      group: runner-e2e-history-publish
+      cancel-in-progress: false
+    permissions:
+      contents: read
+      id-token: write
+    environment:
+      name: runner-e2e-history
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+
+      - uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
+        with:
+          version: 9.15.4
+
+      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+        with:
+          node-version: 24
+
+      - run: pnpm install --frozen-lockfile
+
+      - name: Download access-controlled normalized campaign
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
+        with:
+          name: runner-e2e-report-${{ github.run_id }}-${{ github.run_attempt }}
+          path: runner-e2e-merged-report
+
+      - name: Exchange GitHub OIDC identity for scoped AWS credentials
+        uses: aws-actions/configure-aws-credentials@e6de054238d6b7531b4efff3b6587d9aade6a06c # v6
+        with:
+          role-to-assume: ${{ vars.RUNNER_E2E_HISTORY_AWS_ROLE_ARN }}
+          aws-region: ${{ vars.RUNNER_E2E_HISTORY_AWS_REGION }}
+
+      - name: Prune private evidence and publish immutable campaign history
+        env:
+          PAPERCLIP_RUNNER_E2E_REPORT_DIR: ${{ github.workspace }}/runner-e2e-merged-report/normalized
+          RUNNER_E2E_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_HISTORY_S3_BUCKET }}
+          RUNNER_E2E_HISTORY_PREFIX: ${{ vars.RUNNER_E2E_HISTORY_PREFIX || 'runner-e2e' }}
+          RUNNER_E2E_HISTORY_PUBLIC_BASE_URL: ${{ vars.RUNNER_E2E_HISTORY_PUBLIC_BASE_URL }}
+        run: pnpm test:e2e:runner:history:publish
+
+      - name: Package pruned structured dashboard for GitHub Pages
+        if: vars.RUNNER_FULL_STACK_E2E_PUBLISH_PAGES == 'true'
+        uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4
+        with:
+          path: runner-e2e-merged-report/normalized
+
+  pages:
+    name: Publish latest structured dashboard
+    needs: [report, publish_history]
+    if: always() && needs.report.outputs.history_source_ready == 'true' && needs.publish_history.result == 'success' && vars.RUNNER_FULL_STACK_E2E_PUBLISH_PAGES == 'true'
+    runs-on: ubuntu-latest
+    permissions:
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4
@@ -0,0 +1,134 @@
+name: Runner Live Evals
+
+on:
+  schedule:
+    - cron: "17 6 * * 0"
+  workflow_dispatch:
+
+concurrency:
+  group: runner-live-evals-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  authorize:
+    name: Authorize paid campaign
+    if: github.event_name != 'schedule' || vars.RUNNER_LIVE_EVALS_NIGHTLY_ENABLED == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    permissions:
+      contents: read
+    steps:
+      - name: Require default branch and allowlisted numeric actor IDs
+        env:
+          GH_TOKEN: ${{ github.token }}
+          REF: ${{ github.ref }}
+          DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+          ACTOR: ${{ github.actor }}
+          ACTOR_ID: ${{ github.actor_id }}
+          TRIGGERING_ACTOR: ${{ github.triggering_actor }}
+          ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
+        run: |
+          set -euo pipefail
+          if [ "$REF" != "refs/heads/$DEFAULT_BRANCH" ]; then
+            echo "Paid runner live evals may run only from the default branch." >&2
+            exit 1
+          fi
+          if ! jq -e 'type == "array" and length > 0 and all(.[]; type == "number" and . > 0 and floor == .)' <<< "${ALLOWED_ACTOR_IDS:-}" >/dev/null; then
+            echo "RUNNER_E2E_ALLOWED_ACTOR_IDS must be a non-empty JSON array of numeric GitHub user IDs." >&2
+            exit 1
+          fi
+          triggering_actor_id="$(gh api "users/$TRIGGERING_ACTOR" --jq .id)"
+          if [ "$triggering_actor_id" != "$ACTOR_ID" ] && [ "$TRIGGERING_ACTOR" = "$ACTOR" ]; then
+            echo "GitHub actor identity contexts disagree; refusing the paid run." >&2
+            exit 1
+          fi
+          candidates=("$triggering_actor_id" "$ACTOR_ID")
+          for candidate in "${candidates[@]}"; do
+            if ! jq -e --argjson candidate "$candidate" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null; then
+              echo "The initiating GitHub account is not authorized to run paid runner live evals." >&2
+              exit 1
+            fi
+          done
+
+  live_matrix:
+    name: Balanced provider/model matrix
+    needs: authorize
+    if: github.event_name != 'schedule' || vars.RUNNER_LIVE_EVALS_NIGHTLY_ENABLED == 'true'
+    runs-on: ubuntu-latest
+    timeout-minutes: 180
+    permissions:
+      contents: read
+    environment:
+      name: runner-e2e-paid
+
+    steps:
+      - name: Reauthorize paid execution before provider access
+        env:
+          GH_TOKEN: ${{ github.token }}
+          REF: ${{ github.ref }}
+          DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+          ACTOR_ID: ${{ github.actor_id }}
+          TRIGGERING_ACTOR: ${{ github.triggering_actor }}
+          ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
+        run: |
+          set -euo pipefail
+          test "$REF" = "refs/heads/$DEFAULT_BRANCH"
+          jq -e 'type == "array" and length > 0 and all(.[]; type == "number" and . > 0 and floor == .)' <<< "${ALLOWED_ACTOR_IDS:-}" >/dev/null
+          triggering_actor_id="$(gh api "users/$TRIGGERING_ACTOR" --jq .id)"
+          jq -e --argjson candidate "$triggering_actor_id" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
+          jq -e --argjson candidate "$ACTOR_ID" 'index($candidate) != null' <<< "$ALLOWED_ACTOR_IDS" >/dev/null
+
+      - name: Checkout repository
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+
+      - name: Setup pnpm
+        uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
+        with:
+          version: 9.15.4
+
+      - name: Setup Node.js
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+        with:
+          node-version: 24
+
+      - name: Install dependencies
+        run: pnpm install --frozen-lockfile
+
+      - name: Restore compatible weekly baseline
+        uses: actions/cache@caa296126883cff596d87d8935842f9db880ef25 # v5
+        with:
+          path: packages/paperclip-runner/.paperclip-local/evals/workflows/history
+          key: runner-live-eval-history-${{ github.ref_name }}-${{ github.run_id }}
+          restore-keys: |
+            runner-live-eval-history-${{ github.ref_name }}-
+
+      - name: Build provider-neutral eval kernel
+        run: pnpm --filter @paperclipai/paperclip-eval-kernel build
+
+      - name: Run trend-only live matrix
+        env:
+          OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
+          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+          PAPERCLIP_EVAL_BASELINE_READY: "true"
+          PAPERCLIP_EVAL_RUNNER_BUILD: ${{ github.sha }}
+          PAPERCLIP_EVAL_MAX_CAMPAIGN_COST_USD: "12"
+          PAPERCLIP_EVAL_SCHEDULE_SEED: runner-live-seven-week-v1
+        run: pnpm --filter @paperclipai/paperclip-runner report:runner-live-evals
+
+      - name: Publish job summary
+        if: always()
+        run: |
+          summary=packages/paperclip-runner/.paperclip-local/evals/workflows/github-live-summary.md
+          if [ -f "$summary" ]; then
+            cat "$summary" >> "$GITHUB_STEP_SUMMARY"
+          fi
+
+      - name: Upload safe live eval bundle
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
+        with:
+          name: runner-live-evals-${{ github.run_id }}
+          path: packages/paperclip-runner/.paperclip-local/evals/workflows/
+          retention-days: 30
+          if-no-files-found: error