mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 14:19:00 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - The runner E2E chat evals run real agents against a local Paperclip
instance on a CI host
> - PR #20044 hides the eval harness from local native agents with a
Linux Landlock domain around each runnerd launch
> - That domain is the same for every run in an attempt, so an eval
agent can still read another run's saved state
> - Recent chat-eval runs show agents that read other runs' OpenCode
session databases, another run's run log, the instance `pg_wal`,
`control-plane-state.json`, and `/proc/<pid>/cmdline` of harness
processes
> - This pull request scopes each runnerd launch to its own run and
moves the selected case id off the command line
> - The benefit is that an eval agent cannot learn about the case or
other runs from files outside its own run
## Linked Issues or Issue Description
Refs #20044
**What happened**
In local native OpenCode chat-eval cells, worker agents read files that
belong to other runs or to the harness:
- other runs' OpenCode sqlite session databases under
`runtime/paperclip-runner/durable-sessions/` (a status-cards run)
- `/proc/<pid>/cmdline` of harness processes, which shows harness paths
and the case id (`routines.pause-owned`)
- another run's run-log ndjson, and a listing of the instance
`db/pg_wal` (`routines.pause-owned`)
- a listing of `control-plane-state.json`
(`runs-adapters.list-model-options`)
The Landlock sandbox from #20044 hides only the harness (`tests/`, the
Git store, private attempt directories). All runnerd launches share one
hidden set, so the instance's per-run directories stay readable.
**Expected behavior**
An agent process sees only its own run's durable session, OpenCode
session and provider trace. Other runs' sessions, run logs and traces,
the instance database files, secrets and the shared control-plane state
in the attempt instance are denied. The case id is not on any harness
command line.
**Steps to reproduce**
1. Dispatch a local native OpenCode chat-features cell that has two
agents (for example a status-cards case).
2. In the worker's shell, run `ls
<instance>/runtime/paperclip-runner/durable-sessions/*/` and `ls
<instance>/db/pg_wal`.
3. Run `cat /proc/<launcher pid>/cmdline`.
4. Before this change, all three succeed and show other runs' files and
`--id chat-features...`.
## What Changed
- `agent-filesystem-sandbox.py`: new `--expose PATH` option. It grants a
path beneath a hidden directory. Landlock does not check traversal, so
the run uses its own entry while the parent stays unlistable. The helper
refuses a missing exposure, an exposure outside every hidden path, or
one that would reveal a hidden path. It fails the launch unless every
exposed path is readable.
- `agent-filesystem-sandbox.ts`: the launcher passes the instance root
in the sandbox configuration. `runnerdLaunchSandbox` adds per-launch
hidden paths: `durable-sessions/`, `runner/`, `control-plane/`,
`opencode/` under `runtime/paperclip-runner/`, and `data/run-logs/`,
`data/provider-traces/`, `data/backups/`, `db/`, `secrets/`. It exposes
only the launch's own durable-session directory (from `--state-dir`),
Codex runner state directory, OpenCode session directory (from
`--session-id`, named as the OpenCode driver names it) and provider
trace file.
- The wrapper creates the exposed entries, and the shared instance
directories that the product creates on first use, before runnerd
starts. It logs the run-scoped hidden and exposed paths for each launch.
- New `probeRunScopedSandbox`: at server start, a sandboxed shell runs
against a fixture instance. Another run's session, OpenCode database,
runner state, run log, trace, `pg_wal`, secrets, control-plane state and
the server's `/proc/<pid>/environ` must be denied. The run's own
session, OpenCode directory, ACPX directory and trace must be writable.
In CI any failure stops the attempt. The result goes to `server.log`.
- `selectors.ts` and `launch.ts`: the launcher reads the selected cell
from `PAPERCLIP_RUNNER_E2E_SELECTED_ID` and removes it from its
environment.
- `runner-full-stack-e2e.yml` (Run paid cell) and
`chat-catalog-testops-case.yml`: pass the cell in that variable instead
of `--id`. A target branch that does not read the variable still gets
`--id`.
- `SECURITY.md`: documents the run scope, the `/proc` limit and what
stays readable.
## Verification
- `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts
tests/runner-e2e/agent-filesystem-sandbox.test.ts
tests/runner-e2e/workflow-security.test.ts
tests/runner-e2e/manual-private-workflow.test.ts
tests/runner-e2e/private-trust-workflow-security.test.ts` passes on
macOS. The Landlock host tests (including the new fixture-instance probe
test) run only on Linux CI.
- `npx tsc --noEmit -p tests/runner-e2e` shows no errors in the changed
files.
- Paid run
[42611358428](https://github.com/paperclipai/paperclip/actions/runs/42611358428):
`chat-features.runner-opencode.local.agents.change-manager` on the same
commit applied to `chat-eval/master-candidate` (fca4bb7ce3). Result:
passed. In `server.log` the setup probe reports `failures: []` and
`run:otherDurableSession`, `run:durableSessionListing`,
`run:otherRunnerState`, `run:controlPlaneState`,
`run:otherOpencodeSession`, `run:otherRunLog`, `run:databaseWal`,
`run:secrets`, `run:otherProviderTrace` and `run:harnessEnviron` all
`denied`; `run:ownDurableSession`, `run:ownOpencodeSession`,
`run:acpxRuntime` and `run:ownProviderTrace` `allowed`;
`run:harnessCmdline` `allowed` (recorded, Landlock limit). The runnerd
launch line lists the nine run-scoped hidden instance paths and exposes
only its own durable-session and OpenCode session directories. The agent
finished the case inside that sandbox.
## Risks
- A sandboxed runnerd cannot create new direct entries in the instance
root, `runtime/`, `runtime/paperclip-runner/` or `data/`. The wrapper
creates the known ones first (`acpx`, `companies`, `projects`,
`workspaces`, `runtime-context-assets`, `skills`,
`runtime/provider-packs`). A provider that creates a different new
directory there fails in a sandboxed local cell. The paid run covers
OpenCode only. ACPX cells keep their runtime directory readable.
- A run still reads its own session's control-plane state, and other
agents' homes and task workspaces in the same instance.
- Landlock cannot hide `/proc/<pid>/cmdline`. Harness paths stay visible
there. Only the case id moves to the environment. Scheduled multi-cell
campaigns still pass `--id`.
- Harness-only change. No product code changes.
## Model Used
- Anthropic Claude Opus 5.5 (`claude-opus-5-5`), extended thinking, tool
use (shell, file edits, GitHub CLI).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
324 lines
16 KiB
YAML
324 lines
16 KiB
YAML
name: Chat catalog TestOps case
|
|
|
|
on:
|
|
workflow_call:
|
|
inputs:
|
|
protocol:
|
|
type: string
|
|
required: true
|
|
cell:
|
|
type: string
|
|
required: true
|
|
target_sha:
|
|
type: string
|
|
required: true
|
|
runner:
|
|
type: string
|
|
required: true
|
|
selection_artifact_id:
|
|
type: string
|
|
required: true
|
|
selection_sha256:
|
|
type: string
|
|
required: true
|
|
build_artifact_id:
|
|
type: string
|
|
required: true
|
|
ALLURE_JOB_RUN_ID:
|
|
type: string
|
|
required: true
|
|
secrets:
|
|
OPENROUTER_API_KEY:
|
|
required: false
|
|
ANTHROPIC_API_KEY:
|
|
required: false
|
|
ALLURE_TOKEN:
|
|
required: false
|
|
|
|
permissions:
|
|
contents: read
|
|
actions: read
|
|
|
|
env:
|
|
NODE_OPTIONS: ''
|
|
NODE_PATH: ''
|
|
AWS_EC2_METADATA_DISABLED: 'true'
|
|
|
|
jobs:
|
|
chat:
|
|
name: Execute the original selected case
|
|
runs-on: ${{ inputs.runner }}
|
|
timeout-minutes: ${{ fromJSON(inputs.cell).timeout_minutes }}
|
|
environment: runner-e2e-private-paid
|
|
permissions:
|
|
contents: read
|
|
actions: read
|
|
id-token: write
|
|
env: &retention
|
|
TARGET_SHA: ${{ inputs.target_sha }}
|
|
MANUAL_PRIVATE_IDS: ${{ fromJSON(inputs.cell).private_ids }}
|
|
MANUAL_PRIVATE_APPROVALS_JSON: ${{ fromJSON(inputs.cell).private_policy }}
|
|
RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_S3_BUCKET }}
|
|
steps:
|
|
- &checkout
|
|
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
ref: ${{ github.sha }}
|
|
persist-credentials: false
|
|
- &node
|
|
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
|
with:
|
|
node-version: 24
|
|
- &guard
|
|
name: Verify the exact reusable protocol and main-only caller
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
TESTOPS_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
|
|
CHAT_TESTOPS_PROTOCOL: ${{ inputs.protocol }}
|
|
run: |
|
|
test "$CHAT_TESTOPS_PROTOCOL" = streaming-v1
|
|
node tests/runner-e2e/chat-completion-testops-cli.mjs guard
|
|
- &pnpm
|
|
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6
|
|
env:
|
|
NPM_CONFIG_AUDIT: 'false'
|
|
NPM_CONFIG_FUND: 'false'
|
|
NPM_CONFIG_UPDATE_NOTIFIER: 'false'
|
|
with:
|
|
version: 9.15.4
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
ref: ${{ inputs.target_sha }}
|
|
persist-credentials: false
|
|
- name: Reject target-owned trusted helper paths
|
|
id: target_boundary
|
|
run: test -z "$(git ls-tree --name-only HEAD -- .testops-trusted)"
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
|
with:
|
|
ref: ${{ github.sha }}
|
|
path: .testops-trusted
|
|
persist-credentials: false
|
|
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
artifact-ids: ${{ inputs.build_artifact_id }}
|
|
path: chat-completion-build
|
|
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
artifact-ids: ${{ inputs.selection_artifact_id }}
|
|
path: chat-completion-testops-selection
|
|
- name: Verify source-bound build and authenticated selection before credentials
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
TESTOPS_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
|
|
EXPECTED_SELECTION_SHA256: ${{ inputs.selection_sha256 }}
|
|
ALLURE_JOB_RUN_ID: ${{ inputs.ALLURE_JOB_RUN_ID }}
|
|
run: |
|
|
set -euo pipefail
|
|
test "$(git -C .testops-trusted rev-parse HEAD)" = "$GITHUB_SHA"
|
|
node .testops-trusted/tests/runner-e2e/chat-completion-testops-cli.mjs verify-selection chat-completion-testops-selection/testops-pilot-selection.json
|
|
(cd chat-completion-build && sha256sum -c chat-completion-build.tar.gz.sha256)
|
|
tar -xzf chat-completion-build/chat-completion-build.tar.gz -C "$GITHUB_WORKSPACE"
|
|
pnpm install --frozen-lockfile --ignore-scripts
|
|
(cd .testops-trusted && node tests/runner-e2e/prepare-report-lock.mjs && pnpm install --frozen-lockfile --ignore-scripts --store-dir "$RUNNER_TEMP/chat-completion-retention-store")
|
|
- name: Qualify the pinned provider runtime and preinstalled browser
|
|
run: |
|
|
node packages/paperclip-runner/scripts/materialize-opencode-binary.mjs
|
|
node -e 'const fs=require("node:fs");fs.chmodSync(process.execPath,fs.statSync(process.execPath).mode&~0o022);if(fs.statSync(process.execPath).mode&0o022)throw Error("Writable provider Node")'
|
|
google-chrome --version
|
|
pnpm exec playwright install ffmpeg
|
|
- name: Bind exact private evidence under the authenticated TestOps authority
|
|
id: private
|
|
env:
|
|
TARGET_ROOT: ${{ github.workspace }}
|
|
SELECTED_EXECUTION_ID: ${{ fromJSON(inputs.cell).executionId }}
|
|
run: node .testops-trusted/cli/node_modules/tsx/dist/cli.mjs .testops-trusted/tests/runner-e2e/manual-private-transfer.ts prepare
|
|
- name: Obtain only this cell's exact-key storage session
|
|
id: cell_credentials
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd # v6
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
inline-session-policy: ${{ steps.private.outputs.session_policy }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
- name: Verify private custody before delivering the model key
|
|
env: &cell_session
|
|
SELECTED_EXECUTION_ID: ${{ fromJSON(inputs.cell).executionId }}
|
|
PRIVATE_EVIDENCE_ROOT: ${{ steps.private.outputs.root }}
|
|
AWS_ACCESS_KEY_ID: ${{ steps.cell_credentials.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.cell_credentials.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.cell_credentials.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
run: node .testops-trusted/cli/node_modules/tsx/dist/cli.mjs .testops-trusted/tests/runner-e2e/manual-private-transfer.ts verify
|
|
- name: Execute only the selected original case with zero retries
|
|
env:
|
|
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
|
|
ANTHROPIC_API_KEY: ${{ contains(fromJSON(inputs.cell).required_credentials, 'ANTHROPIC_API_KEY') && secrets.ANTHROPIC_API_KEY || '' }}
|
|
PAPERCLIP_CHAT_COMPLETION_TESTOPS: v1
|
|
ALLURE_JOB_RUN_ID: ${{ inputs.ALLURE_JOB_RUN_ID }}
|
|
PAPERCLIP_RUNNER_E2E_PRIVATE_PREFLIGHT: ${{ steps.private.outputs.root }}/preflight.json
|
|
PRIVATE_EVIDENCE_ROOT: ${{ steps.private.outputs.root }}
|
|
PAPERCLIP_E2E_CAMPAIGN_ID: gha-${{ github.run_id }}-1-${{ fromJSON(inputs.cell).executionId }}
|
|
PAPERCLIP_E2E_ACCOUNTING_CAMPAIGN_ID: gha-${{ github.run_id }}-1
|
|
PAPERCLIP_RUNNER_E2E_SOURCE_SHA: ${{ inputs.target_sha }}
|
|
PAPERCLIP_RUNNER_E2E_SOURCE_REF: refs/heads/codex/chat-catalog-master-integration
|
|
PAPERCLIP_PLAYWRIGHT_CHANNEL: chrome
|
|
PAPERCLIP_RUNNER_E2E_SELECTED_ID: ${{ fromJSON(inputs.cell).executionId }}
|
|
run: |
|
|
# Agents can read any process's argv in /proc; keep the case id off the command line.
|
|
selector=(--id "$PAPERCLIP_RUNNER_E2E_SELECTED_ID")
|
|
if grep -q PAPERCLIP_RUNNER_E2E_SELECTED_ID tests/runner-e2e/selectors.ts 2>/dev/null; then selector=(); fi
|
|
pnpm test:e2e:runner -- "${selector[@]}" --max-parallel 1 --max-automatic-retries 0 > "$PRIVATE_EVIDENCE_ROOT/launcher-output.log" 2>&1
|
|
- name: Retain original evidence after success or failure
|
|
if: always() && steps.cell_credentials.outcome == 'success'
|
|
env: *cell_session
|
|
run: node .testops-trusted/cli/node_modules/tsx/dist/cli.mjs .testops-trusted/tests/runner-e2e/manual-private-transfer.ts upload-cell
|
|
|
|
report:
|
|
name: Retain private originals and build the exact safe TestOps projection
|
|
needs: chat
|
|
if: always() && !cancelled() && (needs.chat.result == 'success' || needs.chat.result == 'failure')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
environment: runner-e2e-private-history
|
|
permissions:
|
|
contents: read
|
|
actions: read
|
|
id-token: write
|
|
env: *retention
|
|
outputs:
|
|
bundle_id: ${{ steps.bundle.outputs.artifact-id }}
|
|
steps:
|
|
- *checkout
|
|
- *node
|
|
- *guard
|
|
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
artifact-ids: ${{ inputs.selection_artifact_id }}
|
|
path: chat-completion-testops-selection
|
|
- *pnpm
|
|
- name: Install trusted reporting dependencies with lifecycle scripts disabled
|
|
run: |
|
|
node tests/runner-e2e/prepare-report-lock.mjs
|
|
pnpm install --frozen-lockfile --ignore-scripts --store-dir "$RUNNER_TEMP/chat-completion-report-store"
|
|
- name: Prepare unchanged exact-key report and cell policies
|
|
id: session
|
|
run: node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/manual-private-transfer.ts report-session
|
|
- name: Obtain exact first-cell reader
|
|
id: reader_0
|
|
uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd # v6
|
|
with:
|
|
role-to-assume: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_ROLE_ARN }}
|
|
aws-region: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
inline-session-policy: ${{ steps.session.outputs.read_policy_0 }}
|
|
output-env-credentials: false
|
|
output-credentials: true
|
|
disable-retry: true
|
|
action-timeout-s: 60
|
|
- name: Download the first exact original
|
|
env:
|
|
PRIVATE_REPORT_ROOT: ${{ github.workspace }}
|
|
SELECTED_CELL_INDEX: '0'
|
|
AWS_ACCESS_KEY_ID: ${{ steps.reader_0.outputs.aws-access-key-id }}
|
|
AWS_SECRET_ACCESS_KEY: ${{ steps.reader_0.outputs.aws-secret-access-key }}
|
|
AWS_SESSION_TOKEN: ${{ steps.reader_0.outputs.aws-session-token }}
|
|
AWS_DEFAULT_REGION: ${{ vars.RUNNER_E2E_PRIVATE_HISTORY_AWS_REGION }}
|
|
run: node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/manual-private-transfer.ts download-cell
|
|
- name: Verify every selected original and its completeness receipt
|
|
env:
|
|
PRIVATE_REPORT_ROOT: ${{ github.workspace }}
|
|
run: node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/manual-private-transfer.ts verify-downloads
|
|
- name: Build the existing report while preserving failed source grades
|
|
id: normalize
|
|
continue-on-error: true
|
|
env:
|
|
PAPERCLIP_RUNNER_E2E_REPORT_ROOT: ${{ github.workspace }}/selected-runner-e2e
|
|
PAPERCLIP_RUNNER_E2E_REPORT_OUT: ${{ github.workspace }}/runner-e2e-merged-report/normalized
|
|
PAPERCLIP_RUNNER_E2E_EXPECTED_IDS: ${{ fromJSON(inputs.cell).private_ids }}
|
|
PAPERCLIP_E2E_CAMPAIGN_ID: gha-${{ github.run_id }}-1
|
|
PAPERCLIP_RUNNER_E2E_SOURCE_SHA: ${{ inputs.target_sha }}
|
|
PAPERCLIP_RUNNER_E2E_SOURCE_REF: refs/heads/codex/chat-catalog-master-integration
|
|
run: |
|
|
mkdir -p runner-e2e-merged-report
|
|
pnpm test:e2e:runner:report > runner-e2e-merged-report/report.log 2>&1
|
|
- name: Export only exact reviewed synthetic fixture results
|
|
id: export
|
|
if: always() && steps.reader_0.outcome == 'success'
|
|
run: node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/chat-completion-testops-export.ts
|
|
- name: Require a complete safe projection even after a failed normalization
|
|
if: always()
|
|
env:
|
|
EXPORT_OUTCOME: ${{ steps.export.outcome }}
|
|
run: test "$EXPORT_OUTCOME" = success
|
|
- name: Freeze one safe Allure bundle before any TestOps transmission
|
|
id: prepare
|
|
if: always() && !cancelled() && steps.export.outcome == 'success'
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
TESTOPS_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
|
|
ALLURE_JOB_RUN_ID: ${{ inputs.ALLURE_JOB_RUN_ID }}
|
|
EXPECTED_SELECTION_SHA256: ${{ inputs.selection_sha256 }}
|
|
CHAT_TESTOPS_EXECUTION_ID: ${{ fromJSON(inputs.cell).executionId }}
|
|
ALLURE_RESULTS: ${{ runner.temp }}/chat-catalog-case-bundle/allure
|
|
run: |
|
|
mkdir -m 700 "$RUNNER_TEMP/chat-catalog-case-bundle"
|
|
cp -R runner-e2e-testops "$RUNNER_TEMP/chat-catalog-case-bundle/projection"
|
|
node tests/runner-e2e/chat-completion-testops-cli.mjs prepare-case-upload "$RUNNER_TEMP/chat-catalog-case-bundle/projection"
|
|
cp "$RUNNER_TEMP/chat-completion-testops-prepare-case-upload/prepared.json" "$RUNNER_TEMP/chat-catalog-case-bundle/prepared.json"
|
|
- name: Retain the immutable metadata-only case bundle
|
|
id: bundle
|
|
if: always() && !cancelled()
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: chat-catalog-case-bundle-${{ github.run_id }}-1-${{ fromJSON(inputs.cell).index }}
|
|
path: ${{ runner.temp }}/chat-catalog-case-bundle/
|
|
retention-days: 30
|
|
if-no-files-found: error
|
|
|
|
upload:
|
|
name: Publish only this original case without closing the shared job run
|
|
needs: report
|
|
if: always() && !cancelled() && (needs.report.result == 'success' || needs.report.result == 'failure')
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
environment: runner-e2e-testops-reporting
|
|
steps:
|
|
- *checkout
|
|
- *node
|
|
- *guard
|
|
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
artifact-ids: ${{ inputs.selection_artifact_id }}
|
|
path: chat-completion-testops-selection
|
|
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
|
|
with:
|
|
artifact-ids: ${{ needs.report.outputs.bundle_id }}
|
|
path: ${{ runner.temp }}/chat-catalog-case-bundle
|
|
- name: Admit the frozen singleton and submit it at most once
|
|
id: publication
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
TESTOPS_ALLOWED_ACTOR_IDS: ${{ vars.RUNNER_E2E_ALLOWED_ACTOR_IDS }}
|
|
ALLURE_TOKEN: ${{ secrets.ALLURE_TOKEN }}
|
|
ALLURE_ENDPOINT: https://paperclip.testops.cloud
|
|
ALLURE_PROJECT_ID: '1'
|
|
ALLURE_JOB_RUN_ID: ${{ inputs.ALLURE_JOB_RUN_ID }}
|
|
EXPECTED_SELECTION_SHA256: ${{ inputs.selection_sha256 }}
|
|
CHAT_TESTOPS_EXECUTION_ID: ${{ fromJSON(inputs.cell).executionId }}
|
|
CHAT_TESTOPS_PREPARED_FILE: ${{ runner.temp }}/chat-catalog-case-bundle/prepared.json
|
|
ALLURE_RESULTS: ${{ runner.temp }}/chat-catalog-case-bundle/allure
|
|
run: node tests/runner-e2e/chat-completion-testops-cli.mjs upload-case "$RUNNER_TEMP/chat-catalog-case-bundle/projection"
|
|
- name: Retain only the closed singleton publication receipt
|
|
if: always() && !cancelled()
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: chat-catalog-case-publication-${{ github.run_id }}-1-${{ fromJSON(inputs.cell).index }}
|
|
path: |
|
|
${{ runner.temp }}/chat-completion-testops-upload-case/publication.json
|
|
${{ runner.temp }}/chat-completion-testops-upload-case/failure.json
|
|
${{ runner.temp }}/chat-completion-testops-upload-case/once-upload_case.json
|
|
retention-days: 30
|
|
if-no-files-found: error
|