fix(evals): run campaign cells when TestOps selection is skipped (#19053)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The Runner Full-Stack E2E workflow runs the paid Product E2E cells,
including the Product regression TestOps plan
> - #18983 added an optional `testops_selection` job and made `catalog`
depend on it
> - That job is skipped in every ordinary manual or scheduled campaign
> - GitHub's implicit `success()` also fails when an ancestor job was
skipped, so every job below `catalog` was skipped
> - Since #18983 merged, ordinary campaigns run no cells: the merge
reports every selected cell as "No result artifact was uploaded"
> - This pull request gives each job below `catalog` an explicit status
condition
> - The benefit is that ordinary, nightly and branch-targeted campaigns
run their cells again, and TestOps campaigns are unchanged

## Linked Issues or Issue Description

Refs #18983, #19000

**What happened?**
Run https://github.com/paperclipai/paperclip/actions/runs/41914430034
selected 20 Product regression cells with the `id` input. `catalog`
passed. `daytona_image`, `build_runner_artifacts`,
`build_remote_provider_pack` and every matrix job were skipped. The
merge failed with 20 "No result artifact was uploaded" rows.

**Expected behavior**
An ordinary manual campaign runs its selected cells when
`testops_selection` is skipped, as it did before #18983.

**Steps to reproduce**
Dispatch `runner-full-stack-e2e.yml` on `master` without `testops_case`
or `ALLURE_JOB_RUN_ID`. The scheduled daily campaign has the same path.

## What Changed

- `daytona_image`, `build_runner_artifacts`,
`build_remote_provider_pack`, `test`, `test_recurring` and
`test_manual_private` now start with `!cancelled()`.
- Each of those jobs now requires `needs.<job>.result == 'success'` for
each of its real dependencies. A failed or skipped build still skips the
cells, as before. The existing schedule, cadence and evidence-route
conditions are unchanged.
- A new regression test parses the workflow. It requires every job below
`testops_selection` to use a status function and to require each real
dependency result.

## Verification

- `node --test tests/runner-e2e/product-regression-testops.test.mjs`:
15/15 pass.
- Negative control: the new test fails against the current `master`
workflow.
- `daytona-image.test.ts`, `workflow-target.test.ts` and
`workflow-timeout.test.ts` pass. `workflow-security.test.ts` has 5
failures. The same 5 fail on unchanged `master` (they concern
`pr-trusted.yml` and `chat-catalog-testops-case.yml`).
- A workflow change takes effect only from `master`, because paid
campaigns run only from the default branch. After merge, dispatch an
ordinary `id=` campaign and confirm the cells run.

## Risks

- Low risk. The conditions keep the previous failure propagation: a job
runs only when each direct dependency succeeded and the run is not
cancelled.
- `catalog` success already implies `authorize` and `target_lock`
success, so those are not repeated.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), via Claude Code with tool use and
code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Opus 5.5 authored and GitHub committed 2026-11-04 17:41:18 -06:00
1 parent 23cd123ff8
commit ac57b183f0
2 files changed
+43 -5

No files matched your search

+23 -5
View File
@@ -541,6 +541,10 @@ jobs:
daytona_image:
name: Publish verified Daytona image
needs: [authorize, target_lock, catalog]
# testops_selection is skipped outside TestOps campaigns. GitHub's implicit
# success() also checks skipped ancestors, so jobs below catalog state
# their real dependency results explicitly.
if: ${{ !cancelled() && needs.catalog.result == 'success' }}
# Keep image builds on the same authorized fleet as campaign execution.
runs-on: ${{ needs.authorize.outputs.test_runner }}
timeout-minutes: 45
@@ -687,7 +691,9 @@ jobs:
build_runner_artifacts:
name: Build reusable runner campaign artifacts
needs: [authorize, target_lock, catalog]
if: github.event_name != 'schedule' || (vars.RUNNER_FULL_STACK_E2E_NIGHTLY_ENABLED == 'true' && vars.RUNNER_FULL_STACK_E2E_OPERATIONS_ENABLED == 'true')
if: >-
!cancelled() && needs.catalog.result == 'success' &&
(github.event_name != 'schedule' || (vars.RUNNER_FULL_STACK_E2E_NIGHTLY_ENABLED == 'true' && vars.RUNNER_FULL_STACK_E2E_OPERATIONS_ENABLED == 'true'))
# Compile native binaries on the same reviewed image used to execute them,
# avoiding libc/architecture drift between GitHub-hosted and AWS lanes.
runs-on: ${{ needs.authorize.outputs.test_runner }}
@@ -815,7 +821,10 @@ jobs:
name: Build reusable remote provider pack
needs:
[authorize, target_lock, catalog, daytona_image, build_runner_artifacts]
if: github.event_name != 'schedule' || (vars.RUNNER_FULL_STACK_E2E_NIGHTLY_ENABLED == 'true' && vars.RUNNER_FULL_STACK_E2E_OPERATIONS_ENABLED == 'true')
if: >-
!cancelled() && needs.catalog.result == 'success' && needs.daytona_image.result == 'success' &&
needs.build_runner_artifacts.result == 'success' &&
(github.event_name != 'schedule' || (vars.RUNNER_FULL_STACK_E2E_NIGHTLY_ENABLED == 'true' && vars.RUNNER_FULL_STACK_E2E_OPERATIONS_ENABLED == 'true'))
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
@@ -951,7 +960,10 @@ jobs:
test:
name: ${{ matrix.executionId }}
if: needs.catalog.outputs.operations_cadence == '' && needs.authorize.outputs.evidence_retention == 'actions'
if: >-
!cancelled() && needs.catalog.result == 'success' && needs.daytona_image.result == 'success' &&
needs.build_runner_artifacts.result == 'success' && needs.build_remote_provider_pack.result == 'success' &&
needs.catalog.outputs.operations_cadence == '' && needs.authorize.outputs.evidence_retention == 'actions'
needs:
[
authorize,
@@ -1443,7 +1455,10 @@ jobs:
test_recurring:
name: ${{ matrix.executionId }}
if: needs.catalog.outputs.operations_cadence != '' && needs.authorize.outputs.target_sha == github.sha && github.ref == format('refs/heads/{0}', github.event.repository.default_branch)
if: >-
!cancelled() && needs.catalog.result == 'success' && needs.daytona_image.result == 'success' &&
needs.build_runner_artifacts.result == 'success' && needs.build_remote_provider_pack.result == 'success' &&
needs.catalog.outputs.operations_cadence != '' && needs.authorize.outputs.target_sha == github.sha && github.ref == format('refs/heads/{0}', github.event.repository.default_branch)
needs: [authorize, target_lock, catalog, daytona_image, build_runner_artifacts, build_remote_provider_pack]
runs-on: ${{ needs.authorize.outputs.test_runner }}
timeout-minutes: ${{ matrix.timeoutMinutes }}
@@ -1460,7 +1475,10 @@ jobs:
test_manual_private:
name: ${{ matrix.executionId }}
if: needs.catalog.outputs.operations_cadence == '' && needs.authorize.outputs.evidence_retention == 'private' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch)
if: >-
!cancelled() && needs.catalog.result == 'success' && needs.daytona_image.result == 'success' &&
needs.build_runner_artifacts.result == 'success' && needs.build_remote_provider_pack.result == 'success' &&
needs.catalog.outputs.operations_cadence == '' && needs.authorize.outputs.evidence_retention == 'private' && github.ref == format('refs/heads/{0}', github.event.repository.default_branch)
needs: [authorize, target_lock, catalog, daytona_image, build_runner_artifacts, build_remote_provider_pack]
runs-on: ${{ needs.authorize.outputs.test_runner }}
timeout-minutes: ${{ matrix.timeoutMinutes }}