feat(runner-e2e): publish declared screenshots (#12895)

## Thinking Path

> - Paperclip uses runner end-to-end reports to compare agent profiles
and execution environments
> - The report dashboard shows each reviewed final-state screenshot as a
thumbnail and gallery item
> - The public history publisher removed all per-attempt images before
it regenerated the dashboard
> - Therefore the public dashboard had the new layout but could not show
the screenshots from the run
> - The publisher needs a narrow rule that keeps only screenshots from
the exact live fixture issue route
> - This pull request keeps those trusted PNG files in every future
public S3 and GitHub Pages report
> - The benefit is that each future report can show its screenshot
gallery without exposing logs, traces, videos, archives, arbitrary
images, or generated report trees

## Linked Issues or Issue Description

**What happened?**

The runner E2E job captured final-state screenshots in its private
artifact. The public S3 and GitHub Pages publication step removed those
screenshots before it regenerated the dashboard. As a result, the public
report showed the new dashboard controls but no screenshot thumbnails or
gallery items.

**Expected behavior**

Each future public runner E2E report must include reviewed PNG
screenshots from the live fixture issue. Other captures and active or
unsafe evidence must stay private.

**Steps to reproduce**

1. Run the runner full-stack E2E workflow on `master` before this
change.
2. Open the private `runner-e2e-report-*` artifact and confirm that it
contains per-attempt PNG screenshots.
3. Open the public campaign URL and confirm that the dashboard has no
screenshot gallery items.

**Paperclip version or commit**

The issue was reproduced on commit `64d8929`, after the report design
change in PR #12889.

**Deployment mode**

GitHub Actions with the public S3 and GitHub Pages report publishers.

Related design work: Refs #12889.

## What Changed

- Mark screenshots from the exact server-created live fixture issue
route with `public-runner-fixture`.
- Keep marked PNG files in both the S3 history bundle and the GitHub
Pages bundle.
- Keep captures from other issue routes, sensitive routes, and external
origins private.
- Bind public files to the normalized execution ID, attempt, and safe
PNG base name.
- Validate every retained image with the existing PNG signature and 12
MiB size checks.
- Skip missing-artifact sentinel results with attempt `0` when they have
no public screenshots.
- Continue to remove unmarked images, videos, traces, archives,
generated HTML reports, and other private evidence.
- Update publisher tests, workflow checks, report copy, and the
public-evidence security documentation.

## Verification

- `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts
tests/runner-e2e/history.test.ts tests/runner-e2e/report.test.ts`
- `pnpm exec vitest run --config tests/runner-e2e/vitest.config.ts
tests/runner-e2e/workflow-security.test.ts -t "uses environment-scoped
OIDC"`
- `pnpm test:e2e:runner:typecheck`
- `PAPERCLIP_PLAYWRIGHT_CHANNEL=chrome pnpm exec playwright test
--config tests/e2e/playwright.config.ts
tests/e2e/runner-e2e-dashboard.spec.ts`
- `pnpm -r typecheck`
- `pnpm build`
- Regenerated the dashboard from retained evidence for Actions run
`33968240659` without a paid matrix rerun. The public-stage proof
contained 121 screenshot gallery items and thumbnail frames, with zero
generated HTML report files. The trusted-fixture marker and route gate
have separate focused tests.
- All pull request CI checks pass on commit `ccd2b49e1`.

## Risks

- This change intentionally makes marked fixture screenshots public at
the campaign URL. A screenshot can show data that a raw-byte secret scan
cannot detect.
- The capture helper marks a screenshot only on the exact loopback issue
route for the fixture that the harness created. A different issue,
sensitive page, or external origin stays private.
- The publisher also requires the marker, a safe normalized path, a
valid PNG signature, and the size limit.
- The change does not publish videos, logs, traces, archives, arbitrary
images, or generated browser report trees.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, model `gpt-5.6-sol`, with high reasoning, repository
tool use, shell execution, browser inspection, and GitHub CLI access.
The working context was the Codex desktop task context.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-09-05 09:49:31 -05:00
1 parent 8f0c1d4548
commit 87832c48fd
12 files changed
+382 -102

No files matched your search

+4 -4
View File
@@ -1132,7 +1132,7 @@ jobs:
fi
publish_history:
name: Publish trusted-summary S3 history and structured Pages bundle
name: Publish S3 history and Pages bundle with declared screenshots
needs: [authorize, catalog, report]
if: always() && needs.catalog.result == 'success' && needs.report.outputs.history_source_ready == 'true'
runs-on: ubuntu-latest
@@ -1183,7 +1183,7 @@ jobs:
role-to-assume: ${{ vars.RUNNER_E2E_HISTORY_AWS_ROLE_ARN }}
aws-region: ${{ vars.RUNNER_E2E_HISTORY_AWS_REGION }}
- name: Publish trusted summary image to S3 and prune the Pages bundle
- name: Publish trusted summary and declared screenshots to public bundles
env:
PAPERCLIP_RUNNER_E2E_REPORT_DIR: ${{ github.workspace }}/runner-e2e-merged-report/normalized
RUNNER_E2E_HISTORY_S3_BUCKET: ${{ vars.RUNNER_E2E_HISTORY_S3_BUCKET }}
@@ -1196,7 +1196,7 @@ jobs:
if: vars.RUNNER_FULL_STACK_E2E_PUBLISH_PAGES == 'true'
run: echo "name=github-pages-${{ github.run_id }}-${{ github.run_attempt }}" >> "$GITHUB_OUTPUT"
- name: Package structured-only dashboard for GitHub Pages
- name: Package pruned dashboard with declared screenshots for GitHub Pages
if: vars.RUNNER_FULL_STACK_E2E_PUBLISH_PAGES == 'true'
uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4
with:
@@ -1204,7 +1204,7 @@ jobs:
path: runner-e2e-merged-report/pages
pages:
name: Publish latest structured dashboard
name: Publish latest dashboard with declared screenshots
needs: [report, publish_history]
if: always() && needs.report.outputs.history_source_ready == 'true' && needs.publish_history.result == 'success' && vars.RUNNER_FULL_STACK_E2E_PUBLISH_PAGES == 'true'
runs-on: ubuntu-latest