fix(evals): apply reviewed lock corrections to reports and qualification images (#17937)

<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/98/changes/paperclip/98
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip manages agents and records their work.
> - Product E2E campaigns measure real agent behavior and model spend.
> - A paid hosted run completed both cases and all ten model receipts.
> - Its report job could not install dependencies from the committed
lock.
> - The trusted checkout needs a reviewed dependency correction.
> - This change applies a reviewed lock correction before the frozen
report install.

## Linked Issues or Issue Description

Refs: #17928

**Description**
The report in [run
40576181788](https://github.com/paperclipai/paperclip/actions/runs/40576181788)
failed with `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`. Both paid cases had
passed. The worker and build jobs already restored the resolved source
lock, but the report did not.

**Steps to reproduce**
Dispatch the bounded daily campaign on the default branch with a stale
committed lock configuration. Workers use the resolved lock. The report
uses the stale lock and fails before report generation.

**Expected behavior**
The report installs frozen dependencies from a verified lock for the
exact trusted source. Evidence can be reported without repeating paid
work.

## What Changed

- Apply a reviewed lock correction from the trusted report checkout
before its frozen install. Use the same correction for the qualification
image build, which also requires a compatible frozen lock.
- Pin the original lock, correction and output SHA-256. The trusted
authorization job rejects unknown inputs before any paid work.
- Apply the same correction to trusted manual history publication.
Qualification image builds apply it only when their target equals the
workflow revision; other targets keep their own lock.
- Update existing Claude ACP and postgres patch identities and link the
declared yaml dependency. Existing package tarball integrity entries
stay unchanged; the declared yaml dependency adds exactly yaml@2.9.1
with pinned integrity.
- Keep runtime registry resolution and worker target locks out of the
credential-bearing report.
- Test valid correction, idempotence and changed-lock, changed-patch and
wrong-output rejection.

## Verification

- Focused workflow and lock-correction tests: 36 passed.
- `actionlint .github/workflows/runner-full-stack-e2e.yml` passed.
- Original worker evidence: 2/2 passed, ten final model receipts, zero
unresolved model requests.
- The original report was regenerated from retained evidence with zero
provider calls and saved privately with replay provenance.
- Broader repository checks run in CI. Prior full Product checks passed
for #17928.

## Risks

The correction is tied to the reviewed committed lock. Dependency input
changes need a reviewed correction update. No new registry version or
lifecycle script can enter the privileged report through this fix. The
original failed workflow stays failed. No root lockfile is committed.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository inspection and
code execution. The exact serving model ID and context window are not
exposed to this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-10-27 20:55:13 -05:00
1 parent 0d22e27dc4
commit a8bd012635
11 files changed
+784 -7

No files matched your search

@@ -187,6 +187,18 @@ jobs:
echo '::notice title=Paid runner routing::Using a standard GitHub-hosted runner'
fi
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
ref: ${{ github.sha }}
persist-credentials: false
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 24
- name: Preflight reviewed report dependencies before paid admission
run: node tests/runner-e2e/prepare-report-lock.mjs
target_lock:
name: Resolve target pnpm lockfile
needs: authorize
@@ -1273,6 +1285,9 @@ jobs:
with:
version: 9.15.4
- name: Apply reviewed reporting lock corrections
run: node tests/runner-e2e/prepare-report-lock.mjs
- name: Install pinned reporting dependencies
run: pnpm install --frozen-lockfile --ignore-scripts --store-dir "${{ runner.temp }}/eval-report-store"
@@ -1500,6 +1515,9 @@ jobs:
# Use a job-local store and disable lifecycle scripts throughout install.
# OIDC is available to every step, even before the AWS credential exchange.
- name: Apply reviewed publication lock corrections
run: node tests/runner-e2e/prepare-report-lock.mjs
- name: Install pinned publication dependencies
run: pnpm install --frozen-lockfile --ignore-scripts --store-dir "${{ runner.temp }}/eval-publisher-store"