mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-28 20:21:50 +01:00
fix(evals): apply reviewed lock corrections to reports and qualification images (#17937)
<!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/98/changes/paperclip/98 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages agents and records their work. > - Product E2E campaigns measure real agent behavior and model spend. > - A paid hosted run completed both cases and all ten model receipts. > - Its report job could not install dependencies from the committed lock. > - The trusted checkout needs a reviewed dependency correction. > - This change applies a reviewed lock correction before the frozen report install. ## Linked Issues or Issue Description Refs: #17928 **Description** The report in [run 40576181788](https://github.com/paperclipai/paperclip/actions/runs/40576181788) failed with `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`. Both paid cases had passed. The worker and build jobs already restored the resolved source lock, but the report did not. **Steps to reproduce** Dispatch the bounded daily campaign on the default branch with a stale committed lock configuration. Workers use the resolved lock. The report uses the stale lock and fails before report generation. **Expected behavior** The report installs frozen dependencies from a verified lock for the exact trusted source. Evidence can be reported without repeating paid work. ## What Changed - Apply a reviewed lock correction from the trusted report checkout before its frozen install. Use the same correction for the qualification image build, which also requires a compatible frozen lock. - Pin the original lock, correction and output SHA-256. The trusted authorization job rejects unknown inputs before any paid work. - Apply the same correction to trusted manual history publication. Qualification image builds apply it only when their target equals the workflow revision; other targets keep their own lock. - Update existing Claude ACP and postgres patch identities and link the declared yaml dependency. Existing package tarball integrity entries stay unchanged; the declared yaml dependency adds exactly yaml@2.9.1 with pinned integrity. - Keep runtime registry resolution and worker target locks out of the credential-bearing report. - Test valid correction, idempotence and changed-lock, changed-patch and wrong-output rejection. ## Verification - Focused workflow and lock-correction tests: 36 passed. - `actionlint .github/workflows/runner-full-stack-e2e.yml` passed. - Original worker evidence: 2/2 passed, ten final model receipts, zero unresolved model requests. - The original report was regenerated from retained evidence with zero provider calls and saved privately with replay provenance. - Broader repository checks run in CI. Prior full Product checks passed for #17928. ## Risks The correction is tied to the reviewed committed lock. Dependency input changes need a reviewed correction update. No new registry version or lifecycle script can enter the privileged report through this fix. The original failed workflow stays failed. No root lockfile is committed. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection and code execution. The exact serving model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
0d22e27dc4
commit
a8bd012635
11 files changed
+784
-7
No files matched your search
@@ -187,6 +187,18 @@ jobs:
|
||||
echo '::notice title=Paid runner routing::Using a standard GitHub-hosted runner'
|
||||
fi
|
||||
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
||||
with:
|
||||
ref: ${{ github.sha }}
|
||||
persist-credentials: false
|
||||
|
||||
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
|
||||
with:
|
||||
node-version: 24
|
||||
|
||||
- name: Preflight reviewed report dependencies before paid admission
|
||||
run: node tests/runner-e2e/prepare-report-lock.mjs
|
||||
|
||||
target_lock:
|
||||
name: Resolve target pnpm lockfile
|
||||
needs: authorize
|
||||
@@ -1273,6 +1285,9 @@ jobs:
|
||||
with:
|
||||
version: 9.15.4
|
||||
|
||||
- name: Apply reviewed reporting lock corrections
|
||||
run: node tests/runner-e2e/prepare-report-lock.mjs
|
||||
|
||||
- name: Install pinned reporting dependencies
|
||||
run: pnpm install --frozen-lockfile --ignore-scripts --store-dir "${{ runner.temp }}/eval-report-store"
|
||||
|
||||
@@ -1500,6 +1515,9 @@ jobs:
|
||||
|
||||
# Use a job-local store and disable lifecycle scripts throughout install.
|
||||
# OIDC is available to every step, even before the AWS credential exchange.
|
||||
- name: Apply reviewed publication lock corrections
|
||||
run: node tests/runner-e2e/prepare-report-lock.mjs
|
||||
|
||||
- name: Install pinned publication dependencies
|
||||
run: pnpm install --frozen-lockfile --ignore-scripts --store-dir "${{ runner.temp }}/eval-publisher-store"
|
||||
|
||||
|
||||
Reference in new issue
Block a user