PaperClipAI

Jannik/PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-30 21:49:18 +01:00

Commit Graph

Author	SHA1	Message	Date
DottaandPaperclip	287b04d72f	fix(evals): keep report dependency proof with lockfile refresh (#18063 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/192/changes/paperclip/192 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages AI agents and runs paid Product E2E evaluations. > - Paid work needs a report that can install its trusted dependencies. > - The report uses a reviewed offline lock correction. > - New UI dependencies can make that correction stale while its lock hash still matches. > - The lockfile bot must refresh the correction with its lock changes. > - This PR also checks manifest consistency offline before model admission. ## Linked Issues or Issue Description What happened? Current master declares new CodeMirror and YAML UI dependencies. Its committed lock omits those UI importer entries. The existing report correction accepts that lock, but a frozen install then fails with `ERR_PNPM_OUTDATED_LOCKFILE`. This can waste completed paid work before the report fails. Refs #18059. That bot PR supplies the matching lock. This PR keeps the root lock bot-owned and supplies its exact offline report correction. The correction changes 21 importer lines and leaves all package records unchanged. Expected behavior The existing bot includes matching report proof in its dependency PR. Paid authorization rejects stale manifests before model work. Report installation still accepts only reviewed hashes and applies no registry resolution. Steps to reproduce 1. Run a frozen pnpm 9.15.4 install on master at `7f10cc058568e3c1ac133cf24f3b4000cbdfca0e`. 2. Observe the missing UI importer entries. 3. Apply the reviewed correction in this PR. The frozen install passes. ## What Changed - Add an explicit refresh mode that derives the offline patch and hashes from a known committed lock and the bot's resolved lock. - Keep the existing unknown-lock, patch hash and output hash guards. - Include the proof files in the existing lockfile bot PR. New bot PRs use complete descriptions; existing descriptions stay intact. - Validate the frozen lock against current manifests offline before paid admission, using the existing pinned pnpm version. - Refresh the current correction and document this process. The root lockfile is not committed here. ## Verification - 24 focused report-lock and workflow-security tests pass. They cover offline replay, tampered input/patch/output rejection, real Git refresh, unknown-base refusal and unchanged proof. - A full frozen install passes after the reviewed correction, with lifecycle scripts disabled. - A fresh source archive validates offline with an empty package store. - A deliberate manifest change fails that same offline check with `ERR_PNPM_OUTDATED_LOCKFILE`. - Both workflow files parse as YAML. All 95 shell steps pass Bash syntax checks. - `git diff --check` passes. These checks make no model requests. - Full repository checks and fresh reviews must pass on the final PR head before merge. ## Risks - Dependency changes must still pass review. Refresh refuses an unknown committed baseline. - The bot now commits two additional proof files. Its write permissions and target branch stay unchanged. Existing PR descriptions are preserved. - The paid authorization job adds an offline manifest check. It runs no package lifecycle scripts. - No budget, provider credential, environment permission, schedule or paid enable gate changes. ## Model Used - OpenAI Codex, GPT-6, with reasoning and code execution tools. The exact serving variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-28 15:30:58 -05:00
DottaandPaperclip	6987c2f20b	test(upgrades): qualify controller replacement with immutable artifacts and phase evidence (#17455 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/105/changes/paperclip/105 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip needs evidence that an admitted execution survives a main-server replacement. > - Runner mechanics and server adoption belong to #17097 and #17439. > - A healthy HTTP endpoint does not prove recovery of the original execution. > - Qualification must identify the serving artifact, natural ownership, authenticated attachment and resumed processing separately. > - It must also retain failed runs and measure normal-operation costs. > - This PR extends the existing qualification machinery and acceptance record. ## Linked Issues or Issue Description What existing behavior does this improve? Operators can assess upgrade continuity from phase evidence tied to exact artifacts and profiles. Current behavior A same-artifact restart or synthetic executor pass cannot establish different-artifact upgrade and rollback support. Idle SQL gates also do not measure active-run capacity. Proposed behavior Use immutable A/B artifacts and strict phase receipts through the existing supervisor. Preserve failed evidence and add measured before/after workload costs. Builds on merged #17097 and #17439. The base is now `master`. #17456 follows for review convenience. This PR lands qualification machinery and the evidence record. The remaining hosted gates stay open for deployment qualification; merging this machinery does not claim those experiments passed. ## What Changed - Extend the main-server supervisor and installed-artifact verification for actual serving revisions, forward migrations and compatible rollback. - Distinguish HTTP readiness, natural ownership, authenticated attachment, resumed processing and backlog settlement. - Retain the original reconnect grace and execution identity checks. Never substitute a new task or edited lease for continuity. - Reuse real-harness and Daytona qualification paths, with exact profile and artifact provenance. - Bind current Copilot attestation inputs in the Daytona image fingerprint. - Keep one acceptance record, including both failed file-only stress phases and the final passing bounded-pool comparison. ## Verification After retargeting to master, all 158 focused qualification cases across 13 files pass, with E2E typecheck. The first 12-file batch has four report-lock test failures and 133 passes. Master #17811 had committed the exact already-approved output lock, while that fixture assumed HEAD must be the uncorrected input. The fixture now checks both pinned forms, executes the committed-lock path, and reconstructs the pinned input to retain all rejection tests. The production helper, policy hashes, patch and dependency resolution are unchanged. The corrected batch explicitly includes both process-preservation suites; the prior 146-case receipt has its own source and selection. Current candidate `8f1d6ec02f` integrates master `09ad9adce2`, including the merged server slice. Its diff contains only qualification code and documentation. The [qualification record](https://github.com/paperclipai/paperclip/blob/8f1d6ec02f5bf6c90e7b27da8156953cabd9a180/doc/plans/2026-10-22-hot-upgrade-qualification.md) is the acceptance ledger and retains source-attributed failures and hosted limits. On the eighth integration, all 146 cases across 12 focused qualification files pass, including report-lock preparation, workflow security and workflow targeting. E2E typecheck passes. The earlier 118-case result remains attributed to its prior source. The first sandbox invocation has 117 passes and one failure because process inspection was denied during inert fixture cleanup. The permitted repeat passes all 118 and a targeted process check finds no remaining fixture daemon. The earlier 96-case receipt remains source-attributed in the ledger. The latest two review regressions first fail: an intermediate replacement can cite another runner/session despite a valid final step, and malformed receipt JSON aborts evidence packaging. Each replacement now binds run/company/issue/runner/session, and corrupt receipts retain explicit invalid metadata plus sanitized diagnostics while other logs and receipts still package. These checks prove qualification machinery with injected supervisor operations, configuration subprocesses and real filesystem packaging. They do not qualify a hosted harness. Three request-reader cases, 14 artifact/configuration/cache/history cases, and three lost-partial-receipt cases fail before their respective corrections. The final collector preserves sanitized startup, readiness and recovery phase receipts through packaging and temporary-root deletion. Earlier embedded-Postgres initialization, report-catalog drift, CI scheduling/cleanup failures and fixture collisions remain recorded with their corrections or limits. The previous Greptile service error is retained as historical review evidence, not a current review pass. The record includes prior source `9bb9d3417e`'s actual Linux contention/fairness successes and the ordinary-warm regression proof from the server slice. The historical 300-session component campaign passes four after phases and retains earlier burst failures. Those component costs do not establish hosted capacity. The hosted API-only pair has zero agents/viewers, fails both absolute latency gates and the candidate memory gate, and does not complete the active-agent/viewer matrix. The strict ten-agent same-artifact restart remains attributed to its original artifact. It does not qualify this updated stack. Actual A→B/B→A, the normal drain-free Cloud rollout, and other hosted harnesses remain open. No new provider turn, deployment or hosted qualification was run during merge preparation. Prior source `57d3767eac` jobs 122590643944, 122590643983 and 122590644161 ended after runner shutdown/cancellation without final Vitest results. The shutdown cause is unproven. No retry was issued before this source was superseded by the required server recovery fix. These attempts remain failures/incomplete evidence. Fresh exact-head CI, legacy status, completed Greptile 5/5, resolved feedback and normal merge eligibility remain required. ## Risks Different-artifact A→B/B→A, the normal drain-free Cloud rollout path, other hosted harnesses and the complete concurrency/viewer matrix remain open. Frozen branch builds are development-build evidence. This PR owns qualification and evidence; production defects must be fixed in their earlier owning slice. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The runtime does not expose a more specific model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 21:58:09 -05:00
DottaandPaperclip	a8bd012635	fix(evals): apply reviewed lock corrections to reports and qualification images (#17937 ) <!-- entire-trail-link-start --> https://entire.com/gh/paperclipai/-/trails/98/changes/paperclip/98 <!-- entire-trail-link-end --> ## Thinking Path > - Paperclip manages agents and records their work. > - Product E2E campaigns measure real agent behavior and model spend. > - A paid hosted run completed both cases and all ten model receipts. > - Its report job could not install dependencies from the committed lock. > - The trusted checkout needs a reviewed dependency correction. > - This change applies a reviewed lock correction before the frozen report install. ## Linked Issues or Issue Description Refs: #17928 Description The report in [run 40576181788](https://github.com/paperclipai/paperclip/actions/runs/40576181788) failed with `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`. Both paid cases had passed. The worker and build jobs already restored the resolved source lock, but the report did not. Steps to reproduce Dispatch the bounded daily campaign on the default branch with a stale committed lock configuration. Workers use the resolved lock. The report uses the stale lock and fails before report generation. Expected behavior The report installs frozen dependencies from a verified lock for the exact trusted source. Evidence can be reported without repeating paid work. ## What Changed - Apply a reviewed lock correction from the trusted report checkout before its frozen install. Use the same correction for the qualification image build, which also requires a compatible frozen lock. - Pin the original lock, correction and output SHA-256. The trusted authorization job rejects unknown inputs before any paid work. - Apply the same correction to trusted manual history publication. Qualification image builds apply it only when their target equals the workflow revision; other targets keep their own lock. - Update existing Claude ACP and postgres patch identities and link the declared yaml dependency. Existing package tarball integrity entries stay unchanged; the declared yaml dependency adds exactly yaml@2.9.1 with pinned integrity. - Keep runtime registry resolution and worker target locks out of the credential-bearing report. - Test valid correction, idempotence and changed-lock, changed-patch and wrong-output rejection. ## Verification - Focused workflow and lock-correction tests: 36 passed. - `actionlint .github/workflows/runner-full-stack-e2e.yml` passed. - Original worker evidence: 2/2 passed, ten final model receipts, zero unresolved model requests. - The original report was regenerated from retained evidence with zero provider calls and saved privately with replay provenance. - Broader repository checks run in CI. Prior full Product checks passed for #17928. ## Risks The correction is tied to the reviewed committed lock. Dependency input changes need a reviewed correction update. No new registry version or lifecycle script can enter the privileged report through this fix. The original failed workflow stays failed. No root lockfile is committed. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository inspection and code execution. The exact serving model ID and context window are not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-27 20:55:13 -05:00

Author

SHA1

Message

Date

DottaandPaperclip

287b04d72f

fix(evals): keep report dependency proof with lockfile refresh (#18063 )

<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/192/changes/paperclip/192
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip manages AI agents and runs paid Product E2E evaluations.
> - Paid work needs a report that can install its trusted dependencies.
> - The report uses a reviewed offline lock correction.
> - New UI dependencies can make that correction stale while its lock
hash still matches.
> - The lockfile bot must refresh the correction with its lock changes.
> - This PR also checks manifest consistency offline before model
admission.

## Linked Issues or Issue Description

**What happened?**

Current master declares new CodeMirror and YAML UI dependencies. Its
committed lock omits those UI importer entries. The existing report
correction accepts that lock, but a frozen install then fails with
`ERR_PNPM_OUTDATED_LOCKFILE`. This can waste completed paid work before
the report fails.

Refs #18059. That bot PR supplies the matching lock. This PR keeps the
root lock bot-owned and supplies its exact offline report correction.
The correction changes 21 importer lines and leaves all package records
unchanged.

**Expected behavior**

The existing bot includes matching report proof in its dependency PR.
Paid authorization rejects stale manifests before model work. Report
installation still accepts only reviewed hashes and applies no registry
resolution.

**Steps to reproduce**

1. Run a frozen pnpm 9.15.4 install on master at
`7f10cc058568e3c1ac133cf24f3b4000cbdfca0e`.
2. Observe the missing UI importer entries.
3. Apply the reviewed correction in this PR. The frozen install passes.

## What Changed

- Add an explicit refresh mode that derives the offline patch and hashes
from a known committed lock and the bot's resolved lock.
- Keep the existing unknown-lock, patch hash and output hash guards.
- Include the proof files in the existing lockfile bot PR. New bot PRs
use complete descriptions; existing descriptions stay intact.
- Validate the frozen lock against current manifests offline before paid
admission, using the existing pinned pnpm version.
- Refresh the current correction and document this process. The root
lockfile is not committed here.

## Verification

- 24 focused report-lock and workflow-security tests pass. They cover
offline replay, tampered input/patch/output rejection, real Git refresh,
unknown-base refusal and unchanged proof.
- A full frozen install passes after the reviewed correction, with
lifecycle scripts disabled.
- A fresh source archive validates offline with an empty package store.
- A deliberate manifest change fails that same offline check with
`ERR_PNPM_OUTDATED_LOCKFILE`.
- Both workflow files parse as YAML. All 95 shell steps pass Bash syntax
checks.
- `git diff --check` passes. These checks make no model requests.
- Full repository checks and fresh reviews must pass on the final PR
head before merge.

## Risks

- Dependency changes must still pass review. Refresh refuses an unknown
committed baseline.
- The bot now commits two additional proof files. Its write permissions
and target branch stay unchanged. Existing PR descriptions are
preserved.
- The paid authorization job adds an offline manifest check. It runs no
package lifecycle scripts.
- No budget, provider credential, environment permission, schedule or
paid enable gate changes.

## Model Used

- OpenAI Codex, GPT-6, with reasoning and code execution tools. The
exact serving variant and context window are not exposed in this
session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-10-28 15:30:58 -05:00

DottaandPaperclip

6987c2f20b

test(upgrades): qualify controller replacement with immutable artifacts and phase evidence (#17455 )

https://entire.com/gh/paperclipai/-/trails/105/changes/paperclip/105

## Thinking Path

> - Paperclip needs evidence that an admitted execution survives a
main-server replacement.
> - Runner mechanics and server adoption belong to #17097 and #17439.
> - A healthy HTTP endpoint does not prove recovery of the original
execution.
> - Qualification must identify the serving artifact, natural ownership,
authenticated attachment and resumed processing separately.
> - It must also retain failed runs and measure normal-operation costs.
> - This PR extends the existing qualification machinery and acceptance
record.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Operators can assess upgrade continuity from phase evidence tied to
exact artifacts and profiles.

**Current behavior**

A same-artifact restart or synthetic executor pass cannot establish
different-artifact upgrade and rollback support. Idle SQL gates also do
not measure active-run capacity.

**Proposed behavior**

Use immutable A/B artifacts and strict phase receipts through the
existing supervisor. Preserve failed evidence and add measured
before/after workload costs.

Builds on merged #17097 and #17439. The base is now `master`. #17456
follows for review convenience. This PR lands qualification machinery
and the evidence record. The remaining hosted gates stay open for
deployment qualification; merging this machinery does not claim those
experiments passed.

## What Changed

- Extend the main-server supervisor and installed-artifact verification
for actual serving revisions, forward migrations and compatible
rollback.
- Distinguish HTTP readiness, natural ownership, authenticated
attachment, resumed processing and backlog settlement.
- Retain the original reconnect grace and execution identity checks.
Never substitute a new task or edited lease for continuity.
- Reuse real-harness and Daytona qualification paths, with exact profile
and artifact provenance.
- Bind current Copilot attestation inputs in the Daytona image
fingerprint.
- Keep one acceptance record, including both failed file-only stress
phases and the final passing bounded-pool comparison.

## Verification

After retargeting to master, all 158 focused qualification cases across
13 files pass, with E2E typecheck. The first 12-file batch has four
report-lock test failures and 133 passes. Master #17811 had committed
the exact already-approved output lock, while that fixture assumed HEAD
must be the uncorrected input. The fixture now checks both pinned forms,
executes the committed-lock path, and reconstructs the pinned input to
retain all rejection tests. The production helper, policy hashes, patch
and dependency resolution are unchanged. The corrected batch explicitly
includes both process-preservation suites; the prior 146-case receipt
has its own source and selection.

Current candidate `8f1d6ec02f` integrates master `09ad9adce2`, including
the merged server slice. Its diff contains only qualification code and
documentation. The [qualification
record](https://github.com/paperclipai/paperclip/blob/8f1d6ec02f5bf6c90e7b27da8156953cabd9a180/doc/plans/2026-10-22-hot-upgrade-qualification.md)
is the acceptance ledger and retains source-attributed failures and
hosted limits.

On the eighth integration, all 146 cases across 12 focused qualification
files pass, including report-lock preparation, workflow security and
workflow targeting. E2E typecheck passes. The earlier 118-case result
remains attributed to its prior source. The first sandbox invocation has
117 passes and one failure because process inspection was denied during
inert fixture cleanup. The permitted repeat passes all 118 and a
targeted process check finds no remaining fixture daemon. The earlier
96-case receipt remains source-attributed in the ledger. The latest two
review regressions first fail: an intermediate replacement can cite
another runner/session despite a valid final step, and malformed receipt
JSON aborts evidence packaging. Each replacement now binds
run/company/issue/runner/session, and corrupt receipts retain explicit
invalid metadata plus sanitized diagnostics while other logs and
receipts still package. These checks prove qualification machinery with
injected supervisor operations, configuration subprocesses and real
filesystem packaging. They do not qualify a hosted harness.

Three request-reader cases, 14 artifact/configuration/cache/history
cases, and three lost-partial-receipt cases fail before their respective
corrections. The final collector preserves sanitized startup, readiness
and recovery phase receipts through packaging and temporary-root
deletion. Earlier embedded-Postgres initialization, report-catalog
drift, CI scheduling/cleanup failures and fixture collisions remain
recorded with their corrections or limits. The previous Greptile service
error is retained as historical review evidence, not a current review
pass.

The record includes prior source `9bb9d3417e`'s actual Linux
contention/fairness successes and the ordinary-warm regression proof
from the server slice. The historical 300-session component campaign
passes four after phases and retains earlier burst failures. Those
component costs do not establish hosted capacity. The hosted API-only
pair has zero agents/viewers, fails both absolute latency gates and the
candidate memory gate, and does not complete the active-agent/viewer
matrix.

The strict ten-agent same-artifact restart remains attributed to its
original artifact. It does not qualify this updated stack. Actual
A→B/B→A, the normal drain-free Cloud rollout, and other hosted harnesses
remain open. No new provider turn, deployment or hosted qualification
was run during merge preparation.

Prior source `57d3767eac` jobs 122590643944, 122590643983 and
122590644161 ended after runner shutdown/cancellation without final
Vitest results. The shutdown cause is unproven. No retry was issued
before this source was superseded by the required server recovery fix.
These attempts remain failures/incomplete evidence. Fresh exact-head CI,
legacy status, completed Greptile 5/5, resolved feedback and normal
merge eligibility remain required.

## Risks

Different-artifact A→B/B→A, the normal drain-free Cloud rollout path,
other hosted harnesses and the complete concurrency/viewer matrix remain
open. Frozen branch builds are development-build evidence. This PR owns
qualification and evidence; production defects must be fixed in their
earlier owning slice.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools and code
execution. The runtime does not expose a more specific model ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-10-27 21:58:09 -05:00

DottaandPaperclip

a8bd012635

fix(evals): apply reviewed lock corrections to reports and qualification images (#17937 )

<!-- entire-trail-link-start -->
https://entire.com/gh/paperclipai/-/trails/98/changes/paperclip/98
<!-- entire-trail-link-end -->

## Thinking Path

> - Paperclip manages agents and records their work.
> - Product E2E campaigns measure real agent behavior and model spend.
> - A paid hosted run completed both cases and all ten model receipts.
> - Its report job could not install dependencies from the committed
lock.
> - The trusted checkout needs a reviewed dependency correction.
> - This change applies a reviewed lock correction before the frozen
report install.

## Linked Issues or Issue Description

Refs: #17928

**Description**
The report in [run
40576181788](https://github.com/paperclipai/paperclip/actions/runs/40576181788)
failed with `ERR_PNPM_LOCKFILE_CONFIG_MISMATCH`. Both paid cases had
passed. The worker and build jobs already restored the resolved source
lock, but the report did not.

**Steps to reproduce**
Dispatch the bounded daily campaign on the default branch with a stale
committed lock configuration. Workers use the resolved lock. The report
uses the stale lock and fails before report generation.

**Expected behavior**
The report installs frozen dependencies from a verified lock for the
exact trusted source. Evidence can be reported without repeating paid
work.

## What Changed

- Apply a reviewed lock correction from the trusted report checkout
before its frozen install. Use the same correction for the qualification
image build, which also requires a compatible frozen lock.
- Pin the original lock, correction and output SHA-256. The trusted
authorization job rejects unknown inputs before any paid work.
- Apply the same correction to trusted manual history publication.
Qualification image builds apply it only when their target equals the
workflow revision; other targets keep their own lock.
- Update existing Claude ACP and postgres patch identities and link the
declared yaml dependency. Existing package tarball integrity entries
stay unchanged; the declared yaml dependency adds exactly yaml@2.9.1
with pinned integrity.
- Keep runtime registry resolution and worker target locks out of the
credential-bearing report.
- Test valid correction, idempotence and changed-lock, changed-patch and
wrong-output rejection.

## Verification

- Focused workflow and lock-correction tests: 36 passed.
- `actionlint .github/workflows/runner-full-stack-e2e.yml` passed.
- Original worker evidence: 2/2 passed, ten final model receipts, zero
unresolved model requests.
- The original report was regenerated from retained evidence with zero
provider calls and saved privately with replay provenance.
- Broader repository checks run in CI. Prior full Product checks passed
for #17928.

## Risks

The correction is tied to the reviewed committed lock. Dependency input
changes need a reviewed correction update. No new registry version or
lifecycle script can enter the privileged report through this fix. The
original failed workflow stays failed. No root lockfile is committed.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, repository inspection and
code execution. The exact serving model ID and context window are not
exposed to this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-10-27 20:55:13 -05:00