Files
PaperClipAI/tests/runner-e2e/fixtures
DottaandPaperclip 48b57e0c94 test(evals): pin held scenario repairs and await Cloud return (#19188)
## Thinking Path

> - Paperclip helps people manage AI agents and their work.
> - Protected TestOps runs admit one immutable catalog source.
> - Three held scenario observers now have reviewed, locally qualified
repairs.
> - A separate Cloud recovery test started its heading wait before the
Cloud return.
> - This PR pins the qualified successor and corrects that test
boundary.
> - It changes no product behavior, model, execution budget, or private
evidence protocol.

## Linked Issues or Issue Description

Refs: #19155, #19093, #19095, #19112.

The next candidate retains the reviewed skill, company-secret, and
team-install observer repairs. It adds a narrow timeline v9 correction:
a recorded task identifier can be the subject of its actual
reassignment. The real timeline attempt remains failed; the diagnostic
correction is not a regrade or a new native pass. Earlier failed
attempts stay retained; a source pin and component tests do not
establish a native pass.

The Cloud recovery fixture failed both direct-task variants in the first
CI attempt of #19155. Its five-second heading check started before the
external Cloud roundtrip ended. The failure image already showed the
expected task shortly afterward. One unchanged local direct-task control
passed. The single authorized CI recovery also passed; this did not
establish a reliable boundary. This PR waits for one handoff and the
exact return URL before the original heading check.

## What Changed

- Pin source `5aef4bffa2b106f2066bf2485338798f04a702bd`, tree
`464be838a448c728d44e2d5593b9d7faea266663`.
- Update the two manifest target literals and expected pin test literal.
Bind the exact v9 descriptor snapshot and its one changed timeline
oracle leaf.
- Add the reviewed two-line Cloud return readiness witness. Keep the
original five-second heading assertion and existing overall test limit.
- Use descriptor
`744a566faaa90b8048fff526e0a30b7475b86c2de88d697783f8da9e02f70e20`. Keep
the launcher, all private helpers, 244 identities, requests, setup,
human/native counts, clocks, profile, model, retry policy, permissions,
and the 196/244 gate unchanged.

## Verification

- Current infrastructure worktree: **131 pin, admission, selection,
streaming, API, and reconciliation controls passed**.
- The fixture owner ran the exact patched Cloud spec: **all five real
auth cases passed** in 37.4s. A six-second Cloud hold passed the
unchanged heading check after return. Missing handoff and missing return
controls rejected. This PR's fresh master base is verified by
current-head CI, not credited from the earlier run.
- Fresh v9 source qualification: **268 focused timeline controls**, the
242-declaration gate, fresh server declarations, and full E2E typecheck
passed under local Node **26.4.0**. The preceding held-helper
qualification (239 Node controls and one offline Chromium route control)
remains evidence for the byte-identical unaffected helper closures. The
original browser sandbox failure is retained. Actual AWS Node and model
configuration remain unchanged.
- All 506 registered SDK input Git blobs and registry
`efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8` equal
the prior qualified source. No new SDK build or runtime refresh.
- All 75 private protocol files, production trees, YAML, lockfile,
launcher, identities, counts, clocks, and sourceFiles declarations equal
the prior source. Only the timeline descriptor oracle advances v8 to v9.
- Prior head `ce18f2bd52ed00ce750302f63aecc4167a3daff3` finished CI with
43 passed, one expected skip, four originating runner interruptions, and
a failed dependent verify aggregate. Two logs show runner shutdown
signals; two annotations show lost runner communication. No assertion
failure was observed. Those outcomes are retained; no unchanged recovery
was requested.
- Required fresh current-head CI, genuine review, thread clearance, and
Root's exact-diff acceptance are pending. No merge or new paid run is
authorized by this PR's publication.
- The earlier 3 PASS / 7 FAIL uniform attempt and all later original
outcomes remain unchanged. No new native pass or whole-catalog readiness
claim is made.

## Risks

The source remains a frozen candidate with unmerged catalog repairs. A
real measured run is still required. The readiness witness requires both
a real handoff and the exact task URL; it does not extend the heading
timeout, bypass auth, skip a case, or change production code. The active
measurement on the previous pin is not altered.

## Model Used

OpenAI GPT-6.1 (`gpt-6.1-sol`), with reasoning, code execution, and
repository tools. The host did not expose a context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched for and linked related public PRs above
- [x] I have linked public PRs and described the issue
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change
- [x] Relevant local pin tests passed; source and fixture qualification
scope is stated above
- [x] I have updated the expected pin test and the observed fixture
boundary
- [x] Owning source metadata and this description document the change
- [x] I have considered and documented the risks
- [ ] All current-head infrastructure CI gates are green
- [ ] Greptile is 5/5 with no unresolved material findings
- [x] I will address reviewer comments before requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-04 23:14:14 -06:00
..