Files
PaperClipAI/.github
DottaandPaperclip a1845b3b64 fix(evals): capture TestOps registration after watch (#18375)
## Thinking Path

> - Paperclip manages AI agents and verifies their work through Product
E2E tests.
> - The fixed Chat feature workflow prepares a TestOps case and plan
before any model run.
> - Its first zero-result bootstrap completed, but the watched child did
not receive the generated job-run ID.
> - The official watch log contained a job-run ID, so the missing child
variable did not prove that registration was absent.
> - This pull request reads official context after watch and validates
the linked backend records.
> - The workflow retains closed registration metadata and stops on
unknown or foreign context.

## Linked Issues or Issue Description

Refs: #18322.

**What happened?**
The successful zero-result bootstrap retained
`official_job_run_id_missing` from its watched child. It could not prove
the backend job and workflow binding needed for the next setup step.

**Expected behavior**
After successful watch, read the generated context, validate its current
GitHub run and fixed workflow, and retain only closed IDs and parameter
metadata.

**Steps to reproduce**
Run the fixed workflow's `bootstrap` operation on master. Inspect its
setup artifact after watch completes. No evaluation or synthetic passing
result is needed.

**Paperclip version or commit**
The original bootstrap used merged #18322. This follow-up starts from
`76fde891ccc29283b3f5c8dcdce089c5b3d937d1`.

**Deployment mode**
GitHub Actions metadata-only setup in the protected reporting
environment.

## What Changed

- Run an internal context reader after successful bootstrap watch in the
same protected step.
- Parse only numeric official job-run and launch IDs in memory. Do not
log or save raw command output.
- Validate current-run, project, launch, job, workflow, and bounded
parameter metadata before writing an exclusive registration receipt.
- Retain the two known registration receipt paths with the existing
setup artifact.
- Preserve the original inside-watch receipt, case reconciliation,
provider isolation, and evaluation limits.

## Verification

- `node --test tests/runner-e2e/chat-feature-testops-api.test.mjs
tests/runner-e2e/testops-chat-reporter.test.mjs`: 48 pure/mock/FS
controls pass (36 prior and 12 new). The actual local Node version was
26.4.0. Normal Node 24 CI remains required.
- Four MJS syntax checks and `git diff --check` pass. No TypeScript
noEmit claim is made for this MJS-only helper change.
- Actionlint passes for the changed workflow.
- Normal [GitHub Actions
CI](https://github.com/paperclipai/paperclip/actions/runs/41123894542)
passes on exact head `58396de10f0c49294b0bbc0f4709c29990d57d53`: 48 jobs
succeeded and one skipped. Both required `ci / e2e` and `ci / verify`
pass.
- Greptile reviewed that exact head at 5/5 with no actionable findings
or unresolved review threads.
- No local CLI, provider, model, app, database, or build qualification
was run. No replacement bootstrap or paid evaluation was dispatched.

## Risks

The official context or backend DTO shape can differ from the supported
format. The reader then stops with a closed failure receipt. Pure tests
do not qualify the remote CLI or backend. The original zero-result
bootstrap and its missing child-context observation remain part of the
history. The case's actual outcome remains unmeasured.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning, code editing, tool
use, and pure test review. The runtime does not expose the exact
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-30 16:07:04 -05:00
..