Files
PaperClipAI/tests
DottaandPaperclip d06c5e0eff fix(evals): preserve opaque TestOps parameters (#18394)
## Thinking Path

> - Paperclip verifies real agent work through Product E2E tests.
> - The fixed Chat feature workflow uses TestOps to select and report
one case.
> - TestOps exposes job parameters as ID/value pairs.
> - The schema does not establish that those IDs identify global
environment variables.
> - The reader used that unsupported relationship to infer parameter
names.
> - This pull request keeps IDs opaque, preserves the strict allowed
values, and records closed read diagnostics.

## Linked Issues or Issue Description

Refs: #18322, #18375, #18385.

**What happened?**
A zero-model bootstrap returned HTTP 404 during context readback. The
exact failed GET was not retained and remains unknown. Schema review
also found no established relationship between job-parameter IDs and
global environment-variable IDs, although the reader queried that
namespace to label parameters.

**Expected behavior**
Read only the existing job-run, launch, and job records. Preserve the
job's exact permitted ID/value vector without inventing parameter names.
Record which fixed GET class failed, its positive resource ID, state,
and available HTTP status without raw responses or URLs.

**Steps to reproduce**
Use the actual API helpers with a job-parameter ID that has no global
environment-variable definition. The new pure controls reject an
unexpected definition lookup and verify the opaque vector and
diagnostics without a backend or model call.

**Paperclip version or commit**
This change starts from `8595efc20bda3ce20a9fbbf34a5d156a6fcf6d57` after
#18385.

**Deployment mode**
GitHub Actions metadata setup in the protected TestOps reporting
environment.

## What Changed

- Remove the unsupported global environment-variable lookup from
job-parameter handling.
- Preserve an exact positive ID and `refs/heads/master` value. Use the
fixed workflow's existing `evaluate` default.
- Treat an operation parameter as such only when its ID matches the
protected configured ID and its value is allowed. Keep rejecting empty,
duplicate, conflicting, or unknown vectors before writes.
- Project only an opaque ID with a fixed-default-ref kind and value, or
an unresolved kind with the value withheld.
- Add closed diagnostics for the existing job-run, launch, and job GETs.
A successful call keeps HTTP status null because the API abstraction
does not expose the status.
- Sync one reporter-test assertion and document opaque IDs. Reporter
runtime and workflow source are unchanged.

## Verification

- Four new actual-helper controls fail on the parent.
- `node --test tests/runner-e2e/chat-feature-testops-api.test.mjs
tests/runner-e2e/testops-chat-reporter.test.mjs`: 58 affected
pure/mock/FS controls pass on Node 26.4.0.
- Three MJS syntax checks and `git diff --check` pass. TypeScript noEmit
was not run for this MJS/documentation change.
- Two intermediate exact-projection assertion failures were retained
before the assertions were synced to the new diagnostic contract.
- [Normal Node 24
CI](https://github.com/paperclipai/paperclip/actions/runs/41137009579)
passes on exact head `3c256fb8f7559093c5fb063170e26c01b6e8a0a4`: 48 jobs
passed and one skipped. Both required `ci / e2e` and `ci / verify` pass.
- Greptile reviewed this exact head at 5/5 with no actionable findings
or unresolved threads. At this verification cut, the optional
Contributor trust check was still in progress; it is not an effective
required status context.
- No actual CLI, API, provider, model, app, database, build, install, or
additional bootstrap was run to validate this change locally.

## Risks

The original HTTP 404 route remains unknown. Removing an unsupported
schema assumption does not prove that it caused that failure. The live
job mapping and dispatch remain unqualified. The strict source, actor,
selection, run-number, nonce, once-only send, private custody, case,
model, grader, and clock checks are unchanged. Unknown metadata still
stops. This PR adds no endpoint, workflow mode, permission, credential
path, or passing result.

## Model Used

OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning, code editing, tool
use, and pure test review. The runtime does not expose the exact
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-30 21:41:05 -05:00
..