Files
PaperClipAI/tests
DottaandPaperclip 7c04adb2d6 test(evals): pin company environment and API search repairs (#19392)
## Thinking Path

> - Paperclip manages AI agents and their governed actions.
> - The TestOps controller runs an immutable product-eval source.
> - The last company-secret attempt rejected a safe metadata question
before private entry.
> - A governed approval also had a redundant server notification and
wake path.
> - Natural-language API search could hide the resource needed for a
timeline answer.
> - This pull request pins the reviewed source and its exact descriptor
snapshot.
> - New native evidence must still prove the affected journeys.

## Linked Issues or Issue Description

Refs #19381, #19093, #19387, #19388. This is the next controller pin for
those reviewed changes. The core pull requests remain unmerged.

## What Changed

- Pin source `2c99c248e839f41b520fc2d7e3702169ac6cb0a2` and its exact
descriptor snapshot.
- Advance company oracle v12 to v13. Admit bounded environment-name
questions without a required authored pattern or exact prompt template.
Preserve independent human-answer validation.
- Include the reviewed interaction-accept notification and API-search
ranking overlays. Record their route and catalog dependencies in the
company and timeline provenance.
- Retain the existing cap of two, zero retries, DeepSeek1017 model, and
all 244 current YAML definitions. Company keeps 480000ms, native2..3,
base human2 plus one independently proved approval. Timeline
keeps720000ms, native3/human1.

## Verification

- Source:396 controls in9 suites,2046 API catalog controls,145 route
controls, full runner TypeScript check, and unchanged SDK506 input guard
passed. Fresh-master API PR has2059 controls; its other baseline tests
are absent from this catalog source.
- Controller:133 contract, API, adapter, reconcile, streaming and
manifest controls passed. Existing report-lock preparation returned
`already-reviewed`.
- All current YAML definitions and launcher bytes remain unchanged from
the prior source. No model or native eval ran for this pin.
- The original company5059 and timeline5060 remain failed. Company
private entry, approval and final binding remain unproved on this
successor.
- Exact-head normal checks are terminal:56 success,3 intentional skips,
and one neutral Paperclip review. Greptile and Paperclip both report5/5
with no findings; review threads are empty. The next TestOps/AWS run
follows the protected controller merge.

## Risks

The new source is not a native feature pass. Its effect on model
completion is unproved. The original480000ms company budget may still be
too short for three native turns. No budget change or count waiver is
included. The full catalog remains outside master and its existing
admission gate remains unchanged. The controller reuses byte-identical
SDK506 evidence; no compiler or registry renewal is claimed. Actual
post-merge helper imports still need qualification against the merged
controller lock. The CI virtual merge includes a newer upstream lock
than the local frozen dependency proof.

## Model Used

OpenAI `gpt-6.1-sol` through Codex, with repository tools and code
execution. The host did not report its context window.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either linked existing issues or described the issue in-PR
- [x] I have not referenced internal or instance-local Paperclip issues
or links
- [x] My branch name describes the change and contains no internal
ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is5/5 with no open findings
- [x] I will address reviewer comments before requesting merge

Co-authored-by: Paperclip <paperclip@paperclip.ing>
2026-11-05 20:05:16 -06:00
..