Remove hardcoded Codex and Pi model admission and keep qualification selections in evaluation fixtures. Preserve explicit provider verification, runtime pins, and persisted recovery identities.
Co-Authored-By: Paperclip <noreply@paperclip.ing>
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Paperclip supports direct adapters and an experimental native Runner
path.
> - These paths need one stable compatibility matrix.
> - The matrix must not launch providers or load credentials during
normal tests.
> - Result handling must reject incomplete output and sensitive values.
> - This pull request adds a credential-free acceptance foundation.
> - The benefit is a reviewable contract for later end-to-end executors.
## Linked Issues or Issue Description
**What existing behavior does this improve?**
This improves verification for direct adapters and Paperclip Runner
providers.
**Subsystem affected**
Cross-cutting test infrastructure for adapters, the server runtime, and
the task thread.
**Current behavior**
The repository has subsystem tests. It does not have one declarative
matrix for direct and native compatibility.
**Proposed behavior**
Add a pure acceptance catalog, result validator, redaction helpers, and
failure classification. Keep all execution authority outside this
change.
**Reason and benefit**
The matrix makes legacy isolation and native recovery requirements
explicit. The helpers let later executors report safe and complete
results.
**Breaking changes**
None. This change does not alter production runtime selection or start
any provider.
## What Changed
- Add a catalog for built-in direct adapters and qualified native
provider profiles.
- Add compatibility cases for runtime selection, task threads,
questions, and flag-change recovery.
- Add pure redaction and transient-failure classification helpers.
- Add fail-closed Markdown and JUnit report aggregation.
- Add isolated test and type-check commands.
- Document the credential-free boundary and deferred live execution
work.
## Verification
GitHub Actions must run:
- `pnpm test:runner-acceptance`
- `pnpm test:runner-acceptance:typecheck`
- The repository test, type-check, build, policy, and security gates.
No local test command was run. The repository owner requested
GitHub-only verification.
## Risks
Low production risk. The change adds test-only files and root scripts.
The catalog can drift when a built-in adapter changes. Its validation
fails closed on that drift.
## Model Used
OpenAI Codex with the GPT-5 agent model. The work used high reasoning,
repository inspection, tool use, and parallel code review.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
