Files
PaperClipAI/tests/runner-acceptance
DottaandPaperclip 91dfaa7d61 fix(runner): let ACP providers verify any requested model
Remove hardcoded Codex and Pi model admission and keep qualification selections in evaluation fixtures. Preserve explicit provider verification, runtime pins, and persisted recovery identities.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-10-06 12:37:18 -05:00
..

Credential-free Runner acceptance foundation

This directory defines the reviewable, deterministic foundation for Runner acceptance checks. It contains no launcher and grants no authority to start Paperclip, a provider process, a browser, a remote environment, or a billable model request.

The catalog covers:

  • every current built-in direct adapter except the explicitly deferred Pi adapter;
  • the external-plugin direct-adapter compatibility contract;
  • Paperclip Runner with Codex;
  • Paperclip Runner with the qualified OpenCode model; and
  • Paperclip Runner with the qualified ACPX Claude and Codex profiles.

The direct-adapter cells assert the legacy boundary: runnerd does not start, native records are not created, direct finalization remains authoritative, and classic task controls remain available. Native cells assert persisted provider identity, runtime authority, structured-question behavior, and recovery of an already-recorded run after the rollout flag changes.

Registered compatibility-only adapters, including the retired acpx_local entry, remain in the direct catalog so a future selection check can prove they never fall into native execution. Their presence is not a claim that the adapter can start a provider session.

Commands

Run the isolated unit suite:

pnpm test:runner-acceptance

Check the standalone TypeScript boundary:

pnpm test:runner-acceptance:typecheck

These commands are credential-free. They validate the catalog and pure support utilities; they do not claim that a provider was contacted.

Result boundary

Future fixture executors may emit paperclip.runner-acceptance.result/v1 objects and pass them to buildRunnerAcceptanceReport. Results must name a catalog cell, report every expected assertion, carry successful redaction, and contain no sensitive-looking structured values. The aggregator produces only in-memory normalized data plus optional Markdown or JUnit strings. It does not write evidence, screenshots, history, or public reports.

Deliberate exclusions

This foundation does not include:

  • Pi or any Pi transitive package;
  • Claude Managed or AWS AgentCore;
  • Daytona, remote images, live provider execution, or cost accounting;
  • provider-key loading, secret references, auth-file discovery, or tracked env templates;
  • screenshots, traces, raw logs, evidence archives, dashboards, history, or publication workflows; or
  • the eval kernel, scenario explorer, browser SDK, or package/runtime replacement work.

Add an executor only in a later, explicitly authorized change. Keep its launch authority separate from this catalog, and make any billable or secret-bearing mode opt-in and independently reviewed.