Files
PaperClipAI/tests
DottaandClaude Opus 5.5 26e77362c5 fix(evals): grade steering.steer-active honestly on runtimes that cannot steer
The OpenCode harness reports steering: false, so the queue for its running
task says steeringDisposition "unsupported" and the page offers Interrupt
instead of Steer. The fixture waited only for "available" and timed out.

Accept either definitive capability. When steering is available, the
same-turn ACK branch is graded as before. When it is unsupported, grade the
honest fallback the scenario names: no usable Steer control, a visible
continuation control, a refused steer attempt (409 steering_unsupported) that
records no steering and keeps the exact message queued, and one normal
continuation that consumes it and renders the blue page. Drop the reply
keyword checks and the /blue/ match on the delivery document; the rendered
page proves the colour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:23:18 -06:00
..