Files
PaperClipAI/tests
DottaandClaude Opus 5.5 2b194d77e2 fix(evals): name each team member's observed readiness and grade configure-team structurally
teams.configure-team failed setup because the QA member's empty OpenRouter key
surfaced as a generic hello-probe error (OpenRouter 401 "Missing Authentication
header") instead of the missing-authentication check, so the missing-versus-ready
split never formed. The product fix (cherry-picked) classifies a provider 401 as
opencode_hello_probe_auth_required.

- The setup blocker now names every member's observed probe status and non-info checks.
- Replace reply keyword grading: drop the member-name substring and the
  premature-ready regex (identification is the rendered link to the missing
  member's own page, already required), and grade the followup by a non-empty
  reply from its own succeeded run instead of matching ready/configured words.
- Calibrate both ways; bump oracleVersion to v3; add parse.ts to sourceFiles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 11:51:44 -06:00
..