mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-31 14:16:39 +01:00
DottaandPaperclip
99d27fd5e7
## Thinking Path > - Paperclip is a control plane for AI agents that do work. > - Its evaluation reports separate recorded results from source coverage. > - A case without a main baseline measurement can still have a candidate run. > - A normal campaign card treats an expected execution without a result as missing evidence. > - Catalog planning data must not create those cards or change a recorded grade. > - This change adds a typed metadata view to the existing Evals dashboard. > - Readers can compare main outcomes, candidate execution, allowances and source review separately. ## Linked Issues or Issue Description **What existing behavior does this improve?** The existing Evals dashboard can show a campaign's retained results. It cannot show the full chat catalog as metadata without synthetic result entries. **Current behavior** An unused initial, an unknown allowance and a completed candidate run can all lack a main baseline result. Showing them as missing campaign artifacts hides these differences. **Proposed behavior** Add one validated metadata view in the existing renderer. Show one row per frozen scenario, separate execution and main outcome columns, filters, and the public request and expected outcome from static YAML. Keep normal campaign cards unchanged. **Reason and benefit** Readers can see which cases ran, which have no main measurement, and which still need preparation. Source review and candidate outcomes cannot add baseline pass credit. **Breaking changes** None to existing campaign inputs or outputs. The new metadata mode uses its own strict schema and requires a new private output directory. No matching open or closed PR or issue was found in the duplicate search. ## What Changed - Add a strict display-only catalog schema with closed enums and fixed catalog scope. - Validate the 244 row identities against static YAML without importing executable case callbacks. - Show main results, recorded execution, allowances and source review in separate columns. - Add search and filters, passing examples, qualification notes and expandable public test descriptions. - Add one dated immutable snapshot link in the existing shared report navigation. This does not change history data or update an already-published page. - Add a private metadata preview command to the existing dashboard owner. Keep existing campaign generation on its current path. - Keep allowance certainty independent of measurements and allow an optional later candidate outcome alongside a main outcome, without changing main counters. - Add focused pure, JSDOM and file-loader checks. Document the metadata command. ## Verification - The owning `catalog-overview.test.ts` Vitest suite passes all fourteen tests under the focused source configuration. The valid-main/unknown-allowance and main-fail/later-candidate-pass controls fail against the old renderer and pass with this repair. The never-run/unknown-allowance control fails againste8686bfand passes with the restored consistency guard. - Scoped TypeScript verification of the changed source and its imports has no diagnostics. - The tests cover closed schema fields, duplicate and invented IDs, candidate/main separation, filters, HTML escaping, private output mode, existing output refusal, symlink and oversized input refusal, static descriptions, and the normal campaign branch. Ceiling values below, equal to and above the target are covered. Unsupported navigation is hidden only in metadata mode. - The source was independently reviewed. Actual generated DOM validation also checked 244 rows and all public descriptions. - No visual browser verification or live product journey is claimed. No credentials, private projection, generated report or evidence files are included in this PR. - Repo-wide builds were not run locally for this report-only slice. Normal remote CI passed on repair head `f813c18`, with 55 successful checks, two skipped checks and one neutral review check. Required `ci / verify` and `ci / e2e` both passed. Current-head Greptile is 5/5 and no review threads remain open. Thee8686bfnormal CI had an initial failed attempt and a successful failed-only replay. Both are retained; that replay does not substitute for current-head verification. - The firstf813c18CI attempt passed 373 tests in serialized server shard 4/9 but failed one existing scheduled-retry test: its intermediate cancellation count was 29 rather than 20. The actual CI-tested test, route, worker, scheduler and DB-fixture source matches prior passing CI. The one failed-only replay passed all 374 tests in that shard on the unchanged head. The first failure remains recorded; no assertion or timeout changed. - Rendering the authenticated existing projection through both old and new generators produces byte-identical HTML and both assets. The already-published dated snapshot is unchanged. ## Risks The view depends on an authenticated metadata projection. Shape validation does not qualify source or execution. The view preserves unknown states and keeps candidate outcomes outside main qualification. The preview command does not publish reports or alter history. Existing campaign result behavior remains separate. Shared navigation gains one dated link. Existing published pages need a separate conditional activation; this PR does not upload or rewrite them. ## Model Used OpenAI GPT-6 through Codex, with source editing, tool use and pure/JSDOM checks. The exact serving model ID and context-window size are not exposed to this worker. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>