2 Commits

Include renames

Author	SHA1	Message	Date
DottaandPaperclip	cd0c86e5e1	fix(evals): support private evidence for the frozen chat catalog (#18170) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Product E2E tests need to retain evidence from both successful and failed work. > - Manual private runs use the existing protected AWS evidence route. > - Its fixed eight-case registry blocks other scenarios in the frozen chat catalog. > - Failed Playwright cells also produce static trace files that need exact owner and content checks. > - This pull request admits the fixed catalog, keeps eight cells per run, and retains the existing report identities. > - Exact source and run approval still controls each private operation. ## Linked Issues or Issue Description Refs: #18134, #17888. **What happened?** The manual private route accepts only its first eight execution IDs. It rejects other IDs from the frozen 244-scenario chat catalog before a run can start. The dotted `area.case` report identity also fails the retained report parser. A failed Playwright cell can produce canonical trace assets that the private evidence owner does not yet recognize. **Expected behavior** The trusted route must recognize exactly the frozen 244 OpenCode/local execution IDs. Each original run must still select at most eight cells and have exact protected approval. The evidence owners must retain complete scanned originals from the actual pinned report producer. Unknown carriers and credential contamination must still fail closed. **Steps to reproduce** Use a known frozen chat ID outside the first eight with the current manual classifier. The classifier rejects it. Parse a dotted chat execution ID with the current report parser. The parser rejects it. Generate a synthetic failed Node-only Playwright test and pass its actual trace/report tree through the private evidence owners. This change includes the narrow parser repair from the existing open #17888. It does not merge or close that PR. The related source work was checked for overlap before this branch was created. ## What Changed - Replace the small private registry with the exact frozen 244 execution IDs. Preserve the two existing undotted standalone task IDs. - Keep the private run limit at eight through classification, approval rows, cell binding and transfer sessions. Preserve nonprivate selector behavior. - Bind private cell report assets to the actual installed pinned producer, selected cell, canonical path and exact bytes before the existing scans and archive checks. Pass that verified owner into both real launcher packaging calls. - Exclude installed build staging from the private evidence walk. Retain the real standard-file cap and reject unknown nested carriers. - Add the existing dotted chat report grammar. Preserve other report families and legacy task IDs. - Use the existing public announcement dismissal control in one external-link/upload E2E test so a real announcement cannot cover its composer button. Preserve the open menu and existing timeout. - Document source compatibility and operator gates. No workflow, product, dependency, lockfile, case callback, query, clock or budget changes. ## Verification - 132 distinct focused controls are closed. The current affected owner suite passes 39 controls under a nondefault temporary directory. The other 93 unchanged controls retain their previous passes. The initial registry, map-order and test-tooling failures remain recorded. - Two actual launcher call-expression controls fail on the prior source because canonical trace files are dropped. Both pass after the private-owner wiring. These controls do not claim full launcher lifecycle or actual E2E qualification. - Temporary paths use the real operating-system temporary directory, including the handoff check. - The controls check all 244 IDs through real private routing, cell binding and original-run approval. They reject unknown IDs, aliases, wrong profiles, traversal, private duplicates and excess private cells. - The controls preserve actual nonprivate 33-ID and duplicate selector behavior. - A real synthetic Node-only failed Playwright test produces the report and automatic trace. The control uses actual package, seal, archive, receipt and restored validation paths with mock AWS transport only. - Owning TypeScript `--noEmit` check passes. No local product server, browser, model, database, SDK build or evaluation was run. - Prior-head CI run 40906120388 failed one E2E test because the real announcement footer intercepted its Add to composer click. The other 28 tests in that shard passed. The narrow fixture uses the existing UI dismissal pattern. The added handler passes an isolated noEmit check against the real Playwright Page type. An additional strict check of the whole browser test fails on five unchanged shared-fixture type errors; checking that unchanged shared file alone gives identical diagnostics. The initial shell exit was misreported and corrected in the validation record. Browser verification is deferred to the next normal CI run; no local browser or server was started. - Repository-wide local build and test commands were not run because this is a scoped source change. Normal required PR checks will provide the repository CI result. ## Risks A registry entry is not proof that a target or feature is ready. A future target must use the exact matching private helper and producer contract, retain any stronger target scanners, and produce fresh source fingerprints. Existing running jobs keep their pinned workflow and target sources. All original attempt limits, declared inputs, private storage checks, fatal scanners and exact administrator-controlled run approval remain in force. The new cell report contract accepts only canonical assets from the actual pinned Playwright installation. Other assets and layouts still fail closed. No private evidence or generated report is included in this PR. ## Model Used OpenAI GPT-6 through Codex, with tool use and code execution. The runtime does not expose a finer model deployment ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 11:42:35 -05:00
DottaandPaperclip	018c4c9c9b	Retain approved manual E2E evidence in private history (#18134) ... ## Thinking Path > - Paperclip lets people manage AI agents and their work. > - The manual AWS E2E workflow records browser and model evidence. > - Public repository Actions artifacts are not a restricted evidence store. > - Some chat evidence needs the existing private-history store. > - A target flag must not grant archive access or enable a public fallback. > - This pull request binds restricted runs to protected exact-source approval and retains their standard evidence with trusted transfer code. > - Operators can retain complete evidence without changing scanner verdicts or scenario outcomes. ## Linked Issues or Issue Description **What existing behavior does this improve?** Manual AWS E2E evidence retention for exact reviewed chat-feature selections. **Subsystem affected** The runner E2E manual workflow, standard evidence handoff and private-history transfer. **Current behavior** Manual cell and merged evidence use Actions artifacts. In a public repository, repository readers can download those files. Scheduled private history does not authorize a manual target to use its role. **Proposed behavior** Classify sensitive selections in the trusted workflow. Require one protected original-run approval. Verify existing private storage before target setup or model secrets. Retain scanned standard originals and final packaged results through a workflow-pinned runtime. Deny public artifact and history fallback on failure. **Reason and benefit** Preserve inspectable standard evidence in restricted storage without treating a private bucket as permission to retain credentials. **Additional context** Related source work is separate: #18067 changes paid runner routing, #18100 changes scoped input delivery, and #17888 changes report catalog parsing. This pull request does not include or revert those changes. ## What Changed - Add a trusted manual classification and exact run, attempt, workflow, target, selection and launcher approval contract. - Add a protected manual private job. Prepare a second workflow-pinned checkout and isolated publication dependencies before archive credentials. - Seal standard originals and final result bytes with exact inventories. Reject credential contamination, unscanned carriers and changed files. - Use existing immutable private-history archives and transfer receipts for cells and merged reports. Keep the existing bounds. - Use per-cell exact-key read sessions and a separate report writer. Check every inline policy against the 2,048-character STS limit. - Scan and seal fresh generated report logs and files with the actual report-step credential roots before transfer. - Require a positive nonprivate route for Actions uploads and public history. Private failures have no public fallback. Preserve normal catalog semantics for nonprivate selections with more than 32 IDs or duplicate IDs. - Authenticate generated-report fonts and every reserved trace-viewer static asset against the exact trusted producer path and bytes, including HTML and JavaScript. Keep standard cell evidence and unknown carriers closed. - Document the first eight supported IDs, deployment gates and the operator trust boundary. ## Verification - 57 new pure, actual-helper and actual-workflow-shell controls pass. - 50 affected existing workflow, routing, target and private-history controls pass. The final committed workflow security controls also pass. - Targeted helper/test and actual launcher noEmit checks pass. - `actionlint -shellcheck= -pyflakes= .github/workflows/runner-full-stack-e2e.yml` passes. - Parent `2f46336d7` required CI is green. The server 11/12 shard first failed on database cleanup/test timing and passed one bounded recheck with unchanged source. The original failure remains recorded. - The preserved docs-only parent `d721c170f` had 59 terminal, nonblocking checks and fresh Greptile 5/5. Three later review comments exposed a genuine static-report asset failure and nonprivate selector regression; their parent counterexamples remain recorded. - The reviewed parent passes 76 focused controls across nine files. The final dependency preflight successor passes all six affected controls, including two new actual `prepare` subprocess controls, and targeted helper/test noEmit. Unchanged suites were not repeated. A synthetic Node-only Playwright HTML/blob/JUnit report and the real dashboard renderer/brand stager pass the actual sealer, strict archive and immutable receipt path; only AWS transport is synthetic. No browser or product setup runs in that test. - Default cell fonts/SVGs, unknown report files, forged paths, altered static bytes, recomputed inventory tampering and registered credential matches remain rejected. Eligibility and remaining-byte checks run before body reads in both sealer and transfer validator; both independent parent read-order failures are preserved. - The pre-credential target comparison also binds the new generated-report asset helper byte for byte. A genuine tampered-dependency parent failure is preserved. - Six additional actual sealer/validator controls expose the standard-extension trace-owner bypass on the parent. All 34 affected controls pass on its correction, including the real producer/archive/receipt test; targeted noEmit passes. Ordinary dynamic report HTML outside the reserved trace directory is unchanged. - Final head `43683c4e0` passes both required gates, `ci / verify` and `ci / e2e`, in normal CI run `40849916302` attempt 1. All 59 rollup checks/statuses are terminal and nonblocking (55 success, two neutral, two skipped). - Fresh exact-head Greptile is 5/5; automated review is terminal and all technical threads are resolved. Human CODEOWNER approval remains a separate gate. Earlier source, genuine counterexamples and CI history remain preserved. - Full local build and live E2E were not run. This task prohibits local SDK/native/server builds and paid or AWS operations. Normal pull request CI must pass before merge readiness. - No existing artifact bodies, archives, media or provider reasoning were inspected. Synthetic owned fixtures prove the privacy boundaries. ## Risks - Deployment still needs the existing private bucket, role and protected environments. Both environments need the exact original-run approval variable. - Targets need the compatible handoff protocol and a separately reviewed exact launcher digest. Existing target branches do not become ready automatically. - Complete duplicated originals and packaged evidence can exceed existing archive limits. Such a transfer fails; files are not silently removed. - Required evidence omissions remain fatal to PASS. Safe failed evidence is retained with an explicit missing list and incomplete required-evidence verdict. Credential contamination still blocks all original uploads. Restricted storage does not waive PNG, HTML, blob or scanner checks. - This is operator-approved source execution. Target processes can inherit the job's OIDC request environment and may re-assume a role. Inline session attenuation is not isolation from malicious approved source. - Operator authorization must bind the actual exact source and original run. A proven administrator-controlled record may supply it without an environment reviewer UI. No new approval UI or environment change is required by this PR. - Required repository CODEOWNER review is still pending. Commented reviews and review requests are not approvals. This PR stays open and unmerged. - This PR does not deploy infrastructure, change IAM, dispatch workflows, publish report payloads, change grades or transfer prior scenario passes. ## Model Used OpenAI `gpt-6.1-sol` in Codex. Used code execution, source analysis and tool calls. The host does not expose the context window size. Independent peer review used the same model family in a separate agent. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-29 08:51:40 -05:00