PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-11-08 22:13:59 +01:00

Author	SHA1	Message	Date
DottaandPaperclip	6a66c09f7f	test(evals): pin secure confirmation and team warning repairs (#19333 ) ## Thinking Path > - Paperclip manages AI agents and human approval for work. > - The chat catalog measures real user journeys through TestOps. > - The last personal-secret journey reached a valid secure-save confirmation that its observer rejected. > - The team journey reported a missing named skill with a typographic apostrophe that its observer rejected. > - A company-secret request returned body validation before the required agent authorization denial. > - This PR pins one reviewed source that addresses these boundaries without changing the requests, models, or time limits. > - New native measurements must establish the result. Existing failures remain failures. ## Linked Issues or Issue Description Refs: #19084 Refs: #19112 Refs: #19325 Refs: #19261 This is the single successor pin for the reviewed personal confirmation, team warning, and company authorization changes. It does not merge those core or full-catalog branches into master. ## What Changed - Pin immutable source `ab0f5246f765058af702b93d9b6637f2ef5b83cd`, tree `6cf28980b0ab6d1dd8e54efda3996708c53aeb98`. - Bind the exact descriptor snapshot `0b57bba133a5c15860ca77b804f7282b5314d7feb8efbfd6f11fca42cdbdee49` and matching target test. - Advance personal-secret v9 to v10 and team-install v4 to v5. Record the personal confirmation helper and its existing contract/flow inputs in source provenance. - Disclose the unmerged company-secret authorization fix from #19325. It denies scoped agent creation before body validation and gives safe human entry guidance. Board creation remains unchanged. - Retain the original query history, including the single previously approved multipart-form clarification. Preserve all 244 identities, the 196/244 gate, model, clocks, native and human bounds, SDK registry, launcher, and private evidence protocol. ## Verification - Pin-local checks: all 38 manifest, adapter, and reconcile controls passed with `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs tests/runner-e2e/chat-full-catalog-testops-reconcile.test.mjs`. - Exact source qualification: the combined domain run passed 328 controls and skipped one optional canary. Its two local listener failures were sandbox EPERM setup failures. The same two controls then passed with the authorized local listener, in a 73-control run. The eight new route authorization controls also passed. These local runs do not prove native completion. - Full E2E TypeScript passed on exact source ab0 after reuse of the healthy, same-source workspace graph. Earlier missing-alias graph failures are retained. A full local server build or repository-wide suite was not run; #19325 has its own core checks. - The source SDK guard verified all 506 registered input Git blobs unchanged, registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. No SDK build or qualification renewal was needed. - Actual anchored controller dependency check: `prepare-report-lock.mjs` returned `already-reviewed`, then pnpm 9.15.4 completed the frozen, ignore-scripts install in 17 seconds. This exercised the current upstream Claude ACP patch and reporting output `dd5934060dd06b8e731e3beeed49daf918d1a0b2a6c32d3da7815f2e7f0b3691`. It made no tracked lock or policy change. Earlier source-lock and controller-lock evidence remain separate. - Existing failed personal/team attempts remain failed. The company initial noncompletion remains uncorrected as an outcome claim; the authorization fix addresses one proven error boundary only. Team repeat/link checks were not reached in the failed measurement and remain required. No uniform-ten PASS or new native feature PASS is claimed. - Next validation is a controlled TestOps measurement after normal pin merge and fresh actual-controller proof. Any later whole-ten regression is separate. All attempts remain recorded; there is no retry-until-green policy. - Exact head `58d4a9b2d3c90be725e8877873010295122fb9c0`: required CI aggregates `ci / e2e` and `ci / verify` passed in run 42099871883. Windows passed in run 42099871391. Genuine Greptile review is 5/5 on that exact head, with zero unresolved threads. The complete rollup has 56 successes, three configured skips, and one neutral check; no pending or adverse check remains. No native measurement ran as part of these checks. ## Risks - Personal v10 accepts two logical human inputs: the original chat request and a separately verified same-human secure-save confirmation. The raw chat count remains one. This is a disclosed fixture variant, not proof from a new native journey. - The new confirmation fields do not change the trusted publisher schema or its closed enums. Existing private helper and launcher bytes remain unchanged. - Team v5 only normalizes the apostrophe in its warning predicate. Saved state, named-skill truth, repeat idempotency, links, and contradictory-success guards still apply. - The authorization fix is present only in the pinned source. #19325 remains unmerged; its effect on the unfinished native turn is unproven. ## Model Used OpenAI `gpt-6.1-sol` through Codex, with reasoning, repository tools, and local code execution. The context window size is not reported by this runtime. No model inference was used for local tests. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-05 14:22:05 -06:00
DottaandPaperclip	cb8df178c8	test(evals): pin the clarified five-case chat repairs (#19261 ) ## Thinking Path > - Paperclip helps people manage AI agents and their work. > - The TestOps catalog executes one reviewed immutable source with declared model, permissions, counts, clocks, and private evidence gates. > - Recent retained attempts exposed specific question, timeline, secret-handoff, and team-reporting contract gaps. > - Five focused repairs now have one qualified source; their component checks do not establish a native pass. > - This PR binds the trusted catalog lane to that source and records its exact changed definitions. > - One visible-form question is clarified and disclosed; all other catalog requests stay fixed. > - The controller, launcher, private evidence helpers, and execution authority stay unchanged. ## Linked Issues or Issue Description Refs: #19086, #19084, #19093, #19112, #19148, #19188. What happened? The retained native attempts include a timeline narrative rejected despite the saved handoff, secure-entry admission failures before downstream verification, and a truthful contextual team caveat rejected after a correct repeat. Multipart attempts4623 and4717 did not complete under the original broad request. All original statuses and evidence remain unchanged. Expected behavior Measure the five reviewed repairs on one immutable source with the original model, limits, and privacy gates. A component replay must not rewrite an old result or count as a product pass. The catalog must display and execute the same versioned request. Steps to reproduce After normal protected merge and fresh controller admission, use the existing TestOps plan102 and job100 to select the intended five cases on master. Root owns the actual Submit and original-evidence review. No evaluation is dispatched by this PR. ## What Changed - Pin only the catalog lane to `6773b1c128a2e7efa7571ff04b38238f00d06117`, tree `5669b199edc1aed3ab812403407b5930f5127da4`. - Copy its exact descriptor snapshot, SHA256 `aaf15ca6755fcfe550daa98bca1ed263e3d625159b53cd1e1f52fefbf9474778`. Advance only multipart53 v4, timeline466 v10, company-secret578 v8, personal-secret580 v9, and team-install610 v4. Add four declared source paths across the two secret cases. - Record exactly one query clarification through the existing per-case changed-field join. Old: “Help me choose the details for the launch page before you build it.” New: “Walk me through the launch-page question form so I can choose the audience and tone and enter my own headline before you build anything.” Preserve the old query and failed results in history. Retain inventory `0cf530…` and disclose clarified inventory `814bb363…`. All other243 catalog requests are unchanged. - Disclose that the company-secret second human message now uses actual saved non-secret metadata instead of repeating the initial request. Its continuation still requires the actual agent identity. No fabricated agent comment or private value establishes the handoff. - Team recovery uses independent saved canonical skill/version/member proof. No recovery-success phrase is required. A contradictory current denial still fails; a still-missing skill still requires truthful disclosure. - Keep all244 identities, the196/244 admission gate, model/profile, human/native counts, clocks, retries0, cap8, launcher, private protocol, and SDK506/efa qualification unchanged. No product code, full catalog, workflow, permissions, secret, runtime, or helper change lands on master. ## Verification - Source owner qualification:680 affected controls on the accepted parent and213 focused successor controls plus two explicit browser canaries, fresh same-source server declarations, and full E2E TypeScript pass on localNode26.4.0. Registered SDK506 Git inputs, production source, private protocol, launcher, and original limits match the accepted source. The source-only contract/flow additions are not copied into this controller. - Pin-local command: `node --test tests/runner-e2e/chat-full-catalog-testops-manifest.test.mjs tests/runner-e2e/chat-full-catalog-testops-reconcile.test.mjs tests/runner-e2e/chat-full-catalog-testops-adapter.test.mjs` —38 pass. - Exact source descriptor and launcher bytes match the bound hashes. The manifest keeps the original metadata source and source history, joins the new descriptor, and permits only the one explicitly recorded53 query clarification. `git diff --check` passes. - Warm controller admission:10 committed helper byte checks, five required source matches, and four trusted module imports passed with the existing exact-main60f dependency graph. No SDK or native rebuild ran. Actual post-merge master and imports still require a fresh check before Submit. - The53 clarification changes the evaluated question. It does not fix or regrade the original300-second failures. All form validation, back retention, custom answer, persisted-answer, continuation, and visible-summary checks remain required. One targeted post-clarification measurement is planned; any later whole-ten regression is separate, with all attempts retained and no retry-until-green. The existing case53 description must be updated and read back after merge so TestOps displays the clarified request. - All five successor scenarios remain unrun. No new native pass, uniform-ten pass, reliability claim, or master catalog qualification is made. Focused source PRs remain separate and unmerged; their independent CI does not replace this PR's required checks. - Full workspace typecheck/build were not run locally for this metadata-only PR. Current-head required CI, Windows, genuine review, and zero unresolved material threads are required before normal protected merge. ## Risks A wrong source or descriptor would misstate the measured definition. Strict target, snapshot, field, and identity joins cover that risk. The clarified question cannot establish success for the old broad request. Safe guidance admission still needs real human entry, private UI persistence, and worker evidence in the next attempt. Component controls do not prove those outcomes or completion within the original limits. ## Model Used OpenAI `gpt-6.1-sol`, with reasoning, repository tools, and local code execution. The host did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-05 10:00:50 -06:00
DottaandPaperclip	48b57e0c94	test(evals): pin held scenario repairs and await Cloud return (#19188 ) ## Thinking Path > - Paperclip helps people manage AI agents and their work. > - Protected TestOps runs admit one immutable catalog source. > - Three held scenario observers now have reviewed, locally qualified repairs. > - A separate Cloud recovery test started its heading wait before the Cloud return. > - This PR pins the qualified successor and corrects that test boundary. > - It changes no product behavior, model, execution budget, or private evidence protocol. ## Linked Issues or Issue Description Refs: #19155, #19093, #19095, #19112. The next candidate retains the reviewed skill, company-secret, and team-install observer repairs. It adds a narrow timeline v9 correction: a recorded task identifier can be the subject of its actual reassignment. The real timeline attempt remains failed; the diagnostic correction is not a regrade or a new native pass. Earlier failed attempts stay retained; a source pin and component tests do not establish a native pass. The Cloud recovery fixture failed both direct-task variants in the first CI attempt of #19155. Its five-second heading check started before the external Cloud roundtrip ended. The failure image already showed the expected task shortly afterward. One unchanged local direct-task control passed. The single authorized CI recovery also passed; this did not establish a reliable boundary. This PR waits for one handoff and the exact return URL before the original heading check. ## What Changed - Pin source `5aef4bffa2b106f2066bf2485338798f04a702bd`, tree `464be838a448c728d44e2d5593b9d7faea266663`. - Update the two manifest target literals and expected pin test literal. Bind the exact v9 descriptor snapshot and its one changed timeline oracle leaf. - Add the reviewed two-line Cloud return readiness witness. Keep the original five-second heading assertion and existing overall test limit. - Use descriptor `744a566faaa90b8048fff526e0a30b7475b86c2de88d697783f8da9e02f70e20`. Keep the launcher, all private helpers, 244 identities, requests, setup, human/native counts, clocks, profile, model, retry policy, permissions, and the 196/244 gate unchanged. ## Verification - Current infrastructure worktree: 131 pin, admission, selection, streaming, API, and reconciliation controls passed. - The fixture owner ran the exact patched Cloud spec: all five real auth cases passed in 37.4s. A six-second Cloud hold passed the unchanged heading check after return. Missing handoff and missing return controls rejected. This PR's fresh master base is verified by current-head CI, not credited from the earlier run. - Fresh v9 source qualification: 268 focused timeline controls, the 242-declaration gate, fresh server declarations, and full E2E typecheck passed under local Node 26.4.0. The preceding held-helper qualification (239 Node controls and one offline Chromium route control) remains evidence for the byte-identical unaffected helper closures. The original browser sandbox failure is retained. Actual AWS Node and model configuration remain unchanged. - All 506 registered SDK input Git blobs and registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8` equal the prior qualified source. No new SDK build or runtime refresh. - All 75 private protocol files, production trees, YAML, lockfile, launcher, identities, counts, clocks, and sourceFiles declarations equal the prior source. Only the timeline descriptor oracle advances v8 to v9. - Prior head `ce18f2bd52ed00ce750302f63aecc4167a3daff3` finished CI with 43 passed, one expected skip, four originating runner interruptions, and a failed dependent verify aggregate. Two logs show runner shutdown signals; two annotations show lost runner communication. No assertion failure was observed. Those outcomes are retained; no unchanged recovery was requested. - Required fresh current-head CI, genuine review, thread clearance, and Root's exact-diff acceptance are pending. No merge or new paid run is authorized by this PR's publication. - The earlier 3 PASS / 7 FAIL uniform attempt and all later original outcomes remain unchanged. No new native pass or whole-catalog readiness claim is made. ## Risks The source remains a frozen candidate with unmerged catalog repairs. A real measured run is still required. The readiness witness requires both a real handoff and the exact task URL; it does not extend the heading timeout, bypass auth, skip a case, or change production code. The active measurement on the previous pin is not altered. ## Model Used OpenAI GPT-6.1 (`gpt-6.1-sol`), with reasoning, code execution, and repository tools. The host did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched for and linked related public PRs above - [x] I have linked public PRs and described the issue - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change - [x] Relevant local pin tests passed; source and fixture qualification scope is stated above - [x] I have updated the expected pin test and the observed fixture boundary - [x] Owning source metadata and this description document the change - [x] I have considered and documented the risks - [ ] All current-head infrastructure CI gates are green - [ ] Greptile is 5/5 with no unresolved material findings - [x] I will address reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-04 23:14:14 -06:00
DottaandPaperclip	088e959d7b	test(evals): pin qualified chat scenario follow-up repairs (#19155 ) ## Thinking Path > - Paperclip helps people manage AI agents and their work. > - The protected TestOps workflow admits one fixed catalog source. > - The retained uniform attempt exposed scenario observation and grading gaps. > - Focused source PRs now contain the reviewed repairs and local controls. > - A new immutable candidate combines those changes without a runtime refresh. > - This PR binds that qualified candidate for a future normal TestOps measurement. ## Linked Issues or Issue Description Refs: #19135, #19148, #19086, #19084, #19093, #19095, #19100, #19112. The previous uniform attempt finished 3 PASS / 7 FAIL. Its original results and the separate failed control result4587 remain retained. The source repairs correct visible multipart summary evidence, canonical skill navigation, bound approval history, timeline attribution, credential guidance, company configuration questions, and team-install observation. Local qualification is component evidence. It does not establish a new passing uniform-ten attempt. ## What Changed - Pin immutable source `2ec214914cdaa11148ab2527d2203303f1485e2f`, tree `f1dc0c47668e1ed1db1b4d1ead4d3d262edefc0a`. - Update eleven existing oracle-version declarations in the manifest and its frozen descriptor snapshot. - Keep all 244 identities, 242 canonical declarations, requests, setup, human/native counts, clocks, model, profile, retries, permissions, and the 196/244 readiness gate. - Preserve historical manifest provenance and prior failure records. - Keep the launcher and all 75 private protocol files byte-identical to the prior source. This PR changes no controller helper or workflow. ## Verification - Existing focused pin, selection, admission, streaming, API, and reconciliation tests: 131 passed. - Source owner qualification on the exact published candidate: 675 Node controls and 20 hire DOM controls passed; all six affected domains are included. The 242-declaration gate passed. - Fresh server declarations and full E2E typecheck passed on the exact candidate under local Node 26.4.0. The temporary Node24/output tree disappeared; the source owner restored exact frozen dependencies and rebuilt only missing identical-source outputs under Root authorization. Actual AWS Node and the model profile remain unchanged. - Existing SDK guard: 506 registered inputs unchanged, registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`; no SDK renewal or runtime build. - Production server, packages, CLI, UI, YAML, and lock bytes equal the prior source. Only eleven oracle-version descriptor leaves change. Query, clock, count, and policy comparisons passed. - Launcher SHA256 stays `2b9a6953ee51f0ea0dbbd475b0491d9a6c39a1d6835eeaa912e0dbb7cf987724`. Descriptor SHA256 is `41038d984f045f593923da53ac036a9a6adefed5edcaedc609e55991e2fa03b7`. - The next controlled acceptance selection is only [53, 466, 564, 580]. Their current source reviews and manual contract reviews are clear. Cases 74, 578, and 610 have separate open observation findings and remain held. This pin does not qualify the whole catalog or a uniform-ten result. - Root approved the final three-SHA pin delta. Exact current-head pin review is 5/5 with no unresolved threads, and Windows passed. Normal CI is terminal and blocked: 42 jobs passed, one expected selector skipped, and six failed. Both direct-task Cloud recovery variants failed the existing initial heading check. The native Runner, package Runner, and server1 test leaves lost communication with their self-hosted runners, as the GitHub annotations confirm. Their host cause is unproven; Spot is not established. The dependent E2E and verify aggregates also failed. Original CI results are retained. One same-head failed-jobs-only recovery was accepted after a fresh terminal/head guard. The independent unchanged direct-task control passed locally in 4.9s; the CI trace shows its five-second heading wait began before the Cloud roundtrip, with the expected page visible in the failure screenshot shortly afterward. This supports a test-boundary hypothesis, not a fixed race or reliability claim. Attempt 2 is pending; no second unchanged retry is authorized. Merge remains blocked until all required gates pass. Individual source PRs retain their separate CI and base-policy limitations; this PR does not claim that all source PRs are merge-ready. - No model call, TestOps submission, archive read, permission change, core merge, or catalog merge occurred during this pin work. ## Risks The candidate contains unmerged catalog changes. The protected source pin keeps that boundary explicit. Component controls cannot replace a real product attempt. Prior failures and unknown historical grades remain unchanged. A future measurement must use the admitted source and original bounds. ## Model Used OpenAI GPT-6.1 (`gpt-6.1-sol`), with reasoning, code execution, and repository tools. The host did not expose a context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched for and linked related public PRs above - [x] I have linked related public PRs and described the issue - [x] I have not referenced internal or instance-local Paperclip links - [x] My branch name describes the change - [x] I have run the relevant pin tests locally and they pass - [x] I have updated the source pin regression and descriptor snapshot - [x] I have updated the owning source provenance conditions - [x] I have considered and documented the risks - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-04 22:32:45 -06:00
Dotta	9c6993f0aa	test(evals): pin corrected chat catalog and restore retained attachments (#19135 ) ## Thinking Path > - Paperclip uses real Product E2E journeys to verify agent work. > - The trusted TestOps controller must bind the reviewed source and retain inspectable evidence. > - The last uniform attempt exposed bounded observation and evidence-retention defects. Its original outcomes remain unchanged. > - The reviewed successor corrects those boundaries with the original model, requests, counts, and clocks. > - Its private framework report stores canonical attachments once. The controller must validate and restore those references before normal reporting. > - This PR ports that reader support and pins the qualified source in one infrastructure change. It does not merge the full catalog or its core repair PRs. ## Linked Issues or Issue Description Refs #19127, #19083, #19086, #19084, #19093, #19095, #19100, #19112. What happened? The last uniform attempt retained real failed outcomes. It also lost the multipart-question original when evidence packaging exceeded its unchanged physical byte allowance. Review found independent boundaries in skill navigation, announcement observation, timeline attribution, hire history, credential questions, and team visibility. Expected behavior The next run must execute the same ten requests against one reviewed source. The controller must retain complete original evidence, validate private attachment references, and report the actual outcome. Historical failures and missing-original UNKNOWN grades must remain unchanged. Steps to reproduce Use the focused source controls linked above. The private framework control uses a real failed Playwright test and a 140 MiB attachment. Legacy duplicate carriers hit the old cap. Canonical original evidence passes both seals and strict archive restoration. The controller restores derived files and the existing TestOps exporter preserves the failed result. Paperclip version/commit Qualified candidate: `155dd1bc1c4b85ae107f6a0cd429720caa281367`. The controller port is based on current master. The full candidate catalog stays on its own immutable branch. Deployment mode Existing AWS/TestOps job 100, plan 102, branch master. This PR does not dispatch a run. ## What Changed - Pin the published qualified source and its exact launcher and descriptor fixtures. - Preserve current manifest history. Update twelve declaration fields across eleven cases: oracle versions and skill source-file provenance only. Preserve all 244 identities, 242 canonical declarations, queries, setup, human/native counts, clocks, profile, model, retry policy, and global cap of 8. - Port the exact source-owned compact framework helper and private-transfer reader. Keep the three existing required source/controller protocol helpers byte-identical. - Validate both sealed inventories before restoring registered attachments into the derived selected report tree. Verify exact bytes and hashes. Preserve seals and unregistered evidence. - Keep deep JSON, decoded-body, sensitive-key, path, and original/final status checks. Plain-text bodies and registered plain files fail closed if existing diagnostic redaction would change them; JSON remains under structured inspection. Keep all existing physical byte, entry, ZIP, time, privacy, and archive bounds. - Preserve registered clean metadata and attachment bytes. Reject content that needs privacy redaction instead of silently rewriting immutable evidence. ## Verification - 131 existing Node pin, selection, admission, adapter, reconciliation, and security controls pass. The explicit new target assertion and drift negatives remain active. - 86 current-controller controls pass. They cover legacy reports, privacy and static assets, canonical reference guards, both seals, strict archive restoration, derived files, the actual report normalizer, and the actual TestOps/Allure exporter. The original failed result stays failed; screenshot and trace hashes resolve. - Actual P1 controls reject both an unknown password in a decoded plain-text body and an unknown command token in a registered text file. Parent code fails both controls; corrected code passes. The composed source passes 87 affected controls, fresh server declarations, and full E2E typechecking. The old cfe head completed all 49 CI jobs without adverse conclusions, but its review found this real privacy defect. That CI result is not credited to this successor. - The existing report-lock preflight returns already-reviewed at `60f4c06bca47af3ab9d74281f2ef2e1ff559d566b862ba51c2fbe6c8e5bac35b`. - The existing source SDK guard confirms 506 registered inputs, unchanged registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`, and unchanged runtime pins. No registry renewal or SDK build was needed. - Source launcher SHA-256: `2b9a6953ee51f0ea0dbbd475b0491d9a6c39a1d6835eeaa912e0dbb7cf987724`. Descriptor SHA-256: `e026d5c000102f8d50397dc59055d187388304f91dbca9a6ccfbe747004cd16d`. - No model call, evaluation dispatch, evidence download, reporting retry, core merge, or permission change was performed. Full current-head CI and review are pending. Separate source PR failures remain disclosed; local controls do not establish universal source CI readiness. - Seven cases have verified passing historical attempts. A uniform passing ten-case run remains unverified. The old lost multipart-question original remains UNKNOWN. ## Risks This changes a trusted evidence reader and the source selected for future measurements. The new helper is included in the controller import closure. Compact references require matching originals; missing, foreign, changed, unsafe, or redaction-required evidence fails closed. Legacy reporter formats remain valid. A single canonical evidence set can still exceed a bound. Current-head CI and material review must pass before normal protected merge. Actual AWS/TestOps evidence remains required afterward. ## Model Used OpenAI `gpt-6.1-sol`, with reasoning, code editing, and tool use. The host does not expose the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-11-04 18:20:22 -06:00
DottaandPaperclip	52b4b3ccd7	test(evals): pin native credential delivery and worker receipt fixes (#19042 ) ## Thinking Path > - Paperclip must deliver agent work through ordinary human controls. > - The latest three-case run retained three failed originals. > - The personal value was saved, but the native OpenCode child did not receive the configured task environment. > - A worker proof also relied on legacy process identity fields outside that environment contract. > - Timeline and credential checks had independent review defects that could reject truthful outcomes or miss unsafe instructions. > - This PR pins the reviewed fixes together and keeps all prior grades unchanged. > - A new AWS measurement must still prove the real outcome. ## Linked Issues or Issue Description Refs #18963, #18967, #18968, #18971, #18972, #18977, #19027, #19031. Follows merged #18995. This is the next infrastructure pin, not another core implementation PR. The latest launch 2657 retained FAILED originals for timeline, approved-tool replay and personal secret. The personal journey reached secure UI save and a worker wake, but the worker reported no credential and made no endpoint call. These are failed attempts, not new passes. Six of the selected ten cases have independent passes across prior attempts. The uniform final ten and the full catalog remain unqualified. No old result was regraded. ## What Changed - Pin immutable candidate `c6cc59282a45ed3c79880a220f7f60aa3ac1d6a4` on `codex/chat-credential-destination-contract-candidate-20261104`. - Retain the frozen candidate baseline. Add the accepted failed-action completion feedback hint, canonical configured environment delivery to OpenCode, actual native worker output receipt checks, timeline attribution controls, and credential clause safety checks. The feedback change leaves the existing acceptance gate in place. - Declare exactly seven oracle-version changes: timeline v5 to v6; personal v5 to v6; company v4 to v5; secure API key, least-scope delivery, secret proposal and vault reference each external v1 to v2. The four external cases have explicit overrides; the shared default is unchanged. - Sync the manifest rows, explicit changed-field records, descriptor snapshot and expected target. Preserve all 244 identities, original queries, human/native counts, clocks, model, profile, launcher bytes, privacy limits and the existing 196/244 qualification gate. - Include the reviewed retry cancellation readiness fixture repair from `815d47e4b5d7756367d2c5e2da656b640273eac2` in this infrastructure PR only. It awaits the first scheduled batch before polling its state. Its production behavior and assertions are unchanged. This test repair is absent from the model candidate. - Keep the core PRs, including #19027 and #19031, unmerged. A normal merge of this pin does not merge those PRs. ## Verification - Pin-local checks: all 189 existing manifest, TestOps API, adapter, reconciliation and streaming controls passed. The reviewed 815 fixture repair passed its owning real-PG control (one selected test). A temporary delayed-first-batch control failed with the old observer and passed with the new observer. Batch limits, assertions, drain wait and overall budget are unchanged. The original CI cause remains unknown. - Final candidate: 89 credential/receipt consumer controls passed. The exact Runner TypeScript build, 68 driver controls and one owning failed-action PG control passed before the later helper-only changes. Their runtime inputs remain byte-identical; those results are reused explicitly, not claimed as new executions on every source commit. - Real pinned OpenCode 1.18.34 executed a canned local-provider bash call. Synthetic `PUBLISHING_API_KEY` reached its child in a 2.89-second passing control with zero paid provider calls. This proves server-to-bash inheritance; driver spawn controls cover the preceding projection boundary. Actual AWS secure-entry, worker delivery and privacy acceptance remain pending. - The latest local candidate declaration loader is NOT GREEN. It exceeded its unchanged 30-second limit in both a combined run and an isolated run. Five other declaration/contract checks passed. Both failures remain recorded. No timeout was widened and no further retry occurred. Descriptor and definition modules match the earlier passing 5aa source; only the transitive credential helper changed. Prior 242-declaration evidence supports unchanged definitions only, not a claim that these failed executions passed. - The snapshot matches the candidate and exactly seven oracle-version leaves differ from source 249. All 244 identities, queries, run/human counts, clocks, model, profile, launcher and privacy limits remain unchanged. The SDK guard reports 506 unchanged inputs, unchanged runtime pins and registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. The driver is outside those SDK roots. No SDK renewal occurred. Reporting-lock and whitespace checks passed. - Genuine exact-head reviews: source 19e3 is 5/5 with no unresolved threads; current pin 988 is 5/5 with no unresolved threads. The prior pin 4/5 and resolved P1 remain historical. Current exact-head normal CI 41917215227 completed successfully: 48 successful jobs and one selector skip, including final verify job 126888275415. Windows 41917215440 succeeded. The full PR rollup is 56 successes and three intentional skips, with no pending or failed checks. Older green heads do not qualify this head. Paperclip has only an earlier-head 5/5 assessment, which is stale for this head and receives no current assessment credit. - Six of ten cases have independent passes across prior attempts. Launch 2657 retains three FAILED originals; no grade was changed. The next measurement is the exact uniform ten, cap eight and zero automatic retries. No uniform-ten or full-catalog acceptance is claimed. - The local-tests checkbox below covers only the 189 pin controls and owning fixture control. It does not claim all candidate or repository tests are green. Full local repository checks were not repeated for this infrastructure diff. Core PRs remain unmerged. No model call, eval dispatch, private archive read, fleet change or merge occurred for this update. ## Risks Environment delivery crosses a real provider process boundary, so secret isolation remains part of the next actual AWS acceptance. A worker receipt must match an actual successful native run and key-authenticated endpoint call; a copied boolean does not qualify. Oracle safety and attribution changes can alter acceptance, so balanced negatives remain required. No current uniform-ten pass is claimed. Normal CI and genuine exact-head review must finish before protected infrastructure merge. Root owns the next TestOps Submit, with the original model, limits, zero automatic retries and maximum eight concurrent case chains. ## Model Used OpenAI `gpt-6.1-sol` through Codex. Capabilities used: source analysis, mechanical integration, local code execution and PR preparation. The host does not expose the context-window limit. The evaluated model remains OpenCode/OpenRouter `deepseek/deepseek-v4.1-flash-1017`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-04 15:52:12 -06:00
DottaandPaperclip	3bd52a9fa0	test(evals): pin task-history discovery and governed handoff repairs (#18995 ) ## Thinking Path > - Paperclip must complete work through ordinary human and agent controls. > - The paired timeline attempt omitted a real human handoff, although the other ten checks passed. > - The agent used the always-exposed active-task history tool as if its comments were a status audit. > - The tool now states its actual scope and directs chronology and handoff questions to the existing discovery and API tools. > - A separate approved-tool attempt preserved permission and write boundaries, but paired Markdown emphasis caused a false outcome rejection. > - The reviewed oracle normalizes that formatting before the existing outcome checks and retains balanced negatives. > - This PR pins those corrections with the already reviewed personal secure-handoff admission repair for one new representative batch. ## Linked Issues or Issue Description Refs #18962, #18968, #18971, #18972, #18977. Follows merged #18976. The original paired results remain FAILED. Result 4412 omitted the actual human handoff; this is a real answer gap, not a date-oracle excuse. Result 4411 correctly handled the first rejection with no writes or Trust action and rejected agent permission, but the observer misread paired inline emphasis in the truthful outcome. The personal-secret result 4378 remains FAILED at admission; secure save and worker proof were not reached. No prior grade is rewritten and no new native pass is claimed. ## What Changed - Pin immutable candidate `249b487a2d2451763913038cc009f7090580d1f1` on `codex/chat-catalog-human-answer-admission-candidate-20261104`. - Extend exact prior source `b1119630003785c9e26e14c763859125642ad756` with only the accepted timeline-discovery and formatting corrections: 12 source, test and generated files, with 72 additions and ten removals. - Use the canonical `get_task_history` description in the actual exposed semantic catalog. It reads only bounded active-task comments; calling `get_task` does not change that active scope. It is not a status or assignment audit. Progress and handoff questions must discover the company timeline or issue activity, then use the existing API call. - Regenerate semantic catalog and protocol projections on this frozen candidate. Preserve all non-description fields, schemas and authority boundaries. Do not copy newer master catalog or document-schema changes. - Normalize paired inline emphasis before approved-replay outcome checks. Declare only that oracle as app-review v6. Keep timeline v5, personal v5 and company v4. - Retain the reviewed personal completion-context, addressed-human, optional-note and rendered-placeholder safety controls. Actual private save, owner isolation, audit, authorized worker delivery and native continuation remain required. - Update only the existing manifest, expected pin and descriptor snapshot in this infrastructure PR. Preserve all 244 identities, original requests, run/human counts, clocks, model, launcher bytes, privacy limits and qualification holds. ## Verification - The combined candidate passed 542 existing owning controls across 19 suites, including catalog contracts, consistency and 242 lazy declarations with no fixture callbacks. The manifest retains two legacy tasks, for 244 identities. - Candidate Runner checks passed: 15 semantic consumer tests and 12 catalog tests. The owning Runner TypeScript compiler passed after the existing eval-kernel prerequisite built. The first compile reported missing ignored eval-kernel output; the repair used candidate-owned ignored dependency links and output only. No repository type or acceptance rule was changed. - All three generated checks passed: semantic-action catalog, semantic contracts and protocol manifest. Deep comparisons prove each generated catalog differs only in `get_task_history.description`. The seeded fixture changes only its catalog hash, and the protocol manifest changes only the hash of that fixture. The compiled canonical catalog hash matches the fixture. The candidate import conflict added only `getTaskHistoryAction`; no unrelated newer document-schema import was copied. - Both owning fixes cleared fresh exact-head Greptile 5/5 with no actionable threads before this publication: timeline `7f6ba9686b5edba005a49cd15c0d0b52248178dc` and approved replay `0015a79c1e0a3af5292edf4a61aa9bab8f381899`. The personal prerequisite `789bec74a0f43011723fff45596ef9d084bfaba7` also cleared exact-head 5/5 and all 12 threads before its earlier integration. - All 131 existing pin, TestOps contract/API, adapter, reconciliation and streaming controls passed. The descriptor snapshot matches the actual candidate bytes. The only new declaration change from the prior candidate is app-review v5 to v6. Whitespace checks passed. - The SDK source guard reports 506 unchanged inputs, runtime pins unchanged, registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. No SDK registry or runtime certificate renewal occurred. The existing reporting-lock guard returned `already-reviewed` with lock `60f4c06bca47af3ab9d74281f2ef2e1ff559d566b862ba51c2fbe6c8e5bac35b`. - Historical CI results remain recorded. Head 73ea had a server-5 cancellation with exact AWS Spot-reclamation proof. Head 16134 had 44 successful jobs, three failed leaves and dependent verify failure, plus one selector skip. Two failed hosts had exact Spot termination proof; server 11 had two real deadline PostgreSQL failures. After an initial libzstd setup block, the unchanged four-test suite passed locally on exact merge `70e68bff5fb89251f769671ce9ab949ab5bf7fc9` in 33.69 seconds with its original deadlines and assertions. This does not establish the CI cause. One recovery POST returned HTTP403; zero replications were accepted. - Prior head c95 finished with 41 successful jobs, six failures, one cancellation and one selector skip. Four leaves had exact Spot-termination proof; three of their execution logs were unavailable. Server 11 instead had a real `live-recovery-work` next-deadline assertion failure, and server 4 had a real workspace-health connection-refused failure. Dependent verify failed. Windows passed. No replay or claimed test fix erased those results; their two assertion causes remain unresolved. - Exact head `1e89df928f7ccb43bf7af3b814f2c2207d75883a` now has successful normal CI run 41888521175 (attempt 2: 48 successful jobs and one selector skip) and successful Windows run 41888521015. The first attempt retained 46 successes, the failed Typecheck leaf and failed dependent verify, plus one selector skip. Typecheck reported runner shutdown without a TypeScript diagnostic; its Spot request reported no capacity, while the later EC2 termination reason was Client.UserInitiatedShutdown. One authorized unchanged-head targeted recovery reran only Typecheck and dependent verify; both passed. This does not claim a code fix or establish the cleanup caller. All other successful results were carried forward, not fresh remeasurements. - Current exact-head Greptile and Paperclip assessments are both 5/5 with no actionable findings; Paperclip check conclusion is neutral, but its final assessment completed and states Ship. Security checks passed and no review thread is unresolved. Full repository build/typecheck/test were not repeated locally for this three-file metadata change. A fresh actual-serving source, reporting-lock and selection guard remains required after a normal merge and before a new Submit. - No model call, eval dispatch, secret mutation, private archive download, fleet change or merge was performed by this update. The single targeted CI recovery is recorded above. Separate core PRs remain unmerged. ## Risks The discovery description may change tool choice; its effect on the missing handoff remains unproved until a new native attempt. Formatting normalization can affect outcome admission, so existing positive and negative semantics remain required. The personal secure-entry and worker phases are still unproved. The next representative set is timeline, approved-tool replay and personal secret. Company selection remains gated on personal secure-entry and worker proof. The uniform ten-case regression remains pending, and the full 244-case catalog remains unqualified. ## Model Used OpenAI `gpt-6.1-sol` through Codex. Capabilities used: source analysis, isolated mechanical integration and local code execution. The host does not expose the context-window limit. The evaluated model remains OpenCode/OpenRouter `deepseek/deepseek-v4.1-flash-1017`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-04 13:00:37 -06:00
DottaandPaperclip	e968131efa	test(evals): pin reviewed outcome and native handoff repairs (#18976 ) ## Thinking Path > - The catalog must measure the original human requests through ordinary product controls. > - The latest four-case run exposed two oracle false negatives and a real suppressed native handoff. > - Preserve those original failures and fix only their demonstrated causes. > - Keep the reviewed company-selector successor and add the accepted timeline, approved-replay and personal-completion corrections. > - Select native standing-chat terminal answers by authority while preserving governed waits and ordinary-task behavior. > - Bind the resulting immutable candidate and its three changed oracle versions through the existing manifest and descriptor snapshot. > - The next personal-secret attempt must prove the complete secure-entry and worker journey before company selection or a final ten-case regression. ## Linked Issues or Issue Description Refs #18971, #18967, #18968, #18972, #18977. Follows #18974. The latest four-case run retained original failures for timeline, approved tool replay and personal-secret setup. Team installation independently passed. The personal native run had actionable final directions, but presentation selected an earlier ordinary comment and suppressed that answer. This pin makes the reviewed corrections available; it does not regrade those attempts or claim a new native pass. ## What Changed - Pin immutable candidate `5f5e2ca8f18063d051c3d948105f0049562fe4ce` on `codex/chat-catalog-four-outcome-repairs-20261104`. - Refresh the existing descriptor snapshot and matching manifest metadata: timeline v5, app-review v5 and personal-secret v4. Company v3 is retained. Preserve the prior reviewed company recipient correction. - Include the reviewed structural standing-chat response selection and finished-turn recovery, preserving earlier comment history, governed interaction ownership and no-final fallback. The superseded prose classifier is absent. - Preserve all 244 identities, original queries, provider/model, human and native counts, clocks, launcher bytes, private evidence limits and qualification holds. The existing SDK registry/compiler proof is unchanged. - The infrastructure change is three metadata files. No product fix is merged by this PR; its owning core PR remains separate. ## Verification - Exact final candidate: 510 owning tests passed across 19 suites, including catalog contract/consistency and lazy declaration loading. The candidate declares 242 cases plus two existing legacy tasks; no fixture callbacks execute during admission. - Candidate response summary suite: 96 tests passed. The owner separately reports 103 summary/notice controls and 14 recovery PostgreSQL controls; these are distinct from candidate results. - Candidate PostgreSQL scope/recovery controls: four passed (three native/ordinary/legacy caller scopes and the new server-finished yielded unknown-channel recovery). The focused filter left 52 unrelated controls unselected. Earlier setup attempts failed before assertions due missing native library links and plugin-sdk output; package-cwd shipped hydration and one owning SDK compiler build enabled this final actual run. No source workaround, timeout change or acceptance weakening was used. - Two deliberate baseline joins preserve the frozen candidate: the caller test adds only its required `issueComments` import; recovery adds only current-conversation matching yielded result/terminal with no owned interaction, retaining all existing succeeded/completed/session/ownership/authorization guards. The test adds only `statusDecisions` and needed fixture options. Fresh-master ordinary `needs_review` behavior and unrelated imports were not copied. - All 131 existing pin, TestOps contract/API, adapter, reconciliation and streaming controls passed. The descriptor snapshot exactly matches the candidate declaration bytes. Diff whitespace checks passed. - SDK source guard: 506 inputs, `changed: false`, `runtimePinsChanged: false`, registry `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. The local compiler invocation supplied missing ignored test output; it is not a renewed SDK qualification or runtime certificate. - Existing reporting-lock guard returned `already-reviewed`. Fresh protected master `86814c6550f52e99ede150a883bdc773cbb540e3` has admitted lock `60f4c06bca47af3ab9d74281f2ef2e1ff559d566b862ba51c2fbe6c8e5bac35b`. Repeat the existing serving-master guard after merge and before a new Submit. - The earlier pin run exposed a pre-existing remote-failure-retention fixture failure. Normal protected merge #18984 repaired that baseline on master `04a24f4e06da68043378bdf3646f50292efd5316`; its 47 real jobs and aggregates succeeded despite delayed GitHub UI/API status projection. This branch normally merges that actual master. Its three pin metadata files are byte-identical to prior reviewed head `dec78737cc520ce6d489fb8bfccafb460b79e870`, with the same immutable candidate, descriptor bytes and model/count/clock/provenance fields. No baseline fixture patch was duplicated in this PR. - On updated head `bd5402d4688e690ffd9f1f80081577733ceeda25`, the existing 131 controls pass, reporting lock is already reviewed, and existing `selectedCases` admits exactly case 580 without execution. Fresh actual master lock remains admitted. - Exact-head normal CI run 41865757115 has completed its full actual job set: 48 succeeded and one legitimate selector skip, including `ci / e2e` and `ci / verify`. RunsOn completion records and exact GitHub aggregate logs corroborate the results. Windows run 41865757064 completed SUCCESS; its exact log reports all executed Rust test results as passing. GitHub UI/API/check-rollup projections still contain stale queued/in-progress rows, as independently observed during #18984; this is not a claim that those displayed rows are green. A normal protected merge must be accepted by GitHub without override. - Fresh exact-head Greptile is 5/5, the current Paperclip assessment explicitly says 5/5 and Ship, all review threads are resolved (zero threads), and the security scan succeeded. Earlier company-only checks are not used to qualify this head. Full repository build/typecheck/test were not repeated locally for this metadata PR; owning source checks and ordinary hosted CI provide the relevant gates. - No model run, dispatch, retry, cancellation, secret mutation or merge was performed by this update. ## Risks The structural correction changes final-answer selection only for admitted native standing conversations; it still needs a new real secure-entry journey. Oracle corrections can admit previously rejected wording, so retained positive and negative controls remain required. Original failed and unknown attempts are immutable. Company selection remains gated on actual personal-secret secure-entry/worker proof, and all 244 cases remain unqualified as a whole. ## Model Used OpenAI `gpt-6.1-sol` through Codex. Capabilities used: source analysis, isolated integration and local checks. The host does not expose the context-window limit. The evaluated model remains OpenCode/OpenRouter `deepseek/deepseek-v4.1-flash-1017`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes — manifest and expected-pin assertion agree - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green — actual job/aggregate results passed; stale UI/API projection is disclosed above - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-04 10:31:23 -06:00
DottaandPaperclip	32e5eada06	test(evals): pin reviewed attribution and secure question repairs (#18974 ) ## Thinking Path > - Paperclip must complete work through ordinary human and agent controls. > - The remote chat campaign exposed observer and setup defects as well as product discovery gaps. > - Case 466 rejected a valid handoff that used headings, pronouns and inline HTML. > - Case 487 seeded three daily runs although its declared journey needs six. > - Secure-entry replies used a real bound question card that the observer did not support. > - Team installation omitted its existing strict request schemas from the API search result. > - This PR pins the reviewed coherent repairs and keeps their metadata in sync. ## Linked Issues or Issue Description Refs #18921. Refs #18964. Refs #18967. Refs #18968. Refs #18970. Refs #18971. Refs #18972. The previous launch remains unchanged. Case 53 passed its latest native attempt. Cases 466, 487, 580 and 610 still retain their prior failures. The next representative run must test the repaired conditions. The company-secret case follows only after the personal-secret journey proves real secure entry and worker use. Its separate case-local follow-up is held for a later pin; case 578 is excluded from the next four-case selection and has no pass credit. ## What Changed - Pin immutable candidate `e757e447371c4781a3eff3a0ea9dc9c4eceb060a` and its existing descriptor snapshot. - Record the explicit oracle versions and source-file changes for timeline attribution, review fixture allowance, and bound personal/company secret questions. - Keep all 244 case identities, human requests, original run and action counts, time allowances, provider/model, private evidence limits and broader qualification holds unchanged. - Preserve the actual request-body schema affordance for team preview and installation, existing response links and prior help. ## Verification - The final integrated candidate passed 442 owning checks across 18 suites. All 242 declarations loaded with zero fixture callbacks; the existing manifest joins those declarations and two legacy tasks into 244 cases. - The team service fixture passed 29 tests. The frozen candidate API consumer passed 2045 tests after the three-line schema exposure change. These are component controls, not a new native pass. - The stale inventory expectation reproduced one failure with 16 passes. Its unchanged derivation traces five previously reviewed YAML corrections. Updating only the expected constant passed 17 controls, and the existing definition-drift and catalog-tamper negatives still passed. No scenario YAML changed in this candidate. - The existing SDK guard confirms all 506 inputs are unchanged and the reviewed registry remains `efa70621e9fb42aa185f7ee616a1a726aa3eccb367e67e28817cedf26ef2ade8`. No compiler build or registry renewal was needed. - The existing pin, TestOps contract, adapter, reconciliation and streaming controls passed locally. Current master also passed the existing reporting-lock guard with its actual current lock. - The final source includes the reviewed immediate-actor/consecutive-pronoun controls and checks every rendered question title, header and submit label for unsafe instructions. The remaining fixed scanner reasons are diagnostic only. The combined source passed 442 checks; all 131 pin/contract/API/streaming controls passed. - Fresh exact-head [normal CI](https://github.com/paperclipai/paperclip/actions/runs/41847423336) passed: 48 successful jobs and one selector skip, including both required aggregates. Greptile reports 5/5 for this head, with no actionable findings or unresolved review threads. The current optional Paperclip review is queued and is not credited as an assessment pass; its prior-head 5/5 remains historical. The finite source peer review found no material issue and did not claim native execution. The prior CI attempt is retained. No new native pass is claimed by this PR. ## Risks The new source repairs observation and setup, but it does not prove the next native outcome. The secure-entry accounting records one real chat request and one authenticated UI answer as the two declared human actions; it keeps the raw comment count separate. The question path requires the bound pending card, actual resolver audit and delivery facts. Ordinary cases retain their existing accounting. Private scans and archive limits can still reject evidence. Prior failed and unknown attempts remain immutable. ## Model Used OpenAI `gpt-6.1-sol` through Codex. Capabilities used: source analysis, integration and local checks. The host does not expose the context-window limit. The evaluated model stays OpenCode/OpenRouter `deepseek/deepseek-v4.1-flash-1017`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-04 08:40:01 -06:00
DottaandPaperclip	c6fe9b143c	test(evals): pin reviewed chat repairs and restore UI test setup (#18921 ) ## Thinking Path > - Paperclip agents must describe recorded work and respect human review decisions. > - The catalog must test the user's request without adding an unrequested format rule. > - Timeline case 466 wrongly required date and timezone tokens in every valid answer. > - Missing private evidence and stalled review admission also need bounded failure facts. > - The reviewed changes correct that contract and add diagnostics without changing product behavior. > - A new retained secure-entry reply also exposed an observer false positive for an explicitly excluded chat destination. > - A retained team-install reply also omitted every resource link, so the reviewed service now returns canonical hrefs. > - This PR pins the coherent corrections and fixes the observed UI-test interruption helper. ## Linked Issues or Issue Description Refs #18914. Refs #18923. Refs #18927. Refs #18780. This prepares the next diagnostic selection. It does not change the three-case run already in progress. The prior case-466 failure remains a failure because its answer omitted the required handoff. The prior case-53 original grade remains unknown. ## What Changed - Pin immutable candidate `25cd5e4ebb6b955701778d2d1d702f86a573cfaa`. - Update the existing descriptor snapshot and matching timeline oracle declaration to v3. Only case 466's expected outcome is clarified: respect a timezone when explicitly requested. - Update case 487's existing source-file declaration to include its diagnostic helper. Its oracle, request, run counts, and clocks stay unchanged. - The candidate adds fixed carrier/reason counts for case-53 evidence failures. It also records bounded, issue-bound delivery/wake/run facts only when case 487's existing settled check fails, then rethrows that original error. - Correct the shared chat-credential matcher for secure-form instructions that explicitly exclude chat. Mixed or later chat requests still fail. Give case 580 an explicit personal-secret v2 oracle identity and update its existing snapshot and manifest declaration. - Include the reviewed additive team-install resourceLinks response, its shared type and actual 201 OpenAPI schema. Preserve the full existing partial-install/repeat help and synchronize the existing typed Storybook result. The core PR remains unmerged. - Fix the UI test announcement handler that timed out during the two-tab attachment check. Only a detached target with no current replacement is ignored; real remaining-target errors still fail. The original functional assertions and deadlines remain. This changes no catalog case 550 behavior. - Keep all 244 case identities, original requests, model, deadlines, private scanner limits, and broader runtime qualification holds unchanged. ## Verification - All 47 existing TestOps contract, adapter, and manifest controls passed on this pin. - Integrated timeline and private-diagnostic controls passed: 162 timeline checks, 14 evidence checks, and full 242-declaration admission with zero fixture callbacks. - The source author ran 53 real PostgreSQL/review controls and the declaration-backed runner typecheck on the exact accepted review diagnostic. Those passed. - The integrated replay passed 179 tests. Its three PostgreSQL observer cases initially could not start because the local native package lacked a shared library. After the existing same-version runtime was resolved through owned dependency links, those three real PostgreSQL cases passed. The setup failures are retained; no source, test, timeout, scanner, or assertion was weakened. - The retained secure-entry consumer replay accepts the truthful secure-form/not-chat handoff with the correction. All 25 shared consumer checks and the fresh full 242-declaration admission passed (28 tests total). The earlier failed result is unchanged. This does not yet prove human entry or worker delivery. - A cheap prefreeze check on actual master passed the existing reporting-lock guard with unchanged reviewed lock bytes. - The resource-link type changes exactly one of 506 SDK inputs. The existing owning helper freshly compiled shared and plugin SDK outputs. Measured SDK runtime JavaScript is unchanged; the type declaration, its map and two build receipts changed. The emitted resource-link consumer compiles and a foreign resource kind fails with TS2322. The old registry/default proof rejected the change; the existing generator updated only that input and the new default proof passed. All 51 SDK and 20 input/build custody controls passed. Compiler, parser, Zod and canary certificate pins are unchanged; held runtime cases remain held. - Core service/wire/OpenAPI controls passed (72 tests), plus four real PostgreSQL checks. The typed story synchronization passed UI typecheck, catalog checks and 30 controls. Candidate token gates passed. - The prior CI two-tab check failed in the announcement locator handler. A controlled real-browser disappearance reproduces it before the fix and passes after it; a genuine remaining-target error still propagates. That prior failed check is retained. Fresh normal CI and review run on this actual correction. - The review-requested real detached-ElementHandle control now covers the catch branch and asserts the target is absent. Removing that predicate fails with the real browser detachment error; the restored three controls pass. This is test coverage only; helper behavior and deadlines are unchanged. - Fresh exact-head normal [PR CI](https://github.com/paperclipai/paperclip/actions/runs/41774814211) and [Windows Runner](https://github.com/paperclipai/paperclip/actions/runs/41774814064) passed. Greptile and the completed Paperclip review both report 5/5 with no actionable feedback; all review threads are resolved. The optional security scan is neutral, not a security assessment PASS. No new catalog runtime PASS is claimed for this candidate. ## Risks The timeline contract correction accepts valid answers without extra date labels. It still rejects explicit dates outside the requested window, missing attribution, failed-run misstatements, hidden work, and unsupported claims. Balanced owning controls retain those negatives. The diagnostics add evidence rather than changing delivery behavior. Case 53 can still fail closed at the existing archive limit. Case 487 can still fail its original settled check. The additive resource links are a product affordance and still need a real agent rerun. Earlier failed grades and attempts remain unchanged. The full 244 runtime catalog is not merged into master; its existing 196/244 gate remains. ## Model Used OpenAI `gpt-6.1-sol` through Codex. Capabilities used: source analysis, code integration, and local validation. The host does not expose the context-window limit. The model under evaluation remains OpenCode/OpenRouter `deepseek/deepseek-v4.1-flash-1017`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 22:56:20 -06:00
DottaandPaperclip	683fa4725e	fix(evals): refresh native runtime and pin reviewed catalog repairs (#18780 ) ## Thinking Path > - Paperclip lets people inspect and manage their agents’ work. > - The timeline must report canonical ownership history, and OpenCode tool failures need bounded diagnostics. > - The measured candidate also had independently reproduced question-navigation and oracle defects. > - This change promotes the reviewed production fixes and pins one immutable, qualified source composition. > - Original queries, model, counts, clocks, privacy controls and qualification holds remain intact. ## Linked Issues or Issue Description Refs #18775, #18777, #18779, #18711, #18713, #18718, #18699, #18798, #18799. What happened? The measured candidate predates merged native-runtime fixes and contains fixture defects. The timeline omitted recorded reassignment/removal history. Finish-input errors lacked a closed correlation and shape witness. Expected behavior: project recorded ownership transitions without inventing facts, distinguish successful native execution from failed verification steps, and retain content-free finish diagnostics. Steps to reproduce: repeat an unchanged assignee while editing another field; then reassign or unassign it. Only recorded transitions should appear. Invalid OpenCode finish input should produce a correlated rejection witness without values or schema text. Related #18699 adds gated company OTLP diagnostics. This change uses the existing local callback; it adds no exporter or capture policy. ## What Changed - Promote canonical timeline creation, reassignment and removal projection from #18777. Initial child ownership requires a recorded fresh-creation receipt; delegation requires a readable recorded parent. Unknown legacy facts fail closed. User actors/owners require active company membership; agent actors/targets must exist in that company. Spans, events and edges share that admission filter. Remove the introduced all-history absence query and use guarded indexed UUID lookups. - Add the shared/OpenAPI `unassigned` event kind, preserving newer main contracts and issue-route/service code through reviewed semantic joins. Existing UI consumers remain generic; no new visual marker is claimed. - Add bounded OpenCode finish diagnostics: schema digest, hashed call identity, closed shape enums and validation category. A throwing sink does not change admission or cancellation. - Pin immutable candidate `fd5e3b1e5a379524c0d1651ed8273e75ffd86541`, composed from combined #18777 source `b734393eff0d7ef85b7ad8abd556d2a66ca3a4e6` plus the reviewed MCP diagnostic delta. It preserves actually merged #18775 `97c07d149b5670088faf15317dcf603c75e2af87` and #18779 `a352c057d475067b4fe67375f06785287dfbe0ce`, and merged runtime fixes #18711, #18713 and #18718. - Qualify narrative handling for adjacent repair/verification sentences, intervening native-failure wording, and negated repair objects. Questions, denials and hypotheticals retain their controls. Preserve Previous question beside the qualified Entity type import; separately restore the already-called `internalResultLink` goal helper binding. That import is a runtime fixture repair, not a type-only change. - Include the narrow older-baseline CI typecheck closure and causal heartbeat test observers in the candidate. All four default typecheck projects execute. Main already includes the protected upstream tooling and newer heartbeat fixture, so neither is overwritten with an older candidate file. - Preserve all 244 identities, original queries, model, native/human counts and clocks, including the explicit prior Studio accounting exception. The full catalog stays on its candidate branch; the 196/244 gate and historical SDK runtime holds remain. ## Verification The primary’s f478 retry repeated the real composer-hidden five-second assertion (1 failure,18 passes,1 skip). Investigation established a setup readiness gap: an outer route response can delay mounting the chat thread beyond the assertion. The successor adds the existing real-thread `aria-busy="false"` readiness barrier before the unchanged composer assertion. The original120-second scenario and digest, unread, source-link, history, and reply checks are unchanged. The full owning browser scenario passed naturally in10.2s and with its first real chat response held6000ms in12.0s. This synchronizes fixture setup; the underlying AWS late-mount latency remains unproved. Current main gates passed: exact head `823dab49fcd8470140c178771e5c0c6b78725e97` has [successful normal CI](https://github.com/paperclipai/paperclip/actions/runs/41724620555), including both required aggregates, fresh Greptile 5/5 and zero unresolved review threads. Paperclip Review is neutral with a completed substantive 5/5 and no actionable defect. The current optional Superagent Security Scan is neutral/inconclusive: Security Graph context retrieval returned HTTP503, so its assessment did not start. That result supplies no security PASS credit. The primary source PR retains its separate current-head gate before Root sequences normal protected merges. - Reviewed MCP controls: 130 on main and 129 on the older candidate. Meaningful parent comparisons failed before implementation/correlation correction. Controls cover actual offered schema identity, null/omitted admission, bounded rejection categories and throwing-sink cancellation. - Exact unchanged production postimages retain 63 Postgres/public-route controls, nine plan controls and the focused index-plan proof. Owner OpenAPI passes 32 controls; main’s newer contract passes 33. Earlier setup/sandbox failures remain recorded; unchanged database controls were not replayed. - Narrative owner controls pass 216, retaining all prior 207 controls. The independent peer passes 144 balanced cases; earlier 108 and 39-case matrices remain retained. Actual consumer parent failures reproduced the review findings before repair. - Main’s server NoEmit passed on its protected baseline; only two pin files changed afterward. The candidate’s prior full server and four-project checks passed. All server/package/script/root-lock inputs remain byte-identical; the changed Runner project’s guard passes on the current source. Compatibility driver controls pass 47 and six owning suites pass 82. The one explicitly reviewed Docker-only unit omission concerns an unimported image-build feature; all four intended project guards still execute. - Pin/adapter/contract controls pass 47. All 242 expanded declarations, the two legacy dispatch rows, and all 244 singleton private policies retain their authenticated identities. Descriptor, launcher and private-protocol bytes remain exact; declaration loading invoked zero fixture callbacks. - All 506 SDK input blobs match the meaningfully compiled `6abe5739` source vector. Prior real shared/SDK builds measured declaration/map changes and zero runtime-JavaScript changes; a compiled consumer accepts the new kind and rejects an unknown kind. The reviewed registry remains unchanged. Default source proof, 51 SDK controls, 20 custody controls and 16 UI consumer controls remain qualified; runtime/compiler/parser/Zod/canary pins and held runtime grades are unchanged. - The full Mac heartbeat run on prior observer source retained 58 passes / six failures in 802.78s. Two nonassignee controls subsequently pass focused checks. Four late timeout causes remain unresolved locally; no flakiness/infrastructure cause or complete local pass is claimed. [Actual AWS CI](https://github.com/paperclipai/paperclip/actions/runs/41712023737/job/126182620050) ran this exact heartbeat file with 64 passes in 107.209s. Its server/package inputs are unchanged in the current narrative-only successor. The source-identical f478 heartbeat file also passed all64 AWS tests in105.812s; source inputs remain unchanged. Current main ordinary CI passed; the primary source PR must independently close its latest-head ordinary CI gate. - Historical main [run 41713049555](https://github.com/paperclipai/paperclip/actions/runs/41713049555) recorded six synchronized runner-shutdown failures and cancellations; the shutdown cause is unknown. A separate chat passive-control assertion observed one `db.transaction` call where zero was expected. Finite local probes passed 2+2 but did not establish a cause or late stack. No external publication, cleanup cause, source fix or flakiness is inferred. The primary’s separate composer-visibility CI failure also remains an inconclusive historical result; no guessed fix is credited. No old-head retry was used. - Earlier UI/inbox fixture failures and the CLI startup timeout remain historical. UI/inbox repairs are now upstream in #18799/#18798 and main preserves those files verbatim. The CLI timeout cause remains unknown. Broader local repository/Rust qualification is not claimed; fresh hosted CI supplies broader coverage. - All four actual repair merges, an exact merged-tree/diagnostic source join, and fresh controller/source/private-policy/active-run guards precede the ten-case TestOps measurement. No paid or model eval was launched for this PR; the original three passes/seven failures remain unchanged. ## Risks The next ten-case model result is unproved. Earlier MCP timeout and continuation-shape causes remain unknown; diagnostics provide future evidence without relaxing validation or rewriting grades. Prior local/CI failures are retained, including the unresolved chat assertion and local heartbeat timeouts. Initial ownership requires canonical in-window audit facts. Missing legacy facts remain unknown. The additive `unassigned` contract is synchronized across shared/OpenAPI/server consumers, with no new UI marker. SDK runtime holds and the broader 196/244 qualification gate remain. Normal protected merges require fresh checks and resolved feedback; no CODEOWNER or branch-protection bypass is authorized here. ## Model Used OpenAI `gpt-6.1-sol` through Codex. Capabilities used: repository analysis, code changes, source validation, and local test execution. The context-window size is not exposed by this harness. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — current focused/type/pin controls pass; the prior full Mac heartbeat failures remain disclosed and receive no local PASS credit - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green — current main normal CI and required aggregates passed; optional neutral reviews and separate primary gate are disclosed above - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — exact current main head; full thread pagination verified - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 16:17:51 -06:00
DottaandPaperclip	76ee5d0886	fix(evals): preserve singleton reports and pin reviewed chat repairs (#18749 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The chat eval lane runs selected cases from one immutable candidate. > - The finalizer must authenticate every original projection and publication. > - The pinned artifact action places one matched artifact at the download root. > - The finalizer expects named child directories, so a valid one-case run fails before final coverage verification. > - This PR supports both valid layouts and binds one reviewed repair candidate. > - The lane keeps exact selection, source, digest and publication checks. ## Linked Issues or Issue Description Refs: #18740, #18736, #18742, #18734, #18717. This PR is a proposed replacement for the source pin in #18740. That PR remains open. The evaluated candidate contains all five repairs from that prior cut and five further reviewed repair conditions. The product PRs remain unmerged. Merge authority for this replacement is pending; this PR does not dispatch a run. A completed one-case run has a verified original PASS and a verified publication. Its final aggregate fails because the pinned artifact action flattens a single matched artifact. The same path assumption also affects the later publication verifier. The final checks must still run; the repair must preserve their identity and digest checks. ## What Changed - Bind the finite candidate `ab2019e976076ec0eb60b026844f3d9211cb0208` through the manifest and exact descriptor snapshot. - Retain the original 244 case identities and the explicit cumulative definition overrides. Keep the historical definition source separate from the current candidate. - Add the reviewed source dependencies for real hire-review readiness and the strict later-human-intent review overlay from unmerged #18742. Preserve all original clocks, queries and model settings. Keep the earlier Studio144 accounting change explicit. - Document one current target and identify the prior Source5 cut as unmeasured. - Preserve named singleton download destinations for both bundles and publications. Authenticate every bundle before gathering its safe projection. Reject mixed, foreign, missing and symbolic-link layouts. Accept only the producer's exact closed once-marker schema. Do not export or upload results again. ## Verification - Candidate controls: 217 pure checks in 13 files; 242 real factory declarations with zero fixture callbacks; 42 admission controls; unchanged 506-input SDK registry; focused actual database-row type check. - Combined trusted pin and layout controls: 124 Node checks in seven real files, plus source-byte closure for all 244 singleton evidence policies. The unchanged parent reproduces both layout failures. Actual producer-marker, multi-case and malformed-layout controls pass. - The two actual PostgreSQL route regressions on core #18742 head `61996a96dd09e90a5788e8711a94fc5ae631f425` ran and passed. That core PR remains unmerged. This is not chat-case acceptance. - The independent layout logic peer and the whole final eight-path PR source peer passed. Current-head CI completed with 48 successful jobs and one skip. Both required checks passed. All 60 exact-head checks completed: 55 success, two neutral and three skipped. The separate Windows Rust check passed. Greptile and Paperclip Review both scored 5/5, with no unresolved threads. - The optional raw-order note is [disposed with source evidence](https://github.com/paperclipai/paperclip/pull/18749#issuecomment-6598680442). The admitted selector writes canonical order before hashing. A changed raw order fails closed before copying. The checked code remains unchanged. - Full local candidate typecheck and build are not claimed. Installed workspace dependency gaps prevent that full local check. The existing remote source and build preflight must pass before a native query. - No new model run or TestOps upload was used to prepare this PR. Prior attempts and grades stay unchanged. ## Risks A permissive layout repair could hide missing or foreign publications. Exact selected names, singleton counts, source identities, hashes and final readback checks must remain enforced. The layout helper must reject symbolic links and mixed shapes. The active default-branch ruleset requires CODEOWNERS review for the workflow. Normal merge is currently blocked on that review. This PR does not enable auto-merge, change branch rules or dispatch an eval. The candidate contains explicit unmerged product overlays. It is not a core release or a full-catalog qualification. The other five SDK cases remain held. SDK115 completion remains unknown. Provider quota failures are not fixed by the skill observer change. No credentials, caps, runner permissions or workflow admission references are added. ## Model Used OpenAI `gpt-6.1-sol`, with code execution, tool use and independent agent source review. The host does not expose the exact context window. No model eval was run to prepare this PR. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes:` / `Closes:` / `Refs:` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub links) - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-03 06:03:25 -06:00
Dotta	2aeceb963d	fix(evals): pin reviewed Source4 chat repair candidate (#18736 ) ## Thinking Path > - Paperclip manages AI agents and their work. > - The chat catalog checks real user journeys through a fixed candidate. > - The current candidate retains several known fixture and observer defects. > - A new source must bind each repaired declaration before paid work starts. > - This pull request pins the reviewed Source4 candidate and its exact descriptor snapshot. > - The next measurements can test those repairs without changing old results or relaxing private evidence guards. ## Linked Issues or Issue Description Refs #18726. Related core changes remain separate and unmerged: #18717 and #18734. What happened? The active lane still uses Source3. It cannot test the accepted primary, app review, CLI, timeline, plan review, hire fixture and SDK input repairs. Old results must remain historical. Expected behavior The trusted lane must bind the new immutable candidate, all 244 case identities, exact current declarations and private source hashes. It must retain the original definition provenance and explain changed conditions. Steps to reproduce 1. Compare the current manifest with candidate `51ef450754370c8a8f2cbe4bd18e1536bbc93966`. 2. Check each of its 242 declarations against the checked descriptor snapshot. 3. Run the focused manifest, contract and reconciliation controls. Run the selected real cases only after normal source delivery and a fresh execution guard. ## What Changed - Pin candidate `51ef450754370c8a8f2cbe4bd18e1536bbc93966` and its exact descriptor snapshot. Keep all 244 identities and aliases. - Record cumulative original-to-current field overrides. Keep the original `de094` definition provenance and case limits. Studio144 retains its earlier explicit 2..3 accounting revision. - Record the reviewed SDK expected build-lock correction and the candidate-only full proposal disclosure from unmerged #18734. Keep compiler/parser/Zod identities, privacy guards and the other SDK holds. - Update the static owning controls and identify current versus historical sources in the lane guide. Runtime JavaScript, declarations, workflow permissions and private transfer code are unchanged in this PR. ## Verification - 47 focused Node controls passed: contract, manifest and idempotent reconciliation. - Candidate controls: 293 pure tests across 16 files passed. All 242 actual factory declarations and 42 admission controls passed without fixture callbacks. - The final candidate SDK registry check matched all 506 inputs. The actual Git launcher, checked snapshot, three private protocol files and all 244 singleton evidence policies matched. - The candidate product disclosure is byte-identical to the reviewed #18734 file. Its token gate passed. The separate core change retains 115 React controls. - Independent full PR and integration source review passed before publication. - Full local workspace typecheck, application build and model execution were not run for this pin. The existing workspace dependency limitations remain recorded. Latest-head CI `41586212647` passed on attempt 1: 48 passed and 1 skipped. Both required checks passed. Greptile reviewed the exact head at 5/5 with no open threads. ## Risks - These are new candidate measurements. A source-qualified repair does not prove a user journey passed. - SDK115 must first pass real admission with its original 300-second/one-native limit. The other five SDK cases and plugin558 canary certificates remain held. - The CLI parser rejects unresolved placeholders and foreign values. It executes the actual returned commands without substitutions. The old failed command remains failed. - The core guidance and review-card PRs remain unmerged. This pin changes neither core merge authority nor the previous full-catalog acceptance gate. - A running Source3 launch remains pinned to its immutable controller and candidate. The next model wave must wait for terminal accounting and cleanup guards. ## Model Used OpenAI `gpt-6.1-sol`, with code execution, tool use and independent agent source review. The host does not expose the exact context window. No model eval was run to prepare this PR. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-11-03 00:04:17 -06:00
DottaandPaperclip	4461905b92	fix(evals): bind repair launcher and Source3 candidate contracts (#18726 ) ## Thinking Path > - Paperclip manages AI agents and their work. > - The private chat catalog tests real user requests through TestOps and AWS. > - Each test must bind its immutable candidate and private evidence owner before it receives model credentials. > - The previous source pin changed the candidate launcher but kept the old launcher digest in the full-catalog evidence policy. > - The private guard rejected the 23-case launch before any original test or model-key step began. > - This pull request binds the full-catalog launcher to its reviewed candidate and selects the next reviewed repair source. > - The result is an explicit source contract for new measurements, with the old grades and privacy gates preserved. ## Linked Issues or Issue Description Refs #18721. Related: #18710 and #18717. The core guidance PR #18717 remains unmerged. A duplicate search found no open PR for this launcher binding defect. What happened? The full-catalog evidence policy used the legacy pilot launcher digest after the candidate launcher changed. All first eight case jobs failed the private binding step. Their original Execute and model-key steps were skipped. The operator cancelled the remaining jobs once. This was an operational failure, not a feature result. Expected behavior Every singleton policy must use the launcher digest from the same reviewed manifest as the immutable full-catalog target. A final source check must compare the digest and private protocol files with the exact Git target before submission. Steps to reproduce Compare the old evidencePolicy digest with launch.ts from target `36dff2e`. The old owning regression fails. The corrected full-lane policy emits the digest of launch.ts from target `43c44ccc`. The legacy pilot digest remains unchanged. ## What Changed - Pin the full catalog to candidate `43c44ccc823a8c972b247fa7159281887e6f5255` and its exact launcher digest. Keep the strict private prepare guard and pilot digest unchanged. - Bind the actual 242-row candidate descriptor snapshot and the explicit changed definition fields. Preserve all 244 case IDs, aliases and original definition provenance. - Select the reviewed source repairs for skill import guidance, immutable skill versions, app review navigation, authorized app testing, adapter prerequisites, the isolated announcement fixture and bounded diagnostics. These are new candidate measurements. - Record the one native-accounting change: Studio144 has a 2..3 envelope and requires three genuine settled source-bound turns. Its query, model, human count and 540-second case limit stay fixed. All other native bounds and all original case clocks stay fixed. - Keep the SDK scanner, compiler/parser identities and canary certificates unchanged. The public SDK source-input registry is synchronized; this does not certify the runtime or release held SDK cases. - Update the current-target documentation and preserve dated source history. ## Verification - The old full-catalog policy fails the new regression with the stale launcher digest. The corrected policy passes for all 244 singleton declarations. - Current CI tests hash the checked exact candidate launcher source bytes and bind their target and Git blob OID. The previous test accepted an advanced target with a stale launcher; the corrected test rejects that mutation. The candidate remains unchanged. - 62 focused Node controls pass across the manifest, API adapter, reconciliation and trusted contract owners. - The source-bound final-GO helper reads the exact Git target launcher and all three private protocol files. It verifies every singleton policy against those bytes. All 244 pass. - The unchanged actual private prepare command passes with the exact candidate bytes. Mutated launcher bytes, a missing launcher and a wrong-target policy fail before outputs. No service or provider credentials are passed in these controls. - Candidate verification: 294 pure controls, eight Git/FS updater controls, actual 242 factory declarations with zero fixture callbacks, and 42 admission/legacy controls pass. SDK registry check reports no change on the final candidate. - Prior head6bda80f passed PR CI41577494646 on attempt1:48 jobs passed and1 intentionally skipped. Both required checks and Greptile5/5 passed. Two late review findings then required declaration synchronization and removal of an unused exported helper. Final head09719566 then passed PR CI41579310890 on attempt1:48 jobs passed and1 intentionally skipped. Both required checks and exact-head Greptile5/5 pass. The whole ten-file diff received independent review after both late findings were fixed. - One actual TypeScript declaration consumer reproduces missing-export/property errors on the prior declarations and passes after synchronization (installed TypeScript6.0.3; isolated consumer only). The finalGO keeps direct exact-source comparisons; no unused guard export remains. - Local full graph typecheck/build and native browser/model execution were not run for this trusted metadata change. Workspace dependency output prevents a clean local full typecheck claim. Latest-head CI is required. No new feature PASS is claimed from these controls. ## Risks - This advances a private evaluation candidate. It does not merge the candidate's core guidance changes to master. Existing result history and the catalog qualification gate remain unchanged. - The new Studio144 accounting and app492 context/outcome are explicit changed conditions. They must not be credited as passes under the original definitions. - SDK input synchronization preserves all scanner guards. Runtime canary certificates still need real qualification; the other five SDK cases remain held. - The isolated announcement fixture contains one public synthetic JSON object. Production announcement feeds and the global index are unchanged. - The cancelled 23-case launch must be reported as operational BROKEN rows with unknown original grades. It must not be retried under the unchanged source. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) assisted with source integration, code, tests and review. The model used tool execution and repository inspection. The host did not report the context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-02 22:54:16 -06:00
Dotta	fea6163811	test(evals): pin synchronized chat repair declarations and definitions (#18721 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The chat catalog runs a fixed candidate through the trusted TestOps lane. > - The first repair source exposed fixture, observation and evidence-readiness defects. > - Changed declarations also need matching static runtime descriptors. > - This pull request pins one corrected candidate and records each changed definition. > - New measurements keep their source and original outcomes separate from past results. ## Linked Issues or Issue Description Refs: #18710, #18693, #18680, #18647. What happened? The first repair candidate left five static descriptors out of sync with its actual declarations. The real no-callback admission check rejects them. A five-case attempt was cancelled before any case or model step. The earlier pin note overstated remote coverage: importing the catalog did not load every actual factory. Expected behavior Every repaired declaration must pass the real factory admission check before a new batch. The trusted pin must record exact source and definition changes. Existing case identities, prior measurements and the full-catalog pass gate must remain intact. Steps to reproduce Run `node --import ./cli/node_modules/tsx/dist/loader.mjs tests/runner-e2e/chat-catalog-consistency.ts` on the old candidate. It rejects five declarations. The corrected candidate admits all 242 declarations without invoking a fixture. The existing legacy-cell scoring and selection controls also pass. ## What Changed - Pin candidate `36dff2eab3dfb6de6efb4fd2ce6c1a58f8a5f383`. Its [candidate-only delta](https://github.com/paperclipai/paperclip/compare/9ab0f951a5b5990c266ef4456610e0a0775aa334...36dff2eab3dfb6de6efb4fd2ce6c1a58f8a5f383) remains separate from master application code. - Record the real descriptor digest and all 19 declaration-field changes across 18 cases. Check them against the exact candidate static descriptor snapshot, including setup policies. The runtime loader still rejects drift. - Keep original definition values and record 19 manifest-field changes across 15 cases. These include the clarified Fast response request, reviewed goal/skill navigation outcomes, and scoped oracle/source ownership changes. - Keep unchanged all 244 ordered identities, model, case clocks, native-run bounds, private evidence rules, workflows and reporting authority. The original ledger and historical results are unchanged. - Require actual factory admission and legacy controls before each repair-batch GO. SDK audit capture remains private diagnosis only; it does not waive scanner rejection or qualify the other SDK cases. ## Verification - Candidate focused controls: 561 passed across 32 pure test files. - Real lazy factory admission: 242 declarations passed, with zero fixture callbacks. The prior source failed 18 declarations before descriptor sync. The two-file admission and legacy scoring suite passed all 42 controls. - Trusted manifest, reconciliation and adapter controls: 38 passed. Contract and dispatch controls: 23 passed. - Independent source reviews cover the repair slices, descriptor sync and this metadata pin. - `git diff --check` passed. - No local app, DB, native or model run was started for this metadata PR. Full harness typecheck/build is not qualified: existing workspace dependency errors remain outside these pure checks. The established credential-free remote Runner build is still required before paid queries. It does not typecheck the full harness graph. - Initial head `90d4ecf` passed first-attempt CI (48 passed, 1 skipped) and Greptile 5/5. A later review found stale current-target prose and an unchecked setup-policy provenance assertion. The corrected head adds the exact candidate descriptor snapshot, computes its digest, and compares every declared field. A wrong-policy mutant accepted by the parent is now rejected, as is a stale digest. Correct metadata controls pass all 38 cases. Latest-head CI41572232196 passed48 jobs with1 skip after one approved failed-only recovery. The first attempt failed one browser cleanup cancellation with PostgreSQL row-lock timeout; the lock holder remains unknown. No server or timeout change was made. All46 successful executions kept their original full step and execution timestamps. Only the failed browser shard and dependent e2e summary reran. Both required checks passed, current-head Greptile is5/5 and all review threads are resolved. An optional review may remain queued; no claim is made that all optional checks finished. No corrected-candidate feature pass is claimed. ## Risks The candidate changes selected setup, observation and oracle conditions. Each new result must retain its exact source and declared changes; it cannot replace an old failed result. The Fast response request and goal/skill outcome changes are explicit new definitions. Static TestOps prose may still show the original definition until a separate supported metadata reconciliation. SDK scanner findings remain unknown until the private audit is captured; the scanner and evidence bounds remain enforced. ## Model Used OpenAI GPT-6.1 Sol (`gpt-6.1-sol`) with reasoning, source editing and tool use. The exact context window is not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the issue template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All required Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge	2026-11-02 21:55:33 -06:00
DottaandPaperclip	8c9dd1e444	test(runner): qualify Kimi Code lifecycle and recovery (#18029 ) Add the Kimi lifecycle and recovery qualification harness. Preserve original grades and cleanup evidence, strict native ownership audits, bounded startup and cleanup, and early rejection of unsupported local fault execution before credentials or resources. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-30 22:54:37 -05:00
DottaandPaperclip	d9354bbfc3	Add a portable agent question and answer resumption journey (#18346 ) ## Thinking Path > - Paperclip helps people manage agent work and keep human decisions attached to tasks. > - Product E2E journeys verify the browser, native agent, saved state, and cleanup. > - The classic suite already checks a choice question followed by a text question. > - Deployment-targeted journeys need the same behavior on an attached instance. > - Production questions can use provider input, the dedicated semantic tool, or the authenticated native API fallback. > - The basic journey must verify the original card and native continuation for each supported path. > - This pull request shares the scenario and browser actions, adds strict API receipt evidence, and verifies the saved document after reload. > - Separate tool-documentation qualification and the original failed measurements keep their existing scope and outcomes. ## Linked Issues or Issue Description What existing behavior does this improve? It adds portable coverage for answering agent questions and resuming the same task on an attached deployment. Current behavior The classic suite has the two-question scenario. The portable catalog does not expose it. Its shared native-path grader recognizes provider and dedicated semantic receipts, but rejects a valid completed native API creation receipt. Proposed behavior Expose `question-answer-resume` as a deployment-targeted journey. Answer one choice question and one text question in the browser. Require server-owned creation evidence, unchanged forms, exact board answers, bound continuation, an agent-authored task document, and readback after reload. Reason and benefit The same story can qualify local and hosted targets through their normal provider setup and cleanup. It catches lost answers, duplicate or foreign work, premature output, and stale documents. ## What Changed - Register the journey with a three-run maximum and positive model allowance. Accept one to three attributed work runs at both collection and retained-result boundaries. - Share the existing two-question scenario, browser answer actions, and behavioral grader. Keep separate tool-documentation grading unchanged. - Recognize a completed native API creation receipt only when its HTTP 201 result retains the original pending card, exact task/company/agent/source run, idempotency key, and full normalized form. Require the receipt to remain unchanged after the answer and through the final checkpoint. Missing, pending, foreign, or altered proof still fails. - Check signed-in board ownership, each native continuation, no premature or extra output, document authorship, and the saved revision after reload. - Wait for the authenticated product region before screenshots. Recheck origin and path after readiness. Bound readiness and capture by the campaign deadline. - Add closed stage-specific errors, catalog coverage, sanitized receipt calibration, and documentation. Include the provider-form and browser-question presentation helpers in the definition fingerprint. ## Verification - Fresh QA staging qualification passed at `7f7de8be286a436010a48af89aba1727b4451493`: all 11 required checks passed, cleanup passed, and all three native runs used the configured QA target and model. The [sanitized run report](https://pages.paperclip.ing/evals-qa-smoke-41121966520-1/) records the measured pass. This is one successful qualification, not a reliability claim. - Final integration head `ae0d9d289831c770a413658d3520fb1dc8ad60e3` contains the one main merge through `fd008979f5794fda3d5f54151d36acd31d0de140` and an exact cherry-pick of the four-file qualified search fix `4edd10804e2b42941aa2038e90c362c13a0b98dc` from #18370. Patch identity matches. It preserves the project and search registrations and atomic search readback. The question live qualification ran at the earlier SHA above; the final integration head has separate local and CI verification. - Eighteen question journey, helper, and test file blobs are identical between the qualified and integration SHAs. This includes clarification flow and evidence, question grader and browser helper, the sanitized receipt fixture, screenshot helper, provider-form helper, browser question presentation, continuation scenario/readiness/flow, shared agent validation, and user actions. The question run-count contract and dispatch branch are also unchanged. Reviewers can compare those paths directly between the two commits. - All 622 selected Product E2E, catalog, question-resumption, necessary-question, report, and readiness tests pass after the main merge. After the exact search delta, all 145 narrow question/search/project/contract tests and all nine real Chromium search regressions pass. Map checks, combined credential-free preview, source fingerprints, and the earlier workspace-link preflight pass. The local TypeScript attempt reports stale Runner declaration errors for `onAccountingEvent` in unchanged native-session code; fresh official CI provides the clean-build type gate. The three search source/test blobs match the qualified search merge, and the three project source blobs are unchanged. - An in-memory probe executes the actual launcher fingerprint expression. Each project, search, and question implementation input changes the digest. Separate red probes first reproduced the omitted provider-form and browser presentation helpers; both pass after their source inputs are added. - The sanitized retained native API response-shape regression fails before the grading correction and passes afterward. Negative controls reject missing, pending, foreign, changed, spoofed, or wrongly bound receipts, answers, continuation, model, environment, and document evidence. All nine provider/semantic/API path combinations pass calibration. - The real local Chromium screenshot probe fails before the readiness repair and passes afterward. Login redirects and sign-in pages at task URLs remain excluded with no screenshot files. - The two original attached failures remain retained: the first stopped at early screenshot capture; the second stopped before answering at the former API provenance gate. An earlier sandbox run also retained one report subprocess failure; the full selected tests then passed with local subprocess IPC access. - The prior integration CI had an unchanged canary mock `EADDRINUSE` failure. That history is retained; its documented retry is owned by the coordinator. No application API, schema, production UI, model, runtime policy, credential, or deployment configuration change is included. Final integration head `ae0d9d2898` passed all 58 CI checks, the external Snyk status, and fresh Greptile 5/5 with no unresolved findings. This clean CI includes the official TypeScript and build gates. Recurring selection follows merge and a separate normal TestOps job verification. ## Risks An API result must prove native creation; form shape and run count cannot establish it. The grader checks the server receipt, exact scope, unchanged card and receipt, real board answers, and bound native continuation. The shared answer helper still serves classic journeys. The screenshot wait affects attached journeys and remains limited to the authenticated product region and campaign deadline. One live QA pass establishes the measured journey outcome at the qualified SHA. It does not establish reliability across models or deployments. The integration SHA preserves its journey and helper blobs, while shared registry changes receive fresh CI and review. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, source inspection, and code execution. The host does not expose the exact underlying model version or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-30 16:18:21 -05:00
DottaandPaperclip	7c247243e5	Qualify and package Copilot for ordinary installations (#15706 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A provider must work through normal installed packages on local and Daytona environments. > - Candidate-only success does not prove that the shipped package contains the manifest and native runtime needed for Daytona. > - The release must preserve saved-token access and task lifecycle guarantees. > - This pull request will qualify Copilot and include the exact-source remote manifest in normal release archives. > - The benefit is a reviewable part of the Copilot production release. ## Linked Issues or Issue Description Refs #15701, #15703 and #15705. Supersedes qualification work in #15563 and the original candidate in #14434. Base: #15705. ## What Changed - Keep Copilot discovery passive during reconnect reconciliation; refresh on explicit opening, account/environment changes and reconnect. - Retain the selected Product cases, permission/continuity evidence and cleanup checks. - Build the Linux provider manifest from the existing pinned Docker stage in the cloud. - Join all four native daemon receipts with that manifest before server packing. - Reject missing or incompatible remote authority during normal release assembly; retain historical workflow compatibility. - Promote the completed profile47 qualification marker, publish the current qualification record and preserve every failed attempt. - Keep one current release checklist and the shipped-canary gates explicit. ## Verification Runtime #15701 is merged at `d855e550ad8d2a267312983f26457aabf5393c3a`. History #15703 is merged at `df80616d98119cbf5abd40549794d9980a5fabf6`. Remaining source [`42bcbd61299655f7e37aabe6bb5f19bba770ad1d`](https://github.com/paperclipai/paperclip/commit/42bcbd61299655f7e37aabe6bb5f19bba770ad1d) contains the four-file authorization repair. The remaining PR trees preserve six nonoverlapping files from the newer public mainline managed-settings fix and the CI-owned lockfile. No unrelated provider is enabled. The security finding is repaired: existing-agent discovery requires `agent_config:update` before connection selection or token resolution. The saved environment is the default. Authorized editors can select a prospective environment. Both denied board-viewer and agent-reader regressions fail on the original source. All49 route tests and server typecheck pass with the fix. Denied requests perform no connection selection, credential resolution or probe. Current-head security scans pass on both remaining PRs; the repaired thread is resolved. Copilot CLI1.0.88 and profile47 retain digest `sha256:55044aa235e88b3fe36ec8a556ee1e3382590870aab907e4462ef7d64af49cea`. The [qualification record](https://github.com/paperclipai/paperclip/blob/42bcbd61299655f7e37aabe6bb5f19bba770ad1d/doc/plans/2026-10-20-copilot-qualification-v47.json) reports20/20 Product cells,154 assertions and7/7 protocol cases,46 assertions against frozen normal installed source `c2616370800f5f6df45246b0890ba35a7cd2faab`. The exact qualification model is `gpt-5.6-luna`. Every passing Product case confirms cleanup. Original failures remain recorded; automatic behavior retries are disabled. The route-only authorization repair changes no bound runtime/profile bytes and does not relabel the frozen campaign. The ordinary installed remote journey passed saved-token selection, model discovery, normal UI save/reopen, metadata-only Test, completed task, accepted receipt and exact19-byte browser download. An open model catalog caused no additional discovery requests during117seconds. Actual installed runtime smokes passed on macOS ARM64, macOS x64 under Rosetta and native Linux x64 on Daytona. Raw token browser entry was not exercised; encrypted connection creation used the normal API. The first Daytona file attempt timed out during delayed semantic dispatch; its precise stalled layer remains unresolved. Independent ordinary remote delivery and the other19 Product cells passed before one instrumented recheck passed8/8 with unchanged source,prompt,assertions and deadlines. No causal latency repair is claimed. Before the authorization repair, all four final PR heads passed required verify/E2E and fresh Greptile5/5 reviews with zero unresolved threads. Current-head CI/review is running for the two repaired heads. Local focused checks retain129 connection,59 runner,72 shared-eval and88 recovery/workspace passes, plus migration safety and token gates. The additive Copilot migration is361; released master history is preserved. [Cloud normal-package run39414239704](https://github.com/paperclipai/paperclip/actions/runs/39414239704) builds the pre-security source4f3f. All four native producers and Linux package assembly pass; the final clean installed consumers are finishing. All18 retained archives match their hashes;37,863 regular files contain zero selected credentials. That package does not contain the later route repair. The actual final published package must include it and pass the local/Daytona live canaries. The frozen Daytona image remains published and independently verified. No separate full-image scan is claimed. Docker builds run in the cloud. Remaining: current-head CI/review, setup and qualification merges, final publication, and ordinary local/Daytona canaries against the actual shipped package. The user explicitly authorizes the maintainer review override; CI remains a merge gate. No production deployment is claimed. Provider USD remains unknown and GitHub-attributed. ## Risks An omitted or mismatched image manifest prevents ordinary Daytona startup. Final normal archives must contain the matching image authority and platform daemons. The completed runtime campaign remains bound to its original source; the final UI-only repair needs its own normal-install setup proof. Credit notices are not measured USD receipts. Native steering, native plan mode and detached work remain outside this release. ## Model Used OpenAI Codex, based on GPT-6. The exact serving model ID and context window are not exposed in this task. Used for code changes, review and tool-assisted verification. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-20 07:33:14 -05:00
DottaandPaperclip	5739a53f5f	fix(tests): sanitize installed-consumer validation processes (#16797 ) Start installed-consumer validation through Node and remove ambient Node startup settings before test processes start. Keep pinned historical source fixtures available in shallow checkouts and set filesystem fixture conditions explicitly. Co-Authored-By: Paperclip <Paperclip-Paperclip@users.noreply.github.com>	2026-10-19 14:12:45 -05:00
DottaandPaperclip	57e977be72	feat: integrate Pi 1.0 into the experimental Runner (#14921 ) ## Thinking Path > - Paperclip manages AI agents and their work. > - The experimental Runner owns provider processes and durable sessions. > - Pi needs working task execution and human controls. > - The five-PR stack must preserve changes already on master. > - Each layer now carries the complete integrated source for a safe sequential fallback. > - This PR belongs to native GitHub stack #15602, ending at #14956. ## Linked Issues or Issue Description Refs #14436, #14631, #14743 and #14956. Ship Pi 1.0 through the experimental Paperclip Runner. The five PRs are #14921, #14922, #14923, #14924 and #14956. The user authorized the complete merge after checks pass. Existing `pi_local` execution is unchanged. Accounting and wider provider/platform qualification remain deferred. ## What Changed - Recover missing final replies after workspace finalization changes owners, using accepted-turn evidence without rerunning work or granting external-chat publication. - Preserve the admitted Pi instruction root across warm runs, while retaining changed-root rejection. - Give Pi a bounded 15-second default shutdown grace so stop, drain acknowledgement and durable suspension can complete. Explicit deadlines and other providers retain their existing behavior. - Integrate the Pi 1.0 runtime and master contracts. - Use Pi profile 22. Preserve explicit caller-selected models and exact native thinking levels. Keep Pi's wrapper, helper, extension and question/control behavior unchanged from the qualified profile-19 runtime. - Preserve master's Dot lifecycle and consent fields, configured task environment, status guards and current Codex/Claude dependency versions. Cursor stays qualified. Copilot stays pending; profile 17 binds the changed shared protocol validation sources. - Exclude general AWS IAM credentials from Pi static/custom provider bindings and selected task projections; preserve the provider-scoped Bedrock bearer key. Profile 21 is retained as historical provenance. Rust and cloud install probes use the current declaration. - Patch bundled brace-expansion 5.0.9 to the exact official 5.0.12 payload. Pin the patch and complete runtime closures. Include the patch in normal installed setup tooling. Keep the upstream Pi shrinkwrap as provenance and permit only this exact security correction. - Include current attestation files in the Docker build context. Keep the repository lockfile unchanged from master. CI and private image builds resolve manifest changes before their frozen installation. ## Verification - Full local `pnpm -r typecheck` passes, including Runner Rust, server and UI. Focused integration checks pass: 194 Runner admission/environment tests, 63 profile/credential tests with one expected skip, 152 Dot/UI configuration tests, and Pi transcript/notice tests. - Full local `pnpm build` passes on the final source. - Fresh final-source checks pass: all 698 Rust workspace tests (32 binaries), 156 credential/profile/controller tests with one expected skip, Runner TypeScript typecheck, and 20 package/setup/sandbox tests. - The profile-21 Pi materializer passes on the native host with the official pinned Node 24.21.0 and its npm. It verifies all 150 locked packages, the patched dependency and the exact closure. Setup/package bundle tests and UI token gates pass. - The old hashes were reproduced for all three supported targets before calculating the patched graph. New closure hashes are darwin-arm64 `282022db10150c6632b3444df421342e7d534bdf5d5fb1097a2e79d0625a2bcf`, darwin-x64 `64e251e19009f755c0b04f73ce2138246faab71a961b0f13d75ebfcc34bef12e`, and linux-x64 `713b1fdff42fb56a1518bdc084f181d70bee8ebadc3e4b1d76321ed9108c8410`. Independent native platform execution is separate from graph identity reproduction. - Historical cloud qualification remains unchanged: all seven core cases pass on shipping source `10dc43c9ec65d88c2f782d62afb296d09494f215`, harness `1a4408a48cfb5a1f094a311141c257c92cd7a893`, image `sha256:5b3a775b383591bda1b0c1889e509acc70ce7f37c53f09733c81d59037f02280`, and accepted Sonnet 4.6/low fixture. All 215 canonical files and all seven cleanup checks pass independent verification. These are profile-19 results and are not relabeled as fresh profile-22 runs. - Current Pi digest: `sha256:e92078bee3c23bec4100aa589013a44613d054cd686826534025d8019e9f39a9`. [The readiness plan](https://github.com/paperclipai/paperclip/blob/codex/pi-production-readiness/doc/plans/2026-10-02-pi-production-readiness.md) preserves campaign and failed-attempt provenance. - Merge only after every PR's current-head CI and fresh review pass. Linux CI covers the full suites, build and browser tests. The local embedded Postgres API-authority suite cannot start on this macOS/Node 26 host, so Linux CI must confirm that suite. ### Fresh profile-22 core qualification — 2026-10-08 All seven accepted core cases pass canonically on Pi profile 22, with `openrouter/anthropic/claude-sonnet-4.6` and native-confirmed low thinking. This model is a fixture; production accepts the caller's explicit Pi provider/model. Runtime/install source: `3241a992f2a7703e59e97ed0fd3e5d6405de4401`. Frozen accepted harness: `1a4408a48cfb5a1f094a311141c257c92cd7a893`. Immutable cloud image: `ghcr.io/paperclipai/paperclip-daytona-runner@sha256:506f22db7edd78f37c0c40bec1cc084af1850455026dbf467194bfbb8fcef141`. Pi digest: `sha256:e92078bee3c23bec4100aa589013a44613d054cd686826534025d8019e9f39a9`. [Hosted Linux image and clean-install verification](https://github.com/paperclipai/paperclip/actions/runs/37868328023) passes, including all 20 source-bound archives, normal CLI/Pi setup, companion import and the production pack reader. This exact installation source includes the latest master integration and the corrected Pi warm instruction-root fence. Full local typecheck/build and current-head hosted CI verify the final stack. All 13 focused real-root regressions pass. The full local executor suite passed 662 tests; 15 database tests could not start the Mac embedded PostgreSQL service. Hosted Linux CI passes the full required verification and E2E checks. These fresh results keep their own source identity; profile-19 results remain historical. \| Core path \| Canonical campaign \| Retained archive SHA-256 \| \| --- \| --- \| --- \| \| File edit, validation, download and Done \| `pi-core22-replyfix-0-1791511228` \| 23 files; `a473e8603a3dd4737863291f8d3d1e392391f0b16d433c3e0e0e9d8baf7a97b0` \| \| Pending question and controller restart \| `pi-core22-replyfix-1-1791511376` \| 33 files; `6b829c4eb74e1f32a89c692a4ae7130dbfc1c6d3cf13915effe2103d9e242c8e` \| \| Three-turn session/process/workspace continuity \| `pi-core22-replyfix-2-1791511587` \| 23 files; `7a87021f8f9a3fdd3c58bb4467f8d82c635e3ea4795d6e75f144d9aa14818df8` \| \| Four typed questions and browser reconnects \| `pi-core22-replyfix-3-1791511881` \| 42 files; `9e31755252be1f4f9cb0626c984c142d4d1ae5f5bee3a7af08444db8d12c280a` \| \| Plan approval and completion \| `pi-core22-replyfix-4-1791512031` \| 22 files; `a0383ce1aab38e7b5a25ce0e9dd3bebea5c037ebd96ae6b29dae19015da2ae2c` \| \| Same-turn steering and permission denial \| `pi-core22-replyfix-5-1791512261` \| 39 files; `c929b8c7070f0b66aedc17e65ca46e6beab1e363926ac9f7e2a75fb250f05949` \| \| Stop during pending permission \| `pi-core22-replyfix-6-1791512390` \| 33 files; `7f0a58ae0f4d5bfc76149435f4e322537089c5bd16e7ffe9b5ad71f10a621a07` \| All 215 canonical files (28714587 bytes) are independently hash-verified. All seven cleanup grades pass, with no owned runtime process or temporary root after each case. Automatic retries are zero. The owned cloud host stopped normally after retention. The prior profile-22 warm attempt remains failed and separately retained: archive SHA-256 `1e54eba5ec72b50cee1534b23d1d1d4f21a090006b8a64501ba70db972abfde5`. Its original canonical classification is preserved. Diagnosis reproduced a product bug comparing an agent-files root against an unset checkpoint-only field. The fix stores the admitted physical root separately from the adopted per-run collection capability. The real-root regression fails before the fix and passes afterward, including rejection of a changed physical root. Fixture, grader, model and all seven accepted case IDs are unchanged; this fresh campaign tests final-reply publication after file registration first. The intermediate restart attempt also remains failed and retained: archive SHA-256 `5dcaefdf1d17cf4cd54fd4cf810f45e736667392339b8ce7caf08bb4e225277f`. Its original canonical classification is preserved. Pi resumed, wrote the verified answer and completed its task; exact runner suspension was proven, but idle stop consumed about 5.2s and left under 3s for the drain acknowledgement. The Pi-only default shutdown grace is now 15s, preserving a full 5s drain round trip and a finite suspension reserve. Explicit caller deadlines, other provider defaults, literal drain receipts and exact suspension identity checks remain unchanged. The timing regression fails before this correction and passes afterward; all 18 focused settlement tests and Runner typecheck pass. The final-source file attempt is also preserved as failed (`candidate_failure`), archive SHA-256 `db6767b6773ea618997927ac77bdb005a5ac81492c7b9c0ffbc900449f829bc9`. Native edit, validation, exact downloadable artifact and Done/succeeded all passed, and the exact final reply was durably recorded. A workspace recovery owner completed before the live heartbeat reached presentation, leaving that reply absent from task chat. Recovery now materializes only a completed final reply from the accepted turn of an ordinary internal Done task, preserving issue/run/contract binding, suppression, external-chat authorization and same-run deduplication. The database regression covers the generated file-preparation receipt, suppression, unapproved external continuation and replay. Server typecheck and all 49 response-selection tests pass; hosted Linux verifies the database regression because embedded PostgreSQL cannot start on this Mac. The delayed-final-answer database regression passes on [the final root-source Linux server shard](https://github.com/paperclipai/paperclip/actions/runs/37868262553/job/113628594152), alongside 1,108 passing tests. The first root Runner shard had one unchanged durable-resume test exceed its 5-second timeout; the identical top-source shard and the isolated exact test passed. One rerun of that failed job and its required aggregate passed without source or test changes. The original failed job log and the single-rerun receipt remain retained. ### October 9 merge verification Current merge head: `5a8fe63512a7166aaef5cf50065a25008aa8b44b`. All current-head checks pass, including `ci / verify` and `ci / e2e`; exact-head Greptile review is 5/5 with no unresolved threads. Current master conflicts are resolved. The user authorized the maintainer override of the code-owner review gate after these checks. The seven retained live core cases remain bound to source `3241a992f2a7703e59e97ed0fd3e5d6405de4401` and its recorded cloud image. ## Risks - The security correction changes the dependency closure and profile identity. Old sessions must reopen on the new profile. Exact identities and credential bindings fail closed. - The runner remains experimental and requires explicit selection. Legacy Pi Local is unchanged. Caller model IDs pass through; the E2E model is a fixture. - Accounting and the broad platform/provider matrix remain deferred. This merge does not publish a release or deploy a service. ## Model Used OpenAI GPT-6 through Codex assisted with reasoning, repository inspection, editing and tool use. The exact serving ID and context window are not exposed in this session. Final live qualification uses Pi 1.0.0 with `openrouter/anthropic/claude-sonnet-4.6` and native-confirmed low thinking. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-09 08:56:41 -05:00
DottaandPaperclip	16b7db35ff	Shorten planning skills and measure task decomposition (#15296 ) ## Thinking Path > - Paperclip manages work for AI agents. > - Planning guidance helps agents choose owners and dependencies. > - The runtime skill favors few tasks, but the catalog skill requires a child-task breakdown. > - Both add repeated process instructions that can distract from the requested outcome. > - This change keeps the ownership and dependency rules and removes the required matrix and repeated checklist. > - A bounded Product E2E comparison measures saved outcomes and task handoffs before qualification. ## Linked Issues or Issue Description Refs #11057. Related measurement work: #15218. What existing behavior does this improve? Planning and delegation through the runtime plan-to-tasks and bundled task-planning skills. Current behavior The two skills contain about 1,900 words and conflicting guidance on whether plans require child tasks. Proposed behavior Keep cohesive work with one owner. Split only for a real owner, parallel output, dependency, independent review, or follow-up lifecycle. Preserve existing authorization and planning mechanics. ## What Changed - Shorten both skills to about 400 words combined. Preserve their keys and installed-version behavior. - Remove the duplicate operational-skill pointer and regenerate affected source metadata. - Add twelve explicit Product E2E cells: four scenarios with current, short and disabled planning skills. - Use the current task composer and actual create-response ID; calibrate public skill APIs and browser creation without providers. - Eliminate an observed collision in chat-test company prefixes with a per-suite sequence. - Grade saved documents, exact author/run attribution, child count, prerequisite execution order, review boundaries and completion handoffs. - Retain current skill bytes and report source, selections, run accounting and failures. ## Verification - `pnpm test:e2e:runner:typecheck`: pass. - `pnpm test:e2e:runner:unit`: 1,287 Vitest tests and 128 Node checks pass. - `pnpm test:e2e:runner -- --list --suite plan-task-guidance`: twelve local Codex cells. - Archived current skills match master `72ff3a9f27e581a27acb49771e8658bbb0bbaa47` exactly. - Corrected fixture: three real public-API/database calibrations pass with zero provider runs; all 35 evaluator checks and Product E2E typecheck pass. - Setup campaign [37399550253](https://github.com/paperclipai/paperclip/actions/runs/37399550253) was canceled after source review found unsupported bundled edits and automatic core reinstallation. Its paid-cell step was skipped: zero provider runs, no behavioral grade. - The next setup [37401094799](https://github.com/paperclipai/paperclip/actions/runs/37401094799) failed before task creation on the old title-field selector: zero actual runs, original FAIL retained, cleanup passed. A real browser/API calibration of the new helper passes with paused non-provider agents and zero runs. - Full local typecheck/build pass. Full local tests retain one unchanged five-minute Git streaming timeout (also fails isolated), 9,591 passes and 5,796 skips. CI's chat failure was a proven random fixture-prefix collision; five affected cases pass after the test-only repair. - Paid behavior comparison and new-head CI/review remain pending. This PR remains a draft. ## Risks - The shorter text may change delegation decisions. Live outcomes are not yet qualified. - The initial comparison uses one profile and one attempt per cell. It cannot establish cross-model reliability or cost trends. - Disabled means unassigned company-owned copies; the company library remains discoverable. This does not qualify global removal, automatic accepted-plan wiring changes, or installed-copy migration. - Skill availability does not prove a model read or cognitively used it. - No provider/tool protocol, permission, timeout or runtime lifecycle behavior changes in production. ## Model Used OpenAI Codex (GPT-6), with repository inspection, code editing and tool use. The exact backend model ID and context-window size are not exposed in this session. The declared eval model is native Codex `gpt-5.6-sol`. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-06 09:36:01 -05:00
DottaandPaperclip	b17019e14d	fix(agents): reduce default instructions and qualify stock harnesses (#14948 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its adapters supply task context and access to Paperclip skills and tools. > - The default hire manual and shared prompts also repeat general work procedures. > - Those procedures overlap with stock provider instructions and the Paperclip skill. > - Existing E2E fixtures supply a QA manual, so they do not qualify the production default. > - This pull request reduces the generic instructions and adds real default-hire coverage. > - The benefit is less competing guidance, with inspectable evidence for preserved skills and task context. ## Linked Issues or Issue Description Refs: #14920. That merged change preserves native Codex base instructions. This PR covers the default manual, shared legacy prompts, operational skill guidance, and the narrowly approved ACP skill-discovery/session-environment repair for measured delivery and credential-persistence failures. What existing behavior does this improve? New non-CEO hires without a custom bundle and legacy task/chat startup and continuation prompts. Current behavior The shipped default manual contains 602 words. Generic task/chat prompts and ordinary resume deltas repeat work procedures already available through the harness and Paperclip skill. Proposed behavior The default manual contains only the eight-word company identity. Shared startup prompts retain identity and connection guidance. Ordinary resume deltas retain current work context without the generic execution contract. Reason and benefit Let the stock harness guide general work. Keep Paperclip-specific capabilities and independently test default hires, skills, ordered comments, and chat restart. Breaking changes New default hires receive less guidance. Existing saved manuals, explicit custom bundles, CEO templates, and specialized wake contracts retain their behavior. The obsolete includeExecutionContract option remains accepted for source compatibility. ## What Changed - Reduce the default hire manual to one sentence. - Reduce shared task/chat defaults and remove the generic ordinary-resume contract. - Keep connection guidance, auth, skills, custom prompts, and specialized wake context. - Add credential-free instruction-boundary gates and 26 explicit Product E2E cells across eight legacy/native profiles, including two focused Paperclip-storage cases. - Capture public hire receipts before providers run, then grade delivered prompts and independent task/chat outcomes. - Add an early legacy skill API recipe for saving a task document, checking the saved revision receipt and linking the document. Improve stock task/heartbeat skill-selection metadata and show a clickable Markdown UI-link example. Keep native tool completion separate. - Advertise bounded routing descriptions and exact successfully staged SKILL.md paths in legacy ACP Claude; keep full bodies on demand and preserve remote path rebasing. - Remove only the provider environment from copied persisted ACP session records, while loading current run credentials and preserving all other options/conversation state. - Regenerate both capability metadata inventories and reject stale manifests/inventories before provider admission. - Publish the original reduction and focused skill-repair comparisons, preserving all failures, automatic recovery, cost coverage and limitations. ## Verification Behavioral qualification remains pending. Original legacy ACP Claude loses the issue document only in the reduced cohort beneath an unchanged credential failure. A source-backed diagnosis finds that neither ordinary assignment reads the staged operational skill, while the runtime persists provider environment in session state. The new common repairs expose skill metadata/path and omit persisted env; strict document and credential guards stay intact. [Inspectable diagnosis and retained hashes](https://github.com/paperclipai/paperclip/blob/9f654db4541d3d002769c988f6e51fc0b08dadbd/doc/plans/2026-10-03-legacy-acp-claude-readiness.md). Current repair head `de0965984ff3edf611ae6d0e7ca5c7d5ae3947bb` incorporates master `569c7203aa24b95440682983ce7940ba1d4247bd` (merged #14961/#15007). All 222 affected adapter tests, adapter-utils/E2E typechecks, and final 96 variant/grader/retry calibrations pass. The frozen historical comparator is `c25697f4260b6f3adfea143c3ae9932e2f42986d`: 8,280 of 8,291 paths identical, exactly two production instruction paths plus nine declared unit expectations differ. The operational skill/discovery/environment repairs, selected model/profile/task/core grader/auth/permissions/retry policy are identical. Both actual launcher prepare→verify admissions pass with zero providers. [Immutable manifest and exact receipts](https://github.com/paperclipai/paperclip/blob/9f654db4541d3d002769c988f6e51fc0b08dadbd/doc/plans/2026-10-03-legacy-acp-claude-evidence/manifest.json). One original legacy ACP Claude cell per variant is authorized, with enforced single campaign attempts, 12-minute deadlines and company/agent 1,000-cent hard stops; every product recovery run/cost is counted. Actual live outcomes are pending. Current normal CI has one failed server shard and failed aggregate verify under diagnosis; other normal gates including typecheck/build/Rust/all eight browser shards pass. Fresh review completed successfully; the valid historical startup/resume masking finding was fixed with per-invocation task/chat checks and strict complete-snapshot capture, calibrated and resolved. Prior heads, failures and campaigns below remain historical evidence, not checks on this repair head. - Prior head `36aa4d81c49a1a8f6f04b1a068fae19aa901955f` is replayed on merged hiring master `862a5758ba0e88a33232c1f1fa645e85c38a3113`. All 52 current-head checks pass with two intentional Storybook skips, including repository typecheck/test/build and the browser shard. Fresh Greptile is 5/5 with zero unresolved review threads. Exact-head stock prerequisites pass 599 assertions (598 TypeScript + 1 Rust), all six gates and retained receipt verification, zero providers/source errors. Fingerprint `a7f5a22d860a88fe20cce213c6d5e0004788f32c8363930729aea4fd740ad16d`. Combined catalog/hiring calibrations pass 67 assertions, E2E typecheck and 26-cell stock discovery pass. Canonical contract/inventory checks and the later issue-derived reference calibration are retained; that reference-only follow-up is not live-qualified by earlier frozen runs. - Prior full repository typecheck/build passed. The complete local Vitest run executed 14,956 tests: 14,870 passed, 83 skipped, three timing failures. All three affected files passed unchanged narrow reruns; original failures remain retained. Current-head CI now passes the full general checks; the original local failures remain retained. - The original 24-pair default-manual/shared-prompt comparison has two new overall classic Claude/OpenCode document-delivery failures plus an additional legacy ACP Claude document loss beneath an unchanged credential-guard failure (not closed by later runs), two newly passing OpenCode ordered cases, seven unchanged failures and 13 unchanged passes. Equal 15/24 totals do not establish behavioral equivalence. [Complete original report](https://github.com/paperclipai/paperclip/blob/875f4c397d9e8c3f12f39dedd59abaf1eaf5236e/doc/plans/2026-10-02-stock-harness-live-comparison.md). - The skill-only repair holds the eight-word manual/shared prompts and merged #14920 fixed. All four matched profile configurations and 203 fixture/behavior files match. Candidate `abd0b628ca642c09a54a4edc56a5227402f6686e` varies only the two skill sources against baseline `bc83fe030234439ac51279502a28803958963e2e`. [Candidate workflow](https://github.com/paperclipai/paperclip/actions/runs/37060885547) and [baseline workflow](https://github.com/paperclipai/paperclip/actions/runs/37060888047) each pass 571 exact-source prerequisites before providers; all eight cells clean up successfully. Failed campaigns publish successfully and remain failed. - Repair pairs: Claude original Fail → Pass; Claude explicit Pass → Pass; both OpenCode cases Fail → Fail. Explicit OpenCode's handoff worsens beneath the unchanged failing UI-link grade: baseline gives a clickable API URL, candidate gives a code-formatted path without an anchor. The request's usable-link wording is narrower in the UI-only oracle. [Complete repair report and safe projection](https://github.com/paperclipai/paperclip/blob/875f4c397d9e8c3f12f39dedd59abaf1eaf5236e/doc/plans/2026-10-02-legacy-document-skill-repair.md). - The subsequent narrow stock metadata/link correction has two matched Pass → Pass cases, zero new machine failures/passes and no pending pairs. Both original-case handoff links remain deficient: candidate uses a wrong PAP prefix, baseline supplies a bare prefix-less slug path; the preserved original oracle only requires a durable document. Both explicit clickable UI-link cases pass revision/content/link grading. All four exact-source 587-check gates, single assignment runs and cleanup pass. This does not establish fix causality because baseline also succeeds. [Candidate workflow](https://github.com/paperclipai/paperclip/actions/runs/37069547401) freezes `fe9dc1e3c518825242ed889ab9c8352986f8c2ed`; [matched baseline](https://github.com/paperclipai/paperclip/actions/runs/37069552374) freezes `0d7ecfa96d72fba79b7f0a25052b42c0686c0488`. This is a skill-only comparison with reduced manuals/shared prompts held constant, not a repeat of the historical-manual comparison. Only original and clarified explicit classic OpenCode cases are selected, two per variant/four expected turns. 8,242 other tracked files and both profile hashes match; protected workflows admit each exact source before credentials. [Complete qualification report](https://github.com/paperclipai/paperclip/blob/74d0d3d945f4c52d0814b5a845ab5bd09f33cd6b/doc/plans/2026-10-02-opencode-skill-routing-link-qualification.md). Candidate original loads Paperclip/reference before saving publicly; baseline original loads it after writing locally, then saves publicly within the same assignment. Reported cost totals are $0.0107824490 candidate / $0.0107909015 baseline, with unmetered runtime. The later reference-only issue-derived link correction is provider-free calibrated and not live-qualified by these frozen runs; no further paid runs. - Retained tool calls show the repaired original OpenCode assignment loads only its assigned output skill before writing locally. Operational Paperclip is first loaded during automatic disposition recovery; its early recipe is visible then, but it never saves the missing document. Explicit candidate loads Paperclip and reads the new reference before saving successfully. All nine actual runs are counted. Reported LLM totals are $0.3802537209 baseline and $0.4918990161 candidate; local runtime is unmetered. - Initial setup, packaging, cancelled/missing-cell recovery, callback test and relative-output attempts remain retained. No completed provider failure was rerun. Frozen measurement branches are unchanged by later canonical metadata maintenance. - Run `pnpm test:e2e:runner:stock-harness`, `pnpm test:e2e:runner:unit`, and `pnpm test:e2e:runner:typecheck`. Select `stock-harness` explicitly for paid execution; it is excluded from `--all`. Prior-head integration: `36aa4d81c49a1a8f6f04b1a068fae19aa901955f` replays this PR on merged hiring #14985 (`862a5758ba0e88a33232c1f1fa645e85c38a3113`), preserving the four explicit custom-CEO-bundle checks, minimal generic manual boundary, and both suites. The combined fixture catalog and hiring calibrations pass 67 assertions; exact-head stock prerequisites pass 599 assertions (598 TypeScript + 1 Rust), all six gates and retained-receipt verification, zero providers/source errors, fingerprint `a7f5a22d860a88fe20cce213c6d5e0004788f32c8363930729aea4fd740ad16d`. E2E typecheck and 26-cell stock discovery pass. Fresh current-head CI passes all 52 checks with two intentional skips, and fresh Greptile is 5/5 with zero unresolved review threads. The prior source-plan browser failure is retained: a deterministic process fixture replayed its last `fixture:plan` command on `chat_task_completed`, writing revision 2 with identical body after the approval handoff. This was not paid provider execution. Rebased current-head CI passes the same assertion without an old-head retry or a change to that browser fixture. The merged hiring change was measured separately on immutable matched unions, with this reduced/shared/operational context and native completion guidance held constant. [Complete original two-profile report](https://github.com/paperclipai/paperclip/blob/f0512647656be78e48abd8c22a3078db8bf6bcd2/doc/plans/2026-10-02-hiring-template-live-comparison.md): [candidate](https://github.com/paperclipai/paperclip/actions/runs/37075466208) / [historical baseline](https://github.com/paperclipai/paperclip/actions/runs/37075469463), 705 provider-free prerequisites each. Both pairs are unchanged Fail → Fail on the exact-five count, with six core delivery checks passing all four cells; 28 actual successful runs include eight automatic completion wakes, zero retries, four successful cleanups. Source-read coverage is uncomparable, actual model charges unknown. Separately versioned provider-free accounting remains analytical work; original verdicts are preserved. This does not rerun or qualify the completed default-manual or native campaigns. ## Risks - Legacy ACP Claude's additional delivery loss is not closed by any later matched run and blocks the no-extra-failing-behavior merge criterion. Legacy document delivery may have relied on the prior manual/shared prompts. The early skill repair improves Claude in one trial; the later OpenCode pairs pass in both variants and cannot establish causality or robust recovery. Both original-case links remain deficient beneath the storage-only grade. The later issue-derived reference correction has only provider-free validation. Native finish/block descriptions must not be supplied to legacy agents. - The comparison holds merged native Codex fix #14920 constant; it cannot measure that fix's before/after task performance. - These bounded skill/context/chat workflows do not measure general coding quality. Unrepresented providers remain unqualified. - Saved manuals and old Codex sessions are not automatically migrated. Codex through ACP still has a separate base-instruction follow-up. ## Model Used OpenAI Codex, GPT-6 family as identified by this session. The exact deployment ID and context-window size are not exposed. The assistant used reasoning, repository tools, code execution, and delegated PR/eval work. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs: #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (relevant suites and all three unchanged narrow reruns pass; complete-run timing failures retained in Verification) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green on the new repair head (prior-head checks retained above) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups on the new repair head - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-03 12:32:42 -05:00
DottaandPaperclip	dd868ed125	fix(runner): share native completion tool guidance (#14961 ) ## Thinking Path > - Paperclip manages AI agents and their work. > - Native Runner agents report completion through finish and block tools. > - The providers receive different descriptions for those tools. > - Completion guidance belongs with the tools that enforce the result. > - This pull request shares the descriptions and refreshes retained catalogs. > - A separate native suite checks completion and blocking on production defaults. > - Legacy agents retain their separate skill and API paths. ## Linked Issues or Issue Description Refs: #14920, #14948, #14985. Current behavior Native Codex and MCP bridges describe finish and block differently. Retained provider sessions can keep old descriptions. Proposed behavior Native providers receive the same finish and block descriptions. The descriptions cover report selection, validation feedback, returned outcomes, approval gates and final-answer timing. Retained native sessions refresh from v13 to v14. Reason and benefit Put the completion procedure next to its native tool. Preserve stock base instructions, schemas, permissions and terminal semantics. This PR now stands alone on master. It contains no reduced manual, shared prompt or operational-skill changes from #14948. ## What Changed - Add canonical native finish and block descriptions. Use them in direct Codex and both native MCP bridges. - Advance the native tool contract to v14. Cover old-v13 refresh without replacing task identity or prior history. - Check authenticated tool catalogs, provider start/resume frames and serialized daemon catalogs. - Add an independent, explicit-only native completion suite. Preserve the original assigned-skill durable-document journey. Pair it with a concrete whole-task blocker across Codex, ACPX Claude and OpenCode. - Verify the actual public production default bundle and budgets before execution. Require independent durable disposition, native result/terminal receipts and observable provider-final ordering. - Correct the blocker browser oracle to accept the requested explanation. Keep exact owner/action/scope checks. Calibrate positive, missing and contradictory replies. - Preserve only actual `tool_call` terminal names (`paperclip_finish` / `paperclip_block`) in the native compatibility run-log projection. Require the same named call ID through its finishing result; retain all other redaction boundaries. - Admit verified hosted shallow checkout/build hydration and bind the selected runnerd to exact source/archive/binary provenance. Hosted cells truthfully reuse the existing trusted build; local admission executes Rust calibration. Forward only public source/run identifiers through both launcher preflight subprocess paths. - Enforce single attempts in the launcher for opted-in fixtures. Keep ordinary retry policy unchanged. Run exact-source, credential-free admission before credential loading. ## Verification - Frozen candidate: `d6e59e4712a3158ab4cd7d58deff1389b4578c21`, based on master `59c07ede72dc08b8aba149a01cc11e0b7a204621`; historical descriptions: `e74ed61a69fbdd8b3a8f15dd6456bc3140246e33`. Exactly the five original native production files and six unit tests differ. Both carry identical corrected fixtures, strict named finishing-call grader, closed compatibility carrier and admission. Defaults, profiles/models/auth/permissions and manifest bytes match. - Actual launcher `prepareNativeCompletionPreflight` → `verifyNativeCompletionPreflight` admission passes on both exact refs with zero providers: candidate 132 / historical 127 selected TypeScript assertions, 128 Node calibrations and one Rust normalization calibration each; E2E typecheck, manifest checks, selected binary provenance and six-cell discovery pass. Each has 257 explicitly skipped unrelated assertions, not coverage. The credential-free environment calibration exercises both real prepare/verify subprocess options with public hosted identifiers and rejects credential/ambient overrides. Complete actual launcher prepare→verify also passes on both frozen refs with explicitly synthetic hosted metadata/verified archives, separately labeled as calibration rather than a trusted GitHub run. Exact framed provenance parsing and mock source identity are calibrated without relaxing the real verifier. - [Complete matched qualification report](https://github.com/paperclipai/paperclip/blob/532066620b88e8731a5211fbe1cbc48ce8c7dd1a/doc/plans/2026-10-02-native-completion-master-qualification.md), [immutable manifest](https://github.com/paperclipai/paperclip/blob/532066620b88e8731a5211fbe1cbc48ce8c7dd1a/doc/plans/2026-10-02-native-completion-calibrated-manifest.json) and [closed retained audit/hashes](https://github.com/paperclipai/paperclip/blob/532066620b88e8731a5211fbe1cbc48ce8c7dd1a/doc/plans/2026-10-02-native-completion-calibrated-results/comparison.json) are inspectable. All six candidate cells pass; historical descriptions pass five. Paired outcomes: zero new failures, one new pass (Codex blocker), five unchanged passes, zero pending pairs. [Candidate campaign](https://github.com/paperclipai/paperclip/actions/runs/37098728980) and [historical campaign](https://github.com/paperclipai/paperclip/actions/runs/37098815696) each execute six original attempt-1 native runs, with no campaign retry and successful cleanup. Their trusted workflow revision is `215586d127e97c9301d86e769a39a15c13298ca2`, separate from measured source. [Candidate public HTML](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37098728980-1/index.html) and [historical public HTML](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/campaigns/gha-37098815696-1/index.html) retain declared screenshots. - Independent candidate evidence agrees with all original grades: 51 strict native checks, 12 served-default/budget checks and 21 original skill/document checks pass. The historical Codex blocker saves the correct whole-task blocker but omits the required marker from its actual provider final and identical saved reply. This is not semantic-summary fallback. Its original browser/matcher failure stays retained; the additional native snapshot/grade and workspace before/after digest were never written and are not fabricated by the separate API/PRP audit. Historical Codex completion has one failed finish followed by success within the same native run; the public receipt records no failure reason. All twelve runs and their usage remain counted. Reported model-cost subtotals are $0.00421482 historical/$0.00437391 candidate; Codex/Claude zero entries have unknown billing type, actual invoices are unverified and hosted execution cost is unmetered. One matched trial supports no extra failure within these six cases, not broad statistical or coding-quality equivalence. - Initial hosted `e18c2cf9` / `459455ac` and subsequent `0a9c5a7` / `00a761b` cohorts each stopped before providers in all twelve cells. The latter failed a mocked-receipt unit test under ambient hosted metadata; all source/build proofs passed. [All twelve later setup receipts](https://github.com/paperclipai/paperclip/blob/402ee94c52273ad58de355ae9a7d562dd22f8101/doc/plans/2026-10-02-native-completion-qualified-hosted-setup.json) are retained. [Exact failed setup receipts](https://github.com/paperclipai/paperclip/blob/27653eb1a8f8ce839776d760f4563f672e5a706c/doc/plans/2026-10-02-native-completion-master-hosted-setup.json) and the original manifest remain intact. Local sandbox-denied loopback and stale anchor-expectation attempts are retained separately; unchanged appropriate assertions were corrected/admitted before paid dispatch. Old anonymous OpenCode streams are not assigned inferred tool names or retroactively passed. - Full provider-free E2E support previously passed 927 tests in 67 files. Exact-head d6 normal CI run `37098409915`, attempt 1 passes full repository typecheck/build/tests, Runner Rust/static checks, all browser shards/aggregate and canary: 52 check-runs pass, four intentional skips, Snyk passes. Fresh Greptile check `111132956342` is 5/5 with zero unresolved threads. Source-specific deterministic tests do not substitute for the bounded live comparison. - Earlier native source `9138f570c341c251a5727c32d6615ce238bc8e03` is archived. Its [complete reduced-manual-context report](https://github.com/paperclipai/paperclip/blob/9138f570c341c251a5727c32d6615ce238bc8e03/doc/plans/2026-10-02-native-completion-live-comparison.md) remains intact, including original failures, grader limits and provider-free replay. It is not current-master-context qualification. ## Risks Changed tool text can change model behavior. The completed six-pair qualification shows no extra failing outcomes in this bounded trial; other tasks and repeated-run variance remain unmeasured. Observable final ordering does not prove provider feedback consumption. Public evidence can fail closed if a provider does not expose the required result sequence. This slice does not remove native fixed prompts or measure general coding quality. No database, schema, permission or legacy completion changes occur. ## Model Used OpenAI Codex, GPT-6 family, with code inspection, execution and tool use. The exact deployment ID and context-window size are not exposed in this session. They are unavailable rather than inferred from the model menu. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-03 06:20:38 -05:00
DottaandPaperclip	862a5758ba	fix(agents): reduce hiring templates to role descriptions (#14985 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - New agents receive role instructions from onboarding, the hiring skill, or a team package. > - These sources repeat harness procedures and impose generic work policies. > - They can crowd out the task and the harness instructions. > - This pull request reduces those sources to short role descriptions. > - It preserves configuration, skills, authentication, reporting lines, and approval controls. > - The benefit is less repeated instruction text with explicit coverage for the default hiring path. ## Linked Issues or Issue Description Refs #3307. The CEO template can impose a fixed delegation route instead of letting the agent choose how to fulfill the request. This change removes that route. It does not implement autonomous goal selection. Related work: #14920 preserves stock Codex base instructions. #14948 reduces the generic manual and shared runtime prompts. #14961 improves native completion-tool descriptions. This PR is separate from those changes. ## What Changed - Select only the short CEO `AGENTS.md` for new default CEO bundles. Keep the three former companion files as compatibility assets. - Reduce the first-agent chief-of-staff prompt and coder, QA, UX, and security role examples. - Reduce seven bundled team role bodies. Preserve their role, reporting, and skill metadata. Regenerate the catalog. - Make hiring examples optional. Replace the long generic role manual with short role drafting guidance. Preserve explicit requester instructions. - Add configuration and import coverage for native and legacy managed bundles, custom instructions, first-agent rendering, and catalog contents. - Add an explicit-only hiring eval that starts from the production CEO default and checks one coder hire, independently computed JSON output, saved instructions, and worker reuse. - Include the full prompt comparison and a separate three-request drafting simulation. Neither is a live provider comparison. Prompt differences: [before and after](doc/plans/2026-10-02-hiring-template-prompt-diff.md). The CEO default falls from 1,897 to 20 words. The coder example falls from 652 to 18 words. Word counts describe instruction size, not outcome quality or billing. ## Verification - PASS: 99 focused server tests and eight shipped-catalog tests. - PASS: catalog generation and validation for four shipped teams. - PASS: hiring skill validation. - PASS: `pnpm -r typecheck`. - PASS: `pnpm build`. - INCOMPLETE: the full local `pnpm test:run` was stopped before rebase. Its original log is retained. This is not a completed full-suite pass. The full current-head GitHub CI workflow passed: https://github.com/paperclipai/paperclip/actions/runs/37073372419. - PASS: `pnpm test:e2e:runner:typecheck` and `pnpm test:e2e:runner:unit` (63 files / 843 tests). - PASS: discovery for the two new hiring cells, 50 existing everyday cells, and the full 438-cell catalog. - PASS after rebase: 99 server tests, 11 catalog tests, 62 selected E2E support tests, and the E2E typecheck. - PASS: all current-head PR checks at `57dcee147ed0b2d2e3cc657cd9e50fb16bf9ec25`: 51 successful check runs, two intentional Storybook skips, and successful Snyk status. Fresh Greptile is 5/5 with zero unresolved threads. - PENDING follow-up: matched live hiring runs on frozen integration refs. No live outcome-quality or non-regression result is claimed from the configuration checks or this merge. The new suite has two local native cells: Codex and ACPX Claude. It expects five provider turns per cell. It compares source-derived bundles, so the historical long templates remain admissible. Missing successful source-read receipts make a pair uncomparable. They do not establish a behavior regression or equivalence. ## Risks - New default roles have fewer prescribed procedures. Live checks must determine whether a removed instruction was needed for an outcome. - Existing custom and saved bundles keep their contents. The retained companion assets avoid a source-file compatibility break. - Specialized Summarizer, Reflection Coach, and Wiki Maintainer prompts remain unchanged. Their product contracts need separate review. - The generic non-CEO fallback reduction is in #14948. This PR alone does not provide its eight-word fallback. - Configuration tests and drafting simulations do not establish live outcome quality. QA, UX, security, and chief-of-staff hiring behavior remain outside the new two-cell comparison. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, code editing, shell tools, and delegated verification. The runtime does not expose the exact deployment model ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-02 17:56:42 -05:00
DottaandPaperclip	b54b2dc35c	fix: preserve warm Codex turns with incremental managed file checkpoints (#14735 ) ## Thinking Path > - Paperclip manages AI agents and keeps their instructions and files durable. > - Native Codex runners can keep a process alive between compatible turns. > - Managed file collection stopped that process after each turn, which defeated warm reuse. > - Agent folders can contain large images and other files, so full copies on every turn are expensive. > - This change keeps one managed directory for the live session and saves only file changes after each turn. > - Ownership, authorization, instruction changes, and process retirement still control when reuse is safe. ## Linked Issues or Issue Description Related: #13710 introduced native warm session reuse. This fixes managed file collection that still forced those sessions to stop. No duplicate open PR or issue was found. What happened? With managed instructions and warm native Codex enabled, consecutive turns reused a Daytona sandbox but started a new runner process each time. The managed directory collector required process termination before saving files. Expected behavior Compatible turns keep the same process and managed `AGENT_HOME`. Each completed turn saves added, changed, and deleted files before the next turn starts. Unchanged large files do not transfer again. Steps to reproduce 1. Use a native Codex agent with managed instructions and a reusable Daytona environment. 2. Enable warm session reuse and run three turns on the same task. 3. Write a large binary on the first turn, edit a small note on each turn, and delete a file on the second turn. 4. Compare process identity across turns and read the canonical files through the public agent-files API. Paperclip version or commit Reproduced on `d30b03bd8c17604cdab1533eeeeb087aba30e8b1`. Deployment mode Local server with remote Daytona execution; cloud native runner uses the same path. ## What Changed - Retain the managed directory only for the verified owner of a live native Codex session. - Checkpoint each completed turn before releasing the session for reuse. Retry unstable captures, then stop and collect when a warm checkpoint cannot be validated. - Compare metadata and cached hashes, stream only changed file payloads, record deletions, and validate path, content, quota, and authorization before saving. - Rotate sessions when canonical files, loaded instructions, credentials, or launch policy change. Fence stale collection and cleanup callbacks from later owners. - Keep cleanup and recovery aware of the current session owner. Recheck canonical files under the writer lock at handoff, attach the successor collector before fallible bookkeeping, and emit one final save receipt on checkpoint fallback. Preserve storage warnings across unchanged checkpoints. - Add regression coverage and a three-turn Daytona test with independent public API file checks, an unchanged 8 MiB binary, deletion checks, and strict process identity checks. - Document checkpoint consistency, lifecycle behavior, and local run-log counters. - Replace a timing assumption in the Daytona teardown test with explicit transfer-arrival gates after CI exposed an unset release callback. ## Verification - Full local `pnpm -r typecheck` and `pnpm build` passed. Server checks were repeated after the final storage-warning fix. - Runner E2E typecheck and 749 runner E2E unit tests passed. - Focused file checkpoint, directory ownership, instruction collection, native session, and merge tests passed. After review fixes, the managed-directory and native-session suites passed 550 tests, including intervening canonical edits, same-run fresh restore, failed handoff collection, and one-call fallback collection. Server typecheck passed again. The Daytona plugin suite passed 218 tests. The quota-warning regression failed before the fix and passed afterward. - Three real Daytona campaigns passed before the final handoff review fixes. The latest kept PID 547 across all three turns. The first checkpoint copied 8,388,635 bytes; the next two copied 36 and 54 bytes. Public API reads verified the binary, note contents, and deletion after every turn. Test cleanup deleted the sandbox. - The final head was also deployed to an isolated cloud staging instance and passed three UI-triggered native Codex turns with managed instructions. All three retained the same process ID/start time, native session, provider session, runner instance, and Daytona sandbox. Checkpoints copied 8,388,643 bytes on turn 1, then only 52 and 78 bytes on turns 2 and 3; those warm captures also hashed only 52 and 78 bytes. Independent canonical API reads verified every byte of the unchanged 8 MiB binary and the exact note contents after every turn; the deleted file returned 404 after turns 2 and 3. After restoring the original lifecycle and agent-auth configuration, removing the temporary secret, pausing the test agent, and deleting both test sandboxes, independent canonical API reads still verified the entire binary, the final 78-byte three-line note, and the deletion. The native runner flag remained enabled and the final serving revision remained the PR head. - Two earlier staging attempts are preserved as failures and are excluded from the acceptance result: a saved ChatGPT login failed with a provider routing 401, and its subsequent stopped-sandbox retry failed before provider startup with a closed-lease admission error. The successful campaign used a fresh sandbox and a temporary encrypted API-key binding. The stopped-lease retry remains unexplained; this campaign does not establish recovery of that failed sandbox. - All [Paperclip CI gates](https://github.com/paperclipai/paperclip/actions/runs/36750397355) pass on `26ef2ef56a389259246809805c0b34a4747eb86b`, including full test partitions, build, typecheck, runner verification, E2E shards, and the Canary clean public-npm install. Greptile reviewed that exact head at 5/5 with no unresolved review threads or outstanding findings. - Full local repository coverage used the existing CI partitions, but the 40,000-file Git streaming stress test timed out and its local retry was interrupted by macOS thermal emergency sleep; this is not a green full local suite claim. The exact stress test passed on the final head in [CI server shard 2/12](https://github.com/paperclipai/paperclip/actions/runs/36750397355/job/110008294290), in 111.9 seconds. - Repeat the live test with configured credentials and a Linux runner artifact: `pnpm test:e2e:runner -- --id daytona-warm-continuity.runner-codex.daytona.warm-three-turn`. ## Risks - This is a file-level checkpoint, not an atomic snapshot of the whole folder. Background writes after a capture are saved by the next checkpoint or final stopped collection. - Metadata scans still visit all paths. Modified files transfer in full; unchanged files do not rehash or transfer. - Incorrect ownership or reuse could collect the wrong directory. Run ownership fences, current authorization, stable capture validation, and stopped collection fallbacks are covered by tests. - Warm reuse remains opt-in. No database migration or fleet default changes. ## Model Used OpenAI GPT-6 through Codex, with reasoning, code editing, tool use, and test execution. The exact serving model ID and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-30 13:33:37 -05:00
DottaandPaperclip	fc36d1ccd3	test(e2e): qualify large Daytona Git workspace continuation (#14316 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A task can keep one agent process and workspace across human review turns. > - Large workspaces must survive each transfer between Daytona and the host. > - Small fixtures do not cross the previous 32 MiB filename-output limit. > - Correct files alone do not prove that finalization and recovery have settled. > - This pull request adds an explicit three-turn test with 60,000 files and independent host checks. > - The test detects lost files, replaced processes, and stale finalization retries. ## Linked Issues or Issue Description Refs: #14253, #14314, #14315, #14402, #14420. This test covers the workspace Git streaming fix in #14253 and the Daytona archive validation and native finalization fixes consolidated from #14315 and #14314 into #14402. These runtime changes are merged into master. This PR adds regression coverage. ## What Changed - Add one explicit-only native Codex Daytona test. Broad matrix runs do not select it. - Create 60,000 small untracked files through ordinary provider execution. Independently check all host file contents and the 39,828,890-byte filename manifest after each turn. Each continuation changes all generated file contents, so stale host copies fail. - Check spaces, newlines, leading hyphens, Unicode, and glob characters in filenames. - Require committed native finalization, successful workspace receipts, no active transfer or runless cleanup, and no scheduled recovery before each continuation and after the final turn. - Use a fixed external instruction bundle so the same runner PID and process identity can continue across the three browser-driven turns. Managed agent folders intentionally stop the process for file collection after #14420. Set the 20-minute idle window on the environment, then verify the admitted policy on every run. Set a 25-minute Daytona auto-stop window for this large-file test. - Save public workspace-operation evidence when an E2E attempt fails. - Bound this large-file fixture to 15 minutes per turn and 50 minutes total, reserving five minutes outside the turns for setup, host verification, and cleanup. A measured CI continuation succeeded in 11 minutes 5 seconds, exceeding the ordinary warm fixture's 10-minute deadline. The ordinary fixture and all file, process, finalization, and cleanup assertions remain unchanged. ## Verification - Final PR head: `b7eed5517426507f5d7912edb8ae49163d1783b0`. - `pnpm test:e2e:runner:unit` — 56 files and 708 tests passed locally. The catalog retains all 407 existing cells and adds this one explicit-only cell. - `pnpm test:e2e:runner:typecheck` — passed locally. - [Final-head CI](https://github.com/paperclipai/paperclip/actions/runs/36640920233) — all gates passed, including repository typechecks, tests, build, browser suites, and canary dry run. The PR has 54 successful checks and two expected skips. Fresh Greptile reviewed all seven files at 5/5; all review threads are resolved. - [Live single-cell Daytona verification](https://github.com/paperclipai/paperclip/actions/runs/36637283902) — passed on the first attempt in 1,822,299 ms (30m 22s) on `227074b7b`, using native Codex `gpt-5.6-sol` and the verified Daytona image. All three turns independently verified every one of the 60,000 file contents, 39,828,890 filename bytes, and five unusual names. All runs committed with one stable runner PID/process fingerprint, native/provider sessions, runner instance, and sandbox. Final browser/download assertions, all nine matchers, explicit cleanup, and report publication passed. Results are published through the [Product E2E history](https://d1p6rlowie26tp.cloudfront.net/runner-e2e/) and [eval hub](https://pages.paperclip.ing/evals/). - The final `b7eed5517` follow-up only extends the total allowance from 45 to 50 minutes, updates its catalog assertion/version, and documents the setup/cleanup margin. Per-turn limits and behavioral assertions are unchanged from the successful live run; that paid run was not repeated for this allowance-only follow-up. - The [first integrated-head live attempt](https://github.com/paperclipai/paperclip/actions/runs/36632127364) is retained: turn 1 passed, then turn 2 hit the old 10-minute deadline while finalizing. Its trace records successful completion after 11m 5s and successful cleanup. Process diagnostics also showed the intentional managed agent-folder stop boundary introduced in #14420. These observations motivated the larger turn budget and fixed external instructions. - The broad local `pnpm test:run` began alongside the build and encountered server setup and port-test failures. The affected setup suites and assertions passed on focused reruns after the build, using canonical macOS temporary paths where needed. The redundant broad local run was stopped; full-suite success is established by final-head CI, not by that local run. ## Risks - The live test makes provider calls and creates a billable Daytona sandbox. It runs only when explicitly selected and deletes its sandbox during cleanup. - The test creates 60,000 files and can take several minutes per transfer. Its longer retention window applies only to this fixture. - The test will fail on a runtime that does not include all three required fixes. ## Model Used OpenAI GPT-6 in Codex assisted with code, terminal tools, and test analysis. The exact serving model suffix and context-window size were not exposed to this task. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-29 17:49:36 -05:00