2 Commits

Include renames

Author	SHA1	Message	Date
3 people	507a0d7098	test: verify repository delivery after real runner conversion (#17089) ... ## Thinking Path > - Paperclip manages agents that perform useful work. > - Repository setup and Runner selection must preserve that work context. > - A successful provider turn cannot prove configuration retention or usable delivery. > - The repaired setup chain needs independent product checks with the recorded models. > - This PR adds explicit repository and conversion journeys to the existing Product E2E harness. > - The observer checks source bytes, Git identity, saved configuration, and downloaded output. > - Development and staging qualification remain separate. ## Linked Issues or Issue Description Refs #17088 and #17047. The complete repository setup repair in #17088 is merged. No competing conversion journey was found in the open PR search. **What happened?** Existing checks do not prove that a real legacy agent retains its configured context when switched to Paperclip Runner and then delivers verified repository work. The projectless repair check also uses different default models from the recorded staging models. **Expected behavior** A configured repository task delivers correct, accessible output. A real legacy-to-native conversion preserves configuration, instructions, environment, and equivalent permission policy before doing subsequent work. A projectless task continues through setup without supervision. **Steps to reproduce** List the configured-repository, legacy-conversion, and automatic-repository-configured-models suites. Run one exact ID from the README. These are explicit paid development checks, excluded from the default selection. ## What Changed - Add four configured-repository cells and four real legacy-conversion cells for Codex gpt-6.1-sol and Claude claude-opus-5-5 on local and Daytona environments. - Seed a fresh reference only in an owned disposable Git checkout. Compare the delivered JSON with its original bytes, hash, and commit. - Switch the same real legacy agent through the public Runner selection API. Compare allowlisted configuration and instruction digests before and after conversion. - Require one committed native run with the configured provider, model, and environment, a visible registered download after reload, Done, and no pending question. - Add four projectless replay cells with the recorded staging models and the unchanged strict continuation oracle. - Use production hire instructions and existing sandbox profiles, budgets, report pipeline, and cleanup. Record API-key and per-turn lifecycle differences explicitly. - Retain the required allowlisted API snapshot after successful automatic setup. Preserve lifecycle IDs, state, ordering, and receipt references without raw provider content. Verify both admissions retain the configured provider and model. ## Verification - Product E2E support after integrating the setup repair: 3,353 passed, two existing skips; source prerequisites: 128 passed. - Product E2E typecheck passes. Discovery lists all twelve new explicit cells. - Independent oracle calibration rejects wrong configuration, instructions, provider, model, source bytes, hash, commit, downloaded output, delivery, and pending questions. - The first full support run exposed a catalog count mismatch and an unprovisioned new profile ID. Both are fixed; the successful run uses the existing trusted sandbox profile IDs without changing workflow authority. - Review found fixture prerequisite defects: optional CLI-only Codex arguments and platform qualification after Runner selection. Both are corrected; the disposable Claude baseline also excludes its unsupported optional CLI turn limit. Seven new setup checks cover probe ordering, absent qualification, immutable images, local conversion, and uncertain probe accounting. The final focused cohort passes 86 checks and E2E typecheck passes. - The first qualification-regression run also had an incorrect test assumption about the legacy timeout; the corrected red reaches the unsupported extraArgs setting. No production conversion restriction was relaxed. - A retained four-cell development campaign delivered independently verified ordinary Codex output and real Codex/Claude conversion results with retained configuration. All three attempts nevertheless failed the required evidence contract because api-state.json was missing. The flow now saves allowlisted task/run lifecycle fields; it does not export prompts, comments, interactions, configuration contents, or provider outputs. The projectless cell delivered valid bytes but used a monitor fallback and failed the unchanged continuation oracle; its driver correction belongs to #17088. All four attempts remain failures; fresh exact-candidate acceptance is pending. These API-key/per-turn checks do not establish subscription authentication, warm reuse, Slack delivery, or the complete milestone. - Four fresh local ordinary-repository and conversion cells passed at `c4e409d819cc0a703736d3f1ed3552064ca690b8`. Independent downloaded artifacts and saved configuration evidence agree. - The repaired automatic setup chain passed all six journey checks for both models, but first failed because the success path omitted api-state.json. The separate capture repair now passes the fresh two-cell [campaign](https://github.com/paperclipai/paperclip/actions/runs/39633381139) at `7a7c26f6688ec6b0e0454562a43fd48ab1a81c97`. Both required API files and their run/intent/dispatch bindings were independently checked. Earlier failures remain failures. - After merging current master, 114 workspace/admission/continuation checks across six suites pass. The provider/model and evidence privacy cohort passes 65 checks, and E2E typecheck passes. The final model-binding assertion has unit calibration; fresh actual-model acceptance of that assertion is still pending. - Exact-head CI/review and complete staging acceptance remain pending. Local development passes do not establish the 24-cell milestone. ## Risks The real provider may expose additional conversion or setup defects. Failures remain evidence and cannot be replaced by catalog registration or a later retry. The conversion uses the production API and does not inject a native result. Only synthetic data is seeded in a resolved fixture-owned directory. Configuration evidence contains digests, not raw environment values. This PR changes evaluation code, not production behavior. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing>	2026-10-21 11:51:50 -05:00
3 people	eece07a747	fix: repair the complete projectless repository setup chain (#17088) ... ## Thinking Path > - Paperclip lets people manage agents that do useful work. > - A projectless task can prepare an authorized repository in its retained workspace. > - Setup must hand work to a later run without user supervision. > - Real task attempts exposed gaps across admission, completion, and queue dispatch. > - A successful provider exit did not prove that the requested result was delivered. > - This PR repairs the existing lifecycle and checks the complete repository journey. ## Linked Issues or Issue Description Refs #17047. This PR includes the preserved repairs from #17049, #17052, and #17082. **What happened?** A projectless native task could stop after requesting a repository. It could also continue through a timer or generic recovery path and lose its saved conversation lineage. The latest real Codex and Claude attempts delivered correct files, but the native continuation had been folded into the finishing predecessor. Both attempts remain failed measurements. **Expected behavior** Prepare the authorized repository, retain the original context, and dispatch one saved continuation after its predecessor finishes. Deliver the verified result in the originating conversation. Stop, current access, approval, and real blocker rules remain authoritative. **Steps to reproduce** Run the explicit automatic-repository suite for native Codex or Claude. Inspect the downloaded result, original context artifact, both committed run records, and continuation receipts. The suite uses one attempt and does not widen the default paid selection. ## What Changed - Resolve the retained local-board identity through current company membership and workspace authority. - Prepare required source dependencies before the isolated server imports native tool authority. - Explain the saved repository continuation in production completion tools. Correct only the exact pending-checkout blocker backed by a live receipt. Reject an unfinished response wait without a real response source. - Preserve the persisted task-session workspace source and its explicit absence of root repository context. Permit Git's exact no-repository result only for a validated plain task directory. Keep other Git errors and linked metadata strict. - Revalidate server-resolved same-agent continuation values against current ownership and the saved kind, summary, and key. Carry this shape through the internal bridge schema. Keep provider-facing schemas restrictive. - Preserve terminal result authority after a correlated successful controller response. Controller rejection creates no result authority. Bind the shared guidance and transport source in Copilot v55 and preserve historical identities. - Admit native status wakes with the existing no-coalescing contract. The existing deferred queue retains a distinct future dispatch instead of retagging the finishing run. - Recover deferred conversation lineage through the actual admitted dispatch. Accept a null outbox run binding only with matching decision, effect, dispatch actor, idempotency, task, company, agent, and requester evidence. Reject a conflicting nonnull binding. - Independently grade retained random context, actual checkout bytes and HEAD, accessible output, distinct continuation receipts, and admission order. ## Verification - Real database and Git regressions fail before their repairs. Prior focused cohorts cover local-board authority, optional repository waits, plain task directories, and first/reused workspace provenance. - The new valid decision/effect-backed assembled-heartbeat regression reproduces coalescing into its active predecessor before the queue repair. With the repair, it dispatches one successor after the predecessor finishes. Duplicate sweeps create no extra run. - The deferred-outbox conversation regression fails before its repair. The repaired test also rejects a conflicting predecessor binding. - The current queue, continuation, publication, cancellation, stale-admission, and domain cohort passes 344 checks across 11 suites. Server and Product E2E typechecks pass. The strengthened automatic-repository oracle passes 34 calibration checks. - Three omitted/null/resolved forms pass through the compiled Rust runner and controlled provider. The prior shared bridge cohort passes 492 TypeScript checks and 399 Rust core checks. Runner source identity, TypeScript, Rust formatting, and locked Rust typechecks pass. - Six real local development cells passed ordinary repository, conversion, and necessary-question journeys. Independent artifacts and rendered question/result evidence agree. They do not qualify staging or Slack. - Both latest automatic-repository attempts delivered correct files but failed the strict continuation check. They remain failures. A fresh replay of this queue repair is pending. - On the combined candidate `380047f4b923807a8c4f7ef6016e4fe130ab4a3a`, `pnpm -r typecheck` and `pnpm build` pass. Product E2E support passes 3,525 checks with two existing skips, plus 128 source checks. The complete local `pnpm test:run` is running and is not yet a pass. - The first replay selection failed catalog validation due to an operator profile-ID error. A second selection used the wrong default-model suite and was cancelled before any provider job existed. Both setup attempts remain recorded. The corrected campaign selects the recorded GPT-6.1 and Opus 5.5 models plus the local Stop/resume journey. - Exact-head CI, review, and deployment acceptance remain pending. GitHub reports delayed PR updates and webhook delivery; the current head and checks have now propagated. No earlier-head green badge establishes acceptance of this commit. ## Risks This changes several boundaries of repository setup. Full product and deployment acceptance remain required. The queue uses the existing deferred admission and delivery contracts. Batch deduplication remains in place. The provenance repair requires the actual bound dispatch and does not grant publication authority. Current authorization is still checked at publication. No accepted provider result is rewritten, and completed provider work is not rerun to repair persistence or delivery. ## Model Used OpenAI Codex, GPT-6 family, with repository tools and code execution. The runtime does not expose its exact deployment ID or context-window size. Those values are not inferred. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Paperclip <paperclip@paperclip.ing>	2026-10-21 11:21:55 -05:00