2 Commits

Include renames

Author	SHA1	Message	Date
Devin FoleyandPaperclip	1aa0c36397	Fix execution after onboarding plan approval (#19698) ... ## Thinking Path > - Paperclip manages agents and their work. > - First-task onboarding turns a user's goal into an approved proposal. > - The interview offers a plan and a team to carry out that plan. > - The instructions leave execution after acceptance unclear. > - An agent can add a plan-only approval boundary and finish without doing the requested work. > - This change makes the approval scope explicit and tests both execution and plan-only outcomes. ## Linked Issues or Issue Description **What happened?** An onboarding agent can treat an interview about building a product as a request for a plan only. It can then retain the approved plan and close onboarding without creating execution tasks. The interview evaluation previously accepted that result. **Expected behavior** Acceptance of an execution proposal starts its selected work. An explicit plan-only request stops after planning. User-requested separate approvals and existing permission and budget controls still apply. **Steps to reproduce** 1. Choose the onboarding interview and describe a deliverable. 2. Accept the saved plan proposal. 3. Check that the agent starts the approved work instead of closing with only a plan. ## What Changed - Clarify execution intent, approval wording, continuation, and completion in the first-task prompt. - Require assigned child work and saved output in the interview acceptance evaluation. Use checkbox acceptance in the regular interview journey and retain conversational acceptance in the approval-reply suite. - Add an explicit plan-only acceptance journey and nine grader regression checks. - Update evaluation counts, grading versions, and onboarding verification guidance. ## Verification - Passed: all applicable CI checks on `198a6b47cac7`, including typechecks, the general and serialized server tests, Runner checks, build, and browser E2E shards. [CI run](https://github.com/paperclipai/paperclip/actions/runs/42324855376). - Passed locally: the full Runner support suite, 6,860 tests across 323 files (three skipped), plus 13 onboarding asset tests. - Passed locally: the full workspace build, workspace package typechecks (server rechecked after restoring excluded-plugin SDK links), and the Runner evaluation TypeScript check. - Passed: map validation and whitespace checks. - Initial broad local checks failed during checkout preparation because PostgreSQL library symlinks and runtime/SDK assets were missing. Restoring the dependency setup made the affected controls pass. The full root test suite passed in CI; it was not rerun locally after that repair. - Live model journeys have not been run. Grader tests do not prove model compliance. ## Risks - Prompt behavior needs live qualification across providers. This change adds no server-side enforcement or permission bypass. - The new plan-only case adds four local evaluation cells. A full campaign now has 56 first-task cells. - Existing proposals that reserve later approval keep that restriction. This change does not rewrite tenant data or start existing onboarding tasks. - No schema migration or deployment is included. ## Model Used - OpenAI `gpt-6.1-sol`, xhigh reasoning effort, with repository tools and local test execution. Context window size is not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused and full Runner support locally; full root coverage passed in CI) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-06 17:56:27 -08:00
DottaandPaperclip	7c66b6b1f6	test(onboarding): cover plan, hiring, and execution approval history (#19069) ... ## Thinking Path > - Paperclip manages agent work and human decisions. > - Onboarding can approve a plan, a hire, and an execution step in separate turns. > - Tests covered single approvals more closely than that full history. > - The repaired gate must retain valid plan approval through later decisions. > - Every consumer must recognize the saved confirmation kind and selected scope. > - This change tests that contract and registers a complete interview-to-execution journey. ## Linked Issues or Issue Description Refs #19043, #17914, #18283. The earlier tests admitted one accepted proposal at a time. Decomposition fixtures used standard confirmations. The direct-task E2E path forbids hiring. Those cases did not cover an accepted checkbox plan followed by hiring and a separate untargeted execution approval. ## What Changed - Add eight PostgreSQL contract cases for standard and checkbox plan approvals. Native tools create the cards. An authenticated human accepts them through HTTP. A later native run reads the approved revision and actual selected scope. - Exercise native task creation, generic REST creation, child REST creation, and accepted-plan decomposition. Check the assigned child and one creation audit event. Check durable replay on the three entry points that support it. A pending execution confirmation still prevents creation. - Register the explicit-only `onboarding-approval-history` suite for local native Codex and ACPX Claude. The browser chooses the interview, selects only drafting in the saved plan, approves one writer hire, and approves a separate Start drafting confirmation. The model must hire the writer and use that writer's own child execution to save the welcome document. - Grade each human decision, the unchanged plan revision and selection, approval timing, one assigned child and worker execution, the exact writer-authored saved revision, no pending requests, and persistence after reload. Calibrate the oracle against missing, forged, premature, duplicate, and changed outcomes. - Preserve synthetic option choices, document/run bindings, and saved revision authorship in the existing public first-task evidence projection. Keep private provider/account data excluded. Document the workflow, evidence, billing, and cleanup bounds. - Align the approval-history fixture with its completion oracle: the onboarding parent must be Done. Test both native profiles against graded completion evidence and reject a completed child whose parent remains In review. Count parent heartbeats separately from the writer: accept one to seven parent records while retaining the twelve-company-run cap and the independent three-approval/one-writer oracle. - Retain the existing 52-cell first-task suite. The new suite has one attempt, seven expected work turns, a twelve-run maximum, a fifteen-minute deadline, and the existing $5 company task budget. It is excluded from `--all`. - Update two stale cleanup assertions to include the existing thirty-second deletion timeout. ## Verification - Red control: temporarily restore the pre-fix admission module and run the new checkbox/native history case. It fails with the expected `First-task proposal requires human acceptance` error after plan, hiring, and execution acceptance. Restore the repaired module after the probe. - The final post-rebase history matrix and existing onboarding safety suite passed together: 74 tests, no skips. - Final focused Runner E2E support passed: 413 Vitest tests and all 370 Node support checks. This covers catalog discovery, journey controls, wrong-outcome calibration, public evidence, and existing first-task behavior. - Review controls: four authorship cases fail before the revision check. A delayed browser-response control fails before the approval-time fix. Both pass after repair. The final oracle requires the exact current document revision, writer ID, native child run ID, matching body, and readback after reload. - Credential-free discovery lists both new native cells. No provider call was made. `OPENAI_API_KEY` and `ANTHROPIC_API_KEY` are absent, so the real provider journey remains unqualified. - The earlier broad credential-free E2E support run reported 6,728 passing cases, three skipped cases, and nine failures in three unrelated files. All nine reproduce with the changed E2E source restored to the unchanged baseline: three expired overhead-fixture cases, one stale Pi profile-version assertion, and five workflow security assertions. This change does not alter those files, paid workflow policy, or provider pins. - Previous full workspace typecheck/build and [Linux CI](https://github.com/paperclipai/paperclip/actions/runs/41921754638) passed on `4f431f5aae99d072cc75027e90d7b847b4c342ca`, including the new eight-case PostgreSQL file. Ponytail subsequently found a fixture/oracle terminal-status mismatch. Both native-cell controls fail on that fixture (`done` observed versus `in_review` expected), while the parent-in-review negative control passes. A second comparison uses the actual `matchesRunCount` predicate: graded evidence with six parent runs and one writer run fails the old exact-seven-parent requirement. Both fixes pass these controls, all 413 focused support tests, all 370 Node checks, and the owning E2E project typecheck. [Fresh CI](https://github.com/paperclipai/paperclip/actions/runs/41936699686) passed on `91c980b110252da451b92eda597c168517de1276`: 48 successful jobs and one expected skip. Greptile reviewed this head at 5/5 with no unresolved findings. Ponytail's terminal-status finding is fixed and its thread is resolved; its current-head re-review remains queued. Live provider cells remain unqualified. ## Risks The production API and approval gate are unchanged. The new paid journey can expose model or product failures at an intermediate approval boundary. It retains those failures and uses the existing diagnostics and cleanup. Registration and deterministic tests do not establish a live provider pass. The unrelated baseline support failures remain explicit. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, and code execution. The exact serving model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused tests; unrelated baseline failures are recorded above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-11-04 21:46:55 -06:00