2 Commits

Include renames

Author	SHA1	Message	Date
DottaandClaude Opus 5.5	a12fcb8f26	fix(runner-e2e): find saved slugify source in a workspace project folder (#19471) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The runner E2E everyday `recover-controller` case proves that a task survives a control-plane restart. > - The case waits for an active run with a saved `slugify.py` before it kills the controller. > - The harness looked for `slugify.py` only at the workspace root. > - The task brief asks for `slugify.py` "in the project workspace". It does not ask for the root. ACPX Claude saved it in `slugify-tool/slugify.py`. > - The harness timed out before the restart, so the case did not test restart recovery at all. > - This pull request makes the harness find the saved source in a bounded set of project folders. > - The benefit is that the case reaches its real fault boundary for agents that make a project folder. ## Linked Issues or Issue Description **What happened?** The local cell `everyday-workflows.runner-acpx-claude.local.recover-controller` failed with "Timed out waiting for active run with saved source". The agent finished the task and delivered a correct ZIP. It saved `slugify.py` in `<workspace>/slugify-tool/`. The harness read only `<workspace>/slugify.py`, so it never saw the saved source. It also recorded `saved-source-unavailable-at-final-capture`. The controller restart never happened. **Expected behavior** The harness finds the source that the agent saved in the project workspace. Then it kills the controller while the run is active and grades the recovery. **Steps to reproduce** 1. Run `pnpm test:e2e:runner -- --max-automatic-retries 0 --id everyday-workflows.runner-acpx-claude.local.recover-controller` with a provider that saves the project in a subfolder. 2. See the timeout before the `controller-restarted` timeline entry. **Paperclip version or commit** `master` at `6cafcaf89b` (also reproduced at `d87f6fa4c2`). ## What Changed - Added `readWorkspaceSource` in `tests/runner-e2e/everyday-interruption.ts`. It reads `slugify.py` at the workspace root first. Then it searches subfolders to depth 2, shallowest and alphabetical first. It skips hidden folders, `node_modules`, `__pycache__` and `venv`. - `recordSource` (the stop-redirect and recover-controller boundaries) and the final local source capture in `tests/runner-e2e/everyday-flow.ts` use this helper. - The stop-redirect workspace snapshot (`workspaceFiles`) now includes the same bounded project folders. A stop check therefore still sees edits when the project is below the root. - Added `tests/runner-e2e/everyday-interruption.test.ts`. ## Verification - Red before the fix: `npx vitest run --config tests/runner-e2e/vitest.config.ts everyday-interruption` failed 2 of 4 tests (the nested-folder cases returned `undefined`). - Green after the fix: the same command passes 4 of 4. - `pnpm test:e2e:runner:typecheck` passes. - Live rerun of the cell after the fix (local, native ACPX Claude, `claude-sonnet-5`): the harness observed the active run with saved source, submitted the follow-up, and restarted the controller (`source-saved-before-interruption` with `active: true`, then `controller-restarted`). The original timeout did not occur. - Two valid live reruns after the fix both reached the restart. Both still did not pass. After the restart, the interrupted ACPX turn ended as `native_provider_terminal_failed: Provider turn failed`. In the second rerun the task stayed Blocked with the follow-up comment still queued and a Retry offered. This is a separate product signature. It is not in the scope of this harness fix. A follow-up handles it (see Risks). - One earlier rerun failed before the task started because the new worktree had no built runner binary (`paperclip_runner_binary_missing`). That was local setup only. The runner was then built with `pnpm --filter @paperclipai/paperclip-runner build`. ## Risks - Low risk. The change is test-harness only. - If a workspace has several `slugify.py` files, the harness picks the root copy, else the shallowest copy. A stale copy in another folder could be chosen. The search depth and skip list keep this bounded. - Known remaining failure in the same cell (separate signature): after a hard controller restart, the ACPX runner restore path (`acpx_provider_backend.rs`, `restore_session_if_needed`) closes the in-flight turn with `code: acpx_active_turn_recovery_closed` and no `error` object. The run shows the generic "Provider turn failed". The Codex fix in #15297 gives Codex a recoverable `provider_turn_lost_on_restore` cause and a same-conversation continuation. ACPX does not have this yet. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (Anthropic), model ID `claude-opus-5-5`, used through Claude Code with tool use (shell, file edit) for diagnosis, the regression test, the fix, and live reruns. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (not applicable: test-harness-only change) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-11-06 09:24:33 -06:00
3 people	3790ca2f13	fix(runner): repair approval and Stop races and eval infrastructure (#13750) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Runner tasks must continue after approval and stop when the user presses Stop. > - Live evals found races at approval delivery and provider startup. > - Browser readiness and CI setup errors also hid the actual task results. > - This pull request fixes those races and the related test infrastructure. > - Regression tests and saved live reports show which cases now pass. ## Linked Issues or Issue Description Companion eval definitions PR: https://github.com/paperclipai/paperclip-evals/pull/25 (AgentCore paused and provider/environment infrastructure). Related: #13741 now supplies the late-startup Stop fence and warm-attachment recovery; this PR retains that fence and extends startup tracking and regression coverage to both native backend paths. #13539 introduced queued approvals during active runs. #13738 fixes child assignment, task replies, and warm process continuity and is already in the base. #13291 concerns automatic continuation of interrupted legacy sandbox runs; this PR fixes native startup cancellation and does not change that recovery policy. **What happened?** An accepted service approval could wait after its source run stopped. Stop could return success before the provider handle existed. Work could then start after Stop, or a cancelled run could be recorded as failed. Some E2E tests also failed on unloaded browser content or irrelevant reply wording. Runner CI could fail before model work because of dependency or sandbox setup. **Expected behavior** Deliver each settled approval once after its source run stops. Do not start work after an acknowledged Stop. Preserve the audited cancellation. Test the intended product behavior with a ready browser and verified runtime dependencies. **Steps to reproduce** 1. Approve a service request while its source run is active. Let the run finish. Check that its result starts one continuation. 2. Delay provider startup. Press Stop before its handle is available. Check cancellation, then submit `/new`. 3. Run the browser, warm-workspace, and Stop-and-redirect cases from the linked report. **Paperclip version or commit** The branch includes master at `9d19f98b5`. The report records the original source for each focused attempt. **Deployment mode** Isolated local development instances and disposable Daytona sandboxes. ## What Changed - Deliver settled tool-action results for the exact company and source run during final cleanup. Keep the existing idempotent receipt and periodic recovery sweep. - Wait for startup to hand off its provider handle before acknowledging Stop. Reject first-turn admission after cancellation. Preserve a matching audited pending or acknowledged cancellation. - Wait for mounted task history and connector controls in browser tests. Record failure evidence. Grade workspace contents and process continuity separately from exact reply wording. Require each warm-turn marker once and in order, allowing surrounding prose. - Stop-and-redirect now checks that the source file exists and work is active before Stop. - Resolve target dependency locks in an uncredentialed CI job. Verify the lock artifact hash. Keep orchestration and publication on the trusted workflow revision. - Materialize the pinned OpenCode executable and configure the exact Codex executable's user-namespace profile before provider credentials are available. - Compress Daytona directory uploads with gzip. Preserve files, executable modes, symlinks, empty directories, and confinement checks. - Classify file-transfer RPC deadlines as infrastructure. Keep unrelated runner RPC failures visible. ## Verification - [Focused live report with screenshots and original attempts](https://pages.paperclip.ing/runner-reliability-20260921/): 14 of 15 selected Product E2E cases pass across the recorded revisions. Claude and Codex Stop → `/new`, Claude service approval, delegation, both hiring/reuse cases, and native Daytona warm continuity pass. - Two credentialed Runner smoke cases pass. These are not full protocol coverage. - E2E harness after the master merge: 429 tests pass. E2E and server TypeScript checks pass. - Daytona plugin: 239 tests pass, 6 skipped. Plugin TypeScript build passes. The compression test fails against the old code and passes with the change. - Runner backend/runtime regression group: 161 tests pass. Cancellation/startup selection: 26 tests pass. Approval delivery: 34 real-database tests pass. - Workflow security: 7 tests pass. Both edited workflows pass actionlint. Runner TypeScript and Rust builds pass. - After merging master, all 389 native executor tests pass, including both native backend paths and late startup after the Stop deadline. - Post-merge `pnpm -r typecheck` and `pnpm build` pass. The monolithic local `pnpm test:run` was interrupted to integrate master and is inconclusive. The [hosted CI test partitions](https://github.com/paperclipai/paperclip/actions/runs/35620461738) pass on `50a3e43822bcba1e0d07b1b45b0be91cbf9312da`. An unchanged sandbox callback schema test initially received HTTP 503. It passed five isolated local runs, its full local test file, and one failed-job CI retry. No assertion was weakened. ## Risks - Stop can wait for the bounded startup handoff. If it cannot settle, the existing pending-recovery state remains instead of a false acknowledgement. - Immediate approval delivery must remain idempotent across cleanup and recovery sweeps. Tests cover duplicate delivery and company/run boundaries. - The workflow changes still need hosted Linux verification. They retain the trusted workflow and credential boundaries. - Gzip reduces the observed provider upload from about 1.8 GB to 663 MB. It does not yet fix the remaining Claude Daytona transfer timeout. That recovery test never reached Claude, so recovery remains unverified. Use a matching image with the verified provider package preinstalled for the next recovery test; retain cold-upload coverage separately. - The report preserves diagnostic runs with missing source metadata and marks them as such. It does not claim a new full-suite pass. - This PR adds no new prompt policy or historical status reconciliation. ## Model Used OpenAI GPT-6 through Codex performed the primary implementation and review. The exact primary backend model ID is not exposed in this session. OpenAI `gpt-5.6-luna` assisted with bounded infrastructure work and verification. The agents used repository tools, code execution, and browser tests. The exact backend revision and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: OpenAI GPT-6 <noreply@openai.com>	2026-09-21 12:50:50 -05:00