5 Commits

Include renames

Author	SHA1	Message	Date
3 people	8b27af77db	fix(heartbeat): wait for a live holder of a reused shared execution workspace (#16371) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The heartbeat pre-dispatch gate decides if a run can use a shared workspace in its execution environment. > - A child task can inherit the shared execution workspace of its parent. The parent run is often still live when the child run starts. > - On a sandbox environment, the gate failed the child run with `configuration_incomplete`. It did this for every concurrency policy, also for the default `auto` policy. > - Recovery then blocked the task and told the operator to bind missing secrets. That was not the real cause. An operator had to select a different workspace by hand. > - Shared project workspaces already use the `auto`/`serialize` policy to wait through the `workspace_busy` retry path. > - This pull request applies the same policy to a reused shared execution workspace. > - The benefit: the child task waits for the parent run and then continues automatically. It does not fail and become blocked. ## Linked Issues or Issue Description **What happened?** A parent agent created a child task. The child inherited the parent's shared execution workspace. The child agent used a sandbox environment. The child run started about 1.5 seconds after the parent created the task, while the parent run was still live. The run failed before the adapter started with this error: "This environment cannot place concurrent agents in one physical shared folder. Use local/SSH sharing or isolated task workspaces." Recovery set the task to `blocked` with the message "required secret/env bindings are missing". **Expected behavior** With the default `auto` policy, the child run waits for the parent run (`workspace_busy`) and then continues automatically. This is the same behavior as for a shared project workspace on a sandbox. **Paperclip version or commit** Staging release `8ffc5748`. The same code is on `master`. **Deployment mode** Hosted cloud (staging), agent on a sandbox execution environment. **Steps to reproduce** 1. Use an agent with a sandbox environment. 2. Bind two issues to the same `shared_workspace` execution workspace. Set `executionWorkspacePreference: reuse_existing` on the second issue. 3. Keep a live run on the first issue. 4. Wake the agent on the second issue. Before this change, the run fails with `configuration_incomplete`. After this change, the run is deferred with `workspace_busy`. ## What Changed - `environment-selection.ts`: When a reused shared execution workspace has a live holder on a non-local/non-SSH environment, the gate now reads `sharedWorkspaceConcurrency`. With `auto` or `serialize`, it throws `WorkspaceBusyDeferral`. Only `allow` still fails with `shared_realization_unsupported`. - `retries.ts`: Added `findExecutionWorkspaceHolder` next to `findSharedWorkspaceHolder`. It keeps the old match rule. A running holder that has had no output for more than one hour (the existing staleness cutoff) is still returned, with `stale: true`. A live holder is preferred over a stale one. - `environment-selection.ts`: A stale holder still owns the folder, because silence does not prove that the run stopped. The child run therefore fails with `configuration_incomplete` (`shared_realization_unsupported`, `holderStale: true`). It does not share the folder with the holder, and it does not wait without limit. This is the same result as before this pull request, with a more specific message. - `retries.ts`: `WorkspaceBusyDeferral` can now carry an `executionWorkspaceId`. The run result, the run event, and the cancellation reason name the execution workspace when it applies. - `stranded-notice.ts`: Added a specific recovery notice for `shared_realization_unsupported`, and a second notice for a stale holder. They no longer tell the operator to bind missing secrets. - `doc/experimental/issue-worktree-support.md`: Documented that `allow` is only concurrent on local/SSH, and that a reused shared execution workspace follows the same rules. ## Verification - `cd server && npx vitest run src/__tests__/heartbeat-workspace-busy.test.ts -t "reused shared execution workspace"` covers 4 tests: - `auto` on a sandbox with a live holder: the run is cancelled with `workspace_busy`, a retry is scheduled, no Sentry run failure is reported, and the issue does not become blocked. - After the holder finishes, the scheduled retry is promoted and runs. The adapter gets the inherited shared folder, and the issue keeps its execution workspace. - `allow` on a sandbox: the run still fails with the shared-folder message and no retry is scheduled. - A holder that is still running but silent past the staleness cutoff: the run fails with `holderStale: true`, the adapter does not start, no retry is scheduled, and the holder is not touched. - `npx vitest run src/services/recovery/stranded-notice.test.ts src/services/heartbeat/environment-selection.test.ts src/services/heartbeat/retries.test.ts` covers the notice and the selection unit tests. - First commit (`d0b7a092`): the 3 original integration tests and the unit suites passed locally, and all CI checks passed. - Follow-up commit (`3c7e515`, which addresses the Greptile review): these tests were **not** run locally. Dependency install was killed twice by the sandbox's 4 GB memory limit. CI is the first run of the new and changed tests. - Known local failure: `does not hide ambiguous, malformed, provider, or transport cancellations behind workspace_busy` fails in this sandbox also on unchanged `master` (checked with `git stash`). This change does not cause it. - Not run locally: the full server `tsc --noEmit`, for the same memory reason. CI runs the typecheck. ## Risks - Behavior change: before, a run with a live holder of a reused shared execution workspace on a sandbox failed at once. Now it waits with `auto`/`serialize`. A long parent run can delay the child. If the holder has no output for more than one hour, the child fails as before instead of waiting without limit. - An explicit `allow` policy keeps the old failure. - Local and SSH environments do not change. - No schema or migration change. ## Model Used - Anthropic Claude Opus 5.5 (`claude-opus-5-5`), with tool use and code execution in the Paperclip agent runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass (first commit only; see Verification) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Devin Foley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-14 20:45:29 -07:00
Devin FoleyandPaperclip	4b8dc416da	Fix default isolation for projects without workspace configuration (#13636) ... Require a company-scoped configured workspace before applying the operator default for Git worktree isolation. Projects that only have a plain managed directory retain their existing behavior. Explicit isolation requests still require a valid checkout. Add policy and heartbeat integration regressions and document the default. The regression fails before the fix. An isolated checkout passes 541 relevant tests, and the server TypeScript check passes. Local repo-wide typecheck and build require the missing Rust toolchain; all CI lanes passed, and Greptile reviewed the refreshed head at 5/5 with no comments. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-09-18 14:49:45 -07:00
DottaandPaperclip	84fe89906d	fix: complete native agent review handoffs (#13581) ... ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native execution uses durable runs, issue locks, wake requests, and typed tool authority > - A child can finish with a native agent review request while its original assignee stays responsible for the work > - The reviewer then needs a bounded execution path that can inspect the child, record one decision, and finish safely > - Before this change, assignee-only gates rejected the reviewer or left the parent waiting after the child review ended > - This pull request adds typed reviewer admission, scoped reviewer tools, durable wake and recovery handling, and parent continuation evidence > - The benefit is that native review handoffs complete without changing child ownership or granting broad mutation access ## Linked Issues or Issue Description Refs: #13314 Refs: #13574 **What happened?** A native child run could report `needs_review` for an agent reviewer. The reviewer wake then failed assignee and execution-lock checks. The child remained in review and the parent remained waiting. **Expected behavior** The named reviewer should receive one durable wake. The reviewer should inspect the child and resolve the exact review card. The child assignee should stay unchanged. The parent should receive the recorded review outcome after the child reaches its terminal state. **Steps to reproduce** 1. Run a native task with a different named agent reviewer. 2. Keep the child assigned to its original worker. 3. Let the worker finish with a native completion review request. 4. Start the durable reviewer wake. 5. Resolve the review and finish the reviewer run. 6. Observe the child and parent state. **Paperclip version or commit** Base: `e926b1301`. PR head: `b31ad9ab8`. Live reviewer verification source: `eea171aae`. **Deployment mode** Built from source. **Installation method** Built from source (pnpm build). **Agent adapter(s) involved** Not adapter-specific (core bug). **Access context** Both. **Database mode** Embedded PostgreSQL in the isolated live test fixtures. ## What Changed - Add server-validated native review assignment facts. - Admit only the exact company, issue, source run, decision, revision, addressee, and resolver policy. - Give reviewer runs a narrow set of Paperclip read and resolve tools. File and shell access follow the configured agent and environment policy, so reviewers can run tests. - Separate server-owned reviewer instructions from untrusted persisted review data. Escape the data boundary; retain server-enforced authorization. - Keep the child assignee unchanged. Atomically claim the reviewer run, wake request, and issue execution lock. A competing lock prevents provider startup. - Require the exact running reviewer session and current issue lock to resolve its assigned card. Reject missing, unrelated, or terminal reviewer runs. - Add durable reviewer wake, lock, stale-card, and abandoned-run recovery handling. - Prevent duplicate native wake dispatches during deferred admission and recovery. - Carry accepted or rejected child review outcomes into parent task context and continuation evidence. - Add focused server, runner, and native protocol coverage. - Preserve upstream continuation rules. Add child review decisions as separate evidence, while keeping real human answers in their own field. - Return actionable completion validation feedback to both providers. Permit a corrected completion after rejection. Keep strict terminal acknowledgment validation. - Apply exclusive shared-workspace locks to sandbox environments. Local and SSH folders can run concurrently, including when old settings request serialization. - Repair test timing, native event parsing, and the review artifact assertion. Allow a valid reject, correct, and accept review sequence. Check the accepted card against its reviewer run and decision. Keep polling within the existing deadline when review acceptance precedes the parent wake projection; report a specific missing-continuation error at timeout. - Apply the ACPX pending-call limit to reserved finish/block calls, with capacity-release and cancellation tests. ## Verification - `pnpm build`: passed on `eea171aae`. - `pnpm -r typecheck`: passed on `eea171aae`. - `pnpm test:e2e:runner:unit`: 359 tests passed in 30 files on `b31ad9ab8`; runner E2E typecheck also passed. - `pnpm check:token-gates`: passed. - Focused DB review, reviewer authority, and prompt-boundary checks: 31 tests passed. They cover invalid reviewer runs, competing locks, atomic admission, duplicate claims, and valid resolution. - Heartbeat, workspace, and recovery checks: 30 tests passed. - ACPX sidecar suite: 27 tests passed. Moving the capacity guard back below reserved handling makes both new regression cases fail. - Four focused live continuation checks passed on their first attempt at `f15f55e0a`: answer updates scope (6/6 each on Codex and Claude) and question tool guidance (12/12 each). These cases do not use the reviewer prompt path changed afterward. - Fresh Codex and Claude review-handoff checks passed all 29 native checks each on their first attempt at `eea171aae`. Both runs received the expected fixed prompt and completed cleanup. Only the six selected live flows were tested; no full paid provider catalog run. - The final commit only extracts the existing test-harness timeout diagnostic into a shared helper and adds positive and negative coverage. Removing the accepted-review guard makes two regression assertions fail; restoring it passes all six timeout tests. Production runtime code, prompts, deadlines, and grading criteria are unchanged by this final commit. - Deadline regressions: a valid continuation delayed 20 seconds succeeds within its 30-second unit-test deadline; an absent wake returns a specific candidate-failure diagnostic at that same deadline. Both assertions failed before the fix. Production E2E deadlines remain unchanged. - Historical native failures remain recorded: Docker availability failures; a valid reject/correct/accept sequence that the first-card grader misread; and a test that rejected the gap between accepted child review and parent wake projection. No failed result was regraded. The latest tests use a protected reference to the pinned Docker image and the unchanged artifact oracle and time limits. - Full repository verification runs in GitHub CI. Local verification uses the focused suites above, full build, and full typecheck. An unchanged Codex shutdown timing test failed once in CI, passed in isolation, and its full shard passed on the final commit without changes to that test or its causal code path. The original failure is retained in the verification record. Greptile reviewed `b31ad9ab8` at 5/5 with no outstanding actionable findings. All review threads are resolved. All current-head CI gates passed, including the isolated native runner Docker build (55 successful checks; two skipped by the workflow). ## Risks - Reviewer admission depends on exact persisted decision and interaction bindings. A stale or changed card is rejected. - Paperclip control-plane tools are limited to inspection and review resolution. This is not a filesystem permission boundary; provider file and shell access retain the configured policy. - Deferred wake recovery changes dispatch receipt coalescing. A scheduler regression could delay a continuation if the receipt state is wrong. - Parent review outcomes are evidence for the model. They do not grant tool authority or change issue ownership. - This change does not address legacy lease-hold handoff behavior. > Roadmap review: native execution, review gates, and durable recovery are existing roadmap capabilities. This PR completes a narrow reliability path for those capabilities. ## Model Used OpenAI `gpt-6-astra` with reasoning, tool use, and code execution. OpenAI `gpt-5.6-luna` assisted with bounded implementation, review, and journal work. Context window size is not exposed by this session. Live test subjects use `gpt-5.6-sol` and `claude-sonnet-5`; they are not the PR authors. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-09-17 15:52:19 -05:00
DottaandPaperclip	ba396c608c	feat(server): configure shared workspace concurrency (#10759) ... <!-- Write all pull request text in Simplified Technical English (ASD-STE100): short sentences, one instruction per sentence, simple approved vocabulary, and the active voice. --> ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The heartbeat service starts agent runs in local, SSH, sandbox, plugin, or Kubernetes environments. > - Runs can share one project working tree. > - PR #10699 made every shared working tree single-file, including trusted local and SSH hosts that support coordinated concurrent work. > - Operators need a policy that keeps remote environments safe and restores local multi-agent work. > - This pull request adds an `auto`, `serialize`, or `allow` concurrency policy and applies it to the final execution environment. > - The benefit is safe serialization by default for sandboxed targets and useful concurrency by default for persistent local targets. ## Linked Issues or Issue Description Related work: #10699 introduced the busy gate that this change makes configurable. #7852 covers a separate environment-lease race and does not provide this dispatch policy. **Subsystem affected** `server/` heartbeat orchestration and `packages/shared/` workspace policy contracts. **Problem or motivation** The shared-workspace busy gate always defers a second run. This behavior prevents local multi-agent projects from running concurrently even when operators expect agents to coordinate through commits. **Proposed solution** Add `sharedWorkspaceConcurrency` with `auto`, `serialize`, and `allow` values. Default `auto` permits local and SSH concurrency. It serializes sandbox, plugin, and forced Kubernetes execution. **Alternatives considered** Keeping unconditional serialization is too restrictive for persistent host working trees. Always allowing overlap removes the protection that sandboxed and remote targets need. **Roadmap alignment** This is a focused correction to the shipped cloud and sandbox execution milestone. It does not add a new roadmap feature. ## What Changed - Added the optional tri-state field to project policy and issue override contracts and validators. - Added a pure resolver with issue override, project policy, and `auto` default precedence. - Moved final environment and Kubernetes resolution before the shared-workspace busy gate. - Kept the existing deferral and retry behavior for every path that resolves to serialization. - Added a task-context warning and a structured log when a run dispatches beside a live holder. - Added policy and heartbeat coverage for all requested policy and environment combinations. - Documented the new policy and its default behavior. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/execution-workspace-policy.test.ts src/__tests__/heartbeat-workspace-busy.test.ts` — 32 tests passed. - `pnpm -r typecheck` — passed. - `PAPERCLIP_IN_WORKTREE=false PAPERCLIP_DATABASE_RESTORE_IN_PROGRESS=false PAPERCLIP_RESTORE_IN_PROGRESS=false pnpm test:run` — all 3,528 server tests passed. The later UI/workspace phase passed 3,384 tests and hit one unrelated `CompanyEnvironments` navigation timing failure. - `pnpm --filter @paperclipai/ui exec vitest run src/pages/CompanyEnvironments.test.tsx -t "opens the edit form on a standalone page with existing values and closes after save"` — the unrelated UI test passed in isolation. - `pnpm build` — passed. Policy matrix: | Policy | Final target | Expected result | Result | | --- | --- | --- | --- | | `auto` | local | Dispatch with holder note | Passed | | `auto` | sandbox | Defer with `workspace_busy` | Passed | | `auto` | instance-forced Kubernetes | Defer with `workspace_busy` | Passed | | `serialize` | local | Defer with `workspace_busy` | Passed | | `allow` | sandbox | Dispatch with holder note | Passed | Existing serialization, retry, and stale-holder tests also pass. ## Risks - `auto` changes the post-#10699 local and SSH behavior back to concurrent dispatch. Concurrent agents can mutate the same working tree, so each dispatched run receives an explicit coordination warning. - `allow` is an operator override and can permit overlap in sandbox or plugin environments. - Unknown environment drivers serialize in `auto` mode. This keeps the fallback conservative. - There is no database migration. An absent field resolves to `auto`. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 (Codex). The exact API snapshot and context-window size are not exposed to the agent. Reasoning, repository editing, terminal tool use, and local code execution were enabled. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-08-03 14:25:05 -05:00
Dotta	b0b7ec779a	updating paths	2026-03-10 14:43:34 -05:00