PaperClipAI

0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-25 09:41:04 +01:00

Author	SHA1	Message	Date
DottaandPaperclip	d0c8f0b090	fix: explain project access denials and add an authorized repair path (#17509 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Low-trust agents may read only the projects in their allowed scope. > - Workspace setup correctly stops a task outside that scope. > - The task previously showed a generic failure, and agent settings did not offer an additive repair. > - This pull request explains the denial and lets authorized operators add projects while preserving other restrictions. > - The saved task then offers an explicit retry. Saving access does not start work. ## Linked Issues or Issue Description What happened? A project access denial appeared as an ordinary failed attempt. Operators could not add a second allowed project from agent settings. Expected behavior The task should explain the denied access and link to a working permission editor. The editor should preserve existing scope and restrictions, enforce operator authority, and retain drafts after a failed save. Steps to reproduce 1. Assign a low-trust agent to a task whose project is outside its allowed projects. 2. Let workspace setup deny project access. 3. Open the task, follow Manage project access, add the project, save, reload, and return to the task. 4. Confirm that the saved message remains and Retry task is a separate action. ## What Changed - Record a typed, non-retryable project denial before provider dispatch. Project denials do not enter automatic continuation or provider comment policy. - Project current and historical setup denials into a safe task notice. Keep unrelated failures, prior assignees, and approval/pause/budget gates separate. Recheck current project authority, including standard trust and root-issue grants. Accepted retries keep their working state. - Add board-only project-access reads and additive saves under the agent row lock. Enforce company scope, agent-management authority, pending-approval restrictions, and an audit record for new additions. - Add the production notice and searchable multi-project editor, including read-only, saving, cancellation, and failed-save states. Preserve the return-to-task query through canonical agent redirects. - Register board-only project-access routes in OpenAPI with a strict request schema. - Add production Storybook examples and API/persistence/concurrency regressions. Retain the approved design snapshot and update the behavior maps and implementation contract. ## Verification - Passed: `pnpm -r typecheck`, `pnpm build`, `pnpm build-storybook`, `pnpm check:token-gates`, `node scripts/check-maps.mjs`, and `node scripts/storybook-catalog.mjs`. - Passed: 11 permission/projection tests and 30 run-completion tests against an isolated PostgreSQL database. These cover historical failures, concurrent additions, idempotent audit, preserved restrictions, foreign projects, read-only users, agent self-grants, and no automatic wakeup. - Passed: all 27 OpenAPI route tests, including route coverage and the new board-only request/response contract. - Browser: the real app displayed a historical denial; Manage project access opened the editor; saving retained the original project and survived reload; Return to task showed the ready notice. Saving created one audit event and zero runs. Explicit Retry returned 202 and reached the separate sandbox requirement, which remained enforced. No external provider was used. - All five production Storybook interaction checks pass: blocked task, save, cancellation, failed-save recovery, and read-only settings. These use fixture callbacks; durable authorization is covered separately above. - `pnpm test:run` was attempted locally but stopped after repeated embedded PostgreSQL startup failures caused by macOS semaphore exhaustion. Full-suite CI is required before merge. The separate UI suite passed all 9,095 tests. The 41 focused server regressions also passed after rebasing onto the latest master. A further 26 recovery-classification tests pass, including a red-to-green regression for automatic continuation after a project denial. ## Risks - No schema migration. The new endpoint only adds same-company project IDs and preserves the rest of the stored policy. - The historical notice recognizes one exact setup denial message. It does not display arbitrary run errors or disclose a different source project's identity. - The retry still passes all current admission and isolation checks. The notice does not promise that other execution requirements are satisfied. - Related open work: #13309 covers inherited secret restrictions; #11234 covers the broader repository catalog. This change does not replace either policy surface. ## Model Used OpenAI Codex (GPT-6), with code execution and browser tools. The precise runtime model ID and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused server and full UI suites; full-suite infrastructure limit above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 16:16:29 -07:00
Devin FoleyandPaperclip	a3675ddcdf	Add bounded database response size diagnostics (#17504 ) Count database response value sizes by public caller and static table for one explicit UTC window of at most 60 seconds. Keep diagnostics off by default and exclude SQL, parameters, returned values, errors, and credentials from output. Preserve lazy execution, transaction boundaries, and no-replay behavior. Verification: full build and recursive typecheck; 63 focused database and PostgreSQL tests; eight physical disconnect tests with capture enabled. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-24 15:39:28 -07:00
Devin FoleyandPaperclip	470cb34c0e	Read reply repair and ownership evidence without execution payloads (#17501 ) Narrow native reply repair and conversation ownership reads to their consumed evidence while preserving full stored history, result receipts, locks, current authority checks, and publication authorization. Verify successful repair from equivalent unpublished states, exact final replies and decisions, no duplicate repair, preserved history, and 213 recovery/presentation tests. Build/typecheck and all CI pass. Greptile 5/5 Medium. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-24 15:14:48 -07:00
Devin FoleyandPaperclip	a7006a9506	Reclaim finished isolated task directories with restore points (#17472 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents do task work in execution workspaces on the instance disk > - A task without a project repository runs in a plain (non-Git) task directory under `<instance>/isolated-workspaces/<company>/<issue>/` > - Nothing removes these directories after the work is done, so the disk fills up > - The directories can hold the only copy of agent work, so plain deletion is not safe > - This pull request saves each finished task directory in an encrypted restore point, removes it, and restores it when the work reopens. It also removes orphaned directories after a grace period > - The benefit is that finished task directories stop using disk, and no agent work is lost silently ## Linked Issues or Issue Description What happened? On one instance, `instances/<id>/isolated-workspaces/` held 251 task directories and 12 GB, and it grew every day. 61 directories belonged to open workspace rows whose issues closed more than 7 days before (3.9 GB). 12 belonged to issues closed 3 to 7 days before (0.7 GB). 4.0 GB were directories with no open workspace row (the row was archived or missing). The cause has two parts: 1. The terminal workspace reaper archives a workspace only when it proves delivery through Git (a merged pull request or a merged branch). A task directory is not a Git checkout, so the Git status inspection fails and the reaper always skips the row as "undelivered". Git cache expiration handles only `git_worktree` workspaces. 2. When a person archives such a workspace by hand, artifact cleanup removes a `local_fs` directory only when `metadata.createdByRuntime` is true. Task directory rows never set it, so the directory stays and the row becomes `cleanup_failed`. Expected behavior Paperclip reclaims the task directories of finished work by itself. It must not lose files that only exist in those directories. Steps to reproduce 1. Run an agent on an issue without a project repository (for example a low-trust sandbox task, or an issue with no project), so it works in `isolated-workspaces/<company>/<issue>/`. 2. Let the agent write files there, then mark the issue `done`. 3. Wait any number of days. The directory and its workspace row stay. The reaper logs the row as skipped (undelivered). 4. Archive the workspace by hand. The directory still stays. Paperclip version or commit `master` at `fe85fd7ac4` Deployment mode Self-hosted, local disk storage. Agent adapter(s) involved Any adapter that runs in a task directory (seen with local and sandbox runs). ## What Changed - Added `isolated-task-directory-snapshot.ts`: a deterministic snapshot of a plain directory (files with modes, directories, symbolic links as links). It never follows symbolic links, refuses special files, and enforces the restore point limits (20,000 files, 64 MiB per file, 256 MiB in total). A `node_modules` directory next to a `package.json` is left out. Restore writes to a staging directory, verifies the digest, then renames it into place. - Added `isolated-task-directory-retention.ts`: - `expireIsolatedTaskDirectory` saves the directory in a restore point, archives the workspace with the `isolatedTaskDirectoryRetention` state in one transaction, checks again that the directory did not change, renames it to trash and deletes it. A change after the capture rolls the workspace back to active and drops the restore point. An interrupted removal finishes on a later sweep. - It uses the git cache expiration consumer checks (`isGitWorkspaceExpirationEligible`): no live run, unsettled finalization, live lease, live runtime service, unresolved recovery action or reopen fence, and no other open workspace or project workspace in the directory. It also requires the source issue to be bound to the workspace, because reopen restores through that binding. - `restoreIsolatedTaskDirectory` restores the directory on reopen. It replaces an empty directory and fails closed on a non-empty directory that does not match. When the restore point expired, it reopens an empty directory and posts a system comment. - `sweepIsolatedTaskDirectories` walks `isolated-workspaces/` in bounded batches (8 per sweep, with a cursor). A closed workspace that its issue is still bound to is saved and removed like an open one. Other folders are orphans and are removed after a grace period when no live row, project workspace, runtime service, run, finalization, lease or recovery action refers to them. - Reused the git restore point ledger, storage and encryption. `storeSealedRestorePoint` and `readRestorePointObject` were extracted from `storeGitRestorePoint`. `markGitRestorePointExpired` now also marks a task directory restore point as expired. - `isGitWorkspaceExpirationEligible` takes `closedSiblingsAllowed`, so closed rows for the same issue directory do not block a task directory forever. Git expiration keeps its old behavior. - Terminal workspace reaper (`execution-workspaces.ts`): handles open task directory rows after the terminal delay and registers that moment as a deadline. It finishes pending removals, runs the folder walk at the start of a rotation (once per server start, then only at real deadlines), and saves at most 8 directories per sweep. - Reopen accepts a closed `shared_workspace` row that holds task directory state. The run path also reopens (and restores) a closed workspace with that state before preparing the run. - Settings: `PAPERCLIP_TASK_DIRECTORY_RETENTION_ENABLED` (default `false`), `PAPERCLIP_TASK_DIRECTORY_RETENTION_DRY_RUN` (default `true`), `PAPERCLIP_TASK_DIRECTORY_ORPHAN_GRACE_DAYS` (default `7`, minimum `1`). The delay is `PAPERCLIP_GIT_RESTORE_POINT_TERMINAL_DELAY_DAYS`. Removal needs `PAPERCLIP_GIT_RESTORE_POINTS_ENABLED=true`. - Logs: `execution_workspace.task_directory_retention`, `execution_workspace.task_directory_cleanup` and `execution_workspace.task_directory_restore`, with outcome, skip reason, candidate bytes and reclaimed bytes. - Documented the behavior in `doc/recoverable-disk-cleanup.md` and the settings in `doc/DEVELOPING.md`. Preservation decision. The code shows that these directories are not scratch space. Nothing pushes or copies their contents, and the next run of the same issue reuses the same path. So the files are saved before removal, with the same ledger, encryption and retention as git restore points, and restored on reopen. Orphans are not saved: no workspace row can reopen them. The docs and the Risks section state this. Scheduling. The work follows `rules/sleep-aware-background-work.mdc`. It adds no timer and no polling. Open rows use the existing reaper and its per-workspace deadlines (the end of the terminal delay, or one hour after a retryable skip). The folder walk runs once at startup. After that, workspace archive and issue close notifications (the existing terminal workspace hints) point the reaper at the affected folder, and a kept folder is checked again only at its own deadline (end of a grace period or terminal delay, or one hour after a busy lock or live consumer). When the tracked state is full, the reaper schedules another walk instead of dropping work. With nothing pending, it registers nothing. ## Verification - `cd server && pnpm exec vitest run src/__tests__/isolated-task-directory-retention.test.ts src/__tests__/isolated-task-directory-retention-config.test.ts src/__tests__/heartbeat-task-directory-restore.test.ts` (embedded Postgres): 48 tests pass. They cover: - a closed issue's task directory is archived and removed after the delay by the reaper, and reopen restores files, modes and symbolic links exactly; - the same through reopen for a `shared_workspace` row, and through a real heartbeat run (the run restores the files and runs in them); - the directory stays before the delay (and the deadline is the end of the delay), while the issue is open, and when retention is off; - the directory stays while each consumer check fails: live run, live runtime service, reopen fence, unsettled finalization, live lease, unresolved recovery action, issue bound elsewhere, live sibling workspace in the directory; - no removal without restore points, with a non-durable key, before `notBefore`, or over the limits; - dry run changes nothing; a change after the capture rolls back; an interrupted removal finishes on the next sweep; an expired restore point reopens empty with a warning; a diverged directory fails the reopen and stays untouched; - folder sweep: an orphan is removed after the grace period, kept when recently modified, in dry run, when a live row points into it, and when its issue is open; an archived unbound row's folder is removed; an archived bound row's folder is saved, removed and restored on reopen; the walk is bounded and continues from its cursor; - review follow-ups: a file name with a backslash saves and restores; after the startup walk, a workspace archived later is found through its change hint; an orphan used by a live run is retried; skipped rows do not starve later rows of the per-sweep budget; overflowing tracked deadlines schedule another walk instead of being dropped; a closing child sends the reaper back to its archived parent's folder; a pending lease cleanup on a closed workspace keeps an orphan; a deep path at the platform path limit restores; an expiry recorded during removal survives and the archive and removal are recorded as activity; a task closed beyond retention keeps its new restore point for the full period; - symbolic links: a task directory that is a link, an orphan that is a link, and a link inside a directory are never followed; the link targets stay untouched. - Mutation checks. Each change below made at least one of the tests above fail, and the test passed again after revert: - drop the run-path reopen condition; drop the skipped-row cache; drop hint tracking; reject backslashes again; drop the live-consumer retry for folders; evict overflowing folder deadlines; skip ancestor hints; ignore workspace-attached leases for orphans; stage restores under the old longer name; overwrite the whole metadata on the final write; retain task directory restore points from the tree close only; drop the removal activity; force `closedSiblingsAllowed` off; make live siblings not block; skip the digest recheck before rename; skip the issue binding check; remove both symbolic link refusals (task directory); remove the orphan symbolic link refusal; ignore live rows in the folder walk; skip the orphan modification grace; ignore dry run for orphans; skip the restore on reopen; skip the terminal delay; skip the restore points requirement; skip the orphan open-issue check. - Removing only one of the two task directory symbolic link checks did not fail a test, because the other check still refuses the link. Removing both fails the test. - Related suites: `pnpm exec vitest run execution-workspaces isolated-task-directory workspace-runtime git-restore-point git-workspace-expiration terminal-workspace workspace-reaper heartbeat-git-restore heartbeat-task-directory isolated-repository-rehome nested-clone-migration environment-selection` in `server/`: 34 files, 749 tests pass, 4 skipped. Two `workspace-runtime.test.ts` tests fail (`does not diagnose a missing remote branch as a rejected credential`, `rejects explicit main on a master-only remote ...`). They fail the same way on unchanged `master` on the local machine, so they are not caused by this change. - `pnpm run typecheck` in `server/`: passes. - Not run: the full server suite, a live instance rollout, and a check that the host disk usage drops after removal. ## Risks - The feature is off by default, and it starts as a dry run when enabled. It needs restore points to remove anything. Rollback: set `PAPERCLIP_TASK_DIRECTORY_RETENTION_ENABLED=false`. Saved restore points and restore on reopen keep working. - Orphan folders are removed without a restore point. This applies only when no workspace row is bound to the issue at that path. If such an issue is reopened later without another workspace, its next run starts in an empty directory. The grace period (default 7 days of a closed tree and no changes) and the dry run limit this risk. - A directory over the restore point limits stays on disk (logged with its reason). Special files keep a directory too. - `node_modules` next to a `package.json` is not saved. A reopened task must install its packages again. - The restore point lives as long as git restore points (90 days by default), counted from the later of the tree close and the removal. Only task directory restore points use the removal time; git restore points keep their retention. After that, reopen starts with an empty directory and posts a warning comment. - Reopen now accepts a closed `shared_workspace` row, but only when it holds task directory state written by this feature. - No schema migration. The ledger table and its columns are unchanged. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 14:16:58 -07:00
Devin FoleyandPaperclip	cb6c878c38	Read execution blocker evidence without full execution payloads (#17491 ) Read the lifecycle, process, and typed context evidence consumed by execution blockers. Preserve full result evidence, native workspace receipts, current state checks, and stored history. Validated with 458 real PostgreSQL and recovery tests, full build, recursive typecheck, all55 CI checks, and Greptile 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-24 14:14:46 -07:00
Devin FoleyandPaperclip	1cfaf973cd	Read release evidence and telemetry without unused output (#17485 ) Read execution release evidence and terminal telemetry without unused output. Keep full stored history, in-flight guards, and accepted messages. Validated with real PostgreSQL tests, build, typecheck, green CI, and Greptile 5/5. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-24 14:01:41 -07:00
Devin FoleyandPaperclip	991c47bc6e	fix(server): project recovery result metadata (#17481 ) Keep full execution output in storage while selecting only recovery metadata for repeated latest-run and legacy eligibility checks. Retain complete results for provider-text classification and preserve all admission and write fences. Verified 456 focused PostgreSQL tests, full build and workspace typecheck, all 55 exact-head CI checks, and Greptile 5/5 with medium impact and no findings. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-24 13:36:54 -07:00
Devin FoleyandPaperclip	fe85fd7ac4	Expire closed workspaces that open descendants do not use (#17464 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Git cache expiration removes closed issues' worktrees and saves restore points for unsaved work > - It treated a workspace as closed only when the source issue and every descendant were done or cancelled > - A routine that closes a parent and leaves children open for days therefore kept every parent worktree, although the children work in their own workspaces > - The idle path did not help, because the children are updated every day > - This pull request counts only the issues that use the workspace when it decides that a closed workspace can expire > - The benefit is that closed parent worktrees are removed while their children work elsewhere, and a child that later binds to the workspace gets it back from the restore point ## Linked Issues or Issue Description Refs #17271. Bug report described in-PR (bug report template). What happened? On one instance, an hourly routine creates a parent issue, works it in its own worktree, marks the parent done, and leaves child issues open (`blocked`, `in_review` or `in_progress`) for days. The children work in their own workspaces. They are not bound to the parent's workspace. Git cache expiration kept every parent worktree (about 300 MB each), because the closed-tree check and the workspace reaper required all descendants to be closed. In one project, 18 of 19 long-closed parent worktrees were kept only by open children that did not use them. The idle path never ran, because the children were updated daily. Expected behavior A closed issue's worktree expires while descendants are open, as long as no open descendant uses that workspace. Steps to reproduce 1. Enable git cache expiration and restore points. 2. Close an issue that has an isolated git worktree, and leave a child issue open that works in its own workspace (or has no workspace binding). 3. Wait past the terminal delay and run the workspace reaper: the parent worktree stays, and the reaper counts it as `skippedNonTerminalTree`. Paperclip version or commit: `master` at `410e5c6eed`. Deployment mode: any. Agent adapter(s) involved: not adapter-specific (core). ## What Changed - `isGitWorkspaceExpirationEligible` (closed path, not the idle path): the "tree closed for one day" check now covers only the issues in the tree that use the workspace. An issue uses the workspace when it is the source issue, when its `execution_workspace_id` is the workspace, or when its current `workspace_selection` or its `workspace_pending_selection` selects it as the `existing` workspace. Inheritance from a parent (`reuse_existing`) and `existing` selections are both stored as `execution_workspace_id`, and heartbeat workspace resolution reuses an existing workspace only through that binding or a run's persisted native binding. An open descendant without one of these references gets a new workspace on its next run, so it no longer keeps the parent's worktree. - All other checks stay the same: any open issue bound to the workspace, live or unsettled runs of any issue in the tree or of the workspace (including a run whose context names the workspace), finalizations, leases, recovery actions, runtime services, shared cwd and the reopen fence. - Workspace reaper: when the source issue is closed, git cache expiration applies, and no reopen is pending, the reaper routes the workspace by the same rule. It takes the restore point due time from the source issue and the descendants that use the workspace, also after every descendant is closed. A recently closed child that used the workspace still holds it for the terminal delay. The compaction delay and the archive cooldown still use the whole tree when it is closed. - Workspace reaper: compaction and the normal archive path keep their behavior. While any descendant is open, the reaper does not compact the worktree, and the expiration branch handles the workspace and continues. The archive path still requires a closed subtree. With expiration off, or a pending reopen, routing is unchanged. - New exported helper `listIssuesUsingGitWorkspace` shares the SQL "uses the workspace" predicate between eligibility and the reaper. - Docs: `doc/recoverable-disk-cleanup.md` and the `PAPERCLIP_GIT_RESTORE_POINT_TERMINAL_DELAY_DAYS` entry in `doc/DEVELOPING.md`. - The existing `preserves live descendants and pending reopening` case now binds its open child to the workspace. An unbound open child no longer keeps the worktree by design. ## Verification - `pnpm exec vitest run src/__tests__/git-restore-point-expiration.test.ts src/__tests__/git-workspace-expiration.test.ts src/__tests__/heartbeat-git-restore-point-resume.test.ts` in `server/`: all pass. - `pnpm exec vitest run src/__tests__/git-workspace-expiration src/__tests__/git-restore-point src/__tests__/execution-workspace src/__tests__/heartbeat-git-restore-point-resume src/services/heartbeat/environment-selection src/__tests__/isolated-repository` in `server/`: 15 of 17 files passed in one parallel run. The other failures were the updated descendant case (now fixed) and 45-second timeouts in restore point tests while 17 embedded Postgres files ran at the same time. Both files pass when run on their own. - `pnpm run typecheck` in `server/`: passes. - New embedded Postgres cases with real git repositories: - A closed parent with open `blocked`, `in_review`, `in_progress` and `todo` children and a grandchild in their own workspaces (or with none): the worktree expires with a restore point and the reopen restores it exactly. - The reaper archives it, timed from the issues that use the workspace. A child closed one minute ago with its own workspace does not delay it. - A recently closed child bound to the workspace holds it until its terminal delay passes. - When every child is closed and the most recent one used its own workspace, the reaper does not restart the restore point delay. - The worktree is kept when an open child is bound to it, when an open grandchild under a child with its own workspace is bound to it, when a child created later with `issueService.create` inherits it with `reuse_existing`, when an open child selects it through `workspace_selection` or `workspace_pending_selection`, when a child run that references it is live, and when the parent was closed inside the delay. - The reaper never compacts the worktree while children are open, with expiration on or off. A pending reopen keeps its current fence handling. - After expiry, an open child that binds to the workspace reopens it with the saved commits, index, working tree and untracked files. - Heartbeat: a queued run for an open child that binds to the expired workspace of a closed parent restores the workspace and runs the agent in it with the saved state. - Mutation checks (each run against the two restore point test files): - Closed-tree check restored to the whole tree: 7 tests fail. - `execution_workspace_id` removed from the rule: 3 fail. - `workspace_selection` removed from the rule: 1 fails. - `workspace_pending_selection` removed from the rule: 1 fails. - Reaper `latestTerminalAt` from the whole tree: 3 fail. - Reaper routing by the whole tree: 2 fail. - Reaper restore point due time from the whole tree once it is closed: 1 fails. - Pending-reopen gate removed: 1 fails. - Compaction gate (`!treeOpen`) removed: no test fails. Compaction's own guard under the lifecycle lock still rejects open trees, so the gate only skips work that cannot succeed. - Not run: the full server suite and the UI suites. CI covers them. ## Risks - Closed parent worktrees now expire while their children are open. A child that later binds to the workspace depends on the restore point and on the run-side reopen from #17271. The new tests cover this path. When restore points are off, only exactly reproducible worktrees expire. - An open child with no workspace binding no longer keeps the parent's worktree. Heartbeat resolution never reuses a workspace without a binding, so this does not change where the child runs. - The rule is conservative. A workspace selected only by an issue's current or pending selection is kept, and live runs of any issue in the tree still block. - The reaper runs one extra indexed query for each closed workspace when git cache expiration applies. No migration. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 12:43:57 -07:00
Devin FoleyandPaperclip	3a2c56780c	fix(server): avoid execution payloads in run authority checks (#17467 ) ## Thinking Path > - Paperclip manages AI agents and their tasks. > - Each operation must use current task and user authority. > - Chat recovery also checks the exact source-run ancestry. > - These checks can repeatedly read full run payloads. > - Their decisions need identity fields and two typed task keys. > - This change selects those fields and keeps the current checks and locks. > - Complete execution context and output remain stored and available. ## Linked Issues or Issue Description What happened? Ancestry checks, instruction identity reads, task locks, and identity activation receipts transfer full execution context or full run rows for metadata decisions. Expected behavior Select only required identity and binding fields. Keep fresh membership and key revocation checks, task-before-run locking, native task precedence, and strict validation of legacy JSON task identifiers. Steps to reproduce 1. Store a run with large context, Runner profile, and output fields. 2. Read chat recovery ancestry or authorize an instruction operation. 3. Initialize or refresh its run identity. 4. Observe full context or full-row responses in these metadata paths. Paperclip version or commit Baseline: `14e1f59fbf4d9cc7875ed35ebaa9e997f6e2ea7c`. Complementary work: Refs #17460 and Refs #17463. Searched open and closed PRs for ancestry projections and run read optimizations. No duplicate was found. This PR has no dependency on #17463. Deployment mode External PostgreSQL. Tests and measurements use disposable real PostgreSQL. ## What Changed - Share a SQL projection of typed legacy issue/task keys with task-binding readers. - Select only source identity and retry links while checking chat ancestry. - Select current identity fields during instruction authorization. - Narrow identity task-lock reads and activation receipts to task binding fields. - Preserve existing authorization, ancestry proof, revocation checks, and lock order. ## Verification - 72 focused tests pass across read projections, run identity, instruction authority, and chat admission retry. - 49 assembled heartbeat recovery tests pass for chat and automatic ancestry; 359 unrelated tests were outside the filter. - Real PostgreSQL tests include large payloads, task/company/agent mismatch, current membership revocation, malformed JSON types, native binding precedence, and invalid legacy identifiers. - Real ancestry service SQL: 526,477 to 306 serialized response bytes (99.942% smaller) with three 174,764-byte incompressible execution fields. - Every selected non-context value matches the baseline. Both complete service calls return the same result. Typed binding cases are checked in PostgreSQL. - Five alternating warm samples after two warmups: median 2.152 ms before, 0.812 ms after. These are local query/client timings, not hosted request latency. - Full build and recursive typecheck pass. The full local stable suite is running; no full local pass is claimed. Current-head CI and review must pass before merge. - The first new-fixture run failed eight tests because it reused a unique company prefix. Each fixture now has its own prefix; the focused rerun passes all 72 tests. - All current-head CI checks pass on `ecc951b25d67fddc7dc2e0e3e2de861a40ee5ed0`. Greptile: 5/5, medium impact, no findings or unresolved threads. - Staging effect is not measured yet. ## Risks Missing identity fields could change an authorization or recovery decision. The tests exercise current authority and fail-closed behavior. The change keeps full execution reads intact, preserves JSON types, and does not cache authority. There is no migration, message deletion, timing change, or data truncation. ## Model Used OpenAI GPT-6 through Codex, with source inspection, code execution, and GitHub tools. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 12:25:33 -07:00
DottaandPaperclip	6d46851920	test(e2e): add attended GitHub review bot journey (#17457 ) ## Thinking Path > - Paperclip manages agents and the work they perform. > - Product E2E checks verify real user paths through those agents. > - A recorded GitHub review-bot walkthrough had no repeatable registered case. > - App setup, automatic triggering, publication, and task completion need separate evidence. > - This pull request adds an explicit attended journey to the existing catalog and report tools. > - Reusable setup profiles and independent assertions preserve the same outcome across compatible targets. ## Linked Issues or Issue Description Refs: #17262. This case remains an explicit attended attached-target flow. It does not replace the shared deployment-target or paid-operations framework. What existing behavior does this improve? The Product E2E catalog can now select and retain evidence for GitHub review-bot setup and its first automatic review. Current behavior Recorded walkthroughs cannot be rerun as a fixed registered case. A published review alone can hide a failed run or unfinished task. Proposed behavior Create and install an owned App, save automatic new-PR settings, create a defective PR, and require correlated review delivery, an inline finding, a consistent check, a successful run, and a Done task. Retain failures and confirm owned-resource cleanup. Reason and benefit The case makes a useful external integration reproducible without treating setup, unit checks, or report rendering as proof of a completed live journey. ## What Changed - Add two explicit attended cells for an existing configured agent in local or Daytona execution. - Declare hosting, account, connection, and credential prerequisites through private setup bindings. - Add fixed browser controls, ownership checks, independent grading, and cleanup. - Package declared screenshots through the existing sanitizer, manifest, validator, and dashboard. - Reject operations-managed execution until budgeted routing is qualified. - Document hosted QA credential setup and evidence limits. ## Verification - Focused catalog, oracle, report, connection, and ownership checks: 127 tests passed. - Inherited-environment preflight regression failed against the previous commit and passed after the fix. Actual-run environment verification remains required. - Harness TypeScript check and full workspace typecheck passed. Full build passed. The local full-test attempt was interrupted after repeated embedded PostgreSQL startup failures; its exit code of zero does not establish a pass; Exact-head Linux CI passed the full test, typecheck, build, Runner, serialized, browser, and canary gates on `abbddeade848edefc8de75f11cc9008b2cd52764`. - A real hosted attempt published a rated review and inline finding. It failed when its native session timed out before the task reached Done. Cleanup passed. Full live qualification remains open. - Credential scan found no known credential values in the authored files or retained report corpus. ## Risks - The full case creates external test resources and uses the configured provider. It requires an explicit selection and current task authority. It is excluded from default and CI matrices. - Time and run limits are not a hard dollar cap. Missing dollar cost stays unknown. Fixture setup does not weaken budget policy. - Fresh personal OAuth, unattended authentication, runtime skill-read proof, and cross-host qualification are separate claims. - The selected live fixture needs environment/lifecycle recovery before it can adopt a longer provider deadline. ## Model Used OpenAI GPT-6 family through Codex, with reasoning, code execution, and browser tools. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used, including the available version and capability details - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the enhancement template - [x] I have not referenced internal or instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket ID - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation - [x] I have considered and documented risks - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open findings - [x] I will address all Greptile and reviewer comments before merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 12:16:26 -07:00
Devin FoleyandPaperclip	7bd449dbb6	fix(recovery): project context evidence for latest-run checks (#17463 ) ## Thinking Path > - Paperclip manages AI agents and their tasks. > - Recovery sweeps decide whether tasks have a valid next step. > - Those decisions inspect saved run metadata. > - Latest-run queries also return full prompts and execution setup data. > - This repeats large responses when a sweep leaves a task unchanged. > - This change projects the context evidence recovery actually reads. > - A row-version check keeps stale evidence from changing a task. ## Linked Issues or Issue Description What happened? The latest-run recovery queries retrieve large stored contexts for each candidate. The decision usually consumes small recovery, message, or workspace-mode fields. Expected behavior Keep all decision evidence while selecting a small response. Preserve complete stored contexts and full execution reads. Detect concurrent source changes. Steps to reproduce 1. Store a terminal run with large task Markdown, wake, workspace, and continuation bodies. 2. Call `reconcileStrandedAssignedIssues` for its assigned task. 3. Observe the full context returned by the latest-run query on each sweep. Paperclip version or commit Baseline: `410e5c6eedba98588a87e4104153898fb0437532`. Related read optimization: Refs #17405. Complementary fix: Refs #17460. These PRs have no code dependency. Searched existing PRs and issues for run projections, recovery reads, and database transfer. No duplicate was found. Deployment mode External PostgreSQL. Verification uses disposable real PostgreSQL and synthetic data. ## What Changed - Project typed context evidence in all three latest-run recovery queries. - Keep only the used workspace mode and external-interaction flag from nested setup objects. - Preserve original JSON types, message references, retry evidence, and existing query scopes and order. - Read PostgreSQL `xmin` with the run. Require the same row version during native blocked-wait repair. - Retain the original full-context comparison for callers that supply a full run without a row version. - Test large payloads, malformed JSON shapes, review-message recovery, and concurrent source edits. ## Verification - Final-source integration checks: 450 passed across context projections, native finalization recovery, and the assembled heartbeat recovery service. - Focused issue-lock race check: 9 passed (401 outside the filter), including omitted-context and late-output edits and successful repair on a fresh sweep. - Final-source recursive typecheck and full build pass. The full local stable suite is still running; current-head CI must also pass before merge. - Real-service SQL capture against the baseline: one returned row falls from 699,724 to 573 serialized JSON bytes (99.918% smaller). - The fixture uses four 174,764-byte incompressible execution fields. All unaffected returned columns compare equal. Regression tests check the retained decision evidence and unchanged stored records. - Five alternating warm samples after two warmups: baseline median 1.479 ms, treatment 1.084 ms. Ranges overlap; no request-latency speedup is claimed. Plans were captured outside the timing interval. - An initial full-context hash projection cut bytes but raised the local median to 7.457 ms. The final row-version check removes that cost. - One integration run overlapped the hash-export rename and failed seven tests on the removed export. A fresh final-source run passed all 450 tests. The earlier mixed-source failure remains recorded and is not counted as a pass. - All current-head CI checks pass on `17add20e391c30ed0496f4b39eb9c59779efa6b4`. Greptile: 5/5, medium impact, no findings or unresolved threads on that head. - Hosted transfer and product behavior remain to be verified on staging after merge. ## Risks The context selection must track its recovery consumers. Tests preserve typed flags, retry budgets, external-chat references, and malformed-input behavior. The row-version guard is conservative: any intervening run update defers native blocker repair to a fresh sweep. Concurrent omitted-context and output edits are covered through the real service and issue-lock wait; a fresh sweep must still repair. There is no migration, prompt truncation, history deletion, or authorization cache. ## Model Used OpenAI GPT-6 through Codex, with source inspection, code execution, and GitHub tools. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 12:14:41 -07:00
Nicky Leach	60ca3a903a	Keep warm sessions across project revision transitions (#17461 )	2026-10-24 12:11:26 -07:00
Devin FoleyandPaperclip	14e1f59fbf	fix(heartbeat): avoid large run reads during deferred-message recovery (#17460 ) ## Thinking Path > - Paperclip manages AI agents and their tasks. > - Deferred messages must survive restarts and execution cleanup. > - Periodic recovery checks the latest run before it attempts release. > - That check and two release checks load large execution records. > - These checks use only admission evidence and completion identity. > - This change selects those fields while keeping the release policy and fresh reads. > - Full execution records remain available for execution and detail reads. ## Linked Issues or Issue Description What happened? Deferred-message recovery repeatedly transfers stored context, Runner profiles, and output excerpts to make small admission and release decisions. Expected behavior Select the fields each decision consumes. Preserve queued messages, current admission checks, cleanup ownership, and fresh completion reads. Steps to reproduce 1. Save a terminal run with large context, profile, and output fields. 2. Leave an accepted message deferred after the issue lock is released. 3. Call `resumeQueuedRuns` and follow execution release. 4. Observe full-row reads before the admission decision. Paperclip version or commit Baseline: `410e5c6eedb`. Related read optimization: Refs #17405. Searched open and closed PRs for run projections and stranded-queue reads. Searched open issues for database transfer. No matching fix was found. Deployment mode External PostgreSQL. The regression tests use disposable real PostgreSQL. ## What Changed - Select seven admission fields for the stranded queue's latest-run lookup. - Select authentication classification evidence before execution release. - Read completion status and typed issue identifiers after post-commit effects. - Keep company scoping on both release reads. - Add large-payload and fresh-state regressions. Document the read contract. ## Verification - Focused PostgreSQL tests: 68 passed across queue, run control, and the new release-read suite. - Release reads return less than 2 KiB with three large synthetic execution fields. - Queue recovery passes less than 512 bytes to release, retaining the full stored message. - Coverage includes legacy/native owners, paused recipients, live successors, authentication repair, and fresh completion identity after post-commit effects. - Root recursive typecheck and full build pass. The full local stable suite remains in progress; no local full-suite pass is claimed. - All current-head CI checks pass on `c819fbb6b46bd5480a197407f06e02f245667792`. Greptile: 5/5, medium impact, zero comments and zero unresolved threads on that head. - The first focused attempt found a syntax error in the new test; it was corrected before the 68-test passing run. A redundant standalone compiler was stopped while the required recursive check continued. - Hosted impact is not measured yet. Staging verification follows merge. ## Risks The projection must retain all admission evidence and preserve JSON types. The release policy still owns transactions, locks, cleanup, and promotion. There is no schema migration, polling change, message deletion, or API change. Configuration repair evidence is read directly, including when unrelated result output is large. ## Model Used OpenAI GPT-6 through Codex, with source inspection, code execution, and GitHub tools. The exact served model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 12:07:45 -07:00
DottaandPaperclip	0eb2c964a7	fix(agents): allow runner qualification with agent configuration permission (#17448 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent owners configure the harness and computer used for that work. > - A legacy runner upgrade must first qualify an unverified sandbox. > - The upgrade called an instance-admin probe, so a company owner with agent configuration permission received HTTP 403. > - This pull request qualifies the saved computer through the agent's existing configuration authorization. > - Company owners can complete the upgrade while general instance environment management stays restricted. ## Linked Issues or Issue Description Refs #17361. Related: #17122. What happened? Upgrade runner displayed for an existing Claude agent on a Daytona Paperclip Computer. Clicking it returned “Instance admin access required” for a cloud company owner. The agent remained on the legacy runner. Expected behavior A caller permitted to update the agent can qualify that agent's saved computer and upgrade. Instance-admin access is not required for this agent-scoped operation. Steps to reproduce 1. Sign in as a cloud company owner with `agents:configure` and no instance-admin role. 2. Open an existing legacy Claude agent on an unqualified Daytona snapshot. 3. Select Upgrade runner. ## What Changed - Add `POST /api/agents/{id}/qualify-runner` with the existing `agent_config:update` guard. - Reuse the server's effective environment resolver. The submitted environment ID only detects stale state; it cannot select a different computer. - Preserve company binding checks, check binding again at lease acquisition, and record agent-scoped audit activity. - Return qualification status and the lease receipt without instance-level provider metadata or raw provider errors. - Use the new endpoint for the existing-agent upgrade. Keep the generic environment probe restricted to instance admins. - Disable Upgrade runner when an unsaved computer change resolves to a different target, with a save-first explanation. Other draft settings still accompany an upgrade. - Add real-database authorization regression tests and update the API contract and access maps. ## Verification - Red: the existing UI called the instance-wide probe and failed the new agent-scoped call assertions in eight upgrade cases. - Green: 349 related route, provider qualification, runner selection, and UI tests passed. - Green: 12 authorization cases passed with real PostgreSQL, real HTTP routes, real saved role grants, and a simulated provider boot. They cover the cloud owner, removed grants, foreign agent/computer, inherited targets, stale target, invalid body, archived/local target, redacted failure, audit scope, and unchanged agent settings. - Review regression: two cases first reproduced the enabled upgrade button after an unsaved computer change; both now pass, along with all 168 related UI tests. Qualified/unqualified computers and restoring the saved selection are covered. - All 26 OpenAPI cases passed; the 12 real-database authorization cases also passed after rebase. - The host exhausted PostgreSQL semaphores. For the 12 cases, a temporary local-only setup used disposable PostgreSQL 18 in Docker instead of the embedded database launcher. The committed tests use the normal embedded database helper. No authorization decisions were mocked. - `pnpm -r typecheck`, `pnpm build`, `pnpm check:token-gates`, map validation, module-boundary validation, and `git diff --check` passed. - `pnpm test:run` was attempted but did not establish a full local pass: the host could not start embedded PostgreSQL clusters. All current-head CI gates passed, including the full suite in CI. The commitperclip review workflow timed out before reporting any executed steps on its first attempt and passed on one retry. - Fresh staging acceptance remains pending deployment. Verify persisted native configuration and a successful runtime test on the existing Daytona computer after merge. Earlier live Daytona proof from #17361 is separate evidence. ## Risks - Qualification can start a fresh, temporary sandbox. It uses the existing probe lifecycle and cleanup contract. - The new path must remain scoped to the saved agent and its company. Denial and redaction tests cover that boundary. General instance probe authorization is unchanged. - No migration, role change, credential change, or automatic agent upgrade is included. ## Model Used - OpenAI GPT-6 through Codex, with reasoning, code execution, and browser tools. The exact deployed model variant and context-window size were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 10:31:40 -07:00
DottaandPaperclip	7ae81c9fa7	fix(slack): support rich questions and complete review delivery (#17255 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Questions and review decisions must work in the conversation where a task started. > - Slack omitted question context and did not publish several review kinds. > - Some answers saved in Paperclip also lost their authorized continuation or their final Slack reply. > - A settled card and a successful run did not prove that the user received the result. > - This pull request preserves complete questions and follows the existing answer and delivery receipts through continuation. > - Users can read the request, submit an authorized decision, and receive the resumed task's result in the same conversation. ## Linked Issues or Issue Description Refs #17045 and #17218. What happened? Slack question cards could omit instructions and editable starting text. Checkbox reviews, item verdicts, task suggestions, and connection requests had no publication path. Complex question answers submitted in Paperclip could fail continuation authorization. Several review continuations completed without delivering their result to Slack. Expected behavior Show the actual question and context. Use native Slack controls when they preserve the form contract. Otherwise provide an explicit Paperclip review link. Persist one authorized response, resume through the existing delivery path, settle the card, and deliver the useful result to its originating conversation. Steps to reproduce 1. Start a task in Slack and ask a text or choice question with instructions and help text. 2. Submit native text and choice responses, then submit a complex form through Paperclip. 3. Accept a checklist, submit item decisions, select suggested tasks, or decline a connection request. 4. Inspect the saved answer, continuation run, original Slack card, and final Slack reply independently. ## What Changed - Preserve full prompts, context, descriptions, recommendations, help, and editable starting text. Use direct buttons for short choices, inline searchable selects for longer lists, and native checkboxes with explicit submission for multiple choices. - Add native custom-answer and validated-text modals, ordinary confirmation buttons, checklist decisions, per-item verdicts with reasons, suggested-task creation/dismissal, and connection decline. Keep governed approvals, permission grants, and forms that cannot be represented faithfully in the full Paperclip review surface. - Keep Slack-specific canonical-validation feedback in its owning platform module. Shared callbacks delegate without interpreting provider presentation. - Reuse the existing canonical interaction transactions, callback receipts, publication queue, and continuation delivery. Share ordinary confirmation handling across Telegram and Slack. Recheck current authorization and commit response receipts atomically. - Settle cards with the saved answer and remove stale controls. Ignore passive Slack link callbacks so opening a task does not display a false answer error. - Validate exact multi-select receipt mappings during native continuation. Recover automation-origin connection replies from their published prompt after validating the durable connection receipt. - Keep oversized multi-select handoffs free of unused native tokens, and retain the complete Paperclip form for 101–128 choices plus Other. Make the original-conversation audience explicit before submitting a Slack modal. - Add rollback, provider serialization, callback, continuation, and negative authorization regressions. Update the connector playbook and feature/access maps. No React or application UI changes are included. - Add a durable Q01–Q11 qualification index and provider coverage matrix. Require distinct functional, good-experience, and best-available-control assessments; real unanswered/answering/settled evidence; saved results, continuation and navigation proof; and retained failures. Mark other providers unaudited rather than assuming Slack parity. ## Verification Candidate: `e0017d446b1016a38d65382366cd1408cce33039`. The eleven-core-format live Slack gallery is pinned to `c3d458b2a06b660448940fd8cf33c5906f0058d5`; subsequent boundary fixes and the sharing notice have automated verification, not new live screenshots. - Real Slack UI testing covered all 11 formats: text, closed choice, long single select, multiple choices, choices with a custom answer, pattern-validated text, confirmation, checklist, item verdicts, suggested tasks, and connection requests. Each has unanswered/answering/answered screenshots as applicable, an inspected saved result, one continuation, and an individual experience review in the local HTML gallery. - The dedicated private QA app/channel used signed callbacks to the local candidate. A separately approved password-protected, read-only preview proved connection-review, suggested-task review, and created-task navigation. Anonymous access, writes, and credential routes were denied. No OAuth grant was attempted. - Real testing exposed and repaired a multi-select continuation failure and a misleading error on link-button clicks. Failed attempts remain in the evidence. The final navigation replay opened the exact pending connection request and declined it in Slack without granting access. Suggested-task creation produced the selected child with the correct parent and description. - Final review-fix verification: 32 form tests, 28 provider/form-rendering tests, 30 publication tests, and 171 native continuation/authorization tests passed. Earlier 3 atomic-resolution rollback, 9 targeted callback/publication/Telegram, and 28 review-continuation cases passed. The final Slack/Teams modal subset also passed. The automation-origin connection cases, 101/128 custom-choice cases, and oversized multi-select continuation reproduced before their fixes and passed afterward; failing logs remain retained. These counts describe relevant runs, not a reliability percentage. - CI on `b5885b85fd` exposed a test fixture that assumed every Slack modal block was an input; the new sharing notice correctly renders a non-input block. The fixture now selects input blocks safely. The exact signed-modal correction/one-continuation test passed locally, together with 14 Slack form/modal tests. The failing CI log and its specific cause remain recorded. - `pnpm -r typecheck` and `pnpm build` passed. Server typechecking was repeated after the final source changes. Map and module-boundary checks and `git diff --check` passed. - The full `pnpm test:run` was attempted and stopped after repeated embedded PostgreSQL semaphore exhaustion on this host. Its 1,395-test chat suite found two stale native-control assertions; both were corrected and re-tested in a focused run. Focused database suites used separate disposable databases on existing local PostgreSQL. The full suite is not claimed green; new-head CI must complete before merge. - Final-head CI passed on `e0017d446b1016a38d65382366cd1408cce33039`: all test/build jobs and required `ci / verify` and `ci / e2e` gates are green ([run](https://github.com/paperclipai/paperclip/actions/runs/40115430194)). Greptile and Ponytail both reviewed this exact head at 5/5, the security scan passed, and all review threads are resolved. - All three QA app callbacks were restored to staging. Both temporary tunnels and local proxies were closed. The local endpoint was archived and its local credential copies removed. The candidate app was stopped. Limits: provider execution used a deterministic Claude protocol fixture, not an LLM. This is local real-provider UX proof, not deployed staging or broader Codie/Claude qualification. The public preview intentionally blocked ancillary artifact routes, so its destination pages could show a partial-history warning. The complete permission-grant flow and every oversized/mixed-form branch were not driven manually. Earlier fixture, test, and provider failures remain recorded. ## Risks - New reply routes require durable origin proof. Incorrect receipt matching could lose a reply or route it incorrectly. Tests cover missing, forged, mismatched, and revoked authority. - Governed forms keep their Paperclip resolution checks. This change does not grant new approval authority to Slack. - Existing completed review results can become deliverable through the repaired publication path without another provider run. - No schema, lockfile, workflow, public task-status model, or runtime-instruction change is included. This branch is not deployed. ## Model Used OpenAI Codex, GPT-6 family, with code execution and tool use. The exact backend model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run focused tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green on the new head - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups on the new head - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 10:29:15 -07:00
Nicky Leach	62f60bf158	Measure first deliverable text in native run traces (#17434 )	2026-10-24 09:40:08 -07:00
Devin FoleyandPaperclip	785d54636e	Delete re-homed isolated clones when their contents are reproducible (#17410 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Isolated task workspaces are Git worktrees on the server disk, and the terminal workspace reaper keeps that disk usage bounded > - #17369 moves old workspaces off their private per-issue clones onto the project's shared clone and then deletes the private clone > - On one instance, the first live sweep moved 12 workspaces but deleted none of their clones (about 600 MB each), and 3 workspaces could not move because their restore point was "too large" > - The kept clones are not referenced by any workspace, so no other cleanup removes them > - This pull request replaces the clone check with a real reproducibility check, sweeps the orphaned clones, lifts the restore point limits for local transfers, and fixes a false `source_identity_changed` > - The benefit is that the disk space #17369 was meant to reclaim is reclaimed, without deleting unique work ## Linked Issues or Issue Description Refs #17369. Bug report described in-PR (bug report template). What happened? On one instance, with `PAPERCLIP_ISOLATED_SOURCE_REHOME_ENABLED=true` and dry run off, the first sweep re-homed 12 workspaces, skipped 5 with `live_or_unresolved_consumer` and 3 with `restore_point_too_large`. No clone was removed: 11 logged `cloneKeptReason: "clone_checkout_has_files"` and 1 logged `"source_identity_changed"`. Each clone's own checkout was clean (`git status --porcelain --untracked-files=all` empty). Its only history that the shared clone lacked was remote branches and release tags that the shared clone never fetched, which exist on the remote. An operator removed 11 clones by hand after verifying them. Expected behavior A re-homed clone whose checkout is clean and whose local history is in the shared clone or on the remote is deleted. A clone with real unique work is kept. Workspaces whose changes exceed the upload limits still move, because the transfer is local. Steps to reproduce 1. Use a project repository whose `.gitignore` (or the clone's `info/exclude`) contains `.paperclip/`, with a release tag on a commit that is not on any branch. 2. Re-home an isolated workspace of that repository with #17369. 3. The old worktree is removed, the empty `.paperclip/worktrees` directory stays, and `git ls-files --others --ignored --directory` lists `.paperclip/`. The clone is kept with `clone_checkout_has_files`. Without the ignore line, the local release tag keeps it with `unique_clone_history`. 4. Add a file larger than 64 MiB to a workspace. The move skips with `restore_point_too_large`. Paperclip version or commit: `master` at `aa0037a137` (rebased on `ee29e2850d`, after #17407). Deployment mode: any. Agent adapter(s) involved: not adapter-specific (core). ## What Changed - New clone reproducibility check (`cloneDisposable` in `server/src/services/isolated-repository-rehome.ts`), used before deleting a re-homed clone: - `git worktree prune`, then only the clone's own worktree may remain (`base_has_worktrees`); - no tracked changes and no untracked files (`clone_checkout_has_files`); the only untracked exception is Paperclip's compaction trash; - ignored files only inside regenerable directories (`node_modules`, `dist`, ...) or Paperclip's compaction trash. Ignored entries are listed with `--no-empty-directory`, and a non-regenerable ignored directory is expanded file by file, so empty directories such as the old worktree's `.paperclip/worktrees` parent no longer count; - only `refs/heads`, `refs/tags` and `refs/remotes` (no stash, notes or replacement refs: `unsupported_clone_refs`); - every commit reachable from local branches, local tags, non-remote reflogs, `ORIG_HEAD` and `HEAD` (so a detached checkout counts) must be on a remote-tracking ref, present in the shared clone, or on the remote. The remote check reuses `hasCommitsMissingUpstream` (now exported), which fetches the remote's branches and tags into a temporary store, and it runs only when some commit is in neither place. A local tag is accepted when its commit is on the remote. Otherwise the clone is kept (`unique_clone_history`), or kept with `managed_checkout_upstream_unavailable` when the remote cannot be read; - existing 5,000 budgets for refs, reflog entries and unique commits (`clone_history_budget`), plus at most 1,000 ignored directories outside regenerable ones, read in one Git call (`clone_file_budget`); - the log entry now includes `cloneKeptPaths`: up to five paths, worktrees, refs or commits that kept the clone. - Cleanup also checks that the clone's `origin` still matches the workspace `repoUrl` before deleting it. - Old worktree ownership (#4): it is proven by `git rev-parse --git-common-dir`, or, when that fails, by the worktree's `.git` pointer into `<clone>/.git/worktrees/`. - Orphan sweep (`sweepOrphanedIsolatedRepositorySources`): at the first reaper rotation after server start (feature on), it walks `isolated-repository-sources/<company>/<issue>/checkout`. It holds the isolated source lock and skips clones that a workspace still needs (an open workspace with matching `isolatedRemoteSource`, a closed one that is not expired with its worktree removed, a project workspace or a running service), clones with a pending move, and clones whose re-homed workspace still has its own cleanup pending. The repository URL comes only from the database: the issue's workspaces must record exactly one URL and the clone's `origin` must equal it (`unproven_repository` or `source_identity_changed` otherwise), so a rewritten `origin` is never fetched. It then applies the same check (using the shared clone of a workspace of the same issue and repository when one exists), renames to trash and deletes. It logs `event: "execution_workspace.isolated_source_orphan_cleanup"` and removes leftover trash directories from interrupted removals. It inspects at most eight issue directories per reaper sweep and resumes from the saved position in the next sweep, so slow remotes cannot hold up other cleanup. When a full walk kept any orphan, it registers a deadline one day later in the reaper's existing deadline set. Otherwise it registers nothing until the next server start. No new timer. - Restore point limits for re-home (#3): `ISOLATED_REHOME_RESTORE_POINT_LIMITS` (500,000 files, 1 GiB per file and in total, 1.5 GiB bundle, 1,000,000 commits) replaces the upload defaults for this local transfer. Verification is unchanged (exact snapshot equality). In `git-restore-point.ts`, `applyRestorePoint` now streams `git diff --binary` into `git apply` instead of buffering it (and skips the pipe when there is no diff, so no new Git version is needed). On a failure it kills both processes and waits for both. The ignored-file read sizes its buffer from the tree instead of the default limit. - Docs: `doc/recoverable-disk-cleanup.md`. ### What I found for `source_identity_changed` (#4) In #17369, cleanup ran `git rev-parse --git-common-dir` in the old worktree and used `""` when the command failed. Then it compared that to the clone's `.git` path, so any Git failure in the old worktree was reported as `source_identity_changed`. Git fails there when the old worktree's registration in the clone (`<clone>/.git/worktrees/<name>`) is gone while the directory still exists. Its `.git` file then points to a missing directory. I reproduced this by deleting the registration after a re-home: the old code keeps the clone with `source_identity_changed`, the new code removes the old worktree and the clone. I could not inspect the instance, so the exact event that removed that one registration is not confirmed. The check was a false positive in that case, and the fix proves ownership from the `.git` pointer instead. A worktree whose pointer leads outside the clone is still kept. ## Verification - `pnpm exec vitest run src/__tests__/isolated-repository-rehome.test.ts` in `server/`: 28 passed (real Git repositories, embedded Postgres). New cases: - a clean clone whose only extra history is a remote branch and a release tag on a commit outside any branch, both absent from the shared clone, with `.paperclip/` ignored in the clone: deleted; - a clone with a local branch commit that is neither upstream nor in the shared clone: kept (`unique_clone_history`, the commit is in `cloneKeptPaths`); - a clone with a tracked change, then an ignored non-regenerable file next to an ignored `node_modules`, then an untracked `build/notes.txt`: each kept (`clone_checkout_has_files` with the path), then deleted after cleanup; - a clone with a second registered worktree: kept (`base_has_worktrees`); - a clone whose detached `HEAD` holds the only copy of a commit (reflogs expired): kept (`unique_clone_history`); - an old worktree whose registration was pruned: old worktree and clone removed; - an orphan whose `origin` was rewritten is kept with `source_identity_changed`, and with `unproven_repository` once no workspace records its repository; - 1,001 ignored directories keep a clone (`clone_file_budget`); - orphan sweep: a batch of one returns a resume position and the next batch finishes the walk; a dry run changes nothing. Then an orphan whose history is in the shared clone or on the remote is deleted, and an orphan with no workspace rows and an unpushed commit is kept, with the next deadline one day later; - a workspace with 65 MiB untracked and ignored files (default capture refuses them with `restore_point_too_large`) moves with an exact snapshot match. - After rebasing on #17407 (nested clone migration, which reuses helpers from the re-home module): `isolated-repository-rehome`, `nested-clone-migration`, `execution-workspaces-service`, `execution-workspaces-routes`, `execution-workspace-reopen`, `execution-workspace-compaction`, `execution-workspace-policy`, `execution-workspace-runtime-control-conflict`, `execution-workspace-runtime-lease-route`, `git-workspace-expiration`, `git-restore-point`, `git-restore-point-expiration`: 405 passed; typecheck passes. - Related suites in `server/` (before the rebase): `git-workspace-expiration`, `git-restore-point`, `git-restore-point-expiration`, `execution-workspace-reopen`, `execution-workspaces-service`, `execution-workspace-compaction`, `heartbeat-git-restore-point-resume`, `heartbeat-project-repositories`, `git-reproducible-managed-checkouts`, `isolated-remote-workspace-source`, with the new suite: 398 passed. - `pnpm run typecheck` in `server/` passes. - Mutation checks, each failing a test: - remote proof replaced by "always unique": the unfetched-history and orphan tests fail; - remote proof replaced by "never unique": the local-commit and orphan tests fail; - `.git` pointer fallback removed: the pruned-registration test fails; - re-home limits removed: the large-file test fails; - orphan sweep removed from the reaper: the orphan test fails; - worktree count check removed: the second-worktree test fails; - regenerable-directory exception applied to untracked files: the dirty-checkout test fails; - orphan batch limit removed: the orphan test fails; - orphan URL taken from the clone again: the rewritten-origin test fails; - ignored directory budget removed: the many-directory test fails; - `HEAD` left out of the checked commits: the detached-HEAD test fails. - Dropping only `--no-empty-directory` does not fail a test, because the per-directory expansion also accepts an empty ignored directory. The two mechanisms are redundant on purpose. - Not run: the full server suite, or the change on a live instance. ## Risks - Deleting a clone now depends on reading the remote when some local commit is neither on a remote-tracking ref nor in the shared clone. If the remote cannot be read, the clone is kept. - Commits that only remote-tracking refs hold (for example a remote branch deleted upstream after the clone fetched it) are treated as not unique, as before. They were pushed once. - The orphan sweep deletes clones that no workspace refers to. A closed workspace that is expired with its worktree removed does not count as a user, because reopening it rebuilds from the shared clone or clones again. - Re-home transfers can now use up to about 1.5 GiB of memory for a single restore point payload, so a very large workspace uses more memory during its move than before. - The orphan sweep runs once per server start and then only while it keeps an orphan, so a clone that becomes orphaned later waits for the next restart. - No migration. No schema change. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file editing, Git, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 09:17:28 -07:00
DottaandPaperclip	71c04c0bfb	fix: stop idle environment cleanup polling (#17145 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent environments can leave pending teardown or unverified provider resources. > - Recovery must preserve those resources until ownership and workspace safety allow cleanup. > - Empty maintenance scans keep an idle instance and its database active. > - The existing rows, transaction signals, and scheduler can represent the actual recovery obligation. > - This pull request schedules pending cleanup and historical verification only while work remains. > - An empty or fully verified instance stops these scans without losing cleanup recovery. ## Linked Issues or Issue Description What happened? The server scanned pending environment cleanup on its maintenance tick. The run reaper also scanned it even when it recovered no runs or active leases. Historical resource verification scanned every five minutes, including when all history already had valid cleanup proof. What did you expect? Writes should wake cleanup. Pending rows, deferred orphan buffers, and unverified history should retain recovery deadlines. When none remain, these workers should make no periodic database or provider calls. Steps to reproduce Start a configured instance with no pending leases and fully verified history. Leave it idle. Observe repeated pending-cleanup and historical-verification reads from the maintenance paths. Related: #17106 supplies the preceding login-cleanup migration. #13523 improves lease-reaper query indexes. #16809 addresses reusable-sandbox recovery authority. This PR keeps those authority checks in their existing services. ## What Changed - Register two workers with the existing delivery coordinator and shared scheduler. Remove their unconditional maintenance-tick calls. Run the reaper's inline pending cleanup only after it recovers actual work. - Add commit-aware signals to the audited lease and custom-image writers. Existing transactions notify directly; other writers use one small transaction helper. No tables or migrations are added. - Derive pending cleanup deadlines from the existing row backoff and retry or claim metadata. Retain a historical retry only while an unverified candidate exists. - Register unwritten orphan buffers across runtimes that share the database handle. Buffer registration wakes recovery without a database connection. Load the existing spool at startup and retry failed or capacity-limited loads while needed. - Isolate spool failures per runtime. Attempt all buffers and persisted pending rows, then retain a retry for any failed spool. - Document producer coverage, idle behavior, and remaining blockers. Update the sleep inventory. - Repair the inherited CSS-recovery E2E test: the current build no longer emits task-page CSS. Exercise the actual lazy environment stylesheet, preserving real network failure, recovery, and bounded reload assertions. ## Verification October 24 integration update: merged current master and resolved the runtime/spool conflicts. Preserve serialized ingestion of records appended after startup. Plugin replacement failures now register deferred recovery with an idle hold; concurrent callers share one persistence attempt. The producer audit includes this new writer. - Focused cleanup selection: 270 tests now pass across five suites. The initial run lacked a built plugin SDK and executed no tests; building the SDK fixed setup. The new concurrent-flush assertion then reproduced duplicate rows; a single-flight guard fixed it and all 18 plugin integration tests passed on rerun. - Server typecheck passed. After review fixes, all 211 tests across the cleanup worker, real orphan spool, environment runtime, and plugin integration suites passed. These overlap the earlier focused selection. The rate-limit test initially expected a write after twenty idle minutes to wait; corrected it to assert that subsequent writes cannot accelerate the five-minute cadence. - Review fixes preserve retries for unreadable spool directories/records, keep historical provider checks off the startup wait while joining them at shutdown, and restore their independent five-minute minimum spacing. The complete [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/40108905033) passed on final commit `79ae1589c3c7b924db4bedb337d0c7ed3d95020b`, including all E2E shards, server/workspace tests, runner checks, typecheck, build/public-install verification, and security checks. - CI found a stale E2E asset matcher: `issue-detail-*.css` is no longer emitted. Both corrected browser tests passed against a fresh production UI build and disposable local instance. Fresh Greptile review is 5/5 on that same final commit, with all review threads resolved. Previous implementation verification: - `pnpm -r typecheck` and `pnpm build` passed. - New PostgreSQL worker suite: 12 tests passed. Covers quiet configured state, verified history, writes, outer commit and rollback, retry deadlines, restart, drain, deferred orphan buffers, spool failure, pagination, provider uncertainty, shutdown, and the idle run-reaper path. - Worker and environment runtime suites after review fix: 183 tests passed. A new assertion reproduced buffer starvation before the fix. Updated transaction fault injection and isolated simulated-instance database handles. Server typecheck also passed after the fix. - Run-recovery suites: 58 tests passed. Pending-cleanup suite: 38 tests passed after updating the logging fixture to seed real orphan recovery; an idle reaper no longer invokes cleanup. - Stop and workspace-recovery integration selection: 268 tests passed; two live-provider tests skipped by their environment gate. This overlaps the worker suite and is not an additional unique-test count. - Started the full local `pnpm test:run`, then stopped that older duplicate run after CI found the fixture mismatch. The corrected 38-test suite passed locally. The full sharded [CI matrix](https://github.com/paperclipai/paperclip/actions/runs/39678174479) passed on final commit `fc0ff3c4554ca4190a8b0b81ec06ef7065b8ad2c`, including build, typecheck, runner, server, workspace, and E2E checks. This is not a claim that the local full run completed. ## Risks - Missing a future producer signal could delay discovery until restart. The producer audit is documented in `doc/idle-environment-cleanup.md`. Independent SQL writers and another server process are outside the existing process-local notification contract. - Historical candidates blocked by ownership or provider uncertainty keep a five-minute retry until they can be verified. Provider destruction, claim fencing, Stop retention, and workspace safety remain controlled by their existing services. - This does not remove orphan-active discovery or legacy shutdown settlement from run recovery. Active custom-image expiry is also still periodic. External wake coverage does not change. - Local tests prove these workers become quiet and recover. They do not establish deployed instance suspension or fleet cost savings. ## Model Used OpenAI Codex (GPT-6), with code execution and reasoning. The exact serving model ID and context size were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-24 09:04:49 -07:00
Nicky Leach	e5c9e60c5a	Avoid session resets when reusing default task directories (#17419 )	2026-10-24 08:26:57 -07:00
Nicky Leach	f7716ec65b	Allow exact warm-state reads during retained task retirement (#17418 )	2026-10-24 08:26:48 -07:00
Nicky Leach	6b4bdc515b	Fix GitHub ownership checks for warm plugin tasks (#17417 )	2026-10-24 08:26:28 -07:00
Devin Foley devinfoley Claude Opus 5.5	515ff0f284	fix(server): report which S3 call failed in storage error events (#17424 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The server keeps attachments and other files in a storage provider. Hosted instances use the S3 provider. > - Sentry received `SignatureDoesNotMatch` errors from the S3 provider on two staging hosts. > - The events have only AWS SDK stack frames. They do not show which S3 call failed, the HTTP status, or the S3 request ID. > - Without these facts, we cannot find the cause or make a safe fix. > - This pull request records a small, fixed-shape failure record on the S3 error and sends it with the Sentry event. > - The benefit is that the next event shows the failed operation and the provider request ID, and no private data goes to Sentry. ## Linked Issues or Issue Description What happened? The S3 storage provider threw `SignatureDoesNotMatch` four times on staging. The Sentry events do not identify the S3 operation or the caller. Expected behavior A storage provider failure in Sentry shows which storage call failed and the provider's correlation fields. Steps to reproduce 1. Configure the S3 storage provider against an endpoint that rejects the request signature. 2. Read, write, check, or delete a stored file. 3. Look at the Sentry event. It has no first-party stack frame and no operation name. No public GitHub issue exists. I searched open and closed pull requests for `SignatureDoesNotMatch` and storage failure diagnostics. No pull request covers this. ## What Changed - Add `server/src/storage/failure-diagnostics.ts`. It keeps a frozen, fixed-shape record for each failed storage error in a `WeakMap`. - Add `server/src/storage/s3-failure.ts`. It wraps each S3 call and records the operation, error code, HTTP status, request ID, and SDK attempt count. It reads only data properties. A value that does not match its bounded format becomes `unknown` or `null`. - Wrap `PutObject`, `GetObject`, `HeadObject`, `DeleteObject`, `CreateMultipartUpload`, `UploadPart`, and `CompleteMultipartUpload`. - In `captureException`, add the `storage_operation` and `storage_error_code` tags and a `storage_failure` context to the event of that error only. The original error and its default grouping do not change. - Document the new context in `doc/observability.md`. ## Verification - `pnpm exec vitest run src/__tests__/storage-s3-failure-diagnostics.test.ts src/__tests__/sentry.test.ts` (in `server/`): 37 passed, 5 skipped. The skipped tests are the existing real-SDK tests, which need an environment flag. - `pnpm exec vitest run src/__tests__/storage-s3-multipart.test.ts src/__tests__/storage-local-provider.test.ts`: 8 passed. - The new tests use a local HTTP server that returns a real S3 `SignatureDoesNotMatch` XML body with a `StringToSign` field. The tests make sure that the record has the operation, code, status and request ID, and that it does not contain the private string. - I typechecked the changed source files with a scoped `tsc` project. They have no errors. I could not run the full `pnpm -r typecheck` and `pnpm build` in my sandbox: it has a 4 GiB memory limit and no `cargo` for the runner binary. CI runs the full checks. ## Risks - Low risk. The change adds a `try`/`catch` that records data and throws the same error again. Storage behavior, not-found handling, and HTTP responses do not change. - Secret-sensitive routes replace the error before reporting, so those events do not get the record. This is intended. - The record never contains the bucket, object key, endpoint, credentials, error message, or provider response body. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic. Used through Claude Code with tool use (shell, file edit) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-24 05:08:56 -07:00
Devin FoleyandPaperclip	1a79522a4e	perf: narrow run-to-issue authorization reads (#17405 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Run pages show links to issues associated with a run. > - The issue-link route checks the run's company and issue visibility before it reads those links. > - That check needs four binding fields, but the route reads the full run. > - Full runs can include large prompt, output, and result fields. > - This pull request uses the existing narrow authorization read. > - The route transfers less database data and keeps its permission checks and response contract. ## Linked Issues or Issue Description Problem or motivation `GET /api/heartbeat-runs/:runId/issues` calls `heartbeat.getRun` for a permission check. Its consumers only need the run ID, company ID, scope kind, and issue ID. The full read also fetches context, output excerpts, result data, and runtime metadata. Repeated link reads pay for that unused data. Proposed solution Use `getRunReadAccess`, the existing four-field read from #17348. Keep authentication, company access, company-scope authorization, run issue visibility, and linked-issue filtering in their current owners. Keep the existing empty response for missing and inaccessible runs. Alternatives considered A new query contract is unnecessary because the existing read provides all required fields. A permission cache could become stale. Changing the full run reader could affect consumers that need run details. Roadmap alignment This is a focused improvement to an existing route. It adds no feature, route, provider, cache, or background work. Searches for run-issue links, narrow run authorization, and `getRunReadAccess` found no open duplicate. Related work: #17348. ## What Changed - Change the issue-link route to use the existing narrow run read. - Reject full run reads in the route tests. Cover readable company and issue scopes, denied issue access, missing bindings, anonymous requests, and the missing/cross-company empty response. - Add a developer note about the read contract. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/activity-routes.test.ts src/services/heartbeat/run-state.test.ts src/__tests__/agent-live-run-routes.test.ts`: 126 tests passed. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - The route tests use the real run visibility helper with only the four binding fields. The full reader throws if a route calls it. The existing run-state suite checks the exact four-column selection and covers database wiring with disposable Postgres. - `git diff --check`: passed. Reviewed the committed diff for secrets and private identifiers before push. - Apex review: 5/5 on exact head `888a4ce9ea88bb2b284eb5e28fafc01b6f48b386`, completed successfully, with zero review threads. - Full current-head CI passed: 54 successful checks and two expected skips. [Workflow attempt 2](https://github.com/paperclipai/paperclip/actions/runs/40051399577/attempts/2). [Initial serialized shard failure](https://github.com/paperclipai/paperclip/actions/runs/40051399577/job/120744710081): the first company-history case in unchanged `agent-live-run-routes.test.ts` hit its 15-second timeout; the shard had 391 passing tests and one failure. That file, its agent route, the heartbeat state reader, and the service export are unchanged from base. The five company-history cases pass on unchanged base `9fb28d06ffebbb615f413765078aeeb7c4291311` (9.12s suite) and this head (8.29s suite). They also passed in the 126-test focused run. The timeout was not reproduced locally; its cause is not established. Only the failed shard was retried on the same head. It passed in attempt 2. This establishes recovery; the initial timeout remains in this record and its cause is unproven. - The prior local full-suite failures and baseline triage remain documented in #17384. No new local full-suite pass is claimed here. This change uses focused local tests and the full current-head CI gate. - No hosted user journey, staging deployment, production deployment, or billed savings is claimed for this follow-up. ## Risks - Run visibility depends on the canonical `scopeKind` and `issueId`. The existing helper returns those same fields. It does not use prompt or result data for access decisions. - Linked issues still pass the existing SQL visibility filter. Missing and cross-company runs still return `200 []`. Anonymous requests still fail before the run read. - The database query count stays the same. The change reduces data selected for this authorization read. - No schema or API response changes. ## Model Used OpenAI Codex, GPT-6 family. The exact deployment model ID and context window are not exposed in this session. Used reasoning, repository tools, and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 19:59:21 -07:00
Devin FoleyandPaperclip	70de7d49e9	Migrate standalone nested clones onto the shared project clone (#17407 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Each task runs in a git worktree. Extra project repositories sit in the task under `.paperclip-repositories/`. > - #17382 made new nested checkouts git worktrees of one shared clone per project repository. Existing standalone nested clones stayed as full clones. > - Long-lived tasks keep those clones for a long time. Many tasks hold the same large repository, so most of the disk is duplicate `.git` data. > - #17369 added a locked and verified move of isolated workspaces onto the shared clone, with a reaper sweep behind a setting. > - This pull request adds the same kind of locked and verified conversion for standalone nested clones in open tasks, and runs it from the same reaper path behind a new setting. > - The benefit is that the duplicate clones go away without losing any state or local history, and the disk stops filling. ## Linked Issues or Issue Description Refs #17382 #17369. Bug report described in-PR (bug report template). What happened? On one instance, 44 open task worktrees in one project each hold a standalone nested clone of the same large repository. Together they use about 25 GB. About 16 GB of that is duplicate `.git` data. Another project has 33 smaller clones. The clones are removed only when their tasks expire, and many tasks stay open for a long time. The disk keeps filling. Expected behavior Existing standalone nested clones in open tasks become worktrees of the project's shared clone, with the same state and local history, and the standalone clones are deleted. Steps to reproduce 1. Prepare a task with a nested project repository before #17382, or without a shared clone, so that `.paperclip-repositories/<name>` is a standalone clone. 2. Create the project's shared clone, as #17382 does for new tasks. 3. Observe that the old checkout stays a full clone until the task expires. Paperclip version or commit: `master` at `8549ecaf9a`. Deployment mode: any. Agent adapter(s) involved: not adapter-specific (core). ## What Changed - New `server/src/services/nested-clone-migration.ts`. It converts one standalone nested clone in an open execution workspace into a worktree of the shared clone: - It holds the source lock and workspace lifecycle lock (as git cache expiration does), the per-shared-clone lock from #17382, and the expiration lock files (`index.lock`, `HEAD.lock`, current branch ref lock) in the clone. - It requires the #17369 no-live-consumer eligibility (no live runs, leased finalizations, unreleased leases, running services, or reopen flag). It checks again before the swap. - It identifies the repository by managed checkout name, project workspace rows and the origin URL, and it requires a verified shared clone. - It brings local history into the shared clone under the checkout's own `paperclip/<id>/` prefix. The current branch becomes `paperclip/<id>/<branch>`, tracking `origin/<branch>`. Unpushed local branches become `paperclip/<id>/<branch>`. Unpushed tags become `paperclip/<id>/tags/<tag>`. A restore point carries the commits, the `HEAD` reflog and `ORIG_HEAD`. - It stages the worktree at `<name>.clone-migrating`, applies the restore point, copies the `HEAD` reflog and `ORIG_HEAD`, and moves regenerable dirs, untracked `.env` files and `.claude/settings.local.json` by rename. Old and new must then match exactly (snapshot, history, branch, created refs, moved items). - Swap: it renames the clone to `<name>.clone-standalone`, runs `git worktree move` and `git worktree repair`, updates a saved restoration recipe that lists the checkout, and then renames the old clone to trash and deletes it. - A `nestedCloneMigrationPending.<name>` metadata record makes recovery possible. A crash before the swap rolls back (moves items back, removes the staged worktree and the created branches). A crash after the swap rolls forward. - Skip reasons (nothing changes): `live_or_unresolved_consumer`, `unknown_managed_checkout`, `local_source_checkout`, `foreign_origin`, `no_shared_base`, `shared_base_repo_mismatch`, `unsupported_base_git_config`, `nested_clone_has_stash` (stashes are skipped, because `refs/stash` would be shared by every worktree), `nested_clone_has_worktrees`, `nested_clone_has_hooks`, `unsupported_local_git_config`, `unsupported_local_git_state`, `unsupported_info_exclude`, `unsupported_nested_clone_refs`, `unsupported_index_flags`, `unique_branch_reflog_commits`, `nested_clone_ref_conflict`, `snapshot_mismatch`, `history_mismatch`, restore point limits, and `nested_migration_recovery_conflict`. - Reaper: `sweepTerminalWorkspaces` runs the conversion for open workspaces when `PAPERCLIP_NESTED_CLONE_MIGRATION_ENABLED=true` (default off). `PAPERCLIP_NESTED_CLONE_MIGRATION_DRY_RUN=true` only reports. Each attempt logs `event: "execution_workspace.nested_clone_migration"` with outcome, reason, branch, candidate bytes and reclaimed bytes. There is no new timer. A retry deadline is registered on the existing reaper deadlines only for a pending record or a transient skip (live consumer, busy lock, changed row). A workspace is attempted again at most hourly. A workspace without standalone clones costs one directory listing. - `isolated-repository-rehome.ts`: I exported the shared pieces (eligibility, locks, safe base config check, transfer plan, history boundary, snapshot and history keys, item moves, rollback and verification) instead of copying them. Re-home behavior does not change. - `nested-repository-worktrees.ts`: I exported the stale registration cleanup and added the `moveNestedRepositoryWorktree` (move and repair) and `trackNestedWorktreeUpstream` helpers. - Docs: `doc/recoverable-disk-cleanup.md` (new section about the conversion) and `doc/DEVELOPING.md` (settings). ## Verification - `cd server && pnpm exec vitest run src/__tests__/nested-clone-migration.test.ts`: 19 passed. It uses embedded Postgres and real git repositories, and covers: - Conversion with an unpushed commit, a second branch with unpushed commits, a lightweight tag and an annotated tag, staged and unstaged edits, a deleted file, an untracked file with mode 0600, an ignored file, `node_modules`, `.env` and the settings file. The result is a worktree of the shared clone (`--git-common-dir`), with the same HEAD, status, staged content and reflog, the owned branch tracking `origin/main`, and the history under the task's prefix. No other branch in the shared clone changes. The old clone is gone and reclaimed bytes are reported. - Skips with nothing changed: stash, live run, foreign origin, unknown checkout, no shared clone, local excludes, and a change between capture and verification (hook). - Two tasks convert concurrently against one shared clone. The test holds the shared-clone lock and shows that neither conversion advances. Then they run one at a time (peak 1), and each keeps its own branches. - Crash recovery at `afterAside`, `afterMove` and `afterFinalize` (roll forward), and before the swap (roll back, then convert). A dry run reports the pending record and changes nothing. - Expiration and restore of a converted checkout work afterwards. Unpushed branches in a converted checkout block expiration (`unpushed_managed_checkout_branches`). - Reaper: disabled by default; dry run changes nothing and registers no deadline; a live run registers a retry deadline that survives a second sweep; enabled conversion leaves no deadline. - Commits that only the clone's `origin` remote-tracking refs hold are kept as `paperclip/<id>/remotes/origin/<branch>`. - A pending conversion is finished after the workspace is archived, and forgotten (keeping unpushed branches) when the worktree is gone. - A filter planted in the shared clone right before the worktree add is refused (`unsupported_base_git_config`) and never runs. - Only checkouts named after configured project repositories are considered; at most 8 per sweep, pending first, rotating (unit test). - Related suites (run before review fixes; the nested, re-home and execution workspace service suites were rerun after them): `pnpm exec vitest run` on execution-workspaces-routes, execution-workspaces-service, git-reproducible-managed-checkouts, git-restore-point (all 5 files), git-workspace-contents, git-workspace-expiration, git-workspace-expiration-eligibility, heartbeat-git-restore-point-resume, heartbeat-project-repositories, isolated-repository-rehome, nested-clone-migration: 15 files, 393 tests passed. - `cd server && pnpm run typecheck` (`tsc --noEmit`): passes. - Mutation checks. Each mutation makes the targeted test fail, and I reverted each one: - Remove both eligibility checks: the live run test fails. - Remove the snapshot comparison: the capture/verify change test fails. - Disable roll forward after the swap: the recovery tests fail. - Drop the `paperclip/<id>/` prefix from transferred branches: the conversion and concurrency tests fail. - Remove the shared-clone lock: the concurrency test fails. - Drop remote-tracking refs from the plan, the deadline restore during backoff, archived recovery, or the config check before the worktree add: each targeted test fails. - Not run: the full server suite and a run on a live instance. ## Risks - The setting is off by default. Enable a dry run first and read the `execution_workspace.nested_clone_migration` logs. - A clone that a run writes to between verification and the swap rename can lose those writes. This window is the same as in #17369. Locks, the eligibility check and the git lock files make it small. - A crash in the short time between the rename aside and `git worktree move` can collide with a run that recreates `<name>`. In that case the conversion stops with `nested_migration_recovery_conflict` and keeps both copies. Expiration keeps such a workspace because of the `.clone-` names. - Stashes, annotated tag messages (the tagged commit is kept as a branch), custom hooks, custom local config and per-branch reflog-only history are not converted. Clones with them are skipped with a named reason. Only unpushed branches and tags get a name in the shared clone. A pushed local branch name is not kept, but its commits stay on the remote. - The conversion fetches the shared clone first. A failed fetch is ignored, and then the restore point can be larger. - Pushing from a converted checkout needs an explicit destination, as for all #17382 nested worktrees. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file editing, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 19:52:07 -07:00
Nicky Leach	aa0037a137	Support Codex warm plugin task reuse and recovery (#17390 )	2026-10-23 19:14:14 -07:00
Devin FoleyandPaperclip	0c3b5ce273	perf: filter sandbox provider lookups in Postgres (#17395 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Sandbox execution selects a provider from installed plugin manifests. > - Driver-key lookup reads every plugin row before it finds a matching declaration. > - Repeated lookups transfer manifests for unrelated plugins. > - The registry owns this database read and can filter candidates before returning rows. > - This pull request adds that filter and keeps the existing provider selection checks. > - The returned data now grows with matching providers instead of the full catalog. ## Linked Issues or Issue Description Problem or motivation `resolvePluginSandboxProviderDriverByKey` calls the unfiltered registry catalog on each lookup. The caller only needs plugins with a sandbox-provider declaration for one driver key. Unrelated plugin manifests still cross the database connection and are decoded in the server. Proposed solution Add an optional sandbox-provider driver-key filter to `pluginRegistryService().list`. Match the key and kind on the same JSON declaration in Postgres. Pass the key from the existing resolver. Keep full matching rows, install order, status checks and worker checks. Alternatives considered A catalog cache would need an invalidation contract for plugin updates and independent writers. This change keeps fresh database reads. Returning only one SQL row could skip a later eligible provider when an earlier worker is unavailable. The filter therefore keeps all matching candidates. Roadmap alignment This is a small performance fix to existing plugin execution. It adds no product feature or provider. Searches for sandbox/driver lookup and plugin catalog/filter work found no duplicate PR or issue. ## What Changed - Add an optional JSONB containment filter to the existing registry list read. - Use it in sandbox-provider lookup by driver key. The unfiltered catalog call keeps its existing behavior. - Add real-Postgres regression coverage and a short developer note. ## Verification - Four focused provider-selection/recovery/seam suites passed: 27 tests. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - The new Postgres tests seed large unrelated manifests and verify the actual resolver SQL filters rows in the database. They cover key/kind matching within one declaration, unfiltered reads, install order, disabled and unavailable workers, live status/declaration changes, unknown keys and bound unusual input. - The filter keeps the existing SQL call count. It does not add a network call, cache, timer, or background worker. - `git diff --check`: passed. Reviewed the staged diff for secrets and private account data before push. - Full PR CI passed on `6f97c984a931416b6347e9885d840557c54d8aab`: 54 successful checks and two expected skips. [Workflow evidence](https://github.com/paperclipai/paperclip/actions/runs/40041294925). - Apex review: 5/5 on that exact head, completed cleanly, with no comments or unresolved threads. Merge state is clean. - The prior local full run on parent work is documented in #17384. It did not pass because of test setup, host Git-default assumptions and transient failures. Those results were preserved and triaged against unchanged code. This change uses focused local checks and full current-head CI; no new local full-suite pass is claimed. - No hosted user journey, live provider throughput, production deployment or billed savings is claimed. ## Risks - JSONB containment checks the stored manifest before returning rows. It still scans candidate metadata in the database; no index or SQL-call reduction is claimed. - Both key and kind must match the same declaration. The regression fixture includes a plugin with those values on different declarations. - Selection still uses the existing install-order and live readiness/worker checks. Matching disabled records remain visible to callers that do not require a running worker. - No schema, API response, provider protocol, file-copy behavior or scheduling changes. ## Model Used OpenAI Codex, GPT-6 family. The exact deployment model ID and context window are not exposed in this session. Used reasoning, repository tools and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 18:31:36 -07:00
Devin FoleyandPaperclip	8549ecaf9a	Re-home isolated repository workspaces onto the shared project clone (#17369 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Each isolated task workspace is a Git worktree on the server disk, and the terminal workspace reaper and git cache expiration keep that disk usage bounded > - For some days, workspace preparation gave each isolated `git_worktree` workspace its own full clone under `isolated-repository-sources/`, with the worktree inside it. #16963 removed that creation path, but the existing clones stay, and restoring an expired workspace still recreates its private clone > - On one instance, 30 such clones use about 23 GB: 12.1 GB of `.git` data, 5.2 GB of unused default-branch checkouts and 5.75 GB of real worktrees. 28 of them back open workspaces, and the disk is nearly full > - This pull request moves each such open workspace onto the project's shared managed clone with its exact state, deletes the private clone, and makes restore use the shared clone > - The benefit is that disk use drops to one shared clone per project repository, with no lost work and no change for workspaces that cannot move safely ## Linked Issues or Issue Description Refs #16963. Bug report described in-PR (bug report template). What happened? On one instance, 30 per-issue clones under `isolated-repository-sources/<companyId>/<issueId>/checkout` use about 23 GB. 28 of them back open workspaces. No process moves these workspaces to the shared project clone that new workspaces use, and git cache expiration does not remove a clone while its workspace is open. Reopening an expired workspace whose recipe has `sourceKind: "isolated_remote"` clones the full repository again. Separately, a restore point with a non-default file mode cannot be restored: the reference read back from jsonb has reordered `modes` keys, so `loadGitRestorePoint` fails with `restore_point_manifest_mismatch`. Expected behavior An open workspace with a private clone moves to the shared project clone with the same branch, HEAD, index, worktree, untracked and ignored files, and file modes. The private clone is then deleted. Reopening an expired workspace uses the shared clone when one exists. A restore point with non-default file modes restores. Steps to reproduce 1. Create an isolated `git_worktree` workspace with `materializeIsolatedRemoteWorkspaceSource` and record `isolatedRemoteSource` metadata, as preparation did before #16963. 2. Keep its issue open. The clone stays, because no sweep moves or removes it. 3. Expire the workspace with a restore point that has a file with mode `0600`, then reopen it. The private clone is created again, and the restore fails with `restore_point_manifest_mismatch`. Paperclip version or commit: `master` at `ec2e0d6407`. Deployment mode: any. Agent adapter(s) involved: not adapter-specific (core). ## What Changed - New `server/src/services/isolated-repository-rehome.ts` with `rehomeIsolatedRepositoryWorkspace`. It holds the isolated source lock, the `execution_workspace_lifecycle:<id>` advisory lock and expiration-style Git lock files (`index.lock`, `HEAD.lock` and the branch ref lock in the clone). - Eligibility uses the safety checks of git cache expiration that matter while a workspace stays open: no live or unfinished relevant runs, no unsettled native finalizations, no unreleased environment leases, no running runtime services, no pending reopen, and no other workspace on the same path. It does not require a closed or idle issue tree. The check runs again inside the update transaction. - The target base is the project's existing shared managed clone for the same `repoUrl` (the paths `ensureManagedProjectWorkspace` uses, verified with the same origin and storage checks as restoration). - The move captures a restore point of the old worktree. The upstream boundary is commits that both repositories have, so the bundle is small. It imports the bundle into the shared clone, creates `refs/heads/<branch>` or fast-forwards it from an ancestor, and adds the worktree at `<base>/.paperclip/worktrees/<branch>`. Then it applies the restore point. It copies the `HEAD` reflog and `ORIG_HEAD`, copies the branch's upstream config when it creates the branch, and moves regenerable directories, untracked `.env`/`.env.` files, `.claude/settings.local.json` and nested `.paperclip-repositories/` checkouts by rename. Standalone nested clones move as is. Nested checkouts that are worktrees of their own shared clone (#17382) move too, followed by `git worktree repair` in that clone so the registration follows the new path, and their `paperclip/<hash>/` branch is renamed to the new path's prefix so ownership checks still pass. Entries inside a moved nested checkout (for example its `node_modules`) move with it. Rollback and interrupted-move recovery repair them again after moving them back, and verification checks that each one still resolves to the same shared clone. - Verification: the restore point snapshot (HEAD, index, worktree, ignored files, modes) of the new worktree must equal the old one before and after the move. The reflog and `ORIG_HEAD` must match, and every moved item must exist. Only then does one transaction set `cwd`/`providerRef`, drop `isolatedRemoteSource` and `gitCacheExpiration`, rewrite a saved active restoration recipe to `managed_project`, and record `isolatedRepositoryRehome: { at, from, to, cleanedAt }`. A failure before that removes the new worktree, moves items back, restores the branch, and leaves the old worktree untouched. - Safety of the checkout in the shared clone: the move skips with `unsupported_base_git_config` when the shared clone's local config sets filter drivers, includes, attribute files, worktree config or command settings, or when it has `info/attributes`. It checks before planning and again right before `git worktree add`. - Crash safety before the update: the move records `isolatedRepositoryRehomePending` (base, target, branch, head, previous branch tip, planned moves) before the first change to the shared clone. The next attempt undoes an interrupted move under the same locks: it moves items back, removes the new worktree only when it is proven to be this move's (a linked worktree of the shared clone on the issue branch that no workspace row references), and restores the branch. A live failure removes the target only after this attempt created it. When an item exists in both places, rollback stops with `rehome_rollback_conflict` and keeps the target and the record, so no copy is lost. - Cleanup after the update: remove the old worktree (rename to trash, then delete, then `worktree prune`). Delete the clone (rename to trash, then delete) only when no other workspace, project workspace or running service uses it, it has no other worktree, its checkout has no changes or non-regenerable files, its refs are only branches, tags and remote-tracking refs, and every local commit is on a remote-tracking ref or present in the shared clone. Cleanup reads at most 5,000 refs, reflog entries and unique commits (`clone_history_budget`). `cleanedAt` records completion. A crash after the update leaves a pending record that the next pass finishes, also after the workspace is archived: the reaper scan includes archived rows with unfinished re-home cleanup when the feature is on. - Reaper integration in `sweepTerminalWorkspaces`: behind `PAPERCLIP_ISOLATED_SOURCE_REHOME_ENABLED` (default `false`) and `PAPERCLIP_ISOLATED_SOURCE_REHOME_DRY_RUN`, only when git cache expiration is enabled. It logs `event: "execution_workspace.isolated_source_rehome"` with outcome, reason, candidate bytes and reclaimed bytes. A skipped workspace gets a retry deadline one hour later in the reaper's existing deadline set. No new timer. - Restore: `reopenClosedIsolatedExecutionWorkspaceForIssue` calls `rehomeExpiredIsolatedRestoration` first. For an expired workspace with an `isolated_remote` recipe, it uses the shared clone when one exists and the branch there is absent or at the saved commit. It keeps the old behavior for recipes with worktree-backed nested checkouts on branches, because those branch names are bound to the old path. It also requires that the shared clone has every base commit (prerequisite) of the saved restore point bundle. It rewrites the row and recipe to the shared clone and the standard worktree path, then the existing restore runs (`ensureGitRestorePointCommit`, `ensureSavedGitCommit`, apply). Without a shared clone, the old behavior stays. - Fix: restore point `modes` comparison in `git-restore-point-storage.ts` compares by path and mode, so jsonb key order does not matter. - Small exports: `allocatedBytes`, `expirationScope`, `managedSourcePaths`, `verifiedGitCacheSource`. - Docs: new "Isolated repository sources" section in `doc/recoverable-disk-cleanup.md`, new settings in `doc/DEVELOPING.md`. Deviations from the requested design: - The sweep does not create a missing shared clone. When none exists, it skips with `no_shared_base`, because a new network clone in a cleanup path can use more disk. Restore follows the same rule and keeps the old behavior. - History moves through the restore point bundle, not a separate fetch from the clone. The bundle contains HEAD and every reflog/`ORIG_HEAD` commit that the shared clone does not have. The move then checks that all of them are present. - The standard location is `<base>/.paperclip/worktrees/<branch>`. A worktree outside the clone's standard location is skipped (`nonstandard_worktree_location`). - Lock files use a token derived from the workspace id, so a crashed attempt's locks are recovered without a database marker. - A clone that another open workspace still uses is kept and marked `cloneRetained: "base_has_consumers"`. That workspace's own move removes it later. Skip reasons: `live_or_unresolved_consumer`, `unsupported_base_git_config`, `rehome_rollback_conflict`, `nested_branch_conflict`, `invalid_rehome_record`, `interrupted_rehome_pending` (dry run only), `lifecycle_busy`, `source_busy`, `workspace_changed`, `no_shared_base`, `shared_base_repo_mismatch`, `missing_worktree`, `nonstandard_worktree_location`, `source_identity_changed`, `branch_identity_changed`, `unproven_worktree_admin`, `unsupported_worktree_git_state`, `unique_worktree_refs`, `unknown_worktree_git_logs`, `unsupported_local_git_state`, `unique_git_refs`, `unsupported_index_flags`, `unsupported_branch_name`, `invalid_restoration_recipe`, `unsupported_restoration_recipe`, `target_worktree_exists`, `base_branch_checked_out`, `base_branch_diverged`, `untransferable_ignored_files`, `untransferable_managed_checkouts`, `worktree_git_busy`, `head_changed`, `history_not_transferred`, `transfer_conflict`, `snapshot_mismatch`, `history_mismatch`, `transfer_incomplete`, and restore point reasons such as `untracked_nested_repository`, `submodules_unsupported`, `filtered_paths`, `restore_point_too_large`. Clone kept reasons: `base_has_consumers`, `base_has_live_services`, `base_has_worktrees`, `clone_checkout_has_files`, `unsupported_clone_refs`, `unique_clone_history`, `clone_history_budget`. ## Verification - `pnpm exec vitest run src/__tests__/isolated-repository-rehome.test.ts` in `server/`: 18 passed. It uses real Git repositories and embedded Postgres. It covers: - a move with an unpushed commit, staged and unstaged edits, a deletion, an untracked file with mode `0600`, an ignored file, `node_modules`, `.env`, `.claude/settings.local.json` and a nested `.paperclip-repositories` checkout. Snapshots, branch, HEAD, `git status` and reflog match, the row points to the new path without `isolatedRemoteSource`, and the old worktree and clone are gone; - skips with nothing changed on disk or in the database: live run, branch in the shared clone at an unrelated commit, no shared clone, an untracked nested repository, and a concurrent change between capture and verify (hook). The new worktree is removed and the old one keeps its moved items; - the clone is kept while another workspace references it; - a worktree-backed nested checkout moves with the task worktree, stays registered in its shared clone at the new path (not prunable), and keeps its changes, with its branch renamed to the new path's prefix. On rollback it returns to the old path with its registration repaired and its branch name restored. A standalone nested checkout moves together with its own ignored `node_modules`; - a file recreated in the old worktree before verification stops the rollback and keeps both copies, then the move completes once the conflict is gone; - a hostile `filter..smudge` in the shared clone's config makes the move skip without running it; - a clone with more than 5,000 refs is kept, and is deleted once the refs are gone; - a move interrupted before the database update is reported (not undone) by a dry run, then undone and completed on the next attempt; - a target that another party creates right before `git worktree add` is never deleted; - a crash after the database update; after the row is archived, a later reaper sweep removes the old worktree and clone; - restore keeps the private-clone fallback when the shared clone lacks the restore point's base commits; - disabled by default (both flags needed), dry run changes nothing, and the retry deadline is registered; - reopening an expired `isolated_remote` workspace rebuilds it in the shared clone with the restore point applied and no clone created. - After the round 4 fixes, the related suites (`git-workspace-expiration`, `git-restore-point-expiration`, `execution-workspace-reopen`, `execution-workspaces-service`, `heartbeat-git-restore-point-resume`, `heartbeat-project-repositories`, `isolated-remote-workspace-source`) with the new suite: 326 passed. - After rebasing on #17382, the related suites (adding `heartbeat-project-repositories`) with the new suite: 374 passed. - After the first review fixes, the related suites (`git-workspace-expiration`, `git-restore-point`, `git-restore-point-expiration`, `execution-workspace-reopen`, `execution-workspaces-service`, `execution-workspace-compaction`, `heartbeat-git-restore-point-resume`, `isolated-remote-workspace-source`) with the new suite: 301 passed. - Related suites in `server/` (first commit): `pnpm exec vitest run` on `git-workspace-expiration`, `git-workspace-expiration-eligibility`, `git-restore-point`, `git-restore-point-expiration`, `git-restore-point-expiration-config`, `git-restore-point-ledger`, `git-restore-point-ledger-config`, `execution-workspace-reopen`, `execution-workspaces-service`, `execution-workspace-compaction`, `heartbeat-git-restore-point-resume`, `isolated-remote-workspace-source`, `isolated-remote-workspace-source-lock`, `git-workspace-base-storage`, `git-reproducible-managed-checkouts`, `git-expiration-auth-fallback` and `workspace-runtime`: 550 passed, 2 failed. The 2 failures are in `workspace-runtime.test.ts` (base ref resolution) and also fail on unmodified `master` on the same machine. - `pnpm run typecheck` in `server/` passes (after `pnpm -r --filter "./packages/*" build`). - Mutation checks. Each reverted guard fails one test in the new suite: - eligibility always true: "keeps a workspace with a live run" fails; - snapshot verification removed: "removes the new worktree and keeps the old one when the workspace changes during the move" fails; - clone reference check removed: "keeps the clone while another workspace still references it" fails; - restore re-home removed from reopen: "restores an expired isolated workspace into the shared project clone" fails; - old `JSON.stringify` modes comparison: the same restore test fails; - interrupted-move recovery removed: the interrupted-move test fails; - target existence check before `git worktree add` replaced by assumed ownership: the foreign-target test fails; - bundle prerequisite check removed: the stale shared clone restore test fails; - archived-row cleanup scan removed: the crash test fails; - dry run check before interrupted-move recovery removed: the interrupted-move test fails; - `git worktree repair` after the forward move removed: both nested worktree tests fail. The repair after moving back removed: the rollback test fails; - nested branch rename forward or back removed: the matching nested test fails; - rollback conflict check removed: the reappearing-file test fails; - covered-entry filter removed: the full re-home test fails; - shared clone config check removed: the hostile filter test fails; - ref budget removed: the many-ref test fails. - Not run: the full server suite, and the move on a live instance. ## Risks - Workspace paths change for re-homed workspaces. Agents that resume an old session can remember the old path. The run's working directory comes from the updated row, but text in a resumed conversation can still name the old path. - The feature is off by default. It runs only with `PAPERCLIP_ISOLATED_SOURCE_REHOME_ENABLED=true` and git cache expiration enabled. Dry run is available. The restore change is not behind the flag: reopening an expired private-clone workspace now uses the shared clone when one exists. - A run can start between the last eligibility check and the database update. The update transaction checks eligibility again, and the old worktree is deleted only after another check. This is the same window that git cache expiration accepts. - Imported restore point objects stay in the shared clone as unreachable objects after a failed attempt until Git garbage collection removes them. Base expiration keeps a base with unreachable objects, so this only delays base expiration. - A workspace that cannot move (for example `no_shared_base`) is retried each hour while the feature is on. Each retry runs a few Git commands for that workspace. - No migration. No schema change. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file editing, Git, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 18:25:26 -07:00
Nicky Leach	100477e887	Avoid live workspace inspection in task context reads (#17396 )	2026-10-23 17:05:06 -07:00
Devin FoleyandPaperclip	5a7e740d82	fix: validate managed AI accounts before verification allocation (#17384 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - A saved agent must pass execution configuration checks before it can run. > - Managed AI account selection belongs to the existing company-scoped account service. > - Verification allocated its execution environment before it checked that account. > - A missing default or revoked account could create a sandbox on each retry. > - This pull request checks account selection before allocation and keeps the final check. > - Invalid account settings now fail before they cause avoidable provider work. ## Linked Issues or Issue Description What happened? Saved-agent lifecycle verification allocates a test execution target before `withManagedAiProbe` selects the managed AI account. An unavailable account can cause sandbox creation and cleanup on every retry. Expected behavior Check account compatibility, responsible-user access, default selection, grant health and agent eligibility before target allocation. Check current account access again before the real probe. Steps to reproduce 1. Configure a saved OpenCode agent with an OpenRouter personal-default binding. 2. Leave that user's OpenRouter default empty, or revoke the selected grant. 3. Let lifecycle verification use a sandbox execution environment. 4. Before this change, verification can allocate the target and then reject the account. After this change, account selection rejects the configuration before allocation. Paperclip version or commit Source baseline `b5bf86ea09042a864b60910f80c8fdf7e64e9ef0`. Deployment mode Server with Postgres and a remote execution environment. Tests use disposable local Postgres and a controlled target allocation boundary. Searched public PRs and issues for verification, connection preflight, probes and lifecycle work. No duplicate fix found. Related public issue #16128 concerns credential probing and reauthorization. This change does not alter reauthorization or credential refresh. ## What Changed - Call the existing managed account selector with the resolved adapter and model before allocating a verification execution target. - Keep router resolution, secret handling, final account selection and target release with their existing owners. - Add six Postgres tests and document the saved-agent ordering contract. ## Verification - Focused verification suites: 30 tests passed across two files. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - Full PR CI passed on `ddc0879d46b60310c533e6d3c4b8acdd178d979c`: 54 successful checks and two expected skips. [Workflow evidence](https://github.com/paperclipai/paperclip/actions/runs/40033261909). This includes typecheck, tests, build and browser checks. - Apex review: 5/5 on that exact head, clean check, no comments or unresolved threads. Merge state is clean. - Additional local `pnpm test:run` is still running. The complete local suite is not yet a pass claim. - The first focused run passed the missing-default case but failed five setup hooks because fixtures reused a unique company prefix. Each fixture now has a distinct prefix. The complete focused rerun passed. - Tests use real account selection, memberships, grants, personal defaults, encrypted fixture credentials and managed runtime preparation. Only execution-target allocation and the provider probe are controlled boundaries. No live provider request is required. - Tests prove that missing defaults, revoked grants and incompatible models do not allocate a target. A valid account still reaches the managed probe. Revocation during allocation still fails the final account check and releases the target. Probe failure also releases it. - `git diff --check`: passed. Reviewed the diff for secrets and private account data before push. - No hosted login or production deployment is claimed for this follow-up. ## Risks - Adds one read-only account selection to each managed saved-agent verification attempt. The selector can read company accounts and grants. It makes no provider request and creates no execution target. - Final selection stays fresh. A successful preflight does not reserve an account or cache authorization. - Missing configuration still follows the existing durable lifecycle retry schedule. This change prevents needless allocation; it does not remove those database retries. - No schema, scheduler, provider-account choice, capacity or public API changes. ## Model Used OpenAI Codex, GPT-6 family. The exact deployment model ID and context window are not exposed in this session. Used reasoning, repository tools and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 16:31:18 -07:00
Devin FoleyandPaperclip	ec2e0d6407	Share one clone across nested project repository checkouts (#17382 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A project can attach extra repositories, and every task worktree gets a managed checkout of each one under `.paperclip-repositories/<name>` > - Each of these nested checkouts is a full clone. The clone uses `--dissociate`, so no objects are shared between tasks > - On one instance, 44 task worktrees held nested checkouts of the same repository: 24 GB of 27 GB of worktrees. Each new task added about 0.5 GB, and the disk filled twice in one day > - Main task worktrees already avoid this, because they are git worktrees of one managed project clone > - This pull request makes nested checkouts git worktrees of one shared clone per project and repository, and adapts expiration, restore points, restore and cleanup to them without weaker safety checks > - The benefit is that repository objects are stored once, and a new task adds only its working files ## Linked Issues or Issue Description No public issue exists. This is the in-PR bug report. What happened? Every task in a project with extra repositories got a full, independent clone of each extra repository under `.paperclip-repositories/`. On one instance this used 24 GB of 27 GB of task worktrees (16 GB of it `.git` data) and filled the disk twice in one day. Expected behavior Nested checkouts of the same repository share one object store, like main task worktrees share the managed project clone. Disk use per new task is only its working files. Steps to reproduce 1. Create a project with a primary repository and a second repository that has a repository URL and no local folder. 2. Run several tasks in isolated git worktrees in that project. 3. Look at `<task worktree>/.paperclip-repositories/<name>/.git`: each one is a full directory with its own copy of every object. Paperclip version or commit: `master` at `ff835a3fc6`. Deployment mode: any. Agent adapter(s) involved: not adapter-specific (core). Related: #15662 (SSH restores skip nested worktrees) touches a different path; this PR does not change adapter sync. ## What Changed - New `server/src/services/nested-repository-worktrees.ts`. It finds the shared clone (the managed project checkout for the repository URL, from `ensureManagedProjectWorkspace`), adds nested worktrees, restores them, removes their registrations, and serializes all of these operations and fetches per shared clone with an in-process lock. - `prepareProjectRepositoryWorkspaces` takes a `sharedBase` (company and project). For a repository without a configured local folder, it fetches the shared clone and adds a worktree at the same `.paperclip-repositories/<name>-<key>` path. Each worktree gets its own branch `paperclip/<id>/<branch>` that tracks `origin/<branch>` (`<id>` derives from the worktree path), or a detached HEAD for a tag or commit ref. An existing worktree must belong to the project's shared clone. Configured local-folder copies and existing standalone clones are unchanged. Moving a worktree to `detached-repositories` repairs its registration. Heartbeat preparation and fresh-start reopening pass `sharedBase`. - `assertReproducibleManagedCheckout` and `provenManagedCheckoutHistory` accept worktree-backed checkouts. They verify the worktree link, its admin directory, the shared clone identity and origin, and that HEAD is detached or on a branch under the worktree's own prefix. History checks scope to HEAD, the per-worktree HEAD reflog, ORIG_HEAD and the owned branches with their reflogs. Other tasks' branches are not inspected. Stashes, unsupported refs and custom config in the shared clone still block. The upstream proof reads objects from the common git directory. - Restoration contents record `worktree: { base, admin }` for a worktree-backed checkout. Validation checks its shape and the owned branch name, and every use checks the base against the project's expected clone paths. - Expiration locks for a worktree-backed checkout sit in the shared clone (`.git/worktrees/<admin>/index.lock`, `HEAD.lock`, and the owned branch ref lock). `gitCacheExpirationLockLocations` is now async. - `finishGitCacheTrash` removes the expired worktree's registration and deletes its owned branch only at the recorded commit. - `restoreGitWorkspaceContents` recreates a worktree-backed checkout from the shared clone, and clones the shared clone again if it is gone. It imports the restore point into the shared clone, adds the worktree on the saved branch and commit, writes reflog-only commits back into the new HEAD reflog, and applies the restore point. - `cleanupExecutionWorkspaceArtifacts` (terminal reaper archival, close and delete) collects the task's nested worktrees first. After the task directory is gone, it removes their registrations and deletes owned branches only when every commit is on a remote-tracking ref. An unpushed branch stays in the shared clone. - Review follow-ups. Owned branches always track origin, including names with dots, and the config check accepts their tracking keys. A ref held by a retained detached checkout gets `paperclip/<id>/r<n>/<branch>`. Restore derives the upstream from the saved source ref. Each new worktree fetches with `--prune`, refreshes `origin/HEAD` when no ref is configured, and strips `refs/heads/` from branch refs. Git commands on the shared clone run with hooks disabled. The worktree `gitdir` pointer is read through a bounded, non-following, non-blocking handle, and more than 64 owned branches stop the proof before reflog reads. - Docs: new "Nested project repository worktrees" section in `doc/recoverable-disk-cleanup.md`, and updates in `doc/DEVELOPING.md`. ## Verification - `pnpm -r --filter "./packages/*" build`, then `pnpm run typecheck` in `server/`: passes. - `pnpm exec vitest run` in `server/` for `git-workspace-expiration`, `git-reproducible-managed-checkouts`, `git-workspace-contents`, `git-restore-point`, `heartbeat-project-repositories`, `execution-workspaces-service`, `execution-workspaces-routes`, `execution-workspace-compaction`, `workspace-compaction`, `heartbeat-git-restore-point-resume`, `managed-repository-checkout`, `task-workspace-repositories`, `native-workspace-sync`, `heartbeat-managed-clone-credentials`, `services/heartbeat/workspace-preparation`, `project-repositories-persistence`, `git-workspace-base-storage`, `agent-directory-probe`, `workspace-runtime`, `workspace-preparation-history` and `native-completion-feedback`: 866 passed, 2 failed. The 2 failures are in `workspace-runtime.test.ts` (`rejects explicit main on a master-only remote...` and `does not diagnose a missing remote branch...`). They fail the same way on unmodified `master` on this machine. - New tests (embedded Postgres, real git repositories) in `git-workspace-expiration.test.ts`, section "nested checkouts as worktrees of a shared base": - Two tasks get worktrees of one shared clone. `--git-common-dir` points to it, each task has a distinct `paperclip/<id>/main` branch, upstream is `origin/main` and the remote is the repository URL. - Concurrent preparation of three tasks succeeds, and preparation waits while the shared clone lock is held. - A legacy standalone clone is still adopted, and no shared clone is created for it. - Expiring a clean nested worktree removes only its registration and branch. The sibling worktree and branch stay. Reopening recreates the worktree from the shared clone on the same branch and commit, with its local settings. - Reopening clones a deleted shared clone again. - Unpushed commits and uncommitted files block plain expiration, are saved in a restore point and are restored exactly. - Reflog-only history is saved and written back into the restored HEAD reflog, and it survives `gc --prune=now` of the shared clone. - Stashes, a checked-out branch outside the worktree's prefix, and unpushed owned branches or owned reflog entries outside HEAD history block expiration. - Another task's unpushed branch does not block, and it stays after expiration. - A worktree of a foreign clone at the nested path is rejected by preparation and by the prover. - Expiration locks in the shared clone are recovered after a process exit. - Deleting a task removes only its nested registrations, keeps its unpushed branch and deletes the sibling's pushed branch when the sibling is deleted. - Follow-up tests: dotted branch tracking, a ref switched away and back with a retained checkout, a changed remote default branch, a renumbered branch restored next to a same-named remote branch, a fully qualified branch ref after the remote advances, a branch deleted upstream, planted `post-checkout` and `reference-transaction` hooks, more than 64 owned branches, and a FIFO, oversized or symlinked `gitdir` pointer. Each fix was mutation-checked. - `heartbeat-project-repositories.test.ts`: a repository-URL-only sibling is now a worktree of the shared clone on an owned branch, and its detached copy is still a valid checkout after its repository leaves the project. In `workspace-preparation.test.ts`, the publication guard now runs before any worktree exists. - Mutation checks. I reverted one guard at a time and confirmed that a test fails: the owned-HEAD check, the owned-branch check in restore point mode, the owned-reflog check, the shared-clone identity check, the pushed-only branch deletion on cleanup, the cleanup release hook, the registration removal on expiration, the reflog-only history write-back on restore, and the shared clone lock. The owned-branch-tip check alone is also covered by the owned-reflog check, so removing only one of the two keeps the tests green. - Not run: the full server test suite, the UI and e2e suites, and a live instance migration. ## Risks - Behavior change for agents: a nested checkout of a URL-only repository is now on `paperclip/<id>/<branch>`, not `<branch>`. `git pull` and `git status` work as before. A plain `git push` now fails because the branch names differ. Agents must push with an explicit destination, for example `git push origin HEAD:<branch>`. Branches in the shared clone are visible to all tasks of the project, like main task worktree branches. - Agent-created branches outside the worktree's prefix, stashes, and local config live in the shared clone. Stashes and custom config block expiration of every worktree of that clone (fail-closed). Other branches stay in the shared clone after a task is deleted. - The shared clone is the managed project checkout for that repository URL. Fetches into it update its remote-tracking refs. A concurrent restore point capture of a sibling can then roll back and retry on the next sweep. - The base lock is per process. Across processes, git's own ref and index locks apply. - No migration in this PR. Existing standalone nested clones keep their disk use until their tasks are removed or expired. A safe migration helper is a follow-up. Task repositories (`task-repo-*`) and configured local-folder copies stay standalone clones. - Restoration recipes now can carry `contents.checkouts[].worktree`. An older server rejects such a recipe as invalid (fail-closed) and does not restore without it. ## Model Used Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file editing, test execution). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 16:05:13 -07:00
DottaandPaperclip	bf223596ab	fix(runner): bootstrap fresh sessions in replacement sandboxes (#17320 ) Reuse the untouched-session proof and atomic directory claim when a sandbox replacement carries a provider session that has never run. Retain verified restore for saved backups and all prior or ambiguous session activity. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-23 15:35:24 -07:00
Devin FoleyandPaperclip	4660b22f6b	perf: narrow run bindings during event ingestion (#17378 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent runs append native events and retry-exhaustion receipts. > - These writes lock the run row and verify its company and agent bindings. > - The binding check fetched the full run, including prompts, configuration, and output. > - This pull request reads only the two fields that the check uses. > - The benefit is less database transfer while the same row lock protects sequence allocation and replay checks. ## Linked Issues or Issue Description Related: #17355 and #17348 reduce other run-read payloads. GitHub searches for event-read and narrow-run PRs found no duplicate of this event-ingestion change. What existing behavior does this improve? The run binding read during native event and retry-exhaustion ingestion. Current behavior The transaction uses a full-row `SELECT ... FOR UPDATE` to check only `companyId` and `agentId`. Large run context, runner configuration, and output cross the database connection for each such check. Proposed behavior Select only the company and agent IDs. Keep the same run ID predicate, row lock, transaction, sequence allocator, replay cursor, and binding checks. Reason and benefit The check does not use the other fields. Narrowing the query removes that repeated transfer without a cache or an authorization change. Breaking changes None. No API response, schema, event, or scheduling changes. ## What Changed - Narrow the existing run-row read inside `appendHeartbeatRunEvent`. The event service continues to own its transaction and binding validation. - Exercise native and retry-exhaustion writes and replays with large stored payloads. Record the executed SQL to prove the lock reads only the two binding fields. - Extend missing-run and company/agent mismatch coverage to both transaction paths. Keep the existing concurrent-writer and rollback tests. - Document the read boundary. ## Verification - Focused PostgreSQL event append and sequencing suites: 9 tests passed. They include concurrent writers, replay conflicts, rollback, cursor checks, large payloads, and company/agent/missing-run denials. - The first new retry-exhaustion fixture failed its duplicate assertion because it omitted the exhaustion fields from the event payload. The existing producer contract stores those fields in the payload. The fixture now does the same; the rerun passed. No production retry or replay logic changed. - `pnpm build`: passed. - `pnpm -r typecheck`: passed. - Greptile Apex: 5/5 on `76a6d3ee813ae47919b1ec8e8b88e01e58d5655d`, clean check and no review threads. The [full PR workflow](https://github.com/paperclipai/paperclip/actions/runs/40028758665) passed on that exact head: 54 checks passed and two conditional checks were skipped. The branch is conflict-free. - The full local all-in-one suite is not repeated for this follow-up. Full exact-head CI will supply the broader suite coverage. - No hosted deployment or bill reduction is claimed for this follow-up. ## Risks - Low scope: only the selected columns change. The `FOR UPDATE` lock, transaction boundaries, sequence reservation, duplicate checks, and failure handling remain in place. - Native replay and legacy event writers must still serialize on the same row. Existing real PostgreSQL concurrency tests cover that boundary. - No migration, provider change, new timer, or operator configuration is required. ## Model Used OpenAI Codex (GPT-6 family), with code execution and repository tools. The runtime does not expose the exact model ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 15:09:29 -07:00
Nicky LeachandPaperclip	a3e8fa568a	Reduce repeated work during plugin Runner admission (#17377 ) ## Thinking Path > - Paperclip manages agents and their execution authority. > - A plugin can host Runner tasks on a persistent execution host. > - Admission checks must observe current lifecycle state and credentials. > - The current runtime reader also resolves files, skills, and connections for each check. > - Some providers also repeat the same project authentication check during submit. > - This pull request adds a compact fresh reader and an explicit submit-time validation contract. > - Providers can remove repeated work while Core keeps lease and scope checks. ## Linked Issues or Issue Description What existing behavior does this improve? Plugin Runner startup before task submission. Current behavior One measured startup made 30 full runtime reads during bootstrap. These reads took 5.759 seconds. Each guard can resolve instructions, skills, and connection assignments even when it only needs current identity and lifecycle state. Providers that already enforce project authentication in submit must also perform a separate Core preflight. Proposed behavior Use a lifecycle-owned runtime identity reader for guards. Keep the full runtime reader for synchronization inputs. A trusted provider can declare `projectAuthCheckAtSubmit: true` in its prepared result. Core pins the exact configuration, provider, and project set on the lease. The provider must reject authentication conflicts before credential installation or task start. Providers that omit the flag retain the existing preflight. Reason and benefit Estimated savings with a paired consumer: 4–5 seconds from compact runtime checks, plus roughly 0.5–0.9 seconds from removing the duplicate authentication operation. These estimates overlap and are not a measured deployed result. Refs #17367 for the separate run-log HEAD optimization; this PR does not duplicate it. Breaking changes No migration. The SDK adds a host method and an optional prepared-result field. Consumers that use them require this Core host version. Older providers retain their existing flow. ## What Changed - Add `agents.getRuntimeIdentity` across SDK, RPC, host service, and test fixtures. - Reuse the existing company, plugin ownership, readiness, and capability checks. Read lifecycle state and protected instance secrets fresh on every call. - Skip instruction, organization, skill, and connection hydration in the compact reader. - Allow explicit submit-time authentication enforcement, with prepared scope pinned and checked again at dispatch. - Emit fixed preparation, project-auth, submit, and connection timing phases. ## Verification - 72 targeted SDK capability, plugin task-owner, and environment-task mediator tests passed. - 33 real embedded-Postgres lifecycle and secret tests passed, including pause, content revision, company isolation, plugin disablement, and secret rotation. - SDK and server TypeScript checks passed. Local Runner/eval TypeScript outputs were built to satisfy server type resolution. - Local Postgres required missing library aliases in ignored dependencies; no product source was changed for that repair. - Full repository suites are delegated to GitHub CI. No live rollout or speedup claim. ## Risks The submit-time flag delegates enforcement to a trusted provider. It is not proof that prepare already checked authentication. The provider must check before VM credential installation or task admission. Core still checks exact prepared scope, active lease, run identity, credential type, and dispatch fencing. Existing providers keep the separate check. Roll back a paired consumer before rolling back this host. ## Model Used OpenAI GPT-6 (Codex), with code execution and repository tools. Exact deployment build and context-window size are not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 14:54:25 -07:00
Devin Foley devinfoley Claude Opus 5.5	3c4050e3ad	fix(connections): keep Cloud refresh reauthorization diagnostics (#17362 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Managed connections (Gmail, Google Workspace, GitHub, Asana, Slack manager) get and refresh their OAuth tokens through the Paperclip Cloud connector broker. > - When a provider refuses a refresh grant (the user revoked it, it expired, it was already rotated, or it no longer matches the profile), the broker sent `502 PROVIDER_OPERATION_FAILED`. That is the same answer it sends for a provider outage. > - Paperclip then kept the grant `active`. Each health check, catalog refresh, or catalog read failed again with a 502, and the server reported each failure as a crash. The user did not get a reconnect prompt. > - The broker now sends `409 PROVIDER_REAUTHORIZATION_REQUIRED` for a dead grant. The connector already maps any 409 to `REAUTHORIZATION_REQUIRED`. Thus the grant becomes `needs_reauthorization` and the caller gets a 422 reconnect response. > - This pull request adds the new reason to the diagnostic allowlist, pins the producer contract, and tests the reconnect path from end to end. > - The benefit is that a dead managed grant asks the user to reconnect. It is not reported as a server crash. Real provider outages keep their 502 response. ## Linked Issues or Issue Description What happened? A managed Paperclip Cloud connection refresh failed with `Paperclip Cloud connector rejected the request (operation=refresh, status=502, reason=PROVIDER_OPERATION_FAILED)`. The connection health-check and catalog routes wrap that error as an HTTP 502. The error handler reports every status of 500 or more, so the failure went to error monitoring. The grant stayed `active`, so the same failure can occur again on each use. Expected behavior If the provider refuses the refresh grant, Paperclip marks the grant `needs_reauthorization` and shows the reconnect path. Only a real provider or Cloud outage gives a 502. Steps to reproduce 1. Connect a managed Gmail connection through Paperclip Cloud. 2. Revoke the app access in the Google account, or let the refresh grant expire. 3. Wait until the access token needs a refresh. Then run a connection health check. 4. Before the broker change, the health check returns 502 and the server reports it. The grant stays `active`. Additional context The broker-side classification ships separately in Paperclip Cloud. This pull request is safe before or after that change, because the connector already maps any broker 409 to `REAUTHORIZATION_REQUIRED`. Without this pull request, the 409 still works, but diagnostics show `reason=UNKNOWN_BROKER_ERROR`. ## What Changed - `server/src/services/paperclip-cloud-connector.ts`: Add `PROVIDER_REAUTHORIZATION_REQUIRED` to the allowlist of broker reasons that errors can keep. - `server/src/services/fixtures/cloud-connector-refresh-reauthorization.json`: Add the producer HTTP contract (status 409 and the exact JSON error body). - `server/src/services/paperclip-cloud-connector.test.ts`: Test that a refresh with this contract gives `REAUTHORIZATION_REQUIRED`, status 409, the retained reason, and no refresh-token text. - `server/src/__tests__/tool-access-service.test.ts`: Test that `refreshOAuthGrantCredentials` changes a reauthorization refresh failure into a 422 `oauth_reauthorization_required` response and sets the grant to `needs_reauthorization`. - `doc/connections/GMAIL.md`: Add a troubleshooting row that tells the two refresh reasons apart. ## Verification - `cd server && npx vitest run src/services/paperclip-cloud-connector.test.ts`: 48 passed. Without the allowlist change, the new test fails (`reason=UNKNOWN_BROKER_ERROR`). - `cd server && npx vitest run src/__tests__/tool-access-service.test.ts -t "managed grant for reconnection\|orders managed reconnect"`: 2 passed, against the embedded test database. - I did not run the full `pnpm -r typecheck`, `pnpm test:run`, and `pnpm build` locally. The sandbox has a 4 GB memory limit, and the server `tsc` process was stopped because it used too much memory. CI runs these checks. ## Risks - Low risk. The status mapping does not change: a 409 already meant `REAUTHORIZATION_REQUIRED`. This change only keeps one more fixed reason code in the error message. - The reason allowlist accepts fixed protocol codes only. The broker message, request data, and tokens stay out of errors and logs. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, in Claude Code agent mode with tool use (shell, file edit, code search, and a sub-agent for the code-path trace). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: devinfoley <139239+devinfoley@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-23 14:39:22 -07:00
Devin FoleyandPaperclip	dfe7df0015	fix(native): expire settled native workspace sync folders (#17358 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs sync their workspace with a remote sandbox through a per-run folder under `<instance>/native-workspace-sync/<run>/` > - The folder holds a descriptor, a manifest, and workspace and git seed archives, and a seed can be more than a gigabyte > - Only a few finalization commit paths delete the folder, so many runs leave it on disk forever > - Instance disks fill up with folders that no code will read again > - This pull request adds a bounded, deadline-driven retention worker that removes a folder only when no reader can still need it > - The benefit is that instance disks stop growing with stale sync folders, and recovery and export paths keep their files ## Linked Issues or Issue Description What happened? `cleanupNativeWorkspaceSync(runId)` is the only code that deletes `native-workspace-sync/<run>/`. It runs only when finalization commits through specific paths. Runs that end in `terminal_failure`, retained recovery, reconstruction, superseded runs, commits through other paths, and server crashes leave the folder forever. On one instance, 22 folders older than 7 days held 6.8 GB. Single seed archives were up to 1.4 GB. Expected behavior A run's sync folder is removed after a retention window, once the run and its finalization are settled and no recovery or export path can still read it. The shared `generations` seed cache stays. Steps to reproduce 1. Run a native agent with a remote sandbox and let the run end in `terminal_failure` (or crash the server during finalization). 2. Wait more than 7 days. 3. Look in `<instance>/native-workspace-sync/`. The run's folder and its seed archives are still there. Paperclip version or commit `master` at `837add0caa` Deployment mode Any deployment that runs native agents with remote sandboxes. Agent adapter(s) involved Native runtime with remote sandbox workspace sync (for example Codex on Daytona). ## What Changed - Added `server/src/services/native-runtime/native-workspace-sync-retention.ts`. It removes `native-workspace-sync/<name>/` when `<name>` is a UUID, the entry is a real directory, its newest file is older than the window, and one of these holds: - no `heartbeat_runs` row has that id (orphan); - the run has no valid `nativeWorkspaceSync` reference and is settled: terminal status, `finished_at` before the window, no live recorded process (see below), no live controller lease, no `nativeWorkspaceFinalizationOwner` receipt; - the run is settled as above, its `native_run_finalizations` row (if any) has no `lease_owner` and no `next_attempt_at` and is `committed` (any recovery state) or `terminal_failure` with no or `blocked` recovery state, no active/escalated export-cause or owner-unverified recovery action names it, no lease carries `nativeWorkspaceExportResume` or `pendingCleanupInFlight`, retained recovery cannot still use it (referenced lease destroyed, or no scheduled/running/awaiting_evidence system recovery and no queued/recovering workspace recovery), and historical recovery adoption (`adoptHistoricalWorkspaceRecoveries`) has no pending candidate for it. - Each folder is decided in one transaction that holds `pg_try_advisory_xact_lock` on the run's finalization ownership key. The rule is evaluated while the lock is held. A held lock skips the folder. The folder is renamed to `.expired-<uuid>` in that transaction and deleted after commit. The next sweep removes leftover `.expired-` folders. `generations`, non-UUID names, symlinks and files are never touched. Each tick tries at most 50 leftovers, then checks at most 200 folders and removes at most 50. Leftovers and folders each continue from their own cursor, so undeletable leftovers do not block run folders. A leftover that cannot be deleted schedules the 15-minute retry. Deadline discovery reads run state 200 IDs at a time and stops when asked to. It supports dry run and `shouldContinue`. - A recorded `process_pid` / `process_group_id` blocks removal only while it can still be the run's own process. This reuses the rule from `conversation-continuation.ts`: PIDs are ignored when every lease of the run is remote (they are sandbox PIDs), and a local PID or process group blocks only while it exists on this host and its start time matches `process_started_at`. An unreadable start time keeps the folder. A recycled PID does not. - Separate finding, not changed here: native runs do not clear `process_pid` when they finish. It is cleared only by the status arbiter's `release_run_resources` effect (cancellations and commits) and by the recovery paths. A failed run gets a `schedule_retry` effect instead, so its PID stays set. For remote targets the stored value is the sandbox PID. Other readers already treat these PIDs as liveness hints only, so this PR does not change that write path. - Exported `nativeWorkspaceFinalizationLockKey` from `native-workspace-finalization-ownership.ts` and used it in `withNativeWorkspaceFinalizationOwnership` and the retention worker, so both use the same key derivation. - Exported `nativeWorkspaceSyncStateRoot()` from `native-workspace-sync.ts`; `descriptorDirectory` now uses it. - Registered the worker on the shared delivery-work coordinator (`DELIVERY_QUEUES.nativeWorkspaceSyncRetention`) in `server/src/index.ts`, next to the git restore point worker. It follows `rules/sleep-aware-background-work.mdc`: no interval. It runs at startup and at real deadlines: the earliest settled run's finish (or an orphan's newest file) plus the window; a recheck one window later for folders it kept; and a 15-minute retry after a failure or a held lock. With no folders, `nextRunAt` is `null`. The run folder is now created through one helper in `native-workspace-sync.ts` that notifies process-local listeners; the worker wakes on that signal only while it has no deadline, because any scheduled deadline is never later than a new folder's own deadline. `index.ts` unsubscribes the listener on shutdown. It pauses passive maintenance during idle drain. - Added `PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_ENABLED` (default on unless `false`), `PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_DRY_RUN` (default off) and `PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_DAYS` (default `7`, minimum `1`) in `server/src/config.ts`. Documented them in `doc/DEVELOPING.md` and added a "Native workspace sync folders" section to `doc/recoverable-disk-cleanup.md`. ## Verification - `pnpm exec vitest run src/__tests__/native-workspace-sync-retention.test.ts src/__tests__/native-workspace-sync-retention-config.test.ts` in `server/`: 62 tests pass. They cover: - deletes a settled committed run (also with a left-over `resuming_session` recovery state), a blocked `terminal_failure` with no recovery owner, an orphan, and an unreferenced settled run; - keeps runs whose local process or process group is still alive, or whose live process start time cannot be read; keeps a running run with a PID; deletes terminal runs whose recorded local process exited, whose PID was recycled, or whose PID came from a destroyed sandbox; - keeps recent files, recently finished runs, unsettled runs, a live controller lease, an owner receipt, `observed` and `retryable_failure` phases, a live `lease_owner`, a scheduled `next_attempt_at`, and each non-blocked recovery state on `terminal_failure`; - keeps runs with an open export-cause or owner-unverified recovery action, `nativeWorkspaceExportResume` or `pendingCleanupInFlight` lease metadata, retained recovery in flight on a lease that was not destroyed, or pending historical recovery adoption; deletes when the retained lease was destroyed; - never touches `generations`, non-UUID names, symlinks or files; dry run deletes nothing; leftover `.expired-` folders are removed; an advisory lock held by another transaction makes it skip; ticks are bounded and resume from a cursor; - `nextDeadlineAt` returns `null` with no folders, now with leftovers, the settled finish plus the window, the orphan mtime plus the window, and one window later for kept folders; the registered worker goes quiet with no folders, retries after a held lock or an undeletable leftover, and the real delivery-work coordinator wakes again when the real producer (`writeDescriptor`) creates a run folder; leftover removal is bounded and undeletable leftovers do not block run folders. - Mutation checks: I removed each guard one at a time (advisory lock, owner receipt, process pid, finished_at window, lease owner, next attempt, phase, recovery state, export recovery, owner-unverified, export resume, retained recovery, backfill pending, symlink/real-directory check, dry run, and both mtime checks together, plus the folder-creation wake, the process liveness rule (restoring the old `process_pid IS NULL` rule, the sandbox exemption, the process group check, the recycled-PID check, the unreadable start time), the leftover batch limit, the leftover cursor and the leftover retry delay). Each mutation made at least one test fail. I restored the code after each one. Removing only the first mtime check does not fail a test, because the second check inside the transaction still keeps the folder. - `pnpm exec vitest run` on related suites in `server/`: `native-workspace-sync.test.ts`, `native-workspace-sync-history.test.ts`, `native-workspace-finalizer.test.ts`, `native-workspace-finalizer-failure.test.ts`, `native-harness-backup-expiration.test.ts`, `git-restore-point-ledger.test.ts`, `delivery-work-coordinator.test.ts`, `chat-delivery-work.test.ts`. All pass. - `pnpm run typecheck` in `server/` passes. - Greptile review: earlier rounds; all findings fixed with tests; final score 5/5 with no open threads. - Not run: the full server test suite and an end-to-end run against a real instance. I did not measure disk savings on a live instance. ## Risks - Deleting a folder that a reader still needs would make that path fall back or fail. Same-run resume and export run under the finalization ownership lock or before the run settles, and the worker checks the rule while it holds the same lock. Retained workspace recovery already handles a missing folder (ENOENT) by snapshotting the sandbox. The rule keeps the folder for every in-flight or reopenable recovery state listed above. - A committed finalization row with a stale recovery state and no lease owner is treated as settled. Nothing reads the recovery state of committed rows. - The worker is on by default. Operators can set `PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_DRY_RUN=true` first to see candidates in the log, or turn it off with `PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_ENABLED=false`. - The first sweep on a large instance does more filesystem work. It is bounded to 200 folders checked and 50 removed per tick. - The folder-creation wake is process-local. A folder created by another server process sharing the same instance directory is picked up at the next deadline or restart. - A stale `process_pid` no longer blocks removal. If a run's local process is still alive with its recorded start time, the folder stays. Live work is also still guarded by the run status, controller lease, finalization lease and ownership lock checks. - No schema migration. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 14:05:55 -07:00
DottaandPaperclip	4d90634f74	feat(connections): add 1Password service-account connector (#17249 ) ## Thinking Path > - Paperclip lets people manage AI agents and govern their access to outside services. > - Apps connections keep provider credentials in the instance vault and mediate every action through grants, policies, and audit. > - Agents currently have no Apps connection for reading or managing 1Password items with an operator-supplied service-account token. > - 1Password supports this through its official SDK; its documentation MCP server does not expose vault operations. > - This pull request adds a one-field service-account connection and registers eight reviewed SDK actions through the existing native-provider catalog. > - Operators can select vault permissions in 1Password and agent/action access in Paperclip without installing a CLI or handing durable credentials to agents. ## Linked Issues or Issue Description Related: #16144 adds 1Password as Paperclip's secret-storage backend. This PR adds the separate Apps/tool-access flow with a per-connection credential; it does not change secret providers or use a server-wide ambient token. Subsystem affected Apps catalog, native tool-provider registry, and shared connection setup. Problem or motivation People need to paste their own `OP_SERVICE_ACCOUNT_TOKEN`, find official service-account setup instructions, and authorize agents to use the selected vaults through ordinary Paperclip governance. Proposed solution Add `1password/service-account`, with a password field labeled Paste your 1Password Service Account token and a link to [1Password's creation instructions](https://www.1password.dev/service-accounts/get-started#create-a-service-account). Use the pinned official `@1password/sdk` 0.5.0 for vault/item listing, item and secret-reference reads, and item create/update/archive/delete. Alternatives considered A remote MCP card would not provide vault access. A local CLI or a third-party MCP package adds installation and credential surfaces that the official SDK avoids. ## What Changed - Added the source and generated catalog definition, permission review, official light/dark artwork, documentation, and capability-map references. - Added a provider-owned SDK bridge with strict schemas, account-domain validation, contract hashes, fixed errors, bounded output, and disposable workers with cancellation/deadlines. - Registered the provider through master's native-provider catalog; existing setup, health, discovery, gateway, policy, and audit owners remain in place. - Labeled the single password field Paste your 1Password Service Account token. The shared formatter preserves explicit credential prompts; ordinary key labels retain their existing presentation. - Added credential-ownership, gateway read/write, policy/revocation, worker, setup UI, and interactive Storybook coverage. ## Verification - Final review follow-up: added two deterministic worker-entry tests for nested exact-token redaction and oversized-output rejection. Removing each guard caused exactly its regression test to fail. The focused provider suite passed 15 tests, and the official server typecheck passed after rebuilding stale workspace SDK artifacts. Current head `569cb7f9bd57db791fb13a6034ce69bf50878cb1` passed the full [CI run](https://github.com/paperclipai/paperclip/actions/runs/40016166260), including typecheck, build, unit/integration tests, and all eight browser shards. The required `ci / verify` and `ci / e2e` gates passed. [Greptile](https://github.com/paperclipai/paperclip/pull/17249#issuecomment-6353731805) and [Paperclip Review](https://github.com/paperclipai/paperclip/pull/17249#issuecomment-6372165895) each rated this exact head 5/5, with no open findings. Paperclip Review completed with a neutral check because its scoped reviewer could not inspect CI; the full CI result was verified separately. Other neutral checks and intentionally skipped Storybook jobs contain no actionable findings. - CI recovery on the same commit: the first attempt lost six jobs. Five logged simultaneous runner shutdowns at 18:49 UTC on 2026-10-23; the sixth stopped logging before a test result and its root cause was not established. No assertion failure appeared in the retrieved logs. All six jobs and the aggregate gates passed one failed-job retry. [CI run](https://github.com/paperclipai/paperclip/actions/runs/40005754936). - Copy follow-up: the catalog, provider, and setup UI suites passed 275 tests. The governed gateway lifecycle passed 3 tests (one live-only test skipped), including create, update, archive, and delete. UI token gates and map checks passed. The branch was rebased onto `9c442cf48e` to resolve additive catalog conflicts while retaining the Vercel and Supabase entries. The existing live write proof below remains applicable; this follow-up changes presentation only. - Five affected suites, including the existing Daytona consumer, passed 290 tests (one opt-in live test skipped in fixture mode). Full `pnpm -r typecheck` and `pnpm build` passed. The local broad-suite attempt could not complete because PostgreSQL semaphore exhaustion prevented a competing integration suite from starting; the complete affected-file retry passed after disposable-process cleanup. The prior commit `f208a8aff6dc28859cc3f343497e18d85f124579` passed the complete CI matrix. All CI build, typecheck, test, and browser gates also passed on the earlier commit `9e2f3d3140368a08da43b004b2e8d3a0f4c0fa4d`. The separate Paperclip Review reported a worker-result regression-test gap on that head; the follow-up above addresses it. Its review thread is resolved, and both fresh current-head reviews are clean as recorded above. - Token gates, module boundaries, brand validation, map checks, and Storybook catalog checks passed. - Direct SDK create/archive/delete and rejected-call coverage passed (13 provider tests); the Storybook Permissions/Test play completed successfully in Chromium. Server and UI typechecks passed after those fixes. Greptile passed prior commit `9e2f3d3140368a08da43b004b2e8d3a0f4c0fa4d` at 5/5. Both reviewers also passed the worker-test follow-up at 5/5 as recorded above. - Live 1Password proof on 2026-10-22: connect, eight-action discovery, vault listing through an actual run-scoped gateway, refresh, reconnect preserving identity, another read, removal, and denial through the retained session passed. Credential absence was asserted in responses/audit; an exact-token scan of inspected logs and changed files passed. - Real browser: Connectors → 1Password → token entry → personal access limited to one disposable agent → completion → Permissions → Test `onepassword_list_vaults` succeeded. The disposable connection was removed afterward. - Compiled-worker smoke initially exposed inherited Node launch flags. Workers now use explicit empty `execArgv`; the compiled worker then authenticated successfully and listed one vault. - Live write proof on 2026-10-23: the replacement Keychain token passed all eight actions through the actual run-scoped gateway for both personal and shared credentials. The test created, listed, read, updated, resolved a secret from, archived, and deleted two synthetic items. Both were absent from subsequent item lists. Refresh/reconnect, identity preservation, removal, retained-session denial, and credential absence passed. The live suite passed 3 tests (one fixture-only test skipped) in 116.65 seconds. The expanded fixture suite passed 3 tests (one live-only test skipped); the 13 provider tests and server typecheck passed. An exact-token scan passed over 64 source/log files. - Failure history: the original token denied creation on 2026-10-22 and created no item. The first 2026-10-23 attempt stopped before provider calls because host PostgreSQL semaphore limits prevented test-database startup. A temporary ten-connection test profile allowed the complete retry to pass without changing shared services or host limits. A new delete-result assertion also exposed an outdated fixture response (`success` instead of `deleted`); the fixture was corrected and the complete fixture suite passed. Official contract sources: [service accounts](https://www.1password.dev/service-accounts/get-started), [SDKs](https://www.1password.dev/sdks), [item management](https://www.1password.dev/sdks/manage-items), [official JavaScript SDK](https://github.com/1Password/onepassword-sdk-js). ## Risks - Authorized item/secret reads intentionally return secret values to the agent and ordinary tool-result history. Select trusted agents and use restricted service-account vaults; the connection token itself is redacted. - New connections retain existing Paperclip action defaults, including Allowed writes/destructive actions for selected agents. 1Password still enforces the token's permissions. - Service-account access cannot be changed in place. Create a replacement account and reconnect to change vault permissions; read-only keys can connect but cannot write. - SDK authentication happens per action in a disposable worker. Calls have a 30-second maximum; cancellation stops local work, while a mutation already accepted by 1Password can still finish. Writes are never automatically retried. - Additive dependency and catalog change; no database migration. Disable/remove the connection to withdraw access, or remove the provider from catalog visibility for rollback. ## Model Used OpenAI Codex, based on GPT-6, with code editing, shell/browser tools, and live provider verification. A more specific runtime model ID and context-window limit were not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 13:27:33 -07:00
Devin FoleyandPaperclip	ef9e17304c	perf: narrow runtime progress database reads (#17355 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Active runs publish workspace and provider progress. > - Each update must check the current run status before it publishes. > - This check fetched the full run, including prompts and runner configuration. > - This pull request reads only the four fields that progress publication uses. > - The benefit is less database transfer while each update still checks durable state. ## Linked Issues or Issue Description Related: #17348 narrows authorization reads. This PR applies a separate projection to runtime progress. GitHub searches for runtime progress and run-status read PRs found no duplicate. What existing behavior does this improve? The database read before a runtime progress update. Current behavior Each update calls `getRun`. This reads large run context and runner configuration even though progress publication only uses the run ID, company ID, agent ID, and status. Proposed behavior Use a fresh four-column read for each update. Keep the existing active-status check, event fields, error propagation, and late-progress suppression. Reason and benefit Transfer progress can emit many updates for one run. A small status check avoids repeatedly transferring payloads that it does not use. Breaking changes None. No API response, schema, scheduling, or cancellation policy changes. ## What Changed - Add `getRunStatus` to the run-state owner. The full reader remains available to consumers that need execution data. - Wire the narrow reader into the lifecycle progress check. Remove the unused context field from that check's input contract. - Cover the exact selected fields, missing runs, database errors, large stored payloads, terminal transitions, and the public heartbeat service wiring. Document the read boundary. ## Verification - Run-state and lifecycle suites: 44 tests passed before the additional public-service regression. - Lifecycle, public-service wiring, and native cancellation suites: 36 tests passed. - `pnpm -r typecheck`: passed. - `pnpm build`: passed. - The [full PR workflow](https://github.com/paperclipai/paperclip/actions/runs/40013092548) passed on `95bf496f689049812646b5ccddcdf2cca21efad0`. All 54 reported checks passed; two conditional checks were skipped. This includes test shards, browser checks, Runner verification, production bundle boot, and clean npm installation. - `GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=init.defaultBranch GIT_CONFIG_VALUE_0=master pnpm test:run`: an additional local all-in-one run is still in progress; no full-local-pass claim. The process-scoped Git default matches existing master-only fixtures and does not change user configuration. - The initial description check failed because the dedup-search checkbox was missing. Searches are complete and the template checkbox is now included. The rerun passed on the same source commit. - Greptile Apex: 5/5 on `95bf496f689049812646b5ccddcdf2cca21efad0`, no findings or review threads. - No hosted deployment or bill reduction is claimed. CI and Apex are complete; this PR is merge-ready. ## Risks - Low scope: the query selects fewer columns and preserves the existing status read on every progress callback. It adds no worker, cache, timer, or idle query. - A run can still finish after the status read and before publication, as it could before this change. This PR does not change that concurrency boundary. - No migration or operator configuration is required. ## Model Used OpenAI Codex (GPT-6 family). The runtime does not expose the exact model ID or context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (focused suites above; full suite is still running) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 12:50:36 -07:00
Devin FoleyandPaperclip	837add0caa	fix(recovery): deliver review-path repair instructions to agent prompts (#17353 ) ## Thinking Path > - Paperclip keeps agent tasks moving through owned execution and durable waiting paths. > - A consumed review path gets one repair wake before board escalation. > - That wake records an instruction to save a new monitor or another real review path. > - The canonical wake builder omitted that instruction from the agent prompt. > - This pull request delivers it through the existing disposition-repair section. > - The repair limit and execution authority stay unchanged. ## Linked Issues or Issue Description What happened? An agent opened a pull request and promised to watch CI and external review. It ended without scheduling a monitor. Its bounded review repair also ended without a maintained path, so the task became blocked for a board decision. The saved repair context contained the instruction, but the canonical wake payload and rendered prompt did not. Expected behavior The repair prompt tells the agent to save a one-shot monitor before promising another check. Both fresh and resumed sessions receive that instruction and the one-attempt limit. Steps to reproduce 1. Create a review-path recovery with `decideIssueReviewPathRecovery` for a task whose review path was consumed. 2. Pass its recorded context through `buildPaperclipWakePayload` and wake serialization. 3. Render a fresh or resumed prompt. Before this fix, it has the wake reason but no repair instruction. Paperclip version or commit Base: `9c442cf48e6`. Current head: `f38d65891c8d9f2b38eb76a8578e870debc2b1e6`. Deployment mode Observed on managed sandbox execution with the legacy Claude ACP adapter. The corrected canonical payload serves the shared adapter prompt path. Refs #15764 introduced the bounded exhausted-repair disposition. This change preserves that limit. Open monitor-retry work such as #17171 addresses an already scheduled check; this defect occurs before a check exists. Searches found no duplicate for this missing prompt field. ## What Changed - Build the review-path repair disposition in its owning recovery module, using the server instruction and fixed one-attempt bound. - Project it into the existing canonical `dispositionRepair` field. No new adapter field or execution protocol is added. - Ignore arbitrary replacement instructions and attempt-limit values. A pending repair keeps its instruction across a connection-pool retry that changes the wake reason. Ordinary wakes without the typed repair marker ignore raw instruction text. Existing legacy disposition repairs retain their values. - Add a regression across the real recovery decision, wake builder, serializer and fresh/resumed prompt selectors. Update execution guidance and the affected recovery maps. ## Verification - Verified red before the source fix: two regression cases failed because the instruction was absent; the sibling repair case passed. The same cases pass after the fix. - Passed 194 focused and safety tests across wake preparation, review recovery, adapter prompt handling, and database-backed liveness escalation. The connection-pool retry case also failed before its correction and passes after it. - Replayed the recorded incident wake through the fixed boundary. The old canonical prompt lacked its saved instruction. The corrected serialized prompt includes it with a 1/1 bound. This replay did not execute a provider turn or change the source task. Private evidence stays outside the repository. - Passed `pnpm -r typecheck` after the pool-retry correction. Passed `pnpm build` on final head `f38d65891c8d9f2b38eb76a8578e870debc2b1e6`. Map checks and `git diff --check` pass. - [Full CI](https://github.com/paperclipai/paperclip/actions/runs/40005890857) passed on that head: 54 checks passed and two optional checks were skipped. This includes the full test shards, typecheck, build, and browser suites. The local `pnpm test:run` attempt was stopped after the CI test shards passed. No complete local full-suite pass is claimed. - [Apex review](https://github.com/paperclipai/paperclip/pull/17353#issuecomment-6372160694) is 5/5 on the final head, with no open findings. The first review found the pool-retry case. Its regression and correction are included. The branch has no merge conflicts. - Test setup first required the plugin SDK build, then an empty credential-history database boundary. Those setup errors are not the claimed red. The temporary recorded-wake probe was removed before final validation. - Hosted follow-up acceptance is not run. The source task remains blocked and needs a retry after rollout. This PR does not claim that prompt delivery itself schedules a monitor or that every model will follow the instruction. ## Risks - This fixes instruction delivery. The agent must still save a valid monitor or another waiting path. The existing bounded escalation remains if it does not. - The wake stays a normal-model repair. It does not become status-only recovery and can continue useful work. Existing ownership, approval, budget, pause, and monitor admission gates remain in place. - No schema, credential, telemetry, API, model, or retry-budget change. ## Model Used OpenAI Codex, based on GPT-6, assisted with diagnosis, code editing, shell execution and verification. The exact serving model ID and context-window limit are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (194 focused and safety cases; full coverage passed in CI) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 12:07:07 -07:00
DottaandPaperclip	9c442cf48e	feat(connections): deploy and configure Supabase and Vercel (#17217 ) Add governed Supabase and Vercel deployment/configuration connectors with vaulted credentials, agent access policies, approvals, audit, and bounded provider operations. Deliver reviewed REST actions to native agents and guard Vercel preview intent on empty projects. Preserve secret-safe request identity and durable retry behavior. Verified by current-head CI, fresh Greptile 5/5 review, and real-agent deployment/configuration walkthroughs on disposable provider resources. Test resources and temporary provider tokens were removed after verification. Co-Authored-By: Paperclip <noreply@paperclip.ing>	2026-10-23 11:37:29 -07:00
Devin FoleyandPaperclip	f406b32d08	perf: narrow run authorization reads (#17348 ) ## Thinking Path > - Paperclip lets people manage and observe AI agents at work. > - Live events and run history must enforce current task permissions. > - Some permission checks load the complete run record for each event or request. > - Run context, runner configuration, results, and output can make those records large. > - This pull request reads only the persisted fields each access check needs. > - The checks still read current permissions and fail closed. ## Linked Issues or Issue Description Related: #17251, #9155, and #16229. Those changes cover other run projections, recovery polls, and task history. This change covers live-event delivery, workspace-operation run checks, and event-list access. What happened? Live-event delivery loaded every run column for each subscribed viewer, although the permission helper used only the company, run scope, and task binding. Workspace-operation checks did the same. The run-event endpoint also loaded complete run records before returning separate event rows. Large contexts and runner configurations therefore increased database transfer for ordinary access checks. Expected behavior Read only the stored access binding. Keep company checks, permission freshness, deleted-task tombstones, and response bodies unchanged. Steps to reproduce 1. Create an issue-bound run with large context, runner configuration, and output fields. 2. Subscribe an authorized viewer to company live events and publish run updates. 3. Read the run events and a linked workspace operation. 4. Inspect the run SELECT projections. The previous queries load fields that the access checks do not use. Paperclip version or commit The defect is present on the current master base, `62c78e56e53`. Deployment mode Authenticated server with PostgreSQL. The defect is not adapter-specific. ## What Changed - Select only company, scope, and issue bindings for live-event and workspace-operation run authorization. - Add a four-field run-access read for event-list requests. Reuse those columns in the existing log-access projection. - Preserve per-delivery and per-request permission checks, company predicates, and event responses. - Add real PostgreSQL and WebSocket coverage with large run fixtures. Cover grant revocation, grant restoration, deleted tasks, missing and foreign-company runs, and failed reads. - Document the bounded access reads in the existing polling-load guide. ## Verification - Six focused suites pass: 40 tests. They cover the changed paths, run-state reads, run logs, disconnect retries, and native event replay. - The new WebSocket test uses the production delivery path and a real database. It verifies that the selected SQL omits large run fields while authorization changes take effect on the same open socket. - The event-route test preserves the returned event content and denies the next request after grant revocation. - Local recursive typecheck, full build, and module-boundary checks pass. [Exact-head CI](https://github.com/paperclipai/paperclip/actions/runs/39997370119) completed successfully, covering the full grouped unit suites, typecheck, build/runtime boot, and E2E shards. The separate serial local `pnpm test:run` is still running and is not claimed as completed. - A later CI shard failed an unchanged email test expecting exactly one task after permission grants. The unchanged-base test passes at normal speed; adding only a 2.2-second pause before granting access reproduces the extra tasks because previously denied deliveries become due for retry. No email code or assertions are changed in this PR. The original failure is retained. The single shard rerun passed: 92 suites and 1,489 tests passed, with one suite and two tests skipped by the existing gates. - CI initially found a route-test double missing the new access helper. The follow-up fixes that fixture and adds event reads to malformed-ID and missing/cross-company checks. All 122 tests across the four changed-path suites pass; a final rerun of the expanded route suite passes all 92 tests. - The first CI run also timed out in the signoff changes-requested handoff while waiting for a reviewer run after a cancelled execution blocker. The focused test passes locally on this source. The same focused test passes three times on unchanged master. Controller-lease cancellation messages appear on both versions. The original CI timeout is preserved as an intermittent, unreproduced failure; its exact cause remains unknown. The signoff shard passes in the final exact-head CI run. - Initial fresh-checkout tests could not start because workspace SDK output and native PostgreSQL library links were missing. Dependency builds and the package's native-link setup corrected those prerequisites. The subsequent focused run passed. Focused command: ```sh pnpm --filter @paperclipai/server exec vitest run src/__tests__/live-events-run-read.test.ts src/__tests__/heartbeat-run-privacy-routes.test.ts src/services/heartbeat/run-state.test.ts src/__tests__/native-observer-replay.test.ts src/__tests__/heartbeat-run-log.test.ts src/__tests__/heartbeat-run-privacy-disconnect-retry.test.ts ``` ## Risks - A missing access field could cause an incorrect allow or denial. Tests exercise current grants, revocation, company scope, and deleted-task bindings through the real consumers. - No successful permission decision is cached. Every delivery and request still checks current database state. - No schema, migration, polling interval, API response, or deployment configuration changes are included. - This reduces bytes for the affected reads. It does not establish a whole-account savings percentage. Hosted transfer measurements remain separate validation after staging deployment. ## Model Used OpenAI Codex, a GPT-6 based coding agent, used repository tools, code execution, and database-backed tests. The session does not expose the exact backend model identifier or context-window size; those values are not guessed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 11:01:06 -07:00
Nicky LeachandPaperclip	a708a13336	Separate plugin workspace completion from maintenance inspection (#17329 ) ## Thinking Path > - Paperclip manages AI agents and their work. > - Storage plugins own remote task workspaces. > - Run completion currently requests the same inspection used for storage cleanup. > - That inspection can fetch remote refs and scan files after the agent has finished. > - Completion needs retained workspace identity and durable writes, not permission to delete files. > - This change adds an optional completion operation and keeps maintenance inspection separate. ## Linked Issues or Issue Description What happened? A successful plugin-backed run can fail during completion because the storage provider performs a slow remote-recoverability inspection. Dirty or unpublished output does not need a deletion-safety proof to remain valid task output. Expected behavior Completion verifies the assigned workspace and confirms durable writes through the provider. Cleanup still requires the existing full inspection. Steps to reproduce Run a task with plugin-owned storage whose inspection fetches remote refs. Let the agent finish, then observe completion waiting for the maintenance inspection. Related context: #16428 added finalization observability. The duplicate search found no existing PR for this provider contract. ## What Changed - Add optional `workspaceProvider.finalizationVersion: 1` and strict `finalize` / `finalized` schemas. - Pass original run and agent context through live completion and recovered completion. - Give the new operation a 30-second deadline and keep ownership checks and lifecycle serialization. - Retain the inspection fallback for providers without the capability. Keep maintenance inspection and removal unchanged. ## Verification - SDK build passed. - Targeted SDK and completion unit tests: 7 passed. - Added integration coverage for capability selection, legacy fallback, maintenance separation, and failure propagation. - Local integration setup timed out starting PostgreSQL; the integration suite passed in GitHub CI. - Full build/typecheck and broader tests passed in GitHub CI (55 successful checks, 2 skipped). Local checks remained targeted as requested. - Diff reviewed for secrets and operational identifiers. ## Risks The capability is opt-in. Providers must not advertise it until their completion implementation is available. An unavailable provider or failed durability barrier remains retryable. No schema or stored-binding migration is required. Test-file size exception: the provider integration suite grows from 973 to 1,002 lines. The 29-line case follows the same prepared binding through legacy completion, opted-in completion, and maintenance inspection using the existing single-connection database fixture. Keeping it there avoids a second embedded-Postgres lifecycle for one transition test. The completion helper unit tests are in a separate focused file. ## Model Used OpenAI GPT-6 Codex, with reasoning and code/test tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 09:07:35 -07:00
Devin FoleyandPaperclip	0fb152ee94	fix: save stale remote-tracking commits in nested restore points (#17289 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Git cache expiration removes closed issues' worktrees once their remote, or a restore point, can reproduce them > - A nested `.paperclip-repositories` checkout's remote-tracking refs can point at commits the remote has since dropped, such as a branch deleted after a squash merge > - Strict expiration rightly keeps such a checkout, because it may hold the only copy; but restore point mode also kept it forever, because there was nothing else to save > - This pull request saves those commits in the checkout's restore point, so the workspace can be reclaimed without losing them > - The benefit is that long-lived multi-repository workspaces are reclaimed in restore point mode ## Linked Issues or Issue Description Refs #17257. Bug report described in-PR (bug report template). What happened? With restore points enabled, four closed workspaces on one instance were kept with `unpushed_managed_checkout`, after a dry run had counted them as candidates. Each nested `paperclip` checkout was clean, on `main` at a commit that is on GitHub `main`. But it held about 1,000 remote-tracking refs, and between 28 and 41 of them pointed at branches the remote has since deleted or force-pushed (147 to 203 commits it no longer has). The relaxed check passed, so the checkout needed no restore point beyond those commits. Capture saved only `HEAD` history, so it found nothing to save and stored no restore point. The final check before deletion then ran the strict check, which keeps stale remote-tracking commits, and rolled back. The dry run never reaches that final check, so it over-reported. Together these workspaces held about 2.4 GB. Expected behavior In restore point mode, commits that exist only in remote-tracking refs are saved, and the workspace expires. Steps to reproduce 1. In a nested checkout, fetch a remote branch, so `refs/remotes/origin/<branch>` exists. 2. Delete that branch on the remote and garbage-collect it. 3. With restore points enabled, close the issue and run expiration: the workspace is kept with `unpushed_managed_checkout`. Paperclip version or commit: `master` at `412914cfa7`. Deployment mode: any. Agent adapter(s) involved: not adapter-specific (core). ## What Changed - Strict mode is unchanged. An earlier revision of this pull request ignored remote-tracking refs. Review pointed to `does not trust a stale remote-tracking ref`, which deliberately protects those commits, so this pull request keeps that behavior and the test is unchanged. - Relaxed nested check (`assertReproducibleManagedCheckout(..., { restorePoint: true })`): - reads all reflogs in one bounded call (`--format=%H %gD`), deduplicates them by commit, and splits them into local and remote-tracking entries before applying the tip budget; - local entries still have to be covered or upstream, as before; - remote-tracking tips and remote reflog entries that the remote no longer has are returned as `preserve` (new `tipsMissingUpstream`, sharing `commitsMissingUpstream` with the existing check). - Capture: the proof is shared as `provenManagedCheckoutHistory`. `storeGitRestorePoint({ resolvePreserveCommits })` calls it after taking the history digest and before capture, then compares the digest again after capture. A fetch before the digest is included in the preserved commits, and one after it changes the digest and aborts capture. The commits flow into `captureRestorePoint`. They are bundled as history-only commits, within the existing limits (50 preserved tips, 1,000 commits, 256 MB bundle). The checkout therefore gets a restore point, and the final check uses the relaxed check for it. - Final check: `restorePointHistoryDigest` now also covers remote-tracking refs and their reflogs, so a fetch, ref change or reflog-only change after capture rolls expiration back. - Restore: the commits are imported as objects; the stale remote-tracking refs themselves are not recreated. - Docs: `doc/recoverable-disk-cleanup.md`. No code comments were added. ## Verification - `pnpm exec vitest run src/__tests__/git-workspace-expiration.test.ts src/__tests__/git-reproducible-managed-checkouts.test.ts -t "stale remote-tracking\|remote-tracking refs change"` (in `server/`): 5 passed. - The existing strict test passes unchanged. - A clean nested checkout whose remote-tracking ref points at a deleted, garbage-collected branch is kept strictly, expires with a restore point, and reopens on its commit with the stale commit present. - A remote-tracking ref created after capture rolls expiration back, and so does a reflog-only change to one. - 6,000 repeated reflog entries for one commit do not hit the tip budget. - Mutation checks: each of these fails a case: - dropping the preserved commits reproduces the instance's symptom (`unpushed_managed_checkout`); - dropping remote-tracking refs or their reflogs from the digest; - removing deduplication (`managed_checkout_local_tip_budget`). - `pnpm exec vitest run git-restore-point git-workspace-expiration git-workspace-contents git-workspace-restoration git-reproducible-managed-checkouts execution-workspaces heartbeat-git-restore-point-resume` (in `server/`): 261 passed. - `pnpm run typecheck` (in `server/`): pass. - Not run: server build and boot smoke. ## Risks - Only with restore points enabled. Strict expiration is unchanged. - Retention: saved stale commits are kept until the restore point expires (90 days after the tree closes by default), then deleted with it. - Limits: checkouts with more than 50 stale tips, or bundles over the limits, skip capture and stay kept, as before. - Spurious rollbacks: a fetch during the short window between capture and deletion rolls back; the next sweep retries. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, tests, read-only inspection of a live instance). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 08:24:53 -07:00
Devin FoleyandPaperclip	6877760904	feat: save ignored files in git restore points (#17290 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Git cache expiration removes closed issues' worktrees after saving a restore point of anything the remote cannot reproduce > - Restore points left ignored files out, so any worktree with an ignored file that was not a known regenerable directory was kept > - On one instance that was the most common reason a closed worktree was still kept, mostly build caches and agent evidence files > - This pull request saves those ignored files in the restore point too > - The benefit is that more closed worktrees are reclaimed, and nothing an agent left behind is lost ## Linked Issues or Issue Description Refs #17248, #17257. Bug report described in-PR (bug report template). What happened? With restore points enabled, one instance kept 15 of 43 inspected closed worktrees with `unique_ignored_files`. Among 19 such worktrees on disk: - 9 held `ui/tsconfig.tsbuildinfo`; - others held build output outside the regenerable directory names (`server/ui-dist/`, `test-results/`); - 2 held materialized skill copies under `.claude/skills/`; - about 5 held small files agents kept out of git: patches, evidence archives, recovery receipts. Expected behavior Ignored files that are not regenerable are saved with the restore point and restored on reopen, so the worktree can be removed. Steps to reproduce 1. In a closed issue's worktree, leave an ignored file outside `node_modules`, `dist` and the other regenerable directories (for example `ui/tsconfig.tsbuildinfo`). 2. With restore points enabled, run expiration: the worktree is kept with `unique_ignored_files`. Paperclip version or commit: `master` at `412914cfa7`. Deployment mode: any. Agent adapter(s) involved: not adapter-specific (core). ## What Changed Capture (`server/src/services/git-restore-point.ts`): - `restorePointSavesIgnoredFile` defines which ignored files are saved. It leaves out: - `.env` and `.env.` files; - `.paperclip-repositories/` (nested checkouts have their own restore points); - `.claude/settings.local.json` (saved with the workspace contents); - anything inside a directory named in `REGENERABLE_WORKSPACE_DIR_NAMES`. - Those files are scanned with the same limits and checks as untracked files, and snapshotted into a separate `ignored` tree, committed on top of `HEAD`. Its ref travels in the bundle. - They are hashed with `hash-object --no-filters` (symlinks by their raw target bytes), so `.gitattributes` conversions such as `text=auto` never change their bytes. Every path given to `--stdin-paths` is C-quoted, so git always reads the literal workspace file, even for names that start with `"`. - An ignored nested repository skips capture, which keeps the worktree. - A worktree whose only extra files are ignored is no longer "nothing to save". Format:* a payload with an ignored tree uses `paperclip.git-restore-point-payload.v2`, and one without stays `v1`. Decoding requires `ignored` exactly for `v2`, so older servers reject the new payloads instead of restoring without the files. Restore: `applyRestorePoint` writes the ignored files directly from their blobs, read in one `cat-file --batch` call, so no filter is applied. - It checks every target before writing, refuses any path that would leave the worktree, and never overwrites an existing file: a conflict fails the restore closed. - If a write fails partway, it removes the files it wrote, so a retry starts clean. The post-apply verification, the before-deletion snapshot check and `restorePointSnapshotMatches` all include the ignored tree. The stored reference (`GitRestorePointRef.ignored`) is validated and checked against the decoded manifest. Expiration (`git-workspace-restoration.ts`): in restore point mode, the reproducibility check accepts the ignored files a restore point saves. Strict mode is unchanged. Docs: `doc/recoverable-disk-cleanup.md`. No code comments were added. ## Verification - `pnpm exec vitest run src/__tests__/git-restore-point.test.ts` (in `server/`): 27 passed. New cases: - Round trip: ignored files round-trip as a `v2` payload with their modes, while `dist/`, `node_modules/` and `.env` stay out. - Conflict: an existing conflicting file fails the restore closed and is left as it was. - Nested repository: an ignored nested repository skips capture. - Forged payload: a `v1` payload claiming an ignored tree is rejected. - Line endings: an ignored CRLF file in a `* text=auto` repository restores byte for byte. - Partial failure: a restore that fails partway leaves nothing behind, and a retry succeeds. - Quoted names: a crafted ignored path that looks like a quoted absolute path saves and restores the workspace file, never the outside file it names. - Symlinks: a symlink target with bytes that are not valid UTF-8 restores exactly. - `pnpm exec vitest run src/__tests__/git-restore-point-expiration.test.ts` (in `server/`): new cases. - A closed worktree whose only extra files are an ignored build cache and evidence file is kept without restore points, expires with one, and reopens with those files and without `node_modules`. - An ignored file changed after capture rolls back. - Mutation checks: each of these fails at least one case: - not relaxing the reproducibility check; - not capturing ignored files (the end-to-end case then catches the loss); - leaving the ignored tree out of the comparison; - not writing the files on apply; - hashing with filters; - dropping the partial-restore rollback; - passing unquoted paths; - decoding symlink targets as UTF-8. - `pnpm exec vitest run git-restore-point git-workspace-expiration git-workspace-contents git-workspace-restoration execution-workspaces workspace-compaction heartbeat-git-restore-point-resume src/services/heartbeat/environment-selection` (in `server/`): 296 passed. - `pnpm run typecheck` (in `server/`): pass. - Not run: server build and boot smoke. ## Risks - Only with restore points enabled. Strict expiration is unchanged. - Bigger restore points. Ignored files count toward the existing limits (20,000 files, 64 MB per file, 256 MB total). Larger caches outside the regenerable names make capture skip, which keeps the worktree as before. - Rollback: older servers cannot restore `v2` restore points; they fail closed. Disable restore points before downgrading. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, tests, read-only inspection of a live instance). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 08:24:31 -07:00
DottaandPaperclip	3f1ad6b25b	fix(chat): resume task Retry after retained-source containment (#17317 ) ## Thinking Path > - Paperclip manages agent work through task and execution ownership. > - A task Retry must retain the user's request while the system verifies an old executor. > - Connected conversations also retain their external principal and accepted source batch. > - Chat Retry admission required containment before it could save a wait for that recovery. > - A saved chat receipt could also remain deferred after containment became available. > - This pull request lets the exact recovery-bound request wait and resume through its chat owner. > - Normal admission still checks containment, current access and policy before one successor starts. ## Linked Issues or Issue Description What happened? An ordinary task Retry on a connected conversation failed with `failed_chat_retry_not_authorized` while its retained remote executor still needed verification. The new recovery generation existed, but the chat action failed before it could save a dispatch wait. This occurred after the repairs in #17267 were confirmed live. The local regression found a second gap. A processed chat action with a deferred wake was not reconsidered after containment. Re-entering normal admission also tried to insert its existing durable receipt again. A historical hold for the same failed run could also hide the current Retry's workspace strategy, so the request still waited after the old executor was contained. Preparation could then read that sibling instead of the recovery record that owned the fresh successor, losing the fresh-workspace decision. Expected behavior Save the exact authorized Retry while its matching recovery generation verifies the executor. Once containment is recorded, reconsider the same chat receipt through normal admission. Start one successor with the original source batch. Keep publication, principal, pause and budget checks. Steps to reproduce 1. Admit a GitHub or Telegram request and record a failed native run with a retained remote lease. 2. Leave the executor's termination unconfirmed and exhaust its prior recovery generation. 3. Use ordinary task Retry to create a new recovery generation. 4. Process the connected Retry. The old admission code fails instead of retaining a wait. 5. With a deferred receipt, record exact containment and reconcile again. The old dispatch path leaves it stranded. Paperclip version or commit Reproduced from `412914cfa78d5abefcc34d3eb8d4bb86e461caac`. The branch is rebased onto current master. Deployment mode Hosted tenant for the reported failure. Isolated local PostgreSQL and the assembled heartbeat/chat services for the regression. Refs #17267 and #16965. This change does not add a Cloud dependency or change the absence protocol. ## What Changed - Permit a saved admission wait only for the exact current task Retry and active source recovery generation. - Keep promotion and execution dependent on independent containment proof. - Bind a Retry to active automatic recovery without replacing its generation, attempt count or deadline. - Share the current exact Retry strategy across historical holds for the same run. Retain each hold's stop and effect checks. - Select the canonical recovery record that proves the committed fresh successor during workspace preparation. - Let task receipt reconciliation return the same deferred action to its chat owner. - Recheck current board and external authority, then update the existing wake atomically with its successor. - Preserve the exact chat source batch. Keep generic board-comment filtering unchanged. - Add GitHub and Telegram cases for waiting, containment, one successor, stale generations, source edits, endpoint pause and supersession. - Update recovery documentation and capability maps. ## Verification - Red: real PostgreSQL tests reproduced the admission rejection for GitHub and Telegram. Six active-recovery cases then failed on the generation-ID assumption. Two same-run historical-hold cases remained deferred after containment on the old strategy lookup. A preparation regression then returned no readiness when the historical hold was stored first. - Green: 73 selected chat integration tests and 394 continuation, task Retry, workspace recovery, recovery-state and workspace-preparation tests passed. The chat tests use the real chat admission and heartbeat queue. Unrelated capacity stays occupied so no real provider starts. They assert no executor before containment, one successor afterward, unchanged source leases while waiting, preserved recovery budgets, and denial after authority changes. A separate source hold still blocks work. Both sibling insertion orders pass through the committed-readiness lookup and real workspace preparation: the new directory has independent Git storage and files, and unpublished old work remains intact. - `pnpm -r typecheck`, `node scripts/check-maps.mjs` and `git diff --check` passed after the review fixes. The source build passed on `f6660efa48e37a80cd9857217a537606de85b5c8`. - The local host reached its PostgreSQL semaphore limit. Successful runs used a test-only loader to reduce disposable cluster connection and worker limits. No repository database helper or application setting changed. - The full local `pnpm test:run` began before the final review edits, failed in its first server phase and was stopped after those failures. Later phases did not run. This is not a full local pass. The transient Retry-notice case passes after fixture isolation; the four Discord close-owned publication cases also pass in isolation. Other broad-run failure causes are not established. - Exact-head CI and security checks pass on `f6660efa48e37a80cd9857217a537606de85b5c8`; Greptile is 5/5 with zero unresolved review threads. Optional Paperclip Review completed neutral with a 5/5 static assessment of all 11 changed files and no findings; it did not run tests or hosted acceptance. All 57 current-head checks are terminal: 54 success, two skipped and the optional neutral review, with no failures or pending checks. - The real hosted workflow confirmed that the deployed absence repair recorded containment. A later ordinary UI Retry was accepted and saved a durable request, but it remains behind the historical source hold. The useful existing review remains unchanged. This follow-up is not deployed; a live successful successor and final task disposition remain unverified. No missing files are claimed recovered. ## Risks - The existing task reconciliation sweep can restage a deferred chat action. This reuses its receipt and generation; it does not grant new execution or publication authority. - Dispatch must keep the board Retry identity separate from the connected request's principal. The regression checks these bindings and preserves current chat authorization at each phase. - Historical holds reuse a strategy only from the locked same-run recovery that names the current Retry intent. A different run, malformed recovery record or independent source hold remains fenced. - Unknown executors without a matching active recovery still fail admission. Promotion and execution require containment even when admission saved a wait. - No schema or API shape changes. No credential changes, manual deployment or service restart. ## Model Used OpenAI GPT-6 through Codex, with repository tools, code execution and browser verification. The exact served model variant and context window are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 07:51:34 -07:00
DottaandPaperclip	3bd709df8b	Operationalize Product E2E accounting, tracing, and scheduling (#17009 ) ## Thinking Path > - Paperclip manages agents and the work they perform. > - Product E2E tests exercise real agents, tools, setup and cleanup. > - Repeated runs need retained costs and traces, including failed attempts. > - A passing retry must not hide earlier spending or feedback. > - Recurring work also needs admission limits, private reports and failure detection. > - This change supplies those controls while keeping paid activation behind existing gates. ## Linked Issues or Issue Description Refs #16996 and #17158. This is the consolidated landing of the reviewed eval operations stack: #17019, #17024, #17025, #17027, #17030, #17033, #17035, #17038, #17043, #17044, #17048, #17050, #17051, #17092, #17096, #17100, #17107, #17109, #17112, #17115, #17117, #17121, #17123, #17127, #17130, #17133, #17134, #17137, #17140, #17143, #17154. The repository permits squash merges only. A local rehearsal showed that squashing each slice into master causes repeated conflicts in dependent slices. This PR preserves their commits and lands their combined implementation in one reviewed integration. Superseded slices will close after this merge. ## What Changed - Retain every attempt's cost receipt, including setup, failures, interruptions and retries. Separate reported amounts, estimates, missing costs and reserved capacity in reports and dashboards. - Add bounded request admission, exact model pricecards, gateway error accounting, cache policy and reconciliation evidence. Refuse unsupported routes and requests that cannot fit the allowance. - Integrate Braintrust gateway traces and OTLP correlation, with secret filtering and experiment imports. Record phase timing, including setup, feedback capture and cleanup. - Collect complaint and suggestion evidence after each attempt, including recovery after failures. Retain explicit review dispositions independently from the product grade. - Add gated daily and weekly selections, measured forecasts, setup/readiness checks, wall-time limits and retained monitor notifications. Include declared non-model overhead in admission while reporting the existing subscription separately. - Keep recurring history private, bound uploads and retention, and use isolated archive environments. Keep public publication an explicit operation. Report and publication jobs install only the committed lockfile, with scripts disabled and isolated stores; stale pins fail closed. - Integrate current master, preserve its Product E2E journey billing and privacy checks, and test that shared setup receipts do not double-count estimated agent usage. Preserve upstream fixture repairs. ## Verification - Complete integration baseline `0649a5ce7544ab20825c2183d2f421157db49f24` includes master `a3e1d915a7110a4e115b46396a3f79e6dfb4d05e` and every original stack head as ancestors. - On that integration baseline, `pnpm test:e2e:runner:unit`: 4,012 Vitest tests passed, two skipped, across 193 files; all 137 Node source-contract tests passed. - Focused billing and Product E2E journey contract checks: 24 tests passed. They cover estimates, unknown receipts, single-owner setup costs, browser-only execution and private publication guards. - Monitor and workflow wiring: all 96 Node tests passed. `node scripts/check-token-gates.mjs`: all gates clean. - `pnpm test:e2e:runner:typecheck` passes after the standard Runner TypeScript build refreshed local declarations. The first attempt rejected the new master `runner_prepared_launch` union member because this checkout retained an older generated declaration; the source union already contained that member. - Current head `492b0708a1c531a023d9ac10f0f3dc6383cc1754` removes dependency re-resolution from archive-authorized reporting jobs after the security scan flagged it. All 15 workflow-security tests pass after updating the old two-install assertion. - Hosted [CI run 39965460180](https://github.com/paperclipai/paperclip/actions/runs/39965460180) passes on the current repaired head, including typecheck, build, general tests, Runner checks, browser shards and canary dry run. Greptile is 5/5 on this exact head with no unresolved threads. The security scan passes after the fixed lockfile-resolution thread was resolved. - Paperclip Review returned a terminal neutral check but an incomplete 0-file assessment on the repaired head: its platform blocked both diff responses as prompt-injection content. This is not a successful code review. No bypass or fabricated pass was attempted; current-head Greptile, security and hosted CI are independent gates. - No paid test, reservation release or recurring-run activation accompanies this landing. Live route qualification and invoice reconciliation remain separate evidence. <details> <summary>Retained earlier validation failures</summary> Earlier broad local runs had two 15-second test timeouts (CLI poll cadence and Retry validation), one embedded-database setup timeout, and later PostgreSQL semaphore exhaustion. The exact timed-out tests passed separately without larger timeouts; the broad-run causes were not established. Those local failures remain distinct from subsequent hosted CI results. The independent #17158 fix records its Muse fixture collision, its UUID correction and hosted verification. Local Muse execution was also blocked by macOS embedded-PostgreSQL startup. No historical failure is treated as a successful paid qualification. </details> ## Risks - Admission is a local control, not a provider invoice cap. Missing provider receipts or resource charges retain unknown amounts and holds; estimates do not become billed dollars. - The bounded path admits only documented local suites and provider routes. Ordinary legacy or separate Runner protocol campaigns have different controls. Merging this code does not qualify additional routes. - Recurrence still needs the existing enable gates and current qualification/allowance evidence. Archive permissions and resources must match the documented private-environment contract. - Legacy reports may lack complete attempt or phase evidence. Reports preserve that limitation and do not infer zero usage from missing data. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools and code execution. The exact deployment variant and context-window size are not exposed by this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (targeted checks; full-suite host limitation disclosed above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 07:18:37 -07:00
DottaandPaperclip	4173059081	fix(inbox): keep advisory GitHub review waits quiet (#17318 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Mine shows a person's participation and requests that need their response. > - Automatic GitHub reviews should remain quiet until the assigned work needs human input. > - Review bots used completion requests to keep tasks in review while CI or a PR was still open. > - Those requests correctly entered Mine, despite the automatic-work filter from #16356. > - This pull request gives published reviews an existing provider-event wait in the in-review state and withdraws superseded completion requests. > - Real input requests and separate approval gates keep their current behavior. ## Linked Issues or Issue Description Refs #16356 Refs #13314 An advisory score below 5/5, pending or failed CI, or an unmerged PR does not by itself require a person's approval in Paperclip. The bot can track those conditions without adding a completion card to Mine. ## What Changed - Explain the quiet review lifecycle in the GitHub review instructions. Use a yielded provider-event wait, or an existing saved monitor, after publication. Report done when the configured completion conditions are met. - Keep a confirmed, published review in `in_review` during that wait. Reuse existing authorization and monitor delivery. Add no schema, timer, or polling loop. - Recognize the committed external wait in review attention and recovery. Shared execution receives a verified publication flag; GitHub receipt lookup stays with the GitHub module. Other chat waits keep their current status. - Withdraw an older system-created completion request only when the same bot has published both reviews and its new report has no attention request. Keep explicit cards, separate approval gates, and requests without the prior review evidence. Finish validation stays read-only; withdrawal commits atomically with the new status after checking current governance, the admitted result, and both low-risk contracts. - Update the product rule, capability maps, and regression coverage. ## Verification - Final commit `7c502118eec36028d1b4537bb8797ef893940bf0`: 54 GitHub checks passed, two intentionally skipped, none pending or failed. Greptile is 5/5, the security scan passed, and all review threads are resolved. - Passed: `pnpm -r typecheck` and `pnpm build`. Rechecked server typecheck and build after the final authorization guard. - Passed: 195 tests across `external-chat-wait.integration.test.ts`, `status-arbiter.test.ts`, `review-instructions.test.ts`, and `issue-liveness.test.ts`. The database suite used an isolated PostgreSQL 18 container through `PAPERCLIP_TEST_DATABASE_URL`. - Acceptance coverage: 3/5 stays in review outside Mine; 5/5 with pending completion conditions stays in review; 5/5 with satisfied conditions finishes quietly; a real repository-access request enters Mine until resolved. Also covers obsolete-card withdrawal, transaction rollback, rejected reports, separate approval gates, multiple reviewed heads, saved monitors, and revoked endpoints, repositories, or membership. - Passed: `node scripts/check-maps.mjs`, `node scripts/check-module-boundaries.mjs`, and `git diff --check`. - The initial acceptance run exposed a missing authenticated user in the test fixture. Added the user required by the existing addressee authorization check. All 154 tests in that database suite then passed, including the final rerun. - Review found premature withdrawal during validation, stale repository access, and a missed receipt when a run reviewed multiple heads. The revision fixes all three and includes acceptance regressions. The saved-monitor commit guard now accepts the verified published-review state. - `pnpm test:run` was attempted but could not complete on this Mac. Embedded PostgreSQL failed to allocate semaphores (`semget: No space left on device`); a direct initialization probe confirmed the host resource limit. The isolated database run proves the affected suite, not the full repository suite. - These tests exercise persisted review receipts, finalization, and Mine queries. They do not prove a live model followed the new instructions or submit a live GitHub review. ## Risks - The bot must follow the new lifecycle instructions. A real explicit attention request still enters Mine. - Existing obsolete cards are withdrawn when the same bot publishes a later review and reports no human dependency. This change does not run a bulk cleanup of historical inbox entries. - Review attention now recognizes an external response path. It requires an applied current status decision, a successful bound run, and current conversation, repository, and principal authorization. - No migration or UI change. The full local test suite is unverified because of the host database limit; CI passes on the final commit. ## Model Used OpenAI Codex, GPT-6, with reasoning, repository inspection, code editing, and test execution. The exact deployment model ID and context-window size are not exposed in this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (195 focused tests; full-suite limitation above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 07:14:40 -07:00
DottaandPaperclip	5ef385f7e6	Run reusable Product E2E journeys across deployment targets (#17262 ) ## Thinking Path > - Paperclip manages agents and their work. > - Product E2E tests check complete user journeys. > - The current harness prepares self-hosted instances. > - Hosted workspaces need separate authentication, setup, and cleanup. > - The same in-app story should run on either deployment. > - This pull request adds deployment targets and explicit prerequisites. > - The private Cloud adapter owns fleet access and lifecycle operations. ## Linked Issues or Issue Description Subsystem affected Product E2E harness, fixture registry, and reports. Problem or motivation Self-hosted test setup cannot qualify a deployed Cloud workspace. Copying each story into a second harness would make its assertions diverge. Proposed solution Add explicit deployment targets, setup recipes, and ordered campaigns. Reuse a no-model task-properties story on both deployments. Let fresh Cloud onboarding provide verified context for that same story. Keep Cloud lifecycle and credentials behind a private target provider. Roadmap alignment This supports Cloud deployment qualification and the existing agent eval system. It adds no application endpoint or scheduler. Related work: #17112. The separate onboarding gateway UI prerequisite is #17253. ## What Changed - Add explicit target configuration and a credential-free preparation preview. Keep historical selectors and the current `--all` matrix unchanged. - Resolve requested fixture prerequisites, including a run-owned backlog task prepared before measurement. Reuse verified onboarding context without counting recipe setup as onboarding coverage. - Add task assignment persistence and the actual onboarding parent/child first-task journey, with an independent artifact oracle and native trace requirement. - Preserve failed boundaries and blocked dependents, deployment identity, costs, phase outcomes, and private evidence in the existing dashboard. - Validate success assertions and accounting before successful cleanup. Bound collection by the campaign deadline and cleanup/evidence hooks by a separate ten-minute grace, with cancellation and stop-only fallback. - Stage dashboard rendering separately so a timed-out renderer cannot overwrite retained failure results. Add evidence finalization hooks so the private provider can save evidence before target deletion and lease release. - Reject private journey bundles in the public history publisher. Browser-only results have no model or provider. ## Verification - The unchanged task-properties story passed on managed self-hosted and a registered testing Cloud workspace. Authentication, setup, journey, collection, and cleanup passed. Model calls: zero. Earlier attempts remain private and retained. - Final journey and report tests: 96 passed in eight files. Catalog compatibility: 39 passed. The wider focused run exposed three stale master catalog expectations; these now match the already shipped case-handoff v4 metadata and four-cell matrix. The default 385-cell selection is unchanged. - Review regressions reproduced premature success for a zero-request receipt and a missing required assertion. Both now fail before success cleanup while retaining cost receipts. The initial lifecycle fixes passed 33 checks. A subsequent review exposed a late renderer write that could erase saved failures; rendering now writes into isolated staging and publishes HTML only after it settles. Combined campaign, fixture-contract, and retained-report checks: 67 passed, including late writes, delayed timer callbacks, stalled collection, release, stop, evidence hooks, and browser close. Harness typecheck passed after the fixes. - A broader harness unit attempt completed 3,601 passing cases, two failures, and two skips. The failures exposed a stale Pi count and an omitted active Copilot fixture in the Daytona image input closure. The focused compatibility fixes passed 83 tests. After all source fixes, the full harness support suite passed: 160 files, 3,621 tests, two skips, plus 128 native-source checks. The earlier failures remain recorded. - Harness typecheck, repository typecheck, repository build, map checks, and Storybook catalog checks passed on the rebased branch. - The required full `pnpm test:run` attempt timed out after 35 minutes. It had 1,195 completed passing cases and one chat integration failure before its first suite completed. That case and nine adjacent cases pass in isolation. The original failure remains unresolved and retained; this is not a full-suite pass. - Current-head remote checks (`39b2a1ed8c49222fabd1ba8e225cac81bff5c6f7`): 53 passed, two skipped, none pending or failing. Greptile completed successfully at 5/5 with no unresolved review threads. - Fresh paid Cloud onboarding, dependent reuse, and disposable cleanup have not been qualified live. They require deployment of the separate UI prerequisite and actual Cloud gateway budget admission. The trusted Cloud workflow remains disabled. - Final live verification at `39b2a1ed8c`: the unchanged shared story passed on managed self-hosted and the registered testing Cloud stack using the prepared backlog-task recipe. Both cleaned up successfully with zero model calls. Independent private S3 checks confirmed final evidence and released QA identity/stack leases. Cloud evidence remains private. ## Risks - A deployed UI or Cloud control-plane change can invalidate selectors or provider contracts. Version drift fails the attempt. - Cloud evidence and lifecycle recovery are private-provider responsibilities. A pending cleanup or evidence upload must keep the target unavailable for reuse. - No release is deployed or pinned by this change. No automatic story retry is added. The initial pilot retains its explicit $10, one disposable workspace, and 30-minute bounds. ## Model Used OpenAI GPT-6 through Codex, with reasoning, repository tools, code execution, and delegated agents. The runtime does not expose a more specific deployment model identifier or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-23 06:27:49 -07:00
Nicky Leach	412914cfa7	Read native plugin task output files on the execution host (#17285 )	2026-10-22 23:38:09 -07:00
Devin FoleyandPaperclip	d69631b01b	feat: reopen a fresh workspace with a warning when its restore point expired (#17269 ) ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - #17248 and #17257 let git cache expiration remove closed issues' worktrees after saving encrypted restore points > - Restore points are kept for 90 days after an issue tree closes, then deleted > - Reopening a workspace whose restore point had expired failed, so the issue could not get a workspace back > - This pull request starts a fresh workspace on the same branch instead, and warns that the saved work is gone > - The benefit is that old issues can always be resumed, with a clear record of what was lost ## Linked Issues or Issue Description Refs #17248, #17257. Bug report described in-PR (bug report template). What happened? After `PAPERCLIP_GIT_RESTORE_POINT_RETENTION_DAYS` passes, the retention worker marks the restore point expired and deletes it. Reopening that workspace then fails with the git cache restoration error, every time. Expected behavior Reopening succeeds with a fresh workspace on the issue's branch, and the issue says that unsaved work from before the cleanup is no longer available. Steps to reproduce 1. Enable restore points and let a closed issue's worktree expire with unsaved work. 2. Let its restore point expire. 3. Reopen the issue: the workspace rebuild fails. Paperclip version or commit: `master` at `650581b2f0`. Deployment mode: any. Agent adapter(s) involved: not adapter-specific (core). ## What Changed - Detection: `gitRestorePointExpired(recipe)` is true when the main restore point or any nested checkout's restore point has `expiredAt`. - Reopen (`execution-workspaces.ts`): - `restoreGitCacheBase(..., { freshStart: true })` prepares the base without the saved commit or the bundle. - The saved recipe is left out of the metadata handed to the runtime, and dropped from the stored metadata only when the reopen succeeds, so a failed attempt can be retried and still warns. The runtime recreates the worktree on its branch: the branch tip if the repository still has it, otherwise a new branch from the base ref. When the expired recipe had nested checkouts, the project's repositories are prepared with `prepareProjectRepositoryWorkspaces` (without the expired saved state) right after the worktree is recreated and before the provision command runs, with the same copy identity and access rechecks as other reopen copies. - The result carries `warning`, the `execution_workspace.reopened` activity records `restorePointExpired` and the warning, and a system comment on the issue posts the warning in the same transaction. - Retention: `markGitRestorePointExpired` now also marks nested checkouts' restore points, matched by object key. - Docs: `doc/recoverable-disk-cleanup.md` describes the fresh start. No code comments were added. ## Verification - `pnpm exec vitest run src/__tests__/git-restore-point-expiration.test.ts` (in `server/`): 13 passed. New case: an expired restore point reopens fresh with the warning and the system comment, on the original branch, even after the base lost the branch and its unpushed commit; the recipe is gone. A first attempt that fails keeps the expired recipe, and the retry still warns and comments exactly once. - New nested case in `git-workspace-expiration.test.ts`: an expired nested checkout restore point is marked on the checkout entry and reopens fresh with the warning, with the nested repository cloned again at its pushed commit and without its expired uncommitted files. - Mutation checks: each of these fails a case: - removing the fresh base path (reopen fails); - removing the system comment; - handing the runtime the old recipe; - dropping the stored recipe before the rebuild succeeds; - marking only the main restore point expired; - skipping nested repository preparation on a fresh start. - `pnpm exec vitest run git-workspace-expiration git-workspace-contents git-restore-point execution-workspaces` plus the environment selection and heartbeat resume suites (in `server/`): 257 passed. - `pnpm run typecheck` (in `server/`): pass. - Not run: server build and boot smoke. ## Risks - Only with restore points enabled: recipes get `expiredAt` only from the restore point retention worker, so nothing changes otherwise. - Lost work is intended: the fresh workspace has no uncommitted files and, when the repository no longer has the branch, no unpushed commits. The warning says so. - Recipe dropped: after a fresh start the workspace no longer carries the expired recipe, so later expirations start from a clean slate. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code with tool use (shell, file edits, tests). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>	2026-10-22 22:20:52 -07:00
Nicky Leach	2cd6eb27c7	fix(runner): record the plugin-prepared working directory (#17243 )	2026-10-22 20:42:26 -07:00

1 2 3 4 5 ...