mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-08 12:33:05 +01:00
c6cb7e2a2d42743c0c3c4ead2cbad4a299f9a2e7
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nicky LeachandClaude Opus 5.5
|
be20d4ad97 |
fix(projects): keep projects on the storage chosen at creation (#19036)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A project stores its work in Core/local storage or in a storage plugin > - Today, Core finds the installed default each time it reads a project > - So installing a default storage plugin moves existing projects, without a saved change > - Agents that cannot reach the new storage then lose access to new workspaces > - This pull request makes Core apply the default once, when it creates a project > - The benefit is that a project stays on its storage until a person moves it ## Linked Issues or Issue Description No issue exists. This describes the problem. I searched open pull requests for related work and found no duplicate. **What existing behavior does this improve?** How Core chooses the storage for a project when a plugin claims `defaultForProjects`. **Current behavior** Installing such a plugin changes the storage of every existing project with no explicit choice. Disabling the plugin changes it back. Nothing is saved, so the project has no fixed owner. Execution environments and storage are separate contracts, so an agent can lose access to new workspaces on a project that it used before. **Proposed behavior** Core saves the storage choice on the project when it creates the project. A project with no saved choice uses Core/local storage. A plugin-backed project stays attached to its plugin. A Core/local project can move to a plugin. **Reason and benefit** A project stays on its storage until a person moves it, so installing or disabling a plugin does not change where work goes. **Alternatives considered** - Keep the default at read time and add a process that saves a choice on existing projects when a plugin arrives. This adds a background step and a race with new projects. - Add a new database column. The existing policy field already holds the choice, so a migration is not necessary. ## What Changed - `resolveProjectWorkspaceProvider` reads only the saved choice. It no longer applies an installed default. - `withDefaultProjectStorage` applies the default when Core creates a project, including a personal private project. With more than one default, a project with no explicit choice is refused. This is the existing conflict rule, now applied at creation. - `projectService.update` refuses a change of storage on a plugin-backed project. An update that omits the field keeps the attachment. - Plugin lifecycle delivery sends project events to a storage provider only for projects attached to it, and for deleted projects. Core acknowledges the other events for that provider. Other plugins continue to receive all project events. - The project settings control has no "Inherit" option. It is locked on a plugin-backed project. - `ProjectWorkspaceProviderConfiguration.source` no longer has the `"default"` value. - A project with no repository or folder ignores its saved storage choice when a task runs. A storage plugin cannot manage this type of project, so its tasks use a plain task directory. Without this change, the second run of a task reset the agent session, and remote agents could not use an empty task directory. - The plugin SDK README and the plugin authoring guide describe the new rules. ## Verification - `npx vitest run server/src/__tests__/project-workspace-provider.integration.test.ts server/src/__tests__/plugin-lifecycle-delivery.test.ts server/src/__tests__/plugin-workspaces.integration.test.ts packages/plugins/sdk/tests/workspace-provider.test.ts`: 93 tests pass. - `npx vitest run src/components/ProjectProperties.concurrency.test.tsx` in `ui/`: 9 tests pass. - `pnpm --filter @paperclipai/server typecheck` and `pnpm --filter @paperclipai/ui typecheck` pass. - `npx vitest run` on `heartbeat-session-preparation`, `workspace-preparation`, `remote-workspace-source`, `project-workspace-provider.integration`, and `plugin-workspaces.integration` in `server/`: 231 tests pass. - I did not run the full test suite locally. CI runs it. - I did not test this change on a running instance. ## Risks - **Behavior change at upgrade:** an existing project that uses a plugin only through the installed default, with no saved choice, moves to Core/local storage after upgrade. Its plugin files stay in place, but new work does not use them until an operator sets `storageProviderPluginId` on the project. This pull request has no automatic backfill. - **Missed lifecycle events:** a provider does not receive events for a Core/local project. If the project moves to that provider later, the update event starts its reconcile. - **One-way move:** a person cannot move a plugin-backed project back through the API or the UI. This is intended. If a plugin is disabled, the project fails closed, as it does today for an explicit choice. - This pull request does not stop a person from assigning an agent to a project with storage that the agent cannot reach. That is follow-up work. - This changes core plugin behavior, and I did not discuss it in `#dev` first. Maintainers can tell me if they want a discussion before review. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), through Claude Code, with tool use and code execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky LeachandClaude Opus 5.5
|
11ac90e31f |
fix(build): typecheck server/src/__tests__ and fix every type error (#18739)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server has about 1,100 test and helper files in `server/src/__tests__` > - `server/tsconfig.json` excludes that directory, Vitest does not check types, and no script runs the tsconfig in that directory > - So a test can call a function with the wrong arguments and still pass, and no tool reports it > - This gap hid 2,168 real type errors, and some of them hid tests that did not test what they claim > - This pull request fixes all of these errors and adds the directory to the server layers from paperclipai/paperclip#18737 > - The benefit is that a type error in a server test now fails the typecheck, with no baseline and no backlog ## Linked Issues or Issue Description Refs paperclipai/paperclip#18737, which is merged and added the server layers. Refs paperclipai/paperclip#18738, which is merged and checks the test projects outside `pnpm -r`, so it is not a duplicate. Refs paperclipai/paperclip#18137, which records that this directory has no check. **What happened?** No tool checked the types of 1,074 of the 1,101 files in `server/src/__tests__`. I searched `package.json`, the workflows, and all tsconfig files. 27 files are in the program of another tsconfig, and these 27 have 0 errors. 11 of the 27 are in `helpers/maintenance/tsconfig.json`. No script ran that project when I measured this. paperclipai/paperclip#18738 now runs it. **Expected behavior** The server typecheck checks these files, and a type error fails the check. **Steps to reproduce** 1. On the base commit, run `tsc -p server/src/__tests__/tsconfig.json --noEmit`. It reports 6,536 errors and needs 12.3 GiB. 2. Add a type error to any file in `server/src/__tests__`. 3. Run `pnpm --filter @paperclipai/server typecheck`. It passes. **Paperclip version or commit** I measured the error breakdown below on commit `8dfd632452`, with TypeScript 7.0.2. The branch now includes master up to paperclipai/paperclip#18865. **Deployment mode** Not applicable. This issue applies to development and CI. No production source file in the server changes. ### Error breakdown Each row adds one change to the row above. | Step | Total errors | Change | Cause | | --- | --- | --- | --- | | Config as committed | 6,536 | | | | Add `src/types/*.d.ts` | 4,447 | -2,089 | Config. The Express request types were missing. | | Set `rootDir` to the repository root | 2,436 | -2,011 | Config. | | Set `allowImportingTsExtensions` | 2,181 | -255 | Config. Vitest accepts `.ts` imports. | | Add `ES2024.Promise` to `lib` | 2,138 | -43 | Config. Node 24 has `Promise.withResolvers`. | | Set `allowJs` for `.mjs` imports | 2,148 | +10 | Config. It removes 8 `TS7016` errors and shows new real errors. | | Set `declaration` to false | 2,142 | -6 | Config. Tests emit nothing. | | Count only files in `server/src` | 2,058 | -84 | Other packages: 53 Daytona, 29 Runner test, 2 UI. | So 4,478 of the 6,536 errors (69%) came from config or scope, and 2,058 are real type errors in 314 files. The 29 Runner test errors are also real, which gives 2,087 errors. Tests that master added after this measurement had 81 more errors, so this pull request fixes 2,168 errors. This table classifies the first 2,058 errors. | Kind of real error | Errors | | --- | --- | | Wrong argument or assignment type | 1,023 | | Stale or partial fixture objects | 525 | | Property that the type does not have | 317 | | Missing narrowing of null or unknown | 101 | | Implicit any | 60 | | Missing name or module | 16 | | Other | 16 | ### What the type errors hid Most errors were stale fixtures and loose mocks. These errors hid real test problems, and this pull request fixes them: - **A test that never ran.** `instrumentation.test.ts` required `sdk-trace-base`, which is not installed. So 3 span tests were skipped. They now run and pass. - **Assertions that could not fail.** - `issue-thread-interactions-service` used a company ID as an issue ID. - `status-card-evaluation-work` compared `[]` with `[]` in a rollback check. - `slack-conversation-lifecycle` read `item.issueId`, which does not exist. - `email-channels` used a permission key that does not exist. - `heartbeat-project-env` imported `LOW_TRUST_REVIEW_PRESET`, which is not exported. The value was undefined. - **Postgres processes that did not stop.** 11 test files called `stop` on the test database helper. That method does not exist, and the helper method is `cleanup`. - **Arguments that the service ignored.** - `methodKey` is now `connectionMethodKey` in the Daytona, Unikraft, 1Password, and Railway tests. - 108 calls passed `"owner-user"` to a parameter that expects an options object. - 9 chat wizard fixtures passed `managementContext` as `source`, so they ran `existing_app` and not `manifest`. - 2 tests passed a `{ userId }` actor that the function ignores. - Inserts into `environments` passed a `companyId` column that the table does not have. - **Problems in tests that master added during this work.** - `sandbox-creation-operations` read `database.connectionString` when an external test database was set. In that mode `database` is undefined, so the fixture would stop with a TypeError. It now uses the external URL. - `idle-completed-history` set a Dot operation to `dispatched`. That is a Muse status, and the Dot type does not have it. The fixture now uses `admitted`, which is a real in-flight Dot status. ### Possible product problems, not changed These may be real problems in production code or types. I did not change them, because this pull request does not change production behavior: - `decisions-service` `service.create` does not validate the lenient `cancel_issue_tree` input. - `createConnection` ignores `input.connectionPurpose`. - The `createConnection` route schema default stores `{}` for `transportConfig`. - `prepareManagedAiRuntime` accepted the invalid value `mode: "connection"`. - `createBinding` accepts invalid `projectionClass` strings outside class 3. - `ensureManagedProjectWorkspace` forwards an untyped `repoRef` by spread. - `routineVariableSchema.label` is optional, but `RoutineVariable.label` is required. - `ToolRunContext.projectId` may be too strict. - `withdrawInteraction` reads `issue.status`, which its type omits. - `issueService.list` and `reconcileNativeFinalizations` omit fields that they return at runtime. - `createGitHubExternalObjectProvider` handles `tokenProvider: null`, but its type forbids this value. ## What Changed - Fixed the 2,168 type errors. The diff changes 322 test and helper files. - Corrected `server/src/__tests__/tsconfig.json`. Editors and the typecheck now read the same options and report the same errors. - Added three `test-tree` layers to `server/scripts/typecheck-layers.mjs`. They read the server layers as declarations, and any error fails them. A hash of the path selects the shard, so a new file does not move other files. - Added `server/scripts/typecheck-stubs/daytona-sdk.d.ts`. Server tests reach Daytona plugin source, and the root install does not have `@daytonaio/sdk`. - Changed two imports in `ui/src/lib/issue-execution-policy.ts` to use `.js` extensions, because server tests import this file. - Made `declarationsConfig` and `--declarations-only` skip the `test-tree` layers, because these layers emit nothing. Other projects read only the source layers. Added a test for this. - Updated `doc/DEVELOPING.md`. ## Verification Run these commands: ```sh pnpm --filter @paperclipai/server typecheck pnpm exec vitest run server/scripts/typecheck-layers.test.mjs ``` Results on macOS (arm64), Node 24, 16 cores. Each check ran after the merge of paperclipai/paperclip#18738 unless the row says otherwise: | Check | Commit | Result | | --- | --- | --- | | Server typecheck, `PAPERCLIP_TYPECHECK_LOW_MEMORY=1`, all 7 layers | `6e45a08573` | Exit 0, 0 errors, 62 seconds, peak 3.52 GiB | | `node scripts/typecheck-test-projects.mjs --low-memory` | `a3306bd13e` | Exit 0, 0 errors, 42 seconds, peak 3.25 GiB | | The same script with `--reuse-declarations` | `1b878aabc0` | Exit 0, 0 errors | | `typecheck-layers.test.mjs` and `live-recovery-work.test.ts` | `01db5634e4` | 90 passed. These include real compiler runs for a pass, a type error, and an invalid tsconfig. | | `tsconfig.declarations.json` after a full server typecheck | `0fb1e4fb0a` | It references only `below`, `cycle`, and `above`. No generated config names a `tsconfig.layers.json` file. | | Owned runtime test in `idle-completed-history` | `e8f2f448ac` | Passed, with the fix that master later replaced | | `idle-incoming-wake`, `plugin-idle-manager`, and the changed `chat-channels` test | `6e45a08573` | 30 passed | | Runner `native-execution.test.ts` | `eab4dcc1f4` | 114 passed | | The 19 server test files that changed since `eab4dcc1f4`, and `typecheck-layers.test.mjs` | `259737b700` | 4,275 of 4,277 passed. The 2 failures are in `chat-channels` and are in the list of 7 below. | The peak is the largest single process, from `/usr/bin/time -l`. Peaks for the server typecheck were between 3.42 GiB and 3.56 GiB across 8 runs. Peaks for the test projects were 3.25 GiB and 3.37 GiB across 2 runs. I did not measure a cgroup. **Runtime check of the fixes.** Types do not prove that a test still works. So I ran all 309 changed server test files, 11,929 tests, before the rebase onto paperclipai/paperclip#18737: | Result | Tests | Detail | | --- | --- | --- | | Passed | 11,913 | Includes the 2 tests and the 54 `dot-runner` tests that passed in a later run, as described below. | | Failed | 7 | The same 7 tests also fail on the commit before the fixes. | | Skipped | 9 | These tests have their own skip conditions: `sentry` 5, and 1 each in `bundled-plugin-registration`, `customer-success-cloud`, `onepassword-connection`, and `workspace-runtime-https-live-exercise`. | The 7 tests that fail with and without this change, on my machine: - 3 tests in `chat-channels` and `email-channels` stop with `ENOENT` on a `SKILL.md` file. The path is in the parent checkout, above the git worktree that I used, so I expect that the worktree is the cause. I did not prove this. - `workspace-instance-cleanup` compares `/private/var/...` with `/var/...`. This is a macOS symlink difference. - 2 tests in `git-reproducible-managed-checkouts` get `unknown_base_git_files` and expect `unpushed_managed_checkout`. I did not find the cause. - 1 more test in `chat-channels`, `materializes direct external-chat finals once`, gets `undefined`. I did not find the cause. CI is the check for these 7 tests. `ci / verify` passed on `eab4dcc1f4`. Failures during this work: - My fix to `agent-runtime-connector-access` changed an invalid permission key in the setup, but not in the assertion. The runtime check found this. I corrected the assertion, and the file passes. - `heartbeat-plugin-environment` failed one wait in the full run. It passed three times alone, so I think that load caused the failure. It can fail again under load. - `dot-runner` skipped all 54 tests in the full run because `cargo` was not on my `PATH`. It then passed 54 of 54 with the toolchain on the `PATH`. - My first `lib` value removed the default DOM types, and the error count went up by 375. I added the DOM libraries. - My Daytona stub with bare `any` left 48 errors, because generic helpers then inferred `unknown`. I added shapes that return promises. - One fix to `agent-secrets-routes` used `class_3_static_lease`, and the test failed. That class allows only specific paths. I reverted to `unclassified`. - `ci / General tests (server (8/12))` failed on `259737b700` because of the flaky port in `idle-completed-history`. paperclipai/paperclip#18853 fixed this test on master, so this pull request keeps that fix. **Audit of the diff.** I counted these patterns in added and removed lines, to find fixes that only hide an error: | Pattern | Added | Removed | | --- | --- | --- | | `@ts-ignore` | 0 | 0 | | `@ts-expect-error` | 0 | 1 | | `.skip` | 0 | 0 | | `as any` | 9 | 9 | | `: any` | 11 | 2 | | `as never` | 8 | 2 | | `as unknown as` | 35 | 4 | | `Promise<unknown>` | 16 | 5 | | `expect(` | 128 | 113 | No hunk removes more `expect(` calls than it adds. Three removals of test content were intentional, and I checked each one: - `muse-control` removed `signalToken`. Line 1012 already asserts that it is absent. - `merged-pr-confirmation-sweep` removed legacy aliases. Other tests cover them. - `heartbeat-workspace-branch-containment` removed a `finalize` branch that no test row can reach. Some partial fixtures stay unchecked through `as unknown as` and `Promise<unknown>`, mostly in `issue-agent-mutation-ownership-routes` and `issue-comment-reopen-routes`. **Accuracy of the check.** One program over the tree and the three shards reported the same errors at the same positions. Memory, as the peak of the largest process on one thread, was 3.93 GiB for one program and 2.43 GiB for the largest shard. I measured these two values before the fixes. Not run or not proven: - I did not run the full `pnpm test:run`, or a root `pnpm build`. I expect CI to run the full suites. - I did not run the 4 GiB container check, or the full `scripts/typecheck-low-memory.mjs`. The peaks above come from macOS, so they are not a Linux cgroup measurement. - I did not run the UI typecheck after the merges. - `workspace-runtime-https-live-exercise` is skipped, so its `cleanup` fix did not run. ## Risks - **A large diff in test files.** 328 files change. A pull request that edits the same tests can get merge conflicts. New tests on master can also add type errors before this merges. Any such error fails this check and needs a fix here. - **A higher cost for changes to source types.** A change to a type can add errors in test files that the author did not touch. The author must then fix them. This is the purpose of the check, but it is new work. - **Changed test behavior.** Some tests now run paths that they did not run before, such as the 9 chat wizard fixtures and the 3 span tests. They pass here, but they can be more sensitive to timing or to the environment. - **The Daytona stub can go stale.** The stub is written by hand. If the plugin uses a new SDK member, the typecheck reports an error until someone updates the stub. - **The release path.** `release-verify.yml` and `scripts/release.sh` run `pnpm -r typecheck`, so they now run this check. - **The tsconfig change.** The test transform reads `server/src/__tests__/tsconfig.json`. The options that I changed are for type checks, and the runs above passed. - **A gap that remains.** Tsconfigs in `packages/paperclip-runner` and in most sandbox provider plugins exclude `*.test.ts`. This check does not read most of those test files, so they can still drift. That work is separate from this pull request. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used Anthropic Claude Opus 5.5 (`claude-opus-5-5`) through Claude Code, with reasoning, tool use, code execution, and subagents. The runtime does not show the context window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [ ] I have run tests locally and they pass (7 tests fail locally, with and without this change. See Verification.) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky LeachandPaperclip
|
aab87ce707 |
Prove workspace delivery before inspecting cleanup candidates (#18493)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Completed tasks leave workspaces that need safe cleanup. > - Providers without ancestry inspection currently need a recorded merged PR before cleanup can start. > - A squash merge changes commit identity, so direct ancestry cannot prove that delivery. > - This change proves delivery from the finalized HEAD and captured destination before storage inspection. > - Fresh storage checks and a second delivery check still protect removal. > - This reuses the existing worker and receipts. It adds no table or timer. ## Linked Issues or Issue Description Refs #18410. Related: #13400 addresses workspace identity during inspection; this change adds destination-aware delivery discovery. ## What Changed - Resolve direct ancestry or discover an associated merged GitHub PR without a recorded work product. Require exact HEAD, destination repository and branch, and a reachable integration commit. - Use the latest settled finalization receipt as a candidate. Reject missing, failed, pending, conflicting, or changed evidence. - Inspect storage only after positive remote proof for providers that disable ancestry inspection. Recheck HEAD, branch, source identity and delivery before removal intent. - Add an optional provider capability for a specific PR recovery hint and an optional prepared destination receipt. Older providers keep their existing protocol. - Wake cleanup when finalization writes settle or gain a workspace binding. Keep remote-only retries for concrete unresolved delivery. Park missing or unsupported candidate identity. - Reuse the source project's explicit GitHub consent for delivery checks through the existing authorization service. Revalidate the bound repository and consent before removal; missing or revoked consent retains storage without switching identities. Public sources remain anonymous. - Update the contracts, documentation, access map, and targeted regressions. ## Verification - Initial-head GitHub CI passed at `5acd4c818be6f1764f8319f42891f9956c357c26`, including full build and typecheck. Standard Greptile is 5/5 with no open threads. The first Apex review scored 3/5 and found startup latency from remote proof lookups and removal retry identity changes across provider capability changes. Both corrections are pushed in `9d1a38df3d`, with the qualified destination-ref length fix. Red regressions reproduced each issue. Latest targeted checks passed: cleanup/worker 26, provider integration 41, SDK contract 8. Head `9d1a38df3d` passed standard and Apex reviews at 5/5 with no open threads, plus all CI including full build/typecheck. Canary Dry Run initially failed on an npm 404 fetching `rollup-4.67.3.tgz`; the exact tarball subsequently returned HTTP 200, and one targeted retry passed without source changes. Aggregate verification also passed. After #18403 merged, master introduced conflicts in shared workspace cleanup and metadata handling. Resolved them in `7700832536`, preserving remote snapshot cleanup and Git delivery-proof cleanup. Targeted merge verification: 93 tests passed in the first run (59 opt-in skips), then all 16 remote lifecycle tests passed with explicit database opt-in; map checks passed. The isolated database is stopped. The merge head received standard review 5/5. Its CI found two failures in unchanged test cleanup paths: native runtime temporary-directory ENOTEMPTY and agent-chat cancellation lock timeout. Those failures remain recorded and require successful final-head verification. The subsequent project-consent change has targeted coverage: authorization36, resolver33 and cleanup19 tests passed, including revoked consent, anonymous public access and source-binding races. The positive consent test failed before the fix. Current head `c772679e05` has green GitHub CI, including full build/typecheck and both previously failing test jobs, plus standard Greptile 5/5 with no open threads. Neither cleanup failure required changes to its test path. Apex also completed at 5/5 on this exact head (check `124651548532`, summary updated 2026-10-31 20:45 UTC), with no open review threads. The PR remains draft. Earlier failures remain recorded above. - Final focused run on the rebased branch: **182 tests passed across six files**, covering the resolver, delivery cleanup, provider bridge, finalization notifications, existing workspace service and SDK contracts. The disposable test database was stopped afterward. - Targeted resolver tests: 33 passed. These cover ancestry, discovered squash PRs, wrong HEAD/base/repository, removed integration commits, bounded responses, cancellation and credential isolation. - Targeted cleanup integration tests: 17 passed. Provider bridge integration: 36 passed. SDK contract tests: 7 passed. Finalization notification tests: 9 passed. - A negative control that removed the final delivery check failed the race regression as expected. The check was restored. - Read-only live GitHub probe: the new resolver discovered PR #18410 from HEAD `3f5ade4cd42b707040e9a82fdaed343e3d736a09`, without PR hints. It proved integration commit `6484f96c28bfb11a793ce4be6cc5ac15643a5a02` reachable from observed master `5bc029bf6c1db6647e96acea323f3723fd1d3eba`. GitHub also advertised the exact original HEAD at `refs/pull/18410/head`. - `node scripts/check-maps.mjs` passed. Structural checks do not prove deployment behavior. - Local tests are targeted by user request. Full build, typecheck and test suite run in GitHub CI. A standalone server typecheck could not complete because local Runner build declarations were absent; no changed-file diagnostics were reported. - Initial local execution hit a stalled pnpm wrapper, sandbox socket denial and stale generated SDK declarations. Direct installed binaries, the authorized isolated test database and rebuilt SDK resolved those setup failures. No deployment was performed. ## Risks - Destructive cleanup still requires fresh storage safety and lifecycle checks. Remote proof alone never authorizes removal. A persisted removal intent preserves existing recovery semantics. - Legacy workspaces without an unambiguous destination or completion HEAD remain retained. This does not guess the original default branch. - Private repositories use the explicit standing GitHub consent on the bound source project, with current membership, grant, repository access and trust checks. No separate company token is required when that consent is valid. Missing or revoked consent and unavailable or rate-limited access retain storage; the worker does not borrow a task or agent identity. - A known undelivered commit keeps the existing five-minute retry because external GitHub changes have no guaranteed ingress to this lane. This can keep cleanup scheduled. Missing evidence parks instead. - Providers need the new recovery capability to clean a squash merge whose original HEAD is reachable only through its PR ref. Providers without it remain conservative. - The resolver currently supports GitHub.com. Existing local and other-provider ancestry paths stay in place. ## Model Used OpenAI Codex, GPT-6 family, with reasoning, repository tools and parallel code agents. The exact runtime model ID and context-window size were not exposed to this session. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky Leach
|
6484f96c28 | fix(workspaces): inspect storage only when needed (#18410) | ||
|
Nicky LeachandPaperclip
|
a708a13336 |
Separate plugin workspace completion from maintenance inspection (#17329)
## Thinking Path > - Paperclip manages AI agents and their work. > - Storage plugins own remote task workspaces. > - Run completion currently requests the same inspection used for storage cleanup. > - That inspection can fetch remote refs and scan files after the agent has finished. > - Completion needs retained workspace identity and durable writes, not permission to delete files. > - This change adds an optional completion operation and keeps maintenance inspection separate. ## Linked Issues or Issue Description **What happened?** A successful plugin-backed run can fail during completion because the storage provider performs a slow remote-recoverability inspection. Dirty or unpublished output does not need a deletion-safety proof to remain valid task output. **Expected behavior** Completion verifies the assigned workspace and confirms durable writes through the provider. Cleanup still requires the existing full inspection. **Steps to reproduce** Run a task with plugin-owned storage whose inspection fetches remote refs. Let the agent finish, then observe completion waiting for the maintenance inspection. Related context: #16428 added finalization observability. The duplicate search found no existing PR for this provider contract. ## What Changed - Add optional `workspaceProvider.finalizationVersion: 1` and strict `finalize` / `finalized` schemas. - Pass original run and agent context through live completion and recovered completion. - Give the new operation a 30-second deadline and keep ownership checks and lifecycle serialization. - Retain the inspection fallback for providers without the capability. Keep maintenance inspection and removal unchanged. ## Verification - SDK build passed. - Targeted SDK and completion unit tests: 7 passed. - Added integration coverage for capability selection, legacy fallback, maintenance separation, and failure propagation. - Local integration setup timed out starting PostgreSQL; the integration suite passed in GitHub CI. - Full build/typecheck and broader tests passed in GitHub CI (55 successful checks, 2 skipped). Local checks remained targeted as requested. - Diff reviewed for secrets and operational identifiers. ## Risks The capability is opt-in. Providers must not advertise it until their completion implementation is available. An unavailable provider or failed durability barrier remains retryable. No schema or stored-binding migration is required. Test-file size exception: the provider integration suite grows from 973 to 1,002 lines. The 29-line case follows the same prepared binding through legacy completion, opted-in completion, and maintenance inspection using the existing single-connection database fixture. Keeping it there avoids a second embedded-Postgres lifecycle for one transition test. The completion helper unit tests are in a separate focused file. ## Model Used OpenAI GPT-6 Codex, with reasoning and code/test tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Nicky LeachandPaperclip
|
8b4ab65b1c |
Add plugin-owned project storage and task workspaces (#17001)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Tasks can use isolated repository workspaces. > - Core currently prepares those repositories on its own filesystem. > - A storage provider may only expose repositories inside remote agent computers. > - Project storage and task workspace preparation must have one provider owner. > - This pull request adds that contract and keeps execution as a separate plugin interface. > - Core keeps authorization, durable task bindings, and lifecycle fences. ## Linked Issues or Issue Description **Subsystem affected** Plugin SDK, project workspace policy, heartbeat preparation, execution leases, and workspace lifecycle. **Problem or motivation** Core cannot prepare a task workspace when its repository storage is unavailable on the Core host. An execution lease must not be required just to prepare persistent storage. **Proposed solution** Let one plugin provide project resource lifecycle and workspace prepare, inspect, restore, and remove behavior. Core reserves a task workspace before it calls the plugin. The plugin returns an opaque durable reference. The matching execution plugin receives that reference on its task RPC. **Alternatives considered** Preparing repositories on Core would require a provider mount on Core. Preparing them only during task launch would tie persistent workspace lifecycle to execution leases and agent availability. **Roadmap alignment** This extends the remote provider model in ROADMAP.md. Related execution work is in #16978. This PR does not replace that task bridge. Native execution of managed storage requires the bridge; this change fails closed when it is absent. ## What Changed - Add one storage provider declaration with supported modes/strategies, a capability, a validated RPC contract, and a test harness. - Persist provider ownership, repository revision, and execution-workspace identity before external effects. Reconcile retries with a stable operation ID. - Prepare provider storage before environment acquisition. Keep Core filesystem preparation out of this path. - Pin the durable reference to the matching execution plugin and validate the lease binding before task dispatch. - Route inspection, reopen, and removal through the existing lifecycle process. Keep local data when remote recovery is uncertain. - Keep the application database pool free during lifecycle locks. Preserve pending workspace selections, validate source edits before retirement, and retry unfinished closed storage cleanup. - Protect provider metadata and live repository sources from caller edits. Storage edits retire only closed records with confirmed removal; retired records cannot reopen. ## Verification - Current head `8214b3ca5`: all 25 provider lifecycle tests pass with `maxConnections=1`. They cover preparation, inspection, reopening, removal, a paused provider with responsive ordinary SQL and a competing source edit, pending selection preservation, rejected source edits, cleanup retry scheduling, and providers that cache operation results, and committed cleanup notifications for storage intent and recovered state. - Final frozen-tree lifecycle run on `8214b3ca5`: all 318 tests passed in 10 files, covering workspace services, reopen, compaction, task binding, project sources, heartbeat preparation, display caching, and source locks. - Direct server `tsc --noEmit` passes. The SDK build and all 128 SDK tests passed during initial verification; later SDK changes are documentation only. - Full CI on the previous head `ec5c7209a` passed all 55 applicable checks; 2 conditional checks skipped. Standard Greptile reviewed current head `342e6d9c3` at 5/5 with no unresolved threads. All 55 applicable CI checks passed on `342e6d9c3`; 2 conditional checks skipped. The Slack browser and CLI startup jobs passed on their single unchanged-head rerun. Its Apex review returned 4/5 with one pending-restoration issue. Commit `1b58943b1` reconciles that durable intent before cleanup, advances the generation without reopening the Core row, and blocks direct removal or retirement while intent remains unresolved. Standard Greptile reviewed `1b58943b1` at 5/5 with no new findings. All 54 applicable CI checks passed on `1b58943b1`; 3 conditional checks skipped. The chat job passed on its single unchanged-head rerun. Apex then returned 4/5 for a missing cleanup wake after recovery. Commit `8214b3ca5` atomically registers cleanup intent before storage intent and recovery-result writes. Its provider test observes real committed notification hints and the visible advanced generation. Standard Greptile reviewed `8214b3ca5` at 5/5 with no new findings. All 55 applicable current-head CI checks passed on the first run; 2 conditional checks skipped. Final Apex verification completed at 5/5 on this exact head: 38 files reviewed, no new comments, and no unresolved threads. The PR is ready for review and mergeable. - The first Apex review found five issues after a standard 5/5: application-pool starvation, stale pending selections, retirement during rejected source edits, lost cleanup retries, and reused inspection identities. Commits `6d72e7fd4` and `342e6d9c3` fix each with regression coverage. Archive lock waits use dedicated connections, and source edits fail promptly if the lifecycle is busy. Retirement compares normalized values. The normalized-source regression initially failed with the raw-value comparison; after moving the check after normalization, a clean provider-suite rerun passed all 24 tests at that stage. Those threads are resolved; the earlier Apex 0/5 is not a passing acceptance result. - Earlier local integration attempts could not bind localhost inside the sandbox. The same suites passed with permission to start local PostgreSQL. - The earlier broad local `pnpm test:run` overlapped edits and reported stale-revision and checkout-path/Git-fixture failures. It was not a final-tree result. Full CI subsequently passed; the original local failure remains part of verification history. - Local repository-wide typecheck/build cannot complete because `cargo` is unavailable. Direct server TypeScript checks and affected tests pass; full Rust and build validation comes from CI. - An earlier browser run on `ff918c5a8` failed in the unchanged case-filter journey: expected “Done,” received “All statuses,” with an announcement handler recorded during the assertion. The cause is not confirmed. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39500611857/job/118896954508) remains verification history. The unchanged shard passed in the next full CI run on `ec5c7209a`; no UI or fixture change was made. - Current-head browser shard 2 failed waiting five seconds for the mocked Slack setup Success heading. This PR changes neither that UI nor its fixture. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39506070513/job/118916021232) remains verification evidence. The cause is not confirmed; the single unchanged-shard rerun passed; no UI or fixture change was made, and the original cause remains unconfirmed. - The workspace test shard passed its 3,978 general tests, then its CLI company-import/export fixture timed out after 30 seconds waiting for startup health before creating any company or workspace. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39506070513/job/118916021855) remains verification history. The child reached external-PostgreSQL startup without exiting or reporting an exception. Its cause is not confirmed; the single unchanged-job rerun passed, including the CLI fixture. No startup or fixture change was made, and the original cause remains unconfirmed. - The new cached-removal regression initially failed because its fake worker advertised availability for unrelated fixture plugins, so another workspace changed its fake checkout state. Worker availability now matches the fixture plugin; the complete 25-test provider suite passes. - On `1b58943b1`, the chat shard timed out polling for one conversation after an authorized GitHub description edit. [The failed job](https://github.com/paperclipai/paperclip/actions/runs/39510521657/job/118930193418) remains evidence. The exact case passes locally against PostgreSQL (1 passed, 1,380 unrelated cases skipped), and this PR changes neither that test nor its handler. The original cause is unconfirmed; the single unchanged-job CI rerun passed without a code or fixture change. The original cause remains unconfirmed. - The diff was checked for credentials, private references, and generated artifacts. ## Risks This is an additive provider contract with no database migration. Existing projects keep their local behavior unless their policy selects a storage plugin. A selected provider must be ready and enabled. Unknown or unsafe cleanup fails closed. Managed native execution depends on #16978. Core refuses a host Runner fallback. Providers must fence lifecycle generations, preserve ambiguous retries, and report remote recovery honestly. Provider-specific live storage and guest mount qualification remain separate from these Core contract tests. ## Model Used OpenAI Codex (GPT-6), with repository inspection, code execution, and tests. The exact model variant was not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either linked existing issues or described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links - [x] My branch name describes the change and contains no internal ticket id - [x] I have run tests locally and they pass (affected suites; broad-run limits are documented above) - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |