mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-10 12:12:25 +01:00
11c07c622c52c0c8bea2427b0b8ff94f2ea6eb21
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nicky LeachandClaude Opus 5.5
|
0b73c805e8 |
feat(environments): fill an unset instance default with a managed plugin environment (#19551)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A managed instance gets its instance-level environments from the `PAPERCLIP_MANAGED_CONFIG` document, which can declare plugin environments next to the managed sandbox > - A platform that runs managed instances wants a new instance to start with one of those plugin environments as its default, without changing the default of any instance that already has one > - #19632 made boot fill the instance default only when it is unset, local or deleted, in one locked transaction, and removed the revert, so every existing managed instance keeps a set default > - This pull request adds an `instanceDefault` flag to a plugin entry. That entry fills an unset default through the same locked fill, before the managed sandbox does > - The fill follows the same readiness rule as an admin's default write, so it never moves ready agents onto a plugin that has not prepared them > - The platform also needs a way to know whether a release understands plugin entries, so the release now ships a small capabilities file > - The benefit: new instances get the plugin environment as their default, and existing instances and admin choices are never changed ## Linked Issues or Issue Description Refs #19632 (the locked fill-if-unset rule this builds on), #19508 (plugin entries in the managed config) **Subsystem affected** Server environments: the managed config parser and the managed environment bootstrap. **Problem or motivation** The managed config can declare plugin environments, but it can't make one the instance default on a new instance. An external platform also has no way to tell whether a given release supports plugin entries before it sends one. **Proposed solution** - Add `instanceDefault: true` to at most one plugin entry. Right after that entry's boot ensure succeeds, it fills the instance default through `fillUnsetInstanceDefault`. The managed sandbox fill runs after it. - On a new instance the plugin environment wins. On an existing instance the default is already set, so neither fill changes it. - The fill declines while an active agent that follows the default hasn't been prepared by that plugin. This is the rule the instance-settings route enforces. When the plugin fill declines, the managed sandbox fills as usual. - Ship `server/src/services/managed-config-capabilities.json`, which lists `pluginEnvironments`. **Alternatives considered** - A recorded one-time decision with a retry worker. It needed pending states and extra checks for many writer and trigger combinations. Filling only an unset default, under the existing lock, needs none of that. - Rejecting the fill whenever agents exist. That is stricter than the admin route and blocks harmless cases, such as agents that are paused or already prepared. **Roadmap alignment** This extends existing managed-environment behaviour. It adds no UI. ## What Changed - `server/src/services/managed-config.ts`: - `ManagedPluginEnvironmentSpec.instanceDefault` (boolean, default `false`). - The parser refuses `instanceDefault` on more than one plugin entry, on the sandbox entry (even `false`), or as a non-boolean. - `MANAGED_CONFIG_CAPABILITIES` and `MANAGED_CONFIG_CAPABILITIES_SCHEMA_VERSION` constants. A plugin entry is refused unless `pluginEnvironments` is listed. - `server/src/services/managed-config-capabilities.json`: `{ "schemaVersion": 1, "capabilities": ["pluginEnvironments"] }`. A test pins it to the constants and to this literal contract. - `server/src/services/managed-environments.ts`: - New `fillPluginInstanceDefault`. It runs right after a successful boot ensure of the flagged entry and logs a fill or a decline. Recovery after boot never fills. - The managed sandbox fill now runs once, after the plugin entries. - `server/src/services/managed-instance-default.ts`: `fillUnsetInstanceDefault` runs `agentsNeedSetupForEnvironmentChange` inside its locked transaction and returns `declined: "agents_need_setup"`. A sandbox target never needs agent setup, so the managed sandbox fill behaves as before. - `server/src/services/plugin-environment-driver.ts`: the readiness query is factored into `agentsNeedSetupForEnvironmentChange`. `assertAgentsReadyForEnvironmentChange` wraps it and still throws the same conflict. - `doc/DEVELOPING.md`: documents `instanceDefault`, the fill order, the readiness decline, and the capabilities file. ## Verification - `cd server && npx vitest run src/__tests__/managed-plugin-environments.test.ts` (embedded Postgres), under `instanceDefault (fill only an unset default)`: - A new instance gets the plugin row as its default. - An existing sandbox default is kept, and so is an admin's choice. - If the plugin was down at the first boot and the sandbox filled, a later recovery and boot keep the sandbox default. - An admin-cleared or local default is declined while a ready or verifying agent follows the default without the plugin's setup. The managed sandbox then fills it. - It fills when the agents that follow the default are paused or already prepared, or are pinned to another environment. - Without the flag, nothing happens. - A new agent that follows the default gets the plugin in `requiredPluginIds`. - `npx vitest run src/__tests__/managed-instance-default.test.ts` (embedded Postgres): the earlier fill, lock and overlap cases pass. A sandbox fill also succeeds while active agents follow the default. - `npx vitest run src/services/managed-environments-plugin-entries.test.ts`: - The fill order is plugin then sandbox. - When the plugin is down there is no fill, and recovery never fills. - A failed ensure gives only the sandbox fill, and a failed fill keeps the ensure counted. - When the plugin fill declines, the sandbox still fills. - `npx vitest run src/__tests__/managed-config.test.ts src/services/managed-config-capabilities.test.ts`: parser refusals and the capabilities contract. - `npx vitest run src/services/managed-environments.test.ts src/__tests__/instance-settings-routes.test.ts src/__tests__/environment-routes.test.ts`: pass. The route still throws the same readiness conflict. - `pnpm --filter @paperclipai/server typecheck` passes. - I did not run the full `pnpm test:run`, `pnpm -r typecheck` or `pnpm build`. CI covers them. ## Risks - An instance whose default is unset at the first boot after this release gets the plugin environment, if its agents are ready for it. That covers a new instance, and an instance whose admin cleared the default or chose the local environment. This is the intended behaviour. - The readiness check doesn't lock agent rows. If an agent becomes ready during the boot fill, it can miss the check, which is the same race the admin route has. Agents created after the fill are prepared for the plugin, because setup reads the current default. - If the plugin is down at a new instance's first boot, the instance keeps the managed sandbox default from then on. - No schema change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (file edits, shell, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky LeachandClaude Opus 5.5
|
913102870d |
fix(environments): fill the managed instance default once, under a lock (#19632)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - On a managed instance under `enableManagedSandboxOnly`, boot points the instance default at the managed sandbox when nothing deliberate is set > - That write read the default and wrote it later without a lock. Two overlapping boots, or a boot and an admin write, could overwrite a value the other had just committed > - A revert path also reset that stamped default whenever the mode was turned off, so a platform-filled default could change again later > - A planned plugin-environment default should fill only an unset default and never change a set one, which needs every instance's default to stay set > - This pull request makes the fill one locked transaction and removes the revert, so a filled default is an ordinary set value > - The benefit is a single, race-free rule: boot fills an unset default once, and nothing changes a set default afterwards ## Linked Issues or Issue Description Refs #19551 (the plugin-environment default that builds on this rule) **Subsystem affected** Server environments: the managed environment bootstrap and the instance settings route. **Problem or motivation** `applyManagedEnvironments` read the instance default, decided, and wrote it in separate steps with no lock, so a concurrent boot or admin write could be overwritten. When `enableManagedSandboxOnly` was not declared, a revert reset a default the platform had filled. That needed a `managedDefaultStamped` marker, plus a marker clear on every admin default write. A later boot could therefore change a default that had already been set. **Proposed solution** Fill the default in one transaction that locks the instance default row and rechecks it. Fill only an unset, local or deleted default. Remove the revert and its marker, so a filled default behaves like any other set value. **Alternatives considered** Keeping the revert and adding locks to it. A revert on a mode change would still move an existing instance's default later, which a fill-only plugin default cannot tolerate. **Roadmap alignment** This narrows existing managed-environment behaviour. It adds no product surface. ## What Changed - New `server/src/services/managed-instance-default.ts`: `fillUnsetInstanceDefault(db, environmentId)`. In one transaction it creates the settings row if needed, locks the instance default row (`FOR UPDATE`), and rechecks the current value. It writes only when the default is unset, names the local environment, or names a deleted environment. Any other set default is kept. - `server/src/services/managed-environments.ts`: - Under `enableManagedSandboxOnly`, the managed sandbox fill now uses that helper, through a new `fillInstanceDefault` test seam. - The mode-off revert and the `managedDefaultStamped` marker are removed. - The unused `instanceSettings` seam option is removed, along with the `getById`, `update` and `findManagedSandboxEnvironment` methods of the environments seam. - `server/src/routes/instance-settings.ts`: an explicit default write is a plain `update` again. The marker clear and its transaction are gone. - `doc/DEVELOPING.md`: the "Managed environments" section states the fill rule. Existing rows may still carry `managedDefaultStamped` in their metadata. Nothing reads it any more, and it is not part of the managed stock hash. ## Verification - `cd server && npx vitest run src/__tests__/managed-instance-default.test.ts` (embedded Postgres): - The fill writes an unset default, including on a fresh instance with no settings row. It also writes a default that names the local environment, or one whose environment was deleted. - It never changes a default set to an ssh, plugin or other sandbox environment, and leaves a default that already names the target alone. - Controlled overlap: an uncommitted admin default write makes the fill wait, and the fill then keeps the admin's value. This test fails without the row lock. - Two overlapping fills: exactly one writes. - Real boot pass: the default is filled under managed-sandbox-only and stays the same after a boot with the mode off. A set default stays on every boot. - `npx vitest run src/services/managed-environments.test.ts src/services/managed-environments-plugin-entries.test.ts`: the fill runs only under managed-sandbox-only, with the managed sandbox row. It never runs for a plugin environment. Nothing is written when the mode is off. A failed fill does not fail the ensure. - `npx vitest run src/__tests__/instance-settings-routes.test.ts`: an explicit default write is a plain update that touches no environment row. - `npx vitest run src/__tests__/managed-plugin-environments.test.ts src/__tests__/environment-routes.test.ts src/__tests__/server-startup-feedback-export.test.ts src/__tests__/instance-settings-service.test.ts`: pass. - `pnpm --filter @paperclipai/server typecheck` passes. - I did not run the full `pnpm test:run`, `pnpm -r typecheck` or `pnpm build`. CI covers them. ## Risks - Behaviour change: when `enableManagedSandboxOnly` is turned off for an instance, a default the platform filled with the managed sandbox now stays. Local execution is still allowed then, but it is no longer the default unless an admin picks it. - No schema change. Stale `managedDefaultStamped` keys stay in some row metadata and are ignored. - The fill holds the instance default row lock for one short transaction at boot. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (file edits, shell, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky LeachandClaude Opus 5.5
|
1206373a95 |
fix(environments): close Apex findings on managed plugin environments (#19526)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A managed instance gets its instance-level environments from the `PAPERCLIP_MANAGED_CONFIG` document, and #19508 added plugin environment entries to it > - Greptile's Apex review of #19508 arrived after it merged, with three findings > - A credential inside an array nested in an array passed the secret check and the response redaction > - Recovery of an archived environment listened on one worker handle, which a restart replaces, and its attempts were not tracked through idle checks and shutdown > - This pull request closes all three findings > - The benefit is that credentials never pass the check, an environment comes back after any worker restart, and shutdown never cuts off a recovery write ## Linked Issues or Issue Description Refs #19508 (the Apex findings this pull request fixes) **Subsystem affected** Server environments: the managed-config parser, the environment routes' response floor, the managed environment bootstrap, and the plugin worker manager. **Problem or motivation** Greptile's Apex review of #19508 found three problems. (1) `findSecretLikeConfigKey` and `redactSecretLikeConfigKeys` skip array elements that are arrays, so `{"groups":[[{"apiToken":"..."}]]}` passes the boot check and is echoed to restricted readers. (2) Recovery subscribes to `ready` on one worker handle. A restart such as a settings save replaces that handle, so the new worker's `ready` never reaches the listener, and the environment stays archived. (3) A recovery attempt is fire-and-forget. The worker manager's work token ends when the worker replies, but the database write continues, so an idle check or the database shutdown can overlap it. **Proposed solution** Walk arrays at any depth in both functions. Add a manager-wide `onWorkerReady` subscription and use it for recovery. Run every recovery attempt through a controller that tracks it as idle work and that shutdown stops and awaits before the database closes. **Alternatives considered** Re-subscribing on each new handle after a restart. That needs a hook in every restart path. The manager-wide subscription covers all of them in one place. **Roadmap alignment** This only fixes code that #19508 added or touched. It adds no product surface. ## What Changed - `server/src/services/managed-config.ts`: `findSecretLikeConfigKey` walks arrays at any depth (new `findSecretLikeConfigKeyInValue`). This covers sandbox `config` and plugin `driverConfig`. - `server/src/routes/environments.ts`: `redactSecretLikeConfigKeys` walks arrays at any depth (new `redactSecretLikeConfigValue`). - `server/src/services/plugin-worker-manager.ts`: new optional `onWorkerReady(listener)` on `PluginWorkerManager`. It fires on every worker `ready`, including a replacement handle from a restart. A listener error is logged and does not affect the worker. - `server/src/services/managed-environments.ts`: - Sandbox and plugin recovery subscribe through `subscribeWorkerReady`. It prefers `onWorkerReady` and falls back to the handle when the manager has no such method. - New `createManagedEnvironmentRecovery()` controller. Each attempt runs inside `trackIdleWork` for its whole duration. `stop()` removes every `ready` subscription, refuses new attempts, and awaits unfinished ones. - `server/src/index.ts`: creates the controller, passes it to `applyManagedEnvironments`, and awaits `stop()` during shutdown, after the delivery workers stop and before the application teardown and database close. ## Verification - `cd server && npx vitest run src/__tests__/managed-config.test.ts`: a secret-like key in an array nested in an array refuses boot, for plugin `driverConfig` and sandbox `config`. - `npx vitest run src/__tests__/environment-routes.test.ts`: a nested-array credential is removed from the response, and other nested values stay. - `npx vitest run src/services/managed-environments-plugin-entries.test.ts`: - With `onWorkerReady`, recovery never consults the handle. It ignores another plugin's `ready`, re-ensures on this plugin's `ready`, and then unsubscribes. - An attempt counts in `idleWorkSnapshot()` until its ensure settles. `stop()` unsubscribes, waits for the unfinished attempt, and later `ready` events start nothing. - `npx vitest run src/services/managed-environments.test.ts src/__tests__/managed-plugin-environments.test.ts`: pass. - `pnpm --filter @paperclipai/server typecheck` passes. - I did not run the full `pnpm test:run`, `pnpm -r typecheck` or `pnpm build`. CI covers them. ## Risks - The redaction now also walks nested arrays in sandbox config responses. Only credential-shaped keys are removed; other values are unchanged. - The boot check now refuses a document that hides a secret-like key in a nested array. A valid document has no such key. - `onWorkerReady` is optional, so other `PluginWorkerManager` implementations and test fakes keep working. - Shutdown now waits for an unfinished recovery attempt. An attempt is one worker call plus one short transaction. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (file edits, shell, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky LeachandClaude Opus 5.5
|
98ebeefbc9 |
feat(environments): let managed config declare plugin environments (#19508)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - A managed instance gets its instance-level environments from the `PAPERCLIP_MANAGED_CONFIG` document > - #19490 added the plugin environment rows, their marker index and write floor, but nothing creates them yet > - The managed config still accepts only one sandbox entry, so a managed instance cannot get a plugin-provided environment without manual setup > - This pull request lets the managed config declare plugin entries beside the sandbox entry, and provisions one locked row for each at boot > - The benefit is that a managed instance gets its plugin environments with no manual setup, and the rows recover by themselves when their plugin restarts ## Linked Issues or Issue Description Refs #19490 (the plugin environment rows this pull request provisions) **Subsystem affected** Server environments: the managed-config parser, the managed environment bootstrap, and the environment routes' write floor. **Problem or motivation** A managed instance cannot get a plugin environment from its managed config. The `environments` section accepts one entry, and that entry always provisions the managed sandbox. Company settings cannot create a plugin environment either. So each managed instance that needs one requires manual setup. **Proposed solution** Let an `environments` entry declare `driver: "plugin"`. Boot provisions one locked row for each plugin entry through the row layer from #19490. A later pull request adds a one-time instance default for one plugin entry, and advertises the new managed-config capability. **Alternatives considered** Manual setup on each instance, and a new control-plane endpoint. Both need a step outside the deploy. The managed config already delivers the sandbox entry. **Roadmap alignment** This extends the existing managed-config `environments` section, which already provisions the managed sandbox. It adds no new product surface. ## What Changed - `server/src/services/managed-config.ts`: an `environments` entry may set `driver` to `"sandbox"` or `"plugin"`. An entry without `driver` is still the sandbox entry, so today's documents parse unchanged. - A plugin entry has `name`, optional `description`, `pluginKey` (the plugin's manifest id), `driverKey` and optional `driverConfig`. - At most one sandbox entry. Names are unique across the section, and `pluginKey`–`driverKey` pairs are unique. Secret-like keys in `driverConfig` refuse startup, as for sandbox `config`. An unknown key or driver still refuses startup, and that includes `instanceDefault` until the follow-up pull request. - Plugin entries parse into a new `pluginEnvironments` list. The `environments` list keeps only the sandbox entry, so its consumers do not change. - `MANAGED_ENVIRONMENT_DRIVERS` lists the accepted drivers. - `server/src/services/managed-environments.ts` (`applyManagedEnvironments`): - Boot refuses to start unless an auto-installed catalog entry installs each plugin entry's `pluginKey`. `plugins.autoInstall` holds catalog keys, not manifest ids, so the parser cannot check this. Boot uses the manifest ids `createApp` already resolves from the bundled and distribution catalogs (`app.locals.managedBundledPluginKeys`, passed in by `server/src/index.ts`). Without that list it resolves the catalog itself. - Plugin entries run after the sandbox entry and reuse its readiness handling. Boot awaits `pluginsReady`, then requires the plugin `ready`, declaring the entry's driver, and running a worker. Otherwise boot archives the row and re-ensures it on the worker's `ready`. - Recovery re-ensures on every worker `ready` until one ensure succeeds or the plugin rejects the config (HTTP 422). - If the ensure fails because the worker died, boot archives the row and waits for `ready`. If the worker is still running (for example the call timed out), boot keeps the row as it is, retries once at once and on the next `ready`, and otherwise at the next boot. No timer is added. - Removing an entry never archives its row. - A plugin-only document does not conflict with `PAPERCLIP_EXECUTION_MODE`, because plugin rows do not use the sandbox slot. - Plugin entries never change the instance default. The sandbox stamp still fills only an unset or local default, so it never replaces a plugin default. - `server/src/routes/environments.ts`: `isPlatformSlotEnvironment` treats a marked plugin row as a live slot while the document declares its entry. So the marker-clear escape hatch is refused then, and allowed again after the entry leaves the document. - `doc/DEVELOPING.md`: new "Managed environments" section. ## Verification - `cd server && npx vitest run src/__tests__/managed-config.test.ts`: old documents parse unchanged. Sandbox plus plugin parses. Each of these refuses startup: two sandbox entries, `driver: null`, duplicate names, a duplicate pair, a secret-like key, an unknown key (including `instanceDefault`), an unknown driver, a malformed `driverKey` or `driverConfig`. - `npx vitest run src/services/managed-environments-plugin-entries.test.ts` (the sandbox-entry tests stay in `managed-environments.test.ts`; shared fixtures are in `managed-environments.test-support.ts`): - A plugin entry is ensured with the validator's worker manager, after the sandbox entry. - Boot refuses a plugin entry whose `pluginKey` no auto-installed catalog entry installs, both from the passed list and from its own catalog lookup. - A plugin that is down archives the row, and worker recovery re-ensures it. A missing plugin, a plugin without the driver, or a worker manager without `call` counts as unavailable. - Recovery stays armed after a failed attempt and stops after one success or a permanent config rejection (422). A `ready` during an attempt queues one more. - A worker that dies during the boot ensure archives the row and arms recovery. A failure while the worker keeps running leaves the row as it is and retries. - Removing an entry does not archive. Plugin entries and the sandbox stamp never change a plugin default. - An ensure failure is counted, not thrown. - `npx vitest run src/__tests__/managed-plugin-environments.test.ts` (embedded Postgres, real `applyManagedEnvironments`): the first boot creates the row and the second changes nothing. A plugin that is down archives the row, and recovery brings it back. Removing the entry does not archive the row. The instance default stays unset. The row tests from #19490 still pass. - `npx vitest run src/__tests__/environment-routes.test.ts`: the marker clear is refused while the document declares the entry, and allowed after it leaves. - `npx vitest run src/__tests__/instance-settings-service.test.ts src/__tests__/instance-settings-cloud-defaults.test.ts`: pass with the new `pluginEnvironments` field. - `pnpm --filter @paperclipai/server typecheck` passes. - I did not run the full `pnpm test:run`, `pnpm -r typecheck` or `pnpm build`. CI covers them. ## Risks - No schema change. - A managed-config document with a plugin entry now provisions a row at boot. A document without one behaves as before. - Adoption renames an unmarked row for the same plugin and driver to the declared name and resets its driver config to the declared one. The row keeps its id, agents and other metadata, and it is then locked like the managed sandbox. - Boot now refuses to start when a plugin entry names a plugin that no auto-installed catalog entry provides. This is the same fail-closed posture as an unknown sandbox provider. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code with tool use (file edits, shell, test runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nicky LeachandClaude Sonnet 5
|
9effe51b63 |
fix(server): let the cloud-harness sandbox environment self-heal past operator-drift protection (#13177)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every cloud-harness-managed stack gets one platform-owned "Paperclip Computer" sandbox environment, reconciled from `PAPERCLIP_MANAGED_CONFIG` on boot > - That reconciler deliberately refuses to overwrite a row it classifies as operator-modified, to protect a self-hosted operator's hand-edited environment (#10979) > - But for the cloud-harness-managed row specifically, no operator has any path to hand-edit it at all — so a hash mismatch there can only be drift between two platform-driven reconciliation passes, never a real customization > - Roughly 10 staging stacks got stuck on a broken sandbox image because of exactly this: a `sandbox_image` campaign correctly delivered a fixed snapshot, but the reconciler classified the row as operator-modified and silently skipped applying it > - This pull request adds an explicit `platformFullyManaged` flag so the cloud-harness caller can assert that guarantee and let its own drift self-heal, without weakening the protection for every other caller (self-hosted kubernetes-execution-mode, tests, admin routes) where an operator genuinely can edit the row > - The benefit is that a sandbox-image rollout can no longer get silently stuck fleet-wide, while self-hosted operator customization keeps exactly the protection #10979 built ## Linked Issues or Issue Description No public issue exists for this internal-instance-discovered bug; opening directly per CONTRIBUTING.md path B, following the bug report template fields. **What happened?** After a `sandbox_image` campaign delivered a fixed Daytona snapshot fleet-wide, ~10 of 79 active staging stacks kept booting agents against the old, broken snapshot. Their `PAPERCLIP_MANAGED_CONFIG` env var and the reconciler's own bookkeeping (`built_in_managed_resources`) both correctly showed the new snapshot — but `environments.config.snapshot`, the field the runtime actually reads to acquire a sandbox lease, was never updated on those rows. **Expected behavior** A `sandbox_image` campaign (or any `PAPERCLIP_MANAGED_CONFIG` delivery) to the cloud-harness-managed sandbox environment should always converge that row's `config` to the newly-desired value, since no operator can have a competing edit to protect. **Steps to reproduce** 1. Boot a cloud-harness-managed stack; let the reconciler create the managed sandbox row and record its stock hash in `built_in_managed_resources`. 2. Somehow cause the row's live content hash to no longer match the recorded binding hash without an operator ever touching it (in the field, this happened via drift between two platform-driven reconciliation passes carried out across a catalog-version bump — the exact trigger wasn't fully pinned down, but is irrelevant to the fix). 3. Deliver a new `PAPERCLIP_MANAGED_CONFIG` (e.g. via a `sandbox_image` campaign). 4. Observe `ensureManagedSandboxEnvironment` classify the row `operator_modified` and skip writing `config`, even though `updateAvailable: true` is reported and the binding itself already advanced to the new stock hash. **Paperclip version or commit** `master` as of this PR. **Deployment mode** Cloud-managed stacks with `enableManagedSandboxOnly` declared (any Paperclip Cloud–provisioned staging or production stack). Related PR for context (not a duplicate — this is additive to it, not a revert): #10979, which introduced the `operator_modified` classification this PR narrowly opts the cloud-harness path out of. ## What Changed - `server/src/services/environments.ts`: added `platformFullyManaged?: boolean` to `ManagedSandboxEnvironmentInput`. When set, a plain content-hash mismatch against a real prior binding (i.e. `operator_modified` that isn't an archive-reaffirmation) is reclassified as `stock_update_available` before the skip-vs-apply branch, so it flows through the normal update path instead of being frozen. - `server/src/services/managed-environments.ts`: pass `platformFullyManaged: true` from both `ensureManagedSandboxEnvironment` call sites — the main boot ensure and the provider-recovery reactivation path. These are the *only* two callers driven by `PAPERCLIP_MANAGED_CONFIG`; `ensureKubernetesEnvironment` (self-hosted `kubernetes-execution-mode` bootstrap) and every other caller are untouched and keep the original protective default. - `server/src/services/managed-environments.test.ts`: updated the two `toHaveBeenCalledWith` assertions that now include the flag. - `server/src/__tests__/environment-service.test.ts`: two new tests — one confirming the bypass applies drift under `platformFullyManaged`, one confirming archive-reaffirmation still wins even under the flag. Archive-reaffirmation is deliberately *not* bypassed even under `platformFullyManaged`: a `sandbox_image` update must never resurrect a row something else deliberately kept archived after Paperclip's own provider-unavailability archival. That's a distinct, still-real signal, orthogonal to config drift. ## Verification - `vitest run` on `managed-environments.test.ts` and `managed-resource-drift.test.ts`: 26/26 pass, including the two updated assertions. - `environment-service.test.ts` — the file both new tests live in, and the file holding the two pre-existing tests this change must not regress ("classifies operator drift, preserves the row, and exposes the pending stock update" and "preserves an existing unmanaged sandbox row holding the desired name") — requires a real embedded-Postgres instance (`describeEmbeddedPostgres`) not available in the sandbox this was developed in; `getEmbeddedPostgresTestSupport()` reports unsupported there, so the whole file is skipped locally. I traced the reconciliation logic by hand against all four relevant tests (the two new ones plus the two pre-existing ones) line by line to confirm the expected outcomes, but **CI running this suite for real is the actual gate here**, not this description — please don't merge on a green run of everything else alone if this suite doesn't show as executed. - `tsc --noEmit`: zero errors in any of the four touched files. The pre-existing ~229 errors elsewhere in `server` are unrelated missing-module issues from packages needing a build step first, confirmed unchanged by this diff. - Manually reproduced the underlying bug against real staging data (a `paperclip-cloud`-managed stack whose `environments` row was stuck exactly this way) before writing the fix, and confirmed via direct SQL inspection that the recorded `built_in_managed_resources` baseline already held the correct desired snapshot on every affected stack — i.e. the reconciler already *knew* the right answer, it was just refusing to apply it. That data point is what ruled out "the campaign didn't actually deliver the update" as the cause. ## Risks - Scope is intentionally narrow: only the two `PAPERCLIP_MANAGED_CONFIG`-driven call sites pass the new flag; every other caller of `ensureManagedSandboxEnvironment`/`ensureKubernetesEnvironment` is byte-for-byte unchanged. The two pre-existing regression tests that specifically cover self-hosted operator-edit protection don't pass this flag and are unmodified. - The main residual risk is the unresolved root cause of *why* the hash drifted in the first place (a race between two close-together reconciliation passes, or a catalog-version-dependent change to what gets hashed, most likely) — this PR makes that drift self-healing rather than fixing whatever produces it. If the drift is being caused by a genuine concurrency bug (rather than an expected, occasional side effect of a stock-field/catalog-version change), that bug still exists and could recur; it just no longer gets stuck when it does. - Low risk of behavior change for real self-hosted deployments: none of them can reach the new code path, since only the two now-flagged call sites exist inside `managed-environments.ts`, itself gated to `PAPERCLIP_MANAGED_CONFIG` (which self-hosted `kubernetes-execution-mode` explicitly refuses to run alongside — see the existing mutual-exclusivity check this PR does not touch). ## Model Used Claude Sonnet 5 (`claude-sonnet-5`), via Claude Code, with tool use (file edits, shell/git, `gh` CLI, direct Postgres inspection of live staging data via `psql`/`pg`, Railway SSH for on-host diagnosis). No extended-thinking mode. Standard Claude Code context window. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass — see Verification: the file holding the four load-bearing tests can't run in this sandbox (no embedded-Postgres support); traced by hand instead, CI is the real gate - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes — none applicable beyond the inline doc comments this PR adds - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green — pending CI run on this PR - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — pending review - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Devin Foley
|
0044fa8904 |
Let tenants edit env vars on managed sandbox environments; add managed-sandbox-only mode (#11200)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environments give each agent run an execution target: the local host, SSH, or a sandbox provider > - A managed deployment can provision one platform-managed sandbox environment through the `PAPERCLIP_MANAGED_CONFIG` `environments` section > - That row is fully locked today. A tenant cannot add environment variables for their agents. There is also no way to hide local execution — run selection falls back to the local row > - A platform that manages the sandbox for its tenants needs both: the tenant adds env vars (and nothing else), and local execution is neither visible nor reachable > - This pull request opens exactly one tenant edit (env vars) on the managed sandbox row, and adds an `enableManagedSandboxOnly` mode that hides local and makes run selection fail closed > - The benefit is a complete managed-sandbox experience with no change for self-hosted instances ## Linked Issues or Issue Description **Subsystem affected** Environments (managed sandbox provisioning, environment routes, run environment selection) and the environments UI. **Problem or motivation** Platform-provisioned sandbox environments (`metadata.managedByPaperclip`) reject every write on cloud-managed instances. Agents often need environment variables inside their sandbox. The tenant has no way to set them on the managed row. Separately, an operator cannot remove local execution: the environment list always shows the local row, and run selection falls back to it when no default is set. **Proposed solution** Allow an envVars-only PATCH on the managed sandbox row, and echo those env vars back for editing. Add a managed-tier feature (`enableManagedSandboxOnly`) that hides the local environment from all read surfaces and redirects local-landing run selection to the managed sandbox environment, failing closed when it is unavailable. **Alternatives considered** UI-only hiding of the local row. This was rejected: it does not stop a run from resolving to local, so it is presentation without enforcement. Full unlock of the managed row was also rejected: name, driver, and config stay platform-owned so boot reconciliation cannot fight tenant edits. ## What Changed - `server/src/routes/environments.ts`: the platform-provisioned write floor admits an envVars-only PATCH on the generalized managed sandbox row (sandbox driver, `managedByPaperclip`, not legacy kubernetes-marker rows). Name, driver, config, status, metadata, and DELETE stay rejected. The read floor stops blanking env vars on that row; credential-shaped config keys stay redacted for every actor. Legacy kubernetes-marker rows keep the full floor. - Same file: under `enableManagedSandboxOnly`, the environments list and the by-id read omit the local row for every actor, including instance admins. - `server/src/services/execution-workspace-policy.ts`: `resolveExecutionWorkspaceEnvironmentId` gains the managed-sandbox-only inputs. A selection that lands on the local environment is redirected to the managed sandbox environment. With no active managed row it throws `ManagedSandboxUnavailableError` — never local. Non-local selections (ssh, user-created sandboxes) are untouched. - `server/src/services/heartbeat.ts`: the run path reads the flag, looks up the managed row (`findManagedSandboxEnvironment`, new read-only finder in `environments.ts`), and passes both to the resolver. Mirrors the forced-kubernetes precedent, which keeps precedence when both regimes are on. - `server/src/services/managed-environments.ts`: after a successful reconcile, the instance default environment moves to the managed sandbox row when the current default is unset, local, or dangling. A tenant-chosen custom environment is never overridden. - `packages/shared`: new `enableManagedSandboxOnly` key (schema default false, catalog tier `managed`, cloudDefault false, selfHostedDefault false) and the matching interface field. - UI: managed rows show a "Managed by Paperclip" lock badge; editing one opens a dedicated env-vars-only editor that sends the one PATCH shape the server admits (the old full form failed with a 403 on save). New `ui/src/lib/managed-sandbox-environment.ts` mirrors the local filter for cached lists (applied in the project picker; the agent picker already excluded local). The experimental settings page gains the toggle at its alphabetical card position. `environmentsApi.update` now declares the `envVars` field it already sent. - Tests: environment route floor coverage (envVars-only accepted, mixed bodies rejected, legacy rows still blanked and locked, local hidden and 404 under the flag, self-hosted unchanged), an embedded-postgres service test pinning that boot reconciliation never touches tenant env vars, resolver redirect/fail-closed cases, managed-environments default-stamping cases, and UI lib/settings tests. ## Verification - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/environment-routes.test.ts src/__tests__/environment-service.test.ts src/__tests__/execution-workspace-policy.test.ts src/services/managed-environments.test.ts` — all pass. - `pnpm --filter @paperclipai/shared exec vitest run` — 425 pass (catalog/schema default parity is pinned by an existing test). - `pnpm --filter @paperclipai/ui exec tsc --noEmit` and the affected UI suites (CompanyEnvironments, InstanceExperimentalSettings incl. card-order test, new lib test) — all pass. - Full workspace `pnpm test`: 3,414 passed. 17 files report failures on this machine; the identical 17 fail on a clean `origin/master` worktree in the same environment (git-worktree/skills/embedded-postgres environment dependencies and plugin-SDK zero-test collections). One additional file (`issue-monitor-scheduler.test.ts`) failed one timing-sensitive test in one of two full-suite runs and passes 7/7 in isolation on this branch — a flake in a domain this diff does not touch. The branch introduces no new failures. - Self-hosted zero-delta: every new behavior is gated on the cloud-managed instance check or the new flag, which defaults to false in schema and catalog; pinned by the "does not floor platform-marked rows on self-hosted instances" and flag-off tests. ## Risks - Behavior is opt-in twice over: the write-floor exception applies only to rows the managed-config provisioner stamps, and the hiding/forcing applies only when `enableManagedSandboxOnly` is on (default false everywhere). Self-hosted instances see no change. - The env-vars echo is scoped to the generalized managed sandbox row; legacy kubernetes-marker rows keep the blanket floor because pre-generalization builds may have written platform values there. - Fail-closed run selection means a managed instance with the flag on and an archived managed row (provider plugin down) refuses runs with a precise error instead of running locally. That is the intended posture. ## Model Used Claude Fable 5 (`claude-fable-5`), extended thinking, agentic tool use via Claude Code CLI. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
DottaandPaperclip
|
f6c6452b25 |
fix(server): preserve managed environment drift on boot (#10979)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip creates a managed sandbox environment for each company during boot > - Operators can change environment fields after Paperclip creates the environment > - The boot reconciler replaced those changes without checking for drift > - This pull request adds stock hashes and transactional drift reconciliation > - The benefit is that Paperclip can update untouched stock fields without losing operator work ## Linked Issues or Issue Description **What happened?** The managed sandbox boot reconciler rewrote the stock description, configuration, metadata, and status on every start. It did not detect operator changes first. A restart could therefore remove an operator's changes. **Expected behavior** Paperclip must preserve operator changes by default. It must update an untouched stock environment when Paperclip ships new stock values. It must perform each row update and stock-hash update atomically. **Steps to reproduce** 1. Start Paperclip and let it create the managed sandbox environment. 2. Change one Paperclip-owned stock field on that environment. 3. Restart Paperclip. 4. Observe that the previous reconciler replaced the change. **Paperclip version or commit** This bug reproduces on `master` before this change. **Deployment mode** Local development and self-hosted server boot are affected. ## What Changed - Add a shared deterministic stock-hash and drift classifier for built-in resources. - Track the managed sandbox stock hash with the company-scoped built-in resource binding. - Reconcile the environment and its stock metadata in one transaction with a row lock. - Preserve operator-modified and unmanaged rows and report their skipped update state. - Use archive ownership tokens so provider recovery reactivates only Paperclip-archived rows and preserves later operator archive decisions. - Keep operator-owned environment variables and unrelated metadata out of the stock fingerprint. - Add activity records for managed environment creation, updates, skipped drift, tracking initialization, and archive changes. - Add regression tests for current stock, available stock updates, operator drift, unmanaged rows, archive and reactivation, user-owned fields, and concurrent reconciliation. ## Verification - `pnpm -r typecheck` - `pnpm build` - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY pnpm test:run -- --mode serialized` (128 suites passed) - Repository general server, UI, CLI, shared, skills catalog, database, adapter, plugin SDK, and plugin creator projects passed. Two embedded-Postgres tests exceeded the host's five-second default under the aggregate run and passed in the complete database project with `--testTimeout=20000`. One timing-sensitive sandbox stream test passed on its focused retry. - Focused managed-environment unit and integration coverage passed: 49 tests across the drift classifier, boot report, and environment service suites. ## Risks - The main risk is an incorrect ownership boundary in the stock fingerprint. The fingerprint includes only Paperclip-owned stock fields. Tests confirm that environment variables and unrelated metadata survive reconciliation. - Concurrent reconciliation could otherwise overwrite a late operator edit. The implementation locks the environment row and updates the row and hash binding in one transaction. A concurrency test covers this path. - There is no schema migration. Existing managed rows initialize tracking without replacing their current values. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex. The exact serving snapshot and context-window size were not exposed. The model used reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
273315a4d0 |
feat(server): provision managed sandbox environments from the managed config (#10324)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Hosted/managed deployments configure instances entirely from the control plane: `PAPERCLIP_MANAGED_CONFIG` already delivers feature flags and `plugins.autoInstall` (bundled sandbox provider plugins), parsed fail-closed at boot > - A plugin alone is not usable for execution: runs need an instance-level `driver: "sandbox"` environment row pointing at the provider, and today only Kubernetes has a boot path for that (`PAPERCLIP_EXECUTION_MODE` → `ensureKubernetesEnvironment`); every other provider requires a manual product-API call the control plane cannot make on a managed instance > - Adding one `ensureXxxEnvironment` per provider would multiply near-identical boot hooks and env-var surfaces > - This pull request generalizes the existing Kubernetes machinery: the managed-config document gains an optional `environments` section that declares a sandbox environment for any bundled provider, ensured idempotently at boot by a provider-agnostic service function (the Kubernetes hook becomes a thin wrapper over it) > - The benefit is that a managed fleet can provision any sandbox provider (Daytona, Modal, E2B, …) purely from configuration — no per-provider code, no manual API calls, no secrets in the document — while self-hosted behavior is untouched ## Linked Issues or Issue Description No public issue exists. Refs #10157 (the cloud image variant that bundles sandbox provider plugins — this PR is the configuration half that makes an installed provider usable). **Problem (feature-request shape):** on a managed instance the control plane can auto-install a bundled sandbox provider plugin via `PAPERCLIP_MANAGED_CONFIG.plugins.autoInstall`, but cannot create the environment row that makes the provider schedulable. The only boot-time environment provisioning is Kubernetes-specific (`PAPERCLIP_EXECUTION_MODE=kubernetes` + `PAPERCLIP_K8S_*`). A generic, config-driven path is needed so any bundled provider can be provisioned without per-plugin code or manual API calls. ## What Changed - `server/src/services/managed-config.ts`: optional `environments` top-level section — `[{ name, description?, provider, config? }]` — validated fail-closed: unknown keys, more than one entry (the DB permits exactly one Paperclip-managed sandbox row, `environments_managed_sandbox_idx`), a `provider` not present in `plugins.autoInstall`, `config.provider`, or secret-looking config keys at any depth (`api_key`/`token`/`secret`/`password`/`credential`) all refuse startup. Absent section ⇒ `environments: []`, so pre-section documents keep booting newer builds. - `server/src/services/environments.ts`: new provider-agnostic `ensureManagedSandboxEnvironment({ name, description?, provider, config?, extraMetadata? })` — idempotently owns the single managed sandbox row: refreshes name/description/config each call, adopts the slot across provider switches (dropping the stale `managedKubernetesSandbox` marker), adopts a same-name unmanaged sandbox row (stamping it managed) instead of colliding on `environments_name_idx` every boot, and falls back to keeping the current name if the desired name belongs to a different row. `ensureKubernetesEnvironment` is now a thin wrapper that pins `provider: "kubernetes"` and stamps the legacy marker. - `server/src/services/managed-environments.ts` (new): `applyManagedEnvironments` boot step — no-op for self-hosted/empty; throws (fail startup) when `PAPERCLIP_EXECUTION_MODE` is also set, since both would own the same managed sandbox row; otherwise ensures each declared environment fail-safe per entry (log + continue boot, matching bundled-plugin provisioning posture). - `server/src/index.ts`: runs the new boot step right after the execution-policy bootstrap, before the heartbeat resumes queued runs. - `server/src/services/index.ts`: exports `applyManagedEnvironments` and `ManagedEnvironmentSpec`. - Secrets stay out of the document by construction: provider credentials reach managed instances only as process env vars (each provider's documented fallback, e.g. `DAYTONA_API_KEY` for the Daytona plugin). ## Verification ```sh cd server pnpm exec tsc --noEmit -p tsconfig.json pnpm exec vitest run \ src/__tests__/managed-config.test.ts \ src/services/managed-environments.test.ts \ src/services/execution-policy-bootstrap.test.ts \ src/__tests__/environment-service.test.ts \ src/__tests__/environment-instance-routes.test.ts \ src/__tests__/environment-routes.test.ts \ src/__tests__/plugin-install-guard.test.ts \ src/__tests__/environment-execution-target.test.ts \ src/__tests__/instance-settings-managed-overlay.test.ts \ src/__tests__/bundled-plugins.test.ts ``` All pass locally (typecheck clean; environment-service suite runs against embedded Postgres and exercises the refactored Kubernetes wrapper plus the new generic ensure: create/refresh, provider switch, unmanaged-row adoption, name-conflict fallback). New tests cover the parser (12 cases incl. secret-key rejection at depth) and the boot step (no-op, mutual exclusion, pass-through, fail-safe). ## Risks - **Self-hosted: none intended.** Without `PAPERCLIP_MANAGED_CONFIG` nothing new executes; the `PAPERCLIP_EXECUTION_MODE=kubernetes` path is regression-covered by the existing bootstrap/service/route suites (all green). - **Behavioral shift in `ensureKubernetesEnvironment` (deliberate):** it now also refreshes `name`/`description` to their managed defaults each boot (desired-state semantics, same as config today) and adopts a `managedByPaperclip` sandbox row that lacks the Kubernetes marker — previously that state made the ensure throw every boot. - **New startup failure modes are all explicit misconfigurations** (malformed section, provider not auto-installed, secret in config, execution-mode conflict) and fail with precise errors; DB-side ensure failures never block boot (fail-safe per entry, logged). - No migrations; no API surface changes. ## Model Used Claude Fable 5 (Anthropic, model ID `claude-fable-5`) with extended thinking and tool use, driving the change end-to-end inside a Claude Code / agent-harness session (code, tests, and verification runs). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes (the managed-config module header is the contract doc) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |