Commit Graph
2 Commits
Author SHA1 Message Date
Nicky LeachandClaude Opus 5.5 98ebeefbc9 feat(environments): let managed config declare plugin environments (#19508)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A managed instance gets its instance-level environments from the
`PAPERCLIP_MANAGED_CONFIG` document
> - #19490 added the plugin environment rows, their marker index and
write floor, but nothing creates them yet
> - The managed config still accepts only one sandbox entry, so a
managed instance cannot get a plugin-provided environment without manual
setup
> - This pull request lets the managed config declare plugin entries
beside the sandbox entry, and provisions one locked row for each at boot
> - The benefit is that a managed instance gets its plugin environments
with no manual setup, and the rows recover by themselves when their
plugin restarts

## Linked Issues or Issue Description

Refs #19490 (the plugin environment rows this pull request provisions)

**Subsystem affected**
Server environments: the managed-config parser, the managed environment
bootstrap, and the environment routes' write floor.

**Problem or motivation**
A managed instance cannot get a plugin environment from its managed
config. The `environments` section accepts one entry, and that entry
always provisions the managed sandbox. Company settings cannot create a
plugin environment either. So each managed instance that needs one
requires manual setup.

**Proposed solution**
Let an `environments` entry declare `driver: "plugin"`. Boot provisions
one locked row for each plugin entry through the row layer from #19490.
A later pull request adds a one-time instance default for one plugin
entry, and advertises the new managed-config capability.

**Alternatives considered**
Manual setup on each instance, and a new control-plane endpoint. Both
need a step outside the deploy. The managed config already delivers the
sandbox entry.

**Roadmap alignment**
This extends the existing managed-config `environments` section, which
already provisions the managed sandbox. It adds no new product surface.

## What Changed

- `server/src/services/managed-config.ts`: an `environments` entry may
set `driver` to `"sandbox"` or `"plugin"`. An entry without `driver` is
still the sandbox entry, so today's documents parse unchanged.
- A plugin entry has `name`, optional `description`, `pluginKey` (the
plugin's manifest id), `driverKey` and optional `driverConfig`.
- At most one sandbox entry. Names are unique across the section, and
`pluginKey`–`driverKey` pairs are unique. Secret-like keys in
`driverConfig` refuse startup, as for sandbox `config`. An unknown key
or driver still refuses startup, and that includes `instanceDefault`
until the follow-up pull request.
- Plugin entries parse into a new `pluginEnvironments` list. The
`environments` list keeps only the sandbox entry, so its consumers do
not change.
  - `MANAGED_ENVIRONMENT_DRIVERS` lists the accepted drivers.
- `server/src/services/managed-environments.ts`
(`applyManagedEnvironments`):
- Boot refuses to start unless an auto-installed catalog entry installs
each plugin entry's `pluginKey`. `plugins.autoInstall` holds catalog
keys, not manifest ids, so the parser cannot check this. Boot uses the
manifest ids `createApp` already resolves from the bundled and
distribution catalogs (`app.locals.managedBundledPluginKeys`, passed in
by `server/src/index.ts`). Without that list it resolves the catalog
itself.
- Plugin entries run after the sandbox entry and reuse its readiness
handling. Boot awaits `pluginsReady`, then requires the plugin `ready`,
declaring the entry's driver, and running a worker. Otherwise boot
archives the row and re-ensures it on the worker's `ready`.
- Recovery re-ensures on every worker `ready` until one ensure succeeds
or the plugin rejects the config (HTTP 422).
- If the ensure fails because the worker died, boot archives the row and
waits for `ready`. If the worker is still running (for example the call
timed out), boot keeps the row as it is, retries once at once and on the
next `ready`, and otherwise at the next boot. No timer is added.
  - Removing an entry never archives its row.
- A plugin-only document does not conflict with
`PAPERCLIP_EXECUTION_MODE`, because plugin rows do not use the sandbox
slot.
- Plugin entries never change the instance default. The sandbox stamp
still fills only an unset or local default, so it never replaces a
plugin default.
- `server/src/routes/environments.ts`: `isPlatformSlotEnvironment`
treats a marked plugin row as a live slot while the document declares
its entry. So the marker-clear escape hatch is refused then, and allowed
again after the entry leaves the document.
- `doc/DEVELOPING.md`: new "Managed environments" section.

## Verification

- `cd server && npx vitest run src/__tests__/managed-config.test.ts`:
old documents parse unchanged. Sandbox plus plugin parses. Each of these
refuses startup: two sandbox entries, `driver: null`, duplicate names, a
duplicate pair, a secret-like key, an unknown key (including
`instanceDefault`), an unknown driver, a malformed `driverKey` or
`driverConfig`.
- `npx vitest run
src/services/managed-environments-plugin-entries.test.ts` (the
sandbox-entry tests stay in `managed-environments.test.ts`; shared
fixtures are in `managed-environments.test-support.ts`):
- A plugin entry is ensured with the validator's worker manager, after
the sandbox entry.
- Boot refuses a plugin entry whose `pluginKey` no auto-installed
catalog entry installs, both from the passed list and from its own
catalog lookup.
- A plugin that is down archives the row, and worker recovery re-ensures
it. A missing plugin, a plugin without the driver, or a worker manager
without `call` counts as unavailable.
- Recovery stays armed after a failed attempt and stops after one
success or a permanent config rejection (422). A `ready` during an
attempt queues one more.
- A worker that dies during the boot ensure archives the row and arms
recovery. A failure while the worker keeps running leaves the row as it
is and retries.
- Removing an entry does not archive. Plugin entries and the sandbox
stamp never change a plugin default.
  - An ensure failure is counted, not thrown.
- `npx vitest run src/__tests__/managed-plugin-environments.test.ts`
(embedded Postgres, real `applyManagedEnvironments`): the first boot
creates the row and the second changes nothing. A plugin that is down
archives the row, and recovery brings it back. Removing the entry does
not archive the row. The instance default stays unset. The row tests
from #19490 still pass.
- `npx vitest run src/__tests__/environment-routes.test.ts`: the marker
clear is refused while the document declares the entry, and allowed
after it leaves.
- `npx vitest run src/__tests__/instance-settings-service.test.ts
src/__tests__/instance-settings-cloud-defaults.test.ts`: pass with the
new `pluginEnvironments` field.
- `pnpm --filter @paperclipai/server typecheck` passes.
- I did not run the full `pnpm test:run`, `pnpm -r typecheck` or `pnpm
build`. CI covers them.

## Risks

- No schema change.
- A managed-config document with a plugin entry now provisions a row at
boot. A document without one behaves as before.
- Adoption renames an unmarked row for the same plugin and driver to the
declared name and resets its driver config to the declared one. The row
keeps its id, agents and other metadata, and it is then locked like the
managed sandbox.
- Boot now refuses to start when a plugin entry names a plugin that no
auto-installed catalog entry provides. This is the same fail-closed
posture as an unknown sandbox provider.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code
with tool use (file edits, shell, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 17:35:13 +00:00
Nicky LeachandClaude Opus 5.5 21a1e78852 feat(environments): add managed plugin environment rows (#19490)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A managed instance gets its instance-level environments from the
`PAPERCLIP_MANAGED_CONFIG` document
> - That document cannot declare a plugin environment yet, so a managed
instance cannot get a plugin-provided environment without manual setup
> - The full change is split into three pull requests. This first one
adds the plugin environment rows and their write floor, and nothing
calls them yet
> - The second pull request lets the managed config declare plugin
entries and provisions them at boot. The third sets the instance default
once and advertises the new managed-config capability
> - The benefit is a smaller review for each step, and no instance
changes behavior until the last step lands

## Linked Issues or Issue Description

**Subsystem affected**
Server environments (managed-config environment provisioning,
environment routes) and the board UI's adapter Test target.

**Problem or motivation**
A managed instance cannot get a plugin environment from its managed
config. Company settings cannot create one either. So each managed
instance that needs one requires manual setup.

**Proposed solution**
Over three pull requests, let an `environments` entry in the managed
config declare `driver: "plugin"`. Boot then provisions one locked row
for each plugin entry, and makes one of them the instance default once
on new instances. This pull request adds only the row layer: create or
adopt a marked row, archive it, find it, and keep tenants from editing
it.

**Alternatives considered**
Manual setup on each instance, and a new control-plane endpoint. Both
need a step outside the deploy. The managed config already delivers the
sandbox entry.

**Roadmap alignment**
This extends the existing managed-config `environments` section, which
already provisions the managed sandbox. It adds no new product surface.

## What Changed

- New `server/src/services/managed-plugin-environments.ts`, used through
new `environmentService` methods (`ensureManagedPluginEnvironment`,
`archiveManagedPluginEnvironment`, `findManagedPluginEnvironment`):
- The ensure validates the driver config through
`normalizeEnvironmentConfig` and
`validatePluginEnvironmentDriverConfig`, the same path an
operator-created plugin environment takes.
- It marks the row `managedByPaperclip: true` and
`managedPluginEnvironment: "<pluginKey>:<driverKey>"`.
- It adopts an existing unmarked row for the same plugin and driver
instead of adding a second one, so a hand-made row keeps its agents.
- A row that already matches gets no write. Changes write activity log
entries.
- Archive sets an active marked row to `archived`. The next ensure
reactivates it.
- Migration `0437`: unique index
`environments_managed_plugin_environment_idx` on `metadata ->>
'managedPluginEnvironment'`. This is the only schema change of the three
pull requests. Snapshot `0432` is pruned to keep the five newest.
- `server/src/routes/environments.ts`:
`isPlatformProvisionedEnvironment` also covers the new marker (write
floor and redaction). `managedPluginEnvironment` is a reserved marker
key, so a client cannot stamp it on its own row. Nothing provisions
marked rows yet, so the marker-clear escape hatch stays open for them.
The next pull request refuses it while the managed config declares the
row's entry.
- `ui/src/lib/adapter-test-environment.ts`:
`resolveManagedSandboxEnvironmentId` requires `driver === "sandbox"`.
The plugin row also carries `managedByPaperclip`, so the old "not local"
check could take it for the managed sandbox.

## Verification

- `cd server && npx vitest run
src/__tests__/managed-plugin-environments.test.ts` (embedded Postgres):
- The ensure creates the marked row through the plugin validator. A
second ensure writes nothing and logs nothing.
- A hand-made row for the same plugin and driver is adopted. It keeps
its id and its agents.
  - Archive, then ensure, reactivates the row.
- The ensure refuses while the plugin has no running worker, and refuses
a name that another environment holds.
  - The unique index allows one row per marker.
- `npx vitest run src/__tests__/environment-routes.test.ts`: on a
cloud-managed instance, edit, env-var edit, status change and delete of
a marked plugin row are refused, including for instance admins. The
marker alone keeps the floor. A client cannot stamp the marker. The
marker clear is allowed.
- `npx vitest run src/__tests__/environment-service.test.ts`: passes.
- `cd ui && npx vitest run src/lib/adapter-test-environment.test.ts`:
the plugin row is never taken for the managed sandbox. Under
managed-sandbox-only, an agent on the sandbox tests in the sandbox, and
an agent on the plugin environment tests there.
- `pnpm --filter @paperclipai/server typecheck`, `pnpm --filter
@paperclipai/ui typecheck` and `pnpm --filter @paperclipai/db
check:migrations` pass.
- I did not run the full `pnpm test:run`, `pnpm -r typecheck` or `pnpm
build`. CI covers them. `pnpm verify` asks for broader verification
because of the migration.

## Risks

- Migration: one new partial unique index on a small table. No row
carries the marker before this change, so the index cannot fail on
existing data.
- No caller uses the new ensure yet, so no instance changes behavior.
- The UI change affects only lists that hold a non-sandbox row with
`managedByPaperclip`. Today that is only the local row, which the old
check already excluded.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code
with tool use (file edits, shell, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 16:52:43 +00:00