fix(cloud): persist configuration and load release-image packages (#18335)

Land the approved Cloud configuration and release-image package stack.

Includes the separately reviewed adapter settings (#18335), external adapters (#18333), bundled plugins (#18336), startup configuration (#18339), and migration/package checks (#18341).

Includes the final image-owned loader and linked-dependency inventory corrections.

Verified stack head: 462716170d.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-10-30 19:21:33 +00:00
1 parent 36f4422389
commit 8f822bfcf7
57 files changed
+3670 -250

No files matched your search

+63
View File
@@ -722,6 +722,60 @@ connection, before enabling the new server and UI.
- The repo keeps only the newest 5 snapshots. `generate` runs `prune:snapshots` afterwards to delete older ones. Drizzle only reads the newest snapshot, and each snapshot is a full copy of the schema (over 1 MB each). Older snapshots are still in git history.
- `packages/db/src/migration-snapshot-drift.test.ts` is the enforcement backstop. It repeats the diff that `generate` performs and fails when the newest snapshot no longer matches `packages/db/src/schema/`.
## Managed adapter settings
Cloud-managed instances keep adapter selection preferences in the private
`instance_settings` singleton `adapter-settings:v1`. Its `general` JSON contains
`version: 1`, a monotonic `revision`, and `disabledTypes`. It is separate from
the public `default` settings row and never appears in the instance settings API.
Each update locks this row in a transaction. The update and company activity
records commit together. Discovery and new-agent admission read the database
at their asynchronous operation boundaries; no polling or mutable settings cache
is used. Disabled adapters remain available to existing agents.
`PAPERCLIP_ADAPTERS` and `PAPERCLIP_ADAPTERS_FILE` declare the effective operator
adapter list. The declaration is parsed once for a process configuration and
validated at startup. It does not rewrite stored preferences. A conflicting
adapter toggle returns 403. Removing the declaration restores saved choices.
Operator file edits require a restart. Self-hosted instances continue to use
`adapter-settings.json` for saved preferences.
The server-only adapter settings service exposes `inspectStored()` and
`seedIfAbsent({ disabledTypes }, transaction?)` for migration tooling. Seeding
uses the unique singleton key, returns `{ seeded, current }`, and never replaces
an existing record. The caller can use the same transaction for its import
receipt. Before admission, `initializeAdapterSettings` checks the durable row.
When absent, it imports a valid legacy settings file once, retaining unknown
adapter types, or seeds empty preferences if no file exists. It rejects files
larger than 64 KiB, malformed input, and read failures. An existing durable row
wins over every stale file. Ordinary Cloud reads do not read a legacy file.
The focused PostgreSQL suite can use an owned disposable PostgreSQL database via
`PAPERCLIP_ADAPTER_SETTINGS_TEST_DATABASE_URL` when embedded PostgreSQL is
unavailable. It applies migrations and inserts test data, so never point this
variable at an existing application database.
## Cloud configuration migration receipt
The private `instance_settings` singleton `cloud-configuration-migration:v1`
records atomic import completion, semantic source digests, imported revisions,
and metadata-only external adapter blockers. It composes the adapter and
startup-config owners without replacing an existing authoritative row. Receipt
locking and the import transaction handle concurrent attempts and retries; no
local completion marker participates in conflict resolution.
Qualification records current configuration revisions and selected image package
evidence before normal plugin reconciliation. Image inspection does not hold
admin write locks; configuration and registration snapshots are rechecked in a
short transaction before the report is saved. A historical successful report
never grants current eligibility. External metadata blockers survive loss of the
legacy home and require a deliberate revision-checked recheck of a present valid
original source file. Package mismatches remain recorded after maintenance
reconciliation. See the [bounded migration and qualification procedure](cloud-configuration-migration.md)
for the stopped-writer prerequisite, commands, retry semantics, integrity limits,
and the explicit exclusion of secret bytes, whole-server zero downtime, and
volume removal.
## Cloud runtime identity singleton
The private `instance_settings` row whose singleton key is
@@ -1490,3 +1544,12 @@ It uses disposable embedded PostgreSQL by default. A disposable local database
named `review_wait_benchmark...` can be supplied through
`PAPERCLIP_TEST_DATABASE_URL` to compare another PostgreSQL major version.
DataRow bytes exclude response metadata and transport overhead.
## Cloud runtime configuration singleton
The private `instance_settings` key `cloud-runtime-config:v1` holds a strictly
validated, versioned nonsecret configuration and revision. It is separate from
the ordinary instance settings API. Seed inserts preserve an existing winner;
replacements lock the row and reject stale revisions. Database/transaction
handles permit atomic migration receipts. Connection URLs, API keys and unknown
fields are excluded. See [field ownership and startup order](cloud-runtime-configuration.md).
+12 -1
View File
@@ -1542,7 +1542,9 @@ Paperclip also persists an empty `OPENAI_API_KEY` override for those agents so a
If the `codex` CLI is not installed or not on `PATH`, `codex_local` agent runs fail at execution time with a clear adapter error. Quota polling uses a short-lived `codex app-server` subprocess: when `codex` cannot be spawned, that provider reports `ok: false` in aggregated quota results and the API server keeps running (it must not exit on a missing binary).
Local adapters require their corresponding CLI/session setup on the machine running Paperclip. External adapters are installed through the adapter/plugin flow and should not require hardcoded imports in `server/` or `ui/`.
Local adapters require their corresponding CLI/session setup on the machine running Paperclip. Self-hosted instances can install external adapters through the adapter/plugin flow without hardcoded imports in `server/` or `ui/`.
Cloud-managed instances use only adapter implementations supplied by the release image. A tenant server token or the presence of `PAPERCLIP_MANAGED_CONFIG` enforces this boundary. Startup and UI parser requests do not read external adapter registrations or packages. Install, reinstall, reload, remove, and override operations return `403` with `adapter_install_platform_managed`, including for instance admins. Legacy `adapter-plugins.json` files stay untouched and do not change the built-in registry.
### Filesystem access boundary (2026-10-20)
@@ -3869,3 +3871,12 @@ in `_test` for `PAPERCLIP_MCP_SLEEP_TEST_DATABASE_URL`,
`PAPERCLIP_DOT_RUNNER_TEST_DATABASE_URL`. The interaction-service regression suite
also accepts `PAPERCLIP_INTERACTIONS_TEST_DATABASE_URL`. The suites migrate and write to those
URLs; the sleep suite truncates company data between tests.
## Cloud startup configuration
Cloud stores nonsecret startup choices in a private PostgreSQL singleton.
Bootstrap connection credentials stay in the supplied environment. A bounded
one-time config-file seed preserves existing choices, then later starts use the
database without loading home `.env` files or repairing worktree config. See
[Cloud runtime configuration](cloud-runtime-configuration.md) for field ownership,
secret handling, initialization order and migration interfaces.
+148
View File
@@ -0,0 +1,148 @@
# Cloud configuration migration and package qualification
This procedure covers saved adapter preferences, the allowlisted nonsecret
startup configuration, and selected bundled plugin packages. The result is a
point-in-time report. It does not authorize a deployment, guarantee zero
downtime, establish general Railway admission, or prove that a volume can be
removed. Secret and key bytes, database credentials, artifacts, logs, local
storage, workspaces, and plugin state outside PostgreSQL need their own storage
and replacement checks. This includes the decision signing key and local
encryption keys. Their byte authority is unchanged by this procedure.
Runtime environment and secret equality, plus an actual HTTP handoff between
application processes, remain separate rollout prerequisites. The package digest
does not establish those conditions.
## Migration prerequisite
Stop every legacy file writer before the first import. This includes old
application replicas and operator commands that can write configuration or
adapter settings. One maintenance window is sufficient. The importer cannot
make an old file writer participate in its database transaction. Keep those
writers stopped after import; later file edits do not update database authority.
Keep the normal database backup and recovery procedure for the instance. Use the
new release with the same managed environment and database connection. The
server runs migration before constructing configuration-dependent services.
For an explicit source-checkout invocation, use:
```sh
pnpm exec tsx server/src/qualify-cloud-configuration.ts --migrate
```
The command uses `DATABASE_URL` and the normal Cloud environment. It does not
accept credentials as arguments. It writes the receipt to PostgreSQL and prints
a JSON report; no report file or local completion marker is required. Exit code
0 means the requested configuration/package check passed, 2 means the report has
blockers, and 1 means the operation failed. A fresh installation can report a
missing plugin registration before normal boot installation completes. Run the
check again after that installation settles.
## Database authority and retry behavior
The private `instance_settings` singleton `cloud-configuration-migration:v1`
contains the versioned receipt. The transaction locks that row and composes the
adapter and startup-config owners' atomic seed methods. Completion and both
seeds commit together. A failed write or crash before commit leaves no completed
receipt. A retry uses committed database state and never replaces a later
settings revision. Earlier independently seeded owner records remain authoritative.
The receipt records source kind, a digest of the nonsecret semantic candidate,
the imported revision, and bounded legacy external-registration evidence.
An absent startup configuration remains an authoritative `config: null`; it is
distinct from an uninitialized database record. No raw secret values are copied.
Once the receipt exists, ordinary boots do not reread legacy source files.
A replacement home can be empty or absent without changing saved authority.
All legacy readers have a 64 KiB limit. They require regular files, reject
symlinks and FIFOs, and use bounded reads. External registration inspection reads
metadata only. It never loads a registered module or follows a registered package
path. A nonempty, invalid, or unreadable registration file creates a durable
blocker. Removing the old home does not clear that blocker.
To resolve an external-registration blocker, stop its writer, deliberately
remove the unsupported registration metadata at the original source location,
and leave a present valid empty JSON array file. Read the current receipt revision
from the report, then run:
```sh
pnpm exec tsx server/src/qualify-cloud-configuration.ts --recheck-external-revision 3
```
Use the actual current revision in place of `3`. A changed revision, a different
source location, or a missing/invalid source file rejects the recheck. Reusing
the same container path with an empty home is insufficient. This is an explicit
operator resolution; qualification never clears the blocker automatically.
## Current packages and overlap comparison
Every qualification reads current settings revisions and plugin registrations.
It resolves selected packages from this image with the same bundled package
identity rules as normal loading. It verifies package bytes before importing a
trusted image manifest, then compares that manifest with the approved database
registration. Missing packages, missing evidence, incompatible versions,
unselected registrations, and integrity failures prevent qualification. Legacy
package paths and npm discovery cannot supply missing image bytes.
Image inspection happens outside database write locks. A short transaction then
locks and rereads the current configuration and registration records. A change
during inspection produces a blocker and requires a new check. The report gives
`adapterRevision`, `configRevision`, `configurationDigest`, and
`packageEvidenceDigest`. A historical completed receipt is evidence, not current
authorization. Call qualification again for a current result.
The server records this report before ordinary bundled-plugin reconciliation.
It retains `lastPackageMismatch` after later successful checks, so reconciliation
cannot erase the evidence seen before an upgrade. The report does not block
normal maintenance upgrades or replace capability approval. After an upgrade
settles, its current packages can qualify normally.
For a same-package comparison, capture the current release's
`packageEvidenceDigest` and pass it explicitly on the replacement:
```sh
pnpm exec tsx server/src/qualify-cloud-configuration.ts --expected-package-evidence sha256:REPLACE_WITH_THE_64_HEX_DIGEST
```
`configurationPackagesEligible` describes current database/configuration/package
compatibility. `samePackageOverlapEligible` is true only when an explicit expected
digest also matches. There is no permanent first-release baseline that would
prevent future maintenance releases from establishing their own evidence.
The package evidence digest covers selected packages, not container image bytes.
Distribution packages reuse and reverify their catalog digest. Builtin packages
have a bounded inventory of physical file bytes, modes, and logical symlink
targets, including standalone pnpm dependency trees. With the selected catalog
root, inventory may follow links only into that provider, the catalog SDK,
`packages/shared`, and the current image's root `node_modules`. Reached pnpm
package units include their sibling dependency links; unrelated root packages
are not scanned. Canonical directory/file deduplication bounds cycles, and
logical root labels preserve the digest when identical image bytes relocate.
Dangling or escaping links, special files, more than 50,000 entries, depth above
32, metadata above 128 KiB, or more than 256 MiB per package fail verification.
The measured cached Cloud Daytona closure has about 22,253 entries and 130 MB;
that artifact predates this change and is not a complete current release proof.
Inventory work is synchronous and linear in selected package bytes. It runs only during startup before the application is constructed
or in the explicit operator process, with no idle polling. The report includes
`packageInspectionDurationMs`. Packages must remain immutable throughout a check.
## Verification scope
The focused PostgreSQL tests use two independent clients and separate home/image
roots. They import legacy settings, change saved revisions, remove the old roots,
reconstruct from the database, and start a real plugin worker from the replacement
image fixture. They also cover concurrent migration, commit failure and retry,
database failure, stale sources, durable external blockers, package mismatch,
maintenance reconciliation, and changes during manifest inspection.
The bundled-plugin registration suite also has an actual prebuilt Daytona case.
Build its standalone package with the normal SDK linker before running that case;
it is skipped when the provider build is absent. It qualifies the real linked
SDK/shared package graph and uses the production loader and worker manager for
credential-free initialize/health calls. The runtime-path tests also invoke the
production server bundler to verify its preserved module locations.
Set `PAPERCLIP_CLOUD_CONFIGURATION_TEST_DATABASE_URL` only to an owned disposable
PostgreSQL database when embedded PostgreSQL is unavailable. The suite applies
migrations and writes/deletes test records. A complete release image, hosted
replacement, overlapping production processes, secret-byte migration, and volume
removal are not established by these local fixtures.
+118
View File
@@ -0,0 +1,118 @@
# Cloud runtime configuration
Cloud-managed servers store nonsecret startup and application choices in the
private `instance_settings` singleton `cloud-runtime-config:v1`. The row contains
`version: 1`, an increasing `revision`, and a strict allowlist of configuration
fields. It is separate from the normal instance settings row and is not exposed
by the instance settings API.
The canonical managed-instance predicate applies: a nonblank Cloud tenant token
or the presence of `PAPERCLIP_MANAGED_CONFIG`, including a blank value, selects
this configuration path. Invalid managed feature documents still retain their
existing validation at application startup.
## Startup and authority
1. The hosting environment supplies `DATABASE_URL` and, if needed,
`DATABASE_MIGRATION_URL`. Cloud requires a PostgreSQL URL and never falls back
to embedded PostgreSQL. Credentials, trust keys, provider configuration and
deployment wiring needed before the database connection remain supplied by
the environment or the existing secret system. This phase parses only the
database connection record. Runtime mode, host binding and authentication
checks wait until the saved snapshot has loaded.
2. After schema migration and database connection, startup reads the private
runtime configuration row. An existing invalid row or a database error stops
startup. Neither condition selects a legacy file or an old process snapshot.
3. If no row exists, startup imports the legacy config once. It uses
`PAPERCLIP_CONFIG` when supplied, otherwise the instance home config path.
It does not search working-directory ancestors. A truly absent file saves
an authoritative `config: null` marker. This retains the existing dynamic
environment/image defaults, including authentication and secret strictness.
The marker never enables a later file fallback. A present malformed, linked, nonregular, or oversized file
stops initialization. Reads and stored snapshots are limited to 64 KiB.
4. The seed uses an insert that does not replace an existing row. Concurrent
initializers read the committed winner. Later starts do not inspect the
legacy file, even when it is absent, corrupt, or changed.
5. Startup publishes a detached configuration snapshot, restores the signed
Cloud runtime identity, then initializes telemetry, secret-provider defaults,
authentication and application services. A saved telemetry opt-out therefore
takes effect before a telemetry client can be created.
Existing operator environment overlays keep precedence over saved values.
Reading an overlaid value does not save it back to the database. Credentials and
connection URLs are never copied into this JSONB row.
Cloud does not load instance or working-directory `.env` files and does not run
legacy worktree configuration repair. The hosting system must supply those
bootstrap variables directly. In particular, a database connection kept only
in `.env` cannot bootstrap this path. Self-hosted configuration and dotenv
behavior remain unchanged.
## Field ownership
The table lists every field consumed from the existing config file and the
remaining known schema fields. Environment-only `Config` fields, such as
heartbeat maintenance flags and feedback-export credentials, retain their
existing environment sources and are not frozen into the saved snapshot.
| Config file fields | Cloud owner |
| --- | --- |
| `server.deploymentMode`, `exposure`, `bind`, `customBindHost`, `host`, `port`, `allowedHostnames`, `serveUi` | Saved runtime configuration, with existing environment overlays. |
| `auth.baseUrlMode`, `publicBaseUrl`, `disableSignUp` | Saved runtime configuration, with environment overlays and the signed Cloud identity restoration. Saved URLs cannot contain userinfo, query parameters, or fragments. |
| `database.backup.enabled`, `intervalMinutes`, `retentionDays`, `dir` | Saved runtime configuration, with existing backup environment overlays. |
| `database.mode`, `connectionString` | Cloud bootstrap requires the environment `DATABASE_URL`. The legacy URL is never saved. |
| `database.embeddedPostgresDataDir`, `embeddedPostgresPort` | Local embedded-database settings. Cloud does not use embedded PostgreSQL or these paths. |
| `storage.provider`, `localDisk.baseDir`, `s3.bucket`, `s3.region`, `s3.endpoint`, `s3.prefix`, `s3.forcePathStyle` | Saved runtime configuration, with existing environment overlays. Saved endpoints cannot contain userinfo, query parameters, or fragments. |
| `secrets.provider`, `strictMode`, `localEncrypted.keyFilePath` | Saved nonsecret provider choice and locator, with existing environment overlays. No key bytes are imported or saved. |
| `telemetry.enabled` | Saved choice, passed to the existing telemetry initialization and opt-out policy before client creation. |
| `llm.provider` | Saved provider metadata. |
| `llm.apiKey` | Never saved. The existing OpenAI model-discovery consumer uses `OPENAI_API_KEY` first. A legacy OpenAI key requires that environment credential before import, or startup stops. Claude key metadata has no server consumer. |
| `logging.mode`, `logDir`, `updates.checkEnabled` | Nonsecret legacy/CLI metadata is preserved. Server logging continues to use its existing environment configuration. |
| `$meta` | CLI file-write metadata is not application authority. The compatibility view synthesizes valid metadata. |
| Unknown extension keys | No server config consumer exists. They are excluded from the strict saved allowlist, including any unknown credential fields. |
A saved path is a locator. This change does not make local storage objects,
backup bytes, logging files, or secret-key bytes durable. Their storage owners
and existing environment/secret mechanisms retain responsibility for those
bytes. Restart tests prove configuration restoration under a different home;
they do not prove durable file, secret, or backup storage.
The decision signing service still uses `PAPERCLIP_DECISION_SIGNING_SECRET` when
supplied and otherwise reads or creates its instance key file before the database
connection. The local encrypted secret provider still uses its environment key
or existing key-file mechanism. Those byte-storage fallbacks are outside this
configuration migration. Qualification must account for them separately; this
PR does not establish file-free secret storage on Cloud.
## Internal update and migration interface
`cloudRuntimeConfigService(db)` provides `inspectStored(handle?)`,
`seedIfAbsent(config, handle?)`, and `replace(config, expectedRevision, handle?)`.
Each method accepts a database or transaction handle. A migration can compose a
config seed, related settings and a receipt in one transaction.
Replacement validates the same allowlist, locks the singleton row, checks the
expected revision, and increases it. A stale revision fails. There is no new
public configuration endpoint or settings UI. Startup consumers use a snapshot;
a saved replacement takes effect on the next initialization/restart.
`readLegacyCloudRuntimeConfig(env?)` returns the validated nonsecret candidate
without writing it. `initializeCloudRuntimeConfig` performs the standalone
one-time seed and publishes its committed snapshot. Integrated migration
receipts and deployed Cloud qualification are separate consumers of this
interface.
## Verification
`server/src/__tests__/cloud-runtime-config.test.ts` exercises real PostgreSQL
clients, concurrent writes, transaction rollback, changed homes, a fresh Node
process, environment precedence, secret exclusion, invalid data and database
failure. It uses embedded PostgreSQL by default. A dedicated disposable database
can be selected with `PAPERCLIP_RUNTIME_CONFIG_TEST_DATABASE_URL` when the host
cannot start embedded PostgreSQL. That database must be reserved for this test.
The test applies the current schema before use.
The startup orchestration controls in
`server-startup-feedback-export.test.ts` hold configuration initialization and
verify that telemetry, authentication and app construction wait for it. These
local controls do not claim a live Cloud rollout.
+36
View File
@@ -34,6 +34,42 @@ Current limitations to keep in mind:
In practice, that means the current implementation is a good fit for local development and self-hosted persistent deployments, but not yet for multi-instance cloud plugin distribution.
### Managed image packages
Managed Cloud provisions only plugins selected from the current release image
catalog. A Cloud tenant token or managed configuration keeps this image-only
policy active. Without an explicit selected package list, Cloud selects no
packages and cannot fall back to local installs. An explicit operator-managed
list also keeps this policy active outside Cloud. The database registration's
plugin key, package name, version and
approved manifest identify that package. Its legacy `packagePath` remains for
local/self-hosted compatibility; Cloud never uses that path to locate code.
Workers, plugin migrations and UI files resolve through the same current-image
package resolver. Cloud does not install npm dependencies, build packages, use
the development UI proxy, or fall back to a server home directory.
Prebuilt workers use the current release's baked CLI `tsx` loader for the SDK's
workspace TypeScript dependencies. The loader is located from the owning server
package in both regular and bundled server layouts. Missing loaders and loader
links outside the release's CLI or root `node_modules` fail activation. This
loads image-supplied code without installing dependencies or building packages.
Equivalent packages at different image roots reuse the existing registration,
configuration, approval status and company scopes without rewriting its stored
path. Concurrent first registration of the same package retains one plugin ID.
Conflicting package identities fail explicitly. Normal maintenance boot retains
the existing bundled manifest/version reconciliation and approval gates.
`createBundledPluginPackageResolver(...).inspect(registration, verifiedManifest)`
is a read-only qualification hook. Resolve the current image path, load its
manifest with the validated plugin loader, and inspect the registration before
normal boot reconciliation. Missing manifest evidence and incompatible packages
produce explicit blockers. Distribution packages retain their catalog digest
verification; built-in packages trust the immutable release image and must bump
their version when their code changes. Qualification supports identical package
sets only. This package-resolution boundary does not coordinate workers,
background jobs, or deployment overlap between servers.
## 1. Scope
This spec covers:
@@ -74,6 +74,9 @@ API: `present` — representative operation IDs:
- `PATCH /api/instance/settings`.
- `GET /api/instance/settings/experimental`.
- `PATCH /api/instance/settings/experimental`.
- `GET /api/adapters`.
- `GET /api/adapters/{type}`.
- `PATCH /api/adapters/{type}`.
Agent access: `restricted` — Board/admin instance settings; CLI configure/env; local
configuration files.
@@ -88,9 +91,38 @@ Verification: `not-run` — assert the documented agent denial with an owned tes
verify any listed agent request/result path separately. A delegated operation needs its
own policy and access review.
Cloud runtime configuration uses a separate private PostgreSQL singleton and
adds no public read or mutation endpoint. Existing instance settings APIs do not
expose it. Bootstrap database credentials and trust keys remain in the supplied
environment; the strict saved allowlist excludes API keys and connection URLs.
Internal configuration updates require a matching revision and transaction lock.
[Database controls](../../server/src/__tests__/cloud-runtime-config.test.ts) cover
seed/restart, credential exclusion, conflicts and rollback. These are local
controls, not agent authorization or deployed Cloud qualification.
Gaps: Settings APIs cover parts of configuration; pre-start database/binding/storage
configuration is local operator tooling.
Adapter toggles require instance-admin authority and remain blocked when
`instance.adapters` is hidden. Ordinary board members can read the catalog;
agent keys cannot toggle adapters. Managed Cloud preferences are stored in a
private database singleton. The operator adapter declaration sets the effective
list and rejects conflicting toggles. [PostgreSQL regression coverage](../../server/src/services/adapter-settings-postgres.test.ts)
exercises persistence, concurrent writers, admission freshness, activity audit,
and failed writes; [authorization tests](../../server/src/__tests__/adapter-routes-authz.test.ts)
cover admin and hidden-setting gates. These are local tests, not deployed proof.
Cloud configuration migration and package qualification are operator commands,
not agent-accessible REST capabilities. They require the managed environment and
database operator access. The [command](../../server/src/qualify-cloud-configuration.ts)
composes transaction-safe owner seeds and writes a private receipt. Every
qualification rereads current revisions and image package evidence; a completed
historical receipt grants no deployment or execution authority. [PostgreSQL coverage](../../server/src/__tests__/cloud-configuration-migration-postgres.test.ts)
uses independent clients, lost-home reconstruction, a real replacement fixture
worker, rollback/retry, and durable blockers. [Reader coverage](../../server/src/__tests__/cloud-configuration-readers.test.ts)
checks bounded sources and pnpm-style link integrity. These are local fixtures;
see the [scope and operator procedure](../../doc/cloud-configuration-migration.md).
Cloud stack Admins may update experimental settings through the trusted Cloud
actor without receiving other instance-admin powers. This uses the current
stack role on each request; company membership alone does not grant it.
+8 -1
View File
@@ -55,7 +55,14 @@ Policy: Company model discovery requires company access; account-backed discover
also enforces its account/pool and target-agent rules. Instance adapter inventory
and installation retain board/admin gates. Delegated managed
AI-account login operations are enumerated under `model-accounts`; they do not
grant raw host administration or execution-session reset.
grant raw host administration or execution-session reset. On Cloud-managed
instances, external adapter install, reinstall, reload, removal and override
operations return `403` with `adapter_install_platform_managed`, including for
instance admins. Built-in metadata remains available to board users. Startup
and UI parser reads do not consume saved external registrations or package code.
[Local fixture controls](../../server/src/adapters/cloud-external-adapters.test.ts)
and [route controls](../../server/src/__tests__/adapter-routes-authz.test.ts)
cover this boundary; they do not establish live agent or provider qualification.
Evidence: `source-only` — [implementation](../../server/src/routes/agents.ts#L6608) at
the reviewed commit above.
+30
View File
@@ -63,8 +63,38 @@ unfinished application write must still prevent sleep after its client leaves.
### `settings-backup`
Cloud startup reads nonsecret application choices from a private, versioned
PostgreSQL singleton after a bounded one-time legacy config seed. A fresh
process and different home restore the saved choices without consulting a stale
`.env` file. Invalid saved data or an unavailable database prevents startup;
saved telemetry opt-out applies before client creation. The
[database controls](../../server/src/__tests__/cloud-runtime-config.test.ts) and
[startup-order controls](../../server/src/__tests__/server-startup-feedback-export.test.ts)
cover these local boundaries. See [field ownership](../../doc/cloud-runtime-configuration.md).
These controls do not qualify a deployed Cloud rollout or durable file storage.
Automated: [experimental settings](../../ui/src/pages/InstanceExperimentalSettings.test.tsx) and [database backup routes](../../server/src/__tests__/instance-database-backups-routes.test.ts) cover their UI/API contracts.
Cloud migration: in one maintenance window, stop legacy file writers and run the
[configuration/package qualification command](../../doc/cloud-configuration-migration.md).
Inspect its database receipt, update a saved preference, then repeat against a
second owned application/image root with the first home unavailable. Use an
explicit package evidence digest for same-package comparison. Confirm stale
qualification and missing legacy files cannot clear blockers. This recipe covers
configuration/packages and does not qualify whole-server zero downtime, secret
bytes, or volume removal. [Local PostgreSQL evidence path](../../server/src/__tests__/cloud-configuration-migration-postgres.test.ts)
includes a real fixture worker; it does not establish a hosted replacement.
Managed adapters: change a visible adapter preference as an instance admin,
then use a second application process against the same disposable database to
confirm catalog and new-agent selection reflect it. Verify saved agents remain
usable. Restart with an operator adapter declaration, confirm its effective
list, and reject conflicting toggles. Remove the declaration and confirm saved
preferences return. [Database tests](../../server/src/services/adapter-settings-postgres.test.ts)
cover local persistence, concurrent writers, audit rollback, and failure responses;
[local compatibility tests](../../server/src/services/adapter-settings-local.test.ts)
cover the legacy file. No deployed walkthrough is claimed.
Manual: Change a harmless experimental setting, reload, and confirm the affected route honors it. Back up a disposable instance, inspect the reported file and metadata, and restore using the documented procedure into a second isolated target. Verify a known record after restoration.
Cloud staging: On a managed `*.staging.paperclip.app` instance, verify experiments
+9
View File
@@ -170,6 +170,15 @@ controls do not establish live provider acceptance.
### `adapter-setup`
Cloud-managed instances load adapters from the release image. Saved external
registrations cannot replace built-ins or supply UI parser code. Install,
reinstall, reload, remove, and override operations remain platform-managed.
The [fixture loader controls](../../server/src/adapters/cloud-external-adapters.test.ts)
exercise real packages, import side effects, both managed-instance signals and
self-hosted compatibility. The [route authorization controls](../../server/src/__tests__/adapter-routes-authz.test.ts)
check managed denials before package or registration access. These are local
controls, not live Cloud provider qualification.
Automated: [adapter validation routes](../../server/src/__tests__/agent-adapter-validation-routes.test.ts) and [model refresh](../../server/src/__tests__/adapter-model-refresh-routes.test.ts) cover server behavior.
Claude native setup uses the existing Anthropic API-key or subscription connection
@@ -1,6 +1,6 @@
{
"name": "@paperclipai/plugin-daytona",
"version": "0.1.1",
"version": "0.1.9",
"description": "Daytona sandbox provider plugin for Paperclip environments",
"license": "MIT",
"homepage": "https://github.com/paperclipai/paperclip",
@@ -1,6 +1,6 @@
{
"name": "@paperclipai/plugin-kubernetes",
"version": "0.1.0",
"version": "0.1.0-alpha.1",
"description": "Kubernetes sandbox provider plugin for Paperclip environments",
"license": "MIT",
"homepage": "https://github.com/paperclipai/paperclip",
@@ -300,7 +300,7 @@ describe("adapter management route authorization", () => {
mocks.reloadExternalAdapter.mockImplementation(async (type: string) =>
createAdapter(type),
);
}, 20_000);
}, 60_000);
afterEach(() => {
unregisterServerAdapter(EXTERNAL_ADAPTER_TYPE);
@@ -373,35 +373,31 @@ describe("adapter management route authorization", () => {
},
);
describe("cloud-managed adapter code install floor", () => {
beforeEach(() => {
process.env.PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN = "test-server-token";
describe("cloud-managed adapter code floor", () => {
afterEach(() => vi.unstubAllEnvs());
it.each(
["PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", "PAPERCLIP_MANAGED_CONFIG"].flatMap((signal) =>
["install", "reinstall", "reload", "override", "delete"].map((routeName) => [signal, routeName]),
),
)("blocks %s adapter %s before package or registration access", async (signal, routeName) => {
seedInstalledExternalAdapter();
vi.clearAllMocks();
vi.stubEnv(signal, signal === "PAPERCLIP_MANAGED_CONFIG" ? "" : "test-server-token");
const app = createApp(instanceAdmin);
const res = await sendMutatingRequest(app, routeName);
expect(res.status, `${routeName}: ${JSON.stringify(res.body)}`).toBe(403);
expect(res.body.details).toMatchObject({ code: "adapter_install_platform_managed" });
expect(mocks.execFile).not.toHaveBeenCalled();
expect(mocks.loadExternalAdapterPackage).not.toHaveBeenCalled();
expect(mocks.reloadExternalAdapter).not.toHaveBeenCalled();
expect(mocks.getAdapterPluginByType).not.toHaveBeenCalled();
expect(mocks.addAdapterPlugin).not.toHaveBeenCalled();
expect(mocks.removeAdapterPlugin).not.toHaveBeenCalled();
expect(mocks.externalRecords.get(EXTERNAL_ADAPTER_TYPE)).toEqual(installedRecord());
});
afterEach(() => {
delete process.env.PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN;
});
it.each(["install", "reinstall"] as const)(
"floors adapter %s off for instance admins on cloud-managed instances",
async (routeName) => {
resetInstalledExternalAdapterState();
if (routeName !== "install") {
seedInstalledExternalAdapter();
}
const app = createApp(instanceAdmin);
const res = await sendMutatingRequest(app, routeName);
expect(res.status, `${routeName}: ${JSON.stringify(res.body)}`).toBe(
403,
);
expect(res.body.details).toMatchObject({
code: "adapter_install_platform_managed",
});
expect(mocks.execFile).not.toHaveBeenCalled();
expect(mocks.loadExternalAdapterPackage).not.toHaveBeenCalled();
},
);
});
describe("operator-hidden adapter management floor", () => {
@@ -103,6 +103,11 @@ vi.mock("../services/instance-settings.js", () => ({
instanceSettingsService: () => mockInstanceSettingsService,
}));
vi.mock("../services/adapter-settings.js", async (importOriginal) => ({
...(await importOriginal<typeof import("../services/adapter-settings.js")>()),
adapterSettingsService: () => ({ getDisabledTypes: async () => [] }),
}));
vi.mock("../services/dot-runner-broker.js", () => ({
dotRunnerBroker: () => ({ enabled: async () => true, bindingForAgent: async () => null }),
}));
@@ -0,0 +1,86 @@
import { rmSync, writeFileSync, symlinkSync, mkdirSync, readFileSync } from "node:fs";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { bundledImageFixture } from "./helpers/bundled-image-fixture.js";
import { createBundledPluginPackageResolver } from "../services/bundled-plugin-package.js";
import { BUNDLED_PLUGIN_CATALOG, resolveBundledPluginInstalls } from "../services/bundled-plugins.js";
import { readDistributionPluginCatalog } from "../services/distribution-plugin-catalog.js";
import boxd from "../../../packages/plugins/sandbox-providers/boxd/src/manifest.js";
import createos from "../../../packages/plugins/sandbox-providers/createos/src/manifest.js";
import cloudflare from "../../../packages/plugins/sandbox-providers/cloudflare/src/manifest.js";
import daytona from "../../../packages/plugins/sandbox-providers/daytona/src/manifest.js";
import e2b from "../../../packages/plugins/sandbox-providers/e2b/src/manifest.js";
import exe_dev from "../../../packages/plugins/sandbox-providers/exe-dev/src/manifest.js";
import kubernetes from "../../../packages/plugins/sandbox-providers/kubernetes/src/manifest.js";
import modal from "../../../packages/plugins/sandbox-providers/modal/src/manifest.js";
import novita from "../../../packages/plugins/sandbox-providers/novita/src/manifest.js";
const manifests = { boxd, createos, cloudflare, daytona, e2b, "exe-dev": exe_dev, kubernetes, modal, novita };
const roots: string[] = [];
function fixture() { const result = bundledImageFixture(); roots.push(result.root); return result; }
afterEach(() => { for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); });
describe("bundled image package identity", () => {
it("resolves independent image roots from the same registration without any previous path", () => {
const first = fixture();
const second = fixture();
const registration = { ...first.registration, packagePath: "/lost/old/home/plugins/fixture" };
expect(first.resolver.resolve(registration)).toBe(first.packageRoot);
expect(second.resolver.resolve(registration)).toBe(second.packageRoot);
expect(first.resolver.inspect(registration, first.manifest)).toEqual(second.resolver.inspect(registration, second.manifest));
expect(registration.packagePath).toBe("/lost/old/home/plugins/fixture");
});
it("requires verified manifest evidence and blocks same-version manifest drift", () => {
const { resolver, registration, manifest } = fixture();
expect(resolver.inspect(registration)).toMatchObject({ compatible: false, code: "missing_manifest_evidence" });
expect(resolver.inspect(registration, { ...manifest, description: "changed" })).toMatchObject({ compatible: false, code: "incompatible_bundled_package", reason: expect.stringContaining("Approved plugin manifest") });
expect(resolver.inspect(registration, manifest)).toMatchObject({ compatible: true, identity: { kind: "bundled_image", catalogKey: "fixture", pluginKey: manifest.id, version: "1.0.0" } });
});
it("rejects mismatched package names, versions, identity and escaping entrypoints", () => {
const { resolver, registration } = fixture();
for (const update of [
{ packageName: "@other/plugin" }, { version: "2.0.0" }, { pluginKey: "other.plugin" },
{ manifestJson: { ...registration.manifestJson, version: "2.0.0" } },
{ manifestJson: { ...registration.manifestJson, entrypoints: { worker: "../worker.js" } } },
]) expect(() => resolver.resolve({ ...registration, ...update })).toThrow();
});
it("fails explicitly for missing or deselected bundles even when a stale npm package exists", () => {
const { root, packageRoot, resolver, registration } = fixture();
const fallback = path.join(root, "home", "node_modules", registration.packageName);
mkdirSync(fallback, { recursive: true });
writeFileSync(path.join(fallback, "worker.js"), "throw new Error('unsafe fallback');");
rmSync(packageRoot, { recursive: true });
expect(() => resolver.resolve(registration)).toThrow(/ENOENT/);
expect(() => createBundledPluginPackageResolver([]).resolve(registration)).toThrow(/absent or not selected/);
});
it("rejects tampering before manifest import and symlink escapes before worker resolution", () => {
const { root, packageRoot, registration, resolver } = fixture();
writeFileSync(path.join(packageRoot, "dist", "manifest.js"), "throw new Error('must never import');");
expect(() => readDistributionPluginCatalog(root, BUNDLED_PLUGIN_CATALOG)).toThrow(/digest mismatch/);
rmSync(path.join(packageRoot, "dist", "worker.js"));
symlinkSync(path.join(root, "distribution", "catalog.json"), path.join(packageRoot, "dist", "worker.js"));
expect(() => resolver.resolve(registration)).toThrow(/escapes/);
});
it.each(BUNDLED_PLUGIN_CATALOG)("accepts the real shipped $key package and manifest contract", (entry) => {
const manifest = manifests[entry.key as keyof typeof manifests];
const pkg = JSON.parse(readFileSync(new URL(`../../../packages/plugins/${entry.relativePath}/package.json`, import.meta.url), "utf8"));
const { root } = fixture();
const packageRoot = path.join(root, entry.relativePath);
mkdirSync(path.join(packageRoot, "dist"), { recursive: true });
writeFileSync(path.join(packageRoot, "package.json"), JSON.stringify(pkg));
writeFileSync(path.join(packageRoot, "dist", "manifest.js"), `export default ${JSON.stringify(manifest)};`);
writeFileSync(path.join(packageRoot, "dist", "worker.js"), "export default {};");
const installs = resolveBundledPluginInstalls([entry.key], { catalogRoot: root, enforceCatalogRoot: true, distributionPlugins: [], env: {} });
const resolver = createBundledPluginPackageResolver(installs);
const registration = { pluginKey: entry.pluginKey, packageName: pkg.name, version: manifest.version, manifestJson: manifest };
expect(pkg.version).toBe(manifest.version);
expect(resolver.resolve(registration)).toBe(packageRoot);
expect(resolver.inspect(registration, manifest)).toMatchObject({ compatible: true });
});
});
@@ -0,0 +1,287 @@
import express from "express";
import request from "supertest";
import { execFileSync } from "node:child_process";
import { rmSync, mkdirSync, writeFileSync, cpSync, readFileSync, existsSync, mkdtempSync } from "node:fs";
import path from "node:path";
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
import { applyPendingMigrations, closeRegisteredClients, companies, createDb, plugins } from "@paperclipai/db";
import { eq } from "drizzle-orm";
import type { PluginRecord } from "@paperclipai/shared";
import { createPluginWorkerManager } from "../services/plugin-worker-manager.js";
import { pluginLoader, type PluginRuntimeServices } from "../services/plugin-loader.js";
import { linkSdkInto } from "../../../scripts/link-plugin-dev-sdk.mjs";
import { fileURLToPath } from "node:url";
import { createBundledPluginPackageResolver } from "../services/bundled-plugin-package.js";
import { inspectCloudPackages } from "../services/cloud-package-qualification.js";
import { resolveBundledPluginInstalls } from "../services/bundled-plugins.js";
import { pluginRegistryService } from "../services/plugin-registry.js";
import { pluginUiStaticRoutes } from "../routes/plugin-ui-static.js";
import { errorHandler } from "../middleware/error-handler.js";
import { derivePluginDatabaseNamespace } from "../services/plugin-database.js";
import { ensureBundledPlugins } from "../services/bundled-plugins.js";
import daytonaManifest from "../../../packages/plugins/sandbox-providers/daytona/src/manifest.js";
import { bundledImageFixture } from "./helpers/bundled-image-fixture.js";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
const externalDatabaseUrl = process.env.PAPERCLIP_BUNDLED_PLUGIN_TEST_DATABASE_URL?.trim();
const support = externalDatabaseUrl ? { supported: true } : await getEmbeddedPostgresTestSupport();
const describePostgres = support.supported ? describe : describe.skip;
describePostgres("bundled image registration in PostgreSQL", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: ReturnType<typeof createDb>;
let peer: ReturnType<typeof createDb>;
const roots: string[] = [];
const fixture = () => { const result = bundledImageFixture(); roots.push(result.root); return result; };
beforeAll(async () => {
if (externalDatabaseUrl) {
await applyPendingMigrations(externalDatabaseUrl);
database = { connectionString: externalDatabaseUrl, cleanup: () => closeRegisteredClients(externalDatabaseUrl) };
} else {
database = await startEmbeddedPostgresTestDatabase("bundled-image-registration-");
}
db = createDb(database.connectionString);
peer = createDb(database.connectionString);
}, 90_000);
afterEach(async () => {
await db.delete(plugins);
await db.delete(companies);
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true });
});
afterAll(async () => { await database?.cleanup(); });
function runtime() {
const output: string[] = [];
const services = {
workerManager: {
startWorker: vi.fn(async (_id: string, options: { entrypointPath: string }) => { output.push(execFileSync(process.execPath, [options.entrypointPath, "--source"], { encoding: "utf8" })); }),
call: vi.fn(async () => ({})),
},
lifecycleManager: { markError: vi.fn() },
jobStore: { syncJobDeclarations: vi.fn() }, jobScheduler: { registerPlugin: vi.fn() },
toolDispatcher: {}, eventBus: { forPlugin: vi.fn(), subscriptionCount: vi.fn(() => 0) },
buildHostHandlers: vi.fn(() => ({})), instanceInfo: { instanceId: "fixture", hostVersion: "1.0.0" },
};
return { services, output };
}
function loaderFor(image: ReturnType<typeof fixture>, databaseClient = db, services?: PluginRuntimeServices) {
return pluginLoader(databaseClient, {
localPluginDir: path.join(image.root, "lost-home"), prebuiltOnly: true,
resolvePackageRoot: image.resolver.resolve, assertPackageActivation: image.resolver.assertActivation,
enableLocalFilesystem: false, enableNpmDiscovery: false,
}, services);
}
it("starts current-image workers after losing the old home and retains each company's config", async () => {
const old = fixture();
const first = fixture();
const second = fixture();
await loaderFor(old).installPlugin({ localPath: old.packageRoot });
const registry = pluginRegistryService(db);
let row = (await registry.getByKey(old.manifest.id))!;
await registry.updateStatus(row.id, { status: "ready" });
const [companyA, companyB, unconfigured] = await db.insert(companies).values([
{ name: "Company A", issuePrefix: "CFA" }, { name: "Company B", issuePrefix: "CFB" }, { name: "Unconfigured", issuePrefix: "CFC" },
]).returning();
await registry.upsertConfig(row.id, companyA!.id, { configJson: { token: "only-A" } });
await registry.upsertConfig(row.id, companyB!.id, { configJson: { token: "only-B" } });
row = (await registry.getById(row.id))!;
rmSync(old.root, { recursive: true });
const original = { ...row };
for (const image of [first, second]) {
const current = runtime();
const loader = loaderFor(image, db, current.services as unknown as PluginRuntimeServices);
const result = await loader.loadSingle(row.id);
expect(result.success).toBe(true);
expect(current.output).toEqual([expect.stringContaining(image.packageRoot)]);
expect(current.services.workerManager.startWorker).toHaveBeenCalledWith(row.id, expect.objectContaining({
entrypointPath: path.join(image.packageRoot, "dist", "worker.js"), config: {}, proactiveCompanyScopes: [companyA!.id, companyB!.id].sort(),
}));
expect(current.services.workerManager.call.mock.calls).toEqual(expect.arrayContaining([
[row.id, "configChanged", { companyId: companyA!.id, config: { token: "only-A" } }],
[row.id, "configChanged", { companyId: companyB!.id, config: { token: "only-B" } }],
]));
expect(current.services.workerManager.call).toHaveBeenCalledTimes(2);
expect(await registry.getConfig(row.id, unconfigured!.id)).toBeNull();
const app = express();
app.use((req, _res, next) => {
req.actor = { type: "board", userId: "fixture-user", source: "session", isInstanceAdmin: false, companyIds: [companyA!.id] };
next();
});
app.use(pluginUiStaticRoutes(db, { localPluginDir: path.join(image.root, "lost-home"), resolvePackageRoot: image.resolver.resolve }));
app.use(errorHandler);
const served = await request(app).get(`/_plugins/${row.id}/ui/index.js`).query({ companyId: companyA!.id });
expect(served.status).toBe(200);
expect(served.text).toContain("current-image");
expect((await request(app).get(`/_plugins/${row.id}/ui/index.js`).query({ companyId: companyB!.id })).status).toBe(403);
}
expect(await registry.getById(row.id)).toEqual(original);
});
it("completes the real worker initialize and health protocol from a relocated image", async () => {
const old = fixture(); const image = fixture();
await loaderFor(old).installPlugin({ localPath: old.packageRoot });
const registry = pluginRegistryService(db);
const row = (await registry.getByKey(old.manifest.id))!;
await registry.updateStatus(row.id, { status: "ready" });
rmSync(old.root, { recursive: true });
const manager = createPluginWorkerManager();
try {
const services = { ...runtime().services, workerManager: manager } as unknown as PluginRuntimeServices;
const result = await loaderFor(image, db, services).loadSingle(row.id);
expect(result.success).toBe(true);
expect(await manager.call(row.id, "health", {})).toMatchObject({ status: "ok", details: { source: expect.stringContaining(image.packageRoot), configs: {} } });
} finally { await manager.stopAll(); }
});
it("qualifies and initializes the current release SDK/shared exports through the baked loader", async () => {
const image = fixture();
const catalogRoot = fileURLToPath(new URL("../../../packages/plugins", import.meta.url));
const provider = mkdtempSync(path.join(catalogRoot, "sandbox-providers", "image-sdk-proof-"));
roots.push(provider);
cpSync(image.packageRoot, provider, { recursive: true });
linkSdkInto(provider);
writeFileSync(path.join(provider, "dist", "worker.js"), 'import { definePlugin, runWorker } from "@paperclipai/plugin-sdk"; runWorker(definePlugin({ async setup() {}, async onHealth() { return { status: "ok", message: "Current image SDK/shared loaded" }; } }), import.meta.url);');
const installs = [{ key: "sdk-proof", pluginKey: image.manifest.id, localPath: provider }];
const resolver = createBundledPluginPackageResolver(installs);
const manager = createPluginWorkerManager();
const loader = pluginLoader(db, {
prebuiltOnly: true, resolvePackageRoot: resolver.resolve, assertPackageActivation: resolver.assertActivation,
enableLocalFilesystem: false, enableNpmDiscovery: false,
}, { ...runtime().services, workerManager: manager } as unknown as PluginRuntimeServices);
const registry = pluginRegistryService(db);
try {
await loader.installPlugin({ localPath: provider });
const installed = (await registry.getByKey(image.manifest.id))!;
await registry.updateStatus(installed.id, { status: "ready" });
const registration = (await registry.getById(installed.id))!;
expect((await inspectCloudPackages({ installs, catalogRoot, registrations: [registration], loadManifest: (root) => loader.loadManifest(root) })).blockers).toEqual([]);
expect((await loader.loadSingle(installed.id)).success).toBe(true);
expect(await manager.call(installed.id, "health", {})).toMatchObject({ status: "ok", message: "Current image SDK/shared loaded" });
} finally { await manager.stopAll(); }
}, 90_000);
it.skipIf(!existsSync(fileURLToPath(new URL("../../../packages/plugins/sandbox-providers/daytona/dist/worker.js", import.meta.url))))("qualifies and initializes the actual prebuilt Daytona release package", async () => {
const catalogRoot = fileURLToPath(new URL("../../../packages/plugins", import.meta.url));
const installs = resolveBundledPluginInstalls(["daytona"], { catalogRoot, env: {}, enforceCatalogRoot: true, distributionPlugins: [] });
const resolver = createBundledPluginPackageResolver(installs);
const manager = createPluginWorkerManager();
const services = { ...runtime().services, workerManager: manager } as unknown as PluginRuntimeServices;
const loader = pluginLoader(db, {
localPluginDir: "/lost/release/home/plugins", prebuiltOnly: true,
resolvePackageRoot: resolver.resolve, assertPackageActivation: resolver.assertActivation,
enableLocalFilesystem: false, enableNpmDiscovery: false,
}, services);
const registry = pluginRegistryService(db);
vi.stubEnv("DAYTONA_API_KEY", "");
try {
await loader.installPlugin({ localPath: installs[0]!.localPath });
const installed = (await registry.getByKey(daytonaManifest.id))!;
await registry.update(installed.id, { packagePath: "/lost/release/home/daytona" });
await registry.updateStatus(installed.id, { status: "ready" });
const registration = (await registry.getById(installed.id))!;
const inspected = await inspectCloudPackages({ installs, catalogRoot, registrations: [registration], loadManifest: (root) => loader.loadManifest(root) });
expect(inspected.blockers).toEqual([]);
expect(inspected.packages[0]?.packageDigest).toMatch(/^sha256:/);
expect((await loader.loadSingle(installed.id)).success).toBe(true);
expect(await manager.call(installed.id, "health", {})).toMatchObject({ status: "ok", message: "Daytona sandbox provider plugin healthy" });
} finally { await manager.stopAll(); vi.unstubAllEnvs(); }
}, 90_000);
it("registers the same package concurrently from two roots without replacing the winner", async () => {
const first = fixture(); const second = fixture();
await Promise.all([
loaderFor(first).installPlugin({ localPath: first.packageRoot }),
loaderFor(second, peer).installPlugin({ localPath: second.packageRoot }),
]);
const rows = await pluginRegistryService(db).listInstalled();
expect(rows).toHaveLength(1);
const original = rows[0]!;
expect([first.packageRoot, second.packageRoot]).toContain(original.packagePath);
await loaderFor(second, peer).installPlugin({ localPath: second.packageRoot });
expect(await pluginRegistryService(db).getById(original.id)).toEqual(original);
await expect(pluginRegistryService(peer).install({ packageName: original.packageName }, { ...first.manifest, description: "different package metadata" }, { idempotent: true })).rejects.toThrow(/already installed/);
});
it("preserves disabled registrations and avoids path writes during managed boot", async () => {
const old = fixture(); const image = fixture();
await loaderFor(old).installPlugin({ localPath: old.packageRoot });
const registry = pluginRegistryService(db);
const row = (await registry.getByKey(old.manifest.id))!;
await db.update(plugins).set({ status: "disabled" }).where(eq(plugins.id, row.id));
const original = await registry.getById(row.id);
await ensureBundledPlugins(image.installs, { registry, loader: loaderFor(image), lifecycle: { load: vi.fn() }, logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() } }, { reinstallUninstalled: true, preservePackagePath: true });
expect(await registry.getById(row.id)).toEqual(original);
});
it("fails before worker startup for missing image bytes without using a home fallback or building", async () => {
const image = fixture();
await loaderFor(image).installPlugin({ localPath: image.packageRoot });
const registry = pluginRegistryService(db);
const row = (await registry.getByKey(image.manifest.id))!;
await registry.updateStatus(row.id, { status: "ready" });
const fallback = path.join(image.root, "lost-home", "node_modules", row.packageName);
mkdirSync(fallback, { recursive: true });
writeFileSync(path.join(fallback, "package.json"), JSON.stringify({ name: row.packageName }));
writeFileSync(path.join(fallback, "worker.js"), "throw new Error('must not start');");
rmSync(path.join(image.packageRoot, "dist", "worker.js"));
const current = runtime();
const loader = loaderFor(image, db, current.services as unknown as PluginRuntimeServices);
const result = await loader.loadSingle(row.id);
expect(result.success).toBe(false);
expect(result.error).toContain("ENOENT");
expect(current.services.workerManager.startWorker).not.toHaveBeenCalled();
await expect(loader.installPlugin({ packageName: row.packageName })).rejects.toThrow(/current image catalog/);
await loader.cleanupInstallArtifacts(row as PluginRecord);
expect(image.resolver.inspect(row, image.manifest).compatible).toBe(false);
});
it("applies plugin migrations from the current image after the legacy package disappears", async () => {
const image = bundledImageFixture({
capabilities: ["database.namespace.migrate", "database.namespace.read", "database.namespace.write"],
database: { migrationsDir: "migrations" },
});
roots.push(image.root);
const [row] = await db.insert(plugins).values({
...image.registration, packagePath: "/lost/legacy/home/plugin", apiVersion: 1, categories: image.manifest.categories, status: "ready",
}).returning();
const current = runtime();
const result = await loaderFor(image, db, current.services as unknown as PluginRuntimeServices).loadSingle(row!.id);
expect(result.success).toBe(true);
expect(current.services.workerManager.startWorker).toHaveBeenCalledWith(row!.id, expect.objectContaining({ databaseNamespace: derivePluginDatabaseNamespace(image.manifest.id) }));
await db.execute((await import("drizzle-orm")).sql.raw(`DROP SCHEMA "${derivePluginDatabaseNamespace(image.manifest.id)}" CASCADE`));
});
it("retains registry-install provider credentials for an ordinary npm package", async () => {
const npmManifest = { ...daytonaManifest, id: "paperclip.image-npm-fixture" };
const image = bundledImageFixture(npmManifest);
roots.push(image.root);
const localPluginDir = path.join(image.root, "npm-home");
const packageRoot = path.join(localPluginDir, "node_modules", "@paperclipai", "plugin-daytona");
mkdirSync(path.dirname(packageRoot), { recursive: true });
cpSync(image.packageRoot, packageRoot, { recursive: true });
const metadata = JSON.parse(readFileSync(path.join(packageRoot, "package.json"), "utf8"));
metadata.name = "@paperclipai/plugin-daytona";
writeFileSync(path.join(packageRoot, "package.json"), JSON.stringify(metadata));
const registry = pluginRegistryService(db);
const row = (await registry.install({ packageName: metadata.name }, npmManifest))!;
await registry.updateStatus(row.id, { status: "ready" });
vi.stubEnv("DAYTONA_API_KEY", "synthetic-provider-key");
try {
const current = runtime();
const loader = pluginLoader(db, { localPluginDir, enableNpmDiscovery: false, enableLocalFilesystem: false }, current.services as unknown as PluginRuntimeServices);
expect((await loader.loadSingle(row.id)).success).toBe(true);
expect(current.services.workerManager.startWorker).toHaveBeenCalledWith(row.id, expect.objectContaining({ env: expect.objectContaining({ DAYTONA_API_KEY: "synthetic-provider-key" }) }));
} finally { vi.unstubAllEnvs(); }
});
it("keeps self-hosted local-path loading available without a managed resolver", async () => {
const image = fixture();
const loader = pluginLoader(db, { localPluginDir: path.join(image.root, "selfhost"), enableNpmDiscovery: false, enableLocalFilesystem: false });
await loader.installPlugin({ localPath: image.packageRoot });
const row = (await pluginRegistryService(db).getByKey(image.manifest.id))!;
expect(row.packagePath).toBe(image.packageRoot);
await expect(loader.installPlugin({ localPath: image.packageRoot })).rejects.toThrow(/already installed/);
});
});
@@ -0,0 +1,72 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { execFileSync } from "node:child_process";
import { fileURLToPath } from "node:url";
import { afterEach, describe, expect, it } from "vitest";
import { build, transform } from "esbuild";
import { bundleRuntime } from "../../scripts/bundle-runtime.mjs";
import { resolveBundledPluginRuntimeLoader } from "../services/bundled-plugin-runtime.js";
const roots: string[] = [];
function fixture() {
const root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), "image-plugin-runtime-")));
roots.push(root);
fs.mkdirSync(path.join(root, "server", "dist-runtime"), { recursive: true });
fs.writeFileSync(path.join(root, "server", "package.json"), JSON.stringify({ name: "@paperclipai/server", type: "module" }));
const loader = path.join(root, "cli", "node_modules", "tsx", "dist", "loader.mjs");
fs.mkdirSync(path.dirname(loader), { recursive: true });
fs.writeFileSync(loader, "export {};\n");
return { root, loader };
}
afterEach(() => { for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); });
describe("current image plugin runtime loader", () => {
it.each(["src/services", "dist/services", "dist-runtime"])("resolves the owning release from server/%s", (location) => {
const { root, loader } = fixture();
const moduleDirectory = path.join(root, "server", location);
fs.mkdirSync(moduleDirectory, { recursive: true });
expect(resolveBundledPluginRuntimeLoader(moduleDirectory)).toBe(loader);
});
it("resolves a bundled entrypoint without preserved source module locations", async () => {
const { root, loader } = fixture();
const runtime = path.join(root, "server", "dist-runtime", "index.mjs");
await build({
stdin: { contents: `import { resolveBundledPluginRuntimeLoader } from ${JSON.stringify(fileURLToPath(new URL("../services/bundled-plugin-runtime.ts", import.meta.url)))}; process.stdout.write(resolveBundledPluginRuntimeLoader());`, resolveDir: process.cwd() },
outfile: runtime, bundle: true, platform: "node", format: "esm", target: "node24", logLevel: "silent",
});
expect(execFileSync(process.execPath, [runtime], { encoding: "utf8" })).toBe(loader);
});
it("uses the production bundler's preserved module location at the dist-runtime entrypoint", async () => {
const { root, loader } = fixture();
const service = path.join(root, "server", "dist", "services", "bundled-plugin-runtime.js");
fs.mkdirSync(path.dirname(service), { recursive: true });
const source = fs.readFileSync(new URL("../services/bundled-plugin-runtime.ts", import.meta.url), "utf8");
fs.writeFileSync(service, (await transform(source, { loader: "ts", format: "esm", target: "node24" })).code);
const entry = path.join(root, "server", "dist", "probe.js");
fs.writeFileSync(entry, 'import { resolveBundledPluginRuntimeLoader } from "./services/bundled-plugin-runtime.js"; process.stdout.write(resolveBundledPluginRuntimeLoader());');
const bundled = await bundleRuntime({ entry, outdir: path.join(root, "server", "dist-runtime") });
expect(execFileSync(process.execPath, [bundled.outfile], { encoding: "utf8" })).toBe(loader);
});
it("accepts the baked pnpm target inside the current image node_modules", () => {
const { root, loader } = fixture();
const physical = path.join(root, "node_modules", ".pnpm", "tsx@1", "node_modules", "tsx", "dist", "loader.mjs");
fs.mkdirSync(path.dirname(physical), { recursive: true });
fs.writeFileSync(physical, "export {};\n");
fs.rmSync(loader); fs.symlinkSync(path.relative(path.dirname(loader), physical), loader);
expect(resolveBundledPluginRuntimeLoader(path.join(root, "server", "dist-runtime"))).toBe(physical);
});
it("fails explicitly for missing or legacy-home loaders", () => {
const { root, loader } = fixture();
const moduleDirectory = path.join(root, "server", "dist-runtime");
fs.rmSync(loader);
expect(() => resolveBundledPluginRuntimeLoader(moduleDirectory)).toThrow("missing from the current image");
const outside = fixture().loader;
fs.symlinkSync(outside, loader);
expect(() => resolveBundledPluginRuntimeLoader(moduleDirectory)).toThrow("escapes the current image");
});
});
@@ -9,6 +9,7 @@ import {
ensureBundledPlugins,
resolveBundledCatalogRoot,
resolveBundledPluginInstalls,
resolveManagedBundledPluginKeys,
type BundledPluginProvisionerDeps,
type ResolvedBundledPlugin,
} from "../services/bundled-plugins.js";
@@ -30,6 +31,21 @@ afterAll(() => {
// Catalog-root resolution (fail-to-start allowlist)
// ---------------------------------------------------------------------------
describe("managed bundled plugin selection", () => {
it.each([
{ env: {}, selected: undefined, expected: null },
{ env: { PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN: "tenant-fixture" }, selected: undefined, expected: [] },
{ env: { PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN: " " }, selected: undefined, expected: null },
{ env: { PAPERCLIP_MANAGED_CONFIG: "" }, selected: undefined, expected: [] },
{ env: { PAPERCLIP_MANAGED_CONFIG: "{}" }, selected: null, expected: [] },
{ env: { PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN: "tenant-fixture" }, selected: ["daytona"], expected: ["daytona"] },
{ env: {}, selected: ["modal"], expected: ["modal"] },
{ env: {}, selected: [], expected: [] },
])("uses the canonical Cloud floor and preserves explicit selection: $env / $selected", ({ env, selected, expected }) => {
expect(resolveManagedBundledPluginKeys(selected, env)).toEqual(expected);
});
});
describe("resolveBundledPluginInstalls", () => {
it("resolves the optional CreateOS sandbox provider without making it a self-hosted default", () => {
expect(resolveBundledPluginInstalls(["createos"], {
@@ -0,0 +1,93 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { cloudBundledPackageDigest } from "../services/cloud-bundled-package-digest.js";
const roots: string[] = [];
function file(name: string, content = "export const value = 1;") {
fs.mkdirSync(path.dirname(name), { recursive: true }); fs.writeFileSync(name, content);
}
function link(target: string, name: string) {
fs.mkdirSync(path.dirname(name), { recursive: true }); fs.symlinkSync(path.relative(path.dirname(name), target), name);
}
function fixture() {
const root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), "image-package-graph-"))); roots.push(root);
const catalogRoot = path.join(root, "packages", "plugins");
const provider = path.join(catalogRoot, "sandbox-providers", "daytona");
const sdk = path.join(catalogRoot, "sdk");
const shared = path.join(root, "packages", "shared");
const dependency = path.join(root, "node_modules", ".pnpm", "dependency@1", "node_modules", "dependency");
const transitive = path.join(root, "node_modules", ".pnpm", "transitive@1", "node_modules", "transitive");
for (const directory of [provider, sdk, shared, dependency, transitive]) file(path.join(directory, "index.js"));
link(sdk, path.join(provider, "node_modules", "@paperclipai", "plugin-sdk"));
link(shared, path.join(sdk, "node_modules", "@paperclipai", "shared"));
link(dependency, path.join(shared, "node_modules", "dependency"));
link(transitive, path.join(path.dirname(dependency), "transitive"));
return { root, catalogRoot, provider, sdk, shared, dependency, transitive };
}
afterEach(() => { for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); });
describe("bounded current-image package dependency inventory", () => {
it("hashes external SDK, shared, and pnpm sibling dependency bytes", () => {
const image = fixture();
const digest = () => cloudBundledPackageDigest(image.provider, { catalogRoot: image.catalogRoot });
let previous = digest();
for (const directory of [image.sdk, image.shared, image.dependency, image.transitive]) {
file(path.join(directory, "index.js"), "export const value = 2;");
const next = digest(); expect(next).not.toBe(previous); previous = next;
}
});
it("keeps the digest stable when an identical release moves to a new root", () => {
const original = fixture(); const replacement = fixture();
expect(cloudBundledPackageDigest(original.provider, { catalogRoot: original.catalogRoot }))
.toBe(cloudBundledPackageDigest(replacement.provider, { catalogRoot: replacement.catalogRoot }));
});
it("rejects legacy-home links and preserves package-contained defaults", () => {
const image = fixture();
expect(() => cloudBundledPackageDigest(image.provider)).toThrow("escapes its package");
const outside = fixture();
link(path.join(outside.sdk, "index.js"), path.join(image.shared, "legacy-home.js"));
expect(() => cloudBundledPackageDigest(image.provider, { catalogRoot: image.catalogRoot })).toThrow("authorized image roots");
});
it("deduplicates cycles without scanning unrelated image node_modules", () => {
const image = fixture();
link(image.sdk, path.join(image.shared, "node_modules", "cycle"));
const unrelated = path.join(image.root, "node_modules", "unrelated", "special");
file(unrelated);
const first = cloudBundledPackageDigest(image.provider, { catalogRoot: image.catalogRoot });
file(unrelated, "changed unrelated code");
expect(cloudBundledPackageDigest(image.provider, { catalogRoot: image.catalogRoot })).toBe(first);
});
it.each(["", ".pnpm", ".pnpm/node_modules"])("rejects aggregate image dependency targets: %s", (aggregate) => {
const image = fixture();
const target = path.join(image.root, "node_modules", aggregate);
fs.mkdirSync(target, { recursive: true });
link(target, path.join(image.sdk, "node_modules", "aggregate"));
expect(() => cloudBundledPackageDigest(image.provider, { catalogRoot: image.catalogRoot })).toThrow("select an individual package");
});
it("accepts the shipped provider's observed entry class", () => {
const image = fixture();
const directory = path.join(image.provider, "many"); fs.mkdirSync(directory);
for (let index = 0; index < 16_000; index++) fs.writeFileSync(path.join(directory, `${index}.js`), "");
expect(cloudBundledPackageDigest(image.provider, { catalogRoot: image.catalogRoot })).toMatch(/^sha256:/);
});
it("rejects oversized metadata and excessive depth", () => {
const image = fixture();
file(path.join(image.sdk, "package.json"), " ".repeat(128 * 1024 + 1));
expect(() => cloudBundledPackageDigest(image.provider, { catalogRoot: image.catalogRoot })).toThrow("metadata");
fs.rmSync(path.join(image.sdk, "package.json"));
const large = path.join(image.shared, "large.js");
fs.closeSync(fs.openSync(large, "w")); fs.truncateSync(large, 256 * 1024 * 1024 + 1);
expect(() => cloudBundledPackageDigest(image.provider, { catalogRoot: image.catalogRoot })).toThrow("byte limit");
fs.rmSync(large);
file(path.join(image.sdk, ...Array.from({ length: 33 }, () => "deep"), "file.js"));
expect(() => cloudBundledPackageDigest(image.provider, { catalogRoot: image.catalogRoot })).toThrow("depth limit");
});
});
@@ -0,0 +1,325 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { applyPendingMigrations, closeRegisteredClients, createDb, instanceSettings, plugins, type Db } from "@paperclipai/db";
import { eq, inArray, sql } from "drizzle-orm";
import { paperclipConfigSchema } from "@paperclipai/shared";
import { adapterSettingsService, initializeAdapterSettings } from "../services/adapter-settings.js";
import { cloudRuntimeConfigService, initializeCloudRuntimeConfig } from "../services/cloud-runtime-config.js";
import { cloudConfigurationMigrationService } from "../services/cloud-configuration-migration.js";
import { qualifyCloudConfigurationForCurrentImage } from "../services/cloud-configuration-qualification.js";
import { runCloudConfigurationQualification } from "../qualify-cloud-configuration.js";
import { pluginLoader, type PluginRuntimeServices } from "../services/plugin-loader.js";
import { pluginRegistryService } from "../services/plugin-registry.js";
import { createPluginWorkerManager } from "../services/plugin-worker-manager.js";
import { ensureBundledPlugins } from "../services/bundled-plugins.js";
import { bundledImageFixture } from "./helpers/bundled-image-fixture.js";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
const externalDatabaseUrl = process.env.PAPERCLIP_CLOUD_CONFIGURATION_TEST_DATABASE_URL?.trim();
const support = externalDatabaseUrl ? { supported: true } : await getEmbeddedPostgresTestSupport();
const keys = ["adapter-settings:v1", "cloud-runtime-config:v1", "cloud-configuration-migration:v1"];
(support.supported ? describe : describe.skip)("Cloud configuration migration and current-image qualification", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: Db;
let peer: Db;
let homeA: string;
let homeB: string;
const roots: string[] = [];
const fixture = (version = "1.0.0") => { const image = bundledImageFixture({ version }); roots.push(image.root); return image; };
const env = (home = homeA) => ({ PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN: "test-cloud-token", PAPERCLIP_HOME: home, PAPERCLIP_CONFIG: path.join(home, "config.json") });
const migration = (client = db, home = homeA) => cloudConfigurationMigrationService(client, { runtimeEnv: env(home) });
beforeAll(async () => {
if (externalDatabaseUrl) {
await applyPendingMigrations(externalDatabaseUrl);
database = { connectionString: externalDatabaseUrl, cleanup: () => closeRegisteredClients(externalDatabaseUrl) };
} else database = await startEmbeddedPostgresTestDatabase("cloud-configuration-migration-");
db = createDb(database.connectionString);
peer = createDb(database.connectionString);
}, 90_000);
beforeEach(async () => {
homeA = await fs.mkdtemp(path.join(os.tmpdir(), "cloud-legacy-A-"));
homeB = await fs.mkdtemp(path.join(os.tmpdir(), "cloud-replacement-B-"));
roots.push(homeA, homeB);
await db.delete(instanceSettings).where(inArray(instanceSettings.singletonKey, keys));
await db.delete(plugins);
});
afterEach(async () => {
vi.restoreAllMocks();
for (const root of roots.splice(0)) await fs.rm(root, { recursive: true, force: true });
});
afterAll(async () => { await database?.cleanup(); });
async function legacy() {
const config = paperclipConfigSchema.parse({
$meta: { version: 1, updatedAt: "2026-10-30T00:00:00.000Z", source: "configure" },
database: { mode: "postgres" }, logging: { mode: "cloud" },
server: { port: 3199 }, telemetry: { enabled: false },
});
await fs.writeFile(path.join(homeA, "config.json"), JSON.stringify(config));
await fs.writeFile(path.join(homeA, "adapter-settings.json"), JSON.stringify({ disabledTypes: ["claude_local", "future_adapter"] }));
await fs.writeFile(path.join(homeA, "adapter-plugins.json"), "[]");
return config;
}
function loaderFor(image: ReturnType<typeof fixture>, client = db, services?: PluginRuntimeServices) {
return pluginLoader(client, {
localPluginDir: path.join(homeB, "plugins"), prebuiltOnly: true,
resolvePackageRoot: image.resolver.resolve, assertPackageActivation: image.resolver.assertActivation,
enableLocalFilesystem: false, enableNpmDiscovery: false,
}, services);
}
async function install(image: ReturnType<typeof fixture>) {
await loaderFor(image).installPlugin({ localPath: image.packageRoot });
const registry = pluginRegistryService(db);
const row = (await registry.getByKey(image.manifest.id))!;
await registry.updateStatus(row.id, { status: "ready" });
return row;
}
function qualify(image: ReturnType<typeof fixture>, client = db, home = homeA, expectedPackageEvidenceDigest?: string) {
const loader = loaderFor(image, client);
return migration(client, home).qualify({ installs: image.installs, loadManifest: (root) => loader.loadManifest(root), expectedPackageEvidenceDigest });
}
it("imports A atomically, retains later DB revisions, and starts B's real image worker after A disappears", async () => {
await legacy();
const oldImage = fixture();
const newImage = fixture();
const row = await install(oldImage);
const first = await migration().migrate();
expect(first.migration.adapter.authority).toBe("legacy_file");
expect(first.migration.config.authority).toBe("legacy_file");
const original = await qualify(oldImage);
expect(original.configurationPackagesEligible).toBe(true);
expect(original.samePackageOverlapEligible).toBe(false);
await adapterSettingsService(peer, { runtimeEnv: env() }).setDisabled("codex_local", true);
const config = (await cloudRuntimeConfigService(peer).inspectStored())!;
expect(config.config).not.toBeNull();
await cloudRuntimeConfigService(peer).replace({ ...config.config!, server: { ...config.config!.server, port: 3299 } }, config.revision);
await fs.rm(homeA, { recursive: true });
await fs.rm(oldImage.root, { recursive: true });
expect((await migration(peer, homeB).migrate()).migration).toEqual(first.migration);
await initializeAdapterSettings(peer, { runtimeEnv: env(homeB) });
const restored = await initializeCloudRuntimeConfig(peer, { runtimeEnv: env(homeB) });
expect(restored?.current.config?.server.port).toBe(3299);
expect(await adapterSettingsService(peer, { runtimeEnv: env(homeB) }).getDisabledTypes()).toEqual(["claude_local", "codex_local", "future_adapter"]);
const replacement = await qualify(newImage, peer, homeB, original.packageEvidenceDigest);
expect(replacement).toMatchObject({ adapterRevision: 1, configRevision: 1, configurationPackagesEligible: true, samePackageOverlapEligible: true });
expect(replacement.configurationDigest).not.toBe(original.configurationDigest);
const manager = createPluginWorkerManager();
try {
const services = {
workerManager: manager, lifecycleManager: { markError: vi.fn() },
jobStore: { syncJobDeclarations: vi.fn() }, jobScheduler: { registerPlugin: vi.fn() },
toolDispatcher: {}, eventBus: { forPlugin: vi.fn(), subscriptionCount: vi.fn(() => 0) },
buildHostHandlers: vi.fn(() => ({})), instanceInfo: { instanceId: "qualification-fixture", hostVersion: "1.0.0" },
} as unknown as PluginRuntimeServices;
expect((await loaderFor(newImage, peer, services).loadSingle(row.id)).success).toBe(true);
expect(await manager.call(row.id, "health", {})).toMatchObject({ status: "ok", details: { source: expect.stringContaining(newImage.packageRoot) } });
} finally { await manager.stopAll(); }
});
it("serializes concurrent attempts and never imports stale inputs over later DB changes", async () => {
await legacy();
const [first, second] = await Promise.all([migration().migrate(), migration(peer).migrate()]);
expect(first).toEqual(second);
await adapterSettingsService(peer, { runtimeEnv: env() }).setDisabled("claude_local", false);
await fs.writeFile(path.join(homeA, "config.json"), "invalid stale file");
await fs.writeFile(path.join(homeA, "adapter-settings.json"), "invalid stale file");
expect(await migration().migrate()).toEqual(first);
expect(await adapterSettingsService(db, { runtimeEnv: env() }).getDisabledTypes()).toEqual(["future_adapter"]);
});
it("rolls back both seeds and the receipt if completion fails, then retries after the crash boundary", async () => {
await legacy();
await db.execute(sql`CREATE FUNCTION reject_cloud_migration_receipt() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN IF NEW.singleton_key = 'cloud-configuration-migration:v1' THEN RAISE EXCEPTION 'test receipt failure'; END IF; RETURN NEW; END $$`);
await db.execute(sql`CREATE TRIGGER reject_cloud_migration_receipt BEFORE UPDATE ON instance_settings FOR EACH ROW EXECUTE FUNCTION reject_cloud_migration_receipt()`);
try {
await expect(migration().migrate()).rejects.toMatchObject({ cause: { message: expect.stringContaining("test receipt failure") } });
expect(await peer.select().from(instanceSettings).where(inArray(instanceSettings.singletonKey, keys))).toEqual([]);
} finally {
await db.execute(sql`DROP TRIGGER reject_cloud_migration_receipt ON instance_settings`);
await db.execute(sql`DROP FUNCTION reject_cloud_migration_receipt()`);
}
expect((await migration(peer).migrate()).revision).toBe(0);
expect((await peer.select().from(instanceSettings).where(inArray(instanceSettings.singletonKey, keys))).length).toBe(3);
});
it("fails without partial authority for malformed legacy config and preserves already-authoritative rows", async () => {
await legacy();
await fs.writeFile(path.join(homeA, "config.json"), "invalid");
await expect(migration().migrate()).rejects.toThrow("Legacy Cloud runtime configuration");
expect(await peer.select().from(instanceSettings).where(inArray(instanceSettings.singletonKey, keys))).toEqual([]);
await adapterSettingsService(db, { runtimeEnv: env() }).seedIfAbsent({ disabledTypes: ["codex_local"] });
await cloudRuntimeConfigService(db).seedIfAbsent(null);
await fs.writeFile(path.join(homeA, "adapter-settings.json"), "invalid");
const receipt = await migration(peer).migrate();
expect(receipt.migration.adapter.authority).toBe("database");
expect(receipt.migration.config.authority).toBe("database");
expect((await cloudRuntimeConfigService(peer).inspectStored())?.config).toBeNull();
});
it("returns database errors instead of falling back to files or a completed receipt", async () => {
await legacy();
await migration().migrate();
const broken = createDb(database.connectionString);
await broken.$client.end({ timeout: 0 });
await expect(migration(broken).migrate()).rejects.toThrow();
const image = fixture();
await expect(qualify(image, broken)).rejects.toThrow();
});
it("keeps external registration blockers durable when B has no legacy home and requires deliberate original-source resolution", async () => {
await legacy();
const sentinel = path.join(homeA, "external-code-executed");
await fs.writeFile(path.join(homeA, "external.js"), `require('node:fs').writeFileSync(${JSON.stringify(sentinel)}, 'executed');`);
await fs.writeFile(path.join(homeA, "adapter-plugins.json"), JSON.stringify([{ type: "legacy_external", packageName: "legacy-adapter", localPath: path.join(homeA, "external.js") }]));
const image = fixture();
await install(image);
await migration().migrate();
expect((await qualify(image)).blockers).toContainEqual(expect.objectContaining({ code: "unsupported_external_adapters" }));
await expect(fs.stat(sentinel)).rejects.toMatchObject({ code: "ENOENT" });
await migration(peer, homeB).migrate();
const replacement = await qualify(image, peer, homeB);
expect(replacement.configurationPackagesEligible).toBe(false);
await expect(migration(peer, homeB).recheckLegacyExternalRegistrations(replacement.receiptRevision)).rejects.toThrow("original legacy source");
await fs.unlink(path.join(homeA, "adapter-plugins.json"));
await expect(migration().recheckLegacyExternalRegistrations(replacement.receiptRevision)).rejects.toThrow("present, valid registration file");
expect((await migration().inspectStored())?.migration.externalRegistrations.blocker).toBe("unsupported_external_adapters");
await fs.writeFile(path.join(homeA, "adapter-plugins.json"), "[]");
await expect(migration().recheckLegacyExternalRegistrations(0)).rejects.toThrow("revision conflict");
await migration().recheckLegacyExternalRegistrations(replacement.receiptRevision);
expect((await qualify(image)).configurationPackagesEligible).toBe(true);
});
it("retains a metadata blocker for malformed or oversized registration files", async () => {
await legacy();
await fs.writeFile(path.join(homeA, "adapter-plugins.json"), Buffer.alloc(65_537));
const receipt = await migration().migrate();
expect(receipt.migration.externalRegistrations.blocker).toBe("invalid_external_registration_metadata");
await fs.rm(homeA, { recursive: true });
expect((await migration(peer, homeB).migrate()).migration.externalRegistrations).toEqual(receipt.migration.externalRegistrations);
});
it("records image mismatches before reconciliation, permits maintenance upgrades, and requires explicit fresh overlap evidence", async () => {
await legacy();
const oldImage = fixture();
const nextImage = fixture("1.1.0");
await install(oldImage);
await migration().migrate();
const old = await qualify(oldImage);
const mismatch = await qualify(nextImage, peer, homeB, old.packageEvidenceDigest);
expect(mismatch.configurationPackagesEligible).toBe(false);
expect(mismatch.samePackageOverlapEligible).toBe(false);
expect(mismatch.blockers).toContainEqual(expect.objectContaining({ code: "incompatible_bundled_package", registeredVersion: "1.0.0", imageVersion: "1.1.0" }));
const registry = pluginRegistryService(db);
await ensureBundledPlugins(nextImage.installs, {
registry, loader: loaderFor(nextImage), lifecycle: { load: vi.fn() },
logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() },
}, { reinstallUninstalled: true, preservePackagePath: true });
const current = await qualify(nextImage);
expect(current.configurationPackagesEligible).toBe(true);
expect(current.samePackageOverlapEligible).toBe(false);
expect((await migration().inspectStored())?.lastPackageMismatch?.packageEvidenceDigest).toBe(mismatch.packageEvidenceDigest);
expect((await qualify(nextImage, peer, homeB, current.packageEvidenceDigest)).samePackageOverlapEligible).toBe(true);
});
it("fails qualification for missing package or manifest bytes without loading a home fallback", async () => {
await legacy();
const image = fixture();
await install(image);
await migration().migrate();
const good = await qualify(image);
await fs.rm(path.join(image.packageRoot, "dist", "manifest.js"));
const broken = await qualify(image, peer, homeB, good.packageEvidenceDigest);
expect(broken.configurationPackagesEligible).toBe(false);
expect(broken.samePackageOverlapEligible).toBe(false);
expect(broken.blockers.length).toBeGreaterThan(0);
await fs.rm(image.packageRoot, { recursive: true });
expect((await qualify(image)).blockers).toContainEqual(expect.objectContaining({ code: "missing_bundled_package" }));
});
it("rejects changed image bytes before manifest import and never uses stale completed qualification", async () => {
await legacy();
const image = fixture();
await install(image);
await migration().migrate();
const original = await qualify(image);
await fs.appendFile(path.join(image.packageRoot, "dist", "worker.js"), "\nprocess.exit(99);\n");
const loadManifest = vi.fn();
const current = await migration(peer, homeB).qualify({ installs: image.installs, loadManifest, expectedPackageEvidenceDigest: original.packageEvidenceDigest });
expect(current.configurationPackagesEligible).toBe(false);
expect(current.samePackageOverlapEligible).toBe(false);
expect(current.blockers).toContainEqual(expect.objectContaining({ code: "package_integrity_mismatch" }));
expect(loadManifest).not.toHaveBeenCalled();
});
it("allows an administrator update while manifest inspection waits and refuses to certify the stale snapshot", async () => {
await legacy();
const image = fixture();
await install(image);
await migration().migrate();
let started!: () => void;
let resume!: () => void;
const entered = new Promise<void>((resolve) => { started = resolve; });
const release = new Promise<void>((resolve) => { resume = resolve; });
const loader = loaderFor(image);
const pending = migration().qualify({ installs: image.installs, loadManifest: async (root) => { started(); await release; return loader.loadManifest(root); } });
await entered;
try {
const change = adapterSettingsService(peer, { runtimeEnv: env() }).setDisabled("codex_local", true);
let timer: ReturnType<typeof setTimeout> | undefined;
try {
expect(await Promise.race([change, new Promise<never>((_resolve, reject) => { timer = setTimeout(() => reject(new Error("Admin update was blocked by image inspection")), 2_000); })])).toBe(true);
} finally { clearTimeout(timer); }
} finally { resume(); }
const result = await pending;
expect(result).toMatchObject({ configurationPackagesEligible: false, adapterRevision: 1 });
expect(result.blockers).toContainEqual(expect.objectContaining({ code: "configuration_changed_during_inspection" }));
expect((await qualify(image)).configurationPackagesEligible).toBe(true);
});
it("uses the current managed catalog through the operator command and durably records catalog failure", async () => {
await legacy();
const image = fixture();
await install(image);
await migration().migrate();
const runtimeEnv = {
...env(homeB), DATABASE_URL: database.connectionString, PAPERCLIP_BUNDLED_PLUGIN_ROOT: image.root,
PAPERCLIP_MANAGED_CONFIG: JSON.stringify({ v: 1, mode: "cloud", catalogVersion: "fixture", features: {}, plugins: { autoInstall: ["fixture"] } }),
};
const initial = await qualifyCloudConfigurationForCurrentImage(db, { runtimeEnv });
expect(initial?.configurationPackagesEligible).toBe(true);
const stdout = vi.spyOn(process.stdout, "write").mockImplementation(() => true);
try {
expect(await runCloudConfigurationQualification(["--expected-package-evidence", initial!.packageEvidenceDigest], runtimeEnv)).toBe(0);
const report = stdout.mock.calls.map((call) => String(call[0])).find((text) => text.startsWith('{\n "receiptRevision"'));
expect(report && JSON.parse(report).samePackageOverlapEligible).toBe(true);
} finally { stdout.mockRestore(); }
const missing = await qualifyCloudConfigurationForCurrentImage(peer, { runtimeEnv: { ...runtimeEnv, PAPERCLIP_MANAGED_CONFIG: runtimeEnv.PAPERCLIP_MANAGED_CONFIG.replace('"fixture"]', '"absent"]') } });
expect(missing?.configurationPackagesEligible).toBe(false);
expect(missing?.blockers).toContainEqual(expect.objectContaining({ code: "current_image_catalog_unavailable" }));
expect((await migration().inspectStored())?.lastPackageMismatch?.blockers).toContainEqual(expect.objectContaining({ code: "current_image_catalog_unavailable" }));
});
it.skipIf(process.platform === "win32")("detects changed pnpm dependency bytes in a builtin package overlap comparison", async () => {
await legacy();
const image = fixture();
await install(image);
await migration().migrate();
const dependency = path.join(image.packageRoot, "node_modules", ".pnpm", "dep@1.0.0", "node_modules", "dep");
await fs.mkdir(dependency, { recursive: true });
await fs.writeFile(path.join(dependency, "index.js"), "export const value = 1;");
await fs.symlink(path.relative(path.join(image.packageRoot, "node_modules"), dependency), path.join(image.packageRoot, "node_modules", "dep"));
const installs = image.installs.map(({ distribution: _distribution, ...install }) => install);
const loader = loaderFor(image);
const original = await migration().qualify({ installs, loadManifest: (root) => loader.loadManifest(root) });
expect(original.configurationPackagesEligible).toBe(true);
await fs.writeFile(path.join(dependency, "index.js"), "export const value = 2;");
const changed = await migration(peer, homeB).qualify({ installs, loadManifest: (root) => loader.loadManifest(root), expectedPackageEvidenceDigest: original.packageEvidenceDigest });
expect(changed.configurationPackagesEligible).toBe(true);
expect(changed.samePackageOverlapEligible).toBe(false);
expect(changed.overlapBlockers).toContainEqual(expect.objectContaining({ code: "package_evidence_changed" }));
});
});
@@ -0,0 +1,48 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { execFileSync } from "node:child_process";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { readLegacyAdapterSettings } from "../services/adapter-settings.js";
import { inspectLegacyExternalRegistrations } from "../services/cloud-configuration-evidence.js";
import { cloudBundledPackageDigest } from "../services/cloud-bundled-package-digest.js";
vi.mock("../adapters/plugin-loader.js", () => ({ buildExternalAdapters: async () => [] }));
describe("bounded Cloud legacy readers and image inventory", () => {
let root: string;
beforeEach(async () => { root = await fs.mkdtemp(path.join(os.tmpdir(), "cloud-config-reader-")); });
afterEach(async () => { await fs.rm(root, { recursive: true, force: true }); });
it("distinguishes an absent adapter file from a saved empty preference", async () => {
expect(await readLegacyAdapterSettings({ PAPERCLIP_HOME: root })).toBeNull();
await fs.writeFile(path.join(root, "adapter-settings.json"), '{"disabledTypes":[]}');
expect(await readLegacyAdapterSettings({ PAPERCLIP_HOME: root })).toEqual({ disabledTypes: [] });
});
it.skipIf(process.platform === "win32")("rejects symlinks and FIFOs without waiting for a writer", async () => {
const target = path.join(root, "target.json");
const settings = path.join(root, "adapter-settings.json");
await fs.writeFile(target, '{"disabledTypes":[]}');
await fs.symlink(target, settings);
await expect(readLegacyAdapterSettings({ PAPERCLIP_HOME: root })).rejects.toMatchObject({ code: "ELOOP" });
expect((await inspectLegacyExternalRegistrations(settings)).blocker).toBe("unreadable_external_registration_metadata");
await fs.unlink(settings);
execFileSync("mkfifo", [settings]);
await expect(readLegacyAdapterSettings({ PAPERCLIP_HOME: root })).rejects.toThrow("regular file");
expect((await inspectLegacyExternalRegistrations(settings)).blocker).toBe("invalid_external_registration_metadata");
});
it.skipIf(process.platform === "win32")("hashes pnpm-style contained links and the physical dependency bytes", async () => {
const dependency = path.join(root, "node_modules", ".pnpm", "dep@1.0.0", "node_modules", "dep");
await fs.mkdir(dependency, { recursive: true });
await fs.writeFile(path.join(dependency, "index.js"), "export const value = 1;");
await fs.symlink(path.relative(path.join(root, "node_modules"), dependency), path.join(root, "node_modules", "dep"));
const first = cloudBundledPackageDigest(root);
await fs.writeFile(path.join(dependency, "index.js"), "export const value = 2;");
expect(cloudBundledPackageDigest(root)).not.toBe(first);
await fs.unlink(path.join(root, "node_modules", "dep"));
await fs.symlink("/", path.join(root, "node_modules", "dep"));
expect(() => cloudBundledPackageDigest(root)).toThrow("escapes its package");
});
});
@@ -0,0 +1,302 @@
import fs from "node:fs";
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { fileURLToPath, pathToFileURL } from "node:url";
import os from "node:os";
import path from "node:path";
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { applyPendingMigrations, closeRegisteredClients, createDb, instanceSettings, type Db } from "@paperclipai/db";
import { eq } from "drizzle-orm";
import { paperclipConfigSchema } from "@paperclipai/shared";
import { loadCloudBootstrapConfig, loadConfig } from "../config.js";
import { readConfigFile } from "../config-file.js";
import {
CLOUD_RUNTIME_CONFIG_KEY,
cloudRuntimeConfigService,
initializeCloudRuntimeConfig,
readLegacyCloudRuntimeConfig,
} from "../services/cloud-runtime-config.js";
import {
getCloudRuntimeConfigSnapshot,
projectCloudRuntimeConfig,
setCloudRuntimeConfigSnapshot,
} from "../services/cloud-runtime-config-snapshot.js";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
const externalDatabaseUrl = process.env.PAPERCLIP_RUNTIME_CONFIG_TEST_DATABASE_URL;
const support = externalDatabaseUrl ? { supported: true } : await getEmbeddedPostgresTestSupport();
const describePostgres = support.supported ? describe : describe.skip;
let home: string;
let configPath: string;
function legacyConfig() {
return {
$meta: { version: 1, updatedAt: "2026-10-30T00:00:00.000Z", source: "configure" },
llm: { provider: "openai", apiKey: "legacy-openai-secret" },
database: {
mode: "postgres",
connectionString: "postgres://legacy:database-secret@example.test/paperclip",
backup: { enabled: false, intervalMinutes: 73, retentionDays: 11, dir: "/tmp/legacy-backups" },
},
logging: { mode: "cloud", logDir: "/tmp/legacy-logs" },
server: { deploymentMode: "authenticated", exposure: "private", port: 3210, serveUi: false, allowedHostnames: ["saved.example.test"] },
auth: { baseUrlMode: "explicit", publicBaseUrl: "https://saved.example.test", disableSignUp: true },
storage: { provider: "s3", s3: { bucket: "saved-bucket", region: "eu-west-1", prefix: "saved/", forcePathStyle: true } },
secrets: { provider: "local_encrypted", strictMode: true, localEncrypted: { keyFilePath: "/tmp/legacy-master.key" } },
telemetry: { enabled: false },
updates: { checkEnabled: false },
unsupportedSecret: "unknown-secret-value",
};
}
beforeEach(() => {
home = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-runtime-config-"));
configPath = path.join(home, "config.json");
vi.stubEnv("PAPERCLIP_HOME", home);
vi.stubEnv("PAPERCLIP_CONFIG", configPath);
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", "test-token");
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", undefined);
vi.stubEnv("DATABASE_MIGRATION_URL", undefined);
vi.stubEnv("PAPERCLIP_DEPLOYMENT_MODE", undefined);
vi.stubEnv("PAPERCLIP_AUTH_PUBLIC_BASE_URL", undefined);
vi.stubEnv("PAPERCLIP_AUTH_BASE_URL_MODE", undefined);
vi.stubEnv("PAPERCLIP_PUBLIC_URL", undefined);
vi.stubEnv("BETTER_AUTH_URL", undefined);
vi.stubEnv("BETTER_AUTH_BASE_URL", undefined);
vi.stubEnv("PAPERCLIP_ALLOWED_HOSTNAMES", undefined);
vi.stubEnv("PAPERCLIP_TAILNET_BIND_HOST", "100.100.0.1");
vi.stubEnv("OPENAI_API_KEY", "environment-api-secret");
setCloudRuntimeConfigSnapshot(null);
});
afterEach(() => {
vi.restoreAllMocks();
vi.unstubAllEnvs();
setCloudRuntimeConfigSnapshot(null);
fs.rmSync(home, { recursive: true, force: true });
});
describe("Cloud configuration file boundary", () => {
it.each(["null", "{", JSON.stringify({ unexpected: true })])("rejects malformed present legacy data %s", async (contents) => {
fs.writeFileSync(configPath, contents);
await expect(readLegacyCloudRuntimeConfig()).rejects.toThrow(/Legacy Cloud runtime configuration/);
});
it("rejects oversized and linked legacy sources", async () => {
fs.writeFileSync(configPath, " ".repeat(65_537));
await expect(readLegacyCloudRuntimeConfig()).rejects.toThrow(/safely/);
fs.rmSync(configPath);
const target = path.join(home, "target.json");
fs.writeFileSync(target, JSON.stringify(legacyConfig()));
fs.symlinkSync(target, configPath);
await expect(readLegacyCloudRuntimeConfig()).rejects.toThrow(/safely/);
});
it("requires an environment credential for the legacy OpenAI consumer", async () => {
fs.writeFileSync(configPath, JSON.stringify(legacyConfig()));
vi.stubEnv("OPENAI_API_KEY", undefined);
await expect(readLegacyCloudRuntimeConfig()).rejects.toThrow(/OPENAI_API_KEY/);
expect(() => readConfigFile()).toThrow(/not been initialized/);
});
it.each(["https://user:secret@saved.example.test", "https://saved.example.test?key=secret", "https://saved.example.test#secret"])("rejects secret-bearing URL %s", async (url) => {
const config = legacyConfig();
config.auth.publicBaseUrl = url;
fs.writeFileSync(configPath, JSON.stringify(config));
await expect(readLegacyCloudRuntimeConfig()).rejects.toThrow(/configuration is invalid/);
});
it("loads the minimal Cloud bootstrap from environment only", () => {
fs.writeFileSync(configPath, "malformed");
vi.stubEnv("DATABASE_URL", "postgres://bootstrap:bootstrap-secret@127.0.0.1:5432/runtime");
expect(loadCloudBootstrapConfig()).toMatchObject({ databaseMode: "postgres", databaseUrl: process.env.DATABASE_URL });
vi.stubEnv("DATABASE_URL", undefined);
expect(() => loadCloudBootstrapConfig()).toThrow(/requires DATABASE_URL/);
vi.stubEnv("DATABASE_URL", "sqlite:///tmp/db");
expect(() => loadCloudBootstrapConfig()).toThrow(/must use PostgreSQL/);
});
it.each(["", " "])("treats an empty migration URL as absent (%j)", (migrationUrl) => {
vi.stubEnv("DATABASE_URL", "postgres://bootstrap:password@127.0.0.1/runtime");
vi.stubEnv("DATABASE_MIGRATION_URL", migrationUrl);
expect(loadCloudBootstrapConfig().databaseMigrationUrl).toBeUndefined();
});
it("normalizes and validates an explicit migration URL", () => {
vi.stubEnv("DATABASE_URL", "postgres://bootstrap:password@127.0.0.1/runtime");
vi.stubEnv("DATABASE_MIGRATION_URL", " postgres://migration:password@127.0.0.1/runtime ");
expect(loadCloudBootstrapConfig().databaseMigrationUrl).toBe("postgres://migration:password@127.0.0.1/runtime");
vi.stubEnv("DATABASE_MIGRATION_URL", "https://wrong.example.test");
expect(() => loadCloudBootstrapConfig()).toThrow("DATABASE_MIGRATION_URL must use PostgreSQL");
});
it("preserves local config file behavior without managed signals", () => {
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", undefined);
fs.writeFileSync(configPath, JSON.stringify(legacyConfig()));
expect(readConfigFile()?.llm?.apiKey).toBe("legacy-openai-secret");
expect(loadConfig()).toMatchObject({ port: 3210, telemetryEnabled: false, storageS3Bucket: "saved-bucket" });
});
});
describePostgres("Cloud runtime configuration in PostgreSQL", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let first: Db;
let second: Db;
beforeAll(async () => {
if (externalDatabaseUrl) {
await applyPendingMigrations(externalDatabaseUrl);
database = { connectionString: externalDatabaseUrl, cleanup: () => closeRegisteredClients(externalDatabaseUrl, { scope: "database" }) };
} else {
database = await startEmbeddedPostgresTestDatabase("paperclip-runtime-config-db-");
}
first = createDb(database.connectionString);
second = createDb(database.connectionString);
}, 90_000);
beforeEach(async () => {
await first.delete(instanceSettings).where(eq(instanceSettings.singletonKey, CLOUD_RUNTIME_CONFIG_KEY));
vi.stubEnv("DATABASE_URL", database.connectionString);
});
afterAll(async () => { await database?.cleanup(); });
it.each(["PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", "PAPERCLIP_MANAGED_CONFIG"])("seeds a fresh database with %s", async (signal) => {
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", undefined);
vi.stubEnv(signal, signal === "PAPERCLIP_MANAGED_CONFIG" ? "" : "test-token");
const result = await initializeCloudRuntimeConfig(first);
expect(result).toMatchObject({ seeded: true, current: { version: 1, revision: 0, config: null } });
expect(await cloudRuntimeConfigService(second).inspectStored()).toEqual(result?.current);
expect(fs.existsSync(configPath)).toBe(false);
vi.stubEnv("PAPERCLIP_DEPLOYMENT_MODE", "authenticated");
vi.stubEnv("PAPERCLIP_PUBLIC_URL", "https://environment.example.test");
expect(loadConfig()).toMatchObject({ databaseMode: "postgres", authBaseUrlMode: "explicit", secretsStrictMode: true, telemetryEnabled: true });
});
it("loads the saved authenticated mode before validating an environment host bind", async () => {
fs.writeFileSync(configPath, JSON.stringify(legacyConfig()));
vi.stubEnv("HOST", "0.0.0.0");
vi.stubEnv("PAPERCLIP_DEPLOYMENT_MODE", undefined);
expect(loadCloudBootstrapConfig()).toEqual({
databaseMode: "postgres",
databaseUrl: database.connectionString,
databaseMigrationUrl: process.env.DATABASE_MIGRATION_URL,
});
await initializeCloudRuntimeConfig(first);
expect(loadConfig()).toMatchObject({ deploymentMode: "authenticated", host: "0.0.0.0", port: 3210 });
});
it("imports existing choices without credentials and restores them from a second client and new home", async () => {
fs.writeFileSync(configPath, JSON.stringify(legacyConfig()));
const initial = await initializeCloudRuntimeConfig(first);
expect(loadConfig()).toMatchObject({ port: 3210, telemetryEnabled: false, databaseBackupEnabled: false, storageS3Bucket: "saved-bucket", authDisableSignUp: true });
const [row] = await second.select().from(instanceSettings).where(eq(instanceSettings.singletonKey, CLOUD_RUNTIME_CONFIG_KEY));
const persisted = JSON.stringify(row.general);
for (const secret of ["legacy-openai-secret", "database-secret", "unknown-secret-value", "environment-api-secret", database.connectionString]) {
expect(persisted).not.toContain(secret);
}
const replacementHome = fs.mkdtempSync(path.join(home, "replacement-"));
const replacementConfigDir = path.join(replacementHome, ".paperclip");
fs.mkdirSync(replacementConfigDir);
const envPath = path.join(replacementConfigDir, ".env");
const envContents = "PORT=9999\nPAPERCLIP_RUNTIME_CONFIG_ENV_SENTINEL=should-not-load\n";
fs.writeFileSync(envPath, envContents);
vi.stubEnv("PAPERCLIP_HOME", replacementHome);
vi.stubEnv("PAPERCLIP_CONFIG", path.join(replacementConfigDir, "config.json"));
setCloudRuntimeConfigSnapshot(null);
const files = vi.spyOn(fs, "readFileSync");
expect(await initializeCloudRuntimeConfig(second)).toEqual({ seeded: false, current: initial?.current });
expect(loadConfig()).toMatchObject({ port: 3210, telemetryEnabled: false, storageS3Bucket: "saved-bucket" });
expect(files.mock.calls.map(([file]) => String(file)).filter(file => file.startsWith(replacementHome))).toEqual([]);
files.mockRestore();
const root = fileURLToPath(new URL("../../../", import.meta.url));
const childScript = path.join(home, "restore.mts");
fs.writeFileSync(childScript, `
import { createDb } from ${JSON.stringify(pathToFileURL(path.join(root, "packages/db/src/index.ts")).href)};
import { initializeCloudRuntimeConfig } from ${JSON.stringify(pathToFileURL(path.join(root, "server/src/services/cloud-runtime-config.ts")).href)};
import { loadConfig } from ${JSON.stringify(pathToFileURL(path.join(root, "server/src/config.ts")).href)};
const db = createDb(process.env.DATABASE_URL);
await initializeCloudRuntimeConfig(db);
const config = loadConfig();
console.log(JSON.stringify({ port: config.port, telemetryEnabled: config.telemetryEnabled, storageS3Bucket: config.storageS3Bucket,
envSentinel: process.env.PAPERCLIP_RUNTIME_CONFIG_ENV_SENTINEL ?? null }));
await db.$client.end({ timeout: 1 });
`);
const child = await promisify(execFile)(process.execPath, [path.join(root, "server/node_modules/tsx/dist/cli.mjs"), childScript], {
cwd: replacementHome,
env: { ...process.env },
timeout: 30_000,
});
expect(JSON.parse(child.stdout.trim())).toEqual({ port: 3210, telemetryEnabled: false, storageS3Bucket: "saved-bucket", envSentinel: null });
expect(fs.readFileSync(envPath, "utf8")).toBe(envContents);
}, 45_000);
it("keeps environment overlays out of saved settings and ignores corrupt old files after seed", async () => {
fs.writeFileSync(configPath, JSON.stringify(legacyConfig()));
await initializeCloudRuntimeConfig(first);
vi.stubEnv("PORT", "4567");
vi.stubEnv("PAPERCLIP_STORAGE_S3_BUCKET", "operator-bucket");
vi.stubEnv("PAPERCLIP_AUTH_DISABLE_SIGN_UP", "false");
fs.writeFileSync(configPath, "malformed legacy contents");
await initializeCloudRuntimeConfig(second);
expect(loadConfig()).toMatchObject({ port: 4567, storageS3Bucket: "operator-bucket", authDisableSignUp: false, telemetryEnabled: false });
expect((await cloudRuntimeConfigService(first).inspectStored())?.config).toMatchObject({ server: { port: 3210 }, storage: { s3: { bucket: "saved-bucket" } }, auth: { disableSignUp: true } });
});
it("keeps one committed seed when two clients propose different initial values", async () => {
const config = projectCloudRuntimeConfig(paperclipConfigSchema.parse(legacyConfig()));
const results = await Promise.all([
cloudRuntimeConfigService(first).seedIfAbsent({ ...config, telemetry: { enabled: true } }),
cloudRuntimeConfigService(second).seedIfAbsent({ ...config, telemetry: { enabled: false } }),
]);
expect(results.filter(result => result.seeded)).toHaveLength(1);
expect(results[0].current).toEqual(results[1].current);
expect(await cloudRuntimeConfigService(first).inspectStored()).toEqual(results[0].current);
});
it("rejects unknown or credential fields on the direct mutation path", async () => {
const config = projectCloudRuntimeConfig(paperclipConfigSchema.parse(legacyConfig()));
await cloudRuntimeConfigService(first).seedIfAbsent(config);
await expect(cloudRuntimeConfigService(first).replace({ ...config, llm: { provider: "openai", apiKey: "must-not-store" } } as never, 0)).rejects.toThrow(/configuration is invalid/);
expect((await cloudRuntimeConfigService(second).inspectStored())?.revision).toBe(0);
});
it("serializes revisions and rejects competing stale writes from two database clients", async () => {
const config = projectCloudRuntimeConfig(paperclipConfigSchema.parse(legacyConfig()));
await cloudRuntimeConfigService(first).seedIfAbsent(config);
const results = await Promise.allSettled([
cloudRuntimeConfigService(first).replace({ ...config, telemetry: { enabled: true } }, 0),
cloudRuntimeConfigService(second).replace({ ...config, telemetry: { enabled: false } }, 0),
]);
expect(results.filter(result => result.status === "fulfilled")).toHaveLength(1);
expect(results.filter(result => result.status === "rejected")).toHaveLength(1);
expect((await cloudRuntimeConfigService(second).inspectStored())?.revision).toBe(1);
});
it("composes a seed with the caller transaction and rolls back both on failure", async () => {
const config = projectCloudRuntimeConfig(paperclipConfigSchema.parse(legacyConfig()));
await expect(first.transaction(async tx => {
await cloudRuntimeConfigService(first).seedIfAbsent(config, tx);
await tx.insert(instanceSettings).values({ singletonKey: "runtime-config-test-receipt", general: {} });
throw new Error("injected migration failure");
})).rejects.toThrow(/injected migration failure/);
expect(await cloudRuntimeConfigService(second).inspectStored()).toBeNull();
expect(await second.select().from(instanceSettings).where(eq(instanceSettings.singletonKey, "runtime-config-test-receipt"))).toEqual([]);
});
it("fails closed on invalid database settings without consulting a valid legacy file", async () => {
fs.writeFileSync(configPath, JSON.stringify(legacyConfig()));
await first.insert(instanceSettings).values({ singletonKey: CLOUD_RUNTIME_CONFIG_KEY, general: { version: 99, config: { secret: "hidden" } } });
await expect(initializeCloudRuntimeConfig(second)).rejects.toThrow(/Persisted Cloud runtime configuration is invalid/);
expect(() => getCloudRuntimeConfigSnapshot()).toThrow(/not been initialized/);
});
it("does not fall back to a file or old snapshot when the database fails", async () => {
fs.writeFileSync(configPath, JSON.stringify(legacyConfig()));
await initializeCloudRuntimeConfig(first);
const unavailable = createDb(database.connectionString);
await unavailable.$client.end({ timeout: 1 });
await expect(initializeCloudRuntimeConfig(unavailable)).rejects.toThrow();
expect(() => readConfigFile()).toThrow(/not been initialized/);
});
});
@@ -0,0 +1,50 @@
import { mkdtempSync, mkdirSync, realpathSync, writeFileSync } from "node:fs";
import path from "node:path";
import os from "node:os";
import type { PaperclipPluginManifestV1 } from "@paperclipai/shared";
import { BUNDLED_PLUGIN_CATALOG, resolveBundledPluginInstalls } from "../../services/bundled-plugins.js";
import { distributionBundleDigest, readDistributionPluginCatalog } from "../../services/distribution-plugin-catalog.js";
import { derivePluginDatabaseNamespace } from "../../services/plugin-database.js";
import { createBundledPluginPackageResolver } from "../../services/bundled-plugin-package.js";
export function bundledImageFixture(manifestOverrides: Partial<PaperclipPluginManifestV1> = {}) {
const root = realpathSync(mkdtempSync(path.join(os.tmpdir(), "bundled-image-")));
const packageRoot = path.join(root, "distribution", "fixture");
const manifest: PaperclipPluginManifestV1 = {
id: "paperclip.image-fixture", apiVersion: 1, version: "1.0.0", displayName: "Image Fixture",
description: "Image package relocation fixture", author: "Paperclip", categories: ["automation"], capabilities: ["issues.read"],
entrypoints: { worker: "dist/worker.js", ui: "dist/ui" }, ...manifestOverrides,
};
mkdirSync(path.join(packageRoot, "dist", "ui"), { recursive: true });
writeFileSync(path.join(packageRoot, "package.json"), JSON.stringify({
name: "@paperclipai/plugin-image-fixture", version: manifest.version, type: "module",
paperclipPlugin: { manifest: "dist/manifest.js", ...manifest.entrypoints },
}));
writeFileSync(path.join(packageRoot, "dist", "manifest.js"), `export default ${JSON.stringify(manifest)};`);
writeFileSync(path.join(packageRoot, "dist", "worker.js"), `import readline from "node:readline";
if (process.argv.includes("--source")) process.stdout.write(import.meta.url);
else {
const configs = {};
readline.createInterface({ input: process.stdin }).on("line", line => {
const request = JSON.parse(line);
let result = {};
if (request.method === "initialize") result = { ok: true, supportedMethods: ["configChanged", "health"] };
if (request.method === "configChanged") configs[request.params.companyId] = request.params.config;
if (request.method === "health") result = { status: "ok", details: { source: import.meta.url, configs } };
process.stdout.write(JSON.stringify({ jsonrpc: "2.0", id: request.id, result }) + "\\n");
if (request.method === "shutdown") setImmediate(() => process.exit(0));
});
}`);
writeFileSync(path.join(packageRoot, "dist", "ui", "index.js"), "export const source = 'current-image';");
if (manifest.database) {
mkdirSync(path.join(packageRoot, manifest.database.migrationsDir), { recursive: true });
writeFileSync(path.join(packageRoot, manifest.database.migrationsDir, "001_fixture.sql"), `CREATE TABLE ${derivePluginDatabaseNamespace(manifest.id)}.image_rows (id uuid PRIMARY KEY);`);
}
const entry = { key: "fixture", pluginKey: manifest.id, version: manifest.version, directory: "fixture", digest: distributionBundleDigest(packageRoot) };
writeFileSync(path.join(root, "distribution", "catalog.json"), JSON.stringify({ schemaVersion: 1, plugins: [entry] }));
const catalog = readDistributionPluginCatalog(root, BUNDLED_PLUGIN_CATALOG);
const installs = resolveBundledPluginInstalls(["fixture"], { catalogRoot: root, enforceCatalogRoot: true, env: {}, distributionPlugins: catalog });
const resolver = createBundledPluginPackageResolver(installs);
const registration = { pluginKey: manifest.id, packageName: "@paperclipai/plugin-image-fixture", version: manifest.version, manifestJson: manifest };
return { root, packageRoot, manifest, catalog, installs, resolver, registration };
}
@@ -20,6 +20,22 @@ import { gzipSync } from "node:zlib";
import { tmpdir } from "node:os";
import path from "node:path";
const cloudRuntimeStartupMocks = vi.hoisted(() => ({
initialize: vi.fn(async () => null),
bootstrap: vi.fn(),
telemetry: vi.fn(),
}));
vi.mock("../services/adapter-settings.js", async () => ({
...await vi.importActual<typeof import("../services/adapter-settings.js")>("../services/adapter-settings.js"),
initializeAdapterSettings: vi.fn(async () => null),
}));
vi.mock("../services/cloud-runtime-config.js", () => ({ initializeCloudRuntimeConfig: cloudRuntimeStartupMocks.initialize }));
const cloudMigrationMocks = vi.hoisted(() => ({ migrate: vi.fn(async () => null), qualify: vi.fn(async () => null) }));
vi.mock("../services/cloud-configuration-migration.js", () => ({ migrateCloudConfiguration: cloudMigrationMocks.migrate }));
vi.mock("../services/cloud-configuration-qualification.js", () => ({ qualifyCloudConfigurationForCurrentImage: cloudMigrationMocks.qualify }));
vi.mock("../telemetry.js", () => ({ initTelemetry: cloudRuntimeStartupMocks.telemetry, getTelemetryClient: vi.fn(() => null) }));
const ORIGINAL_PAPERCLIP_API_URL = process.env.PAPERCLIP_API_URL;
const ORIGINAL_PAPERCLIP_RUNTIME_API_URL = process.env.PAPERCLIP_RUNTIME_API_URL;
const ORIGINAL_PAPERCLIP_RUNTIME_API_CANDIDATES_JSON = process.env.PAPERCLIP_RUNTIME_API_CANDIDATES_JSON;
@@ -343,6 +359,7 @@ vi.mock("../services/native-runtime/native-safe-replacement.js", () => ({
vi.mock("../config.js", () => ({
loadConfig: loadConfigMock,
loadCloudBootstrapConfig: cloudRuntimeStartupMocks.bootstrap,
}));
vi.mock("../middleware/logger.js", () => ({
@@ -586,6 +603,8 @@ describe("startServer feedback export wiring", () => {
process.env.PAPERCLIP_DECISION_SIGNING_SECRET = "fedcba9876543210fedcba9876543210";
process.env.PAPERCLIP_AGENT_JWT_SECRET = "0123456789abcdef0123456789abcdef";
loadConfigMock.mockReturnValue(buildTestConfig());
cloudRuntimeStartupMocks.initialize.mockReset().mockResolvedValue(null);
cloudRuntimeStartupMocks.bootstrap.mockReturnValue({ databaseMode: "postgres", databaseUrl: buildTestConfig().databaseUrl, databaseMigrationUrl: undefined });
resolveHeartbeatSchedulingSuppressionMock.mockReturnValue({
suppressed: false,
reason: null,
@@ -595,6 +614,64 @@ describe("startServer feedback export wiring", () => {
process.env.BETTER_AUTH_SECRET = "test-secret";
});
it("awaits saved Cloud configuration before telemetry, authentication and app construction", async () => {
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", "startup-test-token");
vi.stubEnv("HOST", "0.0.0.0");
vi.stubEnv("PAPERCLIP_DEPLOYMENT_MODE", undefined);
cloudRuntimeStartupMocks.bootstrap.mockReturnValue({ databaseMode: "postgres", databaseUrl: buildTestConfig().databaseUrl, databaseMigrationUrl: undefined });
let release: (() => void) | undefined;
cloudRuntimeStartupMocks.initialize.mockImplementationOnce(() => new Promise<null>(resolve => {
release = () => {
loadConfigMock.mockReturnValue(buildTestConfig({ telemetryEnabled: false, authDisableSignUp: true, port: 4321, host: "0.0.0.0", bind: "all" }));
resolve(null);
};
}));
try {
const starting = startServer();
for (let index = 0; index < 30 && !release; index += 1) await new Promise<void>(resolve => setImmediate(resolve));
expect(release).toBeTypeOf("function");
expect(cloudMigrationMocks.migrate.mock.invocationCallOrder[0]).toBeLessThan(cloudRuntimeStartupMocks.initialize.mock.invocationCallOrder[0]!);
expect(cloudRuntimeStartupMocks.telemetry).not.toHaveBeenCalled();
expect(createBetterAuthInstanceMock).not.toHaveBeenCalled();
expect(createAppMock).not.toHaveBeenCalled();
expect(loadConfigMock).not.toHaveBeenCalled();
release!();
const started = await starting;
await settleStartupHousekeeping();
expect(cloudRuntimeStartupMocks.telemetry).toHaveBeenCalledExactlyOnceWith({ enabled: false });
expect(createBetterAuthInstanceMock).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({ authDisableSignUp: true, port: 4321 }), expect.anything());
expect(started.listenPort).toBe(4321);
expect(cloudMigrationMocks.qualify.mock.invocationCallOrder[0]).toBeLessThan(createAppMock.mock.invocationCallOrder[0]!);
} finally {
release?.();
vi.unstubAllEnvs();
}
});
it("does not create telemetry or the app when saved Cloud configuration fails", async () => {
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", "startup-test-token");
cloudRuntimeStartupMocks.initialize.mockRejectedValueOnce(new Error("persisted config unavailable"));
try {
await expect(startServer()).rejects.toThrow("persisted config unavailable");
expect(cloudRuntimeStartupMocks.telemetry).not.toHaveBeenCalled();
expect(createBetterAuthInstanceMock).not.toHaveBeenCalled();
expect(createAppMock).not.toHaveBeenCalled();
} finally {
vi.unstubAllEnvs();
}
});
it("stops before configuration consumers when the atomic Cloud migration fails", async () => {
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", "startup-test-token");
cloudMigrationMocks.migrate.mockRejectedValueOnce(new Error("migration did not commit"));
try {
await expect(startServer()).rejects.toThrow("migration did not commit");
expect(cloudRuntimeStartupMocks.initialize).not.toHaveBeenCalled();
expect(cloudRuntimeStartupMocks.telemetry).not.toHaveBeenCalled();
expect(createAppMock).not.toHaveBeenCalled();
} finally { vi.unstubAllEnvs(); }
});
it.each([false, true])("registers runner evidence independently from the backup scan (%s)", async enabled => {
loadConfigMock.mockReturnValue(buildTestConfig({ nativeHarnessBackupExpirationEnabled: enabled }));
await startServer(); await settleStartupHousekeeping();
@@ -1509,6 +1586,8 @@ describe("startServer authenticated auth origin setup", () => {
vi.clearAllMocks();
process.env.PAPERCLIP_DECISION_SIGNING_SECRET = "fedcba9876543210fedcba9876543210";
loadConfigMock.mockReturnValue(buildTestConfig());
cloudRuntimeStartupMocks.initialize.mockReset().mockResolvedValue(null);
cloudRuntimeStartupMocks.bootstrap.mockReturnValue({ databaseMode: "postgres", databaseUrl: buildTestConfig().databaseUrl, databaseMigrationUrl: undefined });
createBetterAuthInstanceMock.mockReturnValue({});
deriveAuthTrustedOriginsMock.mockReturnValue([]);
process.env.BETTER_AUTH_SECRET = "test-secret";
@@ -1570,6 +1649,8 @@ describe("startServer PAPERCLIP_API_URL handling", () => {
vi.clearAllMocks();
process.env.PAPERCLIP_DECISION_SIGNING_SECRET = "fedcba9876543210fedcba9876543210";
loadConfigMock.mockReturnValue(buildTestConfig());
cloudRuntimeStartupMocks.initialize.mockReset().mockResolvedValue(null);
cloudRuntimeStartupMocks.bootstrap.mockReturnValue({ databaseMode: "postgres", databaseUrl: buildTestConfig().databaseUrl, databaseMigrationUrl: undefined });
process.env.BETTER_AUTH_SECRET = "test-secret";
delete process.env.PAPERCLIP_API_URL;
});
@@ -0,0 +1,136 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
let home: string;
let localPackage: string;
let npmPackage: string;
let storePath: string;
let sentinel: string;
const parserSource = "export function parseStdoutLine() { return { message: 'external parser' }; }";
const managedError = { status: 403, details: { code: "adapter_install_platform_managed" } };
function writePackage(packageDir: string, type: string): void {
fs.mkdirSync(packageDir, { recursive: true });
fs.writeFileSync(path.join(packageDir, "package.json"), JSON.stringify({
name: `fixture-${type}`,
type: "module",
exports: { ".": "./index.mjs", "./ui-parser": "./ui-parser.mjs" },
paperclip: { adapterUiParser: "1.0.0" },
}));
fs.writeFileSync(path.join(packageDir, "index.mjs"), `
import fs from "node:fs";
fs.appendFileSync(${JSON.stringify(sentinel)}, ${JSON.stringify(type + "\n")});
export function createServerAdapter() {
return {
type: ${JSON.stringify(type)},
agentConfigurationDoc: "External fixture",
models: [],
execute: async () => ({ exitCode: 41, signal: null, timedOut: false }),
testEnvironment: async () => ({ status: "pass", checks: [] }),
};
}
`);
fs.writeFileSync(path.join(packageDir, "ui-parser.mjs"), parserSource);
}
beforeEach(() => {
vi.resetModules();
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", undefined);
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", undefined);
home = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-cloud-adapters-"));
vi.stubEnv("PAPERCLIP_HOME", home);
sentinel = path.join(home, "external-code-executed");
localPackage = path.join(home, "local-package");
npmPackage = path.join(home, "adapter-plugins", "node_modules", "fixture-npm");
storePath = path.join(home, "adapter-plugins.json");
writePackage(localPackage, "codex_local");
writePackage(npmPackage, "fixture_npm");
fs.writeFileSync(storePath, JSON.stringify([
{ packageName: "fixture-local", localPath: localPackage, type: "codex_local", installedAt: "2026-01-01" },
{ packageName: "fixture-npm", type: "fixture_npm", installedAt: "2026-01-01" },
]));
});
afterEach(() => {
vi.restoreAllMocks();
vi.unstubAllEnvs();
fs.rmSync(home, { recursive: true, force: true });
});
describe("Cloud image adapter boundary", () => {
it.each(["PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", "PAPERCLIP_MANAGED_CONFIG"])(
"cold boots built-ins without reading or executing persisted packages with %s",
async (signal) => {
vi.stubEnv(signal, signal === "PAPERCLIP_MANAGED_CONFIG" ? "" : "test-token");
const registrationBefore = fs.readFileSync(storePath, "utf8");
const reads = vi.spyOn(fs, "readFileSync");
const registry = await import("./registry.js");
await registry.waitForExternalAdapters();
const loader = await import("./plugin-loader.js");
const store = await import("../services/adapter-plugin-store.js");
expect(registry.requireServerAdapter("codex_local").agentConfigurationDoc).not.toBe("External fixture");
expect(registry.requireServerAdapter("process").execute).toBeTypeOf("function");
expect(registry.findServerAdapter("fixture_npm")).toBeNull();
expect(registry.hasActiveAdapterOverride("codex_local")).toBe(false);
expect(await loader.buildExternalAdapters()).toEqual([]);
expect(store.listAdapterPlugins()).toEqual([]);
expect(store.getAdapterPluginByType("codex_local")).toBeUndefined();
expect(loader.getUiParserSource("codex_local")).toBeUndefined();
expect(loader.getOrExtractUiParserSource("codex_local")).toBeUndefined();
await expect(loader.loadExternalAdapterPackage("fixture-local", localPackage)).rejects.toMatchObject(managedError);
await expect(loader.loadExternalAdapterPackage("fixture-npm")).rejects.toMatchObject(managedError);
await expect(loader.reloadExternalAdapter("codex_local")).rejects.toMatchObject(managedError);
expect(() => registry.registerServerAdapter(registry.requireServerAdapter("codex_local"))).toThrow();
expect(() => registry.unregisterServerAdapter("codex_local")).toThrow();
expect(() => registry.setOverridePaused("codex_local", false)).toThrow();
expect(() => store.addAdapterPlugin({ packageName: "fixture", type: "fixture", installedAt: "2026-01-01" })).toThrow();
expect(() => store.removeAdapterPlugin("codex_local")).toThrow();
expect(() => store.getAdapterPluginsDir()).toThrow();
expect(store.getAdapterPluginStorePath()).toBe(storePath);
expect(reads.mock.calls.map(([file]) => String(file)).filter(file =>
file === storePath || file.startsWith(localPackage + path.sep) || file.startsWith(npmPackage + path.sep),
)).toEqual([]);
reads.mockRestore();
expect(fs.existsSync(sentinel)).toBe(false);
expect(fs.readFileSync(storePath, "utf8")).toBe(registrationBefore);
},
);
it("preserves local and npm package loading, overrides, reload, and lazy UI parsing on self-hosted instances", async () => {
const loader = await import("./plugin-loader.js");
expect(loader.getOrExtractUiParserSource("codex_local")).toBe(parserSource);
const registry = await import("./registry.js");
await registry.waitForExternalAdapters();
const override = registry.requireServerAdapter("codex_local");
expect(override.agentConfigurationDoc).toBe("External fixture");
expect(registry.hasActiveAdapterOverride("codex_local")).toBe(true);
expect(registry.requireServerAdapter("fixture_npm").agentConfigurationDoc).toBe("External fixture");
expect(await override.execute({} as never)).toMatchObject({ exitCode: 41 });
expect(fs.readFileSync(sentinel, "utf8").trim().split("\n")).toEqual(["codex_local", "fixture_npm"]);
expect(loader.getUiParserSource("fixture_npm")).toBe(parserSource);
const reloaded = await loader.reloadExternalAdapter("codex_local");
expect(reloaded?.type).toBe("codex_local");
expect(await reloaded?.execute({} as never)).toMatchObject({ exitCode: 41 });
expect(registry.setOverridePaused("codex_local", true)).toBe(true);
expect(registry.findActiveServerAdapter("codex_local")?.agentConfigurationDoc).not.toBe("External fixture");
expect(registry.setOverridePaused("codex_local", false)).toBe(true);
registry.unregisterServerAdapter("codex_local");
expect(registry.requireServerAdapter("codex_local").agentConfigurationDoc).not.toBe("External fixture");
});
it("does not expose a cached registration or parser after a managed signal becomes present", async () => {
const loader = await import("./plugin-loader.js");
const store = await import("../services/adapter-plugin-store.js");
await loader.loadExternalAdapterPackage("fixture-local", localPackage);
expect(loader.getUiParserSource("codex_local")).toBe(parserSource);
expect(store.listAdapterPlugins()).toHaveLength(2);
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", "{}");
expect(store.listAdapterPlugins()).toEqual([]);
expect(loader.getUiParserSource("codex_local")).toBeUndefined();
expect(loader.getOrExtractUiParserSource("codex_local")).toBeUndefined();
});
});
+7
View File
@@ -15,6 +15,8 @@ import { pathToFileURL } from "node:url";
import type { ServerAdapterModule } from "./types.js";
import { validateAdapterLoginCapability } from "@paperclipai/adapter-utils";
import { logger } from "../middleware/logger.js";
import { isCloudManagedInstance } from "../services/cloud-instance.js";
import { assertExternalAdapterCodeAllowed } from "../services/adapter-code-policy.js";
import {
listAdapterPlugins,
@@ -30,6 +32,7 @@ import type { AdapterPluginRecord } from "../services/adapter-plugin-store.js";
const uiParserCache = new Map<string, string>();
export function getUiParserSource(adapterType: string): string | undefined {
if (isCloudManagedInstance()) return undefined;
return uiParserCache.get(adapterType);
}
@@ -38,6 +41,7 @@ export function getUiParserSource(adapterType: string): string | undefined {
* Makes the ui-parser.js endpoint self-healing.
*/
export function getOrExtractUiParserSource(adapterType: string): string | undefined {
if (isCloudManagedInstance()) return undefined;
const cached = uiParserCache.get(adapterType);
if (cached) return cached;
@@ -182,6 +186,7 @@ export async function loadExternalAdapterPackage(
packageName: string,
localPath?: string,
): Promise<ServerAdapterModule> {
assertExternalAdapterCodeAllowed();
const packageDir = localPath
? path.resolve(localPath)
: path.resolve(getAdapterPluginsDir(), "node_modules", packageName);
@@ -221,6 +226,7 @@ async function loadFromRecord(record: AdapterPluginRecord): Promise<ServerAdapte
export async function reloadExternalAdapter(
type: string,
): Promise<ServerAdapterModule | null> {
assertExternalAdapterCodeAllowed();
const record = getAdapterPluginByType(type);
if (!record) return null;
@@ -271,6 +277,7 @@ export async function reloadExternalAdapter(
* Build all external adapter modules from the plugin store.
*/
export async function buildExternalAdapters(): Promise<ServerAdapterModule[]> {
if (isCloudManagedInstance()) return [];
const results: ServerAdapterModule[] = [];
const storeRecords = listAdapterPlugins();
+7 -8
View File
@@ -136,7 +136,8 @@ import {
import { agentConfigurationDoc as piAgentConfigurationDoc } from "@paperclipai/adapter-pi-local";
import { BUILTIN_ADAPTER_TYPES } from "./builtin-adapter-types.js";
import { buildExternalAdapters } from "./plugin-loader.js";
import { getDisabledAdapterTypes } from "../services/adapter-plugin-store.js";
import { isCloudManagedInstance } from "../services/cloud-instance.js";
import { assertExternalAdapterCodeAllowed } from "../services/adapter-code-policy.js";
import { NativeEnvironmentProbeCleanupError } from "../vendor/paperclip-runner/index.js";
import { processAdapter } from "./process/index.js";
import { httpAdapter } from "./http/index.js";
@@ -997,11 +998,6 @@ registerBuiltInAdapters();
// applies the registry fallback for externals overriding a built-in type.
// ---------------------------------------------------------------------------
/** Cached sync wrapper — the store is a simple JSON file read, safe to call frequently. */
function getDisabledAdapterTypesFromStore(): string[] {
return getDisabledAdapterTypes();
}
/**
* Merge an external adapter module with host-provided session management.
*
@@ -1034,6 +1030,7 @@ export function resolveExternalAdapterRegistration(
* and avoid racing against the loading window.
*/
const externalAdaptersReady: Promise<void> = (async () => {
if (isCloudManagedInstance()) return;
try {
const externalAdapters = await buildExternalAdapters();
for (const externalAdapter of externalAdapters) {
@@ -1069,6 +1066,7 @@ export function waitForExternalAdapters(): Promise<void> {
}
export function registerServerAdapter(adapter: ServerAdapterModule): void {
assertExternalAdapterCodeAllowed();
if (BUILTIN_ADAPTER_TYPES.has(adapter.type) && !builtinFallbacks.has(adapter.type)) {
const existing = adaptersByType.get(adapter.type);
if (existing) {
@@ -1079,6 +1077,7 @@ export function registerServerAdapter(adapter: ServerAdapterModule): void {
}
export function unregisterServerAdapter(type: string): void {
assertExternalAdapterCodeAllowed();
if (type === processAdapter.type || type === httpAdapter.type) return;
if (builtinFallbacks.has(type)) {
pausedOverrides.delete(type);
@@ -1186,8 +1185,7 @@ export function listServerAdapters(): ServerAdapterModule[] {
* Used for menus and agent creation flows — disabled adapters remain
* functional for existing agents but hidden from selection.
*/
export function listEnabledServerAdapters(): ServerAdapterModule[] {
const disabled = getDisabledAdapterTypesFromStore();
export function listEnabledServerAdapters(disabled: string[]): ServerAdapterModule[] {
const disabledSet = disabled.length > 0 ? new Set(disabled) : null;
return disabledSet
? Array.from(adaptersByType.values()).filter((a) => !disabledSet.has(a.type))
@@ -1227,6 +1225,7 @@ export async function detectAdapterModel(
* an override or was already in the requested state.
*/
export function setOverridePaused(type: string, paused: boolean): boolean {
assertExternalAdapterCodeAllowed();
if (!builtinFallbacks.has(type)) return false;
const wasPaused = pausedOverrides.has(type);
if (paused && !wasPaused) {
+13 -4
View File
@@ -183,7 +183,9 @@ import {
ensureBundledPlugins,
resolveBundledCatalogRoot,
resolveBundledPluginInstalls,
resolveManagedBundledPluginKeys,
} from "./services/bundled-plugins.js";
import { createBundledPluginPackageResolver } from "./services/bundled-plugin-package.js";
import { readDistributionPluginCatalog, distributionPluginActivationGuard } from "./services/distribution-plugin-catalog.js";
import {
createPluginWorkerManager,
@@ -623,7 +625,7 @@ export async function createApp(
res.set("Cache-Control", "no-store"); res.set("Referrer-Policy", "no-referrer");
res.redirect(303, await chatChannels.githubWizard.identityCallback(String(req.query.state ?? ""), String(req.query.code ?? ""), req.actor));
});
const managedAutoInstallKeys = opts.managedPluginAutoInstall ?? null;
const managedAutoInstallKeys = resolveManagedBundledPluginKeys(opts.managedPluginAutoInstall);
const bundledCatalogRoot =
opts.bundledPluginCatalogRoot ?? resolveBundledCatalogRoot(process.env);
const distributionPlugins = readDistributionPluginCatalog(bundledCatalogRoot, BUNDLED_PLUGIN_CATALOG);
@@ -636,6 +638,7 @@ export async function createApp(
distributionPlugins,
},
);
const bundledPackageResolver = managedAutoInstallKeys === null ? undefined : createBundledPluginPackageResolver(bundledPluginInstalls);
const managedBundledPluginKeys =
managedAutoInstallKeys !== null
? bundledPluginInstalls.map((install) => install.pluginKey)
@@ -906,7 +909,11 @@ export async function createApp(
{
localPluginDir: opts.localPluginDir ?? DEFAULT_LOCAL_PLUGIN_DIR,
migrationDb: opts.pluginMigrationDb,
assertPackageActivation: distributionPluginActivationGuard(bundledCatalogRoot, distributionPlugins, managedAutoInstallKeys),
assertPackageActivation: bundledPackageResolver?.assertActivation ?? distributionPluginActivationGuard(bundledCatalogRoot, distributionPlugins, managedAutoInstallKeys),
resolvePackageRoot: bundledPackageResolver?.resolve,
prebuiltOnly: managedAutoInstallKeys !== null,
enableLocalFilesystem: managedAutoInstallKeys === null,
enableNpmDiscovery: managedAutoInstallKeys === null,
},
{
workerManager,
@@ -965,6 +972,7 @@ export async function createApp(
);
api.use(
adapterRoutes({
db,
resolveRunnerTarget: (companyId, environmentId) => resolveAgentRunnerTargetForCompany(db, companyId, environmentId),
getNativeRunnerEnabled: async () =>
(await instanceSettingsService(db).getExperimental())
@@ -1014,6 +1022,7 @@ export async function createApp(
app.use(
pluginUiStaticRoutes(db, {
localPluginDir: opts.localPluginDir ?? DEFAULT_LOCAL_PLUGIN_DIR,
resolvePackageRoot: bundledPackageResolver?.resolve,
}),
);
@@ -1253,7 +1262,7 @@ export async function createApp(
void toolDispatcher.initialize().catch((err) => {
logger.error({ err }, "Failed to initialize plugin tool dispatcher");
});
const devWatcher = createPluginDevWatcher(
const devWatcher = managedAutoInstallKeys !== null ? null : createPluginDevWatcher(
lifecycle,
async (pluginId) =>
(await pluginRegistry.getById(pluginId))?.packagePath ?? null,
@@ -1299,7 +1308,7 @@ export async function createApp(
// owns provisioning); self-hosted leaves an operator's uninstall alone.
// Disabled plugins never start automatically. Added distribution permissions
// still enter upgrade_pending so enabling them requires an operator decision.
{ reinstallUninstalled: managedAutoInstallKeys !== null },
{ reinstallUninstalled: managedAutoInstallKeys !== null, preservePackagePath: managedAutoInstallKeys !== null },
)
.then(() => loader.loadAll())
.then((result) => {
+3
View File
@@ -8,6 +8,8 @@ import {
} from "@paperclipai/shared";
import { ZodError } from "zod";
import { resolvePaperclipConfigPath } from "./paths.js";
import { isCloudManagedInstance } from "./services/cloud-instance.js";
import { getCloudRuntimeConfigSnapshot, toPaperclipConfig } from "./services/cloud-runtime-config-snapshot.js";
function formatConfigValidationError(error: ZodError): string {
return error.issues
@@ -19,6 +21,7 @@ function formatConfigValidationError(error: ZodError): string {
}
export function readConfigFile(): PaperclipConfig | null {
if (isCloudManagedInstance()) return toPaperclipConfig(getCloudRuntimeConfigSnapshot().config);
const configPath = resolvePaperclipConfigPath();
if (!fs.existsSync(configPath)) return null;
+46 -22
View File
@@ -1,4 +1,7 @@
import { readConfigFile, resolveDeploymentMode } from "./config-file.js";
import { isCloudManagedInstance } from "./services/cloud-instance.js";
import { StartupRefusalError } from "./startup-refusals.js";
import type { PaperclipConfig } from "@paperclipai/shared";
import { parseChatWebhookPublicBaseUrl } from "./chat-webhook-public-url.js";
import { execFileSync } from "node:child_process";
import { existsSync, realpathSync } from "node:fs";
@@ -32,27 +35,29 @@ import {
resolveHomeAwarePath,
} from "./home-paths.js";
const PAPERCLIP_ENV_FILE_PATH = resolvePaperclipEnvPath();
if (existsSync(PAPERCLIP_ENV_FILE_PATH)) {
loadDotenv({ path: PAPERCLIP_ENV_FILE_PATH, override: false, quiet: true });
}
if (!isCloudManagedInstance()) {
const PAPERCLIP_ENV_FILE_PATH = resolvePaperclipEnvPath();
if (existsSync(PAPERCLIP_ENV_FILE_PATH)) {
loadDotenv({ path: PAPERCLIP_ENV_FILE_PATH, override: false, quiet: true });
}
const CWD_ENV_PATH = resolve(process.cwd(), ".env");
const cwdEnvExists = existsSync(CWD_ENV_PATH);
const isSameFile = cwdEnvExists && existsSync(PAPERCLIP_ENV_FILE_PATH)
? realpathSync(CWD_ENV_PATH) === realpathSync(PAPERCLIP_ENV_FILE_PATH)
: CWD_ENV_PATH === PAPERCLIP_ENV_FILE_PATH;
if (shouldLoadWorkingDirectoryEnv({
cwdEnvExists,
isPaperclipEnvFile: isSameFile,
})) {
loadDotenv({ path: CWD_ENV_PATH, override: false, quiet: true });
}
const CWD_ENV_PATH = resolve(process.cwd(), ".env");
const cwdEnvExists = existsSync(CWD_ENV_PATH);
const isSameFile = cwdEnvExists && existsSync(PAPERCLIP_ENV_FILE_PATH)
? realpathSync(CWD_ENV_PATH) === realpathSync(PAPERCLIP_ENV_FILE_PATH)
: CWD_ENV_PATH === PAPERCLIP_ENV_FILE_PATH;
if (shouldLoadWorkingDirectoryEnv({
cwdEnvExists,
isPaperclipEnvFile: isSameFile,
})) {
loadDotenv({ path: CWD_ENV_PATH, override: false, quiet: true });
}
if (basename(dirname(PAPERCLIP_ENV_FILE_PATH)) === ".paperclip") {
applyEmptyWorktreeSigningSecrets(dirname(dirname(PAPERCLIP_ENV_FILE_PATH)));
if (basename(dirname(PAPERCLIP_ENV_FILE_PATH)) === ".paperclip") {
applyEmptyWorktreeSigningSecrets(dirname(dirname(PAPERCLIP_ENV_FILE_PATH)));
}
maybeRepairLegacyWorktreeConfigAndEnvFiles();
}
maybeRepairLegacyWorktreeConfigAndEnvFiles();
const TAILSCALE_DETECT_TIMEOUT_MS = 3000;
@@ -147,13 +152,12 @@ function detectTailnetBindHost(): string | undefined {
}
}
export function loadConfig(): Config {
const fileConfig = readConfigFile();
export function loadConfig(fileConfig: PaperclipConfig | null = readConfigFile()): Config {
const fileDatabaseMode =
(fileConfig?.database.mode === "postgres" ? "postgres" : "embedded-postgres") as DatabaseMode;
(isCloudManagedInstance() || fileConfig?.database.mode === "postgres" ? "postgres" : "embedded-postgres") as DatabaseMode;
const fileDbUrl =
fileDatabaseMode === "postgres"
!isCloudManagedInstance() && fileDatabaseMode === "postgres"
? fileConfig?.database.connectionString
: undefined;
const fileDatabaseBackup = fileConfig?.database.backup;
@@ -460,3 +464,23 @@ export function loadConfig(): Config {
announcementsFeedUrl: process.env.PAPERCLIP_ANNOUNCEMENTS_FEED_URL?.trim() || "https://pages.paperclip.ing/announcements/v1/current.json",
};
}
export type CloudBootstrapConfig = {
databaseMode: "postgres";
databaseUrl: string;
databaseMigrationUrl: string | undefined;
};
export function loadCloudBootstrapConfig(): CloudBootstrapConfig {
if (!isCloudManagedInstance()) throw new Error("Cloud bootstrap configuration requires a managed instance");
const url = process.env.DATABASE_URL?.trim();
if (!url) throw new StartupRefusalError("database-contract-unmet", "Cloud bootstrap requires DATABASE_URL from the environment");
const migrationUrl = process.env.DATABASE_MIGRATION_URL?.trim() || undefined;
for (const [name, value] of [["DATABASE_URL", url], ["DATABASE_MIGRATION_URL", migrationUrl]]) {
if (!value) continue;
let parsed: URL;
try { parsed = new URL(value); } catch { throw new Error(`Cloud bootstrap ${name} must use PostgreSQL`); }
if (!["postgres:", "postgresql:"].includes(parsed.protocol)) throw new Error(`Cloud bootstrap ${name} must use PostgreSQL`);
}
return { databaseMode: "postgres", databaseUrl: url, databaseMigrationUrl: migrationUrl };
}
+42 -21
View File
@@ -104,7 +104,8 @@ import {
} from "@paperclipai/db";
import detectPort from "detect-port";
import { createApp } from "./app.js";
import { loadConfig } from "./config.js";
import { initializeCloudRuntimeConfig } from "./services/cloud-runtime-config.js";
import { loadCloudBootstrapConfig, loadConfig, type Config } from "./config.js";
import { logger } from "./middleware/logger.js";
import { createIncompleteTaskWatchdogScanReporter } from "./services/task-watchdog-diagnostics.js";
import { setStartupRecoveryPhase } from "./startup-recovery-state.js";
@@ -186,6 +187,9 @@ import {
loadWithoutCoordinatedShutdownSignalHooks,
} from "./shutdown.js";
import { initializeCloudRuntimeIdentity } from "./services/cloud-runtime-identity.js";
import { initializeAdapterSettings } from "./services/adapter-settings.js";
import { migrateCloudConfiguration } from "./services/cloud-configuration-migration.js";
import { qualifyCloudConfigurationForCurrentImage } from "./services/cloud-configuration-qualification.js";
import { systemdNotify } from "./services/systemd-notify.js";
import { flushInFlightRunLogMirrors } from "./services/run-log-store.js";
import {
@@ -251,6 +255,19 @@ async function endDatabaseClient(client: unknown, timeoutSeconds: number): Promi
await sql.end({ timeout: timeoutSeconds });
}
function initializeRuntimeConfigConsumers(config: ReturnType<typeof loadConfig>): void {
initTelemetry({ enabled: config.telemetryEnabled });
if (process.env.PAPERCLIP_SECRETS_PROVIDER === undefined) {
process.env.PAPERCLIP_SECRETS_PROVIDER = config.secretsProvider;
}
if (process.env.PAPERCLIP_SECRETS_STRICT_MODE === undefined) {
process.env.PAPERCLIP_SECRETS_STRICT_MODE = config.secretsStrictMode ? "true" : "false";
}
if (process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE === undefined) {
process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE = config.secretsMasterKeyFilePath;
}
}
export async function startServer(): Promise<StartedServer> {
const startupDatabase: StartupDatabaseTeardown = { close: null };
try {
@@ -278,17 +295,8 @@ async function startServerWithDatabaseTeardown(
// sentry.ts.
await sentryReady;
ensureDecisionSigningSecret();
let config = loadConfig();
initTelemetry({ enabled: config.telemetryEnabled });
if (process.env.PAPERCLIP_SECRETS_PROVIDER === undefined) {
process.env.PAPERCLIP_SECRETS_PROVIDER = config.secretsProvider;
}
if (process.env.PAPERCLIP_SECRETS_STRICT_MODE === undefined) {
process.env.PAPERCLIP_SECRETS_STRICT_MODE = config.secretsStrictMode ? "true" : "false";
}
if (process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE === undefined) {
process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE = config.secretsMasterKeyFilePath;
}
const startupConfig = isCloudManagedInstance() ? loadCloudBootstrapConfig() : loadConfig();
if ("deploymentMode" in startupConfig) initializeRuntimeConfigConsumers(startupConfig);
type MigrationSummary =
| "skipped"
@@ -390,7 +398,7 @@ async function startServerWithDatabaseTeardown(
}
}
function assertCloudDatabaseContract(): void {
function assertCloudDatabaseContract(config: Config): void {
if (config.deploymentMode !== "authenticated" || config.deploymentExposure !== "public") {
return;
}
@@ -483,17 +491,19 @@ async function startServerWithDatabaseTeardown(
let startupDbInfo:
| { mode: "external-postgres"; connectionString: string }
| { mode: "embedded-postgres"; dataDir: string; port: number };
assertCloudDatabaseContract();
if (config.databaseUrl) {
const migrationUrl = config.databaseMigrationUrl ?? config.databaseUrl;
if ("deploymentMode" in startupConfig) assertCloudDatabaseContract(startupConfig);
if (startupConfig.databaseUrl) {
const migrationUrl = startupConfig.databaseMigrationUrl ?? startupConfig.databaseUrl;
migrationSummary = await ensureMigrations(migrationUrl, "PostgreSQL");
db = createDb(config.databaseUrl);
pluginMigrationDb = config.databaseMigrationUrl ? createDb(config.databaseMigrationUrl) : db;
db = createDb(startupConfig.databaseUrl);
pluginMigrationDb = startupConfig.databaseMigrationUrl ? createDb(startupConfig.databaseMigrationUrl) : db;
logger.info("Using external PostgreSQL via DATABASE_URL/config");
activeDatabaseConnectionString = config.databaseUrl;
startupDbInfo = { mode: "external-postgres", connectionString: config.databaseUrl };
activeDatabaseConnectionString = startupConfig.databaseUrl;
startupDbInfo = { mode: "external-postgres", connectionString: startupConfig.databaseUrl };
} else {
if (!("deploymentMode" in startupConfig)) throw new Error("Cloud bootstrap requires an external PostgreSQL connection");
const config = startupConfig;
const moduleName = "embedded-postgres";
let EmbeddedPostgres: EmbeddedPostgresCtor;
try {
@@ -711,12 +721,19 @@ async function startServerWithDatabaseTeardown(
};
startupDatabase.close = closeDatabaseClients;
setSentryRunLinkResolver(createSentryRunLinkResolver(db));
await migrateCloudConfiguration(db as any);
await initializeAdapterSettings(db as any);
await initializeCloudRuntimeConfig(db as any);
let config = "deploymentMode" in startupConfig ? startupConfig : loadConfig();
// A claimed warm-pool stack may restart while its provider environment still
// names the pool host. Restore the signed, durable identity before Better
// Auth, routes, or child-runtime configuration capture any public URL.
const restoredCloudRuntimeIdentity = await initializeCloudRuntimeIdentity(db as any);
if (restoredCloudRuntimeIdentity) config = loadConfig();
if (isCloudManagedInstance()) initializeRuntimeConfigConsumers(config);
const isWarmStandby = await createCloudWarmStandby(db as any);
if (config.deploymentMode === "local_trusted" && !isLoopbackHost(config.host)) {
@@ -963,6 +980,10 @@ async function startServerWithDatabaseTeardown(
// document parsed fail-closed above (`plugins.autoInstall`). Absent env means
// self-hosted: createApp falls back to its built-in kubernetes-only default.
const managedPluginAutoInstall = managedConfig?.plugins.autoInstall ?? null;
const cloudQualification = await qualifyCloudConfigurationForCurrentImage(db as any);
if (cloudQualification && !cloudQualification.configurationPackagesEligible) {
logger.warn({ scope: cloudQualification.scope, blockerCodes: cloudQualification.blockers.map((blocker) => blocker.code) }, "Cloud configuration and package qualification has blockers");
}
const app = await createApp(db as any, {
prepareIdleDatabaseBackup: idleDatabaseBackupEnabled
? async () => (await runServerDatabaseBackup("idle")) !== null
+2
View File
@@ -1,6 +1,7 @@
import fs from "node:fs";
import path from "node:path";
import { resolveDefaultConfigPath } from "./home-paths.js";
import { isCloudManagedInstance } from "./services/cloud-instance.js";
const PAPERCLIP_CONFIG_BASENAME = "config.json";
const PAPERCLIP_ENV_FILENAME = ".env";
@@ -26,6 +27,7 @@ function findConfigFileFromAncestors(startDir: string): string | null {
export function resolvePaperclipConfigPath(overridePath?: string): string {
if (overridePath) return path.resolve(overridePath);
if (process.env.PAPERCLIP_CONFIG) return path.resolve(process.env.PAPERCLIP_CONFIG);
if (isCloudManagedInstance()) return resolveDefaultConfigPath();
return findConfigFileFromAncestors(process.cwd()) ?? resolveDefaultConfigPath();
}
+47
View File
@@ -0,0 +1,47 @@
import path from "node:path";
import { pathToFileURL } from "node:url";
import { createDb } from "@paperclipai/db";
import { cloudConfigurationMigrationService } from "./services/cloud-configuration-migration.js";
import { qualifyCloudConfigurationForCurrentImage } from "./services/cloud-configuration-qualification.js";
export async function runCloudConfigurationQualification(args: string[], env: NodeJS.ProcessEnv = process.env): Promise<number> {
const allowed = new Set(["--migrate", "--expected-package-evidence", "--recheck-external-revision"]);
let migrate = false;
let expectedPackageEvidenceDigest: string | undefined;
let recheckRevision: number | undefined;
for (let index = 0; index < args.length; index += 1) {
const argument = args[index];
if (!allowed.has(argument)) throw new Error(`Unknown qualification argument: ${argument}`);
if (argument === "--migrate") migrate = true;
if (argument === "--expected-package-evidence") {
expectedPackageEvidenceDigest = args[++index];
if (!expectedPackageEvidenceDigest || !/^sha256:[a-f0-9]{64}$/.test(expectedPackageEvidenceDigest)) throw new Error("A SHA-256 package evidence digest is required");
}
if (argument === "--recheck-external-revision") {
const value = args[++index];
if (!value || !/^\d+$/.test(value) || !Number.isSafeInteger(Number(value))) throw new Error("A nonnegative receipt revision is required");
recheckRevision = Number(value);
}
}
const url = env.DATABASE_URL?.trim();
if (!url) throw new Error("DATABASE_URL is required for Cloud configuration qualification");
const db = createDb(url);
try {
const service = cloudConfigurationMigrationService(db, { runtimeEnv: env });
if (migrate) await service.migrate();
if (recheckRevision !== undefined) await service.recheckLegacyExternalRegistrations(recheckRevision);
const report = await qualifyCloudConfigurationForCurrentImage(db, { runtimeEnv: env, expectedPackageEvidenceDigest });
if (!report) throw new Error("Cloud configuration qualification requires a managed instance");
process.stdout.write(`${JSON.stringify(report, null, 2)}\n`);
return report.configurationPackagesEligible && (!expectedPackageEvidenceDigest || report.samePackageOverlapEligible) ? 0 : 2;
} finally {
await db.$client.end({ timeout: 5 });
}
}
if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) {
runCloudConfigurationQualification(process.argv.slice(2)).then((code) => { process.exitCode = code; }).catch((error) => {
process.stderr.write(`${error instanceof Error ? error.message : "Cloud configuration qualification failed"}\n`);
process.exitCode = 1;
});
}
+37 -27
View File
@@ -19,11 +19,13 @@ import { readFile } from "node:fs/promises";
import path from "node:path";
import { promisify } from "node:util";
import { Router } from "express";
import { companies, type Db } from "@paperclipai/db";
import { adapterSettingsService } from "../services/adapter-settings.js";
import { logActivity, publishActivity, type ActivityPublication } from "../services/activity-log.js";
import {
listServerAdapters,
findServerAdapter,
findActiveServerAdapter,
listEnabledServerAdapters,
registerServerAdapter,
resolveExternalAdapterRegistration,
unregisterServerAdapter,
@@ -37,8 +39,6 @@ import {
removeAdapterPlugin,
getAdapterPluginByType,
getAdapterPluginsDir,
getDisabledAdapterTypes,
setAdapterDisabled,
} from "../services/adapter-plugin-store.js";
import type { AdapterPluginRecord } from "../services/adapter-plugin-store.js";
import type { ServerAdapterModule, AdapterConfigSchema } from "../adapters/types.js";
@@ -49,9 +49,9 @@ import type {
import { loadExternalAdapterPackage, getUiParserSource, getOrExtractUiParserSource, reloadExternalAdapter } from "../adapters/plugin-loader.js";
import { logger } from "../middleware/logger.js";
import { badRequest, forbidden } from "../errors.js";
import { isCloudManagedInstance } from "../services/cloud-instance.js";
import { assertExternalAdapterCodeAllowed } from "../services/adapter-code-policy.js";
import { getHiddenSettings } from "../services/settings-visibility.js";
import { assertBoardOrgAccess, assertCompanyAccess, assertInstanceAdmin } from "./authz.js";
import { assertBoardOrgAccess, assertCompanyAccess, assertInstanceAdmin, getActorInfo } from "./authz.js";
import { BUILTIN_ADAPTER_TYPES } from "../adapters/builtin-adapter-types.js";
import { agentRunnerAvailability, type RunnerTarget } from "../services/agent-runner-selection.js";
import type { AgentRunnerAvailability } from "@paperclipai/shared";
@@ -60,22 +60,6 @@ import { resolveAntigravityCandidateAdmission } from "../services/native-runtime
const execFileAsync = promisify(execFile);
let warnedMalformedAntigravityAdmission = false;
/**
* Floor: on cloud-managed instances adapter code is bundled into the platform
* image; fetching and loading external adapter packages at runtime stays off
* for every actor, including instance admins. Adapter code executes in the
* server process, so a runtime install would let an instance admin read the
* platform trust anchors from the process environment (mirrors the
* bundled-only plugin install floor in plugin-install-guard.ts).
*/
function assertAdapterCodeInstallAllowed() {
if (isCloudManagedInstance()) {
throw forbidden("Adapter installation is platform-managed on cloud-managed instances", {
code: "adapter_install_platform_managed",
});
}
}
/**
* Floor: when the hosting operator hides the Adapters settings surface
* (`instance.adapters` in PAPERCLIP_HIDDEN_SETTINGS), adapter management
@@ -209,7 +193,7 @@ function buildAdapterInfo(adapter: ServerAdapterModule, externalRecord: AdapterP
loaded: true, // If it's in the registry, it's loaded
disabled: disabledSet.has(adapter.type),
capabilities: buildAdapterCapabilities(adapter),
...agentRunnerAvailability(adapter.type, target),
...agentRunnerAvailability(adapter.type, target, [...disabledSet]),
...(adapter.acp ? { acp: adapter.acp } : {}),
overriddenBuiltin: externalRecord ? BUILTIN_ADAPTER_TYPES.has(adapter.type) : undefined,
overridePaused: BUILTIN_ADAPTER_TYPES.has(adapter.type) ? isOverridePaused(adapter.type) : undefined,
@@ -265,11 +249,13 @@ function registerWithSessionManagement(adapter: ServerAdapterModule): void {
// ---------------------------------------------------------------------------
export function adapterRoutes(options: {
db?: Db;
getNativeRunnerEnabled?: () => Promise<boolean>;
getOpenAiDotEnabled?: () => Promise<boolean>;
resolveRunnerTarget?: (companyId: string, environmentId?: string) => Promise<RunnerTarget>;
} = {}) {
const router = Router();
const settings = adapterSettingsService(options.db);
/**
* GET /api/adapters
@@ -303,7 +289,7 @@ export function adapterRoutes(options: {
const externalRecords = new Map(
listAdapterPlugins().map((r) => [r.type, r]),
);
const disabledSet = new Set(getDisabledAdapterTypes());
const disabledSet = new Set(await settings.getDisabledTypes());
// One shared implementation, with independent provider rollouts. Explicit
// adapter-admin disabling still applies to both choices.
@@ -343,7 +329,7 @@ export function adapterRoutes(options: {
*/
router.post("/adapters/install", async (req, res) => {
assertInstanceAdmin(req);
assertAdapterCodeInstallAllowed();
assertExternalAdapterCodeAllowed();
assertAdapterManagementVisible();
const { packageName, isLocalPath = false, version } = req.body as AdapterInstallRequest;
@@ -477,7 +463,7 @@ export function adapterRoutes(options: {
}
const externalRecord = getAdapterPluginByType(adapterType);
const disabledSet = new Set(getDisabledAdapterTypes());
const disabledSet = new Set(await settings.getDisabledTypes());
res.json(buildAdapterInfo(adapter, externalRecord, disabledSet));
});
@@ -509,7 +495,28 @@ export function adapterRoutes(options: {
return;
}
const changed = setAdapterDisabled(adapterType, disabled);
const publications: ActivityPublication[] = [];
const changed = await settings.setDisabled(adapterType, disabled, async (tx, current) => {
const actor = getActorInfo(req);
const companyIds = await tx.select({ id: companies.id }).from(companies);
for (const company of companyIds) {
await logActivity(tx as Db, {
companyId: company.id,
...actor,
action: "instance.adapter.updated",
entityType: "instance_settings",
entityId: "adapter-settings:v1",
details: { adapterType, disabled, revision: current.revision },
}, publications);
}
});
for (const publication of publications) {
try {
publishActivity(publication);
} catch (error) {
logger.error({ err: error, companyId: publication.companyId }, "failed to publish adapter settings activity");
}
}
if (changed) {
logger.info({ type: adapterType, disabled }, "Adapter enabled/disabled");
@@ -528,6 +535,7 @@ export function adapterRoutes(options: {
*/
router.patch("/adapters/:type/override", async (req, res) => {
assertInstanceAdmin(req);
assertExternalAdapterCodeAllowed();
assertAdapterManagementVisible();
@@ -558,6 +566,7 @@ export function adapterRoutes(options: {
*/
router.delete("/adapters/:type", async (req, res) => {
assertInstanceAdmin(req);
assertExternalAdapterCodeAllowed();
assertAdapterManagementVisible();
const adapterType = req.params.type;
@@ -635,6 +644,7 @@ export function adapterRoutes(options: {
*/
router.post("/adapters/:type/reload", async (req, res) => {
assertInstanceAdmin(req);
assertExternalAdapterCodeAllowed();
assertAdapterManagementVisible();
const type = req.params.type;
@@ -688,7 +698,7 @@ export function adapterRoutes(options: {
// package name, but without the risk of losing the store record.
router.post("/adapters/:type/reinstall", async (req, res) => {
assertInstanceAdmin(req);
assertAdapterCodeInstallAllowed();
assertExternalAdapterCodeAllowed();
assertAdapterManagementVisible();
const type = req.params.type;
+2 -11
View File
@@ -154,7 +154,7 @@ import type {
} from "@paperclipai/adapter-utils";
import { evaluateCodexCredentialReadiness } from "@paperclipai/adapter-codex-local/server";
import type { AdapterAuthSignal, AdapterAuthSignalResponse, CodexAccountBindingClaim } from "@paperclipai/shared";
import { getDisabledAdapterTypes } from "../services/adapter-plugin-store.js";
import { adapterSettingsService } from "../services/adapter-settings.js";
import { skillVersionSelectionMap } from "../services/runtime-skill-selections.js";
import { isFixedClaudeOAuthBinding, secretService } from "../services/secrets.js";
import { authorizationDeniedDetails } from "../services/authorization.js";
@@ -1859,15 +1859,6 @@ export function agentRoutes(
* Adapter validation for the paths that CHOOSE a harness for a new agent
* (hire + create), as opposed to the paths that operate on an existing one.
*
* A disabled adapter is one this instance cannot run — most often because a
* declarative registry (PAPERCLIP_ADAPTERS) curated it out, which
* reconcileAdapterAvailability turns into a disabled type at boot. Registered
* but disabled still passes assertKnownAdapterType, so an agent could be
* created on it and then fail EVERY run at lease time with
* `Adapter "..." is not in the configured adapter registry` — an error that
* arrives minutes later, in a run log, with no way back to the choice that
* caused it. Refuse at selection time instead, and name what can be chosen.
*
* Existing agents on a now-disabled adapter are deliberately untouched
* (listEnabledServerAdapters documents the same rule: hidden from selection,
* still functional for agents that already use them).
@@ -1902,7 +1893,7 @@ export function agentRoutes(
);
}
}
const disabled = new Set(getDisabledAdapterTypes());
const disabled = new Set(await adapterSettingsService(db).getDisabledTypes());
if (!disabled.has(adapterType)) return adapterType;
const available = listServerAdapters()
.map((a) => a.type)
+12 -7
View File
@@ -32,6 +32,7 @@ import path from "node:path";
import fs from "node:fs";
import crypto from "node:crypto";
import type { Db } from "@paperclipai/db";
import type { PluginRecord } from "@paperclipai/shared";
import { pluginRegistryService } from "../services/plugin-registry.js";
import { logger } from "../middleware/logger.js";
import { assertCompanyAccess } from "./authz.js";
@@ -192,6 +193,7 @@ export interface PluginUiStaticRouteOptions {
* Defaults to the standard `~/.paperclip/plugins/` location.
*/
localPluginDir: string;
resolvePackageRoot?: (plugin: PluginRecord) => string;
}
/**
@@ -304,7 +306,7 @@ export function pluginUiStaticRoutes(db: Db, options: PluginUiStaticRouteOptions
if (typeof devUiUrl === "string" && devUiUrl.length > 0) {
// Dev proxy is only available in development mode
if (process.env.NODE_ENV === "production") {
if (options.resolvePackageRoot || process.env.NODE_ENV === "production") {
log.warn(
{ pluginId: plugin.id },
"plugin-ui-static: devUiUrl ignored in production",
@@ -405,12 +407,15 @@ export function pluginUiStaticRoutes(db: Db, options: PluginUiStaticRouteOptions
}
// Step 3: Resolve the plugin's UI directory
const uiDir = resolvePluginUiDir(
options.localPluginDir,
plugin.packageName,
manifest.entrypoints.ui,
plugin.packagePath,
);
let uiDir: string | null;
try {
uiDir = options.resolvePackageRoot
? path.resolve(options.resolvePackageRoot(plugin), manifest.entrypoints.ui)
: resolvePluginUiDir(options.localPluginDir, plugin.packageName, manifest.entrypoints.ui, plugin.packagePath);
} catch {
res.status(503).json({ error: "Plugin package is unavailable in the current image" });
return;
}
if (!uiDir) {
log.warn(
@@ -0,0 +1,10 @@
import { forbidden } from "../errors.js";
import { isCloudManagedInstance } from "./cloud-instance.js";
export function assertExternalAdapterCodeAllowed(): void {
if (isCloudManagedInstance()) {
throw forbidden("Adapter installation is platform-managed on cloud-managed instances", {
code: "adapter_install_platform_managed",
});
}
}
+12 -2
View File
@@ -15,6 +15,8 @@
import fs from "node:fs";
import path from "node:path";
import { resolvePaperclipHomeDir } from "../home-paths.js";
import { isCloudManagedInstance } from "./cloud-instance.js";
import { assertExternalAdapterCodeAllowed } from "./adapter-code-policy.js";
// ---------------------------------------------------------------------------
// Types
@@ -43,8 +45,8 @@ interface AdapterSettings {
// Paths
// ---------------------------------------------------------------------------
function adapterPluginPaths() {
const paperclipDir = resolvePaperclipHomeDir();
function adapterPluginPaths(homeOverride?: string) {
const paperclipDir = resolvePaperclipHomeDir(homeOverride);
return {
adapterPluginsDir: path.join(paperclipDir, "adapter-plugins"),
adapterPluginsStorePath: path.join(paperclipDir, "adapter-plugins.json"),
@@ -79,6 +81,7 @@ function ensureDirs(): string {
}
function readStore(): AdapterPluginRecord[] {
if (isCloudManagedInstance()) return [];
const { adapterPluginsStorePath } = adapterPluginPaths();
if (storeCache?.path === adapterPluginsStorePath) return storeCache.records;
try {
@@ -130,11 +133,16 @@ function writeSettings(settings: AdapterSettings): void {
// Public API
// ---------------------------------------------------------------------------
export function getAdapterPluginStorePath(homeOverride?: string): string {
return adapterPluginPaths(homeOverride).adapterPluginsStorePath;
}
export function listAdapterPlugins(): AdapterPluginRecord[] {
return readStore();
}
export function addAdapterPlugin(record: AdapterPluginRecord): void {
assertExternalAdapterCodeAllowed();
const store = [...readStore()];
const idx = store.findIndex((r) => r.type === record.type);
if (idx >= 0) {
@@ -146,6 +154,7 @@ export function addAdapterPlugin(record: AdapterPluginRecord): void {
}
export function removeAdapterPlugin(type: string): boolean {
assertExternalAdapterCodeAllowed();
const store = [...readStore()];
const idx = store.findIndex((r) => r.type === type);
if (idx < 0) return false;
@@ -159,6 +168,7 @@ export function getAdapterPluginByType(type: string): AdapterPluginRecord | unde
}
export function getAdapterPluginsDir(): string {
assertExternalAdapterCodeAllowed();
return ensureDirs();
}
@@ -32,8 +32,7 @@ describe("reconcileAdapterAvailability", () => {
]);
expect(result.enabled).toEqual(["opencode_local"]);
expect(result.disabled.sort()).toEqual(["claude_local", "pi_local"]);
expect(setAdapterDisabled).toHaveBeenCalledWith("claude_local", true);
expect(setAdapterDisabled).toHaveBeenCalledWith("opencode_local", false);
expect(setAdapterDisabled).not.toHaveBeenCalled();
});
it("throws when a declared adapter has no installed implementation", () => {
@@ -1,73 +1,8 @@
/**
* Declarative adapter-registry bootstrap.
*
* One source (`PAPERCLIP_ADAPTERS` inline JSON, or `PAPERCLIP_ADAPTERS_FILE` a
* path to a JSON file) feeds two consumers:
* 1. Availability: reconcile the file-backed disabled-set so the picker shows
* exactly the declared, enabled set (runs for any instance).
* 2. k8s runtime: the same registry rides on the Kubernetes environment config
* (see execution-policy-bootstrap) so the plugin resolves runtime
* image/envKeys/allowFqdns/probe/defaultEnv from it.
*
* Parsing is pure + fails loud on malformed/invalid config, mirroring
* execution-policy-bootstrap.
*/
import fs from "node:fs";
import { adapterRegistrySchema, type AdapterRegistryEntryParsed } from "@paperclipai/shared";
import type { AdapterRegistryEntryParsed } from "@paperclipai/shared";
import { logger } from "../middleware/logger.js";
import { listServerAdapters } from "../adapters/registry.js";
import { setAdapterDisabled } from "./adapter-plugin-store.js";
export { parseAdapterRegistryEnv, type AdapterRegistryEnv } from "./adapter-registry-config.js";
export type AdapterRegistryEnv = Record<string, string | undefined>;
/**
* Parse the declarative registry from env. Returns null when unconfigured
* (built-in defaults; local/OSS unchanged). Throws on malformed/invalid input.
*/
export function parseAdapterRegistryEnv(
env: AdapterRegistryEnv = process.env,
): AdapterRegistryEntryParsed[] | null {
const inline = env.PAPERCLIP_ADAPTERS?.trim();
const filePath = env.PAPERCLIP_ADAPTERS_FILE?.trim();
if (!inline && !filePath) return null;
let rawText: string;
if (inline) {
rawText = inline;
} else {
try {
rawText = fs.readFileSync(filePath as string, "utf-8");
} catch (err) {
throw new Error(
`PAPERCLIP_ADAPTERS_FILE could not be read at "${filePath}": ${(err as Error).message}`,
);
}
}
let parsed: unknown;
try {
parsed = JSON.parse(rawText);
} catch (err) {
throw new Error(`PAPERCLIP_ADAPTERS must be valid JSON: ${(err as Error).message}`);
}
const result = adapterRegistrySchema.safeParse(parsed);
if (!result.success) {
throw new Error(
`PAPERCLIP_ADAPTERS failed validation: ${result.error.issues
.map((i) => `${i.path.join(".")}: ${i.message}`)
.join("; ")}`,
);
}
return result.data;
}
/**
* Reconcile availability: every known server adapter NOT enabled in the declared
* registry is disabled; declared+enabled ones are enabled. Throws if a declared
* adapterType has no installed adapter (cannot offer a harness with no
* implementation). No-op when `registry` is null.
*/
export function reconcileAdapterAvailability(
registry: AdapterRegistryEntryParsed[] | null,
): { enabled: string[]; disabled: string[] } {
@@ -88,10 +23,9 @@ export function reconcileAdapterAvailability(
for (const type of knownTypes) {
const entry = declared.get(type);
const shouldEnable = entry !== undefined && entry.enabled !== false;
setAdapterDisabled(type, !shouldEnable);
(shouldEnable ? enabled : disabled).push(type);
}
logger.info({ enabled, disabled }, "reconciled adapter availability from PAPERCLIP_ADAPTERS");
logger.info({ enabled, disabled }, "validated adapter availability from PAPERCLIP_ADAPTERS");
return { enabled, disabled };
}
@@ -0,0 +1,47 @@
import fs from "node:fs";
import { adapterRegistrySchema, type AdapterRegistryEntryParsed } from "@paperclipai/shared";
export type AdapterRegistryEnv = Record<string, string | undefined>;
let cached: { inline: string | undefined; filePath: string | undefined; registry: AdapterRegistryEntryParsed[] } | undefined;
export function parseAdapterRegistryEnv(
env: AdapterRegistryEnv = process.env,
): AdapterRegistryEntryParsed[] | null {
const inline = env.PAPERCLIP_ADAPTERS?.trim();
const filePath = env.PAPERCLIP_ADAPTERS_FILE?.trim();
if (!inline && !filePath) return null;
if (cached && cached.inline === inline && cached.filePath === filePath) return structuredClone(cached.registry);
let rawText: string;
if (inline) {
rawText = inline;
} else {
try {
rawText = fs.readFileSync(filePath as string, "utf-8");
} catch (err) {
throw new Error(
`PAPERCLIP_ADAPTERS_FILE could not be read at "${filePath}": ${(err as Error).message}`,
);
}
}
let parsed: unknown;
try {
parsed = JSON.parse(rawText);
} catch (err) {
throw new Error(`PAPERCLIP_ADAPTERS must be valid JSON: ${(err as Error).message}`);
}
const result = adapterRegistrySchema.safeParse(parsed);
if (!result.success) {
throw new Error(
`PAPERCLIP_ADAPTERS failed validation: ${result.error.issues
.map((i) => `${i.path.join(".")}: ${i.message}`)
.join("; ")}`,
);
}
cached = { inline, filePath, registry: result.data };
return structuredClone(result.data);
}
@@ -0,0 +1,87 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
vi.mock("../adapters/registry.js", () => ({ listServerAdapters: () => [{ type: "codex_local" }, { type: "claude_local" }] }));
import { adapterSettingsService, type AdapterSettingsDb } from "./adapter-settings.js";
import { getDisabledAdapterTypes, setAdapterDisabled } from "./adapter-plugin-store.js";
import { parseAdapterRegistryEnv } from "./adapter-registry-config.js";
describe("local adapter settings compatibility", () => {
let directory: string;
beforeEach(() => {
directory = fs.mkdtempSync(path.join(os.tmpdir(), "adapter-settings-local-"));
vi.stubEnv("PAPERCLIP_HOME", directory);
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", undefined);
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", undefined);
vi.stubEnv("PAPERCLIP_ADAPTERS", undefined);
vi.stubEnv("PAPERCLIP_ADAPTERS_FILE", undefined);
});
afterEach(() => {
vi.restoreAllMocks();
vi.unstubAllEnvs();
fs.rmSync(directory, { recursive: true, force: true });
});
it("reads and writes the existing file without a database", async () => {
fs.writeFileSync(path.join(directory, "adapter-settings.json"), JSON.stringify({ disabledTypes: ["claude_local"] }));
const service = adapterSettingsService(undefined);
expect(await service.getDisabledTypes()).toEqual(["claude_local"]);
expect(await service.setDisabled("codex_local", true)).toBe(true);
expect(getDisabledAdapterTypes()).toEqual(["claude_local", "codex_local"]);
expect(setAdapterDisabled("claude_local", false)).toBe(true);
expect(await service.getDisabledTypes()).toEqual(["codex_local"]);
expect(JSON.parse(fs.readFileSync(path.join(directory, "adapter-settings.json"), "utf8"))).toEqual({ disabledTypes: ["codex_local"] });
});
it("loads an operator file once and leaves the saved file intact", async () => {
setAdapterDisabled("codex_local", true);
const file = path.join(directory, "operator-adapters.json");
fs.writeFileSync(file, JSON.stringify([{ adapterType: "codex_local", enabled: true }]));
const env = { PAPERCLIP_ADAPTERS_FILE: file };
parseAdapterRegistryEnv(env);
const read = vi.spyOn(fs, "readFileSync");
const service = adapterSettingsService(undefined, { runtimeEnv: env });
expect(await service.getDisabledTypes()).toEqual(["claude_local"]);
expect(await service.getDisabledTypes()).toEqual(["claude_local"]);
expect(read).not.toHaveBeenCalled();
expect(getDisabledAdapterTypes()).toEqual(["codex_local"]);
});
it("restores the saved choice when its audit fails and audits a later retry", async () => {
const transaction = vi.fn(async (callback: (tx: AdapterSettingsDb) => Promise<unknown>) => callback(db));
const db = { transaction } as unknown as AdapterSettingsDb;
const service = adapterSettingsService(db);
const audit = vi.fn().mockRejectedValueOnce(new Error("audit unavailable")).mockResolvedValue(undefined);
await expect(service.setDisabled("codex_local", true, audit)).rejects.toThrow("audit unavailable");
expect(await service.getDisabledTypes()).toEqual([]);
expect(JSON.parse(fs.readFileSync(path.join(directory, "adapter-settings.json"), "utf8"))).toEqual({ disabledTypes: [] });
expect(await service.setDisabled("codex_local", true, audit)).toBe(true);
expect(await service.getDisabledTypes()).toEqual(["codex_local"]);
expect(audit).toHaveBeenCalledTimes(2);
expect(transaction).toHaveBeenCalledTimes(2);
});
it("keeps a failed audit rollback from overwriting a concurrent save", async () => {
const db = { transaction: async (callback: (tx: AdapterSettingsDb) => Promise<unknown>) => callback(db) } as unknown as AdapterSettingsDb;
const first = adapterSettingsService(db);
const second = adapterSettingsService(db);
let failAudit!: (reason: Error) => void;
const auditWait = new Promise<void>((_resolve, reject) => { failAudit = reject; });
const audit = vi.fn(() => auditWait);
const rejected = expect(first.setDisabled("codex_local", true, audit)).rejects.toThrow("audit unavailable");
await vi.waitFor(() => expect(audit).toHaveBeenCalledOnce());
const nextAudit = vi.fn(async () => undefined);
const next = second.setDisabled("codex_local", true, nextAudit);
await Promise.resolve();
expect(nextAudit).not.toHaveBeenCalled();
failAudit(new Error("audit unavailable"));
await rejected;
expect(await next).toBe(true);
expect(await first.getDisabledTypes()).toEqual(["codex_local"]);
expect(nextAudit).toHaveBeenCalledOnce();
});
});
@@ -0,0 +1,228 @@
import express from "express";
import request from "supertest";
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { activityLog, applyPendingMigrations, companies, createDb, instanceSettings, type Db } from "@paperclipai/db";
import { eq, sql } from "drizzle-orm";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../__tests__/helpers/embedded-postgres.js";
const legacy = vi.hoisted(() => ({ get: vi.fn(() => [] as string[]), set: vi.fn() }));
vi.mock("./adapter-plugin-store.js", () => ({
getDisabledAdapterTypes: legacy.get,
setAdapterDisabled: legacy.set,
listAdapterPlugins: () => [],
getAdapterPluginByType: () => undefined,
}));
vi.mock("../adapters/registry.js", () => ({
listServerAdapters: () => [{ type: "codex_local" }, { type: "claude_local" }, { type: "paperclip_runner" }],
findServerAdapter: (type: string) => ({ type }),
findActiveServerAdapter: (type: string) => ({ type }),
isOverridePaused: () => false,
hasActiveAdapterOverride: () => false,
waitForExternalAdapters: async () => {},
}));
vi.mock("../adapters/plugin-loader.js", () => ({}));
import { adapterSettingsService, initializeAdapterSettings } from "./adapter-settings.js";
import { adapterRoutes } from "../routes/adapters.js";
import { errorHandler } from "../middleware/error-handler.js";
import { resolveNewAgentRunnerForCompany } from "./agent-runner-selection.js";
import * as activities from "./activity-log.js";
const externalDatabaseUrl = process.env.PAPERCLIP_ADAPTER_SETTINGS_TEST_DATABASE_URL;
const support = externalDatabaseUrl ? { supported: true } : await getEmbeddedPostgresTestSupport();
(support.supported ? describe : describe.skip)("durable managed adapter settings", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: Db;
let peer: Db;
let directory: string;
const managed = { runtimeEnv: { PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN: "synthetic-test-token" } };
const service = () => adapterSettingsService(db, managed);
beforeAll(async () => {
database = externalDatabaseUrl
? { connectionString: externalDatabaseUrl, cleanup: async () => {} }
: await startEmbeddedPostgresTestDatabase("adapter-settings-");
if (externalDatabaseUrl) await applyPendingMigrations(externalDatabaseUrl);
db = createDb(database.connectionString);
peer = createDb(database.connectionString);
await db.insert(companies).values({ name: "Adapter settings test", issuePrefix: "AST" });
}, 90_000);
afterAll(async () => {
await Promise.all([db?.$client.end({ timeout: 0 }), peer?.$client.end({ timeout: 0 })]);
await database?.cleanup();
}, 60_000);
beforeEach(async () => {
directory = await fs.mkdtemp(path.join(os.tmpdir(), "adapter-settings-import-"));
await db.delete(instanceSettings).where(eq(instanceSettings.singletonKey, "adapter-settings:v1"));
legacy.get.mockClear();
legacy.set.mockClear();
vi.stubEnv("PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN", "synthetic-test-token");
});
afterEach(async () => {
vi.restoreAllMocks();
vi.unstubAllEnvs();
await fs.rm(directory, { recursive: true, force: true });
});
function app() {
const app = express();
app.use(express.json());
app.use((req, _res, next) => {
req.actor = { type: "board", source: "local_implicit", userId: "local-board", isInstanceAdmin: true };
next();
});
app.use(adapterRoutes({ db }));
app.use(errorHandler);
return app;
}
it("persists across independent connections and reconstructed stores without file reads", async () => {
await expect(service().getDisabledTypes()).rejects.toThrow("not been initialized");
await service().seedIfAbsent({ disabledTypes: [] });
expect(await service().setDisabled("codex_local", true)).toBe(true);
expect(await adapterSettingsService(peer, managed).getDisabledTypes()).toEqual(["codex_local"]);
await db.$client.end({ timeout: 0 });
db = createDb(database.connectionString);
expect(await service().inspectStored()).toEqual({ version: 1, revision: 1, disabledTypes: ["codex_local"] });
expect(legacy.get).not.toHaveBeenCalled();
expect(legacy.set).not.toHaveBeenCalled();
});
it("acknowledges the committed save when its live activity event fails", async () => {
await service().seedIfAbsent({ disabledTypes: [] });
const publication = vi.spyOn(activities, "publishActivity").mockImplementation(() => { throw new Error("event unavailable"); });
const response = await request(app()).patch("/adapters/codex_local").send({ disabled: true });
expect(response.status, JSON.stringify(response.body)).toBe(200);
expect(publication).toHaveBeenCalled();
expect(await adapterSettingsService(peer, managed).getDisabledTypes()).toEqual(["codex_local"]);
const events = await peer.select().from(activityLog).where(eq(activityLog.action, "instance.adapter.updated"));
expect(events.some((event) => event.details?.adapterType === "codex_local")).toBe(true);
});
it("preserves simultaneous first writes and concurrent removals from other connections", async () => {
const writers = [service(), adapterSettingsService(peer, managed)];
await Promise.all(Array.from({ length: 12 }, (_, index) => writers[index % 2].setDisabled(`adapter_${index}`, true)));
expect((await service().inspectStored())?.disabledTypes).toHaveLength(12);
await Promise.all([
writers[0].setDisabled("adapter_0", false),
writers[1].setDisabled("adapter_1", false),
writers[0].setDisabled("extra", true),
]);
const stored = await service().inspectStored();
expect(stored?.revision).toBe(15);
expect(stored?.disabledTypes).not.toContain("adapter_0");
expect(stored?.disabledTypes).not.toContain("adapter_1");
expect(stored?.disabledTypes).toContain("extra");
expect(await writers[1].setDisabled("extra", true)).toBe(false);
expect((await service().inspectStored())?.revision).toBe(15);
});
it("imports legacy choices once before admission and ignores stale files after the DB record exists", async () => {
const runtimeEnv = { ...managed.runtimeEnv, PAPERCLIP_HOME: directory };
const file = path.join(directory, "adapter-settings.json");
await fs.writeFile(file, JSON.stringify({ disabledTypes: ["codex_local", "future_adapter"] }));
const results = await Promise.all([
initializeAdapterSettings(db, { runtimeEnv }),
initializeAdapterSettings(peer, { runtimeEnv }),
]);
expect(results.filter((result) => result?.seeded)).toHaveLength(1);
expect(await service().getDisabledTypes()).toEqual(["codex_local", "future_adapter"]);
await service().setDisabled("codex_local", false);
await fs.writeFile(file, "invalid stale JSON");
expect((await initializeAdapterSettings(peer, { runtimeEnv }))?.seeded).toBe(false);
expect(await service().getDisabledTypes()).toEqual(["future_adapter"]);
});
it("uses defaults only for an absent legacy file and rejects malformed or oversized input", async () => {
const runtimeEnv = { ...managed.runtimeEnv, PAPERCLIP_HOME: directory };
const file = path.join(directory, "adapter-settings.json");
await fs.writeFile(file, JSON.stringify({ disabledTypes: [42] }));
await expect(initializeAdapterSettings(db, { runtimeEnv })).rejects.toThrow("Persisted adapter settings are invalid");
expect(await service().inspectStored()).toBeNull();
await fs.writeFile(file, Buffer.alloc(65_537));
await expect(initializeAdapterSettings(db, { runtimeEnv })).rejects.toThrow("size limit");
expect(await service().inspectStored()).toBeNull();
await fs.unlink(file);
expect((await initializeAdapterSettings(db, { runtimeEnv }))?.current.disabledTypes).toEqual([]);
expect(await service().getDisabledTypes()).toEqual([]);
});
it("composes atomic import and receipts with the caller transaction", async () => {
await expect(db.transaction(async (tx) => {
expect(await service().seedIfAbsent({ disabledTypes: ["codex_local"] }, tx)).toEqual({
seeded: true, current: { version: 1, revision: 0, disabledTypes: ["codex_local"] },
});
throw new Error("receipt failed");
})).rejects.toThrow("receipt failed");
expect(await service().inspectStored()).toBeNull();
await Promise.all([
service().seedIfAbsent({ disabledTypes: ["codex_local"] }),
adapterSettingsService(peer, managed).seedIfAbsent({ disabledTypes: ["claude_local"] }),
]).then((results) => {
expect(results.filter((result) => result.seeded)).toHaveLength(1);
expect(results[0].current).toEqual(results[1].current);
});
});
it("applies operator restrictions without rewriting saved choices", async () => {
await service().setDisabled("codex_local", true);
const env = { ...managed.runtimeEnv, PAPERCLIP_ADAPTERS: JSON.stringify([{ adapterType: "codex_local", enabled: true }]) };
const overlay = adapterSettingsService(peer, { runtimeEnv: env });
expect(await overlay.getDisabledTypes()).toEqual(["claude_local", "paperclip_runner"]);
expect(await overlay.inspectStored()).toEqual(await service().inspectStored());
await expect(overlay.setDisabled("claude_local", false)).rejects.toThrow("hosting operator");
await expect(overlay.setDisabled("codex_local", true)).rejects.toThrow("hosting operator");
expect(await overlay.getDisabledTypes()).toEqual(["claude_local", "paperclip_runner"]);
expect(await service().getDisabledTypes()).toEqual(["codex_local"]);
});
it("refreshes the disabled harness at each new-agent admission", async () => {
await service().seedIfAbsent({ disabledTypes: [] });
const input = { adapterType: "codex_local", runner: "legacy" as const };
expect((await resolveNewAgentRunnerForCompany(db, "company", input)).adapterType).toBe("codex_local");
await adapterSettingsService(peer, managed).setDisabled("codex_local", true);
await expect(resolveNewAgentRunnerForCompany(db, "company", input)).rejects.toMatchObject({ status: 422, details: { code: "agent_runner_unavailable" } });
await adapterSettingsService(peer, managed).setDisabled("codex_local", false);
expect((await resolveNewAgentRunnerForCompany(db, "company", input)).adapterType).toBe("codex_local");
});
it("rolls back the setting when its audit write fails", async () => {
await expect(service().setDisabled("codex_local", true, async () => { throw new Error("audit failed"); })).rejects.toThrow("audit failed");
expect(await service().inspectStored()).toBeNull();
await expect(adapterSettingsService(peer, managed).getDisabledTypes()).rejects.toThrow("not been initialized");
});
it("publishes catalog changes across stores and records the instance mutation", async () => {
const res = await request(app()).patch("/adapters/codex_local").send({ disabled: true });
expect(res.status).toBe(200);
expect(res.body).toEqual({ type: "codex_local", disabled: true, changed: true });
const catalog = await request(app()).get("/adapters");
expect(catalog.status).toBe(200);
expect(catalog.body.find((entry: { type: string }) => entry.type === "codex_local").disabled).toBe(true);
const [audit] = await peer.select().from(activityLog).where(eq(activityLog.action, "instance.adapter.updated"));
expect(audit.details).toMatchObject({ adapterType: "codex_local", disabled: true, revision: 1 });
});
it("returns failure when PostgreSQL rejects the update and keeps the durable choice", async () => {
await service().seedIfAbsent({ disabledTypes: [] });
await db.execute(sql`CREATE FUNCTION reject_adapter_settings() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN RAISE EXCEPTION 'test adapter settings failure'; END $$`);
await db.execute(sql`CREATE TRIGGER reject_adapter_settings BEFORE UPDATE ON instance_settings FOR EACH ROW EXECUTE FUNCTION reject_adapter_settings()`);
try {
const res = await request(app()).patch("/adapters/codex_local").send({ disabled: true });
expect(res.status).toBe(500);
expect(await adapterSettingsService(peer, managed).getDisabledTypes()).toEqual([]);
} finally {
await db.execute(sql`DROP TRIGGER reject_adapter_settings ON instance_settings`);
await db.execute(sql`DROP FUNCTION reject_adapter_settings()`);
}
});
it("fails closed on malformed stored settings and unavailable database", async () => {
await db.insert(instanceSettings).values({ singletonKey: "adapter-settings:v1", general: { version: 2, disabledTypes: [] } });
await expect(service().getDisabledTypes()).rejects.toThrow("Persisted adapter settings are invalid");
await expect(adapterSettingsService(undefined, managed).getDisabledTypes()).rejects.toThrow("database is required");
expect(legacy.get).not.toHaveBeenCalled();
});
});
+149
View File
@@ -0,0 +1,149 @@
import type { Db } from "@paperclipai/db";
import { instanceSettings } from "@paperclipai/db";
import { eq } from "drizzle-orm";
import { open } from "node:fs/promises";
import { constants } from "node:fs";
import path from "node:path";
import { z } from "zod";
import { listServerAdapters } from "../adapters/registry.js";
import { forbidden } from "../errors.js";
import { resolvePaperclipHomeDir } from "../home-paths.js";
import { getDisabledAdapterTypes, setAdapterDisabled } from "./adapter-plugin-store.js";
import { parseAdapterRegistryEnv } from "./adapter-registry-config.js";
import { isCloudManagedInstance } from "./cloud-instance.js";
const SINGLETON_KEY = "adapter-settings:v1";
let localSettingsMutation: Promise<unknown> = Promise.resolve();
type Transaction = Parameters<Parameters<Db["transaction"]>[0]>[0];
export type AdapterSettingsDb = Pick<Db | Transaction, "select" | "insert" | "update" | "transaction">;
export type StoredAdapterSettings = { version: 1; revision: number; disabledTypes: string[] };
const storedSettingsSchema = z.object({
version: z.literal(1),
revision: z.number().int().nonnegative(),
disabledTypes: z.array(z.string().refine((type) => type.trim().length > 0)),
});
function parseStored(value: Record<string, unknown>): StoredAdapterSettings {
const parsed = storedSettingsSchema.safeParse(value);
if (!parsed.success) throw new Error("Persisted adapter settings are invalid");
return { ...parsed.data, disabledTypes: [...new Set(parsed.data.disabledTypes)].sort() };
}
export function effectiveDisabledAdapterTypes(saved: string[], knownTypes: string[] | (() => string[]), env: Record<string, string | undefined> = process.env): string[] {
const registry = parseAdapterRegistryEnv(env);
if (!registry) return [...saved];
const enabled = new Set(registry.filter((entry) => entry.enabled !== false).map((entry) => entry.adapterType));
const types = typeof knownTypes === "function" ? knownTypes() : knownTypes;
return [...new Set(types.filter((type) => !enabled.has(type)))].sort();
}
export function adapterSettingsService(db: AdapterSettingsDb | undefined, options: { runtimeEnv?: Record<string, string | undefined> } = {}) {
const env = options.runtimeEnv ?? process.env;
const managed = isCloudManagedInstance(env);
function requireDb(): AdapterSettingsDb {
if (!db) throw new Error("A database is required for managed adapter settings");
return db;
}
async function inspectStored(handle: AdapterSettingsDb = requireDb()): Promise<StoredAdapterSettings | null> {
const [row] = await handle.select({ general: instanceSettings.general }).from(instanceSettings)
.where(eq(instanceSettings.singletonKey, SINGLETON_KEY)).limit(1);
return row ? parseStored(row.general) : null;
}
async function seedIfAbsent(value: { disabledTypes: string[] }, handle: AdapterSettingsDb = requireDb()) {
const snapshot = parseStored({ version: 1, revision: 0, disabledTypes: value.disabledTypes });
const inserted = await handle.insert(instanceSettings).values({ singletonKey: SINGLETON_KEY, general: snapshot })
.onConflictDoNothing({ target: instanceSettings.singletonKey }).returning({ id: instanceSettings.id });
const current = await inspectStored(handle);
if (!current) throw new Error("Adapter settings seed was not persisted");
return { seeded: inserted.length > 0, current };
}
return {
inspectStored,
seedIfAbsent,
async getDisabledTypes(): Promise<string[]> {
const current = managed ? await inspectStored() : null;
if (managed && !current) throw new Error("Managed adapter settings have not been initialized");
const stored = managed ? current!.disabledTypes : getDisabledAdapterTypes();
return effectiveDisabledAdapterTypes(stored, () => listServerAdapters().map((adapter) => adapter.type), env);
},
async setDisabled(type: string, disabled: boolean, afterChange?: (handle: AdapterSettingsDb, current: StoredAdapterSettings) => Promise<void>): Promise<boolean> {
if (!type.trim()) throw new Error("Adapter type is required");
const registry = parseAdapterRegistryEnv(env);
if (registry && disabled === registry.some((entry) => entry.adapterType === type && entry.enabled !== false)) {
throw forbidden("Adapter availability is managed by the hosting operator", { code: "settings_operator_managed" });
}
if (!managed) {
const mutation = localSettingsMutation.then(async () => {
const previous = getDisabledAdapterTypes().includes(type);
const changed = setAdapterDisabled(type, disabled);
if (!changed || !db || !afterChange) return changed;
try {
await db.transaction(async (tx) => {
await afterChange(tx, { version: 1, revision: 0, disabledTypes: getDisabledAdapterTypes() });
});
} catch (error) {
setAdapterDisabled(type, previous);
throw error;
}
return true;
});
localSettingsMutation = mutation.catch(() => undefined);
return mutation;
}
return requireDb().transaction(async (tx) => {
await seedIfAbsent({ disabledTypes: [] }, tx);
const [row] = await tx.select().from(instanceSettings).where(eq(instanceSettings.singletonKey, SINGLETON_KEY)).for("update");
if (!row) throw new Error("Adapter settings are unavailable");
const current = parseStored(row.general);
const types = new Set(current.disabledTypes);
if (types.has(type) === disabled) return false;
if (disabled) types.add(type); else types.delete(type);
const next: StoredAdapterSettings = { version: 1, revision: current.revision + 1, disabledTypes: [...types].sort() };
await tx.update(instanceSettings).set({ general: next, updatedAt: new Date() }).where(eq(instanceSettings.id, row.id));
await afterChange?.(tx, next);
return true;
});
},
};
}
export type AdapterSettingsService = Pick<ReturnType<typeof adapterSettingsService>, "getDisabledTypes" | "setDisabled">;
export async function initializeAdapterSettings(db: AdapterSettingsDb, options: { runtimeEnv?: Record<string, string | undefined> } = {}) {
const env = options.runtimeEnv ?? process.env;
if (!isCloudManagedInstance(env)) return null;
const service = adapterSettingsService(db, options);
const current = await service.inspectStored();
if (current) return { seeded: false, current };
return service.seedIfAbsent(await readLegacyAdapterSettings(env) ?? { disabledTypes: [] });
}
export async function readLegacyAdapterSettings(env: Record<string, string | undefined> = process.env): Promise<{ disabledTypes: string[] } | null> {
const filePath = path.join(resolvePaperclipHomeDir(env.PAPERCLIP_HOME), "adapter-settings.json");
try {
const file = await open(filePath, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK);
try {
const stat = await file.stat();
if (!stat.isFile() || stat.size > 65_536) throw new Error("Legacy adapter settings exceed the size limit or are not a regular file");
const buffer = Buffer.alloc(65_537);
let total = 0;
while (total < buffer.length) {
const { bytesRead } = await file.read(buffer, total, buffer.length - total, total);
if (bytesRead === 0) break;
total += bytesRead;
}
if (total > 65_536) throw new Error("Legacy adapter settings exceed the size limit");
const value = JSON.parse(buffer.subarray(0, total).toString("utf8"));
return { disabledTypes: parseStored({ version: 1, revision: 0, disabledTypes: value?.disabledTypes }).disabledTypes };
} finally {
await file.close();
}
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error;
return null;
}
}
@@ -3,6 +3,10 @@ import { agentHarnessType, agentRunner, paperclipRunnerProfileForHarness, paperc
const state = vi.hoisted(() => ({ disabled: [] as string[], overridden: new Set<string>(), settings: {} as Record<string, unknown>, getEnvironment: vi.fn(), bindings: vi.fn(), activeTemplate: vi.fn().mockResolvedValue(null), secretPaths: vi.fn().mockResolvedValue(new Set(["apiKey"])), managedEnvironment: vi.fn(), kubernetesEnvironment: vi.fn(), ensureKubernetesEnvironment: vi.fn(), ssh: vi.fn(), resolveEnvironment: vi.fn(), sandboxQualification: vi.fn(), pluginByKey: vi.fn() }));
vi.mock("../adapters/registry.js", () => ({ findActiveServerAdapter: (type: string) => ({ type }), hasActiveAdapterOverride: (type: string) => state.overridden.has(type), waitForExternalAdapters: async () => {} }));
vi.mock("./adapter-plugin-store.js", () => ({ getDisabledAdapterTypes: () => state.disabled }));
vi.mock("./adapter-settings.js", async (original) => ({
...await original(),
adapterSettingsService: () => ({ getDisabledTypes: async () => state.disabled }),
}));
vi.mock("./instance-settings.js", () => ({ instanceSettingsService: () => ({ get: async () => state.settings }) }));
vi.mock("./environments.js", () => ({ environmentService: () => ({ getById: state.getEnvironment, listBoundCompanyIds: state.bindings, findManagedSandboxEnvironment: state.managedEnvironment, findKubernetesEnvironment: state.kubernetesEnvironment, ensureKubernetesEnvironment: state.ensureKubernetesEnvironment }) }));
vi.mock("./environment-custom-images.js", () => ({ environmentCustomImageService: () => ({ getActiveTemplate: state.activeTemplate }) }));
@@ -351,7 +355,7 @@ describe("server-owned Codex runner selection", () => {
state.settings = { general: { executionMode: "kubernetes" } };
const selected = await resolveAgentRunnerTargetForCompany({} as never, "company");
expect(selected).toEqual({ driver: "sandbox" });
expect(agentRunnerAvailability("codex_local", selected)).toEqual({ supportedRunners: ["legacy"], defaultRunner: "legacy" });
expect(agentRunnerAvailability("codex_local", selected, state.disabled)).toEqual({ supportedRunners: ["legacy"], defaultRunner: "legacy" });
expect(state.kubernetesEnvironment).toHaveBeenCalledWith("company");
expect(state.ensureKubernetesEnvironment).not.toHaveBeenCalled();
expect(state.resolveEnvironment).not.toHaveBeenCalled();
@@ -477,14 +481,14 @@ describe("server-owned Codex runner selection", () => {
config: { provider: "daytona", snapshot: "replacement-snapshot" }, metadata: { managedByPaperclip: true } });
state.resolveEnvironment.mockRejectedValue(new Error("Saved provider secret is unavailable."));
const selected = await resolveAgentRunnerTargetForCompany({} as never, "company", "sandbox");
expect(agentRunnerAvailability(adapterType, selected).defaultRunner).toBe("paperclip");
expect(agentRunnerAvailability(adapterType, selected, state.disabled).defaultRunner).toBe("paperclip");
expect(await resolveNewAgentRunnerForCompany({} as never, "company", {
adapterType, defaultEnvironmentId: "sandbox", adapterConfig: { model: adapterType === "opencode_local" ? "provider/model" : undefined },
})).toMatchObject({ adapterType: "paperclip_runner" });
expect(state.sandboxQualification).not.toHaveBeenCalled();
expect(state.resolveEnvironment).not.toHaveBeenCalled();
state.disabled.push("paperclip_runner");
expect(agentRunnerAvailability(adapterType, selected).defaultRunner).toBe("legacy");
expect(agentRunnerAvailability(adapterType, selected, state.disabled).defaultRunner).toBe("legacy");
});
it.each(["self-hosted", "custom-environment", "captured-template", "stale-marker"])("keeps qualification for a %s sandbox", async kind => {
vi.stubEnv("PAPERCLIP_MANAGED_CONFIG", kind === "self-hosted" ? undefined : JSON.stringify({ ...managedCloudConfig,
@@ -495,7 +499,7 @@ describe("server-owned Codex runner selection", () => {
metadata: { managedByPaperclip: kind !== "custom-environment" } });
state.activeTemplate.mockResolvedValue(kind === "captured-template" ? { id: "template", provider: "daytona", templateRef: "custom-snapshot" } : null);
const selected = await resolveAgentRunnerTargetForCompany({} as never, "company", "sandbox");
expect(agentRunnerAvailability("codex_local", selected).defaultRunner).toBe("legacy");
expect(agentRunnerAvailability("codex_local", selected, state.disabled).defaultRunner).toBe("legacy");
expect(state.sandboxQualification).toHaveBeenCalled();
});
it("ignores a stale captured image and preserves a matching custom image without resolving credentials", async () => {
@@ -507,17 +511,17 @@ describe("server-owned Codex runner selection", () => {
sourceEnvironmentConfigFingerprint: fingerprint({ ...config, snapshot: "previous-snapshot" }, { excludePaths: ["apiKey"] }) };
state.activeTemplate.mockResolvedValue(template);
const current = await resolveAgentRunnerTargetForCompany({} as never, "company", "sandbox");
expect(agentRunnerAvailability("codex_local", current).defaultRunner).toBe("paperclip");
expect(agentRunnerAvailability("codex_local", current, state.disabled).defaultRunner).toBe("paperclip");
expect(await resolveNewAgentRunnerForCompany({} as never, "company", { adapterType: "codex_local", runner: "paperclip", defaultEnvironmentId: "sandbox" }))
.toMatchObject({ adapterType: "paperclip_runner" });
expect(state.sandboxQualification).not.toHaveBeenCalled();
state.activeTemplate.mockResolvedValue({ ...template, sourceEnvironmentConfigFingerprint: fingerprint(config, { excludePaths: ["apiKey"] }) });
const custom = await resolveAgentRunnerTargetForCompany({} as never, "company", "sandbox");
expect(agentRunnerAvailability("codex_local", custom).defaultRunner).toBe("legacy");
expect(agentRunnerAvailability("codex_local", custom, state.disabled).defaultRunner).toBe("legacy");
expect(state.sandboxQualification).toHaveBeenCalled();
state.getEnvironment.mockResolvedValue({ id: "sandbox", driver: "sandbox", status: "active", config: { ...config, timeoutMs: "invalid" }, metadata: { managedByPaperclip: true } });
const invalid = await resolveAgentRunnerTargetForCompany({} as never, "company", "sandbox");
expect(agentRunnerAvailability("codex_local", invalid).defaultRunner).toBe("legacy");
expect(agentRunnerAvailability("codex_local", invalid, state.disabled).defaultRunner).toBe("legacy");
expect(state.resolveEnvironment).not.toHaveBeenCalled();
});
it("checks company access before trusting a managed Cloud image", async () => {
+23 -10
View File
@@ -5,6 +5,9 @@ import { resolveClaudeModel } from "@paperclipai/adapter-claude-local";
import { DEFAULT_GROK_LOCAL_MODEL } from "@paperclipai/adapter-grok-local";
import { findActiveServerAdapter, hasActiveAdapterOverride, waitForExternalAdapters } from "../adapters/registry.js";
import { getDisabledAdapterTypes } from "./adapter-plugin-store.js";
import { adapterSettingsService, effectiveDisabledAdapterTypes } from "./adapter-settings.js";
import { listServerAdapters } from "../adapters/registry.js";
import { isCloudManagedInstance } from "./cloud-instance.js";
import { claudeRunnerUnsupportedSettings, PaperclipRunnerProviderProfileError, resolvePaperclipRunnerNativeProviderInput } from "./native-runtime/provider-profile.js";
import { forbidden, unprocessable } from "../errors.js";
import { runSshCommand } from "@paperclipai/adapter-utils/ssh";
@@ -24,6 +27,7 @@ export type RunnerTarget = {
export const AGENT_RUNNER_SSH_PLATFORM_COMMAND = "uname -s; uname -m; if paperclip_glibc_version=$(getconf GNU_LIBC_VERSION 2>/dev/null) && [ -n \"$paperclip_glibc_version\" ]; then printf 'glibc\\n%s\\n' \"$paperclip_glibc_version\"; else case \"$(ldd --version 2>&1)\" in *musl*) printf 'musl\\n' ;; *) printf 'unknown\\n' ;; esac; fi";
export interface AgentRunnerSelection {
adapterType?: string | null;
disabledTypes?: string[];
adapterConfig?: Record<string, unknown>;
runner?: AgentRunnerChoice;
target?: RunnerTarget;
@@ -32,8 +36,13 @@ export interface AgentRunnerSelection {
onConfigurationWarnings?: (warnings: AgentConfigurationWarning[]) => void;
}
function runnerAdaptersAvailable(harness: string, driver: string): boolean {
const disabled = new Set(getDisabledAdapterTypes());
function localDisabledTypes(): string[] {
if (isCloudManagedInstance()) throw new Error("Managed runner selection requires a fresh adapter settings snapshot");
return effectiveDisabledAdapterTypes(getDisabledAdapterTypes(), () => listServerAdapters().map((adapter) => adapter.type));
}
function runnerAdaptersAvailable(harness: string, driver: string, disabledTypes: string[]): boolean {
const disabled = new Set(disabledTypes);
return !!paperclipRunnerProfileForHarness(harness)
&& !disabled.has(harness) && !disabled.has("paperclip_runner")
&& !!findActiveServerAdapter(harness) && !!findActiveServerAdapter("paperclip_runner")
@@ -41,13 +50,13 @@ function runnerAdaptersAvailable(harness: string, driver: string): boolean {
&& isEnvironmentDriverSupportedForAdapter("paperclip_runner", driver);
}
export function agentRunnerAvailability(harness: string, target: RunnerTarget = localAgentRunnerTarget()): AgentRunnerAvailability {
export function agentRunnerAvailability(harness: string, target: RunnerTarget = localAgentRunnerTarget(), disabledTypes: string[] = localDisabledTypes()): AgentRunnerAvailability {
if (target.runnerTasks || target.managedCloudImage) {
return runnerAdaptersAvailable(harness, target.driver)
return runnerAdaptersAvailable(harness, target.driver, disabledTypes)
? { supportedRunners: target.runnerTasks ? ["paperclip"] : ["paperclip", "legacy"], defaultRunner: "paperclip" }
: { supportedRunners: ["legacy"], defaultRunner: "legacy" };
}
const supported = runnerAdaptersAvailable(harness, target.driver)
const supported = runnerAdaptersAvailable(harness, target.driver, disabledTypes)
&& (target.driver !== "sandbox" || Boolean(target.platform && target.architecture
&& (target.platform !== "linux" || (target.libc === "glibc" && target.glibcVersion))))
&& (["claude_local", "grok_local"].includes(harness)
@@ -166,11 +175,12 @@ export function resolveNewAgentRunner(input: AgentRunnerSelection): { adapterTyp
function resolveAgentRunner(input: AgentRunnerSelection): { adapterType: string; adapterConfig: Record<string, unknown> } {
const adapterType = input.adapterType ?? "process";
let config = { ...input.adapterConfig };
const disabledTypes = input.disabledTypes ?? localDisabledTypes();
const harness = agentHarnessType(adapterType, config);
const native = adapterType === "paperclip_runner";
if (paperclipRunnerProfileForHarness(adapterType) && getDisabledAdapterTypes().includes(adapterType)) throw unprocessable(`${adapterType === "grok_local" ? "Grok Build" : adapterType === "pi_local" ? "Pi" : adapterType === "opencode_local" ? "OpenCode" : adapterType === "claude_local" ? "Claude" : "Codex"} is disabled on this instance. Enable its adapter or choose another harness.`, { code: "agent_runner_unavailable" });
if (paperclipRunnerProfileForHarness(adapterType) && disabledTypes.includes(adapterType)) throw unprocessable(`${adapterType === "grok_local" ? "Grok Build" : adapterType === "pi_local" ? "Pi" : adapterType === "opencode_local" ? "OpenCode" : adapterType === "claude_local" ? "Claude" : "Codex"} is disabled on this instance. Enable its adapter or choose another harness.`, { code: "agent_runner_unavailable" });
if (native && (input.runner === undefined || input.runner === "paperclip" || (harness !== "codex_local" && input.runner !== "legacy"))) return { adapterType, adapterConfig: normalizeLegacyRunnerProvider(config) };
const availability = agentRunnerAvailability(harness, input.target);
const availability = agentRunnerAvailability(harness, input.target, disabledTypes);
const choice = input.runner ?? "auto";
const useNative = choice === "paperclip" || (choice === "auto" && availability.defaultRunner === "paperclip");
if (!useNative) {
@@ -318,13 +328,14 @@ function resolveAgentRunner(input: AgentRunnerSelection): { adapterType: string;
}
export async function resolveNewAgentRunnerForCompany(db: Db, companyId: string, input: AgentRunnerSelection & { defaultEnvironmentId?: string | null }) {
input = { ...input, disabledTypes: await adapterSettingsService(db).getDisabledTypes() };
if (!paperclipRunnerProfileForHarness(agentHarnessType(input.adapterType ?? "process", input.adapterConfig))
|| input.runner === "legacy" || (input.adapterType === "paperclip_runner" && (input.runner === undefined || input.runner === "paperclip"))) return resolveNewAgentRunner(input);
const target = await resolveAgentRunnerTargetForCompany(db, companyId, input.defaultEnvironmentId);
const target = await resolveAgentRunnerTargetForCompany(db, companyId, input.defaultEnvironmentId, input.disabledTypes);
return resolveNewAgentRunner({ ...input, target });
}
export async function resolveAgentRunnerTargetForCompany(db: Db, companyId: string, defaultEnvironmentId?: string | null): Promise<RunnerTarget> {
export async function resolveAgentRunnerTargetForCompany(db: Db, companyId: string, defaultEnvironmentId?: string | null, disabledTypes?: string[]): Promise<RunnerTarget> {
await waitForExternalAdapters();
const { environment, driver } = await resolveAgentRunnerEnvironmentForCompany(db, companyId, defaultEnvironmentId);
let target: RunnerTarget = driver === "local" ? localAgentRunnerTarget()
@@ -356,7 +367,9 @@ export async function resolveAgentRunnerTargetForCompany(db: Db, companyId: stri
const qualified = await readSandboxPlatformQualification(db, environment, companyId);
if (qualified) target = { driver, ...qualified };
}
if (driver === "ssh" && environment && ["codex_local", "claude_local", "opencode_local", "pi_local", "grok_local"].some(harness => runnerAdaptersAvailable(harness, driver))) {
const disabledForSsh = driver === "ssh" && environment
? disabledTypes ?? await adapterSettingsService(db).getDisabledTypes() : [];
if (driver === "ssh" && environment && ["codex_local", "claude_local", "opencode_local", "pi_local", "grok_local"].some(harness => runnerAdaptersAvailable(harness, driver, disabledForSsh))) {
const parsed = await resolveEnvironmentDriverConfigForRuntime(db, companyId, environment);
if (parsed.driver !== "ssh") throw unprocessable("The selected SSH environment is unavailable.");
let output: string;
@@ -0,0 +1,108 @@
import fs from "node:fs";
import path from "node:path";
import { isDeepStrictEqual } from "node:util";
import type { PaperclipPluginManifestV1, PluginRecord } from "@paperclipai/shared";
import type { ResolvedBundledPlugin } from "./bundled-plugins.js";
import { assertDistributionManifestCapabilities } from "./distribution-plugin-catalog.js";
export type BundledPluginRegistration = Pick<PluginRecord, "pluginKey" | "packageName" | "version" | "manifestJson">;
export interface BundledPluginPackageIdentity {
kind: "bundled_image";
catalogKey: string;
pluginKey: string;
packageName: string;
version: string;
digest?: string;
}
export type BundledPluginPackageInspection =
| { compatible: true; identity: BundledPluginPackageIdentity }
| { compatible: false; pluginKey: string; code: "incompatible_bundled_package" | "missing_manifest_evidence"; reason: string };
function containedEntrypoint(root: string, declared: unknown, directory: boolean): string {
const relative = typeof declared === "string" ? declared.replace(/^\.\//, "").replace(/\/$/, "") : "";
if (!relative || path.isAbsolute(relative) || path.win32.isAbsolute(relative) || relative.includes("\\") || relative.split("/").length > 32 || relative.split("/").some((part) => !part || part === "." || part === "..")) {
throw new Error("Bundled plugin entrypoint must stay inside its package");
}
const target = fs.realpathSync(path.join(root, relative));
const resolvedRelative = path.relative(root, target);
if (!resolvedRelative || resolvedRelative.startsWith("..") || path.isAbsolute(resolvedRelative)) {
throw new Error("Bundled plugin entrypoint escapes its package");
}
const stat = fs.statSync(target);
if (directory ? !stat.isDirectory() : !stat.isFile()) throw new Error("Bundled plugin entrypoint is not prebuilt");
return relative;
}
export function createBundledPluginPackageResolver(installs: readonly ResolvedBundledPlugin[]) {
const byKey = new Map(installs.map((install) => [install.pluginKey, install]));
function read(install: ResolvedBundledPlugin) {
const root = fs.realpathSync(install.localPath);
if (root !== path.resolve(install.localPath)) throw new Error("Bundled plugin package root changed after catalog resolution");
containedEntrypoint(root, "package.json", false);
const metadataPath = path.join(root, "package.json");
if (fs.statSync(metadataPath).size > 128 * 1024) throw new Error("Bundled plugin package metadata exceeds verification limits");
const pkg = JSON.parse(fs.readFileSync(metadataPath, "utf8"));
if (typeof pkg.name !== "string" || typeof pkg.version !== "string" || !pkg.paperclipPlugin) {
throw new Error("Bundled plugin package identity or entrypoints are missing");
}
if (install.distribution && pkg.version !== install.distribution.version) throw new Error("Bundled plugin package version does not match the catalog");
const manifest = containedEntrypoint(root, pkg.paperclipPlugin.manifest, false);
const worker = containedEntrypoint(root, pkg.paperclipPlugin.worker, false);
const ui = pkg.paperclipPlugin.ui === undefined ? undefined : containedEntrypoint(root, pkg.paperclipPlugin.ui, true);
if (install.distribution && (worker !== install.distribution.entrypoints.worker || ui !== install.distribution.entrypoints.ui)) {
throw new Error("Bundled plugin entrypoints do not match the verified package");
}
return { root, pkg, manifest, worker, ui };
}
function assertManifest(install: ResolvedBundledPlugin, manifest: PaperclipPluginManifestV1, previousManifest?: PaperclipPluginManifestV1) {
const bundle = read(install);
if (manifest.id !== install.pluginKey || manifest.version !== bundle.pkg.version) throw new Error("Bundled plugin manifest identity/version does not match the image");
const worker = containedEntrypoint(bundle.root, manifest.entrypoints.worker, false);
const ui = manifest.entrypoints.ui === undefined ? undefined : containedEntrypoint(bundle.root, manifest.entrypoints.ui, true);
if (worker !== bundle.worker || ui !== bundle.ui) throw new Error("Bundled plugin manifest entrypoints do not match the image");
assertDistributionManifestCapabilities(manifest);
if (previousManifest) {
const approved = new Set(previousManifest.capabilities);
if (manifest.capabilities.some((capability) => !approved.has(capability))) throw new Error("Bundled plugin capabilities require approval");
}
return bundle;
}
function resolve(plugin: BundledPluginRegistration): string {
const install = byKey.get(plugin.pluginKey);
if (!install) throw new Error(`Bundled plugin is absent or not selected in this deployment: ${plugin.pluginKey}`);
const bundle = assertManifest(install, plugin.manifestJson);
if (plugin.packageName !== bundle.pkg.name || plugin.version !== bundle.pkg.version) throw new Error("Bundled plugin registration identity/version does not match the image");
return bundle.root;
}
return {
resolve,
inspect(plugin: BundledPluginRegistration, verifiedManifest?: PaperclipPluginManifestV1): BundledPluginPackageInspection {
try {
resolve(plugin);
const install = byKey.get(plugin.pluginKey)!;
if (!verifiedManifest) return { compatible: false, pluginKey: plugin.pluginKey, code: "missing_manifest_evidence", reason: "Image manifest evidence is required" };
assertManifest(install, verifiedManifest);
if (!isDeepStrictEqual(verifiedManifest, plugin.manifestJson)) throw new Error("Approved plugin manifest does not match the image");
return { compatible: true, identity: {
kind: "bundled_image", catalogKey: install.key, pluginKey: plugin.pluginKey,
packageName: plugin.packageName, version: plugin.version,
...(install.distribution ? { digest: install.distribution.digest } : {}),
} };
} catch (error) {
return { compatible: false, pluginKey: plugin.pluginKey, code: "incompatible_bundled_package", reason: error instanceof Error ? error.message : String(error) };
}
},
assertActivation(input: { pluginKey?: string; packageRoot: string; manifest?: PaperclipPluginManifestV1; previousManifest?: PaperclipPluginManifestV1 }) {
const install = input.pluginKey ? byKey.get(input.pluginKey) : installs.find((candidate) => candidate.localPath === input.packageRoot);
if (!install || path.resolve(input.packageRoot) !== install.localPath) throw new Error("Bundled plugin is absent or not selected in this deployment");
read(install);
if (input.manifest) assertManifest(install, input.manifest, input.previousManifest);
},
};
}
@@ -0,0 +1,37 @@
import fs from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
export function resolveBundledPluginRuntimeLoader(
moduleDirectory = path.dirname(fileURLToPath(import.meta.url)),
): string {
let directory = fs.realpathSync(moduleDirectory);
let imageRoot: string | undefined;
for (let depth = 0; depth < 5; depth++) {
const metadataPath = path.join(directory, "package.json");
const metadata = fs.lstatSync(metadataPath, { throwIfNoEntry: false });
if (metadata?.isFile() && metadata.size <= 128 * 1024
&& JSON.parse(fs.readFileSync(metadataPath, "utf8")).name === "@paperclipai/server") {
imageRoot = path.dirname(directory);
break;
}
const parent = path.dirname(directory);
if (parent === directory) break;
directory = parent;
}
if (!imageRoot) throw new Error("Managed plugin server package is missing from the current image");
const declared = path.join(imageRoot, "cli/node_modules/tsx/dist/loader.mjs");
let loader: string;
try {
loader = fs.realpathSync(declared);
if (!fs.statSync(loader).isFile()) throw new Error("not a file");
} catch {
throw new Error("Managed plugin runtime loader is missing from the current image");
}
const roots = [path.join(imageRoot, "cli"), path.join(imageRoot, "node_modules")];
if (!roots.some((root) => {
const relative = path.relative(root, loader);
return relative !== "" && !relative.startsWith("..") && !path.isAbsolute(relative);
})) throw new Error("Managed plugin runtime loader escapes the current image");
return loader;
}
+13 -4
View File
@@ -1,6 +1,7 @@
import path from "node:path";
import fs from "node:fs";
import { isDeepStrictEqual } from "node:util";
import { isCloudManagedInstance } from "./cloud-instance.js";
import type { PaperclipPluginManifestV1 } from "@paperclipai/shared";
import { assertDistributionManifestCapabilities, readDistributionPluginCatalog, type DistributionPlugin } from "./distribution-plugin-catalog.js";
@@ -118,6 +119,13 @@ export const BUNDLED_PLUGIN_CATALOG: readonly BundledPluginCatalogEntry[] = [
*/
export const SELF_HOSTED_AUTO_INSTALL_KEYS: readonly string[] = ["kubernetes"];
export function resolveManagedBundledPluginKeys(
keys: readonly string[] | null | undefined,
env: Record<string, string | undefined> = process.env,
): readonly string[] | null {
return keys ?? (isCloudManagedInstance(env) ? [] : null);
}
export function resolveBundledCatalogRoot(
env: Record<string, string | undefined>,
): string {
@@ -203,7 +211,7 @@ export function resolveBundledPluginInstalls(
`bundled plugin "${key}" resolves to "${localPath}", outside the bundled catalog root "${opts.catalogRoot}"; refusing to start`,
);
}
resolved.push({ key: entry.key, pluginKey: entry.pluginKey, localPath });
resolved.push({ key: entry.key, pluginKey: entry.pluginKey, localPath: opts.enforceCatalogRoot ? canonicalize(localPath) : localPath });
}
return resolved;
}
@@ -269,6 +277,7 @@ async function reconcileBundledPluginManifest(
deps: BundledPluginProvisionerDeps,
bundleManifestExists: (localPath: string) => boolean,
verifiedManifest?: PaperclipPluginManifestV1,
preservePackagePath = false,
): Promise<"upgrade_pending" | undefined> {
try {
// Managed reinstalls take the install path instead; this branch means the
@@ -283,7 +292,7 @@ async function reconcileBundledPluginManifest(
const approved = new Set(existing.manifestJson.capabilities ?? []);
requiresApproval = bundleManifest.capabilities.some((capability) => !approved.has(capability));
}
const rebindPackage = install.distribution && existing.packagePath !== install.localPath && existing.status !== "uninstalled";
const rebindPackage = !preservePackagePath && install.distribution && existing.packagePath !== install.localPath && existing.status !== "uninstalled";
const refreshDistribution = install.distribution && !isDeepStrictEqual(bundleManifest, existing.manifestJson);
if (bundleManifest.version === existing.version && !rebindPackage && !requiresApproval && !refreshDistribution) return;
await deps.registry.update(existing.id, {
@@ -389,7 +398,7 @@ async function reenableErroredBundledPlugin(
export async function ensureBundledPlugins(
installs: readonly ResolvedBundledPlugin[],
deps: BundledPluginProvisionerDeps,
opts: { reinstallUninstalled: boolean },
opts: { reinstallUninstalled: boolean; preservePackagePath?: boolean },
): Promise<void> {
const bundleManifestExists = deps.bundleManifestExists ?? defaultBundleManifestExists;
for (const install of installs) {
@@ -411,7 +420,7 @@ export async function ensureBundledPlugins(
// existing install, because the auto-install below skips a present
// plugin. Distribution entries also replace a legacy/npm package path
// before loadAll resolves the worker. Configuration and status stay put.
const reconciledStatus = await reconcileBundledPluginManifest(existing, install, deps, bundleManifestExists, verifiedManifest);
const reconciledStatus = await reconcileBundledPluginManifest(existing, install, deps, bundleManifestExists, verifiedManifest, opts.preservePackagePath);
if (reconciledStatus === "upgrade_pending") continue;
if (existing.status === "error") {
await reenableErroredBundledPlugin(existing, install, deps);
@@ -0,0 +1,95 @@
import fs from "node:fs";
import path from "node:path";
import { createHash } from "node:crypto";
import { cloudConfigurationDigest } from "./cloud-configuration-evidence.js";
export function cloudBundledPackageDigest(packageRoot: string, options: { catalogRoot?: string } = {}): string {
const root = fs.realpathSync(packageRoot);
const roots = [{ key: "package", directory: root }];
let imageNodeModules: string | undefined;
if (options.catalogRoot) {
const catalogRoot = fs.realpathSync(options.catalogRoot);
const relative = path.relative(catalogRoot, root);
if (!relative || relative.startsWith("..") || path.isAbsolute(relative)) throw new Error("Image package is outside the selected catalog root");
imageNodeModules = path.resolve(catalogRoot, "../../node_modules");
roots.push(
{ key: "plugin-sdk", directory: path.join(catalogRoot, "sdk") },
{ key: "shared", directory: path.resolve(catalogRoot, "../shared") },
{ key: "image-node-modules", directory: imageNodeModules },
);
}
roots.sort((left, right) => right.directory.length - left.directory.length);
const inventory = new Map<string, [string, string, string, number]>();
const directories = new Set<string>();
const files = new Set<string>();
const buffer = Buffer.alloc(65_536);
let bytes = 0;
let entries = 0;
function logical(file: string): string {
for (const candidate of roots) {
const relative = path.relative(candidate.directory, file);
if (relative === "" || (!relative.startsWith("..") && !path.isAbsolute(relative))) {
if (relative.split(path.sep).length > 32) throw new Error("Image package exceeds the directory depth limit");
return `${candidate.key}/${relative.split(path.sep).join("/")}`;
}
}
throw new Error(options.catalogRoot ? "Image dependency escapes its authorized image roots" : "Image package symlink escapes its package");
}
function follow(target: string) {
logical(target);
if (imageNodeModules) {
const relative = path.relative(imageNodeModules, target).split(path.sep);
if ((relative.length === 1 && relative[0] === "")
|| (relative[0] === ".pnpm" && (relative.length === 1 || (relative.length === 2 && relative[1] === "node_modules")))) {
throw new Error("Image dependency must select an individual package");
}
if (relative[0] === ".pnpm" && relative.length > 2 && relative[1] && relative[1] !== "node_modules") {
walk(path.join(imageNodeModules, ".pnpm", relative[1]));
return;
}
}
walk(target);
}
function walk(file: string) {
const name = logical(file);
const stat = fs.lstatSync(file);
if (stat.isSymbolicLink()) {
const target = fs.realpathSync(file);
inventory.set(name, [name, "link", logical(target), stat.mode & 0o777]);
follow(target);
} else if (stat.isDirectory()) {
const real = fs.realpathSync(file);
logical(real);
if (directories.has(real)) return;
directories.add(real);
for (const child of fs.readdirSync(real).sort()) {
if (++entries > 50_000) throw new Error("Image package exceeds the entry limit");
walk(path.join(real, child));
}
} else if (stat.isFile()) {
const real = fs.realpathSync(file);
if (files.has(real)) return;
files.add(real);
if (path.basename(file) === "package.json" && stat.size > 128 * 1024) throw new Error("Image package metadata exceeds the byte limit");
if (bytes + stat.size > 256 * 1024 * 1024) throw new Error("Image package exceeds the byte limit");
const handle = fs.openSync(file, "r");
const hash = createHash("sha256");
try {
while (true) {
const count = fs.readSync(handle, buffer);
if (!count) break;
bytes += count;
if (bytes > 256 * 1024 * 1024) throw new Error("Image package exceeds the byte limit");
hash.update(buffer.subarray(0, count));
}
} finally { fs.closeSync(handle); }
inventory.set(name, [name, "file", `sha256:${hash.digest("hex")}`, stat.mode & 0o777]);
} else throw new Error("Image package contains a special file");
}
walk(root);
if (!inventory.size) throw new Error("Image package is empty");
return cloudConfigurationDigest([...inventory.values()].sort((left, right) => left[0] < right[0] ? -1 : left[0] > right[0] ? 1 : 0));
}
@@ -0,0 +1,74 @@
import { createHash } from "node:crypto";
import { constants } from "node:fs";
import { open } from "node:fs/promises";
import { z } from "zod";
function canonical(value: unknown): unknown {
if (Array.isArray(value)) return value.map(canonical);
if (value && typeof value === "object") {
return Object.fromEntries(Object.entries(value).sort(([left], [right]) => left < right ? -1 : left > right ? 1 : 0).map(([key, child]) => [key, canonical(child)]));
}
return value;
}
export function cloudConfigurationDigest(value: unknown): string {
return `sha256:${createHash("sha256").update(JSON.stringify(canonical(value))).digest("hex")}`;
}
export const externalRegistrationEvidenceSchema = z.object({
sourceLocatorDigest: z.string(),
sourceDigest: z.string(),
state: z.enum(["missing", "parsed", "invalid", "unreadable"]),
adapterTypes: z.array(z.string()),
blocker: z.enum(["unsupported_external_adapters", "invalid_external_registration_metadata", "unreadable_external_registration_metadata"]).nullable(),
}).strict();
export type ExternalRegistrationEvidence = z.infer<typeof externalRegistrationEvidenceSchema>;
const legacyRegistrationSchema = z.array(z.object({
type: z.string().regex(/^[a-zA-Z0-9_.-]{1,128}$/),
packageName: z.string().min(1).max(1024),
version: z.string().max(128).optional(),
localPath: z.string().max(4096).optional(),
disabled: z.boolean().optional(),
}).passthrough()).max(100);
export async function inspectLegacyExternalRegistrations(filePath: string): Promise<ExternalRegistrationEvidence> {
const sourceLocatorDigest = cloudConfigurationDigest(filePath);
try {
const file = await open(filePath, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK);
let raw: unknown;
try {
const stat = await file.stat();
if (!stat.isFile() || stat.size > 65_536) throw new Error("invalid legacy registration file");
const buffer = Buffer.alloc(65_537);
let total = 0;
while (total < buffer.length) {
const { bytesRead } = await file.read(buffer, total, buffer.length - total, total);
if (!bytesRead) break;
total += bytesRead;
}
if (total > 65_536) throw new Error("oversized legacy registration file");
raw = JSON.parse(buffer.subarray(0, total).toString("utf8"));
} finally {
await file.close();
}
const records = legacyRegistrationSchema.safeParse(raw);
if (!records.success) {
return { sourceLocatorDigest, sourceDigest: cloudConfigurationDigest({ state: "invalid" }), state: "invalid", adapterTypes: [], blocker: "invalid_external_registration_metadata" };
}
const metadata = records.data.map((record) => ({ type: record.type, packageName: record.packageName, version: record.version, disabled: record.disabled, hasLocalPath: record.localPath !== undefined }));
return {
sourceLocatorDigest, sourceDigest: cloudConfigurationDigest(metadata), state: "parsed",
adapterTypes: [...new Set(metadata.map((record) => record.type))].sort(),
blocker: metadata.length ? "unsupported_external_adapters" : null,
};
} catch (error) {
const absent = (error as NodeJS.ErrnoException).code === "ENOENT";
const invalid = error instanceof SyntaxError || (error instanceof Error && /legacy registration file/.test(error.message));
const state = absent ? "missing" : invalid ? "invalid" : "unreadable";
return {
sourceLocatorDigest, sourceDigest: cloudConfigurationDigest({ state }), state, adapterTypes: [],
blocker: absent ? null : invalid ? "invalid_external_registration_metadata" : "unreadable_external_registration_metadata",
};
}
}
@@ -0,0 +1,193 @@
import { instanceSettings, plugins, type Db } from "@paperclipai/db";
import { eq, inArray, ne } from "drizzle-orm";
import { z } from "zod";
import { adapterSettingsService, readLegacyAdapterSettings } from "./adapter-settings.js";
import { getAdapterPluginStorePath } from "./adapter-plugin-store.js";
import { cloudRuntimeConfigService, readLegacyCloudRuntimeConfig } from "./cloud-runtime-config.js";
import { isCloudManagedInstance, type CloudInstanceEnv } from "./cloud-instance.js";
import { cloudConfigurationDigest, externalRegistrationEvidenceSchema, inspectLegacyExternalRegistrations } from "./cloud-configuration-evidence.js";
import { cloudPackageEvidenceSchema, inspectCloudPackages, qualificationBlockerSchema } from "./cloud-package-qualification.js";
import type { ResolvedBundledPlugin } from "./bundled-plugins.js";
import type { PaperclipPluginManifestV1 } from "@paperclipai/shared";
const RECEIPT_KEY = "cloud-configuration-migration:v1";
const CONFIGURATION_KEYS = ["adapter-settings:v1", "cloud-runtime-config:v1"];
type Transaction = Parameters<Parameters<Db["transaction"]>[0]>[0];
export type CloudConfigurationDb = Pick<Db | Transaction, "select" | "insert" | "update" | "transaction">;
const sourceSchema = z.object({
authority: z.enum(["database", "legacy_file", "default"]), sourceDigest: z.string(),
importedRevision: z.number().int().nonnegative(),
}).strict();
const qualificationSchema = z.object({
scope: z.literal("configuration_and_packages"), checkedAt: z.string(),
adapterRevision: z.number().int().nonnegative(), configRevision: z.number().int().nonnegative(),
configurationDigest: z.string(), packageEvidenceDigest: z.string(),
packageInspectionDurationMs: z.number().nonnegative(),
expectedPackageEvidenceDigest: z.string().nullable(),
configurationPackagesEligible: z.boolean(), samePackageOverlapEligible: z.boolean(),
packages: z.array(cloudPackageEvidenceSchema), blockers: z.array(qualificationBlockerSchema),
overlapBlockers: z.array(qualificationBlockerSchema),
}).strict();
const receiptSchema = z.object({
version: z.literal(1), revision: z.number().int().nonnegative(),
migration: z.object({ completedAt: z.string(), adapter: sourceSchema, config: sourceSchema, externalRegistrations: externalRegistrationEvidenceSchema }).strict(),
qualification: qualificationSchema.nullable(), lastPackageMismatch: qualificationSchema.nullable(),
}).strict();
export type CloudConfigurationReceipt = z.infer<typeof receiptSchema>;
function parseReceipt(value: unknown): CloudConfigurationReceipt {
const result = receiptSchema.safeParse(value);
if (!result.success) throw new Error("Persisted Cloud configuration migration receipt is invalid");
return result.data;
}
export function cloudConfigurationMigrationService(db: CloudConfigurationDb, options: { runtimeEnv?: CloudInstanceEnv; now?: () => Date } = {}) {
const env = options.runtimeEnv ?? process.env;
const now = () => (options.now?.() ?? new Date()).toISOString();
const adapters = adapterSettingsService(db, { runtimeEnv: env });
const config = cloudRuntimeConfigService(db);
function assertManaged() {
if (!isCloudManagedInstance(env)) throw new Error("Cloud configuration qualification requires a managed instance");
}
async function readRow(handle: CloudConfigurationDb, lock = false) {
const query = handle.select().from(instanceSettings).where(eq(instanceSettings.singletonKey, RECEIPT_KEY));
const [row] = lock ? await query.for("update") : await query;
return row ?? null;
}
async function currentConfiguration(handle: CloudConfigurationDb) {
await handle.select({ id: instanceSettings.id }).from(instanceSettings).where(inArray(instanceSettings.singletonKey, CONFIGURATION_KEYS))
.orderBy(instanceSettings.singletonKey).for("update");
const adapter = await adapters.inspectStored(handle);
const runtime = await config.inspectStored(handle);
if (!adapter || !runtime) throw new Error("Cloud configuration authority is incomplete");
return { adapter, runtime };
}
async function registrationSnapshot(handle: CloudConfigurationDb, lock = false) {
const query = handle.select({
pluginKey: plugins.pluginKey, packageName: plugins.packageName, version: plugins.version,
manifestJson: plugins.manifestJson, status: plugins.status,
}).from(plugins).where(ne(plugins.status, "uninstalled")).orderBy(plugins.pluginKey);
return lock ? query.for("share") : query;
}
async function save(handle: CloudConfigurationDb, id: string, receipt: CloudConfigurationReceipt) {
const validated = parseReceipt(receipt);
const updated = await handle.update(instanceSettings).set({ general: validated, updatedAt: new Date() }).where(eq(instanceSettings.id, id)).returning({ id: instanceSettings.id });
if (updated.length !== 1) throw new Error("Cloud configuration migration receipt was not persisted");
return validated;
}
return {
async inspectStored(): Promise<CloudConfigurationReceipt | null> {
const row = await readRow(db);
return row ? parseReceipt(row.general) : null;
},
async migrate(): Promise<CloudConfigurationReceipt> {
assertManaged();
return db.transaction(async (tx) => {
const inserted = await tx.insert(instanceSettings).values({ singletonKey: RECEIPT_KEY, general: { version: 1 } })
.onConflictDoNothing({ target: instanceSettings.singletonKey }).returning({ id: instanceSettings.id });
const row = await readRow(tx, true);
if (!row) throw new Error("Cloud configuration migration receipt is unavailable");
if (!inserted.length) {
const receipt = parseReceipt(row.general);
await currentConfiguration(tx);
return receipt;
}
await tx.select({ id: instanceSettings.id }).from(instanceSettings).where(inArray(instanceSettings.singletonKey, CONFIGURATION_KEYS))
.orderBy(instanceSettings.singletonKey).for("update");
const storedAdapter = await adapters.inspectStored(tx);
const storedConfig = await config.inspectStored(tx);
const adapterCandidate = storedAdapter ? null : await readLegacyAdapterSettings(env);
const configCandidate = storedConfig ? null : await readLegacyCloudRuntimeConfig(env);
const adapterSeed = storedAdapter ? null : await adapters.seedIfAbsent({ disabledTypes: adapterCandidate?.disabledTypes ?? [] }, tx);
const configSeed = storedConfig ? null : await config.seedIfAbsent(configCandidate, tx);
const adapter = storedAdapter ?? adapterSeed!.current;
const runtime = storedConfig ?? configSeed!.current;
const adapterFromDatabase = Boolean(storedAdapter || !adapterSeed?.seeded);
const configFromDatabase = Boolean(storedConfig || !configSeed?.seeded);
const receipt: CloudConfigurationReceipt = {
version: 1, revision: 0,
migration: {
completedAt: now(),
adapter: { authority: adapterFromDatabase ? "database" : adapterCandidate ? "legacy_file" : "default", sourceDigest: cloudConfigurationDigest(adapterFromDatabase ? { disabledTypes: adapter.disabledTypes } : adapterCandidate), importedRevision: adapter.revision },
config: { authority: configFromDatabase ? "database" : configCandidate ? "legacy_file" : "default", sourceDigest: cloudConfigurationDigest(configFromDatabase ? runtime.config : configCandidate), importedRevision: runtime.revision },
externalRegistrations: await inspectLegacyExternalRegistrations(getAdapterPluginStorePath(env.PAPERCLIP_HOME)),
},
qualification: null, lastPackageMismatch: null,
};
return save(tx, row.id, receipt);
});
},
async recheckLegacyExternalRegistrations(expectedRevision: number): Promise<CloudConfigurationReceipt> {
assertManaged();
return db.transaction(async (tx) => {
const row = await readRow(tx, true);
if (!row) throw new Error("Cloud configuration migration has not completed");
const receipt = parseReceipt(row.general);
if (receipt.revision !== expectedRevision) throw new Error("Cloud configuration migration receipt revision conflict");
const filePath = getAdapterPluginStorePath(env.PAPERCLIP_HOME);
if (cloudConfigurationDigest(filePath) !== receipt.migration.externalRegistrations.sourceLocatorDigest) {
throw new Error("External registration recheck requires the original legacy source location");
}
const evidence = await inspectLegacyExternalRegistrations(filePath);
if (evidence.state !== "parsed") throw new Error("External registration recheck requires a present, valid registration file");
receipt.migration.externalRegistrations = evidence;
receipt.revision += 1;
receipt.qualification = null;
return save(tx, row.id, receipt);
});
},
async qualify(input: {
installs: readonly ResolvedBundledPlugin[];
catalogRoot?: string;
loadManifest: (packageRoot: string) => Promise<PaperclipPluginManifestV1 | null>;
expectedPackageEvidenceDigest?: string;
imageCatalogUnavailable?: boolean;
}) {
assertManaged();
const adapter = await adapters.inspectStored();
const runtime = await config.inspectStored();
if (!adapter || !runtime) throw new Error("Cloud configuration authority is incomplete");
const configurationBefore = cloudConfigurationDigest({ adapter, runtime });
const registrationsBefore = await registrationSnapshot(db);
const evidence = await inspectCloudPackages({ ...input, registrations: registrationsBefore });
return db.transaction(async (tx) => {
const row = await readRow(tx, true);
if (!row) throw new Error("Cloud configuration migration has not completed");
const receipt = parseReceipt(row.general);
const current = await currentConfiguration(tx);
const registrations = await registrationSnapshot(tx, true);
const blockers = [...evidence.blockers];
if (configurationBefore !== cloudConfigurationDigest(current) || cloudConfigurationDigest(registrationsBefore) !== cloudConfigurationDigest(registrations)) {
blockers.push({ code: "configuration_changed_during_inspection", reason: "Configuration or plugin registration changed during image inspection; run qualification again" });
}
if (input.imageCatalogUnavailable) blockers.push({ code: "current_image_catalog_unavailable", reason: "The current image package catalog could not be verified" });
const external = receipt.migration.externalRegistrations;
if (external.blocker) blockers.push({ code: external.blocker, reason: "Legacy external adapter registration metadata requires an explicit operator resolution" });
const overlapBlockers = input.expectedPackageEvidenceDigest && input.expectedPackageEvidenceDigest !== evidence.packageEvidenceDigest
? [{ code: "package_evidence_changed", reason: "Current packages do not match the expected package evidence digest" }] : [];
const qualification: z.infer<typeof qualificationSchema> = {
scope: "configuration_and_packages", checkedAt: now(),
adapterRevision: current.adapter.revision, configRevision: current.runtime.revision,
configurationDigest: cloudConfigurationDigest(current), packageEvidenceDigest: evidence.packageEvidenceDigest,
packageInspectionDurationMs: evidence.packageInspectionDurationMs,
expectedPackageEvidenceDigest: input.expectedPackageEvidenceDigest ?? null,
configurationPackagesEligible: blockers.length === 0,
samePackageOverlapEligible: Boolean(input.expectedPackageEvidenceDigest && blockers.length === 0 && overlapBlockers.length === 0),
packages: evidence.packages, blockers, overlapBlockers,
};
receipt.revision += 1;
receipt.qualification = qualification;
if (evidence.blockers.length || input.imageCatalogUnavailable || overlapBlockers.length) receipt.lastPackageMismatch = qualification;
await save(tx, row.id, receipt);
return { receiptRevision: receipt.revision, ...qualification };
});
},
};
}
export async function migrateCloudConfiguration(db: CloudConfigurationDb, options: { runtimeEnv?: CloudInstanceEnv } = {}) {
if (!isCloudManagedInstance(options.runtimeEnv ?? process.env)) return null;
return cloudConfigurationMigrationService(db, options).migrate();
}
@@ -0,0 +1,33 @@
import type { Db } from "@paperclipai/db";
import { BUNDLED_PLUGIN_CATALOG, resolveBundledCatalogRoot, resolveBundledPluginInstalls, resolveManagedBundledPluginKeys, type ResolvedBundledPlugin } from "./bundled-plugins.js";
import { readDistributionPluginCatalog } from "./distribution-plugin-catalog.js";
import { createBundledPluginPackageResolver } from "./bundled-plugin-package.js";
import { cloudConfigurationMigrationService } from "./cloud-configuration-migration.js";
import { isCloudManagedInstance, type CloudInstanceEnv } from "./cloud-instance.js";
import { getManagedInstanceConfig } from "./managed-config.js";
import { pluginLoader } from "./plugin-loader.js";
export async function qualifyCloudConfigurationForCurrentImage(db: Db, options: {
runtimeEnv?: CloudInstanceEnv;
expectedPackageEvidenceDigest?: string;
} = {}) {
const env = options.runtimeEnv ?? process.env;
if (!isCloudManagedInstance(env)) return null;
const service = cloudConfigurationMigrationService(db, { runtimeEnv: env });
const catalogRoot = resolveBundledCatalogRoot(env);
let installs: ResolvedBundledPlugin[];
try {
const distribution = readDistributionPluginCatalog(catalogRoot, BUNDLED_PLUGIN_CATALOG);
installs = resolveBundledPluginInstalls(resolveManagedBundledPluginKeys(getManagedInstanceConfig(env)?.plugins.autoInstall, env) ?? [], {
catalogRoot, env, enforceCatalogRoot: true, distributionPlugins: distribution,
});
} catch {
return service.qualify({ installs: [], loadManifest: async () => null, imageCatalogUnavailable: true, expectedPackageEvidenceDigest: options.expectedPackageEvidenceDigest });
}
const resolver = createBundledPluginPackageResolver(installs);
const loader = pluginLoader(db, {
prebuiltOnly: true, enableLocalFilesystem: false, enableNpmDiscovery: false,
assertPackageActivation: resolver.assertActivation, resolvePackageRoot: resolver.resolve,
});
return service.qualify({ installs, catalogRoot, loadManifest: (root) => loader.loadManifest(root), expectedPackageEvidenceDigest: options.expectedPackageEvidenceDigest });
}
@@ -0,0 +1,72 @@
import type { PaperclipPluginManifestV1 } from "@paperclipai/shared";
import { z } from "zod";
import { createBundledPluginPackageResolver, type BundledPluginRegistration } from "./bundled-plugin-package.js";
import type { ResolvedBundledPlugin } from "./bundled-plugins.js";
import { distributionBundleDigest } from "./distribution-plugin-catalog.js";
import { cloudConfigurationDigest } from "./cloud-configuration-evidence.js";
import { cloudBundledPackageDigest } from "./cloud-bundled-package-digest.js";
export const qualificationBlockerSchema = z.object({
code: z.string(), pluginKey: z.string().optional(), reason: z.string(),
registeredVersion: z.string().optional(), imageVersion: z.string().optional(),
}).strict();
export type QualificationBlocker = z.infer<typeof qualificationBlockerSchema>;
export const cloudPackageEvidenceSchema = z.object({
catalogKey: z.string(), pluginKey: z.string(), packageDigest: z.string().nullable(),
imageManifestDigest: z.string().nullable(), approvedManifestDigest: z.string().nullable(),
}).strict();
export type CloudPackageEvidence = z.infer<typeof cloudPackageEvidenceSchema>;
export async function inspectCloudPackages(input: {
installs: readonly ResolvedBundledPlugin[];
registrations: readonly BundledPluginRegistration[];
catalogRoot?: string;
loadManifest: (packageRoot: string) => Promise<PaperclipPluginManifestV1 | null>;
}) {
const started = performance.now();
const resolver = createBundledPluginPackageResolver(input.installs);
const registrations = new Map(input.registrations.map((record) => [record.pluginKey, record]));
const selected = new Set(input.installs.map((install) => install.pluginKey));
const blockers: QualificationBlocker[] = input.registrations.filter((record) => !selected.has(record.pluginKey)).map((record) => ({
code: "unsupported_plugin_registration", pluginKey: record.pluginKey, reason: "An installed plugin is not selected from the current image", registeredVersion: record.version,
}));
const packages: CloudPackageEvidence[] = [];
for (const install of [...input.installs].sort((left, right) => left.pluginKey < right.pluginKey ? -1 : left.pluginKey > right.pluginKey ? 1 : 0)) {
const registration = registrations.get(install.pluginKey);
const evidence: CloudPackageEvidence = {
catalogKey: install.key, pluginKey: install.pluginKey, packageDigest: null,
imageManifestDigest: null, approvedManifestDigest: registration ? cloudConfigurationDigest(registration.manifestJson) : null,
};
packages.push(evidence);
let stage = "missing_bundled_package";
try {
resolver.assertActivation({ packageRoot: install.localPath });
stage = "unverified_package_integrity";
evidence.packageDigest = install.distribution ? distributionBundleDigest(install.localPath) : cloudBundledPackageDigest(install.localPath, { catalogRoot: input.catalogRoot });
if (install.distribution && evidence.packageDigest !== install.distribution.digest) {
blockers.push({ code: "package_integrity_mismatch", pluginKey: install.pluginKey, reason: "Current package bytes do not match the verified distribution digest" });
continue;
}
stage = "missing_manifest_evidence";
const manifest = await input.loadManifest(install.localPath);
if (!manifest) {
blockers.push({ code: stage, pluginKey: install.pluginKey, reason: "The current image has no verified plugin manifest" });
continue;
}
evidence.imageManifestDigest = cloudConfigurationDigest(manifest);
if (!registration) {
blockers.push({ code: "missing_plugin_registration", pluginKey: install.pluginKey, reason: "The current image package is not registered in the database", imageVersion: manifest.version });
continue;
}
const inspected = resolver.inspect(registration, manifest);
if (!inspected.compatible) blockers.push({
code: inspected.code, pluginKey: install.pluginKey, reason: inspected.reason,
registeredVersion: registration.version, imageVersion: manifest.version,
});
} catch {
blockers.push({ code: stage, pluginKey: install.pluginKey, reason: "The current image package could not be verified" });
}
}
const packageEvidenceDigest = cloudConfigurationDigest(packages.map(({ approvedManifestDigest: _approved, ...evidence }) => evidence));
return { packages, blockers, packageEvidenceDigest, packageInspectionDurationMs: Math.round(performance.now() - started) };
}
@@ -0,0 +1,129 @@
import {
authConfigSchema,
databaseBackupConfigSchema,
llmConfigSchema,
loggingConfigSchema,
paperclipConfigSchema,
secretsConfigSchema,
secretsLocalEncryptedConfigSchema,
serverConfigSchema,
storageConfigSchema,
storageLocalDiskConfigSchema,
storageS3ConfigSchema,
type PaperclipConfig,
} from "@paperclipai/shared";
import { z } from "zod";
const publicUrl = z.string().url().refine((value) => {
const url = new URL(value);
return !url.username && !url.password && !url.search && !url.hash;
});
export const cloudRuntimeConfigSchema = z.object({
server: serverConfigSchema.strict(),
auth: authConfigSchema.extend({ publicBaseUrl: publicUrl.optional() }).strict(),
backup: databaseBackupConfigSchema.strict(),
storage: storageConfigSchema.extend({
localDisk: storageLocalDiskConfigSchema.strict(),
s3: storageS3ConfigSchema.extend({ endpoint: publicUrl.optional() }).strict(),
}).strict(),
secrets: secretsConfigSchema.extend({
localEncrypted: secretsLocalEncryptedConfigSchema.strict(),
}).strict(),
telemetry: z.object({ enabled: z.boolean() }).strict(),
logging: loggingConfigSchema.strict(),
updates: z.object({ checkEnabled: z.boolean() }).strict().optional(),
llm: llmConfigSchema.omit({ apiKey: true }).strict().optional(),
}).strict();
export type CloudRuntimeConfig = z.infer<typeof cloudRuntimeConfigSchema>;
export type StoredCloudRuntimeConfig = { version: 1; revision: number; config: CloudRuntimeConfig | null };
const storedSchema = z.object({
version: z.literal(1),
revision: z.number().int().nonnegative(),
config: cloudRuntimeConfigSchema.nullable(),
}).strict();
export function parseStoredCloudRuntimeConfig(value: unknown): StoredCloudRuntimeConfig {
const parsed = storedSchema.safeParse(value);
if (!parsed.success || Buffer.byteLength(JSON.stringify(parsed.data), "utf8") > 65_536) {
throw new Error("Persisted Cloud runtime configuration is invalid");
}
const full = parsed.data.config === null ? { success: true } : paperclipConfigSchema.safeParse(toPaperclipConfig(parsed.data.config));
if (!full.success) throw new Error("Persisted Cloud runtime configuration is invalid");
return parsed.data;
}
export function toPaperclipConfig(config: CloudRuntimeConfig | null): PaperclipConfig | null {
if (config === null) return null;
const { backup, ...settings } = structuredClone(config);
return {
$meta: { version: 1, updatedAt: "1970-01-01T00:00:00.000Z", source: "configure" },
database: {
mode: "postgres",
embeddedPostgresDataDir: "",
embeddedPostgresPort: 54329,
backup,
},
...settings,
};
}
export function projectCloudRuntimeConfig(source: PaperclipConfig): CloudRuntimeConfig {
const raw = {
server: {
deploymentMode: source.server.deploymentMode,
exposure: source.server.exposure,
bind: source.server.bind,
customBindHost: source.server.customBindHost,
host: source.server.host,
port: source.server.port,
allowedHostnames: source.server.allowedHostnames,
serveUi: source.server.serveUi,
},
auth: {
baseUrlMode: source.auth.baseUrlMode,
publicBaseUrl: source.auth.publicBaseUrl,
disableSignUp: source.auth.disableSignUp,
},
backup: {
enabled: source.database.backup.enabled,
intervalMinutes: source.database.backup.intervalMinutes,
retentionDays: source.database.backup.retentionDays,
dir: source.database.backup.dir,
},
storage: {
provider: source.storage.provider,
localDisk: { baseDir: source.storage.localDisk.baseDir },
s3: {
bucket: source.storage.s3.bucket,
region: source.storage.s3.region,
endpoint: source.storage.s3.endpoint,
prefix: source.storage.s3.prefix,
forcePathStyle: source.storage.s3.forcePathStyle,
},
},
secrets: {
provider: source.secrets.provider,
strictMode: source.secrets.strictMode,
localEncrypted: { keyFilePath: source.secrets.localEncrypted.keyFilePath },
},
telemetry: { enabled: source.telemetry.enabled },
logging: { mode: source.logging.mode, logDir: source.logging.logDir },
updates: source.updates ? { checkEnabled: source.updates.checkEnabled } : undefined,
llm: source.llm ? { provider: source.llm.provider } : undefined,
};
return parseStoredCloudRuntimeConfig({ version: 1, revision: 0, config: raw }).config!;
}
let snapshot: StoredCloudRuntimeConfig | null = null;
export function setCloudRuntimeConfigSnapshot(value: StoredCloudRuntimeConfig | null): void {
snapshot = value ? parseStoredCloudRuntimeConfig(value) : null;
}
export function getCloudRuntimeConfigSnapshot(): StoredCloudRuntimeConfig {
if (!snapshot) throw new Error("Cloud runtime configuration has not been initialized");
return structuredClone(snapshot);
}
+105
View File
@@ -0,0 +1,105 @@
import { constants } from "node:fs";
import { open } from "node:fs/promises";
import path from "node:path";
import { eq } from "drizzle-orm";
import { instanceSettings, type Db } from "@paperclipai/db";
import { paperclipConfigSchema } from "@paperclipai/shared";
import { resolvePaperclipConfigPathForInstance } from "@paperclipai/shared/home-paths";
import { isCloudManagedInstance, type CloudInstanceEnv } from "./cloud-instance.js";
import {
parseStoredCloudRuntimeConfig,
projectCloudRuntimeConfig,
setCloudRuntimeConfigSnapshot,
type CloudRuntimeConfig,
type StoredCloudRuntimeConfig,
} from "./cloud-runtime-config-snapshot.js";
export const CLOUD_RUNTIME_CONFIG_KEY = "cloud-runtime-config:v1";
type Transaction = Parameters<Parameters<Db["transaction"]>[0]>[0];
export type CloudRuntimeConfigDb = Pick<Db | Transaction, "select" | "insert" | "update" | "transaction">;
export function cloudRuntimeConfigService(db: CloudRuntimeConfigDb) {
async function inspectStored(handle: CloudRuntimeConfigDb = db): Promise<StoredCloudRuntimeConfig | null> {
const [row] = await handle.select({ general: instanceSettings.general }).from(instanceSettings)
.where(eq(instanceSettings.singletonKey, CLOUD_RUNTIME_CONFIG_KEY)).limit(1);
return row ? parseStoredCloudRuntimeConfig(row.general) : null;
}
async function seedIfAbsent(config: CloudRuntimeConfig | null, handle: CloudRuntimeConfigDb = db) {
const next = parseStoredCloudRuntimeConfig({ version: 1, revision: 0, config });
const inserted = await handle.insert(instanceSettings)
.values({ singletonKey: CLOUD_RUNTIME_CONFIG_KEY, general: next })
.onConflictDoNothing({ target: instanceSettings.singletonKey }).returning({ id: instanceSettings.id });
const current = await inspectStored(handle);
if (!current) throw new Error("Cloud runtime configuration seed was not persisted");
return { seeded: inserted.length > 0, current };
}
return {
inspectStored,
seedIfAbsent,
async replace(config: CloudRuntimeConfig | null, expectedRevision: number, handle: CloudRuntimeConfigDb = db) {
const validated = parseStoredCloudRuntimeConfig({ version: 1, revision: expectedRevision, config });
return handle.transaction(async (tx) => {
const [row] = await tx.select().from(instanceSettings)
.where(eq(instanceSettings.singletonKey, CLOUD_RUNTIME_CONFIG_KEY)).for("update");
if (!row) throw new Error("Cloud runtime configuration has not been initialized");
const current = parseStoredCloudRuntimeConfig(row.general);
if (current.revision !== expectedRevision) throw new Error("Cloud runtime configuration revision conflict");
const next = { ...validated, revision: current.revision + 1 };
await tx.update(instanceSettings).set({ general: next, updatedAt: new Date() }).where(eq(instanceSettings.id, row.id));
return next;
});
},
};
}
export function legacyCloudRuntimeConfigPath(env: CloudInstanceEnv = process.env): string {
return env.PAPERCLIP_CONFIG
? path.resolve(env.PAPERCLIP_CONFIG)
: resolvePaperclipConfigPathForInstance({ homeDir: env.PAPERCLIP_HOME, instanceId: env.PAPERCLIP_INSTANCE_ID });
}
export async function readLegacyCloudRuntimeConfig(env: CloudInstanceEnv = process.env): Promise<CloudRuntimeConfig | null> {
let raw: unknown;
try {
const file = await open(legacyCloudRuntimeConfigPath(env), constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK);
try {
const stat = await file.stat();
if (!stat.isFile() || stat.size > 65_536) throw new Error("invalid file");
const buffer = Buffer.alloc(65_537);
let total = 0;
while (total < buffer.length) {
const { bytesRead } = await file.read(buffer, total, buffer.length - total, total);
if (bytesRead === 0) break;
total += bytesRead;
}
if (total > 65_536) throw new Error("oversized file");
raw = JSON.parse(buffer.subarray(0, total).toString("utf8"));
} finally {
await file.close();
}
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw new Error("Legacy Cloud runtime configuration could not be read safely");
return null;
}
const parsed = paperclipConfigSchema.safeParse(raw);
if (!parsed.success) throw new Error("Legacy Cloud runtime configuration is invalid");
if (parsed.data.llm?.provider === "openai" && parsed.data.llm.apiKey?.trim() && !env.OPENAI_API_KEY?.trim()) {
throw new Error("Move the legacy OpenAI API key to OPENAI_API_KEY before initializing Cloud runtime configuration");
}
return projectCloudRuntimeConfig(parsed.data);
}
export async function initializeCloudRuntimeConfig(db: CloudRuntimeConfigDb, options: { runtimeEnv?: CloudInstanceEnv } = {}) {
const env = options.runtimeEnv ?? process.env;
if (!isCloudManagedInstance(env)) return null;
setCloudRuntimeConfigSnapshot(null);
const service = cloudRuntimeConfigService(db);
const current = await service.inspectStored();
const result = current
? { seeded: false, current }
: await service.seedIfAbsent(await readLegacyCloudRuntimeConfig(env));
setCloudRuntimeConfigSnapshot(result.current);
return result;
}
+21 -17
View File
@@ -1,3 +1,4 @@
import { resolveBundledPluginRuntimeLoader } from "./bundled-plugin-runtime.js";
import { finishServiceShutdown } from "../shutdown.js";
/**
* PluginLoader — discovery, installation, and runtime activation of plugins.
@@ -31,7 +32,7 @@ import { execFile } from "node:child_process";
import os from "node:os";
import path from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
import { promisify } from "node:util";
import { isDeepStrictEqual, promisify } from "node:util";
import type { Db } from "@paperclipai/db";
import { PLUGIN_RPC_ERROR_CODES } from "@paperclipai/plugin-sdk";
import type {
@@ -271,6 +272,8 @@ function getDeclaredPageRoutePaths(manifest: PaperclipPluginManifestV1): string[
* Options for the plugin loader service.
*/
export interface PluginLoaderOptions {
resolvePackageRoot?: (plugin: PluginRecord) => string;
prebuiltOnly?: boolean;
/** Image-owned deployment policy, checked before importing code and starting workers. */
assertPackageActivation?: (input: {
pluginKey?: string;
@@ -1157,6 +1160,8 @@ export function pluginLoader(
enableLocalFilesystem = true,
enableNpmDiscovery = true,
assertPackageActivation,
resolvePackageRoot,
prebuiltOnly = false,
} = options;
const registry = pluginRegistryService(db);
@@ -1211,6 +1216,7 @@ export function pluginLoader(
installOptions: PluginInstallOptions,
): Promise<DiscoveredPlugin> {
const { packageName, localPath, version, installDir } = installOptions;
if (prebuiltOnly && !localPath) throw new Error("Managed plugins must come from the current image catalog");
if (!packageName && !localPath) {
throw new Error("Either packageName or localPath must be provided");
@@ -1286,7 +1292,7 @@ export function pluginLoader(
const pkgJson = await readPackageJson(resolvedPackagePath);
if (!pkgJson) throw new Error(`Missing package.json at ${resolvedPackagePath}`);
if (localPath) {
if (localPath && !prebuiltOnly) {
await ensureLocalPluginBuilt(resolvedPackagePath, pkgJson);
}
@@ -1398,7 +1404,7 @@ export function pluginLoader(
}
assertPackageActivation?.({ packageRoot, installedPackagePath: plugin.packagePath, pluginKey: plugin.pluginKey, manifest, previousManifest: plugin.manifestJson });
if (JSON.stringify(manifest) === JSON.stringify(plugin.manifestJson)) {
if (isDeepStrictEqual(manifest, plugin.manifestJson)) {
return plugin;
}
@@ -1749,6 +1755,7 @@ export function pluginLoader(
packagePath: discovered.source === "local-filesystem" ? discovered.packagePath : undefined,
},
manifest,
{ idempotent: prebuiltOnly },
);
if (!installed) {
@@ -1803,12 +1810,7 @@ export function pluginLoader(
newManifest: PaperclipPluginManifestV1;
discovered: DiscoveredPlugin;
}> {
const plugin = (await registry.getById(pluginId)) as {
id: string;
packageName: string;
packagePath: string | null;
manifestJson: PaperclipPluginManifestV1;
} | null;
const plugin = await registry.getById(pluginId);
if (!plugin) throw new Error(`Plugin not found: ${pluginId}`);
const oldManifest = plugin.manifestJson;
@@ -1817,7 +1819,7 @@ export function pluginLoader(
// For local-path installs, fall back to the stored packagePath so
// `upgradePlugin` can re-read the manifest from disk without needing
// the caller to re-supply the path every time.
localPath = plugin.packagePath ?? undefined,
localPath = resolvePackageRoot ? resolvePackageRoot(plugin) : plugin.packagePath ?? undefined,
version,
} = upgradeOptions;
@@ -1887,6 +1889,7 @@ export function pluginLoader(
// -----------------------------------------------------------------------
async cleanupInstallArtifacts(plugin: PluginRecord): Promise<void> {
if (prebuiltOnly) return;
const managedTargets = new Set<string>();
const managedNodeModulesDir = resolveManagedInstallPackageDir(localPluginDir, plugin.packageName);
const directManagedDir = path.join(localPluginDir, plugin.packageName);
@@ -2262,12 +2265,14 @@ export function pluginLoader(
// ------------------------------------------------------------------
// 1. Resolve worker entrypoint
// ------------------------------------------------------------------
const packageRoot = resolvePluginPackageRoot(activePlugin, localPluginDir);
const packageRoot = resolvePackageRoot ? resolvePackageRoot(activePlugin) : resolvePluginPackageRoot(activePlugin, localPluginDir);
assertPackageActivation?.({ pluginKey, packageRoot, installedPackagePath: activePlugin.packagePath });
activePlugin = await refreshPluginManifestFromPackage(activePlugin, packageRoot);
manifest = activePlugin.manifestJson;
assertPackageActivation?.({ pluginKey, packageRoot, installedPackagePath: activePlugin.packagePath, manifest });
const workerEntrypoint = resolveWorkerEntrypoint(activePlugin, localPluginDir);
const workerEntrypoint = resolvePackageRoot
? path.resolve(packageRoot, manifest.entrypoints.worker)
: resolveWorkerEntrypoint(activePlugin, localPluginDir);
// ------------------------------------------------------------------
// 2. Apply restricted database migrations before worker startup
@@ -2334,7 +2339,7 @@ export function pluginLoader(
env: buildPluginWorkerEnv({
manifest,
packageName: activePlugin.packageName,
packagePath: activePlugin.packagePath,
packagePath: resolvePackageRoot ? packageRoot : activePlugin.packagePath,
instanceInfo,
}),
// Authorize the worker to act on each configured company from its
@@ -2345,10 +2350,9 @@ export function pluginLoader(
proactiveCompanyScopes: configRows.map((row) => row.companyId),
};
// Repo-local plugin installs can resolve workspace TS sources at runtime
// (for example @paperclipai/shared exports). Run those workers through
// the tsx loader so first-party example plugins work in development.
if (activePlugin.packagePath && existsSync(DEV_TSX_LOADER_PATH)) {
if (prebuiltOnly) {
workerOptions.execArgv = ["--import", resolveBundledPluginRuntimeLoader()];
} else if (activePlugin.packagePath && existsSync(DEV_TSX_LOADER_PATH)) {
workerOptions.execArgv = ["--import", DEV_TSX_LOADER_PATH];
}
+17 -6
View File
@@ -1,3 +1,4 @@
import { isDeepStrictEqual } from "node:util";
import { writePluginJobChange } from "./plugin-job-work-notifications.js";
import { asc, eq, isNull, ne, sql, and } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
@@ -31,6 +32,10 @@ import type {
import { conflict, notFound } from "../errors.js";
import { DELIVERY_QUEUES, notifyDeliveryWork } from "./delivery-work-notifications.js";
function sameBundledPluginRegistration(existing: typeof plugins.$inferSelect, packageName: string, manifest: PaperclipPluginManifestV1): boolean {
return existing.pluginKey === manifest.id && existing.packageName === packageName && existing.version === manifest.version && isDeepStrictEqual(existing.manifestJson, manifest);
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
@@ -140,10 +145,11 @@ export function pluginRegistryService(db: Db) {
* manifest from the package. This method persists the plugin row and
* assigns the next install order.
*/
install: async (input: InstallPlugin, manifest: PaperclipPluginManifestV1) => {
install: async (input: InstallPlugin, manifest: PaperclipPluginManifestV1, options: { idempotent?: boolean } = {}) => {
const existing = await getByKey(manifest.id);
if (existing) {
if (existing.status !== "uninstalled") {
if (options.idempotent && sameBundledPluginRegistration(existing, input.packageName, manifest)) return existing;
throw conflict(`Plugin already installed: ${manifest.id}`);
}
@@ -153,7 +159,7 @@ export function pluginRegistryService(db: Db) {
.update(plugins)
.set({
packageName: input.packageName,
packagePath: input.packagePath ?? null,
packagePath: options.idempotent ? existing.packagePath : input.packagePath ?? null,
version: manifest.version,
apiVersion: manifest.apiVersion,
categories: manifest.categories,
@@ -170,7 +176,7 @@ export function pluginRegistryService(db: Db) {
const installOrder = await nextInstallOrder();
try {
const rows = await db
const insertion = db
.insert(plugins)
.values({
pluginKey: manifest.id,
@@ -182,9 +188,14 @@ export function pluginRegistryService(db: Db) {
status: "installed" as PluginStatus,
installOrder,
packagePath: input.packagePath ?? null,
})
.returning();
return rows[0];
});
const rows = await (options.idempotent
? insertion.onConflictDoNothing({ target: plugins.pluginKey })
: insertion).returning();
if (rows[0]) return rows[0];
const concurrent = await getByKey(manifest.id);
if (concurrent && concurrent.status !== "uninstalled" && sameBundledPluginRegistration(concurrent, input.packageName, manifest)) return concurrent;
throw conflict(`Plugin registration differs from the current image: ${manifest.id}`);
} catch (error) {
if (isPluginKeyConflict(error)) {
throw conflict(`Plugin already installed: ${manifest.id}`);