Forward provider credentials from verified image catalogs (#21186)

## Thinking Path

> - Paperclip manages AI agents for work.
> - Application images can ship plugins that provide task environments.
> - Paperclip starts plugin workers with a restricted process
environment.
> - An image author can know which provider credentials an image-bundled
worker needs.
> - The image catalog already identifies selected bundles and verifies
their content.
> - This change adds a way for the image catalog to declare the provider
keys that a verified worker receives.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Image-bundled environment drivers can use the worker credential delivery
already available to built-in providers.

**Current behavior**

The host provides fixed credential forwarding for certain built-in
providers.

**Proposed behavior**

A verified image bundle can receive provider API keys declared on its
own catalog entry.

**Reason and benefit**

Image authors can scope deployment credentials to the verified worker
that needs them.

## What Changed

- Add an optional provider credential declaration to the image-owned
distribution catalog.
- Scope forwarding to the verified worker for the selected bundle.
- Publish distribution catalog support for image builders.

## Verification

- The new loader regression failed before the change because the old
catalog rejected the credential declaration.
- Ran six focused Vitest suites for bundled activation, package
identity, distribution catalog validation, worker environment, and
managed configuration. 166 tests passed and one existing test was
skipped.
- Ran `pnpm --filter @paperclipai/server typecheck`. It passed.
- Reviewed the diff for secret values and personal information. Test
keys are synthetic.
- A downstream image build fixture exercised the catalog reader, bundle
verification, and worker environment together. The supported build
passed only the declared provider key to the verified worker. The build
for an older target omitted the declaration.
- Local verification used embedded PostgreSQL and prebuilt fixtures.

## Risks

- Deploy images with catalog credential declarations only to Core
versions that support the field. Older hosts reject unknown catalog
fields.
- The image author controls which verified bundle receives each provider
API key.

## Model Used

- Implementation: GPT-6.1 Sol (`gpt-6.1-sol`), delegated through Codex.
Independent Codex review and verification.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs
- [x] I have described the issue in the PR
- [x] I have used public references
- [x] My branch name describes the change
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented risks
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address reviewer comments before requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Nicky LeachandPaperclip authored and GitHub committed 2026-11-12 20:53:08 +00:00
1 parent 267983356c
commit 2168f50741
9 files changed
+127 -9

No files matched your search

+20
View File
@@ -41,6 +41,26 @@ in operation. This detects packaging errors but does not authenticate an
untrusted image builder. Image provenance and signatures remain deployment
responsibilities.
### Provider credentials
An image author can set `workerCredentialEnv` on a distribution catalog
entry to pass provider API keys from the host environment. For example,
`"workerCredentialEnv": ["EXAMPLE_API_KEY"]` lets the worker receive
`EXAMPLE_API_KEY` when the deployment sets it. List 1 to 16 distinct
uppercase environment variable names ending in `_API_KEY`, each at most
128 characters. Store the names in the catalog and set the values through
the deployment.
Core grants these variables to the selected, digest-verified worker when
it declares `environment.drivers.register`. This grant is available through
managed image activation. The ordinary `plugins.workerEnv` credential
restriction applies independently.
Before emitting this field, image builders can check for
`workerCredentialEnv` in the capability list at
`server/src/services/distribution-plugin-catalog-capabilities.json` in the
target Core source revision. Older versions reject the unknown catalog field.
## Selection, upgrades and rollback
Managed instances select a distribution key through the existing
@@ -32,7 +32,11 @@ describePostgres("bundled image registration in PostgreSQL", () => {
let db: ReturnType<typeof createDb>;
let peer: ReturnType<typeof createDb>;
const roots: string[] = [];
const fixture = () => { const result = bundledImageFixture(); roots.push(result.root); return result; };
const fixture = (...args: Parameters<typeof bundledImageFixture>) => {
const result = bundledImageFixture(...args);
roots.push(result.root);
return result;
};
beforeAll(async () => {
if (externalDatabaseUrl) {
await applyPendingMigrations(externalDatabaseUrl);
@@ -72,6 +76,55 @@ describePostgres("bundled image registration in PostgreSQL", () => {
}, services);
}
it("forwards catalog credentials only to the verified bundle with the declaration", async () => {
const manifestOverrides = {
capabilities: ["environment.drivers.register"],
environmentDrivers: [{
driverKey: "fixture-tasks",
kind: "environment_driver",
displayName: "Fixture tasks",
configSchema: { type: "object", additionalProperties: false },
}],
} satisfies Partial<ReturnType<typeof fixture>["manifest"]>;
const image = fixture(manifestOverrides, { workerCredentialEnv: ["TEST_PROVIDER_API_KEY"] });
const imageWithoutGrant = fixture(manifestOverrides);
const { services } = runtime();
const managed = loaderFor(image, db, services as unknown as PluginRuntimeServices);
const managedWithoutGrant = loaderFor(imageWithoutGrant, db, services as unknown as PluginRuntimeServices);
const unmanaged = pluginLoader(db, {}, services as unknown as PluginRuntimeServices);
await managed.installPlugin({ localPath: image.packageRoot });
const registry = pluginRegistryService(db);
const row = (await registry.getByKey(image.manifest.id))!;
await registry.updateStatus(row.id, { status: "ready" });
vi.stubEnv("TEST_PROVIDER_API_KEY", "fixture-provider-key");
vi.stubEnv("TEST_OTHER_API_KEY", "fixture-other-key");
try {
expect((await managed.loadSingle(row.id)).success).toBe(true);
const worker = services.workerManager.startWorker.mock.lastCall?.[1];
expect(worker).toMatchObject({
entrypointPath: path.join(image.packageRoot, "dist", "worker.js"),
env: { TEST_PROVIDER_API_KEY: "fixture-provider-key" },
});
expect(worker).not.toHaveProperty("env.TEST_OTHER_API_KEY");
expect((await managedWithoutGrant.loadSingle(row.id)).success).toBe(true);
const workerWithoutGrant = services.workerManager.startWorker.mock.lastCall?.[1];
expect(workerWithoutGrant).toMatchObject({ entrypointPath: path.join(imageWithoutGrant.packageRoot, "dist", "worker.js") });
expect(workerWithoutGrant).not.toHaveProperty("env.TEST_PROVIDER_API_KEY");
const copy = path.join(image.root, "operator-copy");
cpSync(image.packageRoot, copy, { recursive: true });
await registry.update(row.id, { packagePath: copy });
expect((await unmanaged.loadSingle(row.id)).success).toBe(true);
const unverifiedWorker = services.workerManager.startWorker.mock.lastCall?.[1];
expect(unverifiedWorker).toMatchObject({ entrypointPath: path.join(copy, "dist", "worker.js") });
expect(unverifiedWorker).not.toHaveProperty("env.TEST_PROVIDER_API_KEY");
} finally {
vi.unstubAllEnvs();
}
});
it("starts current-image workers after losing the old home and retains each company's config", async () => {
const old = fixture();
const first = fixture();
@@ -29,6 +29,19 @@ describe("image-owned plugin catalogs", () => {
expect(installs.map(({ key }) => key)).toEqual(["daytona", "acme.widget"]);
expect(installs[1]?.localPath).toBe(localPath);
});
it("accepts only bounded and distinct provider API key names as worker credentials", () => {
const { root, entry, save } = fixture();
save([{ ...entry, workerCredentialEnv: ["EXAMPLE_API_KEY", "SECOND_API_KEY"] }]);
expect(readDistributionPluginCatalog(root, BUNDLED_PLUGIN_CATALOG)[0]?.workerCredentialEnv)
.toEqual(["EXAMPLE_API_KEY", "SECOND_API_KEY"]);
for (const invalid of [
[], ["NODE_OPTIONS"], ["DATABASE_URL"], ["example_API_KEY"], ["EXAMPLE_API_KEY=value"],
["EXAMPLE_API_KEY", "EXAMPLE_API_KEY"], Array.from({ length: 17 }, (_, i) => `EXAMPLE_${i}_API_KEY`),
]) {
save([{ ...entry, workerCredentialEnv: invalid }]);
expect(() => readDistributionPluginCatalog(root, BUNDLED_PLUGIN_CATALOG)).toThrow(/workerCredentialEnv/);
}
});
it("rejects tampered bundle bytes before importing manifest code", () => {
const { root, localPath } = fixture();
writeFileSync(path.join(localPath, "dist", "manifest.js"), "throw new Error('must never execute');");
@@ -7,7 +7,7 @@ import { distributionBundleDigest, readDistributionPluginCatalog } from "../../s
import { derivePluginDatabaseNamespace } from "../../services/plugin-database.js";
import { createBundledPluginPackageResolver } from "../../services/bundled-plugin-package.js";
export function bundledImageFixture(manifestOverrides: Partial<PaperclipPluginManifestV1> = {}, options: { workerPrelude?: string } = {}) {
export function bundledImageFixture(manifestOverrides: Partial<PaperclipPluginManifestV1> = {}, options: { workerPrelude?: string; workerCredentialEnv?: string[] } = {}) {
const root = realpathSync(mkdtempSync(path.join(os.tmpdir(), "bundled-image-")));
const packageRoot = path.join(root, "distribution", "fixture");
const manifest: PaperclipPluginManifestV1 = {
@@ -41,7 +41,7 @@ else {
mkdirSync(path.join(packageRoot, manifest.database.migrationsDir), { recursive: true });
writeFileSync(path.join(packageRoot, manifest.database.migrationsDir, "001_fixture.sql"), `CREATE TABLE ${derivePluginDatabaseNamespace(manifest.id)}.image_rows (id uuid PRIMARY KEY);`);
}
const entry = { key: "fixture", pluginKey: manifest.id, version: manifest.version, directory: "fixture", digest: distributionBundleDigest(packageRoot) };
const entry = { key: "fixture", pluginKey: manifest.id, version: manifest.version, directory: "fixture", digest: distributionBundleDigest(packageRoot), workerCredentialEnv: options.workerCredentialEnv };
writeFileSync(path.join(root, "distribution", "catalog.json"), JSON.stringify({ schemaVersion: 1, plugins: [entry] }));
const catalog = readDistributionPluginCatalog(root, BUNDLED_PLUGIN_CATALOG);
const installs = resolveBundledPluginInstalls(["fixture"], { catalogRoot: root, enforceCatalogRoot: true, env: {}, distributionPlugins: catalog });
+18 -1
View File
@@ -152,8 +152,9 @@ describe("buildPluginWorkerEnv", () => {
const env = buildPluginWorkerEnv({
manifest: { capabilities: [] },
instanceInfo,
verifiedDistribution: { workerCredentialEnv: ["TEST_PROVIDER_API_KEY"] },
grantedEnvVars: ["PROVIDER_ORIGIN", "PROVIDER_BLANK", "PROVIDER_MISSING"],
processEnv: { PROVIDER_ORIGIN: "https://provider.test", PROVIDER_BLANK: " ", DATABASE_URL: "postgres://host", OPENAI_API_KEY: "openai-token" },
processEnv: { PROVIDER_ORIGIN: "https://provider.test", PROVIDER_BLANK: " ", DATABASE_URL: "postgres://host", OPENAI_API_KEY: "openai-token", TEST_PROVIDER_API_KEY: "fixture-provider-key" },
});
expect(env).toEqual({
PAPERCLIP_DEPLOYMENT_MODE: "authenticated",
@@ -288,6 +289,22 @@ describe("buildPluginWorkerEnv", () => {
});
});
it.each([null, "/app/packages/plugins/distribution/fixture", "/operator/plugins/fixture"])(
"denies catalog credentials without verified image provenance for %s",
(packagePath) => {
expect(buildPluginWorkerEnv({
manifest: {
capabilities: ["environment.drivers.register"],
environmentDrivers: [{ driverKey: "fixture-tasks" }],
},
packageName: "@paperclipai/plugin-image-fixture",
packagePath,
instanceInfo,
processEnv: { TEST_PROVIDER_API_KEY: "fixture-provider-key" },
})).not.toHaveProperty("TEST_PROVIDER_API_KEY");
},
);
it("passes the credential to a first-party plugin installed from the bundled catalog", () => {
const env = buildPluginWorkerEnv({
manifest: {
@@ -102,7 +102,9 @@ export function createBundledPluginPackageResolver(installs: readonly ResolvedBu
const install = input.pluginKey ? byKey.get(input.pluginKey) : installs.find((candidate) => candidate.localPath === input.packageRoot);
if (!install || path.resolve(input.packageRoot) !== install.localPath) throw new Error("Bundled plugin is absent or not selected in this deployment");
read(install);
if (input.manifest) assertManifest(install, input.manifest, input.previousManifest);
if (!input.manifest) return;
assertManifest(install, input.manifest, input.previousManifest);
return install.distribution;
},
};
}
@@ -0,0 +1 @@
{ "schemaVersion": 1, "capabilities": ["workerCredentialEnv"] }
@@ -6,6 +6,8 @@ import type { PaperclipPluginManifestV1 } from "@paperclipai/shared";
import { pluginCapabilityValidator } from "./plugin-capability-validator.js";
const segment = z.string().regex(/^[a-z][a-z0-9.-]{0,99}$/);
const workerCredentialEnv = z.array(z.string().max(128).regex(/^[A-Z][A-Z0-9_]*_API_KEY$/))
.min(1).max(16).refine((names) => new Set(names).size === names.length, "Worker credential names must be distinct");
export const distributionPluginCatalogSchema = z.object({
schemaVersion: z.literal(1),
plugins: z.array(z.object({
@@ -14,6 +16,7 @@ export const distributionPluginCatalogSchema = z.object({
version: z.string().regex(/^\d+\.\d+\.\d+(?:-[a-zA-Z0-9.-]+)?(?:\+[a-zA-Z0-9.-]+)?$/),
directory: segment,
digest: z.string().regex(/^sha256:[a-f0-9]{64}$/),
workerCredentialEnv: workerCredentialEnv.optional(),
}).strict()).max(100),
}).strict();
+13 -4
View File
@@ -52,6 +52,7 @@ import type { PluginJobStore } from "./plugin-job-store.js";
import type { PluginToolDispatcher } from "./plugin-tool-dispatcher.js";
import type { PluginLifecycleManager } from "./plugin-lifecycle.js";
import { pluginDatabaseService } from "./plugin-database.js";
import type { DistributionPlugin } from "./distribution-plugin-catalog.js";
import { resolveBundledCatalogRoot } from "./bundled-plugins.js";
import { managedPluginWorkerEnvGrant } from "./managed-plugin-worker-env.js";
import { createWorkScheduler } from "./work-scheduler.js";
@@ -160,6 +161,8 @@ export function buildPluginWorkerEnv(input: {
packageName?: string;
/** Local install path (`PluginRecord.packagePath`); null for registry installs. */
packagePath?: string | null;
/** Selected image distribution verified during managed package activation. */
verifiedDistribution?: Pick<DistributionPlugin, "workerCredentialEnv">;
/** Test seam; defaults to the repo plugin tree and the bundled catalog root. */
trustedLocalPluginRoots?: readonly string[];
grantedEnvVars?: readonly string[];
@@ -194,7 +197,12 @@ export function buildPluginWorkerEnv(input: {
)
? credentialEntry.envVars
: [];
for (const key of [...ADAPTER_ENV_PASSTHROUGH, ...K8S_IN_CLUSTER_ENV_PASSTHROUGH, ...credentialKeys]) {
for (const key of [
...ADAPTER_ENV_PASSTHROUGH,
...K8S_IN_CLUSTER_ENV_PASSTHROUGH,
...credentialKeys,
...(input.verifiedDistribution?.workerCredentialEnv ?? []),
]) {
const value = processEnv[key];
if (value && value.trim().length > 0) {
env[key] = value;
@@ -281,7 +289,7 @@ function getDeclaredPageRoutePaths(manifest: PaperclipPluginManifestV1): string[
export interface PluginLoaderOptions {
resolvePackageRoot?: (plugin: PluginRecord) => string;
prebuiltOnly?: boolean;
/** Image-owned deployment policy, checked before importing code and starting workers. */
/** Check image policy and return verified distribution evidence when available. */
assertPackageActivation?: (input: {
pluginKey?: string;
packageRoot: string;
@@ -290,7 +298,7 @@ export interface PluginLoaderOptions {
manifest?: PaperclipPluginManifestV1;
/** Persisted grants, supplied before a runtime manifest refresh is saved. */
previousManifest?: PaperclipPluginManifestV1;
}) => void;
}) => DistributionPlugin | void;
/**
* Path to the local plugin directory to scan.
* Defaults to ~/.paperclip/plugins/
@@ -2370,7 +2378,7 @@ export function pluginLoader(
assertPackageActivation?.({ pluginKey, packageRoot, installedPackagePath: activePlugin.packagePath });
activePlugin = await refreshPluginManifestFromPackage(activePlugin, packageRoot);
manifest = activePlugin.manifestJson;
assertPackageActivation?.({ pluginKey, packageRoot, installedPackagePath: activePlugin.packagePath, manifest });
const verifiedDistribution = assertPackageActivation?.({ pluginKey, packageRoot, installedPackagePath: activePlugin.packagePath, manifest });
const workerEntrypoint = resolvePackageRoot
? path.resolve(packageRoot, manifest.entrypoints.worker)
: resolveWorkerEntrypoint(activePlugin, localPluginDir);
@@ -2441,6 +2449,7 @@ export function pluginLoader(
manifest,
packageName: activePlugin.packageName,
packagePath: resolvePackageRoot ? packageRoot : activePlugin.packagePath,
verifiedDistribution: resolvePackageRoot && verifiedDistribution ? verifiedDistribution : undefined,
instanceInfo,
grantedEnvVars: managedPluginWorkerEnvGrant(activePlugin.pluginKey),
}),