Files
PaperClipAI/server/src/services/environment-custom-image-runtime.ts
T
Devin FoleyandPaperclip b4815bf964 Scope environment custom images to instance environments (#8850)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Environments are now managed as instance-level runtime resources
rather than per-company rows
> - The custom environment image setup tables were introduced with their
own `company_id` columns and route query parameters
> - That split made one saved environment image state depend on an extra
company context even though the environment itself is the durable owner
> - It also made saved-environment probes harder because applying the
active custom image template could require a company context when no
secret-backed config needed one
> - This pull request scopes custom image templates and setup sessions
directly to the saved environment
> - The benefit is that reusable environment images follow the same
instance-scoped model as environments while secret resolution still uses
company context only when secrets require it

## Linked Issues or Issue Description

No matching public GitHub issue was found.

Bug report:

### What happened?

saved environment custom-image routes and persistence required a
`companyId` even though environments are instance-scoped, and saved
sandbox probes did not opt into active custom-image template application
unless a company context was present.

### Expected behavior

custom-image templates and setup sessions should be owned by the saved
environment, and saved sandbox probes should apply the active template
while still requiring a company context only for secret-backed runtime
config.

### Steps to reproduce

1. Configure an instance-scoped sandbox environment with custom-image
setup support.
2. Start or inspect a custom-image session or template for that saved
environment.
3. Probe the saved environment without a custom-image-specific
`companyId` query parameter.

### Paperclip version or commit

current `master` after the environment custom-image template migration.

### Deployment mode

Local dev (pnpm dev) or authenticated local Paperclip instance.

### Installation method

Built from source (pnpm dev / pnpm build).

### Agent adapter(s) involved

Not adapter-specific (core bug).

### Database mode

Embedded PGlite/Postgres dev database.

### Access context

Board human operator.

### Privacy checklist

No logs, secrets, tokens, private URLs, or local machine paths are
included.

Duplicate search performed:

- `gh search prs "environment custom image companyId
repo:paperclipai/paperclip" --state open --limit 20`
- `gh search prs "custom image environment scoped
repo:paperclipai/paperclip" --state open --limit 20`
- `gh search issues "environment custom image
repo:paperclipai/paperclip" --state open --limit 20`

The returned results were unrelated adapter, Docker, auth, or
stale-workspace items.

## What Changed

- Removed redundant `company_id` columns from environment custom-image
templates and setup sessions.
- Added migration `0127_environment_custom_images_instance_scoped` to
collapse duplicate active rows per environment before dropping the old
company-scoped indexes/columns.
- Updated custom-image services, route handlers, shared validators, and
UI API/query keys to use environment-scoped custom-image state.
- Kept runtime secret resolution company-aware only when secret refs or
bindings require a company context.
- Made saved sandbox environment probes opt into active custom-image
template application.
- Updated DB, shared, server, and UI tests for the new
environment-scoped contract.

## Verification

- `pnpm --filter @paperclipai/db run check:migrations`
- `pnpm exec vitest run
packages/db/src/environment-custom-images-schema.test.ts
packages/shared/src/environment-custom-images.test.ts
server/src/__tests__/environment-custom-image-routes.test.ts
server/src/__tests__/environment-custom-images-service.test.ts
server/src/__tests__/environment-routes.test.ts
ui/src/pages/CompanyEnvironments.test.tsx`
- `pnpm -r typecheck`
- `pnpm test:run` before rebasing onto latest `master`; after the rebase
only the migration number changed, and the migration check plus focused
suite, typecheck, and build were rerun.
- `pnpm build`

## Risks

- Migration safety: the migration supersedes duplicate active templates
per environment and fails duplicate active setup sessions before adding
environment-only unique indexes. Operators with duplicate historical
active rows should review which active template is kept.
- Behavior shift: plugin custom-image setup calls now receive
`companyId: "instance"` when no secret binding determines a concrete
company context.
- Secret-backed configs still require an explicit or uniquely inferable
company context; environments with secret bindings spread across
multiple companies continue to fail fast.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex via the `codex_local` adapter, GPT-5-based coding model
with tool-enabled repository inspection, editing, testing, git, and
GitHub CLI access. Exact context-window metadata was not exposed by the
runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-01 23:57:31 -07:00

174 lines
6.8 KiB
TypeScript

import { createHash } from "node:crypto";
import { and, desc, eq } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import { environmentCustomImageTemplates } from "@paperclipai/db";
import {
ENVIRONMENT_CUSTOM_IMAGE_TEMPLATE_KINDS,
type EnvironmentCustomImageTemplate,
type EnvironmentCustomImageTemplateKind,
type SandboxEnvironmentConfig,
} from "@paperclipai/shared";
import { writeConfigValueAtPath } from "./json-schema-secret-refs.js";
type TemplateRow = typeof environmentCustomImageTemplates.$inferSelect;
export const ENVIRONMENT_CUSTOM_IMAGE_RUNTIME_CONFIG_BINDING_METADATA_KEY = "runtimeConfigBinding";
export interface EnvironmentCustomImageRuntimeConfigBinding {
field: string;
unsetFields: string[];
}
function isRecord(value: unknown): value is Record<string, unknown> {
return Boolean(value) && typeof value === "object" && !Array.isArray(value);
}
function stableStringify(value: unknown): string {
if (Array.isArray(value)) {
return `[${value.map((entry) => stableStringify(entry)).join(",")}]`;
}
if (isRecord(value)) {
return `{${Object.keys(value).sort().map((key) => `${JSON.stringify(key)}:${stableStringify(value[key])}`).join(",")}}`;
}
return JSON.stringify(value) ?? "null";
}
export function readEnvironmentCustomImageTemplateKind(
value: string | null,
): EnvironmentCustomImageTemplateKind {
return (ENVIRONMENT_CUSTOM_IMAGE_TEMPLATE_KINDS as readonly string[]).includes(value ?? "")
? value as EnvironmentCustomImageTemplateKind
: "unknown";
}
export function defaultEnvironmentCustomImageRuntimeConfigBinding(
templateKind: string | null | undefined,
): EnvironmentCustomImageRuntimeConfigBinding {
const kind = readEnvironmentCustomImageTemplateKind(templateKind ?? null);
if (kind === "snapshot") return { field: "snapshot", unsetFields: ["image"] };
if (kind === "image") return { field: "image", unsetFields: ["snapshot"] };
if (kind === "provider_template") return { field: "template", unsetFields: [] };
return { field: "templateRef", unsetFields: [] };
}
function isValidRuntimeConfigBindingField(value: unknown): value is string {
return typeof value === "string"
&& /^[A-Za-z_][A-Za-z0-9_-]*$/.test(value)
&& value !== "provider";
}
export function normalizeEnvironmentCustomImageRuntimeConfigBinding(
value: unknown,
): EnvironmentCustomImageRuntimeConfigBinding | null {
if (!isRecord(value) || !isValidRuntimeConfigBindingField(value.field)) return null;
const unsetFields = Array.isArray(value.unsetFields)
? value.unsetFields.filter((field): field is string =>
isValidRuntimeConfigBindingField(field) && field !== value.field,
)
: [];
return {
field: value.field,
unsetFields: Array.from(new Set(unsetFields)),
};
}
export function resolveEnvironmentCustomImageRuntimeConfigBinding(input: {
templateKind: string | null | undefined;
metadata?: Record<string, unknown> | null;
}): EnvironmentCustomImageRuntimeConfigBinding {
return normalizeEnvironmentCustomImageRuntimeConfigBinding(
input.metadata?.[ENVIRONMENT_CUSTOM_IMAGE_RUNTIME_CONFIG_BINDING_METADATA_KEY],
) ?? defaultEnvironmentCustomImageRuntimeConfigBinding(input.templateKind);
}
export function fingerprintEnvironmentSandboxProviderConfig(
config: SandboxEnvironmentConfig,
options?: { excludePaths?: Iterable<string> },
): string {
let normalized = config as Record<string, unknown>;
for (const path of options?.excludePaths ?? []) {
normalized = writeConfigValueAtPath(normalized, path, undefined);
}
return createHash("sha256")
.update(stableStringify(normalized))
.digest("hex");
}
export function applyCustomImageTemplateToSandboxConfig(
config: SandboxEnvironmentConfig,
template: Pick<EnvironmentCustomImageTemplate, "templateKind" | "templateRef" | "metadata">,
): SandboxEnvironmentConfig {
if (!template.templateRef) return config;
const next = { ...(config as Record<string, unknown>) };
const binding = resolveEnvironmentCustomImageRuntimeConfigBinding({
templateKind: template.templateKind,
metadata: template.metadata,
});
for (const field of binding.unsetFields) {
delete next[field];
}
next[binding.field] = template.templateRef;
return next as SandboxEnvironmentConfig;
}
export function environmentCustomImageTemplateFromRow(row: TemplateRow): EnvironmentCustomImageTemplate {
return {
id: row.id,
environmentId: row.environmentId,
provider: row.provider,
templateKind: readEnvironmentCustomImageTemplateKind(row.templateKind),
templateRef: row.templateRef,
sourceTemplateRef: row.sourceTemplateRef ?? null,
sourceEnvironmentConfigFingerprint: row.sourceEnvironmentConfigFingerprint ?? null,
status: row.status,
createdByUserId: row.createdByUserId ?? null,
createdByAgentId: row.createdByAgentId ?? null,
capturedAt: row.capturedAt ?? null,
lastUsedAt: row.lastUsedAt ?? null,
supersededByTemplateId: row.supersededByTemplateId ?? null,
metadata: row.metadata ?? null,
createdAt: row.createdAt,
updatedAt: row.updatedAt,
};
}
export async function resolveActiveEnvironmentCustomImageTemplateForRuntime(
db: Db,
input: {
environmentId: string;
baseConfig: SandboxEnvironmentConfig;
runtimeConfig: SandboxEnvironmentConfig;
now?: Date;
},
): Promise<SandboxEnvironmentConfig> {
const row = await db
.select()
.from(environmentCustomImageTemplates)
.where(and(
eq(environmentCustomImageTemplates.environmentId, input.environmentId),
eq(environmentCustomImageTemplates.provider, input.baseConfig.provider),
eq(environmentCustomImageTemplates.status, "active"),
))
.orderBy(desc(environmentCustomImageTemplates.capturedAt), desc(environmentCustomImageTemplates.createdAt))
.then((rows) => rows[0] ?? null);
if (!row) return input.runtimeConfig;
const active = environmentCustomImageTemplateFromRow(row);
if (!active.templateRef) return input.runtimeConfig;
// An active template is an explicit, environment+provider-scoped artifact: the
// captured snapshot/image fully replaces the base image at create time, so it is
// applied whenever present. We deliberately do not gate on a base-config
// fingerprint — the config dialog re-saves the environment right after capture,
// and ordinary resource/lifecycle tweaks (cpu/memory/lease knobs that don't
// affect image identity and are dropped for snapshot creation) would otherwise
// silently discard the captured setup state and provider metadata. Re-running
// setup supersedes the template when the user wants a fresh capture.
const now = input.now ?? new Date();
await db
.update(environmentCustomImageTemplates)
.set({ lastUsedAt: now, updatedAt: now })
.where(eq(environmentCustomImageTemplates.id, active.id));
return applyCustomImageTemplateToSandboxConfig(input.runtimeConfig, active);
}