mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-26 18:31:34 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Environments are now managed as instance-level runtime resources rather than per-company rows > - The custom environment image setup tables were introduced with their own `company_id` columns and route query parameters > - That split made one saved environment image state depend on an extra company context even though the environment itself is the durable owner > - It also made saved-environment probes harder because applying the active custom image template could require a company context when no secret-backed config needed one > - This pull request scopes custom image templates and setup sessions directly to the saved environment > - The benefit is that reusable environment images follow the same instance-scoped model as environments while secret resolution still uses company context only when secrets require it ## Linked Issues or Issue Description No matching public GitHub issue was found. Bug report: ### What happened? saved environment custom-image routes and persistence required a `companyId` even though environments are instance-scoped, and saved sandbox probes did not opt into active custom-image template application unless a company context was present. ### Expected behavior custom-image templates and setup sessions should be owned by the saved environment, and saved sandbox probes should apply the active template while still requiring a company context only for secret-backed runtime config. ### Steps to reproduce 1. Configure an instance-scoped sandbox environment with custom-image setup support. 2. Start or inspect a custom-image session or template for that saved environment. 3. Probe the saved environment without a custom-image-specific `companyId` query parameter. ### Paperclip version or commit current `master` after the environment custom-image template migration. ### Deployment mode Local dev (pnpm dev) or authenticated local Paperclip instance. ### Installation method Built from source (pnpm dev / pnpm build). ### Agent adapter(s) involved Not adapter-specific (core bug). ### Database mode Embedded PGlite/Postgres dev database. ### Access context Board human operator. ### Privacy checklist No logs, secrets, tokens, private URLs, or local machine paths are included. Duplicate search performed: - `gh search prs "environment custom image companyId repo:paperclipai/paperclip" --state open --limit 20` - `gh search prs "custom image environment scoped repo:paperclipai/paperclip" --state open --limit 20` - `gh search issues "environment custom image repo:paperclipai/paperclip" --state open --limit 20` The returned results were unrelated adapter, Docker, auth, or stale-workspace items. ## What Changed - Removed redundant `company_id` columns from environment custom-image templates and setup sessions. - Added migration `0127_environment_custom_images_instance_scoped` to collapse duplicate active rows per environment before dropping the old company-scoped indexes/columns. - Updated custom-image services, route handlers, shared validators, and UI API/query keys to use environment-scoped custom-image state. - Kept runtime secret resolution company-aware only when secret refs or bindings require a company context. - Made saved sandbox environment probes opt into active custom-image template application. - Updated DB, shared, server, and UI tests for the new environment-scoped contract. ## Verification - `pnpm --filter @paperclipai/db run check:migrations` - `pnpm exec vitest run packages/db/src/environment-custom-images-schema.test.ts packages/shared/src/environment-custom-images.test.ts server/src/__tests__/environment-custom-image-routes.test.ts server/src/__tests__/environment-custom-images-service.test.ts server/src/__tests__/environment-routes.test.ts ui/src/pages/CompanyEnvironments.test.tsx` - `pnpm -r typecheck` - `pnpm test:run` before rebasing onto latest `master`; after the rebase only the migration number changed, and the migration check plus focused suite, typecheck, and build were rerun. - `pnpm build` ## Risks - Migration safety: the migration supersedes duplicate active templates per environment and fails duplicate active setup sessions before adding environment-only unique indexes. Operators with duplicate historical active rows should review which active template is kept. - Behavior shift: plugin custom-image setup calls now receive `companyId: "instance"` when no secret binding determines a concrete company context. - Secret-backed configs still require an explicit or uniquely inferable company context; environments with secret bindings spread across multiple companies continue to fail fast. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex via the `codex_local` adapter, GPT-5-based coding model with tool-enabled repository inspection, editing, testing, git, and GitHub CLI access. Exact context-window metadata was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
174 lines
6.8 KiB
TypeScript
174 lines
6.8 KiB
TypeScript
import { createHash } from "node:crypto";
|
|
import { and, desc, eq } from "drizzle-orm";
|
|
import type { Db } from "@paperclipai/db";
|
|
import { environmentCustomImageTemplates } from "@paperclipai/db";
|
|
import {
|
|
ENVIRONMENT_CUSTOM_IMAGE_TEMPLATE_KINDS,
|
|
type EnvironmentCustomImageTemplate,
|
|
type EnvironmentCustomImageTemplateKind,
|
|
type SandboxEnvironmentConfig,
|
|
} from "@paperclipai/shared";
|
|
import { writeConfigValueAtPath } from "./json-schema-secret-refs.js";
|
|
|
|
type TemplateRow = typeof environmentCustomImageTemplates.$inferSelect;
|
|
|
|
export const ENVIRONMENT_CUSTOM_IMAGE_RUNTIME_CONFIG_BINDING_METADATA_KEY = "runtimeConfigBinding";
|
|
|
|
export interface EnvironmentCustomImageRuntimeConfigBinding {
|
|
field: string;
|
|
unsetFields: string[];
|
|
}
|
|
|
|
function isRecord(value: unknown): value is Record<string, unknown> {
|
|
return Boolean(value) && typeof value === "object" && !Array.isArray(value);
|
|
}
|
|
|
|
function stableStringify(value: unknown): string {
|
|
if (Array.isArray(value)) {
|
|
return `[${value.map((entry) => stableStringify(entry)).join(",")}]`;
|
|
}
|
|
if (isRecord(value)) {
|
|
return `{${Object.keys(value).sort().map((key) => `${JSON.stringify(key)}:${stableStringify(value[key])}`).join(",")}}`;
|
|
}
|
|
return JSON.stringify(value) ?? "null";
|
|
}
|
|
|
|
export function readEnvironmentCustomImageTemplateKind(
|
|
value: string | null,
|
|
): EnvironmentCustomImageTemplateKind {
|
|
return (ENVIRONMENT_CUSTOM_IMAGE_TEMPLATE_KINDS as readonly string[]).includes(value ?? "")
|
|
? value as EnvironmentCustomImageTemplateKind
|
|
: "unknown";
|
|
}
|
|
|
|
export function defaultEnvironmentCustomImageRuntimeConfigBinding(
|
|
templateKind: string | null | undefined,
|
|
): EnvironmentCustomImageRuntimeConfigBinding {
|
|
const kind = readEnvironmentCustomImageTemplateKind(templateKind ?? null);
|
|
if (kind === "snapshot") return { field: "snapshot", unsetFields: ["image"] };
|
|
if (kind === "image") return { field: "image", unsetFields: ["snapshot"] };
|
|
if (kind === "provider_template") return { field: "template", unsetFields: [] };
|
|
return { field: "templateRef", unsetFields: [] };
|
|
}
|
|
|
|
function isValidRuntimeConfigBindingField(value: unknown): value is string {
|
|
return typeof value === "string"
|
|
&& /^[A-Za-z_][A-Za-z0-9_-]*$/.test(value)
|
|
&& value !== "provider";
|
|
}
|
|
|
|
export function normalizeEnvironmentCustomImageRuntimeConfigBinding(
|
|
value: unknown,
|
|
): EnvironmentCustomImageRuntimeConfigBinding | null {
|
|
if (!isRecord(value) || !isValidRuntimeConfigBindingField(value.field)) return null;
|
|
const unsetFields = Array.isArray(value.unsetFields)
|
|
? value.unsetFields.filter((field): field is string =>
|
|
isValidRuntimeConfigBindingField(field) && field !== value.field,
|
|
)
|
|
: [];
|
|
return {
|
|
field: value.field,
|
|
unsetFields: Array.from(new Set(unsetFields)),
|
|
};
|
|
}
|
|
|
|
export function resolveEnvironmentCustomImageRuntimeConfigBinding(input: {
|
|
templateKind: string | null | undefined;
|
|
metadata?: Record<string, unknown> | null;
|
|
}): EnvironmentCustomImageRuntimeConfigBinding {
|
|
return normalizeEnvironmentCustomImageRuntimeConfigBinding(
|
|
input.metadata?.[ENVIRONMENT_CUSTOM_IMAGE_RUNTIME_CONFIG_BINDING_METADATA_KEY],
|
|
) ?? defaultEnvironmentCustomImageRuntimeConfigBinding(input.templateKind);
|
|
}
|
|
|
|
export function fingerprintEnvironmentSandboxProviderConfig(
|
|
config: SandboxEnvironmentConfig,
|
|
options?: { excludePaths?: Iterable<string> },
|
|
): string {
|
|
let normalized = config as Record<string, unknown>;
|
|
for (const path of options?.excludePaths ?? []) {
|
|
normalized = writeConfigValueAtPath(normalized, path, undefined);
|
|
}
|
|
return createHash("sha256")
|
|
.update(stableStringify(normalized))
|
|
.digest("hex");
|
|
}
|
|
|
|
export function applyCustomImageTemplateToSandboxConfig(
|
|
config: SandboxEnvironmentConfig,
|
|
template: Pick<EnvironmentCustomImageTemplate, "templateKind" | "templateRef" | "metadata">,
|
|
): SandboxEnvironmentConfig {
|
|
if (!template.templateRef) return config;
|
|
const next = { ...(config as Record<string, unknown>) };
|
|
const binding = resolveEnvironmentCustomImageRuntimeConfigBinding({
|
|
templateKind: template.templateKind,
|
|
metadata: template.metadata,
|
|
});
|
|
for (const field of binding.unsetFields) {
|
|
delete next[field];
|
|
}
|
|
next[binding.field] = template.templateRef;
|
|
return next as SandboxEnvironmentConfig;
|
|
}
|
|
|
|
export function environmentCustomImageTemplateFromRow(row: TemplateRow): EnvironmentCustomImageTemplate {
|
|
return {
|
|
id: row.id,
|
|
environmentId: row.environmentId,
|
|
provider: row.provider,
|
|
templateKind: readEnvironmentCustomImageTemplateKind(row.templateKind),
|
|
templateRef: row.templateRef,
|
|
sourceTemplateRef: row.sourceTemplateRef ?? null,
|
|
sourceEnvironmentConfigFingerprint: row.sourceEnvironmentConfigFingerprint ?? null,
|
|
status: row.status,
|
|
createdByUserId: row.createdByUserId ?? null,
|
|
createdByAgentId: row.createdByAgentId ?? null,
|
|
capturedAt: row.capturedAt ?? null,
|
|
lastUsedAt: row.lastUsedAt ?? null,
|
|
supersededByTemplateId: row.supersededByTemplateId ?? null,
|
|
metadata: row.metadata ?? null,
|
|
createdAt: row.createdAt,
|
|
updatedAt: row.updatedAt,
|
|
};
|
|
}
|
|
|
|
export async function resolveActiveEnvironmentCustomImageTemplateForRuntime(
|
|
db: Db,
|
|
input: {
|
|
environmentId: string;
|
|
baseConfig: SandboxEnvironmentConfig;
|
|
runtimeConfig: SandboxEnvironmentConfig;
|
|
now?: Date;
|
|
},
|
|
): Promise<SandboxEnvironmentConfig> {
|
|
const row = await db
|
|
.select()
|
|
.from(environmentCustomImageTemplates)
|
|
.where(and(
|
|
eq(environmentCustomImageTemplates.environmentId, input.environmentId),
|
|
eq(environmentCustomImageTemplates.provider, input.baseConfig.provider),
|
|
eq(environmentCustomImageTemplates.status, "active"),
|
|
))
|
|
.orderBy(desc(environmentCustomImageTemplates.capturedAt), desc(environmentCustomImageTemplates.createdAt))
|
|
.then((rows) => rows[0] ?? null);
|
|
if (!row) return input.runtimeConfig;
|
|
|
|
const active = environmentCustomImageTemplateFromRow(row);
|
|
if (!active.templateRef) return input.runtimeConfig;
|
|
|
|
// An active template is an explicit, environment+provider-scoped artifact: the
|
|
// captured snapshot/image fully replaces the base image at create time, so it is
|
|
// applied whenever present. We deliberately do not gate on a base-config
|
|
// fingerprint — the config dialog re-saves the environment right after capture,
|
|
// and ordinary resource/lifecycle tweaks (cpu/memory/lease knobs that don't
|
|
// affect image identity and are dropped for snapshot creation) would otherwise
|
|
// silently discard the captured setup state and provider metadata. Re-running
|
|
// setup supersedes the template when the user wants a fresh capture.
|
|
const now = input.now ?? new Date();
|
|
await db
|
|
.update(environmentCustomImageTemplates)
|
|
.set({ lastUsedAt: now, updatedAt: now })
|
|
.where(eq(environmentCustomImageTemplates.id, active.id));
|
|
return applyCustomImageTemplateToSandboxConfig(input.runtimeConfig, active);
|
|
}
|