mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-23 00:53:38 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Paperclip creates a managed sandbox environment for each company during boot > - Operators can change environment fields after Paperclip creates the environment > - The boot reconciler replaced those changes without checking for drift > - This pull request adds stock hashes and transactional drift reconciliation > - The benefit is that Paperclip can update untouched stock fields without losing operator work ## Linked Issues or Issue Description **What happened?** The managed sandbox boot reconciler rewrote the stock description, configuration, metadata, and status on every start. It did not detect operator changes first. A restart could therefore remove an operator's changes. **Expected behavior** Paperclip must preserve operator changes by default. It must update an untouched stock environment when Paperclip ships new stock values. It must perform each row update and stock-hash update atomically. **Steps to reproduce** 1. Start Paperclip and let it create the managed sandbox environment. 2. Change one Paperclip-owned stock field on that environment. 3. Restart Paperclip. 4. Observe that the previous reconciler replaced the change. **Paperclip version or commit** This bug reproduces on `master` before this change. **Deployment mode** Local development and self-hosted server boot are affected. ## What Changed - Add a shared deterministic stock-hash and drift classifier for built-in resources. - Track the managed sandbox stock hash with the company-scoped built-in resource binding. - Reconcile the environment and its stock metadata in one transaction with a row lock. - Preserve operator-modified and unmanaged rows and report their skipped update state. - Use archive ownership tokens so provider recovery reactivates only Paperclip-archived rows and preserves later operator archive decisions. - Keep operator-owned environment variables and unrelated metadata out of the stock fingerprint. - Add activity records for managed environment creation, updates, skipped drift, tracking initialization, and archive changes. - Add regression tests for current stock, available stock updates, operator drift, unmanaged rows, archive and reactivation, user-owned fields, and concurrent reconciliation. ## Verification - `pnpm -r typecheck` - `pnpm build` - `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY pnpm test:run -- --mode serialized` (128 suites passed) - Repository general server, UI, CLI, shared, skills catalog, database, adapter, plugin SDK, and plugin creator projects passed. Two embedded-Postgres tests exceeded the host's five-second default under the aggregate run and passed in the complete database project with `--testTimeout=20000`. One timing-sensitive sandbox stream test passed on its focused retry. - Focused managed-environment unit and integration coverage passed: 49 tests across the drift classifier, boot report, and environment service suites. ## Risks - The main risk is an incorrect ownership boundary in the stock fingerprint. The fingerprint includes only Paperclip-owned stock fields. Tests confirm that environment variables and unrelated metadata survive reconciliation. - Concurrent reconciliation could otherwise overwrite a late operator edit. The implementation locks the environment row and updates the row and hash binding in one transaction. A concurrency test covers this path. - There is no schema migration. Existing managed rows initialize tracking without replacing their current values. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex. The exact serving snapshot and context-window size were not exposed. The model used reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
464 lines
16 KiB
TypeScript
464 lines
16 KiB
TypeScript
import { describe, expect, it, vi } from "vitest";
|
|
import type { Db } from "@paperclipai/db";
|
|
import { MANAGED_CONFIG_ENV_KEY, parseManagedConfigEnv } from "./managed-config.js";
|
|
import {
|
|
applyManagedEnvironments,
|
|
type ApplyManagedEnvironmentsOptions,
|
|
} from "./managed-environments.js";
|
|
|
|
const noDb = null as unknown as Db;
|
|
|
|
function parsedConfig(overrides: Record<string, unknown> = {}) {
|
|
const config = parseManagedConfigEnv({
|
|
[MANAGED_CONFIG_ENV_KEY]: JSON.stringify({
|
|
v: 1,
|
|
mode: "cloud",
|
|
catalogVersion: "2026.720.0",
|
|
features: {},
|
|
plugins: { autoInstall: ["daytona"] },
|
|
...overrides,
|
|
}),
|
|
});
|
|
if (!config) throw new Error("expected a parsed managed config");
|
|
return config;
|
|
}
|
|
|
|
function environmentRow(overrides: Record<string, unknown> = {}) {
|
|
const now = new Date();
|
|
return {
|
|
id: "env-1",
|
|
name: "Daytona",
|
|
description: null,
|
|
driver: "sandbox",
|
|
status: "active",
|
|
config: {},
|
|
envVars: {},
|
|
metadata: {},
|
|
createdAt: now,
|
|
updatedAt: now,
|
|
...overrides,
|
|
};
|
|
}
|
|
|
|
function reconciliationResult(
|
|
overrides: Record<string, unknown> = {},
|
|
) {
|
|
return {
|
|
environment: environmentRow(),
|
|
action: "added" as const,
|
|
stockStatus: "missing" as const,
|
|
updateAvailable: false,
|
|
stockHash: "sha256:stock",
|
|
...overrides,
|
|
};
|
|
}
|
|
|
|
function readyDriverResolver(status = "ready") {
|
|
return vi.fn(async () => ({
|
|
plugin: { id: "plugin-1", pluginKey: "sandbox-providers/daytona", status },
|
|
}));
|
|
}
|
|
|
|
function runningWorkerManager(running = true) {
|
|
return { isRunning: vi.fn(() => running), getWorker: vi.fn(() => undefined) };
|
|
}
|
|
|
|
type WorkerManagerSeam = NonNullable<ApplyManagedEnvironmentsOptions["workerManager"]>;
|
|
type RecoveryHandle = NonNullable<ReturnType<WorkerManagerSeam["getWorker"]>>;
|
|
|
|
/** A worker handle fake that records `ready` listeners so tests can fire them. */
|
|
function recoveryHandle() {
|
|
const readyListeners: Array<(payload: { pluginId: string }) => void> = [];
|
|
const handle: RecoveryHandle = {
|
|
on: vi.fn((_event, listener) => {
|
|
readyListeners.push(listener);
|
|
}),
|
|
off: vi.fn((_event, listener) => {
|
|
const at = readyListeners.indexOf(listener);
|
|
if (at !== -1) readyListeners.splice(at, 1);
|
|
}),
|
|
};
|
|
return {
|
|
handle,
|
|
fireReady: () => {
|
|
for (const listener of [...readyListeners]) listener({ pluginId: "plugin-1" });
|
|
},
|
|
};
|
|
}
|
|
|
|
/** Lets the fire-and-forget recovery ensure settle. */
|
|
function tick() {
|
|
return new Promise((resolve) => setImmediate(resolve));
|
|
}
|
|
|
|
type EnvironmentsSeam = NonNullable<ApplyManagedEnvironmentsOptions["environments"]>;
|
|
|
|
function environmentsSeam(overrides: Partial<EnvironmentsSeam> = {}): EnvironmentsSeam {
|
|
return {
|
|
ensureManagedSandboxEnvironment:
|
|
overrides.ensureManagedSandboxEnvironment ?? vi.fn().mockResolvedValue(reconciliationResult()),
|
|
archiveManagedSandboxEnvironment:
|
|
overrides.archiveManagedSandboxEnvironment ?? vi.fn().mockResolvedValue(null),
|
|
};
|
|
}
|
|
|
|
describe("applyManagedEnvironments", () => {
|
|
it("no-ops for self-hosted instances and for documents without environments", async () => {
|
|
expect(await applyManagedEnvironments(noDb, null)).toBeNull();
|
|
expect(await applyManagedEnvironments(noDb, parsedConfig())).toBeNull();
|
|
});
|
|
|
|
it("refuses startup when a forced execution mode also claims the managed sandbox slot", async () => {
|
|
const config = parsedConfig({
|
|
environments: [{ name: "Daytona", provider: "daytona" }],
|
|
});
|
|
await expect(
|
|
applyManagedEnvironments(noDb, config, {
|
|
env: { PAPERCLIP_EXECUTION_MODE: "kubernetes" },
|
|
}),
|
|
).rejects.toThrow(/mutually exclusive/);
|
|
});
|
|
|
|
it("ensures each declared environment through the provider-agnostic service call", async () => {
|
|
const ensureManagedSandboxEnvironment = vi
|
|
.fn()
|
|
.mockResolvedValue(reconciliationResult());
|
|
const config = parsedConfig({
|
|
environments: [
|
|
{
|
|
name: "Daytona",
|
|
description: "Managed Daytona sandbox.",
|
|
provider: "daytona",
|
|
config: { target: "us" },
|
|
},
|
|
],
|
|
});
|
|
|
|
const resolveSandboxProviderDriver = readyDriverResolver();
|
|
const workerManager = runningWorkerManager();
|
|
const result = await applyManagedEnvironments(noDb, config, {
|
|
env: {},
|
|
workerManager,
|
|
environments: environmentsSeam({ ensureManagedSandboxEnvironment }),
|
|
resolveSandboxProviderDriver,
|
|
});
|
|
|
|
expect(result).toMatchObject({ ensured: 1, failed: 0, added: 1, skipped: 0 });
|
|
expect(result).toMatchObject({ removed: 0, backedUp: 0 });
|
|
expect(resolveSandboxProviderDriver).toHaveBeenCalledWith({ db: noDb, driverKey: "daytona" });
|
|
expect(workerManager.isRunning).toHaveBeenCalledWith("plugin-1");
|
|
expect(ensureManagedSandboxEnvironment).toHaveBeenCalledTimes(1);
|
|
expect(ensureManagedSandboxEnvironment).toHaveBeenCalledWith({
|
|
name: "Daytona",
|
|
description: "Managed Daytona sandbox.",
|
|
provider: "daytona",
|
|
config: { target: "us" },
|
|
stockVersion: "2026.720.0",
|
|
});
|
|
// The frozen parsed config must not leak into the service (the row's
|
|
// config is mutated downstream when the provider key is forced in).
|
|
const passedConfig = ensureManagedSandboxEnvironment.mock.calls[0]?.[0]?.config;
|
|
expect(Object.isFrozen(passedConfig)).toBe(false);
|
|
});
|
|
|
|
it("waits for the bundled-plugin startup pass before ensuring anything", async () => {
|
|
const ensureManagedSandboxEnvironment = vi
|
|
.fn()
|
|
.mockResolvedValue(reconciliationResult());
|
|
const config = parsedConfig({
|
|
environments: [{ name: "Daytona", provider: "daytona" }],
|
|
});
|
|
|
|
let releasePlugins!: () => void;
|
|
const pluginsReady = new Promise<void>((resolve) => {
|
|
releasePlugins = resolve;
|
|
});
|
|
|
|
const pending = applyManagedEnvironments(noDb, config, {
|
|
env: {},
|
|
pluginsReady,
|
|
workerManager: runningWorkerManager(),
|
|
environments: environmentsSeam({ ensureManagedSandboxEnvironment }),
|
|
resolveSandboxProviderDriver: readyDriverResolver(),
|
|
});
|
|
|
|
// Give the ensure every chance to (incorrectly) run early.
|
|
await new Promise((resolve) => setImmediate(resolve));
|
|
expect(ensureManagedSandboxEnvironment).not.toHaveBeenCalled();
|
|
|
|
releasePlugins();
|
|
expect(await pending).toMatchObject({ ensured: 1, failed: 0, added: 1 });
|
|
expect(ensureManagedSandboxEnvironment).toHaveBeenCalledTimes(1);
|
|
});
|
|
|
|
it("reports operator-modified rows as skipped with a stock update still available", async () => {
|
|
const environments = environmentsSeam({
|
|
ensureManagedSandboxEnvironment: vi.fn().mockResolvedValue(reconciliationResult({
|
|
action: "skipped",
|
|
stockStatus: "operator_modified",
|
|
updateAvailable: true,
|
|
})),
|
|
});
|
|
const result = await applyManagedEnvironments(noDb, parsedConfig({
|
|
environments: [{ name: "Daytona", provider: "daytona" }],
|
|
}), {
|
|
env: {},
|
|
workerManager: runningWorkerManager(),
|
|
environments,
|
|
resolveSandboxProviderDriver: readyDriverResolver(),
|
|
});
|
|
|
|
expect(result).toMatchObject({
|
|
ensured: 0,
|
|
failed: 0,
|
|
added: 0,
|
|
updated: 0,
|
|
unchanged: 0,
|
|
skipped: 1,
|
|
outcomes: [{
|
|
action: "skipped",
|
|
stockStatus: "operator_modified",
|
|
updateAvailable: true,
|
|
}],
|
|
});
|
|
});
|
|
|
|
it("skips an entry whose provider plugin is missing and archives its stale row", async () => {
|
|
const environments = environmentsSeam({
|
|
archiveManagedSandboxEnvironment: vi.fn().mockResolvedValue(environmentRow({ status: "archived" })),
|
|
});
|
|
const config = parsedConfig({
|
|
environments: [{ name: "Daytona", provider: "daytona" }],
|
|
});
|
|
|
|
const result = await applyManagedEnvironments(noDb, config, {
|
|
env: {},
|
|
workerManager: runningWorkerManager(),
|
|
environments,
|
|
resolveSandboxProviderDriver: vi.fn(async () => null),
|
|
});
|
|
|
|
expect(result).toMatchObject({ ensured: 0, failed: 1 });
|
|
expect(environments.ensureManagedSandboxEnvironment).not.toHaveBeenCalled();
|
|
expect(environments.archiveManagedSandboxEnvironment).toHaveBeenCalledWith({
|
|
provider: "daytona",
|
|
});
|
|
});
|
|
|
|
it("skips (and counts failed) an entry whose provider plugin is not ready", async () => {
|
|
const environments = environmentsSeam();
|
|
const config = parsedConfig({
|
|
environments: [{ name: "Daytona", provider: "daytona" }],
|
|
});
|
|
|
|
const result = await applyManagedEnvironments(noDb, config, {
|
|
env: {},
|
|
workerManager: runningWorkerManager(),
|
|
environments,
|
|
resolveSandboxProviderDriver: readyDriverResolver("disabled"),
|
|
});
|
|
|
|
expect(result).toMatchObject({ ensured: 0, failed: 1 });
|
|
expect(environments.ensureManagedSandboxEnvironment).not.toHaveBeenCalled();
|
|
expect(environments.archiveManagedSandboxEnvironment).toHaveBeenCalledWith({
|
|
provider: "daytona",
|
|
});
|
|
});
|
|
|
|
it("skips an entry whose plugin record is ready but whose worker is not running", async () => {
|
|
const environments = environmentsSeam();
|
|
const config = parsedConfig({
|
|
environments: [{ name: "Daytona", provider: "daytona" }],
|
|
});
|
|
|
|
const workerManager = runningWorkerManager(false);
|
|
const result = await applyManagedEnvironments(noDb, config, {
|
|
env: {},
|
|
workerManager,
|
|
environments,
|
|
resolveSandboxProviderDriver: readyDriverResolver(),
|
|
});
|
|
|
|
expect(result).toMatchObject({ ensured: 0, failed: 1 });
|
|
expect(workerManager.isRunning).toHaveBeenCalledWith("plugin-1");
|
|
expect(environments.ensureManagedSandboxEnvironment).not.toHaveBeenCalled();
|
|
expect(environments.archiveManagedSandboxEnvironment).toHaveBeenCalledWith({
|
|
provider: "daytona",
|
|
});
|
|
// No registered handle → no respawn is coming; degraded until next boot.
|
|
expect(workerManager.getWorker).toHaveBeenCalledWith("plugin-1");
|
|
});
|
|
|
|
it("reactivates the archived environment once when the provider worker recovers", async () => {
|
|
const environments = environmentsSeam();
|
|
const config = parsedConfig({
|
|
environments: [
|
|
{ name: "Daytona", provider: "daytona", config: { target: "us" } },
|
|
],
|
|
});
|
|
|
|
const { handle, fireReady } = recoveryHandle();
|
|
const workerManager = {
|
|
isRunning: vi.fn(() => false),
|
|
getWorker: vi.fn(() => handle),
|
|
};
|
|
const result = await applyManagedEnvironments(noDb, config, {
|
|
env: {},
|
|
workerManager,
|
|
environments,
|
|
resolveSandboxProviderDriver: readyDriverResolver(),
|
|
});
|
|
|
|
// The boot pass itself stays degraded: archived, counted failed.
|
|
expect(result).toMatchObject({ ensured: 0, failed: 1 });
|
|
expect(environments.archiveManagedSandboxEnvironment).toHaveBeenCalledWith({
|
|
provider: "daytona",
|
|
});
|
|
expect(environments.ensureManagedSandboxEnvironment).not.toHaveBeenCalled();
|
|
|
|
// Worker manager respawns the worker → the handle re-emits `ready`.
|
|
fireReady();
|
|
fireReady();
|
|
await tick();
|
|
|
|
expect(environments.ensureManagedSandboxEnvironment).toHaveBeenCalledTimes(1);
|
|
expect(environments.ensureManagedSandboxEnvironment).toHaveBeenCalledWith({
|
|
name: "Daytona",
|
|
description: undefined,
|
|
provider: "daytona",
|
|
config: { target: "us" },
|
|
stockVersion: "2026.720.0",
|
|
});
|
|
expect(handle.off).toHaveBeenCalledTimes(1);
|
|
});
|
|
|
|
it("re-checks isRunning after subscribing so a recovery during the archive is not missed", async () => {
|
|
const environments = environmentsSeam();
|
|
const config = parsedConfig({
|
|
environments: [{ name: "Daytona", provider: "daytona" }],
|
|
});
|
|
|
|
const { handle } = recoveryHandle();
|
|
// Down at the gate check, already back up by the time the listener is
|
|
// registered — the `ready` event has been missed.
|
|
const workerManager = {
|
|
isRunning: vi
|
|
.fn()
|
|
.mockReturnValueOnce(false)
|
|
.mockReturnValue(true),
|
|
getWorker: vi.fn(() => handle),
|
|
};
|
|
const result = await applyManagedEnvironments(noDb, config, {
|
|
env: {},
|
|
workerManager,
|
|
environments,
|
|
resolveSandboxProviderDriver: readyDriverResolver(),
|
|
});
|
|
await tick();
|
|
|
|
expect(result).toMatchObject({ ensured: 0, failed: 1 });
|
|
expect(environments.ensureManagedSandboxEnvironment).toHaveBeenCalledTimes(1);
|
|
});
|
|
|
|
it("does not subscribe for recovery when the plugin record is missing or not ready", async () => {
|
|
const config = parsedConfig({
|
|
environments: [{ name: "Daytona", provider: "daytona" }],
|
|
});
|
|
|
|
for (const resolveSandboxProviderDriver of [
|
|
vi.fn(async () => null),
|
|
readyDriverResolver("disabled"),
|
|
]) {
|
|
const workerManager = runningWorkerManager(false);
|
|
await applyManagedEnvironments(noDb, config, {
|
|
env: {},
|
|
workerManager,
|
|
environments: environmentsSeam(),
|
|
resolveSandboxProviderDriver,
|
|
});
|
|
expect(workerManager.getWorker).not.toHaveBeenCalled();
|
|
}
|
|
});
|
|
|
|
it("logs, not throws, when the recovery re-ensure fails", async () => {
|
|
const environments = environmentsSeam({
|
|
ensureManagedSandboxEnvironment: vi.fn().mockRejectedValue(new Error("db exploded")),
|
|
});
|
|
const config = parsedConfig({
|
|
environments: [{ name: "Daytona", provider: "daytona" }],
|
|
});
|
|
|
|
const { handle, fireReady } = recoveryHandle();
|
|
const workerManager = {
|
|
isRunning: vi.fn(() => false),
|
|
getWorker: vi.fn(() => handle),
|
|
};
|
|
await applyManagedEnvironments(noDb, config, {
|
|
env: {},
|
|
workerManager,
|
|
environments,
|
|
resolveSandboxProviderDriver: readyDriverResolver(),
|
|
});
|
|
|
|
fireReady();
|
|
await tick();
|
|
// The rejection is swallowed into a log line; reaching this point without
|
|
// an unhandled rejection is the assertion.
|
|
expect(environments.ensureManagedSandboxEnvironment).toHaveBeenCalledTimes(1);
|
|
});
|
|
|
|
it("fails closed when no worker manager is provided", async () => {
|
|
const environments = environmentsSeam();
|
|
const config = parsedConfig({
|
|
environments: [{ name: "Daytona", provider: "daytona" }],
|
|
});
|
|
|
|
const result = await applyManagedEnvironments(noDb, config, {
|
|
env: {},
|
|
environments,
|
|
resolveSandboxProviderDriver: readyDriverResolver(),
|
|
});
|
|
|
|
expect(result).toMatchObject({ ensured: 0, failed: 1 });
|
|
expect(environments.ensureManagedSandboxEnvironment).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it("counts a skipped entry once even when archiving its stale row fails", async () => {
|
|
const environments = environmentsSeam({
|
|
archiveManagedSandboxEnvironment: vi.fn().mockRejectedValue(new Error("db exploded")),
|
|
});
|
|
const config = parsedConfig({
|
|
environments: [{ name: "Daytona", provider: "daytona" }],
|
|
});
|
|
|
|
const result = await applyManagedEnvironments(noDb, config, {
|
|
env: {},
|
|
workerManager: runningWorkerManager(),
|
|
environments,
|
|
resolveSandboxProviderDriver: vi.fn(async () => null),
|
|
});
|
|
|
|
expect(result).toMatchObject({ ensured: 0, failed: 1 });
|
|
expect(environments.archiveManagedSandboxEnvironment).toHaveBeenCalledTimes(1);
|
|
});
|
|
|
|
it("is fail-safe per entry: an ensure failure is counted, not thrown", async () => {
|
|
const environments = environmentsSeam({
|
|
ensureManagedSandboxEnvironment: vi.fn().mockRejectedValue(new Error("db exploded")),
|
|
});
|
|
const config = parsedConfig({
|
|
environments: [{ name: "Daytona", provider: "daytona" }],
|
|
});
|
|
|
|
const result = await applyManagedEnvironments(noDb, config, {
|
|
env: {},
|
|
workerManager: runningWorkerManager(),
|
|
environments,
|
|
resolveSandboxProviderDriver: readyDriverResolver(),
|
|
});
|
|
|
|
expect(result).toMatchObject({ ensured: 0, failed: 1 });
|
|
expect(environments.archiveManagedSandboxEnvironment).not.toHaveBeenCalled();
|
|
});
|
|
});
|