Files
PaperClipAI/server/scripts/bundle-runtime.test.mjs
T
DottaandPaperclip d941ed3879 feat(server): verify existing-volume ephemeral enrollment (#18529)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Cloud web servers can now keep durable state in PostgreSQL and S3.
> - Existing servers still have files and a decision signing key on
their volume.
> - The volume must stay attached until every data owner verifies its
migration.
> - This pull request adds a dedicated maintenance process for that
transition.
> - Cloud can preserve the original key and remove the volume only after
a matching receipt.

## Linked Issues or Issue Description

Refs #18518 and #18526. Existing-volume enrollment is a required step in
the zero-downtime web deployment work. This is the Core maintenance
side; the Cloud transition is a separate private change.

## What Changed

- Package a standalone maintenance entry that starts no normal app
routes or execution workers.
- Coordinate the existing agent-file, run-evidence and skill migration
owners. Follow every page and verify shared bytes before final
eligibility.
- Require stopped writers. Refuse active execution, missing local bytes,
local workspaces and external instructions.
- Read the existing decision key without generating a replacement.
Preserve normal key creation for ordinary server startup.
- Require an attempt token for status and export. Export the original
key only after successful migration. Keep status and errors free of
secret material.
- Document the Cloud ownership boundary, deadline, retry and retention
requirements.

## Verification

- Full workspace typecheck and build passed before the review fixes. The
final server TypeScript compilation passed after correcting Drizzle's
optional predicate type.
- Production runtime bundling passed: 2,702 inputs, 122 chunks,
including the maintenance executable.
- The focused suite passed 21 cases using PostgreSQL and real
S3-compatible storage. It covers multi-page migration of 26 agents,
exact reads after deleting the predecessor home, corrupt-object
rejection, active/missing-data refusal, and the bundled plain-Node
process with authenticated export and clean SIGTERM exit.
- Regression cases refuse unsaved terminal working copies, require
explicit stopped/cleaned receipts for unavailable copies, clean 101
expired traces through the owning service, and enforce the deadline
after migration succeeds.
- Initial integration attempts exposed fixture cleanup ordering
problems; corrected runs pass. CI also caught a Drizzle predicate type
error, now fixed and compiled locally.
- All repository CI gates passed on a4db955403, including typecheck,
build, browser suites and serialized tests. Greptile reviewed that head
at 5/5; all four review threads are resolved. Hosted enrollment and
active-agent continuity remain separate acceptance work.

## Risks

- This is an explicit maintenance entry. The caller must confirm all
prior deployments stopped and keep normal tenant routing closed through
detachment.
- A timeout blocks export but an owner may finish its current bounded
operation before observing cancellation. Cloud must confirm process exit
before detaching. SIGTERM has a ten-second hard exit deadline.
- The authenticated export contains the original signing key. The Cloud
caller must put it only in the stack vault and must not log the body.
- Local-workspace and external-instruction migrations remain their
existing owners' responsibility. Enrollment fails closed until those
data are eligible.
- No database schema changes. Ordinary server startup is unchanged
unless this maintenance entry is explicitly launched.

## Model Used

OpenAI Codex, GPT-6. Tool use, code execution and repository review. The
runtime does not expose a more specific model ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-01 03:56:44 -06:00

169 lines
10 KiB
JavaScript

import { execFile } from "node:child_process";
import { mkdirSync, mkdtempSync, realpathSync, rmSync, symlinkSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import path from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
import { promisify } from "node:util";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { bundleRuntime, preserveModuleLocation } from "./bundle-runtime.mjs";
const execFileAsync = promisify(execFile);
function write(file, content) {
mkdirSync(path.dirname(file), { recursive: true });
writeFileSync(file, content);
}
describe("bundleRuntime", () => {
let root;
beforeEach(() => {
root = realpathSync(mkdtempSync(path.join(tmpdir(), "paperclip-bundle-runtime-")));
const pkg = path.join(root, "packages", "fixture");
write(path.join(pkg, "package.json"), JSON.stringify({ name: "@paperclipai/fixture", type: "module", exports: { ".": "./src/index.ts" } }));
write(path.join(pkg, "src", "asset.txt"), "asset beside its source\n");
write(path.join(pkg, "src", "helper.ts"), "export const double = (value: number): number => value * 2;\n");
write(path.join(pkg, "src", "index.ts"), [
"import { readFileSync } from \"node:fs\";",
"import { double } from \"./helper.js\";",
"import { onlyForPackage } from \"package-private-dep\";",
"import { nodeBuild } from \"dual-build-dep\";",
"export const asset = readFileSync(new URL(\"./asset.txt\", import.meta.url), \"utf8\").trim();",
"export const location = import.meta.dirname;",
"export const values = { doubled: double(21), onlyForPackage, nodeBuild };",
"",
].join("\n"));
write(path.join(pkg, "node_modules", "package-private-dep", "package.json"), JSON.stringify({ name: "package-private-dep", type: "module", main: "index.js" }));
write(path.join(pkg, "node_modules", "package-private-dep", "index.js"), "export const onlyForPackage = \"resolved from the package\";\n");
write(path.join(pkg, "node_modules", "dual-build-dep", "package.json"), JSON.stringify({
name: "dual-build-dep", exports: { ".": { module: "./bundler/index.js", default: "./node.cjs" } },
}));
write(path.join(pkg, "node_modules", "dual-build-dep", "node.cjs"), "exports.nodeBuild = \"node build\";\n");
write(path.join(pkg, "node_modules", "dual-build-dep", "bundler", "index.js"), "export * from \"./extensionless\";\n");
const server = path.join(root, "server");
mkdirSync(path.join(server, "node_modules", "@paperclipai"), { recursive: true });
symlinkSync(pkg, path.join(server, "node_modules", "@paperclipai", "fixture"), "dir");
write(path.join(server, "package.json"), JSON.stringify({ name: "server", type: "module" }));
for (const [version, flavor] of [["1", "installed at build"], ["2", "replaced after build"]]) {
const store = path.join(root, "node_modules", ".pnpm", `server-dep@${version}`);
write(path.join(store, "package.json"), JSON.stringify({ name: "server-dep", type: "module", main: "index.js" }));
write(path.join(store, "index.js"), `export const flavor = ${JSON.stringify(flavor)};\n`);
}
symlinkSync(path.join(root, "node_modules", ".pnpm", "server-dep@1"), path.join(server, "node_modules", "server-dep"), "dir");
write(path.join(server, "dist", "late.js"), "import { values } from \"@paperclipai/fixture\";\nexport const late = () => values.doubled + 1;\n");
write(path.join(server, "dist", "index.js"), [
"import { asset, location, values } from \"@paperclipai/fixture\";",
"import { flavor } from \"server-dep\";",
"import { pathToFileURL } from \"node:url\";",
"const { late } = await import(\"./late.js\");",
"// The server only starts when it is the main module.",
"if (pathToFileURL(process.argv[1]).href === import.meta.url) {",
" console.log(JSON.stringify({ asset, location, values, flavor, late: late(), entryDir: import.meta.dirname }));",
"}",
"",
].join("\n"));
});
afterEach(() => {
rmSync(root, { recursive: true, force: true });
});
it("packages a maintenance entry without starting the application or needing a TypeScript loader", async () => {
const maintenanceEntry = path.join(root, "server", "dist", "maintenance.js");
write(maintenanceEntry, 'import { asset } from "@paperclipai/fixture"; console.log(JSON.stringify({ asset, phase: process.argv.at(-1) }));');
const outdir = path.join(root, "server", "dist-runtime");
await bundleRuntime({ entry: path.join(root, "server", "dist", "index.js"), outdir, maintenanceEntry });
const { stdout } = await execFileAsync(process.execPath, [path.join(outdir, "agent-files.mjs"), "--phase=eligibility"], { timeout: 30_000 });
expect(JSON.parse(stdout)).toEqual({ asset: "asset beside its source", phase: "--phase=eligibility" });
});
it("packages standalone enrollment without starting the app", async () => {
const ephemeralMigrationEntry = path.join(root, "server", "dist", "migration.js");
write(ephemeralMigrationEntry, 'import { values } from "@paperclipai/fixture"; console.log(JSON.stringify({ migrated: values.doubled }));');
const outdir = path.join(root, "server", "dist-runtime");
await bundleRuntime({ entry: path.join(root, "server", "dist", "index.js"), outdir, ephemeralMigrationEntry });
const { stdout } = await execFileAsync(process.execPath, [path.join(outdir, "ephemeral-migration.mjs")], { timeout: 30_000 });
expect(JSON.parse(stdout)).toEqual({ migrated: 42 });
});
it("bundles workspace TypeScript and runs with plain node", async () => {
const outdir = path.join(root, "server", "dist-runtime");
const summary = await bundleRuntime({ entry: path.join(root, "server", "dist", "index.js"), outdir });
expect(summary.outfile).toBe(path.join(outdir, "index.mjs"));
const { stdout } = await execFileAsync(process.execPath, [summary.outfile], { cwd: path.join(root, "server"), timeout: 30_000 });
const output = JSON.parse(stdout);
expect(output.values).toEqual({ doubled: 42, onlyForPackage: "resolved from the package", nodeBuild: "node build" });
expect(output.late).toBe(43);
expect(output.asset).toBe("asset beside its source");
expect(output.location).toBe(path.join(root, "packages", "fixture", "src"));
expect(output.entryDir).toBe(path.join(root, "server", "dist"));
});
it("keeps database caller locations public in a source-mapped runtime bundle", async () => {
const captureModule = fileURLToPath(new URL("../../packages/db/src/response-capture.ts", import.meta.url));
write(path.join(root, "server", "dist", "services", "response-caller.js"), [
`import { publicDatabaseCallers } from ${JSON.stringify(captureModule)};`,
'export function callers() { return publicDatabaseCallers(new Error("private-capture-message").stack); }',
].join("\n"));
write(path.join(root, "server", "dist", "index.js"), [
'import { callers } from "./services/response-caller.js";',
'console.log(JSON.stringify({ callers: callers() }));',
].join("\n"));
const summary = await bundleRuntime({ entry: path.join(root, "server", "dist", "index.js"), outdir: path.join(root, "server", "dist-runtime") });
const env = { ...process.env };
delete env.NODE_OPTIONS;
delete env.PAPERCLIP_DB_RESPONSE_CAPTURE_WINDOW;
const options = { cwd: path.join(root, "server"), env, timeout: 30_000 };
const plain = JSON.parse((await execFileAsync(process.execPath, [summary.outfile], options)).stdout);
const mapped = JSON.parse((await execFileAsync(process.execPath, ["--enable-source-maps", summary.outfile], options)).stdout);
expect(plain.callers).toEqual([]);
expect(mapped.callers).toEqual([expect.stringMatching(/^services\/response-caller\.js:\d+:\d+$/)]);
for (const result of [plain, mapped]) {
const output = JSON.stringify(result);
expect(output).not.toContain(root);
expect(output).not.toContain(captureModule);
expect(output).not.toContain("private-capture-message");
}
});
it("emits a server module that loads without starting the server", async () => {
const outdir = path.join(root, "server", "dist-runtime");
const summary = await bundleRuntime({ entry: path.join(root, "server", "dist", "index.js"), outdir });
expect(summary.serverModule).toBe(path.join(outdir, "server.mjs"));
const probe = path.join(root, "server", "probe.mjs");
write(probe, `await import(${JSON.stringify(pathToFileURL(summary.serverModule).href)});\nconsole.log("loaded");\n`);
const loaded = await execFileAsync(process.execPath, [probe], { cwd: path.join(root, "server"), timeout: 30_000 });
expect(loaded.stdout.trim()).toBe("loaded");
const started = await execFileAsync(process.execPath, [summary.outfile], { cwd: path.join(root, "server"), timeout: 30_000 });
expect(JSON.parse(started.stdout).late).toBe(43);
});
it("leaves the server's own dependencies to runtime resolution", async () => {
const outdir = path.join(root, "server", "dist-runtime");
const summary = await bundleRuntime({ entry: path.join(root, "server", "dist", "index.js"), outdir });
rmSync(path.join(root, "server", "node_modules", "server-dep"));
symlinkSync(path.join(root, "node_modules", ".pnpm", "server-dep@2"), path.join(root, "server", "node_modules", "server-dep"), "dir");
const { stdout } = await execFileAsync(process.execPath, [summary.outfile], { cwd: path.join(root, "server"), timeout: 30_000 });
expect(JSON.parse(stdout).flavor).toBe("replaced after build");
});
it("keeps third-party packages external", async () => {
const outdir = path.join(root, "server", "dist-runtime");
await bundleRuntime({ entry: path.join(root, "server", "dist", "index.js"), outdir });
const { stdout } = await execFileAsync("grep", ["-rl", "resolved from the package", outdir], { encoding: "utf8" }).catch(() => ({ stdout: "" }));
expect(stdout.trim()).toBe("");
});
});
describe("preserveModuleLocation", () => {
it("replaces import.meta location properties with the original path", () => {
const file = "/srv/app/packages/x/src/mod.ts";
const out = preserveModuleLocation("a(import.meta.url); b(import.meta.dirname); c(import.meta.filename); d(import.meta.resolve);", file);
expect(out).toBe(`a("file:///srv/app/packages/x/src/mod.ts"); b("/srv/app/packages/x/src"); c("/srv/app/packages/x/src/mod.ts"); d(import.meta.resolve);`);
});
});