mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 23:43:16 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its web server also owns controller admission and user commands. > - A rolling deployment starts a replacement before the old server exits. > - Shared database state alone does not preserve local agent files, browser chunks, or in-process commands. > - This pull request adds an opt-in ephemeral server contract and a local overlap test. > - The benefit is a web server that can relinquish traffic and local files without losing acknowledged work. ## Linked Issues or Issue Description Refs #18403, #18327, #18335, #18341. **Current behavior** The control-plane web server still has local agent-file authority and local command targets. A new container can miss saved bytes or send a command to the wrong process. An existing browser tab also needs chunks from its original build. **Proposed behavior** Eligible servers share only PostgreSQL, object storage and stable secrets. A replacement waits for readiness. The old server drains admitted requests, keeps owner commands available through handoff, and releases controller leases. Explicit maintenance adopts existing files before the volume is removed. **Reason and benefit** This is the application prerequisite for rolling app-only deployments. Schema migrations and permanent multi-replica operation are outside this change. ## What Changed - Store agent-file manifests in PostgreSQL and immutable bytes in S3. Treat local roots as disposable projections. Preserve existing permission checks and edit conflicts. - Add an explicit migration, eligibility and object-retention command to the production runtime bundle. - Add `/api/ready`, admitted-request drain, browser restart signaling and retained immutable UI chunks. - Carry transaction-aware work hints across controllers during handoff and active execution. Keep listeners closed when idle. - Deliver durable Stop and runtime responses to the execution owner. Recheck request lifecycle and access before response dispatch. - Reject local-only configuration in ephemeral mode. Keep the feature disabled for existing installations by default. - Add an isolated PostgreSQL/S3/proxy/browser deployment harness with independent server homes and retained failure evidence. ## Verification - All CI checks pass on `ce17b95019d3cf1cabd357d19b6d37f96782cc5c`: workspace typecheck, build, Runner checks, server/general tests and browser shards. Greptile reviewed the runtime changes at 5/5 with all findings resolved; the final commit updates maps and the plan. - Local targeted checks passed: coordinator 27; Stop/recovery 33; PostgreSQL command delivery 6; route/workspace regressions 8; bootstrap Stop 1; restart recovery 75. PostgreSQL command tests simulate the provider edge. - Local browser proof passed 91 reads and 19 saves; reconnect took 1,013ms. The tab opened Audit from S3 after deletion of A's whole home/build. This uses compiled workspace code and local-trusted auth, not released containers or live agents. - Local workspace typecheck/build passed; current server and runtime bundle passed. The earlier broad local suite was stopped after three hours and is not a passing receipt. Its retained failures and isolated reruns are documented in the plan. - Stacked #18526 adds stricter graceful-exit proof, an unready replacement control, live Runner qualification and a Stop handoff correction. Hosted enablement remains gated on those results and Cloud #1096. No hosted stack has been enabled. ## Risks - Apply migration 0422 before enabling the feature. Existing data must pass each owner's migration and eligibility checks. Do not discard a volume based on a single migration page. - Durable manifests remain authoritative after the opt-in is disabled. Rolling back to a binary that ignores them is unsupported. - Cold directory restoration and complete shutdown timing still need qualification before enabling a hosted stack. - Retained UI assets must cover the supported browser and rollback window. Cleanup is explicit; no new idle maintenance polling is introduced. - This change does not claim production zero downtime or authorize a fleet-wide enablement. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window limit are not exposed in this session. Used reasoning, code editing, shell execution and browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
113 lines
4.3 KiB
JavaScript
113 lines
4.3 KiB
JavaScript
|
|
import { readFile, rm } from "node:fs/promises";
|
|
import { dirname, join } from "node:path";
|
|
import { fileURLToPath, pathToFileURL } from "node:url";
|
|
import { build } from "esbuild";
|
|
|
|
const scriptDir = dirname(fileURLToPath(import.meta.url));
|
|
const serverDir = join(scriptDir, "..");
|
|
|
|
const REQUIRE_SHIM = "import { createRequire as __paperclipCreateRequire } from 'node:module'; const require = __paperclipCreateRequire(import.meta.url);";
|
|
|
|
export function preserveModuleLocation(source, file) {
|
|
return source
|
|
.replace(/\bimport\.meta\.url\b/g, JSON.stringify(pathToFileURL(file).href))
|
|
.replace(/\bimport\.meta\.dirname\b/g, JSON.stringify(dirname(file)))
|
|
.replace(/\bimport\.meta\.filename\b/g, JSON.stringify(file));
|
|
}
|
|
|
|
function runtimeEntry(entry) {
|
|
return {
|
|
name: "runtime-entry",
|
|
setup(builder) {
|
|
builder.onResolve({ filter: /^paperclip-runtime-entry$/ }, () => ({ path: "paperclip-runtime-entry", namespace: "runtime-entry" }));
|
|
builder.onLoad({ filter: /.*/, namespace: "runtime-entry" }, () => ({
|
|
contents: `process.argv[1] = ${JSON.stringify(entry)};\nawait import(${JSON.stringify(entry)});\n`,
|
|
loader: "js",
|
|
resolveDir: dirname(entry),
|
|
}));
|
|
},
|
|
};
|
|
}
|
|
|
|
const keepModuleLocations = {
|
|
name: "keep-module-locations",
|
|
setup(builder) {
|
|
builder.onLoad({ filter: /\.(?:[cm]?js|[cm]?ts)$/ }, async (args) => {
|
|
if (args.path.includes("/node_modules/")) return undefined;
|
|
const source = await readFile(args.path, "utf8");
|
|
if (!source.includes("import.meta")) return undefined;
|
|
return {
|
|
contents: preserveModuleLocation(source, args.path),
|
|
loader: /\.[cm]?ts$/.test(args.path) ? "ts" : "js",
|
|
resolveDir: dirname(args.path),
|
|
};
|
|
});
|
|
},
|
|
};
|
|
|
|
function externalizeThirdParty(isBundled, runtimeDir) {
|
|
return {
|
|
name: "externalize-third-party",
|
|
setup(builder) {
|
|
builder.onResolve({ filter: /^[^./]/ }, async (args) => {
|
|
if (args.pluginData?.externalizeThirdParty || isBundled(args.path)) return undefined;
|
|
const resolved = await builder.resolve(args.path, {
|
|
kind: args.kind,
|
|
resolveDir: args.resolveDir,
|
|
importer: args.importer,
|
|
pluginData: { externalizeThirdParty: true },
|
|
});
|
|
if (resolved.errors.length > 0 || resolved.external) return { path: args.path, external: true };
|
|
const fromServer = await builder.resolve(args.path, {
|
|
kind: args.kind,
|
|
resolveDir: runtimeDir,
|
|
pluginData: { externalizeThirdParty: true },
|
|
});
|
|
if (fromServer.errors.length === 0 && fromServer.path === resolved.path) return { path: args.path, external: true };
|
|
return { path: resolved.path, external: true };
|
|
});
|
|
},
|
|
};
|
|
}
|
|
|
|
export async function bundleRuntime({ entry, outdir, maintenanceEntry, isBundled = (specifier) => specifier.startsWith("@paperclipai/") }) {
|
|
await rm(outdir, { recursive: true, force: true });
|
|
const result = await build({
|
|
entryPoints: { index: "paperclip-runtime-entry", server: entry, ...(maintenanceEntry ? { "agent-files": maintenanceEntry } : {}) },
|
|
outdir,
|
|
outExtension: { ".js": ".mjs" },
|
|
chunkNames: "chunk-[hash]",
|
|
bundle: true,
|
|
splitting: true,
|
|
format: "esm",
|
|
platform: "node",
|
|
target: "node24",
|
|
mainFields: ["main"],
|
|
conditions: ["node"],
|
|
banner: { js: REQUIRE_SHIM },
|
|
sourcemap: "linked",
|
|
metafile: true,
|
|
logLevel: "error",
|
|
plugins: [runtimeEntry(entry), keepModuleLocations, externalizeThirdParty(isBundled, dirname(entry))],
|
|
});
|
|
const outputs = Object.keys(result.metafile.outputs).filter((file) => file.endsWith(".mjs"));
|
|
return {
|
|
outfile: join(outdir, "index.mjs"),
|
|
serverModule: join(outdir, "server.mjs"),
|
|
inputs: Object.keys(result.metafile.inputs).length,
|
|
chunks: outputs.length,
|
|
};
|
|
}
|
|
|
|
async function main() {
|
|
const started = performance.now();
|
|
const summary = await bundleRuntime({ entry: join(serverDir, "dist", "index.js"), outdir: join(serverDir, "dist-runtime"),
|
|
maintenanceEntry: join(serverDir, "dist", "maintenance", "agent-files.js") });
|
|
console.log(`[bundle-runtime] ${summary.inputs} inputs -> ${summary.chunks} chunks in ${Math.round(performance.now() - started)} ms: ${summary.outfile}`);
|
|
}
|
|
|
|
if (process.argv[1] && pathToFileURL(process.argv[1]).href === import.meta.url) {
|
|
await main();
|
|
}
|