mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-03 09:37:19 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Its web server also owns controller admission and user commands. > - A rolling deployment starts a replacement before the old server exits. > - Shared database state alone does not preserve local agent files, browser chunks, or in-process commands. > - This pull request adds an opt-in ephemeral server contract and a local overlap test. > - The benefit is a web server that can relinquish traffic and local files without losing acknowledged work. ## Linked Issues or Issue Description Refs #18403, #18327, #18335, #18341. **Current behavior** The control-plane web server still has local agent-file authority and local command targets. A new container can miss saved bytes or send a command to the wrong process. An existing browser tab also needs chunks from its original build. **Proposed behavior** Eligible servers share only PostgreSQL, object storage and stable secrets. A replacement waits for readiness. The old server drains admitted requests, keeps owner commands available through handoff, and releases controller leases. Explicit maintenance adopts existing files before the volume is removed. **Reason and benefit** This is the application prerequisite for rolling app-only deployments. Schema migrations and permanent multi-replica operation are outside this change. ## What Changed - Store agent-file manifests in PostgreSQL and immutable bytes in S3. Treat local roots as disposable projections. Preserve existing permission checks and edit conflicts. - Add an explicit migration, eligibility and object-retention command to the production runtime bundle. - Add `/api/ready`, admitted-request drain, browser restart signaling and retained immutable UI chunks. - Carry transaction-aware work hints across controllers during handoff and active execution. Keep listeners closed when idle. - Deliver durable Stop and runtime responses to the execution owner. Recheck request lifecycle and access before response dispatch. - Reject local-only configuration in ephemeral mode. Keep the feature disabled for existing installations by default. - Add an isolated PostgreSQL/S3/proxy/browser deployment harness with independent server homes and retained failure evidence. ## Verification - All CI checks pass on `ce17b95019d3cf1cabd357d19b6d37f96782cc5c`: workspace typecheck, build, Runner checks, server/general tests and browser shards. Greptile reviewed the runtime changes at 5/5 with all findings resolved; the final commit updates maps and the plan. - Local targeted checks passed: coordinator 27; Stop/recovery 33; PostgreSQL command delivery 6; route/workspace regressions 8; bootstrap Stop 1; restart recovery 75. PostgreSQL command tests simulate the provider edge. - Local browser proof passed 91 reads and 19 saves; reconnect took 1,013ms. The tab opened Audit from S3 after deletion of A's whole home/build. This uses compiled workspace code and local-trusted auth, not released containers or live agents. - Local workspace typecheck/build passed; current server and runtime bundle passed. The earlier broad local suite was stopped after three hours and is not a passing receipt. Its retained failures and isolated reruns are documented in the plan. - Stacked #18526 adds stricter graceful-exit proof, an unready replacement control, live Runner qualification and a Stop handoff correction. Hosted enablement remains gated on those results and Cloud #1096. No hosted stack has been enabled. ## Risks - Apply migration 0422 before enabling the feature. Existing data must pass each owner's migration and eligibility checks. Do not discard a volume based on a single migration page. - Durable manifests remain authoritative after the opt-in is disabled. Rolling back to a binary that ignores them is unsupported. - Cold directory restoration and complete shutdown timing still need qualification before enabling a hosted stack. - Retained UI assets must cover the supported browser and rollback window. Cleanup is explicit; no new idle maintenance polling is introduced. - This change does not claim production zero downtime or authorize a fleet-wide enablement. ## Model Used OpenAI Codex, GPT-6 family. The exact serving model ID and context-window limit are not exposed in this session. Used reasoning, code editing, shell execution and browser automation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
77 lines
5.0 KiB
TypeScript
77 lines
5.0 KiB
TypeScript
import { and, eq, isNotNull, isNull, sql } from "drizzle-orm";
|
|
import { agentFileRoots, agents, executionWorkspaces, heartbeatRuns, providerTraceRecords, type Db } from "@paperclipai/db";
|
|
import type { Config } from "../config.js";
|
|
import { isValidLocalEncryptedMasterKey } from "../secrets/local-encrypted-provider.js";
|
|
import { agentInstructionsBundleMode } from "./agent-instructions.js";
|
|
import { skillInstallationMaintenance } from "./skill-installation-maintenance.js";
|
|
|
|
export const ephemeralServerEnabled = () => process.env.PAPERCLIP_EPHEMERAL_SERVER === "true";
|
|
|
|
/** Validate before consumers can generate a replacement local identity. Error
|
|
* messages name settings only, never credential values or connection URLs. */
|
|
export function assertEphemeralServerConfig(config: Pick<Config, "databaseUrl" | "storageProvider">, env = process.env): string | null {
|
|
if (env.PAPERCLIP_EPHEMERAL_SERVER !== "true") return null;
|
|
const refuse = (reason: string): never => { throw new Error(`Volume-free server configuration: ${reason}`); };
|
|
if (config.storageProvider !== "s3" || env.PAPERCLIP_DURABLE_AGENT_FILES !== "true" || env.PAPERCLIP_SHARED_UI_ASSETS !== "true") {
|
|
refuse("shared S3 storage, durable agent files and shared UI assets are required");
|
|
}
|
|
if (!env.PAPERCLIP_SECRETS_MASTER_KEY || !isValidLocalEncryptedMasterKey(env.PAPERCLIP_SECRETS_MASTER_KEY)) {
|
|
refuse("a stable PAPERCLIP_SECRETS_MASTER_KEY is required");
|
|
}
|
|
if ((env.PAPERCLIP_DECISION_SIGNING_SECRET?.trim().length ?? 0) < 32) {
|
|
refuse("a stable PAPERCLIP_DECISION_SIGNING_SECRET of at least 32 characters is required; preserve the existing decision key during migration");
|
|
}
|
|
let runtime: URL, listener: URL;
|
|
try { runtime = new URL(config.databaseUrl ?? ""); listener = new URL(env.PAPERCLIP_DATABASE_LISTEN_URL ?? ""); }
|
|
catch { return refuse("external DATABASE_URL and PAPERCLIP_DATABASE_LISTEN_URL are required"); }
|
|
if (![runtime, listener].every(url => ["postgres:", "postgresql:"].includes(url.protocol))
|
|
|| runtime.username !== listener.username || runtime.password !== listener.password || runtime.pathname !== listener.pathname
|
|
|| runtime.hostname.replace(/-pooler(?=\.)/, "") !== listener.hostname
|
|
|| /pooler|pgbouncer/i.test(listener.hostname)) {
|
|
refuse("PAPERCLIP_DATABASE_LISTEN_URL must be a direct connection to the same database using the runtime role");
|
|
}
|
|
return listener.href;
|
|
}
|
|
|
|
/** A readiness gate, not a periodic scan or an implicit migration. Existing
|
|
* volume data is adopted by the owning maintenance commands before removal. */
|
|
export async function assertEphemeralServerData(db: Db) {
|
|
if (!ephemeralServerEnabled()) return;
|
|
const { blockers } = await readEphemeralServerDataEligibility(db);
|
|
if (blockers.length) throw new Error(`Volume-free server data is not eligible: ${blockers.join(", ")}`);
|
|
}
|
|
|
|
export async function readEphemeralServerDataEligibility(db: Db) {
|
|
const blockers: string[] = [];
|
|
const [missingFiles] = await db.select({ id: agents.id }).from(agents).leftJoin(agentFileRoots,
|
|
and(eq(agentFileRoots.companyId, agents.companyId), eq(agentFileRoots.agentId, agents.id)))
|
|
.where(isNull(agentFileRoots.agentId)).limit(1);
|
|
if (missingFiles) blockers.push("agent_files_require_migration");
|
|
// Only inspect configuration here; filesystem reads must never adopt an
|
|
// empty directory on the replacement machine as the old server's content.
|
|
const configured = await db.select({ id: agents.id, companyId: agents.companyId, name: agents.name, adapterConfig: agents.adapterConfig }).from(agents);
|
|
if (configured.some(agent => agentInstructionsBundleMode(agent) === "external")) blockers.push("external_agent_instructions");
|
|
const [localWorkspace] = await db.select({ id: executionWorkspaces.id }).from(executionWorkspaces).where(and(
|
|
isNotNull(executionWorkspaces.cwd), sql`${executionWorkspaces.providerType} <> 'plugin'`,
|
|
sql`coalesce(${executionWorkspaces.metadata}->'remoteWorkspace'->>'schema', '') <> 'paperclip.remote-workspace.v1'`,
|
|
)).limit(1);
|
|
if (localWorkspace) blockers.push("workspaces_require_migration");
|
|
const [localLog] = await db.select({ id: heartbeatRuns.id }).from(heartbeatRuns).where(and(
|
|
eq(heartbeatRuns.logStore, "local_file"), isNotNull(heartbeatRuns.logRef),
|
|
)).limit(1);
|
|
if (localLog) blockers.push("run_logs_require_migration");
|
|
const [localTrace] = await db.select({ id: providerTraceRecords.id }).from(providerTraceRecords).where(and(
|
|
isNull(providerTraceRecords.deletedAt), sql`${providerTraceRecords.traceRef} not like 's3-segments-v1:%'`,
|
|
)).limit(1);
|
|
if (localTrace) blockers.push("provider_traces_require_migration");
|
|
const skills = skillInstallationMaintenance(db);
|
|
let after: string | undefined;
|
|
for (;;) {
|
|
const page = await skills.eligibility({ after, limit: 50 });
|
|
if (page.results.some(skill => skill.status !== "version_backed")) { blockers.push("skills_require_migration"); break; }
|
|
if (page.inspected < 50 || !page.after) break;
|
|
after = page.after;
|
|
}
|
|
return { eligible: blockers.length === 0, blockers };
|
|
}
|