mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-13 15:55:46 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The server reads its configuration through `loadConfig()` in `server/src/config.ts`. Runtime code calls it often: `getStorageService()` calls it on each use, and the native runtime calls it for backup storage, response delivery and attachment reuse > - `loadConfig()` called `detectTailnetBindHost()` on each call. That function runs `tailscale ip -4` with `execFileSync`, which is a synchronous child process. The event loop stops until the child process ends > - A CPU profile of a busy hosted server showed this cost on the main thread, on a server that does not bind to a tailnet > - The tailnet address is an input only to the bind host, and only the listener that the server binds at startup uses the bind host. Most bind configurations never read the address > - This pull request makes `loadConfig()` detect the address only when the bind configuration reads it, and keeps the result for the life of the process > - The benefit is that a `loadConfig()` call no longer starts a child process on runtime paths, so the event loop stays free for requests ## Linked Issues or Issue Description No public issue exists. I searched open and closed pull requests for `tailscale loadConfig` and `detectTailnetBindHost` and found no related work. **What happened?** Each `loadConfig()` call started `tailscale ip -4` as a synchronous child process. A 30 second CPU profile of the main thread of a hosted server (commit556b249897, Node 24, main thread at 96% of one core) showed `detectTailnetBindHost` ← `loadConfig` at 3.1% inclusive and the native child-process spawn at 3.6% self. The callers in the profile were `getStorageService`, `durableNativeHarnessStorage`, `restoreDurableHarnessForExecution` and `createRunnerdBackendWithinSessionClaim`. The server process was about 4.4 GB resident, which makes each spawn more expensive. The cost is the same when the `tailscale` command is not installed, because the spawn still runs. **Expected behavior** `loadConfig()` starts `tailscale` only when the bind configuration reads a tailnet address: `server.bind=tailnet`, or a host without a bind mode that is not loopback and not all interfaces. It starts it at most one time per process. A change to `PAPERCLIP_TAILNET_BIND_HOST`, to another environment variable or to the config file is still visible to the next `loadConfig()` call. **Steps to reproduce** 1. Run `pnpm --filter @paperclipai/server exec tsx scripts/benchmark-config-load.ts --calls=200` on the parent commit. The benchmark puts a counting `tailscale` stand-in first on `PATH`, so it does not need Tailscale. 2. Read the `tailscale execs` column. Each scenario shows 200 execs for 200 calls. **Paperclip version or commit** `master` ate2cf9a670f. The profile is from556b249897. **Deployment mode** All modes. The profile is from an `authenticated` hosted server. The benchmark covers `bind=lan`, the default loopback bind and `bind=tailnet`. **Node.js version** Profile: Node 24. Local benchmark: Node 26.4.0 on darwin arm64. ## What Changed - `server/src/config.ts`: `loadConfig()` calls `detectTailnetBindHost()` only when the bind configuration reads the address. With a configured bind mode, that is `tailnet` only. Without one, it is a host that `inferBindModeFromHost` cannot resolve to loopback or all interfaces without the address. - `server/src/config.ts`: the result of `tailscale ip -4` is kept for the life of the process, also when the command fails. `PAPERCLIP_TAILNET_BIND_HOST` is not kept: each call reads it first, as before. - `server/src/__tests__/config-tailnet-bind-detection.test.ts` (new): regression tests that count `tailscale` starts. - `server/scripts/benchmark-config-load.ts` (new): the reproduction and benchmark. - `doc/DEVELOPING.md`: removed the advice to set `PAPERCLIP_TAILNET_BIND_HOST` for local fixtures to prevent repeated discovery. The server no longer repeats discovery. - `cli/src/config/server-bind.ts` has its own copy of the detection. It is not changed: it detects only for `bind=tailnet` and runs in short CLI commands, not on a server runtime path. ## Verification Regression test, red then green: - `cd server && pnpm exec vitest run src/__tests__/config-tailnet-bind-detection.test.ts` - Parent commit: 11 of 11 tests fail. Examples: `expected 1 to be +0` for `bind=lan` (the command ran when it must not), `expected 3 to be 1` for `bind=tailnet` (the command ran on each call). - This branch: 11 of 11 tests pass. - The tests assert the number of `tailscale` starts and the resolved bind host. They do not assert time. They cover `bind=loopback`, `lan`, `custom`, loopback and all-interfaces hosts without a bind mode, repeated `getStorageService()`, `bind=tailnet`, bind inference from a host, `PAPERCLIP_TAILNET_BIND_HOST` (used without the command, each change visible), a bind mode change in the environment and in the config file, and the unchanged refusal when no tailnet address exists. Benchmark, same machine, 200 calls per scenario, 3 runs each (ranges shown): | Scenario | Before: `tailscale` execs | Before: ms per call | After: `tailscale` execs | After: ms per call | | --- | --- | --- | --- | --- | | `loadConfig()`, `bind=lan` | 200 | 4.5 to 6.0 | 0 | 0.063 to 0.069 | | `getStorageService()`, `bind=lan` | 200 | 4.0 to 4.5 | 0 | 0.043 to 0.060 | | `loadConfig()`, default loopback | 200 | 4.0 to 4.5 | 0 | 0.046 to 0.053 | | `loadConfig()`, `bind=tailnet` | 200 | 3.8 to 5.1 | 1 | 0.74 to 0.92 | - Command: `pnpm --filter @paperclipai/server exec tsx scripts/benchmark-config-load.ts --calls=200` - Each call is synchronous, so the time per call is time that the event loop is blocked. For 200 calls the block went from about 750 to 1200 ms to about 9 to 14 ms where the bind does not read a tailnet address. - The `bind=tailnet` row after the fix contains the one detection (about 150 ms for the first start of the new stand-in file on macOS). The other 199 calls start no process. - `loadConfig()` also reads and validates the config file on each call. After the fix a full call costs about 0.05 ms locally, so the file read is not a material cost and this pull request does not change it. A hosted server reads a stored snapshot from memory, not a file. Other checks: - `cd server && pnpm exec vitest run src/__tests__/managed-runtime-public-url.test.ts src/__tests__/native-harness-backup-expiration-config.test.ts src/__tests__/chat-webhook-public-url.test.ts src/__tests__/managed-cloud-prerequisites.test.ts src/__tests__/managed-loopback-auth.test.ts src/__tests__/cloud-runtime-config.test.ts`: 84 tests pass. - `tsc --noEmit` over the three changed TypeScript files: clean. - `pnpm verify` exits 2 (broader verification required, because of the new file in `server/scripts`). Required CI supplies that coverage. - Not verified: the benchmark on Linux with a multi-gigabyte process, and the hosted server after deploy. The local numbers come from a 200 MB process, where a spawn is cheaper than on the profiled server. ## Risks - Low risk. The resolved `bind`, `host` and `customBindHost` are the same as before for each configuration. A configured bind mode other than `tailnet` never used the detected address, and a loopback or all-interfaces host resolved before the address was read. - Behavior shift: a server process detects the Tailscale address one time. If Tailscale starts, stops or changes its address later, `loadConfig()` in that process still returns the first result until restart. The server binds its listener one time at startup, so the listener already had this property. From the code (not reproduced): before this change, a `bind=tailnet` server whose Tailscale stopped after startup threw from each later `loadConfig()` call on runtime paths. It now continues with the address from startup. - A failed detection is also kept. With `bind=tailnet` the server refuses to start, as before, and the next start detects again. `PAPERCLIP_TAILNET_BIND_HOST` overrides a kept result at once. ## Model Used - Claude Fable 5.2 (Anthropic, model ID `claude-fable-5-2`), in Claude Code with tool use (shell, file edits, GitHub CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
471 lines
21 KiB
TypeScript
471 lines
21 KiB
TypeScript
import { validateManagedCloudBootstrapPrerequisites, validateManagedCloudPrerequisites } from "./managed-cloud-prerequisites.js";
|
|
import { readConfigFile, resolveDeploymentMode } from "./config-file.js";
|
|
import { isCloudManagedInstance } from "./services/cloud-instance.js";
|
|
import { StartupRefusalError } from "./startup-refusals.js";
|
|
import type { PaperclipConfig } from "@paperclipai/shared";
|
|
import { parseChatWebhookPublicBaseUrl } from "./chat-webhook-public-url.js";
|
|
import { execFileSync } from "node:child_process";
|
|
import { existsSync, realpathSync } from "node:fs";
|
|
import { basename, dirname, resolve } from "node:path";
|
|
import { config as loadDotenv } from "dotenv";
|
|
import { resolvePaperclipEnvPath } from "./paths.js";
|
|
import { maybeRepairLegacyWorktreeConfigAndEnvFiles } from "./worktree-config.js";
|
|
import { shouldLoadWorkingDirectoryEnv } from "./env-file-policy.js";
|
|
import { applyEmptyWorktreeSigningSecrets } from "./dev-runner-worktree.js";
|
|
import {
|
|
AUTH_BASE_URL_MODES,
|
|
BIND_MODES,
|
|
DEPLOYMENT_EXPOSURES,
|
|
SECRET_PROVIDERS,
|
|
type BindMode,
|
|
type AuthBaseUrlMode,
|
|
type DeploymentExposure,
|
|
type DeploymentMode,
|
|
type SecretProvider,
|
|
inferBindModeFromHost,
|
|
resolveRuntimeBind,
|
|
validateConfiguredBindMode,
|
|
} from "@paperclipai/shared";
|
|
import {
|
|
resolveDefaultBackupDir,
|
|
resolveDefaultEmbeddedPostgresDir,
|
|
resolveDefaultSecretsKeyFilePath,
|
|
resolveHomeAwarePath,
|
|
} from "./home-paths.js";
|
|
import { loadStorageConfig, type StorageConfig } from "./storage/config.js";
|
|
|
|
if (!isCloudManagedInstance()) {
|
|
const PAPERCLIP_ENV_FILE_PATH = resolvePaperclipEnvPath();
|
|
if (existsSync(PAPERCLIP_ENV_FILE_PATH)) {
|
|
loadDotenv({ path: PAPERCLIP_ENV_FILE_PATH, override: false, quiet: true });
|
|
}
|
|
|
|
const CWD_ENV_PATH = resolve(process.cwd(), ".env");
|
|
const cwdEnvExists = existsSync(CWD_ENV_PATH);
|
|
const isSameFile = cwdEnvExists && existsSync(PAPERCLIP_ENV_FILE_PATH)
|
|
? realpathSync(CWD_ENV_PATH) === realpathSync(PAPERCLIP_ENV_FILE_PATH)
|
|
: CWD_ENV_PATH === PAPERCLIP_ENV_FILE_PATH;
|
|
if (shouldLoadWorkingDirectoryEnv({
|
|
cwdEnvExists,
|
|
isPaperclipEnvFile: isSameFile,
|
|
})) {
|
|
loadDotenv({ path: CWD_ENV_PATH, override: false, quiet: true });
|
|
}
|
|
|
|
if (basename(dirname(PAPERCLIP_ENV_FILE_PATH)) === ".paperclip") {
|
|
applyEmptyWorktreeSigningSecrets(dirname(dirname(PAPERCLIP_ENV_FILE_PATH)));
|
|
}
|
|
maybeRepairLegacyWorktreeConfigAndEnvFiles();
|
|
}
|
|
|
|
const TAILSCALE_DETECT_TIMEOUT_MS = 3000;
|
|
|
|
type DatabaseMode = "embedded-postgres" | "postgres";
|
|
|
|
export interface Config extends StorageConfig {
|
|
deploymentMode: DeploymentMode;
|
|
deploymentExposure: DeploymentExposure;
|
|
bind: BindMode;
|
|
customBindHost: string | undefined;
|
|
host: string;
|
|
port: number;
|
|
allowedHostnames: string[];
|
|
authBaseUrlMode: AuthBaseUrlMode;
|
|
authPublicBaseUrl: string | undefined;
|
|
chatWebhookPublicBaseUrl: string | undefined;
|
|
authDisableSignUp: boolean;
|
|
databaseMode: DatabaseMode;
|
|
databaseUrl: string | undefined;
|
|
databaseMigrationUrl: string | undefined;
|
|
embeddedPostgresDataDir: string;
|
|
embeddedPostgresPort: number;
|
|
databaseBackupEnabled: boolean;
|
|
databaseBackupIntervalMinutes: number;
|
|
databaseBackupRetentionDays: number;
|
|
databaseBackupDir: string;
|
|
gitCacheExpirationEnabled: boolean;
|
|
gitBaseExpirationEnabled: boolean;
|
|
gitCacheExpirationDryRun: boolean;
|
|
isolatedSourceRehomeEnabled: boolean;
|
|
isolatedSourceRehomeDryRun: boolean;
|
|
nestedCloneMigrationEnabled: boolean;
|
|
nestedCloneMigrationDryRun: boolean;
|
|
workspaceReaperCooldownDays: number;
|
|
workspaceCompactionDelayHours: number | null;
|
|
skillVersionRetentionEnabled: boolean;
|
|
skillVersionRetentionDryRun: boolean;
|
|
skillVersionRetentionKeepNewest: number;
|
|
skillVersionRetentionGraceMinutes: number;
|
|
nativeHarnessBackupExpirationEnabled: boolean;
|
|
nativeHarnessBackupExpirationDryRun: boolean;
|
|
nativeHarnessBackupExpirationDays: number;
|
|
nativeRunnerEvidenceRetentionDays: number;
|
|
nativeWorkspaceSyncRetentionEnabled: boolean;
|
|
nativeWorkspaceSyncRetentionDryRun: boolean;
|
|
nativeWorkspaceSyncRetentionDays: number;
|
|
gitRestorePointRetentionDays: number;
|
|
gitRestorePointsEnabled: boolean;
|
|
gitRestorePointTerminalDelayDays: number;
|
|
gitRestorePointIdleDays: number;
|
|
taskDirectoryRetentionEnabled: boolean;
|
|
taskDirectoryRetentionDryRun: boolean;
|
|
taskDirectoryOrphanGraceDays: number;
|
|
serveUi: boolean;
|
|
uiDevMiddleware: boolean;
|
|
secretsProvider: SecretProvider;
|
|
secretsStrictMode: boolean;
|
|
secretsMasterKeyFilePath: string;
|
|
feedbackExportBackendUrl: string | undefined;
|
|
feedbackExportBackendToken: string | undefined;
|
|
heartbeatSchedulerEnabled: boolean;
|
|
heartbeatSchedulerIntervalMs: number;
|
|
companyDeletionEnabled: boolean;
|
|
telemetryEnabled: boolean;
|
|
announcementsEnabled: boolean;
|
|
announcementsFeedUrl: string;
|
|
}
|
|
|
|
let tailscaleAddressForProcess: { address: string | undefined } | undefined;
|
|
|
|
function detectTailnetBindHost(): string | undefined {
|
|
const explicit = process.env.PAPERCLIP_TAILNET_BIND_HOST?.trim();
|
|
if (explicit) return explicit;
|
|
|
|
tailscaleAddressForProcess ??= { address: runTailscaleAddressCommand() };
|
|
return tailscaleAddressForProcess.address;
|
|
}
|
|
|
|
function runTailscaleAddressCommand(): string | undefined {
|
|
try {
|
|
const stdout = execFileSync("tailscale", ["ip", "-4"], {
|
|
encoding: "utf8",
|
|
stdio: ["ignore", "pipe", "ignore"],
|
|
timeout: TAILSCALE_DETECT_TIMEOUT_MS,
|
|
});
|
|
return stdout
|
|
.split(/\r?\n/)
|
|
.map((line) => line.trim())
|
|
.find(Boolean);
|
|
} catch {
|
|
return undefined;
|
|
}
|
|
}
|
|
|
|
export function loadConfig(override?: PaperclipConfig | null): Config {
|
|
const configFile = override === undefined ? readConfigFile({ includeProvenance: true })
|
|
: override ? { config: override, explicitStorageS3Bucket: true } : null;
|
|
const fileConfig = configFile?.config ?? null;
|
|
const fileDatabaseMode =
|
|
(isCloudManagedInstance() || fileConfig?.database.mode === "postgres" ? "postgres" : "embedded-postgres") as DatabaseMode;
|
|
|
|
const fileDbUrl =
|
|
!isCloudManagedInstance() && fileDatabaseMode === "postgres"
|
|
? fileConfig?.database.connectionString
|
|
: undefined;
|
|
const fileDatabaseBackup = fileConfig?.database.backup;
|
|
const fileSecrets = fileConfig?.secrets;
|
|
|
|
const providerFromEnvRaw = process.env.PAPERCLIP_SECRETS_PROVIDER;
|
|
const providerFromEnv =
|
|
providerFromEnvRaw && SECRET_PROVIDERS.includes(providerFromEnvRaw as SecretProvider)
|
|
? (providerFromEnvRaw as SecretProvider)
|
|
: null;
|
|
const providerFromFile = fileSecrets?.provider;
|
|
const secretsProvider: SecretProvider = providerFromEnv ?? providerFromFile ?? "local_encrypted";
|
|
|
|
const storage = loadStorageConfig(fileConfig);
|
|
validateManagedCloudPrerequisites({ ...storage,
|
|
storageS3Bucket: process.env.PAPERCLIP_STORAGE_S3_BUCKET
|
|
?? (configFile?.explicitStorageS3Bucket ? fileConfig?.storage.s3.bucket : undefined),
|
|
databaseUrl: process.env.DATABASE_URL ?? fileDbUrl,
|
|
}, process.env);
|
|
const feedbackExportBackendUrl =
|
|
process.env.PAPERCLIP_FEEDBACK_EXPORT_BACKEND_URL?.trim() ||
|
|
process.env.PAPERCLIP_TELEMETRY_BACKEND_URL?.trim() ||
|
|
undefined;
|
|
const feedbackExportBackendToken =
|
|
process.env.PAPERCLIP_FEEDBACK_EXPORT_BACKEND_TOKEN?.trim() ||
|
|
process.env.PAPERCLIP_TELEMETRY_BACKEND_TOKEN?.trim() ||
|
|
undefined;
|
|
|
|
const deploymentMode = resolveDeploymentMode(fileConfig);
|
|
const strictModeFromEnv = process.env.PAPERCLIP_SECRETS_STRICT_MODE;
|
|
const secretsStrictMode =
|
|
strictModeFromEnv !== undefined
|
|
? strictModeFromEnv === "true"
|
|
: (fileSecrets?.strictMode ?? deploymentMode === "authenticated");
|
|
const deploymentExposureFromEnvRaw = process.env.PAPERCLIP_DEPLOYMENT_EXPOSURE;
|
|
const deploymentExposureFromEnv =
|
|
deploymentExposureFromEnvRaw &&
|
|
DEPLOYMENT_EXPOSURES.includes(deploymentExposureFromEnvRaw as DeploymentExposure)
|
|
? (deploymentExposureFromEnvRaw as DeploymentExposure)
|
|
: null;
|
|
const deploymentExposure: DeploymentExposure =
|
|
deploymentMode === "local_trusted"
|
|
? "private"
|
|
: (deploymentExposureFromEnv ?? fileConfig?.server.exposure ?? "private");
|
|
const bindFromEnvRaw = process.env.PAPERCLIP_BIND;
|
|
const bindFromEnv =
|
|
bindFromEnvRaw && BIND_MODES.includes(bindFromEnvRaw as BindMode)
|
|
? (bindFromEnvRaw as BindMode)
|
|
: null;
|
|
const configuredHost = process.env.HOST ?? fileConfig?.server.host ?? "127.0.0.1";
|
|
const configuredBind = bindFromEnv ?? fileConfig?.server.bind;
|
|
const bindReadsTailnetAddress = configuredBind
|
|
? configuredBind === "tailnet"
|
|
: inferBindModeFromHost(configuredHost) === "custom";
|
|
const tailnetBindHost = bindReadsTailnetAddress ? detectTailnetBindHost() : undefined;
|
|
const bind = configuredBind ?? inferBindModeFromHost(configuredHost, { tailnetBindHost });
|
|
const customBindHost = process.env.PAPERCLIP_BIND_HOST ?? fileConfig?.server.customBindHost;
|
|
const authBaseUrlModeFromEnvRaw = process.env.PAPERCLIP_AUTH_BASE_URL_MODE;
|
|
const authBaseUrlModeFromEnv =
|
|
authBaseUrlModeFromEnvRaw &&
|
|
AUTH_BASE_URL_MODES.includes(authBaseUrlModeFromEnvRaw as AuthBaseUrlMode)
|
|
? (authBaseUrlModeFromEnvRaw as AuthBaseUrlMode)
|
|
: null;
|
|
const publicUrlFromEnv = process.env.PAPERCLIP_PUBLIC_URL;
|
|
const configuredAuthPublicBaseUrlRaw = [
|
|
process.env.PAPERCLIP_AUTH_PUBLIC_BASE_URL,
|
|
process.env.BETTER_AUTH_URL,
|
|
process.env.BETTER_AUTH_BASE_URL,
|
|
publicUrlFromEnv,
|
|
fileConfig?.auth?.publicBaseUrl,
|
|
].find((value): value is string => typeof value === "string" && value.trim().length > 0);
|
|
const managedRuntimePublicUrl = process.env.PAPERCLIP_MANAGED_RUNTIME_PUBLIC_URL?.trim() || undefined;
|
|
const authPublicBaseUrlRaw = configuredAuthPublicBaseUrlRaw ?? managedRuntimePublicUrl;
|
|
const authPublicBaseUrl = authPublicBaseUrlRaw?.trim() || undefined;
|
|
const authBaseUrlMode: AuthBaseUrlMode =
|
|
authBaseUrlModeFromEnv ??
|
|
(configuredAuthPublicBaseUrlRaw === undefined && managedRuntimePublicUrl
|
|
? "explicit"
|
|
: fileConfig?.auth?.baseUrlMode ?? (authPublicBaseUrl ? "explicit" : "auto"));
|
|
const disableSignUpFromEnv = process.env.PAPERCLIP_AUTH_DISABLE_SIGN_UP;
|
|
const authDisableSignUp: boolean =
|
|
disableSignUpFromEnv !== undefined
|
|
? disableSignUpFromEnv === "true"
|
|
: (fileConfig?.auth?.disableSignUp ?? false);
|
|
const allowedHostnamesFromEnvRaw = process.env.PAPERCLIP_ALLOWED_HOSTNAMES;
|
|
const allowedHostnamesFromEnv = allowedHostnamesFromEnvRaw
|
|
? allowedHostnamesFromEnvRaw
|
|
.split(",")
|
|
.map((value) => value.trim().toLowerCase())
|
|
.filter((value) => value.length > 0)
|
|
: null;
|
|
const publicUrlHostname = authPublicBaseUrl
|
|
? (() => {
|
|
try {
|
|
return new URL(authPublicBaseUrl).hostname.trim().toLowerCase();
|
|
} catch {
|
|
return null;
|
|
}
|
|
})()
|
|
: null;
|
|
const allowedHostnames = Array.from(
|
|
new Set(
|
|
[
|
|
...(allowedHostnamesFromEnv ?? fileConfig?.server.allowedHostnames ?? []),
|
|
...(publicUrlHostname ? [publicUrlHostname] : []),
|
|
]
|
|
.map((value) => value.trim().toLowerCase())
|
|
.filter(Boolean),
|
|
),
|
|
);
|
|
const companyDeletionEnvRaw = process.env.PAPERCLIP_ENABLE_COMPANY_DELETION;
|
|
const companyDeletionEnabled =
|
|
companyDeletionEnvRaw !== undefined
|
|
? companyDeletionEnvRaw === "true"
|
|
: deploymentMode === "local_trusted";
|
|
const databaseBackupEnabled =
|
|
process.env.PAPERCLIP_DB_BACKUP_ENABLED !== undefined
|
|
? process.env.PAPERCLIP_DB_BACKUP_ENABLED === "true"
|
|
: (fileDatabaseBackup?.enabled ?? true);
|
|
const databaseBackupIntervalMinutes = Math.max(
|
|
1,
|
|
Number(process.env.PAPERCLIP_DB_BACKUP_INTERVAL_MINUTES) ||
|
|
fileDatabaseBackup?.intervalMinutes ||
|
|
60,
|
|
);
|
|
const databaseBackupRetentionDays = Math.max(
|
|
1,
|
|
Number(process.env.PAPERCLIP_DB_BACKUP_RETENTION_DAYS) ||
|
|
fileDatabaseBackup?.retentionDays ||
|
|
7,
|
|
);
|
|
const databaseBackupDir = resolveHomeAwarePath(
|
|
process.env.PAPERCLIP_DB_BACKUP_DIR ??
|
|
fileDatabaseBackup?.dir ??
|
|
resolveDefaultBackupDir(),
|
|
);
|
|
// The terminal-workspace reaper waits this many days after an issue tree
|
|
// becomes terminal before it archives the workspace. A person can reopen the
|
|
// work inside this window. A value of 0 disables the cooldown and restores
|
|
// immediate reaping. A negative or non-numeric value falls back to the
|
|
// default. The day granularity obeys the PAPERCLIP_DB_BACKUP_RETENTION_DAYS
|
|
// precedent above. The default is 1 day: regenerable files are compacted
|
|
// after a few hours anyway, so a longer hold only keeps source checkouts of
|
|
// delivered work, and a recurring task produces one workspace per run.
|
|
const workspaceReaperCooldownDaysEnv =
|
|
process.env.PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS?.trim();
|
|
const workspaceReaperCooldownDaysRaw = Number(workspaceReaperCooldownDaysEnv);
|
|
const workspaceReaperCooldownDays =
|
|
workspaceReaperCooldownDaysEnv
|
|
&& Number.isFinite(workspaceReaperCooldownDaysRaw)
|
|
&& workspaceReaperCooldownDaysRaw >= 0
|
|
? workspaceReaperCooldownDaysRaw
|
|
: 1;
|
|
// The terminal-workspace reaper removes regenerable files (installed
|
|
// dependencies, build output, tool caches) from a terminal workspace this
|
|
// many hours after its issue tree becomes terminal. The worktree and every
|
|
// source file stay; a resumed task regenerates the files when its workspace
|
|
// is provisioned again. 0 compacts on the next sweep. A negative, empty, or
|
|
// non-numeric value falls back to the default of 1 hour.
|
|
// PAPERCLIP_WORKSPACE_COMPACTION_ENABLED=false turns compaction off.
|
|
const workspaceCompactionDelayHoursEnv =
|
|
process.env.PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS?.trim();
|
|
const workspaceCompactionDelayHoursRaw = Number(workspaceCompactionDelayHoursEnv);
|
|
const workspaceCompactionDelayHours =
|
|
process.env.PAPERCLIP_WORKSPACE_COMPACTION_ENABLED?.trim().toLowerCase() === "false"
|
|
? null
|
|
: workspaceCompactionDelayHoursEnv
|
|
&& Number.isFinite(workspaceCompactionDelayHoursRaw)
|
|
&& workspaceCompactionDelayHoursRaw >= 0
|
|
? workspaceCompactionDelayHoursRaw
|
|
: 1;
|
|
// Skill version retention removes materialized version directories under
|
|
// `skills/<companyId>/__versions__/` that no current version, agent pin, or
|
|
// live run uses. It keeps this many newest versions per skill as a warm
|
|
// cache. A version directory used within the grace window also stays. A
|
|
// removed directory is written again from its version row on next use.
|
|
// Negative, empty, or non-numeric values fall back to the defaults.
|
|
const nonNegativeIntegerEnv = (name: string, fallback: number) => {
|
|
const raw = process.env[name]?.trim();
|
|
const value = Number(raw);
|
|
return raw && Number.isFinite(value) && value >= 0 ? Math.floor(value) : fallback;
|
|
};
|
|
const skillVersionRetentionKeepNewest = nonNegativeIntegerEnv("PAPERCLIP_SKILL_VERSION_RETENTION_KEEP", 0);
|
|
const skillVersionRetentionGraceMinutes = nonNegativeIntegerEnv("PAPERCLIP_SKILL_VERSION_RETENTION_GRACE_MINUTES", 60);
|
|
const nativeHarnessBackupExpirationDays = Math.max(1, nonNegativeIntegerEnv("PAPERCLIP_NATIVE_HARNESS_BACKUP_EXPIRATION_DAYS", 7));
|
|
const nativeRunnerEvidenceRetentionDays = Math.max(1, nonNegativeIntegerEnv("PAPERCLIP_NATIVE_RUNNER_EVIDENCE_RETENTION_DAYS", 7));
|
|
const nativeWorkspaceSyncRetentionDays = Math.max(1, nonNegativeIntegerEnv("PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_DAYS", 7));
|
|
const gitRestorePointRetentionDays = Math.max(1, nonNegativeIntegerEnv("PAPERCLIP_GIT_RESTORE_POINT_RETENTION_DAYS", 90));
|
|
const gitRestorePointTerminalDelayDays = nonNegativeIntegerEnv("PAPERCLIP_GIT_RESTORE_POINT_TERMINAL_DELAY_DAYS", 3);
|
|
const gitRestorePointIdleDays = nonNegativeIntegerEnv("PAPERCLIP_GIT_RESTORE_POINT_IDLE_DAYS", 7);
|
|
const taskDirectoryOrphanGraceDays = Math.max(1, nonNegativeIntegerEnv("PAPERCLIP_TASK_DIRECTORY_ORPHAN_GRACE_DAYS", 7));
|
|
const bindValidationErrors = validateConfiguredBindMode({
|
|
deploymentMode,
|
|
deploymentExposure,
|
|
bind,
|
|
host: configuredHost,
|
|
customBindHost,
|
|
});
|
|
if (bindValidationErrors.length > 0) {
|
|
throw new Error(bindValidationErrors[0]);
|
|
}
|
|
const resolvedBind = resolveRuntimeBind({
|
|
bind,
|
|
host: configuredHost,
|
|
customBindHost,
|
|
tailnetBindHost,
|
|
});
|
|
if (resolvedBind.errors.length > 0) {
|
|
throw new Error(resolvedBind.errors[0]);
|
|
}
|
|
|
|
return {
|
|
deploymentMode,
|
|
deploymentExposure,
|
|
bind: resolvedBind.bind,
|
|
customBindHost: resolvedBind.customBindHost,
|
|
host: resolvedBind.host,
|
|
port: Number(process.env.PORT) || fileConfig?.server.port || 3100,
|
|
allowedHostnames,
|
|
authBaseUrlMode,
|
|
authPublicBaseUrl,
|
|
chatWebhookPublicBaseUrl: parseChatWebhookPublicBaseUrl(
|
|
process.env.PAPERCLIP_CHAT_WEBHOOK_PUBLIC_URL,
|
|
),
|
|
authDisableSignUp,
|
|
databaseMode: fileDatabaseMode,
|
|
databaseUrl: process.env.DATABASE_URL ?? fileDbUrl,
|
|
databaseMigrationUrl: process.env.DATABASE_MIGRATION_URL,
|
|
embeddedPostgresDataDir: resolveHomeAwarePath(
|
|
fileConfig?.database.embeddedPostgresDataDir ?? resolveDefaultEmbeddedPostgresDir(),
|
|
),
|
|
embeddedPostgresPort: fileConfig?.database.embeddedPostgresPort ?? 54329,
|
|
databaseBackupEnabled,
|
|
databaseBackupIntervalMinutes,
|
|
databaseBackupRetentionDays,
|
|
databaseBackupDir,
|
|
gitCacheExpirationEnabled: process.env.PAPERCLIP_GIT_CACHE_EXPIRATION_ENABLED === "true",
|
|
gitBaseExpirationEnabled: process.env.PAPERCLIP_GIT_BASE_EXPIRATION_ENABLED === "true",
|
|
gitCacheExpirationDryRun: process.env.PAPERCLIP_GIT_CACHE_EXPIRATION_DRY_RUN !== "false",
|
|
isolatedSourceRehomeEnabled: process.env.PAPERCLIP_ISOLATED_SOURCE_REHOME_ENABLED === "true",
|
|
isolatedSourceRehomeDryRun: process.env.PAPERCLIP_ISOLATED_SOURCE_REHOME_DRY_RUN === "true",
|
|
nestedCloneMigrationEnabled: process.env.PAPERCLIP_NESTED_CLONE_MIGRATION_ENABLED === "true",
|
|
nestedCloneMigrationDryRun: process.env.PAPERCLIP_NESTED_CLONE_MIGRATION_DRY_RUN === "true",
|
|
workspaceReaperCooldownDays,
|
|
workspaceCompactionDelayHours,
|
|
skillVersionRetentionEnabled: process.env.PAPERCLIP_SKILL_VERSION_RETENTION_ENABLED !== "false",
|
|
skillVersionRetentionDryRun: process.env.PAPERCLIP_SKILL_VERSION_RETENTION_DRY_RUN === "true",
|
|
skillVersionRetentionKeepNewest,
|
|
skillVersionRetentionGraceMinutes,
|
|
nativeHarnessBackupExpirationEnabled: process.env.PAPERCLIP_NATIVE_HARNESS_BACKUP_EXPIRATION_ENABLED !== "false",
|
|
nativeHarnessBackupExpirationDryRun: process.env.PAPERCLIP_NATIVE_HARNESS_BACKUP_EXPIRATION_DRY_RUN === "true",
|
|
nativeHarnessBackupExpirationDays,
|
|
nativeRunnerEvidenceRetentionDays,
|
|
nativeWorkspaceSyncRetentionEnabled: process.env.PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_ENABLED !== "false",
|
|
nativeWorkspaceSyncRetentionDryRun: process.env.PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_DRY_RUN === "true",
|
|
nativeWorkspaceSyncRetentionDays,
|
|
gitRestorePointRetentionDays,
|
|
gitRestorePointsEnabled: process.env.PAPERCLIP_GIT_RESTORE_POINTS_ENABLED === "true",
|
|
gitRestorePointTerminalDelayDays,
|
|
gitRestorePointIdleDays,
|
|
taskDirectoryRetentionEnabled: process.env.PAPERCLIP_TASK_DIRECTORY_RETENTION_ENABLED === "true",
|
|
taskDirectoryRetentionDryRun: process.env.PAPERCLIP_TASK_DIRECTORY_RETENTION_DRY_RUN !== "false",
|
|
taskDirectoryOrphanGraceDays,
|
|
serveUi:
|
|
process.env.SERVE_UI !== undefined
|
|
? process.env.SERVE_UI === "true"
|
|
: fileConfig?.server.serveUi ?? true,
|
|
uiDevMiddleware: process.env.PAPERCLIP_UI_DEV_MIDDLEWARE === "true",
|
|
secretsProvider,
|
|
secretsStrictMode,
|
|
secretsMasterKeyFilePath:
|
|
resolveHomeAwarePath(
|
|
process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE ??
|
|
fileSecrets?.localEncrypted.keyFilePath ??
|
|
resolveDefaultSecretsKeyFilePath(),
|
|
),
|
|
...storage,
|
|
feedbackExportBackendUrl,
|
|
feedbackExportBackendToken,
|
|
heartbeatSchedulerEnabled: process.env.HEARTBEAT_SCHEDULER_ENABLED !== "false",
|
|
heartbeatSchedulerIntervalMs: Math.max(10000, Number(process.env.HEARTBEAT_SCHEDULER_INTERVAL_MS) || 30000),
|
|
companyDeletionEnabled,
|
|
telemetryEnabled: fileConfig?.telemetry?.enabled ?? true,
|
|
announcementsEnabled: process.env.PAPERCLIP_ANNOUNCEMENTS_ENABLED !== "false",
|
|
announcementsFeedUrl: process.env.PAPERCLIP_ANNOUNCEMENTS_FEED_URL?.trim() || "https://pages.paperclip.ing/announcements/v1/current.json",
|
|
};
|
|
}
|
|
|
|
export type CloudBootstrapConfig = {
|
|
databaseMode: "postgres";
|
|
databaseUrl: string;
|
|
databaseMigrationUrl: string | undefined;
|
|
};
|
|
|
|
export function loadCloudBootstrapConfig(): CloudBootstrapConfig {
|
|
if (!isCloudManagedInstance()) throw new Error("Cloud bootstrap configuration requires a managed instance");
|
|
const url = process.env.DATABASE_URL?.trim();
|
|
if (!url) throw new StartupRefusalError("database-contract-unmet", "Cloud bootstrap requires DATABASE_URL from the environment");
|
|
const migrationUrl = process.env.DATABASE_MIGRATION_URL?.trim() || undefined;
|
|
for (const [name, value] of [["DATABASE_URL", url], ["DATABASE_MIGRATION_URL", migrationUrl]]) {
|
|
if (!value) continue;
|
|
let parsed: URL;
|
|
try { parsed = new URL(value); } catch { throw new Error(`Cloud bootstrap ${name} must use PostgreSQL`); }
|
|
if (!["postgres:", "postgresql:"].includes(parsed.protocol)) throw new Error(`Cloud bootstrap ${name} must use PostgreSQL`);
|
|
}
|
|
validateManagedCloudBootstrapPrerequisites({ databaseUrl: url }, process.env);
|
|
return { databaseMode: "postgres", databaseUrl: url, databaseMigrationUrl: migrationUrl };
|
|
}
|