mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-01 15:35:12 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Managed Cloud instances must retain shared objects and secrets across server replacement. > - Implicit storage defaults can select local disk or an unintended bucket. > - Cloud runtime settings now come from a canonical database snapshot. > - This pull request validates bootstrap prerequisites before database setup and effective S3 settings after snapshot resolution. > - The benefit is an actionable refusal before local signing state or HTTP setup, while valid canonical Cloud settings work. ## Linked Issues or Issue Description **What happened?** A managed Cloud instance could start with local disk storage, an implicit bucket, or a local key file. After canonical Cloud configuration landed, checking only environment or file values would also reject a valid database bucket. Shared schema defaults could make an omitted raw bucket appear explicit. **Expected behavior** Managed Cloud requires external PostgreSQL, a stable environment master key, and effective S3 settings with a nonblank bucket. The canonical snapshot remains the configuration authority. Invalid configuration refuses startup before signing or HTTP effects. Self-hosted defaults remain compatible. **Steps to reproduce** 1. Select managed Cloud with its tenant token or managed configuration document. 2. Leave storage as local disk, or omit the S3 bucket from a raw canonical snapshot. 3. Start the server. Startup must refuse. An explicit valid bucket in the canonical snapshot must work without a file or environment bucket. **Paperclip version or commit** Current base: `4edd10804e2b42941aa2038e90c362c13a0b98dc`. **Deployment mode** Managed Cloud. Ordinary self-hosted defaults remain unchanged. Related PRs: #18335 owns canonical Cloud runtime configuration. #18327 owns S3 run logs. #18352 owns credential persistence. #18347 owns cutover checks. This PR adds configuration prerequisites to those existing boundaries. ## What Changed - Validate external PostgreSQL and a stable environment master key at the Cloud bootstrap boundary, before database setup. - Validate effective S3 settings after canonical snapshot publication and before Cloud signing, runtime consumers, or HTTP setup. - Require a nonblank bucket in the Cloud snapshot schema. Reject omitted or blank buckets in every legacy Cloud file before one-time import normalization, even when an environment bucket is supplied. Keep shared self-hosted schema defaults. - Preserve typed bucket provenance at the existing file-read boundary. Accept explicit canonical bucket names, including `paperclip`, and environment settings when the snapshot is null. - Validate effective legacy storage with the existing resolver before seeding. A rejected candidate leaves no canonical rows, so a corrected source file can be imported on retry. Environment overlays are checked but never persisted. - Add real PostgreSQL snapshot and migration-retry tests, startup refusal tests, and environment/file compatibility controls. Document that Cloud always requires its stable environment key, even with an external default secrets provider. ## Verification - Latest focused acceptance passed 199/199 in six files (54.29 seconds), using the command below plus `server/src/__tests__/cloud-configuration-migration-postgres.test.ts`. Real PostgreSQL proves rejected storage leaves all three migration authority rows absent, then a corrected legacy file imports on retry. An external-default-secrets control proves the deliberate unconditional Cloud master-key contract. - Before the peer correction, the frozen product diff passed 178 tests in five files: managed-cloud-prerequisites, cloud-runtime-config, server-startup-feedback-export, config-file, and cloud-configuration-readers. Command: `pnpm exec vitest run server/src/__tests__/{managed-cloud-prerequisites,cloud-runtime-config,server-startup-feedback-export,config-file,cloud-configuration-readers}.test.ts --maxWorkers=1 --no-file-parallelism`. Real PostgreSQL used a qualified POSIX test-only PG18.6 override because the host cannot supply the stock runtime's SysV semaphore allocation. - Coverage includes raw database and legacy omitted/blank bucket rejection, valid database and environment buckets, null snapshots, explicit default-named buckets, self-hosted compatibility, and refusal before local signing mutation. - The real AWS SDK configured-provider control uses a simulated loopback S3 service. It writes, reads from a fresh provider, and deletes an object. It also reads an encrypted credential with the stable environment key. This does not qualify production cloud permissions. - `git diff --check` passed. Final merges add unrelated master changes from #18241, #18363, and #18370; the focused storage product diff is unchanged. - Peer review reproduced a fresh legacy local-disk import that gained the default bucket before an S3 environment overlay. The importer now requires a raw explicit bucket for every legacy Cloud file. Six controls cover omission/blank values with and without an environment bucket; the full focused rerun passed 184/184 across all five files in 194.89 seconds. - Initial revised startup testing passed 167/168: one fixture omitted the database URL and failed before its intended key assertion. The corrected final run passed 178/178. An initial direct server typecheck failed because Runner declarations were not built; this is a setup failure, not a compile pass. Integration verification rebuilds dependencies before full checks. - Prior head `fae1e4aa` passed complete CI with 53 successes, two intended skips, and Greptile 5/5. `de71345f379cecb71dbfb219f35e14f942c10a3f` then passed complete CI (55 successes, three workflow skips, and a neutral external review), with Greptile 5/5 and zero unresolved threads. The external review identified the preseed rejection problem; its correction is committed at `75e04d0d77e19dcef0a8cb070b6b518f0f173b14`. The final merge head is `19b809ea2823a6a4e5ceb48ef2c2bac4721733d6`; [Fresh complete CI](https://github.com/paperclipai/paperclip/actions/runs/41127309762) passed on that head: 55 successful checks, three workflow skips, and one neutral external review. Greptile reviewed this exact head at 5/5 with no actionable issues. Both earlier external review threads are resolved with the tested fixes. Earlier infrastructure retries remain documented in [the first diagnosis](https://github.com/paperclipai/paperclip/pull/18350#issuecomment-6529964690) and [the runner shutdown evidence](https://github.com/paperclipai/paperclip/pull/18350#issuecomment-6530364769). Earlier missing SDK build and fixture metadata failures were repaired before those passes. - Current frozen-head CI first attempt failed one unchanged Antigravity authentication success case in Runner Vitest 1/2 (`outcome=failure`, 1994 tests passed). The same file passed 11/11 locally in 2.33 seconds, and identical Runner source passed the prior complete CI run. Attempt 2 retried only failed jobs (the Runner shard and dependent aggregate). Both passed without a source change. No second retry was used. The exact timing cause is not established. [Failed job](https://github.com/paperclipai/paperclip/actions/runs/41127309762/job/124309181778). - No production data, buckets, or live credentials were used. Complete local repository suites are not duplicated in this worktree; current-head required CI supplies the full suite gate. ## Risks - Managed deployments with local disk, omitted buckets, file-only keys, or missing external database URLs now refuse startup. Operators must supply valid prerequisites before rollout. - Required deployment environment overlays must remain stable across replacements. For example, a saved local-disk provider with an explicit bucket can use an environment S3 provider; removing that required overlay correctly refuses startup until the environment or canonical configuration is corrected. Environment settings are never silently copied into the canonical row. - Effective legacy storage must also pass before first seeding. A bad candidate does not create authority rows. Existing saved rows remain authoritative and can require environment overrides or a canonical configuration change if they are invalid. - A present legacy Cloud file needs an explicit raw bucket even if its saved provider is local disk or its environment supplies a bucket. Correct the file before one-time import, or use the existing absent-file/null-snapshot environment-only setup. Environment overlays are not persisted as operator choices. - A previously stored canonical nonblank `paperclip` bucket is authoritative. Prior normalization erased whether that value was originally explicit. This change cannot recover that provenance and does not reject existing nonblank canonical values. - Validation proves configuration shape, not cloud permissions, database reachability, or key compatibility with existing encrypted data. Keep the master key stable across replacements. - Cloud snapshot JSON parsing is stricter; omitted or blank raw S3 bucket values need correction. No database migration or provider operation changes are added. These prerequisites do not prove that every server persistence path is disposable. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI Codex, exact model ID `gpt-6.1-sol`. Context window: unknown. The model used reasoning, repository tools, code execution, and test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
463 lines
21 KiB
TypeScript
463 lines
21 KiB
TypeScript
import { validateManagedCloudBootstrapPrerequisites, validateManagedCloudPrerequisites } from "./managed-cloud-prerequisites.js";
|
|
import { readConfigFile, resolveDeploymentMode } from "./config-file.js";
|
|
import { isCloudManagedInstance } from "./services/cloud-instance.js";
|
|
import { StartupRefusalError } from "./startup-refusals.js";
|
|
import type { PaperclipConfig } from "@paperclipai/shared";
|
|
import { parseChatWebhookPublicBaseUrl } from "./chat-webhook-public-url.js";
|
|
import { execFileSync } from "node:child_process";
|
|
import { existsSync, realpathSync } from "node:fs";
|
|
import { basename, dirname, resolve } from "node:path";
|
|
import { config as loadDotenv } from "dotenv";
|
|
import { resolvePaperclipEnvPath } from "./paths.js";
|
|
import { maybeRepairLegacyWorktreeConfigAndEnvFiles } from "./worktree-config.js";
|
|
import { shouldLoadWorkingDirectoryEnv } from "./env-file-policy.js";
|
|
import { applyEmptyWorktreeSigningSecrets } from "./dev-runner-worktree.js";
|
|
import {
|
|
AUTH_BASE_URL_MODES,
|
|
BIND_MODES,
|
|
DEPLOYMENT_EXPOSURES,
|
|
SECRET_PROVIDERS,
|
|
type BindMode,
|
|
type AuthBaseUrlMode,
|
|
type DeploymentExposure,
|
|
type DeploymentMode,
|
|
type SecretProvider,
|
|
inferBindModeFromHost,
|
|
resolveRuntimeBind,
|
|
validateConfiguredBindMode,
|
|
} from "@paperclipai/shared";
|
|
import {
|
|
resolveDefaultBackupDir,
|
|
resolveDefaultEmbeddedPostgresDir,
|
|
resolveDefaultSecretsKeyFilePath,
|
|
resolveHomeAwarePath,
|
|
} from "./home-paths.js";
|
|
import { loadStorageConfig, type StorageConfig } from "./storage/config.js";
|
|
|
|
if (!isCloudManagedInstance()) {
|
|
const PAPERCLIP_ENV_FILE_PATH = resolvePaperclipEnvPath();
|
|
if (existsSync(PAPERCLIP_ENV_FILE_PATH)) {
|
|
loadDotenv({ path: PAPERCLIP_ENV_FILE_PATH, override: false, quiet: true });
|
|
}
|
|
|
|
const CWD_ENV_PATH = resolve(process.cwd(), ".env");
|
|
const cwdEnvExists = existsSync(CWD_ENV_PATH);
|
|
const isSameFile = cwdEnvExists && existsSync(PAPERCLIP_ENV_FILE_PATH)
|
|
? realpathSync(CWD_ENV_PATH) === realpathSync(PAPERCLIP_ENV_FILE_PATH)
|
|
: CWD_ENV_PATH === PAPERCLIP_ENV_FILE_PATH;
|
|
if (shouldLoadWorkingDirectoryEnv({
|
|
cwdEnvExists,
|
|
isPaperclipEnvFile: isSameFile,
|
|
})) {
|
|
loadDotenv({ path: CWD_ENV_PATH, override: false, quiet: true });
|
|
}
|
|
|
|
if (basename(dirname(PAPERCLIP_ENV_FILE_PATH)) === ".paperclip") {
|
|
applyEmptyWorktreeSigningSecrets(dirname(dirname(PAPERCLIP_ENV_FILE_PATH)));
|
|
}
|
|
maybeRepairLegacyWorktreeConfigAndEnvFiles();
|
|
}
|
|
|
|
const TAILSCALE_DETECT_TIMEOUT_MS = 3000;
|
|
|
|
type DatabaseMode = "embedded-postgres" | "postgres";
|
|
|
|
export interface Config extends StorageConfig {
|
|
deploymentMode: DeploymentMode;
|
|
deploymentExposure: DeploymentExposure;
|
|
bind: BindMode;
|
|
customBindHost: string | undefined;
|
|
host: string;
|
|
port: number;
|
|
allowedHostnames: string[];
|
|
authBaseUrlMode: AuthBaseUrlMode;
|
|
authPublicBaseUrl: string | undefined;
|
|
chatWebhookPublicBaseUrl: string | undefined;
|
|
authDisableSignUp: boolean;
|
|
databaseMode: DatabaseMode;
|
|
databaseUrl: string | undefined;
|
|
databaseMigrationUrl: string | undefined;
|
|
embeddedPostgresDataDir: string;
|
|
embeddedPostgresPort: number;
|
|
databaseBackupEnabled: boolean;
|
|
databaseBackupIntervalMinutes: number;
|
|
databaseBackupRetentionDays: number;
|
|
databaseBackupDir: string;
|
|
gitCacheExpirationEnabled: boolean;
|
|
gitBaseExpirationEnabled: boolean;
|
|
gitCacheExpirationDryRun: boolean;
|
|
isolatedSourceRehomeEnabled: boolean;
|
|
isolatedSourceRehomeDryRun: boolean;
|
|
nestedCloneMigrationEnabled: boolean;
|
|
nestedCloneMigrationDryRun: boolean;
|
|
workspaceReaperCooldownDays: number;
|
|
workspaceCompactionDelayHours: number | null;
|
|
skillVersionRetentionEnabled: boolean;
|
|
skillVersionRetentionDryRun: boolean;
|
|
skillVersionRetentionKeepNewest: number;
|
|
skillVersionRetentionGraceMinutes: number;
|
|
nativeHarnessBackupExpirationEnabled: boolean;
|
|
nativeHarnessBackupExpirationDryRun: boolean;
|
|
nativeHarnessBackupExpirationDays: number;
|
|
nativeRunnerEvidenceRetentionDays: number;
|
|
nativeWorkspaceSyncRetentionEnabled: boolean;
|
|
nativeWorkspaceSyncRetentionDryRun: boolean;
|
|
nativeWorkspaceSyncRetentionDays: number;
|
|
gitRestorePointRetentionDays: number;
|
|
gitRestorePointsEnabled: boolean;
|
|
gitRestorePointTerminalDelayDays: number;
|
|
gitRestorePointIdleDays: number;
|
|
taskDirectoryRetentionEnabled: boolean;
|
|
taskDirectoryRetentionDryRun: boolean;
|
|
taskDirectoryOrphanGraceDays: number;
|
|
serveUi: boolean;
|
|
uiDevMiddleware: boolean;
|
|
secretsProvider: SecretProvider;
|
|
secretsStrictMode: boolean;
|
|
secretsMasterKeyFilePath: string;
|
|
feedbackExportBackendUrl: string | undefined;
|
|
feedbackExportBackendToken: string | undefined;
|
|
heartbeatSchedulerEnabled: boolean;
|
|
heartbeatSchedulerIntervalMs: number;
|
|
companyDeletionEnabled: boolean;
|
|
telemetryEnabled: boolean;
|
|
announcementsEnabled: boolean;
|
|
announcementsFeedUrl: string;
|
|
}
|
|
|
|
function detectTailnetBindHost(): string | undefined {
|
|
const explicit = process.env.PAPERCLIP_TAILNET_BIND_HOST?.trim();
|
|
if (explicit) return explicit;
|
|
|
|
try {
|
|
const stdout = execFileSync("tailscale", ["ip", "-4"], {
|
|
encoding: "utf8",
|
|
stdio: ["ignore", "pipe", "ignore"],
|
|
timeout: TAILSCALE_DETECT_TIMEOUT_MS,
|
|
});
|
|
return stdout
|
|
.split(/\r?\n/)
|
|
.map((line) => line.trim())
|
|
.find(Boolean);
|
|
} catch {
|
|
return undefined;
|
|
}
|
|
}
|
|
|
|
export function loadConfig(override?: PaperclipConfig | null): Config {
|
|
const configFile = override === undefined ? readConfigFile({ includeProvenance: true })
|
|
: override ? { config: override, explicitStorageS3Bucket: true } : null;
|
|
const fileConfig = configFile?.config ?? null;
|
|
const fileDatabaseMode =
|
|
(isCloudManagedInstance() || fileConfig?.database.mode === "postgres" ? "postgres" : "embedded-postgres") as DatabaseMode;
|
|
|
|
const fileDbUrl =
|
|
!isCloudManagedInstance() && fileDatabaseMode === "postgres"
|
|
? fileConfig?.database.connectionString
|
|
: undefined;
|
|
const fileDatabaseBackup = fileConfig?.database.backup;
|
|
const fileSecrets = fileConfig?.secrets;
|
|
|
|
const providerFromEnvRaw = process.env.PAPERCLIP_SECRETS_PROVIDER;
|
|
const providerFromEnv =
|
|
providerFromEnvRaw && SECRET_PROVIDERS.includes(providerFromEnvRaw as SecretProvider)
|
|
? (providerFromEnvRaw as SecretProvider)
|
|
: null;
|
|
const providerFromFile = fileSecrets?.provider;
|
|
const secretsProvider: SecretProvider = providerFromEnv ?? providerFromFile ?? "local_encrypted";
|
|
|
|
const storage = loadStorageConfig(fileConfig);
|
|
validateManagedCloudPrerequisites({ ...storage,
|
|
storageS3Bucket: process.env.PAPERCLIP_STORAGE_S3_BUCKET
|
|
?? (configFile?.explicitStorageS3Bucket ? fileConfig?.storage.s3.bucket : undefined),
|
|
databaseUrl: process.env.DATABASE_URL ?? fileDbUrl,
|
|
}, process.env);
|
|
const feedbackExportBackendUrl =
|
|
process.env.PAPERCLIP_FEEDBACK_EXPORT_BACKEND_URL?.trim() ||
|
|
process.env.PAPERCLIP_TELEMETRY_BACKEND_URL?.trim() ||
|
|
undefined;
|
|
const feedbackExportBackendToken =
|
|
process.env.PAPERCLIP_FEEDBACK_EXPORT_BACKEND_TOKEN?.trim() ||
|
|
process.env.PAPERCLIP_TELEMETRY_BACKEND_TOKEN?.trim() ||
|
|
undefined;
|
|
|
|
const deploymentMode = resolveDeploymentMode(fileConfig);
|
|
const strictModeFromEnv = process.env.PAPERCLIP_SECRETS_STRICT_MODE;
|
|
const secretsStrictMode =
|
|
strictModeFromEnv !== undefined
|
|
? strictModeFromEnv === "true"
|
|
: (fileSecrets?.strictMode ?? deploymentMode === "authenticated");
|
|
const deploymentExposureFromEnvRaw = process.env.PAPERCLIP_DEPLOYMENT_EXPOSURE;
|
|
const deploymentExposureFromEnv =
|
|
deploymentExposureFromEnvRaw &&
|
|
DEPLOYMENT_EXPOSURES.includes(deploymentExposureFromEnvRaw as DeploymentExposure)
|
|
? (deploymentExposureFromEnvRaw as DeploymentExposure)
|
|
: null;
|
|
const deploymentExposure: DeploymentExposure =
|
|
deploymentMode === "local_trusted"
|
|
? "private"
|
|
: (deploymentExposureFromEnv ?? fileConfig?.server.exposure ?? "private");
|
|
const bindFromEnvRaw = process.env.PAPERCLIP_BIND;
|
|
const bindFromEnv =
|
|
bindFromEnvRaw && BIND_MODES.includes(bindFromEnvRaw as BindMode)
|
|
? (bindFromEnvRaw as BindMode)
|
|
: null;
|
|
const configuredHost = process.env.HOST ?? fileConfig?.server.host ?? "127.0.0.1";
|
|
const tailnetBindHost = detectTailnetBindHost();
|
|
const bind =
|
|
bindFromEnv ??
|
|
fileConfig?.server.bind ??
|
|
inferBindModeFromHost(configuredHost, { tailnetBindHost });
|
|
const customBindHost = process.env.PAPERCLIP_BIND_HOST ?? fileConfig?.server.customBindHost;
|
|
const authBaseUrlModeFromEnvRaw = process.env.PAPERCLIP_AUTH_BASE_URL_MODE;
|
|
const authBaseUrlModeFromEnv =
|
|
authBaseUrlModeFromEnvRaw &&
|
|
AUTH_BASE_URL_MODES.includes(authBaseUrlModeFromEnvRaw as AuthBaseUrlMode)
|
|
? (authBaseUrlModeFromEnvRaw as AuthBaseUrlMode)
|
|
: null;
|
|
const publicUrlFromEnv = process.env.PAPERCLIP_PUBLIC_URL;
|
|
const configuredAuthPublicBaseUrlRaw = [
|
|
process.env.PAPERCLIP_AUTH_PUBLIC_BASE_URL,
|
|
process.env.BETTER_AUTH_URL,
|
|
process.env.BETTER_AUTH_BASE_URL,
|
|
publicUrlFromEnv,
|
|
fileConfig?.auth?.publicBaseUrl,
|
|
].find((value): value is string => typeof value === "string" && value.trim().length > 0);
|
|
const managedRuntimePublicUrl = process.env.PAPERCLIP_MANAGED_RUNTIME_PUBLIC_URL?.trim() || undefined;
|
|
const authPublicBaseUrlRaw = configuredAuthPublicBaseUrlRaw ?? managedRuntimePublicUrl;
|
|
const authPublicBaseUrl = authPublicBaseUrlRaw?.trim() || undefined;
|
|
const authBaseUrlMode: AuthBaseUrlMode =
|
|
authBaseUrlModeFromEnv ??
|
|
(configuredAuthPublicBaseUrlRaw === undefined && managedRuntimePublicUrl
|
|
? "explicit"
|
|
: fileConfig?.auth?.baseUrlMode ?? (authPublicBaseUrl ? "explicit" : "auto"));
|
|
const disableSignUpFromEnv = process.env.PAPERCLIP_AUTH_DISABLE_SIGN_UP;
|
|
const authDisableSignUp: boolean =
|
|
disableSignUpFromEnv !== undefined
|
|
? disableSignUpFromEnv === "true"
|
|
: (fileConfig?.auth?.disableSignUp ?? false);
|
|
const allowedHostnamesFromEnvRaw = process.env.PAPERCLIP_ALLOWED_HOSTNAMES;
|
|
const allowedHostnamesFromEnv = allowedHostnamesFromEnvRaw
|
|
? allowedHostnamesFromEnvRaw
|
|
.split(",")
|
|
.map((value) => value.trim().toLowerCase())
|
|
.filter((value) => value.length > 0)
|
|
: null;
|
|
const publicUrlHostname = authPublicBaseUrl
|
|
? (() => {
|
|
try {
|
|
return new URL(authPublicBaseUrl).hostname.trim().toLowerCase();
|
|
} catch {
|
|
return null;
|
|
}
|
|
})()
|
|
: null;
|
|
const allowedHostnames = Array.from(
|
|
new Set(
|
|
[
|
|
...(allowedHostnamesFromEnv ?? fileConfig?.server.allowedHostnames ?? []),
|
|
...(publicUrlHostname ? [publicUrlHostname] : []),
|
|
]
|
|
.map((value) => value.trim().toLowerCase())
|
|
.filter(Boolean),
|
|
),
|
|
);
|
|
const companyDeletionEnvRaw = process.env.PAPERCLIP_ENABLE_COMPANY_DELETION;
|
|
const companyDeletionEnabled =
|
|
companyDeletionEnvRaw !== undefined
|
|
? companyDeletionEnvRaw === "true"
|
|
: deploymentMode === "local_trusted";
|
|
const databaseBackupEnabled =
|
|
process.env.PAPERCLIP_DB_BACKUP_ENABLED !== undefined
|
|
? process.env.PAPERCLIP_DB_BACKUP_ENABLED === "true"
|
|
: (fileDatabaseBackup?.enabled ?? true);
|
|
const databaseBackupIntervalMinutes = Math.max(
|
|
1,
|
|
Number(process.env.PAPERCLIP_DB_BACKUP_INTERVAL_MINUTES) ||
|
|
fileDatabaseBackup?.intervalMinutes ||
|
|
60,
|
|
);
|
|
const databaseBackupRetentionDays = Math.max(
|
|
1,
|
|
Number(process.env.PAPERCLIP_DB_BACKUP_RETENTION_DAYS) ||
|
|
fileDatabaseBackup?.retentionDays ||
|
|
7,
|
|
);
|
|
const databaseBackupDir = resolveHomeAwarePath(
|
|
process.env.PAPERCLIP_DB_BACKUP_DIR ??
|
|
fileDatabaseBackup?.dir ??
|
|
resolveDefaultBackupDir(),
|
|
);
|
|
// The terminal-workspace reaper waits this many days after an issue tree
|
|
// becomes terminal before it archives the workspace. A person can reopen the
|
|
// work inside this window. A value of 0 disables the cooldown and restores
|
|
// immediate reaping. A negative or non-numeric value falls back to the
|
|
// default. The day granularity obeys the PAPERCLIP_DB_BACKUP_RETENTION_DAYS
|
|
// precedent above. The default is 1 day: regenerable files are compacted
|
|
// after a few hours anyway, so a longer hold only keeps source checkouts of
|
|
// delivered work, and a recurring task produces one workspace per run.
|
|
const workspaceReaperCooldownDaysEnv =
|
|
process.env.PAPERCLIP_WORKSPACE_REAPER_COOLDOWN_DAYS?.trim();
|
|
const workspaceReaperCooldownDaysRaw = Number(workspaceReaperCooldownDaysEnv);
|
|
const workspaceReaperCooldownDays =
|
|
workspaceReaperCooldownDaysEnv
|
|
&& Number.isFinite(workspaceReaperCooldownDaysRaw)
|
|
&& workspaceReaperCooldownDaysRaw >= 0
|
|
? workspaceReaperCooldownDaysRaw
|
|
: 1;
|
|
// The terminal-workspace reaper removes regenerable files (installed
|
|
// dependencies, build output, tool caches) from a terminal workspace this
|
|
// many hours after its issue tree becomes terminal. The worktree and every
|
|
// source file stay; a resumed task regenerates the files when its workspace
|
|
// is provisioned again. 0 compacts on the next sweep. A negative, empty, or
|
|
// non-numeric value falls back to the default of 1 hour.
|
|
// PAPERCLIP_WORKSPACE_COMPACTION_ENABLED=false turns compaction off.
|
|
const workspaceCompactionDelayHoursEnv =
|
|
process.env.PAPERCLIP_WORKSPACE_COMPACTION_DELAY_HOURS?.trim();
|
|
const workspaceCompactionDelayHoursRaw = Number(workspaceCompactionDelayHoursEnv);
|
|
const workspaceCompactionDelayHours =
|
|
process.env.PAPERCLIP_WORKSPACE_COMPACTION_ENABLED?.trim().toLowerCase() === "false"
|
|
? null
|
|
: workspaceCompactionDelayHoursEnv
|
|
&& Number.isFinite(workspaceCompactionDelayHoursRaw)
|
|
&& workspaceCompactionDelayHoursRaw >= 0
|
|
? workspaceCompactionDelayHoursRaw
|
|
: 1;
|
|
// Skill version retention removes materialized version directories under
|
|
// `skills/<companyId>/__versions__/` that no current version, agent pin, or
|
|
// live run uses. It keeps this many newest versions per skill as a warm
|
|
// cache. A version directory used within the grace window also stays. A
|
|
// removed directory is written again from its version row on next use.
|
|
// Negative, empty, or non-numeric values fall back to the defaults.
|
|
const nonNegativeIntegerEnv = (name: string, fallback: number) => {
|
|
const raw = process.env[name]?.trim();
|
|
const value = Number(raw);
|
|
return raw && Number.isFinite(value) && value >= 0 ? Math.floor(value) : fallback;
|
|
};
|
|
const skillVersionRetentionKeepNewest = nonNegativeIntegerEnv("PAPERCLIP_SKILL_VERSION_RETENTION_KEEP", 0);
|
|
const skillVersionRetentionGraceMinutes = nonNegativeIntegerEnv("PAPERCLIP_SKILL_VERSION_RETENTION_GRACE_MINUTES", 60);
|
|
const nativeHarnessBackupExpirationDays = Math.max(1, nonNegativeIntegerEnv("PAPERCLIP_NATIVE_HARNESS_BACKUP_EXPIRATION_DAYS", 7));
|
|
const nativeRunnerEvidenceRetentionDays = Math.max(1, nonNegativeIntegerEnv("PAPERCLIP_NATIVE_RUNNER_EVIDENCE_RETENTION_DAYS", 7));
|
|
const nativeWorkspaceSyncRetentionDays = Math.max(1, nonNegativeIntegerEnv("PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_DAYS", 7));
|
|
const gitRestorePointRetentionDays = Math.max(1, nonNegativeIntegerEnv("PAPERCLIP_GIT_RESTORE_POINT_RETENTION_DAYS", 90));
|
|
const gitRestorePointTerminalDelayDays = nonNegativeIntegerEnv("PAPERCLIP_GIT_RESTORE_POINT_TERMINAL_DELAY_DAYS", 3);
|
|
const gitRestorePointIdleDays = nonNegativeIntegerEnv("PAPERCLIP_GIT_RESTORE_POINT_IDLE_DAYS", 7);
|
|
const taskDirectoryOrphanGraceDays = Math.max(1, nonNegativeIntegerEnv("PAPERCLIP_TASK_DIRECTORY_ORPHAN_GRACE_DAYS", 7));
|
|
const bindValidationErrors = validateConfiguredBindMode({
|
|
deploymentMode,
|
|
deploymentExposure,
|
|
bind,
|
|
host: configuredHost,
|
|
customBindHost,
|
|
});
|
|
if (bindValidationErrors.length > 0) {
|
|
throw new Error(bindValidationErrors[0]);
|
|
}
|
|
const resolvedBind = resolveRuntimeBind({
|
|
bind,
|
|
host: configuredHost,
|
|
customBindHost,
|
|
tailnetBindHost,
|
|
});
|
|
if (resolvedBind.errors.length > 0) {
|
|
throw new Error(resolvedBind.errors[0]);
|
|
}
|
|
|
|
return {
|
|
deploymentMode,
|
|
deploymentExposure,
|
|
bind: resolvedBind.bind,
|
|
customBindHost: resolvedBind.customBindHost,
|
|
host: resolvedBind.host,
|
|
port: Number(process.env.PORT) || fileConfig?.server.port || 3100,
|
|
allowedHostnames,
|
|
authBaseUrlMode,
|
|
authPublicBaseUrl,
|
|
chatWebhookPublicBaseUrl: parseChatWebhookPublicBaseUrl(
|
|
process.env.PAPERCLIP_CHAT_WEBHOOK_PUBLIC_URL,
|
|
),
|
|
authDisableSignUp,
|
|
databaseMode: fileDatabaseMode,
|
|
databaseUrl: process.env.DATABASE_URL ?? fileDbUrl,
|
|
databaseMigrationUrl: process.env.DATABASE_MIGRATION_URL,
|
|
embeddedPostgresDataDir: resolveHomeAwarePath(
|
|
fileConfig?.database.embeddedPostgresDataDir ?? resolveDefaultEmbeddedPostgresDir(),
|
|
),
|
|
embeddedPostgresPort: fileConfig?.database.embeddedPostgresPort ?? 54329,
|
|
databaseBackupEnabled,
|
|
databaseBackupIntervalMinutes,
|
|
databaseBackupRetentionDays,
|
|
databaseBackupDir,
|
|
gitCacheExpirationEnabled: process.env.PAPERCLIP_GIT_CACHE_EXPIRATION_ENABLED === "true",
|
|
gitBaseExpirationEnabled: process.env.PAPERCLIP_GIT_BASE_EXPIRATION_ENABLED === "true",
|
|
gitCacheExpirationDryRun: process.env.PAPERCLIP_GIT_CACHE_EXPIRATION_DRY_RUN !== "false",
|
|
isolatedSourceRehomeEnabled: process.env.PAPERCLIP_ISOLATED_SOURCE_REHOME_ENABLED === "true",
|
|
isolatedSourceRehomeDryRun: process.env.PAPERCLIP_ISOLATED_SOURCE_REHOME_DRY_RUN === "true",
|
|
nestedCloneMigrationEnabled: process.env.PAPERCLIP_NESTED_CLONE_MIGRATION_ENABLED === "true",
|
|
nestedCloneMigrationDryRun: process.env.PAPERCLIP_NESTED_CLONE_MIGRATION_DRY_RUN === "true",
|
|
workspaceReaperCooldownDays,
|
|
workspaceCompactionDelayHours,
|
|
skillVersionRetentionEnabled: process.env.PAPERCLIP_SKILL_VERSION_RETENTION_ENABLED !== "false",
|
|
skillVersionRetentionDryRun: process.env.PAPERCLIP_SKILL_VERSION_RETENTION_DRY_RUN === "true",
|
|
skillVersionRetentionKeepNewest,
|
|
skillVersionRetentionGraceMinutes,
|
|
nativeHarnessBackupExpirationEnabled: process.env.PAPERCLIP_NATIVE_HARNESS_BACKUP_EXPIRATION_ENABLED !== "false",
|
|
nativeHarnessBackupExpirationDryRun: process.env.PAPERCLIP_NATIVE_HARNESS_BACKUP_EXPIRATION_DRY_RUN === "true",
|
|
nativeHarnessBackupExpirationDays,
|
|
nativeRunnerEvidenceRetentionDays,
|
|
nativeWorkspaceSyncRetentionEnabled: process.env.PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_ENABLED !== "false",
|
|
nativeWorkspaceSyncRetentionDryRun: process.env.PAPERCLIP_NATIVE_WORKSPACE_SYNC_RETENTION_DRY_RUN === "true",
|
|
nativeWorkspaceSyncRetentionDays,
|
|
gitRestorePointRetentionDays,
|
|
gitRestorePointsEnabled: process.env.PAPERCLIP_GIT_RESTORE_POINTS_ENABLED === "true",
|
|
gitRestorePointTerminalDelayDays,
|
|
gitRestorePointIdleDays,
|
|
taskDirectoryRetentionEnabled: process.env.PAPERCLIP_TASK_DIRECTORY_RETENTION_ENABLED === "true",
|
|
taskDirectoryRetentionDryRun: process.env.PAPERCLIP_TASK_DIRECTORY_RETENTION_DRY_RUN !== "false",
|
|
taskDirectoryOrphanGraceDays,
|
|
serveUi:
|
|
process.env.SERVE_UI !== undefined
|
|
? process.env.SERVE_UI === "true"
|
|
: fileConfig?.server.serveUi ?? true,
|
|
uiDevMiddleware: process.env.PAPERCLIP_UI_DEV_MIDDLEWARE === "true",
|
|
secretsProvider,
|
|
secretsStrictMode,
|
|
secretsMasterKeyFilePath:
|
|
resolveHomeAwarePath(
|
|
process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE ??
|
|
fileSecrets?.localEncrypted.keyFilePath ??
|
|
resolveDefaultSecretsKeyFilePath(),
|
|
),
|
|
...storage,
|
|
feedbackExportBackendUrl,
|
|
feedbackExportBackendToken,
|
|
heartbeatSchedulerEnabled: process.env.HEARTBEAT_SCHEDULER_ENABLED !== "false",
|
|
heartbeatSchedulerIntervalMs: Math.max(10000, Number(process.env.HEARTBEAT_SCHEDULER_INTERVAL_MS) || 30000),
|
|
companyDeletionEnabled,
|
|
telemetryEnabled: fileConfig?.telemetry?.enabled ?? true,
|
|
announcementsEnabled: process.env.PAPERCLIP_ANNOUNCEMENTS_ENABLED !== "false",
|
|
announcementsFeedUrl: process.env.PAPERCLIP_ANNOUNCEMENTS_FEED_URL?.trim() || "https://pages.paperclip.ing/announcements/v1/current.json",
|
|
};
|
|
}
|
|
|
|
export type CloudBootstrapConfig = {
|
|
databaseMode: "postgres";
|
|
databaseUrl: string;
|
|
databaseMigrationUrl: string | undefined;
|
|
};
|
|
|
|
export function loadCloudBootstrapConfig(): CloudBootstrapConfig {
|
|
if (!isCloudManagedInstance()) throw new Error("Cloud bootstrap configuration requires a managed instance");
|
|
const url = process.env.DATABASE_URL?.trim();
|
|
if (!url) throw new StartupRefusalError("database-contract-unmet", "Cloud bootstrap requires DATABASE_URL from the environment");
|
|
const migrationUrl = process.env.DATABASE_MIGRATION_URL?.trim() || undefined;
|
|
for (const [name, value] of [["DATABASE_URL", url], ["DATABASE_MIGRATION_URL", migrationUrl]]) {
|
|
if (!value) continue;
|
|
let parsed: URL;
|
|
try { parsed = new URL(value); } catch { throw new Error(`Cloud bootstrap ${name} must use PostgreSQL`); }
|
|
if (!["postgres:", "postgresql:"].includes(parsed.protocol)) throw new Error(`Cloud bootstrap ${name} must use PostgreSQL`);
|
|
}
|
|
validateManagedCloudBootstrapPrerequisites({ databaseUrl: url }, process.env);
|
|
return { databaseMode: "postgres", databaseUrl: url, databaseMigrationUrl: migrationUrl };
|
|
}
|