mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-21 00:05:49 +02:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Human messages start agent work through task, chat, and external-channel intake. > - Agent startup can delay the first visible response. > - A small direct model call can acknowledge the request while normal work starts. > - This response must remain separate from evidence that the agent answered or completed work. > - This pull request adds optional company-wide fast responses with durable receipts, access checks, and cost accounting. > - Users receive prompt acknowledgement without changing the agent execution path. ## Linked Issues or Issue Description **Subsystem affected** Cross-cutting: database, shared contracts, server orchestration, connections, and UI. **Problem or motivation** Users can wait for an agent to start before they receive a contextual response. Generic progress messages do not confirm what the user asked. An acknowledgement must not falsely complete work or satisfy an unanswered turn. **Proposed solution** Add a disabled-by-default experimental fast-response model using an existing shared API connection. Generate one short acknowledgement for each accepted human turn. Use the existing agent identity and external publication outboxes. Retain platform provenance and account for the inference independently. **Alternatives considered** Running the acknowledgement through the agent runtime adds startup latency. Restricting implementation to OpenRouter leaves other existing API connections unsupported. This change supports all existing API connections and uses OpenRouter for live verification. **Roadmap alignment** This extends existing agent communication and connection settings. ROADMAP.md and open pull requests were checked. No duplicate fast-response implementation was found. The maintainer supplied the feature plan directly. ## What Changed - Add Experimental fast responses to instance settings. Keep the feature off by default. Enforce the flag at intake, dispatch, and publication. - Move Decision model and Fast response setup to company Settings → Connections when the experiment is enabled. Place Connections below Environments. Separate the model sections with clear headings and a divider. Keep Decision model in General when the experiment is off. Route Costs setup links to the available settings page. - Make the acknowledgement prompt a multiline template string. Ask for a brief, positive response in the user’s language. - Add settings, connection/model selection, sponsorship, fixed-sample testing, and Costs history. - Add a Vercel AI SDK text service for existing native, routed, Bedrock, and local API connections. - Add durable per-turn requests, a bounded worker, a three-second admission deadline, and one inference attempt. - Integrate task creation, task and agent chat, steering, queued follow-ups, experimental board chat, shared chat intake, and email. - Preserve receipt provenance and exclude receipts from completion, recovery, interaction resolution, and agent wakeups. - Recheck authorization, source validity, real replies, budgets, and destination binding before publication. Preserve accounting for suppressed or uncertain outcomes. - Add provider contracts, database race tests, channel outbox tests, UI tests, Stories, and documentation. Keep decision-model execution unchanged. ## Verification - Local `pnpm -r typecheck`, `pnpm build`, and UI token gates pass after rebase. The UI typecheck and build also pass after the final navigation fixes. All 37 focused navigation tests pass. The first full local `pnpm test:run` invocation reported seven chat failures. All seven passed in isolated reruns after fixture updates and host cleanup. This invocation was stopped after the full final-head CI suite passed. - Initial local database tests hit macOS shared-memory exhaustion. A diagnostic confirmed `shmget: No space left on device` with the system limit of 32 IDs. Three abandoned, unattached PostgreSQL IPC headers were reclaimed. The isolated rerun passed all 31 fast-response and channel tests. The two additional Slack regression cases also passed in a separate rerun. No active database was stopped. - UI token gates pass. Provider contract tests cover all supported API connection types. Fast-response tests cover uniqueness, authorization, budgets, expiry, cancellation, restart, real-reply races, and the experimental flag. - The test drive verified that disabling the experiment hides Connections and redirects its direct URL. Enabling it restores the saved OpenRouter setup. After restart on the final commit, the OpenRouter settings sample returned “Sure, I’ll take care of fixing the border around the settings panel.” in 428 ms. It used 260 input and 31 output tokens and recorded $0.0000576 in reported cost. - Earlier live OpenRouter verification produced a receipt in 1.013 seconds for a new task. The real agent then supplied its substantive answer. A separate settings test took 1.34 seconds, used 246 input and 23 output tokens, and recorded $0.0000361. - Live Slack verification on final commit `24016fa525961ebc7a4acffdc5019ce123970549` passed for a fresh initial task and a follow-up in a disposable bot DM. The initial acknowledgement appeared 2.405 seconds after the Slack message and 1.413 seconds after durable acceptance; the follow-up appeared in 2.322 seconds and 1.160 seconds respectively. The recorded generation/settlement durations were 898 ms and 697 ms. The initial call used 291 input / 39 output tokens and cost $0.00006705; the follow-up used 281 / 47 and cost $0.00007035. - The Slack acknowledgements retained the assigned agent identity and `origin: fast_response`, source-turn references, and no fabricated run ID. A status snapshot after the initial receipt showed the task still `in_progress` with its real execution run. The agent then read the CSV fixture and correctly reported two ready items, two blockers, and their owners. Both replies stayed in the originating DM, and no duplicate generic working message followed the delivered contextual acknowledgements. The `new` command started a fresh conversation session without generating a fast response. - An earlier eligible Slack turn generated a response but suppressed publication. Its reported cost was retained, generic progress remained, and the real agent completed the requested result. This is recorded separately from the two delivered acknowledgements; the history did not expose a precise suppression reason. - Slack verification used real signed provider events, the existing intake/publication paths, and a user-approved temporary webhook-only HTTPS tunnel. Subscriptions were limited to direct-message events. The test bot was paused and the tunnel stopped afterward. Live channel/group-thread and GitHub journeys remain unverified; their mocked tests cover intake, destination binding, publication, and continued underlying work. - Greptile reviewed final commit `24016fa525961ebc7a4acffdc5019ce123970549` at 5/5. Both new review findings are fixed and all threads are resolved. [Final-head CI run](https://github.com/paperclipai/paperclip/actions/runs/38060015730) passed. This includes grouped server/chat/workspace tests, serialized suites, runner checks, browser journeys, typecheck, build, release registry, policy, and the canary dry run. - Manual check: enable Experimental fast responses. Open company Settings → Connections. Select a shared API connection and model. Save and use Test response. Submit a task message and confirm that its receipt does not complete the task. ## Risks - The migration adds receipt and configuration tables plus accounting references. It uses idempotent statements and follows the current migration sequence. - A slow provider, authorization change, or real-agent reply can suppress an acknowledgement. Existing progress feedback remains available. - Unknown provider billing retains a reservation until reconciliation. External deliveries with uncertain outcomes must not be blindly repeated. - Live Slack initial-task and follow-up DM journeys pass. Live Slack channel/group-thread and GitHub journeys remain unverified. The PR remains a draft and records these gaps. Final-head repository CI and automated review pass. ## Model Used OpenAI Codex, based on GPT-6, with reasoning, repository tools, code execution, and browser interaction. The runtime does not expose an exact model snapshot ID or context-window size; these are not asserted. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
594 lines
23 KiB
TypeScript
594 lines
23 KiB
TypeScript
import type { Db } from "@paperclipai/db";
|
|
import { companies, instanceSettings } from "@paperclipai/db";
|
|
|
|
/**
|
|
* A `Db` or an open transaction handle — the subset of query builders the
|
|
* settings writes use. Lets `update` run inside a caller's transaction so
|
|
* it commits atomically with a sibling write.
|
|
*/
|
|
type InstanceSettingsTransaction = Parameters<Parameters<Db["transaction"]>[0]>[0];
|
|
export type InstanceSettingsWriteDb = Pick<
|
|
Db | InstanceSettingsTransaction,
|
|
"select" | "insert" | "update"
|
|
>;
|
|
import {
|
|
DEFAULT_FEEDBACK_DATA_SHARING_PREFERENCE,
|
|
DEFAULT_BACKUP_RETENTION,
|
|
PAPERCLIP_CLOUD_MANAGED_BY,
|
|
instanceGeneralSettingsSchema,
|
|
type InstanceGeneralSettings,
|
|
instanceExperimentalSettingsSchema,
|
|
type InstanceExperimentalSettings,
|
|
type InstanceExperimentalSettingsWithManaged,
|
|
type ManagedExperimentalFeatureKey,
|
|
type ManagedSettingMetadata,
|
|
type PatchInstanceGeneralSettings,
|
|
type InstanceSettings,
|
|
type PatchInstanceSettings,
|
|
type PatchInstanceExperimentalSettings,
|
|
} from "@paperclipai/shared";
|
|
import {
|
|
INSTANCE_FEATURE_CATALOG,
|
|
applyOperatorGeneralDefaults,
|
|
stripOperatorGeneralEchoes,
|
|
} from "@paperclipai/shared";
|
|
import { eq } from "drizzle-orm";
|
|
import { getManagedInstanceConfig, type ManagedInstanceConfig } from "./managed-config.js";
|
|
import { getOperatorSettingDefaults } from "./setting-defaults.js";
|
|
|
|
const DEFAULT_SINGLETON_KEY = "default";
|
|
const instanceGeneralSettingsStorageSchema = instanceGeneralSettingsSchema.strip();
|
|
const instanceExperimentalSettingsStorageSchema = instanceExperimentalSettingsSchema.strip();
|
|
const TRUTHY_RUNTIME_ENV_VALUES = new Set(["1", "true", "yes", "on"]);
|
|
|
|
interface InstanceSettingsServiceOptions {
|
|
runtimeEnv?: Record<string, string | undefined>;
|
|
now?: () => Date;
|
|
}
|
|
|
|
type WorktreeRunExecutionSuppressedReason =
|
|
| "not_worktree_runtime"
|
|
| "flag_disabled"
|
|
| "missing_cutoff"
|
|
| "missing_instance_id"
|
|
| "instance_id_mismatch"
|
|
| "settings_read_error";
|
|
|
|
export type WorktreeRunExecutionActivationState =
|
|
| {
|
|
armed: true;
|
|
cutoff: string;
|
|
activationInstanceId: string;
|
|
reason: null;
|
|
}
|
|
| {
|
|
armed: false;
|
|
cutoff: null;
|
|
activationInstanceId: string | null;
|
|
reason: WorktreeRunExecutionSuppressedReason;
|
|
};
|
|
|
|
export function isTruthyRuntimeEnvValue(value: string | undefined) {
|
|
return typeof value === "string" && TRUTHY_RUNTIME_ENV_VALUES.has(value.trim().toLowerCase());
|
|
}
|
|
|
|
function getRuntimeInstanceId(env: Record<string, string | undefined>) {
|
|
const instanceId = env.PAPERCLIP_INSTANCE_ID?.trim();
|
|
return instanceId ? instanceId : null;
|
|
}
|
|
|
|
function stripServerManagedExperimentalPatchFields(
|
|
patch: PatchInstanceExperimentalSettings | Record<string, unknown>,
|
|
): PatchInstanceExperimentalSettings {
|
|
const {
|
|
worktreeRunExecutionActivatedAt: _ignoredActivatedAt,
|
|
worktreeRunExecutionActivationInstanceId: _ignoredActivationInstanceId,
|
|
...patchable
|
|
} = patch as Record<string, unknown>;
|
|
return patchable as PatchInstanceExperimentalSettings;
|
|
}
|
|
|
|
export function applyExperimentalSettingsPatch(
|
|
current: unknown,
|
|
patch: PatchInstanceExperimentalSettings | Record<string, unknown>,
|
|
options: InstanceSettingsServiceOptions = {},
|
|
): InstanceExperimentalSettings {
|
|
const previousExperimental = normalizeExperimentalSettings(current);
|
|
const patchable = stripServerManagedExperimentalPatchFields(patch);
|
|
const nextExperimental = normalizeExperimentalSettings({
|
|
...previousExperimental,
|
|
...patchable,
|
|
});
|
|
const hasWorktreeRunExecutionPatch = Object.prototype.hasOwnProperty.call(
|
|
patchable,
|
|
"enableWorktreeRunExecution",
|
|
);
|
|
|
|
if (!hasWorktreeRunExecutionPatch) {
|
|
return nextExperimental;
|
|
}
|
|
|
|
if (nextExperimental.enableWorktreeRunExecution !== true) {
|
|
return {
|
|
...nextExperimental,
|
|
worktreeRunExecutionActivatedAt: null,
|
|
worktreeRunExecutionActivationInstanceId: null,
|
|
};
|
|
}
|
|
|
|
if (previousExperimental.enableWorktreeRunExecution === true) {
|
|
return nextExperimental;
|
|
}
|
|
|
|
const runtimeEnv = options.runtimeEnv ?? process.env;
|
|
if (!isTruthyRuntimeEnvValue(runtimeEnv.PAPERCLIP_IN_WORKTREE)) {
|
|
return nextExperimental;
|
|
}
|
|
|
|
return {
|
|
...nextExperimental,
|
|
worktreeRunExecutionActivatedAt: (options.now ?? (() => new Date()))().toISOString(),
|
|
worktreeRunExecutionActivationInstanceId: getRuntimeInstanceId(runtimeEnv),
|
|
};
|
|
}
|
|
|
|
function suppressWorktreeRunExecution(
|
|
reason: WorktreeRunExecutionSuppressedReason,
|
|
activationInstanceId: string | null = null,
|
|
): WorktreeRunExecutionActivationState {
|
|
return {
|
|
armed: false,
|
|
cutoff: null,
|
|
activationInstanceId,
|
|
reason,
|
|
};
|
|
}
|
|
|
|
export function resolveWorktreeRunExecutionActivation(
|
|
experimental: InstanceExperimentalSettings,
|
|
currentInstanceId: string | null | undefined,
|
|
): WorktreeRunExecutionActivationState {
|
|
if (experimental.enableWorktreeRunExecution !== true) {
|
|
return suppressWorktreeRunExecution(
|
|
"flag_disabled",
|
|
experimental.worktreeRunExecutionActivationInstanceId,
|
|
);
|
|
}
|
|
if (!experimental.worktreeRunExecutionActivatedAt) {
|
|
return suppressWorktreeRunExecution(
|
|
"missing_cutoff",
|
|
experimental.worktreeRunExecutionActivationInstanceId,
|
|
);
|
|
}
|
|
if (!currentInstanceId) {
|
|
return suppressWorktreeRunExecution(
|
|
"missing_instance_id",
|
|
experimental.worktreeRunExecutionActivationInstanceId,
|
|
);
|
|
}
|
|
if (experimental.worktreeRunExecutionActivationInstanceId !== currentInstanceId) {
|
|
return suppressWorktreeRunExecution(
|
|
"instance_id_mismatch",
|
|
experimental.worktreeRunExecutionActivationInstanceId,
|
|
);
|
|
}
|
|
return {
|
|
armed: true,
|
|
cutoff: experimental.worktreeRunExecutionActivatedAt,
|
|
activationInstanceId: currentInstanceId,
|
|
reason: null,
|
|
};
|
|
}
|
|
|
|
export async function resolveWorktreeRunExecutionActivationState(options: {
|
|
getExperimental: () => Promise<InstanceExperimentalSettings>;
|
|
runtimeEnv?: Record<string, string | undefined>;
|
|
}): Promise<WorktreeRunExecutionActivationState> {
|
|
const runtimeEnv = options.runtimeEnv ?? process.env;
|
|
if (!isTruthyRuntimeEnvValue(runtimeEnv.PAPERCLIP_IN_WORKTREE)) {
|
|
return suppressWorktreeRunExecution("not_worktree_runtime");
|
|
}
|
|
try {
|
|
return resolveWorktreeRunExecutionActivation(
|
|
await options.getExperimental(),
|
|
getRuntimeInstanceId(runtimeEnv),
|
|
);
|
|
} catch {
|
|
return suppressWorktreeRunExecution("settings_read_error");
|
|
}
|
|
}
|
|
|
|
function normalizeGeneralSettings(raw: unknown): InstanceGeneralSettings {
|
|
const parsed = instanceGeneralSettingsStorageSchema.safeParse(raw ?? {});
|
|
if (parsed.success) {
|
|
return {
|
|
censorUsernameInLogs: parsed.data.censorUsernameInLogs ?? false,
|
|
feedbackDataSharingPreference:
|
|
parsed.data.feedbackDataSharingPreference ?? DEFAULT_FEEDBACK_DATA_SHARING_PREFERENCE,
|
|
backupRetention: parsed.data.backupRetention ?? DEFAULT_BACKUP_RETENTION,
|
|
// Absent => unrestricted; only carry through an explicit policy.
|
|
...(parsed.data.executionMode ? { executionMode: parsed.data.executionMode } : {}),
|
|
};
|
|
}
|
|
return {
|
|
censorUsernameInLogs: false,
|
|
feedbackDataSharingPreference: DEFAULT_FEEDBACK_DATA_SHARING_PREFERENCE,
|
|
backupRetention: DEFAULT_BACKUP_RETENTION,
|
|
};
|
|
}
|
|
|
|
export function normalizeExperimentalSettings(raw: unknown): InstanceExperimentalSettings {
|
|
const parsed = instanceExperimentalSettingsStorageSchema.safeParse(raw ?? {});
|
|
if (parsed.success) {
|
|
return {
|
|
enableEnvironments: parsed.data.enableEnvironments ?? false,
|
|
enableNativeRunner: parsed.data.enableNativeRunner ?? true,
|
|
enableAiConnectionRouters: parsed.data.enableAiConnectionRouters ?? false,
|
|
enableManagedSandboxOnly: parsed.data.enableManagedSandboxOnly ?? false,
|
|
enableIsolatedWorkspaces: parsed.data.enableIsolatedWorkspaces ?? false,
|
|
enableIsolatedWorkspacesByDefault: parsed.data.enableIsolatedWorkspacesByDefault ?? false,
|
|
enableStreamlinedLeftNavigation: parsed.data.enableStreamlinedLeftNavigation ?? true,
|
|
enableStreamlinedUi: parsed.data.enableStreamlinedUi ?? true,
|
|
// Apps graduated from Experimental. Ignore historical off values while
|
|
// continuing to accept the compatibility key in stored settings.
|
|
enableApps: true,
|
|
enableMcpAggregators: true,
|
|
enablePublicMcp: parsed.data.enablePublicMcp ?? false,
|
|
enableOpenAiDot: parsed.data.enableOpenAiDot ?? false,
|
|
enableChatConnectors: parsed.data.enableChatConnectors ?? false,
|
|
enableGitHubReviewBots: parsed.data.enableGitHubReviewBots ?? false,
|
|
enableMemoryConnectors: parsed.data.enableMemoryConnectors ?? false,
|
|
enablePipelines: parsed.data.enablePipelines ?? false,
|
|
enableCases: parsed.data.enableCases ?? false,
|
|
enableAgentChat: parsed.data.enableAgentChat ?? false,
|
|
enableCombinedInboxTasks: parsed.data.enableCombinedInboxTasks ?? false,
|
|
enableConferenceRoomChat: parsed.data.enableConferenceRoomChat ?? false,
|
|
enableClassicTaskInterface: parsed.data.enableClassicTaskInterface ?? false,
|
|
enableIssuePlanDecompositions: parsed.data.enableIssuePlanDecompositions ?? false,
|
|
enableExperimentalFileViewer: parsed.data.enableExperimentalFileViewer ?? false,
|
|
enableExternalObjects: parsed.data.enableExternalObjects ?? false,
|
|
enableSmokeLab: parsed.data.enableSmokeLab ?? false,
|
|
enableBuiltInAgents: parsed.data.enableBuiltInAgents ?? false,
|
|
enableBetaSkills: parsed.data.enableBetaSkills ?? false,
|
|
enableSummaries: parsed.data.enableSummaries ?? false,
|
|
enableStatusCards: parsed.data.enableStatusCards ?? false,
|
|
enableDecisions: parsed.data.enableDecisions ?? false,
|
|
enableFastResponses: parsed.data.enableFastResponses ?? false,
|
|
enableGoalsSidebarLink: parsed.data.enableGoalsSidebarLink ?? false,
|
|
enableServerInfoDebugView: parsed.data.enableServerInfoDebugView ?? false,
|
|
enablePaperclipDeveloperMode: parsed.data.enablePaperclipDeveloperMode ?? false,
|
|
enableSimplifiedEnglishInteractions: parsed.data.enableSimplifiedEnglishInteractions ?? false,
|
|
enableFirstTaskPlanProposal: parsed.data.enableFirstTaskPlanProposal ?? false,
|
|
autoRestartDevServerWhenIdle: parsed.data.autoRestartDevServerWhenIdle ?? false,
|
|
enableWorkspaceBranchReconcileForward: parsed.data.enableWorkspaceBranchReconcileForward ?? true,
|
|
enableWorkspaceDirtyQuarantineRepair: parsed.data.enableWorkspaceDirtyQuarantineRepair ?? true,
|
|
enableOwnerInstanceAdmin: parsed.data.enableOwnerInstanceAdmin ?? false,
|
|
enableSandboxDuplexBridge: parsed.data.enableSandboxDuplexBridge ?? false,
|
|
enableRunnerPreviewIngress: parsed.data.enableRunnerPreviewIngress ?? false,
|
|
enableWorktreeRunExecution: parsed.data.enableWorktreeRunExecution ?? false,
|
|
worktreeRunExecutionActivatedAt: parsed.data.worktreeRunExecutionActivatedAt ?? null,
|
|
worktreeRunExecutionActivationInstanceId:
|
|
parsed.data.worktreeRunExecutionActivationInstanceId ?? null,
|
|
};
|
|
}
|
|
return {
|
|
enableEnvironments: false,
|
|
enableNativeRunner: true,
|
|
enableAiConnectionRouters: false,
|
|
enableManagedSandboxOnly: false,
|
|
enableIsolatedWorkspaces: false,
|
|
enableIsolatedWorkspacesByDefault: false,
|
|
enableStreamlinedLeftNavigation: true,
|
|
enableStreamlinedUi: true,
|
|
enableApps: true,
|
|
enableMcpAggregators: true,
|
|
enablePublicMcp: false,
|
|
enableOpenAiDot: false,
|
|
enableChatConnectors: false,
|
|
enableGitHubReviewBots: false,
|
|
enableMemoryConnectors: false,
|
|
enablePipelines: false,
|
|
enableCases: false,
|
|
enableAgentChat: false,
|
|
enableCombinedInboxTasks: false,
|
|
enableConferenceRoomChat: false,
|
|
enableClassicTaskInterface: false,
|
|
enableIssuePlanDecompositions: false,
|
|
enableExperimentalFileViewer: false,
|
|
enableExternalObjects: false,
|
|
enableSmokeLab: false,
|
|
enableBuiltInAgents: false,
|
|
enableBetaSkills: false,
|
|
enableSummaries: false,
|
|
enableStatusCards: false,
|
|
enableDecisions: false,
|
|
enableFastResponses: false,
|
|
enableGoalsSidebarLink: false,
|
|
enableServerInfoDebugView: false,
|
|
enablePaperclipDeveloperMode: false,
|
|
enableSimplifiedEnglishInteractions: false,
|
|
enableFirstTaskPlanProposal: false,
|
|
autoRestartDevServerWhenIdle: false,
|
|
enableWorkspaceBranchReconcileForward: true,
|
|
enableWorkspaceDirtyQuarantineRepair: true,
|
|
enableOwnerInstanceAdmin: false,
|
|
enableSandboxDuplexBridge: false,
|
|
enableRunnerPreviewIngress: false,
|
|
enableWorktreeRunExecution: false,
|
|
worktreeRunExecutionActivatedAt: null,
|
|
worktreeRunExecutionActivationInstanceId: null,
|
|
};
|
|
}
|
|
|
|
export type ManagedExperimentalKeyMetadata = Partial<
|
|
Record<ManagedExperimentalFeatureKey, ManagedSettingMetadata>
|
|
>;
|
|
|
|
/**
|
|
* Overlay the cloud managed-config feature values over normalized settings.
|
|
*
|
|
* Read-time precedence: code floor (cloud) > managed overlay > tenant DB
|
|
* value > schema default. (No code floors are expressed as flags today —
|
|
* floors are enforced in code at the guarded routes, independent of any
|
|
* flag value.) The overlay is deliberately never persisted: it re-asserts on
|
|
* every read, so a DB restore or manual row edit cannot resurrect a
|
|
* capability the harness has disabled.
|
|
*/
|
|
export function applyManagedExperimentalOverlay(
|
|
experimental: InstanceExperimentalSettings,
|
|
managedConfig: ManagedInstanceConfig | null,
|
|
): { experimental: InstanceExperimentalSettings; managedKeys: ManagedExperimentalKeyMetadata } {
|
|
if (!managedConfig) return { experimental, managedKeys: {} };
|
|
const next: InstanceExperimentalSettings = { ...experimental };
|
|
const managedKeys: ManagedExperimentalKeyMetadata = {};
|
|
for (const [key, value] of Object.entries(managedConfig.features) as Array<
|
|
[ManagedExperimentalFeatureKey, boolean]
|
|
>) {
|
|
// Existing Cloud stack configs may still carry retired flags. Accept the
|
|
// document during rollout, but never let retired flags disable Apps or MCP aggregators.
|
|
if (key === "enableApps" || key === "enableMcpAggregators") continue;
|
|
next[key] = value;
|
|
managedKeys[key] = { managed: true, managedBy: PAPERCLIP_CLOUD_MANAGED_BY };
|
|
}
|
|
return { experimental: next, managedKeys };
|
|
}
|
|
|
|
/**
|
|
* Keep self-hosted-only defaults out of Cloud.
|
|
*
|
|
* The experimental schema carries one default per flag, and the feature
|
|
* catalog pins it to `selfHostedDefault`. A flag that is on by default for
|
|
* self-hosted but off by default for Cloud (`selfHostedDefault: true`,
|
|
* `cloudDefault: false`) would therefore normalize to "on" for a managed
|
|
* instance whose tenant row and managed overlay both leave it unset. Re-assert
|
|
* the declared Cloud default for exactly those flags. An explicit tenant value
|
|
* or a managed feature value still wins (the overlay is applied afterwards).
|
|
*/
|
|
export function applyCloudCatalogDefaults(
|
|
experimental: InstanceExperimentalSettings,
|
|
rawStored: unknown,
|
|
managedConfig: ManagedInstanceConfig | null,
|
|
): InstanceExperimentalSettings {
|
|
if (!managedConfig) return experimental;
|
|
const stored =
|
|
rawStored && typeof rawStored === "object" && !Array.isArray(rawStored)
|
|
? (rawStored as Record<string, unknown>)
|
|
: {};
|
|
const next: InstanceExperimentalSettings = { ...experimental };
|
|
for (const [key, entry] of Object.entries(INSTANCE_FEATURE_CATALOG)) {
|
|
if (entry.cloudDefault !== false || entry.selfHostedDefault !== true) continue;
|
|
if (typeof stored[key] === "boolean") continue;
|
|
if (typeof managedConfig.features[key as ManagedExperimentalFeatureKey] === "boolean") continue;
|
|
(next as unknown as Record<string, unknown>)[key] = false;
|
|
}
|
|
return next;
|
|
}
|
|
|
|
/**
|
|
* Keep the write path from freezing a self-hosted default into a Cloud row.
|
|
*
|
|
* `updateExperimental` persists the whole normalized object, and the schema
|
|
* normalizes an omitted flag to its self-hosted default. Without this step an
|
|
* unrelated experimental write (say, turning on pipelines) would store
|
|
* `enableNativeRunner: true` on a managed instance whose tenant row had never
|
|
* mentioned the flag; every later read would then treat the stored boolean as
|
|
* an explicit tenant choice and stop re-asserting the Cloud default.
|
|
*
|
|
* For each guarded flag (see `applyCloudCatalogDefaults`), the stored key is
|
|
* left absent unless the tenant already stored a boolean or this patch sets
|
|
* the flag to something other than the Cloud default. A patch value equal to
|
|
* the Cloud default is a full-GET echo of the read-time overlay, not a
|
|
* choice, and is stripped the same way `stripOperatorGeneralEchoes` treats
|
|
* operator defaults. Self-hosted rows are returned untouched.
|
|
*/
|
|
export function stripCloudCatalogDefaultEchoes(
|
|
rawStored: unknown,
|
|
patch: PatchInstanceExperimentalSettings | Record<string, unknown>,
|
|
next: InstanceExperimentalSettings,
|
|
managedConfig: ManagedInstanceConfig | null,
|
|
): Partial<InstanceExperimentalSettings> {
|
|
if (!managedConfig) return next;
|
|
const stored =
|
|
rawStored && typeof rawStored === "object" && !Array.isArray(rawStored)
|
|
? (rawStored as Record<string, unknown>)
|
|
: {};
|
|
const patchRecord = patch as Record<string, unknown>;
|
|
const result: Record<string, unknown> = { ...next };
|
|
for (const [key, entry] of Object.entries(INSTANCE_FEATURE_CATALOG)) {
|
|
if (entry.cloudDefault !== false || entry.selfHostedDefault !== true) continue;
|
|
if (typeof stored[key] === "boolean") continue;
|
|
if (
|
|
Object.prototype.hasOwnProperty.call(patchRecord, key) &&
|
|
typeof patchRecord[key] === "boolean" &&
|
|
patchRecord[key] !== entry.cloudDefault
|
|
) {
|
|
continue;
|
|
}
|
|
delete result[key];
|
|
}
|
|
return result as Partial<InstanceExperimentalSettings>;
|
|
}
|
|
|
|
export function instanceSettingsService(db: Db, options: InstanceSettingsServiceOptions = {}) {
|
|
// Fail closed: a malformed PAPERCLIP_MANAGED_CONFIG throws here (and at
|
|
// boot in index.ts) rather than silently running without the overlay.
|
|
const managedConfig = getManagedInstanceConfig(options.runtimeEnv ?? process.env);
|
|
// Same posture for PAPERCLIP_SETTING_DEFAULTS: parsed once, applied per
|
|
// read, never persisted (see applyOperatorGeneralDefaults) — including on
|
|
// the write path, where a full-GET echo of the overlaid value is stripped
|
|
// back to the schema default (see stripOperatorGeneralEchoes).
|
|
const operatorDefaults = getOperatorSettingDefaults(options.runtimeEnv ?? process.env);
|
|
|
|
function toGeneralView(raw: unknown): InstanceGeneralSettings {
|
|
return applyOperatorGeneralDefaults(normalizeGeneralSettings(raw), operatorDefaults);
|
|
}
|
|
|
|
function toExperimentalView(raw: unknown): InstanceExperimentalSettingsWithManaged {
|
|
const { experimental, managedKeys } = applyManagedExperimentalOverlay(
|
|
applyCloudCatalogDefaults(normalizeExperimentalSettings(raw), raw, managedConfig),
|
|
managedConfig,
|
|
);
|
|
// Self-hosted responses stay byte-identical: no managedKeys field at all.
|
|
return managedConfig ? { ...experimental, managedKeys } : experimental;
|
|
}
|
|
|
|
function toInstanceSettings(row: typeof instanceSettings.$inferSelect): InstanceSettings {
|
|
return {
|
|
id: row.id,
|
|
defaultEnvironmentId: row.defaultEnvironmentId ?? null,
|
|
general: toGeneralView(row.general),
|
|
experimental: toExperimentalView(row.experimental),
|
|
createdAt: row.createdAt,
|
|
updatedAt: row.updatedAt,
|
|
} as InstanceSettings;
|
|
}
|
|
async function getOrCreateRow(runner: InstanceSettingsWriteDb = db) {
|
|
const existing = await runner
|
|
.select()
|
|
.from(instanceSettings)
|
|
.where(eq(instanceSettings.singletonKey, DEFAULT_SINGLETON_KEY))
|
|
.then((rows) => rows[0] ?? null);
|
|
if (existing) return existing;
|
|
|
|
const now = new Date();
|
|
const [created] = await runner
|
|
.insert(instanceSettings)
|
|
.values({
|
|
singletonKey: DEFAULT_SINGLETON_KEY,
|
|
general: {},
|
|
experimental: {},
|
|
createdAt: now,
|
|
updatedAt: now,
|
|
})
|
|
.onConflictDoUpdate({
|
|
target: [instanceSettings.singletonKey],
|
|
set: {
|
|
updatedAt: now,
|
|
},
|
|
})
|
|
.returning();
|
|
|
|
if (created) return created;
|
|
|
|
const raced = await runner
|
|
.select()
|
|
.from(instanceSettings)
|
|
.where(eq(instanceSettings.singletonKey, DEFAULT_SINGLETON_KEY))
|
|
.then((rows) => rows[0] ?? null);
|
|
if (raced) return raced;
|
|
|
|
throw new Error("Failed to initialize instance settings row");
|
|
}
|
|
|
|
return {
|
|
get: async (): Promise<InstanceSettings> => toInstanceSettings(await getOrCreateRow()),
|
|
|
|
update: async (
|
|
patch: PatchInstanceSettings,
|
|
writeOptions?: { db?: InstanceSettingsWriteDb },
|
|
): Promise<InstanceSettings> => {
|
|
// The write may run inside a caller-supplied transaction so it commits
|
|
// atomically with a sibling write (e.g. clearing the managed-default
|
|
// stamp on the environment row alongside a defaultEnvironmentId
|
|
// change). Reads use the same runner so the row is visible to the tx.
|
|
const runner = writeOptions?.db ?? db;
|
|
const current = await getOrCreateRow(runner);
|
|
const now = new Date();
|
|
const [updated] = await runner
|
|
.update(instanceSettings)
|
|
.set({
|
|
...(Object.prototype.hasOwnProperty.call(patch, "defaultEnvironmentId")
|
|
? { defaultEnvironmentId: patch.defaultEnvironmentId ?? null }
|
|
: {}),
|
|
updatedAt: now,
|
|
})
|
|
.where(eq(instanceSettings.id, current.id))
|
|
.returning();
|
|
return toInstanceSettings(updated ?? current);
|
|
},
|
|
|
|
getGeneral: async (
|
|
readOptions?: { db?: InstanceSettingsWriteDb },
|
|
): Promise<InstanceGeneralSettings> => {
|
|
const row = await getOrCreateRow(readOptions?.db);
|
|
return toGeneralView(row.general);
|
|
},
|
|
|
|
getExperimental: async (): Promise<InstanceExperimentalSettingsWithManaged> => {
|
|
const row = await getOrCreateRow();
|
|
return toExperimentalView(row.experimental);
|
|
},
|
|
|
|
updateGeneral: async (patch: PatchInstanceGeneralSettings): Promise<InstanceSettings> => {
|
|
const current = await getOrCreateRow();
|
|
const storedGeneral = normalizeGeneralSettings(current.general);
|
|
// A full-GET echo carries the overlaid operator value for a field the
|
|
// user never chose; stripping it keeps the overlay strictly read-time,
|
|
// so changing or unsetting the variable later still takes effect.
|
|
const nextGeneral = stripOperatorGeneralEchoes(
|
|
storedGeneral,
|
|
normalizeGeneralSettings({ ...storedGeneral, ...patch }),
|
|
operatorDefaults,
|
|
);
|
|
const now = new Date();
|
|
const [updated] = await db
|
|
.update(instanceSettings)
|
|
.set({
|
|
general: { ...nextGeneral },
|
|
updatedAt: now,
|
|
})
|
|
.where(eq(instanceSettings.id, current.id))
|
|
.returning();
|
|
return toInstanceSettings(updated ?? current);
|
|
},
|
|
|
|
updateExperimental: async (patch: PatchInstanceExperimentalSettings): Promise<InstanceSettings> => {
|
|
const current = await getOrCreateRow();
|
|
// Guarded Cloud flags stay absent from the row unless chosen, so the
|
|
// read-time catalog default keeps applying (see stripCloudCatalogDefaultEchoes).
|
|
const nextExperimental = stripCloudCatalogDefaultEchoes(
|
|
current.experimental,
|
|
patch,
|
|
applyExperimentalSettingsPatch(current.experimental, patch, options),
|
|
managedConfig,
|
|
);
|
|
const now = new Date();
|
|
const [updated] = await db
|
|
.update(instanceSettings)
|
|
.set({
|
|
experimental: { ...nextExperimental },
|
|
updatedAt: now,
|
|
})
|
|
.where(eq(instanceSettings.id, current.id))
|
|
.returning();
|
|
return toInstanceSettings(updated ?? current);
|
|
},
|
|
|
|
listCompanyIds: async (): Promise<string[]> =>
|
|
db
|
|
.select({ id: companies.id })
|
|
.from(companies)
|
|
.then((rows) => rows.map((row) => row.id)),
|
|
};
|
|
}
|