Files
PaperClipAI/server/src/services/execution-workspace-policy.ts
T
dcac49a4fd feat(workspaces): defer isolated setup until runtime start (#10653)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Isolated workspaces give each task a safe and reproducible checkout.
> - The existing setup cloned the development database before an agent
needed to run the app.
> - This made worktree creation slower and heavier for tasks that never
start a service.
> - Runtime services already use one server start path for heartbeat,
operator, and startup recovery flows.
> - This pull request moves heavy setup to that start path and keeps
worktree creation lean.
> - The benefit is faster isolated workspace creation with the same
reliable runtime setup when a service starts.

## Linked Issues or Issue Description

Related pull request: #10652 covers the initial deferred
database-seeding slice. This pull request supersedes it with end-to-end
runtime provisioning and safe cleanup.

**What existing behavior does this improve?**

This improves isolated worktree creation, runtime service startup, and
isolated instance cleanup.

**Subsystem affected**

Cross-cutting: CLI worktree setup, server runtime orchestration, shared
workspace contracts, and development scripts.

**Current behavior**

Paperclip seeds an isolated development database during worktree
creation. It can also leave an isolated instance directory after
workspace teardown. This work happens even when no runtime service
starts.

**Proposed behavior**

Paperclip creates the worktree with a lean eager setup. It runs an
idempotent runtime provision command before the first managed service
spawn. Concurrent starts share one provision attempt. Teardown removes
the isolated instance safely.

**Reason and benefit**

Many agent tasks only edit and test code. They do not need a running
Paperclip instance. Deferring the database seed reduces workspace
startup cost while preserving automatic setup for tasks that start the
app.

**Breaking changes**

None. The new runtime provision command is optional. Existing workspace
behavior is unchanged when it is absent.

## What Changed

- Split Paperclip worktree setup into a lean eager script and an
idempotent runtime provision script.
- Added `runtimeProvisionCommand` to project, issue, realized workspace,
and persisted workspace contracts.
- Added a per-workspace provision mutex before local service spawn for
heartbeat, operator, and startup recovery flows.
- Added a persisted `provisioning` service state and the
`workspace_runtime_provision` operation phase.
- Kept provision time outside the service readiness timeout and made
failed attempts visible and retryable.
- Reclaimed isolated instance data during safe workspace teardown.
- Serialized deferred database seeding across processes and bound
teardown to the instance root captured in persisted workspace metadata.
- Added tests for config flow, concurrency, retry, no-op behavior,
readiness timing, scripts, CLI commands, and cleanup.
- Documented the eager and runtime provisioning contracts.

## Verification

- `pnpm -r typecheck`
- `pnpm build`
- `pnpm test:run` (server: 3,201 passed; UI: 3,345 passed; the CLI phase
exposed one environment-sensitive AWS doctor assertion because the agent
runtime injects static AWS credentials)
- `env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY pnpm exec vitest
run cli/src/__tests__/secrets.test.ts -t 'passes AWS doctor checks when
non-secret provider config is present'`
- Focused runtime tests cover serialized provisioning, retry after
stderr failure, absent-command no-op behavior, operation logging,
persisted state order, and readiness timeout exclusion.
- Focused CLI and cleanup tests cover concurrent seed serialization,
stale-lock fail-closed behavior, persisted instance ownership, and
rewritten sibling pointers.

## Risks

- A faulty runtime provision script blocks service startup. Paperclip
records stderr, marks the service failed, and retries on the next start.
- Concurrent service requests share an in-process provision attempt,
while the seed command uses an atomic filesystem lock across processes.
A stale lock fails closed and requires an operator to verify no seed is
running before removing it.
- Isolated instance cleanup is destructive. The cleanup service
validates ownership and path containment before removal.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, `gpt-5.6-sol`, with agentic reasoning, tool use, and
code execution. The service does not expose the context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-02 10:37:10 -05:00

351 lines
14 KiB
TypeScript

import type {
ExecutionWorkspaceMode,
ExecutionWorkspaceStrategy,
IssueExecutionWorkspaceSettings,
ProjectExecutionWorkspaceDefaultMode,
ProjectExecutionWorkspacePolicy,
} from "@paperclipai/shared";
import { asString, parseObject } from "../adapters/utils.js";
export type ParsedExecutionWorkspaceMode = Exclude<ExecutionWorkspaceMode, "inherit" | "reuse_existing">;
export const WORKSPACE_WORKTREE_REQUIRES_PROJECT_CODE = "workspace_worktree_requires_project";
export const WORKSPACE_WORKTREE_REQUIRES_PROJECT_REMEDIATION =
"Attach a project to the task, or bind a reusable execution workspace, then retry.";
export const WORKSPACE_WORKTREE_REQUIRES_PROJECT_MESSAGE =
`This task is set to run in an isolated git worktree, but it has no project and no reusable execution workspace to create the worktree from. ${WORKSPACE_WORKTREE_REQUIRES_PROJECT_REMEDIATION}`;
type WorkspaceStrategyType = ExecutionWorkspaceStrategy["type"];
export type UnrunnableWorktreeIssueRef = {
projectId?: string | null;
projectWorkspaceId?: string | null;
executionWorkspaceId?: string | null;
executionWorkspacePreference?: string | null;
};
function cloneRecord(value: Record<string, unknown> | null | undefined): Record<string, unknown> | null {
if (!value) return null;
return { ...value };
}
function parseExecutionWorkspaceStrategy(raw: unknown): ExecutionWorkspaceStrategy | null {
const parsed = parseObject(raw);
const type = asString(parsed.type, "");
if (type !== "project_primary" && type !== "git_worktree" && type !== "adapter_managed" && type !== "cloud_sandbox") {
return null;
}
return {
type,
...(typeof parsed.baseRef === "string" ? { baseRef: parsed.baseRef } : {}),
...(typeof parsed.branchTemplate === "string" ? { branchTemplate: parsed.branchTemplate } : {}),
...(typeof parsed.worktreeParentDir === "string" ? { worktreeParentDir: parsed.worktreeParentDir } : {}),
...(typeof parsed.provisionCommand === "string" ? { provisionCommand: parsed.provisionCommand } : {}),
...(typeof parsed.runtimeProvisionCommand === "string"
? { runtimeProvisionCommand: parsed.runtimeProvisionCommand }
: {}),
...(typeof parsed.teardownCommand === "string" ? { teardownCommand: parsed.teardownCommand } : {}),
};
}
export function resolveEffectiveWorkspaceStrategyType(
mode: ParsedExecutionWorkspaceMode,
config: Record<string, unknown> | null | undefined,
): WorkspaceStrategyType {
const workspaceStrategy = parseObject(config?.workspaceStrategy);
const type = asString(workspaceStrategy.type, "");
if (type === "project_primary" || type === "git_worktree" || type === "adapter_managed" || type === "cloud_sandbox") {
return type;
}
// Default mirrors workspace-runtime.ts realizeExecutionWorkspace: missing type -> "project_primary".
// agent_default is a metadata-only mode that never creates a worktree, so it keeps "adapter_managed".
return mode === "agent_default" ? "adapter_managed" : "project_primary";
}
export function resolvePinnedIssueWorkspaceStrategyType(input: {
mode: ParsedExecutionWorkspaceMode;
issueSettings: IssueExecutionWorkspaceSettings | null;
}): WorkspaceStrategyType {
const strategyType = input.issueSettings?.workspaceStrategy?.type;
if (
strategyType === "project_primary" ||
strategyType === "git_worktree" ||
strategyType === "adapter_managed" ||
strategyType === "cloud_sandbox"
) {
return strategyType;
}
// When no explicit strategy type is set, mirror the runtime default (project_primary for most
// modes; adapter_managed for agent_default). Mode alone never implies git_worktree.
return input.mode === "agent_default" ? "adapter_managed" : "project_primary";
}
export function hasReusableExecutionWorkspaceBinding(issue: UnrunnableWorktreeIssueRef): boolean {
return Boolean(issue.executionWorkspaceId && issue.executionWorkspacePreference === "reuse_existing");
}
export function isUnrunnableWorktreeCombo(input: {
issue: UnrunnableWorktreeIssueRef;
resolvedMode: ParsedExecutionWorkspaceMode;
resolvedStrategy: string | null | undefined;
reusableExecutionWorkspaceAvailable?: boolean | null;
hasResolvablePriorSessionWorkspace?: boolean | null;
}): boolean {
if (input.resolvedMode !== "isolated_workspace" && input.resolvedMode !== "operator_branch") return false;
if (input.resolvedStrategy !== "git_worktree") return false;
if (input.issue.projectId || input.issue.projectWorkspaceId) return false;
const hasReusableWorkspace =
input.reusableExecutionWorkspaceAvailable ?? hasReusableExecutionWorkspaceBinding(input.issue);
if (hasReusableWorkspace) return false;
return input.hasResolvablePriorSessionWorkspace !== true;
}
export function parseProjectExecutionWorkspacePolicy(raw: unknown): ProjectExecutionWorkspacePolicy | null {
const parsed = parseObject(raw);
if (Object.keys(parsed).length === 0) return null;
const enabled = typeof parsed.enabled === "boolean" ? parsed.enabled : false;
const workspaceStrategy = parseExecutionWorkspaceStrategy(parsed.workspaceStrategy);
const defaultMode = asString(parsed.defaultMode, "");
const defaultProjectWorkspaceId =
typeof parsed.defaultProjectWorkspaceId === "string" ? parsed.defaultProjectWorkspaceId : undefined;
const allowIssueOverride =
typeof parsed.allowIssueOverride === "boolean" ? parsed.allowIssueOverride : undefined;
const normalizedDefaultMode = (() => {
if (
defaultMode === "shared_workspace" ||
defaultMode === "isolated_workspace" ||
defaultMode === "operator_branch" ||
defaultMode === "adapter_default"
) {
return defaultMode as ProjectExecutionWorkspaceDefaultMode;
}
if (defaultMode === "project_primary") return "shared_workspace";
if (defaultMode === "isolated") return "isolated_workspace";
return undefined;
})();
return {
enabled,
...(normalizedDefaultMode ? { defaultMode: normalizedDefaultMode } : {}),
...(allowIssueOverride !== undefined ? { allowIssueOverride } : {}),
...(defaultProjectWorkspaceId ? { defaultProjectWorkspaceId } : {}),
...(workspaceStrategy ? { workspaceStrategy } : {}),
...(parsed.workspaceRuntime && typeof parsed.workspaceRuntime === "object" && !Array.isArray(parsed.workspaceRuntime)
? { workspaceRuntime: { ...(parsed.workspaceRuntime as Record<string, unknown>) } }
: {}),
...(parsed.branchPolicy && typeof parsed.branchPolicy === "object" && !Array.isArray(parsed.branchPolicy)
? { branchPolicy: { ...(parsed.branchPolicy as Record<string, unknown>) } }
: {}),
...(parsed.pullRequestPolicy && typeof parsed.pullRequestPolicy === "object" && !Array.isArray(parsed.pullRequestPolicy)
? { pullRequestPolicy: { ...(parsed.pullRequestPolicy as Record<string, unknown>) } }
: {}),
...(parsed.runtimePolicy && typeof parsed.runtimePolicy === "object" && !Array.isArray(parsed.runtimePolicy)
? { runtimePolicy: { ...(parsed.runtimePolicy as Record<string, unknown>) } }
: {}),
...(parsed.cleanupPolicy && typeof parsed.cleanupPolicy === "object" && !Array.isArray(parsed.cleanupPolicy)
? { cleanupPolicy: { ...(parsed.cleanupPolicy as Record<string, unknown>) } }
: {}),
...(parsed.authorizationPolicy && typeof parsed.authorizationPolicy === "object" && !Array.isArray(parsed.authorizationPolicy)
? { authorizationPolicy: { ...(parsed.authorizationPolicy as Record<string, unknown>) } }
: {}),
};
}
export function gateProjectExecutionWorkspacePolicy(
projectPolicy: ProjectExecutionWorkspacePolicy | null,
isolatedWorkspacesEnabled: boolean,
): ProjectExecutionWorkspacePolicy | null {
if (!isolatedWorkspacesEnabled) return null;
return projectPolicy;
}
type ParseIssueExecutionWorkspaceSettingsOptions = {
includeEnvironmentId?: boolean;
};
export function parseIssueExecutionWorkspaceSettings(
raw: unknown,
options: ParseIssueExecutionWorkspaceSettingsOptions = {},
): IssueExecutionWorkspaceSettings | null {
const parsed = parseObject(raw);
if (Object.keys(parsed).length === 0) return null;
const workspaceStrategy = parseExecutionWorkspaceStrategy(parsed.workspaceStrategy);
const mode = asString(parsed.mode, "");
const normalizedMode = (() => {
if (
mode === "inherit" ||
mode === "shared_workspace" ||
mode === "isolated_workspace" ||
mode === "operator_branch" ||
mode === "reuse_existing" ||
mode === "agent_default"
) {
return mode;
}
if (mode === "project_primary") return "shared_workspace";
if (mode === "isolated") return "isolated_workspace";
return "";
})();
const networkEgress = parseObject(parsed.networkEgress);
const allowFqdns = Array.isArray(networkEgress.allowFqdns)
? networkEgress.allowFqdns
.filter((value): value is string => typeof value === "string" && value.trim().length > 0)
.map((value) => value.trim().toLowerCase())
: [];
const allowCidrs = Array.isArray(networkEgress.allowCidrs)
? networkEgress.allowCidrs
.filter((value): value is string => typeof value === "string" && value.trim().length > 0)
.map((value) => value.trim())
: [];
return {
...(normalizedMode
? { mode: normalizedMode as IssueExecutionWorkspaceSettings["mode"] }
: {}),
...(options.includeEnvironmentId && (typeof parsed.environmentId === "string" || parsed.environmentId === null)
? { environmentId: parsed.environmentId }
: {}),
...(workspaceStrategy ? { workspaceStrategy } : {}),
...(parsed.workspaceRuntime && typeof parsed.workspaceRuntime === "object" && !Array.isArray(parsed.workspaceRuntime)
? { workspaceRuntime: { ...(parsed.workspaceRuntime as Record<string, unknown>) } }
: {}),
...(allowFqdns.length > 0 || allowCidrs.length > 0
? { networkEgress: { allowFqdns, allowCidrs } }
: {}),
};
}
export function selectEnvironmentExecutionWorkspaceSettings(
parsedSettings: IssueExecutionWorkspaceSettings | null,
isolatedWorkspacesEnabled: boolean,
): IssueExecutionWorkspaceSettings | null {
if (!parsedSettings) return null;
if (isolatedWorkspacesEnabled) return parsedSettings;
return parsedSettings.networkEgress
? { networkEgress: parsedSettings.networkEgress }
: null;
}
export type ExecutionWorkspaceEnvironmentSource =
| "agent"
| "instance"
| "default";
export type ExecutionWorkspaceEnvironmentResolution = {
environmentId: string;
source: ExecutionWorkspaceEnvironmentSource;
};
export function resolveExecutionWorkspaceEnvironmentId(input: {
agentDefaultEnvironmentId: string | null;
instanceDefaultEnvironmentId: string | null;
localDefaultEnvironmentId: string;
}): ExecutionWorkspaceEnvironmentResolution {
if (input.agentDefaultEnvironmentId) {
return {
environmentId: input.agentDefaultEnvironmentId,
source: "agent",
};
}
if (input.instanceDefaultEnvironmentId) {
return {
environmentId: input.instanceDefaultEnvironmentId,
source: "instance",
};
}
return {
environmentId: input.localDefaultEnvironmentId,
source: "default",
};
}
export function defaultIssueExecutionWorkspaceSettingsForProject(
projectPolicy: ProjectExecutionWorkspacePolicy | null,
): IssueExecutionWorkspaceSettings | null {
if (!projectPolicy?.enabled) return null;
return {
mode:
projectPolicy.defaultMode === "isolated_workspace"
? "isolated_workspace"
: projectPolicy.defaultMode === "operator_branch"
? "operator_branch"
: projectPolicy.defaultMode === "adapter_default"
? "agent_default"
: "shared_workspace",
};
}
export function issueExecutionWorkspaceModeForPersistedWorkspace(
mode: string | null | undefined,
): IssueExecutionWorkspaceSettings["mode"] {
if (mode === null || mode === undefined) {
return "agent_default";
}
if (mode === "isolated_workspace" || mode === "operator_branch" || mode === "shared_workspace") {
return mode;
}
if (mode === "adapter_managed" || mode === "cloud_sandbox") {
return "agent_default";
}
return "shared_workspace";
}
export function resolveExecutionWorkspaceMode(input: {
projectPolicy: ProjectExecutionWorkspacePolicy | null;
issueSettings: IssueExecutionWorkspaceSettings | null;
legacyUseProjectWorkspace: boolean | null;
}): ParsedExecutionWorkspaceMode {
const issueMode = input.issueSettings?.mode;
if (issueMode && issueMode !== "inherit" && issueMode !== "reuse_existing") {
return issueMode;
}
if (input.projectPolicy?.enabled) {
if (input.projectPolicy.defaultMode === "isolated_workspace") return "isolated_workspace";
if (input.projectPolicy.defaultMode === "operator_branch") return "operator_branch";
if (input.projectPolicy.defaultMode === "adapter_default") return "agent_default";
return "shared_workspace";
}
if (input.legacyUseProjectWorkspace === false) {
return "agent_default";
}
return "shared_workspace";
}
export function buildExecutionWorkspaceAdapterConfig(input: {
agentConfig: Record<string, unknown>;
projectPolicy: ProjectExecutionWorkspacePolicy | null;
issueSettings: IssueExecutionWorkspaceSettings | null;
mode: ParsedExecutionWorkspaceMode;
legacyUseProjectWorkspace: boolean | null;
}): Record<string, unknown> {
const nextConfig = { ...input.agentConfig };
const projectHasPolicy = Boolean(input.projectPolicy?.enabled);
const issueHasWorkspaceOverrides = Boolean(
input.issueSettings?.mode ||
input.issueSettings?.workspaceStrategy ||
input.issueSettings?.workspaceRuntime,
);
const hasWorkspaceControl = projectHasPolicy || issueHasWorkspaceOverrides || input.legacyUseProjectWorkspace === false;
if (hasWorkspaceControl) {
if (input.mode === "isolated_workspace") {
const strategy =
input.issueSettings?.workspaceStrategy ??
input.projectPolicy?.workspaceStrategy ??
parseExecutionWorkspaceStrategy(nextConfig.workspaceStrategy) ??
({ type: "git_worktree" } satisfies ExecutionWorkspaceStrategy);
nextConfig.workspaceStrategy = strategy as unknown as Record<string, unknown>;
} else {
delete nextConfig.workspaceStrategy;
}
if (input.mode === "agent_default") {
delete nextConfig.workspaceRuntime;
} else if (input.issueSettings?.workspaceRuntime) {
nextConfig.workspaceRuntime = cloneRecord(input.issueSettings.workspaceRuntime) ?? undefined;
} else if (input.projectPolicy?.workspaceRuntime) {
nextConfig.workspaceRuntime = cloneRecord(input.projectPolicy.workspaceRuntime) ?? undefined;
}
}
return nextConfig;
}