refactor(heartbeat): extract workspace preparation (#15807)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The heartbeat service prepares the workspace and dispatches each
agent run.
> - After the runtime extraction, `heartbeat.ts` still had 6,417 lines.
> - Workspace setup occupied one contiguous 857-line block inside
`executeRun`.
> - This pull request moves that block into one workspace preparation
module.
> - The executor keeps dispatch gates and terminal cleanup. Reviewers
can inspect workspace setup as one phase.

## Linked Issues or Issue Description

Refs #15803 and #15578. This continues the heartbeat refactor series.
The search found no open duplicate workspace preparation extraction.

**What existing behavior does this improve?**

It makes heartbeat workspace preparation easier to maintain. It does not
change the workspace policy or execution behavior.

**Current behavior**

`executeRun` contains workspace resolution, freshness decisions,
provisioning, persistence, issue binding, environment acquisition, and
realization inline.

**Proposed behavior**

`prepareHeartbeatWorkspace` owns that sequence in
`server/src/services/heartbeat/workspace-preparation.ts`. `heartbeat.ts`
calls it at the same point and uses its returned workspace, lease,
target, and metadata.

**Reason and benefit**

The main file falls from 6,417 to 5,597 lines. The workspace phase has a
typed boundary and direct regression tests.

**Breaking changes**

None. There are no schema, API, feature-flag, or execution-policy
changes.

## What Changed

- Extract the 857-line workspace preparation block without changing its
operations or their order.
- Preserve native recovery bindings, workspace persistence failure
cleanup, issue rebinding after realization, and the cloned restore
lease.
- Keep environment selection, dispatch admission, terminalization, and
final lease release in `heartbeat.ts`.
- Add 12 boundary tests with disposable PostgreSQL, real Git
repositories, and real local environment leases.
- Make the existing Stop and plan-only recovery tests await executor
completion instead of short polling deadlines. Assert exactly one
continuation for the specific plan-only source run.
- Keep server-only Node preload hooks out of the deterministic process
adapter used by Stop browser acceptance.
- Refresh workspace recovery browser acceptance for the current task
composer, and give its disposable test-drive an explicit 10-second
snapshot deadline. Production snapshot settings stay unchanged.
- Document the workspace preparation boundary in `doc/DEVELOPING.md`.

## Verification

- AST comparison: all 857 moved lines match the original block. The
remaining module body differs only at the extraction call.
- `pnpm build`: passed.
- `pnpm -r typecheck`: passed.
- Focused regression coverage: 651 distinct cases across eight suites
passed across targeted runs, including all 12 new boundary tests. The
complete 400-test recovery suite passed together on the final commit.
- Workspace bootstrap browser acceptance: both journeys passed (4.2
minutes), including preserved source edits, ignored-file exclusion,
bounded retries, and persistence across reload.
- Stop/queue browser acceptance: both process and native runner journeys
passed (2.2 minutes), covering queueing, Stop, subtree pause/cancel,
resume, and unrelated-task isolation.
- [Full repository
CI](https://github.com/paperclipai/paperclip/actions/runs/38062408589):
all 47 jobs passed, including the general and serialized test suites,
eight browser shards, build, typecheck, native runner checks, and canary
dry run.
- The local `pnpm test:run` was interrupted to prioritize the recovery
regression; the complete repository suite was verified in CI.
- Greptile: 5/5 on `c28e698b885cf5fd12f36f7b4052e7a2e6f879d7`, with no
open review threads.

## Risks

- This code writes durable workspace bindings and acquires environment
leases. A changed operation order could break recovery or cleanup after
partial setup. The extraction preserves the original statements and
exception boundaries.
- The new tests cover creation, reuse, immutable recovery bindings,
persistence failure, acquisition and realization failure, controller
ownership loss, isolation, execution policy, and lease snapshot
independence.
- Browser acceptance uses real application services with deterministic
local workers and a fake Codex protocol provider. It does not qualify
live cloud providers.

## Model Used

OpenAI GPT-6 through Codex, with code editing, shell execution, and test
tools. The exact serving model identifier and context-window size are
not exposed in this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-10-10 10:30:17 -05:00
1 parent 19e9facc5b
commit 07fee85eaf
9 files changed
+1445 -917

No files matched your search

+8
View File
@@ -1360,6 +1360,14 @@ lifecycle and ownership callbacks update its teardown state immediately, includi
when preparation later throws. Public sandbox lifecycle and model fallback helpers
remain re-exported by `heartbeat.ts`.
Workspace preparation is in `server/src/services/heartbeat/workspace-preparation.ts`.
`prepareHeartbeatWorkspace` resolves and provisions the workspace, persists its
metadata and issue binding, then acquires and realizes the execution environment.
It preserves the immutable workspace binding used by native recovery and records
preparation spans even when acquisition or realization fails. The executor retains
environment selection, dispatch gates, and terminal lease cleanup. Persistence
failures still clean newly created workspace artifacts before propagating.
Runtime execution is in `server/src/services/heartbeat/runtime-execution.ts`.
`executeHeartbeatRuntime` dispatches a prepared native session or legacy adapter,
captures usage, finalizes workspace operations, and releases gateway tokens and
@@ -8455,12 +8455,10 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
.set({ status: "paused" })
.where(eq(agents.id, agentId));
release();
if (next!)
await vi.waitFor(async () =>
expect((await heartbeat.getRun(next!.id))?.status).not.toBe(
"running",
),
);
// Wait for the released executor and its cleanup, not the default
// one-second polling window. The next case must not inherit live work.
await heartbeat.drainActiveRunExecutions();
if (next!) expect((await heartbeat.getRun(next!.id))?.status).not.toBe("running");
}
},
);
@@ -12100,7 +12098,9 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
const legacyReviewBefore = withLegacyReview
? await db.select().from(issues).where(eq(issues.id, legacyReviewId))
: [];
mockAdapterExecute.mockImplementationOnce(async () => {
let planOnlyRunId = "";
mockAdapterExecute.mockImplementationOnce(async (input?: unknown) => {
planOnlyRunId = (input as { runId: string }).runId;
if (withLegacyReview) {
// These pre-dispatch cancellations used to satisfy both the no-comment
// and churn thresholds and suppress an otherwise valid continuation.
@@ -12124,17 +12124,23 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
await heartbeat.reconcileStrandedAssignedIssues();
await heartbeat.promoteDueScheduledRetries(new Date(Date.now() + 31_000));
await heartbeat.resumeQueuedRuns();
// The continuation is produced by completion; wait for that durable boundary.
await heartbeat.drainActiveRunExecutions();
const livenessWake = await waitForValue(async () => {
expect(planOnlyRunId).not.toBe("");
const livenessWakes = await waitForValue(async () => {
const rows = await db
.select()
.from(agentWakeupRequests)
.where(eq(agentWakeupRequests.agentId, agentId));
return (
rows.find((row) => row.reason === "issue_disposition_repair") ?? null
);
// Follow-up repairs can already exist. Match this plan-only result,
// rather than relying on the database's unordered row order.
const matching = rows.filter((row) => row.reason === "issue_disposition_repair"
&& row.payload?.retryOfRunId === planOnlyRunId);
return matching.length > 0 ? matching : null;
});
expect(livenessWake).toBeTruthy();
expect(livenessWakes).toHaveLength(1);
const livenessWake = livenessWakes![0]!;
expect(livenessWake?.payload).toMatchObject({
issueId,
dispositionRepairAttempt: 1,
@@ -12143,7 +12149,7 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
const sourceRunId = (
livenessWake?.payload as Record<string, unknown> | null
)?.retryOfRunId;
expect(sourceRunId).toBeTruthy();
expect(sourceRunId).toBe(planOnlyRunId);
const sourceRun = await db
.select()
.from(heartbeatRuns)
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,236 @@
import { execFile as execFileCallback } from "node:child_process";
import { mkdtemp, mkdir, readFile, rm, writeFile, access } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { promisify } from "node:util";
import { agents, companies, createDb, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS, environmentLeases, heartbeatRuns, issues, projects, projectWorkspaces, type Db } from "@paperclipai/db";
import { eq, sql } from "drizzle-orm";
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
import { environmentRunOrchestrator } from "../environment-run-orchestrator.js";
import { environmentService } from "../environments.js";
import { executionWorkspaceService } from "../execution-workspaces.js";
import { instanceSettingsService } from "../instance-settings.js";
import { issueService } from "../issues.js";
import { workspaceOperationService } from "../workspace-operations.js";
import { createHeartbeatWorkspaceResolver, resolveExecutionWorkspaceReuseRequestForIssue } from "./workspaces.js";
import { prepareHeartbeatWorkspace, type HeartbeatWorkspacePreparationInput } from "./workspace-preparation.js";
const execFile = promisify(execFileCallback);
const support = await getEmbeddedPostgresTestSupport();
describe.skipIf(!support.supported)("heartbeat workspace preparation boundary", () => {
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
let db: Db;
let home: string;
beforeAll(async () => {
home = await mkdtemp(join(tmpdir(), "paperclip-workspace-preparation-"));
vi.stubEnv("PAPERCLIP_HOME", home);
database = await startEmbeddedPostgresTestDatabase("paperclip-workspace-preparation");
db = createDb(database.connectionString);
await db.execute(sql`set client_min_messages = warning`);
}, EMBEDDED_POSTGRES_TEST_TIMEOUT_MS);
afterAll(async () => {
await db?.$client.end();
await database?.cleanup();
vi.unstubAllEnvs();
if (home) await rm(home, { recursive: true, force: true });
});
beforeEach(async () => {
vi.restoreAllMocks();
await db.execute(sql`truncate table companies restart identity cascade`);
await instanceSettingsService(db).updateExperimental({ enableIsolatedWorkspaces: true });
});
async function fixture(isolated = false) {
const [company] = await db.insert(companies).values({ name: "Workspace preparation", issuePrefix: "WSP" }).returning();
const [agent] = await db.insert(agents).values({ companyId: company.id, name: "Workspace agent", status: "running", adapterType: "process" }).returning();
const [project] = await db.insert(projects).values({ companyId: company.id, name: "Source project" }).returning();
const source = join(home, project.id);
await mkdir(source);
await execFile("git", ["init", "--initial-branch=main", source]);
await execFile("git", ["-C", source, "config", "user.email", "test@example.com"]);
await execFile("git", ["-C", source, "config", "user.name", "Workspace test"]);
await writeFile(join(source, "README.md"), "source content\n");
await execFile("git", ["-C", source, "add", "."]);
await execFile("git", ["-C", source, "commit", "-m", "Source"]);
const [projectWorkspace] = await db.insert(projectWorkspaces).values({ companyId: company.id, projectId: project.id, name: "Source", cwd: source, isPrimary: true }).returning();
const [issue] = await db.insert(issues).values({ companyId: company.id, title: "Prepare this workspace", identifier: "WSP-1", projectId: project.id, projectWorkspaceId: projectWorkspace.id, assigneeAgentId: agent.id }).returning();
const [run] = await db.insert(heartbeatRuns).values({ companyId: company.id, agentId: agent.id, issueId: issue.id, scopeKind: "issue", status: "running", runtimeMode: "legacy" }).returning();
const localEnvironment = await environmentService(db).ensureLocalEnvironment(company.id);
const envOrchestrator = environmentRunOrchestrator(db);
const resolver = createHeartbeatWorkspaceResolver(db);
const input: HeartbeatWorkspacePreparationInput = {
run, agent,
task: { issueRef: issue, issueId: issue.id, context: { issueId: issue.id, projectId: project.id }, executionProjectId: project.id, responsibleUserId: null, previousSessionParams: null },
policy: {
trustPreset: { kind: "standard", preset: "standard", boundary: null, sourcePresets: {} },
isolatedWorkspacesEnabled: true,
effectiveExecutionWorkspaceMode: isolated ? "isolated_workspace" : "shared_workspace",
requestedExecutionWorkspaceMode: isolated ? "isolated_workspace" : "shared_workspace",
useIsolatedTaskDirectory: false, nativeChatWorkspaceScope: null,
projectExecutionWorkspacePolicy: null, issueExecutionWorkspaceSettings: null, environmentExecutionWorkspaceSettings: null,
executionPolicy: { executionMode: "any" },
},
environment: { selectedEnvironmentId: localEnvironment.id, localEnvironment, selectedEnvironmentForConfig: localEnvironment, environmentResolution: { environmentId: localEnvironment.id, source: "default" }, resolvedInstanceSettings: await instanceSettingsService(db).get() },
config: { mergedConfig: isolated ? { workspaceStrategy: { type: "git_worktree", baseRef: "main", branchTemplate: "test-{{issue.identifier}}" } } : {}, configSnapshot: null, secretManifest: [] },
reuse: { requestedShouldReuseExisting: false, existingExecutionWorkspace: null, reusableExistingExecutionWorkspace: null, workspaceReuseRequest: resolveExecutionWorkspaceReuseRequestForIssue({ issueExecutionWorkspaceId: null, issueExecutionWorkspacePreference: null, existingExecutionWorkspaceStatus: null }), nativeRecoveryExecutionWorkspaceId: null, persistedNativeExecutionInput: null, isDotRun: false, runOptions: {} },
services: { envOrchestrator, executionWorkspacesSvc: executionWorkspaceService(db), workspaceOperationsSvc: workspaceOperationService(db), issuesSvc: issueService(db), ...resolver, appendRunEvent: vi.fn() },
controllerLease: { assertOwned: vi.fn(async () => {}) }, nativeRunnerPreparationSpans: [],
};
return { input, source, issue, run, envOrchestrator };
}
const readIssue = (id: string) => db.select().from(issues).where(eq(issues.id, id)).then(rows => rows[0]!);
const readRun = (id: string) => db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, id)).then(rows => rows[0]!);
const leases = (id: string) => db.select().from(environmentLeases).where(eq(environmentLeases.heartbeatRunId, id));
async function reuse(input: HeartbeatWorkspacePreparationInput, workspace: NonNullable<Awaited<ReturnType<typeof prepareHeartbeatWorkspace>>["persistedExecutionWorkspace"]>) {
input.reuse.requestedShouldReuseExisting = true;
input.reuse.existingExecutionWorkspace = workspace;
input.reuse.reusableExistingExecutionWorkspace = workspace;
input.reuse.workspaceReuseRequest = resolveExecutionWorkspaceReuseRequestForIssue({ issueExecutionWorkspaceId: workspace.id, issueExecutionWorkspacePreference: "reuse_existing", existingExecutionWorkspaceStatus: workspace.status });
input.task.issueRef = { ...input.task.issueRef!, executionWorkspaceId: workspace.id, executionWorkspacePreference: "reuse_existing" };
const [run] = await db.insert(heartbeatRuns).values({ companyId: input.run.companyId, agentId: input.agent.id, issueId: input.task.issueId, scopeKind: "issue", status: "running", runtimeMode: "legacy" }).returning();
input.run = run;
}
it("persists the workspace and issue binding before acquiring its environment", async () => {
const { input, source, issue, envOrchestrator } = await fixture();
const acquire = envOrchestrator.acquireForRun;
vi.spyOn(envOrchestrator, "acquireForRun").mockImplementation(async args => {
expect((await readIssue(issue.id)).executionWorkspaceId).toBe(args.persistedExecutionWorkspace?.id);
expect((await readRun(input.run.id)).contextSnapshot?.executionWorkspaceId).toBe(args.persistedExecutionWorkspace?.id);
return acquire(args);
});
const result = await prepareHeartbeatWorkspace(db, input);
expect(result.executionWorkspace.cwd).toBe(source);
expect(result.persistedExecutionWorkspace).toMatchObject({ companyId: issue.companyId, projectId: issue.projectId, projectWorkspaceId: issue.projectWorkspaceId, mode: "shared_workspace" });
expect(await leases(input.run.id)).toMatchObject([{ status: "active", executionWorkspaceId: result.persistedExecutionWorkspace!.id }]);
expect(input.nativeRunnerPreparationSpans.map(span => span.name)).toEqual(["environment.acquire", "environment.workspace.realize"]);
expect(input.controllerLease.assertOwned).toHaveBeenCalledTimes(2);
});
it("creates a real isolated worktree and reuses its recorded workspace", async () => {
const { input, source, issue } = await fixture(true);
const first = await prepareHeartbeatWorkspace(db, input);
expect(first.executionWorkspace.cwd).not.toBe(source);
expect(await readFile(join(first.executionWorkspace.cwd, "README.md"), "utf8")).toBe("source content\n");
expect(await readIssue(issue.id)).toMatchObject({ executionWorkspaceId: first.persistedExecutionWorkspace!.id, executionWorkspacePreference: "reuse_existing" });
await writeFile(join(first.executionWorkspace.cwd, "retained.txt"), "keep this work");
await reuse(input, first.persistedExecutionWorkspace!);
const next = await prepareHeartbeatWorkspace(db, input);
expect(next.reusedExecutionWorkspace).toMatchObject({ cwd: first.executionWorkspace.cwd });
expect(next.persistedExecutionWorkspace!.id).toBe(first.persistedExecutionWorkspace!.id);
expect(await readFile(join(next.executionWorkspace.cwd, "retained.txt"), "utf8")).toBe("keep this work");
});
it("keeps the admitted recovery workspace when the issue now points elsewhere", async () => {
const { input, issue } = await fixture(true);
const first = await prepareHeartbeatWorkspace(db, input);
await reuse(input, first.persistedExecutionWorkspace!);
await db.update(issues).set({ executionWorkspaceId: null, projectWorkspaceId: null }).where(eq(issues.id, issue.id));
input.task.issueRef = { ...input.task.issueRef!, executionWorkspaceId: null, projectWorkspaceId: null };
input.reuse.nativeRecoveryExecutionWorkspaceId = first.persistedExecutionWorkspace!.id;
const anchor = vi.spyOn(input.services, "resolveReusedGitWorkspaceAnchor");
const next = await prepareHeartbeatWorkspace(db, input);
expect(next.persistedExecutionWorkspace!.id).toBe(first.persistedExecutionWorkspace!.id);
expect(anchor).toHaveBeenCalledWith(expect.objectContaining({ immutableNativeBinding: true, explicitProjectWorkspaceId: first.persistedExecutionWorkspace!.projectWorkspaceId }));
expect(await readIssue(issue.id)).toMatchObject({ executionWorkspaceId: null, projectWorkspaceId: null });
});
it("removes a newly created worktree when persistence fails and preserves the original error", async () => {
const { input, source } = await fixture(true);
const error = new Error("workspace persistence unavailable");
let createdCwd = "";
vi.spyOn(input.services.executionWorkspacesSvc, "create").mockImplementation(async args => { createdCwd = args.cwd!; throw error; });
await expect(prepareHeartbeatWorkspace(db, input)).rejects.toBe(error);
expect(createdCwd).not.toBe("");
await expect(access(createdCwd)).rejects.toThrow();
expect(await readFile(join(source, "README.md"), "utf8")).toBe("source content\n");
expect(await leases(input.run.id)).toHaveLength(0);
});
it("preserves a reused worktree when its metadata update fails", async () => {
const { input } = await fixture(true);
const first = await prepareHeartbeatWorkspace(db, input);
await reuse(input, first.persistedExecutionWorkspace!);
await writeFile(join(first.executionWorkspace.cwd, "retained.txt"), "unsaved work");
const error = new Error("workspace update unavailable");
vi.spyOn(input.services.executionWorkspacesSvc, "update").mockRejectedValueOnce(error);
await expect(prepareHeartbeatWorkspace(db, input)).rejects.toBe(error);
expect(await readFile(join(first.executionWorkspace.cwd, "retained.txt"), "utf8")).toBe("unsaved work");
expect(await leases(input.run.id)).toHaveLength(0);
});
it.each(["acquire", "realize", "ownership"] as const)("preserves %s errors and durable lease state for outer cleanup", async failure => {
const { input, envOrchestrator } = await fixture();
const error = new Error(`${failure} failed`);
if (failure === "acquire") vi.spyOn(envOrchestrator, "acquireForRun").mockRejectedValueOnce(error);
if (failure === "realize") vi.spyOn(envOrchestrator, "realizeForRun").mockRejectedValueOnce(error);
if (failure === "ownership") vi.mocked(input.controllerLease.assertOwned).mockResolvedValueOnce(undefined).mockRejectedValueOnce(error);
await expect(prepareHeartbeatWorkspace(db, input)).rejects.toBe(error);
expect(input.nativeRunnerPreparationSpans.at(-1)).toMatchObject({ name: failure === "realize" ? "environment.workspace.realize" : "environment.acquire", outcome: "failed" });
expect((await readRun(input.run.id)).status).toBe("running");
expect(await leases(input.run.id)).toHaveLength(failure === "acquire" ? 0 : 1);
if (failure !== "acquire") {
expect(await leases(input.run.id)).toMatchObject([{ status: "active" }]);
// The production executor terminalizes first, then releases by durable run ID.
await db.update(heartbeatRuns).set({ status: "failed" }).where(eq(heartbeatRuns.id, input.run.id));
await envOrchestrator.releaseForRun({ heartbeatRunId: input.run.id, companyId: input.run.companyId, agentId: input.agent.id, status: "released" });
expect(await leases(input.run.id)).toMatchObject([{ status: "released" }]);
}
});
it("checks the acquired environment allowlist before realization", async () => {
const { input, envOrchestrator } = await fixture();
input.policy.executionPolicy = { managedSandboxOnly: true };
const realize = vi.spyOn(envOrchestrator, "realizeForRun");
await expect(prepareHeartbeatWorkspace(db, input)).rejects.toThrow("forbids local execution");
expect(realize).not.toHaveBeenCalled();
expect(await leases(input.run.id)).toMatchObject([{ status: "active" }]);
});
it("rejects low-trust local execution before resolving or provisioning a workspace", async () => {
const { input } = await fixture(true);
input.policy.trustPreset = {
kind: "low_trust_review", preset: "low_trust_review", sourcePresets: {},
boundary: { mode: "low_trust_review", companyId: input.agent.companyId, rootIssueId: input.task.issueId! },
};
const resolve = vi.spyOn(input.services, "resolveWorkspaceForRun");
const create = vi.spyOn(input.services.executionWorkspacesSvc, "create");
await expect(prepareHeartbeatWorkspace(db, input)).rejects.toThrow("requires a sandbox environment");
expect(resolve).not.toHaveBeenCalled();
expect(create).not.toHaveBeenCalled();
expect(await leases(input.run.id)).toHaveLength(0);
});
it("rejects remote recovery without its retained sync binding before acquiring a lease", async () => {
const { input, envOrchestrator } = await fixture();
input.reuse.runOptions.nativeRestartRecovery = {
kind: "reattach_remote_runner", runId: input.run.id, leaseOwner: "controller",
controllerGeneration: 1, providerAttempt: 1, restartKind: "hard", recoveryRequestId: null,
remote: { providerLeaseId: "original-provider-lease", remoteCwd: "/original" },
};
const acquire = vi.spyOn(envOrchestrator, "acquireForRun");
await expect(prepareHeartbeatWorkspace(db, input)).rejects.toThrow("native_remote_recovery_lease_mismatch");
expect(acquire).not.toHaveBeenCalled();
expect(await leases(input.run.id)).toHaveLength(0);
expect(input.nativeRunnerPreparationSpans).toMatchObject([{ name: "environment.acquire", outcome: "failed" }]);
});
it("binds a replacement workspace from realization and snapshots the restore lease", async () => {
const { input, envOrchestrator, issue } = await fixture();
const realize = envOrchestrator.realizeForRun;
vi.spyOn(envOrchestrator, "realizeForRun").mockImplementation(async args => {
const result = await realize(args);
const workspace = await input.services.executionWorkspacesSvc.create({ companyId: issue.companyId, projectId: issue.projectId!, projectWorkspaceId: issue.projectWorkspaceId, sourceIssueId: issue.id, name: "Realized workspace", mode: "shared_workspace", strategyType: "project_primary", cwd: args.executionWorkspace.cwd });
return { ...result, persistedExecutionWorkspace: workspace, lease: { ...result.lease, metadata: { remoteCwd: "/original" } } };
});
const result = await prepareHeartbeatWorkspace(db, input);
expect((await readIssue(issue.id)).executionWorkspaceId).toBe(result.persistedExecutionWorkspace!.id);
expect(result.persistedExecutionWorkspace!.name).toBe("Realized workspace");
result.activeEnvironmentLease.lease.metadata!.remoteCwd = "/rebound";
expect(result.workspaceRestoreSource.metadata?.remoteCwd).toBe("/original");
});
});
File diff suppressed because it is too large. Load diff
+3
View File
@@ -205,6 +205,9 @@ for (const adapter of ["process", "paperclip_runner"] as const) {
"-e",
"console.log('stop fixture ready'); setInterval(() => console.log('working'), 200);",
],
// The child only runs this script; server-only fixture
// loaders must not follow NODE_OPTIONS into the adapter.
env: { NODE_OPTIONS: "" },
graceSec: 1,
}
: { provider: "codex", model: "gpt-5.1-codex-mini" },
+5 -6
View File
@@ -9,14 +9,13 @@ timeout only when the test explicitly arms a marker inside its repository.
From a development worktree, start the foreground instance:
```sh
BOOTSTRAP_FIXTURE_KEY=not-a-provider-key \
PATH="$PWD/tests/e2e/workspace-bootstrap/bin:$PATH" \
NODE_ENV=test \
node cli/node_modules/tsx/dist/cli.mjs cli/src/index.ts test-drive \
--harness codex --api-key-env BOOTSTRAP_FIXTURE_KEY \
--company-name 'Workspace Recovery QA' --no-browser
node cli/node_modules/tsx/dist/cli.mjs tests/e2e/workspace-bootstrap/launch.ts
```
The launcher creates an isolated test-drive, installs the Git shim, and sets a
10-second snapshot deadline after test-drive clears inherited configuration.
Production's longer snapshot deadline would outlast the injected failure.
Set `BOOTSTRAP_REAL_GIT` to the absolute Git executable if it is not
`/usr/bin/git`. Do not point this fixture at a normal development or production
instance. The seeder checks the loopback host, trusted-local mode, and company
+18
View File
@@ -0,0 +1,18 @@
import path from "node:path";
import { prepareTestDriveEnvironment, testDriveCommand } from "../../../cli/src/commands/test-drive.js";
const options = {
harness: "codex",
apiKeyEnv: "BOOTSTRAP_FIXTURE_KEY",
companyName: "Workspace Recovery QA",
browser: false,
} as const;
process.env.BOOTSTRAP_FIXTURE_KEY = "not-a-provider-key";
await prepareTestDriveEnvironment(options);
process.env.NODE_ENV = "test";
// Test-drive clears PAPERCLIP_* settings. Apply the fixture's short deadline
// after isolation; production snapshots may legitimately take much longer.
process.env.PAPERCLIP_WORKSPACE_GIT_SNAPSHOT_TIMEOUT_MS = "10000";
process.env.PATH = `${path.join(import.meta.dirname, "bin")}${path.delimiter}${process.env.PATH ?? ""}`;
await testDriveCommand(options);
+14 -8
View File
@@ -22,11 +22,11 @@ for (const persistent of [false, true]) {
if (await announcement.isVisible()) await announcement.click();
await page.getByRole("link", { name: "Tasks", exact: true }).click();
await page.getByRole("button", { name: "New Task", exact: true }).last().click();
await page.getByPlaceholder("Task title (optional)", { exact: true }).fill(title);
await page.getByRole("button", { name: "Assignee", exact: true }).click();
await page.getByRole("button", { name: fixture.agentName, exact: true }).click();
await page.getByRole("textbox", { name: "editable markdown" }).fill(title);
await page.getByRole("button", { name: "Select assignee", exact: true }).click();
await page.getByRole("option").filter({ hasText: fixture.agentName }).click();
// Let the closing popover unmount before clicking another popover trigger.
await expect(page.getByRole("textbox", { name: "Search assignees...", includeHidden: true })).toHaveCount(0);
await expect(page.getByRole("searchbox", { name: "Search assignees", includeHidden: true })).toHaveCount(0);
// The preceding popover's focus restoration can consume the first click.
await expect(async () => {
if (!await page.getByRole("textbox", { name: "Search projects..." }).isVisible()) {
@@ -36,15 +36,21 @@ for (const persistent of [false, true]) {
}).toPass({ timeout: 10_000, intervals: [1_000] });
await page.getByRole("button", { name: fixture.projectName, exact: true }).click();
await expect(page.getByRole("textbox", { name: "Search projects...", includeHidden: true })).toHaveCount(0);
await page.getByRole("button", { name: "Create Task", exact: true }).click();
const taskLink = page.getByRole("complementary").getByRole("link", { name: title, exact: true });
const created = page.waitForResponse(response =>
response.request().method() === "POST"
&& new URL(response.url()).pathname === `/api/companies/${fixture.companyId}/issues`,
);
await page.getByRole("button", { name: "Create task", exact: true }).click();
const response = await created;
expect(response.ok(), await response.text()).toBeTruthy();
const task = await response.json();
expect(task.description).toBe(title);
const taskLink = page.getByRole("complementary").getByRole("link", { name: task.title, exact: true });
await expect(taskLink).toBeVisible();
// Follow the UI's actual persisted link, including across task-tab state.
const href = await taskLink.getAttribute("href");
expect(href).toBeTruthy();
await page.goto(new URL(href!, base).href);
const tasks = await api(`/companies/${fixture.companyId}/issues`);
const task = tasks.find((row: { title: string }) => row.title === title);
const runs = async () => (await api(`/companies/${fixture.companyId}/heartbeat-runs`)).filter((row: { agentId: string }) => row.agentId === fixture.agentId);
await expect.poll(async () => (await runs()).some((row: { errorCode: string }) => row.errorCode === "workspace_git_scan_timeout"), { timeout: 30_000 }).toBe(true);
await expect(page.getByText(/Agent resumes in/)).toBeVisible();