Files
PaperClipAI/server/src/services/heartbeat/workspaces.test.ts
T
Nicky LeachandPaperclip 00331e0254 Connect native Runner admission to plugin-owned workspaces (#16978)
## Thinking Path

> - Paperclip manages work for AI agents.
> - A plugin can own an execution environment and its files.
> - #17029 supplies the task driver and bounded setup verification.
> - Heartbeat still prepares workspace files on Core and expects a shell
transport.
> - This PR gives the selected plugin ownership of workspace preparation
and native task execution.
> - Core keeps admission, tool authority, and recovery checks.

## Linked Issues or Issue Description

**Subsystem affected**

Heartbeat workspace admission and native Runner execution.

**Problem or motivation**

A task-only plugin cannot run through Core filesystem staging or shell
execution. Native admission and recovery must use the same lease and
workspace ownership.

**Proposed solution**

Use the task driver from #17029. Resolve workspace and project
identities without Core file operations. Connect native dispatch and
restart recovery to that driver.

**Alternatives considered**

Keep workspace admission and dispatch in one change. A workspace change
alone would leave dispatch on the previous transport. An execution
change alone would leave Core staging files for the wrong host.

## What Changed

- Enable plugin environment selection for native workspace sessions.
- Resolve workspace metadata and authorized project references without
Core checkout, copy, or provision operations.
- Preserve agent-default and primary workspace policies. Use the pinned
workspace provider for plugin-backed project storage. Reject unsupported
Core service commands and cross-owner reuse.
- Connect native execution to the prepared task driver. Keep assigned
MCP authority on Core.
- Save exact checkpoint identity, transfer selected account refreshes,
and require termination proof before replacement or cleanup.
- Complete managed plugin workspaces through their storage provider. Use
the same check after a restart. Keep branch validation and leave
task-owned files on the plugin.
- Include default instruction content in runtime synchronization when no
instruction directory exists.
- Defer new-agent Copilot verification until the saved-agent lifecycle
can acquire the agent-owned computer.

## Verification

- Rebased onto `origin/master` at `898144f4b3`. Master advanced once
during verification. Both manual conflicts joined import blocks: one in
`native-session-executor.ts`, and one in
`heartbeat/runtime-execution.ts`. The seven patches retain their logic.
The first rebased head passed repository typecheck, build, all 862
focused execution/finalizer cases, and all CI gates. The latest head
`93e5a72892` passes server typecheck and all 880 focused execution,
finalization, and execution-control notification tests. All 54 CI/review
checks pass, with two expected skips. Greptile ran automatically and
scored this head 5/5 with no unresolved findings. No new review or Apex
request was made for these import-only resolutions.
- The successful CI workflow initially left three stale job-check
reports. Targeted policy and summary reruns cleared the reports. No
source change or test fix was needed.
- Repository typecheck and build pass: `pnpm -r typecheck` and `pnpm
build`.
- Focused native execution and restart tests pass: 795 cases. The
account and harness suites pass: 159 cases. Workspace admission passes:
48 cases. The SDK task lifecycle, workspace API, runtime-hook, and
instruction checks also pass.
- The heartbeat integration case uses the merged workspace provider
contract and reaches native dispatch with no project files on Core. The
new-agent case confirms that creation leaves the agent in `preparing`
until lifecycle verification.
- Map validation and the Copilot source-identity check pass. No Runner
implementation or source-identity bump is in this PR.
- The first rebased workspace run found two stale path assertions and a
recovery guard that did not recognize task-owned targets. These are
fixed. The storage integration fixture also needed its persisted source
binding; the production authorization check stays in place.
- Two prior review findings required rejection of Core service startup
and cross-owner workspace reuse. The existing admission fixtures now
check both failures. The installed-account authentication finding was
fixed in merged #17029.
- Tests reuse the current fixtures. No new provider test matrix or live
Runner process is added for these fixes.
- The prior head `2db273ec12` passed CI on its first attempt and reached
standard Greptile 5/5. Apex scored 4/5 and found local-folder checks
during plugin completion and recovery.
- The completion fix reproduces both failures. It passes all 110 cases
in the runtime execution, workspace finalizer, and workspace provider
integration suites. Server typecheck and map checks pass. CI passed on
`62d8335b4e` on its first attempt, and standard Greptile scored 5/5. The
next Apex review scored 4/5. It found that a failed plugin inspection
could abort restart recovery.
- The restart fix records a failed workspace receipt and uses the
current run retry policy. A regression first reproduced the aborted
pass. It now checks retry state, retained results, and completion of
another workspace in the same pass. All 68 finalizer cases and server
typecheck pass. CI passed on `7d4016323b` on its first attempt. Standard
Greptile and Apex both scored 5/5. Apex left one P2 about lost failure
codes.
- The diagnostic fix keeps the known branch and owner mismatch codes in
receipts and run recovery state. It gives repair guidance for those
codes and keeps unknown plugin errors generic. All 96 affected finalizer
cases and server typecheck pass. Before the rebase, required CI passed
on `c2fa61c6c8` on its first attempt. Standard Greptile and Apex both
scored 5/5 on that head, with no unresolved findings. The new head
retains these reviewed patches.
- The earlier Superagent scan on `c2fa61c6c8` was inconclusive after a
pinned merge-base checkout failed or timed out. The current scan on
`93e5a72892` passes.
- The exact rebased head passes all 48 workspace cases and server
typecheck. The first local test launch could not bind PostgreSQL inside
the sandbox. The authorized local-server run passed.
- The prior head reached standard Greptile 5/5. Its first CI run hit an
unchanged Runner cleanup timeout and an external build-runner shutdown.
Both jobs passed on the single retry. The timeout cause is not
established.
- A local regression check found that a retained Core workspace could
also enter a plugin task without file transfer. Core now rejects this
move before provider work. The existing ledger test covers both
directions and confirms that the source files stay intact.
- The first active standard review scored 4/5. It found that a fresh
plugin retry could enter Core Git preparation. This mode now uses the
existing unsupported-storage error before workspace resolution or
acquisition. Both unsupported-mode cases pass, and server typecheck
passes.
- No live Cloud or provider validation was run.

## Risks

- The prerequisite #17029 is merged. This PR targets master.
- Native plugin tasks support per-turn workspace sessions. Core service
commands and fresh retries that import retained files remain
unsupported. A workspace storage plugin can use the modes and strategies
it declares.
- Core-managed credentials support Codex and direct Claude accounts.
Other managed credential routes fail closed. Other harnesses can use an
installed account.
- A provider must preserve its prepared configuration and report process
termination correctly. Uncertain termination keeps the task pending
cleanup.
- Cloud deployment and live provider validation remain separate.

## Model Used

OpenAI Codex, GPT-6. The exact model ID and context window are not
exposed in this session. Tool use includes code editing, local tests,
and the GitHub CLI.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have described the issue in-PR following the relevant issue
template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal issue
details
- [x] I have run focused tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-21 08:22:48 -07:00

253 lines
13 KiB
TypeScript

import { execFile } from "node:child_process";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { promisify } from "node:util";
import type { agents, Db, projectWorkspaces } from "@paperclipai/db";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import {
ensureManagedProjectWorkspace as legacyEnsureManagedProjectWorkspace,
WorkspaceValidationFailure as LegacyWorkspaceValidationFailure,
} from "../heartbeat.js";
import { resolveDefaultAgentWorkspaceDir } from "../../home-paths.js";
import {
createHeartbeatWorkspaceResolver,
ensureManagedProjectWorkspace,
WorkspaceValidationFailure,
} from "./workspaces.js";
const runGit = promisify(execFile);
let testRoot: string;
beforeEach(async () => {
testRoot = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-workspace-module-"));
vi.stubEnv("PAPERCLIP_HOME", testRoot);
vi.stubEnv("PAPERCLIP_INSTANCE_ID", "workspace-test");
vi.stubEnv("PAPERCLIP_MULTI_PROJECT_WORKSPACE_SYNC", "false");
});
afterEach(async () => {
vi.unstubAllEnvs();
await fs.rm(testRoot, { recursive: true, force: true });
});
function agent(id = "agent-one") {
return { id, companyId: "company-one" } as typeof agents.$inferSelect;
}
function workspace(id: string, projectId: string, cwd: string) {
return {
id, projectId, companyId: "company-one", cwd,
repoUrl: null, repoRef: null, sourceType: "local_path",
} as typeof projectWorkspaces.$inferSelect;
}
// The existing service integration suites cover the SQL and authorization rules.
// These query results exercise the new factory's database binding and call order.
function databaseWithResults(...results: unknown[][]) {
const select = vi.fn(() => {
const rows = results.shift() ?? [];
const query = {
from: () => query,
where: () => query,
orderBy: async () => rows,
then: (resolve: (value: unknown[]) => unknown) => Promise.resolve(rows).then(resolve),
};
return query;
});
return { db: { select } as unknown as Db, select };
}
function deferred() {
let resolve!: () => void;
const promise = new Promise<void>(complete => { resolve = complete; });
return { promise, resolve };
}
async function createRepository(parent: string, text: string) {
const cwd = path.join(testRoot, parent, "repo");
await fs.mkdir(cwd, { recursive: true });
await runGit("git", ["init", cwd]);
await runGit("git", ["-C", cwd, "config", "user.email", "workspace-test@example.com"]);
await runGit("git", ["-C", cwd, "config", "user.name", "Workspace Test"]);
await fs.writeFile(path.join(cwd, "README.md"), text);
await runGit("git", ["-C", cwd, "add", "README.md"]);
await runGit("git", ["-C", cwd, "commit", "-m", "initial"]);
return cwd;
}
describe("heartbeat workspace module", () => {
it("preserves the legacy error class and checkout entry point", () => {
expect(LegacyWorkspaceValidationFailure).toBe(WorkspaceValidationFailure);
expect(legacyEnsureManagedProjectWorkspace).toBe(ensureManagedProjectWorkspace);
const failure = new WorkspaceValidationFailure("invalid checkout", { reason: "test" });
expect(failure).toBeInstanceOf(LegacyWorkspaceValidationFailure);
expect(failure.code).toBe("workspace_validation_failed");
});
it("binds each resolver to its own database", async () => {
const firstCwd = path.join(testRoot, "first");
const secondCwd = path.join(testRoot, "second");
await Promise.all([fs.mkdir(firstCwd), fs.mkdir(secondCwd)]);
const first = databaseWithResults([workspace("workspace-one", "project-one", firstCwd)]);
const second = databaseWithResults([workspace("workspace-two", "project-two", secondCwd)]);
const [firstResult, secondResult] = await Promise.all([
createHeartbeatWorkspaceResolver(first.db).resolveWorkspaceForRun(agent(), { projectId: "project-one" }, null),
createHeartbeatWorkspaceResolver(second.db).resolveWorkspaceForRun(agent(), { projectId: "project-two" }, null),
]);
expect(firstResult).toMatchObject({ cwd: firstCwd, projectId: "project-one", workspaceId: "workspace-one", source: "project_primary", additionalWorkspaces: [] });
expect(secondResult).toMatchObject({ cwd: secondCwd, projectId: "project-two", workspaceId: "workspace-two", source: "project_primary", additionalWorkspaces: [] });
expect(first.select).toHaveBeenCalledTimes(2);
expect(second.select).toHaveBeenCalledTimes(2);
});
it("uses the current issue's project and selected workspace over stale context", async () => {
const selectedCwd = path.join(testRoot, "selected");
await fs.mkdir(selectedCwd);
const database = databaseWithResults(
[{ projectId: "current-project", projectWorkspaceId: "selected-workspace" }],
[{ projectId: "current-project" }],
[workspace("default-workspace", "current-project", path.join(testRoot, "missing")), workspace("selected-workspace", "current-project", selectedCwd)],
);
const result = await createHeartbeatWorkspaceResolver(database.db).resolveWorkspaceForRun(
agent(),
{ issueId: "issue-one", projectId: "stale-project", projectWorkspaceId: "stale-workspace" },
null,
);
expect(result).toMatchObject({ cwd: selectedCwd, projectId: "current-project", workspaceId: "selected-workspace", warnings: [], baseCwdFallback: false });
expect(database.select).toHaveBeenCalledTimes(4);
});
it("resolves plugin storage without creating or reading a Core checkout", async () => {
const missingCwd = path.join(testRoot, "provider-only-repository");
const database = databaseWithResults([workspace("remote-source", "project-one", missingCwd)],
[{ policy: { storageProviderPluginId: "10000000-0000-4000-8000-000000000001" } }]);
const result = await createHeartbeatWorkspaceResolver(database.db).resolveWorkspaceForRun(agent(), { projectId: "project-one" }, null);
expect(result).toMatchObject({ cwd: "/workspaces/remote-source", workspaceId: "remote-source", projectId: "project-one", source: "project_primary" });
expect(await fs.stat(missingCwd).catch(() => null)).toBeNull();
});
it("resolves plugin project identity without Core files", async () => {
const repoUrl = "/unavailable-controller-source.git";
const missingCwd = path.join(testRoot, "exists-only-on-plugin");
const configured = { ...workspace("selected", "project-one", missingCwd), repoUrl };
const database = databaseWithResults([configured], [{ id: "project-one" }]);
const stat = vi.spyOn(fs, "stat"); const mkdir = vi.spyOn(fs, "mkdir");
try {
const result = await createHeartbeatWorkspaceResolver(database.db).resolveWorkspaceForRun(
{ ...agent(), adapterType: "paperclip_runner" }, { projectId: "project-one" }, null,
{ executionEnvironmentDriver: "plugin" });
expect(result).toMatchObject({ cwd: "/workspaces/selected", projectId: "project-one", workspaceId: "selected", repoUrl,
source: "project_primary", baseCwdFallback: false, additionalWorkspaces: [] });
expect(stat).not.toHaveBeenCalled(); expect(mkdir).not.toHaveBeenCalled();
} finally { stat.mockRestore(); mkdir.mockRestore(); }
});
it("keeps a projectless plugin scope without creating an agent directory on Core", async () => {
const mkdir = vi.spyOn(fs, "mkdir");
try {
const result = await createHeartbeatWorkspaceResolver(databaseWithResults().db).resolveWorkspaceForRun(
{ ...agent(), adapterType: "paperclip_runner" }, {}, null, { executionEnvironmentDriver: "plugin" });
expect(result).toMatchObject({ cwd: "/workspaces/agent-one", projectId: null, workspaceId: null });
expect(mkdir).not.toHaveBeenCalled();
} finally { mkdir.mockRestore(); }
});
it("rejects plugin project scope outside the company", async () => {
const database = databaseWithResults([], []);
await expect(createHeartbeatWorkspaceResolver(database.db).resolveWorkspaceForRun(
{ ...agent(), adapterType: "paperclip_runner" }, { projectId: "foreign" }, null,
{ executionEnvironmentDriver: "plugin" })).rejects.toMatchObject({ code: "workspace_validation_failed" });
});
it("fails closed with workspace diagnostics when a configured directory is unavailable", async () => {
const missingCwd = path.join(testRoot, "missing-configured-workspace");
const database = databaseWithResults([workspace("selected", "project-one", missingCwd)]);
await expect(createHeartbeatWorkspaceResolver(database.db).resolveWorkspaceForRun(
agent(), { projectId: "project-one" }, null,
)).rejects.toMatchObject({
code: "workspace_validation_failed",
resultJson: { workspaceValidation: {
reason: "configured_workspace_unavailable", resolvedProjectId: "project-one", baseCwdFallback: false,
} },
});
expect(await fs.stat(missingCwd).catch(() => null)).toBeNull();
});
it("retains materialization diagnostics when a configured repository cannot be cloned", async () => {
const missingRepo = path.join(testRoot, "missing-repository.git");
const database = databaseWithResults([{
...workspace("selected", "project-one", ""), repoUrl: missingRepo, sourceType: "git_remote",
}]);
await expect(createHeartbeatWorkspaceResolver(database.db).resolveWorkspaceForRun(
agent(), { projectId: "project-one" }, null,
)).rejects.toMatchObject({
code: "workspace_validation_failed",
resultJson: { workspaceValidation: {
reason: "git_worktree_base_materialization_failed",
materializationFailures: [expect.objectContaining({ projectWorkspaceId: "selected", repoUrl: missingRepo, error: expect.any(String) })],
} },
});
});
it("retains a valid prior session workspace and falls back when it disappears", async () => {
const priorCwd = path.join(testRoot, "prior-session");
await fs.mkdir(priorCwd);
const database = databaseWithResults();
const resolver = createHeartbeatWorkspaceResolver(database.db);
const previousSession = { cwd: priorCwd, workspaceId: "previous-workspace" };
expect(await resolver.resolveWorkspaceForRun(agent(), {}, previousSession)).toMatchObject({ cwd: priorCwd, source: "task_session", workspaceId: "previous-workspace", warnings: [] });
await fs.rm(priorCwd, { recursive: true });
const fallback = await resolver.resolveWorkspaceForRun(agent(), {}, previousSession);
expect(fallback).toMatchObject({ cwd: resolveDefaultAgentWorkspaceDir(agent().id), source: "agent_home", workspaceId: null });
expect(fallback.warnings).toEqual([expect.stringContaining("is not available")]);
expect((await fs.stat(fallback.cwd)).isDirectory()).toBe(true);
expect(database.select).not.toHaveBeenCalled();
});
it("shares one in-flight checkout across legacy and direct module callers", async () => {
const repoUrl = await createRepository("source", "shared checkout\n");
const started = deferred();
const release = deferred();
const resolveGitAuth = vi.fn(async () => {
started.resolve();
await release.promise;
return null;
});
const input = { companyId: "company-one", projectId: "shared-project", repoUrl, resolveGitAuth };
const attempts = Array.from({ length: 8 }, (_, index) =>
(index % 2 ? legacyEnsureManagedProjectWorkspace : ensureManagedProjectWorkspace)(input),
);
await started.promise;
release.resolve();
const results = await Promise.all(attempts);
expect(resolveGitAuth).toHaveBeenCalledTimes(1);
expect(new Set(results.map(result => result.cwd)).size).toBe(1);
expect(await fs.readFile(path.join(results[0]!.cwd, "README.md"), "utf8")).toBe("shared checkout\n");
expect((await runGit("git", ["-C", results[0]!.cwd, "remote", "get-url", "origin"])).stdout.trim()).toBe(repoUrl);
});
it("keeps concurrent repositories with the same basename in separate checkouts", async () => {
const [firstRepo, secondRepo] = await Promise.all([
createRepository("first-source", "first repository\n"),
createRepository("second-source", "second repository\n"),
]);
const [first, second] = await Promise.all([firstRepo, secondRepo].map(repoUrl =>
ensureManagedProjectWorkspace({ companyId: "company-one", projectId: "shared-project", repoUrl }),
));
expect(first!.cwd).not.toBe(second!.cwd);
expect(await fs.readFile(path.join(first!.cwd, "README.md"), "utf8")).toBe("first repository\n");
expect(await fs.readFile(path.join(second!.cwd, "README.md"), "utf8")).toBe("second repository\n");
});
it("releases a failed checkout attempt so the next caller can retry", async () => {
const repoUrl = await createRepository("retry-source", "retry succeeded\n");
const resolveGitAuth = vi.fn()
.mockRejectedValueOnce(new Error("credential lookup failed"))
.mockResolvedValue(null);
const input = { companyId: "company-one", projectId: "retry-project", repoUrl, resolveGitAuth };
await expect(ensureManagedProjectWorkspace(input)).rejects.toThrow("credential lookup failed");
const result = await legacyEnsureManagedProjectWorkspace(input);
expect(resolveGitAuth).toHaveBeenCalledTimes(2);
expect(await fs.readFile(path.join(result.cwd, "README.md"), "utf8")).toBe("retry succeeded\n");
});
});