fix(ssh): transport project repositories as their own git checkouts (#14782)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - A project can attach more than one repository. The task workspace
keeps the selected repository at its root and puts the other project
repositories under `.paperclip-repositories/<name>-<key>`, each with its
own `.git`
> - Agents can run on an SSH execution environment. Paperclip copies the
task workspace to the remote host before the run and restores it after
the run
> - The SSH copy excludes `.git` at every depth, but the restore
baseline excludes it only at the workspace root
> - So the other project repositories reach the remote host without Git,
and the restore then deletes their `.git` directories on the Paperclip
host
> - The next run of the same task fails during workspace setup, and the
agent cannot commit to those repositories on the remote host
> - This pull request transports each project repository as a Git
workspace of its own, the same way the sandbox path already handles them
> - The benefit is that multi-repository projects work on SSH
environments across consecutive runs

## Linked Issues or Issue Description

Refs #11632 (SSH workspace transfer exclude list). Related SSH workspace
PRs: #14233, #14428, #14472. I found no issue or PR for this bug.

**What happened?**

A project has two repositories and its agent runs on an SSH environment.
After the first run, the second repository under
`.paperclip-repositories/` has no `.git` directory on the Paperclip
host. The next run of the same task fails during setup with `Managed
workspace path "…/.paperclip-repositories/<repo>" already exists but is
not a git checkout.` On the remote host, `git` inside that repository
resolves to the parent repository.

**Expected behavior**

Each project repository reaches the remote host as a Git checkout with
its local changes. Remote commits and edits come back after the run. The
next run of the same task starts normally.

**Steps to reproduce**

1. Create a project with two repositories.
2. Configure an SSH execution environment and make it the agent's
default environment.
3. Assign a task to the agent and let it run once.
4. Look at `.paperclip-repositories/<repo>` in the task workspace:
`.git` is gone.
5. Wake the agent on the same task again: the run fails with
`setup_failed`.

**Paperclip version or commit**

Reproduced on `v2026.916.1` and on `master` (`5edf55d73`).

**Deployment mode**

Self-hosted (Docker), authenticated, with an SSH execution environment.

## What Changed

- `ssh.ts`: `prepareWorkspaceForSshExecution` lists the project
repositories under `.paperclip-repositories/`. It applies the discovery
rules of `readGitWorkspaceSnapshot`: each entry must be a directory with
a valid name and must be a Git repository root, else the prepare step
fails before any transfer.
- `ssh.ts`: the anchor copy leaves `.paperclip-repositories/` out. Each
project repository then gets the same import, sync, and deleted-path
steps as the anchor. The remote anchor repository ignores
`/.paperclip-repositories/`, as the local checkout does.
- `ssh.ts`: `prepareWorkspaceForSshExecution` returns the transported
repositories (the field is present only when there are repositories).
`restoreWorkspaceFromSshExecution` accepts them with their baselines. It
validates each path and baseline first, then restores the repositories
before the anchor and stops at the first failure, as the sandbox restore
does.
- `remote-managed-runtime.ts`: the anchor baseline excludes
`.paperclip-repositories/`, and each project repository gets its own
baseline for the restore merge.
- `ssh-fixture.test.ts`: regression tests for two consecutive managed
runs and for the direct restore path, on a workspace with a project
repository (commits, dirty edits, and a deleted file). Two tests for the
new validation.
-
`docs/guides/board-operator/execution-workspaces-and-runtime-services.md`:
one line about project repositories in the SSH round trip.

## Verification

- The new regression test fails on `master` (`expected 'backend
initial\n?? ../\n' to contain 'frontend initial'`) and passes with this
change.
- `PAPERCLIP_ENABLE_DARWIN_SSH_ENV_LAB=1 npx vitest run
packages/adapter-utils/src/ssh-fixture.test.ts
packages/adapter-utils/src/remote-managed-runtime.test.ts`: 32 passed,
with the sshd fixture running.
- `tsc --noEmit` passes for `packages/adapter-utils` and `server`, and
`pnpm -r typecheck` passes for the other workspaces. The Rust step of
`@paperclipai/paperclip-runner` did not run locally because `cargo` is
not installed.
- `node ./scripts/check-no-git-push.mjs` and `pnpm
check:module-boundaries` pass.
- `pnpm test:run` did not complete locally. Before it stopped, 5 tests
failed: 2 in `server/src/__tests__/workspace-runtime.test.ts` and 3 in
`server/src/__tests__/company-skills-service.test.ts`. The same 5 tests
also fail on the base commit `5edf55d73` without this change. CI runs
the full suite.
- `pnpm build` passes for all workspaces except
`@paperclipai/paperclip-runner` and `server`, because their build
compiles the Rust runner binary and `cargo` is not installed. `tsc
--noEmit` passes for `server`.
- Manual test on a self-hosted `v2026.916.1` instance with the same
change applied: a project with two repositories and an SSH environment.
Two runs on the same task passed. After each run, the second repository
keeps its `.git` on the host. On the remote host it is a Git checkout,
and the remote anchor ignores it.

## Risks

- Low risk. Workspaces without `.paperclip-repositories/` take the same
path as before, and the return value is unchanged for them.
- A workspace with an invalid entry under `.paperclip-repositories/` now
fails the SSH prepare step. The sandbox path already rejects such
entries.
- If one repository fails to restore, the restore stops, as in the
sandbox path. The remote run directory keeps the agent's work.
- Each project repository adds one bundle import and one restore per
run. The time grows with the number and size of the repositories.
- Out of scope: other nested `.git` directories (for example a vendored
checkout inside a repository) keep the existing SSH behavior.

## Model Used

- Provider and model: Anthropic Claude Opus 5.5 (`claude-opus-5-5`), in
Claude Code.
- Capabilities: extended thinking, tool use, and code execution. The
context window size was not recorded.
- Use: the model investigated the bug, wrote the change and the tests,
and ran the checks. A separate Claude Code agent reviewed the diff. The
author reviewed the change. The manual test ran on the author's
self-hosted instance.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
This commit is contained in:
Barış ÖZDEMİR authored and GitHub committed 2026-10-05 22:21:00 -07:00
1 parent bb73f2fe39
commit bf14f803d5
4 files changed
+396 -23

No files matched your search

@@ -138,6 +138,7 @@ fetches with it. Details and caveats:
Code state moves between runs through the local execution-workspace cwd alone — not through a git remote.
- Each run's prepare step bundles the local worktree to the run's remote dir over ssh, with no `git remote` configured.
- Other project repositories under `.paperclip-repositories/` make the same round trip. Each one is bundled and restored as its own Git checkout.
- The adapter's restore step at the end of the run writes any new remote commits back into the local worktree directly.
- Adapters must never `git push` from runtime code, and must never assume a remote exists.
- A failed restore is a run-level error and records `workspace_finalize=failed` on the execution workspace, which gates dependent issue wakes until the next successful finalize.
@@ -1,5 +1,5 @@
import path from "node:path";
import { GIT_ARCHIVE_EXCLUDES } from "./git-workspace-sync.js";
import { GIT_ARCHIVE_EXCLUDES, PROJECT_REPOSITORIES_DIR } from "./git-workspace-sync.js";
import {
type SshRemoteExecutionSpec,
prepareWorkspaceForSshExecution,
@@ -13,7 +13,7 @@ import {
type SandboxAdditionalSource,
type SandboxManagedRuntimeAssetRestoreContext,
} from "./sandbox-managed-runtime.js";
import { captureDirectorySnapshot } from "./workspace-restore-merge.js";
import { captureDirectorySnapshot, type DirectorySnapshot } from "./workspace-restore-merge.js";
import type { RuntimeProgressSink } from "./runtime-progress.js";
// The fixed heavy-directory excludes every referenced project drops,
@@ -145,13 +145,27 @@ export async function prepareRemoteManagedRuntime(input: {
workspaceExclude: input.workspaceExclude,
})
: null;
const projectRepositories = preparedWorkspace?.repositories ?? [];
const baselineSnapshot = preparedWorkspace
? await captureDirectorySnapshot(input.workspaceLocalDir, {
exclude: preparedWorkspace.gitBacked
? [...GIT_ARCHIVE_EXCLUDES, ".paperclip-runtime"]
? [
...GIT_ARCHIVE_EXCLUDES,
".paperclip-runtime",
...(projectRepositories.length > 0 ? [PROJECT_REPOSITORIES_DIR] : []),
]
: [".paperclip-runtime", ...(input.workspaceFileMode === "all" ? input.workspaceExclude ?? [] : [])],
})
: null;
const repositoryBaselines: Array<{ path: string; baselineSnapshot: DirectorySnapshot }> = [];
for (const repository of projectRepositories) {
repositoryBaselines.push({
path: repository,
baselineSnapshot: await captureDirectorySnapshot(path.join(input.workspaceLocalDir, repository), {
exclude: [...GIT_ARCHIVE_EXCLUDES, ".paperclip-runtime"],
}),
});
}
const assetDirs: Record<string, string> = {};
try {
@@ -177,6 +191,7 @@ export async function prepareRemoteManagedRuntime(input: {
baselineSnapshot,
restoreGitHistory: preparedWorkspace.gitBacked,
onProgress: input.onProgress,
repositories: repositoryBaselines,
});
}
throw error;
@@ -244,6 +259,7 @@ export async function prepareRemoteManagedRuntime(input: {
baselineSnapshot,
restoreGitHistory: preparedWorkspace.gitBacked,
onProgress,
repositories: repositoryBaselines,
});
}
for (const asset of input.assets ?? []) {
@@ -21,6 +21,16 @@ import {
import { prepareRemoteManagedRuntime } from "./remote-managed-runtime.js";
const SSH_FIXTURE_TEST_TIMEOUT_MS = 30_000;
const UNREACHABLE_SSH_SPEC = {
host: "127.0.0.1",
port: 1,
username: "nobody",
remoteWorkspacePath: "/nonexistent",
remoteCwd: "/nonexistent",
privateKey: null,
knownHosts: null,
strictHostKeyChecking: false,
} as const;
let sshEnvLabUnsupportedReason: string | null = null;
// One entry per fixture root directory, registered at creation time so
@@ -850,6 +860,205 @@ describe("ssh env-lab fixture", () => {
await expect(readFile(path.join(localRepo, "run-b.txt"), "utf8")).resolves.toBe("from run b\n");
}, SSH_FIXTURE_TEST_TIMEOUT_MS);
it("round-trips project repositories nested in .paperclip-repositories across consecutive SSH runs", async () => {
const rootDir = await createFixtureRootDir();
const statePath = path.join(rootDir, "state.json");
const localRepo = path.join(rootDir, "local-workspace");
const nestedRelative = ".paperclip-repositories/frontend-0123456789ab";
const nestedRepo = path.join(localRepo, nestedRelative);
await mkdir(localRepo, { recursive: true });
await git(localRepo, ["init"]);
await git(localRepo, ["checkout", "-b", "main"]);
await git(localRepo, ["config", "user.name", "Paperclip Test"]);
await git(localRepo, ["config", "user.email", "test@paperclip.dev"]);
await writeFile(path.join(localRepo, "backend.txt"), "backend base\n", "utf8");
await git(localRepo, ["add", "backend.txt"]);
await git(localRepo, ["commit", "-m", "backend initial"]);
await writeFile(path.join(localRepo, ".git", "info", "exclude"), "\n/.paperclip-repositories/\n", { flag: "a" });
await mkdir(nestedRepo, { recursive: true });
await git(nestedRepo, ["init"]);
await git(nestedRepo, ["checkout", "-b", "main"]);
await git(nestedRepo, ["config", "user.name", "Paperclip Test"]);
await git(nestedRepo, ["config", "user.email", "test@paperclip.dev"]);
await writeFile(path.join(nestedRepo, "frontend.txt"), "frontend base\n", "utf8");
await writeFile(path.join(nestedRepo, "obsolete.txt"), "tracked\n", "utf8");
await git(nestedRepo, ["add", "frontend.txt", "obsolete.txt"]);
await git(nestedRepo, ["commit", "-m", "frontend initial"]);
await writeFile(path.join(nestedRepo, "frontend.txt"), "frontend dirty local\n", "utf8");
await rm(path.join(nestedRepo, "obsolete.txt"));
const started = await startSshEnvLabFixtureOrSkip(statePath, "SSH nested project repositories test");
if (!started) return;
const config = await buildSshEnvLabFixtureConfig(started);
const spec = {
...config,
remoteCwd: started.workspaceDir,
} as const;
const first = await prepareRemoteManagedRuntime({
spec,
runId: "run-1",
adapterKey: "test-adapter",
workspaceLocalDir: localRepo,
});
const remoteNested = path.posix.join(first.workspaceRemoteDir, nestedRelative);
const remoteNestedStatus = await runSshCommand(
config,
`cd ${JSON.stringify(remoteNested)} && git log -1 --pretty=%s && git status --short`,
);
expect(remoteNestedStatus.stdout).toContain("frontend initial");
expect(remoteNestedStatus.stdout).toContain("M frontend.txt");
expect(remoteNestedStatus.stdout).toContain("D obsolete.txt");
const remoteAnchorStatus = await runSshCommand(
config,
`cd ${JSON.stringify(first.workspaceRemoteDir)} && git status --short --untracked-files=all`,
);
expect(remoteAnchorStatus.stdout).not.toContain(".paperclip-repositories");
await runSshCommand(
config,
[
`cd ${JSON.stringify(remoteNested)}`,
`git config user.name "Paperclip SSH"`,
`git config user.email "ssh@paperclip.dev"`,
`git add frontend.txt`,
`git commit -m "remote frontend update" >/dev/null`,
`printf "frontend remote dirty\\n" > frontend.txt`,
`cd ${JSON.stringify(first.workspaceRemoteDir)}`,
`git config user.name "Paperclip SSH"`,
`git config user.email "ssh@paperclip.dev"`,
`printf "backend remote\\n" > backend.txt`,
`git add backend.txt`,
`git commit -m "remote backend update" >/dev/null`,
].join(" && "),
{ timeoutMs: 30_000, maxBuffer: 256 * 1024 },
);
await first.restoreWorkspace();
await expect(stat(path.join(nestedRepo, ".git"))).resolves.toBeDefined();
expect(await git(nestedRepo, ["log", "-1", "--pretty=%s"])).toBe("remote frontend update");
await expect(readFile(path.join(nestedRepo, "frontend.txt"), "utf8")).resolves.toBe("frontend remote dirty\n");
expect(await git(localRepo, ["log", "-1", "--pretty=%s"])).toBe("remote backend update");
expect(await git(localRepo, ["status", "--short", "--untracked-files=all"])).not.toContain(".paperclip-repositories");
const second = await prepareRemoteManagedRuntime({
spec,
runId: "run-2",
adapterKey: "test-adapter",
workspaceLocalDir: localRepo,
});
const secondNestedLog = await runSshCommand(
config,
`cd ${JSON.stringify(path.posix.join(second.workspaceRemoteDir, nestedRelative))} && git log -1 --pretty=%s`,
);
expect(secondNestedLog.stdout.trim()).toBe("remote frontend update");
await second.restoreWorkspace();
await expect(stat(path.join(nestedRepo, ".git"))).resolves.toBeDefined();
await expect(readFile(path.join(nestedRepo, "frontend.txt"), "utf8")).resolves.toBe("frontend remote dirty\n");
}, SSH_FIXTURE_TEST_TIMEOUT_MS * 2);
it("restores project repositories through the direct SSH restore path", async () => {
const rootDir = await createFixtureRootDir();
const statePath = path.join(rootDir, "state.json");
const localRepo = path.join(rootDir, "local-workspace");
const nestedRelative = ".paperclip-repositories/frontend-0123456789ab";
const nestedRepo = path.join(localRepo, nestedRelative);
for (const repo of [localRepo, nestedRepo]) {
await mkdir(repo, { recursive: true });
await git(repo, ["init"]);
await git(repo, ["checkout", "-b", "main"]);
await git(repo, ["config", "user.name", "Paperclip Test"]);
await git(repo, ["config", "user.email", "test@paperclip.dev"]);
await writeFile(path.join(repo, "tracked.txt"), "base\n", "utf8");
await git(repo, ["add", "tracked.txt"]);
await git(repo, ["commit", "-m", "initial"]);
}
await writeFile(path.join(localRepo, ".git", "info", "exclude"), "\n/.paperclip-repositories/\n", { flag: "a" });
const started = await startSshEnvLabFixtureOrSkip(statePath, "SSH direct restore with project repositories test");
if (!started) return;
const config = await buildSshEnvLabFixtureConfig(started);
const spec = {
...config,
remoteCwd: started.workspaceDir,
} as const;
const prepared = await prepareWorkspaceForSshExecution({
spec,
localDir: localRepo,
remoteDir: started.workspaceDir,
});
expect(prepared).toEqual({ gitBacked: true, repositories: [nestedRelative] });
await runSshCommand(
config,
[
`cd ${JSON.stringify(path.posix.join(started.workspaceDir, nestedRelative))}`,
`git config user.name "Paperclip SSH"`,
`git config user.email "ssh@paperclip.dev"`,
`printf "remote\\n" > tracked.txt`,
`git commit -am "remote nested update" >/dev/null`,
].join(" && "),
{ timeoutMs: 30_000, maxBuffer: 256 * 1024 },
);
await restoreWorkspaceFromSshExecution({
spec,
localDir: localRepo,
remoteDir: started.workspaceDir,
repositories: (prepared.repositories ?? []).map((repository) => ({ path: repository })),
});
await expect(stat(path.join(nestedRepo, ".git"))).resolves.toBeDefined();
expect(await git(nestedRepo, ["log", "-1", "--pretty=%s"])).toBe("remote nested update");
await expect(readFile(path.join(nestedRepo, "tracked.txt"), "utf8")).resolves.toBe("remote\n");
expect(await git(localRepo, ["status", "--short", "--untracked-files=all"])).toBe("");
}, SSH_FIXTURE_TEST_TIMEOUT_MS * 2);
it("fails closed when .paperclip-repositories holds something other than a git checkout", async () => {
const rootDir = await createFixtureRootDir();
const localRepo = path.join(rootDir, "local-workspace");
await mkdir(path.join(localRepo, ".paperclip-repositories", "not-a-repo"), { recursive: true });
await git(localRepo, ["init"]);
await git(localRepo, ["checkout", "-b", "main"]);
await git(localRepo, ["config", "user.name", "Paperclip Test"]);
await git(localRepo, ["config", "user.email", "test@paperclip.dev"]);
await writeFile(path.join(localRepo, "tracked.txt"), "base\n", "utf8");
await git(localRepo, ["add", "tracked.txt"]);
await git(localRepo, ["commit", "-m", "initial"]);
await expect(prepareWorkspaceForSshExecution({ spec: UNREACHABLE_SSH_SPEC, localDir: localRepo, remoteDir: "/nonexistent" }))
.rejects.toThrow("Project repository is not a Git checkout: .paperclip-repositories/not-a-repo");
}, SSH_FIXTURE_TEST_TIMEOUT_MS);
it("rejects inconsistent project repository restores before any SSH connection", async () => {
const localDir = await createFixtureRootDir();
const repositoryBaseline = { exclude: [".git", ".git/*", ".paperclip-runtime"], entries: new Map() };
const restore = (input: Partial<Parameters<typeof restoreWorkspaceFromSshExecution>[0]>) =>
restoreWorkspaceFromSshExecution({
spec: UNREACHABLE_SSH_SPEC,
localDir,
remoteDir: "/nonexistent",
baselineSnapshot: { exclude: [".git", ".git/*", ".paperclip-runtime", ".paperclip-repositories"], entries: new Map() },
restoreGitHistory: true,
...input,
});
await expect(restore({ repositories: [{ path: "../outside", baselineSnapshot: repositoryBaseline }] }))
.rejects.toThrow("Invalid project repository path: ../outside");
await expect(restore({ repositories: [{ path: ".paperclip-repositories/frontend-0123456789ab" }] }))
.rejects.toThrow("Project repository has no workspace baseline: .paperclip-repositories/frontend-0123456789ab");
await expect(restore({
baselineSnapshot: { exclude: [".git", ".git/*", ".paperclip-runtime"], entries: new Map() },
repositories: [{ path: ".paperclip-repositories/frontend-0123456789ab", baselineSnapshot: repositoryBaseline }],
})).rejects.toThrow("Workspace baseline must exclude .paperclip-repositories");
}, SSH_FIXTURE_TEST_TIMEOUT_MS);
it("preserves nested per-run files across sequential SSH restores with stale baselines", async () => {
const rootDir = await createFixtureRootDir();
const statePath = path.join(rootDir, "state.json");
+167 -20
View File
@@ -9,6 +9,7 @@ import type { CommandManagedRuntimeRunner } from "./command-managed-runtime.js";
import {
createUnrelatedHistoryGraftCommit,
GIT_SYNC_COMMIT_IDENTITY_ARGS,
PROJECT_REPOSITORIES_DIR,
readSanitizedOriginRemoteUrl,
} from "./git-workspace-sync.js";
import type { RunProcessResult } from "./server-utils.js";
@@ -1071,6 +1072,91 @@ async function removeDeletedPathsOnSsh(input: {
});
}
const PROJECT_REPOSITORY_NAME_PATTERN = /^[a-zA-Z0-9_-]+$/;
async function isLocalGitRepositoryRoot(localDir: string): Promise<boolean> {
try {
const toplevel = await runLocalGit(localDir, ["rev-parse", "--show-toplevel"], {
timeout: 10_000,
maxBuffer: 16 * 1024,
});
const [directory, repository] = await Promise.all([
fs.realpath(localDir),
fs.realpath(toplevel.stdout.trim()),
]);
return directory === repository;
} catch {
return false;
}
}
async function listLocalProjectRepositories(localDir: string): Promise<string[]> {
const root = path.join(localDir, PROJECT_REPOSITORIES_DIR);
const rootStat = await fs.lstat(root).catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return null;
throw error;
});
if (!rootStat) return [];
if (!rootStat.isDirectory() || rootStat.isSymbolicLink()) {
throw new Error("Invalid project repositories directory");
}
const repositories: string[] = [];
for (const entry of await fs.readdir(root, { withFileTypes: true })) {
if (!entry.isDirectory() || !PROJECT_REPOSITORY_NAME_PATTERN.test(entry.name)) {
throw new Error("Invalid project repository directory");
}
const relative = `${PROJECT_REPOSITORIES_DIR}/${entry.name}`;
if (!(await isLocalGitRepositoryRoot(path.join(localDir, relative)))) {
throw new Error(`Project repository is not a Git checkout: ${relative}`);
}
repositories.push(relative);
}
return repositories.sort();
}
async function transportGitWorkspaceToSsh(input: {
spec: SshRemoteExecutionSpec;
localDir: string;
remoteDir: string;
snapshot: LocalGitWorkspaceSnapshot;
exclude?: string[];
onProgress?: RuntimeProgressSink;
progressLabel: string;
}): Promise<void> {
await importGitWorkspaceToSsh({
spec: input.spec,
localDir: input.localDir,
remoteDir: input.remoteDir,
snapshot: input.snapshot,
onProgress: input.onProgress,
});
await syncDirectoryToSsh({
spec: input.spec,
localDir: input.localDir,
remoteDir: input.remoteDir,
exclude: [".git", ".paperclip-runtime", ...(input.exclude ?? [])],
onProgress: input.onProgress,
progressLabel: input.progressLabel,
});
await removeDeletedPathsOnSsh({
spec: input.spec,
remoteDir: input.remoteDir,
deletedPaths: input.snapshot.deletedPaths,
});
}
async function excludeProjectRepositoriesOnSsh(spec: SshConnectionConfig, remoteDir: string): Promise<void> {
const pattern = `/${PROJECT_REPOSITORIES_DIR}/`;
const excludeFile = path.posix.join(remoteDir, ".git", "info", "exclude");
await runSshScript(
spec,
`mkdir -p ${shellQuote(path.posix.dirname(excludeFile))} && ` +
`{ grep -qxF ${shellQuote(pattern)} ${shellQuote(excludeFile)} 2>/dev/null || ` +
`printf '\\n%s\\n' ${shellQuote(pattern)} >> ${shellQuote(excludeFile)}; }`,
{ timeoutMs: 30_000 },
);
}
async function allocateLoopbackPort(host: string): Promise<number> {
return await new Promise<number>((resolve, reject) => {
const server = net.createServer();
@@ -1560,32 +1646,38 @@ export async function prepareWorkspaceForSshExecution(input: {
onProgress?: RuntimeProgressSink;
workspaceFileMode?: "all";
workspaceExclude?: string[];
}): Promise<{ gitBacked: boolean }> {
}): Promise<{ gitBacked: boolean; repositories?: string[] }> {
const remoteDir = input.remoteDir ?? input.spec.remoteCwd;
const gitSnapshot = input.workspaceFileMode === "all" ? null : await readLocalGitWorkspaceSnapshot(input.localDir);
if (gitSnapshot) {
await importGitWorkspaceToSsh({
const repositories = await listLocalProjectRepositories(input.localDir);
await transportGitWorkspaceToSsh({
spec: input.spec,
localDir: input.localDir,
remoteDir,
snapshot: gitSnapshot,
onProgress: input.onProgress,
});
await syncDirectoryToSsh({
spec: input.spec,
localDir: input.localDir,
remoteDir,
exclude: [".git", ".paperclip-runtime"],
exclude: repositories.length > 0 ? [PROJECT_REPOSITORIES_DIR] : [],
onProgress: input.onProgress,
progressLabel: "workspace",
});
await removeDeletedPathsOnSsh({
spec: input.spec,
remoteDir,
deletedPaths: gitSnapshot.deletedPaths,
});
return { gitBacked: true };
if (repositories.length > 0) {
await excludeProjectRepositoriesOnSsh(input.spec, remoteDir);
}
for (const relative of repositories) {
const localDir = path.join(input.localDir, relative);
const snapshot = await readLocalGitWorkspaceSnapshot(localDir);
if (!snapshot) throw new Error(`Cannot read the Git state of project repository: ${relative}`);
await transportGitWorkspaceToSsh({
spec: input.spec,
localDir,
remoteDir: path.posix.join(remoteDir, relative),
snapshot,
onProgress: input.onProgress,
progressLabel: relative,
});
}
return { gitBacked: true, ...(repositories.length > 0 ? { repositories } : {}) };
}
await clearRemoteDirectory({
@@ -1611,8 +1703,62 @@ export async function restoreWorkspaceFromSshExecution(input: {
baselineSnapshot?: DirectorySnapshot;
restoreGitHistory?: boolean;
onProgress?: RuntimeProgressSink;
repositories?: Array<{ path: string; baselineSnapshot?: DirectorySnapshot }>;
}): Promise<void> {
const remoteDir = input.remoteDir ?? input.spec.remoteCwd;
const repositories = input.repositories ?? [];
for (const repository of repositories) {
if (
path.posix.dirname(repository.path) !== PROJECT_REPOSITORIES_DIR ||
!PROJECT_REPOSITORY_NAME_PATTERN.test(path.posix.basename(repository.path))
) {
throw new Error(`Invalid project repository path: ${repository.path}`);
}
if (input.baselineSnapshot && !repository.baselineSnapshot) {
throw new Error(`Project repository has no workspace baseline: ${repository.path}`);
}
}
if (
input.baselineSnapshot &&
repositories.length > 0 &&
!input.baselineSnapshot.exclude.includes(PROJECT_REPOSITORIES_DIR)
) {
throw new Error(`Workspace baseline must exclude ${PROJECT_REPOSITORIES_DIR} when project repositories are restored separately`);
}
for (const repository of repositories) {
await restoreWorkspaceRootFromSsh({
spec: input.spec,
localDir: path.join(input.localDir, repository.path),
remoteDir: path.posix.join(remoteDir, repository.path),
baselineSnapshot: repository.baselineSnapshot,
restoreGitHistory: input.restoreGitHistory,
onProgress: input.onProgress,
progressLabel: repository.path,
});
}
await restoreWorkspaceRootFromSsh({
spec: input.spec,
localDir: input.localDir,
remoteDir,
baselineSnapshot: input.baselineSnapshot,
restoreGitHistory: input.restoreGitHistory,
onProgress: input.onProgress,
progressLabel: "workspace",
hasProjectRepositories: repositories.length > 0,
});
}
async function restoreWorkspaceRootFromSsh(input: {
spec: SshRemoteExecutionSpec;
localDir: string;
remoteDir: string;
baselineSnapshot?: DirectorySnapshot;
restoreGitHistory?: boolean;
onProgress?: RuntimeProgressSink;
progressLabel: string;
hasProjectRepositories?: boolean;
}): Promise<void> {
const remoteDir = input.remoteDir;
if (input.baselineSnapshot) {
const stagingDir = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-ssh-sync-back-"));
const importedRef = input.restoreGitHistory
@@ -1635,7 +1781,7 @@ export async function restoreWorkspaceFromSshExecution(input: {
localDir: stagingDir,
exclude: input.baselineSnapshot.exclude,
onProgress: input.onProgress,
progressLabel: "workspace",
progressLabel: input.progressLabel,
});
await mergeDirectoryWithBaseline({
baseline: input.baselineSnapshot,
@@ -1666,6 +1812,7 @@ export async function restoreWorkspaceFromSshExecution(input: {
const gitSnapshot = await readLocalGitWorkspaceSnapshot(input.localDir);
if (gitSnapshot) {
const projectRepositoryEntries = input.hasProjectRepositories ? [PROJECT_REPOSITORIES_DIR] : [];
await exportGitWorkspaceFromSsh({
spec: input.spec,
remoteDir,
@@ -1676,10 +1823,10 @@ export async function restoreWorkspaceFromSshExecution(input: {
spec: input.spec,
remoteDir,
localDir: input.localDir,
exclude: [".git", ".paperclip-runtime"],
preserveLocalEntries: [".git"],
exclude: [".git", ".paperclip-runtime", ...projectRepositoryEntries],
preserveLocalEntries: [".git", ...projectRepositoryEntries],
onProgress: input.onProgress,
progressLabel: "workspace",
progressLabel: input.progressLabel,
});
return;
}
@@ -1690,7 +1837,7 @@ export async function restoreWorkspaceFromSshExecution(input: {
localDir: input.localDir,
exclude: [".paperclip-runtime"],
onProgress: input.onProgress,
progressLabel: "workspace",
progressLabel: input.progressLabel,
});
}