mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-22 20:29:18 +01:00
fix: provision all project repositories for local and sandbox tasks (#13442)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Projects can now attach several source repositories. > - Task preparation still treated these sources as alternative workspaces. > - Sandbox sync preserved Git history only for the selected repository. > - A task needs every attached repository to complete work across the project. > - This pull request prepares all distinct project repositories and preserves their separate Git histories through sandbox restore. ## Linked Issues or Issue Description **What happened?** A user reported that a project with two repositories received only the first repository in Daytona. Repository-only project rows also reached the agent with null local paths. Managed checkouts with matching repository names could resolve to the same directory. **Expected behavior** Local and sandbox tasks receive every distinct repository attached to their project. Repository-only sources work without preconfigured local folders. Each repository keeps its own Git history and working files. **Steps to reproduce** 1. Create a project with two repository sources and no local folder paths. 2. Assign a task to the project and run it in Daytona. 3. Inspect the task workspace and the repository paths exposed to the agent. 4. Observe that the original implementation supplies only the selected checkout. Related change: #13010 added multiple repository selection. The open repository-catalog proposals #11234 and #11228 cover a different data model. This fix uses the existing project workspaces. ## What Changed - Materialize each additional distinct repository as an editable checkout inside the task root. Seed configured local sources with their current working files and retain task edits across runs. - Pass materialized repository paths to local agents and native sandbox task prompts. Apply existing run-scoped Git credentials to each remote clone. - Preserve each repository's Git history, dirty files, and restore baseline during sandbox staging and durable recovery. Apply each repository's ignore rules and the operator's workspace exclusions. - Keep same-name managed repositories in separate directories. Report additional clone failures before the task starts. - Add task-level, checkout, sandbox round-trip, environment-hint, and recovery-descriptor regression coverage. Document checkout and restore behavior. ## Verification - Red: the original implementation fails the sandbox test because the second repository has no Git directory. It also fails the same-name checkout test and both real-database task tests because repository hints have no local path. - Green: focused tests pass for one and two repository-only sources, local source edits, clone failures, per-repository credentials, separate Git histories, ignored files, and recovery from remote or durable seed state. - Live Daytona smoke passed with two disposable repositories through the production provider sync functions. Both repositories arrived with Git history. Commits from both restored locally. Ignored files stayed excluded. The disposable sandbox was deleted. - Passed on final commit `93ab76763`: `pnpm -r typecheck` and `pnpm build`. - Final focused coverage: 254 assertions across the six changed test areas passed across the serial run and an isolated rerun of the existing process-kill timing test. The live Daytona smoke also passed. - The local `pnpm test:run` overlapped source edits and retained stale transformed code. Its first phase reported 12,240 passed assertions, nine failed assertions, three hook failures, and one worker error; later phases did not run locally. This run is not claimed as green. Fresh focused tests verify the changes, and every general/workspace and serialized-server CI shard passes on the final commit. - Final CI is green on `93ab76763`: all test shards, all three browser shards, typecheck, build, runner verification, canary dry run, and security checks. The initial unrelated chat-delivery browser timing failure passed in the final CI run. Optional Storybook visual checks were skipped. - Greptile is 5/5 on the final commit with no unresolved review threads. Its checkout-race finding was reproduced with a failing test, fixed, and rechecked. ## Risks - Additional repositories need disk space and clone time. Access failure for an attached repository stops preparation. - Additional checkouts live under `.paperclip-repositories/` and keep independent histories. Changes stay in those task copies; they do not overwrite configured source folders. - Detached or reconfigured repository copies are retained under `.paperclip-runtime/detached-repositories/`. Sandbox recovery retains per-repository merge baselines. - No database migration, UI contract change, or new credential delegation is required. Referenced projects retain their separate read-only behavior. ## Model Used OpenAI Codex, based on GPT-6, with repository inspection, code execution, and tool use. The runtime does not expose a more specific model deployment ID or context-window size. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
0e9b24c821
commit
8f1905d34d
12 files changed
+504
-21
No files matched your search
@@ -4,7 +4,7 @@ import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import { ensureManagedProjectWorkspace } from "../services/heartbeat.ts";
|
||||
import { ensureManagedProjectWorkspace, prepareProjectRepositoryWorkspaces } from "../services/heartbeat.ts";
|
||||
import { buildGitAuthInvocation, GIT_CREDENTIAL_TOKEN_ENV_KEY } from "../services/git-credentials.ts";
|
||||
import { sanitizeRuntimeServiceBaseEnv } from "../services/workspace-runtime.ts";
|
||||
import { resolveManagedProjectWorkspaceDir } from "../home-paths.ts";
|
||||
@@ -38,6 +38,108 @@ async function createLocalSourceRepo() {
|
||||
}
|
||||
|
||||
describe("ensureManagedProjectWorkspace clone credentials", () => {
|
||||
it("materializes every repository-only project row inside the task workspace and reuses local edits", async () => {
|
||||
const first = await createLocalSourceRepo();
|
||||
const second = await createLocalSourceRepo();
|
||||
try {
|
||||
const anchor = await ensureManagedProjectWorkspace({ companyId: "repo-only", projectId: "two", repoUrl: first });
|
||||
await execFile("git", ["checkout", "-b", "project-branch"], { cwd: second });
|
||||
const resolveGitAuth = vi.fn(async () => null);
|
||||
const input = {
|
||||
cwd: anchor.cwd, anchorRepoUrl: first, resolveGitAuth,
|
||||
workspaces: [
|
||||
{ id: "first", repoUrl: first, repoRef: null },
|
||||
{ id: "duplicate-first", repoUrl: first, repoRef: null },
|
||||
{ id: "second", repoUrl: second, repoRef: "project-branch" },
|
||||
],
|
||||
};
|
||||
const additional = await prepareProjectRepositoryWorkspaces(input);
|
||||
expect(additional).toHaveLength(1);
|
||||
expect(additional[0]!.workspaceId).toBe("second");
|
||||
expect(path.relative(anchor.cwd, additional[0]!.cwd)).toMatch(/^\.paperclip-repositories\//);
|
||||
expect((await execFile("git", ["branch", "--show-current"], { cwd: additional[0]!.cwd })).stdout.trim()).toBe("project-branch");
|
||||
expect(resolveGitAuth).toHaveBeenCalledWith(second);
|
||||
await fs.writeFile(path.join(additional[0]!.cwd, "README.md"), "work in progress");
|
||||
expect(await prepareProjectRepositoryWorkspaces(input)).toEqual(additional);
|
||||
expect(await fs.readFile(path.join(additional[0]!.cwd, "README.md"), "utf8")).toBe("work in progress");
|
||||
expect((await execFile("git", ["status", "--porcelain"], { cwd: anchor.cwd })).stdout).toBe("");
|
||||
} finally {
|
||||
await Promise.all([first, second].map((cwd) => fs.rm(cwd, { recursive: true, force: true })));
|
||||
}
|
||||
});
|
||||
|
||||
it("fails task preparation when any attached repository cannot be cloned", async () => {
|
||||
const first = await createLocalSourceRepo();
|
||||
try {
|
||||
const anchor = await ensureManagedProjectWorkspace({ companyId: "repo-failure", projectId: "two", repoUrl: first });
|
||||
await expect(prepareProjectRepositoryWorkspaces({
|
||||
cwd: anchor.cwd, anchorRepoUrl: first,
|
||||
workspaces: [{ id: "missing", repoUrl: path.join(first, "missing.git"), repoRef: null }],
|
||||
})).rejects.toThrow("Failed to prepare managed checkout");
|
||||
} finally {
|
||||
await fs.rm(first, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("seeds a configured second local checkout with its uncommitted work and ignores", async () => {
|
||||
const first = await createLocalSourceRepo();
|
||||
const second = await createLocalSourceRepo();
|
||||
try {
|
||||
const anchor = await ensureManagedProjectWorkspace({ companyId: "local-project", projectId: "two", repoUrl: first });
|
||||
await fs.writeFile(path.join(second, "README.md"), "local changes");
|
||||
await fs.writeFile(path.join(second, ".gitignore"), "secret.txt\n");
|
||||
await fs.writeFile(path.join(second, "secret.txt"), "private");
|
||||
const [repo] = await prepareProjectRepositoryWorkspaces({
|
||||
cwd: anchor.cwd, anchorRepoUrl: first,
|
||||
workspaces: [{ id: "second", cwd: second, repoUrl: "https://github.com/example/backend.git", repoRef: null }],
|
||||
});
|
||||
expect(await fs.readFile(path.join(repo!.cwd, "README.md"), "utf8")).toBe("local changes");
|
||||
await expect(fs.stat(path.join(repo!.cwd, "secret.txt"))).rejects.toMatchObject({ code: "ENOENT" });
|
||||
expect((await execFile("git", ["remote", "get-url", "origin"], { cwd: repo!.cwd })).stdout.trim()).toBe("https://github.com/example/backend.git");
|
||||
} finally {
|
||||
await Promise.all([first, second].map((cwd) => fs.rm(cwd, { recursive: true, force: true })));
|
||||
}
|
||||
});
|
||||
it("keeps different repositories with the same name separate within one project", async () => {
|
||||
const first = await createLocalSourceRepo();
|
||||
const second = await createLocalSourceRepo();
|
||||
try {
|
||||
const results = await Promise.all([first, second].map((repoUrl) => ensureManagedProjectWorkspace({
|
||||
companyId: "company-multiple", projectId: "project-multiple", repoUrl,
|
||||
})));
|
||||
expect(results[0]!.cwd).not.toBe(results[1]!.cwd);
|
||||
for (let i = 0; i < results.length; i++) {
|
||||
const origin = await execFile("git", ["remote", "get-url", "origin"], { cwd: results[i]!.cwd });
|
||||
expect(origin.stdout.trim()).toBe([first, second][i]);
|
||||
}
|
||||
} finally {
|
||||
await Promise.all([first, second].map((cwd) => fs.rm(cwd, { recursive: true, force: true })));
|
||||
}
|
||||
});
|
||||
it("rechecks the repository when another process wins the checkout rename", async () => {
|
||||
const first = await createLocalSourceRepo();
|
||||
const second = await createLocalSourceRepo();
|
||||
const companyId = "cross-process-race";
|
||||
const projectId = "same-name";
|
||||
const sharedCwd = resolveManagedProjectWorkspaceDir({ companyId, projectId });
|
||||
try {
|
||||
// A different process does not share managedCheckoutMaterializations. Publish
|
||||
// its completed checkout after this caller chose its destination, before rename.
|
||||
const resolveGitAuth = vi.fn(async () => {
|
||||
if (!(await fs.stat(sharedCwd).catch(() => null))) {
|
||||
await execFile("git", ["clone", second, sharedCwd]);
|
||||
}
|
||||
return null;
|
||||
});
|
||||
const result = await ensureManagedProjectWorkspace({ companyId, projectId, repoUrl: first, resolveGitAuth });
|
||||
expect((await execFile("git", ["remote", "get-url", "origin"], { cwd: result.cwd })).stdout.trim()).toBe(first);
|
||||
expect((await execFile("git", ["remote", "get-url", "origin"], { cwd: sharedCwd })).stdout.trim()).toBe(second);
|
||||
expect(result.cwd).not.toBe(sharedCwd);
|
||||
} finally {
|
||||
await Promise.all([first, second].map((cwd) => fs.rm(cwd, { recursive: true, force: true })));
|
||||
}
|
||||
});
|
||||
|
||||
it("clones exactly as before when no auth provider is configured", async () => {
|
||||
const sourceRepo = await createLocalSourceRepo();
|
||||
try {
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { mkdtemp, mkdir, writeFile, readFile, realpath, rm } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import { agents, companies, createDb, issues, projects, projectWorkspaces } from "@paperclipai/db";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
import { heartbeatService } from "../services/heartbeat.ts";
|
||||
import { drainHeartbeatRunsToQuiescence } from "./helpers/drain-heartbeat-runs.js";
|
||||
|
||||
const execute = vi.hoisted(() => vi.fn(async (_input: any) => ({ exitCode: 0, signal: null, timedOut: false })));
|
||||
vi.mock("../adapters/index.js", () => ({
|
||||
getServerAdapter: () => ({ type: "codex_local", execute, supportsLocalAgentJwt: false }),
|
||||
findActiveServerAdapter: () => ({ type: "codex_local", execute, supportsLocalAgentJwt: false }),
|
||||
runningProcesses: new Map(),
|
||||
}));
|
||||
|
||||
const support = await getEmbeddedPostgresTestSupport();
|
||||
const suite = support.supported ? describe : describe.skip;
|
||||
suite("task project repository provisioning", () => {
|
||||
let database: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>>;
|
||||
let db: ReturnType<typeof createDb>;
|
||||
let root: string;
|
||||
let heartbeat: ReturnType<typeof heartbeatService>;
|
||||
beforeAll(async () => {
|
||||
root = await mkdtemp(path.join(os.tmpdir(), "paperclip-project-repos-"));
|
||||
vi.stubEnv("PAPERCLIP_HOME", path.join(root, "home"));
|
||||
vi.stubEnv("PAPERCLIP_MULTI_PROJECT_WORKSPACE_SYNC", "false");
|
||||
database = await startEmbeddedPostgresTestDatabase("project-repositories");
|
||||
db = createDb(database.connectionString);
|
||||
heartbeat = heartbeatService(db);
|
||||
execute.mockImplementation(async (input) => {
|
||||
await db.update(issues).set({ status: "done" }).where(eq(issues.id, input.context.issueId));
|
||||
return { exitCode: 0, signal: null, timedOut: false };
|
||||
});
|
||||
}, 30_000);
|
||||
afterAll(async () => {
|
||||
if (db && heartbeat) await drainHeartbeatRunsToQuiescence(db, heartbeat);
|
||||
await db?.$client.end({ timeout: 5 });
|
||||
await database?.cleanup();
|
||||
vi.unstubAllEnvs();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}, 60_000);
|
||||
|
||||
it.each([1, 2])("gives a task all %i repositories without any configured local folders", async (count) => {
|
||||
const companyId = randomUUID();
|
||||
const projectId = randomUUID();
|
||||
const agentId = randomUUID();
|
||||
const issueId = randomUUID();
|
||||
const repositoryRows = [];
|
||||
for (let index = 0; index < count; index++) {
|
||||
const source = path.join(root, companyId, `source-${index}`);
|
||||
await mkdir(source, { recursive: true });
|
||||
const git = (...args: string[]) => execFileSync("git", args, { cwd: source, stdio: "ignore" });
|
||||
git("init", "-b", "main");
|
||||
await writeFile(path.join(source, "README.md"), `repository ${index}`);
|
||||
git("add", ".");
|
||||
git("-c", "user.name=Test", "-c", "user.email=test@example.com", "commit", "-m", "seed");
|
||||
repositoryRows.push({ id: randomUUID(), companyId, projectId, name: `Repo ${index}`, sourceType: "git_repo", repoUrl: pathToFileURL(source).href, cwd: null, isPrimary: index === 0 });
|
||||
}
|
||||
await db.insert(companies).values({ id: companyId, name: "Repo test", issuePrefix: `R${companyId.slice(0, 6)}`, defaultResponsibleUserId: "responsible-user" });
|
||||
await db.insert(projects).values({ id: projectId, companyId, name: "Multi-repo", status: "in_progress" });
|
||||
await db.insert(projectWorkspaces).values(repositoryRows);
|
||||
await db.insert(agents).values({ id: agentId, companyId, name: "Test", role: "engineer", status: "idle", adapterType: "codex_local", adapterConfig: {}, runtimeConfig: {}, permissions: {} });
|
||||
await db.insert(issues).values({ id: issueId, companyId, projectId, title: "Use project repositories", status: "todo", assigneeAgentId: agentId });
|
||||
const run = await heartbeat.wakeup(agentId, { source: "on_demand", triggerDetail: "manual", contextSnapshot: { issueId, projectId } });
|
||||
expect(run).not.toBeNull();
|
||||
await vi.waitFor(async () => {
|
||||
const latest = await heartbeat.getRun(run!.id);
|
||||
expect({ status: latest?.status, error: latest?.error }).toEqual({ status: "succeeded", error: null });
|
||||
}, { timeout: 15_000 });
|
||||
const input = execute.mock.calls.find(([ctx]) => ctx.runId === run!.id)![0];
|
||||
const hints = input.context.paperclipWorkspaces as Array<{ workspaceId: string; cwd: string }>;
|
||||
for (let index = 0; index < count; index++) {
|
||||
const hint = hints.find((entry) => entry.workspaceId === repositoryRows[index]!.id);
|
||||
expect(hint?.cwd).toBeTruthy();
|
||||
expect(await readFile(path.join(hint!.cwd, "README.md"), "utf8")).toBe(`repository ${index}`);
|
||||
expect(execFileSync("git", ["rev-parse", "--show-toplevel"], { cwd: hint!.cwd, encoding: "utf8" }).trim()).toBe(await realpath(hint!.cwd));
|
||||
}
|
||||
}, 25_000);
|
||||
});
|
||||
@@ -134,7 +134,7 @@ describe("native workspace sync durable metadata", () => {
|
||||
).toThrow("native_workspace_sync_descriptor_digest_invalid");
|
||||
});
|
||||
|
||||
it("writes one immutable descriptor when the same state is replayed", async () => {
|
||||
it.each([false, true])("writes one immutable descriptor when the same state is replayed (multiple repositories: %s)", async (multipleRepositories) => {
|
||||
const paperclipHome = await mkdtemp(
|
||||
path.join(os.tmpdir(), "paperclip-native-workspace-sync-"),
|
||||
);
|
||||
@@ -164,7 +164,12 @@ describe("native workspace sync durable metadata", () => {
|
||||
state: "prepared" as const,
|
||||
baselineSha256: directorySnapshotSha256(baseline),
|
||||
baseline: serializeDirectorySnapshot(baseline),
|
||||
gitSnapshot: null,
|
||||
gitSnapshot: multipleRepositories ? {
|
||||
headCommit: "a".repeat(40), branchName: "main", overlayPaths: [], deletedPaths: [], ignoredPaths: [],
|
||||
repositories: [{ path: ".paperclip-repositories/backend", snapshot: {
|
||||
headCommit: "b".repeat(40), branchName: "backend-work", overlayPaths: ["dirty.txt"], deletedPaths: [], ignoredPaths: ["secret.txt"],
|
||||
} }],
|
||||
} : null,
|
||||
seed: null,
|
||||
createdAt: "2026-01-01T00:00:00.000Z",
|
||||
finalizedAt: null,
|
||||
|
||||
@@ -28,6 +28,8 @@ import { buildHeartbeatRunStatusLiveEventPayload } from "./heartbeat-run-status-
|
||||
export { buildHeartbeatRunStatusLiveEventPayload } from "./heartbeat-run-status-payload.js";
|
||||
import { buildExecutionContinuation } from "./execution-continuation.js";
|
||||
import { renderPaperclipWakePrompt } from "@paperclipai/adapter-utils/server-utils";
|
||||
import { PROJECT_REPOSITORIES_DIR, readGitWorkspaceSnapshot } from "@paperclipai/adapter-utils/git-workspace-sync";
|
||||
import { captureDirectorySnapshot, mergeDirectoryWithBaseline } from "@paperclipai/adapter-utils/workspace-restore-merge";
|
||||
import { initializeRunIdentity, explicitOperatorRunIdentity } from "./run-identity.js";
|
||||
import {
|
||||
assertDurableChatWakeupReceipt,
|
||||
@@ -2372,24 +2374,49 @@ export async function ensureManagedProjectWorkspace(input: {
|
||||
/** Optional git credential source for cloning private repos; null/absent preserves ambient behavior. */
|
||||
resolveGitAuth?: GitRemoteAuthProvider | null;
|
||||
}): Promise<{ cwd: string; warning: string | null }> {
|
||||
const cwd = resolveManagedProjectWorkspaceDir({
|
||||
const defaultCwd = resolveManagedProjectWorkspaceDir({
|
||||
companyId: input.companyId,
|
||||
projectId: input.projectId,
|
||||
repoName: deriveRepoNameFromRepoUrl(input.repoUrl),
|
||||
});
|
||||
let cwd = defaultCwd;
|
||||
if (input.repoUrl && await fs.stat(path.join(cwd, ".git")).catch(() => null)) {
|
||||
const origin = await execFile("git", ["-C", cwd, "remote", "get-url", "origin"], { timeout: 10_000 })
|
||||
.then((result) => result.stdout.trim()).catch(() => null);
|
||||
if (origin && origin !== input.repoUrl) {
|
||||
cwd = `${cwd}-${createHash("sha256").update(input.repoUrl).digest("hex").slice(0, 12)}`;
|
||||
}
|
||||
}
|
||||
const inFlight = managedCheckoutMaterializations.get(cwd);
|
||||
if (inFlight) return inFlight;
|
||||
if (inFlight) {
|
||||
await inFlight;
|
||||
// The winner may have cloned a different repository with the same basename.
|
||||
return ensureManagedProjectWorkspace(input);
|
||||
}
|
||||
const attempt = materializeManagedProjectWorkspace(cwd, input).finally(() => {
|
||||
managedCheckoutMaterializations.delete(cwd);
|
||||
});
|
||||
managedCheckoutMaterializations.set(cwd, attempt);
|
||||
return attempt;
|
||||
const result = await attempt;
|
||||
if (input.repoUrl) {
|
||||
// A different server process can publish a same-name checkout between the
|
||||
// initial origin check and the atomic rename. Never adopt its other repo.
|
||||
const origin = await execFile("git", ["-C", cwd, "remote", "get-url", "origin"], { timeout: 10_000 })
|
||||
.then((value) => value.stdout.trim()).catch(() => null);
|
||||
if (origin && origin !== input.repoUrl) {
|
||||
if (cwd !== defaultCwd) throw new Error("Managed checkout origin does not match the requested repository");
|
||||
return ensureManagedProjectWorkspace(input);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
async function materializeManagedProjectWorkspace(
|
||||
cwd: string,
|
||||
input: {
|
||||
repoUrl: string | null;
|
||||
repoRef?: string | null;
|
||||
localSource?: string | null;
|
||||
resolveGitAuth?: GitRemoteAuthProvider | null;
|
||||
},
|
||||
): Promise<{ cwd: string; warning: string | null }> {
|
||||
@@ -2427,14 +2454,14 @@ async function materializeManagedProjectWorkspace(
|
||||
// is never created in a partial state and never removed on failure, so a concurrent
|
||||
// materialization (another process, or a run racing this one) can neither adopt a broken
|
||||
// checkout nor lose its own completed one.
|
||||
const auth = input.resolveGitAuth
|
||||
const auth = input.resolveGitAuth && !input.localSource
|
||||
? await input.resolveGitAuth(input.repoUrl)
|
||||
: null;
|
||||
const cloneTmpDir = await fs.mkdtemp(`${cwd}.clone-`);
|
||||
try {
|
||||
await execFile(
|
||||
"git",
|
||||
[...(auth?.configArgs ?? []), "clone", input.repoUrl, cloneTmpDir],
|
||||
[...(auth?.configArgs ?? []), "clone", "--no-hardlinks", "--", input.localSource ?? input.repoUrl, cloneTmpDir],
|
||||
{
|
||||
env: {
|
||||
// Spread order matters: the sanitizer strips PAPERCLIP_*, which would remove the
|
||||
@@ -2448,6 +2475,15 @@ async function materializeManagedProjectWorkspace(
|
||||
timeout: MANAGED_WORKSPACE_GIT_CLONE_TIMEOUT_MS,
|
||||
},
|
||||
);
|
||||
if (input.localSource) {
|
||||
const snapshot = await readGitWorkspaceSnapshot(input.localSource, false);
|
||||
if (!snapshot) throw new Error("Configured repository folder is not a Git checkout");
|
||||
const baseline = await captureDirectorySnapshot(cloneTmpDir, { exclude: [".git", ".paperclip-runtime", PROJECT_REPOSITORIES_DIR, ...snapshot.ignoredPaths] });
|
||||
await mergeDirectoryWithBaseline({ baseline, sourceDir: input.localSource, targetDir: cloneTmpDir });
|
||||
await execFile("git", ["-C", cloneTmpDir, "remote", "set-url", "origin", input.repoUrl], { timeout: 10_000 });
|
||||
} else if (input.repoRef) {
|
||||
await execFile("git", ["-C", cloneTmpDir, "checkout", input.repoRef], { timeout: MANAGED_WORKSPACE_GIT_CLONE_TIMEOUT_MS });
|
||||
}
|
||||
} catch (error) {
|
||||
await fs
|
||||
.rm(cloneTmpDir, { recursive: true, force: true })
|
||||
@@ -2484,6 +2520,59 @@ async function materializeManagedProjectWorkspace(
|
||||
return { cwd, warning: null };
|
||||
}
|
||||
|
||||
/** Keep every distinct project repository inside the task's writable/synced root. */
|
||||
export async function prepareProjectRepositoryWorkspaces(input: {
|
||||
cwd: string;
|
||||
anchorRepoUrl: string | null;
|
||||
workspaces: Array<Pick<typeof projectWorkspaces.$inferSelect, "id" | "repoUrl" | "repoRef"> & { cwd?: string | null }>;
|
||||
resolveGitAuth?: GitRemoteAuthProvider | null;
|
||||
}): Promise<Array<{ workspaceId: string; cwd: string; repoUrl: string; repoRef: string | null }>> {
|
||||
const identity = (url: string) => url.trim().replace(/\.git\/?$/, "").replace(/\/$/, "");
|
||||
const seen = new Set(input.anchorRepoUrl ? [identity(input.anchorRepoUrl)] : []);
|
||||
const selected = input.workspaces.filter((workspace) => {
|
||||
if (!workspace.repoUrl || seen.has(identity(workspace.repoUrl))) return false;
|
||||
seen.add(identity(workspace.repoUrl));
|
||||
return true;
|
||||
});
|
||||
const root = path.join(input.cwd, PROJECT_REPOSITORIES_DIR);
|
||||
if (selected.length === 0 && !(await fs.lstat(root).catch(() => null))) return [];
|
||||
await fs.mkdir(root, { recursive: true });
|
||||
if (await fs.realpath(root) !== path.join(await fs.realpath(input.cwd), PROJECT_REPOSITORIES_DIR)) {
|
||||
throw new Error("Project repositories directory escapes the task workspace");
|
||||
}
|
||||
const excludePath = await execFile("git", ["-C", input.cwd, "rev-parse", "--git-path", "info/exclude"], { timeout: 10_000 })
|
||||
.then((result) => path.resolve(input.cwd, result.stdout.trim()));
|
||||
const exclude = await fs.readFile(excludePath, "utf8").catch(() => "");
|
||||
if (!exclude.split(/\r?\n/).includes(`/${PROJECT_REPOSITORIES_DIR}/`)) {
|
||||
await fs.mkdir(path.dirname(excludePath), { recursive: true });
|
||||
await fs.appendFile(excludePath, `\n/${PROJECT_REPOSITORIES_DIR}/\n`);
|
||||
}
|
||||
const results = [];
|
||||
for (const workspace of selected) {
|
||||
const repoUrl = workspace.repoUrl!;
|
||||
const name = (deriveRepoNameFromRepoUrl(repoUrl) ?? "repo").replace(/[^a-zA-Z0-9_-]/g, "-");
|
||||
const key = `${name}-${createHash("sha256").update(JSON.stringify([workspace.id, identity(repoUrl), workspace.repoRef, workspace.cwd ?? null])).digest("hex").slice(0, 12)}`;
|
||||
const cwd = path.join(root, key);
|
||||
const existing = await fs.lstat(cwd).catch(() => null);
|
||||
if (existing && (!existing.isDirectory() || existing.isSymbolicLink())) throw new Error("Invalid project repository checkout path");
|
||||
const localSource = workspace.cwd && workspace.cwd !== REPO_ONLY_CWD_SENTINEL
|
||||
&& await fs.stat(workspace.cwd).then((entry) => entry.isDirectory()).catch(() => false)
|
||||
? workspace.cwd : null;
|
||||
const result = await materializeManagedProjectWorkspace(cwd, { repoUrl, repoRef: workspace.repoRef, localSource, resolveGitAuth: input.resolveGitAuth });
|
||||
if (result.warning) throw new Error(result.warning);
|
||||
results.push({ workspaceId: workspace.id, cwd, repoUrl, repoRef: workspace.repoRef });
|
||||
}
|
||||
// Retain detached checkout work outside the synchronized repository set.
|
||||
const active = new Set(results.map((repo) => path.basename(repo.cwd)));
|
||||
for (const entry of await fs.readdir(root)) {
|
||||
if (active.has(entry) || entry.includes(".clone-")) continue;
|
||||
const retained = path.join(input.cwd, ".paperclip-runtime", "detached-repositories", randomUUID());
|
||||
await fs.mkdir(path.dirname(retained), { recursive: true });
|
||||
await fs.rename(path.join(root, entry), retained);
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve one project workspace row to a usable cwd. The anchor path and each additional
|
||||
* referenced project share this step: use the configured cwd when present, otherwise clone or
|
||||
@@ -21576,6 +21665,25 @@ export function heartbeatService(
|
||||
);
|
||||
}
|
||||
await bindIssueToPersistedExecutionWorkspace(persistedExecutionWorkspace);
|
||||
const projectRepositoryPaths: string[] = [];
|
||||
if (executionWorkspace.projectId && resolvedWorkspace.source === "project_primary" && !resolvedWorkspace.baseCwdFallback) {
|
||||
const repositoryRows = await db.select().from(projectWorkspaces).where(and(
|
||||
eq(projectWorkspaces.companyId, agent.companyId),
|
||||
eq(projectWorkspaces.projectId, executionWorkspace.projectId),
|
||||
)).orderBy(asc(projectWorkspaces.createdAt), asc(projectWorkspaces.id));
|
||||
const repositories = await prepareProjectRepositoryWorkspaces({
|
||||
cwd: executionWorkspace.cwd,
|
||||
anchorRepoUrl: executionWorkspace.repoUrl,
|
||||
workspaces: repositoryRows,
|
||||
resolveGitAuth: workspaceGitAuthProvider,
|
||||
});
|
||||
const paths = new Map(repositories.map((repo) => [repo.workspaceId, repo.cwd]));
|
||||
projectRepositoryPaths.push(...repositories.map((repo) => path.relative(executionWorkspace.cwd, repo.cwd)));
|
||||
if (resolvedWorkspace.workspaceId) paths.set(resolvedWorkspace.workspaceId, executionWorkspace.cwd);
|
||||
resolvedWorkspace.workspaceHints = resolvedWorkspace.workspaceHints.map((hint) => ({
|
||||
...hint, cwd: paths.get(hint.workspaceId) ?? hint.cwd,
|
||||
}));
|
||||
}
|
||||
if (persistedExecutionWorkspace) {
|
||||
context.executionWorkspaceId = persistedExecutionWorkspace.id;
|
||||
await db
|
||||
@@ -22913,13 +23021,17 @@ export function heartbeatService(
|
||||
companyId: agent.companyId,
|
||||
runId: run.id,
|
||||
issue: issueRef,
|
||||
taskPrompt:
|
||||
taskPrompt: [
|
||||
readNonEmptyString(
|
||||
selectPaperclipTaskMarkdown(context, {
|
||||
resumedSession,
|
||||
}),
|
||||
) ??
|
||||
`# ${issueRef.identifier ?? issueRef.id}: ${issueRef.title}`,
|
||||
projectRepositoryPaths.length > 0
|
||||
? `## Project repositories\nThe task workspace also contains these editable Git repositories:\n${projectRepositoryPaths.map((repo) => `- ${repo}`).join("\n")}`
|
||||
: null,
|
||||
].filter(Boolean).join("\n\n"),
|
||||
wakePayload: context.paperclipWake,
|
||||
resumedSession,
|
||||
conversationMode: context.conversationMode === true,
|
||||
|
||||
@@ -322,6 +322,7 @@ export function readNativeWorkspaceSyncReference(
|
||||
|
||||
function parseGitSnapshot(
|
||||
value: unknown,
|
||||
nested = false,
|
||||
): GitWorkspaceSnapshot | null | undefined {
|
||||
if (value === null) return null;
|
||||
const candidate = parseObject(value);
|
||||
@@ -348,12 +349,26 @@ function parseGitSnapshot(
|
||||
) {
|
||||
return undefined;
|
||||
}
|
||||
const repositories: NonNullable<GitWorkspaceSnapshot["repositories"]> = [];
|
||||
if (candidate.repositories !== undefined) {
|
||||
if (nested || !Array.isArray(candidate.repositories)) return undefined;
|
||||
const seen = new Set<string>();
|
||||
for (const raw of candidate.repositories) {
|
||||
const repo = parseObject(raw);
|
||||
if (typeof repo.path !== "string" || !/^\.paperclip-repositories\/[a-zA-Z0-9_-]+$/.test(repo.path) || seen.has(repo.path)) return undefined;
|
||||
const snapshot = parseGitSnapshot(repo.snapshot, true);
|
||||
if (!snapshot) return undefined;
|
||||
seen.add(repo.path);
|
||||
repositories.push({ path: repo.path, snapshot });
|
||||
}
|
||||
}
|
||||
return {
|
||||
headCommit: candidate.headCommit,
|
||||
branchName: candidate.branchName as string | null,
|
||||
overlayPaths: [...(candidate.overlayPaths as string[])],
|
||||
deletedPaths: [...(candidate.deletedPaths as string[])],
|
||||
ignoredPaths: [...(candidate.ignoredPaths as string[])],
|
||||
...(repositories.length ? { repositories } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user