mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-11 13:24:55 +01:00
fix: restore nested project repositories and settings after expiration (#16691)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agents use isolated Git workspaces with managed project repository copies. > - Finished workspaces can expire when Git can reproduce their contents. > - The saved recipe records the main commit but omits nested commits and local settings. > - Reopening can therefore use newer nested code and lose merged user settings. > - This pull request saves and restores those contents before workspace provisioning. > - It also permits verified FETCH_HEAD metadata so ordinary fetches do not block safe expiration. ## Linked Issues or Issue Description Refs #16457, #16586, #16621. Related performance work: #16579. **What happened?** After a finished workspace expires, its nested project checkout can reopen at the current upstream commit instead of its previous commit. Cleanup also treats local Claude settings as disposable, although that file can contain user settings. Ordinary linked-worktree FETCH_HEAD metadata prevents otherwise safe cleanup. **Expected behavior** Restore the main and nested repositories at their saved commits. Preserve local settings bytes and permissions. Skip files and Git state that cannot be reproduced. Permit standard fetched metadata only when its commits remain recoverable. **Steps to reproduce** 1. Prepare a managed nested project repository at commit A and add ignored local settings. 2. Advance upstream to B, finish the task tree, and expire its workspace and base. 3. Reopen the workspace through the normal service path. Before this fix, nested preparation selects B. ## What Changed - Extend the existing restoration recipe with exact nested commits, branches, source identity, and encrypted local settings. Keep the schema compatible with older recipes. - Restore nested repositories and settings before provisioning. Recheck company/project source access and preserve conflicting files on retry. - Recheck recorded contents under cleanup locks before deletion. Persist lock ownership before atomic publication so process exits recover safely and foreign locks stay intact. Preserve changed or unsupported contents, including tracked settings. - Keep saved project repository selections through normal preparation. Checkpoint restored contents before provisioning so failed setup can retry without overwriting later edits. - Accept bounded linked-worktree FETCH_HEAD records only after local ancestry or fresh upstream proof. Preserve local-only fetched commits. - Add real Git/Postgres lifecycle cases and focused content, access, publication-retry, and fetched-state controls. Document the behavior. ## Verification - `pnpm -r typecheck` passed. - `pnpm build` passed. - The original real Git/Postgres experiment failed with nested B instead of A. The same path passes with this change. - The integrated lifecycle suite passed 42 cases in 91.66s. Related preparation, credential, content, and Git metadata controls passed 46 cases in 24.41s. The final heartbeat checkpoint persistence case passed separately in 8.17s. - Coverage includes exact main/nested commits, encrypted settings and modes, upstream failure and retry, provisioning order, concurrent reopening, changed unique files, tracked settings, malformed metadata, and company boundaries. - Exact-head CI passed the implementation, typecheck, build, security, and test matrices. One browser shard passed on a single rerun after an unrelated fixture teardown raced an activity-log write; its feature assertions passed in the initial attempt. - The broad local test run was stopped after about 12 minutes of unrelated serial tests. Full parallel CI is required before merge; no local full-suite pass is claimed. - Single-workspace deployment, real-backlog reclamation, and ordinary product reopening remain required acceptance work after merge. ## Risks - Expired Git repositories still depend on upstream access. A failed restoration returns the existing retryable error and never substitutes the default branch. - Saved settings use the existing local encryption provider. Recovery requires that encryption key to remain available. - Unsupported files, tracked settings, unknown sources, and unverified Git state remain on disk. This can reduce reclaimed space. - No database migration, recovery-retention change, public API, or fleet-default change is included. ## Model Used OpenAI Codex, GPT-6 family, with tool use, code execution, and an independent agent review. The runtime does not expose a more specific model identifier. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
e6dcc6b095
commit
790b729086
14 files changed
+1258
-37
No files matched your search
+19
-5
@@ -2626,9 +2626,10 @@ recipe to workspace metadata. Current runs, previews, pending reopening,
|
||||
unresolved recovery, dirty/untracked files, unknown ignored files, unproven
|
||||
ownership, and unsupported or unique Git state prevent deletion. Ignored files
|
||||
can be discarded only when they are repository-ignored regenerable output,
|
||||
Paperclip's `.claude/settings.local.json` (rewritten on every Claude run), or a
|
||||
managed project checkout under `.paperclip-repositories/<repo>-<hash>` that the
|
||||
next run re-clones. A checkout qualifies only when its name maps back to one of
|
||||
ignored `.claude/settings.local.json` saved as an encrypted snapshot of at most
|
||||
64 KiB, or a managed project checkout under `.paperclip-repositories/<repo>-<hash>`
|
||||
whose exact commit, branch, and source identity are saved in the restoration recipe.
|
||||
Tracked settings, oversized settings, symbolic links, and unknown files remain protected. A checkout qualifies only when its name maps back to one of
|
||||
the project's repositories and it passes the same checks as the worktree
|
||||
(clean tree, no hidden index flags, allowed local config, no in-progress
|
||||
state), stock Git storage (no custom hooks), no linked worktrees, no
|
||||
@@ -2637,7 +2638,10 @@ reachable from a current upstream branch or tag, read fresh from the project
|
||||
repository's URL with the company's Git identity. Task repositories
|
||||
(`task-repo-*`) always prevent deletion. Inspection runs Git with
|
||||
`core.fsmonitor` disabled, so a repository's local config cannot make it run a
|
||||
command.
|
||||
command. Standard linked-worktree `FETCH_HEAD` records are retained unless every
|
||||
referenced commit is an ancestor of the saved HEAD or passes a fresh upstream fetch.
|
||||
Malformed records, symbolic links, and more than 32 distinct fetched commits prevent
|
||||
expiration.
|
||||
Deletion first renames the worktree to recorded trash; restart and reopening
|
||||
finish that cleanup under the same source/lifecycle locks.
|
||||
|
||||
@@ -2645,7 +2649,17 @@ finish that cleanup under the same source/lifecycle locks.
|
||||
owned isolated remote-source base after all consumers expire and all remaining
|
||||
refs and files are verified reproducible. Shared project clones remain intact.
|
||||
Normal authorized reopening recreates a missing source and checks out the saved
|
||||
commit, even after upstream advances. Inaccessible upstream produces an explicit
|
||||
commit, including nested project repositories, even after upstream advances.
|
||||
Settings bytes and permissions restore before provisioning; their encrypted payloads
|
||||
are bound to the company, workspace, and relative path and require the instance
|
||||
secrets master key. The worker rechecks the complete saved contents before deletion.
|
||||
A changed workspace is retained and made usable instead of deleting newer work.
|
||||
Interrupted settings publication and partial nested restoration are retryable.
|
||||
Normal preparation retains the saved nested repository selection when project
|
||||
ref or path settings change. A durable contents checkpoint lets provisioning
|
||||
retry without overwriting files it changed. Expiration records ownership before
|
||||
publishing native Git locks; restart recovery removes only those owned locks.
|
||||
Inaccessible upstream produces an explicit
|
||||
retryable restoration failure. It never substitutes the default branch.
|
||||
Reopening failures reject the consuming request with HTTP 409 before saving a
|
||||
comment or changing its task. The response preserves the restoration message, so
|
||||
|
||||
@@ -33,7 +33,89 @@ Each slice gets focused tests, required repository checks, review, single-worksp
|
||||
deployment at an exact SHA, dry-run inspection, measured reclamation, and ordinary
|
||||
product acceptance. Rollback disables deletion and retains restoration support.
|
||||
|
||||
## Current state (2026-10-15, 19:27 UTC)
|
||||
## Current delivery state (2026-10-17)
|
||||
|
||||
The earlier fixture acceptance proved recovery, but did not reclaim the real backlog.
|
||||
The Oct 16 recurrence measured 129 GiB used and 8.24 GiB free. Nested project copies
|
||||
accounted for 21.31 GiB; 36 fully finished task trees held 9.23 GiB of candidate
|
||||
nested copies. Those are candidate bytes, not reclaimed bytes. Runner backups and
|
||||
legacy skill-run pinning remain separate follow-ups.
|
||||
|
||||
Upstream PRs #16586, #16614, and #16621 now recognize managed nested checkouts,
|
||||
short-circuit shared paths, and verify public Git commits without a run identity.
|
||||
The remaining implementation gap is exact nested commit and local settings recovery.
|
||||
Current work builds on master `1f27427ffdefaf5fb2760d1d53c0d78cb9683d4c`, branch
|
||||
`codex/worktree-nested-restoration`. Serving SHA observed before work:
|
||||
`27ab902d3db2770ada79c9f42a9c06e1d346f34b`. No deployment or real reclamation yet.
|
||||
Review: [PR #16691](https://github.com/paperclipai/paperclip/pull/16691).
|
||||
Its first head `4696ba2fd171ec3970b7e116bcb5f77fe9331785` passed CI. Greptile
|
||||
found historical project-selection and failed-provision retry gaps; both now have
|
||||
fixes and real lifecycle controls. The updated head still needs review and CI.
|
||||
|
||||
| Required outcome | Current evidence | Remaining gap | Next action / owner |
|
||||
| --- | --- | --- | --- |
|
||||
| Restore every nested repository at its recorded commit | Real Git/Postgres regression now restores A after upstream advances to B, before provisioning | Review and deployed proof | CI and one-target rollout / this task |
|
||||
| Preserve local settings and unique work | Encrypted bytes/modes, mutation fencing, interrupted publication and retry controls pass | Review and deployed proof | CI and protected-data comparison / this task |
|
||||
| Reclaim real finished backlog on staging | Oct 16 disk survey and skip reasons retained | Reviewed merge, one-target rollout, actual measured deletion | Existing deployment workflow / this task |
|
||||
| Ordinary reopening and subsequent work | Prior main-repository fixture proof only | Real expired nested workspace journey plus independent inspection | Product acceptance / this task |
|
||||
|
||||
Retained experiment evidence: `/tmp/paperclip-disk-20261016/nested-baseline-built.log`.
|
||||
After 17s dependency installation and one SDK build, the useful baseline took 12.23s
|
||||
(6.91s test, 3.07s transform), using bare upstream repositories and normal preparation,
|
||||
expiration, and reopening. It failed because nested HEAD became B instead of A.
|
||||
Treatment passed the same recovery case in 14.44s. The affected suites passed
|
||||
37 cases in 142.37s; six nested mutation/retry cases passed in 56.72s. Independent
|
||||
review found interrupted hard-link publication and tracked-settings mode risks;
|
||||
three added controls passed in 19.07s after fixes. The final affected run passed
|
||||
55 of 56 cases in 98.38s; one fault-injection fixture used a noncanonical macOS
|
||||
path and missed its intended failure. Correcting that path made the case pass
|
||||
in 16.94s. All five content controls passed again after the bounded-read fix.
|
||||
Full workspace typecheck and build passed. The broad local test run was stopped
|
||||
after approximately 12 minutes of unrelated serial server tests; mandatory
|
||||
parallel CI remains required before merge. No full-suite local pass is claimed.
|
||||
|
||||
Review fixes preserve saved project ref/cwd selection through normal preparation
|
||||
and subsequent expiration. A separate durable contents checkpoint survives failed
|
||||
provisioning and transaction rollback, and heartbeat persistence retains it. Native
|
||||
cleanup locks now have persisted random ownership and atomic publication; actual
|
||||
child-process exits prove worker and reopening recovery without removing foreign
|
||||
locks. The integrated lifecycle file passed 42 cases in 91.66s; 46 preparation,
|
||||
credential, contents, and Git controls passed in 24.41s. The final heartbeat
|
||||
checkpoint persistence case passed separately in 8.17s. Server typecheck passed.
|
||||
The representative crash experiment took 20.84s, mostly fixture Git/process work
|
||||
(72% tests, 21% transforms); it justified the integrated regression run.
|
||||
|
||||
The live pre-cleanup baseline records 50 old terminal workspaces in 74.543s,
|
||||
including 19 with nested copies and 21 dirty controls. Evidence is
|
||||
`/tmp/paperclip-cleanup-20261017/eligible-baseline.jsonl`; no file contents or
|
||||
credentials are printed. The ordinary staging UI is authenticated and reachable.
|
||||
Current disk observation: 129,519,296,512 bytes used and 17,779,265,536 free (88%).
|
||||
|
||||
Live skip inspection found ordinary linked-worktree FETCH_HEAD files blocking
|
||||
45 of 200 sampled worker events. The local baseline reproduced rejection in
|
||||
8.31s; the verified-fetch treatment passed five cases in 15.23s. Unknown,
|
||||
malformed, and unverified local-only fetched state stays protected.
|
||||
|
||||
The existing fleet campaign completed for this target during diagnosis. Current
|
||||
serving SHA is `d30648dee89720358271c276b9fc28cf83f18be7`, artifact
|
||||
`cloud-39088110403-1`. The target is temporarily user-managed, with no operation
|
||||
lock; Git dry-run was staged for the next deployment. Fleet defaults are unchanged.
|
||||
Reconcile the pin after acceptance without rolling back restoration support.
|
||||
Staging read-only baseline at 2026-10-18 03:05 UTC: 127,859,752,960 bytes used,
|
||||
19,438,809,088 free (87%). Only one real-company workspace had a completed
|
||||
expiration record. These observations do not establish reclamation by this change.
|
||||
|
||||
First attempts exposed loopback sandbox denial and a missing SDK build; both setup
|
||||
issues were corrected before judging behavior. Reuse this fixture for treatment.
|
||||
|
||||
All prior temporary SSH keys have been revoked and removed. This iteration registered
|
||||
`paperclip-cleanup-20261017`, fingerprint `SHA256:h6kA+d64fu/OoA4bLNfjW0J3V5YsapuSWVbZpR+VcdA`.
|
||||
Revoke it and remove `/tmp/paperclip-cleanup-20261017/railway-key{,.pub}` and
|
||||
`~/.ssh/paperclip_cleanup_20261017{,.pub}` at completion. Historical details below describe prior
|
||||
revisions and must not be treated as current deployment or access state.
|
||||
|
||||
## Historical state (2026-10-15, 19:27 UTC)
|
||||
|
||||
|
||||
- Skill [PR #16433](https://github.com/paperclipai/paperclip/pull/16433) merged as
|
||||
`86c965ce7ab1499864f74b8dd6ed0d2d8db15197`. Git
|
||||
|
||||
@@ -81,9 +81,9 @@ describe("assertReproducibleGitFiles with Paperclip-managed files", () => {
|
||||
const expectAccepted = () => expect(check()).resolves.toBeDefined();
|
||||
const expectRejected = (reason: string | RegExp) => expect(check()).rejects.toThrow(reason);
|
||||
|
||||
it("accepts Paperclip's Claude settings file", async () => {
|
||||
it("preserves Claude settings without a restoration capture", async () => {
|
||||
await write(path.join(work, ".claude", "settings.local.json"), "{}\n");
|
||||
await expectAccepted();
|
||||
await expectRejected("unique_ignored_files");
|
||||
});
|
||||
|
||||
it("accepts a clean managed checkout, including its regenerable directories", async () => {
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
import { execFile as execFileCallback } from "node:child_process";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
captureGitWorkspaceContents,
|
||||
CLAUDE_SETTINGS_PATH,
|
||||
readManagedCheckoutContents,
|
||||
restoreGitWorkspaceContents,
|
||||
type GitWorkspaceContents,
|
||||
} from "../services/git-workspace-contents.js";
|
||||
import { managedProjectCheckoutName } from "../services/managed-project-checkout-name.js";
|
||||
|
||||
const execFile = promisify(execFileCallback);
|
||||
|
||||
describe("Git workspace settings preservation", () => {
|
||||
let root: string;
|
||||
let cwd: string;
|
||||
let identity: { companyId: string; workspaceId: string };
|
||||
const original = Buffer.from('{"env":{"PRIVATE_VALUE":"saved-workspace-setting"}}\n');
|
||||
|
||||
async function repository(directory: string) {
|
||||
await fs.mkdir(directory, { recursive: true });
|
||||
const git = (...args: string[]) => execFile("git", ["-C", directory,
|
||||
"-c", "user.name=Fixture", "-c", "user.email=fixture@example.test", "-c", "commit.gpgsign=false", ...args],
|
||||
{ env: { ...process.env, GIT_CONFIG_NOSYSTEM: "1", GIT_CONFIG_GLOBAL: os.devNull } });
|
||||
await git("init", "--initial-branch=main");
|
||||
await fs.writeFile(path.join(directory, ".gitignore"), ".claude/settings.local.json\n.paperclip-repositories/\n");
|
||||
await git("add", ".gitignore");
|
||||
await git("commit", "-m", "Fixture");
|
||||
return directory;
|
||||
}
|
||||
|
||||
async function capture() {
|
||||
const target = path.join(cwd, CLAUDE_SETTINGS_PATH);
|
||||
await fs.mkdir(path.dirname(target), { recursive: true });
|
||||
await fs.writeFile(target, original, { mode: 0o600 });
|
||||
return captureGitWorkspaceContents(cwd, identity, []);
|
||||
}
|
||||
|
||||
function restore(contents: GitWorkspaceContents, selectedIdentity = identity) {
|
||||
return restoreGitWorkspaceContents({ cwd, ...selectedIdentity, contents,
|
||||
resolveGitAuth: async () => null, authorizeSource: async () => {} });
|
||||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", "31".repeat(32));
|
||||
root = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-workspace-contents-")));
|
||||
cwd = await repository(path.join(root, "work"));
|
||||
identity = { companyId: randomUUID(), workspaceId: randomUUID() };
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
vi.unstubAllEnvs();
|
||||
await fs.rm(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("binds encrypted settings to their company and workspace while restoring the original bytes and mode", async () => {
|
||||
const contents = await capture();
|
||||
const target = path.join(cwd, CLAUDE_SETTINGS_PATH);
|
||||
expect(JSON.stringify(contents)).not.toContain("saved-workspace-setting");
|
||||
expect(JSON.stringify(contents)).not.toContain(original.toString("base64"));
|
||||
await fs.unlink(target);
|
||||
for (const differentIdentity of [
|
||||
{ ...identity, companyId: randomUUID() },
|
||||
{ ...identity, workspaceId: randomUUID() },
|
||||
]) {
|
||||
await expect(restore(contents, differentIdentity)).rejects.toThrow("could not be restored");
|
||||
await expect(fs.lstat(target)).rejects.toMatchObject({ code: "ENOENT" });
|
||||
}
|
||||
await restore(contents);
|
||||
expect(await fs.readFile(target)).toEqual(original);
|
||||
expect((await fs.stat(target)).mode & 0o777).toBe(0o600);
|
||||
});
|
||||
|
||||
it("rejects moving the saved settings ciphertext into a nested checkout", async () => {
|
||||
const contents = await capture();
|
||||
const source = { id: randomUUID(), repoUrl: "https://example.test/project.git", repoRef: "main", cwd: null };
|
||||
const name = managedProjectCheckoutName(source);
|
||||
const nested = await repository(path.join(cwd, ".paperclip-repositories", name));
|
||||
await execFile("git", ["-C", nested, "remote", "add", "origin", source.repoUrl]);
|
||||
const checkout = await readManagedCheckoutContents(nested, name, source);
|
||||
const moved = { ...contents, settings: null, checkouts: [{ ...checkout, settings: contents.settings }] };
|
||||
await expect(restore(moved)).rejects.toThrow("could not be restored");
|
||||
await expect(fs.lstat(path.join(nested, CLAUDE_SETTINGS_PATH))).rejects.toMatchObject({ code: "ENOENT" });
|
||||
expect(await fs.readFile(path.join(cwd, CLAUDE_SETTINGS_PATH))).toEqual(original);
|
||||
});
|
||||
|
||||
it("rejects symlinked settings files and parents without changing the links or their targets", async () => {
|
||||
const contents = await capture();
|
||||
const outside = path.join(root, "outside");
|
||||
const outsideSettings = path.join(outside, "settings.local.json");
|
||||
const outsideBytes = Buffer.from("outside workspace contents\n");
|
||||
await fs.mkdir(outside);
|
||||
await fs.writeFile(outsideSettings, outsideBytes);
|
||||
for (const linkKind of ["parent", "file"]) {
|
||||
await fs.rm(path.join(cwd, ".claude"), { recursive: true, force: true });
|
||||
const link = path.join(cwd, linkKind === "parent" ? ".claude" : CLAUDE_SETTINGS_PATH);
|
||||
if (linkKind === "file") await fs.mkdir(path.dirname(link));
|
||||
await fs.symlink(linkKind === "parent" ? outside : outsideSettings, link);
|
||||
await expect(captureGitWorkspaceContents(cwd, identity, [])).rejects.toThrow();
|
||||
await expect(restore(contents)).rejects.toThrow("could not be restored");
|
||||
expect((await fs.lstat(link)).isSymbolicLink()).toBe(true);
|
||||
expect(await fs.readFile(outsideSettings)).toEqual(outsideBytes);
|
||||
}
|
||||
});
|
||||
|
||||
it("preserves settings exceeding the recovery size limit", async () => {
|
||||
const contents = await capture();
|
||||
const target = path.join(cwd, CLAUDE_SETTINGS_PATH);
|
||||
const oversized = Buffer.alloc(64 * 1024 + 1, "x");
|
||||
await fs.writeFile(target, oversized);
|
||||
await expect(captureGitWorkspaceContents(cwd, identity, [])).rejects.toThrow("unsupported_local_settings");
|
||||
await expect(restore(contents)).rejects.toThrow("could not be restored");
|
||||
expect(await fs.readFile(target)).toEqual(oversized);
|
||||
expect((await fs.stat(target)).mode & 0o777).toBe(0o600);
|
||||
});
|
||||
|
||||
it("leaves conflicting settings and their permissions intact", async () => {
|
||||
const contents = await capture();
|
||||
const target = path.join(cwd, CLAUDE_SETTINGS_PATH);
|
||||
const edited = Buffer.from('{"env":{"PRIVATE_VALUE":"newer-user-setting"}}\n');
|
||||
await fs.writeFile(target, edited);
|
||||
await fs.chmod(target, 0o640);
|
||||
await expect(restore(contents)).rejects.toThrow("could not be restored");
|
||||
expect(await fs.readFile(target)).toEqual(edited);
|
||||
expect((await fs.stat(target)).mode & 0o777).toBe(0o640);
|
||||
expect(await fs.readdir(path.dirname(target))).toEqual(["settings.local.json"]);
|
||||
});
|
||||
});
|
||||
@@ -1,19 +1,24 @@
|
||||
import { promises as fs } from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { execFileSync, spawnSync } from "node:child_process";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { createRequire } from "node:module";
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { companies, projects, issues, executionWorkspaces, agents, heartbeatRuns, nativeRunFinalizations, workspaceRuntimeServices, createDb } from "@paperclipai/db";
|
||||
import { companies, projects, projectWorkspaces, issues, executionWorkspaces, agents, heartbeatRuns, nativeRunFinalizations, workspaceRuntimeServices, createDb } from "@paperclipai/db";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "./helpers/embedded-postgres.js";
|
||||
import { resolveManagedProjectWorkspaceDir } from "../home-paths.js";
|
||||
import { materializeManagedProjectWorkspace } from "../services/managed-repository-checkout.js";
|
||||
import { materializeIsolatedRemoteWorkspaceSource } from "../services/isolated-remote-workspace-source.js";
|
||||
import { realizeExecutionWorkspace, ensurePersistedExecutionWorkspaceAvailable } from "../services/workspace-runtime.js";
|
||||
import { executionWorkspaceService } from "../services/execution-workspaces.js";
|
||||
import { prepareProjectRepositoryWorkspaces, mergeExecutionWorkspaceMetadataForPersistence } from "../services/heartbeat/workspaces.js";
|
||||
import { expireReconstructableGitWorkspace } from "../services/git-workspace-expiration.js";
|
||||
import * as activityLog from "../services/activity-log.js";
|
||||
import { readGitCacheRestoration } from "../services/git-workspace-restoration.js";
|
||||
import { gitCacheExpirationLockLocations, readGitCacheExpiration } from "../services/git-workspace-restoration.js";
|
||||
|
||||
const gitEnv = { ...process.env, GIT_AUTHOR_NAME: "Fixture", GIT_AUTHOR_EMAIL: "fixture@example.test",
|
||||
GIT_COMMITTER_NAME: "Fixture", GIT_COMMITTER_EMAIL: "fixture@example.test", GIT_TERMINAL_PROMPT: "0" };
|
||||
@@ -48,7 +53,7 @@ describeEmbeddedPostgres("reconstructable Git expiration", () => {
|
||||
await fs.mkdir(source, { recursive: true });
|
||||
git(source, "init", "--initial-branch=main");
|
||||
await fs.writeFile(path.join(source, "marker.txt"), "commit A\n");
|
||||
await fs.writeFile(path.join(source, ".gitignore"), "node_modules/\nprivate-notes/\n");
|
||||
await fs.writeFile(path.join(source, ".gitignore"), "node_modules/\nprivate-notes/\n.claude/settings.local.json\n");
|
||||
git(source, "add", "."); git(source, "commit", "-m", "A");
|
||||
const commitA = git(source, "rev-parse", "HEAD");
|
||||
git(location, "clone", "--bare", source, remote); git(source, "remote", "add", "origin", remote);
|
||||
@@ -78,7 +83,7 @@ describeEmbeddedPostgres("reconstructable Git expiration", () => {
|
||||
await fs.writeFile(path.join(source, "marker.txt"), "commit B\n"); git(source, "add", "."); git(source, "commit", "-m", "B"); git(source, "push", "origin", "main");
|
||||
return git(source, "rev-parse", "HEAD");
|
||||
};
|
||||
return { companyId, issueId, workspaceId, cwd: workspace.cwd, baseCwd, base, remote, source, commitA, row, expire, reopen, advance };
|
||||
return { companyId, projectId, issueId, workspaceId, cwd: workspace.cwd, baseCwd, base, remote, source, commitA, row, expire, reopen, advance };
|
||||
}
|
||||
|
||||
it("expires through the lifecycle worker and restores the recorded commit after upstream advances", async () => {
|
||||
@@ -112,6 +117,356 @@ describeEmbeddedPostgres("reconstructable Git expiration", () => {
|
||||
expect(git(f.cwd, "rev-parse", "HEAD")).toBe(continued);
|
||||
});
|
||||
|
||||
it("expires a worktree after an ordinary fetch and restores its older recorded commit", async () => {
|
||||
const f = await fixture();
|
||||
await f.advance();
|
||||
git(f.cwd, "fetch", "origin", "main");
|
||||
expect(await f.expire()).toMatchObject({ expired: true, baseExpired: true });
|
||||
expect(await f.reopen()).toMatchObject({ ok: true });
|
||||
expect(git(f.cwd, "rev-parse", "HEAD")).toBe(f.commitA);
|
||||
});
|
||||
|
||||
it("preserves the contents checkpoint through normal runtime metadata persistence", async () => {
|
||||
const f = await fixture();
|
||||
const settingsPath = path.join(f.cwd, ".claude/settings.local.json");
|
||||
await fs.mkdir(path.dirname(settingsPath));
|
||||
await fs.writeFile(settingsPath, "original settings\n");
|
||||
expect((await f.expire({ expireBase: false })).expired).toBe(true);
|
||||
const service = executionWorkspaceService(db);
|
||||
const stored = (await service.getStoredById(f.companyId, f.workspaceId))!;
|
||||
const input = { db, base: f.base, workspace: stored,
|
||||
issue: { id: f.issueId, identifier: "GC-1", title: "Resumed task" },
|
||||
agent: { id: null, name: "Fixture", companyId: f.companyId }, skipHostProvision: true };
|
||||
const restored = (await ensurePersistedExecutionWorkspaceAvailable(input))!;
|
||||
await service.update(f.workspaceId, { metadata: mergeExecutionWorkspaceMetadataForPersistence({
|
||||
existingMetadata: stored.metadata, source: restored.source, createdByRuntime: restored.branchCreatedByRuntime,
|
||||
strategyType: restored.strategy, configSnapshot: null, shouldReuseExisting: true,
|
||||
baseRef: restored.repoRef, baseRefSha: restored.baseRefSha,
|
||||
}) });
|
||||
expect((await f.row()).metadata?.gitCacheRestoration).toMatchObject({ contentsRestoredAt: expect.any(String) });
|
||||
await fs.writeFile(settingsPath, "later task settings\n");
|
||||
await ensurePersistedExecutionWorkspaceAvailable({ ...input, workspace: (await service.getStoredById(f.companyId, f.workspaceId))! });
|
||||
expect(await fs.readFile(settingsPath, "utf8")).toBe("later task settings\n");
|
||||
});
|
||||
|
||||
async function nestedFixture() {
|
||||
const f = await fixture(true);
|
||||
const nestedSource = path.join(path.dirname(f.source), "nested-author");
|
||||
const nestedRemote = path.join(path.dirname(f.source), "nested.git");
|
||||
await fs.mkdir(nestedSource);
|
||||
git(nestedSource, "init", "--initial-branch=main");
|
||||
await fs.writeFile(path.join(nestedSource, "nested.txt"), "nested A\n");
|
||||
await fs.writeFile(path.join(nestedSource, ".gitignore"), ".claude/settings.local.json\n");
|
||||
git(nestedSource, "add", "."); git(nestedSource, "commit", "-m", "Nested A");
|
||||
const nestedA = git(nestedSource, "rev-parse", "HEAD");
|
||||
git(path.dirname(nestedSource), "clone", "--bare", nestedSource, nestedRemote);
|
||||
const [source] = await db.insert(projectWorkspaces).values({ companyId: f.companyId, projectId: f.projectId!,
|
||||
name: "Nested repository", repoUrl: pathToFileURL(nestedRemote).href, repoRef: "main" }).returning();
|
||||
const preparation = { cwd: f.cwd, anchorRepoUrl: f.base.repoUrl, workspaces: [source!], resolveGitAuth: async () => null };
|
||||
const [nested] = await prepareProjectRepositoryWorkspaces(preparation);
|
||||
const settings = Buffer.from('{"permissions":{"allow":["Read(/user/path)"]},"env":{"LOCAL_SETTING":"preserve-me"}}\n');
|
||||
const settingsPath = path.join(f.cwd, ".claude/settings.local.json");
|
||||
await fs.mkdir(path.dirname(settingsPath));
|
||||
await fs.writeFile(settingsPath, settings, { mode: 0o600 });
|
||||
const nestedSettingsPath = path.join(nested!.cwd, ".claude/settings.local.json");
|
||||
await fs.mkdir(path.dirname(nestedSettingsPath));
|
||||
await fs.writeFile(nestedSettingsPath, settings, { mode: 0o640 });
|
||||
return { ...f, nestedSource, nestedRemote, nestedA, preparation, nested: nested!, settings, settingsPath, nestedSettingsPath };
|
||||
}
|
||||
|
||||
async function crashWithExpirationLocks(f: Awaited<ReturnType<typeof nestedFixture>>, beforePublication = false) {
|
||||
expect(await f.expire({ beforeDelete: async () => { throw new Error("stop_before_locks"); } }))
|
||||
.toMatchObject({ expired: false, skipReason: "stop_before_locks" });
|
||||
const row = await f.row();
|
||||
const locations = gitCacheExpirationLockLocations(row);
|
||||
const token = readGitCacheExpiration(row)!.lockToken!;
|
||||
const module = new URL("../services/git-workspace-expiration-locks.ts", import.meta.url).href;
|
||||
const child = spawnSync(process.execPath, ["--import", createRequire(import.meta.url).resolve("tsx"), "--input-type=module", "-e", `
|
||||
import fs from 'node:fs/promises';
|
||||
import { acquireGitExpirationLocks } from ${JSON.stringify(module)};
|
||||
${beforePublication ? "fs.link = async () => process.exit(71);" : ""}
|
||||
await acquireGitExpirationLocks(${JSON.stringify(locations)}, ${JSON.stringify(token)});
|
||||
process.exit(71);
|
||||
`], { encoding: "utf8" });
|
||||
expect(child.stderr).toBe("");
|
||||
expect(child.status).toBe(71);
|
||||
return locations;
|
||||
}
|
||||
|
||||
it("recovers expiration-owned main and nested locks after process exit before retrying cleanup", async () => {
|
||||
const f = await nestedFixture();
|
||||
const locations = await crashWithExpirationLocks(f);
|
||||
for (const location of locations) expect(await exists(path.join(location.root, location.relativePath))).toBe(true);
|
||||
expect(await f.expire()).toMatchObject({ expired: true });
|
||||
expect(await f.reopen()).toMatchObject({ ok: true, reopened: true });
|
||||
expect(git(f.cwd, "rev-parse", "HEAD")).toBe(f.commitA);
|
||||
expect(git(f.nested.cwd, "rev-parse", "HEAD")).toBe(f.nestedA);
|
||||
expect(await fs.readFile(f.nestedSettingsPath)).toEqual(f.settings);
|
||||
});
|
||||
|
||||
it("ordinary reopening recovers process-abandoned main and nested locks", async () => {
|
||||
const f = await nestedFixture();
|
||||
const locations = await crashWithExpirationLocks(f);
|
||||
expect(await f.reopen()).toMatchObject({ ok: true, reopened: true });
|
||||
for (const location of locations) expect(await exists(path.join(location.root, location.relativePath))).toBe(false);
|
||||
await fs.writeFile(path.join(f.nested.cwd, "continued.txt"), "continued after crash\n");
|
||||
git(f.nested.cwd, "add", "."); git(f.nested.cwd, "commit", "-m", "Continued after crash");
|
||||
expect(git(f.nested.cwd, "rev-parse", "HEAD")).not.toBe(f.nestedA);
|
||||
expect(await fs.readFile(f.settingsPath)).toEqual(f.settings);
|
||||
});
|
||||
|
||||
it("recovers an unpublished lock temporary file and a surviving branch lock after rename", async () => {
|
||||
const f = await nestedFixture();
|
||||
const locations = await crashWithExpirationLocks(f, true);
|
||||
for (const location of locations) expect(await exists(path.join(location.root, location.relativePath))).toBe(false);
|
||||
expect(await f.reopen()).toMatchObject({ ok: true });
|
||||
const firstAdmin = path.dirname(path.join(locations[0]!.root, locations[0]!.relativePath));
|
||||
expect((await fs.readdir(firstAdmin)).filter(name => name.startsWith(".paperclip-expiration-"))).toEqual([]);
|
||||
|
||||
const g = await nestedFixture();
|
||||
const remaining = await crashWithExpirationLocks(g);
|
||||
const state = readGitCacheExpiration(await g.row())!;
|
||||
await fs.rename(g.cwd, state.worktreeTrash);
|
||||
expect(await g.reopen()).toMatchObject({ ok: true });
|
||||
for (const location of remaining) expect(await exists(path.join(location.root, location.relativePath))).toBe(false);
|
||||
expect(git(g.cwd, "rev-parse", "HEAD")).toBe(g.commitA);
|
||||
expect(git(g.nested.cwd, "rev-parse", "HEAD")).toBe(g.nestedA);
|
||||
});
|
||||
|
||||
it("retains lock ownership when rollback is interrupted before releasing native locks", async () => {
|
||||
const f = await nestedFixture();
|
||||
const unlink = fs.unlink.bind(fs);
|
||||
const failure = vi.spyOn(fs, "unlink").mockImplementation(async target => {
|
||||
if (path.basename(String(target)) === "HEAD.lock") throw new Error("lock_release_failed");
|
||||
return unlink(target);
|
||||
});
|
||||
expect(await f.expire({ beforeDelete: async () => { await fs.writeFile(f.settingsPath, "changed during expiration\n"); } }))
|
||||
.toMatchObject({ expired: false, skipReason: "lock_release_failed" });
|
||||
expect((await f.row()).status).toBe("archived");
|
||||
expect(readGitCacheExpiration(await f.row())?.lockToken).toMatch(/^[a-f0-9]{64}$/);
|
||||
failure.mockRestore();
|
||||
expect(await f.expire()).toMatchObject({ expired: false, skipReason: "local_settings_changed" });
|
||||
expect((await f.row()).status).toBe("active");
|
||||
expect((await f.row()).metadata?.gitCacheExpiration).toBeUndefined();
|
||||
expect(await fs.readFile(f.settingsPath, "utf8")).toBe("changed during expiration\n");
|
||||
await fs.writeFile(path.join(f.nested.cwd, "continued.txt"), "continued after rollback\n");
|
||||
git(f.nested.cwd, "add", "."); git(f.nested.cwd, "commit", "-m", "Continued after rollback");
|
||||
});
|
||||
|
||||
it("preserves a foreign Git lock when recovering an interrupted expiration", async () => {
|
||||
const f = await nestedFixture();
|
||||
const locations = await crashWithExpirationLocks(f);
|
||||
const target = path.join(locations[0]!.root, locations[0]!.relativePath);
|
||||
await fs.writeFile(target, "foreign Git lock\n");
|
||||
expect(await f.expire()).toMatchObject({ expired: false, skipReason: "git_expiration_lock_conflict" });
|
||||
expect(await f.reopen()).toMatchObject({ ok: false, code: "rebuild_failed" });
|
||||
expect(await fs.readFile(target, "utf8")).toBe("foreign Git lock\n");
|
||||
expect(await fs.readFile(f.settingsPath)).toEqual(f.settings);
|
||||
expect(git(f.nested.cwd, "rev-parse", "HEAD")).toBe(f.nestedA);
|
||||
});
|
||||
|
||||
it("restores nested checkout commits and local settings after upstream advances", async () => {
|
||||
const f = await nestedFixture();
|
||||
const { nestedSource, nestedRemote, nestedA, preparation, nested, settings, settingsPath } = f;
|
||||
await fs.writeFile(path.join(nestedSource, "nested.txt"), "nested B\n");
|
||||
git(nestedSource, "add", "."); git(nestedSource, "commit", "-m", "Nested B");
|
||||
git(nestedSource, "push", nestedRemote, "main");
|
||||
await f.advance();
|
||||
expect(await f.expire()).toMatchObject({ expired: true });
|
||||
expect(await exists(f.cwd)).toBe(false);
|
||||
expect(await f.reopen()).toMatchObject({ ok: true, reopened: true });
|
||||
await prepareProjectRepositoryWorkspaces(preparation);
|
||||
expect(git(f.cwd, "rev-parse", "HEAD")).toBe(f.commitA);
|
||||
expect(git(nested!.cwd, "rev-parse", "HEAD")).toBe(nestedA);
|
||||
expect(await fs.readFile(settingsPath)).toEqual(settings);
|
||||
expect((await fs.stat(settingsPath)).mode & 0o777).toBe(0o600);
|
||||
expect(await fs.readFile(f.nestedSettingsPath)).toEqual(settings);
|
||||
expect((await fs.stat(f.nestedSettingsPath)).mode & 0o777).toBe(0o640);
|
||||
expect(JSON.stringify((await f.row()).metadata)).not.toContain("preserve-me");
|
||||
await fs.writeFile(path.join(nested.cwd, "continued.txt"), "nested C\n");
|
||||
git(nested.cwd, "add", "."); git(nested.cwd, "commit", "-m", "Nested continued");
|
||||
const continued = git(nested.cwd, "rev-parse", "HEAD");
|
||||
await fs.writeFile(settingsPath, "user changes after reopening\n");
|
||||
await ensurePersistedExecutionWorkspaceAvailable({ db, base: f.base,
|
||||
workspace: (await executionWorkspaceService(db).getStoredById(f.companyId, f.workspaceId))!,
|
||||
issue: { id: f.issueId, identifier: "GC-1", title: "Resumed task" },
|
||||
agent: { id: null, name: "Fixture", companyId: f.companyId }, skipHostProvision: true });
|
||||
await prepareProjectRepositoryWorkspaces(preparation);
|
||||
expect(git(nested.cwd, "rev-parse", "HEAD")).toBe(continued);
|
||||
expect(await fs.readFile(settingsPath, "utf8")).toBe("user changes after reopening\n");
|
||||
});
|
||||
|
||||
it("restores saved contents before the provision command observes the workspace", async () => {
|
||||
const f = await nestedFixture();
|
||||
const probe = path.join(path.dirname(f.source), "provision-probe.cjs");
|
||||
const observed = path.join(path.dirname(f.source), "provision-observed.txt");
|
||||
await fs.writeFile(probe, `const fs = require('node:fs'); const cp = require('node:child_process');
|
||||
if (cp.execFileSync('git', ['-C', ${JSON.stringify(f.nested.cwd)}, 'rev-parse', 'HEAD'], {encoding:'utf8'}).trim() !== ${JSON.stringify(f.nestedA)}) throw Error('wrong nested commit');
|
||||
if (fs.readFileSync(${JSON.stringify(f.settingsPath)}).toString('base64') !== ${JSON.stringify(f.settings.toString('base64'))}) throw Error('wrong settings');
|
||||
fs.writeFileSync(${JSON.stringify(observed)}, 'restored before provision');`);
|
||||
const [primary] = await db.insert(projectWorkspaces).values({ companyId: f.companyId, projectId: f.projectId!,
|
||||
name: "Primary repository", cwd: f.baseCwd, repoUrl: f.base.repoUrl, isPrimary: true }).returning();
|
||||
await db.update(issues).set({ projectId: f.projectId }).where(eq(issues.id, f.issueId));
|
||||
const row = await f.row();
|
||||
await db.update(executionWorkspaces).set({ projectWorkspaceId: primary!.id, metadata: { ...row.metadata,
|
||||
config: { provisionCommand: `node ${JSON.stringify(probe)}` } } }).where(eq(executionWorkspaces.id, f.workspaceId));
|
||||
expect((await f.expire()).expired).toBe(true);
|
||||
expect(await f.reopen()).toMatchObject({ ok: true });
|
||||
expect(await fs.readFile(observed, "utf8")).toBe("restored before provision");
|
||||
});
|
||||
|
||||
it.each(["failed_provision", "transaction_rollback"])("retries %s without discarding its restored workspace edits", async failure => {
|
||||
const f = await nestedFixture();
|
||||
const probe = path.join(path.dirname(f.source), "provision-retry.cjs");
|
||||
const observed = path.join(path.dirname(f.source), "provision-retry.txt");
|
||||
const changedSettings = "settings updated by provisioning\n";
|
||||
await fs.writeFile(probe, `const fs = require('node:fs');
|
||||
if (!fs.existsSync(${JSON.stringify(observed)})) {
|
||||
fs.writeFileSync(${JSON.stringify(f.settingsPath)}, ${JSON.stringify(changedSettings)});
|
||||
fs.writeFileSync(${JSON.stringify(observed)}, 'first attempt');
|
||||
process.exit(1);
|
||||
}
|
||||
if (fs.readFileSync(${JSON.stringify(f.settingsPath)}, 'utf8') !== ${JSON.stringify(changedSettings)}) throw Error('lost provision edits');
|
||||
fs.writeFileSync(${JSON.stringify(observed)}, 'second attempt succeeded');`);
|
||||
const [primary] = await db.insert(projectWorkspaces).values({ companyId: f.companyId, projectId: f.projectId!,
|
||||
name: "Primary repository", cwd: f.baseCwd, repoUrl: f.base.repoUrl, isPrimary: true }).returning();
|
||||
await db.update(issues).set({ projectId: f.projectId }).where(eq(issues.id, f.issueId));
|
||||
const row = await f.row();
|
||||
await db.update(executionWorkspaces).set({ projectWorkspaceId: primary!.id, metadata: { ...row.metadata,
|
||||
config: { provisionCommand: `node ${JSON.stringify(probe)}` } } }).where(eq(executionWorkspaces.id, f.workspaceId));
|
||||
expect((await f.expire()).expired).toBe(true);
|
||||
if (failure === "transaction_rollback") {
|
||||
const record = activityLog.logActivity;
|
||||
vi.spyOn(activityLog, "logActivity").mockImplementation(async (...args) => {
|
||||
if (args[1].action === "execution_workspace.reopen_failed") throw new Error("interrupted_reopen_transaction");
|
||||
return record(...args);
|
||||
});
|
||||
}
|
||||
expect(await f.reopen()).toMatchObject({ ok: false, code: "rebuild_failed" });
|
||||
expect(await fs.readFile(f.settingsPath, "utf8")).toBe(changedSettings);
|
||||
expect((await f.row()).metadata?.gitCacheRestoration).toMatchObject({ contentsRestoredAt: expect.any(String) });
|
||||
expect((await f.row()).metadata?.gitCacheRestoration).not.toHaveProperty("restoredAt");
|
||||
vi.restoreAllMocks();
|
||||
expect(await f.reopen()).toMatchObject({ ok: true, reopened: true });
|
||||
expect(await fs.readFile(observed, "utf8")).toBe("second attempt succeeded");
|
||||
expect(git(f.nested.cwd, "rev-parse", "HEAD")).toBe(f.nestedA);
|
||||
expect(await fs.readFile(f.settingsPath, "utf8")).toBe(changedSettings);
|
||||
});
|
||||
|
||||
it("resets the contents checkpoint before retrying a missing restored workspace", async () => {
|
||||
const f = await nestedFixture();
|
||||
expect((await f.expire()).expired).toBe(true);
|
||||
expect(await f.reopen()).toMatchObject({ ok: true });
|
||||
expect((await f.row()).metadata?.gitCacheRestoration).toMatchObject({ contentsRestoredAt: expect.any(String), restoredAt: expect.any(String) });
|
||||
await fs.rm(f.cwd, { recursive: true });
|
||||
await db.update(executionWorkspaces).set({ status: "archived" }).where(eq(executionWorkspaces.id, f.workspaceId));
|
||||
await fs.rename(f.nestedRemote, `${f.nestedRemote}.offline`);
|
||||
expect(await f.reopen()).toMatchObject({ ok: false, code: "rebuild_failed" });
|
||||
expect(await exists(f.cwd)).toBe(true);
|
||||
expect((await f.row()).metadata?.gitCacheRestoration).toMatchObject({ contentsRestoredAt: null, restoredAt: expect.any(String) });
|
||||
await fs.rename(`${f.nestedRemote}.offline`, f.nestedRemote);
|
||||
expect(await f.reopen()).toMatchObject({ ok: true, reopened: true });
|
||||
expect(git(f.nested.cwd, "rev-parse", "HEAD")).toBe(f.nestedA);
|
||||
expect(await fs.readFile(f.settingsPath)).toEqual(f.settings);
|
||||
});
|
||||
|
||||
it("resumes settings publication after linking but before temporary-file removal", async () => {
|
||||
const f = await nestedFixture();
|
||||
const settingsDirectory = await fs.realpath(path.dirname(f.settingsPath));
|
||||
expect((await f.expire()).expired).toBe(true);
|
||||
const originalRemove = fs.rm.bind(fs);
|
||||
const remove = vi.spyOn(fs, "rm").mockImplementation(async (target, options) => {
|
||||
if (path.dirname(String(target)) === settingsDirectory && String(target).includes(".settings-restore-")) throw new Error("interrupted_settings_publication");
|
||||
return originalRemove(target, options);
|
||||
});
|
||||
expect(await f.reopen()).toMatchObject({ ok: false, code: "rebuild_failed" });
|
||||
expect((await fs.stat(f.settingsPath)).nlink).toBe(2);
|
||||
remove.mockRestore();
|
||||
expect(await f.reopen()).toMatchObject({ ok: true });
|
||||
expect((await fs.stat(f.settingsPath)).nlink).toBe(1);
|
||||
expect(await fs.readFile(f.settingsPath)).toEqual(f.settings);
|
||||
});
|
||||
|
||||
it("retains tracked local settings whose permissions Git cannot reproduce", async () => {
|
||||
const f = await nestedFixture();
|
||||
git(f.cwd, "add", "-f", ".claude/settings.local.json");
|
||||
git(f.cwd, "commit", "-m", "Tracked settings");
|
||||
git(f.cwd, "push", "origin", "HEAD:refs/heads/fixture-task");
|
||||
expect(await f.expire()).toMatchObject({ expired: false, skipReason: "tracked_local_settings" });
|
||||
expect((await fs.stat(f.settingsPath)).mode & 0o777).toBe(0o600);
|
||||
});
|
||||
|
||||
it.each(["repoRef", "cwd"])("keeps the restored checkout through preparation after project %s changes", async selection => {
|
||||
const f = await nestedFixture();
|
||||
await fs.writeFile(path.join(f.nestedSource, "nested.txt"), "nested B\n");
|
||||
git(f.nestedSource, "add", "."); git(f.nestedSource, "commit", "-m", "Nested B");
|
||||
git(f.nestedSource, "push", f.nestedRemote, "HEAD:refs/heads/release");
|
||||
expect((await f.expire()).expired).toBe(true);
|
||||
const updated = await db.update(projectWorkspaces).set(selection === "repoRef" ? { repoRef: "release" } : { cwd: f.nestedSource })
|
||||
.where(eq(projectWorkspaces.id, f.preparation.workspaces[0]!.id)).returning();
|
||||
expect(await f.reopen()).toMatchObject({ ok: true });
|
||||
const row = await f.row();
|
||||
const prepared = await prepareProjectRepositoryWorkspaces({ ...f.preparation, workspaces: updated,
|
||||
restoredContents: readGitCacheRestoration(row)?.contents });
|
||||
expect(prepared).toEqual([f.nested]);
|
||||
expect(git(f.nested.cwd, "rev-parse", "HEAD")).toBe(f.nestedA);
|
||||
expect(await fs.readFile(f.nestedSettingsPath)).toEqual(f.settings);
|
||||
expect(await exists(path.join(f.cwd, ".paperclip-runtime", "detached-repositories"))).toBe(false);
|
||||
await db.update(executionWorkspaces).set({ metadata: { ...row.metadata, reopenPendingConsumption: false } })
|
||||
.where(eq(executionWorkspaces.id, f.workspaceId));
|
||||
expect(await f.expire()).toMatchObject({ expired: true });
|
||||
});
|
||||
|
||||
it("does not replace a missing restored checkout or admit a changed repository", async () => {
|
||||
const f = await nestedFixture();
|
||||
expect((await f.expire()).expired).toBe(true);
|
||||
expect(await f.reopen()).toMatchObject({ ok: true });
|
||||
const restoredContents = readGitCacheRestoration(await f.row())?.contents;
|
||||
await expect(prepareProjectRepositoryWorkspaces({ ...f.preparation, restoredContents,
|
||||
workspaces: [{ ...f.preparation.workspaces[0]!, repoUrl: pathToFileURL(f.source).href }] })).rejects.toThrow("could not be restored");
|
||||
await fs.rename(f.nested.cwd, `${f.nested.cwd}.retained`);
|
||||
await expect(prepareProjectRepositoryWorkspaces({ ...f.preparation, restoredContents })).rejects.toThrow("could not be restored");
|
||||
expect(await exists(f.nested.cwd)).toBe(false);
|
||||
expect(git(`${f.nested.cwd}.retained`, "rev-parse", "HEAD")).toBe(f.nestedA);
|
||||
});
|
||||
|
||||
it("retries a partial nested restoration when upstream returns", async () => {
|
||||
const f = await nestedFixture();
|
||||
expect((await f.expire()).expired).toBe(true);
|
||||
await fs.rename(f.nestedRemote, `${f.nestedRemote}.offline`);
|
||||
expect(await f.reopen()).toMatchObject({ ok: false, code: "rebuild_failed" });
|
||||
expect(await exists(f.cwd)).toBe(true);
|
||||
expect((await f.row()).metadata?.gitCacheRestoration).not.toHaveProperty("restoredAt");
|
||||
await fs.rename(`${f.nestedRemote}.offline`, f.nestedRemote);
|
||||
expect(await f.reopen()).toMatchObject({ ok: true });
|
||||
expect(git(f.nested.cwd, "rev-parse", "HEAD")).toBe(f.nestedA);
|
||||
expect(await fs.readFile(f.settingsPath)).toEqual(f.settings);
|
||||
});
|
||||
|
||||
it.each(["settings", "checkout", "legacy_recipe"])("retains changed %s and permits ordinary reopening", async kind => {
|
||||
const f = await nestedFixture();
|
||||
if (kind === "legacy_recipe") {
|
||||
expect((await f.expire({ beforeDelete: async () => { throw new Error("stop_before_delete"); } })).expired).toBe(false);
|
||||
const row = await f.row();
|
||||
const recipe = { ...(row.metadata!.gitCacheRestoration as Record<string, unknown>) };
|
||||
delete recipe.contents;
|
||||
await db.update(executionWorkspaces).set({ metadata: { ...row.metadata, gitCacheRestoration: recipe } }).where(eq(executionWorkspaces.id, f.workspaceId));
|
||||
}
|
||||
const outcome = await f.expire({ beforeDelete: async () => {
|
||||
if (kind === "settings") await fs.writeFile(f.settingsPath, "new local settings\n");
|
||||
if (kind === "checkout") {
|
||||
await fs.writeFile(path.join(f.nested.cwd, "nested.txt"), "nested B\n");
|
||||
git(f.nested.cwd, "add", "."); git(f.nested.cwd, "commit", "-m", "Nested B");
|
||||
git(f.nested.cwd, "push", "origin", "main");
|
||||
}
|
||||
} });
|
||||
expect(outcome).toMatchObject({ expired: false, reclaimedBytes: 0 });
|
||||
expect(await exists(f.cwd)).toBe(true);
|
||||
expect(await f.reopen()).toMatchObject({ ok: true });
|
||||
expect(await fs.readFile(f.settingsPath, "utf8")).toBe(kind === "settings" ? "new local settings\n" : f.settings.toString());
|
||||
expect(await fs.readFile(path.join(f.nested.cwd, "nested.txt"), "utf8")).toBe(kind === "checkout" ? "nested B\n" : "nested A\n");
|
||||
});
|
||||
|
||||
it("expires again after a reopened task consumes its fence and finishes", async () => {
|
||||
const f = await fixture();
|
||||
expect((await f.expire()).expired).toBe(true);
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
import { execFile as execFileCallback } from "node:child_process";
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||
import { assertReproducibleGitFiles, verifyRemoteGitCommit } from "../services/git-workspace-restoration.js";
|
||||
|
||||
const execFile = promisify(execFileCallback);
|
||||
|
||||
async function git(cwd: string, ...args: string[]) {
|
||||
const { stdout } = await execFile("git", ["-C", cwd, "-c", "user.name=Fixture", "-c", "user.email=fixture@example.test",
|
||||
"-c", "commit.gpgsign=false", ...args],
|
||||
{ env: { ...process.env, GIT_CONFIG_NOSYSTEM: "1", GIT_CONFIG_GLOBAL: os.devNull } });
|
||||
return stdout.trim();
|
||||
}
|
||||
|
||||
describe("reconstructable worktree FETCH_HEAD", () => {
|
||||
let root: string;
|
||||
let source: string;
|
||||
let remote: string;
|
||||
let worktree: string;
|
||||
let fetchHead: string;
|
||||
let initialCommit: string;
|
||||
|
||||
async function commit(value: string) {
|
||||
await fs.writeFile(path.join(source, "marker.txt"), `${value}\n`);
|
||||
await git(source, "add", "marker.txt");
|
||||
await git(source, "commit", "-m", value);
|
||||
return git(source, "rev-parse", "HEAD");
|
||||
}
|
||||
|
||||
beforeEach(async () => {
|
||||
root = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-fetch-head-")));
|
||||
source = path.join(root, "source");
|
||||
remote = path.join(root, "remote.git");
|
||||
worktree = path.join(root, "worktree");
|
||||
const base = path.join(root, "base");
|
||||
await fs.mkdir(source);
|
||||
await git(source, "init", "--initial-branch=main");
|
||||
initialCommit = await commit("A");
|
||||
await git(root, "clone", "--bare", source, remote);
|
||||
await git(root, "clone", remote, base);
|
||||
await git(base, "worktree", "add", "-b", "task", worktree, "HEAD");
|
||||
fetchHead = path.join(await git(worktree, "rev-parse", "--absolute-git-dir"), "FETCH_HEAD");
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await fs.rm(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("accepts ordinary fetched metadata already contained in the workspace HEAD", async () => {
|
||||
await git(worktree, "fetch", "origin", "main");
|
||||
expect(await fs.readFile(fetchHead, "utf8")).toContain(`${initialCommit}\t\t`);
|
||||
await expect(assertReproducibleGitFiles(worktree)).resolves.toEqual([]);
|
||||
});
|
||||
|
||||
it("verifies a fetched upstream commit before discarding metadata ahead of the workspace", async () => {
|
||||
const upstream = await commit("B");
|
||||
await git(source, "push", remote, "main");
|
||||
await git(worktree, "fetch", "origin", "main");
|
||||
await git(worktree, "fetch", "--append", "origin", "main");
|
||||
const original = await fs.readFile(fetchHead);
|
||||
await expect(assertReproducibleGitFiles(worktree)).rejects.toThrow("unverified_fetched_commit");
|
||||
const verified: string[] = [];
|
||||
await expect(assertReproducibleGitFiles(worktree, undefined, {
|
||||
verifyFetchedCommit: async (sha, signal) => {
|
||||
await verifyRemoteGitCommit(remote, sha, async () => null, signal);
|
||||
verified.push(sha);
|
||||
},
|
||||
})).resolves.toEqual([]);
|
||||
expect(verified).toEqual([upstream]);
|
||||
expect(await git(worktree, "rev-parse", "HEAD")).toBe(initialCommit);
|
||||
expect(await fs.readFile(fetchHead)).toEqual(original);
|
||||
});
|
||||
|
||||
it("preserves a fetched local-only commit that the configured upstream cannot restore", async () => {
|
||||
const unpublished = await commit("Unpublished");
|
||||
await git(worktree, "fetch", source, "main");
|
||||
const original = await fs.readFile(fetchHead);
|
||||
expect(await git(worktree, "rev-parse", "FETCH_HEAD")).toBe(unpublished);
|
||||
await expect(assertReproducibleGitFiles(worktree, undefined, {
|
||||
verifyFetchedCommit: (sha, signal) => verifyRemoteGitCommit(remote, sha, async () => null, signal),
|
||||
})).rejects.toThrow("could not be restored");
|
||||
expect(await fs.readFile(fetchHead)).toEqual(original);
|
||||
expect(await git(worktree, "cat-file", "-t", unpublished)).toBe("commit");
|
||||
});
|
||||
|
||||
it("preserves malformed or oversized FETCH_HEAD and bounds distinct commit proofs", async () => {
|
||||
const variants = [
|
||||
{ value: "operator notes\n", reason: "unsupported_worktree_fetch_head" },
|
||||
{ value: `${initialCommit}\tunknown-marker\tbranch 'main'\n`, reason: "unsupported_worktree_fetch_head" },
|
||||
{ value: `${initialCommit}\t\tbranch 'main'`, reason: "unsupported_worktree_fetch_head" },
|
||||
{ value: `${initialCommit}\t\tbranch 'main'\0\n`, reason: "unsupported_worktree_fetch_head" },
|
||||
{ value: "x".repeat(64 * 1024 + 1), reason: "unsupported_worktree_fetch_head" },
|
||||
{ value: Array.from({ length: 33 }, (_, index) => `${index.toString(16).padStart(40, "0")}\t\tbranch 'main'\n`).join(""),
|
||||
reason: "worktree_fetch_head_budget" },
|
||||
];
|
||||
for (const { value, reason } of variants) {
|
||||
await fs.writeFile(fetchHead, value);
|
||||
await expect(assertReproducibleGitFiles(worktree, undefined, {
|
||||
verifyFetchedCommit: (sha, signal) => verifyRemoteGitCommit(remote, sha, async () => null, signal),
|
||||
})).rejects.toThrow(reason);
|
||||
expect(await fs.readFile(fetchHead, "utf8")).toBe(value);
|
||||
}
|
||||
});
|
||||
|
||||
it("preserves FETCH_HEAD when its path is a symlink or directory", async () => {
|
||||
await git(worktree, "fetch", "origin", "main");
|
||||
const original = await fs.readFile(fetchHead);
|
||||
const outside = path.join(root, "retained-fetch-head");
|
||||
await fs.rename(fetchHead, outside);
|
||||
await fs.symlink(outside, fetchHead);
|
||||
await expect(assertReproducibleGitFiles(worktree)).rejects.toThrow("unsupported_worktree_fetch_head");
|
||||
expect((await fs.lstat(fetchHead)).isSymbolicLink()).toBe(true);
|
||||
expect(await fs.readFile(outside)).toEqual(original);
|
||||
await fs.unlink(fetchHead);
|
||||
await fs.mkdir(fetchHead);
|
||||
await expect(assertReproducibleGitFiles(worktree)).rejects.toThrow("unsupported_worktree_fetch_head");
|
||||
expect((await fs.lstat(fetchHead)).isDirectory()).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -3,7 +3,7 @@ import { measureRunPhase } from "./execution-preparation-timing.js";
|
||||
import { assertTaskWorkspaceAccess, assertTaskWorkspaceSourceProjectAccess } from "./task-workspace-source-access.js";
|
||||
import { verifiedRecordedIsolatedRemoteSource, withIsolatedRepositorySourceLock } from "./isolated-remote-workspace-source.js";
|
||||
import { expireReconstructableGitWorkspace, hasPendingGitExpiration } from "./git-workspace-expiration.js";
|
||||
import { readGitCacheRestoration, restoreGitCacheBase, GitCacheRestorationFailure } from "./git-workspace-restoration.js";
|
||||
import { readGitCacheRestoration, restoreGitCacheBase, recoverGitCacheExpirationLocks, GitCacheRestorationFailure } from "./git-workspace-restoration.js";
|
||||
import { parseObject } from "../adapters/utils.js";
|
||||
import { resolveCoreTrustPreset } from "./trust-preset-resolver.js";
|
||||
import { CONTROLLER_PROVISION_COPY_ONLY, workspaceRequiresCopyOnlyProvisioning } from "./workspace-host-provision.js";
|
||||
@@ -3808,6 +3808,8 @@ export function executionWorkspaceService(db: Db, opts: ExecutionWorkspaceServic
|
||||
return { ok: false, code: "not_reopenable", message: "Execution workspace source changed during reopen" };
|
||||
}
|
||||
}
|
||||
try { await recoverGitCacheExpirationLocks(row); }
|
||||
catch { return { ok: false, code: "rebuild_failed", message: new GitCacheRestorationFailure().message }; }
|
||||
const verifyCopyIdentity = capturedSource && authorizedTarget ? async (sourceCwd: string, targetCwd: string) => {
|
||||
if (!capturedSource!.roots.includes(await fs.realpath(sourceCwd))
|
||||
|| await canonicalRepositoryRoot(targetCwd) !== authorizedTarget) {
|
||||
@@ -3863,6 +3865,27 @@ export function executionWorkspaceService(db: Db, opts: ExecutionWorkspaceServic
|
||||
const realized = await ensurePersistedExecutionWorkspaceAvailable({
|
||||
db: tx as unknown as Db,
|
||||
resolveGitAuth: gitAuth,
|
||||
persistContentsRestoredAt: async at => {
|
||||
const currentRecipe = nextMetadata.gitCacheRestoration;
|
||||
const updated = await db.update(executionWorkspaces).set({
|
||||
metadata: sql`jsonb_set(${executionWorkspaces.metadata}, '{gitCacheRestoration,contentsRestoredAt}', ${JSON.stringify(at)}::jsonb)`,
|
||||
}).where(and(eq(executionWorkspaces.id, row.id), eq(executionWorkspaces.companyId, row.companyId),
|
||||
sql`${executionWorkspaces.metadata}->'gitCacheRestoration' = ${JSON.stringify(currentRecipe)}::jsonb`))
|
||||
.returning({ id: executionWorkspaces.id });
|
||||
if (updated.length !== 1) throw new GitCacheRestorationFailure();
|
||||
nextMetadata.gitCacheRestoration = { ...currentRecipe as Record<string, unknown>, contentsRestoredAt: at };
|
||||
},
|
||||
prepareRestoredLocalSource: async sourceCwd => {
|
||||
const captured = await assertIsolatedWorkspaceSourceReadAccess(tx as unknown as Db, {
|
||||
companyId: row.companyId, sourceCwd, actor: input.authorizationActor, copyKind: "repository",
|
||||
});
|
||||
const recheck = () => retainIsolatedWorkspaceSourceAccess(tx as unknown as Db, {
|
||||
companyId: row.companyId, sourceCwd, targetCwd: row.cwd!, actor: input.authorizationActor,
|
||||
captured, issueId: issue.id, copyKind: "repository",
|
||||
});
|
||||
await recheck();
|
||||
return recheck;
|
||||
},
|
||||
skipHostProvision: trust.kind === "low_trust_review" || copyOnlyProvisioning,
|
||||
beforeSourceCopy: verifyCopyIdentity,
|
||||
afterSourceCopy: verifyCopyIdentity,
|
||||
@@ -3969,7 +3992,7 @@ export function executionWorkspaceService(db: Db, opts: ExecutionWorkspaceServic
|
||||
// Keep the refresh guard after reopening so normal preparation cannot
|
||||
// advance the saved checkout to upstream HEAD. Its intact branch may now
|
||||
// receive newer task commits; the next expiration records a fresh recipe.
|
||||
if (recipe) nextMetadata.gitCacheRestoration = { ...recipe, restoredAt: now().toISOString() };
|
||||
if (recipe) nextMetadata.gitCacheRestoration = { ...nextMetadata.gitCacheRestoration as Record<string, unknown>, restoredAt: now().toISOString() };
|
||||
const activeMetadata = setMetadataReopenPendingConsumption(nextMetadata, now());
|
||||
await measureRunPhase("workspace.notification", () => notifyTerminalWorkspaceWork(tx, `workspace:${row.id}`));
|
||||
const activeRow = await tx
|
||||
|
||||
@@ -0,0 +1,229 @@
|
||||
import fs from "node:fs/promises";
|
||||
import { constants } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { createHash } from "node:crypto";
|
||||
import { isDeepStrictEqual } from "node:util";
|
||||
import { PROJECT_REPOSITORIES_DIR } from "@paperclipai/adapter-utils/git-workspace-sync";
|
||||
import { localEncryptedProvider } from "../secrets/local-encrypted-provider.js";
|
||||
import type { StoredSecretVersionMaterial } from "../secrets/types.js";
|
||||
import type { GitRemoteAuthProvider } from "./git-credentials.js";
|
||||
import { localRepositoryPath } from "./local-repository-path.js";
|
||||
import { managedProjectCheckoutName } from "./managed-project-checkout-name.js";
|
||||
import { materializeManagedProjectWorkspace } from "./managed-repository-checkout.js";
|
||||
import { ensureSavedGitCommit, gitCacheCommand, GitCacheRestorationFailure } from "./git-workspace-restoration.js";
|
||||
|
||||
export const CLAUDE_SETTINGS_PATH = ".claude/settings.local.json";
|
||||
const SETTINGS_LIMIT = 64 * 1024;
|
||||
const CHECKOUT_LIMIT = 50;
|
||||
type Identity = { companyId: string; workspaceId: string };
|
||||
type Settings = { material: StoredSecretVersionMaterial } | null;
|
||||
export type ManagedCheckoutSource = { id: string; repoUrl: string; repoRef: string | null; cwd: string | null };
|
||||
export type ManagedCheckoutContents = {
|
||||
name: string; source: ManagedCheckoutSource; commit: string; branchName: string | null;
|
||||
};
|
||||
export type GitWorkspaceContents = {
|
||||
schema: "paperclip.git-workspace-contents.v1";
|
||||
settings: Settings;
|
||||
checkouts: Array<ManagedCheckoutContents & { settings: Settings }>;
|
||||
};
|
||||
|
||||
async function entry(target: string) {
|
||||
return fs.lstat(target).catch((error: NodeJS.ErrnoException) => {
|
||||
if (error.code === "ENOENT") return null;
|
||||
throw error;
|
||||
});
|
||||
}
|
||||
|
||||
async function checkedDirectory(cwd: string, relative: string, create = false) {
|
||||
let current = await fs.realpath(cwd);
|
||||
for (const segment of relative.split("/")) {
|
||||
current = path.join(current, segment);
|
||||
if (create) await fs.mkdir(current, { mode: 0o700 }).catch((error: NodeJS.ErrnoException) => {
|
||||
if (error.code !== "EEXIST") throw error;
|
||||
});
|
||||
const stat = await entry(current);
|
||||
if (!stat) return null;
|
||||
if (!stat.isDirectory() || stat.isSymbolicLink()) throw new GitCacheRestorationFailure();
|
||||
}
|
||||
return current;
|
||||
}
|
||||
|
||||
async function readSettings(cwd: string) {
|
||||
if (!await checkedDirectory(cwd, ".claude")) return null;
|
||||
const target = path.join(cwd, CLAUDE_SETTINGS_PATH);
|
||||
const initial = await entry(target);
|
||||
if (!initial) return null;
|
||||
if (!initial.isFile() || initial.isSymbolicLink() || initial.nlink !== 1 || initial.size > SETTINGS_LIMIT) throw new Error("unsupported_local_settings");
|
||||
if (await gitCacheCommand(cwd, ["ls-files", "--error-unmatch", "--", CLAUDE_SETTINGS_PATH]).then(() => true, () => false)) throw new Error("tracked_local_settings");
|
||||
const handle = await fs.open(target, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK);
|
||||
try {
|
||||
const stat = await handle.stat();
|
||||
if (!stat.isFile() || stat.nlink !== 1 || stat.size > SETTINGS_LIMIT) throw new Error("unsupported_local_settings");
|
||||
const buffer = Buffer.alloc(SETTINGS_LIMIT + 1);
|
||||
let length = 0;
|
||||
while (length < buffer.length) {
|
||||
const { bytesRead } = await handle.read(buffer, length, buffer.length - length, length);
|
||||
if (bytesRead === 0) break;
|
||||
length += bytesRead;
|
||||
}
|
||||
if (length > SETTINGS_LIMIT) throw new Error("unsupported_local_settings");
|
||||
return { mode: stat.mode & 0o777, bytes: buffer.subarray(0, length).toString("base64") };
|
||||
} finally { await handle.close(); }
|
||||
}
|
||||
|
||||
async function saveSettings(cwd: string, identity: Identity, relativePath: string): Promise<Settings> {
|
||||
const value = await readSettings(cwd);
|
||||
if (!value) return null;
|
||||
const { material } = await localEncryptedProvider.createVersion({ value: JSON.stringify({ ...identity, relativePath, ...value }) });
|
||||
return { material };
|
||||
}
|
||||
|
||||
async function savedSettings(settings: Settings, identity: Identity, relativePath: string) {
|
||||
if (settings === null) return null;
|
||||
const value = JSON.parse(await localEncryptedProvider.resolveVersion({ material: settings.material, externalRef: null }));
|
||||
if (value.companyId !== identity.companyId || value.workspaceId !== identity.workspaceId || value.relativePath !== relativePath
|
||||
|| !Number.isInteger(value.mode) || value.mode < 0 || value.mode > 0o777
|
||||
|| typeof value.bytes !== "string" || value.bytes.length > Math.ceil(SETTINGS_LIMIT / 3) * 4
|
||||
|| Buffer.from(value.bytes, "base64").toString("base64") !== value.bytes) throw new GitCacheRestorationFailure();
|
||||
return { mode: value.mode as number, bytes: value.bytes as string };
|
||||
}
|
||||
|
||||
export function validateGitWorkspaceContents(contents: GitWorkspaceContents | undefined) {
|
||||
if (contents === undefined) return;
|
||||
const settingsValid = (settings: Settings) => settings === null || !!settings && typeof settings.material === "object"
|
||||
&& settings.material !== null && JSON.stringify(settings.material).length <= SETTINGS_LIMIT * 2;
|
||||
if (!contents || contents.schema !== "paperclip.git-workspace-contents.v1" || !settingsValid(contents.settings)
|
||||
|| !Array.isArray(contents.checkouts) || contents.checkouts.length > CHECKOUT_LIMIT) throw new GitCacheRestorationFailure();
|
||||
const names = new Set<string>();
|
||||
for (const checkout of contents.checkouts) {
|
||||
if (!checkout || !checkout.source || typeof checkout.source.id !== "string" || typeof checkout.source.repoUrl !== "string"
|
||||
|| !(checkout.source.repoRef === null || typeof checkout.source.repoRef === "string")
|
||||
|| !(checkout.source.cwd === null || typeof checkout.source.cwd === "string")
|
||||
|| typeof checkout.name !== "string" || !/^[A-Za-z0-9_-]+-[a-f0-9]{12}$/.test(checkout.name)
|
||||
|| checkout.name !== managedProjectCheckoutName(checkout.source) || names.has(checkout.name)
|
||||
|| !/^[a-f0-9]{40}$/.test(checkout.commit)
|
||||
|| !(checkout.branchName === null || typeof checkout.branchName === "string" && checkout.branchName.length > 0
|
||||
&& checkout.branchName.length <= 1024 && !checkout.branchName.startsWith("-") && !checkout.branchName.includes("\0"))
|
||||
|| !settingsValid(checkout.settings)) throw new GitCacheRestorationFailure();
|
||||
names.add(checkout.name);
|
||||
}
|
||||
}
|
||||
|
||||
export function selectRestoredProjectRepositories<T extends { id: string; repoUrl: string | null; repoRef: string | null; cwd?: string | null }>(sources: T[], contents?: GitWorkspaceContents): T[] {
|
||||
validateGitWorkspaceContents(contents);
|
||||
for (const checkout of contents?.checkouts ?? []) {
|
||||
if (!sources.some(source => source.id === checkout.source.id && source.repoUrl === checkout.source.repoUrl)) throw new GitCacheRestorationFailure();
|
||||
}
|
||||
return sources.map(source => {
|
||||
const saved = contents?.checkouts.find(checkout => checkout.source.id === source.id)?.source;
|
||||
return saved ? { ...source, repoRef: saved.repoRef, cwd: saved.cwd } : source;
|
||||
});
|
||||
}
|
||||
|
||||
export async function readManagedCheckoutContents(checkout: string, name: string, source: ManagedCheckoutSource, signal?: AbortSignal): Promise<ManagedCheckoutContents> {
|
||||
const commit = (await gitCacheCommand(checkout, ["rev-parse", "HEAD"], signal)).trim();
|
||||
const branchName = (await gitCacheCommand(checkout, ["symbolic-ref", "--quiet", "--short", "HEAD"], signal).catch(() => "")).trim() || null;
|
||||
return { name, source, commit, branchName };
|
||||
}
|
||||
|
||||
export async function captureGitWorkspaceContents(cwd: string, identity: Identity, checkouts: ManagedCheckoutContents[]): Promise<GitWorkspaceContents> {
|
||||
if (checkouts.length > CHECKOUT_LIMIT) throw new Error("managed_checkout_budget");
|
||||
const contents: GitWorkspaceContents = { schema: "paperclip.git-workspace-contents.v1",
|
||||
settings: await saveSettings(cwd, identity, CLAUDE_SETTINGS_PATH), checkouts: [] };
|
||||
for (const checkout of checkouts.sort((a, b) => a.name.localeCompare(b.name))) {
|
||||
const relative = `${PROJECT_REPOSITORIES_DIR}/${checkout.name}`;
|
||||
if (!await checkedDirectory(cwd, relative)) throw new GitCacheRestorationFailure();
|
||||
contents.checkouts.push({ ...checkout, settings: await saveSettings(path.join(cwd, relative), identity, `${relative}/${CLAUDE_SETTINGS_PATH}`) });
|
||||
}
|
||||
validateGitWorkspaceContents(contents);
|
||||
return contents;
|
||||
}
|
||||
|
||||
export async function assertGitWorkspaceContentsUnchanged(cwd: string, identity: Identity, contents: GitWorkspaceContents | undefined, checkouts: ManagedCheckoutContents[]) {
|
||||
validateGitWorkspaceContents(contents);
|
||||
const recorded = contents?.checkouts.map(({ settings: _settings, ...checkout }) => checkout) ?? [];
|
||||
if (!isDeepStrictEqual(recorded, checkouts.sort((a, b) => a.name.localeCompare(b.name)))) throw new Error("managed_checkouts_changed");
|
||||
const assertSettings = async (root: string, saved: Settings, relative: string) => {
|
||||
if (!isDeepStrictEqual(await readSettings(root), await savedSettings(saved, identity, relative))) throw new Error("local_settings_changed");
|
||||
};
|
||||
await assertSettings(cwd, contents?.settings ?? null, CLAUDE_SETTINGS_PATH);
|
||||
for (const checkout of contents?.checkouts ?? []) {
|
||||
const relative = `${PROJECT_REPOSITORIES_DIR}/${checkout.name}`;
|
||||
if (!await checkedDirectory(cwd, relative)) throw new Error("managed_checkouts_changed");
|
||||
await assertSettings(path.join(cwd, relative), checkout.settings, `${relative}/${CLAUDE_SETTINGS_PATH}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function restoreSettings(cwd: string, settings: Settings, identity: Identity, relative: string) {
|
||||
const saved = await savedSettings(settings, identity, relative);
|
||||
const directory = await checkedDirectory(cwd, ".claude", saved !== null);
|
||||
if (!directory) return;
|
||||
const target = path.join(directory, "settings.local.json");
|
||||
const temporary = path.join(directory, `.settings-restore-${createHash("sha256").update(JSON.stringify(settings)).digest("hex")}`);
|
||||
const pending = await entry(temporary);
|
||||
if (pending) {
|
||||
if (!saved || !pending.isFile() || pending.isSymbolicLink() || pending.size > SETTINGS_LIMIT) throw new GitCacheRestorationFailure();
|
||||
const current = await entry(target);
|
||||
const expected = Buffer.from(saved.bytes, "base64");
|
||||
const bytes = await fs.readFile(temporary);
|
||||
const unpublished = pending.nlink === 1 && !current && bytes.length <= expected.length
|
||||
&& bytes.equals(expected.subarray(0, bytes.length)) && [0o600, saved.mode].includes(pending.mode & 0o777);
|
||||
const published = pending.nlink === 2 && current?.ino === pending.ino && current.dev === pending.dev
|
||||
&& bytes.equals(expected) && (pending.mode & 0o777) === saved.mode;
|
||||
if (!unpublished && !published) throw new GitCacheRestorationFailure();
|
||||
await fs.unlink(temporary);
|
||||
}
|
||||
const current = await readSettings(cwd);
|
||||
if (current) {
|
||||
if (!isDeepStrictEqual(current, saved)) throw new GitCacheRestorationFailure();
|
||||
return;
|
||||
}
|
||||
if (!saved) return;
|
||||
try {
|
||||
await fs.writeFile(temporary, Buffer.from(saved.bytes, "base64"), { flag: "wx", mode: 0o600 });
|
||||
await fs.chmod(temporary, saved.mode);
|
||||
await fs.link(temporary, target);
|
||||
} finally { await fs.rm(temporary, { force: true }); }
|
||||
}
|
||||
|
||||
export async function restoreGitWorkspaceContents(input: Identity & { cwd: string; contents?: GitWorkspaceContents;
|
||||
prepareLocalSource?: (sourceCwd: string) => Promise<() => Promise<void>>;
|
||||
resolveGitAuth: GitRemoteAuthProvider; authorizeSource: (source: ManagedCheckoutSource) => Promise<void> }) {
|
||||
if (!input.contents) return;
|
||||
validateGitWorkspaceContents(input.contents);
|
||||
try {
|
||||
for (const checkout of input.contents.checkouts) {
|
||||
await input.authorizeSource(checkout.source);
|
||||
const localSource = localRepositoryPath(checkout.source.repoUrl);
|
||||
if (localSource && !input.prepareLocalSource) throw new GitCacheRestorationFailure();
|
||||
const recheck = localSource ? await input.prepareLocalSource!(localSource) : undefined;
|
||||
const root = await checkedDirectory(input.cwd, PROJECT_REPOSITORIES_DIR, true);
|
||||
const destination = path.join(root!, checkout.name);
|
||||
const relative = `${PROJECT_REPOSITORIES_DIR}/${checkout.name}/${CLAUDE_SETTINGS_PATH}`;
|
||||
const existing = await entry(destination);
|
||||
if (existing) {
|
||||
if (!existing.isDirectory() || existing.isSymbolicLink() || !(await entry(path.join(destination, ".git")))?.isDirectory()
|
||||
|| (await fs.realpath(path.join(destination, ".git"))) !== path.join(destination, ".git")) throw new GitCacheRestorationFailure();
|
||||
if ((await gitCacheCommand(destination, ["rev-parse", "--show-toplevel"])).trim() !== destination
|
||||
|| (await gitCacheCommand(destination, ["config", "--local", "--null", "--get-all", "remote.origin.url"])) !== `${checkout.source.repoUrl}\0`) throw new GitCacheRestorationFailure();
|
||||
const actual = await readManagedCheckoutContents(destination, checkout.name, checkout.source);
|
||||
if (actual.commit !== checkout.commit || actual.branchName !== checkout.branchName) throw new GitCacheRestorationFailure();
|
||||
await restoreSettings(destination, checkout.settings, input, relative);
|
||||
continue;
|
||||
}
|
||||
const result = await materializeManagedProjectWorkspace(destination, { repoUrl: checkout.source.repoUrl,
|
||||
resolveGitAuth: input.resolveGitAuth, beforePublish: async temporary => {
|
||||
await ensureSavedGitCommit(temporary, checkout.source.repoUrl, checkout.commit, input.resolveGitAuth);
|
||||
if (checkout.branchName) await gitCacheCommand(temporary, ["check-ref-format", "--branch", checkout.branchName]);
|
||||
await gitCacheCommand(temporary, checkout.branchName
|
||||
? ["checkout", "-B", checkout.branchName, checkout.commit] : ["checkout", "--detach", checkout.commit]);
|
||||
await gitCacheCommand(temporary, ["reflog", "expire", "--expire=now", "--all"]);
|
||||
await restoreSettings(temporary, checkout.settings, input, relative);
|
||||
await input.authorizeSource(checkout.source);
|
||||
await recheck?.();
|
||||
} });
|
||||
if (result.warning) throw new GitCacheRestorationFailure();
|
||||
}
|
||||
await restoreSettings(input.cwd, input.contents.settings, input, CLAUDE_SETTINGS_PATH);
|
||||
} catch { throw new GitCacheRestorationFailure(); }
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
import fs from "node:fs/promises";
|
||||
import { constants } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { createHash } from "node:crypto";
|
||||
|
||||
export type GitExpirationLockLocation = { root: string; relativePath: string; checkout: string | null };
|
||||
|
||||
function marker(token: string) {
|
||||
if (!/^[a-f0-9]{64}$/.test(token)) throw new Error("invalid_expiration_lock_token");
|
||||
return Buffer.from(`paperclip-git-expiration-lock-v1:${token}\n`);
|
||||
}
|
||||
|
||||
async function lockPaths(location: GitExpirationLockLocation, token: string, create: boolean) {
|
||||
const segments = location.relativePath.split(path.sep);
|
||||
if (path.isAbsolute(location.relativePath) || segments.some(segment => !segment || segment === "." || segment === "..")) throw new Error("unsafe_expiration_lock_path");
|
||||
const root = await fs.lstat(location.root).catch((error: NodeJS.ErrnoException) => {
|
||||
if (error.code === "ENOENT") return null;
|
||||
throw error;
|
||||
});
|
||||
if (!root) {
|
||||
if (create) throw new Error("missing_expiration_lock_root");
|
||||
return null;
|
||||
}
|
||||
if (!root.isDirectory() || root.isSymbolicLink()) throw new Error("unsafe_expiration_lock_path");
|
||||
let parent = await fs.realpath(location.root);
|
||||
for (const segment of segments.slice(0, -1)) {
|
||||
parent = path.join(parent, segment);
|
||||
if (create) await fs.mkdir(parent, { mode: 0o700 }).catch((error: NodeJS.ErrnoException) => { if (error.code !== "EEXIST") throw error; });
|
||||
const stat = await fs.lstat(parent).catch((error: NodeJS.ErrnoException) => { if (error.code === "ENOENT") return null; throw error; });
|
||||
if (!stat) return null;
|
||||
if (!stat.isDirectory() || stat.isSymbolicLink()) throw new Error("unsafe_expiration_lock_path");
|
||||
}
|
||||
const digest = createHash("sha256").update(location.relativePath).digest("hex").slice(0, 16);
|
||||
return { lock: path.join(parent, segments.at(-1)!), temporary: path.join(parent, `.paperclip-expiration-${token}-${digest}`) };
|
||||
}
|
||||
|
||||
async function removeOwnedFile(target: string, expected: Buffer, allowPartial: boolean) {
|
||||
const handle = await fs.open(target, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK)
|
||||
.catch((error: NodeJS.ErrnoException) => { if (error.code === "ENOENT") return null; throw error; });
|
||||
if (!handle) return;
|
||||
try {
|
||||
const stat = await handle.stat();
|
||||
if (!stat.isFile() || stat.size > expected.length || stat.nlink > 2) throw new Error("git_expiration_lock_conflict");
|
||||
const bytes = Buffer.alloc(expected.length + 1);
|
||||
const { bytesRead } = await handle.read(bytes, 0, bytes.length, 0);
|
||||
const actual = bytes.subarray(0, bytesRead);
|
||||
if (actual.length !== stat.size) throw new Error("git_expiration_lock_conflict");
|
||||
if (!(allowPartial && stat.nlink === 1 ? actual.equals(expected.subarray(0, actual.length)) : actual.equals(expected))) {
|
||||
throw new Error("git_expiration_lock_conflict");
|
||||
}
|
||||
const current = await fs.lstat(target);
|
||||
if (current.ino !== stat.ino || current.dev !== stat.dev || !current.isFile()) throw new Error("git_expiration_lock_conflict");
|
||||
await fs.unlink(target);
|
||||
} finally { await handle.close(); }
|
||||
}
|
||||
|
||||
export async function recoverGitExpirationLocks(locations: GitExpirationLockLocation[], token: string) {
|
||||
const expected = marker(token);
|
||||
for (const location of locations) {
|
||||
const paths = await lockPaths(location, token, false);
|
||||
if (!paths) continue;
|
||||
await removeOwnedFile(paths.temporary, expected, true);
|
||||
await removeOwnedFile(paths.lock, expected, false);
|
||||
}
|
||||
}
|
||||
|
||||
export async function acquireGitExpirationLocks(locations: GitExpirationLockLocation[], token: string) {
|
||||
const expected = marker(token);
|
||||
const owned: GitExpirationLockLocation[] = [];
|
||||
try {
|
||||
for (const location of locations) {
|
||||
const paths = (await lockPaths(location, token, true))!;
|
||||
const handle = await fs.open(paths.temporary, "wx", 0o600);
|
||||
owned.push(location);
|
||||
try { await handle.writeFile(expected); await handle.sync(); } finally { await handle.close(); }
|
||||
await fs.link(paths.temporary, paths.lock);
|
||||
await fs.unlink(paths.temporary);
|
||||
}
|
||||
} catch (error) {
|
||||
await recoverGitExpirationLocks(owned, token);
|
||||
throw error;
|
||||
}
|
||||
return { release: () => recoverGitExpirationLocks(owned, token) };
|
||||
}
|
||||
@@ -1,5 +1,8 @@
|
||||
import { captureGitWorkspaceContents, assertGitWorkspaceContentsUnchanged, readManagedCheckoutContents, selectRestoredProjectRepositories, type ManagedCheckoutContents } from "./git-workspace-contents.js";
|
||||
import fs from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { acquireGitExpirationLocks } from "./git-workspace-expiration-locks.js";
|
||||
import { and, eq, ne, sql } from "drizzle-orm";
|
||||
import { executionWorkspaces, issues, heartbeatRuns, nativeRunFinalizations, environmentLeases,
|
||||
workspaceRuntimeServices, issueRecoveryActions, projectWorkspaces, withDedicatedDbConnection, type Db } from "@paperclipai/db";
|
||||
@@ -8,7 +11,7 @@ import { withIsolatedRepositorySourceLock, verifiedRecordedIsolatedRemoteSource
|
||||
import { managedProjectCheckoutName } from "./managed-project-checkout-name.js";
|
||||
import type { GitRemoteAuthProvider } from "./git-credentials.js";
|
||||
import { assertReproducibleGitFiles, assertReproducibleBaseGitStorage, assertReproducibleManagedCheckout, inspectGitCacheRecipe, readGitCacheRestoration,
|
||||
verifyRemoteGitCommit, GitCacheRestorationFailure, type ManagedCheckoutProver, gitCacheCommand, readGitCacheExpiration, finishGitCacheTrash, type GitCacheExpiration, GIT_CACHE_RESTORATION_KEY, GIT_CACHE_EXPIRATION_REASON } from "./git-workspace-restoration.js";
|
||||
verifyRemoteGitCommit, gitCacheExpirationLockLocations, recoverGitCacheExpirationLocks, GitCacheRestorationFailure, type ManagedCheckoutProver, gitCacheCommand, readGitCacheExpiration, finishGitCacheTrash, type GitCacheExpiration, GIT_CACHE_RESTORATION_KEY, GIT_CACHE_EXPIRATION_REASON } from "./git-workspace-restoration.js";
|
||||
|
||||
async function allocatedBytes(target: string, shouldContinue: () => boolean): Promise<number> {
|
||||
if (!shouldContinue()) throw new Error("scan_stopped");
|
||||
@@ -135,17 +138,18 @@ async function inspectBase(db: Db, workspace: Workspace, signal?: AbortSignal, a
|
||||
return allocatedBytes(recipe.sourcePath, () => { signal?.throwIfAborted(); return true; });
|
||||
}
|
||||
|
||||
function managedCheckoutProver(db: Db, workspace: Workspace, auth: GitRemoteAuthProvider): ManagedCheckoutProver {
|
||||
function managedCheckoutProver(db: Db, workspace: Workspace, auth: GitRemoteAuthProvider, checkouts: Map<string, ManagedCheckoutContents>): ManagedCheckoutProver {
|
||||
let rows: Promise<Array<{ id: string; repoUrl: string | null; repoRef: string | null; cwd: string | null }>> | undefined;
|
||||
return async (checkoutPath, name, signal) => {
|
||||
if (!workspace.projectId) throw new Error("unknown_managed_checkout");
|
||||
rows ??= db.select({ id: projectWorkspaces.id, repoUrl: projectWorkspaces.repoUrl, repoRef: projectWorkspaces.repoRef,
|
||||
cwd: projectWorkspaces.cwd }).from(projectWorkspaces)
|
||||
.where(and(eq(projectWorkspaces.companyId, workspace.companyId), eq(projectWorkspaces.projectId, workspace.projectId)));
|
||||
const source = (await rows).find(row => row.repoUrl
|
||||
const source = selectRestoredProjectRepositories(await rows, readGitCacheRestoration(workspace)?.contents).find(row => row.repoUrl
|
||||
&& managedProjectCheckoutName({ id: row.id, repoUrl: row.repoUrl, repoRef: row.repoRef, cwd: row.cwd }) === name);
|
||||
if (!source?.repoUrl) throw new Error("unknown_managed_checkout");
|
||||
await assertReproducibleManagedCheckout(checkoutPath, { repoUrl: source.repoUrl, resolveGitAuth: auth, signal });
|
||||
checkouts.set(name, await readManagedCheckoutContents(checkoutPath, name, { ...source, repoUrl: source.repoUrl }, signal));
|
||||
};
|
||||
}
|
||||
|
||||
@@ -169,9 +173,12 @@ export async function expireReconstructableGitWorkspace(db: Db, snapshot: Worksp
|
||||
if (!hasPendingGitExpiration(workspace) && !["active", "idle", "in_review"].includes(workspace.status)) throw new Error("not_expirable");
|
||||
if (["active", "idle", "in_review"].includes(workspace.status)) state = null;
|
||||
const auth = expirationGitAuth(db, workspace);
|
||||
const proveManagedCheckout = managedCheckoutProver(db, workspace, auth);
|
||||
const checkouts = new Map<string, ManagedCheckoutContents>();
|
||||
const identity = { companyId: workspace.companyId, workspaceId: workspace.id };
|
||||
const proveManagedCheckout = managedCheckoutProver(db, workspace, auth, checkouts);
|
||||
let recipe = state ? readGitCacheRestoration(workspace)! : await inspectGitCacheRecipe(workspace,
|
||||
auth, options.signal, { proveManagedCheckout });
|
||||
if (!state) recipe.contents = await captureGitWorkspaceContents(recipe.worktreePath, identity, [...checkouts.values()]);
|
||||
const present = await exists(recipe.worktreePath);
|
||||
if (!state && !present) throw new Error("missing_unverified_worktree");
|
||||
result.candidate = true;
|
||||
@@ -194,7 +201,7 @@ export async function expireReconstructableGitWorkspace(db: Db, snapshot: Worksp
|
||||
}
|
||||
state = { schema: "paperclip.git-cache-expiration.v1", generation, workspaceRemoved: false,
|
||||
worktreeTrash: path.join(path.dirname(recipe.worktreePath), `.expired-git-worktree-${workspace.id}-${generation}`),
|
||||
worktreeAdmin, baseTrash: null, completed: false };
|
||||
worktreeAdmin, baseTrash: null, completed: false, lockToken: randomBytes(32).toString("hex") };
|
||||
if (await exists(state.worktreeTrash)) throw new Error("unknown_worktree_trash");
|
||||
// Commit the immutable restoration recipe BEFORE touching disk. The separate
|
||||
// lifecycle-lock connection remains held through deletion and final state.
|
||||
@@ -208,18 +215,43 @@ export async function expireReconstructableGitWorkspace(db: Db, snapshot: Worksp
|
||||
[workspace] = await db.update(executionWorkspaces).set({ metadata: { ...workspace!.metadata, [EXPIRATION_KEY]: state }, updatedAt: new Date() })
|
||||
.where(eq(executionWorkspaces.id, snapshot.id)).returning();
|
||||
};
|
||||
await options.beforeDelete?.(workspace);
|
||||
if (!state.lockToken) {
|
||||
state.lockToken = randomBytes(32).toString("hex");
|
||||
await persist();
|
||||
}
|
||||
await recoverGitCacheExpirationLocks(workspace!);
|
||||
const lockLocations = gitCacheExpirationLockLocations(workspace!);
|
||||
await options.beforeDelete?.(workspace!);
|
||||
if (!await isGitWorkspaceExpirationEligible(tx, workspace, options.now)) throw new Error("live_or_unresolved_consumer");
|
||||
if (!state.workspaceRemoved && present) {
|
||||
const { acquireGitWorktreeCleanupLock } = await import("./workspace-runtime.js");
|
||||
const lock = await acquireGitWorktreeCleanupLock(recipe.worktreePath);
|
||||
const locks = [await acquireGitExpirationLocks(lockLocations.filter(location => location.checkout === null), state.lockToken)];
|
||||
try {
|
||||
await assertReproducibleGitFiles(recipe.worktreePath, options.signal, { proveManagedCheckout });
|
||||
if ((await gitCacheCommand(recipe.worktreePath, ["rev-parse", "HEAD"], options.signal)).trim() !== recipe.commit) throw new Error("head_changed");
|
||||
try {
|
||||
checkouts.clear();
|
||||
await assertReproducibleGitFiles(recipe.worktreePath, options.signal, { proveManagedCheckout, preserveSettings: true,
|
||||
verifyFetchedCommit: (commit, signal) => verifyRemoteGitCommit(recipe.repoUrl, commit, auth, signal) });
|
||||
for (const checkout of checkouts.values()) {
|
||||
const cwd = path.join(recipe.worktreePath, ".paperclip-repositories", checkout.name);
|
||||
locks.push(await acquireGitExpirationLocks(lockLocations.filter(location => location.checkout === checkout.name), state.lockToken));
|
||||
if ((await gitCacheCommand(cwd, ["status", "--porcelain=v1", "--untracked-files=all", "-z"], options.signal)).length) throw new Error("managed_checkouts_changed");
|
||||
checkouts.set(checkout.name, await readManagedCheckoutContents(cwd, checkout.name, checkout.source, options.signal));
|
||||
}
|
||||
await assertGitWorkspaceContentsUnchanged(recipe.worktreePath, identity, recipe.contents, [...checkouts.values()]);
|
||||
if ((await gitCacheCommand(recipe.worktreePath, ["symbolic-ref", "--short", "HEAD"], options.signal)).trim() !== recipe.branchName) throw new Error("branch_identity_changed");
|
||||
if ((await gitCacheCommand(recipe.worktreePath, ["rev-parse", "HEAD"], options.signal)).trim() !== recipe.commit) throw new Error("head_changed");
|
||||
} catch (error) {
|
||||
for (const lock of [...locks].reverse()) await lock.release();
|
||||
const metadata: Record<string, unknown> = { ...workspace.metadata, lifecycleGeneration: state.generation + 1,
|
||||
[GIT_CACHE_RESTORATION_KEY]: { ...recipe, restoredAt: new Date().toISOString() } };
|
||||
delete metadata[EXPIRATION_KEY];
|
||||
await db.update(executionWorkspaces).set({ status: "active", closedAt: null, cleanupEligibleAt: null,
|
||||
cleanupReason: null, metadata, updatedAt: new Date() }).where(eq(executionWorkspaces.id, workspace.id));
|
||||
throw error;
|
||||
}
|
||||
await fs.rename(recipe.worktreePath, state.worktreeTrash);
|
||||
await finishGitCacheTrash(workspace);
|
||||
result.reclaimedBytes += result.candidateBytes;
|
||||
} finally { await lock.release(); }
|
||||
} finally { for (const lock of locks.reverse()) await lock.release(); }
|
||||
}
|
||||
await finishGitCacheTrash(workspace);
|
||||
state.workspaceRemoved = true;
|
||||
|
||||
@@ -1,4 +1,7 @@
|
||||
import { validateGitWorkspaceContents, type GitWorkspaceContents } from "./git-workspace-contents.js";
|
||||
import { recoverGitExpirationLocks, type GitExpirationLockLocation } from "./git-workspace-expiration-locks.js";
|
||||
import fs from "node:fs/promises";
|
||||
import { constants } from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { execFile as execFileCallback } from "node:child_process";
|
||||
@@ -31,15 +34,17 @@ export type GitCacheRestoration = {
|
||||
gitBranchOwnershipVersion: 1;
|
||||
verifiedAt: string;
|
||||
restoredAt?: string;
|
||||
contentsRestoredAt?: string | null;
|
||||
contents?: GitWorkspaceContents;
|
||||
};
|
||||
export type GitCacheExpiration = {
|
||||
schema: "paperclip.git-cache-expiration.v1"; generation: number; workspaceRemoved: boolean;
|
||||
worktreeTrash: string; worktreeAdmin: string; baseTrash: string | null; completed: boolean;
|
||||
worktreeTrash: string; worktreeAdmin: string; baseTrash: string | null; completed: boolean; lockToken?: string;
|
||||
};
|
||||
type Workspace = {
|
||||
id: string; companyId: string; projectId: string | null; sourceIssueId: string | null; repoUrl: string | null;
|
||||
mode: string | null; strategyType: string | null; branchName: string | null;
|
||||
cwd: string | null; providerRef: string | null; metadata: Record<string, unknown> | null;
|
||||
cwd: string | null; providerRef: string | null; status?: string; metadata: Record<string, unknown> | null;
|
||||
};
|
||||
|
||||
function managedSourcePaths(workspace: Workspace) {
|
||||
@@ -86,9 +91,11 @@ export function readGitCacheRestoration(workspace: Workspace): GitCacheRestorati
|
||||
|| (recipe.sourceKind === "isolated_remote" ? recipe.sourcePath !== recordedIsolatedRemoteSource(workspace)
|
||||
: recipe.sourceKind !== "managed_project" || recordedIsolatedRemoteSource(workspace) !== null || !managedSourcePaths(workspace).includes(recipe.sourcePath))
|
||||
|| !shaPattern.test(recipe.commit) || !Number.isFinite(Date.parse(recipe.verifiedAt))
|
||||
|| (recipe.restoredAt !== undefined && !Number.isFinite(Date.parse(recipe.restoredAt)))) {
|
||||
|| (recipe.restoredAt !== undefined && !Number.isFinite(Date.parse(recipe.restoredAt)))
|
||||
|| (recipe.contentsRestoredAt != null && !Number.isFinite(Date.parse(recipe.contentsRestoredAt)))) {
|
||||
throw new GitCacheRestorationFailure();
|
||||
}
|
||||
validateGitWorkspaceContents(recipe.contents);
|
||||
return recipe;
|
||||
}
|
||||
|
||||
@@ -187,7 +194,7 @@ export async function assertReproducibleManagedCheckout(checkout: string, input:
|
||||
const worktrees = (await gitCacheCommand(checkout, ["worktree", "list", "--porcelain"], signal))
|
||||
.split("\n").filter(line => line.startsWith("worktree "));
|
||||
if (worktrees.length !== 1) throw new Error("managed_checkout_has_worktrees");
|
||||
await assertReproducibleGitFiles(checkout, signal);
|
||||
await assertReproducibleGitFiles(checkout, signal, { preserveSettings: true });
|
||||
await assertReproducibleBaseGitStorage(checkout, signal);
|
||||
const refs = (await gitCacheCommand(checkout, ["for-each-ref", "--format=%(refname)"], signal)).trim().split("\n").filter(Boolean);
|
||||
if (refs.some(ref => !/^refs\/(heads|tags|remotes\/origin)\//.test(ref))) throw new Error("unsupported_managed_checkout_refs");
|
||||
@@ -230,8 +237,57 @@ async function hasCommitsMissingUpstream(checkout: string, localTips: string[],
|
||||
}
|
||||
}
|
||||
|
||||
async function assertReproducibleFetchHead(cwd: string, gitDir: string, signal?: AbortSignal,
|
||||
verifyFetchedCommit?: (commit: string, signal?: AbortSignal) => Promise<void>) {
|
||||
const target = path.join(gitDir, "FETCH_HEAD");
|
||||
const entry = await fs.lstat(target).catch((error: NodeJS.ErrnoException) => {
|
||||
if (error.code === "ENOENT") return null;
|
||||
throw error;
|
||||
});
|
||||
if (!entry) return;
|
||||
const byteLimit = 64 * 1024;
|
||||
if (!entry.isFile() || entry.isSymbolicLink() || entry.size > byteLimit) throw new Error("unsupported_worktree_fetch_head");
|
||||
const handle = await fs.open(target, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK);
|
||||
let contents: Buffer;
|
||||
try {
|
||||
const stat = await handle.stat();
|
||||
if (!stat.isFile() || stat.nlink !== 1 || stat.size > byteLimit) throw new Error("unsupported_worktree_fetch_head");
|
||||
const buffer = Buffer.alloc(byteLimit + 1);
|
||||
let length = 0;
|
||||
while (length < buffer.length) {
|
||||
signal?.throwIfAborted();
|
||||
const { bytesRead } = await handle.read(buffer, length, buffer.length - length, length);
|
||||
if (bytesRead === 0) break;
|
||||
length += bytesRead;
|
||||
}
|
||||
if (length > byteLimit) throw new Error("unsupported_worktree_fetch_head");
|
||||
contents = buffer.subarray(0, length);
|
||||
} finally { await handle.close(); }
|
||||
const text = contents.toString("utf8");
|
||||
if (!Buffer.from(text, "utf8").equals(contents) || text && !text.endsWith("\n")) throw new Error("unsupported_worktree_fetch_head");
|
||||
const commits = new Set<string>();
|
||||
for (const line of text ? text.slice(0, -1).split("\n") : []) {
|
||||
const match = /^([a-f0-9]{40})\t(?:not-for-merge)?\t[^\t\r\n\0]*$/.exec(line);
|
||||
if (!match) throw new Error("unsupported_worktree_fetch_head");
|
||||
commits.add(match[1]!);
|
||||
if (commits.size > 32) throw new Error("worktree_fetch_head_budget");
|
||||
}
|
||||
for (const commit of commits) {
|
||||
signal?.throwIfAborted();
|
||||
const objectType = await gitCacheCommand(cwd, ["cat-file", "-t", commit], signal).catch(() => "");
|
||||
const ancestor = objectType.trim() === "commit" && await gitCacheCommand(cwd, ["merge-base", "--is-ancestor", commit, "HEAD"], signal)
|
||||
.then(() => true, () => false);
|
||||
signal?.throwIfAborted();
|
||||
if (ancestor) continue;
|
||||
if (!verifyFetchedCommit) throw new Error("unverified_fetched_commit");
|
||||
await verifyFetchedCommit(commit, signal);
|
||||
}
|
||||
}
|
||||
|
||||
export async function assertReproducibleGitFiles(cwd: string, signal?: AbortSignal, options: {
|
||||
proveManagedCheckout?: ManagedCheckoutProver;
|
||||
preserveSettings?: boolean;
|
||||
verifyFetchedCommit?: (commit: string, signal?: AbortSignal) => Promise<void>;
|
||||
} = {}) {
|
||||
if ((await gitCacheCommand(cwd, ["status", "--porcelain=v1", "--untracked-files=all", "-z"], signal)).length) throw new Error("dirty_or_untracked_files");
|
||||
if ((await gitCacheCommand(cwd, ["ls-files", "-v"], signal)).split("\n").some(line => /^[a-zS] /.test(line))) throw new Error("unsupported_index_flags");
|
||||
@@ -239,7 +295,7 @@ export async function assertReproducibleGitFiles(cwd: string, signal?: AbortSign
|
||||
if (localConfig.split("\n").filter(Boolean).some(key => !/^(core\.(repositoryformatversion|filemode|bare|logallrefupdates|ignorecase|precomposeunicode)|remote\.origin\.(url|fetch)|branch\.[^.]+\.(remote|merge))$/.test(key))) throw new Error("unsupported_local_git_config");
|
||||
const regenerable = await findRegenerableWorkspaceDirs(cwd, { signal });
|
||||
const ignored = (await gitCacheCommand(cwd, ["ls-files", "--others", "--ignored", "--exclude-standard", "-z"], signal)).split("\0").filter(Boolean);
|
||||
const unexplained = ignored.filter(file => file !== PAPERCLIP_CLAUDE_SETTINGS_FILE && !regenerable.some(dir => file.startsWith(`${dir}/`)));
|
||||
const unexplained = ignored.filter(file => !(options.preserveSettings && file === PAPERCLIP_CLAUDE_SETTINGS_FILE) && !regenerable.some(dir => file.startsWith(`${dir}/`)));
|
||||
const checkouts = new Set<string>();
|
||||
for (const file of unexplained) {
|
||||
const [namespace, name, rest] = file.split("/", 3);
|
||||
@@ -266,8 +322,9 @@ export async function assertReproducibleGitFiles(cwd: string, signal?: AbortSign
|
||||
if (original) await gitCacheCommand(cwd, ["merge-base", "--is-ancestor", original.trim(), "HEAD"], signal)
|
||||
.catch(() => { throw new Error("unique_original_head"); });
|
||||
if (gitDir !== common) {
|
||||
const allowed = new Set(["HEAD", "ORIG_HEAD", "index", "commondir", "gitdir", "logs", "refs", "COMMIT_EDITMSG", "index.lock", "HEAD.lock"]);
|
||||
const allowed = new Set(["HEAD", "ORIG_HEAD", "FETCH_HEAD", "index", "commondir", "gitdir", "logs", "refs", "COMMIT_EDITMSG", "index.lock", "HEAD.lock"]);
|
||||
if ((await fs.readdir(gitDir)).some(name => !allowed.has(name))) throw new Error("unknown_worktree_git_files");
|
||||
await assertReproducibleFetchHead(cwd, gitDir, signal, options.verifyFetchedCommit);
|
||||
async function assertEmptyRefs(root: string): Promise<void> {
|
||||
const entries = await fs.readdir(root, { withFileTypes: true }).catch((error: NodeJS.ErrnoException) => {
|
||||
if (error.code === "ENOENT") return []; throw error;
|
||||
@@ -316,7 +373,8 @@ export async function inspectGitCacheRecipe(workspace: Workspace, resolveGitAuth
|
||||
if (!(await fs.lstat(path.join(workspace.cwd, ".git"))).isFile()) throw new Error("not_linked_worktree");
|
||||
if ((await gitCacheCommand(workspace.cwd, ["rev-parse", "--path-format=absolute", "--git-common-dir"], signal)).trim() !== path.join(await fs.realpath(source), ".git")) throw new Error("source_identity_changed");
|
||||
if ((await gitCacheCommand(workspace.cwd, ["symbolic-ref", "--short", "HEAD"], signal)).trim() !== workspace.branchName) throw new Error("branch_identity_changed");
|
||||
await assertReproducibleGitFiles(workspace.cwd, signal, options);
|
||||
await assertReproducibleGitFiles(workspace.cwd, signal, { ...options, preserveSettings: true,
|
||||
verifyFetchedCommit: (commit, verificationSignal) => verifyRemoteGitCommit(workspace.repoUrl!, commit, resolveGitAuth, verificationSignal) });
|
||||
const commit = (await gitCacheCommand(workspace.cwd, ["rev-parse", "HEAD"], signal)).trim();
|
||||
await verifyRemoteGitCommit(workspace.repoUrl, commit, resolveGitAuth, signal);
|
||||
return { schema: "paperclip.git-cache-restoration.v1", companyId: workspace.companyId, workspaceId: workspace.id,
|
||||
@@ -331,6 +389,7 @@ export function readGitCacheExpiration(workspace: Workspace): GitCacheExpiration
|
||||
const recipe = readGitCacheRestoration(workspace);
|
||||
if (!recipe || state.schema !== "paperclip.git-cache-expiration.v1" || !Number.isSafeInteger(state.generation)
|
||||
|| state.generation !== workspace.metadata?.lifecycleGeneration
|
||||
|| (state.lockToken !== undefined && !/^[a-f0-9]{64}$/.test(state.lockToken))
|
||||
|| state.worktreeTrash !== path.join(path.dirname(recipe.worktreePath), `.expired-git-worktree-${workspace.id}-${state.generation}`)
|
||||
|| path.dirname(state.worktreeAdmin) !== path.join(recipe.sourcePath, ".git", "worktrees")
|
||||
|| (state.baseTrash !== null && state.baseTrash !== path.join(path.dirname(recipe.sourcePath), `.expired-git-base-${workspace.id}-${state.generation}`))) {
|
||||
@@ -339,6 +398,38 @@ export function readGitCacheExpiration(workspace: Workspace): GitCacheExpiration
|
||||
return state;
|
||||
}
|
||||
|
||||
export function gitCacheExpirationLockLocations(workspace: Workspace): GitExpirationLockLocation[] {
|
||||
const state = readGitCacheExpiration(workspace);
|
||||
const recipe = readGitCacheRestoration(workspace);
|
||||
if (!state || !recipe || !state.lockToken || state.completed) return [];
|
||||
const branchPath = (branch: string) => {
|
||||
if (!branch || branch.split("/").some(part => !part || part === "." || part === "..") || branch.includes("\\")) throw new GitCacheRestorationFailure();
|
||||
return `refs/heads/${branch}.lock`;
|
||||
};
|
||||
const locations: GitExpirationLockLocation[] = [
|
||||
...["index.lock", "HEAD.lock"].map(name => ({ root: recipe.sourcePath,
|
||||
relativePath: path.relative(recipe.sourcePath, path.join(state.worktreeAdmin, name)), checkout: null })),
|
||||
{ root: recipe.sourcePath, relativePath: path.join(".git", branchPath(recipe.branchName)), checkout: null },
|
||||
];
|
||||
for (const checkout of recipe.contents?.checkouts ?? []) {
|
||||
for (const name of ["index.lock", "HEAD.lock", ...(checkout.branchName ? [branchPath(checkout.branchName)] : [])]) {
|
||||
locations.push({ root: recipe.worktreePath, relativePath: path.join(PROJECT_REPOSITORIES_DIR, checkout.name, ".git", name), checkout: checkout.name });
|
||||
}
|
||||
}
|
||||
return locations;
|
||||
}
|
||||
|
||||
export async function recoverGitCacheExpirationLocks(workspace: Workspace) {
|
||||
const state = readGitCacheExpiration(workspace);
|
||||
if (!state?.lockToken || state.completed || !["archived", "cleanup_failed"].includes(workspace.status ?? "")) return;
|
||||
const recipe = readGitCacheRestoration(workspace)!;
|
||||
if (await fs.lstat(recipe.sourcePath).then(() => true, (error: NodeJS.ErrnoException) => {
|
||||
if (error.code === "ENOENT") return false;
|
||||
throw error;
|
||||
})) await verifiedGitCacheSource(workspace, recipe.sourcePath);
|
||||
await recoverGitExpirationLocks(gitCacheExpirationLockLocations(workspace), state.lockToken);
|
||||
}
|
||||
|
||||
// Both restart recovery and reopening finish only recorded, uniquely named trash.
|
||||
// No directory scan or age heuristic can authorize deleting an arbitrary path.
|
||||
export async function finishGitCacheTrash(workspace: Workspace) {
|
||||
@@ -423,6 +514,9 @@ export function savedGitWorkspaceRestoration(input: { metadata?: Record<string,
|
||||
|| value.worktreePath !== input.cwd || value.sourcePath !== input.baseCwd
|
||||
|| value.repoUrl !== input.repoUrl || value.branchName !== input.branchName
|
||||
|| !isRuntimeOwnedGitBranch(input.metadata)
|
||||
|| (value.restoredAt !== undefined && !Number.isFinite(Date.parse(value.restoredAt)))) throw new GitCacheRestorationFailure();
|
||||
return { commit: value.commit, restored: value.restoredAt !== undefined };
|
||||
|| (value.restoredAt !== undefined && !Number.isFinite(Date.parse(value.restoredAt)))
|
||||
|| (value.contentsRestoredAt != null && !Number.isFinite(Date.parse(value.contentsRestoredAt)))) throw new GitCacheRestorationFailure();
|
||||
validateGitWorkspaceContents(value.contents);
|
||||
return { commit: value.commit, restored: value.restoredAt !== undefined, contents: value.contents,
|
||||
contentsRestored: value.contentsRestoredAt === undefined ? value.restoredAt !== undefined : value.contentsRestoredAt !== null };
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
import { nativeExecutionUsesTaskWorkspace } from "./external-provider.js";
|
||||
import { readGitCacheRestoration } from "../git-workspace-restoration.js";
|
||||
import { createHeartbeatPreparationTiming } from "./preparation-timing.js";
|
||||
import {
|
||||
resolveWorkspaceAfterLowTrustPreflight,
|
||||
@@ -572,6 +573,10 @@ async function prepareHeartbeatWorkspaceLocked(db: Db, input: HeartbeatWorkspace
|
||||
restoreExistingWorkspace: reusableExistingExecutionWorkspace
|
||||
? () =>
|
||||
ensurePersistedExecutionWorkspaceAvailable({
|
||||
prepareRestoredLocalSource: issueId ? sourceCwd => executionWorkspacesSvc.prepareConfiguredRepositorySource({
|
||||
companyId: agent.companyId, issueId, workspaceId: reusableExistingExecutionWorkspace.id,
|
||||
cwd: reusableExistingExecutionWorkspace.cwd!, agentId: agent.id, runId: run.id, sourceCwd,
|
||||
}) : undefined,
|
||||
skipHostProvision,
|
||||
db,
|
||||
base: executionWorkspaceBase,
|
||||
@@ -986,6 +991,7 @@ async function prepareHeartbeatWorkspaceLocked(db: Db, input: HeartbeatWorkspace
|
||||
cwd: executionWorkspace.cwd,
|
||||
anchorRepoUrl: executionWorkspace.repoUrl,
|
||||
workspaces: repositoryRows,
|
||||
restoredContents: persistedExecutionWorkspace ? readGitCacheRestoration(persistedExecutionWorkspace)?.contents : undefined,
|
||||
resolveGitAuth: workspaceGitAuthProvider,
|
||||
prepareLocalSource: persistedExecutionWorkspace && issueId
|
||||
? (sourceCwd) => executionWorkspacesSvc.prepareConfiguredRepositorySource({
|
||||
@@ -999,13 +1005,14 @@ async function prepareHeartbeatWorkspaceLocked(db: Db, input: HeartbeatWorkspace
|
||||
}) : undefined,
|
||||
}));
|
||||
const paths = new Map(repositories.map((repo) => [repo.workspaceId, repo.cwd]));
|
||||
const preparedRefs = new Map(repositories.map(repo => [repo.workspaceId, repo.repoRef]));
|
||||
projectRepositoryPaths.push(...repositories.map((repo) => path.relative(executionWorkspace.cwd, repo.cwd)));
|
||||
if (resolvedWorkspace.workspaceId) paths.set(resolvedWorkspace.workspaceId, executionWorkspace.cwd);
|
||||
resolvedWorkspace.workspaceHints = repositoryRows.map((source) => ({
|
||||
workspaceId: source.id,
|
||||
cwd: paths.get(source.id) ?? readNonEmptyString(source.cwd),
|
||||
repoUrl: readNonEmptyString(source.repoUrl),
|
||||
repoRef: readNonEmptyString(source.repoRef),
|
||||
repoRef: preparedRefs.has(source.id) ? preparedRefs.get(source.id)! : readNonEmptyString(source.repoRef),
|
||||
}));
|
||||
}
|
||||
if (persistedExecutionWorkspace) {
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
import { assertIsolatedWorkspaceSourceReadAccess, retainIsolatedWorkspaceSourceAccess } from "../execution-workspace-repositories.js";
|
||||
import { selectRestoredProjectRepositories, type GitWorkspaceContents } from "../git-workspace-contents.js";
|
||||
import { GitCacheRestorationFailure } from "../git-workspace-restoration.js";
|
||||
import { deriveRepoNameFromRepoUrl, managedProjectCheckoutName, projectRepositoryIdentity } from "../managed-project-checkout-name.js";
|
||||
import { forbidden } from "../../errors.js";
|
||||
import { readGitConnectionFailure } from "../git-connection-failure.js";
|
||||
@@ -498,14 +500,16 @@ export async function prepareProjectRepositoryWorkspaces(input: {
|
||||
resolveGitAuth?: GitRemoteAuthProvider | null;
|
||||
beforePublish?: () => Promise<void>;
|
||||
prepareLocalSource?: (sourceCwd: string) => Promise<() => Promise<void>>;
|
||||
restoredContents?: GitWorkspaceContents;
|
||||
}): Promise<Array<{ workspaceId: string; cwd: string; repoUrl: string; repoRef: string | null }>> {
|
||||
const identity = projectRepositoryIdentity;
|
||||
const seen = new Set(input.anchorRepoUrl ? [identity(input.anchorRepoUrl)] : []);
|
||||
const selected = input.workspaces.filter((workspace) => {
|
||||
const selected = selectRestoredProjectRepositories(input.workspaces, input.restoredContents).filter((workspace) => {
|
||||
if (!workspace.repoUrl || seen.has(identity(workspace.repoUrl))) return false;
|
||||
seen.add(identity(workspace.repoUrl));
|
||||
return true;
|
||||
});
|
||||
if (input.restoredContents?.checkouts.some(checkout => !selected.some(source => source.id === checkout.source.id))) throw new GitCacheRestorationFailure();
|
||||
const root = path.join(input.cwd, PROJECT_REPOSITORIES_DIR);
|
||||
if (selected.length === 0 && !(await fs.lstat(root).catch(() => null))) return [];
|
||||
await fs.mkdir(root, { recursive: true });
|
||||
@@ -519,6 +523,7 @@ export async function prepareProjectRepositoryWorkspaces(input: {
|
||||
const key = managedProjectCheckoutName({ id: workspace.id, repoUrl, repoRef: workspace.repoRef, cwd: workspace.cwd });
|
||||
const cwd = path.join(root, key);
|
||||
const existing = await fs.lstat(cwd).catch(() => null);
|
||||
if (!existing && input.restoredContents?.checkouts.some(checkout => checkout.name === key)) throw new GitCacheRestorationFailure();
|
||||
if (existing && (!existing.isDirectory() || existing.isSymbolicLink())) throw new Error("Invalid project repository checkout path");
|
||||
const configuredLocalSource = workspace.cwd && workspace.cwd !== REPO_ONLY_CWD_SENTINEL ? workspace.cwd : null;
|
||||
const localSource = configuredLocalSource
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { restoreGitWorkspaceContents } from "./git-workspace-contents.js";
|
||||
import { savedGitWorkspaceRestoration, ensureSavedGitCommit, GitCacheRestorationFailure } from "./git-workspace-restoration.js";
|
||||
import { sanitizeRuntimeServiceBaseEnv } from "./runtime-service-env.js";
|
||||
import { verifiedRecordedIsolatedRemoteSource } from "./isolated-remote-workspace-source.js";
|
||||
@@ -36,7 +37,7 @@ import {
|
||||
type WorkspaceRuntimeDesiredState,
|
||||
type WorkspaceRuntimeServiceStateMap,
|
||||
} from "@paperclipai/shared";
|
||||
import { and, desc, eq, gte, inArray, isNull, lte, ne, or } from "drizzle-orm";
|
||||
import { and, desc, eq, gte, inArray, isNull, lte, ne, or, sql } from "drizzle-orm";
|
||||
import { asNumber, asString, parseObject, renderTemplate } from "../adapters/utils.js";
|
||||
import { conflict, unprocessable } from "../errors.js";
|
||||
import { resolveHomeAwarePath } from "../home-paths.js";
|
||||
@@ -3769,6 +3770,8 @@ export async function realizeExecutionWorkspace(input: {
|
||||
}
|
||||
|
||||
export async function ensurePersistedExecutionWorkspaceAvailable(input: {
|
||||
prepareRestoredLocalSource?: (sourceCwd: string) => Promise<() => Promise<void>>;
|
||||
persistContentsRestoredAt?: (at: string | null) => Promise<void>;
|
||||
db?: Db | null;
|
||||
base: ExecutionWorkspaceInput;
|
||||
workspace: {
|
||||
@@ -3853,6 +3856,41 @@ export async function ensurePersistedExecutionWorkspaceAvailable(input: {
|
||||
workspaceId: input.workspace.id, cwd, baseCwd, repoUrl: input.workspace.repoUrl ?? input.base.repoUrl,
|
||||
branchName: input.workspace.branchName ?? null });
|
||||
const savedCommit = restoration?.commit ?? null;
|
||||
let currentRecipe = input.workspace.metadata?.gitCacheRestoration;
|
||||
const persistContentsRestoredAt = async (at: string | null) => {
|
||||
if (!restoration?.contents || !input.workspace.id) return;
|
||||
if (input.persistContentsRestoredAt) await input.persistContentsRestoredAt(at);
|
||||
else {
|
||||
if (!input.db) throw new GitCacheRestorationFailure();
|
||||
const updated = await input.db.update(executionWorkspaces).set({
|
||||
metadata: sql`jsonb_set(${executionWorkspaces.metadata}, '{gitCacheRestoration,contentsRestoredAt}', ${JSON.stringify(at)}::jsonb)`,
|
||||
}).where(and(eq(executionWorkspaces.id, input.workspace.id), eq(executionWorkspaces.companyId, input.agent.companyId),
|
||||
sql`${executionWorkspaces.metadata}->'gitCacheRestoration' = ${JSON.stringify(currentRecipe)}::jsonb`))
|
||||
.returning({ id: executionWorkspaces.id });
|
||||
if (updated.length !== 1) throw new GitCacheRestorationFailure();
|
||||
}
|
||||
currentRecipe = { ...currentRecipe as Record<string, unknown>, contentsRestoredAt: at };
|
||||
input.workspace.metadata!.gitCacheRestoration = currentRecipe;
|
||||
};
|
||||
const restoreContents = async () => {
|
||||
if (!restoration?.contents || !input.workspace.id) return;
|
||||
await restoreGitWorkspaceContents({ cwd, companyId: input.agent.companyId, workspaceId: input.workspace.id,
|
||||
contents: restoration.contents, resolveGitAuth: async url => {
|
||||
const auth = await input.resolveGitAuth?.(url);
|
||||
if (!auth) return null;
|
||||
const source = auth.source;
|
||||
if (source !== "company_secret" && source !== "managed_connection" && source !== "server_env") throw new GitCacheRestorationFailure();
|
||||
return { ...auth, source, secretName: auth.secretName ?? null };
|
||||
},
|
||||
authorizeSource: async source => {
|
||||
if (!input.db || !input.workspace.projectId) throw new GitCacheRestorationFailure();
|
||||
const [row] = await input.db.select().from(projectWorkspaces).where(and(
|
||||
eq(projectWorkspaces.id, source.id), eq(projectWorkspaces.companyId, input.agent.companyId),
|
||||
eq(projectWorkspaces.projectId, input.workspace.projectId)));
|
||||
if (!row || row.repoUrl !== source.repoUrl) throw new GitCacheRestorationFailure();
|
||||
}, prepareLocalSource: input.prepareRestoredLocalSource });
|
||||
await persistContentsRestoredAt(new Date().toISOString());
|
||||
};
|
||||
const repoRoot = await runGit(["rev-parse", "--show-toplevel"], baseCwd);
|
||||
await input.beforeSourceCopy?.(repoRoot, realized.worktreePath ?? cwd);
|
||||
const recordedBaseRefSha = readRecordedBaseRefSha(input.workspace.metadata);
|
||||
@@ -3934,6 +3972,7 @@ export async function ensurePersistedExecutionWorkspaceAvailable(input: {
|
||||
realized.warnings = [...repairWarnings, ...baseRefreshWarnings, ...baseDrift.warnings];
|
||||
realized.baseRefSha = refresh.baseRefSha ?? recordedBaseRefSha ?? baseDrift.branchBaseRefSha ?? baseDrift.currentBaseRefSha;
|
||||
await input.afterSourceCopy?.(repoRoot, realized.worktreePath ?? cwd);
|
||||
if (restoration && !restoration.contentsRestored) await restoreContents();
|
||||
await provisionExecutionWorktree({
|
||||
skipHostProvision: input.skipHostProvision,
|
||||
strategy: {
|
||||
@@ -3958,6 +3997,7 @@ export async function ensurePersistedExecutionWorkspaceAvailable(input: {
|
||||
throw new Error(`Execution workspace "${cwd}" is missing and cannot be restored because no branch name is recorded.`);
|
||||
}
|
||||
|
||||
await persistContentsRestoredAt(null);
|
||||
await fs.mkdir(path.dirname(worktreePath), { recursive: true });
|
||||
await runGit(["worktree", "prune"], repoRoot).catch(() => {});
|
||||
if (savedCommit) {
|
||||
@@ -4035,6 +4075,7 @@ export async function ensurePersistedExecutionWorkspaceAvailable(input: {
|
||||
});
|
||||
|
||||
await input.afterSourceCopy?.(repoRoot, worktreePath);
|
||||
await restoreContents();
|
||||
await provisionExecutionWorktree({
|
||||
skipHostProvision: input.skipHostProvision,
|
||||
strategy: {
|
||||
|
||||
Reference in new issue
Block a user