fix: preserve shared Git sources during online workspace migration (#19688)

## Thinking Path

> - Paperclip preserves the work produced by its agents.
> - Remote workspace migration must preserve legacy Git history and
local edits.
> - Linked checkouts can share administration and objects with other
tasks.
> - A task lock alone cannot exclude writers of a shared or borrowed
store.
> - The online source owner must derive and retain the full physical
ownership closure.
> - This change reuses task, project, lifecycle and finalizer locks for
that closure.
> - Idle legacy Git work can then migrate while unrelated remote turns
continue.

## Linked Issues or Issue Description

Refs: #19642.

**What happened?**

Online migration keeps legacy Git sources blocked because several task
checkouts can share a project base or a private Git cache. A read-only
source observation found 160 pending Git records, 120 distinct paths and
nine common stores. Its largest project cohort contained 126 records and
110 task owners. An ordinary-directory fence cannot preserve these
sources by itself.

**Expected behavior**

Derive the physical source owners from stable Git metadata. Hold their
existing source and lifecycle keys through publication. Retain all
history and recovery obligations. Reject foreign aliases, unknown stores
and changed sources. Let unrelated object-backed work continue.

**Steps to reproduce**

1. Create a managed project checkout and linked task worktrees.
2. Keep a settled prepared recovery source on a sibling task.
3. Keep unrelated remote work running.
4. Request an online import of an idle leaf checkout.
5. The existing owner keeps the shared source blocked.

**Paperclip version or commit**

This branch builds on merged #19642 (`7ac99718a4`) and fresh master
`fb3cf1a072`. It adds the Git closure. It does not add a writers-stopped
shortcut.

**Deployment mode**

Hosted native Runner with remote workspace storage. Local qualification
uses owned synthetic Git repositories, disposable PostgreSQL databases
and the actual archive and storage owners.

## What Changed

- Read bounded stable Git pointers, common-directory metadata and local
alternates. Derive the complete physical source ownership cohort.
Preserve all object stores.
- Retain common and borrowed store owners in preparation, lifecycle,
runtime, expiration and actual reopen paths. Keep company boundaries and
the existing 128-key bound.
- Hold exact sibling native recovery proofs under their existing
physical finalizer keys. Recheck descriptors, allocations, recovery
actions and source identities before capture and publication. Preserve
original states and warnings.
- Require leaf-first nested imports. Reuse the existing immutable child
download, restore and source comparison before parent exclusion.
- Bind the self-contained archive owner to the held Git namespaces. Keep
its three walks, history normalization, fsck, bytes and digest format.
Reject bare, detached, network, unknown or changed Git storage.
- Check both foreign path fields. Keep foreign metadata private.
Document the proof, bounds, source observation and remaining eligibility
limits.

## Verification

- Final revision `2934e41c923b93e16508b978c849b747c86be645`: all checks
are terminal, with 56 successful and four skipped, zero failures and
zero pending. The [required CI
aggregate](https://github.com/paperclipai/paperclip/actions/runs/42326464408/job/128266279210)
passed. Build, types, all server/Runner/serialized suites, end-to-end
aggregate and Snyk passed. [Fresh exact-head Greptile
review](https://github.com/paperclipai/paperclip/pull/19688#issuecomment-6699760104)
is 5/5 with no open threads. GitHub reports MERGEABLE and CLEAN.

- Review then found two correctness gaps: an ordinary managed directory
inside an unrelated host repository was blocked, and a Git parent could
miss a completed nested operation after its child snapshot became
missing or deleted. Three actual-owner regressions reproduced both gaps
in 9.70 seconds. The corrected owner passes 12 focused cases in 17.08
seconds. It imports and restores only the ordinary root bytes, preserves
the outer repository, rejects both missing/deleted child receipts before
a parent journal, and retains valid/changed child coverage and
private-cache contention. Its wrapper exited 0 and cleanup completed.
The first bootstrap attempt failed before tests ran because the host
exhausted its 32 shared-memory segment limit; no shared services were
stopped and no pass is claimed for that attempt. The final qualified
source/test type graph, module boundaries and diff checks pass.
- Receipt scans keep canonical path resolution. A lexical SQL prefix can
omit children reached through symlinked ancestors. The observed full
metadata closure completed in 1.17 seconds; no measured bottleneck
justifies weakening that alias proof. Writer admission still rejects
uncertain Git ownership before mutation. Expiration returns its typed
skip, and reopen returns its structured failure. Runtime and lifecycle
do not bypass an unknown source. No broad error-contract change was
added.

- Full CI at `720104ea975e` exposed five existing boundary cases across
three server shards. Four exact cases reproduced locally as failures in
25.98 seconds. Reopen now preserves its structured failure when source
inspection rejects the path, without reconstructing it. Runtime
recognizes only the exact regular ignore-only `.git/info/exclude`
fragment; migration still rejects that fragment. The original
arbitrary-path and file-URL import/delete-host/resume fixtures keep both
supported input forms and use the genuine offline setup path after their
sole writer finishes. The production mode gate is unchanged. A new
real-owner control proves the offline flag still rejects a running
legacy database writer.
- The final focused run passed all ten cases across four suites in 22.75
seconds, including all five CI failures, invalid-pointer admission,
fragment rejection, offline writer exclusion and actual nested reopen
lock ordering. Its wrapper exited 0 and cleanup completed. The known
invalid-pointer case remains fail-closed; the separate unexpected-I/O
diagnostic retains safe error metadata without command text or
credentials. Existing Git stderr redaction coverage remains intact. The
final metadata reader suite passed 20 cases in 2.68 seconds. The
qualified server/test source type graph, module boundaries and diff
checks pass. No production assertions or timeout defaults were weakened.
Fresh CI and review are required for the repair revision.

- A later review found that the ordinary early return still used only
one foreign path field. The actual ordinary providerRef-only regression
failed as incorrectly eligible. One source-path owner now enumerates
both local locators for company/foreign ordinary and Git scans, layout
checks, early admission, lifecycle/reopen/runtime and
expiration/move/preparation retention. Capture and display keep their
original preferred path. The actual owner canary passed 49 cases across
two suites in 109.15 seconds. It covers
cwd-only/ref-only/different/symlink locators, same-company mismatches,
foreign owners, lifecycle ordering, Git/recovery imports and
expiration/reopen. Three selected-locator fallback cases also failed
before their guard; the final eight-case control passed in 31.05
seconds, including the configured alias and valid import/private-child
controls. All wrappers exited 0 and completed cleanup. The
metadata/locator suite passed 19 cases. The final source/test type
graph, module boundaries and diff checks pass.

- Review found two P1 gaps. Both configured-directory symlink cases
failed before canonicalization. The actual parent import lost an
already-imported child's unique private cache and failed its archive
fence instead of retaining that owner. The corrected real-owner canary
passed four cases in 20.78 seconds: configured symlink runtime
provisioning, private-cache contention before the parent journal,
successful parent publication after release, and unchanged/changed child
coverage. The metadata suite passed 15 cases in 4.14 seconds. Both
wrappers exited 0 and cleanup completed.
- A fresh source read-only observation found 160 pending Git records and
nine common stores. An additional exact receipt query found 1,243
completed migration receipts and 557 nested source paths under pending
Git roots. All 126 records in the largest project were present in the
captured Git metadata set. For all 110 present candidate workspace
records, the derived closure fits the unchanged 128-key cap. The worst
case needs 110 issue keys and one project key, with 17 keys of headroom.
Its 173 imported children introduce no physical issue owner outside
those 110 logical task owners. This checks the observed key bound only.
It does not claim migration eligibility or a coherent snapshot.
- The initial stacked CI did not run four Runner test lanes: trusted
master passed new --package-lane/--native-server options to the old base
lane script. The branch is now rebased onto actual merged master and
retargeted before synchronization. It includes the canonical current
lane parser without an unrelated Runner patch. Fresh new-head CI is
required.

- The actual importer/source owner run passed 36 cases across three
suites in 187.75 seconds. It covers sibling prepared recovery, unrelated
running remote work, common-cache contention, changed
coordinator/allocation/path state during upload, exact nested coverage,
dirty unpushed work, and existing expiration/reopen owners. Wrapper exit
0 and cleanup completed.
- A separate previously hanging lifecycle-worker restore passed in 17.33
seconds with the unchanged 15-second test-body limit. An earlier run
timed out in three expiration/reopen cases and blocked cleanup; its
wrapper later exited 1. These failures remain recorded. They are not
counted as passes.
- Two boundary controls first failed: a foreign providerRef-only alias
was incorrectly eligible, and actual reopen did not admit its distinct
cache owner at the initial source boundary. The narrow guards now
inspect both foreign locators and derive physical Git keys before the
raw lifecycle transaction. The foreign alias and existing real nested
reopen contention passed. The private-cache control first retained
contradictory recorded source provenance and stayed fail-closed after
release. The corrected present legacy-worktree fixture passed in 12.61
seconds. These failed attempts remain recorded; they are not passes.
- The metadata reader and serialized/local archive controls use actual
Git repositories. They compare archive bytes and digests, remove every
original source, restore staged and unstaged work, and run fsck. The
metadata reader passed 13 cases in 2.85 seconds. The local/serialized
archive controls passed nine cases in 9.27 seconds. All processes exited
0. An additional actual importer control changed a relative commondir
pointer to the absolute name of the same store during upload.
Publication was rejected, the uploaded journal remained available, and
original files stayed intact (one case, 5.86 seconds; wrapper exit 0 and
cleanup complete).
- The final rebased full local server/test type graph passes with
db/shared/Runner source aliases because reused dependency declarations
predate new master exports. The first rebased attempt reported those
stale declaration fields/tables, and is not counted as a pass. Module
boundaries and diff checks pass. Standard package builds, types and
broad regression checks remain required in CI.
- No hosted import, admission hold, cancellation, provider mutation,
source filesystem write or shared-cache Git command was performed for
this change. The source observation is read-only and is not a locked
eligibility proof.

## Risks

- A missing writer owner could make physical source exclusion
incomplete. Existing mutation owners retain the same
company/task/project keys, and uncertain metadata fails closed.
- A project can contain a large cohort. Limits remain finite. A bound
failure keeps the source blocked.
- Bare, detached, unowned or network Git stores remain blocked online.
Missing archived sources retain their existing authenticated
reconstruction path.
- Exact recovery proofs preserve authority and obligations. They do not
infer provider termination or mark a prepared descriptor finalized.
- This change does not claim all observed Git records are immediately
eligible. Existing preparation, provider, recovery, runtime and
source-change blockers remain.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, tool use and code execution.
The host does not expose a more specific backend model ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-11-06 20:08:19 -06:00
1 parent a894b11a2b
commit d5c4f7dd02
20 files changed
+1151 -134

No files matched your search

@@ -0,0 +1,90 @@
# Online Git source preservation
The online importer must exclude every writer of a selected checkout, its Git
administration, and all common or borrowed object stores. A checkout's task key
alone does not cover a shared project base or a private cache owned by another
task. Unrelated remote work must continue.
The source owner reads bounded, stable Git pointer, `commondir`, and alternates
metadata. It maps each physical directory to the existing company/task source
key or managed-project source key. It derives a transitive company cohort from
those owners and declared project sources. It rejects foreign physical aliases,
unknown stores, bare repositories, symlink metadata, network alternates, and
uncertain or excessive metadata. It preserves every object store; HEAD alone is
not a source closure. The bounds are 1,000 company records, 256 cohort records,
128 source keys, 32 borrowed stores, and four alternate levels. Foreign minimal
paths use the existing bounded scan of 256 records per page, at most 40 pages.
Configured directory aliases resolve through the existing canonical path owner
before Git inspection. Git pointer files and administration/object-store links
retain their strict checks. A configured symlink does not make a direct symlink
migration source admissible.
The runtime retains the existing plain-task-directory treatment of an exact
regular `.git/info/exclude` fragment. It is not an operational Git repository.
Git pointers and object stores retain their topology checks, and migration
rejects the fragment instead of treating it as an ordinary snapshot. A blocked
reopen returns its existing structured failure without attempting reconstruction.
Arbitrary self-hosted Git paths and file URLs retain the genuine offline
maintenance path; online migration requires a proven physical source owner.
Physical exclusion enumerates both `cwd` and `providerRef` through one source
path owner. Company/foreign alias checks, early admission, layout checks and
filesystem mutation owners use that set. A managed or Git source in either
locator requires a proven scope before the legacy fallback. Capture and display
keep their existing preferred-path selection. Missing archived sources retain
their existing reconstruction path.
Preparation, lifecycle, runtime-service, expiration, and reopen owners retain
the same physical keys before filesystem mutation or lifecycle exclusion. The
importer holds all cohort task/project and lifecycle keys through publication.
It also holds the existing per-run physical finalizer keys for each exact native
recovery proof. It re-inspects the complete source proof before capture and
publication. Changed descriptors, allocations, owners, aliases, or Git metadata
reject publication. Prepared sources and unresolved recovery warnings retain
their original state and evidence. They are not marked finalized or cleared.
Present nested workspaces must import before their parent. The existing owning
nested-import path downloads and restores each immutable child snapshot and
compares it with a fresh source capture before excluding that child. Missing or
changed coverage remains blocked. The archive retains its three source walks,
history normalization, Git/fsck checks, manifest ordering, and byte format. Its
self-contained remote function checks that every Git store belongs to the held
physical source roots. It preserves the original source files.
Ordinary capture does not read an enclosing host repository outside its root.
Its canonical ordinary proof remains valid there. Any `.git` marker at the
selected root still requires Git admission. A completed nested operation stays
in the parent layout guard unless the derived child proof covers that exact
operation and physical path. Missing or revoked snapshot coverage cannot turn
the leftover child into an ordinary parent directory.
Receipt path scans retain canonical resolution for aliases. A lexical SQL prefix
could hide a child reached through a symlinked ancestor. The finite receipt bound
remains in force; query optimization must preserve that alias coverage.
Imported child paths come from the same exact completed migration/snapshot
selector used by nested source verification. Their physical Git owners remain
in the parent fence after their database rows become remote. The selector bounds
company migration receipts to 10,240 and binds each child path, operation,
snapshot, digest and Git metadata proof. Current remote child execution does not
reacquire authority over those historical host files.
A source metadata observation at `2026-11-06T23:15:08.979Z` examined 160 pending
Git records and 120 unique paths. It found 92 Git pointers, 49 Git directories,
19 absent archived roots, and nine common/object stores. No alternate or uncertain
metadata was observed. Project cohort sizes were 126, 27, 4, 1, 1, and 1; the
largest contained 110 distinct task owners. These observations bound useful
cohorts. They are not a coherent snapshot, acquired lock proof, or claim that all
160 are eligible. Missing archived sources keep their existing authenticated
reconstruction path. Remaining provider, recovery, preparation, runtime, and
source-change blockers remain authoritative.
Focused verification uses the real importer and local storage owner. It covers
prepared sibling recovery, unrelated running remote work, private cache ownership,
source changes during upload, leaf-first exact child coverage, and existing Git
expiration/reopen consumers. Archive tests exercise both local calls and the
serialized remote function, compare archive bytes and digests, remove every
original source, restore uncommitted/staged work, and run fsck. No hosted import,
hold, cancellation, provider call, or shared-cache Git command is part of these
local tests.
@@ -20,7 +20,7 @@ import { materializeIsolatedRemoteWorkspaceSource } from "../services/isolated-r
import { realizeExecutionWorkspace, ensurePersistedExecutionWorkspaceAvailable } from "../services/workspace-runtime.js";
import { executionWorkspaceService } from "../services/execution-workspaces.js";
import { withExecutionWorkspaceLifecycleLock } from "../services/execution-workspace-lifecycle-lock.js";
import { withWorkspaceSourceLocks } from "../services/workspace-source-lock.js";
import { IsolatedRepositorySourceBusy, withWorkspaceSourceLocks } from "../services/workspace-source-lock.js";
import { prepareProjectRepositoryWorkspaces, mergeExecutionWorkspaceMetadataForPersistence } from "../services/heartbeat/workspaces.js";
import { describeGitExpirationFailure, expireReconstructableGitWorkspace } from "../services/git-workspace-expiration.js";
import { nestedWorktreeBranchPrefix, restoreNestedRepositoryWorktree, withNestedRepositoryBaseLock } from "../services/nested-repository-worktrees.js";
@@ -220,6 +220,30 @@ describeEmbeddedPostgres("reconstructable Git expiration", () => {
expect((await fs.stat(f.nestedSettingsPath)).mode & 0o777).toBe(0o640);
});
it("retains a distinct private Git cache before the actual reopen lifecycle", async () => {
const f = await fixture(), sourceIssueId = randomUUID();
await db.insert(issues).values({ id: sourceIssueId, companyId: f.companyId, title: "Reopen consumer" });
const metadata = { ...(await f.row()).metadata };
delete metadata.isolatedRemoteSource;
await db.update(executionWorkspaces).set({ sourceIssueId, metadata }).where(eq(executionWorkspaces.id, f.workspaceId));
let entered!: () => void, release!: () => void;
const started = new Promise<void>(resolve => { entered = resolve; });
const pending = new Promise<void>(resolve => { release = resolve; });
const owner = withWorkspaceSourceLocks(db, { companyId: f.companyId, issueIds: [f.issueId] }, async () => {
entered(); await pending;
});
await started;
try {
await expect(f.reopen()).rejects.toBeInstanceOf(IsolatedRepositorySourceBusy);
const key = `execution_workspace_lifecycle:${f.workspaceId}`;
const locks = await db.execute(sql`select count(*)::int n from pg_locks where locktype='advisory' and granted
and classid=((hashtextextended(${key},0)>>32)&4294967295)::oid and objid=(hashtextextended(${key},0)&4294967295)::oid`);
expect(locks[0].n).toBe(0);
} finally { release(); await owner; }
expect(await f.reopen()).toMatchObject({ ok: true, reopened: false });
expect(git(f.cwd, "rev-parse", "HEAD")).toBe(f.commitA);
});
it("takes the project fence before the lifecycle lock during an actual nested reopen", async () => {
const f = await nestedFixture(true, true);
expect(await f.expire({ expireBase: false })).toMatchObject({ expired: true, baseExpired: false });
@@ -897,10 +921,22 @@ describeEmbeddedPostgres("reconstructable Git expiration", () => {
it("describes an unexpected inspection failure without the command or credentials", async () => {
const f = await fixture();
await fs.writeFile(path.join(f.cwd, ".git"), `gitdir: ${path.join(f.cwd, "missing-admin")}\n`);
const failure = Object.assign(new Error("inspection command token=sk-secret-fixture"), { code: "EACCES" });
vi.spyOn(fs, "open").mockRejectedValueOnce(failure);
const result = await f.expire();
expect(result).toMatchObject({ expired: false, skipReason: "inspection_failed",
failure: { name: "Error", code: expect.any(String), at: expect.stringMatching(/^at /), stderr: expect.stringMatching(/not a git repository|gitfile does not point to a valid repository/) } });
failure: { name: "Error", code: "EACCES", at: expect.stringMatching(/^at /), stderr: null } });
expect(JSON.stringify(result)).not.toContain("sk-secret-fixture");
expect(JSON.stringify(result)).not.toContain("inspection command");
expect(await exists(f.cwd)).toBe(true);
});
it("retains an invalid Git pointer rejected by source admission without exposing its command", async () => {
const f = await fixture();
await fs.writeFile(path.join(f.cwd, ".git"), `gitdir: ${path.join(f.cwd, "missing-admin")}\n`);
const result = await f.expire();
expect(result).toMatchObject({ expired: false, skipReason: "workspace_git_source_unproven" });
expect(result.failure).toBeUndefined();
expect(await exists(f.cwd)).toBe(true);
});
@@ -1,6 +1,8 @@
import { executionWorkspaces, withDedicatedDbConnection, type Db } from "@paperclipai/db";
import { eq, sql } from "drizzle-orm";
import { isolatedTaskSourceIdentity, managedProjectSourceIdentity, withWorkspaceSourceLocks } from "./workspace-source-lock.js";
import { isDeepStrictEqual } from "node:util";
import { workspaceSourceIdentities, withWorkspaceSourceLocks } from "./workspace-source-lock.js";
import { localWorkspaceSourcePaths } from "./execution-workspace-source-roots.js";
import { withWorkspaceLockKeepalive } from "./native-runtime/native-workspace-lock-keepalive.js";
type Transaction = Parameters<Parameters<Db["transaction"]>[0]>[0];
@@ -20,23 +22,25 @@ export async function withExecutionWorkspaceLifecycleLock<T>(
work: (tx: Transaction) => Promise<T>,
): Promise<T> {
const [source] = await db.select({ companyId: executionWorkspaces.companyId, projectId: executionWorkspaces.projectId,
providerType: executionWorkspaces.providerType, cwd: executionWorkspaces.cwd, providerRef: executionWorkspaces.providerRef }).from(executionWorkspaces).where(eq(executionWorkspaces.id, workspaceId));
const projectId = source && (source.cwd ?? source.providerRef)
? await managedProjectSourceIdentity(source.companyId, (source.cwd ?? source.providerRef)!) : null;
const issueId = source && (source.cwd ?? source.providerRef)
? await isolatedTaskSourceIdentity(source.companyId, (source.cwd ?? source.providerRef)!) : null;
const projectIds = [...new Set([projectId, source && ["local_fs", "git_worktree"].includes(source.providerType)
? source.projectId : null].filter((id): id is string => Boolean(id)))];
providerType: executionWorkspaces.providerType, sourceIssueId: executionWorkspaces.sourceIssueId, cwd: executionWorkspaces.cwd, providerRef: executionWorkspaces.providerRef }).from(executionWorkspaces).where(eq(executionWorkspaces.id, workspaceId));
const sourceRoots = source ? await Promise.all(localWorkspaceSourcePaths(source)
.map(cwd => workspaceSourceIdentities(source.companyId, cwd))) : [];
const projectIds = [...new Set([...sourceRoots.flatMap(root => root.projectIds),
source && ["local_fs", "git_worktree"].includes(source.providerType) ? source.projectId : null,
].filter((id): id is string => Boolean(id)))].sort();
const issueIds = [...new Set([...sourceRoots.flatMap(root => root.issueIds),
source && ["local_fs", "git_worktree"].includes(source.providerType) ? source.sourceIssueId : null,
].filter((id): id is string => Boolean(id)))].sort();
const workWithLifecycle = () => withDedicatedDbConnection(db, (connection) =>
connection.transaction(async (tx) => {
await acquireExecutionWorkspaceLifecycleLock(tx, workspaceId);
if ((projectIds.length || issueId) && source) {
if ((projectIds.length || issueIds.length) && source) {
const [fresh] = await tx.select({ companyId: executionWorkspaces.companyId, projectId: executionWorkspaces.projectId,
providerType: executionWorkspaces.providerType, cwd: executionWorkspaces.cwd, providerRef: executionWorkspaces.providerRef }).from(executionWorkspaces).where(eq(executionWorkspaces.id, workspaceId));
providerType: executionWorkspaces.providerType, sourceIssueId: executionWorkspaces.sourceIssueId, cwd: executionWorkspaces.cwd, providerRef: executionWorkspaces.providerRef }).from(executionWorkspaces).where(eq(executionWorkspaces.id, workspaceId));
const freshRoots = fresh ? await Promise.all(localWorkspaceSourcePaths(fresh)
.map(cwd => workspaceSourceIdentities(fresh.companyId, cwd))) : [];
if (!fresh || fresh.companyId !== source.companyId || fresh.projectId !== source.projectId
|| fresh.providerType !== source.providerType || !(fresh.cwd ?? fresh.providerRef)
|| await managedProjectSourceIdentity(fresh.companyId, (fresh.cwd ?? fresh.providerRef)!) !== projectId
|| await isolatedTaskSourceIdentity(fresh.companyId, (fresh.cwd ?? fresh.providerRef)!) !== issueId) {
|| fresh.providerType !== source.providerType || fresh.sourceIssueId !== source.sourceIssueId || !isDeepStrictEqual(freshRoots, sourceRoots)) {
throw new Error("workspace_source_scope_changed");
}
}
@@ -50,8 +54,8 @@ export async function withExecutionWorkspaceLifecycleLock<T>(
);
}),
);
return (projectIds.length || issueId) && source
? withWorkspaceSourceLocks(db, { companyId: source.companyId, projectIds, issueIds: issueId ? [issueId] : [] }, workWithLifecycle)
return (projectIds.length || issueIds.length) && source
? withWorkspaceSourceLocks(db, { companyId: source.companyId, projectIds, issueIds }, workWithLifecycle)
: workWithLifecycle();
}
@@ -1,6 +1,10 @@
import fs from "node:fs/promises";
import path from "node:path";
export function localWorkspaceSourcePaths(workspace: { cwd?: string | null; providerRef?: string | null }): string[] {
return [...new Set([workspace.cwd, workspace.providerRef].filter((value): value is string => typeof value === "string" && value.length > 0))];
}
export async function canonicalRepositoryRoot(cwd: string): Promise<string> {
const absolute = path.resolve(cwd);
try { return await fs.realpath(absolute); }
+13 -7
View File
@@ -8,7 +8,8 @@ import { measureRunTransaction } from "./execution-transaction-timing.js";
import { measureRunPhase } from "./execution-preparation-timing.js";
import { assertExecutionWorkspaceRecordAccess, assertTaskWorkspaceExecutionAccess, assertTaskWorkspaceSourceProjectAccess } from "./task-workspace-access.js";
import { verifiedRecordedIsolatedRemoteSource, withIsolatedRepositorySourceLock } from "./isolated-remote-workspace-source.js";
import { managedProjectSourceIdentity, retainCurrentWorkspaceSource, withWorkspaceSourceLocks } from "./workspace-source-lock.js";
import { workspaceSourceIdentities, retainCurrentWorkspaceSource, withWorkspaceSourceLocks } from "./workspace-source-lock.js";
import { localWorkspaceSourcePaths } from "./execution-workspace-source-roots.js";
import { expireReconstructableGitWorkspace, gitWorkspaceLastActivityAt, hasPendingGitExpiration, listIssuesUsingGitWorkspace } from "./git-workspace-expiration.js";
import { expireIsolatedTaskDirectory, hasPendingTaskDirectoryRetention, isIsolatedTaskDirectoryWorkspace, isRetryableTaskDirectorySkip,
mayBeIsolatedTaskDirectoryWorkspace, readTaskDirectoryRetention, restoreIsolatedTaskDirectory, supersedeTaskDirectoryRestorePoint,
@@ -4774,14 +4775,19 @@ export function executionWorkspaceService(db: Db, opts: ExecutionWorkspaceServic
return result;
};
if (freshSnapshot.providerType === "plugin") return reopenSource();
const projectIds = new Set<string>();
const projectIds = new Set<string>(), issueIds = new Set<string>();
if (freshSnapshot.projectId) projectIds.add(freshSnapshot.projectId);
for (const cwd of new Set([freshSnapshot.cwd, freshSnapshot.providerRef])) {
if (!cwd) continue;
const projectId = await managedProjectSourceIdentity(freshSnapshot.companyId, cwd);
if (projectId) projectIds.add(projectId);
try {
for (const cwd of localWorkspaceSourcePaths(freshSnapshot)) {
const identities = await workspaceSourceIdentities(freshSnapshot.companyId, cwd);
for (const id of identities.projectIds) projectIds.add(id);
for (const id of identities.issueIds) issueIds.add(id);
}
} catch {
return { ok: false, code: "rebuild_failed", message: freshSnapshot.metadata?.gitCacheRestoration
? new GitCacheRestorationFailure().message : "Failed to rebuild the execution workspace" };
}
return withWorkspaceSourceLocks(db, { companyId: freshSnapshot.companyId, projectIds: [...projectIds] }, reopenSource);
return withWorkspaceSourceLocks(db, { companyId: freshSnapshot.companyId, projectIds: [...projectIds], issueIds: [...issueIds] }, reopenSource);
};
return authorizedSnapshot.sourceIssueId
? withIsolatedRepositorySourceLock(db, authorizedSnapshot.companyId, authorizedSnapshot.sourceIssueId, reopen)
@@ -12,6 +12,8 @@ import { TASK_DIRECTORY_RETENTION_KEY } from "./isolated-task-directory-snapshot
import { currentRestorePointSnapshot, RestorePointSkip, type RestorePointLimits } from "./git-restore-point.js";
import { abandonGitRestorePoint, abandonGitRestorePointsByKey, activateGitRestorePoints, restorePointHistoryUnchanged, restorePointSnapshotMatches, storeGitRestorePoint } from "./git-restore-point-storage.js";
import { withIsolatedRepositorySourceLock, verifiedRecordedIsolatedRemoteSource } from "./isolated-remote-workspace-source.js";
import { retainCurrentWorkspaceSource } from "./workspace-source-lock.js";
import { localWorkspaceSourcePaths } from "./execution-workspace-source-roots.js";
import { managedProjectCheckoutName } from "./managed-project-checkout-name.js";
import { nestedRepositoryBaseCandidates } from "./nested-repository-worktrees.js";
import type { GitRemoteAuthProvider } from "./git-credentials.js";
@@ -292,8 +294,11 @@ export async function expireReconstructableGitWorkspace(db: Db, snapshot: Worksp
let pendingRestorePoints: string[] = [];
if (!snapshot.sourceIssueId) return { ...result, skipReason: "missing_source_issue" };
try {
return await withIsolatedRepositorySourceLock(db, snapshot.companyId, snapshot.sourceIssueId, () =>
withDedicatedDbConnection(db, connection => connection.transaction(async tx => {
return await withIsolatedRepositorySourceLock(db, snapshot.companyId, snapshot.sourceIssueId, async () => {
for (const cwd of localWorkspaceSourcePaths(snapshot)) await retainCurrentWorkspaceSource(cwd);
const retainedSource = readGitCacheRestoration(snapshot)?.sourcePath;
if (retainedSource) await retainCurrentWorkspaceSource(retainedSource);
return withDedicatedDbConnection(db, connection => connection.transaction(async tx => {
const acquired = await tx.execute<{ locked: boolean }>(sql`select pg_try_advisory_xact_lock(hashtextextended(${`execution_workspace_lifecycle:${snapshot.id}`}, 0)) as locked`);
if (!acquired[0]?.locked) throw new Error("lifecycle_busy");
let workspace = (await db.select().from(executionWorkspaces).where(and(eq(executionWorkspaces.id, snapshot.id), eq(executionWorkspaces.companyId, snapshot.companyId))))[0];
@@ -487,7 +492,8 @@ export async function expireReconstructableGitWorkspace(db: Db, snapshot: Worksp
state.completed = recipe.sourceKind !== "isolated_remote" || !options.expireBase || (result.baseExpired || !await exists(recipe.sourcePath)) && (!state.baseTrash || !await exists(state.baseTrash));
await persist();
return result;
})), true);
}));
}, true);
} catch (error) {
for (const ledgerId of pendingRestorePoints) await abandonGitRestorePoint(db, ledgerId).catch(() => undefined);
options.signal?.throwIfAborted();
@@ -662,8 +662,15 @@ describe.skipIf(!support.supported && !externalDatabaseUrl)("heartbeat workspace
providerType: "local_fs", strategyType: "project_primary", mode: "shared_workspace", status: "active" }).returning();
await db.update(issues).set({ executionWorkspaceId: original.id }).where(eq(issues.id, issue.id));
const storage = createStorageService(createLocalDiskStorageProvider(join(home, `import-objects-${randomUUID()}`)));
await expect(migrateRemoteWorkspace({ db, storage, companyId: issue.companyId, workspaceId: original.id, apply: true }))
.resolves.toMatchObject({ applied: true });
const migrationBefore = process.env.PAPERCLIP_EPHEMERAL_MIGRATION;
process.env.PAPERCLIP_EPHEMERAL_MIGRATION = "true";
try {
await expect(migrateRemoteWorkspace({ db, storage, companyId: issue.companyId, workspaceId: original.id,
apply: true, writersStopped: true })).resolves.toMatchObject({ applied: true });
} finally {
if (migrationBefore === undefined) delete process.env.PAPERCLIP_EPHEMERAL_MIGRATION;
else process.env.PAPERCLIP_EPHEMERAL_MIGRATION = migrationBefore;
}
const [imported] = await db.select().from(executionWorkspaces).where(eq(executionWorkspaces.id, original.id));
const saved = readRemoteWorkspaceAuthority(imported.metadata)!;
expect(saved).toMatchObject({ source: { kind: "git", repoUrl, ref: "main", branchName: "main" },
+2 -1
View File
@@ -1,4 +1,5 @@
import { retainCurrentWorkspaceSource } from "../workspace-source-lock.js";
import { localWorkspaceSourcePaths } from "../execution-workspace-source-roots.js";
import { effectiveProjectWorkspacePolicy } from "../project-workspace-provider.js";
import { canonicalRepositoryRoot } from "../execution-workspace-source-roots.js";
import { assertTaskWorkspaceExecutionAccess } from "../task-workspace-access.js";
@@ -3401,7 +3402,7 @@ export function createHeartbeatWorkspaceResolver(db: Db) {
managedBase && workspace.repoUrl
? `${managedBase}-${createHash("sha256").update(workspace.repoUrl).digest("hex").slice(0, 12)}` : null,
].filter((value): value is string => Boolean(value));
await retainCurrentWorkspaceSource(workspace.cwd ?? workspace.providerRef ?? "./");
for (const sourceCwd of localWorkspaceSourcePaths(workspace)) await retainCurrentWorkspaceSource(sourceCwd);
const cwd = await resolvePersistedGitWorkspaceSource({
companyId: agent.companyId,
projectId: sourceProjectId,
@@ -19,7 +19,7 @@ import { applyRestorePoint, captureRestorePoint, currentRestorePointSnapshot, im
import { findRegenerableWorkspaceDirs, REGENERABLE_WORKSPACE_DIR_NAMES, WORKSPACE_COMPACTION_TRASH_DIR } from "./workspace-compaction.js";
import { ensureManagedRepositoriesIgnored } from "./managed-repository-checkout.js";
import { withWorkspaceLockKeepalive } from "./native-runtime/native-workspace-lock-keepalive.js";
import { canonicalRepositoryRoot } from "./execution-workspace-source-roots.js";
import { canonicalRepositoryRoot, localWorkspaceSourcePaths } from "./execution-workspace-source-roots.js";
import { withExecutionWorkspaceLifecycleLock } from "./execution-workspace-lifecycle-lock.js";
import { loadGitRestorePoint } from "./git-restore-point-storage.js";
import { nestedWorktreeBranchPrefix, readLinkedNestedCheckout, repairMovedNestedRepositoryWorktree, withNestedRepositoryBaseLock } from "./nested-repository-worktrees.js";
@@ -355,8 +355,7 @@ async function undoMove(db: Db | Tx, workspaceId: string, pending: IsolatedRepos
export function withWorkspaceMoveLocks<T>(db: Db, workspace: Workspace, work: (tx: Tx) => Promise<T>) {
return withIsolatedRepositorySourceLock(db, workspace.companyId, workspace.sourceIssueId!, () =>
withWorkspaceSourceLocks(db, { companyId: workspace.companyId, projectIds: workspace.projectId ? [workspace.projectId] : [] }, async () => {
const cwd = workspace.cwd ?? workspace.providerRef;
if (cwd) await retainCurrentWorkspaceSource(cwd);
for (const cwd of localWorkspaceSourcePaths(workspace)) await retainCurrentWorkspaceSource(cwd);
return withDedicatedDbConnection(db, connection => connection.transaction(async tx => {
const rows = await tx.execute<{ locked: boolean; idle_timeout_ms: string | number | null }>(sql`select
pg_try_advisory_xact_lock(hashtextextended(${`execution_workspace_lifecycle:${workspace.id}`}, 0)) as locked,
@@ -0,0 +1,181 @@
import fs from "node:fs/promises";
import path from "node:path";
import { and, eq, gt, inArray, ne, sql } from "drizzle-orm";
import { executionWorkspaces, heartbeatRuns, issueRecoveryActions, type Db } from "@paperclipai/db";
import { resolvePaperclipInstanceRoot } from "../home-paths.js";
import { inspectImportedWorkspaceSourcePaths, inspectOnlineOrdinarySource, physicalPath, requiresOnlineOrdinarySource, type WorkspaceSourceScope } from "./remote-workspace-migration-tree.js";
import { workspaceSourceIdentities } from "./workspace-source-lock.js";
import { readWorkspaceGitSourceRoots } from "./workspace-git-source-roots.js";
import { inspectOnlineLegacyWorkspaceRecoveryHold, inspectOnlineLegacyWorkspaceRecoverySource, type OnlineLegacyWorkspaceRecoveryHold } from "./online-legacy-workspace-recovery.js";
import type { LegacyWorkspaceRecoverySource } from "./legacy-workspace-recovery-migration.js";
import { hasPendingGitExpiration } from "./git-workspace-expiration.js";
import { readGitCacheRestoration } from "./git-workspace-restoration.js";
import { localWorkspaceSourcePaths } from "./execution-workspace-source-roots.js";
type Workspace = typeof executionWorkspaces.$inferSelect;
type ReadDb = Pick<Db, "select">;
type Identities = Awaited<ReturnType<typeof workspaceSourceIdentities>>;
export type OnlineWorkspaceSource = WorkspaceSourceScope & {
kind: "ordinary" | "git"; projectIds: string[];
gitSourceFingerprints?: Array<{ workspaceId: string; fingerprint: string | null }>;
importedChildren?: Array<Awaited<ReturnType<typeof inspectImportedWorkspaceSourcePaths>>[number] & { gitFingerprint: string | null }>;
companions: Array<{ workspaceId: string; sources: LegacyWorkspaceRecoverySource[];
holds: Array<OnlineLegacyWorkspaceRecoveryHold & { actionId: string }> }>;
};
const intersects = (values: string[], keys: Set<string>) => values.some(value => keys.has(value));
export async function requiresOnlineWorkspaceSource(workspace: Workspace): Promise<boolean> {
if (await requiresOnlineOrdinarySource(workspace)) return true;
if (!["local_fs", "git_worktree"].includes(workspace.providerType)) return false;
for (const cwd of localWorkspaceSourcePaths(workspace)) {
if (!path.isAbsolute(cwd)) continue;
const root = await physicalPath(cwd);
const entry = await fs.lstat(root).catch((error: NodeJS.ErrnoException) => { if (error.code !== "ENOENT") throw error; return null; });
if (!entry?.isDirectory() || entry.isSymbolicLink()) continue;
if (workspace.repoUrl || workspace.branchName || workspace.strategyType === "git_worktree"
|| await fs.lstat(path.join(root, ".git")).then(() => true,
(error: NodeJS.ErrnoException) => { if (error.code !== "ENOENT") throw error; return false; })) return true;
}
return false;
}
async function companion(db: ReadDb, workspace: Workspace) {
const runs = await db.select({ id: heartbeatRuns.id, issueId: heartbeatRuns.nativeIssueId,
state: sql<string>`${heartbeatRuns.runnerProfileJson}#>>'{nativeWorkspaceSync,state}'` }).from(heartbeatRuns).where(and(
eq(heartbeatRuns.companyId, workspace.companyId),
sql`${heartbeatRuns.runnerProfileJson}#>>'{nativeWorkspaceSync,workspaceId}'=${workspace.id}`,
sql`${heartbeatRuns.runnerProfileJson}#>>'{nativeWorkspaceSync,state}' in ('prepared','finalized')`,
)).orderBy(heartbeatRuns.id);
const sources: LegacyWorkspaceRecoverySource[] = [];
for (const run of runs.filter(run => run.state === "prepared")) {
const source = await inspectOnlineLegacyWorkspaceRecoverySource(db, { workspace, runId: run.id });
if (!source) return null;
sources.push(source);
}
const issueIds = [...new Set([workspace.sourceIssueId, ...runs.map(run => run.issueId)].filter((id): id is string => Boolean(id)))];
const actions = issueIds.length ? await db.select().from(issueRecoveryActions).where(and(
eq(issueRecoveryActions.companyId, workspace.companyId), inArray(issueRecoveryActions.sourceIssueId, issueIds),
inArray(issueRecoveryActions.status, ["active", "escalated"]),
)).orderBy(issueRecoveryActions.id) : [];
const holds: Array<OnlineLegacyWorkspaceRecoveryHold & { actionId: string }> = [];
for (const recovery of actions) {
const hold = await inspectOnlineLegacyWorkspaceRecoveryHold(db, { workspace, recovery });
if (!hold) return null;
holds.push({ actionId: recovery.id, ...hold });
}
return { workspaceId: workspace.id, sources, holds, recoveryIssueIds: issueIds };
}
export async function inspectOnlineWorkspaceSource(db: ReadDb, workspace: Workspace): Promise<OnlineWorkspaceSource | null> {
const cwd = workspace.cwd ?? workspace.providerRef;
if (!cwd || !path.isAbsolute(cwd) || !["local_fs", "git_worktree"].includes(workspace.providerType)) return null;
const ordinary = await inspectOnlineOrdinarySource(db, workspace);
if (ordinary) {
const marker = await fs.lstat(path.join(ordinary.root, ".git")).catch((error: NodeJS.ErrnoException) => {
if (error.code !== "ENOENT") throw error; return null;
});
if (!marker) return { ...ordinary, kind: "ordinary", projectIds: ordinary.projectId ? [ordinary.projectId] : [], companions: [] };
}
let selected: Identities;
try { selected = await workspaceSourceIdentities(workspace.companyId, cwd); } catch { return null; }
if (!selected.git || selected.git.root !== await physicalPath(cwd) || selected.unownedPaths.length || !workspace.sourceIssueId
|| workspace.cwd && workspace.providerRef && workspace.cwd !== workspace.providerRef) return null;
const head = await fs.lstat(path.join(selected.git.gitDirectory.path, "HEAD")).catch((error: NodeJS.ErrnoException) => {
if (error.code !== "ENOENT") throw error; return null;
});
if (!head || !head.isFile() && !head.isSymbolicLink()) return null;
const entry = await fs.lstat(cwd);
const rows = await db.select().from(executionWorkspaces).where(and(eq(executionWorkspaces.companyId, workspace.companyId),
inArray(executionWorkspaces.providerType, ["local_fs", "git_worktree"]))).orderBy(executionWorkspaces.id).limit(1001);
if (rows.length > 1000) return null;
const nodes: Array<{ row: Workspace; identities: Identities }> = [];
for (const row of rows) {
const paths = localWorkspaceSourcePaths(row);
if (!paths.length || paths.some(cwd => !path.isAbsolute(cwd))) return null;
try {
const identities = await workspaceSourceIdentities(workspace.companyId, paths[0]);
for (const cwd of paths.slice(1)) {
const alternate = await workspaceSourceIdentities(workspace.companyId, cwd);
identities.projectIds = [...new Set([...identities.projectIds, ...alternate.projectIds])].sort();
identities.issueIds = [...new Set([...identities.issueIds, ...alternate.issueIds])].sort();
}
nodes.push({ row, identities });
} catch { return null; }
}
const projectIds = new Set(selected.projectIds), physicalIssueIds = new Set(selected.issueIds), aliases = new Set<string>();
const importedChildren: NonNullable<OnlineWorkspaceSource["importedChildren"]> = [];
try {
for (const child of await inspectImportedWorkspaceSourcePaths(db, workspace)) {
const identities = await workspaceSourceIdentities(workspace.companyId, child.sourcePath);
if (identities.unownedPaths.length) return null;
for (const id of identities.projectIds) projectIds.add(id);
for (const id of identities.issueIds) physicalIssueIds.add(id);
importedChildren.push({ ...child, gitFingerprint: identities.git?.fingerprint ?? null });
}
} catch { return null; }
for (;;) {
let changed = false;
for (const node of nodes) {
if (aliases.has(node.row.id)) continue;
if (node.row.id !== workspace.id && !intersects(node.identities.projectIds, projectIds)
&& !intersects(node.identities.issueIds, physicalIssueIds)
&& !(node.row.projectId && projectIds.has(node.row.projectId))
&& !(node.row.sourceIssueId && physicalIssueIds.has(node.row.sourceIssueId))) continue;
if (!node.row.sourceIssueId || node.row.cwd && node.row.providerRef && node.row.cwd !== node.row.providerRef
|| !["active", "idle", "in_review", "archived"].includes(node.row.status)
|| node.row.metadata?.remoteWorkspace || hasPendingGitExpiration(node.row)) return null;
aliases.add(node.row.id); changed = true;
for (const id of node.identities.projectIds) projectIds.add(id);
for (const id of node.identities.issueIds) physicalIssueIds.add(id);
if (node.row.projectId) projectIds.add(node.row.projectId);
}
if (!changed) break;
}
if (!aliases.has(workspace.id) || aliases.size > 256) return null;
const included = nodes.filter(node => aliases.has(node.row.id));
const issueIds = [...new Set([...physicalIssueIds, ...included.map(node => node.row.sourceIssueId!)])].sort();
if (issueIds.length + projectIds.size > 128) return null;
const instance = await physicalPath(resolvePaperclipInstanceRoot());
const gitSourceRoots = [...[...projectIds].map(id => path.join(instance, "projects", workspace.companyId, id)),
...issueIds.flatMap(id => ["isolated-workspaces", "isolated-workspace-attempts", "isolated-repository-sources"].map(kind =>
path.join(instance, kind, workspace.companyId, id, ...(kind === "isolated-repository-sources" ? ["checkout"] : []))))].sort();
const owned = (target: string) => gitSourceRoots.some(root => target === root || target.startsWith(root + path.sep));
let after: string | undefined;
for (let page = 0; ; page++) {
const foreign = await db.select({ id: executionWorkspaces.id, cwd: executionWorkspaces.cwd, providerRef: executionWorkspaces.providerRef })
.from(executionWorkspaces).where(and(ne(executionWorkspaces.companyId, workspace.companyId),
inArray(executionWorkspaces.providerType, ["local_fs", "git_worktree"]), ...(after ? [gt(executionWorkspaces.id, after)] : [])))
.orderBy(executionWorkspaces.id).limit(256);
for (const row of foreign) {
for (const cwd of localWorkspaceSourcePaths(row)) {
if (!path.isAbsolute(cwd)) return null;
try {
if (owned(await physicalPath(cwd))) return null;
const git = await readWorkspaceGitSourceRoots(cwd, { ancestors: true });
if (git && [git.gitDirectory.path, git.commonDirectory.path, ...git.objectDirectories.map(root => root.path)].some(owned)) return null;
} catch { return null; }
}
}
if (foreign.length < 256) break;
if (page === 39) return null;
after = foreign.at(-1)!.id;
}
const recipe = readGitCacheRestoration(workspace);
if (recipe && (!recipe.restoredAt || recipe.contents && !recipe.contentsRestoredAt
|| recipe.restorePoint && !recipe.restorePoint.expiredAt && !recipe.restorePoint.appliedAt)) return null;
const companions: OnlineWorkspaceSource["companions"] = [], settledPreparedWorkspaceIds: string[] = [], settledRecoveryIssueIds = new Set<string>();
for (const node of included) {
const proof = await companion(db, node.row);
if (!proof) return null;
companions.push({ workspaceId: proof.workspaceId, sources: proof.sources, holds: proof.holds });
if (proof.sources.length) settledPreparedWorkspaceIds.push(proof.workspaceId);
for (const id of proof.recoveryIssueIds) settledRecoveryIssueIds.add(id);
}
return { kind: "git", root: selected.git.root, projectId: null, projectIds: [...projectIds].sort(), issueIds,
workspaceIds: [...aliases].sort(), device: entry.dev, inode: entry.ino, gitSourceRoots, companions,
physicalRoots: included.flatMap(node => localWorkspaceSourcePaths(node.row)).sort(),
settledPreparedWorkspaceIds: settledPreparedWorkspaceIds.sort(), settledRecoveryIssueIds: [...settledRecoveryIssueIds].sort(),
settledFinalizationRunIds: [...new Set(companions.flatMap(proof => proof.sources.map(source => source.runId)))].sort(),
gitSourceFingerprints: included.map(node => ({ workspaceId: node.row.id, fingerprint: node.identities.git?.fingerprint ?? null })), importedChildren,
};
}
@@ -1,11 +1,13 @@
import fs from "node:fs/promises";
import path from "node:path";
import { isDeepStrictEqual } from "node:util";
import { and, eq, gt, inArray, ne, sql } from "drizzle-orm";
import { and, asc, eq, gt, inArray, ne, sql } from "drizzle-orm";
import { executionWorkspaces, executionWorkspaceSnapshots, heartbeatRuns, workspaceOperations, type Db } from "@paperclipai/db";
import { resolvePaperclipInstanceRoot } from "../home-paths.js";
import { adapterExecutionControls } from "./adapter-execution-control.js";
import { managedProjectSourceIdentity } from "./workspace-source-lock.js";
import { readWorkspaceGitSourceRoots } from "./workspace-git-source-roots.js";
import { localWorkspaceSourcePaths } from "./execution-workspace-source-roots.js";
import { offlineMigrationExecutionWhere } from "./offline-migration-execution.js";
import type { StorageService } from "../storage/types.js";
import { captureRemoteWorkspaceDirectory, restoreRemoteWorkspaceArchive } from "./remote-workspace-snapshot-archive.js";
@@ -49,25 +51,32 @@ export async function offlineWorkspaceMigrationBlockers(db: ReadDb) {
return active ? ["active_execution_requires_drain"] : [];
}
export type OnlineOrdinarySource = {
export type WorkspaceSourceScope = {
root: string; projectId: string | null; issueIds: string[]; workspaceIds: string[]; device: number; inode: number;
physicalRoots?: string[]; gitSourceRoots?: string[]; settledPreparedWorkspaceIds?: string[]; settledRecoveryIssueIds?: string[]; settledFinalizationRunIds?: string[];
importedChildren?: Array<{ operationId: string; sourcePath: string }>;
};
export async function requiresOnlineOrdinarySource(workspace: Workspace): Promise<boolean> {
if (!workspace.cwd || workspace.providerType !== "local_fs" || workspace.repoUrl || workspace.branchName) return false;
const parts = path.relative(resolvePaperclipInstanceRoot(), workspace.cwd).split(path.sep);
const managed = parts[1] === workspace.companyId && Boolean(workspace.sourceIssueId) && (
parts[0] === "projects" && parts.length === 4 && /^[a-f0-9-]{36}$/.test(parts[2])
|| parts[2] === workspace.sourceIssueId && (parts[0] === "isolated-workspaces" && parts.length === 3
|| parts[0] === "isolated-workspace-attempts" && parts.length === 4));
if (!managed) return false;
const entry = await fs.lstat(workspace.cwd).catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return null; throw error;
});
return Boolean(entry?.isDirectory() && !entry.isSymbolicLink());
if (workspace.providerType !== "local_fs" || workspace.repoUrl || workspace.branchName) return false;
const instance = await physicalPath(resolvePaperclipInstanceRoot());
for (const cwd of localWorkspaceSourcePaths(workspace)) {
const root = await physicalPath(cwd);
const parts = path.relative(instance, root).split(path.sep);
const managed = parts[1] === workspace.companyId && Boolean(workspace.sourceIssueId) && (
parts[0] === "projects" && parts.length === 4 && /^[a-f0-9-]{36}$/.test(parts[2])
|| parts[2] === workspace.sourceIssueId && (parts[0] === "isolated-workspaces" && parts.length === 3
|| parts[0] === "isolated-workspace-attempts" && parts.length === 4));
if (!managed) continue;
const entry = await fs.lstat(root).catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return null; throw error;
});
if (entry?.isDirectory() && !entry.isSymbolicLink()) return true;
}
return false;
}
export async function inspectOnlineOrdinarySource(db: ReadDb, workspace: Workspace): Promise<OnlineOrdinarySource | null> {
export async function inspectOnlineOrdinarySource(db: ReadDb, workspace: Workspace): Promise<Omit<WorkspaceSourceScope, "importedChildren"> | null> {
if (!workspace.cwd || workspace.providerType !== "local_fs" || workspace.repoUrl || workspace.branchName) return null;
const root = await physicalPath(workspace.cwd);
const instance = await physicalPath(resolvePaperclipInstanceRoot());
@@ -98,10 +107,10 @@ export async function inspectOnlineOrdinarySource(db: ReadDb, workspace: Workspa
if (rows.length > 1000) return null;
const aliases: typeof rows = [];
for (const row of rows) {
const cwd = rootOf(row);
let physical: string;
try { physical = cwd ? await physicalPath(cwd) : ""; } catch { return null; }
if (!physical || !overlaps(root, physical)) continue;
const paths = localWorkspaceSourcePaths(row);
let physical: string[];
try { physical = await Promise.all(paths.map(physicalPath)); } catch { return null; }
if (!physical.some(cwd => overlaps(root, cwd))) continue;
if (row.cwd !== root || row.providerRef && row.providerRef !== root
|| row.providerType !== "local_fs" || row.repoUrl || row.branchName || !row.sourceIssueId
|| isolated && row.sourceIssueId !== workspace.sourceIssueId
@@ -119,8 +128,9 @@ export async function inspectOnlineOrdinarySource(db: ReadDb, workspace: Workspa
...(after ? [gt(executionWorkspaces.id, after)] : []),
)).orderBy(executionWorkspaces.id).limit(256);
for (const row of foreign) {
const cwd = rootOf(row);
try { if (cwd && overlaps(root, await physicalPath(cwd))) return null; } catch { return null; }
for (const cwd of localWorkspaceSourcePaths(row)) {
try { if (overlaps(root, await physicalPath(cwd))) return null; } catch { return null; }
}
}
if (foreign.length < 256) break;
if (page === 39) return null;
@@ -132,9 +142,12 @@ export async function inspectOnlineOrdinarySource(db: ReadDb, workspace: Workspa
}
export async function onlineOrdinarySourceBlockers(db: Pick<Db, "execute">, companyId: string,
source: OnlineOrdinarySource, workspace: Pick<Workspace, "id" | "sourceIssueId">, operationId?: string): Promise<string[]> {
source: WorkspaceSourceScope, workspace: Pick<Workspace, "id" | "sourceIssueId">, operationId?: string): Promise<string[]> {
const workspaceIds = sql.join(source.workspaceIds.map(id => sql`${id}::uuid`), sql`, `);
const issueIds = sql.join(source.issueIds.map(id => sql`${id}::uuid`), sql`, `);
const settledWorkspaces = sql.join((source.settledPreparedWorkspaceIds?.length ? source.settledPreparedWorkspaceIds : ["00000000-0000-0000-0000-000000000000"]).map(id => sql`${id}`), sql`, `);
const settledFinalizers = sql.join((source.settledFinalizationRunIds?.length ? source.settledFinalizationRunIds : ["00000000-0000-0000-0000-000000000000"]).map(id => sql`${id}::uuid`), sql`, `);
const settledRecoveries = sql.join((source.settledRecoveryIssueIds?.length ? source.settledRecoveryIssueIds : ["00000000-0000-0000-0000-000000000000"]).map(id => sql`${id}::uuid`), sql`, `);
const rows = await db.execute<{ reason: string }>(sql`
select 'active_execution' reason where exists (select 1 from heartbeat_runs r where r.company_id=${companyId}::uuid
and r.status in ('queued','running') and (r.issue_id in (${issueIds}) or r.native_issue_id in (${issueIds})
@@ -154,18 +167,19 @@ export async function onlineOrdinarySourceBlockers(db: Pick<Db, "execute">, comp
and r.runner_profile_json#>>'{nativeWorkspaceSync,workspaceId}' in (${sql.join(source.workspaceIds.map(id => sql`${id}`), sql`, `)})
and (r.runner_profile_json#>>'{nativeWorkspaceSync,workspaceId}'<>${workspace.id}
and r.runner_profile_json#>>'{nativeWorkspaceSync,state}'='prepared'
and r.runner_profile_json#>>'{nativeWorkspaceSync,workspaceId}' not in (${settledWorkspaces})
or r.runner_profile_json->'nativeWorkspaceFinalizationOwner' is not null
or r.controller_lease_expires_at>clock_timestamp()))
union all select 'execution_finalization_unresolved' where exists (select 1 from native_run_finalizations f where f.company_id=${companyId}::uuid
and (f.issue_id in (${issueIds}) and f.issue_id<>${workspace.sourceIssueId}::uuid or f.run_id in
(select r.id from heartbeat_runs r where r.company_id=${companyId}::uuid
and r.runner_profile_json#>>'{nativeWorkspaceSync,workspaceId}' in (${sql.join(source.workspaceIds.map(id => sql`${id}`), sql`, `)})
and r.runner_profile_json#>>'{nativeWorkspaceSync,workspaceId}'<>${workspace.id})) and (f.phase not in ('committed','terminal_failure') or f.lease_owner is not null))
and r.runner_profile_json#>>'{nativeWorkspaceSync,workspaceId}'<>${workspace.id})) and f.run_id not in (${settledFinalizers}) and (f.phase not in ('committed','terminal_failure') or f.lease_owner is not null))
union all select 'workspace_recovery_unresolved' where exists (select 1 from issue_recovery_actions a where a.company_id=${companyId}::uuid
and (a.source_issue_id in (${issueIds}) and a.source_issue_id<>${workspace.sourceIssueId}::uuid or a.source_issue_id in
(select r.native_issue_id from heartbeat_runs r where r.company_id=${companyId}::uuid
and r.runner_profile_json#>>'{nativeWorkspaceSync,workspaceId}' in (${sql.join(source.workspaceIds.map(id => sql`${id}`), sql`, `)})
and r.runner_profile_json#>>'{nativeWorkspaceSync,workspaceId}'<>${workspace.id})) and a.status in ('active','escalated'))
and r.runner_profile_json#>>'{nativeWorkspaceSync,workspaceId}'<>${workspace.id})) and a.source_issue_id not in (${settledRecoveries}) and a.status in ('active','escalated'))
union all select 'repository_preparation_unresolved' where exists (select 1 from execution_workspace_repositories r where r.company_id=${companyId}::uuid
and r.execution_workspace_id in (${workspaceIds}) and r.state<>'ready')`);
const services = await db.execute<{ cwd: string | null; provider: string }>(sql`
@@ -174,9 +188,17 @@ export async function onlineOrdinarySourceBlockers(db: Pick<Db, "execute">, comp
if (services.length > 1000) reasons.push("workspace_runtime_service_unsettled");
else for (const service of services) {
try {
if (service.cwd ? overlaps(source.root, await physicalPath(service.cwd)) : service.provider !== "adapter_managed") {
reasons.push("workspace_runtime_service_unsettled"); break;
const cwd = service.cwd ? await physicalPath(service.cwd) : null;
let overlap = cwd ? (source.physicalRoots ?? [source.root]).some(root => overlaps(root, cwd)) : service.provider !== "adapter_managed";
if (!overlap && cwd && source.gitSourceRoots) {
const owned = (target: string) => source.gitSourceRoots!.some(root => target === root || target.startsWith(root + path.sep));
overlap = owned(cwd);
if (!overlap) {
const git = await readWorkspaceGitSourceRoots(cwd, { ancestors: true });
overlap = Boolean(git && [git.gitDirectory.path, git.commonDirectory.path, ...git.objectDirectories.map(root => root.path)].some(owned));
}
}
if (overlap) { reasons.push("workspace_runtime_service_unsettled"); break; }
} catch { reasons.push("workspace_runtime_service_unsettled"); break; }
}
const controlled = [...adapterExecutionControls.keys()];
@@ -191,7 +213,7 @@ export async function onlineOrdinarySourceBlockers(db: Pick<Db, "execute">, comp
/** Only the maintenance entry point can acknowledge that every old deployment
* has stopped. A task-level migration hold cannot fence other path aliases. */
export async function workspaceMigrationLayoutBlockers(db: ReadDb, workspace: Workspace, writersStopped = false, onlineSource?: OnlineOrdinarySource) {
export async function workspaceMigrationLayoutBlockers(db: ReadDb, workspace: Workspace, writersStopped = false, onlineSource?: WorkspaceSourceScope) {
if (writersStopped) return offlineWorkspaceMigrationBlockers(db);
const cwd = rootOf(workspace);
if (!cwd) return [];
@@ -201,45 +223,73 @@ export async function workspaceMigrationLayoutBlockers(db: ReadDb, workspace: Wo
inArray(executionWorkspaces.providerType, ["local_fs", "git_worktree"]),
));
const root = await physicalPath(cwd);
for (const row of local) if (!onlineSource?.workspaceIds.includes(row.id) && rootOf(row) && overlaps(root, await physicalPath(rootOf(row)!)))
return ["overlapping_workspace_requires_maintenance"];
const migrated = await db.select({ cwd: workspaceOperations.cwd }).from(workspaceOperations).where(and(
for (const row of local) {
for (const cwd of localWorkspaceSourcePaths(row)) {
const physical = await physicalPath(cwd);
if (onlineSource?.gitSourceRoots && physical.startsWith(`${root}/.paperclip/worktrees/`)) {
const entry = await fs.lstat(physical).catch((error: NodeJS.ErrnoException) => { if (error.code !== "ENOENT") throw error; return null; });
if (entry) return ["nested_workspace_import_required"];
}
if (!onlineSource?.workspaceIds.includes(row.id) && overlaps(root, physical)) return ["overlapping_workspace_requires_maintenance"];
}
}
const migrated = await db.select({ id: workspaceOperations.id, cwd: workspaceOperations.cwd }).from(workspaceOperations).where(and(
eq(workspaceOperations.companyId, workspace.companyId), eq(workspaceOperations.phase, "remote_workspace_migration"),
eq(workspaceOperations.status, "succeeded"), ne(workspaceOperations.executionWorkspaceId, workspace.id),
));
for (const row of migrated) if (row.cwd && (await physicalPath(row.cwd)).startsWith(`${root}/.paperclip/worktrees/`))
return ["overlapping_workspace_requires_maintenance"];
for (const row of migrated) {
if (!row.cwd) continue;
const sourcePath = await physicalPath(row.cwd);
if (sourcePath.startsWith(`${root}/.paperclip/worktrees/`)
&& !(onlineSource?.gitSourceRoots && onlineSource.importedChildren?.some(child => child.operationId === row.id && child.sourcePath === sourcePath)))
return ["overlapping_workspace_requires_maintenance"];
}
return [];
}
async function importedWorkspaceCandidates(db: ReadDb, workspace: Workspace) {
const sourceRoot = rootOf(workspace);
if (!sourceRoot) return [];
const root = await physicalPath(sourceRoot);
const rows = await db.select({ operation: workspaceOperations, snapshot: executionWorkspaceSnapshots })
.from(workspaceOperations).innerJoin(executionWorkspaceSnapshots, and(
eq(executionWorkspaceSnapshots.companyId, workspaceOperations.companyId),
eq(executionWorkspaceSnapshots.executionWorkspaceId, workspaceOperations.executionWorkspaceId),
eq(executionWorkspaceSnapshots.sourceOperationId, workspaceOperations.id),
)).where(and(eq(workspaceOperations.companyId, workspace.companyId),
ne(workspaceOperations.executionWorkspaceId, workspace.id), eq(workspaceOperations.phase, "remote_workspace_migration"),
eq(workspaceOperations.status, "succeeded"), inArray(executionWorkspaceSnapshots.status, ["committed", "superseded"]),
sql`${workspaceOperations.metadata}->>'snapshotId' = ${executionWorkspaceSnapshots.id}::text`,
)).orderBy(asc(workspaceOperations.id)).limit(10241);
if (rows.length > 10240) throw new Error("nested_workspace_scope_limit");
const located = [];
for (const row of rows) {
if (!row.operation.cwd) continue;
located.push({ ...row, sourcePath: await physicalPath(row.operation.cwd) });
}
return located.filter(row => row.sourcePath.startsWith(`${root}/.paperclip/worktrees/`))
.sort((a, b) => a.sourcePath.length - b.sourcePath.length || a.operation.id.localeCompare(b.operation.id));
}
export async function inspectImportedWorkspaceSourcePaths(db: ReadDb, workspace: Workspace) {
return (await importedWorkspaceCandidates(db, workspace)).map(({ operation, snapshot, sourcePath }) => ({
workspaceId: snapshot.executionWorkspaceId, operationId: operation.id, snapshotId: snapshot.id,
sourcePath, manifestDigest: snapshot.manifestDigest, operationManifestDigest: operation.metadata?.manifestDigest,
}));
}
/** Runtime-created nested task folders can be omitted only after an exact
* committed import covers their current bytes. Ordinary user folders stay in
* the parent archive. The snapshot owner retains the resulting dependencies. */
export async function prepareNestedWorkspaceImports(input: {
db: Db; storage: StorageService; workspace: Workspace; scratch: string; signal?: AbortSignal;
recordedChildren?: ImportedWorkspaceChild[];
recordedChildren?: ImportedWorkspaceChild[]; gitSourceRoots?: string[];
}): Promise<ImportedWorkspaceChild[]> {
const sourceRoot = rootOf(input.workspace);
if (!sourceRoot) return [];
if (input.recordedChildren && !input.recordedChildren.length) return [];
if (!sourceRoot || input.recordedChildren && !input.recordedChildren.length) return [];
const root = await physicalPath(sourceRoot);
const rows = await input.db.select({ operation: workspaceOperations, snapshot: executionWorkspaceSnapshots })
.from(workspaceOperations).innerJoin(executionWorkspaceSnapshots, and(
eq(executionWorkspaceSnapshots.companyId, workspaceOperations.companyId),
eq(executionWorkspaceSnapshots.executionWorkspaceId, workspaceOperations.executionWorkspaceId),
eq(executionWorkspaceSnapshots.sourceOperationId, workspaceOperations.id),
)).where(and(eq(workspaceOperations.companyId, input.workspace.companyId),
ne(workspaceOperations.executionWorkspaceId, input.workspace.id), eq(workspaceOperations.phase, "remote_workspace_migration"),
eq(workspaceOperations.status, "succeeded"), inArray(executionWorkspaceSnapshots.status, ["committed", "superseded"]),
sql`${workspaceOperations.metadata}->>'snapshotId' = ${executionWorkspaceSnapshots.id}::text`,
));
const located = [];
for (const row of rows) {
if (!row.operation.cwd || input.recordedChildren && !input.recordedChildren.some(child => child.operationId === row.operation.id)) continue;
located.push({ ...row, sourcePath: await physicalPath(row.operation.cwd) });
}
const candidates = located.filter(row => row.sourcePath.startsWith(`${root}/.paperclip/worktrees/`))
.sort((a, b) => a.sourcePath.length - b.sourcePath.length || a.operation.id.localeCompare(b.operation.id));
const candidates = (await importedWorkspaceCandidates(input.db, input.workspace)).filter(row =>
!input.recordedChildren || input.recordedChildren.some(child => child.operationId === row.operation.id));
const covered: ImportedWorkspaceChild[] = [];
for (const { operation, snapshot, sourcePath } of candidates) {
input.signal?.throwIfAborted();
@@ -259,7 +309,7 @@ export async function prepareNestedWorkspaceImports(input: {
async function verifyCoveredWorkspace(input: {
db: Db; storage: StorageService; sourcePath: string; snapshot: typeof executionWorkspaceSnapshots.$inferSelect;
operationId: string; expectedDigest: unknown; scratch: string; signal?: AbortSignal; depth: number;
operationId: string; expectedDigest: unknown; scratch: string; signal?: AbortSignal; depth: number; gitSourceRoots?: string[];
}) {
input.signal?.throwIfAborted();
if (input.depth > 32) throw new Error("nested_workspace_coverage_invalid");
@@ -287,6 +337,6 @@ async function verifyCoveredWorkspace(input: {
await loadWorkspaceSnapshotArchive({ storage: input.storage, reference, targetPath: archivePath, signal: input.signal });
await restoreRemoteWorkspaceArchive({ archivePath, targetDir: restored, expectedManifestDigest: reference.manifestDigest, signal: input.signal });
const current = await captureRemoteWorkspaceDirectory({ cwd: input.sourcePath, archivePath: path.join(area, "current.pcws"),
exclude: children.map(child => child.relativePath), signal: input.signal });
exclude: children.map(child => child.relativePath), gitSourceRoots: input.gitSourceRoots, signal: input.signal });
if (current.manifestDigest !== reference.manifestDigest) throw new Error("nested_workspace_source_changed");
}
@@ -521,7 +521,8 @@ describe.skipIf(!support.supported)("remote workspace migration", () => {
});
async function gitArchiveFixture(repoUrl = "https://example.invalid/repository.git", home = root) {
const f = await seed(), projectId = randomUUID(), projectWorkspaceId = randomUUID(), branchName = "saved-task";
vi.stubEnv("PAPERCLIP_HOME", home); vi.stubEnv("PAPERCLIP_INSTANCE_ID", "default");
await fs.mkdir(home, { recursive: true });
vi.stubEnv("PAPERCLIP_HOME", await fs.realpath(home)); vi.stubEnv("PAPERCLIP_INSTANCE_ID", "default");
const base = resolveManagedProjectWorkspaceDir({ companyId: f.companyId, projectId, repoName: "repository" });
await fs.mkdir(base, { recursive: true }); await git(base, "init", "-q", "-b", "main");
await fs.writeFile(path.join(base, "base.txt"), "base\n"); await git(base, "add", "."); await git(base, "commit", "-qm", "base");
@@ -1143,7 +1144,13 @@ describe.skipIf(!support.supported)("remote workspace migration", () => {
expect(await fs.readFile(path.join(f.cwd, "saved.txt"), "utf8")).toBe("new local edits\n");
});
it("imports dirty unpushed Git work and restores it after the original folder disappears", async () => {
const f = await seed(); await git(f.cwd, "init", "-q", "-b", "feature");
let f = await seed();
vi.stubEnv("PAPERCLIP_HOME", await fs.realpath(root));
const cwd = path.join(resolvePaperclipInstanceRoot(), "isolated-workspaces", f.companyId, f.issueId);
await fs.mkdir(path.dirname(cwd), { recursive: true }); await fs.rename(f.cwd, cwd);
await db.update(executionWorkspaces).set({ cwd, providerRef: cwd }).where(eq(executionWorkspaces.id, f.workspaceId));
f = { ...f, cwd };
await git(f.cwd, "init", "-q", "-b", "feature");
await git(f.cwd, "add", "."); await git(f.cwd, "commit", "-qm", "unpushed work");
await git(f.cwd, "remote", "add", "origin", "https://example.invalid/repository.git");
const head = await git(f.cwd, "rev-parse", "HEAD");
@@ -22,8 +22,9 @@ import { loadTaskDirectoryRestorePoint, mayBeIsolatedTaskDirectoryWorkspace, rea
import { inspectArchivedGitMigrationSource, lockArchivedGitMigrationSources, materializeArchivedGitMigrationSource, type ArchivedGitMigrationSource } from "./remote-workspace-migration-git.js";
import { readNativeWorkspaceSyncReference } from "./native-runtime/native-workspace-sync.js";
import { isLegacyWorkspaceTaskRelated, readLegacyWorkspaceRecoveryAllocations, LEGACY_WORKSPACE_RECOVERY_MIGRATION, type LegacyWorkspaceRecoverySource } from "./legacy-workspace-recovery-migration.js";
import { inspectOnlineOrdinarySource, requiresOnlineOrdinarySource, onlineOrdinarySourceBlockers, type OnlineOrdinarySource, offlineWorkspaceMigrationBlockers, prepareNestedWorkspaceImports, validateImportedWorkspaceChildren, workspaceMigrationLayoutBlockers } from "./remote-workspace-migration-tree.js";
import { onlineOrdinarySourceBlockers, offlineWorkspaceMigrationBlockers, prepareNestedWorkspaceImports, validateImportedWorkspaceChildren, workspaceMigrationLayoutBlockers } from "./remote-workspace-migration-tree.js";
import { inspectOnlineWorkspaceSource, requiresOnlineWorkspaceSource, type OnlineWorkspaceSource } from "./online-workspace-source.js";
import { withWorkspaceSourceLocks, IsolatedRepositorySourceBusy } from "./workspace-source-lock.js";
import { inspectOnlineLegacyWorkspaceRecoverySource, inspectOnlineLegacyWorkspaceRecoveryHold, lockOnlineLegacyWorkspaceRecoverySources, type OnlineLegacyWorkspaceRecoveryHold } from "./online-legacy-workspace-recovery.js";
import { inspectNativeWorkspaceMigration, nativeWorkspaceMigrationMarker } from "./native-runtime/native-workspace-migration.js";
@@ -84,7 +85,7 @@ async function isUnmaterializedRepositoryRequest(workspace: Workspace,
});
}
async function inspect(db: ReadDb, input: { companyId: string; workspaceId: string; operationId?: string; writersStopped?: boolean; onlineSource?: OnlineOrdinarySource; onlineSourceUnproven?: boolean }) {
async function inspect(db: ReadDb, input: { companyId: string; workspaceId: string; operationId?: string; writersStopped?: boolean; onlineSource?: OnlineWorkspaceSource; onlineSourceUnproven?: boolean }) {
const [workspace] = await db.select().from(executionWorkspaces).where(and(eq(executionWorkspaces.companyId, input.companyId), eq(executionWorkspaces.id, input.workspaceId)));
if (!workspace) throw new Error("remote_workspace_migration_workspace_missing");
const authority = readRemoteWorkspaceAuthority(workspace.metadata);
@@ -95,7 +96,7 @@ async function inspect(db: ReadDb, input: { companyId: string; workspaceId: stri
if (input.writersStopped) reasons.push(...await offlineWorkspaceMigrationBlockers(db));
if (input.onlineSourceUnproven) reasons.push("online_workspace_source_scope_unproven");
if (input.onlineSource) {
const currentScope = await inspectOnlineOrdinarySource(db, workspace);
const currentScope = await inspectOnlineWorkspaceSource(db, workspace);
if (canonical(currentScope) !== canonical(input.onlineSource)) reasons.push("online_workspace_source_scope_changed");
else reasons.push(...await onlineOrdinarySourceBlockers(db as Db, workspace.companyId, currentScope!, workspace, allowedOperation));
}
@@ -311,9 +312,9 @@ export async function inventoryRemoteWorkspaceMigration(db: Db, input: { company
const [workspace] = !input.writersStopped ? await db.select().from(executionWorkspaces).where(and(
eq(executionWorkspaces.companyId, input.companyId), eq(executionWorkspaces.id, input.workspaceId))) : [];
const onlineSource = workspace && !readRemoteWorkspaceAuthority(workspace.metadata)
? await inspectOnlineOrdinarySource(db, workspace) ?? undefined : undefined;
? await inspectOnlineWorkspaceSource(db, workspace) ?? undefined : undefined;
const result = await inspect(db, { ...input, onlineSource, onlineSourceUnproven: Boolean(workspace && !onlineSource
&& !readRemoteWorkspaceAuthority(workspace.metadata) && await requiresOnlineOrdinarySource(workspace)) });
&& !readRemoteWorkspaceAuthority(workspace.metadata) && await requiresOnlineWorkspaceSource(workspace)) });
return { companyId: input.companyId, workspaceId: input.workspaceId, eligible: result.reasons.length === 0,
alreadyMigrated: Boolean(result.authority), reasons: result.reasons, localPath: result.workspace.cwd ?? result.workspace.providerRef,
source: result.source, operationId: typeof result.marker.operationId === "string" ? result.marker.operationId : null,
@@ -329,9 +330,9 @@ export async function migrateRemoteWorkspace(input: {
const [sourceWorkspace] = !input.writersStopped ? await input.db.select().from(executionWorkspaces).where(and(
eq(executionWorkspaces.companyId, input.companyId), eq(executionWorkspaces.id, input.workspaceId))) : [];
const onlineSource = sourceWorkspace && !readRemoteWorkspaceAuthority(sourceWorkspace.metadata)
? await inspectOnlineOrdinarySource(input.db, sourceWorkspace) ?? undefined : undefined;
? await inspectOnlineWorkspaceSource(input.db, sourceWorkspace) ?? undefined : undefined;
if (sourceWorkspace && !onlineSource && !readRemoteWorkspaceAuthority(sourceWorkspace.metadata)
&& await requiresOnlineOrdinarySource(sourceWorkspace)) {
&& await requiresOnlineWorkspaceSource(sourceWorkspace)) {
throw new Error("remote_workspace_migration_blocked:online_workspace_source_scope_unproven");
}
let committedPublication: Parameters<typeof publishActivity>[0] | undefined;
@@ -390,7 +391,8 @@ export async function migrateRemoteWorkspace(input: {
let operation = reserved.operation!;
if (onlineSource) {
if (!await lockOnlineLegacyWorkspaceRecoverySources(lock as unknown as Db, { companyId: input.companyId,
sources: reserved.legacyRecoverySources, holds: reserved.onlineRecoveryHolds }))
sources: [...reserved.legacyRecoverySources, ...onlineSource.companions.flatMap(proof => proof.sources)],
holds: [...reserved.onlineRecoveryHolds, ...onlineSource.companions.flatMap(proof => proof.holds)] }))
throw new Error("remote_workspace_migration_blocked:remote_copyback_owned");
const current = await inspect(input.db, { ...input, onlineSource, operationId: operation.id });
if (current.reasons.length || canonical(current.legacyRecoverySources) !== canonical(reserved.legacyRecoverySources)
@@ -420,8 +422,8 @@ export async function migrateRemoteWorkspace(input: {
// archive identity even if a later batch has already imported the leaves.
const recordedChildren = scope.state === "fixed" ? validateImportedWorkspaceChildren(scope.children)
: scope.state === "pending" ? undefined : [];
const nested = input.writersStopped && !retained && !reserved.archivedGitSource
? await prepareNestedWorkspaceImports({ ...input, workspace: reserved.workspace, scratch, recordedChildren }) : [];
const nested = (input.writersStopped || onlineSource?.kind === "git") && !retained && !reserved.archivedGitSource
? await prepareNestedWorkspaceImports({ ...input, workspace: reserved.workspace, scratch, recordedChildren, gitSourceRoots: onlineSource?.gitSourceRoots }) : [];
if (scope.state === "pending") {
const [saved] = await input.db.update(workspaceOperations).set({ metadata: { ...operation.metadata,
nestedWorkspaceScope: { version: 1, state: "fixed", children: nested } }, updatedAt: new Date() })
@@ -431,7 +433,7 @@ export async function migrateRemoteWorkspace(input: {
operation = saved;
}
const exclude = nested.map(child => child.relativePath);
const captured = recovered ?? await captureRemoteWorkspaceDirectory({ cwd: sourceRoot!, archivePath, exclude, requireNoGit: Boolean(onlineSource), signal: input.signal });
const captured = recovered ?? await captureRemoteWorkspaceDirectory({ cwd: sourceRoot!, archivePath, exclude, requireNoGit: onlineSource?.kind === "ordinary", gitSourceRoots: onlineSource?.gitSourceRoots, signal: input.signal });
const ref = await uploadWorkspaceSnapshot({ db: input.db, storage: input.storage, snapshotId: snapshot.id,
companyId: input.companyId, workspaceId: input.workspaceId, archivePath, manifestDigest: captured.manifestDigest, signal: input.signal,
// Private Git reconstruction can change administrative bytes between
@@ -444,7 +446,7 @@ export async function migrateRemoteWorkspace(input: {
await restoreRemoteWorkspaceArchive({ archivePath: restoredArchive, targetDir: restored, expectedManifestDigest: ref.manifestDigest, signal: input.signal });
if (!recovered) {
const currentArchive = path.join(scratch, "current.pcws");
const current = await captureRemoteWorkspaceDirectory({ cwd: sourceRoot!, archivePath: currentArchive, exclude, requireNoGit: Boolean(onlineSource), signal: input.signal });
const current = await captureRemoteWorkspaceDirectory({ cwd: sourceRoot!, archivePath: currentArchive, exclude, requireNoGit: onlineSource?.kind === "ordinary", gitSourceRoots: onlineSource?.gitSourceRoots, signal: input.signal });
if (current.manifestDigest !== captured.manifestDigest) throw new Error("remote_workspace_migration_source_changed");
}
if (nested.length) {
@@ -561,7 +563,7 @@ export async function migrateRemoteWorkspace(input: {
if (onlineSource) {
const result = await withWorkspaceSourceLocks(input.db, {
companyId: input.companyId, issueIds: onlineSource.issueIds,
projectIds: onlineSource.projectId ? [onlineSource.projectId] : [],
projectIds: onlineSource.projectIds,
}, async lock => {
for (const workspaceId of onlineSource.workspaceIds) {
if (!await tryAcquireExecutionWorkspaceLifecycleLock(lock, workspaceId))
@@ -1,3 +1,5 @@
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { randomUUID } from "node:crypto";
import fs from "node:fs/promises";
import os from "node:os";
@@ -74,9 +76,153 @@ describe.skipIf(!support.supported)("online ordinary physical source fence", ()
}
return { companyId, projectId, cwd, aliases, workspaceId: aliases[0].workspace.id };
}
const execute = promisify(execFile);
const git = async (cwd: string, ...args: string[]) => (await execute("git", ["-c", `core.hooksPath=${os.devNull}`, "-C", cwd, ...args], {
env: { ...process.env, GIT_CONFIG_NOSYSTEM: "1", GIT_CONFIG_GLOBAL: os.devNull, GIT_TERMINAL_PROMPT: "0",
GIT_AUTHOR_NAME: "Source fixture", GIT_AUTHOR_EMAIL: "source@example.test", GIT_COMMITTER_NAME: "Source fixture", GIT_COMMITTER_EMAIL: "source@example.test" },
})).stdout.trim();
async function gitFixture(f = undefined as Awaited<ReturnType<typeof fixture>> | undefined) {
const base = f ?? await fixture();
await git(base.cwd, "init", "-q", "-b", "main");
await git(base.cwd, "add", "."); await git(base.cwd, "commit", "-qm", "Saved legacy work");
const repoUrl = "https://example.invalid/saved-work.git";
await git(base.cwd, "remote", "add", "origin", repoUrl);
for (const alias of base.aliases) await db.update(executionWorkspaces).set({ repoUrl, branchName: "main" })
.where(eq(executionWorkspaces.id, alias.workspace.id));
const [issue] = await db.insert(issues).values({ companyId: base.companyId, projectId: base.projectId, title: "Shared Git child" }).returning();
const cwd = path.join(base.cwd, ".paperclip", "worktrees", "child");
await git(base.cwd, "worktree", "add", "-qb", "child", cwd);
const [workspace] = await db.insert(executionWorkspaces).values({ companyId: base.companyId, projectId: base.projectId,
sourceIssueId: issue.id, name: "Shared Git child", mode: "isolated_workspace", strategyType: "git_worktree",
providerType: "git_worktree", cwd, providerRef: cwd, repoUrl, branchName: "child", baseRef: "main" }).returning();
await db.update(issues).set({ executionWorkspaceId: workspace.id }).where(eq(issues.id, issue.id));
return { ...base, child: workspace, repoUrl };
}
async function stored(workspaceId: string) {
return (await db.select().from(executionWorkspaces).where(eq(executionWorkspaces.id, workspaceId)))[0];
}
it("imports an ordinary managed directory without capturing its enclosing host repository", async () => {
await git(home, "init", "-q", "-b", "main");
await fs.writeFile(path.join(home, "host-only.txt"), "outside the workspace\n");
await git(home, "add", "host-only.txt"); await git(home, "commit", "-qm", "Enclosing host repository");
const head = await git(home, "rev-parse", "HEAD"), f = await fixture();
expect(await inventoryRemoteWorkspaceMigration(db, f)).toMatchObject({ eligible: true });
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).resolves.toMatchObject({ applied: true });
const authority = readRemoteWorkspaceAuthority((await stored(f.workspaceId)).metadata)!;
const reference = await getWorkspaceSnapshot(db, { ...f, snapshotId: authority.snapshotId! });
const archivePath = path.join(home, "ordinary.pcws"), targetDir = path.join(home, "ordinary-restored");
await loadWorkspaceSnapshotArchive({ storage, reference: reference!, targetPath: archivePath });
await restoreRemoteWorkspaceArchive({ archivePath, targetDir, expectedManifestDigest: reference!.manifestDigest });
expect(await fs.readFile(path.join(targetDir, "saved.txt"), "utf8")).toBe("saved legacy edits\n");
expect(await fs.lstat(path.join(targetDir, ".git")).catch(() => null)).toBeNull();
expect(await fs.lstat(path.join(targetDir, "host-only.txt")).catch(() => null)).toBeNull();
expect(await git(home, "rev-parse", "HEAD")).toBe(head);
});
it.each([false, true])("does not admit a Git fragment as an ordinary migration source (objects: %s)", async objects => {
const f = await fixture(), info = path.join(f.cwd, ".git", "info");
await fs.mkdir(info, { recursive: true });
await fs.writeFile(path.join(info, "exclude"), "/.paperclip-repositories/\n");
if (objects) await fs.mkdir(path.join(f.cwd, ".git", "objects"));
expect(await inventoryRemoteWorkspaceMigration(db, f)).toMatchObject({ eligible: false,
reasons: expect.arrayContaining(["online_workspace_source_scope_unproven"]) });
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("online_workspace_source_scope_unproven");
expect(await db.select().from(workspaceOperations)).toEqual([]);
expect(await fs.readFile(path.join(info, "exclude"), "utf8")).toBe("/.paperclip-repositories/\n");
});
it("preserves unmanaged Git import in genuine offline mode while retaining online and drain guards", async () => {
const f = await fixture(), cwd = path.join(home, "self-hosted-repository"), repoUrl = "https://example.invalid/self-hosted.git";
await fs.mkdir(cwd); await git(cwd, "init", "-q", "-b", "main");
await fs.writeFile(path.join(cwd, "saved.txt"), "self-hosted original work\n");
await git(cwd, "add", "."); await git(cwd, "commit", "-qm", "Self-hosted source");
await git(cwd, "remote", "add", "origin", repoUrl);
for (const alias of f.aliases.slice(1)) await db.update(executionWorkspaces).set({ providerType: "plugin" })
.where(eq(executionWorkspaces.id, alias.workspace.id));
await db.update(executionWorkspaces).set({ cwd, providerRef: cwd, repoUrl, branchName: "main" })
.where(eq(executionWorkspaces.id, f.workspaceId));
expect(await inventoryRemoteWorkspaceMigration(db, f)).toMatchObject({ eligible: false,
reasons: expect.arrayContaining(["online_workspace_source_scope_unproven"]) });
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("online_workspace_source_scope_unproven");
vi.stubEnv("PAPERCLIP_EPHEMERAL_MIGRATION", "true");
const [agent] = await db.insert(agents).values({ companyId: f.companyId, name: "Offline writer guard" }).returning();
const [run] = await db.insert(heartbeatRuns).values({ companyId: f.companyId, agentId: agent.id,
issueId: f.aliases[0].issue.id, status: "running", runtimeMode: "legacy" }).returning();
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true, writersStopped: true }))
.rejects.toThrow("active_execution_requires_drain");
expect(await db.select().from(workspaceOperations)).toEqual([]);
await db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date() }).where(eq(heartbeatRuns.id, run.id));
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true, writersStopped: true })).resolves.toMatchObject({ applied: true });
expect(await fs.readFile(path.join(cwd, "saved.txt"), "utf8")).toBe("self-hosted original work\n");
});
it.each(["ordinary", "Git"].flatMap(kind => ["cwd", "providerRef", "different", "alias"].map(locator => ({ kind, locator }))))(
"rejects foreign $kind ownership through $locator source locators", async ({ kind, locator }) => {
const base = await fixture(), f = kind === "Git" ? await gitFixture(base) : null;
const workspaceId = f?.child.id ?? base.workspaceId, sourcePath = f?.child.cwd ?? base.cwd;
const companyId = randomUUID(), unrelated = path.join(home, "foreign-display-root"), alias = path.join(home, "foreign-path-alias");
await fs.mkdir(unrelated); await fs.symlink(sourcePath, alias);
await db.insert(companies).values({ id: companyId, name: "Foreign source owner", issuePrefix: "FOREIGN" });
await db.insert(executionWorkspaces).values({ companyId, name: "Foreign source alias", mode: "shared_workspace",
strategyType: "project_primary", providerType: "local_fs",
cwd: locator === "cwd" ? sourcePath : locator === "providerRef" ? null : unrelated,
providerRef: locator === "cwd" ? null : locator === "alias" ? alias : sourcePath });
const input = { companyId: base.companyId, workspaceId };
expect(await inventoryRemoteWorkspaceMigration(db, input)).toMatchObject({ eligible: false });
await expect(migrateRemoteWorkspace({ db, storage, ...input, apply: true })).rejects.toThrow("remote_workspace_migration_blocked");
expect(readRemoteWorkspaceAuthority((await stored(workspaceId)).metadata)).toBeNull();
expect(await db.select().from(executionWorkspaceSnapshots)).toEqual([]);
expect(await fs.readFile(path.join(sourcePath, "saved.txt"), "utf8")).toBe("saved legacy edits\n");
});
it.each(["cwd", "providerRef"] as const)("rejects a mismatched same-company ordinary %s owner", async locator => {
const f = await fixture(), unrelated = path.join(home, "another-source");
await fs.mkdir(unrelated);
await db.update(executionWorkspaces).set({ cwd: locator === "cwd" ? f.cwd : unrelated,
providerRef: locator === "cwd" ? unrelated : f.cwd }).where(eq(executionWorkspaces.id, f.aliases[1].workspace.id));
expect(await inventoryRemoteWorkspaceMigration(db, f)).toMatchObject({ eligible: false });
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("online_workspace_source_scope_unproven");
expect(await db.select().from(workspaceOperations)).toEqual([]);
});
it.each(["providerRef", "different", "alias", "secondary-Git"] as const)("requires a proven source for the selected %s locator before fallback", async locator => {
const base = await fixture(), f = locator === "secondary-Git" ? await gitFixture(base) : null;
for (const alias of base.aliases.slice(f ? 0 : 1)) await db.update(executionWorkspaces).set({ providerType: "plugin" })
.where(eq(executionWorkspaces.id, alias.workspace.id));
const workspaceId = f?.child.id ?? base.workspaceId, sourcePath = f?.child.cwd ?? base.cwd;
const unrelated = path.join(home, "selected-display-root"); await fs.mkdir(unrelated);
const alias = path.join(home, "selected-path-alias"); await fs.symlink(sourcePath, alias);
await db.update(executionWorkspaces).set({ cwd: locator === "providerRef" ? null : unrelated,
providerRef: locator === "alias" ? alias : sourcePath, repoUrl: null, branchName: null,
strategyType: "project_primary" }).where(eq(executionWorkspaces.id, workspaceId));
const input = { companyId: base.companyId, workspaceId };
expect(await inventoryRemoteWorkspaceMigration(db, input)).toMatchObject({ eligible: false,
reasons: expect.arrayContaining(["online_workspace_source_scope_unproven"]) });
await expect(migrateRemoteWorkspace({ db, storage, ...input, apply: true })).rejects.toThrow("online_workspace_source_scope_unproven");
expect(await db.select().from(workspaceOperations)).toEqual([]);
expect(await fs.readFile(path.join(sourcePath, "saved.txt"), "utf8")).toBe("saved legacy edits\n");
});
it.each(["cwd", "providerRef", "different", "alias"] as const)("reserves every %s source locator before the actual lifecycle owner", async locator => {
const f = await fixture(), unrelated = path.join(home, "lifecycle-display-root"), alias = path.join(home, "lifecycle-path-alias");
await fs.mkdir(unrelated); await fs.symlink(f.cwd, alias);
await db.update(executionWorkspaces).set({ projectId: null,
cwd: locator === "cwd" ? f.cwd : locator === "providerRef" ? null : unrelated,
providerRef: locator === "cwd" ? null : locator === "alias" ? alias : f.cwd }).where(eq(executionWorkspaces.id, f.workspaceId));
let entered!: () => void, release!: () => void, mutated = false;
const started = new Promise<void>(resolve => { entered = resolve; });
const pending = new Promise<void>(resolve => { release = resolve; });
const owner = withWorkspaceSourceLocks(db, { companyId: f.companyId, projectIds: [f.projectId] }, async () => { entered(); await pending; });
await started;
const lifecycle = withExecutionWorkspaceLifecycleLock(db, f.workspaceId, async () => { mutated = true; });
try {
await vi.waitFor(async () => {
const waiting = await db.execute(sql`select count(*)::int n from pg_stat_activity where datname=current_database() and wait_event='advisory'`);
expect(waiting[0].n).toBeGreaterThan(0);
});
expect(mutated).toBe(false);
const key = `execution_workspace_lifecycle:${f.workspaceId}`;
const locks = await db.execute(sql`select count(*)::int n from pg_locks where locktype='advisory' and granted
and classid=((hashtextextended(${key},0)>>32)&4294967295)::oid and objid=(hashtextextended(${key},0)&4294967295)::oid`);
expect(locks[0].n).toBe(0);
} finally { release(); await owner; await lifecycle; }
expect(mutated).toBe(true);
expect(await fs.readFile(path.join(f.cwd, "saved.txt"), "utf8")).toBe("saved legacy edits\n");
});
function uploadBarrier() {
let entered!: () => void, release!: () => void;
const started = new Promise<void>(resolve => { entered = resolve; });
@@ -192,11 +338,13 @@ describe.skipIf(!support.supported)("online ordinary physical source fence", ()
repoUrl: null, repoRef: null, strategy: "project_primary", branchName: null, branchCreatedByRuntime: false,
worktreePath: null, warnings: [], created: false } })).toEqual([]);
});
it.each(["workspace", "configured"] as const)("fences project command provisioning and its running %s CWD without an execution workspace ID", async location => {
it.each(["workspace", "configured", "symlink"] as const)("fences project command provisioning and its running %s CWD without an execution workspace ID", async location => {
const f = await fixture();
const startProjectId = location === "configured" ? randomUUID() : f.projectId;
if (startProjectId !== f.projectId) await db.insert(projects).values({ id: startProjectId, companyId: f.companyId, name: "Service owner" });
const startCwd = resolveManagedProjectWorkspaceDir({ companyId: f.companyId, projectId: startProjectId });
const startCwd = location === "symlink" ? path.join(home, "configured-service-source")
: resolveManagedProjectWorkspaceDir({ companyId: f.companyId, projectId: startProjectId });
if (location === "symlink") await fs.symlink(f.cwd, startCwd);
await fs.mkdir(startCwd, { recursive: true });
const [agent] = await db.insert(agents).values({ companyId: f.companyId, name: "Project command owner" }).returning();
const [projectWorkspace] = await db.insert(projectWorkspaces).values({ companyId: f.companyId, projectId: startProjectId,
@@ -360,6 +508,146 @@ describe.skipIf(!support.supported)("online ordinary physical source fence", ()
status: "active", evidence: { runId } });
return { ...f, run, owner, lease, warning };
}
it("imports a shared Git child while a terminal sibling recovery and unrelated remote execution remain intact", async () => {
const recovery = await recoveryFixture(), f = await gitFixture(recovery);
const unrelated = await remoteWriter(f.companyId, recovery.run.agentId);
await fs.writeFile(path.join(f.child.cwd!, "saved.txt"), "staged child work\n");
await git(f.child.cwd!, "add", ".");
await fs.writeFile(path.join(f.child.cwd!, "saved.txt"), "unstaged child work\n");
const input = { companyId: f.companyId, workspaceId: f.child.id };
expect(await inventoryRemoteWorkspaceMigration(db, input)).toMatchObject({ eligible: true });
await expect(migrateRemoteWorkspace({ db, storage, ...input, apply: true })).resolves.toMatchObject({ applied: true });
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, unrelated.id)))[0]).toEqual(unrelated);
expect((await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, recovery.run.id)))[0]).toEqual(recovery.run);
expect((await db.select().from(nativeRunFinalizations).where(eq(nativeRunFinalizations.runId, recovery.run.id)))[0]).toEqual(recovery.owner);
expect((await db.select().from(issueRecoveryActions).where(eq(issueRecoveryActions.id, recovery.warning.id)))[0]).toEqual(recovery.warning);
expect((await db.select().from(environmentLeases).where(eq(environmentLeases.id, recovery.lease.id)))[0]).toEqual(recovery.lease);
const authority = readRemoteWorkspaceAuthority((await stored(f.child.id)).metadata)!;
const reference = await getWorkspaceSnapshot(db, { companyId: f.companyId, workspaceId: f.child.id, snapshotId: authority.snapshotId! });
const archivePath = path.join(home, "git-child.pcws"), targetDir = path.join(home, "git-child-restored");
await loadWorkspaceSnapshotArchive({ storage, reference: reference!, targetPath: archivePath });
expect(await fs.readFile(path.join(f.cwd, "saved.txt"), "utf8")).toBe("saved legacy edits\n");
await fs.rm(f.cwd, { recursive: true });
await restoreRemoteWorkspaceArchive({ archivePath, targetDir, expectedManifestDigest: reference!.manifestDigest });
expect(await git(targetDir, "show", ":saved.txt")).toBe("staged child work");
expect(await fs.readFile(path.join(targetDir, "saved.txt"), "utf8")).toBe("unstaged child work\n");
await git(targetDir, "fsck", "--full", "--no-dangling");
});
it("takes the original private Git store owner before reserving an import journal", async () => {
const f = await fixture();
const [seed] = await db.insert(issues).values({ companyId: f.companyId, title: "Private seed owner" }).returning();
const source = path.join(resolvePaperclipInstanceRoot(), "isolated-repository-sources", f.companyId, seed.id, "checkout");
await fs.mkdir(source, { recursive: true }); await git(source, "init", "-q", "-b", "main");
await fs.writeFile(path.join(source, "saved.txt"), "private seed work\n");
await git(source, "add", "."); await git(source, "commit", "-qm", "Private seed");
const repoUrl = "https://example.invalid/private-seed.git";
await git(source, "remote", "add", "origin", repoUrl);
const [issue] = await db.insert(issues).values({ companyId: f.companyId, title: "Linked task" }).returning();
const cwd = path.join(f.cwd, ".paperclip", "worktrees", "private-child");
await git(source, "worktree", "add", "-qb", "private-child", cwd);
const [workspace] = await db.insert(executionWorkspaces).values({ companyId: f.companyId, projectId: f.projectId,
sourceIssueId: issue.id, name: "Private linked source", mode: "isolated_workspace", strategyType: "git_worktree",
providerType: "git_worktree", cwd, providerRef: cwd, repoUrl, branchName: "private-child" }).returning();
let entered!: () => void, release!: () => void;
const started = new Promise<void>(resolve => { entered = resolve; });
const pending = new Promise<void>(resolve => { release = resolve; });
const owner = withIsolatedRepositorySourceLock(db, f.companyId, seed.id, async () => { entered(); await pending; });
await started;
const input = { companyId: f.companyId, workspaceId: workspace.id };
try {
expect(await inventoryRemoteWorkspaceMigration(db, input)).toMatchObject({ eligible: true });
await expect(migrateRemoteWorkspace({ db, storage, ...input, apply: true })).rejects.toBeInstanceOf(IsolatedRepositorySourceBusy);
expect(await db.select().from(workspaceOperations)).toEqual([]);
} finally { release(); await owner; }
await expect(migrateRemoteWorkspace({ db, storage, ...input, apply: true })).resolves.toMatchObject({ applied: true });
expect(await fs.readFile(path.join(source, "saved.txt"), "utf8")).toBe("private seed work\n");
});
it.each(["coordinator", "allocation", "locator", "git-metadata"] as const)("rechecks the shared Git companion %s before publication", async change => {
const recovery = await recoveryFixture(), f = await gitFixture(recovery), barrier = uploadBarrier();
const migration = migrateRemoteWorkspace({ db, storage, companyId: f.companyId, workspaceId: f.child.id, apply: true })
.then(value => ({ value }), error => ({ error }));
await Promise.race([barrier.started, migration.then(result => { if ("error" in result) throw result.error; throw new Error("capture did not upload"); })]);
try {
if (change === "coordinator") await db.update(nativeRunFinalizations).set({ leaseOwner: randomUUID(), leaseExpiresAt: new Date(Date.now() - 1000) })
.where(eq(nativeRunFinalizations.runId, recovery.run.id));
if (change === "allocation") await db.update(environmentLeases).set({ metadata: { remoteCwd: "/changed-companion-source" } })
.where(eq(environmentLeases.id, recovery.lease.id));
if (change === "locator") await db.update(executionWorkspaces).set({ providerRef: path.join(home, "unproven-companion") })
.where(eq(executionWorkspaces.id, f.aliases[1].workspace.id));
if (change === "git-metadata") {
const administration = await git(f.child.cwd!, "rev-parse", "--absolute-git-dir");
await fs.writeFile(path.join(administration, "commondir"), path.join(f.cwd, ".git") + "\n");
}
} finally { barrier.release(); }
const result = await migration;
expect(result).toHaveProperty("error");
expect("error" in result ? String(result.error) : "").toContain("online_workspace_source_scope_changed");
expect(readRemoteWorkspaceAuthority((await stored(f.child.id)).metadata)).toBeNull();
expect((await db.select().from(executionWorkspaceSnapshots))[0].status).toBe("uploaded");
expect(await fs.readFile(path.join(f.child.cwd!, "saved.txt"), "utf8")).toBe("saved legacy edits\n");
});
it("retains an imported nested child's private cache owner through parent source verification", async () => {
const f = await gitFixture(), sourceIssueId = randomUUID();
const source = path.join(resolvePaperclipInstanceRoot(), "isolated-repository-sources", f.companyId, sourceIssueId, "checkout");
await git(f.cwd, "worktree", "remove", f.child.cwd!);
await fs.mkdir(path.dirname(source), { recursive: true });
await git(home, "clone", "--no-hardlinks", f.cwd, source);
await git(source, "remote", "set-url", "origin", f.repoUrl);
await git(source, "worktree", "add", "-qb", "private-child", f.child.cwd!);
await db.update(executionWorkspaces).set({ branchName: "private-child" }).where(eq(executionWorkspaces.id, f.child.id));
await migrateRemoteWorkspace({ db, storage, companyId: f.companyId, workspaceId: f.child.id, apply: true });
expect((await stored(f.child.id)).providerType).toBe("cloud_sandbox");
let entered!: () => void, release!: () => void;
const started = new Promise<void>(resolve => { entered = resolve; });
const pending = new Promise<void>(resolve => { release = resolve; });
const owner = withWorkspaceSourceLocks(db, { companyId: f.companyId, issueIds: [sourceIssueId] }, async () => {
entered(); await pending;
});
await started;
try {
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toBeInstanceOf(IsolatedRepositorySourceBusy);
expect(await db.select().from(workspaceOperations).where(eq(workspaceOperations.executionWorkspaceId, f.workspaceId))).toEqual([]);
} finally { release(); await owner; }
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).resolves.toMatchObject({ applied: true });
const authority = readRemoteWorkspaceAuthority((await stored(f.workspaceId)).metadata)!;
const [snapshot] = await db.select().from(executionWorkspaceSnapshots).where(eq(executionWorkspaceSnapshots.id, authority.snapshotId!));
expect(snapshot.metadata?.importedWorkspaceChildren).toEqual([expect.objectContaining({ workspaceId: f.child.id })]);
expect(await fs.readFile(path.join(source, "saved.txt"), "utf8")).toBe("saved legacy edits\n");
expect(await fs.readFile(path.join(f.child.cwd!, "saved.txt"), "utf8")).toBe("saved legacy edits\n");
});
it.each(["missing", "deleted"] as const)("rejects a Git parent with %s imported-child snapshot coverage", async state => {
const f = await gitFixture();
await migrateRemoteWorkspace({ db, storage, companyId: f.companyId, workspaceId: f.child.id, apply: true });
const authority = readRemoteWorkspaceAuthority((await stored(f.child.id)).metadata)!;
if (state === "missing") await db.delete(executionWorkspaceSnapshots).where(eq(executionWorkspaceSnapshots.id, authority.snapshotId!));
else await db.update(executionWorkspaceSnapshots).set({ status: "deleted" }).where(eq(executionWorkspaceSnapshots.id, authority.snapshotId!));
expect(await inventoryRemoteWorkspaceMigration(db, f)).toMatchObject({ eligible: false,
reasons: expect.arrayContaining(["overlapping_workspace_requires_maintenance"]) });
await expect(migrateRemoteWorkspace({ db, storage, ...f, apply: true })).rejects.toThrow("overlapping_workspace_requires_maintenance");
expect(readRemoteWorkspaceAuthority((await stored(f.workspaceId)).metadata)).toBeNull();
expect(await db.select().from(workspaceOperations).where(eq(workspaceOperations.executionWorkspaceId, f.workspaceId))).toEqual([]);
expect(await fs.readFile(path.join(f.child.cwd!, "saved.txt"), "utf8")).toBe("saved legacy edits\n");
});
it.each([false, true])("requires exact imported-child coverage for an online shared Git parent (%s)", async changed => {
const f = await gitFixture();
expect(await inventoryRemoteWorkspaceMigration(db, f)).toMatchObject({ eligible: false,
reasons: expect.arrayContaining(["nested_workspace_import_required"]) });
await migrateRemoteWorkspace({ db, storage, companyId: f.companyId, workspaceId: f.child.id, apply: true });
if (changed) await fs.writeFile(path.join(f.child.cwd!, "new-work.txt"), "changed after child import\n");
expect(await inventoryRemoteWorkspaceMigration(db, f)).toMatchObject({ eligible: true });
const migration = migrateRemoteWorkspace({ db, storage, ...f, apply: true });
if (changed) {
await expect(migration).rejects.toThrow("nested_workspace_source_changed");
expect(readRemoteWorkspaceAuthority((await stored(f.workspaceId)).metadata)).toBeNull();
expect(await fs.readFile(path.join(f.child.cwd!, "new-work.txt"), "utf8")).toBe("changed after child import\n");
} else {
await expect(migration).resolves.toMatchObject({ applied: true });
const authority = readRemoteWorkspaceAuthority((await stored(f.workspaceId)).metadata)!;
const [snapshot] = await db.select().from(executionWorkspaceSnapshots).where(eq(executionWorkspaceSnapshots.id, authority.snapshotId!));
expect(snapshot.metadata?.importedWorkspaceChildren).toEqual([expect.objectContaining({ workspaceId: f.child.id })]);
}
});
it("imports a terminal prepared source while retaining its original provider and finalization obligations", async () => {
const f = await recoveryFixture(), put = storage.putFile.bind(storage);
const unrelated = await remoteWriter(f.companyId, f.run.agentId);
@@ -81,6 +81,33 @@ describe("portable remote workspace archives", () => {
expect(await sourceBytes(cwd)).toBe(before);
await expect(capture(cwd)).resolves.toMatchObject({ manifestDigest: expect.any(String) });
});
it.each(["linked", "borrowed"].flatMap(kind => ["local", "serialized"].map(mode => ({ kind, mode }))))(
"enforces owned Git source roots for $kind $mode capture without changing archive bytes", async ({ kind, mode }) => {
const base = await source(); await git(base, "init", "-q", "-b", "main");
await fs.writeFile(path.join(base, "work.txt"), "base work\n"); await git(base, "add", "."); await git(base, "commit", "-qm", "base");
const cwd = path.join(root, "selected");
if (kind === "linked") await git(base, "worktree", "add", "-qb", "selected", cwd);
else await git(root, "clone", "-q", "--shared", base, cwd);
await fs.writeFile(path.join(cwd, "work.txt"), "staged work\n"); await git(cwd, "add", ".");
await fs.writeFile(path.join(cwd, "work.txt"), "unstaged work\n");
const baseline = await capture(cwd), archivePath = path.join(root, "owned.pcws");
const prefix = remoteWorkspaceArchiveScript().split("const [kind, first, second, digest]")[0];
const fenced = async (roots: string[]) => {
const input = { kind: "capture", cwd, archivePath, gitSourceRoots: roots };
return mode === "local" ? captureRemoteWorkspaceDirectory(input)
: JSON.parse((await execute(process.execPath, ["--eval", `${prefix}archive(${JSON.stringify(input)}).then(value=>process.stdout.write(JSON.stringify(value))).catch(error=>{process.stderr.write(error.message);process.exitCode=1;});`], { env })).stdout);
};
await expect(fenced([await fs.realpath(cwd)])).rejects.toThrow("online_source_git_requires_shared_fence");
await expect(fs.access(archivePath)).rejects.toMatchObject({ code: "ENOENT" });
const snapshot = { archivePath, ...await fenced([await fs.realpath(root)]) };
expect(snapshot).toMatchObject({ manifestDigest: baseline.manifestDigest, sha256: baseline.sha256, byteSize: baseline.byteSize });
expect(await fs.readFile(archivePath)).toEqual(await fs.readFile(baseline.archivePath));
await fs.rm(base, { recursive: true }); await fs.rm(cwd, { recursive: true });
const target = await restore(snapshot);
expect(await git(target, "show", ":work.txt")).toBe("staged work");
expect(await fs.readFile(path.join(target, "work.txt"), "utf8")).toBe("unstaged work\n");
await git(target, "fsck", "--full", "--no-dangling");
});
it("requires a shared Git fence for a nested bare repository borrowing external objects", async () => {
const cwd = await source(), upstream = path.join(root, "upstream"), repository = path.join(cwd, "bare.git");
await fs.mkdir(upstream);
@@ -92,8 +119,10 @@ describe("portable remote workspace archives", () => {
expect(await fs.readFile(path.join(repository, "objects", "info", "alternates"), "utf8"))
.toContain(path.join(upstream, ".git", "objects"));
const before = await sourceBytes(cwd), external = await sourceBytes(upstream);
await expect(captureRemoteWorkspaceDirectory({ cwd, archivePath: path.join(root, "bare.pcws"), requireNoGit: true }))
.rejects.toThrow("online_source_git_requires_shared_fence");
for (const options of [{ requireNoGit: true }, { gitSourceRoots: [await fs.realpath(root)] }]) {
await expect(captureRemoteWorkspaceDirectory({ cwd, archivePath: path.join(root, "bare.pcws"), ...options }))
.rejects.toThrow("online_source_git_requires_shared_fence");
}
expect(await sourceBytes(cwd)).toBe(before);
expect(await sourceBytes(upstream)).toBe(external);
});
@@ -11,6 +11,7 @@ async function workspaceArchive(input: {
expectedManifestDigest?: string;
exclude?: string[];
requireNoGit?: boolean;
gitSourceRoots?: string[];
signal?: AbortSignal;
maxEntries?: number;
maxManifestBytes?: number;
@@ -273,6 +274,46 @@ async function workspaceArchive(input: {
const output = await fs.open(destination, "w", 0o600);
try { await readStableFile(source, bytes => output.writeFile(bytes)); } finally { await output.close(); }
};
const assertGitSourceFence = async (repo: string) => {
if (!input.gitSourceRoots) return null;
if (!input.gitSourceRoots.length || input.gitSourceRoots.length > 384
|| input.gitSourceRoots.some(root => !path.isAbsolute(root) || path.resolve(root) !== root)) fail("online_source_git_requires_shared_fence");
const directory = async (target: string) => {
const actual = await fs.realpath(target), entry = await fs.lstat(target, { bigint: true });
if (!entry.isDirectory() || entry.isSymbolicLink()
|| !input.gitSourceRoots!.some(root => actual === root || actual.startsWith(`${root}${path.sep}`))) {
fail("online_source_git_requires_shared_fence");
}
return { path: actual, device: String(entry.dev), inode: String(entry.ino) };
};
const gitDirectory = await directory((await git(repo, ["rev-parse", "--absolute-git-dir"])).trim());
const commonDirectory = await directory((await git(repo, ["rev-parse", "--path-format=absolute", "--git-common-dir"])).trim());
const stores: unknown[] = [], seen = new Set<string>();
const visit = async (target: string, depth: number): Promise<void> => {
if (depth > 4 || seen.size >= 32) fail("online_source_git_requires_shared_fence");
const current = await directory(target);
if (seen.has(current.path)) return;
seen.add(current.path);
if (await fs.lstat(path.join(current.path, "info", "http-alternates")).then(() => true,
(error: NodeJS.ErrnoException) => { if (error.code !== "ENOENT") throw error; return false; })) fail("online_source_git_requires_shared_fence");
const alternatePath = path.join(current.path, "info", "alternates");
const entry = await fs.lstat(alternatePath).catch((error: NodeJS.ErrnoException) => { if (error.code !== "ENOENT") throw error; return null; });
if (!entry) { stores.push(current); return; }
if (!entry.isFile() || entry.isSymbolicLink() || entry.size > 64 * 1024) fail("online_source_git_requires_shared_fence");
const chunks: Buffer[] = []; let size = 0;
const saved = await readStableFile(alternatePath, async bytes => {
size += bytes.length;
if (size > 64 * 1024) fail("online_source_git_requires_shared_fence");
chunks.push(bytes);
});
const values = Buffer.concat(chunks).toString("utf8").split("\n").filter(Boolean);
if (values.length > 32 || values.some(value => /[\0\r]/.test(value) || value.startsWith('"'))) fail("online_source_git_requires_shared_fence");
stores.push({ ...current, alternatesSha256: saved.sha256 });
for (const value of values) await visit(path.resolve(current.path, value), depth + 1);
};
await visit(path.join(commonDirectory.path, "objects"), 0);
return { gitDirectory, commonDirectory, stores };
};
const gitStatePath = async (repo: string, name: string) => (await git(repo, ["rev-parse", "--path-format=absolute", "--git-path", name])).trim();
const gitStateHash = async (repo: string, name: string) => {
const file = await gitStatePath(repo, name);
@@ -325,7 +366,7 @@ async function workspaceArchive(input: {
const directoryStat = await fs.lstat(directory);
if (!directoryStat.isDirectory() || directoryStat.isSymbolicLink() || await fs.realpath(directory) !== directory) fail("changed");
const names = (await fs.readdir(directory)).sort();
if (input.requireNoGit && (names.includes("commondir") || names.includes("gitdir")
if (skipGit && (input.requireNoGit || input.gitSourceRoots) && (names.includes("commondir") || names.includes("gitdir")
|| names.includes("objects") && names.some(name => ["HEAD", "refs", "packed-refs"].includes(name))
|| names.includes("HEAD") && names.includes("config")
|| path.basename(directory) === "info" && names.some(name => ["alternates", "http-alternates"].includes(name)))) {
@@ -411,13 +452,14 @@ async function workspaceArchive(input: {
const repositories = [...new Set(directories)];
const gitSignatures = new Map<string, string>();
const signature = async (repo: string, settings?: Awaited<ReturnType<typeof portableGitSettings>>) => {
const fence = await assertGitSourceFence(repo);
const operation = await assertGitOperationComplete(repo);
const index = (await git(repo, ["rev-parse", "--path-format=absolute", "--git-path", "index"])).trim();
const indexHash = await hashFile(index).catch((error: NodeJS.ErrnoException) => error.code === "ENOENT" ? null : Promise.reject(error));
const config = settings ?? await portableGitSettings(repo), shallow = await gitStateHash(repo, "shallow");
const partial = config.some(setting => /^remote\..*\.promisor$/u.test(setting.key) && setting.value === "true");
return JSON.stringify([await git(repo, ["status", "--porcelain=v2", "-z", "--untracked-files=no"]), await git(repo, ["for-each-ref", "--format=%(objectname) %(refname)"]), await git(repo, ["symbolic-ref", "-q", "HEAD"]).catch(() => git(repo, ["rev-parse", "HEAD"])), indexHash, config,
shallow, await gitStateHash(repo, "info/sparse-checkout"), shallow || partial ? (await gitObjectFiles(repo)).files : null, operation]);
shallow, await gitStateHash(repo, "info/sparse-checkout"), shallow || partial ? (await gitObjectFiles(repo)).files : null, operation, fence]);
};
for (let index = 0; index < repositories.length; index++) {
const relative = repositories[index]!, repo = path.join(root, relative), normalized = path.join(scratch, `repo-${index}.git`);
@@ -587,7 +629,7 @@ async function workspaceArchive(input: {
}
}
export function captureRemoteWorkspaceDirectory(input: { cwd: string; archivePath: string; manifestPath?: string; exclude?: string[]; requireNoGit?: boolean; signal?: AbortSignal }) {
export function captureRemoteWorkspaceDirectory(input: { cwd: string; archivePath: string; manifestPath?: string; exclude?: string[]; requireNoGit?: boolean; gitSourceRoots?: string[]; signal?: AbortSignal }) {
return workspaceArchive({ ...input, kind: "capture" });
}
@@ -0,0 +1,143 @@
import { randomUUID } from "node:crypto";
import { execFile } from "node:child_process";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { promisify } from "node:util";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { readWorkspaceGitSourceRoots } from "./workspace-git-source-roots.js";
import { workspaceSourceIdentities } from "./workspace-source-lock.js";
import { localWorkspaceSourcePaths } from "./execution-workspace-source-roots.js";
import { resolveManagedProjectWorkspaceDir, resolvePaperclipInstanceRoot } from "../home-paths.js";
const execute = promisify(execFile);
let root: string;
const git = async (cwd: string, ...args: string[]) => execute("git", ["-c", `core.hooksPath=${os.devNull}`, "-C", cwd, ...args], {
env: { ...process.env, GIT_CONFIG_GLOBAL: os.devNull, GIT_CONFIG_NOSYSTEM: "1", GIT_TERMINAL_PROMPT: "0",
GIT_AUTHOR_NAME: "Source fixture", GIT_AUTHOR_EMAIL: "source@example.test", GIT_COMMITTER_NAME: "Source fixture", GIT_COMMITTER_EMAIL: "source@example.test" },
});
beforeEach(async () => { root = await fs.realpath(await fs.mkdtemp(path.join(os.tmpdir(), "pc-git-source-"))); });
afterEach(async () => { vi.unstubAllEnvs(); await fs.rm(root, { recursive: true, force: true }); });
async function repository(name = "repository") {
const cwd = path.join(root, name);
await fs.mkdir(cwd, { recursive: true }); await git(cwd, "init", "--initial-branch=main");
await fs.writeFile(path.join(cwd, "saved.txt"), "original work\n");
await git(cwd, "add", "."); await git(cwd, "commit", "-m", "Original");
return cwd;
}
describe("Git physical source roots", () => {
it("distinguishes an ignore-only fragment from Git source metadata", async () => {
const cwd = path.join(root, "plain-task"), marker = path.join(cwd, ".git");
await fs.mkdir(path.join(marker, "info"), { recursive: true });
await fs.writeFile(path.join(marker, "info", "exclude"), "/.paperclip-repositories/\n");
expect(await readWorkspaceGitSourceRoots(cwd)).toBeNull();
for (const name of ["HEAD", "config", "commondir"]) {
await fs.writeFile(path.join(marker, name), name === "commondir" ? ".\n" : "partial Git state\n");
await expect(readWorkspaceGitSourceRoots(cwd)).rejects.toThrow("workspace_git_source_unproven");
await fs.rm(path.join(marker, name));
}
await fs.mkdir(path.join(marker, "objects"));
expect(await readWorkspaceGitSourceRoots(cwd)).toMatchObject({ objectDirectories: [{ path: path.join(marker, "objects") }] });
});
it.each([
{ cwd: "/source/a", providerRef: null, paths: ["/source/a"] },
{ cwd: null, providerRef: "/source/a", paths: ["/source/a"] },
{ cwd: "/source/a", providerRef: "/source/b", paths: ["/source/a", "/source/b"] },
{ cwd: "/source/a", providerRef: "/source/a", paths: ["/source/a"] },
])("retains every distinct local locator $cwd $providerRef", ({ cwd, providerRef, paths }) => {
expect(localWorkspaceSourcePaths({ cwd, providerRef })).toEqual(paths);
});
it.each([false, true])("maps a configured directory symlink before strict Git inspection (%s)", async isGit => {
vi.stubEnv("PAPERCLIP_HOME", root);
const companyId = randomUUID(), projectId = randomUUID();
const cwd = resolveManagedProjectWorkspaceDir({ companyId, projectId });
await fs.mkdir(cwd, { recursive: true });
if (isGit) await git(cwd, "init", "-q");
const alias = path.join(root, "configured-source");
await fs.symlink(cwd, alias);
expect(await workspaceSourceIdentities(companyId, alias)).toMatchObject({ projectIds: [projectId], unownedPaths: [] });
expect(await workspaceSourceIdentities(companyId, alias)).toEqual(await workspaceSourceIdentities(companyId, cwd));
await expect(readWorkspaceGitSourceRoots(alias)).rejects.toThrow("workspace_git_source_unproven");
});
it("binds actual linked worktree administration and the full shared object-store root", async () => {
const source = await repository(), cwd = path.join(root, "task worktree");
await git(source, "worktree", "add", "-b", "task", cwd);
const actual = await readWorkspaceGitSourceRoots(cwd);
expect(actual).toMatchObject({ root: cwd, commonDirectory: { path: path.join(source, ".git") },
objectDirectories: [{ path: path.join(source, ".git", "objects") }] });
expect(actual!.gitDirectory.path.startsWith(path.join(source, ".git", "worktrees") + path.sep)).toBe(true);
expect(actual!.metadata.map(file => path.basename(file.path)).sort()).toEqual([".git", "commondir"]);
expect(await readWorkspaceGitSourceRoots(cwd)).toEqual(actual);
expect(await fs.readFile(path.join(cwd, "saved.txt"), "utf8")).toBe("original work\n");
});
it("retains borrowed object stores instead of treating HEAD as the complete source", async () => {
const source = await repository(), cwd = path.join(root, "borrower");
await git(root, "clone", "--shared", source, cwd);
const actual = await readWorkspaceGitSourceRoots(cwd);
expect(actual!.objectDirectories.map(directory => directory.path).sort()).toEqual([
path.join(cwd, ".git", "objects"), path.join(source, ".git", "objects"),
].sort());
expect(actual!.metadata.some(file => file.path.endsWith("info/alternates"))).toBe(true);
});
it.each(["symlink", "fifo", "oversized", "http-alternate", "bare"] as const)("refuses unproven %s metadata", async kind => {
const cwd = await repository();
if (kind === "symlink") {
await fs.rename(path.join(cwd, ".git"), path.join(cwd, "administration"));
await fs.symlink("administration", path.join(cwd, ".git"));
}
if (kind === "fifo" || kind === "oversized") {
await fs.rm(path.join(cwd, ".git"), { recursive: true });
if (kind === "fifo") await execute("mkfifo", [path.join(cwd, ".git")]);
else await fs.writeFile(path.join(cwd, ".git"), "x".repeat(4097));
}
if (kind === "http-alternate") await fs.writeFile(path.join(cwd, ".git", "objects", "info", "http-alternates"), "https://example.invalid/objects\n");
if (kind === "bare") { await fs.rm(cwd, { recursive: true }); await git(root, "init", "--bare", cwd); }
await expect(readWorkspaceGitSourceRoots(cwd)).rejects.toThrow("workspace_git_source_unproven");
});
it("changes the proof when a valid pointer selects another administration directory", async () => {
const source = await repository(), first = path.join(root, "first"), second = path.join(root, "second");
await git(source, "worktree", "add", "-b", "first", first);
await git(source, "worktree", "add", "-b", "second", second);
const before = await readWorkspaceGitSourceRoots(first);
await fs.writeFile(path.join(first, ".git"), await fs.readFile(path.join(second, ".git")));
const after = await readWorkspaceGitSourceRoots(first);
expect(after!.gitDirectory).not.toEqual(before!.gitDirectory);
expect(after!.fingerprint).not.toBe(before!.fingerprint);
});
it.each([false, true])("maps a managed linked source and its service subdirectory to both physical owners (%s)", async nested => {
vi.stubEnv("PAPERCLIP_HOME", root);
const companyId = randomUUID(), projectId = randomUUID(), sourceIssueId = randomUUID();
const source = await repository(path.relative(root, path.join(resolvePaperclipInstanceRoot(),
"isolated-repository-sources", companyId, sourceIssueId, "checkout")));
const cwd = path.join(resolveManagedProjectWorkspaceDir({ companyId, projectId }), ".paperclip", "worktrees", "task");
await fs.mkdir(path.dirname(cwd), { recursive: true });
await git(source, "worktree", "add", "-b", "task", cwd);
const target = nested ? path.join(cwd, "service") : cwd;
if (nested) await fs.mkdir(target);
const identities = await workspaceSourceIdentities(companyId, target);
expect(identities).toMatchObject({ projectIds: [projectId], issueIds: [sourceIssueId], unownedPaths: [] });
expect(identities.git!.commonDirectory.path).toBe(path.join(source, ".git"));
});
it("rejects a physical Git source owned by another company", async () => {
vi.stubEnv("PAPERCLIP_HOME", root);
const companyId = randomUUID(), foreignCompanyId = randomUUID();
const source = await repository(path.relative(root, path.join(resolvePaperclipInstanceRoot(),
"isolated-repository-sources", foreignCompanyId, randomUUID(), "checkout")));
const cwd = resolveManagedProjectWorkspaceDir({ companyId, projectId: randomUUID() });
await fs.mkdir(path.dirname(cwd), { recursive: true });
await git(source, "worktree", "add", "-b", "task", cwd);
await expect(workspaceSourceIdentities(companyId, cwd)).rejects.toThrow("workspace_source_company_mismatch");
});
it("keeps an external borrowed store explicitly unowned", async () => {
vi.stubEnv("PAPERCLIP_HOME", root);
const companyId = randomUUID(), projectId = randomUUID(), source = await repository("external");
const cwd = resolveManagedProjectWorkspaceDir({ companyId, projectId });
await fs.mkdir(path.dirname(cwd), { recursive: true });
await git(root, "clone", "--shared", source, cwd);
expect(await workspaceSourceIdentities(companyId, cwd)).toMatchObject({ projectIds: [projectId],
unownedPaths: [path.join(source, ".git", "objects")] });
});
it("leaves absent and ordinary directories outside the Git proof", async () => {
expect(await readWorkspaceGitSourceRoots(path.join(root, "absent"))).toBeNull();
expect(await readWorkspaceGitSourceRoots(root)).toBeNull();
});
});
@@ -0,0 +1,115 @@
import fs from "node:fs/promises";
import { constants } from "node:fs";
import path from "node:path";
import { createHash } from "node:crypto";
type Identity = { path: string; device: string; inode: string };
type Metadata = Identity & { sha256: string; bytes: number };
export type WorkspaceGitSourceRoots = {
root: string; gitDirectory: Identity; commonDirectory: Identity;
objectDirectories: Identity[]; metadata: Metadata[]; fingerprint: string;
};
const invalid = () => new Error("workspace_git_source_unproven");
const sha256 = (value: string | Buffer) => createHash("sha256").update(value).digest("hex");
const entry = (target: string) => fs.lstat(target, { bigint: true }).catch((error: NodeJS.ErrnoException) => {
if (error.code === "ENOENT") return null;
throw error;
});
async function directoryIdentity(target: string): Promise<Identity> {
const stat = await entry(target);
if (!stat?.isDirectory() || stat.isSymbolicLink()) throw invalid();
const actual = await fs.realpath(target);
return { path: actual, device: String(stat.dev), inode: String(stat.ino) };
}
async function readMetadata(target: string, limit: number): Promise<{ metadata: Metadata; text: string } | null> {
const before = await entry(target);
if (!before) return null;
if (!before.isFile() || before.isSymbolicLink() || before.nlink !== 1n || before.size > BigInt(limit)) throw invalid();
const handle = await fs.open(target, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK);
try {
const opened = await handle.stat({ bigint: true });
if (opened.dev !== before.dev || opened.ino !== before.ino || !opened.isFile()) throw invalid();
const buffer = Buffer.alloc(limit + 1), { bytesRead } = await handle.read(buffer, 0, buffer.length, 0);
const after = await handle.stat({ bigint: true }), current = await entry(target);
if (!current || bytesRead !== Number(before.size) || bytesRead > limit
|| [opened, after, current].some(stat => stat.dev !== before.dev || stat.ino !== before.ino || stat.size !== before.size
|| stat.mtimeNs !== before.mtimeNs || stat.ctimeNs !== before.ctimeNs)) throw invalid();
const bytes = buffer.subarray(0, bytesRead);
return { metadata: { path: target, device: String(before.dev), inode: String(before.ino), bytes: bytesRead, sha256: sha256(bytes) },
text: bytes.toString("utf8") };
} finally { await handle.close(); }
}
function metadataPath(text: string): string {
const value = text.replace(/\r?\n$/, "");
if (!value || /[\0\r\n]/.test(value)) throw invalid();
return value;
}
async function onlyEntry(directory: string, name: string): Promise<boolean> {
let found = false;
for await (const child of await fs.opendir(directory)) {
if (found || child.name !== name) return false;
found = true;
}
return found;
}
export async function readWorkspaceGitSourceRoots(cwd: string, options: { ancestors?: boolean } = {}): Promise<WorkspaceGitSourceRoots | null> {
let root = path.resolve(cwd), marker = "", markerEntry: Awaited<ReturnType<typeof entry>>;
for (let depth = 0; ; depth++) {
if (depth > 32) throw invalid();
const source = await entry(root);
if (!source) return null;
if (!source.isDirectory() || source.isSymbolicLink()) throw invalid();
root = await fs.realpath(root);
marker = path.join(root, ".git"); markerEntry = await entry(marker);
if (markerEntry) break;
if (await entry(path.join(root, "objects")) && await entry(path.join(root, "HEAD"))) throw invalid();
if (!options.ancestors || path.dirname(root) === root) return null;
root = path.dirname(root);
}
const metadata: Metadata[] = [];
let gitDirectory: Identity;
if (markerEntry.isDirectory() && !markerEntry.isSymbolicLink()) {
if (!await entry(path.join(marker, "objects")) && await onlyEntry(marker, "info")) {
const info = path.join(marker, "info"), infoEntry = await entry(info);
if (infoEntry?.isDirectory() && !infoEntry.isSymbolicLink() && await onlyEntry(info, "exclude")) {
const exclude = await entry(path.join(info, "exclude"));
if (exclude?.isFile() && !exclude.isSymbolicLink() && exclude.nlink === 1n) return null;
}
}
gitDirectory = await directoryIdentity(marker);
}
else {
const pointer = await readMetadata(marker, 4096);
if (!pointer || !pointer.text.startsWith("gitdir: ")) throw invalid();
metadata.push(pointer.metadata);
gitDirectory = await directoryIdentity(path.resolve(root, metadataPath(pointer.text.slice(8))));
}
const commonPointer = await readMetadata(path.join(gitDirectory.path, "commondir"), 4096);
if (commonPointer) metadata.push(commonPointer.metadata);
const commonDirectory = commonPointer
? await directoryIdentity(path.resolve(gitDirectory.path, metadataPath(commonPointer.text))) : gitDirectory;
const objectDirectories: Identity[] = [], seen = new Set<string>();
const visit = async (target: string, depth: number): Promise<void> => {
if (depth > 4 || seen.size >= 32) throw invalid();
const directory = await directoryIdentity(target);
if (seen.has(directory.path)) return;
seen.add(directory.path); objectDirectories.push(directory);
if (await entry(path.join(directory.path, "info", "http-alternates"))) throw invalid();
const alternates = await readMetadata(path.join(directory.path, "info", "alternates"), 64 * 1024);
if (!alternates) return;
metadata.push(alternates.metadata);
const values = alternates.text.split("\n").filter(Boolean);
if (values.length > 32 || values.some(value => /[\0\r]/.test(value) || value.startsWith('"'))) throw invalid();
for (const value of values) await visit(path.resolve(directory.path, value), depth + 1);
};
await visit(path.join(commonDirectory.path, "objects"), 0);
objectDirectories.sort((a, b) => a.path.localeCompare(b.path));
metadata.sort((a, b) => a.path.localeCompare(b.path));
const result = { root, gitDirectory, commonDirectory, objectDirectories, metadata };
return { ...result, fingerprint: sha256(JSON.stringify(result)) };
}
+7 -6
View File
@@ -8,7 +8,8 @@ import { getStorageService } from "../storage/index.js";
import type { StorageService } from "../storage/types.js";
import { sanitizeRuntimeServiceBaseEnv } from "./runtime-service-env.js";
import { verifiedRecordedIsolatedRemoteSource } from "./isolated-remote-workspace-source.js";
import { assertCurrentWorkspaceSourceHeld, isolatedTaskSourceIdentity, managedProjectSourceIdentity, withWorkspaceSourceLocks } from "./workspace-source-lock.js";
import { assertCurrentWorkspaceSourceHeld, workspaceSourceIdentities, withWorkspaceSourceLocks } from "./workspace-source-lock.js";
import { localWorkspaceSourcePaths } from "./execution-workspace-source-roots.js";
export { sanitizeRuntimeServiceBaseEnv } from "./runtime-service-env.js";
import { execFile, spawn, type ChildProcess } from "node:child_process";
import { existsSync, lstatSync, readdirSync, readFileSync, realpathSync } from "node:fs";
@@ -7803,14 +7804,14 @@ async function withRuntimeServiceSourceLocks<T>(input: {
issue: ExecutionWorkspaceIssueRef | null; adapterEnv: Record<string, string>; services: Record<string, unknown>[];
}, action: () => Promise<T>): Promise<T> {
if (!input.db || input.services.length === 0) return action();
const paths = [input.workspace.cwd, input.workspace.baseCwd, ...input.services.map(service =>
const paths = [...localWorkspaceSourcePaths({ cwd: input.workspace.cwd, providerRef: input.workspace.worktreePath }),
input.workspace.baseCwd, ...input.services.map(service =>
resolveRuntimeServiceReuseIdentity({ ...input, service, scopeType: "project_workspace", scopeId: input.workspace.workspaceId }).serviceCwd)];
const projectIds: string[] = [], issueIds: string[] = [];
for (const cwd of new Set(paths)) {
const projectId = await managedProjectSourceIdentity(input.agent.companyId, cwd);
const issueId = await isolatedTaskSourceIdentity(input.agent.companyId, cwd);
if (projectId) projectIds.push(projectId);
if (issueId) issueIds.push(issueId);
const identities = await workspaceSourceIdentities(input.agent.companyId, cwd);
projectIds.push(...identities.projectIds);
issueIds.push(...identities.issueIds);
}
return withWorkspaceSourceLocks(input.db, { companyId: input.agent.companyId, projectIds, issueIds }, action);
}
+25 -19
View File
@@ -4,6 +4,7 @@ import { sql } from "drizzle-orm";
import path from "node:path";
import { resolvePaperclipInstanceRoot } from "../home-paths.js";
import { canonicalRepositoryRoot } from "./execution-workspace-source-roots.js";
import { readWorkspaceGitSourceRoots } from "./workspace-git-source-roots.js";
import { withWorkspaceLockKeepalive } from "./native-runtime/native-workspace-lock-keepalive.js";
type SourceTransaction = Parameters<Parameters<Db["transaction"]>[0]>[0];
@@ -38,7 +39,7 @@ export async function withWorkspaceSourceLocks<T>(db: Db,
if (held?.db === db) {
if (new Set([...held.keys, ...keys]).size > 128) throw new Error("workspace_source_scope_limit");
if (held.companyId !== input.companyId) throw new Error("workspace_source_company_mismatch");
const onlyProjectKeys = keys.every(key => key.startsWith("managed_project_source:"));
const onlyProjectKeys = keys.filter(key => !held.keys.has(key)).every(key => key.startsWith("managed_project_source:"));
const alreadyHasProject = [...held.keys].some(key => key.startsWith("managed_project_source:"));
await acquire(held, !onlyProjectKeys || alreadyHasProject);
return action(held.transaction);
@@ -65,7 +66,8 @@ export async function managedProjectSourceIdentity(companyId: string, cwd: strin
export async function isolatedTaskSourceIdentity(companyId: string, cwd: string): Promise<string | null> {
const root = await canonicalRepositoryRoot(resolvePaperclipInstanceRoot());
const relative = path.relative(root, await canonicalRepositoryRoot(cwd)).split(path.sep);
if (!["isolated-workspaces", "isolated-workspace-attempts"].includes(relative[0])) return null;
if (!["isolated-workspaces", "isolated-workspace-attempts", "isolated-repository-sources"].includes(relative[0])) return null;
if (relative[0] === "isolated-repository-sources" && relative[3] !== "checkout") return null;
if (relative[1] !== companyId) throw new Error("workspace_source_company_mismatch");
return relative[2] && /^[a-f0-9-]{36}$/.test(relative[2]) ? relative[2] : null;
}
@@ -77,31 +79,35 @@ export async function retainManagedProjectSource(db: Db, companyId: string, cwd:
await withWorkspaceSourceLocks(db, { companyId, projectIds: [projectId] }, async () => undefined, true);
}
export async function workspaceSourceIdentities(companyId: string, cwd: string) {
cwd = await canonicalRepositoryRoot(cwd);
const git = await readWorkspaceGitSourceRoots(cwd, { ancestors: true });
const roots = [cwd, ...(git ? [git.root, git.gitDirectory.path, git.commonDirectory.path,
...git.objectDirectories.map(directory => directory.path)] : [])];
const projectIds = new Set<string>(), issueIds = new Set<string>(), unownedPaths: string[] = [];
for (const target of new Set(roots)) {
const projectId = await managedProjectSourceIdentity(companyId, target);
const issueId = await isolatedTaskSourceIdentity(companyId, target);
if (projectId) projectIds.add(projectId);
if (issueId) issueIds.add(issueId);
if (!projectId && !issueId) unownedPaths.push(target);
}
return { projectIds: [...projectIds].sort(), issueIds: [...issueIds].sort(), unownedPaths, git };
}
export async function retainCurrentWorkspaceSource(cwd: string): Promise<void> {
const scope = sourceLocks.getStore();
if (!scope) return;
const instance = await canonicalRepositoryRoot(resolvePaperclipInstanceRoot());
const task = path.relative(instance, await canonicalRepositoryRoot(cwd)).split(path.sep);
if (["isolated-workspaces", "isolated-workspace-attempts"].includes(task[0]) && task.length >= 3) {
const issueId = await isolatedTaskSourceIdentity(scope.companyId, cwd);
if (issueId) await withWorkspaceSourceLocks(scope.db, { companyId: scope.companyId, issueIds: [issueId] }, async () => undefined, true);
return;
}
const root = await canonicalRepositoryRoot(path.join(resolvePaperclipInstanceRoot(), "projects"));
const target = await canonicalRepositoryRoot(cwd), relative = path.relative(root, target);
if (relative === ".." || relative.startsWith(`..${path.sep}`) || path.isAbsolute(relative)) return;
const [companyId, projectId] = relative.split(path.sep);
if (!companyId || !projectId || !/^[a-f0-9-]{36}$/.test(companyId) || !/^[a-f0-9-]{36}$/.test(projectId)) return;
await retainManagedProjectSource(scope.db, companyId, cwd);
const identities = await workspaceSourceIdentities(scope.companyId, cwd);
await withWorkspaceSourceLocks(scope.db, { companyId: scope.companyId, ...identities }, async () => undefined, true);
}
export async function assertCurrentWorkspaceSourceHeld(cwd: string): Promise<void> {
const scope = sourceLocks.getStore();
if (!scope) throw new Error("workspace_source_admission_scope_required");
const projectId = await managedProjectSourceIdentity(scope.companyId, cwd);
const issueId = await isolatedTaskSourceIdentity(scope.companyId, cwd);
if (projectId && !scope.keys.has(`managed_project_source:${scope.companyId}:${projectId}`)
|| issueId && !scope.keys.has(`isolated_repository_source:${scope.companyId}:${issueId}`)) {
const identities = await workspaceSourceIdentities(scope.companyId, cwd);
if (identities.projectIds.some(id => !scope.keys.has(`managed_project_source:${scope.companyId}:${id}`))
|| identities.issueIds.some(id => !scope.keys.has(`isolated_repository_source:${scope.companyId}:${id}`))) {
throw new IsolatedRepositorySourceBusy("workspace_source_busy");
}
}