mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-23 17:33:51 +02:00
feat: add a library to capture and restore git workspace restore points (#17238)
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Each issue's agent works in a git worktree that Paperclip keeps on
the instance's disk
> - Git cache expiration removes a closed issue's worktree only when the
remote can reproduce it exactly: every commit pushed, nothing
uncommitted
> - Most closed worktrees fail that test, so they are kept forever and
disks fill up
> - A restore point that saves only the part the remote cannot reproduce
would let those worktrees be removed without losing work
> - This pull request adds the library to capture, verify, encrypt and
restore such restore points, with no callers yet
> - The benefit is a small, reviewable foundation for removing unpushed
and uncommitted worktrees safely in follow-up changes
## Linked Issues or Issue Description
Bug report described in-PR (bug report template).
**What happened?**
Closed issues' worktrees accumulate without bound. With
`PAPERCLIP_GIT_CACHE_EXPIRATION_ENABLED=true` and dry run off, one
instance's last 111 expiration checks expired nothing. The skip reasons
were:
| Skip reason | Checks |
|---|---|
| `dirty_or_untracked_files` | 39 |
| `unpushed_managed_checkout` | 32 |
| `live_or_unresolved_consumer` | 21 |
| `unique_ignored_files` | 9 |
| commit not on the remote | 10 |
Without expiration, the terminal workspace reaper only archives merged,
clean worktrees. One local instance archived 37 workspaces across about
6,200 sweeps.
**Expected behavior**
A closed worktree can be removed once anything the remote cannot
reproduce is saved, and restored exactly if the work resumes.
**Steps to reproduce**
1. Close an issue whose worktree has an unpushed commit or an untracked
file.
2. Enable git cache expiration with dry run off.
3. The worktree is never removed; expiration reports
`dirty_or_untracked_files` or `upstream_commit_unverified`.
**Paperclip version or commit:** `master` at def4726bea.
**Deployment mode:** any; observed on a server instance and a local
instance.
**Agent adapter(s) involved:** not adapter-specific (core bug).
## What Changed
New `server/src/services/git-restore-point.ts`, a library that nothing
calls yet:
- **`captureRestorePoint`**
- Snapshots the index and the working tree as deterministic commits in a
temporary object store. The store borrows the repository's objects
through alternates and writes none of its own into the repository.
- Bundles those snapshot commits, unpushed commits, and commits
reachable only from the reflog or `ORIG_HEAD`, against the remote
commits as prerequisites.
- Returns `null` when the remote can already reproduce everything.
Refuses to capture when more than 50 history-only commits exist, rather
than dropping any.
- **Leaves out:** ignored files (such as `node_modules`), and untracked
`.env` and `.env.*` files. Provisioning recreates environment files with
current secrets. Edits to tracked environment files, such as
`.env.example`, are captured.
- **Skips** nested untracked repositories, paths with a filter
attribute, gitlinks and changes over size limits rather than capturing
them partially. Special files are ignored, because git never tracks
them.
- **`currentRestorePointSnapshot`** recomputes the snapshot ids, so a
caller can prove nothing changed between capture and deletion.
- **`importRestorePoint`** verifies the bundle's prerequisites and refs
against the manifest, then imports the objects without leaving refs
behind.
- **`applyRestorePoint`**
- Requires `HEAD` to match.
- Restores the working tree (`git apply`), the index (`read-tree`) and
non-default file modes.
- Then recomputes the snapshot and requires it to equal the captured
one.
- **`sealRestorePoint` / `openRestorePoint`**
- Encrypt with a fresh AES-256-GCM key per restore point, with the
restore point's identity as additional authenticated data.
- The data key is wrapped by the instance secrets provider
(`localEncryptedProvider`), the same pattern already used for saved
Claude settings.
- Opening checks the ciphertext hash, the key's bound identity, the GCM
tag, and the plaintext hash and size.
- **Isolation:** every git command runs with system and global config
disabled, hooks disabled, and line-ending conversion off.
- New `server/src/__tests__/git-restore-point.test.ts`.
- No code comments were added.
## Verification
- `pnpm exec vitest run src/__tests__/git-restore-point.test.ts` (in
`server/`): 18 passed, using real repositories and a bare remote. The
cases:
- **Full round trip** into a fresh clone that lacks the unpushed
commits, covering:
- two unpushed commits;
- staged and unstaged edits of the same file;
- a deletion, an untracked file, a symlink, `0755` and `0600` modes;
- `node_modules` and `.env` files left out.
- **No-op:** capture returns `null` for a reproducible workspace, and
creates no objects in the repository (`fsck --unreachable` stays empty).
- **Snapshot ids** are deterministic and change when contents change.
- **History-only commits** are preserved and leave no refs after import.
- **Skips:** nested repositories and size limits. Special files are
ignored.
- **Rejections:** missing prerequisites and `HEAD` mismatches.
- **Encryption:** tampering, truncation, a wrong identity and a
different master key are all rejected.
- **Review regressions:** edits to a tracked `.env.example` are
captured; staged files whose contents match the index get limits and
modes; permission-only changes change the snapshot; a configured `diff`
textconv does not alter restored bytes; more than 50 history-only
commits refuses capture.
- **Mutation checks:** removing the `.env` exclusion or the index
restore fails the round trip, and removing each review fix fails its
regression test.
- **Stability:** 12 consecutive runs passed. An earlier version copied
the index for the working-tree snapshot and was racy when a file changed
within the same second. It now starts from `HEAD`'s tree instead.
- `pnpm run typecheck` (in `server/`): pass.
## Risks
- **No behavior change:** nothing calls the library yet.
- **Follow-ups:** wiring it into expiration (storage, metadata
versioning, restore on reopen and on the next run, and retention) comes
in separate PRs behind configuration that defaults to off.
- **Key dependence:** a restore point is only as recoverable as the
instance's secrets master key. The follow-up wiring will refuse to
create restore points when that key is not durable.
- **Rollback:** revert.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), 1M context, through Claude Code
with tool use (shell, file edits, tests, read-only measurements on live
instances).
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
---------
Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
059ae49cab
commit
30f402f5fe
2 files changed
+752
No files matched your search
@@ -0,0 +1,304 @@
|
||||
import { execFileSync } from "node:child_process";
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
applyRestorePoint,
|
||||
captureRestorePoint,
|
||||
currentRestorePointSnapshot,
|
||||
importRestorePoint,
|
||||
openRestorePoint,
|
||||
RestorePointRestoreFailure,
|
||||
RestorePointSkip,
|
||||
sealRestorePoint,
|
||||
} from "../services/git-restore-point.ts";
|
||||
|
||||
const identity = { companyId: "company-1", workspaceId: "workspace-1", restorePointId: "restore-1", generation: 1 };
|
||||
|
||||
function git(cwd: string, ...args: string[]) {
|
||||
return execFileSync("git", ["-C", cwd, ...args], {
|
||||
encoding: "utf8",
|
||||
env: { ...process.env, GIT_CONFIG_NOSYSTEM: "1", GIT_CONFIG_GLOBAL: os.devNull, GIT_AUTHOR_NAME: "Test", GIT_AUTHOR_EMAIL: "t@example.com", GIT_COMMITTER_NAME: "Test", GIT_COMMITTER_EMAIL: "t@example.com" },
|
||||
}).trim();
|
||||
}
|
||||
|
||||
async function write(root: string, relative: string, contents: string, mode?: number) {
|
||||
await fs.mkdir(path.dirname(path.join(root, relative)), { recursive: true });
|
||||
await fs.writeFile(path.join(root, relative), contents);
|
||||
if (mode !== undefined) await fs.chmod(path.join(root, relative), mode);
|
||||
}
|
||||
|
||||
async function read(root: string, relative: string) {
|
||||
return fs.readFile(path.join(root, relative), "utf8");
|
||||
}
|
||||
|
||||
describe("git restore points", () => {
|
||||
let root: string;
|
||||
let remote: string;
|
||||
let work: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", "31".repeat(32));
|
||||
root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-restore-point-test-"));
|
||||
remote = path.join(root, "remote.git");
|
||||
work = path.join(root, "work");
|
||||
git(root, "init", "-q", "--bare", "-b", "master", remote);
|
||||
git(root, "clone", "-q", remote, work);
|
||||
await write(work, ".gitignore", "node_modules/\n.env\n");
|
||||
await write(work, "src/app.ts", "export const value = 1;\n");
|
||||
await write(work, "src/remove-me.ts", "export const gone = true;\n");
|
||||
await write(work, "README.md", "hello\n");
|
||||
git(work, "add", "-A");
|
||||
git(work, "commit", "-q", "-m", "initial");
|
||||
git(work, "push", "-q", "origin", "HEAD:master");
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
vi.unstubAllEnvs();
|
||||
await fs.rm(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
const upstream = () => [git(work, "rev-parse", "origin/master")];
|
||||
|
||||
async function freshClone(name: string) {
|
||||
const target = path.join(root, name);
|
||||
git(root, "clone", "-q", remote, target);
|
||||
return target;
|
||||
}
|
||||
|
||||
it("returns nothing when the workspace is fully reproducible from the remote", async () => {
|
||||
await write(work, "node_modules/pkg/index.js", "ignored\n");
|
||||
expect(await captureRestorePoint({ cwd: work, upstreamCommits: upstream() })).toBeNull();
|
||||
});
|
||||
|
||||
it("round-trips unpushed commits, staged and unstaged edits, untracked files, deletions and modes", async () => {
|
||||
git(work, "checkout", "-q", "-b", "feature");
|
||||
await write(work, "src/app.ts", "export const value = 2;\n");
|
||||
git(work, "commit", "-q", "-am", "unpushed one");
|
||||
await write(work, "src/feature.ts", "export const feature = true;\n");
|
||||
git(work, "add", "src/feature.ts");
|
||||
git(work, "commit", "-q", "-m", "unpushed two");
|
||||
await write(work, "src/app.ts", "export const value = 3;\n");
|
||||
git(work, "add", "src/app.ts");
|
||||
await write(work, "src/app.ts", "export const value = 4;\n");
|
||||
await fs.rm(path.join(work, "src/remove-me.ts"));
|
||||
await write(work, "notes/todo.md", "untracked work\n");
|
||||
await write(work, "scripts/run.sh", "#!/bin/sh\necho hi\n", 0o755);
|
||||
await write(work, "private/key.txt", "secret-ish\n", 0o600);
|
||||
await fs.symlink("../README.md", path.join(work, "notes/readme-link"));
|
||||
await write(work, "node_modules/pkg/index.js", "ignored\n");
|
||||
await write(work, ".env", "TOKEN=ignored\n");
|
||||
await write(work, "config/.env.local", "TOKEN=untracked-env\n");
|
||||
|
||||
const captured = await captureRestorePoint({ cwd: work, upstreamCommits: upstream() });
|
||||
expect(captured).not.toBeNull();
|
||||
const { payload, manifest } = captured!;
|
||||
expect(manifest.head).toBe(git(work, "rev-parse", "HEAD"));
|
||||
expect(manifest.prerequisites).toEqual(upstream());
|
||||
expect(await currentRestorePointSnapshot(work)).toEqual({ head: manifest.head, index: manifest.index, worktree: manifest.worktree, modes: manifest.modes });
|
||||
|
||||
const restored = await freshClone("restored");
|
||||
expect(() => git(restored, "cat-file", "-e", `${manifest.head}^{commit}`)).toThrow();
|
||||
const imported = await importRestorePoint({ repositoryCwd: restored, payload });
|
||||
git(restored, "checkout", "-q", "-b", "feature", imported.head);
|
||||
await applyRestorePoint({ cwd: restored, manifest: imported });
|
||||
|
||||
expect(await read(restored, "src/app.ts")).toBe("export const value = 4;\n");
|
||||
expect(git(restored, "show", ":src/app.ts")).toBe("export const value = 3;");
|
||||
expect(await read(restored, "src/feature.ts")).toBe("export const feature = true;\n");
|
||||
await expect(fs.access(path.join(restored, "src/remove-me.ts"))).rejects.toMatchObject({ code: "ENOENT" });
|
||||
expect(await read(restored, "notes/todo.md")).toBe("untracked work\n");
|
||||
expect(git(restored, "status", "--porcelain", "--", "notes/todo.md")).toBe("?? notes/todo.md");
|
||||
expect((await fs.stat(path.join(restored, "scripts/run.sh"))).mode & 0o777).toBe(0o755);
|
||||
expect((await fs.stat(path.join(restored, "private/key.txt"))).mode & 0o777).toBe(0o600);
|
||||
expect(await fs.readlink(path.join(restored, "notes/readme-link"))).toBe("../README.md");
|
||||
await expect(fs.access(path.join(restored, "node_modules"))).rejects.toMatchObject({ code: "ENOENT" });
|
||||
await expect(fs.access(path.join(restored, ".env"))).rejects.toMatchObject({ code: "ENOENT" });
|
||||
await expect(fs.access(path.join(restored, "config/.env.local"))).rejects.toMatchObject({ code: "ENOENT" });
|
||||
});
|
||||
|
||||
it("captures edits to tracked environment files", async () => {
|
||||
await write(work, ".env.example", "TOKEN=\n");
|
||||
git(work, "add", ".env.example");
|
||||
git(work, "commit", "-q", "-m", "example env");
|
||||
git(work, "push", "-q", "origin", "HEAD:master");
|
||||
git(work, "fetch", "-q");
|
||||
await write(work, ".env.example", "TOKEN=\nOTHER=\n");
|
||||
const captured = await captureRestorePoint({ cwd: work, upstreamCommits: upstream() });
|
||||
expect(captured).not.toBeNull();
|
||||
const restored = await freshClone("restored");
|
||||
await importRestorePoint({ repositoryCwd: restored, payload: captured!.payload });
|
||||
await applyRestorePoint({ cwd: restored, manifest: captured!.manifest });
|
||||
expect(await read(restored, ".env.example")).toBe("TOKEN=\nOTHER=\n");
|
||||
});
|
||||
|
||||
it("applies safeguards and modes to staged files whose contents match the index", async () => {
|
||||
await write(work, "private/staged.txt", "staged secret\n", 0o600);
|
||||
git(work, "add", "private/staged.txt");
|
||||
await expect(captureRestorePoint({ cwd: work, upstreamCommits: upstream(), limits: { maxFiles: 0 } }))
|
||||
.rejects.toMatchObject({ reason: "restore_point_too_large" });
|
||||
const captured = await captureRestorePoint({ cwd: work, upstreamCommits: upstream() });
|
||||
expect(captured!.manifest.modes).toEqual([{ path: "private/staged.txt", mode: 0o600 }]);
|
||||
const restored = await freshClone("restored");
|
||||
await importRestorePoint({ repositoryCwd: restored, payload: captured!.payload });
|
||||
await applyRestorePoint({ cwd: restored, manifest: captured!.manifest });
|
||||
expect((await fs.stat(path.join(restored, "private/staged.txt"))).mode & 0o777).toBe(0o600);
|
||||
expect(git(restored, "status", "--porcelain", "--", "private/staged.txt")).toBe("A private/staged.txt");
|
||||
});
|
||||
|
||||
it("detects permission-only changes after capture", async () => {
|
||||
await write(work, "private/key.txt", "secret\n", 0o600);
|
||||
const before = await currentRestorePointSnapshot(work);
|
||||
await fs.chmod(path.join(work, "private/key.txt"), 0o644);
|
||||
expect(await currentRestorePointSnapshot(work)).not.toEqual(before);
|
||||
});
|
||||
|
||||
it("restores exact bytes when the repository configures a diff text conversion", async () => {
|
||||
await write(work, ".gitattributes", "*.txt diff=upper\n");
|
||||
await write(work, "notes.txt", "original\n");
|
||||
git(work, "add", ".gitattributes", "notes.txt");
|
||||
git(work, "commit", "-q", "-m", "textconv");
|
||||
git(work, "push", "-q", "origin", "HEAD:master");
|
||||
git(work, "fetch", "-q");
|
||||
await write(work, "notes.txt", "changed contents\n");
|
||||
const captured = await captureRestorePoint({ cwd: work, upstreamCommits: upstream() });
|
||||
const restored = await freshClone("restored");
|
||||
git(restored, "config", "diff.upper.textconv", "tr a-z A-Z <");
|
||||
await importRestorePoint({ repositoryCwd: restored, payload: captured!.payload });
|
||||
await applyRestorePoint({ cwd: restored, manifest: captured!.manifest });
|
||||
expect(await read(restored, "notes.txt")).toBe("changed contents\n");
|
||||
});
|
||||
|
||||
it("captures a small edit despite a long ordinary history", async () => {
|
||||
for (let index = 0; index < 60; index += 1) {
|
||||
await write(work, "log.txt", `${index}\n`);
|
||||
git(work, "commit", "-q", "-am", `history ${index}`, "--allow-empty");
|
||||
}
|
||||
git(work, "add", "-A");
|
||||
git(work, "commit", "-q", "-m", "tracked log", "--allow-empty");
|
||||
git(work, "push", "-q", "origin", "HEAD:master");
|
||||
git(work, "fetch", "-q");
|
||||
await write(work, "notes/todo.md", "small edit\n");
|
||||
const captured = await captureRestorePoint({ cwd: work, upstreamCommits: upstream() });
|
||||
expect(captured!.manifest.preserved).toEqual([]);
|
||||
expect(captured!.manifest.worktree).not.toBeNull();
|
||||
});
|
||||
|
||||
it("refuses to capture when history-only commits exceed the supported limit", async () => {
|
||||
git(work, "checkout", "-q", "-b", "feature");
|
||||
for (let index = 0; index < 51; index += 1) {
|
||||
await write(work, `scratch/${index}.txt`, `${index}\n`);
|
||||
git(work, "add", "-A");
|
||||
git(work, "commit", "-q", "-m", `scratch ${index}`);
|
||||
git(work, "reset", "-q", "--hard", "origin/master");
|
||||
}
|
||||
await expect(captureRestorePoint({ cwd: work, upstreamCommits: upstream() }))
|
||||
.rejects.toMatchObject({ reason: "restore_point_history_too_large" });
|
||||
});
|
||||
|
||||
it("keeps capture objects out of the repository", async () => {
|
||||
await write(work, "notes/todo.md", "untracked work\n");
|
||||
const before = git(work, "count-objects", "-v");
|
||||
expect(await captureRestorePoint({ cwd: work, upstreamCommits: upstream() })).not.toBeNull();
|
||||
expect(git(work, "count-objects", "-v")).toBe(before);
|
||||
expect(git(work, "fsck", "--no-reflogs", "--unreachable", "--no-progress")).toBe("");
|
||||
});
|
||||
|
||||
it("produces the same snapshot ids for unchanged contents and different ids after a change", async () => {
|
||||
await write(work, "notes/todo.md", "one\n");
|
||||
const first = await currentRestorePointSnapshot(work);
|
||||
expect(await currentRestorePointSnapshot(work)).toEqual(first);
|
||||
await write(work, "notes/todo.md", "two\n");
|
||||
expect(await currentRestorePointSnapshot(work)).not.toEqual(first);
|
||||
});
|
||||
|
||||
it("bundles commits only reachable from history that is not on any branch", async () => {
|
||||
git(work, "checkout", "-q", "-b", "feature");
|
||||
await write(work, "src/lost.ts", "export const lost = true;\n");
|
||||
git(work, "add", "src/lost.ts");
|
||||
git(work, "commit", "-q", "-m", "rewritten away");
|
||||
const lost = git(work, "rev-parse", "HEAD");
|
||||
git(work, "reset", "-q", "--hard", "origin/master");
|
||||
const captured = await captureRestorePoint({ cwd: work, upstreamCommits: upstream() });
|
||||
expect(captured!.manifest.preserved).toEqual([lost]);
|
||||
const restored = await freshClone("restored");
|
||||
await importRestorePoint({ repositoryCwd: restored, payload: captured!.payload });
|
||||
expect(git(restored, "cat-file", "-t", lost)).toBe("commit");
|
||||
expect(git(restored, "for-each-ref", "refs/paperclip-restore-import/")).toBe("");
|
||||
});
|
||||
|
||||
it("skips a nested untracked repository", async () => {
|
||||
git(root, "init", "-q", path.join(work, "vendor/other"));
|
||||
await write(work, "vendor/other/file.txt", "x\n");
|
||||
await expect(captureRestorePoint({ cwd: work, upstreamCommits: upstream() }))
|
||||
.rejects.toMatchObject({ reason: "untracked_nested_repository" });
|
||||
});
|
||||
|
||||
it("ignores special files, which git never tracks", async () => {
|
||||
execFileSync("mkfifo", [path.join(work, "pipe")]);
|
||||
expect(await captureRestorePoint({ cwd: work, upstreamCommits: upstream() })).toBeNull();
|
||||
});
|
||||
|
||||
it("skips workspaces over the size limits", async () => {
|
||||
await write(work, "big.bin", "x".repeat(2048));
|
||||
await expect(captureRestorePoint({ cwd: work, upstreamCommits: upstream(), limits: { maxFileBytes: 1024 } }))
|
||||
.rejects.toBeInstanceOf(RestorePointSkip);
|
||||
await expect(captureRestorePoint({ cwd: work, upstreamCommits: upstream(), limits: { maxFiles: 0 } }))
|
||||
.rejects.toMatchObject({ reason: "restore_point_too_large" });
|
||||
});
|
||||
|
||||
it("refuses to import a bundle whose prerequisites are missing", async () => {
|
||||
git(work, "checkout", "-q", "-b", "feature");
|
||||
await write(work, "src/app.ts", "export const value = 2;\n");
|
||||
git(work, "commit", "-q", "-am", "unpushed");
|
||||
const captured = await captureRestorePoint({ cwd: work, upstreamCommits: upstream() });
|
||||
const empty = path.join(root, "empty");
|
||||
git(root, "init", "-q", empty);
|
||||
await expect(importRestorePoint({ repositoryCwd: empty, payload: captured!.payload }))
|
||||
.rejects.toMatchObject({ reason: "bundle_prerequisites_missing" });
|
||||
});
|
||||
|
||||
it("refuses to apply onto a different commit", async () => {
|
||||
await write(work, "notes/todo.md", "work\n");
|
||||
const captured = await captureRestorePoint({ cwd: work, upstreamCommits: upstream() });
|
||||
const restored = await freshClone("restored");
|
||||
await write(restored, "other.txt", "x\n");
|
||||
git(restored, "add", "other.txt");
|
||||
git(restored, "commit", "-q", "-m", "diverged");
|
||||
await expect(applyRestorePoint({ cwd: restored, manifest: captured!.manifest }))
|
||||
.rejects.toMatchObject({ reason: "head_mismatch" });
|
||||
});
|
||||
|
||||
describe("encryption", () => {
|
||||
const payload = Buffer.from("restore point payload");
|
||||
|
||||
it("round-trips and binds to the restore point identity", async () => {
|
||||
const sealed = await sealRestorePoint(payload, identity);
|
||||
expect(sealed.ciphertext.includes(payload)).toBe(false);
|
||||
expect(await openRestorePoint({ ...sealed, identity })).toEqual(payload);
|
||||
await expect(openRestorePoint({ ...sealed, identity: { ...identity, workspaceId: "workspace-2" } }))
|
||||
.rejects.toMatchObject({ reason: "key_identity_mismatch" });
|
||||
});
|
||||
|
||||
it("rejects tampered, truncated or rehashed ciphertext", async () => {
|
||||
const sealed = await sealRestorePoint(payload, identity);
|
||||
const flipped = Buffer.from(sealed.ciphertext);
|
||||
flipped[flipped.length - 20] ^= 1;
|
||||
await expect(openRestorePoint({ ...sealed, ciphertext: flipped, identity }))
|
||||
.rejects.toMatchObject({ reason: "ciphertext_hash_mismatch" });
|
||||
const { createHash } = await import("node:crypto");
|
||||
await expect(openRestorePoint({ ...sealed, ciphertext: flipped, sha256: createHash("sha256").update(flipped).digest("hex"), identity }))
|
||||
.rejects.toMatchObject({ reason: "ciphertext_authentication_failed" });
|
||||
const truncated = sealed.ciphertext.subarray(0, 10);
|
||||
await expect(openRestorePoint({ ...sealed, ciphertext: truncated, sha256: createHash("sha256").update(truncated).digest("hex"), identity }))
|
||||
.rejects.toBeInstanceOf(RestorePointRestoreFailure);
|
||||
});
|
||||
|
||||
it("cannot be opened with a different master key", async () => {
|
||||
const sealed = await sealRestorePoint(payload, identity);
|
||||
vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", "42".repeat(32));
|
||||
await expect(openRestorePoint({ ...sealed, identity })).rejects.toThrow();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,448 @@
|
||||
import { execFile as execFileCallback } from "node:child_process";
|
||||
import { createCipheriv, createDecipheriv, createHash, randomBytes } from "node:crypto";
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { localEncryptedProvider } from "../secrets/local-encrypted-provider.js";
|
||||
import type { StoredSecretVersionMaterial } from "../secrets/types.js";
|
||||
|
||||
const execFile = promisify(execFileCallback);
|
||||
|
||||
export const RESTORE_POINT_PAYLOAD_SCHEMA = "paperclip.git-restore-point-payload.v1";
|
||||
const PAYLOAD_MAGIC = Buffer.from("PCRP-PLAIN1\n");
|
||||
const SEALED_MAGIC = Buffer.from("PCRP1");
|
||||
const KEY_SCHEMA = "paperclip.git-restore-point-key.v1";
|
||||
const SHA = /^[0-9a-f]{40}$/;
|
||||
const SNAPSHOT_IDENTITY = {
|
||||
GIT_AUTHOR_NAME: "Paperclip",
|
||||
GIT_AUTHOR_EMAIL: "restore-point@paperclip.invalid",
|
||||
GIT_AUTHOR_DATE: "@0 +0000",
|
||||
GIT_COMMITTER_NAME: "Paperclip",
|
||||
GIT_COMMITTER_EMAIL: "restore-point@paperclip.invalid",
|
||||
GIT_COMMITTER_DATE: "@0 +0000",
|
||||
};
|
||||
const EXCLUDED_UNTRACKED = [":(exclude,glob)**/.env", ":(exclude,glob)**/.env.*"];
|
||||
const MAX_PRESERVED_COMMITS = 50;
|
||||
|
||||
export type RestorePointLimits = {
|
||||
maxFiles: number;
|
||||
maxFileBytes: number;
|
||||
maxTotalBytes: number;
|
||||
maxCommits: number;
|
||||
maxBundleBytes: number;
|
||||
};
|
||||
|
||||
export const DEFAULT_RESTORE_POINT_LIMITS: RestorePointLimits = {
|
||||
maxFiles: 20_000,
|
||||
maxFileBytes: 64 * 1024 * 1024,
|
||||
maxTotalBytes: 256 * 1024 * 1024,
|
||||
maxCommits: 1_000,
|
||||
maxBundleBytes: 256 * 1024 * 1024,
|
||||
};
|
||||
|
||||
export type RestorePointManifest = {
|
||||
schema: typeof RESTORE_POINT_PAYLOAD_SCHEMA;
|
||||
head: string;
|
||||
index: string | null;
|
||||
worktree: string | null;
|
||||
preserved: string[];
|
||||
prerequisites: string[];
|
||||
modes: Array<{ path: string; mode: number }>;
|
||||
};
|
||||
|
||||
export type RestorePointSnapshot = { head: string; index: string | null; worktree: string | null; modes: Array<{ path: string; mode: number }> };
|
||||
|
||||
export type RestorePointIdentity = { companyId: string; workspaceId: string; restorePointId: string; generation: number };
|
||||
|
||||
export type SealedRestorePoint = {
|
||||
ciphertext: Buffer;
|
||||
sha256: string;
|
||||
key: StoredSecretVersionMaterial;
|
||||
};
|
||||
|
||||
export class RestorePointSkip extends Error {
|
||||
constructor(readonly reason: string) {
|
||||
super(reason);
|
||||
this.name = "RestorePointSkip";
|
||||
}
|
||||
}
|
||||
|
||||
export class RestorePointRestoreFailure extends Error {
|
||||
constructor(readonly reason: string) {
|
||||
super(`The workspace restore point could not be applied: ${reason}`);
|
||||
this.name = "RestorePointRestoreFailure";
|
||||
}
|
||||
}
|
||||
|
||||
type GitOptions = { env?: Record<string, string>; input?: Buffer | string; signal?: AbortSignal; maxBuffer?: number };
|
||||
|
||||
function isolatedEnv(extra: Record<string, string> = {}) {
|
||||
return {
|
||||
...process.env,
|
||||
GIT_CONFIG_NOSYSTEM: "1",
|
||||
GIT_CONFIG_GLOBAL: os.devNull,
|
||||
GIT_TERMINAL_PROMPT: "0",
|
||||
GIT_OPTIONAL_LOCKS: "0",
|
||||
...extra,
|
||||
};
|
||||
}
|
||||
|
||||
async function git(cwd: string, args: string[], options: GitOptions = {}): Promise<Buffer> {
|
||||
const child = execFile("git", [
|
||||
"-c", "core.fsmonitor=false", "-c", "core.autocrlf=false", "-c", "core.safecrlf=false",
|
||||
"-c", `core.hooksPath=${os.devNull}`, "-C", cwd, ...args,
|
||||
], {
|
||||
env: isolatedEnv(options.env),
|
||||
encoding: "buffer",
|
||||
maxBuffer: options.maxBuffer ?? 64 * 1024 * 1024,
|
||||
timeout: 120_000,
|
||||
signal: options.signal,
|
||||
});
|
||||
if (options.input !== undefined) child.child.stdin?.end(options.input);
|
||||
const { stdout } = await child;
|
||||
return stdout as Buffer;
|
||||
}
|
||||
|
||||
async function gitText(cwd: string, args: string[], options: GitOptions = {}) {
|
||||
return (await git(cwd, args, options)).toString("utf8").trim();
|
||||
}
|
||||
|
||||
function nulList(output: Buffer) {
|
||||
return output.toString("utf8").split("\0").filter(Boolean);
|
||||
}
|
||||
|
||||
async function objectsDirectory(cwd: string) {
|
||||
const common = await gitText(cwd, ["rev-parse", "--git-common-dir"]);
|
||||
return path.resolve(cwd, common, "objects");
|
||||
}
|
||||
|
||||
async function snapshotStore(cwd: string, temporary: string) {
|
||||
const store = path.join(temporary, "store.git");
|
||||
await execFile("git", ["init", "-q", "--bare", store], { env: isolatedEnv() });
|
||||
const repositoryObjects = await objectsDirectory(cwd);
|
||||
await fs.writeFile(path.join(store, "objects", "info", "alternates"), `${repositoryObjects}\n`);
|
||||
return {
|
||||
store,
|
||||
env: {
|
||||
GIT_OBJECT_DIRECTORY: path.join(store, "objects"),
|
||||
GIT_ALTERNATE_OBJECT_DIRECTORIES: repositoryObjects,
|
||||
...SNAPSHOT_IDENTITY,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function copyIndex(cwd: string, target: string) {
|
||||
const indexPath = path.resolve(cwd, await gitText(cwd, ["rev-parse", "--git-path", "index"]));
|
||||
await fs.copyFile(indexPath, target).catch((error: NodeJS.ErrnoException) => {
|
||||
if (error.code !== "ENOENT") throw error;
|
||||
});
|
||||
}
|
||||
|
||||
async function scanChangedPaths(cwd: string, limits: RestorePointLimits, signal?: AbortSignal) {
|
||||
const changed = new Set([
|
||||
...nulList(await git(cwd, ["ls-files", "-z", "--modified"], { signal })),
|
||||
...nulList(await git(cwd, ["diff", "--cached", "--name-only", "--no-renames", "-z", "HEAD"], { signal })),
|
||||
...nulList(await git(cwd, ["ls-files", "-z", "--others", "--exclude-standard", "--", ".", ...EXCLUDED_UNTRACKED], { signal })),
|
||||
]);
|
||||
if (changed.size > limits.maxFiles) throw new RestorePointSkip("restore_point_too_large");
|
||||
let totalBytes = 0;
|
||||
const modes: Array<{ path: string; mode: number }> = [];
|
||||
for (const relative of changed) {
|
||||
if (relative.endsWith("/")) throw new RestorePointSkip("untracked_nested_repository");
|
||||
const stat = await fs.lstat(path.join(cwd, relative)).catch((error: NodeJS.ErrnoException) => {
|
||||
if (error.code === "ENOENT") return null;
|
||||
throw error;
|
||||
});
|
||||
if (!stat) continue;
|
||||
if (stat.isDirectory()) throw new RestorePointSkip("untracked_nested_repository");
|
||||
if (!stat.isFile() && !stat.isSymbolicLink()) throw new RestorePointSkip("unsupported_special_file");
|
||||
if (stat.isFile()) {
|
||||
if (stat.size > limits.maxFileBytes) throw new RestorePointSkip("restore_point_too_large");
|
||||
totalBytes += stat.size;
|
||||
const mode = stat.mode & 0o777;
|
||||
if (mode !== 0o644 && mode !== 0o755) modes.push({ path: relative, mode });
|
||||
}
|
||||
if (totalBytes > limits.maxTotalBytes) throw new RestorePointSkip("restore_point_too_large");
|
||||
}
|
||||
if (changed.size > 0) {
|
||||
const attributes = nulList(await git(cwd, ["check-attr", "-z", "--stdin", "filter"], { input: [...changed].join("\0"), signal }));
|
||||
for (let index = 2; index < attributes.length; index += 3) {
|
||||
if (attributes[index] !== "unspecified" && attributes[index] !== "unset") throw new RestorePointSkip("filtered_paths");
|
||||
}
|
||||
}
|
||||
return { modes: modes.sort((a, b) => a.path.localeCompare(b.path)) };
|
||||
}
|
||||
|
||||
async function snapshotTrees(cwd: string, env: Record<string, string>, temporary: string, signal?: AbortSignal) {
|
||||
const indexCopy = path.join(temporary, "index");
|
||||
await copyIndex(cwd, indexCopy);
|
||||
const indexTree = await gitText(cwd, ["write-tree"], { env: { ...env, GIT_INDEX_FILE: indexCopy }, signal })
|
||||
.catch(() => { throw new RestorePointSkip("unmerged_index"); });
|
||||
const worktreeIndex = path.join(temporary, "worktree-index");
|
||||
const worktreeEnv = { ...env, GIT_INDEX_FILE: worktreeIndex };
|
||||
await git(cwd, ["read-tree", "HEAD"], { env: worktreeEnv, signal });
|
||||
await git(cwd, ["add", "-u", "--", "."], { env: worktreeEnv, signal });
|
||||
await git(cwd, ["add", "-A", "--", ".", ...EXCLUDED_UNTRACKED], { env: worktreeEnv, signal });
|
||||
const worktreeTree = await gitText(cwd, ["write-tree"], { env: worktreeEnv, signal });
|
||||
const gitlinks = await gitText(cwd, ["ls-tree", "-r", worktreeTree], { env, signal });
|
||||
if (/^160000 /m.test(gitlinks)) throw new RestorePointSkip("submodules_unsupported");
|
||||
return { indexTree, worktreeTree };
|
||||
}
|
||||
|
||||
async function snapshotCommits(cwd: string, env: Record<string, string>, temporary: string, modes: RestorePointSnapshot["modes"], signal?: AbortSignal): Promise<RestorePointSnapshot> {
|
||||
const head = await gitText(cwd, ["rev-parse", "--verify", "HEAD^{commit}"], { signal });
|
||||
const headTree = await gitText(cwd, ["rev-parse", "HEAD^{tree}"], { signal });
|
||||
const { indexTree, worktreeTree } = await snapshotTrees(cwd, env, temporary, signal);
|
||||
if (indexTree === headTree && worktreeTree === headTree) return { head, index: null, worktree: null, modes: [] };
|
||||
const index = await gitText(cwd, ["commit-tree", indexTree, "-p", head, "-m", "paperclip restore point index"], { env, signal });
|
||||
const worktree = await gitText(cwd, ["commit-tree", worktreeTree, "-p", head, "-p", index, "-m", "paperclip restore point worktree"], { env, signal });
|
||||
return { head, index, worktree, modes };
|
||||
}
|
||||
|
||||
const UNBOUNDED_LIMITS: RestorePointLimits = {
|
||||
maxFiles: Number.POSITIVE_INFINITY,
|
||||
maxFileBytes: Number.POSITIVE_INFINITY,
|
||||
maxTotalBytes: Number.POSITIVE_INFINITY,
|
||||
maxCommits: Number.POSITIVE_INFINITY,
|
||||
maxBundleBytes: Number.POSITIVE_INFINITY,
|
||||
};
|
||||
|
||||
export async function currentRestorePointSnapshot(input: string, signal?: AbortSignal): Promise<RestorePointSnapshot> {
|
||||
const cwd = await fs.realpath(input);
|
||||
const { modes } = await scanChangedPaths(cwd, UNBOUNDED_LIMITS, signal);
|
||||
const temporary = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-restore-point-"));
|
||||
try {
|
||||
const { env } = await snapshotStore(cwd, temporary);
|
||||
return await snapshotCommits(cwd, env, temporary, modes, signal);
|
||||
} finally {
|
||||
await fs.rm(temporary, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
async function existingCommits(cwd: string, commits: readonly string[], signal?: AbortSignal) {
|
||||
const present: string[] = [];
|
||||
for (const commit of commits) {
|
||||
if (!SHA.test(commit)) continue;
|
||||
if (await git(cwd, ["cat-file", "-e", `${commit}^{commit}`], { signal }).then(() => true, () => false)) present.push(commit);
|
||||
}
|
||||
return present;
|
||||
}
|
||||
|
||||
async function unpreservedHistory(cwd: string, head: string, boundary: readonly string[], signal?: AbortSignal) {
|
||||
const candidates = new Set<string>();
|
||||
const reflog = await gitText(cwd, ["reflog", "show", "--format=%H", "HEAD"], { signal }).catch(() => "");
|
||||
for (const line of reflog.split("\n")) if (SHA.test(line)) candidates.add(line);
|
||||
const original = await gitText(cwd, ["rev-parse", "--verify", "-q", "ORIG_HEAD^{commit}"], { signal }).catch(() => "");
|
||||
if (SHA.test(original)) candidates.add(original);
|
||||
candidates.delete(head);
|
||||
if (candidates.size === 0) return [];
|
||||
const unreachable = new Set((await gitText(cwd, ["rev-list", "--stdin"], {
|
||||
signal, input: [...candidates, `^${head}`, ...boundary.map((commit) => `^${commit}`)].join("\n") + "\n",
|
||||
})).split("\n").filter((line) => SHA.test(line)));
|
||||
const preserved = [...candidates].filter((commit) => unreachable.has(commit));
|
||||
if (preserved.length > MAX_PRESERVED_COMMITS) throw new RestorePointSkip("restore_point_history_too_large");
|
||||
return preserved.sort();
|
||||
}
|
||||
|
||||
function bundlePrerequisites(bundle: Buffer) {
|
||||
const headerEnd = bundle.indexOf("\n\n");
|
||||
if (headerEnd < 0) throw new RestorePointSkip("invalid_bundle");
|
||||
return bundle.subarray(0, headerEnd).toString("utf8").split("\n").slice(1)
|
||||
.filter((line) => line.startsWith("-")).map((line) => line.slice(1, 41)).filter((sha) => SHA.test(sha)).sort();
|
||||
}
|
||||
|
||||
function encodePayload(manifest: RestorePointManifest, bundle: Buffer) {
|
||||
const manifestBytes = Buffer.from(JSON.stringify(manifest));
|
||||
const length = Buffer.alloc(4);
|
||||
length.writeUInt32BE(manifestBytes.length);
|
||||
return Buffer.concat([PAYLOAD_MAGIC, length, manifestBytes, bundle]);
|
||||
}
|
||||
|
||||
export function decodeRestorePointPayload(payload: Buffer): { manifest: RestorePointManifest; bundle: Buffer } {
|
||||
if (payload.length < PAYLOAD_MAGIC.length + 4 || !payload.subarray(0, PAYLOAD_MAGIC.length).equals(PAYLOAD_MAGIC)) {
|
||||
throw new RestorePointRestoreFailure("invalid_payload");
|
||||
}
|
||||
const length = payload.readUInt32BE(PAYLOAD_MAGIC.length);
|
||||
const start = PAYLOAD_MAGIC.length + 4;
|
||||
if (start + length > payload.length) throw new RestorePointRestoreFailure("invalid_payload");
|
||||
const manifest = JSON.parse(payload.subarray(start, start + length).toString("utf8")) as RestorePointManifest;
|
||||
const shas = [manifest.head, ...(manifest.index ? [manifest.index] : []), ...(manifest.worktree ? [manifest.worktree] : []),
|
||||
...manifest.preserved, ...manifest.prerequisites];
|
||||
if (manifest.schema !== RESTORE_POINT_PAYLOAD_SCHEMA || shas.some((sha) => !SHA.test(sha))
|
||||
|| (manifest.index === null) !== (manifest.worktree === null)
|
||||
|| manifest.modes.some((entry) => typeof entry.path !== "string" || path.isAbsolute(entry.path)
|
||||
|| entry.path.split("/").includes("..") || !Number.isInteger(entry.mode) || entry.mode < 0 || entry.mode > 0o777)) {
|
||||
throw new RestorePointRestoreFailure("invalid_payload");
|
||||
}
|
||||
return { manifest, bundle: payload.subarray(start + length) };
|
||||
}
|
||||
|
||||
export async function captureRestorePoint(input: {
|
||||
cwd: string;
|
||||
upstreamCommits: readonly string[];
|
||||
limits?: Partial<RestorePointLimits>;
|
||||
signal?: AbortSignal;
|
||||
}): Promise<{ payload: Buffer; manifest: RestorePointManifest } | null> {
|
||||
const limits = { ...DEFAULT_RESTORE_POINT_LIMITS, ...input.limits };
|
||||
const cwd = await fs.realpath(input.cwd);
|
||||
const { modes } = await scanChangedPaths(cwd, limits, input.signal);
|
||||
const temporary = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-restore-point-"));
|
||||
try {
|
||||
const { store, env } = await snapshotStore(cwd, temporary);
|
||||
const snapshot = await snapshotCommits(cwd, env, temporary, modes, input.signal);
|
||||
const boundary = await existingCommits(cwd, input.upstreamCommits, input.signal);
|
||||
const preserved = await unpreservedHistory(cwd, snapshot.head, boundary, input.signal);
|
||||
const headOnUpstream = await (async () => {
|
||||
for (const tip of boundary) {
|
||||
if (await git(cwd, ["merge-base", "--is-ancestor", snapshot.head, tip], { signal: input.signal }).then(() => true, () => false)) return true;
|
||||
}
|
||||
return false;
|
||||
})();
|
||||
if (headOnUpstream && snapshot.worktree === null && preserved.length === 0) return null;
|
||||
const refs: Array<[string, string]> = [["refs/restore/head", snapshot.head]];
|
||||
if (snapshot.index && snapshot.worktree) refs.push(["refs/restore/index", snapshot.index], ["refs/restore/worktree", snapshot.worktree]);
|
||||
preserved.forEach((commit, index) => refs.push([`refs/restore/preserved/${index}`, commit]));
|
||||
for (const [ref, commit] of refs) await git(store, ["update-ref", ref, commit], { env, signal: input.signal });
|
||||
const exclusions = boundary.map((commit) => `^${commit}`);
|
||||
const commitCount = Number(await gitText(store, ["rev-list", "--count", ...refs.map(([ref]) => ref), ...exclusions], { env, signal: input.signal }));
|
||||
if (commitCount > limits.maxCommits) throw new RestorePointSkip("restore_point_too_large");
|
||||
if (commitCount === 0) return null;
|
||||
const bundlePath = path.join(temporary, "restore.bundle");
|
||||
await git(store, ["bundle", "create", "-q", bundlePath, "--stdin"], {
|
||||
env, signal: input.signal, input: [...refs.map(([ref]) => ref), ...exclusions].join("\n") + "\n",
|
||||
});
|
||||
const bundle = await fs.readFile(bundlePath);
|
||||
if (bundle.length > limits.maxBundleBytes) throw new RestorePointSkip("restore_point_too_large");
|
||||
const prerequisites = bundlePrerequisites(bundle);
|
||||
const manifest: RestorePointManifest = {
|
||||
schema: RESTORE_POINT_PAYLOAD_SCHEMA,
|
||||
head: snapshot.head,
|
||||
index: snapshot.index,
|
||||
worktree: snapshot.worktree,
|
||||
preserved,
|
||||
prerequisites,
|
||||
modes: snapshot.modes,
|
||||
};
|
||||
return { payload: encodePayload(manifest, bundle), manifest };
|
||||
} finally {
|
||||
await fs.rm(temporary, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
export async function importRestorePoint(input: { repositoryCwd: string; payload: Buffer; signal?: AbortSignal }) {
|
||||
const { manifest, bundle } = decodeRestorePointPayload(input.payload);
|
||||
const temporary = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-restore-point-"));
|
||||
try {
|
||||
const bundlePath = path.join(temporary, "restore.bundle");
|
||||
await fs.writeFile(bundlePath, bundle, { mode: 0o600 });
|
||||
await git(input.repositoryCwd, ["bundle", "verify", "-q", bundlePath], { signal: input.signal })
|
||||
.catch(() => { throw new RestorePointRestoreFailure("bundle_prerequisites_missing"); });
|
||||
const heads = (await gitText(input.repositoryCwd, ["bundle", "list-heads", bundlePath], { signal: input.signal }))
|
||||
.split("\n").filter(Boolean).map((line) => line.split(" "));
|
||||
const expected = new Map<string, string>([["refs/restore/head", manifest.head]]);
|
||||
if (manifest.index && manifest.worktree) {
|
||||
expected.set("refs/restore/index", manifest.index);
|
||||
expected.set("refs/restore/worktree", manifest.worktree);
|
||||
}
|
||||
manifest.preserved.forEach((commit, index) => expected.set(`refs/restore/preserved/${index}`, commit));
|
||||
if (heads.some(([sha, ref]) => expected.get(ref ?? "") !== sha)) throw new RestorePointRestoreFailure("bundle_manifest_mismatch");
|
||||
const listed = new Set(heads.map(([, ref]) => ref));
|
||||
for (const [ref, commit] of expected) {
|
||||
if (listed.has(ref)) continue;
|
||||
const present = await git(input.repositoryCwd, ["cat-file", "-e", `${commit}^{commit}`], { signal: input.signal }).then(() => true, () => false);
|
||||
if (!present) throw new RestorePointRestoreFailure("bundle_manifest_mismatch");
|
||||
}
|
||||
if (listed.size > 0) {
|
||||
await git(input.repositoryCwd, ["fetch", "-q", "--no-tags", "--no-write-fetch-head", bundlePath,
|
||||
...[...listed].map((ref) => `${ref}:refs/paperclip-restore-import/${ref!.slice("refs/restore/".length)}`)], { signal: input.signal });
|
||||
}
|
||||
await git(input.repositoryCwd, ["for-each-ref", "--format=delete %(refname)", "refs/paperclip-restore-import/"], { signal: input.signal })
|
||||
.then((output) => git(input.repositoryCwd, ["update-ref", "--stdin"], { input: output, signal: input.signal }));
|
||||
return manifest;
|
||||
} finally {
|
||||
await fs.rm(temporary, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
export async function applyRestorePoint(input: { cwd: string; manifest: RestorePointManifest; signal?: AbortSignal }) {
|
||||
const { manifest } = input;
|
||||
const cwd = await fs.realpath(input.cwd);
|
||||
const head = await gitText(cwd, ["rev-parse", "--verify", "HEAD^{commit}"], { signal: input.signal });
|
||||
if (head !== manifest.head) throw new RestorePointRestoreFailure("head_mismatch");
|
||||
if (manifest.index && manifest.worktree) {
|
||||
const diff = await git(cwd, ["diff", "--binary", "--no-color", "--no-ext-diff", "--no-textconv", "--no-renames", "HEAD", manifest.worktree], { signal: input.signal });
|
||||
if (diff.length > 0) await git(cwd, ["apply", "--binary", "--whitespace=nowarn", "-"], { input: diff, signal: input.signal });
|
||||
await git(cwd, ["read-tree", `${manifest.index}^{tree}`], { signal: input.signal });
|
||||
}
|
||||
for (const entry of manifest.modes) {
|
||||
const target = path.join(cwd, entry.path);
|
||||
if (path.relative(cwd, target).startsWith("..")) throw new RestorePointRestoreFailure("invalid_payload");
|
||||
const stat = await fs.lstat(target).catch(() => null);
|
||||
if (stat?.isFile()) await fs.chmod(target, entry.mode);
|
||||
}
|
||||
const restored = await currentRestorePointSnapshot(cwd, input.signal);
|
||||
if (restored.head !== manifest.head || restored.index !== manifest.index || restored.worktree !== manifest.worktree
|
||||
|| JSON.stringify(restored.modes) !== JSON.stringify(manifest.modes)) {
|
||||
throw new RestorePointRestoreFailure("restored_contents_mismatch");
|
||||
}
|
||||
}
|
||||
|
||||
function additionalData(identity: RestorePointIdentity) {
|
||||
return Buffer.from(JSON.stringify([identity.companyId, identity.workspaceId, identity.restorePointId, identity.generation]));
|
||||
}
|
||||
|
||||
export async function sealRestorePoint(payload: Buffer, identity: RestorePointIdentity): Promise<SealedRestorePoint> {
|
||||
const dataKey = randomBytes(32);
|
||||
const iv = randomBytes(12);
|
||||
const cipher = createCipheriv("aes-256-gcm", dataKey, iv);
|
||||
cipher.setAAD(additionalData(identity));
|
||||
const ciphertext = Buffer.concat([SEALED_MAGIC, iv, cipher.update(payload), cipher.final(), cipher.getAuthTag()]);
|
||||
const { material } = await localEncryptedProvider.createVersion({
|
||||
value: JSON.stringify({
|
||||
schema: KEY_SCHEMA,
|
||||
...identity,
|
||||
dataKey: dataKey.toString("base64"),
|
||||
plaintextSha256: createHash("sha256").update(payload).digest("hex"),
|
||||
plaintextBytes: payload.length,
|
||||
}),
|
||||
} as Parameters<typeof localEncryptedProvider.createVersion>[0]);
|
||||
return { ciphertext, sha256: createHash("sha256").update(ciphertext).digest("hex"), key: material };
|
||||
}
|
||||
|
||||
export async function openRestorePoint(input: {
|
||||
ciphertext: Buffer;
|
||||
sha256: string;
|
||||
key: StoredSecretVersionMaterial;
|
||||
identity: RestorePointIdentity;
|
||||
}): Promise<Buffer> {
|
||||
if (createHash("sha256").update(input.ciphertext).digest("hex") !== input.sha256) {
|
||||
throw new RestorePointRestoreFailure("ciphertext_hash_mismatch");
|
||||
}
|
||||
const minimum = SEALED_MAGIC.length + 12 + 16;
|
||||
if (input.ciphertext.length < minimum || !input.ciphertext.subarray(0, SEALED_MAGIC.length).equals(SEALED_MAGIC)) {
|
||||
throw new RestorePointRestoreFailure("invalid_ciphertext");
|
||||
}
|
||||
const unwrapped = JSON.parse(await localEncryptedProvider.resolveVersion({ material: input.key, externalRef: null } as Parameters<typeof localEncryptedProvider.resolveVersion>[0]));
|
||||
const { identity } = input;
|
||||
if (unwrapped.schema !== KEY_SCHEMA || unwrapped.companyId !== identity.companyId || unwrapped.workspaceId !== identity.workspaceId
|
||||
|| unwrapped.restorePointId !== identity.restorePointId || unwrapped.generation !== identity.generation) {
|
||||
throw new RestorePointRestoreFailure("key_identity_mismatch");
|
||||
}
|
||||
const iv = input.ciphertext.subarray(SEALED_MAGIC.length, SEALED_MAGIC.length + 12);
|
||||
const tag = input.ciphertext.subarray(input.ciphertext.length - 16);
|
||||
const body = input.ciphertext.subarray(SEALED_MAGIC.length + 12, input.ciphertext.length - 16);
|
||||
const decipher = createDecipheriv("aes-256-gcm", Buffer.from(unwrapped.dataKey, "base64"), iv);
|
||||
decipher.setAAD(additionalData(identity));
|
||||
decipher.setAuthTag(tag);
|
||||
let payload: Buffer;
|
||||
try {
|
||||
payload = Buffer.concat([decipher.update(body), decipher.final()]);
|
||||
} catch {
|
||||
throw new RestorePointRestoreFailure("ciphertext_authentication_failed");
|
||||
}
|
||||
if (payload.length !== unwrapped.plaintextBytes || createHash("sha256").update(payload).digest("hex") !== unwrapped.plaintextSha256) {
|
||||
throw new RestorePointRestoreFailure("plaintext_hash_mismatch");
|
||||
}
|
||||
return payload;
|
||||
}
|
||||
Reference in new issue
Block a user