mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-03 09:37:19 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Accounting, credential recovery, and sandbox cleanup can retain legacy files on an instance volume. > - Replacing that volume while files remain can lose unresolved work. > - There is no verified production backlog, so an empty installation needs no migration. > - The recovery owners must decide whether each record is valid and settled. > - This pull request adds a bounded file inventory and an optional cutover gate. > - The benefit is a safe check that preserves obligations without reading secret material. ## Linked Issues or Issue Description **Problem or motivation** Operators need to check the actual affected instance directories before replacing a web server. A skipped corrupt file or failed read must not appear clear. **Proposed solution** Add a Node script that inspects only the three legacy recovery directories for an explicitly chosen instance. Default mode reports counts. The optional check mode blocks when files or uncertainty remain. **Alternatives considered** A bulk migration system has no established backlog to migrate. Reimplementing the owner parsers could expose credentials and still would not prove settlement. This check never reads file bytes. **Roadmap alignment** This is a small operational prerequisite for safe cloud server replacement. It does not add a recovery backend or change resource ownership. Related recovery-owner changes: #18354 (accounting), #18352 (credentials), and #18357 (sandbox ownership). This inventory remains independently usable with existing files. ## What Changed - Add a pure classification policy and a bounded filesystem boundary. - Add `node scripts/recovery-cutover.mjs --instance-root /absolute/instance` with optional `--check`, directory override, and entry budget. - Report observed lower-bound counts and safe status summaries. Do not print paths, filenames, identifiers, raw errors, or file contents. - Add one dependency-free CLI acceptance suite and a Vitest bridge so the same cases run in normal server CI. - Document existing owner entry points, override selection, stopped producers, and retained-volume requirements. ## Verification - `node --test scripts/recovery-cutover.test.mjs`: 9 cases pass. The suite starts the actual CLI with synthetic legacy directories. - Pending receipts, nested credential files, orphan records, malformed bytes, temporary files, unreadable files, unsafe paths, symlinks, read failures, bounds, and configured overrides all block safely. - Empty or absent recovery directories clear. Source file bytes remain unchanged. Report-only mode and check-mode exit codes are tested. - `pnpm exec vitest run server/src/__tests__/recovery-cutover.test.ts --pool=forks --maxWorkers=1`: the normal CI bridge passes and runs the same 9 cases. - Cached frozen dependency installation, script syntax checks, and `git diff --check` pass. The lockfile is unchanged. - Independent source review found the read-only boundary sound. - [Complete current-head CI](https://github.com/paperclipai/paperclip/actions/runs/41097614666) passed at `bf5cc7f067041c10cd0294ff26756678da943976`: 53 successful checks and two intended skips, including typecheck, full tests, build, and E2E. Greptile is 5/5 with no review threads. - One failed-job retry was needed for an existing heartbeat timing failure in server shard 4. The relevant heartbeat source and tests match the base branch. Local diagnostic runs had 61 pass/2 fail and 7 pass/1 fail, with changing timing failures; neither run is claimed as green. The bounded retry and all failed evidence are [recorded here](https://github.com/paperclipai/paperclip/pull/18347#issuecomment-6529629804). No heartbeat code or test was changed. ## Risks - The scan is a point-in-time inventory. Stop producers for the final gate. It is not an atomic drain certificate or protection against hostile path mutation. - All files remain unverified. The check cannot certify validity or Postgres settlement. Existing owners retain that responsibility. - The scan does not inspect other instances, volumes, Runner delivery, or database obligations. It does not establish that production cutover is ready. - No migration, deletion, decryption, or replay command is added. Symlink paths are rejected, including system aliases; use canonical absolute paths. ## Model Used - OpenAI `gpt-6.1-sol`, with reasoning, tool use, and code execution. Context window size was not supplied. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
113 lines
6.2 KiB
Markdown
113 lines
6.2 KiB
Markdown
# Check existing recovery files before web-server replacement
|
|
|
|
This check covers only observed legacy files on one explicitly selected instance.
|
|
There is no verified production backlog. An empty installation needs no migration.
|
|
This check does not inspect Postgres obligations, Runner delivery, other volumes,
|
|
other instances, or the rest of web-server replacement readiness.
|
|
|
|
Use the retained volume and the instance root from the old server configuration.
|
|
The instance root is the directory that contains `accounting-receipts/` and
|
|
`quota-credential-recovery/`. It is not the parent directory of all instances.
|
|
Use canonical absolute paths. The check rejects symlink components, including
|
|
system aliases such as `/var` on macOS; use the actual `/private/var` path there.
|
|
The chosen instance directory must exist. Missing recovery directories are normal.
|
|
|
|
Run this command from a checkout that includes the check. Node is the only runtime
|
|
dependency. It needs no database connection, master key, or package installation.
|
|
|
|
```sh
|
|
node scripts/recovery-cutover.mjs --instance-root /absolute/retained/instance
|
|
```
|
|
|
|
The default command reports JSON and exits successfully even when work remains.
|
|
For an automation gate, add `--check`:
|
|
|
|
```sh
|
|
node scripts/recovery-cutover.mjs --instance-root /absolute/retained/instance --check
|
|
```
|
|
|
|
| Exit code | Meaning |
|
|
| --- | --- |
|
|
| `0` | Report produced; with `--check`, all three scans are clear |
|
|
| `1` | With `--check`, unresolved files or an unknown/incomplete scan block cutover |
|
|
| `2` | Invalid arguments or the inventory could not run |
|
|
|
|
The sandbox directory defaults to `data/sandbox-orphan-cleanup/` below the chosen
|
|
instance. If the old server uses `SANDBOX_ORPHAN_CLEANUP_SPOOL_DIR`, supply that same
|
|
environment value when running the check. An explicit `--orphan-directory` takes
|
|
precedence. A relative or empty override is rejected; supply its canonical absolute
|
|
path for this check. An override is an additional explicitly selected root and can
|
|
be outside the instance directory. The check does not search for other overrides.
|
|
|
|
```sh
|
|
node scripts/recovery-cutover.mjs --instance-root /absolute/retained/instance \
|
|
--orphan-directory /absolute/retained/orphans --check
|
|
```
|
|
|
|
The report gives one summary for each of the three recovery kinds. It never
|
|
prints filenames, input paths, file contents, identifiers, token values, decrypted
|
|
material, or raw filesystem errors. `unverifiedFiles` includes every regular file,
|
|
including JSON, temporary, probe, truncated, corrupt, and unreadable files. The
|
|
check does not read file bytes and cannot claim that a file is valid or settled.
|
|
That decision belongs to the existing recovery owner and its format definitions.
|
|
|
|
Only the credential directory's immediate UUID company directories are traversed;
|
|
empty company directories are normal after recovery. Accounting and orphan files
|
|
use flat directories. Unexpected directories, deeper nesting, symlinks, and
|
|
special entries count as `unknownEntries` and block. The check does not follow
|
|
symlinks or inspect unexpected subtrees. Directory/stat/close failures increment
|
|
`readFailures` or leave the scan incomplete. An invalid or missing instance root
|
|
also blocks; it is not an empty installation.
|
|
|
|
The default shared budget is 10,000 observed entries across all three scans,
|
|
including company directories. `--max-entries` accepts 1 through 100,000. At the
|
|
limit the scanner reads at most one extra entry per open directory to detect
|
|
truncation. An unfinished scan reports `complete: false` and status `unknown`.
|
|
Counts are observed lower bounds, not total records or total pending operations.
|
|
A complete scan with any files reports `blocked`; any uncertainty reports
|
|
`unknown`. Only a complete empty scan reports `clear`.
|
|
|
|
For the final cutover gate, stop the old producers first and retain the old volume.
|
|
Run the check after the relevant recovery owners finish their work and producers
|
|
are stopped again. A live scan is a point-in-time inventory, not an atomic drain
|
|
certificate. Concurrent renames or writes invalidate the snapshot; the tool does
|
|
not lock writers or protect against a hostile process that changes paths during
|
|
inspection. Do not use it to certify a live writable or untrusted volume.
|
|
|
|
If files exist, keep the old volume and let the existing owners settle actual
|
|
entries with their existing permissions and database checks:
|
|
|
|
- Accounting: `run-cost-accounting.ts` uses the `usage-receipts.ts` replay and
|
|
per-run recovery paths. The receipt owner validates the accounting envelope and
|
|
handles database conflicts and settlement.
|
|
- Credentials: `ai-connections.ts` and `ai-connection-runtime.ts` use
|
|
`quota-credential-recovery.ts` during account credential access and refresh.
|
|
The owner validates encrypted material with the original instance master key
|
|
and preserves grant, company and credential version fences.
|
|
- Orphan cleanup: `environment-runtime.ts` exposes `flushDeferredOrphanCleanups`
|
|
to the existing cleanup lifecycle. `sandbox-orphan-cleanup-spool.ts` owns the
|
|
record format and removes a file only after the recovery path saves ownership.
|
|
|
|
These are service entry points, not standalone operator commands. This change
|
|
adds no import, deletion, replay, decryption, or bulk migration command. Do not
|
|
trigger new paid runs or token exchanges merely to empty the report. If an owner
|
|
cannot settle a record, preserve it and record an explicit reconciliation outcome
|
|
through the existing incident/operations process. Corrupt files need owner review;
|
|
they do not disappear from the gate because a recovery reader skipped them.
|
|
Never delete unresolved files to obtain a clear result. An approved exception
|
|
requires separate evidence; this tool continues to block while the files remain.
|
|
Retain the volume and required secrets through rollback compatibility review.
|
|
|
|
Verify the check without any real instance data:
|
|
|
|
```sh
|
|
node --test scripts/recovery-cutover.test.mjs
|
|
```
|
|
|
|
The tests use disposable synthetic directories and spawn the actual CLI. They
|
|
cover pending legacy layouts, malformed bytes, temporary files, empty directories,
|
|
symlinks, overrides, scan limits, safe diagnostics, failure injection, exit codes,
|
|
and preservation of source bytes. They establish file inventory behavior only.
|
|
They do not establish that any production volume is clear or that domain recovery
|
|
has completed.
|