Files
PaperClipAI/server/src/services
DottaandPaperclip d2efe3392b fix(native): preserve retired authority evidence (#19806)
## Thinking Path

> - Paperclip manages work that AI agents perform for users.
> - Native sessions retain control-plane files when execution authority
moves to a later run.
> - Volume migration must preserve these historical bytes before a
server can stop depending on its old volume.
> - The existing preservation owner archives the current controller and
supervisor, but refuses a nonempty authority-epoch directory.
> - A settled four-run session can therefore remain blocked after its
current controller archive is complete.
> - This pull request adds a closed historical inventory to the existing
encrypted evidence archive.
> - The benefit is preserved source evidence and database-only
replacement validation without granting execution or recovery authority.

## Linked Issues or Issue Description

Refs #19748. A search of pull requests for filesystem independence and
issues for authority-epoch history found no duplicate change. This fixes
its uncovered historical-file case.

**What happened?**

A settled native session retained three prior controller files in
authority-epoch directories. Its three historical database checkpoint
headers were absent. Maintenance preserved the current controller, then
refused filesystem independence because those historical files remained
uncovered.

**Expected behavior**

Preserve the exact historical bytes and modes through the existing
encrypted evidence owner. Keep original files, prior archive rows, run
outcomes, warnings, stamps, session pointers, and recovery policy
unchanged. Validate the resulting receipt on a replacement without the
source volume.

**Steps to reproduce**

Use the included four-run fixture with a current-controller-only
archive, three prior authority epochs, destroyed provider receipts,
released ownership, and absent historical checkpoint headers. Call the
canonical filesystem-independence owner. The base rejects the nonempty
directory; the treatment preserves the complete evidence and validates
it after removing source-volume identity from the test process.

**Paperclip version/commit**

Baseline `ef83d566cacf1d5f740d701560597dad2bfa4619`; final branch
rebased onto `27170dca49`.

**Deployment mode**

Original-volume maintenance followed by an ephemeral replacement. This
PR performs no hosted migration or provider operation.

## What Changed

- Add optional closed authority-epoch entries to the existing
`original_volume_source_evidence` archive. Preserve nested file bytes
and modes through the existing encryption, readback, ledger, audit, and
retention owners.
- Bind each epoch to its retained run, canonical checkpoint identity
when present, or explicit header absence plus the exact admitted
authority binding. Keep current-source checkpoint and artifact checks
strict. Historical evidence never becomes an executable checkpoint.
- Recheck the complete historical inventory at archive and receipt
publication fences. Keep physical locks, all-run ownership and
termination checks, source capability checks, and final CAS.
- Bind the history inventory and its database facts into the
authenticated archive manifest. Replacement validation uses database
bindings only. Later header presence, removal, retirement, or changed
identity invalidates the receipt.
- Leave history-free manifest bytes and receipt shapes unchanged. Keep
an existing current-controller-only archive when a later complete
archive is added. Unknown children and other nonempty history
directories remain blockers.

## Verification

- Node 24.22.2, PostgreSQL 18.1, macOS arm64. The final four-run fixture
fails on the baseline owner at its nonempty-history guard. The same
final fixture and treatment pass all 51 focused PostgreSQL/filesystem
cases.
- The positive case performs actual encryption, object-store
persistence, authenticated restoration, and exact byte/mode comparisons.
It proves replacement database-only validation with absent source paths
and storage reads refused.
- Controls cover foreign identities, unknown children, owned prior runs,
changed bytes or new files after upload, missing or newly present
historical headers, malformed and retired headers, an unknown authority
override, receipt-identity tampering, and a late concurrent winner.
Existing current-controller evidence and history-free manifest bytes
remain intact.
- The baseline/treatment driver restored only its two owned source files
in `finally` and verified their exact hashes. Earlier setup and
treatment failures remain recorded: missing bundled PostgreSQL library
links, an overly narrow raw-schema check, and an incorrect synthetic
transient-workspace binding. The final baseline and treatment use the
corrected same fixture; those earlier attempts are not counted as
passes.
- Review follow-up `21183b75f4` closes the missing-history-field gap.
Deleting history from an actual published receipt reproduced an invalid
true result before the fix. Unconditional manifest reconstruction now
rejects it; all 46 owner cases pass. Genuine history-free manifests
retain their exact bytes.
- Review follow-up `f3d305dd36` adds suspended historical Runner-state
bytes and mode checks through authenticated restore, active/foreign
state refusals, exact 16 MiB and 64 MiB boundaries, and retention of an
oversized file without publication. All 51 cases and server TypeScript
pass; production behavior is unchanged.
- CI run `42362784772` on the previous head had one unrelated 30-second
timeout in `manual-private-cell-report-assets.test.ts`, with 6,864
support tests passing. The failure remains recorded. The prior head
`21183b75f4b223cd3ec28124cdc194f24b4f79ba` passes [required CI run
42363408227](https://github.com/paperclipai/paperclip/actions/runs/42363408227),
including Runner support: 55 successful checks and four intended skips,
plus successful Snyk. The final test/documentation head
`f3d305dd3690a7db38b57c4a77389c17206cb76b` passes [required CI run
42364958386](https://github.com/paperclipai/paperclip/actions/runs/42364958386):
55 successful checks, four intended skips, one neutral review check, and
successful Snyk. Fresh Greptile is 5/5; all four review threads are
resolved. No timeout or assertion was weakened.
- The count warning is resolved as independent bounds: the actual
descriptor accepts 32 control-plane-only epochs (97 entries) and 32
epochs with 31 Runner states (128 entries), and rejects 129 entries. The
32-epoch and 128-entry maxima both apply; the capture budget is
unchanged.
- Server TypeScript and module-boundary checks pass. The local
verification planner selects ten broad application lanes; required CI
supplies that wider coverage instead of duplicating it locally.
- Reproduce the focused owner suite from `server`:

```sh
node ../cli/node_modules/tsx/dist/cli.mjs ../packages/db/src/test-postgres-command.ts -- node ../node_modules/vitest/vitest.mjs run src/services/native-runtime/native-harness-filesystem-independence.integration.test.ts --hookTimeout 90000 --maxWorkers 1 --fsModuleCache
```

## Risks

This changes a maintenance trust boundary. A source capture is limited
to 128 entries, 16 MiB per file, and 64 MiB total. These limits are
intentionally below the Runner's 256 MiB state maximum. Larger valid
epochs remain on the original volume and continue blocking filesystem
independence. It uses bounded directory reads and file descriptors,
rejects aliases and unknown paths, and rechecks bytes before
publication. Storage failures retain original files and ledger evidence;
they do not settle recovery or authorize a retry.

Startup adds bounded database reads for retained historical header
bindings or their absence. It adds no source filesystem or storage
reads. The selected cohort's serving startup cost still needs
measurement before enrollment. Other historical layouts require their
own canonical source qualification. No missing provider history is
declared recovered, and no automatic corruption repair is added.

## Model Used

OpenAI Codex, GPT-6 family, with reasoning, source editing, shell tools,
and PostgreSQL/filesystem test execution. The runtime does not expose a
more specific serving model ID or context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-07 04:55:18 -06:00
..
…
…
…
…