Files
PaperClipAI/server
DottaandPaperclip 5c3dce09b5 fix(runner): preserve verified remote artifacts across controller releases (#17512)
## Thinking Path

> - Paperclip manages AI agents and owns their task authority.
> - Remote native agents can keep working while their controller
restarts.
> - The controller must authenticate the same runner and verify its
provider files before recovery.
> - The current path compares a retained provider pack with the new
controller release. A release change can reject intact files.
> - The new controller binary digest can also reject the original daemon
during PRP authentication.
> - This PR records the admitted release authority in private controller
state and uses it for exact process reattachment.

## Linked Issues or Issue Description

**What happened?**

A remote Claude session started and used tools. After controller
shutdown, recovery failed with
`runner_remote_provider_artifact_incompatible`. The transport combined
stderr into stdout, so the error omitted the verifier reason. The
historical rejected manifest is unavailable. This PR reproduces
controller release drift as a cause; it does not claim that the
historical mismatch is proven. Reaching pack verification implies the
earlier exact-process/controller-identity checks passed. That supports a
preserved-state recovery path but is not a direct inspection of the
deployed volume.

**Expected behavior**

A compatible, unchanged remote runner can reattach after a controller
release changes. Full package hashes, exact process scope, protocol
capabilities, and authenticated runner identity must still pass. Fresh
launches use the current release.

**Steps to reproduce**

1. Verify and admit a remote native session with release A.
2. Preserve its runner process and replace the controller with release
B.
3. Reattach the original process. The old path compares its pack with B
and fails at `manifest mismatch`.
4. With this fix, the controller reads its saved A authority and
verifies all A hashes. Changed files still fail.

**Paperclip version or commit**

Reproduced from master `470cb34c0e42cb9d0b232d0bcf5d45cc7b0a81ca`;
rebased onto `a3675ddcdf`.

**Deployment mode**

Native remote runner on a Daytona-backed Paperclip Computer. Local
acceptance uses isolated Linux containers and actual packaged artifacts.

Related work: #17361 and #17448 handle runner upgrade and qualification
permissions. #17097 and #17439 add durable controller checkpoints and
server adoption. That stack can skip remote artifact preparation with a
saved checkpoint. This PR fixes the current file-backed path and retains
full provider-pack verification; it has no checkpoint schema or
empty-disk controller recovery.

## What Changed

- Save the complete verified pack manifest and authenticated runner
version/digest after successful session admission.
- Bind the receipt to company, agent, workspace, session, runner,
environment lease, remote directory, and provider descriptor. Keep it
outside guest backup directories.
- Reuse the saved authority for exact reattachment. Keep current release
selection for fresh launches and explicit session replacement.
- Fail closed for mismatched or corrupt receipts. Historical sessions
without a receipt retain the current release integrity gate.
- Report bounded, redacted verifier output from both transport streams.
Warn on receipt write failure while returning the already admitted
session to its owner.
- Add scope, persistence, replacement, storage failure, and backend
wiring regressions. Update recovery documentation and the feature map.

## Verification

- Test-first red: the original selection chose release B instead of A
after controller recreation. The new receipt regression passes. A
separate red test showed the missing stdout diagnostic; it passes after
the diagnostic fix.
- Authority and diagnostic suites: 49 tests pass. Backend pack, process
adoption, and harness persistence selection: 47 tests pass.
- Actual Linux provider image: complete A manifest/artifact verification
passed; controller source-revision drift failed on the old path; saved A
authority passed; a modified ACPX sidecar failed its SHA-256 check.
- Actual native Linux runner: authenticated a session, detached the
controller, changed the controller manifest and binary, then
authenticated the original PID and provider session with saved
authority. There were zero duplicate spawns, zero new provider turns, no
stop/suspend, and the workspace file remained. This probe used the real
runner with a deterministic Codex fixture; it is separate from the real
ACPX package integrity proof.
- Full workspace typecheck and build passed. The full native executor
suite first hit host PostgreSQL semaphore exhaustion. The complete
changed files then passed 864/864 tests with separate real PostgreSQL
databases in a disposable Linux container. The temporary local fixture
changed only database provisioning and used normal repository
migrations. Current-head CI also passed the full required test, Runner,
browser, typecheck, and build lanes.
- `node scripts/check-maps.mjs` and `git diff --check` passed. These
structural checks do not prove live recovery.
- Current head `48fca2ae33d99f7f6d2fe12f431cf76d25185c70`: 55 successful
checks, two conditional skips, no pending or failed checks; fresh
Greptile 5/5 with zero findings and zero unresolved review threads.
Commitperclip quality review passed.
- No original task was retried or changed. Existing Daytona access could
not resolve the original sandbox or its image; a fresh staging Claude
recovery is not claimed.

## Risks

This prevents recurrence for sessions admitted with the new receipt. It
does not retroactively recover the historical failed run. Saved
authority exists only after this change has admitted a session. It
cannot repair an ambiguous historical launch or restore a receipt on an
empty replacement controller. A stale protocol or modified pack remains
a hard failure. Receipt write errors warn without losing the live
session handle, but later recovery still needs matching authority. The
broader checkpoint stack changes the same server preparation and
runner-artifact option and will require a deliberate integration.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools, and code
execution. The runtime does not expose a more specific model ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-24 16:44:50 -07:00
..