Files
PaperClipAI/doc
DottaandPaperclip 0819f4ea78 fix(native): verify controller ownership before retrying saved turns (#19638)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Native task turns can keep a provider conversation from an earlier
run.
> - A copied conversation checkpoint names the new run before its
controller has attached.
> - A failed attachment can make recovery treat that copied identity as
current execution authority.
> - The current run needs its own durable controller checkpoint, or a
valid staged attachment, before exact resume.
> - This pull request checks that ownership and uses the normal Retry
path to continue a saved user turn after verified cleanup.
> - The task can continue once with its saved work and history.

## Linked Issues or Issue Description

Refs #19399 and #18860.

**What happened?**

A new user comment resumed a completed native conversation. The retained
runner process was gone before the new controller attached. The new run
still had a copied provider session ID. Recovery selected exact resume
for that new run and then failed its unchanged owner and lease checks.
Ordinary first-attempt failure settlement also omitted the saved-user
Retry handoff that restart recovery already uses.

**Expected behavior**

Exact resume requires current controller evidence. A saved user turn can
request one normal Retry after this infrastructure failure. The normal
path must verify provider stop, saved files, current user authority and
all execution gates before it starts a fresh session.

**Steps to reproduce**

1. Finish a native warm run and retain its provider checkpoint.
2. Admit a user comment as a new run and copy the provider checkpoint
through the real resume producer.
3. Stop the retained runner before attachment. The real remote process
check reports that the process is not running.
4. Observe that the copied provider ID selects exact current-run resume.
Observe that ordinary failure settlement saves no Retry request.

## What Changed

- Check a bounded encrypted controller header for fenced remote Runner
recovery. Keep local Runner recovery on its existing disk-backed path.
Match the run, conversation and runner identity. Preserve the existing
staged-attachment path.
- Share the existing saved-user Retry admission between restart recovery
and ordinary terminal native settlement. Keep its exact infrastructure
errors, user wake, comment, provider-turn and retry-chain checks.
- Add real PostgreSQL and stopped-process regressions. Verify malformed
and foreign checkpoint holds, current-owner resume, user authority and
Stop. Verify saved-file recovery creates one fresh-session successor.
- Document the ownership and Retry contract.

## Verification

- RED: the real checkpoint rebind producer and stopped-process check
selected exact current-run resume without a current controller
checkpoint.
- RED: ordinary settlement created zero Retry receipts for the real
native failure code; expected one.
- Review RED: a real current local controller file existed with no
database checkpoint rows. The initial gate returned ambiguous state. The
fix takes a required trusted execution-target kind and applies the
database requirement only to the remote persistence path.
- Initial local revision: `3662055880ccf8e09c3c50a340897e16c78627e7`,
macOS ARM64, Node 26.4.0, pnpm 9.15.4, embedded PostgreSQL.
- Initial server Vitest: 42 selected cases passed across controller
recovery integration, remote restart adoption, ordinary run completion
and retained workspace recovery. The cases cover the stopped-process
producer, current/foreign/malformed ownership, automatic user
continuation, Stop, unsettled turns, system wakes and a manual Retry on
first use.
- `pnpm --filter @paperclipai/server typecheck`: passed with this
worktree's Runner, shared and SDK builds and all seven server layers.
- `node scripts/check-module-boundaries.mjs`, whitespace checks and the
added-code-comment audit: passed.
- Review correction: `8c795e2e0e44b5209da3e228593c5cd35e948cae`. The
real local/remote controller case passed against a fresh PostgreSQL
cluster. It verifies local disk recovery without database rows, remote
current-owner resume, and remote missing/foreign/malformed holds. Fresh
`node scripts/typecheck-layers.mjs` passed all seven server layers. The
existing offline chat reader now requests metadata only and retains its
full independent native-state safety check.
- A broader local recheck hit PostgreSQL setup/cleanup hook timeouts.
These are retained as failed setup attempts; the fresh-cluster
regression above passed. Required CI supplies broad final-head coverage.
- Settlement-order correction:
`dafb721c3bbb7ae86fbc3c9c6c953ace29ca088b`. The real PostgreSQL
regression first proved both failures: Stop after an already-failed row
created one Retry instead of zero; a DB-native promotion with an old
legacy caller snapshot created zero instead of one. All six settlement
scenarios now pass. The hook requires the local failed outcome and the
canonical settled native row. Fresh seven-layer server typecheck passed
again.
- Required shared fixture base integrated at
`d22d8f9d5ee3dc36550a94d55c1d8376f3b9aab2`. The six runner-skills
fixture cases and all seven server typecheck layers passed. Required CI
passed for this head:
[workflow](https://github.com/paperclipai/paperclip/actions/runs/42309419592),
including all server layers, tests, build, Runner verification and E2E.
The final snapshot has 57 successful checks, three intended skips and
one neutral result. Current-head Greptile is 5/5; both concrete review
threads are resolved. The independent re-review ended without a final
response; its prior Stop finding has a real RED/GREEN repair.

The focused command was run from `server/`:

```sh
../node_modules/.bin/vitest run --config vitest.config.ts src/services/heartbeat/run-completion.test.ts src/services/native-runtime/native-controller-recovery.integration.test.ts src/services/native-runtime/native-session-executor.test.ts src/services/workspace-recovery.test.ts -t 'replays saved user intent|does not assign current-run resume authority|remote controller restart adoption|continues a user turn once|does not continue automatically'
```

## Risks

- A terminal failure of a fenced Runner adds one exact-run header read.
The read selects one encrypted header and limits inspection to 1 MiB.
Invalid or missing ownership evidence stays held.
- Saved user input can request one automatic Retry. It uses normal
preservation, current authorization, budget, approval, Stop and cleanup
gates. Unsettled provider turns and system wakes do not start an
automatic successor.
- This PR changes no provider process, credential, model or lease
authority checks.

## Model Used

OpenAI GPT-6 family through Codex, with reasoning, tool use and code
execution. The model is inherited; no override was used. The exact
serving model ID and context-window size are not exposed by this
runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-06 17:31:23 -06:00
..
…
…
…
…
…