mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 06:10:44 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - Native runs save their workspace after the result is accepted, under
a copyback owner receipt
> - A deploy can restart the server while that save is in progress
> - The new server runs in a new container, so it cannot see the old
process and fails closed
> - The run then waits on a board action that asks a person to verify
the previous controller
> - Recoveries must continue without a human step, and copy must not ask
anyone to inspect a server
> - This pull request adds a deployment-level rule to the shared
owner-liveness primitive and uses it in workspace finalization
> - The benefit is that a save interrupted by a deploy resumes on its
own, with no second writer
## Linked Issues or Issue Description
**What happened?**
A deploy restarted the server while three runs were saving their
accepted results. Each run kept a copyback owner receipt that names the
old Railway deployment on another container. The restarted server
reported `native_workspace_finalization_owner_unverified`, owned by the
board, with the next action "Verify the previous controller and its
workspace-copyback processes have stopped...". The cloud harness stop
receipt did not arrive, so the runs stayed in "Recovery needed" for
about an hour, until a person confirmed the stop by hand.
**Expected behavior**
The restarted server proves that the previous controller is gone,
resumes the save, and settles the run with no human step. No user-facing
text asks a person to verify a server or controller.
**Steps to reproduce**
1. Start a native run on a Railway service with a volume.
2. Let the result be accepted and the workspace save start.
3. Deploy, so that the server restarts during the save.
4. See that the run waits for a board confirmation.
**Paperclip version or commit**
Observed on e2d490c61. Reproduced on master before this change. The
owner-proof work in #19848, #19851, #19853, #19868, #19871 and #19883
does not cover this shape: `proveOwnerGone` returns `unprovable`
(`different_host`) for an owner on another container.
## What Changed
- `owner-proof.ts`: `LocalOwnerStamp` can carry the platform `worker`
(deployment identity). `ProcessProbe` can carry this process's
deployment and whether the platform gives it an exclusive volume. New
verdict reason `deployment_replaced`.
- New rule in `proveOwnerGone`, after the current-boot check: the owner
is gone when this server runs with a Railway volume
(`RAILWAY_VOLUME_ID`, `RAILWAY_VOLUME_MOUNT_PATH`) and the stamp names
the same project, environment and service with a different deployment
ID. Railway stops the previous deployment before it mounts the volume
into the new one, and a service with a volume has no replicas. So every
process of the old deployment, including orphaned copyback children, has
exited.
- `native-workspace-finalization-ownership.ts`: evaluate that proof
before the cloud harness request. On `deployment_replaced`, claim the
exact owner receipt under the existing advisory lock and row-lock
revalidation, log `issue.workspace_owner_stop_verified` with
`deploymentProof`, mark the recovery `resuming`, and continue the save
with no provider turn. Other cases keep the harness request and the
existing fences.
- New owner receipts also record the mounted volume ID (`volumeId`).
When a receipt has it, the proof requires this server to mount the same
volume. Both `RAILWAY_VOLUME_ID` and `RAILWAY_VOLUME_MOUNT_PATH` must be
set. The finalizer reaches this check only while it holds the run's
copyback advisory lock, so the old owner's lock session has also ended.
- Copy: the hold's next action, the route conflict and the two Retry
wait messages now describe the wait. They no longer ask a person to
verify a server or controller. The hold's text still names the Resume
saved result action for platforms where neither this rule nor the
harness can confirm the stop.
- `doc/native-workspace-finalization-recovery.md`: describe the rule.
Previous-release story: receipts written by earlier releases already
record `worker.deploymentId`, so they carry forward and take this path
after the upgrade. No schema change.
## Verification
- `cd server && npx vitest run
src/services/native-runtime/native-workspace-owner-restart-recovery.test.ts
src/services/owner-proof.test.ts`
- Before the fix: "resumes the interrupted save automatically, with no
board action" fails (`acquired: false`).
- After the fix: all pass.
- The restart test seeds the staging receipt shape (old deployment,
other hostname, PID 3, existing `operator_required` hold). It checks
that the save resumes once, the hold becomes `resuming`, the task
presentation no longer waits for a user, the receipt clears, and no
agent wake is created.
- Negative cases stay fenced: same deployment, a service without a
volume, a half-configured volume, another service, another environment,
and a receipt that names a different volume. Unit tests cover the same
rule in `proveOwnerGone`, including that the current boot stays alive.
- Related suites pass: finalizer, queued-wake recovery, route ownership,
workspace recovery gates and dispatch, explicit continuation, recovery
services (1005 tests).
- `pnpm --filter @paperclipai/server typecheck` passes.
## Risks
- The rule depends on Railway volume exclusivity. It applies only when
this server has a volume mounted and the receipt names the same service
and environment. Without a volume, the existing harness and board paths
still apply.
- A receipt from a deployment that ran before the volume was attached
could, in theory, overlap during that one redeploy. Volumes are attached
at stack creation, so this is not a practical case.
- Copy changes are text only. Error codes are unchanged.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>