Files
PaperClipAI/server
DottaandClaude Opus 5.5 f56b097fbd fix(recovery): move stranded legacy subtasks off a parent's held workspace (#20314)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agents split work into subtasks. Since #19860, every subtask gets
its own execution workspace from the project base, and a subtask that an
older release bound to its parent's workspace moves to its own workspace
on its next admission
> - Some legacy subtasks never get a next admission. Another task held
the shared workspace with unsaved files, so their runs failed setup
before any provider work. Stranded-work recovery then blocked them and
handed them to the board
> - On staging, a parent and two of its children stayed blocked for four
days this way. The children could have run in their own workspace at any
time
> - This pull request makes the scheduled recovery-action sweep move
such a child to its own workspace and continue it, with no person
> - The benefit is that a legacy subtask never stays blocked on a
workspace that it should not share, and the parent keeps its own
workspace

## Linked Issues or Issue Description

Refs #19860 (subtasks get their own workspace; legacy children move at
admission)
Refs #20295 (the held workspace's exhausted save now settles
automatically)
Refs #20273 (stall wakes wait while a workspace hold refuses a run)

**What happened?**
A parent task and three children shared the parent's remote workspace,
because an older release created the children before #19860. One child's
ended run held the workspace with unsaved files. The other two children
were woken, but each run failed setup with "The workspace still has an
unfinished writer or save operation." (`setup_failed`,
`executionRecovery.kind=bootstrap`, `providerWorkStarted=false`).
Stranded-work recovery then set each child to Blocked with a board-owned
escalation (`board_escalation_no_takeover_v1`) that asked a board
operator to retry. The #19860 move happens only at the next admission,
and a blocked child gets no admission.

**Expected behavior**
A subtask that failed only because of a legacy shared-workspace binding
continues in its own workspace from the project base automatically. The
parent and the shared workspace stay unchanged.

**Steps to reproduce**
1. With a release before #19860, create a parent task and children that
share its remote workspace.
2. Let one child's run end with unsaved files whose save is not settled.
3. Wake another child. Its run fails setup on the workspace hold, and
stranded-work recovery blocks it with a board escalation.
4. See that the blocked child is never moved to its own workspace and
never runs again.

**Root cause**
`server/src/services/execution-workspaces.ts`
(`releaseAncestorWorkspaceBindingAtAdmission`) moves a legacy child only
when a run of the child is admitted.
`server/src/services/recovery/service.ts`
(`reconcileStrandedAssignedIssues`) ignores Blocked tasks, so nothing
admits a child that a setup failure on the shared hold blocked.

## What Changed

- `server/src/services/execution-workspaces.ts`: extract the binding
release into `releaseAncestorWorkspaceBinding(tx, …)`, which runs inside
the caller's transaction. Admission uses it unchanged (same guards: no
other running run of the task, no unsettled workspace recovery, only a
relative's workspace).
- New `server/src/services/recovery/legacy-workspace-rehome.ts`
(`rehomeStrandedLegacyWorkspaceTasks`). It selects a board-owned
stranded escalation whose latest run failed setup on the workspace
writer hold before any provider work, for a Blocked task bound to a
relative's workspace. In one transaction it releases the binding,
resolves the escalation with the outcome
`legacy_shared_workspace_rehomed`, sets the task to `todo`, records a
pending wake (`legacyWorkspaceRehome.wake: "pending"`), and records
`issue.legacy_workspace_rehomed` system activity. After commit it wakes
the assignee once (`issue_recovery_action_restored`, with a fresh
session) and marks the wake `delivered` only when a run is created. A
wake lost to a crash, refused (`null`), or failed stays pending and
backs off (5 minutes, doubling, at most an hour); the pass returns the
earliest owed wake as its retry deadline, so the recovery worker
schedules it. Candidates are paged by action id.
- Guards: the Blocked status must come from that escalation (set after
the failed run, and no status decision at or after the block), the
shared dependency-readiness rule must allow work (cancelled blockers and
done blockers that still save files keep it blocked), the failed run
must belong to the current assignee, and the binding release must be
allowed. Otherwise nothing changes.
- `reconcileActiveRecoveryActions` (the company-scoped sweep that the
scheduled issue-recovery worker runs for each company with an active
recovery action) runs this pass first. `recoveryActionSweepCondition()`
also selects a company whose moved task still owes its wake.
- `doc/execution-semantics.md`: documents the move and what a moved
child sees.

### What a moved child sees

A fresh workspace from the project base, exactly like a new top-level
task in the project. It does not get the parent's changes. It does not
get its own earlier files from the shared workspace either, because
nothing of its work ran there (its runs failed setup before any provider
work). The child that still holds the shared workspace with unsaved
files is not moved by this change: its save recovery owns the hold
(#20295 settles it automatically), its files stay in the shared
workspace and in that recovery's saved-work receipts, and it moves on
its first admission after that recovery settles (the #19860 rule).

### Previous-release story (`rules/upgrade-safe-run-state.mdc`)

- **Migrated, no schema change.** The change reads escalations and task
bindings that older releases saved, and it changes only the task's
binding row, the task status, and the escalation. It does not change
saved run identity (native execution input, leases, checkpoints,
sessions). The regression test seeds the exact staging shape (Nov 4–5
evidence fields, `retryReason: workspace_busy`, `latestRunErrorCode:
setup_failed`).
- **Mid-run deploy.** A child with a running run or an unsettled
workspace recovery is not moved (the same guards as the admission
release).

## Verification

- Red (the move disabled), `npx vitest run
src/__tests__/legacy-workspace-rehome.test.ts`: 1 failed, 2 passed. The
staging-shape case failed because the stranded child stayed Blocked and
bound to the parent's workspace.
- Green: 8 of 8 pass. All cases run the scheduled path: the company is
selected by `recoveryActionSweepCondition()` and the pass is
`recoveryService(...).reconcileActiveRecoveryActions({ companyId })`
with the real heartbeat wake.
- Staging shape: a parent and 3 children share the parent's workspace.
One child holds it with a dirty writer run. Two children are blocked by
board escalations after setup failed on that hold. After the
stranded-work sweep, both children are unbound from the shared
workspace, their escalations are resolved by the system with text that
asks no person to act, and each child is woken exactly once with a fresh
session. Each child runs once through the real heartbeat path outside
the shared workspace and completes. The parent and the writer child keep
their bindings, the shared workspace row is unchanged, and nothing runs
for them. A later pass changes nothing.
- A move whose wake was lost to a crash: the scheduled worker still
selects the company, delivers the wake once, and the child runs and
completes.
- A refused wake (`null`) or a thrown enqueue stays pending, backs off,
does not hold up other wakes, is reported to the worker as the next
deadline, and keeps the company selected after a restart; once accepted,
it settles.
- Thirty rejected escalations do not hide a later eligible one (paging).
- A child with a cancelled blocker stays blocked with its board
escalation.
- A child whose last status decision came before the block moves; one
decided after the block stays blocked.
- A stranded child whose setup failed for another reason stays blocked
with its board escalation.
- A task bound to the workspace it created (the parent) is never moved.
- Related suites pass: `subtask-workspace-independence`,
`heartbeat-workspace-busy`, `heartbeat-remote-workspace-busy`,
`heartbeat-held-comment-wake`, `recovery-deadline-acceptance`,
`heartbeat-retry-scheduling`, `task-workspace-binding` (278 tests), and
after the review changes `issue-blocker-stall-wakeups`,
`issue-recovery-deadlines`, `issue-recovery-work-lifecycle`,
`recovery-durable-wait-projection`, `owner-gone-attestation` (124 tests
with the two workspace suites).
- `pnpm verify`: exit 2 ("broader verification required", because
`doc/execution-semantics.md` is a global CI input). Required CI supplies
the broad coverage.
- `npx tsc --noEmit -p server/tsconfig.json`: passes.

## Risks

- The pass changes a Blocked task to `todo` only under narrow guards,
and only for a task bound to a relative's workspace whose last run
failed setup on that workspace's writer hold. A Blocked status from
another decision, or an unresolved blocker, keeps the task blocked.
- The moved child starts again from the project base. This is the
product rule for subtasks. Nothing of its own work is dropped, because
its runs failed before any provider work.
- The pass runs inside the existing active recovery action sweep and
selects at most 25 escalations and 25 pending wakes per pass.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used through Claude
Code with tool use (shell, file edits, browser read-only API checks) and
extended reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 12:15:25 -06:00
..