Files
PaperClipAI/doc
DottaandPaperclip b7ef307a65 fix(workspaces): wait for an ended writer's save instead of failing setup (#19074)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent runs can use a remote task workspace in a sandbox. One writer
run at a time holds that workspace.
> - When a writer run ends, its workspace save and sandbox release
finish a few minutes later. The writer hold stays until then.
> - A new run that started in that interval failed with `setup_failed`:
"The workspace still has an unfinished writer or save operation."
> - A live writer already makes the new run wait (`workspace_busy`,
added in #18691). An ended writer that is still settling did not.
> - This pull request reports the ended writer as the holder too, so the
new run waits instead of failing.
> - The benefit is that work continues automatically after a short
settle interval. Users do not see a failed run and a blocked task.

## Linked Issues or Issue Description

Refs #18691

**What happened?**
A task run failed during workspace setup with `setup_failed` and the
error "The workspace still has an unfinished writer or save operation."
The previous run on the same task had ended a few seconds before. Its
sandbox lease was released 4 to 17 minutes later. A staging soak
recorded 31 of these failures over 14 tasks in about 29 hours.

**What did you expect to happen?**
The new run waits for the workspace, as it does when the previous writer
is still running. It continues when the save and release settle.

**Steps to reproduce**
1. Bind a task to a remote workspace whose authority has `dirty: true`
and a `writerRunId` for a run with status `failed` that ended now.
2. Wake the task's agent.
3. The run fails with `setup_failed`. No retry is scheduled.

## What Changed

- `reserveRemoteWorkspace` now throws `RemoteWorkspaceWriterActiveError`
for an ended writer whose save or sandbox release has not settled.
Before, it threw a plain 409 conflict. The error keeps the same message,
status and `remote_workspace_writer_pending` code. It uses the writer's
`finishedAt` as its last activity and sets `ended: true`.
- Workspace preparation already converts that error to a
`workspace_busy` deferral with the one-hour holder cutoff. It now passes
`holderEnded` to the deferral.
- `WorkspaceBusyDeferral` says "ended, but its workspace save has not
settled" for an ended holder, instead of "is still running".
- A writer that is missing or has no task still gets the plain conflict,
as before.
- `doc/execution-semantics.md` describes the wait and its cutoff.

## Verification

- Red before the fix: the new test "waits for a writer that just ended
while its save or sandbox release settles" failed. The run had `status:
failed`, `errorCode: setup_failed`, `executionRecovery.kind: bootstrap`.
This is the same result as the staging failures.
- Green after the fix:
`cd server && npx vitest run
src/__tests__/heartbeat-remote-workspace-busy.test.ts
src/services/remote-workspace-reservation.test.ts`
  (16 tests pass.)
- Safety cases in the new tests:
- The waiting run does not change the workspace authority. The unsaved
writer's files and hold stay unchanged.
- A writer that ended more than one hour ago still causes the
unfinished-writer setup failure, with no scheduled retry. The wait is
bounded.
- Existing cases still pass: live writer wait with Stop and
reassignment, the concurrent reservation race
(`remote_workspace_writer_pending`), and the unused-reservation
takeover.
- Also run: `remote-workspaces.integration.test.ts`,
`heartbeat-workspace-busy.test.ts`,
`remote-workspace-save-concurrency.test.ts`,
`workspace-recovery.test.ts`. All 6 suites pass (442 tests).
- Not done: a live run on staging at this revision. The fix is not
deployed.

## Risks

- A task whose ended writer never settles now makes up to about one hour
of short `workspace_busy` waits (one scheduled retry every 1 to 2
minutes) before it fails. Before, it failed at once. The end state after
the cutoff is the same.
- Some of the staging failures have other causes that this change does
not fix:
- A previous run that stays `running` with no output for more than one
hour still causes this failure, as intended.
- A dirty workspace whose writer ended without a saved result (for
example a workspace finalization retry that never runs) still blocks the
task after the cutoff. These need separate fixes.
- No schema, API or UI changes.

## Model Used

- Anthropic Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with
tool use (shell, browser reads of staging run records) and extended
reasoning.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-11-04 17:39:26 -06:00
..
…
…
…
…
…