mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-08 12:33:05 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work.
> - Remote agent runs call a sandbox provider plugin, such as Daytona,
through a plugin worker process.
> - Sometimes that worker is not running. It can be stopped, restarting
after a crash, or retired by the host.
> - Then the host refuses the call before it sends anything to the
worker.
> - Setup recorded this refusal as a generic `setup_failed`. It
scheduled no retry, so the task became stranded and needed a person.
> - This pull request types the refusal and gives it its own transient
setup code. The bounded bootstrap retry then runs.
> - The benefit is that the task continues on its own when the worker is
available again. Users do not see a blocked task.
## Linked Issues or Issue Description
Refs #19150 (removed one trigger for the worker retirement)
Refs #19245 (restarts a worker after a fail-closed retirement)
**What happened?**
A plugin worker stopped. Each run that then called `environmentExecute`
during setup failed with `Cannot call "environmentExecute" — worker for
"<plugin>" is stopped`. The run got `setup_failed`, no retry was
scheduled, and the task moved to board recovery. On one staging
instance, 15 runs failed this way in 12 minutes.
**Expected behavior**
No provider work started, so the failure is safe to retry. The run must
retry within the bootstrap budget, as it does for a reusable sandbox
resume failure or a preparation timeout.
**Steps to reproduce**
1. Stop the sandbox provider plugin worker, or let the host retire it.
2. Wake an agent whose run needs that provider during setup.
3. The run fails with `setup_failed`. No retry is scheduled.
**Paperclip version or commit**
master at 76e43852.
**Deployment mode**
Authenticated cloud deployment (staging).
## What Changed
- Add `PluginWorkerUnavailableError` with the code
`plugin_worker_unavailable`
(`server/src/services/plugin-worker-unavailable-error.ts`).
- The plugin worker manager now rejects with this error when the worker
is stopped, crashed, restarting, not running, or not registered. The
message text does not change.
- Setup failure handling maps this error to `plugin_worker_unavailable`,
but only before provider dispatch and only for a legacy run with no
native session or runner instance. These are the same guards as for the
sibling transient codes.
- `isTransientWorkspacePreparationCode` includes the new code, so setup
schedules the bounded bootstrap retry.
- Legacy execution recovery and the recovery service treat the code like
the other bootstrap-only codes. The stranded-task notice has a title and
a next action for it.
## Verification
- Red: the new `run-completion` test failed on the unchanged code. A
fresh refusal got `setup_failed` and no retry.
- Green: the same test passes. A fresh refusal gets
`plugin_worker_unavailable`, `providerWorkStarted: false` and a bounded
retry. A started, untrusted (message-only) or native run keeps
`setup_failed` and gets no retry.
- New worker manager test: a stopped handle and a missing worker both
reject with the typed error. The call is not sent, and `onWorkerSettled`
runs once.
- `pnpm --filter @paperclipai/server exec tsc --noEmit -p .` passes.
- Related suites pass: run-completion, plugin-worker-manager,
legacy-execution-recovery, heartbeat-retry-scheduling and recovery (17
files, 439 tests).
## Risks
- Low risk. The retry uses the existing bounded bootstrap budget. When
the budget is exhausted, the existing board recovery action takes over.
- The retry applies only before provider dispatch. A worker that stopped
after work started keeps the old behavior.
- A wrapper that drops the error `cause` keeps the old `setup_failed`
behavior. It does not cause a wrong retry.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code
execution in Claude Code.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
31 lines
1.3 KiB
TypeScript
31 lines
1.3 KiB
TypeScript
/**
|
|
* Setup failure code for a call that reached a plugin worker which was not
|
|
* running (stopped, crashed, restarting, or not registered). No request was
|
|
* sent to the worker, so the run is retried with the bounded bootstrap budget
|
|
* once the worker is available again.
|
|
*/
|
|
export const PLUGIN_WORKER_UNAVAILABLE_CODE = "plugin_worker_unavailable";
|
|
|
|
/** A host call was refused because its plugin worker was not running. */
|
|
export class PluginWorkerUnavailableError extends Error {
|
|
readonly code = PLUGIN_WORKER_UNAVAILABLE_CODE;
|
|
readonly pluginId: string;
|
|
/** The worker status at refusal, or null when no worker was registered. */
|
|
readonly workerStatus: string | null;
|
|
|
|
constructor(input: { pluginId: string; workerStatus: string | null; message: string }) {
|
|
super(input.message);
|
|
this.name = "PluginWorkerUnavailableError";
|
|
this.pluginId = input.pluginId;
|
|
this.workerStatus = input.workerStatus;
|
|
}
|
|
}
|
|
|
|
/** Find a worker refusal through the orchestrator's error wrappers. */
|
|
export function findPluginWorkerUnavailableError(error: unknown): PluginWorkerUnavailableError | null {
|
|
for (let cause = error, depth = 0; cause && depth < 4; cause = (cause as { cause?: unknown }).cause, depth++) {
|
|
if (cause instanceof PluginWorkerUnavailableError) return cause;
|
|
}
|
|
return null;
|
|
}
|