mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
## Thinking Path
> - Paperclip is the open source app people use to manage AI agents for
work
> - A project can store its task workspaces with a workspace storage
provider plugin. Core calls the plugin's `workspaceOperation` method to
prepare, inspect, finalize and remove a workspace.
> - `callWorkspaceProvider` catches every rejection of that call and
throws the same conflict: "Workspace storage operation is incomplete;
retry the same workspace".
> - A provider error, an RPC timeout, a crashed worker and a worker that
never got the call all look the same in the run, the log and the API.
The message tells the user to retry, also when the failure is a fixed
configuration problem that a retry cannot fix.
> - In one diagnosis, three different configuration failures in a row
showed this same message. Finding each cause took trace span counts and
database reads.
> - This pull request lets a provider send a reason from a closed set,
and makes Core tell transport failures apart from provider failures.
> - The benefit is that a failed run says what failed and whether a
retry can help. Provider text and values still never cross into Core.
## Linked Issues or Issue Description
**What happened?**
A workspace provider's `workspaceOperation` failed. The run failed with
"Workspace storage operation is incomplete; retry the same workspace",
`errorCode: setup_failed`, and no reason in the run record or the
`heartbeat execution setup failed` log. Retrying did not help, because
the cause was a provider configuration problem.
**Expected behavior**
Core tells these cases apart: the worker did not receive the call, the
call timed out, the worker stopped during the call, or the provider
reported a failure. For a provider failure, Core shows a bounded reason.
Core tells the user to retry only when a retry can help.
**Steps to reproduce**
1. Install a workspace storage provider plugin whose
`onWorkspaceOperation` throws during `prepare` (for example, because of
missing provider settings).
2. Run an agent task in a project that uses that provider.
3. See the failed run: the error is the generic retry message, and the
run record and logs do not show the cause.
**Paperclip version / commit**
`master` at 5ac05871b5.
**Deployment mode**
Any deployment with a workspace storage provider plugin.
## What Changed
- **SDK reason contract.** New
`packages/plugins/sdk/src/workspace-operation-error.ts`:
`WORKSPACE_OPERATION_FAILURE_REASONS` (11 reasons),
`PluginWorkspaceOperationError`, `workspaceOperationErrorData` and
`readWorkspaceOperationFailureReason`. It uses the same envelope design
as `environment-task-error.ts`. The worker host attaches the envelope
only to `workspaceOperation` errors. It copies only the reason, never
the message.
- **Duck-typed reason.** The worker host also forwards an own
`workspaceFailureReason` property from any error, when the value is in
the closed set. A provider that builds against an older SDK can set the
reason before it can import the class. An inherited or unknown value is
ignored. The value can only select one of Core's fixed messages.
- **Core classification.** New
`server/src/services/workspace-provider-failure.ts`.
`callWorkspaceProvider` maps each rejection to a
`WorkspaceProviderFailureError` (409) with `kind`, `reason` and
`retryable`:
- `worker_unavailable`: no request was sent. The
`PluginWorkerUnavailableError` stays as the cause, so run setup retries
it as `plugin_worker_unavailable` (the existing bounded bootstrap
retry).
- `timeout` and `worker_exited`: the outcome is unknown, so the message
still says to retry the same workspace.
- `transport`: any other rejection. The message is the old message.
- `provider` with a reason: a fixed message per reason. Only transient
reasons say to retry.
- `provider` without a known reason: "Workspace storage provider
reported a failure; retry the same workspace".
- **Run settlement.** `failRunSetup` records
`resultJson.workspaceProviderFailure` (`kind`, `reason`, `retryable`).
The `heartbeat execution setup failed` log carries the same fields and
the run's `errorCode`. The run keeps `errorCode: setup_failed` (except
for the worker-not-running case above), so the existing setup-failure
retry, recovery and chat paths do not change.
- **Span allowlist.** The host accepts a `plugin.workspace.failure` span
and a `plugin.failure_reason` attribute. The value must be a reason from
the closed set or `unknown`.
- **Docs.** `doc/plugins/PLUGIN_RUNNER_EXECUTION.md` describes the
failure kinds and the reasons. `doc/observability.md` lists the new span
and attribute.
## Verification
- `cd packages/plugins/sdk && npx vitest run`: 21 files, 261 tests pass.
The new `tests/workspace-operation-error.test.ts` sends errors through
the real worker RPC host.
- `cd server && npx vitest run
src/__tests__/workspace-provider-failure.test.ts
src/__tests__/plugin-host-services-span.test.ts
src/services/heartbeat/run-completion.test.ts`: 3 files, 118 tests pass.
These tests cover:
- every transport kind, and that a worker refusal is still found as the
cause;
- a real worker RPC rejection with a reason, and without a reason;
- a fixed message and retry decision for every reason;
- span clamping of `plugin.failure_reason`;
- run setup for a provider reason (no retry, reason in `resultJson` and
in the log) and for a stopped worker (bounded retry).
- `cd server && npx vitest run` for the 4 plugin workspace integration
suites (`plugin-workspaces`, `plugin-fresh-workspaces`,
`project-workspace-provider`, `workspace-delivery-cleanup`): 91 tests
pass. Their lost-reply fixtures still get the old "incomplete" message.
- `npx tsc --noEmit` in `packages/plugins/sdk`: no errors. `npx tsc
--noEmit -p server/tsconfig.json` shows no errors in the changed files.
The remaining errors come from workspace packages that I did not build
locally.
- Not run locally: the full `pnpm -r typecheck`, `pnpm test:run` and
`pnpm build`. CI runs them.
## Risks
- **Behavior change for a stopped worker.** A `workspaceOperation` call
that the worker did not receive now gets `errorCode:
plugin_worker_unavailable` and the bounded bootstrap retry, as other
plugin calls already do. Before, it was a `setup_failed` run with no
retry. No request reached the provider, so the retry is safe.
- **Message changes.** Provider-reported failures and timeouts get new
fixed messages. Code that matches the exact old message only sees it for
`transport` and `remote_incomplete` failures. I did not find a match on
this message outside tests.
- **No new error codes.** Provider failures keep `setup_failed`. Retry,
recovery and chat-retry routing does not change.
- **Duck typing.** Any error with an own `workspaceFailureReason` from
the closed set is forwarded. A provider can only select one of Core's
fixed messages. It cannot send text or values.
- **Observability.** The new span attribute is a closed set and stays
inside the host allowlist. No Telemetry change.
## Model Used
- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, through Claude Code
with tool use (shell, file edits, test runs). Extended thinking was on.
## Checklist
- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge
Related:
[paperclipai/paperclip#19326](https://github.com/paperclipai/paperclip/pull/19326)
added the same envelope design for Runner revision mismatches. I found
no duplicate PR for this change.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>