Files
PaperClipAI/doc/plugins
Nicky LeachandClaude Opus 5.5 960a2b781e perf(workspaces): hold kept plugin workspaces instead of re-inspecting them every sweep (#22054)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Workspace storage plugins hold task workspaces, and Core's terminal
workspace sweep removes them after their tasks finish
> - The sweep inspects each candidate workspace through the provider. A
provider inspection can start a sandbox or call a remote API, so each
one has a real cost
> - When an inspection fails, the sweep backs off. When an inspection
succeeds but keeps the workspace (local data, or work that is not
merged), there is no backoff, so the sweep inspects the same workspace
again on every pass, forever
> - Kept workspaces also kept a past-due sweep deadline, so targeted
sweeps came back to them every few minutes, not only once per hourly
rotation
> - With many closed workspaces, this repeated work grows without limit
and gives the same answer each time
> - This pull request records a hold after an inspection keeps a
provider workspace, and skips the provider inspection while nothing that
could change the answer has changed
> - The benefit is far fewer provider operations for kept workspaces,
with no delay for workspaces that can be removed

## Linked Issues or Issue Description

**Subsystem affected**
Server: the terminal workspace sweep for workspace storage plugins
(`server/src/services/execution-workspaces.ts`).

**Problem or motivation**
The terminal workspace sweep inspects every kept plugin workspace again
on every pass. For a workspace kept for local data, the answer cannot
change until something changes the workspace. For a clean, unmerged
workspace, the answer changes only when the remote changes. Each
repeated inspection is a provider operation, which can start a sandbox
or call a remote API.

**Proposed solution**
After an inspection keeps a plugin workspace, Core records a hold. A
workspace kept for local data is not inspected again until it or its
task tree changes, or a change notice names it. A clean, unmerged
workspace is checked again after 1 hour, then on an interval that
doubles up to 24 hours. Workspaces that can be removed are removed as
before.

**Alternatives considered**
An in-memory hold only. It is lost on restart, and the existing
in-memory inspection cache holds only 256 entries, so a large set of
kept workspaces would be inspected again after every restart.

## What Changed

- `server/src/services/terminal-workspace-retry.ts`: add the
`terminalInspectionHold` workspace metadata record (generation,
fingerprint, reason, HEAD, check count, check time, next check time).
The write is fenced on the lifecycle generation. Add the recheck delay:
1 hour, doubling, capped at 24 hours.
- `server/src/services/execution-workspaces.ts`
(`sweepTerminalWorkspaces`):
- Before it inspects a plugin workspace, the sweep compares a
fingerprint of the inputs that can change the answer: the row's
`updatedAt`, status, provider ref, repository, branch, base ref,
provider binding, delivery candidate, and the task tree's terminal
state. The lifecycle generation scopes the record. While the hold
applies, the sweep skips the inspection and counts the workspace as
`skippedBackoff`.
- After an inspection keeps the workspace, the sweep records a
`local_data` hold (dirty or untracked files, or local-only data; no
recheck time) or an `unmerged` hold (growing recheck interval). A new
inspected HEAD, a new fingerprint, or a change notice restarts the
interval.
- A kept workspace now gets a sweep deadline at its recheck time, or
none. Before, it kept its past-due archive deadline, so targeted sweeps
came back to it every few minutes.
- A provider refresh notice (`plugin-workspaces`: activation, upgrade,
capability change, company re-enablement) ends every hold, because it
can change how any plugin workspace inspects.
- A `workspace:` or `issue:` change notice ends the holds that it names
(the workspace, or a workspace whose task tree or linked tasks contain
the issue). A notice without a scope ends every hold. The next sweep
visit clears an ended hold from the row, so the reset survives a
restart. The in-memory notice entries are pruned after the next full
rotation.
- When an inspection finds the workspace removable, the sweep clears the
hold and continues to removal as before.
- The hold applies only to plugin workspaces. Local Git worktree
inspections are cheap, and a person can change those folders directly,
so they keep the current behavior.
- Core owns the new metadata key. Workspace metadata updates keep it,
like `terminalCleanupRetry`.
- `doc/plugins/PLUGIN_RUNNER_EXECUTION.md`: describe the hold for
provider authors.
- Optional item not done: `removePluginWorkspace` still inspects again
before it removes. That inspection runs right before a destructive
operation, passes the recovery PR hint to the provider, and checks
`remotelyRecoverable`, which the sweep's inspection result does not
carry. Skipping it is not clearly safe.

## Verification

- `npx vitest run
src/__tests__/plugin-workspace-terminal-hold.integration.test.ts` (new,
embedded Postgres):
- Local-data hold: no inspection after 30 days; a workspace notice, an
issue notice, a generation bump, a metadata update, a new task
completion time, and a provider refresh notice each cause exactly one
new inspection; no sweep deadline is left. The provider refresh step
fails without its fix (6 inspections, not 7).
- Unmerged backoff: recheck delays are 1, 2, 4, 8, 16, 24, 24 hours; a
new HEAD resets to 1 hour; a workspace notice resets to 1 hour.
- Removable workspaces: a clean merged workspace is removed on its first
sweep (inspection count unchanged at 2); a held dirty workspace is
removed on the next sweep after a notice, and the hold is cleared; a
closed held workspace is removed after an unscoped notice.
- Failure backoff: delays stay 5 and 10 minutes, and a failed inspection
does not record a hold.
- Red check: the five new tests fail against the unchanged service and
pass with the change.
- Related suites pass locally (10 files, 204 tests):
`execution-workspaces-service`, `plugin-workspaces.integration`,
`terminal-workspace-deadlines`, `terminal-workspace-idle`,
`terminal-workspace-page-batch`, `terminal-workspace-rotation-limits`,
`terminal-workspace-work`, `workspace-delivery-cleanup.integration`,
`plugin-workspace-terminal-retry.integration`, and the new file.
- A scratch check confirmed the old deadline behavior: after a sweep
kept a workspace, the old code left it due with a past deadline. The new
code leaves no deadline.
- `plugin-workspace-terminal-retry.integration.test.ts`: one step now
sends a workspace notice before it checks that a new failure restarts
the backoff at 1, because the preceding successful inspection now holds
the dirty workspace.
- `cd server && npx tsc --noEmit -p .` passes.

## Risks

- A held `local_data` workspace is inspected again only after a change.
If a change notice is lost (for example a restart between the commit and
the notice) and the change does not touch the workspace row, its
generation, binding, status, delivery candidate, or task tree terminal
state, the workspace stays held until one of those changes. Runs end
with an `issue:` notice and new task completion times, so the normal
paths are covered.
- An `unmerged` workspace that becomes merged on the remote waits until
its next recheck (at most 24 hours), not the next hourly rotation.
- Each sweep page now runs one extra query for the linked tasks of held
workspaces. Each kept workspace gets one metadata write per inspection.
- No schema migration. The record is in existing workspace metadata.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, test runs).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 13:40:35 -08:00
..