Files
PaperClipAI/ui/src/components/RunWorkspaceRecoverySurface.tsx
T
f3ca4d24bc fix: repair dirty/foreign-branch execution worktrees (#9297)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Execution workspaces are the bridge between Paperclip's control
plane and a local agent's checked-out repository state.
> - When a workspace is restored after a failed or interrupted run, the
recorded branch can disagree with the branch currently checked out on
disk.
> - A clean branch mismatch can be reconciled safely, but a dirty
mismatch needs a lossless path that does not discard uncommitted agent
work.
> - This pull request adds a quarantine-and-restore path that saves
dirty work to a rescue branch, restores the recorded branch, and exposes
the repair from the board UI and run page.
> - The benefit is that operators can recover wedged execution
workspaces without losing work or moving another live branch
unexpectedly.

## Linked Issues or Issue Description

No public GitHub issue exists for this workspace-recovery failure, so
this PR includes the bug report inline.

**What happened**

A git worktree-backed execution workspace could become wedged when
Paperclip expected one branch but found a different checked-out branch
with dirty tracked or untracked files. The existing safe repair path
refused the restore, leaving the source task blocked with no lossless
one-click recovery path.

**Expected behavior**

Paperclip should preserve dirty work before restoring the recorded
workspace branch. If another live workspace claims the checked-out
branch, or an attached runtime service is active, the repair should
refuse with clear operator-facing evidence instead of risking work loss
or file contention.

**Steps to reproduce**

Create a git worktree execution workspace whose persisted branch name
differs from the checked-out branch, add dirty tracked or untracked
files in that worktree, then trigger workspace validation or use the
branch reconcile endpoint. Before this change, the dirty mismatch
remained blocked because Paperclip had no quarantine restore mode.

**Paperclip version or commit**

Observed on the pre-fix workspace-recovery implementation. Verified on
this PR head after rebasing onto current `master`.

**Deployment mode**

Local trusted development/worktree deployments using git worktree
execution workspaces and optional workspace runtime services.

## What Changed

- Added dirty-worktree quarantine repair that creates a rescue branch,
commits dirty tracked and untracked files there, restores the recorded
branch, writes audit comments/activity, and preserves the live foreign
branch ref.
- Added `quarantine_restore` branch reconcile API support,
recovery-action resolution, source-task wake behavior, execution-review
preservation, claimant refusal, runtime-service refusal, and coverage
for the non-transactional git ordering.
- Added board UI controls for the repair action in the recovery card
plus a compact failed-run workspace recovery surface that uses the same
reconcile handlers.
- Hardened Greptile follow-up cases by best-effort restoring the
recorded branch after a mid-sequence rescue commit failure and by
refusing quarantine restore while attached runtime services are active.

## Verification

- `pnpm exec vitest run
server/src/__tests__/execution-workspaces-service.test.ts -t
"quarantine_restore"`
- `pnpm exec vitest run server/src/__tests__/workspace-runtime.test.ts
-t "workspace dirty quarantine branch repair"`
- `pnpm exec vitest run
server/src/__tests__/heartbeat-workspace-finalize-branch.test.ts -t
"repairs clean unrecorded branch drift|adopts unrecorded forward branch
drift"`
- `pnpm exec vitest run
server/src/__tests__/workspace-runtime-routes-authz.test.ts`
- `pnpm --filter @paperclipai/server typecheck`
- Earlier PR verification covered the route, service, heartbeat, UI
component, and run-page recovery surfaces; Cutter posted public preview
screenshots for the repair popover and run-page panel at
https://github.com/paperclipai/paperclip/pull/9297#issuecomment-4926934211.
- GitHub PR checks are green on
`dcac76b05f4cf6e1ee16544c2831d83c7857e475`.
- Greptile is 5/5 with zero annotations and no unresolved review threads
on `dcac76b05f4cf6e1ee16544c2831d83c7857e475`.

## Risks

- Moderate risk because the change intentionally runs git commands
against local worktrees; the implementation refuses dirty repair when
another claimant or active runtime service is detected and records
rescue refs for auditability.
- Compatibility / release-note callout for self-hosted operators:
existing instances that left `enableWorkspaceBranchReconcileForward`
unset now get automatic forward branch reconciliation during heartbeat
workspace recovery. Operators who want the previous advisory-only
behavior can set `experimental.enableWorkspaceBranchReconcileForward` to
`false`; dirty quarantine repair can likewise be disabled with
`experimental.enableWorkspaceDirtyQuarantineRepair: false`.
- If the git rescue succeeds but a later database write fails, the
worktree may already be restored while the recovery action remains open;
this ordering is documented in code because git side effects cannot
participate in the database transaction.
- UI risk is limited to the workspace recovery surfaces and covered by
component tests plus the existing Cutter visual preview.

## Model Used

OpenAI Codex coding agent based on GPT-5, with repository tool use,
shell execution, and local test execution. Earlier preserved commits on
this branch also show Claude Code / Claude Opus 4.8 assistance in their
commit metadata.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] Branch naming exception documented: this PR preserves the existing
worktree branch requested for publication while keeping the PR title and
body public-facing
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 18:19:49 -05:00

290 lines
11 KiB
TypeScript

import { useCallback } from "react";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import type { HeartbeatRun } from "@paperclipai/shared";
import { useNavigate } from "@/lib/router";
import { issuesApi } from "../api/issues";
import { executionWorkspacesApi } from "../api/execution-workspaces";
import { accessApi } from "../api/access";
import { queryKeys } from "../lib/queryKeys";
import { useToastActions } from "../context/ToastContext";
import {
IssueRecoveryActionCard,
type RecoveryReissueRequest,
type RecoveryResolveOutcome,
} from "./IssueRecoveryActionCard";
import {
canBoardManageRuntime,
readRecoveryReconcileWorkspaceId,
} from "../lib/recovery-reconcile";
/** The run errorCode Paperclip stamps when it declines a run over a git workspace it can't validate. */
export const WORKSPACE_VALIDATION_RUN_ERROR_CODE = "workspace_validation_failed";
function asRecord(value: unknown): Record<string, unknown> | null {
return value && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: null;
}
function asNonEmptyString(value: unknown): string | null {
if (typeof value !== "string") return null;
const trimmed = value.trim();
return trimmed.length > 0 ? trimmed : null;
}
/**
* Reads the source issue id a failed run was working when it was declined over workspace validation.
* The run's context snapshot pins the issue the recovery action lives on.
*/
function readRunIssueId(run: HeartbeatRun): string | null {
const context = asRecord(run.contextSnapshot);
if (!context) return null;
return asNonEmptyString(context.issueId);
}
/**
* Run-page recovery surface. When a run *failed* with workspace-validation evidence, this fetches
* the source issue's active recovery action and renders the same `IssueRecoveryActionCard`
* (compact) that `IssueDetail` shows — wired to the same reconcile-forward / repair / re-issue /
* break-glass / resolve handlers — so the divergence can be resolved from the run view directly.
*
* Renders nothing unless the run is a workspace-validation failure whose source issue still carries
* a live `workspace_validation` recovery action.
*/
export function RunWorkspaceRecoverySurface({ run }: { run: HeartbeatRun }) {
const navigate = useNavigate();
const queryClient = useQueryClient();
const { pushToast } = useToastActions();
const isWorkspaceValidationFailure =
run.status === "failed" && run.errorCode === WORKSPACE_VALIDATION_RUN_ERROR_CODE;
const issueId = readRunIssueId(run);
const { data: issue } = useQuery({
queryKey: queryKeys.issues.detail(issueId ?? "__none__"),
queryFn: () => issuesApi.get(issueId!),
enabled: Boolean(isWorkspaceValidationFailure && issueId),
});
const { data: boardAccess } = useQuery({
queryKey: queryKeys.access.currentBoardAccess,
queryFn: () => accessApi.getCurrentBoardAccess(),
enabled: Boolean(isWorkspaceValidationFailure && issueId),
retry: false,
});
const recoveryAction = issue?.activeRecoveryAction ?? null;
const canManageBoardRuntime = canBoardManageRuntime(run.companyId, boardAccess);
// Prefer the workspace pinned by the recovery action's evidence (the workspace that actually
// diverged) over the page-level id, which can drift after a re-issue rebinds the issue.
const reconcileWorkspaceId =
readRecoveryReconcileWorkspaceId(recoveryAction) ?? issue?.executionWorkspaceId ?? null;
const invalidate = useCallback(() => {
if (issueId) {
queryClient.invalidateQueries({ queryKey: queryKeys.issues.detail(issueId) });
}
queryClient.invalidateQueries({ queryKey: queryKeys.runIssues(run.id) });
queryClient.invalidateQueries({ queryKey: queryKeys.runDetail(run.id) });
}, [issueId, queryClient, run.id]);
const reconcile = useMutation({
mutationFn: (
input:
| { workspaceId: string; mode: "forward" }
| { workspaceId: string; mode: "override"; reason: string }
| { workspaceId: string; mode: "quarantine_restore" },
) => {
const { workspaceId, ...body } = input;
return executionWorkspacesApi.reconcile(workspaceId, body);
},
onSuccess: (_result, variables) => {
invalidate();
pushToast(
variables.mode === "quarantine_restore"
? {
title: "Workspace repaired",
body: "Dirty changes were quarantined onto a rescue branch and the recorded branch restored; the task will resume.",
tone: "success",
}
: {
title: "Workspace branch reconciled",
body: "The recorded branch now matches the live branch; the task will resume.",
tone: "success",
},
);
},
onError: (err) => {
pushToast({
title: "Reconcile failed",
body: err instanceof Error ? err.message : "Unable to reconcile the workspace branch.",
tone: "error",
});
},
});
const reissue = useMutation({
mutationFn: async (request: RecoveryReissueRequest) => {
if (!issue) throw new Error("Task is not loaded yet.");
const sourceLabel = issue.identifier ?? "the stalled task";
const descriptionLines = [
`Re-issued from ${sourceLabel} on an isolated git worktree after a workspace branch divergence.`,
"",
`- Base ref (live branch): \`${request.baseRef}\``,
...(request.expectedBranch ? [`- Recorded branch: \`${request.expectedBranch}\``] : []),
"",
"---",
"",
issue.description ?? "",
];
return issuesApi.create(issue.companyId, {
title: `Re-issue (isolated): ${issue.title ?? sourceLabel}`,
description: descriptionLines.join("\n"),
priority: issue.priority,
projectId: issue.projectId ?? null,
parentId: issue.parentId ?? null,
assigneeAgentId:
recoveryAction?.returnOwnerAgentId ??
recoveryAction?.previousOwnerAgentId ??
issue.assigneeAgentId ??
null,
executionWorkspacePreference: "isolated_workspace",
executionWorkspaceSettings: {
mode: "isolated_workspace",
workspaceStrategy: { type: "git_worktree", baseRef: request.baseRef },
},
});
},
onSuccess: (created) => {
invalidate();
pushToast({
title: "Isolated re-issue created",
body: created.identifier
? `${created.identifier} will run on a fresh isolated workspace.`
: "A fresh isolated re-issue was created.",
tone: "success",
});
if (created.identifier) {
navigate(`/issues/${created.identifier}`);
}
},
onError: (err) => {
pushToast({
title: "Re-issue failed",
body: err instanceof Error ? err.message : "Unable to create an isolated re-issue.",
tone: "error",
});
},
});
const resolve = useMutation({
mutationFn: (data: {
outcome: "restored" | "false_positive";
sourceIssueStatus: "todo" | "done" | "in_review";
}) => {
if (!issueId || !recoveryAction) throw new Error("No recovery action to resolve.");
return issuesApi.resolveRecoveryAction(issueId, {
actionId: recoveryAction.id,
outcome: data.outcome,
sourceIssueStatus: data.sourceIssueStatus,
});
},
onSuccess: () => {
invalidate();
},
onError: (err) => {
pushToast({
title: "Recovery resolution failed",
body: err instanceof Error ? err.message : "Unable to resolve recovery action",
tone: "error",
});
},
});
const handleReconcileForward = useCallback(() => {
if (!reconcileWorkspaceId) return;
void reconcile.mutateAsync({ workspaceId: reconcileWorkspaceId, mode: "forward" });
}, [reconcile, reconcileWorkspaceId]);
const handleBreakGlass = useCallback(
(reason: string) => {
if (!reconcileWorkspaceId) return;
void reconcile.mutateAsync({ workspaceId: reconcileWorkspaceId, mode: "override", reason });
},
[reconcile, reconcileWorkspaceId],
);
const handleQuarantineRestore = useCallback(() => {
if (!reconcileWorkspaceId) return;
void reconcile.mutateAsync({ workspaceId: reconcileWorkspaceId, mode: "quarantine_restore" });
}, [reconcile, reconcileWorkspaceId]);
const handleReissue = useCallback(
(request: RecoveryReissueRequest) => {
void reissue.mutateAsync(request);
},
[reissue],
);
const handleResolve = useCallback(
(outcome: RecoveryResolveOutcome) => {
switch (outcome) {
case "todo":
void resolve.mutateAsync({ outcome: "restored", sourceIssueStatus: "todo" });
return;
case "done":
void resolve.mutateAsync({ outcome: "restored", sourceIssueStatus: "done" });
return;
case "in_review":
void resolve.mutateAsync({ outcome: "restored", sourceIssueStatus: "in_review" });
return;
case "false_positive_done":
void resolve.mutateAsync({ outcome: "false_positive", sourceIssueStatus: "done" });
return;
case "false_positive_in_review":
void resolve.mutateAsync({ outcome: "false_positive", sourceIssueStatus: "in_review" });
return;
}
},
[resolve],
);
if (!isWorkspaceValidationFailure || !issueId) return null;
if (!recoveryAction || recoveryAction.kind !== "workspace_validation") return null;
return (
<div className="space-y-2" data-testid="run-workspace-recovery-surface">
<div className="flex items-center justify-between gap-2">
<span className="text-xs font-medium text-muted-foreground">Workspace recovery</span>
{issue?.identifier ? (
<a
href={`/issues/${issue.identifier}`}
className="font-mono text-xs text-muted-foreground underline-offset-2 hover:underline"
onClick={(event) => {
event.preventDefault();
navigate(`/issues/${issue.identifier}`);
}}
>
{issue.identifier}
</a>
) : null}
</div>
<IssueRecoveryActionCard
action={recoveryAction}
variant="compact"
onResolve={handleResolve}
onReissueIsolated={handleReissue}
reissuePending={reissue.isPending}
onReconcileForward={handleReconcileForward}
onBreakGlassOverride={handleBreakGlass}
onQuarantineRestore={handleQuarantineRestore}
quarantineRestorePending={reconcile.isPending}
reconcilePending={reconcile.isPending}
canBreakGlass={canManageBoardRuntime}
/>
</div>
);
}
export default RunWorkspaceRecoverySurface;