Files
PaperClipAI/ui/src/components/IssueRunLedger.tsx
T
DottaandClaude Opus 5.5 e7e28ba241 feat(queue): show tasks waiting for an agent's slot as waiting, with their place in line (#21405)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Each agent runs at most `runtimeConfig.heartbeat.maxConcurrentRuns`
tasks at once. Other runs wait in the agent's queue with status `queued`
> - On hosted staging, one agent had all 20 slots full and 295 queued
runs. Some tasks waited 3 to 7 hours and never started
> - The UI showed these tasks as live or working: a pulsing "Live" badge
in the task header and in task rows, a spinning "Working … for 4 hours"
in the task chat, and "N live" counts that included every queued run
> - People could not tell that work was waiting, why it waited, or what
to change
> - This pull request makes a deep queue an honest, visible state:
"Waiting for Codie · 20 slots max · 41 of 295 in line · Configure"
> - The benefit is that operators see the real state and the one setting
that changes it

## Linked Issues or Issue Description

Refs #21391, #21349, #21242

**What happened?**

A queued run that never started looked the same as a running run. The
task header, the task rows in lists and the inbox, the task chat and the
sidebar counts all showed queued runs as live or working.

**Expected behavior**

A run that waits for one of its agent's concurrency slots shows that it
waits. It names the agent, the agent's limit and the run's place in
line, and it links to where the limit is set. The place in line is the
order in which the server starts the agent's queued runs.

**Why does it matter?**

With a deep queue, the UI said "working" for hours while nothing ran.
Operators could not see the cause or the fix.

## What Changed

- Server: a new module
`server/src/services/heartbeat/queued-run-line.ts` now holds the facts
that the queued-run dispatcher orders by (`queuedRunDispatchFacts`). The
dispatch path in `queue.ts` and the new "place in line" read use this
one builder, so the line the UI shows is the dispatch order. The logic
moved without change, including the recovery-share fix from #21391.
- Server: `executionProjectionsForRuns` adds `agentQueue` (agent, limit,
running count, position, queued count) and the label "Waiting for
<agent>" with cause `agent_at_capacity` to a queued, never-started run
when its agent has no free slot. It reads one batch of agents and
running counts. Then it builds one line per full agent (at most 8 agents
per read), using only the three context keys that ordering needs. No new
polling.
- Server: the place in line uses `queuedRunDispatchLine`, an O(n log n)
replay of the dispatcher. The plain replay rescans the queue after each
step, which is quadratic (measured: 3.6 ms at 300 runs, 76 ms at 3,000,
600 ms at 10,000, all on the event loop). Dispatch itself is unchanged:
it stops after the free slots. Each agent's line is memoized for 5 s and
shared by concurrent reads. In a preview worktree, the line applies the
same creation-time cutoff as dispatch.
- Server: the task list summary adds `runningDescendantCount`
(descendants whose run started) next to `liveDescendantCount`.
`liveDescendantCount` still decides whether a blocked parent is covered.
The "n live below" chip uses the new count.
- Shared: the new `ExecutionAgentQueue` type and the optional field
`ExecutionProjection.agentQueue`.
- Server: `/companies/:companyId/live-runs` lists running runs before
queued runs, so a deep queue cannot push running work out of the 50-row
limit.
- UI: a new `AgentQueueWait` component shows "Waiting for Codie · 20
slots max · 41 of 295 in line · Configure", with no spinner and no
elapsed timer. Configure opens the agent's Advanced view, where the run
policy (`maxConcurrentRuns`) is edited. The task chat (both renderers),
the run ledger and the Storybook story use it.
- UI: the task header and the task rows (task list, inbox, legacy views)
show a quiet "Waiting" instead of the pulsing "Live" when every live run
of the task has not started. The full sentence shows on hover. "n live
below" counts only started sub-tasks. A waiting sub-task still covers
its blocked parent.
- UI: the sidebar "N live" counts (Dashboard, agents) count only started
runs.
- UI: the task chat pill now names other server holds on a queued run
(for example "Waiting for maintenance") instead of "Working".
- Docs: DESIGN.md records the waiting-for-slot hold.
- Storybook: `runs-agent-queue-wait` stories (full line, task chat, task
row, one slot, mobile), with play checks.

## Verification

- Red/green server test:
`server/src/services/execution-projection-agent-queue.test.ts` (embedded
Postgres). Before this change, the projection has no `agentQueue`. Now
it checks the agent name, limit, running count, positions 1 to 4 in
dispatch order (a person's retry first, then priority, not creation
order), no wait while a slot is free, and isolation from other
companies' runs. It also checks that the positions equal the line that
the dispatch builder produces.
- `pnpm exec vitest run` on the new test, `queued-run-dispatch-order`,
`recovery-wake-herd`, `heartbeat-queued-run-claim-isolation` and
`agent-live-run-routes` in `server/`.
- Runtime cost (rule `runtime-dependency-boundaries`): the assumption is
that the line stays cheap for a deep queue. `queued-run-line.test.ts`
checks that `queuedRunDispatchLine` equals the dispatcher's
claim-every-run order over 400 random queues (ages around the aging
boundary, recovery and person runs, blocked runs, deferred share runs,
all share counters). It also orders 10,000 runs in under 250 ms (the
quadratic replay takes about 600 ms). The DB test projects a 300-run
queue end to end with correct positions 1 to 300. Remaining limit: the
line is up to 5 s stale.
- Red/green: `issue-list-assignee-filter-routes.test.ts` now asserts
`runningDescendantCount: 0` for a parent whose only live descendant is
queued and never started.
- UI tests: `liveIssueIds.test.ts` (waiting tasks, started runs keep
"Live", no "running below" count for waiting sub-tasks) and
`TaskChatLiveRunPill.test.tsx` (waiting line with no spinner and the
Configure link, other holds named). The suites for the run ledger,
sidebar, sidebar agents, task detail, task columns and task chat also
pass.
- The Storybook stories render in headless Chromium and their play
checks pass.
- After deploy: open a queued task on staging and confirm the waiting
state.

## Risks

- Cost on read: a projection read that includes a queued run of a full
agent now reads that agent's queued runs (only small columns) and the
dispatch facts. This is at most one line per full agent every 5 s, for
at most 8 agents per read. Reads with no queued, never-started runs do
no more work.
- The place in line is the server's prediction of dispatch order. A
claim that is refused at start time (a gate, a blocker) can let the next
run start first. The position updates on the next refresh.
- The process-local "declined aged claim" hint that the dispatcher uses
is not visible to other servers, so the projection does not use it. This
can change only the aged-oldest share step.
- Configure opens the agent's Advanced view (the YAML run policy). The
agent's Runtime tab hides the run-policy fields today.
- Low risk otherwise: no schema change and no migration.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use
(shell, file edits, headless browser) and subagents for the UI survey.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 10:24:20 -06:00

1052 lines
38 KiB
TypeScript

import { useEffect, useMemo, useState, type ReactNode } from "react";
import { AgentQueueWait } from "./AgentQueueWait";
import { agentQueueWaitOf, isWaitingToStart } from "../lib/agent-queue-wait";
import type { ActivityEvent, Issue, Agent, ProviderTraceMetadata } from "@paperclipai/shared";
import {
isResponsibleUserDenialCode,
normalizeRunFailureCause,
responsibleUserLabel,
} from "@paperclipai/shared";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { Link } from "@/lib/router";
import { accessApi, type CurrentBoardAccess } from "../api/access";
import {
activityApi,
type RunForIssue,
type RunLivenessState,
} from "../api/activity";
import { ApiError } from "../api/client";
import {
heartbeatsApi,
type ActiveRunForIssue,
type LiveRunForIssue,
type WatchdogDecisionInput,
} from "../api/heartbeats";
import { useToastActions } from "../context/ToastContext";
import { cn, relativeTime } from "../lib/utils";
import { queryKeys } from "../lib/queryKeys";
import { keepPreviousDataForSameQueryTail } from "../lib/query-placeholder-data";
import { describeRunRetryState } from "../lib/runRetryState";
import { readSourceResolvedWatchdogFold } from "../lib/source-resolved-watchdog-fold";
import { SourceResolvedFoldBadge } from "./SourceResolvedFoldBadge";
import { ResponsibleUserDenialNotice } from "./ResponsibleUserDenialNotice";
import { RunnerInspector } from "./RunnerInspector";
import { agentsApi } from "../api/agents";
import {
ProviderTraceStatusBadge,
runRequestedProviderTrace,
} from "./ProviderTraceStatusBadge";
type IssueRunLedgerProps = {
issueId: string;
companyId: string;
issueStatus: Issue["status"];
childIssues: Issue[];
agentMap: ReadonlyMap<string, Agent>;
hasLiveRuns: boolean;
activityEvents?: ActivityEvent[];
renderActivityEvent?: (event: ActivityEvent) => ReactNode;
resolveUserLabel?: (userId: string) => string | null | undefined;
};
type IssueRunLedgerContentProps = {
runs: RunForIssue[];
liveRuns?: LiveRunForIssue[];
activeRun?: ActiveRunForIssue | null;
issueStatus: Issue["status"];
childIssues: Issue[];
agentMap: ReadonlyMap<string, Pick<Agent, "name">>;
activityEvents?: ActivityEvent[];
renderActivityEvent?: (event: ActivityEvent) => ReactNode;
resolveUserLabel?: (userId: string) => string | null | undefined;
pendingWatchdogDecision?: WatchdogDecisionInput["decision"] | null;
canRecordWatchdogDecisions?: boolean;
watchdogDecisionError?: string | null;
onWatchdogDecision?: (input: WatchdogDecisionInput) => void;
onRerunWithTrace?: (run: RunForIssue) => void;
providerTraceMetadata?: ReadonlyMap<string, ProviderTraceMetadata>;
};
type LedgerRun = RunForIssue & {
isLive?: boolean;
agentName?: string;
outputSilence?: ActiveRunForIssue["outputSilence"];
};
type LedgerFeedItem =
| {
kind: "run";
id: string;
timestamp: string;
run: LedgerRun;
}
| {
kind: "activity";
id: string;
timestamp: string;
event: ActivityEvent;
};
type LivenessCopy = {
label: string;
tone: string;
description: string;
};
const LIVENESS_COPY: Record<RunLivenessState, LivenessCopy> = {
completed: {
label: "Completed",
tone: "border-emerald-500/30 bg-emerald-500/10 text-emerald-700 dark:text-emerald-300",
description: "Task reached a terminal state.",
},
advanced: {
label: "Advanced",
tone: "border-cyan-500/30 bg-cyan-500/10 text-cyan-700 dark:text-cyan-300",
description: "Run produced concrete evidence of progress.",
},
plan_only: {
label: "Plan only",
tone: "border-amber-500/30 bg-amber-500/10 text-amber-700 dark:text-amber-300",
description: "Run described future work without concrete action evidence.",
},
empty_response: {
label: "Empty response",
tone: "border-orange-500/30 bg-orange-500/10 text-orange-700 dark:text-orange-300",
description: "Run finished without useful output.",
},
blocked: {
label: "Blocked",
tone: "border-yellow-500/30 bg-yellow-500/10 text-yellow-700 dark:text-yellow-300",
description: "Run or task declared a blocker.",
},
failed: {
label: "Failed",
tone: "border-red-500/30 bg-red-500/10 text-red-700 dark:text-red-300",
description: "Run ended unsuccessfully.",
},
needs_followup: {
label: "Needs follow-up",
tone: "border-sky-500/30 bg-sky-500/10 text-sky-700 dark:text-sky-300",
description:
"Run produced useful output but did not prove concrete progress.",
},
};
const PENDING_LIVENESS_COPY: LivenessCopy = {
label: "Checks after finish",
tone: "border-border bg-background text-muted-foreground",
description: "Liveness is evaluated after the run finishes.",
};
const RETRY_PENDING_LIVENESS_COPY: LivenessCopy = {
label: "Retry pending",
tone: "border-blue-500/30 bg-blue-500/10 text-blue-700 dark:text-blue-300",
description: "Paperclip queued an automatic retry that has not started yet.",
};
const MISSING_LIVENESS_COPY: LivenessCopy = {
label: "No liveness data",
tone: "border-border bg-background text-muted-foreground",
description: "This run has no persisted liveness classification.",
};
const TERMINAL_CHILD_STATUSES = new Set<Issue["status"]>(["done", "cancelled"]);
const ACTIVE_RUN_STATUSES = new Set(["queued", "running"]);
type RunOutputSilenceLevel = NonNullable<
ActiveRunForIssue["outputSilence"]
>["level"];
type RunOutputSilenceCopy = {
label: string;
tone: string;
};
const RUN_OUTPUT_SILENCE_COPY: Partial<
Record<RunOutputSilenceLevel, RunOutputSilenceCopy>
> = {
suspicious: {
label: "Output silence",
tone: "border-amber-500/30 bg-amber-500/10 text-amber-700 dark:text-amber-300",
},
critical: {
label: "Critical silence",
tone: "border-red-500/30 bg-red-500/10 text-red-700 dark:text-red-300",
},
snoozed: {
label: "Silence snoozed",
tone: "border-cyan-500/30 bg-cyan-500/10 text-cyan-700 dark:text-cyan-300",
},
};
function asRecord(value: unknown): Record<string, unknown> | null {
if (typeof value !== "object" || value === null || Array.isArray(value))
return null;
return value as Record<string, unknown>;
}
function readString(value: unknown) {
return typeof value === "string" && value.trim().length > 0
? value.trim()
: null;
}
function readNumber(value: unknown) {
return typeof value === "number" && Number.isFinite(value) ? value : null;
}
function formatDuration(
start: string | Date | null | undefined,
end: string | Date | null | undefined,
) {
if (!start) return null;
const startMs = new Date(start).getTime();
const endMs = end ? new Date(end).getTime() : Date.now();
if (!Number.isFinite(startMs) || !Number.isFinite(endMs)) return null;
const totalSeconds = Math.max(0, Math.round((endMs - startMs) / 1000));
if (totalSeconds < 60) return `${totalSeconds}s`;
const minutes = Math.floor(totalSeconds / 60);
const seconds = totalSeconds % 60;
if (minutes < 60)
return seconds > 0 ? `${minutes}m ${seconds}s` : `${minutes}m`;
const hours = Math.floor(minutes / 60);
const remainingMinutes = minutes % 60;
return remainingMinutes > 0 ? `${hours}h ${remainingMinutes}m` : `${hours}h`;
}
function toIsoString(value: string | Date | null | undefined) {
if (!value) return null;
return value instanceof Date ? value.toISOString() : value;
}
function liveRunToLedgerRun(
run: LiveRunForIssue | ActiveRunForIssue,
): LedgerRun {
return {
runId: run.id,
status: run.status,
agentId: run.agentId,
agentName: run.agentName,
adapterType: run.adapterType,
startedAt: toIsoString(run.startedAt),
finishedAt: toIsoString(run.finishedAt),
createdAt: toIsoString(run.createdAt) ?? new Date().toISOString(),
invocationSource: run.invocationSource,
usageJson: null,
resultJson: null,
isLive: run.status === "queued" || run.status === "running",
outputSilence: run.outputSilence,
};
}
function mergeRuns(
runs: RunForIssue[],
liveRuns: LiveRunForIssue[] | undefined,
activeRun: ActiveRunForIssue | null | undefined,
) {
const byId = new Map<string, LedgerRun>();
for (const run of runs) byId.set(run.runId, run);
for (const run of liveRuns ?? []) {
const existing = byId.get(run.id);
byId.set(
run.id,
existing
? {
...existing,
isLive: true,
agentName: run.agentName,
outputSilence: run.outputSilence,
}
: liveRunToLedgerRun(run),
);
}
if (activeRun) {
const existing = byId.get(activeRun.id);
if (existing) {
byId.set(activeRun.id, {
...existing,
isLive: isActiveRun(existing) || isActiveRun(activeRun),
agentName: activeRun.agentName,
outputSilence: activeRun.outputSilence,
});
} else {
byId.set(activeRun.id, liveRunToLedgerRun(activeRun));
}
}
return [...byId.values()].sort((a, b) => {
const aTime = new Date(a.startedAt ?? a.createdAt).getTime();
const bTime = new Date(b.startedAt ?? b.createdAt).getTime();
if (aTime !== bTime) return bTime - aTime;
return b.runId.localeCompare(a.runId);
});
}
function statusLabel(status: string) {
return status.replace(/_/g, " ");
}
function isActiveRun(run: Pick<LedgerRun, "status" | "isLive">) {
return run.isLive || ACTIVE_RUN_STATUSES.has(run.status);
}
function runSummary(
run: LedgerRun,
agentMap: ReadonlyMap<string, Pick<Agent, "name">>,
) {
const agentName = compactAgentName(run, agentMap);
if (run.status === "running") return `Running now by ${agentName}`;
if (run.status === "queued") return `Queued for ${agentName}`;
if (run.status === "scheduled_retry")
return `Automatic retry scheduled for ${agentName}`;
return `${statusLabel(run.status)} by ${agentName}`;
}
function livenessCopyForRun(run: LedgerRun) {
if (run.status === "scheduled_retry") return RETRY_PENDING_LIVENESS_COPY;
if (run.livenessState) return LIVENESS_COPY[run.livenessState];
return isActiveRun(run) ? PENDING_LIVENESS_COPY : MISSING_LIVENESS_COPY;
}
function stopReasonLabel(run: RunForIssue) {
const result = asRecord(run.resultJson);
const stopReason = readString(result?.stopReason);
const timeoutFired = result?.timeoutFired === true;
const effectiveTimeoutSec = readNumber(result?.effectiveTimeoutSec);
const timeoutText =
effectiveTimeoutSec && effectiveTimeoutSec > 0
? `${effectiveTimeoutSec}s timeout`
: null;
if (timeoutFired || stopReason === "timeout") {
return timeoutText ? `timeout (${timeoutText})` : "timeout";
}
if (
stopReason === "max_turns_exhausted" ||
stopReason === "turn_limit_exhausted"
)
return "max turns exhausted";
if (stopReason === "budget_paused") return "budget paused";
if (stopReason === "cancelled") return "cancelled";
if (stopReason === "paused") return "paused by board";
if (stopReason === "process_lost") return "process lost";
if (stopReason === "unmanaged_background_task_stopped")
return "unmanaged background task stopped";
if (stopReason === "adapter_failed") {
// Older runs have no recorded cause and keep the plain label.
const failureCause = normalizeRunFailureCause(run.failureCause);
return failureCause ? `adapter failed (${failureCause})` : "adapter failed";
}
if (stopReason === "completed")
return timeoutText ? `completed (${timeoutText})` : "completed";
return timeoutText;
}
function stopStatusLabel(run: LedgerRun, stopReason: string | null) {
if (stopReason) return stopReason;
if (run.status === "scheduled_retry") return "Retry pending";
if (run.status === "queued") return "Waiting to start";
if (run.status === "running") return "Still running";
if (!run.livenessState) return "Unavailable";
return "No stop reason";
}
function lastUsefulActionLabel(run: LedgerRun) {
if (run.status === "scheduled_retry") return "Waiting for next attempt";
if (run.lastUsefulActionAt) return relativeTime(run.lastUsefulActionAt);
if (isActiveRun(run)) return "No action recorded yet";
if (
run.livenessState === "plan_only" ||
run.livenessState === "needs_followup"
) {
return "No concrete action";
}
if (run.livenessState === "empty_response") return "No useful output";
if (!run.livenessState) return "Unavailable";
return "None recorded";
}
function continuationLabel(run: LedgerRun) {
if (!run.continuationAttempt || run.continuationAttempt <= 0) return null;
return `Continuation attempt ${run.continuationAttempt}`;
}
function hasExhaustedContinuation(run: RunForIssue) {
return /continuation attempts exhausted/i.test(run.livenessReason ?? "");
}
function childIssueSummary(childIssues: Issue[]) {
const active = childIssues.filter(
(issue) => !TERMINAL_CHILD_STATUSES.has(issue.status),
);
const done = childIssues.filter((issue) => issue.status === "done").length;
const cancelled = childIssues.filter(
(issue) => issue.status === "cancelled",
).length;
return { active, done, cancelled, total: childIssues.length };
}
function compactAgentName(
run: LedgerRun,
agentMap: ReadonlyMap<string, Pick<Agent, "name">>,
) {
return (
run.agentName ?? agentMap.get(run.agentId)?.name ?? run.agentId.slice(0, 8)
);
}
function formatSilenceAge(ms: number | null | undefined) {
if (!ms || ms <= 0) return null;
const totalMinutes = Math.floor(ms / 60_000);
if (totalMinutes < 1) return "under 1 minute";
if (totalMinutes < 60)
return `${totalMinutes} minute${totalMinutes === 1 ? "" : "s"}`;
const hours = Math.floor(totalMinutes / 60);
const minutes = totalMinutes % 60;
if (minutes === 0) return `${hours} hour${hours === 1 ? "" : "s"}`;
return `${hours}h ${minutes}m`;
}
function canBoardRecordWatchdogDecision(
companyId: string,
boardAccess: CurrentBoardAccess | undefined,
) {
if (!boardAccess) return false;
if (boardAccess.source === "local_implicit" || boardAccess.isInstanceAdmin)
return true;
const membership = boardAccess.memberships?.find(
(item) => item.companyId === companyId && item.status === "active",
);
if (!membership)
return (
boardAccess.companyIds.includes(companyId) && !boardAccess.memberships
);
return (
membership.membershipRole !== "viewer" && membership.membershipRole !== null
);
}
function watchdogDecisionErrorMessage(error: unknown) {
if (error instanceof ApiError && error.status === 403) {
return "Only the board or the assigned recovery owner can record watchdog decisions";
}
return error instanceof Error && error.message.trim().length > 0
? error.message
: "Paperclip could not record the watchdog decision.";
}
export function IssueRunLedger({
issueId,
companyId,
issueStatus,
childIssues,
agentMap,
hasLiveRuns,
activityEvents,
renderActivityEvent,
resolveUserLabel,
}: IssueRunLedgerProps) {
const queryClient = useQueryClient();
const { pushToast } = useToastActions();
const [watchdogDecisionError, setWatchdogDecisionError] = useState<
string | null
>(null);
const { data: boardAccess } = useQuery({
queryKey: queryKeys.access.currentBoardAccess,
queryFn: () => accessApi.getCurrentBoardAccess(),
});
const { data: runs } = useQuery({
queryKey: queryKeys.issues.runs(issueId),
queryFn: () => activityApi.runsForIssue(issueId),
refetchInterval:
hasLiveRuns || issueStatus === "in_progress" ? 5000 : false,
placeholderData: keepPreviousDataForSameQueryTail<RunForIssue[]>(issueId),
});
const { data: liveRuns } = useQuery({
queryKey: queryKeys.issues.liveRuns(issueId),
queryFn: () => heartbeatsApi.liveRunsForIssue(issueId),
enabled: hasLiveRuns,
refetchInterval: 3000,
placeholderData:
keepPreviousDataForSameQueryTail<LiveRunForIssue[]>(issueId),
});
const { data: activeRun = null } = useQuery({
queryKey: queryKeys.issues.activeRun(issueId),
queryFn: () => heartbeatsApi.activeRunForIssue(issueId),
enabled: hasLiveRuns || issueStatus === "in_progress",
refetchInterval: hasLiveRuns ? false : 3000,
placeholderData: keepPreviousDataForSameQueryTail<ActiveRunForIssue | null>(
issueId,
),
});
const traceRunIds = useMemo(
() => (runs ?? []).slice(0, 100).map((run) => run.runId),
[runs],
);
const canInspectProviderTrace =
boardAccess?.source === "local_implicit" || boardAccess?.isInstanceAdmin === true;
const { data: providerTraceRows } = useQuery({
queryKey: queryKeys.providerTraceMetadata(companyId, traceRunIds),
queryFn: () => heartbeatsApi.providerTraceMetadata(companyId, traceRunIds),
enabled: canInspectProviderTrace && traceRunIds.length > 0,
});
const providerTraceMetadata = useMemo(
() => new Map((providerTraceRows ?? []).map((trace) => [trace.runId, trace])),
[providerTraceRows],
);
const watchdogDecision = useMutation({
mutationFn: (input: WatchdogDecisionInput) =>
heartbeatsApi.recordWatchdogDecision(input),
onMutate: () => {
setWatchdogDecisionError(null);
},
onSuccess: () => {
setWatchdogDecisionError(null);
queryClient.invalidateQueries({
queryKey: queryKeys.issues.activeRun(issueId),
});
queryClient.invalidateQueries({
queryKey: queryKeys.issues.liveRuns(issueId),
});
},
onError: (error) => {
const message = watchdogDecisionErrorMessage(error);
const dedupeSuffix =
error instanceof ApiError ? String(error.status) : "error";
setWatchdogDecisionError(message);
pushToast({
title: "Watchdog decision not recorded",
body: message,
tone: "error",
dedupeKey: `watchdog-decision:${issueId}:${dedupeSuffix}`,
});
},
});
const rerunWithTrace = useMutation({
mutationFn: async (run: RunForIssue) => {
const context = asRecord(run.contextSnapshot);
const payload: Record<string, unknown> = {};
for (const key of ["issueId", "taskId", "taskKey"] as const) {
const value = readString(context?.[key]);
if (value) payload[key] = value;
}
const result = await agentsApi.wakeup(
run.agentId,
{
source: "on_demand",
triggerDetail: "manual",
reason: "rerun_with_provider_trace",
payload,
debug: { providerTrace: "raw" },
},
companyId,
);
if (!("id" in result))
throw new Error(result.message ?? "Trace re-run was skipped.");
return result;
},
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: queryKeys.issues.runs(issueId),
});
queryClient.invalidateQueries({
queryKey: queryKeys.issues.liveRuns(issueId),
});
},
onError: (error) =>
pushToast({
title: "Trace re-run not started",
body:
error instanceof Error
? error.message
: "Paperclip could not start the trace re-run.",
tone: "error",
dedupeKey: `provider-trace-rerun:${issueId}`,
}),
});
return (
<IssueRunLedgerContent
runs={runs ?? []}
liveRuns={liveRuns}
activeRun={activeRun}
issueStatus={issueStatus}
childIssues={childIssues}
agentMap={agentMap}
activityEvents={activityEvents}
renderActivityEvent={renderActivityEvent}
resolveUserLabel={resolveUserLabel}
pendingWatchdogDecision={watchdogDecision.variables?.decision ?? null}
canRecordWatchdogDecisions={canBoardRecordWatchdogDecision(
companyId,
boardAccess,
)}
watchdogDecisionError={watchdogDecisionError}
onWatchdogDecision={(input) => watchdogDecision.mutate(input)}
onRerunWithTrace={
canInspectProviderTrace
? (run) => rerunWithTrace.mutate(run)
: undefined
}
providerTraceMetadata={providerTraceMetadata}
/>
);
}
export function IssueRunLedgerContent({
runs,
liveRuns,
activeRun,
issueStatus,
childIssues,
agentMap,
activityEvents,
renderActivityEvent,
resolveUserLabel,
pendingWatchdogDecision,
canRecordWatchdogDecisions = true,
watchdogDecisionError,
onWatchdogDecision,
onRerunWithTrace,
providerTraceMetadata = new Map(),
}: IssueRunLedgerContentProps) {
const [inspectedRun, setInspectedRun] = useState<LedgerRun | null>(null);
const ledgerRuns = useMemo(
() => mergeRuns(runs, liveRuns, activeRun),
[activeRun, liveRuns, runs],
);
useEffect(() => {
if (inspectedRun || typeof window === "undefined") return;
const requestedRunId = new URLSearchParams(window.location.search).get("inspectRun");
if (!requestedRunId) return;
const requestedRun = ledgerRuns.find((run) => run.runId === requestedRunId);
if (requestedRun) setInspectedRun(requestedRun);
}, [inspectedRun, ledgerRuns]);
const latestRun = ledgerRuns[0] ?? null;
const latestSilentRun = useMemo(
() =>
ledgerRuns.find(
(run) =>
isActiveRun(run) &&
(run.outputSilence?.level === "critical" ||
run.outputSilence?.level === "suspicious"),
) ?? null,
[ledgerRuns],
);
const children = childIssueSummary(childIssues);
const canRenderActivityEvents = Boolean(renderActivityEvent);
const feedItems = useMemo<LedgerFeedItem[]>(() => {
const items: LedgerFeedItem[] = [];
for (const run of ledgerRuns) {
items.push({
kind: "run",
id: run.runId,
timestamp: run.startedAt ?? run.createdAt,
run,
});
}
if (canRenderActivityEvents) {
for (const event of activityEvents ?? []) {
items.push({
kind: "activity",
id: event.id,
timestamp:
event.createdAt instanceof Date
? event.createdAt.toISOString()
: String(event.createdAt),
event,
});
}
}
return items.sort((a, b) => {
const aTime = new Date(a.timestamp).getTime();
const bTime = new Date(b.timestamp).getTime();
if (aTime !== bTime) return bTime - aTime;
if (a.kind !== b.kind) return a.kind === "run" ? -1 : 1;
return b.id.localeCompare(a.id);
});
}, [activityEvents, canRenderActivityEvents, ledgerRuns]);
return (
<section className="space-y-3" aria-label="Task run ledger">
<div className="flex items-center justify-between gap-2">
<div className="min-w-0">
<h3 className="text-sm font-medium text-muted-foreground">
Run ledger
</h3>
<p className="text-xs text-muted-foreground">
{latestRun
? runSummary(latestRun, agentMap)
: issueStatus === "in_progress"
? "Waiting for the first run record."
: "No runs linked yet."}
</p>
</div>
{latestRun ? (
<Link
to={`/agents/${latestRun.agentId}/runs/${latestRun.runId}`}
className="shrink-0 rounded-md border border-border px-2 py-1 text-xs text-muted-foreground hover:text-foreground"
>
Latest run
</Link>
) : null}
</div>
{children.total > 0 ? (
<div className="rounded-md border border-border/70 px-3 py-2">
<div className="flex flex-wrap items-center gap-2 text-xs">
<span className="font-medium text-foreground">Child work</span>
<span className="text-muted-foreground">
{children.active.length > 0
? `${children.active.length} active, ${children.done} done, ${children.cancelled} cancelled`
: `all ${children.total} terminal (${children.done} done, ${children.cancelled} cancelled)`}
</span>
</div>
{children.active.length > 0 ? (
<div className="mt-2 flex flex-wrap gap-1.5">
{children.active.slice(0, 4).map((child) => (
<Link
key={child.id}
to={`/issues/${child.identifier ?? child.id}`}
className="inline-flex min-w-0 max-w-full items-center gap-1 rounded-md border border-border bg-background px-2 py-1 text-(length:--text-micro) hover:bg-accent/40"
>
<span className="shrink-0 font-mono text-muted-foreground">
{child.identifier ?? child.id.slice(0, 8)}
</span>
<span className="truncate">{child.title}</span>
<span className="shrink-0 text-muted-foreground">
{statusLabel(child.status)}
</span>
</Link>
))}
{children.active.length > 4 ? (
<span className="rounded-md border border-border px-2 py-1 text-(length:--text-micro) text-muted-foreground">
+{children.active.length - 4} more
</span>
) : null}
</div>
) : null}
</div>
) : null}
{latestSilentRun?.outputSilence ? (
<div
className={cn(
"rounded-md border px-3 py-2 text-xs",
latestSilentRun.outputSilence.level === "critical"
? "border-red-500/30 bg-red-500/10 text-red-900 dark:text-red-200"
: "border-amber-500/30 bg-amber-500/10 text-amber-900 dark:text-amber-200",
)}
>
<p className="font-medium">
{latestSilentRun.outputSilence.level === "critical"
? "Critical output silence"
: "Output silence watchdog warning"}
</p>
<p className="mt-1">
Latest active run has been silent for{" "}
{formatSilenceAge(latestSilentRun.outputSilence.silenceAgeMs) ??
"an extended period"}
.
{latestSilentRun.outputSilence.evaluationIssueIdentifier ? (
<>
{" "}
Review{" "}
<Link
to={`/issues/${latestSilentRun.outputSilence.evaluationIssueIdentifier}`}
className="font-medium underline underline-offset-2"
>
{latestSilentRun.outputSilence.evaluationIssueIdentifier}
</Link>{" "}
for recovery context.
</>
) : null}
</p>
<p className="mt-1">
{latestSilentRun.outputSilence.evaluationIssueIdentifier
? "This signal is informational. Paperclip did not create new delegated recovery work."
: "This signal is informational. Paperclip did not create or assign a recovery task."}
</p>
{onWatchdogDecision && canRecordWatchdogDecisions ? (
<div className="mt-2 flex flex-wrap gap-1.5">
<button
type="button"
className="rounded-md border border-border bg-background/80 px-2 py-1 text-(length:--text-micro) text-foreground hover:bg-background"
onClick={() =>
onWatchdogDecision({
runId: latestSilentRun.runId,
decision: "continue",
evaluationIssueId:
latestSilentRun.outputSilence?.evaluationIssueId ?? null,
})
}
disabled={pendingWatchdogDecision != null}
>
Continue monitoring
</button>
<button
type="button"
className="rounded-md border border-border bg-background/80 px-2 py-1 text-(length:--text-micro) text-foreground hover:bg-background"
onClick={() =>
onWatchdogDecision({
runId: latestSilentRun.runId,
decision: "snooze",
evaluationIssueId:
latestSilentRun.outputSilence?.evaluationIssueId ?? null,
snoozedUntil: new Date(
Date.now() + 60 * 60 * 1000,
).toISOString(),
reason: "Snoozed from issue run ledger",
})
}
disabled={pendingWatchdogDecision != null}
>
Snooze 1h
</button>
<button
type="button"
className="rounded-md border border-border bg-background/80 px-2 py-1 text-(length:--text-micro) text-foreground hover:bg-background"
onClick={() =>
onWatchdogDecision({
runId: latestSilentRun.runId,
decision: "dismissed_false_positive",
evaluationIssueId:
latestSilentRun.outputSilence?.evaluationIssueId ?? null,
reason: "Dismissed from issue run ledger",
})
}
disabled={pendingWatchdogDecision != null}
>
Mark false positive
</button>
</div>
) : null}
{watchdogDecisionError ? (
<p className="mt-2 rounded-md border border-red-500/30 bg-red-500/10 px-2 py-1 text-(length:--text-micro) text-red-900 dark:text-red-200">
{watchdogDecisionError}
</p>
) : null}
</div>
) : null}
{feedItems.length === 0 ? (
<div className="rounded-md border border-dashed border-border px-3 py-3 text-sm text-muted-foreground">
{renderActivityEvent
? "Runs and activity will appear here once this task has history."
: "Historical runs without liveness metadata will appear here once linked to this task."}
</div>
) : (
<div className="space-y-1.5">
{feedItems.slice(0, 20).map((item) => {
if (item.kind === "activity") {
return (
<div key={`activity:${item.id}`}>
{renderActivityEvent?.(item.event)}
</div>
);
}
const run = item.run;
const nextAction = run.execution ? run.execution.nextAction : run.nextAction;
const agentQueue = isWaitingToStart(run) ? agentQueueWaitOf(run.execution) : null;
const liveness = livenessCopyForRun(run);
const stopReason = stopReasonLabel(run);
const duration = formatDuration(run.startedAt, run.finishedAt);
const exhausted = hasExhaustedContinuation(run);
const continuation = continuationLabel(run);
const retryState = describeRunRetryState(run);
const agentName = compactAgentName(run, agentMap);
const onBehalfOfLabel = run.responsibleUserId
? responsibleUserLabel(resolveUserLabel?.(run.responsibleUserId))
: null;
const denialCode = isResponsibleUserDenialCode(run.errorCode)
? run.errorCode
: null;
const sourceResolvedFold = readSourceResolvedWatchdogFold(
run.resultJson,
);
return (
<article
key={`run:${run.runId}`}
className="space-y-1.5 rounded-lg border border-border/60 px-3 py-2 text-xs text-muted-foreground"
>
<div className="flex flex-wrap items-center gap-1.5">
<span className="font-medium text-foreground">Run</span>
<Link
to={`/agents/${run.agentId}/runs/${run.runId}`}
className="min-w-0 max-w-full truncate font-mono text-foreground hover:underline"
>
{run.runId.slice(0, 8)}
</Link>
<span>by {agentName}</span>
{onBehalfOfLabel ? (
<span
data-testid="run-on-behalf-of"
className="min-w-0 max-w-full truncate text-muted-foreground"
title={`Acting on behalf of ${onBehalfOfLabel}`}
>
on behalf of{" "}
<span className="text-foreground">{onBehalfOfLabel}</span>
</span>
) : null}
<span className="rounded-md border border-border px-1.5 py-0.5 text-(length:--text-micro) capitalize text-muted-foreground">
{statusLabel(run.status)}
</span>
{run.isLive && !isWaitingToStart(run) ? (
<span className="inline-flex items-center gap-1 rounded-md border border-blue-500/30 bg-blue-500/10 px-1.5 py-0.5 text-(length:--text-micro) text-blue-700 dark:text-blue-300">
<span className="h-1.5 w-1.5 rounded-full bg-blue-500" />
live
</span>
) : null}
<ProviderTraceStatusBadge
trace={providerTraceMetadata.get(run.runId)}
requested={runRequestedProviderTrace(run.contextSnapshot)}
showOff
/>
<span
className={cn(
"rounded-md border px-1.5 py-0.5 text-(length:--text-micro) font-medium",
liveness.tone,
)}
title={liveness.description}
>
{liveness.label}
</span>
{exhausted ? (
<span className="rounded-md border border-red-500/30 bg-red-500/10 px-1.5 py-0.5 text-(length:--text-micro) font-medium text-red-700 dark:text-red-300">
Exhausted
</span>
) : null}
{continuation ? (
<span className="text-(length:--text-micro) text-muted-foreground">
{continuation}
</span>
) : null}
{retryState ? (
<span
className={cn(
"rounded-md border px-1.5 py-0.5 text-(length:--text-micro) font-medium",
retryState.tone,
)}
>
{retryState.badgeLabel}
</span>
) : null}
{run.outputSilence &&
RUN_OUTPUT_SILENCE_COPY[run.outputSilence.level] ? (
<span
className={cn(
"rounded-md border px-1.5 py-0.5 text-(length:--text-micro) font-medium",
RUN_OUTPUT_SILENCE_COPY[run.outputSilence.level]?.tone,
)}
>
{RUN_OUTPUT_SILENCE_COPY[run.outputSilence.level]?.label}
</span>
) : null}
{sourceResolvedFold ? <SourceResolvedFoldBadge /> : null}
<span className="ml-auto shrink-0">
{relativeTime(item.timestamp)}
</span>
<button
type="button"
className="rounded-md border border-border px-1.5 py-0.5 text-(length:--text-micro) text-foreground hover:bg-accent/40"
onClick={() => setInspectedRun(run)}
>
Inspect run
</button>
</div>
<div className="grid gap-2 text-xs text-muted-foreground sm:grid-cols-3">
<div className="min-w-0">
<span className="text-foreground">Elapsed</span>{" "}
{duration ?? "unknown"}
</div>
<div className="min-w-0">
<span className="text-foreground">Last useful action</span>{" "}
{lastUsefulActionLabel(run)}
</div>
<div className="min-w-0">
<span className="text-foreground">Stop</span>{" "}
{stopStatusLabel(run, stopReason)}
</div>
</div>
{retryState ? (
<div className="rounded-md border border-border/70 bg-accent/20 px-2 py-2 text-xs leading-5 text-muted-foreground">
{retryState.detail ? <p>{retryState.detail}</p> : null}
{retryState.secondary ? (
<p>{retryState.secondary}</p>
) : null}
{retryState.retryOfRunId ? (
<p>
Retry of{" "}
<Link
to={`/agents/${run.agentId}/runs/${retryState.retryOfRunId}`}
className="font-mono text-foreground hover:underline"
>
{retryState.retryOfRunId.slice(0, 8)}
</Link>
</p>
) : null}
</div>
) : null}
{run.livenessReason ? (
<p className="min-w-0 break-words text-xs leading-5 text-muted-foreground">
{run.livenessReason}
</p>
) : null}
{denialCode ? (
<ResponsibleUserDenialNotice
code={denialCode}
userName={
run.responsibleUserId
? resolveUserLabel?.(run.responsibleUserId)
: null
}
/>
) : null}
{agentQueue ? (
<AgentQueueWait queue={agentQueue} className="text-xs" />
) : nextAction ? (
<div className="min-w-0 rounded-md bg-accent/40 px-2 py-1.5 text-xs leading-5">
<span className="font-medium text-foreground">
Next action:{" "}
</span>
<span className="break-words text-muted-foreground">
{nextAction}
</span>
</div>
) : null}
</article>
);
})}
{feedItems.length > 20 ? (
<div className="px-3 py-2 text-xs text-muted-foreground">
{feedItems.length - 20} older items not shown
</div>
) : null}
</div>
)}
{inspectedRun ? (
<RunnerInspector
runId={inspectedRun.runId}
run={inspectedRun}
open
onOpenChange={(nextOpen) => {
if (!nextOpen) setInspectedRun(null);
}}
onRerunWithTrace={
!["queued", "running"].includes(inspectedRun.status) &&
onRerunWithTrace
? () => onRerunWithTrace(inspectedRun)
: undefined
}
/>
) : null}
</section>
);
}