Files
PaperClipAI/ui/src/components/transcript/useLiveRunTranscripts.ts
T
DottaandPaperclip 6cc765bd81 feat(live): deliver recoverable transcripts to interested views (#17456)
## Thinking Path

> - Paperclip should show live agent output without repeating
presentation work for every view.
> - Execution, acknowledgements and durable checkpoints must remain
independent of browser interest.
> - The server already owns authorization and redaction at delivery.
> - A shared browser connection can declare which visible runs need full
transcripts.
> - Catch-up must handle subscription races, disconnects, terminal
output and slow consumers.
> - This PR delivers the backend and UI contract together without a
database migration.

## Linked Issues or Issue Description

**What existing behavior does this improve?**

Visible transcript consumers receive recoverable output through one
shared browser connection.

**Current behavior**

Independent views can create duplicate connections, presentation work
and recurring history reads. A subscription race or filtered native
history can lose the saved tail.

**Proposed behavior**

Negotiate run interests, subscribe before catch-up and retain
independent canonical recovery cursors. Keep lightweight lifecycle and
attention events available without full transcript delivery.

PR 4 of #17097 → #17439 → #17455 → #17456. The runtime feature is
independent of hot-upgrade recovery; the stack order is for review
convenience.

## What Changed

- Add additive WebSocket capability negotiation, replacement interests,
revision acknowledgement and resynchronization notices.
- Reference-count visible consumer interests on one company-scoped
browser connection. Preserve older-client/server behavior.
- Reuse canonical structural/current-user/registered-secret redaction
for plugin run notifications through the same bounded delivery queue.
Keep producer and uninterested streams free of recipient lookups.
- Refresh application queries after the matching recovery
acknowledgement, including one follow-up after an older in-flight read.
Ordinary interest acknowledgements add no refresh.
- Keep authorization and redaction at delivery. Browser interest cannot
change harness execution, PRP ACKs, checkpoints, accounting or canonical
logs.
- Deduplicate by sequence and log identity. Catch up terminal output and
saved tails without recurring reads during healthy negotiated streaming.
- Bound delivery to 1,000 items and 1 MiB. Overflow requires
resynchronization; persistent congestion closes the connection.
- Preserve compact-history expansion from current master across
paginated catch-up. Snapshot the original read window before React can
defer its update.

## Verification

On current source `f6710f0b3c`, Linux [job
122604880503](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122604880503)
executes all five run-read permission cases successfully (2.574
seconds), including grant and revocation on the same connection. Its
shard passes all 2,430 cases.

Current [server-shard job
122604880416](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122604880416)
has 1,513 passes, one failure and three skips: an unchanged nested-clone
expiration/restore test exceeds its 15-second deadline. The preceding
`5efc6c48bf` source has identical non-document bytes; [its corresponding
job
122601359272](https://github.com/paperclipai/paperclip/actions/runs/40623017917/job/122601359272)
passes all 21 nested-clone cases, including that case in 9.505 seconds.
The exact timing cause is unproven. The single diagnostic [retry job
122609332597](https://github.com/paperclipai/paperclip/actions/runs/40624101891/job/122609332597)
passes all 21 nested-clone cases (61.316 seconds), including the
formerly timed-out case in 7.003 seconds, and all 1,514 shard cases
(three skips). The failure remains recorded; no timeout or assertion was
changed.

After the server squash merge and qualification retarget, all 50 focused
server cases and all 91 focused UI cases pass again, with both
typechecks and token gates. Runner vendor preparation rebuilds current
master inputs before those tests. This source needs fresh CI and
reviews; prior CI and browser receipts do not qualify it by inference.

Current candidate `f6710f0b3c` now targets master `6987c2f20b`, after
merged #17097, #17439 and #17455. Its diff contains only transcript
delivery and its documentation. It adds no database migration.
Retargeting changes only the evidence document; all production and test
code is byte-identical to `5efc6c48bf`.

On the eighth integration, all 50 focused server delivery cases and all
91 focused UI cases pass, with both typechecks and token gates. The
following 35-case receipt belongs to the earlier delivery correction.
Four low-queue authorization/decoration stall variants and the
late-rejection case fail before correction. They now verify the
five-second read deadline, old and negotiated clients, revisions that
cannot reset the deadline, unaffected sibling connections, rejected late
results, and timer cleanup. The first combined toast run has four
loopback EPERM failures and 31 passes; its permitted repeat passes all
35. Earlier 28-case evidence excluded the toast file.

The permission correction requires the current HTTP company-scope
transcript permission, plus task access, for both native events and
legacy logs. Prior Linux source `49a8783e9b`, job 122549913643, executes
all five run-read cases, including grant/revocation on the same
connection, successfully. That source placed this file in server shard
11; later shard assignments can differ and must be checked against their
actual inventories. The older `405bedd189` receipt retains a separate
scheduled-retry teardown race and inherited fixture collision; neither
is waived. The new head must pass again in Linux CI.

The seven extracted stream-recovery tests preserve their assertions. All
39 related legacy/native cases, UI typecheck and token gates pass after
restoring the initially missed fixture helper. Earlier full workspace
build/typecheck and Chrome journeys at `445a76563c` cover two views,
company switches, reconnect, terminal catch-up, reload and zero
recurring healthy transcript reads. Those browser receipts are
historical source evidence and do not qualify this new head by
inference. The [qualification
record](https://github.com/paperclipai/paperclip/blob/f6710f0b3c84c6fdb2021df67206261b446dfc97/doc/plans/2026-10-22-hot-upgrade-qualification.md)
retains exact earlier UI/history, source-boundary and cleanup failures
with their fixes and limits.

The workload-cost test preserves bounds and authorization callback
counts for zero interests, one viewer, multiple browsers and a slow
viewer. Its socket congestion is simulated. It measures neither SQL nor
network costs. Current permission checks and the final hosted
active-agent/viewer matrix remain unmeasured; no production savings are
claimed.

Fresh exact-head CI, legacy status, completed Greptile 5/5, resolved
feedback and ordinary merge eligibility remain required.

The earlier recipient corrections pass all 47
registry/plugin/delivery/WebSocket/toast cases and all 174 UI
provider/recovery/interest cases, both typechecks and token gates on
source `b202e59a25`. A real bus/host-service test registers a synthetic
run secret through the production registry and reproduces its raw
notification before the fix; native/legacy/final notifications now
redact it, disposal prevents late worker notification. A later P1 review
reproduces lost final notifications on settings-read and
secret-resolution failure. Both cases fail before correction. Any
delivery failure or subscription timeout now sends one content-free
stream error and removes the plugin subscription. It never changes the
agent execution or run status. Later events start no new redaction
reads. The final integrated six-file server lane passes all 50 cases,
with server typecheck and token gates. Existing browser delivery
behavior is unchanged. Storage and secret-provider seams are scripted.
Two real React Query/provider regressions first miss a run during
resync; both now refresh after the valid replacement ACK, including
pending-read and stale-scope cases. The prerequisite qualification batch
passes all 118 cases and E2E typecheck after master integration. Fresh
CI, security scan and reviews remain pending on this head.

The contribution-rule review correction removes 104 explanatory comments
introduced by this slice and keeps the ordering, deadline and
plugin-delivery explanations in the delivery document. Existing comments
and functional test/lint directives remain. All edited source files have
identical parsed non-comment tokens before and after the change. All 50
server delivery cases and all 91 cases in six affected UI suites pass,
along with token gates. The new head requires fresh CI, security and
review.

## Risks

A slow or disconnected viewer must resynchronize from authorized
canonical history. It may see a display gap until catch-up completes.
Company changes and revoked access must remove delivery interests. This
slice adds no database migration and cannot change execution or recovery
authority. Full transcript delivery evaluates current company-scope
permission per delivery. Active plugin recipients also perform canonical
redaction reads; viewer-load database costs on this final source remain
unmeasured.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, repository tools and code
execution. The runtime does not expose a more specific model ID or
context-window size.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-10-27 22:23:30 -05:00

610 lines
26 KiB
TypeScript

import { TranscriptSequence } from "../../lib/transcript-sequence";
import { useTranscriptLiveStream } from "../../context/LiveUpdatesProvider";
import { usePageVisibility } from "../../lib/page-visibility";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { readTranscriptRequest } from "./read-transcript-request";
import { useQuery } from "@tanstack/react-query";
import type { LiveEvent } from "@paperclipai/shared";
import { ApiError } from "../../api/client";
import { instanceSettingsApi } from "../../api/instanceSettings";
import { heartbeatsApi } from "../../api/heartbeats";
import { buildTranscript, getUIAdapter, onAdapterChange, type RunLogChunk, type TranscriptEntry } from "../../adapters";
import { queryKeys } from "../../lib/queryKeys";
import {
mergeRunLogChunks,
parsePersistedLogContent,
readChunkSeq,
type ChunkRetentionBudget,
} from "../../lib/run-log-chunks";
const LOG_POLL_INTERVAL_MS = 2000;
const LOG_READ_LIMIT_BYTES = 256_000;
const REALTIME_FALLBACK_POLL_INTERVAL_MS = 30_000;
const EMPTY_RUN_LOG_CHUNKS: RunLogChunk[] = [];
// Retained transcript payload budget for full task views. A byte budget (rather
// than a tiny chunk count) keeps the whole streamed scrollback intact — a
// delta-streaming run emits thousands of one-token chunks in seconds, and the
// old 200-chunk cap discarded just-rendered messages off the top irreversibly.
// If a run genuinely exceeds this, the oldest output collapses behind a visible
// marker instead of vanishing (see `applyRetentionBudget`).
const TASK_VIEW_MAX_BYTES_PER_RUN = 2_000_000;
// Grace period before an accumulated transcript buffer is pruned for a run that
// has vanished from the `runs` list. The parent refetches runs on its own
// interval, and a single transient empty/errored poll would otherwise wipe the
// buffer — forcing a rehydration that skips to the last `LOG_READ_LIMIT_BYTES`
// and silently drops already-rendered scrollback (PAP-462 B3). A run that is
// genuinely gone stays absent past this window and is then pruned as before.
const RUN_ABSENCE_PRUNE_GRACE_MS = 20_000;
export interface RunTranscriptSource {
createdAt?: string | Date;
id: string;
status: string;
adapterType: string;
runtimeMode?: "legacy" | "native";
hasStoredOutput?: boolean;
logBytes?: number | null;
lastOutputBytes?: number | null;
logRef?: string | null;
}
interface UseLiveRunTranscriptsOptions {
runs: RunTranscriptSource[];
companyId?: string | null;
/**
* Compact chunk-count cap for ticker-style consumers (dashboard). When set,
* trimming is silent — the historical behavior. Full task views omit this and
* use the byte budget below instead.
*/
maxChunksPerRun?: number;
maxBytesPerRun?: number;
logPollIntervalMs?: number;
logReadLimitBytes?: number;
enableRealtimeUpdates?: boolean;
enabled?: boolean;
}
function readString(value: unknown): string | null {
return typeof value === "string" && value.trim().length > 0 ? value : null;
}
function isTerminalStatus(status: string): boolean {
return status === "failed" || status === "timed_out" || status === "cancelled" || status === "interrupted" || status === "succeeded";
}
function canReadPersistedLog(run: RunTranscriptSource): boolean {
return run.status === "running" || isTerminalStatus(run.status);
}
function runKnownLogBytes(run: RunTranscriptSource): number | null {
const bytes = run.status === "queued"
? run.logBytes
: run.lastOutputBytes ?? run.logBytes;
return typeof bytes === "number" && Number.isFinite(bytes) && bytes > 0 ? bytes : null;
}
export function resolveInitialLogOffset(run: RunTranscriptSource, limitBytes: number): number {
const knownBytes = runKnownLogBytes(run);
if (knownBytes === null) return 0;
return Math.max(0, knownBytes - Math.max(0, limitBytes));
}
export function useLiveRunTranscripts({
runs,
companyId,
maxChunksPerRun,
maxBytesPerRun = TASK_VIEW_MAX_BYTES_PER_RUN,
logPollIntervalMs = LOG_POLL_INTERVAL_MS,
logReadLimitBytes = LOG_READ_LIMIT_BYTES,
enableRealtimeUpdates = true,
enabled = true,
}: UseLiveRunTranscriptsOptions) {
// Ticker consumers opt into the silent chunk-count cap; full task views use a
// byte budget that collapses (not discards) the oldest output when exceeded.
const { visible } = usePageVisibility();
const retentionBudget: ChunkRetentionBudget = useMemo(
() =>
typeof maxChunksPerRun === "number"
? { maxChunks: maxChunksPerRun }
: { maxBytes: maxBytesPerRun, collapseTrimmed: true },
[maxChunksPerRun, maxBytesPerRun],
);
const runsKey = useMemo(
() =>
runs
.map((run) => {
const logBytes = isTerminalStatus(run.status) && typeof run.logBytes === "number" ? run.logBytes : "";
const lastOutputBytes = isTerminalStatus(run.status) && typeof run.lastOutputBytes === "number" ? run.lastOutputBytes : "";
return `${run.id}:${run.status}:${run.adapterType}:${run.hasStoredOutput === true ? "1" : "0"}:${logBytes}:${lastOutputBytes}:${run.logRef ?? ""}`;
})
.sort((a, b) => a.localeCompare(b))
.join(","),
[runs],
);
const normalizedRuns = useMemo(() => runs.map((run) => ({ ...run })), [runsKey]);
const [chunksByRun, setChunksByRun] = useState<Map<string, RunLogChunk[]>>(new Map());
const [hydratedRunIds, setHydratedRunIds] = useState<Set<string>>(new Set());
const [errorsByRun, setErrorsByRun] = useState<ReadonlyMap<string, Error>>(new Map());
const [retryGeneration, setRetryGeneration] = useState(0);
const retry = useCallback(() => {
missingTerminalLogRunIdsRef.current.clear();
setRetryGeneration((value) => value + 1);
}, []);
const seenChunkKeysRef = useRef(new Set<string>());
// Highest sequenced chunk trimmed out of a run's retained window; older
// records re-delivered by the other transport are dropped instead of being
// re-inserted ahead of newer output.
const trimmedSeqFloorByRunRef = useRef(new Map<string, number>());
const pendingLogRowsByRunRef = useRef(new Map<string, string>());
const logOffsetByRunRef = useRef(new Map<string, number>());
const logReadLimitByRunRef = useRef(new Map<string, number>());
const logIdentityRef = useRef(new Map<string, string>());
const retiredLogIdentitiesRef = useRef(new Map<string, Set<string>>());
const sequencesRef = useRef(new Map<string, TranscriptSequence>());
const sequence = (id: string) => {
let value = sequencesRef.current.get(id);
if (!value) { value = new TranscriptSequence(); sequencesRef.current.set(id, value); }
return value;
};
const gapPendingRef = useRef(new Set<string>());
const refreshByRunRef = useRef(new Map<string, (force?: boolean) => void>());
const requestCatchUp = (runId: string) => {
if (gapPendingRef.current.has(runId)) return;
gapPendingRef.current.add(runId);
refreshByRunRef.current.get(runId)?.();
};
const bindLog = (runId: string, store: unknown, ref: unknown): "same" | "changed" | "stale" => {
if (typeof ref !== "string" || !ref) return "same";
const identity = `${typeof store === "string" ? store : ""}:${ref}`;
const prior = logIdentityRef.current.get(runId);
if (prior === identity) return "same";
if (retiredLogIdentitiesRef.current.get(runId)?.has(identity)) return "stale";
logIdentityRef.current.set(runId, identity);
if (!prior) return "same";
const retired = retiredLogIdentitiesRef.current.get(runId) ?? new Set<string>();
retired.add(prior);
if (retired.size > 8) retired.delete(retired.values().next().value!);
retiredLogIdentitiesRef.current.set(runId, retired);
logOffsetByRunRef.current.set(runId, 0); logReadLimitByRunRef.current.delete(runId);
pendingLogRowsByRunRef.current.delete(runId); pendingLogRowsByRunRef.current.delete(`${runId}:records`);
trimmedSeqFloorByRunRef.current.delete(runId); sequencesRef.current.delete(runId);
for (const key of seenChunkKeysRef.current) if (key.includes(`:${runId}:`)) seenChunkKeysRef.current.delete(key);
setChunksByRun(previous => new Map(previous).set(runId, []));
setTruncatedRunIds(previous => {
if (!previous.has(runId)) return previous;
const next = new Set(previous); next.delete(runId); return next;
});
return "changed";
};
const [truncatedRunIds, setTruncatedRunIds] = useState<ReadonlySet<string>>(new Set());
const missingTerminalLogRunIdsRef = useRef(new Set<string>());
// PAP-462 B3: buffered runs that dropped out of the `runs` list, mapped to the
// wall-clock deadline (ms) after which their buffer may be pruned. A run still
// inside its grace window is retained across the empty poll; `pruneTick` fires
// the effect again once the nearest deadline elapses so a run that stays gone
// is eventually cleaned up even if the `runs` list never changes again.
const absenceDeadlineByRunRef = useRef(new Map<string, number>());
const prevKnownRunIdsRef = useRef(new Set<string>());
const [pruneTick, setPruneTick] = useState(0);
const transcriptCacheRef = useRef(new Map<string, {
adapterType: string;
chunks: RunLogChunk[];
censorUsernameInLogs: boolean;
parserTick: number;
transcript: TranscriptEntry[];
}>());
const previousCompanyRef = useRef(companyId);
useEffect(() => {
if (previousCompanyRef.current === companyId) return;
previousCompanyRef.current = companyId;
setChunksByRun(new Map()); setHydratedRunIds(new Set()); setErrorsByRun(new Map());
seenChunkKeysRef.current.clear(); trimmedSeqFloorByRunRef.current.clear();
pendingLogRowsByRunRef.current.clear(); logOffsetByRunRef.current.clear(); logReadLimitByRunRef.current.clear();
logIdentityRef.current.clear(); retiredLogIdentitiesRef.current.clear(); sequencesRef.current.clear();
gapPendingRef.current.clear(); missingTerminalLogRunIdsRef.current.clear();
absenceDeadlineByRunRef.current.clear(); prevKnownRunIdsRef.current.clear(); transcriptCacheRef.current.clear();
}, [companyId]);
// Tick counter to force transcript recomputation when dynamic parser loads
const [parserTick, setParserTick] = useState(0);
useEffect(() => {
return onAdapterChange(() => setParserTick((t) => t + 1));
}, []);
const { data: generalSettings } = useQuery({
queryKey: queryKeys.instance.generalSettings,
queryFn: () => instanceSettingsApi.getGeneral(),
});
const runById = useMemo(() => new Map(normalizedRuns.map((run) => [run.id, run])), [normalizedRuns]);
const activeRunIds = useMemo(
() => new Set(normalizedRuns.filter((run) => run.status === "running").map((run) => run.id)),
[normalizedRuns],
);
const runIdsKey = useMemo(
() => normalizedRuns.map((run) => run.id).sort((a, b) => a.localeCompare(b)).join(","),
[normalizedRuns],
);
const appendChunks = (runId: string, chunks: Array<RunLogChunk & { dedupeKey: string }>) => {
if (chunks.length === 0) return;
setChunksByRun((prev) => {
const prevChunks = prev.get(runId) ?? [];
const { chunks: merged, changed } = mergeRunLogChunks(
runId,
prevChunks,
chunks,
{
seenChunkKeys: seenChunkKeysRef.current,
trimmedSeqFloorByRun: trimmedSeqFloorByRunRef.current,
},
retentionBudget,
);
if (!changed) return prev;
const next = new Map(prev);
next.set(runId, merged);
return next;
});
};
useEffect(() => {
const knownRunIds = new Set(normalizedRuns.map((run) => run.id));
const now = Date.now();
const deadlines = absenceDeadlineByRunRef.current;
// PAP-462 B3: a run that just disappeared from the list starts its grace
// clock; one that reappeared clears any pending deadline. Comparing against
// the previous known set means a transient empty poll only *arms* the timer
// rather than pruning the buffer outright.
for (const runId of prevKnownRunIdsRef.current) {
if (!knownRunIds.has(runId) && !deadlines.has(runId)) {
deadlines.set(runId, now + RUN_ABSENCE_PRUNE_GRACE_MS);
}
}
for (const runId of knownRunIds) {
deadlines.delete(runId);
}
prevKnownRunIdsRef.current = knownRunIds;
// Retain known runs plus any absent run still inside its grace window; only
// runs absent past their deadline are actually pruned.
const retainedRunIds = new Set(knownRunIds);
let soonestExpiryMs = Number.POSITIVE_INFINITY;
for (const [runId, deadline] of deadlines) {
if (deadline > now) {
retainedRunIds.add(runId);
soonestExpiryMs = Math.min(soonestExpiryMs, deadline);
} else {
deadlines.delete(runId);
}
}
setChunksByRun((prev) => {
const next = new Map<string, RunLogChunk[]>();
for (const [runId, chunks] of prev) {
if (retainedRunIds.has(runId)) {
next.set(runId, chunks);
}
}
return next.size === prev.size ? prev : next;
});
setHydratedRunIds((prev) => {
const next = new Set<string>();
for (const runId of prev) {
if (retainedRunIds.has(runId)) {
next.add(runId);
}
}
return next.size === prev.size ? prev : next;
});
setErrorsByRun((previous) => {
const next = new Map([...previous].filter(([id]) => retainedRunIds.has(id)));
return next.size === previous.size ? previous : next;
});
setTruncatedRunIds((previous) => {
const next = new Set([...previous].filter((id) => retainedRunIds.has(id)));
return next.size === previous.size ? previous : next;
});
for (const key of pendingLogRowsByRunRef.current.keys()) {
const runId = key.replace(/:records$/, "");
if (!retainedRunIds.has(runId)) {
pendingLogRowsByRunRef.current.delete(key);
}
}
for (const runId of logOffsetByRunRef.current.keys()) {
if (!retainedRunIds.has(runId)) {
logOffsetByRunRef.current.delete(runId);
logReadLimitByRunRef.current.delete(runId);
}
}
for (const runId of trimmedSeqFloorByRunRef.current.keys()) {
if (!retainedRunIds.has(runId)) {
trimmedSeqFloorByRunRef.current.delete(runId);
}
}
for (const runId of missingTerminalLogRunIdsRef.current.keys()) {
if (!retainedRunIds.has(runId)) {
missingTerminalLogRunIdsRef.current.delete(runId);
}
}
for (const runId of transcriptCacheRef.current.keys()) {
if (!retainedRunIds.has(runId)) {
transcriptCacheRef.current.delete(runId);
}
}
for (const map of [logIdentityRef.current, retiredLogIdentitiesRef.current, sequencesRef.current]) {
for (const id of map.keys()) if (!retainedRunIds.has(id)) map.delete(id);
}
for (const id of gapPendingRef.current) if (!retainedRunIds.has(id)) gapPendingRef.current.delete(id);
// Re-run once the nearest grace window elapses so a run that stays gone is
// pruned even if `normalizedRuns` never changes again.
if (soonestExpiryMs !== Number.POSITIVE_INFINITY) {
const timer = window.setTimeout(
() => setPruneTick((tick) => tick + 1),
Math.max(0, soonestExpiryMs - now) + 50,
);
return () => window.clearTimeout(timer);
}
}, [normalizedRuns, pruneTick]);
const stream = useTranscriptLiveStream([...activeRunIds], { companyId, enabled: enabled && enableRealtimeUpdates }, event => {
if (event.companyId !== companyId) return;
const payload = event.payload ?? {};
const runId = readString(payload["runId"]);
if (!runId) return;
if (!runById.has(runId)) return;
if (event.type === "heartbeat.run.log") {
const binding = bindLog(runId, payload.logStore, payload.logRef);
if (binding === "stale") return;
const seq = readChunkSeq(payload.seq);
if (payload.truncated === true) sequence(runId).requireHistory(seq);
const gap = sequence(runId).observe(seq);
if (binding === "changed" || payload.truncated === true || gap) requestCatchUp(runId);
if (payload.truncated === true) return;
const chunk = readString(payload["chunk"]);
if (!chunk) return;
const ts = readString(payload["ts"]) ?? event.createdAt;
const stream =
readString(payload["stream"]) === "stderr"
? "stderr"
: readString(payload["stream"]) === "system"
? "system"
: "stdout";
appendChunks(runId, [{
ts,
stream,
chunk,
seq: readChunkSeq(payload["seq"]),
dedupeKey: `log:${runId}:${ts}:${stream}:${chunk}`,
}]);
return;
}
if (event.type === "heartbeat.run.event") {
const seq = typeof payload["seq"] === "number" ? payload["seq"] : null;
const eventType = readString(payload["eventType"]) ?? "event";
const messageText = readString(payload["message"]) ?? eventType;
appendChunks(runId, [{
ts: event.createdAt,
stream: eventType === "error" ? "stderr" : "system",
chunk: messageText,
dedupeKey: `socket:event:${runId}:${seq ?? `${eventType}:${messageText}:${event.createdAt}`}`,
}]);
return;
}
if (event.type === "heartbeat.run.status") {
const status = readString(payload["status"]) ?? "updated";
if (isTerminalStatus(status)) refreshByRunRef.current.get(runId)?.(true);
appendChunks(runId, [{
ts: event.createdAt,
stream: isTerminalStatus(status) && status !== "succeeded" ? "stderr" : "system",
chunk: `run ${status}`,
dedupeKey: `socket:status:${runId}:${status}:${readString(payload["finishedAt"]) ?? ""}`,
}]);
}
});
useEffect(() => {
if (!visible || !enabled) return;
const readableRuns = normalizedRuns.filter(canReadPersistedLog);
if (readableRuns.length === 0) return;
let cancelled = false;
const controller = new AbortController();
const inFlightRunIds = new Set<string>();
const retryTimers = new Map<string, number>();
const pendingRefresh = new Set<string>();
const readRunLog = async (run: RunTranscriptSource) => {
if (cancelled || missingTerminalLogRunIdsRef.current.has(run.id) || inFlightRunIds.has(run.id)) {
return;
}
pendingRefresh.delete(run.id);
window.clearTimeout(retryTimers.get(run.id));
retryTimers.delete(run.id);
inFlightRunIds.add(run.id);
let failed = false;
let alignStartToRecord = !logOffsetByRunRef.current.has(run.id);
let offset = logOffsetByRunRef.current.get(run.id) ?? resolveInitialLogOffset(run, logReadLimitBytes);
let rewinding = false;
const previousReadLimit = logReadLimitByRunRef.current.get(run.id);
if (previousReadLimit !== undefined && logReadLimitBytes > previousReadLimit) {
const expandedOffset = resolveInitialLogOffset(run, logReadLimitBytes);
if (expandedOffset < offset) {
alignStartToRecord = true;
offset = expandedOffset;
rewinding = true;
}
}
let recordInitialWindow = previousReadLimit === undefined || rewinding;
try {
for (;;) {
const result = await readTranscriptRequest(
(signal) => heartbeatsApi.log(run.id, offset, logReadLimitBytes, {
signal,
...(alignStartToRecord && offset > 0 ? { alignStartToRecord: true } : {}),
}), controller.signal,
);
if (cancelled) return;
const binding = bindLog(run.id, result.store, result.logRef);
if (binding === "stale") { failed = true; return; }
if (binding === "changed" && offset !== 0) { offset = 0; continue; }
if (recordInitialWindow || binding === "changed") {
const windowIsTruncated = offset > 0;
setTruncatedRunIds(previous => {
if (previous.has(run.id) === windowIsTruncated) return previous;
const next = new Set(previous);
if (windowIsTruncated) next.add(run.id);
else next.delete(run.id);
return next;
});
recordInitialWindow = false;
}
logReadLimitByRunRef.current.set(run.id, logReadLimitBytes);
if (rewinding) {
pendingLogRowsByRunRef.current.delete(run.id);
pendingLogRowsByRunRef.current.delete(`${run.id}:records`);
rewinding = false;
}
alignStartToRecord = false;
setErrorsByRun(previous => {
if (!previous.has(run.id)) return previous;
const next = new Map(previous); next.delete(run.id); return next;
});
const rows = parsePersistedLogContent(run.id, result.content, pendingLogRowsByRunRef.current);
appendChunks(run.id, rows);
for (const row of rows) sequence(run.id).confirm(row.seq);
const next = result.nextOffset ?? offset + new TextEncoder().encode(result.content).byteLength;
logOffsetByRunRef.current.set(run.id, next);
if (result.nextOffset === undefined || result.content.length === 0 || next <= offset) break;
offset = next;
}
failed = sequence(run.id).needsCatchup;
if (!failed) gapPendingRef.current.delete(run.id);
} catch (error) {
if (cancelled) return;
failed = true;
if (error instanceof ApiError && error.status === 404) {
setErrorsByRun((previous) => {
if (!previous.has(run.id)) return previous;
const next = new Map(previous);
next.delete(run.id);
return next;
});
// A newly started run may not have created its log yet.
if (isTerminalStatus(run.status)) { missingTerminalLogRunIdsRef.current.add(run.id); failed = false; }
} else {
setErrorsByRun((previous) => {
if (previous.has(run.id)) return previous;
const next = new Map(previous);
next.set(run.id, error instanceof Error ? error : new Error("Run history could not be loaded"));
return next;
});
}
} finally {
inFlightRunIds.delete(run.id);
if (!cancelled) {
if (pendingRefresh.has(run.id)) {
missingTerminalLogRunIdsRef.current.delete(run.id);
void readRunLog(run);
} else if (failed && stream.healthy) {
const timer = window.setTimeout(() => { retryTimers.delete(run.id); void readRunLog(run); }, LOG_POLL_INTERVAL_MS);
retryTimers.set(run.id, timer);
}
setHydratedRunIds((prev) => {
if (prev.has(run.id)) return prev;
const next = new Set(prev);
next.add(run.id);
return next;
});
}
}
};
for (const run of readableRuns) {
refreshByRunRef.current.set(run.id, (force = false) => {
if (force) { pendingRefresh.add(run.id); missingTerminalLogRunIdsRef.current.delete(run.id); }
void readRunLog(run);
});
void readRunLog(run);
}
const activeRuns = readableRuns.filter((run) => run.status === "running");
// The realtime websocket is the primary live source when enabled, so the
// recurring poll only needs to run as a slow fallback rather than doubling
// the live update work every couple of seconds.
const effectivePollMs = enableRealtimeUpdates
? Math.max(logPollIntervalMs, REALTIME_FALLBACK_POLL_INTERVAL_MS)
: logPollIntervalMs;
const interval = !stream.healthy && activeRuns.length > 0 && effectivePollMs > 0
? window.setInterval(() => {
void Promise.all(activeRuns.map((run) => readRunLog(run)));
}, effectivePollMs)
: null;
return () => {
cancelled = true;
refreshByRunRef.current.clear();
controller.abort();
for (const timer of retryTimers.values()) window.clearTimeout(timer);
if (interval !== null) window.clearInterval(interval);
};
}, [visible, enableRealtimeUpdates, logPollIntervalMs, logReadLimitBytes, normalizedRuns, runIdsKey, retryGeneration, enabled, companyId, stream.healthy, stream.epoch]);
const transcriptByRun = useMemo(() => {
const next = new Map<string, TranscriptEntry[]>();
const censorUsernameInLogs = generalSettings?.censorUsernameInLogs === true;
const cache = transcriptCacheRef.current;
const currentRunIds = new Set<string>();
for (const run of normalizedRuns) {
currentRunIds.add(run.id);
const chunks = chunksByRun.get(run.id) ?? EMPTY_RUN_LOG_CHUNKS;
const cached = cache.get(run.id);
if (
cached &&
cached.adapterType === run.adapterType &&
cached.chunks === chunks &&
cached.censorUsernameInLogs === censorUsernameInLogs &&
cached.parserTick === parserTick
) {
next.set(run.id, cached.transcript);
continue;
}
const adapter = getUIAdapter(run.adapterType);
const transcript = buildTranscript(chunks, adapter, {
censorUsernameInLogs,
});
cache.set(run.id, {
adapterType: run.adapterType,
chunks,
censorUsernameInLogs,
parserTick,
transcript,
});
next.set(run.id, transcript);
}
for (const runId of cache.keys()) {
if (!currentRunIds.has(runId)) {
cache.delete(runId);
}
}
return next;
}, [chunksByRun, generalSettings?.censorUsernameInLogs, normalizedRuns, parserTick]);
return {
transcriptByRun,
hydratedRunIds, truncatedRunIds, errorsByRun, retry,
isInitialHydrating: normalizedRuns.some((run) => canReadPersistedLog(run) && !hydratedRunIds.has(run.id)),
hasOutputForRun(runId: string) {
return (chunksByRun.get(runId)?.length ?? 0) > 0 || runById.get(runId)?.hasStoredOutput === true;
},
};
}