mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-09 07:10:34 +01:00
perf(chat): speed up cached returns and preserve bottom follow (#19452)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Agent Chat shows saved messages and native run output in one conversation. > - Reopening a chat rebuilt terminal transcripts and waited for network hydration. > - A queued scroll event could also disable bottom-follow after a cached editor increased the content height. > - This pull request reuses terminal history and preserves bottom-follow through that event order. > - The benefit is a faster cached return that opens at the latest reply. ## Linked Issues or Issue Description **What happened?** Returning to an agent chat could leave its latest reply below the viewport. Saved native transcripts also held the first reveal behind repeated network reads. **Expected behavior** A new chat navigation opens at the latest reply. A reader who scrolls up keeps their position. Back restores the earlier reading anchor. Saved history can render while the normal catch-up read runs. **Steps to reproduce** 1. Open an agent chat with saved native runs and task question cards. 2. Open Tasks, then reopen the same agent from Recents. 3. Observe the wait and the distance from the latest reply. **Paperclip version or commit** Reproduced on `523818ea9bf81f96abf4e6b73e81c2ce87518c07`. Rebased onto master `2f9d3d43c3`. **Deployment mode** Built from source with production UI assets and an isolated synthetic database. Related work: #18358 added recent-window chat loading. #18461 deferred completed transcripts for compact task opens. Open PR #14667 changes the reveal gate for task threads. This PR keeps that gate and caches hydrated terminal native history across visits. ## What Changed - Retain successful terminal native history in the application query cache. Keys include company, run, status and projection. - Render matching saved history immediately and continue the normal cursor-based catch-up read. - Preserve bottom-follow when content grows before a queued scroll event is delivered. Upward gestures and Back anchors retain priority. - Add regression coverage for remounts, catch-up, cache clearing, projection changes, cached question editors and scroll event ordering. - Repair two browser assertions: use a final 422 failure for explicit history retry, and accept saved task-view query parameters. - Document the local before/after comparison, including unchanged paths. ## Verification - Focused transcript and scroll suites: 51 tests passed after the review fixes, including a 300-run cache stability case. - Full UI suite on the rebased head: 896 files passed; 11,257 tests passed and one skipped. - Final review-fixed production UI: all 24 desktop/mobile chat journeys passed. - Final head: `pnpm -r typecheck`, `pnpm build` and `pnpm check:token-gates` passed. The complete GitHub CI test matrix, E2E, build and release checks passed for `4429a9aab4`. - The local repo-wide test command was stopped before performance sampling. Local acceptance used the full UI suite and the chat browser rig; CI completed the full repository matrix. - A parallel local build initially read a partially regenerated runner module while typecheck was emitting files. After typecheck finished, the serial full build passed. No runner source change was required. - The first browser run exposed the retry fixture's synthetic 500 being retried by the shared query policy. A final 422 response now verifies explicit retry. The Back assertion also now accepts the task collection's saved view query. - Re-measured the final reviewed build against the original baseline in five alternating pairs. The same retained 600-message/300-run fixture, Chromium at 1440×900 and 150 ms simulated latency produced cached-return medians of 580 ms (568–594) before and 155 ms (130–161) after. Median paired saving: 433 ms. All five pairs improved. A separate loopback asset host kept the manual test-drive build unchanged. - The initial full comparison found no reliable change to loopback cached returns, cold opens or uncached in-app entry. These are local results, not deployed staging measurements. - Greptile identified cache-expiry row loss and missing older output after a cached projection switch. Regression cases reproduce those paths. Mounted history now retains restored rows, and each projection retains its confirmed snapshot. The fresh review on commit `4429a9aab4` is 5/5 with no open findings. - One browser run overlapped a local rebuild and received a root-page 500. The server log records the missing generated index at that time. The run also timed out waiting for wheel acknowledgement after older rows had loaded; that input-acknowledgement cause was not established. After the build completed, the full serial run passed all 24 cases. - Run the opt-in browser rig with `tests/perf/agent-chat/playwright.config.ts` against a disposable loopback instance. See `tests/perf/agent-chat/README.md`. ## Risks - Cached history must not cross company, status or projection boundaries. Regression tests cover those cache misses and cache clearing. - Terminal output can receive late events. Cached history still performs catch-up reads, and retry, reconnect and mode changes retain their reads. - Programmatic scroll events can race with a reader gesture. Tests cover upward intent, delayed media and Back restoration. - This change does not improve cold-start latency. No schema, server authorization or exported data changes. ## Model Used OpenAI Codex using GPT-6. The runtime does not expose a more specific model ID or context-window size. Used reasoning, shell execution, repository editing, tests and browser profiling. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
1 parent
51a65af9b8
commit
eb5c24b61c
6 files changed
+274
-12
No files matched your search
@@ -38,6 +38,47 @@ visible recent conversation at its intended position with an editable composer,
|
||||
not the loading shell. Retain per-frame geometry and server timings alongside
|
||||
sample medians/ranges; do not treat fewer transcript requests as latency proof.
|
||||
|
||||
Terminal native transcripts read during a visit are retained in the application's
|
||||
query cache by company, run, status and projection. A cached return can reveal
|
||||
those rows immediately while the normal cursor-based catch-up read runs in the
|
||||
background. Live runs and uncached projections still wait for hydration. Cache
|
||||
reuse does not remove reconnect or retry reads. Return navigation also exercises
|
||||
cached question editors: a scroll event queued before their layout grows must
|
||||
retain bottom-follow, while an upward gesture and Back reading anchors still win.
|
||||
Mounted rows survive query-cache eviction. Expanding a cached preview restores
|
||||
the full projection before catch-up, including older rows and late events.
|
||||
|
||||
### Local comparison, 2026-11-06
|
||||
|
||||
Compared baseline `523818ea9bf81f96abf4e6b73e81c2ce87518c07` with the
|
||||
terminal-history cache and scroll-event fix using production assets, Chromium
|
||||
151.0.7922.34, a 1440×900 viewport and the same retained 600-message/300-run
|
||||
fixture (with the rig's saved live replies). Five alternating pairs per profile;
|
||||
no CPU throttling, profiling, builds or this task's test suites during samples.
|
||||
Other worktrees remained active on the shared host. Browser contexts had fresh
|
||||
HTTP/application caches for direct opens; in-app entry started from a freshly
|
||||
loaded task collection; return reused that visit's application cache.
|
||||
|
||||
Times are median milliseconds, with min–max in parentheses. Readiness means a
|
||||
revealed conversation at the bottom with an editable composer.
|
||||
|
||||
| Network | Journey | Baseline | Candidate | Median paired saving |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Loopback | Direct open | 439 (420–489) | 425 (409–474) | 10 |
|
||||
| Loopback | In-app entry | 277 (268–284) | 279 (271–285) | −3 |
|
||||
| Loopback | Cached return | 115 (106–120) | 118 (113–120) | −4 |
|
||||
| 150 ms latency | Direct open | 3586 (3573–3597) | 3575 (3418–3595) | 10 |
|
||||
| 150 ms latency | In-app entry | 1956 (1930–1959) | 1958 (1955–1961) | −4 |
|
||||
| 150 ms latency | Cached return | 547 (538–553) | 131 (120–137) | 416 |
|
||||
|
||||
Only the slower-network cached return establishes a reliable speed improvement:
|
||||
all five paired deltas are positive. Direct opens and uncached navigation remain
|
||||
unchanged. The unchanged server's issue-read timing medians were 18.2 vs 16.9 ms
|
||||
on the latency profile (20 reads per variant). Baseline and candidate entry assets
|
||||
were `index-8fGouAl3.js` and `index-BVkn60XK.js`. This is local synthetic proof,
|
||||
not a claim about deployed staging latency. The cached-question reproduction went
|
||||
from a persistent 198-pixel bottom gap to zero.
|
||||
|
||||
The chat's pending task questions use the attention service's scoped question
|
||||
projection. It keeps current access, audience, dismissal, queue, snooze and archive
|
||||
rules without constructing the company feed. Ranks are relative to the returned
|
||||
|
||||
@@ -2,6 +2,7 @@ import fs from "node:fs/promises";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { expect, test, type Page } from "@playwright/test";
|
||||
import { longChatFixture, type ChatFixture } from "./agent-chat-fixture";
|
||||
import { coordinatorTaskQuestion, coordinatorTaskQuestionAttention } from "../../../ui/src/fixtures/coordinatorTaskQuestion";
|
||||
|
||||
let fixture: ChatFixture;
|
||||
let chatPath: string;
|
||||
@@ -96,6 +97,38 @@ test("cold long chat reveals one recent window without full-history reads or red
|
||||
await page.screenshot({ path: info.outputPath("recent-chat.png") });
|
||||
});
|
||||
|
||||
test("clicking away and reopening cached questions keeps the latest reply in view", async ({ page }, info) => {
|
||||
const question = {
|
||||
...coordinatorTaskQuestion,
|
||||
payload: { ...coordinatorTaskQuestion.payload, questionSet: {
|
||||
schema: "paperclip.question_set.v1", questions: [
|
||||
{ id: "reply", prompt: "Give a reply", answerMode: "text", required: true },
|
||||
],
|
||||
} },
|
||||
};
|
||||
await page.route("**/pending-questions", route => route.fulfill({ json: [coordinatorTaskQuestionAttention] }));
|
||||
await page.route("**/api/issues/task-launch/interactions", async route => {
|
||||
await new Promise(resolve => setTimeout(resolve, 600));
|
||||
await route.fulfill({ json: [question] });
|
||||
});
|
||||
const gap = () => scroller(page).evaluate(root => root.scrollHeight - root.clientHeight - root.scrollTop);
|
||||
await page.goto(chatPath); await ready(page);
|
||||
await expect(page.getByTestId("question-text-answer-composer")).toBeVisible();
|
||||
await expect.poll(gap).toBeLessThanOrEqual(2);
|
||||
await editor(page).fill("Keep my draft on return.");
|
||||
const navigation = page.viewportSize()!.width < 768
|
||||
? page.getByRole("navigation", { name: "Mobile navigation" })
|
||||
: page.getByRole("navigation", { name: "Workspace navigation" });
|
||||
await navigation.getByRole("link", { name: "Tasks", exact: true }).click();
|
||||
await navigation.getByRole("link", { name: page.viewportSize()!.width < 768 ? /^Chat/ : "History Fixture", exact: true }).click();
|
||||
await ready(page);
|
||||
await expect.poll(gap).toBeLessThanOrEqual(2);
|
||||
await expect(editor(page)).toContainText("Keep my draft on return.");
|
||||
await page.waitForTimeout(400);
|
||||
expect(await gap()).toBeLessThanOrEqual(2);
|
||||
await page.screenshot({ path: info.outputPath("cached-return.png") });
|
||||
});
|
||||
|
||||
test("upward scrolling loads one delayed page and preserves the row, viewport and draft", async ({ page }, info) => {
|
||||
let release!: () => void;
|
||||
const held = new Promise<void>(resolve => { release = resolve; });
|
||||
@@ -129,7 +162,7 @@ test("upward scrolling loads one delayed page and preserves the row, viewport an
|
||||
test("failed history stops automatic retries and leaves a working inline retry", async ({ page }) => {
|
||||
let attempts = 0;
|
||||
await page.route(`**/api/issues/${fixture.issue.id}/comments?**`, async route => {
|
||||
if (++attempts === 1) await route.fulfill({ status: 500, json: { error: "Synthetic older-page failure" } });
|
||||
if (++attempts === 1) await route.fulfill({ status: 422, json: { error: "Synthetic older-page failure" } });
|
||||
else await route.continue();
|
||||
});
|
||||
await page.goto(chatPath); await ready(page);
|
||||
@@ -157,7 +190,7 @@ test("Back restores paged history and the reading position", async ({ page }) =>
|
||||
? page.getByRole("navigation", { name: "Mobile navigation" })
|
||||
: page.getByRole("navigation", { name: "Workspace navigation" });
|
||||
await navigation.getByRole("link", { name: "Tasks", exact: true }).click();
|
||||
await expect(page).toHaveURL(/\/issues$/);
|
||||
await expect(page).toHaveURL(/\/issues(?:\?|$)/);
|
||||
await page.goBack(); await ready(page);
|
||||
await expect(page.locator('[id^="comment-"]')).toHaveCount(70);
|
||||
expect(Math.abs(await offset(page, saved.id) - saved.top)).toBeLessThanOrEqual(2);
|
||||
|
||||
@@ -186,6 +186,23 @@ describe("TaskMessageScroller", () => {
|
||||
expect(scroller().scrollTop).toBe(1000);
|
||||
});
|
||||
|
||||
it("keeps following when content grows before the previous scroll event is delivered", async () => {
|
||||
render(0);
|
||||
fakeGeometry(scroller(), { clamp: true });
|
||||
render(1);
|
||||
expect(scroller().scrollTop).toBe(600);
|
||||
Object.defineProperty(scroller(), "scrollHeight", { value: 1400, configurable: true });
|
||||
await dispatch(scroller(), new Event("scroll", { bubbles: true }));
|
||||
expect(scroller().scrollTop).toBe(1000);
|
||||
expect(pill()).toBeNull();
|
||||
scroller().dispatchEvent(new WheelEvent("wheel", { bubbles: true, deltaY: -100 }));
|
||||
Object.defineProperty(scroller(), "scrollHeight", { value: 1600, configurable: true });
|
||||
await scrollTo(scroller(), 900);
|
||||
render(2);
|
||||
expect(scroller().scrollTop).toBe(900);
|
||||
await waitForPill(true);
|
||||
});
|
||||
|
||||
it("retains the reading anchor when a resize scroll event precedes the next width reflow", async () => {
|
||||
const resize: (() => void)[] = [];
|
||||
vi.stubGlobal("ResizeObserver", class {
|
||||
|
||||
@@ -68,6 +68,7 @@ export function TaskMessageScroller({ children, contentKey, className, scrolling
|
||||
const scrollbarIdleDelayRef = useRef<number | null>(null);
|
||||
const [pillPhase, setPillPhase] = useState<PillPhase>("hidden");
|
||||
const previousScrollTop = useRef(0);
|
||||
const previousScrollHeight = useRef(0);
|
||||
const touchStartY = useRef<number | null>(null);
|
||||
const requestingHistory = useRef(false);
|
||||
const upwardIntent = useRef(false);
|
||||
@@ -186,6 +187,7 @@ export function TaskMessageScroller({ children, contentKey, className, scrolling
|
||||
// the reader scrolls or navigation selects a different position.
|
||||
rememberAnchor(false);
|
||||
previousScrollTop.current = el.scrollTop;
|
||||
previousScrollHeight.current = el.scrollHeight;
|
||||
}, [rememberAnchor, scrollToBottom, hidePill]);
|
||||
|
||||
const handleScroll = useCallback(() => {
|
||||
@@ -194,6 +196,12 @@ export function TaskMessageScroller({ children, contentKey, className, scrolling
|
||||
const moved = top !== previousScrollTop.current;
|
||||
const scrollingUp = top < previousScrollTop.current;
|
||||
previousScrollTop.current = top;
|
||||
if (previousScrollHeight.current > 0 && pinnedRef.current && !scrollingUp && !upwardIntent.current &&
|
||||
ref.current.scrollHeight > previousScrollHeight.current) {
|
||||
reconcileContent();
|
||||
hidePill();
|
||||
return;
|
||||
}
|
||||
if ((scrollingUp || upwardIntent.current) && top <= 192 && !easingRef.current) requestEarlierHistory();
|
||||
showScrollbarWhileScrolling();
|
||||
// A growing composer shrinks this viewport. Some browsers dispatch the
|
||||
@@ -218,6 +226,7 @@ export function TaskMessageScroller({ children, contentKey, className, scrolling
|
||||
return;
|
||||
}
|
||||
pinnedRef.current = pinned;
|
||||
if (pinned) upwardIntent.current = false;
|
||||
rememberAnchor(moved);
|
||||
if (pinned) hidePill();
|
||||
else showPill();
|
||||
@@ -229,10 +238,12 @@ export function TaskMessageScroller({ children, contentKey, className, scrolling
|
||||
showPill,
|
||||
showScrollbarWhileScrolling,
|
||||
requestEarlierHistory,
|
||||
reconcileContent,
|
||||
]);
|
||||
|
||||
const handleJumpToLatest = useCallback(() => {
|
||||
pendingHashTarget.current = null;
|
||||
upwardIntent.current = false;
|
||||
const el = ref.current;
|
||||
if (!el) return;
|
||||
if (isPinned()) {
|
||||
|
||||
@@ -5,6 +5,7 @@ import { createRoot, type Root } from "react-dom/client";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { useNativeRunTranscripts } from "./useNativeRunTranscripts";
|
||||
import { TRANSCRIPT_REQUEST_TIMEOUT_MS } from "./read-transcript-request";
|
||||
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
|
||||
|
||||
const eventsMock = vi.hoisted(() => vi.fn());
|
||||
const conversationEventsMock = vi.hoisted(() => vi.fn());
|
||||
@@ -39,6 +40,7 @@ describe("useNativeRunTranscripts", () => {
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
eventsMock.mockReset();
|
||||
conversationEventsMock.mockReset();
|
||||
container = document.createElement("div");
|
||||
document.body.appendChild(container);
|
||||
root = createRoot(container);
|
||||
@@ -73,6 +75,111 @@ describe("useNativeRunTranscripts", () => {
|
||||
expect(state!.transcriptByRun.get("old")?.filter((entry) => entry.kind === "assistant").map((entry) => entry.text)).toEqual(["Earlier answer", "Latest answer"]);
|
||||
});
|
||||
|
||||
it("reopens saved terminal history immediately and catches up without duplicating saved events", async () => {
|
||||
const client = new QueryClient();
|
||||
let latest!: ReturnType<typeof useNativeRunTranscripts>;
|
||||
const row = (seq: number) => ({ id: `event-${seq}`, runId: "saved", companyId: "company", createdAt: new Date().toISOString(), seq, eventType: "item.completed", payload: { prpEvent: {
|
||||
schema: "paperclip.prp.event.v1", schemaVersion: 1, eventType: "item.completed", runId: "saved",
|
||||
emittedAt: new Date().toISOString(), itemId: `answer-${seq}`, payload: { kind: "agentMessage", text: `Answer ${seq}` },
|
||||
} } });
|
||||
function Saved({ companyId = "company", compact = true, status = "succeeded", extra = false }) {
|
||||
latest = useNativeRunTranscripts([
|
||||
{ id: "saved", status, runtimeMode: "native", conversationOnly: compact },
|
||||
...(extra ? [{ id: "earlier", status, runtimeMode: "native" as const, conversationOnly: compact }] : []),
|
||||
], { companyId });
|
||||
return null;
|
||||
}
|
||||
async function mount(props = {}) {
|
||||
await act(async () => root.render(<QueryClientProvider client={client}><Saved {...props} /></QueryClientProvider>));
|
||||
}
|
||||
async function leave() {
|
||||
await act(async () => root.render(null));
|
||||
}
|
||||
conversationEventsMock.mockResolvedValueOnce([row(1)]);
|
||||
await mount();
|
||||
expect(latest.hydratedRunIds.has("saved")).toBe(true);
|
||||
await leave();
|
||||
let finish!: (rows: ReturnType<typeof row>[]) => void;
|
||||
conversationEventsMock.mockImplementationOnce(() => new Promise(resolve => { finish = resolve; }));
|
||||
await mount();
|
||||
expect(latest.isInitialHydrating).toBe(false);
|
||||
expect(latest.transcriptByRun.has("saved")).toBe(true);
|
||||
expect(conversationEventsMock.mock.calls.at(-1)?.[1]).toBe(1);
|
||||
await act(async () => finish([row(2)]));
|
||||
expect(latest.transcriptByRun.get("saved")?.filter(entry => entry.kind === "assistant").map(entry => entry.text)).toEqual(["Answer 1", "Answer 2"]);
|
||||
await leave();
|
||||
conversationEventsMock.mockImplementation(() => new Promise(() => {}));
|
||||
await mount();
|
||||
expect(conversationEventsMock.mock.calls.at(-1)?.[1]).toBe(2);
|
||||
expect(latest.isInitialHydrating).toBe(false);
|
||||
expect(latest.transcriptByRun.get("saved")?.filter(entry => entry.kind === "assistant").map(entry => entry.text)).toEqual(["Answer 1", "Answer 2"]);
|
||||
await leave();
|
||||
conversationEventsMock.mockResolvedValue([]);
|
||||
await mount();
|
||||
client.removeQueries({ queryKey: ["native-run-transcript"] });
|
||||
await mount({ extra: true });
|
||||
expect(latest.transcriptByRun.get("saved")?.filter(entry => entry.kind === "assistant").map(entry => entry.text)).toEqual(["Answer 1", "Answer 2"]);
|
||||
conversationEventsMock.mockImplementation(() => new Promise(() => {}));
|
||||
for (const props of [{ companyId: "other" }, { compact: false }, { status: "running" }]) {
|
||||
await leave();
|
||||
eventsMock.mockImplementation(() => new Promise(() => {}));
|
||||
await mount(props);
|
||||
expect(latest.isInitialHydrating).toBe(true);
|
||||
expect(latest.transcriptByRun.has("saved")).toBe(false);
|
||||
}
|
||||
await leave();
|
||||
client.clear();
|
||||
await mount();
|
||||
expect(latest.isInitialHydrating).toBe(true);
|
||||
expect(latest.transcriptByRun.has("saved")).toBe(false);
|
||||
await leave();
|
||||
client.clear();
|
||||
eventsMock.mockResolvedValueOnce([row(1), row(2)]);
|
||||
await mount({ compact: false });
|
||||
conversationEventsMock.mockResolvedValueOnce([row(2)]);
|
||||
await mount();
|
||||
await leave();
|
||||
conversationEventsMock.mockResolvedValueOnce([row(3)]);
|
||||
await mount();
|
||||
expect(latest.transcriptByRun.get("saved")?.filter(entry => entry.kind === "assistant").map(entry => entry.text)).toEqual(["Answer 2", "Answer 3"]);
|
||||
eventsMock.mockResolvedValueOnce([row(3)]);
|
||||
await mount({ compact: false });
|
||||
expect(latest.transcriptByRun.get("saved")?.filter(entry => entry.kind === "assistant").map(entry => entry.text)).toEqual(["Answer 1", "Answer 2", "Answer 3"]);
|
||||
await leave();
|
||||
client.clear();
|
||||
});
|
||||
|
||||
it("reuses a 300-run saved window without rescanning its cache on unchanged renders", async () => {
|
||||
const client = new QueryClient();
|
||||
const runs = Array.from({ length: 300 }, (_, index) => ({
|
||||
id: `saved-${index}`, status: "succeeded", runtimeMode: "native" as const, conversationOnly: true,
|
||||
}));
|
||||
for (const run of runs) {
|
||||
client.setQueryData(["native-run-transcript", "company", run.id, true, run.status], { events: [], cursor: 0 });
|
||||
}
|
||||
const originalRead = client.getQueryData.bind(client);
|
||||
let reads = 0;
|
||||
vi.spyOn(client, "getQueryData").mockImplementation(((key) => {
|
||||
if (key[0] === "native-run-transcript" && ++reads > runs.length) throw new Error("Saved history was rescanned");
|
||||
return originalRead(key);
|
||||
}) as typeof client.getQueryData);
|
||||
conversationEventsMock.mockImplementation(() => new Promise(() => {}));
|
||||
let state!: ReturnType<typeof useNativeRunTranscripts>;
|
||||
function Window({ tick }: { tick: number }) {
|
||||
state = useNativeRunTranscripts(runs, { companyId: "company" });
|
||||
return <span>{tick}</span>;
|
||||
}
|
||||
for (let tick = 0; tick < 3; tick++) {
|
||||
await act(async () => root.render(<QueryClientProvider client={client}><Window tick={tick} /></QueryClientProvider>));
|
||||
expect(state.isInitialHydrating).toBe(false);
|
||||
expect(state.transcriptByRun.size).toBe(300);
|
||||
}
|
||||
expect(reads).toBe(300);
|
||||
await act(async () => root.render(null));
|
||||
client.clear();
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it("exposes event transport failures and retries terminal runs until recovery", async () => {
|
||||
eventsMock
|
||||
.mockRejectedValueOnce(new Error("event endpoint unavailable"))
|
||||
|
||||
@@ -2,7 +2,8 @@ import { TranscriptSequence } from "../../lib/transcript-sequence";
|
||||
import { useTranscriptLiveStream } from "../../context/LiveUpdatesProvider";
|
||||
import { usePageVisibility } from "../../lib/page-visibility";
|
||||
import { liveRunEvent, mergeRunEvents } from "../../lib/live-run-event";
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
import { useCallback, useContext, useEffect, useMemo, useRef, useState } from "react";
|
||||
import { QueryClientContext } from "@tanstack/react-query";
|
||||
import type { HeartbeatRunEvent } from "@paperclipai/shared";
|
||||
import type { TranscriptEntry } from "@/adapters";
|
||||
import { heartbeatsApi } from "@/api/heartbeats";
|
||||
@@ -12,6 +13,14 @@ import { readTranscriptRequest } from "./read-transcript-request";
|
||||
const EVENT_PAGE_SIZE = 1_000;
|
||||
const EVENT_POLL_INTERVAL_MS = 2_000;
|
||||
|
||||
interface SavedNativeHistory {
|
||||
events: HeartbeatRunEvent[];
|
||||
cursor: number;
|
||||
}
|
||||
|
||||
const historyKey = (companyId: string, run: NativeRunTranscriptSource) =>
|
||||
["native-run-transcript", companyId, run.id, Boolean(run.conversationOnly), run.status] as const;
|
||||
|
||||
export interface NativeRunTranscriptSource {
|
||||
id: string;
|
||||
status: string;
|
||||
@@ -29,6 +38,7 @@ function isLive(status: string): boolean {
|
||||
}
|
||||
|
||||
export function useNativeRunTranscripts(runs: readonly NativeRunTranscriptSource[], options: { companyId?: string | null; enabled?: boolean } = {}) {
|
||||
const queryClient = useContext(QueryClientContext);
|
||||
const { visible } = usePageVisibility();
|
||||
const nativeRunsKey = runs
|
||||
.filter((run) => run.runtimeMode === "native")
|
||||
@@ -41,7 +51,19 @@ export function useNativeRunTranscripts(runs: readonly NativeRunTranscriptSource
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
[nativeRunsKey],
|
||||
);
|
||||
const savedHistory = useMemo(() => {
|
||||
const saved = new Map<string, SavedNativeHistory>();
|
||||
if (queryClient && options.companyId && options.enabled !== false) {
|
||||
for (const run of nativeRuns) {
|
||||
if (isLive(run.status)) continue;
|
||||
const history = queryClient.getQueryData<SavedNativeHistory>(historyKey(options.companyId, run));
|
||||
if (history) saved.set(run.id, history);
|
||||
}
|
||||
}
|
||||
return saved;
|
||||
}, [queryClient, options.companyId, options.enabled, nativeRuns]);
|
||||
const [eventsByRun, setEventsByRun] = useState<Map<string, HeartbeatRunEvent[]>>(new Map());
|
||||
const persistedEventsByRunRef = useRef(new Map<string, HeartbeatRunEvent[]>());
|
||||
const [errorsByRun, setErrorsByRun] = useState<Map<string, NativeRunTranscriptError>>(new Map());
|
||||
const [hydratedRunIds, setHydratedRunIds] = useState<ReadonlySet<string>>(new Set());
|
||||
const [hydratedStatusByRun, setHydratedStatusByRun] = useState(new Map<string, string>());
|
||||
@@ -68,6 +90,7 @@ export function useNativeRunTranscripts(runs: readonly NativeRunTranscriptSource
|
||||
previousCompanyRef.current = options.companyId;
|
||||
setEventsByRun(new Map()); setErrorsByRun(new Map()); setHydratedRunIds(new Set()); setHydratedStatusByRun(new Map());
|
||||
projectionCacheRef.current.clear(); cursorByRunRef.current.clear(); conversationOnlyByRunRef.current.clear();
|
||||
persistedEventsByRunRef.current.clear();
|
||||
sequencesRef.current.clear(); gapPendingRef.current.clear(); interruptedReadsRef.current.clear();
|
||||
}, [options.companyId]);
|
||||
const stream = useTranscriptLiveStream(nativeRuns.filter(run => isLive(run.status)).map(run => run.id), options, event => {
|
||||
@@ -98,7 +121,12 @@ export function useNativeRunTranscripts(runs: readonly NativeRunTranscriptSource
|
||||
const changedModes = new Set<string>();
|
||||
for (const run of nativeRuns) {
|
||||
const mode = Boolean(run.conversationOnly);
|
||||
if (conversationOnlyByRunRef.current.get(run.id) !== mode) { changedModes.add(run.id); cursorByRunRef.current.delete(run.id); sequencesRef.current.delete(run.id); }
|
||||
if (conversationOnlyByRunRef.current.get(run.id) !== mode) {
|
||||
changedModes.add(run.id); cursorByRunRef.current.delete(run.id); sequencesRef.current.delete(run.id);
|
||||
}
|
||||
if (changedModes.has(run.id) || !persistedEventsByRunRef.current.has(run.id)) {
|
||||
persistedEventsByRunRef.current.set(run.id, savedHistory.get(run.id)?.events ?? []);
|
||||
}
|
||||
conversationOnlyByRunRef.current.set(run.id, mode);
|
||||
}
|
||||
let cancelled = false;
|
||||
@@ -111,7 +139,16 @@ export function useNativeRunTranscripts(runs: readonly NativeRunTranscriptSource
|
||||
const next = new Map([...previous].filter(([id]) => retainedIds.has(id)));
|
||||
return next.size === previous.size ? previous : next;
|
||||
};
|
||||
setEventsByRun(retainMap);
|
||||
setEventsByRun(previous => {
|
||||
let next = retainMap(previous);
|
||||
for (const [id, saved] of savedHistory) {
|
||||
if (!next.has(id) || changedModes.has(id)) {
|
||||
if (next === previous) next = new Map(next);
|
||||
next.set(id, mergeRunEvents(saved.events, next.get(id) ?? []));
|
||||
}
|
||||
}
|
||||
return next;
|
||||
});
|
||||
setErrorsByRun(retainMap);
|
||||
setHydratedStatusByRun(retainMap);
|
||||
setHydratedRunIds((previous) => {
|
||||
@@ -123,6 +160,9 @@ export function useNativeRunTranscripts(runs: readonly NativeRunTranscriptSource
|
||||
cursorByRunRef.current.delete(id); sequencesRef.current.delete(id); gapPendingRef.current.delete(id);
|
||||
}
|
||||
}
|
||||
for (const id of persistedEventsByRunRef.current.keys()) {
|
||||
if (!retainedIds.has(id)) persistedEventsByRunRef.current.delete(id);
|
||||
}
|
||||
|
||||
const refreshRun = async (run: NativeRunTranscriptSource) => {
|
||||
if (cancelled || historyReadsRef.current.has(run.id)) return;
|
||||
@@ -132,7 +172,8 @@ export function useNativeRunTranscripts(runs: readonly NativeRunTranscriptSource
|
||||
historyReadsRef.current.set(run.id, controller.signal);
|
||||
let failed = false;
|
||||
try {
|
||||
let cursor = cursorByRunRef.current.get(run.id) ?? 0;
|
||||
const saved = savedHistory.get(run.id);
|
||||
let cursor = cursorByRunRef.current.get(run.id) ?? saved?.cursor ?? 0;
|
||||
const observedBeforeRead = sequence(run.id).observed;
|
||||
const incoming: HeartbeatRunEvent[] = [];
|
||||
for (;;) {
|
||||
@@ -159,9 +200,19 @@ export function useNativeRunTranscripts(runs: readonly NativeRunTranscriptSource
|
||||
if (!failed) gapPendingRef.current.delete(run.id);
|
||||
if (incoming.length > 0) setEventsByRun((previous) => {
|
||||
const next = new Map(previous);
|
||||
next.set(run.id, mergeRunEvents(previous.get(run.id) ?? [], incoming));
|
||||
next.set(run.id, mergeRunEvents(previous.get(run.id) ?? saved?.events ?? [], incoming));
|
||||
return next;
|
||||
});
|
||||
const previousEvents = persistedEventsByRunRef.current.get(run.id) ?? [];
|
||||
const persistedEvents = incoming.length ? mergeRunEvents(previousEvents, incoming) : previousEvents;
|
||||
persistedEventsByRunRef.current.set(run.id, persistedEvents);
|
||||
if (!failed && !isLive(run.status) && queryClient && options.companyId) {
|
||||
const key = historyKey(options.companyId, run);
|
||||
queryClient.setQueryData<SavedNativeHistory>(key, {
|
||||
events: persistedEvents,
|
||||
cursor: coveredCursor,
|
||||
});
|
||||
}
|
||||
setErrorsByRun((previous) => {
|
||||
if (!previous.has(run.id)) return previous;
|
||||
const next = new Map(previous);
|
||||
@@ -217,12 +268,12 @@ export function useNativeRunTranscripts(runs: readonly NativeRunTranscriptSource
|
||||
}
|
||||
for (const timer of timers.values()) window.clearTimeout(timer);
|
||||
};
|
||||
}, [nativeRuns, retryGeneration, visible, options.enabled, options.companyId, stream.healthy, stream.epoch]);
|
||||
}, [nativeRuns, retryGeneration, visible, options.enabled, options.companyId, stream.healthy, stream.epoch, queryClient, savedHistory]);
|
||||
|
||||
const transcriptByRun = useMemo(() => {
|
||||
const transcripts = new Map<string, TranscriptEntry[]>();
|
||||
for (const run of nativeRuns) {
|
||||
const events = eventsByRun.get(run.id);
|
||||
const events = eventsByRun.get(run.id) ?? savedHistory.get(run.id)?.events;
|
||||
if (!events) continue;
|
||||
let cached = projectionCacheRef.current.get(run.id);
|
||||
if (!cached || cached.events !== events) {
|
||||
@@ -235,14 +286,16 @@ export function useNativeRunTranscripts(runs: readonly NativeRunTranscriptSource
|
||||
if (!transcripts.has(id)) projectionCacheRef.current.delete(id);
|
||||
}
|
||||
return transcripts;
|
||||
}, [eventsByRun, nativeRuns]);
|
||||
}, [eventsByRun, nativeRuns, savedHistory]);
|
||||
|
||||
const terminalHydratedRunIds = useMemo(() => new Set(nativeRuns
|
||||
.filter((run) => !isLive(run.status) && hydratedStatusByRun.get(run.id) === run.status)
|
||||
.map((run) => run.id)), [nativeRuns, hydratedStatusByRun]);
|
||||
|
||||
const availableRunIds = useMemo(() => new Set([...savedHistory.keys(), ...hydratedRunIds]), [savedHistory, hydratedRunIds]);
|
||||
const availableTerminalRunIds = useMemo(() => new Set([...savedHistory.keys(), ...terminalHydratedRunIds]), [savedHistory, terminalHydratedRunIds]);
|
||||
return {
|
||||
transcriptByRun, errorsByRun, hydratedRunIds, terminalHydratedRunIds, retry,
|
||||
isInitialHydrating: nativeRuns.some((run) => !hydratedRunIds.has(run.id)),
|
||||
transcriptByRun, errorsByRun, hydratedRunIds: availableRunIds, terminalHydratedRunIds: availableTerminalRunIds, retry,
|
||||
isInitialHydrating: nativeRuns.some((run) => !availableRunIds.has(run.id)),
|
||||
};
|
||||
}
|
||||
Reference in new issue
Block a user