mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-16 20:12:20 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every pull request runs the "Runner support suite" CI job (`pnpm test:e2e:runner:unit`) > - That job took 8.1–10.6 minutes, depending on the spot instance type, and was one of the last PR jobs to finish > - The target for every PR job is 7 minutes or less > - Measurement showed the suite is CPU-bound: each of the 693 Vitest files loads `@playwright/test`, the runner catalog and shared modules again in its own worker, and a few modules did quadratic or repeated work when they loaded > - This pull request removes that repeated work and reuses compiled code across workers > - The benefit is a job 75–86 s (13–16%) shorter on each instance type, with the same files, tests and assertions. It does not reach 7 minutes; the floor is explained below ## Linked Issues or Issue Description **What existing behavior does this improve?** The duration of the "Runner support suite" PR job. **Subsystem affected** CI, the Runner E2E support suite under `tests/runner-e2e/`, and one shared module (`packages/shared/src/connection-routing.ts`). **Current behavior** In 26 recent green PR runs the job took a median of 526 s on `m7i-flex.xlarge` (17 runs), 549 s on `m7i.xlarge` (3 runs) and 604 s on `m6a.xlarge` (6 runs). Vitest ran 693 files on 3 workers. The CPU was about 82% busy. 52% of the tracked time was module evaluation. **Proposed behavior** The job is shorter on every instance type. The same files and tests pass, and no assertion changes. **Reason and benefit** The PR workflow finishes sooner, so every pull request gets its result sooner. **Breaking changes** None. ## What Changed Measurements below are from the job's own runs on this PR. The spot fleet picks `m7i-flex`, `m7i`, `m6a` or `m5a` (all 4 vCPU); speed differs by up to 2x between them, so each comparison uses the same instance type. This suite has no Vitest retries, so a flaky test fails the job. ### Baseline (recent green PR runs on master, before this PR) | Run (job id) | Job | Test step | Vitest wall | Vitest split | | --- | --- | --- | --- | --- | | 131933126599 | 523 s | 474 s | 469.8 s | import 52%, tests 43%, transform 5% | | 131932775003 | 510 s | 461 s | 457.1 s | import 51%, tests 44%, transform 5% | | 131926764360 | 527 s | 479 s | 475.2 s | import 51%, tests 43%, transform 5% | | 131924082813 | 541 s | 496 s | – | – | | 131912527065 | 517 s | 468 s | – | – | - One Vitest invocation (693 files, 3 workers, the Vitest default of CPUs − 1), then 386 `node:test` tests in about 3 s. About 45–50 s of the job is checkout and dependency restore before the test step. - Summed tracked time was about 1,297 s for 470 s of wall time: effective concurrency 2.76 of 3 workers. Module evaluation was 674 s: 4,164 modules evaluated 77,699 times, because each file runs in a fresh isolated worker. - A CPU sampler (temporary, removed) showed 82% average CPU use with 3 workers and 95–99% with 4 workers. With 4 workers and no other change the wall time did not improve (455 s). The suite was CPU-bound, not idle. - Top modules by summed evaluation time (diagnostic run): `@playwright/test` 158 s over 390 files, `tests/runner-e2e/catalog.ts` 56 s over 141 files, `packages/shared/src/connection-routing.ts` 40 s over 317 files, `acpx/dist/runtime.js` 39 s. - Slowest files: `chat-rendered-links` 43 s, `results-root` 43 s, `chat-feature-flow` 43 s, `report` 40 s, `manual-private-transfer` 26 s, `private-framework-report` 24 s, `chat-catalog-fingerprint` 24 s, `hire-delegate-flow` 23 s, `source-sdk-evidence` 19 s, `unit-launcher` 16 s. No single file is near the 7-minute target, so there is no longest-file floor. - Slowest tests: the 256 MiB archive-cap test 23 s, the reload-once reply wait 20 s (plus two at 10 s), the documented pnpm validation 16 s, the retained name-only identity replay 14 s, the selected-case fingerprint replay 13 s. ### Changes, one at a time, with what each one bought 1. **Compile cache.** `run-unit.mjs` sets `NODE_COMPILE_CACHE` to `<tmpdir>/paperclip-runner-e2e-compile-cache` unless it is already set. Vitest workers and the Node processes that tests start reuse V8 compiled code. Node validates each entry against the source. `@playwright/test` evaluation went from 158 s to 107–124 s, `acpx` from 39 s to 26–27 s, and `zod` from 17 s to 9–10 s. Vitest went from 478 s to 441 s on a comparable run (this run also included change 2). 2. **Reuse import scans in the chat catalog fingerprint.** `fingerprintChatDependencyClosure` ran one esbuild parse for each of about 2,300 files on every call. The parse result is now cached by file path and content SHA-256 for the process. The fingerprint inputs and digest are unchanged. `chat-feature-flow` went from 43 s to 12–14 s, `chat-catalog-fingerprint` from 24 s to 11–13 s, and `chat-campaign-identity` from 13 s to 8 s. 3. **Hash each suite definition once.** `buildRunnerMatrix` computed `suiteDefinitionHash(suite)` for every execution, thousands of times, every time a test file loaded the catalog. It now computes it once per suite. The values are the same. `catalog.ts` evaluation went from 51 s to 9.6 s. Vitest went from 441 s to 389 s. 4. **Fake timers for the reload-once reply waits.** These three tests in `chat-rendered-links.test.ts` waited out the real 10 s render floor twice. They now advance fake time with `vi.runAllTimersAsync()`. The assertions on results, reload counts and attempt phases are unchanged. The file went from 43 s to 2.8 s. This freed a worker slot; on its own it did not shorten the CPU-bound run. 5. **Derive each aggregator identity once.** `connection-routing.ts` called `aggregatorAppIdentity` (two regex replaces) for every index entry for each of the 133 Arcade apps while it loaded, in 317 test files and in the server. The identity of each entry is now computed once. I compared the resulting `AGGREGATOR_SUPPORT_INDEX` (1,579 entries) and `searchAggregatorServices` results with master: they are identical. The SDK public input pins are refreshed for this change. 6. **Not shipped: four workers.** With the CPU work above removed, four workers cut Vitest from 379 s to 341 s (job 418 s). In the next run, `chat-installed-lane.test.ts` failed: the descendant of a timed-out preparation child was still visible to `process.kill(pid, 0)` right after cleanup. This is the process-tree-under-load risk, so the suite keeps the Vitest default of 3 workers. Docs: `tests/runner-e2e/README.md` describes the compile cache, how to turn it off, why the worker count stays at the default, and the cost of work at module load. ### Result | Run | Job | Test step | Vitest wall | Effective concurrency | | --- | --- | --- | --- | --- | | Baseline median, `m7i-flex` (17 runs) | 526 s | 465 s | 462 s | 2.76 | | Baseline median, `m7i` (3 runs) | 549 s | 476 s | 473 s | – | | Baseline median, `m6a` (6 runs) | 604 s | 535 s | 531 s | – | | This PR, `m7i-flex` (job 132007703795) | 440 s | 383 s | 379 s | 2.79 | | This PR, `m7i` (job 132027384596) | 474 s | 424 s | 420 s | 2.78 | | This PR, `m6a` (job 132033745288) | 526 s | 471 s | 466 s | 2.79 | | This PR, `m5a` (job 132040621237; dependency cache miss) | 880 s | 768 s | 761 s | – | The last three rows are the final configuration. The `m7i-flex` row is the same code before a README-only change. Module evaluation fell from 77,699 to 63,135 evaluations; part of that drop came from master changes merged into this branch, so I attribute time only to the per-change measurements above. ## Verification - CI on this PR is the experiment. Every push ran the real job on the real runner. The table above lists each run. - Local correctness checks, one process at a time: `chat-rendered-links.test.ts` (58 passed), the six chat fingerprint test files (37 passed), `catalog.test.ts` (92 passed), `packages/shared` `connection-routing.test.ts` and `connection-search.test.ts` (21 passed), and `node tests/runner-e2e/run-unit.mjs connection-auth.test.ts` (launcher path, 386 node:test passed). - Equivalence check for the connection routing change: the index and search results are byte-identical to master. - Flakiness: 10 runs of this branch with 3 workers (including the three final runs on `m7i`, `m6a` and the 2x slower `m5a`) passed with 0 failed tests and 0 timeouts. No port conflicts, OOM kills or `ps` timeouts appear in the logs. The suite has no retries. The 4-worker setting failed once in 3 runs, so it is not used. ## Risks - Low risk. No test is skipped or removed. No assertion is weakened. - The worker count is unchanged, so the load on process, port and temp-directory tests is lower than before (less CPU work per file). - The compile cache directory lives in the OS temp directory. Node rejects a stale or partial entry and compiles again. `NODE_DISABLE_COMPILE_CACHE=1` turns it off. - The connection routing change runs in the server too. It produces the same index, and it makes loading the module cheaper. ### What sets the remaining floor The job is about 45–55 s of checkout and dependency restore plus the Vitest run. The Vitest run is CPU-bound on 4 vCPU (2 physical cores). A fourth worker gives about 40 s more but fails a process-tree cleanup test under load (`chat-installed-lane.test.ts`, which checks the descendant immediately after cleanup). The largest remaining cost is per-file isolation: every file loads `@playwright/test` (about 0.3 s CPU each, 390 files), `zod` schemas from `packages/shared`, and `acpx` again. Vitest estimates that `isolate: false` would save about 140 s now. I did not use it: 116 files use `vi.mock`, and module state shared between files would make failures depend on which files share a worker. If more headroom is needed, the next step is a split across two machines (out of scope here because it changes `.github/`). Two duration-balanced halves (`vitest --shard=1/2` and `--shard=2/2`, about 190–235 s of Vitest each on `m7i-flex`/`m6a`) would put each job at about 4–5 minutes on every instance type except `m5a`. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Claude Code agent with tool use (shell, GitHub CLI) and extended reasoning. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable (one test moved to fake timers; behavior changes are equivalence-checked) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
263 lines
23 KiB
TypeScript
263 lines
23 KiB
TypeScript
// @vitest-environment jsdom
|
|
import { readFileSync } from "node:fs";
|
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
|
import { APP_ROUTE_ROOTS, collectConversationLinks, collectConversationLinksWithDiagnostics, conversationLinkBindings, unservedLinkReason, mislabeledIssueLinks, readConversationDomLinks, readLatestConversationReplyRender, waitForLatestConversationReplyRender, waitForSavedTurnReplyRender, waitForSavedTurnReplyRenderReloadingOnce, type ConversationLinkInput } from "./chat-rendered-links.js";
|
|
|
|
const card = '<div data-thread-item-kind="project_created" data-thread-anchor="project-created:project"><article aria-label="Project created: Launch"><div><p>Project created</p><a href="/RUN/projects/project">Launch</a></div></article></div>';
|
|
const thread = (content = card) => `<div data-testid="task-chat-thread"><div data-testid="task-chat-history-content"><div data-testid="task-chat-scroller"><div id="comment-human" data-thread-item-kind="human">Create a project called Launch for the new website.</div>${content}<div id="comment-reply" data-thread-item-kind="agent">Created the Launch project. No repositories are connected yet.</div></div></div></div>`;
|
|
function input(): ConversationLinkInput {
|
|
return { origin: "https://fixture.test", companyId: "company", companyPrefix: "RUN", sourceId: "source", leadId: "ceo",
|
|
comments: [{ id: "human", issueId: "source", companyId: "company" }, { id: "reply", issueId: "source", companyId: "company", authorAgentId: "ceo", createdByRunId: "run" }],
|
|
runs: [{ id: "run", companyId: "company", agentId: "ceo", status: "succeeded", contextSnapshot: { issueId: "source" } }],
|
|
projects: [{ id: "project", companyId: "company", name: "Launch" }],
|
|
activity: [{ id: "created", companyId: "company", action: "project.created", entityType: "project", entityId: "project", actorType: "agent", actorId: "ceo", agentId: "ceo", runId: "run" }] };
|
|
}
|
|
const collect = (value = input()) => readConversationDomLinks(conversationLinkBindings(value));
|
|
beforeEach(() => { document.body.innerHTML = thread(); vi.spyOn(Element.prototype, "getClientRects").mockReturnValue([{ width: 20, height: 20 }] as unknown as DOMRectList); });
|
|
afterEach(() => vi.restoreAllMocks());
|
|
it("collects the real supported sibling project-card structure when reply-only collection has no link", () => {
|
|
expect(document.querySelectorAll('#comment-reply a[href]')).toHaveLength(0);
|
|
expect(collect()).toEqual([{ threadTestId: "task-chat-thread", href: "/RUN/projects/project", source: "project-created-card", conversationId: "source", anchorCommentId: "human", resourceId: "project", cardLabel: "Project created: Launch", createdByRunId: "run" }]);
|
|
});
|
|
it("keeps the original creation card usable on an idempotent retry without requiring another card", () => {
|
|
const value = input(); value.comments.push({ id: "retry", issueId: "source", companyId: "company", authorAgentId: "ceo", createdByRunId: "retry-run" });
|
|
value.runs.push({ id: "retry-run", companyId: "company", agentId: "ceo", status: "succeeded", contextSnapshot: { issueId: "source" } });
|
|
document.querySelector('[data-testid="task-chat-scroller"]')!.insertAdjacentHTML('beforeend', '<div id="comment-retry">The project already exists.</div>');
|
|
expect(collect(value).map(row => row.href)).toEqual(["/RUN/projects/project"]);
|
|
});
|
|
it("retains visible ordinary reply links including task, goal and artifact links", () => {
|
|
document.querySelector('#comment-reply')!.innerHTML = '<a href="/RUN/issues/task">Task</a><a href="/RUN/goals/goal">Goal</a><a href="/api/attachments/file/content">Artifact</a><a href="https://github.com/owner/repository/issues/12">External issue</a>';
|
|
expect(collect().filter(row => row.source === 'reply').map(row => row.href)).toEqual(['/RUN/issues/task','/RUN/goals/goal','/api/attachments/file/content','https://github.com/owner/repository/issues/12']);
|
|
});
|
|
it.each([
|
|
['wrong saved company', (v:ConversationLinkInput) => { v.projects[0]!.companyId='other'; }],
|
|
['wrong saved resource', (v:ConversationLinkInput) => { v.projects[0]!.id='other'; }],
|
|
['wrong saved name', (v:ConversationLinkInput) => { v.projects[0]!.name='Other'; }],
|
|
['missing creation audit', (v:ConversationLinkInput) => { v.activity=[]; }],
|
|
['wrong audit company', (v:ConversationLinkInput) => { v.activity[0]!.companyId='other'; }],
|
|
['wrong audit actor type', (v:ConversationLinkInput) => { v.activity[0]!.actorType='user'; }],
|
|
['wrong audit actor', (v:ConversationLinkInput) => { v.activity[0]!.actorId='other'; }],
|
|
['wrong audit agent', (v:ConversationLinkInput) => { v.activity[0]!.agentId='other'; }],
|
|
['wrong audit entity type', (v:ConversationLinkInput) => { v.activity[0]!.entityType='issue'; }],
|
|
['wrong audit resource', (v:ConversationLinkInput) => { v.activity[0]!.entityId='other'; }],
|
|
['unrelated historical run', (v:ConversationLinkInput) => { v.activity[0]!.runId='old'; }],
|
|
['run in another company', (v:ConversationLinkInput) => { v.runs[0]!.companyId='other'; }],
|
|
['run for another conversation', (v:ConversationLinkInput) => { v.runs[0]!.contextSnapshot={issueId:'other'}; }],
|
|
['wrong reply author', (v:ConversationLinkInput) => { v.comments[1]!.authorAgentId='other'; }],
|
|
['reply from another conversation', (v:ConversationLinkInput) => { v.comments[1]!.issueId='other'; }],
|
|
['comments from another company', (v:ConversationLinkInput) => { v.comments.forEach(row=>row.companyId='other'); }],
|
|
] as const)('rejects %s instead of treating any matching project link as evidence', (_label, mutate) => { const value=input(); mutate(value); expect(collect(value)).toEqual([]); });
|
|
it.each([
|
|
['sidebar only', () => { document.body.innerHTML=thread('')+`<aside>${card}</aside>`; }],
|
|
['unrelated conversation', () => { document.body.innerHTML=thread('')+thread().replaceAll('comment-human','comment-other-human').replaceAll('comment-reply','comment-other-reply'); }],
|
|
['hidden sibling', () => { document.querySelector('article')!.setAttribute('hidden',''); }],
|
|
['hidden ancestor', () => { document.querySelector('[data-testid="task-chat-history-content"]')!.setAttribute('style','visibility:hidden'); }],
|
|
['inert ancestor', () => { document.querySelector('[data-testid="task-chat-history-content"]')!.setAttribute('inert',''); }],
|
|
['aria-hidden ancestor', () => { document.querySelector('[data-testid="task-chat-history-content"]')!.setAttribute('aria-hidden','true'); }],
|
|
['display:none', () => { document.querySelector('article')!.setAttribute('style','display:none'); }],
|
|
['transparent card', () => { document.querySelector('article')!.setAttribute('style','opacity:0'); }],
|
|
['missing layout box', () => { vi.mocked(Element.prototype.getClientRects).mockReturnValue([] as unknown as DOMRectList); }],
|
|
['missing supported card wrapper', () => { document.querySelector('[data-thread-item-kind="project_created"]')!.removeAttribute('data-thread-item-kind'); }],
|
|
['wrong item resource', () => { document.querySelector('[data-thread-item-kind="project_created"]')!.setAttribute('data-thread-anchor','project-created:other'); }],
|
|
['wrong card kind', () => { document.querySelector('article')!.setAttribute('aria-label','Goal created: Launch'); }],
|
|
['wrong card resource', () => { document.querySelector('article a')!.setAttribute('href','/RUN/projects/other'); }],
|
|
['wrong company route', () => { document.querySelector('article a')!.setAttribute('href','/OTHER/projects/project'); }],
|
|
['other origin', () => { document.querySelector('article a')!.setAttribute('href','https://other.test/RUN/projects/project'); }],
|
|
['wrong resource kind', () => { document.querySelector('article a')!.setAttribute('href','/RUN/goals/project'); }],
|
|
['extra destination path', () => { document.querySelector('article a')!.setAttribute('href','/RUN/projects/project/other'); }],
|
|
['duplicate visible source conversation', () => { document.body.insertAdjacentHTML('beforeend',thread()); }],
|
|
] as const)('rejects %s in actual DOM scope', (_label, mutate) => { mutate(); expect(collect()).toEqual([]); });
|
|
it("excludes hidden, wrong-company and unrelated inline anchors while keeping the current reply", () => {
|
|
document.querySelector('#comment-reply')!.innerHTML='<a href="/RUN/issues/task">Task</a><a hidden href="/RUN/issues/hidden">Hidden</a><a href="/OTHER/issues/task">Other company</a>';
|
|
document.body.insertAdjacentHTML('beforeend','<aside><a href="/RUN/issues/sidebar">Sidebar</a></aside>');
|
|
expect(collect().filter(row=>row.source==='reply').map(row=>row.href)).toEqual(['/RUN/issues/task']);
|
|
});
|
|
|
|
it("treats a link inside an authenticated reply as an ordinary inline link, not a sibling creation receipt", () => {
|
|
document.querySelector('#comment-reply')!.append(document.querySelector('article')!);
|
|
expect(collect().map(row=>row.source)).toEqual(['reply']);
|
|
});
|
|
|
|
it("recognizes the minimized authentic retained DOM and server identities without changing the original measurement", () => {
|
|
const fixture = JSON.parse(readFileSync(`${import.meta.dirname}/chat-rendered-links.retained-project.json`, "utf8"));
|
|
document.body.innerHTML = fixture.html;
|
|
const reply = fixture.input.comments.find((row: { authorAgentId?: string }) => row.authorAgentId);
|
|
expect(document.getElementById(`comment-${reply.id}`)!.querySelectorAll("a[href]")).toHaveLength(0);
|
|
expect(collect(fixture.input).map(row => row.href)).toEqual([fixture.expectedHref]);
|
|
});
|
|
|
|
it("retains ordinary inline links in the supported classic issue-chat root", () => {
|
|
document.body.innerHTML=thread('').replace('task-chat-thread','thread-root');
|
|
document.querySelector('#comment-reply')!.innerHTML='<a href="/RUN/issues/task">Task</a>';
|
|
expect(collect().map(row=>[row.threadTestId,row.source,row.href])).toEqual([['thread-root','reply','/RUN/issues/task']]);
|
|
});
|
|
|
|
// Full cohort on 9296c74e8d (choose-primary, case-history, set-budget): the API saw the saved reply ~2s before
|
|
// the open conversation rendered it, so links read in between were empty although the reply linked correctly.
|
|
describe("waits for the latest saved reply to render before reading links", () => {
|
|
const page = () => ({ evaluate: async <T, A>(read: (arg: A) => T, arg: A) => read(arg) }) as unknown as Parameters<typeof collectConversationLinks>[0];
|
|
it("reads the reply's links once the reply renders after the API saw it", async () => {
|
|
document.querySelector('#comment-reply')!.remove();
|
|
setTimeout(() => document.querySelector('[data-testid="task-chat-scroller"]')!.insertAdjacentHTML("beforeend", '<div id="comment-reply"><a href="/RUN/issues/task">Task</a></div>'), 600);
|
|
const value = input(); value.projects = []; value.activity = [];
|
|
expect((await collectConversationLinks(page(), value)).map(row => row.href)).toEqual(["/RUN/issues/task"]);
|
|
});
|
|
it("does not wait when the conversation is not open, and waits at most once for a reply that never renders", async () => {
|
|
const bindings = conversationLinkBindings(input());
|
|
document.body.innerHTML = "<main>Other page</main>";
|
|
expect(readLatestConversationReplyRender({ anchors: bindings.anchors, latest: "reply" })).toBe("closed");
|
|
document.body.innerHTML = thread(); document.querySelector('#comment-reply')!.remove();
|
|
const p = page();
|
|
expect(await waitForLatestConversationReplyRender(p, bindings, 300, 50)).toBe("pending");
|
|
const started = Date.now(); expect(await waitForLatestConversationReplyRender(p, bindings, 5_000, 50)).toBe("pending"); expect(Date.now() - started).toBeLessThan(1_000);
|
|
});
|
|
// Cohort 3 on dfd2a6efca (access.other-company, teams.install-team and six more): the final evidence read
|
|
// ran on a freshly loaded conversation whose wait had expired in an earlier document, so it read instantly
|
|
// and saved no links although the reply rendered them. A new document gets its own bounded wait.
|
|
it("waits again in a newly loaded document after an earlier document's wait expired", async () => {
|
|
const bindings = conversationLinkBindings(input());
|
|
document.body.innerHTML = thread(); document.querySelector('#comment-reply')!.remove();
|
|
const p = page();
|
|
expect(await waitForLatestConversationReplyRender(p, bindings, 200, 50)).toBe("pending");
|
|
delete (window as typeof window & { __paperclipConversationLinkDocument?: string }).__paperclipConversationLinkDocument;
|
|
setTimeout(() => document.querySelector('[data-testid="task-chat-scroller"]')!.insertAdjacentHTML("beforeend", '<div id="comment-reply"><a href="/RUN/issues/task">Task</a></div>'), 300);
|
|
expect(await waitForLatestConversationReplyRender(p, bindings, 5_000, 50)).toBe("rendered");
|
|
});
|
|
});
|
|
|
|
// Full cohort 3 (fast-responses, choose-primary, set-budget): the run was recorded finished 0.2-1.3s before its
|
|
// reply comment was saved, so a link read right after the run finished saw no reply at all.
|
|
describe("waits for a finished run's reply to be saved, then rendered, before its links are read", () => {
|
|
const page = () => ({ evaluate: async <T, A>(read: (arg: A) => T, arg: A) => read(arg) }) as unknown as Parameters<typeof collectConversationLinks>[0];
|
|
it("waits for the reply saved after the run finished and for its render", async () => {
|
|
const saved = input().comments, before = saved.filter(row => row.id !== "reply");
|
|
document.querySelector('#comment-reply')!.remove();
|
|
let reads = 0; const comments = async () => (reads++ < 3 ? before : saved);
|
|
setTimeout(() => document.querySelector('[data-testid="task-chat-scroller"]')!.insertAdjacentHTML("beforeend", '<div id="comment-reply"><a href="/RUN/issues/task">Task</a></div>'), 400);
|
|
expect(await waitForSavedTurnReplyRender(page(), { comments, leadId: "ceo", runIds: ["run"] }, 3_000, 50)).toBe("rendered");
|
|
expect(reads).toBeGreaterThan(3);
|
|
});
|
|
it("reports a run that saved no reply in time, without waiting past its bound", async () => {
|
|
const before = input().comments.filter(row => row.id !== "reply");
|
|
const started = Date.now();
|
|
expect(await waitForSavedTurnReplyRender(page(), { comments: async () => before, leadId: "ceo", runIds: ["run"] }, 300, 50)).toBe("unsaved");
|
|
expect(Date.now() - started).toBeLessThan(1_500);
|
|
});
|
|
});
|
|
|
|
// Full cohort 5 on 98c18784c6 (cases.case-history, tasks.ambiguous-task): the final reply was saved by run
|
|
// finalization after the run's terminal status and logged no comment_added, so the open conversation never
|
|
// fetched it; the thread kept only the person's message. One reload reads the saved conversation.
|
|
describe("reloads once when a saved reply never renders in the open conversation", () => {
|
|
beforeEach(() => { vi.useFakeTimers(); });
|
|
afterEach(() => { vi.useRealTimers(); });
|
|
const settle = async <T>(waiting: Promise<T>) => { await vi.runAllTimersAsync(); return waiting; };
|
|
const reloading = (onReload: () => void) => {
|
|
let reloads = 0;
|
|
const page = { evaluate: async <T, A>(read: (arg: A) => T, arg: A) => read(arg),
|
|
reload: async () => { reloads++; delete (window as typeof window & { __paperclipConversationLinkDocument?: string }).__paperclipConversationLinkDocument; onReload(); } };
|
|
return { page: page as unknown as Parameters<typeof waitForSavedTurnReplyRenderReloadingOnce>[0], reloads: () => reloads };
|
|
};
|
|
it("renders after the reload and records both waits", async () => {
|
|
const comments = async () => input().comments;
|
|
document.querySelector('#comment-reply')!.remove();
|
|
const p = reloading(() => document.querySelector('[data-testid="task-chat-scroller"]')!.insertAdjacentHTML("beforeend", '<div id="comment-reply"><a href="/RUN/issues/task">Task</a></div>'));
|
|
const wait = await settle(waitForSavedTurnReplyRenderReloadingOnce(p.page, { comments, leadId: "ceo", runIds: ["run"] }, 300, 50));
|
|
expect(wait.result).toBe("rendered"); expect(p.reloads()).toBe(1);
|
|
expect(wait.attempts.map(row => [row.phase, row.result])).toEqual([["open-document", "pending"], ["after-reload", "rendered"]]);
|
|
});
|
|
it("waits for the reloaded app to mount the conversation before the second wait", async () => {
|
|
const comments = async () => input().comments;
|
|
document.querySelector('#comment-reply')!.remove();
|
|
const saved = document.body.innerHTML;
|
|
const p = reloading(() => {
|
|
document.body.innerHTML = "<main>Loading</main>";
|
|
setTimeout(() => { document.body.innerHTML = saved; document.querySelector('[data-testid="task-chat-scroller"]')!.insertAdjacentHTML("beforeend", '<div id="comment-reply"><a href="/RUN/issues/task">Task</a></div>'); }, 300);
|
|
});
|
|
const wait = await settle(waitForSavedTurnReplyRenderReloadingOnce(p.page, { comments, leadId: "ceo", runIds: ["run"] }, 1_000, 50));
|
|
expect(wait.attempts.map(row => [row.phase, row.result])).toEqual([["open-document", "pending"], ["after-reload", "rendered"]]);
|
|
});
|
|
it("does not reload a reply that rendered live, and still reports a reply that never renders", async () => {
|
|
const comments = async () => input().comments;
|
|
const live = reloading(() => undefined);
|
|
expect((await settle(waitForSavedTurnReplyRenderReloadingOnce(live.page, { comments, leadId: "ceo", runIds: ["run"] }, 300, 50))).result).toBe("rendered");
|
|
expect(live.reloads()).toBe(0);
|
|
document.querySelector('#comment-reply')!.remove();
|
|
const never = reloading(() => undefined);
|
|
const wait = await settle(waitForSavedTurnReplyRenderReloadingOnce(never.page, { comments, leadId: "ceo", runIds: ["run"] }, 200, 50));
|
|
expect(wait.result).toBe("pending"); expect(never.reloads()).toBe(1); expect(wait.attempts).toHaveLength(2);
|
|
});
|
|
});
|
|
|
|
describe("task links whose visible identifier names another task", () => {
|
|
it("accepts links whose label names the task they open, or names no task", () => {
|
|
expect(mislabeledIssueLinks("Open [COM-1](/COM/issues/COM-1) in [Launch](/COM/projects/launch).")).toEqual([]);
|
|
expect(mislabeledIssueLinks("The [open COM-1 task](/COM/issues/com-1#document-plan) and [this one](/COM/issues/COM-2).")).toEqual([]);
|
|
expect(mislabeledIssueLinks("[COM-1 Launch open website](/COM/issues/task-uuid)", id => id === "task-uuid" ? "COM-1" : undefined)).toEqual([]);
|
|
expect(mislabeledIssueLinks("[SHA-256 notes for COM-1](/COM/issues/COM-1)")).toEqual([]);
|
|
});
|
|
it("rejects a link whose label names a different task than it opens", () => {
|
|
expect(mislabeledIssueLinks("Open tasks: [COM-2 Launch open website](/COM/issues/COM-1)... correction:")).toEqual(["[COM-2 Launch open website](/COM/issues/COM-1)"]);
|
|
expect(mislabeledIssueLinks("[COM-2](/COM/issues/task-uuid)", id => id === "task-uuid" ? "COM-1" : undefined)).toHaveLength(1);
|
|
});
|
|
});
|
|
|
|
// Full cohort 4 (case-history, goals.explain-purpose, preview-service): a phase capture that saved no links could
|
|
// not say whether the thread was closed, showed another conversation, or rendered the reply without anchors.
|
|
describe("records thread and anchor diagnostics when a saved reply yields no rendered links", () => {
|
|
const page = () => ({ evaluate: async <T, A>(read: (arg: A) => T, arg: A) => read(arg) }) as unknown as Parameters<typeof collectConversationLinks>[0];
|
|
it("reports the rendered thread, its saved comment anchors and the reply's anchor count", async () => {
|
|
const value = input(); value.projects = []; value.activity = [];
|
|
vi.spyOn(console, "info").mockImplementation(() => undefined);
|
|
const result = await collectConversationLinksWithDiagnostics(page(), value);
|
|
expect(result.links).toEqual([]);
|
|
expect(result.diagnostics).toEqual({ render: "rendered", expectedReplyIds: ["reply"], savedCommentCount: 2,
|
|
threads: [{ testId: "task-chat-thread", visible: true, renderedCommentIds: ["human", "reply"], foreignCommentIds: [], replyAnchorCounts: { reply: 0 } }] });
|
|
});
|
|
it("adds no diagnostics when links were read", async () => {
|
|
document.querySelector('#comment-reply')!.innerHTML = '<a href="/RUN/issues/task">Task</a>';
|
|
const value = input(); value.projects = []; value.activity = [];
|
|
const result = await collectConversationLinksWithDiagnostics(page(), value);
|
|
expect(result.links.map(row => row.href)).toEqual(["/RUN/issues/task"]); expect(result.diagnostics).toBeUndefined();
|
|
});
|
|
});
|
|
|
|
// Full cohort 9 (documents-artifacts.produce-download, agents.edit-instructions): a reply's download link was written as
|
|
// /COM/api/attachments/.../content?download=1, which the server does not serve (its API is only at /api), and a reply's
|
|
// Morgan link was saved as //COM/agents/..., which the board repairs but notifications and chat channels deliver as a
|
|
// link to a host named COM. Both counted as valid links.
|
|
describe("only links that open a served app page or API route count", () => {
|
|
const origin = "https://fixture.test";
|
|
it("accepts company pages, unprefixed board pages, global pages and served API downloads", () => {
|
|
for (const href of ["/RUN/issues/RUN-1", "/run/agents/morgan/overview", "/RUN/company/settings/connections", "/RUN/routines/routine", "/RUN",
|
|
"/issues/RUN-1", "/instance/settings", "/api/attachments/file/content?download=1", "/api/attachments/file/content", "https://github.com/owner/repository"])
|
|
expect(unservedLinkReason(href, origin), href).toBeUndefined();
|
|
});
|
|
it("rejects a company-prefixed API path, a doubled prefix and other paths the app does not route", () => {
|
|
for (const href of ["/RUN/api/attachments/file/content?download=1", "/RUN/RUN/issues/RUN-1", "/RUN/attachments/file/content", "/attachments/file/content"])
|
|
expect(unservedLinkReason(href, origin), href).toBe("not-a-served-route");
|
|
});
|
|
it("rejects a protocol-relative company link, rendered raw or repaired by the board, but not the same page saved correctly", () => {
|
|
expect(unservedLinkReason("//RUN/agents/morgan", origin)).toBe("protocol-relative");
|
|
expect(unservedLinkReason("/RUN/agents/morgan", origin, ["//RUN/agents/morgan"])).toBe("protocol-relative");
|
|
expect(unservedLinkReason("/RUN/agents/morgan", origin, ["/RUN/agents/morgan"])).toBeUndefined();
|
|
expect(unservedLinkReason("/RUN/agents/morgan", origin, ["//RUN/agents/morgan", "/RUN/agents/morgan"])).toBeUndefined();
|
|
});
|
|
it("lists exactly the first path segments that the board router declares", () => {
|
|
const app = readFileSync(`${import.meta.dirname}/../../ui/src/App.tsx`, "utf8");
|
|
const roots = new Set([...app.matchAll(/path="\/?([^"/]+)/g)].map(match => match[1]!).filter(root => !root.startsWith(":") && root !== "*"));
|
|
expect([...APP_ROUTE_ROOTS].sort()).toEqual([...roots].sort());
|
|
expect(APP_ROUTE_ROOTS).not.toContain("api");
|
|
});
|
|
it("drops unserved reply links from collection, keeps them as rejected evidence and keeps served ones", async () => {
|
|
const page = { evaluate: async <T, A>(read: (arg: A) => T, arg: A) => read(arg) } as unknown as Parameters<typeof collectConversationLinks>[0];
|
|
vi.spyOn(console, "info").mockImplementation(() => undefined);
|
|
document.querySelector('#comment-reply')!.innerHTML = '<a href="/api/attachments/file/content">website.html</a><a href="/RUN/api/attachments/file/content?download=1">Download</a><a href="/RUN/agents/morgan">Morgan</a>';
|
|
const value = input(); value.projects = []; value.activity = [];
|
|
value.comments[1]!.body = "Prepared [website.html](/api/attachments/file/content). [Download](/RUN/api/attachments/file/content?download=1) from [Morgan](//RUN/agents/morgan).";
|
|
const result = await collectConversationLinksWithDiagnostics(page, value);
|
|
expect(result.links.map(row => row.href)).toEqual(["/api/attachments/file/content"]);
|
|
expect(result.rejected?.map(row => [row.href, row.reason])).toEqual([["/RUN/api/attachments/file/content?download=1", "not-a-served-route"], ["/RUN/agents/morgan", "protocol-relative"]]);
|
|
});
|
|
});
|