mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-12 06:10:44 +01:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Runner, plugin, and Playwright tests verify how those agents run, connect, and appear in the board > - Most package tsconfigs exclude tests, and Vitest strips types, so a type error in those tests fails nothing > - The tests drifted to 803 real type errors, and some of those errors hid assertions that checked nothing > - paperclipai/paperclip#18739 and paperclipai/paperclip#18738 check the server test tree and four test projects, but not these files > - This pull request adds a required check for these test files, fixes every error, and fails CI if a test file is in no program > - The benefit is that a type error in any test file outside `server/src` now fails CI ## Linked Issues or Issue Description Refs #18739 Refs #18738 **What happened?** 517 test files were in no program that required CI checks. A type error in those files did not fail CI. **Expected behavior** CI fails on a type error in any test file. **Steps to reproduce** Add `export const value: number = "text";` to a test file under `packages/paperclip-runner/src` on master. CI stays green, because `tsconfig.json` excludes test files and Vitest strips types. **Measured gap** Each row is a program with its tests included, on one thread. File and error counts are for head `7d4b5e907a`. Memory peaks were measured before the two latest merges of master. | Group | Test files | Real errors | Peak | | --- | ---: | ---: | ---: | | `paperclip-runner` Node source | 290 | 644 | 1.09 GiB | | `paperclip-runner` surfaces | 9 | 12 | 0.42 GiB | | `paperclip-runner` devtools and examples | 14 | 0 | 0.68 GiB | | `sandbox-providers/kubernetes` | 21 | 65 | 0.44 GiB | | 8 other sandbox providers and `bridge-template` | 32 | 4 | 0.34 GiB | | `plugins/sdk` | 22 | 15 | 0.41 GiB | | `plugin-workspace-diff` | 5 | 0 | 0.37 GiB | | `plugin-orchestration-smoke-example` | 1 | 1 | 0.15 GiB | | `kv-demo-mcp-server` | 3 | 0 | 0.13 GiB | | `packages/db` scripts | 1 | 0 | 1.01 GiB | | 8 Playwright suites under `tests/` | 52 | 25 | 1.72 GiB | | `tests/runner-acceptance` | 3 | 0 | 0.58 GiB | | `tests/e2e`, through layer declarations | 63 | 37 | at most 1.59 GiB | | `ui` `AttentionInteractionResolver.test.tsx` | 1 | 0 | n/a | | **Total** | **517** | **803** | | 27 of the 803 errors came from master while this pull request was open. Each merge of master brought new test code that no check had read. Errors that were config errors, not real errors: - One TS2688 for each provider, because the root install does not install provider dependencies. - 9 in the Runner Node program (TS7016 and TS6059). - 2 in `tsconfig.browser.json`. TypeScript 7 removed `baseUrl`, so that config was invalid. - 213 in the Playwright suites (`@/*` aliases and Vite types). - About 60 in `tests/e2e` (import extensions and aliases). `packages/plugins/create-paperclip-plugin` has no gap. Its tsconfig excludes its test, but `cli/tsconfig.json` reaches that test through an import. `server/test-baselines` also has no gap, because `tests/lifecycle-baseline` reads it. ## What Changed - Added `scripts/typecheck-test-programs.mjs` and its tests. The `Typecheck + Release Registry` job runs it. It checks each program in `PROGRAMS` on one thread. - The script fails if a tracked `*.test.ts(x)` or `*.spec.ts(x)` file outside `server/src` is in no program. The compiler file list decides this, because `include` drops `foo.test.tsx` when `foo.test.ts` exists. - Added a `tsconfig.test.json` to each uncovered package, and `tests/tsconfig.playwright.json`. - Added `tests/e2e` to `PROJECTS` in `scripts/typecheck-test-projects.mjs`, with a `tsconfig.json` and a `tsconfig.layers.json`. - Added `scripts/typecheck-sandbox-providers`. It is a small workspace that installs provider SDKs from one lockfile. I did not add a lockfile inside each provider, because `scripts/build-standalone-public-packages.mjs` uses a frozen install when one exists, and that changes what a release resolves. - Corrected `packages/paperclip-runner/tsconfig.browser.json` for TypeScript 7. - Set `rootDir` to `..` in `packages/paperclip-runner/tsconfig.test.json`, because one Runner test imports from `packages/adapter-utils/src`. The program does not emit files, so this changes no build output. - Moved the Kubernetes provider from `vitest ^3.2.4` to `^4.1.8`, the version the other providers use. Vitest 3 requires `tinypool ^1.1.1`, which has two critical advisories (GHSA-85c8-ppgw-ccpr and GHSA-5gmw-xhrv-c9v3). The first patched version is 2.1.2. Its tests have the same result on both versions: 189 passed, and the same 12 `file-sync` tests failed. - Moved the Kubernetes provider from `zod ^3.24.2` to `^4.6.6`, and gave its two `z.record` calls a key schema. This is the only production change, and it stays in this pull request. See "Production change for review". - Renamed `AttentionInteractionResolver.test.ts` to `AttentionInteractionResolver.replace.test.ts`, so TypeScript reads the `.test.tsx` beside it. - Fixed all 803 errors, with changes to test files and test support only. The fix for `contracts/native-execution.test.ts` is the same as the fix in paperclipai/paperclip#18739, so those two changes merge cleanly. - Updated `doc/DEVELOPING.md`. This pull request does not edit `server/scripts/typecheck-layers.mjs`. paperclipai/paperclip#18739 and paperclipai/paperclip#18738 have both merged, and this branch includes them. ### Production change for review The new lockfile made dependency review flag `zod@3.25.76` as a high-severity advisory (GHSA-mrhx-c7xc-m3w5). The first patched version is 4.6.6, so 3.x has no fix. The published Kubernetes plugin uses zod to validate operator config. The zod 3 source does not compile against zod 4, so a typecheck-only override cannot fix this. The existing tests do not prove this change, because they also passed with the invalid one-argument `z.record`. So I parsed 45 inputs with the same schema on zod 3.25.76 and zod 4.6.7: | Result | Inputs | Detail | | --- | ---: | --- | | Identical | 23 | Includes defaults, valid config, and a `__proto__` key | | Same outcome, new error wording | 21 | For example, `Invalid url` is now `Invalid URL`. `validateConfig` shows this text to operators. | | Different outcome | 1 | `jobTtlSecondsAfterFinished: 2 ** 60` was valid and is now rejected. The new limit is `Number.MAX_SAFE_INTEGER`. | I did not test this against a real cluster. The decision is to keep this change in this pull request. Since then, paperclipai/paperclip#17364 has moved the rest of the repository to zod 4.6.x, so the Kubernetes plugin now matches it. ### Behavior change for review `reducedMotion` under `use` is not a Playwright option. It had no effect in 11 configs and 2 `test.use` calls, so those suites ran with `no-preference`. The options are now under `contextOptions`, so reduced motion now takes effect. `app-shell-design.spec.ts` asserts a real transition, so it now asks for `no-preference` itself. The baseline manifest has `snapshotCount: 0`, so this invalidates no published baseline. A frozen trial reference PNG can differ. ### Findings These are product questions. I did not change them, and I did not hide them with casts: 1. **`PrpEvent` and its schema disagree.** The TypeScript type in `src/protocol/replay-contract.ts` requires `normalizedSessionId`. The `required` list in `protocol/schemas/event.schema.json` does not include it. `validatePrpEvent` accepts events that the type forbids. 2. **`requestCheckboxConfirmation` is stricter than its runtime.** It uses the output type, so plugin authors must pass `continuationPolicy` and `allowDeclineReason`, although the schema gives them defaults. `z.input` would fix this. 3. **`RetainedRunnerdMaintenanceInput.backend.kind` is a free string.** The quarantine domain comes from `descriptor.kind`, but the release uses `proof.backend.kind`. If they differ, `completeRetainedNativeSessionCleanup` returns 0 and the quarantine stays. The production caller passes `"runner"` today, so this is not a live bug. 4. **`CodexHarnessSession.close(input?)` is optional, but the contract requires `{ reason }`.** These are tests that verified nothing. I corrected them, and they pass: - `backends/native-instruction-measurement.test.ts` filtered on `placement === "core"`, which is not a valid placement. It measured an empty tool list. It now measures 14 tools. - `drivers/acpx/codex-acpx-driver.test.ts` checked for `usage.updated`, which is not an event type. - The `hostFixture()` helper in `drivers/acpx/runtime-host.test.ts` ignored two overrides, so `expect(prepareSandbox).not.toHaveBeenCalled()` could not fail. - `live/live-session.test.ts` set `agent: "pi"` before a spread that overwrote it. - `native-distribution-integrity.test.ts` passed an AbortSignal where `openCommand` expects `{ signal }`. These are gaps that this pull request does not close: - **The root lockfile still has some old zod versions.** paperclipai/paperclip#17364 moved the direct zod dependencies to 4.6.x. `pnpm-lock.yaml` still resolves `zod@3.25.76`, `4.1.11`, and `4.4.3`, which are below the patched 4.6.6. `packages/paperclip-runner/scripts/pi-distribution/package.json` pins `zod` to `3.25.76`, and some agent SDKs resolve `4.4.3`. Dependency review does not flag them, because this pull request does not add them. - **Dead GitHub coverage.** `chat-adapters-ui-providers.spec.ts` returned early for GitHub, so four later `provider === "github"` branches never ran. I removed them. They included the private key file and paste match, and the reconnect flow. Nothing asserts those behaviors in that spec now. - **Provider tests do not run in CI.** The root `vitest.config.ts` lists only Daytona. The other 8 providers, `bridge-template` included, are typechecked now, but CI does not run their tests. - **One `packages/db` test never runs.** `scripts/clean-poisoned-claude-sessions.test.ts` is outside the `include` of `packages/db/vitest.config.ts`. - **`plugin-orchestration-smoke-example` does not collect.** `@paperclipai/shared` is not linked in its `node_modules`. It passes 3 of 3 with an alias. ## Verification Run these commands: ```sh node scripts/typecheck-test-programs.mjs --low-memory pnpm --filter @paperclipai/server typecheck node scripts/typecheck-test-projects.mjs --low-memory --reuse-declarations node --test scripts/typecheck-test-programs.test.mjs ``` Results on macOS (arm64), Node 26.12. CI uses Node 24, and its `Typecheck + Release Registry` job also passes. At head `7d4b5e907a`, the test programs, `tests/e2e`, and the script tests pass again with 0 errors. The times and peaks below come from the run before the two latest merges of master: | Check | Result | | --- | --- | | Test programs, `--low-memory` | Exit 0, 0 errors, 42 seconds, peak 1.78 GiB | | Server layers, low memory | Exit 0, 0 errors, 38 seconds, peak 3.41 GiB | | Test projects, including `tests/e2e` | Exit 0, 0 errors, 13 seconds, peak 1.59 GiB | | Script tests | 12 passed | | Injected type error in each of 23 places | Each one failed its program with TS2322 | | Coverage check | 0 unchecked files, 10 seconds | I did not run this in a 4 GiB container. The peaks above are `maximum resident set size` for the largest single process. **Runtime check of the fixes.** Types do not prove that a test still works, so I ran the suites before the two latest merges of master: | Suite | Result | | --- | --- | | Runner, full suite | 4,885 passed, 3 failed, 13 skipped | | Sandbox providers, 10 packages | 939 passed, 12 failed, 36 skipped | | Plugin SDK | 200 passed | | `kv-demo-mcp-server` | 12 passed | | Renamed UI test and its sibling | 4 passed | All 15 failures exist without these fixes or come from the local machine: - **3 Cursor Cloud timeouts.** `artifact-delivery.test.ts` has 1 and `cursor-cloud-driver.test.ts` has 2. They time out in a full local run, and all 133 tests in those files with `codex-security-config.test.ts` pass when run alone. They fail in the same way with those two files at their unedited versions. - **12 Kubernetes `file-sync.test.ts` failures.** They fail in the same way before and after the fixes. The first failure is that `/proc/self/fd` does not exist on macOS. After each of the two latest merges of master, I ran only the Runner test files that I changed: | Merge | Result | | --- | --- | | `885c249cb9`, 8 files | 534 passed | | `7d4b5e907a`, 7 files | 672 passed, 16 failed | The 16 failures are in the two `source-reconciliation` test files. They fail in the same way with master's unedited versions of those files. 15 of them fail with `ACPX sidecar exited at initialize`, so I think my local build of the Rust and sidecar binaries is out of date. I did not rebuild them to check. CI builds them from scratch, and all Runner jobs pass in CI at `7d4b5e907a`. **Not run:** - `tests/e2e` and the 8 Playwright suites need a server, so most of them have only compile checks. `playwright test --list` loads 308 tests from the main e2e config. - `packages/plugins/sandbox-providers/kubernetes/test/integration` needs a cluster, so it has only compile checks. - I did not run the full `pnpm test:run` or root `pnpm build`. Required CI runs them. **Reduced motion.** I ran 6 motion-sensitive Storybook specs in system Chrome with a temporary config, once with `reduce` and once with `no-preference`. Both runs had 45 passed and the same 5 failed. So reduced motion did not change those results. The 5 failures do not depend on motion. At least one comes from the browser, because system Chrome prints `oklch(55.6% 0 0)` where the test expects `oklch(0.556 0 0)`. I did not run the other Storybook specs. The `storybook-visual` label runs that workflow. ## Risks - **A new lockfile to maintain.** A change to a provider `package.json` now fails the typecheck job until someone refreshes `scripts/typecheck-sandbox-providers/pnpm-lock.yaml`. The failure message gives the command. - **The job has a new install.** The step downloads provider SDKs, so it depends on the npm registry. It uses `--frozen-lockfile` and `--ignore-scripts`. - **The local install replaces provider `node_modules`.** A contributor with a standalone provider install must reinstall that provider after running the script. - **Reduced motion can change Storybook results.** See "Behavior change for review". That workflow is not part of required CI. - **Kubernetes plugin validation changed.** A release of this plugin ships zod 4. Error wording changes, and very large integers are now rejected. See "Production change for review". - **Master moves fast.** Each merge of master brought new test code with type errors, 27 so far. After this merges, the check reports those errors in the pull request that adds them. ## Model Used - Provider: Anthropic Claude, through Claude Code - Model: Claude Opus 5.5 (`claude-opus-5-5`) - Capabilities: tool use, code execution, and subagents for the type fixes in separate files ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details. The app generated the branch name. - [ ] I have run tests locally and they pass. See the failures explained in Verification. - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
413 lines
27 KiB
TypeScript
413 lines
27 KiB
TypeScript
import { expect, test, type Locator, type Page } from "@playwright/test";
|
||
import { THREAD_PREVIEW_CASES } from "../../ui/src/components/design-sandbox/task-thread-fixtures";
|
||
|
||
async function openStory(page: Page, id: string) {
|
||
await page.goto(`/iframe.html?id=${id}&viewMode=story&globals=theme:dark`);
|
||
}
|
||
|
||
async function resolvedColor(locator: Locator, token: string): Promise<string> {
|
||
return locator.evaluate((element, tokenName) => {
|
||
const probe = document.createElement("span");
|
||
probe.style.color = `var(${tokenName})`;
|
||
element.append(probe);
|
||
const color = getComputedStyle(probe).color;
|
||
probe.remove();
|
||
return color;
|
||
}, token);
|
||
}
|
||
|
||
test("proposal typography reaches shared buttons, prose and identifiers", async ({ page }) => {
|
||
await openStory(page, "design-foundations-typography--type-specification");
|
||
const guide = page.getByTestId("typography-foundations-preview");
|
||
await expect(guide).toBeVisible();
|
||
await page.evaluate(() => document.fonts.ready);
|
||
await expect(guide.locator("[data-ramp]")).toHaveCount(9);
|
||
await expect(guide.locator('[data-type-sample^="weight-"]')).toHaveCount(2);
|
||
await expect(guide.getByRole("button", { name: "Continue", exact: true })).toHaveCSS("font-weight", "400");
|
||
for (const selector of ['[data-type-sample="detail-id"]', ".paperclip-markdown code", '.task-row-columns [data-column="taskId"]']) {
|
||
const element = guide.locator(selector).first();
|
||
await expect(element).toHaveCSS("font-family", /"Paper Mono"/);
|
||
}
|
||
await expect(guide.locator(".paperclip-markdown a")).toHaveCSS("font-weight", "400");
|
||
await expect(guide.locator(".paperclip-markdown strong")).toHaveCSS("font-weight", "600");
|
||
await expect(guide.locator('[data-type-sample="message-time"]')).toHaveCSS("font-size", "12px");
|
||
await expect(guide.locator('[data-type-sample="message-time"]')).toHaveCSS("line-height", "16px");
|
||
const label = (id: string) => guide.locator(`[data-type-sample="${id}"]`).evaluate(element => {
|
||
const style = getComputedStyle(element);
|
||
return [style.fontFamily, style.fontSize, style.fontWeight, style.lineHeight, style.letterSpacing, style.color, style.textTransform];
|
||
});
|
||
expect(await label("property-eyebrow")).toEqual(await label("nav-eyebrow"));
|
||
});
|
||
|
||
test("shared sidebar and portaled account menu use the simplified weights and xs metadata", async ({ page }) => {
|
||
await openStory(page, "design-sidebar--default");
|
||
const organization = page.getByRole("button", { name: /organization switcher/ }).locator("[data-cell-marquee]");
|
||
await expect(organization).toHaveCSS("font-weight", "600");
|
||
await expect(page.getByRole("button", { name: "New Task", exact: true })).toHaveCSS("font-weight", "400");
|
||
await page.getByRole("button", { name: "Open account menu", exact: true }).click();
|
||
const header = page.locator("[data-flyout-profile]");
|
||
await expect(header).toBeVisible();
|
||
await expect(header.locator("p")).toHaveCSS("font-size", "12px");
|
||
await expect(header.locator("p")).toHaveCSS("line-height", "16px");
|
||
await expect(page.getByRole("link", { name: "Settings", exact: true })).toHaveCSS("font-weight", "400");
|
||
});
|
||
|
||
test("leaving a proposal restores the ordinary foundation typography", async ({ page }) => {
|
||
await openStory(page, "design-foundations-typography--type-specification");
|
||
await expect(page.getByRole("button", { name: "Continue", exact: true })).toHaveCSS("font-weight", "400");
|
||
await openStory(page, "foundations-primitive-matrix--all-primitives");
|
||
await expect(page.getByRole("heading", { name: "Primitives and interaction states", exact: true })).toBeVisible();
|
||
await expect(page.locator(".design-proposal-theme")).toHaveCount(0);
|
||
await expect(page.getByRole("button", { name: "default", exact: true }).first()).toHaveCSS("font-weight", "500");
|
||
await expect(page.locator(".font-mono").first()).not.toHaveCSS("font-family", /"Paper Mono"/);
|
||
expect(await page.locator("html").evaluate(element => {
|
||
const style = getComputedStyle(element);
|
||
return [style.getPropertyValue("--font-weight-medium").trim(), style.getPropertyValue("--font-weight-bold").trim(), style.getPropertyValue("--text-micro").trim()];
|
||
})).toEqual(["500", "700", "11px"]);
|
||
});
|
||
|
||
|
||
test("conversation typography uses the sandbox's complete state catalog and grouped composition", async ({ page }) => {
|
||
await openStory(page, "design-foundations-typography--conversation-styles");
|
||
const preview = page.getByTestId("sandbox-task-thread");
|
||
await expect(preview).toHaveAttribute("data-preview-state", "all");
|
||
expect(THREAD_PREVIEW_CASES).toHaveLength(72);
|
||
await expect(preview.locator("[data-preview-case]")).toHaveCount(THREAD_PREVIEW_CASES.length);
|
||
const selector = preview.getByRole("combobox", { name: "Thread preview state" });
|
||
expect(await selector.locator("optgroup option").evaluateAll(options => options.map(option => ({
|
||
id: (option as HTMLOptionElement).value, label: option.textContent,
|
||
group: option.parentElement?.getAttribute("label"),
|
||
}))).then(options => options.sort((a, b) => a.id.localeCompare(b.id)))).toEqual(THREAD_PREVIEW_CASES.map(({ id, label, group }) => ({ id, label, group })).sort((a, b) => a.id.localeCompare(b.id)));
|
||
expect(await preview.locator("[data-preview-case]").evaluateAll(elements => elements.map(element => element.getAttribute("data-preview-case"))))
|
||
.toEqual(THREAD_PREVIEW_CASES.map(example => example.id));
|
||
|
||
await selector.selectOption("staging-conversation");
|
||
await expect(preview.locator("[data-preview-case]")).toHaveCount(1);
|
||
await expect(preview.getByTestId("task-chat-live-run-pill")).toBeVisible();
|
||
await expect(preview.getByTestId("task-chat-runner-turn")).toHaveCount(0);
|
||
await expect(preview.getByText("Thought through the task", { exact: true })).toBeVisible();
|
||
await expect(preview.getByText("Read files", { exact: true })).toBeVisible();
|
||
await expect(preview.getByTestId("task-chat-bubble-attached-turn")).toHaveCount(1);
|
||
await page.getByText("Typography measurements", { exact: true }).click();
|
||
await expect(page.getByTestId("conversation-style-inventory").locator("tbody tr")).not.toHaveCount(0);
|
||
|
||
await selector.selectOption("runner-running");
|
||
await expect(preview.getByTestId("task-chat-runner-turn")).toBeVisible();
|
||
await expect(preview.getByTestId("task-chat-live-run-pill")).toHaveCount(0);
|
||
});
|
||
|
||
test("conversation typography keeps permission decisions and replay in the shared preview", async ({ page }) => {
|
||
await openStory(page, "design-foundations-typography--conversation-styles");
|
||
const preview = page.getByTestId("sandbox-task-thread");
|
||
await preview.getByRole("combobox", { name: "Thread preview state" }).selectOption("permission-pending");
|
||
const allow = preview.getByRole("button", { name: "Allow once", exact: true });
|
||
await allow.click();
|
||
await expect(allow).toHaveCount(0);
|
||
await preview.getByRole("button", { name: "Reset / replay", exact: true }).click();
|
||
await expect(allow).toBeVisible();
|
||
await preview.getByRole("button", { name: "Show saved thread", exact: true }).click();
|
||
await expect(preview).toHaveAttribute("data-preview-state", "saved");
|
||
await expect(preview.locator("[data-preview-case]")).toHaveCount(0);
|
||
await expect(preview.getByText(/Saved task data is available in the sandbox/)).toBeVisible();
|
||
});
|
||
|
||
test("conversation type spec measures the shared states and preserves actual sample typography", async ({ page }) => {
|
||
await openStory(page, "design-foundations-typography--conversation-type-specification");
|
||
const guide = page.getByTestId("conversation-type-spec");
|
||
await expect(guide.getByTestId("conversation-type-census")).toContainText("72 states measured");
|
||
const preview = guide.getByTestId("conversation-spec-shared-preview");
|
||
await expect(preview.locator("[data-preview-case]")).toHaveCount(THREAD_PREVIEW_CASES.length);
|
||
await expect(guide.locator("[data-conversation-weight]")).toHaveCount(2);
|
||
await expect(guide.locator("[data-conversation-candidate]").first()).toBeVisible();
|
||
await guide.getByRole("combobox", { name: "Surface", exact: true }).selectOption("Human message");
|
||
const row = guide.locator("[data-conversation-type-role]").first();
|
||
const properties = (element: Element) => {
|
||
const style = getComputedStyle(element);
|
||
return [style.fontFamily, style.fontSize, style.lineHeight, style.fontWeight, style.color, style.letterSpacing];
|
||
};
|
||
expect(await row.locator("[data-conversation-type-sample]").evaluate(properties))
|
||
.toEqual(await preview.getByTestId("task-chat-human-bubble").first().locator("p").first().evaluate(properties));
|
||
await row.getByText("Source & recipe", { exact: true }).click();
|
||
await expect(row.getByText("ui/src/components/task-chat/TaskChatBubble.tsx", { exact: true })).toBeVisible();
|
||
await guide.getByRole("searchbox", { name: "Find a conversation text role" }).fill("no-such-type-recipe");
|
||
await expect(guide.locator("[data-conversation-type-role]")).toHaveCount(0);
|
||
await expect(guide.getByText("No roles match these filters.", { exact: true })).toBeVisible();
|
||
});
|
||
|
||
test("conversation type spec can measure expanded shared content without deciding requests", async ({ page }) => {
|
||
await openStory(page, "design-foundations-typography--conversation-type-specification");
|
||
const guide = page.getByTestId("conversation-type-spec");
|
||
await expect(guide.getByTestId("conversation-type-census")).toContainText("72 states measured");
|
||
const preview = guide.getByTestId("conversation-spec-shared-preview");
|
||
await preview.getByRole("combobox", { name: "Thread preview state" }).selectOption("working");
|
||
await guide.getByRole("button", { name: "Expand details & measure", exact: true }).click();
|
||
await expect(guide.getByTestId("conversation-type-census")).toContainText("1 states measured");
|
||
await expect(preview.getByTestId("task-chat-runner-activity-detail").first()).toBeVisible();
|
||
await expect(guide.locator('[data-conversation-type-role="Activity detail · code / command"]').first()).toBeVisible();
|
||
await preview.getByRole("combobox", { name: "Thread preview state" }).selectOption("permission-pending");
|
||
await guide.getByRole("button", { name: "Expand details & measure", exact: true }).click();
|
||
await expect(preview.getByRole("button", { name: "Allow once", exact: true })).toBeVisible();
|
||
await guide.getByRole("button", { name: "Show all states", exact: true }).click();
|
||
await expect(preview.locator("[data-preview-case]")).toHaveCount(THREAD_PREVIEW_CASES.length);
|
||
await guide.getByRole("button", { name: "Refresh type inventory", exact: true }).click();
|
||
await expect(guide.getByTestId("conversation-type-census")).toContainText("72 states measured");
|
||
});
|
||
|
||
test("proposal conversation uses two sizes with muted progress, contrasting bubbles and preserved shimmer", async ({ page }) => {
|
||
await openStory(page, "design-foundations-typography--conversation-type-specification");
|
||
const guide = page.getByTestId("conversation-type-spec");
|
||
await expect(guide.getByTestId("conversation-type-census")).toContainText("72 states measured");
|
||
const preview = guide.getByTestId("conversation-spec-shared-preview");
|
||
expect(await guide.locator("[data-conversation-size]").evaluateAll(elements => elements.map(element => element.getAttribute("data-conversation-size")).sort())).toEqual(["12px", "14px"]);
|
||
const human = preview.locator('[data-conversation-text="human"] p').first();
|
||
const final = preview.locator('[data-conversation-text="final"] p').first();
|
||
const progress = preview.locator('[data-conversation-text="progress"] p').first();
|
||
for (const element of [human, final, progress]) {
|
||
await expect(element).toHaveCSS("font-size", "14px");
|
||
await expect(element).toHaveCSS("line-height", "20px");
|
||
}
|
||
await expect(human).toHaveCSS("color", "rgb(255, 255, 255)");
|
||
const foregroundToken = await preview.evaluate(element => getComputedStyle(element).getPropertyValue("--foreground").trim());
|
||
await expect(final).toHaveCSS("color", await resolvedColor(preview, "--foreground"));
|
||
await expect(progress).toHaveCSS("color", await resolvedColor(preview, "--muted-foreground"));
|
||
const pill = preview.locator('[data-preview-case="session-start"]').getByTestId("task-chat-live-run-pill");
|
||
await expect(pill.locator(".shimmer-text")).toHaveCSS("font-size", "14px");
|
||
await expect(pill.locator(".shimmer-text")).toHaveCSS("--shimmer-base", foregroundToken);
|
||
await expect(pill.locator("span").filter({ hasText: /^for / }).last()).toHaveCSS("font-size", "12px");
|
||
await guide.getByRole("button", { name: "Expand details & measure", exact: true }).click();
|
||
expect(await guide.locator("[data-conversation-size]").evaluateAll(elements => elements.map(element => element.getAttribute("data-conversation-size")).sort())).toEqual(["12px", "14px"]);
|
||
await expect(preview.locator('[data-testid="task-chat-runner-activity-detail"] .paperclip-markdown p').first()).toHaveCSS("font-size", "14px");
|
||
const streamingFinal = preview.locator('[data-preview-case="streaming-final"] [data-conversation-text="final"] p').last();
|
||
await expect(streamingFinal).toHaveCSS("color", await resolvedColor(preview, "--foreground"));
|
||
await expect(preview.locator('[data-testid="task-chat-turn-status-header"] [data-conversation-meta]').last()).toHaveCSS("font-size", "12px");
|
||
await page.emulateMedia({ reducedMotion: "no-preference" });
|
||
await expect(pill.locator(".shimmer-text")).toHaveCSS("animation-name", "shimmer-text-slide");
|
||
});
|
||
|
||
test("conversation playback progresses through one thread and supports pause and replay", async ({ page }) => {
|
||
await page.clock.install();
|
||
await openStory(page, "design-foundations-typography--conversation-playback");
|
||
const playback = page.getByTestId("conversation-playback");
|
||
await expect(playback).toHaveAttribute("data-playback-state", "idle");
|
||
await expect(page.getByRole("combobox", { name: "Thread preview state" })).toHaveCount(0);
|
||
await playback.getByRole("button", { name: "Play conversation", exact: true }).click();
|
||
await expect(playback.getByTestId("task-chat-human-bubble")).toHaveCount(1);
|
||
await page.clock.runFor(2500);
|
||
await expect(playback).toHaveAttribute("data-playback-stage", "1");
|
||
await expect(playback.getByTestId("task-chat-live-run-pill")).toBeVisible();
|
||
await playback.getByRole("button", { name: "Pause", exact: true }).click();
|
||
const paused = await playback.innerText();
|
||
await page.clock.runFor(5000);
|
||
expect(await playback.innerText()).toBe(paused);
|
||
await expect(playback).toHaveAttribute("data-playback-state", "paused");
|
||
await playback.getByRole("button", { name: "Resume conversation", exact: true }).click();
|
||
await page.clock.runFor(3000);
|
||
await expect(playback).toHaveAttribute("data-playback-stage", "3");
|
||
await expect(playback.getByTestId("task-chat-human-bubble")).toHaveCount(1);
|
||
await expect(playback.getByText(/inspect the component alias/)).toBeVisible();
|
||
const runningLabel = playback.locator('[data-conversation-tool-running="true"]').first();
|
||
await page.emulateMedia({ reducedMotion: "no-preference" });
|
||
await expect(runningLabel).toHaveCSS("animation-name", "shimmer-text-slide");
|
||
const foreground = await resolvedColor(playback, "--foreground");
|
||
await expect(runningLabel).toHaveCSS("color", foreground);
|
||
await page.clock.runFor(7500);
|
||
await expect(playback).toHaveAttribute("data-playback-stage", "6");
|
||
await expect(playback.locator('[data-conversation-text="final"] .paperclip-markdown').last()).toHaveCSS("color", foreground);
|
||
await page.clock.runFor(7500);
|
||
await expect(playback).toHaveAttribute("data-playback-stage", "9");
|
||
await expect(playback.getByText("Which reconnect policy should I test?", { exact: true })).toBeVisible();
|
||
await expect(playback.getByTestId("task-chat-human-bubble")).toHaveCount(2);
|
||
await page.clock.runFor(11_000);
|
||
await expect(playback).toHaveAttribute("data-playback-stage", "13");
|
||
await expect(playback.locator('[data-conversation-tool-running="true"]')).toHaveCount(0);
|
||
await page.clock.runFor(5000);
|
||
await expect(playback).toHaveAttribute("data-playback-stage", "15");
|
||
await expect(playback.locator('[data-conversation-tool-running="true"]').first()).toHaveCSS("animation-name", "shimmer-text-slide");
|
||
await page.clock.runFor(10_000);
|
||
await expect(playback).toHaveAttribute("data-playback-state", "complete");
|
||
await expect(playback.getByTestId("task-chat-agent-bubble")).toHaveCount(2);
|
||
await expect(playback.getByText("26 component source files", { exact: true })).toBeVisible();
|
||
await playback.getByTestId("task-chat-turn-summary").last().click();
|
||
await expect(playback.getByTestId("task-chat-activity-phase-toggle").first()).toBeVisible();
|
||
await playback.getByRole("button", { name: "Replay conversation", exact: true }).click();
|
||
await expect(playback).toHaveAttribute("data-playback-stage", "0");
|
||
await expect(playback.getByTestId("task-chat-agent-bubble")).toHaveCount(0);
|
||
});
|
||
|
||
|
||
test("conversation metadata has hover and focus feedback and agent names stay body-sized", async ({ page }) => {
|
||
await openStory(page, "design-foundations-typography--conversation-styles");
|
||
const preview = page.getByTestId("sandbox-task-thread");
|
||
const name = preview.getByTestId("task-chat-agent-identity").first().locator(":scope > span:last-child");
|
||
await expect(name).toHaveCSS("font-size", "14px");
|
||
await expect(name).toHaveCSS("line-height", "20px");
|
||
await expect(name).toHaveCSS("font-weight", "600");
|
||
const foreground = await resolvedColor(preview, "--foreground");
|
||
await expect(name).toHaveCSS("color", foreground);
|
||
const heading = preview.locator('[data-conversation-text="final"] h2').first();
|
||
await expect(heading).toHaveCSS("font-size", "14px");
|
||
await expect(heading).toHaveCSS("font-weight", "400");
|
||
await expect(heading).toHaveCSS("color", foreground);
|
||
const activity = preview.getByTestId("task-chat-turn-summary").first();
|
||
const label = activity.locator("span").filter({ hasText: /^Activity$/ });
|
||
const base = await label.evaluate(element => getComputedStyle(element).color);
|
||
await activity.hover();
|
||
await expect(label).toHaveCSS("color", foreground);
|
||
await expect(activity).not.toHaveCSS("background-color", "rgba(0, 0, 0, 0)");
|
||
await page.mouse.move(0, 0);
|
||
await expect(label).toHaveCSS("color", base);
|
||
await activity.focus();
|
||
await expect(label).toHaveCSS("color", foreground);
|
||
await expect(activity).toHaveCSS("font-size", "12px");
|
||
});
|
||
|
||
test("playback appends complete words with a fade while retaining previous word nodes", async ({ page }) => {
|
||
await page.clock.install();
|
||
await page.emulateMedia({ reducedMotion: "no-preference" });
|
||
await openStory(page, "design-foundations-typography--conversation-playback");
|
||
const playback = page.getByTestId("conversation-playback");
|
||
await playback.getByRole("button", { name: "Play conversation", exact: true }).click();
|
||
await page.clock.runFor(3500);
|
||
const prose = playback.locator('[data-conversation-text="progress"] .paperclip-markdown').last();
|
||
const words = prose.locator(".conversation-stream-word");
|
||
await expect(words.first()).toHaveText("I’ll");
|
||
await expect(words.first()).toHaveCSS("animation-name", "conversation-word-in");
|
||
await expect(words.first()).toHaveCSS("animation-duration", "0.32s");
|
||
const initialCount = await words.count();
|
||
const oldWord = await words.first().elementHandle();
|
||
await page.clock.runFor(200);
|
||
expect(await words.count()).toBeGreaterThan(initialCount);
|
||
expect(await oldWord!.evaluate(element => element.isConnected)).toBe(true);
|
||
const text = await prose.textContent();
|
||
expect("I’ll inspect the component alias first, then check the file list. No code changes are needed.".startsWith(text!)).toBe(true);
|
||
await page.emulateMedia({ reducedMotion: "reduce" });
|
||
await expect(words.first()).toHaveCSS("animation-name", "none");
|
||
});
|
||
|
||
test("completion fades activity before folding its space and preserves the mounted answer", async ({ page }) => {
|
||
await page.clock.install();
|
||
await page.emulateMedia({ reducedMotion: "no-preference" });
|
||
await openStory(page, "design-foundations-typography--conversation-playback");
|
||
const playback = page.getByTestId("conversation-playback");
|
||
await playback.getByRole("button", { name: "Play conversation", exact: true }).click();
|
||
await page.clock.runFor(15_800);
|
||
const turn = playback.getByTestId("task-chat-turn-handoff").first();
|
||
const activity = turn.getByTestId("task-chat-handoff-activity");
|
||
const answer = turn.getByTestId("task-chat-agent-bubble");
|
||
const node = await answer.elementHandle();
|
||
const prose = await answer.locator("p").first().elementHandle();
|
||
const before = await answer.boundingBox();
|
||
const expandedHeight = (await activity.boundingBox())!.height;
|
||
await page.clock.runFor(240);
|
||
await expect(turn).toHaveAttribute("data-handoff-state", "settling");
|
||
await expect(activity).toHaveAttribute("aria-hidden", "true");
|
||
await expect(activity).toHaveAttribute("inert", "");
|
||
await expect(activity).toHaveCSS("transition-delay", "0s, 0.18s");
|
||
await expect(activity).toHaveCSS("transition-duration", "0.18s, 0.48s");
|
||
expect(await node!.evaluate(element => element.isConnected)).toBe(true);
|
||
expect(await prose!.evaluate(element => element.isConnected)).toBe(true);
|
||
// Observe intermediate rendered geometry, not just a class name or endpoint.
|
||
await expect.poll(async () => Number(await activity.evaluate(element => getComputedStyle(element).opacity))).toBeLessThan(1);
|
||
await expect.poll(async () => (await activity.boundingBox())!.height).toBeLessThan(expandedHeight);
|
||
await expect(turn).toHaveAttribute("data-handoff-state", "settled");
|
||
expect((await activity.boundingBox())!.height).toBe(0);
|
||
expect((await answer.boundingBox())!.y).toBeLessThan(before!.y);
|
||
expect(await node!.evaluate(element => element.isConnected)).toBe(true);
|
||
expect(await prose!.evaluate(element => element.isConnected)).toBe(true);
|
||
await expect(turn.getByTestId("task-chat-turn-summary")).toBeVisible();
|
||
await turn.getByTestId("task-chat-turn-summary").click();
|
||
await expect(turn.getByTestId("task-chat-activity-phase-toggle").first()).toBeVisible();
|
||
|
||
// Replay during an exit must discard its retained activity and old timers.
|
||
await playback.getByRole("button", { name: "Replay", exact: true }).click();
|
||
await page.clock.runFor(15_800);
|
||
await page.clock.runFor(240);
|
||
await expect(playback.getByTestId("task-chat-turn-handoff").first()).toHaveAttribute("data-handoff-state", "settling");
|
||
await playback.getByRole("button", { name: "Replay", exact: true }).click();
|
||
await expect(playback.getByTestId("task-chat-turn-handoff")).toHaveCount(1);
|
||
await expect(playback.getByTestId("task-chat-agent-bubble")).toHaveCount(0);
|
||
});
|
||
|
||
test("reduced motion settles the final response without an activity exit", async ({ page }) => {
|
||
await page.clock.install();
|
||
await page.emulateMedia({ reducedMotion: "reduce" });
|
||
await openStory(page, "design-foundations-typography--conversation-playback");
|
||
const playback = page.getByTestId("conversation-playback");
|
||
await playback.getByRole("button", { name: "Play conversation", exact: true }).click();
|
||
await page.clock.runFor(16_040);
|
||
const turn = playback.getByTestId("task-chat-turn-handoff").first();
|
||
await expect(turn).toHaveAttribute("data-handoff-state", "settled");
|
||
await expect(turn.getByTestId("task-chat-handoff-activity")).toHaveCSS("transition-property", "none");
|
||
await expect(turn.getByTestId("task-chat-turn-summary")).toBeVisible();
|
||
await expect(turn.getByTestId("task-chat-live-run-pill")).toHaveCount(0);
|
||
});
|
||
|
||
test("one current item owns shimmer and startup plumbing stays out of the conversation", async ({ page }) => {
|
||
await page.clock.install();
|
||
await page.emulateMedia({ reducedMotion: "no-preference" });
|
||
await openStory(page, "design-foundations-typography--conversation-playback");
|
||
const playback = page.getByTestId("conversation-playback");
|
||
const owners = playback.locator('.shimmer-text, [data-conversation-tool-running="true"]');
|
||
await playback.getByRole("button", { name: "Play conversation", exact: true }).click();
|
||
await expect(owners).toHaveCount(1);
|
||
await expect(owners).toHaveText("Working");
|
||
await expect(playback).not.toContainText(/Preparing the turn|Preparing the transcript|Waiting for transcript|run phase:/i);
|
||
await page.clock.runFor(1600);
|
||
await expect(owners).toHaveCount(1);
|
||
await expect(owners).toHaveText("Thinking");
|
||
const working = playback.getByTestId("task-chat-live-run-pill").getByText("Working", { exact: true });
|
||
const muted = await resolvedColor(playback, "--muted-foreground");
|
||
await expect(working).toHaveCSS("color", muted);
|
||
await expect(working).toHaveCSS("animation-name", "none");
|
||
await page.clock.runFor(3800);
|
||
await expect(owners).toHaveCount(1);
|
||
await expect(owners).toHaveAttribute("data-conversation-tool-running", "true");
|
||
await expect(owners).toHaveCSS("animation-name", "shimmer-text-slide");
|
||
await expect(working).toHaveCSS("color", muted);
|
||
await page.clock.runFor(7400);
|
||
await expect(playback).toHaveAttribute("data-playback-stage", "6");
|
||
await expect(owners).toHaveCount(0);
|
||
});
|
||
|
||
test("mixed-font tool text shares one uninterrupted shimmer loop", async ({ page }) => {
|
||
await page.clock.install();
|
||
await page.emulateMedia({ reducedMotion: "no-preference" });
|
||
await openStory(page, "design-foundations-typography--conversation-playback");
|
||
const playback = page.getByTestId("conversation-playback");
|
||
await playback.getByRole("button", { name: "Play conversation", exact: true }).click();
|
||
await page.clock.runFor(9600);
|
||
const line = playback.locator('[data-conversation-tool-running="true"]');
|
||
await expect(line).toHaveCount(1);
|
||
await expect(line).toContainText("Running a command");
|
||
await expect(line.locator("span").first()).toHaveCSS("font-family", /Inter/);
|
||
await expect(line.locator("span").last()).toHaveCSS("font-family", /Paper Mono/);
|
||
for (const child of await line.locator(":scope > span").all()) {
|
||
await expect(child).toHaveCSS("animation-name", "none");
|
||
await expect(child).toHaveCSS("background-image", "none");
|
||
await expect(child).toHaveCSS("-webkit-text-fill-color", "rgba(0, 0, 0, 0)");
|
||
}
|
||
const sweep = await line.evaluate(element => {
|
||
const animations = element.getAnimations({ subtree: true });
|
||
const animation = animations.find(candidate => (candidate as CSSAnimation).animationName === "shimmer-text-slide")!;
|
||
const timing = animation.effect!.getTiming();
|
||
const keyframes = (animation.effect as KeyframeEffect).getKeyframes();
|
||
// Sample the old dead zone: the background must keep moving after 60%.
|
||
animation.pause();
|
||
animation.currentTime = Number(timing.duration) * 0.65;
|
||
const first = getComputedStyle(element).backgroundPosition;
|
||
animation.currentTime = Number(timing.duration) * 0.95;
|
||
const second = getComputedStyle(element).backgroundPosition;
|
||
animation.play();
|
||
return { count: animations.length, easing: timing.easing, forever: timing.iterations === Infinity, offsets: keyframes.map(frame => frame.offset), first, second };
|
||
});
|
||
expect(sweep.count).toBe(1);
|
||
expect(sweep.easing).toBe("linear");
|
||
expect(sweep.forever).toBe(true);
|
||
expect(sweep.offsets).toEqual([0, 1]);
|
||
expect(sweep.first).not.toBe(sweep.second);
|
||
await page.emulateMedia({ reducedMotion: "reduce" });
|
||
await expect(line).toHaveCSS("animation-name", "none");
|
||
await expect(line).toHaveCSS("background-image", "none");
|
||
await expect(line.locator("span").last()).toHaveCSS("-webkit-text-fill-color", await resolvedColor(playback, "--foreground"));
|
||
});
|