test(storybook): add production examples for feature map UI gaps (#16196)

## Thinking Path

> - Paperclip manages AI agents and their work.
> - The feature map records the user-facing capabilities.
> - Storybook uses the same feature index.
> - Step 2 found 49 UI capabilities with no authored examples.
> - This pull request adds shipped-component examples with isolated
fixtures.
> - The catalog records each example's states and remaining production
checks.

## Linked Issues or Issue Description

Refs: #16167. The verification inventory is merged; this follow-up now
targets master.

## What Changed

- Add 37 suites and 257 scenarios for all 49 UI capabilities from the
step-2 inventory. Keep existing stories and IDs.
- Use shipped pages and components. Simulate API replies, sockets,
props, and callbacks at their boundaries.
- Activate story API fixtures before the application providers read
data. Block unregistered API mutations and restore owned storage/fetch
state.
- Add browser checks for rendering, declared play assertions, fixture
reset, and selected mobile/theme states. Update the catalog and
verification inventory.

## Verification

- `pnpm -r typecheck`: passed.
- `pnpm build`: passed.
- UI Vitest suite: 730 files and 8,208 tests passed. Fixture boundary
tests: 4 passed.
- Map/catalog checks, 27 generator/map tests, and token gates: passed.
- `pnpm build-storybook`: passed. All 2,248 previous built entry IDs
remain present.
- Pre-reconciliation static browser suite: 260 checks passed (257
scenarios, token fixture reset, and two mobile/theme cases). All
non-play stories assert an explicit subject state; 129 play functions
passed.
- `pnpm test:run` could not complete on this Mac. Embedded PostgreSQL
exhausted host shared-memory allocation. The run also reported
native-runner test timeouts. Only this task's verified runner was
stopped; other jobs and host settings were preserved.
- Reconciled onto merged master `252d9b5077`. The six draft stories and
new same-browser reset regression passed (7 checks); 16
draft-persistence tests and UI typecheck passed. Static merge
verification: 32 affected-subject/draft checks passed, including
same-browser reset. Rebuilt Storybook retains all 2,248 original IDs in
its 2,517-entry index. Four additional play-result checks passed:
visible onboarding reset, pending probe, filtered-empty runs, and
optimistic queued-response removal. UI typecheck and 27 catalog/map
tests also passed. Final static browser suite on
`3fab35df8d912935fa86ce0de7d32d07e2b5f4c3`: all 261 checks passed (257
scenarios plus fixture-reset and mobile checks). Latest head
`3fab35df8d912935fa86ce0de7d32d07e2b5f4c3`: all 55 CI/status checks
passed; the two optional Storybook workflow jobs were skipped. Greptile
5/5 with no unresolved threads.
- No pinned Linux visual-baseline comparison or real provider journey is
claimed.

## Risks

Fixtures can drift from production contracts. The catalog records
inspected sources and their fingerprints. Capability links provide
initial isolated UI coverage; they do not certify every recipe variant,
real permissions, persistence, or provider behavior.

The new examples expose existing UI limits. Plugin discovery failure
stays on Loading. Document and environment-list reads lack dedicated
failure feedback; Cases and Inbox fall back to empty views after errors.
Goal alignment has display views but no assignment control. These
findings remain visible in the inventory.

## Model Used

OpenAI GPT-6 via Codex. The exact runtime deployment ID and context
window are not exposed in this session. Used code generation, shell
execution, and browser verification.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-10-13 13:24:09 -05:00
1 parent 252d9b5077
commit c8efdbfba4
54 files changed
+6759 -73

No files matched your search

@@ -0,0 +1,321 @@
import { expect, type FrameLocator } from "@playwright/test";
// These expectations name shipped UI states, not captured visual baselines.
// Explicit fallback cases document where production has no separate error UI.
type State = {
text?: string | RegExp;
absent?: string | RegExp;
skeleton?: true;
selector?: string;
emptyRootReason?: string;
tokenHistory?: "active" | "expired" | "empty";
editable?: true;
defaultEnvironment?: true;
};
const failed: State = { text: "The request failed. Please try again." };
const denied: State = {
text: "You do not have permission to perform this action.",
};
const skeleton: State = { skeleton: true };
const documentFallback: State = {
text: "New document",
absent: "Run the release checks.",
};
const environmentFallback: State = {
defaultEnvironment: true,
absent: "Release sandbox",
};
const states: Record<string, Record<string, State>> = {
"feature-cli-authorization": {
pending: { text: "Approve Paperclip CLI access" },
loading: { text: "Loading CLI auth challenge..." },
expired: { text: "CLI auth challenge expired" },
unavailable: { text: "CLI auth challenge unavailable" },
},
"feature-membership": {
members: { text: "Release reviewer" },
loading: { text: "Loading organization access…" },
empty: { text: "No user memberships found for this organization yet." },
failed,
"permission-denied": {
text: "You do not have permission to manage organization members.",
},
},
"feature-sign-in": {
"signed-out": { text: "Sign in to Paperclip" },
loading: { text: "Loading…" },
"session-failed": {
text: "Unable to check sign-in. Refresh and try again.",
},
},
"feature-run-audit": {
loading: { text: "Loading runs…" },
empty: { text: "No runs yet." },
},
"feature-conference-room": {
empty: { text: "Conference Room", editable: true },
},
"feature-catalog-review": {
quarantined: { text: "Review 1 new action" },
empty: { text: "No actions match “”. Clear the search to see them all." },
refreshing: { text: "Review 1 new action", selector: ".animate-spin" },
saving: { text: "Saving…" },
"read-only": { text: "Actions", absent: "Review 1 new action" },
},
"feature-budget-resolution": {
loading: skeleton,
empty: { text: /No budget policies yet\./ },
"load-failed": {
text: "Budget data could not be loaded. Please try again shortly.",
},
"permission-denied": {
text: "Budget data could not be loaded. Please try again shortly.",
},
},
// Cases currently uses its empty presentation after a failed list query.
"feature-case-browse": {
loading: skeleton,
empty: { text: "No cases yet" },
failed: { text: "No cases yet" },
"permission-denied": { text: "No cases yet" },
},
"feature-case-detail": {
loading: skeleton,
failed: { text: "Case not found." },
"permission-denied": { text: "Case not found." },
"arbitrary-fields": { text: "0.3.1" },
},
"feature-channel-attachments": {
"no-files": { text: "Here is the release report.", absent: /Files ·/ },
"multiple-files": { text: "Files · 2" },
"missing-filename": { text: "attachment" },
},
"feature-company-import": { "choose-source": { text: "Import source" } },
"feature-company-selection": {
organizations: { text: "Archived Research" },
loading: { text: "Loading organizations..." },
empty: { text: "New Organization", absent: "Paperclip Storybook" },
failed,
"permission-denied": {
text: "New Organization",
absent: "Paperclip Storybook",
},
},
"feature-document-revisions": {
empty: documentFallback,
loading: documentFallback,
"read-failed": documentFallback,
"permission-denied": documentFallback,
edit: { text: "Run the release checks.", editable: true },
},
"feature-environment-images": {
"no-image": { text: "Not configured" },
"image-read-failed": { text: "Image metadata is unavailable." },
"unsupported-provider": { text: "Unsupported provider" },
},
"feature-execution-environments": {
loading: environmentFallback,
empty: environmentFallback,
failed: environmentFallback,
"permission-denied": environmentFallback,
"unsupported-provider": { text: "Isolated release checks" },
"edit-configuration": {
text: "Edit environment",
selector: '[data-testid="environment-form-page"]',
},
},
"feature-gateway-tokens": {
"active-token": { tokenHistory: "active" },
"no-tokens": { tokenHistory: "empty" },
"expired-token": { tokenHistory: "expired" },
},
"feature-goal-alignment": {
"empty-relationships": { text: "No sub-goals." },
loading: skeleton,
failed,
"permission-denied": denied,
},
"feature-inbox-views": {
mine: { text: "Create super-detailed storybooks for the project" },
recent: { text: "Create super-detailed storybooks for the project" },
unread: { text: "Create super-detailed storybooks for the project" },
blocked: { text: "No work is stopped." },
empty: { text: "Inbox zero." },
loading: skeleton,
failed: { text: "Inbox zero." },
"permission-denied": { text: "Inbox zero." },
},
// Order reads are allowed to fall back to navigation; section collapse is local.
"feature-sidebar-preferences": {
"saved-order": {
text: "Paperclip Storybook",
selector: 'a[href="/PAP/dashboard"]',
},
"loading-order": {
text: "Paperclip Storybook",
selector: 'a[href="/PAP/dashboard"]',
},
"order-read-failed": {
text: "Paperclip Storybook",
selector: 'a[href="/PAP/dashboard"]',
},
"order-permission-denied": {
text: "Paperclip Storybook",
selector: 'a[href="/PAP/dashboard"]',
},
},
"feature-onboarding-resume": {
"malformed-draft": {
text: "Create your first agent",
selector: '[data-testid="onboarding-wizard"]',
},
},
"feature-pipeline-learnings": {
loading: skeleton,
empty: { text: "No learnings yet." },
failed: { text: "Could not load learnings." },
"permission-denied": { text: "Could not load learnings." },
},
"feature-pipeline-board": {
loading: skeleton,
empty: { text: "0 total items", absent: "Ship the release notes" },
failed: { text: "Pipeline not found." },
"permission-denied": { text: "Pipeline not found." },
},
"feature-pipeline-review": {
loading: skeleton,
empty: { text: "Nothing needs you right now." },
failed: { text: "Some items need attention. Try again in a moment." },
"permission-denied": {
text: "Some items need attention. Try again in a moment.",
},
},
"feature-pipeline-stages": {
loading: skeleton,
failed,
"permission-denied": denied,
},
"feature-plugin-configuration": {
configured: {
text: "Preview plugin",
selector: 'input[value="Release preview"]',
},
loading: { text: "Loading plugin details..." },
unavailable: { text: "Failed to load plugins." },
},
"feature-plugin-contributions": {
loading: { text: "Loading…" },
"missing-contribution": { text: "Preview plugin" },
"discovery-failure": { text: "Loading…" },
},
"feature-plugin-management": {
installed: { text: "Preview plugin" },
empty: { text: "No plugins installed" },
loading: { text: "Loading plugins..." },
failed: { text: "Failed to load plugins." },
"permission-denied": { text: "Failed to load plugins." },
},
"feature-routine-run-now": {
overview: { text: "Review a release" },
loading: skeleton,
failed,
"permission-denied": denied,
},
"feature-secret-proposal-review": {
pending: { text: "dev/github/token" },
loading: { text: "Loading proposals…" },
empty: { text: "No pending proposals" },
failed: { text: "Couldn’t load proposals. Try again." },
},
"feature-skill-test-runs": {
"completed-test": { text: "succeeded" },
queued: { text: "queued" },
running: { text: "running" },
"test-failed": { text: "failed" },
empty: { text: "No test runs yet. Pick an agent and Run." },
loading: { text: "Loading skill…" },
"load-failed": { text: "Skill not found." },
"permission-denied": { text: "Skill not found." },
},
"feature-status-card-states": {
loading: { text: "Loading cards…" },
empty: { text: "No status cards yet" },
"load-failed": { text: "Could not load status cards" },
"permission-denied": {
text: "You do not have permission to perform this action.",
},
"setting-up": { text: "Setting up your card…" },
"generation-failed": { text: "Last update failed" },
"stale-summary": { text: "3 changes since last update" },
},
"feature-composer-draft-recovery": {
restored: { text: "Review the release before publishing.", editable: true },
empty: { editable: true, absent: "Review the release before publishing." },
"uncertain-submission": {
text: /We couldn’t confirm whether this comment was saved\./,
selector: '[role="alert"]',
},
},
"feature-durable-response-queue": {
empty: {
emptyRootReason:
"The shipped queued-message component returns null for zero entries.",
},
},
};
export async function assertRenderState(frame: FrameLocator, id: string) {
const [suite, variant] = id.split("--");
const state = states[suite]?.[variant];
if (!state)
throw new Error(`Declare an explicit render expectation for ${id}`);
expect(
state,
`Declare an explicit render expectation for ${id}`,
).toBeDefined();
if (state.emptyRootReason) {
await expect(frame.locator("#storybook-root")).toBeEmpty();
return;
}
if (state.text)
await expect(
frame
.getByText(state.text, { exact: typeof state.text === "string" })
.first(),
).toBeVisible();
if (state.absent)
await expect(
frame.getByText(state.absent, {
exact: typeof state.absent === "string",
}),
).toHaveCount(0);
if (state.skeleton)
await expect(frame.locator('[data-slot="skeleton"]').first()).toBeVisible();
if (state.selector)
await expect(frame.locator(state.selector).first()).toBeVisible();
if (state.defaultEnvironment) {
const select = frame.getByRole("combobox", { name: "Default environment" });
await expect(select).toBeVisible();
await expect(select).toHaveValue("");
await expect(
select.getByRole("option", { name: "Local", exact: true }),
).toBeAttached();
}
if (state.editable)
await expect(frame.getByRole("textbox").first()).toBeVisible();
if (state.tokenHistory) {
await frame.getByRole("button", { name: /Token history/ }).click();
if (state.tokenHistory === "empty")
await expect(
frame.getByText("No tokens have been issued for this gateway."),
).toBeVisible();
else {
await expect(
frame.getByText("Preview client", { exact: true }).first(),
).toBeVisible();
if (state.tokenHistory === "expired")
await expect(
frame.getByText("Expired", { exact: true }).first(),
).toBeVisible();
}
}
}
@@ -0,0 +1,17 @@
import { defineConfig } from "@playwright/test";
export default defineConfig({
testDir: ".",
testMatch: "feature-gap-stories.spec.ts",
timeout: 60_000,
expect: { timeout: 15_000 },
retries: 0,
fullyParallel: true,
workers: 4,
reporter: "list",
outputDir: "./test-results/feature-gaps",
use: {
baseURL: process.env.PAPERCLIP_GAP_STORYBOOK_URL ?? "http://localhost:6136",
viewport: { width: 1280, height: 900 },
reducedMotion: "reduce",
},
});
@@ -0,0 +1,170 @@
import { assertRenderState } from "./feature-gap-render-states";
import { readFileSync, writeFileSync } from "node:fs";
import { expect, test } from "@playwright/test";
import { scanStories } from "../../scripts/storybook-catalog.mjs";
const root = new URL("../../", import.meta.url).pathname;
const catalog = JSON.parse(
readFileSync(
new URL("../../maps/storybook-map/catalog.json", import.meta.url),
"utf8",
),
);
const suites = catalog.entries.filter(
(entry: { stableId: string }) =>
entry.stableId.startsWith("feature-") &&
!entry.stableId.startsWith("feature-map-"),
);
const parsed = scanStories(root);
let index: { entries: Record<string, { tags: string[] }> };
test.beforeAll(async ({ request }) => {
const response = await request.get("/index.json");
expect(response.ok()).toBe(true);
index = await response.json();
});
test.afterEach(async ({ page }, info) => {
if (info.status !== info.expectedStatus) {
const managerPath = info.outputPath("manager.txt");
writeFileSync(managerPath, await page.locator("body").innerText());
await info.attach("storybook-page", {
path: managerPath,
contentType: "text/plain",
});
const frame = page.frameLocator("#storybook-preview-iframe");
const subjectPath = info.outputPath("subject.txt");
writeFileSync(subjectPath, await frame.locator("body").innerText());
await info.attach("storybook-subject", {
path: subjectPath,
contentType: "text/plain",
});
}
});
for (const suite of suites) {
const file = parsed.find(
(story: { file: string }) => story.file === suite.file,
);
for (const id of file.ids) {
test(id, async ({ page }, info) => {
expect(
index.entries[id],
`Rebuild Storybook to include ${id}`,
).toBeDefined();
const errors: string[] = [];
page.on("pageerror", (error) => errors.push(error.message));
// Opening an addon during play steals focus from inputs and dismisses menus.
await page.goto(
`/?path=/story/${id}&addonPanel=storybook/interactions/panel`,
);
const frame = page.frameLocator("#storybook-preview-iframe");
await expect(frame.locator("body")).toHaveClass(/sb-show-main/);
await expect(frame.locator(".sb-show-errordisplay")).toHaveCount(0);
await expect(
page.getByRole("tab", { name: /^Interactions/ }),
).toHaveAttribute("aria-selected", "true");
const status = page.locator('[aria-label^="Story status:"]');
if (index.entries[id]?.tags.includes("play-fn")) {
await expect(status).toHaveAttribute(
"aria-label",
"Story status: Pass",
);
}
if (!index.entries[id].tags.includes("play-fn"))
await assertRenderState(frame, id);
await expect(frame.locator("#storybook-root")).toBeAttached();
expect(errors).toEqual([]);
if (
id.includes("accepted") ||
id.includes("permission") ||
id.includes("save-retention")
)
await page.screenshot({ path: info.outputPath(`${id}.png`) });
});
}
}
test("token fixtures reset when switching stories in one browser", async ({
page,
}) => {
await page.goto(
"/?path=/story/feature-gateway-tokens--issued-token&addonPanel=storybook/interactions/panel",
);
await expect(
page.getByRole("tab", { name: /^Interactions/ }),
).toHaveAttribute("aria-selected", "true");
await expect(page.locator('[aria-label^="Story status:"]')).toHaveAttribute(
"aria-label",
"Story status: Pass",
);
await page.getByRole("link", { name: "Active Token", exact: true }).click();
const frame = page.frameLocator("#storybook-preview-iframe");
await frame.getByRole("button", { name: /Token history/ }).click();
await expect(
frame.getByText("Preview client", { exact: true }).first(),
).toBeVisible();
await expect(frame.getByText("Review client", { exact: true })).toHaveCount(
0,
);
});
test("draft fixtures reset when switching away from uncertain submission", async ({
page,
}) => {
await page.goto(
"/?path=/story/feature-composer-draft-recovery--uncertain-submission&addonPanel=storybook/interactions/panel",
);
const frame = page.frameLocator("#storybook-preview-iframe");
const warning = frame.getByRole("alert").filter({
hasText: "We couldn’t confirm whether this comment was saved.",
});
const editor = frame.getByRole("textbox");
await expect(warning).toContainText(
"We couldn’t confirm whether this comment was saved.",
);
await page.getByRole("link", { name: "Empty", exact: true }).click();
await expect(warning).toHaveCount(0);
await expect(editor).toHaveText("");
for (const story of ["Sent Clears Draft", "Send Failed Keeps Draft"]) {
await page
.getByRole("link", { name: "Uncertain Submission", exact: true })
.click();
await expect(warning).toContainText(
"We couldn’t confirm whether this comment was saved.",
);
// Keep the browser/storage context, but reload for play stories so the
// manager cannot report the previous no-play story's Pass during navigation.
const id =
story === "Sent Clears Draft"
? "sent-clears-draft"
: "send-failed-keeps-draft";
await page.goto(
`/?path=/story/feature-composer-draft-recovery--${id}&addonPanel=storybook/interactions/panel`,
);
await expect(frame.locator("body")).toHaveClass(/sb-show-main/);
await expect(editor).toBeVisible();
await expect(warning).toHaveCount(0);
await expect(page.locator('[aria-label^="Story status:"]')).toHaveAttribute(
"aria-label",
"Story status: Pass",
);
await expect(editor).toHaveText(
story === "Sent Clears Draft"
? ""
: "Review the release before publishing.",
);
}
});
for (const theme of ["light", "dark"]) {
test(`backup retention renders on mobile in ${theme}`, async ({
page,
}, info) => {
await page.setViewportSize({ width: 390, height: 844 });
await page.goto(
`/iframe.html?id=feature-backup-retention--current-policy&viewMode=story&globals=theme:${theme}`,
);
await expect(page.getByRole("button", { name: "14 days" })).toBeVisible();
await expect(page.locator(".sb-show-errordisplay")).toHaveCount(0);
await page.screenshot({
path: info.outputPath(`backup-mobile-${theme}.png`),
});
});
}