mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-10-10 12:07:09 +02:00
DottaandPaperclip
d3919713bc
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Storybook visual baselines protect UI surfaces from unintended visual drift. > - Pixel-perfect screenshot baselines are sensitive to OS, font rasterization, and browser environment. > - The suite already stores external baseline artifacts and has an opt-in CI path. > - Local runs on non-matching platforms can report false-positive diffs unless the platform lock is explicit. > - This pull request documents the Linux/Ubuntu baseline constraint and makes the local static server port explicit. > - The benefit is clearer visual-review guidance and more predictable Playwright web server startup. ## Linked Issues or Issue Description No public GitHub issue exists. ### What happened? The Storybook visual baseline suite requires a matching Linux capture environment for pixel-exact comparisons, but the docs did not clearly warn local users that non-Linux environments can produce false-positive diffs. The Playwright web server command also relied on the static server's default port instead of passing the configured port explicitly. ### Expected behavior Developers should see clear Linux/Ubuntu baseline guidance before running the visual suite locally, and Playwright should start the Storybook static server on the same explicit port that the test config expects. ### Steps to reproduce 1. Review the Storybook visual docs before this PR. 2. Run or inspect the Storybook visual Playwright config. 3. Notice the missing platform guidance and implicit static server port coupling. ### Paperclip version or commit Reproducible on `master` before this branch. ### Deployment mode Local dev (pnpm dev) / built from source. ## What Changed - Documents the Linux/Ubuntu-only baseline limitation in the developer docs and visual-suite README. - Adds `--port` parsing and validation to the Storybook static server helper. - Adds regression coverage for `--port` followed by another flag. - Passes the Playwright web server port explicitly from the Storybook visual config. ## Verification - Passed: `node --check scripts/serve-storybook-static.mjs` - Passed: `node --test scripts/__tests__/serve-storybook-static.test.mjs` - Passed: `node --test scripts/__tests__/storybook-visual-baseline.test.mjs` - Greptile: 5/5 with no unresolved review threads after commit `94a649755a2ae7c4a34a3e8a1f16ec4d26d738fd`. - Not run: full `pnpm test:storybook-visual`, because it builds Storybook and runs the browser visual suite; this PR only changes docs plus server port plumbing. ## Risks Low risk. The server still defaults to port 6106 when no explicit port is provided, and invalid port values now fail fast with a clear error before the Playwright server waits for an unreachable URL. ## Model Used OpenAI GPT-5 Codex coding agent with local command execution and repository editing tools. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
67 lines
3.0 KiB
Markdown
67 lines
3.0 KiB
Markdown
# Storybook Visual Baselines
|
|
|
|
The visual suite compares built Storybook stories against PNG snapshots stored
|
|
outside git. The checked-in manifest at `baseline-manifest.json` pins the
|
|
immutable archive URL, SHA-256, byte size, snapshot count, and capture
|
|
environment.
|
|
|
|
## Commands
|
|
|
|
```sh
|
|
pnpm storybook-visual:baseline download
|
|
pnpm storybook-visual:baseline verify
|
|
pnpm test:storybook-visual
|
|
pnpm test:storybook-visual:update
|
|
```
|
|
|
|
`download` fetches the archive, verifies its SHA-256 and byte size, unpacks it to
|
|
`tests/storybook-visual/.snapshots/`, and checks the PNG count. The same snapshot
|
|
directory can be overridden with `STORYBOOK_VISUAL_SNAPSHOT_DIR`.
|
|
|
|
## Known Limitation: Linux Baselines
|
|
|
|
Storybook visual baselines are platform-locked. The checked-in manifest records
|
|
the capture environment as `ubuntu-24.04`, and Playwright compares screenshots
|
|
with `maxDiffPixels: 0`. Pixel-exact results are only meaningful when local runs
|
|
use the same Linux/Ubuntu capture platform as the baseline.
|
|
|
|
macOS, Windows, and other non-matching local environments can produce
|
|
false-positive diffs from font rasterization and subpixel rendering differences.
|
|
Use the `Storybook Visual` GitHub Actions workflow on `ubuntu-latest` as the
|
|
source of truth for cross-platform review, or run the suite locally in a matching
|
|
Linux environment before accepting or updating baselines.
|
|
|
|
## CI and Review Artifacts
|
|
|
|
Storybook visual tests are opt-in while the suite stabilizes. Add the
|
|
`storybook-visual` label to a pull request, or run the `Storybook Visual`
|
|
workflow manually, to download the pinned baseline, build Storybook, and run the
|
|
Playwright visual suite on GitHub Actions.
|
|
|
|
The workflow uploads `tests/storybook-visual/playwright-report/` and
|
|
`tests/storybook-visual/test-results/` as a `storybook-visual-report-*` artifact
|
|
on every run. When screenshots differ, Playwright writes the actual, expected,
|
|
and diff PNGs into `test-results`, so reviewers can inspect the failure without
|
|
rerunning the suite locally.
|
|
|
|
Normal PR visual runs use repository read-only permissions and never upload or
|
|
modify baseline objects. To review intentional visual changes before updating
|
|
`baseline-manifest.json`, run the workflow manually with `update_snapshots`
|
|
enabled. That produces a `storybook-visual-baseline-review-*` artifact containing
|
|
the packed candidate snapshot archive for review. Publishing that bundle to the
|
|
baseline bucket still requires the explicit maintainer upload command below.
|
|
|
|
## Updating Baselines
|
|
|
|
1. Run `pnpm test:storybook-visual:update` after reviewing intentional visual
|
|
diffs.
|
|
2. Run `pnpm storybook-visual:baseline pack` to create
|
|
`tests/storybook-visual/baseline-review/snapshots.tgz`.
|
|
3. Upload the archive from a trusted maintainer environment with
|
|
`STORYBOOK_VISUAL_S3_URI=s3://bucket/baselines/storybook-visual/<sha>/snapshots.tgz pnpm storybook-visual:baseline upload`.
|
|
4. Copy the printed `snapshotCount` and `archive` fields into
|
|
`baseline-manifest.json`.
|
|
|
|
Generated snapshots, review bundles, Playwright reports, and downloaded caches
|
|
are ignored by git.
|