0 Fork 0

mirror of https://github.com/paperclipai/paperclip.git synced 2026-10-06 21:05:21 +02:00

Files

T

DottaandPaperclip d3919713bc [codex] Document Storybook visual baseline platform lock (#9216 )

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - Storybook visual baselines protect UI surfaces from unintended
visual drift.
> - Pixel-perfect screenshot baselines are sensitive to OS, font
rasterization, and browser environment.
> - The suite already stores external baseline artifacts and has an
opt-in CI path.
> - Local runs on non-matching platforms can report false-positive diffs
unless the platform lock is explicit.
> - This pull request documents the Linux/Ubuntu baseline constraint and
makes the local static server port explicit.
> - The benefit is clearer visual-review guidance and more predictable
Playwright web server startup.

## Linked Issues or Issue Description

No public GitHub issue exists.

### What happened?

The Storybook visual baseline suite requires a matching Linux capture
environment for pixel-exact comparisons, but the docs did not clearly
warn local users that non-Linux environments can produce false-positive
diffs. The Playwright web server command also relied on the static
server's default port instead of passing the configured port explicitly.

### Expected behavior

Developers should see clear Linux/Ubuntu baseline guidance before
running the visual suite locally, and Playwright should start the
Storybook static server on the same explicit port that the test config
expects.

### Steps to reproduce

1. Review the Storybook visual docs before this PR.
2. Run or inspect the Storybook visual Playwright config.
3. Notice the missing platform guidance and implicit static server port
coupling.

### Paperclip version or commit

Reproducible on `master` before this branch.

### Deployment mode

Local dev (pnpm dev) / built from source.

## What Changed

- Documents the Linux/Ubuntu-only baseline limitation in the developer
docs and visual-suite README.
- Adds `--port` parsing and validation to the Storybook static server
helper.
- Adds regression coverage for `--port` followed by another flag.
- Passes the Playwright web server port explicitly from the Storybook
visual config.

## Verification

- Passed: `node --check scripts/serve-storybook-static.mjs`
- Passed: `node --test
scripts/__tests__/serve-storybook-static.test.mjs`
- Passed: `node --test
scripts/__tests__/storybook-visual-baseline.test.mjs`
- Greptile: 5/5 with no unresolved review threads after commit
`94a649755a2ae7c4a34a3e8a1f16ec4d26d738fd`.
- Not run: full `pnpm test:storybook-visual`, because it builds
Storybook and runs the browser visual suite; this PR only changes docs
plus server port plumbing.

## Risks

Low risk. The server still defaults to port 6106 when no explicit port
is provided, and invalid port values now fail fast with a clear error
before the Playwright server waits for an unreachable URL.

## Model Used

OpenAI GPT-5 Codex coding agent with local command execution and
repository editing tools.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

2026-07-08 07:23:02 -05:00

3.0 KiB

Raw Blame History

Storybook Visual Baselines

The visual suite compares built Storybook stories against PNG snapshots stored outside git. The checked-in manifest at baseline-manifest.json pins the immutable archive URL, SHA-256, byte size, snapshot count, and capture environment.

Commands

pnpm storybook-visual:baseline download
pnpm storybook-visual:baseline verify
pnpm test:storybook-visual
pnpm test:storybook-visual:update

download fetches the archive, verifies its SHA-256 and byte size, unpacks it to tests/storybook-visual/.snapshots/, and checks the PNG count. The same snapshot directory can be overridden with STORYBOOK_VISUAL_SNAPSHOT_DIR.

Known Limitation: Linux Baselines

Storybook visual baselines are platform-locked. The checked-in manifest records the capture environment as ubuntu-24.04, and Playwright compares screenshots with maxDiffPixels: 0. Pixel-exact results are only meaningful when local runs use the same Linux/Ubuntu capture platform as the baseline.

macOS, Windows, and other non-matching local environments can produce false-positive diffs from font rasterization and subpixel rendering differences. Use the Storybook Visual GitHub Actions workflow on ubuntu-latest as the source of truth for cross-platform review, or run the suite locally in a matching Linux environment before accepting or updating baselines.

CI and Review Artifacts

Storybook visual tests are opt-in while the suite stabilizes. Add the storybook-visual label to a pull request, or run the Storybook Visual workflow manually, to download the pinned baseline, build Storybook, and run the Playwright visual suite on GitHub Actions.

The workflow uploads tests/storybook-visual/playwright-report/ and tests/storybook-visual/test-results/ as a storybook-visual-report-* artifact on every run. When screenshots differ, Playwright writes the actual, expected, and diff PNGs into test-results, so reviewers can inspect the failure without rerunning the suite locally.

Normal PR visual runs use repository read-only permissions and never upload or modify baseline objects. To review intentional visual changes before updating baseline-manifest.json, run the workflow manually with update_snapshots enabled. That produces a storybook-visual-baseline-review-* artifact containing the packed candidate snapshot archive for review. Publishing that bundle to the baseline bucket still requires the explicit maintainer upload command below.

Updating Baselines

Run pnpm test:storybook-visual:update after reviewing intentional visual diffs.
Run pnpm storybook-visual:baseline pack to create tests/storybook-visual/baseline-review/snapshots.tgz.
Upload the archive from a trusted maintainer environment with STORYBOOK_VISUAL_S3_URI=s3://bucket/baselines/storybook-visual/<sha>/snapshots.tgz pnpm storybook-visual:baseline upload.
Copy the printed snapshotCount and archive fields into baseline-manifest.json.

Generated snapshots, review bundles, Playwright reports, and downloaded caches are ignored by git.

3.0 KiB Raw Blame History