fix(runner): re-record the Phase 7 UI matrix under the Inter font environment

The committed matrix was captured while fontconfig resolved the sans stack
to the DejaVu Sans fallback; ambient user-font churn then made Inter (the
stack's first choice) resolve, drifting all 24 shots at once. Re-recorded
under Chromium 151.0.7922.34 with Inter active, taught the recorder to
refuse to record or compare when Inter does not resolve distinctly, and
wrote the recording environment into the generated evidence index.

PAP-17001

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
DottaandClaude Fable 5 committed 2026-08-10 22:57:07 +00:00
1 parent b0f8b1e91e
commit 31b1e228b3
27 files changed
+73 -9

No files matched your search

@@ -168,11 +168,17 @@ Hosts without the Playwright chromium system libraries can either run
`pnpm --filter @paperclipai/paperclip-runner verify:rootless` or set
`PAPERCLIP_RUNNER_CHROMIUM_PATH` to a preinstalled Chromium.
The committed PNGs are pinned to the Chromium build listed in the Phase 7G
verification evidence, so `check:phase7:ui` needs that same browser. Point
`PAPERCLIP_RUNNER_CHROMIUM_PATH` at it before comparing; a different build makes
every shot mismatch at once, which reads like a regression but is not one. The
check prints the Chromium version it recorded with whenever it reports drift.
The committed PNGs are pinned to the Chromium build and font environment listed
in `knowledge/evidence/phase-07/ui/index.md`, so `check:phase7:ui` needs that
same browser and font set. Point `PAPERCLIP_RUNNER_CHROMIUM_PATH` at the
recorded browser before comparing — and when that path is the agent-browser
wrapper, also set `PAPERCLIP_CHROMIUM_BIN` to the exact binary, because the
wrapper otherwise picks the newest installed Playwright Chromium. A different
build or font set makes every shot mismatch at once, which reads like a
regression but is not one. The recorder probes that `Inter` resolves distinctly
from the DejaVu Sans fallback and refuses to record or compare when it does
not; the drift report prints the Chromium version and font probe it recorded
with.
## Accessibility
Binary file not shown.

Before

Width:  |  Height:  |  Size: 96 KiB

After

Width:  |  Height:  |  Size: 104 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 39 KiB

After

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 67 KiB

After

Width:  |  Height:  |  Size: 72 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 45 KiB

After

Width:  |  Height:  |  Size: 49 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 65 KiB

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 43 KiB

After

Width:  |  Height:  |  Size: 48 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 65 KiB

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 43 KiB

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 65 KiB

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 42 KiB

After

Width:  |  Height:  |  Size: 46 KiB

@@ -27,3 +27,12 @@ Deterministic `fake` mode; no provider, runner, or credential is involved.
- `thread-baseline--mobile.png`
- `turn-streaming--desktop.png`
- `turn-streaming--mobile.png`
## Recording environment
- Chromium 151.0.7922.34 — pin the exact binary with `PAPERCLIP_CHROMIUM_BIN`
(honoured by the agent-browser wrapper) alongside
`PAPERCLIP_RUNNER_CHROMIUM_PATH`.
- Sans stack resolves to Inter (probe: Inter 549px vs
DejaVu Sans 641px for the title string). The recorder
refuses to run when Inter does not resolve.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 65 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

After

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 57 KiB

After

Width:  |  Height:  |  Size: 62 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 36 KiB

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 57 KiB

After

Width:  |  Height:  |  Size: 62 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 35 KiB

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 63 KiB

After

Width:  |  Height:  |  Size: 69 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 42 KiB

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 65 KiB

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 42 KiB

After

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 64 KiB

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 43 KiB

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 67 KiB

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 45 KiB

After

Width:  |  Height:  |  Size: 49 KiB

@@ -81,6 +81,35 @@ async function startPreview() {
// Set by `record` so a drift report can name the Chromium build that produced it.
let browserVersion = "unknown";
let fontProbe = { interWidth: 0, dejavuWidth: 0, interResolves: false };
/**
* The committed matrix is recorded with Inter active. When fontconfig cannot
* resolve Inter, every text node falls back to DejaVu Sans and all 24 shots
* drift at once — refuse to record or compare under that environment instead
* of producing a whole-page mystery diff.
*/
async function probeFonts(browser) {
const context = await browser.newContext();
const page = await context.newPage();
const probe = await page.evaluate(() => {
const canvas = document.createElement("canvas").getContext("2d");
const sample = "Wire the runner spike to the mock control plane";
const width = (font) => {
canvas.font = font;
return Math.round(canvas.measureText(sample).width);
};
const interWidth = width("700 24px Inter");
const dejavuWidth = width('700 24px "DejaVu Sans"');
return {
interWidth,
dejavuWidth,
interResolves: document.fonts.check("16px Inter") && interWidth !== dejavuWidth,
};
});
await context.close();
return probe;
}
async function record(targetDir) {
await mkdir(targetDir, { recursive: true });
@@ -92,6 +121,15 @@ async function record(targetDir) {
browserVersion = browser.version();
const recorded = [];
try {
fontProbe = await probeFonts(browser);
if (!fontProbe.interResolves) {
throw new Error(
`font environment mismatch: "Inter" does not resolve distinctly ` +
`(Inter ${fontProbe.interWidth}px vs DejaVu Sans ${fontProbe.dejavuWidth}px). ` +
"The committed matrix is recorded with Inter active; make Inter visible to " +
"fontconfig before recording or checking.",
);
}
for (const viewport of VIEWPORTS) {
for (const slug of PHASE7_UI_SHOT_SLUGS) {
const context = await browser.newContext({
@@ -146,6 +184,15 @@ async function main() {
"",
...manifest.map((name) => `- \`${name}\``),
"",
"## Recording environment",
"",
`- Chromium ${browserVersion} — pin the exact binary with \`PAPERCLIP_CHROMIUM_BIN\``,
" (honoured by the agent-browser wrapper) alongside",
" `PAPERCLIP_RUNNER_CHROMIUM_PATH`.",
`- Sans stack resolves to Inter (probe: Inter ${fontProbe.interWidth}px vs`,
` DejaVu Sans ${fontProbe.dejavuWidth}px for the title string). The recorder`,
" refuses to run when Inter does not resolve.",
"",
].join("\n"),
"utf8",
);
@@ -183,10 +230,12 @@ async function main() {
"Phase 7 UI evidence is not byte-stable:",
...drift,
"",
`Recorded with Chromium ${browserVersion}.`,
"Committed PNGs are pinned to one Chromium build. A wholesale mismatch",
"usually means a different browser, not a UI regression: set",
"PAPERCLIP_RUNNER_CHROMIUM_PATH to the Chromium that recorded them.",
`Recorded with Chromium ${browserVersion}; sans probe Inter=${fontProbe.interWidth}px, DejaVu Sans=${fontProbe.dejavuWidth}px.`,
"Committed PNGs are pinned to one Chromium build and one font environment.",
"A wholesale mismatch usually means a different browser or font set, not a",
"UI regression: set PAPERCLIP_RUNNER_CHROMIUM_PATH (and PAPERCLIP_CHROMIUM_BIN",
"for the wrapper) to the Chromium that recorded them, and compare the font",
"probe against knowledge/evidence/phase-07/ui/index.md.",
"",
].join("\n"),
);