fix(runner): re-record the Phase 7 UI matrix under the Inter font environment
The committed matrix was captured while fontconfig resolved the sans stack to the DejaVu Sans fallback; ambient user-font churn then made Inter (the stack's first choice) resolve, drifting all 24 shots at once. Re-recorded under Chromium 151.0.7922.34 with Inter active, taught the recorder to refuse to record or compare when Inter does not resolve distinctly, and wrote the recording environment into the generated evidence index. PAP-17001 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
No files matched your search
@@ -168,11 +168,17 @@ Hosts without the Playwright chromium system libraries can either run
|
||||
`pnpm --filter @paperclipai/paperclip-runner verify:rootless` or set
|
||||
`PAPERCLIP_RUNNER_CHROMIUM_PATH` to a preinstalled Chromium.
|
||||
|
||||
The committed PNGs are pinned to the Chromium build listed in the Phase 7G
|
||||
verification evidence, so `check:phase7:ui` needs that same browser. Point
|
||||
`PAPERCLIP_RUNNER_CHROMIUM_PATH` at it before comparing; a different build makes
|
||||
every shot mismatch at once, which reads like a regression but is not one. The
|
||||
check prints the Chromium version it recorded with whenever it reports drift.
|
||||
The committed PNGs are pinned to the Chromium build and font environment listed
|
||||
in `knowledge/evidence/phase-07/ui/index.md`, so `check:phase7:ui` needs that
|
||||
same browser and font set. Point `PAPERCLIP_RUNNER_CHROMIUM_PATH` at the
|
||||
recorded browser before comparing — and when that path is the agent-browser
|
||||
wrapper, also set `PAPERCLIP_CHROMIUM_BIN` to the exact binary, because the
|
||||
wrapper otherwise picks the newest installed Playwright Chromium. A different
|
||||
build or font set makes every shot mismatch at once, which reads like a
|
||||
regression but is not one. The recorder probes that `Inter` resolves distinctly
|
||||
from the DejaVu Sans fallback and refuses to record or compare when it does
|
||||
not; the drift report prints the Chromium version and font probe it recorded
|
||||
with.
|
||||
|
||||
## Accessibility
|
||||
|
||||
|
||||
|
Before Width: | Height: | Size: 96 KiB After Width: | Height: | Size: 104 KiB |
|
Before Width: | Height: | Size: 39 KiB After Width: | Height: | Size: 41 KiB |
|
Before Width: | Height: | Size: 67 KiB After Width: | Height: | Size: 72 KiB |
|
Before Width: | Height: | Size: 45 KiB After Width: | Height: | Size: 49 KiB |
|
Before Width: | Height: | Size: 65 KiB After Width: | Height: | Size: 71 KiB |
|
Before Width: | Height: | Size: 43 KiB After Width: | Height: | Size: 48 KiB |
|
Before Width: | Height: | Size: 65 KiB After Width: | Height: | Size: 70 KiB |
|
Before Width: | Height: | Size: 43 KiB After Width: | Height: | Size: 47 KiB |
|
Before Width: | Height: | Size: 65 KiB After Width: | Height: | Size: 71 KiB |
|
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 46 KiB |
@@ -27,3 +27,12 @@ Deterministic `fake` mode; no provider, runner, or credential is involved.
|
||||
- `thread-baseline--mobile.png`
|
||||
- `turn-streaming--desktop.png`
|
||||
- `turn-streaming--mobile.png`
|
||||
|
||||
## Recording environment
|
||||
|
||||
- Chromium 151.0.7922.34 — pin the exact binary with `PAPERCLIP_CHROMIUM_BIN`
|
||||
(honoured by the agent-browser wrapper) alongside
|
||||
`PAPERCLIP_RUNNER_CHROMIUM_PATH`.
|
||||
- Sans stack resolves to Inter (probe: Inter 549px vs
|
||||
DejaVu Sans 641px for the title string). The recorder
|
||||
refuses to run when Inter does not resolve.
|
||||
|
Before Width: | Height: | Size: 59 KiB After Width: | Height: | Size: 65 KiB |
|
Before Width: | Height: | Size: 38 KiB After Width: | Height: | Size: 41 KiB |
|
Before Width: | Height: | Size: 57 KiB After Width: | Height: | Size: 62 KiB |
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 39 KiB |
|
Before Width: | Height: | Size: 57 KiB After Width: | Height: | Size: 62 KiB |
|
Before Width: | Height: | Size: 35 KiB After Width: | Height: | Size: 38 KiB |
|
Before Width: | Height: | Size: 63 KiB After Width: | Height: | Size: 69 KiB |
|
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 45 KiB |
|
Before Width: | Height: | Size: 65 KiB After Width: | Height: | Size: 71 KiB |
|
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 64 KiB After Width: | Height: | Size: 70 KiB |
|
Before Width: | Height: | Size: 43 KiB After Width: | Height: | Size: 47 KiB |
|
Before Width: | Height: | Size: 67 KiB After Width: | Height: | Size: 73 KiB |
|
Before Width: | Height: | Size: 45 KiB After Width: | Height: | Size: 49 KiB |
@@ -81,6 +81,35 @@ async function startPreview() {
|
||||
|
||||
// Set by `record` so a drift report can name the Chromium build that produced it.
|
||||
let browserVersion = "unknown";
|
||||
let fontProbe = { interWidth: 0, dejavuWidth: 0, interResolves: false };
|
||||
|
||||
/**
|
||||
* The committed matrix is recorded with Inter active. When fontconfig cannot
|
||||
* resolve Inter, every text node falls back to DejaVu Sans and all 24 shots
|
||||
* drift at once — refuse to record or compare under that environment instead
|
||||
* of producing a whole-page mystery diff.
|
||||
*/
|
||||
async function probeFonts(browser) {
|
||||
const context = await browser.newContext();
|
||||
const page = await context.newPage();
|
||||
const probe = await page.evaluate(() => {
|
||||
const canvas = document.createElement("canvas").getContext("2d");
|
||||
const sample = "Wire the runner spike to the mock control plane";
|
||||
const width = (font) => {
|
||||
canvas.font = font;
|
||||
return Math.round(canvas.measureText(sample).width);
|
||||
};
|
||||
const interWidth = width("700 24px Inter");
|
||||
const dejavuWidth = width('700 24px "DejaVu Sans"');
|
||||
return {
|
||||
interWidth,
|
||||
dejavuWidth,
|
||||
interResolves: document.fonts.check("16px Inter") && interWidth !== dejavuWidth,
|
||||
};
|
||||
});
|
||||
await context.close();
|
||||
return probe;
|
||||
}
|
||||
|
||||
async function record(targetDir) {
|
||||
await mkdir(targetDir, { recursive: true });
|
||||
@@ -92,6 +121,15 @@ async function record(targetDir) {
|
||||
browserVersion = browser.version();
|
||||
const recorded = [];
|
||||
try {
|
||||
fontProbe = await probeFonts(browser);
|
||||
if (!fontProbe.interResolves) {
|
||||
throw new Error(
|
||||
`font environment mismatch: "Inter" does not resolve distinctly ` +
|
||||
`(Inter ${fontProbe.interWidth}px vs DejaVu Sans ${fontProbe.dejavuWidth}px). ` +
|
||||
"The committed matrix is recorded with Inter active; make Inter visible to " +
|
||||
"fontconfig before recording or checking.",
|
||||
);
|
||||
}
|
||||
for (const viewport of VIEWPORTS) {
|
||||
for (const slug of PHASE7_UI_SHOT_SLUGS) {
|
||||
const context = await browser.newContext({
|
||||
@@ -146,6 +184,15 @@ async function main() {
|
||||
"",
|
||||
...manifest.map((name) => `- \`${name}\``),
|
||||
"",
|
||||
"## Recording environment",
|
||||
"",
|
||||
`- Chromium ${browserVersion} — pin the exact binary with \`PAPERCLIP_CHROMIUM_BIN\``,
|
||||
" (honoured by the agent-browser wrapper) alongside",
|
||||
" `PAPERCLIP_RUNNER_CHROMIUM_PATH`.",
|
||||
`- Sans stack resolves to Inter (probe: Inter ${fontProbe.interWidth}px vs`,
|
||||
` DejaVu Sans ${fontProbe.dejavuWidth}px for the title string). The recorder`,
|
||||
" refuses to run when Inter does not resolve.",
|
||||
"",
|
||||
].join("\n"),
|
||||
"utf8",
|
||||
);
|
||||
@@ -183,10 +230,12 @@ async function main() {
|
||||
"Phase 7 UI evidence is not byte-stable:",
|
||||
...drift,
|
||||
"",
|
||||
`Recorded with Chromium ${browserVersion}.`,
|
||||
"Committed PNGs are pinned to one Chromium build. A wholesale mismatch",
|
||||
"usually means a different browser, not a UI regression: set",
|
||||
"PAPERCLIP_RUNNER_CHROMIUM_PATH to the Chromium that recorded them.",
|
||||
`Recorded with Chromium ${browserVersion}; sans probe Inter=${fontProbe.interWidth}px, DejaVu Sans=${fontProbe.dejavuWidth}px.`,
|
||||
"Committed PNGs are pinned to one Chromium build and one font environment.",
|
||||
"A wholesale mismatch usually means a different browser or font set, not a",
|
||||
"UI regression: set PAPERCLIP_RUNNER_CHROMIUM_PATH (and PAPERCLIP_CHROMIUM_BIN",
|
||||
"for the wrapper) to the Chromium that recorded them, and compare the font",
|
||||
"probe against knowledge/evidence/phase-07/ui/index.md.",
|
||||
"",
|
||||
].join("\n"),
|
||||
);
|
||||
|
||||