test(screenshots): restore screenshot-neutrality after the determinism refactor

The determinism conversion silently dropped/renamed screenshots vs the committed
baselines, so CI flagged ~35 baselines as "removed". Restore them:

- pl_editor-*: re-add the "pl_editor-" prefix the conversion dropped (5).
- Buffer screenshots that lost their path: (eeschema draw-wires/loaded-css,
  pcbnew draw-lines/loaded-css, pcbnew-move, dark-mode, eeschema-crosshair,
  aui-resize) re-add path:'test-results/<name>' so they SAVE and still feed the
  in-test pixel-diff analysis (17).
- Dropped animating/loading captures re-added via stableShot / raw screenshot:
  timer-03/05/06, dialogs-timer-running/started, wxgrid-02/03, 01-loading (7).

Delete 5 obsolete wizard baselines (eeschema/gerbview/pl_editor/symbol_editor/
generic -wizard-00-initial): every app is now seeded to bypass the first-run
setup wizard (that was the point of removing the wizard for/if loop), so those
states are no longer produced — pl_editor.spec asserts "no wizard appears"
instead.

Net: every committed baseline has a producer again (0 "removed"); determinism
guard green; affected specs pass (e2e 40, kicad 12) and emit the restored PNGs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BVX1pHMvRPYHdp6ZfEawrk
This commit is contained in:
Viktor Vaczi 2026-07-07 11:35:39 +02:00
commit c1eb08108b
16 changed files with 47 additions and 23 deletions

Binary file not shown.

Before

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 64 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 104 KiB

View file

@ -242,11 +242,11 @@ test.describe('wxAuiManager dock-sash resize UX (pcbjam#20)', () => {
// intermediate frame while the button is still held.
await page.mouse.move(sash.x - 90, sash.y, { steps: 10 });
await page.waitForTimeout(70); // eslint-disable-line -- documented interaction dwell: let the mid-drag frame paint before capturing it for the pixel-diff
const midShot = await page.screenshot({ scale: 'css' });
const midShot = await page.screenshot({ path: 'test-results/aui-resize-03-mid-drag.png', scale: 'css' });
await page.mouse.up();
await page.waitForTimeout(200); // eslint-disable-line -- documented interaction dwell: let the release settle before capturing the after frame for the pixel-diff
const afterShot = await page.screenshot({ scale: 'css' });
const afterShot = await page.screenshot({ path: 'test-results/aui-resize-04-after.png', scale: 'css' });
const midFrac = await stippleFraction(page, midShot, region);
const afterFrac = await stippleFraction(page, afterShot, region);

View file

@ -134,11 +134,13 @@ test.describe('Dialogs Tab Tests', () => {
await stableShot(page, 'dialogs-timer-initial.png', { fullPage: true });
// Click "Start Timer" button using element registry. No screenshot of the running
// timer: the label increments once a second, so a running frame never holds still
// for stableShot to stabilise and it asserted nothing (started/running dropped).
// Click "Start Timer" button using element registry.
await clickByLabel(page, 'Start Timer');
await stableShot(page, 'dialogs-timer-started.png', { fullPage: true });
await stableShot(page, 'dialogs-timer-running.png', { fullPage: true });
// Click "Stop Timer" (Start/Stop are separate always-present buttons; the queued
// button events run in order, so no dwell is needed between them).
await clickByLabel(page, 'Stop Timer');

View file

@ -144,6 +144,8 @@ test.describe('Grid Tab Tests', () => {
// Click on a cell (would be at ~y=175 if grid existed)
await page.mouse.click(box.x + 180, box.y + 175);
await stableShot(page, 'wxgrid-02-selection.png', { fullPage: true });
// The click's effect is the cell-selection console event — poll for it
// (replaces the 200ms sleep + hasGridEvent assertion).
await expect.poll(() => testLogger.consoleLogs.some(l => l.includes('Grid cell')),
@ -165,6 +167,8 @@ test.describe('Grid Tab Tests', () => {
await page.keyboard.type('1.5');
await page.keyboard.press('Enter');
await stableShot(page, 'wxgrid-03-editing.png', { fullPage: true });
// The edit's effect is the cell-changed console event — poll for it
// (replaces the 200ms sleep + hasEditEvent assertion).
await expect.poll(() => testLogger.consoleLogs.some(l => l.includes('Grid cell') && l.includes('changed')),

View file

@ -39,6 +39,7 @@ test.describe('wxTimer Tests', () => {
// replaces waitForTimeout(1500)).
await expect.poll(() => testLogger.consoleLogs.some(l => l.includes('Slow timer tick')),
{ message: 'slow timer should tick', timeout: 8000 }).toBe(true);
await stableShot(page, 'timer-03-ticked.png', { fullPage: true });
// Click Stop for the slow timer.
const stopButton = await findByLabel(page, 'Stop', { exact: true });
@ -57,12 +58,14 @@ test.describe('wxTimer Tests', () => {
await clickByLabel(page, 'Start Fast');
await expect.poll(() => testLogger.consoleLogs.some(l => l.includes('Fast timer started')),
{ message: 'Should log fast timer started' }).toBe(true);
await stableShot(page, 'timer-05-fast-started.png', { fullPage: true });
// Let the fast timer tick (deterministic). No screenshot while it runs: the fast
// timer continuously animates the gauge, so no two frames are ever identical and
// stableShot can't stabilize — and the original mid-run shots asserted nothing.
// Let the fast timer tick (deterministic). The fast timer continuously animates the
// gauge, so this mid-run frame is timing-dependent; stableShot just captures a frame
// for the offline gate (non-asserting) rather than trying to stabilize it.
await expect.poll(() => testLogger.consoleLogs.some(l => l.includes('Fast timer tick')),
{ message: 'fast timer should tick', timeout: 8000 }).toBe(true);
await stableShot(page, 'timer-06-fast-running.png', { fullPage: true });
await clickByLabel(page, 'Stop Fast');
await expect.poll(() => testLogger.consoleLogs.some(l => l.includes('Fast timer stopped')),

View file

@ -67,6 +67,10 @@ test.describe('wxWidgets WASM - Diagnostics', () => {
await page.goto('/minimal_test.html');
// Loading-state capture: raw + immediate (no settle) on purpose — this shot documents the
// app mid-load, before the canvas is up, so it must NOT wait for the render to finish.
await page.screenshot({ path: 'test-results/01-loading.png', fullPage: true });
// Deterministic app readiness: canvas visible + wx registry populated.
// Fails loudly (replaces the try/catch waitForSelector + 1s settle).
await waitForWxApp(page);

View file

@ -38,7 +38,10 @@ test.describe('PCBnew dark-mode browser', () => {
await waitForEditorReady(page);
await hideCursor(page);
const cssScreenshot = await page.screenshot({ scale: 'css' });
const cssScreenshot = await page.screenshot({
path: 'test-results/pcbnew-dark-mode-loaded.png',
scale: 'css'
});
const reference = await compareToReference(page, cssScreenshot, PCBNEW_REFERENCE, PCBNEW_HEADER_REGION);

View file

@ -204,7 +204,7 @@ test.describe('Eeschema crosshair modes', () => {
await page.mouse.move(probe.x, probe.y);
await waitForCanvasStable(page, box.sel);
const shotSmall = await page.screenshot({ scale: 'css' });
const shotSmall = await page.screenshot({ path: 'test-results/eeschema-crosshair-00-small.png', scale: 'css' });
// click 1 -> full-window
await clickAndSettle();
@ -212,7 +212,7 @@ test.describe('Eeschema crosshair modes', () => {
message: 'one click should advance to Full-Window Crosshairs', timeout: 6000,
}).toContain('Full-Window Crosshairs');
await waitForCanvasStable(page, box.sel);
const shotFull = await page.screenshot({ scale: 'css' });
const shotFull = await page.screenshot({ path: 'test-results/eeschema-crosshair-01-full.png', scale: 'css' });
expect((await compareScreenshots(page, shotSmall, shotFull, diffRegion)).diffPixels,
'full-window crosshair should visibly differ from the small crosshair').toBeGreaterThan(200);
@ -222,7 +222,7 @@ test.describe('Eeschema crosshair modes', () => {
message: 'second click should advance to 45 Degree Crosshairs', timeout: 6000,
}).toContain('45 Degree Crosshairs');
await waitForCanvasStable(page, box.sel);
const shot45 = await page.screenshot({ scale: 'css' });
const shot45 = await page.screenshot({ path: 'test-results/eeschema-crosshair-02-45.png', scale: 'css' });
expect((await compareScreenshots(page, shotFull, shot45, diffRegion)).diffPixels,
'45-degree crosshair should visibly differ from the full-window crosshair').toBeGreaterThan(200);
@ -231,6 +231,8 @@ test.describe('Eeschema crosshair modes', () => {
await expect.poll(tooltipNow, {
message: 'third click should cycle back to Small crosshairs', timeout: 6000,
}).toContain('Small crosshairs');
await waitForCanvasStable(page, box.sel);
await page.screenshot({ path: 'test-results/eeschema-crosshair-03-small-again.png', scale: 'css' });
const realErrors = testLogger.errors.filter((error: string) => !error.includes('favicon'));
expect(realErrors).toEqual([]);

View file

@ -170,6 +170,7 @@ test.describe('Eeschema WASM', () => {
expect(viewport![3]).toBe(glCanvas.height);
await hideCursor(page);
await stableShot(page, 'eeschema-loaded-css.png');
await stableShot(page, 'eeschema-loaded.png');
const canvasCount = await page.locator('canvas').count();
@ -203,6 +204,8 @@ test.describe('Eeschema WASM', () => {
expect(isToolChecked(drawWiresTool)).toBe(false);
const baselineErrorCount = testLogger.errors.length;
await stableShot(page, 'eeschema-draw-wires-00-before-tool-click.png');
// Select the tool and confirm it latches checked.
expect(await clickByTooltip(page, 'Draw Wires', { elementType: 'tool' })).toBe(true);
await expect.poll(async () => isToolChecked(await findByTooltip(page, 'Draw Wires', { elementType: 'tool' })), {
@ -242,7 +245,7 @@ test.describe('Eeschema WASM', () => {
// crosshair IS a visible change, so this settle is deterministic.
await page.mouse.move(640, 360);
await waitForCanvasStable(page, glSel);
const afterToolClick = await page.screenshot({ scale: 'css' });
const afterToolClick = await page.screenshot({ path: 'test-results/eeschema-draw-wires-01-after-click.png', scale: 'css' });
// Draw a wire: click a start vertex, then an end vertex. These two waits are the
// ONE place in the converted suite that still uses a fixed delay: a wire vertex
@ -258,7 +261,7 @@ test.describe('Eeschema WASM', () => {
await page.mouse.click(endPoint.x, endPoint.y);
await page.waitForTimeout(750); // eslint-disable-line -- see comment above
const afterDrawing = await page.screenshot({ scale: 'css' });
const afterDrawing = await page.screenshot({ path: 'test-results/eeschema-draw-wires-02-after-drawing.png', scale: 'css' });
const diffRegion: DiffRegion = {
x: Math.max(0, Math.min(startPoint.x, endPoint.x) - 24),

View file

@ -142,7 +142,7 @@ test.describe('PCBnew move with "m" (#9)', () => {
const drawnId = newItems[0].id;
const pos0 = await getPos(page, drawnId);
const beforeMove = await page.screenshot({ scale: 'css' });
const beforeMove = await page.screenshot({ path: 'test-results/pcbnew-move-00-before.png', scale: 'css' });
// Hover onto the line, select it, press m, nudge right, commit with Enter. These
// are documented interaction dwells: selection, move-mode entry, and per-arrow
@ -166,7 +166,7 @@ test.describe('PCBnew move with "m" (#9)', () => {
await page.keyboard.press('Enter');
await page.waitForTimeout(500); // eslint-disable-line -- documented interaction dwell
const afterMove = await page.screenshot({ scale: 'css' });
const afterMove = await page.screenshot({ path: 'test-results/pcbnew-move-01-after.png', scale: 'css' });
const pos1 = await getPos(page, drawnId);
const dx = pos1.x - pos0.x;

View file

@ -192,6 +192,7 @@ test.describe('PCBnew WASM', () => {
expect(appearancePane!.width).toBeLessThanOrEqual(240);
await hideCursor(page);
await stableShot(page, 'pcbnew-loaded-css.png');
await stableShot(page, 'pcbnew-loaded.png');
const canvasCount = await page.locator('canvas').count();
@ -221,6 +222,8 @@ test.describe('PCBnew WASM', () => {
expect(isToolChecked(drawLinesTool)).toBe(false);
const baselineErrorCount = testLogger.errors.length;
await stableShot(page, 'pcbnew-draw-lines-00-before-tool-click.png');
expect(await clickByTooltip(page, 'Draw Lines', { elementType: 'tool' })).toBe(true);
await expect.poll(async () => isToolChecked(await findByTooltip(page, 'Draw Lines', { elementType: 'tool' })), {
message: 'Draw Lines tool should stay selected after the click',
@ -248,7 +251,7 @@ test.describe('PCBnew WASM', () => {
// Move the crosshair onto the canvas (a visible change → deterministic settle).
await page.mouse.move(640, 360);
await waitForCanvasStable(page, glSel);
const afterToolClick = await page.screenshot({ scale: 'css' });
const afterToolClick = await page.screenshot({ path: 'test-results/pcbnew-draw-lines-01-after-click.png', scale: 'css' });
const startPoint = { x: Math.round(box.x + box.width * 0.28), y: Math.round(box.y + box.height * 0.36) };
const endPoint = { x: Math.round(box.x + box.width * 0.48), y: Math.round(box.y + box.height * 0.47) };
@ -272,7 +275,7 @@ test.describe('PCBnew WASM', () => {
await page.mouse.up();
await page.waitForTimeout(750); // eslint-disable-line -- see comment above
const afterDrawing = await page.screenshot({ scale: 'css' });
const afterDrawing = await page.screenshot({ path: 'test-results/pcbnew-draw-lines-02-after-drawing.png', scale: 'css' });
const diffRegion: DiffRegion = {
x: Math.max(0, Math.min(startPoint.x, endPoint.x) - 24),

View file

@ -32,7 +32,7 @@ test.describe('pl_editor WASM', () => {
test('app loads, canvas visible, no WASM abort', async ({ page, testLogger }) => {
await waitForEditorReady(page);
await stableShot(page, '01-loaded.png');
await stableShot(page, 'pl_editor-01-loaded.png');
expect(hasAbort(testLogger), 'no WASM abort during load').toBe(false);
@ -56,7 +56,7 @@ test.describe('pl_editor WASM', () => {
expect(blockingDialogs, 'no setup wizard/dialog should be visible (seed skipped it)').toBe(0);
expect(hasAbort(testLogger), 'no WASM abort during launch').toBe(false);
await stableShot(page, '02-no-wizard.png');
await stableShot(page, 'pl_editor-02-no-wizard.png');
});
test('File menu exposes Open... and Save As...', async ({ page, testLogger }) => {
@ -78,7 +78,7 @@ test.describe('pl_editor WASM', () => {
'File menu items rendered',
);
await stableShot(page, '03-file-menu.png');
await stableShot(page, 'pl_editor-03-file-menu.png');
const menuLabels = await page.evaluate(() => {
const registry = window.wxElementRegistry!;
@ -128,7 +128,7 @@ test.describe('pl_editor WASM', () => {
// the inner file list isn't painted. stableShot's stabilization waits for the
// list to finish painting — deterministically replacing the old waitForTimeout(600)
// that used to catch the dialog as a black rectangle.
await stableShot(page, '04-save-as-dialog.png');
await stableShot(page, 'pl_editor-04-save-as-dialog.png');
// The bug: pressing Enter on a folder name treated it as a file and surfaced
// "Unable to load /dev file". After the OnOk fix, the dialog should navigate
@ -136,7 +136,7 @@ test.describe('pl_editor WASM', () => {
await page.keyboard.type('/dev');
await page.keyboard.press('Enter');
await stableShot(page, '04b-after-enter.png');
await stableShot(page, 'pl_editor-04b-after-enter.png');
// The wxFileDialog should still be visible — we navigated into /dev, didn't close it.
const dialogStillOpen = await page.evaluate(() => {