e2e/CI: dual-engine suites, per-engine screenshots, SwiftShader retired, prod web suite, CI-coverage gate

Squash of experiment/ff-big-modules vs main.

Big-module routing removed: native-EH shrank kicad_editor below
SpiderMonkey's x86-64 code budget (runs 29355049705/29356152413 green on
stock Firefox), so BIG_MODULE_SPECS routing and the baseline-only-JIT
crutch are gone — kicad-firefox and kicad-chromium both run the full
suite, with the module compiled the way real users' browsers compile it.

Per-engine screenshots end to end: stableShot/shotPath write
test-results/<engine>/<name>.png; baselines move to
baseline-screenshots/{chromium,firefox}/ and the whole tools/screenshots
pipeline (compare/promote/manifest/spec-map/changelog/Discord) keys on
<engine>/<name>. Previously Firefox and Chromium renders of one spec
overwrote each other and Firefox renders were never actually gated.
Seeded from CI run 29421380806 (92 new firefox baselines, +24 chromium
web-suite shots); manifest generated from the baseline tree.

One merged playwright.config.ts (kicad/asyncify/coroutine/perf as
projects); ~25 dead npm scripts dropped. The web suite is gated in CI for
the first time ever (4 rotted specs fixed, 5 broken lib-bridge specs
triaged as fixme in docs/features/web-e2e-rot/); cheap lint step after
npm ci; last 26 blind-sleep violations fixed.

SwiftShader retired: CI Chromium renders WebGL on ANGLE → Mesa llvmpipe
(--use-gl=angle --use-angle=gl --ignore-gpu-blocklist; the blocklist flag
is mandatory — llvmpipe is blocklisted and WebGL is silently unavailable
without it) in BOTH configs. Under WORKERS=4 congestion SwiftShader
transiently failed the first post-board-load draw and the recovery
cascade ended in a silent permanent Cairo fallback — that engine flip was
the "~1.2% changedRatio both directions" occ-export baseline flake.
Validated 160/160 across two 80-repeat rigs; full analysis in
docs/features/wx-parity-bugs/occ-export-context-eviction.md. Chromium
baselines shift slightly on llvmpipe — promote once from the first green
run. Deflakes the new coverage exposed: presence baselines settle before
capture; presence fixtures declare current file formats; perf gets its
own outputDir so CI evidence survives; occ-export settles the board paint
before the export dialog; menu-item waits (waitForRenderedByLabel before
clickMenuItem) in 4 specs + the TESTING.md rule.

Web suite runs the PROD build, in parallel: webServer becomes backend
`start` + the standalone's e2e:preview (build-preview.mjs: link-wasm →
stash the public/wasm symlink aside during vite build, build-demo.mjs's
move — then vite preview as the persistent server). The wasm middleware
serves /wasm/* in preview and emits COOP/COEP/CORP itself (a pthread
worker script's own response must carry COEP or Chrome kills it with
ERR_BLOCKED_BY_RESPONSE). VITE_* flags bake at build time;
VITE_ALLOW_USER_OVERRIDE joins turbo globalEnv. fullyParallel + default
workers: 5.2m → 1.4m. Determinism fixes the parallel run exposed:
shared-page specs become serial groups; locks.spec grabs alice's exact
item via the new kicadCollabTestSelectByUuid hook (cross-tab "first
footprint" order is not a ysync invariant); quit specs poll page.url()
(quit supersedes its own navigation — NS_BINDING_ABORTED on Firefox).
Suite: 51 passed / 12 skipped / 0 failed in 1.6m.

CI-coverage gate (lint:ci-coverage): every tests/**/*.spec.ts must be
reachable from the npm scripts the workflows invoke — scraped from
.github/workflows/, resolved through package.json, coverage asked from
playwright --list itself. Rules: uncovered-spec + orphan-project (with a
documented LOCAL_ONLY_PROJECTS allowlist). Gating next to
lint:determinism; 138 spec files / 13 projects accounted for.

Product fixes kept from the investigations (reachable on real GPUs too):
wx 7799fd1be5 — paint flags clear before dispatch + Invalidate always
propagates; kicad 3dcfea5e45 — SwiftShader pass-boundary flush +
per-instance font texture + first-frame GL-error drain (GAL recovery
recovers instead of falling back to Cairo) + the user-facing eeschema
switch navigates again under __EMSCRIPTEN__ (project-sync's
FaceRegistered gate had rerouted it into the hidden sync player; caught
by the newly-gated web suite).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018eUxiPApHgGiu9NFyQfhAq
This commit is contained in:
Viktor Vaczi 2026-07-17 12:10:40 +02:00
commit 63ed1f3c1f
556 changed files with 2470 additions and 1389 deletions

View file

@ -1,9 +1,14 @@
import { test, expect, devices, type Page, type Browser } from '@playwright/test';
import { openOverlayMenu } from './overlay-menu';
import { shotPath, settledShot } from '../e2e/utils/element-tracker';
// One shared page across the checks (beforeAll boots pcbnew ONCE — each boot
// costs minutes). Serial group: fullyParallel would re-boot per test.
test.describe.configure({ mode: 'serial' });
/**
* Mobile canvas-only mode e2e (features/mobile) — runs under the
* `mobile-chromium` project (Pixel 7 emulation, system Chrome).
* `web-mobile` project (Pixel 7 emulation on the bundled Chromium).
*
* Boots pcbnew ONCE on the demo board with `?mobile=1` (the wasm runtime is
* process-global and each boot costs minutes, so the four checks share one
@ -83,7 +88,8 @@ async function fingerDrag(
const x = from.x + ((to.x - from.x) * i) / steps;
const y = from.y + ((to.y - from.y) * i) / steps;
await touch(pg, 'touchmove', [{ id: 1, x, y }]);
await pg.waitForTimeout(30);
// The pacing IS the input: it gives the gesture a finger-plausible velocity.
await pg.waitForTimeout(30); // eslint-disable-line -- documented interaction dwell: gesture pacing
}
await touch(pg, 'touchend', [], [{ id: 1, x: to.x, y: to.y }]);
}
@ -104,7 +110,8 @@ async function pinch(
for (let i = 1; i <= steps; i++) {
const s = spreadFrom + ((spreadTo - spreadFrom) * i) / steps;
await touch(pg, 'touchmove', at(s));
await pg.waitForTimeout(30);
// The pacing IS the input: it gives the gesture a finger-plausible velocity.
await pg.waitForTimeout(30); // eslint-disable-line -- documented interaction dwell: gesture pacing
}
await touch(pg, 'touchend', [], at(spreadTo));
}
@ -176,7 +183,14 @@ async function diffRatio(
}
const shot = (name: string) =>
page.screenshot({ path: `test-results/mobile-${name}.png`, scale: 'css' });
page.screenshot({ path: shotPath(page, `mobile-${name}.png`), scale: 'css' });
/** Settle the render (two consecutive identical canvas screenshots), then capture
* the named gate shot — the deterministic replacement for sleep-then-shoot. */
const settledCapture = async (name: string): Promise<Buffer> => {
await settledShot(page.locator('#canvas'), expect);
return shot(name);
};
test.beforeAll(async ({ browser }: { browser: Browser }) => {
test.setTimeout(420_000); // cold load: 136 MB wasm download + compile
@ -193,8 +207,9 @@ test.beforeAll(async ({ browser }: { browser: Browser }) => {
// in WasmTool) must be GONE — the gesture tests diff screenshots, and an
// overlay screenshot diffs to zero (learned the hard way on a cold load).
await expect(page.locator('div.inset-0.z-30')).toHaveCount(0, { timeout: 180000 });
// let the first paint + fit-to-view and the chrome-hide poll settle
await page.waitForTimeout(4000);
// First paint + fit-to-view + the chrome-hide poll: settled when two
// consecutive canvas screenshots are byte-identical.
await settledShot(page.locator('#canvas'), expect);
});
test.afterAll(async () => {
@ -212,9 +227,19 @@ test('tap contract: a touch tap synthesizes a LEFT click, no phantom middle-drag
const p = { id: 1, x: box.x + box.width * 0.5, y: box.y + box.height * 0.5 };
await touch(page, 'touchstart', [p]);
await page.waitForTimeout(60);
// A tap's physical duration — instantaneous taps can be filtered as noise.
await page.waitForTimeout(60); // eslint-disable-line -- documented interaction dwell: tap duration
await touch(page, 'touchend', [], [p]);
await page.waitForTimeout(300);
// The synthesized click is the observable: wait for the mouseup to land in the log.
await expect
.poll(
() =>
page.evaluate(
() => (window as unknown as { __mouseLog: [string, number][] }).__mouseLog.length,
),
{ timeout: 10000, message: 'tap never synthesized any mouse events' },
)
.toBeGreaterThanOrEqual(2);
const log = await page.evaluate(
() => (window as unknown as { __mouseLog: [string, number][] }).__mouseLog,
@ -237,14 +262,12 @@ test('pinch: two-finger pinch-out zooms in, pinch-in restores', async () => {
const box = await getGlBox(page);
const centre = { x: box.x + box.width * 0.5, y: box.y + box.height * 0.45 };
const base = await shot('pinch-00-base');
const base = await settledCapture('pinch-00-base');
await pinch(page, centre, 40, 140); // pinch OUT = zoom in
await page.waitForTimeout(500);
const zoomed = await shot('pinch-01-zoomed');
const zoomed = await settledCapture('pinch-01-zoomed');
await pinch(page, centre, 140, 40); // pinch IN at the same centre = zoom back out
await page.waitForTimeout(500);
const restored = await shot('pinch-02-restored');
const restored = await settledCapture('pinch-02-restored');
const dIn = await diffRatio(page, base, zoomed, box);
const dBack = await diffRatio(page, base, restored, box);
@ -264,14 +287,12 @@ test('pan: one-finger drag translates the view (not a rubber-band select)', asyn
const from = { x: box.x + box.width * 0.15, y: box.y + box.height * 0.88 };
const to = { x: box.x + box.width * 0.55, y: box.y + box.height * 0.7 };
const base = await shot('pan-00-base');
const base = await settledCapture('pan-00-base');
await fingerDrag(page, from, to);
await page.waitForTimeout(500);
const panned = await shot('pan-01-panned');
const panned = await settledCapture('pan-01-panned');
await fingerDrag(page, to, from); // drag back — translation is invertible
await page.waitForTimeout(500);
const restored = await shot('pan-02-restored');
const restored = await settledCapture('pan-02-restored');
const dPan = await diffRatio(page, base, panned, box);
const dBack = await diffRatio(page, base, restored, box);