e2e/CI: dual-engine suites, per-engine screenshots, SwiftShader retired, prod web suite, CI-coverage gate
Squash of experiment/ff-big-modules vs main.
Big-module routing removed: native-EH shrank kicad_editor below
SpiderMonkey's x86-64 code budget (runs 29355049705/29356152413 green on
stock Firefox), so BIG_MODULE_SPECS routing and the baseline-only-JIT
crutch are gone — kicad-firefox and kicad-chromium both run the full
suite, with the module compiled the way real users' browsers compile it.
Per-engine screenshots end to end: stableShot/shotPath write
test-results/<engine>/<name>.png; baselines move to
baseline-screenshots/{chromium,firefox}/ and the whole tools/screenshots
pipeline (compare/promote/manifest/spec-map/changelog/Discord) keys on
<engine>/<name>. Previously Firefox and Chromium renders of one spec
overwrote each other and Firefox renders were never actually gated.
Seeded from CI run 29421380806 (92 new firefox baselines, +24 chromium
web-suite shots); manifest generated from the baseline tree.
One merged playwright.config.ts (kicad/asyncify/coroutine/perf as
projects); ~25 dead npm scripts dropped. The web suite is gated in CI for
the first time ever (4 rotted specs fixed, 5 broken lib-bridge specs
triaged as fixme in docs/features/web-e2e-rot/); cheap lint step after
npm ci; last 26 blind-sleep violations fixed.
SwiftShader retired: CI Chromium renders WebGL on ANGLE → Mesa llvmpipe
(--use-gl=angle --use-angle=gl --ignore-gpu-blocklist; the blocklist flag
is mandatory — llvmpipe is blocklisted and WebGL is silently unavailable
without it) in BOTH configs. Under WORKERS=4 congestion SwiftShader
transiently failed the first post-board-load draw and the recovery
cascade ended in a silent permanent Cairo fallback — that engine flip was
the "~1.2% changedRatio both directions" occ-export baseline flake.
Validated 160/160 across two 80-repeat rigs; full analysis in
docs/features/wx-parity-bugs/occ-export-context-eviction.md. Chromium
baselines shift slightly on llvmpipe — promote once from the first green
run. Deflakes the new coverage exposed: presence baselines settle before
capture; presence fixtures declare current file formats; perf gets its
own outputDir so CI evidence survives; occ-export settles the board paint
before the export dialog; menu-item waits (waitForRenderedByLabel before
clickMenuItem) in 4 specs + the TESTING.md rule.
Web suite runs the PROD build, in parallel: webServer becomes backend
`start` + the standalone's e2e:preview (build-preview.mjs: link-wasm →
stash the public/wasm symlink aside during vite build, build-demo.mjs's
move — then vite preview as the persistent server). The wasm middleware
serves /wasm/* in preview and emits COOP/COEP/CORP itself (a pthread
worker script's own response must carry COEP or Chrome kills it with
ERR_BLOCKED_BY_RESPONSE). VITE_* flags bake at build time;
VITE_ALLOW_USER_OVERRIDE joins turbo globalEnv. fullyParallel + default
workers: 5.2m → 1.4m. Determinism fixes the parallel run exposed:
shared-page specs become serial groups; locks.spec grabs alice's exact
item via the new kicadCollabTestSelectByUuid hook (cross-tab "first
footprint" order is not a ysync invariant); quit specs poll page.url()
(quit supersedes its own navigation — NS_BINDING_ABORTED on Firefox).
Suite: 51 passed / 12 skipped / 0 failed in 1.6m.
CI-coverage gate (lint:ci-coverage): every tests/**/*.spec.ts must be
reachable from the npm scripts the workflows invoke — scraped from
.github/workflows/, resolved through package.json, coverage asked from
playwright --list itself. Rules: uncovered-spec + orphan-project (with a
documented LOCAL_ONLY_PROJECTS allowlist). Gating next to
lint:determinism; 138 spec files / 13 projects accounted for.
Product fixes kept from the investigations (reachable on real GPUs too):
wx 7799fd1be5 — paint flags clear before dispatch + Invalidate always
propagates; kicad 3dcfea5e45 — SwiftShader pass-boundary flush +
per-instance font texture + first-frame GL-error drain (GAL recovery
recovers instead of falling back to Cairo) + the user-facing eeschema
switch navigates again under __EMSCRIPTEN__ (project-sync's
FaceRegistered gate had rerouted it into the hidden sync player; caught
by the newly-gated web suite).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018eUxiPApHgGiu9NFyQfhAq
This commit is contained in:
parent
8a20f69c5e
commit
63ed1f3c1f
556 changed files with 2470 additions and 1389 deletions
|
|
@ -1,9 +1,14 @@
|
|||
import { test, expect, devices, type Page, type Browser } from '@playwright/test';
|
||||
import { openOverlayMenu } from './overlay-menu';
|
||||
import { shotPath, settledShot } from '../e2e/utils/element-tracker';
|
||||
|
||||
// One shared page across the checks (beforeAll boots pcbnew ONCE — each boot
|
||||
// costs minutes). Serial group: fullyParallel would re-boot per test.
|
||||
test.describe.configure({ mode: 'serial' });
|
||||
|
||||
/**
|
||||
* Mobile canvas-only mode e2e (features/mobile) — runs under the
|
||||
* `mobile-chromium` project (Pixel 7 emulation, system Chrome).
|
||||
* `web-mobile` project (Pixel 7 emulation on the bundled Chromium).
|
||||
*
|
||||
* Boots pcbnew ONCE on the demo board with `?mobile=1` (the wasm runtime is
|
||||
* process-global and each boot costs minutes, so the four checks share one
|
||||
|
|
@ -83,7 +88,8 @@ async function fingerDrag(
|
|||
const x = from.x + ((to.x - from.x) * i) / steps;
|
||||
const y = from.y + ((to.y - from.y) * i) / steps;
|
||||
await touch(pg, 'touchmove', [{ id: 1, x, y }]);
|
||||
await pg.waitForTimeout(30);
|
||||
// The pacing IS the input: it gives the gesture a finger-plausible velocity.
|
||||
await pg.waitForTimeout(30); // eslint-disable-line -- documented interaction dwell: gesture pacing
|
||||
}
|
||||
await touch(pg, 'touchend', [], [{ id: 1, x: to.x, y: to.y }]);
|
||||
}
|
||||
|
|
@ -104,7 +110,8 @@ async function pinch(
|
|||
for (let i = 1; i <= steps; i++) {
|
||||
const s = spreadFrom + ((spreadTo - spreadFrom) * i) / steps;
|
||||
await touch(pg, 'touchmove', at(s));
|
||||
await pg.waitForTimeout(30);
|
||||
// The pacing IS the input: it gives the gesture a finger-plausible velocity.
|
||||
await pg.waitForTimeout(30); // eslint-disable-line -- documented interaction dwell: gesture pacing
|
||||
}
|
||||
await touch(pg, 'touchend', [], at(spreadTo));
|
||||
}
|
||||
|
|
@ -176,7 +183,14 @@ async function diffRatio(
|
|||
}
|
||||
|
||||
const shot = (name: string) =>
|
||||
page.screenshot({ path: `test-results/mobile-${name}.png`, scale: 'css' });
|
||||
page.screenshot({ path: shotPath(page, `mobile-${name}.png`), scale: 'css' });
|
||||
|
||||
/** Settle the render (two consecutive identical canvas screenshots), then capture
|
||||
* the named gate shot — the deterministic replacement for sleep-then-shoot. */
|
||||
const settledCapture = async (name: string): Promise<Buffer> => {
|
||||
await settledShot(page.locator('#canvas'), expect);
|
||||
return shot(name);
|
||||
};
|
||||
|
||||
test.beforeAll(async ({ browser }: { browser: Browser }) => {
|
||||
test.setTimeout(420_000); // cold load: 136 MB wasm download + compile
|
||||
|
|
@ -193,8 +207,9 @@ test.beforeAll(async ({ browser }: { browser: Browser }) => {
|
|||
// in WasmTool) must be GONE — the gesture tests diff screenshots, and an
|
||||
// overlay screenshot diffs to zero (learned the hard way on a cold load).
|
||||
await expect(page.locator('div.inset-0.z-30')).toHaveCount(0, { timeout: 180000 });
|
||||
// let the first paint + fit-to-view and the chrome-hide poll settle
|
||||
await page.waitForTimeout(4000);
|
||||
// First paint + fit-to-view + the chrome-hide poll: settled when two
|
||||
// consecutive canvas screenshots are byte-identical.
|
||||
await settledShot(page.locator('#canvas'), expect);
|
||||
});
|
||||
|
||||
test.afterAll(async () => {
|
||||
|
|
@ -212,9 +227,19 @@ test('tap contract: a touch tap synthesizes a LEFT click, no phantom middle-drag
|
|||
|
||||
const p = { id: 1, x: box.x + box.width * 0.5, y: box.y + box.height * 0.5 };
|
||||
await touch(page, 'touchstart', [p]);
|
||||
await page.waitForTimeout(60);
|
||||
// A tap's physical duration — instantaneous taps can be filtered as noise.
|
||||
await page.waitForTimeout(60); // eslint-disable-line -- documented interaction dwell: tap duration
|
||||
await touch(page, 'touchend', [], [p]);
|
||||
await page.waitForTimeout(300);
|
||||
// The synthesized click is the observable: wait for the mouseup to land in the log.
|
||||
await expect
|
||||
.poll(
|
||||
() =>
|
||||
page.evaluate(
|
||||
() => (window as unknown as { __mouseLog: [string, number][] }).__mouseLog.length,
|
||||
),
|
||||
{ timeout: 10000, message: 'tap never synthesized any mouse events' },
|
||||
)
|
||||
.toBeGreaterThanOrEqual(2);
|
||||
|
||||
const log = await page.evaluate(
|
||||
() => (window as unknown as { __mouseLog: [string, number][] }).__mouseLog,
|
||||
|
|
@ -237,14 +262,12 @@ test('pinch: two-finger pinch-out zooms in, pinch-in restores', async () => {
|
|||
const box = await getGlBox(page);
|
||||
const centre = { x: box.x + box.width * 0.5, y: box.y + box.height * 0.45 };
|
||||
|
||||
const base = await shot('pinch-00-base');
|
||||
const base = await settledCapture('pinch-00-base');
|
||||
await pinch(page, centre, 40, 140); // pinch OUT = zoom in
|
||||
await page.waitForTimeout(500);
|
||||
const zoomed = await shot('pinch-01-zoomed');
|
||||
const zoomed = await settledCapture('pinch-01-zoomed');
|
||||
|
||||
await pinch(page, centre, 140, 40); // pinch IN at the same centre = zoom back out
|
||||
await page.waitForTimeout(500);
|
||||
const restored = await shot('pinch-02-restored');
|
||||
const restored = await settledCapture('pinch-02-restored');
|
||||
|
||||
const dIn = await diffRatio(page, base, zoomed, box);
|
||||
const dBack = await diffRatio(page, base, restored, box);
|
||||
|
|
@ -264,14 +287,12 @@ test('pan: one-finger drag translates the view (not a rubber-band select)', asyn
|
|||
const from = { x: box.x + box.width * 0.15, y: box.y + box.height * 0.88 };
|
||||
const to = { x: box.x + box.width * 0.55, y: box.y + box.height * 0.7 };
|
||||
|
||||
const base = await shot('pan-00-base');
|
||||
const base = await settledCapture('pan-00-base');
|
||||
await fingerDrag(page, from, to);
|
||||
await page.waitForTimeout(500);
|
||||
const panned = await shot('pan-01-panned');
|
||||
const panned = await settledCapture('pan-01-panned');
|
||||
|
||||
await fingerDrag(page, to, from); // drag back — translation is invertible
|
||||
await page.waitForTimeout(500);
|
||||
const restored = await shot('pan-02-restored');
|
||||
const restored = await settledCapture('pan-02-restored');
|
||||
|
||||
const dPan = await diffRatio(page, base, panned, box);
|
||||
const dBack = await diffRatio(page, base, restored, box);
|
||||
|
|
|
|||
Loading…
Reference in a new issue