pcbjam/tests/TESTING.md

Testing rules

Determinism rules for the Playwright specs (tests/e2e, tests/kicad, tests/jspi, tests/web). Enforced by npm run lint:determinism (tools/lint-determinism.ts, gating in CI). Run specs from tests/ via npm run test:e2e (the full CI project set: wx-chromium, kicad-firefox, kicad-chromium, jspi-firefox, coroutine-firefox) or npm run test:kicad (kicad-firefox only) — not playwright directly. One spec on one engine: npx playwright test --project=kicad-firefox kicad/pcbnew.spec.ts.

Waits — never blind

• No page.waitForTimeout(n). Wait on a condition: expect.poll(() => predicate), a web-first assertion (expect(locator).toBeVisible()), or waitUntil(page, fn, desc) (throws loudly on timeout).
• App readiness: waitForWxApp(page) (canvas visible + element registry populated) for widget/editor harnesses; waitForCanvasApp(page) for registry-less canvas apps.
• The only allowed waitForTimeout is an irreducible interaction dwell — a canvas/keyboard commit with no JS-observable signal — and it MUST carry a same-line marker: // eslint-disable-line -- documented interaction dwell: <why>.
• Menu clicks: wait for the specific item, not a count. Popup items register in the element registry progressively as they paint, so a coarse gate ("N menuitems rendered") can pass before the item you're about to click exists — and clickMenuItem is single-shot. Before every clickMenuItem(page, 'X'), await waitForRenderedByLabel(page, 'X', { elementType: 'menuitem' }) (same matcher as the click). clickMenuItemByText already waits internally and needs no guard. A submenu click needs its own wait: the parent menu's still-rendered items satisfy any count gate before the submenu paints.

No defensive branches

• No if (await el.count()) el.click(). Assert the element exists, then act: expect(await clickByLabel(page, 'X'), '...').toBe(true). Use clickMenuItemByText (normalizes & / ... / …) instead of try-A-else-A…-else-A fallback chains.
• No swallowed .catch(() => {}). Let it throw, or assert the tolerated outcome. A genuinely best-effort op must carry a marker explaining why.

Screenshots — stableShot, compared offline

• Capture with stableShot(page, 'name.png', { fullPage }) — it settles the render (in-page canvas-hash over animation frames) then writes a raw PNG to test-results/<engine>/ (chromium/firefox, derived from the running browser — the same spec on two engines writes two files). It does not assert. Never use Playwright's toHaveScreenshot. Raw page.screenshot/fs writers must route through shotPath(page, 'name.png') for the same engine scoping.
• Comparison is offline and per-engine: npm run screenshots:check diffs test-results/<engine>/ against the baselines in tests/baseline-screenshots/<engine>/ (+ the still-committed 3d-regression/, gal-regression/).
• Baselines live in the private R2 bucket pcbjam-ci-screenshots, not git: the R2-hosted manifest baselines/pcbjam/manifest.json pins each {name, engine} to a sha256, and baseline-screenshots/ + .baseline-manifest.json are gitignored caches — materialize them with npm run screenshots:fetch-manifest && npm run screenshots:fetch (needs the R2 credentials; see tools/screenshots/README.md). Nothing screenshot-related is committed to git.
• CI's Linux render is the source of truth; baselines are promoted from a CI run in the morelli review app (https://pcbjam-morelli-staging.pcbjam-staging.workers.dev) — CI uploads each run's renders to R2 (30-day retention), morelli shows the diffs and writes the manifest on Promote. A local (Mac) check shows font/render noise and is not the gate.
• A continuously-animating state (timer, mid-slide) can't be a stable baseline — drop the shot.

Retries

• retries: 0 in both configs (playwright.config.ts — the merged wasm-suite config — and playwright-web.config.ts). A failure is real; don't mask it with a retry.

Every spec runs in CI — lint:ci-coverage

npm run lint:ci-coverage (tools/lint-ci-coverage.ts, gating in CI next to the determinism lint) proves every *.spec.ts under tests/ is actually executed by CI: it scrapes the npm run test:… invocations from .github/workflows/, resolves them through package.json to their playwright test --config/--project flags, and asks Playwright itself (--list) which files those runs cover. No hand-maintained lists — adding a spec in a brand-new directory is exactly what it catches.

When it fires:

• uncovered-spec — your new spec matches no CI-run project. Put it in a covered testDir, adjust a project's testMatch, or add the project to a CI npm script.
• orphan-project — you added a config project no CI script selects. Wire it into a CI script, or (for deliberately-local system-browser projects) add it to LOCAL_ONLY_PROJECTS in the lint with a comment saying why.

Where things are

• Per-test logs (JS console + cpp): tests/logs/{wxwidgets,kicad}/<test-name>/.
• Guards: npm run lint:determinism, npm run lint:ci-coverage. Screenshot gate: npm run screenshots:check.