Commit graph pcbjam/tests/README.md
Author SHA1 Message Date
Viktor Vaczi
63ed1f3c1f e2e/CI: dual-engine suites, per-engine screenshots, SwiftShader retired, prod web suite, CI-coverage gate
Squash of experiment/ff-big-modules vs main.

Big-module routing removed: native-EH shrank kicad_editor below
SpiderMonkey's x86-64 code budget (runs 29355049705/29356152413 green on
stock Firefox), so BIG_MODULE_SPECS routing and the baseline-only-JIT
crutch are gone — kicad-firefox and kicad-chromium both run the full
suite, with the module compiled the way real users' browsers compile it.

Per-engine screenshots end to end: stableShot/shotPath write
test-results/<engine>/<name>.png; baselines move to
baseline-screenshots/{chromium,firefox}/ and the whole tools/screenshots
pipeline (compare/promote/manifest/spec-map/changelog/Discord) keys on
<engine>/<name>. Previously Firefox and Chromium renders of one spec
overwrote each other and Firefox renders were never actually gated.
Seeded from CI run 29421380806 (92 new firefox baselines, +24 chromium
web-suite shots); manifest generated from the baseline tree.

One merged playwright.config.ts (kicad/asyncify/coroutine/perf as
projects); ~25 dead npm scripts dropped. The web suite is gated in CI for
the first time ever (4 rotted specs fixed, 5 broken lib-bridge specs
triaged as fixme in docs/features/web-e2e-rot/); cheap lint step after
npm ci; last 26 blind-sleep violations fixed.

SwiftShader retired: CI Chromium renders WebGL on ANGLE → Mesa llvmpipe
(--use-gl=angle --use-angle=gl --ignore-gpu-blocklist; the blocklist flag
is mandatory — llvmpipe is blocklisted and WebGL is silently unavailable
without it) in BOTH configs. Under WORKERS=4 congestion SwiftShader
transiently failed the first post-board-load draw and the recovery
cascade ended in a silent permanent Cairo fallback — that engine flip was
the "~1.2% changedRatio both directions" occ-export baseline flake.
Validated 160/160 across two 80-repeat rigs; full analysis in
docs/features/wx-parity-bugs/occ-export-context-eviction.md. Chromium
baselines shift slightly on llvmpipe — promote once from the first green
run. Deflakes the new coverage exposed: presence baselines settle before
capture; presence fixtures declare current file formats; perf gets its
own outputDir so CI evidence survives; occ-export settles the board paint
before the export dialog; menu-item waits (waitForRenderedByLabel before
clickMenuItem) in 4 specs + the TESTING.md rule.

Web suite runs the PROD build, in parallel: webServer becomes backend
`start` + the standalone's e2e:preview (build-preview.mjs: link-wasm →
stash the public/wasm symlink aside during vite build, build-demo.mjs's
move — then vite preview as the persistent server). The wasm middleware
serves /wasm/* in preview and emits COOP/COEP/CORP itself (a pthread
worker script's own response must carry COEP or Chrome kills it with
ERR_BLOCKED_BY_RESPONSE). VITE_* flags bake at build time;
VITE_ALLOW_USER_OVERRIDE joins turbo globalEnv. fullyParallel + default
workers: 5.2m → 1.4m. Determinism fixes the parallel run exposed:
shared-page specs become serial groups; locks.spec grabs alice's exact
item via the new kicadCollabTestSelectByUuid hook (cross-tab "first
footprint" order is not a ysync invariant); quit specs poll page.url()
(quit supersedes its own navigation — NS_BINDING_ABORTED on Firefox).
Suite: 51 passed / 12 skipped / 0 failed in 1.6m.

CI-coverage gate (lint:ci-coverage): every tests/**/*.spec.ts must be
reachable from the npm scripts the workflows invoke — scraped from
.github/workflows/, resolved through package.json, coverage asked from
playwright --list itself. Rules: uncovered-spec + orphan-project (with a
documented LOCAL_ONLY_PROJECTS allowlist). Gating next to
lint:determinism; 138 spec files / 13 projects accounted for.

Product fixes kept from the investigations (reachable on real GPUs too):
wx 7799fd1be5 — paint flags clear before dispatch + Invalidate always
propagates; kicad 3dcfea5e45 — SwiftShader pass-boundary flush +
per-instance font texture + first-frame GL-error drain (GAL recovery
recovers instead of falling back to Cairo) + the user-facing eeschema
switch navigates again under __EMSCRIPTEN__ (project-sync's
FaceRegistered gate had rerouted it into the hidden sync player; caught
by the newly-gated web suite).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018eUxiPApHgGiu9NFyQfhAq
2026-07-17 12:21:54 +02:00
Viktor Vaczi
4e92baf5d0
test(ysync): repro tests for review bugs 01-07 + v2 items-wire e2e port (miss 11)
The 2026-07-02 sync review (docs/features/ysync-review, ysync-review branch)
found 7 bugs and that the two-tab e2e only exercised the DEAD legacy scalar
wire. This lands plan doc 15 in full; results + empirical findings in doc 16.

- tests/collab/browser-entry-v2.ts (+build.mjs): the PRODUCTION v2 stack
  bundled for e2e (connectKicadDoc + attachKicadCollab, kdoc_* keys), with
  in-page renderActiveDoc/singleSeedRender/driftReport helpers and yjs forced
  to ONE copy (the two web pnpm workspaces otherwise bundle two
  instanceof-incompatible instances).
- tests/kicad/ysync-two-tab.spec.ts: pl_editor green baseline (A↔B edits,
  ITEM-level drift silence) + divergent-uuid adopt; bug-01 pcb/ee fresh-room
  repros (Chromium-only: two kicad_editor tabs exceed Firefox's per-process
  wasm budget); bug-06 concurrent-seed race; bug-03 Y-half.
- tests/kicad/ysync-repros-{pcbnew,eeschema}.spec.ts: bugs 02/03/05 + the
  bug-04 matrix (anchor-centred fp rotation, pad resize, endpoint drag,
  symbol rotation, Value-field edit), each with green landed-preconditions;
  the "local move emits" controls double as headless-emit probes — GREEN on
  both tools, so every emit-dependent repro is a live test.fail.
- wasm/bindings: 7 local-edit test hooks via real commits
  (CallAfter+COROUTINE) — TestRemoveItem/TestRotateItem (both tools,
  dispatched in the merged image), TestSetPadSize/TestMoveEndpoint (pcbnew),
  TestSetFieldText (eeschema).
- web/standalone ysync-repros.test.ts: bug-01 units (C++-faithful fake gating
  emit on ensureBridge) + bug-07a/b (stale DOWN hook, real sheet-manager gap).
- web/pcbjam-shared bump: bug-03/06 unit repros.

Convention: every repro asserts the CORRECT behavior and is expected-fail
(test.fail/it.fails) naming its bug doc; a fix flips it to "unexpected pass",
forcing marker removal — the repro becomes the regression test. Every
expected failure verified (JSON reporter) to fail at its documented assert.
Suite state: 39 passed / 0 failed / 0 flaky / 5 skipped (2 firefox guards,
2 pre-existing legacy two-tab skips, 1 pre-existing roundtrip fixme).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPfrVhfYgPPgtawjSssZfn
2026-07-06 08:57:01 +02:00
Viktor Vaczi
c1ef489cfa feat(wasm-eh): migrate the WASM build to native wasm exceptions (+ 3D viewer default-on)
Replace the legacy Emscripten JS-exceptions model with native wasm-EH (legacy
encoding) across the whole build, keeping Asyncify coroutines working via a
from-source Binaryen --hoist-cpp-catches pre-pass. Net result: native-EH is the
only build mode, the 3D viewer is on by default, and pcbnew shrinks substantially.

Highlights:
- Binaryen submodule everywhere + --hoist-cpp-catches integration in apply-asyncify;
  post-link Asyncify covers every app wasm (not just standalone test wasm).
- Build deps (incl. OpenCASCADE without OCC_CONVERT_SIGNALS) and all KiCad apps
  with -fwasm-exceptions; emscripten_sleep added to the post-link asyncify-imports.
- libcontext fiber entry wired under native exceptions; while-loop main loop +
  currData shim injected into all wx apps.
- Native-EH collab apply fixed: DEBUG-define the embind TU + match all out-of-CMake
  C++ TUs' ABI flags to the core, fixing the vtable-layout skew / mis-dispatch.
- 3D viewer enabled by default (real raytracer linked, not the stub).
- Retire the EH-spike scaffolding; flip the asyncify-races ablation pins to
  shim-redundancy pins (native-EH stays clean with the legacy shims ablated).
- Fix the asyncify-races quiescence check to not require Asyncify.currData==0:
  under the native-EH per-frame-yield top loop the main stack is asyncify-suspended
  every frame, so currData legitimately churns (a freed-but-not-yet-nulled buffer,
  not a leak). Refresh the pcbnew toolbar screenshot baseline for the new kicad.
- CI: drop the obsolete binaryen_version input/env (the build uses the binaryen
  submodule fork's wasm-opt, not a version download); key the wasm-output cache on
  the binaryen submodule SHA instead.

Bumps the wxwidgets + binaryen submodules to their squashed feature commits.

Validated green: all 7 apps native-EH (real 3D in pcbnew); KiCad e2e 63/63
Firefox + Chromium (3D viewer renders); wx 336; coroutine 34/34 both engines;
asyncify 7/7 both engines.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 09:40:26 +02:00
Viktor Vaczi
fa408c0c06 fix(wasm): bump wxwidgets for the wxMenuBar Detach() UAF fix
Submodule pointer bump for the wxMenuBar dangling-child fix that resolved the 18 DOM-port kicad e2e failures. Also adds the async-debug dossier (docs/features/async/08-10) root-causing the 'indirect call to null' / 'signature mismatch' family to the menu-bar use-after-free, and folds the asyncify harness into 'npm test' (new test:wx for wx-only); tests/README + WHATWORKS updated to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 14:10:05 +02:00
Viktor Vaczi
7629ab978d test(e2e): fix wx suite — header deps, port pinning, drop legacy-GL tests, wx+pthread worker fix
Suite went 64 failed → 291 passed / 0 failed:
- Makefile.wasm: compile with -MMD -MP and include .d files — stale objects
  relinked against a newer wx lib crashed apps at startup with "function
  signature mismatch" after any header/vtable change
- playwright(.coroutine).config.ts: pin the server port for the whole run
  (resolvePort from playwright-kicad.config.ts); the 60s freshness window made
  workers restarted after a failure rotate to a dead port (ERR_CONNECTION_REFUSED
  cascade across ~50 tests)
- drop legacy-GL testing: KiCad renders via WebGL GAL (gal-webgl.spec.ts);
  remove minimal_test's OpenGL tab/GLTestCanvas, opengl.spec.ts, GL z-order
  describes, -sLEGACY_GL_EMULATION + gl_immediate_shim.js from the test build,
  and the orphaned GL baselines
- coroutine-pthread repro apps join the `all` target (make clean used to delete
  them while all never rebuilt them); clean no longer eats checked-in JS like
  worker_dom_stub.js
- bump wxwidgets: guard module-eval document access so wx+pthread apps
  (threadpool tests) survive Web Worker eval

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 13:40:00 +02:00
Viktor Vaczi
251056e506 Clean up test infrastructure and fix test assertions
- Replace test.skip() with proper expect() assertions when app fails to load
- Remove button-finder utility (no longer needed with element registry)
- Remove kicad tests (tested separately)
- Add findByName/clickByName helpers for bitmap buttons
- Fix element lookups: use clickByName for bitmap buttons, clickTreeItem
  for treebook pages, selectComboItem for wxChoice items
- Add SetName() to shape buttons in bitmapbuttons_test.cpp
- Remove verbose logging from print and threadpool tests
- Update README to reflect current test infrastructure

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 20:18:19 +01:00
Viktor Vaczi
0a093b918e Add element registry for semantic E2E test automation
Replace hardcoded pixel coordinates with semantic element lookups in tests.
The element registry (added to wxWidgets) tracks all wxWindow instances,
enabling tests to find buttons by label text instead of pixel positions.

Changes:
- Add element-tracker.ts with clickByLabel, findByLabel, findByType, etc.
- Migrate clipboard, dialog, timer, filedialog, logerror tests to use registry
- Update fixtures.ts to export element-tracker utilities
- Update README with element registry documentation
- Update wxwidgets submodule with registry implementation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 12:11:33 +01:00
Viktor Vaczi
a26b40c063 Reorganize project structure for clarity
- Rename tests/wasm-app/ to tests/apps/ for brevity
- Move lib/gl_immediate_shim.js to wasm/shims/ (consolidates WASM files)
- Delete docs/ directory (outdated Nov 2024 research docs)
- Delete patches/ directory (already applied to fork submodules)
- Delete wasm/patches/ and wasm/config/ (empty/unused)
- Update all file references in scripts, configs, and documentation
- Update .gitignore for new tests/apps/ paths

Verified both build workflows pass:
- wxWidgets tests: 255 passed
- Docker KiCad build: completed successfully

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 14:06:23 +01:00
Viktor Vaczi
9a78db1528 Add wxDataViewCtrl, wxHtmlWindow, and wxStyledTextCtrl test apps
Test 3 previously untested KiCad-critical wxWidgets controls:
- wxDataViewCtrl: List/tree views for Zone Manager, Net Inspector (10 tests)
- wxHtmlWindow: HTML rendering for About dialogs, error formatting (8 tests)
- wxStyledTextCtrl: Syntax highlighting for DRC rules, Python console (10 tests)

Also add button-finder config and update docs. All 119 tests passing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-03 11:27:00 +01:00
Viktor Vaczi
802b1e871c Implement wxClipboard and add button finder utility for canvas testing
wxClipboard:
- Update wxwidgets submodule with full wxClipboard implementation
- Add clipboard async functions to ASYNCIFY_IMPORTS in Makefile.wasm
- Update clipboard tests with correct button positions
- Add clipboard permissions to playwright config
- Update WHATWORKS.md to mark wxClipboard as working

Button Finder Utility:
- Add parametric button-finder.spec.ts for scanning canvas apps
- Utility scans for clickable buttons by detecting console log responses
- Excluded from regular test runs via testIgnore
- Supports APP_URL, START_Y, END_Y, STEP environment variables
- Document usage in tests/README.md

All 91 tests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-03 10:19:50 +01:00
Viktor Vaczi
67f5a88a34 Fix wxGrid test crash and add debug build support
The wxGrid test was crashing with "memory access out of bounds" due to
an initialization order bug in grid_test.cpp: during CreateGrid(), wxGrid
fires cell selection events which triggered the OnGridCellSelect handler
that called m_log->AppendText() before m_log was initialized.

Fix: Add null check in LogEvent() to guard against events firing before
m_log is created.

Also adds --debug flag to build-wasm-test.sh for debugging WASM crashes:
- Builds with DWARF symbols (-g) and source maps (-gsource-map)
- No optimization (-O0) preserves debugging context
- Stack traces show actual function names instead of wasm-function[N]

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-29 22:15:56 +01:00
Viktor Vaczi
26181bb929 Refactor test logging into shared utilities
- Extract common logging code to tests/e2e/utils/
  - fixtures.ts: Playwright fixture with automatic log capture
  - test-utils.ts: Logging utilities and helper functions
- Each test now automatically captures:
  - Console logs with timestamps and log levels
  - Page errors with full stack traces
- Log files written per test to tests/logs/
  - <test-name>.log for all console output
  - <test-name>.errors.log only when errors occur
- Update all 13 spec files to use shared fixtures
- Update README with test structure and logging docs
- Net reduction of ~700 lines by removing duplicated code

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-29 20:54:54 +01:00
Viktor Vaczi
3f67fed173 Add Playwright test infrastructure for wxWidgets WASM port
- Add tests/ directory with Playwright e2e tests
- Add minimal wxWidgets WASM test application
- Add build script for test app (scripts/build-wasm-test.sh)
- Update wxWidgets submodule with WASM build fixes
- Update .gitignore for test artifacts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-28 13:10:05 +01:00