Make the Playwright e2e + kicad suites deterministic so screenshot flake stops
tracing to timing races.
- Blind page.waitForTimeout -> condition waits (expect.poll, web-first
assertions, waitUntil) + readiness helpers (waitForWxApp, waitForCanvasApp).
Remaining sleeps are documented interaction dwells (annotated).
- Defensive "if element exists" branches -> loud asserts; label-fallback chains
-> normalized clickMenuItemByText. First-run wizard for/if loops removed by
seeding calculator/gerbview/pcbnew HTMLs.
- Screenshots: new stableShot(page, name) settles the render in-page (canvas
hash over rAF) then writes a raw PNG to test-results/ for the existing offline
gate (tools/screenshots vs baseline-screenshots). Replaces toHaveScreenshot,
which did inline compare + its own baselines and had decoupled the specs from
the real gate. scale:'css' pinned.
- retries: 0 in both configs.
- Guard: tests/tools/lint-determinism.ts (npm run lint:determinism) bans blind
sleeps / toHaveScreenshot / inline retries / swallowed catches in specs;
documented exceptions carry a marker. Rules in tests/TESTING.md.
Assertions, coverage, and renders unchanged (semantic-equivalence reviewed;
captures pixel-identical modulo inherent timer/timestamp/3d-raytrace variance).
Both suites green at retries:0 (e2e 340, kicad 92); ~35-61% faster.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BVX1pHMvRPYHdp6ZfEawrk
The 3d-webgl merge (kicad eb13ff3bdc: the viewer now defaults to the real
OpenGL renderer via wasm/gl1, and occ-split moves STEP parsing into the
occ_service worker) made the raytracer-era orchestration on this branch moot —
main's chromium-ci phase is green at 15-way parallelism (28666407570 /
28698861536). Drop what no longer earns its complexity, keep the diagnostics,
fix main's live flake, and make the deadlock spec test what it was written for.
- REVERT the chromium-ci-3d serial project, the two-phase test:kicad:ci, the
SwiftShader GPU-process flags, and the resize-drag/models skips: config and
package.json are byte-for-byte back to main's shape. The raytracer contention
they guarded is no longer on the CI path.
- FIX main's live flake: run 28698861536 is green only via retry
(3d-viewer.spec:26 flaky) and 28666407570's deadlock red sampled an ALL-ZERO
pixel signature — the viewer's first frame lags the canvas's creation on
software WebGL under parallel load, and sampling too early reads an all-black
backbuffer. New waitForThreeDRender() gates render assertions on actual
pixels (1s-interval full-frame CPU reads) instead of fixed sleeps, used by
3d-viewer.spec:26 and the models render tail.
- KEEP the storm-proofed samplers (one full-frame getImageData on a
willReadFrequently canvas replacing 256 per-pixel GPU round-trips per sample
— the "GPU stall due to ReadPixels" trigger) and the logThreeDDiag
instrumentation: engine-independent, and they de-risk every remaining
software-GL pixel read.
- models spec: bridge assertions stay front-loaded (the protocol regression
signal is independent of the render); the occ_service parse verdict is now
POLLED — it lands async relative to the bridge ensures, so asserting it
immediately raced the worker; the render tail runs again everywhere. (The
pre-webgl raytracer+models renderer-death documented in a17f3be does not
affect the OpenGL default path — the raytracer-toggle+models combination
remains untested product surface, tracked outside this branch.)
- deadlock spec: the deadlock it guards is raytracer-specific and the viewer
now defaults to OpenGL — on the GL engine it either passes vacuously (fast
renders make every liveness assertion trivial, 28698861536) or fails on the
black first frame (28666407570). It now flips the engine via the "Use
raytracing" toolbar toggle (loud assert if the toggle moved) and
cross-checks engagement by requiring the canvas pixels to CHANGE after the
flip with no input in between (the raytraced frame is lit differently; a GL
re-render reproduces identical pixels; heap growth is unusable — mimalloc
satisfies the raytracer from freed arena pages). That guard immediately
caught a REAL defect: on the webgl-era wasm build the toggle is INERT (the
click lands and "Reload time" updates, but the canvas never changes —
suspects: DoRePaint's silent catch(runtime_error) freezing the canvas after
a raytracer Redraw throw, or ToggleRaytracing writing m_boardAdapter.m_Cfg
while RenderEngineChanged() reads GetAppSettings<…>(), possibly different
instances in the merged bundle). The spec is therefore test.skip-annotated
as a KNOWN ISSUE with the full engine-force machinery in place — unskipping
it self-validates the product fix. The CI-skip also stays (raytracer
liveness needs real-GPU pacing; the Worker-boot deadlock mechanism is
covered on CI by the standalone wx harnesses).
- 180s viewer-open waits kept as pure CI headroom (never slow a passing run).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The headed Mesa-llvmpipe switch (ce44636) was built on a misdiagnosis. With
--use-gl=angle --use-angle=gl under Xvfb, Chromium gets NO WebGL AT ALL on the
GPU-less CI VM: runs 28652367347 + 28664038296 both log
`glcanvas count before opening 3D viewer: 0` (even the pcbnew board's own GAL
canvas fails to create) and every failure is the FIRST viewer-open wait timing
out. "Timeouts instead of crashes" meant no GL context existed to crash, not
that llvmpipe was stable-but-slow. The Mesa/GLX path works for the Firefox
project, but not for Chromium on this image.
Headless SwiftShader is the environment where GL provably works — the
serialized run 28649537489 opened the viewer and rendered/sampled 56 colours
(viewer:26/130/209 all passed single-tab). Its remaining failures split into
two families, each addressed on its own evidence:
1) SwiftShader GPU-process kills (deadlock spec's lost canvas/black frame):
- chromium-ci-3d back to headless + --enable-unsafe-swiftshader, plus
--disable-gpu-watchdog and --disable-gpu-process-crash-limit: heavy churn
stalls the software-GL GPU process until the watchdog shoots it
(CONTEXT_LOST -> removed GL canvas -> "Target crashed"); with the watchdog
off a slow op completes, and without the crash limit a lost GPU process
keeps restarting for later tests. Same args locally (no-ops on a healthy
real GPU).
- Storm-proof every canvas sampler (3d-viewer, models, deadlock specs): one
full-frame getImageData on a willReadFrequently (CPU-backed) 2D canvas +
JS grid sampling, replacing 256 per-pixel getImageData GPU round-trips per
sample — the "GPU stall due to ReadPixels" trigger, worst in the deadlock
spec's 1.5s-interval settle polling. Sampled pixels and sig order are
unchanged.
- edge-resize spec: keep the open + 5-resize-handles assertions on CI, then
test.skip the drag — each resize step re-raytraces synchronously on the
wasm main thread and the 12-step drag blocked mouse.move past the 240s
budget even single-tab. Real-GPU-paced; still runs locally (deadlock
spec's CI-skip from d9900fe unchanged, same rationale).
2) KNOWN PRODUCT BUG uncovered in the models spec (NOT SwiftShader, NOT CI):
raytracing a scene WITH component models kills the Chromium renderer
process outright ~6s after scene build — silent process death with a FLAT
wasm heap (531-637 MB watched at 2s intervals; not OOM), no console error,
no wasm abort, no crash report. Deterministic on a real GPU (5/5),
fixture-independent (700 KB USB-C and 61 KB USON-8 STEP crash identically);
the same death hits CI SwiftShader at ~66s. Board-only raytraces complete
and stay alive, so the bug is specific to the model path. The spec's one
historical green ended at raytrace-age ~6s — inside the death window by
luck; every CI run of it ever died. Fix here: front-load the bridge
assertions (ensure requests fire during scene BUILD, before the raytrace),
so protocol regressions still fail the suite everywhere, and test.skip the
render tail unconditionally with the bug documented at the skip site. The
raytrace->blit pipeline stays gated on CI by 3d-viewer.spec's board-only
render. Bug tracked for a dedicated fix session.
Correct the now-wrong llvmpipe comments (config, open-wait rationale, deadlock
skip reason). Expected CI matrix: viewer open/render/stacking/titlebar +
models-bridge on SwiftShader serial; edge-resize drag + camera-move deadlock
on real GPUs locally; the Worker-boot deadlock mechanism stays covered on CI
by the standalone wx harnesses.
Locally validated on a real GPU (--project=chromium): 5 passed + 1 skipped
(the documented models render tail).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to the headed-llvmpipe switch. That run (28652367347) stopped CRASHING —
every failure became a plain timeout, confirming llvmpipe gives a stable WebGL context
(no more CONTEXT_LOST / "Target crashed") but software raytracing is just slow:
- The first board raytrace takes ~60s on llvmpipe, right at openThreeDViewer's 60s
wait → raise it to 180s (shared helper + the models spec's inline copy). Real GPU
returns in ~2s, so it is only CI headroom.
- The deadlock spec asserts the wasm main thread stays responsive within ~15s DURING a
raytrace — a premise only true on a fast (real) GPU. Under slow software rendering a
legitimately-slow raytrace is indistinguishable from a deadlock, so skip it on CI
(test.skip on process.env.CI). The Worker-boot deadlock MECHANISM stays covered on CI
by the standalone coroutine-pthread-ondemand / raytrace-threads wx harnesses; the spec
still runs locally on a real GPU.
- Gate the headed + llvmpipe launch flags on CI so chromium-ci-3d behaves like a normal
bundled-Chromium project locally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The three heavy 3D-viewer specs (3d-viewer, 3d-viewer-deadlock, 3d-viewer-models)
crashed on CI (run 28604015154): "Target crashed", "browser has been closed", and a
black frozen canvas. Each boots the 3D-enabled pcbnew build, which pre-warms
~hardwareConcurrency*2+8 (~68 on the 30-core VM) Web Workers AND runs a multi-threaded
CPU raytracer over SwiftShader software-WebGL. Under fullyParallel the three spec FILES
run in separate browser processes concurrently, and the simultaneous raytrace tabs
exhaust the shared GPU process's ~16 live-WebGL-context limit / renderer memory. The
pool pre-warm (the deadlock fix, 7630c7e) is correct and present in the tested binary;
per-file isolation prevented in-process Worker accumulation but NOT this cross-process
concurrency — the gap these specs' first CI exposure hit.
Fix (test-orchestration only; no wasm/submodule changes):
- Route the three specs to a dedicated `chromium-ci-3d` Playwright project (own
outputDir=pw-artifacts/kicad-3d so its failure traces survive the later perf run's
wipe; the existing tests/pw-artifacts/** upload already captures it).
- test:kicad:ci runs it as a SECOND `playwright test` pass with --workers=1, so at most
one raytrace tab is alive at a time (the environment in which a single 3D load is
reliably green — docs/features/3d-raytracer/README.md §7). An exit-code-aggregating
sequencer runs both phases and fails if either fails (no silent skip of the 3D pass).
- Cap phase-1 CI workers to 12 as headroom under the WebGL-context cap.
- Log a resource-diagnostic snapshot (hardwareConcurrency, pthread pool, GL-canvas
count, wasm/JS heap) before each 3D interaction so any recurrence is attributable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move OpenCASCADE out of the merged editor image into occ_service: a separate
emscripten module (-sASYNCIFY=0, MODULARIZE, in-container -Oz finalize, 2N+8
pre-warmed pthread pool) booted lazily in a dedicated Web Worker on the first
STEP export or STEP/IGES model parse. kicad_editor.wasm ~190 MB -> 130 MB;
sessions that never touch OCC never fetch its 57 MB. STEP export works in the
browser for the first time: the unchanged desktop dialog runs EXPORTER_STEP,
whose wasm shadow suspends into globalThis.occService and the export bytes go
straight to a browser download (never entering the editor heap). STEP/IGES 3D
models parse in the worker via the oce shadow (S3D WriteCache/ReadCache wire).
- wasm/occ-service/: service CMake target (hooked from the kicad fork's
top-level CMakeLists, wasm/editor pattern), embind entry
(occExport/occLoadModel), wxConfig pre-js.
- wasm/stubs/{exporter_step,oce_plugin}_stub.cpp: EM_ASYNC_JS worker bridges
(callee-shadowing; no caller #ifdefs).
- web/standalone: provider installed whenever the kicad_editor bundle boots
(cross-face safe); ONE shared worker-boot source occ-worker.js (vite ?raw;
the e2e stub reads the same file) — blob worker with locateFile absolutized
against the glue URL; export download-name guard.
- deps: OCC builds with RapidJSON so its glTF/GLB writer exists — pinned to
the vcpkg master snapshot 2025-02-26 (24b5e7a8b27f), the same code official
KiCad consumes via vcpkg.json's opencascade[rapidjson]; rapidjson's latest
tag (v1.1.0, 2016) is ill-formed under modern clang.
- tests: occ-export dialog e2e (lazy-fetch boundary + STEP download bytes),
occ-probe incl. a 9-format matrix (step/stpz/brep/xao/ply/stl/glb/u3d/pdf),
3d-viewer-models hard-asserts the worker parse; occ provider stub installed
ambiently by the kicad fixtures.
Validated against desktop kicad-cli 10.0.4: geometric exact equality (bbox
delta 0 um, volume delta 0.0000%) for STEP/GLB/STL/BREP/STPZ across three
boards and option sweeps — with desktop OCC 7.9 vs wasm OCC 7.8; PLY/XAO/PDF
structurally equal; U3D same-size (quantizer float LSBs differ). Full kicad
e2e green on Firefox and Chromium; standalone verified end to end (lazy fetch
only on the Export click; export.step 60,628 B ISO-10303-21; loadModel 700 KB
STEP -> 569 KB scenegraph cache).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New tooling in tests/tools/screenshots/ (TypeScript via tsx):
- compare.ts: one pixelmatch engine (AA-excluded), connected-component
"where to look" boxes, old|new+boxes|heatmap triptych, per-engine floors.
- promote.ts: churn-free updater — overwrite a baseline only when decoded
pixels differ beyond the floor, copying CI bytes verbatim (no re-encode
churn); pulls a CI run via `gh run download` or a local --from dir.
- post-discord.ts: always-on CI-on-main report (SHA + e2e status + the
track-only runtime-perf table), then screenshot triptychs, batched +
size-capped + flood-collapsed + 429-aware.
- perf-report.ts: perf table with Δ vs the previous main run (via gh).
- changelog.ts: no-build git-history baseline differ (Discord trigger B).
- noise.ts / gen-manifest.ts: calibration + manifest generation.
CI wiring:
- wasm-build.yml: post-test step runs the gate + report on the already-
produced test-results (no extra build); report-only (continue-on-error),
posts only on push to main, inert without DISCORD_WEBHOOK_URL.
- ci-ubicloud.yml: secrets: inherit (pass the webhook through).
- screenshot-changelog.yml: ~30s no-build changelog on baseline changes.
screenshot-manifest.json: canonical 354-name set + best-effort engine tags
(313 chromium-swiftshader / 41 firefox-llvmpipe).
Normalize scale:'device'->'css' across 18 spec files (no-op at CI DSF=1)
so committed baselines are uniformly css-scaled.
Design: CI's Linux render is the single source of truth; no pinned
container (accept rare env drift -> re-promote); dev commits via promote.
Replaces the byte-cmp compare-screenshots.sh + file-size-proxy
update-baseline-screenshots.sh (kept for now until the first re-baseline).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Measures the current build's cold load, open+render, and pan/zoom FPS
(1x/4x/6x CPU-throttle sweep) for both editors, writing
tests/test-results/perf-*.json. Track-only: asserts only that the app booted
and the document opened — no perf thresholds, so it never gates CI.
Runs on a new Chromium `perf` Playwright project (CDP throttling; pcbnew needs
V8 anyway) via `npm run test:perf`, and is wired into wasm-build.yml as a
continue-on-error step so the numbers are captured/uploaded without flaking the
gate. Reuses the existing fixtures, ready-signal, fs-inject, and board-ready
helpers; FPS counter is cancel-able so a throttle sweep on one page doesn't
accumulate rAF loops.
.gitignore: ignore /benchmark-builds/ (disposable prebuilt-WASM + standalone
harness bundle used to reproduce the native-vs-JS-EH comparison locally).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Builds KiCad's 3D viewer for the browser (opt in with BUILD_3D_VIEWER=ON) and
wires up the legacy-GL emulation it needs.
- build-kicad-target.sh / docker/build.sh: BUILD_3D_VIEWER →
-DKICAD_BUILD_3D_VIEWER_WASM=ON + -sLEGACY_GL_EMULATION + the GL js-library.
- wasm/shims/gl_immediate_shim.js: display-list emulation (record/replay board
layers), fixed-function + GLU stubs, throw-guards for unsupported pnames, and
per-context GLImmediate init for the viewer's SECOND WebGL context.
- tests/kicad/3d-viewer.spec.ts + utils/pcbnew-ready.ts: open View → 3D Viewer.
- bumps the kicad submodule (EMSCRIPTEN-guarded GL changes).
WIP: the viewer window + UI render and geometry draws (glError=0x0), but the
board is not yet visibly rendered — GLImmediate's FFP shader program isn't
linked on the viewer's 2nd WebGL context. Full status + next steps in
features/feat/add-3d-view/notes.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds an end-to-end test that drives File→Open in pcbnew, injects
the .kicad_pcb and .kicad_pro files into MEMFS at the dialog's
default starting directory, drives the menu + filename text input
+ Enter accept path, and screenshots the loaded board. Parametrized
for both kicad/demos/microwave (RF polygon footprints) and
kicad/demos/pic_programmer (full multi-IC layout).
Without the rtree fix bumped in via the kicad submodule, the load
would abort on every PCB at rtree.h:1771 Classify; the test asserts
no [RTREE-DIAG] line and no Aborted(. The post-load clipboard
RuntimeError in __asyncjs__js_clipboardHasText is a separate,
pre-existing wasm-port limitation that we explicitly do not regress
on here.
- tests/kicad/load-pcb.spec.ts: serial-mode parametrized spec
- tests/kicad/load-pcb-probe.spec.ts: one-shot diagnostic probe
for inspecting wxFileDialog state on the canvas
- tests/kicad/utils/fs-inject.ts: FS.writeFile bridge from Node fs
- tests/kicad/utils/board-ready.ts: poll-for-no-dialogs readiness
- tests/baseline-screenshots/load-pcb-*.png: 6 baselines covering
both demos at pcbnew-ready / dialog-open / loaded states
- features/.../rtree-debug-findings.md: full diagnosis trail with
an upstream-reportable summary the maintainer can lift verbatim
- kicad submodule bumped to 07d8130d44 (shape_poly_set rtree fix)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
