pcbjam/tests/3d-regression

3D Renderer Regression Suite (OpenGL → WebGL port)

Screenshot-baseline TDD harness for porting KiCad's 3D viewer OpenGL renderer (RENDER_3D_OPENGL, pure GL 1.x fixed-function) to WebGL2 — the same approach used for the 2D GAL port (tests/gal-regression/), but compared with the CI pixelmatch engine instead of ImageMagick.

Shared C++ scenarios call real KiCad 3D-viewer code (draw helpers, display lists, materials, camera, stencil hole-subtraction, VBO models…) and are compiled two ways:

• native/ — macOS app on real desktop OpenGL (2.1 compatibility context via the vendored glad loader). Renders each scenario into a fixed 800×600 offscreen FBO (GL_RGBA8 + GL_DEPTH24_STENCIL8, the EDA_3D_CANVAS::RenderToFrameBuffer recipe) → golden baselines.
• wasm/ — the same scenario TUs compiled with em++ (planned; red state first: linked against wasm/stubs/gl_ffp_stub.c no-ops, every render blank until the port makes them green).

Directory layout

scenarios/        shared scenario sources (native + wasm) — the registry in
                  scene3d_test_scenarios.cpp is the single source of truth
native/           golden generator (CMake; homebrew wxWidgets + OpenGL.framework)
wasm/             WebGL harness (planned)
baseline/         committed native goldens: 3d-<name>.png
baseline-webgl/   committed browser renders (port era, CI-promoted)
output/           run output + diffs (gitignored)
manifest.json     {width,height,scenarios[]} written by the native harness;
                  committed — the anti-drift anchor for specs and orchestrator
floors.json       pixelmatch verdict floors per comparison level

Running

./scripts/test-3d-regression.sh            # build + render + gate (native; webgl when present)
./scripts/test-3d-regression.sh native     # native phase only
./scripts/test-3d-regression.sh compare    # comparisons only
./scripts/test-3d-regression.sh promote    # promote output/native -> baseline/ (byte-diff guarded)

Logs land in logs/test-3d-regression/ (build logs in tests/logs/).

Comparison levels

Engine: tests/tools/screenshots/compare-dirs.ts (pixelmatch {threshold: 0.1, includeAA: false} — AA edge pixels ignored), floors from floors.json. Diff triptychs/heatmaps + report.json land in output/diff/<level>/.

Level	Pair	Floor (changedRatio)	Role
native-self	baseline/ vs output/native/	0.001 (measured noise: exactly 0 on Apple M5)	gating regression check on the dev Mac
webgl-vs-native	baseline/ vs output/webgl/	0.02, report-only	the TDD port-progress meter (npm run 3d:check:parity)
webgl-self	baseline-webgl/ vs output/webgl/	0.005	browser regression anchor (once the port renders)

npm scripts (from tests/): 3d:check, 3d:check:webgl, 3d:check:parity, 3d:compare (generic dir pair), 3d:test:webgl.

Updating baselines

Baselines are generated on the dev Mac (CI has no native GL — same model as the GAL suite). After an intentional render change:

1. ./scripts/test-3d-regression.sh native (fails with triptychs in output/diff/native-self/ — eyeball them),
2. ./scripts/test-3d-regression.sh promote (byte-diff-guarded copy, zero churn) and commit baseline/ + manifest.json.

The orchestrator cmps the freshly-written manifest against the committed one every run, so a scenario registry change can't silently drift past the baselines. Scenario names are append-only — never rename or renumber (they are the PNG names and the WebGL test IDs).

TDD red state

Once wasm/ exists, the Playwright spec (tests/e2e/3d-webgl.spec.ts) is capture-only and stays green; the red signal is npm run 3d:check:parity reporting ~100% changed for every scenario. Port progress = scenarios dropping out of that report. glLineWidth > 1 (model bbox scenario) has no WebGL equivalent — expect that one to need quad emulation to go green.

Scenario tiers (47 scenarios)

• Tier 1 (30) — standalone TUs: opengl_utils (arrows/segments/bbox/half-cylinder), ogl_utils (background gradient, materials, textures), TRIANGLE_DISPLAY_LIST/OPENGL_RENDER_LIST (display lists, seg-ends alpha-test texture, DrawCulled stencil subtraction, z-transform, transparency), MODEL_3D (VBO/IBO, material modes, bboxes), SPHERES_GIZMO, camera (perspective/ortho/preset views, isolated lights).
• Tier 2 (14) — RENDER_3D_OPENGL private generators (generateCylinder/Disk/Dimple/InvCone, all five addObjectTriangles overloads, appendPostMachiningGeometry, via composite, the four grid densities, setupMaterials/setLayerMaterial/setArrowMaterial, createBoard) via the rob-template accessor (native/render3d_test_accessor.*) over the synthetic BOARD_ADAPTER (native/board_adapter_test_impl.cpp — its InitSettings is the test-data seam).
• Tier 3 (3) — full reload() + Redraw() composites over the synthetic mini-board: redraw-empty, redraw-mini-board (copper/silk/mask/stencil holes), redraw-mini-board-navigator (grid + gizmo — the port-complete gate).

Known upstream bug (documented by 3d-post-machining.png)

appendPostMachiningGeometry's COUNTERSINK path adds middle-contour quads with AddQuad but never calls AddNormal, so the normals array ends up half the vertex count and OPENGL_RENDER_LIST::generate_middle_triangles rejects the whole middle list — a countersunk hole silently erases the walls of any geometry batched into the same TRIANGLE_DISPLAY_LIST (the real viewer has the same defect). The scenario keeps counterbore and countersink in separate lists so the counterbore renders correctly while the countersink half records the buggy (empty) upstream output.

Lighting semantics worth knowing: init_lights() runs once at context init under the identity modelview, so the two directional lights are anchored in eye space (they follow the camera) — the harness replicates that (SCENE3D_CTX::InitOnce), and only the headlight is repositioned per frame like Redraw() does. Also avoid toggling GL_LIGHTx between draws inside one frame: the Apple GL driver drops the first draw after a mid-frame toggle (the real renderer never does this; the isolated-light scenarios use one light per frame instead).