test(3d): screenshot-baseline TDD suite for the 3D viewer OpenGL->WebGL port — 47 native goldens + red-state WebGL harness
tests/3d-regression mirrors the gal-regression pattern at renderer scale: shared C++ scenarios call real KiCad 3D-viewer code (opengl_utils, display lists + DrawCulled stencil subtraction, MODEL_3D VBOs, private generators via a rob-template accessor, and full reload()+Redraw() composites over a synthetic BOARD_ADAPTER). A native macOS harness renders them on real OpenGL into 47 committed goldens (bit-deterministic, FBO capture); the wasm harness compiles the same TUs against wasm/stubs/gl_ffp_stub.c no-ops so every scenario renders blank — the TDD red state (parity meter: 47/47 changed). Comparisons use the CI pixelmatch engine via the new generic compare-dirs.ts (floors.json levels; manifest.json cmp-guards registry drift). Documents an upstream bug: appendPostMachiningGeometry's countersink path adds middle quads without normals, silently erasing the walls of any display list it is batched into (3d-post-machining.png keeps the lists separate to record it). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
1a24c2894e
commit
ce1473c9ab
85 changed files with 6797 additions and 1 deletions
124
tests/3d-regression/README.md
Normal file
124
tests/3d-regression/README.md
Normal file
|
|
@ -0,0 +1,124 @@
|
|||
# 3D Renderer Regression Suite (OpenGL → WebGL port)
|
||||
|
||||
Screenshot-baseline TDD harness for porting KiCad's 3D viewer OpenGL renderer
|
||||
(`RENDER_3D_OPENGL`, pure GL 1.x fixed-function) to WebGL2 — the same approach
|
||||
used for the 2D GAL port (`tests/gal-regression/`), but compared with the CI
|
||||
pixelmatch engine instead of ImageMagick.
|
||||
|
||||
Shared C++ **scenarios** call real KiCad 3D-viewer code (draw helpers, display
|
||||
lists, materials, camera, stencil hole-subtraction, VBO models…) and are
|
||||
compiled two ways:
|
||||
|
||||
- **native/** — macOS app on real desktop OpenGL (2.1 compatibility context via
|
||||
the vendored glad loader). Renders each scenario into a fixed 800×600
|
||||
offscreen FBO (`GL_RGBA8` + `GL_DEPTH24_STENCIL8`, the
|
||||
`EDA_3D_CANVAS::RenderToFrameBuffer` recipe) → **golden baselines**.
|
||||
- **wasm/** — the same scenario TUs compiled with em++ (planned; red state
|
||||
first: linked against `wasm/stubs/gl_ffp_stub.c` no-ops, every render blank
|
||||
until the port makes them green).
|
||||
|
||||
## Directory layout
|
||||
|
||||
```
|
||||
scenarios/ shared scenario sources (native + wasm) — the registry in
|
||||
scene3d_test_scenarios.cpp is the single source of truth
|
||||
native/ golden generator (CMake; homebrew wxWidgets + OpenGL.framework)
|
||||
wasm/ WebGL harness (planned)
|
||||
baseline/ committed native goldens: 3d-<name>.png
|
||||
baseline-webgl/ committed browser renders (port era, CI-promoted)
|
||||
output/ run output + diffs (gitignored)
|
||||
manifest.json {width,height,scenarios[]} written by the native harness;
|
||||
committed — the anti-drift anchor for specs and orchestrator
|
||||
floors.json pixelmatch verdict floors per comparison level
|
||||
```
|
||||
|
||||
## Running
|
||||
|
||||
```
|
||||
./scripts/test-3d-regression.sh # build + render + gate (native; webgl when present)
|
||||
./scripts/test-3d-regression.sh native # native phase only
|
||||
./scripts/test-3d-regression.sh compare # comparisons only
|
||||
./scripts/test-3d-regression.sh promote # promote output/native -> baseline/ (byte-diff guarded)
|
||||
```
|
||||
|
||||
Logs land in `logs/test-3d-regression/` (build logs in `tests/logs/`).
|
||||
|
||||
## Comparison levels
|
||||
|
||||
Engine: `tests/tools/screenshots/compare-dirs.ts` (pixelmatch
|
||||
`{threshold: 0.1, includeAA: false}` — AA edge pixels ignored), floors from
|
||||
`floors.json`. Diff triptychs/heatmaps + `report.json` land in
|
||||
`output/diff/<level>/`.
|
||||
|
||||
| Level | Pair | Floor (changedRatio) | Role |
|
||||
|---|---|---|---|
|
||||
| `native-self` | `baseline/` vs `output/native/` | 0.001 (measured noise: exactly 0 on Apple M5) | gating regression check on the dev Mac |
|
||||
| `webgl-vs-native` | `baseline/` vs `output/webgl/` | 0.02, **report-only** | the TDD port-progress meter (`npm run 3d:check:parity`) |
|
||||
| `webgl-self` | `baseline-webgl/` vs `output/webgl/` | 0.005 | browser regression anchor (once the port renders) |
|
||||
|
||||
npm scripts (from `tests/`): `3d:check`, `3d:check:webgl`, `3d:check:parity`,
|
||||
`3d:compare` (generic dir pair), `3d:test:webgl`.
|
||||
|
||||
## Updating baselines
|
||||
|
||||
Baselines are generated on the dev Mac (CI has no native GL — same model as the
|
||||
GAL suite). After an intentional render change:
|
||||
|
||||
1. `./scripts/test-3d-regression.sh native` (fails with triptychs in
|
||||
`output/diff/native-self/` — eyeball them),
|
||||
2. `./scripts/test-3d-regression.sh promote` (byte-diff-guarded copy, zero
|
||||
churn) and commit `baseline/` + `manifest.json`.
|
||||
|
||||
The orchestrator `cmp`s the freshly-written manifest against the committed one
|
||||
every run, so a scenario registry change can't silently drift past the
|
||||
baselines. Scenario names are append-only — never rename or renumber (they are
|
||||
the PNG names and the WebGL test IDs).
|
||||
|
||||
## TDD red state
|
||||
|
||||
Once `wasm/` exists, the Playwright spec (`tests/e2e/3d-webgl.spec.ts`) is
|
||||
capture-only and stays green; the red signal is `npm run 3d:check:parity`
|
||||
reporting ~100% changed for every scenario. Port progress = scenarios dropping
|
||||
out of that report. `glLineWidth > 1` (model bbox scenario) has no WebGL
|
||||
equivalent — expect that one to need quad emulation to go green.
|
||||
|
||||
## Scenario tiers (47 scenarios)
|
||||
|
||||
- **Tier 1 (30)** — standalone TUs: `opengl_utils`
|
||||
(arrows/segments/bbox/half-cylinder), `ogl_utils` (background gradient,
|
||||
materials, textures), `TRIANGLE_DISPLAY_LIST`/`OPENGL_RENDER_LIST` (display
|
||||
lists, seg-ends alpha-test texture, `DrawCulled` stencil subtraction,
|
||||
z-transform, transparency), `MODEL_3D` (VBO/IBO, material modes, bboxes),
|
||||
`SPHERES_GIZMO`, camera (perspective/ortho/preset views, isolated lights).
|
||||
- **Tier 2 (14)** — `RENDER_3D_OPENGL` private generators
|
||||
(`generateCylinder/Disk/Dimple/InvCone`, all five `addObjectTriangles`
|
||||
overloads, `appendPostMachiningGeometry`, via composite, the four grid
|
||||
densities, `setupMaterials`/`setLayerMaterial`/`setArrowMaterial`,
|
||||
`createBoard`) via the rob-template accessor
|
||||
(`native/render3d_test_accessor.*`) over the synthetic `BOARD_ADAPTER`
|
||||
(`native/board_adapter_test_impl.cpp` — its `InitSettings` is the test-data
|
||||
seam).
|
||||
- **Tier 3 (3)** — full `reload()` + `Redraw()` composites over the synthetic
|
||||
mini-board: `redraw-empty`, `redraw-mini-board` (copper/silk/mask/stencil
|
||||
holes), `redraw-mini-board-navigator` (grid + gizmo — the port-complete
|
||||
gate).
|
||||
|
||||
## Known upstream bug (documented by `3d-post-machining.png`)
|
||||
|
||||
`appendPostMachiningGeometry`'s COUNTERSINK path adds middle-contour quads
|
||||
with `AddQuad` but never calls `AddNormal`, so the normals array ends up half
|
||||
the vertex count and `OPENGL_RENDER_LIST::generate_middle_triangles` rejects
|
||||
the whole middle list — a countersunk hole silently erases the walls of any
|
||||
geometry batched into the same `TRIANGLE_DISPLAY_LIST` (the real viewer has
|
||||
the same defect). The scenario keeps counterbore and countersink in separate
|
||||
lists so the counterbore renders correctly while the countersink half records
|
||||
the buggy (empty) upstream output.
|
||||
|
||||
Lighting semantics worth knowing: `init_lights()` runs once at context init
|
||||
under the identity modelview, so the two directional lights are anchored in
|
||||
**eye space** (they follow the camera) — the harness replicates that
|
||||
(`SCENE3D_CTX::InitOnce`), and only the headlight is repositioned per frame
|
||||
like `Redraw()` does. Also avoid toggling `GL_LIGHTx` between draws inside one
|
||||
frame: the Apple GL driver drops the first draw after a mid-frame toggle (the
|
||||
real renderer never does this; the isolated-light scenarios use one light per
|
||||
frame instead).
|
||||
Loading…
Reference in a new issue