pcbjam/docs/features/async/migration-evidence/jspi-vs-asyncify-bench-2026-08.md
Viktor Vaczi f419a1fedd build+perf: wasm-opt the shipped wasm, and measure real frames in CI
emcc only runs Binaryen at link -O2+ (link.py: should_run_binaryen_optimizer
returns OPT_LEVEL >= 2) and we link at -O1, so the shipped module had never seen
wasm-opt at all — it kept its entire 19.56 MB name section, ~20% of the editor
(-sJSPI sets ASYNCIFY=2, which suppresses wasm-ld's --strip-debug, leaving
wasm-opt as the only thing that would drop it). Step 8.2 runs it post-link and
in-container, so CI's cached compile phase covers it and the host post-process
stays pure-host.

Default -O2, picked by measuring every level on the same module: -O0 already
captures 27% of the raw win (it is mostly the name section), -O2 costs 23 s and
gives the best frame rate, and -O3/-O4/-Os/-Oz cost 48-132 s for at most 1.5%
more brotli — -O4 is not even smaller than -O3. Targets that already link -O2/-Oz
(occ_service, kicad_tools) are skipped by testing for the target_features
section, which emcc strips whenever it ran the optimizer itself, so there is no
hard-coded target list to drift. Feature flags come from the module's own
target_features section and so cannot diverge from the link.

The perf specs reported requestAnimationFrame ticks as "FPS". That is not a frame
rate: rAF fires on the compositor's schedule whether or not the GAL redrew, and
it read 120/s on a board where the renderer completed zero frames in six seconds.
measureInteractionFps now counts completed GAL frames — runs of draws to the
default framebuffer, exactly one per frame in every AA mode — and drives a pure
middle-drag pan after a zoom-to-fit. Mixing wheel zoom into the drive made the
result depend on where the wheel left the view: +-20% across identical repeats,
against +-2% for pan alone. The report gains a GAL fps column with a regression
flag on the 1x number; rAF is kept so historical runs stay comparable.

CI has no GPU, so its number is a software-rasteriser redraw rate — a regression
signal, not a user-facing frame rate. Method and measurements in the bench report.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016X9eh1s5sTx1o9Em9KBuwR
2026-08-22 12:53:00 +02:00

507 lines
27 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# JSPI vs Asyncify: editor benchmark (2026-08-14)
A/B benchmark of the `kicad_editor` app built two ways:
- **Arm A — asyncify**: `pcbjam` @ `29c61b8` (branch `staging`), emscripten **4.0.2**,
`-sASYNCIFY=1` + host post-link pipeline (finalize → hoist-cpp-catches →
`wasm-opt --asyncify``wasm-opt -O1`, Binaryen fork v130).
- **Arm B — JSPI**: worktree @ `9c475a8` (branch `experiment/jspi`), emscripten **6.0.6**,
`-sJSPI`, single-phase link, no post-link transform (only the ~1 s env-shim patch).
**Stated confound, up front:** the two arms differ by emscripten major (4.0.2 vs
6.0.6) — libc/libc++/linker/JS-glue all moved with it. That skew is inseparable
from the backend switch (strict-JSPI needs emscripten 6); every number below is
"the shipped asyncify build vs the shipped JSPI build", not a pure
suspension-mechanism isolate. What IS matched: same KiCad commit (`4bfed3f1`),
wxWidgets 3.3.1, EH flags (`-fwasm-exceptions -sSUPPORT_LONGJMP=wasm
-sWASM_LEGACY_EXCEPTIONS=1`), mimalloc, memory config (256 MB initial / 4 GB
max), **DEBUG mode on both** (TU `-g -O1`, link `-O1 -g -gseparate-dwarf` — the
default), docker caps 10 CPU / 32 GB, `-j 10`, `BINARYEN_CORES=8`,
`HOIST_KEEP_NAMES` unset, runs serialized on an idle machine.
Machine: Apple M4 Max, 16 cores, 64 GB; Docker Desktop VM 12 CPU / 28 GiB;
macOS 26.5.2. Raw data: [`bench-data-2026-08/`](bench-data-2026-08/).
> **Revised 2026-08-14 (evening).** The build, size, load and board-open numbers
> stand. The **interaction-FPS numbers were wrong and have been re-measured** —
> see [§4.4](#44-interaction-fps-re-measured-2026-08-14-evening). The original
> `distinctFps` counter hashed a 48×48 downscale of the canvas, which (a) misses
> redraws that change few pixels and (b) stalls the renderer through the pixel
> readback, badly so under software rasterisation. Frames are now counted from
> the WebGL command stream, the view is zoomed-to-fit before every measurement so
> each run sees the same geometry, and both a real GPU and a software rasteriser
> are reported. Direction of the result is unchanged; the magnitudes are not.
>
> **Second revision, same evening.** The two arms were never built the same way:
> the Asyncify pipeline always finished with a host-side `wasm-opt -O1`, while
> the JSPI build got **no Binaryen pass at all** (emcc only runs wasm-opt at link
> -O2+, and we link at -O1). So every number above compares an optimised module
> against an unoptimised one — in JSPI's favour, since JSPI won anyway. A
> post-link `wasm-opt` step has now been added to the build and re-benchmarked:
> [§6](#6-post-link-wasm-opt). The original unoptimised numbers are kept
> throughout; §7 adds the third arm.
## TL;DR
| Metric | Asyncify | JSPI | Δ |
|---|---|---|---|
| Editor wasm, raw | 113.1 MB | 94.1 MB | **17 %** |
| Editor wasm, gzip 9 | 36.7 MB | 18.6 MB | **49 %** |
| Editor wasm, brotli | 24.5 MB | 13.5 MB | **45 %** |
| Post-link tail (per build!) | 63.4 s, 6.1 GB RSS | 1.6 s, 49 MB RSS | **97 %** |
| Clean rebuild (warm ccache) | 165 s | 95 s | **42 %** |
| Cold load (median of 5) | 2 576 ms | 1 538 ms | **40 %** |
| Open vme-wren (27.7 MB board) | 6 465 ms | 3 551 ms | **45 %** |
| GAL frames/s, vme-wren pan (GPU) | 12.7 | 19.5 | **+54 %** |
| GAL frames/s, vme-wren zoom (GPU) | 26.2 | 39.5 | **+51 %** |
| GAL frames/s, jetson pan (GPU) | 11.7 | 15.5 | **+32 %** |
| Wasm heap after big-board open | 962 MB | 802 MB | **17 %** |
| Jetson 80.9 MB board open | 14.6 s, peak 1.90 GB | 8.6 s, peak 1.73 GB | **41 %** |
| JSPI wasm, raw — **with `wasm-opt -O1`** | — | **63.3 MB** | **36 %** vs unoptimised JSPI |
| JSPI wasm, brotli — **with `wasm-opt -O1`** | — | **10.6 MB** | **15 %** vs unoptimised JSPI |
(The last two rows are the third arm added in [§6](#6-post-link-wasm-opt); every
other row is the original A/B, both arms as they were actually built.)
The asyncify tax was real on every axis: bytes (instrumented code also
compresses ~2× worse), build time (a mandatory 6-GB-RSS host pass per link),
load (bigger download + more code to tier), compute (slower opens, ~5070 %
fewer rendered frames), and memory. The frame-rate advantage holds at a roughly
*constant ratio* across CPU throttle rates (+63 % / +51 % / +73 % at 1× / 4× /
6×), i.e. JSPI does less work per frame rather than merely loading a smaller
module. The earlier claim that the gap *widened* under throttle (+21 % → +68 %)
does not survive re-measurement — it was an artefact of the old counter being
clipped at 1×.
## 1. Build time
Scenarios (identical commands both arms, run serially): **S1** no-change
rebuild (`./docker/build.sh kicad_editor -j 10` twice, 2nd reported), **S2**
clean editor rebuild with warm ccache (`--clean-kicad`, twice), **S3** clean
rebuild with `CCACHE_DISABLE=1` (manual `compose exec`, compile cost only).
Both trees exhibit an **mtime ping-pong** in the wx re-enter (a reconfigure
regenerates files the next rsync restores → every other build pays a wx
reconfigure). Numbers are therefore given per observed mode; phase splits from
the `@KW@` markers.
| Scenario | Asyncify | JSPI |
|---|---|---|
| S1 no-change, quiet mode | **175 s** (cfg 40 + link 40 + finalize 4 + **asyncify 58**) | **154 s** (cfg 56 + link 46, tail ≈0) |
| S1 no-change, wx-churn mode | 427 s (wx 89 + cfg 132 + compile 124 + tail 63) | 379 s (first run after container recreate) |
| S2 clean, warm ccache, quiet | **165 s** (1 819 TU in 29 s + tail 61) | **95 s** (1 819 TU in 26 s) |
| S2 clean, wx-churn mode | 248 s | 122 s |
| S3 clean, no ccache (kicad only) | **365 s** (cfg 154 + compile 211) | **304 s** (cfg 136 + compile 168) |
| Container compile-only, steady | 91 s | — (contained in S2) |
Reading: the container-side compile is broadly comparable (emscripten 6 is
~15-20 % faster on the uncached compile); **the structural difference is the
host tail** — asyncify pays 58-63 s *on every single link*, JSPI pays ~1.6 s.
In the dev loop that is the difference between a ~2.5-minute and a ~4-minute
edit-to-browser cycle (or 7 min when the wx churn mode hits, which the tail
compounds).
Note: A-S3's wall figure included a 453 s wx rebuild triggered by the bench's
own `touch` of the wx build Makefile (used to pin the wx skip path) — excluded
above; the kicad configure+compile figures are uncontaminated.
## 2. Build memory
- **Container peak** (cgroup v2 `memory.peak`, fresh cgroup per run): both arms
1824 GiB across all scenarios — dominated by the `-j 10` compile and the
DWARF-heavy `wasm-ld` link, essentially backend-independent.
- **Host post-link tail** (`/usr/bin/time -l`):
- Asyncify: **63.4 s wall, 307 s user, peak RSS 6.09 GiB**
(finalize ≈4 s + hoist+asyncify+`-O1` ≈58 s at `BINARYEN_CORES=8`).
Historical logs show 59154 s and one OOM-killed build (exit 137).
- JSPI: **1.6 s wall, peak RSS 49 MB** (node env-shim patch).
## 3. Bundle size
Same-day builds, DEBUG mode, complete cold-load set
(`kicad_editor.{wasm,js}`, `wx.js`, `wx-dom.js`, `images.tar.gz`):
| | raw | gzip 9 | brotli |
|---|---|---|---|
| Asyncify wasm | 118 557 371 B (113.1 MB) | 36.7 MB | 24.5 MB |
| JSPI wasm | 98 664 038 B (94.1 MB) | 18.6 MB | 13.5 MB |
| Asyncify total set | 118.8 MB | 42.0 MB | 29.5 MB |
| JSPI total set | 99.8 MB | 23.8 MB | 18.5 MB |
The compressed delta (49 % gzip) is much larger than the raw delta (17 %):
asyncify's spilled-locals/branch-table instrumentation is high-entropy. The
over-the-wire cost of a cold editor load roughly **halves** under JSPI.
Side note: separate DWARF is 2.27 GB vs 1.91 GB (16 %); dev-tools-only fetch.
Other apps (existing artifacts, not rebuilt today): pl_editor 19 %,
gerbview 19 %, calculator 26 %, occ_service 1 % (never asyncified),
ngspice_service 0 %.
## 4. Runtime
One harness for both arms: the jspi tree's Playwright 1.62.1 `perf` project
(bundled Chromium, no JSPI-specific launch flags), `tests/kicad/
pcbnew-large-perf.spec.ts`, artifact sets swapped in `tests/apps/kicad/` with
sha256 recorded per row (jspi `6f03e62d…`, asyncify `6dab4931…`). The harness
page seeds KiCad config for both settings versions (9.99 + 10.0) so the
asyncify binary boots wizard-free. Local static server serves wasm gzipped
(hence `transferSize` ≈ the gzip column above). Headless; per doc-12, headless
FPS is a comparative indicator, not an absolute.
**Cold load** (fresh context ×5, navigation → fully-ready editor):
| | Asyncify | JSPI |
|---|---|---|
| loadMs median (range) | 2 576 (2 5212 629) | 1 538 (1 1171 841) |
| wasm fetch (gzipped wire) | ~990 ms / 32.1 MB | ~570 ms / 17.1 MB |
| wasm heap at boot | 556.8 MB | 386.7 MB |
CDP-attach sanity: loadMs with a CDP session pre-attached fell inside each
arm's normal range (2 514 / 1 117 ms) — no tier-down artifact from the
throttling channel.
**Board open** (`Module.kicadOpenFile`, ×3 each):
| Board | Asyncify | JSPI |
|---|---|---|
| demo (155 KB, 15 fp) | 467 ms | 358698 ms (par — too small to discriminate) |
| vme-wren (27.7 MB, 1 508 fp, 24 858 seg) | **6 465 ms** median, peak heap 9621 183 MB | **3 551 ms** median, peak heap 802 MB |
| jetson-agx-thor (80.9 MB, 1 125 fp) | loaded, **14 584 ms**, peak 1.90 GB | loaded, **8 620 ms**, peak 1.73 GB |
Neither arm OOMs even on the 80.9 MB board — both stay well under the 4 GB cap.
### 4.4 Interaction FPS (re-measured 2026-08-14 evening)
**What the first pass got wrong.** `distinctFps` counted samples of a 48×48
downscale hash of the GAL canvas at ~30 Hz. That under-counts real redraws by
23.5× (a moving crosshair changes too few pixels to survive the downscale), it
is capped at the 30 Hz sample rate, and the `drawImage`+`getImageData` readback
it performs is itself expensive — under software rasterisation it was a
significant share of the very thing being measured. The `rafFps` column was
never a frame rate at all: rAF ticks on the compositor's schedule whether or not
the GAL redrew anything.
**The corrected metric.** A GAL frame ends with the compositor blitting to the
default framebuffer, so a *run* of draw calls issued while no framebuffer is
bound is exactly one completed frame. The run must be collapsed — the number of
present draws per frame varies with the AA mode (1 under supersampling, 2 under
`AA_NONE`, +1 when the crosshair is drawn) — but a run boundary happens once per
frame in every mode, so no divisor is needed. This is now what
`measureInteractionFps()` in `tests/kicad/utils/perf-utils.ts` reports, and what
CI records.
**Two further methodology fixes.** Each pattern gets a discarded warm-up drive
(the first pass pays first-time tessellation and measures caching, not steady
state), and the view is **zoomed to fit before every drive**. Without the reset a
preceding wheel-zoom leaves an arbitrary zoom level and the next pattern sees a
different amount of geometry — run-to-run spread reached 2×. With it, repeats
land within ~1% (measured: 21.3 / 21.3 / 21.1 fps over three runs). Zoom-to-fit
also means every number below is the *whole board in view*, i.e. the worst case.
Both arms were re-run back-to-back on the same machine in the same session, from
their own static servers (asyncify served read-only out of the `staging`
worktree's `output/`; the jspi tree's harness page and board fixtures shared by
both, so only the 5-file artifact set differs).
**vme-wren (27.7 MB, 1 508 fp, 24 858 seg) — GAL frames/s:**
| Renderer | Pattern | Asyncify | JSPI | Δ |
|---|---|---|---|---|
| GPU (ANGLE Metal) | crosshair only | 66.4 | 64.9 | 2 % |
| GPU (ANGLE Metal) | wheel zoom | 26.2 | **39.5** | **+51 %** |
| GPU (ANGLE Metal) | middle-drag pan | 12.7 | **19.5** | **+54 %** |
| Software (SwiftShader) | crosshair only | 59.5 | 65.9 | +11 % |
| Software (SwiftShader) | wheel zoom | 11.6 | **39.3** | **+239 %** |
| Software (SwiftShader) | middle-drag pan | 9.9 | 9.5 | 4 % |
**jetson-agx-thor (80.9 MB) — GAL frames/s:**
| Renderer | Pattern | Asyncify | JSPI | Δ |
|---|---|---|---|---|
| GPU (ANGLE Metal) | crosshair only | 66.0 | 63.7 | 3 % |
| GPU (ANGLE Metal) | wheel zoom | 16.0 | **19.4** | **+21 %** |
| GPU (ANGLE Metal) | middle-drag pan | 11.7 | **15.5** | **+32 %** |
| Software (SwiftShader) | any | **0** | **0** | — |
**CPU-throttle sweep** (vme-wren, GPU, middle-drag pan, GAL frames/s):
| Throttle | Asyncify | JSPI | Δ |
|---|---|---|---|
| 1× | 12.5 | 20.4 | **+63 %** |
| 4× | 4.1 | 6.2 | **+51 %** |
| 6× | 2.6 | 4.5 | **+73 %** |
Readings:
- **JSPI's interaction advantage is real and larger than first reported**: ~+50 %
on vme-wren and ~+30 % on jetson, on a real GPU, where the first pass claimed
+21 %.
- **Crosshair-only motion is identical on both arms** (~65 fps everywhere, zero
vertex upload). Moving the cursor only re-composites; it never touches
geometry, so the suspension backend has nothing to do with it. This is the
control case, and it behaving as a control is a good sign for the rest.
- **The earlier "jetson saturates, the backend stops mattering" conclusion was an
artefact** of the broken counter. On a real GPU the backend still separates the
arms by ~30 % at 80.9 MB. What *is* true is that neither arm renders the jetson
board at all under software rasterisation: no frame completes within a 6 s
window, and re-running with a 45 s settle and a 20 s window still yields zero
(rAF 2.8/s, 233 GL calls/s). A single redraw there takes over 20 seconds.
- **The advantage does not widen under throttle** — it is a roughly constant
ratio (+51 % to +73 %, the spread being run noise). That still points at less
work per frame rather than a pure module-size effect, but the original
"+21 % → +68 % widening" reading was an artefact: the old counter's 30 Hz
ceiling and under-counting compressed the measured 1× gap specifically.
- **Software-rasteriser numbers do not rank the arms reliably** — vme-wren pan is
a tie (9.9 vs 9.5) while zoom is a 3.4× gap. Rank the backends on the GPU
numbers; treat the software column as the CI-shaped regression signal it is.
Absolute frame rates here are lower than the old `distinctFps` figures would
suggest at first glance only because zoom-to-fit puts the entire board in view.
Load and board-open times re-measured in the same runs reproduced §4's originals
(e.g. vme-wren open 6 475 vs 2 088 ms; jetson open 14 510 vs 8 094 ms), which is
the cross-check that the new rig measures the same builds as the old one.
**Memory checkpoints** (wasm linear memory; Chromium `usedJSHeapSize` tracked
alongside, differences <10 %): boot 557 vs 387 MB; after vme-wren open 962 vs
802 MB; unchanged after the FPS sweep on both arms.
## 5. Methodology notes & gotchas
- Asyncify arm reused the warm `kicad-wasm-main` compose volume (62 GB); its
branch-default `kicad-wasm-staging` volume is a cold stub — running without
`COMPOSE_PROJECT_NAME=kicad-wasm-main` would have benchmarked a multi-hour
cold dep build. The other session's tree was verified idle before/after and
`git status` byte-identical.
- Container `memory.peak` read from a per-run fresh cgroup (`compose stop`
between runs, `KICAD_KEEP_CONTAINER=1` so the read happens before teardown),
plus a 5 s `docker stats` sampler (CSV in bench-data).
- The first S3 attempt died in wx's PCRE `aclocal` regen (manual `compose
exec` skips build.sh's rsync, which perturbs the wx reconfigure check); fixed
by pre-touching the wx build Makefile — which on arm A then triggered the wx
rebuild noted in §1. Symmetric procedure both arms.
- The jspi-arm FPS rows in the archived ndjson appear twice: the first pass
sampled the static `#canvas` (distinct ≈ 0, marked SUPERSEDED); the glcanvas
re-run is authoritative and matches what `perf-utils.measureFpsDetailed` now
does. Arm A ran entirely with the fixed sampler.
- Playwright clears `tests/test-results/` per invocation — it ate the arm B
ndjson during the arm A run (reconstructed from the run logs, two timestamps
approximate). The spec now writes to `tests/bench-results/` instead.
- The live dev app at :3048 was kept on the JSPI build throughout (its
`public/wasm` symlink pointed at a stash during the swap window) and
verified serving the JSPI wasm afterwards.
## 6. Post-link wasm-opt
**The asymmetry.** Arm A's pipeline ended with `wasm-opt --hoist-cpp-catches` →
`--asyncify` → **`-O1`** on the host. Arm B ended with nothing: emcc only runs
Binaryen at link `-O2`+ (`tools/link.py`, `should_run_binaryen_optimizer()`
returns `settings.OPT_LEVEL >= 2`), and at -O0/-O1 the pass list comes back empty
so `wasm-opt` is never even spawned. The section tables confirm it — the asyncify
module has **no name section** (its host `wasm-opt` stripped it) while the JSPI
module still carries **19.56 MB** of names, because `-sJSPI` sets `ASYNCIFY=2`,
which suppresses wasm-ld's `--strip-debug`, leaving wasm-opt as the only thing
that would have dropped them.
So §§34 compare an optimised module against an unoptimised one. That understates
JSPI: on a like-for-like code-section basis it was 66.7 MB (never optimised)
against asyncify's 106.3 MB (optimised).
**The build step.** Added to `scripts/kicad/build-kicad-target.sh` as step 8.2,
after the link and before `copy-output`. It runs in-container (wasm-opt ships
with emsdk, so CI's cached `--compile-only` phase covers it and the host
`--postprocess-only` phase stays pure-host). Defaults to `-O1`, matching the
Asyncify-era pipeline; override with `KICAD_WASM_OPT` (`-O2`, `-O1 -g` to keep
the name section, `off` to skip). No feature flags are passed: Binaryen reads the
module's own `target_features` section, so the enabled-feature list cannot drift
from the link. That same section is the skip signal — emcc strips it via
`--strip-target-features` whenever it ran the optimizer itself, so targets that
link at -O2/-Oz (`occ_service`, `kicad_tools`) are detected and skipped rather
than hard-coded by name. A stamp file makes it idempotent across relink-free
rebuilds, and a failure leaves the linked module untouched and fails the build.
**Cost: 9 s**, in a 121 s no-change rebuild (~7 %). For contrast the Asyncify host
tail cost 63.4 s and 6.09 GB RSS per link. The "wasm-opt OOMs on large modules"
comment that justified `-O0` in the release path is Asyncify-era: it predates
JSPI and was written when instrumentation had roughly doubled the function count.
**What it does to the module:** code 66.7 → 51.0 MB, name section 19.56 MB → 0,
`external_debug_info` preserved (so `-gseparate-dwarf` DWARF still resolves).
### 6.1 Size
| Editor wasm | Asyncify | JSPI | JSPI + `wasm-opt -O1` |
|---|---|---|---|
| raw | 118.6 MB | 98.7 MB | **63.3 MB** |
| gzip 9 | 38.5 MB | 19.5 MB | **16.1 MB** |
| brotli | 21.0 MB | 12.5 MB | **10.6 MB** |
Against unoptimised JSPI: **36 % raw, 17 % gzip, 15 % brotli**. Against
asyncify: 47 % raw, 58 % gzip, 49 % brotli. The raw delta is much larger than
the compressed one because most of it is the name section, which compresses well.
### 6.2 Runtime
Three arms, same session, GPU (ANGLE Metal), zoom-to-fit, 6 s drives. Pan is the
headline: it repeats within ~2 %. Zoom was dropped from the harness entirely
after this run — the wheel drive continuously changes how much geometry is
visible, so it spread ±20 % run to run (34.7 / 42.1 / 27.1 on three identical
repeats) and discriminated nothing. The CI perf specs now drive a pure
middle-drag pan for the same reason. Cursor is the control: crosshair-only motion
touches no geometry.
| vme-wren, GPU | Asyncify | JSPI | JSPI + `wasm-opt -O1` |
|---|---|---|---|
| cold load | 2 145 ms | 1 153 ms | **1 063 ms** |
| open board | 6 347 ms | 3 381 ms | 3 317 ms |
| **pan, GAL fps** | 11.9 | 18.3 | **18.1** |
| cursor, GAL fps (control) | 65.4 | 64.8 | 65.3 |
| zoom, GAL fps (noisy) | 20.0 | 32.4 | 23.0 |
| jetson-agx-thor, GPU | Asyncify | JSPI | JSPI + `wasm-opt -O1` |
|---|---|---|---|
| cold load | 2 135 ms | 1 173 ms | **1 050 ms** |
| open board | 14 651 ms | 8 080 ms | 8 013 ms |
| **pan, GAL fps** | 11.4 | 14.3 | **14.4** |
**Reading: `wasm-opt -O1` is a size and startup win, not a frame-rate win.**
Frame rate is unchanged within noise on both boards (18.3 → 18.1 and 14.3 → 14.4
— the two arms are indistinguishable). Cold load improves 810 % (1 153 → 1 063
and 1 173 → 1 050), which is what a 36 % smaller download and less code to tier
buys. Board open is flat, consistent with it being dominated by parsing and
connectivity rather than code quality.
That frame rate does not move is the expected result rather than a
disappointment: the per-frame hot path is scene traversal in code LLVM already
optimised at the translation-unit level, and Binaryen `-O1` on top of clang -O1
mostly removes cross-module redundancy and dead weight. It also means the
original A/B's *interaction* conclusions are not disturbed by the asymmetry — the
JSPI-vs-asyncify frame-rate gap was never an artefact of the missing pass. The
size and load-time comparisons in §§34, however, were: JSPI's real advantage
there is larger than those sections state.
### 6.3 Every optimization level, measured
All seven Binaryen levels run on the same pristine unoptimised module
(98.7 MB), then each served against identical glue and benchmarked. `-O0` is the
control: it optimises nothing, so it isolates what a plain round-trip plus
dropping the name section is worth on its own.
| Level | wasm-opt wall | raw | gzip 9 | **brotli** | pan GAL fps | cold load |
|---|---|---|---|---|---|---|
| none | — | 98.7 MB | 19.47 MB | **12.51 MB** | 18.8 | 1 2571 609 ms |
| `-O0` | 4 s | 72.3 MB | 17.39 MB | **11.21 MB** | 19.5 | ~1 145 ms |
| `-O1` *(shipped default)* | 8 s | 63.3 MB | 16.12 MB | **10.61 MB** | 19.5 | ~1 090 ms |
| `-O2` | 23 s | 60.7 MB | 15.95 MB | **10.66 MB** | **21.1** | ~1 090 ms |
| `-O3` | 98 s | 60.1 MB | 15.83 MB | **10.53 MB** | 20.4 | ~1 070 ms |
| `-O4` | 132 s | 60.1 MB | 15.86 MB | **10.56 MB** | 21.8 | ~1 094 ms |
| `-Os` | 48 s | 60.0 MB | 15.85 MB | **10.53 MB** | 22.1 | ~1 068 ms |
| `-Oz` | 97 s | 57.2 MB | 15.68 MB | **10.49 MB** | 20.4 | ~1 076 ms |
Pan is the mean of two reps where two were run (none, `-O0`, `-O2`, `-Oz`);
repeats agreed within ~5%, and the `settled` time — how long until the app goes
quiet after opening — repeated within 0.4% (e.g. none 6 941 / 6 938 ms against
`-O2` 6 106 / 6 123 ms), which is what gives confidence the gaps are real.
Three things fall out:
- **Most of the size win is not optimisation.** `-O0` — which does no
optimisation at all — already captures 26.7% of the raw reduction, because the
bulk of it is the 19.56 MB name section being dropped.
- **Compressed size is flat from `-O1` onward.** Brotli is what actually goes over
the wire, and every level from `-O1` to `-Oz` lands in a 10.4910.66 MB band —
a 1.6% spread. Raw size keeps falling to `-Oz` (57.2 MB), which matters for
parse and memory but not for download.
- **Build cost explodes for nothing.** `-O3`, `-O4`, `-Os` and `-Oz` cost 48132 s
against `-O2`'s 23 s and `-O1`'s 8 s, and buy at most 1.5% more brotli. `-O4`
is not smaller than `-O3` at all (60 097 004 vs 60 091 979 bytes) while taking
35% longer.
**Recommendation: `-O2` is the value pick** — 23 s for the best measured frame
rate (+13% pan over unoptimised, versus +4% at `-O1`) and essentially the same
download as anything more expensive. The build currently defaults to `-O1` to
mirror the Asyncify-era pipeline; moving it is a one-word change to
`KICAD_WASM_OPT`. Anything past `-O2` is not worth its build time on this module.
### 6.4 Release (non-debug) build
The build that ships is the **debug** one (`DEBUG_BUILD` defaults to 1). This
tests whether dropping debug mode buys anything, with `wasm-opt -O2` held
constant on both sides so the only variable is the compile/link mode:
| | Debug | Release |
|---|---|---|
| KiCad + deps + wx TUs | `-g -O1` | `-O2` |
| link | `-O1 -g -gseparate-dwarf` | `-O0` |
| post-link | `wasm-opt -O2` | `wasm-opt -O2` |
**Consistency first.** `check_stamp()` is only `[ -f "$stamp_file" ]` — it does
not encode the build mode — so flipping `DEBUG_BUILD=0` rebuilds KiCad while
silently reusing debug-built dependencies. That matters beyond lost optimisation:
wxWidgets' ABI depends on its debug level. Every stamp was therefore wiped and wx
forced through reconfigure, and the result verified by checking for DWARF, which
only a `-g` build emits: OCCT (`libTKernel`), cairo, freetype, harfbuzz,
boost_locale, protobuf, wx base, wx core and the KiCad objects all came back
DWARF-free. Since `env.sh` ties `-g -O1` and `-O2` to the same switch, no DWARF
proves the `-O2` branch. (`docker/build.sh` already forwards `DEBUG_BUILD` into
the container for exactly this reason — its comment notes that `--release` alone
reaches KiCad's flag block but never `DEBUG_CFLAGS`/`WX_DEBUG_FLAGS`.)
**Size — release is slightly *worse*:**
| | Debug + `wasm-opt -O2` | Release + `wasm-opt -O2` | Δ |
|---|---|---|---|
| raw | 60.7 MB | 62.5 MB | **+3.0 %** |
| gzip 9 | 15.95 MB | 16.24 MB | +1.8 % |
| brotli | 10.66 MB | 10.80 MB | +1.3 % |
| code section | 48.5 MB | 50.3 MB | +3.7 % |
| data section | 12.0 MB | 12.0 MB | — |
`-O2` inlines more, so it trades size for speed — the code section grows while
data is byte-identical.
**Speed — no measurable win:**
| vme-wren, GPU | Debug + `wasm-opt -O2` | Release + `wasm-opt -O2` |
|---|---|---|
| pan GAL fps (2 reps) | 19.4 / 19.5 | 19.0 / 20.8 |
| cold load | 1 091 / 1 041 ms | 1 067 / 1 081 ms |
| settle after open | 6 790 / 6 907 ms | 6 671 / 6 003 ms |
Frame rate is **+2.3 % on the means, inside run noise** (the release arm's two
reps, 19.0 and 20.8, straddle the debug arm's pair). Cold load is identical.
Settle is ~7 % faster, the only consistent signal.
**Reading: `clang -O2` and `wasm-opt -O2` are largely redundant here — you want
one of them, not both.** The measured jump is from *no* whole-module optimisation
to *any* (+4 % at `wasm-opt -O1`, +13 % at `-O2`); which tool supplies it barely
matters. That also explains the earlier observation that unoptimised debug loaded
much slower than the optimised variants: that gap was the 19.56 MB **name
section** inflating the download, not code quality. With names stripped on both
sides here, load times converge exactly.
So debug mode is not costing meaningful speed, and release costs 13 % more
bytes while giving up all debug info — no separate DWARF, no symbolised stack
traces. On this evidence there is no reason to switch, which is a useful thing to
know: the cheap `wasm-opt` step already captured what was available, and the
remaining render cost is structural (see §6.2's reading), not a missing flag.
**Still not measured:** LTO (`-flto`) and `-msimd128`, neither of which appears
anywhere in the build. LTO is the interesting one — this is a merged multi-app
binary, so there is a large cross-TU inlining surface that neither per-file
`-O2` nor Binaryen can reach. Also unmeasured: a `-O3`/`-Oz` *link* level, whose
metadce pass (gated on `OPT_LEVEL >= 3` or `SHRINK_LEVEL >= 1`, so it does not
engage at `-O2`) would drop unused JS-library and wasm exports.
## 7. Reproduce
```bash
# build side (each arm, serialized; see §1 for scenario commands)
KICAD_NO_MONITOR=1 KICAD_KEEP_CONTAINER=1 ./docker/build.sh kicad_editor -j 10
# runtime side (from tests/, artifacts staged in tests/apps/kicad/)
cp kicad/qa/data/pcbnew/vme-wren.kicad_pcb tests/apps/kicad/board/
cp kicad/demos/jetson-agx-thor-baseboard/jetson-agx-thor-baseboard.kicad_pcb tests/apps/kicad/board/
BENCH_ARM=<arm> PERF_LARGE=1 npx playwright test --project=perf pcbnew-large-perf.spec.ts --workers=1
# → tests/bench-results/perf-bench-<arm>.ndjson
```