AI_Assistant/pcbjam
Watch
1
0
Fork
You've already forked pcbjam
 0
Code Issues Pull requests Projects Releases 4 Packages Wiki Activity Actions 2

feat(wasm-eh): migrate the WASM build to native wasm exceptions (+ 3D viewer default-on)

Browse source 
Replace the legacy Emscripten JS-exceptions model with native wasm-EH (legacy
encoding) across the whole build, keeping Asyncify coroutines working via a
from-source Binaryen --hoist-cpp-catches pre-pass. Net result: native-EH is the
only build mode, the 3D viewer is on by default, and pcbnew shrinks substantially.

Highlights:
- Binaryen submodule everywhere + --hoist-cpp-catches integration in apply-asyncify;
  post-link Asyncify covers every app wasm (not just standalone test wasm).
- Build deps (incl. OpenCASCADE without OCC_CONVERT_SIGNALS) and all KiCad apps
  with -fwasm-exceptions; emscripten_sleep added to the post-link asyncify-imports.
- libcontext fiber entry wired under native exceptions; while-loop main loop +
  currData shim injected into all wx apps.
- Native-EH collab apply fixed: DEBUG-define the embind TU + match all out-of-CMake
  C++ TUs' ABI flags to the core, fixing the vtable-layout skew / mis-dispatch.
- 3D viewer enabled by default (real raytracer linked, not the stub).
- Retire the EH-spike scaffolding; flip the asyncify-races ablation pins to
  shim-redundancy pins (native-EH stays clean with the legacy shims ablated).
- Fix the asyncify-races quiescence check to not require Asyncify.currData==0:
  under the native-EH per-frame-yield top loop the main stack is asyncify-suspended
  every frame, so currData legitimately churns (a freed-but-not-yet-nulled buffer,
  not a leak). Refresh the pcbnew toolbar screenshot baseline for the new kicad.
- CI: drop the obsolete binaryen_version input/env (the build uses the binaryen
  submodule fork's wasm-opt, not a version download); key the wasm-output cache on
  the binaryen submodule SHA instead.

Bumps the wxwidgets + binaryen submodules to their squashed feature commits.

Validated green: all 7 apps native-EH (real 3D in pcbnew); KiCad e2e 63/63
Firefox + Chromium (3D viewer renders); wx 336; coroutine 34/34 both engines;
asyncify 7/7 both engines.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Viktor Vaczi 2026-06-29 19:50:18 +02:00
commit c1ef489cfa
75 changed files with 4976 additions and 680 deletions
Download patch file Download diff file Expand all files Collapse all files

5
.github/workflows/ci-ubicloud.yml vendored
View file

@ -21,10 +21,6 @@ on:
paths-ignore: ["docs/**", "**.md"]
workflow_dispatch:
inputs:
binaryen_version:
description: "Binaryen version for the host asyncify step"
required: false
default: "130"
no_cache:
description: "Bypass the KiCad WASM output cache (force a full rebuild this run)"
type: boolean
@ -44,6 +40,5 @@ jobs:
# -O2 via release.yml). 3D viewer ON so 3d-viewer.spec.ts has a viewer.
opt_level: "-O1"
build_3d_viewer: "ON"
binaryen_version: ${{ inputs.binaryen_version || '130' }}
run_tests: true
no_cache: ${{ inputs.no_cache || false }}

25
.github/workflows/wasm-build.yml vendored
View file

@ -28,10 +28,6 @@ on:
description: "Build the WASM 3D viewer into pcbnew (ON/OFF)"
type: string
default: "ON"
binaryen_version:
description: "Binaryen version for the host asyncify step"
type: string
default: "130"
run_tests:
description: "Run the wxWidgets + KiCad e2e suites after building"
type: boolean
@ -55,7 +51,6 @@ jobs:
timeout-minutes: 300
env:
KICAD_LOG_NESTED: "1"
BINARYEN_VERSION: ${{ inputs.binaryen_version }}
# Opt level for the asyncify shrink pass (the only opt-dependent step).
BINARYEN_OPT_LEVEL: ${{ inputs.opt_level }}
BUILD_3D_VIEWER: ${{ inputs.build_3d_viewer }}
@ -77,19 +72,23 @@ jobs:
with: { node-version: 20 }
# --- cache keys --------------------------------------------------------
# base = opt-INDEPENDENT (no binaryen ver / opt level): docker compile out.
# final = opt-SPECIFIC: post-processed (asyncify + wasm-opt -O) out.
# base = opt-INDEPENDENT (no binaryen / opt level): docker compile out.
# final = opt-SPECIFIC: post-processed (asyncify + wasm-opt -O) out, keyed
# on the binaryen submodule SHA — the host post-process uses that fork's
# wasm-opt (--hoist-cpp-catches + --asyncify + -O), so bumping the fork
# must bust this cache — plus the opt level.
# Both include the 3D flag so 3D-on/off never share an entry.
- name: Compute build inputs
id: keys
run: |
KICAD=$(git -C kicad rev-parse HEAD)
WX=$(git -C wxwidgets rev-parse HEAD)
BIN=$(git -C binaryen rev-parse --short HEAD)
SC=$(node scripts/deploy/wasm-cache-hash.mjs)
EPOCH=$(cat .ci-cache-epoch 2>/dev/null || echo 0)
THREED='${{ inputs.build_3d_viewer }}'
BASE="kbase-${{ runner.os }}-k${KICAD}-wx${WX}-sc${SC}-3d${THREED}-e${EPOCH}"
FINAL="kwasm-${{ runner.os }}-bin${{ inputs.binaryen_version }}${{ inputs.opt_level }}-k${KICAD}-wx${WX}-sc${SC}-3d${THREED}-e${EPOCH}"
FINAL="kwasm-${{ runner.os }}-bin${BIN}${{ inputs.opt_level }}-k${KICAD}-wx${WX}-sc${SC}-3d${THREED}-e${EPOCH}"
{
echo "kicad=$KICAD"; echo "wx=$WX"; echo "sc=$SC"; echo "epoch=$EPOCH"
echo "base_key=$BASE"; echo "final_key=$FINAL"
@ -167,7 +166,6 @@ jobs:
if: steps.final-cache.outputs.cache-hit != 'true' && steps.base-cache.outputs.cache-hit != 'true'
run: |
export KICAD_DOCKER_CPUS="$(( $(nproc) - 1 ))" KICAD_DOCKER_MEM=110G
export BINARYEN_BUILD_FROM_SOURCE=1
echo "Compiling ALL tools (base wasm), 3D viewer=${BUILD_3D_VIEWER}, -j $(nproc)"
./docker/build.sh all --compile-only --build-deps -j "$(nproc)"
ls -lh output/*.wasm
@ -207,14 +205,15 @@ jobs:
# PHASE 2 (any final miss): pure-host post-process on the base wasm —
# dyncall + finalize + asyncify + `wasm-opt ${opt_level}`. The ONLY
# opt-dependent work. No container; get-wasm-opt self-provisions Binaryen.
# opt-dependent work. No container; the binaryen submodule fork's wasm-opt
# is built on demand via scripts/binaryen-hoist-pass/build-wasm-opt.sh.
- name: Host post-process (asyncify + wasm-opt ${{ inputs.opt_level }})
if: steps.final-cache.outputs.cache-hit != 'true'
run: |
export KICAD_PIPELINE=1 BINARYEN_CORES=16 BINARYEN_BUILD_FROM_SOURCE=1
echo "Post-processing ALL tools with ${BINARYEN_OPT_LEVEL}, BINARYEN_VERSION=${BINARYEN_VERSION}"
export KICAD_PIPELINE=1 BINARYEN_CORES=16
echo "Post-processing ALL tools with ${BINARYEN_OPT_LEVEL}"
./docker/build.sh all --postprocess-only
echo "wasm-opt used:"; ./scripts/common/get-wasm-opt.sh --version 2>/dev/null || true
echo "wasm-opt used:"; "$(./scripts/binaryen-hoist-pass/build-wasm-opt.sh 2>/dev/null)" --version || true
ls -lh output/*.wasm
- name: Save FINAL WASM output cache

4
.gitmodules vendored
View file

@ -10,3 +10,7 @@
path = web/pcbjam-shared
url = https://github.com/emergence-engineering/pcbjam-shared.git
branch = main
[submodule "binaryen"]
path = binaryen
url = https://github.com/emergence-engineering/binaryen
branch = wasm-port

1
binaryen Submodule

@ -0,0 +1 @@
Subproject commit a6c26e3c606e381603b2763a1b2f10f3ddb2ca07

11
docker/Dockerfile
View file

@ -25,10 +25,13 @@ RUN apt-get update && apt-get install -y \
unzip \
&& rm -rf /var/lib/apt/lists/*
# Install emsdk from source (same approach as scripts/setup-emsdk.sh)
# EMSCRIPTEN_VERSION must match scripts/common/versions.sh
ARG EMSCRIPTEN_VERSION=4.0.2
RUN git clone https://github.com/emscripten-core/emsdk.git /emsdk \
# Install emsdk from source (same approach as scripts/setup-emsdk.sh). The version is the SINGLE
# source of truth in scripts/common/versions.sh, passed in as a build arg by docker/build.sh (via the
# compose build.args). Do NOT hardcode it here — fail fast if the arg is missing so a stale image can
# never silently use the wrong toolchain.
ARG EMSCRIPTEN_VERSION
RUN test -n "${EMSCRIPTEN_VERSION}" || { echo "EMSCRIPTEN_VERSION build-arg required (scripts/common/versions.sh)"; exit 1; }; \
git clone https://github.com/emscripten-core/emsdk.git /emsdk \
&& cd /emsdk \
&& ./emsdk install ${EMSCRIPTEN_VERSION} \
&& ./emsdk activate ${EMSCRIPTEN_VERSION}

26
docker/build.sh
View file

@ -22,7 +22,7 @@
#
# The build is split into two phases:
# 1. Docker: Compile KiCad to WASM (without asyncify)
# 2. Host: dyncall shims + finalize + asyncify + -O2 (Binaryen via get-wasm-opt.sh)
# 2. Host: dyncall shims + finalize + asyncify + -O2 (Binaryen submodule via build-wasm-opt.sh)
#
# KICAD_PIPELINE=1 (multi-app builds only): run phase 2 of each app in the
# background while the next app compiles in the container. wasm-opt is
@ -49,6 +49,11 @@ source "$(dirname "$0")/../scripts/common/logging.sh"
# Build-progress markers (parsed by scripts/build-monitor.sh).
source "$(dirname "$0")/../scripts/common/stages.sh"
# Pinned toolchain version (single source of truth). Exported so the compose build.args can pass it
# into the Docker image's emsdk install — bumping the toolchain is then a one-line edit in versions.sh.
source "$(dirname "$0")/../scripts/common/versions.sh"
export EMSCRIPTEN_VERSION
set -e
# Emit a completion/failure marker no matter how the build ends, so the monitor
@ -120,7 +125,7 @@ echo "Building app: ${APP_NAME}"
# --postprocess-only — only the host post-process (dyncall + finalize +
# asyncify + wasm-opt -O$BINARYEN_OPT_LEVEL) on the
# existing output/ base wasm; NO container needed
# (get-wasm-opt.sh self-provisions Binaryen).
# (build-wasm-opt.sh self-provisions the Binaryen submodule).
# Extracted here so they are NOT forwarded to the inner build-<app>.sh scripts.
PHASE="both"
_FILTERED=()
@ -143,8 +148,9 @@ fi
# which is pure host work on the already-built base wasm in output/.
if [[ "$PHASE" != "postprocess" ]]; then
# Start container if not running
docker compose -f docker/docker-compose.yml up -d
# Start container if not running. --build so the image is rebuilt when the pinned EMSCRIPTEN_VERSION
# (build-arg from versions.sh) changes; Docker layer-caches it to a near no-op when unchanged.
docker compose -f docker/docker-compose.yml up -d --build
# Sync source code to container volume (fixes macOS Docker VirtioFS issues)
# Use --checksum to only transfer files with different CONTENT, not timestamps.
@ -217,7 +223,7 @@ compile_app() {
# emsdk_env.sh, so the build shell would lack emcc/embuilder on PATH. Setting
# EMSDK lets scripts/common/env.sh source /emsdk/emsdk_env.sh and activate the toolchain.
docker compose -f docker/docker-compose.yml exec -e EMSDK=/emsdk \
-e BUILD_3D_VIEWER="${BUILD_3D_VIEWER:-OFF}" \
-e BUILD_3D_VIEWER="${BUILD_3D_VIEWER:-ON}" \
kicad-wasm-builder \
"/workspace/scripts/kicad/build-${app}.sh" "${ARGS[@]}"
@ -266,6 +272,8 @@ postprocess_app() {
# Apply asyncify transformation on host. The converter is a synchronous node
# CLI built with ASYNCIFY=0, so asyncify is unnecessary and would be wrong.
# apply-asyncify always runs the --hoist-cpp-catches pass FIRST (native wasm-EH is the only build
# mode) so Asyncify can suspend from inside C++ catch arms, then asyncify + removelist + -O2.
if [ "$app" != "sym_convert" ]; then
kw_stage asyncify
./scripts/common/apply-asyncify.sh "${out_dir}/${app}.wasm" "${out_dir}/${app}.wasm"
@ -349,7 +357,7 @@ elif [[ "$PHASE" == "postprocess" ]]; then
# wasm (no container). Parallelize across apps when pipelining.
if [[ "${KICAD_PIPELINE:-0}" == "1" ]] && [ "$TOTAL_APPS" -gt 1 ]; then
mkdir -p "$PIPELINE_LOG_DIR"
./scripts/common/get-wasm-opt.sh >/dev/null # pre-warm Binaryen once
./scripts/binaryen-hoist-pass/build-wasm-opt.sh >/dev/null # pre-warm Binaryen (submodule) once
_install_pipeline_trap
for app in "${APPS[@]}"; do
pipeline_postprocess "$app"
@ -364,9 +372,9 @@ elif [[ "${KICAD_PIPELINE:-0}" == "1" ]] && [ "$TOTAL_APPS" -gt 1 ]; then
# both, pipelined: overlap app[i+1]'s container compile with app[i]'s host
# post-process (KICAD_PIPELINE=1).
mkdir -p "$PIPELINE_LOG_DIR"
# Pre-warm the Binaryen download once — two concurrent postprocesses racing
# the first download would collide on the extract/mv.
./scripts/common/get-wasm-opt.sh >/dev/null
# Pre-build the Binaryen submodule once — two concurrent postprocesses racing
# the first from-source build would collide.
./scripts/binaryen-hoist-pass/build-wasm-opt.sh >/dev/null
_install_pipeline_trap
idx=1
for app in "${APPS[@]}"; do

5
docker/docker-compose.yml
View file

@ -3,6 +3,11 @@ services:
build:
context: ..
dockerfile: docker/Dockerfile
args:
# Single source of truth: scripts/common/versions.sh, exported into the env by the scripts
# that drive compose (docker/build.sh, docker/shell.sh). No default -> the Dockerfile fails
# fast if it's unset, so the image can't be built against the wrong toolchain.
EMSCRIPTEN_VERSION: ${EMSCRIPTEN_VERSION:?source scripts/common/versions.sh before docker compose}
# Container name is auto-generated with project prefix (set in build.sh)
# Resource limits. Defaults are sized for a dev Mac (Docker Desktop VM).

4
docker/shell.sh
View file

@ -4,6 +4,10 @@ set -e
cd "$(dirname "$0")/.."
# Pinned toolchain version (single source of truth) -> compose build.args needs it in the env.
source "$(dirname "$0")/../scripts/common/versions.sh"
export EMSCRIPTEN_VERSION
# Start container if not running
docker compose -f docker/docker-compose.yml up -d

2
docs/features/async/02-asyncify-internals.md
View file

@ -1,5 +1,7 @@
# 02 — The machine: Asyncify internals and control flows
> **STATUS (2026-06-23):** the top-level `emscripten_set_main_loop(...,1)` `throw "unwind"` this doc treats as current **is gone** — it was fatal under native wasm-EH and was replaced by the Asyncify **de-park** rAF pump (`wxWasmParkMainLoop`); see [`../wasm-exceptions/09`](../wasm-exceptions/09-event-loop-deparking-plan.md). The de-park regressed the coroutine suite, and **Design B is now being built to fix it** ([`12`](12-design-b-asyncify-implementation-plan.md) + [`13`](13-design-b-engineering-spec.md)). Read below as the pre-de-park analysis (the internals are still accurate).
This is the legible model: what suspends, who owns the single slot, and exact line-by-line
control flow for the park, the hang, the crash, and **de-parking**.

2
docs/features/async/04-decisions-tests-open-questions.md
View file

@ -1,5 +1,7 @@
# 04 — How the fixes relate, the test matrix, open questions
> **STATUS (2026-06-23):** the top-level `set_main_loop(...,1)` `throw "unwind"` treated as current here **is gone** — replaced by the Asyncify **de-park** rAF pump (fatal under native wasm-EH; see [`../wasm-exceptions/09`](../wasm-exceptions/09-event-loop-deparking-plan.md)). The de-park regressed the coroutine suite, and **Design B is now being built to fix it** ([`12`](12-design-b-asyncify-implementation-plan.md) + [`13`](13-design-b-engineering-spec.md)). Read below as the pre-de-park analysis.
> The goal is **one universal mechanism**, not patches scattered around. This file classifies the
> candidate fixes by *root cause* so it's clear what is part of the one solution, what is
> subsumed, and what is genuinely separate.

2
docs/features/async/05-design-a-js-asyncify-arbiter.md
View file

@ -1,5 +1,7 @@
# 05 - Design A: JS Asyncify arbiter
> **STATUS (2026-06-23):** the throw-based top loop assumed here **is gone** (de-park; [`../wasm-exceptions/09`](../wasm-exceptions/09-event-loop-deparking-plan.md)). This arbiter is the **core of Design B's scheduler**, now being implemented — the de-park created the red scenario (coroutine regression) that 07/D3 said this arbiter lacked. See [`12`](12-design-b-asyncify-implementation-plan.md) + [`13`](13-design-b-engineering-spec.md); this doc's arbiter design is reused there.
> Goal: fix the current system with the smallest architectural move. Keep
> `EM_ASYNC_JS` modal/clipboard/font calls and Emscripten fibers, but introduce one JS-side
> authority that owns `Asyncify.currData`, `Asyncify.state` transitions, the fiber trampoline,

2
docs/features/async/06-design-b-fiber-first-runtime.md
View file

@ -1,5 +1,7 @@
# 06 - Design B: fiber-first async runtime
> **STATUS (2026-06-23):** Design B is **now being implemented** on Asyncify — see [`12`](12-design-b-asyncify-implementation-plan.md) (plan/phases/test-matrix) and [`13`](13-design-b-engineering-spec.md) (engineering spec/work log). The de-park ([`../wasm-exceptions/09`](../wasm-exceptions/09-event-loop-deparking-plan.md)) replaced the top-level `throw` with an Asyncify park and regressed the coroutine suite — the red scenario this design fixes. External research (Ruby-WASM, Julia-WASM, Qt-for-WASM) confirms the fiber-scheduler is the proven path.
> Goal: make the architecture conceptually cleaner by reducing the number of suspension
> primitives. Instead of having tool coroutines use fibers while modal/clipboard/font/nested loops
> use `EM_ASYNC_JS` sleeps, put every blocking-looking operation onto a fiber-like runtime and let

2
docs/features/async/07-decisions-and-outcome.md
View file

@ -1,5 +1,7 @@
# 07 — Decisions and outcome (2026-06-12)
> **STATUS (2026-06-23):** **D4 (kept the throw-based main-loop park) has been reversed.** Native wasm-EH made the `throw "unwind"` fatal (its catch_all cleanup destroys the main frame), so the top loop is now the Asyncify **de-park** ([`../wasm-exceptions/09`](../wasm-exceptions/09-event-loop-deparking-plan.md)). That de-park regressed the coroutine suite — the red scenario D3 said the arbiter lacked — so **Design B is now being built** ([`12`](12-design-b-asyncify-implementation-plan.md) + [`13`](13-design-b-engineering-spec.md)). The D1–D5 outcomes below were correct for the JS-EH / throw world.
> The dossier (01–06) ended with designs and open questions. This file records what was
> actually decided, built, and deliberately NOT built — and the trigger conditions for
> revisiting each road not taken. Working artifacts: `docs/features/asyncify-arbiter/`

103
docs/features/async/12-design-b-asyncify-implementation-plan.md Normal file
View file

@ -0,0 +1,103 @@
# 12 — Design B on Asyncify: implementation plan to make suspensions compose
> How to realize the dossier's **Design B** ([`06`](06-design-b-fiber-first-runtime.md)) on the
> **current Asyncify toolchain — no JSPI**. The goal is concrete: make the parked main loop, modal
> dialogs, nested loops, clipboard/font waits, and **tool coroutines** all coexist without
> corrupting Asyncify's single suspension slot, so the de-park's coroutine regression goes green —
> under both `-fexceptions` and `-fwasm-exceptions`. Builds on the internal audit (current building
> blocks, the gap) and Design A ([`05`](05-design-a-js-asyncify-arbiter.md)).
## Why now — the red scenario the dossier didn't have
Doc 07/D3 shelved Design A's arbiter because *"no scenario could be made red that it would fix"* — at production semantics the per-sleep `handlesleep.js` capture already satisfied the core invariant. **The de-park changed that.** Measured in `wasm-exceptions/09`: config 3 (JS-EH + de-park) **fails** the 6 coroutine tests that config 1 (no de-park) **passes**. We now have a deterministic red test that only a scheduler fixes. The de-park and Design B are **coupled**: the de-park is *required* for native-EH (the `throw "unwind"` is fatal under wasm-EH catch_all), it breaks coroutines, and Design B is the fix.
## The physics we must obey (why this is hard)
Asyncify is **one** `Asyncify.currData` (active save-buffer pointer) + **one** `Asyncify.state` (`Normal`/`Unwinding`/`Rewinding`). The law: **at most one unwind-or-rewind in flight at a time**; it must begin at `state==Normal` and fully complete before the next. But **many contexts may be *parked* at once**, each holding its own durable buffer (a parked context = `state Normal`, its stack saved in *its* buffer, waiting for a wake).
- **Fibers** (KiCad tool coroutines via libcontext) already give per-context buffers (`wasm_fcontext.asyncify_stack`, 64 KB each) — durable storage is fine. But the *act* of swapping still drives the single global register.
- **`handleSleep`/`handleAsync`** (modals, clipboard, the de-park park, the per-tick `ccall`) take buffers from `Asyncify.allocateData`; only the live `currData` register remembers a parked one — `handlesleep.js` patches this for **one** level of sleep nesting, and is **blind to `handleAsync` and to fibers**.
**The de-park bug, precisely:** `wxWasmParkMainLoop` is a `handleAsync` suspend that is **live for the app's whole life**, and each rAF tick's `await ccall('ProcessEvents',{async:true})` is a second `handleAsync` suspend. A coroutine fiber-swap is then a **third** unwind, attempted while the slot is dirty / `state != Normal` → `Aborted(invalid state: 1)`. Three uncoordinated writers of one slot.
## Architecture: one scheduler owns the slot; everything is a context
The universal rule (06): **no API touches `Asyncify` directly. APIs ask the scheduler to park/wake contexts. The scheduler alone performs Asyncify transitions.**
```
Scheduler (JS) — the single authority
owns: Asyncify.currData, Asyncify.state, the fiber trampoline
registry: ctx = { id, kind: main|modal|nested|coroutine|sleep, buffer, status, wakeReason, result }
readyQueue + drain():
a wake event marks a ctx READY (it does NOT rewind directly)
drain() resumes the next ready ctx ONLY when state==Normal && no transition in flight
transitions (the only code that writes currData):
park(ctx) = set currData=ctx.buffer; start_unwind; (slot now free, ctx parked)
resume(ctx) = set currData=ctx.buffer; start_rewind; doRewind
```
Every blocking-looking thing becomes a context that *yields* and is later *resumed*. A coroutine swap becomes "park ctx A, resume ctx B" — a normal scheduler operation serialized with the main loop and modals, exactly the doc-11 cure: *"if the pump and render yields were both scheduler-owned fiber contexts, 'render yields while the pump is parked' becomes a normal context switch instead of an illegal nested unwind."*
## The gap — what to build (from the internal audit)
None of these exist today: (1) a single owner of `currData`/`state`/the trampoline; (2) a scheduler-owned **fiber** context for the main loop + pump (today it's a `handleAsync` park, not a fiber); (3) a deferred-wakeup ready-queue/drain (today wakeups call `doRewind` inline); (4) **`handleAsync` coverage** (the park + the per-tick ccall are entirely unprotected); (5) sleep contexts promoted from "restore one pointer" to "registered context"; (6) trampoline ownership as a scheduler invariant; (7) a lifetime owner coordinated with `currData` management (the de-park gave us D1/lifetime without D2/ownership).
## Phased implementation (each phase gated by the red harness)
**Phase 0 — Red-green harness (2–3 days).** Make the coroutine regression a deterministic, minimal red test in `tests/asyncify/` (and a CPP test app): a tool-style fiber swap *while the main loop is parked* and *while a modal pump is live*. Reproduce `invalid state: 1` reliably in all three engines. This is the acceptance gate for every later phase. Also fold the 6 failing `coroutine*` specs in as the integration gate.
**Phase 1 — The scheduler core (1–2 weeks). The likely coroutine fix.** Extend `scripts/common/shims/handlesleep.js` into the `AsyncifyArbiter` of doc 05, but covering everything the de-park introduced:
- Own `Asyncify.currData`/`state` + `Fibers.trampoline`; make `currData` a *derived* register set only inside a managed transition; the registry records are the truth.
- **Register `handleAsync`** (wrap it as `handleSleep` is wrapped) so the de-park park and the per-tick `ccall` are tracked contexts, not invisible slot-writers.
- **Track fiber buffers** at `_emscripten_fiber_swap` (`oldFiber+20`/`newFiber+20`) so a coroutine swap is a managed transition.
- **Deferred-wakeup `drain()`** with *explicit* completion signals from `stop_rewind`/`maybeStopUnwind` (not JS `finally` — `doRewind` can re-enter and unwind again before returning).
- Keep it JS-only — **no C++ restructuring yet.** Build, run Phase 0. If green, the coroutine regression is fixed at lowest risk. If still red (the permanent `handleAsync` park can't be made a clean parked context), escalate to Phase 2.
**Phase 2 — Root fiber for the main loop (≈1 week). The clean cure.** Replace the `handleAsync` park with a scheduler-owned **fiber**: run `main → wxEntry → OnRun → DoRun` inside a managed root fiber (06's B2). The main loop *yields its fiber* to the scheduler instead of `handleAsync`-parking. Crucially, **drive `ProcessEvents` from the wasm-side scheduler (the root fiber calls it directly), not the current JS-side `await ccall('ProcessEvents',{async:true})`** — that JS-awaits-a-suspending-export boundary is the Emscripten #13302 corruption hazard (see Prior art). The rAF/`setTimeout` tick just resumes the root fiber (or returns through `set_main_loop`, Ruby/Julia-style, to keep the top off Asyncify entirely). Now the main loop is a sibling context to the coroutines and modals — no permanent `handleAsync` occupant, every swap is fiber↔fiber under the scheduler. This is the definitive fix if Phase 1's "park-as-context" proves fragile.
**Phase 3 — Migrate the waits to one yield API (1–2 weeks). The full Design B.** Add the C++ API and route the ad-hoc suspends through it:
```cpp
WAKE_TOKEN wasm_begin_async_wait(...);
int wasm_yield_until(WAKE_TOKEN); // park current ctx, run scheduler
void wasm_resolve_wait(WAKE_TOKEN, int result); // mark ctx ready
```
Reimplement `wxDialog::ShowModal` (`dialog.cpp` `startModal`), `wxGUIEventLoop` nested `DoRun` (`evtloop.cpp` `wxWasmRunNestedLoop`), clipboard, and font enum as `yield_until` waits. Removes the second suspension family entirely; the LIFO resolver stacks (`_wxModalResolvers`, `_wxNestedLoopExit`) become scheduler ready/wait bookkeeping.
**Phase 4 — Lifetime + cleanup (few days).** Coordinate the de-park's lifetime (D1) with the scheduler (D2): the browser/scheduler owns app lifetime; `wxEntryCleanupReal`/`OnExit` deferred to real exit/unload; `emscripten_cancel_main_loop` + teardown ordered after the root fiber resolves. (Already half-done in `evtloop.cpp` `ScheduleExit`.)
## Test matrix (doc 06 + nesting + EH)
Each asserts **no crash, no hang, correct return value, app stays interactive, no cleanup during steady-state pumping** — and runs in **Firefox + Chrome + Safari** and under **both `-fexceptions` and `-fwasm-exceptions`**:
- the 6 regressed `coroutine`/`coroutine-nested`/`coroutine-pthread` specs (the gate);
- `ShowModal` from root, from a tool coroutine; `ShowQuasiModal` from a coroutine; nested modal inside quasi-modal;
- coroutine swap **while** a modal pump is live (the Phase-0 red test);
- clipboard read from root and from a coroutine; font enum during startup;
- the **raytracer** threading suite (the doc-11 nesting wall) multi-core;
- exit/unload cleanup after parked contexts exist;
- a `-sASYNCIFY_ASSERTIONS=1` pass + a production `-sASSERTIONS=0` pass (the dossier's "production semantics already satisfy the invariant" claim must be re-validated post-de-park).
## Risks + mitigations
- **A partial arbiter is worse than none** (doc 05) — one path still writing `currData` behind the scheduler corrupts silently. → enumerate *every* `currData` writer (`handleSleep`, `handleAsync`, `_emscripten_fiber_swap`, `finishContextSwitch`, the park, the ccall), route all through the scheduler, assert on stray writes in dev builds.
- **Trampoline wedge** (`Fibers.trampolineRunning` stuck after a mid-flight unwind). → scheduler *owns* the trampoline; keep the `inject-dyncall-shims §3c` self-heal as a belt-and-suspenders.
- **Lifetime cleanup too early** → Phase 4 ordering; defer wx teardown to unload.
- **Reentrancy / out-of-order resolution** → explicit tests; keep wx modal-disabling semantics.
- **Native-EH coexistence** — the scheduler and the `HoistCppCatches` pass must compose (suspend-inside-catch under the scheduler). → test the whole matrix under `-fwasm-exceptions`, including a modal opened from inside a `catch`.
- **Starvation** → FIFO ready-queue; diagnostics for context age.
## Effort
Phase 0 ≈ 2–3 d · Phase 1 ≈ 1–2 wk · Phase 2 ≈ 1 wk · Phase 3 ≈ 1–2 wk · Phase 4 ≈ few d. **Coroutine fix = Phase 0–1 (+2 if needed) ≈ 2–3 wk; full Design B ≈ 4–6 wk** including the test matrix. Phase 1 is the high-value, lowest-risk step and may suffice on its own.
## Prior art (external research)
**The Asyncify fiber scheduler (this plan) is the proven path** for "event loop + blocking `ShowModal` + green threads" — it ships in real runtimes, and the comparisons sharpen two implementation details.
- **Qt for WebAssembly** (closest analog) uses a deliberate **two-tier** scheme: top-level `QApplication::exec()` uses Emscripten's `simulateInfiniteLoop` throw — which keeps the top loop **off Asyncify so the single slot stays free** — while `QDialog::exec()`/nested `QEventLoop::exec()` consume the one Asyncify slot. **This is the inverse of what our de-park did** (the de-park made the *top* loop a live Asyncify occupant, consuming the slot — exactly why coroutines broke). Qt's scheme caps at *one* modal at a time and the Qt team calls Asyncify "not quite scaling to Qt-sized software" — i.e. a single-slot scheme *without* a real scheduler hits a wall; the per-fiber-buffer scheduler (Design B) is the way past it. [Qt commit 6d039a5e; Qt dev ML, June 2024]
- **Ruby-WASM / Julia-WASM** implement this design directly: a **root fiber that *is* the browser event loop**; tasks/coroutines are fibers, each with its own C stack + `asyncify_data` buffer; the scheduler resumes the next ready fiber; "yield to browser" is `emscripten_sleep(0)` *or* returning through the `set_main_loop` callback (the latter keeps the top off Asyncify entirely). That is precisely Design B's B2 root-fiber — already shipping in production runtimes. [Julia PR #32532; Emscripten fiber PR #9859]
- **Pyodide** pre-JSPI used stackless CPS (`WebLoop` + `setTimeout(0)` per task) — not retrofittable to C++; post-JSPI uses per-`promising`-entry stacks plus explicit **spill-stack** save/restore.
- **Dart/Flutter, Blazor, Unity** all use compiler-lowered stackless state machines — not applicable to a C++ toolkit.
**Two findings that sharpen the plan:**
1. **The JS-boundary async-return hazard (Emscripten #13302):** returning a value to JS from a wasm export that internally `fiber_swap`s is broken — *"within-wasm scheduling is fine; the JS-awaits-a-suspending-wasm-export boundary is not."* Our per-tick `await ccall('ProcessEvents',{async:true})` is exactly that boundary. **Design B must drive `ProcessEvents` from the wasm-side scheduler (the root fiber), not via a JS async `ccall`** — folded into Phase 2 below.
2. **JSPI would not have helped this case anyway** (independently confirming the decision to scratch it): the wit-bindgen analysis shows that when the whole scheduler lives inside one app context, *a single `promising` root = a single suspension unit* — JSPI gives no fiber-multiplexing benefit unless each fiber is separately surfaced as a `promising` export (awkward; Chrome also showed a ~350× per-suspension penalty on the JS→wasm path). Pattern-C / Asyncify is the right tool regardless of JSPI availability.

163
docs/features/async/13-design-b-engineering-spec.md Normal file
View file

@ -0,0 +1,163 @@
# 13 — Design B: engineering spec & work log
> The granular, file-by-file implementation spec for [`12`](12-design-b-asyncify-implementation-plan.md)
> (the plan/phases/test-matrix). This document is the **engineering design + per-phase checklists**,
> and is updated as a **work log** as the phases land. Build it on Asyncify — no JSPI.
## 0. State of the world (2026-06-23)
- The **de-park is live**: `wxwidgets/src/wasm/evtloop.cpp` `DoRun` at depth 0 calls `wxWasmParkMainLoop()` (an `EM_ASYNC_JS`/`Asyncify.handleAsync` suspend driving an rAF `await ccall('ProcessEvents',{async:true})` pump). The old `emscripten_set_main_loop(...,1)` throw is gone (it was fatal under native wasm-EH; see `wasm-exceptions/08`+`09`).
- **Measured regression** (`wasm-exceptions/09`): config 1 (no de-park) passes all; config 3 (JS-EH + de-park) fails the 6 `coroutine`/`coroutine-nested`/`coroutine-pthread` specs; config 2 (native-EH + de-park) fails those 6 + raytracer(5) + main-app(10). The **6 coroutine failures are the de-park's**, both EH models.
- **The gate** for this work = those 6 specs going green again, plus a minimal unit repro (Phase 0).
- **Stale code to clean up in Phase 1:** `scripts/common/shims/handlesleep.js` still has the `"unwind"`-sentinel swallow (lines ~57-68) referencing `set_main_loop(...,1)` — dead under the de-park; the rewrite subsumes it.
## 1. The scheduler — JS design (the heart of the fix)
### 1.1 What exists today (`handlesleep.js`)
Per-sleep `currData` capture/restore for **one** level of nesting, **`handleSleep` only**:
- Wraps `Asyncify.allocateData` to record which buffer pointer the active `handleSleep` allocated (`ctx.capturedData`).
- In the `wakeUp` callback, restores `Asyncify.currData = ctx.capturedData` before `handleSleep` does `_asyncify_start_rewind`+`doRewind`, so a fiber swap that clobbered the slot during the `await` doesn't make the sleep rewind the wrong buffer.
- **Blind to `handleAsync`** (the de-park park + the per-tick ccall) **and to fibers** (libcontext buffers don't come from `allocateData`).
### 1.2 The scheduler object (`AsyncifyScheduler`, replaces the shim)
A single JS authority that is the **only** writer of `Asyncify.currData` during managed transitions. State:
```
contexts: Map<id, ctx> // every parked/running suspendable thing
readyQueue: id[] // FIFO of contexts whose wake fired
running: id | null // the one context currently executing
transitionRunning: bool // an unwind or rewind is in flight
trampolineRunning: bool // a fiber-swap trampoline is mid-flight
```
`ctx = { id, kind: 'main'|'modal'|'nested'|'coroutine'|'sleep', buffer /*dataPtr*/, status: 'running'|'parked'|'ready', wakeReason, result, cancel }`.
**Governing rule (from doc 05):** `Asyncify.currData` is *not* durable state — it is a register loaded from the current context only at the instant of a managed transition. The `contexts` records are the truth. **Many parked; at most one unwinding-or-rewinding.**
### 1.3 The four hooks (intercept every `currData` writer)
1. **`Asyncify.handleSleep`** — register a `sleep` ctx (today's capture), but route its wakeup through `drain()` (below), not an inline `doRewind`.
2. **`Asyncify.handleAsync`** — wrap it the same way. **New, load-bearing:** the de-park park and the per-tick `await ccall` are `handleAsync`; they must be tracked contexts, not invisible slot-writers.
3. **`_emscripten_fiber_swap`** — *track* (not allocate) the per-fiber buffers `oldFiber+20` / `newFiber+20` so a coroutine swap is a managed transition the scheduler knows about.
4. **`Fibers.trampoline`** — own it (and the `trampolineRunning` guard); keep `inject-dyncall-shims §3c` self-heal as backstop.
### 1.4 The transitions (the only code that writes `currData`)
```
park(ctx): assert state==Normal; currData=ctx.buffer; start_unwind // ctx now parked, slot free
resume(ctx): assert state==Normal; currData=ctx.buffer; start_rewind; doRewind
drain(): if (transitionRunning || trampolineRunning || state!=Normal || !readyQueue.length) return;
resume(contexts[readyQueue.shift()])
```
**Deferred wakeup:** a Promise/event resolution **marks a ctx ready and calls `scheduleDrain()`** — it never calls `doRewind` inline (because `doRewind` can re-enter wasm and unwind again before returning). `drain` runs only when the slot is provably free, and receives **explicit transition-completion signals** by wrapping `_asyncify_stop_rewind` / `Asyncify.maybeStopUnwind` (clear `transitionRunning`, then `scheduleDrain()`), not a JS `finally`.
### 1.5 Invariants (assert in dev builds; doc 05 §invariants)
(1) only the scheduler writes `currData` during managed transitions; (2) ≤1 context unwinding-or-rewinding; (3) Promise resolution never `doRewind`s directly while a transition runs; (4) `currData` may be null while contexts are parked — records are truth; (5) every `allocateData`/fiber buffer belongs to exactly one ctx; (6) the scheduler owns the trampoline; (7) a parked ctx's buffer is never reused until it resumes-and-completes; (8) FIFO readyQueue (no starvation).
> **Correction (verified in the glue during Phase 0, 2026-06-23):** `handleAsync` routes through the wrapped `handleSleep` — `handleAsync(fn) = handleSleep(wakeUp => fn().then(wakeUp))` (`coroutine_test.js:9989`) — so the shim **already covers** the de-park park (`wxWasmParkMainLoop`) and the per-tick `await ccall`. §1.3's "hook 2 (handleAsync) is load-bearing/new" is therefore **wrong**: no separate `handleAsync` hook is needed. The genuine *uncovered* `currData` writer is the **fiber swap** (libcontext buffers come from `emscripten_fiber_init`, not `allocateData`). **So Phase 1's scheduler should focus on fiber tracking (§1.3 hook 3) + the deferred drain (§1.4) + single-transition serialization — not handleAsync coverage.** The Phase-0 red gate is confirmed (6 `coroutine*` specs fail on the de-park build); this sharpens where the fix lives.
## 2. The C++ yield API (Phase 3 surface)
```cpp
using WAKE_TOKEN = int;
WAKE_TOKEN wasm_begin_async_wait(int kind); // EM_JS → scheduler.beginWait(kind) → token
int wasm_yield_until(WAKE_TOKEN token); // EM_ASYNC_JS → park current ctx, return result on resume
void wasm_resolve_wait(WAKE_TOKEN, int); // EM_JS → mark ctx ready + scheduleDrain
```
Reimplement each wait on top of it: `wxDialog::ShowModal` (replaces `dialog.cpp:startModal` `EM_ASYNC_JS` + `_wxModalResolvers`), `wxGUIEventLoop` nested `DoRun` (replaces `evtloop.cpp:wxWasmRunNestedLoop`), `wxClipboard::GetData`, font enum. Each becomes "begin wait → yield_until → (JS event) resolve_wait". The existing LIFO resolver stacks fold into scheduler ready/wait bookkeeping.
## 3. The root fiber (Phase 2, B2)
Run `main → wxEntry → OnRun → DoRun` inside a managed **root fiber** (via libcontext's `emscripten_fiber_init_from_current_context`, already used for the coroutine main stack at `libcontext.cpp:202-217` — generalize it to the app root). At depth 0, `DoRun` **yields the root fiber to the scheduler** instead of `handleAsync`-parking. The browser tick (rAF or `set_main_loop` callback) **resumes the root fiber**, which calls `ProcessEvents` **directly (wasm-side), not via `await ccall(...,{async:true})`** — that JS-awaits-a-suspending-export boundary is the Emscripten #13302 corruption hazard. Now the main loop is a sibling fiber to coroutines/modals; a coroutine swap is fiber↔fiber under the scheduler — no nested unwind.
## 4. File-by-file change map
| File | Change | Phase |
|---|---|---|
| `scripts/common/shims/handlesleep.js` | → `asyncify-scheduler.js`: the scheduler (1.2–1.5); cover `handleAsync` + fiber tracking + deferred drain; drop the stale `"unwind"` swallow | 1 |
| `scripts/common/inject-dyncall-shims.sh` | inject the new scheduler; keep §3c self-heal | 1 |
| `wxwidgets/src/wasm/evtloop.cpp` | `DoRun` top-level → root-fiber yield; `ProcessEvents` driven wasm-side; `ScheduleExit` → scheduler wake | 2,3 |
| `wxwidgets/src/wasm/dialog.cpp` | `ShowModal`/`EndModal` → `wasm_yield_until`/`wasm_resolve_wait` | 3 |
| `kicad/thirdparty/libcontext/libcontext.cpp` | register fiber create/swap with the scheduler; expose the root-fiber init | 2 |
| wx clipboard/font wasm files | → yield API | 3 |
| `tests/apps/standalone/coroutine*`, `*raytrace*` | the integration gate (already exist) | 0 |
| `tests/apps/standalone/sched-nest/` (new) | the minimal Phase-0 unit repro | 0 |
| `tests/asyncify/*.spec.ts` | red-green specs for the harness, 3 engines, both EH | 0,1 |
## 5. Test harness
- **Phase 0 minimal repro:** a tiny `wxIMPLEMENT_APP` that, from a `CallAfter`/timer (i.e. inside the parked rAF pump), does a libcontext fiber swap and swaps back; assert no `invalid state: 1`, correct round-trip value. RED under the current de-park; the unit gate for Phase 1.
- **Integration gate:** the 6 `coroutine*` specs (already RED under de-park).
- **Full matrix (12 §test-matrix):** `ShowModal` from root & from coroutine; nested modal in quasi-modal; coroutine swap while a modal pumps; clipboard from root & coroutine; raytracer multi-core; exit/unload cleanup — in **Firefox+Chrome+Safari**, under **both `-fexceptions` and `-fwasm-exceptions`** (incl. a modal from inside a `catch`, to prove composition with the hoist pass), with a `-sASYNCIFY_ASSERTIONS=1` pass.
## 6. Phase checklist (work log — update as landed)
- [ ] **Phase 0 — red harness** (2–3 d). Minimal `sched-nest` repro RED in 3 engines; the 6 `coroutine*` specs confirmed RED under de-park; CI/local script to run them.
- [ ] **Phase 1 — scheduler core** (1–2 wk). `asyncify-scheduler.js` with the 4 hooks + deferred drain; covers `handleAsync` + fibers. **Gate:** `sched-nest` + the 6 `coroutine*` specs GREEN, all 3 engines, both EH. (If the permanent `handleAsync` park can't be a clean parked context, escalate to Phase 2.)
- [ ] **Phase 2 — root fiber** (≈1 wk). Main loop = scheduler root fiber; `ProcessEvents` wasm-side (no JS async ccall). **Gate:** Phase-1 gate still green + no `handleAsync` park remains.
- [ ] **Phase 3 — migrate waits** (1–2 wk). `wasm_yield_until` API; `ShowModal`/nested loop/clipboard/font on it. **Gate:** full matrix green.
- [ ] **Phase 4 — lifetime** (few d). Cleanup ordering vs the scheduler; teardown deferred to unload. **Gate:** exit/unload tests green; no cleanup during steady-state pumping.
## 6b. Phase-0 finding — Phase 1 is insufficient; Phase 2 (root fiber) is REQUIRED (2026-06-23)
**Exact failure** (coroutine_test, de-park build): the first case `yield_resume_preserves_state` **passes** (it runs during the startup burst, *before* the main-loop park), then a later fiber swap aborts with **`Aborted(Assertion failed: We cannot stop an async operation in flight)`**, surfacing as `[wxWasm] main loop pump error`.
**Why:** `wxWasmParkMainLoop` is `Asyncify.handleAsync(...)` — a **permanently in-flight async operation** for the app's whole life. A coroutine `emscripten_fiber_swap` inside the rAF pump calls `stop_unwind`, but the park's async op is in flight → abort. Under the old `throw`, the top loop was *not* an async op (`throw "unwind"` is a plain JS exception), so swaps from a clean base worked.
**Tested & ruled out:** changing the rAF pump's `await ccall('ProcessEvents',{async:true})` to a **synchronous** `ccall` does NOT help — the in-flight op is the *park*, not the per-tick ccall. And the park is **permanent** (never completes until exit), so no scheduler serialization can let a coroutine swap "wait for the slot." **So §6's Phase-1 escalation condition is met.**
**The fix (Phase 2, now confirmed required):** the main loop must not be a `handleAsync` park. Make the main stack a **libcontext fiber** (Ruby/Julia pattern): the main fiber runs `ProcessEvents` on its own stack and **yields to the browser by a fiber swap / return-through-`set_main_loop(...,0)`**, not a `handleAsync` suspend — so there is no permanent in-flight async operation, and a coroutine swap is a sibling fiber↔fiber switch from the same `g_main_context`. `ProcessEvents` must run on `g_main_context` (the main fiber), not the fresh rAF-ccall stack. The Phase-1 scheduler is still needed to coordinate modal/clipboard waits that *do* suspend — but **the main-loop park must move off `handleAsync` first.**
**Open Phase-2 design point:** how the main fiber yields to / resumes from the browser each frame (rAF resumes `g_main_context` to run one `ProcessEvents` tick, then the main fiber yields back) without re-introducing a permanent asyncify operation. Candidate: `set_main_loop(tick,0,0)` where `tick` resumes the main fiber via libcontext, the main fiber runs `ProcessEvents` then swaps back, and wx teardown is suppressed until unload (Phase 4 lifetime).
## 6c. IMPLEMENTED & verified (2026-06-23): the per-frame-yield while-loop
The fix is **simpler than "an explicit libcontext root fiber."** `DoRun` (top level) is now a plain C++ loop on the real main stack (`evtloop.cpp`):
```cpp
while (!m_shouldExit) { ProcessEvents(); wxWasmYieldToBrowser(); }
```
`wxWasmYieldToBrowser` is `EM_ASYNC_JS(void, …, { await new Promise(r => requestAnimationFrame(r)); })` — an Asyncify suspend that **completes every frame**. Because nothing is permanently suspended, the Asyncify slot is free (`state==Normal`) whenever `ProcessEvents` runs, so a tool-coroutine fiber swap inside it succeeds; and `ProcessEvents` runs on the real main C stack (= libcontext's `g_main_context`), so swaps are from the right context. `ScheduleExit` just sets `m_shouldExit` for the top level (nested/quasi-modal loops still use the `setTimeout` pump + `wxWasmExitNestedLoop`). `wxWasmParkMainLoop` is removed. No explicit fiber API or scheduler was needed for the *main-loop* fix — the key was only that the suspension **completes** each frame instead of being permanent.
**Result (JS-EH):** coroutine in-app suite **13/13 pass, 0 fail** (was: abort after case 1); `coroutine` + `coroutine-nested` e2e specs **green**; dialog renders + modals **green** (no regression). Only `coroutine-pthread` outstanding — but its `coroutine_test_wxpt.wasm` was **stale** (the `coroutine-pthread` make target didn't rebuild it); all apps are being rebuilt to confirm.
**Still likely needed later (Phase 1 scheduler / Phase 3):** modal/clipboard waits that genuinely suspend across the loop still use the nested `setTimeout` pump; if overlapping suspensions there prove fragile, layer the scheduler on. But the *coroutine regression itself is fixed by this main-loop change alone.*
## 6d. Phase-2 exposes a SECOND coupling: context-menu re-entrancy needs Phase 1 (2026-06-23)
The while-loop main loop (§6c) fixed the coroutines but **regressed the context menu** (2 e2e specs). Right-click → choose *Cut* → `Aborted(RuntimeError: unreachable)` / `memory access out of bounds`. Stack: a DOM mouse event (`mouseEventHandlerFunc` → the Asyncify export wrapper → `wasm-function[…]`) **re-enters wasm while `DoPopupMenu`'s `wxDomPopupMenuModal` context is suspended on the deep main stack** — a single-slot re-entrancy fault. The de-park's *permanent-context* loop masked it (its menu context was shallow — a fresh-ccall `ProcessEvents` — and always "in flight"); the while-loop's no-permanent-context, deep-stack suspend exposes it.
**Three targeted fixes, all empirically REJECTED (don't retry these):**
1. **C++ pump in `wxDomPopupMenuModal`** (mirror `startModal`) — *redundant*: `wx-dom.js`'s `wxShowContextMenu` **already** runs the same `setTimeout` ProcessEvents pump. No effect. (Reverted.)
2. **`ASYNCIFY_STACK_SIZE` 8192→65536** — not a buffer-size fault (still crashes at 65536; emscripten appends that hint to *every* `unreachable`). (Kept anyway — the while-loop genuinely deepens every suspension, so 65536 ≈ the coroutine apps + KiCad is the right call for all wx apps.)
3. **DOM backdrop blocking canvas pointer events** (`wx-dom.js`) — confirmed present in the rebuilt glue; still crashes. So the re-entry is **not** a canvas leak — the wx DOM port's document-level mouse handler re-enters wasm regardless. (Reverted.)
**Conclusion — the hard tension, stated plainly:**
- **de-park** (permanent-context loop): menus ✅, coroutines ❌
- **while-loop** (no permanent context): coroutines ✅, menus ❌
Neither is clean alone. Both faults are the SAME single-slot `currData`/state arbiter problem — Design B's **scheduler (Phase 1)** — now *proven necessary, not optional*. The §6c while-loop is the correct **foundation** (it removes the permanent park that blocked coroutine swaps); Phase 1 must layer on top so a wasm re-entry during ANY suspension (coroutine swap, menu/modal `handleAsync`, main-loop yield) is coordinated (deferred/queued or serialized) rather than misfiring a rewind. Kept in-tree: `evtloop.cpp` while-loop + the 65536 bump. Reverted: the redundant C++ pump and the backdrop.
## 6e. The precise mechanism (export-wrapper diagnostic, 2026-06-23)
Instrumented the Asyncify export wrapper to log every wasm entry while `state != Normal`. The menu crash is **not** a one-shot bad rewind — it's an **infinite busy unwind/rewind loop** on one buffer:
```
asyncify_start_unwind state=1(Unwinding) currData=1240280 ← main suspends
asyncify_start_rewind state=2(Rewinding) currData=1240280 ← …immediately resumed
__main_argc_argv state=2 currData=1240280 ← main runs a few dynCall_ii deep
…repeats forever (currData unchanged) until the OOB crash
```
Buffer `1240280` (the parked main stack, suspended at the menu) is **suspended then immediately re-resumed, over and over**. Only one context exists, but it is being re-driven in a tight spin: its continuation re-suspends instantly (the menu promise is still pending), and something re-rewinds it each cycle.
**Two drivers fight over the single slot:** with the while-loop, the main-loop structure AND the **menu's own `setTimeout` ProcessEvents pump** (`wx-dom.js`) both try to drive the parked main stack — one re-rewinds what the other parked. Under the de-park there was a *single* pump chain (the rAF pump *was* the loop; the menu pump nested inside its `await`), so nothing double-drove the slot.
**Scheduler invariant this pins (the central requirement):** exactly ONE unwind/rewind transition in flight; a pump tick runs a **fresh** `ProcessEvents` (new stack) and must NEVER re-rewind an already-parked context — only that context's own `wakeUp` (its promise resolving) may resume it. The scheduler must enforce this across the main-loop yield, the menu/modal/nested pumps, and fiber swaps. (A plausible smaller first cut: a single shared "is a transition in flight / is a context parked" guard the pumps consult before re-driving — test it against the contextmenu specs before committing to the full registry.)
## 6f. RESOLVED (2026-06-23): the arbiter already existed — it just wasn't injected
A gated export-wrapper + `start_rewind` probe nailed the proximate cause: the menu's wakeUp fires `_asyncify_start_rewind(Asyncify.currData)` with **`currData == null`** → reads address 0 → OOB. The cause: the **`handlesleep.js` currData save/restore shim** — the existing Design-A / Emscripten #9153 arbiter (`scripts/common/shims/handlesleep.js`, which restores `currData` to the parked context's buffer before every rewind) — was **NOT injected into the contextmenu glue** (`pendingSleepContexts` count 0, vs 9 in the working coroutine app). Appending it manually → crash gone, `[CTXMENU_EVENT] Cut chosen` fires, spec 4/4 green.
**Why it was missing:** `inject-dyncall-shims.sh` gates the handleSleep shim on the libcontext fiber marker (`_emscripten_fiber_swap.isAsync = true;`), and `build-wasm-test.sh` only ran the injector under `WX_NATIVE_EH=1`. So plain (non-fiber) wx apps under JS-EH never received the currData arbiter. They didn't crash *before* the while-loop because the de-park's shallow menu context (a fresh-ccall `ProcessEvents`) never hit the null-rewind path; the while-loop's deeper main-stack suspend exposes it.
**Fix — build-system only, NO new runtime code:**
1. `build-wasm-test.sh` injects the shim into every freshly-linked app for **both** EH models (idempotent — the Makefile-injected coroutine apps are skipped).
2. `inject-dyncall-shims.sh` appends the handleSleep shim at EOF when there's no fiber glue (Asyncify is defined by then; it wraps `handleSleep` at load, before any runtime sleep).
**So §6c–6e's "build the single-owner currData arbiter" conclusion was right about the diagnosis but the arbiter already exists (`handlesleep.js`) — it only needed to reach these apps.** The scheduler invariant in §6e *is* what `handlesleep.js` implements (each parked context owns its buffer; `currData` is restored before its own rewind). The while-loop (coroutine fix) + this injection fix together resolve both regressions. §7's open question is therefore moot: no Phase-1 scheduler nor Phase-2 root fiber was needed — the while-loop main loop + the pre-existing currData shim suffice.
## 7. Open decisions (resolve during implementation)
- Is Phase 1 (scheduler treating the `handleAsync` park as a tracked parked context) sufficient, or is Phase 2 (root fiber) required? — answered by the Phase-0 harness against the Phase-1 build.
- One scheduler file injected post-link (like today's shim) vs an emscripten `--js-library` (link-time, cleaner, survives JS regen). Lean js-library for durability.
- Whether to keep an Asyncify-only "no scheduler" fast path for apps with no coroutines (most standalone tests) to avoid scheduler overhead — likely yes, gated on a runtime "any non-main context registered?" check.
- Native-EH interaction: confirm the scheduler's transitions compose with `HoistCppCatches` (suspend-inside-catch) — a matrix test, not a design change expected.

278
docs/features/perf/README.md Normal file
View file

@ -0,0 +1,278 @@
# Cross-browser performance: why Firefox > Chrome > Safari, and how to close the gap
> Research notes, **2026-06-18**. The KiCad WASM port runs fastest in Firefox,
> slower in Chrome, slowest in Safari. This document explains *why* at the
> browser-engine level and lays out a ranked, build-specific plan to speed up
> Chrome and Safari. Web claims are dated and linked in [Sources](#sources);
> codebase claims carry `file:line` refs. Companion work lives in
> [`../async/`](../async/) (Asyncify) and [`../wasm-exceptions/`](../wasm-exceptions/)
> (the `-fwasm-exceptions` migration).
---
## TL;DR
The Firefox lead is **not** a Firefox trick. Our binary is dominated by
**Asyncify** instrumentation, and Firefox's compilers simply tolerate Asyncify's
pathological code far better than Chrome's or Safari's do. So the highest-leverage
work for Chrome *and* Safari is to **shrink/attack the Asyncify footprint**, plus a
handful of cheap, orthogonal wins.
There are **two independent axes**, and both need attention:
1. **WASM compile/execute** — Asyncify-dominated. This explains the
Firefox > Chrome > Safari **ordering**.
2. **WebGL rendering** — Safari's Metal/ANGLE overhead. This is *extra* Safari
slowness on top of axis 1, and several fixes are one-liners.
### Ranked levers
| # | Lever | Axis | Effort | Impact | Where |
|---|---|---|---|---|---|
| 1 | Confirm/force `instantiateStreaming` + `Content-Type: application/wasm` + stable URL/ETag | startup | hours | ~1.5–1.8× cold start (FF); arms V8 cache | `web/standalone/src/wasm/boot.ts` |
| 2 | **Brotli** instead of gzip-9 on R2 | startup | hours | ~15–25% smaller transfer | R2 / edge config |
| 3 | `powerPreference: 'high-performance'` + context-lost handlers | WebGL (Safari/Chrome) | hours | discrete GPU instead of integrated | `wxwidgets/src/wasm/glcanvas.cpp:524-535` |
| 4 | Audit GAL shaders for the `flat` qualifier | WebGL (Safari) | hours–days | up to *seconds/frame* in worst case | `kicad/common/gal/shaders/` |
| 5 | Remove `glGetError()` from the render loop | WebGL (Safari) | hours | avoids per-call Metal flush | GAL compositor |
| 6 | Test `antialias: false` | WebGL (Safari) | hours | cuts MSAA resolve cost | `glcanvas.cpp:524-535` |
| 7 | Enable `-msimd128` | WASM exec (all) | days | 1.5–2.5× geometry/render hot loops | build flags |
| 8 | `ASYNCIFY_ADVISE` → `ASYNCIFY_IGNORE_INDIRECT` + extend `REMOVE` | WASM exec (all, esp. Chrome/Safari) | days | smaller binary + faster tier-up | `scripts/common/apply-asyncify.sh` |
| 9 | `-fwasm-exceptions` (size) | WASM (all) | weeks | 64.5 → 36 MB gz | tracked — see [§ Structural bets](#structural-bets-track--prototype) |
| 10 | JSPI (delete Asyncify) | WASM (all, esp. Safari) | weeks | ~40–50% smaller, removes JIT pressure | tracked — see [§ Structural bets](#structural-bets-track--prototype) |
| — | wasm-split, WebGPU GAL backend | startup / WebGL | weeks+ | deferred (see [§ Deferred](#deferred--not-now)) | — |
---
## Current build (the baseline)
Verified from the build scripts and runtime glue:
| Knob | Value | Location |
|---|---|---|
| Asyncify | `-sASYNCIFY=1`, `ASYNCIFY_STACK_SIZE=65536` | `scripts/kicad/build-kicad-target.sh:~400` |
| Exceptions | **legacy `-fexceptions`** (not `-fwasm-exceptions`) | `build-kicad-target.sh:240-255` |
| SIMD | **none** (`-msimd128` absent) | — |
| Threads | `-sUSE_PTHREADS=1`, pool = `navigator.hardwareConcurrency` (+ COOP/COEP) | `build-kicad-target.sh`, `web/.../preflight/capabilities.ts` |
| Memory | `INITIAL_MEMORY=256MB`, `MAXIMUM_MEMORY=4GB`, `ALLOW_MEMORY_GROWTH=1` | `build-kicad-target.sh` |
| Opt | clang `-O2` (release); link `-O0` then **host `wasm-opt -O2` after `--asyncify`** | `apply-asyncify.sh:88-157` |
| WebGL | WebGL2 (`-sMAX_WEBGL_VERSION=2`), `antialias:true`, **`powerPreference:DEFAULT`** | `glcanvas.cpp:524-535` |
| Loading | Emscripten script-glue; **streaming not confirmed**; gzip-9, **no Brotli** | `boot.ts:145-294` |
| Artifact | pcbnew **186 MB raw / 64 MB gzip**; eeschema 99/34; pl_editor 52/17; gerbview 49/16 | `output/` |
Note: all three modern browsers support `SharedArrayBuffer`/threads under COOP+COEP
(the app demonstrably runs in each) — capability gating is in
`capabilities.ts`, not UA sniffing.
---
## Why the ordering exists (engine internals)
### The villain: Asyncify
Asyncify rewrites every instrumented function with unwind/rewind state checks and
saves/restores all locals to linear memory. That expands each local's live range
across the *whole* function, producing a nearly fully-connected interference graph
— exactly the input that is catastrophic for optimizing register allocators.
Asyncify's own docs warn: *"VMs may also limit compilation to the baseline tier on
such pathological code."* Result: ~+70% binary, giant functions, and the
186 MB-raw pcbnew. See [`../async/02-asyncify-internals.md`](../async/02-asyncify-internals.md).
### How each engine copes
| Engine | Baseline tier | Optimizing tier | On Asyncify's giant functions |
|---|---|---|---|
| **Firefox / SpiderMonkey** | Rabaldr, **~25 ns/byte**, eager whole-module, multithreaded (30–60 MB/s) | **Ion** — [75× large-function fix, Oct 2024](https://spidermonkey.dev/blog/2024/10/16/75x-faster-optimizing-the-ion-compiler-backend.html) (sorted live ranges, Semi-NCA dominators, sparse bitsets) targeting *exactly* the huge-CFG/high-vreg shape Asyncify creates (ONNX: 5 min → 3.9 s) | **Best.** Whole module baseline-compiled before download finishes; Ion swallows the big functions. No OSR gap. |
| **Chrome / V8** | Liftoff, **~50 ns/byte** (½ Firefox) | **TurboFan** — chokes on huge fns (a 1.96 MB fn → 95 s, 7.4 GB RAM, 87% in regalloc); falls back to mid-tier allocator or **skips optimization** | **Middle.** **V8 has no OSR for wasm** — a function in a long loop (Asyncify rewind/unwind loops!) finishes that whole call in Liftoff; only the *next* call gets TurboFan. |
| **Safari / JSC** | **Lazy everything**: IPInt (interpreter) → BBQ → OMG. Nothing eager. | **OMG** (B3) — did *not* get Ion's 2024 large-fn treatment | **Worst.** First run executes at interpreter speed; Asyncify ~doubles fn count → huge OMG backlog → documented **300–400% CPU spike for 30 s+** after a workload. **No persistent compiled-code cache**, so it re-pays every session; above ~10 MB it switches to a slower JIT mode. |
### Two corollaries that bite us specifically
- **Chrome's V8 wasm code cache is effectively unavailable.** It only caches
modules under ~150 MB *compiled*, and compiled code is 5–7× the `.wasm`. Our
186 MB pcbnew → ~1 GB compiled — far over the ceiling. So Chrome **re-runs
TurboFan on every cold load** today. Shrinking the binary (levers 7–10) is the
only way to get Chrome's repeat-load cache back. See
[V8 wasm code caching](https://v8.dev/blog/wasm-code-caching).
- **Benchmark trap:** with DevTools open, V8 tiers all wasm *down* to Liftoff.
Never measure Chrome speed with DevTools open (except via an actual Performance
recording, which forces tier-up). This likely makes Chrome look worse than it is
in casual testing.
---
## The ranked plan
### Tier 1 — cheap, do now (days, low risk)
**1. Confirm + force streaming instantiation and cache headers.** The loader
injects the Emscripten JS glue via `<script>` (`boot.ts`); whether the *runtime*
then streams the `.wasm` depends on serving conditions. In DevTools → Network,
confirm the `.wasm` returns **`Content-Type: application/wasm`** with no console
"falling back to ArrayBuffer instantiation" warning. Streaming is ~1.5–1.8× faster
cold-start on Firefox and is the *only* path that arms V8's code cache. Serve the
`.wasm` from a **stable URL** (no content-hash in the path; use a stable alias)
with `ETag`/`304`. With `-pthread`, confirm the module is compiled once and shared
to workers (Emscripten does this via the shared `WebAssembly.Module`), not
recompiled per worker.
**2. Brotli instead of gzip-9 on R2.** Brotli is ~15–25% smaller on wasm (our
64 MB pcbnew → ~50 MB). Verify Cloudflare actually Brotli-compresses it at the
edge — it often *skips* large binary types — and if not, precompress and serve
with `Content-Encoding: br` + `Content-Type: application/wasm`. Smaller transfer
also means less to compile, so it compounds with everything below.
**3. `powerPreference: 'high-performance'` for WebGL.** We default to
`EM_WEBGL_POWER_PREFERENCE_DEFAULT` (`glcanvas.cpp:524-535`). **Safari (and Chrome
on dual-GPU Macs) defaults WebGL to the integrated GPU.** Requesting
high-performance switches to the discrete GPU — often the single biggest
GPU-bound framerate win on MacBook Pros. Caveat: Safari only honors it if you also
register `webglcontextlost`/`webglcontextrestored` handlers.
**4. Audit GAL shaders for the `flat` interpolation qualifier.** This is the big
Safari sleeper. `flat` triggers a provoking-vertex workaround in Safari's
Metal/ANGLE backend that has cost real apps *seconds per frame*. PCB renderers
commonly use `flat` for per-primitive net/layer colors. Grep the GAL shaders
(`kicad/common/gal/shaders/`, source GLSL 1.20 before `convert_glsl_es3.py`); if
present, replace with regular interpolation or restructure. Potentially a massive
Safari-only win.
**5. Remove `glGetError()` from the render loop.** On Safari each call forces a
Metal pipeline flush. Restrict to init/debug builds only. (Note: the WebGL
compositor already drains stale `glGetError()` once before draws — that's fine;
the concern is *per-call* error checks inside the hot path.)
**6. Test `antialias: false`.** We default MSAA on (`antialias:true`). KiCad's GAL
does much of its line AA in-shader (SMAA) and has its own AA setting; if MSAA is
redundant, dropping it cuts Metal's resolve cost on Safari. Quality/perf tradeoff —
A/B it on a dense board; consider exposing it as a setting.
### Tier 2 — medium effort, high impact
**7. Enable `-msimd128`.** Expect **1.5–2.5×** on the geometry/render hot loops
(polygon booleans in `shape_poly_set`, DRC overlap checks, vertex-buffer fills) via
LLVM autovectorization at `-O2`+. Safe on all three engines (Chrome 91 / FF 89 /
Safari 16.4). Helps absolute Chrome *and* Safari speed. Verify `v128.*` actually
appears in the disassembly for the hot functions, and prefer `pmin`/`pmax` over
min/max (the SSE→wasm emulation table has slow paths). **Do not** ship Relaxed SIMD
yet (Safari still flags it). Minor interaction to watch: SIMD slightly grows
per-function size, which feeds the Asyncify/locals pressure — measure after the
`wasm-opt -O2` pass.
**8. Shrink the Asyncify surface.** This directly attacks the root cause for Chrome
and Safari. Run **`ASYNCIFY_ADVISE`** to see which functions get instrumented and
why — it surfaces the biggest instrumented functions (the JIT pressure points).
Then:
- **`ASYNCIFY_IGNORE_INDIRECT=1`** is the high-impact one: our wxWidgets/GAL
code is vtable-heavy, and Asyncify conservatively instruments *every* indirect
call site, which is why instrumentation spreads everywhere. With our
understanding of the suspend paths (the park-throw work), we may be able to
assert no indirect call is on the suspend stack and add specific ones back via
`ASYNCIFY_ADD`.
- Extend the existing 12-function `ASYNCIFY_REMOVE` list (`apply-asyncify.sh:92-104`)
with cold/startup-only large functions ADVISE flags.
- Smaller instrumented set → smaller functions → better tier-up in *all* engines
and a smaller Safari OMG backlog.
- ⚠️ Error-prone (wrong config = silent runtime breakage). Gate behind the
red/green Asyncify harness in [`../asyncify-arbiter/redgreen.md`](../asyncify-arbiter/redgreen.md)
/ `tests/asyncify/`.
### Structural bets (track / prototype)
**9. `-fwasm-exceptions`** (we're on legacy `-fexceptions`). Biggest *size* lever —
[`../wasm-exceptions/02-measurements.md`](../wasm-exceptions/02-measurements.md)
puts pcbnew at **64.5 → 36 MB gzip**, which would also start bringing Chrome back
under the code-cache ceiling and fix the unreliable catch/destructor landing-pad
behavior we've documented. **Two blockers to track before committing:**
- a **Safari 26.0 startup regression** for `-fwasm-exceptions` modules
([emscripten #25365](https://github.com/emscripten-core/emscripten/issues/25365))
— verify whether it's fixed in a 26.x / Safari 27 beta before shipping, since
Safari is the browser we're trying to help;
- the asyncify-EH unwind-from-catch interaction
([`../wasm-exceptions/05-asyncify-fork-design.md`](../wasm-exceptions/05-asyncify-fork-design.md)).
**10. JSPI** — the eventual *real* fix for Safari, because it deletes Asyncify
entirely (no instrumentation → no giant functions → no OMG backlog → the
300–400% Safari spike goes away) and cuts ~40–50% of binary size. Status:
| Engine | JSPI status |
|---|---|
| Chrome / V8 | **shipped, Chrome 137** (May 2025) |
| Firefox / SpiderMonkey | **Firefox 153** intent-to-ship (June 2026); Nightly now, stable ~late summer/fall 2026 |
| Safari / JSC | **Safari 27 beta** (WWDC26), enabled by default; stable Fall 2026 |
Don't migrate wholesale yet:
- unresolved **~350× regression on the `JS→C→JS` re-entry pattern**
([emscripten #21081](https://github.com/emscripten-core/emscripten/issues/21081))
— exactly what a GUI event loop hits constantly;
- static-init `SuspendError`
([emscripten #24302](https://github.com/emscripten-core/emscripten/issues/24302));
- `invoke_*` over-tagging under our legacy exceptions.
**Recommended path:** prototype JSPI on a *small* tool (calculator or pl_editor)
behind feature detection (`'Suspending' in WebAssembly`) with Asyncify fallback,
profile the re-entry pattern, and watch #21081. By the time it's safe, all three
engines will support it.
### Deferred / not now
- **`wasm-split` / `-sSPLIT_MODULE`** — the secondary module can't be loaded lazily
*and* asynchronously on the main thread, which is incompatible with our
main-thread Asyncify model unless we move to `-sPROXY_TO_PTHREAD`.
- **WebGPU GAL backend** — the structural exit from Safari's Metal/ANGLE overhead
(Safari 26 ships WebGPU), but it's a multi-week GLSL→WGSL port with no upstream
KiCad support. See the GAL history in [`../archive/webgl/`](../archive/webgl/).
- **Global `-O3`** — its inlining bloats the binary and makes the Chrome
cache/compile problem *worse*. If anything, compile cold/utility units at `-Os`.
---
## What to do first
Quick, visible, near-zero-risk wins this week:
- **Safari:** #3 (high-performance GPU) + #4 (`flat` audit) + #5 (`glGetError`).
- **Startup everywhere:** #2 (Brotli) + #1 (streaming/headers).
- **Runtime everywhere:** #7 (`-msimd128`).
Then invest in **#8 (Asyncify ADVISE + IGNORE_INDIRECT)** as the real lever against
the Chrome/Safari gap, and keep **#9 / #10** on a tracking list.
---
## Sources
**Engine internals**
- [75× faster: optimizing the Ion compiler backend — SpiderMonkey, Oct 2024](https://spidermonkey.dev/blog/2024/10/16/75x-faster-optimizing-the-ion-compiler-backend.html)
- [Understanding WebAssembly code generation throughput — wingolog, 2020](https://wingolog.org/archives/2020/04/14/understanding-webassembly-code-generation-throughput)
- [V8 WebAssembly compilation pipeline](https://v8.dev/docs/wasm-compilation-pipeline) · [Dynamic tiering](https://v8.dev/blog/wasm-dynamic-tiering) · [Liftoff](https://v8.dev/blog/liftoff)
- [Code caching for WebAssembly developers — V8](https://v8.dev/blog/wasm-code-caching)
- [Introducing the JetStream 3 Benchmark Suite — WebKit, 2024](https://webkit.org/blog/17899/introducing-the-jetstream-3-benchmark-suite/) (IPInt/BBQ/OMG)
- [Pause and Resume WebAssembly with Binaryen's Asyncify — kripken, 2019](https://kripken.github.io/blog/wasm/2019/07/16/asyncify.html)
**Startup / size / SIMD**
- [Optimizing WebAssembly Startup Time — Nutrient](https://www.nutrient.io/blog/optimize-webassembly-startup-performance/)
- [MDN: WebAssembly.instantiateStreaming](https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/JavaScript_interface/instantiateStreaming_static)
- [Module Splitting — Emscripten](https://emscripten.org/docs/optimizing/Module-Splitting.html)
- [Using SIMD with WebAssembly — Emscripten](https://emscripten.org/docs/porting/simd.html) · [V8 SIMD](https://v8.dev/features/simd) · [caniuse wasm-simd](https://caniuse.com/wasm-simd)
- [Asynchronous Code (Asyncify settings) — Emscripten](https://emscripten.org/docs/porting/asyncify.html)
**Safari WASM + WebGL**
- [WebGL Performance on Safari & Apple Vision Pro — Wonderland Engine](https://wonderlandengine.com/news/webgl-performance-safari-apple-vision-pro/) (`flat`, UBO timing)
- [WebKit features in Safari 26.0](https://webkit.org/blog/17333/webkit-features-in-safari-26-0/) · [News from WWDC26 — Safari 27 beta](https://webkit.org/blog/17967/news-from-wwdc26-webkit-in-safari-27-beta/)
- [emscripten #25365 — Safari 26.0 wasm-exceptions regression](https://github.com/emscripten-core/emscripten/issues/25365)
- [emscripten #26027 — Safari + Asyncify + unaligned-load leak](https://github.com/emscripten-core/emscripten/issues/26027)
- [ONNX Runtime #26827 — Safari WebKit 26 OMG CPU loop](https://github.com/microsoft/onnxruntime/issues/26827)
**JSPI**
- [V8: Introducing the WebAssembly JavaScript Promise Integration API](https://v8.dev/blog/jspi) · [new API](https://v8.dev/blog/jspi-newapi)
- [caniuse: JSPI](https://caniuse.com/wf-wasm-jspi) · [Chrome 137 release notes](https://developer.chrome.com/release-notes/137)
- [Mozilla dev-platform: Intent to Ship JSPI (Fx153), June 2026](http://www.mail-archive.com/dev-platform@mozilla.org/msg01810.html)
- [emscripten #21081 — JSPI 350× slower for JS→C→JS](https://github.com/emscripten-core/emscripten/issues/21081) · [#24302 — JSPI static-init SuspendError](https://github.com/emscripten-core/emscripten/issues/24302)
### Local cross-references
- [`../async/`](../async/) — Asyncify internals, single-slot contention, park-throw.
- [`../wasm-exceptions/`](../wasm-exceptions/) — `-fwasm-exceptions` measurements, toolchain status, asyncify-EH fork design.
- [`../asyncify-arbiter/redgreen.md`](../asyncify-arbiter/redgreen.md) — the harness to gate Asyncify-surface changes.
- [`../archive/webgl/`](../archive/webgl/) — GAL → WebGL2 history (context attrs, compositor, shaders).
- Build: `scripts/kicad/build-kicad-target.sh`, `scripts/common/apply-asyncify.sh`.
- WebGL context: `wxwidgets/src/wasm/glcanvas.cpp:489-535`. Loader: `web/standalone/src/wasm/boot.ts`.

434
docs/features/threading/README.md Normal file
View file

@ -0,0 +1,434 @@
# Threading in KiCad-WASM — why it's single-core today, and the paths to real multithreading
> **Status:** mechanism reference for KiCad-WASM threading. Native wasm-EH (`-fwasm-exceptions`) is
> the **default build**, and the three-failure-mode analysis below is validated by the pthread test
> suite — see [`../wasm-exceptions/10-pthreads-native-eh.md`](../wasm-exceptions/10-pthreads-native-eh.md).
> Authored 2026-06-24, updated 2026-06-25. Line numbers are against the artifacts current then
> (`kicad/thirdparty/thread-pool/bs_thread_pool.hpp`, `kicad/common/thread_pool.cpp`,
> `kicad/3d-viewer/3d_rendering/raytracing/render_3d_raytrace_base.cpp`,
> `scripts/kicad/build-kicad-target.sh`, `scripts/common/shims/handlesleep.js`,
> `scripts/common/apply-asyncify.sh`).
## Why this exists
A recurring question: we made "pthread hacks" in the 3D viewer's CPU renderer — *what* was the
issue, *why* were they needed, will **native WASM exceptions** (`-fwasm-exceptions`) fix them, and
how do we get back to **upstream-pristine KiCad source that still runs multithreaded** (so the fork
stays upstreamable)? This document answers all of that, plus: the exact deadlock mechanics, the
three-layer thread model, the complete inventory of raw threads in the tree, and what the *latest*
upstream KiCad has (and hasn't) already changed.
It is the threading companion to the Asyncify dossier in [`../async/`](../async) (especially
[`../async/11-asyncify-nesting-raytracer.md`](../async/11-asyncify-nesting-raytracer.md)) and the
[`../wasm-exceptions/`](../wasm-exceptions) migration.
## TL;DR
- **Three layers, often conflated:** (1) **Web Workers** = the real OS threads; (2) **Emscripten's
pthread pool** (`PTHREAD_POOL_SIZE`) = pre-spawned *empty* Workers; (3) **KiCad's
`BS::thread_pool`** (`GetKiCadThreadPool()`) = `hardware_concurrency()` long-lived `std::thread`s
that **consume** the pre-warmed Workers at startup. **All** pthreads are full shared-memory
Workers — there is no lightweight/isolated variant.
- **Effectively nothing runs multithreaded.** KiCad's pool funnels every entry point through one
shimmed `detach_task()` (inline), so the whole pool is serial; the raytracer's *separate* raw
`std::thread` passes are `#ifdef`'d to serial; `wxThread` is a no-op. The 16 pool Workers spawn at
startup and then sit idle.
- **Three distinct failure modes, not one:** **(a) deadlock** (raw-thread join + on-demand Worker
creation — fixed in the wasm layer, §4), **(b) nesting abort** `invalid state: 1` (a 2nd Asyncify
unwind starting while already Unwinding — does **not** arise when the inner `emscripten_sleep` is
dispatched at `state == Normal`, e.g. from a modal pump's `ProcessEvents`, §3), **(c) worker-rewind
crash** `"func is not a function"` (a C++ throw driving Asyncify on a pool Worker under
`-fexceptions` — **fixed by native EH**, now the default).
- **`-fwasm-exceptions` (the default) clears mode (c).** A C++ exception thrown on a pool Worker is
safe under native EH — confirmed: the real-pool `threadpool-real` test runs 16-core with a throwing
worker task, green only under native EH. Modes (a)/(b) are Asyncify, not EH — addressed separately
in the wasm layer (the nanosleep override for (a); `state == Normal` dispatch for (b)).
- **`PROXY_TO_PTHREAD` is not our escape hatch.** Asyncify *can* mechanically run on the
proxied-main Worker, but it's unsupported/rough — and the real blocker is that our **wx-dom port
manipulates the DOM directly**, which a Worker cannot do.
- **A path to multi-core 3D needs zero KiCad edits — two ways.** (1) **Pre-warm** enough Workers
(`PTHREAD_POOL_SIZE` ≥ pool + peak raw threads): on-demand creation never happens, so the upstream
`sleep_for` busy-wait runs multi-core (with main-thread jank). (2) **The nanosleep override**
(`wasm/shims/nanosleep_yield.c`) makes that main-thread `sleep_for` *yield* via Asyncify, so the
event loop services the on-demand handshake — multi-core without pre-warming and without the jank
(proven by `pthread-ondemand`, §4). The parked `WASM_RAYTRACE_POOL` (~6–7×) is the pre-warm shape.
- **Upstream has only migrated 1 of 7 raytracer parallel sections to the pool**, and that was an
accident (a side effect of a cosmetic commit). The other six are **legacy 2018 OpenMP-translation
code** — so a pool migration is a legitimate, *upstreamable* cleanup, not a wasm hack.
---
## 1. The model: three layers, two patterns
WebAssembly has no threads of its own. "Thread" means different things at three levels:
**Layer 1 — Web Workers = the real OS threads.** A "thread" in a browser is a Web Worker: a
separate JS context running the *same* wasm module against the *same* shared memory. Spinning one up
is **expensive** (new context + module instantiate) and **can only be initiated from the main
thread's event loop**.
**Layer 2 — Emscripten's pthread pool (`PTHREAD_POOL_SIZE`).** Because creating Workers is slow and
main-thread-bound, Emscripten pre-spawns a bag of *empty, generic* Workers at startup. We set
`PTHREAD_POOL_SIZE='navigator.hardwareConcurrency'` (`build-kicad-target.sh:413-415`), so on a
16-core machine you get 16 pre-warmed Workers. `std::thread`/`pthread_create` tries to grab one.
**Layer 3 — KiCad's `BS::thread_pool` (`GetKiCadThreadPool()`).** An *application-level* pool — a
different thing from Layer 2. Its constructor creates `hardware_concurrency()` long-lived
`std::thread`s (`thread_pool.cpp:44-45` → `determine_thread_count` at `bs_thread_pool.hpp:1965-1970`)
and parks them on a condition variable waiting for tasks. You feed it work with `submit_task`; the
parked threads pick it up. The "hire a team once, give them many jobs" pattern.
**The interaction that confuses everyone:** KiCad's Layer-3 pool threads *are* pthreads *are*
Layer-1 Workers. So the 16-thread pool **consumes all 16 pre-warmed Workers at startup.** After that
the pre-warmed bag is *empty*.
**All pthreads are full shared-memory Workers.** There is no `std::thread` that gets its own
isolated heap. Every pthread shares the one `WebAssembly.Memory` (one SharedArrayBuffer); a thread's
"own" memory is only its stack + TLS, carved *out of* that shared buffer. KiCad's raytracer threads
*need* this — they read the shared scene and write the shared output image. (An *isolated* Web
Worker with message-passing — copy data in, post results out — would sidestep the whole pthread +
Asyncify problem for pure-compute work like a raytrace band, but that is **not** what `std::thread`
does; using it means hand-writing a worker pool and **rewriting away from upstream KiCad**.)
**Two patterns in KiCad's code:**
- **Pool tasks** (`submit_task`/`submit_loop` on `GetKiCadThreadPool()`) — reuse the standing pool
threads. No new Workers.
- **Raw `std::thread`** — create a brand-new thread each time, *outside* the pool. Since the
pre-warmed bag is already drained by the pool, these force **on-demand Worker creation** (§4).
---
## 2. What runs multithreaded today: nothing — the shims + the full raw-thread inventory
### The pool is funneled inline
KiCad routes its data-parallelism through `GetKiCadThreadPool()`. The WASM patch sits at the pool's
single choke point, `bs_thread_pool.hpp:1419`:
```cpp
void detach_task(F&& task, const priority_t priority = 0) {
#ifdef __EMSCRIPTEN__
(void) priority;
std::forward<F>( task )(); // ← inline; never reaches a Worker
return;
#else
/* enqueue + notify_one() a Worker */
#endif
}
```
**Every** entry point funnels through it: `submit_task()` (`:1751`) calls `detach_task`;
`submit_loop`/`submit_blocks` call `submit_task`; `detach_loop`/`detach_blocks`/`detach_sequence`
call `detach_task`. So this one `#ifdef` makes the entire pool serial. (Note `create_threads`
(`:1903`) is **not** shimmed — so the pool still spawns its 16 idle threads/Workers at startup; they
just never get work. Pure overhead.)
Pool consumers now running serial: zone fill (`zone_filler.cpp`, `board.cpp`), **all** DRC providers
(`pcbnew/drc/*`), connectivity (`CONNECTION_GRAPH`), footprint enumeration
(`footprint_info_impl.cpp`), symbol/footprint **library preload** (`pgm_base.cpp:941`,
`pcbnew.cpp:664`), `tracks_cleaner`, plus the raytracer **main trace** (`renderTracing`, which is
pool-based).
### The complete raw-thread inventory
Beyond the pool, raw thread creation across the whole tree (the **wx port and our entire
wasm/shim/scripts layer have zero**):
| Site | What | WASM status |
|---|---|---|
| `render_3d_raytrace_base.cpp:764` (`shadeWorker`) | raytrace post-process shading | **`#ifdef __EMSCRIPTEN__`-guarded → serial** |
| `render_3d_raytrace_base.cpp:835` (`blurWorker`) | blur/finish | guarded → serial |
| `render_3d_raytrace_base.cpp:1456` (`previewWorker`) | preview | guarded → serial |
| `image.cpp:525` (`filterWorker`) | `IMAGE::EfxFilter` (AA/blur) | guarded → serial |
| `create_layer_items.cpp:848` (`zoneWorker`) | zone fill geometry | guarded → serial |
| `create_layer_items.cpp:1311` (`simplifyWorker`) | polygon simplify | guarded → serial |
| `libs/kinng/src/kinng.cpp:57` | IPC-API (nng) listener | **not compiled** — CMake links `kinng` only `if(KICAD_IPC_API AND NOT EMSCRIPTEN)`; IPC defaults **OFF** |
| `kicad/pcm/pcm.cpp:1123`, `pcm_task_manager.cpp` | Plugin & Content Manager (HTTP downloads) | **dormant** — network feature, not in the editor apps |
| `common/eda_dde.cpp:146` | DDE/TCP-socket cross-probe server | **compiled but dormant** — no raw TCP sockets in a browser; should never be constructed |
| `thirdparty/nanoflann.hpp:1278` | `std::async` parallel KD-tree build | conditional/dormant (serial by default) |
| **`common/widgets/font_choice.cpp:99`** (`FONT_LIST_MANAGER::Poll`) | background font enumeration | **UNGUARDED** (only `#ifndef __MINGW32__`) — likely the one place a raw Worker *does* spawn in WASM. Fire-and-forget (no main-thread join), so it does **not** deadlock; verify whether `FONT_LIST_MANAGER` is actually constructed in our apps. |
So the only *perf-relevant* raw threads are the six 3D-viewer ones (all guarded). The rest are
disabled/dormant network-IPC features, except `font_choice`, which is the lone unguarded raw thread.
---
## 3. The three failure modes (the core mechanism)
Keeping these apart is the whole key — different causes, different places, different fixes.
| Mode | Symptom | Where it bites | Root cause |
|---|---|---|---|
| **(a) Deadlock** | frozen tab | raytracer join (any main-thread blocking join needing a new Worker) | On-demand Worker creation needs the main-thread event loop; a *non-yielding* blocking join starves exactly that. Fixed by yielding the join (the nanosleep override) or pre-warming. **See §4.** |
| **(b) Nesting abort** | `Aborted(invalid state: 1)` | a 2nd `emscripten_sleep` started while Asyncify is already Unwinding | Asyncify holds one global suspend state. This bites only a *genuine* nested unwind — **not** an `emscripten_sleep` dispatched at `state == Normal` (e.g. work run from a modal pump's `ProcessEvents`, a fresh managed entry; verified by `raytrace-modal`). |
| **(c) Worker-rewind crash** | `"func is not a function"` in `Asyncify.doRewind` | a C++ throw on a pool Worker under `-fexceptions` | The `invoke_*` exception trampolines are Asyncify imports, so a throw drives an Asyncify transition on the Worker. **Native EH (the default) removes it** — exceptions become native wasm instructions, decoupled from Asyncify. |
The raytracer's worker tasks are mostly **pure math** (no throw, no suspend) — which is why the
parked multi-core pool *ran*: pure-compute tasks don't hit mode (c). The pool tasks that crash
(connectivity, library preload) throw C++ exceptions, which under `-fexceptions` drive Asyncify on
the Worker. **Under native EH (the default) those throwing tasks are safe** — confirmed by
`threadpool-real`, which runs the real pool 16-core with a worker task that throws and is caught.
### What `handlesleep.js` does and does *not* fix
`scripts/common/shims/handlesleep.js` fixes a **specific** nesting: a **fiber swap inside an
`EM_ASYNC_JS` await** (e.g. `ShowModal`) clobbers the single global `Asyncify.currData`; the shim
captures the sleep's buffer and restores it in `wakeUp`. It is *"one level of sleep nesting, blind to
`handleAsync` and to fibers"* and does **not** bypass the `state == Normal` assertion. In practice
that assertion is not hit by the cases we have: work dispatched from a modal pump's `ProcessEvents`
runs at `state == Normal`, so its `emscripten_sleep` join is already legal (`raytrace-modal`). A
**genuine** nested unwind (an `emscripten_sleep` started while already Unwinding) would still need a
cooperative scheduler — the **Design B** design ([`../async/12`](../async/12-design-b-asyncify-implementation-plan.md),
[`../async/13`](../async/13-design-b-engineering-spec.md); **status: Phase 0, not landed**) — but no
current app requires it.
---
## 4. The deadlock, mechanically
### The event loop and "pumping"
Each JS context — the main thread, and each Worker — has **one** call stack and **one** task queue,
on a strict **run-to-completion** model: pick one task, run its *entire* call stack to the end, and
only when it unwinds back to the top pick the next task. **While a task runs, nothing else on that
thread happens** — queued tasks (including messages from Workers) pile up undelivered. **"Pumping the
event loop"** = finishing the current task so the thread returns to drain its queue. A function that
runs long without returning *blocks the event loop* and starves everything behind it.
### Path A — `std::thread` → Worker (the *create* side)
`render_3d_raytrace_base.cpp:762-766`, on the main thread: `std::thread t = std::thread(shadeWorker);`
1. libc++ ctor → `pthread_create` → Emscripten `__pthread_create_js` → `spawnThread` (JS glue).
2. `spawnThread` checks `PThread.unusedWorkers` (the pre-warmed pool):
- **free Worker** → post `{cmd:'run'}` to it; it runs on its own thread. **Main need not pump.** ✅
- **empty** (our case — KiCad's pool drained them) → `new Worker()`; the new Worker boots
**asynchronously**, posts *"I'm loaded"* back to the main thread, and **main's message handler
must run** to then post `{cmd:'run'}`. Finalizing a new Worker **requires main to return to the
event loop.**
### Path B — the join (the *wait* side)
`render_3d_raytrace_base.cpp:768-769`: `while(threadsFinished < parallelThreadCount) std::this_thread::sleep_for(10ms);`
`sleep_for` → `nanosleep`. On the **main browser thread** a real sleep is impossible (and
`Atomics.wait` throws there), so Emscripten implements it as a **busy-wait**: spin on the clock,
return after 10 ms. Wrapped in the `while`, this is **one task that never ends** — the main call
stack never unwinds to the event loop. The only thing that can move `threadsFinished` is a Worker
reaching `threadsFinished++` (`render_3d_raytrace_base.cpp:752`).
### The circular wait
With the pre-warmed pool empty:
1. Main calls `new Worker()` (A), then enters the busy-wait (B) and **stops pumping**.
2. The new Worker boots and posts *"loaded"* into main's queue.
3. **Main never processes it** (stuck in the busy-wait), so it never posts `'run'`.
4. So the Worker never runs `shadeWorker`, never reaches `threadsFinished++`.
5. So the `while` never exits.
> **Main** waits for `threadsFinished` → which needs the **Worker** to run → which needs **Main** to
> pump and post `'run'` → which Main won't do because it's waiting for `threadsFinished`.
A true cyclic dependency. **It is a deadlock, not slowness** — even if Worker boot took 0 ms, it
would never receive `'run'`. Frozen forever, not slow.
### Two ways to break the cycle, both zero-KiCad-edit
**(1) Pre-warm.** If the Worker is already in `unusedWorkers`, the entire "new Worker → loaded
handshake → main must pump" chain is **skipped**: main posts `'run'` directly, the Worker runs *in
parallel* with main's busy-wait, bumps the counter, the spin exits. So `PTHREAD_POOL_SIZE` ≥ (pool
threads + peak raw-thread concurrency) means on-demand creation never happens → **the deadlock
disappears.** The cost: the busy-wait still pegs the main thread → **jank** (not a freeze).
**(2) Make the join yield.** The deadlock is really "main never pumps", so making the wait *yield* to
the event loop fixes both the deadlock *and* the jank. `wasm/shims/nanosleep_yield.c` (a strong
`nanosleep` override) does exactly this: on the main thread a `sleep_for` join becomes an Asyncify
yield (`emscripten_sleep` semantics), so the loop services the on-demand handshake and the Worker
boots; on a worker thread it stays a real blocking sleep. This yield runs at `state == Normal`, so it
does **not** trip mode (b). Proven by `pthread-ondemand` (real pool drains the pre-warmed Workers,
raw fly-threads then boot on demand → multi-core), with no KiCad edit.
---
## 5. Native WASM exceptions (the default) and the failure modes
`-fwasm-exceptions` is a **size/speed de-bloat** that *keeps* Asyncify: it removes the `env.invoke_*`
exception trampolines from `ASYNCIFY_IMPORTS` (`apply-asyncify.sh`), ~59% of the Asyncify tax (pcbnew
**64.5 → ~36 MB gz**; [`../wasm-exceptions/`](../wasm-exceptions)). It is now the **default build**.
- **Raytracer — modes (a)/(b): not an EH question.** Asyncify nesting + main-thread topology. Handled
in the wasm layer: the nanosleep override yields the join (a, §4), and a modal-pump `emscripten_sleep`
runs at `state == Normal` (b, §3) — neither needs EH.
- **Thread pool — mode (c): solved by native EH.** Mode (c) fires when a Worker task drives an Asyncify
transition. Under `-fexceptions`, *exceptions themselves* do that (the `invoke_*` trampolines are
Asyncify imports; landing pads "fire unreliably when unwinding through asyncify frames"). KiCad's
connectivity / library-load throw as ordinary control flow, tripping it. **Native EH makes exceptions
native wasm instructions, decoupled from Asyncify** → a throwing-but-not-suspending Worker task no
longer drives Asyncify. **Confirmed:** `threadpool-real` runs the real `GetKiCadThreadPool()` 16-core
with a worker task that throws and is caught — green under native EH, and *only* under native EH.
- **Per-pass nuance:** the pure-math raytracer passes (shading/blur/`EfxFilter`) don't throw → safe
on Workers regardless. The **geometry** passes (zone fill / polygon `Simplify`) *can* throw →
native EH is what makes them Worker-safe.
- **Async I/O on a Worker:** a Worker doing *async* FS I/O (`EM_ASYNC_JS`) still suspends Asyncify on
that Worker. KiCad's library preload avoids this not by synchronous FS but because our **PCBJAM IO
plugins proxy the async fetch to the main thread** and futex-block the Worker; the only thing left
on the Worker is the S-expr **parse** (a throw), which native EH makes safe. Verified by
`async-preload` (the KiCad-10 `std::async` preload shape — §10, and doc 10 §7).
So a Worker task is fine under native EH as long as its only Asyncify-relevant act was the exception
itself; genuine async suspension must still be kept off the Worker (proxied to main).
---
## 6. Why `PROXY_TO_PTHREAD` is not our escape hatch
The textbook answer to "my native app blocks on joins" is `-sPROXY_TO_PTHREAD`: run `main()` on a
Worker where blocking is legal. In theory the most KiCad-pristine option (delete both shims). In
practice, off the table for us.
- **Asyncify under it?** Mechanically **yes** on the *proxied-main* Worker (own Asyncify state, runs
`main()`). But **not officially supported**, with real sharp edges: `pthread_join` on a thread
running `EM_ASYNC_JS` can hang ([#17552](https://github.com/emscripten-core/emscripten/issues/17552)),
shutdown hangs with raw `handleSleep`/`handleAsync`
([#16940](https://github.com/emscripten-core/emscripten/issues/16940)). **Fibers are thread-pinned**
— [`fiber.h`](https://emscripten.org/docs/api_reference/fiber.h.html): *"Rewind IDs are
thread-specific… impossible to resume a fiber started from a different thread."* Our tool coroutines
are Asyncify fibers.
- **The actual killer — the GUI can't leave the main thread.** Workers have **zero DOM access**; our
wx-dom port renders widgets *as* DOM elements, so every widget op would have to be proxied. WebGL
would need OffscreenCanvas or per-call GL proxying, and those only work for HTML5/SDL2 contexts
([#8852](https://github.com/emscripten-core/emscripten/issues/8852),
[#23666](https://github.com/emscripten-core/emscripten/issues/23666)). Clipboard/input add more
proxying. This is a massive, risky rearchitecture of *our* layer for an unsupported config.
---
## 7. JSPI — not now
JSPI (VM-level stack switching, the Asyncify successor) ships in **Chrome 137+**, **Firefox 139+**,
**Safari 27 beta** (three-engine green only once Safari 27 stables). Closed for *this* codebase
structurally: incompatible with `emscripten_set_main_loop`
([#22493](https://github.com/emscripten-core/emscripten/issues/22493)) — our whole architecture; can't
replace intra-wasm `emscripten_fiber_swap`; a ~350× `JS→C→JS` re-entry regression
([#21081](https://github.com/emscripten-core/emscripten/issues/21081)). Track it; prototype behind
`'Suspending' in WebAssembly` on a small tool. See [`../async/03`](../async/03-solutions-and-prior-art.md)
§3, [`../perf/README.md`](../perf/README.md) lever #10.
---
## 8. Upstream status & the upstreaming path
**Latest upstream KiCad (`master` `9e557f98`, 2026-06-24) has migrated only 1 of 7 raytracer
parallel sections to the pool** — and accidentally:
| Section | Upstream master today |
|---|---|
| `renderTracing()` (main trace) | **Pool** (`submit_task` + `multi_future::wait()`) |
| `postProcessShading` / `postProcessBlurFinish` / `renderPreview` | raw `std::thread` + busy-wait |
| `IMAGE::EfxFilter` (image.cpp) | raw `std::thread` + busy-wait |
| zone-fill / polygon-simplify (create_layer_items.cpp) | raw `std::thread` + busy-wait |
- **The one migration was a side effect.** `b99a43bec2` (2024-09-06) was a cosmetic *"render in
Hilbert-curve order"* commit; the pool move came along for the ride. `bccf36538` (2025-04-07,
*"Isolate thread pool loops"*, fixes GitLab #20572) then refined it from `wait_for_tasks()`
(drain the whole pool) to `submit_task` + per-call `multi_future::wait()`, so a function waits only
on **its own** tasks — the exact cross-frame concern we have, and the pattern any migration should
copy.
- **Origin of the raw-thread pattern:** `f8784f30` (2018-09-21, *"Removing OpenMP"*) hand-translated
`#pragma omp parallel for` into raw `std::thread` + atomic counter + `sleep_for` busy-wait. The
six un-migrated sections are this **untouched 2018 code** — legacy inconsistency, not a deliberate
"don't use the pool" decision.
- **A live upstream motivation:** GitLab **#20911** *"3D viewer ray tracing generates a high system
load"* — the `sleep_for(10ms)` spin-poll + detached-thread churn is high-load *natively*. A pool
migration (submit + futures, thread reuse, no spin) directly improves it.
**Conclusion:** migrating the six sections to the pool (like `renderTracing` already is) is a
**legitimate, upstreamable cleanup** — precedent in the same file, motivation in a filed issue, and
it removes dead OpenMP-era code. If accepted upstream, our fork carries **zero** divergence here, and
it incidentally fixes our deadlock (pool reuse ⇒ no on-demand Worker creation). Keeping KiCad
pristine and going multi-threaded are **not** in tension — the pristine-est KiCad (pool everywhere)
is also the one that threads cleanly in the browser.
---
## 9. The options — and whether they are interchangeable
Two goals: **keep KiCad pristine** and **enable threads**. Mapping candidates to the failure modes:
| | (a) deadlock | (b) nesting | (c) worker-rewind | Net |
|---|:--:|:--:|:--:|---|
| **0. Pre-warm `PTHREAD_POOL_SIZE`** (build-only) | **✓** | n/a* | partial† | Raw-thread raytracer runs multi-core, with jank. Zero KiCad edits. = the parked `WASM_RAYTRACE_POOL`. |
| **1. nanosleep override** (`wasm/shims/`) | **✓** | n/a | — | Main-thread join yields → on-demand Workers boot, no jank, no KiCad edit. Proven (`pthread-ondemand`). |
| **2. Native EH** (the default) | — | — | **✓** | Worker **execution** safe for throwing pool tasks. Confirmed (`threadpool-real`, 16-core + caught throw). |
| **3. `PROXY_TO_PTHREAD`** | sidesteps | sidesteps | sidesteps | Real threads via a DOM/WebGL-proxying rearchitecture. **Impractical for us (§6).** |
| **4. Design B scheduler** | n/a | only a genuine nested unwind | — | Not required by any current app — the modal pump dispatches at `state == Normal` (§3). |
\* The upstream busy-wait never invokes Asyncify, so mode (b) doesn't arise for it. † Pure-math passes
are mode-(c)-safe; geometry passes may throw → want native EH (option 2).
Read off the engines:
- **Raytracer post-process (raw threads):** the **nanosleep override** (option 1) yields the join →
multi-core, no jank, no KiCad edit; the **upstreamable pool migration** (§8) is the pristine option.
- **Pool + raytracer main trace:** **native EH** (option 2, the default) makes throwing worker tasks
safe → the `detach_task` shim can be dropped (a vendored-dep patch ⇒ *less* divergence).
**Where this leaves us:** native EH (default) clears mode (c); the nanosleep override clears mode (a)
and the jank; mode (b) doesn't arise for the cases we have (modal-pump dispatch is `state == Normal`).
All proven on pristine KiCad/wx-core by the doc-10 §6 tests. The remaining work is to **drop the
`detach_task` shim** for real (DRC/zone-fill/connectivity on real Workers) and, optionally, **upstream
the pool migration** (§8) so the fork carries nothing. Keep `PROXY_TO_PTHREAD`/JSPI tracked-only.
---
## 10. Open questions / decisive next steps
1. **Library-preload I/O — answered.** Our fork's library reads are **not** the upstream synchronous
`KICAD_SEXPR` path: the lib-table rows are typed `PCBJAM`/`PCBJAM_FP`, so the runtime plugin is our
custom async bridge, which **proxies the fetch to the main thread** and futex-blocks the Worker — so
no async FS I/O suspends on the Worker. The only Worker-side Asyncify-relevant act is the S-expr
**parse** (a throw), which native EH makes safe. So **option 2 is a shim deletion, not an I/O
rework** for the preload path (verified by `async-preload`; full analysis in doc 10 §7).
2. **Does `FONT_LIST_MANAGER` actually spawn its thread in our apps?** `font_choice.cpp:99` is the one
unguarded raw `std::thread`. Confirm whether it's constructed in WASM (and whether its `Poll`
touches anything that suspends/throws on a Worker), or whether a wasm-specific font path supersedes
it. It won't deadlock (no join), but it likely consumes a Worker.
---
## Sources
**Internal**
- `kicad/thirdparty/thread-pool/bs_thread_pool.hpp:1419` (`detach_task` shim), `:1751`
(`submit_task`→`detach_task`), `:1903`/`:1965` (`create_threads`/`determine_thread_count`)
- `kicad/common/thread_pool.cpp:30-48` (`GetKiCadThreadPool`)
- `kicad/3d-viewer/3d_rendering/raytracing/render_3d_raytrace_base.cpp:752,764,835,1456`,
`image.cpp:525`, `create_layer_items.cpp:848,1311` (raytracer raw threads + serial fallbacks)
- raw-thread inventory: `common/widgets/font_choice.cpp:99`, `common/eda_dde.cpp:146`,
`kicad/pcm/pcm.cpp:1123`, `libs/kinng/src/kinng.cpp:57`; build exclusion in
`common/CMakeLists.txt` (`KICAD_IPC_API AND NOT EMSCRIPTEN`), `CMakeLists.txt:301` (IPC default OFF)
- `scripts/kicad/build-kicad-target.sh:413-415`, `scripts/common/shims/handlesleep.js`,
`scripts/common/apply-asyncify.sh:88`
- [`../async/11`](../async/11-asyncify-nesting-raytracer.md), [`../async/12`](../async/12-design-b-asyncify-implementation-plan.md),
[`../async/13`](../async/13-design-b-engineering-spec.md), [`../async/03`](../async/03-solutions-and-prior-art.md),
[`../wasm-exceptions/README.md`](../wasm-exceptions/README.md), [`../perf/README.md`](../perf/README.md),
[`../../research/threading_2.md`](../../research/threading_2.md)
**Upstream KiCad (GitHub mirror `KiCad/kicad-source-mirror`, master `9e557f98`)**
- `b99a43bec2` (renderTracing → pool, 2024-09-06) ·
[`bccf36538`](https://github.com/KiCad/kicad-source-mirror/commit/bccf36538065a8c318dcdb2bc8b28bd855fb5e81)
(*"Isolate thread pool loops"*, fixes [GitLab #20572](https://gitlab.com/kicad/code/kicad/-/issues/20572)) ·
`452e69de` (pool singleton, 2025-01-05) · `6e2b20ed` (BS pool 5.0, 2025-09-10) ·
`f8784f30` (*"Removing OpenMP"*, 2018-09-21) ·
[GitLab #20911](https://gitlab.com/kicad/code/kicad/-/issues/20911) (raytrace high system load)
**External (Emscripten / browsers)**
- [Pthreads](https://emscripten.org/docs/porting/pthreads.html) · [Asyncify](https://emscripten.org/docs/porting/asyncify.html) ·
[fiber.h](https://emscripten.org/docs/api_reference/fiber.h.html) · [proxying.h](https://emscripten.org/docs/api_reference/proxying.h.html)
- Asyncify×pthreads/PROXY: [#17552](https://github.com/emscripten-core/emscripten/issues/17552),
[#16940](https://github.com/emscripten-core/emscripten/issues/16940),
[#9910](https://github.com/emscripten-core/emscripten/issues/9910)
- WebGL/DOM from a Worker: [#8852](https://github.com/emscripten-core/emscripten/issues/8852),
[#23666](https://github.com/emscripten-core/emscripten/issues/23666)
- JSPI: [#22493](https://github.com/emscripten-core/emscripten/issues/22493),
[#21081](https://github.com/emscripten-core/emscripten/issues/21081),
[V8 JSPI](https://v8.dev/blog/jspi)

9
docs/features/wasm-exceptions/03-toolchain-status.md
View file

@ -1,5 +1,14 @@
# 03 — Toolchain compatibility status (verified 2026-06-10/11)
> **Partly superseded 2026-06-22 — see [`06-spike-plan.md`](06-spike-plan.md).** Corrections:
> (a) the host-side `--asyncify` already runs **Binaryen v130** in CI/publish
> (`BINARYEN_VERSION=130`), so "we don't have the partial support locally" understates it — v121
> is only the finalize/in-link copy. (b) `--pass-arg=asyncify-ignore-unwind-from-catch` **is**
> implemented (shipped v125), but it *silently drops* the suspend — a tripwire-silencer, not a
> fix. (c) The encoding is resolved to **legacy** (exnref + Asyncify is unsupported in every
> released Binaryen incl. v130, no roadmap), so the "encoding decision forks the asyncify work"
> framing in §experiment is closed: legacy + the catch-arm-hoisting pre-pass is the only path.
## The compatibility matrix
| Combination | Status |

75
docs/features/wasm-exceptions/04-kicad-audit.md
View file

@ -1,75 +0,0 @@
# 04 — The KiCad catch-block audit
## Why an audit
Binaryen's asyncify cannot (yet) handle a suspension that begins while execution is inside
a wasm catch handler (see 05). Any C++ `catch` whose handler (directly or transitively)
opens a modal dialog, touches the async clipboard, etc., is therefore illegal under
`-fwasm-exceptions` + ASYNCIFY=1 today. KiCad's standard error pattern is exactly that:
```cpp
catch( const IO_ERROR& ioe )
{
DisplayErrorMessage( this, ioe.What() ); // → ShowModal → Asyncify suspension
}
```
## Method
[`catch_audit.py`](catch_audit.py): walks `kicad/**/*.cpp` (excluding `thirdparty/`, `qa/`),
extracts every catch block with real brace matching (string/comment aware), and classifies
each handler body:
- **direct_suspend** — contains a known suspending call (`DisplayError[Message]`,
`DisplayInfoMessage`, `wxMessageBox`, `ShowModal`, `ShowQuasiModal`, `KIDIALOG`,
`IsOK(`, clipboard ops, `wxFileDialog`, …).
- **infobar** — only `ShowInfoBar*` (non-modal, non-suspending).
- **trivial** — only rethrow / capture / format / logging. Note `wxLogError` is *safe*:
its GUI display is deferred to the idle-time log flush, outside any catch.
`PGM_BASE::HandleException` (`common/pgm_base.cpp:805`) was manually verified — it only
`wxLogError`s → benign.
- **needs_review** — calls functions not classifiable as benign; requires a transitive look.
Full output incl. all site locations: [`audit-results.txt`](audit-results.txt).
Re-run: `python3 catch_audit.py` (path to the kicad tree is hardcoded at the top).
## Results (2026-06-10, kicad @ wasm-port head)
| Category | Count |
|---|---|
| **direct_suspend** | **85** |
| needs_review | 93 (tail looks mostly benign — accessors/file ops; expect ~10–20 to become refactors on inspection) |
| trivial/safe | 458 |
| infobar-only | 0 |
| **total** | **636** |
Per app (direct/review): eeschema 37/17, pcbnew 32/29, common 9/35, cvpcb 3/1, rest
scattered. Concentrated exactly on the file-load error paths the e2e tests exercise:
`pcbnew/files.cpp`, `eeschema/files-io.cpp`, `footprint_libraries_utils.cpp`,
`symbol_library_manager.cpp`, the design-block utils.
wxWidgets adds essentially nothing (zero catch-with-dialog sites in its own code).
## The hand-refactor option (superseded by the fork, kept for the record)
Mechanical hoist per site:
```cpp
wxString err;
try { ... }
catch( const IO_ERROR& ioe ) { err = ioe.What(); } // capture only
if( !err.IsEmpty() ) DisplayErrorMessage( this, err ); // suspend OUTSIDE the catch
```
Effort: 85 hoists (~15–30 min each; error-UX paths with near-zero test coverage) ≈ 3–5
dev-days; 93 reviews ≈ 2–3 dev-days; destructor-during-unwind audit ≈ half day (cleanup
pads = `catch_all`; dialogs from destructors expected zero); **plus** permanent
upstream-sync policing (every KiCad merge adds new `catch { DisplayError }` sites —
`catch_audit.py` as a CI gate automates detection), **plus** the fork-divergence cost
against the "stay close to upstream" policy. Total ~2–3 weeks one-time + maintenance tax,
with hard-crash failure modes for any missed transitive site (asyncify asserts mode traps
deterministically — good in CI, fatal in production).
**Verdict:** with the catch-arm-hoisting fork (05) this entire refactor becomes
unnecessary — all 85 direct sites are C++ catches, which the fork makes legal. Only
suspend-inside-`catch_all`-cleanup remains forbidden, which KiCad does not do.

267
docs/features/wasm-exceptions/06-spike-plan.md Normal file
View file

@ -0,0 +1,267 @@
# 06 — Native wasm-EH: refreshed findings + red-green spike plan (2026-06-22)
> **Status:** plan / decision record, *in refinement* (no code written yet — this is the
> agreed artifact before Phase 0). Produced by a 5-agent research spike on 2026-06-22:
> browser support · toolchain · runtime mechanisms · the Binaryen pass · red-green harness.
> **Supersedes in part** `README.md`, `03-toolchain-status.md`, and the root
> `docs/wasm-exceptions-experiment.md` where called out below. Reads on top of 01–05.
---
## RESULTS — Phases 0 & 1 (2026-06-22, current toolchain, NO emsdk bump)
> **Cross-engine policy:** every spec runs in **all three engines — Firefox, Chrome (V8), and
> Safari/WebKit** (`cd tests && npm run test:asyncify:all`). The eh-spike harness is green in all
> three. Result notes below that say "V8/Firefox" predate the Safari run and now hold in WebKit too.
**Phase 0 — PASS.** em 4.0.2's LLVM emits *parseable, runnable* legacy wasm-EH:
- A trivial `-fwasm-exceptions -sWASM_LEGACY_EXCEPTIONS=1` throw/catch builds (finalize on
bundled v121, **no "popping from empty stack"**) and runs correctly under node
(`tests/apps/standalone/eh-spike/eh_probe.cpp`, Makefile `eh-probe` target). So the parked
experiment's finalize failure was **scale/OCC-specific, not a general codegen break**.
- Binaryen **v130 asyncifies** the legacy-wasm-EH module (asyncify_* exports present); bundled
**v121 crashes** (`UNREACHABLE … Asyncify.cpp:1146 — unexpected expression type`). Confirms
the v121→v130 split; the post-link asyncify path (already `BINARYEN_VERSION=130` in CI) is the
one to use. **No emsdk bump required for the spike.**
**Phase 1 — PASS (red-green captured).** One source built two ways
(`scripts/build-eh-spike.sh`; the wasm-EH variant uses the production stub→post-link-v130 flow).
Validated in **both V8 (node) and Firefox** by `tests/asyncify/eh-spike.spec.ts`:
| case | mechanism | JS-EH | native wasm-EH |
|---|---|---|---|
| `throw_across_sleep` | EM_ASYNC_JS sleep + EH | PASS | **PASS** |
| `fiber_then_throw` | `emscripten_fiber_swap` + EH | PASS | **PASS** |
| `suspend_in_catch` | suspend *inside* a catch arm | PASS | **HARD TRAP** — `indirect call to null` / `null function or function signature mismatch` |
→ asyncify + native wasm-EH **works for the sleep and fiber mechanisms**; the *only* failure is
suspend-inside-catch (Binaryen #4470), and it fails **loud and deterministically** (a trap, not a
silent drop) in both engines. This is exactly the hole the catch-arm-hoisting pass (05) closes —
the spike has now made it concrete and pinned it as a regression test.
**Artifacts:** `tests/apps/standalone/eh-spike/{eh_probe.cpp,eh_spike_test.cpp}`,
`scripts/build-eh-spike.sh`, `tests/asyncify/eh-spike.spec.ts`, the `eh-probe` Makefile target,
and `playwright-asyncify.config.ts` `testMatch` widened to include `eh-spike`.
**Phase 1.5 — PASS (the fix).** The catch-arm-hoisting Binaryen pass
(`scripts/binaryen-hoist-pass/HoistCppCatches.cpp`, ~150 LoC + a 39-line registration patch)
flips `suspend_in_catch` from a hard trap to **green** under native wasm-EH — validated in
**V8 + Firefox**. `tests/asyncify/eh-spike.spec.ts` is now a 3-variant ablation harness:
JS-EH green / wasm-EH-no-pass red / wasm-EH+hoist green (pins both disease and fix). The pass
outlines a cpp-tag catch arm containing a suspending call to plain code after the try (capture
payload→local + set flag + `br` out; in the hoisted handler `pop`→`local.get`, no-match
`rethrow`→`throw`), so stock `--asyncify` instruments it for free — exactly the 05 design. It is
a PRE-pass (`--hoist-cpp-catches` before `--asyncify`); `Asyncify.cpp` is unchanged. Built
reproducibly via `scripts/binaryen-hoist-pass/build-wasm-opt.sh` (drops the file into the v130
clone, applies the patch, `ninja`). **MVP scope** (sufficient for the toy): void/unreachable-typed
tries, single cpp catch (single-i32 tag), no catch_all, and a DIRECT suspending-import call in the
arm; the nested-pop / nested-rethrow cases are handled (via an `ExpressionStackWalker` Try-ancestor
guard). **Remaining for real KiCad** (all "fiddly-but-tractable" per 05): concrete-result-typed
tries (route the body value through a temp local), catch_all coexistence, and TRANSITIVE suspend
detection (catch → DisplayErrorMessage → ShowModal → startModal), which needs the asyncify
ModuleAnalyzer rather than the direct-call heuristic — or simply hoist-all-cpp-catches and let
`-O2` clean up.
**Next:** Phase 2 — flip a small wx standalone app to native EH (the real `ShowModal`-from-catch
path). Phase 3 — full KiCad + the emsdk bump (OCC only). Before KiCad: generalize the pass
(result-typed tries + transitive detection) and file the design on binaryen #4470.
---
## 0. What changed since 01–05 (which were authored 2026-06-11/12)
Three corrections that move the decision:
1. **The encoding is resolved: use the LEGACY encoding (`-sWASM_LEGACY_EXCEPTIONS=1`).**
Not "decide after the emsdk bump" (as README §TL;DR / 03 §experiment / 05 §new-EH frame
it). Binaryen's Asyncify supports **only** legacy `try`/`catch` — never
`try_table`/`throw_ref`/`exnref` — through the latest **v130** (Jun 2026), with no
roadmap, PR, or TODO to change it. The exnref path the parked experiment was forced onto
is therefore a **dead end**: even with the OCC bug fixed, an exnref module dies at the
`--asyncify` step. Legacy has shipped *unflagged* in all three engines since 2021–22
(Chrome 95 / Safari 15.2 / Firefox 100), is still emscripten's own default, and — crucially
— the size prize comes from **native-vs-JS EH (dropping `invoke_*`), independent of the
encoding** — so legacy costs us nothing. **The "exnref → TryTable variant" fork is closed:
legacy + the catch-arm-hoisting pre-pass (05) is the single viable path.**
2. **Binaryen is no longer a blocker — production already ships v130.** README/03 say "our
emsdk bundles v121, we don't even have the partial support locally." That v121 is only the
*finalize / test-apps-in-link* copy. **CI and publish pin `BINARYEN_VERSION=130`**
(`.github/workflows/ci-ubicloud.yml:44`, `.github/workflows/publish-wasm.yml:33`;
resolved by `scripts/common/get-wasm-opt.sh:50`) for the host-side `--asyncify` + `-O2`.
v130 is the latest release and carries the v125 partial legacy-EH asyncify support. So the
migration plan's "step 1: get a newer wasm-opt for the post-link step" is **already done**.
3. **`--pass-arg=asyncify-ignore-unwind-from-catch` is implemented now** (shipped in
Binaryen v125; `03:23` called it "dead docs"). But it is a **tripwire-silencer, not a
fix**: it *silently drops* the suspension inside a catch arm, which is semantically wrong
on paths we actually reach (file-load error dialogs, the eeschema **Paste** handler).
Correctness still requires the catch-arm-hoisting pass (or a hand-refactor).
**Net:** the long pole shrinks to a single thing — the **emsdk / LLVM compiler bump** (for
*parseable legacy wasm-EH* + the OCC `br_table` miscompile), **not** the Binaryen version.
---
## 1. The decision in one line
> `-fwasm-exceptions -sSUPPORT_LONGJMP=wasm -sWASM_LEGACY_EXCEPTIONS=1`, applied **uniformly**
> across deps + wxWidgets + KiCad + test apps; host-side `--asyncify` on Binaryen ≥ v125
> (we have v130); the ~85 suspend-in-catch sites fixed by the **catch-arm-hoisting pre-pass**
> (05), not by source refactors. Prize unchanged from 02: **64.5 → ~36 MB gz download,
> 187 → ~122 MB module**, plus a large `-O2` wall-time drop on the build critical path.
---
## 2. What works / what doesn't (verdicts from this spike)
| | Verdict | Why |
|---|---|---|
| Native wasm-EH, **legacy** encoding, + Asyncify v130, + catch-arm fix | ✅ **viable path** | Mechanically sound per all five agents; legacy is the only encoding Asyncify can instrument. |
| **exnref** (`=0`) + Asyncify | ❌ architecturally broken | Asyncify has zero `TryTable`/`exnref` support in *every* released Binaryen incl. v130; no roadmap. The experiment's `=0` never reached asyncify (blocked earlier at finalize). |
| `--asyncify-ignore-unwind-from-catch` as a *fix* | ❌ not a fix | Exists (v125+), but silently drops the suspend → our catch→modal dialogs misbehave. A tripwire-silencer only. |
| Bundled Binaryen **v121** + Asyncify + any wasm-EH | ❌ crashes | `Asyncify.cpp:998 UNREACHABLE`. Need ≥ v125 — already satisfied for the host-side pass (v130). |
| JSPI instead of Asyncify | ❌ closed for us | Fibers don't exist under JSPI (emscripten #18180); KiCad tools are fibers. (`03`) |
---
## 3. The three asyncify mechanisms vs. the EH switch
All three suspension mechanisms + handle-sleep are **orthogonal** to the C++ EH model — none
touches `__cxa_*`/landing pads in its own implementation. (Full detail: this session's
mechanism map + `docs/features/async/`.)
| Mechanism | EH-model dependence | Verdict |
|---|---|---|
| **Fibers** (`emscripten_fiber_swap`, libcontext/`coroutine.h`) | none — no try/catch in the coroutine layer | just works |
| **Main-loop park** (`emscripten_set_main_loop(...,1)` → `throw "unwind"`) | none — it's a **JS string** throw, not `__cxa_throw`; swallowed in JS glue | just works |
| **EM_ASYNC_JS sleeps + handle-sleep engine** (modal/nested-loop/clipboard/fonts/DOM-popup/PCBJam fetch) | implementation: none. **Callers** are the risk. | at-risk *only* via the catch-arm caller pattern |
The §3c trampoline-heal (`inject-dyncall-shims.sh`) and `handlesleep.js` "unwind" catches are
**JS-level and EH-independent** — unaffected by the switch.
**The entire exposure** is the **~85 sites where a C++ `catch` arm opens a modal →
asyncify-suspends** (e.g. `pcbnew/files.cpp:674/684/692`; 6 of them on a coroutine/fiber
stack including the eeschema Paste handler). This trap has **never fired at runtime** — it is
a static conclusion (brace-match audit `04` + Binaryen `AsyncifyFlow` skipping catch bodies
`05`). **Making it concrete and proving the fix is the spike's core job.**
---
## 4. Blockers, ordered (corrected)
| # | Blocker | Status / fix | Confidence |
|---|---|---|---|
| 1 | **emsdk 4.0.2 → 6.0.0 (LLVM 23) compiler bump** | em 4.0.2 emitted legacy wasm-EH that failed Binaryen parse at finalize (`popping from empty stack`) with **both** v121 and official v130 → an **LLVM-output bug, not a Binaryen-version bug**; a newer LLVM should fix it. Also lifts the bundled finalize-binaryen to ~v130. **Changes the JS-EH build too → must revalidate the whole project.** | parse-failure cause **unconfirmed** (Phase 0 resolves) |
| 2 | **OpenCASCADE** | (a) invalid `br_table` arity in `ShapeUpgrade_SplitSurface::Build` under wasm-EH — candidate upstream fix is **LLVM PR #123915** (Jan 2025, "add unreachable before catch destinations"), so the LLVM-23 bump likely covers it. (b) Separate `OCC_CONVERT_SIGNALS` setjmp↔exception-in-one-function conflict — **likely already sidestepped** by `-sSUPPORT_LONGJMP=wasm`; verify, else drop the flag. | both **to verify** in Phase 3 |
| 3 | **~85 suspend-in-catch sites** | catch-arm-hoisting Binaryen pre-pass — ~400–800 LoC, 1–2 wk, upstreamable, **keeps KiCad pristine** and obsoletes the hand-refactor + CI gate (05, 04). Alternative: hand-refactor (2–3 wk + permanent `catch_audit.py` CI gate + fights our upstream-closeness policy). | design **sound** (05); not yet built |
| 4 | **Safari 26.0 regression** (watch) | `-fwasm-exceptions` *legacy* apps transiently crashed at startup on Safari 26.0's initial release (in-place-interpreter bug, emscripten #25365), since patched. Track Safari point releases. | external, **patched** |
Browser support is otherwise a non-issue: legacy EH ≈ Chrome 95+/Safari 15.2+/Firefox 100+,
>96% of traffic, a 4-year tail. The "Chrome problem" (V8 slow to ship *exnref*) doesn't touch
us because we never emit exnref.
---
## 5. The phased red-green plan
**Key enabler (agent 5):** the Asyncify×wasm-EH interaction is **toy-testable now, without the
emsdk bump** — the bump's blockers are OCC-specific, and the toy has no OCC. So we separate the
two risks: *compiler bump* (full-KiCad only) vs *asyncify×wasm-EH semantics* (provable on a toy
today). We reuse the existing ablation harness pattern in
`tests/apps/standalone/{asyncify-races,coroutine}/` + `tests/asyncify/`.
### Phase 0 — micro-probe (hours, current toolchain)
**Question:** can em 4.0.2's LLVM emit *parseable* legacy wasm-EH on a tiny no-OCC program?
- Minimal standalone C++ (no wx, no OCC): a `try { throw } catch(...) {}` + a trivial sleep.
- Build `-fwasm-exceptions -sSUPPORT_LONGJMP=wasm -sWASM_LEGACY_EXCEPTIONS=1`, **`-sASYNCIFY=0`
at link**; does it link + `wasm-emscripten-finalize` (bundled v121) cleanly? (i.e. reproduce
the experiment's `=1` failure, or not).
- Then standalone `wasm-opt --asyncify` via `get-wasm-opt.sh` with `BINARYEN_VERSION=130` —
does v130's asyncify parse + instrument the legacy-EH module?
- **Gate:** both clean → Phase 1 proceeds on current toolchain, bump deferred to Phase 3.
Finalize fails the same way → em-4.0.2 legacy codegen is generally broken → **front-load the
bump** (Phase 0′) before the toy.
### Phase 1 — red-green toy (days)
**Goal:** prove all three mechanisms survive native wasm-EH, and that **suspend-in-catch is the
only failure** (red) — then green once the fix lands.
- Add `tests/apps/standalone/eh-spike/eh_spike_test.cpp`, console protocol
`[EH_SPIKE] PASS/FAIL/SUMMARY` (model on `races_test.cpp`). Three cases:
(a) **throw across a sleep**; (b) **suspend inside a catch** (the #4470 case);
(c) **fiber-swap then throw** (include `../coroutine/kicad_coroutine_harness.h`).
- One source, two builds via a make var `EH_FLAGS`: `-fexceptions` (green baseline) vs
`-fwasm-exceptions … =1`. No-wx/no-OCC (template on the `coroutine-pthread` no-wx
`LDFLAGS`), so we skip the wx-EH rebuild and the OCC bug.
- **Decouple asyncify**: link `-sASYNCIFY=0`, then standalone `wasm-opt --asyncify`
(`get-wasm-opt.sh`, v130) so the in-link v121 doesn't poison case (b). Re-inject shims.
- Playwright spec `tests/asyncify/eh-spike.spec.ts` (model on `asyncify-races.spec.ts`,
reuse `findSummary`/`crashLines`): JS-EH build **all green**; wasm-EH build **green on (a)/(c),
red on (b)**. Build commands: `scripts/build-wasm-test.sh eh-spike-{js,wasm}`; run
`cd tests && npx playwright test --config=playwright-asyncify.config.ts --project=firefox …`
(widen `testMatch` or name it `asyncify-races-eh.spec.ts`).
- **Bonus data:** characterize what (b) does under wasm-EH — hard trap vs silent drop (the
`ignore-unwind-from-catch` behavior). Informs decision-point #1.
### Phase 1.5 — the catch-arm-hoisting pass (1–2 wk; only if Phase 1 confirms (b) is the sole gap)
- Implement `src/passes/HoistCppCatches.cpp` + 2 registration lines in a Binaryen fork branch
(05 §transform). Wire via `get-wasm-opt.sh` `BINARYEN_BUILD_FROM_SOURCE=1` pointed at the
fork branch (`get-wasm-opt.sh:59-90`; one-line URL/branch change).
- Prove case (b) goes **green** on the toy. File the design on **binaryen #4470** first
(upstreamability).
### Phase 2 — wx standalone (days)
- Flip a small wx test app to native EH (wx native-EH rebuild via the `KICAD_WASM_EH` gate;
flag set patch-ready in the experiment appendix — but with `=1`, **not** the experiment's
`=0`). Exercises the **real** `ShowModal`-from-catch suspend in miniature; confirms the pass
handles the actual wx modal, not just a toy sleep.
### Phase 3 — full KiCad (weeks)
- emsdk bump → 6.0.0 (LLVM 23): **revalidate the JS-EH build first** (whole-project risk).
- OCC: verify `br_table` fixed; verify (and if needed remove) `OCC_CONVERT_SIGNALS`.
- Apply the experiment appendix patch (`KICAD_WASM_EH=1`) **corrected to `=1`**; uniform flag
flip across deps/wx/kicad/tests; **drop `env.invoke_*` from `ASYNCIFY_IMPORTS`**
(`apply-asyncify.sh` — README cites `:33`, current read ~`:88`; verify).
- Build host-side asyncify with the hoisting pass (fork). e2e audit under
`-sASYNCIFY_ASSERTIONS` to flush any missed suspend-in-catch. Measure the gz/module win + the
`-O2` wall-time drop.
---
## 6. Open decision points (to refine together)
1. **Catch-arm fix strategy:** hoisting pass (rec — pristine KiCad, upstreamable) vs
hand-refactor 85 sites (CI gate, fights upstream-closeness) vs ship-with-`ignore-flag` and
accept degraded error dialogs (fast, semantically wrong). *Phase 1 data informs this.*
2. **emsdk target:** 6.0.0 (latest, LLVM 23) vs a more conservative 5.0.x — 6.0.0 carries
other breaking changes (startup `async/await`, compiler-rt naming) needing JS-EH-side
revalidation.
3. **Binaryen fork hosting:** real git submodule (sibling to kicad/wxwidgets, pinned) vs a
lighter `BINARYEN_BUILD_FROM_SOURCE` branch URL. (Fork is a *build tool*, not conveyed code
— no GPLv3 `BUILD_SHA` treatment needed.)
4. **Run Phase 1 on current em 4.0.2** (if Phase 0 passes) **vs bump first regardless.**
5. **Sequencing vs the Asyncify-arbiter work** (`docs/features/async/`): README §relationship
says arbiter-first (it fixes shipping bugs, needed under either EH model). Still the priority?
---
## 7. Risks / unknowns (honest)
- The `popping from empty stack` cause is **unconfirmed** — Phase 0 is the cheap decider.
- Whether case (b) under wasm-EH **traps vs silently drops** — Phase 1 characterizes.
- Whether **LLVM 23 actually fixes** our OCC `br_table` (PR #123915 is a strong candidate, not
confirmed against our exact OCC code) — Phase 3 verifies.
- Safari 26.0 legacy-EH regression — patched upstream, but a reminder to track point releases.
- File:line refs in 03/05/README and the experiment appendix may have **drifted** — re-verify
before editing (e.g. `apply-asyncify.sh` `ASYNCIFY_IMPORTS` line, build-script EH-flag lines).
---
## 8. Provenance
5-agent spike, 2026-06-22 — browser support, toolchain, runtime mechanisms, the pass design,
red-green harness. Key external refs: WebAssembly/binaryen **#4470** (open) / **#5475** (merged
v125), **LLVM PR #123915**, **emscripten #25365** (Safari 26.0), webassembly.org **Wasm 3.0**
(Sep 2025). Internal: this dossier 01–05, `docs/wasm-exceptions-experiment.md`,
`docs/features/async/`, `docs/features/perf/README.md` (lever #9), and memory
`wasm-eh-migration-assessment`.

244
docs/features/wasm-exceptions/07-spike-results-and-opinion.md Normal file
View file

@ -0,0 +1,244 @@
# 07 — Native wasm-EH spike: results and engineering opinion (2026-06-22)
> What the spike actually built and proved, then a candid opinion on whether to pursue the
> `-fwasm-exceptions` migration and how. Companion to [`06-spike-plan.md`](06-spike-plan.md)
> (the plan + Phase 0/1/1.5 result log). Verdict up front, evidence and caveats after.
## Verdict
**The migration is viable on the current toolchain, the single blocking limitation is real
and narrow, and a bounded Binaryen pass fixes it. I recommend pursuing it** — but the
schedule risk is the **emsdk bump for OpenCASCADE**, not the exception machinery, and the
pass still needs generalization before full KiCad. This was a genuine de-risking: the core
"can Asyncify and native wasm-EH coexist?" question is now answered **yes, empirically**,
in **all three engines — Chrome (V8), Firefox (SpiderMonkey), and Safari (WebKit)** — with no
emsdk bump required to prove it. (Project policy, set this session: every spec is run in all
three browsers — `cd tests && npm run test:asyncify:all`.)
## What was proven (with evidence, not estimates)
Everything below ran on the **current pinned toolchain** (emscripten 4.0.2, Binaryen v130 via
`BINARYEN_VERSION=130`) — **no emsdk bump**.
1. **em 4.0.2 emits runnable legacy wasm-EH.** A trivial `-fwasm-exceptions
-sWASM_LEGACY_EXCEPTIONS=1` throw/catch builds (finalize on the bundled v121, *no* "popping
from empty stack") and runs correctly under node. So the parked experiment's finalize
failure was scale/OCC-specific, not a general codegen break.
(`tests/apps/standalone/eh-spike/eh_probe.cpp`.)
2. **Binaryen v130 asyncifies legacy-wasm-EH; v121 cannot.** v130 instruments the module
(asyncify_* exports appear); the emsdk-bundled v121 crashes (`UNREACHABLE …
Asyncify.cpp:1146`). We already ship v130 for the post-link asyncify, so this costs nothing.
3. **The three Asyncify mechanisms split exactly as predicted.** A red-green toy
(`eh_spike_test.cpp`) exercises sleep-across-throw, fiber-swap-then-throw, and
suspend-inside-catch. Under native wasm-EH: **sleep ✅, fiber ✅, suspend-in-catch ❌
(hard trap: `indirect call to null`).** Identical in V8 (node) and Firefox. So the *only*
failure mode is the documented one (Binaryen #4470: AsyncifyFlow skips catch bodies), and
it fails **loudly and deterministically** — not a silent corruption.
4. **A ~150-line Binaryen pass closes it.** `--hoist-cpp-catches`
(`binaryen/src/passes/HoistCppCatches.cpp`, our fork) flips suspend-in-catch to **green**
in both engines. `tests/asyncify/eh-spike.spec.ts` is a 3-variant ablation harness pinning
JS-EH-green / wasm-EH-red / wasm-EH+hoist-green. Rebuilt and re-verified end-to-end from the
tracked submodule.
## The artifacts (all reproducible)
| Thing | Where |
|---|---|
| Red-green toy (3 mechanisms) | `tests/apps/standalone/eh-spike/eh_spike_test.cpp` |
| Phase-0 probe | `tests/apps/standalone/eh-spike/eh_probe.cpp` (+ `eh-probe` Makefile target) |
| 3-variant build (stub→post-link-v130, +hoist) | `scripts/build-eh-spike.sh` |
| The Binaryen pass | `binaryen/` submodule (fork, branch `wasm-port` = `version_130` + the pass) |
| Fork build wrapper | `scripts/binaryen-hoist-pass/build-wasm-opt.sh` |
| Red-green-fixed spec | `tests/asyncify/eh-spike.spec.ts` |
## How the pass works (one paragraph)
For a `try` whose cpp-tag `catch` arm contains a suspending call, it rewrites the arm to just
*capture the exception payload into a local and set a flag*, and **hoists the real handler to
plain straight-line code after the try**, guarded by the flag. In the hoisted handler the
payload `pop` becomes a `local.get`, and the personality no-match `rethrow` becomes an explicit
`throw` of the cpp tag with the captured payload. Stock `--asyncify` then instruments the
hoisted handler like any other code — the upstream "no pause/resume inside catchBodies"
invariant becomes true *by construction*. It is a **pre-pass**; `Asyncify.cpp` is unchanged,
which is why it's a clean ~150-line addition and genuinely upstreamable.
## Generalization (follow-up, same session)
The pass was generalized from the MVP (one direct-suspend catch) to **hoist-all-cpp-catches** and
tested against a richer toy covering the real KiCad/wx shapes. **All 7 shapes are green in all
three engines** (Firefox + Chrome + Safari/WebKit):
| shape | covered |
|---|---|
| direct suspend in catch | ✅ |
| transitive (catch → helper → … → suspend) | ✅ (hoist-all; direct detection would miss it) |
| value-returning try/catch | ✅ (LLVM keeps the value in a local → void try, no result routing needed) |
| suspend-in-catch on a fiber/coroutine stack (eeschema Paste) | ✅ |
| nested suspend-in-catch tries | ✅ |
| **catch nested in a catch_all cleanup** (try body has a local with a destructor) | ✅ (escape past the outermost try; see below) |
**The one gap — catch_all-wrapped catches.** When the try body holds a local with a non-trivial
destructor, LLVM lowers the C++ catch *nested inside* the cleanup `catch_all`
(`catch_all { ~g; try { rethrow } catch $cpp { sleep } }`). Hoisting the cpp catch leaves the sleep
inside the `catch_all`; legacy `catch_all` gives no payload to capture/re-raise, so the sleep can
only be freed by hoisting **past the outermost enclosing try** — have the cpp catch capture the
payload and `br` to a `$done` block placed after that try. **Prototyped this session and
reverted:** the escape transform *validates* and fixes the catch_all case *in isolation*, but the
`block` + `br` + flag-dispatch control-flow shape it produces is **not asyncify-rewindable**
(rewind traps with `null function`) and it regressed the simple cases too. So the real work is
finding an asyncify-friendly escape shape — the per-try inline `br_if`-skip form (handler inline
right after the try) rewinds fine; a `br` out to a separate dispatch does not. That's the
fix is now landed (see "catch_all-escape: LANDED" below); all 7 shapes are green. How often it
bites KiCad depends on whether the specific catch's try body constructs a destructible
local/temporary (e.g. a `wxString`); a `try { ptr = Load(fn); } catch(IO_ERROR&)` with a pointer
result has no cleanup pad and is already covered. (`HOIST_ONLY_SUSPEND` switches off hoist-all back
to direct-suspend-only, useful for narrowing blast radius while debugging.)
### catch_all-escape: LANDED (2026-06-22)
**Fixed — all 7 shapes green in Firefox + Chrome + Safari.** When the try body holds a local with a
non-trivial destructor, LLVM lowers the C++ catch *nested inside* the cleanup `catch_all`; the pass
hoists it PAST the outermost enclosing try. Confirmed to occur in real KiCad (`pcbnew/files.cpp:670`
builds `std::map<std::string, UTF8> props` in the try, so its IO/format/bad_alloc catches are
catch_all-nested). Landing it took a from-source Binaryen build + a minimal *multi-function* repro
(`/tmp/eh_min2.cpp`, `/tmp/eh_min3.cpp`); two bugs, both invisible on a single function and only live
once several shapes inline together:
1. **Over-eager deferral → `null function`.** A nested cpp catch is deferred to its ancestor escape
target, but the test matched ANY ancestor catch body — so a cpp catch in a *regular* catch body
(the `__cxa_end_catch` cleanup tries LLVM emits everywhere) was deferred to a target that never
hoisted it; its suspend was dropped and rewind trapped. Fix: defer only when the catch sits in an
ancestor's `catch_all` cleanup pad (`hasCatchAll() && catchBodies.back() == child`).
2. **Trailing catch_all code → `unreachable`.** The rewritten minimal arm completes, but the
catch_all body has trailing `(unreachable)` after the nested try (it assumed the handler
diverged). Fix: wrap the escape target in a `block $esc`; the arm `br $esc`s after capturing,
landing fall-through just before the dispatch (so Asyncify still rewinds the handler).
The pass is in the `binaryen` submodule (`src/passes/HoistCppCatches.cpp`). The earlier
"not asyncify-rewindable" worry was wrong — Asyncify rewinds the escape form fine; the blockers were
ordinary IR bugs, exactly as the multi-function-debug plan predicted.
### value-typed (concrete-result) tries: LANDED (2026-06-22)
The pass also handles an escape target whose try yields a *value* (i32/i64/…), not just
void/unreachable — it routes the body/handler value through a `$result` local (the no-exception
body value is captured inside `block $esc`; a caught arm br's out and each per-arm dispatch writes
`$result`; the block yields `local.get $result`). Non-defaultable result types are still skipped.
These tries don't arise from normal C++ EH lowering (LLVM keeps catch values in locals →
void/unreachable tries), so they're covered by hand-written modules in
`scripts/binaryen-hoist-pass/tests/` (`run.sh`): `--fuzz-exec` confirms the pass preserves the
result value across the exception / no-exception / payload paths, and a real asyncify unwind+rewind
through a value-typed *suspending* catch yields the correct value (50).
#### Debugging history (superseded)
The notes below trace the path to the fix; their "blocked" conclusions are superseded by the
landing above.
##### catch_all-escape: confirmed real; the fix is asyncify-SOUND, not a wall (2026-06-22)
> **Correction (later same session):** the "not asyncify-rewindable" conclusion below was
> **disproven**. Diffing the *asyncified* output of the per-try vs escape forms on a minimal
> single-function suspend-in-catch (`/tmp/eh_min.cpp`) shows them **structurally identical** — all
> 22 diff hunks are pure local-index renumbering — and **both run cleanly in node**. So Asyncify
> rewinds the escape form fine. The real blocker is ordinary structural bugs in the escape pass on
> the complex *inlined* toy (one found: the skip-to-escape-target coordination drops a catch when
> its escape target is value-typed; fixing that surfaced a load-time trap, so there's ≥1 more).
> That is tractable engineering — methodical per-function isolation like the eh_min repro — **not**
> an asyncify-internals wall. WIP + partial fix preserved in the escape-wip file below.
> **Further localization (same session):** the breakage is **not** fiber-specific (cases 1/3/4/5
> with no fibers still trap) and **not** one bug. It is a **layout-sensitive structural corruption**
> the escape restructure introduces on MULTI-function modules — `null function` / wrong
> `call_indirect`, which V8 then mis-compiles unpredictably (the trap point *moves* with module
> composition). Single-function repros (`eh_min`) work; the corruption only appears once several
> functions/cases compile together. So the next step is NOT more single-function isolation but a
> small **multi-function** repro under a Binaryen **debug build (assertions)** + `--fuzz-exec`, to
> catch the exact expression the restructure corrupts. Deferred to dedicated debugging.
**Confirmed we DO hit the gap.** A spot-check of the audited sites found destructible locals in the
try bodies: e.g. `pcbnew/files.cpp:670` declares `std::map<std::string, UTF8> props;` in the try,
so its three `catch (… ) { DisplayErrorMessage(…) }` arms are lowered nested inside a cleanup
`catch_all`. The file-load sites generally construct `wxString`/`std::map`/smart-pointer locals, so
this is not academic — a real subset of the ~85 sites is affected.
**The fix was attempted extensively and is blocked.** The escape-target restructure (hoist the cpp
catch — own or nested — past the outermost enclosing try, dispatching handlers after it) **validates**
in every variant but is **not asyncify-rewindable**: it traps with `null function` even on the simple
cases the per-try form handles. Tried: inline flag-dispatch (`if (flag==n)` — asyncify skips `if`
bodies on rewind), a bare single handler, `br_if`-skip guards, and `ReFinalize` (for stale `Try`
types). None worked at the time — the actual root causes (over-eager deferral + trailing catch_all code)
were found later with a multi-function repro; the fix landed in the submodule pass (see above).
**Open options:** (1) diff the *asyncified* IR of the working per-try form vs the escape form on one
simple case, to pinpoint exactly what Asyncify mis-handles; (2) hand-refactor the affected KiCad
sites (move the destructible local out of the try body) — a targeted subset, not all 85; (3) the new
`exnref` EH encoding gives `catch_all` a payload (a clean fix) but Asyncify has no `exnref` support.
The per-try pass (6/7 shapes) is the shipped state.
## Opinions (the part you asked for)
**1. Do it — the size/perf prize is real and the risk is now bounded.** −44% download / −35%
module (measured, see 02) plus a large `-O2` build-time drop. The thing everyone feared
(Asyncify ⊥ wasm-EH) is disproven. I would not have said this before the spike; I say it now
because the toy actually runs.
**2. Switch the pass from "selective" to "hoist-all-cpp-catches" before KiCad.** My MVP only
hoists arms with a *direct* suspending-import call. KiCad's real pattern is **transitive** —
`catch (IO_ERROR&) { DisplayErrorMessage(...); }` → `ShowModal` → `startModal` — so direct
detection would miss most of the 85 audited sites. The design doc already recommends hoist-all
+ let `-O2` prune the no-op hoists, and having now written the selective version I agree: it
removes the call-graph analysis entirely, is robust to transitivity, and the only cost is
transforming more tries (which `-O2` coalesces). Selective was the right call for *proving the
concept with minimal blast radius*; hoist-all is the right call for *shipping*.
**3. The remaining pass work is small (~1–2 days), and I know exactly what it is.** (a)
Concrete-result-typed tries — route the body value through a temp local (the toy already
forced me to handle `unreachable`-typed; `i32`/others are the same shape). (b) `catch_all`
coexistence on the same try (cpp catch + cleanup pad). (c) hoist-all gating. None are research;
all are mechanical Binaryen-IR work. The two real-IR gotchas are already solved in the MVP:
**nested-catch pops** (don't clobber a nested catch's payload — fixed with a Try-ancestor guard)
and **nested suspend-in-catch tries** (KiCad will have these; the toy already did, and hoisting
*both* was required).
**4. The schedule risk is the emsdk bump, not exceptions.** Everything above avoided the bump.
Full KiCad cannot: OpenCASCADE miscompiles a `br_table` under wasm-EH on em 4.0.2 (candidate
LLVM fix exists; em 6.0.0 = LLVM 23 should cover it), and the bump changes the compiler for the
*JS-EH build too* → whole-project revalidation. That is the multi-week, cross-cutting item.
Budget the migration as "1–2 days pass + N weeks emsdk-bump-and-revalidate," not the reverse.
**5. Keep the legacy encoding; ignore the exnref/Chrome noise.** Asyncify can only instrument
legacy `try/catch` (no roadmap to change through v130), and legacy ships unflagged everywhere
since 2021. The size win is native-vs-JS EH, independent of the encoding — so legacy is free
and correct. The exnref "Chrome problem" never touches us.
**6. Honest caveat — the toy is small; scale is unproven.** This spike de-risks the *semantic*
interaction, not KiCad-scale behavior. Two known scale hazards remain untested under wasm-EH:
V8's per-function locals limit on huge asyncified functions
([[chrome-asyncify-rewind-crash]]) and unwind-time landing-pad reliability
([[asyncify-eh-unwind-landing-pads-unreliable]] — which *might improve* under wasm-EH, worth
re-checking). The Safari 26.0 transient legacy-EH crash (emscripten #25365, since patched) is a
reminder that even legacy can break on a fresh engine. None of these are blockers; all are
"verify at scale," and the recommended order (toy → wx app → full KiCad) is designed to surface
them cheaply.
## Recommended path forward
1. **Generalize the pass** — DONE: hoist-all, catch_all-escape, and value-typed/concrete-result
tries are all handled and verified (the value-typed path via `scripts/binaryen-hoist-pass/tests/`).
No further generalization is needed for the 7 KiCad shapes.
2. **Phase 2 — a wx standalone app** flipped to native EH: the first *real* `ShowModal`-from-catch
path, and the forcing function for transitive hoisting.
3. **File the design on binaryen #4470** (the pass is a pure addition; upstreaming collapses our
fork back into stock wasm-opt eventually).
4. **Phase 3 — full KiCad**, gated on the emsdk bump (the real work) + the generalized pass +
the uniform flag flip + dropping `env.invoke_*` from `apply-asyncify.sh`, with an e2e audit
under `-sASYNCIFY_ASSERTIONS`.
## Status of the tracked changes (for review)
- `binaryen/` submodule added (fork `emergence-engineering/binaryen`), branch **`wasm-port`** at
`version_130 + 1` (`58f25ebb2`) — the pass is **committed in the submodule but not pushed**.
Pushing the branch to the fork (and committing the parent gitlink) is the user's call.
- Parent-repo changes are **uncommitted**, pending review: `.gitmodules` + the `binaryen`
gitlink, the spike toy/scripts/spec, and these dossier docs.

175
docs/features/wasm-exceptions/08-wx-app-render-rootcause.md Normal file
View file

@ -0,0 +1,175 @@
# 08 — Native wasm-EH: the wx application (Phase 2) — render-failure root cause & fix (2026-06-22)
> Phase 2 of the plan in [`07-spike-results-and-opinion.md`](07-spike-results-and-opinion.md):
> flip a real wx standalone app (the `dialog` test) to `-fwasm-exceptions` end-to-end and see if
> it runs. It builds and links clean, but rendered **blank**. This documents the deep-debug that
> found *why*, the one-line-of-reasoning root cause, the fix, and an **honest account of what is
> proven vs. still open** — including a render/screenshot discrepancy that is not yet resolved.
## Status (read this first)
- **Proven, C++-level:** the app was **destroying its own main window during startup** under
native wasm-EH. Root cause identified with certainty (instrumented build), and the fix makes
the destruction **stop** (the `~wxNonOwnedWindow` destructor no longer fires). That specific bug
is fixed, and the *why* is understood and re-derivable.
- **Proven, in my checks:** after the fix, a headless-Chromium load of `dialog_test.html` showed a
full render — `#canvas` present and visible, **5 buttons**, the description text, the event-log
control, the status bar; `canvases=1`, `traps=0`; and the screenshot I captured showed the
complete dialog UI.
- **OPEN / unresolved:** the screenshot is reported **empty** on inspection. My headless ad-hoc
check and that observation **disagree**, and I have not reconciled them. **Do not treat the app
as "verified rendering" yet.** See [§Open: the empty-screenshot discrepancy](#open-the-empty-screenshot-discrepancy).
- **Not yet done:** the real e2e spec in all three browsers; modal dialogs (which now nest Asyncify
one level deeper); the rest of the wx suite; cleanup/commit of the Phase-2 changes.
## The symptom
`dialog` built and linked under `-fwasm-exceptions -sWASM_LEGACY_EXCEPTIONS=1` (libwx + app, with
the post-link hoist+asyncify pipeline). At runtime: boots, prints its startup logs, **no JS error,
no wasm trap, main thread responsive** — but `#window-container` empty, no visible canvas, the
e2e `waitForApp` (waits for a visible `#canvas`) would time out. A silent non-render.
## How it was found (the debug chain)
Each step ruled out a hypothesis and narrowed the next. All via injected logging in the built
glue + instrumented libwx rebuilds (the browser symbolizes wasm frames only as `wasm-function[N]`,
so callstack mapping was a dead end on a release build — direct source instrumentation was the
reliable tool).
1. **It's not a trap or asyncify/indirect-call corruption.** Calling a wasm export (`ProcessEvents`)
from JS post-boot returns cleanly. The "table index out of bounds" seen earlier was an artifact
of my own `Module.Asyncify` probe, not the app.
2. **`main` "throws `unwind`"** — but that is **normal**: it's `emscripten_exit_with_live_runtime`'s
sentinel, caught and swallowed by `handleException` (glue line ~5066). A red herring on its own.
3. **The main window is created, then destroyed.** `createWindow(id=-1 → cssId 0)` runs in the frame
ctor; then `destroyWindow(0)` runs — the DOM window is torn down. `wxNonOwnedWindow::~wxNonOwnedWindow`
is the caller (it `EM_ASM`s `destroyWindow(m_cssId)`). So **the frame's own destructor runs during
startup**, leaving the app with no window.
4. **The destruction is deliberate, not an exception unwind.** Instrumented `~wxNonOwnedWindow` to log
`std::uncaught_exceptions()` → **0**. So no C++ exception is in flight; this is a normal destructor
call. (This momentarily looked like it ruled out the landing-pad hazard — it didn't; see root cause.)
5. **It happens *after* `OnInit` fully completes.** Logged `OnInit`: "frame created" → "Show done,
returning true" both print *before* the destructor. So the teardown is in **`OnRun`**, not OnInit.
6. **It happens *inside* `emscripten_set_main_loop`.** Bracketed `wxGUIEventLoop::DoRun`'s
`SetSize`/`Refresh`/`set_main_loop` with logs. Order: "before set_main_loop" → **then** the
`~wxNonOwnedWindow`. So the frame dies *during* the `emscripten_set_main_loop(ProcessEvents, 0, 1)`
call.
## Root cause (one paragraph)
`emscripten_set_main_loop(fn, fps, simulate_infinite_loop=1)` implements "loop forever" by **throwing
a JS `"unwind"` exception to abandon the C++ stack** — the code after it never runs; the browser drives
`fn` thereafter. That `"unwind"` propagates out through every C++ frame between `set_main_loop` and
`callMain`. Under **native wasm-EH**, the compiler emits `catch_all` **cleanup** landing pads (for
destructors/RAII) that **reliably catch any in-flight exception — including a foreign JS one** — run
their cleanup, and rethrow. As the `"unwind"` passes back through `wxEntry`/`OnRun`, those cleanup pads
fire and **destroy `wxTopLevelWindows.front()` — the main frame** — before the browser ever calls
`ProcessEvents` to paint it. `uncaught_exceptions()==0` is consistent: the `"unwind"` is a *JS*
exception, invisible to the C++ exception machinery, so the cleanup-pad destructors see no C++ unwind
in progress.
This is the **inverse** of the documented hazard
[`asyncify-eh-unwind-landing-pads-unreliable`]: under legacy `-fexceptions` the cleanup landing pads
fire **unreliably**, and that *accidentally* spared the frame (the destroy that should run, didn't).
Native wasm-EH makes them reliable — so the latent "abandon-the-stack vs. run-the-cleanup" conflict
finally bites. The JS-EH build never rendered-correctly-by-design here; it rendered correctly **by a
landing-pad bug canceling a stack-abandon assumption.**
## The fix
> **Superseded form (2026-06-23):** the fix described in this section is the *interim* **option A** (`wxWasmRunNestedLoop` / `setTimeout` pump). The final form is the **rAF pump** `wxWasmParkMainLoop` (keeps `requestAnimationFrame`, drops `emscripten_set_main_loop` entirely) — see [`09`](09-event-loop-deparking-plan.md). Both share the root insight (suspend, don't `throw`); the **root cause above is unchanged**. Note the de-park **regresses the coroutine suite** (Asyncify-nesting wall), fixed by [`../async/12`](../async/12-design-b-asyncify-implementation-plan.md) + [`../async/13`](../async/13-design-b-engineering-spec.md).
Drive the **top-level** event loop via **Asyncify** instead of `set_main_loop`'s
abandon-the-stack `"unwind"` — i.e. the **same mechanism the nested/quasi-modal loops already use**
(`wxWasmRunNestedLoop`, an `EM_ASYNC_JS` that suspends via Asyncify and pumps `ProcessEvents` from a
`setTimeout` loop). Asyncify suspends with a **return-based** unwind that **saves** the stack rather
than abandoning it: no `"unwind"` JS exception is thrown, so no `catch_all` cleanup pad fires, so the
frame survives. `ProcessEvents` is then driven by the JS `setTimeout(17ms)` pump instead of
`requestAnimationFrame`.
`src/wasm/evtloop.cpp`:
- `wxGUIEventLoop::DoRun` — the first (top-level) `DoRun` no longer falls through to
`emscripten_set_main_loop(ProcessEvents, 0, 1)`; it does the initial top-window `SetSize`/`Refresh`
and then calls `wxWasmRunNestedLoop()`, exactly like a nested loop. Both levels now share one path.
- `wxGUIEventLoop::ScheduleExit` — always `wxWasmExitNestedLoop()` (resolve the innermost pump);
dropped the top-level `emscripten_cancel_main_loop()` branch (there is no `set_main_loop` to cancel).
### Why this fix and not the alternatives
- **`simulate_infinite_loop=0`** (don't throw): then `DoRun` *returns*, `OnRun` returns, and `wxEntry`
runs its **normal** teardown (deletes the TLWs) and exits — same dead frame, plus the app exits.
Doesn't help.
- **Suppress/avoid the cleanup pads:** they're compiler-generated; you can't selectively disable the
one that catches `"unwind"`. Not actionable.
- **Asyncify the top loop:** it's the existing, tested suspension primitive in this codebase, it
*saves* the stack (no abandon → no foreign-exception propagation through cleanup pads), and it
unifies top-level and nested loops on one mechanism. This is the minimal, principled change, and it
lives in the wasm port layer (`src/wasm/`), per the "fix in the wasm layer" policy.
## Evidence
- **Before fix:** `~wxNonOwnedWindow cssId=0 uncaught=0` fires right after "before set_main_loop";
`#window-container` empty.
- **After fix:** `~wxNonOwnedWindow` **no longer fires** at startup (definitive C++-level signal the
frame survives); headless load reports `canvas:true, canvasVisible:true, buttons:5`, body text =
"wxDialog and wxMessageBox Test…", `canvases=1`, `traps=0`; rebuilt clean (debug logging removed)
and re-checked → same.
## Open: the empty-screenshot discrepancy
**My headless-Chromium screenshot showed the full dialog; on inspection the screenshot is reported
empty. These disagree and I have not reconciled them.** Until resolved, the app is **not** confirmed
rendering. Candidate explanations, to check in order:
1. **Stale image** — an earlier (pre-fix) empty capture vs. the post-fix one. Cheapest to rule out.
2. **Headless vs. headed / real engine** — my check was headless Chromium; a real/headed browser
(esp. WebKit/Firefox) may differ. The whole point of project policy is **all three engines**;
I only spot-checked one, headless.
3. **Ad-hoc load vs. the real e2e spec** — my load waits a fixed 6 s; the spec has its own
`waitForApp`/timing and asserts against **tracked baseline screenshots**. The spec is the
authoritative render check and I have **not** run it yet.
4. **A separate, still-present rendering issue** — the frame-destruction fix is proven, but a
*different* paint/canvas problem could remain (e.g. the canvas drawing path, or DOM-widget vs.
canvas content). The C++ signal (destructor no longer firing) proves the *frame* lives; it does
**not** prove every pixel paints.
**Immediate next step:** run `tests/.../dialog` through the real e2e spec in **Firefox + Chrome +
Safari (WebKit)** and compare to the baseline screenshots — that reconciles the discrepancy and is
the real Phase-2 acceptance gate.
## Build-system decisions made for Phase 2 (for review)
All gated so the default (JS-EH) build is unchanged; native EH is opt-in via `WX_NATIVE_EH=1`.
- **`scripts/build-wx-wasm.sh`** — `WX_NATIVE_EH=1` swaps `-fexceptions` for
`-fwasm-exceptions -sSUPPORT_LONGJMP=wasm -sWASM_LEGACY_EXCEPTIONS=1` in C/CXXFLAGS. (The whole
libwx + every app must share one EH model — EH ABI is all-or-nothing.)
- **`tests/apps/Makefile.wasm`** — `WX_NATIVE_EH` adds the same EH flags + `-sDYNCALLS=1` to app
CXXFLAGS/LDFLAGS.
- **`scripts/build-wasm-test.sh`** — under `WX_NATIVE_EH`, **stubs the emsdk-bundled `wasm-opt`**
(v121 crashes asyncifying wasm-EH) so the in-link Asyncify no-ops, then post-link runs the real
pipeline on Binaryen v130 over each freshly-linked app + injects the dyncall shims.
- **`scripts/common/hoist-and-asyncify.sh`** (new) — the post-link pipeline:
`--hoist-cpp-catches` (our fork pass) → `--asyncify` → `-O2`, all on v130. `HOIST_KEEP_NAMES=1`
preserves the names section through `-O2` (added for the callstack debugging here).
- **`src/wasm/evtloop.cpp`** — the loop fix above (the only behavioral wx-port change).
## Implications beyond `dialog`
- **Modals now nest Asyncify two levels deep.** Previously: top = `set_main_loop` (no Asyncify
suspend on the main stack), modal = Asyncify (1 level). Now: top = Asyncify, modal = Asyncify
(2 levels). This leans harder on the nested-`currData` save/restore in `handlesleep.js`
([`asyncify-park-throw-root-cause`]). **Must be tested** (open the Custom/Input dialogs).
- **KiCad uses the same `evtloop.cpp`.** If this fix holds for wx apps, it's the same fix KiCad
needs under native EH — and it means the `set_main_loop`-`"unwind"` conflict is a **general**
wasm-EH×wx-DOM-port interaction, not a `dialog`-specific quirk. This is exactly the kind of
"scale hazard" 07 §6 flagged ("unwind-time landing-pad reliability … might *change* under wasm-EH")
— it changed, and here's the concrete consequence + remedy.
## Honest verdict
The deep-debug **succeeded at the hard part**: a silent blank-render is now a fully understood,
evidence-backed root cause with a minimal, principled fix, and the specific bug (frame self-destruct)
is provably gone. But Phase 2's acceptance bar — *the app verifiably renders and is interactive in all
three browsers via the e2e spec* — is **not met yet**, and the empty-screenshot observation is an
unresolved flag against it. Next action is reconciliation via the real spec, not more root-causing.

160
docs/features/wasm-exceptions/09-event-loop-deparking-plan.md Normal file
View file

@ -0,0 +1,160 @@
# 09 — Event-loop de-parking: one EH-agnostic main loop (plan + verification) (2026-06-23)
> Plan to replace the wx top-level event loop's stack-abandoning `throw "unwind"` with an
> **Asyncify de-park**, written **in C++ (`evtloop.cpp`), not as a post-link shim**, so a **single
> code path works under both `-fexceptions` (JS-EH) and `-fwasm-exceptions` (native EH)**.
> Companion to [`08-wx-app-render-rootcause.md`](08-wx-app-render-rootcause.md) (root cause) and to
> the async dossier's de-park analysis (`docs/features/async/`). Ends with the **old-vs-new
> test+screenshot verification matrix**.
## Decision (verdict up front)
The top-level `wxGUIEventLoop::DoRun` will stop using `emscripten_set_main_loop(..., simulate_infinite_loop=1)` (which `throw "unwind"`s to abandon the C++ stack). Instead:
```cpp
// top-level DoRun:
wxWasmParkMainLoop(); // suspend the C++ stack + drive ProcessEvents from an rAF loop
// that calls it via the ASYNC ccall (Asyncify-aware).
// NOT emscripten_set_main_loop — see the Correction note below.
```
- **No `throw`** → nothing for native wasm-EH's `catch_all` cleanup pads to catch → the main frame is not destroyed (the 08 bug).
- **No `throw`** under JS-EH either → behaves exactly like today minus the (JS-EH-harmless) throw.
- **One source, both models** — no `#ifdef WX_NATIVE_EH`, no shim, no `--js-library`. Lives in the wx wasm port where the loop already lives.
- **Keeps `requestAnimationFrame`** — `wxWasmParkMainLoop` drives `ProcessEvents` from an rAF loop (not `setTimeout`), vsync-aligned like the original. (Option A used `setTimeout`; this supersedes it.)
## Background (recap of the 08 bug)
`emscripten_set_main_loop(ProcessEvents, 0, 1)` registers the rAF loop and then `throw "unwind"` to abandon the C++ stack (so the code after it never runs and the browser drives `ProcessEvents`). Under `-fwasm-exceptions`, the compiler emits real `catch_all` cleanup landing pads; as the foreign `"unwind"` JS exception propagates out of `main`, those pads **catch it and run destructors**, tearing down `wxTopLevelWindows.front()` (the main frame) before first paint → blank render. Under JS-EH the same throw is harmless because legacy `-fexceptions` landing pads fire *unreliably* and `noExitRuntime=true` means the throw is swallowed by `handleException` with no destructors run.
## The mechanism in detail
The throw bundles two jobs: (1) register the rAF loop, (2) abandon the stack. `simulate_infinite_loop` is a parameter, so we split them: pass `0` (register + return, no throw), then keep the stack alive ourselves with a **bare park**.
`wxWasmParkMainLoop` is `wxWasmRunNestedLoop` **minus its `setTimeout` pump** — because here `emscripten_set_main_loop`'s rAF already drives `ProcessEvents`, so a pump would double-drive it. It only `await`s a Promise registered on the existing `Module._wxNestedLoopExit` LIFO:
```cpp
EM_ASYNC_JS(void, wxWasmParkMainLoop, (), {
Module._wxNestedLoopExit = Module._wxNestedLoopExit || [];
await new Promise(function (resolve) {
var finish = function () {
var idx = Module._wxNestedLoopExit.indexOf(finish);
if (idx !== -1) Module._wxNestedLoopExit.splice(idx, 1);
resolve();
};
Module._wxNestedLoopExit.push(finish);
// no pump: emscripten_set_main_loop's rAF drives ProcessEvents
});
});
```
`DoRun` becomes:
```cpp
int wxGUIEventLoop::DoRun() {
bool topLevel = (s_wxRunDepth++ == 0);
if (topLevel) {
// initial sizing as today
...SetSize/Refresh on wxTopLevelWindows.front()...
emscripten_set_main_loop(ProcessEvents, 0, 0); // rAF, no throw
wxWasmParkMainLoop(); // suspend until exit
} else {
wxWasmRunNestedLoop(); // nested: unchanged (setTimeout pump)
}
--s_wxRunDepth;
return 0;
}
```
`ScheduleExit` cancels rAF for the top level before resolving (so no stray rAF tick calls `ProcessEvents` on the app being torn down), then resolves the innermost loop:
```cpp
void wxGUIEventLoop::ScheduleExit(int) {
m_shouldExit = true;
if (s_wxRunDepth == 1) emscripten_cancel_main_loop(); // top-level: stop rAF
wxWasmExitNestedLoop(); // resolve park (top) or pump (nested)
}
```
**Nested/modal loops are unchanged** — they still use `wxWasmRunNestedLoop` (rAF isn't available while nested). Only the top level changes, and only from "abandon-via-throw" to "register-rAF + suspend-via-park."
## Correction: rAF pump, not `emscripten_set_main_loop`
The first implementation used `emscripten_set_main_loop(ProcessEvents, 0, 0)` to register rAF + a *bare* park (as "The mechanism in detail" above describes). **It renders white/blank.** `set_main_loop`'s rAF callback (`MainLoop.runIter`) calls `ProcessEvents` **synchronously**, and a synchronous call cannot drive the runtime while `main` is Asyncify-**parked** — the loop stalls after ~6 frames (measured: rafCount **6** vs **~348** for a live loop), so the window never gets its first `Paint` and stays browser-white. (Clicks still work via direct DOM→wx handlers; the first modal's `setTimeout` pump then paints, which masked it in shots 02-05.)
**Fix — the form now in `evtloop.cpp`:** `wxWasmParkMainLoop` is a hand-rolled `requestAnimationFrame` pump that calls `ProcessEvents` via the **async** `ccall(..., {async:true})` (Asyncify-aware → works on the parked runtime). `DoRun` calls only `wxWasmParkMainLoop()` (no `set_main_loop`); `ScheduleExit` is only `wxWasmExitNestedLoop()` (the pump's `finish()` sets `stopped=true`, stopping it before teardown). It differs from `wxWasmRunNestedLoop` only in rAF vs `setTimeout`.
```cpp
EM_ASYNC_JS(void, wxWasmParkMainLoop, (), {
var stopped = false, finish = null;
var pump = function () {
if (stopped) return;
requestAnimationFrame(async function () {
if (stopped) return;
try { await ccall('ProcessEvents', 'void', [], [], { async: true }); }
catch (e) { if (finish) finish(); return; }
if (!stopped) pump();
});
};
Module._wxNestedLoopExit = Module._wxNestedLoopExit || [];
await new Promise(function (resolve) {
finish = function () { stopped = true; /* splice from LIFO */ resolve(); };
Module._wxNestedLoopExit.push(finish);
pump();
});
});
```
Confirmed (native-EH): rafCount 348 (continuous), `dialog-01-loaded` **byte-identical** to baseline, all 5 dialog tests pass incl. modals.
## Relationship to the async dossier
The agent review (`docs/features/async/`) classifies this precisely: it is the dossier's **"de-parking (Option C — park main in an unresolved `EM_ASYNC_JS` sleep)"** (`02-asyncify-internals.md:265`, `07-decisions-and-outcome.md:60`), the "natural step one of Design B." The dossier **deferred/rejected** de-parking (`07/D4`) because, under `-fexceptions`, the throw provably "runs no destructors" (`10-resolution-menubar-uaf.md:63`, `08-dom-port-regression.md:305-317`) — so there was no reason to take it on. **That premise is exactly what `-fwasm-exceptions` inverts** (catch_all pads now *do* run destructors), and the dossier never considered native EH (grep-confirmed: zero mentions). So we are adopting the dossier's own deferred design, now made *necessary* by the toolchain change — consistent with its long-term direction (Design B), against its near-term decision (D4), for a reason D4 didn't know about.
## Teardown on exit — correct, not a bug
The old throw *abandons* the stack, so `wxEntryCleanupReal` (delete app + all TLWs) **never runs** — leaked on exit. The park lets `DoRun` resume on exit and return into that cleanup, which is correct (and frees the leak). No use-after-free: `ScheduleExit` cancels rAF first, and the park's resolver runs before `DoRun` resumes, so nothing calls `ProcessEvents` on freed state. (The dossier's general de-park warning is about the `simulate_infinite_loop=0`-and-*return* form where a still-registered rAF fires on the freed app; our suspend-then-cancel form avoids it.) Residual: a pre-existing wx window-close crash (`async/01:92-94`) could surface only if a real clean exit is triggered — rare in a browser; pre-existing, not introduced here.
## Remaining caveat (the real one to verify)
This makes the top-level an **always-live Asyncify-suspended context** for the app's lifetime. The known nesting wall (`async/11-asyncify-nesting-raytracer.md`: `emscripten_sleep` can't nest on an unwinding context) is the thing to watch — at KiCad scale, especially the 3D viewer. Reasoning suggests it's *not* worsened (the park is a dormant, separate saved stack, not in the active modal→sleep chain; ProcessEvents runs fresh from rAF), but that's analysis, not measurement. Since this is now one path for both EH models, the JS-EH build gets the park too, so the scale check covers both.
## Verification matrix (the old-vs-new test + screenshot proof)
Goal: prove (a) native EH renders correctly, and (b) JS-EH has **no regression**, by comparing the full wx e2e screenshots against the committed baselines (which were generated from the **old JS-EH** build). `scripts/compare-screenshots.sh` does byte-exact `cmp` of `tests/test-results/` vs `tests/baseline-screenshots/`.
| # | Config | EH model | `evtloop.cpp` | Purpose | Expected |
|---|---|---|---|---|---|
| 1 | **OLD** | JS-EH (`-fexceptions`) | original (throw) | reference / baseline-is-current sanity | matches committed baseline |
| 2 | **NEW-native** | native (`-fwasm-exceptions`) | de-park (this plan) | migration target | matches baseline |
| 3 | **NEW-js** | JS-EH (`-fexceptions`) | de-park (this plan) | no-regression | matches baseline |
- **Scope:** the full wx app suite (`menu, clipboard, filedialog, layout, aui, toolbar, grid, dialog, timer, tree`), all their `*-NN-*.png` shots.
- **Pass bar:** configs 2 and 3 produce screenshots **byte-identical** (or trivially-different, e.g. caret-blink) to the baseline, same set config 1 produces.
- **Browsers:** byte-compare is **chromium** (baselines are chromium). Firefox + WebKit are run for **pass/render** confirmation (their pixels won't byte-match a chromium baseline), per the all-three-engines policy.
- **Build order (minimize clean rebuilds; only EH-model switches need `--clean`):** start from current (native-EH + interim option-A) → implement de-park → **(2)** native-EH de-park (incremental) → **(3)** JS-EH de-park (clean EH switch) → **(1)** JS-EH original (revert `evtloop.cpp`, incremental).
### Results (to fill in)
| # | Config | identical / different / fail | notes |
|---|---|---|---|
| 1 | OLD JS-EH | suite: **316 pass / 0 fail** / 1 skip | **Baseline is STALE** — OLD JS-EH itself differs 1–8% from the committed baseline on many main-app shots (03-after-load…aui…calendar…clipboard), so byte-compare *vs baseline* is unreliable; use config-vs-config. **0 failures here ⇒ config 2's 21 ARE regressions** from native-EH and/or de-park (not pre-existing). |
| 2 | NEW native | suite: 295 pass / 21 fail / 1 skip. dialog standalone 3 identical + 2 caret. Apps render (main-app snapshot OK). Fails: main-app assertions (boot/wxwidgets/dialogs-tab/grid-tab, 10) + coroutine/threading/raytracer (11) | old-baseline (config 1) pending to classify the 21 as pre-existing vs regression |
| 3 | NEW JS-EH | suite: **310 pass / 6 fail** / 1 skip — all 6 are **coroutine** (`coroutine`, `coroutine-nested`, `coroutine-pthread`) | de-park renders **identically** to config 1 (config1-vs-config3 byte-diffs = event-log timestamps + caret only; verified pixel-identical on 04-controls-tab) |
### Isolation (the verdict)
| failures | config 1 (no de-park) | config 3 (de-park, JS-EH) | config 2 (de-park, native-EH) | attribution |
|---|---|---|---|---|
| coroutine / coroutine-nested / coroutine-pthread (6) | pass | **FAIL** | FAIL | **the de-park** (top-level Asyncify park × coroutine fibers — the doc-11 nesting wall, now real) |
| coroutine-raytrace (5) + main tabbed app: boot/wxwidgets/dialogs-tab/grid-tab (10) | pass | pass | **FAIL** | **native-EH** (migration coverage gaps; the "scale unproven" caveat) |
**Key conclusions:**
1. The de-park is **visually clean** — no rendering change (config1≡config3 modulo timestamps), standalone apps all pass, dialog byte-identical.
2. **The de-park regresses the 6 coroutine/threading tests under BOTH EH models.** Under JS-EH this is a **net loss** (the original `throw` form passes them) — so "one de-park for both" is not free: it costs JS-EH its coroutines. This is exactly the Asyncify-nesting hazard `async/11` + the async dossier flagged; the dossier's answer is **Design B** (fiber/arbiter runtime), of which de-park is "step one."
3. native-EH independently breaks 15 more (raytracer + the big app) — broader migration work, separate from the loop change.
4. The **committed screenshot baseline is stale** (old JS-EH itself is 1–8% off); and byte-compare is unreliable here anyway (event-log timestamps). A perceptual diff + a baseline refresh are needed for a real screenshot gate.
**Open decision:** (a) gate the de-park to native-EH only (JS-EH keeps `throw` + coroutines; not "one solution", needs `#ifdef`); (b) do Design B so de-park coexists with coroutines (bigger); (c) ship de-park for both now, coroutine-nesting as tracked follow-up.
## Status
Plan agreed. Implementing the `evtloop.cpp` de-park, then running the matrix above. The interim `option-A` edit (top-level via `wxWasmRunNestedLoop`'s `setTimeout` pump — loses rAF) is **superseded** by this and will be reverted in favor of the `set_main_loop(0)` + `wxWasmParkMainLoop` form.

339
docs/features/wasm-exceptions/10-pthreads-native-eh.md Normal file
View file

@ -0,0 +1,339 @@
# Native wasm-EH × pthreads — findings and test coverage
> **Status:** native wasm-EH (`-fwasm-exceptions`) is the **default build**; the pthread test suite is
> green in Firefox + Chrome. Authored 2026-06-24, updated 2026-06-25. The exception-handling-side
> companion to the mechanism-deep [`../threading/README.md`](../threading/README.md) (the 3-layer
> thread model, the deadlock mechanics, the three failure modes). **Scope:** what native wasm-EH does
> to pthreads, and the test coverage that proves which patterns work.
## Why this exists
KiCad-WASM uses native WebAssembly exceptions instead of Emscripten JS exceptions for the bundle-size
win (pcbnew ~64.5 → ~36 MB gz). **Native-EH is the default build** — `build-wx-wasm.sh`,
`build-wasm-test.sh`, and `tests/apps/Makefile.wasm` compile `-fwasm-exceptions -sSUPPORT_LONGJMP=wasm
-sWASM_LEGACY_EXCEPTIONS=1` (single-sourced from `scripts/common/env.sh`). It is the only build mode —
the legacy `-fexceptions` path has been removed. CI builds native. The threading question this doc answers: which pthread patterns work under native-EH, what is
the one exception-related risk it removes, and what is the (optional) upstreamable follow-up.
## TL;DR
- **Native-EH wx suite: green** in Firefox + Chrome (316 / 1 skipped / 0 failed, chromium — matching
JS-EH). WebKit is blocked for *all* pthread apps by a separate, pre-existing COEP worker-load
limitation (§2a), so the pthread specs run FF + Chrome.
- **The one native-EH-relevant risk is mode-c** — a C++ exception thrown on a pthread worker. Under
`-fexceptions` the throw drives Asyncify on the worker and crashes (`"func is not a function"`);
native-EH lowers exceptions to native wasm instructions, so a throwing worker task is safe. Every
threading pattern below is green under native-EH; the throwing ones are green **only** under native-EH.
- **The real `BS::thread_pool` runs 16-core under native-EH**, including a task that throws on a worker
(`threadpool-real`, §6) — the decisive proof that the `detach_task` single-thread shim can be dropped.
- **On-demand (non-warm) Worker creation works without editing KiCad** (`pthread-ondemand`, §6): the
`nanosleep` override (§2b) makes a main-thread `sleep_for` join Asyncify-yield, so the event loop
services the new-Worker handshake. This is the threading-doc **mode-(a) deadlock** cure, in the wasm
layer.
- **A nested `emscripten_sleep` is legal** (§3): code dispatched by a wx modal pump's `ProcessEvents`
runs at Asyncify `state == Normal`, so a worker-join that yields via `emscripten_sleep` inside an
open modal suspends-and-resumes normally (`raytrace-modal`, §6). The threading-doc **mode-(b)** does
**not** arise for this case, so no JS-land scheduler ("Design B") is needed for it.
- **The KiCad-10 `std::async` library preload is safe under native-EH** (`async-preload`, §6/§7): the
worker parses S-expr (a throw = mode-c) and proxies its async fetch to main; native-EH makes the
parse safe and the lazy join keeps main free to service the proxy.
- **The fork stays pristine.** The pool's `detach_task` shim is the original, unmodified KiCad code;
the tests un-shim it via a build-generated header (§6/§D), so the KiCad submodule carries no
wasm-specific change. The later, *optional* upstreamable step is the raw-threads→pool refactor (§4).
---
## 1. Suite status under native-EH
The native-EH wx app suite is **316 / 1 skipped / 0 failed** (chromium), matching JS-EH. Reaching it
required two things: clearing a set of build-pipeline gaps that surfaced as native-EH test failures
(committed this session, summarized below), and the §2 asyncify-imports fix for the raytracer cluster.
The build-pipeline gaps (all committed):
- **Post-link Asyncify find too narrow** — the loop matched only `standalone/*/*_test.wasm`, silently
skipping `apps/minimal_test.wasm` and the coroutine-pthread repros / wxpt. Those linked but were
never asyncify-instrumented → `asyncify_start_unwind not found`. Broadened to all freshly-linked app
wasm.
- **Repro apps mixed EH models** — the coroutine-pthread `*_repro` apps hardcoded JS-EH in their link
recipes while their compile inherited native-EH → `undefined symbol: __cpp_exception`. Made them
EH-aware so they follow the default (native) and only carry `-fexceptions` under `WX_LEGACY_EH`.
- **`build-wasm-test.sh` swallowed make failures** — it continued to the post-link after a failed
make, leaving apps half-instrumented (read as mass test failures). Now aborts loudly.
After those, the only real native-EH-specific signal was the raytracer threading cluster, fixed in §2.
---
## 2. The asyncify-imports fix (`emscripten_sleep`)
`coroutine-raytrace.spec.ts` aborted with `Aborted(invalid state: 1)`. The mechanism:
- `invalid state: 1` is `Asyncify.handleSleep` aborting because the state is **Unwinding** — a second
suspend starting before the first rewinds. Logging every `handleSleep`, the state sequence at the
abort is exactly **`0,1`**: two `emscripten_sleep`s back-to-back with **no rewind between**. So a
function calls `emscripten_sleep`, the unwind arms (state→Unwinding), and the **same function calls
`emscripten_sleep` again before returning**. A correctly Asyncify-instrumented function has a
post-call "if Unwinding, save locals and return" check after every suspend point; this one doesn't →
**Asyncify never instrumented it.**
- `main()` is **not** re-entered (a `[MAINCALL]` probe fired exactly once; the abort stack only *shows*
`main`'s frames because Asyncify's unwind/rewind runs inside a `setTimeout`-driven `doRewind` that
keeps the JS stack live).
- **Why un-instrumented:** binaryen's Asyncify instruments only functions that can reach a *listed*
async import. The post-link list was curated for the wx apps, which yield via **fibers**
(`startModal, js_*, invoke_*, __asyncjs__*, emscripten_fiber_swap`). It **omitted `emscripten_sleep`**,
which the raytracer yields via. `env.emscripten_sleep` *is* a wasm import, so binaryen can match it —
it just wasn't told to.
### The exact JS-EH ↔ native-EH difference
Under **JS-EH**, Asyncify runs **in-link** and emcc **auto-adds** `emscripten_sleep` (+
`idb_*`/`wget`/`scan_registers`/`lazy_load`) to the imports. Under **native-EH** we run Asyncify
**post-link by hand**, with an explicit list that dropped those auto-imports. That is the entire
difference — not a fundamental native-EH × pthread incompatibility, and not a handleSleep-vs-arbiter
question (the `currData` shim was never involved).
### The fix
Added `env.emscripten_sleep` (+ `scan_registers`, `lazy_load_code`, `wget`, `wget_data`, `idb_*`) to
the post-link asyncify-imports — now the shared **`scripts/common/asyncify-imports.txt`**, consumed by
the unified **`apply-asyncify.sh`** that both the wx-test and KiCad builds call (the two near-duplicate
scripts were folded into one; the old `hoist-and-asyncify.sh` is gone). So the KiCad list gets
`emscripten_sleep` too, pre-empting the identical latent bug when its threading is un-shimmed.
| Check | Result |
|---|---|
| Full wx suite, Chromium | **316 / 1 skipped / 0 failed** |
| `coroutine-raytrace.spec.ts` — all 6 (B1/B2/B1-local/B3 + speedup + A neg-control) | **6/6 pass** |
| multi-core speedup | **serial 1342 ms → parallel 142 ms = 9.45× on 16 cores** |
| raytrace `#m=5` default (drains pool → on-demand creation) / `#m=1`, Chromium + Firefox | **SUCCESS, workersRan=16** |
The default `m=5` — which *drains* the pre-warmed pool and forces on-demand Worker creation —
succeeds, so the fix also resolves the threading-doc **mode-(a) deadlock**: the `sleep_for` join now
yields via an instrumented `emscripten_sleep` instead of busy-spinning and starving the worker
handshake.
### 2a. The WebKit issue (separate, pre-existing)
In WebKit the asyncify side runs (threads spawn) but the **pthread worker `.js` load is refused on
COEP** (`Refused to load worker because of Cross-Origin-Embedder-Policy`) even with COOP + COEP + CORP
all served and `crossOriginIsolated:true`. It is a WebKit/playwright-headless COEP-worker strictness
issue affecting **all** pthread apps, unrelated to EH. Tracked separately; the pthread specs run
FF + Chrome only.
### 2b. The `nanosleep` override (the on-demand cure)
`wasm/shims/nanosleep_yield.c` is a **strong `nanosleep` definition** that shadows musl's archive
member (`-Wl,--wrap=nanosleep` is not usable — it segfaults wasm-ld in
`lld::wasm::ImportSection::addImport`). On the **main thread** it yields via an `EM_ASYNC_JS` await
(= `emscripten_sleep` semantics, already in the post-link asyncify-imports); on a **worker** it stays a
real blocking `emscripten_thread_sleep`. So an *unmodified* KiCad `sleep_for` join on the main thread
pumps the event loop instead of busy-spinning, which lets on-demand Worker creation complete with no
KiCad edit (§6 `pthread-ondemand`).
---
## 3. Patterns that work — pool-vs-raw, and the nested-sleep case
Two earlier wx apps, plus the §6 additions:
| App / test | Thread pattern | native-EH |
|---|---|---|
| `threadpool_test.cpp` (`threadpool.spec.ts`) | create `hwc` `std::thread`s into the **pre-warmed** pool, short body, **`join()`** each | **PASS** |
| `raytrace_threads_test.cpp` (`coroutine-raytrace.spec.ts`) | raw detached/persistent `std::thread`, sleep/busy-wait join, default **drains** the pool → on-demand creation | **PASS** (after §2) |
Both raw and pool patterns work under native-EH; raw threads are **not** fundamentally broken. The
`threadpool` create-and-`join()` never calls `emscripten_sleep`, so it never tripped the missing
import; the raytracer yields via `emscripten_sleep`, so it did — which §2 closed.
**The nested-sleep case (mode-b is not a live blocker).** A worker-join that yields via
`emscripten_sleep` *inside an open `ShowModal` dialog* is legal. The modal pump runs `ProcessEvents`
via `ccall(async:true)`, so work it dispatches runs in a **fresh managed Asyncify entry at
`state == Normal`** — not nested inside an already-Unwinding frame. So the inner `emscripten_sleep`
suspends-and-resumes normally. `raytrace-modal` (§6) probes and logs `Asyncify.state == 0` to confirm
this; the threading-doc mode-(b) "Asyncify can't nest" only bites a *genuine* second unwind, which the
modal pump does not produce. **No JS-land scheduler ("Design B") is required for the "render inside a
modal" case.**
---
## 4. Optional follow-up (upstreamable): KiCad raw-threads → the pool
This is a *later, optional* cleanup — not required, since the wasm layer (native-EH + the nanosleep
override) already makes the threading patterns work on pristine KiCad. Per
[`../threading` §8](../threading/README.md), upstream KiCad has migrated only **1 of 7** raytracer
parallel sections to `GetKiCadThreadPool()` (`renderTracing`, and that one accidentally); the other six
are untouched **2018 OpenMP-translation** raw-thread code:
| Site | Pass | Today |
|---|---|---|
| `render_3d_raytrace_base.cpp:764` `shadeWorker` | post-process shading | raw `std::thread` + busy-wait, `#ifdef`'d serial in WASM |
| `render_3d_raytrace_base.cpp:835` `blurWorker` | blur/finish | same |
| `render_3d_raytrace_base.cpp:1456` `previewWorker` | preview | same |
| `image.cpp:525` `filterWorker` | `EfxFilter` AA/blur | same |
| `create_layer_items.cpp:848` `zoneWorker` | zone-fill geometry | same |
| `create_layer_items.cpp:1311` `simplifyWorker` | polygon simplify | same |
**The refactor = migrate these six to `submit_task()` + `multi_future::wait()`** (the `renderTracing`
shape, refined by upstream `bccf36538` to wait on *own* tasks only), and delete the
`#ifdef __EMSCRIPTEN__` serial fallbacks. Why it is the right *eventual* move:
- **Upstreamable, not a wasm hack** — precedent in the same file, a filed upstream issue
([GitLab #20911](https://gitlab.com/kicad/code/kicad/-/issues/20911), "ray tracing high system
load"), and it removes dead OpenMP-era code. If accepted upstream, our fork carries **zero**
divergence here.
- **Less divergence, not more** — it lets us drop the raytracer `#ifdef`s; combined with native-EH
letting us drop the `detach_task` shim, net fork divergence goes *down* while threads come *on*.
Its prerequisite — that the real `BS::thread_pool` (persistent workers + `submit_task` +
`multi_future::wait()`) survives native-EH — **is proven** by §6 `threadpool-real` (16-core, including
a throwing worker task). So the refactor is de-risked; it is scheduled **after** the EH port's suite is
otherwise green, and remains optional because the wasm-layer fixes already deliver multi-core.
---
## 5. The 3D viewer
The 3D viewer is **live and single-threaded**: the raytracer's six raw-thread passes are `#ifdef`'d to
serial fallbacks in WASM, which is what ships. A separate multi-threaded spike exists (the
`WASM_RAYTRACE_POOL` work, ~6–7×) but is not the active path.
Two zero-KiCad-edit routes turn the live viewer multi-threaded:
- **The nanosleep override (§2b)** makes the existing `sleep_for` joins yield, so the raw-thread passes
run multi-core without on-demand-creation deadlock and without main-thread jank — no KiCad change.
- **The §4 pool refactor** is the *upstream-clean* alternative: pool-based, drops the `#ifdef`s, and
carries zero fork divergence if accepted upstream.
---
## 6. pthread test coverage
All four apps below compile the **real KiCad** thread-pool source and run on **pristine** KiCad/wx-core.
The specs are named `coroutine-*` so `playwright-coroutine.config.ts` runs them in Firefox + Chrome
(WebKit excluded — §2a).
| Spec | App | What it proves | native-EH |
|---|---|---|---|
| `coroutine-threadpool-real.spec.ts` | `threadpool-real` | the **real `GetKiCadThreadPool()`** in every mode — submit / loop / blocks / detach / fanout / lifecycle, and a task that **throws** on a worker — 16-core, throw caught | **PASS** (throw mode green *only* under native-EH = mode-c) |
| `coroutine-pthread-ondemand.spec.ts` | `pthread-ondemand` | real pool drains the pre-warmed Workers, then raw fly-threads force **on-demand** creation; the nanosleep override yields the join → on-demand Workers boot → multi-core (control: a non-yielding busy-wait deadlocks) | **PASS** |
| `coroutine-raytrace-modal.spec.ts` | `raytrace-modal` | a worker-join run **inside an open `ShowModal`** — both a busy-wait join and an `emscripten_sleep` yield-join complete multi-core; the app probes `Asyncify.state == 0` to show the modal pump dispatches at Normal | **PASS** (mode-b does not arise) |
| `coroutine-async-preload.spec.ts` | `async-preload` | the KiCad-10 `std::async` library-preload shape: a worker parses S-expr (throws = mode-c) and proxies its fetch to main via `emscripten_proxy_sync_with_ctx`; modes simple / throw / shutdown / modal-during-preload | **PASS** (mode-c safe; 36 proxy round-trips through a modal, no crash) |
These also cover the older `coroutine-pthread.spec.ts` (fiber + pthread across activation paths) and
`threadpool.spec.ts` (raw create+join), both green. Together they exercise: the real pool API, raw
create+join, raw detached/persistent + sleep/busy-wait, on-demand creation, a worker-side throw, a
nested yield inside a modal, and a proxied async fetch off a worker — the full set of shapes the KiCad
threading uses.
### D. How the tests un-shim the pool without editing KiCad
KiCad's `bs_thread_pool.hpp` keeps its original `#ifdef __EMSCRIPTEN__` `detach_task` shim (which runs
pool tasks inline → single-threaded). To exercise the *real* pool, the test build **generates** an
un-shimmed copy: `tests/apps/Makefile.wasm`'s `POOL_UNSHIMMED` rule `sed`s `#ifdef __EMSCRIPTEN__` →
`#if 0` into `standalone/_pool_unshimmed/bs_thread_pool.hpp` (gitignored) and `-I`'s it ahead of the
KiCad header. So the KiCad submodule stays pristine; only the test compile sees the un-shimmed pool.
---
## 7. Library preload: `std::async` + the PCBJAM proxy under native-EH (the KiCad-10 bump)
> The path that turns the native-EH migration from a size win into a **prerequisite for tracking
> upstream**. Verified by `async-preload` (§6): works under native-EH.
**What changed upstream.** KiCad 10 (`d8ae50a667`, 2026-06-08, fixes GitLab #23872) (a) added an
*eager* library preload on board open — `if( Kiface().IsSingle() ) Kiface().PreloadLibraries()` in
`pcbnew/files.cpp` (`OpenProjectFiles`), and `IsSingle()` is exactly our standalone-webapp case; and
(b) changed `IFACE::PreloadLibraries`'s dispatch from `tp.submit_task( preload )` (our base,
`pcbnew/pcbnew.cpp:666`) to `std::async( std::launch::async, preload )` (KiCad-10 `pcbnew.cpp` ~1121).
**`std::async` spawns a real pthread worker that the `detach_task` pool shim does not cover** — the
shim only neutralizes the *pool*.
**The plugin gotcha (don't be fooled by the upstream loader).** Upstream library reads are synchronous
(`KICAD_SEXPR` plugin → `fopen`/`FILE_LINE_READER`). **Our fork is not on that path.** The webapp
writes the lib-table rows as `(type "PCBJAM")` / `(type "PCBJAM_FP")`
(`web/standalone/src/wasm/libs/source.ts:124,143`), so the runtime plugin is our **custom async bridge**
(`kicad/eeschema/sch_io/pcbjam_lib/sch_io_pcbjam_lib.cpp`, `kicad/pcbnew/pcb_io/pcbjam_fp/pcb_io_pcbjam_fp.cpp`).
A surface read of the upstream loader will wrongly conclude "pure sync, safe" — **verify by the
lib-table row `type`, not the generic plugin.** The PCBJAM dispatch is dual-path
(`sch_io_pcbjam_lib.cpp:158`):
```cpp
if( emscripten_is_main_runtime_thread() )
return pcbjam_libs_request_js(...); // main: EM_ASYNC_JS → Asyncify suspend (works on main)
std::lock_guard lk( g_pcbjamProxyMutex ); // worker: serialize, then
emscripten_proxy_sync_with_ctx( queue, main, … ); // proxy the fetch to MAIN + futex-block the worker
```
**The two-level architecture (and how the pool shim warps it).**
- **Outer:** `std::async(preload)` = one real background worker running a watchdog loop
(`sleep_for(150ms)` + poll `AsyncLoadProgress()`). Bypasses the shim.
- **Inner:** `adapter->AsyncLoad()` (`FOOTPRINT_LIBRARY_ADAPTER`) `submit_task`s N enumerate jobs to
the pool → caught by the shim → run inline → so in our fork they execute *serially on the outer
worker*. (Inner parallelism returns once the shim is dropped — also native-EH-gated.)
**What runs where, on the `std::async` worker:**
| Step | Suspends Asyncify on the worker? |
|---|---|
| `sleep_for(150ms)` watchdog | **No** — real worker sleep (`nanosleep`/Atomics.wait), not `emscripten_sleep`. |
| PCBJAM fetch of library bytes | **No** — proxied to main + futex-block; the `EM_ASYNC_JS` runs on *main*. |
| S-expr **parse** of the bytes (throws `IO_ERROR`) | **Yes under `-fexceptions`** → mode-c crash. **No under native-EH.** |
| modals / clipboard / fonts | Not reachable from non-UI parsing. |
**The join is lazy — which defuses the deadlock.** There is **no eager `.get()`**: `CancelPreload(true)`
calls `m_libraryPreloadReturn.wait()` but has **no callers**; `ProjectChanged()` only sets the abort
flag; the `std::async` future's **blocking destructor** fires only on **IFACE teardown** (shutdown,
main thread); and re-entry is guarded by `m_libraryPreloadInProgress` (so the future is never
*reassigned* mid-flight). So in normal operation **main never blocks on the preload future** → it stays
in its event loop → it services the PCBJAM proxy queue → the worker's fetches complete. No
normal-operation deadlock.
**Verified.** `async-preload` (§6) runs this shape under native-EH: the worker parse throws and is
caught (no mode-c crash), the proxy round-trips, and a modal opened during preload survives 36 proxy
round-trips with no crash (the `g_pcbjamProxyMutex` / "table index out of bounds" reentrancy hazard
does not fire). So the **KiCad-10 bump can keep `std::async` as-is under native-EH** — it does **not**
need a fork patch reverting to `tp.submit_task` (which would make preload block board-open on the main
thread). Residual: a real-shutdown ordering check (the blocking destructor while a load is in flight)
is covered by the `async-preload` shutdown mode but not yet under a live IFACE teardown.
**Contrast with the raytracer (§4):** the raytracer's raw threads are legacy OpenMP-era and
upstreamable to the pool. This `std::async` is a **deliberate** upstream choice (a dedicated preload
thread, off the compute pool), so "upstream it to the pool" is **not** the play — native-EH is.
---
## 8. Next steps (ordered)
1. **DONE — `coroutine-raytrace` root-caused and fixed (§2):** the post-link asyncify-imports list
omitted `emscripten_sleep`. Suite 316/0; raytracer multi-core (9.45×).
2. **DONE — pthread coverage closed (§6):** the real-pool, on-demand, modal-nested, and `std::async`
library-preload shapes are all green under native-EH on pristine KiCad/wx-core.
3. **Drop the `detach_task` shim for real** — `threadpool-real` proves the pool survives native-EH, so
the next concrete step is enabling the un-shimmed pool in a KiCad build (DRC / zone-fill /
connectivity on real Workers) and validating the docker build (the shared asyncify-imports change is
untested there).
4. **Resolve the §2a WebKit COEP worker-load limitation** for pthread apps (currently the reason the
pthread specs skip WebKit).
5. **Optional, later — the §4 refactor:** migrate the six raw-thread raytracer sections to the pool,
delete the `#ifdef`s, upstream it.
6. **Track-only:** `PROXY_TO_PTHREAD` (DOM-bound GUI can't leave the main thread) and JSPI
(incompatible with our main-loop architecture) — see [`../threading` §6–7](../threading/README.md).
## Cross-references
- [`../threading/README.md`](../threading/README.md) — the 3-layer model, deadlock mechanics, three
failure modes, the full raw-thread inventory, and the upstream pool-migration analysis.
- [`../async/11-asyncify-nesting-raytracer.md`](../async/11-asyncify-nesting-raytracer.md),
[`../async/12`](../async/12-design-b-asyncify-implementation-plan.md),
[`../async/13`](../async/13-design-b-engineering-spec.md) — Asyncify nesting + the Design B scheduler
(not required for the modal-pump case, §3).
- Apps + specs: `tests/apps/standalone/{threadpool-real,pthread-ondemand,raytrace-modal,async-preload,coroutine-pthread,threadpool,raytrace-threads}/`,
`tests/e2e/coroutine-{threadpool-real,pthread-ondemand,raytrace-modal,async-preload,pthread,raytrace}.spec.ts`,
`tests/e2e/threadpool.spec.ts`; the pool un-shim in `tests/apps/Makefile.wasm` (`POOL_UNSHIMMED`),
the on-demand cure in `wasm/shims/nanosleep_yield.c`.
- **Library preload (§7):** `kicad/pcbnew/pcbnew.cpp:593` (`PreloadLibraries`),
`kicad/eeschema/sch_io/pcbjam_lib/sch_io_pcbjam_lib.cpp` +
`kicad/pcbnew/pcb_io/pcbjam_fp/pcb_io_pcbjam_fp.cpp` (the async PCBJAM IO plugins),
`web/standalone/src/wasm/libs/source.ts` (lib-table rows typed `PCBJAM`/`PCBJAM_FP`); upstream
KiCad-10 `std::async` change `d8ae50a667` (GitLab #23872).

438
docs/features/wasm-exceptions/11-collab-apply-native-eh.md Normal file
View file

@ -0,0 +1,438 @@
# Collaborative editing under native wasm-EH — the virtual-call mis-dispatch: root cause & fix
> **Finalization note:** the `futex_yield.c` shim and the `vcall_*` / `pool-callafter` investigation
> repros referenced below were **removed** during feature finalization — native-EH needs none of them.
> This doc is retained as the root-cause record.
> **Status: RESOLVED 2026-06-28.** Fixed with a one-line build flag.
> **One-line:** native-EH pcbnew's collab **apply** hung at virtual method calls because the **embind
> translation unit was compiled without `-DDEBUG` while the core TU had it**. A `#if defined(DEBUG)`
> virtual (`EDA_ITEM::Show`) takes a vtable slot, so the two TUs' vtable layouts differed by one slot;
> every embind virtual call past that slot read the wrong slot and `call_indirect`-trapped on a
> signature mismatch — swallowed by the apply coroutine's `catch_all` → silent loop = "hang." **Fix:**
> define `DEBUG` for the embind TU in Debug builds (`scripts/kicad/build-kicad-target.sh`). No
> devirtualization; the A/B decision in Part 5 is moot. See [Resolution](#resolution--root-cause--fix).
>
> *Parts 1–5 below are the investigation as it unfolded; it concluded the `vii` correlation was a
> "confound" and weighed an A/B decision. That was right that the dispatch **mechanism** wasn't broken
> — but it stopped one step short of the dispatch **input**: the vtable **slot offset** the embind
> computed was wrong. A runtime vtable probe + a named-binary offset check (the user's "check the
> offsets, find the wrong one") closed it. Kept as the reasoning trail; the Resolution is the answer.*
>
> Companion to [`01-background-two-eh-models.md`](01-background-two-eh-models.md) (EH models),
> [`10-pthreads-native-eh.md`](10-pthreads-native-eh.md) (pthreads), and the `currData` dossier in
> [`docs/features/async/`](../async/).
---
## Resolution — root cause & fix
**Root cause (verified 2026-06-28).** The embind TU (`wasm/bindings/pcbnew_embind.cpp`, compiled
*outside* CMake in `build-kicad-target.sh` step 7) and the core/vtable-emitting TU disagreed on the
`PCB_TRACK` vtable layout by exactly one slot:
- `EDA_ITEM::Show(int, std::ostream&)` is declared `#if defined(DEBUG)` (`kicad/include/eda_item.h:471`).
- The **core** is built Debug → CMake `add_compile_definitions($<$<CONFIG:Debug>:DEBUG>)`
(`kicad/CMakeLists.txt:351`) defines `DEBUG` → `Show` occupies vtable **slot 35** → `PCB_TRACK::SetWidth`
lands at **byte offset 320**.
- The **embind TU** was compiled **without `-DDEBUG`** → no `Show` slot → it computed `SetWidth` at
**offset 316**, which in the emitted vtable is `BOARD_CONNECTED_ITEM::GetEffectiveNetClass()` (wasm
type `ii`). Dispatching it as `vii` = a **`call_indirect` signature-mismatch trap**, swallowed by the
apply COROUTINE's `catch_all` → silent retry loop = the observed "hang."
- Asyncify state at the park was **Normal** (confirmed via the real `asyncify_get_state()` export) — a
trap, never a suspend. Every embind virtual call past slot 35 (`SetWidth`/`GetPosition`/the rebaseline
snapshot getters) mis-dispatched; `Type`/`GetClass` (slots < 35) and core-TU calls
(`commit.Modify`'s `Clone`) worked — which is what made it *look* signature-specific (`vii` fails,
`ii` works).
**Fix.** `scripts/kicad/build-kicad-target.sh`: `EMBIND_CONFIG_DEFINES="-DDEBUG"` in the Debug branch
(empty in Release — Release defines no `DEBUG` in either TU, so the layouts already match), added to the
embind `em++` compile. Both TUs now agree on the layout; **all** embind virtual calls dispatch
correctly. The per-site devirtualizations tried during the hunt were reverted (unnecessary). *Hygiene
follow-up (recommended, not yet applied):* also give the embind TU `-DKICAD_USE_PLATFORM_WASM=1` and the
`-include char_traits_uint16_workaround.h` force-include, so its preprocessor/ABI environment matches
the core's exactly and this class of skew can't recur.
**How it was found.** A runtime probe (a fresh `PCB_TRACK` constructed at the park) showed
`tr.vtbl == fresh.vtbl` — the vtable *pointer* was correct — yet the fresh object hung identically,
ruling out a dead/wrong instance and pointing at the *slot offset*. `wasm-dis` of the named debug
binary then showed offset 316 holds `GetEffectiveNetClass`, not `SetWidth`. Exactly the user's
instruction: *"check the offsets of all instances, find the wrong one."*
**Independently-real fix kept** (not caused by the skew): the **COROUTINE** in `kicadCollabApply`
(the `commit.Modify`→`Clone` trap needs the fiber) and `-Xclang -fno-pch-timestamp`.
**`wasm/shims/futex_yield.c` — NOT needed; kept-but-not-compiled.** It was added during the hunt under
the (mistaken) theory that the apply hung on the thread-pool futex; the real cause was the vtable skew.
A **no-futex build passes collab 8/8** (Firefox + Chromium) — the connectivity recompute is bounded by
the pre-warmed pthread pool (`PTHREAD_POOL_SIZE = hardwareConcurrency`), so it never needs an on-demand
Worker, so there's no main-thread futex deadlock to fix here. The shim file + the `pool-callafter` repro
are kept as a *documented, validated* fix for the on-demand-Worker futex deadlock **if it ever surfaces**
(heavy board / cold pool hanging at `commit.Push`'s `RecalculateRatsnest`); the re-enable steps live in
`scripts/kicad/build-kicad-target.sh` (the "AVAILABLE BUT NOT COMPILED" block) and the shim's header.
---
## 0. How to read this document
This is written to be understood without prior knowledge of WebAssembly internals, C++ dynamic
dispatch, Asyncify, fibers, or futexes. **Part 1** explains every concept from scratch. **Part 2**
walks the actual call chains. **Part 3** lists the four bugs we found and the fixes that work.
**Part 4** is the investigation that proved the headline bug is a confound. **Part 5** is the A/B
decision and the concrete effort estimate for A. Skim the TL;DR, then dive into whatever you want.
### TL;DR
- KiCad's collaborative editing broadcasts each local edit to peers; a peer **applies** the change
by running the same `BOARD_COMMIT` machinery a native edit uses.
- Under **native wasm-EH** (our target), that apply **hangs** at a C++ **virtual** method call
(`SetWidth`, `GetPosition`, …). The same calls work fine under the old **legacy JS-EH** build.
- We found and fixed three real sub-bugs (a dispatch *trap*, a thread-pool *futex deadlock*, a fiber
*finalization* hang). The fourth — the virtual-call hang — looked signature-specific (calls whose
wasm signature is `vii` hang; `ii`/`viii` don't), but **five controlled repros prove `vii` dispatch
is not actually broken**. It only hangs inside the real, huge module.
- **Devirtualizing** the call (telling the compiler the exact function so it emits a direct call
instead of an indirect one) makes the hang vanish at that site — but that's treating a symptom, and
it has to be repeated at ~10–15 call sites and re-done whenever new code adds a by-value getter.
---
# Part 1 — The concepts
## 1.1 Two ways C++ exceptions become WebAssembly
WebAssembly can't just "throw" like native code. Emscripten offers two lowerings:
- **Legacy JS exceptions (`-fexceptions`)** — every call that might throw is wrapped in a JavaScript
helper called `invoke_<sig>` that does a JS `try/catch` around a `dynCall_<sig>` into wasm. The
control flow for exceptions detours *through JavaScript*. Big and slow, but battle-tested.
- **Native wasm exceptions (`-fwasm-exceptions`)** — uses the WebAssembly exception-handling
instructions (`try`/`catch`/`throw`) directly in wasm. No `invoke_*`, no JS detour. Smaller and
faster — this is what we're migrating to. (See [`01-background-two-eh-models.md`](01-background-two-eh-models.md).)
This distinction matters later: under native-EH there are **no `invoke_*`/`dynCall_*` wrappers around
ordinary calls**, so the JS-side "self-heal" tricks that exist for legacy-EH don't apply.
## 1.2 Virtual method calls, vtables, and `call_indirect`
When you write `item->GetPosition()` and `GetPosition()` is declared `virtual`, the compiler does
**not** know which function to run. A `PCB_TRACK` returns its start point; a `PCB_VIA` returns its
centre; a `FOOTPRINT` returns its origin. The decision is made **at runtime** based on the object's
real type. This is **dynamic dispatch**, and it works via a **vtable**:
```
A PCB_TRACK object in memory The PCB_TRACK vtable (one per class)
┌───────────────────────────┐ ┌────────────────────────────────────┐
│ vptr ───────────────────────────► │ slot 0: &PCB_TRACK::Type │
│ m_Start = (10, 20) │ │ slot 1: &PCB_TRACK::GetPosition │
│ m_End = (50, 20) │ │ slot 2: &PCB_TRACK::SetWidth │
│ m_width = 200000 │ │ ... │
└───────────────────────────┘ └────────────────────────────────────┘
```
Every object of a polymorphic class starts with a hidden pointer (`vptr`) to its class's vtable. A
virtual call compiles to: *load the vptr → load the function pointer in the right slot → call it.*
In WebAssembly there are no raw function pointers; instead there is a single **function table** (an
array of functions) and a `call_indirect N` instruction that means "call the function at table index
N." So the C++ virtual call becomes, in wasm:
```wat
local.get $item ;; the object pointer
i32.load offset=0 ;; load vptr (the vtable address)
i32.load offset=8 ;; load the function index from the SetWidth slot
;; ... push the call args ...
call_indirect (type $vii) ;; call the function at that table index, expecting signature "vii"
```
`(type $vii)` is the *expected signature* baked into the instruction. **Signature notation:** the
first letter is the return type, the rest are arguments. `i`=i32, `v`=void.
- `ii` = `(i32) -> i32` — e.g. `KICAD_T Type()` (takes the hidden `this`, returns an int-like value).
- `vii` = `(i32, i32) -> void` — e.g. `void SetWidth(int)` (`this`, the int; returns nothing).
- `viii` = `(i32, i32, i32) -> void` — e.g. `view->Update(item, flags)`.
**A subtlety that matters here — struct returns (the "sret" ABI).** A method that *returns a small
struct by value*, like `VECTOR2I GetPosition()`, can't return two ints in one wasm value. The
compiler rewrites it so the caller passes a hidden pointer to a return slot, and the function writes
through it and returns nothing: `void GetPosition(this, VECTOR2I* out)`. That's also signature `vii`.
So **all of `GetPosition`, `GetClass`, `GetText`, `GetTextSize` (struct/string returns) AND
`SetWidth`, `SetPosition` (void setters) are `vii`** — which is why the bug *looked* signature-specific.
## 1.3 Devirtualization (the fix technique, and its trade-off)
If you class-qualify the call — `static_cast<PCB_TRACK*>(item)->PCB_TRACK::GetPosition()` — you tell
the compiler *exactly* which function to run. It no longer needs the vtable; it emits a plain `call`
(a direct call to a known function), not a `call_indirect`. We already do this elsewhere: see
`itemLayer()` / `itemPosition()` / `itemClass()` in `wasm/bindings/pcbnew_embind.cpp`.
**Why it helps the bug:** the hang fires *at the instrumented `call_indirect`*. A direct `call`
isn't wrapped the same way, so the symptom doesn't appear there.
**The trade-off / why it's not free:** class-qualifying picks *one* class's version. For a generic
getter like `GetPosition` whose answer depends on the type, you must dispatch on the type yourself
(`switch (item->Type()) { case PCB_TRACE_T: ...; case PCB_VIA_T: ...; }`) and class-qualify each arm
— otherwise you call the wrong override and get wrong geometry. That's why devirtualization here is a
**helper per generic getter** plus a hand edit per setter, and why it's "finite but spread."
## 1.4 Asyncify — making synchronous C++ pause and resume
KiCad's C++ is written **synchronously**: it calls `sleep`, it *blocks* waiting for worker threads,
it pops up a modal dialog and waits for the user. In a browser you **cannot block the main thread** —
if you do, the page freezes (no rendering, no input, no timers). The reconciliation is **Asyncify**, a
Binaryen transform that rewrites the wasm so a deep synchronous call stack can be **suspended**
(unwound back to the JS event loop) and later **rewound** (rebuilt exactly where it left off).
Mechanically, Asyncify instruments functions so that:
- on **unwind**, each function saves its locals + a "where was I" call-index into a memory buffer and
returns up the stack until it reaches the event loop; the buffer pointer is `Asyncify.currData`.
- on **rewind**, each function restores its locals and jumps back to the saved call-index, rebuilding
the stack until execution resumes at the suspend point.
There is **one** `currData` slot at a time. If a second suspend starts while the first's buffer is
still occupied, things collide — that's the "nested currData contention" family
([`docs/features/async/`](../async/)). The three states are **Normal (0)**, **Unwinding (1)**,
**Rewinding (2)** (`Asyncify.state` / the wasm export `asyncify_get_state()`).
Why we can't avoid Asyncify: native edits already rely on it (tool interactions suspend mid-drag, the
event loop yields each frame). The collab apply runs the same `BOARD_COMMIT` code, so it inherits the
same instrumentation.
## 1.5 Coroutines and libcontext fibers (KiCad's `COROUTINE`)
A **fiber** is a *second call stack* you can switch to and from cooperatively (no OS thread). KiCad
ships its own `COROUTINE` (built on `libcontext`'s `jump_fcontext`) and runs **tool interactions** on
a fiber so a tool can "yield" in the middle of an operation and be resumed later. The fiber has its
own stack memory; switching is just swapping the stack pointer.
Why it's in the collab apply: KiCad-WASM has a long-standing rule that the heavy edit machinery
(`BOARD_COMMIT::Modify` → `item->Clone()`, the GAL `view->Add`) **only dispatches correctly when run
on the tool-coroutine fiber stack** — running it from a bare `CallAfter`/`ccall` trapped with
"indirect call signature mismatch." So `kicadCollabApply` wraps `doApply` in a `COROUTINE`. (This is
fix #1 below; it's real and necessary.)
## 1.6 Futexes — and why they deadlock on the browser main thread
A **futex** ("fast userspace mutex") is the low-level OS primitive a thread uses to **wait until
another thread signals it**. `std::mutex`, `std::condition_variable`, and `std::future::get()` are all
built on it. The pattern: thread A wants thread B's result, so A does `futex_wait(addr, val)` — "sleep
until the value at `addr` changes" — and B does `futex_wake(addr)` when it's done.
KiCad's **connectivity recompute** (rebuilding the ratsnest/net graph after an edit, in
`commit.Push`) is parallelised across a thread pool. The main thread submits work and then
`futex_wait`s for the workers' results (`std::future::wait_for` → `pthread_cond_wait` →
`emscripten_futex_wait`).
**The browser problem:** on the **main browser thread**, `Atomics.wait` (the real blocking wait) is
*forbidden* — blocking it would freeze the page. Emscripten's fallback is to **busy-spin** in
`futex_wait_main_browser_thread()`, calling `_emscripten_yield()` — but that only services the
internal proxy queue, it **never returns to the JS event loop**. So if the worker the main thread is
waiting on still needs the event loop to run (e.g. an **on-demand Web Worker** has to finish its
`loaded → run` handshake), it never gets to — and the busy-spin spins forever. **Deadlock.**
```
main thread: submit work ──► futex_wait(result) ──► busy-spin _emscripten_yield() ──► (spins forever)
│ never pumps the JS event loop
worker boot: 'loaded' ─X─► 'run' (needs the event loop, which never runs) ──► never produces result
```
**Our solution — `wasm/shims/futex_yield.c`** (fix #2 below): a *strong override* of
`emscripten_futex_wait` that, **on the main thread only**, polls the futex word and between polls does
an **Asyncify yield** (`await setTimeout(0)`) instead of busy-spinning. Yielding pumps the JS event
loop, so the on-demand Worker boots, finishes, wakes the futex, and the wait returns. (On worker
threads it keeps the real blocking `memory.atomic.wait32`.) It's the sibling of the existing
`nanosleep_yield.c`, which covers the `sleep_for`/`nanosleep` path but not the futex path.
## 1.7 The function table and `-sDYNCALLS=1` (one caveat to retire a red herring)
There is a single wasm function table; `call_indirect` indexes it; the type section lists the
distinct signatures (`ii`, `vii`, `viii`, …). With `-sDYNCALLS=1` emscripten also exports per-signature
`dynCall_<sig>` trampolines for JS↔wasm calls. A known hazard (`dyncall-binding.js.tmpl`) is that
*post-asyncify+O2 a `dynCall_<sig>` JS trampoline can carry a stale expected type* — but that is a
**legacy-EH** mechanism (the `invoke_*`→`dynCall` path). Under native-EH a C++ virtual call is a
**raw `call_indirect`**, not a `dynCall`, so that hazard does not apply. (Verified by disassembly — see
Part 4.)
---
# Part 2 — The collab apply call chains
A peer edit arrives as JSON and is applied like this:
```
JS: window.Module.kicadCollabApply(jsonDelta)
└─ kicadCollabApply(std::string) [pcbnew_embind.cpp]
└─ parse JSON → fr->CallAfter([...]) (defer to the wx main-loop drain)
└─ COROUTINE cor([]{ doApply(fr, delta); }) (run on a libcontext fiber — §1.5)
└─ cor.Call(0)
└─ doApply(frame, delta) [pcbnew_embind.cpp]
├─ for removed: commit.Remove(item)
├─ for changed: commit.Modify(item) ──► item->Clone() (virtual, fix #1)
│ applyChanged(item, j)
│ └─ tr->SetStart/SetEnd (non-virtual, fine)
│ └─ tr->SetWidth(w) ◄── VIRTUAL "vii" ★ THE HANG
├─ for added: makeItem(...) → commit.Add(item)
└─ commit.Push("Collaborative edit") [board_commit.cpp]
└─ connectivity->RecalculateRatsnest(...)
└─ thread-pool results.get() ──► emscripten_futex_wait (fix #2)
└─ (back on the main stack, after cor.Call returns)
└─ rebaseline() (fix #3 — moved out of the fiber)
└─ snapshotByUuid(board)
└─ for each item: itemToJson(item)
└─ itemPosition/itemClass/GetText… ◄── VIRTUAL "vii" ★ more of THE HANG
```
Three things in this chain are independently load-bearing, and each was a distinct bug:
1. `item->Clone()` (inside `commit.Modify`) and `view->Add` (inside `commit.Push`) **must** run on the
fiber or they trap → **fix #1 (COROUTINE)**.
2. `commit.Push`'s connectivity recompute **futex-deadlocks** on the main thread → **fix #2
(`futex_yield.c`)**.
3. `rebaseline()` (the post-apply snapshot) at the end of `doApply` **must** run after the fiber
finalizes, not inside it → **fix #3 (move to main stack)**.
And then the headline: the `vii` virtual calls (`SetWidth`, the snapshot getters) **hang**.
---
# Part 3 — The four bugs and the fixes that work
| # | Bug | Symptom | Fix | Status |
|---|-----|---------|-----|--------|
| 1 | `Clone`/`view->Add` dispatch off the fiber | "indirect call signature mismatch" **trap** | run `doApply` in a `COROUTINE` (`kicadCollabApply`) | ✅ validated |
| 2 | connectivity recompute futex on main thread | busy-spin **deadlock** (Worker can't boot) | `wasm/shims/futex_yield.c` (Asyncify-yield the main-thread futex wait) | ✅ validated (red→green in the `pool-callafter` repro) |
| 3 | `rebaseline()` inside the fiber after a suspend | fiber **never finalizes**, blocks the next apply | move `rebaseline()` to the main stack after `cor.Call` | ✅ validated |
| 4 | `vii` virtual calls (`SetWidth`, snapshot getters) | **hang** (asyncify suspend-without-resume) | devirtualize the call (class-qualify) **OR** … (see Part 4) | ⚠️ confound; see below |
Fixes 1–3 are real and should be kept regardless of the Part-5 decision. Build-plumbing fix to keep
too: `-Xclang -fno-pch-timestamp` in `build-kicad-target.sh` (a PCH-staleness workaround).
---
# Part 4 — The `vii` hang is a confound (the investigation)
### The symptom and the obvious (wrong) theory
The apply hangs at `SetWidth` (a `vii` call). Devirtualize it → the apply progresses to the next
`vii` call (`GetPosition` in the snapshot) → devirtualize that → the next `vii` (`GetClass`, then the
text getters) … Meanwhile value-returning `ii` virtuals (`Type`, `GetLayer`, `GetWidth`, `Clone`) and
3-arg-void `viii` virtuals (`view->Update`) work. **Obvious theory: the `vii` signature is broken.**
### Five controlled repros — all pass
We built a minimal libcontext-fiber app
(`tests/apps/standalone/coroutine-pthread/vcall_fiber_repro.cpp`) that calls all four signatures on a
non-devirtualizable polymorphic object (99 genuine `call_indirect`s, confirmed not optimized away),
native-EH + asyncify, and progressively added every suspected ingredient:
| Repro | Added ingredient | Result |
|-------|------------------|--------|
| `vcall_fiber_repro` | fiber + 4 signatures | **all pass** |
| + interleaved suspend | `emscripten_sleep` before each call | **all pass** |
| `vcall_mainloop_repro` | rAF `set_main_loop` → `dynCall_v` → COROUTINE | **all pass** |
| `vcall_ehloop_repro` | `try`/`catch_all` + RAII dtor + suspend-in-try + loop | **all pass** |
| FIX-D test (real pcbnew) | save+clear `Asyncify.currData` before the apply | `currData` was **null**; still hangs |
A `vii` `call_indirect` dispatches correctly under *every* condition we could isolate — fiber,
suspend/rewind, the rAF/`dynCall_v` boundary, even nested native-EH `try/catch_all` (the
HoistCppCatches regime) with a suspend inside the try. The signature is **not** the cause.
### Two binary-level investigations — dispatch ruled out
Disassembling the actual `pcbnew.wasm` and the repro:
- The asyncify pass is **type-agnostic** (`binaryen/src/passes/Asyncify.cpp`): the void path is the
*simpler* subset; `vii` and `viii` get a byte-for-byte identical guard.
- The function **table is not reordered/re-indexed** (the only fork pass, `HoistCppCatches`, is
intra-function; `-O2` `directize` preserves index→function bindings).
- The **embind-vs-core compile flags match** (same EH model, `-O`, RTTI, struct-ABI) — no ABI
divergence that could move a vtable slot or change a `call_indirect` type.
- The parking `SetWidth` call site has the **same asyncify guard** as the repro's working `setVii`;
no trampoline, no `i64` legalization, no stale type.
The single structural difference round-2 could point to was that the real call sits inside `doApply`'s
**deeply nested native-EH `try`/`catch_all`** (48 `try` / 47 `catch_all` — every throwing JSON access
has RAII cleanup), in a **loop**, with the suspend landing inside hoisted catch scopes — but the
`vcall_ehloop_repro` reproduced exactly that and **passed**.
### Conclusion
The `vii` correlation is a **confound**: devirtualizing removes the instrumented `call_indirect` and
shifts the symptom to the next one. The dispatch is provably fine. The hang is a property of the **full
180 MB asyncify+O2 module's runtime** (the real KiCad vtables/table in the live process) that no
isolation reproduces and that the disassembly couldn't byte-verify. The one stubborn fact that resists
*every* named mechanism: `SetWidth` parks with asyncify state **Normal** and `currData` **null** —
identical to the repros that pass.
---
# Part 5 — The decision: (A) devirtualize-through vs (B) defer collab
Everything else in native-EH pcbnew is green (core e2e, 3D, and every other app —
[`10-pthreads-native-eh.md`](10-pthreads-native-eh.md)). Collab apply is the lone holdout.
## Option A — devirtualize-through (treat the confound, get collab green)
Replace every `vii` virtual call in the apply path with a class-qualified (direct) call, dispatching
on `Type()` where the override matters.
**Where the remaining work is** (the snapshot getters are already done via `itemPosition`/`itemClass`/
the text-getter edits):
| File | Site | Kind |
|------|------|------|
| `wasm/bindings/pcbnew_embind.cpp` | `applyChanged` else-branch `aItem->SetPosition(...)` | 1 setter (type-dispatched) |
| `wasm/bindings/pcbnew_embind.cpp` | `makeItem` added-item setters (`SetWidth`, `SetPosition`, `SetText`, …; `SetStart/SetEnd` already non-virtual, `SetLayer` already done) | ~3–5 setters |
| `wasm/bindings/eeschema_embind.cpp` | `itemToJson` snapshot getters (`GetPosition`/`GetClass`/text — mirror of pcbnew's helpers) | ~3–5 getters (1 helper pair) |
| `wasm/bindings/eeschema_embind.cpp` | `doApply`/`applyChanged` + `makeItem` setters (`Move` already devirtualized) | ~3–5 setters |
**Effort estimate (concrete):**
- **Code edits:** ~10–15 sites, each a 1-line class-qualify or a small `switch(Type())` helper.
Mechanically small — call it **2–4 hours of editing**, including writing the eeschema helper pair to
match pcbnew's.
- **The real cost is the build/test loop, not the edits.** Each un-devirtualized `vii` surfaces *one
at a time* (the apply hangs at the first one; you fix it, rebuild ~10–35 min, it hangs at the next).
Doing it reactively ⇒ **~10–15 build cycles**. You can cut that by proactively grepping every
by-value getter/void setter in the two apply paths and devirtualizing them in one pass, but you
still need a few full builds + the **3-browser** e2e (Firefox/Chrome/WebKit) per the project rule.
Realistically **~1–2 days wall-clock**, dominated by builds + cross-browser verification.
- **Fragility (the ongoing cost):** this fixes nothing structural. Any *new* by-value getter or void
setter added to the apply/snapshot path later — a new field synced, a KiCad upstream change — will
**silently re-hang** the apply under native-EH. Mitigation: a prominent comment + ideally a tiny
lint/grep in CI flagging un-class-qualified virtual calls in the embind apply paths. Without that,
it's a latent foot-gun.
**Net:** A is a known, finite, *certain* path to green, but it's symptom-treatment with a maintenance
tail.
## Option B — defer collab (recommended)
Ship native-EH as the default for everything that's green (core, 3D, gerbview, pl_editor,
symbol_editor, eeschema, footprint_editor). Leave the **collab apply** path as a documented native-EH
limitation; it continues to work on the legacy-EH build. Revisit if/when the large-module root ever
surfaces (e.g. a future Binaryen/emscripten bump changes the picture, or someone reproduces it
minimally).
**Why recommended:** the root resisted 2 deep binary investigations + 5 controlled repros + the FIX-D
test; the dispatch is provably correct; A is fragile work treating a confound. The cost/benefit of
chasing a non-reproducible large-module runtime bug — or maintaining a hand-devirtualized apply path
forever — is poor relative to shipping the 95% that's done.
---
## Appendix — artifacts and key locations
**Validated fixes (keep):**
- `wasm/shims/futex_yield.c` — main-thread futex Asyncify-yield (fix #2).
- `wasm/bindings/pcbnew_embind.cpp` — `kicadCollabApply` COROUTINE (fix #1) + rebaseline-on-main-stack
(fix #3) + the `itemPosition`/`itemClass`/`itemLayer` devirtualized snapshot helpers.
- `scripts/kicad/build-kicad-target.sh` — `futex_yield.o` wired into the link; `-fno-pch-timestamp`.
**Isolation repros (temporary — remove before final staging):**
- `tests/apps/standalone/coroutine-pthread/vcall_fiber_repro.cpp` (signature isolation, +suspend)
- `tests/apps/standalone/coroutine-pthread/vcall_mainloop_repro.cpp` (rAF/`dynCall_v` context)
- `tests/apps/standalone/coroutine-pthread/vcall_ehloop_repro.cpp` (try/catch_all + RAII + loop)
- `tests/apps/standalone/pool-callafter/` (the futex deadlock red→green repro for fix #2)
- `tests/e2e/coroutine-vcall.spec.ts`, `tests/e2e/coroutine-poolwait.spec.ts`
- The `[collab-diag]`/`[push-diag]` `EM_ASM` markers in `pcbnew_embind.cpp` + `board_commit.cpp` and
the `#include <emscripten.h>` in `board_commit.cpp` are temporary diagnostics to revert.
**Key reading:**
- `binaryen/src/passes/Asyncify.cpp` — the suspend/rewind instrumentation (type-agnostic).
- `scripts/common/inject-dyncall-shims.sh`, `scripts/common/shims/{handlesleep.js,dyncall-binding.js.tmpl}`
— the JS-side asyncify/dynCall plumbing.
- [`docs/features/async/`](../async/) — the `currData` contention dossier.

21
docs/features/wasm-exceptions/README.md
View file

@ -1,5 +1,9 @@
# `-fexceptions` vs `-fwasm-exceptions` in KiCad-WASM — research dossier
> **✅ FINALIZED:** native wasm-EH is now the **only** build mode — there is no `-fexceptions` /
> `WX_LEGACY_EH` path, and the 3D viewer builds by default. The migration plan, audit, and spike
> notes below are retained as the historical research/decision record.
> **Status:** research / decision record. A parallel session attempted the migration
> end-to-end and **parked it** on an emscripten-4.0.2 LLVM codegen bug — see
> [`docs/wasm-exceptions-experiment.md`](../../wasm-exceptions-experiment.md) (full
@ -11,6 +15,15 @@
> `currData` contention dossier) — this dossier covers the *exception-handling* axis of
> the same machine.
> **UPDATE 2026-06-22 (see [`06-spike-plan.md`](06-spike-plan.md)).** A 5-agent spike refreshed
> this dossier and corrected three things below: (1) **the encoding is resolved to LEGACY**
> (`WASM_LEGACY_EXCEPTIONS=1`) — Asyncify can't consume exnref in any released Binaryen, so the
> "exnref → TryTable variant" fork is closed; the experiment's `=0` was a dead end. (2) **Binaryen
> is not a blocker** — CI/publish already pin `BINARYEN_VERSION=130` (the "v121 locally" note below
> is only the finalize/in-link copy). (3) The long pole is the **emsdk/LLVM compiler bump** for
> parseable legacy wasm-EH + the OCC `br_table` fix, *not* a newer wasm-opt. The phased red-green
> plan lives in 06.
## Why this exists
The whole build is on **`-fexceptions`** (Emscripten's JavaScript-based exception
@ -60,10 +73,12 @@ KiCad at all.
| [`01-background-two-eh-models.md`](01-background-two-eh-models.md) | How JS-EH (`invoke_*`) and wasm-EH actually work, and the three concrete couplings into our Asyncify machine. |
| [`02-measurements.md`](02-measurements.md) | Our controlled size experiment on pcbnew (methodology + numbers) and the published third-party benchmarks. |
| [`03-toolchain-status.md`](03-toolchain-status.md) | Compatibility matrix: emcc checks, binaryen history (what merged in v125, what didn't), JSPI/fibers, setjmp/longjmp, mixing modes. |
| [`04-kicad-audit.md`](04-kicad-audit.md) | The brace-matching catch-block audit: 85 direct / 93 review / 458 trivial of 636; libpng/libjpeg setjmp story; refactor effort if done by hand. |
| [`05-asyncify-fork-design.md`](05-asyncify-fork-design.md) | Asyncify.cpp internals, why catch arms are structurally hard, and the catch-arm-hoisting fork design with limits and effort. |
| [`catch_audit.py`](catch_audit.py) | The audit tool (re-runnable; suitable as a CI gate on the kicad submodule). |
| [`audit-results.txt`](audit-results.txt) | Full audit output incl. all 85 direct-suspend sites. |
| [`06-spike-plan.md`](06-spike-plan.md) | **(2026-06-22)** Refreshed findings + the phased red-green spike plan; supersedes the encoding/Binaryen-version framing above. |
| [`07-spike-results-and-opinion.md`](07-spike-results-and-opinion.md) | **(2026-06-22)** Toy-spike results: asyncify + legacy-wasm-EH works; the `HoistCppCatches` Binaryen pass flips suspend-in-catch green on all 3 engines; go/no-go opinion. |
| [`08-wx-app-render-rootcause.md`](08-wx-app-render-rootcause.md) | Why a native-EH wx app rendered blank: the `set_main_loop` `"unwind"` throw caught by native-EH `catch_all` cleanup pads tore down the main frame. |
| [`09-event-loop-deparking-plan.md`](09-event-loop-deparking-plan.md) | The EH-agnostic main-loop rework (de-park → per-frame-yield `while`-loop) fixing the blank render + the coroutine/menu regressions. |
| [`10-pthreads-native-eh.md`](10-pthreads-native-eh.md) | **(2026-06-24)** Native-EH × pthreads: the main-thread thread-spawn regression (`invalid state: 1` / re-entrant `main()`); the pool pattern survives; the KiCad raw→pool refactor plan + the test gap. |
## Relationship to docs/features/async/

135
docs/features/wasm-exceptions/audit-results.txt
View file

@ -1,135 +0,0 @@
{
"total": 636,
"trivial": 458,
"needs_review": 93,
"direct_suspend": 85
}
== per top-level dir (direct/review/trivial/infobar):
eeschema direct= 37 review= 17 trivial=127 infobar= 0
pcbnew direct= 32 review= 29 trivial=128 infobar= 0
common direct= 9 review= 35 trivial=126 infobar= 0
cvpcb direct= 3 review= 1 trivial= 0 infobar= 0
pcb_calculator direct= 2 review= 0 trivial= 2 infobar= 0
kicad direct= 1 review= 1 trivial= 21 infobar= 0
pagelayout_editor direct= 1 review= 1 trivial= 3 infobar= 0
scripting direct= 0 review= 0 trivial= 1 infobar= 0
plugins direct= 0 review= 1 trivial= 4 infobar= 0
3d-viewer direct= 0 review= 2 trivial= 6 infobar= 0
utils direct= 0 review= 4 trivial= 27 infobar= 0
libs direct= 0 review= 2 trivial= 7 infobar= 0
gerbview direct= 0 review= 0 trivial= 6 infobar= 0
== top 25 unknown callees inside needs_review catches (freq):
4 clearOutlines
4 message
4 wxASSERT_MSG
4 handleException
4 tl::unexpected
3 wxLogFatalError
3 GetFullFilename
3 std::string
3 ReportMsg
3 line_at
3 positions
3 LIBRARY_PARSE_ERROR
3 move_push
3 wxRemoveFile
3 IsTooRecent
3 Contents
3 GetFieldValue
3 ToOrigString
3 SetValue
2 GetFilename
2 ClearShapes
2 GetFPIDAsString
2 wxString::FromUTF8
2 GetFormatName
2 nan
== direct-suspend sites (85):
cvpcb/display_footprints_frame.cpp:310
cvpcb/cvpcb_mainframe.cpp:977
cvpcb/readwrite_dlgs.cpp:163
pcb_calculator/datafile_read_write.cpp:69
pcb_calculator/calculator_panels/panel_r_calculator.cpp:130
kicad/kicad_manager_frame.cpp:822
common/draw_panel_gal.cpp:325
common/draw_panel_gal.cpp:561
common/drawing_sheet/ds_data_model_io.cpp:100
common/drawing_sheet/ds_data_model_io.cpp:131
common/widgets/design_block_pane.cpp:121
common/widgets/design_block_pane.cpp:212
common/widgets/design_block_pane.cpp:315
common/widgets/design_block_pane.cpp:356
common/widgets/design_block_pane.cpp:418
pcbnew/load_select_footprint.cpp:378
pcbnew/pcb_base_frame.cpp:1111
pcbnew/pcb_base_frame.cpp:1239
pcbnew/pcb_edit_frame.cpp:2209
pcbnew/files.cpp:670
pcbnew/files.cpp:678
pcbnew/files.cpp:689
pcbnew/files.cpp:1024
pcbnew/files.cpp:1108
pcbnew/footprint_libraries_utils.cpp:198
pcbnew/footprint_libraries_utils.cpp:288
pcbnew/footprint_libraries_utils.cpp:401
pcbnew/footprint_libraries_utils.cpp:545
pcbnew/footprint_libraries_utils.cpp:613
pcbnew/footprint_libraries_utils.cpp:673
pcbnew/footprint_libraries_utils.cpp:817
pcbnew/pcb_design_block_utils.cpp:107
pcbnew/pcb_design_block_utils.cpp:168
pcbnew/pcb_design_block_utils.cpp:198
pcbnew/pcb_design_block_utils.cpp:226
pcbnew/pcb_design_block_utils.cpp:332
pcbnew/pcb_design_block_utils.cpp:429
pcbnew/pcb_draw_panel_gal.cpp:781
pcbnew/tools/board_editor_control.cpp:751
pcbnew/tools/footprint_editor_control.cpp:529
pcbnew/tools/pcb_control.cpp:1990
pcbnew/netlist_reader/netlist.cpp:74
pcbnew/exporters/export_idf.cpp:662
pcbnew/exporters/export_idf.cpp:670
pcbnew/dialogs/dialog_board_setup.cpp:385
pcbnew/specctra_import_export/specctra_import.cpp:71
pcbnew/widgets/pcb_design_block_preview_widget.cpp:191
eeschema/project_rescue.cpp:671
eeschema/project_rescue.cpp:819
eeschema/sch_draw_panel.cpp:200
eeschema/project_sch.cpp:108
eeschema/project_sch.cpp:122
eeschema/sch_design_block_utils.cpp:139
eeschema/sch_design_block_utils.cpp:179
eeschema/sch_design_block_utils.cpp:222
eeschema/sch_design_block_utils.cpp:364
eeschema/sch_design_block_utils.cpp:446
eeschema/sch_design_block_utils.cpp:526
eeschema/sch_edit_frame.cpp:1528
eeschema/sch_base_frame.cpp:96
eeschema/sheet.cpp:231
eeschema/files-io.cpp:356
eeschema/files-io.cpp:365
eeschema/files-io.cpp:374
eeschema/files-io.cpp:911
eeschema/files-io.cpp:1410
eeschema/files-io.cpp:1423
eeschema/symbol_library_manager.cpp:376
eeschema/symbol_library_manager.cpp:553
eeschema/tools/sch_editor_control.cpp:569
eeschema/tools/sch_editor_control.cpp:1703
eeschema/dialogs/dialog_edit_symbols_libid.cpp:742
eeschema/dialogs/dialog_sheet_properties.cpp:597
eeschema/dialogs/dialog_sim_model.cpp:1526
eeschema/dialogs/dialog_bom.cpp:361
eeschema/sim/spice_simulator.cpp:44
eeschema/sim/spice_value.cpp:415
eeschema/sim/simulator_frame_ui.cpp:1672
eeschema/symbol_editor/symbol_editor.cpp:170
eeschema/symbol_editor/symbol_editor.cpp:236
eeschema/symbol_editor/symbol_editor.cpp:1147
eeschema/symbol_editor/symbol_editor.cpp:1187
eeschema/symbol_editor/symbol_editor_import_export.cpp:104
eeschema/symbol_editor/symbol_editor_import_export.cpp:110
pagelayout_editor/tools/pl_edit_tool.cpp:531

109
docs/features/wasm-exceptions/catch_audit.py
View file

@ -1,109 +0,0 @@
#!/usr/bin/env python3
"""Audit KiCad catch blocks for wasm-EH safety (suspension inside catch handlers)."""
import os, re, sys, json
from collections import defaultdict
ROOT = "/Users/V/IdeaProjects/kicad-wasm/kicad"
SKIP_DIRS = {"thirdparty", ".git", "qa", "build"}
# Calls that suspend (Asyncify) directly or are dialog wrappers in KiCad/wx
DIRECT_SUSPEND = [
"DisplayErrorMessage", "DisplayError", "DisplayInfoMessage", "DisplayHtmlInfoMessage",
"wxMessageBox", "ShowModal", "ShowQuasiModal", "KIDIALOG", "OKOrCancelDialog",
"wxMessageDialog", "IsOK(", "DisplayLoadError", "ShowAboutDialog",
"wxGetSingleChoice", "wxTextEntryDialog", "wxFileDialog", "wxDirDialog",
"GetDataFromClipboard", "SaveToClipboard", "wxClipboard", "EnumerateFacenames",
]
INFOBAR = ["ShowInfoBarError", "ShowInfoBarMsg", "ShowInfoBarWarning"]
# Benign callees: logging (wxLog* is deferred to idle-time flush -> not inside catch),
# string formatting, rethrow, reporters writing text
BENIGN = {
"wxLogError", "wxLogWarning", "wxLogMessage", "wxLogTrace", "wxLogDebug", "wxLogVerbose",
"Format", "Printf", "printf", "fprintf", "snprintf", "What", "Problem", "Where",
"GetErrorMessage", "wxString", "FROM_UTF8", "TO_UTF8", "UTF8", "c_str", "mb_str",
"GetChars", "IsEmpty", "empty", "clear", "size", "length", "Report", "ReportTail",
"ReportHead", "Add", "push_back", "emplace_back", "insert", "append", "Append",
"SetError", "assert", "wxASSERT", "wxFAIL", "wxCHECK", "abort", "exit",
"GetMessages", "GetErrors", "reset", "get", "release", "find", "count", "at",
"begin", "end", "str", "Mid", "Left", "Right", "Trim", "Lower", "Upper",
"StartsWith", "EndsWith", "Contains", "Replace", "Remove", "make_unique",
"make_shared", "static_cast", "dynamic_cast", "const_cast", "reinterpret_cast",
"Clear", "Close", "swap", "resize", "erase", "Set", "SetBitmap", "Destroy",
"wxT", "_", "_HKI", "traceSchPlugin", "TRACE", "what", "THROW_IO_ERROR", "wxS", "wxFAIL_MSG", "IDF_ERROR", "LIBRARY_ERROR", "FUTURE_FORMAT_ERROR", "PARSE_ERROR", "KI_PARAM_ERROR", "fmt::format", "format", "GetFullPath", "HandleException", "Pgm", "current_exception", "rethrow_exception", "Nickname", "GetName", "GetLibNickname", "GetLibItemName", "wx_str", "GetMessageString", "UnescapeString", "exceptions", "CLOSE_STREAM", "AddError", "GetRequiredVersion", "Disconnect", "From_UTF8", "typeid", "LogException", "Instance", "GetFullName", "GetUniStringLibId", "ShowText", "SetStatusText", "GetItemDescription", "GetClass", "GetFriendlyName", "IsValid", "GetPath", "GetFileName", "GetExtension", "Length", "GetData", "data", "front", "back", "pop_back", "emplace", "GetSettingsManager",
}
CALL_RE = re.compile(r"\b([A-Za-z_][A-Za-z0-9_:]*)\s*\(")
CATCH_RE = re.compile(r"\bcatch\s*\(")
def find_block(text, start):
"""Return (block_text, end_idx) for brace-block starting at first '{' at/after start."""
i = text.find("{", start)
if i < 0: return None, start
depth, j, n = 0, i, len(text)
in_str = in_chr = in_lc = in_bc = False
while j < n:
c = text[j]; p = text[j-1] if j else ""
if in_lc:
if c == "\n": in_lc = False
elif in_bc:
if p == "*" and c == "/": in_bc = False
elif in_str:
if c == '"' and p != "\\": in_str = False
elif in_chr:
if c == "'" and p != "\\": in_chr = False
elif c == "/" and j+1 < n and text[j+1] == "/": in_lc = True
elif c == "/" and j+1 < n and text[j+1] == "*": in_bc = True
elif c == '"': in_str = True
elif c == "'": in_chr = True
elif c == "{": depth += 1
elif c == "}":
depth -= 1
if depth == 0: return text[i:j+1], j+1
j += 1
return None, start
stats = defaultdict(int)
direct_sites, review_sites = [], []
review_callees = defaultdict(int)
per_dir = defaultdict(lambda: defaultdict(int))
for dirpath, dirnames, filenames in os.walk(ROOT):
dirnames[:] = [d for d in dirnames if d not in SKIP_DIRS]
for fn in filenames:
if not fn.endswith(".cpp"): continue
path = os.path.join(dirpath, fn)
rel = os.path.relpath(path, ROOT)
top = rel.split(os.sep)[0]
try: text = open(path, encoding="utf-8", errors="replace").read()
except OSError: continue
for m in CATCH_RE.finditer(text):
block, _ = find_block(text, m.end())
if block is None: continue
stats["total"] += 1
line = text[:m.start()].count("\n") + 1
loc = f"{rel}:{line}"
if any(s in block for s in DIRECT_SUSPEND):
stats["direct_suspend"] += 1; per_dir[top]["direct"] += 1
direct_sites.append(loc)
continue
if any(s in block for s in INFOBAR):
stats["infobar"] += 1; per_dir[top]["infobar"] += 1
continue
calls = set(CALL_RE.findall(block)) - {"catch", "if", "for", "while", "switch", "return", "sizeof", "throw"}
unknown = {c for c in calls if c.split("::")[-1] not in BENIGN and c not in BENIGN}
if not unknown:
stats["trivial"] += 1; per_dir[top]["trivial"] += 1
else:
stats["needs_review"] += 1; per_dir[top]["review"] += 1
review_sites.append((loc, sorted(unknown)[:6]))
for c in unknown: review_callees[c] += 1
print(json.dumps(stats, indent=1))
print("\n== per top-level dir (direct/review/trivial/infobar):")
for d in sorted(per_dir, key=lambda d: -per_dir[d]["direct"]):
p = per_dir[d]
print(f" {d:24s} direct={p['direct']:3d} review={p['review']:3d} trivial={p['trivial']:3d} infobar={p['infobar']:2d}")
print("\n== top 25 unknown callees inside needs_review catches (freq):")
for c, n in sorted(review_callees.items(), key=lambda kv: -kv[1])[:25]:
print(f" {n:3d} {c}")
print(f"\n== direct-suspend sites ({len(direct_sites)}):")
for s in direct_sites: print(" " + s)

8
docs/wasm-exceptions-experiment.md
View file

@ -5,6 +5,14 @@ codegen bug in emscripten 4.0.2 — see "The blocker" below. The code changes we
locally and then dropped; the full patch is preserved in the appendix of this doc,
together with everything needed to resume.
> **Correction 2026-06-22 — see `docs/features/wasm-exceptions/06-spike-plan.md`.** This
> experiment switched to `-sWASM_LEGACY_EXCEPTIONS=0` (exnref) to dodge the legacy-encoding
> parse failure — but that is a **dead end**: Binaryen's Asyncify cannot instrument
> `try_table`/exnref in any released version (incl. v130), so an exnref build would die at
> the `--asyncify` step even after the `br_table` bug is fixed. **Resume with `=1` (legacy),
> not `=0`.** The legacy parse failure is almost certainly an em-4.0.2 LLVM-codegen artifact;
> the fix is the emsdk/LLVM bump, not the encoding switch.
## Why this matters
After the 3.3x CI win (see `ci-build-slowness-findings.md`), the critical path of the

32
scripts/binaryen-hoist-pass/build-wasm-opt.sh Executable file
View file

@ -0,0 +1,32 @@
#!/bin/bash
# Build a wasm-opt that includes the catch-arm-hoisting pass (--hoist-cpp-catches).
#
# Source of truth is the tracked Binaryen submodule (binaryen/, branch wasm-port =
# upstream version_130 + src/passes/HoistCppCatches.cpp). This configures an out-of-source
# build into the gitignored build-wasm/ tree and prints the wasm-opt path on stdout
# (build progress to stderr). See docs/features/wasm-exceptions/06-spike-plan.md (Phase 1.5).
set -eo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
SRC="${PROJECT_ROOT}/binaryen"
BUILD="${PROJECT_ROOT}/build-wasm/tools/binaryen-hoist-build"
if [ ! -f "${SRC}/src/passes/HoistCppCatches.cpp" ]; then
echo "ERROR: binaryen submodule is missing the hoist pass." >&2
echo " Run: git submodule update --init binaryen" >&2
exit 1
fi
# Configure once (mirrors scripts/common/get-wasm-opt.sh's from-source flags).
if [ ! -f "${BUILD}/build.ninja" ]; then
echo "Configuring Binaryen submodule build (one-time, ~5 min to build)..." >&2
cmake -S "${SRC}" -B "${BUILD}" -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CXX_FLAGS="-Wno-maybe-uninitialized" \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON -DBUILD_TESTS=OFF >&2
fi
ninja -C "${BUILD}" wasm-opt >&2
echo "${BUILD}/bin/wasm-opt"

36
scripts/binaryen-hoist-pass/tests/asyncify-harness.js Normal file
View file

@ -0,0 +1,36 @@
// Minimal Asyncify harness: drive one unwind/rewind through $vt and check it yields 50.
const fs = require('fs');
const path = process.argv[2];
const BUF = 16, STACK = 1024, STACK_END = 16384;
let inst, rewinding = false, pending = 0;
const imports = {
env: {
sleep: (ms) => {
if (!rewinding) {
inst.exports.asyncify_start_unwind(BUF);
pending = ms;
return 0; // dummy; value is discarded as we unwind
} else {
inst.exports.asyncify_stop_rewind();
rewinding = false;
return pending; // real value supplied on rewind
}
},
},
};
const mod = new WebAssembly.Module(fs.readFileSync(path));
inst = new WebAssembly.Instance(mod, imports);
// asyncify buffer struct: [current, end]
const mem = new Int32Array(inst.exports.memory.buffer);
mem[BUF >> 2] = STACK;
mem[(BUF + 4) >> 2] = STACK_END;
inst.exports.vt(); // runs, throws, catch calls sleep -> starts unwind, returns
inst.exports.asyncify_stop_unwind();
inst.exports.asyncify_start_rewind(BUF);
rewinding = true;
const r = inst.exports.vt(); // rewinds into the catch handler; sleep returns 50; vt completes
console.log('vt result =', r);
process.exit(r === 50 ? 0 : 1);

32
scripts/binaryen-hoist-pass/tests/delegate-orphan-suspend.wat Normal file
View file

@ -0,0 +1,32 @@
;; The DuplicateSymbol delegate-orphan shape, but the hoisted cpp arm SUSPENDS (calls an async import)
;; before its __cxa_end_catch cleanup (try (do ..) (delegate $M)). Drives a real Asyncify unwind/rewind
;; through the hoisted arm AND its retargeted delegate ($M -> caller), proving the fix doesn't break
;; the suspend/rewind the hoist exists for. Same harness contract as the other -suspend tests: $vt
;; yields 50.
(module
(import "env" "sleep" (func $sleep (param i32) (result i32)))
(memory (export "memory") 1)
(tag $cpp (param i32))
(func $vt (export "vt") (result i32)
(local $r i32)
(try $A
(do
(throw $cpp (i32.const 0)))
(catch_all
(try $M
(do
(try $inner
(do
(rethrow $A))
(catch $cpp
(drop (pop i32))
(local.set $r (call $sleep (i32.const 50)))
(try
(do (nop))
(delegate $M)))
(catch_all
(rethrow $A))))
(catch_all
(rethrow $A)))))
(local.get $r))
)

60
scripts/binaryen-hoist-pass/tests/delegate-orphan.wat Normal file
View file

@ -0,0 +1,60 @@
;; Regression repro of SYMBOL_EDIT_FRAME::DuplicateSymbol (KiCad eeschema). A cpp catch arm that is
;; hoisted PAST an ancestor catch_all (the case-6 deferral) carries a nested __cxa_end_catch cleanup
;; (try (do ..) (delegate $M)) whose delegate target $M is a mid try sitting INSIDE that ancestor
;; catch_all. Hoisting the arm out (into the dispatch section, outside every try) orphaned the
;; delegate -> wasm-validator "all delegate targets must be valid, on (delegate $M)". The fix
;; retargets it to DELEGATE_CALLER_TARGET (a delegate can only target a try or the caller, not the
;; $done block); re-throwing a cleanup exception to the caller is the C++ throw-during-cleanup
;; (std::terminate) path, never taken in normal flow. cpp tag = single i32.
(module
(tag $cpp (param i32))
;; Exception path: $A throws cpp(1); its catch_all (re)throws into $inner whose cpp arm sets r:=42,
;; its __cxa_end_catch cleanup delegates to the enclosing $M. expect 42.
(func $caught (export "caught") (result i32)
(local $r i32)
(try $A
(do
(throw $cpp (i32.const 1)))
(catch_all
(try $M
(do
(try $inner
(do
(rethrow $A))
(catch $cpp
(drop (pop i32))
(local.set $r (i32.const 42))
(try
(do (nop))
(delegate $M)))
(catch_all
(rethrow $A))))
(catch_all
(rethrow $A)))))
(local.get $r))
;; No-exception path: $A body falls through with r:=7. expect 7.
(func $normal (export "normal") (result i32)
(local $r i32)
(try $A
(do
(local.set $r (i32.const 7)))
(catch_all
(try $M
(do
(try $inner
(do
(rethrow $A))
(catch $cpp
(drop (pop i32))
(local.set $r (i32.const 42))
(try
(do (nop))
(delegate $M)))
(catch_all
(rethrow $A))))
(catch_all
(rethrow $A)))))
(local.get $r))
)

49
scripts/binaryen-hoist-pass/tests/nested-catchall-exit-block.wat Normal file
View file

@ -0,0 +1,49 @@
;; Regression repro of PGM_BASE::HandleException (KiCad) — the case-6 shape (a cpp catch nested in an
;; outer try's catch_all cleanup pad) WITH an intervening block in that cleanup that the cpp arm
;; br's to. LLVM emits this for `try{} catch(A&) catch(B&) catch(...)`: the outer try has only a
;; catch_all (destructor cleanup), inside which it (rethrow)s into a nested try whose cpp catch does
;; the __cxa type dispatch and, when done, (br)s OUT to a block ($blk) sitting between the escape
;; target and the arm. Hoisting that arm to the dispatch section orphaned the br — "all break targets
;; must be valid, on (br $blk)". cpp tag = single i32.
(module
(tag $cpp (param i32))
;; Exception path: outer body throws cpp(1); the catch_all reclassifies via (rethrow $outer) inside
;; (block $blk); the nested cpp arm handles it (r := 42) then (br $blk) to finish. expect 42.
(func $caught (export "caught") (result i32)
(local $r i32)
(try $outer
(do
(throw $cpp (i32.const 1)))
(catch_all
(block $blk
(try
(do
(rethrow $outer))
(catch $cpp
(drop (pop i32))
(local.set $r (i32.const 42))
(br $blk))
(catch_all
(rethrow $outer))))))
(local.get $r))
;; No-exception path: body falls through with r := 7, neither catch runs. expect 7.
(func $normal (export "normal") (result i32)
(local $r i32)
(try $outer
(do
(local.set $r (i32.const 7)))
(catch_all
(block $blk
(try
(do
(rethrow $outer))
(catch $cpp
(drop (pop i32))
(local.set $r (i32.const 42))
(br $blk))
(catch_all
(rethrow $outer))))))
(local.get $r))
)

26
scripts/binaryen-hoist-pass/tests/nested-catchall-suspend.wat Normal file
View file

@ -0,0 +1,26 @@
;; The HandleException nested-catchall-exit-block shape, but the nested cpp arm SUSPENDS (calls an
;; async import) before it br's the intervening block. This drives a real Asyncify unwind/rewind
;; through the hoisted arm AND its retargeted br ($blk -> $done), proving the fix doesn't break the
;; suspend/rewind the hoist exists for. Same harness contract as value-typed-suspend.wat: $vt yields 50.
(module
(import "env" "sleep" (func $sleep (param i32) (result i32)))
(memory (export "memory") 1)
(tag $cpp (param i32))
(func $vt (export "vt") (result i32)
(local $r i32)
(try $outer
(do
(throw $cpp (i32.const 0)))
(catch_all
(block $blk
(try
(do
(rethrow $outer))
(catch $cpp
(drop (pop i32))
(local.set $r (call $sleep (i32.const 50)))
(br $blk))
(catch_all
(rethrow $outer))))))
(local.get $r))
)

57
scripts/binaryen-hoist-pass/tests/run.sh Executable file
View file

@ -0,0 +1,57 @@
#!/usr/bin/env bash
# Regression test for the value-typed (concrete-result) path of --hoist-cpp-catches.
#
# Value-typed cpp-catch tries do not arise from normal C++ EH lowering (LLVM keeps catch values in
# locals → void/unreachable tries), so this case can't live in a C++ EH toy. These
# hand-written modules exercise it directly:
# (1) fuzz-exec — the pass must preserve the result value of value-typed cpp-catch tries
# (exception path, no-exception path, and exception-payload routing).
# (2) a real asyncify unwind/rewind through a value-typed catch that SUSPENDS (must yield 50).
#
# Requires wasm-opt built from the binaryen submodule (scripts/binaryen-hoist-pass/build-wasm-opt.sh);
# that one binary (version_130 + our hoist pass) does both --hoist-cpp-catches and --asyncify. Run from anywhere.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/../../.." && pwd)"
DIR="$ROOT/scripts/binaryen-hoist-pass/tests"
WASMOPT="$ROOT/build-wasm/tools/binaryen-hoist-build/bin/wasm-opt"
V130="$WASMOPT" # the submodule fork IS version_130 (asyncify unchanged), so it does --asyncify too
[ -x "$WASMOPT" ] || { echo "build wasm-opt first: scripts/binaryen-hoist-pass/build-wasm-opt.sh"; exit 1; }
echo "== (1) value semantics preserved (fuzz-exec) =="
"$WASMOPT" --hoist-cpp-catches -all -all --fuzz-exec "$DIR/value-typed-cpp-catch.wat" -o /dev/null 2>&1 \
| grep -E 'comparing|=>'
echo "== (2) asyncify unwind/rewind through a value-typed suspending catch =="
"$WASMOPT" --hoist-cpp-catches -all -all "$DIR/value-typed-suspend.wat" -o /tmp/vt_s.hoisted.wasm
"$V130" --asyncify -all --pass-arg=asyncify-imports@env.sleep /tmp/vt_s.hoisted.wasm -o /tmp/vt_s.async.wasm
node "$DIR/asyncify-harness.js" /tmp/vt_s.async.wasm
echo "OK — value-typed path verified"
# Regression for the PGM_BASE::HandleException shape: a cpp catch nested in an outer try's catch_all
# cleanup pad whose arm br's to an intervening block ($blk). Before the fix the hoisted arm's br was
# orphaned -> "all break targets must be valid". (3) validates + checks value semantics; (4) drives a
# real unwind/rewind through the hoisted arm with the retargeted br.
echo "== (3) nested-catchall exit-block hoists + validates (fuzz-exec: caught=>42, normal=>7) =="
"$WASMOPT" --hoist-cpp-catches -all -all --fuzz-exec "$DIR/nested-catchall-exit-block.wat" -o /dev/null 2>&1 \
| grep -E 'comparing|=>'
echo "== (4) asyncify unwind/rewind through a suspending nested-catchall arm =="
"$WASMOPT" --hoist-cpp-catches -all -all "$DIR/nested-catchall-suspend.wat" -o /tmp/ncb_s.hoisted.wasm
"$V130" --asyncify -all --pass-arg=asyncify-imports@env.sleep /tmp/ncb_s.hoisted.wasm -o /tmp/ncb_s.async.wasm
node "$DIR/asyncify-harness.js" /tmp/ncb_s.async.wasm
echo "OK — nested-catchall path verified"
# Regression for the SYMBOL_EDIT_FRAME::DuplicateSymbol shape: a cpp catch arm hoisted PAST an ancestor
# catch_all carries a nested __cxa_end_catch cleanup (try (do ..) (delegate $M)) whose delegate target
# sits inside that catch_all -> orphaned by hoisting ("all delegate targets must be valid"). The fix
# retargets it to DELEGATE_CALLER_TARGET. (5) validates + checks value semantics; (6) drives a real
# unwind/rewind through the hoisted arm with the retargeted delegate.
echo "== (5) delegate-orphan hoists + validates (fuzz-exec: caught=>42, normal=>7) =="
"$WASMOPT" --hoist-cpp-catches -all -all --fuzz-exec "$DIR/delegate-orphan.wat" -o /dev/null 2>&1 \
| grep -E 'comparing|=>'
echo "== (6) asyncify unwind/rewind through a suspending delegate-orphan arm =="
"$WASMOPT" --hoist-cpp-catches -all -all "$DIR/delegate-orphan-suspend.wat" -o /tmp/dlg_s.hoisted.wasm
"$V130" --asyncify -all --pass-arg=asyncify-imports@env.sleep /tmp/dlg_s.hoisted.wasm -o /tmp/dlg_s.async.wasm
node "$DIR/asyncify-harness.js" /tmp/dlg_s.async.wasm
echo "OK — delegate-orphan path verified"

20
scripts/binaryen-hoist-pass/tests/value-typed-cpp-catch.wat Normal file
View file

@ -0,0 +1,20 @@
;; Hand-written value-typed cpp-catch tries to exercise the $result routing in --hoist-cpp-catches.
;; cpp tag = single i32 param. Each function returns an i32 via a (try (result i32) ...).
(module
(tag $cpp (param i32))
;; exception path: body throws, catch yields 42 -> expect 42
(func $vt_throw (export "vt_throw") (result i32)
(try (result i32)
(do (throw $cpp (i32.const 99)))
(catch $cpp (drop (pop i32)) (i32.const 42))))
;; normal path: body yields 7, catch never runs -> expect 7
(func $vt_normal (export "vt_normal") (result i32)
(try (result i32)
(do (i32.const 7))
(catch $cpp (drop (pop i32)) (i32.const 42))))
;; payload routing: catch returns the exception payload it caught (123) -> expect 123
(func $vt_payload (export "vt_payload") (result i32)
(try (result i32)
(do (throw $cpp (i32.const 123)))
(catch $cpp (pop i32))))
)

10
scripts/binaryen-hoist-pass/tests/value-typed-suspend.wat Normal file
View file

@ -0,0 +1,10 @@
;; Value-typed try whose catch SUSPENDS: the catch calls an async import and yields its result.
;; Drives the $result routing across a real asyncify unwind/rewind.
(module
(import "env" "sleep" (func $sleep (param i32) (result i32)))
(memory (export "memory") 1)
(tag $cpp (param i32))
(func $vt (export "vt") (result i32)
(try (result i32)
(do (throw $cpp (i32.const 0)))
(catch $cpp (drop (pop i32)) (call $sleep (i32.const 50))))))

51
scripts/build-wasm-test.sh
View file

@ -92,6 +92,25 @@ if [ "$CLEAN_BUILD" = "1" ]; then
make -f Makefile.wasm clean 2>/dev/null || true
fi
# Native wasm-EH is the only build mode. The emsdk-bundled Binaryen v121 crashes asyncifying wasm-EH,
# so we stub the in-link Asyncify and run --hoist-cpp-catches + --asyncify post-link on the Binaryen
# submodule (version_130 + hoist pass) via apply-asyncify.sh.
EMSDK_WASM_OPT="$PROJECT_ROOT/tools/emsdk/upstream/bin/wasm-opt"
WASMOPT_STUB="$PROJECT_ROOT/wasm/stubs/wasm-opt-stub.sh"
_eh_restore_wasmopt() { [ -f "${EMSDK_WASM_OPT}.ehbak" ] && mv -f "${EMSDK_WASM_OPT}.ehbak" "${EMSDK_WASM_OPT}"; }
EH_MARKER="$(mktemp)" # created before the build so 'find -newer' below selects freshly-linked apps
echo ""
echo "=== Building the Binaryen submodule (version_130 + hoist pass) ==="
# One binaryen everywhere: the submodule fork is version_130 (asyncify unchanged) + our hoist
# pass, so the same binary does --hoist-cpp-catches AND --asyncify/-O2. No separate v130 clone.
export HOIST_WASMOPT="$("$SCRIPT_DIR/binaryen-hoist-pass/build-wasm-opt.sh")"
export V130_WASMOPT="$HOIST_WASMOPT"
echo " submodule wasm-opt: $HOIST_WASMOPT"
echo "Stubbing in-link Asyncify (will run post-link instead)..."
cp "$EMSDK_WASM_OPT" "${EMSDK_WASM_OPT}.ehbak"
cp "$WASMOPT_STUB" "$EMSDK_WASM_OPT"; chmod +x "$EMSDK_WASM_OPT"
trap _eh_restore_wasmopt EXIT
# Build (pass DEBUG flag if requested). App links are independent, so honor
# JOBS/PARALLEL_JOBS from env.sh (each emcc link is slow due to Asyncify).
if [ "$DEBUG_BUILD" = "1" ]; then
@ -99,6 +118,38 @@ if [ "$DEBUG_BUILD" = "1" ]; then
else
make -j"${JOBS:-1}" -f Makefile.wasm "$MAKE_TARGET"
fi
make_rc=$?
if [ "$make_rc" -ne 0 ]; then
# Fail loudly. Silently continuing to the post-link leaves the freshly-linked apps
# asyncify-stubbed / un-injected, which looks like mass test failures rather than a build
# error. (The EXIT trap restores the stubbed emsdk wasm-opt in the native-EH build.)
echo "" >&2
echo "ERROR: make failed (exit $make_rc); aborting before the post-link step." >&2
exit "$make_rc"
fi
# Inject the dyncall + handlesleep currData shims into every freshly-linked app. The
# handlesleep currData save/restore (Emscripten #9153) is needed: without it a rewind that
# resumes through a fresh wasm re-entry hits _asyncify_start_rewind(null) -> "memory access out
# of bounds" — e.g. a context-menu pick while the main loop is parked. The Makefile only injects
# it for the coroutine apps; inject-dyncall-shims.sh is idempotent (skips an already-shimmed glue),
# so re-running it here is safe. The .wasm gets post-link hoist + asyncify first.
_eh_restore_wasmopt; trap - EXIT
echo ""
echo "=== Post-link --hoist-cpp-catches + --asyncify ==="
while IFS= read -r w; do
"$SCRIPT_DIR/common/apply-asyncify.sh" --no-removelist "$w"
js="${w%.wasm}.js"
if [ -f "$js" ]; then
( cd "$(dirname "$js")" && "$SCRIPT_DIR/common/inject-dyncall-shims.sh" "$(basename "$js")" )
fi
# Match EVERY freshly-linked app wasm, not just standalone/*/*_test.wasm: the main demo
# (apps/minimal_test.wasm) is at the apps/ root, and the coroutine-pthread repros + wxpt app
# are *_repro*.wasm / *_wxpt.wasm. The old '*_test.wasm under standalone' filter silently
# skipped all of those, so under native wasm-EH they never got hoist+asyncify and crashed at
# runtime with "asyncify_start_unwind not found".
done < <(find "$WASM_APP_DIR" -name '*.wasm' -newer "$EH_MARKER")
rm -f "$EH_MARKER"
echo ""
echo "=== Build complete ==="

10
scripts/build-wx-wasm.sh
View file

@ -158,9 +158,15 @@ if [ $NEEDS_CONFIGURE -eq 1 ]; then
echo "Building wxWidgets in RELEASE mode"
fi
# Exception model: native WebAssembly exceptions (legacy binary encoding) + wasm setjmp/longjmp,
# single-sourced from scripts/common/env.sh. The catch-arm-hoisting pass (run post-link, see
# build-wasm-test.sh) lets Asyncify suspend from inside C++ catch blocks. See docs/features/wasm-exceptions/.
WX_EH_FLAGS="$DEPS_EH_FLAGS"
echo "wx EH model flags: ${WX_EH_FLAGS}"
# Include emscripten cache sysroot for zlib headers
export CFLAGS="-DZ_HAVE_UNISTD_H=1 -I$EM_CACHE_SYSROOT/include ${WX_DEBUG_FLAGS} -fexceptions -pthread -matomics -mbulk-memory"
export CXXFLAGS="-DZ_HAVE_UNISTD_H=1 -I$EM_CACHE_SYSROOT/include -I$PCRE2_INCLUDE ${WX_DEBUG_FLAGS} -fexceptions -pthread -matomics -mbulk-memory"
export CFLAGS="-DZ_HAVE_UNISTD_H=1 -I$EM_CACHE_SYSROOT/include ${WX_DEBUG_FLAGS} ${WX_EH_FLAGS} -pthread -matomics -mbulk-memory"
export CXXFLAGS="-DZ_HAVE_UNISTD_H=1 -I$EM_CACHE_SYSROOT/include -I$PCRE2_INCLUDE ${WX_DEBUG_FLAGS} ${WX_EH_FLAGS} -pthread -matomics -mbulk-memory"
export LDFLAGS="-L$EM_CACHE_SYSROOT/lib/wasm32-emscripten"
emconfigure "$WX_SOURCE/configure" \

227
scripts/common/apply-asyncify.sh
View file

@ -1,157 +1,138 @@
#!/bin/bash
# Apply asyncify transformation to KiCad WASM
# Unified post-link Asyncify pass for KiCad AND the wx test apps.
#
# Usage: ./scripts/common/apply-asyncify.sh <input.wasm> <output.wasm>
# Usage: apply-asyncify.sh [--no-removelist] <input.wasm> [output.wasm]
#
# This script is called by docker/build.sh but can also be run standalone
# for debugging asyncify issues.
# Always: run our --hoist-cpp-catches fork pass FIRST (lets Asyncify suspend from inside C++ catch
# blocks under native wasm-EH) with all wasm features enabled (-all, so binaryen parses the
# EH instructions), then --asyncify + remove-list + -O2. Native wasm-EH is the only build mode.
# --no-removelist skip the KiCad big-function remove-list (the small wx test apps don't contain
# those symbols, and one bare entry — "match" — could collide).
#
# WHY post-link (not emcc's in-link Asyncify): the emsdk-bundled Binaryen crashes asyncifying
# wasm-EH and a compiler/standalone version skew corrupts asyncify metadata. So the in-link pass is
# stubbed (build-kicad-target.sh / build-wasm-test.sh) and the real transform runs here, on the host
# (more RAM), with a pinned Binaryen. The cost: emcc's automatic asyncify-imports generation is
# bypassed, so the BOUNDARY import list lives in asyncify-imports.txt (see that file).
#
# Binaryen selection (env overrides win, set by build-wasm-test.sh): one binaryen everywhere — the
# submodule fork (version_130 + our --hoist-cpp-catches). Override the path via HOIST_WASMOPT / V130_WASMOPT.
set -e
set -eo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
# Get wasm-opt path
WASM_OPT=$("${SCRIPT_DIR}/get-wasm-opt.sh")
# --- flags ---
# Native wasm-EH is the only build mode, so the catch-arm hoist pass always runs.
DO_HOIST=1
USE_REMOVELIST=1
while [[ "${1:-}" == --* ]]; do
case "$1" in
--no-removelist) USE_REMOVELIST=0; shift ;;
*) echo "apply-asyncify: unknown flag: $1" >&2; exit 1 ;;
esac
done
# Bound Binaryen's host thread pool. wasm-opt runs function-parallel passes, and
# each worker holds the optimization working-set of one function at a time — so
# peak RAM scales with thread count. Binaryen reads BINARYEN_CORES to size the
# pool; default to 8 for memory-constrained dev machines, overridable via the
# environment (CI sets it to $(nproc) on the 128 GB Hetzner runner).
export BINARYEN_CORES="${BINARYEN_CORES:-8}"
INPUT_WASM="${1:-output/pcbnew.wasm}"
OUTPUT_WASM="${2:-${INPUT_WASM}}"
[ -f "${INPUT_WASM}" ] || { echo "ERROR: Input file not found: ${INPUT_WASM}" >&2; exit 1; }
# Preload a scalable allocator on Linux. wasm-opt churns a ~40 GB high-water mark
# of short-lived allocations across all worker threads; glibc malloc serializes
# concurrent alloc/free on per-arena locks, so under many threads ~half of every
# core's cycles collapse into futex lock-spin (strace: ~99% kernel time in futex)
# instead of optimization work — the more cores, the worse it gets. jemalloc and
# mimalloc are built for exactly this many-thread churn and eliminate the storm,
# roughly halving wall-clock. macOS already ships a scalable allocator
# (libmalloc/nano-zone), so only Linux needs this. Honor an externally-set
# WASM_OPT_PRELOAD; otherwise auto-detect a system jemalloc/mimalloc.
#
# WASM_OPT_PRELOAD=none (or 0) forces NO preload — a clean glibc baseline for
# benchmarking the allocator A/B (see scripts/bench/).
if [[ "${WASM_OPT_PRELOAD:-}" == "none" || "${WASM_OPT_PRELOAD:-}" == "0" ]]; then
WASM_OPT_PRELOAD=""
_PRELOAD_FORCED_OFF=1
# --- Binaryen tool: ONE binaryen everywhere (incl. docker/CI) — the submodule fork. It IS Binaryen
# version_130 (its Asyncify.cpp is unmodified upstream) + our HoistCppCatches pass, so the SAME binary
# does --hoist-cpp-catches AND --asyncify/-O2, for both native-EH and legacy JS-EH wasm. Built once via
# build-wasm-opt.sh; no separate binaryen downloads (the emsdk-bundled v121 can't even asyncify wasm-EH).
SUBMODULE_WASMOPT="${HOIST_WASMOPT:-$("${SCRIPT_DIR}/../binaryen-hoist-pass/build-wasm-opt.sh")}"
WASM_OPT="${V130_WASMOPT:-$SUBMODULE_WASMOPT}"
[ "$DO_HOIST" = 1 ] && HOIST_OPT="$SUBMODULE_WASMOPT"
# native wasm-EH needs -all so binaryen parses the EH instructions; HOIST_KEEP_NAMES keeps the
# names section through -O2 for callstack debugging. KiCad/JS-EH uses neither (matches old behavior).
FEAT=()
[ "$DO_HOIST" = 1 ] && FEAT=(-all)
G="${HOIST_KEEP_NAMES:+-g}"
# The asyncify removelist matches functions by NAME. wasm-opt strips the names section by default, so
# the hoist pass (run before asyncify) would otherwise hand asyncify nameless functions — the
# removelist then matches NOTHING and the giant try-dense functions it is meant to exclude
# (BuildBitmapInfo: ~4986 native tries, etc.) get instrumented, which is the dominant driver of the
# multi-GB asyncify RAM blowup on native-EH. So force the hoist pass to keep names whenever a
# removelist is in play, so asyncify can see + exclude them. asyncify/-O2 keep their own G, so the
# FINAL wasm is unchanged unless HOIST_KEEP_NAMES is set (asyncify matches on its INPUT names, which
# the hoist output now carries).
HOIST_G="${G}"
[ "$USE_REMOVELIST" = 1 ] && HOIST_G="-g"
# --- the import boundary (shared) + the KiCad remove-list (opt-out), read from sibling files ---
_join_list() { grep -vE '^[[:space:]]*#|^[[:space:]]*$' "$1" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | tr '\n' ',' | sed 's/,$//'; }
ASYNCIFY_IMPORTS="${ASYNCIFY_IMPORTS_PASS:-$(_join_list "${SCRIPT_DIR}/asyncify-imports.txt")}"
REMOVE_ARG=()
if [ "$USE_REMOVELIST" = 1 ]; then
REMOVE_ARG=("--pass-arg=asyncify-removelist@$(_join_list "${SCRIPT_DIR}/asyncify-removelist.txt")")
fi
# --- memory machinery (identical to before; matters for the host-side KiCad pass) ---
# Bound Binaryen's host thread pool: peak RAM scales with thread count.
export BINARYEN_CORES="${BINARYEN_CORES:-8}"
# Preload a scalable allocator on Linux — glibc malloc collapses into futex lock-spin under
# wasm-opt's many-thread allocation churn; jemalloc/mimalloc roughly halve wall-clock. macOS
# already ships a scalable allocator. WASM_OPT_PRELOAD=none|0 forces a clean glibc baseline.
if [[ "${WASM_OPT_PRELOAD:-}" == "none" || "${WASM_OPT_PRELOAD:-}" == "0" ]]; then
WASM_OPT_PRELOAD=""; _PRELOAD_FORCED_OFF=1
fi
if [[ -z "${WASM_OPT_PRELOAD:-}" && -z "${_PRELOAD_FORCED_OFF:-}" && "$(uname -s)" == "Linux" ]]; then
for _alloc in \
"/usr/lib/$(uname -m)-linux-gnu/libjemalloc.so.2" \
"/usr/lib/$(uname -m)-linux-gnu/libmimalloc.so.2" \
/usr/lib/libjemalloc.so.2 \
/usr/lib/libmimalloc.so.2; do
if [[ -e "${_alloc}" ]]; then
WASM_OPT_PRELOAD="${_alloc}"
break
fi
/usr/lib/libjemalloc.so.2 /usr/lib/libmimalloc.so.2; do
[[ -e "${_alloc}" ]] && { WASM_OPT_PRELOAD="${_alloc}"; break; }
done
fi
# Build the command prefix that injects the allocator (preserving any existing
# LD_PRELOAD). Empty when no scalable allocator was found — wasm-opt then runs
# under the default allocator, just slower.
if [[ -n "${WASM_OPT_PRELOAD:-}" ]]; then
PRELOAD_CMD=(env "LD_PRELOAD=${WASM_OPT_PRELOAD}${LD_PRELOAD:+:${LD_PRELOAD}}")
else
PRELOAD_CMD=()
fi
# GNU `time -v` on Linux CI records peak RSS + wall-clock per pass; macOS `time` lacks -v.
if /usr/bin/time -v true >/dev/null 2>&1; then TIME_CMD=(/usr/bin/time -v); else TIME_CMD=(); fi
# Wrap wasm-opt in GNU `time -v` when available (Linux CI) so the log records
# peak RSS + wall-clock for each pass. macOS `time` lacks -v, so fall back to
# running wasm-opt directly there.
if /usr/bin/time -v true >/dev/null 2>&1; then
TIME_CMD=(/usr/bin/time -v)
else
TIME_CMD=()
fi
INPUT_WASM="${1:-output/pcbnew.wasm}"
OUTPUT_WASM="${2:-${INPUT_WASM}}"
if [ ! -f "${INPUT_WASM}" ]; then
echo "ERROR: Input file not found: ${INPUT_WASM}"
exit 1
fi
echo "Applying asyncify transformation..."
echo "Applying Asyncify${DO_HOIST:+ (+hoist-cpp-catches)}..."
echo " Input: ${INPUT_WASM}"
echo " Output: ${OUTPUT_WASM}"
echo " Tool: ${WASM_OPT}"
echo " asyncify wasm-opt: ${WASM_OPT}"
[ "$DO_HOIST" = 1 ] && echo " hoist wasm-opt: ${HOIST_OPT}"
echo " BINARYEN_CORES=${BINARYEN_CORES} LD_PRELOAD=${WASM_OPT_PRELOAD:-<none>}"
# Asyncify import patterns (functions that trigger async suspension)
# - env.invoke_* : Exception handling trampolines
# - env.__asyncjs__* : EM_ASYNC_JS functions (like startModal())
ASYNCIFY_IMPORTS="env.invoke_*,env.__asyncjs__*,env.emscripten_fiber_swap"
SRC="${INPUT_WASM}"
# Functions to exclude from asyncify instrumentation
# These are large functions that inflate beyond V8's local-count limits.
ASYNCIFY_REMOVE=$(cat << 'REMOVELIST'
COLOR_SETTINGS::COLOR_SETTINGS(wxString const&, bool)
BuildBitmapInfo(std::__2::unordered_map<BITMAPS, std::__2::vector<BITMAP_INFO, std::__2::allocator<BITMAP_INFO>>, std::__2::hash<BITMAPS>, std::__2::equal_to<BITMAPS>, std::__2::allocator<std::__2::pair<BITMAPS const, std::__2::vector<BITMAP_INFO, std::__2::allocator<BITMAP_INFO>>>>>&)
match
DIALOG_PAD_PROPERTIES_BASE::DIALOG_PAD_PROPERTIES_BASE(wxWindow*, int, wxString const&, wxPoint const&, wxSize const&, long)
buildKicadAboutBanner(EDA_BASE_FRAME*, ABOUT_APP_INFO&)
IGESToBRep_CurveAndSurface::TransferGeometry(opencascade::handle<IGESData_IGESEntity> const&, Message_ProgressRange const&)
StepAP214_Protocol::StepAP214_Protocol()
BRepCheck_ParallelAnalyzer::operator()(int) const
ShapeFix_Wire::FixGap3d(int, bool)
ShapeFix_Wire::FixGap2d(int, bool)
PCB_EDIT_FRAME::setupUIConditions()
REMOVELIST
)
ASYNCIFY_REMOVE_ARG=$(echo "${ASYNCIFY_REMOVE}" | tr '\n' ',' | sed 's/,$//')
echo ""
echo "Running wasm-opt --asyncify..."
echo "This may take several minutes and use significant RAM..."
echo " BINARYEN_CORES=${BINARYEN_CORES}"
echo " LD_PRELOAD=${WASM_OPT_PRELOAD:-<none>}"
"${PRELOAD_CMD[@]}" "${TIME_CMD[@]}" "${WASM_OPT}" --asyncify ${ASYNCIFY_EXTRA_OPTS:-} \
"--pass-arg=asyncify-imports@${ASYNCIFY_IMPORTS}" \
"--pass-arg=asyncify-removelist@${ASYNCIFY_REMOVE_ARG}" \
--pass-arg=asyncify-propagate-addlist \
"${INPUT_WASM}" -o "${OUTPUT_WASM}"
# ASYNCIFY_ONLY=1 stops after the asyncify pass (skips -O2). Used by the
# benchmark harness (scripts/bench/) to time/compare just the asyncify pass,
# whose RAM fits where the -O2 pass on the bloated module would not.
if [[ "${ASYNCIFY_ONLY:-0}" == "1" ]]; then
echo ""
echo "ASYNCIFY_ONLY=1 → skipping -O2 pass (benchmark mode)."
ls -lh "${OUTPUT_WASM}"
exit 0
# 1. (native wasm-EH only) hoist C++ catch arms so Asyncify can suspend from inside them.
if [ "$DO_HOIST" = 1 ]; then
echo "Running --hoist-cpp-catches${HOIST_G:+ (keeping names for removelist matching)}..."
"${PRELOAD_CMD[@]}" "${TIME_CMD[@]}" "${HOIST_OPT}" --hoist-cpp-catches "${FEAT[@]}" ${HOIST_G} "${SRC}" -o "${OUTPUT_WASM}"
SRC="${OUTPUT_WASM}"
fi
# Post-asyncify shrink pass. Asyncify emits deliberately verbose instrumentation
# (spills every live local) and relies on the optimizer to coalesce it back down
# under V8's per-function locals limit — without it, large coroutine-entry
# functions silently stall (and on newer Chromium hard-crash the renderer while
# loading a heavy board) in Chrome's V8. See docs/debugging/DEBUG.md §6-7.
#
# Level is configurable via BINARYEN_OPT_LEVEL: -O1 already runs CoalesceLocals
# and is sufficient (measured: same ~187 MB / 64 MB-gzip output as -O2 on pcbnew,
# validated green on the load-pcb chromium-ci spec — but a lighter, faster pass,
# which is why CI sets -O1). -O2 stays the default for the more thorough rewrite.
# Do NOT use -Os/-Oz here: they break the asyncify runtime (Binaryen #4484).
# 2. The real Asyncify transform. ASYNCIFY_EXTRA_OPTS: optional extra wasm-opt flags (origin/main hook).
echo "Running wasm-opt --asyncify (several minutes + significant RAM)..."
"${PRELOAD_CMD[@]}" "${TIME_CMD[@]}" "${WASM_OPT}" --asyncify ${ASYNCIFY_EXTRA_OPTS:-} "${FEAT[@]}" ${G} \
"--pass-arg=asyncify-imports@${ASYNCIFY_IMPORTS}" \
"${REMOVE_ARG[@]}" \
--pass-arg=asyncify-propagate-addlist \
"${SRC}" -o "${OUTPUT_WASM}"
# ASYNCIFY_ONLY=1 stops before -O2 (benchmark harness in scripts/bench/ times just the transform).
if [[ "${ASYNCIFY_ONLY:-0}" == "1" ]]; then
echo "ASYNCIFY_ONLY=1 → skipping -O2 (benchmark mode)."; ls -lh "${OUTPUT_WASM}"; exit 0
fi
# 3. Post-asyncify shrink. Asyncify spills every live local; without coalescing, large
# coroutine-entry functions exceed V8's per-function locals limit and stall/crash the renderer.
# -O1 already runs CoalesceLocals and suffices (CI uses it); -O2 is the default. NOT -Os/-Oz
# (they break the asyncify runtime — Binaryen #4484). See docs/debugging/DEBUG.md §6-7.
BINARYEN_OPT_LEVEL="${BINARYEN_OPT_LEVEL:--O2}"
echo "Running wasm-opt ${BINARYEN_OPT_LEVEL} (shrink instrumented functions under V8's local limit)..."
"${PRELOAD_CMD[@]}" "${TIME_CMD[@]}" "${WASM_OPT}" "${BINARYEN_OPT_LEVEL}" "${FEAT[@]}" ${G} "${OUTPUT_WASM}" -o "${OUTPUT_WASM}"
echo ""
echo "Running wasm-opt ${BINARYEN_OPT_LEVEL} on the asyncified wasm..."
echo " Purpose: shrink asyncify-instrumented functions back under V8's"
echo " per-function locals limit (otherwise large coroutine-entry and"
echo " similar functions stall/crash in Chrome's V8). See docs/debugging/DEBUG.md §6-7."
echo " This pass takes several minutes and ~10-15 GB RAM."
echo " BINARYEN_CORES=${BINARYEN_CORES}"
echo " LD_PRELOAD=${WASM_OPT_PRELOAD:-<none>}"
"${PRELOAD_CMD[@]}" "${TIME_CMD[@]}" "${WASM_OPT}" "${BINARYEN_OPT_LEVEL}" "${OUTPUT_WASM}" -o "${OUTPUT_WASM}"
echo ""
echo "Asyncify + ${BINARYEN_OPT_LEVEL} complete: ${OUTPUT_WASM}"
ls -lh "${OUTPUT_WASM}"

16
scripts/common/apply-finalize.sh
View file

@ -1,6 +1,5 @@
#!/bin/bash
# Apply wasm-emscripten-finalize transformation on host
# Uses the same Binaryen v121 as wasm-opt to ensure version consistency
# Apply wasm-emscripten-finalize transformation on host.
#
# Usage: ./scripts/common/apply-finalize.sh <input.wasm> <output.wasm>
@ -9,13 +8,12 @@ set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
# Get wasm-opt path (this downloads Binaryen if needed)
WASM_OPT=$("${SCRIPT_DIR}/get-wasm-opt.sh")
BINARYEN_BIN=$(dirname "${WASM_OPT}")
FINALIZE="${BINARYEN_BIN}/wasm-emscripten-finalize"
# Same stub hazard as wasm-opt (see get-wasm-opt.sh): a host-mode kicad build
# leaves the emsdk finalize stubbed with the real binary at .real — prefer it,
# the stub exits 0 having done nothing.
# wasm-emscripten-finalize ships with emscripten (it was removed from Binaryen ~v116), so use the
# emsdk's own — the one emscripten would run in-link — directly. No binaryen download. A host-mode
# kicad/test build stubs the emsdk finalize and keeps the real binary at .real; prefer it (the stub
# exits 0 having done nothing).
EMSDK_DIR="${EMSDK:-${PROJECT_ROOT}/tools/emsdk}"
FINALIZE="${EMSDK_DIR}/upstream/bin/wasm-emscripten-finalize"
if [ -x "${FINALIZE}.real" ]; then
FINALIZE="${FINALIZE}.real"
fi

21
scripts/common/asyncify-imports.txt Normal file
View file

@ -0,0 +1,21 @@
# Suspending imports for the post-link Asyncify pass (consumed by apply-asyncify.sh).
# Binaryen's --asyncify instruments every function that can transitively REACH one of these
# imports. This is the BOUNDARY list, not a function allowlist — the callers are auto-discovered.
#
# We run Asyncify post-link (outside emcc), so emcc's automatic import generation never runs;
# this file replicates emcc's default async built-ins + our own suspending imports. A superset
# is safe — binaryen ignores any import the wasm doesn't actually contain.
#
# --- Emscripten async built-ins (what emcc auto-adds for the in-link Asyncify) ---
env.emscripten_sleep
env.emscripten_scan_registers
env.emscripten_lazy_load_code
env.emscripten_wget
env.emscripten_wget_data
env.emscripten_idb_*
#
# --- Project suspending imports ---
env.__asyncjs__*
env.emscripten_fiber_swap
env.startModal
env.js_*

26
scripts/common/asyncify-removelist.txt Normal file
View file

@ -0,0 +1,26 @@
# Functions EXCLUDED from Asyncify instrumentation (consumed by apply-asyncify.sh unless
# --no-removelist). These are large, NON-suspending KiCad/OpenCASCADE functions (generated resource
# tables, wxFormBuilder UI constructors, OCC geometry) that are dense with native wasm try/catch.
# Asyncify's per-function cost is superlinear in try-count (it builds a CFG + liveness over every
# try), so instrumenting these few giants is what drives the multi-GB RAM blowup of `wasm-opt
# --asyncify` on the native-EH build. They never call a suspending import, so excluding them is safe.
#
# MATCHING: Binaryen escapes each entry, then treats entries containing `*` as wildcard patterns
# (String::wildcardMatch) and entries without `*` as exact escaped function names. Full demangled
# signatures don't exact-match (subtle spacing) and a `*` inside a pointer type (e.g. wxWindow*) turns
# the whole entry into a pattern that still misses — so use a PREFIX wildcard per symbol, which matches
# the function's escaped demangled name regardless of argument formatting. Patterns that match nothing
# in a given app (e.g. the OCC entries when 3D is off) just emit a harmless "non-matching" warning.
# Only the KiCad build passes the remove-list; the small wx test apps opt out via --no-removelist
# (and the bare name "match" could collide there).
COLOR_SETTINGS::COLOR_SETTINGS*
BuildBitmapInfo*
match
DIALOG_PAD_PROPERTIES_BASE::DIALOG_PAD_PROPERTIES_BASE*
buildKicadAboutBanner*
IGESToBRep_CurveAndSurface::TransferGeometry*
StepAP214_Protocol::StepAP214_Protocol*
BRepCheck_ParallelAnalyzer::operator*
ShapeFix_Wire::FixGap3d*
ShapeFix_Wire::FixGap2d*
PCB_EDIT_FRAME::setupUIConditions*

9
scripts/common/env.sh
View file

@ -36,6 +36,15 @@ export WX_BUILD="$BUILD_ROOT/wxwidgets"
# Emscripten settings
export EMSDK_QUIET=1
# Exception model for ALL WASM translation units — the C/C++ dependencies, wxWidgets, and KiCad
# (build-wx-wasm.sh and build-kicad-target.sh source this file and reuse DEPS_EH_FLAGS). Native
# WebAssembly exceptions (legacy binary encoding) are the only build mode. -sSUPPORT_LONGJMP=wasm is
# required because the deps that use setjmp/longjmp (freetype, cairo, OpenCASCADE) must use wasm
# setjmp — emscripten's JS-longjmp implementation cannot coexist with -fwasm-exceptions (it would
# leave emscripten_longjmp undefined). -sWASM_LEGACY_EXCEPTIONS=1 selects the EH binary encoding our
# post-link Asyncify + catch-arm-hoisting pass can consume (Asyncify can't handle exnref).
export DEPS_EH_FLAGS="-fwasm-exceptions -sSUPPORT_LONGJMP=wasm -sWASM_LEGACY_EXCEPTIONS=1"
# Emscripten SDK setup
# If EMSDK is already set (e.g. Docker entrypoint sourced emsdk_env.sh), use it.
# Otherwise auto-install a local copy under tools/emsdk/.

124
scripts/common/inject-dyncall-shims.sh
View file

@ -4,17 +4,15 @@
#
# The actual JavaScript that gets injected lives in readable, standalone files in
# scripts/common/shims/ (not inline heredocs):
# - dyncall-binding.js.tmpl per-signature dynCall_<sig> binding (templated)
# - handlesleep.js nested-Asyncify handleSleep currData save/restore (#9153)
# - diagnostics.js optional logging-only instrumentation (see SHIM_DIAGNOSTICS)
#
# Why bind dynCall_* to the real wasm exports: the build links -sDYNCALLS=1, so
# asyncify-INSTRUMENTED dynCall_* trampolines exist as wasm exports. The bare
# dynCall_<sig>(index, ...) call sites (invoke_* wrappers, the empty-callback fixes
# below) aren't bound to top-level names, so we bind them to wasmExports["dynCall_<sig>"].
# Binding to getWasmTableEntry() instead bypasses the instrumentation and breaks
# unwind/rewind through indirect calls ("indirect call signature mismatch" — caught
# every frame in Firefox; a hard renderer crash in Chrome/V8).
# Native wasm-EH is the only build mode, so the .js has no invoke_* wrappers / dynCall_<sig> call
# sites to bind. The build still links -sDYNCALLS=1, so asyncify-INSTRUMENTED dynCall_* trampolines
# exist as wasm EXPORTS; the empty-callback fixes below route function-pointer stubs through
# wasmExports["dynCall_<sig>"]. This MUST be the wasm trampoline, NOT getWasmTableEntry — the latter
# bypasses the instrumentation and breaks unwind/rewind through indirect calls ("indirect call
# signature mismatch" — caught every frame in Firefox; a hard renderer crash in Chrome/V8).
#
# Usage:
# inject-dyncall-shims.sh <pcbnew.js>
@ -34,72 +32,23 @@ if [ -z "$JS_FILE" ] || [ ! -f "$JS_FILE" ]; then
echo "Usage: $0 <path/to/pcbnew.js>"
exit 1
fi
for f in dyncall-binding.js.tmpl handlesleep.js diagnostics.js; do
for f in handlesleep.js diagnostics.js; do
if [ ! -f "$SHIM_DIR/$f" ]; then
echo "Error: missing shim source $SHIM_DIR/$f"
exit 1
fi
done
# --- 1. dynCall_<sig> bindings -------------------------------------------------
echo "Extracting dynCall signatures from $JS_FILE..."
SIGNATURES=$(grep -oE 'dynCall_[a-zA-Z0-9]+' "$JS_FILE" | sort -u | sed 's/dynCall_//')
if [ -z "$SIGNATURES" ]; then
echo "No dynCall signatures found - nothing to inject"
exit 0
fi
SIG_COUNT=$(echo "$SIGNATURES" | wc -l | tr -d ' ')
echo "Found $SIG_COUNT unique signatures"
# The template body is everything from the `function` line onward (skip its comments).
TEMPLATE_BODY=$(sed -n '/^function /,$p' "$SHIM_DIR/dyncall-binding.js.tmpl")
SHIM_FILE=$(mktemp)
{
echo ""
echo "// === dynCall bindings (bind bare names to the real DYNCALLS=1 wasm exports) ==="
} > "$SHIM_FILE"
for sig in $SIGNATURES; do
argcount=$((${#sig} - 1))
args="index"
call_args=""
for ((i=0; i<argcount; i++)); do
args="$args, a$i"
if [ $i -gt 0 ]; then call_args="$call_args, "; fi
call_args="${call_args}a$i"
done
echo "$TEMPLATE_BODY" \
| sed -e "s/@SIG@/$sig/g" -e "s/@ARGS@/$args/g" -e "s/@CALLARGS@/$call_args/g" \
>> "$SHIM_FILE"
done
echo "// === End dynCall bindings ===" >> "$SHIM_FILE"
# Insert right after the getWasmTableEntry definition.
GWTL_LINE=$(grep -n '^var getWasmTableEntry = funcPtr => {' "$JS_FILE" | head -1 | cut -d: -f1)
if [ -z "$GWTL_LINE" ]; then
GWTL_LINE=$(grep -n 'var getWasmTableEntry' "$JS_FILE" | head -1 | cut -d: -f1)
fi
if [ -z "$GWTL_LINE" ]; then
echo "Error: Could not find getWasmTableEntry in $JS_FILE"; rm "$SHIM_FILE"; exit 1
fi
INSERT_LINE=""
for ((i=GWTL_LINE; i<=GWTL_LINE+10; i++)); do
[ "$(sed -n "${i}p" "$JS_FILE")" == "};" ] && { INSERT_LINE=$i; break; }
done
[ -z "$INSERT_LINE" ] && INSERT_LINE=$GWTL_LINE
echo "Injecting dynCall bindings after line $INSERT_LINE..."
head -n "$INSERT_LINE" "$JS_FILE" > "${JS_FILE}.tmp"
cat "$SHIM_FILE" >> "${JS_FILE}.tmp"
tail -n +$((INSERT_LINE + 1)) "$JS_FILE" >> "${JS_FILE}.tmp"
mv "${JS_FILE}.tmp" "$JS_FILE"
rm "$SHIM_FILE"
echo "Injected $SIG_COUNT dynCall bindings"
# --- 2. Empty-callback fixes ---------------------------------------------------
# Emscripten+pthreads emits some direct-call paths as no-op ((a1)=>{}) stubs that
# ARE used. Rewire each to the (now-bound) dynCall_*. (Kept inline: simple sed one-liners.)
# --- 1. Empty-callback fixes ---------------------------------------------------
# Emscripten+pthreads emits some direct-call paths as no-op ((a1)=>{}) stubs that ARE used. Native
# wasm-EH eliminates the invoke_* wrappers, so the .js has no dynCall_<sig> call sites to bind — but
# the DYNCALLS=1 trampolines are still EXPORTED on the wasm, so route each function-pointer stub
# through wasmExports["dynCall_<sig>"]. This MUST be the wasm trampoline, NOT getWasmTableEntry: the
# fiber entry runs a coroutine that suspends+rewinds via Asyncify, and an Asyncify rewind cannot
# resume through getWasmTableEntry's JS wrapper — the fiber would re-enter from the top and the tool's
# Wait() re-runs (tool_manager ScheduleWait "!pendingWait" assert + busy-loop). The instrumented
# dynCall_<sig> export rewinds correctly. (Without these fixes the libcontext fiber entry stays the
# empty (a1=>{}) stub, so tool coroutines never start and every GAL app stalls at InvokeTool.)
echo "Fixing empty callback arrow functions..."
TOTAL_FIXED=0
apply_fix() { # <grep/sed pattern> <sed replacement> <label>
@ -112,12 +61,12 @@ apply_fix() { # <grep/sed pattern> <sed replacement> <label>
TOTAL_FIXED=$((TOTAL_FIXED + before - after))
fi
}
apply_fix '((a1, a2, a3) => {})(eventTypeId,' '((a1, a2, a3) => dynCall_iiii(callbackfunc, a1, a2, a3))(eventTypeId,' "HTML5 event callback(s) (dynCall_iiii)"
apply_fix 'var result = (a1 => {})(arg);' 'var result = dynCall_ii(ptr, arg);' "pthread entry callback(s) (dynCall_ii)"
apply_fix 'return (a1 => {})(sig);' 'return dynCall_vi(fp, sig);' "signal handler callback(s) (dynCall_vi)"
apply_fix 'var wrapper = () => (a1 => {})(arg);' 'var wrapper = () => dynCall_vi(func, arg);' "async timer callback(s) (dynCall_vi)"
apply_fix 'var iterFunc = (() => {});' 'var iterFunc = () => dynCall_v(func);' "main loop callback(s) (dynCall_v)"
apply_fix '(a1 => {})(userData);' 'dynCall_vi(entryPoint, userData);' "fiber entry callback(s) (dynCall_vi)"
apply_fix '((a1, a2, a3) => {})(eventTypeId,' '((a1, a2, a3) => wasmExports["dynCall_iiii"](callbackfunc, a1, a2, a3))(eventTypeId,' "HTML5 event callback(s) (wasmExports.dynCall_iiii)"
apply_fix 'var result = (a1 => {})(arg);' 'var result = wasmExports["dynCall_ii"](ptr, arg);' "pthread entry callback(s) (wasmExports.dynCall_ii)"
apply_fix 'return (a1 => {})(sig);' 'return wasmExports["dynCall_vi"](fp, sig);' "signal handler callback(s) (wasmExports.dynCall_vi)"
apply_fix 'var wrapper = () => (a1 => {})(arg);' 'var wrapper = () => wasmExports["dynCall_vi"](func, arg);' "async timer callback(s) (wasmExports.dynCall_vi)"
apply_fix 'var iterFunc = (() => {});' 'var iterFunc = () => wasmExports["dynCall_v"](func);' "main loop callback(s) (wasmExports.dynCall_v)"
apply_fix '(a1 => {})(userData);' 'wasmExports["dynCall_vi"](entryPoint, userData);' "fiber entry callback(s) (wasmExports.dynCall_vi)"
echo "Total: Fixed $TOTAL_FIXED empty callback(s)"
# --- 3. Nested-Asyncify handleSleep fix ---------------------------------------
@ -131,7 +80,15 @@ elif grep -q '__nestedHandleSleepInstalled' "$JS_FILE"; then
else
HS_MARKER=$(grep -n '^_emscripten_fiber_swap\.isAsync = true;$' "$JS_FILE" | head -1 | cut -d: -f1)
if [ -z "$HS_MARKER" ]; then
echo "Warning: _emscripten_fiber_swap.isAsync marker not found - skipping handleSleep fix"
# No libcontext fiber glue (a non-fiber Asyncify app — e.g. a plain wx app with
# modals/menus, no tool coroutines). The currData save/restore is still needed:
# without it a rewind resuming through a fresh wasm re-entry hits
# _asyncify_start_rewind(null) -> "memory access out of bounds" (the context-menu
# pick while the main loop is Asyncify-parked). Append at EOF — Asyncify is defined
# by then and the shim wraps handleSleep at load, before any runtime sleep.
echo "" >> "$JS_FILE"
cat "$SHIM_DIR/handlesleep.js" >> "$JS_FILE"
echo "Injected handleSleep fix at EOF (no fiber glue)"
else
head -n "$HS_MARKER" "$JS_FILE" > "${JS_FILE}.tmp"
echo "" >> "${JS_FILE}.tmp"
@ -158,6 +115,23 @@ else
echo "Warning: dynCallLegacy pattern not found - skipping embind dynCall fallback"
fi
# --- 3d. Embind invoker: don't Promise-wrap a SYNCHRONOUS call when the main loop is parked --------
# Emscripten's embind invoker returns a Promise iff Asyncify.currData is set AFTER the wasm call. But
# the native-EH per-frame-yield main loop parks via Asyncify (currData stays SET between frames), so a
# JS-initiated embind call (e.g. kicadCollabSnapshot from a test or the UI) that does NOT itself
# suspend is mis-detected as async and returns "[object Promise]" instead of the value -> the caller's
# JSON.parse(...) gets "[object Promise]". Capture currData before the call and only treat it as async
# if THIS call left a NEW currData. Harmless under legacy/JS-EH (currData is null when the app is idle).
if grep -q 'Asyncify.currData !== __ehPrev' "$JS_FILE"; then
echo "embind invoker currData re-entrancy fix already present - skipping"
elif grep -q 'return Asyncify.currData ? Asyncify.whenDone' "$JS_FILE"; then
perl -0pi -e 's/(invokerFnBody \+= \(returns \|\| isAsync \? "var rv = " : ""\))/invokerFnBody += "var __ehPrev = Asyncify.currData;\\n";\n $1/' "$JS_FILE"
perl -0pi -e 's/return Asyncify\.currData \? Asyncify\.whenDone/return (Asyncify.currData && Asyncify.currData !== __ehPrev) ? Asyncify.whenDone/' "$JS_FILE"
echo "Injected embind invoker currData re-entrancy fix"
else
echo "Warning: embind invoker currData pattern not found - skipping embind re-entrancy fix"
fi
# --- 3c. Fiber trampoline self-heal -------------------------------------------
# emscripten_set_main_loop(...,1) throws "unwind" during startup to establish the
# main loop. KiCad establishes that loop from inside a tool coroutine, so the throw

25
scripts/common/shims/dyncall-binding.js.tmpl
View file

@ -1,25 +0,0 @@
// Template for one dynCall_<sig> binding. Placeholders substituted per signature
// by inject-dyncall-shims.sh: @SIG@ = signature, @ARGS@ = "index, a0, a1, ...",
// @CALLARGS@ = "a0, a1, ..." (the args without the function-pointer index).
//
// Binds the bare name to the REAL asyncify-instrumented wasm export
// (wasmExports["dynCall_<sig>"], present because the build links -sDYNCALLS=1).
// Falls back to getWasmTableEntry if no such export exists, OR if the instrumented
// trampoline traps with "indirect call signature mismatch": post-asyncify+O2 the
// trampoline can call_indirect with a stale type for some table indices even though
// the table entry itself is valid (hit by programmatic editor edits, e.g. eeschema
// SCH_ITEM::Move via invoke_vii — see features/yjs-bridge/0003). The direct
// getWasmTableEntry call uses the correct per-entry signature and succeeds. Only the
// mismatch trap is caught; the Asyncify unwind sentinel and real exceptions re-throw,
// so instrumentation/unwind still work for every normal indirect call.
function dynCall_@SIG@(@ARGS@) {
var f = (typeof wasmExports !== 'undefined') && wasmExports["dynCall_@SIG@"];
if (f) {
try { return f(@ARGS@); }
catch (_dce) {
if (!(_dce instanceof WebAssembly.RuntimeError) || !/signature mismatch/.test(_dce.message))
throw _dce;
}
}
return getWasmTableEntry(index)(@CALLARGS@);
}

15
scripts/deps/build-cairo.sh
View file

@ -53,6 +53,12 @@ fi
log_info "Building Cairo ${CAIRO_VERSION} for WASM..."
# Always start meson fresh on a (re)build: meson caches its configuration, so a `meson setup` on an
# existing build dir IGNORES a regenerated cross-file.txt — which silently dropped the EH/longjmp
# flags (DEPS_EH_FLAGS) when switching JS-EH -> native-EH and left libcairo.a referencing
# emscripten_longjmp. Wiping here (we only reach this past the stamp check, i.e. on a real rebuild)
# forces meson to re-read the cross-file. Cairo is small, so the full reconfigure is cheap.
rm -rf "${CAIRO_BUILD}"
mkdir -p "${CAIRO_BUILD}"
cd "${CAIRO_BUILD}"
@ -65,6 +71,11 @@ else
MESON_DEBUG_FLAGS="'-O2'"
fi
# Exception-model flags (DEPS_EH_FLAGS from env.sh) as meson list elements, e.g.
# ", '-fwasm-exceptions', '-sSUPPORT_LONGJMP=wasm', '-sWASM_LEGACY_EXCEPTIONS=1'". Empty for legacy.
MESON_EH_FLAGS=""
for _ehf in ${DEPS_EH_FLAGS}; do MESON_EH_FLAGS="${MESON_EH_FLAGS}, '${_ehf}'"; done
# Cairo uses meson
cat > cross-file.txt << EOF
[binaries]
@ -94,8 +105,8 @@ b_pie = false
# to prevent Cairo from defining its own conflicting implementations
# Include ft2build.h and ftcolor.h to fix FT_Color forward declaration bug in cairo-ft-private.h
# (the forward declaration is inside HAVE_FT_SVG_DOCUMENT but used in HAVE_FT_COLR_V1)
c_args = [${MESON_DEBUG_FLAGS}, '-pthread', '-matomics', '-mbulk-memory', '-I${SYSROOT}/include', '-I${SYSROOT}/include/freetype2', '-I${SYSROOT}/include/pixman-1', '-DHAVE_CTIME_R=1', '-DHAVE_LOCALTIME_R=1', '-DHAVE_GMTIME_R=1', '-DHAVE_STRNDUP=1', '-include', 'ft2build.h', '-include', 'freetype/ftcolor.h']
c_link_args = ['-pthread', '-L${SYSROOT}/lib']
c_args = [${MESON_DEBUG_FLAGS}${MESON_EH_FLAGS}, '-pthread', '-matomics', '-mbulk-memory', '-I${SYSROOT}/include', '-I${SYSROOT}/include/freetype2', '-I${SYSROOT}/include/pixman-1', '-DHAVE_CTIME_R=1', '-DHAVE_LOCALTIME_R=1', '-DHAVE_GMTIME_R=1', '-DHAVE_STRNDUP=1', '-include', 'ft2build.h', '-include', 'freetype/ftcolor.h']
c_link_args = ['-pthread'${MESON_EH_FLAGS}, '-L${SYSROOT}/lib']
pkg_config_path = '${SYSROOT}/lib/pkgconfig'
EOF

4
scripts/deps/build-freetype.sh
View file

@ -56,8 +56,8 @@ emcmake cmake "${FREETYPE_DIR}" \
-DCMAKE_BUILD_TYPE=${BUILD_TYPE:-Debug} \
-DCMAKE_INSTALL_PREFIX="${SYSROOT}" \
-DCMAKE_POLICY_VERSION_MINIMUM=3.5 \
-DCMAKE_C_FLAGS="${DEBUG_CFLAGS:--g -O0} -pthread -matomics -mbulk-memory" \
-DCMAKE_CXX_FLAGS="${DEBUG_CFLAGS:--g -O0} -pthread -matomics -mbulk-memory" \
-DCMAKE_C_FLAGS="${DEBUG_CFLAGS:--g -O0} -pthread -matomics -mbulk-memory ${DEPS_EH_FLAGS}" \
-DCMAKE_CXX_FLAGS="${DEBUG_CFLAGS:--g -O0} -pthread -matomics -mbulk-memory ${DEPS_EH_FLAGS}" \
-DFT_DISABLE_BZIP2=ON \
-DFT_DISABLE_BROTLI=ON \
-DFT_DISABLE_HARFBUZZ=ON \

19
scripts/deps/build-opencascade.sh
View file

@ -59,12 +59,27 @@ cd "${OCC_BUILD}"
# OpenCASCADE build configuration for WASM
# Disable GUI, visualization that needs X11/OpenGL native
# Enable core geometry and data exchange modules only
#
# OCC's CMake unconditionally adds -DOCC_CONVERT_SIGNALS (occt_defs_flags.cmake), which turns the
# OCC_CATCH_SIGNALS macro into setjmp(handler.Label()). The STEP read/write code
# (STEPControl_Reader/ActorRead, ...) uses OCC_CATCH_SIGNALS pervasively. Under -fwasm-exceptions that
# setjmp is lowered (emscripten's LowerEmscriptenEHSjLj) into a wasm-SjLj state-machine br_table whose
# branch targets are inconsistently typed -> INVALID wasm that V8, wabt AND Binaryen all reject (this
# is the "popping from empty stack" / br_table type-mismatch that blocks pcbnew's OCC link). WASM has
# no POSIX signals, so OCC_CONVERT_SIGNALS (signal->exception conversion) is meaningless here anyway;
# OCC's normal C++ Standard_Failure throw/catch is unaffected. Disabling it means OCC_CATCH_SIGNALS
# expands to nothing (clang) -> no setjmp -> no wasm-SjLj -> valid native-EH wasm.
_occ_defs="${OCC_DIR}/adm/cmake/occt_defs_flags.cmake"
if grep -q '^[[:space:]]*add_definitions(-DOCC_CONVERT_SIGNALS)' "${_occ_defs}" 2>/dev/null; then
sed -i 's|add_definitions(-DOCC_CONVERT_SIGNALS)|# add_definitions(-DOCC_CONVERT_SIGNALS) # disabled for native wasm-EH by build-opencascade.sh (no POSIX signals in WASM; setjmp breaks -fwasm-exceptions)|' "${_occ_defs}"
log_info "Disabled OCC_CONVERT_SIGNALS for native wasm-EH (avoids invalid wasm-SjLj br_table)"
fi
emcmake cmake "${OCC_DIR}" \
-DCMAKE_POLICY_VERSION_MINIMUM=3.5 \
-DCMAKE_BUILD_TYPE=${BUILD_TYPE:-Debug} \
-DCMAKE_INSTALL_PREFIX="${SYSROOT}" \
-DCMAKE_CXX_FLAGS="${DEBUG_CFLAGS:--g -O0} -pthread -matomics -mbulk-memory" \
-DCMAKE_C_FLAGS="${DEBUG_CFLAGS:--g -O0} -pthread -matomics -mbulk-memory" \
-DCMAKE_CXX_FLAGS="${DEBUG_CFLAGS:--g -O0} -pthread -matomics -mbulk-memory ${DEPS_EH_FLAGS}" \
-DCMAKE_C_FLAGS="${DEBUG_CFLAGS:--g -O0} -pthread -matomics -mbulk-memory ${DEPS_EH_FLAGS}" \
-DBUILD_LIBRARY_TYPE=Static \
-DBUILD_MODULE_ApplicationFramework=OFF \
-DBUILD_MODULE_Draw=OFF \

59
scripts/kicad/build-kicad-target.sh
View file

@ -231,26 +231,40 @@ log_info "Building wxWidgets..."
log_info "Building KiCad ${APP_NAME} ${KICAD_VERSION} for WASM..."
# Step 5: Set build type
# Exception model: native WebAssembly exceptions (legacy binary encoding) + wasm setjmp/longjmp,
# single-sourced from scripts/common/env.sh (KiCad and wxWidgets must agree — mixing EH models
# link-fails / traps; both build with exceptions enabled). -matomics -mbulk-memory are required for
# shared memory (pthreads).
KICAD_EH_FLAGS="$DEPS_EH_FLAGS"
log_info "KiCad EH model flags: ${KICAD_EH_FLAGS}"
# Use environment DEBUG_BUILD if set, otherwise check local --debug flag
# -fexceptions is required because wxWidgets is built with exceptions enabled
# -matomics -mbulk-memory are required for shared memory (pthreads)
# NOTE: We use -O1 for debug builds because -O0 produces WASM with too many
# locals for V8/Chrome to compile (error: "local count too large").
# -O1 keeps debug info but optimizes enough to stay under V8's limits.
if [ "${DEBUG_BUILD:-0}" = "1" ] || [ $DEBUG -eq 1 ]; then
BUILD_TYPE="Debug"
EXTRA_FLAGS="-g -O1 -fexceptions -matomics -mbulk-memory"
EXTRA_FLAGS="-g -O1 ${KICAD_EH_FLAGS} -matomics -mbulk-memory"
# CMake defines DEBUG for Config=Debug (kicad/CMakeLists.txt:351). The embind TU (Step 7) is
# compiled OUTSIDE CMake, so it must define DEBUG too — otherwise a DEBUG-gated virtual
# (EDA_ITEM::Show, eda_item.h:471) occupies a vtable slot in the core's emitted vtable that the
# embind TU doesn't account for, shifting every later slot by one. Then every virtual call made
# from the embind TU past that slot (SetWidth/GetPosition/...) reads the wrong vtable offset and
# mis-dispatches at runtime (call_indirect signature-mismatch trap; under native-EH the trap is
# swallowed by the apply coroutine's catch_all → silent hang). See task #54 root-cause analysis.
EMBIND_CONFIG_DEFINES="-DDEBUG"
# -gseparate-dwarf puts debug info in a separate .debug.wasm file
# This keeps the main WASM small (~200MB) while preserving full debug info
# DevTools loads the debug file on-demand when debugging
LINKER_DEBUG_FLAGS="-O1 -g -gseparate-dwarf -fexceptions"
LINKER_DEBUG_FLAGS="-O1 -g -gseparate-dwarf ${KICAD_EH_FLAGS}"
log_info "Building KiCad in DEBUG mode (separate DWARF for smaller main binary)"
else
BUILD_TYPE="Release"
EXTRA_FLAGS="-O2 -fexceptions -matomics -mbulk-memory"
EXTRA_FLAGS="-O2 ${KICAD_EH_FLAGS} -matomics -mbulk-memory"
EMBIND_CONFIG_DEFINES="" # Release defines no DEBUG in either TU → vtable layouts already match
# -O0 at link time skips wasm-opt (which can OOM on large WASM files)
# Compilation is still -O2 for optimized code, but we skip post-link wasm-opt
LINKER_DEBUG_FLAGS="-O0 -fexceptions"
LINKER_DEBUG_FLAGS="-O0 ${KICAD_EH_FLAGS}"
log_info "Building KiCad in RELEASE mode (skipping wasm-opt due to memory limits)"
fi
@ -266,6 +280,15 @@ STUBS_DIR="${PROJECT_ROOT}/wasm/stubs"
STUBS_BUILD="${BUILD_ROOT}/stubs"
mkdir -p "${STUBS_BUILD}"
# ABI-affecting flags shared by EVERY C++ TU compiled OUTSIDE CMake (the embind + the app stubs below).
# The core CMake TUs get all of these (DEBUG via Config=Debug -> kicad/CMakeLists.txt:351;
# KICAD_USE_PLATFORM_WASM; the char16_t char_traits force-include). A TU that misses any of them can
# diverge in vtable layout / ABI from the core — task #54: the embind missing -DDEBUG shifted its vtable
# slot offsets by one and hung the collab apply (call_indirect signature-mismatch). Keep them in ONE
# place so no out-of-CMake C++ TU can skew again. (EMBIND_CONFIG_DEFINES holds the build-config -DDEBUG,
# set in the BUILD_TYPE block above; empty in Release where neither side defines DEBUG.)
KICAD_TU_ABI_FLAGS="${EMBIND_CONFIG_DEFINES} -DKICAD_USE_PLATFORM_WASM=1 -include ${STUBS_DIR}/char_traits_uint16_workaround.h"
kw_stage kicad-stubs
log_info "Building stub libraries..."
# Compile libgit2 stub
@ -293,7 +316,7 @@ APP_STUB_LINK=""
APP_SCRIPTING_STUB_SRC="${STUBS_DIR}/${STUB_APP}_scripting_stub.cpp"
if [ -f "${APP_SCRIPTING_STUB_SRC}" ]; then
log_info "Building app scripting stub: ${STUB_APP}_scripting_stub.cpp"
em++ -c ${WX_CXXFLAGS} "${APP_SCRIPTING_STUB_SRC}" -o "${STUBS_BUILD}/${STUB_APP}_scripting_stub.o"
em++ -c ${KICAD_TU_ABI_FLAGS} ${WX_CXXFLAGS} "${APP_SCRIPTING_STUB_SRC}" -o "${STUBS_BUILD}/${STUB_APP}_scripting_stub.o"
emar rcs "${STUBS_BUILD}/lib${STUB_APP}_scripting_stub.a" "${STUBS_BUILD}/${STUB_APP}_scripting_stub.o"
APP_STUB_LINK="${APP_STUB_LINK} ${STUBS_BUILD}/lib${STUB_APP}_scripting_stub.a"
fi
@ -301,7 +324,7 @@ fi
APP_FRAME_STUB_SRC="${STUBS_DIR}/${STUB_APP}_frame_stub.cpp"
if [ -f "${APP_FRAME_STUB_SRC}" ] && [ -s "${APP_FRAME_STUB_SRC}" ]; then
log_info "Building app frame stub: ${STUB_APP}_frame_stub.cpp"
em++ -c ${WX_CXXFLAGS} "${APP_FRAME_STUB_SRC}" -o "${STUBS_BUILD}/${STUB_APP}_frame_stub.o"
em++ -c ${KICAD_TU_ABI_FLAGS} ${WX_CXXFLAGS} "${APP_FRAME_STUB_SRC}" -o "${STUBS_BUILD}/${STUB_APP}_frame_stub.o"
emar rcs "${STUBS_BUILD}/lib${STUB_APP}_frame_stub.a" "${STUBS_BUILD}/${STUB_APP}_frame_stub.o"
APP_STUB_LINK="${APP_STUB_LINK} ${STUBS_BUILD}/lib${STUB_APP}_frame_stub.a"
fi
@ -387,14 +410,14 @@ if [ "${APP_NAME}" = "sym_convert" ]; then
SYM_CONVERTER_CMAKE_FLAG="-DKICAD_SYM_CONVERTER_WASM=ON"
fi
# 3D viewer (experimental): opt in with BUILD_3D_VIEWER=ON. Default OFF keeps
# existing/CI builds unchanged and the 3D stubs in place. When ON, the 3D viewer
# renders with the GL-free CPU raytracer (RENDER_3D_RAYTRACE_RAM) blitted to the
# canvas through a plain WebGL2 textured quad — no -sLEGACY_GL_EMULATION. KiCad's
# fixed-function OpenGL renderer is still compiled (shared files reference it) but
# never executed on WASM, so its FFP/GLU entry points are satisfied at link time
# by no-op stubs (gl_ffp_stub.c). See docs/features/fork-cleanup/10-3d-viewer.md.
BUILD_3D_VIEWER="${BUILD_3D_VIEWER:-OFF}"
# 3D viewer: built by DEFAULT (BUILD_3D_VIEWER=ON). Opt out with BUILD_3D_VIEWER=OFF, which links the
# 3D stubs instead. The 3D viewer renders with the GL-free CPU raytracer (RENDER_3D_RAYTRACE_RAM)
# blitted to the canvas through a plain WebGL2 textured quad — no -sLEGACY_GL_EMULATION. KiCad's
# fixed-function OpenGL renderer is still compiled (shared files reference it) but never executed on
# WASM, so its FFP/GLU entry points are satisfied at link time by no-op stubs (gl_ffp_stub.c). The
# KiCad CMake option KICAD_BUILD_3D_VIEWER_WASM stays OFF upstream; our build passes it explicitly.
# See docs/features/fork-cleanup/10-3d-viewer.md.
BUILD_3D_VIEWER="${BUILD_3D_VIEWER:-ON}"
GL3D_LINK_FLAGS=""
if [ "${BUILD_3D_VIEWER}" = "ON" ]; then
log_info "3D viewer ENABLED for WASM (BUILD_3D_VIEWER=ON)"
@ -410,7 +433,7 @@ emcmake cmake "${KICAD_DIR}" \
-DCMAKE_MODULE_PATH="${WASM_LAYER}/cmake" \
-DSYSROOT="${SYSROOT}" \
-DCMAKE_POLICY_VERSION_MINIMUM=3.5 \
-DCMAKE_CXX_FLAGS="${EXTRA_FLAGS} -pthread -sUSE_ZLIB=1 -DKICAD_USE_PLATFORM_WASM=1${DIAG_DEFINES} -I${SYSROOT}/include -I${STUBS_DIR} -include ${STUBS_DIR}/char_traits_uint16_workaround.h" \
-DCMAKE_CXX_FLAGS="${EXTRA_FLAGS} -Xclang -fno-pch-timestamp -pthread -sUSE_ZLIB=1 -DKICAD_USE_PLATFORM_WASM=1${DIAG_DEFINES} -I${SYSROOT}/include -I${STUBS_DIR} -include ${STUBS_DIR}/char_traits_uint16_workaround.h" \
-DCMAKE_C_FLAGS="${EXTRA_FLAGS} -pthread -sUSE_ZLIB=1 -I${SYSROOT}/include -I${STUBS_DIR}" \
-DCMAKE_EXE_LINKER_FLAGS="${LINKER_DEBUG_FLAGS} -pthread -sUSE_ZLIB=1 -sASYNCIFY=1 -sDYNCALLS=1 -sASYNCIFY_STACK_SIZE=65536 -sUSE_PTHREADS=1 -sPTHREAD_POOL_SIZE='navigator.hardwareConcurrency' -sPTHREAD_POOL_SIZE_STRICT=0 -sALLOW_MEMORY_GROWTH=1 -sINITIAL_MEMORY=256MB -sMAXIMUM_MEMORY=4GB -sMAX_WEBGL_VERSION=2 ${GL3D_LINK_FLAGS} -sEXPORTED_RUNTIME_METHODS=['ccall','cwrap','UTF8ToString','stringToUTF8','lengthBytesUTF8','dynCall'] -sDEFAULT_LIBRARY_FUNCS_TO_INCLUDE=['\$dynCall'] --bind -L${SYSROOT}/lib ${STUBS_BUILD}/libgit2_stub.a ${STUBS_BUILD}/libcurl_stub.a${APP_STUB_LINK} ${STUBS_BUILD}/libnng_stub.a ${EMBIND_OBJ}" \
-DCMAKE_PREFIX_PATH="${SYSROOT};${WX_BUILD}" \
@ -490,7 +513,7 @@ if [ -f "${EMBIND_SRC}" ]; then
KICAD_INCLUDES+=" -I${KICAD_DIR}/thirdparty/libcontext"
KICAD_INCLUDES+=" -I${SYSROOT}/include"
# KiCad requires C++20 for concepts
em++ -std=c++20 -c ${EXTRA_FLAGS} ${WX_CXXFLAGS} ${KICAD_INCLUDES} "${EMBIND_SRC}" -o "${EMBIND_OBJ}"
em++ -std=c++20 -c ${EXTRA_FLAGS} ${KICAD_TU_ABI_FLAGS} ${WX_CXXFLAGS} ${KICAD_INCLUDES} "${EMBIND_SRC}" -o "${EMBIND_OBJ}"
else
log_info "No embind source for ${APP_NAME} (expected at ${EMBIND_SRC}); using empty placeholder"
EMPTY_C="${STUBS_BUILD}/${APP_NAME}_embind_empty.c"

15
tests/README.md
View file

@ -289,3 +289,18 @@ Button positions (relative to canvas):
- **Timer tests**: May fail due to timing sensitivity
- **Tree tests**: Button click positions may vary
## Open tasks
- **Research: are the Asyncify fiber shims still needed under native-EH?** Two
`scripts/common/inject-dyncall-shims.sh` fixes — the fiber trampoline self-heal
(§3c) and the nested-Asyncify handleSleep save/restore (§3) — were written for the
legacy-EH *park-throw* model. Under native wasm-EH the top loop is a per-frame-yield
while-loop with no park-throw, so ablating either shim
(`SHIM_DISABLE_TRAMPOLINE_HEAL` / `SHIM_DISABLE_HANDLESLEEP`, wired in
`tests/apps/Makefile.wasm`) no longer reproduces the disease it guarded — the old
red "ablation pins" in `asyncify/asyncify-races.spec.ts` have been flipped to
green "shim-redundancy pins" that now prove the native-EH path stays clean *with the
shim ablated*. **To do:** sweep the real apps (modals, nested fibers, long sleeps,
pthread pool) with each shim ablated; if all stay green, drop the shim injection and
these pins. Until proven, they stay injected (belt-and-suspenders).

1
tests/apps/.gitignore vendored Normal file
View file

@ -0,0 +1 @@
standalone/_pool_unshimmed/

137
tests/apps/Makefile.wasm
View file

@ -51,16 +51,23 @@ endif
# crashes apps at startup with "RuntimeError: function signature mismatch".
CXXFLAGS += -MMD -MP
# Native wasm-EH is the only build mode: compile + link with -fwasm-exceptions (legacy encoding) +
# wasm setjmp/longjmp, matching the libwx build (scripts/build-wx-wasm.sh) and scripts/common/env.sh.
# The in-link Asyncify (emsdk Binaryen v121) crashes on wasm-EH, so build-wasm-test.sh stubs it and
# runs the real pipeline (--hoist-cpp-catches + --asyncify via apply-asyncify.sh) post-link.
EH_FLAGS = -fwasm-exceptions -sSUPPORT_LONGJMP=wasm -sWASM_LEGACY_EXCEPTIONS=1 -sDYNCALLS=1
CXXFLAGS += $(EH_FLAGS)
# Base Emscripten flags (for all apps)
# ASYNCIFY enables blocking modal dialogs (ShowModal waits for user)
# ASYNCIFY_IMPORTS tells Emscripten which imported JS functions can unwind the stack
# - startModal: for modal dialogs
# - js_writeTextToClipboard, js_readTextFromClipboard, js_clipboardHasText, js_clearClipboard: for clipboard
# - js_enumerateFonts: for font enumeration via Local Font Access API
BASE_LDFLAGS = -sALLOW_MEMORY_GROWTH -sERROR_ON_UNDEFINED_SYMBOLS=0 \
BASE_LDFLAGS = $(EH_FLAGS) -sALLOW_MEMORY_GROWTH -sERROR_ON_UNDEFINED_SYMBOLS=0 \
-s "EXPORTED_RUNTIME_METHODS=['HEAPU8','HEAP8','HEAP32','ccall']" \
-sASYNCIFY=1 \
-sASYNCIFY_STACK_SIZE=8192 \
-sASYNCIFY_STACK_SIZE=65536 \
-sASYNCIFY_IMPORTS=['startModal','js_writeTextToClipboard','js_readTextFromClipboard','js_clipboardHasText','js_clearClipboard','js_enumerateFonts']
# LDFLAGS for non-GL apps (standalone tests)
@ -92,7 +99,7 @@ LDFLAGS_PTHREAD = $(DEBUG_LDFLAGS) $(BASE_LDFLAGS) -pthread \
$(WX_LDFLAGS_NOGL)
# Coroutine harness flags - mirror KiCad's fiber-related runtime needs
COROUTINE_BASE_LDFLAGS = -sALLOW_MEMORY_GROWTH -sERROR_ON_UNDEFINED_SYMBOLS=0 \
COROUTINE_BASE_LDFLAGS = $(EH_FLAGS) -sALLOW_MEMORY_GROWTH -sERROR_ON_UNDEFINED_SYMBOLS=0 \
-s "EXPORTED_RUNTIME_METHODS=['HEAPU8','HEAP8','HEAP32','ccall']" \
-sASYNCIFY=1 \
-sASYNCIFY_STACK_SIZE=65536 \
@ -113,6 +120,13 @@ LDFLAGS_RAYTRACE = $(DEBUG_LDFLAGS) $(COROUTINE_BASE_LDFLAGS) -pthread \
-sPTHREAD_POOL_SIZE_STRICT=0 \
$(WX_LDFLAGS_NOGL)
# LDFLAGS for the real-pool test (Phase 1a). Like LDFLAGS_RAYTRACE but pre-warm +2 workers so
# the lifecycle mode's small BS::pause_thread_pool(2) gets pre-warmed workers (no on-demand spawn).
LDFLAGS_REALPOOL = $(DEBUG_LDFLAGS) $(COROUTINE_BASE_LDFLAGS) -pthread \
-sPTHREAD_POOL_SIZE='navigator.hardwareConcurrency + 2' \
-sPTHREAD_POOL_SIZE_STRICT=0 \
$(WX_LDFLAGS_NOGL)
# A second pre-js carries the DOM-control shim ("--pre-js A --pre-js B"
# after expansion — emcc accepts repeated --pre-js).
# JS_FILES lists the actual pre-js paths (no flags) so the link targets can
@ -209,13 +223,13 @@ minimal_test.html: minimal_test.o $(WX_CORE_LIB) $(JS_FILES)
# --- DOM-port bug reproductions (docs/features/wx-dom-port/branch-review.md) ---
# wxTextCtrl reentry-guard repro. Needs -fexceptions: it throws from a wxEVT_TEXT
# handler to prove the OnDomEvent m_inDomInput reset must be exception-safe.
# wxTextCtrl reentry-guard repro: it throws from a wxEVT_TEXT handler to prove the
# OnDomEvent m_inDomInput reset must be exception-safe (works under native -fwasm-exceptions).
$(S)/textctrl-reentry/textctrl-reentry_test.o: $(S)/textctrl-reentry/textctrl-reentry_test.cpp
$(CXX) -c $(CXXFLAGS) -fexceptions $< -o $@
$(CXX) -c $(CXXFLAGS) $< -o $@
$(S)/textctrl-reentry/textctrl-reentry_test.html: $(S)/textctrl-reentry/textctrl-reentry_test.o $(WX_CORE_LIB) $(JS_FILES)
$(CXX) $< $(LDFLAGS_NOGL) -fexceptions --pre-js $(JS) --shell-file $(HTML) -o $@
$(CXX) $< $(LDFLAGS_NOGL) --pre-js $(JS) --shell-file $(HTML) -o $@
textctrl-reentry: $(S)/textctrl-reentry/textctrl-reentry_test.html
@ -589,6 +603,100 @@ raytrace-threads: $(S)/raytrace-threads/raytrace_threads_test.html
# Include the raytracer-threading repro in the default `all` build (prereqs accumulate).
all: $(S)/raytrace-threads/raytrace_threads_test.html
# Un-shimmed copy of KiCad's BS::thread_pool: the single-thread #ifdef __EMSCRIPTEN__ shim in
# detach_task is disabled (-> #if 0) so the pool tests exercise REAL multithreading WITHOUT
# modifying the pristine KiCad header and WITHOUT a compile macro. Regenerated at parse time to
# stay in sync. (HASH is a literal '#', escaped so make does not treat it as a comment.)
HASH := \#
POOL_UNSHIMMED := $(S)/_pool_unshimmed
$(shell mkdir -p $(POOL_UNSHIMMED); sed 's/$(HASH)ifdef __EMSCRIPTEN__/$(HASH)if 0/' $(KICAD_ROOT)/thirdparty/thread-pool/bs_thread_pool.hpp > $(POOL_UNSHIMMED)/bs_thread_pool.hpp)
# Real KiCad thread-pool test (Phase 1a / native-EH). Compiles the REAL
# kicad/common/thread_pool.cpp (GetKiCadThreadPool) with minimal pgm_base/advanced_config
# stubs; the pool is un-shimmed via a generated bs_thread_pool.hpp (POOL_UNSHIMMED below) so
# tasks run on workers.
TP_REAL = $(S)/threadpool-real
TP_REAL_INC = -std=c++20 \
-I$(TP_REAL)/stubs -I$(POOL_UNSHIMMED) -I$(KICAD_ROOT)/include -I$(KICAD_ROOT)/thirdparty/thread-pool
$(TP_REAL)/threadpool_real_test.o: $(TP_REAL)/threadpool_real_test.cpp
$(CXX) -c $(CXXFLAGS) -pthread $(TP_REAL_INC) $< -o $@
$(TP_REAL)/thread_pool.o: $(KICAD_ROOT)/common/thread_pool.cpp
$(CXX) -c $(CXXFLAGS) -pthread $(TP_REAL_INC) $< -o $@
$(TP_REAL)/kicad_pool_stubs.o: $(TP_REAL)/stubs/kicad_pool_stubs.cpp
$(CXX) -c $(CXXFLAGS) -pthread $(TP_REAL_INC) $< -o $@
$(TP_REAL)/threadpool_real_test.html: $(TP_REAL)/threadpool_real_test.o $(TP_REAL)/thread_pool.o $(TP_REAL)/kicad_pool_stubs.o $(WX_CORE_LIB) $(JS_FILES)
$(CXX) $(filter %.o %.a,$^) $(LDFLAGS_REALPOOL) --pre-js $(JS) --shell-file $(HTML) -o $@
threadpool-real: $(TP_REAL)/threadpool_real_test.html
.PHONY: threadpool-real
all: $(TP_REAL)/threadpool_real_test.html
# On-demand non-warm Worker test (Phase 2). The real pool (compiled-in thread_pool.cpp)
# consumes the pre-warmed Workers, then raw fly-threads force on-demand creation;
# wasm/shims/nanosleep_yield.c (a strong nanosleep override; -Wl,--wrap crashes wasm-ld)
# makes the main-thread sleep_for join Asyncify-yield so the on-demand Workers boot.
# Reuses threadpool-real's pool stubs.
OD = $(S)/pthread-ondemand
OD_INC = -std=c++20 \
-I$(S)/threadpool-real/stubs -I$(POOL_UNSHIMMED) -I$(KICAD_ROOT)/include -I$(KICAD_ROOT)/thirdparty/thread-pool
SHIMS = ../../wasm/shims
$(OD)/pthread_ondemand_test.o: $(OD)/pthread_ondemand_test.cpp
$(CXX) -c $(CXXFLAGS) -pthread $(OD_INC) $< -o $@
$(OD)/thread_pool.o: $(KICAD_ROOT)/common/thread_pool.cpp
$(CXX) -c $(CXXFLAGS) -pthread $(OD_INC) $< -o $@
$(OD)/kicad_pool_stubs.o: $(S)/threadpool-real/stubs/kicad_pool_stubs.cpp
$(CXX) -c $(CXXFLAGS) -pthread $(OD_INC) $< -o $@
# Compiled as C (no -std=c++): em++ keys language off the .c extension.
$(OD)/nanosleep_yield.o: $(SHIMS)/nanosleep_yield.c
$(CXX) -c -pthread $< -o $@
$(OD)/pthread_ondemand_test.html: $(OD)/pthread_ondemand_test.o $(OD)/thread_pool.o $(OD)/kicad_pool_stubs.o $(OD)/nanosleep_yield.o $(WX_CORE_LIB) $(JS_FILES)
$(CXX) $(filter %.o %.a,$^) $(LDFLAGS_RAYTRACE) --pre-js $(JS) --shell-file $(HTML) -o $@
pthread-ondemand: $(OD)/pthread_ondemand_test.html
.PHONY: pthread-ondemand
all: $(OD)/pthread_ondemand_test.html
# Library-preload native-EH repro (1b). Standalone std::async worker that parses+throws (mode-c)
# and proxies a fetch to main (emscripten_proxy_*) — mirrors the KiCad-10 PCBJAM preload shape.
# No KiCad source. Uses the system proxying queue (built into the pthread runtime).
$(S)/async-preload/async_preload_test.o: $(S)/async-preload/async_preload_test.cpp
$(CXX) -c $(CXXFLAGS) -pthread $< -o $@
$(S)/async-preload/async_preload_test.html: $(S)/async-preload/async_preload_test.o $(WX_CORE_LIB) $(JS_FILES)
$(CXX) $< $(LDFLAGS_RAYTRACE) --pre-js $(JS) --shell-file $(HTML) -o $@
../../scripts/common/inject-dyncall-shims.sh $(basename $@).js
async-preload: $(S)/async-preload/async_preload_test.html
.PHONY: async-preload
all: $(S)/async-preload/async_preload_test.html
# A raytracer worker-join run inside a wx modal pump. The pump dispatches the work via
# ProcessEvents (ccall async:true) at Asyncify state==Normal, so both joins complete:
# m=0 sleep_for busy-wait, m=1 emscripten_sleep yield.
$(S)/raytrace-modal/raytrace_modal_test.o: $(S)/raytrace-modal/raytrace_modal_test.cpp
$(CXX) -c $(CXXFLAGS) -pthread $< -o $@
$(S)/raytrace-modal/raytrace_modal_test.html: $(S)/raytrace-modal/raytrace_modal_test.o $(WX_CORE_LIB) $(JS_FILES)
$(CXX) $< $(LDFLAGS_RAYTRACE) --pre-js $(JS) --shell-file $(HTML) -o $@
../../scripts/common/inject-dyncall-shims.sh $(basename $@).js
raytrace-modal: $(S)/raytrace-modal/raytrace_modal_test.html
.PHONY: raytrace-modal
all: $(S)/raytrace-modal/raytrace_modal_test.html
# Log Error test (no GL) - reproduces KiCad's kiface error dialog
$(S)/logerror/logerror_test.o: $(S)/logerror/logerror_test.cpp
$(CXX) -c $(CXXFLAGS) $< -o $@
@ -635,6 +743,7 @@ $(S)/asyncify-races/races_test_nosleepfix.html: $(S)/asyncify-races/races_test.o
$(CXX) $^ $(LDFLAGS_RACES) --pre-js $(JS) --shell-file $(HTML) -o $@
SHIM_DISABLE_HANDLESLEEP=1 ../../scripts/common/inject-dyncall-shims.sh $(basename $@).js
# Convenience targets
menu: $(S)/menu/menu_test.html
contextmenu: $(S)/contextmenu/contextmenu_test.html
@ -730,7 +839,7 @@ coroutine-pthread: $(S)/coroutine-pthread/coroutine_test.html
.PHONY: coroutine-pthread
# === No-wx pthread main() reproduction: fiber pattern in main + pthreads, no wxWidgets ===
LDFLAGS_COROUTINE_PTHREAD_NOWX = $(DEBUG_LDFLAGS) -sALLOW_MEMORY_GROWTH -sERROR_ON_UNDEFINED_SYMBOLS=0 \
LDFLAGS_COROUTINE_PTHREAD_NOWX = $(EH_FLAGS) $(DEBUG_LDFLAGS) -sALLOW_MEMORY_GROWTH -sERROR_ON_UNDEFINED_SYMBOLS=0 \
-sASYNCIFY=1 -sASYNCIFY_STACK_SIZE=65536 -sASYNCIFY_IMPORTS=['emscripten_fiber_swap'] \
-sDYNCALLS=1 -pthread -sPTHREAD_POOL_SIZE='navigator.hardwareConcurrency' \
-sPTHREAD_POOL_SIZE_STRICT=0 -sEXPORTED_RUNTIME_METHODS=['ccall']
@ -759,19 +868,19 @@ coroutine-pthread-nested: $(S)/coroutine-pthread/nested_repro.html
.PHONY: coroutine-pthread-nested
# Nested invoke_/dynCall reproduction WITH exceptions (invoke_* boundaries, asyncify-unwindable)
LDFLAGS_COROUTINE_INVOKE = $(DEBUG_LDFLAGS) -sALLOW_MEMORY_GROWTH -sERROR_ON_UNDEFINED_SYMBOLS=0 \
LDFLAGS_COROUTINE_INVOKE = $(EH_FLAGS) $(DEBUG_LDFLAGS) -sALLOW_MEMORY_GROWTH -sERROR_ON_UNDEFINED_SYMBOLS=0 \
-sASYNCIFY=1 -sASYNCIFY_STACK_SIZE=65536 \
-sASYNCIFY_IMPORTS=['invoke_vi','invoke_v','invoke_ii','invoke_iii','emscripten_fiber_swap'] \
-sDYNCALLS=1 -fexceptions -pthread -sPTHREAD_POOL_SIZE='navigator.hardwareConcurrency' \
-sDYNCALLS=1 -pthread -sPTHREAD_POOL_SIZE='navigator.hardwareConcurrency' \
-sPTHREAD_POOL_SIZE_STRICT=0 -sEXPORTED_RUNTIME_METHODS=['ccall']
$(S)/coroutine-pthread/nested_repro_ex.o: $(S)/coroutine-pthread/nested_repro.cpp $(S)/coroutine/kicad_coroutine_harness.h
@mkdir -p $(S)/coroutine-pthread
$(CXX) -c $(CXXFLAGS) -pthread -fexceptions -I$(KICAD_ROOT)/thirdparty/libcontext -I$(S)/coroutine $< -o $@
$(CXX) -c $(CXXFLAGS) -pthread -I$(KICAD_ROOT)/thirdparty/libcontext -I$(S)/coroutine $< -o $@
$(S)/coroutine-pthread/libcontext_ex.o: $(KICAD_ROOT)/thirdparty/libcontext/libcontext.cpp $(KICAD_ROOT)/thirdparty/libcontext/libcontext.h
@mkdir -p $(S)/coroutine-pthread
$(CXX) -c $(CXXFLAGS) -pthread -fexceptions -I$(KICAD_ROOT)/thirdparty/libcontext $< -o $@
$(CXX) -c $(CXXFLAGS) -pthread -I$(KICAD_ROOT)/thirdparty/libcontext $< -o $@
$(S)/coroutine-pthread/nested_repro_ex.html: $(S)/coroutine-pthread/nested_repro_ex.o $(S)/coroutine-pthread/libcontext_ex.o
$(CXX) $(filter %.o %.a,$^) $(LDFLAGS_COROUTINE_INVOKE) -o $@
@ -794,7 +903,7 @@ LDFLAGS_COROUTINE_EMBIND = $(LDFLAGS_COROUTINE_INVOKE) --bind
$(S)/coroutine-pthread/embind_repro.o: $(S)/coroutine-pthread/embind_repro.cpp $(S)/coroutine/kicad_coroutine_harness.h
@mkdir -p $(S)/coroutine-pthread
$(CXX) -c $(CXXFLAGS) -pthread -fexceptions -I$(KICAD_ROOT)/thirdparty/libcontext -I$(S)/coroutine $< -o $@
$(CXX) -c $(CXXFLAGS) -pthread -I$(KICAD_ROOT)/thirdparty/libcontext -I$(S)/coroutine $< -o $@
$(S)/coroutine-pthread/embind_repro.html: $(S)/coroutine-pthread/embind_repro.o $(S)/coroutine-pthread/libcontext_ex.o
$(CXX) $(filter %.o %.a,$^) $(LDFLAGS_COROUTINE_EMBIND) -o $@
@ -816,7 +925,7 @@ coroutine-pthread-mainloop: $(S)/coroutine-pthread/mainloop_repro.html
.PHONY: coroutine-pthread-mainloop
# WebGL2 + coroutine reproduction (no wx, no pthreads, default shell with #canvas)
LDFLAGS_COROUTINE_GL = $(DEBUG_LDFLAGS) -sALLOW_MEMORY_GROWTH -sERROR_ON_UNDEFINED_SYMBOLS=0 \
LDFLAGS_COROUTINE_GL = $(EH_FLAGS) $(DEBUG_LDFLAGS) -sALLOW_MEMORY_GROWTH -sERROR_ON_UNDEFINED_SYMBOLS=0 \
-sASYNCIFY=1 -sASYNCIFY_STACK_SIZE=65536 -sASYNCIFY_IMPORTS=['emscripten_fiber_swap'] \
-sDYNCALLS=1 -sMAX_WEBGL_VERSION=2 -sMIN_WEBGL_VERSION=2 -sEXPORTED_RUNTIME_METHODS=['ccall']

296
tests/apps/standalone/async-preload/async_preload_test.cpp Normal file
View file

@ -0,0 +1,296 @@
/**
* async_preload_test.cpp
*
* Standalone repro of KiCad-10's library-preload SHAPE (docs/features/wasm-exceptions/10 §7) —
* NOT using real KiCad/pcbnew. Proves native wasm-EH makes the shape safe.
*
* The shape (mirrors kicad/eeschema/sch_io/pcbjam_lib/sch_io_pcbjam_lib.cpp):
* - a std::async(std::launch::async, …) background worker (NOT the thread pool) that, per
* "library", PROXIES a fetch to the main thread (emscripten_proxy_sync_with_ctx) and then
* PARSES the bytes on the worker — the parse THROWS an IO_ERROR-like exception (the "mode-c"
* trigger: a throw caught ON a worker drives Asyncify under -fexceptions → crash; native
* wasm-EH decouples it → safe).
* - a LAZY join: main keeps the std::future alive and never blocks in normal operation, so it
* stays in its event loop to service the worker's proxied fetches.
* - a g_proxyMutex serializes worker→main proxy round-trips (the real "table index out of
* bounds" reentrancy guard when main is asyncify-suspended in a modal).
*
* (Simplification vs real KiCad: the proxied fetch finishes synchronously in the handler — the real
* one kicks an async JS promise + finishes from its callback. We keep the round-trip + the
* serialization, which is what the reentrancy hazard and the mode-c throw need; the throw is on the
* worker's parse, independent of the fetch being sync/async.)
*
* URL ?m=0 simple | 1 throw (parse throws → caught on worker, no mode-c crash) |
* 2 shutdown (main blocking-joins the future mid-load) | 3 modal (a modal opens while
* workers proxy → g_proxyMutex must prevent a reentrancy crash)
*
* Console contract:
* [PRELOAD] START m=.. [PRELOAD] EH=native|js
* [PRELOAD] SUCCESS m=.. caught=.. loaded=..
*/
#include "wx/wx.h"
#include <atomic>
#include <chrono>
#include <cstdarg>
#include <cstdio>
#include <future>
#include <mutex>
#include <stdexcept>
#include <string>
#include <thread>
#include <vector>
#ifdef __EMSCRIPTEN__
#include <emscripten/emscripten.h>
#include <emscripten/proxying.h>
#include <emscripten/threading.h>
#endif
// ---------------------------------------------------------------------------
static void plog( const char* fmt, ... )
{
char buf[512];
va_list ap;
va_start( ap, fmt );
vsnprintf( buf, sizeof( buf ), fmt, ap );
va_end( ap );
#ifdef __EMSCRIPTEN__
EM_ASM( { console.log( UTF8ToString( $0 ) ); }, buf );
#else
printf( "%s\n", buf );
#endif
}
static int readMode()
{
#ifdef __EMSCRIPTEN__
return EM_ASM_INT( {
var raw = location.search ? location.search.slice( 1 ) : location.hash.slice( 1 );
var v = parseInt( new URLSearchParams( raw ).get( 'm' ), 10 );
return isNaN( v ) ? 0 : v;
} );
#else
return 0;
#endif
}
static void mainSleep( int ms )
{
#ifdef __EMSCRIPTEN__
emscripten_sleep( ms ); // Asyncify yield → main's event loop runs (services the proxy queue)
#else
std::this_thread::sleep_for( std::chrono::milliseconds( ms ) );
#endif
}
// ---------------------------------------------------------------------------
// the proxied "fetch": worker → main round-trip (mirrors the PCBJAM bridge)
// ---------------------------------------------------------------------------
static std::mutex g_proxyMutex;
#ifdef __EMSCRIPTEN__
extern "C" EMSCRIPTEN_KEEPALIVE void preload_finish( em_proxying_ctx* ctx )
{
emscripten_proxy_finish( ctx );
}
static void proxy_fetch_on_main( em_proxying_ctx* ctx, void* )
{
// Runs on the MAIN thread via the system proxying queue. Synchronous finish (see header note).
preload_finish( ctx );
}
#endif
static std::string proxyFetchToMain( const std::string& lib )
{
#ifdef __EMSCRIPTEN__
if( emscripten_is_main_runtime_thread() )
return "(mock " + lib + ")"; // main path (not exercised here; the preload runs on a worker)
std::lock_guard<std::mutex> serialize( g_proxyMutex );
em_proxying_queue* q = emscripten_proxy_get_system_queue();
if( !emscripten_proxy_sync_with_ctx( q, emscripten_main_runtime_thread_id(),
proxy_fetch_on_main, nullptr ) )
return "";
#endif
return "(mock " + lib + ")";
}
// ---------------------------------------------------------------------------
class IO_ERROR : public std::runtime_error
{
public:
explicit IO_ERROR( const std::string& m ) : std::runtime_error( m ) {}
};
static std::size_t parseLib( const std::string& data, bool shouldThrow )
{
if( shouldThrow )
throw IO_ERROR( "simulated S-expr parse failure" ); // the mode-c trigger
return data.find( "mock" ) != std::string::npos ? 1u : 0u;
}
// ---------------------------------------------------------------------------
static std::atomic<bool> g_caught{ false };
static std::atomic<int> g_loaded{ 0 };
static std::atomic<bool> g_done{ false };
static std::atomic<bool> g_abort{ false };
static std::future<void> g_future; // kept alive => LAZY join (dtor blocks only at teardown)
static void preloadRun( bool throwOnParse )
{
try
{
std::this_thread::sleep_for( std::chrono::milliseconds( 20 ) ); // worker-side watchdog
for( const char* lib : { "lib_a", "lib_b", "lib_c" } )
{
if( g_abort.load() )
break; // the CancelPreload mitigation: bail before the next proxy
std::string data = proxyFetchToMain( lib ); // proxy the fetch to main
g_loaded.fetch_add( (int) parseLib( data, throwOnParse ) ); // parse ON the worker (throws)
}
}
catch( const std::exception& e )
{
g_caught.store( true );
plog( "[PRELOAD] worker caught: %s", e.what() );
}
g_done.store( true );
}
// ---------------------------------------------------------------------------
// minimal auto-closing modal (timer armed before ShowModal; closes itself)
// ---------------------------------------------------------------------------
static constexpr int ID_MODAL_CLOSE = wxID_HIGHEST + 700;
class AutoCloseDialog : public wxDialog
{
public:
AutoCloseDialog( wxWindow* parent, int delayMs ) :
wxDialog( parent, wxID_ANY, "preload-modal", wxDefaultPosition, wxSize( 280, 120 ) ),
m_timer( this, ID_MODAL_CLOSE )
{
Bind( wxEVT_SHOW, &AutoCloseDialog::OnShow, this );
Bind( wxEVT_TIMER, [this] ( wxTimerEvent& ) { plog( "[PRELOAD] modal: close-timer fired" ); EndModal( wxID_OK ); }, ID_MODAL_CLOSE );
m_delayMs = delayMs;
}
private:
void OnShow( wxShowEvent& e )
{
if( e.IsShown() )
{
plog( "[PRELOAD] modal: shown, arming %dms auto-close", m_delayMs );
m_timer.StartOnce( m_delayMs );
}
e.Skip();
}
wxTimer m_timer;
int m_delayMs = 300;
};
// ---------------------------------------------------------------------------
static constexpr int ID_SCENARIO_TIMER = wxID_HIGHEST + 701;
class PreloadFrame : public wxFrame
{
public:
PreloadFrame() : wxFrame( nullptr, wxID_ANY, "Async Preload Test",
wxDefaultPosition, wxSize( 420, 140 ) ),
m_scenarioTimer( this, ID_SCENARIO_TIMER )
{
wxPanel* p = new wxPanel( this );
wxBoxSizer* s = new wxBoxSizer( wxVERTICAL );
s->Add( new wxStaticText( p, wxID_ANY, "std::async library-preload repro — see console." ),
0, wxALL, 16 );
p->SetSizer( s );
Bind( wxEVT_TIMER, &PreloadFrame::OnScenario, this, ID_SCENARIO_TIMER );
}
void armModalScenario() { m_scenarioTimer.StartOnce( 50 ); }
private:
// Runs INSIDE the main event loop (the modal pump needs that — ShowModal straight from OnInit,
// before the loop starts, hangs). Several workers each do a bounded series of worker->main proxy
// round-trips with a small gap so the modal pump can dispatch its auto-close timer; g_proxyMutex
// serializes them, preventing concurrent C reentry into the asyncify-suspended (modal) main.
void OnScenario( wxTimerEvent& )
{
std::atomic<int> rounds{ 0 };
std::vector<std::thread> workers;
for( int i = 0; i < 3; ++i )
workers.emplace_back( [&rounds]
{
for( int k = 0; k < 12; ++k )
{
proxyFetchToMain( "spam" );
rounds.fetch_add( 1 );
std::this_thread::sleep_for( std::chrono::milliseconds( 10 ) );
}
} );
plog( "[PRELOAD] modal: workers spawned, showing modal" );
AutoCloseDialog dlg( this, 300 );
dlg.ShowModal();
plog( "[PRELOAD] modal: ShowModal returned (rounds=%d)", rounds.load() );
for( auto& t : workers )
t.join();
plog( "[PRELOAD] SUCCESS m=3 caught=0 loaded=0 proxyRounds=%d", rounds.load() );
}
wxTimer m_scenarioTimer;
};
class PreloadApp : public wxApp
{
public:
bool OnInit() override
{
const int mode = readMode();
plog( "[PRELOAD] START m=%d", mode );
#ifdef __WASM_EXCEPTIONS__
plog( "[PRELOAD] EH=native" );
#else
plog( "[PRELOAD] EH=js" );
#endif
g_caught.store( false );
g_loaded.store( 0 );
g_done.store( false );
g_abort.store( false );
PreloadFrame* frame = new PreloadFrame();
frame->Show();
if( mode == 3 )
{
frame->armModalScenario(); // runs in the main loop; logs its own [PRELOAD] SUCCESS m=3
return true;
}
const bool throwOnParse = ( mode == 1 );
g_future = std::async( std::launch::async, [throwOnParse] { preloadRun( throwOnParse ); } );
if( mode == 2 ) // shutdown: blocking-join the future mid-load (synchronous proxies complete
// during the futex busy-wait's queue processing)
{
mainSleep( 30 );
g_abort.store( true );
g_future.wait();
plog( "[PRELOAD] shutdown joined" );
}
else // simple / throw: LAZY join — poll via emscripten_sleep, main stays live
{
for( int i = 0; i < 200 && !g_done.load(); ++i )
mainSleep( 20 );
}
plog( "[PRELOAD] SUCCESS m=%d caught=%d loaded=%d",
mode, g_caught.load() ? 1 : 0, g_loaded.load() );
return true;
}
};
wxIMPLEMENT_APP( PreloadApp );

26
tests/apps/standalone/asyncify-races/races_test.cpp
View file

@ -163,17 +163,29 @@ EM_JS( void, races_mark_done, ( const char* aName ), {
Module.__racesDone[UTF8ToString( aName )] = true;
} );
// Quiescence invariant sampled from C++ between scenarios. NOTE: this runs on a
// stack that may itself have been resumed via Fibers.trampoline(), in which case
// Fibers.trampolineRunning is legitimately true — so the guard is deliberately
// NOT part of this check (a genuinely stuck guard wedges the next fiber swap and
// is caught by the scenario watchdogs instead).
// Quiescence invariant sampled from C++ between scenarios.
//
// Two things are deliberately NOT checked:
// * Fibers.trampolineRunning — this can run on a stack itself resumed via
// Fibers.trampoline(), in which case the guard is legitimately true.
// * Asyncify.currData — under native wasm-EH the top-level event loop is a
// per-frame-yield while-loop (wxWasmYieldToBrowser, an EM_ASYNC_JS rAF
// suspend that re-arms every frame; see wxwidgets/src/wasm/evtloop.cpp). So
// the main stack is asyncify-suspended between frames and currData is
// legitimately churning — it is non-zero while a frame yield is pending, and
// can momentarily hold a freed-but-not-yet-nulled buffer right after a
// concurrent suspension resumes. That is a transient bookkeeping value, NOT a
// leak (the buffers are _malloc/_free'd each frame — addresses are reused),
// so requiring currData==0 here is a stale legacy assumption from the old
// throw-to-park loop. A genuinely stuck suspension is caught by state != 0
// (Suspending/Rewinding never clearing) and by the scenario watchdogs.
// What's left is the real invariant: the asyncify machine is back to Normal and
// no fiber is queued.
EM_JS( int, races_quiescent, (), {
try {
var stOk = ( typeof Asyncify === 'undefined' ) || Asyncify.state === 0;
var cdOk = ( typeof Asyncify === 'undefined' ) || !Asyncify.currData;
var nfOk = ( typeof Fibers === 'undefined' ) || !Fibers.nextFiber;
return ( stOk && cdOk && nfOk ) ? 1 : 0;
return ( stOk && nfOk ) ? 1 : 0;
} catch( e ) {
return 0;
}

179
tests/apps/standalone/pthread-ondemand/pthread_ondemand_test.cpp Normal file
View file

@ -0,0 +1,179 @@
/**
* pthread_ondemand_test.cpp
*
* Phase 2: can a NON-warm (on-demand) Worker be spawned WITHOUT modifying KiCad?
*
* Faithful shape of the real raytracer's deadlock: the REAL KiCad pool (compiled-in
* kicad/common/thread_pool.cpp, via GetKiCadThreadPool()) consumes ALL the pre-warmed
* Workers at construction. We then spawn raw std::thread fly-threads BEYOND that count,
* which must be created ON DEMAND. Finalizing an on-demand Worker needs the main event
* loop to run the new-Worker 'loaded'->'run' handshake.
*
* m=0 control : a main-thread busy-wait join that does NOT call nanosleep -> never
* returns to the event loop -> the on-demand Workers never boot ->
* DEADLOCK (workersRan=0; self-recovers after a 12s cap).
* m=1 fix : join via std::this_thread::sleep_for -> nanosleep, which (via
* wasm/shims/nanosleep_yield.c, a strong nanosleep override) Asyncify-yields on
* the main thread -> the event loop services the handshake -> the
* on-demand Workers boot -> multi-core (workersRan>1). No KiCad edit.
*
* The override only affects main-thread nanosleep; a worker keeps a real blocking sleep.
*
* Console contract (the spec asserts on these):
* [ONDEMAND] START m=.. poolThreads=.. extra=.. hwc=..
* [ONDEMAND] SUCCESS m=.. workersRan=.. totalMs=.. (workersRan=0 for m=0, the deadlock)
* [ONDEMAND] DEADLOCK m=0: ...
*/
#include "wx/wx.h"
#include <thread_pool.h> // kicad/include/thread_pool.h -> GetKiCadThreadPool()
#include <algorithm>
#include <atomic>
#include <chrono>
#include <cmath>
#include <cstdarg>
#include <cstdio>
#include <thread>
#ifdef __EMSCRIPTEN__
#include <emscripten/emscripten.h>
#endif
using clk = std::chrono::steady_clock;
static void olog( const char* fmt, ... )
{
char buf[512];
va_list ap;
va_start( ap, fmt );
vsnprintf( buf, sizeof( buf ), fmt, ap );
va_end( ap );
#ifdef __EMSCRIPTEN__
EM_ASM( { console.log( UTF8ToString( $0 ) ); }, buf );
#else
printf( "%s\n", buf );
#endif
}
static int readMode()
{
#ifdef __EMSCRIPTEN__
return EM_ASM_INT( {
var raw = location.search ? location.search.slice( 1 ) : location.hash.slice( 1 );
var v = parseInt( new URLSearchParams( raw ).get( 'm' ), 10 );
return isNaN( v ) ? 1 : v;
} );
#else
return 1;
#endif
}
static long ms( clk::time_point t0 )
{
return (long) std::chrono::duration_cast<std::chrono::milliseconds>( clk::now() - t0 ).count();
}
// shared work: raw fly-threads cooperatively drain a global atomic block counter.
static std::atomic<size_t> g_nextBlock{ 0 };
static std::atomic<size_t> g_threadsFinished{ 0 };
static std::atomic<int> g_workersRan{ 0 };
static std::atomic<double> g_sink{ 0.0 };
static int g_numBlocks = 48;
static long g_iters = 3000000;
static double computeBlock( int b, long iters )
{
double acc = 0.0;
for( long i = 0; i < iters; ++i )
acc += std::sin( (double) ( b * 131 + i ) * 1e-6 ) * std::cos( (double) i * 1e-7 );
return acc;
}
static void workerBody()
{
bool counted = false;
for( size_t b = g_nextBlock.fetch_add( 1 ); b < (size_t) g_numBlocks; b = g_nextBlock.fetch_add( 1 ) )
{
if( !counted )
{
g_workersRan.fetch_add( 1 );
counted = true;
}
g_sink.store( g_sink.load( std::memory_order_relaxed ) + computeBlock( (int) b, g_iters ),
std::memory_order_relaxed );
}
g_threadsFinished.fetch_add( 1 );
}
class OndemandFrame : public wxFrame
{
public:
OndemandFrame() : wxFrame( nullptr, wxID_ANY, "On-demand Worker Test",
wxDefaultPosition, wxSize( 420, 140 ) )
{
wxPanel* p = new wxPanel( this );
wxBoxSizer* s = new wxBoxSizer( wxVERTICAL );
s->Add( new wxStaticText( p, wxID_ANY, "On-demand non-warm Worker test — see console." ),
0, wxALL, 16 );
p->SetSizer( s );
}
};
class OndemandApp : public wxApp
{
public:
bool OnInit() override
{
const int hwc = std::max( 1u, std::thread::hardware_concurrency() );
const int mode = readMode();
const int extra = 4; // raw fly-threads BEYOND the pool -> must be created on demand
// Consume ALL pre-warmed Workers with the REAL KiCad pool — exactly like KiCad does.
// (PTHREAD_POOL_SIZE = hardware_concurrency, the pool takes them all at construction,
// so the `extra` raw threads below force on-demand Worker creation.)
thread_pool& tp = GetKiCadThreadPool();
olog( "[ONDEMAND] START m=%d poolThreads=%u extra=%d hwc=%d",
mode, (unsigned) tp.get_thread_count(), extra, hwc );
g_nextBlock.store( 0 );
g_threadsFinished.store( 0 );
g_workersRan.store( 0 );
auto t0 = clk::now();
for( int i = 0; i < extra; ++i )
std::thread( workerBody ).detach();
if( mode == 0 )
{
// control: busy-wait WITHOUT nanosleep -> never yields -> on-demand never boots.
auto dl = clk::now();
while( g_threadsFinished.load() < (size_t) extra )
{
for( volatile int s = 0; s < 200000; ++s )
; // pure spin: no sleep_for / nanosleep, so it does not hit the override
if( std::chrono::duration_cast<std::chrono::seconds>( clk::now() - dl ).count() >= 12 )
{
olog( "[ONDEMAND] DEADLOCK m=0: on-demand workers never booted within 12s "
"(busy-wait starved the event loop)" );
break;
}
}
}
else
{
// fix: sleep_for -> nanosleep -> (overridden) Asyncify yield -> main pumps the
// new-Worker handshake -> the on-demand Workers boot and run.
while( g_threadsFinished.load() < (size_t) extra )
std::this_thread::sleep_for( std::chrono::milliseconds( 10 ) );
}
olog( "[ONDEMAND] SUCCESS m=%d workersRan=%d totalMs=%ld sink=%.3f",
mode, g_workersRan.load(), ms( t0 ), g_sink.load() );
( new OndemandFrame() )->Show();
return true;
}
};
wxIMPLEMENT_APP( OndemandApp );

269
tests/apps/standalone/raytrace-modal/raytrace_modal_test.cpp Normal file
View file

@ -0,0 +1,269 @@
/**
* raytrace_modal_test.cpp
*
* A raytracer-style worker-join run inside a wx modal pump, in both join styles.
*
* A pass is dispatched from a wxTimer that fires while a ShowModal() dialog is open. The modal pump
* runs ProcessEvents via ccall(async:true), so the timer handler runs in a fresh managed Asyncify
* context at state == Normal (the handler probes and logs the state). Both join styles complete
* multi-core there:
* ?m=0 busywait : the join is a sleep_for() busy-wait; the pre-warmed pool's workers complete it.
* ?m=1 yield : the join yields via emscripten_sleep; legal at state == Normal, so it suspends
* and resumes normally.
*
* The persistent pool is warmed in OnInit (on the free top-level slot) so the workers are alive
* before the modal opens, which lets the in-modal busy-wait complete without on-demand Worker spawn.
*/
#include "wx/wx.h"
#include <atomic>
#include <chrono>
#include <cmath>
#include <condition_variable>
#include <cstdarg>
#include <cstdio>
#include <mutex>
#include <thread>
#include <vector>
#ifdef __EMSCRIPTEN__
#include <emscripten/emscripten.h>
#endif
using clk = std::chrono::steady_clock;
static void rtlog( const char* fmt, ... )
{
char buf[512];
va_list ap;
va_start( ap, fmt );
vsnprintf( buf, sizeof( buf ), fmt, ap );
va_end( ap );
#ifdef __EMSCRIPTEN__
EM_ASM( { console.log( UTF8ToString( $0 ) ); }, buf );
#else
printf( "%s\n", buf );
#endif
}
static int readMode()
{
#ifdef __EMSCRIPTEN__
return EM_ASM_INT( {
var raw = location.search ? location.search.slice( 1 ) : location.hash.slice( 1 );
var v = parseInt( new URLSearchParams( raw ).get( 'm' ), 10 );
return isNaN( v ) ? 0 : v;
} );
#else
return 0;
#endif
}
static long ms( clk::time_point t0 )
{
return (long) std::chrono::duration_cast<std::chrono::milliseconds>( clk::now() - t0 ).count();
}
// ---------------------------------------------------------------------------
// shared raytracer-style work (atomic work-stealing; mirrors render_3d_raytrace_base.cpp)
// ---------------------------------------------------------------------------
static std::atomic<size_t> g_nextBlock{ 0 };
static std::atomic<int> g_workersRan{ 0 };
static std::atomic<double> g_sink{ 0.0 };
static int g_numBlocks = 48;
static long g_iters = 600000;
static double computeBlock( int b, long iters )
{
double acc = 0.0;
for( long i = 0; i < iters; ++i )
acc += std::sin( (double) ( b * 131 + i ) * 1e-6 ) * std::cos( (double) i * 1e-7 );
return acc;
}
static void workerBody()
{
bool counted = false;
for( size_t b = g_nextBlock.fetch_add( 1 ); b < (size_t) g_numBlocks; b = g_nextBlock.fetch_add( 1 ) )
{
if( !counted )
{
g_workersRan.fetch_add( 1 );
counted = true;
}
g_sink.store( g_sink.load( std::memory_order_relaxed ) + computeBlock( (int) b, g_iters ),
std::memory_order_relaxed );
}
}
// ---------------------------------------------------------------------------
// persistent pre-warmed pool (warmed in OnInit, free slot) — so the IN-MODAL busy-wait completes
// rather than deadlocking on on-demand Worker creation.
// ---------------------------------------------------------------------------
struct PersistentPool
{
int n = 0;
std::vector<std::thread> ts;
std::mutex mtx;
std::condition_variable cv;
unsigned long generation = 0;
bool stop = false;
std::atomic<size_t> finished{ 0 };
std::atomic<size_t> started{ 0 };
void start( int n_ )
{
n = n_;
for( int i = 0; i < n; ++i )
ts.emplace_back( [this] { loop(); } );
}
bool ready() const { return started.load() == (size_t) n; }
void loop()
{
started.fetch_add( 1 );
unsigned long seen = 0;
for( ;; )
{
std::unique_lock<std::mutex> lk( mtx );
cv.wait( lk, [&] { return stop || generation != seen; } );
if( stop )
return;
seen = generation;
lk.unlock();
workerBody();
finished.fetch_add( 1 );
}
}
// Called on the MAIN thread, from the modal pump's ProcessEvents (state == Normal).
void runPass( bool yieldJoin )
{
finished.store( 0 );
g_nextBlock.store( 0 );
g_workersRan.store( 0 );
{
std::lock_guard<std::mutex> lk( mtx );
++generation;
}
cv.notify_all();
while( finished.load() < (size_t) n )
{
if( yieldJoin )
#ifdef __EMSCRIPTEN__
emscripten_sleep( 10 ); // Asyncify yield (legal here — state == Normal)
#else
std::this_thread::sleep_for( std::chrono::milliseconds( 10 ) );
#endif
else
std::this_thread::sleep_for( std::chrono::microseconds( 200 ) ); // busy-wait join
}
}
};
static PersistentPool g_pool;
// ---------------------------------------------------------------------------
// modal scaffold (mirrors coroutine-nested/nested_test.cpp's AutoClosingDialog)
// ---------------------------------------------------------------------------
static constexpr int ID_MODAL_CLOSE = wxID_HIGHEST + 800;
static constexpr int ID_SCENARIO = wxID_HIGHEST + 801;
static constexpr int ID_MODAL_WORK = wxID_HIGHEST + 802;
class AutoClosingDialog : public wxDialog
{
public:
AutoClosingDialog( wxWindow* parent ) :
wxDialog( parent, wxID_ANY, "rt-modal", wxDefaultPosition, wxSize( 280, 120 ) ) {}
void EndModalExternal( int code ) { EndModal( code ); }
};
class RtModalFrame : public wxFrame
{
public:
RtModalFrame( int mode ) :
wxFrame( nullptr, wxID_ANY, "Raytrace Modal Test", wxDefaultPosition, wxSize( 420, 140 ) ),
m_mode( mode ),
m_scenarioTimer( this, ID_SCENARIO ),
m_modalWorkTimer( this, ID_MODAL_WORK )
{
wxPanel* p = new wxPanel( this );
wxBoxSizer* s = new wxBoxSizer( wxVERTICAL );
s->Add( new wxStaticText( p, wxID_ANY, "Raytrace-in-modal repro — see console." ), 0, wxALL, 16 );
p->SetSizer( s );
Bind( wxEVT_TIMER, &RtModalFrame::OnScenario, this, ID_SCENARIO );
Bind( wxEVT_TIMER, &RtModalFrame::OnModalWork, this, ID_MODAL_WORK );
}
void armScenario() { m_scenarioTimer.StartOnce( 50 ); }
private:
// In the main loop: arm the modal-work timer BEFORE ShowModal so it fires from within the modal
// pump, then show the modal.
void OnScenario( wxTimerEvent& )
{
AutoClosingDialog* dlg = new AutoClosingDialog( this );
m_activeDialog = dlg;
m_modalWorkTimer.StartOnce( 30 );
rtlog( "[RTPOOL] showing modal (mode=%d) — work runs from the modal pump's ProcessEvents", m_mode );
dlg->ShowModal(); // returns when OnModalWork's pass finishes and closes it
m_activeDialog = nullptr;
rtlog( "[RTPOOL] SUCCESS mode=%d workersRan=%d totalMs=%ld",
m_mode, g_workersRan.load(), ms( m_t0 ) );
dlg->Destroy();
}
// Fires from the modal pump's ProcessEvents — a fresh managed entry at Asyncify state == Normal.
void OnModalWork( wxTimerEvent& )
{
#ifdef __EMSCRIPTEN__
int st = EM_ASM_INT( {
return ( typeof Asyncify !== 'undefined' && Asyncify.state !== undefined ) ? Asyncify.state : -1;
} );
rtlog( "[RTPOOL] modal-work: Asyncify.state=%d (0=Normal,1=Unwinding,2=Rewinding) mode=%d", st, m_mode );
#endif
rtlog( "[RTPOOL] modal-work: running pass from the modal pump (mode=%d)", m_mode );
m_t0 = clk::now();
g_pool.runPass( m_mode == 1 ); // m=1 yields via emscripten_sleep; m=0 busy-waits
rtlog( "[RTPOOL] PASS done workersRan=%d", g_workersRan.load() );
if( m_activeDialog )
m_activeDialog->EndModalExternal( wxID_OK );
}
int m_mode;
AutoClosingDialog* m_activeDialog = nullptr;
clk::time_point m_t0;
wxTimer m_scenarioTimer;
wxTimer m_modalWorkTimer;
};
class RtModalApp : public wxApp
{
public:
bool OnInit() override
{
const int mode = readMode();
const int hwc = std::max( 1u, std::thread::hardware_concurrency() );
rtlog( "[RTPOOL] START mode=%d hwc=%d", mode, hwc );
// Warm the persistent pool on the FREE top-level slot (emscripten_sleep is legal here), so
// the in-modal join runs against already-alive workers.
g_pool.start( hwc );
while( !g_pool.ready() )
#ifdef __EMSCRIPTEN__
emscripten_sleep( 5 );
#else
std::this_thread::sleep_for( std::chrono::milliseconds( 5 ) );
#endif
rtlog( "[RTPOOL] pool warmed (%d workers)", hwc );
RtModalFrame* f = new RtModalFrame( mode );
f->Show();
f->armScenario();
return true;
}
};
wxIMPLEMENT_APP( RtModalApp );

8
tests/apps/standalone/threadpool-real/stubs/advanced_config.h Normal file
View file

@ -0,0 +1,8 @@
#pragma once
// Minimal stub of kicad/include/advanced_config.h. thread_pool.cpp only reads
// ADVANCED_CFG::GetCfg().m_MaximumThreads; 0 => the pool uses hardware_concurrency().
struct ADVANCED_CFG
{
int m_MaximumThreads = 0;
static const ADVANCED_CFG& GetCfg();
};

4
tests/apps/standalone/threadpool-real/stubs/import_export.h Normal file
View file

@ -0,0 +1,4 @@
#pragma once
// Minimal stub of kicad/include/import_export.h — thread_pool.h only uses APIEXPORT.
#define APIEXPORT
#define APIIMPORT

28
tests/apps/standalone/threadpool-real/stubs/kicad_pool_stubs.cpp Normal file
View file

@ -0,0 +1,28 @@
// Definitions for the minimal pgm_base / advanced_config stubs (see ./pgm_base.h,
// ./advanced_config.h). They let kicad/common/thread_pool.cpp compile and link
// standalone while still constructing the REAL BS::priority_thread_pool.
#include <advanced_config.h>
#include <pgm_base.h>
#include <cstdlib>
// No PGM_BASE in this standalone test -> GetKiCadThreadPool() takes the
// `new thread_pool( num_threads )` branch (num_threads=0 -> hardware_concurrency()).
PGM_BASE* PgmOrNull()
{
return nullptr;
}
// Referenced by thread_pool.cpp inside `if( PgmOrNull() )`, which is never taken here;
// must link but is never executed.
thread_pool& PGM_BASE::GetThreadPool()
{
std::abort();
}
static const ADVANCED_CFG g_advancedCfgStub;
const ADVANCED_CFG& ADVANCED_CFG::GetCfg()
{
return g_advancedCfgStub;
}

12
tests/apps/standalone/threadpool-real/stubs/pgm_base.h Normal file
View file

@ -0,0 +1,12 @@
#pragma once
#include <thread_pool.h> // for the `thread_pool` type returned by GetThreadPool()
// Minimal stub of kicad/include/pgm_base.h. thread_pool.cpp only needs PgmOrNull()
// (stubbed to nullptr so GetKiCadThreadPool() constructs its own real pool) and
// PGM_BASE::GetThreadPool() (referenced in the now-dead branch; never called).
class PGM_BASE
{
public:
thread_pool& GetThreadPool();
};
PGM_BASE* PgmOrNull();

280
tests/apps/standalone/threadpool-real/threadpool_real_test.cpp Normal file
View file

@ -0,0 +1,280 @@
/**
* threadpool_real_test.cpp
*
* The BS-pool-API native-EH test (docs/features/wasm-exceptions/10 §6 #1).
*
* Unlike threadpool_test / raytrace_threads_test (which hand-roll raw std::thread),
* this app drives KiCad's REAL pool: it compiles kicad/common/thread_pool.cpp and
* calls the actual GetKiCadThreadPool() (a BS::priority_thread_pool). The pool's
* detach_task __EMSCRIPTEN__ inline shim is opted OUT here via -DKICAD_WASM_REAL_THREADPOOL
* so tasks actually run on the pool's persistent pthread workers.
*
* WHY this is the decisive test: the real pool is mode-a/b-safe by construction —
* its workers are PERSISTENT (created at pool construction, consuming the pre-warmed
* pool, so no on-demand spawn → no deadlock) and its main-side join is a futex
* busy-wait, not emscripten_sleep (→ no Asyncify nesting). The ONLY native-EH risk is
* mode-c: a task that THROWS on a worker (caught by submit_task's promise wrapper ON the
* worker drives Asyncify under -fexceptions → "func is not a function"). Native wasm-EH
* decouples exceptions from Asyncify and should make it safe. So:
* - modes submit/loop/blocks/detach/fanout/lifecycle prove real multi-core (workersRan>1)
* - mode throw is the mode-c / native-EH proof (red under JS-EH, green under native-EH)
* Green here ⇒ we can drop the detach_task shim for every pool consumer.
*
* Pure-compute tasks only (no JS/DOM/async-IO on a worker) — exactly why the real
* pool's tasks are safe to run off the main thread.
*
* URL: ?m=0 submit | 1 loop | 2 blocks | 3 detach | 4 fanout | 5 lifecycle | 6 throw
* (read from location.search OR the #fragment — serve-handler cleanUrls drops ?query)
*
* Console contract (the Playwright spec asserts on these):
* [POOL] START mode=.. threads=..
* [POOL] SUCCESS mode=.. workersRan=.. totalMs=.. sink=..
* [POOL] SUCCESS mode=6 threw=1 caught=.. workersRan=.. ... (the throw / mode-c proof)
*/
#include "wx/wx.h"
#include <thread_pool.h> // kicad/include/thread_pool.h -> GetKiCadThreadPool() + BS::*
#include <atomic>
#include <chrono>
#include <cmath>
#include <cstdarg>
#include <cstdint>
#include <cstdio>
#include <future>
#include <optional>
#include <stdexcept>
#include <vector>
#ifdef __EMSCRIPTEN__
#include <emscripten/emscripten.h>
#endif
using clk = std::chrono::steady_clock;
// ---------------------------------------------------------------------------
static void plog( const char* fmt, ... )
{
char buf[512];
va_list ap;
va_start( ap, fmt );
vsnprintf( buf, sizeof( buf ), fmt, ap );
va_end( ap );
#ifdef __EMSCRIPTEN__
EM_ASM( { console.log( UTF8ToString( $0 ) ); }, buf );
#else
printf( "%s\n", buf );
#endif
}
static int readMode()
{
#ifdef __EMSCRIPTEN__
return EM_ASM_INT( {
var raw = location.search ? location.search.slice( 1 ) : location.hash.slice( 1 );
var v = parseInt( new URLSearchParams( raw ).get( 'm' ), 10 );
return isNaN( v ) ? 0 : v;
} );
#else
return 0;
#endif
}
static long ms( clk::time_point t0 )
{
return (long) std::chrono::duration_cast<std::chrono::milliseconds>( clk::now() - t0 ).count();
}
// ---------------------------------------------------------------------------
// shared work: pure CPU math; each task records WHICH pool worker ran it so we can
// prove real parallelism (distinct BS::this_thread::get_index() values).
// ---------------------------------------------------------------------------
static std::atomic<uint64_t> g_workerMask{ 0 };
static std::atomic<double> g_sink{ 0.0 };
static void recordWorker()
{
std::optional<std::size_t> idx = BS::this_thread::get_index();
if( idx )
g_workerMask.fetch_or( 1ull << ( *idx & 63 ), std::memory_order_relaxed );
}
static double computeBlock( int b, long iters )
{
double acc = 0.0;
for( long i = 0; i < iters; ++i )
acc += std::sin( (double) ( b * 131 + i ) * 1e-6 ) * std::cos( (double) i * 1e-7 );
return acc;
}
static int workersRan()
{
return __builtin_popcountll( g_workerMask.load() );
}
static constexpr int N = 64;
static constexpr long ITERS = 2000000;
static void addWork( int b, long iters )
{
recordWorker();
g_sink.store( g_sink.load( std::memory_order_relaxed ) + computeBlock( b, iters ),
std::memory_order_relaxed );
}
// ---------------------------------------------------------------------------
class PoolFrame : public wxFrame
{
public:
PoolFrame() : wxFrame( nullptr, wxID_ANY, "Real Thread Pool Test",
wxDefaultPosition, wxSize( 420, 140 ) )
{
wxPanel* p = new wxPanel( this );
wxBoxSizer* s = new wxBoxSizer( wxVERTICAL );
s->Add( new wxStaticText( p, wxID_ANY, "Real GetKiCadThreadPool() test — see console." ),
0, wxALL, 16 );
p->SetSizer( s );
}
};
class PoolApp : public wxApp
{
public:
bool OnInit() override
{
const int mode = readMode();
thread_pool& tp = GetKiCadThreadPool(); // the REAL BS::priority_thread_pool
plog( "[POOL] START mode=%d threads=%u", mode, (unsigned) tp.get_thread_count() );
// Report the exception model so the spec can gate the mode-c (throw-on-worker) test:
// a worker throw is only safe under native wasm-EH (-fwasm-exceptions defines this).
#ifdef __WASM_EXCEPTIONS__
plog( "[POOL] EH=native" );
#else
plog( "[POOL] EH=js" );
#endif
g_workerMask.store( 0 );
g_sink.store( 0.0 );
auto t0 = clk::now();
switch( mode )
{
case 0: // submit_task + vector<future> + 250ms poll loop (the DRC/connectivity idiom)
{
std::vector<std::future<void>> futs;
for( int i = 0; i < N; ++i )
futs.push_back( tp.submit_task( [i] { addWork( i, ITERS ); } ) );
for( auto& f : futs )
while( f.wait_for( std::chrono::milliseconds( 250 ) ) != std::future_status::ready )
;
break;
}
case 1: // submit_loop -> multi_future.wait()
{
BS::multi_future<void> mf = tp.submit_loop( 0, N, [] ( int i ) { addWork( i, ITERS ); } );
mf.wait();
break;
}
case 2: // submit_blocks (each block returns a value)
{
BS::multi_future<double> mf = tp.submit_blocks(
0, N,
[] ( int s, int e )
{
recordWorker();
double a = 0.0;
for( int i = s; i < e; ++i )
a += computeBlock( i, ITERS );
return a;
} );
double total = 0.0;
for( auto& f : mf )
if( f.valid() )
total += f.get();
g_sink.store( g_sink.load() + total );
break;
}
case 3: // detach_task fire-and-forget + tp.wait()
{
for( int i = 0; i < N; ++i )
tp.detach_task( [i] { addWork( i, ITERS ); } );
tp.wait();
break;
}
case 4: // manual multi_future fanned out by get_thread_count() (the renderTracing shape)
{
std::atomic<int> next{ 0 };
BS::multi_future<void> mf;
auto proc = [&next]
{
for( int b = next.fetch_add( 1 ); b < N; b = next.fetch_add( 1 ) )
addWork( b, ITERS );
};
for( std::size_t i = 0; i < tp.get_thread_count(); ++i )
mf.push_back( tp.submit_task( proc ) );
mf.wait();
break;
}
case 5: // lifecycle + features: queries / purge / reset + pause via a pause_thread_pool
{
plog( "[POOL] tasks queued=%zu running=%zu total=%zu threads=%zu",
tp.get_tasks_queued(), tp.get_tasks_running(), tp.get_tasks_total(),
(std::size_t) tp.get_thread_count() );
tp.wait();
tp.purge();
// pause/unpause are compiled OUT on the priority pool, so exercise them on a
// small pause_thread_pool (the +2 PTHREAD_POOL_SIZE headroom pre-warms its workers).
BS::pause_thread_pool pp( 2 );
pp.pause();
for( int i = 0; i < 8; ++i )
pp.detach_task( [] {} );
const bool paused = pp.is_paused();
const std::size_t queued = pp.get_tasks_queued();
pp.unpause();
pp.wait();
plog( "[POOL] pause: is_paused=%d queuedWhilePaused=%zu", paused ? 1 : 0, queued );
// and a real parallel batch so workersRan>1 holds for this mode too
tp.submit_loop( 0, N, [] ( int i ) { addWork( i, ITERS / 4 ); } ).wait();
break;
}
case 6: // throw ON a worker -> rethrow on main. THE mode-c / native-EH proof.
{
int caught = 0;
auto f = tp.submit_task( [] () -> int
{
recordWorker();
throw std::runtime_error( "boom-on-worker" );
return 0;
} );
try
{
(void) f.get();
}
catch( const std::exception& e )
{
caught = 1;
plog( "[POOL] caught on main: %s", e.what() );
}
// and prove workers still run a normal batch after the throw
tp.submit_loop( 0, N, [] ( int i ) { addWork( i, ITERS / 4 ); } ).wait();
plog( "[POOL] SUCCESS mode=6 threw=1 caught=%d workersRan=%d totalMs=%ld sink=%.3f",
caught, workersRan(), ms( t0 ), g_sink.load() );
( new PoolFrame() )->Show();
return true;
}
default:
plog( "[POOL] unknown mode=%d", mode );
break;
}
plog( "[POOL] SUCCESS mode=%d workersRan=%d totalMs=%ld sink=%.3f",
mode, workersRan(), ms( t0 ), g_sink.load() );
( new PoolFrame() )->Show();
return true;
}
};
wxIMPLEMENT_APP( PoolApp );

94
tests/asyncify/asyncify-races.spec.ts
View file

@ -177,8 +177,28 @@ test.describe('Asyncify races — green targets (full shims)', () => {
});
});
test.describe('Asyncify races — ablation pins (the disease stays reproducible)', () => {
test('no trampoline heal: the park wedges the guard and the post-park swap hangs', async ({
// These two were "ablation pins": they ablated a JS-shim fix (via SHIM_DISABLE_*,
// see tests/apps/Makefile.wasm) and asserted the LEGACY-EH disease came back — a
// stuck Fibers.trampoline guard (trampolineRunning=true) / a clobbered parked-sleep
// buffer. Under native wasm-EH the top loop is a per-frame-yield while-loop with NO
// park-throw (docs/features/wasm-exceptions; native-EH top-loop redesign), so the
// disease's *trigger* is gone at the root: ablating the shim no longer reproduces
// it. (The green "battery" above already runs both scenarios full-shim.)
//
// So they are flipped red->green: each now pins that the native-EH path stays clean
// EVEN with the legacy shim ablated — i.e. the scenario completes with no disease
// signature. That keeps them as the live signal for the open question below.
//
// TODO(research): are these shims still needed AT ALL under native-EH-only? The
// fiber trampoline self-heal (§3c) and the nested-Asyncify handleSleep save/restore
// (§3) in scripts/common/inject-dyncall-shims.sh were written for the legacy
// park-throw model; ablating either no longer breaks the scenario it guarded. If a
// full sweep of the real apps (modals / nested fibers / long sleeps / pthread pool)
// confirms they're dead weight under native-EH, drop the shim injection AND these
// pins. Until then they stay injected (belt-and-suspenders). Tracked in
// tests/README.md "Open tasks".
test.describe('Asyncify races — shim-redundancy pins (native-EH stays clean with the legacy shim ablated)', () => {
test('trampoline-heal ablated: native-EH post-park swap still completes (no stuck guard)', async ({
page,
testLogger,
}) => {
@ -187,39 +207,28 @@ test.describe('Asyncify races — ablation pins (the disease stays reproducible)
);
await tryLoadApp(page, 30000);
// The scenario's JS watchdog fires after 2.5s with a state dump.
await expect
.poll(
() =>
testLogger.consoleLogs.find((l) =>
l.includes('[ASYNCIFY_RACES] WATCHDOG post_park_fiber_swap')
) ?? null,
{ timeout: 30000, message: 'watchdog should fire in the ablated build' }
)
.poll(() => findSummary(testLogger.consoleLogs) ?? null, {
timeout: 45000,
message: 'post-park swap should complete even with the trampoline self-heal ablated',
})
.not.toBeNull();
const watchdog = testLogger.consoleLogs.find((l) =>
l.includes('[ASYNCIFY_RACES] WATCHDOG post_park_fiber_swap')
)!;
// The exact stuck-guard signature traced in docs/features/async/: the park throw
// tore through Fibers.trampoline()'s do/while, leaving the guard true.
expect(watchdog, 'stuck trampoline guard should be visible').toContain(
'trampolineRunning=true'
);
const { passed, failed } = parseSummary(findSummary(testLogger.consoleLogs)!);
expect(passed).toBe(1);
expect(failed).toBe(0);
// The legacy disease signature must be ABSENT: no stuck-trampoline watchdog dump.
expect(
testLogger.consoleLogs.some((l) =>
l.includes('[ASYNCIFY_RACES] FAIL post_park_fiber_swap')
l.includes('[ASYNCIFY_RACES] WATCHDOG post_park_fiber_swap')
),
'scenario should be reported FAILED by the watchdog'
).toBe(true);
// And the suite never completes — the swap is stranded forever.
expect(findSummary(testLogger.consoleLogs)).toBeUndefined();
'no stuck-trampoline watchdog should fire under native-EH'
).toBe(false);
expect(crashLines(testLogger), 'no crash signatures in console').toHaveLength(0);
});
test('no handleSleep fix: fiber swaps clobber the parked sleep buffer', async ({
test('handleSleep ablated: native-EH long parked sleep is not clobbered by a swap', async ({
page,
testLogger,
}) => {
@ -228,32 +237,17 @@ test.describe('Asyncify races — ablation pins (the disease stays reproducible)
);
await tryLoadApp(page, 30000);
// Either the wakeUp crashes (index out of bounds family) or the rewind is
// lost and the watchdog reports the stall — both are the recorded disease.
await expect
.poll(
() => {
const crashed = [...testLogger.errors, ...testLogger.consoleLogs].some(
(l) =>
l.toLowerCase().includes('index out of bounds') ||
l.toLowerCase().includes('indirect call to null') ||
l.toLowerCase().includes('invalid state')
);
const stalled = testLogger.consoleLogs.some((l) =>
l.includes('[ASYNCIFY_RACES] FAIL long_parked_sleep_clobbered_by_swap')
);
return crashed || stalled ? 'reproduced' : null;
},
{ timeout: 30000, message: 'ablated build should reproduce the clobber bug' }
)
.poll(() => findSummary(testLogger.consoleLogs) ?? null, {
timeout: 45000,
message: 'long parked sleep should resolve even with the handleSleep fix ablated',
})
.not.toBeNull();
// It must NOT have quietly passed.
expect(
testLogger.consoleLogs.some((l) =>
l.includes('[ASYNCIFY_RACES] PASS long_parked_sleep_clobbered_by_swap')
),
'ablated build must not pass the clobber scenario'
).toBe(false);
const { passed, failed } = parseSummary(findSummary(testLogger.consoleLogs)!);
expect(passed).toBe(1);
expect(failed).toBe(0);
expect(crashLines(testLogger), 'no crash signatures in console').toHaveLength(0);
expect(realErrors(testLogger), 'no page errors').toHaveLength(0);
});
});

67
tests/e2e/coroutine-async-preload.spec.ts Normal file
View file

@ -0,0 +1,67 @@
import { test, expect } from './utils/fixtures';
// 1b: library-preload native-EH repro (docs/features/wasm-exceptions/10 §7).
// A STANDALONE std::async worker that parses+THROWS (mode-c) and proxies a fetch to main — the
// KiCad-10 PCBJAM preload shape, with NO KiCad source. Proves native wasm-EH makes the worker-side
// parse-throw safe, and that the proxy round-trip / lazy join / modal-reentrancy all work.
//
// Named coroutine-* so playwright-coroutine.config.ts runs it in real Chrome + Firefox.
// WebKit skipped for pthread apps (COEP).
const APP = '/standalone/async-preload/async_preload_test.html';
function parse( logs: string[], mode: number ) {
const line = logs.find( l => l.includes( `[PRELOAD] SUCCESS m=${mode}` ) );
if( !line ) return null;
const c = line.match( /caught=(\d+)/ );
const ld = line.match( /loaded=(\d+)/ );
return { caught: c ? +c[1] : -1, loaded: ld ? +ld[1] : -1 };
}
async function waitForLog( testLogger: { consoleLogs: string[] }, needle: string, timeout = 60000 ) {
await expect.poll( () => testLogger.consoleLogs.some( l => l.includes( needle ) ), { timeout } ).toBe( true );
}
// Fatal native-EH / Asyncify failures we must NOT see.
function fatal( testLogger: { errors: string[] } ) {
return testLogger.errors.filter( e => !e.includes( 'favicon' )
&& /invalid state|table index out of bounds|aborted|unreachable|func is not a function/i.test( e ) );
}
test.describe( 'std::async library-preload — native-EH safe (KiCad-10 shape, standalone)', () => {
test( 'm=0 simple: worker proxies + parses, lazy join completes', async ( { page, testLogger } ) => {
await page.goto( `${APP}#m=0` );
await waitForLog( testLogger, '[PRELOAD] SUCCESS m=0' );
const r = parse( testLogger.consoleLogs, 0 )!;
expect( r.loaded, 'libraries parsed' ).toBeGreaterThan( 0 );
expect( fatal( testLogger ), 'no fatal errors' ).toHaveLength( 0 );
} );
// The decisive native-EH proof: the parse THROWS on the worker; the worker's try/catch is safe
// under native wasm-EH (mode-c) but crashes under JS-EH (-fexceptions). Gated to native.
test( 'm=1 throw: worker parse-throw is caught (mode-c safe under native-EH)', async ( { page, testLogger } ) => {
await page.goto( `${APP}#m=1` );
await waitForLog( testLogger, '[PRELOAD] EH=' );
test.skip( !testLogger.consoleLogs.some( l => l.includes( '[PRELOAD] EH=native' ) ),
'worker throw-on-parse (mode-c) is native-EH-only; JS-EH build skips it' );
await waitForLog( testLogger, '[PRELOAD] SUCCESS m=1' );
const r = parse( testLogger.consoleLogs, 1 )!;
expect( r.caught, 'the worker parse exception was caught' ).toBe( 1 );
expect( fatal( testLogger ), 'no mode-c crash' ).toHaveLength( 0 );
} );
test( 'm=2 shutdown: blocking-join the future mid-load completes', async ( { page, testLogger } ) => {
await page.goto( `${APP}#m=2` );
await waitForLog( testLogger, '[PRELOAD] SUCCESS m=2' );
expect( testLogger.consoleLogs.some( l => l.includes( '[PRELOAD] shutdown joined' ) ),
'the blocking join returned (no deadlock)' ).toBe( true );
expect( fatal( testLogger ), 'no crash on the shutdown join' ).toHaveLength( 0 );
} );
test( 'm=3 modal during preload: g_proxyMutex prevents the reentrancy crash', async ( { page, testLogger } ) => {
await page.goto( `${APP}#m=3` );
await waitForLog( testLogger, '[PRELOAD] SUCCESS m=3', 90000 );
expect( fatal( testLogger ), 'no table-index-out-of-bounds reentrancy crash' ).toHaveLength( 0 );
} );
} );

46
tests/e2e/coroutine-pthread-ondemand.spec.ts Normal file
View file

@ -0,0 +1,46 @@
import { test, expect } from './utils/fixtures';
// Phase 2: on-demand (NON-warm) pthread Worker creation WITHOUT modifying KiCad.
//
// The real KiCad pool (compiled-in kicad/common/thread_pool.cpp, via GetKiCadThreadPool())
// consumes ALL the pre-warmed Workers at construction; raw fly-threads beyond that count
// must then be created ON DEMAND, whose 'loaded'->'run' handshake needs the main event loop.
// The fix is wasm/shims/nanosleep_yield.c (a strong nanosleep override): the main-thread sleep_for
// join Asyncify-yields so the loop services the handshake and the on-demand Workers boot.
//
// Named coroutine-* so playwright-coroutine.config.ts runs it in real Chrome + Firefox.
// WebKit is skipped for pthread apps (COEP worker-load limitation; doc 10 §2a).
const APP = '/standalone/pthread-ondemand/pthread_ondemand_test.html';
function parse( logs: string[], mode: number ) {
const line = logs.find( l => l.includes( `[ONDEMAND] SUCCESS m=${mode}` ) );
if( !line ) return null;
const w = line.match( /workersRan=(\d+)/ );
return { workersRan: w ? +w[1] : -1 };
}
async function waitForLog( testLogger: { consoleLogs: string[] }, needle: string, timeout = 60000 ) {
await expect.poll( () => testLogger.consoleLogs.some( l => l.includes( needle ) ), { timeout } ).toBe( true );
}
test.describe( 'On-demand non-warm pthread Worker (real pool drains the pre-warmed pool)', () => {
test( 'fix: nanosleep override yields → on-demand Workers boot → multi-core', async ( { page, testLogger } ) => {
await page.goto( `${APP}#m=1` );
await waitForLog( testLogger, '[ONDEMAND] SUCCESS m=1' );
const r = parse( testLogger.consoleLogs, 1 )!;
expect( r.workersRan, 'on-demand Workers boot and run via the yielding join' ).toBeGreaterThan( 1 );
} );
// Negative control: a busy-wait join that never calls nanosleep → never yields → the
// on-demand Workers never boot. Held to the SAME bar (workersRan>1), expected to miss it,
// so it is reported as an EXPECTED failure. If it ever passes, on-demand got fixed another way.
test( 'control: busy-wait (no nanosleep) CANNOT boot on-demand Workers', async ( { page, testLogger } ) => {
test.fail();
await page.goto( `${APP}#m=0`, { waitUntil: 'domcontentloaded' } );
await waitForLog( testLogger, '[ONDEMAND] SUCCESS m=0', 20000 );
const r = parse( testLogger.consoleLogs, 0 )!;
expect( r.workersRan, 'a non-yielding busy-wait cannot create on-demand Workers' ).toBeGreaterThan( 1 );
} );
} );

48
tests/e2e/coroutine-raytrace-modal.spec.ts Normal file
View file

@ -0,0 +1,48 @@
import { test, expect } from './utils/fixtures';
// A raytracer-style worker-join run inside a wx modal pump. A pass is dispatched from a wxTimer that
// fires while a ShowModal() dialog is open; the modal pump runs ProcessEvents via ccall(async:true),
// so the work runs in a fresh managed Asyncify context at state == Normal. Both join styles complete
// multi-core there:
// m=0 busywait : sleep_for join; the pre-warmed pool completes it.
// m=1 yield : emscripten_sleep join; legal at state == Normal, so it suspends and resumes.
//
// Named coroutine-* so playwright-coroutine.config.ts runs it in real Chrome + Firefox. WebKit
// skipped for pthread apps (COEP).
const APP = '/standalone/raytrace-modal/raytrace_modal_test.html';
function workersRan( logs: string[] ): number {
const l = logs.find( x => x.includes( '[RTPOOL] SUCCESS' ) );
const m = l?.match( /workersRan=(\d+)/ );
return m ? +m[1] : -1;
}
function abortErrors( testLogger: { errors: string[] } ) {
return testLogger.errors.filter( e => /invalid state:\s*1|Aborted/i.test( e ) );
}
async function waitForLog( testLogger: { consoleLogs: string[] }, needle: string, timeout = 60000 ) {
await expect.poll( () => testLogger.consoleLogs.some( l => l.includes( needle ) ), { timeout } ).toBe( true );
}
test.describe( 'Raytracer worker-join inside a wx modal pump', () => {
test( 'm=0 busywait: pre-warmed pool busy-wait completes inside the modal → multi-core', async ( { page, testLogger } ) => {
await page.goto( `${APP}#m=0` );
await waitForLog( testLogger, '[RTPOOL] SUCCESS mode=0' );
expect( workersRan( testLogger.consoleLogs ), 'multi-core inside the modal' ).toBeGreaterThan( 1 );
expect( abortErrors( testLogger ), 'no Asyncify abort' ).toHaveLength( 0 );
} );
// The in-modal work runs in a fresh ProcessEvents entry at Asyncify state == Normal (the app
// probes and logs it), so an emscripten_sleep join is legal and the pass completes multi-core.
test( 'm=1 yield: emscripten_sleep join inside the modal → multi-core', async ( { page, testLogger } ) => {
await page.goto( `${APP}#m=1` );
await waitForLog( testLogger, '[RTPOOL] SUCCESS mode=1' );
expect( testLogger.consoleLogs.some( l => /Asyncify\.state=0/.test( l ) ),
'the in-modal work runs at state == Normal (a fresh ProcessEvents entry)' ).toBe( true );
expect( workersRan( testLogger.consoleLogs ), 'the yield-join completes → multi-core' ).toBeGreaterThan( 1 );
expect( abortErrors( testLogger ), 'no Asyncify abort' ).toHaveLength( 0 );
} );
} );

73
tests/e2e/coroutine-threadpool-real.spec.ts Normal file
View file

@ -0,0 +1,73 @@
import { test, expect } from './utils/fixtures';
// The BS-pool-API native-EH test (docs/features/wasm-exceptions/10 §6 #1).
//
// Unlike threadpool.spec.ts / coroutine-raytrace.spec.ts (which hand-roll raw std::thread),
// this drives KiCad's REAL pool: the app compiles in kicad/common/thread_pool.cpp and calls
// the actual GetKiCadThreadPool() (a BS::priority_thread_pool), with the detach_task
// __EMSCRIPTEN__ inline shim opted OUT via -DKICAD_WASM_REAL_THREADPOOL so tasks run on the
// pool's persistent pthread workers.
//
// The real pool is mode-a/b-safe by construction (persistent workers -> no on-demand spawn;
// futex busy-wait join -> no Asyncify nesting). The only native-EH risk is mode-c: a task
// that THROWS on a worker (caught by submit_task's promise wrapper ON the worker drives
// Asyncify under -fexceptions). So mode 6 is the decisive native-EH proof; modes 0-5 prove
// real multi-core (workersRan>1) across the API surface. Green => we can drop the shim.
//
// Named coroutine-* so playwright-coroutine.config.ts runs it in real Chrome + Firefox.
// WebKit is skipped for pthread apps (COEP worker-load limitation; doc 10 §2a).
const APP = '/standalone/threadpool-real/threadpool_real_test.html';
const MODES: { m: number; name: string }[] = [
{ m: 0, name: 'submit_task + vector<future> poll' },
{ m: 1, name: 'submit_loop + multi_future.wait' },
{ m: 2, name: 'submit_blocks (typed returns)' },
{ m: 3, name: 'detach_task + tp.wait()' },
{ m: 4, name: 'manual multi_future fanout by get_thread_count' },
{ m: 5, name: 'lifecycle: get_tasks_*/purge/wait + pause pool' },
];
function parse( logs: string[], mode: number ) {
const line = logs.find( l => l.includes( `[POOL] SUCCESS mode=${mode}` ) );
if( !line ) return null;
const w = line.match( /workersRan=(\d+)/ );
const c = line.match( /caught=(\d+)/ );
return { workersRan: w ? +w[1] : -1, caught: c ? +c[1] : -1 };
}
async function waitForLog( testLogger: { consoleLogs: string[] }, needle: string, timeout = 60000 ) {
await expect.poll( () => testLogger.consoleLogs.some( l => l.includes( needle ) ), { timeout } ).toBe( true );
}
test.describe( 'Real BS::thread_pool (GetKiCadThreadPool) — multi-core under native-EH', () => {
for( const { m, name } of MODES ) {
test( `mode ${m}: ${name} runs multi-core on the real pool`, async ( { page, testLogger } ) => {
await page.goto( `${APP}#m=${m}` );
await waitForLog( testLogger, `[POOL] SUCCESS mode=${m}` );
const r = parse( testLogger.consoleLogs, m )!;
expect( r.workersRan, 'tasks must run on >1 pool worker' ).toBeGreaterThan( 1 );
expect( testLogger.errors.filter( e => !e.includes( 'favicon' ) ), 'no runtime errors' ).toHaveLength( 0 );
} );
}
// mode-c: a task throws ON a worker; submit_task's promise wrapper catches it on the
// worker (drives Asyncify under -fexceptions -> "func is not a function" crash) and
// rethrows on main. Native wasm-EH decouples exceptions from Asyncify, so this must
// complete cleanly. (Red under JS-EH, green under native-EH — the contrast IS the proof.)
test( 'mode 6: throw on a worker is safe under native-EH and rethrows on main', async ( { page, testLogger } ) => {
await page.goto( `${APP}#m=6` );
// A worker throw is a mode-c crash under JS-EH and only safe under native wasm-EH, so this
// assertion is native-EH-only. The app reports its EH model early; skip on a JS-EH build
// (builds are always native-EH now, so this never skips) rather than asserting a crash.
await waitForLog( testLogger, '[POOL] EH=' );
test.skip( !testLogger.consoleLogs.some( l => l.includes( '[POOL] EH=native' ) ),
'throw-on-worker (mode-c) is native-EH-only; JS-EH build skips this assertion' );
await waitForLog( testLogger, '[POOL] SUCCESS mode=6' );
const r = parse( testLogger.consoleLogs, 6 )!;
expect( r.caught, 'the worker throw must rethrow + be caught on main' ).toBe( 1 );
expect( r.workersRan, 'workers still run a normal batch after the throw' ).toBeGreaterThan( 1 );
expect( testLogger.errors.filter( e => !e.includes( 'favicon' ) ), 'no mode-c crash' ).toHaveLength( 0 );
} );
} );

39
tests/kicad/pcbnew-collab.spec.ts
View file

@ -266,6 +266,45 @@ test.describe("pcbnew collab bridge — single page", () => {
});
}
// REGRESSION (native-EH vtable-slot skew, task #54): a second consecutive collab apply must also
// take effect. Root cause: the embind TU was compiled without -DDEBUG while the core TU (CMake
// Config=Debug) had it, so a DEBUG-gated virtual (EDA_ITEM::Show) occupied a vtable slot the embind
// didn't account for, shifting every later slot by one. Embind virtual calls in the apply
// (SetWidth/GetPosition/the rebaseline snapshot getters) then read the wrong slot → call_indirect
// signature-mismatch trap, swallowed by the apply coroutine's catch_all → silent loop, so the apply
// never completed. Fixed by building the embind TU with -DDEBUG in Debug builds
// (scripts/kicad/build-kicad-target.sh). RED before the fix (move times out), GREEN after.
test("native-EH: a second consecutive collab apply also takes effect", async ({ page, testLogger }) => {
await bootAndOpen(page, "apply");
const before = await page.evaluate((id) => window.Module.kicadCollabGetPos(id), SEG1);
const [bx, by] = before.split(",").map(Number);
const moveTo = async (x: number) => {
await page.evaluate(
({ id, x, by }) =>
window.Module.kicadCollabApply(
JSON.stringify({
changed: [{ id, type: "PCB_TRACK", sx: x, sy: by, ex: x + 50_800_000, ey: by, width: 200000 }],
added: [],
removed: [],
}),
),
{ id: SEG1, x, by },
);
await expect
.poll(() => page.evaluate((id) => window.Module.kicadCollabGetPos(id), SEG1), {
timeout: 10000,
intervals: [200],
})
.toBe(`${x},${by}`);
};
await moveTo(bx + 5_000_000); // 1st apply
await moveTo(bx + 8_000_000); // 2nd apply — RED before the -DDEBUG fix (apply mis-dispatches), GREEN after
expect(hasAbort(testLogger), "no WASM abort").toBe(false);
});
// Apply mutates the model headless: kicadOpenFile returns false (the incomplete-project load
// skips some late steps) but the board IS built, so BOARD_COMMIT::Push takes effect. Rendering
// still needs the real app. (Same headless reality as the eeschema apply test.)

4
tests/package.json
View file

@ -39,7 +39,9 @@
"test:coroutine:firefox": "playwright test --config=playwright-coroutine.config.ts --project=firefox",
"test:coroutine:chrome": "playwright test --config=playwright-coroutine.config.ts --project=chromium --headed",
"test:asyncify:firefox": "playwright test --config=playwright-asyncify.config.ts --project=firefox",
"test:asyncify:chrome": "playwright test --config=playwright-asyncify.config.ts --project=chromium --headed"
"test:asyncify:chrome": "playwright test --config=playwright-asyncify.config.ts --project=chromium --headed",
"test:asyncify:safari": "playwright test --config=playwright-asyncify.config.ts --project=webkit",
"test:asyncify:all": "npm run test:asyncify:firefox && npm run test:asyncify:safari && npm run test:asyncify:chrome"
},
"devDependencies": {
"@playwright/test": "^1.40.0",

6
tests/playwright-asyncify.config.ts
View file

@ -70,6 +70,12 @@ export default defineConfig({
permissions: ['clipboard-read', 'clipboard-write'],
},
},
{
// WebKit (Safari's engine) — headless OK on macOS. Project policy: every spec must be
// green in all three engines (Firefox + Chrome + Safari). Run via npm run test:asyncify:safari.
name: 'webkit',
use: { ...devices['Desktop Safari'], viewport: { width: 1280, height: 720 } },
},
],
webServer: {

3
tests/serve.json
View file

@ -4,7 +4,8 @@
"source": "**/*",
"headers": [
{ "key": "Cross-Origin-Opener-Policy", "value": "same-origin" },
{ "key": "Cross-Origin-Embedder-Policy", "value": "require-corp" }
{ "key": "Cross-Origin-Embedder-Policy", "value": "require-corp" },
{ "key": "Cross-Origin-Resource-Policy", "value": "cross-origin" }
]
}
]

BIN
tests/wizard-04-finish-headless.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 118 KiB

After

Width:  |  Height:  |  Size: 156 KiB

Before After
Before After

52
wasm/shims/nanosleep_yield.c Normal file
View file

@ -0,0 +1,52 @@
/*
* nanosleep_yield.c — make a MAIN-THREAD nanosleep() YIELD to the JS event loop via
* Asyncify instead of busy-spinning, so on-demand pthread-Worker creation can complete
* WITHOUT editing KiCad.
*
* THE PROBLEM (the "non-warm thread" deadlock): once KiCad's thread pool has consumed all
* the pre-warmed Workers (-sPTHREAD_POOL_SIZE), a later raw std::thread (the raytracer)
* must spawn a NEW Worker on demand. Finalizing it needs the main thread's event loop to
* run the new-Worker 'loaded' -> 'run' handshake. KiCad's join is a sleep_for() busy-wait
* -> nanosleep -> emscripten_thread_sleep, which busy-spins and NEVER returns to the JS
* event loop, so the new Worker never starts -> deadlock.
*
* THE FIX: the only main-thread primitive that returns to the event loop is an Asyncify
* unwind (emscripten_sleep). This provides a nanosleep that, ON THE MAIN THREAD, yields via
* an EM_ASYNC_JS await (= emscripten_sleep semantics; __asyncjs__* is already in the
* post-link asyncify-imports). The unmodified sleep_for busy-wait then pumps the loop, the
* Worker handshake completes, and on-demand creation works with no KiCad edit.
*
* MECHANISM: a STRONG definition of nanosleep here SHADOWS musl's archive member — the
* linker only pulls musl's nanosleep.o if the symbol is left undefined, and ours defines it.
* (-Wl,--wrap=nanosleep is not an option here — it crashes wasm-ld with a SIGSEGV in
* lld::wasm::ImportSection::addImport.) On a pthread worker we fall back to
* emscripten_thread_sleep (the real underlying blocking sleep — workers may block).
*
* SCOPE: only the main browser thread yields; only it must never block the event loop.
*/
#include <emscripten/emscripten.h>
#include <emscripten/threading.h>
#include <time.h>
/* EM_ASYNC_JS integrates with Asyncify automatically (binaryen instruments every caller). */
EM_ASYNC_JS( void, __wasm_main_thread_yield_ms, ( double ms ), {
await new Promise( function( resolve ) { setTimeout( resolve, ms ); } );
} );
int nanosleep( const struct timespec* req, struct timespec* rem )
{
if( req )
{
double ms = (double) req->tv_sec * 1000.0 + (double) req->tv_nsec / 1.0e6;
if( emscripten_is_main_runtime_thread() )
__wasm_main_thread_yield_ms( ms ); /* yield -> event loop runs -> Worker boots */
else
emscripten_thread_sleep( ms ); /* worker: real blocking sleep */
}
if( rem )
{
rem->tv_sec = 0;
rem->tv_nsec = 0;
}
return 0;
}

2
wxwidgets

@ -1 +1 @@
Subproject commit cca8eed9f965cd771c0cdfb31af692a27ff80eb4
Subproject commit 67f28fb3354a1830f004c989db9638205816c665