feat(wasm-eh): migrate the WASM build to native wasm exceptions (+ 3D viewer default-on)
Replace the legacy Emscripten JS-exceptions model with native wasm-EH (legacy encoding) across the whole build, keeping Asyncify coroutines working via a from-source Binaryen --hoist-cpp-catches pre-pass. Net result: native-EH is the only build mode, the 3D viewer is on by default, and pcbnew shrinks substantially. Highlights: - Binaryen submodule everywhere + --hoist-cpp-catches integration in apply-asyncify; post-link Asyncify covers every app wasm (not just standalone test wasm). - Build deps (incl. OpenCASCADE without OCC_CONVERT_SIGNALS) and all KiCad apps with -fwasm-exceptions; emscripten_sleep added to the post-link asyncify-imports. - libcontext fiber entry wired under native exceptions; while-loop main loop + currData shim injected into all wx apps. - Native-EH collab apply fixed: DEBUG-define the embind TU + match all out-of-CMake C++ TUs' ABI flags to the core, fixing the vtable-layout skew / mis-dispatch. - 3D viewer enabled by default (real raytracer linked, not the stub). - Retire the EH-spike scaffolding; flip the asyncify-races ablation pins to shim-redundancy pins (native-EH stays clean with the legacy shims ablated). - Fix the asyncify-races quiescence check to not require Asyncify.currData==0: under the native-EH per-frame-yield top loop the main stack is asyncify-suspended every frame, so currData legitimately churns (a freed-but-not-yet-nulled buffer, not a leak). Refresh the pcbnew toolbar screenshot baseline for the new kicad. - CI: drop the obsolete binaryen_version input/env (the build uses the binaryen submodule fork's wasm-opt, not a version download); key the wasm-output cache on the binaryen submodule SHA instead. Bumps the wxwidgets + binaryen submodules to their squashed feature commits. Validated green: all 7 apps native-EH (real 3D in pcbnew); KiCad e2e 63/63 Firefox + Chromium (3D viewer renders); wx 336; coroutine 34/34 both engines; asyncify 7/7 both engines. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Viktor Vaczi
parent
b8c8dee355
commit
c1ef489cfa
75 changed files with 4976 additions and 680 deletions
|
|
@ -1,5 +1,7 @@
|
|||
# 02 — The machine: Asyncify internals and control flows
|
||||
|
||||
> **STATUS (2026-06-23):** the top-level `emscripten_set_main_loop(...,1)` `throw "unwind"` this doc treats as current **is gone** — it was fatal under native wasm-EH and was replaced by the Asyncify **de-park** rAF pump (`wxWasmParkMainLoop`); see [`../wasm-exceptions/09`](../wasm-exceptions/09-event-loop-deparking-plan.md). The de-park regressed the coroutine suite, and **Design B is now being built to fix it** ([`12`](12-design-b-asyncify-implementation-plan.md) + [`13`](13-design-b-engineering-spec.md)). Read below as the pre-de-park analysis (the internals are still accurate).
|
||||
|
||||
This is the legible model: what suspends, who owns the single slot, and exact line-by-line
|
||||
control flow for the park, the hang, the crash, and **de-parking**.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
# 04 — How the fixes relate, the test matrix, open questions
|
||||
|
||||
> **STATUS (2026-06-23):** the top-level `set_main_loop(...,1)` `throw "unwind"` treated as current here **is gone** — replaced by the Asyncify **de-park** rAF pump (fatal under native wasm-EH; see [`../wasm-exceptions/09`](../wasm-exceptions/09-event-loop-deparking-plan.md)). The de-park regressed the coroutine suite, and **Design B is now being built to fix it** ([`12`](12-design-b-asyncify-implementation-plan.md) + [`13`](13-design-b-engineering-spec.md)). Read below as the pre-de-park analysis.
|
||||
|
||||
> The goal is **one universal mechanism**, not patches scattered around. This file classifies the
|
||||
> candidate fixes by *root cause* so it's clear what is part of the one solution, what is
|
||||
> subsumed, and what is genuinely separate.
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
# 05 - Design A: JS Asyncify arbiter
|
||||
|
||||
> **STATUS (2026-06-23):** the throw-based top loop assumed here **is gone** (de-park; [`../wasm-exceptions/09`](../wasm-exceptions/09-event-loop-deparking-plan.md)). This arbiter is the **core of Design B's scheduler**, now being implemented — the de-park created the red scenario (coroutine regression) that 07/D3 said this arbiter lacked. See [`12`](12-design-b-asyncify-implementation-plan.md) + [`13`](13-design-b-engineering-spec.md); this doc's arbiter design is reused there.
|
||||
|
||||
> Goal: fix the current system with the smallest architectural move. Keep
|
||||
> `EM_ASYNC_JS` modal/clipboard/font calls and Emscripten fibers, but introduce one JS-side
|
||||
> authority that owns `Asyncify.currData`, `Asyncify.state` transitions, the fiber trampoline,
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
# 06 - Design B: fiber-first async runtime
|
||||
|
||||
> **STATUS (2026-06-23):** Design B is **now being implemented** on Asyncify — see [`12`](12-design-b-asyncify-implementation-plan.md) (plan/phases/test-matrix) and [`13`](13-design-b-engineering-spec.md) (engineering spec/work log). The de-park ([`../wasm-exceptions/09`](../wasm-exceptions/09-event-loop-deparking-plan.md)) replaced the top-level `throw` with an Asyncify park and regressed the coroutine suite — the red scenario this design fixes. External research (Ruby-WASM, Julia-WASM, Qt-for-WASM) confirms the fiber-scheduler is the proven path.
|
||||
|
||||
> Goal: make the architecture conceptually cleaner by reducing the number of suspension
|
||||
> primitives. Instead of having tool coroutines use fibers while modal/clipboard/font/nested loops
|
||||
> use `EM_ASYNC_JS` sleeps, put every blocking-looking operation onto a fiber-like runtime and let
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
# 07 — Decisions and outcome (2026-06-12)
|
||||
|
||||
> **STATUS (2026-06-23):** **D4 (kept the throw-based main-loop park) has been reversed.** Native wasm-EH made the `throw "unwind"` fatal (its catch_all cleanup destroys the main frame), so the top loop is now the Asyncify **de-park** ([`../wasm-exceptions/09`](../wasm-exceptions/09-event-loop-deparking-plan.md)). That de-park regressed the coroutine suite — the red scenario D3 said the arbiter lacked — so **Design B is now being built** ([`12`](12-design-b-asyncify-implementation-plan.md) + [`13`](13-design-b-engineering-spec.md)). The D1–D5 outcomes below were correct for the JS-EH / throw world.
|
||||
|
||||
> The dossier (01–06) ended with designs and open questions. This file records what was
|
||||
> actually decided, built, and deliberately NOT built — and the trigger conditions for
|
||||
> revisiting each road not taken. Working artifacts: `docs/features/asyncify-arbiter/`
|
||||
|
|
|
|||
103
docs/features/async/12-design-b-asyncify-implementation-plan.md
Normal file
103
docs/features/async/12-design-b-asyncify-implementation-plan.md
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
# 12 — Design B on Asyncify: implementation plan to make suspensions compose
|
||||
|
||||
> How to realize the dossier's **Design B** ([`06`](06-design-b-fiber-first-runtime.md)) on the
|
||||
> **current Asyncify toolchain — no JSPI**. The goal is concrete: make the parked main loop, modal
|
||||
> dialogs, nested loops, clipboard/font waits, and **tool coroutines** all coexist without
|
||||
> corrupting Asyncify's single suspension slot, so the de-park's coroutine regression goes green —
|
||||
> under both `-fexceptions` and `-fwasm-exceptions`. Builds on the internal audit (current building
|
||||
> blocks, the gap) and Design A ([`05`](05-design-a-js-asyncify-arbiter.md)).
|
||||
|
||||
## Why now — the red scenario the dossier didn't have
|
||||
|
||||
Doc 07/D3 shelved Design A's arbiter because *"no scenario could be made red that it would fix"* — at production semantics the per-sleep `handlesleep.js` capture already satisfied the core invariant. **The de-park changed that.** Measured in `wasm-exceptions/09`: config 3 (JS-EH + de-park) **fails** the 6 coroutine tests that config 1 (no de-park) **passes**. We now have a deterministic red test that only a scheduler fixes. The de-park and Design B are **coupled**: the de-park is *required* for native-EH (the `throw "unwind"` is fatal under wasm-EH catch_all), it breaks coroutines, and Design B is the fix.
|
||||
|
||||
## The physics we must obey (why this is hard)
|
||||
|
||||
Asyncify is **one** `Asyncify.currData` (active save-buffer pointer) + **one** `Asyncify.state` (`Normal`/`Unwinding`/`Rewinding`). The law: **at most one unwind-or-rewind in flight at a time**; it must begin at `state==Normal` and fully complete before the next. But **many contexts may be *parked* at once**, each holding its own durable buffer (a parked context = `state Normal`, its stack saved in *its* buffer, waiting for a wake).
|
||||
|
||||
- **Fibers** (KiCad tool coroutines via libcontext) already give per-context buffers (`wasm_fcontext.asyncify_stack`, 64 KB each) — durable storage is fine. But the *act* of swapping still drives the single global register.
|
||||
- **`handleSleep`/`handleAsync`** (modals, clipboard, the de-park park, the per-tick `ccall`) take buffers from `Asyncify.allocateData`; only the live `currData` register remembers a parked one — `handlesleep.js` patches this for **one** level of sleep nesting, and is **blind to `handleAsync` and to fibers**.
|
||||
|
||||
**The de-park bug, precisely:** `wxWasmParkMainLoop` is a `handleAsync` suspend that is **live for the app's whole life**, and each rAF tick's `await ccall('ProcessEvents',{async:true})` is a second `handleAsync` suspend. A coroutine fiber-swap is then a **third** unwind, attempted while the slot is dirty / `state != Normal` → `Aborted(invalid state: 1)`. Three uncoordinated writers of one slot.
|
||||
|
||||
## Architecture: one scheduler owns the slot; everything is a context
|
||||
|
||||
The universal rule (06): **no API touches `Asyncify` directly. APIs ask the scheduler to park/wake contexts. The scheduler alone performs Asyncify transitions.**
|
||||
|
||||
```
|
||||
Scheduler (JS) — the single authority
|
||||
owns: Asyncify.currData, Asyncify.state, the fiber trampoline
|
||||
registry: ctx = { id, kind: main|modal|nested|coroutine|sleep, buffer, status, wakeReason, result }
|
||||
readyQueue + drain():
|
||||
a wake event marks a ctx READY (it does NOT rewind directly)
|
||||
drain() resumes the next ready ctx ONLY when state==Normal && no transition in flight
|
||||
transitions (the only code that writes currData):
|
||||
park(ctx) = set currData=ctx.buffer; start_unwind; (slot now free, ctx parked)
|
||||
resume(ctx) = set currData=ctx.buffer; start_rewind; doRewind
|
||||
```
|
||||
|
||||
Every blocking-looking thing becomes a context that *yields* and is later *resumed*. A coroutine swap becomes "park ctx A, resume ctx B" — a normal scheduler operation serialized with the main loop and modals, exactly the doc-11 cure: *"if the pump and render yields were both scheduler-owned fiber contexts, 'render yields while the pump is parked' becomes a normal context switch instead of an illegal nested unwind."*
|
||||
|
||||
## The gap — what to build (from the internal audit)
|
||||
|
||||
None of these exist today: (1) a single owner of `currData`/`state`/the trampoline; (2) a scheduler-owned **fiber** context for the main loop + pump (today it's a `handleAsync` park, not a fiber); (3) a deferred-wakeup ready-queue/drain (today wakeups call `doRewind` inline); (4) **`handleAsync` coverage** (the park + the per-tick ccall are entirely unprotected); (5) sleep contexts promoted from "restore one pointer" to "registered context"; (6) trampoline ownership as a scheduler invariant; (7) a lifetime owner coordinated with `currData` management (the de-park gave us D1/lifetime without D2/ownership).
|
||||
|
||||
## Phased implementation (each phase gated by the red harness)
|
||||
|
||||
**Phase 0 — Red-green harness (2–3 days).** Make the coroutine regression a deterministic, minimal red test in `tests/asyncify/` (and a CPP test app): a tool-style fiber swap *while the main loop is parked* and *while a modal pump is live*. Reproduce `invalid state: 1` reliably in all three engines. This is the acceptance gate for every later phase. Also fold the 6 failing `coroutine*` specs in as the integration gate.
|
||||
|
||||
**Phase 1 — The scheduler core (1–2 weeks). The likely coroutine fix.** Extend `scripts/common/shims/handlesleep.js` into the `AsyncifyArbiter` of doc 05, but covering everything the de-park introduced:
|
||||
- Own `Asyncify.currData`/`state` + `Fibers.trampoline`; make `currData` a *derived* register set only inside a managed transition; the registry records are the truth.
|
||||
- **Register `handleAsync`** (wrap it as `handleSleep` is wrapped) so the de-park park and the per-tick `ccall` are tracked contexts, not invisible slot-writers.
|
||||
- **Track fiber buffers** at `_emscripten_fiber_swap` (`oldFiber+20`/`newFiber+20`) so a coroutine swap is a managed transition.
|
||||
- **Deferred-wakeup `drain()`** with *explicit* completion signals from `stop_rewind`/`maybeStopUnwind` (not JS `finally` — `doRewind` can re-enter and unwind again before returning).
|
||||
- Keep it JS-only — **no C++ restructuring yet.** Build, run Phase 0. If green, the coroutine regression is fixed at lowest risk. If still red (the permanent `handleAsync` park can't be made a clean parked context), escalate to Phase 2.
|
||||
|
||||
**Phase 2 — Root fiber for the main loop (≈1 week). The clean cure.** Replace the `handleAsync` park with a scheduler-owned **fiber**: run `main → wxEntry → OnRun → DoRun` inside a managed root fiber (06's B2). The main loop *yields its fiber* to the scheduler instead of `handleAsync`-parking. Crucially, **drive `ProcessEvents` from the wasm-side scheduler (the root fiber calls it directly), not the current JS-side `await ccall('ProcessEvents',{async:true})`** — that JS-awaits-a-suspending-export boundary is the Emscripten #13302 corruption hazard (see Prior art). The rAF/`setTimeout` tick just resumes the root fiber (or returns through `set_main_loop`, Ruby/Julia-style, to keep the top off Asyncify entirely). Now the main loop is a sibling context to the coroutines and modals — no permanent `handleAsync` occupant, every swap is fiber↔fiber under the scheduler. This is the definitive fix if Phase 1's "park-as-context" proves fragile.
|
||||
|
||||
**Phase 3 — Migrate the waits to one yield API (1–2 weeks). The full Design B.** Add the C++ API and route the ad-hoc suspends through it:
|
||||
```cpp
|
||||
WAKE_TOKEN wasm_begin_async_wait(...);
|
||||
int wasm_yield_until(WAKE_TOKEN); // park current ctx, run scheduler
|
||||
void wasm_resolve_wait(WAKE_TOKEN, int result); // mark ctx ready
|
||||
```
|
||||
Reimplement `wxDialog::ShowModal` (`dialog.cpp` `startModal`), `wxGUIEventLoop` nested `DoRun` (`evtloop.cpp` `wxWasmRunNestedLoop`), clipboard, and font enum as `yield_until` waits. Removes the second suspension family entirely; the LIFO resolver stacks (`_wxModalResolvers`, `_wxNestedLoopExit`) become scheduler ready/wait bookkeeping.
|
||||
|
||||
**Phase 4 — Lifetime + cleanup (few days).** Coordinate the de-park's lifetime (D1) with the scheduler (D2): the browser/scheduler owns app lifetime; `wxEntryCleanupReal`/`OnExit` deferred to real exit/unload; `emscripten_cancel_main_loop` + teardown ordered after the root fiber resolves. (Already half-done in `evtloop.cpp` `ScheduleExit`.)
|
||||
|
||||
## Test matrix (doc 06 + nesting + EH)
|
||||
|
||||
Each asserts **no crash, no hang, correct return value, app stays interactive, no cleanup during steady-state pumping** — and runs in **Firefox + Chrome + Safari** and under **both `-fexceptions` and `-fwasm-exceptions`**:
|
||||
- the 6 regressed `coroutine`/`coroutine-nested`/`coroutine-pthread` specs (the gate);
|
||||
- `ShowModal` from root, from a tool coroutine; `ShowQuasiModal` from a coroutine; nested modal inside quasi-modal;
|
||||
- coroutine swap **while** a modal pump is live (the Phase-0 red test);
|
||||
- clipboard read from root and from a coroutine; font enum during startup;
|
||||
- the **raytracer** threading suite (the doc-11 nesting wall) multi-core;
|
||||
- exit/unload cleanup after parked contexts exist;
|
||||
- a `-sASYNCIFY_ASSERTIONS=1` pass + a production `-sASSERTIONS=0` pass (the dossier's "production semantics already satisfy the invariant" claim must be re-validated post-de-park).
|
||||
|
||||
## Risks + mitigations
|
||||
|
||||
- **A partial arbiter is worse than none** (doc 05) — one path still writing `currData` behind the scheduler corrupts silently. → enumerate *every* `currData` writer (`handleSleep`, `handleAsync`, `_emscripten_fiber_swap`, `finishContextSwitch`, the park, the ccall), route all through the scheduler, assert on stray writes in dev builds.
|
||||
- **Trampoline wedge** (`Fibers.trampolineRunning` stuck after a mid-flight unwind). → scheduler *owns* the trampoline; keep the `inject-dyncall-shims §3c` self-heal as a belt-and-suspenders.
|
||||
- **Lifetime cleanup too early** → Phase 4 ordering; defer wx teardown to unload.
|
||||
- **Reentrancy / out-of-order resolution** → explicit tests; keep wx modal-disabling semantics.
|
||||
- **Native-EH coexistence** — the scheduler and the `HoistCppCatches` pass must compose (suspend-inside-catch under the scheduler). → test the whole matrix under `-fwasm-exceptions`, including a modal opened from inside a `catch`.
|
||||
- **Starvation** → FIFO ready-queue; diagnostics for context age.
|
||||
|
||||
## Effort
|
||||
|
||||
Phase 0 ≈ 2–3 d · Phase 1 ≈ 1–2 wk · Phase 2 ≈ 1 wk · Phase 3 ≈ 1–2 wk · Phase 4 ≈ few d. **Coroutine fix = Phase 0–1 (+2 if needed) ≈ 2–3 wk; full Design B ≈ 4–6 wk** including the test matrix. Phase 1 is the high-value, lowest-risk step and may suffice on its own.
|
||||
|
||||
## Prior art (external research)
|
||||
|
||||
**The Asyncify fiber scheduler (this plan) is the proven path** for "event loop + blocking `ShowModal` + green threads" — it ships in real runtimes, and the comparisons sharpen two implementation details.
|
||||
|
||||
- **Qt for WebAssembly** (closest analog) uses a deliberate **two-tier** scheme: top-level `QApplication::exec()` uses Emscripten's `simulateInfiniteLoop` throw — which keeps the top loop **off Asyncify so the single slot stays free** — while `QDialog::exec()`/nested `QEventLoop::exec()` consume the one Asyncify slot. **This is the inverse of what our de-park did** (the de-park made the *top* loop a live Asyncify occupant, consuming the slot — exactly why coroutines broke). Qt's scheme caps at *one* modal at a time and the Qt team calls Asyncify "not quite scaling to Qt-sized software" — i.e. a single-slot scheme *without* a real scheduler hits a wall; the per-fiber-buffer scheduler (Design B) is the way past it. [Qt commit 6d039a5e; Qt dev ML, June 2024]
|
||||
- **Ruby-WASM / Julia-WASM** implement this design directly: a **root fiber that *is* the browser event loop**; tasks/coroutines are fibers, each with its own C stack + `asyncify_data` buffer; the scheduler resumes the next ready fiber; "yield to browser" is `emscripten_sleep(0)` *or* returning through the `set_main_loop` callback (the latter keeps the top off Asyncify entirely). That is precisely Design B's B2 root-fiber — already shipping in production runtimes. [Julia PR #32532; Emscripten fiber PR #9859]
|
||||
- **Pyodide** pre-JSPI used stackless CPS (`WebLoop` + `setTimeout(0)` per task) — not retrofittable to C++; post-JSPI uses per-`promising`-entry stacks plus explicit **spill-stack** save/restore.
|
||||
- **Dart/Flutter, Blazor, Unity** all use compiler-lowered stackless state machines — not applicable to a C++ toolkit.
|
||||
|
||||
**Two findings that sharpen the plan:**
|
||||
1. **The JS-boundary async-return hazard (Emscripten #13302):** returning a value to JS from a wasm export that internally `fiber_swap`s is broken — *"within-wasm scheduling is fine; the JS-awaits-a-suspending-wasm-export boundary is not."* Our per-tick `await ccall('ProcessEvents',{async:true})` is exactly that boundary. **Design B must drive `ProcessEvents` from the wasm-side scheduler (the root fiber), not via a JS async `ccall`** — folded into Phase 2 below.
|
||||
2. **JSPI would not have helped this case anyway** (independently confirming the decision to scratch it): the wit-bindgen analysis shows that when the whole scheduler lives inside one app context, *a single `promising` root = a single suspension unit* — JSPI gives no fiber-multiplexing benefit unless each fiber is separately surfaced as a `promising` export (awkward; Chrome also showed a ~350× per-suspension penalty on the JS→wasm path). Pattern-C / Asyncify is the right tool regardless of JSPI availability.
|
||||
163
docs/features/async/13-design-b-engineering-spec.md
Normal file
163
docs/features/async/13-design-b-engineering-spec.md
Normal file
|
|
@ -0,0 +1,163 @@
|
|||
# 13 — Design B: engineering spec & work log
|
||||
|
||||
> The granular, file-by-file implementation spec for [`12`](12-design-b-asyncify-implementation-plan.md)
|
||||
> (the plan/phases/test-matrix). This document is the **engineering design + per-phase checklists**,
|
||||
> and is updated as a **work log** as the phases land. Build it on Asyncify — no JSPI.
|
||||
|
||||
## 0. State of the world (2026-06-23)
|
||||
|
||||
- The **de-park is live**: `wxwidgets/src/wasm/evtloop.cpp` `DoRun` at depth 0 calls `wxWasmParkMainLoop()` (an `EM_ASYNC_JS`/`Asyncify.handleAsync` suspend driving an rAF `await ccall('ProcessEvents',{async:true})` pump). The old `emscripten_set_main_loop(...,1)` throw is gone (it was fatal under native wasm-EH; see `wasm-exceptions/08`+`09`).
|
||||
- **Measured regression** (`wasm-exceptions/09`): config 1 (no de-park) passes all; config 3 (JS-EH + de-park) fails the 6 `coroutine`/`coroutine-nested`/`coroutine-pthread` specs; config 2 (native-EH + de-park) fails those 6 + raytracer(5) + main-app(10). The **6 coroutine failures are the de-park's**, both EH models.
|
||||
- **The gate** for this work = those 6 specs going green again, plus a minimal unit repro (Phase 0).
|
||||
- **Stale code to clean up in Phase 1:** `scripts/common/shims/handlesleep.js` still has the `"unwind"`-sentinel swallow (lines ~57-68) referencing `set_main_loop(...,1)` — dead under the de-park; the rewrite subsumes it.
|
||||
|
||||
## 1. The scheduler — JS design (the heart of the fix)
|
||||
|
||||
### 1.1 What exists today (`handlesleep.js`)
|
||||
Per-sleep `currData` capture/restore for **one** level of nesting, **`handleSleep` only**:
|
||||
- Wraps `Asyncify.allocateData` to record which buffer pointer the active `handleSleep` allocated (`ctx.capturedData`).
|
||||
- In the `wakeUp` callback, restores `Asyncify.currData = ctx.capturedData` before `handleSleep` does `_asyncify_start_rewind`+`doRewind`, so a fiber swap that clobbered the slot during the `await` doesn't make the sleep rewind the wrong buffer.
|
||||
- **Blind to `handleAsync`** (the de-park park + the per-tick ccall) **and to fibers** (libcontext buffers don't come from `allocateData`).
|
||||
|
||||
### 1.2 The scheduler object (`AsyncifyScheduler`, replaces the shim)
|
||||
A single JS authority that is the **only** writer of `Asyncify.currData` during managed transitions. State:
|
||||
```
|
||||
contexts: Map<id, ctx> // every parked/running suspendable thing
|
||||
readyQueue: id[] // FIFO of contexts whose wake fired
|
||||
running: id | null // the one context currently executing
|
||||
transitionRunning: bool // an unwind or rewind is in flight
|
||||
trampolineRunning: bool // a fiber-swap trampoline is mid-flight
|
||||
```
|
||||
`ctx = { id, kind: 'main'|'modal'|'nested'|'coroutine'|'sleep', buffer /*dataPtr*/, status: 'running'|'parked'|'ready', wakeReason, result, cancel }`.
|
||||
|
||||
**Governing rule (from doc 05):** `Asyncify.currData` is *not* durable state — it is a register loaded from the current context only at the instant of a managed transition. The `contexts` records are the truth. **Many parked; at most one unwinding-or-rewinding.**
|
||||
|
||||
### 1.3 The four hooks (intercept every `currData` writer)
|
||||
1. **`Asyncify.handleSleep`** — register a `sleep` ctx (today's capture), but route its wakeup through `drain()` (below), not an inline `doRewind`.
|
||||
2. **`Asyncify.handleAsync`** — wrap it the same way. **New, load-bearing:** the de-park park and the per-tick `await ccall` are `handleAsync`; they must be tracked contexts, not invisible slot-writers.
|
||||
3. **`_emscripten_fiber_swap`** — *track* (not allocate) the per-fiber buffers `oldFiber+20` / `newFiber+20` so a coroutine swap is a managed transition the scheduler knows about.
|
||||
4. **`Fibers.trampoline`** — own it (and the `trampolineRunning` guard); keep `inject-dyncall-shims §3c` self-heal as backstop.
|
||||
|
||||
### 1.4 The transitions (the only code that writes `currData`)
|
||||
```
|
||||
park(ctx): assert state==Normal; currData=ctx.buffer; start_unwind // ctx now parked, slot free
|
||||
resume(ctx): assert state==Normal; currData=ctx.buffer; start_rewind; doRewind
|
||||
drain(): if (transitionRunning || trampolineRunning || state!=Normal || !readyQueue.length) return;
|
||||
resume(contexts[readyQueue.shift()])
|
||||
```
|
||||
**Deferred wakeup:** a Promise/event resolution **marks a ctx ready and calls `scheduleDrain()`** — it never calls `doRewind` inline (because `doRewind` can re-enter wasm and unwind again before returning). `drain` runs only when the slot is provably free, and receives **explicit transition-completion signals** by wrapping `_asyncify_stop_rewind` / `Asyncify.maybeStopUnwind` (clear `transitionRunning`, then `scheduleDrain()`), not a JS `finally`.
|
||||
|
||||
### 1.5 Invariants (assert in dev builds; doc 05 §invariants)
|
||||
(1) only the scheduler writes `currData` during managed transitions; (2) ≤1 context unwinding-or-rewinding; (3) Promise resolution never `doRewind`s directly while a transition runs; (4) `currData` may be null while contexts are parked — records are truth; (5) every `allocateData`/fiber buffer belongs to exactly one ctx; (6) the scheduler owns the trampoline; (7) a parked ctx's buffer is never reused until it resumes-and-completes; (8) FIFO readyQueue (no starvation).
|
||||
|
||||
> **Correction (verified in the glue during Phase 0, 2026-06-23):** `handleAsync` routes through the wrapped `handleSleep` — `handleAsync(fn) = handleSleep(wakeUp => fn().then(wakeUp))` (`coroutine_test.js:9989`) — so the shim **already covers** the de-park park (`wxWasmParkMainLoop`) and the per-tick `await ccall`. §1.3's "hook 2 (handleAsync) is load-bearing/new" is therefore **wrong**: no separate `handleAsync` hook is needed. The genuine *uncovered* `currData` writer is the **fiber swap** (libcontext buffers come from `emscripten_fiber_init`, not `allocateData`). **So Phase 1's scheduler should focus on fiber tracking (§1.3 hook 3) + the deferred drain (§1.4) + single-transition serialization — not handleAsync coverage.** The Phase-0 red gate is confirmed (6 `coroutine*` specs fail on the de-park build); this sharpens where the fix lives.
|
||||
|
||||
## 2. The C++ yield API (Phase 3 surface)
|
||||
```cpp
|
||||
using WAKE_TOKEN = int;
|
||||
WAKE_TOKEN wasm_begin_async_wait(int kind); // EM_JS → scheduler.beginWait(kind) → token
|
||||
int wasm_yield_until(WAKE_TOKEN token); // EM_ASYNC_JS → park current ctx, return result on resume
|
||||
void wasm_resolve_wait(WAKE_TOKEN, int); // EM_JS → mark ctx ready + scheduleDrain
|
||||
```
|
||||
Reimplement each wait on top of it: `wxDialog::ShowModal` (replaces `dialog.cpp:startModal` `EM_ASYNC_JS` + `_wxModalResolvers`), `wxGUIEventLoop` nested `DoRun` (replaces `evtloop.cpp:wxWasmRunNestedLoop`), `wxClipboard::GetData`, font enum. Each becomes "begin wait → yield_until → (JS event) resolve_wait". The existing LIFO resolver stacks fold into scheduler ready/wait bookkeeping.
|
||||
|
||||
## 3. The root fiber (Phase 2, B2)
|
||||
Run `main → wxEntry → OnRun → DoRun` inside a managed **root fiber** (via libcontext's `emscripten_fiber_init_from_current_context`, already used for the coroutine main stack at `libcontext.cpp:202-217` — generalize it to the app root). At depth 0, `DoRun` **yields the root fiber to the scheduler** instead of `handleAsync`-parking. The browser tick (rAF or `set_main_loop` callback) **resumes the root fiber**, which calls `ProcessEvents` **directly (wasm-side), not via `await ccall(...,{async:true})`** — that JS-awaits-a-suspending-export boundary is the Emscripten #13302 corruption hazard. Now the main loop is a sibling fiber to coroutines/modals; a coroutine swap is fiber↔fiber under the scheduler — no nested unwind.
|
||||
|
||||
## 4. File-by-file change map
|
||||
| File | Change | Phase |
|
||||
|---|---|---|
|
||||
| `scripts/common/shims/handlesleep.js` | → `asyncify-scheduler.js`: the scheduler (1.2–1.5); cover `handleAsync` + fiber tracking + deferred drain; drop the stale `"unwind"` swallow | 1 |
|
||||
| `scripts/common/inject-dyncall-shims.sh` | inject the new scheduler; keep §3c self-heal | 1 |
|
||||
| `wxwidgets/src/wasm/evtloop.cpp` | `DoRun` top-level → root-fiber yield; `ProcessEvents` driven wasm-side; `ScheduleExit` → scheduler wake | 2,3 |
|
||||
| `wxwidgets/src/wasm/dialog.cpp` | `ShowModal`/`EndModal` → `wasm_yield_until`/`wasm_resolve_wait` | 3 |
|
||||
| `kicad/thirdparty/libcontext/libcontext.cpp` | register fiber create/swap with the scheduler; expose the root-fiber init | 2 |
|
||||
| wx clipboard/font wasm files | → yield API | 3 |
|
||||
| `tests/apps/standalone/coroutine*`, `*raytrace*` | the integration gate (already exist) | 0 |
|
||||
| `tests/apps/standalone/sched-nest/` (new) | the minimal Phase-0 unit repro | 0 |
|
||||
| `tests/asyncify/*.spec.ts` | red-green specs for the harness, 3 engines, both EH | 0,1 |
|
||||
|
||||
## 5. Test harness
|
||||
- **Phase 0 minimal repro:** a tiny `wxIMPLEMENT_APP` that, from a `CallAfter`/timer (i.e. inside the parked rAF pump), does a libcontext fiber swap and swaps back; assert no `invalid state: 1`, correct round-trip value. RED under the current de-park; the unit gate for Phase 1.
|
||||
- **Integration gate:** the 6 `coroutine*` specs (already RED under de-park).
|
||||
- **Full matrix (12 §test-matrix):** `ShowModal` from root & from coroutine; nested modal in quasi-modal; coroutine swap while a modal pumps; clipboard from root & coroutine; raytracer multi-core; exit/unload cleanup — in **Firefox+Chrome+Safari**, under **both `-fexceptions` and `-fwasm-exceptions`** (incl. a modal from inside a `catch`, to prove composition with the hoist pass), with a `-sASYNCIFY_ASSERTIONS=1` pass.
|
||||
|
||||
## 6. Phase checklist (work log — update as landed)
|
||||
|
||||
- [ ] **Phase 0 — red harness** (2–3 d). Minimal `sched-nest` repro RED in 3 engines; the 6 `coroutine*` specs confirmed RED under de-park; CI/local script to run them.
|
||||
- [ ] **Phase 1 — scheduler core** (1–2 wk). `asyncify-scheduler.js` with the 4 hooks + deferred drain; covers `handleAsync` + fibers. **Gate:** `sched-nest` + the 6 `coroutine*` specs GREEN, all 3 engines, both EH. (If the permanent `handleAsync` park can't be a clean parked context, escalate to Phase 2.)
|
||||
- [ ] **Phase 2 — root fiber** (≈1 wk). Main loop = scheduler root fiber; `ProcessEvents` wasm-side (no JS async ccall). **Gate:** Phase-1 gate still green + no `handleAsync` park remains.
|
||||
- [ ] **Phase 3 — migrate waits** (1–2 wk). `wasm_yield_until` API; `ShowModal`/nested loop/clipboard/font on it. **Gate:** full matrix green.
|
||||
- [ ] **Phase 4 — lifetime** (few d). Cleanup ordering vs the scheduler; teardown deferred to unload. **Gate:** exit/unload tests green; no cleanup during steady-state pumping.
|
||||
|
||||
## 6b. Phase-0 finding — Phase 1 is insufficient; Phase 2 (root fiber) is REQUIRED (2026-06-23)
|
||||
|
||||
**Exact failure** (coroutine_test, de-park build): the first case `yield_resume_preserves_state` **passes** (it runs during the startup burst, *before* the main-loop park), then a later fiber swap aborts with **`Aborted(Assertion failed: We cannot stop an async operation in flight)`**, surfacing as `[wxWasm] main loop pump error`.
|
||||
|
||||
**Why:** `wxWasmParkMainLoop` is `Asyncify.handleAsync(...)` — a **permanently in-flight async operation** for the app's whole life. A coroutine `emscripten_fiber_swap` inside the rAF pump calls `stop_unwind`, but the park's async op is in flight → abort. Under the old `throw`, the top loop was *not* an async op (`throw "unwind"` is a plain JS exception), so swaps from a clean base worked.
|
||||
|
||||
**Tested & ruled out:** changing the rAF pump's `await ccall('ProcessEvents',{async:true})` to a **synchronous** `ccall` does NOT help — the in-flight op is the *park*, not the per-tick ccall. And the park is **permanent** (never completes until exit), so no scheduler serialization can let a coroutine swap "wait for the slot." **So §6's Phase-1 escalation condition is met.**
|
||||
|
||||
**The fix (Phase 2, now confirmed required):** the main loop must not be a `handleAsync` park. Make the main stack a **libcontext fiber** (Ruby/Julia pattern): the main fiber runs `ProcessEvents` on its own stack and **yields to the browser by a fiber swap / return-through-`set_main_loop(...,0)`**, not a `handleAsync` suspend — so there is no permanent in-flight async operation, and a coroutine swap is a sibling fiber↔fiber switch from the same `g_main_context`. `ProcessEvents` must run on `g_main_context` (the main fiber), not the fresh rAF-ccall stack. The Phase-1 scheduler is still needed to coordinate modal/clipboard waits that *do* suspend — but **the main-loop park must move off `handleAsync` first.**
|
||||
|
||||
**Open Phase-2 design point:** how the main fiber yields to / resumes from the browser each frame (rAF resumes `g_main_context` to run one `ProcessEvents` tick, then the main fiber yields back) without re-introducing a permanent asyncify operation. Candidate: `set_main_loop(tick,0,0)` where `tick` resumes the main fiber via libcontext, the main fiber runs `ProcessEvents` then swaps back, and wx teardown is suppressed until unload (Phase 4 lifetime).
|
||||
|
||||
## 6c. IMPLEMENTED & verified (2026-06-23): the per-frame-yield while-loop
|
||||
|
||||
The fix is **simpler than "an explicit libcontext root fiber."** `DoRun` (top level) is now a plain C++ loop on the real main stack (`evtloop.cpp`):
|
||||
```cpp
|
||||
while (!m_shouldExit) { ProcessEvents(); wxWasmYieldToBrowser(); }
|
||||
```
|
||||
`wxWasmYieldToBrowser` is `EM_ASYNC_JS(void, …, { await new Promise(r => requestAnimationFrame(r)); })` — an Asyncify suspend that **completes every frame**. Because nothing is permanently suspended, the Asyncify slot is free (`state==Normal`) whenever `ProcessEvents` runs, so a tool-coroutine fiber swap inside it succeeds; and `ProcessEvents` runs on the real main C stack (= libcontext's `g_main_context`), so swaps are from the right context. `ScheduleExit` just sets `m_shouldExit` for the top level (nested/quasi-modal loops still use the `setTimeout` pump + `wxWasmExitNestedLoop`). `wxWasmParkMainLoop` is removed. No explicit fiber API or scheduler was needed for the *main-loop* fix — the key was only that the suspension **completes** each frame instead of being permanent.
|
||||
|
||||
**Result (JS-EH):** coroutine in-app suite **13/13 pass, 0 fail** (was: abort after case 1); `coroutine` + `coroutine-nested` e2e specs **green**; dialog renders + modals **green** (no regression). Only `coroutine-pthread` outstanding — but its `coroutine_test_wxpt.wasm` was **stale** (the `coroutine-pthread` make target didn't rebuild it); all apps are being rebuilt to confirm.
|
||||
|
||||
**Still likely needed later (Phase 1 scheduler / Phase 3):** modal/clipboard waits that genuinely suspend across the loop still use the nested `setTimeout` pump; if overlapping suspensions there prove fragile, layer the scheduler on. But the *coroutine regression itself is fixed by this main-loop change alone.*
|
||||
|
||||
## 6d. Phase-2 exposes a SECOND coupling: context-menu re-entrancy needs Phase 1 (2026-06-23)
|
||||
|
||||
The while-loop main loop (§6c) fixed the coroutines but **regressed the context menu** (2 e2e specs). Right-click → choose *Cut* → `Aborted(RuntimeError: unreachable)` / `memory access out of bounds`. Stack: a DOM mouse event (`mouseEventHandlerFunc` → the Asyncify export wrapper → `wasm-function[…]`) **re-enters wasm while `DoPopupMenu`'s `wxDomPopupMenuModal` context is suspended on the deep main stack** — a single-slot re-entrancy fault. The de-park's *permanent-context* loop masked it (its menu context was shallow — a fresh-ccall `ProcessEvents` — and always "in flight"); the while-loop's no-permanent-context, deep-stack suspend exposes it.
|
||||
|
||||
**Three targeted fixes, all empirically REJECTED (don't retry these):**
|
||||
1. **C++ pump in `wxDomPopupMenuModal`** (mirror `startModal`) — *redundant*: `wx-dom.js`'s `wxShowContextMenu` **already** runs the same `setTimeout` ProcessEvents pump. No effect. (Reverted.)
|
||||
2. **`ASYNCIFY_STACK_SIZE` 8192→65536** — not a buffer-size fault (still crashes at 65536; emscripten appends that hint to *every* `unreachable`). (Kept anyway — the while-loop genuinely deepens every suspension, so 65536 ≈ the coroutine apps + KiCad is the right call for all wx apps.)
|
||||
3. **DOM backdrop blocking canvas pointer events** (`wx-dom.js`) — confirmed present in the rebuilt glue; still crashes. So the re-entry is **not** a canvas leak — the wx DOM port's document-level mouse handler re-enters wasm regardless. (Reverted.)
|
||||
|
||||
**Conclusion — the hard tension, stated plainly:**
|
||||
- **de-park** (permanent-context loop): menus ✅, coroutines ❌
|
||||
- **while-loop** (no permanent context): coroutines ✅, menus ❌
|
||||
|
||||
Neither is clean alone. Both faults are the SAME single-slot `currData`/state arbiter problem — Design B's **scheduler (Phase 1)** — now *proven necessary, not optional*. The §6c while-loop is the correct **foundation** (it removes the permanent park that blocked coroutine swaps); Phase 1 must layer on top so a wasm re-entry during ANY suspension (coroutine swap, menu/modal `handleAsync`, main-loop yield) is coordinated (deferred/queued or serialized) rather than misfiring a rewind. Kept in-tree: `evtloop.cpp` while-loop + the 65536 bump. Reverted: the redundant C++ pump and the backdrop.
|
||||
|
||||
## 6e. The precise mechanism (export-wrapper diagnostic, 2026-06-23)
|
||||
|
||||
Instrumented the Asyncify export wrapper to log every wasm entry while `state != Normal`. The menu crash is **not** a one-shot bad rewind — it's an **infinite busy unwind/rewind loop** on one buffer:
|
||||
```
|
||||
asyncify_start_unwind state=1(Unwinding) currData=1240280 ← main suspends
|
||||
asyncify_start_rewind state=2(Rewinding) currData=1240280 ← …immediately resumed
|
||||
__main_argc_argv state=2 currData=1240280 ← main runs a few dynCall_ii deep
|
||||
…repeats forever (currData unchanged) until the OOB crash
|
||||
```
|
||||
Buffer `1240280` (the parked main stack, suspended at the menu) is **suspended then immediately re-resumed, over and over**. Only one context exists, but it is being re-driven in a tight spin: its continuation re-suspends instantly (the menu promise is still pending), and something re-rewinds it each cycle.
|
||||
|
||||
**Two drivers fight over the single slot:** with the while-loop, the main-loop structure AND the **menu's own `setTimeout` ProcessEvents pump** (`wx-dom.js`) both try to drive the parked main stack — one re-rewinds what the other parked. Under the de-park there was a *single* pump chain (the rAF pump *was* the loop; the menu pump nested inside its `await`), so nothing double-drove the slot.
|
||||
|
||||
**Scheduler invariant this pins (the central requirement):** exactly ONE unwind/rewind transition in flight; a pump tick runs a **fresh** `ProcessEvents` (new stack) and must NEVER re-rewind an already-parked context — only that context's own `wakeUp` (its promise resolving) may resume it. The scheduler must enforce this across the main-loop yield, the menu/modal/nested pumps, and fiber swaps. (A plausible smaller first cut: a single shared "is a transition in flight / is a context parked" guard the pumps consult before re-driving — test it against the contextmenu specs before committing to the full registry.)
|
||||
|
||||
## 6f. RESOLVED (2026-06-23): the arbiter already existed — it just wasn't injected
|
||||
|
||||
A gated export-wrapper + `start_rewind` probe nailed the proximate cause: the menu's wakeUp fires `_asyncify_start_rewind(Asyncify.currData)` with **`currData == null`** → reads address 0 → OOB. The cause: the **`handlesleep.js` currData save/restore shim** — the existing Design-A / Emscripten #9153 arbiter (`scripts/common/shims/handlesleep.js`, which restores `currData` to the parked context's buffer before every rewind) — was **NOT injected into the contextmenu glue** (`pendingSleepContexts` count 0, vs 9 in the working coroutine app). Appending it manually → crash gone, `[CTXMENU_EVENT] Cut chosen` fires, spec 4/4 green.
|
||||
|
||||
**Why it was missing:** `inject-dyncall-shims.sh` gates the handleSleep shim on the libcontext fiber marker (`_emscripten_fiber_swap.isAsync = true;`), and `build-wasm-test.sh` only ran the injector under `WX_NATIVE_EH=1`. So plain (non-fiber) wx apps under JS-EH never received the currData arbiter. They didn't crash *before* the while-loop because the de-park's shallow menu context (a fresh-ccall `ProcessEvents`) never hit the null-rewind path; the while-loop's deeper main-stack suspend exposes it.
|
||||
|
||||
**Fix — build-system only, NO new runtime code:**
|
||||
1. `build-wasm-test.sh` injects the shim into every freshly-linked app for **both** EH models (idempotent — the Makefile-injected coroutine apps are skipped).
|
||||
2. `inject-dyncall-shims.sh` appends the handleSleep shim at EOF when there's no fiber glue (Asyncify is defined by then; it wraps `handleSleep` at load, before any runtime sleep).
|
||||
|
||||
**So §6c–6e's "build the single-owner currData arbiter" conclusion was right about the diagnosis but the arbiter already exists (`handlesleep.js`) — it only needed to reach these apps.** The scheduler invariant in §6e *is* what `handlesleep.js` implements (each parked context owns its buffer; `currData` is restored before its own rewind). The while-loop (coroutine fix) + this injection fix together resolve both regressions. §7's open question is therefore moot: no Phase-1 scheduler nor Phase-2 root fiber was needed — the while-loop main loop + the pre-existing currData shim suffice.
|
||||
|
||||
## 7. Open decisions (resolve during implementation)
|
||||
- Is Phase 1 (scheduler treating the `handleAsync` park as a tracked parked context) sufficient, or is Phase 2 (root fiber) required? — answered by the Phase-0 harness against the Phase-1 build.
|
||||
- One scheduler file injected post-link (like today's shim) vs an emscripten `--js-library` (link-time, cleaner, survives JS regen). Lean js-library for durability.
|
||||
- Whether to keep an Asyncify-only "no scheduler" fast path for apps with no coroutines (most standalone tests) to avoid scheduler overhead — likely yes, gated on a runtime "any non-main context registered?" check.
|
||||
- Native-EH interaction: confirm the scheduler's transitions compose with `HoistCppCatches` (suspend-inside-catch) — a matrix test, not a design change expected.
|
||||
278
docs/features/perf/README.md
Normal file
278
docs/features/perf/README.md
Normal file
|
|
@ -0,0 +1,278 @@
|
|||
# Cross-browser performance: why Firefox > Chrome > Safari, and how to close the gap
|
||||
|
||||
> Research notes, **2026-06-18**. The KiCad WASM port runs fastest in Firefox,
|
||||
> slower in Chrome, slowest in Safari. This document explains *why* at the
|
||||
> browser-engine level and lays out a ranked, build-specific plan to speed up
|
||||
> Chrome and Safari. Web claims are dated and linked in [Sources](#sources);
|
||||
> codebase claims carry `file:line` refs. Companion work lives in
|
||||
> [`../async/`](../async/) (Asyncify) and [`../wasm-exceptions/`](../wasm-exceptions/)
|
||||
> (the `-fwasm-exceptions` migration).
|
||||
|
||||
---
|
||||
|
||||
## TL;DR
|
||||
|
||||
The Firefox lead is **not** a Firefox trick. Our binary is dominated by
|
||||
**Asyncify** instrumentation, and Firefox's compilers simply tolerate Asyncify's
|
||||
pathological code far better than Chrome's or Safari's do. So the highest-leverage
|
||||
work for Chrome *and* Safari is to **shrink/attack the Asyncify footprint**, plus a
|
||||
handful of cheap, orthogonal wins.
|
||||
|
||||
There are **two independent axes**, and both need attention:
|
||||
|
||||
1. **WASM compile/execute** — Asyncify-dominated. This explains the
|
||||
Firefox > Chrome > Safari **ordering**.
|
||||
2. **WebGL rendering** — Safari's Metal/ANGLE overhead. This is *extra* Safari
|
||||
slowness on top of axis 1, and several fixes are one-liners.
|
||||
|
||||
### Ranked levers
|
||||
|
||||
| # | Lever | Axis | Effort | Impact | Where |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 | Confirm/force `instantiateStreaming` + `Content-Type: application/wasm` + stable URL/ETag | startup | hours | ~1.5–1.8× cold start (FF); arms V8 cache | `web/standalone/src/wasm/boot.ts` |
|
||||
| 2 | **Brotli** instead of gzip-9 on R2 | startup | hours | ~15–25% smaller transfer | R2 / edge config |
|
||||
| 3 | `powerPreference: 'high-performance'` + context-lost handlers | WebGL (Safari/Chrome) | hours | discrete GPU instead of integrated | `wxwidgets/src/wasm/glcanvas.cpp:524-535` |
|
||||
| 4 | Audit GAL shaders for the `flat` qualifier | WebGL (Safari) | hours–days | up to *seconds/frame* in worst case | `kicad/common/gal/shaders/` |
|
||||
| 5 | Remove `glGetError()` from the render loop | WebGL (Safari) | hours | avoids per-call Metal flush | GAL compositor |
|
||||
| 6 | Test `antialias: false` | WebGL (Safari) | hours | cuts MSAA resolve cost | `glcanvas.cpp:524-535` |
|
||||
| 7 | Enable `-msimd128` | WASM exec (all) | days | 1.5–2.5× geometry/render hot loops | build flags |
|
||||
| 8 | `ASYNCIFY_ADVISE` → `ASYNCIFY_IGNORE_INDIRECT` + extend `REMOVE` | WASM exec (all, esp. Chrome/Safari) | days | smaller binary + faster tier-up | `scripts/common/apply-asyncify.sh` |
|
||||
| 9 | `-fwasm-exceptions` (size) | WASM (all) | weeks | 64.5 → 36 MB gz | tracked — see [§ Structural bets](#structural-bets-track--prototype) |
|
||||
| 10 | JSPI (delete Asyncify) | WASM (all, esp. Safari) | weeks | ~40–50% smaller, removes JIT pressure | tracked — see [§ Structural bets](#structural-bets-track--prototype) |
|
||||
| — | wasm-split, WebGPU GAL backend | startup / WebGL | weeks+ | deferred (see [§ Deferred](#deferred--not-now)) | — |
|
||||
|
||||
---
|
||||
|
||||
## Current build (the baseline)
|
||||
|
||||
Verified from the build scripts and runtime glue:
|
||||
|
||||
| Knob | Value | Location |
|
||||
|---|---|---|
|
||||
| Asyncify | `-sASYNCIFY=1`, `ASYNCIFY_STACK_SIZE=65536` | `scripts/kicad/build-kicad-target.sh:~400` |
|
||||
| Exceptions | **legacy `-fexceptions`** (not `-fwasm-exceptions`) | `build-kicad-target.sh:240-255` |
|
||||
| SIMD | **none** (`-msimd128` absent) | — |
|
||||
| Threads | `-sUSE_PTHREADS=1`, pool = `navigator.hardwareConcurrency` (+ COOP/COEP) | `build-kicad-target.sh`, `web/.../preflight/capabilities.ts` |
|
||||
| Memory | `INITIAL_MEMORY=256MB`, `MAXIMUM_MEMORY=4GB`, `ALLOW_MEMORY_GROWTH=1` | `build-kicad-target.sh` |
|
||||
| Opt | clang `-O2` (release); link `-O0` then **host `wasm-opt -O2` after `--asyncify`** | `apply-asyncify.sh:88-157` |
|
||||
| WebGL | WebGL2 (`-sMAX_WEBGL_VERSION=2`), `antialias:true`, **`powerPreference:DEFAULT`** | `glcanvas.cpp:524-535` |
|
||||
| Loading | Emscripten script-glue; **streaming not confirmed**; gzip-9, **no Brotli** | `boot.ts:145-294` |
|
||||
| Artifact | pcbnew **186 MB raw / 64 MB gzip**; eeschema 99/34; pl_editor 52/17; gerbview 49/16 | `output/` |
|
||||
|
||||
Note: all three modern browsers support `SharedArrayBuffer`/threads under COOP+COEP
|
||||
(the app demonstrably runs in each) — capability gating is in
|
||||
`capabilities.ts`, not UA sniffing.
|
||||
|
||||
---
|
||||
|
||||
## Why the ordering exists (engine internals)
|
||||
|
||||
### The villain: Asyncify
|
||||
|
||||
Asyncify rewrites every instrumented function with unwind/rewind state checks and
|
||||
saves/restores all locals to linear memory. That expands each local's live range
|
||||
across the *whole* function, producing a nearly fully-connected interference graph
|
||||
— exactly the input that is catastrophic for optimizing register allocators.
|
||||
Asyncify's own docs warn: *"VMs may also limit compilation to the baseline tier on
|
||||
such pathological code."* Result: ~+70% binary, giant functions, and the
|
||||
186 MB-raw pcbnew. See [`../async/02-asyncify-internals.md`](../async/02-asyncify-internals.md).
|
||||
|
||||
### How each engine copes
|
||||
|
||||
| Engine | Baseline tier | Optimizing tier | On Asyncify's giant functions |
|
||||
|---|---|---|---|
|
||||
| **Firefox / SpiderMonkey** | Rabaldr, **~25 ns/byte**, eager whole-module, multithreaded (30–60 MB/s) | **Ion** — [75× large-function fix, Oct 2024](https://spidermonkey.dev/blog/2024/10/16/75x-faster-optimizing-the-ion-compiler-backend.html) (sorted live ranges, Semi-NCA dominators, sparse bitsets) targeting *exactly* the huge-CFG/high-vreg shape Asyncify creates (ONNX: 5 min → 3.9 s) | **Best.** Whole module baseline-compiled before download finishes; Ion swallows the big functions. No OSR gap. |
|
||||
| **Chrome / V8** | Liftoff, **~50 ns/byte** (½ Firefox) | **TurboFan** — chokes on huge fns (a 1.96 MB fn → 95 s, 7.4 GB RAM, 87% in regalloc); falls back to mid-tier allocator or **skips optimization** | **Middle.** **V8 has no OSR for wasm** — a function in a long loop (Asyncify rewind/unwind loops!) finishes that whole call in Liftoff; only the *next* call gets TurboFan. |
|
||||
| **Safari / JSC** | **Lazy everything**: IPInt (interpreter) → BBQ → OMG. Nothing eager. | **OMG** (B3) — did *not* get Ion's 2024 large-fn treatment | **Worst.** First run executes at interpreter speed; Asyncify ~doubles fn count → huge OMG backlog → documented **300–400% CPU spike for 30 s+** after a workload. **No persistent compiled-code cache**, so it re-pays every session; above ~10 MB it switches to a slower JIT mode. |
|
||||
|
||||
### Two corollaries that bite us specifically
|
||||
|
||||
- **Chrome's V8 wasm code cache is effectively unavailable.** It only caches
|
||||
modules under ~150 MB *compiled*, and compiled code is 5–7× the `.wasm`. Our
|
||||
186 MB pcbnew → ~1 GB compiled — far over the ceiling. So Chrome **re-runs
|
||||
TurboFan on every cold load** today. Shrinking the binary (levers 7–10) is the
|
||||
only way to get Chrome's repeat-load cache back. See
|
||||
[V8 wasm code caching](https://v8.dev/blog/wasm-code-caching).
|
||||
- **Benchmark trap:** with DevTools open, V8 tiers all wasm *down* to Liftoff.
|
||||
Never measure Chrome speed with DevTools open (except via an actual Performance
|
||||
recording, which forces tier-up). This likely makes Chrome look worse than it is
|
||||
in casual testing.
|
||||
|
||||
---
|
||||
|
||||
## The ranked plan
|
||||
|
||||
### Tier 1 — cheap, do now (days, low risk)
|
||||
|
||||
**1. Confirm + force streaming instantiation and cache headers.** The loader
|
||||
injects the Emscripten JS glue via `<script>` (`boot.ts`); whether the *runtime*
|
||||
then streams the `.wasm` depends on serving conditions. In DevTools → Network,
|
||||
confirm the `.wasm` returns **`Content-Type: application/wasm`** with no console
|
||||
"falling back to ArrayBuffer instantiation" warning. Streaming is ~1.5–1.8× faster
|
||||
cold-start on Firefox and is the *only* path that arms V8's code cache. Serve the
|
||||
`.wasm` from a **stable URL** (no content-hash in the path; use a stable alias)
|
||||
with `ETag`/`304`. With `-pthread`, confirm the module is compiled once and shared
|
||||
to workers (Emscripten does this via the shared `WebAssembly.Module`), not
|
||||
recompiled per worker.
|
||||
|
||||
**2. Brotli instead of gzip-9 on R2.** Brotli is ~15–25% smaller on wasm (our
|
||||
64 MB pcbnew → ~50 MB). Verify Cloudflare actually Brotli-compresses it at the
|
||||
edge — it often *skips* large binary types — and if not, precompress and serve
|
||||
with `Content-Encoding: br` + `Content-Type: application/wasm`. Smaller transfer
|
||||
also means less to compile, so it compounds with everything below.
|
||||
|
||||
**3. `powerPreference: 'high-performance'` for WebGL.** We default to
|
||||
`EM_WEBGL_POWER_PREFERENCE_DEFAULT` (`glcanvas.cpp:524-535`). **Safari (and Chrome
|
||||
on dual-GPU Macs) defaults WebGL to the integrated GPU.** Requesting
|
||||
high-performance switches to the discrete GPU — often the single biggest
|
||||
GPU-bound framerate win on MacBook Pros. Caveat: Safari only honors it if you also
|
||||
register `webglcontextlost`/`webglcontextrestored` handlers.
|
||||
|
||||
**4. Audit GAL shaders for the `flat` interpolation qualifier.** This is the big
|
||||
Safari sleeper. `flat` triggers a provoking-vertex workaround in Safari's
|
||||
Metal/ANGLE backend that has cost real apps *seconds per frame*. PCB renderers
|
||||
commonly use `flat` for per-primitive net/layer colors. Grep the GAL shaders
|
||||
(`kicad/common/gal/shaders/`, source GLSL 1.20 before `convert_glsl_es3.py`); if
|
||||
present, replace with regular interpolation or restructure. Potentially a massive
|
||||
Safari-only win.
|
||||
|
||||
**5. Remove `glGetError()` from the render loop.** On Safari each call forces a
|
||||
Metal pipeline flush. Restrict to init/debug builds only. (Note: the WebGL
|
||||
compositor already drains stale `glGetError()` once before draws — that's fine;
|
||||
the concern is *per-call* error checks inside the hot path.)
|
||||
|
||||
**6. Test `antialias: false`.** We default MSAA on (`antialias:true`). KiCad's GAL
|
||||
does much of its line AA in-shader (SMAA) and has its own AA setting; if MSAA is
|
||||
redundant, dropping it cuts Metal's resolve cost on Safari. Quality/perf tradeoff —
|
||||
A/B it on a dense board; consider exposing it as a setting.
|
||||
|
||||
### Tier 2 — medium effort, high impact
|
||||
|
||||
**7. Enable `-msimd128`.** Expect **1.5–2.5×** on the geometry/render hot loops
|
||||
(polygon booleans in `shape_poly_set`, DRC overlap checks, vertex-buffer fills) via
|
||||
LLVM autovectorization at `-O2`+. Safe on all three engines (Chrome 91 / FF 89 /
|
||||
Safari 16.4). Helps absolute Chrome *and* Safari speed. Verify `v128.*` actually
|
||||
appears in the disassembly for the hot functions, and prefer `pmin`/`pmax` over
|
||||
min/max (the SSE→wasm emulation table has slow paths). **Do not** ship Relaxed SIMD
|
||||
yet (Safari still flags it). Minor interaction to watch: SIMD slightly grows
|
||||
per-function size, which feeds the Asyncify/locals pressure — measure after the
|
||||
`wasm-opt -O2` pass.
|
||||
|
||||
**8. Shrink the Asyncify surface.** This directly attacks the root cause for Chrome
|
||||
and Safari. Run **`ASYNCIFY_ADVISE`** to see which functions get instrumented and
|
||||
why — it surfaces the biggest instrumented functions (the JIT pressure points).
|
||||
Then:
|
||||
- **`ASYNCIFY_IGNORE_INDIRECT=1`** is the high-impact one: our wxWidgets/GAL
|
||||
code is vtable-heavy, and Asyncify conservatively instruments *every* indirect
|
||||
call site, which is why instrumentation spreads everywhere. With our
|
||||
understanding of the suspend paths (the park-throw work), we may be able to
|
||||
assert no indirect call is on the suspend stack and add specific ones back via
|
||||
`ASYNCIFY_ADD`.
|
||||
- Extend the existing 12-function `ASYNCIFY_REMOVE` list (`apply-asyncify.sh:92-104`)
|
||||
with cold/startup-only large functions ADVISE flags.
|
||||
- Smaller instrumented set → smaller functions → better tier-up in *all* engines
|
||||
and a smaller Safari OMG backlog.
|
||||
- ⚠️ Error-prone (wrong config = silent runtime breakage). Gate behind the
|
||||
red/green Asyncify harness in [`../asyncify-arbiter/redgreen.md`](../asyncify-arbiter/redgreen.md)
|
||||
/ `tests/asyncify/`.
|
||||
|
||||
### Structural bets (track / prototype)
|
||||
|
||||
**9. `-fwasm-exceptions`** (we're on legacy `-fexceptions`). Biggest *size* lever —
|
||||
[`../wasm-exceptions/02-measurements.md`](../wasm-exceptions/02-measurements.md)
|
||||
puts pcbnew at **64.5 → 36 MB gzip**, which would also start bringing Chrome back
|
||||
under the code-cache ceiling and fix the unreliable catch/destructor landing-pad
|
||||
behavior we've documented. **Two blockers to track before committing:**
|
||||
- a **Safari 26.0 startup regression** for `-fwasm-exceptions` modules
|
||||
([emscripten #25365](https://github.com/emscripten-core/emscripten/issues/25365))
|
||||
— verify whether it's fixed in a 26.x / Safari 27 beta before shipping, since
|
||||
Safari is the browser we're trying to help;
|
||||
- the asyncify-EH unwind-from-catch interaction
|
||||
([`../wasm-exceptions/05-asyncify-fork-design.md`](../wasm-exceptions/05-asyncify-fork-design.md)).
|
||||
|
||||
**10. JSPI** — the eventual *real* fix for Safari, because it deletes Asyncify
|
||||
entirely (no instrumentation → no giant functions → no OMG backlog → the
|
||||
300–400% Safari spike goes away) and cuts ~40–50% of binary size. Status:
|
||||
|
||||
| Engine | JSPI status |
|
||||
|---|---|
|
||||
| Chrome / V8 | **shipped, Chrome 137** (May 2025) |
|
||||
| Firefox / SpiderMonkey | **Firefox 153** intent-to-ship (June 2026); Nightly now, stable ~late summer/fall 2026 |
|
||||
| Safari / JSC | **Safari 27 beta** (WWDC26), enabled by default; stable Fall 2026 |
|
||||
|
||||
Don't migrate wholesale yet:
|
||||
- unresolved **~350× regression on the `JS→C→JS` re-entry pattern**
|
||||
([emscripten #21081](https://github.com/emscripten-core/emscripten/issues/21081))
|
||||
— exactly what a GUI event loop hits constantly;
|
||||
- static-init `SuspendError`
|
||||
([emscripten #24302](https://github.com/emscripten-core/emscripten/issues/24302));
|
||||
- `invoke_*` over-tagging under our legacy exceptions.
|
||||
|
||||
**Recommended path:** prototype JSPI on a *small* tool (calculator or pl_editor)
|
||||
behind feature detection (`'Suspending' in WebAssembly`) with Asyncify fallback,
|
||||
profile the re-entry pattern, and watch #21081. By the time it's safe, all three
|
||||
engines will support it.
|
||||
|
||||
### Deferred / not now
|
||||
|
||||
- **`wasm-split` / `-sSPLIT_MODULE`** — the secondary module can't be loaded lazily
|
||||
*and* asynchronously on the main thread, which is incompatible with our
|
||||
main-thread Asyncify model unless we move to `-sPROXY_TO_PTHREAD`.
|
||||
- **WebGPU GAL backend** — the structural exit from Safari's Metal/ANGLE overhead
|
||||
(Safari 26 ships WebGPU), but it's a multi-week GLSL→WGSL port with no upstream
|
||||
KiCad support. See the GAL history in [`../archive/webgl/`](../archive/webgl/).
|
||||
- **Global `-O3`** — its inlining bloats the binary and makes the Chrome
|
||||
cache/compile problem *worse*. If anything, compile cold/utility units at `-Os`.
|
||||
|
||||
---
|
||||
|
||||
## What to do first
|
||||
|
||||
Quick, visible, near-zero-risk wins this week:
|
||||
|
||||
- **Safari:** #3 (high-performance GPU) + #4 (`flat` audit) + #5 (`glGetError`).
|
||||
- **Startup everywhere:** #2 (Brotli) + #1 (streaming/headers).
|
||||
- **Runtime everywhere:** #7 (`-msimd128`).
|
||||
|
||||
Then invest in **#8 (Asyncify ADVISE + IGNORE_INDIRECT)** as the real lever against
|
||||
the Chrome/Safari gap, and keep **#9 / #10** on a tracking list.
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
**Engine internals**
|
||||
- [75× faster: optimizing the Ion compiler backend — SpiderMonkey, Oct 2024](https://spidermonkey.dev/blog/2024/10/16/75x-faster-optimizing-the-ion-compiler-backend.html)
|
||||
- [Understanding WebAssembly code generation throughput — wingolog, 2020](https://wingolog.org/archives/2020/04/14/understanding-webassembly-code-generation-throughput)
|
||||
- [V8 WebAssembly compilation pipeline](https://v8.dev/docs/wasm-compilation-pipeline) · [Dynamic tiering](https://v8.dev/blog/wasm-dynamic-tiering) · [Liftoff](https://v8.dev/blog/liftoff)
|
||||
- [Code caching for WebAssembly developers — V8](https://v8.dev/blog/wasm-code-caching)
|
||||
- [Introducing the JetStream 3 Benchmark Suite — WebKit, 2024](https://webkit.org/blog/17899/introducing-the-jetstream-3-benchmark-suite/) (IPInt/BBQ/OMG)
|
||||
- [Pause and Resume WebAssembly with Binaryen's Asyncify — kripken, 2019](https://kripken.github.io/blog/wasm/2019/07/16/asyncify.html)
|
||||
|
||||
**Startup / size / SIMD**
|
||||
- [Optimizing WebAssembly Startup Time — Nutrient](https://www.nutrient.io/blog/optimize-webassembly-startup-performance/)
|
||||
- [MDN: WebAssembly.instantiateStreaming](https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/JavaScript_interface/instantiateStreaming_static)
|
||||
- [Module Splitting — Emscripten](https://emscripten.org/docs/optimizing/Module-Splitting.html)
|
||||
- [Using SIMD with WebAssembly — Emscripten](https://emscripten.org/docs/porting/simd.html) · [V8 SIMD](https://v8.dev/features/simd) · [caniuse wasm-simd](https://caniuse.com/wasm-simd)
|
||||
- [Asynchronous Code (Asyncify settings) — Emscripten](https://emscripten.org/docs/porting/asyncify.html)
|
||||
|
||||
**Safari WASM + WebGL**
|
||||
- [WebGL Performance on Safari & Apple Vision Pro — Wonderland Engine](https://wonderlandengine.com/news/webgl-performance-safari-apple-vision-pro/) (`flat`, UBO timing)
|
||||
- [WebKit features in Safari 26.0](https://webkit.org/blog/17333/webkit-features-in-safari-26-0/) · [News from WWDC26 — Safari 27 beta](https://webkit.org/blog/17967/news-from-wwdc26-webkit-in-safari-27-beta/)
|
||||
- [emscripten #25365 — Safari 26.0 wasm-exceptions regression](https://github.com/emscripten-core/emscripten/issues/25365)
|
||||
- [emscripten #26027 — Safari + Asyncify + unaligned-load leak](https://github.com/emscripten-core/emscripten/issues/26027)
|
||||
- [ONNX Runtime #26827 — Safari WebKit 26 OMG CPU loop](https://github.com/microsoft/onnxruntime/issues/26827)
|
||||
|
||||
**JSPI**
|
||||
- [V8: Introducing the WebAssembly JavaScript Promise Integration API](https://v8.dev/blog/jspi) · [new API](https://v8.dev/blog/jspi-newapi)
|
||||
- [caniuse: JSPI](https://caniuse.com/wf-wasm-jspi) · [Chrome 137 release notes](https://developer.chrome.com/release-notes/137)
|
||||
- [Mozilla dev-platform: Intent to Ship JSPI (Fx153), June 2026](http://www.mail-archive.com/dev-platform@mozilla.org/msg01810.html)
|
||||
- [emscripten #21081 — JSPI 350× slower for JS→C→JS](https://github.com/emscripten-core/emscripten/issues/21081) · [#24302 — JSPI static-init SuspendError](https://github.com/emscripten-core/emscripten/issues/24302)
|
||||
|
||||
### Local cross-references
|
||||
- [`../async/`](../async/) — Asyncify internals, single-slot contention, park-throw.
|
||||
- [`../wasm-exceptions/`](../wasm-exceptions/) — `-fwasm-exceptions` measurements, toolchain status, asyncify-EH fork design.
|
||||
- [`../asyncify-arbiter/redgreen.md`](../asyncify-arbiter/redgreen.md) — the harness to gate Asyncify-surface changes.
|
||||
- [`../archive/webgl/`](../archive/webgl/) — GAL → WebGL2 history (context attrs, compositor, shaders).
|
||||
- Build: `scripts/kicad/build-kicad-target.sh`, `scripts/common/apply-asyncify.sh`.
|
||||
- WebGL context: `wxwidgets/src/wasm/glcanvas.cpp:489-535`. Loader: `web/standalone/src/wasm/boot.ts`.
|
||||
434
docs/features/threading/README.md
Normal file
434
docs/features/threading/README.md
Normal file
|
|
@ -0,0 +1,434 @@
|
|||
# Threading in KiCad-WASM — why it's single-core today, and the paths to real multithreading
|
||||
|
||||
> **Status:** mechanism reference for KiCad-WASM threading. Native wasm-EH (`-fwasm-exceptions`) is
|
||||
> the **default build**, and the three-failure-mode analysis below is validated by the pthread test
|
||||
> suite — see [`../wasm-exceptions/10-pthreads-native-eh.md`](../wasm-exceptions/10-pthreads-native-eh.md).
|
||||
> Authored 2026-06-24, updated 2026-06-25. Line numbers are against the artifacts current then
|
||||
> (`kicad/thirdparty/thread-pool/bs_thread_pool.hpp`, `kicad/common/thread_pool.cpp`,
|
||||
> `kicad/3d-viewer/3d_rendering/raytracing/render_3d_raytrace_base.cpp`,
|
||||
> `scripts/kicad/build-kicad-target.sh`, `scripts/common/shims/handlesleep.js`,
|
||||
> `scripts/common/apply-asyncify.sh`).
|
||||
|
||||
## Why this exists
|
||||
|
||||
A recurring question: we made "pthread hacks" in the 3D viewer's CPU renderer — *what* was the
|
||||
issue, *why* were they needed, will **native WASM exceptions** (`-fwasm-exceptions`) fix them, and
|
||||
how do we get back to **upstream-pristine KiCad source that still runs multithreaded** (so the fork
|
||||
stays upstreamable)? This document answers all of that, plus: the exact deadlock mechanics, the
|
||||
three-layer thread model, the complete inventory of raw threads in the tree, and what the *latest*
|
||||
upstream KiCad has (and hasn't) already changed.
|
||||
|
||||
It is the threading companion to the Asyncify dossier in [`../async/`](../async) (especially
|
||||
[`../async/11-asyncify-nesting-raytracer.md`](../async/11-asyncify-nesting-raytracer.md)) and the
|
||||
[`../wasm-exceptions/`](../wasm-exceptions) migration.
|
||||
|
||||
## TL;DR
|
||||
|
||||
- **Three layers, often conflated:** (1) **Web Workers** = the real OS threads; (2) **Emscripten's
|
||||
pthread pool** (`PTHREAD_POOL_SIZE`) = pre-spawned *empty* Workers; (3) **KiCad's
|
||||
`BS::thread_pool`** (`GetKiCadThreadPool()`) = `hardware_concurrency()` long-lived `std::thread`s
|
||||
that **consume** the pre-warmed Workers at startup. **All** pthreads are full shared-memory
|
||||
Workers — there is no lightweight/isolated variant.
|
||||
- **Effectively nothing runs multithreaded.** KiCad's pool funnels every entry point through one
|
||||
shimmed `detach_task()` (inline), so the whole pool is serial; the raytracer's *separate* raw
|
||||
`std::thread` passes are `#ifdef`'d to serial; `wxThread` is a no-op. The 16 pool Workers spawn at
|
||||
startup and then sit idle.
|
||||
- **Three distinct failure modes, not one:** **(a) deadlock** (raw-thread join + on-demand Worker
|
||||
creation — fixed in the wasm layer, §4), **(b) nesting abort** `invalid state: 1` (a 2nd Asyncify
|
||||
unwind starting while already Unwinding — does **not** arise when the inner `emscripten_sleep` is
|
||||
dispatched at `state == Normal`, e.g. from a modal pump's `ProcessEvents`, §3), **(c) worker-rewind
|
||||
crash** `"func is not a function"` (a C++ throw driving Asyncify on a pool Worker under
|
||||
`-fexceptions` — **fixed by native EH**, now the default).
|
||||
- **`-fwasm-exceptions` (the default) clears mode (c).** A C++ exception thrown on a pool Worker is
|
||||
safe under native EH — confirmed: the real-pool `threadpool-real` test runs 16-core with a throwing
|
||||
worker task, green only under native EH. Modes (a)/(b) are Asyncify, not EH — addressed separately
|
||||
in the wasm layer (the nanosleep override for (a); `state == Normal` dispatch for (b)).
|
||||
- **`PROXY_TO_PTHREAD` is not our escape hatch.** Asyncify *can* mechanically run on the
|
||||
proxied-main Worker, but it's unsupported/rough — and the real blocker is that our **wx-dom port
|
||||
manipulates the DOM directly**, which a Worker cannot do.
|
||||
- **A path to multi-core 3D needs zero KiCad edits — two ways.** (1) **Pre-warm** enough Workers
|
||||
(`PTHREAD_POOL_SIZE` ≥ pool + peak raw threads): on-demand creation never happens, so the upstream
|
||||
`sleep_for` busy-wait runs multi-core (with main-thread jank). (2) **The nanosleep override**
|
||||
(`wasm/shims/nanosleep_yield.c`) makes that main-thread `sleep_for` *yield* via Asyncify, so the
|
||||
event loop services the on-demand handshake — multi-core without pre-warming and without the jank
|
||||
(proven by `pthread-ondemand`, §4). The parked `WASM_RAYTRACE_POOL` (~6–7×) is the pre-warm shape.
|
||||
- **Upstream has only migrated 1 of 7 raytracer parallel sections to the pool**, and that was an
|
||||
accident (a side effect of a cosmetic commit). The other six are **legacy 2018 OpenMP-translation
|
||||
code** — so a pool migration is a legitimate, *upstreamable* cleanup, not a wasm hack.
|
||||
|
||||
---
|
||||
|
||||
## 1. The model: three layers, two patterns
|
||||
|
||||
WebAssembly has no threads of its own. "Thread" means different things at three levels:
|
||||
|
||||
**Layer 1 — Web Workers = the real OS threads.** A "thread" in a browser is a Web Worker: a
|
||||
separate JS context running the *same* wasm module against the *same* shared memory. Spinning one up
|
||||
is **expensive** (new context + module instantiate) and **can only be initiated from the main
|
||||
thread's event loop**.
|
||||
|
||||
**Layer 2 — Emscripten's pthread pool (`PTHREAD_POOL_SIZE`).** Because creating Workers is slow and
|
||||
main-thread-bound, Emscripten pre-spawns a bag of *empty, generic* Workers at startup. We set
|
||||
`PTHREAD_POOL_SIZE='navigator.hardwareConcurrency'` (`build-kicad-target.sh:413-415`), so on a
|
||||
16-core machine you get 16 pre-warmed Workers. `std::thread`/`pthread_create` tries to grab one.
|
||||
|
||||
**Layer 3 — KiCad's `BS::thread_pool` (`GetKiCadThreadPool()`).** An *application-level* pool — a
|
||||
different thing from Layer 2. Its constructor creates `hardware_concurrency()` long-lived
|
||||
`std::thread`s (`thread_pool.cpp:44-45` → `determine_thread_count` at `bs_thread_pool.hpp:1965-1970`)
|
||||
and parks them on a condition variable waiting for tasks. You feed it work with `submit_task`; the
|
||||
parked threads pick it up. The "hire a team once, give them many jobs" pattern.
|
||||
|
||||
**The interaction that confuses everyone:** KiCad's Layer-3 pool threads *are* pthreads *are*
|
||||
Layer-1 Workers. So the 16-thread pool **consumes all 16 pre-warmed Workers at startup.** After that
|
||||
the pre-warmed bag is *empty*.
|
||||
|
||||
**All pthreads are full shared-memory Workers.** There is no `std::thread` that gets its own
|
||||
isolated heap. Every pthread shares the one `WebAssembly.Memory` (one SharedArrayBuffer); a thread's
|
||||
"own" memory is only its stack + TLS, carved *out of* that shared buffer. KiCad's raytracer threads
|
||||
*need* this — they read the shared scene and write the shared output image. (An *isolated* Web
|
||||
Worker with message-passing — copy data in, post results out — would sidestep the whole pthread +
|
||||
Asyncify problem for pure-compute work like a raytrace band, but that is **not** what `std::thread`
|
||||
does; using it means hand-writing a worker pool and **rewriting away from upstream KiCad**.)
|
||||
|
||||
**Two patterns in KiCad's code:**
|
||||
- **Pool tasks** (`submit_task`/`submit_loop` on `GetKiCadThreadPool()`) — reuse the standing pool
|
||||
threads. No new Workers.
|
||||
- **Raw `std::thread`** — create a brand-new thread each time, *outside* the pool. Since the
|
||||
pre-warmed bag is already drained by the pool, these force **on-demand Worker creation** (§4).
|
||||
|
||||
---
|
||||
|
||||
## 2. What runs multithreaded today: nothing — the shims + the full raw-thread inventory
|
||||
|
||||
### The pool is funneled inline
|
||||
|
||||
KiCad routes its data-parallelism through `GetKiCadThreadPool()`. The WASM patch sits at the pool's
|
||||
single choke point, `bs_thread_pool.hpp:1419`:
|
||||
|
||||
```cpp
|
||||
void detach_task(F&& task, const priority_t priority = 0) {
|
||||
#ifdef __EMSCRIPTEN__
|
||||
(void) priority;
|
||||
std::forward<F>( task )(); // ← inline; never reaches a Worker
|
||||
return;
|
||||
#else
|
||||
/* enqueue + notify_one() a Worker */
|
||||
#endif
|
||||
}
|
||||
```
|
||||
|
||||
**Every** entry point funnels through it: `submit_task()` (`:1751`) calls `detach_task`;
|
||||
`submit_loop`/`submit_blocks` call `submit_task`; `detach_loop`/`detach_blocks`/`detach_sequence`
|
||||
call `detach_task`. So this one `#ifdef` makes the entire pool serial. (Note `create_threads`
|
||||
(`:1903`) is **not** shimmed — so the pool still spawns its 16 idle threads/Workers at startup; they
|
||||
just never get work. Pure overhead.)
|
||||
|
||||
Pool consumers now running serial: zone fill (`zone_filler.cpp`, `board.cpp`), **all** DRC providers
|
||||
(`pcbnew/drc/*`), connectivity (`CONNECTION_GRAPH`), footprint enumeration
|
||||
(`footprint_info_impl.cpp`), symbol/footprint **library preload** (`pgm_base.cpp:941`,
|
||||
`pcbnew.cpp:664`), `tracks_cleaner`, plus the raytracer **main trace** (`renderTracing`, which is
|
||||
pool-based).
|
||||
|
||||
### The complete raw-thread inventory
|
||||
|
||||
Beyond the pool, raw thread creation across the whole tree (the **wx port and our entire
|
||||
wasm/shim/scripts layer have zero**):
|
||||
|
||||
| Site | What | WASM status |
|
||||
|---|---|---|
|
||||
| `render_3d_raytrace_base.cpp:764` (`shadeWorker`) | raytrace post-process shading | **`#ifdef __EMSCRIPTEN__`-guarded → serial** |
|
||||
| `render_3d_raytrace_base.cpp:835` (`blurWorker`) | blur/finish | guarded → serial |
|
||||
| `render_3d_raytrace_base.cpp:1456` (`previewWorker`) | preview | guarded → serial |
|
||||
| `image.cpp:525` (`filterWorker`) | `IMAGE::EfxFilter` (AA/blur) | guarded → serial |
|
||||
| `create_layer_items.cpp:848` (`zoneWorker`) | zone fill geometry | guarded → serial |
|
||||
| `create_layer_items.cpp:1311` (`simplifyWorker`) | polygon simplify | guarded → serial |
|
||||
| `libs/kinng/src/kinng.cpp:57` | IPC-API (nng) listener | **not compiled** — CMake links `kinng` only `if(KICAD_IPC_API AND NOT EMSCRIPTEN)`; IPC defaults **OFF** |
|
||||
| `kicad/pcm/pcm.cpp:1123`, `pcm_task_manager.cpp` | Plugin & Content Manager (HTTP downloads) | **dormant** — network feature, not in the editor apps |
|
||||
| `common/eda_dde.cpp:146` | DDE/TCP-socket cross-probe server | **compiled but dormant** — no raw TCP sockets in a browser; should never be constructed |
|
||||
| `thirdparty/nanoflann.hpp:1278` | `std::async` parallel KD-tree build | conditional/dormant (serial by default) |
|
||||
| **`common/widgets/font_choice.cpp:99`** (`FONT_LIST_MANAGER::Poll`) | background font enumeration | **UNGUARDED** (only `#ifndef __MINGW32__`) — likely the one place a raw Worker *does* spawn in WASM. Fire-and-forget (no main-thread join), so it does **not** deadlock; verify whether `FONT_LIST_MANAGER` is actually constructed in our apps. |
|
||||
|
||||
So the only *perf-relevant* raw threads are the six 3D-viewer ones (all guarded). The rest are
|
||||
disabled/dormant network-IPC features, except `font_choice`, which is the lone unguarded raw thread.
|
||||
|
||||
---
|
||||
|
||||
## 3. The three failure modes (the core mechanism)
|
||||
|
||||
Keeping these apart is the whole key — different causes, different places, different fixes.
|
||||
|
||||
| Mode | Symptom | Where it bites | Root cause |
|
||||
|---|---|---|---|
|
||||
| **(a) Deadlock** | frozen tab | raytracer join (any main-thread blocking join needing a new Worker) | On-demand Worker creation needs the main-thread event loop; a *non-yielding* blocking join starves exactly that. Fixed by yielding the join (the nanosleep override) or pre-warming. **See §4.** |
|
||||
| **(b) Nesting abort** | `Aborted(invalid state: 1)` | a 2nd `emscripten_sleep` started while Asyncify is already Unwinding | Asyncify holds one global suspend state. This bites only a *genuine* nested unwind — **not** an `emscripten_sleep` dispatched at `state == Normal` (e.g. work run from a modal pump's `ProcessEvents`, a fresh managed entry; verified by `raytrace-modal`). |
|
||||
| **(c) Worker-rewind crash** | `"func is not a function"` in `Asyncify.doRewind` | a C++ throw on a pool Worker under `-fexceptions` | The `invoke_*` exception trampolines are Asyncify imports, so a throw drives an Asyncify transition on the Worker. **Native EH (the default) removes it** — exceptions become native wasm instructions, decoupled from Asyncify. |
|
||||
|
||||
The raytracer's worker tasks are mostly **pure math** (no throw, no suspend) — which is why the
|
||||
parked multi-core pool *ran*: pure-compute tasks don't hit mode (c). The pool tasks that crash
|
||||
(connectivity, library preload) throw C++ exceptions, which under `-fexceptions` drive Asyncify on
|
||||
the Worker. **Under native EH (the default) those throwing tasks are safe** — confirmed by
|
||||
`threadpool-real`, which runs the real pool 16-core with a worker task that throws and is caught.
|
||||
|
||||
### What `handlesleep.js` does and does *not* fix
|
||||
|
||||
`scripts/common/shims/handlesleep.js` fixes a **specific** nesting: a **fiber swap inside an
|
||||
`EM_ASYNC_JS` await** (e.g. `ShowModal`) clobbers the single global `Asyncify.currData`; the shim
|
||||
captures the sleep's buffer and restores it in `wakeUp`. It is *"one level of sleep nesting, blind to
|
||||
`handleAsync` and to fibers"* and does **not** bypass the `state == Normal` assertion. In practice
|
||||
that assertion is not hit by the cases we have: work dispatched from a modal pump's `ProcessEvents`
|
||||
runs at `state == Normal`, so its `emscripten_sleep` join is already legal (`raytrace-modal`). A
|
||||
**genuine** nested unwind (an `emscripten_sleep` started while already Unwinding) would still need a
|
||||
cooperative scheduler — the **Design B** design ([`../async/12`](../async/12-design-b-asyncify-implementation-plan.md),
|
||||
[`../async/13`](../async/13-design-b-engineering-spec.md); **status: Phase 0, not landed**) — but no
|
||||
current app requires it.
|
||||
|
||||
---
|
||||
|
||||
## 4. The deadlock, mechanically
|
||||
|
||||
### The event loop and "pumping"
|
||||
|
||||
Each JS context — the main thread, and each Worker — has **one** call stack and **one** task queue,
|
||||
on a strict **run-to-completion** model: pick one task, run its *entire* call stack to the end, and
|
||||
only when it unwinds back to the top pick the next task. **While a task runs, nothing else on that
|
||||
thread happens** — queued tasks (including messages from Workers) pile up undelivered. **"Pumping the
|
||||
event loop"** = finishing the current task so the thread returns to drain its queue. A function that
|
||||
runs long without returning *blocks the event loop* and starves everything behind it.
|
||||
|
||||
### Path A — `std::thread` → Worker (the *create* side)
|
||||
|
||||
`render_3d_raytrace_base.cpp:762-766`, on the main thread: `std::thread t = std::thread(shadeWorker);`
|
||||
|
||||
1. libc++ ctor → `pthread_create` → Emscripten `__pthread_create_js` → `spawnThread` (JS glue).
|
||||
2. `spawnThread` checks `PThread.unusedWorkers` (the pre-warmed pool):
|
||||
- **free Worker** → post `{cmd:'run'}` to it; it runs on its own thread. **Main need not pump.** ✅
|
||||
- **empty** (our case — KiCad's pool drained them) → `new Worker()`; the new Worker boots
|
||||
**asynchronously**, posts *"I'm loaded"* back to the main thread, and **main's message handler
|
||||
must run** to then post `{cmd:'run'}`. Finalizing a new Worker **requires main to return to the
|
||||
event loop.**
|
||||
|
||||
### Path B — the join (the *wait* side)
|
||||
|
||||
`render_3d_raytrace_base.cpp:768-769`: `while(threadsFinished < parallelThreadCount) std::this_thread::sleep_for(10ms);`
|
||||
|
||||
`sleep_for` → `nanosleep`. On the **main browser thread** a real sleep is impossible (and
|
||||
`Atomics.wait` throws there), so Emscripten implements it as a **busy-wait**: spin on the clock,
|
||||
return after 10 ms. Wrapped in the `while`, this is **one task that never ends** — the main call
|
||||
stack never unwinds to the event loop. The only thing that can move `threadsFinished` is a Worker
|
||||
reaching `threadsFinished++` (`render_3d_raytrace_base.cpp:752`).
|
||||
|
||||
### The circular wait
|
||||
|
||||
With the pre-warmed pool empty:
|
||||
1. Main calls `new Worker()` (A), then enters the busy-wait (B) and **stops pumping**.
|
||||
2. The new Worker boots and posts *"loaded"* into main's queue.
|
||||
3. **Main never processes it** (stuck in the busy-wait), so it never posts `'run'`.
|
||||
4. So the Worker never runs `shadeWorker`, never reaches `threadsFinished++`.
|
||||
5. So the `while` never exits.
|
||||
|
||||
> **Main** waits for `threadsFinished` → which needs the **Worker** to run → which needs **Main** to
|
||||
> pump and post `'run'` → which Main won't do because it's waiting for `threadsFinished`.
|
||||
|
||||
A true cyclic dependency. **It is a deadlock, not slowness** — even if Worker boot took 0 ms, it
|
||||
would never receive `'run'`. Frozen forever, not slow.
|
||||
|
||||
### Two ways to break the cycle, both zero-KiCad-edit
|
||||
|
||||
**(1) Pre-warm.** If the Worker is already in `unusedWorkers`, the entire "new Worker → loaded
|
||||
handshake → main must pump" chain is **skipped**: main posts `'run'` directly, the Worker runs *in
|
||||
parallel* with main's busy-wait, bumps the counter, the spin exits. So `PTHREAD_POOL_SIZE` ≥ (pool
|
||||
threads + peak raw-thread concurrency) means on-demand creation never happens → **the deadlock
|
||||
disappears.** The cost: the busy-wait still pegs the main thread → **jank** (not a freeze).
|
||||
|
||||
**(2) Make the join yield.** The deadlock is really "main never pumps", so making the wait *yield* to
|
||||
the event loop fixes both the deadlock *and* the jank. `wasm/shims/nanosleep_yield.c` (a strong
|
||||
`nanosleep` override) does exactly this: on the main thread a `sleep_for` join becomes an Asyncify
|
||||
yield (`emscripten_sleep` semantics), so the loop services the on-demand handshake and the Worker
|
||||
boots; on a worker thread it stays a real blocking sleep. This yield runs at `state == Normal`, so it
|
||||
does **not** trip mode (b). Proven by `pthread-ondemand` (real pool drains the pre-warmed Workers,
|
||||
raw fly-threads then boot on demand → multi-core), with no KiCad edit.
|
||||
|
||||
---
|
||||
|
||||
## 5. Native WASM exceptions (the default) and the failure modes
|
||||
|
||||
`-fwasm-exceptions` is a **size/speed de-bloat** that *keeps* Asyncify: it removes the `env.invoke_*`
|
||||
exception trampolines from `ASYNCIFY_IMPORTS` (`apply-asyncify.sh`), ~59% of the Asyncify tax (pcbnew
|
||||
**64.5 → ~36 MB gz**; [`../wasm-exceptions/`](../wasm-exceptions)). It is now the **default build**.
|
||||
|
||||
- **Raytracer — modes (a)/(b): not an EH question.** Asyncify nesting + main-thread topology. Handled
|
||||
in the wasm layer: the nanosleep override yields the join (a, §4), and a modal-pump `emscripten_sleep`
|
||||
runs at `state == Normal` (b, §3) — neither needs EH.
|
||||
- **Thread pool — mode (c): solved by native EH.** Mode (c) fires when a Worker task drives an Asyncify
|
||||
transition. Under `-fexceptions`, *exceptions themselves* do that (the `invoke_*` trampolines are
|
||||
Asyncify imports; landing pads "fire unreliably when unwinding through asyncify frames"). KiCad's
|
||||
connectivity / library-load throw as ordinary control flow, tripping it. **Native EH makes exceptions
|
||||
native wasm instructions, decoupled from Asyncify** → a throwing-but-not-suspending Worker task no
|
||||
longer drives Asyncify. **Confirmed:** `threadpool-real` runs the real `GetKiCadThreadPool()` 16-core
|
||||
with a worker task that throws and is caught — green under native EH, and *only* under native EH.
|
||||
- **Per-pass nuance:** the pure-math raytracer passes (shading/blur/`EfxFilter`) don't throw → safe
|
||||
on Workers regardless. The **geometry** passes (zone fill / polygon `Simplify`) *can* throw →
|
||||
native EH is what makes them Worker-safe.
|
||||
- **Async I/O on a Worker:** a Worker doing *async* FS I/O (`EM_ASYNC_JS`) still suspends Asyncify on
|
||||
that Worker. KiCad's library preload avoids this not by synchronous FS but because our **PCBJAM IO
|
||||
plugins proxy the async fetch to the main thread** and futex-block the Worker; the only thing left
|
||||
on the Worker is the S-expr **parse** (a throw), which native EH makes safe. Verified by
|
||||
`async-preload` (the KiCad-10 `std::async` preload shape — §10, and doc 10 §7).
|
||||
|
||||
So a Worker task is fine under native EH as long as its only Asyncify-relevant act was the exception
|
||||
itself; genuine async suspension must still be kept off the Worker (proxied to main).
|
||||
|
||||
---
|
||||
|
||||
## 6. Why `PROXY_TO_PTHREAD` is not our escape hatch
|
||||
|
||||
The textbook answer to "my native app blocks on joins" is `-sPROXY_TO_PTHREAD`: run `main()` on a
|
||||
Worker where blocking is legal. In theory the most KiCad-pristine option (delete both shims). In
|
||||
practice, off the table for us.
|
||||
|
||||
- **Asyncify under it?** Mechanically **yes** on the *proxied-main* Worker (own Asyncify state, runs
|
||||
`main()`). But **not officially supported**, with real sharp edges: `pthread_join` on a thread
|
||||
running `EM_ASYNC_JS` can hang ([#17552](https://github.com/emscripten-core/emscripten/issues/17552)),
|
||||
shutdown hangs with raw `handleSleep`/`handleAsync`
|
||||
([#16940](https://github.com/emscripten-core/emscripten/issues/16940)). **Fibers are thread-pinned**
|
||||
— [`fiber.h`](https://emscripten.org/docs/api_reference/fiber.h.html): *"Rewind IDs are
|
||||
thread-specific… impossible to resume a fiber started from a different thread."* Our tool coroutines
|
||||
are Asyncify fibers.
|
||||
- **The actual killer — the GUI can't leave the main thread.** Workers have **zero DOM access**; our
|
||||
wx-dom port renders widgets *as* DOM elements, so every widget op would have to be proxied. WebGL
|
||||
would need OffscreenCanvas or per-call GL proxying, and those only work for HTML5/SDL2 contexts
|
||||
([#8852](https://github.com/emscripten-core/emscripten/issues/8852),
|
||||
[#23666](https://github.com/emscripten-core/emscripten/issues/23666)). Clipboard/input add more
|
||||
proxying. This is a massive, risky rearchitecture of *our* layer for an unsupported config.
|
||||
|
||||
---
|
||||
|
||||
## 7. JSPI — not now
|
||||
|
||||
JSPI (VM-level stack switching, the Asyncify successor) ships in **Chrome 137+**, **Firefox 139+**,
|
||||
**Safari 27 beta** (three-engine green only once Safari 27 stables). Closed for *this* codebase
|
||||
structurally: incompatible with `emscripten_set_main_loop`
|
||||
([#22493](https://github.com/emscripten-core/emscripten/issues/22493)) — our whole architecture; can't
|
||||
replace intra-wasm `emscripten_fiber_swap`; a ~350× `JS→C→JS` re-entry regression
|
||||
([#21081](https://github.com/emscripten-core/emscripten/issues/21081)). Track it; prototype behind
|
||||
`'Suspending' in WebAssembly` on a small tool. See [`../async/03`](../async/03-solutions-and-prior-art.md)
|
||||
§3, [`../perf/README.md`](../perf/README.md) lever #10.
|
||||
|
||||
---
|
||||
|
||||
## 8. Upstream status & the upstreaming path
|
||||
|
||||
**Latest upstream KiCad (`master` `9e557f98`, 2026-06-24) has migrated only 1 of 7 raytracer
|
||||
parallel sections to the pool** — and accidentally:
|
||||
|
||||
| Section | Upstream master today |
|
||||
|---|---|
|
||||
| `renderTracing()` (main trace) | **Pool** (`submit_task` + `multi_future::wait()`) |
|
||||
| `postProcessShading` / `postProcessBlurFinish` / `renderPreview` | raw `std::thread` + busy-wait |
|
||||
| `IMAGE::EfxFilter` (image.cpp) | raw `std::thread` + busy-wait |
|
||||
| zone-fill / polygon-simplify (create_layer_items.cpp) | raw `std::thread` + busy-wait |
|
||||
|
||||
- **The one migration was a side effect.** `b99a43bec2` (2024-09-06) was a cosmetic *"render in
|
||||
Hilbert-curve order"* commit; the pool move came along for the ride. `bccf36538` (2025-04-07,
|
||||
*"Isolate thread pool loops"*, fixes GitLab #20572) then refined it from `wait_for_tasks()`
|
||||
(drain the whole pool) to `submit_task` + per-call `multi_future::wait()`, so a function waits only
|
||||
on **its own** tasks — the exact cross-frame concern we have, and the pattern any migration should
|
||||
copy.
|
||||
- **Origin of the raw-thread pattern:** `f8784f30` (2018-09-21, *"Removing OpenMP"*) hand-translated
|
||||
`#pragma omp parallel for` into raw `std::thread` + atomic counter + `sleep_for` busy-wait. The
|
||||
six un-migrated sections are this **untouched 2018 code** — legacy inconsistency, not a deliberate
|
||||
"don't use the pool" decision.
|
||||
- **A live upstream motivation:** GitLab **#20911** *"3D viewer ray tracing generates a high system
|
||||
load"* — the `sleep_for(10ms)` spin-poll + detached-thread churn is high-load *natively*. A pool
|
||||
migration (submit + futures, thread reuse, no spin) directly improves it.
|
||||
|
||||
**Conclusion:** migrating the six sections to the pool (like `renderTracing` already is) is a
|
||||
**legitimate, upstreamable cleanup** — precedent in the same file, motivation in a filed issue, and
|
||||
it removes dead OpenMP-era code. If accepted upstream, our fork carries **zero** divergence here, and
|
||||
it incidentally fixes our deadlock (pool reuse ⇒ no on-demand Worker creation). Keeping KiCad
|
||||
pristine and going multi-threaded are **not** in tension — the pristine-est KiCad (pool everywhere)
|
||||
is also the one that threads cleanly in the browser.
|
||||
|
||||
---
|
||||
|
||||
## 9. The options — and whether they are interchangeable
|
||||
|
||||
Two goals: **keep KiCad pristine** and **enable threads**. Mapping candidates to the failure modes:
|
||||
|
||||
| | (a) deadlock | (b) nesting | (c) worker-rewind | Net |
|
||||
|---|:--:|:--:|:--:|---|
|
||||
| **0. Pre-warm `PTHREAD_POOL_SIZE`** (build-only) | **✓** | n/a* | partial† | Raw-thread raytracer runs multi-core, with jank. Zero KiCad edits. = the parked `WASM_RAYTRACE_POOL`. |
|
||||
| **1. nanosleep override** (`wasm/shims/`) | **✓** | n/a | — | Main-thread join yields → on-demand Workers boot, no jank, no KiCad edit. Proven (`pthread-ondemand`). |
|
||||
| **2. Native EH** (the default) | — | — | **✓** | Worker **execution** safe for throwing pool tasks. Confirmed (`threadpool-real`, 16-core + caught throw). |
|
||||
| **3. `PROXY_TO_PTHREAD`** | sidesteps | sidesteps | sidesteps | Real threads via a DOM/WebGL-proxying rearchitecture. **Impractical for us (§6).** |
|
||||
| **4. Design B scheduler** | n/a | only a genuine nested unwind | — | Not required by any current app — the modal pump dispatches at `state == Normal` (§3). |
|
||||
|
||||
\* The upstream busy-wait never invokes Asyncify, so mode (b) doesn't arise for it. † Pure-math passes
|
||||
are mode-(c)-safe; geometry passes may throw → want native EH (option 2).
|
||||
|
||||
Read off the engines:
|
||||
- **Raytracer post-process (raw threads):** the **nanosleep override** (option 1) yields the join →
|
||||
multi-core, no jank, no KiCad edit; the **upstreamable pool migration** (§8) is the pristine option.
|
||||
- **Pool + raytracer main trace:** **native EH** (option 2, the default) makes throwing worker tasks
|
||||
safe → the `detach_task` shim can be dropped (a vendored-dep patch ⇒ *less* divergence).
|
||||
|
||||
**Where this leaves us:** native EH (default) clears mode (c); the nanosleep override clears mode (a)
|
||||
and the jank; mode (b) doesn't arise for the cases we have (modal-pump dispatch is `state == Normal`).
|
||||
All proven on pristine KiCad/wx-core by the doc-10 §6 tests. The remaining work is to **drop the
|
||||
`detach_task` shim** for real (DRC/zone-fill/connectivity on real Workers) and, optionally, **upstream
|
||||
the pool migration** (§8) so the fork carries nothing. Keep `PROXY_TO_PTHREAD`/JSPI tracked-only.
|
||||
|
||||
---
|
||||
|
||||
## 10. Open questions / decisive next steps
|
||||
|
||||
1. **Library-preload I/O — answered.** Our fork's library reads are **not** the upstream synchronous
|
||||
`KICAD_SEXPR` path: the lib-table rows are typed `PCBJAM`/`PCBJAM_FP`, so the runtime plugin is our
|
||||
custom async bridge, which **proxies the fetch to the main thread** and futex-blocks the Worker — so
|
||||
no async FS I/O suspends on the Worker. The only Worker-side Asyncify-relevant act is the S-expr
|
||||
**parse** (a throw), which native EH makes safe. So **option 2 is a shim deletion, not an I/O
|
||||
rework** for the preload path (verified by `async-preload`; full analysis in doc 10 §7).
|
||||
2. **Does `FONT_LIST_MANAGER` actually spawn its thread in our apps?** `font_choice.cpp:99` is the one
|
||||
unguarded raw `std::thread`. Confirm whether it's constructed in WASM (and whether its `Poll`
|
||||
touches anything that suspends/throws on a Worker), or whether a wasm-specific font path supersedes
|
||||
it. It won't deadlock (no join), but it likely consumes a Worker.
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
**Internal**
|
||||
- `kicad/thirdparty/thread-pool/bs_thread_pool.hpp:1419` (`detach_task` shim), `:1751`
|
||||
(`submit_task`→`detach_task`), `:1903`/`:1965` (`create_threads`/`determine_thread_count`)
|
||||
- `kicad/common/thread_pool.cpp:30-48` (`GetKiCadThreadPool`)
|
||||
- `kicad/3d-viewer/3d_rendering/raytracing/render_3d_raytrace_base.cpp:752,764,835,1456`,
|
||||
`image.cpp:525`, `create_layer_items.cpp:848,1311` (raytracer raw threads + serial fallbacks)
|
||||
- raw-thread inventory: `common/widgets/font_choice.cpp:99`, `common/eda_dde.cpp:146`,
|
||||
`kicad/pcm/pcm.cpp:1123`, `libs/kinng/src/kinng.cpp:57`; build exclusion in
|
||||
`common/CMakeLists.txt` (`KICAD_IPC_API AND NOT EMSCRIPTEN`), `CMakeLists.txt:301` (IPC default OFF)
|
||||
- `scripts/kicad/build-kicad-target.sh:413-415`, `scripts/common/shims/handlesleep.js`,
|
||||
`scripts/common/apply-asyncify.sh:88`
|
||||
- [`../async/11`](../async/11-asyncify-nesting-raytracer.md), [`../async/12`](../async/12-design-b-asyncify-implementation-plan.md),
|
||||
[`../async/13`](../async/13-design-b-engineering-spec.md), [`../async/03`](../async/03-solutions-and-prior-art.md),
|
||||
[`../wasm-exceptions/README.md`](../wasm-exceptions/README.md), [`../perf/README.md`](../perf/README.md),
|
||||
[`../../research/threading_2.md`](../../research/threading_2.md)
|
||||
|
||||
**Upstream KiCad (GitHub mirror `KiCad/kicad-source-mirror`, master `9e557f98`)**
|
||||
- `b99a43bec2` (renderTracing → pool, 2024-09-06) ·
|
||||
[`bccf36538`](https://github.com/KiCad/kicad-source-mirror/commit/bccf36538065a8c318dcdb2bc8b28bd855fb5e81)
|
||||
(*"Isolate thread pool loops"*, fixes [GitLab #20572](https://gitlab.com/kicad/code/kicad/-/issues/20572)) ·
|
||||
`452e69de` (pool singleton, 2025-01-05) · `6e2b20ed` (BS pool 5.0, 2025-09-10) ·
|
||||
`f8784f30` (*"Removing OpenMP"*, 2018-09-21) ·
|
||||
[GitLab #20911](https://gitlab.com/kicad/code/kicad/-/issues/20911) (raytrace high system load)
|
||||
|
||||
**External (Emscripten / browsers)**
|
||||
- [Pthreads](https://emscripten.org/docs/porting/pthreads.html) · [Asyncify](https://emscripten.org/docs/porting/asyncify.html) ·
|
||||
[fiber.h](https://emscripten.org/docs/api_reference/fiber.h.html) · [proxying.h](https://emscripten.org/docs/api_reference/proxying.h.html)
|
||||
- Asyncify×pthreads/PROXY: [#17552](https://github.com/emscripten-core/emscripten/issues/17552),
|
||||
[#16940](https://github.com/emscripten-core/emscripten/issues/16940),
|
||||
[#9910](https://github.com/emscripten-core/emscripten/issues/9910)
|
||||
- WebGL/DOM from a Worker: [#8852](https://github.com/emscripten-core/emscripten/issues/8852),
|
||||
[#23666](https://github.com/emscripten-core/emscripten/issues/23666)
|
||||
- JSPI: [#22493](https://github.com/emscripten-core/emscripten/issues/22493),
|
||||
[#21081](https://github.com/emscripten-core/emscripten/issues/21081),
|
||||
[V8 JSPI](https://v8.dev/blog/jspi)
|
||||
|
|
@ -1,5 +1,14 @@
|
|||
# 03 — Toolchain compatibility status (verified 2026-06-10/11)
|
||||
|
||||
> **Partly superseded 2026-06-22 — see [`06-spike-plan.md`](06-spike-plan.md).** Corrections:
|
||||
> (a) the host-side `--asyncify` already runs **Binaryen v130** in CI/publish
|
||||
> (`BINARYEN_VERSION=130`), so "we don't have the partial support locally" understates it — v121
|
||||
> is only the finalize/in-link copy. (b) `--pass-arg=asyncify-ignore-unwind-from-catch` **is**
|
||||
> implemented (shipped v125), but it *silently drops* the suspend — a tripwire-silencer, not a
|
||||
> fix. (c) The encoding is resolved to **legacy** (exnref + Asyncify is unsupported in every
|
||||
> released Binaryen incl. v130, no roadmap), so the "encoding decision forks the asyncify work"
|
||||
> framing in §experiment is closed: legacy + the catch-arm-hoisting pre-pass is the only path.
|
||||
|
||||
## The compatibility matrix
|
||||
|
||||
| Combination | Status |
|
||||
|
|
|
|||
|
|
@ -1,75 +0,0 @@
|
|||
# 04 — The KiCad catch-block audit
|
||||
|
||||
## Why an audit
|
||||
|
||||
Binaryen's asyncify cannot (yet) handle a suspension that begins while execution is inside
|
||||
a wasm catch handler (see 05). Any C++ `catch` whose handler (directly or transitively)
|
||||
opens a modal dialog, touches the async clipboard, etc., is therefore illegal under
|
||||
`-fwasm-exceptions` + ASYNCIFY=1 today. KiCad's standard error pattern is exactly that:
|
||||
|
||||
```cpp
|
||||
catch( const IO_ERROR& ioe )
|
||||
{
|
||||
DisplayErrorMessage( this, ioe.What() ); // → ShowModal → Asyncify suspension
|
||||
}
|
||||
```
|
||||
|
||||
## Method
|
||||
|
||||
[`catch_audit.py`](catch_audit.py): walks `kicad/**/*.cpp` (excluding `thirdparty/`, `qa/`),
|
||||
extracts every catch block with real brace matching (string/comment aware), and classifies
|
||||
each handler body:
|
||||
|
||||
- **direct_suspend** — contains a known suspending call (`DisplayError[Message]`,
|
||||
`DisplayInfoMessage`, `wxMessageBox`, `ShowModal`, `ShowQuasiModal`, `KIDIALOG`,
|
||||
`IsOK(`, clipboard ops, `wxFileDialog`, …).
|
||||
- **infobar** — only `ShowInfoBar*` (non-modal, non-suspending).
|
||||
- **trivial** — only rethrow / capture / format / logging. Note `wxLogError` is *safe*:
|
||||
its GUI display is deferred to the idle-time log flush, outside any catch.
|
||||
`PGM_BASE::HandleException` (`common/pgm_base.cpp:805`) was manually verified — it only
|
||||
`wxLogError`s → benign.
|
||||
- **needs_review** — calls functions not classifiable as benign; requires a transitive look.
|
||||
|
||||
Full output incl. all site locations: [`audit-results.txt`](audit-results.txt).
|
||||
Re-run: `python3 catch_audit.py` (path to the kicad tree is hardcoded at the top).
|
||||
|
||||
## Results (2026-06-10, kicad @ wasm-port head)
|
||||
|
||||
| Category | Count |
|
||||
|---|---|
|
||||
| **direct_suspend** | **85** |
|
||||
| needs_review | 93 (tail looks mostly benign — accessors/file ops; expect ~10–20 to become refactors on inspection) |
|
||||
| trivial/safe | 458 |
|
||||
| infobar-only | 0 |
|
||||
| **total** | **636** |
|
||||
|
||||
Per app (direct/review): eeschema 37/17, pcbnew 32/29, common 9/35, cvpcb 3/1, rest
|
||||
scattered. Concentrated exactly on the file-load error paths the e2e tests exercise:
|
||||
`pcbnew/files.cpp`, `eeschema/files-io.cpp`, `footprint_libraries_utils.cpp`,
|
||||
`symbol_library_manager.cpp`, the design-block utils.
|
||||
|
||||
wxWidgets adds essentially nothing (zero catch-with-dialog sites in its own code).
|
||||
|
||||
## The hand-refactor option (superseded by the fork, kept for the record)
|
||||
|
||||
Mechanical hoist per site:
|
||||
|
||||
```cpp
|
||||
wxString err;
|
||||
try { ... }
|
||||
catch( const IO_ERROR& ioe ) { err = ioe.What(); } // capture only
|
||||
if( !err.IsEmpty() ) DisplayErrorMessage( this, err ); // suspend OUTSIDE the catch
|
||||
```
|
||||
|
||||
Effort: 85 hoists (~15–30 min each; error-UX paths with near-zero test coverage) ≈ 3–5
|
||||
dev-days; 93 reviews ≈ 2–3 dev-days; destructor-during-unwind audit ≈ half day (cleanup
|
||||
pads = `catch_all`; dialogs from destructors expected zero); **plus** permanent
|
||||
upstream-sync policing (every KiCad merge adds new `catch { DisplayError }` sites —
|
||||
`catch_audit.py` as a CI gate automates detection), **plus** the fork-divergence cost
|
||||
against the "stay close to upstream" policy. Total ~2–3 weeks one-time + maintenance tax,
|
||||
with hard-crash failure modes for any missed transitive site (asyncify asserts mode traps
|
||||
deterministically — good in CI, fatal in production).
|
||||
|
||||
**Verdict:** with the catch-arm-hoisting fork (05) this entire refactor becomes
|
||||
unnecessary — all 85 direct sites are C++ catches, which the fork makes legal. Only
|
||||
suspend-inside-`catch_all`-cleanup remains forbidden, which KiCad does not do.
|
||||
267
docs/features/wasm-exceptions/06-spike-plan.md
Normal file
267
docs/features/wasm-exceptions/06-spike-plan.md
Normal file
|
|
@ -0,0 +1,267 @@
|
|||
# 06 — Native wasm-EH: refreshed findings + red-green spike plan (2026-06-22)
|
||||
|
||||
> **Status:** plan / decision record, *in refinement* (no code written yet — this is the
|
||||
> agreed artifact before Phase 0). Produced by a 5-agent research spike on 2026-06-22:
|
||||
> browser support · toolchain · runtime mechanisms · the Binaryen pass · red-green harness.
|
||||
> **Supersedes in part** `README.md`, `03-toolchain-status.md`, and the root
|
||||
> `docs/wasm-exceptions-experiment.md` where called out below. Reads on top of 01–05.
|
||||
|
||||
---
|
||||
|
||||
## RESULTS — Phases 0 & 1 (2026-06-22, current toolchain, NO emsdk bump)
|
||||
|
||||
> **Cross-engine policy:** every spec runs in **all three engines — Firefox, Chrome (V8), and
|
||||
> Safari/WebKit** (`cd tests && npm run test:asyncify:all`). The eh-spike harness is green in all
|
||||
> three. Result notes below that say "V8/Firefox" predate the Safari run and now hold in WebKit too.
|
||||
|
||||
**Phase 0 — PASS.** em 4.0.2's LLVM emits *parseable, runnable* legacy wasm-EH:
|
||||
- A trivial `-fwasm-exceptions -sWASM_LEGACY_EXCEPTIONS=1` throw/catch builds (finalize on
|
||||
bundled v121, **no "popping from empty stack"**) and runs correctly under node
|
||||
(`tests/apps/standalone/eh-spike/eh_probe.cpp`, Makefile `eh-probe` target). So the parked
|
||||
experiment's finalize failure was **scale/OCC-specific, not a general codegen break**.
|
||||
- Binaryen **v130 asyncifies** the legacy-wasm-EH module (asyncify_* exports present); bundled
|
||||
**v121 crashes** (`UNREACHABLE … Asyncify.cpp:1146 — unexpected expression type`). Confirms
|
||||
the v121→v130 split; the post-link asyncify path (already `BINARYEN_VERSION=130` in CI) is the
|
||||
one to use. **No emsdk bump required for the spike.**
|
||||
|
||||
**Phase 1 — PASS (red-green captured).** One source built two ways
|
||||
(`scripts/build-eh-spike.sh`; the wasm-EH variant uses the production stub→post-link-v130 flow).
|
||||
Validated in **both V8 (node) and Firefox** by `tests/asyncify/eh-spike.spec.ts`:
|
||||
|
||||
| case | mechanism | JS-EH | native wasm-EH |
|
||||
|---|---|---|---|
|
||||
| `throw_across_sleep` | EM_ASYNC_JS sleep + EH | PASS | **PASS** |
|
||||
| `fiber_then_throw` | `emscripten_fiber_swap` + EH | PASS | **PASS** |
|
||||
| `suspend_in_catch` | suspend *inside* a catch arm | PASS | **HARD TRAP** — `indirect call to null` / `null function or function signature mismatch` |
|
||||
|
||||
→ asyncify + native wasm-EH **works for the sleep and fiber mechanisms**; the *only* failure is
|
||||
suspend-inside-catch (Binaryen #4470), and it fails **loud and deterministically** (a trap, not a
|
||||
silent drop) in both engines. This is exactly the hole the catch-arm-hoisting pass (05) closes —
|
||||
the spike has now made it concrete and pinned it as a regression test.
|
||||
|
||||
**Artifacts:** `tests/apps/standalone/eh-spike/{eh_probe.cpp,eh_spike_test.cpp}`,
|
||||
`scripts/build-eh-spike.sh`, `tests/asyncify/eh-spike.spec.ts`, the `eh-probe` Makefile target,
|
||||
and `playwright-asyncify.config.ts` `testMatch` widened to include `eh-spike`.
|
||||
|
||||
**Phase 1.5 — PASS (the fix).** The catch-arm-hoisting Binaryen pass
|
||||
(`scripts/binaryen-hoist-pass/HoistCppCatches.cpp`, ~150 LoC + a 39-line registration patch)
|
||||
flips `suspend_in_catch` from a hard trap to **green** under native wasm-EH — validated in
|
||||
**V8 + Firefox**. `tests/asyncify/eh-spike.spec.ts` is now a 3-variant ablation harness:
|
||||
JS-EH green / wasm-EH-no-pass red / wasm-EH+hoist green (pins both disease and fix). The pass
|
||||
outlines a cpp-tag catch arm containing a suspending call to plain code after the try (capture
|
||||
payload→local + set flag + `br` out; in the hoisted handler `pop`→`local.get`, no-match
|
||||
`rethrow`→`throw`), so stock `--asyncify` instruments it for free — exactly the 05 design. It is
|
||||
a PRE-pass (`--hoist-cpp-catches` before `--asyncify`); `Asyncify.cpp` is unchanged. Built
|
||||
reproducibly via `scripts/binaryen-hoist-pass/build-wasm-opt.sh` (drops the file into the v130
|
||||
clone, applies the patch, `ninja`). **MVP scope** (sufficient for the toy): void/unreachable-typed
|
||||
tries, single cpp catch (single-i32 tag), no catch_all, and a DIRECT suspending-import call in the
|
||||
arm; the nested-pop / nested-rethrow cases are handled (via an `ExpressionStackWalker` Try-ancestor
|
||||
guard). **Remaining for real KiCad** (all "fiddly-but-tractable" per 05): concrete-result-typed
|
||||
tries (route the body value through a temp local), catch_all coexistence, and TRANSITIVE suspend
|
||||
detection (catch → DisplayErrorMessage → ShowModal → startModal), which needs the asyncify
|
||||
ModuleAnalyzer rather than the direct-call heuristic — or simply hoist-all-cpp-catches and let
|
||||
`-O2` clean up.
|
||||
|
||||
**Next:** Phase 2 — flip a small wx standalone app to native EH (the real `ShowModal`-from-catch
|
||||
path). Phase 3 — full KiCad + the emsdk bump (OCC only). Before KiCad: generalize the pass
|
||||
(result-typed tries + transitive detection) and file the design on binaryen #4470.
|
||||
|
||||
---
|
||||
|
||||
## 0. What changed since 01–05 (which were authored 2026-06-11/12)
|
||||
|
||||
Three corrections that move the decision:
|
||||
|
||||
1. **The encoding is resolved: use the LEGACY encoding (`-sWASM_LEGACY_EXCEPTIONS=1`).**
|
||||
Not "decide after the emsdk bump" (as README §TL;DR / 03 §experiment / 05 §new-EH frame
|
||||
it). Binaryen's Asyncify supports **only** legacy `try`/`catch` — never
|
||||
`try_table`/`throw_ref`/`exnref` — through the latest **v130** (Jun 2026), with no
|
||||
roadmap, PR, or TODO to change it. The exnref path the parked experiment was forced onto
|
||||
is therefore a **dead end**: even with the OCC bug fixed, an exnref module dies at the
|
||||
`--asyncify` step. Legacy has shipped *unflagged* in all three engines since 2021–22
|
||||
(Chrome 95 / Safari 15.2 / Firefox 100), is still emscripten's own default, and — crucially
|
||||
— the size prize comes from **native-vs-JS EH (dropping `invoke_*`), independent of the
|
||||
encoding** — so legacy costs us nothing. **The "exnref → TryTable variant" fork is closed:
|
||||
legacy + the catch-arm-hoisting pre-pass (05) is the single viable path.**
|
||||
|
||||
2. **Binaryen is no longer a blocker — production already ships v130.** README/03 say "our
|
||||
emsdk bundles v121, we don't even have the partial support locally." That v121 is only the
|
||||
*finalize / test-apps-in-link* copy. **CI and publish pin `BINARYEN_VERSION=130`**
|
||||
(`.github/workflows/ci-ubicloud.yml:44`, `.github/workflows/publish-wasm.yml:33`;
|
||||
resolved by `scripts/common/get-wasm-opt.sh:50`) for the host-side `--asyncify` + `-O2`.
|
||||
v130 is the latest release and carries the v125 partial legacy-EH asyncify support. So the
|
||||
migration plan's "step 1: get a newer wasm-opt for the post-link step" is **already done**.
|
||||
|
||||
3. **`--pass-arg=asyncify-ignore-unwind-from-catch` is implemented now** (shipped in
|
||||
Binaryen v125; `03:23` called it "dead docs"). But it is a **tripwire-silencer, not a
|
||||
fix**: it *silently drops* the suspension inside a catch arm, which is semantically wrong
|
||||
on paths we actually reach (file-load error dialogs, the eeschema **Paste** handler).
|
||||
Correctness still requires the catch-arm-hoisting pass (or a hand-refactor).
|
||||
|
||||
**Net:** the long pole shrinks to a single thing — the **emsdk / LLVM compiler bump** (for
|
||||
*parseable legacy wasm-EH* + the OCC `br_table` miscompile), **not** the Binaryen version.
|
||||
|
||||
---
|
||||
|
||||
## 1. The decision in one line
|
||||
|
||||
> `-fwasm-exceptions -sSUPPORT_LONGJMP=wasm -sWASM_LEGACY_EXCEPTIONS=1`, applied **uniformly**
|
||||
> across deps + wxWidgets + KiCad + test apps; host-side `--asyncify` on Binaryen ≥ v125
|
||||
> (we have v130); the ~85 suspend-in-catch sites fixed by the **catch-arm-hoisting pre-pass**
|
||||
> (05), not by source refactors. Prize unchanged from 02: **64.5 → ~36 MB gz download,
|
||||
> 187 → ~122 MB module**, plus a large `-O2` wall-time drop on the build critical path.
|
||||
|
||||
---
|
||||
|
||||
## 2. What works / what doesn't (verdicts from this spike)
|
||||
|
||||
| | Verdict | Why |
|
||||
|---|---|---|
|
||||
| Native wasm-EH, **legacy** encoding, + Asyncify v130, + catch-arm fix | ✅ **viable path** | Mechanically sound per all five agents; legacy is the only encoding Asyncify can instrument. |
|
||||
| **exnref** (`=0`) + Asyncify | ❌ architecturally broken | Asyncify has zero `TryTable`/`exnref` support in *every* released Binaryen incl. v130; no roadmap. The experiment's `=0` never reached asyncify (blocked earlier at finalize). |
|
||||
| `--asyncify-ignore-unwind-from-catch` as a *fix* | ❌ not a fix | Exists (v125+), but silently drops the suspend → our catch→modal dialogs misbehave. A tripwire-silencer only. |
|
||||
| Bundled Binaryen **v121** + Asyncify + any wasm-EH | ❌ crashes | `Asyncify.cpp:998 UNREACHABLE`. Need ≥ v125 — already satisfied for the host-side pass (v130). |
|
||||
| JSPI instead of Asyncify | ❌ closed for us | Fibers don't exist under JSPI (emscripten #18180); KiCad tools are fibers. (`03`) |
|
||||
|
||||
---
|
||||
|
||||
## 3. The three asyncify mechanisms vs. the EH switch
|
||||
|
||||
All three suspension mechanisms + handle-sleep are **orthogonal** to the C++ EH model — none
|
||||
touches `__cxa_*`/landing pads in its own implementation. (Full detail: this session's
|
||||
mechanism map + `docs/features/async/`.)
|
||||
|
||||
| Mechanism | EH-model dependence | Verdict |
|
||||
|---|---|---|
|
||||
| **Fibers** (`emscripten_fiber_swap`, libcontext/`coroutine.h`) | none — no try/catch in the coroutine layer | just works |
|
||||
| **Main-loop park** (`emscripten_set_main_loop(...,1)` → `throw "unwind"`) | none — it's a **JS string** throw, not `__cxa_throw`; swallowed in JS glue | just works |
|
||||
| **EM_ASYNC_JS sleeps + handle-sleep engine** (modal/nested-loop/clipboard/fonts/DOM-popup/PCBJam fetch) | implementation: none. **Callers** are the risk. | at-risk *only* via the catch-arm caller pattern |
|
||||
|
||||
The §3c trampoline-heal (`inject-dyncall-shims.sh`) and `handlesleep.js` "unwind" catches are
|
||||
**JS-level and EH-independent** — unaffected by the switch.
|
||||
|
||||
**The entire exposure** is the **~85 sites where a C++ `catch` arm opens a modal →
|
||||
asyncify-suspends** (e.g. `pcbnew/files.cpp:674/684/692`; 6 of them on a coroutine/fiber
|
||||
stack including the eeschema Paste handler). This trap has **never fired at runtime** — it is
|
||||
a static conclusion (brace-match audit `04` + Binaryen `AsyncifyFlow` skipping catch bodies
|
||||
`05`). **Making it concrete and proving the fix is the spike's core job.**
|
||||
|
||||
---
|
||||
|
||||
## 4. Blockers, ordered (corrected)
|
||||
|
||||
| # | Blocker | Status / fix | Confidence |
|
||||
|---|---|---|---|
|
||||
| 1 | **emsdk 4.0.2 → 6.0.0 (LLVM 23) compiler bump** | em 4.0.2 emitted legacy wasm-EH that failed Binaryen parse at finalize (`popping from empty stack`) with **both** v121 and official v130 → an **LLVM-output bug, not a Binaryen-version bug**; a newer LLVM should fix it. Also lifts the bundled finalize-binaryen to ~v130. **Changes the JS-EH build too → must revalidate the whole project.** | parse-failure cause **unconfirmed** (Phase 0 resolves) |
|
||||
| 2 | **OpenCASCADE** | (a) invalid `br_table` arity in `ShapeUpgrade_SplitSurface::Build` under wasm-EH — candidate upstream fix is **LLVM PR #123915** (Jan 2025, "add unreachable before catch destinations"), so the LLVM-23 bump likely covers it. (b) Separate `OCC_CONVERT_SIGNALS` setjmp↔exception-in-one-function conflict — **likely already sidestepped** by `-sSUPPORT_LONGJMP=wasm`; verify, else drop the flag. | both **to verify** in Phase 3 |
|
||||
| 3 | **~85 suspend-in-catch sites** | catch-arm-hoisting Binaryen pre-pass — ~400–800 LoC, 1–2 wk, upstreamable, **keeps KiCad pristine** and obsoletes the hand-refactor + CI gate (05, 04). Alternative: hand-refactor (2–3 wk + permanent `catch_audit.py` CI gate + fights our upstream-closeness policy). | design **sound** (05); not yet built |
|
||||
| 4 | **Safari 26.0 regression** (watch) | `-fwasm-exceptions` *legacy* apps transiently crashed at startup on Safari 26.0's initial release (in-place-interpreter bug, emscripten #25365), since patched. Track Safari point releases. | external, **patched** |
|
||||
|
||||
Browser support is otherwise a non-issue: legacy EH ≈ Chrome 95+/Safari 15.2+/Firefox 100+,
|
||||
>96% of traffic, a 4-year tail. The "Chrome problem" (V8 slow to ship *exnref*) doesn't touch
|
||||
us because we never emit exnref.
|
||||
|
||||
---
|
||||
|
||||
## 5. The phased red-green plan
|
||||
|
||||
**Key enabler (agent 5):** the Asyncify×wasm-EH interaction is **toy-testable now, without the
|
||||
emsdk bump** — the bump's blockers are OCC-specific, and the toy has no OCC. So we separate the
|
||||
two risks: *compiler bump* (full-KiCad only) vs *asyncify×wasm-EH semantics* (provable on a toy
|
||||
today). We reuse the existing ablation harness pattern in
|
||||
`tests/apps/standalone/{asyncify-races,coroutine}/` + `tests/asyncify/`.
|
||||
|
||||
### Phase 0 — micro-probe (hours, current toolchain)
|
||||
**Question:** can em 4.0.2's LLVM emit *parseable* legacy wasm-EH on a tiny no-OCC program?
|
||||
- Minimal standalone C++ (no wx, no OCC): a `try { throw } catch(...) {}` + a trivial sleep.
|
||||
- Build `-fwasm-exceptions -sSUPPORT_LONGJMP=wasm -sWASM_LEGACY_EXCEPTIONS=1`, **`-sASYNCIFY=0`
|
||||
at link**; does it link + `wasm-emscripten-finalize` (bundled v121) cleanly? (i.e. reproduce
|
||||
the experiment's `=1` failure, or not).
|
||||
- Then standalone `wasm-opt --asyncify` via `get-wasm-opt.sh` with `BINARYEN_VERSION=130` —
|
||||
does v130's asyncify parse + instrument the legacy-EH module?
|
||||
- **Gate:** both clean → Phase 1 proceeds on current toolchain, bump deferred to Phase 3.
|
||||
Finalize fails the same way → em-4.0.2 legacy codegen is generally broken → **front-load the
|
||||
bump** (Phase 0′) before the toy.
|
||||
|
||||
### Phase 1 — red-green toy (days)
|
||||
**Goal:** prove all three mechanisms survive native wasm-EH, and that **suspend-in-catch is the
|
||||
only failure** (red) — then green once the fix lands.
|
||||
- Add `tests/apps/standalone/eh-spike/eh_spike_test.cpp`, console protocol
|
||||
`[EH_SPIKE] PASS/FAIL/SUMMARY` (model on `races_test.cpp`). Three cases:
|
||||
(a) **throw across a sleep**; (b) **suspend inside a catch** (the #4470 case);
|
||||
(c) **fiber-swap then throw** (include `../coroutine/kicad_coroutine_harness.h`).
|
||||
- One source, two builds via a make var `EH_FLAGS`: `-fexceptions` (green baseline) vs
|
||||
`-fwasm-exceptions … =1`. No-wx/no-OCC (template on the `coroutine-pthread` no-wx
|
||||
`LDFLAGS`), so we skip the wx-EH rebuild and the OCC bug.
|
||||
- **Decouple asyncify**: link `-sASYNCIFY=0`, then standalone `wasm-opt --asyncify`
|
||||
(`get-wasm-opt.sh`, v130) so the in-link v121 doesn't poison case (b). Re-inject shims.
|
||||
- Playwright spec `tests/asyncify/eh-spike.spec.ts` (model on `asyncify-races.spec.ts`,
|
||||
reuse `findSummary`/`crashLines`): JS-EH build **all green**; wasm-EH build **green on (a)/(c),
|
||||
red on (b)**. Build commands: `scripts/build-wasm-test.sh eh-spike-{js,wasm}`; run
|
||||
`cd tests && npx playwright test --config=playwright-asyncify.config.ts --project=firefox …`
|
||||
(widen `testMatch` or name it `asyncify-races-eh.spec.ts`).
|
||||
- **Bonus data:** characterize what (b) does under wasm-EH — hard trap vs silent drop (the
|
||||
`ignore-unwind-from-catch` behavior). Informs decision-point #1.
|
||||
|
||||
### Phase 1.5 — the catch-arm-hoisting pass (1–2 wk; only if Phase 1 confirms (b) is the sole gap)
|
||||
- Implement `src/passes/HoistCppCatches.cpp` + 2 registration lines in a Binaryen fork branch
|
||||
(05 §transform). Wire via `get-wasm-opt.sh` `BINARYEN_BUILD_FROM_SOURCE=1` pointed at the
|
||||
fork branch (`get-wasm-opt.sh:59-90`; one-line URL/branch change).
|
||||
- Prove case (b) goes **green** on the toy. File the design on **binaryen #4470** first
|
||||
(upstreamability).
|
||||
|
||||
### Phase 2 — wx standalone (days)
|
||||
- Flip a small wx test app to native EH (wx native-EH rebuild via the `KICAD_WASM_EH` gate;
|
||||
flag set patch-ready in the experiment appendix — but with `=1`, **not** the experiment's
|
||||
`=0`). Exercises the **real** `ShowModal`-from-catch suspend in miniature; confirms the pass
|
||||
handles the actual wx modal, not just a toy sleep.
|
||||
|
||||
### Phase 3 — full KiCad (weeks)
|
||||
- emsdk bump → 6.0.0 (LLVM 23): **revalidate the JS-EH build first** (whole-project risk).
|
||||
- OCC: verify `br_table` fixed; verify (and if needed remove) `OCC_CONVERT_SIGNALS`.
|
||||
- Apply the experiment appendix patch (`KICAD_WASM_EH=1`) **corrected to `=1`**; uniform flag
|
||||
flip across deps/wx/kicad/tests; **drop `env.invoke_*` from `ASYNCIFY_IMPORTS`**
|
||||
(`apply-asyncify.sh` — README cites `:33`, current read ~`:88`; verify).
|
||||
- Build host-side asyncify with the hoisting pass (fork). e2e audit under
|
||||
`-sASYNCIFY_ASSERTIONS` to flush any missed suspend-in-catch. Measure the gz/module win + the
|
||||
`-O2` wall-time drop.
|
||||
|
||||
---
|
||||
|
||||
## 6. Open decision points (to refine together)
|
||||
|
||||
1. **Catch-arm fix strategy:** hoisting pass (rec — pristine KiCad, upstreamable) vs
|
||||
hand-refactor 85 sites (CI gate, fights upstream-closeness) vs ship-with-`ignore-flag` and
|
||||
accept degraded error dialogs (fast, semantically wrong). *Phase 1 data informs this.*
|
||||
2. **emsdk target:** 6.0.0 (latest, LLVM 23) vs a more conservative 5.0.x — 6.0.0 carries
|
||||
other breaking changes (startup `async/await`, compiler-rt naming) needing JS-EH-side
|
||||
revalidation.
|
||||
3. **Binaryen fork hosting:** real git submodule (sibling to kicad/wxwidgets, pinned) vs a
|
||||
lighter `BINARYEN_BUILD_FROM_SOURCE` branch URL. (Fork is a *build tool*, not conveyed code
|
||||
— no GPLv3 `BUILD_SHA` treatment needed.)
|
||||
4. **Run Phase 1 on current em 4.0.2** (if Phase 0 passes) **vs bump first regardless.**
|
||||
5. **Sequencing vs the Asyncify-arbiter work** (`docs/features/async/`): README §relationship
|
||||
says arbiter-first (it fixes shipping bugs, needed under either EH model). Still the priority?
|
||||
|
||||
---
|
||||
|
||||
## 7. Risks / unknowns (honest)
|
||||
|
||||
- The `popping from empty stack` cause is **unconfirmed** — Phase 0 is the cheap decider.
|
||||
- Whether case (b) under wasm-EH **traps vs silently drops** — Phase 1 characterizes.
|
||||
- Whether **LLVM 23 actually fixes** our OCC `br_table` (PR #123915 is a strong candidate, not
|
||||
confirmed against our exact OCC code) — Phase 3 verifies.
|
||||
- Safari 26.0 legacy-EH regression — patched upstream, but a reminder to track point releases.
|
||||
- File:line refs in 03/05/README and the experiment appendix may have **drifted** — re-verify
|
||||
before editing (e.g. `apply-asyncify.sh` `ASYNCIFY_IMPORTS` line, build-script EH-flag lines).
|
||||
|
||||
---
|
||||
|
||||
## 8. Provenance
|
||||
|
||||
5-agent spike, 2026-06-22 — browser support, toolchain, runtime mechanisms, the pass design,
|
||||
red-green harness. Key external refs: WebAssembly/binaryen **#4470** (open) / **#5475** (merged
|
||||
v125), **LLVM PR #123915**, **emscripten #25365** (Safari 26.0), webassembly.org **Wasm 3.0**
|
||||
(Sep 2025). Internal: this dossier 01–05, `docs/wasm-exceptions-experiment.md`,
|
||||
`docs/features/async/`, `docs/features/perf/README.md` (lever #9), and memory
|
||||
`wasm-eh-migration-assessment`.
|
||||
244
docs/features/wasm-exceptions/07-spike-results-and-opinion.md
Normal file
244
docs/features/wasm-exceptions/07-spike-results-and-opinion.md
Normal file
|
|
@ -0,0 +1,244 @@
|
|||
# 07 — Native wasm-EH spike: results and engineering opinion (2026-06-22)
|
||||
|
||||
> What the spike actually built and proved, then a candid opinion on whether to pursue the
|
||||
> `-fwasm-exceptions` migration and how. Companion to [`06-spike-plan.md`](06-spike-plan.md)
|
||||
> (the plan + Phase 0/1/1.5 result log). Verdict up front, evidence and caveats after.
|
||||
|
||||
## Verdict
|
||||
|
||||
**The migration is viable on the current toolchain, the single blocking limitation is real
|
||||
and narrow, and a bounded Binaryen pass fixes it. I recommend pursuing it** — but the
|
||||
schedule risk is the **emsdk bump for OpenCASCADE**, not the exception machinery, and the
|
||||
pass still needs generalization before full KiCad. This was a genuine de-risking: the core
|
||||
"can Asyncify and native wasm-EH coexist?" question is now answered **yes, empirically**,
|
||||
in **all three engines — Chrome (V8), Firefox (SpiderMonkey), and Safari (WebKit)** — with no
|
||||
emsdk bump required to prove it. (Project policy, set this session: every spec is run in all
|
||||
three browsers — `cd tests && npm run test:asyncify:all`.)
|
||||
|
||||
## What was proven (with evidence, not estimates)
|
||||
|
||||
Everything below ran on the **current pinned toolchain** (emscripten 4.0.2, Binaryen v130 via
|
||||
`BINARYEN_VERSION=130`) — **no emsdk bump**.
|
||||
|
||||
1. **em 4.0.2 emits runnable legacy wasm-EH.** A trivial `-fwasm-exceptions
|
||||
-sWASM_LEGACY_EXCEPTIONS=1` throw/catch builds (finalize on the bundled v121, *no* "popping
|
||||
from empty stack") and runs correctly under node. So the parked experiment's finalize
|
||||
failure was scale/OCC-specific, not a general codegen break.
|
||||
(`tests/apps/standalone/eh-spike/eh_probe.cpp`.)
|
||||
2. **Binaryen v130 asyncifies legacy-wasm-EH; v121 cannot.** v130 instruments the module
|
||||
(asyncify_* exports appear); the emsdk-bundled v121 crashes (`UNREACHABLE …
|
||||
Asyncify.cpp:1146`). We already ship v130 for the post-link asyncify, so this costs nothing.
|
||||
3. **The three Asyncify mechanisms split exactly as predicted.** A red-green toy
|
||||
(`eh_spike_test.cpp`) exercises sleep-across-throw, fiber-swap-then-throw, and
|
||||
suspend-inside-catch. Under native wasm-EH: **sleep ✅, fiber ✅, suspend-in-catch ❌
|
||||
(hard trap: `indirect call to null`).** Identical in V8 (node) and Firefox. So the *only*
|
||||
failure mode is the documented one (Binaryen #4470: AsyncifyFlow skips catch bodies), and
|
||||
it fails **loudly and deterministically** — not a silent corruption.
|
||||
4. **A ~150-line Binaryen pass closes it.** `--hoist-cpp-catches`
|
||||
(`binaryen/src/passes/HoistCppCatches.cpp`, our fork) flips suspend-in-catch to **green**
|
||||
in both engines. `tests/asyncify/eh-spike.spec.ts` is a 3-variant ablation harness pinning
|
||||
JS-EH-green / wasm-EH-red / wasm-EH+hoist-green. Rebuilt and re-verified end-to-end from the
|
||||
tracked submodule.
|
||||
|
||||
## The artifacts (all reproducible)
|
||||
|
||||
| Thing | Where |
|
||||
|---|---|
|
||||
| Red-green toy (3 mechanisms) | `tests/apps/standalone/eh-spike/eh_spike_test.cpp` |
|
||||
| Phase-0 probe | `tests/apps/standalone/eh-spike/eh_probe.cpp` (+ `eh-probe` Makefile target) |
|
||||
| 3-variant build (stub→post-link-v130, +hoist) | `scripts/build-eh-spike.sh` |
|
||||
| The Binaryen pass | `binaryen/` submodule (fork, branch `wasm-port` = `version_130` + the pass) |
|
||||
| Fork build wrapper | `scripts/binaryen-hoist-pass/build-wasm-opt.sh` |
|
||||
| Red-green-fixed spec | `tests/asyncify/eh-spike.spec.ts` |
|
||||
|
||||
## How the pass works (one paragraph)
|
||||
|
||||
For a `try` whose cpp-tag `catch` arm contains a suspending call, it rewrites the arm to just
|
||||
*capture the exception payload into a local and set a flag*, and **hoists the real handler to
|
||||
plain straight-line code after the try**, guarded by the flag. In the hoisted handler the
|
||||
payload `pop` becomes a `local.get`, and the personality no-match `rethrow` becomes an explicit
|
||||
`throw` of the cpp tag with the captured payload. Stock `--asyncify` then instruments the
|
||||
hoisted handler like any other code — the upstream "no pause/resume inside catchBodies"
|
||||
invariant becomes true *by construction*. It is a **pre-pass**; `Asyncify.cpp` is unchanged,
|
||||
which is why it's a clean ~150-line addition and genuinely upstreamable.
|
||||
|
||||
## Generalization (follow-up, same session)
|
||||
|
||||
The pass was generalized from the MVP (one direct-suspend catch) to **hoist-all-cpp-catches** and
|
||||
tested against a richer toy covering the real KiCad/wx shapes. **All 7 shapes are green in all
|
||||
three engines** (Firefox + Chrome + Safari/WebKit):
|
||||
|
||||
| shape | covered |
|
||||
|---|---|
|
||||
| direct suspend in catch | ✅ |
|
||||
| transitive (catch → helper → … → suspend) | ✅ (hoist-all; direct detection would miss it) |
|
||||
| value-returning try/catch | ✅ (LLVM keeps the value in a local → void try, no result routing needed) |
|
||||
| suspend-in-catch on a fiber/coroutine stack (eeschema Paste) | ✅ |
|
||||
| nested suspend-in-catch tries | ✅ |
|
||||
| **catch nested in a catch_all cleanup** (try body has a local with a destructor) | ✅ (escape past the outermost try; see below) |
|
||||
|
||||
**The one gap — catch_all-wrapped catches.** When the try body holds a local with a non-trivial
|
||||
destructor, LLVM lowers the C++ catch *nested inside* the cleanup `catch_all`
|
||||
(`catch_all { ~g; try { rethrow } catch $cpp { sleep } }`). Hoisting the cpp catch leaves the sleep
|
||||
inside the `catch_all`; legacy `catch_all` gives no payload to capture/re-raise, so the sleep can
|
||||
only be freed by hoisting **past the outermost enclosing try** — have the cpp catch capture the
|
||||
payload and `br` to a `$done` block placed after that try. **Prototyped this session and
|
||||
reverted:** the escape transform *validates* and fixes the catch_all case *in isolation*, but the
|
||||
`block` + `br` + flag-dispatch control-flow shape it produces is **not asyncify-rewindable**
|
||||
(rewind traps with `null function`) and it regressed the simple cases too. So the real work is
|
||||
finding an asyncify-friendly escape shape — the per-try inline `br_if`-skip form (handler inline
|
||||
right after the try) rewinds fine; a `br` out to a separate dispatch does not. That's the
|
||||
fix is now landed (see "catch_all-escape: LANDED" below); all 7 shapes are green. How often it
|
||||
bites KiCad depends on whether the specific catch's try body constructs a destructible
|
||||
local/temporary (e.g. a `wxString`); a `try { ptr = Load(fn); } catch(IO_ERROR&)` with a pointer
|
||||
result has no cleanup pad and is already covered. (`HOIST_ONLY_SUSPEND` switches off hoist-all back
|
||||
to direct-suspend-only, useful for narrowing blast radius while debugging.)
|
||||
|
||||
### catch_all-escape: LANDED (2026-06-22)
|
||||
|
||||
**Fixed — all 7 shapes green in Firefox + Chrome + Safari.** When the try body holds a local with a
|
||||
non-trivial destructor, LLVM lowers the C++ catch *nested inside* the cleanup `catch_all`; the pass
|
||||
hoists it PAST the outermost enclosing try. Confirmed to occur in real KiCad (`pcbnew/files.cpp:670`
|
||||
builds `std::map<std::string, UTF8> props` in the try, so its IO/format/bad_alloc catches are
|
||||
catch_all-nested). Landing it took a from-source Binaryen build + a minimal *multi-function* repro
|
||||
(`/tmp/eh_min2.cpp`, `/tmp/eh_min3.cpp`); two bugs, both invisible on a single function and only live
|
||||
once several shapes inline together:
|
||||
|
||||
1. **Over-eager deferral → `null function`.** A nested cpp catch is deferred to its ancestor escape
|
||||
target, but the test matched ANY ancestor catch body — so a cpp catch in a *regular* catch body
|
||||
(the `__cxa_end_catch` cleanup tries LLVM emits everywhere) was deferred to a target that never
|
||||
hoisted it; its suspend was dropped and rewind trapped. Fix: defer only when the catch sits in an
|
||||
ancestor's `catch_all` cleanup pad (`hasCatchAll() && catchBodies.back() == child`).
|
||||
2. **Trailing catch_all code → `unreachable`.** The rewritten minimal arm completes, but the
|
||||
catch_all body has trailing `(unreachable)` after the nested try (it assumed the handler
|
||||
diverged). Fix: wrap the escape target in a `block $esc`; the arm `br $esc`s after capturing,
|
||||
landing fall-through just before the dispatch (so Asyncify still rewinds the handler).
|
||||
|
||||
The pass is in the `binaryen` submodule (`src/passes/HoistCppCatches.cpp`). The earlier
|
||||
"not asyncify-rewindable" worry was wrong — Asyncify rewinds the escape form fine; the blockers were
|
||||
ordinary IR bugs, exactly as the multi-function-debug plan predicted.
|
||||
|
||||
### value-typed (concrete-result) tries: LANDED (2026-06-22)
|
||||
|
||||
The pass also handles an escape target whose try yields a *value* (i32/i64/…), not just
|
||||
void/unreachable — it routes the body/handler value through a `$result` local (the no-exception
|
||||
body value is captured inside `block $esc`; a caught arm br's out and each per-arm dispatch writes
|
||||
`$result`; the block yields `local.get $result`). Non-defaultable result types are still skipped.
|
||||
These tries don't arise from normal C++ EH lowering (LLVM keeps catch values in locals →
|
||||
void/unreachable tries), so they're covered by hand-written modules in
|
||||
`scripts/binaryen-hoist-pass/tests/` (`run.sh`): `--fuzz-exec` confirms the pass preserves the
|
||||
result value across the exception / no-exception / payload paths, and a real asyncify unwind+rewind
|
||||
through a value-typed *suspending* catch yields the correct value (50).
|
||||
|
||||
#### Debugging history (superseded)
|
||||
|
||||
The notes below trace the path to the fix; their "blocked" conclusions are superseded by the
|
||||
landing above.
|
||||
|
||||
##### catch_all-escape: confirmed real; the fix is asyncify-SOUND, not a wall (2026-06-22)
|
||||
|
||||
> **Correction (later same session):** the "not asyncify-rewindable" conclusion below was
|
||||
> **disproven**. Diffing the *asyncified* output of the per-try vs escape forms on a minimal
|
||||
> single-function suspend-in-catch (`/tmp/eh_min.cpp`) shows them **structurally identical** — all
|
||||
> 22 diff hunks are pure local-index renumbering — and **both run cleanly in node**. So Asyncify
|
||||
> rewinds the escape form fine. The real blocker is ordinary structural bugs in the escape pass on
|
||||
> the complex *inlined* toy (one found: the skip-to-escape-target coordination drops a catch when
|
||||
> its escape target is value-typed; fixing that surfaced a load-time trap, so there's ≥1 more).
|
||||
> That is tractable engineering — methodical per-function isolation like the eh_min repro — **not**
|
||||
> an asyncify-internals wall. WIP + partial fix preserved in the escape-wip file below.
|
||||
|
||||
> **Further localization (same session):** the breakage is **not** fiber-specific (cases 1/3/4/5
|
||||
> with no fibers still trap) and **not** one bug. It is a **layout-sensitive structural corruption**
|
||||
> the escape restructure introduces on MULTI-function modules — `null function` / wrong
|
||||
> `call_indirect`, which V8 then mis-compiles unpredictably (the trap point *moves* with module
|
||||
> composition). Single-function repros (`eh_min`) work; the corruption only appears once several
|
||||
> functions/cases compile together. So the next step is NOT more single-function isolation but a
|
||||
> small **multi-function** repro under a Binaryen **debug build (assertions)** + `--fuzz-exec`, to
|
||||
> catch the exact expression the restructure corrupts. Deferred to dedicated debugging.
|
||||
|
||||
**Confirmed we DO hit the gap.** A spot-check of the audited sites found destructible locals in the
|
||||
try bodies: e.g. `pcbnew/files.cpp:670` declares `std::map<std::string, UTF8> props;` in the try,
|
||||
so its three `catch (… ) { DisplayErrorMessage(…) }` arms are lowered nested inside a cleanup
|
||||
`catch_all`. The file-load sites generally construct `wxString`/`std::map`/smart-pointer locals, so
|
||||
this is not academic — a real subset of the ~85 sites is affected.
|
||||
|
||||
**The fix was attempted extensively and is blocked.** The escape-target restructure (hoist the cpp
|
||||
catch — own or nested — past the outermost enclosing try, dispatching handlers after it) **validates**
|
||||
in every variant but is **not asyncify-rewindable**: it traps with `null function` even on the simple
|
||||
cases the per-try form handles. Tried: inline flag-dispatch (`if (flag==n)` — asyncify skips `if`
|
||||
bodies on rewind), a bare single handler, `br_if`-skip guards, and `ReFinalize` (for stale `Try`
|
||||
types). None worked at the time — the actual root causes (over-eager deferral + trailing catch_all code)
|
||||
were found later with a multi-function repro; the fix landed in the submodule pass (see above).
|
||||
|
||||
**Open options:** (1) diff the *asyncified* IR of the working per-try form vs the escape form on one
|
||||
simple case, to pinpoint exactly what Asyncify mis-handles; (2) hand-refactor the affected KiCad
|
||||
sites (move the destructible local out of the try body) — a targeted subset, not all 85; (3) the new
|
||||
`exnref` EH encoding gives `catch_all` a payload (a clean fix) but Asyncify has no `exnref` support.
|
||||
The per-try pass (6/7 shapes) is the shipped state.
|
||||
|
||||
## Opinions (the part you asked for)
|
||||
|
||||
**1. Do it — the size/perf prize is real and the risk is now bounded.** −44% download / −35%
|
||||
module (measured, see 02) plus a large `-O2` build-time drop. The thing everyone feared
|
||||
(Asyncify ⊥ wasm-EH) is disproven. I would not have said this before the spike; I say it now
|
||||
because the toy actually runs.
|
||||
|
||||
**2. Switch the pass from "selective" to "hoist-all-cpp-catches" before KiCad.** My MVP only
|
||||
hoists arms with a *direct* suspending-import call. KiCad's real pattern is **transitive** —
|
||||
`catch (IO_ERROR&) { DisplayErrorMessage(...); }` → `ShowModal` → `startModal` — so direct
|
||||
detection would miss most of the 85 audited sites. The design doc already recommends hoist-all
|
||||
+ let `-O2` prune the no-op hoists, and having now written the selective version I agree: it
|
||||
removes the call-graph analysis entirely, is robust to transitivity, and the only cost is
|
||||
transforming more tries (which `-O2` coalesces). Selective was the right call for *proving the
|
||||
concept with minimal blast radius*; hoist-all is the right call for *shipping*.
|
||||
|
||||
**3. The remaining pass work is small (~1–2 days), and I know exactly what it is.** (a)
|
||||
Concrete-result-typed tries — route the body value through a temp local (the toy already
|
||||
forced me to handle `unreachable`-typed; `i32`/others are the same shape). (b) `catch_all`
|
||||
coexistence on the same try (cpp catch + cleanup pad). (c) hoist-all gating. None are research;
|
||||
all are mechanical Binaryen-IR work. The two real-IR gotchas are already solved in the MVP:
|
||||
**nested-catch pops** (don't clobber a nested catch's payload — fixed with a Try-ancestor guard)
|
||||
and **nested suspend-in-catch tries** (KiCad will have these; the toy already did, and hoisting
|
||||
*both* was required).
|
||||
|
||||
**4. The schedule risk is the emsdk bump, not exceptions.** Everything above avoided the bump.
|
||||
Full KiCad cannot: OpenCASCADE miscompiles a `br_table` under wasm-EH on em 4.0.2 (candidate
|
||||
LLVM fix exists; em 6.0.0 = LLVM 23 should cover it), and the bump changes the compiler for the
|
||||
*JS-EH build too* → whole-project revalidation. That is the multi-week, cross-cutting item.
|
||||
Budget the migration as "1–2 days pass + N weeks emsdk-bump-and-revalidate," not the reverse.
|
||||
|
||||
**5. Keep the legacy encoding; ignore the exnref/Chrome noise.** Asyncify can only instrument
|
||||
legacy `try/catch` (no roadmap to change through v130), and legacy ships unflagged everywhere
|
||||
since 2021. The size win is native-vs-JS EH, independent of the encoding — so legacy is free
|
||||
and correct. The exnref "Chrome problem" never touches us.
|
||||
|
||||
**6. Honest caveat — the toy is small; scale is unproven.** This spike de-risks the *semantic*
|
||||
interaction, not KiCad-scale behavior. Two known scale hazards remain untested under wasm-EH:
|
||||
V8's per-function locals limit on huge asyncified functions
|
||||
([[chrome-asyncify-rewind-crash]]) and unwind-time landing-pad reliability
|
||||
([[asyncify-eh-unwind-landing-pads-unreliable]] — which *might improve* under wasm-EH, worth
|
||||
re-checking). The Safari 26.0 transient legacy-EH crash (emscripten #25365, since patched) is a
|
||||
reminder that even legacy can break on a fresh engine. None of these are blockers; all are
|
||||
"verify at scale," and the recommended order (toy → wx app → full KiCad) is designed to surface
|
||||
them cheaply.
|
||||
|
||||
## Recommended path forward
|
||||
|
||||
1. **Generalize the pass** — DONE: hoist-all, catch_all-escape, and value-typed/concrete-result
|
||||
tries are all handled and verified (the value-typed path via `scripts/binaryen-hoist-pass/tests/`).
|
||||
No further generalization is needed for the 7 KiCad shapes.
|
||||
2. **Phase 2 — a wx standalone app** flipped to native EH: the first *real* `ShowModal`-from-catch
|
||||
path, and the forcing function for transitive hoisting.
|
||||
3. **File the design on binaryen #4470** (the pass is a pure addition; upstreaming collapses our
|
||||
fork back into stock wasm-opt eventually).
|
||||
4. **Phase 3 — full KiCad**, gated on the emsdk bump (the real work) + the generalized pass +
|
||||
the uniform flag flip + dropping `env.invoke_*` from `apply-asyncify.sh`, with an e2e audit
|
||||
under `-sASYNCIFY_ASSERTIONS`.
|
||||
|
||||
## Status of the tracked changes (for review)
|
||||
|
||||
- `binaryen/` submodule added (fork `emergence-engineering/binaryen`), branch **`wasm-port`** at
|
||||
`version_130 + 1` (`58f25ebb2`) — the pass is **committed in the submodule but not pushed**.
|
||||
Pushing the branch to the fork (and committing the parent gitlink) is the user's call.
|
||||
- Parent-repo changes are **uncommitted**, pending review: `.gitmodules` + the `binaryen`
|
||||
gitlink, the spike toy/scripts/spec, and these dossier docs.
|
||||
175
docs/features/wasm-exceptions/08-wx-app-render-rootcause.md
Normal file
175
docs/features/wasm-exceptions/08-wx-app-render-rootcause.md
Normal file
|
|
@ -0,0 +1,175 @@
|
|||
# 08 — Native wasm-EH: the wx application (Phase 2) — render-failure root cause & fix (2026-06-22)
|
||||
|
||||
> Phase 2 of the plan in [`07-spike-results-and-opinion.md`](07-spike-results-and-opinion.md):
|
||||
> flip a real wx standalone app (the `dialog` test) to `-fwasm-exceptions` end-to-end and see if
|
||||
> it runs. It builds and links clean, but rendered **blank**. This documents the deep-debug that
|
||||
> found *why*, the one-line-of-reasoning root cause, the fix, and an **honest account of what is
|
||||
> proven vs. still open** — including a render/screenshot discrepancy that is not yet resolved.
|
||||
|
||||
## Status (read this first)
|
||||
|
||||
- **Proven, C++-level:** the app was **destroying its own main window during startup** under
|
||||
native wasm-EH. Root cause identified with certainty (instrumented build), and the fix makes
|
||||
the destruction **stop** (the `~wxNonOwnedWindow` destructor no longer fires). That specific bug
|
||||
is fixed, and the *why* is understood and re-derivable.
|
||||
- **Proven, in my checks:** after the fix, a headless-Chromium load of `dialog_test.html` showed a
|
||||
full render — `#canvas` present and visible, **5 buttons**, the description text, the event-log
|
||||
control, the status bar; `canvases=1`, `traps=0`; and the screenshot I captured showed the
|
||||
complete dialog UI.
|
||||
- **OPEN / unresolved:** the screenshot is reported **empty** on inspection. My headless ad-hoc
|
||||
check and that observation **disagree**, and I have not reconciled them. **Do not treat the app
|
||||
as "verified rendering" yet.** See [§Open: the empty-screenshot discrepancy](#open-the-empty-screenshot-discrepancy).
|
||||
- **Not yet done:** the real e2e spec in all three browsers; modal dialogs (which now nest Asyncify
|
||||
one level deeper); the rest of the wx suite; cleanup/commit of the Phase-2 changes.
|
||||
|
||||
## The symptom
|
||||
|
||||
`dialog` built and linked under `-fwasm-exceptions -sWASM_LEGACY_EXCEPTIONS=1` (libwx + app, with
|
||||
the post-link hoist+asyncify pipeline). At runtime: boots, prints its startup logs, **no JS error,
|
||||
no wasm trap, main thread responsive** — but `#window-container` empty, no visible canvas, the
|
||||
e2e `waitForApp` (waits for a visible `#canvas`) would time out. A silent non-render.
|
||||
|
||||
## How it was found (the debug chain)
|
||||
|
||||
Each step ruled out a hypothesis and narrowed the next. All via injected logging in the built
|
||||
glue + instrumented libwx rebuilds (the browser symbolizes wasm frames only as `wasm-function[N]`,
|
||||
so callstack mapping was a dead end on a release build — direct source instrumentation was the
|
||||
reliable tool).
|
||||
|
||||
1. **It's not a trap or asyncify/indirect-call corruption.** Calling a wasm export (`ProcessEvents`)
|
||||
from JS post-boot returns cleanly. The "table index out of bounds" seen earlier was an artifact
|
||||
of my own `Module.Asyncify` probe, not the app.
|
||||
2. **`main` "throws `unwind`"** — but that is **normal**: it's `emscripten_exit_with_live_runtime`'s
|
||||
sentinel, caught and swallowed by `handleException` (glue line ~5066). A red herring on its own.
|
||||
3. **The main window is created, then destroyed.** `createWindow(id=-1 → cssId 0)` runs in the frame
|
||||
ctor; then `destroyWindow(0)` runs — the DOM window is torn down. `wxNonOwnedWindow::~wxNonOwnedWindow`
|
||||
is the caller (it `EM_ASM`s `destroyWindow(m_cssId)`). So **the frame's own destructor runs during
|
||||
startup**, leaving the app with no window.
|
||||
4. **The destruction is deliberate, not an exception unwind.** Instrumented `~wxNonOwnedWindow` to log
|
||||
`std::uncaught_exceptions()` → **0**. So no C++ exception is in flight; this is a normal destructor
|
||||
call. (This momentarily looked like it ruled out the landing-pad hazard — it didn't; see root cause.)
|
||||
5. **It happens *after* `OnInit` fully completes.** Logged `OnInit`: "frame created" → "Show done,
|
||||
returning true" both print *before* the destructor. So the teardown is in **`OnRun`**, not OnInit.
|
||||
6. **It happens *inside* `emscripten_set_main_loop`.** Bracketed `wxGUIEventLoop::DoRun`'s
|
||||
`SetSize`/`Refresh`/`set_main_loop` with logs. Order: "before set_main_loop" → **then** the
|
||||
`~wxNonOwnedWindow`. So the frame dies *during* the `emscripten_set_main_loop(ProcessEvents, 0, 1)`
|
||||
call.
|
||||
|
||||
## Root cause (one paragraph)
|
||||
|
||||
`emscripten_set_main_loop(fn, fps, simulate_infinite_loop=1)` implements "loop forever" by **throwing
|
||||
a JS `"unwind"` exception to abandon the C++ stack** — the code after it never runs; the browser drives
|
||||
`fn` thereafter. That `"unwind"` propagates out through every C++ frame between `set_main_loop` and
|
||||
`callMain`. Under **native wasm-EH**, the compiler emits `catch_all` **cleanup** landing pads (for
|
||||
destructors/RAII) that **reliably catch any in-flight exception — including a foreign JS one** — run
|
||||
their cleanup, and rethrow. As the `"unwind"` passes back through `wxEntry`/`OnRun`, those cleanup pads
|
||||
fire and **destroy `wxTopLevelWindows.front()` — the main frame** — before the browser ever calls
|
||||
`ProcessEvents` to paint it. `uncaught_exceptions()==0` is consistent: the `"unwind"` is a *JS*
|
||||
exception, invisible to the C++ exception machinery, so the cleanup-pad destructors see no C++ unwind
|
||||
in progress.
|
||||
|
||||
This is the **inverse** of the documented hazard
|
||||
[`asyncify-eh-unwind-landing-pads-unreliable`]: under legacy `-fexceptions` the cleanup landing pads
|
||||
fire **unreliably**, and that *accidentally* spared the frame (the destroy that should run, didn't).
|
||||
Native wasm-EH makes them reliable — so the latent "abandon-the-stack vs. run-the-cleanup" conflict
|
||||
finally bites. The JS-EH build never rendered-correctly-by-design here; it rendered correctly **by a
|
||||
landing-pad bug canceling a stack-abandon assumption.**
|
||||
|
||||
## The fix
|
||||
|
||||
> **Superseded form (2026-06-23):** the fix described in this section is the *interim* **option A** (`wxWasmRunNestedLoop` / `setTimeout` pump). The final form is the **rAF pump** `wxWasmParkMainLoop` (keeps `requestAnimationFrame`, drops `emscripten_set_main_loop` entirely) — see [`09`](09-event-loop-deparking-plan.md). Both share the root insight (suspend, don't `throw`); the **root cause above is unchanged**. Note the de-park **regresses the coroutine suite** (Asyncify-nesting wall), fixed by [`../async/12`](../async/12-design-b-asyncify-implementation-plan.md) + [`../async/13`](../async/13-design-b-engineering-spec.md).
|
||||
|
||||
Drive the **top-level** event loop via **Asyncify** instead of `set_main_loop`'s
|
||||
abandon-the-stack `"unwind"` — i.e. the **same mechanism the nested/quasi-modal loops already use**
|
||||
(`wxWasmRunNestedLoop`, an `EM_ASYNC_JS` that suspends via Asyncify and pumps `ProcessEvents` from a
|
||||
`setTimeout` loop). Asyncify suspends with a **return-based** unwind that **saves** the stack rather
|
||||
than abandoning it: no `"unwind"` JS exception is thrown, so no `catch_all` cleanup pad fires, so the
|
||||
frame survives. `ProcessEvents` is then driven by the JS `setTimeout(17ms)` pump instead of
|
||||
`requestAnimationFrame`.
|
||||
|
||||
`src/wasm/evtloop.cpp`:
|
||||
- `wxGUIEventLoop::DoRun` — the first (top-level) `DoRun` no longer falls through to
|
||||
`emscripten_set_main_loop(ProcessEvents, 0, 1)`; it does the initial top-window `SetSize`/`Refresh`
|
||||
and then calls `wxWasmRunNestedLoop()`, exactly like a nested loop. Both levels now share one path.
|
||||
- `wxGUIEventLoop::ScheduleExit` — always `wxWasmExitNestedLoop()` (resolve the innermost pump);
|
||||
dropped the top-level `emscripten_cancel_main_loop()` branch (there is no `set_main_loop` to cancel).
|
||||
|
||||
### Why this fix and not the alternatives
|
||||
|
||||
- **`simulate_infinite_loop=0`** (don't throw): then `DoRun` *returns*, `OnRun` returns, and `wxEntry`
|
||||
runs its **normal** teardown (deletes the TLWs) and exits — same dead frame, plus the app exits.
|
||||
Doesn't help.
|
||||
- **Suppress/avoid the cleanup pads:** they're compiler-generated; you can't selectively disable the
|
||||
one that catches `"unwind"`. Not actionable.
|
||||
- **Asyncify the top loop:** it's the existing, tested suspension primitive in this codebase, it
|
||||
*saves* the stack (no abandon → no foreign-exception propagation through cleanup pads), and it
|
||||
unifies top-level and nested loops on one mechanism. This is the minimal, principled change, and it
|
||||
lives in the wasm port layer (`src/wasm/`), per the "fix in the wasm layer" policy.
|
||||
|
||||
## Evidence
|
||||
|
||||
- **Before fix:** `~wxNonOwnedWindow cssId=0 uncaught=0` fires right after "before set_main_loop";
|
||||
`#window-container` empty.
|
||||
- **After fix:** `~wxNonOwnedWindow` **no longer fires** at startup (definitive C++-level signal the
|
||||
frame survives); headless load reports `canvas:true, canvasVisible:true, buttons:5`, body text =
|
||||
"wxDialog and wxMessageBox Test…", `canvases=1`, `traps=0`; rebuilt clean (debug logging removed)
|
||||
and re-checked → same.
|
||||
|
||||
## Open: the empty-screenshot discrepancy
|
||||
|
||||
**My headless-Chromium screenshot showed the full dialog; on inspection the screenshot is reported
|
||||
empty. These disagree and I have not reconciled them.** Until resolved, the app is **not** confirmed
|
||||
rendering. Candidate explanations, to check in order:
|
||||
|
||||
1. **Stale image** — an earlier (pre-fix) empty capture vs. the post-fix one. Cheapest to rule out.
|
||||
2. **Headless vs. headed / real engine** — my check was headless Chromium; a real/headed browser
|
||||
(esp. WebKit/Firefox) may differ. The whole point of project policy is **all three engines**;
|
||||
I only spot-checked one, headless.
|
||||
3. **Ad-hoc load vs. the real e2e spec** — my load waits a fixed 6 s; the spec has its own
|
||||
`waitForApp`/timing and asserts against **tracked baseline screenshots**. The spec is the
|
||||
authoritative render check and I have **not** run it yet.
|
||||
4. **A separate, still-present rendering issue** — the frame-destruction fix is proven, but a
|
||||
*different* paint/canvas problem could remain (e.g. the canvas drawing path, or DOM-widget vs.
|
||||
canvas content). The C++ signal (destructor no longer firing) proves the *frame* lives; it does
|
||||
**not** prove every pixel paints.
|
||||
|
||||
**Immediate next step:** run `tests/.../dialog` through the real e2e spec in **Firefox + Chrome +
|
||||
Safari (WebKit)** and compare to the baseline screenshots — that reconciles the discrepancy and is
|
||||
the real Phase-2 acceptance gate.
|
||||
|
||||
## Build-system decisions made for Phase 2 (for review)
|
||||
|
||||
All gated so the default (JS-EH) build is unchanged; native EH is opt-in via `WX_NATIVE_EH=1`.
|
||||
|
||||
- **`scripts/build-wx-wasm.sh`** — `WX_NATIVE_EH=1` swaps `-fexceptions` for
|
||||
`-fwasm-exceptions -sSUPPORT_LONGJMP=wasm -sWASM_LEGACY_EXCEPTIONS=1` in C/CXXFLAGS. (The whole
|
||||
libwx + every app must share one EH model — EH ABI is all-or-nothing.)
|
||||
- **`tests/apps/Makefile.wasm`** — `WX_NATIVE_EH` adds the same EH flags + `-sDYNCALLS=1` to app
|
||||
CXXFLAGS/LDFLAGS.
|
||||
- **`scripts/build-wasm-test.sh`** — under `WX_NATIVE_EH`, **stubs the emsdk-bundled `wasm-opt`**
|
||||
(v121 crashes asyncifying wasm-EH) so the in-link Asyncify no-ops, then post-link runs the real
|
||||
pipeline on Binaryen v130 over each freshly-linked app + injects the dyncall shims.
|
||||
- **`scripts/common/hoist-and-asyncify.sh`** (new) — the post-link pipeline:
|
||||
`--hoist-cpp-catches` (our fork pass) → `--asyncify` → `-O2`, all on v130. `HOIST_KEEP_NAMES=1`
|
||||
preserves the names section through `-O2` (added for the callstack debugging here).
|
||||
- **`src/wasm/evtloop.cpp`** — the loop fix above (the only behavioral wx-port change).
|
||||
|
||||
## Implications beyond `dialog`
|
||||
|
||||
- **Modals now nest Asyncify two levels deep.** Previously: top = `set_main_loop` (no Asyncify
|
||||
suspend on the main stack), modal = Asyncify (1 level). Now: top = Asyncify, modal = Asyncify
|
||||
(2 levels). This leans harder on the nested-`currData` save/restore in `handlesleep.js`
|
||||
([`asyncify-park-throw-root-cause`]). **Must be tested** (open the Custom/Input dialogs).
|
||||
- **KiCad uses the same `evtloop.cpp`.** If this fix holds for wx apps, it's the same fix KiCad
|
||||
needs under native EH — and it means the `set_main_loop`-`"unwind"` conflict is a **general**
|
||||
wasm-EH×wx-DOM-port interaction, not a `dialog`-specific quirk. This is exactly the kind of
|
||||
"scale hazard" 07 §6 flagged ("unwind-time landing-pad reliability … might *change* under wasm-EH")
|
||||
— it changed, and here's the concrete consequence + remedy.
|
||||
|
||||
## Honest verdict
|
||||
|
||||
The deep-debug **succeeded at the hard part**: a silent blank-render is now a fully understood,
|
||||
evidence-backed root cause with a minimal, principled fix, and the specific bug (frame self-destruct)
|
||||
is provably gone. But Phase 2's acceptance bar — *the app verifiably renders and is interactive in all
|
||||
three browsers via the e2e spec* — is **not met yet**, and the empty-screenshot observation is an
|
||||
unresolved flag against it. Next action is reconciliation via the real spec, not more root-causing.
|
||||
160
docs/features/wasm-exceptions/09-event-loop-deparking-plan.md
Normal file
160
docs/features/wasm-exceptions/09-event-loop-deparking-plan.md
Normal file
|
|
@ -0,0 +1,160 @@
|
|||
# 09 — Event-loop de-parking: one EH-agnostic main loop (plan + verification) (2026-06-23)
|
||||
|
||||
> Plan to replace the wx top-level event loop's stack-abandoning `throw "unwind"` with an
|
||||
> **Asyncify de-park**, written **in C++ (`evtloop.cpp`), not as a post-link shim**, so a **single
|
||||
> code path works under both `-fexceptions` (JS-EH) and `-fwasm-exceptions` (native EH)**.
|
||||
> Companion to [`08-wx-app-render-rootcause.md`](08-wx-app-render-rootcause.md) (root cause) and to
|
||||
> the async dossier's de-park analysis (`docs/features/async/`). Ends with the **old-vs-new
|
||||
> test+screenshot verification matrix**.
|
||||
|
||||
## Decision (verdict up front)
|
||||
|
||||
The top-level `wxGUIEventLoop::DoRun` will stop using `emscripten_set_main_loop(..., simulate_infinite_loop=1)` (which `throw "unwind"`s to abandon the C++ stack). Instead:
|
||||
|
||||
```cpp
|
||||
// top-level DoRun:
|
||||
wxWasmParkMainLoop(); // suspend the C++ stack + drive ProcessEvents from an rAF loop
|
||||
// that calls it via the ASYNC ccall (Asyncify-aware).
|
||||
// NOT emscripten_set_main_loop — see the Correction note below.
|
||||
```
|
||||
|
||||
- **No `throw`** → nothing for native wasm-EH's `catch_all` cleanup pads to catch → the main frame is not destroyed (the 08 bug).
|
||||
- **No `throw`** under JS-EH either → behaves exactly like today minus the (JS-EH-harmless) throw.
|
||||
- **One source, both models** — no `#ifdef WX_NATIVE_EH`, no shim, no `--js-library`. Lives in the wx wasm port where the loop already lives.
|
||||
- **Keeps `requestAnimationFrame`** — `wxWasmParkMainLoop` drives `ProcessEvents` from an rAF loop (not `setTimeout`), vsync-aligned like the original. (Option A used `setTimeout`; this supersedes it.)
|
||||
|
||||
## Background (recap of the 08 bug)
|
||||
|
||||
`emscripten_set_main_loop(ProcessEvents, 0, 1)` registers the rAF loop and then `throw "unwind"` to abandon the C++ stack (so the code after it never runs and the browser drives `ProcessEvents`). Under `-fwasm-exceptions`, the compiler emits real `catch_all` cleanup landing pads; as the foreign `"unwind"` JS exception propagates out of `main`, those pads **catch it and run destructors**, tearing down `wxTopLevelWindows.front()` (the main frame) before first paint → blank render. Under JS-EH the same throw is harmless because legacy `-fexceptions` landing pads fire *unreliably* and `noExitRuntime=true` means the throw is swallowed by `handleException` with no destructors run.
|
||||
|
||||
## The mechanism in detail
|
||||
|
||||
The throw bundles two jobs: (1) register the rAF loop, (2) abandon the stack. `simulate_infinite_loop` is a parameter, so we split them: pass `0` (register + return, no throw), then keep the stack alive ourselves with a **bare park**.
|
||||
|
||||
`wxWasmParkMainLoop` is `wxWasmRunNestedLoop` **minus its `setTimeout` pump** — because here `emscripten_set_main_loop`'s rAF already drives `ProcessEvents`, so a pump would double-drive it. It only `await`s a Promise registered on the existing `Module._wxNestedLoopExit` LIFO:
|
||||
|
||||
```cpp
|
||||
EM_ASYNC_JS(void, wxWasmParkMainLoop, (), {
|
||||
Module._wxNestedLoopExit = Module._wxNestedLoopExit || [];
|
||||
await new Promise(function (resolve) {
|
||||
var finish = function () {
|
||||
var idx = Module._wxNestedLoopExit.indexOf(finish);
|
||||
if (idx !== -1) Module._wxNestedLoopExit.splice(idx, 1);
|
||||
resolve();
|
||||
};
|
||||
Module._wxNestedLoopExit.push(finish);
|
||||
// no pump: emscripten_set_main_loop's rAF drives ProcessEvents
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
`DoRun` becomes:
|
||||
```cpp
|
||||
int wxGUIEventLoop::DoRun() {
|
||||
bool topLevel = (s_wxRunDepth++ == 0);
|
||||
if (topLevel) {
|
||||
// initial sizing as today
|
||||
...SetSize/Refresh on wxTopLevelWindows.front()...
|
||||
emscripten_set_main_loop(ProcessEvents, 0, 0); // rAF, no throw
|
||||
wxWasmParkMainLoop(); // suspend until exit
|
||||
} else {
|
||||
wxWasmRunNestedLoop(); // nested: unchanged (setTimeout pump)
|
||||
}
|
||||
--s_wxRunDepth;
|
||||
return 0;
|
||||
}
|
||||
```
|
||||
|
||||
`ScheduleExit` cancels rAF for the top level before resolving (so no stray rAF tick calls `ProcessEvents` on the app being torn down), then resolves the innermost loop:
|
||||
```cpp
|
||||
void wxGUIEventLoop::ScheduleExit(int) {
|
||||
m_shouldExit = true;
|
||||
if (s_wxRunDepth == 1) emscripten_cancel_main_loop(); // top-level: stop rAF
|
||||
wxWasmExitNestedLoop(); // resolve park (top) or pump (nested)
|
||||
}
|
||||
```
|
||||
|
||||
**Nested/modal loops are unchanged** — they still use `wxWasmRunNestedLoop` (rAF isn't available while nested). Only the top level changes, and only from "abandon-via-throw" to "register-rAF + suspend-via-park."
|
||||
|
||||
## Correction: rAF pump, not `emscripten_set_main_loop`
|
||||
|
||||
The first implementation used `emscripten_set_main_loop(ProcessEvents, 0, 0)` to register rAF + a *bare* park (as "The mechanism in detail" above describes). **It renders white/blank.** `set_main_loop`'s rAF callback (`MainLoop.runIter`) calls `ProcessEvents` **synchronously**, and a synchronous call cannot drive the runtime while `main` is Asyncify-**parked** — the loop stalls after ~6 frames (measured: rafCount **6** vs **~348** for a live loop), so the window never gets its first `Paint` and stays browser-white. (Clicks still work via direct DOM→wx handlers; the first modal's `setTimeout` pump then paints, which masked it in shots 02-05.)
|
||||
|
||||
**Fix — the form now in `evtloop.cpp`:** `wxWasmParkMainLoop` is a hand-rolled `requestAnimationFrame` pump that calls `ProcessEvents` via the **async** `ccall(..., {async:true})` (Asyncify-aware → works on the parked runtime). `DoRun` calls only `wxWasmParkMainLoop()` (no `set_main_loop`); `ScheduleExit` is only `wxWasmExitNestedLoop()` (the pump's `finish()` sets `stopped=true`, stopping it before teardown). It differs from `wxWasmRunNestedLoop` only in rAF vs `setTimeout`.
|
||||
|
||||
```cpp
|
||||
EM_ASYNC_JS(void, wxWasmParkMainLoop, (), {
|
||||
var stopped = false, finish = null;
|
||||
var pump = function () {
|
||||
if (stopped) return;
|
||||
requestAnimationFrame(async function () {
|
||||
if (stopped) return;
|
||||
try { await ccall('ProcessEvents', 'void', [], [], { async: true }); }
|
||||
catch (e) { if (finish) finish(); return; }
|
||||
if (!stopped) pump();
|
||||
});
|
||||
};
|
||||
Module._wxNestedLoopExit = Module._wxNestedLoopExit || [];
|
||||
await new Promise(function (resolve) {
|
||||
finish = function () { stopped = true; /* splice from LIFO */ resolve(); };
|
||||
Module._wxNestedLoopExit.push(finish);
|
||||
pump();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
Confirmed (native-EH): rafCount 348 (continuous), `dialog-01-loaded` **byte-identical** to baseline, all 5 dialog tests pass incl. modals.
|
||||
|
||||
## Relationship to the async dossier
|
||||
|
||||
The agent review (`docs/features/async/`) classifies this precisely: it is the dossier's **"de-parking (Option C — park main in an unresolved `EM_ASYNC_JS` sleep)"** (`02-asyncify-internals.md:265`, `07-decisions-and-outcome.md:60`), the "natural step one of Design B." The dossier **deferred/rejected** de-parking (`07/D4`) because, under `-fexceptions`, the throw provably "runs no destructors" (`10-resolution-menubar-uaf.md:63`, `08-dom-port-regression.md:305-317`) — so there was no reason to take it on. **That premise is exactly what `-fwasm-exceptions` inverts** (catch_all pads now *do* run destructors), and the dossier never considered native EH (grep-confirmed: zero mentions). So we are adopting the dossier's own deferred design, now made *necessary* by the toolchain change — consistent with its long-term direction (Design B), against its near-term decision (D4), for a reason D4 didn't know about.
|
||||
|
||||
## Teardown on exit — correct, not a bug
|
||||
|
||||
The old throw *abandons* the stack, so `wxEntryCleanupReal` (delete app + all TLWs) **never runs** — leaked on exit. The park lets `DoRun` resume on exit and return into that cleanup, which is correct (and frees the leak). No use-after-free: `ScheduleExit` cancels rAF first, and the park's resolver runs before `DoRun` resumes, so nothing calls `ProcessEvents` on freed state. (The dossier's general de-park warning is about the `simulate_infinite_loop=0`-and-*return* form where a still-registered rAF fires on the freed app; our suspend-then-cancel form avoids it.) Residual: a pre-existing wx window-close crash (`async/01:92-94`) could surface only if a real clean exit is triggered — rare in a browser; pre-existing, not introduced here.
|
||||
|
||||
## Remaining caveat (the real one to verify)
|
||||
|
||||
This makes the top-level an **always-live Asyncify-suspended context** for the app's lifetime. The known nesting wall (`async/11-asyncify-nesting-raytracer.md`: `emscripten_sleep` can't nest on an unwinding context) is the thing to watch — at KiCad scale, especially the 3D viewer. Reasoning suggests it's *not* worsened (the park is a dormant, separate saved stack, not in the active modal→sleep chain; ProcessEvents runs fresh from rAF), but that's analysis, not measurement. Since this is now one path for both EH models, the JS-EH build gets the park too, so the scale check covers both.
|
||||
|
||||
## Verification matrix (the old-vs-new test + screenshot proof)
|
||||
|
||||
Goal: prove (a) native EH renders correctly, and (b) JS-EH has **no regression**, by comparing the full wx e2e screenshots against the committed baselines (which were generated from the **old JS-EH** build). `scripts/compare-screenshots.sh` does byte-exact `cmp` of `tests/test-results/` vs `tests/baseline-screenshots/`.
|
||||
|
||||
| # | Config | EH model | `evtloop.cpp` | Purpose | Expected |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 | **OLD** | JS-EH (`-fexceptions`) | original (throw) | reference / baseline-is-current sanity | matches committed baseline |
|
||||
| 2 | **NEW-native** | native (`-fwasm-exceptions`) | de-park (this plan) | migration target | matches baseline |
|
||||
| 3 | **NEW-js** | JS-EH (`-fexceptions`) | de-park (this plan) | no-regression | matches baseline |
|
||||
|
||||
- **Scope:** the full wx app suite (`menu, clipboard, filedialog, layout, aui, toolbar, grid, dialog, timer, tree`), all their `*-NN-*.png` shots.
|
||||
- **Pass bar:** configs 2 and 3 produce screenshots **byte-identical** (or trivially-different, e.g. caret-blink) to the baseline, same set config 1 produces.
|
||||
- **Browsers:** byte-compare is **chromium** (baselines are chromium). Firefox + WebKit are run for **pass/render** confirmation (their pixels won't byte-match a chromium baseline), per the all-three-engines policy.
|
||||
- **Build order (minimize clean rebuilds; only EH-model switches need `--clean`):** start from current (native-EH + interim option-A) → implement de-park → **(2)** native-EH de-park (incremental) → **(3)** JS-EH de-park (clean EH switch) → **(1)** JS-EH original (revert `evtloop.cpp`, incremental).
|
||||
|
||||
### Results (to fill in)
|
||||
|
||||
| # | Config | identical / different / fail | notes |
|
||||
|---|---|---|---|
|
||||
| 1 | OLD JS-EH | suite: **316 pass / 0 fail** / 1 skip | **Baseline is STALE** — OLD JS-EH itself differs 1–8% from the committed baseline on many main-app shots (03-after-load…aui…calendar…clipboard), so byte-compare *vs baseline* is unreliable; use config-vs-config. **0 failures here ⇒ config 2's 21 ARE regressions** from native-EH and/or de-park (not pre-existing). |
|
||||
| 2 | NEW native | suite: 295 pass / 21 fail / 1 skip. dialog standalone 3 identical + 2 caret. Apps render (main-app snapshot OK). Fails: main-app assertions (boot/wxwidgets/dialogs-tab/grid-tab, 10) + coroutine/threading/raytracer (11) | old-baseline (config 1) pending to classify the 21 as pre-existing vs regression |
|
||||
| 3 | NEW JS-EH | suite: **310 pass / 6 fail** / 1 skip — all 6 are **coroutine** (`coroutine`, `coroutine-nested`, `coroutine-pthread`) | de-park renders **identically** to config 1 (config1-vs-config3 byte-diffs = event-log timestamps + caret only; verified pixel-identical on 04-controls-tab) |
|
||||
|
||||
### Isolation (the verdict)
|
||||
|
||||
| failures | config 1 (no de-park) | config 3 (de-park, JS-EH) | config 2 (de-park, native-EH) | attribution |
|
||||
|---|---|---|---|---|
|
||||
| coroutine / coroutine-nested / coroutine-pthread (6) | pass | **FAIL** | FAIL | **the de-park** (top-level Asyncify park × coroutine fibers — the doc-11 nesting wall, now real) |
|
||||
| coroutine-raytrace (5) + main tabbed app: boot/wxwidgets/dialogs-tab/grid-tab (10) | pass | pass | **FAIL** | **native-EH** (migration coverage gaps; the "scale unproven" caveat) |
|
||||
|
||||
**Key conclusions:**
|
||||
1. The de-park is **visually clean** — no rendering change (config1≡config3 modulo timestamps), standalone apps all pass, dialog byte-identical.
|
||||
2. **The de-park regresses the 6 coroutine/threading tests under BOTH EH models.** Under JS-EH this is a **net loss** (the original `throw` form passes them) — so "one de-park for both" is not free: it costs JS-EH its coroutines. This is exactly the Asyncify-nesting hazard `async/11` + the async dossier flagged; the dossier's answer is **Design B** (fiber/arbiter runtime), of which de-park is "step one."
|
||||
3. native-EH independently breaks 15 more (raytracer + the big app) — broader migration work, separate from the loop change.
|
||||
4. The **committed screenshot baseline is stale** (old JS-EH itself is 1–8% off); and byte-compare is unreliable here anyway (event-log timestamps). A perceptual diff + a baseline refresh are needed for a real screenshot gate.
|
||||
|
||||
**Open decision:** (a) gate the de-park to native-EH only (JS-EH keeps `throw` + coroutines; not "one solution", needs `#ifdef`); (b) do Design B so de-park coexists with coroutines (bigger); (c) ship de-park for both now, coroutine-nesting as tracked follow-up.
|
||||
|
||||
## Status
|
||||
|
||||
Plan agreed. Implementing the `evtloop.cpp` de-park, then running the matrix above. The interim `option-A` edit (top-level via `wxWasmRunNestedLoop`'s `setTimeout` pump — loses rAF) is **superseded** by this and will be reverted in favor of the `set_main_loop(0)` + `wxWasmParkMainLoop` form.
|
||||
339
docs/features/wasm-exceptions/10-pthreads-native-eh.md
Normal file
339
docs/features/wasm-exceptions/10-pthreads-native-eh.md
Normal file
|
|
@ -0,0 +1,339 @@
|
|||
# Native wasm-EH × pthreads — findings and test coverage
|
||||
|
||||
> **Status:** native wasm-EH (`-fwasm-exceptions`) is the **default build**; the pthread test suite is
|
||||
> green in Firefox + Chrome. Authored 2026-06-24, updated 2026-06-25. The exception-handling-side
|
||||
> companion to the mechanism-deep [`../threading/README.md`](../threading/README.md) (the 3-layer
|
||||
> thread model, the deadlock mechanics, the three failure modes). **Scope:** what native wasm-EH does
|
||||
> to pthreads, and the test coverage that proves which patterns work.
|
||||
|
||||
## Why this exists
|
||||
|
||||
KiCad-WASM uses native WebAssembly exceptions instead of Emscripten JS exceptions for the bundle-size
|
||||
win (pcbnew ~64.5 → ~36 MB gz). **Native-EH is the default build** — `build-wx-wasm.sh`,
|
||||
`build-wasm-test.sh`, and `tests/apps/Makefile.wasm` compile `-fwasm-exceptions -sSUPPORT_LONGJMP=wasm
|
||||
-sWASM_LEGACY_EXCEPTIONS=1` (single-sourced from `scripts/common/env.sh`). It is the only build mode —
|
||||
the legacy `-fexceptions` path has been removed. CI builds native. The threading question this doc answers: which pthread patterns work under native-EH, what is
|
||||
the one exception-related risk it removes, and what is the (optional) upstreamable follow-up.
|
||||
|
||||
## TL;DR
|
||||
|
||||
- **Native-EH wx suite: green** in Firefox + Chrome (316 / 1 skipped / 0 failed, chromium — matching
|
||||
JS-EH). WebKit is blocked for *all* pthread apps by a separate, pre-existing COEP worker-load
|
||||
limitation (§2a), so the pthread specs run FF + Chrome.
|
||||
- **The one native-EH-relevant risk is mode-c** — a C++ exception thrown on a pthread worker. Under
|
||||
`-fexceptions` the throw drives Asyncify on the worker and crashes (`"func is not a function"`);
|
||||
native-EH lowers exceptions to native wasm instructions, so a throwing worker task is safe. Every
|
||||
threading pattern below is green under native-EH; the throwing ones are green **only** under native-EH.
|
||||
- **The real `BS::thread_pool` runs 16-core under native-EH**, including a task that throws on a worker
|
||||
(`threadpool-real`, §6) — the decisive proof that the `detach_task` single-thread shim can be dropped.
|
||||
- **On-demand (non-warm) Worker creation works without editing KiCad** (`pthread-ondemand`, §6): the
|
||||
`nanosleep` override (§2b) makes a main-thread `sleep_for` join Asyncify-yield, so the event loop
|
||||
services the new-Worker handshake. This is the threading-doc **mode-(a) deadlock** cure, in the wasm
|
||||
layer.
|
||||
- **A nested `emscripten_sleep` is legal** (§3): code dispatched by a wx modal pump's `ProcessEvents`
|
||||
runs at Asyncify `state == Normal`, so a worker-join that yields via `emscripten_sleep` inside an
|
||||
open modal suspends-and-resumes normally (`raytrace-modal`, §6). The threading-doc **mode-(b)** does
|
||||
**not** arise for this case, so no JS-land scheduler ("Design B") is needed for it.
|
||||
- **The KiCad-10 `std::async` library preload is safe under native-EH** (`async-preload`, §6/§7): the
|
||||
worker parses S-expr (a throw = mode-c) and proxies its async fetch to main; native-EH makes the
|
||||
parse safe and the lazy join keeps main free to service the proxy.
|
||||
- **The fork stays pristine.** The pool's `detach_task` shim is the original, unmodified KiCad code;
|
||||
the tests un-shim it via a build-generated header (§6/§D), so the KiCad submodule carries no
|
||||
wasm-specific change. The later, *optional* upstreamable step is the raw-threads→pool refactor (§4).
|
||||
|
||||
---
|
||||
|
||||
## 1. Suite status under native-EH
|
||||
|
||||
The native-EH wx app suite is **316 / 1 skipped / 0 failed** (chromium), matching JS-EH. Reaching it
|
||||
required two things: clearing a set of build-pipeline gaps that surfaced as native-EH test failures
|
||||
(committed this session, summarized below), and the §2 asyncify-imports fix for the raytracer cluster.
|
||||
|
||||
The build-pipeline gaps (all committed):
|
||||
|
||||
- **Post-link Asyncify find too narrow** — the loop matched only `standalone/*/*_test.wasm`, silently
|
||||
skipping `apps/minimal_test.wasm` and the coroutine-pthread repros / wxpt. Those linked but were
|
||||
never asyncify-instrumented → `asyncify_start_unwind not found`. Broadened to all freshly-linked app
|
||||
wasm.
|
||||
- **Repro apps mixed EH models** — the coroutine-pthread `*_repro` apps hardcoded JS-EH in their link
|
||||
recipes while their compile inherited native-EH → `undefined symbol: __cpp_exception`. Made them
|
||||
EH-aware so they follow the default (native) and only carry `-fexceptions` under `WX_LEGACY_EH`.
|
||||
- **`build-wasm-test.sh` swallowed make failures** — it continued to the post-link after a failed
|
||||
make, leaving apps half-instrumented (read as mass test failures). Now aborts loudly.
|
||||
|
||||
After those, the only real native-EH-specific signal was the raytracer threading cluster, fixed in §2.
|
||||
|
||||
---
|
||||
|
||||
## 2. The asyncify-imports fix (`emscripten_sleep`)
|
||||
|
||||
`coroutine-raytrace.spec.ts` aborted with `Aborted(invalid state: 1)`. The mechanism:
|
||||
|
||||
- `invalid state: 1` is `Asyncify.handleSleep` aborting because the state is **Unwinding** — a second
|
||||
suspend starting before the first rewinds. Logging every `handleSleep`, the state sequence at the
|
||||
abort is exactly **`0,1`**: two `emscripten_sleep`s back-to-back with **no rewind between**. So a
|
||||
function calls `emscripten_sleep`, the unwind arms (state→Unwinding), and the **same function calls
|
||||
`emscripten_sleep` again before returning**. A correctly Asyncify-instrumented function has a
|
||||
post-call "if Unwinding, save locals and return" check after every suspend point; this one doesn't →
|
||||
**Asyncify never instrumented it.**
|
||||
- `main()` is **not** re-entered (a `[MAINCALL]` probe fired exactly once; the abort stack only *shows*
|
||||
`main`'s frames because Asyncify's unwind/rewind runs inside a `setTimeout`-driven `doRewind` that
|
||||
keeps the JS stack live).
|
||||
- **Why un-instrumented:** binaryen's Asyncify instruments only functions that can reach a *listed*
|
||||
async import. The post-link list was curated for the wx apps, which yield via **fibers**
|
||||
(`startModal, js_*, invoke_*, __asyncjs__*, emscripten_fiber_swap`). It **omitted `emscripten_sleep`**,
|
||||
which the raytracer yields via. `env.emscripten_sleep` *is* a wasm import, so binaryen can match it —
|
||||
it just wasn't told to.
|
||||
|
||||
### The exact JS-EH ↔ native-EH difference
|
||||
|
||||
Under **JS-EH**, Asyncify runs **in-link** and emcc **auto-adds** `emscripten_sleep` (+
|
||||
`idb_*`/`wget`/`scan_registers`/`lazy_load`) to the imports. Under **native-EH** we run Asyncify
|
||||
**post-link by hand**, with an explicit list that dropped those auto-imports. That is the entire
|
||||
difference — not a fundamental native-EH × pthread incompatibility, and not a handleSleep-vs-arbiter
|
||||
question (the `currData` shim was never involved).
|
||||
|
||||
### The fix
|
||||
|
||||
Added `env.emscripten_sleep` (+ `scan_registers`, `lazy_load_code`, `wget`, `wget_data`, `idb_*`) to
|
||||
the post-link asyncify-imports — now the shared **`scripts/common/asyncify-imports.txt`**, consumed by
|
||||
the unified **`apply-asyncify.sh`** that both the wx-test and KiCad builds call (the two near-duplicate
|
||||
scripts were folded into one; the old `hoist-and-asyncify.sh` is gone). So the KiCad list gets
|
||||
`emscripten_sleep` too, pre-empting the identical latent bug when its threading is un-shimmed.
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Full wx suite, Chromium | **316 / 1 skipped / 0 failed** |
|
||||
| `coroutine-raytrace.spec.ts` — all 6 (B1/B2/B1-local/B3 + speedup + A neg-control) | **6/6 pass** |
|
||||
| multi-core speedup | **serial 1342 ms → parallel 142 ms = 9.45× on 16 cores** |
|
||||
| raytrace `#m=5` default (drains pool → on-demand creation) / `#m=1`, Chromium + Firefox | **SUCCESS, workersRan=16** |
|
||||
|
||||
The default `m=5` — which *drains* the pre-warmed pool and forces on-demand Worker creation —
|
||||
succeeds, so the fix also resolves the threading-doc **mode-(a) deadlock**: the `sleep_for` join now
|
||||
yields via an instrumented `emscripten_sleep` instead of busy-spinning and starving the worker
|
||||
handshake.
|
||||
|
||||
### 2a. The WebKit issue (separate, pre-existing)
|
||||
|
||||
In WebKit the asyncify side runs (threads spawn) but the **pthread worker `.js` load is refused on
|
||||
COEP** (`Refused to load worker because of Cross-Origin-Embedder-Policy`) even with COOP + COEP + CORP
|
||||
all served and `crossOriginIsolated:true`. It is a WebKit/playwright-headless COEP-worker strictness
|
||||
issue affecting **all** pthread apps, unrelated to EH. Tracked separately; the pthread specs run
|
||||
FF + Chrome only.
|
||||
|
||||
### 2b. The `nanosleep` override (the on-demand cure)
|
||||
|
||||
`wasm/shims/nanosleep_yield.c` is a **strong `nanosleep` definition** that shadows musl's archive
|
||||
member (`-Wl,--wrap=nanosleep` is not usable — it segfaults wasm-ld in
|
||||
`lld::wasm::ImportSection::addImport`). On the **main thread** it yields via an `EM_ASYNC_JS` await
|
||||
(= `emscripten_sleep` semantics, already in the post-link asyncify-imports); on a **worker** it stays a
|
||||
real blocking `emscripten_thread_sleep`. So an *unmodified* KiCad `sleep_for` join on the main thread
|
||||
pumps the event loop instead of busy-spinning, which lets on-demand Worker creation complete with no
|
||||
KiCad edit (§6 `pthread-ondemand`).
|
||||
|
||||
---
|
||||
|
||||
## 3. Patterns that work — pool-vs-raw, and the nested-sleep case
|
||||
|
||||
Two earlier wx apps, plus the §6 additions:
|
||||
|
||||
| App / test | Thread pattern | native-EH |
|
||||
|---|---|---|
|
||||
| `threadpool_test.cpp` (`threadpool.spec.ts`) | create `hwc` `std::thread`s into the **pre-warmed** pool, short body, **`join()`** each | **PASS** |
|
||||
| `raytrace_threads_test.cpp` (`coroutine-raytrace.spec.ts`) | raw detached/persistent `std::thread`, sleep/busy-wait join, default **drains** the pool → on-demand creation | **PASS** (after §2) |
|
||||
|
||||
Both raw and pool patterns work under native-EH; raw threads are **not** fundamentally broken. The
|
||||
`threadpool` create-and-`join()` never calls `emscripten_sleep`, so it never tripped the missing
|
||||
import; the raytracer yields via `emscripten_sleep`, so it did — which §2 closed.
|
||||
|
||||
**The nested-sleep case (mode-b is not a live blocker).** A worker-join that yields via
|
||||
`emscripten_sleep` *inside an open `ShowModal` dialog* is legal. The modal pump runs `ProcessEvents`
|
||||
via `ccall(async:true)`, so work it dispatches runs in a **fresh managed Asyncify entry at
|
||||
`state == Normal`** — not nested inside an already-Unwinding frame. So the inner `emscripten_sleep`
|
||||
suspends-and-resumes normally. `raytrace-modal` (§6) probes and logs `Asyncify.state == 0` to confirm
|
||||
this; the threading-doc mode-(b) "Asyncify can't nest" only bites a *genuine* second unwind, which the
|
||||
modal pump does not produce. **No JS-land scheduler ("Design B") is required for the "render inside a
|
||||
modal" case.**
|
||||
|
||||
---
|
||||
|
||||
## 4. Optional follow-up (upstreamable): KiCad raw-threads → the pool
|
||||
|
||||
This is a *later, optional* cleanup — not required, since the wasm layer (native-EH + the nanosleep
|
||||
override) already makes the threading patterns work on pristine KiCad. Per
|
||||
[`../threading` §8](../threading/README.md), upstream KiCad has migrated only **1 of 7** raytracer
|
||||
parallel sections to `GetKiCadThreadPool()` (`renderTracing`, and that one accidentally); the other six
|
||||
are untouched **2018 OpenMP-translation** raw-thread code:
|
||||
|
||||
| Site | Pass | Today |
|
||||
|---|---|---|
|
||||
| `render_3d_raytrace_base.cpp:764` `shadeWorker` | post-process shading | raw `std::thread` + busy-wait, `#ifdef`'d serial in WASM |
|
||||
| `render_3d_raytrace_base.cpp:835` `blurWorker` | blur/finish | same |
|
||||
| `render_3d_raytrace_base.cpp:1456` `previewWorker` | preview | same |
|
||||
| `image.cpp:525` `filterWorker` | `EfxFilter` AA/blur | same |
|
||||
| `create_layer_items.cpp:848` `zoneWorker` | zone-fill geometry | same |
|
||||
| `create_layer_items.cpp:1311` `simplifyWorker` | polygon simplify | same |
|
||||
|
||||
**The refactor = migrate these six to `submit_task()` + `multi_future::wait()`** (the `renderTracing`
|
||||
shape, refined by upstream `bccf36538` to wait on *own* tasks only), and delete the
|
||||
`#ifdef __EMSCRIPTEN__` serial fallbacks. Why it is the right *eventual* move:
|
||||
|
||||
- **Upstreamable, not a wasm hack** — precedent in the same file, a filed upstream issue
|
||||
([GitLab #20911](https://gitlab.com/kicad/code/kicad/-/issues/20911), "ray tracing high system
|
||||
load"), and it removes dead OpenMP-era code. If accepted upstream, our fork carries **zero**
|
||||
divergence here.
|
||||
- **Less divergence, not more** — it lets us drop the raytracer `#ifdef`s; combined with native-EH
|
||||
letting us drop the `detach_task` shim, net fork divergence goes *down* while threads come *on*.
|
||||
|
||||
Its prerequisite — that the real `BS::thread_pool` (persistent workers + `submit_task` +
|
||||
`multi_future::wait()`) survives native-EH — **is proven** by §6 `threadpool-real` (16-core, including
|
||||
a throwing worker task). So the refactor is de-risked; it is scheduled **after** the EH port's suite is
|
||||
otherwise green, and remains optional because the wasm-layer fixes already deliver multi-core.
|
||||
|
||||
---
|
||||
|
||||
## 5. The 3D viewer
|
||||
|
||||
The 3D viewer is **live and single-threaded**: the raytracer's six raw-thread passes are `#ifdef`'d to
|
||||
serial fallbacks in WASM, which is what ships. A separate multi-threaded spike exists (the
|
||||
`WASM_RAYTRACE_POOL` work, ~6–7×) but is not the active path.
|
||||
|
||||
Two zero-KiCad-edit routes turn the live viewer multi-threaded:
|
||||
|
||||
- **The nanosleep override (§2b)** makes the existing `sleep_for` joins yield, so the raw-thread passes
|
||||
run multi-core without on-demand-creation deadlock and without main-thread jank — no KiCad change.
|
||||
- **The §4 pool refactor** is the *upstream-clean* alternative: pool-based, drops the `#ifdef`s, and
|
||||
carries zero fork divergence if accepted upstream.
|
||||
|
||||
---
|
||||
|
||||
## 6. pthread test coverage
|
||||
|
||||
All four apps below compile the **real KiCad** thread-pool source and run on **pristine** KiCad/wx-core.
|
||||
The specs are named `coroutine-*` so `playwright-coroutine.config.ts` runs them in Firefox + Chrome
|
||||
(WebKit excluded — §2a).
|
||||
|
||||
| Spec | App | What it proves | native-EH |
|
||||
|---|---|---|---|
|
||||
| `coroutine-threadpool-real.spec.ts` | `threadpool-real` | the **real `GetKiCadThreadPool()`** in every mode — submit / loop / blocks / detach / fanout / lifecycle, and a task that **throws** on a worker — 16-core, throw caught | **PASS** (throw mode green *only* under native-EH = mode-c) |
|
||||
| `coroutine-pthread-ondemand.spec.ts` | `pthread-ondemand` | real pool drains the pre-warmed Workers, then raw fly-threads force **on-demand** creation; the nanosleep override yields the join → on-demand Workers boot → multi-core (control: a non-yielding busy-wait deadlocks) | **PASS** |
|
||||
| `coroutine-raytrace-modal.spec.ts` | `raytrace-modal` | a worker-join run **inside an open `ShowModal`** — both a busy-wait join and an `emscripten_sleep` yield-join complete multi-core; the app probes `Asyncify.state == 0` to show the modal pump dispatches at Normal | **PASS** (mode-b does not arise) |
|
||||
| `coroutine-async-preload.spec.ts` | `async-preload` | the KiCad-10 `std::async` library-preload shape: a worker parses S-expr (throws = mode-c) and proxies its fetch to main via `emscripten_proxy_sync_with_ctx`; modes simple / throw / shutdown / modal-during-preload | **PASS** (mode-c safe; 36 proxy round-trips through a modal, no crash) |
|
||||
|
||||
These also cover the older `coroutine-pthread.spec.ts` (fiber + pthread across activation paths) and
|
||||
`threadpool.spec.ts` (raw create+join), both green. Together they exercise: the real pool API, raw
|
||||
create+join, raw detached/persistent + sleep/busy-wait, on-demand creation, a worker-side throw, a
|
||||
nested yield inside a modal, and a proxied async fetch off a worker — the full set of shapes the KiCad
|
||||
threading uses.
|
||||
|
||||
### D. How the tests un-shim the pool without editing KiCad
|
||||
|
||||
KiCad's `bs_thread_pool.hpp` keeps its original `#ifdef __EMSCRIPTEN__` `detach_task` shim (which runs
|
||||
pool tasks inline → single-threaded). To exercise the *real* pool, the test build **generates** an
|
||||
un-shimmed copy: `tests/apps/Makefile.wasm`'s `POOL_UNSHIMMED` rule `sed`s `#ifdef __EMSCRIPTEN__` →
|
||||
`#if 0` into `standalone/_pool_unshimmed/bs_thread_pool.hpp` (gitignored) and `-I`'s it ahead of the
|
||||
KiCad header. So the KiCad submodule stays pristine; only the test compile sees the un-shimmed pool.
|
||||
|
||||
---
|
||||
|
||||
## 7. Library preload: `std::async` + the PCBJAM proxy under native-EH (the KiCad-10 bump)
|
||||
|
||||
> The path that turns the native-EH migration from a size win into a **prerequisite for tracking
|
||||
> upstream**. Verified by `async-preload` (§6): works under native-EH.
|
||||
|
||||
**What changed upstream.** KiCad 10 (`d8ae50a667`, 2026-06-08, fixes GitLab #23872) (a) added an
|
||||
*eager* library preload on board open — `if( Kiface().IsSingle() ) Kiface().PreloadLibraries()` in
|
||||
`pcbnew/files.cpp` (`OpenProjectFiles`), and `IsSingle()` is exactly our standalone-webapp case; and
|
||||
(b) changed `IFACE::PreloadLibraries`'s dispatch from `tp.submit_task( preload )` (our base,
|
||||
`pcbnew/pcbnew.cpp:666`) to `std::async( std::launch::async, preload )` (KiCad-10 `pcbnew.cpp` ~1121).
|
||||
**`std::async` spawns a real pthread worker that the `detach_task` pool shim does not cover** — the
|
||||
shim only neutralizes the *pool*.
|
||||
|
||||
**The plugin gotcha (don't be fooled by the upstream loader).** Upstream library reads are synchronous
|
||||
(`KICAD_SEXPR` plugin → `fopen`/`FILE_LINE_READER`). **Our fork is not on that path.** The webapp
|
||||
writes the lib-table rows as `(type "PCBJAM")` / `(type "PCBJAM_FP")`
|
||||
(`web/standalone/src/wasm/libs/source.ts:124,143`), so the runtime plugin is our **custom async bridge**
|
||||
(`kicad/eeschema/sch_io/pcbjam_lib/sch_io_pcbjam_lib.cpp`, `kicad/pcbnew/pcb_io/pcbjam_fp/pcb_io_pcbjam_fp.cpp`).
|
||||
A surface read of the upstream loader will wrongly conclude "pure sync, safe" — **verify by the
|
||||
lib-table row `type`, not the generic plugin.** The PCBJAM dispatch is dual-path
|
||||
(`sch_io_pcbjam_lib.cpp:158`):
|
||||
|
||||
```cpp
|
||||
if( emscripten_is_main_runtime_thread() )
|
||||
return pcbjam_libs_request_js(...); // main: EM_ASYNC_JS → Asyncify suspend (works on main)
|
||||
std::lock_guard lk( g_pcbjamProxyMutex ); // worker: serialize, then
|
||||
emscripten_proxy_sync_with_ctx( queue, main, … ); // proxy the fetch to MAIN + futex-block the worker
|
||||
```
|
||||
|
||||
**The two-level architecture (and how the pool shim warps it).**
|
||||
- **Outer:** `std::async(preload)` = one real background worker running a watchdog loop
|
||||
(`sleep_for(150ms)` + poll `AsyncLoadProgress()`). Bypasses the shim.
|
||||
- **Inner:** `adapter->AsyncLoad()` (`FOOTPRINT_LIBRARY_ADAPTER`) `submit_task`s N enumerate jobs to
|
||||
the pool → caught by the shim → run inline → so in our fork they execute *serially on the outer
|
||||
worker*. (Inner parallelism returns once the shim is dropped — also native-EH-gated.)
|
||||
|
||||
**What runs where, on the `std::async` worker:**
|
||||
|
||||
| Step | Suspends Asyncify on the worker? |
|
||||
|---|---|
|
||||
| `sleep_for(150ms)` watchdog | **No** — real worker sleep (`nanosleep`/Atomics.wait), not `emscripten_sleep`. |
|
||||
| PCBJAM fetch of library bytes | **No** — proxied to main + futex-block; the `EM_ASYNC_JS` runs on *main*. |
|
||||
| S-expr **parse** of the bytes (throws `IO_ERROR`) | **Yes under `-fexceptions`** → mode-c crash. **No under native-EH.** |
|
||||
| modals / clipboard / fonts | Not reachable from non-UI parsing. |
|
||||
|
||||
**The join is lazy — which defuses the deadlock.** There is **no eager `.get()`**: `CancelPreload(true)`
|
||||
calls `m_libraryPreloadReturn.wait()` but has **no callers**; `ProjectChanged()` only sets the abort
|
||||
flag; the `std::async` future's **blocking destructor** fires only on **IFACE teardown** (shutdown,
|
||||
main thread); and re-entry is guarded by `m_libraryPreloadInProgress` (so the future is never
|
||||
*reassigned* mid-flight). So in normal operation **main never blocks on the preload future** → it stays
|
||||
in its event loop → it services the PCBJAM proxy queue → the worker's fetches complete. No
|
||||
normal-operation deadlock.
|
||||
|
||||
**Verified.** `async-preload` (§6) runs this shape under native-EH: the worker parse throws and is
|
||||
caught (no mode-c crash), the proxy round-trips, and a modal opened during preload survives 36 proxy
|
||||
round-trips with no crash (the `g_pcbjamProxyMutex` / "table index out of bounds" reentrancy hazard
|
||||
does not fire). So the **KiCad-10 bump can keep `std::async` as-is under native-EH** — it does **not**
|
||||
need a fork patch reverting to `tp.submit_task` (which would make preload block board-open on the main
|
||||
thread). Residual: a real-shutdown ordering check (the blocking destructor while a load is in flight)
|
||||
is covered by the `async-preload` shutdown mode but not yet under a live IFACE teardown.
|
||||
|
||||
**Contrast with the raytracer (§4):** the raytracer's raw threads are legacy OpenMP-era and
|
||||
upstreamable to the pool. This `std::async` is a **deliberate** upstream choice (a dedicated preload
|
||||
thread, off the compute pool), so "upstream it to the pool" is **not** the play — native-EH is.
|
||||
|
||||
---
|
||||
|
||||
## 8. Next steps (ordered)
|
||||
|
||||
1. **DONE — `coroutine-raytrace` root-caused and fixed (§2):** the post-link asyncify-imports list
|
||||
omitted `emscripten_sleep`. Suite 316/0; raytracer multi-core (9.45×).
|
||||
2. **DONE — pthread coverage closed (§6):** the real-pool, on-demand, modal-nested, and `std::async`
|
||||
library-preload shapes are all green under native-EH on pristine KiCad/wx-core.
|
||||
3. **Drop the `detach_task` shim for real** — `threadpool-real` proves the pool survives native-EH, so
|
||||
the next concrete step is enabling the un-shimmed pool in a KiCad build (DRC / zone-fill /
|
||||
connectivity on real Workers) and validating the docker build (the shared asyncify-imports change is
|
||||
untested there).
|
||||
4. **Resolve the §2a WebKit COEP worker-load limitation** for pthread apps (currently the reason the
|
||||
pthread specs skip WebKit).
|
||||
5. **Optional, later — the §4 refactor:** migrate the six raw-thread raytracer sections to the pool,
|
||||
delete the `#ifdef`s, upstream it.
|
||||
6. **Track-only:** `PROXY_TO_PTHREAD` (DOM-bound GUI can't leave the main thread) and JSPI
|
||||
(incompatible with our main-loop architecture) — see [`../threading` §6–7](../threading/README.md).
|
||||
|
||||
## Cross-references
|
||||
|
||||
- [`../threading/README.md`](../threading/README.md) — the 3-layer model, deadlock mechanics, three
|
||||
failure modes, the full raw-thread inventory, and the upstream pool-migration analysis.
|
||||
- [`../async/11-asyncify-nesting-raytracer.md`](../async/11-asyncify-nesting-raytracer.md),
|
||||
[`../async/12`](../async/12-design-b-asyncify-implementation-plan.md),
|
||||
[`../async/13`](../async/13-design-b-engineering-spec.md) — Asyncify nesting + the Design B scheduler
|
||||
(not required for the modal-pump case, §3).
|
||||
- Apps + specs: `tests/apps/standalone/{threadpool-real,pthread-ondemand,raytrace-modal,async-preload,coroutine-pthread,threadpool,raytrace-threads}/`,
|
||||
`tests/e2e/coroutine-{threadpool-real,pthread-ondemand,raytrace-modal,async-preload,pthread,raytrace}.spec.ts`,
|
||||
`tests/e2e/threadpool.spec.ts`; the pool un-shim in `tests/apps/Makefile.wasm` (`POOL_UNSHIMMED`),
|
||||
the on-demand cure in `wasm/shims/nanosleep_yield.c`.
|
||||
- **Library preload (§7):** `kicad/pcbnew/pcbnew.cpp:593` (`PreloadLibraries`),
|
||||
`kicad/eeschema/sch_io/pcbjam_lib/sch_io_pcbjam_lib.cpp` +
|
||||
`kicad/pcbnew/pcb_io/pcbjam_fp/pcb_io_pcbjam_fp.cpp` (the async PCBJAM IO plugins),
|
||||
`web/standalone/src/wasm/libs/source.ts` (lib-table rows typed `PCBJAM`/`PCBJAM_FP`); upstream
|
||||
KiCad-10 `std::async` change `d8ae50a667` (GitLab #23872).
|
||||
438
docs/features/wasm-exceptions/11-collab-apply-native-eh.md
Normal file
438
docs/features/wasm-exceptions/11-collab-apply-native-eh.md
Normal file
|
|
@ -0,0 +1,438 @@
|
|||
# Collaborative editing under native wasm-EH — the virtual-call mis-dispatch: root cause & fix
|
||||
|
||||
> **Finalization note:** the `futex_yield.c` shim and the `vcall_*` / `pool-callafter` investigation
|
||||
> repros referenced below were **removed** during feature finalization — native-EH needs none of them.
|
||||
> This doc is retained as the root-cause record.
|
||||
|
||||
> **Status: RESOLVED 2026-06-28.** Fixed with a one-line build flag.
|
||||
> **One-line:** native-EH pcbnew's collab **apply** hung at virtual method calls because the **embind
|
||||
> translation unit was compiled without `-DDEBUG` while the core TU had it**. A `#if defined(DEBUG)`
|
||||
> virtual (`EDA_ITEM::Show`) takes a vtable slot, so the two TUs' vtable layouts differed by one slot;
|
||||
> every embind virtual call past that slot read the wrong slot and `call_indirect`-trapped on a
|
||||
> signature mismatch — swallowed by the apply coroutine's `catch_all` → silent loop = "hang." **Fix:**
|
||||
> define `DEBUG` for the embind TU in Debug builds (`scripts/kicad/build-kicad-target.sh`). No
|
||||
> devirtualization; the A/B decision in Part 5 is moot. See [Resolution](#resolution--root-cause--fix).
|
||||
>
|
||||
> *Parts 1–5 below are the investigation as it unfolded; it concluded the `vii` correlation was a
|
||||
> "confound" and weighed an A/B decision. That was right that the dispatch **mechanism** wasn't broken
|
||||
> — but it stopped one step short of the dispatch **input**: the vtable **slot offset** the embind
|
||||
> computed was wrong. A runtime vtable probe + a named-binary offset check (the user's "check the
|
||||
> offsets, find the wrong one") closed it. Kept as the reasoning trail; the Resolution is the answer.*
|
||||
>
|
||||
> Companion to [`01-background-two-eh-models.md`](01-background-two-eh-models.md) (EH models),
|
||||
> [`10-pthreads-native-eh.md`](10-pthreads-native-eh.md) (pthreads), and the `currData` dossier in
|
||||
> [`docs/features/async/`](../async/).
|
||||
|
||||
---
|
||||
|
||||
## Resolution — root cause & fix
|
||||
|
||||
**Root cause (verified 2026-06-28).** The embind TU (`wasm/bindings/pcbnew_embind.cpp`, compiled
|
||||
*outside* CMake in `build-kicad-target.sh` step 7) and the core/vtable-emitting TU disagreed on the
|
||||
`PCB_TRACK` vtable layout by exactly one slot:
|
||||
|
||||
- `EDA_ITEM::Show(int, std::ostream&)` is declared `#if defined(DEBUG)` (`kicad/include/eda_item.h:471`).
|
||||
- The **core** is built Debug → CMake `add_compile_definitions($<$<CONFIG:Debug>:DEBUG>)`
|
||||
(`kicad/CMakeLists.txt:351`) defines `DEBUG` → `Show` occupies vtable **slot 35** → `PCB_TRACK::SetWidth`
|
||||
lands at **byte offset 320**.
|
||||
- The **embind TU** was compiled **without `-DDEBUG`** → no `Show` slot → it computed `SetWidth` at
|
||||
**offset 316**, which in the emitted vtable is `BOARD_CONNECTED_ITEM::GetEffectiveNetClass()` (wasm
|
||||
type `ii`). Dispatching it as `vii` = a **`call_indirect` signature-mismatch trap**, swallowed by the
|
||||
apply COROUTINE's `catch_all` → silent retry loop = the observed "hang."
|
||||
- Asyncify state at the park was **Normal** (confirmed via the real `asyncify_get_state()` export) — a
|
||||
trap, never a suspend. Every embind virtual call past slot 35 (`SetWidth`/`GetPosition`/the rebaseline
|
||||
snapshot getters) mis-dispatched; `Type`/`GetClass` (slots < 35) and core-TU calls
|
||||
(`commit.Modify`'s `Clone`) worked — which is what made it *look* signature-specific (`vii` fails,
|
||||
`ii` works).
|
||||
|
||||
**Fix.** `scripts/kicad/build-kicad-target.sh`: `EMBIND_CONFIG_DEFINES="-DDEBUG"` in the Debug branch
|
||||
(empty in Release — Release defines no `DEBUG` in either TU, so the layouts already match), added to the
|
||||
embind `em++` compile. Both TUs now agree on the layout; **all** embind virtual calls dispatch
|
||||
correctly. The per-site devirtualizations tried during the hunt were reverted (unnecessary). *Hygiene
|
||||
follow-up (recommended, not yet applied):* also give the embind TU `-DKICAD_USE_PLATFORM_WASM=1` and the
|
||||
`-include char_traits_uint16_workaround.h` force-include, so its preprocessor/ABI environment matches
|
||||
the core's exactly and this class of skew can't recur.
|
||||
|
||||
**How it was found.** A runtime probe (a fresh `PCB_TRACK` constructed at the park) showed
|
||||
`tr.vtbl == fresh.vtbl` — the vtable *pointer* was correct — yet the fresh object hung identically,
|
||||
ruling out a dead/wrong instance and pointing at the *slot offset*. `wasm-dis` of the named debug
|
||||
binary then showed offset 316 holds `GetEffectiveNetClass`, not `SetWidth`. Exactly the user's
|
||||
instruction: *"check the offsets of all instances, find the wrong one."*
|
||||
|
||||
**Independently-real fix kept** (not caused by the skew): the **COROUTINE** in `kicadCollabApply`
|
||||
(the `commit.Modify`→`Clone` trap needs the fiber) and `-Xclang -fno-pch-timestamp`.
|
||||
|
||||
**`wasm/shims/futex_yield.c` — NOT needed; kept-but-not-compiled.** It was added during the hunt under
|
||||
the (mistaken) theory that the apply hung on the thread-pool futex; the real cause was the vtable skew.
|
||||
A **no-futex build passes collab 8/8** (Firefox + Chromium) — the connectivity recompute is bounded by
|
||||
the pre-warmed pthread pool (`PTHREAD_POOL_SIZE = hardwareConcurrency`), so it never needs an on-demand
|
||||
Worker, so there's no main-thread futex deadlock to fix here. The shim file + the `pool-callafter` repro
|
||||
are kept as a *documented, validated* fix for the on-demand-Worker futex deadlock **if it ever surfaces**
|
||||
(heavy board / cold pool hanging at `commit.Push`'s `RecalculateRatsnest`); the re-enable steps live in
|
||||
`scripts/kicad/build-kicad-target.sh` (the "AVAILABLE BUT NOT COMPILED" block) and the shim's header.
|
||||
|
||||
---
|
||||
|
||||
## 0. How to read this document
|
||||
|
||||
This is written to be understood without prior knowledge of WebAssembly internals, C++ dynamic
|
||||
dispatch, Asyncify, fibers, or futexes. **Part 1** explains every concept from scratch. **Part 2**
|
||||
walks the actual call chains. **Part 3** lists the four bugs we found and the fixes that work.
|
||||
**Part 4** is the investigation that proved the headline bug is a confound. **Part 5** is the A/B
|
||||
decision and the concrete effort estimate for A. Skim the TL;DR, then dive into whatever you want.
|
||||
|
||||
### TL;DR
|
||||
|
||||
- KiCad's collaborative editing broadcasts each local edit to peers; a peer **applies** the change
|
||||
by running the same `BOARD_COMMIT` machinery a native edit uses.
|
||||
- Under **native wasm-EH** (our target), that apply **hangs** at a C++ **virtual** method call
|
||||
(`SetWidth`, `GetPosition`, …). The same calls work fine under the old **legacy JS-EH** build.
|
||||
- We found and fixed three real sub-bugs (a dispatch *trap*, a thread-pool *futex deadlock*, a fiber
|
||||
*finalization* hang). The fourth — the virtual-call hang — looked signature-specific (calls whose
|
||||
wasm signature is `vii` hang; `ii`/`viii` don't), but **five controlled repros prove `vii` dispatch
|
||||
is not actually broken**. It only hangs inside the real, huge module.
|
||||
- **Devirtualizing** the call (telling the compiler the exact function so it emits a direct call
|
||||
instead of an indirect one) makes the hang vanish at that site — but that's treating a symptom, and
|
||||
it has to be repeated at ~10–15 call sites and re-done whenever new code adds a by-value getter.
|
||||
|
||||
---
|
||||
|
||||
# Part 1 — The concepts
|
||||
|
||||
## 1.1 Two ways C++ exceptions become WebAssembly
|
||||
|
||||
WebAssembly can't just "throw" like native code. Emscripten offers two lowerings:
|
||||
|
||||
- **Legacy JS exceptions (`-fexceptions`)** — every call that might throw is wrapped in a JavaScript
|
||||
helper called `invoke_<sig>` that does a JS `try/catch` around a `dynCall_<sig>` into wasm. The
|
||||
control flow for exceptions detours *through JavaScript*. Big and slow, but battle-tested.
|
||||
- **Native wasm exceptions (`-fwasm-exceptions`)** — uses the WebAssembly exception-handling
|
||||
instructions (`try`/`catch`/`throw`) directly in wasm. No `invoke_*`, no JS detour. Smaller and
|
||||
faster — this is what we're migrating to. (See [`01-background-two-eh-models.md`](01-background-two-eh-models.md).)
|
||||
|
||||
This distinction matters later: under native-EH there are **no `invoke_*`/`dynCall_*` wrappers around
|
||||
ordinary calls**, so the JS-side "self-heal" tricks that exist for legacy-EH don't apply.
|
||||
|
||||
## 1.2 Virtual method calls, vtables, and `call_indirect`
|
||||
|
||||
When you write `item->GetPosition()` and `GetPosition()` is declared `virtual`, the compiler does
|
||||
**not** know which function to run. A `PCB_TRACK` returns its start point; a `PCB_VIA` returns its
|
||||
centre; a `FOOTPRINT` returns its origin. The decision is made **at runtime** based on the object's
|
||||
real type. This is **dynamic dispatch**, and it works via a **vtable**:
|
||||
|
||||
```
|
||||
A PCB_TRACK object in memory The PCB_TRACK vtable (one per class)
|
||||
┌───────────────────────────┐ ┌────────────────────────────────────┐
|
||||
│ vptr ───────────────────────────► │ slot 0: &PCB_TRACK::Type │
|
||||
│ m_Start = (10, 20) │ │ slot 1: &PCB_TRACK::GetPosition │
|
||||
│ m_End = (50, 20) │ │ slot 2: &PCB_TRACK::SetWidth │
|
||||
│ m_width = 200000 │ │ ... │
|
||||
└───────────────────────────┘ └────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Every object of a polymorphic class starts with a hidden pointer (`vptr`) to its class's vtable. A
|
||||
virtual call compiles to: *load the vptr → load the function pointer in the right slot → call it.*
|
||||
|
||||
In WebAssembly there are no raw function pointers; instead there is a single **function table** (an
|
||||
array of functions) and a `call_indirect N` instruction that means "call the function at table index
|
||||
N." So the C++ virtual call becomes, in wasm:
|
||||
|
||||
```wat
|
||||
local.get $item ;; the object pointer
|
||||
i32.load offset=0 ;; load vptr (the vtable address)
|
||||
i32.load offset=8 ;; load the function index from the SetWidth slot
|
||||
;; ... push the call args ...
|
||||
call_indirect (type $vii) ;; call the function at that table index, expecting signature "vii"
|
||||
```
|
||||
|
||||
`(type $vii)` is the *expected signature* baked into the instruction. **Signature notation:** the
|
||||
first letter is the return type, the rest are arguments. `i`=i32, `v`=void.
|
||||
- `ii` = `(i32) -> i32` — e.g. `KICAD_T Type()` (takes the hidden `this`, returns an int-like value).
|
||||
- `vii` = `(i32, i32) -> void` — e.g. `void SetWidth(int)` (`this`, the int; returns nothing).
|
||||
- `viii` = `(i32, i32, i32) -> void` — e.g. `view->Update(item, flags)`.
|
||||
|
||||
**A subtlety that matters here — struct returns (the "sret" ABI).** A method that *returns a small
|
||||
struct by value*, like `VECTOR2I GetPosition()`, can't return two ints in one wasm value. The
|
||||
compiler rewrites it so the caller passes a hidden pointer to a return slot, and the function writes
|
||||
through it and returns nothing: `void GetPosition(this, VECTOR2I* out)`. That's also signature `vii`.
|
||||
So **all of `GetPosition`, `GetClass`, `GetText`, `GetTextSize` (struct/string returns) AND
|
||||
`SetWidth`, `SetPosition` (void setters) are `vii`** — which is why the bug *looked* signature-specific.
|
||||
|
||||
## 1.3 Devirtualization (the fix technique, and its trade-off)
|
||||
|
||||
If you class-qualify the call — `static_cast<PCB_TRACK*>(item)->PCB_TRACK::GetPosition()` — you tell
|
||||
the compiler *exactly* which function to run. It no longer needs the vtable; it emits a plain `call`
|
||||
(a direct call to a known function), not a `call_indirect`. We already do this elsewhere: see
|
||||
`itemLayer()` / `itemPosition()` / `itemClass()` in `wasm/bindings/pcbnew_embind.cpp`.
|
||||
|
||||
**Why it helps the bug:** the hang fires *at the instrumented `call_indirect`*. A direct `call`
|
||||
isn't wrapped the same way, so the symptom doesn't appear there.
|
||||
|
||||
**The trade-off / why it's not free:** class-qualifying picks *one* class's version. For a generic
|
||||
getter like `GetPosition` whose answer depends on the type, you must dispatch on the type yourself
|
||||
(`switch (item->Type()) { case PCB_TRACE_T: ...; case PCB_VIA_T: ...; }`) and class-qualify each arm
|
||||
— otherwise you call the wrong override and get wrong geometry. That's why devirtualization here is a
|
||||
**helper per generic getter** plus a hand edit per setter, and why it's "finite but spread."
|
||||
|
||||
## 1.4 Asyncify — making synchronous C++ pause and resume
|
||||
|
||||
KiCad's C++ is written **synchronously**: it calls `sleep`, it *blocks* waiting for worker threads,
|
||||
it pops up a modal dialog and waits for the user. In a browser you **cannot block the main thread** —
|
||||
if you do, the page freezes (no rendering, no input, no timers). The reconciliation is **Asyncify**, a
|
||||
Binaryen transform that rewrites the wasm so a deep synchronous call stack can be **suspended**
|
||||
(unwound back to the JS event loop) and later **rewound** (rebuilt exactly where it left off).
|
||||
|
||||
Mechanically, Asyncify instruments functions so that:
|
||||
- on **unwind**, each function saves its locals + a "where was I" call-index into a memory buffer and
|
||||
returns up the stack until it reaches the event loop; the buffer pointer is `Asyncify.currData`.
|
||||
- on **rewind**, each function restores its locals and jumps back to the saved call-index, rebuilding
|
||||
the stack until execution resumes at the suspend point.
|
||||
|
||||
There is **one** `currData` slot at a time. If a second suspend starts while the first's buffer is
|
||||
still occupied, things collide — that's the "nested currData contention" family
|
||||
([`docs/features/async/`](../async/)). The three states are **Normal (0)**, **Unwinding (1)**,
|
||||
**Rewinding (2)** (`Asyncify.state` / the wasm export `asyncify_get_state()`).
|
||||
|
||||
Why we can't avoid Asyncify: native edits already rely on it (tool interactions suspend mid-drag, the
|
||||
event loop yields each frame). The collab apply runs the same `BOARD_COMMIT` code, so it inherits the
|
||||
same instrumentation.
|
||||
|
||||
## 1.5 Coroutines and libcontext fibers (KiCad's `COROUTINE`)
|
||||
|
||||
A **fiber** is a *second call stack* you can switch to and from cooperatively (no OS thread). KiCad
|
||||
ships its own `COROUTINE` (built on `libcontext`'s `jump_fcontext`) and runs **tool interactions** on
|
||||
a fiber so a tool can "yield" in the middle of an operation and be resumed later. The fiber has its
|
||||
own stack memory; switching is just swapping the stack pointer.
|
||||
|
||||
Why it's in the collab apply: KiCad-WASM has a long-standing rule that the heavy edit machinery
|
||||
(`BOARD_COMMIT::Modify` → `item->Clone()`, the GAL `view->Add`) **only dispatches correctly when run
|
||||
on the tool-coroutine fiber stack** — running it from a bare `CallAfter`/`ccall` trapped with
|
||||
"indirect call signature mismatch." So `kicadCollabApply` wraps `doApply` in a `COROUTINE`. (This is
|
||||
fix #1 below; it's real and necessary.)
|
||||
|
||||
## 1.6 Futexes — and why they deadlock on the browser main thread
|
||||
|
||||
A **futex** ("fast userspace mutex") is the low-level OS primitive a thread uses to **wait until
|
||||
another thread signals it**. `std::mutex`, `std::condition_variable`, and `std::future::get()` are all
|
||||
built on it. The pattern: thread A wants thread B's result, so A does `futex_wait(addr, val)` — "sleep
|
||||
until the value at `addr` changes" — and B does `futex_wake(addr)` when it's done.
|
||||
|
||||
KiCad's **connectivity recompute** (rebuilding the ratsnest/net graph after an edit, in
|
||||
`commit.Push`) is parallelised across a thread pool. The main thread submits work and then
|
||||
`futex_wait`s for the workers' results (`std::future::wait_for` → `pthread_cond_wait` →
|
||||
`emscripten_futex_wait`).
|
||||
|
||||
**The browser problem:** on the **main browser thread**, `Atomics.wait` (the real blocking wait) is
|
||||
*forbidden* — blocking it would freeze the page. Emscripten's fallback is to **busy-spin** in
|
||||
`futex_wait_main_browser_thread()`, calling `_emscripten_yield()` — but that only services the
|
||||
internal proxy queue, it **never returns to the JS event loop**. So if the worker the main thread is
|
||||
waiting on still needs the event loop to run (e.g. an **on-demand Web Worker** has to finish its
|
||||
`loaded → run` handshake), it never gets to — and the busy-spin spins forever. **Deadlock.**
|
||||
|
||||
```
|
||||
main thread: submit work ──► futex_wait(result) ──► busy-spin _emscripten_yield() ──► (spins forever)
|
||||
│ never pumps the JS event loop
|
||||
worker boot: 'loaded' ─X─► 'run' (needs the event loop, which never runs) ──► never produces result
|
||||
```
|
||||
|
||||
**Our solution — `wasm/shims/futex_yield.c`** (fix #2 below): a *strong override* of
|
||||
`emscripten_futex_wait` that, **on the main thread only**, polls the futex word and between polls does
|
||||
an **Asyncify yield** (`await setTimeout(0)`) instead of busy-spinning. Yielding pumps the JS event
|
||||
loop, so the on-demand Worker boots, finishes, wakes the futex, and the wait returns. (On worker
|
||||
threads it keeps the real blocking `memory.atomic.wait32`.) It's the sibling of the existing
|
||||
`nanosleep_yield.c`, which covers the `sleep_for`/`nanosleep` path but not the futex path.
|
||||
|
||||
## 1.7 The function table and `-sDYNCALLS=1` (one caveat to retire a red herring)
|
||||
|
||||
There is a single wasm function table; `call_indirect` indexes it; the type section lists the
|
||||
distinct signatures (`ii`, `vii`, `viii`, …). With `-sDYNCALLS=1` emscripten also exports per-signature
|
||||
`dynCall_<sig>` trampolines for JS↔wasm calls. A known hazard (`dyncall-binding.js.tmpl`) is that
|
||||
*post-asyncify+O2 a `dynCall_<sig>` JS trampoline can carry a stale expected type* — but that is a
|
||||
**legacy-EH** mechanism (the `invoke_*`→`dynCall` path). Under native-EH a C++ virtual call is a
|
||||
**raw `call_indirect`**, not a `dynCall`, so that hazard does not apply. (Verified by disassembly — see
|
||||
Part 4.)
|
||||
|
||||
---
|
||||
|
||||
# Part 2 — The collab apply call chains
|
||||
|
||||
A peer edit arrives as JSON and is applied like this:
|
||||
|
||||
```
|
||||
JS: window.Module.kicadCollabApply(jsonDelta)
|
||||
└─ kicadCollabApply(std::string) [pcbnew_embind.cpp]
|
||||
└─ parse JSON → fr->CallAfter([...]) (defer to the wx main-loop drain)
|
||||
└─ COROUTINE cor([]{ doApply(fr, delta); }) (run on a libcontext fiber — §1.5)
|
||||
└─ cor.Call(0)
|
||||
└─ doApply(frame, delta) [pcbnew_embind.cpp]
|
||||
├─ for removed: commit.Remove(item)
|
||||
├─ for changed: commit.Modify(item) ──► item->Clone() (virtual, fix #1)
|
||||
│ applyChanged(item, j)
|
||||
│ └─ tr->SetStart/SetEnd (non-virtual, fine)
|
||||
│ └─ tr->SetWidth(w) ◄── VIRTUAL "vii" ★ THE HANG
|
||||
├─ for added: makeItem(...) → commit.Add(item)
|
||||
└─ commit.Push("Collaborative edit") [board_commit.cpp]
|
||||
└─ connectivity->RecalculateRatsnest(...)
|
||||
└─ thread-pool results.get() ──► emscripten_futex_wait (fix #2)
|
||||
└─ (back on the main stack, after cor.Call returns)
|
||||
└─ rebaseline() (fix #3 — moved out of the fiber)
|
||||
└─ snapshotByUuid(board)
|
||||
└─ for each item: itemToJson(item)
|
||||
└─ itemPosition/itemClass/GetText… ◄── VIRTUAL "vii" ★ more of THE HANG
|
||||
```
|
||||
|
||||
Three things in this chain are independently load-bearing, and each was a distinct bug:
|
||||
|
||||
1. `item->Clone()` (inside `commit.Modify`) and `view->Add` (inside `commit.Push`) **must** run on the
|
||||
fiber or they trap → **fix #1 (COROUTINE)**.
|
||||
2. `commit.Push`'s connectivity recompute **futex-deadlocks** on the main thread → **fix #2
|
||||
(`futex_yield.c`)**.
|
||||
3. `rebaseline()` (the post-apply snapshot) at the end of `doApply` **must** run after the fiber
|
||||
finalizes, not inside it → **fix #3 (move to main stack)**.
|
||||
|
||||
And then the headline: the `vii` virtual calls (`SetWidth`, the snapshot getters) **hang**.
|
||||
|
||||
---
|
||||
|
||||
# Part 3 — The four bugs and the fixes that work
|
||||
|
||||
| # | Bug | Symptom | Fix | Status |
|
||||
|---|-----|---------|-----|--------|
|
||||
| 1 | `Clone`/`view->Add` dispatch off the fiber | "indirect call signature mismatch" **trap** | run `doApply` in a `COROUTINE` (`kicadCollabApply`) | ✅ validated |
|
||||
| 2 | connectivity recompute futex on main thread | busy-spin **deadlock** (Worker can't boot) | `wasm/shims/futex_yield.c` (Asyncify-yield the main-thread futex wait) | ✅ validated (red→green in the `pool-callafter` repro) |
|
||||
| 3 | `rebaseline()` inside the fiber after a suspend | fiber **never finalizes**, blocks the next apply | move `rebaseline()` to the main stack after `cor.Call` | ✅ validated |
|
||||
| 4 | `vii` virtual calls (`SetWidth`, snapshot getters) | **hang** (asyncify suspend-without-resume) | devirtualize the call (class-qualify) **OR** … (see Part 4) | ⚠️ confound; see below |
|
||||
|
||||
Fixes 1–3 are real and should be kept regardless of the Part-5 decision. Build-plumbing fix to keep
|
||||
too: `-Xclang -fno-pch-timestamp` in `build-kicad-target.sh` (a PCH-staleness workaround).
|
||||
|
||||
---
|
||||
|
||||
# Part 4 — The `vii` hang is a confound (the investigation)
|
||||
|
||||
### The symptom and the obvious (wrong) theory
|
||||
The apply hangs at `SetWidth` (a `vii` call). Devirtualize it → the apply progresses to the next
|
||||
`vii` call (`GetPosition` in the snapshot) → devirtualize that → the next `vii` (`GetClass`, then the
|
||||
text getters) … Meanwhile value-returning `ii` virtuals (`Type`, `GetLayer`, `GetWidth`, `Clone`) and
|
||||
3-arg-void `viii` virtuals (`view->Update`) work. **Obvious theory: the `vii` signature is broken.**
|
||||
|
||||
### Five controlled repros — all pass
|
||||
We built a minimal libcontext-fiber app
|
||||
(`tests/apps/standalone/coroutine-pthread/vcall_fiber_repro.cpp`) that calls all four signatures on a
|
||||
non-devirtualizable polymorphic object (99 genuine `call_indirect`s, confirmed not optimized away),
|
||||
native-EH + asyncify, and progressively added every suspected ingredient:
|
||||
|
||||
| Repro | Added ingredient | Result |
|
||||
|-------|------------------|--------|
|
||||
| `vcall_fiber_repro` | fiber + 4 signatures | **all pass** |
|
||||
| + interleaved suspend | `emscripten_sleep` before each call | **all pass** |
|
||||
| `vcall_mainloop_repro` | rAF `set_main_loop` → `dynCall_v` → COROUTINE | **all pass** |
|
||||
| `vcall_ehloop_repro` | `try`/`catch_all` + RAII dtor + suspend-in-try + loop | **all pass** |
|
||||
| FIX-D test (real pcbnew) | save+clear `Asyncify.currData` before the apply | `currData` was **null**; still hangs |
|
||||
|
||||
A `vii` `call_indirect` dispatches correctly under *every* condition we could isolate — fiber,
|
||||
suspend/rewind, the rAF/`dynCall_v` boundary, even nested native-EH `try/catch_all` (the
|
||||
HoistCppCatches regime) with a suspend inside the try. The signature is **not** the cause.
|
||||
|
||||
### Two binary-level investigations — dispatch ruled out
|
||||
Disassembling the actual `pcbnew.wasm` and the repro:
|
||||
- The asyncify pass is **type-agnostic** (`binaryen/src/passes/Asyncify.cpp`): the void path is the
|
||||
*simpler* subset; `vii` and `viii` get a byte-for-byte identical guard.
|
||||
- The function **table is not reordered/re-indexed** (the only fork pass, `HoistCppCatches`, is
|
||||
intra-function; `-O2` `directize` preserves index→function bindings).
|
||||
- The **embind-vs-core compile flags match** (same EH model, `-O`, RTTI, struct-ABI) — no ABI
|
||||
divergence that could move a vtable slot or change a `call_indirect` type.
|
||||
- The parking `SetWidth` call site has the **same asyncify guard** as the repro's working `setVii`;
|
||||
no trampoline, no `i64` legalization, no stale type.
|
||||
|
||||
The single structural difference round-2 could point to was that the real call sits inside `doApply`'s
|
||||
**deeply nested native-EH `try`/`catch_all`** (48 `try` / 47 `catch_all` — every throwing JSON access
|
||||
has RAII cleanup), in a **loop**, with the suspend landing inside hoisted catch scopes — but the
|
||||
`vcall_ehloop_repro` reproduced exactly that and **passed**.
|
||||
|
||||
### Conclusion
|
||||
The `vii` correlation is a **confound**: devirtualizing removes the instrumented `call_indirect` and
|
||||
shifts the symptom to the next one. The dispatch is provably fine. The hang is a property of the **full
|
||||
180 MB asyncify+O2 module's runtime** (the real KiCad vtables/table in the live process) that no
|
||||
isolation reproduces and that the disassembly couldn't byte-verify. The one stubborn fact that resists
|
||||
*every* named mechanism: `SetWidth` parks with asyncify state **Normal** and `currData` **null** —
|
||||
identical to the repros that pass.
|
||||
|
||||
---
|
||||
|
||||
# Part 5 — The decision: (A) devirtualize-through vs (B) defer collab
|
||||
|
||||
Everything else in native-EH pcbnew is green (core e2e, 3D, and every other app —
|
||||
[`10-pthreads-native-eh.md`](10-pthreads-native-eh.md)). Collab apply is the lone holdout.
|
||||
|
||||
## Option A — devirtualize-through (treat the confound, get collab green)
|
||||
|
||||
Replace every `vii` virtual call in the apply path with a class-qualified (direct) call, dispatching
|
||||
on `Type()` where the override matters.
|
||||
|
||||
**Where the remaining work is** (the snapshot getters are already done via `itemPosition`/`itemClass`/
|
||||
the text-getter edits):
|
||||
|
||||
| File | Site | Kind |
|
||||
|------|------|------|
|
||||
| `wasm/bindings/pcbnew_embind.cpp` | `applyChanged` else-branch `aItem->SetPosition(...)` | 1 setter (type-dispatched) |
|
||||
| `wasm/bindings/pcbnew_embind.cpp` | `makeItem` added-item setters (`SetWidth`, `SetPosition`, `SetText`, …; `SetStart/SetEnd` already non-virtual, `SetLayer` already done) | ~3–5 setters |
|
||||
| `wasm/bindings/eeschema_embind.cpp` | `itemToJson` snapshot getters (`GetPosition`/`GetClass`/text — mirror of pcbnew's helpers) | ~3–5 getters (1 helper pair) |
|
||||
| `wasm/bindings/eeschema_embind.cpp` | `doApply`/`applyChanged` + `makeItem` setters (`Move` already devirtualized) | ~3–5 setters |
|
||||
|
||||
**Effort estimate (concrete):**
|
||||
- **Code edits:** ~10–15 sites, each a 1-line class-qualify or a small `switch(Type())` helper.
|
||||
Mechanically small — call it **2–4 hours of editing**, including writing the eeschema helper pair to
|
||||
match pcbnew's.
|
||||
- **The real cost is the build/test loop, not the edits.** Each un-devirtualized `vii` surfaces *one
|
||||
at a time* (the apply hangs at the first one; you fix it, rebuild ~10–35 min, it hangs at the next).
|
||||
Doing it reactively ⇒ **~10–15 build cycles**. You can cut that by proactively grepping every
|
||||
by-value getter/void setter in the two apply paths and devirtualizing them in one pass, but you
|
||||
still need a few full builds + the **3-browser** e2e (Firefox/Chrome/WebKit) per the project rule.
|
||||
Realistically **~1–2 days wall-clock**, dominated by builds + cross-browser verification.
|
||||
- **Fragility (the ongoing cost):** this fixes nothing structural. Any *new* by-value getter or void
|
||||
setter added to the apply/snapshot path later — a new field synced, a KiCad upstream change — will
|
||||
**silently re-hang** the apply under native-EH. Mitigation: a prominent comment + ideally a tiny
|
||||
lint/grep in CI flagging un-class-qualified virtual calls in the embind apply paths. Without that,
|
||||
it's a latent foot-gun.
|
||||
|
||||
**Net:** A is a known, finite, *certain* path to green, but it's symptom-treatment with a maintenance
|
||||
tail.
|
||||
|
||||
## Option B — defer collab (recommended)
|
||||
|
||||
Ship native-EH as the default for everything that's green (core, 3D, gerbview, pl_editor,
|
||||
symbol_editor, eeschema, footprint_editor). Leave the **collab apply** path as a documented native-EH
|
||||
limitation; it continues to work on the legacy-EH build. Revisit if/when the large-module root ever
|
||||
surfaces (e.g. a future Binaryen/emscripten bump changes the picture, or someone reproduces it
|
||||
minimally).
|
||||
|
||||
**Why recommended:** the root resisted 2 deep binary investigations + 5 controlled repros + the FIX-D
|
||||
test; the dispatch is provably correct; A is fragile work treating a confound. The cost/benefit of
|
||||
chasing a non-reproducible large-module runtime bug — or maintaining a hand-devirtualized apply path
|
||||
forever — is poor relative to shipping the 95% that's done.
|
||||
|
||||
---
|
||||
|
||||
## Appendix — artifacts and key locations
|
||||
|
||||
**Validated fixes (keep):**
|
||||
- `wasm/shims/futex_yield.c` — main-thread futex Asyncify-yield (fix #2).
|
||||
- `wasm/bindings/pcbnew_embind.cpp` — `kicadCollabApply` COROUTINE (fix #1) + rebaseline-on-main-stack
|
||||
(fix #3) + the `itemPosition`/`itemClass`/`itemLayer` devirtualized snapshot helpers.
|
||||
- `scripts/kicad/build-kicad-target.sh` — `futex_yield.o` wired into the link; `-fno-pch-timestamp`.
|
||||
|
||||
**Isolation repros (temporary — remove before final staging):**
|
||||
- `tests/apps/standalone/coroutine-pthread/vcall_fiber_repro.cpp` (signature isolation, +suspend)
|
||||
- `tests/apps/standalone/coroutine-pthread/vcall_mainloop_repro.cpp` (rAF/`dynCall_v` context)
|
||||
- `tests/apps/standalone/coroutine-pthread/vcall_ehloop_repro.cpp` (try/catch_all + RAII + loop)
|
||||
- `tests/apps/standalone/pool-callafter/` (the futex deadlock red→green repro for fix #2)
|
||||
- `tests/e2e/coroutine-vcall.spec.ts`, `tests/e2e/coroutine-poolwait.spec.ts`
|
||||
- The `[collab-diag]`/`[push-diag]` `EM_ASM` markers in `pcbnew_embind.cpp` + `board_commit.cpp` and
|
||||
the `#include <emscripten.h>` in `board_commit.cpp` are temporary diagnostics to revert.
|
||||
|
||||
**Key reading:**
|
||||
- `binaryen/src/passes/Asyncify.cpp` — the suspend/rewind instrumentation (type-agnostic).
|
||||
- `scripts/common/inject-dyncall-shims.sh`, `scripts/common/shims/{handlesleep.js,dyncall-binding.js.tmpl}`
|
||||
— the JS-side asyncify/dynCall plumbing.
|
||||
- [`docs/features/async/`](../async/) — the `currData` contention dossier.
|
||||
|
|
@ -1,5 +1,9 @@
|
|||
# `-fexceptions` vs `-fwasm-exceptions` in KiCad-WASM — research dossier
|
||||
|
||||
> **✅ FINALIZED:** native wasm-EH is now the **only** build mode — there is no `-fexceptions` /
|
||||
> `WX_LEGACY_EH` path, and the 3D viewer builds by default. The migration plan, audit, and spike
|
||||
> notes below are retained as the historical research/decision record.
|
||||
|
||||
> **Status:** research / decision record. A parallel session attempted the migration
|
||||
> end-to-end and **parked it** on an emscripten-4.0.2 LLVM codegen bug — see
|
||||
> [`docs/wasm-exceptions-experiment.md`](../../wasm-exceptions-experiment.md) (full
|
||||
|
|
@ -11,6 +15,15 @@
|
|||
> `currData` contention dossier) — this dossier covers the *exception-handling* axis of
|
||||
> the same machine.
|
||||
|
||||
> **UPDATE 2026-06-22 (see [`06-spike-plan.md`](06-spike-plan.md)).** A 5-agent spike refreshed
|
||||
> this dossier and corrected three things below: (1) **the encoding is resolved to LEGACY**
|
||||
> (`WASM_LEGACY_EXCEPTIONS=1`) — Asyncify can't consume exnref in any released Binaryen, so the
|
||||
> "exnref → TryTable variant" fork is closed; the experiment's `=0` was a dead end. (2) **Binaryen
|
||||
> is not a blocker** — CI/publish already pin `BINARYEN_VERSION=130` (the "v121 locally" note below
|
||||
> is only the finalize/in-link copy). (3) The long pole is the **emsdk/LLVM compiler bump** for
|
||||
> parseable legacy wasm-EH + the OCC `br_table` fix, *not* a newer wasm-opt. The phased red-green
|
||||
> plan lives in 06.
|
||||
|
||||
## Why this exists
|
||||
|
||||
The whole build is on **`-fexceptions`** (Emscripten's JavaScript-based exception
|
||||
|
|
@ -60,10 +73,12 @@ KiCad at all.
|
|||
| [`01-background-two-eh-models.md`](01-background-two-eh-models.md) | How JS-EH (`invoke_*`) and wasm-EH actually work, and the three concrete couplings into our Asyncify machine. |
|
||||
| [`02-measurements.md`](02-measurements.md) | Our controlled size experiment on pcbnew (methodology + numbers) and the published third-party benchmarks. |
|
||||
| [`03-toolchain-status.md`](03-toolchain-status.md) | Compatibility matrix: emcc checks, binaryen history (what merged in v125, what didn't), JSPI/fibers, setjmp/longjmp, mixing modes. |
|
||||
| [`04-kicad-audit.md`](04-kicad-audit.md) | The brace-matching catch-block audit: 85 direct / 93 review / 458 trivial of 636; libpng/libjpeg setjmp story; refactor effort if done by hand. |
|
||||
| [`05-asyncify-fork-design.md`](05-asyncify-fork-design.md) | Asyncify.cpp internals, why catch arms are structurally hard, and the catch-arm-hoisting fork design with limits and effort. |
|
||||
| [`catch_audit.py`](catch_audit.py) | The audit tool (re-runnable; suitable as a CI gate on the kicad submodule). |
|
||||
| [`audit-results.txt`](audit-results.txt) | Full audit output incl. all 85 direct-suspend sites. |
|
||||
| [`06-spike-plan.md`](06-spike-plan.md) | **(2026-06-22)** Refreshed findings + the phased red-green spike plan; supersedes the encoding/Binaryen-version framing above. |
|
||||
| [`07-spike-results-and-opinion.md`](07-spike-results-and-opinion.md) | **(2026-06-22)** Toy-spike results: asyncify + legacy-wasm-EH works; the `HoistCppCatches` Binaryen pass flips suspend-in-catch green on all 3 engines; go/no-go opinion. |
|
||||
| [`08-wx-app-render-rootcause.md`](08-wx-app-render-rootcause.md) | Why a native-EH wx app rendered blank: the `set_main_loop` `"unwind"` throw caught by native-EH `catch_all` cleanup pads tore down the main frame. |
|
||||
| [`09-event-loop-deparking-plan.md`](09-event-loop-deparking-plan.md) | The EH-agnostic main-loop rework (de-park → per-frame-yield `while`-loop) fixing the blank render + the coroutine/menu regressions. |
|
||||
| [`10-pthreads-native-eh.md`](10-pthreads-native-eh.md) | **(2026-06-24)** Native-EH × pthreads: the main-thread thread-spawn regression (`invalid state: 1` / re-entrant `main()`); the pool pattern survives; the KiCad raw→pool refactor plan + the test gap. |
|
||||
|
||||
## Relationship to docs/features/async/
|
||||
|
||||
|
|
|
|||
|
|
@ -1,135 +0,0 @@
|
|||
{
|
||||
"total": 636,
|
||||
"trivial": 458,
|
||||
"needs_review": 93,
|
||||
"direct_suspend": 85
|
||||
}
|
||||
|
||||
== per top-level dir (direct/review/trivial/infobar):
|
||||
eeschema direct= 37 review= 17 trivial=127 infobar= 0
|
||||
pcbnew direct= 32 review= 29 trivial=128 infobar= 0
|
||||
common direct= 9 review= 35 trivial=126 infobar= 0
|
||||
cvpcb direct= 3 review= 1 trivial= 0 infobar= 0
|
||||
pcb_calculator direct= 2 review= 0 trivial= 2 infobar= 0
|
||||
kicad direct= 1 review= 1 trivial= 21 infobar= 0
|
||||
pagelayout_editor direct= 1 review= 1 trivial= 3 infobar= 0
|
||||
scripting direct= 0 review= 0 trivial= 1 infobar= 0
|
||||
plugins direct= 0 review= 1 trivial= 4 infobar= 0
|
||||
3d-viewer direct= 0 review= 2 trivial= 6 infobar= 0
|
||||
utils direct= 0 review= 4 trivial= 27 infobar= 0
|
||||
libs direct= 0 review= 2 trivial= 7 infobar= 0
|
||||
gerbview direct= 0 review= 0 trivial= 6 infobar= 0
|
||||
|
||||
== top 25 unknown callees inside needs_review catches (freq):
|
||||
4 clearOutlines
|
||||
4 message
|
||||
4 wxASSERT_MSG
|
||||
4 handleException
|
||||
4 tl::unexpected
|
||||
3 wxLogFatalError
|
||||
3 GetFullFilename
|
||||
3 std::string
|
||||
3 ReportMsg
|
||||
3 line_at
|
||||
3 positions
|
||||
3 LIBRARY_PARSE_ERROR
|
||||
3 move_push
|
||||
3 wxRemoveFile
|
||||
3 IsTooRecent
|
||||
3 Contents
|
||||
3 GetFieldValue
|
||||
3 ToOrigString
|
||||
3 SetValue
|
||||
2 GetFilename
|
||||
2 ClearShapes
|
||||
2 GetFPIDAsString
|
||||
2 wxString::FromUTF8
|
||||
2 GetFormatName
|
||||
2 nan
|
||||
|
||||
== direct-suspend sites (85):
|
||||
cvpcb/display_footprints_frame.cpp:310
|
||||
cvpcb/cvpcb_mainframe.cpp:977
|
||||
cvpcb/readwrite_dlgs.cpp:163
|
||||
pcb_calculator/datafile_read_write.cpp:69
|
||||
pcb_calculator/calculator_panels/panel_r_calculator.cpp:130
|
||||
kicad/kicad_manager_frame.cpp:822
|
||||
common/draw_panel_gal.cpp:325
|
||||
common/draw_panel_gal.cpp:561
|
||||
common/drawing_sheet/ds_data_model_io.cpp:100
|
||||
common/drawing_sheet/ds_data_model_io.cpp:131
|
||||
common/widgets/design_block_pane.cpp:121
|
||||
common/widgets/design_block_pane.cpp:212
|
||||
common/widgets/design_block_pane.cpp:315
|
||||
common/widgets/design_block_pane.cpp:356
|
||||
common/widgets/design_block_pane.cpp:418
|
||||
pcbnew/load_select_footprint.cpp:378
|
||||
pcbnew/pcb_base_frame.cpp:1111
|
||||
pcbnew/pcb_base_frame.cpp:1239
|
||||
pcbnew/pcb_edit_frame.cpp:2209
|
||||
pcbnew/files.cpp:670
|
||||
pcbnew/files.cpp:678
|
||||
pcbnew/files.cpp:689
|
||||
pcbnew/files.cpp:1024
|
||||
pcbnew/files.cpp:1108
|
||||
pcbnew/footprint_libraries_utils.cpp:198
|
||||
pcbnew/footprint_libraries_utils.cpp:288
|
||||
pcbnew/footprint_libraries_utils.cpp:401
|
||||
pcbnew/footprint_libraries_utils.cpp:545
|
||||
pcbnew/footprint_libraries_utils.cpp:613
|
||||
pcbnew/footprint_libraries_utils.cpp:673
|
||||
pcbnew/footprint_libraries_utils.cpp:817
|
||||
pcbnew/pcb_design_block_utils.cpp:107
|
||||
pcbnew/pcb_design_block_utils.cpp:168
|
||||
pcbnew/pcb_design_block_utils.cpp:198
|
||||
pcbnew/pcb_design_block_utils.cpp:226
|
||||
pcbnew/pcb_design_block_utils.cpp:332
|
||||
pcbnew/pcb_design_block_utils.cpp:429
|
||||
pcbnew/pcb_draw_panel_gal.cpp:781
|
||||
pcbnew/tools/board_editor_control.cpp:751
|
||||
pcbnew/tools/footprint_editor_control.cpp:529
|
||||
pcbnew/tools/pcb_control.cpp:1990
|
||||
pcbnew/netlist_reader/netlist.cpp:74
|
||||
pcbnew/exporters/export_idf.cpp:662
|
||||
pcbnew/exporters/export_idf.cpp:670
|
||||
pcbnew/dialogs/dialog_board_setup.cpp:385
|
||||
pcbnew/specctra_import_export/specctra_import.cpp:71
|
||||
pcbnew/widgets/pcb_design_block_preview_widget.cpp:191
|
||||
eeschema/project_rescue.cpp:671
|
||||
eeschema/project_rescue.cpp:819
|
||||
eeschema/sch_draw_panel.cpp:200
|
||||
eeschema/project_sch.cpp:108
|
||||
eeschema/project_sch.cpp:122
|
||||
eeschema/sch_design_block_utils.cpp:139
|
||||
eeschema/sch_design_block_utils.cpp:179
|
||||
eeschema/sch_design_block_utils.cpp:222
|
||||
eeschema/sch_design_block_utils.cpp:364
|
||||
eeschema/sch_design_block_utils.cpp:446
|
||||
eeschema/sch_design_block_utils.cpp:526
|
||||
eeschema/sch_edit_frame.cpp:1528
|
||||
eeschema/sch_base_frame.cpp:96
|
||||
eeschema/sheet.cpp:231
|
||||
eeschema/files-io.cpp:356
|
||||
eeschema/files-io.cpp:365
|
||||
eeschema/files-io.cpp:374
|
||||
eeschema/files-io.cpp:911
|
||||
eeschema/files-io.cpp:1410
|
||||
eeschema/files-io.cpp:1423
|
||||
eeschema/symbol_library_manager.cpp:376
|
||||
eeschema/symbol_library_manager.cpp:553
|
||||
eeschema/tools/sch_editor_control.cpp:569
|
||||
eeschema/tools/sch_editor_control.cpp:1703
|
||||
eeschema/dialogs/dialog_edit_symbols_libid.cpp:742
|
||||
eeschema/dialogs/dialog_sheet_properties.cpp:597
|
||||
eeschema/dialogs/dialog_sim_model.cpp:1526
|
||||
eeschema/dialogs/dialog_bom.cpp:361
|
||||
eeschema/sim/spice_simulator.cpp:44
|
||||
eeschema/sim/spice_value.cpp:415
|
||||
eeschema/sim/simulator_frame_ui.cpp:1672
|
||||
eeschema/symbol_editor/symbol_editor.cpp:170
|
||||
eeschema/symbol_editor/symbol_editor.cpp:236
|
||||
eeschema/symbol_editor/symbol_editor.cpp:1147
|
||||
eeschema/symbol_editor/symbol_editor.cpp:1187
|
||||
eeschema/symbol_editor/symbol_editor_import_export.cpp:104
|
||||
eeschema/symbol_editor/symbol_editor_import_export.cpp:110
|
||||
pagelayout_editor/tools/pl_edit_tool.cpp:531
|
||||
|
|
@ -1,109 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Audit KiCad catch blocks for wasm-EH safety (suspension inside catch handlers)."""
|
||||
import os, re, sys, json
|
||||
from collections import defaultdict
|
||||
|
||||
ROOT = "/Users/V/IdeaProjects/kicad-wasm/kicad"
|
||||
SKIP_DIRS = {"thirdparty", ".git", "qa", "build"}
|
||||
|
||||
# Calls that suspend (Asyncify) directly or are dialog wrappers in KiCad/wx
|
||||
DIRECT_SUSPEND = [
|
||||
"DisplayErrorMessage", "DisplayError", "DisplayInfoMessage", "DisplayHtmlInfoMessage",
|
||||
"wxMessageBox", "ShowModal", "ShowQuasiModal", "KIDIALOG", "OKOrCancelDialog",
|
||||
"wxMessageDialog", "IsOK(", "DisplayLoadError", "ShowAboutDialog",
|
||||
"wxGetSingleChoice", "wxTextEntryDialog", "wxFileDialog", "wxDirDialog",
|
||||
"GetDataFromClipboard", "SaveToClipboard", "wxClipboard", "EnumerateFacenames",
|
||||
]
|
||||
INFOBAR = ["ShowInfoBarError", "ShowInfoBarMsg", "ShowInfoBarWarning"]
|
||||
# Benign callees: logging (wxLog* is deferred to idle-time flush -> not inside catch),
|
||||
# string formatting, rethrow, reporters writing text
|
||||
BENIGN = {
|
||||
"wxLogError", "wxLogWarning", "wxLogMessage", "wxLogTrace", "wxLogDebug", "wxLogVerbose",
|
||||
"Format", "Printf", "printf", "fprintf", "snprintf", "What", "Problem", "Where",
|
||||
"GetErrorMessage", "wxString", "FROM_UTF8", "TO_UTF8", "UTF8", "c_str", "mb_str",
|
||||
"GetChars", "IsEmpty", "empty", "clear", "size", "length", "Report", "ReportTail",
|
||||
"ReportHead", "Add", "push_back", "emplace_back", "insert", "append", "Append",
|
||||
"SetError", "assert", "wxASSERT", "wxFAIL", "wxCHECK", "abort", "exit",
|
||||
"GetMessages", "GetErrors", "reset", "get", "release", "find", "count", "at",
|
||||
"begin", "end", "str", "Mid", "Left", "Right", "Trim", "Lower", "Upper",
|
||||
"StartsWith", "EndsWith", "Contains", "Replace", "Remove", "make_unique",
|
||||
"make_shared", "static_cast", "dynamic_cast", "const_cast", "reinterpret_cast",
|
||||
"Clear", "Close", "swap", "resize", "erase", "Set", "SetBitmap", "Destroy",
|
||||
"wxT", "_", "_HKI", "traceSchPlugin", "TRACE", "what", "THROW_IO_ERROR", "wxS", "wxFAIL_MSG", "IDF_ERROR", "LIBRARY_ERROR", "FUTURE_FORMAT_ERROR", "PARSE_ERROR", "KI_PARAM_ERROR", "fmt::format", "format", "GetFullPath", "HandleException", "Pgm", "current_exception", "rethrow_exception", "Nickname", "GetName", "GetLibNickname", "GetLibItemName", "wx_str", "GetMessageString", "UnescapeString", "exceptions", "CLOSE_STREAM", "AddError", "GetRequiredVersion", "Disconnect", "From_UTF8", "typeid", "LogException", "Instance", "GetFullName", "GetUniStringLibId", "ShowText", "SetStatusText", "GetItemDescription", "GetClass", "GetFriendlyName", "IsValid", "GetPath", "GetFileName", "GetExtension", "Length", "GetData", "data", "front", "back", "pop_back", "emplace", "GetSettingsManager",
|
||||
}
|
||||
CALL_RE = re.compile(r"\b([A-Za-z_][A-Za-z0-9_:]*)\s*\(")
|
||||
CATCH_RE = re.compile(r"\bcatch\s*\(")
|
||||
|
||||
def find_block(text, start):
|
||||
"""Return (block_text, end_idx) for brace-block starting at first '{' at/after start."""
|
||||
i = text.find("{", start)
|
||||
if i < 0: return None, start
|
||||
depth, j, n = 0, i, len(text)
|
||||
in_str = in_chr = in_lc = in_bc = False
|
||||
while j < n:
|
||||
c = text[j]; p = text[j-1] if j else ""
|
||||
if in_lc:
|
||||
if c == "\n": in_lc = False
|
||||
elif in_bc:
|
||||
if p == "*" and c == "/": in_bc = False
|
||||
elif in_str:
|
||||
if c == '"' and p != "\\": in_str = False
|
||||
elif in_chr:
|
||||
if c == "'" and p != "\\": in_chr = False
|
||||
elif c == "/" and j+1 < n and text[j+1] == "/": in_lc = True
|
||||
elif c == "/" and j+1 < n and text[j+1] == "*": in_bc = True
|
||||
elif c == '"': in_str = True
|
||||
elif c == "'": in_chr = True
|
||||
elif c == "{": depth += 1
|
||||
elif c == "}":
|
||||
depth -= 1
|
||||
if depth == 0: return text[i:j+1], j+1
|
||||
j += 1
|
||||
return None, start
|
||||
|
||||
stats = defaultdict(int)
|
||||
direct_sites, review_sites = [], []
|
||||
review_callees = defaultdict(int)
|
||||
per_dir = defaultdict(lambda: defaultdict(int))
|
||||
|
||||
for dirpath, dirnames, filenames in os.walk(ROOT):
|
||||
dirnames[:] = [d for d in dirnames if d not in SKIP_DIRS]
|
||||
for fn in filenames:
|
||||
if not fn.endswith(".cpp"): continue
|
||||
path = os.path.join(dirpath, fn)
|
||||
rel = os.path.relpath(path, ROOT)
|
||||
top = rel.split(os.sep)[0]
|
||||
try: text = open(path, encoding="utf-8", errors="replace").read()
|
||||
except OSError: continue
|
||||
for m in CATCH_RE.finditer(text):
|
||||
block, _ = find_block(text, m.end())
|
||||
if block is None: continue
|
||||
stats["total"] += 1
|
||||
line = text[:m.start()].count("\n") + 1
|
||||
loc = f"{rel}:{line}"
|
||||
if any(s in block for s in DIRECT_SUSPEND):
|
||||
stats["direct_suspend"] += 1; per_dir[top]["direct"] += 1
|
||||
direct_sites.append(loc)
|
||||
continue
|
||||
if any(s in block for s in INFOBAR):
|
||||
stats["infobar"] += 1; per_dir[top]["infobar"] += 1
|
||||
continue
|
||||
calls = set(CALL_RE.findall(block)) - {"catch", "if", "for", "while", "switch", "return", "sizeof", "throw"}
|
||||
unknown = {c for c in calls if c.split("::")[-1] not in BENIGN and c not in BENIGN}
|
||||
if not unknown:
|
||||
stats["trivial"] += 1; per_dir[top]["trivial"] += 1
|
||||
else:
|
||||
stats["needs_review"] += 1; per_dir[top]["review"] += 1
|
||||
review_sites.append((loc, sorted(unknown)[:6]))
|
||||
for c in unknown: review_callees[c] += 1
|
||||
|
||||
print(json.dumps(stats, indent=1))
|
||||
print("\n== per top-level dir (direct/review/trivial/infobar):")
|
||||
for d in sorted(per_dir, key=lambda d: -per_dir[d]["direct"]):
|
||||
p = per_dir[d]
|
||||
print(f" {d:24s} direct={p['direct']:3d} review={p['review']:3d} trivial={p['trivial']:3d} infobar={p['infobar']:2d}")
|
||||
print("\n== top 25 unknown callees inside needs_review catches (freq):")
|
||||
for c, n in sorted(review_callees.items(), key=lambda kv: -kv[1])[:25]:
|
||||
print(f" {n:3d} {c}")
|
||||
print(f"\n== direct-suspend sites ({len(direct_sites)}):")
|
||||
for s in direct_sites: print(" " + s)
|
||||
Loading…
Reference in a new issue