pcbjam/docs/features/wasm-exceptions/10-pthreads-native-eh.md

Native wasm-EH × pthreads — findings and test coverage

Status: native wasm-EH (-fwasm-exceptions) is the default build; the pthread test suite is green in Firefox + Chrome. Authored 2026-06-24, updated 2026-06-25. The exception-handling-side companion to the mechanism-deep ../threading/README.md (the 3-layer thread model, the deadlock mechanics, the three failure modes). Scope: what native wasm-EH does to pthreads, and the test coverage that proves which patterns work.

Why this exists

KiCad-WASM uses native WebAssembly exceptions instead of Emscripten JS exceptions for the bundle-size win (pcbnew ~64.5 → ~36 MB gz). Native-EH is the default build — build-wx-wasm.sh, build-wasm-test.sh, and tests/apps/Makefile.wasm compile -fwasm-exceptions -sSUPPORT_LONGJMP=wasm -sWASM_LEGACY_EXCEPTIONS=1 (single-sourced from scripts/common/env.sh). It is the only build mode — the legacy -fexceptions path has been removed. CI builds native. The threading question this doc answers: which pthread patterns work under native-EH, what is the one exception-related risk it removes, and what is the (optional) upstreamable follow-up.

TL;DR

• Native-EH wx suite: green in Firefox + Chrome (316 / 1 skipped / 0 failed, chromium — matching JS-EH). WebKit is blocked for all pthread apps by a separate, pre-existing COEP worker-load limitation (§2a), so the pthread specs run FF + Chrome.
• The one native-EH-relevant risk is mode-c — a C++ exception thrown on a pthread worker. Under -fexceptions the throw drives Asyncify on the worker and crashes ("func is not a function"); native-EH lowers exceptions to native wasm instructions, so a throwing worker task is safe. Every threading pattern below is green under native-EH; the throwing ones are green only under native-EH.
• The real BS::thread_pool runs 16-core under native-EH, including a task that throws on a worker (threadpool-real, §6) — the decisive proof that the detach_task single-thread shim can be dropped.
• On-demand (non-warm) Worker creation works without editing KiCad (pthread-ondemand, §6): the nanosleep override (§2b) makes a main-thread sleep_for join Asyncify-yield, so the event loop services the new-Worker handshake. This is the threading-doc mode-(a) deadlock cure, in the wasm layer.
• A nested emscripten_sleep is legal (§3): code dispatched by a wx modal pump's ProcessEvents runs at Asyncify state == Normal, so a worker-join that yields via emscripten_sleep inside an open modal suspends-and-resumes normally (raytrace-modal, §6). The threading-doc mode-(b) does not arise for this case, so no JS-land scheduler ("Design B") is needed for it.
• The KiCad-10 std::async library preload is safe under native-EH (async-preload, §6/§7): the worker parses S-expr (a throw = mode-c) and proxies its async fetch to main; native-EH makes the parse safe and the lazy join keeps main free to service the proxy.
• The fork stays pristine. The pool's detach_task shim is the original, unmodified KiCad code; the tests un-shim it via a build-generated header (§6/§D), so the KiCad submodule carries no wasm-specific change. The later, optional upstreamable step is the raw-threads→pool refactor (§4).

---

1. Suite status under native-EH

The native-EH wx app suite is 316 / 1 skipped / 0 failed (chromium), matching JS-EH. Reaching it required two things: clearing a set of build-pipeline gaps that surfaced as native-EH test failures (committed this session, summarized below), and the §2 asyncify-imports fix for the raytracer cluster.

The build-pipeline gaps (all committed):

• Post-link Asyncify find too narrow — the loop matched only standalone/*/*_test.wasm, silently skipping apps/minimal_test.wasm and the coroutine-pthread repros / wxpt. Those linked but were never asyncify-instrumented → asyncify_start_unwind not found. Broadened to all freshly-linked app wasm.
• Repro apps mixed EH models — the coroutine-pthread *_repro apps hardcoded JS-EH in their link recipes while their compile inherited native-EH → undefined symbol: __cpp_exception. Made them EH-aware so they follow the default (native) and only carry -fexceptions under WX_LEGACY_EH.
• build-wasm-test.sh swallowed make failures — it continued to the post-link after a failed make, leaving apps half-instrumented (read as mass test failures). Now aborts loudly.

After those, the only real native-EH-specific signal was the raytracer threading cluster, fixed in §2.

---

2. The asyncify-imports fix (emscripten_sleep)

coroutine-raytrace.spec.ts aborted with Aborted(invalid state: 1). The mechanism:

• invalid state: 1 is Asyncify.handleSleep aborting because the state is Unwinding — a second suspend starting before the first rewinds. Logging every handleSleep, the state sequence at the abort is exactly 0,1: two emscripten_sleeps back-to-back with no rewind between. So a function calls emscripten_sleep, the unwind arms (state→Unwinding), and the same function calls emscripten_sleep again before returning. A correctly Asyncify-instrumented function has a post-call "if Unwinding, save locals and return" check after every suspend point; this one doesn't → Asyncify never instrumented it.
• main() is not re-entered (a [MAINCALL] probe fired exactly once; the abort stack only shows main's frames because Asyncify's unwind/rewind runs inside a setTimeout-driven doRewind that keeps the JS stack live).
• Why un-instrumented: binaryen's Asyncify instruments only functions that can reach a listed async import. The post-link list was curated for the wx apps, which yield via fibers (startModal, js_*, invoke_*, __asyncjs__*, emscripten_fiber_swap). It omitted emscripten_sleep, which the raytracer yields via. env.emscripten_sleep is a wasm import, so binaryen can match it — it just wasn't told to.

The exact JS-EH ↔ native-EH difference

Under JS-EH, Asyncify runs in-link and emcc auto-adds emscripten_sleep (+ idb_*/wget/scan_registers/lazy_load) to the imports. Under native-EH we run Asyncify post-link by hand, with an explicit list that dropped those auto-imports. That is the entire difference — not a fundamental native-EH × pthread incompatibility, and not a handleSleep-vs-arbiter question (the currData shim was never involved).

The fix

Added env.emscripten_sleep (+ scan_registers, lazy_load_code, wget, wget_data, idb_*) to the post-link asyncify-imports — now the shared scripts/common/asyncify-imports.txt, consumed by the unified apply-asyncify.sh that both the wx-test and KiCad builds call (the two near-duplicate scripts were folded into one; the old hoist-and-asyncify.sh is gone). So the KiCad list gets emscripten_sleep too, pre-empting the identical latent bug when its threading is un-shimmed.

Check	Result
Full wx suite, Chromium	316 / 1 skipped / 0 failed
coroutine-raytrace.spec.ts — all 6 (B1/B2/B1-local/B3 + speedup + A neg-control)	6/6 pass
multi-core speedup	serial 1342 ms → parallel 142 ms = 9.45× on 16 cores
raytrace #m=5 default (drains pool → on-demand creation) / #m=1, Chromium + Firefox	SUCCESS, workersRan=16

The default m=5 — which drains the pre-warmed pool and forces on-demand Worker creation — succeeds, so the fix also resolves the threading-doc mode-(a) deadlock: the sleep_for join now yields via an instrumented emscripten_sleep instead of busy-spinning and starving the worker handshake.

2a. The WebKit issue (separate, pre-existing)

In WebKit the asyncify side runs (threads spawn) but the pthread worker .js load is refused on COEP (Refused to load worker because of Cross-Origin-Embedder-Policy) even with COOP + COEP + CORP all served and crossOriginIsolated:true. It is a WebKit/playwright-headless COEP-worker strictness issue affecting all pthread apps, unrelated to EH. Tracked separately; the pthread specs run FF + Chrome only.

2b. The nanosleep override (the on-demand cure)

wasm/shims/nanosleep_yield.c is a strong nanosleep definition that shadows musl's archive member (-Wl,--wrap=nanosleep is not usable — it segfaults wasm-ld in lld::wasm::ImportSection::addImport). On the main thread it yields via an EM_ASYNC_JS await (= emscripten_sleep semantics, already in the post-link asyncify-imports); on a worker it stays a real blocking emscripten_thread_sleep. So an unmodified KiCad sleep_for join on the main thread pumps the event loop instead of busy-spinning, which lets on-demand Worker creation complete with no KiCad edit (§6 pthread-ondemand).

---

3. Patterns that work — pool-vs-raw, and the nested-sleep case

Two earlier wx apps, plus the §6 additions:

App / test	Thread pattern	native-EH
threadpool_test.cpp (threadpool.spec.ts)	create hwc std::threads into the pre-warmed pool, short body, join() each	PASS
raytrace_threads_test.cpp (coroutine-raytrace.spec.ts)	raw detached/persistent std::thread, sleep/busy-wait join, default drains the pool → on-demand creation	PASS (after §2)

Both raw and pool patterns work under native-EH; raw threads are not fundamentally broken. The threadpool create-and-join() never calls emscripten_sleep, so it never tripped the missing import; the raytracer yields via emscripten_sleep, so it did — which §2 closed.

The nested-sleep case (mode-b is not a live blocker). A worker-join that yields via emscripten_sleep inside an open ShowModal dialog is legal. The modal pump runs ProcessEvents via ccall(async:true), so work it dispatches runs in a fresh managed Asyncify entry at state == Normal — not nested inside an already-Unwinding frame. So the inner emscripten_sleep suspends-and-resumes normally. raytrace-modal (§6) probes and logs Asyncify.state == 0 to confirm this; the threading-doc mode-(b) "Asyncify can't nest" only bites a genuine second unwind, which the modal pump does not produce. No JS-land scheduler ("Design B") is required for the "render inside a modal" case.

---

4. Optional follow-up (upstreamable): KiCad raw-threads → the pool

This is a later, optional cleanup — not required, since the wasm layer (native-EH + the nanosleep override) already makes the threading patterns work on pristine KiCad. Per ../threading §8, upstream KiCad has migrated only 1 of 7 raytracer parallel sections to GetKiCadThreadPool() (renderTracing, and that one accidentally); the other six are untouched 2018 OpenMP-translation raw-thread code:

Site	Pass	Today
render_3d_raytrace_base.cpp:764 shadeWorker	post-process shading	raw std::thread + busy-wait, #ifdef'd serial in WASM
render_3d_raytrace_base.cpp:835 blurWorker	blur/finish	same
render_3d_raytrace_base.cpp:1456 previewWorker	preview	same
image.cpp:525 filterWorker	EfxFilter AA/blur	same
create_layer_items.cpp:848 zoneWorker	zone-fill geometry	same
create_layer_items.cpp:1311 simplifyWorker	polygon simplify	same

The refactor = migrate these six to submit_task() + multi_future::wait() (the renderTracing shape, refined by upstream bccf36538 to wait on own tasks only), and delete the #ifdef __EMSCRIPTEN__ serial fallbacks. Why it is the right eventual move:

• Upstreamable, not a wasm hack — precedent in the same file, a filed upstream issue (GitLab #20911, "ray tracing high system load"), and it removes dead OpenMP-era code. If accepted upstream, our fork carries zero divergence here.
• Less divergence, not more — it lets us drop the raytracer #ifdefs; combined with native-EH letting us drop the detach_task shim, net fork divergence goes down while threads come on.

Its prerequisite — that the real BS::thread_pool (persistent workers + submit_task + multi_future::wait()) survives native-EH — is proven by §6 threadpool-real (16-core, including a throwing worker task). So the refactor is de-risked; it is scheduled after the EH port's suite is otherwise green, and remains optional because the wasm-layer fixes already deliver multi-core.

---

5. The 3D viewer

The 3D viewer is live and single-threaded: the raytracer's six raw-thread passes are #ifdef'd to serial fallbacks in WASM, which is what ships. A separate multi-threaded spike exists (the WASM_RAYTRACE_POOL work, ~6–7×) but is not the active path.

Two zero-KiCad-edit routes turn the live viewer multi-threaded:

• The nanosleep override (§2b) makes the existing sleep_for joins yield, so the raw-thread passes run multi-core without on-demand-creation deadlock and without main-thread jank — no KiCad change.
• The §4 pool refactor is the upstream-clean alternative: pool-based, drops the #ifdefs, and carries zero fork divergence if accepted upstream.

---

6. pthread test coverage

All four apps below compile the real KiCad thread-pool source and run on pristine KiCad/wx-core. The specs are named coroutine-* so playwright-coroutine.config.ts runs them in Firefox + Chrome (WebKit excluded — §2a).

Spec	App	What it proves	native-EH
coroutine-threadpool-real.spec.ts	threadpool-real	the real GetKiCadThreadPool() in every mode — submit / loop / blocks / detach / fanout / lifecycle, and a task that throws on a worker — 16-core, throw caught	PASS (throw mode green only under native-EH = mode-c)
coroutine-pthread-ondemand.spec.ts	pthread-ondemand	real pool drains the pre-warmed Workers, then raw fly-threads force on-demand creation; the nanosleep override yields the join → on-demand Workers boot → multi-core (control: a non-yielding busy-wait deadlocks)	PASS
coroutine-raytrace-modal.spec.ts	raytrace-modal	a worker-join run inside an open ShowModal — both a busy-wait join and an emscripten_sleep yield-join complete multi-core; the app probes Asyncify.state == 0 to show the modal pump dispatches at Normal	PASS (mode-b does not arise)
coroutine-async-preload.spec.ts	async-preload	the KiCad-10 std::async library-preload shape: a worker parses S-expr (throws = mode-c) and proxies its fetch to main via emscripten_proxy_sync_with_ctx; modes simple / throw / shutdown / modal-during-preload	PASS (mode-c safe; 36 proxy round-trips through a modal, no crash)

These also cover the older coroutine-pthread.spec.ts (fiber + pthread across activation paths) and threadpool.spec.ts (raw create+join), both green. Together they exercise: the real pool API, raw create+join, raw detached/persistent + sleep/busy-wait, on-demand creation, a worker-side throw, a nested yield inside a modal, and a proxied async fetch off a worker — the full set of shapes the KiCad threading uses.

D. How the tests un-shim the pool without editing KiCad

KiCad's bs_thread_pool.hpp keeps its original #ifdef __EMSCRIPTEN__ detach_task shim (which runs pool tasks inline → single-threaded). To exercise the real pool, the test build generates an un-shimmed copy: tests/apps/Makefile.wasm's POOL_UNSHIMMED rule seds #ifdef __EMSCRIPTEN__ → #if 0 into standalone/_pool_unshimmed/bs_thread_pool.hpp (gitignored) and -I's it ahead of the KiCad header. So the KiCad submodule stays pristine; only the test compile sees the un-shimmed pool.

---

7. Library preload: std::async + the PCBJAM proxy under native-EH (the KiCad-10 bump)

The path that turns the native-EH migration from a size win into a prerequisite for tracking upstream. Verified by async-preload (§6): works under native-EH.

What changed upstream. KiCad 10 (d8ae50a667, 2026-06-08, fixes GitLab #23872) (a) added an eager library preload on board open — if( Kiface().IsSingle() ) Kiface().PreloadLibraries() in pcbnew/files.cpp (OpenProjectFiles), and IsSingle() is exactly our standalone-webapp case; and (b) changed IFACE::PreloadLibraries's dispatch from tp.submit_task( preload ) (our base, pcbnew/pcbnew.cpp:666) to std::async( std::launch::async, preload ) (KiCad-10 pcbnew.cpp ~1121). std::async spawns a real pthread worker that the detach_task pool shim does not cover — the shim only neutralizes the pool.

The plugin gotcha (don't be fooled by the upstream loader). Upstream library reads are synchronous (KICAD_SEXPR plugin → fopen/FILE_LINE_READER). Our fork is not on that path. The webapp writes the lib-table rows as (type "PCBJAM") / (type "PCBJAM_FP") (web/standalone/src/wasm/libs/source.ts:124,143), so the runtime plugin is our custom async bridge (kicad/eeschema/sch_io/pcbjam_lib/sch_io_pcbjam_lib.cpp, kicad/pcbnew/pcb_io/pcbjam_fp/pcb_io_pcbjam_fp.cpp). A surface read of the upstream loader will wrongly conclude "pure sync, safe" — verify by the lib-table row type, not the generic plugin. The PCBJAM dispatch is dual-path (sch_io_pcbjam_lib.cpp:158):

if( emscripten_is_main_runtime_thread() )
    return pcbjam_libs_request_js(...);           // main: EM_ASYNC_JS → Asyncify suspend (works on main)
std::lock_guard lk( g_pcbjamProxyMutex );         // worker: serialize, then
emscripten_proxy_sync_with_ctx( queue, main, … ); // proxy the fetch to MAIN + futex-block the worker

The two-level architecture (and how the pool shim warps it).

• Outer: std::async(preload) = one real background worker running a watchdog loop (sleep_for(150ms) + poll AsyncLoadProgress()). Bypasses the shim.
• Inner: adapter->AsyncLoad() (FOOTPRINT_LIBRARY_ADAPTER) submit_tasks N enumerate jobs to the pool → caught by the shim → run inline → so in our fork they execute serially on the outer worker. (Inner parallelism returns once the shim is dropped — also native-EH-gated.)

What runs where, on the std::async worker:

Step	Suspends Asyncify on the worker?
sleep_for(150ms) watchdog	No — real worker sleep (nanosleep/Atomics.wait), not emscripten_sleep.
PCBJAM fetch of library bytes	No — proxied to main + futex-block; the EM_ASYNC_JS runs on main.
S-expr parse of the bytes (throws IO_ERROR)	Yes under -fexceptions → mode-c crash. No under native-EH.
modals / clipboard / fonts	Not reachable from non-UI parsing.

The join is lazy — which defuses the deadlock. There is no eager .get(): CancelPreload(true) calls m_libraryPreloadReturn.wait() but has no callers; ProjectChanged() only sets the abort flag; the std::async future's blocking destructor fires only on IFACE teardown (shutdown, main thread); and re-entry is guarded by m_libraryPreloadInProgress (so the future is never reassigned mid-flight). So in normal operation main never blocks on the preload future → it stays in its event loop → it services the PCBJAM proxy queue → the worker's fetches complete. No normal-operation deadlock.

Verified. async-preload (§6) runs this shape under native-EH: the worker parse throws and is caught (no mode-c crash), the proxy round-trips, and a modal opened during preload survives 36 proxy round-trips with no crash (the g_pcbjamProxyMutex / "table index out of bounds" reentrancy hazard does not fire). So the KiCad-10 bump can keep std::async as-is under native-EH — it does not need a fork patch reverting to tp.submit_task (which would make preload block board-open on the main thread). Residual: a real-shutdown ordering check (the blocking destructor while a load is in flight) is covered by the async-preload shutdown mode but not yet under a live IFACE teardown.

Contrast with the raytracer (§4): the raytracer's raw threads are legacy OpenMP-era and upstreamable to the pool. This std::async is a deliberate upstream choice (a dedicated preload thread, off the compute pool), so "upstream it to the pool" is not the play — native-EH is.

---

8. Next steps (ordered)

1. DONE — coroutine-raytrace root-caused and fixed (§2): the post-link asyncify-imports list omitted emscripten_sleep. Suite 316/0; raytracer multi-core (9.45×).
2. DONE — pthread coverage closed (§6): the real-pool, on-demand, modal-nested, and std::async library-preload shapes are all green under native-EH on pristine KiCad/wx-core.
3. Drop the detach_task shim for real — threadpool-real proves the pool survives native-EH, so the next concrete step is enabling the un-shimmed pool in a KiCad build (DRC / zone-fill / connectivity on real Workers) and validating the docker build (the shared asyncify-imports change is untested there).
4. Resolve the §2a WebKit COEP worker-load limitation for pthread apps (currently the reason the pthread specs skip WebKit).
5. Optional, later — the §4 refactor: migrate the six raw-thread raytracer sections to the pool, delete the #ifdefs, upstream it.
6. Track-only: PROXY_TO_PTHREAD (DOM-bound GUI can't leave the main thread) and JSPI (incompatible with our main-loop architecture) — see ../threading §6–7.

Cross-references

• ../threading/README.md — the 3-layer model, deadlock mechanics, three failure modes, the full raw-thread inventory, and the upstream pool-migration analysis.
• ../async/11-asyncify-nesting-raytracer.md, ../async/12, ../async/13 — Asyncify nesting + the Design B scheduler (not required for the modal-pump case, §3).
• Apps + specs: tests/apps/standalone/{threadpool-real,pthread-ondemand,raytrace-modal,async-preload,coroutine-pthread,threadpool,raytrace-threads}/, tests/e2e/coroutine-{threadpool-real,pthread-ondemand,raytrace-modal,async-preload,pthread,raytrace}.spec.ts, tests/e2e/threadpool.spec.ts; the pool un-shim in tests/apps/Makefile.wasm (POOL_UNSHIMMED), the on-demand cure in wasm/shims/nanosleep_yield.c.
• Library preload (§7): kicad/pcbnew/pcbnew.cpp:593 (PreloadLibraries), kicad/eeschema/sch_io/pcbjam_lib/sch_io_pcbjam_lib.cpp + kicad/pcbnew/pcb_io/pcbjam_fp/pcb_io_pcbjam_fp.cpp (the async PCBJAM IO plugins), web/standalone/src/wasm/libs/source.ts (lib-table rows typed PCBJAM/PCBJAM_FP); upstream KiCad-10 std::async change d8ae50a667 (GitLab #23872).