9c475a804e
The runtime is JSPI-only; this removes everything that still pretended otherwise. Three exhaustive sweeps (C++/JS+build+CI/tests+docs) drove the inventory; every deletion verified by grep closure + full gates. Broken-right-now fixes: - deploy-staging.yml passed the retired opt_level input — the workflow could not even start. Removed. - env.sh carried dead exports with a live -sASYNCIFY=1 inside (WASM_LDFLAGS/PTHREAD_LDFLAGS, zero consumers). Removed; the WASM_LEGACY_EXCEPTIONS rationale rewritten to the real reason. - docker/build.sh exported PCBJAM_ASYNC_BACKEND (read nowhere). Gone. Dead weight removed: - binaryen submodule (nothing builds or invokes it), wasm-opt-bench workflow + scripts/bench/, get-wasm-opt.sh, diagnostics.js (242 lines of Asyncify-API-only code), the KICAD_PIPELINE background-postprocess scaffolding (existed to parallelize the deleted wasm-opt phase; the postprocess is a seconds-long node script and now runs inline), build-monitor's dead asyncify rows, sched-context orphan build output, dead .gitignore entries, the .jspi-assets spike dir (the two wf-result research JSONs moved to docs/features/async/migration-evidence/). - bindings: fiber_park.h + its 12 embind registrations (broken-if- called under JSPI), the kicadOpenFileStart/OPEN_JOB starter route, main_stack_runner.h + 5 includes, the always-null context-sleep weak hook in nanosleep_yield.c. - shim: the backend field (installed-flag idempotency instead), noteContextWait (dead both sides), the __wxAsyncifyDump alias (+ the WasmTool fallback and string-dump normalize branch). - web: the emscripten-6-ignored mainScriptUrlOrBlob option in boot.ts (gerber-demo keeps it: it loads the deployed CDN release, which predates emscripten 6 — noted inline). Conditionals: all 'backend === jspi' checks reduced to scheduler- presence checks; races_quiescent re-keyed from Asyncify.state (vacuous) to real backlog quiescence (resumeReady/mutatorQueue — NOT _windowLive, which is the probing activation's own window by definition). Renames (identifiers only, no file renames): ASYNC_LINK_FLAGS→ JSPI_LINK_FLAGS and Makefile ASYNC_LDFLAGS→JSPI_LDFLAGS, kicadCollabFiberBusy→kicadCollabBusy (embind + web + tests), collab_common.h fiber*→apply*/coroutine naming, asyncifySignatures→ wasmTrapSignatures (lists byte-identical). Tests: the two remaining vacuous [wx-asyncify]/fiber-resume-refused asserts re-keyed to live JSPI beacons; eeschema-load's failure message no longer sends the developer to a deleted script; wait-beacons' dead families/parser deleted; lane-0 legacy-glue guards removed (lane 0 is unconstructible); the embind test.fail re-gated with the JSPI reason (plain embind invokers cannot suspend — verified still failing); lint-determinism now scans tests/jspi (166 files clean); eeschema-collab local-move gated to chromium (~50% flaky on FF even solo; pcbnew twin covers both engines). Docs: DEBUG.md rewritten as the JSPI debugging guide; build.md describes the single-phase build; docs/features/async/README.md banner-marked historical and repointed at the NEW 23-jspi-runtime.md (current architecture: export census, turnstile, libcontext ownership + refusal contract, embind call shapes, the em-pthread service-wrapper trick, exception policy, known gaps). Gates on the cleaned tree: test:e2e 725 passed / 0 failed (after the quiescence-probe fix; the 3 other reds were verified contention flakes solo-green or the documented FF gate), web 76/0, jspi 18/18 both engines, vitest 295/295 + 17/17, all lints green, live-app census clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016X9eh1s5sTx1o9Em9KBuwR
103 lines
10 KiB
Markdown
103 lines
10 KiB
Markdown
# Asyncify `currData` contention in KiCad-WASM — research dossier
|
||
|
||
> **STATUS (2026-08-14): HISTORICAL.** This directory is the Asyncify-era
|
||
> investigation log. The JSPI migration (2026-08) superseded the TL;DR
|
||
> conclusion below ("we are not switching to JSPI"). Current architecture:
|
||
> [23-jspi-runtime.md](23-jspi-runtime.md).
|
||
|
||
> **Status:** research / understanding only. No implementation has been chosen.
|
||
> Authored 2026-06. All line numbers are against the artifacts current at that time
|
||
> (`tests/apps/kicad/pcbnew.js`, `wxwidgets/src/wasm/*.cpp`,
|
||
> `kicad/thirdparty/libcontext/libcontext.cpp`, `wxwidgets/src/common/init.cpp`).
|
||
|
||
## Why this exists
|
||
|
||
While bringing up `tests/kicad/load-pcb.spec.ts` (load a real `.kicad_pcb` through
|
||
File→Open), three distinct errors surfaced. One is fixed; two remain and turned out to be
|
||
**the same underlying disease**: Emscripten Asyncify has a single global suspension register
|
||
(`Asyncify.currData` + `Asyncify.state`), but KiCad-WASM has **three independent subsystems**
|
||
that all drive it (tool coroutines, modal/clipboard sleeps, and the parked main loop). When any
|
||
two overlap, one reads a buffer that no longer belongs to it → **crash** (`index out of bounds`)
|
||
or **hang** (a swap unwinds but is never rewound).
|
||
|
||
## TL;DR
|
||
|
||
- **The single slot is not a WebAssembly/Binaryen law — it's a choice in Emscripten's JS
|
||
runtime.** The Binaryen Asyncify pass is multi-buffer by design; `asyncify_start_unwind`/
|
||
`asyncify_start_rewind` take the buffer pointer as an argument.
|
||
- **"Give each context its own buffer" is the standard, supported solution** — it's literally
|
||
what Emscripten *fibers* are, and what QEMU/TinyGo/Pyodide do. We already do it for tool
|
||
coroutines (each `wasm_fcontext` owns a buffer). The bug is that the **fiber path and the
|
||
`handleSleep` path both blindly overwrite the one global `currData` register.**
|
||
- **We are not switching to JSPI.** The fix stays within Asyncify, in the wasm/shim layer.
|
||
- **The achievable universal fix** is a single cooperative scheduler that owns `currData` (and
|
||
the fiber trampoline) and treats every suspendable thing — coroutines, modals, clipboard,
|
||
fonts, and the main loop itself — as a registered context with its own buffer.
|
||
|
||
## Document index
|
||
|
||
| File | Contents |
|
||
|---|---|
|
||
| [`01-background-and-findings.md`](01-background-and-findings.md) | The originating session, the e2e test + logs, and the three concrete bugs (rtree=fixed; clipboard crash; tool-open hang). |
|
||
| [`02-asyncify-internals.md`](02-asyncify-internals.md) | The machine, in detail: what "sleeps", the single `currData`/`state`, the three producers, `handleSleep`/`fiber_swap`/trampoline, the #9153 shim, and full control-flow walkthroughs (startup→park, the hang, the crash, **de-parking** down to the JS line). |
|
||
| [`03-solutions-and-prior-art.md`](03-solutions-and-prior-art.md) | Is there a real solution? Per-context buffers, who has done it, why we're not switching to JSPI, and the unified-authority recipe with failure modes. Sources/URLs included. |
|
||
| [`04-decisions-tests-open-questions.md`](04-decisions-tests-open-questions.md) | How the fix options relate (what's subsumed vs. genuinely separate), the one diagnostic that decides scope, the combinatorial test matrix, and open questions. |
|
||
| [`05-design-a-js-asyncify-arbiter.md`](05-design-a-js-asyncify-arbiter.md) | Incremental design: keep current `EM_ASYNC_JS` sleeps and fibers, but put one JS arbiter in charge of `currData`, transition queueing, and the trampoline. Includes concept explanations. |
|
||
| [`06-design-b-fiber-first-runtime.md`](06-design-b-fiber-first-runtime.md) | Cleaner long-term design: make modals, clipboard, fonts, nested loops, and tools all scheduler-owned fiber-like contexts. Explains how this relates to de-parking and app lifetime. |
|
||
| [`07-decisions-and-outcome.md`](07-decisions-and-outcome.md) | **What was decided and shipped (2026-06-12):** root cause, red-green ledger, the arbiter NOT built and why, roads not taken with revisit triggers. |
|
||
| [`08-dom-port-regression.md`](08-dom-port-regression.md) | DOM-port regression investigation after rebasing onto the async hardening: traces, symbolized crash, ruled-out Asyncify/table/removelist hypotheses, and current stale-window diagnosis. |
|
||
| [`09-dom-window-lifetime-hypothesis.md`](09-dom-window-lifetime-hypothesis.md) | Concrete failure story and first fix experiment for the DOM-port stale `wxWindow` hypothesis: destructor ordering, DOM event reentry, and validation plan. |
|
||
| [`10-resolution-menubar-uaf.md`](10-resolution-menubar-uaf.md) | **RESOLVED:** the regression was a freed `wxMenuBar` left in a live frame's child list by `wxMenuBarBase::Detach()` (DOM-port only — the bar is a real child there). One-line fix in `wxMenuBar::Detach()`; full kicad suite green, zero corruption signatures. |
|
||
| [`11-asyncify-nesting-raytracer.md`](11-asyncify-nesting-raytracer.md) | **Finding + decision:** the WASM 3D raytracer is single-core because `emscripten_sleep` can't nest on an already-unwinding Asyncify context — yielding to join worker threads aborts with `invalid state: 1` since the viewer renders inside a suspended wx modal pump. Multi-core pool (~6–7×) built + parked; unpark needs a *nestable* yield (fiber/JSPI). |
|
||
| [`19-quasimodal-fiber-strand.md`](19-quasimodal-fiber-strand.md) | **FIXED 2026-08-06:** the Symbol Properties dialog hang — a tool fiber parked mid-body, the stale-fiber guard refused (and dropped) its resume, the dispatch interlock was never released. Cure: a quasi-modal's nested loop no longer parks on a coroutine stack. Regression pin: `tests/kicad/quasimodal-strand.spec.ts`. Note this removed one INGREDIENT, not the blue-screen cause. |
|
||
| [`20-design-b-core-plan.md`](20-design-b-core-plan.md) | **PARTIALLY DELIVERED 2026-08:** Design B core. D-1 (legacy runtime deleted), D0 (park-site audit + red spec) and D1 (context primitives + memory gate) landed; D2 (dispatch contexts) was reverted; D3 met its goal by other means. Read §10's work log for what each phase actually cost. **Superseded for the remaining work by 22.** |
|
||
| [`21-park-site-audit.md`](21-park-site-audit.md) | **AUDIT 2026-08:** every Asyncify park site classified by whose stack it suspends (tool fiber / entry stack / main loop) with its routing phase — 14 production sites + 3 test levers. Settles the pthread question: all parks are main-thread; the lib bridge's worker path is a blocking proxy, not a park. |
|
||
| [`22-absorbing-libcontext.md`](22-absorbing-libcontext.md) | **PLAN (2026-08-06), the current one:** absorb libcontext's wasm backend into the scheduler so ONE handler owns every js↔asyncify↔fiber switch — the cure for the blue screen (a context recovered twice or by the wrong fiber). Diagnosis of why three guard layers cannot fix it, why the D2/D3 phases knotted, phases A–F with estimates (~4–6 wk), gates, and the traps this implementation run paid for. **Start here.** |
|
||
| [`17-mailbox-scheduler-plan.md`](17-mailbox-scheduler-plan.md) | **PLAN (2026-08):** the mailbox/scheduler implementation plan — Design B's phasing revised with the July–August guard record (dispatch interlock, open-settle gate, v0.1.28 schedule-don't-dispatch). Test inventory with per-test fate (keep / rewrite / retire / new), 7 steps S0–S6 with gates and rollback, ≈5–7 wk. Supersedes 12/13's phasing; overturns 13 §6f's "no scheduler needed". |
|
||
|
||
## Where to start (2026-08-14)
|
||
|
||
**Read [`23-jspi-runtime.md`](23-jspi-runtime.md)** — the current (JSPI)
|
||
architecture. The plan that got there is [`22-absorbing-libcontext.md`](22-absorbing-libcontext.md),
|
||
with prerequisites [`20`](20-design-b-core-plan.md) §10 (what each phase actually cost) and
|
||
[`21`](21-park-site-audit.md) (the migration surface).
|
||
|
||
<details>
|
||
<summary>The original "single decisive next step" (2026-06, answered)</summary>
|
||
|
||
Before designing anything, **measure whether `Asyncify.currData` is clean (null) at the moment
|
||
`wxGUIEventLoop::DoRun()` throws `"unwind"`** (and at the first rAF tick, and at the first
|
||
post-startup `emscripten_fiber_swap`). That one fact determines whether the universal fix must
|
||
also reshape the main loop ("de-parking") or whether a per-context `currData` authority alone
|
||
suffices. Details in [`04-decisions-tests-open-questions.md`](04-decisions-tests-open-questions.md).
|
||
|
||
</details>
|
||
|
||
---
|
||
|
||
## RESOLUTION (2026-06-12) — see [`07-decisions-and-outcome.md`](07-decisions-and-outcome.md) and `docs/features/asyncify-arbiter/`
|
||
|
||
The decisive measurement was answered **by code trace and then pinned by a deterministic
|
||
test** (`tests/asyncify/asyncify-races.spec.ts` + `tests/apps/standalone/asyncify-races/`):
|
||
|
||
- At the park throw, Asyncify state IS clean (`currData==null`, `state==Normal`) — **but the
|
||
JS stack is necessarily still inside `Fibers.trampoline()`'s `do/while`** (any OnInit-era
|
||
fiber swap means main is trampoline-resumed from then on). The throw skips the
|
||
`trampolineRunning = false` reset → the guard wedges → the first post-idle swap hangs.
|
||
That IS the §5 hang; "orphaned currData" (mechanism #1) is structurally impossible.
|
||
The self-heal (`inject-dyncall-shims.sh` §3c, commit `18a9de0`) is therefore the
|
||
*structural cure*, not a band-aid — **no de-parking needed**.
|
||
- When main's last pre-park suspension is a *sleep*, the same throw instead escapes through
|
||
the sleep's wakeUp promise reaction → the long-mystifying `uncaught exception: unwind`
|
||
rejections. Fixed in `scripts/common/shims/handlesleep.js` (catches the sentinel like
|
||
`callMain` does).
|
||
- The full Design-A arbiter was NOT needed: at production semantics (`-sASSERTIONS=0`),
|
||
out-of-order and overlapping-sleep scenarios are already handled by the per-sleep buffer
|
||
capture in `handlesleep.js`. The remaining bugs were wx-layer: single-slot
|
||
`Module._endModal` broke 3-deep nested modals (now a LIFO resolver stack), and the
|
||
modal/nested-loop pumps stalled silently on ProcessEvents rejection (now resolve-on-error).
|
||
Clipboard `IsSupported` no longer runs the 2 s async probe.
|
||
- De-parking (02 §7) and Design B remain documented options, unneeded for correctness today.
|
||
- Upstream status (researched): the Fibers/Asyncify code is unchanged since 2020; the
|
||
single-slot family is WONTFIX (#9153, #12270, #13302, #16291, #18412). The trampoline
|
||
try/finally would be a good upstream PR.
|