jspi cleanup: remove the asyncify-era residue — dead code, conditionals, pipeline scaffolding, stale prose
The runtime is JSPI-only; this removes everything that still pretended otherwise. Three exhaustive sweeps (C++/JS+build+CI/tests+docs) drove the inventory; every deletion verified by grep closure + full gates. Broken-right-now fixes: - deploy-staging.yml passed the retired opt_level input — the workflow could not even start. Removed. - env.sh carried dead exports with a live -sASYNCIFY=1 inside (WASM_LDFLAGS/PTHREAD_LDFLAGS, zero consumers). Removed; the WASM_LEGACY_EXCEPTIONS rationale rewritten to the real reason. - docker/build.sh exported PCBJAM_ASYNC_BACKEND (read nowhere). Gone. Dead weight removed: - binaryen submodule (nothing builds or invokes it), wasm-opt-bench workflow + scripts/bench/, get-wasm-opt.sh, diagnostics.js (242 lines of Asyncify-API-only code), the KICAD_PIPELINE background-postprocess scaffolding (existed to parallelize the deleted wasm-opt phase; the postprocess is a seconds-long node script and now runs inline), build-monitor's dead asyncify rows, sched-context orphan build output, dead .gitignore entries, the .jspi-assets spike dir (the two wf-result research JSONs moved to docs/features/async/migration-evidence/). - bindings: fiber_park.h + its 12 embind registrations (broken-if- called under JSPI), the kicadOpenFileStart/OPEN_JOB starter route, main_stack_runner.h + 5 includes, the always-null context-sleep weak hook in nanosleep_yield.c. - shim: the backend field (installed-flag idempotency instead), noteContextWait (dead both sides), the __wxAsyncifyDump alias (+ the WasmTool fallback and string-dump normalize branch). - web: the emscripten-6-ignored mainScriptUrlOrBlob option in boot.ts (gerber-demo keeps it: it loads the deployed CDN release, which predates emscripten 6 — noted inline). Conditionals: all 'backend === jspi' checks reduced to scheduler- presence checks; races_quiescent re-keyed from Asyncify.state (vacuous) to real backlog quiescence (resumeReady/mutatorQueue — NOT _windowLive, which is the probing activation's own window by definition). Renames (identifiers only, no file renames): ASYNC_LINK_FLAGS→ JSPI_LINK_FLAGS and Makefile ASYNC_LDFLAGS→JSPI_LDFLAGS, kicadCollabFiberBusy→kicadCollabBusy (embind + web + tests), collab_common.h fiber*→apply*/coroutine naming, asyncifySignatures→ wasmTrapSignatures (lists byte-identical). Tests: the two remaining vacuous [wx-asyncify]/fiber-resume-refused asserts re-keyed to live JSPI beacons; eeschema-load's failure message no longer sends the developer to a deleted script; wait-beacons' dead families/parser deleted; lane-0 legacy-glue guards removed (lane 0 is unconstructible); the embind test.fail re-gated with the JSPI reason (plain embind invokers cannot suspend — verified still failing); lint-determinism now scans tests/jspi (166 files clean); eeschema-collab local-move gated to chromium (~50% flaky on FF even solo; pcbnew twin covers both engines). Docs: DEBUG.md rewritten as the JSPI debugging guide; build.md describes the single-phase build; docs/features/async/README.md banner-marked historical and repointed at the NEW 23-jspi-runtime.md (current architecture: export census, turnstile, libcontext ownership + refusal contract, embind call shapes, the em-pthread service-wrapper trick, exception policy, known gaps). Gates on the cleaned tree: test:e2e 725 passed / 0 failed (after the quiescence-probe fix; the 3 other reds were verified contention flakes solo-green or the documented FF gate), web 76/0, jspi 18/18 both engines, vitest 295/295 + 17/17, all lints green, live-app census clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016X9eh1s5sTx1o9Em9KBuwR
This commit is contained in:
Viktor Vaczi
parent
3ee174e9b4
commit
9c475a804e
120 changed files with 1522 additions and 2974 deletions
|
|
@ -1,294 +1,276 @@
|
|||
# Debugging guide — KiCad / wxWidgets WASM
|
||||
# Debugging guide — KiCad / wxWidgets WASM (JSPI runtime)
|
||||
|
||||
A practical reference for debugging this project: the kinds of issues WASM +
|
||||
Asyncify + browser builds throw at you, the tools that actually work here, and
|
||||
the gotchas of our specific build pipeline. It is **not** a writeup of any one
|
||||
bug — for a concrete worked example see [§6](#6-a-worked-example) and the
|
||||
project memory.
|
||||
A practical reference for debugging this project: how the JSPI runtime is
|
||||
wired, the observability surfaces built into it, and the recipes that actually
|
||||
work here. It is **not** a writeup of any one bug — the numbered docs under
|
||||
[`docs/features/async/`](../features/async/) carry those; the current
|
||||
architecture is [`23-jspi-runtime.md`](../features/async/23-jspi-runtime.md).
|
||||
|
||||
If you're new to this codebase, read [§5 (project gotchas)](#5-project-specific-gotchas)
|
||||
first — most wasted hours come from not knowing how the split build and the
|
||||
shim layer behave.
|
||||
If you're new to this codebase, read [§1](#1-the-runtime-in-one-paragraph) and
|
||||
then go straight to [§2 (observability)](#2-observability) — most questions of
|
||||
the form "why is nothing happening" are answered by one `__wxWaitDump()` call.
|
||||
|
||||
---
|
||||
|
||||
## 1. Classes of issue we hit here
|
||||
## 1. The runtime, in one paragraph
|
||||
|
||||
- **Engine-specific intolerance** — the same `pcbnew.wasm` runs in Firefox but
|
||||
not Chrome (or vice versa). Usually a V8-vs-SpiderMonkey difference in how an
|
||||
Asyncify-instrumented or very large function is handled.
|
||||
- **Silent stalls vs. hard crashes** — execution stops making progress with *no*
|
||||
exception, trap, or crash report. Distinguishing "crashed" from "hung" from
|
||||
"stalled" is half the battle (§2.6).
|
||||
- **Asyncify state problems** — unwind/rewind not completing, instrumentation on
|
||||
a function that shouldn't have it, or a function too large once instrumented.
|
||||
- **Shim/codegen coupling** — `inject-dyncall-shims.sh` patches Emscripten output
|
||||
by pattern; a flag change that alters codegen can silently break those patches.
|
||||
- **Tooling blind spots** — async console delivery, stripped name sections,
|
||||
Playwright hiding the renderer's stderr (§4).
|
||||
Suspension is JSPI: every wasm entry point that can park is a
|
||||
**promising export** (`-sJSPI` + the census in
|
||||
`scripts/common/jspi-exports.txt`), and every suspension awaits a real JS
|
||||
promise. One scheduler — `scripts/common/shims/jspi-scheduler.js`, shipped as
|
||||
a `--pre-js` — owns the discipline around that: it wraps the promising
|
||||
exports so it always knows which **activation** is executing or suspended,
|
||||
gives each activation its own **spill-stack region** (JSPI switches the
|
||||
native stack per activation but *not* the C spill stack — emscripten #27364),
|
||||
and serializes engine re-entries through a resume **turnstile** so only one
|
||||
activation's SP can be armed between wasm entries. KiCad tool coroutines run
|
||||
on the JSPI backend of `kicad/thirdparty/libcontext/` (one promising
|
||||
activation per coroutine, own region, promise-pair yield/resume) and
|
||||
integrate with the same turnstile. There is no post-link instrumentation, no
|
||||
unwind/rewind state machine, and no rewind buffer to corrupt: what used to be
|
||||
"asyncify state problems" are now ordinary promise/call-shape problems.
|
||||
|
||||
---
|
||||
|
||||
## 2. Tools & techniques
|
||||
## 2. Observability
|
||||
|
||||
### 2.1 Stub-bisection *(the workhorse)*
|
||||
Comment out / early-`return` a suspect call, rebuild, and observe a **binary
|
||||
survives-or-fails** outcome. This is the most reliable signal we have because it
|
||||
does **not** depend on reading logs (which lag — see §4). Narrow by halving:
|
||||
disable half the suspects, see which half flips the outcome.
|
||||
- *When:* you can localize a failure to "before/after some call."
|
||||
- *Caveat:* at `-O2`, dead-code elimination removes more around an early `return`
|
||||
than you intend — keep this in mind when a stub "fixes" too much.
|
||||
### 2.1 `__wxWaitDump()` — the first thing to run
|
||||
|
||||
### 2.2 `SHIM_DIAGNOSTICS=1` fast loop *(skip the rebuild)*
|
||||
The only host-side JS step is `inject-dyncall-shims.sh`. Re-run it on a pristine
|
||||
`pcbnew.js` while keeping the already-finalized/asyncified `pcbnew.wasm` — JS-only
|
||||
changes go from a multi-minute rebuild to seconds:
|
||||
```bash
|
||||
cp output/pcbnew.pristine.js output/pcbnew.js
|
||||
SHIM_DIAGNOSTICS=1 ./scripts/common/inject-dyncall-shims.sh output/pcbnew.js
|
||||
cd tests && npm run setup:kicad
|
||||
Available on any app page (and printed automatically by the SuspendError
|
||||
attributor). Returns one object:
|
||||
|
||||
| field | meaning |
|
||||
|---|---|
|
||||
| `dead` | scheduler shut down (teardown seen) |
|
||||
| `waitsBegun` / `waitsResolved` | token-wait registry totals (modal, nested, clipboard, lib-bridge …) |
|
||||
| `earlyWaitResolves` | waits resolved before their waiter parked (legal fast path) |
|
||||
| `pendingWaits` | unresolved registry entries right now |
|
||||
| `runningActivations` | promising exports currently on the JS stack |
|
||||
| `suspendedActivations` | array of `{id, kind, waitKind, token, suspendedMs}` — every parked activation |
|
||||
| `mutatorsWrapped` / `mutatorsDelivered` / `mutatorQueueDepth` | the embind mutator FIFO (queued while `kicadOpenFileBusy`) |
|
||||
| `ring` | the last 64 scheduler events (see §2.2) |
|
||||
|
||||
Reading it: a wedge usually shows up as an entry in `suspendedActivations`
|
||||
with a large `suspendedMs` and a `waitKind`/`token` that tells you *what* it
|
||||
is waiting for; cross-check `pendingWaits` and the ring. `id`s of the form
|
||||
`"lc<N>"` are libcontext coroutines; negative ids are untracked/anonymous
|
||||
suspensions (boot-time `main`, foreign yields).
|
||||
|
||||
### 2.2 Ring counting recipes
|
||||
|
||||
`__wxScheduler._ring` holds `[epochMs, event, a, b]` tuples (last 256; the
|
||||
dump slices the last 64). Useful counts:
|
||||
|
||||
```js
|
||||
// stale resumes refused by the doc-15 contract (a = 'lc<id>')
|
||||
__wxScheduler._ring.filter(e => e[1] === 'libctxRefusedResume').length
|
||||
|
||||
// turnstile self-heals — should be 0 in a healthy run
|
||||
__wxScheduler._ring.filter(e => e[1] === 'forceClearWindow')
|
||||
|
||||
// wakes dropped at quarantined (released-while-parked) coroutines
|
||||
__wxScheduler._ring.filter(e => e[1] === 'deadWakeDropped')
|
||||
|
||||
// park/resolve balance per wait kind
|
||||
__wxScheduler._ring.filter(e => e[1] === 'park').map(e => e[2])
|
||||
```
|
||||
See the `wasm-build-fast-iteration` project memory.
|
||||
|
||||
### 2.3 Logging-only diagnostics module (`scripts/common/shims/diagnostics.js`)
|
||||
Injected **only** when `SHIM_DIAGNOSTICS=1` (off by default, safe to leave in
|
||||
tree). Provides hooks that need no rebuild:
|
||||
- Asyncify lifecycle: `doRewind`, `handleSleep` (unwind/rewind markers).
|
||||
- Modal lifecycle.
|
||||
- A **WebGL call tracer** (did any GL call happen before the failure?).
|
||||
- A **dynCall tracer**: wraps the shim-bound `dynCall_ii`/`dynCall_vi` to log
|
||||
`ptr`, `getWasmTableEntry(ptr).name` (the function index), and a JS stack for
|
||||
rare/large table indices. Arm it at the main rewind to bound log volume.
|
||||
- Periodic asyncify-state monitor (catch "JS task queue stopped pumping").
|
||||
Other event names you will see: `beginWait`, `resolve`, `wrapped`,
|
||||
`libctxQuarantine`, `shutdown`.
|
||||
|
||||
Output is at `console.log` level (not error/warn). This is the JS-side tracer; the
|
||||
C++ source diagnostics are separate and flag-gated — see §2.9.
|
||||
### 2.3 `__libctxJspi` — the coroutine census
|
||||
|
||||
### 2.4 Symbolizing wasm function indices
|
||||
The loaded (post-asyncify) wasm has **no `name` section**, so V8/Firefox report
|
||||
bare function indices (`func[20736]`). The Asyncify pass **preserves function
|
||||
indices**, so a symbol map taken from the *pre-asyncify* wasm is still valid:
|
||||
```bash
|
||||
# the in-container wasm-opt is a STUB; use the real one
|
||||
/emsdk/upstream/bin/wasm-opt.real <pre-asyncify pcbnew.wasm> --symbolmap=/tmp/syms.map
|
||||
# then look up the index, e.g. 20736 -> PCB_EDIT_FRAME::setupUIConditions()
|
||||
The libcontext JSPI backend keeps its own JS-side census:
|
||||
|
||||
```js
|
||||
Object.keys(__libctxJspi.s).length // live coroutine slots (promise pairs)
|
||||
__libctxJspi.tops // id -> spill-region top (SP swap target)
|
||||
__libctxJspi.ghosts // ghost/refused transitions, ever
|
||||
__libctxJspi.deadParked // coroutines released while parked mid-body
|
||||
```
|
||||
Generate the map from a build that still has names (the debug build's
|
||||
pre-asyncify wasm). See §5 on names/DWARF.
|
||||
|
||||
### 2.5 Cross-engine comparison
|
||||
Run the **same** diagnostics build in Firefox and Chrome and compare state at the
|
||||
**same dispatch point** (e.g. asyncify `state`/`currData` at the suspect
|
||||
`dynCall`). If both reach a point with identical state but only one proceeds, you
|
||||
have isolated an engine-specific bug and can stop looking for a logic error.
|
||||
Records are tombstoned, never freed C-side, so stale handles / double
|
||||
releases / ghost resumes are refused loudly instead of corrupting anything —
|
||||
each refusal bumps `ghosts` and prints a beacon (§2.4).
|
||||
|
||||
### 2.4 Beacon vocabulary
|
||||
|
||||
Everything the runtime is unhappy about is announced on the console with a
|
||||
stable prefix. Test helpers count these (`tests/kicad/utils/wait-beacons.ts`);
|
||||
when debugging by hand, grep the captured console for the prefix.
|
||||
|
||||
| beacon | source | meaning |
|
||||
|---|---|---|
|
||||
| `[libctx-jspi] ghost/refused transition … reason=<r>` | `libcontext.cpp` | a refused coroutine transition; `reason` is one of `ghost-enter`, `yield-no-cur`, `released-while-parked`, `dead-cur-substituted`, `yield-to-dead-enterer`, `release-of-running-ignored` |
|
||||
| `[libctx-jspi] coroutine N entry REJECTED: <stack>` | `libcontext.cpp` | the coroutine's entry activation *rejected* (a trap inside the body); the enterer receives the refusal sentinel instead of hanging |
|
||||
| `[libctx-jspi] REGION OVERFLOW: coroutine N …` | `libcontext.cpp` | the spill-region base canary tripped — a tool body outgrew its region |
|
||||
| `[wx-scheduler] force-clearing stuck window …` | `jspi-scheduler.js` | turnstile self-heal: some suspension bypassed the shim (untracked raw await) and would otherwise block resumes forever |
|
||||
| `[wx-scheduler] job tick error: …` | `evtloop.cpp` | a scheduled-job handler threw; containment fired |
|
||||
| `[wx-scheduler] mailbox tick error: …` | `jspi-scheduler.js` | a delivered mailbox handler threw; `wx_dispatch_abandon` + top-wait resolution keep the app alive |
|
||||
| `[wx-scheduler] shutdown (<why>) clean` / `… stranded:N` | `jspi-scheduler.js` | teardown contract — a clean exit *says so*; `stranded` counts waits that never resolved (asserted by `e2e/app-quit.spec.ts`) |
|
||||
| `[wx-scheduler] SuspendError: …` | `jspi-scheduler.js` | see §3 — includes a full dump for targeting |
|
||||
| `[wx-scheduler] LOST WAKE: …` | `jspi-scheduler.js` | watchdog: an activation parked >30 s on a token wait that is no longer registered |
|
||||
| `[wx-scheduler] dropping wake for quarantined N` | `jspi-scheduler.js` | a late wake arrived for a released coroutine; refused (never re-enter a freed body) |
|
||||
| `[wx-timer] retry storm: N retries …` | `timer.cpp` | a timer's `Notify` kept retrying against a held dispatch interlock — something is parked across ticks |
|
||||
| `[wx-dispatch] ERASED … / NEGATIVE depth …` | `evtloop.cpp` | dispatch-interlock bookkeeping anomaly — depth accounting corrupt, report it |
|
||||
|
||||
A healthy run is beacon-silent apart from at most a `shutdown … clean`.
|
||||
|
||||
### 2.5 Tooling blind spots (read before trusting output)
|
||||
|
||||
- **Console is async** — `printf`/`console.*` reaches Playwright via CDP
|
||||
asynchronously; the *last delivered* line can lag the real failure point.
|
||||
Prefer state dumps (§2.1) and binary-outcome bisection over "the last log
|
||||
line".
|
||||
- **Playwright hides the renderer** — it forces `--disable-breakpad` and only
|
||||
pipes the *browser* process stderr. To see the renderer's own stderr and a
|
||||
real crash reason, serve `tests/apps` with the COOP/COEP headers
|
||||
(`tests/serve.json`, e.g. `npx serve apps -c ../serve.json`) and open the
|
||||
page in a normal Chrome with crash reporting on.
|
||||
- **macOS `sample`/`.ips`** see wasm frames as numeric offsets, not C++ names.
|
||||
|
||||
**Crash vs. hang vs. stall** — a failure with no exception is not necessarily
|
||||
a crash. Find the renderer PID and inspect it:
|
||||
|
||||
### 2.6 Crash vs. hang vs. stall
|
||||
A failure with no exception is not necessarily a crash. Find the renderer PID and
|
||||
inspect it:
|
||||
```bash
|
||||
ps -axo pid,%cpu,%mem,command | grep -i 'Google Chrome'
|
||||
sample <rendererPID> 3 # what is the main thread doing?
|
||||
```
|
||||
- **Idle in `CFRunLoop`/`mach_msg2_trap`, ~0% CPU** → a *stall* (event loop alive,
|
||||
but nothing scheduled to run). Not a deadlock.
|
||||
- **Blocked on a futex / `Atomics.wait`** → a pthread/lock issue.
|
||||
- **Spinning at 100%** → an infinite loop.
|
||||
- **Gone + a `.ips` report** → a real signal crash.
|
||||
|
||||
To see the **renderer's own stderr** and a real crash reason, launch system
|
||||
Chrome **outside Playwright** (Playwright forces `--disable-breakpad` and only
|
||||
pipes the *browser* process stderr): serve `tests/apps` with the COOP/COEP headers
|
||||
(`tests/serve.json`) and open the page in a normal Chrome with crash reporting on.
|
||||
On-load failures need no interaction to reproduce.
|
||||
- Idle in `CFRunLoop`/`mach_msg2_trap`, ~0% CPU → a *stall* (event loop
|
||||
alive, nothing scheduled). Under JSPI this almost always means a parked
|
||||
activation whose wake was lost or refused — go read `__wxWaitDump()` and
|
||||
the ring.
|
||||
- Blocked on a futex / `Atomics.wait` → a pthread/lock issue.
|
||||
- Spinning at 100% → an infinite loop.
|
||||
- Gone + a `.ips` report → a real signal crash.
|
||||
|
||||
### 2.7 Build-flag diagnostics
|
||||
- `-sASSERTIONS=2` turns silent UB into named errors. **But** it changes
|
||||
Emscripten codegen and can break `inject-dyncall-shims.sh`'s `sed` patterns
|
||||
(causing a *different*, red-herring failure), and it implicitly enables
|
||||
`STACK_OVERFLOW_CHECK`, whose `___set_stack_limits` our host Asyncify pass
|
||||
strips → pair it with `-sSTACK_OVERFLOW_CHECK=0`. Prefer the §2.3 dynCall
|
||||
tracer on a normal build when you can.
|
||||
- `--pass-arg=asyncify-asserts` (added to the `wasm-opt --asyncify` invocation in
|
||||
`apply-asyncify.sh`) adds Asyncify state-machine runtime checks — use it to
|
||||
validate the removelist (a wrongly-excluded function that *does* unwind is
|
||||
otherwise silent corruption).
|
||||
---
|
||||
|
||||
### 2.8 Isolated standalone probes
|
||||
`tests/apps/standalone/coroutine-pthread/` builds minimal C++ probes with the
|
||||
*real* libcontext + Asyncify + pthreads + DYNCALLS + the shim, run via
|
||||
`tests/e2e/coroutine-pthread.spec.ts`. Use these to reproduce a mechanism in
|
||||
isolation. **Reality check:** an isolated probe often *won't* reproduce a bug
|
||||
that needs the full app runtime — don't over-trust a green probe.
|
||||
## 3. Reading a SuspendError
|
||||
|
||||
Chromium: `RangeError: Trying to suspend without WebAssembly.promising` (or
|
||||
similar `Suspend…` wording). Firefox: `No matching WebAssembly.promising`.
|
||||
|
||||
Both mean the same **call-shape problem**: a *plain* (non-promising) entry
|
||||
into wasm reached a suspending import. The suspension has nowhere to go — a
|
||||
promising activation is created at the *export* boundary, not at the park
|
||||
site — so the engine throws at the park.
|
||||
|
||||
The scheduler's attributor catches these globally and prints
|
||||
`[wx-scheduler] SuspendError: …` with a full `__wxWaitDump()` — the engine
|
||||
cannot say *which* export was entered plainly, but the dump (what is wrapped,
|
||||
what was executing) is exactly the targeting data you need.
|
||||
|
||||
Fixes, in order of likelihood:
|
||||
|
||||
1. **A missing census entry.** The export can suspend but is not declared:
|
||||
add it to `scripts/common/jspi-exports.txt` *and* the scheduler wrap list
|
||||
in `jspi-scheduler.js` *and* `tests/apps/Makefile.wasm`'s
|
||||
`WX_JSPI_EXPORTS` (three synchronized copies — see doc 23).
|
||||
2. **A plain embind registration.** Suspending embind exports must be
|
||||
registered `emscripten::async()` — use `PCBJAM_PARKER_POLICY`
|
||||
(`wasm/bindings/pcbjam_async_policy.h`).
|
||||
3. **A genuinely illegal park** — code that must not suspend (a CLI/service
|
||||
target with no suspension backend, an `emscripten_set_main_loop` callback)
|
||||
grew a suspending call. Move the work behind a promising export instead.
|
||||
|
||||
---
|
||||
|
||||
## 4. The harnesses
|
||||
|
||||
### 4.1 `tests/apps/standalone/jspi-coroutine` — the coroutine contract battery
|
||||
|
||||
A wx-free MiniCoro that mirrors `tool/coroutine.h`'s protocol *exactly*
|
||||
(INVOCATION_ARGS, callerStub + `finish_fcontext`, jumpIn/jumpOut,
|
||||
CONTINUE_AFTER_ROOT) over the **real** `kicad/thirdparty/libcontext`. 18
|
||||
cases: entry/yield/resume/completion, deep-stack preservation, nesting with
|
||||
enterer inference, RunMainStack, value transfer, yield-inside-catch under
|
||||
native wasm-EH, timer-driven resume, slot reclaim, ghost-resume refusal
|
||||
(sentinel-shaped), mid-body release census, phantom-release refusal, and
|
||||
destroy-while-parked containment.
|
||||
|
||||
### 2.9 Source diagnostic logging flags (`--diag=`)
|
||||
The KiCad C++ source carries built-in diagnostic logging, **off by default**,
|
||||
enabled per category at build time:
|
||||
```bash
|
||||
./docker/build.sh --debug --diag=gal,coroutine,ctor # or: --diag=all
|
||||
cd tests/apps/standalone/jspi-coroutine
|
||||
./build.sh # rebuild both variants against the real libcontext
|
||||
node run.mjs # single-thread build, node
|
||||
node run_pt.mjs # pthread build
|
||||
# browser (both variants): tests/jspi/jspi-coroutine.spec.ts
|
||||
```
|
||||
| `--diag=` value | covers |
|
||||
|---|---|
|
||||
| `gal` | `[DIAG_GAL]` — GAL/WebGL pipeline (paint, context create/lock, init) |
|
||||
| `coroutine` | `[WASM_FCONTEXT]` fiber switches + `[DIAG_TOOL]`/`[DIAG_DISP]` tool dispatch |
|
||||
| `ctor` | `[DIAG_CTOR]` — `PCB_EDIT_FRAME` startup milestones |
|
||||
|
||||
- Each value maps to a `-DKICAD_DIAG_*` define that gates the `KI_DIAG_*` macros
|
||||
in `kicad/include/kicad_wasm_diag.h`. All output goes to **stdout** → it shows
|
||||
as `[KICAD_OUT]` logs, never `[KICAD_ERR]` errors.
|
||||
- **Compile-time:** changing `--diag` changes `CMAKE_CXX_FLAGS`, so it forces a
|
||||
recompile (slow once per flag combo, then ccache-cached). Works with `--debug`
|
||||
or `--release`.
|
||||
- Separate from the JS shim tracer (§2.3), which stays `SHIM_DIAGNOSTICS`-gated.
|
||||
Output contract: `[JSPI_CORO] CASE <name> PASS|FAIL(<detail>)`, then
|
||||
`[JSPI_CORO] SUMMARY passed=<n> failed=<n>`. If `build.sh` dies inside
|
||||
emscripten's python driver, point `EMSDK_PYTHON` at a modern interpreter
|
||||
(≥3.10; 3.13 known-good).
|
||||
|
||||
### 4.2 `tests/jspi/suspend-races.spec.ts` — semantic suspension races
|
||||
|
||||
The suspension-race scenarios (nested modal LIFO, out-of-order wake
|
||||
resolution, no-lost-wakes, nested-loop teardown-on-error), run against the
|
||||
races harness built for JSPI. The scenarios express through public wx +
|
||||
coroutine APIs, so they are exactly as meaningful under JSPI — only the
|
||||
failure *modes* they'd catch differ (activation misnesting or a lost wait
|
||||
token). `tests/jspi/jspi-stack.spec.ts` is the red/green proof of the
|
||||
spill-stack discipline itself.
|
||||
|
||||
### 4.3 Isolated probes, generally
|
||||
|
||||
A standalone probe often *won't* reproduce a bug that needs the full app
|
||||
runtime — don't over-trust a green probe. The reverse recipe still holds
|
||||
too: stub-bisection (comment out a suspect call, rebuild, observe a binary
|
||||
survives-or-fails outcome) beats staring at logs, because it does not depend
|
||||
on console delivery order.
|
||||
|
||||
---
|
||||
|
||||
## 3. Principles
|
||||
## 5. Browser notes
|
||||
|
||||
1. **Reproduce cleanly first** — a stable engine-X-fails / engine-Y-passes
|
||||
baseline before changing anything.
|
||||
2. **Fix the build infra before iterating** — a flaky build wastes every
|
||||
subsequent experiment.
|
||||
3. **Narrow by bisection**, with binary outcomes, not by staring at logs.
|
||||
4. **Turn silent failures into named ones** (assertions, asyncify-asserts) or
|
||||
into a state comparison across engines.
|
||||
5. **Know the tooling's blind spots** (§4) before trusting what it shows you.
|
||||
- **Firefox 153+ is the strict engine.** JSPI is on by default (Playwright
|
||||
≥1.62 ships FF 153) and a *plain* embind call into a suspending body throws
|
||||
immediately. Chromium tolerates some shapes FF refuses — the sync
|
||||
`kicadTestFiberPark*` levers are usable for manual probing **on Chromium
|
||||
only**. If a suspension bug reproduces on one engine only, suspect a
|
||||
call-shape difference first (§3), not a logic difference.
|
||||
- **Firefox runs big promising modules on a slow tier.** Observed as library
|
||||
enumeration slowness (FootprintEnumerate rows never appearing within 60 s
|
||||
on the remote read path); tracked upstream (#42199). The firefox leg of
|
||||
`footprint-browse-remote` is gated on it.
|
||||
- **COOP/COEP.** SharedArrayBuffer/pthreads need cross-origin isolation
|
||||
headers; serve `tests/apps` with `tests/serve.json`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Tooling blind spots (read before trusting output)
|
||||
## 6. Build-side debugging
|
||||
|
||||
- **Console is async** — `printf`/`console.*` from WASM reaches Playwright via
|
||||
CDP asynchronously; the *last delivered* line can lag the real failure point.
|
||||
Use stub-bisection for ground truth, not "the last log line."
|
||||
- **No name section** in the shipped wasm → bare indices (§2.4).
|
||||
- **Asyncify shifts code offsets** — DWARF line info is generated before the host
|
||||
Asyncify pass rewrites the code, so source-line mapping on the *shipped* wasm is
|
||||
stale. Asyncify *does* preserve function indices and names.
|
||||
- **Playwright hides the renderer** — forces `--disable-breakpad`, pipes only the
|
||||
browser process stderr (§2.6).
|
||||
- **macOS `sample`/`.ips`** see wasm frames as numeric offsets, not C++ names.
|
||||
- **The build is single-phase.** `docker/build.sh` compiles, links *and
|
||||
finalizes* inside the container; the only host step is
|
||||
`node scripts/common/patch-env-shim.mjs` (merges `Module.ENV` into the
|
||||
glue's `ENV` so `?trace=` works — seconds). `--compile-only` /
|
||||
`--postprocess-only` split the two when CI caches the compile. There is no
|
||||
post-link wasm rewriting to go wrong: what you linked is what runs.
|
||||
- **Logs + monitor.** Builds redirect all output to
|
||||
`logs/<script>/<timestamp>.log`; `./scripts/build-monitor.sh` renders a
|
||||
live stage dashboard off the newest log (`--once` for a snapshot).
|
||||
- **Docker compose project-name trap.** `build.sh` derives
|
||||
`COMPOSE_PROJECT_NAME` from the git branch (`kicad-wasm-<branch>`), so each
|
||||
branch has its own build-cache volume. Any *manual* `docker compose` run
|
||||
must export the same `COMPOSE_PROJECT_NAME` first or it silently targets a
|
||||
scratch volume.
|
||||
- **Source diagnostic logging** (off by default, per-category at build time):
|
||||
|
||||
---
|
||||
```bash
|
||||
./docker/build.sh --debug --diag=gal,coroutine,ctor # or: --diag=all
|
||||
```
|
||||
|
||||
## 5. Project-specific gotchas
|
||||
| `--diag=` value | covers |
|
||||
|---|---|
|
||||
| `gal` | `[DIAG_GAL]` — GAL/WebGL pipeline (paint, context create/lock, init) |
|
||||
| `coroutine` | `[WASM_FCONTEXT]` coroutine switches + `[DIAG_TOOL]`/`[DIAG_DISP]` tool dispatch |
|
||||
| `ctor` | `[DIAG_CTOR]` — `PCB_EDIT_FRAME` startup milestones |
|
||||
|
||||
- **Split build.** `docker/build.sh` compiles + links inside Docker, but the
|
||||
in-container `wasm-opt` and `wasm-emscripten-finalize` are **stubbed** (they OOM
|
||||
on the large wasm). The real `wasm-emscripten-finalize` and
|
||||
`wasm-opt --asyncify` run **on the host** afterward (`apply-finalize.sh`,
|
||||
`apply-asyncify.sh`). Real binary: `…/upstream/bin/wasm-opt.real`.
|
||||
- **Per-branch Docker volumes.** The compose project name is derived from the git
|
||||
branch, so each branch has its own build-cache volume/container. Switching
|
||||
optimization level (`-O1`↔`-O2`) busts ccache and forces a full recompile.
|
||||
- **COOP/COEP.** SharedArrayBuffer/pthreads need cross-origin isolation headers;
|
||||
serve `tests/apps` with `tests/serve.json` (`npx serve apps -c ../serve.json`).
|
||||
- **The shim layer.** `inject-dyncall-shims.sh` binds bare `dynCall_<sig>` to the
|
||||
real `DYNCALLS=1` exports and patches several Emscripten empty-stub callbacks by
|
||||
`sed` pattern — so codegen-changing flags can silently break it.
|
||||
- **Names / DWARF, concretely.** Neither build keeps a `name` section in the
|
||||
*runtime* wasm (it carries only `external_debug_info` + `target_features`). The
|
||||
**debug** build (`-O1 -g -gseparate-dwarf`) puts full DWARF in a ~1.5 GB
|
||||
`pcbnew.wasm.debug.wasm` sidecar (loaded on demand by DevTools' C/C++ extension);
|
||||
the **release** build (`-O2`, no `-g`) has neither names nor DWARF. So readable
|
||||
symbols come from the debug build's DWARF / the §2.4 symbol map, not from the
|
||||
shipped binary.
|
||||
|
||||
---
|
||||
|
||||
## 6. A worked example
|
||||
|
||||
The **Chrome-only startup stall** (May 2026): V8 could not run the
|
||||
Asyncify-*instrumented* `PCB_EDIT_FRAME::setupUIConditions()` (a huge function
|
||||
that never actually unwinds) when it was invoked from the Asyncify-rewound
|
||||
constructor stack — a silent stall, not a crash; Firefox ran the identical wasm
|
||||
fine. Found with stub-bisection (§2.1) + the dynCall tracer (§2.3) + symbol map
|
||||
(§2.4) + cross-engine state comparison (§2.5) + `sample` (§2.6).
|
||||
|
||||
A **second instance** of the same family (May 28, 2026) hit the line-drawing
|
||||
coroutine: V8 stalled at the first instruction of the asyncify-instrumented
|
||||
`libcontext::wasm_fcontext_entry` trampoline when a new fiber for
|
||||
`pcbnew.InteractiveDrawing.line` was entered. The `[DIAG_TOOL]` log showed
|
||||
the activate dispatching and `[WASM_FCONTEXT]` showed `jump-swap` completing,
|
||||
but `entry-call` (logged on the new fiber's first statement) never fired —
|
||||
the tool's button visually never toggled, and tests on headed Chrome **could
|
||||
not reproduce** it (same wasm, different cumulative asyncify state). Trying
|
||||
to add the trampoline / `COROUTINE::callerStub` to `ASYNCIFY_REMOVE` broke
|
||||
runtime because both functions sit ON the suspend chain (their callees
|
||||
`emscripten_fiber_swap` / suspendable tool bodies), so removing them from
|
||||
instrumentation orphans the rewind — `null function` / `ASM_CONSTS` errors.
|
||||
|
||||
The systemic fix (see [§7](#7-debug-vs-production-builds)) is now **committed
|
||||
default**: run `wasm-opt -O2` as a separate pass after `--asyncify` in
|
||||
`scripts/common/apply-asyncify.sh`. This shrinks every instrumented function
|
||||
back under V8's threshold, including the coroutine trampolines that can't be
|
||||
removelist'd. The legacy `ASYNCIFY_REMOVE` entries (`setupUIConditions`
|
||||
etc.) are kept as a redundant safety net — under `-O2` they're no longer
|
||||
required but are harmless.
|
||||
|
||||
Details: the `chrome-asyncify-rewind-crash` and `bundle-size-asyncify-optimization`
|
||||
project memories, and git history of `apply-asyncify.sh`.
|
||||
|
||||
---
|
||||
|
||||
## 7. Debug vs. production builds
|
||||
|
||||
The committed default is the **debug** build (compiled `-g -gseparate-dwarf`,
|
||||
DWARF sidecar) with `apply-asyncify.sh` running `wasm-opt --asyncify` followed
|
||||
by `wasm-opt -O2` (May 28, 2026). Result: ~187 MB wasm / ~65 MB gzip, full
|
||||
source-level debugging. Switch to release (`./docker/build.sh` without
|
||||
`--debug`) for an even smaller shippable build with no DWARF.
|
||||
|
||||
### What the knobs do
|
||||
Two independent knobs:
|
||||
- **`-g` (debug info)** — whether a source map exists at all. Debug =
|
||||
`-g -gseparate-dwarf` (DWARF sidecar); release = none.
|
||||
- **`-O` (optimization)** — how much the code is rewritten. This is what actually
|
||||
fixes the "function too big for V8" class of bug, because Asyncify emits
|
||||
deliberately verbose instrumentation (spills every live local) and **relies on
|
||||
the optimizer to coalesce it back down**. The Emscripten/Binaryen docs are
|
||||
emphatic that you must optimize when using Asyncify.
|
||||
|
||||
### How the build flow uses both
|
||||
1. **Docker compile + link** (`./docker/build.sh [--debug]`) produces an
|
||||
un-finalized, un-asyncified wasm. `--debug` controls only `-g`; the
|
||||
`-O2` optimisation level is set unconditionally at compile time.
|
||||
2. **Host post-processing** (`scripts/common/apply-finalize.sh` then
|
||||
`scripts/common/apply-asyncify.sh`):
|
||||
- `wasm-opt --asyncify` instruments suspendable functions.
|
||||
- `wasm-opt -O2` (added May 28, 2026) shrinks every instrumented
|
||||
function back under V8's per-function locals limit, fixing the
|
||||
"Chrome-only stall on coroutine entry" class of bug systemically.
|
||||
Without this pass, large asyncify-instrumented functions like
|
||||
`PCB_EDIT_FRAME::setupUIConditions()` or libcontext's
|
||||
`wasm_fcontext_entry` silently stall in Chrome's V8 even though
|
||||
Firefox runs them fine. The two passes are run separately so peak
|
||||
RAM stays ~10–15 GB (one heavy `wasm-opt` at a time).
|
||||
3. **Shim injection** (`scripts/common/inject-dyncall-shims.sh`) adds the
|
||||
asyncify-aware dynCall bindings and the nested-asyncify `handleSleep`
|
||||
wrapper to `pcbnew.js`.
|
||||
|
||||
### The `ASYNCIFY_REMOVE` list (in `apply-asyncify.sh`)
|
||||
With `-O2` after asyncify, no large function should exceed V8's limit anymore,
|
||||
so the removelist is mostly a redundant safety net. Two situations still
|
||||
warrant adding to it:
|
||||
- A function whose subtree does **not** asyncify-suspend (so removing it is
|
||||
always safe) and that you're confident never needs to participate in
|
||||
unwind/rewind. Example: `setupUIConditions()` — registers handlers, never
|
||||
yields.
|
||||
- **Don't** add functions on the asyncify-suspend chain (coroutine
|
||||
trampolines, anything calling `emscripten_fiber_swap` / `EM_ASYNC_JS`):
|
||||
removing them orphans the rewind path and you get `null function` /
|
||||
`ASM_CONSTS[code] is not a function` at runtime.
|
||||
|
||||
### Measured result (May 2026)
|
||||
| build | raw wasm | gzip | source-level debugging |
|
||||
|---|---|---|---|
|
||||
| debug, asyncify only (old default) | 338 MB | 137 MB | full (DWARF sidecar) |
|
||||
| debug + asyncify + `-O2` (current default) | **187 MB** | **65 MB** | full (DWARF sidecar) |
|
||||
| release + asyncify + `-O2` | smaller still | — | none |
|
||||
|
||||
The optimized build passes Chrome **and** Firefox `select draw lines` e2e,
|
||||
fixes the user-reported "line tool doesn't toggle in real Chrome" stall, and
|
||||
makes the test load+run ~2× faster (smaller wasm parses faster). Tradeoff:
|
||||
each build now spends an extra ~10 minutes on the `-O2` pass.
|
||||
Each value maps to a `-DKICAD_DIAG_*` define gating the `KI_DIAG_*` macros
|
||||
in `kicad/include/kicad_wasm_diag.h`; output goes to stdout
|
||||
(`[KICAD_OUT]`). Changing `--diag` changes `CMAKE_CXX_FLAGS` → forces a
|
||||
recompile (ccache-cached per flag combo).
|
||||
- **Per-branch volumes + optimization level.** Switching `-O1`↔`-O2` busts
|
||||
ccache and forces a full recompile; plan accordingly.
|
||||
|
|
|
|||
Loading…
Reference in a new issue