jspi cleanup: remove the asyncify-era residue — dead code, conditionals, pipeline scaffolding, stale prose
The runtime is JSPI-only; this removes everything that still pretended otherwise. Three exhaustive sweeps (C++/JS+build+CI/tests+docs) drove the inventory; every deletion verified by grep closure + full gates. Broken-right-now fixes: - deploy-staging.yml passed the retired opt_level input — the workflow could not even start. Removed. - env.sh carried dead exports with a live -sASYNCIFY=1 inside (WASM_LDFLAGS/PTHREAD_LDFLAGS, zero consumers). Removed; the WASM_LEGACY_EXCEPTIONS rationale rewritten to the real reason. - docker/build.sh exported PCBJAM_ASYNC_BACKEND (read nowhere). Gone. Dead weight removed: - binaryen submodule (nothing builds or invokes it), wasm-opt-bench workflow + scripts/bench/, get-wasm-opt.sh, diagnostics.js (242 lines of Asyncify-API-only code), the KICAD_PIPELINE background-postprocess scaffolding (existed to parallelize the deleted wasm-opt phase; the postprocess is a seconds-long node script and now runs inline), build-monitor's dead asyncify rows, sched-context orphan build output, dead .gitignore entries, the .jspi-assets spike dir (the two wf-result research JSONs moved to docs/features/async/migration-evidence/). - bindings: fiber_park.h + its 12 embind registrations (broken-if- called under JSPI), the kicadOpenFileStart/OPEN_JOB starter route, main_stack_runner.h + 5 includes, the always-null context-sleep weak hook in nanosleep_yield.c. - shim: the backend field (installed-flag idempotency instead), noteContextWait (dead both sides), the __wxAsyncifyDump alias (+ the WasmTool fallback and string-dump normalize branch). - web: the emscripten-6-ignored mainScriptUrlOrBlob option in boot.ts (gerber-demo keeps it: it loads the deployed CDN release, which predates emscripten 6 — noted inline). Conditionals: all 'backend === jspi' checks reduced to scheduler- presence checks; races_quiescent re-keyed from Asyncify.state (vacuous) to real backlog quiescence (resumeReady/mutatorQueue — NOT _windowLive, which is the probing activation's own window by definition). Renames (identifiers only, no file renames): ASYNC_LINK_FLAGS→ JSPI_LINK_FLAGS and Makefile ASYNC_LDFLAGS→JSPI_LDFLAGS, kicadCollabFiberBusy→kicadCollabBusy (embind + web + tests), collab_common.h fiber*→apply*/coroutine naming, asyncifySignatures→ wasmTrapSignatures (lists byte-identical). Tests: the two remaining vacuous [wx-asyncify]/fiber-resume-refused asserts re-keyed to live JSPI beacons; eeschema-load's failure message no longer sends the developer to a deleted script; wait-beacons' dead families/parser deleted; lane-0 legacy-glue guards removed (lane 0 is unconstructible); the embind test.fail re-gated with the JSPI reason (plain embind invokers cannot suspend — verified still failing); lint-determinism now scans tests/jspi (166 files clean); eeschema-collab local-move gated to chromium (~50% flaky on FF even solo; pcbnew twin covers both engines). Docs: DEBUG.md rewritten as the JSPI debugging guide; build.md describes the single-phase build; docs/features/async/README.md banner-marked historical and repointed at the NEW 23-jspi-runtime.md (current architecture: export census, turnstile, libcontext ownership + refusal contract, embind call shapes, the em-pthread service-wrapper trick, exception policy, known gaps). Gates on the cleaned tree: test:e2e 725 passed / 0 failed (after the quiescence-probe fix; the 3 other reds were verified contention flakes solo-green or the documented FF gate), web 76/0, jspi 18/18 both engines, vitest 295/295 + 17/17, all lints green, live-app census clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016X9eh1s5sTx1o9Em9KBuwR
This commit is contained in:
Viktor Vaczi
parent
3ee174e9b4
commit
9c475a804e
120 changed files with 1522 additions and 2974 deletions
|
|
@ -11,7 +11,7 @@ This document describes how to build KiCad for WebAssembly using the Docker-base
|
|||
|
||||
### Host Tools
|
||||
|
||||
Binaryen (wasm-opt) is downloaded automatically by the build script. No manual installation needed.
|
||||
Node.js (for the seconds-long host postprocess step). Everything else runs inside the container.
|
||||
|
||||
## Quick Start
|
||||
|
||||
|
|
@ -36,47 +36,30 @@ Binaryen (wasm-opt) is downloaded automatically by the build script. No manual i
|
|||
- `build-wasm/kicad-pcbnew/pcbnew/pcbnew.wasm` - WASM binary
|
||||
- `build-wasm/kicad-pcbnew/pcbnew/pcbnew.wasm.map` - Source map (debug builds)
|
||||
|
||||
## Two-Phase Build
|
||||
## Single-Phase Build
|
||||
|
||||
The build is split into two phases due to memory requirements:
|
||||
The build is one pass: `docker/build.sh` compiles, links **and finalizes** the
|
||||
wasm inside the container. The only host-side step is a postprocess on the
|
||||
generated glue — `node scripts/common/patch-env-shim.mjs` merges `Module.ENV`
|
||||
into the runtime's `ENV` (needed for `?trace=` and any future `Module.ENV`
|
||||
use). It takes seconds and is idempotent.
|
||||
|
||||
### Phase 1: Docker Compilation
|
||||
Compiles KiCad to WASM **without** asyncify transformation. This runs inside Docker with 32GB memory limit.
|
||||
`--compile-only` / `--postprocess-only` split the two so CI can cache the
|
||||
expensive compile and re-run just the host tail.
|
||||
|
||||
### Phase 2: Host Asyncify
|
||||
Applies `wasm-opt --asyncify` on the host machine using Binaryen v121 (downloaded automatically to `tools/`). This transformation uses ~20-30GB RAM.
|
||||
### Suspension: JSPI
|
||||
|
||||
**Note:** Binaryen v121 is used because v125 has a regression causing crashes in the asyncify liveness analysis.
|
||||
Blocking calls — `wxDialog::ShowModal()`, `wxMessageBox()`, clipboard,
|
||||
sleeps/waits, board loads — must yield to the browser event loop. This is
|
||||
handled at **link time** by JSPI (JavaScript Promise Integration): every wasm
|
||||
entry point that can suspend is a promising export
|
||||
(`-sJSPI -sJSPI_EXPORTS=@scripts/common/jspi-exports.txt`), and the
|
||||
`jspi-scheduler.js` pre-js supplies the spill-stack and resume-serialization
|
||||
discipline around it. See
|
||||
[docs/features/async/23-jspi-runtime.md](features/async/23-jspi-runtime.md).
|
||||
|
||||
### Why Asyncify?
|
||||
|
||||
Asyncify is an Emscripten transformation that allows WASM code to pause and resume execution. This is required for:
|
||||
|
||||
- **Modal dialogs** - `wxDialog::ShowModal()` blocks until user closes the dialog
|
||||
- **Message boxes** - `wxMessageBox()` waits for user response
|
||||
- **Clipboard operations** - Browser clipboard API is async
|
||||
- **Sleep/wait operations** - Any blocking call that needs to yield to the browser
|
||||
|
||||
Without asyncify, modal dialogs would freeze the browser because WASM cannot yield control back to JavaScript's event loop.
|
||||
|
||||
### How It Works
|
||||
|
||||
1. `docker/build.sh` compiles KiCad in Docker (no asyncify flags)
|
||||
2. Output is copied to `./output/` directory
|
||||
3. `wasm-opt --asyncify` runs on host, transforming the WASM binary
|
||||
4. Final output is ready for browser execution
|
||||
|
||||
### Technical Details
|
||||
|
||||
The asyncify transformation:
|
||||
- Instruments every function that might be on the call stack during an async operation
|
||||
- Adds stack save/restore logic to unwind and rewind the WASM stack
|
||||
- Increases binary size by ~20% (141MB → 171MB for KiCad)
|
||||
- Uses `asyncify-imports` pattern matching to identify async entry points
|
||||
|
||||
Import patterns used:
|
||||
- `env.invoke_*` - Exception handling trampolines
|
||||
- `env.__asyncjs__*` - EM_ASYNC_JS functions (like `startModal()`)
|
||||
There is no post-link binary rewriting: the wasm the container links is the
|
||||
wasm that ships.
|
||||
|
||||
## Docker Architecture
|
||||
|
||||
|
|
@ -155,17 +138,18 @@ The build system is optimized for fast development iteration:
|
|||
- **ccache**: Caches compiled objects by hashing preprocessed source
|
||||
- **wxWidgets**: `configure` runs once, `make` handles file-level dependencies
|
||||
- **KiCad**: CMake tracks dependencies, only recompiles changed files
|
||||
- **Asyncify**: Post-processing runs every build (~1 min, irreducible minimum)
|
||||
- **Host postprocess**: the ENV-shim patch (`patch-env-shim.mjs`) re-runs every build — seconds
|
||||
|
||||
### Performance
|
||||
|
||||
| Scenario | Time |
|
||||
|----------|------|
|
||||
| No changes | ~1.5 min |
|
||||
| Single file change (KiCad or wxWidgets) | ~1.5 min |
|
||||
| No changes | seconds |
|
||||
| Single file change (KiCad or wxWidgets) | dominated by the recompile + relink of that target |
|
||||
| Full rebuild | ~10 min |
|
||||
|
||||
Most time is spent on asyncify post-processing which runs on every build.
|
||||
There is no fixed per-build post-processing cost: an unchanged tree re-runs
|
||||
only the host ENV-shim patch.
|
||||
|
||||
### Debug vs Release
|
||||
|
||||
|
|
@ -259,24 +243,30 @@ The WASM port requires compatibility layers for browser execution:
|
|||
| Directory | Purpose |
|
||||
|-----------|---------|
|
||||
| `wasm/kiplatform/` | Platform abstraction (app, UI, printing, etc.) |
|
||||
| `wasm/libcontext/` | Coroutine/fiber implementation for Asyncify |
|
||||
| `kicad/thirdparty/libcontext/` | Coroutine backend (JSPI: one promising activation per coroutine) |
|
||||
| `wasm/stubs/` | Stub implementations (libgit2, curl) |
|
||||
| `wasm/config/` | Build configuration headers |
|
||||
|
||||
## Emscripten Flags
|
||||
|
||||
Key flags used in the build:
|
||||
Key flags used in the build (browser apps; see
|
||||
`scripts/kicad/build-kicad-target.sh` for the authoritative link surface):
|
||||
|
||||
```
|
||||
-pthread -sUSE_PTHREADS=1 # Threading support
|
||||
-sASYNCIFY=1 # Async coroutine support
|
||||
-sALLOW_MEMORY_GROWTH=1 # Dynamic memory
|
||||
-sINITIAL_MEMORY=256MB # Starting memory
|
||||
-sMAXIMUM_MEMORY=4GB # Maximum memory
|
||||
-sLEGACY_GL_EMULATION # OpenGL compatibility
|
||||
-sMAX_WEBGL_VERSION=2 # WebGL 2.0
|
||||
-pthread -sUSE_PTHREADS=1 # Threading support
|
||||
-sJSPI # JSPI suspension
|
||||
-sJSPI_EXPORTS=@scripts/common/jspi-exports.txt # promising-export census
|
||||
--pre-js scripts/common/shims/jspi-scheduler.js # scheduler/turnstile shim
|
||||
-sALLOW_MEMORY_GROWTH=1 # Dynamic memory
|
||||
-sINITIAL_MEMORY=256MB # Starting memory
|
||||
-sMAXIMUM_MEMORY=4GB # Maximum memory
|
||||
-sMAX_WEBGL_VERSION=2 # WebGL 2.0
|
||||
```
|
||||
|
||||
Headless targets (`kicad_tools`, `occ_service`) link **no suspension
|
||||
backend**: nothing in them may suspend, so they carry none of the three
|
||||
JSPI-related flags above.
|
||||
|
||||
## Testing
|
||||
|
||||
After building, run the test suite:
|
||||
|
|
|
|||
Loading…
Reference in a new issue