feat(wasm-eh): migrate the WASM build to native wasm exceptions (+ 3D viewer default-on)
Replace the legacy Emscripten JS-exceptions model with native wasm-EH (legacy encoding) across the whole build, keeping Asyncify coroutines working via a from-source Binaryen --hoist-cpp-catches pre-pass. Net result: native-EH is the only build mode, the 3D viewer is on by default, and pcbnew shrinks substantially. Highlights: - Binaryen submodule everywhere + --hoist-cpp-catches integration in apply-asyncify; post-link Asyncify covers every app wasm (not just standalone test wasm). - Build deps (incl. OpenCASCADE without OCC_CONVERT_SIGNALS) and all KiCad apps with -fwasm-exceptions; emscripten_sleep added to the post-link asyncify-imports. - libcontext fiber entry wired under native exceptions; while-loop main loop + currData shim injected into all wx apps. - Native-EH collab apply fixed: DEBUG-define the embind TU + match all out-of-CMake C++ TUs' ABI flags to the core, fixing the vtable-layout skew / mis-dispatch. - 3D viewer enabled by default (real raytracer linked, not the stub). - Retire the EH-spike scaffolding; flip the asyncify-races ablation pins to shim-redundancy pins (native-EH stays clean with the legacy shims ablated). - Fix the asyncify-races quiescence check to not require Asyncify.currData==0: under the native-EH per-frame-yield top loop the main stack is asyncify-suspended every frame, so currData legitimately churns (a freed-but-not-yet-nulled buffer, not a leak). Refresh the pcbnew toolbar screenshot baseline for the new kicad. - CI: drop the obsolete binaryen_version input/env (the build uses the binaryen submodule fork's wasm-opt, not a version download); key the wasm-output cache on the binaryen submodule SHA instead. Bumps the wxwidgets + binaryen submodules to their squashed feature commits. Validated green: all 7 apps native-EH (real 3D in pcbnew); KiCad e2e 63/63 Firefox + Chromium (3D viewer renders); wx 336; coroutine 34/34 both engines; asyncify 7/7 both engines. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Viktor Vaczi
parent
b8c8dee355
commit
c1ef489cfa
75 changed files with 4976 additions and 680 deletions
|
|
@ -1,157 +1,138 @@
|
|||
#!/bin/bash
|
||||
# Apply asyncify transformation to KiCad WASM
|
||||
# Unified post-link Asyncify pass for KiCad AND the wx test apps.
|
||||
#
|
||||
# Usage: ./scripts/common/apply-asyncify.sh <input.wasm> <output.wasm>
|
||||
# Usage: apply-asyncify.sh [--no-removelist] <input.wasm> [output.wasm]
|
||||
#
|
||||
# This script is called by docker/build.sh but can also be run standalone
|
||||
# for debugging asyncify issues.
|
||||
# Always: run our --hoist-cpp-catches fork pass FIRST (lets Asyncify suspend from inside C++ catch
|
||||
# blocks under native wasm-EH) with all wasm features enabled (-all, so binaryen parses the
|
||||
# EH instructions), then --asyncify + remove-list + -O2. Native wasm-EH is the only build mode.
|
||||
# --no-removelist skip the KiCad big-function remove-list (the small wx test apps don't contain
|
||||
# those symbols, and one bare entry — "match" — could collide).
|
||||
#
|
||||
# WHY post-link (not emcc's in-link Asyncify): the emsdk-bundled Binaryen crashes asyncifying
|
||||
# wasm-EH and a compiler/standalone version skew corrupts asyncify metadata. So the in-link pass is
|
||||
# stubbed (build-kicad-target.sh / build-wasm-test.sh) and the real transform runs here, on the host
|
||||
# (more RAM), with a pinned Binaryen. The cost: emcc's automatic asyncify-imports generation is
|
||||
# bypassed, so the BOUNDARY import list lives in asyncify-imports.txt (see that file).
|
||||
#
|
||||
# Binaryen selection (env overrides win, set by build-wasm-test.sh): one binaryen everywhere — the
|
||||
# submodule fork (version_130 + our --hoist-cpp-catches). Override the path via HOIST_WASMOPT / V130_WASMOPT.
|
||||
|
||||
set -e
|
||||
set -eo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
|
||||
|
||||
# Get wasm-opt path
|
||||
WASM_OPT=$("${SCRIPT_DIR}/get-wasm-opt.sh")
|
||||
# --- flags ---
|
||||
# Native wasm-EH is the only build mode, so the catch-arm hoist pass always runs.
|
||||
DO_HOIST=1
|
||||
USE_REMOVELIST=1
|
||||
while [[ "${1:-}" == --* ]]; do
|
||||
case "$1" in
|
||||
--no-removelist) USE_REMOVELIST=0; shift ;;
|
||||
*) echo "apply-asyncify: unknown flag: $1" >&2; exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Bound Binaryen's host thread pool. wasm-opt runs function-parallel passes, and
|
||||
# each worker holds the optimization working-set of one function at a time — so
|
||||
# peak RAM scales with thread count. Binaryen reads BINARYEN_CORES to size the
|
||||
# pool; default to 8 for memory-constrained dev machines, overridable via the
|
||||
# environment (CI sets it to $(nproc) on the 128 GB Hetzner runner).
|
||||
export BINARYEN_CORES="${BINARYEN_CORES:-8}"
|
||||
INPUT_WASM="${1:-output/pcbnew.wasm}"
|
||||
OUTPUT_WASM="${2:-${INPUT_WASM}}"
|
||||
[ -f "${INPUT_WASM}" ] || { echo "ERROR: Input file not found: ${INPUT_WASM}" >&2; exit 1; }
|
||||
|
||||
# Preload a scalable allocator on Linux. wasm-opt churns a ~40 GB high-water mark
|
||||
# of short-lived allocations across all worker threads; glibc malloc serializes
|
||||
# concurrent alloc/free on per-arena locks, so under many threads ~half of every
|
||||
# core's cycles collapse into futex lock-spin (strace: ~99% kernel time in futex)
|
||||
# instead of optimization work — the more cores, the worse it gets. jemalloc and
|
||||
# mimalloc are built for exactly this many-thread churn and eliminate the storm,
|
||||
# roughly halving wall-clock. macOS already ships a scalable allocator
|
||||
# (libmalloc/nano-zone), so only Linux needs this. Honor an externally-set
|
||||
# WASM_OPT_PRELOAD; otherwise auto-detect a system jemalloc/mimalloc.
|
||||
#
|
||||
# WASM_OPT_PRELOAD=none (or 0) forces NO preload — a clean glibc baseline for
|
||||
# benchmarking the allocator A/B (see scripts/bench/).
|
||||
if [[ "${WASM_OPT_PRELOAD:-}" == "none" || "${WASM_OPT_PRELOAD:-}" == "0" ]]; then
|
||||
WASM_OPT_PRELOAD=""
|
||||
_PRELOAD_FORCED_OFF=1
|
||||
# --- Binaryen tool: ONE binaryen everywhere (incl. docker/CI) — the submodule fork. It IS Binaryen
|
||||
# version_130 (its Asyncify.cpp is unmodified upstream) + our HoistCppCatches pass, so the SAME binary
|
||||
# does --hoist-cpp-catches AND --asyncify/-O2, for both native-EH and legacy JS-EH wasm. Built once via
|
||||
# build-wasm-opt.sh; no separate binaryen downloads (the emsdk-bundled v121 can't even asyncify wasm-EH).
|
||||
SUBMODULE_WASMOPT="${HOIST_WASMOPT:-$("${SCRIPT_DIR}/../binaryen-hoist-pass/build-wasm-opt.sh")}"
|
||||
WASM_OPT="${V130_WASMOPT:-$SUBMODULE_WASMOPT}"
|
||||
[ "$DO_HOIST" = 1 ] && HOIST_OPT="$SUBMODULE_WASMOPT"
|
||||
|
||||
# native wasm-EH needs -all so binaryen parses the EH instructions; HOIST_KEEP_NAMES keeps the
|
||||
# names section through -O2 for callstack debugging. KiCad/JS-EH uses neither (matches old behavior).
|
||||
FEAT=()
|
||||
[ "$DO_HOIST" = 1 ] && FEAT=(-all)
|
||||
G="${HOIST_KEEP_NAMES:+-g}"
|
||||
|
||||
# The asyncify removelist matches functions by NAME. wasm-opt strips the names section by default, so
|
||||
# the hoist pass (run before asyncify) would otherwise hand asyncify nameless functions — the
|
||||
# removelist then matches NOTHING and the giant try-dense functions it is meant to exclude
|
||||
# (BuildBitmapInfo: ~4986 native tries, etc.) get instrumented, which is the dominant driver of the
|
||||
# multi-GB asyncify RAM blowup on native-EH. So force the hoist pass to keep names whenever a
|
||||
# removelist is in play, so asyncify can see + exclude them. asyncify/-O2 keep their own G, so the
|
||||
# FINAL wasm is unchanged unless HOIST_KEEP_NAMES is set (asyncify matches on its INPUT names, which
|
||||
# the hoist output now carries).
|
||||
HOIST_G="${G}"
|
||||
[ "$USE_REMOVELIST" = 1 ] && HOIST_G="-g"
|
||||
|
||||
# --- the import boundary (shared) + the KiCad remove-list (opt-out), read from sibling files ---
|
||||
_join_list() { grep -vE '^[[:space:]]*#|^[[:space:]]*$' "$1" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | tr '\n' ',' | sed 's/,$//'; }
|
||||
ASYNCIFY_IMPORTS="${ASYNCIFY_IMPORTS_PASS:-$(_join_list "${SCRIPT_DIR}/asyncify-imports.txt")}"
|
||||
REMOVE_ARG=()
|
||||
if [ "$USE_REMOVELIST" = 1 ]; then
|
||||
REMOVE_ARG=("--pass-arg=asyncify-removelist@$(_join_list "${SCRIPT_DIR}/asyncify-removelist.txt")")
|
||||
fi
|
||||
|
||||
# --- memory machinery (identical to before; matters for the host-side KiCad pass) ---
|
||||
# Bound Binaryen's host thread pool: peak RAM scales with thread count.
|
||||
export BINARYEN_CORES="${BINARYEN_CORES:-8}"
|
||||
|
||||
# Preload a scalable allocator on Linux — glibc malloc collapses into futex lock-spin under
|
||||
# wasm-opt's many-thread allocation churn; jemalloc/mimalloc roughly halve wall-clock. macOS
|
||||
# already ships a scalable allocator. WASM_OPT_PRELOAD=none|0 forces a clean glibc baseline.
|
||||
if [[ "${WASM_OPT_PRELOAD:-}" == "none" || "${WASM_OPT_PRELOAD:-}" == "0" ]]; then
|
||||
WASM_OPT_PRELOAD=""; _PRELOAD_FORCED_OFF=1
|
||||
fi
|
||||
if [[ -z "${WASM_OPT_PRELOAD:-}" && -z "${_PRELOAD_FORCED_OFF:-}" && "$(uname -s)" == "Linux" ]]; then
|
||||
for _alloc in \
|
||||
"/usr/lib/$(uname -m)-linux-gnu/libjemalloc.so.2" \
|
||||
"/usr/lib/$(uname -m)-linux-gnu/libmimalloc.so.2" \
|
||||
/usr/lib/libjemalloc.so.2 \
|
||||
/usr/lib/libmimalloc.so.2; do
|
||||
if [[ -e "${_alloc}" ]]; then
|
||||
WASM_OPT_PRELOAD="${_alloc}"
|
||||
break
|
||||
fi
|
||||
/usr/lib/libjemalloc.so.2 /usr/lib/libmimalloc.so.2; do
|
||||
[[ -e "${_alloc}" ]] && { WASM_OPT_PRELOAD="${_alloc}"; break; }
|
||||
done
|
||||
fi
|
||||
|
||||
# Build the command prefix that injects the allocator (preserving any existing
|
||||
# LD_PRELOAD). Empty when no scalable allocator was found — wasm-opt then runs
|
||||
# under the default allocator, just slower.
|
||||
if [[ -n "${WASM_OPT_PRELOAD:-}" ]]; then
|
||||
PRELOAD_CMD=(env "LD_PRELOAD=${WASM_OPT_PRELOAD}${LD_PRELOAD:+:${LD_PRELOAD}}")
|
||||
else
|
||||
PRELOAD_CMD=()
|
||||
fi
|
||||
# GNU `time -v` on Linux CI records peak RSS + wall-clock per pass; macOS `time` lacks -v.
|
||||
if /usr/bin/time -v true >/dev/null 2>&1; then TIME_CMD=(/usr/bin/time -v); else TIME_CMD=(); fi
|
||||
|
||||
# Wrap wasm-opt in GNU `time -v` when available (Linux CI) so the log records
|
||||
# peak RSS + wall-clock for each pass. macOS `time` lacks -v, so fall back to
|
||||
# running wasm-opt directly there.
|
||||
if /usr/bin/time -v true >/dev/null 2>&1; then
|
||||
TIME_CMD=(/usr/bin/time -v)
|
||||
else
|
||||
TIME_CMD=()
|
||||
fi
|
||||
|
||||
INPUT_WASM="${1:-output/pcbnew.wasm}"
|
||||
OUTPUT_WASM="${2:-${INPUT_WASM}}"
|
||||
|
||||
if [ ! -f "${INPUT_WASM}" ]; then
|
||||
echo "ERROR: Input file not found: ${INPUT_WASM}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Applying asyncify transformation..."
|
||||
echo "Applying Asyncify${DO_HOIST:+ (+hoist-cpp-catches)}..."
|
||||
echo " Input: ${INPUT_WASM}"
|
||||
echo " Output: ${OUTPUT_WASM}"
|
||||
echo " Tool: ${WASM_OPT}"
|
||||
echo " asyncify wasm-opt: ${WASM_OPT}"
|
||||
[ "$DO_HOIST" = 1 ] && echo " hoist wasm-opt: ${HOIST_OPT}"
|
||||
echo " BINARYEN_CORES=${BINARYEN_CORES} LD_PRELOAD=${WASM_OPT_PRELOAD:-<none>}"
|
||||
|
||||
# Asyncify import patterns (functions that trigger async suspension)
|
||||
# - env.invoke_* : Exception handling trampolines
|
||||
# - env.__asyncjs__* : EM_ASYNC_JS functions (like startModal())
|
||||
ASYNCIFY_IMPORTS="env.invoke_*,env.__asyncjs__*,env.emscripten_fiber_swap"
|
||||
SRC="${INPUT_WASM}"
|
||||
|
||||
# Functions to exclude from asyncify instrumentation
|
||||
# These are large functions that inflate beyond V8's local-count limits.
|
||||
ASYNCIFY_REMOVE=$(cat << 'REMOVELIST'
|
||||
COLOR_SETTINGS::COLOR_SETTINGS(wxString const&, bool)
|
||||
BuildBitmapInfo(std::__2::unordered_map<BITMAPS, std::__2::vector<BITMAP_INFO, std::__2::allocator<BITMAP_INFO>>, std::__2::hash<BITMAPS>, std::__2::equal_to<BITMAPS>, std::__2::allocator<std::__2::pair<BITMAPS const, std::__2::vector<BITMAP_INFO, std::__2::allocator<BITMAP_INFO>>>>>&)
|
||||
match
|
||||
DIALOG_PAD_PROPERTIES_BASE::DIALOG_PAD_PROPERTIES_BASE(wxWindow*, int, wxString const&, wxPoint const&, wxSize const&, long)
|
||||
buildKicadAboutBanner(EDA_BASE_FRAME*, ABOUT_APP_INFO&)
|
||||
IGESToBRep_CurveAndSurface::TransferGeometry(opencascade::handle<IGESData_IGESEntity> const&, Message_ProgressRange const&)
|
||||
StepAP214_Protocol::StepAP214_Protocol()
|
||||
BRepCheck_ParallelAnalyzer::operator()(int) const
|
||||
ShapeFix_Wire::FixGap3d(int, bool)
|
||||
ShapeFix_Wire::FixGap2d(int, bool)
|
||||
PCB_EDIT_FRAME::setupUIConditions()
|
||||
REMOVELIST
|
||||
)
|
||||
|
||||
ASYNCIFY_REMOVE_ARG=$(echo "${ASYNCIFY_REMOVE}" | tr '\n' ',' | sed 's/,$//')
|
||||
|
||||
echo ""
|
||||
echo "Running wasm-opt --asyncify..."
|
||||
echo "This may take several minutes and use significant RAM..."
|
||||
echo " BINARYEN_CORES=${BINARYEN_CORES}"
|
||||
echo " LD_PRELOAD=${WASM_OPT_PRELOAD:-<none>}"
|
||||
|
||||
"${PRELOAD_CMD[@]}" "${TIME_CMD[@]}" "${WASM_OPT}" --asyncify ${ASYNCIFY_EXTRA_OPTS:-} \
|
||||
"--pass-arg=asyncify-imports@${ASYNCIFY_IMPORTS}" \
|
||||
"--pass-arg=asyncify-removelist@${ASYNCIFY_REMOVE_ARG}" \
|
||||
--pass-arg=asyncify-propagate-addlist \
|
||||
"${INPUT_WASM}" -o "${OUTPUT_WASM}"
|
||||
|
||||
# ASYNCIFY_ONLY=1 stops after the asyncify pass (skips -O2). Used by the
|
||||
# benchmark harness (scripts/bench/) to time/compare just the asyncify pass,
|
||||
# whose RAM fits where the -O2 pass on the bloated module would not.
|
||||
if [[ "${ASYNCIFY_ONLY:-0}" == "1" ]]; then
|
||||
echo ""
|
||||
echo "ASYNCIFY_ONLY=1 → skipping -O2 pass (benchmark mode)."
|
||||
ls -lh "${OUTPUT_WASM}"
|
||||
exit 0
|
||||
# 1. (native wasm-EH only) hoist C++ catch arms so Asyncify can suspend from inside them.
|
||||
if [ "$DO_HOIST" = 1 ]; then
|
||||
echo "Running --hoist-cpp-catches${HOIST_G:+ (keeping names for removelist matching)}..."
|
||||
"${PRELOAD_CMD[@]}" "${TIME_CMD[@]}" "${HOIST_OPT}" --hoist-cpp-catches "${FEAT[@]}" ${HOIST_G} "${SRC}" -o "${OUTPUT_WASM}"
|
||||
SRC="${OUTPUT_WASM}"
|
||||
fi
|
||||
|
||||
# Post-asyncify shrink pass. Asyncify emits deliberately verbose instrumentation
|
||||
# (spills every live local) and relies on the optimizer to coalesce it back down
|
||||
# under V8's per-function locals limit — without it, large coroutine-entry
|
||||
# functions silently stall (and on newer Chromium hard-crash the renderer while
|
||||
# loading a heavy board) in Chrome's V8. See docs/debugging/DEBUG.md §6-7.
|
||||
#
|
||||
# Level is configurable via BINARYEN_OPT_LEVEL: -O1 already runs CoalesceLocals
|
||||
# and is sufficient (measured: same ~187 MB / 64 MB-gzip output as -O2 on pcbnew,
|
||||
# validated green on the load-pcb chromium-ci spec — but a lighter, faster pass,
|
||||
# which is why CI sets -O1). -O2 stays the default for the more thorough rewrite.
|
||||
# Do NOT use -Os/-Oz here: they break the asyncify runtime (Binaryen #4484).
|
||||
# 2. The real Asyncify transform. ASYNCIFY_EXTRA_OPTS: optional extra wasm-opt flags (origin/main hook).
|
||||
echo "Running wasm-opt --asyncify (several minutes + significant RAM)..."
|
||||
"${PRELOAD_CMD[@]}" "${TIME_CMD[@]}" "${WASM_OPT}" --asyncify ${ASYNCIFY_EXTRA_OPTS:-} "${FEAT[@]}" ${G} \
|
||||
"--pass-arg=asyncify-imports@${ASYNCIFY_IMPORTS}" \
|
||||
"${REMOVE_ARG[@]}" \
|
||||
--pass-arg=asyncify-propagate-addlist \
|
||||
"${SRC}" -o "${OUTPUT_WASM}"
|
||||
|
||||
# ASYNCIFY_ONLY=1 stops before -O2 (benchmark harness in scripts/bench/ times just the transform).
|
||||
if [[ "${ASYNCIFY_ONLY:-0}" == "1" ]]; then
|
||||
echo "ASYNCIFY_ONLY=1 → skipping -O2 (benchmark mode)."; ls -lh "${OUTPUT_WASM}"; exit 0
|
||||
fi
|
||||
|
||||
# 3. Post-asyncify shrink. Asyncify spills every live local; without coalescing, large
|
||||
# coroutine-entry functions exceed V8's per-function locals limit and stall/crash the renderer.
|
||||
# -O1 already runs CoalesceLocals and suffices (CI uses it); -O2 is the default. NOT -Os/-Oz
|
||||
# (they break the asyncify runtime — Binaryen #4484). See docs/debugging/DEBUG.md §6-7.
|
||||
BINARYEN_OPT_LEVEL="${BINARYEN_OPT_LEVEL:--O2}"
|
||||
echo "Running wasm-opt ${BINARYEN_OPT_LEVEL} (shrink instrumented functions under V8's local limit)..."
|
||||
"${PRELOAD_CMD[@]}" "${TIME_CMD[@]}" "${WASM_OPT}" "${BINARYEN_OPT_LEVEL}" "${FEAT[@]}" ${G} "${OUTPUT_WASM}" -o "${OUTPUT_WASM}"
|
||||
|
||||
echo ""
|
||||
echo "Running wasm-opt ${BINARYEN_OPT_LEVEL} on the asyncified wasm..."
|
||||
echo " Purpose: shrink asyncify-instrumented functions back under V8's"
|
||||
echo " per-function locals limit (otherwise large coroutine-entry and"
|
||||
echo " similar functions stall/crash in Chrome's V8). See docs/debugging/DEBUG.md §6-7."
|
||||
echo " This pass takes several minutes and ~10-15 GB RAM."
|
||||
echo " BINARYEN_CORES=${BINARYEN_CORES}"
|
||||
echo " LD_PRELOAD=${WASM_OPT_PRELOAD:-<none>}"
|
||||
|
||||
"${PRELOAD_CMD[@]}" "${TIME_CMD[@]}" "${WASM_OPT}" "${BINARYEN_OPT_LEVEL}" "${OUTPUT_WASM}" -o "${OUTPUT_WASM}"
|
||||
|
||||
echo ""
|
||||
echo "Asyncify + ${BINARYEN_OPT_LEVEL} complete: ${OUTPUT_WASM}"
|
||||
ls -lh "${OUTPUT_WASM}"
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
#!/bin/bash
|
||||
# Apply wasm-emscripten-finalize transformation on host
|
||||
# Uses the same Binaryen v121 as wasm-opt to ensure version consistency
|
||||
# Apply wasm-emscripten-finalize transformation on host.
|
||||
#
|
||||
# Usage: ./scripts/common/apply-finalize.sh <input.wasm> <output.wasm>
|
||||
|
||||
|
|
@ -9,13 +8,12 @@ set -e
|
|||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
|
||||
|
||||
# Get wasm-opt path (this downloads Binaryen if needed)
|
||||
WASM_OPT=$("${SCRIPT_DIR}/get-wasm-opt.sh")
|
||||
BINARYEN_BIN=$(dirname "${WASM_OPT}")
|
||||
FINALIZE="${BINARYEN_BIN}/wasm-emscripten-finalize"
|
||||
# Same stub hazard as wasm-opt (see get-wasm-opt.sh): a host-mode kicad build
|
||||
# leaves the emsdk finalize stubbed with the real binary at .real — prefer it,
|
||||
# the stub exits 0 having done nothing.
|
||||
# wasm-emscripten-finalize ships with emscripten (it was removed from Binaryen ~v116), so use the
|
||||
# emsdk's own — the one emscripten would run in-link — directly. No binaryen download. A host-mode
|
||||
# kicad/test build stubs the emsdk finalize and keeps the real binary at .real; prefer it (the stub
|
||||
# exits 0 having done nothing).
|
||||
EMSDK_DIR="${EMSDK:-${PROJECT_ROOT}/tools/emsdk}"
|
||||
FINALIZE="${EMSDK_DIR}/upstream/bin/wasm-emscripten-finalize"
|
||||
if [ -x "${FINALIZE}.real" ]; then
|
||||
FINALIZE="${FINALIZE}.real"
|
||||
fi
|
||||
|
|
|
|||
21
scripts/common/asyncify-imports.txt
Normal file
21
scripts/common/asyncify-imports.txt
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
# Suspending imports for the post-link Asyncify pass (consumed by apply-asyncify.sh).
|
||||
# Binaryen's --asyncify instruments every function that can transitively REACH one of these
|
||||
# imports. This is the BOUNDARY list, not a function allowlist — the callers are auto-discovered.
|
||||
#
|
||||
# We run Asyncify post-link (outside emcc), so emcc's automatic import generation never runs;
|
||||
# this file replicates emcc's default async built-ins + our own suspending imports. A superset
|
||||
# is safe — binaryen ignores any import the wasm doesn't actually contain.
|
||||
#
|
||||
# --- Emscripten async built-ins (what emcc auto-adds for the in-link Asyncify) ---
|
||||
env.emscripten_sleep
|
||||
env.emscripten_scan_registers
|
||||
env.emscripten_lazy_load_code
|
||||
env.emscripten_wget
|
||||
env.emscripten_wget_data
|
||||
env.emscripten_idb_*
|
||||
#
|
||||
# --- Project suspending imports ---
|
||||
env.__asyncjs__*
|
||||
env.emscripten_fiber_swap
|
||||
env.startModal
|
||||
env.js_*
|
||||
26
scripts/common/asyncify-removelist.txt
Normal file
26
scripts/common/asyncify-removelist.txt
Normal file
|
|
@ -0,0 +1,26 @@
|
|||
# Functions EXCLUDED from Asyncify instrumentation (consumed by apply-asyncify.sh unless
|
||||
# --no-removelist). These are large, NON-suspending KiCad/OpenCASCADE functions (generated resource
|
||||
# tables, wxFormBuilder UI constructors, OCC geometry) that are dense with native wasm try/catch.
|
||||
# Asyncify's per-function cost is superlinear in try-count (it builds a CFG + liveness over every
|
||||
# try), so instrumenting these few giants is what drives the multi-GB RAM blowup of `wasm-opt
|
||||
# --asyncify` on the native-EH build. They never call a suspending import, so excluding them is safe.
|
||||
#
|
||||
# MATCHING: Binaryen escapes each entry, then treats entries containing `*` as wildcard patterns
|
||||
# (String::wildcardMatch) and entries without `*` as exact escaped function names. Full demangled
|
||||
# signatures don't exact-match (subtle spacing) and a `*` inside a pointer type (e.g. wxWindow*) turns
|
||||
# the whole entry into a pattern that still misses — so use a PREFIX wildcard per symbol, which matches
|
||||
# the function's escaped demangled name regardless of argument formatting. Patterns that match nothing
|
||||
# in a given app (e.g. the OCC entries when 3D is off) just emit a harmless "non-matching" warning.
|
||||
# Only the KiCad build passes the remove-list; the small wx test apps opt out via --no-removelist
|
||||
# (and the bare name "match" could collide there).
|
||||
COLOR_SETTINGS::COLOR_SETTINGS*
|
||||
BuildBitmapInfo*
|
||||
match
|
||||
DIALOG_PAD_PROPERTIES_BASE::DIALOG_PAD_PROPERTIES_BASE*
|
||||
buildKicadAboutBanner*
|
||||
IGESToBRep_CurveAndSurface::TransferGeometry*
|
||||
StepAP214_Protocol::StepAP214_Protocol*
|
||||
BRepCheck_ParallelAnalyzer::operator*
|
||||
ShapeFix_Wire::FixGap3d*
|
||||
ShapeFix_Wire::FixGap2d*
|
||||
PCB_EDIT_FRAME::setupUIConditions*
|
||||
|
|
@ -36,6 +36,15 @@ export WX_BUILD="$BUILD_ROOT/wxwidgets"
|
|||
# Emscripten settings
|
||||
export EMSDK_QUIET=1
|
||||
|
||||
# Exception model for ALL WASM translation units — the C/C++ dependencies, wxWidgets, and KiCad
|
||||
# (build-wx-wasm.sh and build-kicad-target.sh source this file and reuse DEPS_EH_FLAGS). Native
|
||||
# WebAssembly exceptions (legacy binary encoding) are the only build mode. -sSUPPORT_LONGJMP=wasm is
|
||||
# required because the deps that use setjmp/longjmp (freetype, cairo, OpenCASCADE) must use wasm
|
||||
# setjmp — emscripten's JS-longjmp implementation cannot coexist with -fwasm-exceptions (it would
|
||||
# leave emscripten_longjmp undefined). -sWASM_LEGACY_EXCEPTIONS=1 selects the EH binary encoding our
|
||||
# post-link Asyncify + catch-arm-hoisting pass can consume (Asyncify can't handle exnref).
|
||||
export DEPS_EH_FLAGS="-fwasm-exceptions -sSUPPORT_LONGJMP=wasm -sWASM_LEGACY_EXCEPTIONS=1"
|
||||
|
||||
# Emscripten SDK setup
|
||||
# If EMSDK is already set (e.g. Docker entrypoint sourced emsdk_env.sh), use it.
|
||||
# Otherwise auto-install a local copy under tools/emsdk/.
|
||||
|
|
|
|||
|
|
@ -4,17 +4,15 @@
|
|||
#
|
||||
# The actual JavaScript that gets injected lives in readable, standalone files in
|
||||
# scripts/common/shims/ (not inline heredocs):
|
||||
# - dyncall-binding.js.tmpl per-signature dynCall_<sig> binding (templated)
|
||||
# - handlesleep.js nested-Asyncify handleSleep currData save/restore (#9153)
|
||||
# - diagnostics.js optional logging-only instrumentation (see SHIM_DIAGNOSTICS)
|
||||
#
|
||||
# Why bind dynCall_* to the real wasm exports: the build links -sDYNCALLS=1, so
|
||||
# asyncify-INSTRUMENTED dynCall_* trampolines exist as wasm exports. The bare
|
||||
# dynCall_<sig>(index, ...) call sites (invoke_* wrappers, the empty-callback fixes
|
||||
# below) aren't bound to top-level names, so we bind them to wasmExports["dynCall_<sig>"].
|
||||
# Binding to getWasmTableEntry() instead bypasses the instrumentation and breaks
|
||||
# unwind/rewind through indirect calls ("indirect call signature mismatch" — caught
|
||||
# every frame in Firefox; a hard renderer crash in Chrome/V8).
|
||||
# Native wasm-EH is the only build mode, so the .js has no invoke_* wrappers / dynCall_<sig> call
|
||||
# sites to bind. The build still links -sDYNCALLS=1, so asyncify-INSTRUMENTED dynCall_* trampolines
|
||||
# exist as wasm EXPORTS; the empty-callback fixes below route function-pointer stubs through
|
||||
# wasmExports["dynCall_<sig>"]. This MUST be the wasm trampoline, NOT getWasmTableEntry — the latter
|
||||
# bypasses the instrumentation and breaks unwind/rewind through indirect calls ("indirect call
|
||||
# signature mismatch" — caught every frame in Firefox; a hard renderer crash in Chrome/V8).
|
||||
#
|
||||
# Usage:
|
||||
# inject-dyncall-shims.sh <pcbnew.js>
|
||||
|
|
@ -34,72 +32,23 @@ if [ -z "$JS_FILE" ] || [ ! -f "$JS_FILE" ]; then
|
|||
echo "Usage: $0 <path/to/pcbnew.js>"
|
||||
exit 1
|
||||
fi
|
||||
for f in dyncall-binding.js.tmpl handlesleep.js diagnostics.js; do
|
||||
for f in handlesleep.js diagnostics.js; do
|
||||
if [ ! -f "$SHIM_DIR/$f" ]; then
|
||||
echo "Error: missing shim source $SHIM_DIR/$f"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
# --- 1. dynCall_<sig> bindings -------------------------------------------------
|
||||
echo "Extracting dynCall signatures from $JS_FILE..."
|
||||
SIGNATURES=$(grep -oE 'dynCall_[a-zA-Z0-9]+' "$JS_FILE" | sort -u | sed 's/dynCall_//')
|
||||
if [ -z "$SIGNATURES" ]; then
|
||||
echo "No dynCall signatures found - nothing to inject"
|
||||
exit 0
|
||||
fi
|
||||
SIG_COUNT=$(echo "$SIGNATURES" | wc -l | tr -d ' ')
|
||||
echo "Found $SIG_COUNT unique signatures"
|
||||
|
||||
# The template body is everything from the `function` line onward (skip its comments).
|
||||
TEMPLATE_BODY=$(sed -n '/^function /,$p' "$SHIM_DIR/dyncall-binding.js.tmpl")
|
||||
|
||||
SHIM_FILE=$(mktemp)
|
||||
{
|
||||
echo ""
|
||||
echo "// === dynCall bindings (bind bare names to the real DYNCALLS=1 wasm exports) ==="
|
||||
} > "$SHIM_FILE"
|
||||
|
||||
for sig in $SIGNATURES; do
|
||||
argcount=$((${#sig} - 1))
|
||||
args="index"
|
||||
call_args=""
|
||||
for ((i=0; i<argcount; i++)); do
|
||||
args="$args, a$i"
|
||||
if [ $i -gt 0 ]; then call_args="$call_args, "; fi
|
||||
call_args="${call_args}a$i"
|
||||
done
|
||||
echo "$TEMPLATE_BODY" \
|
||||
| sed -e "s/@SIG@/$sig/g" -e "s/@ARGS@/$args/g" -e "s/@CALLARGS@/$call_args/g" \
|
||||
>> "$SHIM_FILE"
|
||||
done
|
||||
echo "// === End dynCall bindings ===" >> "$SHIM_FILE"
|
||||
|
||||
# Insert right after the getWasmTableEntry definition.
|
||||
GWTL_LINE=$(grep -n '^var getWasmTableEntry = funcPtr => {' "$JS_FILE" | head -1 | cut -d: -f1)
|
||||
if [ -z "$GWTL_LINE" ]; then
|
||||
GWTL_LINE=$(grep -n 'var getWasmTableEntry' "$JS_FILE" | head -1 | cut -d: -f1)
|
||||
fi
|
||||
if [ -z "$GWTL_LINE" ]; then
|
||||
echo "Error: Could not find getWasmTableEntry in $JS_FILE"; rm "$SHIM_FILE"; exit 1
|
||||
fi
|
||||
INSERT_LINE=""
|
||||
for ((i=GWTL_LINE; i<=GWTL_LINE+10; i++)); do
|
||||
[ "$(sed -n "${i}p" "$JS_FILE")" == "};" ] && { INSERT_LINE=$i; break; }
|
||||
done
|
||||
[ -z "$INSERT_LINE" ] && INSERT_LINE=$GWTL_LINE
|
||||
|
||||
echo "Injecting dynCall bindings after line $INSERT_LINE..."
|
||||
head -n "$INSERT_LINE" "$JS_FILE" > "${JS_FILE}.tmp"
|
||||
cat "$SHIM_FILE" >> "${JS_FILE}.tmp"
|
||||
tail -n +$((INSERT_LINE + 1)) "$JS_FILE" >> "${JS_FILE}.tmp"
|
||||
mv "${JS_FILE}.tmp" "$JS_FILE"
|
||||
rm "$SHIM_FILE"
|
||||
echo "Injected $SIG_COUNT dynCall bindings"
|
||||
|
||||
# --- 2. Empty-callback fixes ---------------------------------------------------
|
||||
# Emscripten+pthreads emits some direct-call paths as no-op ((a1)=>{}) stubs that
|
||||
# ARE used. Rewire each to the (now-bound) dynCall_*. (Kept inline: simple sed one-liners.)
|
||||
# --- 1. Empty-callback fixes ---------------------------------------------------
|
||||
# Emscripten+pthreads emits some direct-call paths as no-op ((a1)=>{}) stubs that ARE used. Native
|
||||
# wasm-EH eliminates the invoke_* wrappers, so the .js has no dynCall_<sig> call sites to bind — but
|
||||
# the DYNCALLS=1 trampolines are still EXPORTED on the wasm, so route each function-pointer stub
|
||||
# through wasmExports["dynCall_<sig>"]. This MUST be the wasm trampoline, NOT getWasmTableEntry: the
|
||||
# fiber entry runs a coroutine that suspends+rewinds via Asyncify, and an Asyncify rewind cannot
|
||||
# resume through getWasmTableEntry's JS wrapper — the fiber would re-enter from the top and the tool's
|
||||
# Wait() re-runs (tool_manager ScheduleWait "!pendingWait" assert + busy-loop). The instrumented
|
||||
# dynCall_<sig> export rewinds correctly. (Without these fixes the libcontext fiber entry stays the
|
||||
# empty (a1=>{}) stub, so tool coroutines never start and every GAL app stalls at InvokeTool.)
|
||||
echo "Fixing empty callback arrow functions..."
|
||||
TOTAL_FIXED=0
|
||||
apply_fix() { # <grep/sed pattern> <sed replacement> <label>
|
||||
|
|
@ -112,12 +61,12 @@ apply_fix() { # <grep/sed pattern> <sed replacement> <label>
|
|||
TOTAL_FIXED=$((TOTAL_FIXED + before - after))
|
||||
fi
|
||||
}
|
||||
apply_fix '((a1, a2, a3) => {})(eventTypeId,' '((a1, a2, a3) => dynCall_iiii(callbackfunc, a1, a2, a3))(eventTypeId,' "HTML5 event callback(s) (dynCall_iiii)"
|
||||
apply_fix 'var result = (a1 => {})(arg);' 'var result = dynCall_ii(ptr, arg);' "pthread entry callback(s) (dynCall_ii)"
|
||||
apply_fix 'return (a1 => {})(sig);' 'return dynCall_vi(fp, sig);' "signal handler callback(s) (dynCall_vi)"
|
||||
apply_fix 'var wrapper = () => (a1 => {})(arg);' 'var wrapper = () => dynCall_vi(func, arg);' "async timer callback(s) (dynCall_vi)"
|
||||
apply_fix 'var iterFunc = (() => {});' 'var iterFunc = () => dynCall_v(func);' "main loop callback(s) (dynCall_v)"
|
||||
apply_fix '(a1 => {})(userData);' 'dynCall_vi(entryPoint, userData);' "fiber entry callback(s) (dynCall_vi)"
|
||||
apply_fix '((a1, a2, a3) => {})(eventTypeId,' '((a1, a2, a3) => wasmExports["dynCall_iiii"](callbackfunc, a1, a2, a3))(eventTypeId,' "HTML5 event callback(s) (wasmExports.dynCall_iiii)"
|
||||
apply_fix 'var result = (a1 => {})(arg);' 'var result = wasmExports["dynCall_ii"](ptr, arg);' "pthread entry callback(s) (wasmExports.dynCall_ii)"
|
||||
apply_fix 'return (a1 => {})(sig);' 'return wasmExports["dynCall_vi"](fp, sig);' "signal handler callback(s) (wasmExports.dynCall_vi)"
|
||||
apply_fix 'var wrapper = () => (a1 => {})(arg);' 'var wrapper = () => wasmExports["dynCall_vi"](func, arg);' "async timer callback(s) (wasmExports.dynCall_vi)"
|
||||
apply_fix 'var iterFunc = (() => {});' 'var iterFunc = () => wasmExports["dynCall_v"](func);' "main loop callback(s) (wasmExports.dynCall_v)"
|
||||
apply_fix '(a1 => {})(userData);' 'wasmExports["dynCall_vi"](entryPoint, userData);' "fiber entry callback(s) (wasmExports.dynCall_vi)"
|
||||
echo "Total: Fixed $TOTAL_FIXED empty callback(s)"
|
||||
|
||||
# --- 3. Nested-Asyncify handleSleep fix ---------------------------------------
|
||||
|
|
@ -131,7 +80,15 @@ elif grep -q '__nestedHandleSleepInstalled' "$JS_FILE"; then
|
|||
else
|
||||
HS_MARKER=$(grep -n '^_emscripten_fiber_swap\.isAsync = true;$' "$JS_FILE" | head -1 | cut -d: -f1)
|
||||
if [ -z "$HS_MARKER" ]; then
|
||||
echo "Warning: _emscripten_fiber_swap.isAsync marker not found - skipping handleSleep fix"
|
||||
# No libcontext fiber glue (a non-fiber Asyncify app — e.g. a plain wx app with
|
||||
# modals/menus, no tool coroutines). The currData save/restore is still needed:
|
||||
# without it a rewind resuming through a fresh wasm re-entry hits
|
||||
# _asyncify_start_rewind(null) -> "memory access out of bounds" (the context-menu
|
||||
# pick while the main loop is Asyncify-parked). Append at EOF — Asyncify is defined
|
||||
# by then and the shim wraps handleSleep at load, before any runtime sleep.
|
||||
echo "" >> "$JS_FILE"
|
||||
cat "$SHIM_DIR/handlesleep.js" >> "$JS_FILE"
|
||||
echo "Injected handleSleep fix at EOF (no fiber glue)"
|
||||
else
|
||||
head -n "$HS_MARKER" "$JS_FILE" > "${JS_FILE}.tmp"
|
||||
echo "" >> "${JS_FILE}.tmp"
|
||||
|
|
@ -158,6 +115,23 @@ else
|
|||
echo "Warning: dynCallLegacy pattern not found - skipping embind dynCall fallback"
|
||||
fi
|
||||
|
||||
# --- 3d. Embind invoker: don't Promise-wrap a SYNCHRONOUS call when the main loop is parked --------
|
||||
# Emscripten's embind invoker returns a Promise iff Asyncify.currData is set AFTER the wasm call. But
|
||||
# the native-EH per-frame-yield main loop parks via Asyncify (currData stays SET between frames), so a
|
||||
# JS-initiated embind call (e.g. kicadCollabSnapshot from a test or the UI) that does NOT itself
|
||||
# suspend is mis-detected as async and returns "[object Promise]" instead of the value -> the caller's
|
||||
# JSON.parse(...) gets "[object Promise]". Capture currData before the call and only treat it as async
|
||||
# if THIS call left a NEW currData. Harmless under legacy/JS-EH (currData is null when the app is idle).
|
||||
if grep -q 'Asyncify.currData !== __ehPrev' "$JS_FILE"; then
|
||||
echo "embind invoker currData re-entrancy fix already present - skipping"
|
||||
elif grep -q 'return Asyncify.currData ? Asyncify.whenDone' "$JS_FILE"; then
|
||||
perl -0pi -e 's/(invokerFnBody \+= \(returns \|\| isAsync \? "var rv = " : ""\))/invokerFnBody += "var __ehPrev = Asyncify.currData;\\n";\n $1/' "$JS_FILE"
|
||||
perl -0pi -e 's/return Asyncify\.currData \? Asyncify\.whenDone/return (Asyncify.currData && Asyncify.currData !== __ehPrev) ? Asyncify.whenDone/' "$JS_FILE"
|
||||
echo "Injected embind invoker currData re-entrancy fix"
|
||||
else
|
||||
echo "Warning: embind invoker currData pattern not found - skipping embind re-entrancy fix"
|
||||
fi
|
||||
|
||||
# --- 3c. Fiber trampoline self-heal -------------------------------------------
|
||||
# emscripten_set_main_loop(...,1) throws "unwind" during startup to establish the
|
||||
# main loop. KiCad establishes that loop from inside a tool coroutine, so the throw
|
||||
|
|
|
|||
|
|
@ -1,25 +0,0 @@
|
|||
// Template for one dynCall_<sig> binding. Placeholders substituted per signature
|
||||
// by inject-dyncall-shims.sh: @SIG@ = signature, @ARGS@ = "index, a0, a1, ...",
|
||||
// @CALLARGS@ = "a0, a1, ..." (the args without the function-pointer index).
|
||||
//
|
||||
// Binds the bare name to the REAL asyncify-instrumented wasm export
|
||||
// (wasmExports["dynCall_<sig>"], present because the build links -sDYNCALLS=1).
|
||||
// Falls back to getWasmTableEntry if no such export exists, OR if the instrumented
|
||||
// trampoline traps with "indirect call signature mismatch": post-asyncify+O2 the
|
||||
// trampoline can call_indirect with a stale type for some table indices even though
|
||||
// the table entry itself is valid (hit by programmatic editor edits, e.g. eeschema
|
||||
// SCH_ITEM::Move via invoke_vii — see features/yjs-bridge/0003). The direct
|
||||
// getWasmTableEntry call uses the correct per-entry signature and succeeds. Only the
|
||||
// mismatch trap is caught; the Asyncify unwind sentinel and real exceptions re-throw,
|
||||
// so instrumentation/unwind still work for every normal indirect call.
|
||||
function dynCall_@SIG@(@ARGS@) {
|
||||
var f = (typeof wasmExports !== 'undefined') && wasmExports["dynCall_@SIG@"];
|
||||
if (f) {
|
||||
try { return f(@ARGS@); }
|
||||
catch (_dce) {
|
||||
if (!(_dce instanceof WebAssembly.RuntimeError) || !/signature mismatch/.test(_dce.message))
|
||||
throw _dce;
|
||||
}
|
||||
}
|
||||
return getWasmTableEntry(index)(@CALLARGS@);
|
||||
}
|
||||
Loading…
Reference in a new issue