pcbjam/scripts/binaryen-hoist-pass/build-wasm-opt.sh

51 lines
2.6 KiB
Shell
Raw Normal View History

feat(wasm-eh): migrate the WASM build to native wasm exceptions (+ 3D viewer default-on) Replace the legacy Emscripten JS-exceptions model with native wasm-EH (legacy encoding) across the whole build, keeping Asyncify coroutines working via a from-source Binaryen --hoist-cpp-catches pre-pass. Net result: native-EH is the only build mode, the 3D viewer is on by default, and pcbnew shrinks substantially. Highlights: - Binaryen submodule everywhere + --hoist-cpp-catches integration in apply-asyncify; post-link Asyncify covers every app wasm (not just standalone test wasm). - Build deps (incl. OpenCASCADE without OCC_CONVERT_SIGNALS) and all KiCad apps with -fwasm-exceptions; emscripten_sleep added to the post-link asyncify-imports. - libcontext fiber entry wired under native exceptions; while-loop main loop + currData shim injected into all wx apps. - Native-EH collab apply fixed: DEBUG-define the embind TU + match all out-of-CMake C++ TUs' ABI flags to the core, fixing the vtable-layout skew / mis-dispatch. - 3D viewer enabled by default (real raytracer linked, not the stub). - Retire the EH-spike scaffolding; flip the asyncify-races ablation pins to shim-redundancy pins (native-EH stays clean with the legacy shims ablated). - Fix the asyncify-races quiescence check to not require Asyncify.currData==0: under the native-EH per-frame-yield top loop the main stack is asyncify-suspended every frame, so currData legitimately churns (a freed-but-not-yet-nulled buffer, not a leak). Refresh the pcbnew toolbar screenshot baseline for the new kicad. - CI: drop the obsolete binaryen_version input/env (the build uses the binaryen submodule fork's wasm-opt, not a version download); key the wasm-output cache on the binaryen submodule SHA instead. Bumps the wxwidgets + binaryen submodules to their squashed feature commits. Validated green: all 7 apps native-EH (real 3D in pcbnew); KiCad e2e 63/63 Firefox + Chromium (3D viewer renders); wx 336; coroutine 34/34 both engines; asyncify 7/7 both engines. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:50:18 +02:00
#!/bin/bash
fix(wasm-eh): build wasm-emscripten-finalize from the Binaryen submodule (unbreak CI) The native-EH migration pointed apply-finalize.sh at emsdk's bundled wasm-emscripten-finalize (${EMSDK:-tools/emsdk}/upstream/bin), which exists on a dev machine (tools/emsdk persists from a prior build) but never on the ephemeral CI host — emsdk lives only inside the Docker image, and nothing on the host post-process path provisions it. So the host post-process died at finalize for every app once the binaryen-SHA cache re-key forced it to actually run. (Not the missing-ninja theory — ninja was installed; the run never reached the wasm-opt build, which is downstream of finalize.) Finish the "Binaryen submodule everywhere" migration: build-wasm-opt.sh now also builds wasm-emscripten-finalize, and apply-finalize.sh takes it from that build (next to wasm-opt). The host post-process is now emsdk-free (dyncall=node, finalize+asyncify=submodule v130) and finalize/wasm-opt share one Binaryen version (previously finalize was emsdk's v121, wasm-opt the submodule's v130). get-wasm-opt.sh stays for bench only; dropped it from the wasm cache key and added build-wasm-opt.sh. Also surface the from-source Binaryen build as a "Build Binaryen" stage in build-monitor.sh (it had no marker), emitted at the pipelined pre-warm sites. Validated: a clean from-scratch `docker/build.sh calculator --build-deps` (deps + compile + finalize via the submodule binary + asyncify + -O2) succeeded end-to-end; finalize resolved to build-wasm/tools/binaryen-hoist-build/bin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 11:55:52 +02:00
# Build the host post-process Binaryen tools from our submodule: wasm-opt (with the
# catch-arm-hoisting pass, --hoist-cpp-catches) AND wasm-emscripten-finalize.
feat(wasm-eh): migrate the WASM build to native wasm exceptions (+ 3D viewer default-on) Replace the legacy Emscripten JS-exceptions model with native wasm-EH (legacy encoding) across the whole build, keeping Asyncify coroutines working via a from-source Binaryen --hoist-cpp-catches pre-pass. Net result: native-EH is the only build mode, the 3D viewer is on by default, and pcbnew shrinks substantially. Highlights: - Binaryen submodule everywhere + --hoist-cpp-catches integration in apply-asyncify; post-link Asyncify covers every app wasm (not just standalone test wasm). - Build deps (incl. OpenCASCADE without OCC_CONVERT_SIGNALS) and all KiCad apps with -fwasm-exceptions; emscripten_sleep added to the post-link asyncify-imports. - libcontext fiber entry wired under native exceptions; while-loop main loop + currData shim injected into all wx apps. - Native-EH collab apply fixed: DEBUG-define the embind TU + match all out-of-CMake C++ TUs' ABI flags to the core, fixing the vtable-layout skew / mis-dispatch. - 3D viewer enabled by default (real raytracer linked, not the stub). - Retire the EH-spike scaffolding; flip the asyncify-races ablation pins to shim-redundancy pins (native-EH stays clean with the legacy shims ablated). - Fix the asyncify-races quiescence check to not require Asyncify.currData==0: under the native-EH per-frame-yield top loop the main stack is asyncify-suspended every frame, so currData legitimately churns (a freed-but-not-yet-nulled buffer, not a leak). Refresh the pcbnew toolbar screenshot baseline for the new kicad. - CI: drop the obsolete binaryen_version input/env (the build uses the binaryen submodule fork's wasm-opt, not a version download); key the wasm-output cache on the binaryen submodule SHA instead. Bumps the wxwidgets + binaryen submodules to their squashed feature commits. Validated green: all 7 apps native-EH (real 3D in pcbnew); KiCad e2e 63/63 Firefox + Chromium (3D viewer renders); wx 336; coroutine 34/34 both engines; asyncify 7/7 both engines. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:50:18 +02:00
#
# Source of truth is the tracked Binaryen submodule (binaryen/, branch wasm-port =
# upstream version_130 + src/passes/HoistCppCatches.cpp). This configures an out-of-source
# build into the gitignored build-wasm/ tree and prints the wasm-opt path on stdout
fix(wasm-eh): build wasm-emscripten-finalize from the Binaryen submodule (unbreak CI) The native-EH migration pointed apply-finalize.sh at emsdk's bundled wasm-emscripten-finalize (${EMSDK:-tools/emsdk}/upstream/bin), which exists on a dev machine (tools/emsdk persists from a prior build) but never on the ephemeral CI host — emsdk lives only inside the Docker image, and nothing on the host post-process path provisions it. So the host post-process died at finalize for every app once the binaryen-SHA cache re-key forced it to actually run. (Not the missing-ninja theory — ninja was installed; the run never reached the wasm-opt build, which is downstream of finalize.) Finish the "Binaryen submodule everywhere" migration: build-wasm-opt.sh now also builds wasm-emscripten-finalize, and apply-finalize.sh takes it from that build (next to wasm-opt). The host post-process is now emsdk-free (dyncall=node, finalize+asyncify=submodule v130) and finalize/wasm-opt share one Binaryen version (previously finalize was emsdk's v121, wasm-opt the submodule's v130). get-wasm-opt.sh stays for bench only; dropped it from the wasm cache key and added build-wasm-opt.sh. Also surface the from-source Binaryen build as a "Build Binaryen" stage in build-monitor.sh (it had no marker), emitted at the pipelined pre-warm sites. Validated: a clean from-scratch `docker/build.sh calculator --build-deps` (deps + compile + finalize via the submodule binary + asyncify + -O2) succeeded end-to-end; finalize resolved to build-wasm/tools/binaryen-hoist-build/bin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 11:55:52 +02:00
# (build progress to stderr); wasm-emscripten-finalize lands next to it in the same bin/
# (apply-finalize.sh derives it from the wasm-opt dir). Building both from one submodule
# keeps finalize and asyncify on a single Binaryen version and removes the host emsdk
# dependency from the post-process. See docs/features/wasm-exceptions/06-spike-plan.md (Phase 1.5).
feat(wasm-eh): migrate the WASM build to native wasm exceptions (+ 3D viewer default-on) Replace the legacy Emscripten JS-exceptions model with native wasm-EH (legacy encoding) across the whole build, keeping Asyncify coroutines working via a from-source Binaryen --hoist-cpp-catches pre-pass. Net result: native-EH is the only build mode, the 3D viewer is on by default, and pcbnew shrinks substantially. Highlights: - Binaryen submodule everywhere + --hoist-cpp-catches integration in apply-asyncify; post-link Asyncify covers every app wasm (not just standalone test wasm). - Build deps (incl. OpenCASCADE without OCC_CONVERT_SIGNALS) and all KiCad apps with -fwasm-exceptions; emscripten_sleep added to the post-link asyncify-imports. - libcontext fiber entry wired under native exceptions; while-loop main loop + currData shim injected into all wx apps. - Native-EH collab apply fixed: DEBUG-define the embind TU + match all out-of-CMake C++ TUs' ABI flags to the core, fixing the vtable-layout skew / mis-dispatch. - 3D viewer enabled by default (real raytracer linked, not the stub). - Retire the EH-spike scaffolding; flip the asyncify-races ablation pins to shim-redundancy pins (native-EH stays clean with the legacy shims ablated). - Fix the asyncify-races quiescence check to not require Asyncify.currData==0: under the native-EH per-frame-yield top loop the main stack is asyncify-suspended every frame, so currData legitimately churns (a freed-but-not-yet-nulled buffer, not a leak). Refresh the pcbnew toolbar screenshot baseline for the new kicad. - CI: drop the obsolete binaryen_version input/env (the build uses the binaryen submodule fork's wasm-opt, not a version download); key the wasm-output cache on the binaryen submodule SHA instead. Bumps the wxwidgets + binaryen submodules to their squashed feature commits. Validated green: all 7 apps native-EH (real 3D in pcbnew); KiCad e2e 63/63 Firefox + Chromium (3D viewer renders); wx 336; coroutine 34/34 both engines; asyncify 7/7 both engines. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:50:18 +02:00
set -eo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
SRC="${PROJECT_ROOT}/binaryen"
BUILD="${PROJECT_ROOT}/build-wasm/tools/binaryen-hoist-build"
if [ ! -f "${SRC}/src/passes/HoistCppCatches.cpp" ]; then
echo "ERROR: binaryen submodule is missing the hoist pass." >&2
echo " Run: git submodule update --init binaryen" >&2
exit 1
fi
# CI fast-path: the workflow cache-restores bin/ + lib/ keyed on the exact submodule
# SHA and sets this var on a hit, so the restored binaries are authoritative — skip
# cmake+ninja. Trust only if both tools actually RUN: the binaries dynamically link
# lib/libbinaryen.so, so an existence check alone passes on an incomplete restore
# (bin/ without lib/ shipped a red main, run 28585074335) while --version proves the
# loader resolves everything. Never set the var locally when iterating on the pass:
# uncommitted source edits would be silently ignored (the SHA key can't see them).
perf(ci): cache the per-run toolchain + test-app work (binaryen, emsdk, apps, browsers) Four new caches in wasm-build.yml, keyed on their real inputs: - Binaryen post-process tools (bin/ only, keyed on the submodule SHA): on a hit BINARYEN_TRUST_PREBUILT=1 tells build-wasm-opt.sh to skip cmake+ninja and trust the restored binaries (~46s/run). The guard is env-gated so local pass iteration (uncommitted sources, same SHA) is unaffected. - Built wx test apps (tests/apps minus kicad/ staging + gal-webgl/): skips the whole build step on a hit (~3 min — the post-link hoist+asyncify over ~74 apps dominates). Key covers wx SHA + kicad SHA (some apps compile real KiCad sources: thread_pool.cpp, libcontext), binaryen SHA, tracked app sources, and the build/post-link scripts + shims. Gated on a wx cache HIT: the app build creates the libwx_*.a symlinks the GAL link needs, so on a wx rebuild the apps must rebuild too. - Host emsdk (tools/emsdk, keyed on the pinned EMSCRIPTEN_VERSION): mostly availability insurance — a fresh install is only ~23s but pulls ~340 MB from github.com + storage.googleapis.com on every run. downloads/ tarballs are pruned before the post-job save. - Playwright browsers (~/.cache/ms-playwright keyed on tests/package-lock). Warm-cache runs should drop from ~12 min to ~8-8.5 min (e2e suites are the floor). Measured baselines from runs 28577824366 / 28520845242. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XeiSRRScdaox5jBueJNcyG
2026-07-02 12:40:22 +02:00
if [ "${BINARYEN_TRUST_PREBUILT:-0}" = "1" ] \
&& "${BUILD}/bin/wasm-opt" --version >/dev/null 2>&1 \
&& "${BUILD}/bin/wasm-emscripten-finalize" --version >/dev/null 2>&1; then
perf(ci): cache the per-run toolchain + test-app work (binaryen, emsdk, apps, browsers) Four new caches in wasm-build.yml, keyed on their real inputs: - Binaryen post-process tools (bin/ only, keyed on the submodule SHA): on a hit BINARYEN_TRUST_PREBUILT=1 tells build-wasm-opt.sh to skip cmake+ninja and trust the restored binaries (~46s/run). The guard is env-gated so local pass iteration (uncommitted sources, same SHA) is unaffected. - Built wx test apps (tests/apps minus kicad/ staging + gal-webgl/): skips the whole build step on a hit (~3 min — the post-link hoist+asyncify over ~74 apps dominates). Key covers wx SHA + kicad SHA (some apps compile real KiCad sources: thread_pool.cpp, libcontext), binaryen SHA, tracked app sources, and the build/post-link scripts + shims. Gated on a wx cache HIT: the app build creates the libwx_*.a symlinks the GAL link needs, so on a wx rebuild the apps must rebuild too. - Host emsdk (tools/emsdk, keyed on the pinned EMSCRIPTEN_VERSION): mostly availability insurance — a fresh install is only ~23s but pulls ~340 MB from github.com + storage.googleapis.com on every run. downloads/ tarballs are pruned before the post-job save. - Playwright browsers (~/.cache/ms-playwright keyed on tests/package-lock). Warm-cache runs should drop from ~12 min to ~8-8.5 min (e2e suites are the floor). Measured baselines from runs 28577824366 / 28520845242. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XeiSRRScdaox5jBueJNcyG
2026-07-02 12:40:22 +02:00
echo "Using prebuilt Binaryen tools (BINARYEN_TRUST_PREBUILT=1): ${BUILD}/bin" >&2
echo "${BUILD}/bin/wasm-opt"
exit 0
fi
feat(wasm-eh): migrate the WASM build to native wasm exceptions (+ 3D viewer default-on) Replace the legacy Emscripten JS-exceptions model with native wasm-EH (legacy encoding) across the whole build, keeping Asyncify coroutines working via a from-source Binaryen --hoist-cpp-catches pre-pass. Net result: native-EH is the only build mode, the 3D viewer is on by default, and pcbnew shrinks substantially. Highlights: - Binaryen submodule everywhere + --hoist-cpp-catches integration in apply-asyncify; post-link Asyncify covers every app wasm (not just standalone test wasm). - Build deps (incl. OpenCASCADE without OCC_CONVERT_SIGNALS) and all KiCad apps with -fwasm-exceptions; emscripten_sleep added to the post-link asyncify-imports. - libcontext fiber entry wired under native exceptions; while-loop main loop + currData shim injected into all wx apps. - Native-EH collab apply fixed: DEBUG-define the embind TU + match all out-of-CMake C++ TUs' ABI flags to the core, fixing the vtable-layout skew / mis-dispatch. - 3D viewer enabled by default (real raytracer linked, not the stub). - Retire the EH-spike scaffolding; flip the asyncify-races ablation pins to shim-redundancy pins (native-EH stays clean with the legacy shims ablated). - Fix the asyncify-races quiescence check to not require Asyncify.currData==0: under the native-EH per-frame-yield top loop the main stack is asyncify-suspended every frame, so currData legitimately churns (a freed-but-not-yet-nulled buffer, not a leak). Refresh the pcbnew toolbar screenshot baseline for the new kicad. - CI: drop the obsolete binaryen_version input/env (the build uses the binaryen submodule fork's wasm-opt, not a version download); key the wasm-output cache on the binaryen submodule SHA instead. Bumps the wxwidgets + binaryen submodules to their squashed feature commits. Validated green: all 7 apps native-EH (real 3D in pcbnew); KiCad e2e 63/63 Firefox + Chromium (3D viewer renders); wx 336; coroutine 34/34 both engines; asyncify 7/7 both engines. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:50:18 +02:00
# Configure once (mirrors scripts/common/get-wasm-opt.sh's from-source flags).
if [ ! -f "${BUILD}/build.ninja" ]; then
echo "Configuring Binaryen submodule build (one-time, ~5 min to build)..." >&2
cmake -S "${SRC}" -B "${BUILD}" -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CXX_FLAGS="-Wno-maybe-uninitialized" \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON -DBUILD_TESTS=OFF >&2
fi
fix(wasm-eh): build wasm-emscripten-finalize from the Binaryen submodule (unbreak CI) The native-EH migration pointed apply-finalize.sh at emsdk's bundled wasm-emscripten-finalize (${EMSDK:-tools/emsdk}/upstream/bin), which exists on a dev machine (tools/emsdk persists from a prior build) but never on the ephemeral CI host — emsdk lives only inside the Docker image, and nothing on the host post-process path provisions it. So the host post-process died at finalize for every app once the binaryen-SHA cache re-key forced it to actually run. (Not the missing-ninja theory — ninja was installed; the run never reached the wasm-opt build, which is downstream of finalize.) Finish the "Binaryen submodule everywhere" migration: build-wasm-opt.sh now also builds wasm-emscripten-finalize, and apply-finalize.sh takes it from that build (next to wasm-opt). The host post-process is now emsdk-free (dyncall=node, finalize+asyncify=submodule v130) and finalize/wasm-opt share one Binaryen version (previously finalize was emsdk's v121, wasm-opt the submodule's v130). get-wasm-opt.sh stays for bench only; dropped it from the wasm cache key and added build-wasm-opt.sh. Also surface the from-source Binaryen build as a "Build Binaryen" stage in build-monitor.sh (it had no marker), emitted at the pipelined pre-warm sites. Validated: a clean from-scratch `docker/build.sh calculator --build-deps` (deps + compile + finalize via the submodule binary + asyncify + -O2) succeeded end-to-end; finalize resolved to build-wasm/tools/binaryen-hoist-build/bin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 11:55:52 +02:00
ninja -C "${BUILD}" wasm-opt wasm-emscripten-finalize >&2
feat(wasm-eh): migrate the WASM build to native wasm exceptions (+ 3D viewer default-on) Replace the legacy Emscripten JS-exceptions model with native wasm-EH (legacy encoding) across the whole build, keeping Asyncify coroutines working via a from-source Binaryen --hoist-cpp-catches pre-pass. Net result: native-EH is the only build mode, the 3D viewer is on by default, and pcbnew shrinks substantially. Highlights: - Binaryen submodule everywhere + --hoist-cpp-catches integration in apply-asyncify; post-link Asyncify covers every app wasm (not just standalone test wasm). - Build deps (incl. OpenCASCADE without OCC_CONVERT_SIGNALS) and all KiCad apps with -fwasm-exceptions; emscripten_sleep added to the post-link asyncify-imports. - libcontext fiber entry wired under native exceptions; while-loop main loop + currData shim injected into all wx apps. - Native-EH collab apply fixed: DEBUG-define the embind TU + match all out-of-CMake C++ TUs' ABI flags to the core, fixing the vtable-layout skew / mis-dispatch. - 3D viewer enabled by default (real raytracer linked, not the stub). - Retire the EH-spike scaffolding; flip the asyncify-races ablation pins to shim-redundancy pins (native-EH stays clean with the legacy shims ablated). - Fix the asyncify-races quiescence check to not require Asyncify.currData==0: under the native-EH per-frame-yield top loop the main stack is asyncify-suspended every frame, so currData legitimately churns (a freed-but-not-yet-nulled buffer, not a leak). Refresh the pcbnew toolbar screenshot baseline for the new kicad. - CI: drop the obsolete binaryen_version input/env (the build uses the binaryen submodule fork's wasm-opt, not a version download); key the wasm-output cache on the binaryen submodule SHA instead. Bumps the wxwidgets + binaryen submodules to their squashed feature commits. Validated green: all 7 apps native-EH (real 3D in pcbnew); KiCad e2e 63/63 Firefox + Chromium (3D viewer renders); wx 336; coroutine 34/34 both engines; asyncify 7/7 both engines. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:50:18 +02:00
echo "${BUILD}/bin/wasm-opt"