Commit graph pcbjam/scripts/common/env.sh
Author SHA1 Message Date
Viktor Vaczi
9c475a804e jspi cleanup: remove the asyncify-era residue — dead code, conditionals, pipeline scaffolding, stale prose
The runtime is JSPI-only; this removes everything that still pretended
otherwise. Three exhaustive sweeps (C++/JS+build+CI/tests+docs) drove
the inventory; every deletion verified by grep closure + full gates.

Broken-right-now fixes:
- deploy-staging.yml passed the retired opt_level input — the workflow
  could not even start. Removed.
- env.sh carried dead exports with a live -sASYNCIFY=1 inside
  (WASM_LDFLAGS/PTHREAD_LDFLAGS, zero consumers). Removed; the
  WASM_LEGACY_EXCEPTIONS rationale rewritten to the real reason.
- docker/build.sh exported PCBJAM_ASYNC_BACKEND (read nowhere). Gone.

Dead weight removed:
- binaryen submodule (nothing builds or invokes it), wasm-opt-bench
  workflow + scripts/bench/, get-wasm-opt.sh, diagnostics.js (242 lines
  of Asyncify-API-only code), the KICAD_PIPELINE background-postprocess
  scaffolding (existed to parallelize the deleted wasm-opt phase; the
  postprocess is a seconds-long node script and now runs inline),
  build-monitor's dead asyncify rows, sched-context orphan build
  output, dead .gitignore entries, the .jspi-assets spike dir (the two
  wf-result research JSONs moved to docs/features/async/migration-evidence/).
- bindings: fiber_park.h + its 12 embind registrations (broken-if-
  called under JSPI), the kicadOpenFileStart/OPEN_JOB starter route,
  main_stack_runner.h + 5 includes, the always-null context-sleep weak
  hook in nanosleep_yield.c.
- shim: the backend field (installed-flag idempotency instead),
  noteContextWait (dead both sides), the __wxAsyncifyDump alias (+ the
  WasmTool fallback and string-dump normalize branch).
- web: the emscripten-6-ignored mainScriptUrlOrBlob option in boot.ts
  (gerber-demo keeps it: it loads the deployed CDN release, which
  predates emscripten 6 — noted inline).

Conditionals: all 'backend === jspi' checks reduced to scheduler-
presence checks; races_quiescent re-keyed from Asyncify.state (vacuous)
to real backlog quiescence (resumeReady/mutatorQueue — NOT _windowLive,
which is the probing activation's own window by definition).

Renames (identifiers only, no file renames): ASYNC_LINK_FLAGS→
JSPI_LINK_FLAGS and Makefile ASYNC_LDFLAGS→JSPI_LDFLAGS,
kicadCollabFiberBusy→kicadCollabBusy (embind + web + tests),
collab_common.h fiber*→apply*/coroutine naming, asyncifySignatures→
wasmTrapSignatures (lists byte-identical).

Tests: the two remaining vacuous [wx-asyncify]/fiber-resume-refused
asserts re-keyed to live JSPI beacons; eeschema-load's failure message
no longer sends the developer to a deleted script; wait-beacons' dead
families/parser deleted; lane-0 legacy-glue guards removed (lane 0 is
unconstructible); the embind test.fail re-gated with the JSPI reason
(plain embind invokers cannot suspend — verified still failing);
lint-determinism now scans tests/jspi (166 files clean);
eeschema-collab local-move gated to chromium (~50% flaky on FF even
solo; pcbnew twin covers both engines).

Docs: DEBUG.md rewritten as the JSPI debugging guide; build.md
describes the single-phase build; docs/features/async/README.md
banner-marked historical and repointed at the NEW
23-jspi-runtime.md (current architecture: export census, turnstile,
libcontext ownership + refusal contract, embind call shapes, the
em-pthread service-wrapper trick, exception policy, known gaps).

Gates on the cleaned tree: test:e2e 725 passed / 0 failed (after the
quiescence-probe fix; the 3 other reds were verified contention flakes
solo-green or the documented FF gate), web 76/0, jspi 18/18 both
engines, vitest 295/295 + 17/17, all lints green, live-app census
clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016X9eh1s5sTx1o9Em9KBuwR
2026-08-14 09:25:32 +02:00
Viktor Vaczi
c1ef489cfa feat(wasm-eh): migrate the WASM build to native wasm exceptions (+ 3D viewer default-on)
Replace the legacy Emscripten JS-exceptions model with native wasm-EH (legacy
encoding) across the whole build, keeping Asyncify coroutines working via a
from-source Binaryen --hoist-cpp-catches pre-pass. Net result: native-EH is the
only build mode, the 3D viewer is on by default, and pcbnew shrinks substantially.

Highlights:
- Binaryen submodule everywhere + --hoist-cpp-catches integration in apply-asyncify;
  post-link Asyncify covers every app wasm (not just standalone test wasm).
- Build deps (incl. OpenCASCADE without OCC_CONVERT_SIGNALS) and all KiCad apps
  with -fwasm-exceptions; emscripten_sleep added to the post-link asyncify-imports.
- libcontext fiber entry wired under native exceptions; while-loop main loop +
  currData shim injected into all wx apps.
- Native-EH collab apply fixed: DEBUG-define the embind TU + match all out-of-CMake
  C++ TUs' ABI flags to the core, fixing the vtable-layout skew / mis-dispatch.
- 3D viewer enabled by default (real raytracer linked, not the stub).
- Retire the EH-spike scaffolding; flip the asyncify-races ablation pins to
  shim-redundancy pins (native-EH stays clean with the legacy shims ablated).
- Fix the asyncify-races quiescence check to not require Asyncify.currData==0:
  under the native-EH per-frame-yield top loop the main stack is asyncify-suspended
  every frame, so currData legitimately churns (a freed-but-not-yet-nulled buffer,
  not a leak). Refresh the pcbnew toolbar screenshot baseline for the new kicad.
- CI: drop the obsolete binaryen_version input/env (the build uses the binaryen
  submodule fork's wasm-opt, not a version download); key the wasm-output cache on
  the binaryen submodule SHA instead.

Bumps the wxwidgets + binaryen submodules to their squashed feature commits.

Validated green: all 7 apps native-EH (real 3D in pcbnew); KiCad e2e 63/63
Firefox + Chromium (3D viewer renders); wx 336; coroutine 34/34 both engines;
asyncify 7/7 both engines.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 09:40:26 +02:00
Istvan Matejcsok
9733400734 chore: 🤖 add new worktree setup script 2026-06-16 14:46:18 +02:00
Viktor Vaczi
7dc1bf27e0 refactor: collapse dual-mode plumbing — the DOM port is the only WASM build
The canvas (wxUniversal) mode is gone (wxwidgets submodule); remove every
piece of side-by-side plumbing so there is exactly one build and one test
flow:

- scripts/build-wxuniversal-wasm.sh -> scripts/build-wx-wasm.sh; no
  --dom/--enable-universal; builds into build-wasm/wxwidgets
- build-wasm-test.sh: no DOM_BUILD / apps-dom rsync mirror / PORT=dom;
  apps build straight into tests/apps (Makefile.wasm PORT conditionals
  collapsed; wx.js + wx-dom.js always pre-js)
- docker/build.sh, build-kicad-target.sh, env.sh: WX_PORT / -dom /
  -universal suffixes removed; kicad builds to kicad-<app>, outputs to
  output/; wx.js/wx-dom.js copied from the real source path
  (/workspace/wxwidgets/build/wasm — the old build-wasm path never
  existed and silently failed)
- setup-kicad-wasm.sh: single target dir; the perl wx-dom.js injection is
  gone — the 7 checked-in kicad pages now reference wx-dom.js directly
- playwright configs serve apps/; fixtures drop the test-results/dom and
  logs/wxwidgets/dom namespacing; boot.spec asserts wxDomPort
  unconditionally; pcbnew.spec uses one reference image;
  appearance.spec assertions unconditional
- compare/update-baseline-screenshots.sh: --port removed
- tests/gal-regression/wasm/Makefile: links build-wasm/wxwidgets and
  carries wx-dom.js as a second pre-js — the gal-webgl suite (30 specs)
  now actually builds and runs here (it needed host-side boost+glm via
  scripts/deps; the bundle had been missing, timing the whole spec out)
- tests: clickCanvas() dispatches via page.mouse (DOM widgets
  legitimately cover the canvas; locator actionability refused the
  click); the comprehensive spec drives wxChoice through its native
  <select> (browser-owned popup cannot be coordinate-clicked)
- docs: README/CLAUDE.md/build.md script names and dirs;
  features/wx-dom-port README reframed (DOM is THE port), visual-notes
  bugs 26-28; FindwxWidgets.cmake config label drops 'wasmuniv'
- wxwidgets submodule -> 9dbacc9448 (DOM-only port, fork diff shrunk)

Gate: full wx e2e suite 292 passed / 1 skipped / 0 failed — first run
ever with the gal-webgl specs green (28 scenarios + load + sequential).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:19:48 +02:00
Viktor Vaczi
d557b21eac fix(docker): Replace emsdk image with Ubuntu + emsdk from source
The Docker build used emscripten/emsdk:4.0.2-arm64 as base image but
env.sh couldn't find emsdk there, installing a second copy. The build
then applied wasm-opt/finalize stubs to the wrong emsdk (hardcoded
/emsdk/), so the real wasm-emscripten-finalize ran in Docker and got
OOM-killed.

- Use ubuntu:22.04 base with emsdk installed from source at /emsdk/
- Make stub paths dynamic via $EMSDK instead of hardcoded /emsdk/
- Skip local emsdk install in env.sh when $EMSDK is already active

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 12:44:26 +01:00
Viktor Vaczi
b46b4b69f1 fix(wasm): Fix asyncify rewind with asyncify-aware dynCall shims
Emscripten 4.x removed dynCall_* WASM exports, breaking asyncify
rewind through indirect calls (modal dialogs, event handlers).
Generate JS shims that track Asyncify.exportCallStack and register
in wasmExports so doRewind can find them.

Also fixes empty callback functions ((() => {})) generated by
Emscripten 4.x + pthreads for HTML5 events, pthread entry,
sighandler, async timer, and main loop callbacks.

Build pipeline improvements:
- Stub wasm-opt/finalize in Docker (RAM limits), run on host
- Add setup-emsdk.sh for reproducible Emscripten setup
- Simplify env.sh and version management

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:26:48 +01:00
Viktor Vaczi
a4f444fea8 feat(webgl): Add WebGL GAL test infrastructure (Phase 1)
Add complete test infrastructure for WebGL GAL visual regression testing:

- scripts/test-gal-regression.sh: Master script that builds both backends,
  runs tests, and performs two-level comparison (native vs baseline,
  webgl vs native)
- scripts/build-gal-webgl-test.sh: WASM build using Makefile with em++
- tests/gal-regression/wasm/: WebGL test harness (stub WEBGL_GAL)
- tests/e2e/gal-webgl.spec.ts: Playwright test for screenshot capture

Fix Homebrew Emscripten environment in scripts/common/env.sh:
- Set EMSDK_PYTHON for Python 3.10+ (em++ reads this, not $PYTHON)
- Add bundled LLVM to PATH (Emscripten needs its clang with WASM backend)

Verified: Native vs Baseline passes (28/28), WebGL generates blank
screenshots as expected (WEBGL_GAL implementation is Phase 2).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 16:11:31 +01:00
Viktor Vaczi
dafef9c0a1 Fix Docker macOS timestamp issues and improve build reproducibility
- Add rsync-based source sync for Docker builds to fix macOS VirtioFS
  timestamp inconsistencies that caused autoconf sanity checks to fail
- Add config.sub wrapper (SHELL/CONFIG_SHELL) to support emscripten/wasm32
  targets without modifying submodule files
- Fix protobuf build to use native gcc/g++ for protoc instead of Emscripten
- Add python PATH fix for macOS (Homebrew's python3 symlink) in wxWidgets build

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 01:07:44 +01:00
Viktor Vaczi
5b919ef66f Add ccache and incremental build system for faster development
- Add ccache to Docker image for compiled object caching
- Change build defaults: incremental by default, skip deps by default
- Add new flags: --full, --clean-kicad, --build-deps
- Skip wxWidgets configure if already configured (check Makefile timestamps)
- Remove unused source hash stamp functions (make handles dependencies)
- Update build.md with new build system documentation

Performance improvements:
- Single file change: 7:35 → 1:34 (4.8x faster)
- No-change rebuild: 7:35 → 1:31 (5x faster)
- Asyncify post-processing (~1 min) is now the bottleneck

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-26 20:51:58 +01:00
Viktor Vaczi
9c271996c2 Fix KiCad WASM build and runtime errors
Build improvements:
- Change -O0 to -O1 to fix "local count too large" asyncify error
- Add wasm-emscripten-finalize to host (Docker OOMs on large WASM)
- Use -gseparate-dwarf for smaller main binary with debug info
- Build native protoc for code generation
- Add more functions to asyncify removelist

Runtime fixes:
- Add inject-dyncall-shims.sh to fix "dynCall_* is not defined" in Emscripten 4.x
- Update wxwidgets with browserInfo.name fix

New stubs and bindings:
- Add Embind bindings for JavaScript interop
- Add stubs for scripting, API plugin, PCB frame, navlib

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-22 16:30:53 +01:00
Viktor Vaczi
af2adda002 Improve build system with debug symbols, clean modes, and parallel jobs
- Add build.md documentation for the WASM build system
- Default to debug builds with -gsource-map for WASM debugging
- Add --release flag to disable debug symbols
- Add --clean flag for full rebuild (deps + wxWidgets + KiCad)
- Add -j N flag for parallel compilation (default: 1 for sequential)
- Update Docker memory limit to 32GB
- Propagate DEBUG_BUILD and JOBS to all dependency scripts

Build modes:
- ./docker/build.sh --clean -j8  (full rebuild, parallel)
- ./docker/build.sh              (rebuild KiCad only)
- ./docker/build.sh --no-clean   (incremental)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-12 14:51:38 +01:00
Viktor Vaczi
3034d9d76f Add KiCad WASM build infrastructure and dependency scripts
- Add build environment setup (scripts/common/env.sh, versions.sh, functions.sh)
- Add dependency build scripts for Zstd, GLM, FreeType, HarfBuzz, Pixman, Cairo
- Add placeholder scripts for OpenCASCADE, ngspice, protobuf
- Add WASM compatibility layer (wasm/kiplatform/, wasm/libcontext/)
- Add CMake modules for KiCad WASM cross-compilation
- Add PCBnew test infrastructure (tests/kicad/)
- Add build plan documentation

Successfully tested builds: Zstd 1.5.5, GLM 0.9.9.8, FreeType 2.13.2,
HarfBuzz 8.3.0, Pixman 0.42.2, Cairo 1.18.0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 13:15:04 +01:00