Commit graph pcbjam/scripts/common
Author SHA1 Message Date
Viktor Vaczi
7331619404 diagnostics: configurable --diag logging flags + asyncify setupUIConditions fix
- build-pcbnew.sh: add --diag=<gal,coroutine,ctor,all> -> -DKICAD_DIAG_*,
  off by default (forwarded by docker/build.sh)
- diagnostics.js: emit at console.log level (no longer error/warn); still
  gated by SHIM_DIAGNOSTICS=1
- apply-asyncify.sh: exclude PCB_EDIT_FRAME::setupUIConditions() from
  asyncify instrumentation (V8 cannot run the instrumented huge function
  on the rewound ctor stack -> Chrome startup stall; Firefox unaffected)
- DEBUG.md: reusable WASM/asyncify/browser debugging guide, diagnostic
  flag docs, and a production-build (release + -O2 asyncify) recipe
- tests: standalone coroutine vcall/gl repro probes
- bump kicad + wxwidgets submodules (diagnostic gating / debug cleanup)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 16:14:03 +02:00
Viktor Vaczi
01dce40dc9 test(wasm): coroutine crash reproduction harness + per-engine runner
Investigation scaffolding for the Chrome-only KiCad coroutine renderer crash.
Adds isolated reproduction probes exercising the coroutine/Asyncify/fiber layer
under KiCad-like conditions, runnable in BOTH Firefox and system Chrome.

- tests/playwright-coroutine.config.ts + test:coroutine:firefox|chrome npm
  scripts: run the coroutine specs in Firefox AND system Chrome (the old e2e
  config only used bundled Chromium, which never reproduced the crash).
- tests/apps/standalone/coroutine-pthread/: no-wx + pthreads reproduction probes
  (fiber-in-main, nested invoke_/dynCall boundaries, RunMainStack, embind,
  main-loop/rAF activation) + worker_dom_stub.js for wx+pthreads builds.
- tests/apps/Makefile.wasm: coroutine-pthread{,-main,-nested,-nested-ex,-wx,
  -embind,-mainloop} targets.
- scripts/common/shims/diagnostics.js: add EM_ASYNC_JS handleSleep enter/wake
  tracking (DIAG_SLEEP) to detect nested-async at the crash.

Findings (details in research notes): every isolated factor so far — direct /
nested / RunMainStack fiber, wx event loop + all 13 scenarios incl EM_ASYNC_JS,
pthreads, and main-loop/rAF activation — runs CLEAN in system Chrome. The
coroutine/Asyncify layer is exonerated; GL/WebGL is the remaining untested factor
(next). The reliable FF-pass/Chrome-fail repro is still the KiCad pcbnew e2e.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 18:44:12 +02:00
Viktor Vaczi
a4ad69412e fix(wasm): bind dynCall_* to real DYNCALLS=1 exports; refactor shim into files
The shim bound bare dynCall_* names to JS getWasmTableEntry() calls, bypassing
the asyncify-instrumented dynCall_* wasm trampolines that -sDYNCALLS=1 provides.
That broke Asyncify unwind/rewind through indirect calls -> "indirect call
signature mismatch" (caught every frame in Firefox; fatal renderer crash in
Chrome). Bind the bare names to wasmExports["dynCall_<sig>"] instead.

Result: the PCBnew "select draw lines" e2e is green in Firefox (tool selects and
draws, zero page errors). Dropped the fiber-stabilization block, the shipped
diagnostic block, and the exportCallStack JS hack (all compensated for the wrong
binding); shim shrank 521 -> ~250 lines.

- scripts/common/inject-dyncall-shims.sh: orchestrator only; injected JS extracted
  to scripts/common/shims/
- scripts/common/shims/dyncall-binding.js.tmpl: per-signature binding template
- scripts/common/shims/handlesleep.js: nested-Asyncify handleSleep fix (#9153)
- scripts/common/shims/diagnostics.js: logging-only, opt-in via SHIM_DIAGNOSTICS=1
- tests/package.json: add test:kicad:firefox / test:kicad:chrome scripts

Known issue (tracked separately): Chrome still renderer-crashes on the first
coroutine resume. Asyncify.doRewind replays the deep main-context call stack and
exceeds V8's execution-stack limit (Firefox tolerates the same wasm). Proper fix
is JSPI.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 15:07:10 +02:00
Viktor Vaczi
9a04217788 wip: nested-asyncify fix, wxAuiToolBar registration, tests, research docs
Main-repo side of a multi-part WIP covering the KiCad WASM tool-selection
and nested-Asyncify work. Submodule commits are in kicad@f6e9239aaa
(libcontext hygiene) and wxwidgets@bb80f91e8b (auibar registration +
dialog diagnostics).

## scripts/common/inject-dyncall-shims.sh

Wrap Asyncify.handleSleep / allocateData to save-and-restore Asyncify.currData
around each EM_ASYNC_JS sleep. This fixes the nested Asyncify collision where
a fiber swap that fired during a modal's event loop clobbered currData, and
the modal's later doRewind used the fiber's buffer and hit "RuntimeError:
index out of bounds". Root cause documented as Emscripten Issue #9153
(wontfix upstream).

Diagnostic-rewind logging (forcedBottomOfCallStack, callStack traces) is
retained to help future debugging of Asyncify state corruption.

## tests/

- tests/playwright-kicad.config.ts: add `channel: 'chrome'` for the
  chromium project so --project=chromium --headed uses system Chrome
  (real GPU) instead of SwiftShader on ARM Mac. Also switch trace to
  retain-on-failure + screenshot on-failure for easier E2E debugging.
- tests/kicad/pcbnew.spec.ts: replace `tool.checked` assertions with a
  label-suffix check (`[checked]`) since our auibar registration encodes
  checked state in the label (no schema change to the registry).
- tests/apps/Makefile.wasm: add `coroutine-nested` build target + include
  it in the all: list.
- tests/apps/standalone/coroutine/: kicad_coroutine_harness.h + test app
  reproducing KiCad COROUTINE semantics against real libcontext.
- tests/apps/standalone/coroutine-nested/: nested_test.cpp reproduces the
  EM_ASYNC_JS-modal + fiber-swap nesting bug in isolation. 8 scenarios
  from baseline_modal_alone through nested_fibers_inside_modal.
- tests/e2e/coroutine.spec.ts + coroutine-nested.spec.ts: Playwright specs
  that load the standalone apps and assert all case cases pass via
  [COROUTINE_TEST] SUMMARY log parsing.

## research/ and features/browser-tools/

Three background docs capturing the investigation trajectory:

- features/browser-tools/0001-kicad-wasm-tool-activation-investigation.md
  Early investigation: why tools don't activate; initial dynCall-empty-
  callback hypothesis.
- features/browser-tools/0002-wasm-coroutine-deep-dive.md
  Deep dive on Asyncify internals, fiber API, QEMU's coroutine-wasm
  reference implementation.
- features/browser-tools/0003-wxauitoolbar-registration-fix.md
  The narrow fix: why wxAuiToolBar needs a registration block, where to
  add it, what the fallback plan is.
- research/threading_1.md: corrected root-cause analysis after reading
  runtime logs — nested-Asyncify currData collision, Emscripten #9153.
- research/threading_2.md: extended research on alternative approaches
  (JSPI/WasmFX/state-machines) and why they don't help here.

## Submodule pointer updates

kicad: f6e9239aaa (wip: libcontext WASM hygiene cleanup)
wxwidgets: bb80f91e8b (wip: wxAuiToolBar element-registry registration +
            dialog diagnostics)

## Open threads not yet in scope

- Firefox/Chrome divergent behavior: "indirect call signature mismatch"
  traps in Firefox vs renderer crash in system Chrome (tracked in
  plans/peaceful-hugging-pnueli.md and the research docs).
- E2E pixel-diff for Draw Lines fails because the test's diff region does
  not cover where the line is actually drawn; tool activation works, the
  line is visible in test-results/pcbnew-draw-lines-02-after-drawing.png.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 13:58:16 +02:00
Viktor Vaczi
d557b21eac fix(docker): Replace emsdk image with Ubuntu + emsdk from source
The Docker build used emscripten/emsdk:4.0.2-arm64 as base image but
env.sh couldn't find emsdk there, installing a second copy. The build
then applied wasm-opt/finalize stubs to the wrong emsdk (hardcoded
/emsdk/), so the real wasm-emscripten-finalize ran in Docker and got
OOM-killed.

- Use ubuntu:22.04 base with emsdk installed from source at /emsdk/
- Make stub paths dynamic via $EMSDK instead of hardcoded /emsdk/
- Skip local emsdk install in env.sh when $EMSDK is already active

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 12:44:26 +01:00
Viktor Vaczi
b46b4b69f1 fix(wasm): Fix asyncify rewind with asyncify-aware dynCall shims
Emscripten 4.x removed dynCall_* WASM exports, breaking asyncify
rewind through indirect calls (modal dialogs, event handlers).
Generate JS shims that track Asyncify.exportCallStack and register
in wasmExports so doRewind can find them.

Also fixes empty callback functions ((() => {})) generated by
Emscripten 4.x + pthreads for HTML5 events, pthread entry,
sighandler, async timer, and main loop callbacks.

Build pipeline improvements:
- Stub wasm-opt/finalize in Docker (RAM limits), run on host
- Add setup-emsdk.sh for reproducible Emscripten setup
- Simplify env.sh and version management

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:26:48 +01:00
Viktor Vaczi
014816b7a5 fix(wasm): Fix Emscripten empty callback functions for pthreads builds
Emscripten with pthreads generates empty arrow functions `{}` for callback
paths it assumes won't be used. However, when registering HTML5 events from
the main browser thread, targetThread is 0 and the direct call path IS taken.

This fix post-processes the generated JS to replace empty callbacks with
actual dynCall invocations for:
- HTML5 event callbacks (dynCall_iiii) - 7 instances
- pthread entry points (dynCall_ii) - 1 instance
- Signal handlers (dynCall_vi) - 1 instance
- Async timers (dynCall_vi) - 1 instance

Also improves build logging to show completion status and exit code.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 14:19:13 +01:00
Viktor Vaczi
a4f444fea8 feat(webgl): Add WebGL GAL test infrastructure (Phase 1)
Add complete test infrastructure for WebGL GAL visual regression testing:

- scripts/test-gal-regression.sh: Master script that builds both backends,
  runs tests, and performs two-level comparison (native vs baseline,
  webgl vs native)
- scripts/build-gal-webgl-test.sh: WASM build using Makefile with em++
- tests/gal-regression/wasm/: WebGL test harness (stub WEBGL_GAL)
- tests/e2e/gal-webgl.spec.ts: Playwright test for screenshot capture

Fix Homebrew Emscripten environment in scripts/common/env.sh:
- Set EMSDK_PYTHON for Python 3.10+ (em++ reads this, not $PYTHON)
- Add bundled LLVM to PATH (Emscripten needs its clang with WASM backend)

Verified: Native vs Baseline passes (28/28), WebGL generates blank
screenshots as expected (WEBGL_GAL implementation is Phase 2).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 16:11:31 +01:00
Viktor Vaczi
5d93e11358 Fix indirect call to null error and logging.sh worktree support
wxwidgets submodule:
- Add m_isCreated flag to skip UpdateElementRegistry() during construction
- Prevents virtual method calls on partially constructed objects

scripts/common/logging.sh:
- Change -d to -e for .git check to support git worktrees

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 16:25:11 +01:00
Viktor Vaczi
ea5dc26a96 Restore elementtracker.h and add build logging
- Restore missing elementtracker.h in wxWidgets (required for E2E tests)
- Add automatic log file redirection for build scripts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 11:55:54 +01:00
Viktor Vaczi
98b523c2bc Fix clean build issues
- Add autoreconf step for bundled PCRE in wxWidgets build script
  (fixes automake version mismatch between 1.16.1 and 1.16.5)
- Fix apply-finalize.sh to use get-wasm-opt.sh for consistent tool paths
  (tools moved to build-wasm/tools/ in previous commit)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 15:28:20 +01:00
Viktor Vaczi
9cd5965b2c Move tools/ to build-wasm/tools/
Groups cached build tools (binaryen) with other build artifacts.
The build-wasm/ directory is already gitignored via build-*/ pattern.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 14:08:47 +01:00
Viktor Vaczi
dafef9c0a1 Fix Docker macOS timestamp issues and improve build reproducibility
- Add rsync-based source sync for Docker builds to fix macOS VirtioFS
  timestamp inconsistencies that caused autoconf sanity checks to fail
- Add config.sub wrapper (SHELL/CONFIG_SHELL) to support emscripten/wasm32
  targets without modifying submodule files
- Fix protobuf build to use native gcc/g++ for protoc instead of Emscripten
- Add python PATH fix for macOS (Homebrew's python3 symlink) in wxWidgets build

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 01:07:44 +01:00
Viktor Vaczi
5b919ef66f Add ccache and incremental build system for faster development
- Add ccache to Docker image for compiled object caching
- Change build defaults: incremental by default, skip deps by default
- Add new flags: --full, --clean-kicad, --build-deps
- Skip wxWidgets configure if already configured (check Makefile timestamps)
- Remove unused source hash stamp functions (make handles dependencies)
- Update build.md with new build system documentation

Performance improvements:
- Single file change: 7:35 → 1:34 (4.8x faster)
- No-change rebuild: 7:35 → 1:31 (5x faster)
- Asyncify post-processing (~1 min) is now the bottleneck

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-26 20:51:58 +01:00
Viktor Vaczi
9c271996c2 Fix KiCad WASM build and runtime errors
Build improvements:
- Change -O0 to -O1 to fix "local count too large" asyncify error
- Add wasm-emscripten-finalize to host (Docker OOMs on large WASM)
- Use -gseparate-dwarf for smaller main binary with debug info
- Build native protoc for code generation
- Add more functions to asyncify removelist

Runtime fixes:
- Add inject-dyncall-shims.sh to fix "dynCall_* is not defined" in Emscripten 4.x
- Update wxwidgets with browserInfo.name fix

New stubs and bindings:
- Add Embind bindings for JavaScript interop
- Add stubs for scripting, API plugin, PCB frame, navlib

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-22 16:30:53 +01:00
Viktor Vaczi
8453f9681e Add stamp source hashing and maximize test infrastructure
- Add source hashing to stamp system (functions.sh) for detecting
  when dependencies need rebuild based on source file changes
- Update build-pcbnew.sh to use source stamps for wxWidgets
- Add global-setup.ts to clean logs before test runs
- Add maximize_test standalone test to verify wxFrame::Maximize()
  works correctly in WASM (it does - window is 1280x720)
- Update Makefile.wasm with maximize test build rules

The maximize_test proves wxWidgets display detection works fine.
KiCad's 20x20 window bug is KiCad-specific, not a wxWidgets issue.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 13:28:43 +01:00
Viktor Vaczi
6771e7de74 Fix Asyncify and improve build process
Build system improvements:
- Add wasm-opt stub for Docker build to bypass asyncify in container
- Add apply-asyncify.sh script for host-side asyncify transformation
- Update build-pcbnew.sh to install wasm-opt stub and add EXPORTED_RUNTIME_METHODS
- Add --with-zlib=sys to wxWidgets configure for proper zlib linking
- Improve update-baseline-screenshots.sh to only copy new/significantly changed files

wxWidgets submodule update:
- Fix EM_JS to EM_ASYNC_JS for proper Asyncify registration

Test updates:
- Add new baseline screenshots for bitmapbuttons, dnd, gridrenderers, printpreview, specialized tests
- Add kicad-pcbnew initial screenshot
- Fix wxgrid-controls.png and wxgrid-dedicated-page.png (were showing errors)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 11:59:50 +01:00
Viktor Vaczi
b70f098ae2 Add two-phase build with host asyncify transformation
- Move asyncify from Docker to host to avoid memory issues
- Auto-download Binaryen v121 (v125 has regression bug)
- Use -O1 for debug builds (V8 local count limit)
- Remove asyncify flags from linker (handled by wasm-opt)
- Document two-phase build in build.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 22:13:31 +01:00
Viktor Vaczi
af2adda002 Improve build system with debug symbols, clean modes, and parallel jobs
- Add build.md documentation for the WASM build system
- Default to debug builds with -gsource-map for WASM debugging
- Add --release flag to disable debug symbols
- Add --clean flag for full rebuild (deps + wxWidgets + KiCad)
- Add -j N flag for parallel compilation (default: 1 for sequential)
- Update Docker memory limit to 32GB
- Propagate DEBUG_BUILD and JOBS to all dependency scripts

Build modes:
- ./docker/build.sh --clean -j8  (full rebuild, parallel)
- ./docker/build.sh              (rebuild KiCad only)
- ./docker/build.sh --no-clean   (incremental)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-12 14:51:38 +01:00
Viktor Vaczi
3034d9d76f Add KiCad WASM build infrastructure and dependency scripts
- Add build environment setup (scripts/common/env.sh, versions.sh, functions.sh)
- Add dependency build scripts for Zstd, GLM, FreeType, HarfBuzz, Pixman, Cairo
- Add placeholder scripts for OpenCASCADE, ngspice, protobuf
- Add WASM compatibility layer (wasm/kiplatform/, wasm/libcontext/)
- Add CMake modules for KiCad WASM cross-compilation
- Add PCBnew test infrastructure (tests/kicad/)
- Add build plan documentation

Successfully tested builds: Zstd 1.5.5, GLM 0.9.9.8, FreeType 2.13.2,
HarfBuzz 8.3.0, Pixman 0.42.2, Cairo 1.18.0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 13:15:04 +01:00