Commit graph pcbjam/docker/build.sh
Author SHA1 Message Date
Viktor Vaczi
9c475a804e jspi cleanup: remove the asyncify-era residue — dead code, conditionals, pipeline scaffolding, stale prose
The runtime is JSPI-only; this removes everything that still pretended
otherwise. Three exhaustive sweeps (C++/JS+build+CI/tests+docs) drove
the inventory; every deletion verified by grep closure + full gates.

Broken-right-now fixes:
- deploy-staging.yml passed the retired opt_level input — the workflow
  could not even start. Removed.
- env.sh carried dead exports with a live -sASYNCIFY=1 inside
  (WASM_LDFLAGS/PTHREAD_LDFLAGS, zero consumers). Removed; the
  WASM_LEGACY_EXCEPTIONS rationale rewritten to the real reason.
- docker/build.sh exported PCBJAM_ASYNC_BACKEND (read nowhere). Gone.

Dead weight removed:
- binaryen submodule (nothing builds or invokes it), wasm-opt-bench
  workflow + scripts/bench/, get-wasm-opt.sh, diagnostics.js (242 lines
  of Asyncify-API-only code), the KICAD_PIPELINE background-postprocess
  scaffolding (existed to parallelize the deleted wasm-opt phase; the
  postprocess is a seconds-long node script and now runs inline),
  build-monitor's dead asyncify rows, sched-context orphan build
  output, dead .gitignore entries, the .jspi-assets spike dir (the two
  wf-result research JSONs moved to docs/features/async/migration-evidence/).
- bindings: fiber_park.h + its 12 embind registrations (broken-if-
  called under JSPI), the kicadOpenFileStart/OPEN_JOB starter route,
  main_stack_runner.h + 5 includes, the always-null context-sleep weak
  hook in nanosleep_yield.c.
- shim: the backend field (installed-flag idempotency instead),
  noteContextWait (dead both sides), the __wxAsyncifyDump alias (+ the
  WasmTool fallback and string-dump normalize branch).
- web: the emscripten-6-ignored mainScriptUrlOrBlob option in boot.ts
  (gerber-demo keeps it: it loads the deployed CDN release, which
  predates emscripten 6 — noted inline).

Conditionals: all 'backend === jspi' checks reduced to scheduler-
presence checks; races_quiescent re-keyed from Asyncify.state (vacuous)
to real backlog quiescence (resumeReady/mutatorQueue — NOT _windowLive,
which is the probing activation's own window by definition).

Renames (identifiers only, no file renames): ASYNC_LINK_FLAGS→
JSPI_LINK_FLAGS and Makefile ASYNC_LDFLAGS→JSPI_LDFLAGS,
kicadCollabFiberBusy→kicadCollabBusy (embind + web + tests),
collab_common.h fiber*→apply*/coroutine naming, asyncifySignatures→
wasmTrapSignatures (lists byte-identical).

Tests: the two remaining vacuous [wx-asyncify]/fiber-resume-refused
asserts re-keyed to live JSPI beacons; eeschema-load's failure message
no longer sends the developer to a deleted script; wait-beacons' dead
families/parser deleted; lane-0 legacy-glue guards removed (lane 0 is
unconstructible); the embind test.fail re-gated with the JSPI reason
(plain embind invokers cannot suspend — verified still failing);
lint-determinism now scans tests/jspi (166 files clean);
eeschema-collab local-move gated to chromium (~50% flaky on FF even
solo; pcbnew twin covers both engines).

Docs: DEBUG.md rewritten as the JSPI debugging guide; build.md
describes the single-phase build; docs/features/async/README.md
banner-marked historical and repointed at the NEW
23-jspi-runtime.md (current architecture: export census, turnstile,
libcontext ownership + refusal contract, embind call shapes, the
em-pthread service-wrapper trick, exception policy, known gaps).

Gates on the cleaned tree: test:e2e 725 passed / 0 failed (after the
quiescence-probe fix; the 3 other reds were verified contention flakes
solo-green or the documented FF gate), web 76/0, jspi 18/18 both
engines, vitest 295/295 + 17/17, all lints green, live-app census
clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016X9eh1s5sTx1o9Em9KBuwR
2026-08-14 09:25:32 +02:00
Viktor Vaczi
e14faeca8b jspi: retire the asyncify pipeline — knob, post-link tail, binaryen hooks
Phase 8 in the parent repo. Deleted: asyncify-scheduler.js, apply-asyncify.sh,
apply-finalize.sh, inject-dyncall-shims.sh, asyncify-imports/removelist.txt,
the wasm-opt/finalize stub pair, scripts/binaryen-hoist-pass/ (the fork stays
a dormant submodule; removal is a follow-up), bench/wasm-opt-bench.sh (README
marked historical), wasm/shims/context_sleep.cpp, and the sched-context
harness app + Makefile targets.

PCBJAM_ASYNC_BACKEND is gone: build-wx-wasm.sh hardcodes the jspi stamp
(still force-cleans pre-migration trees), build-kicad-target.sh gives editors
the JSPI link surface and the CLIs nothing (they pin ASYNCIFY=0), the stub
dance is replaced by an unconditional .real-restore, build-wasm-test.sh lost
its whole post-link loop, docker/build.sh's postprocess is the ENV shim only,
and Makefile.wasm links every app JSPI with the scheduler shim as a tracked
prerequisite. pcbjam_async_policy.h keys on __EMSCRIPTEN__.

jspi-scheduler.js: wxWasmMainLoopPump dropped from the wrap census (the
export died with the D5 detach); inert [TRACE] instrumentation removed.

CI: wasm-build.yml rewritten for the single-cache pipeline (one output cache
keyed on compile inputs; post-processed bytes cached after the shim);
opt_level input removed from both callers. wasm-cache-hash.mjs inputs now
cover patch-env-shim.mjs + jspi-scheduler.js + jspi-exports.txt.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NDeBaKKhQztd8KiVtHuyXr
2026-08-13 08:39:12 +02:00
Viktor Vaczi
3f09a46ff5 jspi: migration phases 0-7 — build knob, scheduler shim, test successor suite
Toolchain: emsdk 6.0.6 (versions.sh; cache-hash keys on it). Build knob
PCBJAM_ASYNC_BACKEND=jspi|asyncify: build-kicad-target.sh links editors with
-sJSPI + -sJSPI_EXPORTS=@scripts/common/jspi-exports.txt + --pre-js
jspi-scheduler.js (no DYNCALLS, no post-link asyncify pipeline); wx build
stamps the backend and forces clean on flip or unknown provenance;
docker/build.sh passes the knob, seeds the emscripten ports cache from the
volume every launch, jspi postprocess = patch-env-shim only.

scripts/common/shims/jspi-scheduler.js: the JSPI successor scheduler —
token-wait registry, resume turnstile (one armed resume between engine
re-entries, SP swaps only at microtask boundaries), green-region spill
stacks (16-aligned tops), S1 embind mutator FIFO lane + parker wraps, S6
shutdown, libctx integration hooks (suspend/end/quarantine + g_current
arm/clear), SuspendError attributor, lost-wake + stuck-window watchdogs,
__wxWaitDump observability.

Embind: PARKER registrations get emscripten::async() under PCBJAM_JSPI
(wasm/bindings/pcbjam_async_policy.h). nanosleep yields route via the shim.

Tests: tests/asyncify -> tests/jspi successor suite (jspi-stack red/green
shadow-stack battery, jspi-coroutine MiniCoro harness, suspend-races
semantic scenarios + __wxWaitDump books coherence); projects jspi-firefox/
jspi-chrome (asyncify-webkit retired — no JSPI in WebKit); unconditional
Firefox JSPI pref; guard-beacons -> wait-beacons (+wxScheduler/libctxJspi
families); Makefile.wasm links test apps against JSPI with the shim as a
tracked link prerequisite.

Web: WasmTool setRo await + __wxWaitDump forensics, open-flow contained
promise, scheduler-shim.test.ts retargeted (8 green).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NDeBaKKhQztd8KiVtHuyXr
2026-08-13 07:06:24 +02:00
Viktor Vaczi
703cb010b7 eeschema simulator: lazy ngspice_service worker — static sharedspice (XSPICE registry + CIDER), init_dll ifdef, e2e both engines
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:59:21 +02:00
Gergő Törcsvári
cb27744d76
tasks-runner 0001 R2: kicad_tools joins the CI build set + lint gates
docker/build.sh "all" now includes kicad_tools (finalizes in-container, so
it never contends with the editor's wasm-opt critical path); .ci-cache-epoch
bumped — the cached FINAL output set changes. wasm-build.yml gains the
corpus-lint + CLI-contract gate step (run_tests leg): with the artifact now
built in CI, the skip-when-unbuilt scripts bite. Release runs
(upload_output) start shipping kicad_tools in the wasm-output artifact —
the closed repo's runner-image workflow consumes it from there.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HLua64PCVwkQ1hpWdaf1Gm
2026-07-14 19:02:13 +02:00
Gergő Törcsvári
ad9c4675f9
feat: kicad_tools headless CLI
One merged node-WASM CLI (wasm/tools, 26.5MB) supersedes sym_convert +
pcb_convert: --convert-lib (absolutizes paths — fixes the legacy plugin's
silent empty output on relative paths), --lint (now full-parses .kicad_pcb
via the linked pcbnew parser), --erc, --netlist, --bom, --plot, --drc.
Registered as the only headless CLI app; ASYNCIFY=0 CLIs skip the
nanosleep→Asyncify yield shim and exit via _exit (dieted static dtors trap).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ADzCSeN3Q9DJL3YW2FZTXB
2026-07-10 19:09:01 +02:00
Gergő Törcsvári
28388990ee
ysync 0009 phase 4: sym_convert --lint mode (kicad_lint folded into the converter)
sym_convert --lint [--strict] <file...>: full parse for .kicad_sch (restored
LoadSchematicFile + minimal headless PGM/SETTINGS_MANAGER runtime,
aSetActive=false) and .kicad_sym/.lib; structure-only for other s-expr files;
dup-uuid identity lints (walker) on every s-expr input; errors as
file:line:offset, exit 0/1/2. Convert mode byte-identical (qa gate).

Fixes en route: 64KB stack local overflowed the default wasm stack (mimalloc
corruption); pre-js now inherits process.env into getenv (KICAD_CONFIG_HOME /
SYM_CONVERT_TRACE); font abort stub retired (SCH_SCREEN::Append bbox resolves
draw fonts) -> 18.0MB; six GetMsgPanelInfo typeinfo stubs; BUILD_3D_VIEWER
defaults OFF for headless CLIs (gl1 shim needs glm).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0142yAmMGZ3Ejrq3wwVr2Yq2
2026-07-10 13:13:51 +02:00
Viktor Vaczi
1a24c2894e fix(build): stop the builder container when the build exits
An idle builder keeps the Docker Desktop VM ballooned: each per-worktree
compose project carries its own ${KICAD_DOCKER_MEM:-32G} cap, build.sh left
the container running afterwards ("interactive use"), and two forgotten
builders plus the VM's page-cache/high-water behavior read as ~56 GB of host
RAM during a ~20 GB build.

stop_builder() now runs from the EXIT trap on every path (success, failure,
Ctrl-C — both the sequential and the pipeline-mode trap). The container is
pure scaffolding — all caches live in the named volumes and `up -d` restarts
it in seconds — so stopping costs nothing. KICAD_KEEP_CONTAINER=1 restores
the old keep-running behavior for interactive debugging; docker/shell.sh is
unaffected (it does its own `up -d`).

Verified live: the full `all --build-deps` run on main left zero containers
running at exit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:42:38 +02:00
Viktor Vaczi
db9d6ee04b feat(wasm): occ-split — lazy occ_service worker; kicad_editor drops OCC (−31%)
Move OpenCASCADE out of the merged editor image into occ_service: a separate
emscripten module (-sASYNCIFY=0, MODULARIZE, in-container -Oz finalize, 2N+8
pre-warmed pthread pool) booted lazily in a dedicated Web Worker on the first
STEP export or STEP/IGES model parse. kicad_editor.wasm ~190 MB -> 130 MB;
sessions that never touch OCC never fetch its 57 MB. STEP export works in the
browser for the first time: the unchanged desktop dialog runs EXPORTER_STEP,
whose wasm shadow suspends into globalThis.occService and the export bytes go
straight to a browser download (never entering the editor heap). STEP/IGES 3D
models parse in the worker via the oce shadow (S3D WriteCache/ReadCache wire).

- wasm/occ-service/: service CMake target (hooked from the kicad fork's
  top-level CMakeLists, wasm/editor pattern), embind entry
  (occExport/occLoadModel), wxConfig pre-js.
- wasm/stubs/{exporter_step,oce_plugin}_stub.cpp: EM_ASYNC_JS worker bridges
  (callee-shadowing; no caller #ifdefs).
- web/standalone: provider installed whenever the kicad_editor bundle boots
  (cross-face safe); ONE shared worker-boot source occ-worker.js (vite ?raw;
  the e2e stub reads the same file) — blob worker with locateFile absolutized
  against the glue URL; export download-name guard.
- deps: OCC builds with RapidJSON so its glTF/GLB writer exists — pinned to
  the vcpkg master snapshot 2025-02-26 (24b5e7a8b27f), the same code official
  KiCad consumes via vcpkg.json's opencascade[rapidjson]; rapidjson's latest
  tag (v1.1.0, 2016) is ill-formed under modern clang.
- tests: occ-export dialog e2e (lazy-fetch boundary + STEP download bytes),
  occ-probe incl. a 9-format matrix (step/stpz/brep/xao/ply/stl/glb/u3d/pdf),
  3d-viewer-models hard-asserts the worker parse; occ provider stub installed
  ambiently by the kicad fixtures.

Validated against desktop kicad-cli 10.0.4: geometric exact equality (bbox
delta 0 um, volume delta 0.0000%) for STEP/GLB/STL/BREP/STPZ across three
boards and option sweeps — with desktop OCC 7.9 vs wasm OCC 7.8; PLY/XAO/PDF
structurally equal; U3D same-size (quantizer float LSBs differ). Full kicad
e2e green on Firefox and Chromium; standalone verified end to end (lazy fetch
only on the Export click; export.step 60,628 B ISO-10303-21; loadModel 700 KB
STEP -> 569 KB scenegraph cache).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:39:58 +02:00
Istvan Matejcsok
bbeef6d20e feat(wasm): kicad_editor — merge the pcbnew+eeschema kifaces into ONE bundle (Part 2)
All four editors (PCB / Footprint / Schematic / Symbol) are now runtime --frame
choices of a single kicad_editor.wasm (178 MB at -O1 vs 147+82 separate; shared
wx/common/boost linked once). One editor per page load, as before; frames pcb /
fpedit / sch / symedit.

- wasm/editor/: the merged executable target (single_top + both kiface library sets,
  whole-archive pcbcommon) + the safety-net focus-walk Kiface() dispatch TU. Gated by
  KICAD_WASM_MERGED_EDITOR (kicad submodule bump carries the fork side: per-engine
  Kiface/getter binding + ODR renames + dual-kiface launcher).
- wasm/bindings/: per-editor collab entries renamed pcbCollab*/schCollab* (JS names
  unchanged); duplicate kicadOpenFile/kicadCollabOnSave + shared-name registrations
  guarded behind KICAD_MERGED_EMBIND; new kicad_editor_embind.cpp registers each
  shared JS name once, dispatching on the live frame.
- Build: kicad_editor app (build wrapper, target case arms, 3-object embind compile
  with the ABI-critical flags, STUB_APP=pcbnew); docker/build.sh "all" =
  kicad_editor calculator pl_editor gerbview (pcbnew/eeschema stay as explicit debug
  apps); scripts/kicad/audit-merged-symbols.sh = repeatable ODR-collision audit (run
  on kicad bumps).
- Frontend: Bundle type (bundle ≠ tool); TOOL_BUNDLE maps all four editors to
  kicad_editor; explicit --frame tokens for pcbnew (pcb) and eeschema (sch); publish
  list = the 4 real bundles.
- Tests/CI: five harnesses load kicad_editor.js with explicit frame tokens;
  PCBNEW_FAMILY_SPECS renamed BIG_MODULE_SPECS + the 8 eeschema-family specs (they
  now boot the merged module — SpiderMonkey x86 CI OOM routing); frame-runtime spec
  covers all four frames from the one bundle.

Validated so far: frame-runtime 4/4 (each frame boots with the right title, no
aborts, no duplicate embind registration); 24-spec merged-module regression green;
3D raytracer renders. Known pre-existing failure: 3d-viewer title-bar drag deadlock,
fixed on main by 7630c7e (2N+8 pthread pre-warm) — picked up by the follow-up rebase.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 17:56:02 +02:00
Istvan Matejcsok
c6716e6b60 feat(wasm): unify editor builds — footprint/symbol editors via runtime --frame
The footprint and symbol editors are no longer separate WASM bundles: the frontend loads the parent pcbnew/eeschema bundle and passes --frame=fpedit / --frame=symedit (TOOL_BUNDLE + TOOL_FRAME -> Module.arguments in boot). Drops the two duplicate build+deploy targets and their wrapper scripts + vestigial embind; adds low-level harnesses (footprint_editor.html, symbol_editor.html) and a runtime-frame spec. Bumps the kicad submodule to the runtime --frame launcher.

The frame-runtime spec is listed in PCBNEW_FAMILY_SPECS so CI routes it to the chromium-ci (V8) project — its footprint case boots the pcbnew module, which OOMs SpiderMonkey on x86 CI. Includes the editor-unification dossier (research docs 01-04 + the as-built implementation record 05).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 17:56:02 +02:00
Gergő Törcsvári
e7640d0a97
build(wasm): standardize asyncify wasm-opt tail to -O1 everywhere
The Binaryen wasm-opt shrink after Asyncify was the only -O2 in the
pipeline: -O2 by default (local + tag release) but -O1 in main CI. The
C++/wx/deps compile is already -O1 everywhere (DEBUG_BUILD defaults to 1;
nothing passes --release in CI/release). Pin the tail to -O1 too:

- apply-asyncify.sh: BINARYEN_OPT_LEVEL default -O2 -> -O1
- release.yml: opt_level -O2 -> -O1 (demo now ships -O1)
- ci-ubicloud.yml / wasm-build.yml / docker/build.sh: refresh stale -O2 comments

Because both callers now build at -O1, the FINAL cache key converges, so a
tag release FINAL-cache-hits main's build and skips the asyncify-tail
rebuild entirely (previously the ~1-2h -O2 wasm-opt was rerun per release).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 12:18:57 +02:00
Gergő Törcsvári
89a433fda9
fix(trace): make ?trace= work by merging Module.ENV in the glue
The emscripten glue's `var ENV = {};` never merged Module.ENV, so boot.ts's
Module.ENV.KICAD_TRACE was a silent no-op — and since environ_get on a pthread
proxies to the main thread, the app/UI thread's getenv read the (empty) main
ENV. Net: KICAD_TRACE never arrived and every KI_TRACE was a no-op.

scripts/common/patch-env-shim.mjs post-processes each generated <app>.js to
merge Module.ENV into ENV (idempotent; runtime no-op when unset), wired into
docker/build.sh's per-app host post-process next to the dyncall-shim inject.
Replaces a fragile manual glue edit re-applied after every build. Verified:
symbol-editor eager load now emits KI_TRACE_SYM_CHOOSER fatLoad timing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 11:01:10 +02:00
Viktor Vaczi
714b0a816b fix(wasm-eh): build wasm-emscripten-finalize from the Binaryen submodule (unbreak CI)
The native-EH migration pointed apply-finalize.sh at emsdk's bundled
wasm-emscripten-finalize (${EMSDK:-tools/emsdk}/upstream/bin), which exists on a
dev machine (tools/emsdk persists from a prior build) but never on the ephemeral
CI host — emsdk lives only inside the Docker image, and nothing on the host
post-process path provisions it. So the host post-process died at finalize for
every app once the binaryen-SHA cache re-key forced it to actually run. (Not the
missing-ninja theory — ninja was installed; the run never reached the wasm-opt
build, which is downstream of finalize.)

Finish the "Binaryen submodule everywhere" migration: build-wasm-opt.sh now also
builds wasm-emscripten-finalize, and apply-finalize.sh takes it from that build
(next to wasm-opt). The host post-process is now emsdk-free (dyncall=node,
finalize+asyncify=submodule v130) and finalize/wasm-opt share one Binaryen
version (previously finalize was emsdk's v121, wasm-opt the submodule's v130).
get-wasm-opt.sh stays for bench only; dropped it from the wasm cache key and
added build-wasm-opt.sh.

Also surface the from-source Binaryen build as a "Build Binaryen" stage in
build-monitor.sh (it had no marker), emitted at the pipelined pre-warm sites.

Validated: a clean from-scratch `docker/build.sh calculator --build-deps` (deps +
compile + finalize via the submodule binary + asyncify + -O2) succeeded
end-to-end; finalize resolved to build-wasm/tools/binaryen-hoist-build/bin.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 11:55:52 +02:00
Viktor Vaczi
c1ef489cfa feat(wasm-eh): migrate the WASM build to native wasm exceptions (+ 3D viewer default-on)
Replace the legacy Emscripten JS-exceptions model with native wasm-EH (legacy
encoding) across the whole build, keeping Asyncify coroutines working via a
from-source Binaryen --hoist-cpp-catches pre-pass. Net result: native-EH is the
only build mode, the 3D viewer is on by default, and pcbnew shrinks substantially.

Highlights:
- Binaryen submodule everywhere + --hoist-cpp-catches integration in apply-asyncify;
  post-link Asyncify covers every app wasm (not just standalone test wasm).
- Build deps (incl. OpenCASCADE without OCC_CONVERT_SIGNALS) and all KiCad apps
  with -fwasm-exceptions; emscripten_sleep added to the post-link asyncify-imports.
- libcontext fiber entry wired under native exceptions; while-loop main loop +
  currData shim injected into all wx apps.
- Native-EH collab apply fixed: DEBUG-define the embind TU + match all out-of-CMake
  C++ TUs' ABI flags to the core, fixing the vtable-layout skew / mis-dispatch.
- 3D viewer enabled by default (real raytracer linked, not the stub).
- Retire the EH-spike scaffolding; flip the asyncify-races ablation pins to
  shim-redundancy pins (native-EH stays clean with the legacy shims ablated).
- Fix the asyncify-races quiescence check to not require Asyncify.currData==0:
  under the native-EH per-frame-yield top loop the main stack is asyncify-suspended
  every frame, so currData legitimately churns (a freed-but-not-yet-nulled buffer,
  not a leak). Refresh the pcbnew toolbar screenshot baseline for the new kicad.
- CI: drop the obsolete binaryen_version input/env (the build uses the binaryen
  submodule fork's wasm-opt, not a version download); key the wasm-output cache on
  the binaryen submodule SHA instead.

Bumps the wxwidgets + binaryen submodules to their squashed feature commits.

Validated green: all 7 apps native-EH (real 3D in pcbnew); KiCad e2e 63/63
Firefox + Chromium (3D viewer renders); wx 336; coroutine 34/34 both engines;
asyncify 7/7 both engines.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 09:40:26 +02:00
Gergő Törcsvári
80ee95a785
build: split build.sh into --compile-only / --postprocess-only phases
The container compile produces an OPT-INDEPENDENT base wasm; the only opt-dependent work is the final 'wasm-opt -O$LEVEL' shrink in apply-asyncify (host post-process). Splitting the build at that boundary lets CI cache the expensive compile ONCE (shared across -O1/-O2) and re-run only the asyncify/-O tail per opt level.

--compile-only: in-container compile → output/ base wasm (no post-process). --postprocess-only: pure host dyncall+finalize+asyncify+wasm-opt -O on the existing base (no container; get-wasm-opt self-provisions Binaryen). Default 'both' is unchanged. Phase flags are extracted before ARGS so they are not forwarded to inner build-<app>.sh. bash -n + shellcheck -S error clean; extraction unit-checked incl. the empty-after-filter case.

Enables the two-tier CI cache (opt-independent base + opt-specific final) so an -O2 demo release reuses main's compiled base and only runs asyncify+O2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 16:23:30 +02:00
Istvan Matejcsok
68c711e633 feat: enable the 3D viewer in the WASM port (WIP)
Builds KiCad's 3D viewer for the browser (opt in with BUILD_3D_VIEWER=ON) and
wires up the legacy-GL emulation it needs.

- build-kicad-target.sh / docker/build.sh: BUILD_3D_VIEWER →
  -DKICAD_BUILD_3D_VIEWER_WASM=ON + -sLEGACY_GL_EMULATION + the GL js-library.
- wasm/shims/gl_immediate_shim.js: display-list emulation (record/replay board
  layers), fixed-function + GLU stubs, throw-guards for unsupported pnames, and
  per-context GLImmediate init for the viewer's SECOND WebGL context.
- tests/kicad/3d-viewer.spec.ts + utils/pcbnew-ready.ts: open View → 3D Viewer.
- bumps the kicad submodule (EMSCRIPTEN-guarded GL changes).

WIP: the viewer window + UI render and geometry draws (glError=0x0), but the
board is not yet visibly rendered — GLImmediate's FFP shader program isn't
linked on the viewer's 2nd WebGL context. Full status + next steps in
features/feat/add-3d-view/notes.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 09:13:41 +02:00
Gergő Törcsvári
698f6d7cb7
feat(libs): standalone .lib→.kicad_sym wasm converter (sym_convert)
Headless node-WASM CLI that converts legacy symbol libraries to .kicad_sym via
SCH_IO_MGR::ConvertLibrary. Adds wasm/cli/sym_convert_main.cpp + sym_convert_pre.js
(in-memory localStorage + wxConfig JS-hook shim), registers sym_convert as a
docker/build.sh app (own kicad-sym_convert tree, real wasm-opt/finalize in-container,
skip host postprocess), and bumps the kicad pointer for the matching target.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 12:21:15 +02:00
Gergő Törcsvári
19fce866c1
feat: libs 0009-S — footprint write-path spike DONE + verified (footprint editor as project-scoped WASM tool + PCB_IO_PCBJAM_FP plugin + bridge kind 4th-arg; New Footprint→FootprintSave fires through bridge on main thread, fork-native body v20251028, no wedge — Firefox e2e green); bump kicad + pcbjam-shared 2026-06-16 16:28:21 +02:00
Viktor Vaczi
7dc1bf27e0 refactor: collapse dual-mode plumbing — the DOM port is the only WASM build
The canvas (wxUniversal) mode is gone (wxwidgets submodule); remove every
piece of side-by-side plumbing so there is exactly one build and one test
flow:

- scripts/build-wxuniversal-wasm.sh -> scripts/build-wx-wasm.sh; no
  --dom/--enable-universal; builds into build-wasm/wxwidgets
- build-wasm-test.sh: no DOM_BUILD / apps-dom rsync mirror / PORT=dom;
  apps build straight into tests/apps (Makefile.wasm PORT conditionals
  collapsed; wx.js + wx-dom.js always pre-js)
- docker/build.sh, build-kicad-target.sh, env.sh: WX_PORT / -dom /
  -universal suffixes removed; kicad builds to kicad-<app>, outputs to
  output/; wx.js/wx-dom.js copied from the real source path
  (/workspace/wxwidgets/build/wasm — the old build-wasm path never
  existed and silently failed)
- setup-kicad-wasm.sh: single target dir; the perl wx-dom.js injection is
  gone — the 7 checked-in kicad pages now reference wx-dom.js directly
- playwright configs serve apps/; fixtures drop the test-results/dom and
  logs/wxwidgets/dom namespacing; boot.spec asserts wxDomPort
  unconditionally; pcbnew.spec uses one reference image;
  appearance.spec assertions unconditional
- compare/update-baseline-screenshots.sh: --port removed
- tests/gal-regression/wasm/Makefile: links build-wasm/wxwidgets and
  carries wx-dom.js as a second pre-js — the gal-webgl suite (30 specs)
  now actually builds and runs here (it needed host-side boost+glm via
  scripts/deps; the bundle had been missing, timing the whole spec out)
- tests: clickCanvas() dispatches via page.mouse (DOM widgets
  legitimately cover the canvas; locator actionability refused the
  click); the comprehensive spec drives wxChoice through its native
  <select> (browser-owned popup cannot be coordinate-clicked)
- docs: README/CLAUDE.md/build.md script names and dirs;
  features/wx-dom-port README reframed (DOM is THE port), visual-notes
  bugs 26-28; FindwxWidgets.cmake config label drops 'wasmuniv'
- wxwidgets submodule -> 9dbacc9448 (DOM-only port, fork diff shrunk)

Gate: full wx e2e suite 292 passed / 1 skipped / 0 failed — first run
ever with the gal-webgl specs green (28 scenarios + load + sequential).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:19:48 +02:00
Viktor Vaczi
7fdd635448 fix(wasm-dom): out_dir survives upstream's pipelined postprocess split
Upstream 680edd's pipelining moved dyncall/finalize/asyncify into
postprocess_app(), where build_app's local out_dir was out of scope —
the DOM flavor post-processed '/pcbnew.js'. Shared out_dir_for() helper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:18:48 +02:00
Viktor Vaczi
dba03b237f feat(wasm-dom): Phase 7 — KiCad pcbnew builds and runs on the DOM port
Pipeline: lexer generation (pcbcommon) ordered before the embind
pre-compile — a fresh-build-tree chicken-and-egg any new volume would
hit (generated pcb_lexer.h was included via -I${KICAD_BUILD}/common
before anything generated it). DOM flavor outputs flow through the
host-mounted output/dom/. pcbnew e2e on DOM: editor loads fully (layer
manager, toolbars, WebGL viewport, registry metrics), draw-lines test
passes; the header pixel-reference assertion fails as expected (native
DOM styling vs univ pixels) — port-aware references pending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:18:05 +02:00
Viktor Vaczi
74c392f88f feat(wasm-dom): Phase 7 plumbing — KiCad DOM-flavor build + test pipeline
WX_PORT=dom flows host → docker container → build scripts:
build-kicad-target.sh selects wxwidgets-dom + kicad-<app>-dom trees;
docker/build.sh passes the env through, copies outputs to output-dom/
incl. wx-dom.js, and post-processes there; build-wxuniversal-wasm.sh
honors WX_PORT and accepts --no-clean again (the kicad pipeline calls it
with that flag — Phase 1's stricter parser had broken the canvas docker
path). setup-kicad-wasm.sh syncs DOM bundles into tests/apps-dom/kicad
and injects the wx-dom.js script tag into the kicad pages;
playwright-kicad.config.ts serves apps-dom under WX_PORT=dom.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:18:05 +02:00
Istvan Matejcsok
680eb9dc15 perf(build): 4h05m -> 1h15m full CI build — docker caps, pipelined wasm-opt, Binaryen 130
Three orchestration fixes, validated end-to-end on the Hetzner ccx53
(run 27280051992, 1h14m41s vs 4h05m baseline, e2e identical):

- docker-compose CPU/memory caps were hardcoded to dev-Mac defaults
  (10 CPUs / 32G); now env-tunable via KICAD_DOCKER_CPUS/KICAD_DOCKER_MEM
  (CI exports nproc/110G — the 32-core runner was compiling on 10 cores).
- docker/build.sh: split build_app into compile_app (container) +
  postprocess_app (host-side dyncall shims + finalize + asyncify + -O2)
  and added KICAD_PIPELINE=1 mode that overlaps each tool's host-side
  wasm-opt with the next tool's container compile (max 2 concurrent
  postprocesses — pcbnew -O2 peaks ~34G RSS). Also: comma-separated app
  lists for cheap pipeline repros.
- get-wasm-opt.sh: Binaryen default 121 -> 130 (fixes the v121 -O2 lock
  convoy, ~9x) and BINARYEN_BUILD_FROM_SOURCE=1 support: the official
  x86_64-linux release tarballs (Alpine/musl, no LTO, assertions on) run
  asyncify 4x slower than a stock gcc -O3+LTO build with sha256-identical
  output; self-build takes ~5 min and is instantly repaid.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 07:48:46 +02:00
Balint Ipkovich
4467bcd2c4 feat: gerbview WASM port 2026-06-03 14:23:57 +02:00
Gergő Törcsvári
735e5aa8e9
feat(web): checkpoint web app init
- frontend (Vite/React) + server (Hono/Drizzle) scaffold under web/
- eeschema WASM embind kicadOpenFile hook + programmatic open-flow
- skip KiCad first-run setup wizard by seeding default config in preRun
- dev: auto-sync output/ WASM artifacts into tests/apps/kicad via link-wasm

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 20:32:19 +02:00
Balint Ipkovich
e28f473040 feat: symbol_editor WASM port 2026-06-02 13:37:26 +02:00
Istvan Matejcsok
c7a71ff2a7 chore: 🤖 add build monitor 2026-06-01 17:37:00 +02:00
Balint Ipkovich
d735779e23 feat: pl_editor WASM port + browser file dialog fixes
Brings up KiCad's pagelayout_editor (drawing-sheet editor) in the
browser, to roughly the same "boots, canvas visible, partially usable
in-session" level as the existing pcbnew/eeschema/calculator ports.

Build:
- docker/build.sh: add pl_editor to the unified app dispatch (case,
  subdir map, all-loop).
- scripts/kicad/build-kicad-target.sh: add pl_editor to the case;
  upstream target name pl_editor under source subdir pagelayout_editor.
- scripts/kicad/build-pl_editor.sh: 7-line thin wrapper matching the
  pcbnew/eeschema/calculator pattern.
- tests/scripts/setup-kicad-wasm.sh: copy_app pl_editor.

App glue:
- wasm/stubs/nl_pl_editor_plugin_stub.cpp: no-op SpaceMouse plugin so
  pl_editor_frame.cpp's NL_PL_EDITOR_PLUGIN symbols resolve. Mirrors
  nl_pcbnew_plugin_stub.cpp.
- tests/apps/kicad/pl_editor.html: browser shell. preRun creates
  /home/kicad and FS.chdir there so file dialogs land somewhere
  friendly instead of MEMFS root (/dev/, /proc/, etc.).

E2E coverage:
- tests/kicad/pl_editor.spec.ts: 5 tests — smoke (canvas, no abort),
  wizard, File menu has Open/Save As, file-dialog folder-navigation
  regression, canvas + toolbar metrics.
- tests/e2e/filedialog-folder-nav.spec.ts: wxWidgets-level twin of
  the regression test (exercises the underlying widget directly via
  the standalone filedialog_test app).

Submodule bumps:
- kicad → feature/pl-editor (WASM gating in pagelayout_editor's
  CMakeLists + navlib stub).
- wxwidgets → feature/pl-editor (wxGenericFileDialog::OnOk navigates
  into selected directories; wasm/mouse.cpp emits wxEVT_LEFT_DCLICK
  via timestamp-based double-click detection — the latter benefits
  every wxWidgets-WASM app).

See features/pl-editor/ for the design doc + per-repo diff patches.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 10:30:56 +02:00
Gergő Törcsvári
d6d743340d
refactor(build): fold calculator into unified docker/build.sh dispatch
Convert the calculator port (commit 0464470) from the parallel/copy-paste
pattern to the dispatch pattern used by pcbnew + eeschema.

- docker/build.sh: <app> is now required (no pcbnew default); missing,
  unknown, and --help paths all print "pcbnew | eeschema | calculator | all".
  Adds calculator to build_app() via kicad_subdir_for() (pcb_calculator
  inner subdir, calculator.* output basename). Also fixes the
  EMSDK=/emsdkkicad-wasm-builder typo that snuck in during the calc merge.
- scripts/kicad/build-kicad-target.sh: accepts calculator; introduces
  KICAD_TARGET (pcb_calculator for calc, == APP_NAME otherwise) used for
  the make target, embind include path, and final-log line.
- scripts/kicad/build-calculator.sh: 305-line copy of build-pcbnew.sh
  collapsed to a 7-line wrapper around build-kicad-target.sh.
- tests/scripts/setup-kicad-wasm.sh: copy_app calculator added with the
  same pcb_calculator subdir mapping for the docker-volume fallback path.
- tests/package.json: test:calculator* routed through the shared
  playwright-kicad.config.ts kicad/calculator.spec.ts (mirrors eeschema).
- Delete: docker/build-calculator.sh, tests/scripts/setup-calculator-wasm.sh,
  tests/playwright-calculator.config.ts.

Bumps wxwidgets d1d1627 -> 6fb2eac (origin/wasm-port). The new sha includes
"unic/combobox: add GetCurrentSelection() inline default" which calc's
kicad/pcb_calculator/widgets/unit_selector.cpp needs to compile. Without
this bump the unified dispatch would expose calc as a buildable target but
the build itself would fail. Verified: build.sh all completes clean across
all three apps; pcbnew/eeschema e2e pass; calc compiles and launches (test
stability separate from this refactor).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 16:01:18 +02:00
Gergő Törcsvári
730147d690
feat(schematic): eeschema WASM build + e2e harness 2026-05-29 16:01:17 +02:00
Istvan Matejcsok
0464470733 add calculator build 2026-05-28 17:28:17 +02:00
Viktor Vaczi
a49ed49d5d wasm: validate kicad hi-dpi scaling 2026-03-22 12:48:05 +01:00
Viktor Vaczi
b46b4b69f1 fix(wasm): Fix asyncify rewind with asyncify-aware dynCall shims
Emscripten 4.x removed dynCall_* WASM exports, breaking asyncify
rewind through indirect calls (modal dialogs, event handlers).
Generate JS shims that track Asyncify.exportCallStack and register
in wasmExports so doRewind can find them.

Also fixes empty callback functions ((() => {})) generated by
Emscripten 4.x + pthreads for HTML5 events, pthread entry,
sighandler, async timer, and main loop callbacks.

Build pipeline improvements:
- Stub wasm-opt/finalize in Docker (RAM limits), run on host
- Add setup-emsdk.sh for reproducible Emscripten setup
- Simplify env.sh and version management

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:26:48 +01:00
Viktor Vaczi
a0a6023400 Fix Docker build caching with checksum-based rsync
Replace timestamp-based rsync + touch with checksum-based sync:
- Use --checksum to only transfer files with actual content changes
- Remove touch mechanism that caused all files to appear "newer"
- Exclude .git, logs, .idea, node_modules from sync

The old approach created a perpetual timestamp mismatch:
1. rsync -a preserved host timestamps
2. touch set container timestamps
3. Next build saw mismatch → transferred everything → touched again

With --checksum mode:
- Changed files: transferred with current time → rebuilt (correct)
- Unchanged files: not transferred → keep timestamp → no rebuild (correct)

Tested scenarios:
- Consecutive builds: ~2 min (cached) vs ~6+ min (was full rebuild)
- Modified wxWidgets file: only that file recompiled
- Modified KiCad file: only that file recompiled

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 10:20:08 +01:00
Viktor Vaczi
ea5dc26a96 Restore elementtracker.h and add build logging
- Restore missing elementtracker.h in wxWidgets (required for E2E tests)
- Add automatic log file redirection for build scripts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 11:55:54 +01:00
Viktor Vaczi
2b96d4ba21 Add Quick Start docs and branch-specific Docker builds
- README.md: Add Quick Start section with full build sequence
- docker/README.md: Document branch-specific builds with git worktrees
- docker/build.sh: Use branch name for COMPOSE_PROJECT_NAME
- docker/docker-compose.yml: Remove hardcoded container_name

Each branch now gets isolated Docker containers/volumes, enabling
parallel development across git worktrees.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 09:08:12 +01:00
Viktor Vaczi
b5d379fe70 Add runtime resource loading and fix Docker incremental builds
- Build images.tar.gz bitmap archive for KiCad icons
- Fetch resources at runtime via HTTP instead of bundling in WASM
- Write to Emscripten virtual FS at compile-time KICAD_DATA path
- Fix rsync timestamp handling for correct make change detection
- Update kicad submodule with WASM settings manager fix

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 20:09:26 +01:00
Viktor Vaczi
dafef9c0a1 Fix Docker macOS timestamp issues and improve build reproducibility
- Add rsync-based source sync for Docker builds to fix macOS VirtioFS
  timestamp inconsistencies that caused autoconf sanity checks to fail
- Add config.sub wrapper (SHELL/CONFIG_SHELL) to support emscripten/wasm32
  targets without modifying submodule files
- Fix protobuf build to use native gcc/g++ for protoc instead of Emscripten
- Add python PATH fix for macOS (Homebrew's python3 symlink) in wxWidgets build

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 01:07:44 +01:00
Viktor Vaczi
9c271996c2 Fix KiCad WASM build and runtime errors
Build improvements:
- Change -O0 to -O1 to fix "local count too large" asyncify error
- Add wasm-emscripten-finalize to host (Docker OOMs on large WASM)
- Use -gseparate-dwarf for smaller main binary with debug info
- Build native protoc for code generation
- Add more functions to asyncify removelist

Runtime fixes:
- Add inject-dyncall-shims.sh to fix "dynCall_* is not defined" in Emscripten 4.x
- Update wxwidgets with browserInfo.name fix

New stubs and bindings:
- Add Embind bindings for JavaScript interop
- Add stubs for scripting, API plugin, PCB frame, navlib

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-22 16:30:53 +01:00
Viktor Vaczi
ad5c4f6772 Fix pthread deadlock by using dynamic PTHREAD_POOL_SIZE
The KiCad WASM build was freezing during initialization due to a pthread
deadlock. When hardware_concurrency() > PTHREAD_POOL_SIZE, new Web Worker
creation requires the event loop, but the main thread busy-waits blocking it.

Changes:
- Use navigator.hardwareConcurrency for PTHREAD_POOL_SIZE in KiCad build
- Add threadpool test to reproduce and verify the fix
- Add single-target build support to build-wasm-test.sh
- Default docker/build.sh to -j 10 for faster builds

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 11:47:50 +01:00
Viktor Vaczi
6771e7de74 Fix Asyncify and improve build process
Build system improvements:
- Add wasm-opt stub for Docker build to bypass asyncify in container
- Add apply-asyncify.sh script for host-side asyncify transformation
- Update build-pcbnew.sh to install wasm-opt stub and add EXPORTED_RUNTIME_METHODS
- Add --with-zlib=sys to wxWidgets configure for proper zlib linking
- Improve update-baseline-screenshots.sh to only copy new/significantly changed files

wxWidgets submodule update:
- Fix EM_JS to EM_ASYNC_JS for proper Asyncify registration

Test updates:
- Add new baseline screenshots for bitmapbuttons, dnd, gridrenderers, printpreview, specialized tests
- Add kicad-pcbnew initial screenshot
- Fix wxgrid-controls.png and wxgrid-dedicated-page.png (were showing errors)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 11:59:50 +01:00
Viktor Vaczi
b70f098ae2 Add two-phase build with host asyncify transformation
- Move asyncify from Docker to host to avoid memory issues
- Auto-download Binaryen v121 (v125 has regression bug)
- Use -O1 for debug builds (V8 local count limit)
- Remove asyncify flags from linker (handled by wasm-opt)
- Document two-phase build in build.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 22:13:31 +01:00
Viktor Vaczi
d8d934ffca Enable exceptions and pthreads atomics for KiCad WASM build
Build fixes:
- Enable C++ exceptions in wxWidgets (--enable-exceptions, -fexceptions)
  Required for KiCad's OnExceptionInMainLoop override in debug builds
- Add -matomics -mbulk-memory to all dependency builds for shared memory
  Required for pthreads support with Emscripten
- Fix docker/build.sh output copy path (pcbnew/ not bin/)
- Make workspace mount read-only in docker-compose.yml

Updated dependency scripts:
- build-freetype.sh, build-harfbuzz.sh, build-protobuf.sh
- build-zstd.sh, build-opencascade.sh, build-boost.sh
- build-pixman.sh, build-cairo.sh (meson cross-file)
- build-wxuniversal-wasm.sh, build-pcbnew.sh

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 11:29:45 +01:00
Viktor Vaczi
b5e46a0aba Add Docker build environment for KiCad WASM
- Add Dockerfile with ARM64-native emscripten/emsdk:4.0.2-arm64 image
- Add docker-compose.yml with resource limits (10 CPUs, 16GB RAM)
- Add helper scripts (build.sh, shell.sh, entrypoint.sh)
- Add Docker README with usage instructions
- Update playwright.config.ts to find free port dynamically
- Add .dockerignore and .gitignore entries for build artifacts

The Docker environment provides reproducible builds with:
- Named volume for build cache (faster I/O on macOS)
- Resource limits to prevent system lockups
- Interactive shell access for debugging

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-08 12:07:01 +01:00