Commit graph pcbjam/wasm/shims
Author SHA1 Message Date
Viktor Vaczi
9c475a804e jspi cleanup: remove the asyncify-era residue — dead code, conditionals, pipeline scaffolding, stale prose
The runtime is JSPI-only; this removes everything that still pretended
otherwise. Three exhaustive sweeps (C++/JS+build+CI/tests+docs) drove
the inventory; every deletion verified by grep closure + full gates.

Broken-right-now fixes:
- deploy-staging.yml passed the retired opt_level input — the workflow
  could not even start. Removed.
- env.sh carried dead exports with a live -sASYNCIFY=1 inside
  (WASM_LDFLAGS/PTHREAD_LDFLAGS, zero consumers). Removed; the
  WASM_LEGACY_EXCEPTIONS rationale rewritten to the real reason.
- docker/build.sh exported PCBJAM_ASYNC_BACKEND (read nowhere). Gone.

Dead weight removed:
- binaryen submodule (nothing builds or invokes it), wasm-opt-bench
  workflow + scripts/bench/, get-wasm-opt.sh, diagnostics.js (242 lines
  of Asyncify-API-only code), the KICAD_PIPELINE background-postprocess
  scaffolding (existed to parallelize the deleted wasm-opt phase; the
  postprocess is a seconds-long node script and now runs inline),
  build-monitor's dead asyncify rows, sched-context orphan build
  output, dead .gitignore entries, the .jspi-assets spike dir (the two
  wf-result research JSONs moved to docs/features/async/migration-evidence/).
- bindings: fiber_park.h + its 12 embind registrations (broken-if-
  called under JSPI), the kicadOpenFileStart/OPEN_JOB starter route,
  main_stack_runner.h + 5 includes, the always-null context-sleep weak
  hook in nanosleep_yield.c.
- shim: the backend field (installed-flag idempotency instead),
  noteContextWait (dead both sides), the __wxAsyncifyDump alias (+ the
  WasmTool fallback and string-dump normalize branch).
- web: the emscripten-6-ignored mainScriptUrlOrBlob option in boot.ts
  (gerber-demo keeps it: it loads the deployed CDN release, which
  predates emscripten 6 — noted inline).

Conditionals: all 'backend === jspi' checks reduced to scheduler-
presence checks; races_quiescent re-keyed from Asyncify.state (vacuous)
to real backlog quiescence (resumeReady/mutatorQueue — NOT _windowLive,
which is the probing activation's own window by definition).

Renames (identifiers only, no file renames): ASYNC_LINK_FLAGS→
JSPI_LINK_FLAGS and Makefile ASYNC_LDFLAGS→JSPI_LDFLAGS,
kicadCollabFiberBusy→kicadCollabBusy (embind + web + tests),
collab_common.h fiber*→apply*/coroutine naming, asyncifySignatures→
wasmTrapSignatures (lists byte-identical).

Tests: the two remaining vacuous [wx-asyncify]/fiber-resume-refused
asserts re-keyed to live JSPI beacons; eeschema-load's failure message
no longer sends the developer to a deleted script; wait-beacons' dead
families/parser deleted; lane-0 legacy-glue guards removed (lane 0 is
unconstructible); the embind test.fail re-gated with the JSPI reason
(plain embind invokers cannot suspend — verified still failing);
lint-determinism now scans tests/jspi (166 files clean);
eeschema-collab local-move gated to chromium (~50% flaky on FF even
solo; pcbnew twin covers both engines).

Docs: DEBUG.md rewritten as the JSPI debugging guide; build.md
describes the single-phase build; docs/features/async/README.md
banner-marked historical and repointed at the NEW
23-jspi-runtime.md (current architecture: export census, turnstile,
libcontext ownership + refusal contract, embind call shapes, the
em-pthread service-wrapper trick, exception policy, known gaps).

Gates on the cleaned tree: test:e2e 725 passed / 0 failed (after the
quiescence-probe fix; the 3 other reds were verified contention flakes
solo-green or the documented FF gate), web 76/0, jspi 18/18 both
engines, vitest 295/295 + 17/17, all lints green, live-app census
clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016X9eh1s5sTx1o9Em9KBuwR
2026-08-14 09:25:32 +02:00
Viktor Vaczi
94cac5e4d4 port staging 29c61b8's zero-duration nanosleep guard (allocator must not suspend)
mimalloc's mi_atomic_yield() is sleep(0) on wasm, reached from malloc's slow
path under cross-thread delayed-free contention. Under JSPI a yield there
suspends the activation INSIDE the allocator — any other activation that runs
next can re-enter mimalloc mid-operation. ms==0 now busy-returns like stock
emscripten. (The removelist half of the staging fix is obsolete here — no
asyncify, no removelist. The mimalloc-storm harness + spec arrive with the
staging merge.)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NDeBaKKhQztd8KiVtHuyXr
2026-08-13 09:04:43 +02:00
Viktor Vaczi
e14faeca8b jspi: retire the asyncify pipeline — knob, post-link tail, binaryen hooks
Phase 8 in the parent repo. Deleted: asyncify-scheduler.js, apply-asyncify.sh,
apply-finalize.sh, inject-dyncall-shims.sh, asyncify-imports/removelist.txt,
the wasm-opt/finalize stub pair, scripts/binaryen-hoist-pass/ (the fork stays
a dormant submodule; removal is a follow-up), bench/wasm-opt-bench.sh (README
marked historical), wasm/shims/context_sleep.cpp, and the sched-context
harness app + Makefile targets.

PCBJAM_ASYNC_BACKEND is gone: build-wx-wasm.sh hardcodes the jspi stamp
(still force-cleans pre-migration trees), build-kicad-target.sh gives editors
the JSPI link surface and the CLIs nothing (they pin ASYNCIFY=0), the stub
dance is replaced by an unconditional .real-restore, build-wasm-test.sh lost
its whole post-link loop, docker/build.sh's postprocess is the ENV shim only,
and Makefile.wasm links every app JSPI with the scheduler shim as a tracked
prerequisite. pcbjam_async_policy.h keys on __EMSCRIPTEN__.

jspi-scheduler.js: wxWasmMainLoopPump dropped from the wrap census (the
export died with the D5 detach); inert [TRACE] instrumentation removed.

CI: wasm-build.yml rewritten for the single-cache pipeline (one output cache
keyed on compile inputs; post-processed bytes cached after the shim);
opt_level input removed from both callers. wasm-cache-hash.mjs inputs now
cover patch-env-shim.mjs + jspi-scheduler.js + jspi-exports.txt.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NDeBaKKhQztd8KiVtHuyXr
2026-08-13 08:39:12 +02:00
Viktor Vaczi
3f09a46ff5 jspi: migration phases 0-7 — build knob, scheduler shim, test successor suite
Toolchain: emsdk 6.0.6 (versions.sh; cache-hash keys on it). Build knob
PCBJAM_ASYNC_BACKEND=jspi|asyncify: build-kicad-target.sh links editors with
-sJSPI + -sJSPI_EXPORTS=@scripts/common/jspi-exports.txt + --pre-js
jspi-scheduler.js (no DYNCALLS, no post-link asyncify pipeline); wx build
stamps the backend and forces clean on flip or unknown provenance;
docker/build.sh passes the knob, seeds the emscripten ports cache from the
volume every launch, jspi postprocess = patch-env-shim only.

scripts/common/shims/jspi-scheduler.js: the JSPI successor scheduler —
token-wait registry, resume turnstile (one armed resume between engine
re-entries, SP swaps only at microtask boundaries), green-region spill
stacks (16-aligned tops), S1 embind mutator FIFO lane + parker wraps, S6
shutdown, libctx integration hooks (suspend/end/quarantine + g_current
arm/clear), SuspendError attributor, lost-wake + stuck-window watchdogs,
__wxWaitDump observability.

Embind: PARKER registrations get emscripten::async() under PCBJAM_JSPI
(wasm/bindings/pcbjam_async_policy.h). nanosleep yields route via the shim.

Tests: tests/asyncify -> tests/jspi successor suite (jspi-stack red/green
shadow-stack battery, jspi-coroutine MiniCoro harness, suspend-races
semantic scenarios + __wxWaitDump books coherence); projects jspi-firefox/
jspi-chrome (asyncify-webkit retired — no JSPI in WebKit); unconditional
Firefox JSPI pref; guard-beacons -> wait-beacons (+wxScheduler/libctxJspi
families); Makefile.wasm links test apps against JSPI with the shim as a
tracked link prerequisite.

Web: WasmTool setRo await + __wxWaitDump forensics, open-flow contained
promise, scheduler-shim.test.ts retargeted (8 green).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NDeBaKKhQztd8KiVtHuyXr
2026-08-13 07:06:24 +02:00
Gergő Törcsvári
75209dc07e
Phase E retry: K1 lands — early-resolve retention, not the lossy guard; pthread-ondemand link fixed
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L2SU74acXyviwSxBhMunFe
2026-08-10 10:14:19 +02:00
Gergő Törcsvári
ab6e44e57f
docs 22 + wx bump: DOM entries on the dispatch context; canvas tools green at D-on
The increment doc 22 ordered last round. Every entry that can reach a tool
coroutine - the four DOM callbacks and the mailbox tick - now goes through the
scheduler, so a coroutine is never entered by a star transfer from one path and
a direct symmetric swap from another. At D-on all four canvas-tool specs are
GREEN and the KiCad suite is 136/3 (was 135/5 before the sleep work, with the
tools red throughout).

context_sleep's wake learned the mirror lesson: now that the mailbox runs ON a
context, it must NOT call drain_all from there (drain refuses re-entry, and
should) - it marks ready and lets the outer drain_all perform the entry, with
an armed pump as a backstop.

Recorded honestly, not papered over: two of the three remaining D-on failures
are the timer-park and quasimodal-strand levers, each failing ONE assertion -
"scheduler shim observed the concurrent-park window", expected >0, got 0 -
while fired/done/parked/errors all pass. That counter needs TWO concurrent
in-place Asyncify parks, and the lever stages "timer park x MAIN-LOOP YIELD
PARK"; D5 removed the main loop's Asyncify park, so the overlap cannot occur.
Re-pinning those levers to the post-migration invariant is a Phase F decision
alongside fiber-resume-park's red->green flip, NOT an assert to relax now.

Landing state: STAR_DISPATCH=0, kicad 139 passed / 1 (pre-existing occ-probe).
Next: Phase E - the K1-K7 bridges are the only in-place parks left under a
context, and are exactly what the current()!=0 fallback still tolerates.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LBjomQfKyRa3jBdeAKpmTw
2026-08-10 10:14:19 +02:00
Gergő Törcsvári
80468e7c5b
Phase B: main-thread sleep parks its context; the real D blocker is DOM entries
wasm/shims/context_sleep.cpp: a main-thread nanosleep whose frame stands on a
scheduler context that OWNS the stack arms a mailbox wake and yield_parks that
context instead of suspending the stack in place. It lives in the sleep
primitive rather than in tool_manager.cpp on purpose - KiCad and the wx core
stay untouched (CLAUDE.md's fork rule) and the whole K7 class moves at once,
not just TOOL_MANAGER::RunSynchronousAction's spin loop.

MEASURED AT D-ON, and it is NOT what unblocks Phase D. The four canvas-tool
specs still fail, but the trace now names a different cause: the fatal swap is
old=<libcontext ROOT> new=<tool coroutine> with mouseEventHandlerFunc above it
- a DOM mouse handler entering wasm DIRECTLY on the main stack, bypassing the
tick. So one coroutine is entered two ways: by the tick through the dispatch
context as a STAR TRANSFER, and by DOM handlers as a DIRECT SYMMETRIC SWAP. A
capture written by one path cannot be rewound by the other -> index out of
bounds in doRewind. That is section 7 rule 5 (partial migration is worse than
none) in its purest measured form, and it is why the harness stays green: its
coroutines are only ever entered from one place.

So the next increment is the DOM event entries (mouse/key/wheel/resize must
hand their events to the dispatch context as the tick does), not another park
site. It subsumes the one-root work too: with no dispatch on the main stack,
resolve_root_identity() always answers "the running context".

Landing state: STAR_DISPATCH=0, kicad 139 passed / 1 (pre-existing occ-probe)
= baseline, with the sleep shim in and inert.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LBjomQfKyRa3jBdeAKpmTw
2026-08-10 10:14:19 +02:00
Gergő Törcsvári
88aaba99c9
feat(libs): react load overlay + progress bar; mimalloc mallinfo stub; framed tests
- WasmTool/source.ts: LIB_LOADING_EVENT brackets the fat-load; full-cover
  overlay ('Loading <kind> libraries…') with a per-lib progress bar (done/total
  from listLibs), show-on-first / hide-after-last coalescing.
- mallinfo stub (wasm/shims/mallinfo_stub.c + build-kicad-target.sh): -sMALLOC=
  mimalloc doesn't export glibc mallinfo() that OpenCASCADE OSD_MemInfo (pcbnew
  3D) needs; zeroed no-op unblocks the pcbnew link.
- source/cdn-source tests: updated to the framed Uint8Array 'bodies' contract.
- bump kicad -> e8db3d3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 09:56:32 +02:00
Viktor Vaczi
c1ef489cfa feat(wasm-eh): migrate the WASM build to native wasm exceptions (+ 3D viewer default-on)
Replace the legacy Emscripten JS-exceptions model with native wasm-EH (legacy
encoding) across the whole build, keeping Asyncify coroutines working via a
from-source Binaryen --hoist-cpp-catches pre-pass. Net result: native-EH is the
only build mode, the 3D viewer is on by default, and pcbnew shrinks substantially.

Highlights:
- Binaryen submodule everywhere + --hoist-cpp-catches integration in apply-asyncify;
  post-link Asyncify covers every app wasm (not just standalone test wasm).
- Build deps (incl. OpenCASCADE without OCC_CONVERT_SIGNALS) and all KiCad apps
  with -fwasm-exceptions; emscripten_sleep added to the post-link asyncify-imports.
- libcontext fiber entry wired under native exceptions; while-loop main loop +
  currData shim injected into all wx apps.
- Native-EH collab apply fixed: DEBUG-define the embind TU + match all out-of-CMake
  C++ TUs' ABI flags to the core, fixing the vtable-layout skew / mis-dispatch.
- 3D viewer enabled by default (real raytracer linked, not the stub).
- Retire the EH-spike scaffolding; flip the asyncify-races ablation pins to
  shim-redundancy pins (native-EH stays clean with the legacy shims ablated).
- Fix the asyncify-races quiescence check to not require Asyncify.currData==0:
  under the native-EH per-frame-yield top loop the main stack is asyncify-suspended
  every frame, so currData legitimately churns (a freed-but-not-yet-nulled buffer,
  not a leak). Refresh the pcbnew toolbar screenshot baseline for the new kicad.
- CI: drop the obsolete binaryen_version input/env (the build uses the binaryen
  submodule fork's wasm-opt, not a version download); key the wasm-output cache on
  the binaryen submodule SHA instead.

Bumps the wxwidgets + binaryen submodules to their squashed feature commits.

Validated green: all 7 apps native-EH (real 3D in pcbnew); KiCad e2e 63/63
Firefox + Chromium (3D viewer renders); wx 336; coroutine 34/34 both engines;
asyncify 7/7 both engines.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 09:40:26 +02:00
Istvan Matejcsok
0195ff0941 feat: enable WASM 3D viewer build — FFP link stubs, drop glemu shim
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 09:21:37 +02:00
Istvan Matejcsok
68c711e633 feat: enable the 3D viewer in the WASM port (WIP)
Builds KiCad's 3D viewer for the browser (opt in with BUILD_3D_VIEWER=ON) and
wires up the legacy-GL emulation it needs.

- build-kicad-target.sh / docker/build.sh: BUILD_3D_VIEWER →
  -DKICAD_BUILD_3D_VIEWER_WASM=ON + -sLEGACY_GL_EMULATION + the GL js-library.
- wasm/shims/gl_immediate_shim.js: display-list emulation (record/replay board
  layers), fixed-function + GLU stubs, throw-guards for unsupported pnames, and
  per-context GLImmediate init for the viewer's SECOND WebGL context.
- tests/kicad/3d-viewer.spec.ts + utils/pcbnew-ready.ts: open View → 3D Viewer.
- bumps the kicad submodule (EMSCRIPTEN-guarded GL changes).

WIP: the viewer window + UI render and geometry draws (glError=0x0), but the
board is not yet visibly rendered — GLImmediate's FFP shader program isn't
linked on the viewer's 2nd WebGL context. Full status + next steps in
features/feat/add-3d-view/notes.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 09:13:41 +02:00
Istvan Matejcsok
e6ff74a11e fix: restore gl_immediate_shim.js
Restores the immediate-mode GL shim that the WASM runtime loads; it had
been dropped from the tree while the CI work was in flight.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 07:48:46 +02:00
Istvan Matejcsok
c148443b7b refactor: 💡 remove orphaned files 2026-06-05 13:18:24 +02:00
Viktor Vaczi
6c2cf85177 feat(webgl): Fix GL initialization and add GLSL ES shader conversion
Progress on WEBGL_GAL rendering:

- Fix GLImmediateShim initialization by adding GL function dependencies
  (glBegin, glEnd, glVertex2f, etc.) to $GLImmediateShim__deps
- Add COLOR4D(EDA_COLOR_T) constructor stub to wasm_stubs.cpp
- Fix GAL context locking - add LockContext/UnlockContext calls
- Add exception catching to build (-sNO_DISABLE_EXCEPTION_CATCHING)
- Create GLSL ES 1.00 shader converter in generate_shaders.py:
  - Remove #version 120 directive (ES 1.00 default)
  - Add precision highp float/int qualifiers
  - Convert int * float to float literals (2 * x -> 2.0 * x)
- Update build script to clean output files for linker flag changes
- Add console capture to test spec for debugging

Current status:
- WASM builds and loads successfully
- GLImmediateShim initializes correctly
- WEBGL_GAL creates successfully
- Shader compilation fails due to Emscripten LEGACY_GL_EMULATION
  prepending code that conflicts with custom shader precision

Next: Investigate shader compilation with LEGACY_GL_EMULATION or
consider alternative approaches for GL compatibility.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 21:59:21 +01:00
Viktor Vaczi
a26b40c063 Reorganize project structure for clarity
- Rename tests/wasm-app/ to tests/apps/ for brevity
- Move lib/gl_immediate_shim.js to wasm/shims/ (consolidates WASM files)
- Delete docs/ directory (outdated Nov 2024 research docs)
- Delete patches/ directory (already applied to fork submodules)
- Delete wasm/patches/ and wasm/config/ (empty/unused)
- Update all file references in scripts, configs, and documentation
- Update .gitignore for new tests/apps/ paths

Verified both build workflows pass:
- wxWidgets tests: 255 passed
- Docker KiCad build: completed successfully

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 14:06:23 +01:00