Commit graph pcbjam/scripts
Author SHA1 Message Date
Istvan Matejcsok
c6716e6b60 feat(wasm): unify editor builds — footprint/symbol editors via runtime --frame
The footprint and symbol editors are no longer separate WASM bundles: the frontend loads the parent pcbnew/eeschema bundle and passes --frame=fpedit / --frame=symedit (TOOL_BUNDLE + TOOL_FRAME -> Module.arguments in boot). Drops the two duplicate build+deploy targets and their wrapper scripts + vestigial embind; adds low-level harnesses (footprint_editor.html, symbol_editor.html) and a runtime-frame spec. Bumps the kicad submodule to the runtime --frame launcher.

The frame-runtime spec is listed in PCBNEW_FAMILY_SPECS so CI routes it to the chromium-ci (V8) project — its footprint case boots the pcbnew module, which OOMs SpiderMonkey on x86 CI. Includes the editor-unification dossier (research docs 01-04 + the as-built implementation record 05).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 17:56:02 +02:00
Viktor Vaczi
e01256b54d fix(ci): binaryen tools cache also needs lib/ — bin-only restore broke every post-process
The binopt cache from b4cb10b saved only binaryen-hoist-build/bin, but the
tools dynamically link lib/libbinaryen.so, so the first cache HIT (run
28585074335) failed all 7 tools' finalize with a loader error. Cache bin/ +
lib/, bump the key to binopt-v2 (Ubicloud never re-saves an exact-key hit, so
the poisoned v1 entry can't be repaired in place), and make the
BINARYEN_TRUST_PREBUILT guard exec both tools with --version instead of -x
existence checks so an incomplete restore falls through to a source build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NTxTrY9on6A8gLvogFbGiy
2026-07-02 14:29:20 +02:00
Istvan Matejcsok
7630c7e521 fix: 🐛 raytrace deadlock - pre-warm 2N+8 pthread Workers 2026-07-02 13:01:02 +02:00
Viktor Vaczi
b4cb10b1c6 perf(ci): cache the per-run toolchain + test-app work (binaryen, emsdk, apps, browsers)
Four new caches in wasm-build.yml, keyed on their real inputs:

- Binaryen post-process tools (bin/ only, keyed on the submodule SHA): on a
  hit BINARYEN_TRUST_PREBUILT=1 tells build-wasm-opt.sh to skip cmake+ninja
  and trust the restored binaries (~46s/run). The guard is env-gated so local
  pass iteration (uncommitted sources, same SHA) is unaffected.
- Built wx test apps (tests/apps minus kicad/ staging + gal-webgl/): skips
  the whole build step on a hit (~3 min — the post-link hoist+asyncify over
  ~74 apps dominates). Key covers wx SHA + kicad SHA (some apps compile real
  KiCad sources: thread_pool.cpp, libcontext), binaryen SHA, tracked app
  sources, and the build/post-link scripts + shims. Gated on a wx cache HIT:
  the app build creates the libwx_*.a symlinks the GAL link needs, so on a
  wx rebuild the apps must rebuild too.
- Host emsdk (tools/emsdk, keyed on the pinned EMSCRIPTEN_VERSION): mostly
  availability insurance — a fresh install is only ~23s but pulls ~340 MB
  from github.com + storage.googleapis.com on every run. downloads/ tarballs
  are pruned before the post-job save.
- Playwright browsers (~/.cache/ms-playwright keyed on tests/package-lock).

Warm-cache runs should drop from ~12 min to ~8-8.5 min (e2e suites are the
floor). Measured baselines from runs 28577824366 / 28520845242.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XeiSRRScdaox5jBueJNcyG
2026-07-02 12:40:22 +02:00
Gergő Törcsvári
4330c9344e
feat(3d): live models CDN — 10.0.3 publish wiring + wrl→step fallback
- kicad-packages3D is STEP-only from the 10.x tags: ensureModelInMemfs
  falls back from a missing .wrl to the same-stem .step, written and
  answered under the .step path (the returned path's extension picks
  the parsing plugin — no C++ awareness). Verified against the live
  CDN with the gallery demo board (4/4 wrl refs served as step).
- upload-models-r2.sh: rclone/S3 bulk upload of a local publish layout
  (wrangler-per-object can't move ~14k blobs); published 10.0.3 —
  105 libs / 7,238 models, 3.4GB raw → 500MB brotli — to pcbjam-cdn.
- deploy-demo.yml MODELS_TAG=10.0.3 → build-demo --models-tag →
  VITE_MODELS_MANIFEST_URL (matches LIB_TAG: model refs come from the
  footprints at that release).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014AT7gVHRktDYoQ68S4x6A4
2026-07-02 11:35:37 +02:00
Gergő Törcsvári
6f3f3bcf00
feat(3d): lazy 3D model delivery R2→IDB→WASM
- cdnModelsSource: sparse per-lib stacks over the models CDN layout
  (libs/kicad-models/<tag>/<lib>/manifest + content-addressed blobs)
- models-bridge: board prescan ((model …) scan → prefetch, 4/4 in 74ms
  on the demo board) + the kind=model3d 'ensure' op answering absolute
  MEMFS paths; only board-referenced bodies ever enter IDB
- boot/runner/config/WasmTool wiring (VITE_MODELS_MANIFEST_URL,
  KICAD*_3DMODEL_DIR seeding, prefetch badge)
- StorageUsageCard on HomePage: per-kind cached sizes + delete-3D-cache
- publish-models.ts (brotli content-addressed publish, 606→93MB for the
  pic_programmer lib set) + dev-demo --models-tag/--models-local
- e2e: 3d-viewer-models.spec.ts (bridge normalize/dedup + STEP/WRL
  render); submodule bumps: kicad (static 3D plugins + ensure hook),
  pcbjam-shared (sparse sync layer)

Spec + findings: docs/features/3d-models (private repo).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014AT7gVHRktDYoQ68S4x6A4
2026-07-02 11:35:36 +02:00
Viktor Vaczi
3efcdcf91c fix(tests): enforce screenshot gate; drop shadow dir, flaky retinascale, 12 stale
Finalize the screenshot review system after the first Linux re-baseline
(that run showed 355/356 stable — the placeholder floor is fine as-is):

- ENFORCE the gate: `screenshots:check --fail-on-change` now fails the build on any
  changed/added/removed vs baselines (it still posts the drift report first, so the
  failure is actionable; a real render change → re-promote). The generic "CI failed"
  notice is suppressed for gate failures so we don't double-post on drift.
- Exclude retinascale-01-loaded: a fullPage HiDPI test whose captured height + DPR
  scaling vary run-to-run (~60% inter-run diff) — a flaky test, not render noise.
  IGNORE_SCREENSHOTS in config.ts; compare/promote/gen-manifest skip it; baseline removed.
- Kill the baseline-dir shadowing: drop e2e/baseline-screenshots/ from BASELINE_DIRS +
  delete its 3 files (grid-tab-final, wxgrid-controls, wxgrid-dedicated-page) that
  duplicated names in baseline-screenshots/ with different bytes.
- Prune 12 stale baselines (renamed/removed specs: wizard-01..04, gerbview-wizard-01..04,
  zoom-pl_editor-*, popup-03-palette).
- Delete the dead compare-screenshots.sh / update-baseline-screenshots.sh.
- Regenerate screenshot-manifest.json (355 entries).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 11:30:33 +02:00
Viktor Vaczi
348596c515 perf(build): use all cores in host builds — wx lib, test apps, gal-webgl
env.sh exports a docker-safe JOBS=1 default, and every host build silently
inherited it (the docker kicad path dodges it by passing -j explicitly):

- build-wx-wasm.sh: the intended "${JOBS:-nproc}" fallback sat BELOW the
  env.sh source, so it was dead code — full wx builds ran make -j1 on the
  Mac and on CI. Compute the all-cores default before env.sh instead
  (explicit JOBS/PARALLEL_JOBS still wins); also -j the PCRE pre-build.
- build-wasm-test.sh: same default fix, plus fan the post-link
  hoist+asyncify loop out across JOBS with xargs -P. Per-app wasm-opt
  can't feed many cores (small modules), so serial stays ~4min even with
  BINARYEN_CORES=16; fanning across the 74 independent apps is what
  scales. Safe: apply-asyncify is in-place per wasm, injector tmp is
  per-js, HOIST_WASMOPT resolved once up front.
- build-gal-webgl-test.sh: make had no -j at all.

Measured on a 16-core M4 Max, clean test-app build: 15m36s (-j1) → 2m08s,
peak RAM 10.7 GB summed across all build processes (64 GB machine; CI
runners have 120 GB). Core counts are always derived from nproc at
runtime — nothing hardcoded, CI workers differ.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XeiSRRScdaox5jBueJNcyG
2026-07-02 10:54:14 +02:00
Viktor Vaczi
f49d536f95 fix(build): wx 'zlib.h not found' CI flake — self-heal zlib port + deterministic cache touch
Two fixes for the intermittent wx-build failure on CI (zipstrm.cpp: fatal
error: 'zlib.h' file not found, surviving the serial retry):

- build-wx-wasm.sh: --with-zlib=sys resolves to the Emscripten zlib PORT,
  which only `embuilder build zlib` installs — and that ran only in the
  configure branch. A pre-configured build dir (CI cache) + fresh emsdk
  therefore died on any recompile of a zlib-using TU. Ensure the port
  before every make; --force because embuilder stamps the port on libz.a
  alone, so a half-populated cache would no-op.

- wasm-build.yml: the cache-restore `touch {} +` stamped files ns-apart in
  readdir order; GNU make 4.x compares ns mtimes, so objects touched before
  a generated header they depend on (wx/setup.h, pcre2.h via .deps/*.d)
  recompiled — a random subset per run, which is what made the failure
  flaky. Use one shared timestamp instead: equal mtimes = up to date.

Reproduced + validated locally (remove sysroot zlib.h + one .o → exact CI
failure; with fix, self-heals in 0.5s; Linux make ns/equal-mtime behavior
verified in a container).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XeiSRRScdaox5jBueJNcyG
2026-07-02 09:59:32 +02:00
Gergő Törcsvári
b5ee3da92c
perf(cdn): publish demo WASM as brotli-q11 + no-transform
release.yml's publish-wasm job now uploads .wasm/.js with
`--compress br --quality 11` instead of gzip-6, and IMMUTABLE
content-addressed blobs carry `no-transform` so Cloudflare's edge
can't decompress + re-serve them at its on-the-fly br-4 (and the
original Content-Length passes through).

~15-20% smaller wasm on cdn.pcbjam.com. Forward-only: tool folders
already published stay gzip until their source changes (identity is
hashed over uncompressed bytes, so the compressor switch never
re-versions or re-uploads existing folders).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 12:21:54 +02:00
Gergő Törcsvári
e7640d0a97
build(wasm): standardize asyncify wasm-opt tail to -O1 everywhere
The Binaryen wasm-opt shrink after Asyncify was the only -O2 in the
pipeline: -O2 by default (local + tag release) but -O1 in main CI. The
C++/wx/deps compile is already -O1 everywhere (DEBUG_BUILD defaults to 1;
nothing passes --release in CI/release). Pin the tail to -O1 too:

- apply-asyncify.sh: BINARYEN_OPT_LEVEL default -O2 -> -O1
- release.yml: opt_level -O2 -> -O1 (demo now ships -O1)
- ci-ubicloud.yml / wasm-build.yml / docker/build.sh: refresh stale -O2 comments

Because both callers now build at -O1, the FINAL cache key converges, so a
tag release FINAL-cache-hits main's build and skips the asyncify-tail
rebuild entirely (previously the ~1-2h -O2 wasm-opt was rerun per release).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 12:18:57 +02:00
Gergő Törcsvári
89a433fda9
fix(trace): make ?trace= work by merging Module.ENV in the glue
The emscripten glue's `var ENV = {};` never merged Module.ENV, so boot.ts's
Module.ENV.KICAD_TRACE was a silent no-op — and since environ_get on a pthread
proxies to the main thread, the app/UI thread's getenv read the (empty) main
ENV. Net: KICAD_TRACE never arrived and every KI_TRACE was a no-op.

scripts/common/patch-env-shim.mjs post-processes each generated <app>.js to
merge Module.ENV into ENV (idempotent; runtime no-op when unset), wired into
docker/build.sh's per-app host post-process next to the dyncall-shim inject.
Replaces a fragile manual glue edit re-applied after every build. Verified:
symbol-editor eager load now emits KI_TRACE_SYM_CHOOSER fatLoad timing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 11:01:10 +02:00
Gergő Törcsvári
88aaba99c9
feat(libs): react load overlay + progress bar; mimalloc mallinfo stub; framed tests
- WasmTool/source.ts: LIB_LOADING_EVENT brackets the fat-load; full-cover
  overlay ('Loading <kind> libraries…') with a per-lib progress bar (done/total
  from listLibs), show-on-first / hide-after-last coalescing.
- mallinfo stub (wasm/shims/mallinfo_stub.c + build-kicad-target.sh): -sMALLOC=
  mimalloc doesn't export glibc mallinfo() that OpenCASCADE OSD_MemInfo (pcbnew
  3D) needs; zeroed no-op unblocks the pcbnew link.
- source/cdn-source tests: updated to the framed Uint8Array 'bodies' contract.
- bump kicad -> e8db3d3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 09:56:32 +02:00
Gergő Törcsvári
4025b05218
feat(libs): parallel s-expr parse + mimalloc + copy-as-is transport — symbol chooser cold 274s→44s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 07:58:30 +02:00
Viktor Vaczi
71807db734 feat(wasm,3d-viewer): run upstream multi-threaded CPU raytracer
Bumps the kicad submodule to the threading revert (the 3D-viewer raytracer's
WASM single-threaded #ifdef fallbacks + the bs_thread_pool inline shim are gone;
those four files are now byte-identical to upstream KiCad).

Links wasm/shims/nanosleep_yield.c into every app (mirrors the gl_ffp_stub
pattern): the now-upstream raw-thread raytracer joins via a main-thread
sleep_for busy-wait, and this shim makes that nanosleep Asyncify-yield to the JS
event loop instead of deadlocking on on-demand pthread-Worker creation. Its
EM_ASYNC_JS yield is already covered by env.__asyncjs__* in asyncify-imports.txt.
This is what lets upstream's multi-threaded code run on WASM under native-EH.

Also documents isolated-worktree + submodule-branch creation in the README.

Result: ~11x faster 3D raytrace (21.3s -> 1.9s, 16 threads), pixel-correct
render, full kicad e2e green (63 passed / 0 failed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 18:17:56 +02:00
Viktor Vaczi
714b0a816b fix(wasm-eh): build wasm-emscripten-finalize from the Binaryen submodule (unbreak CI)
The native-EH migration pointed apply-finalize.sh at emsdk's bundled
wasm-emscripten-finalize (${EMSDK:-tools/emsdk}/upstream/bin), which exists on a
dev machine (tools/emsdk persists from a prior build) but never on the ephemeral
CI host — emsdk lives only inside the Docker image, and nothing on the host
post-process path provisions it. So the host post-process died at finalize for
every app once the binaryen-SHA cache re-key forced it to actually run. (Not the
missing-ninja theory — ninja was installed; the run never reached the wasm-opt
build, which is downstream of finalize.)

Finish the "Binaryen submodule everywhere" migration: build-wasm-opt.sh now also
builds wasm-emscripten-finalize, and apply-finalize.sh takes it from that build
(next to wasm-opt). The host post-process is now emsdk-free (dyncall=node,
finalize+asyncify=submodule v130) and finalize/wasm-opt share one Binaryen
version (previously finalize was emsdk's v121, wasm-opt the submodule's v130).
get-wasm-opt.sh stays for bench only; dropped it from the wasm cache key and
added build-wasm-opt.sh.

Also surface the from-source Binaryen build as a "Build Binaryen" stage in
build-monitor.sh (it had no marker), emitted at the pipelined pre-warm sites.

Validated: a clean from-scratch `docker/build.sh calculator --build-deps` (deps +
compile + finalize via the submodule binary + asyncify + -O2) succeeded
end-to-end; finalize resolved to build-wasm/tools/binaryen-hoist-build/bin.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 11:55:52 +02:00
Gergő Törcsvári
6414036580
chore: add binaryen submodule to git-workflow
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 10:06:34 +02:00
Viktor Vaczi
c1ef489cfa feat(wasm-eh): migrate the WASM build to native wasm exceptions (+ 3D viewer default-on)
Replace the legacy Emscripten JS-exceptions model with native wasm-EH (legacy
encoding) across the whole build, keeping Asyncify coroutines working via a
from-source Binaryen --hoist-cpp-catches pre-pass. Net result: native-EH is the
only build mode, the 3D viewer is on by default, and pcbnew shrinks substantially.

Highlights:
- Binaryen submodule everywhere + --hoist-cpp-catches integration in apply-asyncify;
  post-link Asyncify covers every app wasm (not just standalone test wasm).
- Build deps (incl. OpenCASCADE without OCC_CONVERT_SIGNALS) and all KiCad apps
  with -fwasm-exceptions; emscripten_sleep added to the post-link asyncify-imports.
- libcontext fiber entry wired under native exceptions; while-loop main loop +
  currData shim injected into all wx apps.
- Native-EH collab apply fixed: DEBUG-define the embind TU + match all out-of-CMake
  C++ TUs' ABI flags to the core, fixing the vtable-layout skew / mis-dispatch.
- 3D viewer enabled by default (real raytracer linked, not the stub).
- Retire the EH-spike scaffolding; flip the asyncify-races ablation pins to
  shim-redundancy pins (native-EH stays clean with the legacy shims ablated).
- Fix the asyncify-races quiescence check to not require Asyncify.currData==0:
  under the native-EH per-frame-yield top loop the main stack is asyncify-suspended
  every frame, so currData legitimately churns (a freed-but-not-yet-nulled buffer,
  not a leak). Refresh the pcbnew toolbar screenshot baseline for the new kicad.
- CI: drop the obsolete binaryen_version input/env (the build uses the binaryen
  submodule fork's wasm-opt, not a version download); key the wasm-output cache on
  the binaryen submodule SHA instead.

Bumps the wxwidgets + binaryen submodules to their squashed feature commits.

Validated green: all 7 apps native-EH (real 3D in pcbnew); KiCad e2e 63/63
Firefox + Chromium (3D viewer renders); wx 336; coroutine 34/34 both engines;
asyncify 7/7 both engines.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 09:40:26 +02:00
Gergő Törcsvári
39cb04b127
feat(standalone): demo follow-ups — waitlist, analytics, move-to-local, file tree, load progress
- Waitlist signup form on the home page (between Projects and Tools),
  cross-posting to the landing site's /api/waitlist (CORS + OPTIONS added there,
  gated by WAITLIST_ALLOWED_ORIGINS).
- Version badge gains a pcbjam.com landing-page link (VITE_LANDING_URL).
- Optional Plausible analytics from VITE_PLAUSIBLE_DOMAIN (off by default);
  wired into build-demo + both deploy workflows via vars.PLAUSIBLE_DOMAIN.
- Opening a read-only gallery project auto-"moves to local": it forks into a
  writable browser-local project at the same slug, shadowing the gallery row.
- Project + local-folder views use the same iconed tool launcher (ToolGrid)
  and a navigable directory FileTree instead of a flat file list.
- Boot overlay shows real wasm download progress (Module.instantiateWasm +
  streaming byte counter) and a "taking too long" state after 60s.
- Home Libraries section lists the active libs source (the read-only R2/CDN
  set in the demo) with a name filter; useLibs now reads libsSourceConfig.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPJYx4urFVhdMjEmYfMCMD
2026-06-27 07:59:09 +02:00
Gergő Törcsvári
e8d964d871
feat(demo): local dev-demo runner — R2 libs + example gallery, no backend/partykit
scripts/deploy/dev-demo.mjs (+ `pnpm dev:demo`) is the dev sibling of
build-demo.mjs: it runs the standalone via `vite dev` in the no-backend demo
configuration so a local WASM build can be exercised against the live R2 CDN.

  - libraries: live R2 CDN (VITE_LIBS_SOURCE=cdn, libs/kicad/<lib-tag>, default
    10.0.3) — IDB-cached, read-only; the lazy/fat lib-load path
  - projects: the read-only example gallery (deploy/demo/gallery.json → the same
    "Demo Board") built locally via publish-content.mjs and served same-origin at
    /content/<gallery-tag>/; --content-tag pins the live CDN gallery instead;
    --no-gallery falls back to local-folder + IDB only
  - no partykit (VITE_YJS_PROVIDER=broadcastchannel), no REST backend
    (offline.invalid → local-folder loader), browser-local IDB projects
  - WASM: local fresh build at /wasm (default) or --wasm r2

The generated gallery (public/content symlink, web/standalone/.demo-cdn) is
gitignored, mirroring the public/wasm dev symlink.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 08:16:43 +02:00
Istvan Matejcsok
cdaa96250d fix(wasm): KiCad 10 kiplatform/libgit2 stubs, e2e fixtures, asyncify -g hook
- wasm/kiplatform: add EnableDarkMode, IO::TimestampDir, UI::AllowNetworkFileSystems,
  UI::CancelPendingScroll, SECRETS::DeleteSecret (new KiCad 10 functions).
- wasm/stubs/libgit2_stub.c: ~156 no-op stubs for KiCad 10's restructured git
  backend + git-backed local history + text_eval VCS.
- tests/kicad: bump the demo project dir 9.99 -> 10.0 (KiCad 10 stores projects
  under /home/kicad/documents/kicad/10.0/) — fixes load-pcb + 3d-viewer e2e.
- scripts/common/apply-asyncify.sh: ASYNCIFY_EXTRA_OPTS hook (e.g. -g for
  symbolizable wasm traces).
- kicad submodule -> ef00c2d3 (KiCad 10 link + runtime adaptations).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 10:30:25 +02:00
Istvan Matejcsok
fdfabea6c8 chore: 🔧 fork-cleanup tooling + bump kicad submodule
- kicad → a2c634f2: drop redundant wasm guards, restore upstream
  (modified upstream files 95→86, churn 3180→3104)
- kicad-diff-stats.sh: fix fork-point detection (add missing commit authors
  + an @emergence-engineering.com domain net) so it reports the real
  divergence instead of 0
- build-kicad-target.sh: drop dead -DKICAD_PCM and -DBUILD_GITHUB_PLUGIN
  flags (not options in this KiCad version); keep -DKICAD_SPICE=OFF, which is
  read by wasm/cmake/Findngspice.cmake

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 12:17:46 +02:00
Gergő Törcsvári
6b737f95ed
feat(deploy): publish-libs CI + full clone (C1.3)
publish-libs.ts: --clone <dir> shallow-clones kicad-symbols + kicad-footprints at --lib-tag (full, all libs) and uses them as sources, so CI publishes the whole set with one command.

publish-libs.yml: decoupled workflow_dispatch (mirrors publish-wasm.yml) — lib_tag input (default 10.0.3) + force; clones + publishes to R2, skip-if-exists keyed by the tag. Dependency-free import chain ⇒ no workspace install (just node + npx tsx + git + wrangler).

deploy-demo.yml: LIB_TAG=10.0.3 → build-demo --lib-tag, so the demo serves the full library set from libs/kicad/<LIB_TAG>. Validated locally over .cache: publish then skip-if-exists; both workflows valid YAML.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 12:47:38 +02:00
Gergő Törcsvári
55f8d79a78
feat(deploy): publish-libs (C1.2) — full KiCad set as r2-idb-sync static origins
extract-libs.ts gains extractAllLibs(): enumerate EVERY <Lib>.kicad_symdir / <Lib>.pretty under the source dirs into in-memory self-contained bodies (reuses the curated path's parse + extends resolution). At KiCad 10.0.x symbols already ship the one-symbol-per-file .kicad_symdir layout, so no split needed.

publish-libs.ts (tsx): per lib, build a SyncManifest (sha256 per item) + encodeBundle, write libs/kicad/<libTag>/<lib>/{manifest,bundle} (immutable) + a top manifest.json listing every lib. Keyed by upstream KiCad lib tag, skip-if-exists (HEAD the top manifest; --force overrides) so it is decoupled from the app deploy. Local + r2 drivers via cdn-store.

Validated over web/backend/.cache: 8 libs / 2154 items; Device bundle decodes to 537 bodies, its /manifest matches the bundle, and symbol/R body hash matches its manifest entry (warm-sync diff stays empty). Same wire format cdn-source.test reads, so publish and client are pinned to one format.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 11:04:49 +02:00
Gergő Törcsvári
307a66dd77
feat(standalone): cdn libs source (C1.1) — full KiCad set from R2 static origins
Backendless, read-only LibsSource that serves the full default symbol+footprint set from versioned immutable CDN snapshots. Each lib is an r2-idb-sync static origin opened as a one-layer SyncStack (1 bundle cold, 0 fetches warm, IDB-cached, offline) — the layer descriptor is built locally from lib id + tag, no resolve endpoint in the loop.

Wired via VITE_LIBS_SOURCE=cdn + VITE_LIBS_MANIFEST_URL; build-demo gains --lib-tag (points the demo at libs/kicad/<tag>/manifest.json; omitted keeps offline static). Unit-tested against a fake CDN built with the real encodeBundle/sha256Hex codecs, pinning the publish format.

Next: publish-libs.mjs (C1.2) emits this format over the full set; CI keyed by lib tag (C1.3).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 10:30:18 +02:00
Gergő Törcsvári
ce9ede1e9e
feat(standalone): browser-local (IndexedDB) virtual projects
Loading a folder now imports a COPY into a browser-local IndexedDB project with
its own /p/:slug URL (original disk files untouched): edits persist to IDB across
visits, and the project exports via Download .zip or per-file. Gated by
VITE_LOCAL_PROJECTS=idb (on for the demo; off keeps the File System Access
write-back flow), so it's configurable per deployment.

Modular, swappable project sources behind the shared ProjectSource interface,
each self-describing via a SourceDescriptor whose kind is shown verbatim in the
UI — "Local (this browser)", "Remote · read-only", "Remote · editable" — on the
home page, the project view, and inside the editor, so the user always knows
whether/how Save persists:
  - remote      → REST backend (remote-rw)
  - static      → CDN gallery (remote-ro), saves download
  - local (new) → idbProjectStore, writable IDB store
A composite layers the local store over the configured remote/gallery source,
routing per slug so imported/saved projects and the gallery share one namespace.

New, dependency-free (matching sync-client's raw-IDB ethos):
  - lib/idb-project-store.ts  raw IndexedDB store + create/delete/rename/export
  - lib/zip.ts                store-only ZIP writer (verified via system unzip)
  - lib/import-folder.ts      FSA/webkitdirectory folder → in-memory bytes
  - lib/project-source-shared.ts  source descriptors + deterministic uuid
  - components/SourceChip, components/LocalProjectsSection
Home lists local projects (open/export/rename/delete). Verified: typecheck,
project-source tests, prod build, and IDB key-range project isolation in-browser
(prefix-colliding slugs don't leak).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 08:24:14 +02:00
Gergő Törcsvári
0f53984407
feat(standalone): bottom-right version + source badge
A small fixed bottom-right overlay (VersionBadge) shown app-wide — on the home
page AND inside the loaded editor — surfacing this build's release tag and a
link to the source. The tag links to the exact commit when known (the pcbjam
commit pins the kicad + wxwidgets submodule revisions → our GPLv3
corresponding-source pointer, mirroring site Footer's BUILD_SHA), else the tag's
release page, else the repo root.

build-demo.mjs injects VITE_APP_TAG / VITE_GIT_SHA / VITE_REPO_URL (new --repo
flag, defaults to github.com/emergence-engineering/pcbjam).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 08:01:50 +02:00
Gergő Törcsvári
e94f5b31be ci: cache KiCad WASM build output to skip Docker on unchanged sources
Add an output-cache layer to ci-ubicloud.yml so a build whose inputs are
unchanged (typical when only tests/ or web/ change) skips the in-Docker
compile + the 1-2h host asyncify/wasm-opt chain — the bulk of the ~1h52m
run. On a hit, output/ and the GAL sysroot headers are restored from cache
and the deps restore/seed, build.sh, and sysroot export are all gated on a
miss, so Docker is never started.

Key: kwasm-<os>-bin<ver><opt>-k<kicad-sha>-wx<wx-sha>-sc<hash>-e<epoch>.
The "sc" hash (scripts/deploy/wasm-cache-hash.mjs) folds in just the
build-logic files that shape the wasm bytes (host post-processing,
per-tool compile recipes, scripts/deps, docker/Dockerfile+build.sh) plus
itself; it is content-based, order-independent, and identical on macOS/CI.
*.wasm.debug.wasm (5+ GB, unused by tests) is excluded -> ~0.5 GB entry on
Ubicloud's transparent 30 GB/repo/week cache.

Invalidation:
- bump .ci-cache-epoch in a commit for durable busts (inputs the sc-hash
  can't see: base-image/apt drift, a bad cache);
- [no-cache] or [rebuild-wasm] in the commit message / PR title, or
  workflow_dispatch no_cache=true, for a one-off rebuild (split
  restore/save so the bypass still refreshes the entry).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 14:51:33 +02:00
Gergő Törcsvári
5248f3a464
fix(demo): default WRANGLER_CMD to npx + silence expected R2 not-found probes
The publish scripts shelled bare `wrangler` (ENOENT without a global install)
and the existence probes (registry.json / per-tool meta.json) printed wrangler's
'key does not exist' on a fresh bucket. Default to `npx wrangler@4` and capture
stderr on probes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 13:31:06 +02:00
Gergő Törcsvári
5aae2a0d16
feat(demo): demo.pcbjam.com deploy — versioned WASM CDN + static gallery + tag CI
Cross-origin WASM CDN (cdn.pcbjam.com, R2): per-tool content-addressed,
immutable folders + a per-release runtime manifest (snapshot from registry).
boot.ts loads pthread workers cross-origin via a same-origin blob shim.
Static no-backend project source (demo gallery; Save downloads to local),
api.uploadFileBytes kept and config-gated. demo.pcbjam.com on Cloudflare Pages.
deploy-demo.yml (tag v*) snapshots WASM + content + builds + deploys;
publish-wasm.yml builds on Ubicloud. Design/spec docs live in pcbjam-private.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 11:39:09 +02:00
Istvan Matejcsok
0195ff0941 feat: enable WASM 3D viewer build — FFP link stubs, drop glemu shim
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 09:21:37 +02:00
Istvan Matejcsok
68c711e633 feat: enable the 3D viewer in the WASM port (WIP)
Builds KiCad's 3D viewer for the browser (opt in with BUILD_3D_VIEWER=ON) and
wires up the legacy-GL emulation it needs.

- build-kicad-target.sh / docker/build.sh: BUILD_3D_VIEWER →
  -DKICAD_BUILD_3D_VIEWER_WASM=ON + -sLEGACY_GL_EMULATION + the GL js-library.
- wasm/shims/gl_immediate_shim.js: display-list emulation (record/replay board
  layers), fixed-function + GLU stubs, throw-guards for unsupported pnames, and
  per-context GLImmediate init for the viewer's SECOND WebGL context.
- tests/kicad/3d-viewer.spec.ts + utils/pcbnew-ready.ts: open View → 3D Viewer.
- bumps the kicad submodule (EMSCRIPTEN-guarded GL changes).

WIP: the viewer window + UI render and geometry draws (glError=0x0), but the
board is not yet visibly rendered — GLImmediate's FFP shader program isn't
linked on the viewer's 2nd WebGL context. Full status + next steps in
features/feat/add-3d-view/notes.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 09:13:41 +02:00
Gergő Törcsvári
698f6d7cb7
feat(libs): standalone .lib→.kicad_sym wasm converter (sym_convert)
Headless node-WASM CLI that converts legacy symbol libraries to .kicad_sym via
SCH_IO_MGR::ConvertLibrary. Adds wasm/cli/sym_convert_main.cpp + sym_convert_pre.js
(in-memory localStorage + wxConfig JS-hook shim), registers sym_convert as a
docker/build.sh app (own kicad-sym_convert tree, real wasm-opt/finalize in-container,
skip host postprocess), and bumps the kicad pointer for the matching target.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 12:21:15 +02:00
Gergő Törcsvári
19fce866c1
feat: libs 0009-S — footprint write-path spike DONE + verified (footprint editor as project-scoped WASM tool + PCB_IO_PCBJAM_FP plugin + bridge kind 4th-arg; New Footprint→FootprintSave fires through bridge on main thread, fork-native body v20251028, no wedge — Firefox e2e green); bump kicad + pcbjam-shared 2026-06-16 16:28:21 +02:00
Gergő Törcsvári
df4de65ba2
feat: libs 0004-A — symbol write-path spike (provider save op, writable lib + MEMFS placeholder files, rw mount); fix symbol_editor embind link; e2e write-bridge test; bump kicad
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 16:28:20 +02:00
Istvan Matejcsok
5df7f40b3c ci: 🎡 -o1 & test workers 2026-06-16 14:46:18 +02:00
Istvan Matejcsok
9733400734 chore: 🤖 add new worktree setup script 2026-06-16 14:46:18 +02:00
Viktor Vaczi
68430fa87c feat(wasm-dom): tests for context menus + scrollbars; fix clean wx build
- Standalone wx test apps (contextmenu, scrollbar) + e2e specs proving the
  wxScrollBar control, the wxScrolledWindow gutter, and DoPopupMenu.
- KiCad in-app specs: pl_editor + pcbnew canvas right-click context menus and
  scrolled-panel scrollbar gutters, with baseline screenshots.
- build-wx-wasm.sh: serial -j1 fallback when the parallel build trips the
  intermittent clean-build race (a generated header read mid-rewrite by
  concurrent compiles; never happens at -j1). Recovery resumes from built
  objects; a genuine error still fails the serial pass.
- Bump wxwidgets submodule (context menus + draggable scrollbars).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 23:15:49 +02:00
Viktor Vaczi
8dbca8a48b ci: fix stale build-wxuniversal-wasm.sh reference after rename
Commit 7dc1bf2 renamed scripts/build-wxuniversal-wasm.sh to
build-wx-wasm.sh but missed the CI workflow, so the "Build wxWidgets
(wxUniversal WASM)" step ran a script that no longer exists. Update the
run command in ci-ubicloud.yml plus the two stale comments referencing
the old name (ci-ubicloud.yml:43, scripts/common/stages.sh:7).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 19:50:23 +02:00
Viktor Vaczi
7dc1bf27e0 refactor: collapse dual-mode plumbing — the DOM port is the only WASM build
The canvas (wxUniversal) mode is gone (wxwidgets submodule); remove every
piece of side-by-side plumbing so there is exactly one build and one test
flow:

- scripts/build-wxuniversal-wasm.sh -> scripts/build-wx-wasm.sh; no
  --dom/--enable-universal; builds into build-wasm/wxwidgets
- build-wasm-test.sh: no DOM_BUILD / apps-dom rsync mirror / PORT=dom;
  apps build straight into tests/apps (Makefile.wasm PORT conditionals
  collapsed; wx.js + wx-dom.js always pre-js)
- docker/build.sh, build-kicad-target.sh, env.sh: WX_PORT / -dom /
  -universal suffixes removed; kicad builds to kicad-<app>, outputs to
  output/; wx.js/wx-dom.js copied from the real source path
  (/workspace/wxwidgets/build/wasm — the old build-wasm path never
  existed and silently failed)
- setup-kicad-wasm.sh: single target dir; the perl wx-dom.js injection is
  gone — the 7 checked-in kicad pages now reference wx-dom.js directly
- playwright configs serve apps/; fixtures drop the test-results/dom and
  logs/wxwidgets/dom namespacing; boot.spec asserts wxDomPort
  unconditionally; pcbnew.spec uses one reference image;
  appearance.spec assertions unconditional
- compare/update-baseline-screenshots.sh: --port removed
- tests/gal-regression/wasm/Makefile: links build-wasm/wxwidgets and
  carries wx-dom.js as a second pre-js — the gal-webgl suite (30 specs)
  now actually builds and runs here (it needed host-side boost+glm via
  scripts/deps; the bundle had been missing, timing the whole spec out)
- tests: clickCanvas() dispatches via page.mouse (DOM widgets
  legitimately cover the canvas; locator actionability refused the
  click); the comprehensive spec drives wxChoice through its native
  <select> (browser-owned popup cannot be coordinate-clicked)
- docs: README/CLAUDE.md/build.md script names and dirs;
  features/wx-dom-port README reframed (DOM is THE port), visual-notes
  bugs 26-28; FindwxWidgets.cmake config label drops 'wasmuniv'
- wxwidgets submodule -> 9dbacc9448 (DOM-only port, fork diff shrunk)

Gate: full wx e2e suite 292 passed / 1 skipped / 0 failed — first run
ever with the gal-webgl specs green (28 scenarios + load + sequential).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:19:48 +02:00
Viktor Vaczi
a275ca735f fix(wasm-dom): serve:dom script; keep kicad bundles out of the apps-dom mirror
The wx test-app mirror (build-wasm-test.sh --dom, rsync --delete) wiped
the kicad bundles setup-kicad-wasm.sh syncs into apps-dom/kicad —
apps/kicad only carries the checked-in HTML. Exclude kicad/ (that dir is
owned by the setup script). Add 'npm run serve:dom' for browsing the
DOM-port apps; plain 'npm run serve' serves the canvas dir, which has no
kicad bundles in this worktree (canvas KiCad was never built here).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:18:48 +02:00
Viktor Vaczi
cc7faadbc1 chore: drop redundant lexer pre-build — upstream f936eb1 fixed it independently
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:18:48 +02:00
Viktor Vaczi
dba03b237f feat(wasm-dom): Phase 7 — KiCad pcbnew builds and runs on the DOM port
Pipeline: lexer generation (pcbcommon) ordered before the embind
pre-compile — a fresh-build-tree chicken-and-egg any new volume would
hit (generated pcb_lexer.h was included via -I${KICAD_BUILD}/common
before anything generated it). DOM flavor outputs flow through the
host-mounted output/dom/. pcbnew e2e on DOM: editor loads fully (layer
manager, toolbars, WebGL viewport, registry metrics), draw-lines test
passes; the header pixel-reference assertion fails as expected (native
DOM styling vs univ pixels) — port-aware references pending.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:18:05 +02:00
Viktor Vaczi
74c392f88f feat(wasm-dom): Phase 7 plumbing — KiCad DOM-flavor build + test pipeline
WX_PORT=dom flows host → docker container → build scripts:
build-kicad-target.sh selects wxwidgets-dom + kicad-<app>-dom trees;
docker/build.sh passes the env through, copies outputs to output-dom/
incl. wx-dom.js, and post-processes there; build-wxuniversal-wasm.sh
honors WX_PORT and accepts --no-clean again (the kicad pipeline calls it
with that flag — Phase 1's stricter parser had broken the canvas docker
path). setup-kicad-wasm.sh syncs DOM bundles into tests/apps-dom/kicad
and injects the wx-dom.js script tag into the kicad pages;
playwright-kicad.config.ts serves apps-dom under WX_PORT=dom.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:18:05 +02:00
Viktor Vaczi
e36854bab6 feat(wasm-dom): Phase 5 baselines + visual notes (menus/toolbars/tooltips)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:17:10 +02:00
Viktor Vaczi
1c8fe361d8 feat(wasm-dom): Phase 3 B2 baselines; parallel test-app links
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:17:09 +02:00
Viktor Vaczi
249010a582 feat(wasm-dom): dual-mode build scaffold — native DOM port builds, links and boots alongside canvas
Build/test plumbing for the second wxWidgets mode: --dom on
build-wxuniversal-wasm.sh (build-wasm/wxwidgets-dom, no universal,
--disable-tooltips, libwx_wasmu_* stubs) and build-wasm-test.sh
(mirrors sources into tests/apps-dom, PORT=dom). Build script now
reconfigures when Makefile.in or autoconf_inc.m4 regenerate (bakefile
chain). Harness: WX_PORT=dom serves apps-dom, namespaces screenshots
into test-results/dom and logs into logs/wxwidgets/dom; compare/update
screenshot scripts take --port dom with baseline-screenshots-dom.
New port-agnostic boot.spec.ts (green on both ports).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 20:17:09 +02:00
Viktor Vaczi
14ca16cbd3 test(asyncify): red-green race harness + ablation flags, unwind-catch shim, spec tightening, decisions docs
The asyncify single-slot work, executed red-green (full ledger:
docs/features/asyncify-arbiter/redgreen.md; decisions record:
docs/features/async/07-decisions-and-outcome.md):

- tests/apps/standalone/asyncify-races/ + tests/asyncify/ + dedicated
  playwright config: 8 scenarios reproducing the KiCad asyncify failure
  family with the kicad-faithful startup topology (pre-park fiber swap →
  park throw through the live trampoline). Built in 3 variants; the
  SHIM_DISABLE_TRAMPOLINE_HEAL / SHIM_DISABLE_HANDLESLEEP ablation builds
  keep the historical hang and index-out-of-bounds crash reproducible
  forever (mutation-style pins for the existing shims).
- scripts/common/shims/handlesleep.js: catch the "unwind" park sentinel
  in the wakeUp path — when main's last pre-park suspension was a sleep,
  the main-loop park throw escaped through that sleep's promise reaction
  as an uncaught rejection (the calculator/gerbview console errors).
- scripts/common/inject-dyncall-shims.sh: SHIM_DISABLE_* ablation knobs.
- Spec tightening (the acceptance bar): 'uncaught exception: unwind'
  tolerance DELETED from pcbnew/eeschema specs; load-pcb gained a hard
  clean-console gate over 5 asyncify corruption signatures.
- wxwidgets pointer bump: modal LIFO resolvers, pump resolve-on-error,
  sync clipboard IsSupported (014f67e6c1).

Final state: asyncify suite 7/7, wx e2e 291/292 (1 skip), KiCad e2e 40
passed / 2 skipped with ZERO corruption signatures in any log across all
six apps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 17:02:59 +02:00
Gergő Törcsvári
0665b30762
chore(skills): include pcbjam-shared in the git-feature workflow
The pcbjam repo's own git-feature skills + scripts/git-workflow predated the
web/pcbjam-shared submodule and only covered root+kicad+wxwidgets. Add the MIT
contract repo as a fourth member everywhere:

- repos.sh: pcbjam_shared key (path web/pcbjam-shared, main `main`) + layout doc.
- assert-clean.sh / for-each-repo.sh: count-agnostic wording.
- git-feature-start/commit/sync/finish SKILL.md: pcbjam-shared in the branch /
  commit (kicad, wxwidgets, pcbjam-shared, then root) / rebase (root first) /
  finish orders; root pointer-bump staging + diff now include web/pcbjam-shared,
  and the root stage step excludes it like the other submodule trees.

Mirrors the root pcbjam-private skills, which already covered shared.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 14:05:40 +02:00
Istvan Matejcsok
380206dcfc fix(build): host postprocess silently no-ops when emsdk tools are stubbed
build-kicad-target.sh swaps emsdk's wasm-opt/wasm-emscripten-finalize for
no-op stubs (real binaries preserved as *.real) so emcc skips in-link
asyncify — intended for the CONTAINER emsdk, but a host-mode run left the
host tools/emsdk stubbed (since Jun 9). The stub fakes --version and exits 0,
so every local build's host-side finalize/asyncify/-O2 "succeeded" while
doing nothing: output wasm shipped non-asyncified and aborts at boot with
"asyncify_stop_unwind is not a function" (and stayed 122M vs the correct
187M). get-wasm-opt.sh and apply-finalize.sh now prefer the *.real binary
whenever it exists, making the resolution immune to a stubbed emsdk.

Found while validating the CI e2e fix: pcbnew.wasm built this morning could
not boot in any browser. Local artifacts built since Jun 9 may need their
postprocess re-run (apply-finalize.sh + apply-asyncify.sh).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 13:25:02 +02:00