AI_Assistant/pcbjam
Watch
1
0
Fork
You've already forked pcbjam
 0
Code Issues Pull requests Projects Releases 7 Packages Wiki Activity Actions

docs: ✏️ cleanup and organize docs

Browse source
This commit is contained in:
Istvan Matejcsok 2026-06-05 12:15:54 +02:00
commit 8e413f89ec
29 changed files with 85 additions and 3340 deletions
Download patch file Download diff file Expand all files Collapse all files

154
docs/features/archive/ipc-api/ipc-api.md Normal file
View file

@ -0,0 +1,154 @@
# KiCad IPC API - WASM Fork Changes
> **ARCHIVED — open cleanup TODO, not yet actioned.** Describes `#ifdef KICAD_IPC_API` guards added to ~18 KiCad source files that could be reverted to reduce fork diff. Archiving this doc does **not** perform the revert — that remains a separate task.
## Overview
We added `#ifdef KICAD_IPC_API` guards to 18 KiCad source files. These guards wrap:
- `#include` statements for protobuf/API headers
- `Serialize()` and `Deserialize()` methods
**Key insight:** Since our build uses `KICAD_IPC_API=ON` (line 251 of `scripts/kicad/build-pcbnew.sh`), these guards don't actually disable any code - everything compiles. The guards can be safely reverted to reduce fork divergence.
## Modified Files
| File | What We Guarded |
|------|-----------------|
| `common/eda_shape.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `common/eda_text.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `common/netclass.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `include/api/api_utils.h` | Entire namespace content (utility functions) |
| `pcbnew/api/api_pcb_utils.h` | Entire file content |
| `pcbnew/board_connected_item.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `pcbnew/board_stackup_manager/board_stackup.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `pcbnew/footprint.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `pcbnew/pad.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `pcbnew/padstack.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `pcbnew/pcb_dimension.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `pcbnew/pcb_field.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `pcbnew/pcb_group.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `pcbnew/pcb_shape.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `pcbnew/pcb_text.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `pcbnew/pcb_textbox.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `pcbnew/pcb_track.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
| `pcbnew/zone.cpp` | `#include` for API headers, `Serialize()`, `Deserialize()` |
## Example of Our Changes
**Before (upstream):**
```cpp
#include <api/api_enums.h>
#include <api/api_utils.h>
#include <api/board/board_types.pb.h>
void PAD::Serialize( google::protobuf::Any &aContainer ) const
{
// ... serialization code
}
```
**After (our fork):**
```cpp
#ifdef KICAD_IPC_API
#include <api/api_enums.h>
#include <api/api_utils.h>
#include <api/board/board_types.pb.h>
#endif
#ifdef KICAD_IPC_API
void PAD::Serialize( google::protobuf::Any &aContainer ) const
{
// ... serialization code
}
#endif
```
## Why We Added These Guards
Originally added to allow building with `KICAD_IPC_API=OFF`, which would:
- Skip protobuf dependency
- Exclude serialization methods that depend on protobuf types
However, our current build has `KICAD_IPC_API=ON`, making these guards unnecessary.
## How to Revert
To revert these files to upstream (base commit `4bfed3f174`):
```bash
cd kicad
git checkout 4bfed3f174 -- \
common/eda_shape.cpp \
common/eda_text.cpp \
common/netclass.cpp \
include/api/api_utils.h \
pcbnew/api/api_pcb_utils.h \
pcbnew/board_connected_item.cpp \
pcbnew/board_stackup_manager/board_stackup.cpp \
pcbnew/footprint.cpp \
pcbnew/pad.cpp \
pcbnew/padstack.cpp \
pcbnew/pcb_dimension.cpp \
pcbnew/pcb_field.cpp \
pcbnew/pcb_group.cpp \
pcbnew/pcb_shape.cpp \
pcbnew/pcb_text.cpp \
pcbnew/pcb_textbox.cpp \
pcbnew/pcb_track.cpp \
pcbnew/zone.cpp
```
Then rebuild:
```bash
./docker/build.sh --clean-kicad
```
## Note: toolbars_pcb_editor.cpp
This file has `#ifdef KICAD_SCRIPTING` guards (not IPC API guards):
```cpp
#ifdef KICAD_SCRIPTING
#include "../scripting/python_scripting.h"
#endif
...
#ifdef KICAD_SCRIPTING
bool scriptingAvailable = SCRIPTING::IsWxAvailable();
#else
bool scriptingAvailable = false;
#endif
```
**Cannot be fully reverted** because:
- `KICAD_SCRIPTING=OFF` for WASM builds (CMakeLists.txt lines 135-138)
- Without guards, build fails (python_scripting.h not built, SCRIPTING class doesn't exist)
This is a separate concern from the IPC API and is documented here for completeness.
## Impact of Reversion
- **Fork diff reduction:** 18 fewer modified files
- **Build behavior:** No change (KICAD_IPC_API=ON, code compiles identically)
- **Functionality:** No change (serialization methods still available)
## Related: IPC API Architecture
KiCad's IPC API uses:
- **Protobuf messages** for request/response serialization
- **NNG sockets** for transport (stubbed in WASM - see `wasm/stubs/nng_stub.c`)
- **API_HANDLER_PCB** class for handling requests (instantiated in PCB_EDIT_FRAME)
Our stub at `wasm/stubs/api_plugin_stub.cpp` provides no-op implementations for:
- `KICAD_API_SERVER::Start()`, `Stop()`, `Running()`
- `KICAD_API_SERVER::RegisterHandler()`, `DeregisterHandler()`
## Future: JavaScript Bridge
To expose the IPC API to JavaScript (optional enhancement):
1. Modify `api_plugin_stub.cpp` to store registered handlers
2. Create Embind bindings in `wasm/bindings/api_bridge.cpp`
3. Expose `KiCadApi_HandleRequest(protobufData)` function
4. Use `protobuf.js` on JS side for message creation/parsing
This would enable JS automation without modifying any KiCad source files.

264
docs/features/archive/webgl/0001-opengl-gal-webgl-strategy.md Normal file
View file

@ -0,0 +1,264 @@
# OpenGL, GAL, and WebGL Strategy for KiCad WASM
> **ARCHIVED / HISTORICAL** — the WebGL-GAL backend described here was implemented (see `kicad/common/gal/webgl/`, ~22.5k lines). Live test docs: [`tests/gal-regression/README.md`](../../../../tests/gal-regression/README.md). Kept for design rationale.
## Summary
This document analyzes KiCad's graphics architecture and evaluates strategies for WebGL rendering in the WASM build.
## Current Implementation: Emscripten LEGACY_GL_EMULATION
We currently use Emscripten's legacy OpenGL emulation:
```bash
-sLEGACY_GL_EMULATION -sMAX_WEBGL_VERSION=2
```
**How it works:**
```
KiCad/wxWidgets C++ (glBegin, glVertex, glColor, etc.)
│
▼
gl_immediate_shim.js (our fix for color-per-vertex bug)
│
▼
Emscripten GLImmediate (translates legacy GL → shaders + VBOs)
│
▼
WebGL 2.0 (ES 3.0 compatible, shader-based)
```
**Key components:**
| Component | File | Purpose |
|-----------|------|---------|
| wxGLCanvas | `wxwidgets/src/wasm/glcanvas.cpp` | Creates WebGL contexts on dedicated HTML5 canvases |
| GL Shim | `wasm/shims/gl_immediate_shim.js` | Fixes color-per-vertex bug in GLImmediate |
| GLU Tesselator | `wasm/stubs/glu_wasm_impl.cpp` | Polygon triangulation using earcut (GLU not available in WebGL) |
| GLEW Stub | `wasm/cmake/FindGLEW.cmake` | Stubs out GLEW (not needed with Emscripten) |
| kiglew.h | `kicad/include/gal/opengl/kiglew.h` | Maps double→float, stubs unsupported functions |
---
## GLU Status
**KiCad uses GLU** for polygon tesselation (`gluTess*` functions for zones, complex shapes).
**We already solved this** with `/wasm/stubs/glu_wasm_impl.cpp` which implements the GLU tesselator API using KiCad's earcut algorithm. This works with both LEGACY_GL_EMULATION and would work with pure ES3.
GLU is NOT a blocker.
---
## Where KiCad Uses Legacy OpenGL
Two separate systems use legacy OpenGL:
### 1. OPENGL_GAL (2D Rendering)
Used for schematic and PCB editors.
**Files:**
- `kicad/common/gal/opengl/opengl_gal.cpp` - Main implementation
- `kicad/common/gal/opengl/gpu_manager.cpp` - VBO management
- `kicad/common/gal/opengl/opengl_compositor.cpp` - Framebuffer compositing
- `kicad/common/gal/opengl/antialiasing.cpp` - AA effects
**Legacy GL usage in opengl_gal.cpp:**
- Lines 1557-1605: Bitmap rendering with `glBegin(GL_QUADS)`
- Lines 2690-2755: Cursor drawing with `glBegin(GL_LINES)`
- Lines 566-629, 1558-1602, 2704-2751: Matrix stack operations (`glPushMatrix`, `glPopMatrix`, `glMatrixMode`)
### 2. 3D Viewer (3D Rendering)
Separate from GAL, used for 3D board visualization.
**Files using legacy GL:**
- `kicad/3d-viewer/3d_rendering/opengl/render_3d_opengl.cpp`
- `kicad/3d-viewer/3d_rendering/opengl/opengl_utils.cpp`
- `kicad/3d-viewer/3d_rendering/opengl/layer_triangles.cpp`
- `kicad/3d-viewer/3d_rendering/opengl/3d_spheres_gizmo.cpp`
- `kicad/3d-viewer/3d_rendering/opengl/3d_model.cpp`
- `kicad/3d-viewer/3d_model_viewer/eda_3d_model_viewer.cpp`
- `kicad/3d-viewer/3d_canvas/eda_3d_canvas_pivot.cpp`
---
## If We Cover GAL, Is That Enough?
**For 2D editing (schematic + PCB): YES**
The entire 2D rendering pipeline goes through GAL. A working GAL backend enables full schematic and PCB editing.
**For 3D viewer: NO**
The 3D viewer is a separate rendering system. It could be disabled/stubbed initially and added later.
---
## The GAL API
GAL (Graphics Abstraction Layer) is a clean 2D drawing interface defined in `kicad/include/gal/graphics_abstraction_layer.h`.
**NO raw OpenGL is exposed.** The API consists of ~40 virtual methods:
```cpp
// Drawing primitives
virtual void DrawLine(const VECTOR2D& start, const VECTOR2D& end);
virtual void DrawSegment(const VECTOR2D& start, const VECTOR2D& end, double width);
virtual void DrawPolyline(const std::vector<VECTOR2D>& points);
virtual void DrawCircle(const VECTOR2D& center, double radius);
virtual void DrawArc(const VECTOR2D& center, double radius, const EDA_ANGLE& start, const EDA_ANGLE& angle);
virtual void DrawRectangle(const VECTOR2D& start, const VECTOR2D& end);
virtual void DrawPolygon(const SHAPE_POLY_SET& polySet);
virtual void DrawBitmap(const BITMAP_BASE& bitmap, double alpha);
virtual void DrawGlyph(const KIFONT::GLYPH& glyph);
// Attributes
virtual void SetFillColor(const COLOR4D& color);
virtual void SetStrokeColor(const COLOR4D& color);
virtual void SetLineWidth(float width);
virtual void SetIsFill(bool enabled);
virtual void SetIsStroke(bool enabled);
virtual void SetLayerDepth(double depth);
// Transforms (matrix stack)
virtual void Save();
virtual void Restore();
virtual void Transform(const MATRIX3x3D& matrix);
virtual void Rotate(double angle);
virtual void Translate(const VECTOR2D& translation);
virtual void Scale(const VECTOR2D& scale);
// Rendering control
virtual void BeginDrawing();
virtual void EndDrawing();
virtual void SetTarget(RENDER_TARGET target);
virtual void ClearTarget(RENDER_TARGET target);
virtual void ClearScreen();
// Grid and cursor
virtual void DrawGrid();
virtual void DrawCursor(const VECTOR2D& position);
```
**Existing GAL implementations:**
| Class | File | Purpose |
|-------|------|---------|
| `OPENGL_GAL` | `kicad/include/gal/opengl/opengl_gal.h` | OpenGL rendering (uses legacy GL internally) |
| `CAIRO_GAL` | `kicad/include/gal/cairo/cairo_gal.h` | Cairo software rendering |
| `CALLBACK_GAL` | `kicad/include/callback_gal.h` | Hit testing, no actual rendering |
---
## What Breaks with FULL_ES3 (No Legacy Emulation)
If we removed `LEGACY_GL_EMULATION`:
| Feature | Used In | Status in FULL_ES3 |
|---------|---------|-------------------|
| `gluTess*` (tesselation) | Polygon rendering | **Our stub works** |
| `glBegin/glEnd` | Bitmap quads, cursor | Not available |
| `glVertex/glColor` | Immediate mode drawing | Not available |
| `glPushMatrix/glPopMatrix` | Transformations | Not available |
| `glMatrixMode` | Matrix switching | Not available |
| `GL_QUADS` | Bitmap rendering | Not available (only triangles) |
| `glEnableClientState` | Legacy vertex arrays | Not available |
---
## Strategy Options
### Option 1: Keep LEGACY_GL_EMULATION (Current)
**Pros:**
- Works now
- Minimal code changes to KiCad
- All immediate mode functions available
**Cons:**
- ~200KB binary overhead from GLImmediate
- Runtime overhead from emulation
- Color-per-vertex bug requires our shim
- Some edge cases may not work
**Build flags:**
```bash
-sLEGACY_GL_EMULATION -sMAX_WEBGL_VERSION=2
```
### Option 2: Create WEBGL_GAL (New Backend)
Create a new GAL implementation that uses pure WebGL 2.0 (ES 3.0) without legacy emulation.
**Pros:**
- No emulation overhead
- Smaller binary
- Cleaner, more maintainable
- Better performance
**Cons:**
- Development effort (~2-3 weeks)
- Need to maintain separate backend
**Estimated scope:**
```
New class: WEBGL_GAL : public GAL
Files needed:
├── webgl_gal.h (~300 lines)
├── webgl_gal.cpp (~2500 lines)
├── webgl_shaders.cpp (~500 lines)
└── webgl_compositor.cpp (~500 lines)
Total: ~3500-4000 lines
```
**Why it's feasible:**
- GAL API is clean - no raw GL leaks through
- CAIRO_GAL proves it works (~2500 lines)
- All drawing is 2D with simple primitives
- KiCad's shaders already exist and work in WebGL
### Option 3: Cairo-only (Software Rendering)
Use CAIRO_GAL exclusively, render to HTML5 2D canvas.
**Pros:**
- Already exists
- No WebGL needed
- Works everywhere
**Cons:**
- Slow (CPU-only)
- May struggle with complex boards
- No hardware acceleration
---
## Recommendation
**Short-term:** Keep LEGACY_GL_EMULATION - it works and allows rapid development.
**Medium-term:** Create WEBGL_GAL - cleaner architecture, better performance, removes emulation hacks.
**For 3D viewer:** Disable initially, add later as separate effort.
---
## Key Files Reference
| Purpose | File |
|---------|------|
| Build flags | `scripts/common/env.sh` |
| wxGLCanvas WASM | `wxwidgets/src/wasm/glcanvas.cpp` |
| GL immediate shim | `wasm/shims/gl_immediate_shim.js` |
| GLU tesselator | `wasm/stubs/glu_wasm_impl.cpp` |
| KiCad GL compat | `kicad/include/gal/opengl/kiglew.h` |
| KiCad shaders | `kicad/common/gal/shaders/kicad_*.glsl` |
| GAL base class | `kicad/include/gal/graphics_abstraction_layer.h` |
| OPENGL_GAL | `kicad/include/gal/opengl/opengl_gal.h` |
| CAIRO_GAL | `kicad/include/gal/cairo/cairo_gal.h` |
| GL documentation | `tests/GL_README.md` |

289
docs/features/archive/webgl/0002-gal-native-test-architecture.md Normal file
View file

@ -0,0 +1,289 @@
# GAL Native Test Harness Architecture
> **ARCHIVED / HISTORICAL** — the WebGL-GAL backend described here was implemented (see `kicad/common/gal/webgl/`). Live test docs: [`tests/gal-regression/README.md`](../../../../tests/gal-regression/README.md). Kept for design rationale.
## Overview
The GAL native test harness is a standalone macOS application that compiles KiCad's actual `OPENGL_GAL` rendering engine against system wxWidgets. It generates baseline PNG screenshots for visual regression testing of WebGL rendering in the WASM build.
**Purpose**: Compare native OpenGL rendering (ground truth) against WebGL rendering in the browser to detect visual regressions.
---
## Architecture
```
┌─────────────────────────────────────────────────────────────────────┐
│ GAL Native Test │
├─────────────────────────────────────────────────────────────────────┤
│ gal_native_test.cpp (wxApp + wxFrame) │
│ └─ Creates OPENGL_GAL on wxGLCanvas │
│ └─ Runs 11 test scenarios │
│ └─ Captures FBO → PNG for each │
├─────────────────────────────────────────────────────────────────────┤
│ kicad_stubs.cpp │ gal_test_accessor.cpp │
│ • PGM_BASE singleton │ • Template accessor for │
│ • ADVANCED_CFG │ private OPENGL_GAL members │
│ • KIFONT stubs │ • FBO reading │
│ • Observable stubs │ │
├─────────────────────────────────────────────────────────────────────┤
│ KiCad OPENGL_GAL (from submodule) │
│ kicad/common/gal/opengl/*.cpp (18 source files) │
├─────────────────────────────────────────────────────────────────────┤
│ System Dependencies │
│ wxWidgets (via homebrew) │ GLEW │ OpenGL │
└─────────────────────────────────────────────────────────────────────┘
```
---
## GAL Code Source
The test harness compiles **KiCad's actual OPENGL_GAL** from the kicad submodule:
### Source Files (`/kicad/common/gal/opengl/`)
| File | Purpose |
|------|---------|
| `opengl_gal.cpp` | Main GAL implementation - drawing primitives |
| `opengl_compositor.cpp` | FBO management, layer compositing |
| `antialiasing.cpp` | SMAA antialiasing implementation |
| `vertex_manager.cpp` | Vertex buffer accumulation |
| `vertex_item.cpp` | Individual vertex items |
| `vertex_container.cpp` | Vertex storage base class |
| `cached_container.cpp` | Cached geometry container |
| `cached_container_gpu.cpp` | GPU-resident cached geometry |
| `cached_container_ram.cpp` | RAM-backed cached geometry |
| `noncached_container.cpp` | Per-frame geometry |
| `gpu_manager.cpp` | GPU buffer management |
| `shader.cpp` | GLSL shader compilation/linking |
| `utils.cpp` | OpenGL utility functions |
| `gl_resources.cpp` | OpenGL resource management |
| `hidpi_gl_canvas.cpp` | HiDPI-aware GL canvas |
| `graphics_abstraction_layer.cpp` | GAL base class |
| `color4d.cpp` | Color handling |
| `gal_display_options.cpp` | Display options |
### Build Configuration (`CMakeLists.txt`)
```cmake
# Links against system wxWidgets
find_program(WX_CONFIG_EXECUTABLE wx-config
HINTS /opt/homebrew/bin /usr/local/bin)
# Includes KiCad headers from submodule
target_include_directories(gal_native_test PRIVATE
${KICAD_SOURCE}/include
${KICAD_SOURCE}/include/gal
${KICAD_SOURCE}/include/gal/opengl
${KICAD_SOURCE}/libs/kimath/include
${KICAD_SOURCE}/libs/core/include
)
```
---
## Generated Files
### Location
`/tests/gal-regression/native/generated/`
### Purpose
GLSL shaders must be embedded as strings at runtime. `generate_shaders.py` converts shader source files to C++ hex arrays.
### Generator Script
`generate_shaders.py` reads from `/kicad/common/gal/shaders/` and creates:
| Shader | Generated Files | Purpose |
|--------|-----------------|---------|
| `kicad.frag` | `glsl_kicad_frag.cpp/h` | Fragment shader (coloring) |
| `kicad.vert` | `glsl_kicad_vert.cpp/h` | Vertex shader (transforms) |
| `smaa_base.glsl` | `glsl_smaa_base.cpp/h` | SMAA common structures |
| `smaa_pass_1_*.glsl` | 3 file pairs | SMAA edge detection |
| `smaa_pass_2_*.glsl` | 2 file pairs | SMAA blending weights |
| `smaa_pass_3_*.glsl` | 2 file pairs | SMAA neighborhood blending |
### Generated Code Structure
```cpp
// generated/glsl_kicad_frag.cpp
namespace KIGFX {
namespace BUILTIN_SHADERS {
static unsigned char glsl_kicad_frag_bytes[] = { 0x2f, 0x2a, ... };
std::string glsl_kicad_frag = std::string(
reinterpret_cast<char const*>(glsl_kicad_frag_bytes), 4233);
}}
```
---
## Test Harness Components
### `gal_native_test.cpp`
Main test driver with wxApp/wxFrame:
1. Creates `OPENGL_GAL` on a wxGLCanvas
2. Configures coordinate system for 1:1 world-to-screen mapping
3. Iterates through test scenarios
4. Captures FBO contents as PNG screenshots
Key configuration:
```cpp
// Critical: Set 1:1 world-to-screen mapping
// GAL default is for PCB nanometers (3.937e-8), which would
// compress pixel coordinates (0-800) to ~0.003 screen pixels
m_gal->SetWorldUnitLength(1.0 / ADVANCED_CFG::GetCfg().m_ScreenDPI);
```
### `kicad_stubs.cpp`
Minimal implementations for KiCad symbols not included in GAL:
- `PGM_BASE` singleton with `GL_CONTEXT_MANAGER`
- `ADVANCED_CFG` with `m_ScreenDPI = 91`
- `KIFONT` stubs (returns nullptr for fonts)
- `OBSERVABLE_BASE` observer pattern stubs
- UI dialog stubs (`DisplayError`, etc.)
### `gal_test_accessor.cpp`
Uses C++ template technique to access private members:
```cpp
// Access private OPENGL_GAL members without modifying headers
template<typename T, T> struct steal_impl;
template<typename T, T ptr>
struct steal_impl {
friend T get(steal_impl*) { return ptr; }
};
```
Provides:
- `GetCompositorMainBufferTexture()` - For screenshot reading
- `GetCompositorMainFBO()` - FBO ID access
- `ReadCompositorFBOPixels()` - Direct pixel readback
### `gal_test_scenarios.cpp`
11 rendering test scenarios using the GAL API.
---
## Test Coverage Analysis
### Current Scenarios
| # | Name | GAL Features Tested |
|---|------|-------------------|
| 0 | basic-lines | `DrawLine`, `SetStrokeColor`, `SetLineWidth` |
| 1 | line-widths | `DrawLine` with varying widths (0.5 to 12.0) |
| 2 | circles | `DrawCircle` (filled and stroked) |
| 3 | arcs | `DrawArc` |
| 4 | rectangles | `DrawRectangle` (filled and stroked) |
| 5 | polygons | `DrawPolygon`, `DrawPolyline` |
| 6 | alpha-blending | `SetFillColor` with alpha transparency |
| 7 | transforms | `Save`, `Restore`, `Rotate`, `Translate`, `Scale` |
| 8 | grid-cursor | `DrawGrid`, `DrawCursor` |
| 9 | segments | `DrawSegment` |
| 10 | complex-scene | `SetLayerDepth` (z-ordering) |
### Coverage: ~19% of GAL API (14 of 73 methods)
### Missing Coverage (Priority Order)
**High Priority** (Critical for KiCad functionality):
1. `DrawBitmap()` - Image/icon rendering
2. `DrawCurve()` - Bezier curves
3. `DrawGlyph()` / `BitmapText()` - Text rendering
4. `DrawSegmentChain()` - Complex paths
5. `DrawArcSegment()` - Filled arc segments
**Medium Priority** (Performance-critical features):
6. Group methods: `BeginGroup()`, `EndGroup()`, `DrawGroup()`, `ClearCache()`
7. Render targets: `SetTarget()`, offscreen rendering
8. `SetNegativeDrawMode()` - Gerber-style rendering
**Low Priority** (Already implicit or not relevant to WASM):
9. `EnableDepthTest()` - Implicit in complex-scene
10. Context locking - Single-threaded in WASM
---
## Code Quality Assessment
### Strengths
1. **Clean separation** - Stubs, accessor, scenarios in separate files
2. **Minimal stubs** - Only implements what's needed
3. **Safe accessor** - Template technique avoids `#define private public`
4. **Standard shader embedding** - Common practice for GL applications
### Architecture Decisions
| Decision | Rationale |
|----------|-----------|
| Compile actual KiCad GAL | Ground truth for comparison |
| System wxWidgets | Native OpenGL context |
| FBO reading | Clean screenshots without window capture |
| Hex shader embedding | Runtime shader loading like KiCad |
---
## Running the Test
### Build
```bash
./scripts/build-gal-native-test.sh
```
### Execute
```bash
./tests/gal-regression/native/build/gal_native_test --output ./baselines
```
### Options
| Flag | Description |
|------|-------------|
| `--output <dir>` | Output directory for PNG files |
| `--width <w>` | Canvas width (default: 800) |
| `--height <h>` | Canvas height (default: 600) |
| `--show` | Show window instead of headless |
---
## Next Steps
### Immediate: Expand Test Coverage
1. Add `DrawBitmap` scenario with simple test image
2. Add `DrawCurve` (Bezier) scenario
3. Add text rendering scenario (requires KIFONT implementation)
4. Add `DrawSegmentChain` scenario
5. Add `DrawArcSegment` scenario
### Future: WebGL Comparison
1. Run same scenarios in WASM build
2. Compare native vs WebGL screenshots
3. Automate regression detection
---
## File Reference
| File | Location |
|------|----------|
| Test driver | `tests/gal-regression/native/gal_native_test.cpp` |
| Stubs | `tests/gal-regression/native/kicad_stubs.cpp` |
| Private accessor | `tests/gal-regression/native/gal_test_accessor.cpp` |
| Test scenarios | `tests/gal-regression/scenarios/gal_test_scenarios.cpp` |
| CMake config | `tests/gal-regression/native/CMakeLists.txt` |
| Shader generator | `tests/gal-regression/native/generate_shaders.py` |
| Build script | `scripts/build-gal-native-test.sh` |

235
docs/features/archive/webgl/0003-webgl-gal-implementation-plan.md Normal file
View file

@ -0,0 +1,235 @@
# WebGL GAL Port - Implementation Complete
> **ARCHIVED / HISTORICAL** — the WebGL-GAL backend described here was implemented (see `kicad/common/gal/webgl/`). Live test docs: [`tests/gal-regression/README.md`](../../../../tests/gal-regression/README.md). Kept for design rationale.
## Status: ALL PHASES COMPLETE ✅
**Branch:** `webgl` (21+ commits)
## Goal
Port KiCad's GAL (Graphics Abstraction Layer) from OpenGL to WebGL to enable full KiCad functionality in the browser. Use the existing 28-scenario test suite to verify visual parity between native OpenGL and WebGL implementations.
## Final State
- **WebGL GAL**: Fully integrated into `kicad/common/gal/webgl/` (~27,800 lines)
- **GAL Test Suite**: 28 scenarios passing, visual parity with native OpenGL
- **KiCad WASM**: Builds and runs successfully with WebGL GAL
- **3D Viewer**: Disabled with stubs for WASM builds (`KICAD_BUILD_3D_VIEWER_WASM=OFF`)
## Key Decisions
- **Approach**: Copy and modify existing OpenGL GAL code
- **Scope**: Full feature parity (all 28 scenarios)
- **Location**: Developed in `tests/gal-regression/wasm/` first, moved to KiCad for integration
## Architecture
**Two-backend test architecture:**
```
┌─────────────────────────────────────────────────────────────┐
│ SAME 28 SCENARIO FILES │
│ (scenarios/*.cpp - pure GAL API calls) │
└─────────────────────────────────────────────────────────────┘
│
┌───────────────┴───────────────┐
▼ ▼
┌─────────────────────────────┐ ┌─────────────────────────┐
│ NATIVE TEST HARNESS │ │ WEBGL TEST HARNESS │
│ (gal_native_test.cpp) │ │ (gal_webgl_test.cpp) │
│ │ │ │
│ Uses: OPENGL_GAL │ │ Uses: WEBGL_GAL │
│ Runs: macOS native │ │ Runs: Browser/WASM │
└─────────────────────────────┘ └─────────────────────────┘
│ │
▼ ▼
┌─────────────────────────────┐ ┌─────────────────────────┐
│ output/native/gal-*.png │ │ output/webgl/gal-*.png │
└─────────────────────────────┘ └─────────────────────────┘
```
## Master Test Script: `scripts/test-gal-regression.sh`
**This is the only script we run.** Single command to build, test, and compare everything:
```bash
#!/bin/bash
# Single script to build, run, and compare both backends
# 1. BUILD BOTH
scripts/build-gal-native-test.sh
scripts/build-gal-webgl-test.sh
# 2. RUN BOTH TESTS
./tests/gal-regression/native/build/gal_native_test --output tests/gal-regression/output/native/
npx playwright test gal-webgl.spec.ts # outputs to tests/gal-regression/output/webgl/
# 3. COMPARE (two-level)
compare_screenshots output/native/ baseline/ # Catch native regressions
compare_screenshots output/webgl/ output/native/ # Verify WebGL matches native
# 4. REPORT
# Exit 0 if all match, exit 1 if any differ
```
**Two-level comparison:**
1. **native vs baseline** → Catches if native code regressed
2. **webgl vs native** → Verifies WebGL implementation matches
**Output structure:**
```
tests/gal-regression/
├── baseline/ # Committed reference screenshots
├── output/
│ ├── native/ # Fresh native run
│ └── webgl/ # WebGL run via Playwright
```
## Phases - ALL COMPLETE ✅
### Phase 1: Native Test Harness ✅
Created unified test infrastructure with 28 scenarios covering 100% of GAL API.
**Commits:** 490f531 → 051fb87
**Deliverables:**
- [x] `scripts/test-gal-regression.sh` - Master build/test/compare script
- [x] `scripts/build-gal-native-test.sh` - Native build script
- [x] `tests/gal-regression/native/` - Native C++ test harness using OPENGL_GAL
- [x] `tests/gal-regression/scenarios/` - 28 shared test scenarios
- [x] `tests/gal-regression/baseline/` - Native OpenGL reference screenshots
### Phase 2: WebGL GAL Implementation ✅
Full port of OPENGL_GAL to WebGL 2.0 / OpenGL ES 3.0.
**Commits:** a4f444f → 74faa7e
**Key Changes:**
- Replaced legacy `glBegin/glEnd` with VBO-based rendering
- Replaced GL matrix stack with glm matrices
- Converted GLSL shaders to ES 3.0 (`attribute`→`in`, `varying`→`out`, etc.)
- Added VAO support (required for WebGL 2.0)
- Replaced GLU tesselator with earcut.hpp
**Deliverables:**
- [x] `tests/gal-regression/wasm/webgl/` - Initial WebGL GAL implementation
- [x] `scripts/build-gal-webgl-test.sh` - WASM build script
- [x] `tests/e2e/gal-webgl.spec.ts` - Playwright spec for screenshots
### Phase 3: Complete API Coverage ✅
All GAL methods implemented with visual parity.
**Method Groups (all complete):**
- [x] Basic drawing: DrawLine, DrawSegment, DrawCircle, DrawArc
- [x] Shapes: DrawRectangle, DrawPolygon, DrawPolyline
- [x] Advanced: DrawBezier, DrawBezierArc, DrawArcSegment, DrawSegmentChain
- [x] State: Colors, transforms, depth testing, render targets
- [x] Groups: BeginGroup, EndGroup, DrawGroup, ChangeGroupColor/Depth
- [x] Text: DrawGlyph, DrawGlyphs, BitmapText
- [x] Special: DrawGrid, DrawCursor, DrawBitmap
### Phase 4: KiCad Integration ✅
WebGL GAL moved to KiCad source tree with CMake integration.
**Commit:** 37d973f (KiCad submodule: 1b5bb125d2)
**Deliverables:**
- [x] Move `webgl_gal.*` to `kicad/common/gal/webgl/`
- [x] CMake integration for Emscripten builds
- [x] 3D viewer disabled with stubs (`KICAD_BUILD_3D_VIEWER_WASM=OFF`)
- [x] KiCad WASM builds and e2e tests pass
---
## Technical Lessons Learned
### 1. GLSL ES 3.0 Shader Conversion
Desktop OpenGL shaders needed conversion for WebGL 2.0:
```glsl
attribute → in
varying → out (vertex shader) / in (fragment shader)
texture2D() → texture()
gl_FragColor → explicit out variable
```
Automated in `generate_shaders.py`.
### 2. VAO Required for WebGL 2.0
WebGL 2.0 requires Vertex Array Objects (VAOs):
```cpp
glGenVertexArrays(1, &m_vao);
glBindVertexArray(m_vao);
```
### 3. Legacy GL Elimination
All legacy OpenGL calls replaced:
- `glBegin/glEnd` → VBO-based rendering
- `glPushMatrix/glPopMatrix` → glm matrix stack
- `GL_QUADS` → `GL_TRIANGLES` (quads not supported in WebGL)
- `glEnableClientState` → modern vertex attributes
### 4. GLU Tesselator Replacement
GLU not available in WebGL. Implemented using earcut.hpp:
- File: `kicad/common/gal/webgl/glu_tess_impl.cpp`
- Provides `gluNewTess`, `gluTessBeginPolygon`, etc.
### 5. Coordinate System & Retina Scaling
- White background required for screenshot comparison (alpha compositing)
- Retina 2x scaling requires proper `devicePixelRatio` handling
- World-to-screen mapping: `SetWorldUnitLength(1.0 / DPI)`
### 6. Alpha Blending
Proper blend functions required for correct transparency:
```cpp
glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE, GL_ONE_MINUS_SRC_ALPHA);
```
### 7. 3D Viewer Stubbing Strategy
When disabling 3D viewer (`KICAD_BUILD_3D_VIEWER_WASM=OFF`):
- Guard includes/code with `#ifndef __EMSCRIPTEN__`
- Stub classes need wxWidgets event table macros (`BEGIN_EVENT_TABLE`)
- Complex classes (PANEL_PREVIEW_3D_MODEL) require all event handler stubs
## File Structure (Final)
```
kicad/common/gal/webgl/ # WebGL GAL in KiCad source tree
├── webgl_gal.cpp # Main implementation (3184 lines)
├── webgl_gal.h # Class declaration (618 lines)
├── webgl_compositor.cpp # FBO compositing
├── webgl_antialiasing.cpp # SMAA implementation
├── gpu_manager.cpp # VBO/VAO management
├── vertex_manager.cpp # Vertex accumulation
├── shader.cpp # GLSL compilation
├── glu_tess_impl.cpp # GLU tesselator (earcut)
├── earcut.hpp # Polygon triangulation
└── ... (20+ files total)
tests/gal-regression/
├── baseline/ # Native OpenGL reference (28 PNGs)
├── baseline-webgl/ # WebGL reference (29 PNGs)
├── native/ # Native test harness
├── wasm/ # WebGL test harness (uses KiCad GAL)
└── scenarios/ # Shared 28 test scenarios
wasm/stubs/
├── 3d_canvas_stub.cpp # ~500 lines of 3D stubs
├── 3d_viewer_stub.cpp # EDA_3D_VIEWER_FRAME stub
└── 3d_scenegraph_stub.cpp # VRML export stubs
scripts/
├── build-gal-native-test.sh # Native build script
├── build-gal-webgl-test.sh # WASM build script
└── test-gal-regression.sh # Master test script
```
---
## Verification Commands
```bash
# Build KiCad WASM
docker/build.sh
# Run GAL regression tests
scripts/test-gal-regression.sh
# Run KiCad e2e tests
cd tests && npm run test:kicad
```

329
docs/features/browser-tools/0001-kicad-wasm-tool-activation-investigation.md Normal file
View file

@ -0,0 +1,329 @@
# KiCad WASM Tool Activation Investigation
## Summary
This document explains the current investigation into why interactive PCB tools do not work correctly in the browser build of KiCad.
The visible symptom is simple:
- In native KiCad, clicking `Draw Lines` in the right toolbar leaves the tool selected, and two clicks on the board create a line.
- In the browser build, the tool does not remain selected, and board clicks do not start drawing.
The important conclusion so far is that this does **not** look like a normal KiCad tool-definition bug. The evidence points much lower in the stack, into the WebAssembly coroutine/runtime path used to emulate KiCad's native coroutine model in the browser.
## What "Working" Looks Like Natively
On macOS, Linux, and Windows, the flow for a tool like `Draw Lines` is roughly:
```text
User clicks right toolbar button
-> wxAuiToolBar handles mouse up
-> ACTION_TOOLBAR emits tool action
-> TOOL_MANAGER activates the requested tool
-> KiCad starts or resumes the tool coroutine
-> User clicks board canvas
-> GAL / canvas event is forwarded to the active tool
-> Tool consumes the clicks and creates geometry
```
Two details matter here:
1. KiCad tools are not just plain event handlers. Many of them are coroutine-driven.
2. Native builds can rely on real low-level context switching and on the OS windowing system for input routing.
## Core Concepts
### `wxAuiToolBar`
KiCad's right-side drawing toolbar is an AUI toolbar, not a plain `wxToolBar`. That matters because its rendering, hit-testing, and state transitions go through a different path than the simpler wxWidgets toolbar tests.
### `ACTION_TOOLBAR` and `TOOL_MANAGER`
The toolbar does not draw lines by itself. Clicking a button activates a named KiCad action, and the tool manager is responsible for making the corresponding interactive tool current.
### KiCad coroutines
Interactive tools in KiCad depend on coroutine-style control flow. On native platforms this is implemented with `libcontext`, specifically `make_fcontext()` and `jump_fcontext()`.
### `wxGLCanvas` / `WEBGL_GAL`
Once a tool is active, board clicks are handled through KiCad's graphics/input path. In the browser build that path includes a DOM canvas, wxWidgets' WASM port, and KiCad's WebGL GAL layer.
### Emscripten fibers and Asyncify
WebAssembly in the browser cannot perform the same kind of native stack switching that desktop KiCad uses. Our port therefore has to emulate it with:
- Emscripten fibers
- Asyncify
- generated JavaScript glue around fiber switches
That emulation layer is the main place where the browser build can diverge from the native behavior.
## Native Data Flow
The native path is conceptually:
```text
OS mouse event
-> wxWidgets window / child-window dispatch
-> wxAuiToolBar::OnLeftUp()
-> ACTION_TOOLBAR
-> TOOL_MANAGER
-> libcontext coroutine switch
-> active KiCad tool
-> board clicks routed to the tool
-> drawing result appears
```
The key point is that native `libcontext` performs real context switching, and the OS owns the final event routing between the toolbar area and the graphics canvas.
## Browser/WASM Data Flow
The browser path has more moving parts:
```text
Browser pointer event
-> generated wx.js / WASM event bridge
-> wxWidgets WASM windowing layer
-> wxAuiToolBar or wxGLCanvas target
-> KiCad action dispatch
-> Emscripten fiber switch
-> Asyncify suspend/resume bookkeeping
-> KiCad tool coroutine body
-> WebGL canvas input handling
-> drawing result appears
```
This means the browser build must get all of the following correct at the same time:
- AUI toolbar hit-testing
- tool activation
- coroutine entry
- coroutine completion / return
- WebGL canvas event forwarding
If any one of those layers is wrong, the tool appears to "not work".
## What We Observed
At first glance the problem looked like a toolbar-state problem:
- the `Draw Lines` button did not stay selected
- clicking the board did nothing
However, once we added a proper E2E test with logging and delayed screenshots, the picture became clearer:
1. The button state alone was not enough to diagnose the problem.
An immediate screenshot can capture hover or pressed state, not true persistent selection.
2. After we added explicit `checked`-state tracking and a short delay, it became clear that the tool was still not truly active.
3. The deeper failure was not simply "the toolbar forgot its checked state".
## The First Real Root Cause We Found
The first concrete bug was in generated JavaScript around Asyncify fiber switching.
In the generated KiCad JS, the fiber entry callback in `Fibers.finishContextSwitch()` had effectively become a no-op:
```text
(a1 => {})(userData);
```
That means the code switched into the new fiber, but then did not actually call the tool coroutine entry function.
### Why this matters
If the entry callback is replaced by an empty function, the tool activation path can appear to run, but the coroutine body never really starts. From the outside, that looks like:
- the toolbar click "does something"
- but the tool never becomes truly active
- board clicks are ignored because there is no running interactive tool waiting for them
## Change 1: Fix the generated fiber entry callback
We added a new fix in [inject-dyncall-shims.sh](/Users/V/IdeaProjects/kicad-wasm/scripts/common/inject-dyncall-shims.sh) so the generated JS is patched to call the real entry function:
```text
dynCall_vi(entryPoint, userData);
```
### Reasoning
This is a good change because it fixes an objectively broken generated code path at the WASM/JS boundary. It is not a KiCad workaround.
## Change 2: Tell Asyncify that fiber swaps are suspension points
We updated [apply-asyncify.sh](/Users/V/IdeaProjects/kicad-wasm/scripts/common/apply-asyncify.sh) so `env.emscripten_fiber_swap` is included in `ASYNCIFY_IMPORTS`.
### Reasoning
Asyncify needs to know which imports can suspend or unwind control flow. Fiber swaps are exactly that kind of boundary. If Asyncify does not model them correctly, the call stack bookkeeping around coroutine switches becomes unreliable.
This is also a WASM-layer fix, not a KiCad UI workaround.
## What Happened After Those Changes
Those two changes moved the investigation forward, but they did not finish the problem.
After the JS callback fix, the logs started showing that the fiber entry code was actually being reached:
```text
[WASM_FCONTEXT] make
[WASM_FCONTEXT] swap
[WASM_FCONTEXT] entry
```
That is an important result. It means:
- the original "entry callback is a no-op" bug was real
- we did fix it
- but another problem exists after fiber entry
## The Current Deeper Problem
After the fiber starts, the startup sequence still stalls before PCBnew fully finishes bringing up the toolbars.
The clearest evidence comes from the E2E log at:
[pcbnew-spec-ts-pcbnew-wasm-select-draw-lines-and-draw-on-the-board.log](/Users/V/IdeaProjects/kicad-wasm/tests/logs/kicad/pcbnew/pcbnew-spec-ts-pcbnew-wasm-select-draw-lines-and-draw-on-the-board.log)
The rendered-element summary currently ends up as:
```text
{"count":13,"byType":{"sash":2,"searchctrl":3,"searchbutton":4,"auipart":4},"tools":[]}
```
That means the browser-side registry can see some UI pieces, but **no rendered toolbar tools at all**.
So the current failure is no longer best described as "the Draw Lines tool unchecks itself". The stronger diagnosis is:
- PCBnew startup is being interrupted
- the AUI toolbars never fully come online
- the tool registry is therefore empty
- the test cannot even reach a stable active-tool state
## Why Native KiCad Works But Browser KiCad Does Not
Native KiCad works because two hard problems are already solved by the native platform stack:
1. `libcontext` can use real native context switching semantics.
2. The operating system handles input routing across the real window hierarchy.
The browser build does not get either of those for free.
Instead, it must emulate them with:
- generated JS glue
- Asyncify instrumentation
- Emscripten fibers
- a DOM canvas and WebGL canvas bridge
So the browser failure is not evidence that KiCad's tool logic is broken. It is evidence that our WASM adaptation layer is still incomplete.
## Why Existing wxWidgets Tests Can Still Look Fine
This issue can exist even if many wxWidgets tests look correct.
The reason is that the failing KiCad path is more complex than a normal wx control interaction:
- KiCad uses `wxAuiToolBar`, not only basic controls
- KiCad tool activation goes through `ACTION_TOOLBAR` and `TOOL_MANAGER`
- KiCad drawing tools depend on coroutine switching
- the board uses a separate WebGL-backed canvas/input path
Most simple wxWidgets tests do not exercise that exact combination.
## What We Changed During The Investigation
The current working tree contains a mix of real fixes, testing support, and experiments.
### Changes that look fundamentally correct
| Layer | File | Purpose | Why it makes sense |
|------|------|---------|--------------------|
| Build/WASM glue | [scripts/common/inject-dyncall-shims.sh](/Users/V/IdeaProjects/kicad-wasm/scripts/common/inject-dyncall-shims.sh) | Fix empty fiber entry callback in generated JS | Repairs objectively broken generated code |
| Build/WASM glue | [scripts/common/apply-asyncify.sh](/Users/V/IdeaProjects/kicad-wasm/scripts/common/apply-asyncify.sh) | Add `env.emscripten_fiber_swap` to Asyncify imports | Makes Asyncify aware of fiber-switch suspension points |
| Test support | [tests/e2e/utils/element-tracker.ts](/Users/V/IdeaProjects/kicad-wasm/tests/e2e/utils/element-tracker.ts) | Track `checked` state | Lets the test distinguish hover/pressed from real selection |
| Test support | [tests/kicad/pcbnew.spec.ts](/Users/V/IdeaProjects/kicad-wasm/tests/kicad/pcbnew.spec.ts) | Add tool-selection/drawing regression and log capture | Reproduces the bug through the real KiCad flow |
| Test observability | [wxwidgets/src/aui/auibar.cpp](/Users/V/IdeaProjects/kicad-wasm/wxwidgets/src/aui/auibar.cpp) | Export rendered AUI tool items to the browser registry | Gives Playwright a reliable way to see KiCad AUI tools |
### Changes that are investigative, not final
| Layer | File | Purpose | Current assessment |
|------|------|---------|--------------------|
| KiCad WebGL input | [kicad/common/gal/webgl/webgl_gal.cpp](/Users/V/IdeaProjects/kicad-wasm/kicad/common/gal/webgl/webgl_gal.cpp) | Forward mouse events immediately on WASM instead of posting them | Useful experiment, but not yet proven to be the main fix |
| KiCad startup | [kicad/pcbnew/pcb_edit_frame.cpp](/Users/V/IdeaProjects/kicad-wasm/kicad/pcbnew/pcb_edit_frame.cpp) | Skip auto-invoking the selection tool on WASM | Pure debugging aid to see whether startup tool activation was the blocker |
| KiCad third-party porting layer | [kicad/thirdparty/libcontext/libcontext.cpp](/Users/V/IdeaProjects/kicad-wasm/kicad/thirdparty/libcontext/libcontext.cpp) | Experimental Emscripten-fiber implementation of `libcontext` | Likely the right conceptual layer, but the current implementation is not clean/final |
### Changes that are not meaningful
| File | Note |
|------|------|
| [kicad/common/tool/action_toolbar.cpp](/Users/V/IdeaProjects/kicad-wasm/kicad/common/tool/action_toolbar.cpp) | Trailing newline / formatting-only diff |
| [kicad/common/tool/tools_holder.cpp](/Users/V/IdeaProjects/kicad-wasm/kicad/common/tool/tools_holder.cpp) | Trailing newline / formatting-only diff |
| [wxwidgets/build/wasm/wx.js](/Users/V/IdeaProjects/kicad-wasm/wxwidgets/build/wasm/wx.js) | Generated build artifact, not a source-level design change |
## Why the Current `libcontext` Work Is Still Not "Done"
The current experiments in [libcontext.cpp](/Users/V/IdeaProjects/kicad-wasm/kicad/thirdparty/libcontext/libcontext.cpp) show that we can start a fiber, but not yet return from it with semantics that match what KiCad expects from native `jump_fcontext()`.
The native contract is subtle:
- one context transfers control to another
- control can later resume into the previous context
- returned values and ownership of "who resumes whom" must remain consistent
- cleanup of a finished coroutine must not break the surrounding frame startup
In the browser build, once the first startup coroutine finishes, that handoff is still wrong. The result is not necessarily an immediate crash anymore, but the UI startup is interrupted before the toolbars are fully present.
## Current Best Explanation
The current best explanation is:
1. A toolbar click is not the primary problem.
2. The browser port originally had a broken fiber entry callback, which prevented tool coroutines from starting at all.
3. After fixing that, the browser port still mishandles coroutine completion / return.
4. That deeper runtime mismatch interrupts PCBnew startup before the AUI tools fully appear.
5. Because the tools are not fully rendered and the interactive-tool runtime is not stable, the right-side drawing tools do not remain active and board clicks do not draw.
## Clean Direction From Here
The cleanest direction is to keep the fix low in the WASM/runtime layer and avoid papering over the issue in KiCad UI code.
Recommended direction:
1. Keep the JS fiber-entry fix in [inject-dyncall-shims.sh](/Users/V/IdeaProjects/kicad-wasm/scripts/common/inject-dyncall-shims.sh).
2. Keep the Asyncify import fix in [apply-asyncify.sh](/Users/V/IdeaProjects/kicad-wasm/scripts/common/apply-asyncify.sh).
3. Move the `libcontext` solution toward the dedicated WASM layer under [wasm/libcontext](/Users/V/IdeaProjects/kicad-wasm/wasm/libcontext) instead of continuing to patch KiCad's bundled third-party copy in ad hoc ways.
4. Remove temporary KiCad startup and input experiments once the lower-layer coroutine behavior is correct.
5. Keep the AUI rendered-tool export only if we still want browser E2E tests to target KiCad tools by tooltip and checked state.
## Reproduction and Evidence
### Build and run
```bash
./docker/build.sh
cd tests
npm run test:kicad -- --grep "select draw lines"
```
### Where to look
- E2E spec:
[pcbnew.spec.ts](/Users/V/IdeaProjects/kicad-wasm/tests/kicad/pcbnew.spec.ts)
- E2E logs:
[tests/logs/kicad/pcbnew](/Users/V/IdeaProjects/kicad-wasm/tests/logs/kicad/pcbnew)
- Current startup screenshot:
[wizard-00-initial.png](/Users/V/IdeaProjects/kicad-wasm/tests/test-results/wizard-00-initial.png)
## Bottom Line
The current evidence says the KiCad browser tool failure is fundamentally a WebAssembly coroutine/runtime problem.
The right fix direction is:
- not "change KiCad tool logic"
- not "hack the toolbar state"
- but "make the WASM coroutine and fiber handoff behave like native KiCad expects"
The two most defensible changes so far are the generated-JS fiber-entry fix and the Asyncify import update. Everything else should be treated as either observability support or investigation scaffolding until the underlying `libcontext` behavior is corrected.

795
docs/features/browser-tools/0002-wasm-coroutine-deep-dive.md Normal file
View file

@ -0,0 +1,795 @@
# KiCad WASM Coroutine Deep Dive
## Why Our Use Case Is Special
### Most WASM projects don't need coroutines at all
When you think of "compile C/C++ to WebAssembly with Emscripten," the typical projects are:
- **Games** (Unity, Unreal, etc.): They have a main loop that renders frames. The game engine calls `emscripten_set_main_loop(renderFrame, 60, 0)` and Emscripten calls `renderFrame()` 60 times per second. No coroutines needed — everything is event-driven already.
- **Command-line tools** (ffmpeg, SQLite, etc.): They run, produce output, and exit. Linear execution. No coroutines.
- **Simple GUI apps**: They handle events through callbacks. Button clicked → run handler. No need to pause mid-function.
**KiCad is unusual** because its interactive tools use a **synchronous programming model** inside a coroutine:
```cpp
void PCB_TOOL::DrawLine(TOOL_EVENT& evt) {
Point p1 = WaitForClick(); // ← PAUSES HERE, waits for user
Point p2 = WaitForClick(); // ← PAUSES HERE again
CreateLine(p1, p2);
}
```
This code looks simple and linear, but `WaitForClick()` can't actually block in a browser. Instead, KiCad uses a coroutine to pause the function, return control to the browser event loop, and resume later when the click arrives. This requires the ability to **save and restore the entire call stack** — which is what libcontext and fibers do.
### Very few projects need this
The number of large C/C++ applications that:
1. Were designed for desktop with coroutine-based control flow
2. Are now being ported to the browser via Emscripten
3. Need those coroutines to actually work
...is very small. QEMU is one. KiCad is another. Maybe a handful of others.
Because so few people need this, the Emscripten support for it is:
- **Functional** (the fiber API exists and works)
- **But rough around the edges** (bugs in internal code, poor documentation, edge cases not handled)
- **And under-tested** (most users never exercise these code paths)
### Emscripten's priorities
Emscripten's main user base is game engines and simple tools. Their effort goes into:
- Compilation speed
- WASM binary size
- Performance of simple programs
- SIMD, threading, memory64
The fiber/coroutine path is a niche feature. The `makeDynCall` bug persists because almost nobody hits it — the people who do (like us) work around it.
---
## Concepts From the Ground Up
### What is "The Stack"?
Every time you call a function, a new "frame" is pushed onto the call stack holding local variables, return address, and arguments.
```
main() calls drawLine() calls calculatePoint()
Stack (grows downward):
┌──────────────────┐
│ main() │ ← local vars of main
├──────────────────┤
│ drawLine() │ ← local vars of drawLine
├──────────────────┤
│ calculatePoint() │ ← TOP: local vars of calculatePoint
└──────────────────┘
```
**"Stack-local"** = a variable that lives in a frame. When that function returns, the frame is popped and the variable's memory becomes garbage.
### What is a Coroutine?
A function that can **pause** mid-execution and **resume** later. KiCad needs this because in a browser you can't block waiting for a mouse click (the page freezes). So a drawing tool must pause after requesting a click, let the browser run, then resume when the click arrives.
### Stackful vs Stackless
**Stackless** (C++20 `co_await`): Can only pause at the top level. If `waitForClick()` is 5 calls deep, you can't pause.
**Stackful** (KiCad): Can pause from **anywhere** in the call stack. The entire stack is saved and restored. KiCad needs this.
### What is libcontext?
A small C library (from Boost.Context) that performs context switching via three functions:
```cpp
make_fcontext(stack, size, entry_func); // Create a new context
jump_fcontext(&old, new, value); // Switch contexts
release_fcontext(ctx); // Free a context
```
On native: ~20 lines of assembly per platform (x86, ARM, etc.) that saves/loads CPU registers. On WASM: impossible natively, must be emulated.
**libcontext is NOT a separate repo.** It's a directory inside KiCad (`kicad/thirdparty/libcontext/`). Our kicad submodule points to our fork (`VV-EE/kicad-source-mirror.git`), so we already own it. No additional forking needed.
### The Full Stack of Abstractions
```
KiCad COROUTINE class (kicad/include/tool/coroutine.h)
↓ calls
libcontext API (make_fcontext / jump_fcontext)
↓ implemented with (on WASM)
Emscripten Fibers (emscripten_fiber_swap)
↓ built on
Asyncify (wasm-opt binary transformation)
↓ manipulates
WebAssembly call stack (in the browser's WASM runtime)
```
---
## How Asyncify Actually Works
### The Core Idea
WASM doesn't let you save/restore call stacks like native assembly does. Asyncify works around this with a completely different approach: **it rewrites your WASM bytecode** so that every function can cooperatively save its state and return, then later be re-called and skip ahead to where it left off.
Two globals drive everything:
- `__asyncify_state`: 0 = Normal, 1 = Unwinding, 2 = Rewinding
- `__asyncify_data`: pointer to a buffer that holds saved state
### The Asyncify Data Buffer
Each fiber/coroutine has its own buffer (the "asyncify stack"). Layout:
```
[ptr+0] i32: current stack position (grows upward as data is pushed)
[ptr+4] i32: stack end (upper bound)
[ptr+8] i32: rewind_id (which WASM export to re-enter during rewind)
[ptr+12] ... actual saved data (call indices + serialized local variables)
```
### What the Binary Transformation Does
Asyncify (via `wasm-opt --asyncify`) rewrites every function in the WASM module. Here's a before/after:
**Before transformation:**
```c
void foo(int x) {
x = x + 1;
x = x / 2;
bar(x); // ← this call might trigger a pause
while (x & 7) {
x = x + 1;
}
}
```
**After transformation (pseudocode of the generated WASM):**
```c
void foo(int x) {
// PRELUDE: if we're rewinding, restore our saved locals
if (__asyncify_state == REWINDING) {
x = pop_from_asyncify_stack(); // restore x
call_index = pop_from_asyncify_stack(); // which call site to skip to
}
// Normal code: skip during rewind
if (__asyncify_state == NORMAL) {
x = x + 1;
x = x / 2;
}
// The call site: execute if normal, OR if rewinding to this specific call
if (__asyncify_state == NORMAL || call_index == 0) {
bar(x);
// After the call returns: are we unwinding?
if (__asyncify_state == UNWINDING) {
push_to_asyncify_stack(0); // save call index (we were at bar())
push_to_asyncify_stack(x); // save local variable x
return; // cooperatively return up the chain
}
}
// Rest of function: skip during rewind
if (__asyncify_state == NORMAL) {
while (x & 7) {
x = x + 1;
}
}
}
```
**Key insight:** Every function in the call chain gets this treatment. During unwind, each frame saves its state and returns normally. During rewind, each frame skips ahead to the right call site and dives deeper.
### The Complete Unwind Sequence (Pause)
When something wants to pause (e.g., `emscripten_fiber_swap`):
```
1. JS sets __asyncify_state = UNWINDING (1)
2. JS sets __asyncify_data = pointer to this fiber's buffer
3. The call to emscripten_fiber_swap returns to its caller in WASM
4. The caller checks: state == UNWINDING? Yes.
→ Pushes its call index + locals to asyncify_data buffer
→ Returns to ITS caller
5. That caller checks: state == UNWINDING? Yes.
→ Same thing: push call index + locals, return
6. This cascades all the way up until the WASM export returns to JS
7. JS: all WASM frames have returned. Call asyncify_stop_unwind().
→ __asyncify_state = 0 (Normal)
8. The entire WASM call stack is gone. State is saved in the buffer.
```
### The Complete Rewind Sequence (Resume)
When something wants to resume a paused fiber:
```
1. JS sets __asyncify_state = REWINDING (2)
2. JS sets __asyncify_data = pointer to the saved fiber's buffer
3. JS calls the same WASM export function that was running before (e.g., main)
4. main() enters. Sees state == REWINDING.
→ Pops its locals from asyncify_data buffer
→ Pops its call index → skips ahead to that call site
→ Calls the function at that call site
5. That function enters. Sees state == REWINDING.
→ Same thing: pop locals, pop call index, skip ahead, call deeper
6. This continues until we reach the DEEPEST frame (the one that paused)
7. The deepest frame calls asyncify_stop_rewind()
→ __asyncify_state = 0 (Normal)
8. Execution continues normally from exactly where it paused.
```
### How `emscripten_fiber_swap` Coordinates Two Fibers
Each `emscripten_fiber_t` struct contains:
```c
typedef struct {
void* stack_base; // C stack top
void* stack_limit; // C stack bottom
void* stack_ptr; // current C stack pointer (saved on swap)
void (*entry)(void*); // entry function (NULL after first call)
void* user_data; // argument for entry function
asyncify_data_t asyncify_data; // this fiber's own asyncify buffer
} emscripten_fiber_t;
```
The swap sequence for switching from Fiber A to Fiber B:
```
Fiber A is running (state = Normal)
A calls emscripten_fiber_swap(&A, &B):
JS side:
1. state = Unwinding
2. currData = A.asyncify_data (save into A's buffer)
3. asyncify_start_unwind(A.asyncify_data)
4. Save A's C stack pointer into A.stack_ptr
5. Set Fibers.nextFiber = B
6. Return (emscripten_fiber_swap returns to caller)
WASM side:
7. A's call chain unwinds: each frame saves state into A.asyncify_data
8. All WASM frames return to JS
JS side (maybeStopUnwind):
9. asyncify_stop_unwind() → state = Normal
10. Fibers.trampoline() → finishContextSwitch(B)
finishContextSwitch(B):
11. Restore B's C stack pointer + limits
12. Is B.entry != NULL? (first time entering B)
YES → call B.entry(B.user_data) ← this is where dynCall_vi matters!
NO → (B was previously paused)
asyncify_start_rewind(B.asyncify_data)
doRewind() → calls the saved export, which replays B's call chain
B is now running.
Later, B calls emscripten_fiber_swap(&B, &A):
Same process in reverse:
- B's state is saved into B.asyncify_data
- finishContextSwitch(A):
A.entry == NULL → rewind into A.asyncify_data
A's call chain replays until emscripten_fiber_swap
emscripten_fiber_swap's "else" branch runs:
state = Normal
asyncify_stop_rewind()
A continues exactly where it left off.
```
### Why This Is Slow
Every context switch involves:
1. Unwinding the entire call stack (every frame saves state and returns)
2. Rewinding the entire call stack (every frame re-enters, restores state, skips ahead)
Native libcontext: save ~15 CPU registers, change stack pointer. Done in nanoseconds.
Asyncify: serialize/deserialize every frame. Documented overhead: 20-100% slowdown.
---
## The dynCall Problem
### What dynCall Functions Were
`dynCall_vi`, `dynCall_ii`, etc. were JavaScript wrapper functions for calling WASM **function pointers** from JS. Naming convention:
- `v` = void, `i` = int, `f` = float, `d` = double
- First letter = return type, rest = argument types
- `dynCall_vi(ptr, arg)` = "call the WASM function at table index `ptr` with one int `arg`, returning void"
They existed because calling a WASM function pointer from JavaScript requires:
1. Looking up the function in the `WebAssembly.Table` by index
2. Calling it with the right types
Before the WebAssembly.Table API stabilized, Emscripten generated one typed wrapper per signature used in the program.
### Why They Were Removed
Starting Emscripten 2.0.2 (August 2020), removed for performance:
The replacement is `getWasmTableEntry(index)` which directly looks up the function in the table:
```javascript
// Old way:
dynCall_vi(funcPtr, arg1);
// New way:
getWasmTableEntry(funcPtr)(arg1);
```
Benchmarks showed the new way is **60-80% faster** and produces smaller JS output.
### How Emscripten's Internal Code Uses dynCall
Emscripten's own JS library files (the runtime glue) need to call WASM function pointers too. They use a preprocessor macro called `makeDynCall`:
```javascript
// Inside Emscripten's library_async.js, library_html5.js, etc.
// This is a BUILD-TIME macro, expanded by Emscripten's preprocessor
// Old syntax (pre-2.0.9):
{{{ makeDynCall('vi') }}}(funcPtr, arg1)
// New syntax (2.0.9+):
{{{ makeDynCall('vi', 'funcPtr') }}}(arg1)
```
The difference: the old syntax doesn't tell the macro which variable holds the function pointer. The new syntax does.
### The Silent Degradation Bug
Here's what happens when the macro expands. Inside Emscripten's `parseTools.mjs`:
```javascript
function makeDynCall(sig, funcPtr) {
if (funcPtr === undefined) {
// OLD SYNTAX: funcPtr not provided
if (DYNCALLS) {
// -sDYNCALLS=1 is set: use the generated dynCall_vi function
return `dynCall_${sig}`;
}
// DYNCALLS is false (default since ~2.0.3)
// Try to find an exported dynCall_vi... it doesn't exist
// Fall through to:
return `((args) => {} /* a dynamic function call to signature ${sig},
but there are no exported function pointers with that signature,
so this path should never be taken. */)`;
}
// NEW SYNTAX: funcPtr provided → use getWasmTableEntry
return `getWasmTableEntry(${funcPtr})`;
}
```
**The critical problem:** Emscripten's **own internal library files** still use the old syntax in many places. When `DYNCALLS=false` (the default), the macro generates an empty arrow function `(a1 => {})` instead of actually calling the function.
The generated comment even says *"this path should never be taken"* — but it IS taken, because the internal library files trigger it.
### What Breaks
| Location in Emscripten JS | What the no-op replaces | Effect |
|---------------------------|------------------------|--------|
| `Fibers.finishContextSwitch` | `dynCall_vi(entryPoint, userData)` | **Fiber entry function never called** — coroutines are dead |
| `_emscripten_set_main_loop` | `dynCall_v(callback)` | Main loop callback is a no-op |
| `_emscripten_async_call` | `dynCall_vi(callback, arg)` | Timer callbacks never fire |
| `___call_sighandler` | `dynCall_vi(handler, sig)` | Signal handlers are no-ops |
| `invokeEntryPoint` (pthreads) | `dynCall_ii(entry, arg)` | Thread entry never called |
| HTML5 event callbacks | `dynCall_iiii(callback, ...)` | Mouse/keyboard events ignored |
### Why `finishContextSwitch` Matters Most For Us
This is the function that runs when a fiber is being entered for the first time. The flow:
```javascript
finishContextSwitch(newFiber) {
// ... restore C stack ...
var entryPoint = /* read from fiber struct */;
if (entryPoint !== 0) {
// FIRST TIME entering this fiber: call the entry function
var userData = /* read from fiber struct */;
// THIS LINE is what's broken:
{{{ makeDynCall('vi', 'entryPoint') }}}(userData);
//
// With old-syntax makeDynCall and DYNCALLS=false, this becomes:
// (a1 => {})(userData);
//
// The entry function is NEVER CALLED.
// The fiber "starts" but its body never runs.
} else {
// Subsequent entry: rewind via asyncify
// This path works fine
}
}
```
So: every fiber's FIRST entry goes through `dynCall_vi(entryPoint, userData)`. If that's a no-op, the coroutine body never starts. The fiber appears to start (the swap succeeds) but nothing actually happens inside it.
### What Our Fix Does
`inject-dyncall-shims.sh` does two things:
**1. Generates Asyncify-aware dynCall shims:**
```javascript
function dynCall_vi(funcPtr, arg1) {
var func = getWasmTableEntry(funcPtr);
// Track in Asyncify's export call stack so unwind/rewind works
Asyncify.exportCallStack.push('dynCall_vi');
try {
func(arg1);
} finally {
if (Asyncify.currData) {
// We're mid-unwind: set the rewind function
Asyncify.setDataRewindFunc(Asyncify.currData);
}
Asyncify.exportCallStack.pop();
}
}
```
**2. Patches the six empty arrow function patterns** to call the real shims.
### Why Not Just Use `-sDYNCALLS=1`?
You could. Unity does. But there's a subtlety:
Plain `dynCall_vi` from `-sDYNCALLS=1` is just:
```javascript
function dynCall_vi(index, a1) { getWasmTableEntry(index)(a1); }
```
It does **not** push/pop `Asyncify.exportCallStack`. That tracking is needed for Asyncify to know which WASM export to re-enter during rewind. Without it, if a fiber swap happens inside an indirect call (function pointer), Asyncify loses track of the call chain and the rewind fails.
Our shims add this tracking. That's the extra value over `-sDYNCALLS=1`.
### Is This an Emscripten Bug?
**Yes, arguably.** The problem is that Emscripten's own internal library files (`library_async.js`, `library_html5.js`, etc.) use the deprecated `makeDynCall` syntax, which silently degrades to no-ops when `DYNCALLS=false`. The generated comment says "this path should never be taken" but it's taken constantly. Worth filing as a bug.
---
## QEMU: The Gold Standard Reference
### What QEMU actually is
QEMU is a **machine emulator and virtualizer**. It lets you:
- Run an ARM Linux system on your x86 laptop
- Run Windows inside a virtual machine on Linux
- Emulate hardware for embedded development
It's one of the most important open-source infrastructure projects — it powers much of cloud computing (via KVM/QEMU).
### Why QEMU uses coroutines
QEMU's disk I/O layer uses coroutines for the same reason KiCad uses them: to write **synchronous-looking code** that actually runs asynchronously.
When QEMU needs to read from a virtual disk:
```c
void handle_disk_read(Request *req) {
Buffer data = read_from_disk(req->sector); // ← this might take time
send_data_to_guest(req, data);
}
```
`read_from_disk()` might need to wait for actual I/O. Instead of blocking (which would freeze the emulator), QEMU pauses the coroutine, processes other events, and resumes when the data is ready. Exactly the same pattern as KiCad's `WaitForClick()`.
### Why QEMU was recently ported to WASM
People want to run QEMU in the browser — to provide virtual machines in web-based development environments, education tools, etc. The QEMU project accepted patches to build with Emscripten, and part of that work was making coroutines work in WASM.
### Why QEMU's solution is relevant to us
QEMU and KiCad have the **exact same problem**:
- Both are large C/C++ codebases
- Both use stackful coroutines internally
- Both need those coroutines to work when compiled to WASM
- Both use Emscripten's fiber API as the backend
QEMU's solution (`util/coroutine-wasm.c`) was:
1. Written by someone who clearly understood the Emscripten fiber API constraints
2. Reviewed by the QEMU maintainers
3. Accepted into the official QEMU repository
4. Has been running in production
It's only 127 lines. It's the cleanest, most proven reference for "how to do coroutines in Emscripten."
### QEMU's Full Implementation
Source: [github.com/qemu/qemu/blob/master/util/coroutine-wasm.c](https://github.com/qemu/qemu/blob/master/util/coroutine-wasm.c)
**The struct:**
```c
typedef struct {
Coroutine base; // QEMU's base coroutine type
void *stack; // C stack buffer (heap-allocated)
size_t stack_size;
void *asyncify_stack; // Asyncify data buffer (heap-allocated)
size_t asyncify_stack_size;
CoroutineAction action; // Communication channel (YIELD, TERMINATE, etc.)
emscripten_fiber_t fiber; // The Emscripten fiber handle
} CoroutineEmscripten;
```
Each coroutine owns **two** heap-allocated buffers: a C stack and an asyncify stack. Both persist for the coroutine's lifetime.
**The trampoline (most important part):**
```c
static void coroutine_trampoline(void *co_)
{
Coroutine *co = co_;
while (true) { // ← NEVER returns
co->entry(co->entry_arg); // Run the coroutine body
qemu_coroutine_switch(co, co->caller,
COROUTINE_TERMINATE); // Swap back to caller
}
}
```
Walk-through:
1. A new QEMU coroutine is created for some I/O operation.
2. `emscripten_fiber_init()` is called with `coroutine_trampoline` as the entry function.
3. When the coroutine is first entered (someone swaps to it), `coroutine_trampoline` starts running.
4. It calls `co->entry(co->entry_arg)` — this is the actual I/O handler.
5. The I/O handler might pause (yield) many times while waiting for data. Each yield does a `fiber_swap` back to the caller, and each resume does a `fiber_swap` back to this coroutine. But throughout all of that, `coroutine_trampoline` is still on the stack — we're inside the `co->entry()` call.
6. Eventually the I/O handler finishes and returns.
7. `coroutine_trampoline` resumes after the `co->entry()` line.
8. It calls `qemu_coroutine_switch(co, co->caller, COROUTINE_TERMINATE)` — this swaps back to the caller with a "I'm done" flag.
9. The `while(true)` loops back to the top. If nobody ever swaps back to this coroutine, it just stays suspended here forever (which is fine — the fiber is deallocated later).
10. The entry function **never returns**. The `while(true)` guarantees it.
**Creating a coroutine:**
```c
Coroutine *qemu_coroutine_new(void)
{
CoroutineEmscripten *co = g_malloc0(sizeof(*co));
co->stack_size = COROUTINE_STACK_SIZE;
co->stack = qemu_alloc_stack(&co->stack_size);
co->asyncify_stack_size = COROUTINE_STACK_SIZE;
co->asyncify_stack = g_malloc0(co->asyncify_stack_size);
emscripten_fiber_init(
&co->fiber,
coroutine_trampoline, // the infinite-loop entry
&co->base, // user_data
co->stack, co->stack_size,
co->asyncify_stack, co->asyncify_stack_size
);
return &co->base;
}
```
Both stacks are **heap-allocated** and persist for the coroutine's entire lifetime.
**Context switch:**
```c
CoroutineAction qemu_coroutine_switch(Coroutine *from_, Coroutine *to_,
CoroutineAction action)
{
CoroutineEmscripten *from = DO_UPCAST(CoroutineEmscripten, base, from_);
CoroutineEmscripten *to = DO_UPCAST(CoroutineEmscripten, base, to_);
set_current(to_);
to->action = action; // Tell the target why
emscripten_fiber_swap(&from->fiber, &to->fiber); // Swap!
return from->action; // Read what caller set
}
```
Communication between coroutines uses the `action` field: one side sets it before swapping, the other reads it after resuming.
**Main thread bootstrap (lazy init):**
```c
Coroutine *qemu_coroutine_self(void)
{
Coroutine *self = get_current();
if (!self) {
// First call: capture the main thread as a fiber
CoroutineEmscripten *leaderp = g_malloc0(sizeof(*leaderp));
leaderp->asyncify_stack = g_malloc0(leader_asyncify_stack_size);
leaderp->asyncify_stack_size = leader_asyncify_stack_size;
emscripten_fiber_init_from_current_context(
&leaderp->fiber,
leaderp->asyncify_stack,
leaderp->asyncify_stack_size
);
set_leader(leaderp);
self = &leaderp->base;
set_current(self);
}
return self;
}
```
**Cleanup:**
```c
void qemu_coroutine_delete(Coroutine *co_)
{
CoroutineEmscripten *co = DO_UPCAST(CoroutineEmscripten, base, co_);
qemu_free_stack(co->stack, co->stack_size);
g_free(co->asyncify_stack);
g_free(co);
}
```
Both stacks freed when coroutine destroyed. No stack-local temporaries, no abandoned frames.
### The Key Insight: Entry Function Must NEVER Return
Emscripten's fiber API has a rule: **if the fiber's entry function returns, the program terminates**. This is documented in `fiber.h`:
> "If entry_func returns, the entire program will end, as if main had returned."
Why? Because when the entry function returns, control goes... nowhere. The fiber's stack is done. There's no caller to return to (the fiber was started from a swap, not a regular function call). Emscripten handles this by treating it as program exit.
---
## How Our Implementation Compares to QEMU
### Our Code (`kicad/thirdparty/libcontext/libcontext.cpp`)
```cpp
[[noreturn]] void wasm_fcontext_entry(void* aArg)
{
auto* ctx = static_cast<wasm_fcontext*>(aArg);
// Step 1: Run the coroutine body
ctx->entry(ctx->transfer_value);
// Step 2: The coroutine body returned. We're in trouble.
ctx->running = false;
// Step 3: Try to swap back to the caller
if (ctx->return_to)
{
// Create a TEMPORARY fiber just so we have something to swap FROM
emscripten_fiber_t finished_ctx {};
alignas(16) char finished_asyncify_stack[64*1024] {};
emscripten_fiber_init_from_current_context(&finished_ctx, ...);
// Swap to the caller. We'll never come back.
emscripten_fiber_swap(&finished_ctx, &ctx->return_to->fiber);
}
// Step 4: If we get here, kill everything
emscripten_unwind_to_js_event_loop();
}
```
The problems:
1. **Step 2 is dangerous.** The entry function returned. According to Emscripten docs, this should terminate the program. We're in undefined territory.
2. **Step 3 creates stack-local buffers.** `finished_ctx` and `finished_asyncify_stack` (64KB!) are on this function's stack. When we swap away, this stack frame is abandoned. But Asyncify's bookkeeping still holds pointers to `finished_ctx` (because `emscripten_fiber_swap` saves the asyncify state into it). If Asyncify ever tries to do anything with those pointers, it's reading garbage memory.
3. **Step 4 uses `emscripten_unwind_to_js_event_loop()`**. This function says "I'm done with all WASM execution, return to the browser event loop." It tears down the ENTIRE WASM call stack — not just this fiber, but everything. If this happens during KiCad's startup sequence, the startup dies.
### The Three Differences
| Issue | Our Code | QEMU |
|-------|----------|------|
| Entry function returns? | Yes, then handles it | Never - `while(true)` |
| Stack-local asyncify buffers? | Yes (64KB on stack) | No - all heap-allocated |
| `emscripten_unwind_to_js_event_loop`? | Yes, as fallback | Not used |
---
## How Would We Adopt QEMU's Pattern?
### The change is small
The fix is to replace our `wasm_fcontext_entry` with a QEMU-style trampoline:
**QEMU-style replacement:**
```cpp
[[noreturn]] void wasm_fcontext_entry(void* aArg)
{
auto* ctx = static_cast<wasm_fcontext*>(aArg);
while (true) {
// Run the coroutine body
ctx->entry(ctx->transfer_value);
// Coroutine finished. Swap back to whoever started us.
ctx->running = false;
if (ctx->return_to) {
ctx->return_to->transfer_value = 0;
ctx->return_to->running = true;
g_current_context = ctx->return_to;
emscripten_fiber_swap(&ctx->fiber, &ctx->return_to->fiber);
// If we're swapped back to (unlikely), the while(true) loops
}
}
// We never reach here
}
```
Key differences:
1. `while(true)` ensures we never return from the entry function
2. We swap using `ctx->fiber` (the coroutine's own, heap-allocated fiber) instead of creating a stack-local temporary
3. No `emscripten_unwind_to_js_event_loop()` — we just stay in the loop
### How hard is the change?
**Maybe 15-20 lines changed** in one file (`kicad/thirdparty/libcontext/libcontext.cpp`). The architecture is already right — we use Emscripten fibers, we have a `wasm_fcontext` struct with proper fields, we have `g_current_context` tracking. The only wrong part is the entry-return handling.
The change is small but **the testing is critical**. After making it:
1. Rebuild the WASM module (`docker/build.sh`)
2. Run the PCBnew E2E test (`cd tests && npm run test:kicad -- --grep "select draw lines"`)
3. Check if toolbars now fully appear (the `tools: []` should become non-empty)
4. Check if drawing actually works
### What could go wrong?
The biggest risk is that `jump_fcontext`'s semantics don't perfectly match what KiCad's COROUTINE class expects. Specifically:
- KiCad's COROUTINE uses `jump_fcontext(&old_ctx, new_ctx, value)` where the returned `intptr_t` is a pointer to `INVOCATION_ARGS` that tells the coroutine why it was resumed (FROM_ROOT, FROM_ROUTINE, CONTINUE_AFTER_ROOT).
- If the trampoline loop doesn't correctly set the transfer value before swapping back, the caller might misinterpret why the coroutine stopped.
But this is testable — the E2E test will catch it.
---
## Verification of the Investigation Document
The original investigation document (`0001-kicad-wasm-tool-activation-investigation.md`) was reviewed and verified:
| Claim | Verdict |
|-------|---------|
| Fiber entry callback was a no-op | TRUE |
| inject-dyncall-shims.sh fixes it | TRUE |
| emscripten_fiber_swap needed in ASYNCIFY_IMPORTS | TRUE |
| Tools array is empty / startup stalls | TRUE |
| Problem is in WASM coroutine layer, not KiCad UI | TRUE (well-supported) |
| libcontext impl is "experimental / not clean" | FALSE - it's well-structured production code |
| Coroutine return is the remaining issue | PLAUSIBLE but not proven |
| KiCad investigative changes exist in files | NOT FOUND (likely already reverted) |
---
## Summary
### What's already correct
- The `wasm_fcontext` struct design
- The `make_fcontext` / `jump_fcontext` API mapping to fibers
- The `inject-dyncall-shims.sh` JS patching
- The `apply-asyncify.sh` import configuration
- The main context lazy initialization
### What needs fixing
- `wasm_fcontext_entry`: add `while(true)`, remove stack-local buffers, remove `emscripten_unwind_to_js_event_loop()`
- Dead code cleanup: delete `wasm/libcontext/` directory
### Alternative approaches considered
| Approach | Viability | Notes |
|----------|-----------|-------|
| Fix current Asyncify + fibers | HIGH | Adopt QEMU's trampoline pattern |
| JSPI (JS Promise Integration) | MEDIUM | Future Asyncify replacement, limited browser support |
| Event-driven state machines | NOT VIABLE | Rewrites every KiCad tool |
| C++20 stackless coroutines | NOT COMPATIBLE | KiCad needs stackful suspension |
| WASM Stack Switching proposal | FUTURE | Not standardized yet |
### References
- [QEMU coroutine-wasm.c](https://github.com/qemu/qemu/blob/master/util/coroutine-wasm.c) — gold standard implementation
- [Emscripten fiber.h docs](https://emscripten.org/docs/api_reference/fiber.h.html) — API reference
- [Asyncify blog post](https://kripken.github.io/blog/wasm/2019/07/16/asyncify.html) — deep technical dive
- [Binaryen Asyncify.cpp](https://github.com/WebAssembly/binaryen/blob/main/src/passes/Asyncify.cpp) — the compiler pass source
- [Fiber PR #9859](https://github.com/emscripten-core/emscripten/pull/9859) — design discussion
- [minicoro](https://github.com/edubart/minicoro) — single-header coroutine lib with WASM support
- [Issue #13302](https://github.com/emscripten-core/emscripten/issues/13302) — fiber swap return value bug
- [Issue #12733](https://github.com/emscripten-core/emscripten/issues/12733) — dynCall removal discussion

304
docs/features/browser-tools/0003-wxauitoolbar-registration-fix.md Normal file
View file

@ -0,0 +1,304 @@
# wxAuiToolBar Registration — Tool Selection Fix
## Context
The nested Asyncify collision bug (see `0002-wasm-coroutine-deep-dive.md` and `../../research/threading_1.md`) is fixed. KiCad WASM now loads through the startup wizard without crashing, and the full PCBnew UI renders — menus, left drawing-tool sidebar with Line/Circle/Rectangle icons, layer panel, PCB canvas — all visible.
But tools still don't work end-to-end:
- **User observation**: clicking the Draw Lines tool in the browser doesn't visibly select it or make it function.
- **E2E test**: `select draw lines and draw on the board` can't even attempt a click — it fails earlier at `wxElementRegistry.findAllRendered({ elementType: 'tool' })` because the returned array is empty.
Both signals point at the same gap: **wxAuiToolBar never registers its tools with the rendered-element registry**. The intended outcome of this fix is to make all wxAuiToolBar buttons (Draw Lines and siblings) clickable, selectable, and functional in the browser build.
---
## Root Cause
### The registry and how it gets populated
`window.wxElementRegistry` is a JS-side registry of UI elements used by Playwright tests to find controls by type/label/tooltip. For elements that are not standalone wxWindow instances (e.g., toolbar buttons rendered as pixels on a parent canvas), wxWidgets calls a C++ bridge `WasmRegisterRenderedElement()` which invokes the JS helper `wxRenderedElementRegister()`.
Canonical definition: `wxwidgets/src/wasm/window.cpp:182-214`:
```cpp
void WasmRegisterRenderedElement(
wxWindow* parent,
const char* elementType, // "tool", "menuitem", "sash", "auipart", ...
const char* subType,
int index,
const wxString& label,
const wxString& tooltip,
int screenX, int screenY,
int width, int height,
bool enabled);
```
### Where it's currently called
Grepping `wxwidgets/src/` for `WasmRegisterRenderedElement`:
| File | What it registers |
|---|---|
| `src/univ/toolbar.cpp:557-607` | Regular wxToolBar items (in `RecalcToolBitmapCache`) |
| `src/univ/menu.cpp` | Menu bar items, popup menu items |
| `src/aui/framemanager.cpp:2687-2787` | AUI pane captions, close/pin/maximize buttons |
| `src/aui/tabart.cpp:392,1137` | Tab headers |
| `src/univ/textctrl.cpp:4304-4319` | Text control segments |
| `src/propgrid/propgrid.cpp:2509-2537` | Property grid rows |
| `src/stc/stc.cpp:5203-5213` | Styled text cells |
### The gap
`wxwidgets/src/aui/auibar.cpp` has **zero** `__EMSCRIPTEN__` blocks and **zero** calls to `WasmRegisterRenderedElement`. Verified:
```
$ grep -nE "__EMSCRIPTEN__|WasmRegister" wxwidgets/src/aui/auibar.cpp
(no matches)
$ git log --oneline -n 10 src/aui/auibar.cpp
# Only upstream wxWidgets commits — our fork hasn't modified this file.
```
KiCad's left drawing-tool sidebar is a `wxAuiToolBar` (not `wxToolBar`), which is why its tools are invisible to the registry.
### Test log evidence
From `tests/logs/kicad/pcbnew/pcbnew-spec-ts-pcbnew-wasm-select-draw-lines-and-draw-on-the-board.log`:
```
[TEST] rendered summary {"count":42,"byType":{
"searchctrl":3,"searchbutton":4,"sash":2,"auipart":4,
"combobutton":8,"combotextarea":8,"textctrl":1,"tab":3,"menuitem":9
},"tools":[]}
```
Everything else registers. `tools` is the only empty bucket.
### The log is otherwise clean
Filtering out diagnostic output (`WASM_FCONTEXT`, `DIAG_*`, `wxLog DEBUG`) leaves only three substantive lines, all informational:
```
[DIAG_SHOWMODAL] About to call startModal()
Debug: EndModal: 5100
[DIAG_SHOWMODAL] startModal() returned 5100
```
No exceptions, no crashes, no fiber errors, no `jump-ghost`, `main_refresh=1` stable. The nested Asyncify fix is holding. **The only remaining issue between "UI loads" and "tool works" is this registration gap.**
### Separating the two signals
The registry gap directly explains the **test failure**. It does NOT directly explain the **user's manual observation** — registry population is test-only infrastructure and has no effect on in-browser interactivity.
Hypothesis: once tools are registered, the test will click the tool via coordinates from the registry and produce a log that either shows the click succeeded (no bug — user's "doesn't select" was a display misreading because state wasn't exposed) or shows concrete failure evidence (real activation bug, e.g., another variant of coroutine/asyncify interaction). Either way, the registration fix is strictly additive and unblocks diagnosis.
---
## The Fix
### Summary
Two small changes, one new block:
1. **Extend the registry signature** to include `checked` state (needed so the test can verify selection).
2. **Add a registration block** to `wxAuiToolBar::OnPaint()` following the `univ/toolbar.cpp` pattern.
3. **Update existing callers** to pass a `checked` value (`false` for non-toggleable, real state for `wxItemCheck/wxItemRadio`).
### Change 1 — extend `WasmRegisterRenderedElement` signature
**File: `wxwidgets/src/wasm/window.cpp`** (function at line 182)
Add a `bool checked` parameter and pass it through to the JS helper:
```cpp
void WasmRegisterRenderedElement(
wxWindow* parent,
const char* elementType,
const char* subType,
int index,
const wxString& label,
const wxString& tooltip,
int screenX, int screenY,
int width, int height,
bool enabled,
bool checked) // ← NEW
{
if (!parent) return;
uintptr_t parentId = reinterpret_cast<uintptr_t>(parent);
EM_ASM({
var id = $0.toString() + ':' + UTF8ToString($1) + ':' + $2;
wxRenderedElementRegister(
id,
$0.toString(),
UTF8ToString($1), // elementType
UTF8ToString($3), // subType
UTF8ToString($4), // label
UTF8ToString($5), // tooltip
$6, $7, $8, $9, // x, y, w, h
$10 ? true : false, // enabled
$2, // index
$11 ? true : false // ← checked
);
},
parentId, elementType, index, subType,
label.utf8_str().data(), tooltip.utf8_str().data(),
screenX, screenY, width, height,
enabled, checked);
}
```
**File: `wxwidgets/build/wasm/wx.js`** (helper at line 297)
```javascript
function wxRenderedElementRegister(
id, parentId, elementType, subType,
label, tooltip, screenX, screenY, width, height,
enabled, index, checked) // ← NEW
{
if (window.wxElementRegistry) {
window.wxElementRegistry.registerRendered(id, {
id, parentId, elementType, subType,
label, tooltip,
screenX, screenY, width, height,
centerX: screenX + Math.floor(width / 2),
centerY: screenY + Math.floor(height / 2),
enabled,
index,
checked: !!checked, // ← NEW
lastUpdated: Date.now()
});
}
}
```
Also update `wxRenderedElementUpdate` (same file, around line 321) similarly, so subsequent updates can change `checked`.
### Change 2 — register wxAuiToolBar tools
**File: `wxwidgets/src/aui/auibar.cpp`** (inside `OnPaint`, after the main item-paint loop, before the overflow paint at line ~2501)
```cpp
#ifdef __EMSCRIPTEN__
// Update element registry with toolbar tools (for E2E test automation).
// Runs after every paint so state (enabled/checked, layout) stays current.
extern void WasmRegisterRenderedElement(
wxWindow* parent, const char* elementType, const char* subType,
int index, const wxString& label, const wxString& tooltip,
int screenX, int screenY, int width, int height,
bool enabled, bool checked);
extern void WasmUnregisterRenderedElementsByParent(wxWindow* parent);
WasmUnregisterRenderedElementsByParent(this);
wxPoint screenPos = GetScreenPosition();
for (size_t j = 0, itemCount = m_items.GetCount(); j < itemCount; ++j)
{
wxAuiToolBarItem& item = m_items.Item(j);
if (!item.m_sizerItem)
continue;
if (item.m_kind == wxITEM_SEPARATOR)
continue;
wxRect itemRect = item.m_sizerItem->GetRect();
// Skip items scrolled off the end (match the paint loop's cutoff)
if ((horizontal && itemRect.x + itemRect.width >= last_extent) ||
(!horizontal && itemRect.y + itemRect.height >= last_extent))
continue;
const char* subType = (item.m_kind == wxITEM_CONTROL) ? "control" : "button";
bool isEnabled = !(item.m_state & wxAUI_BUTTON_STATE_DISABLED);
bool isChecked = (item.m_state & wxAUI_BUTTON_STATE_CHECKED) != 0;
WasmRegisterRenderedElement(
this,
"tool",
subType,
static_cast<int>(j),
item.m_label,
item.m_shortHelp,
screenPos.x + itemRect.x,
screenPos.y + itemRect.y,
itemRect.width,
itemRect.height,
isEnabled,
isChecked
);
}
#endif
```
Notes:
- `item.m_label` / `item.m_shortHelp` are the verified field names (`wxwidgets/include/wx/aui/auibar.h:231,235`).
- `wxAUI_BUTTON_STATE_CHECKED` is already how `wxAuiToolBar::OnLeftUp` tracks toggle state (see line 2676 `m_actionItem->m_state & wxAUI_BUTTON_STATE_CHECKED`).
- Placing the block inside OnPaint means every repaint refreshes the registry, which keeps `checked`/`enabled` state synchronized with visible state without needing a separate update path.
### Change 3 — update existing callers to pass `checked`
Every existing `WasmRegisterRenderedElement` call must pass a new final arg. Most don't have meaningful checked state:
- `src/univ/menu.cpp` — pass `false` (or `menuItem->IsChecked()` for check-menu-items, already available)
- `src/aui/framemanager.cpp` (pane parts) — pass `false`
- `src/aui/tabart.cpp` — pass `false` for non-selected tabs, `true` for the active tab (`page.active`)
- `src/univ/textctrl.cpp`, `src/propgrid/propgrid.cpp`, `src/stc/stc.cpp` — pass `false`
- `src/univ/toolbar.cpp` — pass `tool->IsToggled()` (real value for the regular wxToolBar path)
This is a small mechanical change: add `, false` (or the appropriate value) to each existing call site.
---
## Verification
### Build
1. **wxWidgets standalone build** (fast): `./scripts/build-wxuniversal-wasm.sh`
2. **KiCad rebuild** (needed because KiCad statically links wxWidgets; this is the slow step): `./docker/build.sh`
3. **Setup test artifacts**: handled automatically by `npm run test:kicad`'s `setup:kicad` step.
### Run
```bash
cd tests
npm run test:kicad
```
Expectations for `pcbnew.spec.ts`:
- **Test 1** (`click through setup wizard to load PCBnew`): still passes (already passing, unaffected by this change).
- **Test 2** (`select draw lines and draw on the board`): now proceeds past the `findAllRendered` poll. Three possible outcomes:
1. **Passes fully** — tools were simply invisible to the test before; the user's "doesn't select" manual report was a display misreading (likely state changed but they didn't see the visual update, or they tested a stale build).
2. **Fails at the `checked` poll (5s)** — tool renders, click reaches it, but activation path (ACTION_TOOLBAR → TOOL_MANAGER → coroutine) has a real functional bug. Follow up using the log.
3. **Fails at the initial `findAllRendered` poll still** — registration isn't firing; something wrong with the build/binding. Debug by inspecting the generated `pcbnew.js` for the new signature.
### Diagnostic signals in the log
After the click, watch for these patterns:
- `[WASM_FCONTEXT] entry-call ctx=…` new fiber created after click → activation coroutine started. Any subsequent failure is in tool logic, not plumbing.
- No fiber activity at all after the click → click didn't route to ACTION_TOOLBAR. Suspect event routing through the canvas (`wxwidgets/src/wasm/window.cpp` mouse handlers, possibly `kicad/common/gal/webgl/webgl_gal.cpp` which has a WASM-specific uncommitted change).
- Fiber starts but never yields / doesn't hit the tool's `Wait()` loop → similar class of coroutine bug to the Asyncify fix, but different trigger.
### Follow-up scenarios
If the test still fails after this fix, use the above signals to narrow to:
- **Rendering-only**: `Refresh(false); Update()` already runs in `wxAuiToolBar::OnLeftUp` at line 2683–2684, so this is unlikely; but if the registry updates yet the canvas visibly doesn't, something is suppressing paint.
- **Coroutine activation**: new variant of nested-asyncify (maybe menu → tool → dialog nesting). Extend `coroutine-nested` standalone test with the matching scenario.
- **Event routing**: audit the DOM-event → wxWidgets-event bridge. If clicks on coordinates in the canvas aren't reaching wxAuiToolBar, the bridge has regressed.
---
## Files Touched
| File | Change |
|---|---|
| `wxwidgets/src/wasm/window.cpp` | Add `bool checked` param to `WasmRegisterRenderedElement` signature |
| `wxwidgets/build/wasm/wx.js` | Add `checked` arg to `wxRenderedElementRegister` and `wxRenderedElementUpdate` JS helpers |
| `wxwidgets/src/aui/auibar.cpp` | **NEW** registration block in `OnPaint()` (~30 lines in `#ifdef __EMSCRIPTEN__`) |
| `wxwidgets/src/univ/toolbar.cpp` | Pass `tool->IsToggled()` as new final arg |
| `wxwidgets/src/univ/menu.cpp` | Pass `false` (or `IsChecked()` for check items) |
| `wxwidgets/src/aui/framemanager.cpp` | Pass `false` |
| `wxwidgets/src/aui/tabart.cpp` | Pass `page.active` where appropriate, else `false` |
| `wxwidgets/src/univ/textctrl.cpp`, `propgrid/propgrid.cpp`, `stc/stc.cpp` | Pass `false` |
Net: +~50 lines of new code, ~8 files touched. The wxWidgets fork drift grows by one localized patch — no protocol or architectural change.

172
docs/features/fix-asyncify-O2-and-modal-promise-rejection/rtree-debug-findings.md Normal file
View file

@ -0,0 +1,172 @@
# RTree `Classify` duplicate-index crash on PCB load (wasm-only)
## TL;DR
A debug-build pcbnew in the browser used to abort on every `.kicad_pcb` open with
```
Aborted(Assertion failed: !a_parVars->m_taken[a_index],
at: kicad/thirdparty/rtree/geometry/rtree.h:1771, Classify)
```
Native debug builds never hit it. Root cause was a **wasm-only integer overflow** triggered by a single bad template argument in upstream KiCad:
```cpp
// kicad/libs/kimath/src/geometry/shape_poly_set.cpp:1927 ← before
RTree<intptr_t, intptr_t, 2, intptr_t> rtree;
// ^^^^^^^^ ELEMTYPEREAL = intptr_t
```
Every other RTree instantiation in KiCad uses `double` for the 4th template parameter (the volume/area type). This one used `intptr_t`, which on wasm32 is `int32_t` — too narrow to hold `RectSphericalVolume`'s `sumOfSquares` for typical KiCad nanometer extents (`~10⁸ per axis → ~10¹⁴ area`). On native targets `intptr_t = int64_t` happens to absorb the result, so the bug stays latent.
Fix: change `intptr_t` to `double` on that one line. After the fix, both `kicad/demos/microwave/microwave.kicad_pcb` and `kicad/demos/pic_programmer/pic_programmer.kicad_pcb` load and render cleanly through pcbnew's File → Open in the wasm debug build — see `tests/kicad/load-pcb.spec.ts` and the baselines under `tests/baseline-screenshots/load-pcb-*.png`.
---
## Investigation trail
### 1. Reproducing and isolating
The crash reproduced deterministically on every non-empty board (`microwave`, `flat_hierarchy`, `pic_programmer`). The first idea — that the bug had something to do with the file-open path itself — was disproven by the spike test at `tests/kicad/load-pcb.spec.ts`: the menu and dialog drive end-to-end, the abort fires after the file is selected and parsing has begun, well inside `PCB_IO_KICAD_SEXPR::LoadBoard()`.
The static read of `rtree.h` left two plausible mechanisms:
1. `seed0 == seed1` in `PickSeeds` because `if (waste >= worst)` never fires, which only happens if `worst` (= `-coverSplitArea - 1`) goes NaN/`-inf`. That points at integer-overflow / UB inside `RectVolume` / `RectSphericalVolume`.
2. Out-of-band corruption of `m_taken[]` — re-entrant Insert during iteration, stack/heap overrun, etc.
Neither could be decided from code alone.
### 2. Targeted instrumentation
Phase A added a temporary `KICAD_RTREE_DIAG`-gated block in `kicad/thirdparty/rtree/geometry/rtree.h` (since reverted) that, on the assertion's would-fire condition, dumped:
- the call-site tag (`PickSeeds-0` / `PickSeeds-1` / `ChoosePartition`, derived from current `m_count[]`),
- `a_index`, `a_group`, `m_total`, `m_minFill`, `m_count[]`,
- `m_taken[]`, `m_partition[]`,
- `m_coverSplitArea`, `m_area[0..1]`,
- every branch's `m_min/m_max/span` per dimension,
and `return`-ed without aborting. Wired up via `./docker/build.sh --diag rtree` (gated through the existing `--diag=<flags>` plumbing in `scripts/kicad/build-pcbnew.sh`).
### 3. The dump that nailed it
Two consecutive runs from the diagnostic build (excerpt — first run shown):
```
[RTREE-DIAG] Classify duplicate src=PickSeeds-1 idx=0 grp=1
total=9 minFill=4 count=[1,0]
coverSplitArea=-2099823776 area=[717833776,0]
[RTREE-DIAG] m_taken=100000000 m_partition=0 -1 -1 -1 -1 -1 -1 -1 -1
[RTREE-DIAG] branchBuf[0].rect=[67429380..67932300 span=502920]
[117822980..117822980 span=0]
[RTREE-DIAG] branchBuf[1].rect=[69441060..69943980 span=502920]
[117805200..117810280 span=5080]
...
[RTREE-DIAG] branchBuf[8].rect=[82008980..82511900 span=502920]
[117955060..117962680 span=7620]
```
Key observations:
- `src=PickSeeds-1` plus `m_taken[0]=1` and `idx=0` → `seed0 == seed1 == 0`. Exactly the smoking-gun for hypothesis (1).
- Every per-dim `span` is small and positive (`5e5`, `5e3`, etc.). No leaf overflow.
- But `coverSplitArea = -2099823776` — **negative**, which `RectSphericalVolume` cannot algebraically produce (`sumOfSquares * unitSphereVolume` is non-negative).
- The magnitude is interesting too: `-2099823776 ≈ -INT_MAX × 0.98`. That's not a noisy double; that's an exact 32-bit signed-integer value sitting in a double slot.
- `area[0] = 717833776` (run 1) and `-919295316` (run 2) — also "round int32" magnitudes, also wildly wrong vs. the expected `~1.99e11` for the seed-0 branch.
### 4. The actual bug
Every value the diag pulled out of a "double" field had the shape of a 32-bit signed integer. That means the field is being computed and stored as `int32_t` somewhere, not as `double`. The `ELEMTYPEREAL` template parameter is supposed to be the wide floating-point type for volume math.
Grep across the kicad submodule:
```
kicad/include/view/view_rtree.h:36 RTree<VIEW_ITEM*, int, 2, double>
kicad/pcbnew/drc/drc_rtree.h:77 RTree<ITEM_WITH_SHAPE*, int, 2, double>
kicad/pcbnew/connectivity/connectivity_rtree.h:45 RTree<T, int, 3, double>
kicad/pcbnew/connectivity/connectivity_items.h:395 RTree<const SHAPE*, int, 2, double>
kicad/eeschema/sch_rtree.h:42 RTree<SCH_ITEM*, int, 3, double>
kicad/libs/kimath/include/geometry/shape_index.h RTree<T, int, 2, double>
kicad/libs/kimath/src/geometry/shape_poly_set.cpp:1927 RTree<intptr_t, intptr_t, 2, intptr_t> ← OUTLIER
```
Every other instantiation passes `double` for `ELEMTYPEREAL`. `splitCollinearOutlines` (called from `SHAPE_POLY_SET::Simplify`, which fires during any board load that has polygon-bearing items like the microwave demo's RF "footprints" or pic_programmer's pads) passes `intptr_t`.
On wasm32, `intptr_t` is `int32_t` because pointers are 32-bit. `RectSphericalVolume`'s loop:
```cpp
ELEMTYPEREAL sumOfSquares = 0;
for (int index = 0; index < NUMDIMS; ++index) {
ELEMTYPEREAL halfExtent =
((ELEMTYPEREAL) max[index] - (ELEMTYPEREAL) min[index]) * 0.5f;
sumOfSquares += halfExtent * halfExtent;
}
return sumOfSquares * m_unitSphereVolume;
```
…becomes int32 arithmetic. For span_x = 1.5×10⁷, halfExtent² = 5.7×10¹³ — far past `INT_MAX = 2.15×10⁹`. The multiplication wraps, `sumOfSquares` ends up as garbage (the `-2099823776` we observed), `m_coverSplitArea` follows, `worst = -coverSplitArea - 1` ends up hugely positive, and PickSeeds' `if (waste >= worst)` never fires for any pair. `seed0 = seed1 = 0` (their default), `Classify(0, 0)` succeeds, `Classify(0, 1)` trips the assertion.
On native (x86_64, arm64), `intptr_t = int64_t` so the same math succeeds even with the wrong template arg. KiCad's CI has therefore never seen the assertion.
### 5. The fix
```diff
- RTree<intptr_t, intptr_t, 2, intptr_t> rtree;
+ // ELEMTYPEREAL must be a wide floating-point type: RectSphericalVolume's
+ // sumOfSquares grows quadratically with extents, easily exceeding 2^31 for
+ // KiCad nanometer coordinates (~10^8 per axis -> ~10^14 area). All other
+ // RTree instantiations in KiCad use `double` for that fourth argument; this
+ // one was using `intptr_t`, which silently overflowed on wasm32 (where
+ // intptr_t is 32-bit) and tripped an assertion deep in PickSeeds during
+ // PCB load. See features/fix-asyncify-O2-and-modal-promise-rejection/
+ // rtree-debug-findings.md for the full diagnosis trail.
+ RTree<intptr_t, intptr_t, 2, double> rtree;
```
`git -C kicad diff origin/master -- thirdparty/rtree/geometry/rtree.h` is empty — the rtree third-party code stays vanilla upstream. The only divergence from upstream is `libs/kimath/src/geometry/shape_poly_set.cpp:1927`.
### 6. Verification
After the fix:
- `npm run test:kicad:firefox -- load-pcb.spec` → both `microwave` and `pic_programmer` tests pass (~38s total).
- `tests/logs/kicad/load-pcb/*.log` contains the `[KICAD] Wrote …{microwave,pic_programmer}.kicad_pcb` injection lines and no `Aborted(` line, no `[RTREE-DIAG]` line.
- `tests/baseline-screenshots/load-pcb-microwave.png` shows the microwave's two distinctive RF polygon footprints as horizontal red bars on F.Cu, `Pads: 8` in the status bar.
- `tests/baseline-screenshots/load-pcb-pic_programmer.png` shows the pic_programmer's fully-routed multi-IC layout with traces visible across F.Cu.
- `pcbnew.spec.ts` (empty-board path) still passes — no regression.
There remains a separate, pre-existing wasm-port issue downstream: after the board has fully rendered, KiCad's clipboard polling path hits a `RuntimeError: index out of bounds` (and `indirect call to null` on Firefox) inside `__asyncjs__js_clipboardHasText` → `Asyncify.handleSleep`. That's not blocking the load (the screenshot is fully painted by then) and is out of scope for this fix; the load-pcb test explicitly filters its assertion to the two things it cares about — no `[RTREE-DIAG]` and no `Aborted(` — leaving downstream clipboard noise for a follow-up.
---
## Reportable summary for upstream KiCad
Below is a self-contained version suitable for an upstream bug report; pull it as-is.
> **Title:** RTree `ELEMTYPEREAL = intptr_t` in `SHAPE_POLY_SET::splitCollinearOutlines` overflows on 32-bit-pointer targets
>
> **File:** `libs/kimath/src/geometry/shape_poly_set.cpp:1927`
>
> ```cpp
> RTree<intptr_t, intptr_t, 2, intptr_t> rtree;
> ```
>
> The 4th template parameter is `ELEMTYPEREAL`, which `RTree::RectVolume` and `RectSphericalVolume` use for the accumulated volume / sum-of-squares math. Every other `RTree<>` instantiation in KiCad passes `double` for that slot (`view_rtree.h`, `drc_rtree.h`, `connectivity_rtree.h`, `connectivity_items.h`, `eeschema/sch_rtree.h`, `kimath/include/geometry/shape_index.h`). This one passes `intptr_t`.
>
> On 64-bit-pointer hosts `intptr_t == int64_t` and the math fits, so the bug is latent. On 32-bit-pointer targets (wasm32, 32-bit Linux, etc.) `intptr_t == int32_t`, so `RectSphericalVolume`'s `sumOfSquares` overflows for typical KiCad nanometer extents (`~10⁸ per axis → halfExtent² ~5×10¹³`, well past `INT_MAX = 2.15×10⁹`). The wrap turns `m_coverSplitArea` negative, makes `worst = -m_coverSplitArea - 1` huge positive in `PickSeeds`, so `if (waste >= worst)` never fires for any pair. `seed0 == seed1 == 0` survives the loop, `Classify(0, 0)` succeeds, `Classify(0, 1)` trips
>
> ```
> Assertion failed: !a_parVars->m_taken[a_index]
> (thirdparty/rtree/geometry/rtree.h, line 1771)
> ```
>
> in debug builds. Release builds silently store a corrupt tree.
>
> Fix is one character class:
>
> ```diff
> -RTree<intptr_t, intptr_t, 2, intptr_t> rtree;
> +RTree<intptr_t, intptr_t, 2, double> rtree;
> ```
>
> Reproduces on any non-empty board on a wasm32 debug build (we've hit it on the `microwave`, `flat_hierarchy`, and `pic_programmer` demos). Should also reproduce on 32-bit Linux debug builds.

74
docs/features/gerbview/0001-gerbview-port.md Normal file
View file

@ -0,0 +1,74 @@
# Gerber Viewer (gerbview) WASM port — design notes
## Goal
Bring up KiCad's Gerber Viewer (`gerbview`, `FRAME_GERBER`) in the browser to the
"boots, canvas visible, click around" level the other ported apps reached. Scope is
launch-only — loading actual Gerber/drill files is out of scope for now.
## Approach
gerbview is its own standalone program + kiface (unlike symbol_editor, which lived
inside the eeschema kiface), so it follows the **pl_editor/pcbnew pattern** almost
verbatim: gate the native dynamic-kiface logic behind `if( EMSCRIPTEN )` and link the
kiface objects statically into the `gerbview` executable. `gerbview.cpp` (the
`KIFACE_GETTER`) is already part of `gerbview_kiface_objects`, so no source hoisting
was needed (unlike eeschema). There was no `#ifdef __EMSCRIPTEN__` frame stub to
remove (gerbview was never gated out, unlike the symbol editor).
## Changes (kicad submodule)
- **`kicad/gerbview/CMakeLists.txt`** — mirror pl_editor's WASM static-link block:
- On EMSCRIPTEN, compile `common/single_top.cpp` with `TOP_FRAME=FRAME_GERBER`
(no `BUILD_KIWAY_DLL`); wrap the native minimal exe link in `if( NOT EMSCRIPTEN )`.
- Hoist the kiface deps into `GERBVIEW_KIFACE_LIBRARIES`; on EMSCRIPTEN link them
directly into the `gerbview` exe with `LINKER:--allow-multiple-definition`.
- Gate `gerbview.cpp` defs: EMSCRIPTEN → `COMPILING_DLL` (no `BUILD_KIWAY_DLL`, so
`KIFACE_GETTER` links statically); else `BUILD_KIWAY_DLL;COMPILING_DLL`.
- **`kicad/gerbview/navlib/CMakeLists.txt`** — add an `if( EMSCRIPTEN )` branch that
builds `gerbview_navlib` from the WASM stub instead of the real 3Dconnexion plugin
(no SpaceMouse driver in the browser). The frame's navlib member uses
`NL_GERBVIEW_PLUGIN` under WASM (`#ifndef __linux__`; emscripten doesn't define it).
- **`#include <wx/choice.h>`** added to three files that use `wxChoice` (the
Cmp/Net/Attr/DCode aux-toolbar combo boxes) but only had the forward declaration:
`gerbview/events_called_functions.cpp`, `gerbview/toolbars_gerber.cpp`,
`gerbview/tools/gerbview_control.cpp`. Native builds pull `wx/choice.h` transitively;
the WASM wxWidgets header config does not, so these failed with "member access into
incomplete type 'wxChoice'". Include-what-you-use fix — behavior-neutral, upstream-safe.
(`gerbview_frame.cpp` already gets it transitively; the generated `_base.cpp` carries
its own includes — both left untouched to keep the fork minimal.)
## Changes (root repo)
- **`wasm/stubs/nl_gerbview_plugin_stub.cpp`** (NEW) — no-op `NL_GERBVIEW_PLUGIN`
ctor/dtor + `SetCanvas`/`SetFocus`, mirroring `nl_pl_editor_plugin_stub.cpp`.
- **`scripts/kicad/build-gerbview.sh`** (NEW) — thin wrapper → `build-kicad-target.sh gerbview`.
- **`scripts/kicad/build-kicad-target.sh`** — add `gerbview` to the `pcbnew|eeschema)`
case arm (target = subdir = `gerbview`); update usage strings.
- **`docker/build.sh`** — add `gerbview` to valid apps, dispatch case, and the `all`
loop (now 6 apps).
- **`tests/scripts/setup-kicad-wasm.sh`** — `copy_app gerbview`.
- **`tests/apps/kicad/gerbview.html`** (NEW) — browser shell (copy of pl_editor.html;
title, `thisProgram=/usr/bin/gerbview`, `gerbview.js`).
- **`tests/kicad/gerbview.spec.ts`** (NEW) + **`tests/package.json`** — launch-only
smoke test (wizard, canvas visible, registry populated, ≥1 toolbar, no abort).
## Build & verify
```
./docker/build.sh gerbview # seed fresh-branch cache from main first (see build-quirks memory)
cd tests && npm run setup:kicad && npm run test:gerbview
```
Expect: the viewer opens — menu bar, top + aux toolbars (with the Cmp/Net/Attr/DCode
combos), left tool toolbar, dark gerber canvas with grid + origin crosshair, and the
Layers/Items manager pane. `gerbview.spec.ts` passes (2/2, no abort).
## Known limitations
- No Gerber/drill files are loaded; the canvas is empty until a file is opened
(file loading untested / out of scope).
- Symbol-editor-style drawing tools that require an open document behave per native
KiCad (some are inactive with no layers loaded).
- No persistent storage (MEMFS only).
</content>

54
docs/features/pl-editor/0001-pl-editor-port.md Normal file
View file

@ -0,0 +1,54 @@
# pl_editor (drawing-sheet editor) WASM port — design notes
## Goal
Bring up KiCad's `pagelayout_editor` sub-app (also known as `pl_editor`, the drawing-sheet editor) in the browser, to the same "boots, canvas visible, partially usable in-session" level as `pcbnew`. Persistence across sessions not required.
## Approach
Mirrors the in-tree pattern that pcbnew/calculator/eeschema use: gate WASM differences behind `if( EMSCRIPTEN )` blocks inside the upstream `pagelayout_editor/CMakeLists.txt`, keeping a single source of truth for the build alongside KiCad's existing platform conditionals (`if( WIN32 )`, `if( APPLE )`).
(An earlier iteration tried an out-of-tree CMake wrapper to keep the kicad submodule bit-for-bit upstream. It worked, but diverged from the team norm — every other WASM-ported app modifies kicad. We converged to the team pattern; the only kicad-side cost is a ~80-line patch in this app's CMakeLists.txt, all WASM-conditional.)
## Changes (see kicad.patch + root.patch + wxwidgets.patch)
### kicad submodule
- **`kicad/pagelayout_editor/CMakeLists.txt`** — mirrors pcbnew's WASM static-linking pattern:
- Drop `BUILD_KIWAY_DLL` from `single_top.cpp` and `pl_editor.cpp` compile defs on EMSCRIPTEN (browser can't `dlopen` a `.kiface` shared library).
- Split `pl_editor_kiface` into an OBJECT library (`pl_editor_kiface_objects`) + an empty MODULE; lets the same compiled objects be linked statically into the exe on WASM and dynamically into the `.kiface` module on native.
- On EMSCRIPTEN, link `pl_editor` directly against `PL_EDITOR_KIFACE_LIBRARIES` with `LINKER:--allow-multiple-definition` (handles wxWidgets/nanosvg duplicate symbols, same as pcbnew).
- **`kicad/pagelayout_editor/navlib/CMakeLists.txt`** — for EMSCRIPTEN, replace the real 3Dconnexion SpaceMouse plugin sources with `wasm/stubs/nl_pl_editor_plugin_stub.cpp` (no USB hardware in the browser).
### Root repo
- **`wasm/stubs/nl_pl_editor_plugin_stub.cpp`** — no-op `NL_PL_EDITOR_PLUGIN` ctor/dtor + `SetCanvas`/`SetFocus`, mirroring `nl_pcbnew_plugin_stub.cpp`.
- **`scripts/kicad/build-pl_editor.sh`** — thin wrapper around `build-kicad-target.sh pl_editor`.
- **`scripts/kicad/build-kicad-target.sh`** — adds `pl_editor` to the `case` (uses upstream target name `pl_editor`, source subdir `pagelayout_editor`).
- **`docker/build.sh`** — adds `pl_editor` to the unified app dispatch (valid apps + `all` loop + `kicad_subdir_for`).
- **`tests/apps/kicad/pl_editor.html`** — browser shell; `preRun` creates `/home/kicad` and `FS.chdir` there so file dialogs land somewhere friendly instead of MEMFS root.
- **`tests/scripts/setup-kicad-wasm.sh`** — `copy_app pl_editor` added to the existing list.
- **`tests/e2e/filedialog-folder-nav.spec.ts`** — regression test for the wxFileDialog folder-navigation fix.
### wxwidgets submodule (file dialog usability fixes)
These were discovered while bringing up pl_editor's file dialog but apply to any wxWidgets-WASM app:
- **`wxwidgets/src/generic/filedlgg.cpp`** — `wxGenericFileDialog::OnOk` now navigates into the selected entry when it's a directory instead of closing the dialog and surfacing the folder path to the caller as if it were a file. Without this, KiCad's "Open Drawing Sheet" produced "Unable to load /dev file" when the user selected `/dev` (a directory in MEMFS).
- **`wxwidgets/src/wasm/mouse.cpp`** — stateful double-click detection. `EmscriptenMouseEvent` has no click-count field, so `wxEVT_LEFT_DCLICK` literally never fired in the WASM build — breaking `EVT_LIST_ITEM_ACTIVATED` on every listctrl. Now two MOUSEDOWNs of the same button within 500ms emit DCLICK.
## Build & verify
```
./docker/build.sh pl_editor
./tests/scripts/setup-kicad-wasm.sh
# serve tests/apps/kicad and open pl_editor.html
```
Expect: window opens, canvas renders, File > Open / Save As dialogs work (folder navigation via single-click + Enter, single-click + OK, or double-click). Dialog lands at `/home/kicad` by default.
## Known limitations
- No persistent storage (MEMFS only); files vanish on tab close.
- No keyboard accelerator for "navigate to parent directory" beyond the up-arrow button + ".." entry.
- Drawing-sheet-specific tooling beyond basic edit/save is untested (out of MVP scope).

54
docs/features/schematic/0001-eeschema-iface-stubs.md Normal file
View file

@ -0,0 +1,54 @@
# 0001 — eeschema IFACE sub-frame stubs (WASM)
## Why
Eeschema's `IFACE::CreateKiWindow` in [kicad/eeschema/eeschema.cpp](../../kicad/eeschema/eeschema.cpp) dispatches `FRAME_T` ids to four frame constructors:
- `FRAME_SCH` → `SCH_EDIT_FRAME` (the schematic editor — what we want)
- `FRAME_SCH_SYMBOL_EDITOR` → `SYMBOL_EDIT_FRAME` (symbol library editor)
- `FRAME_SCH_VIEWER` → `SYMBOL_VIEWER_FRAME` (symbol library viewer)
- `FRAME_SYMBOL_CHOOSER` → `SYMBOL_CHOOSER_FRAME` (symbol chooser dialog)
- `FRAME_SIMULATOR` → `SIMULATOR_FRAME` (already wrapped in try/catch — no patch needed)
The MVP scope for the WASM port is "empty schematic editor + draw a wire", so the three sub-frames (symbol editor / viewer / chooser) are out of scope. They are non-trivial to support in the browser — they need bundled symbol libraries, FS access for `.kicad_sym` lookup, and full dialog plumbing.
Compiling them out at the source level (removing `EESCHEMA_LIBEDIT_SRCS` from the build) cascades into many CMakeLists.txt and symbol-chooser sources; cleaner to keep the sources compiled and just refuse to instantiate the frames at the IFACE switch.
## What changed
`kicad/eeschema/eeschema.cpp`, the three sub-frame cases inside `IFACE::CreateKiWindow` are now guarded:
```cpp
case FRAME_SCH_SYMBOL_EDITOR:
#ifdef __EMSCRIPTEN__
return nullptr;
#else
return new SYMBOL_EDIT_FRAME( aKiway, aParent );
#endif
case FRAME_SCH_VIEWER:
#ifdef __EMSCRIPTEN__
return nullptr;
#else
return new SYMBOL_VIEWER_FRAME( aKiway, aParent );
#endif
case FRAME_SYMBOL_CHOOSER:
{
#ifdef __EMSCRIPTEN__
return nullptr;
#else
// … existing body …
#endif
}
```
`FRAME_SIMULATOR` is untouched — its existing `try/catch (SIMULATOR_INIT_ERR&)` block catches the ngspice init failure that our header stub eventually triggers, and returns nullptr the same way.
## How to apply
Captured under `features/schematic/kicad.patch` via `./scripts/create-feature-patches.sh schematic` once the build is green.
## Tested by
`tests/kicad/eeschema.spec.ts` — wizard completion + `Draw Wires` tool test. Neither test path exercises the stubbed-out frames.

85
docs/features/symbol-editor/0001-symbol-editor-port.md Normal file
View file

@ -0,0 +1,85 @@
# Symbol Editor WASM port — design notes
## Goal
Bring up KiCad's Symbol Editor (`FRAME_SCH_SYMBOL_EDITOR`, the `.kicad_sym`
library editor) in the browser, to the same "boots, canvas visible, click
around" level as the other ported apps. Scope is launch-only — library
load/save and the symbol viewer/chooser sub-frames are out of scope for now.
## Key insight
Unlike pcbnew / pl_editor, the symbol editor is **not a separate program**. It is
served by the **eeschema kiface**: its sources already compile into
`eeschema_kiface_objects` (`EESCHEMA_LIBEDIT_SRCS` + the `symbol_editor_*` tools in
`kicad/eeschema/CMakeLists.txt`). KiCad's universal launcher `common/single_top.cpp`
opens whichever frame the compile-time `TOP_FRAME` macro names.
So the port is just **a second launcher executable (`symbol_editor`) that links the
same eeschema kiface but compiles `single_top.cpp` with
`TOP_FRAME=FRAME_SCH_SYMBOL_EDITOR`** — no new sources, no new kiface, no extracting
symbol-editor code. The eeschema kiface (deps, navlib, stubs, libraries) is reused
verbatim.
## Changes
### kicad submodule (2 files)
- **`kicad/eeschema/CMakeLists.txt`** — a WASM-only (`if( EMSCRIPTEN )`) block adds the
`symbol_editor` executable. Because `single_top.cpp`'s `COMPILE_DEFINITIONS` are
directory-scoped (already pinned to `TOP_FRAME=FRAME_SCH` for the `eeschema` exe),
we `configure_file`-copy it to a private TU (`symbol_editor_single_top.cpp`) and set
that copy's `TOP_FRAME=FRAME_SCH_SYMBOL_EDITOR;PGM_DATA_FILE_EXT="kicad_sym"`. The exe
links `EESCHEMA_KIFACE_LIBRARIES` directly with `LINKER:--allow-multiple-definition`,
mirroring the eeschema/pcbnew static-link pattern.
- **`kicad/eeschema/eeschema.cpp`** — `IFACE::CreateKiWindow`'s `FRAME_SCH_SYMBOL_EDITOR`
case was stubbed to `return nullptr` on `__EMSCRIPTEN__` during the eeschema MVP
(see `../schematic/0001-eeschema-iface-stubs.md`). That stub is now removed so
the frame is constructed in WASM like the native build. This was THE blocker: with the
stub, `Kiway.Player(FRAME_SCH_SYMBOL_EDITOR)` returned null, `single_top`'s `OnInit`
bailed, and the app sat idle with a blank canvas (no abort, no error).
The symbol viewer (`FRAME_SCH_VIEWER`) and chooser (`FRAME_SYMBOL_CHOOSER`) remain
stubbed — out of scope, and the chooser needs bundled libraries we don't ship.
### Root repo
- **`scripts/kicad/build-symbol_editor.sh`** — thin wrapper around
`build-kicad-target.sh symbol_editor`.
- **`scripts/kicad/build-kicad-target.sh`** — adds `symbol_editor` to the `case`. CMake
target is `symbol_editor` but its build subdir is `eeschema` (it's part of that
kiface), so a `KICAD_SUBDIR` variable now distinguishes target name from subdir for
the output-path log and embind include.
- **`docker/build.sh`** — adds `symbol_editor` to valid apps, the `all` loop, and
`kicad_subdir_for` (`symbol_editor → eeschema`); artifacts land at
`build-wasm/kicad-symbol_editor/eeschema/symbol_editor.{js,wasm}`.
- **`tests/apps/kicad/symbol_editor.html`** — browser shell (copy of eeschema.html with
title + `thisProgram=/usr/bin/symbol_editor` + `symbol_editor.js`).
- **`tests/scripts/setup-kicad-wasm.sh`** — `copy_app symbol_editor` + subdir map entry.
- **`tests/kicad/symbol_editor.spec.ts`** + **`tests/package.json`** — launch-scope smoke
test (canvas visible, registry populated, toolbars present, no WASM abort), mirroring
eeschema's wizard-aware flow.
### wxwidgets submodule
No changes needed — the file-dialog and double-click fixes landed with the pl_editor port.
## Build & verify
```
./docker/build.sh symbol_editor
cd tests && npm run setup:kicad && npm run test:symbol_editor
# or serve tests/apps/kicad and open symbol_editor.html
```
Expect: the symbol editor window opens — menu bar, top + left + right toolbars
(incl. pin/rect/circle/line drawing tools), the symbol library tree pane with the
filter box, the gridded canvas with the symbol-origin crosshair, and a status bar.
## Known limitations
- No bundled symbol libraries, so the library tree is empty (`SyncLibraries` reports
`libCount=0`). Opening/creating/saving `.kicad_sym` files is untested (out of scope).
- Symbol viewer and symbol chooser frames are still stubbed out for WASM.
- No persistent storage (MEMFS only).
</content>

323
docs/features/web-init/0001-web-app-spec.md Normal file
View file

@ -0,0 +1,323 @@
# 0001 — KiCad-WASM Web App Spec
Status: **Refined spec, ready for `implement plan`**
Branch: `feature/web-init`
Scope of this iteration: **create a project, open a project, upload files, open a file in a WASM tool via URL.**
This document is the agreed design after a clarification pass. Decisions that were
explicitly chosen by the user are marked **[decided]**. Items intentionally pushed to a
later iteration are marked **[later]**. Sensible defaults that were *not* explicitly
discussed are marked **[default]** and are safe to change during planning.
---
## 1. Goal
A single web application that lets a user create/open KiCad projects, upload files into
them, and open a file in the matching WASM tool (pcbnew / eeschema / calculator) by
visiting a URL such as:
```
/p/project5/pcbnew/nyak.kicad_pcb
```
The WASM apps already exist as `<tool>.js` + `<tool>.wasm` pairs (see `output/` and
`tests/apps/kicad/`). They boot into an Emscripten harness and read files from MEMFS.
This web app wraps that: it manages projects + files server-side, and on a tool URL it
boots the right WASM app and feeds it the project's files via `FS.writeFile`, then drives
File→Open on the target file.
Non-goals this iteration: editing/saving back, collaboration, auth/login.
---
## 2. Key decisions (summary table)
| Area | Decision |
|---|---|
| Frontend | React + TypeScript + Vite + shadcn/ui **[decided]** |
| Backend | Fastify + ts-rest + Zod **[decided]** |
| Shared types | `packages/contract` (ts-rest contract + Zod) imported by FE client & BE router **[decided]** |
| Realtime | None now; pick stack that allows Hocuspocus/WSS later, no WS endpoints yet **[decided]** |
| URL semantics | `/p/:project/:tool/*filepath` — `tool` selects the WASM app, `*filepath` is auto-opened **[decided]** |
| Auth/tenancy | No auth now, but data model namespaced by an owner id for later multi-user **[decided]** |
| Metadata store | Postgres now, accessed via Drizzle (drizzle-zod shares schemas) **[decided]** |
| File blob storage | Pluggable `FileStorage` interface; local-disk impl now, S3 later **[decided]** |
| Open behavior | Sync **whole project tree** into MEMFS, then auto-open target **[decided]**; lazy/partial load **[later]** |
| Upload | Individual files (multi), folder (preserve structure), and `.zip` of a project **[decided]** |
| WASM artifact delivery | Served from a **configurable base location** (URL/dir): local Fastify static from `output/` in dev, public S3 URL in prod **[decided]** |
| Save-back / sync | Read-only open now; write interface defined but unused. Save-back + lazy load land together **[later]** |
| Monorepo | pnpm + turbo workspace under `web/` **[decided]** |
---
## 3. Repository / monorepo layout **[decided: under `web/`]**
```
web/
├── package.json # pnpm workspace root
├── pnpm-workspace.yaml
├── turbo.json
├── .env.example
├── docker-compose.yml # local Postgres (and later: minio for S3 parity)
├── apps/
│ ├── frontend/ # Vite + React + TS + shadcn
│ └── server/ # Fastify + ts-rest + Drizzle
└── packages/
├── contract/ # ts-rest contract + Zod schemas (shared)
├── storage/ # FileStorage interface + local-disk impl (+ S3 later)
└── config/ # shared tsconfig / eslint / env parsing [default]
```
Rationale: isolates the JS/TS app from the C++/WASM build repo at root (`kicad/`,
`wxwidgets/`, `scripts/`, `docker/`). The web app consumes WASM artifacts produced by the
existing build, it does not build them.
Root `.gitignore` should ignore `web/**/node_modules`, `web/**/dist`, build caches.
---
## 4. Domain model
### 4.1 Entities (Postgres, via Drizzle) **[decided: Postgres + Drizzle]**
```
owner -- namespace for "no auth now, multi-user later"
id uuid pk
slug text unique -- e.g. "default" now; becomes real users later
created_at timestamptz
project
id uuid pk
owner_id uuid fk -> owner.id
slug text -- URL segment, unique within owner (e.g. "project5")
name text -- human display name
created_at timestamptz
updated_at timestamptz
unique(owner_id, slug)
project_file -- index of files; bytes live in FileStorage
id uuid pk
project_id uuid fk -> project.id
path text -- POSIX-relative within project, e.g. "pcbnew/nyak.kicad_pcb"
size bigint
content_type text
storage_key text -- opaque key handed to FileStorage
created_at timestamptz
updated_at timestamptz
unique(project_id, path)
```
- **Owner namespace [decided]**: every project belongs to an `owner`. This iteration uses a
single seeded owner (`slug = "default"`); the URL omits owner (`/p/:project/...`) and the
server resolves it to the default owner. Adding real auth later = populate `owner` per
user and prefix routes, **no schema migration needed**.
- `project_file.path` is the canonical project-relative path. The `storage_key` decouples
the logical path from however the blob backend names things (so renames/S3 layout are free).
drizzle-zod derives Zod schemas from these tables; those Zod schemas feed the ts-rest
contract so DB ↔ API ↔ client share one source of truth.
### 4.2 What "project" means at the byte level
A project is a directory tree of files (`.kicad_pro`, `.kicad_pcb`, `.kicad_sch`,
`fp-lib-table`, `sym-lib-table`, footprint/symbol lib dirs, etc.). `project_file` rows
enumerate the tree; bytes live behind `FileStorage`.
---
## 5. Storage abstraction **[decided: pluggable, local now, S3 later]**
`packages/storage` exposes a single interface. The whole iteration is **read-heavy**; write
methods exist so save-back **[later]** needs no redesign.
```ts
export interface FileStorage {
// read path
exists(key: string): Promise<boolean>;
read(key: string): Promise<Uint8Array>;
createReadStream(key: string): NodeJS.ReadableStream; // for large files
stat(key: string): Promise<{ size: number; contentType?: string }>;
list(prefix: string): Promise<string[]>; // keys under a prefix
// write path (used now only by upload; save-back is [later])
write(key: string, data: Uint8Array | NodeJS.ReadableStream, opts?: { contentType?: string }): Promise<void>;
delete(key: string): Promise<void>;
}
```
Implementations:
- `LocalDiskStorage` **[now]** — rooted at a configurable dir (`STORAGE_ROOT`), `key` maps
to a path under it. Streams to/from disk.
- `S3Storage` **[later]** — same interface over an S3-compatible bucket. `docker-compose`
can run MinIO for local S3 parity when we get there.
Storage key scheme **[default]**: `owners/<owner_id>/projects/<project_id>/<project_file.path>`.
Opaque to callers — only `FileStorage` interprets it.
---
## 6. WASM artifact delivery **[decided: configurable base location]**
The big artifacts (`pcbnew.wasm` ~350 MB, `eeschema.wasm` ~180 MB, `calculator.wasm`,
their `.js` glue, `wx.js`, `images.tar.gz`) are **app binaries, not user data** — kept
separate from `FileStorage`.
- The frontend resolves every artifact URL from a single configurable base:
`WASM_ASSET_BASE_URL` **[decided requirement]**.
- **dev**: points at the Fastify server, which serves the artifacts statically from a
configurable dir (default `../../output` relative to the server, i.e. repo `output/`).
- **prod**: points at a public S3/CDN URL. No code change — just env.
- An artifact URL is composed as `${WASM_ASSET_BASE_URL}/${tool}.js` (and the glue then
fetches the sibling `.wasm` / `worker.js` / `images.tar.gz` from the same base). The
Emscripten `locateFile` hook must be wired to this base so `.wasm`/`.worker.js` resolve
correctly regardless of origin.
- **Cross-origin caveat [important]**: KiCad WASM uses threads (`.worker.js` present),
which needs `SharedArrayBuffer` → the **document** must be served with
`Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp`.
When artifacts come from a different origin (S3/CDN), they must be served with
`Cross-Origin-Resource-Policy: cross-origin` (or CORS) so they load under COEP. The
Fastify static route sets COOP/COEP/CORP in dev; the prod bucket/CDN must set CORP/CORS.
Verify against the existing harness behavior in `tests/apps/kicad/`.
Artifacts are **not committed to git** (they're build outputs). The build pipeline
(`docker/build.sh` → `output/`) remains the source.
---
## 7. URL & routing **[decided]**
Frontend routes (client-side router) **[default: react-router]**:
| Route | View |
|---|---|
| `/` | Project list + "Create project" |
| `/p/:project` | Project detail: file tree, upload, "open in tool" actions |
| `/p/:project/:tool/*filepath` | **Tool view**: boots `:tool` WASM app, auto-opens `*filepath` |
- `:tool` ∈ `{ pcbnew, eeschema, calculator }` — selects the WASM app **[decided]**.
(`calculator` takes no file; opening it ignores `*filepath`.)
- `*filepath` is the project-relative path of the file to auto-open
(e.g. `pcbnew/nyak.kicad_pcb`). It must match a `project_file.path` row.
- `:project` is the project **slug** within the default owner.
- Owner is implicit (default owner) now; route gains an owner segment when auth lands
**[later]** — `/p/:project/...` is forward-compatible.
Tool→file-extension mapping (for validation / "open with" UI) **[default]**:
`.kicad_pcb → pcbnew`, `.kicad_sch → eeschema`. Mismatches surface a warning but the
explicit `:tool` segment wins (per the decided semantics).
---
## 8. API (ts-rest contract in `packages/contract`) **[decided: ts-rest + Zod]**
All endpoints under `/api`. Contract is the single typed source; Fastify router
implements it, frontend uses the generated ts-rest react-query client **[default]**.
```
GET /api/projects -> Project[]
POST /api/projects { name, slug? } -> Project # create [scope]
GET /api/projects/:project -> Project + file tree # open [scope]
DELETE /api/projects/:project -> 204 # [default, nice-to-have]
GET /api/projects/:project/files -> ProjectFile[]
GET /api/projects/:project/files/*path -> file bytes (streamed) # used to fill MEMFS
POST /api/projects/:project/files (multipart) -> ProjectFile[] # upload [scope]
POST /api/projects/:project/files/zip (multipart zip) -> ProjectFile[] # upload-zip [scope]
# write/rename/delete of individual files: interface ready, [later] for save-back
```
Upload handling **[decided: files + folder + zip]**:
- **Individual files (multi)**: multipart; each part carries its target project-relative
path. Streamed to `FileStorage`, one `project_file` row each.
- **Folder (preserve structure)**: frontend uses `webkitdirectory`; relative paths derived
from `file.webkitRelativePath` and sent as the per-file path. Server preserves the tree.
- **Zip**: server unpacks (streaming unzip) into the project tree, creating `project_file`
rows per entry. Reject path traversal (`../`) and absolute paths.
Validation: Zod schemas (shared) validate bodies/params; Fastify JSON-schema serialization
for responses. Errors via a consistent ts-rest error shape **[default]**.
---
## 9. Open-a-file flow (the core of the iteration)
Visiting `/p/:project/:tool/*filepath`:
1. Frontend fetches the project's file tree: `GET /api/projects/:project/files`.
2. Frontend loads the WASM glue for `:tool` from `WASM_ASSET_BASE_URL` and instantiates the
Emscripten module into a canvas-bearing harness (reuse the proven shell from
`tests/apps/kicad/pcbnew.html` — `createCanvas`, `images.tar.gz` prefetch+write,
`locateFile`, status/progress UI), ported into a React component.
3. **Sync whole project tree into MEMFS** **[decided]**: for every `project_file`, fetch its
bytes (`GET .../files/*path`) and `FS.mkdirTree` + `FS.writeFile` at the project root
inside MEMFS (mirroring `tests/kicad/utils/fs-inject.ts`). Files land at the path KiCad
expects (e.g. under the default projects dir, confirmed by `load-pcb-probe.spec.ts`:
`/home/kicad/documents/kicad/9.99/projects/...`). The exact MEMFS mount point for an
arbitrary user project is an **open implementation detail** — see §11.
4. Drive File→Open on `*filepath` using the existing element-tracker / menu-driving
helpers (`tests/e2e/utils/element-tracker.ts`, `tests/kicad/load-pcb.spec.ts`). This UI
automation already works for the demo boards and is the reference implementation.
5. Render. Read-only — no write-back **[decided]**.
> **[later]** Lazy/partial loading: instead of syncing the whole tree up front, intercept
> MEMFS reads and fetch siblings on demand. This lands **together with save-back/sync**, as
> a single coherent iteration (both need MEMFS↔storage plumbing). Not now.
---
## 10. Frontend detail **[shadcn decided; rest default]**
- Vite + React + TS, shadcn/ui components, Tailwind.
- Pages: project list (cards + create dialog), project detail (file tree + upload
dropzone + per-file "open in pcbnew/eeschema" buttons), tool view (full-viewport WASM
canvas + status overlay).
- Data layer: ts-rest react-query client generated from the contract.
- Upload UX: drag-drop dropzone supporting multi-file, folder (`webkitdirectory`), and
`.zip`; progress per file; streamed to backend.
- The WASM tool view is a dedicated component that owns the Emscripten lifecycle and tears
it down on unmount (WebGL context, MEMFS) to allow switching tools/projects.
---
## 11. Open questions for `implement plan` (not blocking this spec)
1. **MEMFS mount point for arbitrary projects.** Demos rely on KiCad's default projects
path. For a user project we must decide where in MEMFS the tree is written and whether
pcbnew/eeschema need it under their expected projects dir, or whether File→Open can
target an arbitrary MEMFS path. Resolve by probing (extend `load-pcb-probe`).
2. **Driving File→Open generically.** Current helpers are tuned to the demo dialog flow
(filelist bbox click + filename input + Enter). Confirm it generalizes to arbitrary
paths, or expose a cleaner embind "open file" entry point in the WASM layer.
3. **eeschema/calculator open flows.** Mirror the pcbnew flow; verify eeschema's File→Open
and that calculator (no file) just boots.
4. **Large-tree sync performance.** Whole-tree sync of a big project over many HTTP
requests may be slow; consider a single tar/zip stream endpoint to fill MEMFS in one
shot (still "sync whole tree", just one request). Decide in planning.
5. **COOP/COEP in prod** with cross-origin S3 artifacts — validate header matrix.
6. **Project slug generation/collision** rules; reserved tool names as slugs.
---
## 12. Out of scope this iteration (explicit)
- Auth / login / real multi-user (data model is ready; UI/enforcement is **[later]**).
- WebSocket / realtime / Hocuspocus collaboration (stack chosen to allow it; none built).
- Saving or syncing edits back to storage (**[later]**, paired with lazy load).
- S3 storage implementation (interface ready; **[later]**).
- Editing project files in the browser outside the WASM tools.
---
## 13. Definition of done (this iteration)
- `pnpm install && docker-compose up -d && pnpm dev` in `web/` brings up Postgres,
Fastify (`/api` + WASM static), and the Vite frontend.
- Create a project from the UI; it appears in the list and in Postgres.
- Open the project; upload files via multi-select, folder, and zip — files appear in the
tree and in `FileStorage` + `project_file`.
- Navigate to `/p/<slug>/pcbnew/<path>.kicad_pcb`; the board renders read-only, equivalent
to the existing `tests/kicad/load-pcb` result, sourcing files from project storage.
- Same for an eeschema `.kicad_sch` file.
- Swapping `WASM_ASSET_BASE_URL` between local `output/` and a remote URL requires no code
change.

96
docs/features/web-init/0002-url-regex-modal-followup.md Normal file
View file

@ -0,0 +1,96 @@
# Next-session prompt: fix the URL-detection wxRegEx modal on schematic load (WASM)
Paste everything below the line into a fresh session to work on this.
---
Fix the "Invalid regular expression … UTF-8 error" modal that pops up when eeschema
(WASM) renders a schematic containing text/symbol fields. Branch: `feature/web-init`.
## Symptom
Loading a schematic that has symbol fields / text (e.g.
`kicad/demos/ecc83/ecc83-pp.kicad_sch`) renders the schematic but immediately throws a
**modal dialog**:
```
KiCad Schematic Editor Error
Invalid regular expression '(https?|ftp|file)://([-\w+&@#/%?=~_|!:,.;]*[^.,;<>\s<><73><EFBFBD><EFBFBD><EFBFBD><EFBFBD>])':
UTF-8 error: code points 0xd800-0xdfff are not defined
```
(The `<60><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>` are mojibake where `¶` U+00B6 should be.) It is **non-fatal** — the schematic
draws behind the dialog — but it blocks the UI and means URL detection in text is broken.
An empty / geometry-only schematic does NOT trigger it (that's why the load regression test
`tests/kicad/eeschema-load.spec.ts` uses wires/junctions only).
## Root cause (already localized)
`kicad/common/string_utils.cpp` has two static `wxRegEx` whose pattern ends in a negated
character class containing `¶` (¶, U+00B6):
- `LinkifyHTML()` ~line 672: `wxS( "\\b(https?|ftp|file)://([-\\w+&@#/%?=~|!:,.;]*[^.,:;<>\\(\\)\\s¶])" )`
- `IsURL()` ~line 683: `wxS( "(https?|ftp|file)://([-\\w+&@#/%?=~|!:,.;]*[^.,:;<>\\s¶])" )`
`IsURL()` is called during field/text rendering — `kicad/eeschema/sch_field.cpp:1042` &
`:1081`, `kicad/eeschema/sch_textbox.cpp:352`, `kicad/eeschema/fields_data_model.cpp:377` —
so any schematic with symbol reference/value fields hits it. The error is a **regex
COMPILE** failure of the static pattern (input-independent): the first `IsURL()` call
constructs the static `wxRegEx`, which fails to compile.
Hypothesis: under the emscripten/wxWidgets-WASM build the `¶` in the `wxS(...)` pattern
is mis-encoded (the dialog shows it as invalid UTF-8 in the 0xd800–0xdfff surrogate range),
so `wxRegEx` rejects the pattern. Native KiCad compiles it fine, so this is WASM-specific —
likely the wide-char literal handling and/or the `wxString → wxRegEx` UTF-8 conversion in
the wx-WASM port. NOT yet root-caused to the exact byte; that's step 1.
## What to do
1. Reproduce + pin the exact corruption. Add temporary logging that prints the pattern
bytes (hex) right before the `wxRegEx` ctor in `IsURL()`/`LinkifyHTML()`, and compare
what `¶` becomes in the WASM build vs. what it should be (`0xC2 0xB6`). Determine
whether the corruption is at: the C++ wide/narrow literal, the `wxS`/`wxString` storage,
or the `wxRegEx` UTF-8 conversion (`src/common/regex.cpp` / the wx-WASM regex backend).
2. Fix at the lowest correct layer. Prefer the wx-WASM layer (project rule: keep `kicad/`
close to upstream, fix under `__EMSCRIPTEN__` in `wxwidgets/`). Candidate fixes, cheapest
first — validate which is actually right after step 1:
- If the `wxRegEx` UTF-8 conversion is the bug, fix it in the wx-WASM regex backend so
non-ASCII pattern code points (e.g. `¶`) round-trip.
- If it's the literal/encoding, build the pattern via `wxString::FromUTF8("…\xC2\xB6…")`
instead of `¶`, or otherwise ensure correct encoding. (If this has to live in
`string_utils.cpp`, guard it `#ifdef __EMSCRIPTEN__` and keep the upstream pattern for
native — minimal divergence; run `scripts/kicad-diff-stats.sh` after.)
- Last resort: drop `¶` from the WASM pattern (it only excludes the pilcrow from a
URL's trailing char — cosmetic). Note this in a comment if chosen.
3. Verify: load a text-bearing schematic and confirm NO modal, schematic renders, and a real
URL in a text field is still linkified (don't regress URL detection). Then re-run
`tests/kicad/eeschema-load.spec.ts` (must stay green) and ideally add a text-bearing
schematic case that would have shown the modal.
## Context you need
- The schematic LOAD path now works: the fiber/Asyncify hang was fixed by the trampoline
self-heal shim in `scripts/common/inject-dyncall-shims.sh` (section "3c"). Don't touch it.
- Build eeschema (reuses prebuilt deps volume, ~5 min, don't build deps from scratch, don't
run two builders at once — 32G each on a 37G Docker VM OOMs):
`COMPOSE_PROJECT_NAME=kicad-wasm-feature-schematic ./docker/build.sh eeschema --debug`
Then `cd web/apps/frontend && npm run link-wasm`. Build scripts log to files
(`logs/build/*.log`) — don't pipe them.
- wxWidgets-only changes: `scripts/build-wxuniversal-wasm.sh` (on-machine, faster), then
relink eeschema.
- The webapp's iframe is being removed this round (separate task) to simplify dev — so the
app may load eeschema directly rather than in a same-origin iframe. The e2e harness
(`tests/apps/kicad/eeschema.html`, loaded directly by Playwright) already runs iframe-free;
it's the most reliable repro. Run kicad e2e from `tests/`: `npm run test:eeschema`.
- Symbolizing WASM stack frames (browser shows `wasm-function[N]` with no names): the separate
`*.debug.wasm` is PRE-asyncify and useless. See memory `eeschema_wasm_symbolization` for the
`--profiling-funcs` + asyncify `-g` recipe and the name-section index→name parser. Those are
temporary debug build flags — re-add while debugging, remove before committing.
- Use debug tools / symbols, don't guess (CLAUDE.md). Temporary `fprintf(stderr,"[TAG] …")`
shows as `[KICAD_ERR]` in the browser console; remove before committing.
## Memory pointers (read these first)
- `eeschema_schematic_load_crash` — full root-cause history of the load hang + this regex
follow-up (the "NEW minor follow-up" note).
- `eeschema_wasm_symbolization` — how to get real C++ names into browser WASM stack traces.