pcbjam/docs/features/fork-cleanup/00-effort-and-current-state.md
Istvan Matejcsok c35005589d refactor(cmake): batch F (gerbview/pl_editor #5 + navlib #6) + docs
Bumps kicad to 6f82742e68 (in-place CMake de-dup pass A-F complete) and updates
the fork-cleanup docs: CMake churn 1,702 -> 1,019, build-graph proven byte-identical
pre/post (zero behavior change), only Phase-B cmake/wasm relocation remaining.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 18:50:08 +02:00

227 lines
18 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Fork cleanup: current state, effort & resulting diff
> **Status:** measurement / planning. Authored 2026-06-17. **Supersedes the baseline numbers in
> [README.md](README.md)** — those were taken at KiCad HEAD `ac7d733787`; work has landed since.
> All figures here are against:
> - KiCad fork HEAD `cb82b64410`, upstream merge-base `4bfed3f174`
> (`jeff@rokeby.ie` "Remove previous hack for dealing with ligatures").
> - Reproduce any number with, in `kicad/`: `git diff 4bfed3f174 HEAD -- <path>` (working tree is clean).
> - **Do not** trust `scripts/kicad-diff-stats.sh` for the total — it only recognises two author emails
> and now reports `0` because HEAD is authored by a teammate (this is the doc-07 bug, still unfixed).
> Verify line numbers before editing — they drift.
## What this document answers
Per-change verdict — **revert** (not needed on web) / **re-enable** (stubbed "because web" but works) /
**relocate** (needed, but movable to a new file + thin include) / **keep** / **upstream-PR** / **config**
plus the engineering effort and the resulting diff size. It reconciles docs [01](01-revert-dead-code.md)[12](12-network-stack.md)
against the *current* source and against a ground-truth per-file ledger so churn is not double-counted.
## Bottom line
The KiCad fork currently diverges by **~2,795 churned lines across 88 modified upstream files**
(+2,215 / 580). The 48 *new* files (+30,334 lines, ~28k of which is the WebGL GAL copy under
`common/gal/webgl` + `include/gal/webgl`; `SmaaAreaTex.h` alone is 14,980) are **policy-compliant and
excluded** — additive new files do not hurt a rebase.
- **Pure divergence cleanup is ~1628 engineer-days** (~95165 h at 6 productive h/day).
- Doing it fully lands at **~50 modified files / ~480620 churned lines — a 7883% reduction**.
- A cheaper **~35 day pass removes ~1,460 lines** (steps 16 below) and gets you to ~1,330 residual; the
rest is diminishing-returns CMake relocation.
- **Biggest single lever: CMake — ~1,399 of the 2,795 lines (half the diff)** is mostly mechanical
wrap-and-reindent churn that collapses to ~175250 lines.
- **Feature re-enables that need porting (3D viewer Route C, ngspice) are weeks of product work and remove
almost zero diff until done — keep them out of the cleanup budget.**
## What is already done (since the README baseline)
| Item | Doc | Churn removed | Evidence |
|---|---|---|---|
| `KICAD_IPC_API` gate reverts (18 files) | 01 | ~107 | Commit `c39e8f8689`; all 18 files byte-identical to merge-base |
| `kiglew.h` +189 GLEW-stub revert | 01 | ~190 | `c39e8f8689`; pristine vs upstream |
| `opengl_gal.cpp` Connect→Bind refactor revert | 01 | ~63 | `c39e8f8689`; pristine |
| `KI_DIAG_*` / `wxFprintf` diag call-site deletion | 01 | ~41 | `c39e8f8689`; **zero** `KI_DIAG` call sites remain (diag flags now inert even though `-DKICAD_DIAG_*=1` is still passed) |
| CMake de-churn batches AF (16 files) | 02 | ~683 (CMake churn 1,702→1,019) | Kicad `6f82742e68` (F) on `b5ac6dcbe4` (AE); root `0b4d830`. `-O1` WASM build (pcbnew/eeschema/calculator) + 9/9 e2e, no screenshot change; **build-graph proven byte-identical pre/post for the whole tree** (compile+link+custom commands) → zero behavior change. Moves: `KICAD_WASM_LAYER` var, early-return guards, `list(REMOVE_ITEM)`, `add_shader` redefine via new `wasm/cmake/WasmShaderOverride.cmake`, additive `BUILD_KIWAY_DLL` sets, pcb_calculator restructure, gerbview/pl_editor #5, navlib #6 (behavior-preserving stub de-churn). **The in-place de-dup pass is complete; only Phase-B `cmake/wasm/` relocation remains.** |
| Tooltips implemented in wx fork | 05 | enables ~19 revert | `wxwidgets/include/wx/wasm/tooltip.h` + `src/wasm/tooltip.cpp`, `wxUSE_TOOLTIPS=1` |
| `GetCurrentSelection`/`IsEmpty` already base behavior in wx wasm | 05 | enables ~11 revert | wasm `wxChoice` inherits base `GetCurrentSelection(){return GetSelection();}` |
| Symbol-lib asset pipeline | 09 | 0 (new files) | `boot.ts` seeds `sym-lib-table`; PCBJAM_LIB/FP plugins are additive new files |
| `char_traits_uint16` Altium blocker fixed | 08 | 0 (build infra) | force-`-include`d in `scripts/kicad/build-kicad-target.sh:369` — makes importer re-enable safe |
| ngspice/SPICE source stubs do **not** exist | 11 | 0 | `eeschema/sim/` is pristine; only a 15-line CMake stub block exists |
**Doc [01](01-revert-dead-code.md) is effectively closed** (~401 lines already removed). The
`pcb_edit_frame.cpp` / `pgm_base.cpp` churn that doc 01 originally listed is now `KICAD_SCRIPTING`
gating — a different track (see Reverts).
## The diff math — by action
Current churn ties to the 2,795-line ledger; residual = churn remaining in modified upstream files after
the action.
| Action | Files | Current churn | Residual | Net reduction | Confidence |
|---|---|---|---|---|---|
| **Revert** (dead/native-only, not needed on web) | 11 | ~64 | 0 | **64** | HIGH |
| **Re-enable feature** (works on web, deletes gate) | 11 | ~145 | ~5 | **140** | MEDHIGH (runtime-untested) |
| **Relocate → new file + thin include** | ~26 | ~2,050 | ~330 | **1,720** | MED |
| **Keep-inline** (irreducibly small, needed) | ~14 | ~120 | ~120 | 0 | HIGH |
| **Upstream-PR** (vanishes on next submodule bump) | ~30 | ~115 | ~115 → 0 on merge | **115 on bump** | HIGH |
| **Config-not-code** (runtime setting) | 3 | ~32 | 0 | **32** | HIGH |
| **Native-bug-fix** (restores native, keeps web gate) | 2 | ~9 | ~8 | 1 | HIGH |
| **TOTAL** | **88** | **~2,795** | **~480620** | **2,175 to 2,315** | |
Residual sums to ~578 (relocate 330 + keep 120 + upstream 115 + re-enable 5 + native 8), dropping toward
~460 once the upstream PRs merge. The 480620 range is mostly "how far the CMake relocation is pushed."
**Double-counts resolved while reconciling:** `pcb_io_mgr.cpp`/`sch_io_mgr.cpp` counted once under doc 08
(not doc 06); nng/webview/nanodbc CMake exclusions once under doc 02 (not doc 12); `pcb_base_frame.*` /
`sch_base_frame.*` fswatcher guards once (docs 05 & 12 overlap); `hidpi_gl_canvas.cpp` once (docs 05 & 07);
the eeschema ngspice CMake block (doc 11, 15 lines) is a subset of the doc-02 eeschema total, not additive.
## Reverts — zero-risk diff deletion (restore upstream exactly)
**~64 churn → 0.**
- **`KICAD_SCRIPTING` dead guards (~64 lines, 11 files)** — Python is policy-off and never compiled, so the
gates are dead; reverting restores upstream and the OFF build still links. Files: `pcbnew/board_item.cpp`
(2), `pcbnew/pcbnew_settings.cpp` (2), `include/pgm_base.h` (2), `eeschema/sch_edit_frame.cpp` (2),
`common/tool/action_toolbar.cpp` (2), `pcbnew/menubar_pcb_editor.cpp` (4), `pcbnew/toolbars_pcb_editor.cpp`
(6), `eeschema/toolbars_sch_editor.cpp` (6), `pcbnew/pcbnew.cpp` (Python parts), `pcbnew/pcb_edit_frame.cpp`
(~22), `scripting/CMakeLists.txt` (5). **HIGH.**
- **Redundant wx workarounds (~11 lines)** — `eda_draw_frame.cpp` + `panel_setup_layers.cpp`
`GetCurrentSelection()→GetSelection()` (10) and `dialog_sim_command.cpp` `IsEmpty()→GetCount()>0` (1):
the wasm `wxChoice` already inherits the exact base behavior. No-ops. **HIGH.**
- **Tooltip guards (~19 lines, 8 files)** — wx fork ships working tooltip no-ops + `wxUSE_TOOLTIPS=1`, so
every `#ifndef __EMSCRIPTEN__` / `#if wxUSE_TOOLTIPS` guard in KiCad is dead. (Revert is the zero-risk
win; upstreaming the `wxUSE_TOOLTIPS` spelling is the cleaner long-term route.) **HIGH.**
- **fontconfig gratuitous comment/whitespace churn (~39 lines)** — noise in the still-compiled native arm;
copy back from upstream. **HIGH.**
- **Native-only reverts (doc 07):** `KICAD_IPC_API` default flip (3) + dead build-script flags
`KICAD_SPICE`/`KICAD_PCM`/`BUILD_GITHUB_PLUGIN` (root scripts, 0 kicad churn). **HIGH.**
## Relocate-to-new-file — needed on web, but shrinkable
**~2,050 → ~330.** Move the inline blob to a new TU/file; leave a ≤3-line `#include`/hook.
| Item | Files | Current → residual | Mechanism |
|---|---|---|---|
| **CMake `if(EMSCRIPTEN)` blocks** | 17 CMakeLists | ~1,399 → ~175250 | Create `kicad/cmake/wasm/*.cmake` (**does not exist yet**) + a `KICAD_WASM_LAYER` cache var; early-`return()` guards; `list(REMOVE_ITEM)` instead of move-and-reindent; additive last-wins property sets; relocate static-kiface/stub blocks behind a 1-line `include()` |
| **libcontext fiber backend** | `libcontext.cpp` | 334 → ~0 (+1 new file) | New `libcontext_emscripten.cpp` selected via CMake; revert the 2-line guard on upstream `release_fcontext` |
| **fontconfig WASM stub** | `fontconfig.cpp`/`.h` | 228 → ~4 | New `fontconfig_wasm.cpp`; native-vs-wasm compile selection; move `<fontconfig/fontconfig.h>` into the .cpp |
| **gestfich web tool-launch bridge** | `common/gestfich.cpp` | 37 → ~3 | `EM_ASM` blob → thin extern-C hook into a new wasm-layer file |
| **static-KIFACE loader** | `common/kiway.cpp` | 33 → ~3 | Move the resolver to `wasm_kiface_loader.cpp` or call `OnKifaceStart` from the bindings layer (`include/kiway.h` `LIB_ENV_VAR` 5 lines stays) |
| **WebGL GAL src list** | `common/gal/CMakeLists.txt` | 30 → ~3 | Move `list(APPEND)` to `cmake/wasm/gal_webgl.cmake` |
| **HiDPI scale-factor** | `hidpi_gl_canvas.cpp` | 6 → ~2 | Override `GetScaleFactor` in the wx wasm HIDPI canvas |
**Confidence MED** — most carry link-order / static-kiface risk and need a real WASM build
(`docker/build.sh`) to verify. The CMake relocation depends on first creating `kicad/cmake/wasm/`.
## Re-enable features — net-negative diff (delete the gate)
**KiCad-source diff deletion (cheap, do now):**
- **Importers (doc 08, ~95 lines)** — delete `if(NOT EMSCRIPTEN)` / `#ifndef __EMSCRIPTEN__` gates in
`common/CMakeLists.txt` (Altium), `pcbnew/CMakeLists.txt`, `pcb_io_mgr.cpp`, `eeschema/CMakeLists.txt`,
`sch_io_mgr.cpp/.h`. Altium blocker already neutralized. ⚠️ **Configure-confident but runtime-untested**
the Altium pcbnew loader has never been linked for WASM; smoke-test with sample Eagle + Altium files.
- **Symbol viewer/chooser (doc 09, ~9 lines)** — delete the two `#ifdef __EMSCRIPTEN__ return nullptr`
gates in `eeschema.cpp`; assets already provisioned. ⚠️ Runtime-untested (GAL/LIB_TREE + asyncify modal).
- **3D delete-from-diff (doc 10, ~93 lines)** — `exporter_vrml.cpp` stub (64), `pcbnew_jobs_handler.{cpp,h}`
render job (9), `pcbnew.cpp` 3D panels (6), `eda_3d_canvas.cpp` raytracer gates (18).
**NOT source diff — real porting (weeks, separate):** Route C canvas blit wiring (1640 h, new file),
static model-loading plugins (2460 h), the ngspice port (3070 h). These cut **0** divergence lines until
done — feature work, not cleanup.
## Must-keep / impossible
- **Collab/Yjs persistence hooks (~53 lines)** — `kicadCollabOnSave/OnModify` extern-C call sites at the
save/modify chokepoint: `pcbnew/files.cpp` (13), `eeschema/files-io.cpp` (12), `pagelayout_editor/files.cpp`
(13), `pl_editor_frame.cpp` (15 → ~4 after trimming comments). No upstream listener API for drawing-sheet →
can't relocate behind a notification. **KEEP** (trim comments).
- **WebGL-GAL switch / silent Cairo fallback / warp-cursor** in `draw_panel_gal.cpp` (~17) — genuine web code,
irreducibly small inline. **KEEP.**
- **SpaceMouse gates (~33 lines, 3D)** — 3Dconnexion hardware, impossible in a browser. Collapse to a leading
no-op `__EMSCRIPTEN__` branch on the existing `#ifdef __linux__` ladder + drop redundant null-guards
(`m_spaceMouse` is already default-null) → ~6 residual. Only matters once 3D builds.
- **wx-progress-reporter early-return (9)**, **bs_thread_pool inline shim (16)** — asyncify constraints; the
proper fix lives in the wx layer / a re-vendor, not the rebase. **KEEP.**
- **`libcontext.h` platform `#elif` (9)**, **`include/kiway.h` `LIB_ENV_VAR` (5)**, **kiplatform CMake routing
(19)**, **nng/IPC CMake stub**, **`openGL_includes.h` kiglew redirect (5)** — irreducibly small additive
guards. **KEEP.**
- **ngspice model-data CMake stub (15)** — too-many-locals V8 workaround; only removable if a rebuild proves
`wasm-opt -O2` now resolves it. **KEEP for now.**
## Orphans — divergence the write-up (docs 0112) never covers
- **Drawing-sheet UUID feature (~33 lines, unconditional — no `#ifdef`, affects ALL builds):**
`drawing_sheet.keywords` (1), `drawing_sheet_parser.cpp` (24), `ds_data_model_io.cpp` (4),
`ds_data_item.h` (4). Adds a stable per-item KIID for the Yjs differ and **changes the `.kicad_wks`
on-disk format unconditionally**. Strong **upstream-PR** candidate (KiCad is adding UUIDs to other item
types); until merged it is pure divergence no doc accounts for.
- **Collab save/modify hooks (~53 lines)** — the `kicadCollabOnSave/OnModify` files above; acknowledge as
permanent web-needed divergence. **KEEP.**
- **`bs_thread_pool.hpp` (16)** — async-theme single-thread shim, not 0112. **KEEP / re-vendor.**
## Per-document effort & diff-reduction
Hours → days at 6 productive h/day.
| Doc | Theme | Current churn | Residual | Reduction | Effort (h / days) | Risk | Verdict |
|---|---|---|---|---|---|---|---|
| [01](01-revert-dead-code.md) | Revert dead code | 0 | 0 | 0 | 00.5 / ~0d | low | **DONE** (`c39e8f8689`) |
| [02](02-cmake-dechurn.md) | CMake de-churn | ~1,702 → 1,019 | ~175250 | **683 so far** | 2238 / 3.76.3d | med | **AF (in-place de-dup) DONE** (`6f82742e68`); only Phase-B `cmake/wasm/` relocation remains. Build graph proven identical → zero behavior change |
| [03](03-config-not-code.md) | Config-not-code | ~60 | ~3 | ~57 | 715 / 1.22.5d | lowmed | 4/5 → runtime config |
| [04](04-stub-tu-relocation.md) | Stub-TU relocation | ~571 | ~2031 | **~540551** | 917 / 1.52.8d | lowmed | **2nd-biggest lever** |
| [05](05-wx-layer-fixes.md) | wx-layer fixes | ~37 | ~3 | ~34 | 37 / 0.51.2d | low | Mostly revertable now |
| [06](06-upstreamable-patches.md) | Upstreamable patches | ~122 | ~48 (→~15 post-merge) | ~74+ | 1739 / 2.86.5d | lowmed | PR; UUID+collab stay |
| [07](07-native-build-bugs-and-tooling.md) | Native-build bugs & tooling | ~8 (+root scripts) | ~2 | ~6 | 49 / 0.71.5d | low | Tiny, near-zero WASM risk |
| [08](08-importers.md) | Importers re-enable | ~97 | ~2 | **~95** | 514 / 0.82.3d | lowmed | Net-negative; runtime-untested |
| [09](09-symbol-libraries.md) | Symbol libs re-enable | ~9 | 0 | ~9 | 13 / 0.20.5d | med | Un-gate; assets shipped |
| [10](10-3d-viewer.md) | 3D viewer (Route C) | ~145 | ~12 | ~133 (diff) | 4096 / 6.716d | high | ~93 delete cheap; rest is NEW porting |
| [11](11-ngspice-simulator.md) | ngspice simulator | ~15 | ~15 | 0 now | 4090 / 6.715d | high | Future feature; keep 15 CMake lines |
| [12](12-network-stack.md) | Network stack | ~91 | ~27 | ~64 | 1122 / 1.83.7d | lowmed | fswatcher→PR, kiway→relocate |
**Pure-cleanup total (docs 0108 + revertible parts of 12, excluding the 3D/ngspice feature ports):
~95165 h ≈ 1628 engineer-days.**
## Recommended sequencing (by diff-reduction-per-hour)
Cumulative removed is from the live 2,795 baseline (doc 01's ~401 already gone).
| # | Action | Doc | Reduction | Effort (h) | Cumulative removed | Risk |
|---|---|---|---|---|---|---|
| 1 | Stub-TU relocation (libcontext + fontconfig) | 04 | ~540 | 511 | ~540 | lowmed |
| 2 | CMake de-dup pass (guards + `REMOVE_ITEM`, no relocation yet) — **batches AF DONE** (`6f82742e68`, 683; in-place de-dup complete) | 02 | ~700 | 814 | ~1,240 | lowmed |
| 3 | Importer re-enable (delete gates) | 08 | ~95 | 514 | ~1,335 | lowmed |
| 4 | `KICAD_SCRIPTING` reverts | 03/06 | ~64 | 24 | ~1,399 | low |
| 5 | wx-layer + tooltip reverts | 05 | ~30 | 24 | ~1,429 | low |
| 6 | Config-not-code (warp cursor/hotkey/clock) | 03 | ~32 | 37 | ~1,461 | low |
| 7 | CMake relocation to `cmake/wasm/` (the hard ~450→~200) | 02 | ~250 | 1224 | ~1,711 | med |
| 8 | gestfich + kiway relocate | 12/03 | ~67 | 612 | ~1,778 | med |
| 9 | Symbol viewer/chooser un-gate | 09 | ~9 | 13 | ~1,787 | med |
| 10 | Upstream-PR batch (RTree, hotkey, choice.h, ngspice guard, UUID, fswatcher) | 06/07/12 | ~115 on bump | 1739 | ~1,902 (post-merge) | lowmed |
| 11 | Native-build bug fixes + dead-flag removal | 07 | ~6 | 49 | ~1,908 | low |
**End-state after steps 19 (pure cleanup, pre-merge): ~1,008 residual. After the upstream PRs land
(step 10): ~895. With remaining CMake relocation + the deferred 3D/ngspice gate deletions folded in:
~480620 residual.** The collab-bridge orphans (~86) + irreducible keep-inline gates (~120) are the
permanent floor.
**Highest-ROI first move:** stub-TU relocation (step 1) — ~540 lines for one day at the lowest risk —
then the CMake de-dup pass, which is half the entire diff.
## Caveats
- **480620 is the *full* end-state.** Steps 16 (~2550 h) get you to ~1,330 residual; the rest is
diminishing-returns CMake relocation.
- **Relocate estimates are MED confidence** — they need a real WASM build to verify link order /
static-kiface behavior. Budget a build + e2e run per relocation batch.
- **3D and ngspice are weeks of feature work, not cleanup** — they delete ~100 diff lines but only after
dozens of porting hours; exclude from any diff-reduction budget.
- Per the project's own policy ([fork-divergence research](../../../docs/features/fork-cleanup/06-upstreamable-patches.md)),
reduce divergence in the wasm/build layer first; temporary diagnostics in kicad/wx are OK but must be
reverted before commit.
## Related
- [README.md](README.md) — the master inventory (baseline numbers now superseded by this doc).
- [07-native-build-bugs-and-tooling.md](07-native-build-bugs-and-tooling.md) — includes the
`kicad-diff-stats.sh` author-list bug that makes the script report `0`; fix it to track the real number.