tooling(sourcetrail): code-graph setup + asyncify allocator-suspend plan

sourcetrail/: Sourcetrail 2021.4.19 indexing pipeline for the wasm port —
compile-db transform (rsp expansion, PCH strip, libc++-11 pinning, path
rewrites), removelist-candidate analysis over the indexed call graph, README
with regen steps and the hard-won tricks (relative --project-file hang,
SDK-header poisoning). Heavy artifacts (.srctrldb, compile dbs, libcxx
headers) stay untracked via the folder's .gitignore.

docs/asyncify-allocator-suspend/plan.md: verified plan for the nanosleep-shim
zero-duration guard (mimalloc mi_atomic_yield=sleep(0) can suspend malloc on
the main thread), the red/green contention test, and the adversarially
verified removelist additions with measured payoff (80.9k -> 60.0k
instrumented functions). Planned, not yet executed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P2NHhbzuEHqP2JmcrSD96D

docs/asyncify-allocator-suspend/plan.md

@@ -0,0 +1,131 @@
+<!-- STATUS: PLANNED, NOT EXECUTED (saved 2026-08-10). Verification findings herein are real
+     (measured against the Aug 6 kicad_editor build, emsdk 4.0.2); the shim guard, the test,
+     and the removelist additions have NOT been applied yet. -->
+
+# Nanosleep shim zero-duration guard + mimalloc contention red/green test
+
+## Context
+
+The asyncify investigation established that **memory allocation can suspend on the main thread**: emscripten's vendored mimalloc uses `sleep(0)` as its spin-wait yield (`mi_atomic_yield` fallback — wasm32-emscripten matches no arch case in `include/mimalloc/atomic.h`), and `wasm/shims/nanosleep_yield.c` unconditionally converts main-thread `nanosleep` into an Asyncify event-loop yield. Under cross-thread delayed-free contention, malloc's slow path can unwind mid-allocation — a latent reentrancy hazard that also blocks the planned allocator removelist entries (~14K-function instrumentation win).
+
+Adversarial verification (real kicad_editor binary + emsdk 4.0.2 sources) proved: the `sleep(0)` chain is the allocator's **only** suspend path; all four mimalloc spin sites sleep constant 0; mimalloc is the module's **only zero-duration sleeper** (all other nanosleep callers are ≥1 ms constants); `std::this_thread::yield`/`sched_yield` don't route through nanosleep. Hence an `ms == 0` early return in the shim severs the path completely and affects nothing else.
+
+**This task:** (1) the zero-duration guard in the shim; (2) a standalone red/green C++ wasm test that exercises the mimalloc contention path and detects main-thread event-loop turns — RED before the fix, GREEN after; (3) add the verified-safe memory-touching entry families to `scripts/common/asyncify-removelist.txt` (user-proposed list, adversarially verified against the real module first).
+
+Out of scope (explicit follow-ups): full kicad_editor rebuild + 3D e2e suite run with the new removelist (the new entries only take effect at the next app build's post-process anyway).
+
+## Part 1 — shim fix
+
+**File:** `pcbjam/wasm/shims/nanosleep_yield.c` (52 lines).
+
+In `nanosleep()`: after computing `ms`, wrap the yield/sleep branch in `if (ms > 0.0)` — zero-duration requests return 0 immediately on both main thread and workers (a 0 ms blocking sleep is a no-op anyway; `sleep(0)` arrives as exactly `{0,0}` per musl, so the guard fires deterministically). Extend the header comment: mimalloc's `mi_atomic_yield` is `sleep(0)` (spin-politeness hint, must NOT become an event-loop yield mid-malloc); a zero-duration sleep never promised an event-loop turn; verified 2026-08-10 that mimalloc is the only zero-duration caller in the module.
+
+Rebuild consequences: only binaries that link the shim — future KiCad app builds, `pthread-ondemand`, and the new test. Already-built `output/*.wasm` unchanged. (CI: touching `wasm/**` busts the testapps cache — expected.)
+
+## Part 2 — the test
+
+### App: `tests/apps/standalone/mimalloc-storm/mimalloc_storm_test.cpp`
+
+Plain `int main()` app (non-wx — template: `standalone/coroutine-pthread/main_repro.cpp`), pthreads + asyncify + **`-sMALLOC=mimalloc`** (used nowhere in the test tree today) + the nanosleep shim.
+
+**Storm** (turns the nanosecond `MI_DELAYED_FREEING` windows into a hit-rate game we control):
+- Main thread allocates batches of small same-size blocks (64 B → pages fill fast → full pages enter delayed-free mode; ~200k blocks ≈ ~200 full pages/round).
+- W worker `std::thread`s free the previous round's blocks in tight loops (every cross-thread free of a full page runs the two-CAS `DELAYED_FREEING` bracket).
+- Main concurrently churns alloc/free in the same size class and calls `mi_collect(true)` each round (`extern "C" void mi_collect(bool)` — public API, headers not on include path; reaches `_mi_heap_delayed_free_all`, the spin site).
+- Bounded by ITERATION counts (no wall-clock). Early-exit once ≥50 turns observed (keeps pre-fix RED runs fast); parameters tuned empirically at the red-validation step.
+
+**Detector** (did main return to the event loop mid-storm?):
+- `EM_ASM` arms a self-re-arming `setTimeout(0)` that increments `globalThis.__stormTurns`; a synchronous C++ storm can only let it run if an Asyncify unwind happened inside. Main checks the counter via `EM_ASM_INT` every k iterations.
+- Phase 2 assertion (shim's load-bearing behavior unchanged): re-arm marker, `nanosleep(5 ms)` on main → the marker MUST have run (nonzero sleeps still yield).
+- Console contract (repo idiom — `EM_ASM` console.log markers, self-terminating, documented in the file header):
+  - `[MIMALLOC_STORM] START threads=W blocks=B rounds=R`
+  - `[MIMALLOC_STORM] SUMMARY stormTurns=N sleepTurned=0|1 completed=1`
+
+### Build rules: `tests/apps/Makefile.wasm`
+
+- `LDFLAGS_MIMALLOC_STORM` modeled on `LDFLAGS_COROUTINE_PTHREAD_NOWX` (line ~875: EH_FLAGS, ALLOW_MEMORY_GROWTH, ASYNCIFY=1 + stack size, DYNCALLS, -pthread, pool=`navigator.hardwareConcurrency`, STRICT=0, no wx) **plus `-sMALLOC=mimalloc`**.
+- `.o` rule for the app; `nanosleep_yield.o` reuse per the `pthread-ondemand` pattern (`Makefile.wasm:681`, compiled `-c -pthread`); `.html` rule; `.PHONY: mimalloc-storm`; `all:` accumulation line.
+- No `mallinfo_stub.c` (no OCC). No new suspending import (`env.__asyncjs__*` already in `scripts/common/asyncify-imports.txt`). Post-link asyncify (`apply-asyncify.sh --no-removelist`) + dyncall injection happen automatically via `build-wasm-test.sh`'s `find -newer` fan-out — no driver-script changes.
+
+### Spec: `tests/e2e/mimalloc-storm.spec.ts`
+
+- Placement in `tests/e2e/` → runs under the existing `wx-chromium` project; **zero playwright-config/CI edits** and passes `lint:ci-coverage`.
+- Pattern copied from `tests/e2e/coroutine-pthread.spec.ts` (the non-wx precedent): `testLogger` fixture, best-effort `tryLoadApp(...).catch(() => {})` with the documented marker comment, then `expect.poll` for the `SUMMARY` line; parse it; assert `stormTurns === 0`, `sleepTurned === 1`, `completed === 1`, and no page errors (favicon-filtered).
+- Determinism compliance: no `waitForTimeout`, no inline retries, bounded C++ iterations; the app always emits a terminal SUMMARY (self-capping), so the poll is bounded.
+
+## Part 3 — asyncify-removelist additions (VERIFIED)
+
+Adversarial verification against the real kicad_editor module is complete: **all proposed families SAFE**, conditional on the Part 1 shim guard shipping first. Add to `scripts/common/asyncify-removelist.txt` (exact patterns — see traps below):
+
+```
+# --- Memory-touching families (safe ONLY with the nanosleep zero-duration guard:
+# --- mimalloc's mi_atomic_yield is sleep(0); with the guard, no allocation path suspends.
+# --- Also conditional: no one registers mi_register_deferred_free/output/error hooks or a
+# --- suspending std::new_handler (all unregistered as of 2026-08-10 audit).
+std::__2::basic_string<*>*
+std::__2::char_traits<*>*
+std::__2::vector<*>*
+std::__2::__tree*
+std::__2::to_string*
+std::__2::__itoa*
+std::__2::__split_buffer<*>*
+std::__2::__shared_ptr_emplace<*>*
+std::__2::__shared_ptr_pointer<*>*
+std::__2::__shared_count*
+std::__2::__shared_weak_count*
+std::__2::deque<*>*
+std::__2::__hash_table<*>*
+std::__2::map<*>*
+std::__2::__list_imp<*>*
+std::__2::allocator*
+boost::uuids::*
+operator new*
+operator delete*
+aligned_alloc
+mi_*
+_mi_*
+sbrk
+# Deliberately NOT std::__2::__function* — type-erased std::function invocation (operator())
+# is how tool/dialog callbacks run; 111 __func::operator() bodies reach startModal/wxMilliSleep
+# directly (e.g. ShowPreferences, file dialogs), so __function frames must stay instrumented.
+# Container entries above are still safe: containers only touch callables via their
+# clone/destroy lifecycle ops, none of which reach a suspend (verified 2026-08-10).
+```
+
+**Authoring traps (verified the hard way):**
+- **Bracket balance:** Binaryen's list parser tracks `<>()[]{}` nesting — an unbalanced pattern like `std::__2::basic_string<*` **aborts the whole asyncify pass** (`Fatal: failed to parse lists`). Template patterns must close the bracket: `basic_string<*>*`.
+- **Breadth anchoring:** bare `basic_string*` sweeps in `basic_string_view`/`basic_stringbuf`/`basic_stringstream` (31 extra names); the `<`-anchored form excludes them. `__tree*` deliberately includes `__tree_node_base`/iterators (verified). Bare `vector*`/`char_traits*` (no namespace) are wrong forms.
+
+**Key evidence:** zero direct suspend paths post-guard in every family; `__tree` comparators are monomorphized (no `call_indirect` in e.g. `__find_equal` with `std::less`); container-of-`std::function` ops (e.g. TOOL_MANAGER's `deque<function<void()>>::push_back`) only touch the functor's clone/move/destroy lifecycle slot — of 12,880 `__function` lifecycle ops in the module, zero reach a suspend; all 111 suspend-reaching `__function` members are `operator()` invocation bodies (the excluded family). `boost::uuids` (KIID) seeds `mt19937` once via synchronous `getentropy` (wasi import, not asyncify-relevant).
+
+**Measured payoff** (single-threaded verbose runs, real hoisted module):
+
+| Run | Removelist | Instrumented | Module size |
+|---|---|---|---|
+| A | current file | 80,871 (74%) | 253 MB |
+| B | + allocator family | 66,499 (60%) | 218 MB |
+| **C** | **+ all families above** | **60,047 (55%)** | **204 MB** |
+
+Run C is a strict subset of B (sanity: zero additions); only non-matching warnings are the 5 pre-existing OCC entries (expected — OCC absent from this app). Audit validity: this binary / emsdk 4.0.2; re-run on emsdk bump or libc++ ABI-namespace change (artifacts in `$W`: `analysis.pkl`, `logC.txt`, `runC.wasm`).
+
+## Part 4 — execution sequence
+
+1. **Build the test against the UNPATCHED shim**: `./scripts/build-wasm-test.sh mimalloc-storm` (targeted — the default `all` target is currently broken by pre-existing working-tree deletions, see caveat). Run the spec: `cd tests && npx playwright test --project=wx-chromium e2e/mimalloc-storm.spec.ts`. Expect the spec to FAIL with `stormTurns > 0` — that failure IS the red validation (proves the storm reaches the mimalloc yield). Record the observed turn count; tune B/R/W if turns are marginal (<~10).
+2. **Apply the shim fix** (Part 1), rebuild the same target (make tracks the shim as an explicit prerequisite), re-run the spec → GREEN: `stormTurns === 0`, storm completes (no-deadlock proof), `sleepTurned === 1`.
+3. `cd tests && npm run lint:determinism` (spec must pass the linter).
+4. **Removelist update:** add the verified entries to `scripts/common/asyncify-removelist.txt` with a documented block: the guard dependency (entries safe ONLY with the zero-duration nanosleep guard), the dormant-slot condition (deferred-free/new_handler/output hooks unregistered), and the `std::__2::__function*` exclusion rationale. Sanity-check the new list against the scratch module: one `wasm-opt --asyncify` run on `$W/hoisted.wasm` with the updated file confirms the expected instrumented-function count and that no entry matches zero functions unexpectedly.
+5. Stop before committing — present the diff; commit on main via the user's `/git-feature-commit` flow on request (trunk-based repo).
+
+## Caveats found during exploration
+
+- **Pre-existing working-tree deletions** (not ours; do not touch): `tests/apps/standalone/asyncify-races/races_test.cpp`, `tests/asyncify/asyncify-races.spec.ts`, `tests/web/eeschema-fp-selector.spec.ts` are tracked but deleted, which breaks `Makefile.wasm`'s `all` target and full `npm run test:e2e`. We build only our target and run only our spec; flag the state to the user at the end.
+- `build-wasm-test.sh` requires `build-wasm/wxwidgets/wx-config` (wx libs built) and stubs the emsdk in-link asyncify — the emsdk lives at `tools/emsdk/` (installed via `scripts/setup-emsdk.sh` if absent).
+- Reference precedents: `races_test.cpp` (via `git show HEAD:...`) for Asyncify-state probing from C++ if the setTimeout detector needs corroboration; `pthread_ondemand_test.cpp` for the watchdog/marker idiom.
+
+## Verification summary
+
+- RED observed pre-fix (spec fails on `stormTurns > 0`), GREEN post-fix on identical parameters — the core deliverable.
+- GREEN also proves: spin completes without the yield (no deadlock), 5 ms sleep still turns the event loop (worker-boot behavior preserved).
+- `lint:determinism` clean.
+- Removelist sanity run on `$W/hoisted.wasm` reproduces run C's numbers (60,047 instrumented / 204 MB pre-O1) with no unexpected non-matching-pattern warnings.
+- Follow-ups NOT here: full editor rebuild + 3D e2e suite with the new removelist (entries take effect at the next app build's host post-process).

sourcetrail/.gitignore

@@ -0,0 +1,9 @@
+# Generated/machine-local artifacts — only the README, scripts, and project
+# files are tracked. Everything below is rebuilt by the steps in README.md.
+*.srctrldb
+*.srctrldb_tmp
+*.srctrlbm
+compile_commands.json
+sample_compile_commands.json
+libcxx-11/
+libcxx-11.src.tar.xz

sourcetrail/README.md

@@ -0,0 +1,113 @@
+# Sourcetrail code graph for the KiCad WASM port
+
+Interactive symbol-level code graph (classes, calls, includes, inheritance) of the
+merged `kicad_editor` build — all 2,176 TUs: pcbnew, eeschema, common, 3D viewer,
+and the `wasm/` port layer — indexed by [Sourcetrail](https://github.com/CoatiSoftware/Sourcetrail).
+Lives in `pcbjam/sourcetrail/`: README, scripts and project files are tracked; the heavy
+artifacts (index db, compile db, libc++ headers) are ignored via the local `.gitignore`
+and rebuilt with the steps below.
+
+## What's in this folder
+
+| File | Purpose |
+|---|---|
+| `kicad-wasm.srctrlprj` / `.srctrldb` | Sourcetrail project + indexed database (~440 MB) |
+| `compile_commands.json` | Transformed compile db the indexer consumes |
+| `transform_compile_db.py` | Turns the container's emscripten compile db into the above |
+| `libcxx-11/` | Pinned libc++ 11.1.0 headers (see Tricks) |
+| `asyncify_candidates.py` | Ranks subsystems safe for the asyncify removelist |
+| `sample.srctrlprj` + `sample_compile_commands.json` | 6-file smoke test for the pipeline |
+
+## Install
+
+- **Sourcetrail 2021.4.19** — the last free release (the maintained fork paywalls binaries):
+  <https://github.com/CoatiSoftware/Sourcetrail/releases/tag/2021.4.19>, macOS dmg → `/Applications`,
+  then `xattr -dr com.apple.quarantine /Applications/Sourcetrail.app`. It's x86_64 — needs Rosetta 2
+  on Apple Silicon (works fine).
+- Nothing else: python3 stdlib only; header mirrors are exported from the Docker build.
+
+## Run
+
+```bash
+# browse (GUI)
+open -a Sourcetrail /Users/V/IdeaProjects/pcbjam-private/pcbjam/sourcetrail/kicad-wasm.srctrlprj
+
+# (re)index from the terminal — ABSOLUTE project path, see Tricks
+/Applications/Sourcetrail.app/Contents/MacOS/Sourcetrail index \
+  --project-file /Users/V/IdeaProjects/pcbjam-private/pcbjam/sourcetrail/kicad-wasm.srctrlprj
+```
+
+Full index ≈ 12 min all-cores. In the GUI there is no single whole-project graph: search a
+symbol (Cmd+F — e.g. `BOARD`, `KIPLATFORM`) and click it; the graph pane centers on it and
+expands as you click nodes/edges.
+
+### Refreshing after a KiCad rebuild
+
+The compile db comes from the `main`-branch Docker build cache (KiCad's CMake exports it
+by default — no reconfigure needed). From `pcbjam/`:
+
+```bash
+source scripts/common/versions.sh
+COMPOSE_PROJECT_NAME=kicad-wasm-main docker compose -f docker/docker-compose.yml up -d --build
+
+# compile db + generated sources/headers + .rsp files + deps/wx includes
+docker compose -p kicad-wasm-main -f docker/docker-compose.yml exec -T kicad-wasm-builder bash -c \
+  "cd /workspace && { find build-wasm/kicad-kicad_editor \( -name '*.h' -o -name '*.hpp' -o -name '*.hxx' \
+   -o -name '*.hh' -o -name '*.inc' -o -name '*.rsp' -o -name '*.cc' -o -name '*.cpp' -o -name '*.cxx' \
+   -o -name 'compile_commands.json' \) -type f; echo build-wasm/sysroot/include; \
+   echo build-wasm/wxwidgets/lib/wx/include; } | tar -cf - -T -" | tar -xf - -C .
+
+# emscripten sysroot headers -> tools/emsdk mirror (only after an emsdk bump)
+docker compose -p kicad-wasm-main -f docker/docker-compose.yml exec -T kicad-wasm-builder \
+  tar -cf - -C / emsdk/upstream/emscripten/cache/sysroot/include | tar -xf - -C tools/
+
+cd sourcetrail
+python3 transform_compile_db.py ../kicad-kicad_editor/compile_commands.json compile_commands.json
+# then the `Sourcetrail index` command above
+```
+
+## Tricks (why this isn't just "point Sourcetrail at the cdb")
+
+- **The bundled clang is ~LLVM 11.** Modern libc++ (emsdk 4.x's or the macOS SDK's) does not
+  parse under it. The transform pins `libcxx-11/` via `-nostdinc++` and takes C headers from the
+  emsdk musl sysroot mirror (`tools/emsdk/`). Expected residue: ~14 errors, all in the
+  libc++11/musl locale seam (`_CTYPE_*`, `strtoull_l`, one fatal `xlocale.h`) — harmless to the graph.
+- **Transform surgery:** `@CMakeFiles/*.rsp` response files are expanded inline (old clang's cdb
+  loader can't); PCH is stripped (`-Xclang -include-pch` of clang-20 `.pch` binaries) and replaced
+  with `-include cmake_pch.hxx`; emscripten-only flags (`-sFOO`, `-fwasm-exceptions`) dropped,
+  `--target=wasm32-unknown-emscripten` + `-fexceptions` added; paths rewritten
+  `/workspace` → `pcbjam/`, `/emsdk` → `pcbjam/tools/emsdk/`.
+- **CLI hangs on relative `--project-file` paths.** Silently — idle event loop, log stops after
+  "Maven executable path detection". Always pass absolute paths.
+- **Global header paths are deliberately empty** in
+  `~/Library/Application Support/Sourcetrail/ApplicationSettings.xml`. First launch auto-filled
+  macOS-26-SDK paths, which poison every parse (see clang-11 point). Don't re-run header path
+  detection from Preferences; `has_prefilled_header_search_paths=1` keeps it from coming back.
+- **"N files (126 complete)" undersells the index.** A file counts as complete only if *every* TU
+  touching it had zero errors; the 14 std-header errors are included nearly everywhere, so the
+  flag cascades. The symbols/references themselves are all recorded.
+
+## Asyncify removelist candidates
+
+```bash
+python3 asyncify_candidates.py
+```
+
+Computes, over the indexed call graph (187K call edges + override pseudo-edges for virtual
+dispatch), which functions can NEVER reach a suspend point, aggregated per module. Seeds =
+every `ShowModal`/`Yield`/`Sleep`/progress-dialog/`COROUTINE` function (**strict**), plus
+`ProcessEvent`-style synchronous dispatch (**lenient** — a dispatched handler may suspend and
+unwind through the dispatcher). Functions clean under *lenient* are candidates for
+`scripts/common/asyncify-removelist.txt` (matching rules are documented in that file:
+one prefix wildcard per symbol, e.g. `SHAPE_POLY_SET::*`).
+
+Headline results from the 2026-08-07 index: `kiapi` generated protobuf (15.2K funcs),
+`libs/kimath` (7.3K), `clipper2` (6.3K), `nlohmann_json`/`fmt`/`pegtl`/`zint` are 100% clean;
+`pcbnew/router` is 1,820/1,827 clean (the 7 are the `Wait()` tool-integration layer — the PNS
+shove/optimizer core never suspends).
+
+**Caveats:** the C++ graph can't see calls through `std::function`, event tables, or raw function
+pointers, and asyncify operates on the post-inlining *wasm* call graph, not C++ symbols. Before
+shipping an entry: ground-truth with Binaryen's asyncify verbose/advise output in
+`apply-asyncify.sh`, and rely on e2e — a wrong removal traps loudly (`unreachable`) at the first
+unwind through it. Tweak seeds / path depths at the top of the script.

sourcetrail/asyncify_candidates.py

@@ -0,0 +1,125 @@
+#!/usr/bin/env python3
+"""Rank subsystems by asyncify-removelist safety using the Sourcetrail db.
+
+Taint = "a suspend point is reachable from this function" (reverse BFS over
+call edges from suspend primitives, with base->override pseudo-edges for
+virtual dispatch).  Two levels:
+  strict  — direct suspend primitives only (Yield/Sleep/ShowModal/progress/
+            coroutine/fiber)
+  lenient — strict + synchronous event dispatch (ProcessEvent & co), since a
+            dispatched handler may suspend and unwind through the dispatcher
+
+Functions never tainted even in lenient mode are removelist candidates.
+Blind spots (validate with ASYNCIFY_ADVISE before shipping): std::function /
+event-table indirection, function pointers, wx/libc internals outside the cdb.
+"""
+import re
+import sqlite3
+from collections import defaultdict, deque
+
+DB = "kicad-wasm.srctrldb"
+PCBJAM = "/Users/V/IdeaProjects/pcbjam-private/pcbjam/"
+FUNC_TYPES = (4096, 8192)
+
+STRICT_PATTERNS = [
+    r"\tnShowModal",  # every ShowModal incl. DIALOG_SHIM/KIDIALOG wrappers
+    r"\tnShowWindowModal",
+    r"\tnShowQuasiModal",
+    r"mwxYield", r"mwxSafeYield", r"mwxYieldIfNeeded",
+    r"\tnYield", r"\tnYieldFor", r"\tnDoYieldFor", r"\tnSafeYield",
+    r"\tnSleep", r"mwxSleep", r"mwxMilliSleep", r"mwxMicroSleep",
+    r"sleep_for", r"sleep_until", r"nanosleep", r"emscripten_sleep",
+    r"emscripten_fiber", r"__asyncjs__",
+    r"mwxGenericProgressDialog", r"mwxProgressDialog",
+    r"mwxMessageBox", r"mwxExecute",
+    r"mCOROUTINE<",
+]
+LENIENT_EXTRA = [
+    r"\tnProcessEvent\t", r"\tnSafelyProcessEvent", r"\tnProcessPendingEvents",
+    r"\tnHandleEvent\t", r"\tnProcessEventLocally",
+]
+
+
+def main():
+    con = sqlite3.connect(DB)
+
+    funcs = {}  # id -> serialized_name
+    for nid, name in con.execute(
+            f"SELECT id, serialized_name FROM node WHERE type IN {FUNC_TYPES}"):
+        funcs[nid] = name
+
+    # callee -> callers (reverse call graph)
+    rev = defaultdict(list)
+    for s, t in con.execute("SELECT source_node_id, target_node_id FROM edge WHERE type=8"):
+        rev[t].append(s)
+    # virtual dispatch: caller of Base::f may land in Derived::f, so taint of
+    # Derived::f must flow to callers of Base::f -> pseudo-edge callee=Derived,
+    # caller-side=Base is wrong; we need: if Derived tainted then Base tainted
+    # is NOT true. Correct: call to Base::f can dispatch to Derived::f, so if
+    # Derived::f suspends, callers of Base::f suspend. Model: rev[Derived] gets
+    # nothing; instead treat Base::f as a caller of every override Derived::f.
+    for s, t in con.execute("SELECT source_node_id, target_node_id FROM edge WHERE type=32"):
+        # override edge: Derived::f (source) -> Base::f (target)
+        rev[s].append(t)  # taint flows Derived -> Base -> Base's callers
+
+    def seeds_for(patterns):
+        pats = [re.compile(p) for p in patterns]
+        return {nid for nid, name in funcs.items() if any(p.search(name) for p in pats)}
+
+    def closure(seed_ids):
+        seen = set(seed_ids)
+        q = deque(seed_ids)
+        while q:
+            n = q.popleft()
+            for caller in rev.get(n, ()):
+                if caller not in seen:
+                    seen.add(caller)
+                    q.append(caller)
+        return seen
+
+    strict_seeds = seeds_for(STRICT_PATTERNS)
+    lenient_seeds = strict_seeds | seeds_for(STRICT_PATTERNS + LENIENT_EXTRA)
+    print(f"functions: {len(funcs)}  strict seeds: {len(strict_seeds)}  "
+          f"lenient seeds: {len(lenient_seeds)}")
+
+    tainted_strict = closure(strict_seeds)
+    tainted_lenient = closure(lenient_seeds)
+
+    # function -> file (prefer definition scope locations, type=1)
+    loc = {}
+    for typ in (1, 0):
+        for eid, path in con.execute(
+                "SELECT o.element_id, f.path FROM occurrence o "
+                "JOIN source_location sl ON sl.id=o.source_location_id "
+                "JOIN file f ON f.id=sl.file_node_id WHERE sl.type=?", (typ,)):
+            if eid in funcs and eid not in loc:
+                loc[eid] = path
+
+    def module(path):
+        if not path.startswith(PCBJAM):
+            return None  # std/emsdk/deps headers — not our code
+        rel = path[len(PCBJAM):]
+        parts = rel.split("/")
+        depth = 3 if parts[0] in ("kicad", "build-wasm") else 2
+        return "/".join(parts[:depth]) if len(parts) > depth else "/".join(parts[:-1])
+
+    stats = defaultdict(lambda: [0, 0, 0])  # module -> [total, strict, lenient]
+    for nid in funcs:
+        m = module(loc.get(nid, ""))
+        if m is None:
+            continue
+        stats[m][0] += 1
+        if nid in tainted_strict:
+            stats[m][1] += 1
+        if nid in tainted_lenient:
+            stats[m][2] += 1
+
+    rows = [(m, t, s, l, t - l) for m, (t, s, l) in stats.items() if t >= 20]
+    rows.sort(key=lambda r: -r[4])
+    print(f"\n{'module':<44}{'funcs':>7}{'strict✗':>9}{'lenient✗':>9}{'clean':>7}{'clean%':>8}")
+    for m, t, s, l, clean in rows[:45]:
+        print(f"{m:<44}{t:>7}{s:>9}{l:>9}{clean:>7}{100*clean//t:>7}%")
+
+
+if __name__ == "__main__":
+    main()

sourcetrail/kicad-wasm.srctrlprj

@@ -0,0 +1,21 @@
+<?xml version="1.0" encoding="utf-8" ?>
+<config>
+    <source_groups>
+        <source_group_9a1b2c3d-0000-4000-8000-kicadwasm0001>
+            <build_file_path>
+                <compilation_db_path>./compile_commands.json</compilation_db_path>
+            </build_file_path>
+            <indexed_header_paths>
+                <indexed_header_path>/Users/V/IdeaProjects/pcbjam-private/pcbjam/kicad</indexed_header_path>
+                <indexed_header_path>/Users/V/IdeaProjects/pcbjam-private/pcbjam/wasm</indexed_header_path>
+                <indexed_header_path>/Users/V/IdeaProjects/pcbjam-private/pcbjam/wxwidgets/include</indexed_header_path>
+                <indexed_header_path>/Users/V/IdeaProjects/pcbjam-private/pcbjam/build-wasm/kicad-kicad_editor</indexed_header_path>
+                <indexed_header_path>/Users/V/IdeaProjects/pcbjam-private/pcbjam/build-wasm/wxwidgets/lib/wx/include</indexed_header_path>
+            </indexed_header_paths>
+            <name>KiCad WASM</name>
+            <status>enabled</status>
+            <type>C/C++ from Compilation Database</type>
+        </source_group_9a1b2c3d-0000-4000-8000-kicadwasm0001>
+    </source_groups>
+    <version>8</version>
+</config>

sourcetrail/sample.srctrlprj

@@ -0,0 +1,21 @@
+<?xml version="1.0" encoding="utf-8" ?>
+<config>
+    <source_groups>
+        <source_group_9a1b2c3d-0000-4000-8000-kicadwasm0001>
+            <build_file_path>
+                <compilation_db_path>./sample_compile_commands.json</compilation_db_path>
+            </build_file_path>
+            <indexed_header_paths>
+                <indexed_header_path>/Users/V/IdeaProjects/pcbjam-private/pcbjam/kicad</indexed_header_path>
+                <indexed_header_path>/Users/V/IdeaProjects/pcbjam-private/pcbjam/wasm</indexed_header_path>
+                <indexed_header_path>/Users/V/IdeaProjects/pcbjam-private/pcbjam/wxwidgets/include</indexed_header_path>
+                <indexed_header_path>/Users/V/IdeaProjects/pcbjam-private/pcbjam/build-wasm/kicad-kicad_editor</indexed_header_path>
+                <indexed_header_path>/Users/V/IdeaProjects/pcbjam-private/pcbjam/build-wasm/wxwidgets/lib/wx/include</indexed_header_path>
+            </indexed_header_paths>
+            <name>KiCad WASM sample</name>
+            <status>enabled</status>
+            <type>C/C++ from Compilation Database</type>
+        </source_group_9a1b2c3d-0000-4000-8000-kicadwasm0001>
+    </source_groups>
+    <version>8</version>
+</config>

sourcetrail/transform_compile_db.py

@@ -0,0 +1,151 @@
+#!/usr/bin/env python3
+"""Transform the in-container emscripten compile_commands.json into one
+Sourcetrail 2021.4.19 (bundled clang ~11) can index on the host.
+
+- expands @CMakeFiles/....rsp response files inline
+- strips emscripten-only and PCH flags (old clang can't load clang-20 .pch)
+- replaces the PCH with `-include cmake_pch.hxx` so those headers still parse
+- injects libc++ 11 headers (-nostdinc++) + emscripten sysroot includes
+- rewrites /workspace -> host pcbjam, /emsdk -> host tools/emsdk mirror
+
+Usage: transform_compile_db.py [--no-wasm-target] <in.json> <out.json>
+"""
+import json
+import os
+import re
+import shlex
+import sys
+
+PCBJAM = "/Users/V/IdeaProjects/pcbjam-private/pcbjam"
+ST_DIR = f"{PCBJAM}/sourcetrail"
+EMSDK_HOST = f"{PCBJAM}/tools/emsdk"
+
+PATH_MAP = [("/workspace/", PCBJAM + "/"), ("/emsdk/", EMSDK_HOST + "/")]
+
+DROP_EXACT = {
+    "-fwasm-exceptions",
+    "-Winvalid-pch",
+    "--emit-symbol-map",
+}
+DROP_PREFIX_RE = re.compile(r"^-s[A-Z][A-Z_0-9]*(=.*)?$|^-gseparate-dwarf")
+
+
+def host_path(p: str) -> str:
+    for src, dst in PATH_MAP:
+        if p.startswith(src):
+            return dst + p[len(src):]
+    return p
+
+
+def rewrite_tok(tok: str) -> str:
+    for src, dst in PATH_MAP:
+        tok = tok.replace(src, dst)
+    return tok
+
+
+def expand_rsp(tokens, directory):
+    out = []
+    for tok in tokens:
+        if tok.startswith("@"):
+            rsp = tok[1:]
+            if not os.path.isabs(rsp):
+                rsp = os.path.join(directory, rsp)
+            rsp_host = host_path(rsp)
+            with open(rsp_host) as f:
+                out.extend(shlex.split(f.read()))
+        else:
+            out.append(tok)
+    return out
+
+
+def transform(tokens, directory, wasm_target=True):
+    tokens = expand_rsp(tokens, directory)
+
+    argv0 = tokens[0]
+    lang_cxx = argv0.endswith("++")
+    out = []
+
+    i = 1
+    pch_headers = []
+    while i < len(tokens):
+        tok = tokens[i]
+        if tok == "-Xclang" and i + 1 < len(tokens):
+            nxt = tokens[i + 1]
+            if nxt == "-fno-pch-timestamp":
+                i += 2
+                continue
+            if nxt == "-include-pch":
+                # -Xclang -include-pch -Xclang <path.pch>
+                if i + 3 < len(tokens) and tokens[i + 2] == "-Xclang":
+                    pch = tokens[i + 3]
+                    hdr = pch[:-4] if pch.endswith(".pch") else pch
+                    if os.path.exists(host_path(hdr)):
+                        pch_headers.append(hdr)
+                    i += 4
+                    continue
+                i += 2
+                continue
+            out.extend([tok, nxt])
+            i += 2
+            continue
+        if tok in DROP_EXACT or DROP_PREFIX_RE.match(tok):
+            i += 1
+            continue
+        if not wasm_target and tok in ("-matomics", "-mbulk-memory", "-pthread"):
+            i += 1
+            continue
+        out.append(tok)
+        i += 1
+
+    for hdr in pch_headers:
+        out.extend(["-include", hdr])
+
+    # Rewrite container paths in the ORIGINAL tokens only, then prepend the
+    # host-path injections — rewriting after injection would re-fire on the
+    # /emsdk/ substring inside the host tools/emsdk mirror path.
+    out = [rewrite_tok(t) for t in out]
+
+    inject = ["-isystem", f"{EMSDK_HOST}/upstream/emscripten/cache/sysroot/include",
+              "-fexceptions"]
+    if lang_cxx:
+        inject = ["-nostdinc++", "-isystem", f"{ST_DIR}/libcxx-11/include"] + inject
+    if wasm_target:
+        inject = ["--target=wasm32-unknown-emscripten"] + inject
+
+    return ["clang++" if lang_cxx else "clang"] + inject + out
+
+
+def main():
+    args = sys.argv[1:]
+    wasm_target = True
+    if args and args[0] == "--no-wasm-target":
+        wasm_target = False
+        args = args[1:]
+    src, dst = args
+
+    with open(src) as f:
+        db = json.load(f)
+
+    out_db = []
+    missing = 0
+    for e in db:
+        tokens = shlex.split(e["command"])
+        directory = e["directory"]
+        new_tokens = transform(tokens, directory, wasm_target)
+        file_host = host_path(e["file"])
+        if not os.path.exists(file_host):
+            missing += 1
+            continue
+        out_db.append({
+            "directory": host_path(directory),
+            "command": " ".join(shlex.quote(t) for t in new_tokens),
+            "file": file_host,
+        })
+
+    with open(dst, "w") as f:
+        json.dump(out_db, f, indent=1)
+    print(f"wrote {len(out_db)} entries to {dst} ({missing} skipped: file missing on host)")
+
+
+if __name__ == "__main__":
+    main()