AI_Assistant/pcbjam
Watch
1
0
Fork
You've already forked pcbjam
 0
Code Issues Pull requests Projects Releases 7 Packages Wiki Activity Actions

docs(yjs-bridge): add collaborative Yjs bridge design

Browse source 
Approved design for porting KiCad-wasm editors to collaborative editing via Yjs:
- 0001 general (tool-agnostic) bridge contract: structured CRDT, unified
  ChangeSource + apply, the 3 serialization mechanisms, 4-commit phasing
- 0002 pl_editor (first PoC): identity, snapshot-differ adapter, field mapping
- 0003 eeschema, 0004 pcbnew: native-listener/COMMIT adapters

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Gergő Törcsvári 2026-06-03 13:21:24 +02:00
commit f94114efb8
No known key found for this signature in database
GPG key ID: 8E75F2CDE64E5322
4 changed files with 515 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

228
features/yjs-bridge/0001-general-bridge-design.md Normal file
View file

@ -0,0 +1,228 @@
# Yjs collaborative bridge — general design (tool-agnostic)
Branch: `feature/yjs-bridge` (root + kicad + wxwidgets).
Goal: a bidirectional bridge between a KiCad editor's in-memory document model and a Yjs
`Y.Doc`, so local UI edits become CRDT ops and remote CRDT ops become model mutations.
This file is the **shared architecture**. Per-tool specifics live in:
- `0002-pl-editor-bridge.md` — drawing-sheet editor (first PoC).
- `0003-eeschema-bridge.md` — schematic editor.
- `0004-pcbnew-bridge.md` — PCB editor.
Each tool doc states **only its differences** from this file.
Decisions taken (2026-06-03):
- **CRDT representation: structured** — `Y.Array` of one `Y.Map` per item (stable id + fields).
- **First PoC transport: in-browser, no server** — two tabs share a `Y.Doc` via
`BroadcastChannel`/`y-webrtc`. Server persistence is a later milestone.
- **First PoC target: pl_editor.**
- **One unified bridge contract for all three tools — no profile split** (see §1).
---
## 1. The unified bridge contract (one shape, per-tool adapters)
There is **one** bridge contract, identical for every tool. The C++ side exposes exactly two
things, and the JS side is fully generic:
```
┌───────────────────────── identical across all tools ─────────────────────────┐
C++ editor → ChangeSource → per-item delta JSON → reconciler.ts → Y.Doc(items: Y.Array<Y.Map>)
C++ editor ← apply(delta) ← per-item delta JSON ← reconciler.ts ← Y.Doc ← provider ← peer
└─ per-tool adapter ─┘ └──────────── generic, schema-agnostic ──┘
```
- **`ChangeSource` (emit):** produces a stream of **per-item change events** — `added` /
`removed` / `changed`, each carrying `{id, type, …fields}`. Emitted to JS as delta JSON.
- **`apply(delta)`:** consumes the same per-item delta JSON and applies it to the model.
Both directions speak the **same per-item vocabulary** for every tool. What differs is only the
**per-tool adapter** that *implements* `ChangeSource`/`apply` — an internal detail invisible to
the wire format, the JS reconciler, and the other tools:
| Tool | `ChangeSource` implementation | `apply` implementation | Identity |
|------|-------------------------------|------------------------|----------|
| eeschema | native `SCHEMATIC_LISTENER` | `SCH_COMMIT` Add/Modify/Remove + Push | native `KIID` |
| pcbnew | native `BOARD_LISTENER` (`OnBoardCompositeUpdate`) | `BOARD_COMMIT` Add/Modify/Remove + Push | native `KIID` |
| pl_editor | **in-bridge snapshot-differ** fired from `OnModify` | per-item mutate `DS_DATA_MODEL` by uuid + `SyncDrawItems` | **added** `KIID` (`0002`) |
eeschema/pcbnew get per-item events *natively* from their commit/listener systems. pl_editor
has none, so its adapter **derives** the same per-item events by uuid-diffing successive
whole-model snapshots inside the bridge (trivial at its scale) — and applies per-item by uuid.
The derivation is encapsulated in pl_editor's adapter; nothing downstream can tell the
difference. (We deliberately do **not** rewrite pl_editor's edit/undo core to a real `COMMIT` —
that would diverge heavily from upstream for no runtime benefit; decided 2026-06-03.)
So the only real per-tool work is: (1) pl_editor must **add** `KIID`+`(uuid)` to its format
(§2, `0002`); (2) eeschema/pcbnew must run the correct **recompute** on apply (connectivity /
ERC / ratsnest / hierarchy) by going through their native `COMMIT`.
---
## 2. Item identity (shared semantics)
A structured CRDT needs a per-item key that is (a) **identical on every client** and
(b) **stable across undo/redo**. The only way to guarantee both is for the id to live in the
**serialized form** of the item (so re-parsing — which undo does — preserves it, and two
clients opening the same file derive the same ids):
- **eeschema/pcbnew:** satisfied natively. Items derive from `EDA_ITEM`
(`include/eda_item.h:516` → `const KIID m_Uuid`) and every item writes `(uuid …)`. Nothing to
do; the uuid flows through KiCad ↔ JSON ↔ Y.Doc as the single source of truth, with **no
cross-layer identity assumptions**.
- **pl_editor:** *not* satisfied natively — items have no `KIID` and `.kicad_wks` has no uuid
token. We add **the same `KIID` + `(uuid …)`** the others use (not a bespoke id), so the three
tools share one identity scheme (`0002`). Once added, identity behaves identically.
New items minted locally get a fresh `KIID`; it appears in the next emit, the reconciler keys
a new `Y.Map` by it, and remote peers create the item with that same uuid on apply.
**Backfill caveat for files lacking uuids.** When a tool *adds* uuids to a format (pl_editor),
legacy/upstream files have none, so they're backfilled on load. `KIID` backfill is **random,
not a deterministic content-hash** — so two clients cold-opening the *same never-saved* file
would disagree. Resolve via the **seed-once rule**: the client that finds the `Y.Doc` empty
seeds it and its backfilled uuids win; joiners adopt the `Y.Doc` instead of their own parse.
The first save then persists uuids, making all later opens consistent. (Deterministic
content-seed backfill is an optional later hardening; see `0002`.)
---
## 3. Wire schema + representation granularity
**The contract is minimal and schema-agnostic:** an array of objects, each with a string `id`
and `type`, plus arbitrary additional fields. The JS reconciler hardcodes **no** field names.
Each item's fields can be represented at one of two granularities — **chosen per item-type on
the C++ side; the JS reconciler is identical either way** (it just diffs values keyed by field
name):
- **Decomposed scalars** — `{id, type, x, y, text, …}`. Enables *field-level* merge (two
people editing different fields of the same item never conflict). Best for small/hot types.
- **Opaque blob** — `{id, type, sexpr: "(segment …)"}` reusing the tool's native per-item
serializer. *Item-level* merge granularity (concurrent edits to the same item conflict).
Near-zero C++ mapping. Best for the long tail of complex types.
### Where the field values come from (don't hand-write N serializers)
There are three C++ mechanisms to produce the JSON for an item, in decreasing genericity. Pick
per tool; a tool may mix them and the JS reconciler is unaffected:
1. **`PROPERTY_MANAGER` reflection — write-once, all `EDA_ITEM` types (the "do it once" lever).**
`EDA_ITEM`-based items register their fields generically (`REGISTER_TYPE` + `AddProperty`,
e.g. `pcbnew/board_item.cpp:427+` — *"Position X"*, *"Layer"*, *"Locked"*, …). Enumerate
`propMgr.GetProperties(item->GetTypeId())`, read each via `PROPERTY_BASE::GetValue → wxAny`
→ JSON; write back via `PROPERTY_BASE::Set`. **One converter covers every registered type**
with field-level merge. Cost paid once: a `wxAny↔JSON` type bridge (ints/strings/bools,
enums, `EDA_ANGLE`/`VECTOR2I` units). Coverage isn't 100% — a few unregistered fields still
need (3). Primary path for **pcbnew** (broad coverage) and **eeschema** (partial).
2. **Native per-item S-expr blob — write-once *per tool that has one*.** pcbnew
`PCB_IO_KICAD_SEXPR::Format(BOARD_ITEM*)`; pl_editor `DS_DATA_MODEL_IO::Format(model, item)`
/ `SaveInString({item}, &str)`. Serializes *any* of that tool's items with no per-type code;
item-level merge only. (eeschema has no clean per-item formatter — document-level only.)
3. **Hand-mapped per-type fields.** Explicit, total control, works everywhere (incl. pl_editor,
which has neither reflection nor `EDA_ITEM`). Most code — reserve for tools with few types
(pl_editor's 5) or the hot fields you want field-level merge on.
(There is also `SERIALIZABLE`/`Serialize→protobuf::Any` on `EDA_ITEM`, but it targets the IPC
API — protobuf, partial coverage — wrong fit for JSON; not used here.)
A tool may mix granularities/mechanisms and refine over time **without any JS change**. Nested
data (polygon corner lists, bitmap blobs) is an opaque blob field or a nested `Y.Array`/`Y.Map`.
Numeric coordinates use the model's native units; the reconciler **quantizes before comparing**
to avoid spurious float diffs.
Optional **Zod schema as a validation guard only** at the apply boundary (reject garbage,
coerce number/bool/string) — never as a structural requirement.
---
## 4. The JS reconciler (`web/.../collab/`, identical for all tools)
A single generic module, parameterized only by the `Y.Doc` and the two bridge entry points:
- **Down (model → Y):** keep the last-seen item-set keyed by `id`. On each emitted per-item
delta, write **only** the changes into the `Y.Array<Y.Map>`, inside one `Y.transaction`
tagged with a local origin. (If an adapter ever emits a whole snapshot instead of a delta —
it shouldn't, but pl_editor's differ could — the reconciler still diffs it to minimal ops, so
the wire stays granular regardless.)
- **Up (Y → model):** observe the `Y.Array`; on remote-origin events, build the changed-item
delta JSON and call the C++ `apply` hook. Ignore events whose origin is our own tag
(standard Yjs echo-suppression).
- **Generic mapping:** an item ⇆ `Y.Map` is a flat copy of scalar fields; the diff is
value-equality per key. Adding a C++ field needs zero JS change.
The reconciler is **adapter-blind** — it only ever sees per-item deltas keyed by uuid.
---
## 5. Loop / re-entrancy safety (shared)
- **`s_applyingRemote` guard** in the C++ bridge: set around `apply`; emit is suppressed while
set. Prevents apply → (OnModify | listener fire) → emit → Y write → observe → apply … loops.
Needed by every adapter (a native listener would otherwise re-fire on our own apply; the
pl_editor differ would otherwise re-diff our own apply).
- **Asyncify timing:** apply is invoked from a JS callback and runs synchronously on the main
thread between frames (same context as the existing `OnFileDropped`/`OpenFileCallback`
hooks). It must not run mid-Asyncify-unwind — gate applies on the tool's ready/idle state and
queue any that arrive during a suspension.
- **Yjs origin tags** on all reconciler transactions; observers skip their own origin.
---
## 6. Transport & persistence (shared, phased)
- **PoC:** two tabs, one `Y.Doc`, `BroadcastChannel` (or `y-webrtc`). Zero backend work.
- **Later (M-server):** `y-websocket` provider (standalone or folded into the Fastify server in
`web/apps/server`) for networked sync + persistence; periodically materialize the Y.Doc back
to the canonical `.kicad_*` file via the tool's normal save path.
---
## 7. Shared C++ bridge interface
A thin, `#ifdef __EMSCRIPTEN__`-guarded interface each tool implements, so JS is uniform:
- **Emit → JS:** `EM_ASM({ window.kicadCollab.onDelta(UTF8ToString($0)) }, json)` — `json` is a
per-item delta. Same JS entry point for every tool/adapter.
- **Apply ← JS:** `EMSCRIPTEN_KEEPALIVE void kicadCollabApply(const char* json)` — dispatches to
the active tool's adapter; wrapped in the `s_applyingRemote` guard.
- Copy the proven wx-layer pattern (`EM_ASM` out, `ccall`→`EMSCRIPTEN_KEEPALIVE` in).
- The collab code that touches tool models lives in the **kicad** fork; keep it tightly guarded
and run `kicad-diff-stats.sh`. eeschema/pcbnew adapters add almost no surface (listener
subclass + COMMIT calls). pl_editor adds the most: the `(uuid)` format token + the snapshot-
differ adapter — but still leaves its edit/undo core untouched.
---
## 8. Phasing — one commit per step
1. **pl_editor interface/identity change.** `KIID m_Uuid` on `DS_DATA_ITEM` + `(uuid …)`
keyword/parse(backfill)/save via the shared `FormatUuid`. Independently testable (uuid
round-trip + legacy-file backfill-on-save). Commit. (`0002` §identity)
2. **pl_editor bridge + JSON converters (all 5 item types) + tests.** The snapshot-differ
`ChangeSource`, per-item `apply` by uuid, the generic JS reconciler, `BroadcastChannel`
transport, two-tab demo. Text/line/rect as scalars, poly/bitmap as blobs (`0002`). Commit.
3. **eeschema JSON converters + bridge + tests.** Adapter = `SCHEMATIC_LISTENER` + `SCH_COMMIT`;
converters reflection-first (§3 mechanism 1) + hand-map gaps; multi-sheet hierarchy + symbol
instances. Commit. (`0003`)
4. **pcbnew JSON converters + bridge + tests.** Adapter = `BOARD_LISTENER` + `BOARD_COMMIT`;
converters = reflection (1) for the bulk + native blob (2) for the long tail; mandatory
connectivity/ratsnest recompute; large-board batched initial-sync. Commit. (`0004`)
5. **Later — y-websocket + persistence** across all tools.
pl_editor goes first (steps 1–2) because its adapter is the only one we write from scratch and
it carries the least that can break (no connectivity, tiny docs); once the contract holds
there, the eeschema/pcbnew adapters are a thin re-use over native machinery.
---
## 9. Cross-cutting risks
- **Asyncify timing** of apply (§5) — needs a real under-load test.
- **Representation-granularity drift** — mixing blob and scalar per type must stay consistent
across clients (the `type`→granularity decision lives in C++ and must be deterministic).
- **Coordinate precision** — quantize before diffing (§3).
- **Recompute side-effects (eeschema/pcbnew)** — applying outside the normal UI flow must still
trigger connectivity/ERC/ratsnest/hierarchy; always go through the native COMMIT.
- **Initial sync cost (pcbnew)** — seeding a 100k-item board into a fresh `Y.Doc` is itself
heavy; needs a batched/own-phase strategy (see `0004`).
- **Fork divergence** — pl_editor changes the `.kicad_wks` file format (fork-only files; see
`0002`). eeschema/pcbnew adapters are near-zero divergence.

121
features/yjs-bridge/0002-pl-editor-bridge.md Normal file
View file

@ -0,0 +1,121 @@
# pl_editor (drawing-sheet editor) — bridge specifics
Differences from `0001-general-bridge-design.md`. pl_editor implements the **same unified
bridge contract** as the other two (per-item `ChangeSource` + per-item `apply`); it is the
**first PoC**. It has none of the native machinery, so its `ChangeSource` is the one adapter we
write from scratch — an **in-bridge snapshot-differ** — while its edit/undo core stays
untouched (low upstream divergence; Route 1, decided 2026-06-03).
## What pl_editor lacks natively (why its adapter is custom)
- `DS_DATA_ITEM` does **not** derive from `EDA_ITEM` (`include/drawing_sheet/ds_data_item.h:95`)
→ no `KIID`, and the `COMMIT` base operates on `EDA_ITEM*` so it can't take data items.
- **No COMMIT** anywhere in `pagelayout_editor/` (0 uses); undo is a whole-model S-expr snapshot
(`SaveCopyInUndoList` → `DS_PROXY_UNDO_ITEM`) wired at 13 sites across 8 files.
- **No listener/observer** of any kind.
- Single change chokepoint: `PL_EDITOR_FRAME::OnModify()` (`pl_editor_frame.cpp:379`).
We do **not** rewrite that core. Instead the adapter derives per-item events by diffing, and
applies per-item by uuid — producing the exact same observable behavior as `SCH_COMMIT`/
`BOARD_COMMIT`, encapsulated entirely in the bridge.
## Model
- Singleton `DS_DATA_MODEL::GetTheInstance()`; `std::vector<DS_DATA_ITEM*> m_list`
(`include/drawing_sheet/ds_data_model.h`). Flat, single-document — no sheets/screens, no
connectivity/ERC (a big simplification vs eeschema/pcbnew).
- Item types: `DS_TEXT`, `DS_SEGMENT`, `DS_RECT`, `DS_POLYPOLYGON`, `DS_BITMAP`.
- `DS_DRAW_ITEM_*` render proxies are rebuilt by `DS_DATA_ITEM::SyncDrawItems(view)`; ephemeral.
## Identity — add the SAME `KIID`/`(uuid …)` as eeschema/pcbnew
Use the identical type + token + writer (general §2), not a bespoke id:
- Add `KIID m_Uuid;` to `DS_DATA_ITEM` (`include/kiid.h`). Default-constructed → **random**, so
a new or uuid-less item is auto-backfilled in memory for free.
- Serialize with the shared helper `KICAD_FORMAT::FormatUuid( m_out, item->m_Uuid )`
(`include/io/kicad/kicad_io_utils.h:55`) in `ds_data_model_io.cpp`, exactly as sch/pcb do.
### Save + load + backfill
- **Format:** add `uuid` to `common/drawing_sheet/drawing_sheet.keywords` (absent today). The
wks lexer is **strict** (unknown token → `default:` → `Expecting()`/throws), so the token must
be declared + handled — no silent passthrough.
- **Load:** `case T_uuid: item->m_Uuid = KIID( FromUTF8() ); NeedRIGHT(); break;` in each item
parser (`drawing_sheet_parser.cpp`). If the token is absent, the parser leaves the default
random `m_Uuid` in place → **automatic backfill**.
- **Save:** always write `(uuid …)` per item → the first save of a legacy/upstream uuid-less
`.kicad_wks` **upgrades the file** in place.
- **Cross-client consistency:** files with uuids (our output, and any file after one save) →
all clients agree. A never-saved uuid-less legacy file gets *random* backfill
(`KIID(invalid)` → random, confirmed in `common/kiid.cpp` — not a deterministic hash), so two
cold opens disagree; handled by the **seed-once rule** (general §2). Optional deterministic
content-seed backfill (boost `name_generator` over type+name+pos+index) is deferred hardening.
- **Tradeoff:** files now carry `(uuid …)`; stock desktop KiCad's strict parser rejects the
unknown token → fork-only files. Backfill makes the upgrade graceful.
## `ChangeSource` adapter — in-bridge snapshot-differ (emits per-item deltas)
The bridge holds the **last emitted snapshot** (a `uuid → fields` map). On `OnModify()`:
1. Serialize current `m_list` to a `uuid → fields` map — **all 5 types, none invisible**
(rationale below): text/segment/rect as scalars; poly/bitmap as an opaque S-expr blob.
2. **Diff vs the cached map by uuid:** present-only-in-new → `added`; only-in-old → `removed`;
in-both-with-changed-fields → `changed`.
3. Emit the per-item delta JSON via `EM_ASM window.kicadCollab.onDelta(json)`; update the cache.
This yields the same `added/removed/changed` per-item events the native listeners produce. The
diff is O(items) — negligible for drawing sheets. *(We could instead call the differ directly
from `OnModify`; optionally wrap it as a one-method `DS_DATA_MODEL` listener purely for
registration symmetry with sch/pcb — nice-to-have, not required.)*
## `apply(delta)` — per-item mutation by uuid
For each op in the incoming delta (inside the `s_applyingRemote` guard, general §5):
- **changed:** find `DS_DATA_ITEM` by `m_Uuid` in `m_list`; set the changed fields.
- **added:** create the item (with the given uuid), `DS_DATA_MODEL::Append`.
- **removed:** find by uuid, `Remove` + delete.
Then `SyncDrawItems` on touched items + canvas refresh, and **re-select by uuid** so a remote
apply doesn't drop local selection (a bonus the uuid unlocks). Optionally
`SaveCopyInUndoList()` once after a remote batch so local undo has a coherent checkpoint. **No
connectivity/ERC** — none exists for drawing sheets.
## Text-insert path (working UI flow targeted first)
`PL_DRAWING_TOOLS::PlaceItem` (`tools/pl_drawing_tools.cpp:72`) → `AddDrawingSheetItem(DS_TEXT)`
(`pl_editor_frame.cpp:823`) → `new DS_DATA_ITEM_TEXT("Text")` (fresh `KIID`) → `Append` →
`SyncDrawItems` → `SaveCopyInUndoList()` → `OnModify()` ← differ runs here → emits an `added`.
## Field mapping — all 5 types in one pass, at two fidelities
Cover every type so none is invisible to collaboration (see rationale). pl_editor has no
reflection (`DS_*` is not `EDA_ITEM`), so scalars are hand-mapped (general §3 mechanism 3) and
the awkward two reuse the existing per-item serializer (mechanism 2):
Shared: `id`(m_Uuid) · `type` · `name`(m_Name) · `x,y`(m_Pos.m_Pos, mm) · `anchor`(m_Pos.m_Anchor)
- **text** (scalars): `text`(m_TextBase) · `orient`(°) · `hjustify`/`vjustify` · `italic` ·
`bold` · `sizeX,sizeY`(m_TextSize, mm) — field-level merge.
- **segment/rect** (scalars): `ex,ey`(m_End.m_Pos) · `eanchor` · `linewidth`(m_LineWidth).
- **poly/bitmap** (opaque blob): `sexpr` field from `DS_DATA_MODEL_IO::Format(model, item)` /
`SaveInString({item}, &str)` — *no* nested corner-list / base64 field mapper to write;
item-level merge. Refine to scalars later only if field-level merge is ever wanted.
**Why all 5, not just text/line/rect:** the adapter is a whole-model differ, so a type absent
from the snapshot is **invisible** — a user adding a polygon or logo bitmap simply wouldn't
sync. Including poly/bitmap as blobs closes that correctness gap at near-zero cost (the
serializer already exists). **Bitmap is safe** despite its large base64 payload: the differ
emits a field only when its value changes (string compare), so a static image is cached once
and never re-emitted — just keep it out of any per-keystroke re-encode path.
On **apply**, a blob item is reconstructed by parsing its `sexpr` back into a `DS_DATA_ITEM`
(wrap as a minimal model string → reuse `SetPageLayout`/the parser; a tiny parse-one helper).
## Build steps (maps to phasing commits 1–2 in general §8)
**Commit 1 — interface/identity:** `KIID m_Uuid` on `DS_DATA_ITEM` + `(uuid)`
keyword/parse(backfill)/save via `FormatUuid`. Build once; verify a `.kicad_wks` round-trips
uuids and a legacy file is backfilled on save.
**Commit 2 — bridge + converters (all 5 types) + tests:**
1. `common/drawing_sheet/ds_collab_bridge.{h,cpp}` (`#ifdef __EMSCRIPTEN__`): the snapshot-differ
`ChangeSource` (cache + diff, emit on `OnModify`), `kicadCollabApply` per-item-by-uuid
(incl. blob parse-one for poly/bitmap), `s_applyingRemote` guard.
2. Web: `yjs` + `BroadcastChannel` provider; the generic reconciler (general §4) + `WasmTool`
wiring; optional Zod guard. Two-tab demo.
3. Verify: insert/move text & line & rect in tab A → mirrors in tab B and back; poly/bitmap
round-trip via blob; selection survives remote apply (uuid re-select); e2e + screenshots.
## Risks specific to pl_editor
- Carries the **format-change divergence** (general §9) — the only tool that does.
- Its `ChangeSource` is a **derived** differ (not native tracking). Correct and cheap here, but
the differ pattern is pl_editor's alone — eeschema/pcbnew use native events.
- Never-saved legacy-file cold-open identity race — mitigated by seed-once; deterministic
backfill deferred.

88
features/yjs-bridge/0003-eeschema-bridge.md Normal file
View file

@ -0,0 +1,88 @@
# eeschema (schematic editor) — bridge specifics
Differences from `0001-general-bridge-design.md`. eeschema implements the **same unified bridge
contract** (general §1); its `ChangeSource`/`apply` adapter is a thin re-use of **native**
machinery, so most of pl_editor's hard work (`0002`) disappears here. Two genuinely new things:
**multi-sheet hierarchy** and **symbol instances**.
## Adapter — native (no derived differ, no format change) — and *why*
eeschema has the full machinery natively:
- **Identity free:** `SCH_ITEM` ⊂ `EDA_ITEM` → `const KIID m_Uuid` (`include/eda_item.h:516`),
serialized as `(uuid …)` on every item by `SCH_IO_KICAD_SEXPR`
(`eeschema/sch_io/kicad_sexpr/sch_io_kicad_sexpr.cpp:745,1047,1189,1322,…`; parsed in
`…_parser.cpp:2777,3476,…`). **No format change** (contrast `0002`).
- **Granular change set:** `SCH_COMMIT` (`eeschema/sch_commit.h`), `Push()` loop
(`eeschema/sch_commit.cpp:228–397`) tags each entry `CHT_ADD` / `CHT_REMOVE` / `CHT_MODIFY`.
- **Native observer — hook without patching core:** `SCHEMATIC_LISTENER`
(`eeschema/schematic.h:60–71`):
```cpp
virtual void OnSchItemsAdded ( SCHEMATIC&, std::vector<SCH_ITEM*>& );
virtual void OnSchItemsRemoved ( SCHEMATIC&, std::vector<SCH_ITEM*>& );
virtual void OnSchItemsChanged ( SCHEMATIC&, std::vector<SCH_ITEM*>& );
virtual void OnSchSheetChanged ( SCHEMATIC& );
```
Register via `schematic->AddListener(bridge)` (`schematic.h:359`). `SCH_COMMIT::Push` fires
these in bulk (`sch_commit.cpp:399–417`). → **emit is a listener subclass; near-zero fork
divergence.**
So: no derived differ and no format patch here. The `ChangeSource` is the listener; `apply`
goes through `SCH_COMMIT`.
## Model — multi-sheet (the main structural difference)
- `SCHEMATIC` (`eeschema/schematic.h:87`) → virtual `m_rootSheet`, `m_topLevelSheets`,
current `m_currentSheet`. Each `SCH_SHEET` owns one `SCH_SCREEN` whose items live in an
`EE_RTREE` (`eeschema/sch_screen.h:98–117`).
- **Yjs shape:** because `m_Uuid` is globally unique, a **flat uuid-keyed `Y.Array`** still
works, but each item's `Y.Map` must carry a **`screen`/`sheetPath` field** so apply knows
which `SCH_SCREEN` to target. (Alternative: a `Y.Map` of screens, each a `Y.Array` of items
— more structure, mirrors the model; decide during the eeschema commit. Flat+scope-field is
simpler to start.)
- `OnSchSheetChanged` / hierarchy refresh must propagate as its own doc field/event.
## Symbol instances (the other new wrinkle)
A `SCH_SYMBOL` placed in a shared sheet appears on multiple sheet *instances*; per-instance
data (reference, unit) lives in `m_instanceReferences` (`SCH_SYMBOL_INSTANCE`). Editing the
shared symbol affects all instances; a remote change to reference/unit must update the right
**instance record**, not just the symbol. Treat instance data as nested fields keyed by
sheet-path within the symbol's item.
## ChangeSource (emit)
`SCHEMATIC_LISTENER` subclass → on `OnSchItemsAdded/Removed/Changed`, emit per-item JSON.
**Converter strategy: reflection-first** (general §3 mechanism 1) — `SCH_ITEM : EDA_ITEM`, so
use `PROPERTY_MANAGER` to serialize registered fields write-once; **hand-map (mechanism 3) the
gaps** where properties aren't registered. (Blobs are *not* an option here — see serialization
note.) Include the `screen`/`sheetPath` scope on every item.
## apply — go through SCH_COMMIT
```cpp
SCH_COMMIT commit( toolMgr );
// add: create SCH_ITEM, commit.Add(item, screen)
// modify: item = schematic->ResolveItem(uuid); commit.Modify(item, screen); …edit fields…
// remove: commit.Remove(item, screen)
commit.Push( "collab" ); // triggers the recompute below
```
`Push` (`sch_commit.cpp`) runs the required side-effects:
- **Connectivity/ERC/netlist:** `frame->RecalculateConnections(...)` when any connectable item
changed (`HasConnectivityChanges`); local vs global cleanup chosen automatically.
- **Hierarchy:** `schematic->RefreshHierarchy()` + nav update when a `SCH_SHEET` changed.
## Apply gotchas (enumerated)
- Connectivity is **bidirectional** — adding/removing a wire re-nets connected pins; never skip
`RecalculateConnections`.
- **Symbol instances** — update the per-sheet instance record, not only the shared symbol.
- **Groups** — keep parent/child links in sync (`group->AddItem/RemoveItem`).
- **`SCH_FIELD` remove is special** — it hides, not deletes (`sch_commit.cpp:308`).
- **ERC markers** are uuid-tied; deletes orphan them (respect exclusions).
- **Selection** is transient UI state — never propagate.
## Serialization note (differs from pcbnew)
eeschema's I/O plugin is **document-level**, not per-item — there is **no** clean public
`Format(single SCH_ITEM)`, so the opaque-S-expr-blob path (general §3 mechanism 2) is *not*
available here. This is exactly why the converter is **reflection-first + hand-mapped gaps**:
fields via `PROPERTY_MANAGER` (or accessors), and on apply build complex items via `Clone()` +
field setters rather than re-parsing a blob.
## Scale
~500–5000 items typical (10k+ for large hierarchical designs). Granular deltas (listener) are
strongly preferred; whole-document serialization is expensive (hierarchy + uuid resolve +
connectivity). A pl_editor-style derived differ would be wrong at this scale — use the listener.

78
features/yjs-bridge/0004-pcbnew-bridge.md Normal file
View file

@ -0,0 +1,78 @@
# pcbnew (PCB editor) — bridge specifics
Differences from `0001-general-bridge-design.md`. pcbnew implements the **same unified bridge
contract** (general §1) with the **richest** native hooks of the three. Two things dominate its
specifics: **mandatory connectivity/ratsnest recompute** and **scale** (10k–100k+ items).
## Adapter — native (richest of the three) — and *why*
- **Identity free:** `BOARD_ITEM` ⊂ `EDA_ITEM` → `const KIID m_Uuid` (`include/eda_item.h:516`),
serialized as `(uuid …)` on every item by `PCB_IO_KICAD_SEXPR` (e.g. track
`pcb_io_kicad_sexpr.cpp:2619`, footprint `:1170`, zone `:2647`). Fast lookup via
`std::unordered_map<KIID, BOARD_ITEM*> m_itemByIdCache` (`board.h:1458`). **No format change.**
- **Granular change set:** `BOARD_COMMIT::Push` (`pcbnew/board_commit.cpp:176–631`) tags each
entry `CHT_ADD/REMOVE/MODIFY` and collects bulk add/remove/changed vectors.
- **Native observer — hook without patching core:** `BOARD_LISTENER` (`board.h:283–301`) is the
richest interface — singular *and* bulk callbacks, plus a combined one:
```cpp
OnBoardItemAdded/­ItemsAdded · OnBoardItemRemoved/­ItemsRemoved
OnBoardItemChanged/­ItemsChanged · OnBoardNetSettingsChanged
OnBoardHighlightNetChanged · OnBoardRatsnestChanged
OnBoardCompositeUpdate(board, added[], removed[], changed[]) // ← one event, all three sets
```
Registered via `board->AddListener(bridge)` (`board.cpp:2964`); fired through
`InvokeListeners(...)` (`board.h:1424`). `BOARD_COMMIT::Push` → `OnItemsCompositeUpdate`
(`board_commit.cpp:580` → `board.cpp:3001`). **`OnBoardCompositeUpdate` is the ideal single
emit hook.**
## Model — typed containers (vs flat list / vs rtree)
8 typed containers, not one list (`board.h:1447–1455`): `m_footprints`, `m_tracks`, `m_zones`,
`m_drawings`, `m_groups`, `m_generators`, `m_markers`, `m_points`. The Yjs shape is still a
**flat uuid-keyed `Y.Array`** — `type` tells apply which container `BOARD::Add` routes to;
`m_itemByIdCache` resolves uuid→item in O(1).
## ChangeSource (emit)
`BOARD_LISTENER::OnBoardCompositeUpdate` → emit per-item JSON for added/changed (uuid+type+
fields) and uuid-only for removed. **Converter strategy: reflection for the bulk + native blob
for the long tail** — pcbnew uniquely has *both* generic mechanisms (general §3):
- (1) `BOARD_ITEM` is broadly `INSPECTABLE` (`board_item.cpp:427+`) → `PROPERTY_MANAGER`
serializes most fields **write-once** with field-level merge.
- (2) `PCB_IO_KICAD_SEXPR::Format(const BOARD_ITEM*)` (`pcb_io_kicad_sexpr.h:351`, per-type
`format(PCB_TRACK*/PAD*/ZONE*/…)`) gives a **free opaque blob** for any type reflection
misses.
So: reflection-decompose the hot types you want field-level merge on (track width/endpoints,
position), blob everything else — **no hand-written per-type serializers**.
## apply — go through BOARD_COMMIT; recompute is MANDATORY
```cpp
BOARD_COMMIT commit( toolMgr );
// add: item = deserialize(...); commit.Add(item)
// modify: item = board->GetItem(uuid); commit.Modify(item); …edit…
// remove: commit.Remove(board->GetItem(uuid))
commit.Push( "collab" ); // updates connectivity + recomputes ratsnest
```
`Push` updates `CONNECTIVITY_DATA` per item and calls `RecalculateRatsnest`. **Never** use
direct `BOARD::Add/Remove` for remote ops unless you also manually
`GetConnectivity()->Add/Update/Remove` + `RecalculateRatsnest()` + `OnRatsnestChanged()` —
the commit path does it correctly, so use it.
## Apply gotchas (enumerated)
- **Connectivity + ratsnest are mandatory** after any structural/net change — skipping
corrupts net feedback. This is the single biggest difference from pl_editor.
- **Zones** are expensive — outline edits need a re-fill (not automatic).
- **Footprints** own child items (pads, fp text); their uuids live in `m_itemByIdCache` too —
a footprint add/remove cascades (handled by `BOARD::Add`, `board.cpp:1239`).
- **Vias/teardrops/DRC** — drill changes ripple; `BOARD_COMMIT::Push` handles within commit.
- **Groups** — membership tracked by the commit.
- **Net settings** (`OnBoardNetSettingsChanged`) are board-global, not per-item — model as a
separate doc field, not an items entry.
## Scale — the defining constraint
Real boards: 5k–20k items typical, 50k–500k for large designs. Implications:
- **Snapshot-on-change is non-viable** (per-commit serialization → MB; remote parse blocks UI).
Granular deltas via the listener are **essential** here — never a whole-document approach.
- **Initial sync is itself heavy:** seeding a fresh `Y.Doc` from a 100k-item board must be a
**batched, own-phase** operation (chunked transactions, possibly off the first paint), not a
single emit. Flag as its own sub-task within the pcbnew commit.
- Prefer **opaque-blob** items to keep per-item payloads small and avoid a huge field-mapping
surface across pcbnew's many types.