# Yjs collaborative bridge — general design (tool-agnostic)

Branch: `feature/yjs-bridge` (root + kicad + wxwidgets).
Goal: a bidirectional bridge between a KiCad editor's in-memory document model and a Yjs
`Y.Doc`, so local UI edits become CRDT ops and remote CRDT ops become model mutations.

This file is the **shared architecture**. Per-tool specifics live in:
- `0002-pl-editor-bridge.md` — drawing-sheet editor (first PoC).
- `0003-eeschema-bridge.md` — schematic editor.
- `0004-pcbnew-bridge.md` — PCB editor.

Each tool doc states **only its differences** from this file.

Decisions taken (2026-06-03):
- **CRDT representation: structured** — `Y.Array` of one `Y.Map` per item (stable id + fields).
- **First PoC transport: in-browser, no server** — two tabs share a `Y.Doc` via
  `BroadcastChannel`/`y-webrtc`. Server persistence is a later milestone.
- **First PoC target: pl_editor.**
- **One unified bridge contract for all three tools — no profile split** (see §1).

---

## 1. The unified bridge contract (one shape, per-tool adapters)

There is **one** bridge contract, identical for every tool. The C++ side exposes exactly two
things, and the JS side is fully generic:

```
                ┌───────────────────────── identical across all tools ─────────────────────────┐
  C++ editor → ChangeSource → per-item delta JSON → reconciler.ts → Y.Doc(items: Y.Array<Y.Map>)
  C++ editor ←   apply(delta) ← per-item delta JSON ← reconciler.ts ← Y.Doc ← provider ← peer
                └─ per-tool adapter ─┘                  └──────────── generic, schema-agnostic ──┘
```

- **`ChangeSource` (emit):** produces a stream of **per-item change events** — `added` /
  `removed` / `changed`, each carrying `{id, type, …fields}`. Emitted to JS as delta JSON.
- **`apply(delta)`:** consumes the same per-item delta JSON and applies it to the model.

Both directions speak the **same per-item vocabulary** for every tool. What differs is only the
**per-tool adapter** that *implements* `ChangeSource`/`apply` — an internal detail invisible to
the wire format, the JS reconciler, and the other tools:

| Tool | `ChangeSource` implementation | `apply` implementation | Identity |
|------|-------------------------------|------------------------|----------|
| eeschema | native `SCHEMATIC_LISTENER` | `SCH_COMMIT` Add/Modify/Remove + Push | native `KIID` |
| pcbnew | native `BOARD_LISTENER` (`OnBoardCompositeUpdate`) | `BOARD_COMMIT` Add/Modify/Remove + Push | native `KIID` |
| pl_editor | **in-bridge snapshot-differ** fired from `OnModify` | per-item mutate `DS_DATA_MODEL` by uuid + `SyncDrawItems` | **added** `KIID` (`0002`) |

eeschema/pcbnew get per-item events *natively* from their commit/listener systems. pl_editor
has none, so its adapter **derives** the same per-item events by uuid-diffing successive
whole-model snapshots inside the bridge (trivial at its scale) — and applies per-item by uuid.
The derivation is encapsulated in pl_editor's adapter; nothing downstream can tell the
difference. (We deliberately do **not** rewrite pl_editor's edit/undo core to a real `COMMIT` —
that would diverge heavily from upstream for no runtime benefit; decided 2026-06-03.)

So the only real per-tool work is: (1) pl_editor must **add** `KIID`+`(uuid)` to its format
(§2, `0002`); (2) eeschema/pcbnew must run the correct **recompute** on apply (connectivity /
ERC / ratsnest / hierarchy) by going through their native `COMMIT`.

---

## 2. Item identity (shared semantics)

A structured CRDT needs a per-item key that is (a) **identical on every client** and
(b) **stable across undo/redo**. The only way to guarantee both is for the id to live in the
**serialized form** of the item (so re-parsing — which undo does — preserves it, and two
clients opening the same file derive the same ids):

- **eeschema/pcbnew:** satisfied natively. Items derive from `EDA_ITEM`
  (`include/eda_item.h:516` → `const KIID m_Uuid`) and every item writes `(uuid …)`. Nothing to
  do; the uuid flows through KiCad ↔ JSON ↔ Y.Doc as the single source of truth, with **no
  cross-layer identity assumptions**.
- **pl_editor:** *not* satisfied natively — items have no `KIID` and `.kicad_wks` has no uuid
  token. We add **the same `KIID` + `(uuid …)`** the others use (not a bespoke id), so the three
  tools share one identity scheme (`0002`). Once added, identity behaves identically.

New items minted locally get a fresh `KIID`; it appears in the next emit, the reconciler keys
a new `Y.Map` by it, and remote peers create the item with that same uuid on apply.

**Backfill caveat for files lacking uuids.** When a tool *adds* uuids to a format (pl_editor),
legacy/upstream files have none, so they're backfilled on load. `KIID` backfill is **random,
not a deterministic content-hash** — so two clients cold-opening the *same never-saved* file
would disagree. Resolve via the **seed-once rule**: the client that finds the `Y.Doc` empty
seeds it and its backfilled uuids win; joiners adopt the `Y.Doc` instead of their own parse.
The first save then persists uuids, making all later opens consistent. (Deterministic
content-seed backfill is an optional later hardening; see `0002`.)

---

## 3. Wire schema + representation granularity

**The contract is minimal and schema-agnostic:** an array of objects, each with a string `id`
and `type`, plus arbitrary additional fields. The JS reconciler hardcodes **no** field names.

Each item's fields can be represented at one of two granularities — **chosen per item-type on
the C++ side; the JS reconciler is identical either way** (it just diffs values keyed by field
name):

- **Decomposed scalars** — `{id, type, x, y, text, …}`. Enables *field-level* merge (two
  people editing different fields of the same item never conflict). Best for small/hot types.
- **Opaque blob** — `{id, type, sexpr: "(segment …)"}` reusing the tool's native per-item
  serializer. *Item-level* merge granularity (concurrent edits to the same item conflict).
  Near-zero C++ mapping. Best for the long tail of complex types.

### Where the field values come from (don't hand-write N serializers)
There are three C++ mechanisms to produce the JSON for an item, in decreasing genericity. Pick
per tool; a tool may mix them and the JS reconciler is unaffected:

1. **`PROPERTY_MANAGER` reflection — write-once, all `EDA_ITEM` types (the "do it once" lever).**
   `EDA_ITEM`-based items register their fields generically (`REGISTER_TYPE` + `AddProperty`,
   e.g. `pcbnew/board_item.cpp:427+` — *"Position X"*, *"Layer"*, *"Locked"*, …). Enumerate
   `propMgr.GetProperties(item->GetTypeId())`, read each via `PROPERTY_BASE::GetValue → wxAny`
   → JSON; write back via `PROPERTY_BASE::Set`. **One converter covers every registered type**
   with field-level merge. Cost paid once: a `wxAny↔JSON` type bridge (ints/strings/bools,
   enums, `EDA_ANGLE`/`VECTOR2I` units). Coverage isn't 100% — a few unregistered fields still
   need (3). Primary path for **pcbnew** (broad coverage) and **eeschema** (partial).
2. **Native per-item S-expr blob — write-once *per tool that has one*.** pcbnew
   `PCB_IO_KICAD_SEXPR::Format(BOARD_ITEM*)`; pl_editor `DS_DATA_MODEL_IO::Format(model, item)`
   / `SaveInString({item}, &str)`. Serializes *any* of that tool's items with no per-type code;
   item-level merge only. (eeschema has no clean per-item formatter — document-level only.)
3. **Hand-mapped per-type fields.** Explicit, total control, works everywhere (incl. pl_editor,
   which has neither reflection nor `EDA_ITEM`). Most code — reserve for tools with few types
   (pl_editor's 5) or the hot fields you want field-level merge on.

(There is also `SERIALIZABLE`/`Serialize→protobuf::Any` on `EDA_ITEM`, but it targets the IPC
API — protobuf, partial coverage — wrong fit for JSON; not used here.)

A tool may mix granularities/mechanisms and refine over time **without any JS change**. Nested
data (polygon corner lists, bitmap blobs) is an opaque blob field or a nested `Y.Array`/`Y.Map`.
Numeric coordinates use the model's native units; the reconciler **quantizes before comparing**
to avoid spurious float diffs.

Optional **Zod schema as a validation guard only** at the apply boundary (reject garbage,
coerce number/bool/string) — never as a structural requirement.

---

## 4. The JS reconciler (`web/.../collab/`, identical for all tools)

A single generic module, parameterized only by the `Y.Doc` and the two bridge entry points:

- **Down (model → Y):** keep the last-seen item-set keyed by `id`. On each emitted per-item
  delta, write **only** the changes into the `Y.Array<Y.Map>`, inside one `Y.transaction`
  tagged with a local origin. (If an adapter ever emits a whole snapshot instead of a delta —
  it shouldn't, but pl_editor's differ could — the reconciler still diffs it to minimal ops, so
  the wire stays granular regardless.)
- **Up (Y → model):** observe the `Y.Array`; on remote-origin events, build the changed-item
  delta JSON and call the C++ `apply` hook. Ignore events whose origin is our own tag
  (standard Yjs echo-suppression).
- **Generic mapping:** an item ⇆ `Y.Map` is a flat copy of scalar fields; the diff is
  value-equality per key. Adding a C++ field needs zero JS change.

The reconciler is **adapter-blind** — it only ever sees per-item deltas keyed by uuid.

---

## 5. Loop / re-entrancy safety (shared)

- **`s_applyingRemote` guard** in the C++ bridge: set around `apply`; emit is suppressed while
  set. Prevents apply → (OnModify | listener fire) → emit → Y write → observe → apply … loops.
  Needed by every adapter (a native listener would otherwise re-fire on our own apply; the
  pl_editor differ would otherwise re-diff our own apply).
- **Asyncify timing:** apply is invoked from a JS callback and runs synchronously on the main
  thread between frames (same context as the existing `OnFileDropped`/`OpenFileCallback`
  hooks). It must not run mid-Asyncify-unwind — gate applies on the tool's ready/idle state and
  queue any that arrive during a suspension.
- **Yjs origin tags** on all reconciler transactions; observers skip their own origin.

---

## 6. Transport & persistence (shared, phased)

- **PoC:** two tabs, one `Y.Doc`, `BroadcastChannel` (or `y-webrtc`). Zero backend work.
- **Later (M-server):** `y-websocket` provider (standalone or folded into the Fastify server in
  `web/apps/server`) for networked sync + persistence; periodically materialize the Y.Doc back
  to the canonical `.kicad_*` file via the tool's normal save path.

---

## 7. Shared C++ bridge interface

A thin, `#ifdef __EMSCRIPTEN__`-guarded interface each tool implements, so JS is uniform:

- **Emit → JS:** `EM_ASM({ window.kicadCollab.onDelta(UTF8ToString($0)) }, json)` — `json` is a
  per-item delta. Same JS entry point for every tool/adapter.
- **Apply ← JS:** `EMSCRIPTEN_KEEPALIVE void kicadCollabApply(const char* json)` — dispatches to
  the active tool's adapter; wrapped in the `s_applyingRemote` guard.
- Copy the proven wx-layer pattern (`EM_ASM` out, `ccall`→`EMSCRIPTEN_KEEPALIVE` in).
- The collab code that touches tool models lives in the **kicad** fork; keep it tightly guarded
  and run `kicad-diff-stats.sh`. eeschema/pcbnew adapters add almost no surface (listener
  subclass + COMMIT calls). pl_editor adds the most: the `(uuid)` format token + the snapshot-
  differ adapter — but still leaves its edit/undo core untouched.

---

## 8. Phasing — one commit per step

1. **pl_editor interface/identity change.** `KIID m_Uuid` on `DS_DATA_ITEM` + `(uuid …)`
   keyword/parse(backfill)/save via the shared `FormatUuid`. Independently testable (uuid
   round-trip + legacy-file backfill-on-save). Commit. (`0002` §identity)
2. **pl_editor bridge + JSON converters (all 5 item types) + tests.** The snapshot-differ
   `ChangeSource`, per-item `apply` by uuid, the generic JS reconciler, `BroadcastChannel`
   transport, two-tab demo. Text/line/rect as scalars, poly/bitmap as blobs (`0002`). Commit.
3. **eeschema JSON converters + bridge + tests.** Adapter = `SCHEMATIC_LISTENER` + `SCH_COMMIT`;
   converters reflection-first (§3 mechanism 1) + hand-map gaps; multi-sheet hierarchy + symbol
   instances. Commit. (`0003`)
4. **pcbnew JSON converters + bridge + tests.** Adapter = `BOARD_LISTENER` + `BOARD_COMMIT`;
   converters = reflection (1) for the bulk + native blob (2) for the long tail; mandatory
   connectivity/ratsnest recompute; large-board batched initial-sync. Commit. (`0004`)
5. **Later — y-websocket + persistence** across all tools.

pl_editor goes first (steps 1–2) because its adapter is the only one we write from scratch and
it carries the least that can break (no connectivity, tiny docs); once the contract holds
there, the eeschema/pcbnew adapters are a thin re-use over native machinery.

---

## 9. Cross-cutting risks
- **Asyncify timing** of apply (§5) — needs a real under-load test.
- **Representation-granularity drift** — mixing blob and scalar per type must stay consistent
  across clients (the `type`→granularity decision lives in C++ and must be deterministic).
- **Coordinate precision** — quantize before diffing (§3).
- **Recompute side-effects (eeschema/pcbnew)** — applying outside the normal UI flow must still
  trigger connectivity/ERC/ratsnest/hierarchy; always go through the native COMMIT.
- **Initial sync cost (pcbnew)** — seeding a 100k-item board into a fresh `Y.Doc` is itself
  heavy; needs a batched/own-phase strategy (see `0004`).
- **Fork divergence** — pl_editor changes the `.kicad_wks` file format (fork-only files; see
  `0002`). eeschema/pcbnew adapters are near-zero divergence.