AI_Assistant/pcbjam
Watch
1
0
Fork
You've already forked pcbjam
 0
Code Issues Pull requests Projects Releases 7 Packages Wiki Activity Actions

fix(editor): boot fan-out — parallel presync/doc-room/files, lib-editor enumerate gate, My Symbols 409 (load-ux 0003)

Browse source 
The 7/29 "presync after open" reorder misdiagnosed the slow board open: the
problem was ordering/serialization (libs awaited before serially-fetched
project files), not bandwidth — the ~155 lib requests are latency-bound. It
also regressed the symbol editor, whose frame eagerly enumerates every lib
through the mutex-serialized bridge crossings: with no warm-up running during
the wasm download, each cold lib fetched one-at-a-time inside its own serial
crossing.

- WasmTool: at consent-OK start in parallel — wasm boot, doc-room connect
  (websocket up BEFORE any file fetch; errors captured and rethrown at the
  await), lib presync (concurrency 8, presyncSettled never rejects), and
  project-file staging (the glue runs FS.staticInit() at script-eval, so
  MEMFS staging always overlapped the download).
- Lib editors (fileless): installLibsProvider enumerateGate parks the whole
  "list" op (a plain name list also cold-fetches the bundle) until the
  presync settles, and the boot overlay waits for it before waitForWxUi —
  downloads run 8-wide in JS, the serialized crossings become IDB read +
  parse. pcbnew/eeschema stay ungated (a silent mid-session park would read
  as a hang). The reentry-guard mutex itself is untouched.
- Overlay: "Project files — n/m" staging line (DriveOptions.onFileProgress),
  and the lib line returns worded "Checking <kind> libraries — n/155" (the
  walk checks every lib but downloads only new or changed ones).
- boot: hasWritableLib accepts lib type "org" (the private platform's rename
  of "user") — boot re-created "My Symbols" on every load and the backend
  409'd.

typecheck clean; 239/239 standalone tests; web e2e tools-open 7/7 +
chrome-toggle 2/2 + comments 1/1 (web-firefox).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N925iFAdnVcMCdvS4pzXER
This commit is contained in:
Gergő Törcsvári 2026-07-30 10:37:12 +02:00
commit bd1fbdce85
No known key found for this signature in database
GPG key ID: 8E75F2CDE64E5322
7 changed files with 260 additions and 55 deletions
Download patch file Download diff file Expand all files Collapse all files

155
web/standalone/src/components/WasmTool.tsx
View file

@ -167,8 +167,9 @@ interface ConsentInfo {
toolRawBytes: number | null;
/** A previous version of this bundle was downloaded → word it as an update. */
update: boolean;
/** The lib kind this tool pre-syncs — warmed in the background once the
* editor is open, not ahead of it (see startLibPresync). */
/** The lib kind this tool pre-syncs — warmed in parallel with the wasm
* download (the boot fan-out; see startLibPresync). Editors with a project
* file open without waiting on it; the lib editors wait (enumerate gate). */
libNowKind: "symbol" | "footprint" | null;
libNow: LibsSyncState | null;
/** The other kind a merged-bundle session can pull lazily ("only if used"). */
@ -954,6 +955,12 @@ export function WasmTool({
done: number;
total: number;
} | null>(null);
// Project-file staging progress (fetch + MEMFS write, overlapping the wasm
// download) for the boot overlay's "Project files" line. Null when idle/done.
const [fileSync, setFileSync] = React.useState<{
done: number;
total: number;
} | null>(null);
// Last lib error (e.g. a backend 404 on open), shown as a dismissible toast.
const [libError, setLibError] = React.useState<string | null>(null);
// A collaborator updated library items that are PLACED in the open document
@ -1361,24 +1368,34 @@ export function WasmTool({
setConsent(null);
if (!ok) return; // unmounted while waiting — never start the download
}
// Pre-warm the lib bundles into IDB — started LATER (right after the
// project is staged + opened, below). It used to run in parallel with the
// wasm download, but a scope with ~155 team libs then fired hundreds of
// bundle fetches that contended with the 26MB+ wasm and the project's own
// files on the same connection, so opening a board took minutes before
// anything was interactive. The warm-up is an OPTIMIZATION (the editor's
// first enumerate reads a warm cache instead of freezing on N cold bundle
// fetches), never a precondition — so it yields to everything the user is
// actually blocked on and fills in behind them.
// Boot fan-out (load-fanout): everything that doesn't need the wasm
// starts NOW, in parallel with the (large) wasm download.
//
// - Lib presync: warms the per-lib IDB bundles. It briefly lived AFTER
// the project open (2a47103) because opening a board took minutes —
// but the real problem was ordering (libs awaited before the project
// files, which fetched serially), not bandwidth: the ~155 lib
// requests are latency-bound, and the files now stage 8-wide the
// moment FS is up. So the presync overlaps the whole download again.
// - Doc-room connect: the collab websocket is up BEFORE the project
// files are fetched, so a file changed while we load still reaches
// this client (the room materializes the target; sheet/sibling rooms
// bind after open, as before).
// - Project files: driveProjectIntoTool below starts fetching as soon
// as the glue scripts have evaluated — the non-modularized glue runs
// FS.staticInit() at script-eval, so MEMFS staging overlaps the wasm
// download/compile rather than following it.
const libKind = LIB_KIND_FOR_TOOL[tool];
const startLibPresync = () => {
if (!source?.presync || !libKind) return;
// Fire-and-forget + best-effort, unchanged: a lib that fails to presync
// still loads lazily later, and the SyncStack dedups — a lib the wasm
// reaches mid-presync just awaits the same in-flight fetch. The signal
// stops the walk on unmount; the setLibSync guards keep a late progress
// callback from re-lighting the indicator of a torn-down session.
void source
// Best-effort and NEVER rejects: `presyncSettled` resolves when the
// warm-up finished, failed, or was aborted (immediately when the source
// has none). A lib that fails to presync still loads lazily later, and
// the SyncStack dedups — a lib the wasm reaches mid-presync just awaits
// the same in-flight fetch. The signal stops the walk on unmount; the
// setLibSync guards keep a late progress callback from re-lighting the
// indicator of a torn-down session.
const startLibPresync = (): Promise<void> => {
if (!source?.presync || !libKind) return Promise.resolve();
return source
.presync({
kind: libKind,
signal: presyncAbort.signal,
@ -1393,6 +1410,41 @@ export function WasmTool({
setLibSync(null);
});
};
const presyncSettled = startLibPresync();
// Lib editors (fileless): their frame eagerly enumerates EVERY lib of
// its kind at boot, one mutex-serialized bridge crossing at a time
// (g_pcbjamProxyMutex in the plugins) — a cold lib would network-fetch
// inside its serial crossing. Hold enumerates until the 8-wide presync
// settles so the crossings read warm IDB. pcbnew/eeschema are NOT
// gated: their choosers open on demand, and a mid-session park with no
// overlay would read as a hang.
const enumerateGate =
FILELESS_TOOLS.has(tool) && libKind
? (kind: string) =>
kind === libKind ? presyncSettled : Promise.resolve()
: undefined;
// Connect the doc's collab room in parallel with the wasm download (it
// needs identity, not the wasm). Errors are captured and rethrown at
// the await below — rejecting here would surface as an unhandled
// rejection while the download is still running.
const docSessionReady: Promise<
| { session?: KicadDocSession; targetBytes?: Uint8Array }
| { error: unknown }
> = (async () => {
try {
await identityReady;
return await maybeConnectDocSession(win, {
docSource,
tool,
scopeId,
projectId,
targetPath,
log: append,
});
} catch (error) {
return { error };
}
})();
await bootKicadTool({
tool,
base,
@ -1409,6 +1461,7 @@ export function WasmTool({
warmStart: warm,
onWasmInstantiated: () => markWasmDownloaded(meta.bundle, meta.ver),
libsSource: source,
enumerateGate,
// 3D models: lazy per-board source (null unless the CDN manifest is
// configured) — feeds the board prescan + the viewer's ensure fallback.
modelsSource: modelsSourceConfig(),
@ -1456,14 +1509,11 @@ export function WasmTool({
},
}),
});
const { session, targetBytes } = await maybeConnectDocSession(win, {
docSource,
tool,
scopeId,
projectId,
targetPath,
log: append,
});
// The room connect started in the fan-out above — settle it before
// staging so the target file materializes from the doc when it has one.
const docResult = await docSessionReady;
if ("error" in docResult) throw docResult.error;
const { session, targetBytes } = docResult;
await driveProjectIntoTool(win, {
tool,
slug,
@ -1478,15 +1528,9 @@ export function WasmTool({
: fetchBytes,
log: append,
onStatus: setStatus,
onFileProgress: (done, total) =>
setFileSync(done >= total ? null : { done, total }),
});
// Everything the user is blocked on has now downloaded: the wasm bundle
// is up and the project's own files are staged into MEMFS with the target
// file open. NOW warm the scope's lib bundles, in the background — the
// boot tail below (collab rooms, wx UI build) is latency-bound chatter,
// not bandwidth, so the pre-sync no longer delays the editor becoming
// interactive. Whatever it hasn't reached by the time the user opens a
// chooser is fetched lazily on demand, as it always was.
startLibPresync();
// Read-only viewer (read-only-viewer): lock the wasm frame BEFORE the
// boot overlay drops — the file is open, so the frame exists; poll the
// export like the chrome toggle does. Fails CLOSED (boot error overlay):
@ -1618,6 +1662,10 @@ export function WasmTool({
});
}
}
// Lib editors: the enumerate gate holds their whole-set hydrate until
// the presync settles — wait for it here too, so the boot overlay (with
// its ticking lib line) stays up instead of revealing an empty tree.
if (enumerateGate) await presyncSettled;
// Tool booted + project opened. Wait for the wx UI to actually build
// before dropping the overlay, so we don't reveal a still-blank editor.
await waitForWxUi(win);
@ -1759,11 +1807,21 @@ export function WasmTool({
{status || "Loading…"}
</p>
<DownloadProgress progress={progress} />
{/* The lib pre-sync deliberately has NO line here anymore: it now
starts only once the wasm + project files are in (see the boot
IIFE), so the overlay would be showing a counter the user is not
waiting on. Its progress surfaces in the unobtrusive bottom-left
indicator below, after `ready`. */}
{/* The parallel fan-out's other progress: project files staging
into MEMFS, and the lib warm-up. The lib line says "checking"
on purpose — the walk visits every lib but downloads only new
or changed ones, so a bare 15/155 next to the consent dialog's
MB figures would read as 155 big downloads. */}
{fileSync && fileSync.total > 0 && (
<p className="whitespace-pre font-mono text-xs text-white/50">
Project files — {String(fileSync.done).padStart(String(fileSync.total).length, " ")}/{fileSync.total}
</p>
)}
{libSync && (
<p className="whitespace-pre font-mono text-xs text-white/50">
{libSyncLabel(libSync)}
</p>
)}
<p className="font-mono text-xs text-white/40">
{warmBoot
? "Loading from your browser's cache — no download needed."
@ -2101,8 +2159,8 @@ function libStateLabel(s: LibsSyncState | null): string | null {
function libNowDetail(s: LibsSyncState | null): string {
const base = "this editor browses them";
// No warmth answer at all: say only what stays true regardless of counts —
// the warm-up runs in the background once the editor is open (it moved off
// the critical path, so "downloaded now" would be wrong as well as vague).
// the warm-up runs alongside the editor download and finishes in the
// background, so "downloaded now" would overpromise as well as vague.
if (!s || s.total === 0) return `${base} — fetched in the background`;
const cold = s.total - s.warm;
if (cold === 0) {
@ -2241,15 +2299,18 @@ function DownloadConsent({
}
/**
* Fixed-width lib pre-sync line, e.g. "Syncing symbol libraries — 42/208".
* The prefix is constant and `done` is space-padded to `total`'s digit count,
* so the text stays still while the counter ticks (render it in a font-mono +
* whitespace-pre element so the pad spaces hold their width).
* Fixed-width lib pre-sync line, e.g. "Checking symbol libraries — 42/208".
* "Checking", not "downloading": the walk visits every lib of the kind but
* downloads only the new/changed ones — a bare counter read as 155 downloads
* (standalone-load-ux follow-up). The prefix is constant and `done` is
* space-padded to `total`'s digit count, so the text stays still while the
* counter ticks (render it in a font-mono + whitespace-pre element so the pad
* spaces hold their width).
*/
function libSyncLabel(s: { kind: string; done: number; total: number }): string {
const total = String(s.total);
const done = String(Math.min(s.done, s.total)).padStart(total.length, " ");
return `Syncing ${s.kind} libraries — ${done}/${total}`;
return `Checking ${s.kind} libraries — ${done}/${total}`;
}
/**

24
web/standalone/src/wasm/boot.test.ts Normal file
View file

@ -0,0 +1,24 @@
import { describe, expect, it } from "vitest";
import { hasWritableLib } from "./boot";
import type { LibInfo } from "./libs/source";
const lib = (type?: string): LibInfo => ({ id: `id-${type}`, name: `n-${type}`, type });
describe("hasWritableLib", () => {
it("sees an example-backend 'user' lib", () => {
expect(hasWritableLib([[lib("origin"), lib("user")]])).toBe(true);
});
it("sees a private-platform 'org' lib (renamed from 'user') — the My Symbols 409", () => {
expect(hasWritableLib([[lib("origin")], [lib("mirror"), lib("org")]])).toBe(true);
});
it("false when only read-only libs are listed", () => {
expect(hasWritableLib([[lib("origin"), lib("mirror")], [lib(undefined)]])).toBe(false);
});
it("false on empty lists", () => {
expect(hasWritableLib([[], []])).toBe(false);
expect(hasWritableLib([])).toBe(false);
});
});

30
web/standalone/src/wasm/boot.ts
View file

@ -29,6 +29,18 @@ import pcbjamDarkTheme from "./themes/pcbjam-dark.json";
/** The default user lib boot ensures exists, so there's a writable save target. */
const DEFAULT_USER_LIB_NAME = "My Symbols";
/**
* True when any per-kind lib list already contains a writable lib — "user" on
* the example backend / local sources, "org" on the private platform (a scope's
* own writable lib; renamed from "user" in the team-libs rework). Missing "org"
* here made boot re-create "My Symbols" on every load, which the backend 409'd.
*/
export function hasWritableLib(lists: Iterable<LibInfo[]>): boolean {
return [...lists].some((libs) =>
libs.some((l) => l.type === "user" || l.type === "org"),
);
}
/**
* Boot a KiCad tool directly in the main React document — no iframe.
*
@ -85,6 +97,12 @@ export interface BootOptions {
* (empty lib-tables are seeded). Its libs become sym-lib-table and/or
* fp-lib-table rows depending on the tool (see `libKinds` in doBoot). */
libsSource?: LibsSource | null;
/** Hold provider enumerates for a kind until this resolves — the lib editors
* pass their presync-settled promise so the boot-time whole-set hydrate
* reads a warm IndexedDB instead of cold-fetching one lib at a time inside
* the serialized bridge crossings. See `LibsProviderOptions.enumerateGate`.
* Never gates boot's own `listLibs` table seeding (a direct source call). */
enumerateGate?: (kind: string) => Promise<void>;
/** 3D model source (lazy, per-board). Null/omitted ⇒ the viewer renders the
* bare board only, exactly as before models existed. */
modelsSource?: Model3dSource | null;
@ -237,6 +255,7 @@ async function doBoot(opts: BootOptions): Promise<void> {
onWasmInstantiated,
libsSource,
modelsSource,
enumerateGate,
} = opts;
// Truthful fetch label: a warm start (download-completion marker present)
// reads from the HTTP cache, so "Downloading" would be a lie — and vice versa.
@ -298,7 +317,7 @@ async function doBoot(opts: BootOptions): Promise<void> {
}
if (libsSource && libKinds.length) {
installLibsProvider(libsSource, log);
installLibsProvider(libsSource, log, { enumerateGate });
// 3D models ride the same provider (kind "model3d"): the C++ ensure fallback
// and the board prescan both resolve through this source.
if (modelsSource) {
@ -310,12 +329,9 @@ async function doBoot(opts: BootOptions): Promise<void> {
// kind-agnostic containers and appear in every list).
const listsByKind = new Map<"symbol" | "footprint", LibInfo[]>();
for (const k of libKinds) listsByKind.set(k, await libsSource.listLibs(k));
// Ensure the owner has at least one writable user lib to save items into.
// A user lib holds either kind, so the created lib joins every table.
const hasUserLib = [...listsByKind.values()].some((libs) =>
libs.some((l) => l.type === "user"),
);
if (libsSource.createLib && !hasUserLib) {
// Ensure the owner has at least one writable lib to save items into.
// A writable lib holds either kind, so the created lib joins every table.
if (libsSource.createLib && !hasWritableLib(listsByKind.values())) {
const created = await libsSource.createLib(DEFAULT_USER_LIB_NAME);
if (created) {
for (const libs of listsByKind.values()) libs.push(created);

18
web/standalone/src/wasm/kicad-runner.test.ts
View file

@ -94,6 +94,24 @@ describe("MEMFS project staging", () => {
expect(peak).toBeLessThanOrEqual(8);
});
it("reports per-file progress from 0 up to the total", async () => {
const { win } = fakeWin();
const files = ["a.kicad_sym", "b.kicad_sym", "c.kicad_sym"];
const seen: Array<[number, number]> = [];
await driveProjectIntoTool(win, {
...opts(files, async () => new Uint8Array([1])),
onFileProgress: (done, total) => seen.push([done, total]),
});
// One up-front (0, total) so the line shows immediately, then one tick per
// staged file, ending exactly at the total.
expect(seen[0]).toEqual([0, files.length]);
expect(seen).toHaveLength(files.length + 1);
expect(seen.map(([done]) => done).sort((a, b) => a - b)).toEqual([0, 1, 2, 3]);
expect(seen.at(-1)?.[1]).toBe(files.length);
});
it("rejects when a fetch fails", async () => {
const { win } = fakeWin();
const files = ["ok1.kicad_sym", "bad.kicad_sym", "ok2.kicad_sym"];

7
web/standalone/src/wasm/kicad-runner.ts
View file

@ -22,6 +22,10 @@ export interface DriveOptions {
fetchBytes: (relPath: string) => Promise<Uint8Array>;
log: (msg: string) => void;
onStatus: (text: string) => void;
/** Per-file staging progress (files fetched+written so far, total) — drives
* the boot overlay's "Project files — n/m" line. Reported once up front
* with done=0 so the line appears as soon as staging starts. */
onFileProgress?: (done: number, total: number) => void;
}
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
@ -83,11 +87,14 @@ const STAGE_CONCURRENCY = 8;
async function syncProjectToMemfs(win: ToolWindow, opts: DriveOptions): Promise<void> {
getFS(win).mkdirTree(memfsProjectDir(opts.slug));
let staged = 0;
opts.onFileProgress?.(0, opts.files.length);
const queue = [...opts.files];
const worker = async (): Promise<void> => {
for (let file = queue.shift(); file; file = queue.shift()) {
const bytes = await opts.fetchBytes(file.path);
restageFile(win, opts.slug, file.path, bytes, opts.log);
opts.onFileProgress?.(++staged, opts.files.length);
// 3D models: prefetch every model this board references (R2 → IDB → MEMFS)
// so the 3D viewer's first open resolves locally. Fire-and-forget — project
// open never waits on it; a ref that misses falls back to the C++ per-model

56
web/standalone/src/wasm/libs/source.test.ts
View file

@ -152,3 +152,59 @@ describe("installLibsProvider — fat list (arg=bodies)", () => {
expect(await request("index", "/mnt/pcbjam/", "", "footprint")).toBeNull();
});
});
describe("installLibsProvider — enumerate gate (load-fanout)", () => {
beforeEach(() => {
(globalThis as unknown as { window: unknown }).window = {
location: { search: "" },
dispatchEvent: () => true,
};
});
afterEach(() => {
delete (globalThis as unknown as { window?: unknown }).window;
});
it("parks list (names AND bodies) until the gate resolves; get is never gated", async () => {
let release!: () => void;
const gate = new Promise<void>((r) => {
release = r;
});
const gateKinds: string[] = [];
installLibsProvider(
baseSource({ getAllItems: async () => itemsAsBytes() }),
() => {},
{
enumerateGate: (kind) => {
gateKinds.push(kind);
return gate;
},
},
);
const request = (
globalThis as unknown as { window: { kicadLibs: { request: Req } } }
).window.kicadLibs.request;
let fatDone = false;
let namesDone = false;
const fat = request("list", libUri("Device"), "bodies", "symbol").then((r) => {
fatDone = true;
return r;
});
const names = request("list", libUri("Device"), "", "symbol").then((r) => {
namesDone = true;
return r;
});
// A user-triggered item open resolves while the gate is still held — only
// enumerates are parked.
await expect(request("get", libUri("Device"), "R", "symbol")).resolves.toBe(
"(kicad_symbol_lib (symbol R))",
);
expect(fatDone).toBe(false);
expect(namesDone).toBe(false);
release();
expect(parseFramed(await fat, "symbols")).toHaveLength(2);
expect(JSON.parse((await names) as string)).toEqual({ symbols: ["R", "C"] });
expect(gateKinds).toEqual(["symbol", "symbol"]);
});
});

25
web/standalone/src/wasm/libs/source.ts
View file

@ -13,7 +13,10 @@ export interface LibInfo {
/** Display nickname for the sym-lib-table row. */
name: string;
description?: string | null;
/** 'origin' | 'mirror' | 'user' — drives "ensure a user lib exists" at boot. */
/** 'origin' | 'mirror' | 'user' | 'org' — drives "ensure a writable lib
* exists" at boot. 'org' is the private platform's name for a scope's own
* writable lib (renamed from 'user' in the team-libs rework); the example
* backend and local sources still say 'user'. */
type?: string;
/** Item count, when the source knows it cheaply (manifest/backend) — shown as a
* badge in the home-page library list. Omitted ⇒ no badge. */
@ -310,9 +313,26 @@ export function buildFpLibTable(libsList: LibInfo[]): string {
* "index" -> the publish-time footprint index JSON (source-global; lib arg
* ignored) / null when the source has none — see getFpIndex.
*/
export interface LibsProviderOptions {
/**
* Hold every enumerate ("list", including the fat "bodies" variant — a plain
* name list also opens the lib's sync stack, i.e. cold-fetches its bundle)
* for the given kind until the returned promise resolves. The lib editors
* pass their presync-settled promise: their frame eagerly enumerates EVERY
* lib of its kind at boot, one mutex-serialized bridge crossing at a time
* (see g_pcbjamProxyMutex in the plugins), so a cold lib would network-fetch
* inside its serial crossing — the gate lets the 8-wide parallel presync
* warm IndexedDB first and the crossings become local read + parse.
* MUST always resolve (the presync is best-effort and never rejects);
* "get"/"save"/"index" are user-triggered and never gated.
*/
enumerateGate?: (kind: string) => Promise<void>;
}
export function installLibsProvider(
source: LibsSource,
log: (msg: string) => void,
opts?: LibsProviderOptions,
): void {
if (window.kicadLibs) return;
const delay = artificialDelayMs();
@ -353,6 +373,9 @@ export function installLibsProvider(
try {
switch (op) {
case "list": {
// Enumerate gate (load-fanout): park this crossing (Asyncify) until
// the caller's precondition — typically the presync — has settled.
if (opts?.enumerateGate) await opts.enumerateGate(kind);
// Each plugin parses its own key: footprints / symbols.
const key = kind === "footprint" ? "footprints" : "symbols";
// "bodies" (arg) = the fat list: every item's body in one crossing, so