fix(editor): boot fan-out — parallel presync/doc-room/files, lib-editor enumerate gate, My Symbols 409 (load-ux 0003)
The 7/29 "presync after open" reorder misdiagnosed the slow board open: the problem was ordering/serialization (libs awaited before serially-fetched project files), not bandwidth — the ~155 lib requests are latency-bound. It also regressed the symbol editor, whose frame eagerly enumerates every lib through the mutex-serialized bridge crossings: with no warm-up running during the wasm download, each cold lib fetched one-at-a-time inside its own serial crossing. - WasmTool: at consent-OK start in parallel — wasm boot, doc-room connect (websocket up BEFORE any file fetch; errors captured and rethrown at the await), lib presync (concurrency 8, presyncSettled never rejects), and project-file staging (the glue runs FS.staticInit() at script-eval, so MEMFS staging always overlapped the download). - Lib editors (fileless): installLibsProvider enumerateGate parks the whole "list" op (a plain name list also cold-fetches the bundle) until the presync settles, and the boot overlay waits for it before waitForWxUi — downloads run 8-wide in JS, the serialized crossings become IDB read + parse. pcbnew/eeschema stay ungated (a silent mid-session park would read as a hang). The reentry-guard mutex itself is untouched. - Overlay: "Project files — n/m" staging line (DriveOptions.onFileProgress), and the lib line returns worded "Checking <kind> libraries — n/155" (the walk checks every lib but downloads only new or changed ones). - boot: hasWritableLib accepts lib type "org" (the private platform's rename of "user") — boot re-created "My Symbols" on every load and the backend 409'd. typecheck clean; 239/239 standalone tests; web e2e tools-open 7/7 + chrome-toggle 2/2 + comments 1/1 (web-firefox). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N925iFAdnVcMCdvS4pzXER
This commit is contained in:
parent
74e822134f
commit
bd1fbdce85
7 changed files with 260 additions and 55 deletions
|
|
@ -167,8 +167,9 @@ interface ConsentInfo {
|
||||||
toolRawBytes: number | null;
|
toolRawBytes: number | null;
|
||||||
/** A previous version of this bundle was downloaded → word it as an update. */
|
/** A previous version of this bundle was downloaded → word it as an update. */
|
||||||
update: boolean;
|
update: boolean;
|
||||||
/** The lib kind this tool pre-syncs — warmed in the background once the
|
/** The lib kind this tool pre-syncs — warmed in parallel with the wasm
|
||||||
* editor is open, not ahead of it (see startLibPresync). */
|
* download (the boot fan-out; see startLibPresync). Editors with a project
|
||||||
|
* file open without waiting on it; the lib editors wait (enumerate gate). */
|
||||||
libNowKind: "symbol" | "footprint" | null;
|
libNowKind: "symbol" | "footprint" | null;
|
||||||
libNow: LibsSyncState | null;
|
libNow: LibsSyncState | null;
|
||||||
/** The other kind a merged-bundle session can pull lazily ("only if used"). */
|
/** The other kind a merged-bundle session can pull lazily ("only if used"). */
|
||||||
|
|
@ -954,6 +955,12 @@ export function WasmTool({
|
||||||
done: number;
|
done: number;
|
||||||
total: number;
|
total: number;
|
||||||
} | null>(null);
|
} | null>(null);
|
||||||
|
// Project-file staging progress (fetch + MEMFS write, overlapping the wasm
|
||||||
|
// download) for the boot overlay's "Project files" line. Null when idle/done.
|
||||||
|
const [fileSync, setFileSync] = React.useState<{
|
||||||
|
done: number;
|
||||||
|
total: number;
|
||||||
|
} | null>(null);
|
||||||
// Last lib error (e.g. a backend 404 on open), shown as a dismissible toast.
|
// Last lib error (e.g. a backend 404 on open), shown as a dismissible toast.
|
||||||
const [libError, setLibError] = React.useState<string | null>(null);
|
const [libError, setLibError] = React.useState<string | null>(null);
|
||||||
// A collaborator updated library items that are PLACED in the open document
|
// A collaborator updated library items that are PLACED in the open document
|
||||||
|
|
@ -1361,24 +1368,34 @@ export function WasmTool({
|
||||||
setConsent(null);
|
setConsent(null);
|
||||||
if (!ok) return; // unmounted while waiting — never start the download
|
if (!ok) return; // unmounted while waiting — never start the download
|
||||||
}
|
}
|
||||||
// Pre-warm the lib bundles into IDB — started LATER (right after the
|
// Boot fan-out (load-fanout): everything that doesn't need the wasm
|
||||||
// project is staged + opened, below). It used to run in parallel with the
|
// starts NOW, in parallel with the (large) wasm download.
|
||||||
// wasm download, but a scope with ~155 team libs then fired hundreds of
|
//
|
||||||
// bundle fetches that contended with the 26MB+ wasm and the project's own
|
// - Lib presync: warms the per-lib IDB bundles. It briefly lived AFTER
|
||||||
// files on the same connection, so opening a board took minutes before
|
// the project open (2a47103) because opening a board took minutes —
|
||||||
// anything was interactive. The warm-up is an OPTIMIZATION (the editor's
|
// but the real problem was ordering (libs awaited before the project
|
||||||
// first enumerate reads a warm cache instead of freezing on N cold bundle
|
// files, which fetched serially), not bandwidth: the ~155 lib
|
||||||
// fetches), never a precondition — so it yields to everything the user is
|
// requests are latency-bound, and the files now stage 8-wide the
|
||||||
// actually blocked on and fills in behind them.
|
// moment FS is up. So the presync overlaps the whole download again.
|
||||||
|
// - Doc-room connect: the collab websocket is up BEFORE the project
|
||||||
|
// files are fetched, so a file changed while we load still reaches
|
||||||
|
// this client (the room materializes the target; sheet/sibling rooms
|
||||||
|
// bind after open, as before).
|
||||||
|
// - Project files: driveProjectIntoTool below starts fetching as soon
|
||||||
|
// as the glue scripts have evaluated — the non-modularized glue runs
|
||||||
|
// FS.staticInit() at script-eval, so MEMFS staging overlaps the wasm
|
||||||
|
// download/compile rather than following it.
|
||||||
const libKind = LIB_KIND_FOR_TOOL[tool];
|
const libKind = LIB_KIND_FOR_TOOL[tool];
|
||||||
const startLibPresync = () => {
|
// Best-effort and NEVER rejects: `presyncSettled` resolves when the
|
||||||
if (!source?.presync || !libKind) return;
|
// warm-up finished, failed, or was aborted (immediately when the source
|
||||||
// Fire-and-forget + best-effort, unchanged: a lib that fails to presync
|
// has none). A lib that fails to presync still loads lazily later, and
|
||||||
// still loads lazily later, and the SyncStack dedups — a lib the wasm
|
// the SyncStack dedups — a lib the wasm reaches mid-presync just awaits
|
||||||
// reaches mid-presync just awaits the same in-flight fetch. The signal
|
// the same in-flight fetch. The signal stops the walk on unmount; the
|
||||||
// stops the walk on unmount; the setLibSync guards keep a late progress
|
// setLibSync guards keep a late progress callback from re-lighting the
|
||||||
// callback from re-lighting the indicator of a torn-down session.
|
// indicator of a torn-down session.
|
||||||
void source
|
const startLibPresync = (): Promise<void> => {
|
||||||
|
if (!source?.presync || !libKind) return Promise.resolve();
|
||||||
|
return source
|
||||||
.presync({
|
.presync({
|
||||||
kind: libKind,
|
kind: libKind,
|
||||||
signal: presyncAbort.signal,
|
signal: presyncAbort.signal,
|
||||||
|
|
@ -1393,6 +1410,41 @@ export function WasmTool({
|
||||||
setLibSync(null);
|
setLibSync(null);
|
||||||
});
|
});
|
||||||
};
|
};
|
||||||
|
const presyncSettled = startLibPresync();
|
||||||
|
// Lib editors (fileless): their frame eagerly enumerates EVERY lib of
|
||||||
|
// its kind at boot, one mutex-serialized bridge crossing at a time
|
||||||
|
// (g_pcbjamProxyMutex in the plugins) — a cold lib would network-fetch
|
||||||
|
// inside its serial crossing. Hold enumerates until the 8-wide presync
|
||||||
|
// settles so the crossings read warm IDB. pcbnew/eeschema are NOT
|
||||||
|
// gated: their choosers open on demand, and a mid-session park with no
|
||||||
|
// overlay would read as a hang.
|
||||||
|
const enumerateGate =
|
||||||
|
FILELESS_TOOLS.has(tool) && libKind
|
||||||
|
? (kind: string) =>
|
||||||
|
kind === libKind ? presyncSettled : Promise.resolve()
|
||||||
|
: undefined;
|
||||||
|
// Connect the doc's collab room in parallel with the wasm download (it
|
||||||
|
// needs identity, not the wasm). Errors are captured and rethrown at
|
||||||
|
// the await below — rejecting here would surface as an unhandled
|
||||||
|
// rejection while the download is still running.
|
||||||
|
const docSessionReady: Promise<
|
||||||
|
| { session?: KicadDocSession; targetBytes?: Uint8Array }
|
||||||
|
| { error: unknown }
|
||||||
|
> = (async () => {
|
||||||
|
try {
|
||||||
|
await identityReady;
|
||||||
|
return await maybeConnectDocSession(win, {
|
||||||
|
docSource,
|
||||||
|
tool,
|
||||||
|
scopeId,
|
||||||
|
projectId,
|
||||||
|
targetPath,
|
||||||
|
log: append,
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
return { error };
|
||||||
|
}
|
||||||
|
})();
|
||||||
await bootKicadTool({
|
await bootKicadTool({
|
||||||
tool,
|
tool,
|
||||||
base,
|
base,
|
||||||
|
|
@ -1409,6 +1461,7 @@ export function WasmTool({
|
||||||
warmStart: warm,
|
warmStart: warm,
|
||||||
onWasmInstantiated: () => markWasmDownloaded(meta.bundle, meta.ver),
|
onWasmInstantiated: () => markWasmDownloaded(meta.bundle, meta.ver),
|
||||||
libsSource: source,
|
libsSource: source,
|
||||||
|
enumerateGate,
|
||||||
// 3D models: lazy per-board source (null unless the CDN manifest is
|
// 3D models: lazy per-board source (null unless the CDN manifest is
|
||||||
// configured) — feeds the board prescan + the viewer's ensure fallback.
|
// configured) — feeds the board prescan + the viewer's ensure fallback.
|
||||||
modelsSource: modelsSourceConfig(),
|
modelsSource: modelsSourceConfig(),
|
||||||
|
|
@ -1456,14 +1509,11 @@ export function WasmTool({
|
||||||
},
|
},
|
||||||
}),
|
}),
|
||||||
});
|
});
|
||||||
const { session, targetBytes } = await maybeConnectDocSession(win, {
|
// The room connect started in the fan-out above — settle it before
|
||||||
docSource,
|
// staging so the target file materializes from the doc when it has one.
|
||||||
tool,
|
const docResult = await docSessionReady;
|
||||||
scopeId,
|
if ("error" in docResult) throw docResult.error;
|
||||||
projectId,
|
const { session, targetBytes } = docResult;
|
||||||
targetPath,
|
|
||||||
log: append,
|
|
||||||
});
|
|
||||||
await driveProjectIntoTool(win, {
|
await driveProjectIntoTool(win, {
|
||||||
tool,
|
tool,
|
||||||
slug,
|
slug,
|
||||||
|
|
@ -1478,15 +1528,9 @@ export function WasmTool({
|
||||||
: fetchBytes,
|
: fetchBytes,
|
||||||
log: append,
|
log: append,
|
||||||
onStatus: setStatus,
|
onStatus: setStatus,
|
||||||
|
onFileProgress: (done, total) =>
|
||||||
|
setFileSync(done >= total ? null : { done, total }),
|
||||||
});
|
});
|
||||||
// Everything the user is blocked on has now downloaded: the wasm bundle
|
|
||||||
// is up and the project's own files are staged into MEMFS with the target
|
|
||||||
// file open. NOW warm the scope's lib bundles, in the background — the
|
|
||||||
// boot tail below (collab rooms, wx UI build) is latency-bound chatter,
|
|
||||||
// not bandwidth, so the pre-sync no longer delays the editor becoming
|
|
||||||
// interactive. Whatever it hasn't reached by the time the user opens a
|
|
||||||
// chooser is fetched lazily on demand, as it always was.
|
|
||||||
startLibPresync();
|
|
||||||
// Read-only viewer (read-only-viewer): lock the wasm frame BEFORE the
|
// Read-only viewer (read-only-viewer): lock the wasm frame BEFORE the
|
||||||
// boot overlay drops — the file is open, so the frame exists; poll the
|
// boot overlay drops — the file is open, so the frame exists; poll the
|
||||||
// export like the chrome toggle does. Fails CLOSED (boot error overlay):
|
// export like the chrome toggle does. Fails CLOSED (boot error overlay):
|
||||||
|
|
@ -1618,6 +1662,10 @@ export function WasmTool({
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// Lib editors: the enumerate gate holds their whole-set hydrate until
|
||||||
|
// the presync settles — wait for it here too, so the boot overlay (with
|
||||||
|
// its ticking lib line) stays up instead of revealing an empty tree.
|
||||||
|
if (enumerateGate) await presyncSettled;
|
||||||
// Tool booted + project opened. Wait for the wx UI to actually build
|
// Tool booted + project opened. Wait for the wx UI to actually build
|
||||||
// before dropping the overlay, so we don't reveal a still-blank editor.
|
// before dropping the overlay, so we don't reveal a still-blank editor.
|
||||||
await waitForWxUi(win);
|
await waitForWxUi(win);
|
||||||
|
|
@ -1759,11 +1807,21 @@ export function WasmTool({
|
||||||
{status || "Loading…"}
|
{status || "Loading…"}
|
||||||
</p>
|
</p>
|
||||||
<DownloadProgress progress={progress} />
|
<DownloadProgress progress={progress} />
|
||||||
{/* The lib pre-sync deliberately has NO line here anymore: it now
|
{/* The parallel fan-out's other progress: project files staging
|
||||||
starts only once the wasm + project files are in (see the boot
|
into MEMFS, and the lib warm-up. The lib line says "checking"
|
||||||
IIFE), so the overlay would be showing a counter the user is not
|
on purpose — the walk visits every lib but downloads only new
|
||||||
waiting on. Its progress surfaces in the unobtrusive bottom-left
|
or changed ones, so a bare 15/155 next to the consent dialog's
|
||||||
indicator below, after `ready`. */}
|
MB figures would read as 155 big downloads. */}
|
||||||
|
{fileSync && fileSync.total > 0 && (
|
||||||
|
<p className="whitespace-pre font-mono text-xs text-white/50">
|
||||||
|
Project files — {String(fileSync.done).padStart(String(fileSync.total).length, " ")}/{fileSync.total}
|
||||||
|
</p>
|
||||||
|
)}
|
||||||
|
{libSync && (
|
||||||
|
<p className="whitespace-pre font-mono text-xs text-white/50">
|
||||||
|
{libSyncLabel(libSync)}
|
||||||
|
</p>
|
||||||
|
)}
|
||||||
<p className="font-mono text-xs text-white/40">
|
<p className="font-mono text-xs text-white/40">
|
||||||
{warmBoot
|
{warmBoot
|
||||||
? "Loading from your browser's cache — no download needed."
|
? "Loading from your browser's cache — no download needed."
|
||||||
|
|
@ -2101,8 +2159,8 @@ function libStateLabel(s: LibsSyncState | null): string | null {
|
||||||
function libNowDetail(s: LibsSyncState | null): string {
|
function libNowDetail(s: LibsSyncState | null): string {
|
||||||
const base = "this editor browses them";
|
const base = "this editor browses them";
|
||||||
// No warmth answer at all: say only what stays true regardless of counts —
|
// No warmth answer at all: say only what stays true regardless of counts —
|
||||||
// the warm-up runs in the background once the editor is open (it moved off
|
// the warm-up runs alongside the editor download and finishes in the
|
||||||
// the critical path, so "downloaded now" would be wrong as well as vague).
|
// background, so "downloaded now" would overpromise as well as vague.
|
||||||
if (!s || s.total === 0) return `${base} — fetched in the background`;
|
if (!s || s.total === 0) return `${base} — fetched in the background`;
|
||||||
const cold = s.total - s.warm;
|
const cold = s.total - s.warm;
|
||||||
if (cold === 0) {
|
if (cold === 0) {
|
||||||
|
|
@ -2241,15 +2299,18 @@ function DownloadConsent({
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Fixed-width lib pre-sync line, e.g. "Syncing symbol libraries — 42/208".
|
* Fixed-width lib pre-sync line, e.g. "Checking symbol libraries — 42/208".
|
||||||
* The prefix is constant and `done` is space-padded to `total`'s digit count,
|
* "Checking", not "downloading": the walk visits every lib of the kind but
|
||||||
* so the text stays still while the counter ticks (render it in a font-mono +
|
* downloads only the new/changed ones — a bare counter read as 155 downloads
|
||||||
* whitespace-pre element so the pad spaces hold their width).
|
* (standalone-load-ux follow-up). The prefix is constant and `done` is
|
||||||
|
* space-padded to `total`'s digit count, so the text stays still while the
|
||||||
|
* counter ticks (render it in a font-mono + whitespace-pre element so the pad
|
||||||
|
* spaces hold their width).
|
||||||
*/
|
*/
|
||||||
function libSyncLabel(s: { kind: string; done: number; total: number }): string {
|
function libSyncLabel(s: { kind: string; done: number; total: number }): string {
|
||||||
const total = String(s.total);
|
const total = String(s.total);
|
||||||
const done = String(Math.min(s.done, s.total)).padStart(total.length, " ");
|
const done = String(Math.min(s.done, s.total)).padStart(total.length, " ");
|
||||||
return `Syncing ${s.kind} libraries — ${done}/${total}`;
|
return `Checking ${s.kind} libraries — ${done}/${total}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|
|
||||||
24
web/standalone/src/wasm/boot.test.ts
Normal file
24
web/standalone/src/wasm/boot.test.ts
Normal file
|
|
@ -0,0 +1,24 @@
|
||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import { hasWritableLib } from "./boot";
|
||||||
|
import type { LibInfo } from "./libs/source";
|
||||||
|
|
||||||
|
const lib = (type?: string): LibInfo => ({ id: `id-${type}`, name: `n-${type}`, type });
|
||||||
|
|
||||||
|
describe("hasWritableLib", () => {
|
||||||
|
it("sees an example-backend 'user' lib", () => {
|
||||||
|
expect(hasWritableLib([[lib("origin"), lib("user")]])).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("sees a private-platform 'org' lib (renamed from 'user') — the My Symbols 409", () => {
|
||||||
|
expect(hasWritableLib([[lib("origin")], [lib("mirror"), lib("org")]])).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("false when only read-only libs are listed", () => {
|
||||||
|
expect(hasWritableLib([[lib("origin"), lib("mirror")], [lib(undefined)]])).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("false on empty lists", () => {
|
||||||
|
expect(hasWritableLib([[], []])).toBe(false);
|
||||||
|
expect(hasWritableLib([])).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
@ -29,6 +29,18 @@ import pcbjamDarkTheme from "./themes/pcbjam-dark.json";
|
||||||
/** The default user lib boot ensures exists, so there's a writable save target. */
|
/** The default user lib boot ensures exists, so there's a writable save target. */
|
||||||
const DEFAULT_USER_LIB_NAME = "My Symbols";
|
const DEFAULT_USER_LIB_NAME = "My Symbols";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* True when any per-kind lib list already contains a writable lib — "user" on
|
||||||
|
* the example backend / local sources, "org" on the private platform (a scope's
|
||||||
|
* own writable lib; renamed from "user" in the team-libs rework). Missing "org"
|
||||||
|
* here made boot re-create "My Symbols" on every load, which the backend 409'd.
|
||||||
|
*/
|
||||||
|
export function hasWritableLib(lists: Iterable<LibInfo[]>): boolean {
|
||||||
|
return [...lists].some((libs) =>
|
||||||
|
libs.some((l) => l.type === "user" || l.type === "org"),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Boot a KiCad tool directly in the main React document — no iframe.
|
* Boot a KiCad tool directly in the main React document — no iframe.
|
||||||
*
|
*
|
||||||
|
|
@ -85,6 +97,12 @@ export interface BootOptions {
|
||||||
* (empty lib-tables are seeded). Its libs become sym-lib-table and/or
|
* (empty lib-tables are seeded). Its libs become sym-lib-table and/or
|
||||||
* fp-lib-table rows depending on the tool (see `libKinds` in doBoot). */
|
* fp-lib-table rows depending on the tool (see `libKinds` in doBoot). */
|
||||||
libsSource?: LibsSource | null;
|
libsSource?: LibsSource | null;
|
||||||
|
/** Hold provider enumerates for a kind until this resolves — the lib editors
|
||||||
|
* pass their presync-settled promise so the boot-time whole-set hydrate
|
||||||
|
* reads a warm IndexedDB instead of cold-fetching one lib at a time inside
|
||||||
|
* the serialized bridge crossings. See `LibsProviderOptions.enumerateGate`.
|
||||||
|
* Never gates boot's own `listLibs` table seeding (a direct source call). */
|
||||||
|
enumerateGate?: (kind: string) => Promise<void>;
|
||||||
/** 3D model source (lazy, per-board). Null/omitted ⇒ the viewer renders the
|
/** 3D model source (lazy, per-board). Null/omitted ⇒ the viewer renders the
|
||||||
* bare board only, exactly as before models existed. */
|
* bare board only, exactly as before models existed. */
|
||||||
modelsSource?: Model3dSource | null;
|
modelsSource?: Model3dSource | null;
|
||||||
|
|
@ -237,6 +255,7 @@ async function doBoot(opts: BootOptions): Promise<void> {
|
||||||
onWasmInstantiated,
|
onWasmInstantiated,
|
||||||
libsSource,
|
libsSource,
|
||||||
modelsSource,
|
modelsSource,
|
||||||
|
enumerateGate,
|
||||||
} = opts;
|
} = opts;
|
||||||
// Truthful fetch label: a warm start (download-completion marker present)
|
// Truthful fetch label: a warm start (download-completion marker present)
|
||||||
// reads from the HTTP cache, so "Downloading" would be a lie — and vice versa.
|
// reads from the HTTP cache, so "Downloading" would be a lie — and vice versa.
|
||||||
|
|
@ -298,7 +317,7 @@ async function doBoot(opts: BootOptions): Promise<void> {
|
||||||
}
|
}
|
||||||
|
|
||||||
if (libsSource && libKinds.length) {
|
if (libsSource && libKinds.length) {
|
||||||
installLibsProvider(libsSource, log);
|
installLibsProvider(libsSource, log, { enumerateGate });
|
||||||
// 3D models ride the same provider (kind "model3d"): the C++ ensure fallback
|
// 3D models ride the same provider (kind "model3d"): the C++ ensure fallback
|
||||||
// and the board prescan both resolve through this source.
|
// and the board prescan both resolve through this source.
|
||||||
if (modelsSource) {
|
if (modelsSource) {
|
||||||
|
|
@ -310,12 +329,9 @@ async function doBoot(opts: BootOptions): Promise<void> {
|
||||||
// kind-agnostic containers and appear in every list).
|
// kind-agnostic containers and appear in every list).
|
||||||
const listsByKind = new Map<"symbol" | "footprint", LibInfo[]>();
|
const listsByKind = new Map<"symbol" | "footprint", LibInfo[]>();
|
||||||
for (const k of libKinds) listsByKind.set(k, await libsSource.listLibs(k));
|
for (const k of libKinds) listsByKind.set(k, await libsSource.listLibs(k));
|
||||||
// Ensure the owner has at least one writable user lib to save items into.
|
// Ensure the owner has at least one writable lib to save items into.
|
||||||
// A user lib holds either kind, so the created lib joins every table.
|
// A writable lib holds either kind, so the created lib joins every table.
|
||||||
const hasUserLib = [...listsByKind.values()].some((libs) =>
|
if (libsSource.createLib && !hasWritableLib(listsByKind.values())) {
|
||||||
libs.some((l) => l.type === "user"),
|
|
||||||
);
|
|
||||||
if (libsSource.createLib && !hasUserLib) {
|
|
||||||
const created = await libsSource.createLib(DEFAULT_USER_LIB_NAME);
|
const created = await libsSource.createLib(DEFAULT_USER_LIB_NAME);
|
||||||
if (created) {
|
if (created) {
|
||||||
for (const libs of listsByKind.values()) libs.push(created);
|
for (const libs of listsByKind.values()) libs.push(created);
|
||||||
|
|
|
||||||
|
|
@ -94,6 +94,24 @@ describe("MEMFS project staging", () => {
|
||||||
expect(peak).toBeLessThanOrEqual(8);
|
expect(peak).toBeLessThanOrEqual(8);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("reports per-file progress from 0 up to the total", async () => {
|
||||||
|
const { win } = fakeWin();
|
||||||
|
const files = ["a.kicad_sym", "b.kicad_sym", "c.kicad_sym"];
|
||||||
|
const seen: Array<[number, number]> = [];
|
||||||
|
|
||||||
|
await driveProjectIntoTool(win, {
|
||||||
|
...opts(files, async () => new Uint8Array([1])),
|
||||||
|
onFileProgress: (done, total) => seen.push([done, total]),
|
||||||
|
});
|
||||||
|
|
||||||
|
// One up-front (0, total) so the line shows immediately, then one tick per
|
||||||
|
// staged file, ending exactly at the total.
|
||||||
|
expect(seen[0]).toEqual([0, files.length]);
|
||||||
|
expect(seen).toHaveLength(files.length + 1);
|
||||||
|
expect(seen.map(([done]) => done).sort((a, b) => a - b)).toEqual([0, 1, 2, 3]);
|
||||||
|
expect(seen.at(-1)?.[1]).toBe(files.length);
|
||||||
|
});
|
||||||
|
|
||||||
it("rejects when a fetch fails", async () => {
|
it("rejects when a fetch fails", async () => {
|
||||||
const { win } = fakeWin();
|
const { win } = fakeWin();
|
||||||
const files = ["ok1.kicad_sym", "bad.kicad_sym", "ok2.kicad_sym"];
|
const files = ["ok1.kicad_sym", "bad.kicad_sym", "ok2.kicad_sym"];
|
||||||
|
|
|
||||||
|
|
@ -22,6 +22,10 @@ export interface DriveOptions {
|
||||||
fetchBytes: (relPath: string) => Promise<Uint8Array>;
|
fetchBytes: (relPath: string) => Promise<Uint8Array>;
|
||||||
log: (msg: string) => void;
|
log: (msg: string) => void;
|
||||||
onStatus: (text: string) => void;
|
onStatus: (text: string) => void;
|
||||||
|
/** Per-file staging progress (files fetched+written so far, total) — drives
|
||||||
|
* the boot overlay's "Project files — n/m" line. Reported once up front
|
||||||
|
* with done=0 so the line appears as soon as staging starts. */
|
||||||
|
onFileProgress?: (done: number, total: number) => void;
|
||||||
}
|
}
|
||||||
|
|
||||||
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
|
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
|
||||||
|
|
@ -83,11 +87,14 @@ const STAGE_CONCURRENCY = 8;
|
||||||
async function syncProjectToMemfs(win: ToolWindow, opts: DriveOptions): Promise<void> {
|
async function syncProjectToMemfs(win: ToolWindow, opts: DriveOptions): Promise<void> {
|
||||||
getFS(win).mkdirTree(memfsProjectDir(opts.slug));
|
getFS(win).mkdirTree(memfsProjectDir(opts.slug));
|
||||||
|
|
||||||
|
let staged = 0;
|
||||||
|
opts.onFileProgress?.(0, opts.files.length);
|
||||||
const queue = [...opts.files];
|
const queue = [...opts.files];
|
||||||
const worker = async (): Promise<void> => {
|
const worker = async (): Promise<void> => {
|
||||||
for (let file = queue.shift(); file; file = queue.shift()) {
|
for (let file = queue.shift(); file; file = queue.shift()) {
|
||||||
const bytes = await opts.fetchBytes(file.path);
|
const bytes = await opts.fetchBytes(file.path);
|
||||||
restageFile(win, opts.slug, file.path, bytes, opts.log);
|
restageFile(win, opts.slug, file.path, bytes, opts.log);
|
||||||
|
opts.onFileProgress?.(++staged, opts.files.length);
|
||||||
// 3D models: prefetch every model this board references (R2 → IDB → MEMFS)
|
// 3D models: prefetch every model this board references (R2 → IDB → MEMFS)
|
||||||
// so the 3D viewer's first open resolves locally. Fire-and-forget — project
|
// so the 3D viewer's first open resolves locally. Fire-and-forget — project
|
||||||
// open never waits on it; a ref that misses falls back to the C++ per-model
|
// open never waits on it; a ref that misses falls back to the C++ per-model
|
||||||
|
|
|
||||||
|
|
@ -152,3 +152,59 @@ describe("installLibsProvider — fat list (arg=bodies)", () => {
|
||||||
expect(await request("index", "/mnt/pcbjam/", "", "footprint")).toBeNull();
|
expect(await request("index", "/mnt/pcbjam/", "", "footprint")).toBeNull();
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("installLibsProvider — enumerate gate (load-fanout)", () => {
|
||||||
|
beforeEach(() => {
|
||||||
|
(globalThis as unknown as { window: unknown }).window = {
|
||||||
|
location: { search: "" },
|
||||||
|
dispatchEvent: () => true,
|
||||||
|
};
|
||||||
|
});
|
||||||
|
afterEach(() => {
|
||||||
|
delete (globalThis as unknown as { window?: unknown }).window;
|
||||||
|
});
|
||||||
|
|
||||||
|
it("parks list (names AND bodies) until the gate resolves; get is never gated", async () => {
|
||||||
|
let release!: () => void;
|
||||||
|
const gate = new Promise<void>((r) => {
|
||||||
|
release = r;
|
||||||
|
});
|
||||||
|
const gateKinds: string[] = [];
|
||||||
|
installLibsProvider(
|
||||||
|
baseSource({ getAllItems: async () => itemsAsBytes() }),
|
||||||
|
() => {},
|
||||||
|
{
|
||||||
|
enumerateGate: (kind) => {
|
||||||
|
gateKinds.push(kind);
|
||||||
|
return gate;
|
||||||
|
},
|
||||||
|
},
|
||||||
|
);
|
||||||
|
const request = (
|
||||||
|
globalThis as unknown as { window: { kicadLibs: { request: Req } } }
|
||||||
|
).window.kicadLibs.request;
|
||||||
|
|
||||||
|
let fatDone = false;
|
||||||
|
let namesDone = false;
|
||||||
|
const fat = request("list", libUri("Device"), "bodies", "symbol").then((r) => {
|
||||||
|
fatDone = true;
|
||||||
|
return r;
|
||||||
|
});
|
||||||
|
const names = request("list", libUri("Device"), "", "symbol").then((r) => {
|
||||||
|
namesDone = true;
|
||||||
|
return r;
|
||||||
|
});
|
||||||
|
// A user-triggered item open resolves while the gate is still held — only
|
||||||
|
// enumerates are parked.
|
||||||
|
await expect(request("get", libUri("Device"), "R", "symbol")).resolves.toBe(
|
||||||
|
"(kicad_symbol_lib (symbol R))",
|
||||||
|
);
|
||||||
|
expect(fatDone).toBe(false);
|
||||||
|
expect(namesDone).toBe(false);
|
||||||
|
|
||||||
|
release();
|
||||||
|
expect(parseFramed(await fat, "symbols")).toHaveLength(2);
|
||||||
|
expect(JSON.parse((await names) as string)).toEqual({ symbols: ["R", "C"] });
|
||||||
|
expect(gateKinds).toEqual(["symbol", "symbol"]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,10 @@ export interface LibInfo {
|
||||||
/** Display nickname for the sym-lib-table row. */
|
/** Display nickname for the sym-lib-table row. */
|
||||||
name: string;
|
name: string;
|
||||||
description?: string | null;
|
description?: string | null;
|
||||||
/** 'origin' | 'mirror' | 'user' — drives "ensure a user lib exists" at boot. */
|
/** 'origin' | 'mirror' | 'user' | 'org' — drives "ensure a writable lib
|
||||||
|
* exists" at boot. 'org' is the private platform's name for a scope's own
|
||||||
|
* writable lib (renamed from 'user' in the team-libs rework); the example
|
||||||
|
* backend and local sources still say 'user'. */
|
||||||
type?: string;
|
type?: string;
|
||||||
/** Item count, when the source knows it cheaply (manifest/backend) — shown as a
|
/** Item count, when the source knows it cheaply (manifest/backend) — shown as a
|
||||||
* badge in the home-page library list. Omitted ⇒ no badge. */
|
* badge in the home-page library list. Omitted ⇒ no badge. */
|
||||||
|
|
@ -310,9 +313,26 @@ export function buildFpLibTable(libsList: LibInfo[]): string {
|
||||||
* "index" -> the publish-time footprint index JSON (source-global; lib arg
|
* "index" -> the publish-time footprint index JSON (source-global; lib arg
|
||||||
* ignored) / null when the source has none — see getFpIndex.
|
* ignored) / null when the source has none — see getFpIndex.
|
||||||
*/
|
*/
|
||||||
|
export interface LibsProviderOptions {
|
||||||
|
/**
|
||||||
|
* Hold every enumerate ("list", including the fat "bodies" variant — a plain
|
||||||
|
* name list also opens the lib's sync stack, i.e. cold-fetches its bundle)
|
||||||
|
* for the given kind until the returned promise resolves. The lib editors
|
||||||
|
* pass their presync-settled promise: their frame eagerly enumerates EVERY
|
||||||
|
* lib of its kind at boot, one mutex-serialized bridge crossing at a time
|
||||||
|
* (see g_pcbjamProxyMutex in the plugins), so a cold lib would network-fetch
|
||||||
|
* inside its serial crossing — the gate lets the 8-wide parallel presync
|
||||||
|
* warm IndexedDB first and the crossings become local read + parse.
|
||||||
|
* MUST always resolve (the presync is best-effort and never rejects);
|
||||||
|
* "get"/"save"/"index" are user-triggered and never gated.
|
||||||
|
*/
|
||||||
|
enumerateGate?: (kind: string) => Promise<void>;
|
||||||
|
}
|
||||||
|
|
||||||
export function installLibsProvider(
|
export function installLibsProvider(
|
||||||
source: LibsSource,
|
source: LibsSource,
|
||||||
log: (msg: string) => void,
|
log: (msg: string) => void,
|
||||||
|
opts?: LibsProviderOptions,
|
||||||
): void {
|
): void {
|
||||||
if (window.kicadLibs) return;
|
if (window.kicadLibs) return;
|
||||||
const delay = artificialDelayMs();
|
const delay = artificialDelayMs();
|
||||||
|
|
@ -353,6 +373,9 @@ export function installLibsProvider(
|
||||||
try {
|
try {
|
||||||
switch (op) {
|
switch (op) {
|
||||||
case "list": {
|
case "list": {
|
||||||
|
// Enumerate gate (load-fanout): park this crossing (Asyncify) until
|
||||||
|
// the caller's precondition — typically the presync — has settled.
|
||||||
|
if (opts?.enumerateGate) await opts.enumerateGate(kind);
|
||||||
// Each plugin parses its own key: footprints / symbols.
|
// Each plugin parses its own key: footprints / symbols.
|
||||||
const key = kind === "footprint" ? "footprints" : "symbols";
|
const key = kind === "footprint" ? "footprints" : "symbols";
|
||||||
// "bodies" (arg) = the fat list: every item's body in one crossing, so
|
// "bodies" (arg) = the fat list: every item's body in one crossing, so
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue