AI_Assistant/pcbjam
Watch
1
0
Fork
You've already forked pcbjam
 0
Code Issues Pull requests Projects Releases 7 Packages Wiki Activity Actions

feat(wasm): occ-split — lazy occ_service worker; kicad_editor drops OCC (−31%)

Browse source 
Move OpenCASCADE out of the merged editor image into occ_service: a separate
emscripten module (-sASYNCIFY=0, MODULARIZE, in-container -Oz finalize, 2N+8
pre-warmed pthread pool) booted lazily in a dedicated Web Worker on the first
STEP export or STEP/IGES model parse. kicad_editor.wasm ~190 MB -> 130 MB;
sessions that never touch OCC never fetch its 57 MB. STEP export works in the
browser for the first time: the unchanged desktop dialog runs EXPORTER_STEP,
whose wasm shadow suspends into globalThis.occService and the export bytes go
straight to a browser download (never entering the editor heap). STEP/IGES 3D
models parse in the worker via the oce shadow (S3D WriteCache/ReadCache wire).

- wasm/occ-service/: service CMake target (hooked from the kicad fork's
  top-level CMakeLists, wasm/editor pattern), embind entry
  (occExport/occLoadModel), wxConfig pre-js.
- wasm/stubs/{exporter_step,oce_plugin}_stub.cpp: EM_ASYNC_JS worker bridges
  (callee-shadowing; no caller #ifdefs).
- web/standalone: provider installed whenever the kicad_editor bundle boots
  (cross-face safe); ONE shared worker-boot source occ-worker.js (vite ?raw;
  the e2e stub reads the same file) — blob worker with locateFile absolutized
  against the glue URL; export download-name guard.
- deps: OCC builds with RapidJSON so its glTF/GLB writer exists — pinned to
  the vcpkg master snapshot 2025-02-26 (24b5e7a8b27f), the same code official
  KiCad consumes via vcpkg.json's opencascade[rapidjson]; rapidjson's latest
  tag (v1.1.0, 2016) is ill-formed under modern clang.
- tests: occ-export dialog e2e (lazy-fetch boundary + STEP download bytes),
  occ-probe incl. a 9-format matrix (step/stpz/brep/xao/ply/stl/glb/u3d/pdf),
  3d-viewer-models hard-asserts the worker parse; occ provider stub installed
  ambiently by the kicad fixtures.

Validated against desktop kicad-cli 10.0.4: geometric exact equality (bbox
delta 0 um, volume delta 0.0000%) for STEP/GLB/STL/BREP/STPZ across three
boards and option sweeps — with desktop OCC 7.9 vs wasm OCC 7.8; PLY/XAO/PDF
structurally equal; U3D same-size (quantizer float LSBs differ). Full kicad
e2e green on Firefox and Chromium; standalone verified end to end (lazy fetch
only on the Export click; export.step 60,628 B ISO-10303-21; loadModel 700 KB
STEP -> 569 KB scenegraph cache).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Viktor Vaczi 2026-07-03 12:39:58 +02:00
commit db9d6ee04b
28 changed files with 2296 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

12
tests/kicad/3d-viewer-models.spec.ts
View file

@ -183,6 +183,8 @@ test.describe('3D viewer component models', () => {
fs.readFileSync(fixtureAbs).toString('base64'),
);
await installModelProviderStub(page);
// (.step models parse in the occ_service worker — the oce3d_Load shadow
// suspends on globalThis.occService, installed ambiently by fixtures.)
await loadBoard(page, testLogger);
@ -222,6 +224,16 @@ test.describe('3D viewer component models', () => {
);
expect(servedSize, 'served STEP written into the model root').toBeGreaterThan(1000);
// OCC split: the .step parse runs in the occ_service worker (the oce3d
// shadow bridges to it) and must SUCCEED — a boot/bridge failure logs
// 'oce Load FAILED' and silently skips the model, which the render
// assertions below can miss (hollow green).
const oceLoadLines = testLogger.consoleLogs.filter((l) => l.includes('oce Load'));
expect(oceLoadLines.some((l) => l.includes('oce Load ok')),
'the served STEP must parse in the occ_service worker').toBe(true);
expect(oceLoadLines.some((l) => l.includes('oce Load FAILED')),
'no oce model parse may fail').toBe(false);
// --- render assertion --------------------------------------------------
const render = await page.evaluate(() => {
const list = document.querySelectorAll('canvas[id^="glcanvas-"]');

11
tests/kicad/fixtures.ts
View file

@ -1,11 +1,22 @@
import { test as base } from '@playwright/test';
import * as path from 'path';
import { setupTestLogger, writeTestLogs, TestLogger, MAIN_CANVAS, waitForApp, tryLoadApp, getCanvasBox, KICAD_LOGS_DIR, getTestFileName } from '../e2e/utils/test-utils';
import { installOccServiceStub } from './utils/occ-service';
// Extend base test with automatic logging
export const test = base.extend<{
testLogger: TestLogger;
}>({
// Every harness page gets the occ_service provider ambiently (standalone
// parity: boot.ts installs it whenever the editor bundle boots). Init-script
// based, so it exists from document start on every navigation; the worker
// itself is only fetched on the first occ request, so specs can still assert
// the lazy-load boundary.
page: async ({ page }, use) => {
await installOccServiceStub(page);
await use(page);
},
testLogger: async ({ page }, use, testInfo) => {
// Build test name from describe block + test title
const testName = testInfo.titlePath.join(' - ');

147
tests/kicad/occ-export.spec.ts Normal file
View file

@ -0,0 +1,147 @@
import type { Page } from '@playwright/test';
import { test, expect } from './fixtures';
import { clickMenuBarItem, clickMenuItem } from '../e2e/utils/element-tracker';
import { injectFromSubmodule } from './utils/fs-inject';
import { waitForBoardLoaded } from './utils/board-ready';
import { waitForPcbnew } from './utils/pcbnew-ready';
/**
* STEP export through the occ_service worker (docs/features/occ-split/):
* pcbnew.wasm carries no OCC — DIALOG_EXPORT_STEP's browser branch runs
* EXPORTER_STEP, whose WASM shadow suspends via EM_ASYNC_JS and hands the
* job to `globalThis.occService` (worker with its own OCC-linked module).
*
* Asserted end to end:
* 1. occ_service.{js,wasm} is NOT fetched at boot or board load — only the
* export click triggers it (the lazy-load boundary).
* 2. The (unchanged) export dialog drives the whole chain: menu → dialog →
* Export button → worker → STEP bytes.
* 3. The result is a real STEP file (ISO-10303-21 magic, non-trivial size),
* captured by the provider stub where the app would download it.
*/
const KICAD_VERSION_DIR = '10.0';
const PROJECT_DIR_MEMFS = `/home/kicad/documents/kicad/${KICAD_VERSION_DIR}/projects`;
const DEMO = { name: 'pic_programmer', dir: 'pic_programmer', stem: 'pic_programmer' } as const;
async function loadBoard(page: Page, testLogger: { consoleLogs: string[]; errors: string[] }): Promise<void> {
const pcbFilename = `${DEMO.stem}.kicad_pcb`;
const proFilename = `${DEMO.stem}.kicad_pro`;
await injectFromSubmodule(page, `kicad/demos/${DEMO.dir}/${pcbFilename}`,
`${PROJECT_DIR_MEMFS}/${pcbFilename}`);
await injectFromSubmodule(page, `kicad/demos/${DEMO.dir}/${proFilename}`,
`${PROJECT_DIR_MEMFS}/${proFilename}`);
expect(await clickMenuBarItem(page, 'File'), 'File menu should be findable').toBe(true);
await page.waitForTimeout(400);
expect(await clickMenuItem(page, 'Open...'), 'Open… menu item should be findable').toBe(true);
await page.waitForFunction(() => {
const registry = window.wxElementRegistry;
return !!registry && registry.findAll({ visible: true })
.some((el) => el.typeName === 'wxFileDialog');
}, null, { timeout: 15000 });
await page.waitForTimeout(1000);
const filenameInput = await page.evaluate(() => {
const registry = window.wxElementRegistry;
if (!registry) return null;
const text = registry.findAll({ visible: true })
.find((el) => el.typeName === 'wxTextCtrl' && el.name === 'text');
return text ? { x: text.centerX, y: text.centerY } : null;
});
expect(filenameInput, 'filename text input should be visible').not.toBeNull();
if (!filenameInput) throw new Error('filename text input not found');
await page.mouse.click(filenameInput.x, filenameInput.y);
await page.waitForTimeout(200);
await page.keyboard.type(pcbFilename);
await page.waitForTimeout(300);
await page.keyboard.press('Enter');
await page.waitForTimeout(1000);
const result = await waitForBoardLoaded(page, testLogger, 60000);
console.log(`[TEST] ${DEMO.name} board-ready result: ${result}`);
}
/** Click a visible wx button by label; returns whether it was found. */
async function clickWxButton(page: Page, label: string): Promise<boolean> {
const pos = await page.evaluate((wanted: string) => {
const registry = window.wxElementRegistry;
if (!registry) return null;
const el = registry.findAll({ visible: true })
.find((e) => (e.label === wanted || e.label === `&${wanted}`)
&& (e.typeName ?? '').includes('Button'));
return el ? { x: el.centerX, y: el.centerY } : null;
}, label);
if (!pos) return false;
await page.mouse.click(pos.x, pos.y);
return true;
}
test.describe('OCC export via occ_service worker', () => {
test.describe.configure({ mode: 'serial' });
test.setTimeout(240000);
test('export dialog produces a valid STEP; occ_service fetches lazily', async ({ page, testLogger }) => {
// Track occ_service fetches from the very start — the lazy boundary is
// the core assertion.
const occFetches: string[] = [];
page.on('request', (r) => {
if (r.url().includes('occ_service')) occFetches.push(r.url());
});
await page.goto('/kicad/pcbnew.html');
await waitForPcbnew(page);
await loadBoard(page, testLogger);
expect(occFetches, 'occ_service must NOT be fetched before the export').toHaveLength(0);
// File → Export → STEP/GLB/…
expect(await clickMenuBarItem(page, 'File'), 'File menu').toBe(true);
await page.waitForTimeout(400);
expect(await clickMenuItem(page, 'Export'), 'Export submenu').toBe(true);
await page.waitForTimeout(400);
expect(await clickMenuItem(page, 'STEP/GLB/BREP/XAO/PLY/STL...'),
'STEP export menu item').toBe(true);
// The (unchanged) DIALOG_EXPORT_STEP: wait for its Export button.
await page.waitForFunction(() => {
const registry = window.wxElementRegistry;
return !!registry && registry.findAll({ visible: true })
.some((el) => (el.label === 'Export' || el.label === '&Export')
&& (el.typeName ?? '').includes('Button'));
}, null, { timeout: 20000 });
await page.waitForTimeout(800);
await page.screenshot({ path: 'test-results/occ-export-dialog.png', scale: 'css' });
expect(await clickWxButton(page, 'Export'), 'Export button click').toBe(true);
// The provider stub captures the bytes where the app would download.
await page.waitForFunction(
() => ((window as any).__occExports?.length ?? 0) > 0,
null, { timeout: 180000 });
const exports = await page.evaluate(() => (window as any).__occExports as Array<{
name: string; size: number; magic: string;
}>);
console.log(`[TEST] captured exports: ${JSON.stringify(exports)}`);
expect(exports).toHaveLength(1);
expect(exports[0].name, 'download name comes from the dialog')
.toMatch(/\.step$/i);
expect(exports[0].magic.startsWith('ISO-10303-21'), 'STEP magic').toBe(true);
expect(exports[0].size, 'non-trivial STEP body').toBeGreaterThan(10_000);
expect(occFetches.length, 'occ_service was fetched lazily by the export')
.toBeGreaterThan(0);
// Dismiss the "Export complete" report dialog if present.
await page.waitForTimeout(1000);
await clickWxButton(page, 'OK');
await page.screenshot({ path: 'test-results/occ-export-done.png', scale: 'css' });
});
});

110
tests/kicad/occ-probe.spec.ts Normal file
View file

@ -0,0 +1,110 @@
import { test, expect } from './fixtures';
/**
* Minimal occ_service probe (no pcbnew boot): drive occExport directly from
* the page with a small fixture board, in whatever engine the project runs.
* Isolates worker/module behavior from the editor entirely — when export
* breaks, it answers "service module or editor-side bridge?" in seconds.
* The provider itself arrives via the fixtures' ambient init script.
*/
test.describe('occ_service probe', () => {
test.setTimeout(180000);
test('occExport of the demo board completes', async ({ page }) => {
// Any harness page gives the COI (COOP/COEP) context; don't wait for wasm.
await page.goto('/kicad/pcbnew.html', { waitUntil: 'domcontentloaded' });
const fs = require('fs') as typeof import('fs');
const path = require('path') as typeof import('path');
const board = fs.readFileSync(
path.resolve(__dirname, '..', 'fixtures', 'demo', 'demo.kicad_pcb'), 'utf8');
const res = await page.evaluate(async (boardText: string) => {
const svc = (globalThis as any).occService;
const bytes = new TextEncoder().encode(boardText);
const t0 = performance.now();
const r = await svc.request({
kind: 'export', board: bytes,
jobJson: JSON.stringify({ format: 'step' }),
fileName: 'probe.step',
});
return { ok: r.ok, report: r.report, ms: Math.round(performance.now() - t0) };
}, board);
console.log(`[PROBE] ok=${res.ok} in ${res.ms}ms report=${(res.report || '').slice(0, 300)}`);
const captured = await page.evaluate(() => (window as any).__occExports);
console.log(`[PROBE] captured: ${JSON.stringify(captured)}`);
await page.screenshot({ path: 'test-results/occ-probe-done.png', scale: 'css' });
expect(res.ok, 'demo board exports').toBe(true);
});
// Every format the export dialog offers must round-trip through the worker.
// Validated against kicad-cli 10.0.4 (2026-07-03): STEP/STPZ/BREP/XAO/PLY/
// STL/GLB geometrically exact-equal; U3D same structure (quantizer float
// LSBs differ cross-platform); PDF structurally equal. GLB regressed once
// already (OCC built without RapidJSON compiles the glTF writer out) —
// this matrix is the net for that class of build-flag loss.
test('format matrix: every dialog format exports through the worker', async ({ page }) => {
await page.goto('/kicad/pcbnew.html', { waitUntil: 'domcontentloaded' });
const fs = require('fs') as typeof import('fs');
const path = require('path') as typeof import('path');
const board = fs.readFileSync(
path.resolve(__dirname, '..', 'fixtures', 'demo', 'demo.kicad_pcb'), 'utf8');
// format -> expected magic prefix of the produced bytes (utf8-decoded).
const TEXT_MAGIC: Record<string, string> = {
step: 'ISO-10303-21',
// The file leads with "\nCASCADE Topology V1…" and the stub captures
// only 16 bytes — match the leading-newline-trimmed prefix.
brep: 'CASCADE Topolog',
xao: '<?xml',
ply: 'ply',
stl: 'solid',
glb: 'glTF',
u3d: 'U3D',
pdf: '%PDF',
};
for (const [fmt, magic] of Object.entries(TEXT_MAGIC)) {
const r = await page.evaluate(async ({ boardText, fmt }) => {
const svc = (globalThis as any).occService;
return await svc.request({
kind: 'export',
board: new TextEncoder().encode(boardText),
jobJson: JSON.stringify({ format: fmt, export_components: false }),
fileName: `probe.${fmt}`,
});
}, { boardText: board, fmt });
expect(r.ok, `${fmt} export reports ok`).toBe(true);
}
// stpz is gzip — binary magic, checked by charcode below.
const rz = await page.evaluate(async (boardText: string) => {
const svc = (globalThis as any).occService;
return await svc.request({
kind: 'export',
board: new TextEncoder().encode(boardText),
jobJson: JSON.stringify({ format: 'stpz', export_components: false }),
fileName: 'probe.stpz',
});
}, board);
expect(rz.ok, 'stpz export reports ok').toBe(true);
const captured: Array<{ name: string; size: number; magic: string }> =
await page.evaluate(() => (window as any).__occExports);
console.log(`[PROBE] formats captured: ${JSON.stringify(captured.map(
(c) => ({ name: c.name, size: c.size })))}`);
for (const [fmt, magic] of Object.entries(TEXT_MAGIC)) {
const hit = captured.find((c) => c.name === `probe.${fmt}`);
expect(hit, `${fmt} bytes captured`).toBeTruthy();
expect(hit!.magic.trimStart().startsWith(magic), `${fmt} magic "${magic}"`).toBe(true);
expect(hit!.size, `${fmt} non-trivial size`).toBeGreaterThan(1000);
}
const z = captured.find((c) => c.name === 'probe.stpz');
expect(z, 'stpz bytes captured').toBeTruthy();
expect(z!.magic.charCodeAt(0), 'stpz gzip magic byte 0').toBe(0x1f);
expect(z!.size, 'stpz non-trivial size').toBeGreaterThan(1000);
});
});

102
tests/kicad/utils/occ-service.ts Normal file
View file

@ -0,0 +1,102 @@
import * as fs from 'fs';
import * as path from 'path';
import type { Page } from '@playwright/test';
/**
* Install a REAL `globalThis.occService` provider into a harness page — the
* same worker-backed occ_service boot the standalone app does, minus the CDN
* manifest resolution: the harness serves occ_service.{js,wasm} same-origin
* next to the tool page (tests/scripts/setup-kicad-wasm.sh copies them from
* output/).
*
* The worker-side wrapper is the SHARED source of truth
* (web/standalone/src/wasm/occ-worker.js — the standalone imports it via vite
* `?raw`; the harness reads it off disk and injects it verbatim), so the
* trap-prone boot logic (blob worker + locateFile absolutization + pthread
* mainScriptUrlOrBlob) cannot drift between app and tests.
*
* Differences from the app provider, for assertability:
* - export results are captured into window.__occExports (name, size, magic
* prefix) instead of triggering a browser download;
* - installed as an init script (kicad fixtures do this for every page), so
* it exists from document start on every navigation — standalone parity,
* where boot.ts installs the provider whenever the editor bundle boots.
*
* The worker fetches occ_service.js lazily on the FIRST request — specs can
* assert the lazy-load boundary by watching network requests.
*/
const OCC_WORKER_SRC = fs.readFileSync(
path.resolve(__dirname, '..', '..', '..',
'web', 'standalone', 'src', 'wasm', 'occ-worker.js'),
'utf8');
export async function installOccServiceStub(page: Page): Promise<void> {
await page.addInitScript((workerSrc: string) => {
if ((globalThis as any).occService) return;
(window as any).__occExports = [];
let workerP: Promise<Worker> | null = null;
const pending = new Map<number, (res: any) => void>();
let nextId = 1;
const ensureWorker = (): Promise<Worker> => {
if (!workerP) {
workerP = (async () => {
const glue = new URL('occ_service.js', window.location.href).href;
console.log(`[TEST-OCC] booting occ_service from ${glue}`);
const worker = new Worker(URL.createObjectURL(new Blob(
[`self.OCC_GLUE_URL = ${JSON.stringify(glue)};\n`, workerSrc],
{ type: 'text/javascript' })));
worker.onmessage = (e) => {
const { id, res } = e.data ?? {};
if (typeof id !== 'number') return;
const resolve = pending.get(id);
if (resolve) { pending.delete(id); resolve(res); }
};
await new Promise<void>((resolve, reject) => {
const onFirst = (e: MessageEvent) => {
if (e.data?.ready) { worker.removeEventListener('message', onFirst); resolve(); }
else if (e.data?.bootError) reject(new Error(e.data.bootError));
};
worker.addEventListener('message', onFirst);
});
console.log('[TEST-OCC] occ_service ready');
return worker;
})().catch((e) => { workerP = null; throw e; });
}
return workerP;
};
const request = async (req: any) => {
let worker: Worker;
try {
worker = await ensureWorker();
} catch (e) {
return { ok: false, report: `occ_service unavailable: ${e}` };
}
const id = nextId++;
const transfer = req.kind === 'export' ? [req.board.buffer] : [req.bytes.buffer];
const res: any = await new Promise((resolve) => {
pending.set(id, resolve);
worker.postMessage({ id, req }, transfer);
});
if (req.kind === 'export') {
if (res.ok && res.bytes?.length) {
const magic = new TextDecoder().decode(res.bytes.slice(0, 16));
(window as any).__occExports.push({
name: req.fileName || res.fileName,
size: res.bytes.length,
magic,
});
console.log(`[TEST-OCC] export captured: ${req.fileName} ${res.bytes.length}B "${magic}"`);
}
return { ok: res.ok, report: res.report, fileName: res.fileName };
}
return res;
};
(globalThis as any).occService = { request };
}, OCC_WORKER_SRC);
}