fix(ci): serialize the heavy 3D-viewer e2e specs (dedicated chromium-ci-3d project)

The three heavy 3D-viewer specs (3d-viewer, 3d-viewer-deadlock, 3d-viewer-models)
crashed on CI (run 28604015154): "Target crashed", "browser has been closed", and a
black frozen canvas. Each boots the 3D-enabled pcbnew build, which pre-warms
~hardwareConcurrency*2+8 (~68 on the 30-core VM) Web Workers AND runs a multi-threaded
CPU raytracer over SwiftShader software-WebGL. Under fullyParallel the three spec FILES
run in separate browser processes concurrently, and the simultaneous raytrace tabs
exhaust the shared GPU process's ~16 live-WebGL-context limit / renderer memory. The
pool pre-warm (the deadlock fix, 7630c7e) is correct and present in the tested binary;
per-file isolation prevented in-process Worker accumulation but NOT this cross-process
concurrency — the gap these specs' first CI exposure hit.

Fix (test-orchestration only; no wasm/submodule changes):
- Route the three specs to a dedicated `chromium-ci-3d` Playwright project (own
  outputDir=pw-artifacts/kicad-3d so its failure traces survive the later perf run's
  wipe; the existing tests/pw-artifacts/** upload already captures it).
- test:kicad:ci runs it as a SECOND `playwright test` pass with --workers=1, so at most
  one raytrace tab is alive at a time (the environment in which a single 3D load is
  reliably green — docs/features/3d-raytracer/README.md §7). An exit-code-aggregating
  sequencer runs both phases and fails if either fails (no silent skip of the 3D pass).
- Cap phase-1 CI workers to 12 as headroom under the WebGL-context cap.
- Log a resource-diagnostic snapshot (hardwareConcurrency, pthread pool, GL-canvas
  count, wasm/JS heap) before each 3D interaction so any recurrence is attributable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Istvan Matejcsok 2026-07-03 10:49:23 +02:00
commit add7c60975
6 changed files with 87 additions and 8 deletions

View file

@ -63,6 +63,37 @@ export function countGlCanvases(page: Page): Promise<number> {
return page.evaluate(() => document.querySelectorAll('canvas[id^="glcanvas-"]').length);
}
// Resource-diagnostic snapshot for the heavy 3D-viewer specs. On CI these can kill the tab
// (renderer OOM, a lost WebGL context when the shared GPU process passes its ~16-context cap,
// or a raytracer pthread deadlock) and the failure surfaces identically ("Target crashed" /
// black canvas). Logging hardwareConcurrency, the emscripten pthread-pool ledger, the live
// GL-canvas count, and the wasm/JS heap sizes right before an interaction makes any recurrence
// attributable (GL-context exhaustion vs heap-OOM vs deadlock) — and confirms, on the real CI
// VM, what navigator.hardwareConcurrency actually is (the whole 2N+8 pool sizing depends on it).
// Best-effort: never throws, so it can't itself fail a spec.
export async function logThreeDDiag(page: Page, label: string): Promise<void> {
const snap = await page.evaluate(() => {
const w = window as unknown as { Module?: Record<string, unknown> };
const M = (w.Module ?? {}) as Record<string, unknown>;
const P = (M.PThread ?? {}) as { unusedWorkers?: unknown[]; runningWorkers?: unknown[] };
const heap = M.HEAPU8 as { length?: number } | undefined;
const perf = (performance as unknown as { memory?: Record<string, number> }).memory ?? {};
const mb = (b?: number) => (typeof b === 'number' ? Math.round(b / 1048576) : null);
return {
hardwareConcurrency: navigator.hardwareConcurrency,
// emscripten pool ledger: pre-warmed-but-idle + currently-running Workers.
pthreadUnused: Array.isArray(P.unusedWorkers) ? P.unusedWorkers.length : null,
pthreadRunning: Array.isArray(P.runningWorkers) ? P.runningWorkers.length : null,
glCanvases: document.querySelectorAll('canvas[id^="glcanvas-"]').length,
canvases: document.querySelectorAll('canvas').length,
wasmHeapMB: mb(heap?.length),
jsHeapUsedMB: mb(perf.usedJSHeapSize),
jsHeapLimitMB: mb(perf.jsHeapSizeLimit),
};
}).catch((e: unknown) => ({ error: String(e) }));
console.log(`[DIAG ${label}] ${JSON.stringify(snap)}`);
}
// Open the 3D viewer (View → 3D Viewer, with an Alt+3 fallback) and wait for the
// secondary frame + its NEW `glcanvas-*` to appear. The main pcbnew board view is
// itself a wxGLCanvas, so the viewer is detected by the GL-canvas COUNT increasing.