emcc only runs Binaryen at link -O2+ (link.py: should_run_binaryen_optimizer
returns OPT_LEVEL >= 2) and we link at -O1, so the shipped module had never seen
wasm-opt at all — it kept its entire 19.56 MB name section, ~20% of the editor
(-sJSPI sets ASYNCIFY=2, which suppresses wasm-ld's --strip-debug, leaving
wasm-opt as the only thing that would drop it). Step 8.2 runs it post-link and
in-container, so CI's cached compile phase covers it and the host post-process
stays pure-host.
Default -O2, picked by measuring every level on the same module: -O0 already
captures 27% of the raw win (it is mostly the name section), -O2 costs 23 s and
gives the best frame rate, and -O3/-O4/-Os/-Oz cost 48-132 s for at most 1.5%
more brotli — -O4 is not even smaller than -O3. Targets that already link -O2/-Oz
(occ_service, kicad_tools) are skipped by testing for the target_features
section, which emcc strips whenever it ran the optimizer itself, so there is no
hard-coded target list to drift. Feature flags come from the module's own
target_features section and so cannot diverge from the link.
The perf specs reported requestAnimationFrame ticks as "FPS". That is not a frame
rate: rAF fires on the compositor's schedule whether or not the GAL redrew, and
it read 120/s on a board where the renderer completed zero frames in six seconds.
measureInteractionFps now counts completed GAL frames — runs of draws to the
default framebuffer, exactly one per frame in every AA mode — and drives a pure
middle-drag pan after a zoom-to-fit. Mixing wheel zoom into the drive made the
result depend on where the wheel left the view: +-20% across identical repeats,
against +-2% for pan alone. The report gains a GAL fps column with a regression
flag on the 1x number; rAF is kept so historical runs stay comparable.
CI has no GPU, so its number is a software-rasteriser redraw rate — a regression
signal, not a user-facing frame rate. Method and measurements in the bench report.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016X9eh1s5sTx1o9Em9KBuwR
The committed tests/screenshot-manifest.json is retired. CI now downloads
baselines/pcbjam/manifest.json (manifest v3, written only by the morelli
review app + its seed script) to the gitignored .baseline-manifest.json,
and everything downstream (pull, verify, compare) reads that copy:
- config.ts: MANIFEST_VERSION 3, MANIFEST_PATH .baseline-manifest.json,
R2_BASELINES_MANIFEST_KEY; ManifestEntry grows opaque provenance
- r2-sync.ts: new --manifest mode (atomic fetch; no-creds skip DELETES a
stale copy so the gate skips rather than using old baselines); --push
gone (bytes enter the CAS only via morelli's promote)
- compare.ts: hard-skips when no manifest was fetched — a stale warm
cache can never gate
- wasm-build.yml: fetch-manifest step before the baselines cache; cache
key now hashes the fetched manifest; the gen-manifest --check lint gate
goes with the committed manifest
- deleted: screenshot-manifest.json, promote.ts, changelog.ts,
gen-manifest.ts, screenshot-changelog.yml, promote-screenshots skill
- docs (CLAUDE/README/TESTING/WHATWORKS/tools README): promote flow is
now https://pcbjam-morelli-staging.pcbjam-staging.workers.dev
Validated locally against the real bucket: fetch-manifest (492), cold
pull 492 / warm pull cached=492, no-creds skip chain, compare gate skip.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Durable per-run screenshot store for the morelli review app
(github.com/PCBJam/morelli): after the report step, CI uploads the renders
and a meta.json index (identity, branch/commit, per-shot sha256+dims,
embedded compare summary) to runs/pcbjam/<GITHUB_RUN_ID>/ — 30-day R2
lifecycle; GH artifacts remain the debugging archive. meta.json is written
last as the upload-complete marker. Uses a new optional WRITE keypair
(CI_SCREENSHOTS_S3_WRITE_*); without it the step no-ops, so secretless
callers stay green. R2Store gains putKey() for the non-CAS run keys.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- compare gate: refuse an INCOMPLETE cache, not just an empty one — a partial
R2 fetch (continue-on-error step) previously misreported un-fetched
baselines as ADDED and silently disabled their removed-detection.
- post-discord: a missing report.json now posts "gate SKIPPED", never the
false "✅ no screenshot drift" — a disabled pipeline must look disabled.
- promote: hard-fail on a pre-migration (non-v2) manifest instead of warning;
the warn path could commit manifest hashes never uploaded to R2.
- changelog: tolerate per-image R2 failures (skip + loud note, like the old
git-blob null-skip) instead of aborting the whole post; missing creds warn
+ exit 0 (notification-only workflow); git-show failures are loud errors,
no longer mistaken for "predates the migration".
- version guard: a manifest NEWER than the tooling throws everywhere instead
of reading as a silent no-op.
- manifest ordering: locale-independent code-unit comparator in writer +
checker (localeCompare depends on host locale; writer=dev Mac, checker=CI).
- r2-sync pull: byte-size pre-filter before hashing cached files.
- wasm-build.yml: restore-keys on the baseline cache — manifest changes now
restore the previous tree and download only the delta.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Baseline PNGs (492, 35 MB) leave git: they now live in the private R2 bucket
pcbjam-ci-screenshots as immutable sha256/<hex>.png objects, pinned by the
committed screenshot-manifest.json (v2: name, engine, sha256, bytes, dims).
tests/baseline-screenshots/ becomes a gitignored cache materialized by the new
`npm run screenshots:fetch` (r2-sync.ts + aws4fetch r2-store.ts).
- promote.ts: requires the RW keypair (shell env or gitignored tests/.env),
syncs the cache, uploads new hashes BEFORE rewriting the manifest; the
manifest diff is the only git-visible output. --prune only edits the
manifest — R2 objects are never deleted, old commits still resolve.
- compare.ts: skips the gate (exit 0, no report.json) when the manifest
expects baselines but the cache is empty (secretless callers).
- changelog.ts: diffs the manifest between revs and fetches bytes from R2;
guards against the migration commit (base manifest not v2 → skip).
- gen-manifest --check: v2 schema + resurrection guard (fails if baseline
PNGs are ever re-committed); credential-free so every caller can gate.
- wasm-build.yml: declares optional read-only S3 secrets, caches + fetches
baselines before the lint gate; release.yml/deploy-staging.yml pass them.
- screenshot-changelog.yml: triggers on the manifest path instead of PNGs.
All 492 objects are seeded and hash-verified in the bucket; fetch/compare
degrade to a warn-and-skip without credentials.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Squash of experiment/ff-big-modules vs main.
Big-module routing removed: native-EH shrank kicad_editor below
SpiderMonkey's x86-64 code budget (runs 29355049705/29356152413 green on
stock Firefox), so BIG_MODULE_SPECS routing and the baseline-only-JIT
crutch are gone — kicad-firefox and kicad-chromium both run the full
suite, with the module compiled the way real users' browsers compile it.
Per-engine screenshots end to end: stableShot/shotPath write
test-results/<engine>/<name>.png; baselines move to
baseline-screenshots/{chromium,firefox}/ and the whole tools/screenshots
pipeline (compare/promote/manifest/spec-map/changelog/Discord) keys on
<engine>/<name>. Previously Firefox and Chromium renders of one spec
overwrote each other and Firefox renders were never actually gated.
Seeded from CI run 29421380806 (92 new firefox baselines, +24 chromium
web-suite shots); manifest generated from the baseline tree.
One merged playwright.config.ts (kicad/asyncify/coroutine/perf as
projects); ~25 dead npm scripts dropped. The web suite is gated in CI for
the first time ever (4 rotted specs fixed, 5 broken lib-bridge specs
triaged as fixme in docs/features/web-e2e-rot/); cheap lint step after
npm ci; last 26 blind-sleep violations fixed.
SwiftShader retired: CI Chromium renders WebGL on ANGLE → Mesa llvmpipe
(--use-gl=angle --use-angle=gl --ignore-gpu-blocklist; the blocklist flag
is mandatory — llvmpipe is blocklisted and WebGL is silently unavailable
without it) in BOTH configs. Under WORKERS=4 congestion SwiftShader
transiently failed the first post-board-load draw and the recovery
cascade ended in a silent permanent Cairo fallback — that engine flip was
the "~1.2% changedRatio both directions" occ-export baseline flake.
Validated 160/160 across two 80-repeat rigs; full analysis in
docs/features/wx-parity-bugs/occ-export-context-eviction.md. Chromium
baselines shift slightly on llvmpipe — promote once from the first green
run. Deflakes the new coverage exposed: presence baselines settle before
capture; presence fixtures declare current file formats; perf gets its
own outputDir so CI evidence survives; occ-export settles the board paint
before the export dialog; menu-item waits (waitForRenderedByLabel before
clickMenuItem) in 4 specs + the TESTING.md rule.
Web suite runs the PROD build, in parallel: webServer becomes backend
`start` + the standalone's e2e:preview (build-preview.mjs: link-wasm →
stash the public/wasm symlink aside during vite build, build-demo.mjs's
move — then vite preview as the persistent server). The wasm middleware
serves /wasm/* in preview and emits COOP/COEP/CORP itself (a pthread
worker script's own response must carry COEP or Chrome kills it with
ERR_BLOCKED_BY_RESPONSE). VITE_* flags bake at build time;
VITE_ALLOW_USER_OVERRIDE joins turbo globalEnv. fullyParallel + default
workers: 5.2m → 1.4m. Determinism fixes the parallel run exposed:
shared-page specs become serial groups; locks.spec grabs alice's exact
item via the new kicadCollabTestSelectByUuid hook (cross-tab "first
footprint" order is not a ysync invariant); quit specs poll page.url()
(quit supersedes its own navigation — NS_BINDING_ABORTED on Firefox).
Suite: 51 passed / 12 skipped / 0 failed in 1.6m.
CI-coverage gate (lint:ci-coverage): every tests/**/*.spec.ts must be
reachable from the npm scripts the workflows invoke — scraped from
.github/workflows/, resolved through package.json, coverage asked from
playwright --list itself. Rules: uncovered-spec + orphan-project (with a
documented LOCAL_ONLY_PROJECTS allowlist). Gating next to
lint:determinism; 138 spec files / 13 projects accounted for.
Product fixes kept from the investigations (reachable on real GPUs too):
wx 7799fd1be5 — paint flags clear before dispatch + Invalidate always
propagates; kicad 3dcfea5e45 — SwiftShader pass-boundary flush +
per-instance font texture + first-frame GL-error drain (GAL recovery
recovers instead of falling back to Cairo) + the user-facing eeschema
switch navigates again under __EMSCRIPTEN__ (project-sync's
FaceRegistered gate had rerouted it into the hidden sync player; caught
by the newly-gated web suite).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018eUxiPApHgGiu9NFyQfhAq
Reads the compare-dirs report.json of the parity + webgl-self levels (when
present — inert for builds without the 3D harness) and appends one line:
'✅ 3D renderer: parity 47/47 · webgl-self 47/47', or on drift the changed
count + worst offender with a pointer to the run-artifact triptychs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
compare-dirs gains --artifacts changed|always: 'always' also writes the
side-by-side triptych + heatmap for pairs that PASS the floor (blue-grey
UNCHANGED caption), so a human can eyeball old-vs-new without a failing
gate. The gating scripts keep the default (empty diff dir = green);
3d:review writes the full 47-pair set to output/diff/parity-review/.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- baseline-webgl/: first population from the passing local Chromium renders
(47/47 under the parity floor) — activates the webgl-self regression gate
(floor 0.005). CI may re-promote its own renders if the Linux/swiftshader
rasterization differs beyond the floor (the README's CI-promote flow).
- compare-dirs.ts: recreate the output dir on every run — triptychs/heatmaps
are only written for CHANGED pairs, so artifacts from earlier (redder) runs
lingered and misreported scenarios that had since gone green (a stale
red-state gen-cylinder triptych survived three green runs).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tests/3d-regression mirrors the gal-regression pattern at renderer scale:
shared C++ scenarios call real KiCad 3D-viewer code (opengl_utils, display
lists + DrawCulled stencil subtraction, MODEL_3D VBOs, private generators via
a rob-template accessor, and full reload()+Redraw() composites over a
synthetic BOARD_ADAPTER). A native macOS harness renders them on real OpenGL
into 47 committed goldens (bit-deterministic, FBO capture); the wasm harness
compiles the same TUs against wasm/stubs/gl_ffp_stub.c no-ops so every
scenario renders blank — the TDD red state (parity meter: 47/47 changed).
Comparisons use the CI pixelmatch engine via the new generic compare-dirs.ts
(floors.json levels; manifest.json cmp-guards registry drift).
Documents an upstream bug: appendPostMachiningGeometry's countersink path
adds middle quads without normals, silently erasing the walls of any
display list it is batched into (3d-post-machining.png keeps the lists
separate to record it).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bake a caption strip onto every posted screenshot composite — status + name +
the spec that produces it — for changed, added, and removed; removed now shows
the old baseline image (was a text-only line). Zero native-dep: an embedded
public-domain 8x8 bitmap font (font8x8.ts) rendered by image-ops `withBottomLabel`;
the name→spec attribution is factored out of gen-manifest into a shared
`spec-map.ts` resolver. Bottom strip, colour per status (green/red/orange).
Applies in both compare (drift gate) and changelog (git-history diff), and
post-discord now attaches the captioned removed images.
Also raise the per-engine drift floor 0.2% → 0.5% (changedRatio) to absorb the
sub-1% inter-run flakiness seen after the re-baseline, while still catching real
localized changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Finalize the screenshot review system after the first Linux re-baseline
(that run showed 355/356 stable — the placeholder floor is fine as-is):
- ENFORCE the gate: `screenshots:check --fail-on-change` now fails the build on any
changed/added/removed vs baselines (it still posts the drift report first, so the
failure is actionable; a real render change → re-promote). The generic "CI failed"
notice is suppressed for gate failures so we don't double-post on drift.
- Exclude retinascale-01-loaded: a fullPage HiDPI test whose captured height + DPR
scaling vary run-to-run (~60% inter-run diff) — a flaky test, not render noise.
IGNORE_SCREENSHOTS in config.ts; compare/promote/gen-manifest skip it; baseline removed.
- Kill the baseline-dir shadowing: drop e2e/baseline-screenshots/ from BASELINE_DIRS +
delete its 3 files (grid-tab-final, wxgrid-controls, wxgrid-dedicated-page) that
duplicated names in baseline-screenshots/ with different bytes.
- Prune 12 stale baselines (renamed/removed specs: wizard-01..04, gerbview-wizard-01..04,
zoom-pl_editor-*, popup-03-palette).
- Delete the dead compare-screenshots.sh / update-baseline-screenshots.sh.
- Regenerate screenshot-manifest.json (355 entries).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The screenshot report went silent on green CI runs. CI runs four sequential
`playwright test` invocations (wx, asyncify, kicad, perf) that all default their
outputDir to `test-results`; Playwright clears the whole outputDir at the start
of every run, so each suite wiped the previous suite's screenshots. By the time
`compare` ran, only perf-*.json remained -> all 354 baselines read as "removed"
-> the no-render guard stayed silent.
- Gate Playwright's outputDir off test-results/ in CI only
(process.env.CI ? 'pw-artifacts/<suite>' : 'test-results') in the wx / asyncify
/ kicad configs, so the committed-baseline screenshots (page.screenshot to
'test-results/...') accumulate in test-results/ across the suites for compare.
Local single-suite behavior is unchanged.
- post-discord: replace the silent no-render guard with a loud, image-less
pipeline alert ("No screenshots produced this run") when 0 render; genuine
partial removals still post as the normal capped REMOVED list.
- gitignore pw-artifacts/; upload tests/pw-artifacts/** as a CI artifact.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Refer to demo.pcbjam.com from the landing: an accent "Live demo" link in the
header nav, an "Open the live editor" hero link, a note in the gerber section,
and a "Try the live demo now" line in the final CTA.
Rework the embedded gerber viewer (public/gerber-demo/boot.js) to source the
WASM from the deploy pipeline's versioned CDN instead of the hand-synced
assets.pcbjam.com bucket: resolve gerbview's content-addressed folder at runtime
from the release manifest (manifest-latest -> tag -> manifest-<tag> -> gerbview),
load all assets (glue + wasm + images.tar.gz) from cdn.pcbjam.com, and load the
cross-origin pthread worker via a same-origin blob importScripts shim (mirrors
web/standalone/src/wasm/boot.ts). Bump the config-seed KICAD_VERSION_DIR
9.99 -> 10.0 to match the deployed build. Drop the now-obsolete committed glue
mirror and the old assets.pcbjam.com sync/r2-deploy scripts.
Collapse the header nav to the hamburger below 1025px — the added demo link no
longer fits the 1024px-capped bar on a single row.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New tooling in tests/tools/screenshots/ (TypeScript via tsx):
- compare.ts: one pixelmatch engine (AA-excluded), connected-component
"where to look" boxes, old|new+boxes|heatmap triptych, per-engine floors.
- promote.ts: churn-free updater — overwrite a baseline only when decoded
pixels differ beyond the floor, copying CI bytes verbatim (no re-encode
churn); pulls a CI run via `gh run download` or a local --from dir.
- post-discord.ts: always-on CI-on-main report (SHA + e2e status + the
track-only runtime-perf table), then screenshot triptychs, batched +
size-capped + flood-collapsed + 429-aware.
- perf-report.ts: perf table with Δ vs the previous main run (via gh).
- changelog.ts: no-build git-history baseline differ (Discord trigger B).
- noise.ts / gen-manifest.ts: calibration + manifest generation.
CI wiring:
- wasm-build.yml: post-test step runs the gate + report on the already-
produced test-results (no extra build); report-only (continue-on-error),
posts only on push to main, inert without DISCORD_WEBHOOK_URL.
- ci-ubicloud.yml: secrets: inherit (pass the webhook through).
- screenshot-changelog.yml: ~30s no-build changelog on baseline changes.
screenshot-manifest.json: canonical 354-name set + best-effort engine tags
(313 chromium-swiftshader / 41 firefox-llvmpipe).
Normalize scale:'device'->'css' across 18 spec files (no-op at CI DSF=1)
so committed baselines are uniformly css-scaled.
Design: CI's Linux render is the single source of truth; no pinned
container (accept rare env drift -> re-promote); dev commits via promote.
Replaces the byte-cmp compare-screenshots.sh + file-size-proxy
update-baseline-screenshots.sh (kept for now until the first re-baseline).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
