feat(tests): screenshot regression + Discord review, perf-tracked
New tooling in tests/tools/screenshots/ (TypeScript via tsx): - compare.ts: one pixelmatch engine (AA-excluded), connected-component "where to look" boxes, old|new+boxes|heatmap triptych, per-engine floors. - promote.ts: churn-free updater — overwrite a baseline only when decoded pixels differ beyond the floor, copying CI bytes verbatim (no re-encode churn); pulls a CI run via `gh run download` or a local --from dir. - post-discord.ts: always-on CI-on-main report (SHA + e2e status + the track-only runtime-perf table), then screenshot triptychs, batched + size-capped + flood-collapsed + 429-aware. - perf-report.ts: perf table with Δ vs the previous main run (via gh). - changelog.ts: no-build git-history baseline differ (Discord trigger B). - noise.ts / gen-manifest.ts: calibration + manifest generation. CI wiring: - wasm-build.yml: post-test step runs the gate + report on the already- produced test-results (no extra build); report-only (continue-on-error), posts only on push to main, inert without DISCORD_WEBHOOK_URL. - ci-ubicloud.yml: secrets: inherit (pass the webhook through). - screenshot-changelog.yml: ~30s no-build changelog on baseline changes. screenshot-manifest.json: canonical 354-name set + best-effort engine tags (313 chromium-swiftshader / 41 firefox-llvmpipe). Normalize scale:'device'->'css' across 18 spec files (no-op at CI DSF=1) so committed baselines are uniformly css-scaled. Design: CI's Linux render is the single source of truth; no pinned container (accept rare env drift -> re-promote); dev commits via promote. Replaces the byte-cmp compare-screenshots.sh + file-size-proxy update-baseline-screenshots.sh (kept for now until the first re-baseline). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
05127af431
commit
13551f4f22
36 changed files with 3186 additions and 63 deletions
|
|
@ -223,14 +223,14 @@ test.describe('Eeschema crosshair modes', () => {
|
|||
|
||||
await page.mouse.move(probe.x, probe.y);
|
||||
await page.waitForTimeout(600);
|
||||
const shotSmall = await page.screenshot({ path: 'test-results/eeschema-crosshair-00-small.png', scale: 'device' });
|
||||
const shotSmall = await page.screenshot({ path: 'test-results/eeschema-crosshair-00-small.png', scale: 'css' });
|
||||
|
||||
// click 1 -> full-window
|
||||
await clickAndSettle();
|
||||
await expect.poll(tooltipNow, {
|
||||
message: 'one click should advance to Full-Window Crosshairs', timeout: 6000,
|
||||
}).toContain('Full-Window Crosshairs');
|
||||
const shotFull = await page.screenshot({ path: 'test-results/eeschema-crosshair-01-full.png', scale: 'device' });
|
||||
const shotFull = await page.screenshot({ path: 'test-results/eeschema-crosshair-01-full.png', scale: 'css' });
|
||||
expect((await compareScreenshots(page, shotSmall, shotFull, diffRegion)).diffPixels,
|
||||
'full-window crosshair should visibly differ from the small crosshair').toBeGreaterThan(200);
|
||||
|
||||
|
|
@ -239,7 +239,7 @@ test.describe('Eeschema crosshair modes', () => {
|
|||
await expect.poll(tooltipNow, {
|
||||
message: 'second click should advance to 45 Degree Crosshairs', timeout: 6000,
|
||||
}).toContain('45 Degree Crosshairs');
|
||||
const shot45 = await page.screenshot({ path: 'test-results/eeschema-crosshair-02-45.png', scale: 'device' });
|
||||
const shot45 = await page.screenshot({ path: 'test-results/eeschema-crosshair-02-45.png', scale: 'css' });
|
||||
expect((await compareScreenshots(page, shotFull, shot45, diffRegion)).diffPixels,
|
||||
'45-degree crosshair should visibly differ from the full-window crosshair').toBeGreaterThan(200);
|
||||
|
||||
|
|
@ -248,7 +248,7 @@ test.describe('Eeschema crosshair modes', () => {
|
|||
await expect.poll(tooltipNow, {
|
||||
message: 'third click should cycle back to Small crosshairs', timeout: 6000,
|
||||
}).toContain('Small crosshairs');
|
||||
await page.screenshot({ path: 'test-results/eeschema-crosshair-03-small-again.png', scale: 'device' });
|
||||
await page.screenshot({ path: 'test-results/eeschema-crosshair-03-small-again.png', scale: 'css' });
|
||||
|
||||
const realErrors = testLogger.errors.filter((error: string) => !error.includes('favicon'));
|
||||
expect(realErrors).toEqual([]);
|
||||
|
|
|
|||
Loading…
Reference in a new issue