fix(tests): enforce screenshot gate; drop shadow dir, flaky retinascale, 12 stale
Finalize the screenshot review system after the first Linux re-baseline (that run showed 355/356 stable — the placeholder floor is fine as-is): - ENFORCE the gate: `screenshots:check --fail-on-change` now fails the build on any changed/added/removed vs baselines (it still posts the drift report first, so the failure is actionable; a real render change → re-promote). The generic "CI failed" notice is suppressed for gate failures so we don't double-post on drift. - Exclude retinascale-01-loaded: a fullPage HiDPI test whose captured height + DPR scaling vary run-to-run (~60% inter-run diff) — a flaky test, not render noise. IGNORE_SCREENSHOTS in config.ts; compare/promote/gen-manifest skip it; baseline removed. - Kill the baseline-dir shadowing: drop e2e/baseline-screenshots/ from BASELINE_DIRS + delete its 3 files (grid-tab-final, wxgrid-controls, wxgrid-dedicated-page) that duplicated names in baseline-screenshots/ with different bytes. - Prune 12 stale baselines (renamed/removed specs: wizard-01..04, gerbview-wizard-01..04, zoom-pl_editor-*, popup-03-palette). - Delete the dead compare-screenshots.sh / update-baseline-screenshots.sh. - Regenerate screenshot-manifest.json (355 entries). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
348596c515
commit
3efcdcf91c
24 changed files with 98 additions and 251 deletions
|
|
@ -9,8 +9,12 @@
|
|||
* the npm scripts and CI steps run from).
|
||||
*/
|
||||
|
||||
/** Committed baseline directories, scanned in order. Filenames are the keys. */
|
||||
export const BASELINE_DIRS = ['baseline-screenshots', 'e2e/baseline-screenshots'] as const;
|
||||
/**
|
||||
* Committed baseline directory. Single dir on purpose: `e2e/baseline-screenshots/`
|
||||
* used to also hold a few names (grid-tab-final, wxgrid-*) that ALSO live here with
|
||||
* different bytes — so they got silently shadowed. Consolidated to one dir.
|
||||
*/
|
||||
export const BASELINE_DIRS = ['baseline-screenshots'] as const;
|
||||
|
||||
/** Where Playwright writes the current run's screenshots (gitignored). */
|
||||
export const RESULTS_DIR = 'test-results';
|
||||
|
|
@ -74,6 +78,20 @@ export const FLOORS: Record<string, EngineFloor> = {
|
|||
*/
|
||||
export const IGNORE_REGIONS: Record<string, Array<{ x: number; y: number; width: number; height: number }>> = {};
|
||||
|
||||
/**
|
||||
* Screenshots excluded from comparison entirely (not compared, not counted as
|
||||
* changed/added/removed, not put in the manifest). For nondeterministic captures
|
||||
* that can't be a stable baseline — e.g. `retinascale-01-loaded` is a `fullPage`
|
||||
* HiDPI test whose captured height + DPR scaling vary run-to-run (~60% inter-run
|
||||
* diff observed), a flaky test rather than render noise.
|
||||
*/
|
||||
export const IGNORE_SCREENSHOTS = new Set<string>(['retinascale-01-loaded.png']);
|
||||
|
||||
/** True if a screenshot is excluded from comparison. */
|
||||
export function isIgnored(name: string): boolean {
|
||||
return IGNORE_SCREENSHOTS.has(name);
|
||||
}
|
||||
|
||||
export type ManifestEntry = { name: string; engine: string };
|
||||
export type Manifest = { screenshots: ManifestEntry[] };
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue