fix(tests): enforce screenshot gate; drop shadow dir, flaky retinascale, 12 stale

Finalize the screenshot review system after the first Linux re-baseline
(that run showed 355/356 stable — the placeholder floor is fine as-is):

- ENFORCE the gate: `screenshots:check --fail-on-change` now fails the build on any
  changed/added/removed vs baselines (it still posts the drift report first, so the
  failure is actionable; a real render change → re-promote). The generic "CI failed"
  notice is suppressed for gate failures so we don't double-post on drift.
- Exclude retinascale-01-loaded: a fullPage HiDPI test whose captured height + DPR
  scaling vary run-to-run (~60% inter-run diff) — a flaky test, not render noise.
  IGNORE_SCREENSHOTS in config.ts; compare/promote/gen-manifest skip it; baseline removed.
- Kill the baseline-dir shadowing: drop e2e/baseline-screenshots/ from BASELINE_DIRS +
  delete its 3 files (grid-tab-final, wxgrid-controls, wxgrid-dedicated-page) that
  duplicated names in baseline-screenshots/ with different bytes.
- Prune 12 stale baselines (renamed/removed specs: wizard-01..04, gerbview-wizard-01..04,
  zoom-pl_editor-*, popup-03-palette).
- Delete the dead compare-screenshots.sh / update-baseline-screenshots.sh.
- Regenerate screenshot-manifest.json (355 entries).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Viktor Vaczi 2026-07-02 11:30:33 +02:00
commit 3efcdcf91c
24 changed files with 98 additions and 251 deletions

View file

@ -336,22 +336,30 @@ jobs:
# seeded by `npm run screenshots:noise`). Posts ONLY on push to main and no-ops
# without DISCORD_WEBHOOK_URL (inert on PRs/forks). No extra build — reads the
# already-produced test-results (screenshots + perf-*.json).
- name: Screenshot + perf report (on success)
- name: Screenshot gate + perf report (on success)
id: screenshot_report
if: success() && inputs.run_tests
continue-on-error: true
working-directory: tests
env:
DISCORD_WEBHOOK_URL: ${{ secrets.DISCORD_WEBHOOK_URL }}
GH_TOKEN: ${{ github.token }}
# ENFORCING gate: `screenshots:check --fail-on-change` writes report.json and exits
# nonzero on any changed/added/removed vs the committed baselines. We still post the
# report (drift triptychs + perf) so the failure is actionable, then exit with the
# gate's status so a drift fails the build. (A re-render change → re-promote baselines.)
run: |
npm run screenshots:check
set +e
npm run screenshots:check -- --fail-on-change; GATE=$?
npm run screenshots:report -- --e2e pass
exit $GATE
# ON FAILURE (build or e2e): a minimal text-only "CI failed" notice, nothing
# else (no images / no comparison). Uses curl, NOT the TS reporter, because on a
# build failure the test deps (npm ci) never installed. Main-push only.
- name: Discord CI-failure notice
if: failure() && github.ref == 'refs/heads/main' && github.event_name == 'push'
# Build/e2e failure only — NOT a screenshot-gate failure (that already posted the
# drift report), so we don't double-post "CI failed" on drift.
if: failure() && steps.screenshot_report.outcome != 'failure' && github.ref == 'refs/heads/main' && github.event_name == 'push'
continue-on-error: true
env:
DISCORD_WEBHOOK_URL: ${{ secrets.DISCORD_WEBHOOK_URL }}