fix(tests): keep test-results across CI suites; loud pipeline alert

The screenshot report went silent on green CI runs. CI runs four sequential
`playwright test` invocations (wx, asyncify, kicad, perf) that all default their
outputDir to `test-results`; Playwright clears the whole outputDir at the start
of every run, so each suite wiped the previous suite's screenshots. By the time
`compare` ran, only perf-*.json remained -> all 354 baselines read as "removed"
-> the no-render guard stayed silent.

- Gate Playwright's outputDir off test-results/ in CI only
  (process.env.CI ? 'pw-artifacts/<suite>' : 'test-results') in the wx / asyncify
  / kicad configs, so the committed-baseline screenshots (page.screenshot to
  'test-results/...') accumulate in test-results/ across the suites for compare.
  Local single-suite behavior is unchanged.
- post-discord: replace the silent no-render guard with a loud, image-less
  pipeline alert ("No screenshots produced this run") when 0 render; genuine
  partial removals still post as the normal capped REMOVED list.
- gitignore pw-artifacts/; upload tests/pw-artifacts/** as a CI artifact.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Viktor Vaczi 2026-07-02 09:20:35 +02:00
commit b270507505
6 changed files with 31 additions and 4 deletions

View file

@ -67,6 +67,11 @@ const appsDir = 'apps';
export default defineConfig({
globalSetup: './global-setup.ts',
testDir: './e2e',
// In CI, redirect Playwright's start-of-run outputDir cleanup to a throwaway dir so it
// never wipes test-results/ — the committed-baseline screenshots (page.screenshot to
// 'test-results/…') must survive across the sequential wx/asyncify/kicad/perf runs for the
// screenshot compare. Local dev keeps the default (test-results cleaned each single run).
outputDir: process.env.CI ? 'pw-artifacts/wx' : 'test-results',
fullyParallel: true,
forbidOnly: !!process.env.CI,
// 1 local retry absorbs transient `npx serve` connection refusals under heavy