tests: move screenshot baselines from git to R2 (content-addressed manifest v2)

Baseline PNGs (492, 35 MB) leave git: they now live in the private R2 bucket
pcbjam-ci-screenshots as immutable sha256/<hex>.png objects, pinned by the
committed screenshot-manifest.json (v2: name, engine, sha256, bytes, dims).
tests/baseline-screenshots/ becomes a gitignored cache materialized by the new
`npm run screenshots:fetch` (r2-sync.ts + aws4fetch r2-store.ts).

- promote.ts: requires the RW keypair (shell env or gitignored tests/.env),
  syncs the cache, uploads new hashes BEFORE rewriting the manifest; the
  manifest diff is the only git-visible output. --prune only edits the
  manifest — R2 objects are never deleted, old commits still resolve.
- compare.ts: skips the gate (exit 0, no report.json) when the manifest
  expects baselines but the cache is empty (secretless callers).
- changelog.ts: diffs the manifest between revs and fetches bytes from R2;
  guards against the migration commit (base manifest not v2 → skip).
- gen-manifest --check: v2 schema + resurrection guard (fails if baseline
  PNGs are ever re-committed); credential-free so every caller can gate.
- wasm-build.yml: declares optional read-only S3 secrets, caches + fetches
  baselines before the lint gate; release.yml/deploy-staging.yml pass them.
- screenshot-changelog.yml: triggers on the manifest path instead of PNGs.

All 492 objects are seeded and hash-verified in the bucket; fetch/compare
degrade to a warn-and-skip without credentials.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

.github/workflows/deploy-staging.yml

@@ -28,6 +28,11 @@ jobs:
       build_3d_viewer: "ON"
       run_tests: true
       upload_output: true
+    secrets:
+      # Read-only R2 pair so the screenshot drift report runs on staging builds
+      # too (baselines live in R2 now; without these the fetch/compare steps skip).
+      CI_SCREENSHOTS_S3_ACCESS_KEY_ID: ${{ secrets.CI_SCREENSHOTS_S3_ACCESS_KEY_ID }}
+      CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY: ${{ secrets.CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY }}
 
   publish-and-deploy:
     needs: build

.github/workflows/release.yml

@@ -77,6 +77,11 @@ jobs:
       build_3d_viewer: "ON"
       run_tests: true
       upload_output: true
+    secrets:
+      # Read-only R2 pair so the screenshot drift report runs on tag builds too
+      # (baselines live in R2 now; without these the fetch/compare steps skip).
+      CI_SCREENSHOTS_S3_ACCESS_KEY_ID: ${{ secrets.CI_SCREENSHOTS_S3_ACCESS_KEY_ID }}
+      CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY: ${{ secrets.CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY }}
 
   # 2) Publish the build to the CDN (content-addressed; unchanged tools reuse)
   #    and write manifest-<tag>.json. The slow step is brotli-q11 over ~300MB of

.github/workflows/screenshot-changelog.yml

@@ -1,17 +1,19 @@
 name: screenshot-changelog
 
-# Discord trigger B: when a push to main changes committed baseline PNGs, post an
+# Discord trigger B: when a push to main changes the screenshot manifest (the
-# old | new+boxes | heatmap triptych per changed baseline (ADDED image / REMOVED
+# committed pin of every R2-stored baseline), post an old | new+boxes | heatmap
-# title too). No build, no GPU — it only diffs two git revisions of PNGs that are
+# triptych per changed baseline (ADDED image / REMOVED title too). No build, no
-# already CI-rendered, so it runs in ~30s. Complements the re-render drift gate in
+# GPU — it diffs two git revisions of the manifest and fetches the PNG bytes
-# wasm-build.yml (which catches un-blessed renders); this is the human-facing feed
+# from the R2 CAS bucket (immutable objects, so the old rev's hashes always
-# of intentional baseline updates as they land.
+# resolve), so it runs in ~30s. Complements the re-render drift gate in
+# wasm-build.yml (which catches un-blessed renders); this is the human-facing
+# feed of intentional baseline updates as they land.
 
 on:
   push:
     branches: [main]
     paths:
-      - 'tests/baseline-screenshots/**.png'
+      - 'tests/screenshot-manifest.json'
 
 concurrency:
   group: screenshot-changelog-${{ github.ref }}
@@ -23,7 +25,7 @@ jobs:
     steps:
       - uses: actions/checkout@v4
         with:
-          # HEAD^ is needed to diff the tip commit's baseline changes. (A push of
+          # HEAD^ is needed to diff the tip commit's manifest changes. (A push of
           # multiple commits reports only the tip commit's baseline diff — fine for
           # a changelog; baseline refreshes are single commits in practice.)
           fetch-depth: 2
@@ -37,4 +39,7 @@ jobs:
         working-directory: tests
         env:
           DISCORD_WEBHOOK_URL: ${{ secrets.DISCORD_WEBHOOK_URL }}
+          CI_SCREENSHOTS_S3_ENDPOINT: ${{ vars.CI_SCREENSHOTS_S3_ENDPOINT }}
+          CI_SCREENSHOTS_S3_ACCESS_KEY_ID: ${{ secrets.CI_SCREENSHOTS_S3_ACCESS_KEY_ID }}
+          CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY: ${{ secrets.CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY }}
         run: npm run screenshots:changelog

.github/workflows/wasm-build.yml

@@ -43,6 +43,13 @@ on:
       # screenshot/perf report step just no-ops there.
       DISCORD_WEBHOOK_URL:
         required: false
+      # Read-only S3 keypair for the private R2 screenshot-baseline bucket
+      # (pcbjam-ci-screenshots). Optional for the same reason: without them the
+      # baseline fetch skips and the compare gate no-ops (report-only pipeline).
+      CI_SCREENSHOTS_S3_ACCESS_KEY_ID:
+        required: false
+      CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY:
+        required: false
 
 jobs:
   build-and-test:
@@ -304,10 +311,36 @@ jobs:
         working-directory: tests
         run: npm ci
 
+      # Baseline screenshots live in a private R2 bucket, pinned by the committed
+      # manifest; this cache keyed on the manifest hash makes most fetches a
+      # no-op (r2-sync hash-verifies every restored file, so a stale restore
+      # costs only the changed downloads).
+      - name: Cache screenshot baselines
+        if: inputs.run_tests
+        uses: actions/cache@v4
+        with:
+          path: tests/baseline-screenshots
+          key: baselines-${{ hashFiles('tests/screenshot-manifest.json') }}
+
+      # continue-on-error: the screenshot pipeline is deliberately report-only
+      # (see the report step) — an R2 outage must not fail the build. Without
+      # credentials (release.yml, fork PRs) r2-sync skips with a warning and
+      # compare.ts later no-ops for the same reason.
+      - name: Fetch screenshot baselines from R2
+        if: inputs.run_tests
+        continue-on-error: true
+        working-directory: tests
+        env:
+          CI_SCREENSHOTS_S3_ENDPOINT: ${{ vars.CI_SCREENSHOTS_S3_ENDPOINT }}
+          CI_SCREENSHOTS_S3_ACCESS_KEY_ID: ${{ secrets.CI_SCREENSHOTS_S3_ACCESS_KEY_ID }}
+          CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY: ${{ secrets.CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY }}
+        run: npm run screenshots:fetch
+
       # Cheap hygiene gates (no build needed): the determinism lint keeps the
-      # banned flake patterns out of the specs, the manifest check keeps
+      # banned flake patterns out of the specs, the manifest check validates
-      # screenshot-manifest.json in lockstep with the committed baselines
+      # screenshot-manifest.json (v2 schema, sorted/unique hashes) and fails if
-      # (a stale manifest silently disables removed-screenshot detection), and
+      # baseline PNGs are ever re-committed to git (credential-free — it never
+      # touches R2, so it gates secretless callers identically), and
       # the CI-coverage lint proves every spec file on disk is reachable from
       # the npm scripts THIS workflow invokes (a spec/project that CI never
       # runs is how the web suite once rotted unnoticed).

.gitignore

@@ -40,6 +40,11 @@ wxwidgets-clean/
 /tests/node_modules/
 /tests/playwright-report/
 /tests/test-results/
+# Baseline screenshots are a local cache fetched from R2 (tests/tools/screenshots/r2-sync.ts);
+# only tests/screenshot-manifest.json is committed. The .env holds a dev's
+# R2 keypair (r2-store.ts auto-loads it) — never commit it.
+/tests/baseline-screenshots/
+/tests/.env
 /tests/bench-results/
 /tests/pw-artifacts/
 /tests/.test-port

CLAUDE.md

@@ -7,8 +7,8 @@ The e2e tests are in /tests, with a README and WHATWORKS md files
 Test determinism rules (no blind sleeps/ifs, `stableShot` screenshots, retries:0) are in tests/TESTING.md, enforced by `npm run lint:determinism`.
 The e2e tests are separated per feature
 Wxwidgets wasm port has hooks for finding positions of UI elements, tests use that
-The test screenshots are tracked with git, per engine (tests/baseline-screenshots/{chromium,firefox}/); CI's Linux render is the source of truth (tooling: tests/tools/screenshots/, see its README).
+The test screenshot baselines live in a private R2 bucket (content-addressed by sha256), pinned per engine by the committed tests/screenshot-manifest.json; tests/baseline-screenshots/{chromium,firefox}/ is a gitignored local cache — `cd tests && npm run screenshots:fetch` materializes it (needs the R2 credentials in tests/tools/screenshots/README.md). CI's Linux render is the source of truth (tooling: tests/tools/screenshots/, see its README).
-To update baselines, promote a CI run's render (churn-free — only meaningfully-changed images restage): `cd tests && npm run screenshots:promote -- --run <ci-run-id>`, then commit. `npm run screenshots:check` is the local gate; on each main push CI posts a screenshot-diff + runtime-perf report to Discord.
+To update baselines, promote a CI run's render (churn-free — only meaningfully-changed images re-upload): `cd tests && npm run screenshots:promote -- --run <ci-run-id>` (needs the read-write R2 credentials), then commit the manifest diff — never commit PNGs. `npm run screenshots:check` is the local gate (fetch first); on each main push CI posts a screenshot-diff + runtime-perf report to Discord.
 The tests have log files in tests/logs/{wxwidgets/kicad}/{test-name} after each run where the js console and cpp logs are visible
 Always check screenshots for validating tests
 Run e2e tests from /tests folder: `npm run test:e2e` (full CI project set, one merged playwright.config.ts) or `npm run test:kicad` (firefox shortcut) — not playwright directly. One spec/engine: `npx playwright test --project=kicad-firefox kicad/pcbnew.spec.ts`. Web-app suite: `npm run test:web`.

tests/.gitignore

@@ -1,6 +1,8 @@
 # Screenshot model: specs capture via stableShot() → page.screenshot into test-results/, which is
-# transient and gitignored at the repo root (/tests/test-results/). The committed baselines live in
+# transient and gitignored at the repo root (/tests/test-results/). Baselines live in a private R2
-# tests/baseline-screenshots/ (+ 3d-regression/, gal-regression/), authored from CI's deterministic
+# bucket, content-addressed by sha256 and pinned by the committed screenshot-manifest.json;
-# Linux render and diffed OFFLINE by tools/screenshots. There are no Playwright-native
+# tests/baseline-screenshots/ is a gitignored local cache (`npm run screenshots:fetch`), authored
-# *-snapshots/ baselines — Playwright does no inline screenshot comparison.
+# from CI's deterministic Linux render and diffed OFFLINE by tools/screenshots. (3d-regression/ and
+# gal-regression/ baselines are still committed.) There are no Playwright-native *-snapshots/
+# baselines — Playwright does no inline screenshot comparison.
 logs/