tests: move screenshot baselines from git to R2 (content-addressed manifest v2)

Baseline PNGs (492, 35 MB) leave git: they now live in the private R2 bucket
pcbjam-ci-screenshots as immutable sha256/<hex>.png objects, pinned by the
committed screenshot-manifest.json (v2: name, engine, sha256, bytes, dims).
tests/baseline-screenshots/ becomes a gitignored cache materialized by the new
`npm run screenshots:fetch` (r2-sync.ts + aws4fetch r2-store.ts).

- promote.ts: requires the RW keypair (shell env or gitignored tests/.env),
  syncs the cache, uploads new hashes BEFORE rewriting the manifest; the
  manifest diff is the only git-visible output. --prune only edits the
  manifest — R2 objects are never deleted, old commits still resolve.
- compare.ts: skips the gate (exit 0, no report.json) when the manifest
  expects baselines but the cache is empty (secretless callers).
- changelog.ts: diffs the manifest between revs and fetches bytes from R2;
  guards against the migration commit (base manifest not v2 → skip).
- gen-manifest --check: v2 schema + resurrection guard (fails if baseline
  PNGs are ever re-committed); credential-free so every caller can gate.
- wasm-build.yml: declares optional read-only S3 secrets, caches + fetches
  baselines before the lint gate; release.yml/deploy-staging.yml pass them.
- screenshot-changelog.yml: triggers on the manifest path instead of PNGs.

All 492 objects are seeded and hash-verified in the bucket; fetch/compare
degrade to a warn-and-skip without credentials.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Istvan Matejcsok 2026-08-18 16:01:46 +02:00
commit 6dfa6ebd15
511 changed files with 3214 additions and 617 deletions

View file

@ -43,6 +43,13 @@ on:
# screenshot/perf report step just no-ops there.
DISCORD_WEBHOOK_URL:
required: false
# Read-only S3 keypair for the private R2 screenshot-baseline bucket
# (pcbjam-ci-screenshots). Optional for the same reason: without them the
# baseline fetch skips and the compare gate no-ops (report-only pipeline).
CI_SCREENSHOTS_S3_ACCESS_KEY_ID:
required: false
CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY:
required: false
jobs:
build-and-test:
@ -304,10 +311,36 @@ jobs:
working-directory: tests
run: npm ci
# Baseline screenshots live in a private R2 bucket, pinned by the committed
# manifest; this cache keyed on the manifest hash makes most fetches a
# no-op (r2-sync hash-verifies every restored file, so a stale restore
# costs only the changed downloads).
- name: Cache screenshot baselines
if: inputs.run_tests
uses: actions/cache@v4
with:
path: tests/baseline-screenshots
key: baselines-${{ hashFiles('tests/screenshot-manifest.json') }}
# continue-on-error: the screenshot pipeline is deliberately report-only
# (see the report step) — an R2 outage must not fail the build. Without
# credentials (release.yml, fork PRs) r2-sync skips with a warning and
# compare.ts later no-ops for the same reason.
- name: Fetch screenshot baselines from R2
if: inputs.run_tests
continue-on-error: true
working-directory: tests
env:
CI_SCREENSHOTS_S3_ENDPOINT: ${{ vars.CI_SCREENSHOTS_S3_ENDPOINT }}
CI_SCREENSHOTS_S3_ACCESS_KEY_ID: ${{ secrets.CI_SCREENSHOTS_S3_ACCESS_KEY_ID }}
CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY: ${{ secrets.CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY }}
run: npm run screenshots:fetch
# Cheap hygiene gates (no build needed): the determinism lint keeps the
# banned flake patterns out of the specs, the manifest check keeps
# screenshot-manifest.json in lockstep with the committed baselines
# (a stale manifest silently disables removed-screenshot detection), and
# banned flake patterns out of the specs, the manifest check validates
# screenshot-manifest.json (v2 schema, sorted/unique hashes) and fails if
# baseline PNGs are ever re-committed to git (credential-free — it never
# touches R2, so it gates secretless callers identically), and
# the CI-coverage lint proves every spec file on disk is reachable from
# the npm scripts THIS workflow invokes (a spec/project that CI never
# runs is how the web suite once rotted unnoticed).