tests: move screenshot baselines from git to R2 (content-addressed manifest v2)

Baseline PNGs (492, 35 MB) leave git: they now live in the private R2 bucket
pcbjam-ci-screenshots as immutable sha256/<hex>.png objects, pinned by the
committed screenshot-manifest.json (v2: name, engine, sha256, bytes, dims).
tests/baseline-screenshots/ becomes a gitignored cache materialized by the new
`npm run screenshots:fetch` (r2-sync.ts + aws4fetch r2-store.ts).

- promote.ts: requires the RW keypair (shell env or gitignored tests/.env),
  syncs the cache, uploads new hashes BEFORE rewriting the manifest; the
  manifest diff is the only git-visible output. --prune only edits the
  manifest — R2 objects are never deleted, old commits still resolve.
- compare.ts: skips the gate (exit 0, no report.json) when the manifest
  expects baselines but the cache is empty (secretless callers).
- changelog.ts: diffs the manifest between revs and fetches bytes from R2;
  guards against the migration commit (base manifest not v2 → skip).
- gen-manifest --check: v2 schema + resurrection guard (fails if baseline
  PNGs are ever re-committed); credential-free so every caller can gate.
- wasm-build.yml: declares optional read-only S3 secrets, caches + fetches
  baselines before the lint gate; release.yml/deploy-staging.yml pass them.
- screenshot-changelog.yml: triggers on the manifest path instead of PNGs.

All 492 objects are seeded and hash-verified in the bucket; fetch/compare
degrade to a warn-and-skip without credentials.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Istvan Matejcsok 2026-08-18 16:01:46 +02:00
commit 6dfa6ebd15
511 changed files with 3214 additions and 617 deletions

View file

@ -28,6 +28,11 @@ jobs:
build_3d_viewer: "ON"
run_tests: true
upload_output: true
secrets:
# Read-only R2 pair so the screenshot drift report runs on staging builds
# too (baselines live in R2 now; without these the fetch/compare steps skip).
CI_SCREENSHOTS_S3_ACCESS_KEY_ID: ${{ secrets.CI_SCREENSHOTS_S3_ACCESS_KEY_ID }}
CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY: ${{ secrets.CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY }}
publish-and-deploy:
needs: build

View file

@ -77,6 +77,11 @@ jobs:
build_3d_viewer: "ON"
run_tests: true
upload_output: true
secrets:
# Read-only R2 pair so the screenshot drift report runs on tag builds too
# (baselines live in R2 now; without these the fetch/compare steps skip).
CI_SCREENSHOTS_S3_ACCESS_KEY_ID: ${{ secrets.CI_SCREENSHOTS_S3_ACCESS_KEY_ID }}
CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY: ${{ secrets.CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY }}
# 2) Publish the build to the CDN (content-addressed; unchanged tools reuse)
# and write manifest-<tag>.json. The slow step is brotli-q11 over ~300MB of

View file

@ -1,17 +1,19 @@
name: screenshot-changelog
# Discord trigger B: when a push to main changes committed baseline PNGs, post an
# old | new+boxes | heatmap triptych per changed baseline (ADDED image / REMOVED
# title too). No build, no GPU — it only diffs two git revisions of PNGs that are
# already CI-rendered, so it runs in ~30s. Complements the re-render drift gate in
# wasm-build.yml (which catches un-blessed renders); this is the human-facing feed
# of intentional baseline updates as they land.
# Discord trigger B: when a push to main changes the screenshot manifest (the
# committed pin of every R2-stored baseline), post an old | new+boxes | heatmap
# triptych per changed baseline (ADDED image / REMOVED title too). No build, no
# GPU — it diffs two git revisions of the manifest and fetches the PNG bytes
# from the R2 CAS bucket (immutable objects, so the old rev's hashes always
# resolve), so it runs in ~30s. Complements the re-render drift gate in
# wasm-build.yml (which catches un-blessed renders); this is the human-facing
# feed of intentional baseline updates as they land.
on:
push:
branches: [main]
paths:
- 'tests/baseline-screenshots/**.png'
- 'tests/screenshot-manifest.json'
concurrency:
group: screenshot-changelog-${{ github.ref }}
@ -23,7 +25,7 @@ jobs:
steps:
- uses: actions/checkout@v4
with:
# HEAD^ is needed to diff the tip commit's baseline changes. (A push of
# HEAD^ is needed to diff the tip commit's manifest changes. (A push of
# multiple commits reports only the tip commit's baseline diff — fine for
# a changelog; baseline refreshes are single commits in practice.)
fetch-depth: 2
@ -37,4 +39,7 @@ jobs:
working-directory: tests
env:
DISCORD_WEBHOOK_URL: ${{ secrets.DISCORD_WEBHOOK_URL }}
CI_SCREENSHOTS_S3_ENDPOINT: ${{ vars.CI_SCREENSHOTS_S3_ENDPOINT }}
CI_SCREENSHOTS_S3_ACCESS_KEY_ID: ${{ secrets.CI_SCREENSHOTS_S3_ACCESS_KEY_ID }}
CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY: ${{ secrets.CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY }}
run: npm run screenshots:changelog

View file

@ -43,6 +43,13 @@ on:
# screenshot/perf report step just no-ops there.
DISCORD_WEBHOOK_URL:
required: false
# Read-only S3 keypair for the private R2 screenshot-baseline bucket
# (pcbjam-ci-screenshots). Optional for the same reason: without them the
# baseline fetch skips and the compare gate no-ops (report-only pipeline).
CI_SCREENSHOTS_S3_ACCESS_KEY_ID:
required: false
CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY:
required: false
jobs:
build-and-test:
@ -304,10 +311,36 @@ jobs:
working-directory: tests
run: npm ci
# Baseline screenshots live in a private R2 bucket, pinned by the committed
# manifest; this cache keyed on the manifest hash makes most fetches a
# no-op (r2-sync hash-verifies every restored file, so a stale restore
# costs only the changed downloads).
- name: Cache screenshot baselines
if: inputs.run_tests
uses: actions/cache@v4
with:
path: tests/baseline-screenshots
key: baselines-${{ hashFiles('tests/screenshot-manifest.json') }}
# continue-on-error: the screenshot pipeline is deliberately report-only
# (see the report step) — an R2 outage must not fail the build. Without
# credentials (release.yml, fork PRs) r2-sync skips with a warning and
# compare.ts later no-ops for the same reason.
- name: Fetch screenshot baselines from R2
if: inputs.run_tests
continue-on-error: true
working-directory: tests
env:
CI_SCREENSHOTS_S3_ENDPOINT: ${{ vars.CI_SCREENSHOTS_S3_ENDPOINT }}
CI_SCREENSHOTS_S3_ACCESS_KEY_ID: ${{ secrets.CI_SCREENSHOTS_S3_ACCESS_KEY_ID }}
CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY: ${{ secrets.CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY }}
run: npm run screenshots:fetch
# Cheap hygiene gates (no build needed): the determinism lint keeps the
# banned flake patterns out of the specs, the manifest check keeps
# screenshot-manifest.json in lockstep with the committed baselines
# (a stale manifest silently disables removed-screenshot detection), and
# banned flake patterns out of the specs, the manifest check validates
# screenshot-manifest.json (v2 schema, sorted/unique hashes) and fails if
# baseline PNGs are ever re-committed to git (credential-free — it never
# touches R2, so it gates secretless callers identically), and
# the CI-coverage lint proves every spec file on disk is reachable from
# the npm scripts THIS workflow invokes (a spec/project that CI never
# runs is how the web suite once rotted unnoticed).