tests: harden the R2 baseline pipeline's failure paths (review findings)

- compare gate: refuse an INCOMPLETE cache, not just an empty one — a partial
  R2 fetch (continue-on-error step) previously misreported un-fetched
  baselines as ADDED and silently disabled their removed-detection.
- post-discord: a missing report.json now posts "gate SKIPPED", never the
  false " no screenshot drift" — a disabled pipeline must look disabled.
- promote: hard-fail on a pre-migration (non-v2) manifest instead of warning;
  the warn path could commit manifest hashes never uploaded to R2.
- changelog: tolerate per-image R2 failures (skip + loud note, like the old
  git-blob null-skip) instead of aborting the whole post; missing creds warn
  + exit 0 (notification-only workflow); git-show failures are loud errors,
  no longer mistaken for "predates the migration".
- version guard: a manifest NEWER than the tooling throws everywhere instead
  of reading as a silent no-op.
- manifest ordering: locale-independent code-unit comparator in writer +
  checker (localeCompare depends on host locale; writer=dev Mac, checker=CI).
- r2-sync pull: byte-size pre-filter before hashing cached files.
- wasm-build.yml: restore-keys on the baseline cache — manifest changes now
  restore the previous tree and download only the delta.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Istvan Matejcsok 2026-08-19 08:51:44 +02:00
commit 453df4fb9c
7 changed files with 105 additions and 34 deletions

View file

@ -313,14 +313,19 @@ jobs:
# Baseline screenshots live in a private R2 bucket, pinned by the committed
# manifest; this cache keyed on the manifest hash makes most fetches a
# no-op (r2-sync hash-verifies every restored file, so a stale restore
# costs only the changed downloads).
# no-op. restore-keys makes a manifest change INCREMENTAL (restore the
# previous tree, download only the changed objects) instead of a full
# ~35MB re-download. A stale or partial restore is safe: r2-sync
# hash-verifies every file and deletes unlisted ones, and compare.ts
# refuses to gate against an incomplete cache.
- name: Cache screenshot baselines
if: inputs.run_tests
uses: actions/cache@v4
with:
path: tests/baseline-screenshots
key: baselines-${{ hashFiles('tests/screenshot-manifest.json') }}
restore-keys: |
baselines-
# continue-on-error: the screenshot pipeline is deliberately report-only
# (see the report step) — an R2 outage must not fail the build. Without