tests: harden the R2 baseline pipeline's failure paths (review findings)

- compare gate: refuse an INCOMPLETE cache, not just an empty one — a partial
  R2 fetch (continue-on-error step) previously misreported un-fetched
  baselines as ADDED and silently disabled their removed-detection.
- post-discord: a missing report.json now posts "gate SKIPPED", never the
  false "✅ no screenshot drift" — a disabled pipeline must look disabled.
- promote: hard-fail on a pre-migration (non-v2) manifest instead of warning;
  the warn path could commit manifest hashes never uploaded to R2.
- changelog: tolerate per-image R2 failures (skip + loud note, like the old
  git-blob null-skip) instead of aborting the whole post; missing creds warn
  + exit 0 (notification-only workflow); git-show failures are loud errors,
  no longer mistaken for "predates the migration".
- version guard: a manifest NEWER than the tooling throws everywhere instead
  of reading as a silent no-op.
- manifest ordering: locale-independent code-unit comparator in writer +
  checker (localeCompare depends on host locale; writer=dev Mac, checker=CI).
- r2-sync pull: byte-size pre-filter before hashing cached files.
- wasm-build.yml: restore-keys on the baseline cache — manifest changes now
  restore the previous tree and download only the delta.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Istvan Matejcsok 2026-08-19 08:51:44 +02:00
commit 453df4fb9c
7 changed files with 105 additions and 34 deletions

View file

@ -134,7 +134,12 @@ function buildHeader(report: Report | null, perfBlock: string, meta: { sha?: str
};
lines.push('');
if (!report || (!report.changed.length && !report.added.length && !report.removed.length)) {
if (!report) {
// No report.json = the compare gate never ran (baseline fetch skipped or
// incomplete). Saying "no drift" here would make a disabled pipeline
// indistinguishable from a healthy one.
lines.push('⚠️ screenshot gate SKIPPED — no baseline comparison ran (R2 fetch skipped/failed; check CI_SCREENSHOTS_S3_* secrets and the fetch step log)');
} else if (!report.changed.length && !report.added.length && !report.removed.length) {
lines.push('✅ no screenshot drift');
} else if (report.driftLikely) {
lines.push(`⚠️ **${report.changed.length} screenshots changed broadly** — looks like host env drift → re-promote (\`npm run screenshots:promote\`)`);