tests: harden the R2 baseline pipeline's failure paths (review findings)
- compare gate: refuse an INCOMPLETE cache, not just an empty one — a partial
R2 fetch (continue-on-error step) previously misreported un-fetched
baselines as ADDED and silently disabled their removed-detection.
- post-discord: a missing report.json now posts "gate SKIPPED", never the
false "✅ no screenshot drift" — a disabled pipeline must look disabled.
- promote: hard-fail on a pre-migration (non-v2) manifest instead of warning;
the warn path could commit manifest hashes never uploaded to R2.
- changelog: tolerate per-image R2 failures (skip + loud note, like the old
git-blob null-skip) instead of aborting the whole post; missing creds warn
+ exit 0 (notification-only workflow); git-show failures are loud errors,
no longer mistaken for "predates the migration".
- version guard: a manifest NEWER than the tooling throws everywhere instead
of reading as a silent no-op.
- manifest ordering: locale-independent code-unit comparator in writer +
checker (localeCompare depends on host locale; writer=dev Mac, checker=CI).
- r2-sync pull: byte-size pre-filter before hashing cached files.
- wasm-build.yml: restore-keys on the baseline cache — manifest changes now
restore the previous tree and download only the delta.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
3257ddc966
commit
453df4fb9c
7 changed files with 105 additions and 34 deletions
|
|
@ -229,13 +229,18 @@ function main(): void {
|
|||
}
|
||||
const root = process.cwd();
|
||||
// Baselines are a local cache fetched from R2 (`npm run screenshots:fetch`).
|
||||
// When the manifest expects screenshots but the cache is empty — a secretless
|
||||
// CI caller where the fetch step skipped, or a dev who hasn't fetched — skip
|
||||
// the gate instead of misreporting every baseline as removed. No report.json
|
||||
// is written; post-discord tolerates its absence.
|
||||
const expected = loadManifest(root)?.screenshots.length ?? 0;
|
||||
if (expected > 0 && listEngineKeys(path.join(root, BASELINE_ROOT)).length === 0) {
|
||||
console.log('[compare] baseline cache is empty — run `npm run screenshots:fetch` (needs R2 credentials); skipping');
|
||||
// The gate only runs against a COMPLETE cache: a partial one (an R2 blip on
|
||||
// a few objects — the CI fetch step is continue-on-error) would misreport
|
||||
// the un-fetched baselines' renders as ADDED and silently disable removed-
|
||||
// detection for them. Fully missing = fetch skipped (no credentials) or a
|
||||
// dev who hasn't fetched. Either way skip without writing report.json —
|
||||
// post-discord posts a distinct "gate SKIPPED" line when it's absent.
|
||||
const gateManifest = loadManifest(root);
|
||||
const wanted = (gateManifest?.screenshots ?? []).filter((e) => !isIgnored(`${e.engine}/${e.name}`));
|
||||
const missing = wanted.filter((e) => !fs.existsSync(path.join(root, BASELINE_ROOT, e.engine, e.name)));
|
||||
if (wanted.length && missing.length) {
|
||||
const what = missing.length === wanted.length ? 'empty' : `INCOMPLETE (${missing.length}/${wanted.length} missing — partial fetch?)`;
|
||||
console.log(`[compare] baseline cache is ${what} — run \`npm run screenshots:fetch\` (needs R2 credentials); skipping`);
|
||||
return;
|
||||
}
|
||||
const report = classify(root, (args.sha as string) || process.env.GITHUB_SHA || null);
|
||||
|
|
|
|||
Loading…
Reference in a new issue