screenshots: R2-hosted manifest becomes the baseline source of truth (morelli cutover)

The committed tests/screenshot-manifest.json is retired. CI now downloads
baselines/pcbjam/manifest.json (manifest v3, written only by the morelli
review app + its seed script) to the gitignored .baseline-manifest.json,
and everything downstream (pull, verify, compare) reads that copy:

- config.ts: MANIFEST_VERSION 3, MANIFEST_PATH .baseline-manifest.json,
  R2_BASELINES_MANIFEST_KEY; ManifestEntry grows opaque provenance
- r2-sync.ts: new --manifest mode (atomic fetch; no-creds skip DELETES a
  stale copy so the gate skips rather than using old baselines); --push
  gone (bytes enter the CAS only via morelli's promote)
- compare.ts: hard-skips when no manifest was fetched — a stale warm
  cache can never gate
- wasm-build.yml: fetch-manifest step before the baselines cache; cache
  key now hashes the fetched manifest; the gen-manifest --check lint gate
  goes with the committed manifest
- deleted: screenshot-manifest.json, promote.ts, changelog.ts,
  gen-manifest.ts, screenshot-changelog.yml, promote-screenshots skill
- docs (CLAUDE/README/TESTING/WHATWORKS/tools README): promote flow is
  now https://pcbjam-morelli-staging.pcbjam-staging.workers.dev

Validated locally against the real bucket: fetch-manifest (492), cold
pull 492 / warm pull cached=492, no-creds skip chain, compare gate skip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Istvan Matejcsok 2026-08-19 11:54:20 +02:00
commit ec7a1b0787
20 changed files with 239 additions and 4793 deletions

View file

@ -55,7 +55,7 @@ tests/
├── logs/ # Test logs (auto-generated)
├── test-results/{chromium,firefox}/ # Screenshots per engine (auto-generated)
├── baseline-screenshots/{chromium,firefox}/ # Reference screenshots per engine — gitignored cache of the R2 bucket (npm run screenshots:fetch)
├── screenshot-manifest.json # Committed pin of every baseline: {name, engine, sha256, …} (regenerated by screenshots:promote)
├── .baseline-manifest.json # Gitignored copy of the R2-hosted baseline manifest (npm run screenshots:fetch-manifest)
├── apps/ # Built WASM test applications
│ ├── minimal_test.html # Main test app
│ └── standalone/ # Individual component test apps
@ -83,13 +83,15 @@ Example log format:
## Screenshots
Tests capture raw PNGs to `test-results/<engine>/` (engine-scoped via
`stableShot`/`shotPath`). The offline gate compares them against the committed
per-engine baselines:
`stableShot`/`shotPath`). The offline gate compares them against the per-engine
baselines pinned by the R2-hosted manifest:
```bash
npm run screenshots:fetch-manifest && npm run screenshots:fetch
npm run screenshots:check
```
CI's Linux render is the source of truth — update baselines by promoting a CI
run (`npm run screenshots:promote -- --run <ci-run-id>`), never by copying
run in the morelli review app
(https://pcbjam-morelli-staging.pcbjam-staging.workers.dev), never by copying
local renders. Rules and details: `TESTING.md`.
## Viewing the App Directly