screenshots: R2-hosted manifest becomes the baseline source of truth (morelli cutover)
The committed tests/screenshot-manifest.json is retired. CI now downloads baselines/pcbjam/manifest.json (manifest v3, written only by the morelli review app + its seed script) to the gitignored .baseline-manifest.json, and everything downstream (pull, verify, compare) reads that copy: - config.ts: MANIFEST_VERSION 3, MANIFEST_PATH .baseline-manifest.json, R2_BASELINES_MANIFEST_KEY; ManifestEntry grows opaque provenance - r2-sync.ts: new --manifest mode (atomic fetch; no-creds skip DELETES a stale copy so the gate skips rather than using old baselines); --push gone (bytes enter the CAS only via morelli's promote) - compare.ts: hard-skips when no manifest was fetched — a stale warm cache can never gate - wasm-build.yml: fetch-manifest step before the baselines cache; cache key now hashes the fetched manifest; the gen-manifest --check lint gate goes with the committed manifest - deleted: screenshot-manifest.json, promote.ts, changelog.ts, gen-manifest.ts, screenshot-changelog.yml, promote-screenshots skill - docs (CLAUDE/README/TESTING/WHATWORKS/tools README): promote flow is now https://pcbjam-morelli-staging.pcbjam-staging.workers.dev Validated locally against the real bucket: fetch-manifest (492), cold pull 492 / warm pull cached=492, no-creds skip chain, compare gate skip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
4196958c10
commit
ec7a1b0787
20 changed files with 239 additions and 4793 deletions
|
|
@ -55,7 +55,7 @@ tests/
|
|||
├── logs/ # Test logs (auto-generated)
|
||||
├── test-results/{chromium,firefox}/ # Screenshots per engine (auto-generated)
|
||||
├── baseline-screenshots/{chromium,firefox}/ # Reference screenshots per engine — gitignored cache of the R2 bucket (npm run screenshots:fetch)
|
||||
├── screenshot-manifest.json # Committed pin of every baseline: {name, engine, sha256, …} (regenerated by screenshots:promote)
|
||||
├── .baseline-manifest.json # Gitignored copy of the R2-hosted baseline manifest (npm run screenshots:fetch-manifest)
|
||||
├── apps/ # Built WASM test applications
|
||||
│ ├── minimal_test.html # Main test app
|
||||
│ └── standalone/ # Individual component test apps
|
||||
|
|
@ -83,13 +83,15 @@ Example log format:
|
|||
## Screenshots
|
||||
|
||||
Tests capture raw PNGs to `test-results/<engine>/` (engine-scoped via
|
||||
`stableShot`/`shotPath`). The offline gate compares them against the committed
|
||||
per-engine baselines:
|
||||
`stableShot`/`shotPath`). The offline gate compares them against the per-engine
|
||||
baselines pinned by the R2-hosted manifest:
|
||||
```bash
|
||||
npm run screenshots:fetch-manifest && npm run screenshots:fetch
|
||||
npm run screenshots:check
|
||||
```
|
||||
CI's Linux render is the source of truth — update baselines by promoting a CI
|
||||
run (`npm run screenshots:promote -- --run <ci-run-id>`), never by copying
|
||||
run in the morelli review app
|
||||
(https://pcbjam-morelli-staging.pcbjam-staging.workers.dev), never by copying
|
||||
local renders. Rules and details: `TESTING.md`.
|
||||
|
||||
## Viewing the App Directly
|
||||
|
|
|
|||
Loading…
Reference in a new issue