AI_Assistant/pcbjam
Watch
1
0
Fork
You've already forked pcbjam
 0
Code Issues Pull requests Projects Releases 7 Packages Wiki Activity Actions

screenshots: R2-hosted manifest becomes the baseline source of truth (morelli cutover)

Browse source 
The committed tests/screenshot-manifest.json is retired. CI now downloads
baselines/pcbjam/manifest.json (manifest v3, written only by the morelli
review app + its seed script) to the gitignored .baseline-manifest.json,
and everything downstream (pull, verify, compare) reads that copy:

- config.ts: MANIFEST_VERSION 3, MANIFEST_PATH .baseline-manifest.json,
  R2_BASELINES_MANIFEST_KEY; ManifestEntry grows opaque provenance
- r2-sync.ts: new --manifest mode (atomic fetch; no-creds skip DELETES a
  stale copy so the gate skips rather than using old baselines); --push
  gone (bytes enter the CAS only via morelli's promote)
- compare.ts: hard-skips when no manifest was fetched — a stale warm
  cache can never gate
- wasm-build.yml: fetch-manifest step before the baselines cache; cache
  key now hashes the fetched manifest; the gen-manifest --check lint gate
  goes with the committed manifest
- deleted: screenshot-manifest.json, promote.ts, changelog.ts,
  gen-manifest.ts, screenshot-changelog.yml, promote-screenshots skill
- docs (CLAUDE/README/TESTING/WHATWORKS/tools README): promote flow is
  now https://pcbjam-morelli-staging.pcbjam-staging.workers.dev

Validated locally against the real bucket: fetch-manifest (492), cold
pull 492 / warm pull cached=492, no-creds skip chain, compare gate skip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Istvan Matejcsok 2026-08-19 11:54:20 +02:00
commit ec7a1b0787
20 changed files with 239 additions and 4793 deletions
Download patch file Download diff file Expand all files Collapse all files

21
tests/TESTING.md
View file

@ -46,15 +46,18 @@ only) — not playwright directly. One spec on one engine:
`tests/baseline-screenshots/<engine>/` (+ the still-committed `3d-regression/`,
`gal-regression/`).
- **Baselines live in the private R2 bucket `pcbjam-ci-screenshots`, not git**:
the committed `tests/screenshot-manifest.json` pins each `{name, engine}` to a
sha256, and `baseline-screenshots/` is a gitignored cache — materialize it with
`npm run screenshots:fetch` (needs the R2 credentials; see
`tools/screenshots/README.md`). CI's lint step validates the manifest and fails
if baseline PNGs are ever committed again.
- **CI's Linux render is the source of truth**; baselines are promoted from CI
(`npm run screenshots:promote -- --run <ci-run-id>`, needs the read-write
credentials) — commit only the regenerated manifest diff, never PNGs. A local
(Mac) check shows font/render noise and is not the gate.
the R2-hosted manifest `baselines/pcbjam/manifest.json` pins each
`{name, engine}` to a sha256, and `baseline-screenshots/` +
`.baseline-manifest.json` are gitignored caches — materialize them with
`npm run screenshots:fetch-manifest && npm run screenshots:fetch` (needs the
R2 credentials; see `tools/screenshots/README.md`). Nothing
screenshot-related is committed to git.
- **CI's Linux render is the source of truth**; baselines are promoted from a
CI run in the morelli review app
(https://pcbjam-morelli-staging.pcbjam-staging.workers.dev) — CI uploads each
run's renders to R2 (30-day retention), morelli shows the diffs and writes
the manifest on Promote. A local (Mac) check shows font/render noise and is
not the gate.
- A continuously-animating state (timer, mid-slide) can't be a stable baseline — drop the shot.
## Retries