screenshots: R2-hosted manifest becomes the baseline source of truth (morelli cutover)

The committed tests/screenshot-manifest.json is retired. CI now downloads
baselines/pcbjam/manifest.json (manifest v3, written only by the morelli
review app + its seed script) to the gitignored .baseline-manifest.json,
and everything downstream (pull, verify, compare) reads that copy:

- config.ts: MANIFEST_VERSION 3, MANIFEST_PATH .baseline-manifest.json,
  R2_BASELINES_MANIFEST_KEY; ManifestEntry grows opaque provenance
- r2-sync.ts: new --manifest mode (atomic fetch; no-creds skip DELETES a
  stale copy so the gate skips rather than using old baselines); --push
  gone (bytes enter the CAS only via morelli's promote)
- compare.ts: hard-skips when no manifest was fetched — a stale warm
  cache can never gate
- wasm-build.yml: fetch-manifest step before the baselines cache; cache
  key now hashes the fetched manifest; the gen-manifest --check lint gate
  goes with the committed manifest
- deleted: screenshot-manifest.json, promote.ts, changelog.ts,
  gen-manifest.ts, screenshot-changelog.yml, promote-screenshots skill
- docs (CLAUDE/README/TESTING/WHATWORKS/tools README): promote flow is
  now https://pcbjam-morelli-staging.pcbjam-staging.workers.dev

Validated locally against the real bucket: fetch-manifest (492), cold
pull 492 / warm pull cached=492, no-creds skip chain, compare gate skip.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Istvan Matejcsok 2026-08-19 11:54:20 +02:00
commit ec7a1b0787
20 changed files with 239 additions and 4793 deletions

View file

@ -1,45 +0,0 @@
name: screenshot-changelog
# Discord trigger B: when a push to main changes the screenshot manifest (the
# committed pin of every R2-stored baseline), post an old | new+boxes | heatmap
# triptych per changed baseline (ADDED image / REMOVED title too). No build, no
# GPU — it diffs two git revisions of the manifest and fetches the PNG bytes
# from the R2 CAS bucket (immutable objects, so the old rev's hashes always
# resolve), so it runs in ~30s. Complements the re-render drift gate in
# wasm-build.yml (which catches un-blessed renders); this is the human-facing
# feed of intentional baseline updates as they land.
on:
push:
branches: [main]
paths:
- 'tests/screenshot-manifest.json'
concurrency:
group: screenshot-changelog-${{ github.ref }}
cancel-in-progress: false
jobs:
changelog:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
# HEAD^ is needed to diff the tip commit's manifest changes. (A push of
# multiple commits reports only the tip commit's baseline diff — fine for
# a changelog; baseline refreshes are single commits in practice.)
fetch-depth: 2
- uses: actions/setup-node@v4
with:
node-version: 20
- name: Install screenshot tooling deps
working-directory: tests
run: npm ci
- name: Post baseline changelog to Discord
working-directory: tests
env:
DISCORD_WEBHOOK_URL: ${{ secrets.DISCORD_WEBHOOK_URL }}
CI_SCREENSHOTS_S3_ENDPOINT: ${{ vars.CI_SCREENSHOTS_S3_ENDPOINT }}
CI_SCREENSHOTS_S3_ACCESS_KEY_ID: ${{ secrets.CI_SCREENSHOTS_S3_ACCESS_KEY_ID }}
CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY: ${{ secrets.CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY }}
run: npm run screenshots:changelog

View file

@ -318,19 +318,37 @@ jobs:
working-directory: tests
run: npm ci
# Baseline screenshots live in a private R2 bucket, pinned by the committed
# manifest; this cache keyed on the manifest hash makes most fetches a
# no-op. restore-keys makes a manifest change INCREMENTAL (restore the
# previous tree, download only the changed objects) instead of a full
# ~35MB re-download. A stale or partial restore is safe: r2-sync
# hash-verifies every file and deletes unlisted ones, and compare.ts
# refuses to gate against an incomplete cache.
# The baseline manifest lives in R2 (written only by the morelli review
# app + its seed script) — download it FIRST so this whole run pins to
# one manifest version and the cache step below can key on its hash.
# continue-on-error + the no-creds skip (which also deletes any stale
# local copy) keep secretless callers green — every later screenshot
# step then skips rather than gating on outdated baselines.
- name: Fetch baseline manifest from R2
if: inputs.run_tests
continue-on-error: true
working-directory: tests
env:
CI_SCREENSHOTS_S3_ENDPOINT: ${{ vars.CI_SCREENSHOTS_S3_ENDPOINT }}
CI_SCREENSHOTS_S3_ACCESS_KEY_ID: ${{ secrets.CI_SCREENSHOTS_S3_ACCESS_KEY_ID }}
CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY: ${{ secrets.CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY }}
run: npm run screenshots:fetch-manifest
# Baseline screenshots live in a private R2 bucket, pinned by the manifest
# fetched above; this cache keyed on the manifest hash makes most fetches
# a no-op (hashFiles evaluates at step run time, AFTER the fetch; a
# skipped fetch hashes to empty → restore-keys still warms the tree).
# restore-keys makes a manifest change INCREMENTAL (restore the previous
# tree, download only the changed objects) instead of a full ~35MB
# re-download. A stale or partial restore is safe: r2-sync hash-verifies
# every file and deletes unlisted ones, and compare.ts refuses to gate
# against an incomplete cache.
- name: Cache screenshot baselines
if: inputs.run_tests
uses: actions/cache@v4
with:
path: tests/baseline-screenshots
key: baselines-${{ hashFiles('tests/screenshot-manifest.json') }}
key: baselines-${{ hashFiles('tests/.baseline-manifest.json') }}
restore-keys: |
baselines-
@ -349,19 +367,17 @@ jobs:
run: npm run screenshots:fetch
# Cheap hygiene gates (no build needed): the determinism lint keeps the
# banned flake patterns out of the specs, the manifest check validates
# screenshot-manifest.json (v2 schema, sorted/unique hashes) and fails if
# baseline PNGs are ever re-committed to git (credential-free — it never
# touches R2, so it gates secretless callers identically), and
# the CI-coverage lint proves every spec file on disk is reachable from
# the npm scripts THIS workflow invokes (a spec/project that CI never
# runs is how the web suite once rotted unnoticed).
- name: Lint test determinism + screenshot manifest + CI coverage
# banned flake patterns out of the specs, and the CI-coverage lint proves
# every spec file on disk is reachable from the npm scripts THIS workflow
# invokes (a spec/project that CI never runs is how the web suite once
# rotted unnoticed). (The old screenshot-manifest --check gate went with
# the committed manifest — the manifest now lives in R2, written only by
# the morelli app, which validates on every promote.)
- name: Lint test determinism + CI coverage
if: inputs.run_tests
working-directory: tests
run: |
npm run lint:determinism
npm run screenshots:manifest -- --check
npm run lint:ci-coverage
- name: Install web workspace deps (collab bundle)