screenshots: R2-hosted manifest becomes the baseline source of truth (morelli cutover)
The committed tests/screenshot-manifest.json is retired. CI now downloads baselines/pcbjam/manifest.json (manifest v3, written only by the morelli review app + its seed script) to the gitignored .baseline-manifest.json, and everything downstream (pull, verify, compare) reads that copy: - config.ts: MANIFEST_VERSION 3, MANIFEST_PATH .baseline-manifest.json, R2_BASELINES_MANIFEST_KEY; ManifestEntry grows opaque provenance - r2-sync.ts: new --manifest mode (atomic fetch; no-creds skip DELETES a stale copy so the gate skips rather than using old baselines); --push gone (bytes enter the CAS only via morelli's promote) - compare.ts: hard-skips when no manifest was fetched — a stale warm cache can never gate - wasm-build.yml: fetch-manifest step before the baselines cache; cache key now hashes the fetched manifest; the gen-manifest --check lint gate goes with the committed manifest - deleted: screenshot-manifest.json, promote.ts, changelog.ts, gen-manifest.ts, screenshot-changelog.yml, promote-screenshots skill - docs (CLAUDE/README/TESTING/WHATWORKS/tools README): promote flow is now https://pcbjam-morelli-staging.pcbjam-staging.workers.dev Validated locally against the real bucket: fetch-manifest (492), cold pull 492 / warm pull cached=492, no-creds skip chain, compare gate skip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
4196958c10
commit
ec7a1b0787
20 changed files with 239 additions and 4793 deletions
45
.github/workflows/screenshot-changelog.yml
vendored
45
.github/workflows/screenshot-changelog.yml
vendored
|
|
@ -1,45 +0,0 @@
|
|||
name: screenshot-changelog
|
||||
|
||||
# Discord trigger B: when a push to main changes the screenshot manifest (the
|
||||
# committed pin of every R2-stored baseline), post an old | new+boxes | heatmap
|
||||
# triptych per changed baseline (ADDED image / REMOVED title too). No build, no
|
||||
# GPU — it diffs two git revisions of the manifest and fetches the PNG bytes
|
||||
# from the R2 CAS bucket (immutable objects, so the old rev's hashes always
|
||||
# resolve), so it runs in ~30s. Complements the re-render drift gate in
|
||||
# wasm-build.yml (which catches un-blessed renders); this is the human-facing
|
||||
# feed of intentional baseline updates as they land.
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'tests/screenshot-manifest.json'
|
||||
|
||||
concurrency:
|
||||
group: screenshot-changelog-${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
changelog:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
# HEAD^ is needed to diff the tip commit's manifest changes. (A push of
|
||||
# multiple commits reports only the tip commit's baseline diff — fine for
|
||||
# a changelog; baseline refreshes are single commits in practice.)
|
||||
fetch-depth: 2
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
- name: Install screenshot tooling deps
|
||||
working-directory: tests
|
||||
run: npm ci
|
||||
- name: Post baseline changelog to Discord
|
||||
working-directory: tests
|
||||
env:
|
||||
DISCORD_WEBHOOK_URL: ${{ secrets.DISCORD_WEBHOOK_URL }}
|
||||
CI_SCREENSHOTS_S3_ENDPOINT: ${{ vars.CI_SCREENSHOTS_S3_ENDPOINT }}
|
||||
CI_SCREENSHOTS_S3_ACCESS_KEY_ID: ${{ secrets.CI_SCREENSHOTS_S3_ACCESS_KEY_ID }}
|
||||
CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY: ${{ secrets.CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY }}
|
||||
run: npm run screenshots:changelog
|
||||
50
.github/workflows/wasm-build.yml
vendored
50
.github/workflows/wasm-build.yml
vendored
|
|
@ -318,19 +318,37 @@ jobs:
|
|||
working-directory: tests
|
||||
run: npm ci
|
||||
|
||||
# Baseline screenshots live in a private R2 bucket, pinned by the committed
|
||||
# manifest; this cache keyed on the manifest hash makes most fetches a
|
||||
# no-op. restore-keys makes a manifest change INCREMENTAL (restore the
|
||||
# previous tree, download only the changed objects) instead of a full
|
||||
# ~35MB re-download. A stale or partial restore is safe: r2-sync
|
||||
# hash-verifies every file and deletes unlisted ones, and compare.ts
|
||||
# refuses to gate against an incomplete cache.
|
||||
# The baseline manifest lives in R2 (written only by the morelli review
|
||||
# app + its seed script) — download it FIRST so this whole run pins to
|
||||
# one manifest version and the cache step below can key on its hash.
|
||||
# continue-on-error + the no-creds skip (which also deletes any stale
|
||||
# local copy) keep secretless callers green — every later screenshot
|
||||
# step then skips rather than gating on outdated baselines.
|
||||
- name: Fetch baseline manifest from R2
|
||||
if: inputs.run_tests
|
||||
continue-on-error: true
|
||||
working-directory: tests
|
||||
env:
|
||||
CI_SCREENSHOTS_S3_ENDPOINT: ${{ vars.CI_SCREENSHOTS_S3_ENDPOINT }}
|
||||
CI_SCREENSHOTS_S3_ACCESS_KEY_ID: ${{ secrets.CI_SCREENSHOTS_S3_ACCESS_KEY_ID }}
|
||||
CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY: ${{ secrets.CI_SCREENSHOTS_S3_SECRET_ACCESS_KEY }}
|
||||
run: npm run screenshots:fetch-manifest
|
||||
|
||||
# Baseline screenshots live in a private R2 bucket, pinned by the manifest
|
||||
# fetched above; this cache keyed on the manifest hash makes most fetches
|
||||
# a no-op (hashFiles evaluates at step run time, AFTER the fetch; a
|
||||
# skipped fetch hashes to empty → restore-keys still warms the tree).
|
||||
# restore-keys makes a manifest change INCREMENTAL (restore the previous
|
||||
# tree, download only the changed objects) instead of a full ~35MB
|
||||
# re-download. A stale or partial restore is safe: r2-sync hash-verifies
|
||||
# every file and deletes unlisted ones, and compare.ts refuses to gate
|
||||
# against an incomplete cache.
|
||||
- name: Cache screenshot baselines
|
||||
if: inputs.run_tests
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: tests/baseline-screenshots
|
||||
key: baselines-${{ hashFiles('tests/screenshot-manifest.json') }}
|
||||
key: baselines-${{ hashFiles('tests/.baseline-manifest.json') }}
|
||||
restore-keys: |
|
||||
baselines-
|
||||
|
||||
|
|
@ -349,19 +367,17 @@ jobs:
|
|||
run: npm run screenshots:fetch
|
||||
|
||||
# Cheap hygiene gates (no build needed): the determinism lint keeps the
|
||||
# banned flake patterns out of the specs, the manifest check validates
|
||||
# screenshot-manifest.json (v2 schema, sorted/unique hashes) and fails if
|
||||
# baseline PNGs are ever re-committed to git (credential-free — it never
|
||||
# touches R2, so it gates secretless callers identically), and
|
||||
# the CI-coverage lint proves every spec file on disk is reachable from
|
||||
# the npm scripts THIS workflow invokes (a spec/project that CI never
|
||||
# runs is how the web suite once rotted unnoticed).
|
||||
- name: Lint test determinism + screenshot manifest + CI coverage
|
||||
# banned flake patterns out of the specs, and the CI-coverage lint proves
|
||||
# every spec file on disk is reachable from the npm scripts THIS workflow
|
||||
# invokes (a spec/project that CI never runs is how the web suite once
|
||||
# rotted unnoticed). (The old screenshot-manifest --check gate went with
|
||||
# the committed manifest — the manifest now lives in R2, written only by
|
||||
# the morelli app, which validates on every promote.)
|
||||
- name: Lint test determinism + CI coverage
|
||||
if: inputs.run_tests
|
||||
working-directory: tests
|
||||
run: |
|
||||
npm run lint:determinism
|
||||
npm run screenshots:manifest -- --check
|
||||
npm run lint:ci-coverage
|
||||
|
||||
- name: Install web workspace deps (collab bundle)
|
||||
|
|
|
|||
Loading…
Reference in a new issue