pcbjam/scripts/bench/sweep.conf
Istvan Matejcsok df25275843 bench: wasm-opt benchmark harness — Hetzner config sweeps + local QEMU VM
Tooling that produced the perf findings (committed for rerunnability):
- scripts/bench/wasm-opt-bench.sh + o2-config-sweep.sh + sweep.conf:
  replay asyncify/-O2 over a cached fixture under allocator/THP/core
  matrices on the Hetzner runner.
- scripts/bench/setup-vm.sh + cloud-init/ + vm-build.sh: local QEMU
  (HVF) Ubuntu guest with Docker CE to verify Linux builds without
  burning paid runners.
- .gitignore: bench fixtures/results and the VM image stay local.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 07:48:46 +02:00

33 lines
1.9 KiB
Text

# Parameters for the wasm-opt config sweep (.github/workflows/wasm-opt-bench.yml).
# Edit, commit, push to a bench/** branch to launch a run.
#
# Config grammar: <preset>[@<cores>] (see scripts/bench/o2-config-sweep.sh)
# presets: baseline(v121 -O2) v130-O2 v130-O1 v121-O1 v130-light v121-light
# mimalloc-retain thp-off (last two are proven-dead controls)
# @<cores> overrides BINARYEN_CORES for that cell (thread sweep).
# CAP_SECONDS_CONF: >0 = windowed storm sample (~CAP/60 min/cell, whole matrix in
# one run); 0 = run each to completion (true wall-clock). FIXTURE_RUN_ID_CONF:
# reuse a prior run's cached fixture (blank = build it ~40 min).
#
# VERDICT from run #1 (27197360957): the -O2 cost is ~90% FUTEX LOCK CONTENTION
# in Binaryen (type mutex), NOT allocator/THP (madvise=0, compaction=0, identical
# under glibc & mimalloc; mimalloc-retain saved only 8%). So allocator/THP are
# dead; the levers are NEWER Binaryen (devs fixed this after v121) and FEWER
# threads, plus lighter passes. Run #2 triages those windowed (fast):
# win_sysfrac = lock contention (LOWER better)
# win_usercores = real-work rate (HIGHER better → shorter wall for same passes)
# ---- Run #2: windowed triage (10 min/cell) on the cached run-#1 fixture ----
# baseline = v121 -O2 @32c (control: expect sys_frac~0.88, user_cores~2.6)
# v130-O2 = newer Binaryen @32c (does the version fix the lock?)
# baseline@8 = v121 -O2 @8c (do fewer threads cut contention?)
# v130-O2@8 = newer + fewer threads
# v121-O1 = lighter passes @32c (does less work progress faster?)
CONFIGS_CONF="baseline v130-O2 baseline@8 v130-O2@8 v121-O1"
CORES_CONF=""
FIXTURE_RUN_ID_CONF="27197360957"
DIAGNOSTIC_CONF="0"
CAP_SECONDS_CONF="600"
# ---- Run #3 (planned): full run (CAP_SECONDS_CONF="0") of the winner(s) for true
# wall-clock + output validity, then Chrome e2e. e.g. CONFIGS="v130-O2 v130-O2@8"