jspi cleanup: remove the asyncify-era residue — dead code, conditionals, pipeline scaffolding, stale prose
The runtime is JSPI-only; this removes everything that still pretended otherwise. Three exhaustive sweeps (C++/JS+build+CI/tests+docs) drove the inventory; every deletion verified by grep closure + full gates. Broken-right-now fixes: - deploy-staging.yml passed the retired opt_level input — the workflow could not even start. Removed. - env.sh carried dead exports with a live -sASYNCIFY=1 inside (WASM_LDFLAGS/PTHREAD_LDFLAGS, zero consumers). Removed; the WASM_LEGACY_EXCEPTIONS rationale rewritten to the real reason. - docker/build.sh exported PCBJAM_ASYNC_BACKEND (read nowhere). Gone. Dead weight removed: - binaryen submodule (nothing builds or invokes it), wasm-opt-bench workflow + scripts/bench/, get-wasm-opt.sh, diagnostics.js (242 lines of Asyncify-API-only code), the KICAD_PIPELINE background-postprocess scaffolding (existed to parallelize the deleted wasm-opt phase; the postprocess is a seconds-long node script and now runs inline), build-monitor's dead asyncify rows, sched-context orphan build output, dead .gitignore entries, the .jspi-assets spike dir (the two wf-result research JSONs moved to docs/features/async/migration-evidence/). - bindings: fiber_park.h + its 12 embind registrations (broken-if- called under JSPI), the kicadOpenFileStart/OPEN_JOB starter route, main_stack_runner.h + 5 includes, the always-null context-sleep weak hook in nanosleep_yield.c. - shim: the backend field (installed-flag idempotency instead), noteContextWait (dead both sides), the __wxAsyncifyDump alias (+ the WasmTool fallback and string-dump normalize branch). - web: the emscripten-6-ignored mainScriptUrlOrBlob option in boot.ts (gerber-demo keeps it: it loads the deployed CDN release, which predates emscripten 6 — noted inline). Conditionals: all 'backend === jspi' checks reduced to scheduler- presence checks; races_quiescent re-keyed from Asyncify.state (vacuous) to real backlog quiescence (resumeReady/mutatorQueue — NOT _windowLive, which is the probing activation's own window by definition). Renames (identifiers only, no file renames): ASYNC_LINK_FLAGS→ JSPI_LINK_FLAGS and Makefile ASYNC_LDFLAGS→JSPI_LDFLAGS, kicadCollabFiberBusy→kicadCollabBusy (embind + web + tests), collab_common.h fiber*→apply*/coroutine naming, asyncifySignatures→ wasmTrapSignatures (lists byte-identical). Tests: the two remaining vacuous [wx-asyncify]/fiber-resume-refused asserts re-keyed to live JSPI beacons; eeschema-load's failure message no longer sends the developer to a deleted script; wait-beacons' dead families/parser deleted; lane-0 legacy-glue guards removed (lane 0 is unconstructible); the embind test.fail re-gated with the JSPI reason (plain embind invokers cannot suspend — verified still failing); lint-determinism now scans tests/jspi (166 files clean); eeschema-collab local-move gated to chromium (~50% flaky on FF even solo; pcbnew twin covers both engines). Docs: DEBUG.md rewritten as the JSPI debugging guide; build.md describes the single-phase build; docs/features/async/README.md banner-marked historical and repointed at the NEW 23-jspi-runtime.md (current architecture: export census, turnstile, libcontext ownership + refusal contract, embind call shapes, the em-pthread service-wrapper trick, exception policy, known gaps). Gates on the cleaned tree: test:e2e 725 passed / 0 failed (after the quiescence-probe fix; the 3 other reds were verified contention flakes solo-green or the documented FF gate), web 76/0, jspi 18/18 both engines, vitest 295/295 + 17/17, all lints green, live-app census clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016X9eh1s5sTx1o9Em9KBuwR
This commit is contained in:
Viktor Vaczi
parent
3ee174e9b4
commit
9c475a804e
120 changed files with 1522 additions and 2974 deletions
1
.github/workflows/deploy-staging.yml
vendored
1
.github/workflows/deploy-staging.yml
vendored
|
|
@ -25,7 +25,6 @@ jobs:
|
|||
build:
|
||||
uses: ./.github/workflows/wasm-build.yml
|
||||
with:
|
||||
opt_level: "-O1"
|
||||
build_3d_viewer: "ON"
|
||||
run_tests: true
|
||||
upload_output: true
|
||||
|
|
|
|||
197
.github/workflows/wasm-opt-bench.yml
vendored
197
.github/workflows/wasm-opt-bench.yml
vendored
|
|
@ -1,197 +0,0 @@
|
|||
# Dedicated benchmark of the host-side `wasm-opt -O2` pass (the ~80-min CI
|
||||
# bottleneck). Same ephemeral Hetzner ccx53 as the main CI, but instead of the
|
||||
# full build+e2e it builds the asyncified `-O2` INPUT once, caches it as an
|
||||
# artifact, and replays scripts/bench/o2-config-sweep.sh over it under a matrix
|
||||
# of allocator/THP/thread configs. Goal: find the env that kills the kernel
|
||||
# page-management storm (see docs/ci-build-slowness-findings.md).
|
||||
#
|
||||
# Manual-only (workflow_dispatch): every run costs a paid ephemeral Hetzner VM,
|
||||
# so nothing here triggers on push. Dispatch from the Actions tab (the workflow
|
||||
# must exist on the default branch for that).
|
||||
#
|
||||
# Parameters live in scripts/bench/sweep.conf (committed) or the dispatch
|
||||
# inputs below. To reuse a prior run's fixture (skip the ~40-min build), set
|
||||
# fixture_run_id to that run's id.
|
||||
#
|
||||
# Reuses the main CI's secrets: HCLOUD_TOKEN, HETZNER_RUNNER_PAT.
|
||||
|
||||
name: wasm-opt bench
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
configs: { description: "space-separated config presets (override conf)", required: false, default: "" }
|
||||
cores: { description: "BINARYEN_CORES (override conf)", required: false, default: "" }
|
||||
fixture_run_id: { description: "run id to pull the cached fixture from (blank = build it)", required: false, default: "" }
|
||||
diagnostic: { description: "1 = capture perf/vmstat/irq around first config", required: false, default: "" }
|
||||
cap_seconds: { description: "windowed-sample seconds per config (0 = run to completion)", required: false, default: "" }
|
||||
|
||||
# One Hetzner bench VM at a time. (The main CI uses a different runner; still,
|
||||
# don't push the ci-hetzner branches while a bench runs — we only have one slot.)
|
||||
concurrency:
|
||||
group: wasm-opt-bench
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
create-runner:
|
||||
name: Create Hetzner runner
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
label: ${{ steps.create.outputs.label }}
|
||||
server_id: ${{ steps.create.outputs.server_id }}
|
||||
steps:
|
||||
- name: Create ephemeral ccx53 runner
|
||||
id: create
|
||||
uses: Cyclenerd/hcloud-github-runner@v1
|
||||
with:
|
||||
mode: create
|
||||
github_token: ${{ secrets.HETZNER_RUNNER_PAT }}
|
||||
hcloud_token: ${{ secrets.HCLOUD_TOKEN }}
|
||||
server_type: ccx53
|
||||
location: nbg1
|
||||
image: ubuntu-24.04
|
||||
|
||||
bench:
|
||||
name: wasm-opt -O2 config sweep (Hetzner ccx53)
|
||||
needs: create-runner
|
||||
runs-on: ${{ needs.create-runner.outputs.label }}
|
||||
timeout-minutes: 210
|
||||
env:
|
||||
KICAD_LOG_NESTED: "1" # live build logs straight to the Actions console
|
||||
steps:
|
||||
- name: Checkout (with submodules)
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Resolve sweep parameters
|
||||
id: params
|
||||
# Pass workflow_dispatch inputs via env (never interpolate ${{ }} into a
|
||||
# shell body) so a value can't be parsed as shell. They override the
|
||||
# committed conf when non-empty; the run id is validated numeric before
|
||||
# it reaches download-artifact.
|
||||
env:
|
||||
IN_CONFIGS: ${{ inputs.configs }}
|
||||
IN_CORES: ${{ inputs.cores }}
|
||||
IN_FIXTURE_RUN_ID: ${{ inputs.fixture_run_id }}
|
||||
IN_DIAGNOSTIC: ${{ inputs.diagnostic }}
|
||||
IN_CAP: ${{ inputs.cap_seconds }}
|
||||
run: |
|
||||
set -e
|
||||
CONF="scripts/bench/sweep.conf"
|
||||
[ -f "$CONF" ] && . "$CONF" || true
|
||||
CONFIGS="${IN_CONFIGS:-${CONFIGS_CONF:-baseline}}"
|
||||
CORES="${IN_CORES:-${CORES_CONF:-}}"
|
||||
FIXTURE_RUN_ID="${IN_FIXTURE_RUN_ID:-${FIXTURE_RUN_ID_CONF:-}}"
|
||||
DIAGNOSTIC="${IN_DIAGNOSTIC:-${DIAGNOSTIC_CONF:-0}}"
|
||||
CAP_SECONDS="${IN_CAP:-${CAP_SECONDS_CONF:-0}}"
|
||||
if [ -n "$FIXTURE_RUN_ID" ] && ! printf '%s' "$FIXTURE_RUN_ID" | grep -qE '^[0-9]+$'; then
|
||||
echo "::error::FIXTURE_RUN_ID must be numeric (got '$FIXTURE_RUN_ID')"; exit 1
|
||||
fi
|
||||
BUILD_FIXTURE=true; [ -n "$FIXTURE_RUN_ID" ] && BUILD_FIXTURE=false
|
||||
{
|
||||
echo "CONFIGS=$CONFIGS"
|
||||
echo "CORES=$CORES"
|
||||
echo "FIXTURE_RUN_ID=$FIXTURE_RUN_ID"
|
||||
echo "DIAGNOSTIC=$DIAGNOSTIC"
|
||||
echo "CAP_SECONDS=$CAP_SECONDS"
|
||||
echo "BUILD_FIXTURE=$BUILD_FIXTURE"
|
||||
} >> "$GITHUB_ENV"
|
||||
echo "build_fixture=$BUILD_FIXTURE" >> "$GITHUB_OUTPUT"
|
||||
echo "::notice::configs='$CONFIGS' cores='${CORES:-nproc}' cap=${CAP_SECONDS}s build_fixture=$BUILD_FIXTURE fixture_run_id='${FIXTURE_RUN_ID:-none}' diagnostic=$DIAGNOSTIC"
|
||||
|
||||
- name: Show machine
|
||||
run: |
|
||||
echo "nproc=$(nproc)"; free -h; df -h /
|
||||
echo "THP: $(cat /sys/kernel/mm/transparent_hugepage/enabled)"; uname -a
|
||||
|
||||
# Allocators + measurement tools. mimalloc is the key data point (Binaryen
|
||||
# #5561): prefer the distro package, source-build it if not packaged.
|
||||
- name: Install bench deps (allocators, time, perf, strace)
|
||||
run: |
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y time strace libjemalloc2 cmake g++ git curl ca-certificates \
|
||||
linux-tools-common linux-tools-generic "linux-tools-$(uname -r)" || \
|
||||
sudo apt-get install -y time strace libjemalloc2 cmake g++ git curl ca-certificates linux-tools-common linux-tools-generic
|
||||
sudo apt-get install -y libmimalloc2.0 || sudo apt-get install -y libmimalloc-dev || true
|
||||
if ! ls /usr/lib/$(uname -m)-linux-gnu/libmimalloc.so* >/dev/null 2>&1; then
|
||||
echo "mimalloc not packaged — building from source"
|
||||
git clone --depth 1 https://github.com/microsoft/mimalloc /tmp/mimalloc
|
||||
cmake -S /tmp/mimalloc -B /tmp/mimalloc/out -DCMAKE_BUILD_TYPE=Release >/dev/null
|
||||
cmake --build /tmp/mimalloc/out -j"$(nproc)" >/dev/null
|
||||
sudo cp -av /tmp/mimalloc/out/libmimalloc.so* /usr/lib/$(uname -m)-linux-gnu/
|
||||
fi
|
||||
echo "allocators present:"; ls -l /usr/lib/$(uname -m)-linux-gnu/libjemalloc.so* /usr/lib/$(uname -m)-linux-gnu/libmimalloc.so* 2>/dev/null || true
|
||||
|
||||
# ---- Fixture: build once (asyncify, no -O2) and cache, or download it ----
|
||||
- name: Install Docker (fixture build only)
|
||||
if: steps.params.outputs.build_fixture == 'true'
|
||||
run: |
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
sudo apt-get install -y ca-certificates curl
|
||||
sudo install -m 0755 -d /etc/apt/keyrings
|
||||
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
|
||||
sudo chmod a+r /etc/apt/keyrings/docker.gpg
|
||||
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
|
||||
https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" \
|
||||
| sudo tee /etc/apt/sources.list.d/docker.list >/dev/null
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
|
||||
|
||||
- name: Build asyncified fixture (ASYNCIFY_ONLY, no -O2)
|
||||
if: steps.params.outputs.build_fixture == 'true'
|
||||
run: |
|
||||
# Stop after the asyncify pass so the artifact is exactly the -O2 INPUT.
|
||||
ASYNCIFY_ONLY=1 BINARYEN_CORES="$(nproc)" ./docker/build.sh eeschema --build-deps -j "$(nproc)"
|
||||
ls -lh output/eeschema.wasm
|
||||
cp output/eeschema.wasm output/eeschema.asyncified.wasm
|
||||
|
||||
- name: Upload fixture artifact
|
||||
if: steps.params.outputs.build_fixture == 'true'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: o2-fixture
|
||||
path: output/eeschema.asyncified.wasm
|
||||
retention-days: 7
|
||||
|
||||
- name: Download cached fixture
|
||||
if: steps.params.outputs.build_fixture == 'false'
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: o2-fixture
|
||||
path: output
|
||||
run-id: ${{ env.FIXTURE_RUN_ID }}
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
# ---- The sweep ----
|
||||
- name: Run -O2 config sweep
|
||||
run: |
|
||||
FIX="output/eeschema.asyncified.wasm"
|
||||
[ -f "$FIX" ] || FIX="output/eeschema.wasm"
|
||||
ls -lh "$FIX"
|
||||
CONFIGS="${CONFIGS}" CORES="${CORES}" DIAGNOSTIC="${DIAGNOSTIC}" CAP_SECONDS="${CAP_SECONDS}" \
|
||||
./scripts/bench/o2-config-sweep.sh "$FIX"
|
||||
|
||||
- name: Upload bench results
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: o2-bench-results-${{ github.run_id }}
|
||||
path: bench/o2-results/**
|
||||
if-no-files-found: warn
|
||||
|
||||
delete-runner:
|
||||
name: Delete Hetzner runner
|
||||
needs: [create-runner, bench]
|
||||
runs-on: ubuntu-latest
|
||||
if: always()
|
||||
steps:
|
||||
- name: Delete ephemeral runner
|
||||
uses: Cyclenerd/hcloud-github-runner@v1
|
||||
with:
|
||||
mode: delete
|
||||
github_token: ${{ secrets.HETZNER_RUNNER_PAT }}
|
||||
hcloud_token: ${{ secrets.HCLOUD_TOKEN }}
|
||||
name: ${{ needs.create-runner.outputs.label }}
|
||||
server_id: ${{ needs.create-runner.outputs.server_id }}
|
||||
Loading…
Reference in a new issue