6953fbd894
Per run, a fresh ccx53 VM is created (Cyclenerd/hcloud-github-runner), builds + tests run on it, and it is always torn down — no persistent infra, no lingering cost. Secrets: HCLOUD_TOKEN, HETZNER_RUNNER_PAT. - ci-full-build.yml: the main-branch CI and the only push-triggered workflow. Full build of all 6 tools (pipelined wasm-opt, self-built Binaryen 130, ~1h15m) + KiCad e2e; docs-only pushes excluded via paths-ignore. - ci.yml (eeschema-only smoke) and wasm-opt-bench.yml (config sweeps): workflow_dispatch-only, since every push-run costs a paid VM. - Shared one-Hetzner-slot concurrency group across the workflows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
184 lines
8 KiB
YAML
184 lines
8 KiB
YAML
# Ephemeral Hetzner Cloud CI: a fresh ccx53 VM is created per run, the full
|
|
# build + e2e suites run on it, then it is destroyed. No persistent infra.
|
|
#
|
|
# Lifecycle (Cyclenerd/hcloud-github-runner): create-runner spins up a Hetzner
|
|
# VM registered as an ephemeral self-hosted runner -> build-and-test runs on it
|
|
# -> delete-runner tears it down (always, even on failure -> no lingering cost).
|
|
#
|
|
# Required repo secrets (emergence-engineering/pcbjam -> Settings -> Secrets):
|
|
# HCLOUD_TOKEN Hetzner Cloud API token, Read & Write
|
|
# HETZNER_RUNNER_PAT GitHub PAT with Administration: read & write (registers
|
|
# and unregisters the self-hosted runner)
|
|
#
|
|
# Submodules: kicad/wxwidgets use SSH URLs (.gitmodules) but point at PUBLIC repos,
|
|
# so actions/checkout rewrites them to HTTPS and the default GITHUB_TOKEN reads them.
|
|
|
|
name: CI (Hetzner)
|
|
|
|
# Manual-only (workflow_dispatch): every run costs a paid ephemeral Hetzner VM.
|
|
# Main-branch CI is ci-full-build.yml.
|
|
on:
|
|
workflow_dispatch:
|
|
inputs:
|
|
binaryen_cores:
|
|
description: "BINARYEN_CORES for the wasm-opt/asyncify step (blank = all cores / nproc). Use to A/B core counts."
|
|
required: false
|
|
default: ""
|
|
|
|
concurrency:
|
|
group: ci-hetzner-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
create-runner:
|
|
name: Create Hetzner runner
|
|
runs-on: ubuntu-latest # or ubicloud-standard-2 if GitHub-hosted is disabled
|
|
outputs:
|
|
label: ${{ steps.create.outputs.label }}
|
|
server_id: ${{ steps.create.outputs.server_id }}
|
|
steps:
|
|
- name: Create ephemeral ccx53 runner
|
|
id: create
|
|
uses: Cyclenerd/hcloud-github-runner@v1
|
|
with:
|
|
mode: create
|
|
github_token: ${{ secrets.HETZNER_RUNNER_PAT }}
|
|
hcloud_token: ${{ secrets.HCLOUD_TOKEN }}
|
|
server_type: ccx53
|
|
location: nbg1
|
|
image: ubuntu-24.04
|
|
|
|
build-and-test:
|
|
name: Build & test (Hetzner ccx53)
|
|
needs: create-runner
|
|
runs-on: ${{ needs.create-runner.outputs.label }}
|
|
timeout-minutes: 160 # cold --build-deps dominates; + wx build + both suites
|
|
env:
|
|
# Stream build-script output straight to the Actions log instead of
|
|
# redirecting it to logs/<script>/<ts>.log (see scripts/common/logging.sh,
|
|
# which returns early when KICAD_LOG_NESTED=1). Nothing consumes
|
|
# KICAD_LOG_FILE, so this is safe and gives live, greppable CI output.
|
|
KICAD_LOG_NESTED: "1"
|
|
steps:
|
|
# Hetzner's ubuntu-24.04 cloud image ships neither Docker nor git (Ubicloud
|
|
# preinstalled both). Install Docker CE + git before the Docker-based build.
|
|
- name: Install system deps (Docker, git)
|
|
run: |
|
|
export DEBIAN_FRONTEND=noninteractive
|
|
sudo apt-get update
|
|
# libjemalloc2: scalable allocator preloaded into the host-side wasm-opt
|
|
# passes (scripts/common/apply-asyncify.sh) to avoid glibc malloc
|
|
# arena-lock contention, which otherwise burns ~half of wasm-opt's CPU
|
|
# in futex lock-spin under many threads.
|
|
# autoconf/automake: build-wxuniversal-wasm.sh regenerates wxWidgets'
|
|
# ./configure from configure.in on a fresh checkout (git doesn't
|
|
# preserve mtimes, so configure.in looks newer). make: the host-side
|
|
# wxWidgets + wasm-test builds drive emconfigure/emmake/make directly.
|
|
# All of this runs on the bare host (not in Docker), and the Hetzner
|
|
# ubuntu-24.04 cloud image is minimal, so these must be installed here.
|
|
sudo apt-get install -y ca-certificates curl git libjemalloc2 autoconf automake make
|
|
# toolchain for BINARYEN_BUILD_FROM_SOURCE (get-wasm-opt.sh)
|
|
sudo apt-get install -y cmake ninja-build g++
|
|
sudo install -m 0755 -d /etc/apt/keyrings
|
|
curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
|
|
| sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
|
|
sudo chmod a+r /etc/apt/keyrings/docker.gpg
|
|
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
|
|
https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" \
|
|
| sudo tee /etc/apt/sources.list.d/docker.list >/dev/null
|
|
sudo apt-get update
|
|
sudo apt-get install -y docker-ce docker-ce-cli containerd.io \
|
|
docker-buildx-plugin docker-compose-plugin
|
|
|
|
- name: Checkout (with submodules)
|
|
uses: actions/checkout@v4
|
|
with:
|
|
submodules: recursive
|
|
|
|
- name: Show disk space
|
|
run: df -h
|
|
|
|
- name: Set up Node
|
|
uses: actions/setup-node@v4
|
|
with:
|
|
node-version: 20
|
|
|
|
# 1) Cold build KiCad WASM app + all dependencies (Docker; longest step).
|
|
# ccx53 has 32 vCPU / 128 GB RAM, so use every core: -j $(nproc) for the
|
|
# C++ compile and, by default, for the host-side wasm-opt/asyncify
|
|
# (apply-asyncify.sh runs on the host and reads BINARYEN_CORES). The
|
|
# workflow_dispatch input overrides BINARYEN_CORES so we can A/B core
|
|
# counts (e.g. 8 vs 32) without editing the workflow; blank = nproc.
|
|
- name: Build KiCad WASM (all, --build-deps)
|
|
env:
|
|
BINARYEN_CORES_INPUT: ${{ github.event.inputs.binaryen_cores }}
|
|
run: |
|
|
CORES="${BINARYEN_CORES_INPUT:-$(nproc)}"
|
|
# Lift the docker-compose dev-Mac resource caps (10 CPUs / 32G) so the
|
|
# container compile actually uses the ccx53's 32 cores / 128 GB.
|
|
export KICAD_DOCKER_CPUS="$(nproc)" KICAD_DOCKER_MEM=110G
|
|
# Self-built wasm-opt: official Linux tarball runs asyncify 4x slower
|
|
# (see get-wasm-opt.sh / run 27276830256). Identical output, sha256-verified.
|
|
export BINARYEN_BUILD_FROM_SOURCE=1
|
|
echo "Build parallelism: compile -j $(nproc) on ${KICAD_DOCKER_CPUS} CPUs, BINARYEN_CORES=${CORES}"
|
|
BINARYEN_CORES="${CORES}" ./docker/build.sh eeschema --build-deps -j "$(nproc)"
|
|
|
|
# 2) Host-side wxWidgets build that produces the wxWidgets test apps consumed
|
|
# by `npm run test` (NOT produced by docker/build.sh). Auto-installs emsdk.
|
|
- name: Build wxWidgets (wxUniversal WASM)
|
|
run: ./scripts/build-wxuniversal-wasm.sh
|
|
|
|
- name: Build wxWidgets test apps
|
|
run: ./scripts/build-wasm-test.sh
|
|
|
|
# 3) Test deps + browsers (firefox for KiCad headless, chromium for wxWidgets).
|
|
- name: Install test deps
|
|
working-directory: tests
|
|
run: npm ci
|
|
|
|
- name: Install Playwright browsers
|
|
working-directory: tests
|
|
run: npx playwright install --with-deps firefox chromium
|
|
|
|
# 4) Stage KiCad artifacts from ./output into tests/apps/kicad.
|
|
- name: Setup KiCad WASM for tests
|
|
working-directory: tests
|
|
run: npm run setup:kicad
|
|
|
|
# 5) Both e2e suites. GitHub sets CI=true automatically -> workers:1, retries,
|
|
# fresh per-run web server (see playwright configs).
|
|
- name: wxWidgets e2e (npm run test)
|
|
working-directory: tests
|
|
run: npm run test
|
|
|
|
- name: KiCad e2e (npm run test:kicad)
|
|
working-directory: tests
|
|
run: npm run test:kicad
|
|
|
|
# 6) Always capture logs/screenshots for post-mortem.
|
|
- name: Upload test logs & results
|
|
if: always()
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: e2e-logs
|
|
path: |
|
|
tests/logs/**
|
|
tests/test-results/**
|
|
tests/playwright-report/**
|
|
logs/**
|
|
if-no-files-found: ignore
|
|
|
|
delete-runner:
|
|
name: Delete Hetzner runner
|
|
needs: [create-runner, build-and-test]
|
|
runs-on: ubuntu-latest # match create-runner's choice
|
|
if: always() # tear down even if the build fails -> no lingering cost
|
|
steps:
|
|
- name: Delete ephemeral runner
|
|
uses: Cyclenerd/hcloud-github-runner@v1
|
|
with:
|
|
mode: delete
|
|
github_token: ${{ secrets.HETZNER_RUNNER_PAT }}
|
|
hcloud_token: ${{ secrets.HCLOUD_TOKEN }}
|
|
name: ${{ needs.create-runner.outputs.label }}
|
|
server_id: ${{ needs.create-runner.outputs.server_id }}
|