wip: nested-asyncify fix, wxAuiToolBar registration, tests, research docs
Main-repo side of a multi-part WIP covering the KiCad WASM tool-selection and nested-Asyncify work. Submodule commits are in kicad@f6e9239aaa (libcontext hygiene) and wxwidgets@bb80f91e8b (auibar registration + dialog diagnostics). ## scripts/common/inject-dyncall-shims.sh Wrap Asyncify.handleSleep / allocateData to save-and-restore Asyncify.currData around each EM_ASYNC_JS sleep. This fixes the nested Asyncify collision where a fiber swap that fired during a modal's event loop clobbered currData, and the modal's later doRewind used the fiber's buffer and hit "RuntimeError: index out of bounds". Root cause documented as Emscripten Issue #9153 (wontfix upstream). Diagnostic-rewind logging (forcedBottomOfCallStack, callStack traces) is retained to help future debugging of Asyncify state corruption. ## tests/ - tests/playwright-kicad.config.ts: add `channel: 'chrome'` for the chromium project so --project=chromium --headed uses system Chrome (real GPU) instead of SwiftShader on ARM Mac. Also switch trace to retain-on-failure + screenshot on-failure for easier E2E debugging. - tests/kicad/pcbnew.spec.ts: replace `tool.checked` assertions with a label-suffix check (`[checked]`) since our auibar registration encodes checked state in the label (no schema change to the registry). - tests/apps/Makefile.wasm: add `coroutine-nested` build target + include it in the all: list. - tests/apps/standalone/coroutine/: kicad_coroutine_harness.h + test app reproducing KiCad COROUTINE semantics against real libcontext. - tests/apps/standalone/coroutine-nested/: nested_test.cpp reproduces the EM_ASYNC_JS-modal + fiber-swap nesting bug in isolation. 8 scenarios from baseline_modal_alone through nested_fibers_inside_modal. - tests/e2e/coroutine.spec.ts + coroutine-nested.spec.ts: Playwright specs that load the standalone apps and assert all case cases pass via [COROUTINE_TEST] SUMMARY log parsing. ## research/ and features/browser-tools/ Three background docs capturing the investigation trajectory: - features/browser-tools/0001-kicad-wasm-tool-activation-investigation.md Early investigation: why tools don't activate; initial dynCall-empty- callback hypothesis. - features/browser-tools/0002-wasm-coroutine-deep-dive.md Deep dive on Asyncify internals, fiber API, QEMU's coroutine-wasm reference implementation. - features/browser-tools/0003-wxauitoolbar-registration-fix.md The narrow fix: why wxAuiToolBar needs a registration block, where to add it, what the fallback plan is. - research/threading_1.md: corrected root-cause analysis after reading runtime logs — nested-Asyncify currData collision, Emscripten #9153. - research/threading_2.md: extended research on alternative approaches (JSPI/WasmFX/state-machines) and why they don't help here. ## Submodule pointer updates kicad: f6e9239aaa (wip: libcontext WASM hygiene cleanup) wxwidgets: bb80f91e8b (wip: wxAuiToolBar element-registry registration + dialog diagnostics) ## Open threads not yet in scope - Firefox/Chrome divergent behavior: "indirect call signature mismatch" traps in Firefox vs renderer crash in system Chrome (tracked in plans/peaceful-hugging-pnueli.md and the research docs). - E2E pixel-diff for Draw Lines fails because the test's diff region does not cover where the line is actually drawn; tool activation works, the line is visible in test-results/pcbnew-draw-lines-02-after-drawing.png. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Viktor Vaczi
parent
a49ed49d5d
commit
9a04217788
17 changed files with 5190 additions and 46 deletions
|
|
@ -30,11 +30,10 @@ echo " Tool: ${WASM_OPT}"
|
|||
# Asyncify import patterns (functions that trigger async suspension)
|
||||
# - env.invoke_* : Exception handling trampolines
|
||||
# - env.__asyncjs__* : EM_ASYNC_JS functions (like startModal())
|
||||
ASYNCIFY_IMPORTS="env.invoke_*,env.__asyncjs__*"
|
||||
ASYNCIFY_IMPORTS="env.invoke_*,env.__asyncjs__*,env.emscripten_fiber_swap"
|
||||
|
||||
# Functions to exclude from asyncify instrumentation
|
||||
# These are large functions that inflate beyond V8's 7.65MB function limit
|
||||
# The names must match exactly as they appear in the WASM (C++ mangled names)
|
||||
# These are large functions that inflate beyond V8's local-count limits.
|
||||
ASYNCIFY_REMOVE=$(cat << 'REMOVELIST'
|
||||
COLOR_SETTINGS::COLOR_SETTINGS(wxString const&, bool)
|
||||
BuildBitmapInfo(std::__2::unordered_map<BITMAPS, std::__2::vector<BITMAP_INFO, std::__2::allocator<BITMAP_INFO>>, std::__2::hash<BITMAPS>, std::__2::equal_to<BITMAPS>, std::__2::allocator<std::__2::pair<BITMAPS const, std::__2::vector<BITMAP_INFO, std::__2::allocator<BITMAP_INFO>>>>>&)
|
||||
|
|
@ -49,7 +48,6 @@ ShapeFix_Wire::FixGap2d(int, bool)
|
|||
REMOVELIST
|
||||
)
|
||||
|
||||
# Convert newlines to commas for the removelist
|
||||
ASYNCIFY_REMOVE_ARG=$(echo "${ASYNCIFY_REMOVE}" | tr '\n' ',' | sed 's/,$//')
|
||||
|
||||
echo ""
|
||||
|
|
|
|||
|
|
@ -137,6 +137,200 @@ rm "$SHIM_FILE"
|
|||
|
||||
echo "Successfully injected $SIG_COUNT dynCall shims into $JS_FILE"
|
||||
|
||||
# Stabilize Asyncify rewind targets for Emscripten fibers.
|
||||
# When a fiber resumes via rewind, the outer JS frame can collapse to the
|
||||
# innermost dynCall wrapper on the next yield. Giving each fiber its own stable
|
||||
# rewind wrapper preserves the original re-entry target across later yields.
|
||||
if grep -q 'Fiber rewind stabilization for Asyncify fibers' "$JS_FILE"; then
|
||||
echo "Fiber rewind stabilization already present - skipping"
|
||||
else
|
||||
FIBER_PATCH_FILE=$(mktemp)
|
||||
cat > "$FIBER_PATCH_FILE" << 'FIBERPATCH'
|
||||
|
||||
// === Fiber rewind stabilization for Asyncify fibers ===
|
||||
if (typeof Asyncify !== "undefined" && typeof Fibers !== "undefined" && typeof _emscripten_fiber_swap !== "undefined") {
|
||||
var __originalAsyncifySetDataRewindFunc = Asyncify.setDataRewindFunc.bind(Asyncify);
|
||||
Asyncify.setDataRewindFunc = function(ptr, forcedBottomOfCallStack) {
|
||||
if (forcedBottomOfCallStack) {
|
||||
var rewindId = Asyncify.getCallStackId(forcedBottomOfCallStack);
|
||||
GROWABLE_HEAP_I32()[(((ptr) + (8)) >> 2)] = rewindId;
|
||||
return;
|
||||
}
|
||||
return __originalAsyncifySetDataRewindFunc(ptr);
|
||||
};
|
||||
|
||||
Fibers.rewindTargetByFiber ||= {};
|
||||
Fibers.rewindWrapperByFiber ||= {};
|
||||
Fibers.entryWrapperByFiber ||= {};
|
||||
Fibers.entryKeyByFiber ||= {};
|
||||
Fibers.shouldUseStableRewindTarget ||= function(fiber) {
|
||||
return GROWABLE_HEAP_U32()[(((fiber) + (16)) >> 2)] !== 0;
|
||||
};
|
||||
Fibers.ensureRewindWrapper ||= function(fiber) {
|
||||
var key = "__fiber_rewind_" + fiber;
|
||||
|
||||
if (!Fibers.rewindWrapperByFiber[fiber]) {
|
||||
var target = Fibers.rewindTargetByFiber[fiber];
|
||||
|
||||
Fibers.rewindWrapperByFiber[fiber] = function() {
|
||||
Asyncify.exportCallStack.push(key);
|
||||
try {
|
||||
return wasmExports[target]();
|
||||
} finally {
|
||||
if (!ABORT) {
|
||||
Asyncify.exportCallStack.pop();
|
||||
Asyncify.maybeStopUnwind();
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
wasmExports[key] = Fibers.rewindWrapperByFiber[fiber];
|
||||
}
|
||||
|
||||
return key;
|
||||
};
|
||||
|
||||
var __originalFinishContextSwitch = Fibers.finishContextSwitch;
|
||||
Fibers.finishContextSwitch = function(newFiber) {
|
||||
var entryPoint = GROWABLE_HEAP_U32()[(((newFiber) + (12)) >> 2)];
|
||||
|
||||
if (entryPoint !== 0 && !Fibers.entryWrapperByFiber[newFiber]) {
|
||||
var userData = GROWABLE_HEAP_U32()[(((newFiber) + (16)) >> 2)];
|
||||
var entryKey = "__fiber_entry_" + newFiber;
|
||||
|
||||
Fibers.entryWrapperByFiber[newFiber] = function() {
|
||||
return dynCall_vi(entryPoint, userData);
|
||||
};
|
||||
Fibers.entryKeyByFiber[newFiber] = entryKey;
|
||||
wasmExports[entryKey] = Fibers.entryWrapperByFiber[newFiber];
|
||||
}
|
||||
|
||||
return __originalFinishContextSwitch(newFiber);
|
||||
};
|
||||
|
||||
var __originalEmscriptenFiberSwap = _emscripten_fiber_swap;
|
||||
_emscripten_fiber_swap = (oldFiber, newFiber) => {
|
||||
if (ABORT) return;
|
||||
|
||||
if (Asyncify.state === Asyncify.State.Normal) {
|
||||
Asyncify.state = Asyncify.State.Unwinding;
|
||||
|
||||
var asyncifyData = oldFiber + 20;
|
||||
if (Fibers.shouldUseStableRewindTarget(oldFiber)) {
|
||||
if (!Fibers.rewindTargetByFiber[oldFiber]) {
|
||||
Fibers.rewindTargetByFiber[oldFiber] = Fibers.entryKeyByFiber[oldFiber] || Asyncify.exportCallStack[0];
|
||||
}
|
||||
|
||||
var rewindKey = Fibers.ensureRewindWrapper(oldFiber);
|
||||
Asyncify.setDataRewindFunc(asyncifyData, rewindKey);
|
||||
} else {
|
||||
Asyncify.setDataRewindFunc(asyncifyData);
|
||||
}
|
||||
Asyncify.currData = asyncifyData;
|
||||
_asyncify_start_unwind(asyncifyData);
|
||||
|
||||
var stackTop = stackSave();
|
||||
GROWABLE_HEAP_U32()[(((oldFiber) + (8)) >> 2)] = stackTop;
|
||||
|
||||
Fibers.nextFiber = newFiber;
|
||||
return;
|
||||
}
|
||||
|
||||
return __originalEmscriptenFiberSwap(oldFiber, newFiber);
|
||||
};
|
||||
|
||||
_emscripten_fiber_swap.isAsync = true;
|
||||
|
||||
// --- handleSleep save/restore for nested Asyncify ---
|
||||
// Asyncify.currData is a single-slot global. When a fiber swap runs inside an
|
||||
// EM_ASYNC_JS Promise await (e.g., wxDialog::ShowModal via startModal), the
|
||||
// fiber swap overwrites currData with the fiber's asyncify_data, losing the
|
||||
// sleep's own buffer. On Promise resolution, handleSleep's doRewind then uses
|
||||
// the wrong buffer and crashes with "index out of bounds" or "unreachable".
|
||||
//
|
||||
// Workaround: intercept Asyncify.allocateData to record which pointer belongs
|
||||
// to the active handleSleep; restore it to Asyncify.currData inside the wakeUp
|
||||
// callback before handleSleep proceeds to _asyncify_start_rewind + doRewind.
|
||||
// Documented upstream as Emscripten Issue #9153 (wontfix).
|
||||
if (typeof Asyncify.handleSleep === "function"
|
||||
&& typeof Asyncify.allocateData === "function"
|
||||
&& !Asyncify.__nestedHandleSleepInstalled) {
|
||||
// Stack of handleSleep contexts awaiting their allocateData association.
|
||||
// Each context learns its data pointer when allocateData fires.
|
||||
Asyncify.__pendingSleepContexts = [];
|
||||
|
||||
var __originalAllocateData = Asyncify.allocateData.bind(Asyncify);
|
||||
Asyncify.allocateData = function() {
|
||||
var ptr = __originalAllocateData();
|
||||
// Associate with the innermost pending handleSleep that hasn't been
|
||||
// linked yet. allocateData is called once per sleep's unwind, so the
|
||||
// innermost un-linked context is ours.
|
||||
for (var i = Asyncify.__pendingSleepContexts.length - 1; i >= 0; --i) {
|
||||
var ctx = Asyncify.__pendingSleepContexts[i];
|
||||
if (!ctx.capturedData) {
|
||||
ctx.capturedData = ptr;
|
||||
break;
|
||||
}
|
||||
}
|
||||
return ptr;
|
||||
};
|
||||
|
||||
var __originalHandleSleep = Asyncify.handleSleep.bind(Asyncify);
|
||||
Asyncify.handleSleep = function(startAsync) {
|
||||
var sleepCtx = { capturedData: null, cleanedUp: false };
|
||||
Asyncify.__pendingSleepContexts.push(sleepCtx);
|
||||
|
||||
var cleanup = function() {
|
||||
if (sleepCtx.cleanedUp) return;
|
||||
sleepCtx.cleanedUp = true;
|
||||
var idx = Asyncify.__pendingSleepContexts.indexOf(sleepCtx);
|
||||
if (idx !== -1) Asyncify.__pendingSleepContexts.splice(idx, 1);
|
||||
};
|
||||
|
||||
try {
|
||||
return __originalHandleSleep(function(wakeUp) {
|
||||
return startAsync(function(result) {
|
||||
// wakeUp runs from pure JS on Promise resolution. Fiber swaps
|
||||
// during the await may have overwritten Asyncify.currData with
|
||||
// a fiber's buffer pointer. Restore OUR buffer so handleSleep's
|
||||
// _asyncify_start_rewind and doRewind use the right data.
|
||||
if (sleepCtx.capturedData) {
|
||||
Asyncify.currData = sleepCtx.capturedData;
|
||||
}
|
||||
cleanup();
|
||||
return wakeUp(result);
|
||||
});
|
||||
});
|
||||
} catch (e) {
|
||||
cleanup();
|
||||
throw e;
|
||||
}
|
||||
};
|
||||
|
||||
Asyncify.__nestedHandleSleepInstalled = true;
|
||||
}
|
||||
}
|
||||
// === End fiber rewind stabilization ===
|
||||
FIBERPATCH
|
||||
|
||||
FIBER_INSERT_LINE=$(grep -n '^_emscripten_fiber_swap\.isAsync = true;$' "$JS_FILE" | head -1 | cut -d: -f1)
|
||||
|
||||
if [ -z "$FIBER_INSERT_LINE" ]; then
|
||||
echo "Error: Could not find _emscripten_fiber_swap.isAsync marker in $JS_FILE"
|
||||
rm "$FIBER_PATCH_FILE"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Injecting fiber rewind stabilization after line $FIBER_INSERT_LINE..."
|
||||
|
||||
head -n "$FIBER_INSERT_LINE" "$JS_FILE" > "${JS_FILE}.tmp"
|
||||
cat "$FIBER_PATCH_FILE" >> "${JS_FILE}.tmp"
|
||||
tail -n +$((FIBER_INSERT_LINE + 1)) "$JS_FILE" >> "${JS_FILE}.tmp"
|
||||
|
||||
mv "${JS_FILE}.tmp" "$JS_FILE"
|
||||
rm "$FIBER_PATCH_FILE"
|
||||
fi
|
||||
|
||||
# Fix empty callback arrow functions generated by Emscripten with pthreads
|
||||
# When pthreads is enabled, Emscripten generates empty {} for some direct-call paths
|
||||
# because it assumes they won't be used. But they ARE used in certain cases.
|
||||
|
|
@ -205,8 +399,123 @@ if [ "$COUNT_BEFORE" -gt 0 ]; then
|
|||
TOTAL_FIXED=$((TOTAL_FIXED + FIXED))
|
||||
fi
|
||||
|
||||
# Fix 6: Asyncify fiber entry callback (1 arg) - signature vi (void return)
|
||||
# Pattern in Fibers.finishContextSwitch: (a1 => {})(userData);
|
||||
# Function pointer is 'entryPoint'
|
||||
COUNT_BEFORE=$(grep -c '(a1 => {})(userData);' "$JS_FILE" || true)
|
||||
if [ "$COUNT_BEFORE" -gt 0 ]; then
|
||||
sed -i '' 's/(a1 => {})(userData);/dynCall_vi(entryPoint, userData);/g' "$JS_FILE"
|
||||
COUNT_AFTER=$(grep -c '(a1 => {})(userData);' "$JS_FILE" || true)
|
||||
FIXED=$((COUNT_BEFORE - COUNT_AFTER))
|
||||
echo " Fixed $FIXED fiber entry callback(s) (dynCall_vi)"
|
||||
TOTAL_FIXED=$((TOTAL_FIXED + FIXED))
|
||||
fi
|
||||
|
||||
if [ "$TOTAL_FIXED" -gt 0 ]; then
|
||||
echo "Total: Fixed $TOTAL_FIXED empty callback(s)"
|
||||
else
|
||||
echo "No empty callbacks found - nothing to fix"
|
||||
fi
|
||||
|
||||
# === Diagnostic logging for modal/asyncify/fiber interactions ===
|
||||
# This instruments key JS functions to trace "index out of bounds" crashes
|
||||
# that occur after EndModal during Asyncify rewind.
|
||||
echo "Injecting diagnostic logging for modal/asyncify interactions..."
|
||||
|
||||
DIAG_PATCH_FILE=$(mktemp)
|
||||
cat > "$DIAG_PATCH_FILE" << 'DIAGPATCH'
|
||||
|
||||
// === Diagnostic: modal/asyncify/fiber interaction logging ===
|
||||
(function() {
|
||||
var _diagTimerId = 0;
|
||||
var _diagModalActive = false;
|
||||
var _diagScheduledTimers = {};
|
||||
|
||||
// 1. Instrument _emscripten_async_call to log function pointer validity
|
||||
if (typeof _emscripten_async_call !== "undefined") {
|
||||
var __diag_orig_async_call = _emscripten_async_call;
|
||||
_emscripten_async_call = function(func, arg, millis) {
|
||||
var tableSize = wasmTable ? wasmTable.length : -1;
|
||||
var inBounds = func < tableSize;
|
||||
var id = ++_diagTimerId;
|
||||
console.warn('[DIAG_ASYNC_CALL] id=' + id + ' func=' + func + ' arg=' + arg +
|
||||
' millis=' + millis + ' tableSize=' + tableSize +
|
||||
' inBounds=' + inBounds + ' modalActive=' + _diagModalActive +
|
||||
' asyncifyState=' + (typeof Asyncify !== 'undefined' ? Asyncify.state : 'N/A'));
|
||||
if (!inBounds) {
|
||||
console.error('[DIAG_ASYNC_CALL] ALREADY OUT OF BOUNDS at schedule time! func=' + func);
|
||||
console.trace();
|
||||
}
|
||||
_diagScheduledTimers[id] = { func: func, arg: arg, millis: millis, scheduledDuringModal: _diagModalActive };
|
||||
return __diag_orig_async_call(func, arg, millis);
|
||||
};
|
||||
}
|
||||
|
||||
// 2. Instrument Asyncify.setDataRewindFunc to log rewind target changes
|
||||
if (typeof Asyncify !== "undefined" && Asyncify.setDataRewindFunc) {
|
||||
var __diag_orig_setRewind = Asyncify.setDataRewindFunc.bind(Asyncify);
|
||||
Asyncify.setDataRewindFunc = function(ptr, forcedBottomOfCallStack) {
|
||||
console.warn('[DIAG_REWIND_FUNC] ptr=' + ptr + ' forced=' + forcedBottomOfCallStack +
|
||||
' state=' + Asyncify.state + ' modalActive=' + _diagModalActive +
|
||||
' callStack=' + JSON.stringify(Asyncify.exportCallStack));
|
||||
return __diag_orig_setRewind(ptr, forcedBottomOfCallStack);
|
||||
};
|
||||
}
|
||||
|
||||
// 3. Track modal lifecycle via _endModal
|
||||
// startModal sets Module._endModal; we wrap that setter to detect modal start/end.
|
||||
var __diag_origDefineProperty = Object.defineProperty;
|
||||
var __diag_endModalSet = false;
|
||||
// Instead of defineProperty (complex), poll-wrap: patch the endModal assignment
|
||||
// by wrapping startModal's promise resolution. We detect modal start when
|
||||
// _endModal appears on Module, and modal end when it's deleted.
|
||||
if (typeof Module !== "undefined") {
|
||||
var __diag_checkInterval = setInterval(function() {
|
||||
if (Module._endModal && !__diag_endModalSet) {
|
||||
__diag_endModalSet = true;
|
||||
_diagModalActive = true;
|
||||
console.warn('[DIAG_MODAL] Modal started (Module._endModal appeared)' +
|
||||
' asyncifyState=' + (typeof Asyncify !== 'undefined' ? Asyncify.state : 'N/A'));
|
||||
|
||||
var __diag_origEndModal = Module._endModal;
|
||||
Module._endModal = function(code) {
|
||||
console.warn('[DIAG_MODAL] EndModal called with code=' + code +
|
||||
' asyncifyState=' + (typeof Asyncify !== 'undefined' ? Asyncify.state : 'N/A'));
|
||||
_diagModalActive = false;
|
||||
return __diag_origEndModal(code);
|
||||
};
|
||||
} else if (!Module._endModal && __diag_endModalSet) {
|
||||
__diag_endModalSet = false;
|
||||
console.warn('[DIAG_MODAL] Modal cleanup complete (Module._endModal deleted)' +
|
||||
' asyncifyState=' + (typeof Asyncify !== 'undefined' ? Asyncify.state : 'N/A'));
|
||||
}
|
||||
}, 50);
|
||||
}
|
||||
|
||||
// 4. Instrument dynCall_vi to catch the exact out-of-bounds call
|
||||
if (typeof dynCall_vi !== "undefined") {
|
||||
var __diag_orig_dynCall_vi = dynCall_vi;
|
||||
dynCall_vi = function(funcPtr, arg1) {
|
||||
var tableSize = wasmTable ? wasmTable.length : -1;
|
||||
if (funcPtr >= tableSize || funcPtr < 0) {
|
||||
console.error('[DIAG_DYNCALL_VI] OUT OF BOUNDS! funcPtr=' + funcPtr +
|
||||
' tableSize=' + tableSize + ' arg1=' + arg1 +
|
||||
' modalActive=' + _diagModalActive +
|
||||
' asyncifyState=' + (typeof Asyncify !== 'undefined' ? Asyncify.state : 'N/A'));
|
||||
console.trace();
|
||||
return; // Skip the call to prevent crash, let execution continue
|
||||
}
|
||||
return __diag_orig_dynCall_vi(funcPtr, arg1);
|
||||
};
|
||||
dynCall_vi.isAsync = true;
|
||||
}
|
||||
|
||||
console.warn('[DIAG] Modal/asyncify diagnostic logging installed');
|
||||
})();
|
||||
// === End diagnostic logging ===
|
||||
DIAGPATCH
|
||||
|
||||
# Append diagnostic patch at the end of the JS file
|
||||
cat "$DIAG_PATCH_FILE" >> "$JS_FILE"
|
||||
rm "$DIAG_PATCH_FILE"
|
||||
echo "Diagnostic logging injected"
|
||||
|
|
|
|||
Loading…
Reference in a new issue