From 9a39320117ebe441607c61183aaf0ff5ccc70c7f Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Sat, 11 Jul 2026 00:28:18 +1000 Subject: [PATCH] GOALS: add G-059 WSL test driving + G-060 QEMU cross-platform matrix (both proposed) - G-059: capability-probed WSL constraint for driving unix-side tests from Windows (multishell sh-payload execution, runtime.bash behaviour). Native-filesystem execution is a design constraint: per-test artifacts staged into a WSL-native tempdir with one-way copy-in/out, no repo checkout inside WSL for these cases, future full-suite mode via a separate native clone - the Windows checkout is never operated on via /mnt (DrvFs slowness + cross-boundary stat differences force git index re-hashing and fossil phantom changes; two VCS clients must not share a working tree). Acceptance includes verifying the Windows checkout git/fossil state is untouched by a WSL-gated run. - G-060: QEMU guests as strictly external tooling behind a hypervisor-agnostic guest-driving contract (provision/stage/run/collect, push-based - same pattern as G-059; guests never share a working tree with the host). GPL-safe posture recorded: external-process invocation only, no QEMU binaries or derived code in punkshell or punkbin, guest OS images not redistributed - fetched/built per machine. First target FreeBSD x86_64; config home deferred to the buildsuites toml direction. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- GOALS.md | 12 +++++++ goals/G-059-wsl-test-driving.md | 64 +++++++++++++++++++++++++++++++++ goals/G-060-qemu-test-matrix.md | 62 ++++++++++++++++++++++++++++++++ 3 files changed, 138 insertions(+) create mode 100644 goals/G-059-wsl-test-driving.md create mode 100644 goals/G-060-qemu-test-matrix.md diff --git a/GOALS.md b/GOALS.md index b8ae7590..56c9f551 100644 --- a/GOALS.md +++ b/GOALS.md @@ -397,3 +397,15 @@ Scope: src/vfs/_config/punk_main.tcl (boot auto_path/tm path filtering), src/mod Detail: goals/G-058-static-runtime-packages.md Goal: punkshell running on a runtime with statically-linked/builtin packages (tclsfe-x64: Thread/twapi/sqlite3/tdbc; punkbin runtimes' builtin Thread/tcllibc/vfs/vlerq; the expected shape of future zig-built static runtimes, G-005) keeps those packages resolvable in every interp and thread punkshell fabricates: boot captures the static baseline (info loaded entries with empty filename, plus their provided versions) before replacing package search paths, code interps and punkshell-created threads are seeded with ifneeded scripts mapping each static package to 'load {} ', and punk::packagepreference resolves static-vs-bundled by a documented version-aware policy instead of blindly loading a bundled dll over an already-provided static package. Acceptance: a punk91-style kit (tclsfe-x64 + punk9win.vfs, no thread dll in the vfs) boots to a working repl with no "can't find package Thread" - punk::console loads in the code interp, and package require Thread succeeds there and in a punkshell-created worker thread, resolving to the static version; in the same kit, twapi resolves per the documented policy (no repeat of the observed static-Twapi-masked-by-older-vfs-twapi-5.0.2 double load; a genuinely newer bundled copy remains reachable by that policy); dll-based kits (punk902z) boot and pass their existing shell test baseline unchanged, as does a plain tclsh dev launch; the seeding mechanism is generic - driven by the captured baseline, no runtime-specific package naming - and the boot-time static baseline is introspectable at the repl; the seeding/preference logic is covered by un-gated unit tests against simulated baselines (runnable under plain tclsh), while kit-boot integration tests are gated behind a capability-probed tcltest constraint (a built kit whose baseline shows static entries including Thread) that skips cleanly when no such kit is present; the runtimes used for verification (tclsfe-x64.exe at minimum) are added to the punkbin artifact repository under win32-x86_64 with sha1sums.txt updated, so the constraint is satisfiable on other machines via the existing runtime-retrieval path; the punk91 code-interp vfs/vfs::zip load failure is re-diagnosed after the fix and either resolved or recorded as a distinct issue/candidate goal. + +### G-059 [proposed] WSL detection and suitability probing for driving unix-side tests from Windows +Scope: src/tests/ (capability probe helpers + constraint-gated cases in existing suites, e.g the unix sh-payload execution test in modules/punk/mix/testsuites/scriptwrap/multishell.test and a runtime.bash behaviour test), src/tests/AGENTS.md (enablement notes) +Detail: goals/G-059-wsl-test-driving.md +Goal: test runs on a Windows dev machine can exercise selected unix-side behaviour through WSL when it is present AND suitable - a capability-probed tcltest constraint (not mere wsl.exe existence) verifies a launchable distro, required tools (bash, coreutils/sha1 tooling), and working one-way staging into the guest's NATIVE filesystem - with all guest-side execution happening on that native filesystem: per-test artifacts are staged into a WSL-native tempdir and results collected back, the Windows checkout is never operated on via the shared /mnt path (DrvFs is slow and cross-boundary stat differences make git re-hash its index and fossil see phantom changes), and any future full-suite mode uses a separate native clone rather than the shared working tree. +Acceptance: a documented probe helper yields a wsl_linux_available constraint whose checks are capability-based (distro launches and answers uname/tool probes; staging into a native tempdir works) and which cannot misfire on wsl.exe-present-but-unusable installs (no distro, WSL1 limitations, broken interop); the currently unix-gated multishell sh-payload execution test runs green via WSL on a suitable machine and still skips cleanly elsewhere; at least one runtime.bash behaviour test (active/use/run resolution against a fixture runtime folder) runs inside WSL - all such tests executing from a native-filesystem staging dir with the shared path used only for one-way copy-in/out; the Windows checkout's git and fossil state is untouched by a WSL-gated run (verifiable: git status/fossil changes identical before and after); suite results on a WSL-less machine are unchanged (skips, not failures); enablement/limitations and the staging pattern recorded in src/tests/AGENTS.md. + +### G-060 [proposed] QEMU-based cross-platform test matrix (arm's-length integration, GPL-safe posture) +Scope: build/test orchestration config and scripts (location to be settled with the buildsuites toml work - see G-005/buildsuites direction), goals/G-060-qemu-test-matrix.md (workflow + license posture), src/tests/ (any guest-driving hooks) +Detail: goals/G-060-qemu-test-matrix.md +Goal: comprehensive cross-platform verification (linux variants, FreeBSD/NetBSD/OpenBSD, arm architectures) is achievable from a single dev machine by driving QEMU guests as strictly external tooling - punkshell invokes qemu as a separate process with declarative per-guest config, and nothing in punkshell or its artifact repos bundles, links against, derives from or otherwise couples to QEMU (GPLv2) - so the project's BSD licensing posture is unaffected and QEMU remains a swappable convenience: the guest-driving contract is push-based artifact staging and result collection (the same pattern G-059 establishes for WSL - guests never share a working tree with the host) and must be satisfiable by real hardware, WSL or another hypervisor. +Acceptance: a documented, repeatable workflow provisions at least one non-Windows QEMU guest (e.g FreeBSD x86_64) that fetches a punkbin runtime and runs the source-tree suite inside the guest, driven from the Windows dev machine with results collected back; the guest-driving interface is hypervisor-agnostic (documented contract; QEMU is one provider); the license posture is recorded in the detail file (external-process invocation only, no QEMU binaries or derived code committed to punkshell or punkbin, guest OS images not redistributed by the project - fetched/built per machine); suite runs on machines without QEMU are unaffected. diff --git a/goals/G-059-wsl-test-driving.md b/goals/G-059-wsl-test-driving.md new file mode 100644 index 00000000..661338ca --- /dev/null +++ b/goals/G-059-wsl-test-driving.md @@ -0,0 +1,64 @@ +# G-059 WSL detection and suitability probing for driving unix-side tests from Windows + +Status: proposed +Scope: src/tests/ (capability probe helpers + constraint-gated cases in existing suites, e.g the unix sh-payload execution test in modules/punk/mix/testsuites/scriptwrap/multishell.test and a runtime.bash behaviour test), src/tests/AGENTS.md (enablement notes) +Acceptance: see GOALS.md index entry (canonical). + +## Context + +Development happens primarily on Windows, but punkshell carries genuinely unix-side +behaviour that currently goes unexercised there: the MULTISHELL polyglot's sh/bash +payload path (execution-tested only behind an isunix constraint - on Windows the shell +layer deliberately re-dispatches to the win32 nextshell), runtime.bash's fetch/use/run +logic (checksum parity added 2026-07-11, verified only by bash/zsh syntax check on +Windows), and eventually suite runs against linux punkbin runtimes. WSL can close part +of that gap without leaving the dev machine. + +## Approach + +### Native-filesystem execution is a design constraint, not an optimization + +User experience (recorded 2026-07-11): operating on the same repo checkout through the +WSL shared path (/mnt/c, DrvFs) is slow, and cross-boundary stat differences cause VCS +recalculation - git re-hashes its index (mtime/inode/dev mismatch between Windows and +WSL git; git status on /mnt/c is very slow generally) and fossil's mtime scan slows and +can see phantom exec-bit changes from DrvFs permission mapping. Two VCS clients with +incompatible stat caches must never share one working tree. + +Therefore: +- Tier 1 (this goal's test cases): NO repo checkout inside WSL at all. The inputs are + small (generated polyglot, runtime.bash, fixture runtime folders) - the harness + stages per-test artifacts into a WSL-native tempdir (mktemp -d on ext4) with a single + one-way copy-in over the shared path (bulk copy-in is fine; per-file stat traffic + during runs is what hurts), executes natively, and collects results back the same way. +- Tier 2 (future full-suite mode, out of scope here beyond being designed-for): a + SEPARATE native clone inside the distro (from gitea, or git fetch using the Windows + repo as a file:// remote for committed state). Working-tree-uncommitted testing would + be an explicit diff-overlay step, never shared-path operation. +- The Windows checkout's git and fossil state must be untouched by a WSL-gated run - + acceptance makes this verifiable (git status / fossil changes identical before/after). + +### Suitability probing + +The constraint (e.g wsl_linux_available) is capability-probed in the same spirit as +G-058's statickitavailable - wsl.exe existence proves nothing (installs can be +distro-less, WSL1-limited, or interop-broken). Probe: launch the default distro, answer +uname/tool probes (bash, coreutils sha1 tooling), create a native tempdir and round-trip +a staged file. Any probe failure -> skip cleanly (never fail the suite). + +### Consumers + +- multishell.test scriptwrap_multishell_exec_unix_sh (currently isunix-gated) gains a + WSL-driven sibling (or the constraint ORs in) running the generated polyglot under the + distro's sh/bash from a native staging dir. +- runtime.bash behaviour test: active/use/run resolution against a fixture runtime + folder inside WSL (network-dependent fetch cases separately env-gated - corp gitea + reachability from WSL2 varies with network setup). + +## Notes + +- Related: [[G-060]] adopts the same push-based staging/collection contract for QEMU + guests (which have no shared path to be tempted by); WSL is effectively the first, + cheapest provider of that guest-driving contract. +- WSL itself is a Windows feature (no licensing entanglement); distro choice is the + user's - the probe should not assume a specific distro, only capabilities. diff --git a/goals/G-060-qemu-test-matrix.md b/goals/G-060-qemu-test-matrix.md new file mode 100644 index 00000000..9fb895a1 --- /dev/null +++ b/goals/G-060-qemu-test-matrix.md @@ -0,0 +1,62 @@ +# G-060 QEMU-based cross-platform test matrix (arm's-length integration, GPL-safe posture) + +Status: proposed +Scope: build/test orchestration config and scripts (location to be settled with the buildsuites toml work - see G-005/buildsuites direction), goals/G-060-qemu-test-matrix.md (workflow + license posture), src/tests/ (any guest-driving hooks) +Acceptance: see GOALS.md index entry (canonical). + +## Context + +punkshell targets Windows primarily with Linux, macOS and FreeBSD as secondary targets, +and punkbin carries (or will carry) runtimes for linux-x86_64/linux-arm/macosx/freebsd +and more. Real verification on those platforms currently requires real machines. QEMU +can provide the matrix from one dev box - but QEMU is GPLv2, and punkshell is BSD: the +integration must be shaped so no licensing entanglement is possible. + +## License posture (the point of the "arm's-length" framing) + +- QEMU is invoked strictly as an EXTERNAL PROCESS - the same relationship a shell has + to any tool it execs. No linking, no derived code, no embedding. +- No QEMU binaries are committed to punkshell or punkbin; each machine installs QEMU + from its own distribution channel. (punkbin's binary policy is for punk runtimes and + build tools we produce/pin - not third-party GPL hypervisors.) +- Guest OS images are not redistributed by the project - fetched or built per machine + from upstream (BSDs/linux have their own licenses and trademark rules; staying out of + image distribution avoids that entire surface). +- Nothing in punkshell's runtime or module code may depend on QEMU's presence; only + optional test-orchestration tooling knows it exists, and even that through a + hypervisor-agnostic contract (below). If QEMU disappeared, the contract remains. + +## Approach + +### Guest-driving contract (hypervisor-agnostic) + +The punkshell-owned artifact is a small contract, not a QEMU integration: +- provision: bring up a named guest from declarative per-guest config (platform, arch, + how to boot, how to reach it - ssh being the obvious transport) +- stage: push artifacts (runtimes, test fixtures, or a source snapshot/clone) into the + guest's native filesystem - push-based staging and result collection, the same + pattern [[G-059]] establishes for WSL; guests never share a working tree with the + host (the DrvFs/VCS-recalculation lesson generalizes: no host checkout is ever + operated on from a guest) +- run: execute a command in the guest (e.g fetch a punkbin runtime, run + src/tests/runtests.tcl from a native clone) +- collect: bring results back to the host +QEMU is one provider of this contract; real hardware, WSL (G-059) or another hypervisor +must be able to satisfy it identically. Config lives with the buildsuites-era toml +direction rather than a new ad-hoc location. + +### Sequencing + +- G-059 (WSL) first proves the staging/run/collect pattern cheaply on linux. +- First QEMU guest target: FreeBSD x86_64 (a declared secondary target with no WSL + equivalent, and punkbin already anticipates freebsd folders). +- Guest provisioning recipes (image fetch, first-boot setup, ssh keys) documented per + guest in the detail/config - repeatable, not artisanal. + +## Notes + +- Related: [[G-005]] (zig builds - the same guests could eventually host build + verification for non-Windows artifacts), [[G-006]] (punkbin retrieval inside guests + exercises runtime.cmd's unix payload for real), [[G-059]] (contract sibling). +- Performance expectations should stay modest (emulated arm will be slow); the matrix + is for correctness verification, not speed benchmarking.