Browse Source

GOALS: add G-059 WSL test driving + G-060 QEMU cross-platform matrix (both proposed)

- G-059: capability-probed WSL constraint for driving unix-side tests from Windows (multishell sh-payload execution, runtime.bash behaviour). Native-filesystem execution is a design constraint: per-test artifacts staged into a WSL-native tempdir with one-way copy-in/out, no repo checkout inside WSL for these cases, future full-suite mode via a separate native clone - the Windows checkout is never operated on via /mnt (DrvFs slowness + cross-boundary stat differences force git index re-hashing and fossil phantom changes; two VCS clients must not share a working tree). Acceptance includes verifying the Windows checkout git/fossil state is untouched by a WSL-gated run.
- G-060: QEMU guests as strictly external tooling behind a hypervisor-agnostic guest-driving contract (provision/stage/run/collect, push-based - same pattern as G-059; guests never share a working tree with the host). GPL-safe posture recorded: external-process invocation only, no QEMU binaries or derived code in punkshell or punkbin, guest OS images not redistributed - fetched/built per machine. First target FreeBSD x86_64; config home deferred to the buildsuites toml direction.

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 4 days ago
parent
commit
9a39320117
  1. 12
      GOALS.md
  2. 64
      goals/G-059-wsl-test-driving.md
  3. 62
      goals/G-060-qemu-test-matrix.md

12
GOALS.md

@ -397,3 +397,15 @@ Scope: src/vfs/_config/punk_main.tcl (boot auto_path/tm path filtering), src/mod
Detail: goals/G-058-static-runtime-packages.md
Goal: punkshell running on a runtime with statically-linked/builtin packages (tclsfe-x64: Thread/twapi/sqlite3/tdbc; punkbin runtimes' builtin Thread/tcllibc/vfs/vlerq; the expected shape of future zig-built static runtimes, G-005) keeps those packages resolvable in every interp and thread punkshell fabricates: boot captures the static baseline (info loaded entries with empty filename, plus their provided versions) before replacing package search paths, code interps and punkshell-created threads are seeded with ifneeded scripts mapping each static package to 'load {} <pkg>', and punk::packagepreference resolves static-vs-bundled by a documented version-aware policy instead of blindly loading a bundled dll over an already-provided static package.
Acceptance: a punk91-style kit (tclsfe-x64 + punk9win.vfs, no thread dll in the vfs) boots to a working repl with no "can't find package Thread" - punk::console loads in the code interp, and package require Thread succeeds there and in a punkshell-created worker thread, resolving to the static version; in the same kit, twapi resolves per the documented policy (no repeat of the observed static-Twapi-masked-by-older-vfs-twapi-5.0.2 double load; a genuinely newer bundled copy remains reachable by that policy); dll-based kits (punk902z) boot and pass their existing shell test baseline unchanged, as does a plain tclsh dev launch; the seeding mechanism is generic - driven by the captured baseline, no runtime-specific package naming - and the boot-time static baseline is introspectable at the repl; the seeding/preference logic is covered by un-gated unit tests against simulated baselines (runnable under plain tclsh), while kit-boot integration tests are gated behind a capability-probed tcltest constraint (a built kit whose baseline shows static entries including Thread) that skips cleanly when no such kit is present; the runtimes used for verification (tclsfe-x64.exe at minimum) are added to the punkbin artifact repository under win32-x86_64 with sha1sums.txt updated, so the constraint is satisfiable on other machines via the existing runtime-retrieval path; the punk91 code-interp vfs/vfs::zip load failure is re-diagnosed after the fix and either resolved or recorded as a distinct issue/candidate goal.
### G-059 [proposed] WSL detection and suitability probing for driving unix-side tests from Windows
Scope: src/tests/ (capability probe helpers + constraint-gated cases in existing suites, e.g the unix sh-payload execution test in modules/punk/mix/testsuites/scriptwrap/multishell.test and a runtime.bash behaviour test), src/tests/AGENTS.md (enablement notes)
Detail: goals/G-059-wsl-test-driving.md
Goal: test runs on a Windows dev machine can exercise selected unix-side behaviour through WSL when it is present AND suitable - a capability-probed tcltest constraint (not mere wsl.exe existence) verifies a launchable distro, required tools (bash, coreutils/sha1 tooling), and working one-way staging into the guest's NATIVE filesystem - with all guest-side execution happening on that native filesystem: per-test artifacts are staged into a WSL-native tempdir and results collected back, the Windows checkout is never operated on via the shared /mnt path (DrvFs is slow and cross-boundary stat differences make git re-hash its index and fossil see phantom changes), and any future full-suite mode uses a separate native clone rather than the shared working tree.
Acceptance: a documented probe helper yields a wsl_linux_available constraint whose checks are capability-based (distro launches and answers uname/tool probes; staging into a native tempdir works) and which cannot misfire on wsl.exe-present-but-unusable installs (no distro, WSL1 limitations, broken interop); the currently unix-gated multishell sh-payload execution test runs green via WSL on a suitable machine and still skips cleanly elsewhere; at least one runtime.bash behaviour test (active/use/run resolution against a fixture runtime folder) runs inside WSL - all such tests executing from a native-filesystem staging dir with the shared path used only for one-way copy-in/out; the Windows checkout's git and fossil state is untouched by a WSL-gated run (verifiable: git status/fossil changes identical before and after); suite results on a WSL-less machine are unchanged (skips, not failures); enablement/limitations and the staging pattern recorded in src/tests/AGENTS.md.
### G-060 [proposed] QEMU-based cross-platform test matrix (arm's-length integration, GPL-safe posture)
Scope: build/test orchestration config and scripts (location to be settled with the buildsuites toml work - see G-005/buildsuites direction), goals/G-060-qemu-test-matrix.md (workflow + license posture), src/tests/ (any guest-driving hooks)
Detail: goals/G-060-qemu-test-matrix.md
Goal: comprehensive cross-platform verification (linux variants, FreeBSD/NetBSD/OpenBSD, arm architectures) is achievable from a single dev machine by driving QEMU guests as strictly external tooling - punkshell invokes qemu as a separate process with declarative per-guest config, and nothing in punkshell or its artifact repos bundles, links against, derives from or otherwise couples to QEMU (GPLv2) - so the project's BSD licensing posture is unaffected and QEMU remains a swappable convenience: the guest-driving contract is push-based artifact staging and result collection (the same pattern G-059 establishes for WSL - guests never share a working tree with the host) and must be satisfiable by real hardware, WSL or another hypervisor.
Acceptance: a documented, repeatable workflow provisions at least one non-Windows QEMU guest (e.g FreeBSD x86_64) that fetches a punkbin runtime and runs the source-tree suite inside the guest, driven from the Windows dev machine with results collected back; the guest-driving interface is hypervisor-agnostic (documented contract; QEMU is one provider); the license posture is recorded in the detail file (external-process invocation only, no QEMU binaries or derived code committed to punkshell or punkbin, guest OS images not redistributed by the project - fetched/built per machine); suite runs on machines without QEMU are unaffected.

64
goals/G-059-wsl-test-driving.md

@ -0,0 +1,64 @@
# G-059 WSL detection and suitability probing for driving unix-side tests from Windows
Status: proposed
Scope: src/tests/ (capability probe helpers + constraint-gated cases in existing suites, e.g the unix sh-payload execution test in modules/punk/mix/testsuites/scriptwrap/multishell.test and a runtime.bash behaviour test), src/tests/AGENTS.md (enablement notes)
Acceptance: see GOALS.md index entry (canonical).
## Context
Development happens primarily on Windows, but punkshell carries genuinely unix-side
behaviour that currently goes unexercised there: the MULTISHELL polyglot's sh/bash
payload path (execution-tested only behind an isunix constraint - on Windows the shell
layer deliberately re-dispatches to the win32 nextshell), runtime.bash's fetch/use/run
logic (checksum parity added 2026-07-11, verified only by bash/zsh syntax check on
Windows), and eventually suite runs against linux punkbin runtimes. WSL can close part
of that gap without leaving the dev machine.
## Approach
### Native-filesystem execution is a design constraint, not an optimization
User experience (recorded 2026-07-11): operating on the same repo checkout through the
WSL shared path (/mnt/c, DrvFs) is slow, and cross-boundary stat differences cause VCS
recalculation - git re-hashes its index (mtime/inode/dev mismatch between Windows and
WSL git; git status on /mnt/c is very slow generally) and fossil's mtime scan slows and
can see phantom exec-bit changes from DrvFs permission mapping. Two VCS clients with
incompatible stat caches must never share one working tree.
Therefore:
- Tier 1 (this goal's test cases): NO repo checkout inside WSL at all. The inputs are
small (generated polyglot, runtime.bash, fixture runtime folders) - the harness
stages per-test artifacts into a WSL-native tempdir (mktemp -d on ext4) with a single
one-way copy-in over the shared path (bulk copy-in is fine; per-file stat traffic
during runs is what hurts), executes natively, and collects results back the same way.
- Tier 2 (future full-suite mode, out of scope here beyond being designed-for): a
SEPARATE native clone inside the distro (from gitea, or git fetch using the Windows
repo as a file:// remote for committed state). Working-tree-uncommitted testing would
be an explicit diff-overlay step, never shared-path operation.
- The Windows checkout's git and fossil state must be untouched by a WSL-gated run -
acceptance makes this verifiable (git status / fossil changes identical before/after).
### Suitability probing
The constraint (e.g wsl_linux_available) is capability-probed in the same spirit as
G-058's statickitavailable - wsl.exe existence proves nothing (installs can be
distro-less, WSL1-limited, or interop-broken). Probe: launch the default distro, answer
uname/tool probes (bash, coreutils sha1 tooling), create a native tempdir and round-trip
a staged file. Any probe failure -> skip cleanly (never fail the suite).
### Consumers
- multishell.test scriptwrap_multishell_exec_unix_sh (currently isunix-gated) gains a
WSL-driven sibling (or the constraint ORs in) running the generated polyglot under the
distro's sh/bash from a native staging dir.
- runtime.bash behaviour test: active/use/run resolution against a fixture runtime
folder inside WSL (network-dependent fetch cases separately env-gated - corp gitea
reachability from WSL2 varies with network setup).
## Notes
- Related: [[G-060]] adopts the same push-based staging/collection contract for QEMU
guests (which have no shared path to be tempted by); WSL is effectively the first,
cheapest provider of that guest-driving contract.
- WSL itself is a Windows feature (no licensing entanglement); distro choice is the
user's - the probe should not assume a specific distro, only capabilities.

62
goals/G-060-qemu-test-matrix.md

@ -0,0 +1,62 @@
# G-060 QEMU-based cross-platform test matrix (arm's-length integration, GPL-safe posture)
Status: proposed
Scope: build/test orchestration config and scripts (location to be settled with the buildsuites toml work - see G-005/buildsuites direction), goals/G-060-qemu-test-matrix.md (workflow + license posture), src/tests/ (any guest-driving hooks)
Acceptance: see GOALS.md index entry (canonical).
## Context
punkshell targets Windows primarily with Linux, macOS and FreeBSD as secondary targets,
and punkbin carries (or will carry) runtimes for linux-x86_64/linux-arm/macosx/freebsd
and more. Real verification on those platforms currently requires real machines. QEMU
can provide the matrix from one dev box - but QEMU is GPLv2, and punkshell is BSD: the
integration must be shaped so no licensing entanglement is possible.
## License posture (the point of the "arm's-length" framing)
- QEMU is invoked strictly as an EXTERNAL PROCESS - the same relationship a shell has
to any tool it execs. No linking, no derived code, no embedding.
- No QEMU binaries are committed to punkshell or punkbin; each machine installs QEMU
from its own distribution channel. (punkbin's binary policy is for punk runtimes and
build tools we produce/pin - not third-party GPL hypervisors.)
- Guest OS images are not redistributed by the project - fetched or built per machine
from upstream (BSDs/linux have their own licenses and trademark rules; staying out of
image distribution avoids that entire surface).
- Nothing in punkshell's runtime or module code may depend on QEMU's presence; only
optional test-orchestration tooling knows it exists, and even that through a
hypervisor-agnostic contract (below). If QEMU disappeared, the contract remains.
## Approach
### Guest-driving contract (hypervisor-agnostic)
The punkshell-owned artifact is a small contract, not a QEMU integration:
- provision: bring up a named guest from declarative per-guest config (platform, arch,
how to boot, how to reach it - ssh being the obvious transport)
- stage: push artifacts (runtimes, test fixtures, or a source snapshot/clone) into the
guest's native filesystem - push-based staging and result collection, the same
pattern [[G-059]] establishes for WSL; guests never share a working tree with the
host (the DrvFs/VCS-recalculation lesson generalizes: no host checkout is ever
operated on from a guest)
- run: execute a command in the guest (e.g fetch a punkbin runtime, run
src/tests/runtests.tcl from a native clone)
- collect: bring results back to the host
QEMU is one provider of this contract; real hardware, WSL (G-059) or another hypervisor
must be able to satisfy it identically. Config lives with the buildsuites-era toml
direction rather than a new ad-hoc location.
### Sequencing
- G-059 (WSL) first proves the staging/run/collect pattern cheaply on linux.
- First QEMU guest target: FreeBSD x86_64 (a declared secondary target with no WSL
equivalent, and punkbin already anticipates freebsd folders).
- Guest provisioning recipes (image fetch, first-boot setup, ssh keys) documented per
guest in the detail/config - repeatable, not artisanal.
## Notes
- Related: [[G-005]] (zig builds - the same guests could eventually host build
verification for non-Windows artifacts), [[G-006]] (punkbin retrieval inside guests
exercises runtime.cmd's unix payload for real), [[G-059]] (contract sibling).
- Performance expectations should stay modest (emulated arm will be slow); the matrix
is for correctness verification, not speed benchmarking.
Loading…
Cancel
Save