From 8cbdf86fff744f9d16837af68ab924e3bc4e1d1d Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Mon, 27 Jul 2026 03:25:49 +1000 Subject: [PATCH] G-129 drafted (user-approved): kit boot finds its payload wherever the archive mounted Arising from the third-party 32-bit kit investigation. src/vfs/_config/punk_main.tcl decides whether the running kit has an attached zipfs image with two assumptions that held for every runtime punkshell had until now: it probes with 'info commands tcl::zipfs::root' and then looks for the mount at a compiled-in //zipfs:/app. Christian Werner's androwish / undroidwish backport gives Tcl 8.6 a working zipfs but no 'root', and mounts the executable's attached archive at the EXECUTABLE'S OWN PATH, so [info library] reads /tcl8.6. Against such a runtime punkshell concludes "no zipfs", neither the zipfs nor the kitpath branch runs, and the kit silently gets no internal module or lib paths at all - the failure surfaces later as packages that cannot be found. project_main.tcl carries the identical code, so the same defect ships into every generated project; both files are in Scope. Two third-party 32-bit runtimes now in the win32-ix86 tier bracket the problem and are named as the verification pair: tclsh8.6.10-luck-zip.exe (Lean Undroidwish Construction Kit - no 'root', no ::zipfs ensemble, mounts at the exe path, library at tcl8.6/, and no zipfs at all in child or thread interps) and tclsh9.1b0-tclsfe.exe (apnadkarni/tcl-sfe - the modern //zipfs:/app convention on the same architecture). The Acceptance requires both: the tclsfe kit proves the 32-bit target itself needs nothing special and that the derivation did not regress the ordinary case, the LUCK kit is the actual work. Both are untracked per G-004, so tests naming them self-gate on presence as the G-124 piperepl pin does; the LUCK build is reproducible from its .cfg build request. Approach settled in the draft: derive the mount point from tcl::zipfs::mount itself, whose entries pair mountpoint with mounted file - the entry naming this executable is this kit's archive wherever it landed. Detect zipfs by a command the backport also provides rather than by 'root'. Leave the kitpath and cookfs branches alone. Report on stderr when an archive is attached but no mount entry matches this executable, because silence is today's failure shape and is what made this expensive to diagnose. Overlap survey named G-031 (restructures this same file into a thin main plus shared boot core - whichever lands second adopts the other's shape, and the derivation belongs in the core), G-101 (its candidate container list already includes zipvfs-over-tclvfs boot, and the LUCK runtime is a ready-made specimen carrying zipfs, vfs 1.4.2 and starkit together), G-123 (these are punkbin candidates; kit-wrappability decides whether a runtime publishes as a usable tier), G-125 (achieved - its gate learned the tcl./ library convention so such a kit can now be BUILT; making it BOOT is this goal), plus G-114 and G-127 as deliberate non-overlaps - same file, different axes. Also records in G-024 a design direction from the developer (non-contract Notes): the toml move is where NAMED GROUPINGS of mapvfs entries should be considered, prompted by wanting verification kits active but obviously not shipped products. Grouping would also give bake and bakelist a name to select on beyond per-kit names, and the commented-out historical entries somewhere honest to live. Assisted-by: harness=claude; primary-model=claude-opus-5[1m]; api-location=anthropic.com --- GOALS.md | 4 + goals/G-024-mapvfs-toml.md | 8 ++ goals/G-129-kitboot-mountpoint-derivation.md | 111 +++++++++++++++++++ 3 files changed, 123 insertions(+) create mode 100644 goals/G-129-kitboot-mountpoint-derivation.md diff --git a/GOALS.md b/GOALS.md index baf7b7d4..0a90bdac 100644 --- a/GOALS.md +++ b/GOALS.md @@ -402,3 +402,7 @@ Detail: goals/G-127-crosstarget-vfs-bake.md ### G-128 [proposed] Portable PE resource stamping: a vendored zig tool that icons a finished kit from any host Scope: src/tools/punkres/ (vendored zig source - tool name settled in the work - with provenance and licence records); src/make.tcl (mechanism selection at the G-057 seam, sharing G-126's tool build step); bin/tools/zig* (pinned toolchain as consumed); bin/punkres.exe (untracked build output); src/tests/ (characterization - icon replaced, payload intact, idempotent, overlay-unsafe refusal); TEMP_REFERENCE/tcl-sfe (read-only reference for the resource structures) Detail: goals/G-128-portable-pe-resource-stamping.md + +### G-129 [proposed] Kit boot finds its payload wherever the archive mounted, not only at //zipfs:/app +Scope: src/vfs/_config/punk_main.tcl and src/vfs/_config/project_main.tcl (vfs capability detection plus mount-point derivation for the kit's internal module/lib path assembly - both carry the same code today); src/runtime/mapvfs.config (an active kit entry per verification runtime); bin/runtime/win32-ix86/ (the two third-party 32-bit runtimes that verify it - read-only input); src/tests/shell/testsuites/punkexe/ (characterization of the derivation); src/AGENTS.md + bin/AGENTS.md (what a runtime must provide to be kit-wrappable) +Detail: goals/G-129-kitboot-mountpoint-derivation.md diff --git a/goals/G-024-mapvfs-toml.md b/goals/G-024-mapvfs-toml.md index 939aca8f..1d9be3c1 100644 --- a/goals/G-024-mapvfs-toml.md +++ b/goals/G-024-mapvfs-toml.md @@ -59,6 +59,14 @@ with the G-014 direction of toml + tomlish for declared configuration. - The commented-out historical entries in mapvfs.config (broken-runtime notes, test cases) are documentation of past pitfalls - carry the useful ones into toml comments or this file rather than losing them. +- Design direction from the developer, recorded 2026-07-27: the toml move is where NAMED + GROUPINGS of entries should be considered, since toml gives the structure the current + line format cannot. The prompt for it was verification kits - two third-party 32-bit + runtimes were added to the win32-ix86 tier to verify G-129's boot-mount derivation, and + they want to sit as active entries while being obviously a verification set rather than + shipped products. Grouping would also give `make.tcl bake` and `bakelist` a name to + select on beyond per-kit names (the G-121 selective-bake surface takes kit names today), + and would give the commented-out historical entries above somewhere honest to live. - Related: G-023 (first scheme consumer), G-014 (toml+tomlish direction and the tomlish punk::args-documentation precondition - that precondition gates punk::config consumption; build-tooling use here can proceed with tomlish diff --git a/goals/G-129-kitboot-mountpoint-derivation.md b/goals/G-129-kitboot-mountpoint-derivation.md new file mode 100644 index 00000000..95f5ff44 --- /dev/null +++ b/goals/G-129-kitboot-mountpoint-derivation.md @@ -0,0 +1,111 @@ +# G-129 Kit boot finds its payload wherever the archive mounted + +Status: proposed +Scope: src/vfs/_config/punk_main.tcl and src/vfs/_config/project_main.tcl (vfs capability detection plus mount-point derivation for the kit's internal module/lib path assembly - both carry the same code today); src/runtime/mapvfs.config (an active kit entry per verification runtime); bin/runtime/win32-ix86/ (the two third-party 32-bit runtimes that verify it - read-only input); src/tests/shell/testsuites/punkexe/ (characterization of the derivation); src/AGENTS.md + bin/AGENTS.md (what a runtime must provide to be kit-wrappable) +Goal: a punk kit boots from wherever its attached archive actually mounted, so a runtime whose zipfs mounts at the executable's own path rather than at //zipfs:/app - as the androwish/undroidwish 8.6 backport does - yields a kit that finds its own modules and libs. The boot derives the mount point from the runtime's own mount list and detects zipfs by a command every generation provides, instead of keying on tcl::zipfs::root (a 9-era command absent from the 8.6 backport) and a compiled-in //zipfs:/app. +Acceptance: a kit baked from a runtime that mounts its attached archive at the executable's own path boots and resolves its own payload with no external package paths in play - the kit's internal modules/ and lib/ trees appear in tcl::tm::list and auto_path and a package that exists only inside the kit resolves - with bin/runtime/win32-ix86/tclsh8.6.10-luck-zip.exe recorded here as that verification runtime; a kit baked from bin/runtime/win32-ix86/tclsh9.1b0-tclsfe.exe (32-bit, modern //zipfs:/app convention) boots the same way, proving the 32-bit target itself needs nothing special and that the derivation did not regress the ordinary case; punkshell's existing 64-bit kits are unaffected, verified by comparing each kit's internal tcl::tm::list and auto_path entries before and after the change across at least one zipfs kit and one starkit-family kit (identical); a runtime that has an attached archive the boot cannot locate says so on stderr rather than continuing silently with no internal paths, which is today's failure shape; and the runtime requirements for kit-wrapping - which zipfs commands and mount conventions are supported - are documented in bin/AGENTS.md. + +## Context + +`src/vfs/_config/punk_main.tcl` decides whether the running kit has an attached zipfs +image, and where it is, with two assumptions that were true for every runtime punkshell +had until now: + + set has_zipfs [expr {[info commands tcl::zipfs::root] ne ""}] + ... + set zipbase [file join [tcl::zipfs::root] app] + if {"$zipbase" in [tcl::zipfs::mount]} { ...add kit modules/ lib/ paths... } + +`tcl::zipfs::root` arrived with the 8.7/9 zipfs. Christian Werner's androwish / +undroidwish backport gives Tcl 8.6 a working zipfs - `mount unmount info list exists +mkzip mkimg mkkey lmkzip lmkimg` - but no `root`, and it mounts the executable's attached +archive at the EXECUTABLE'S OWN PATH rather than at `//zipfs:/app`, so `[info library]` +reads `/tcl8.6`. Against such a runtime punkshell concludes "no zipfs", the +`//zipfs:/app` branch never runs, `::tcl::kitpath` is unset so the starkit branch does not +run either, and the kit silently gets no internal module or lib paths at all. Nothing is +reported; the failure surfaces later as packages that cannot be found. + +Measured 2026-07-27 on two third-party 32-bit windows runtimes now in the store, which +bracket the problem: + +| | tclsh8.6.10-luck-zip.exe | tclsh9.1b0-tclsfe.exe | +|---|---|---| +| provenance | Lean Undroidwish Construction Kit | apnadkarni/tcl-sfe releases | +| Tcl | 8.6.10 | 9.1b0 | +| `tcl::zipfs::root` | absent | `//zipfs:/` | +| `::zipfs` ensemble | absent | present | +| mount point | `` | `//zipfs:/app` | +| `[info library]` | `/tcl8.6` | `//zipfs:/app/tcl_library` | +| zipfs in child/thread interps | none | present | +| attached archive | archive-relative, 1757 members | archive-relative, 892 members | +| `vfs::zip` / `starkit` | 1.0.4 / 1.3.3 | absent | + +The LUCK runtime already drives `make.tcl shell` fine - in that mode punkshell loads from +the source tree, not from a kit payload - which is what made the gap easy to miss. + +## Approach + +- Detect zipfs by a command the backport also provides (`tcl::zipfs::mount` is the natural + one, and is what the mount list comes from anyway) rather than by `tcl::zipfs::root`. +- Derive the mount point from `tcl::zipfs::mount` itself. The mount list pairs mountpoint + with the file mounted there, so the entry whose file is `[info nameofexecutable]` names + this kit's own archive wherever it landed - `//zipfs:/app` on a modern runtime, the exe + path on the backport. Prefer that over any compiled-in constant; the existing + `//zipfs:/app` remains the expected answer, not a hardcoded requirement. +- Keep `::tcl::kitpath` (starkit/tclkit) and the cookfs branch exactly as they are - this + is about the zipfs branch only. +- When an archive is attached but no mount entry can be matched to this executable, say so + on stderr. Silence is the current failure shape and it is what made this expensive to + diagnose. + +## Alternatives considered + +- Special-case the androwish backport by name or by Tcl version - rejected: the mount list + already carries the ground truth, and a version test would need revisiting for every + runtime family that does something reasonable but different. +- Require runtimes to mount at `//zipfs:/app` and declare the backport unsupported - + rejected: the mount point is chosen by the runtime's own boot code (TclZipfs_AppHook), + not by the image we assemble, so this is not something a bake could enforce; and 8.6 is + a supported target (src/AGENTS.md) where this backport is one of the few working + zipfs options. +- Have the bake stamp the expected mount point into the kit's main.tcl - rejected: it + would bake a build-host assumption into the artifact, and the running kit can simply + look. + +## Notes + +- Related: G-031 - the closest structural overlap. That goal restructures this same file + into a thin project-owned main plus a shared boot core; this changes logic that would + live inside that core. Whichever lands second adopts the other's shape - the derivation + belongs in the boot core, not in a per-project main. +- Related: G-101 - its candidate container list already includes "zipvfs-over-tclvfs + boot". A working 8.6 zipfs backport that mounts at the exe path is direct evidence for + that assessment, and the LUCK runtime is a ready-made specimen: it carries zipfs, vfs + 1.4.2 and starkit 1.3.3 together. +- Related: G-123 - these two runtimes are third-party punkbin candidates, and whether a + runtime is kit-wrappable is what decides if it publishes as a usable runtime tier rather + than only as a curiosity. The bin/AGENTS.md requirements this goal writes are the + criterion that publication would cite. +- Related: G-125 (achieved 2026-07-27 - see goals/archive/G-125-unbootable-kit-deploy-gate.md) + - the bake's boot-precondition gate learned the LUCK runtime's `tcl./` library + convention (punkboot::utils 0.4.0) so it no longer refuses such a kit as unbootable. That + fix only made the kit BUILDABLE; making it BOOT is this goal. +- Deliberate non-overlap: G-114 (per-platform tm module roots) and G-127 (cross-target bake + output locations) touch punk_main.tcl and the kit surfaces too, but on the tm-roots and + output-location axes - neither reads or writes the vfs-detection logic. +- `src/vfs/_config/project_main.tcl` carries the identical probe and mount derivation, so + the same defect ships into every generated project. Both files are in Scope; if G-031 + lands first this collapses to one place. +- Both verification runtimes are untracked (`bin/runtime/` is not in VCS, per G-004), so any + test that names them self-gates on their presence - the same pattern + `zipreader_piperepl_runtime` uses for `tclsh90b4_piperepl.exe` in + src/tests/modules/punk/zip/testsuites/zip/zipreader.test. The LUCK build is reproducible + from the service: `bin/runtime/win32-ix86/tclsh8.6.10-luck-zip.cfg` is the base64 build + request (output name, the 38 selected packages, apptext, cmdline) and `LUCK.url` names + the generator. +- The 32-bit dimension is wider than these two runtimes: the developer's standing interest + (recorded 2026-07-27) is 32-bit zig builds for punk, because old firewalled or sandboxed + systems - industrial ones especially - may carry tcl8-only or tcl9-only binary modules + that have to be dropped in, and no single source ships current Tcl for both major + versions. That is separate work; this goal only ensures a 32-bit third-party runtime can + carry a kit, and leaves two verification specimens in the win32-ix86 tier for it.