diff --git a/GOALS.md b/GOALS.md index 0a90bdac..b5f85de4 100644 --- a/GOALS.md +++ b/GOALS.md @@ -406,3 +406,11 @@ Detail: goals/G-128-portable-pe-resource-stamping.md ### G-129 [proposed] Kit boot finds its payload wherever the archive mounted, not only at //zipfs:/app Scope: src/vfs/_config/punk_main.tcl and src/vfs/_config/project_main.tcl (vfs capability detection plus mount-point derivation for the kit's internal module/lib path assembly - both carry the same code today); src/runtime/mapvfs.config (an active kit entry per verification runtime); bin/runtime/win32-ix86/ (the two third-party 32-bit runtimes that verify it - read-only input); src/tests/shell/testsuites/punkexe/ (characterization of the derivation); src/AGENTS.md + bin/AGENTS.md (what a runtime must provide to be kit-wrappable) Detail: goals/G-129-kitboot-mountpoint-derivation.md + +### G-130 [proposed] 32-bit windows runtimes from the buildsuites, both Tcl generations +Scope: src/buildsuites/suite_tcl90/ and src/buildsuites/suite_tcl86/ (the win32-ix86 target invocation and any recipe or flag divergence 32-bit forces); src/buildsuites/ (target dimension as settled by G-105 - no per-target tree copies); bin/runtime/win32-ix86/ (output tier - already holds the third-party comparison runtimes); punkbin per-target artifact layout with G-103 metadata carrying the target +Detail: goals/G-130-32bit-windows-buildsuites.md + +### G-131 [proposed] One runtime, either payload: boot-time container autodetection (metakit or zipfs) +Scope: src/vfs/_config/punk_main.tcl and src/vfs/_config/project_main.tcl (container detection - extends the G-129 zipfs derivation to metakit; both files carry the same code); src/buildsuites/suite_tcl86/ and src/buildsuites/suite_tcl90/ (runtimes carrying both container capabilities, which the detection presupposes); src/make.tcl (the source/target kit-type mismatch check, which today assumes a runtime implies one container); src/runtime/mapvfs.config (kit type as a bake-time choice); bin/runtime/win32-ix86/ (zipfs specimens for the no-regression check); src/tests/shell/testsuites/punkexe/ (characterization); src/AGENTS.md + bin/AGENTS.md (what a runtime must provide) +Detail: goals/G-131-boot-payload-autodetection.md diff --git a/goals/G-129-kitboot-mountpoint-derivation.md b/goals/G-129-kitboot-mountpoint-derivation.md index 95f5ff44..1b9538cd 100644 --- a/goals/G-129-kitboot-mountpoint-derivation.md +++ b/goals/G-129-kitboot-mountpoint-derivation.md @@ -74,6 +74,25 @@ the source tree, not from a kit payload - which is what made the gap easy to mis ## Notes +- THIS GOAL IS THE LAST BLOCKER for baking a working kit on a third-party runtime, which + is why it stands alone rather than waiting on any larger container work. The chain is: + read the runtime's attached payload (G-124, achieved), do not let the boot-precondition + gate falsely refuse it (G-125, achieved - plus the punkboot::utils 0.4.0 third-convention + fix), and then have the baked kit FIND that payload at boot - this goal. The first two + are done; without the third a kit baked on such a runtime builds, deploys and starts, and + then resolves none of its own modules. + Measured 2026-07-27 by running punk_main.tcl's detection logic verbatim under + `bin/runtime/win32-ix86/tclsh8.6.10-luck-zip.exe`: + + has_zipfs (punk_main.tcl:290 verbatim): 0 + zipbase: NEVER COMPUTED - the //zipfs:/app branch is skipped entirely + kitpath branch: SKIPPED - kitpath unset + => internal modules/ + lib/ paths added: NONE + (while tcl::zipfs::mount really does report the archive mounted at the exe path) + + So the failure is not "mounted at an unexpected place and therefore mis-resolved" - it is + "concluded there is no zipfs at all, and skipped every internal-path branch". + - Related: G-031 - the closest structural overlap. That goal restructures this same file into a thin project-owned main plus a shared boot core; this changes logic that would live inside that core. Whichever lands second adopts the other's shape - the derivation diff --git a/goals/G-130-32bit-windows-buildsuites.md b/goals/G-130-32bit-windows-buildsuites.md new file mode 100644 index 00000000..9c1ba58e --- /dev/null +++ b/goals/G-130-32bit-windows-buildsuites.md @@ -0,0 +1,79 @@ +# G-130 32-bit windows runtimes from the buildsuites, both Tcl generations + +Status: proposed +Scope: src/buildsuites/suite_tcl90/ and src/buildsuites/suite_tcl86/ (the win32-ix86 target invocation and any recipe or flag divergence 32-bit forces); src/buildsuites/ (target dimension as settled by G-105 - no per-target tree copies); bin/runtime/win32-ix86/ (output tier - already holds the third-party comparison runtimes); punkbin per-target artifact layout with G-103 metadata carrying the target +Goal: punkshell's own buildsuites produce current 32-bit windows (win32-ix86) runtimes for BOTH Tcl generations from source with the pinned zig toolchain, so a machine that is old, firewalled or sandboxed - and that carries tcl8-only or tcl9-only binary modules which must be dropped in - can be served whichever generation its modules need from one supplier we control, instead of depending on third-party builds that no single source keeps current for both majors. +Acceptance: suite_tcl86 and suite_tcl90 each produce a win32-ix86 runtime through the documented target invocation, without disturbing the native build and without a per-target copy of the suite tree; each artifact runs on windows executing a script and resolving its bundled companions (thread and tclvfs for the 8.6 runtime, thread for the 9 runtime), with tcl_platform(pointerSize) 4 recorded; each loads a 32-bit BINARY EXTENSION - the motivating capability - verified against a real one rather than a stub (twapi 4.7.2 as shipped in bin/runtime/win32-ix86/tclsh8.6.10-luck-zip.exe is a 32-bit dll already in the tree for the 8.6 side; the 9-side extension is named at activation); each suite's core-testsuite run is recorded here against its own 32-bit baseline with every divergence from the corresponding 64-bit baseline dispositioned; any pointer-size or ABI assumption the 32-bit target exposes in the suite recipes is recorded; and the artifacts land in bin/runtime/win32-ix86/ and publish with G-103 metadata whose target names the 32-bit platform, so 'punk-runtime.cmd list -platform win32-ix86' shows them beside the third-party runtimes they are intended to replace. + +## Context + +The motivation is not architectural completeness, it is supply. Old, firewalled or +sandboxed systems - industrial ones especially - are where 32-bit still turns up, and +those are exactly the machines that cannot go and fetch what they need. Such a system may +carry binary Tcl modules built for only one major version, so the runtime has to match the +modules rather than the other way round. There is no single supplier keeping current 32-bit +windows Tcl available for BOTH majors, which means today the answer is assembled from +third-party builds of differing vintage and provenance. + +punkshell already builds Tcl from source with a pinned zig toolchain for both generations +(suite_tcl90, suite_tcl86 - G-096/G-098/G-099/G-100), and G-105 establishes the target as +an invocation dimension rather than a tree copy. 32-bit windows is the first target where +the change is one of BITNESS rather than OS, which is what makes it worth its own goal: it +exercises pointer-size and ABI assumptions that a linux-x86_64 cross does not. + +Two third-party 32-bit runtimes now sit in `bin/runtime/win32-ix86/` and serve as the +comparison baseline for what a suite-built pair would replace: +`tclsh8.6.10-luck-zip.exe` (Lean Undroidwish Construction Kit, Tcl 8.6.10, zipfs backport, +38 bundled packages) and `tclsh9.1b0-tclsfe.exe` (apnadkarni/tcl-sfe, Tcl 9.1b0). Between +them they demonstrate the shape is achievable; neither is ours, and neither is current for +the other's generation. + +## Approach + +- Treat 32-bit windows as one more value of G-105's target dimension, not a suite fork. + If it cannot be expressed that way, that is a finding about the dimension's design and + belongs back in G-105 before this goal proceeds. +- Build BOTH generations before declaring the goal met. A single-generation result does + not serve the motivating case, which is precisely that the machine's modules dictate the + generation. +- Gate on loading a real 32-bit binary extension. Producing a runtime that starts is not + the outcome; producing one that can accept the drop-in modules those systems carry is. +- Compare against the third-party specimens rather than only against a self-consistent + baseline - they are the status quo this replaces. + +## Alternatives considered + +- Keep sourcing 32-bit windows Tcl from third parties - rejected as the status quo the + goal exists to end: no single supplier is current for both majors, and provenance for a + locked-down deployment target is exactly where third-party binaries are weakest + (G-026/G-063 direction). +- Build 32-bit for one generation only (whichever is easier) - rejected: the motivating + systems are those whose existing binary modules force the choice of generation, so a + one-generation answer leaves half of them unserved. +- Defer until FreeBSD and other targets land - rejected on the developer's sequencing + (recorded 2026-07-27): this should slot in before or around the FreeBSD work, because + the bitness axis is likelier to surface recipe assumptions than another OS on the same + bitness, and finding those early is cheaper. + +## Notes + +- Depends on: G-105 - the target-as-invocation-dimension mechanism and the + punkshell-platform-dir vs zig-triple naming mapping. This goal is a consumer of that + decision and a second exercise of it after linux. +- Related: G-101 - which container a suite-built 8.6 runtime can be wrapped into is + decided there, not here. THIS GOAL PRODUCES RUNTIMES, NOT KITS: kit/container type is + deliberately out of Scope, so a 32-bit runtime that is not yet wrappable still satisfies + it. The two meet when G-101's decision lands. +- Related: G-123 - the artifacts publish through the runtime-tier mechanism, and + `punk-runtime.cmd list -platform win32-ix86` is where a user would see suite-built + runtimes appear beside the third-party ones. +- Related: G-129 - the third-party 8.6 specimen in this tier is what exposed punkshell's + kit-boot mount-point assumption. That is a boot-path concern and separate from producing + the runtime, but both goals use the same two specimens. +- Related: G-018 (superseded) - plain zig-built tclsh kits carrying no punk infrastructure; + the same "punkshell as a Tcl supplier, not only a shell" direction this goal serves. +- `punk::platform::local` already reports `win32-ix86` correctly from a 32-bit interpreter + (verified 2026-07-27 on both third-party runtimes), and `make.tcl check` derives + `host=win32-ix86 target=win32-ix86 store=bin/runtime/win32-ix86` when driven by one. No + platform-canon work is required; the tier name the store already uses is the canonical + one. diff --git a/goals/G-131-boot-payload-autodetection.md b/goals/G-131-boot-payload-autodetection.md new file mode 100644 index 00000000..db9fdf32 --- /dev/null +++ b/goals/G-131-boot-payload-autodetection.md @@ -0,0 +1,97 @@ +# G-131 One runtime, either payload: boot-time container autodetection + +Status: proposed +Scope: src/vfs/_config/punk_main.tcl and src/vfs/_config/project_main.tcl (container detection - extends the G-129 zipfs derivation to metakit; both files carry the same code); src/buildsuites/suite_tcl86/ and src/buildsuites/suite_tcl90/ (runtimes carrying both container capabilities, which the detection presupposes); src/make.tcl (the source/target kit-type mismatch check, which today assumes a runtime implies one container); src/runtime/mapvfs.config (kit type as a bake-time choice); bin/runtime/win32-ix86/ (zipfs specimens for the no-regression check); src/tests/shell/testsuites/punkexe/ (characterization); src/AGENTS.md + bin/AGENTS.md (what a runtime must provide) +Goal: one punk runtime per platform and Tcl generation boots from whichever payload container is attached to it - metakit or zipfs - so the container becomes a bake-time choice rather than a property fixed when the runtime is acquired, and the number of runtime deliverables is set by platform and generation alone. +Acceptance: ONE runtime binary, byte-identical between the two runs, boots and resolves its own payload from a metakit-wrapped kit AND from a zipfs-attached kit built from the same .vfs - internal modules/ and lib/ trees on tcl::tm::list and auto_path, and a package existing only inside the kit resolving, in both cases; container selection reads what is actually attached, with no compile-time or bake-time constant naming it, verified by wrapping that one runtime both ways from the same source tree; the zipfs arm is inherited from G-129 rather than re-specified - this goal's zipfs obligation is only that G-129's derivation still holds when reached through container detection, checked against the existing bin/runtime/win32-ix86 specimens; a runtime carrying a payload the boot cannot identify says so on stderr rather than continuing silently with no internal paths; punkshell's existing kits are unaffected, verified by comparing internal tcl::tm::list and auto_path before and after across at least one zipfs kit and one starkit-family kit; make.tcl's source/target kit-type mismatch check is reconciled with runtimes that legitimately support more than one container, rather than warning or skipping on them; cookfs and other exotics are explicitly out of scope with their existing branches untouched; and the deliverable reduction is recorded concretely - which runtimes collapse into which, and what the per-platform-per-generation count becomes. + +## Context + +Today a runtime's container is decided when the runtime is acquired, not when a kit is +baked: a tclkit-lineage runtime carries metakit and boots starkits, a zipfs runtime boots +zipfs images, and `src/runtime/mapvfs.config` pairs each kit with a runtime that can boot +the kit type it names. `make.tcl` encodes that coupling directly - it probes the source +runtime's capabilities and reports a `kit_type_mismatch` when they disagree with the +target kit type, warning and (under `-confirm 0`) skipping the kit. + +The consequence is a deliverable count multiplied by container. For each platform and each +Tcl generation punkshell must obtain, verify, publish and keep current a runtime PER +container it wants to be able to produce. That is also why a container decision like G-101's +feels weighty: picking the 8.6 container is picking what the 8.6 deliverable IS. + +If instead the boot detected what is attached, the container would become a bake-time +choice - the same runtime wrapped as a starkit or as a zipfs image depending on what the +bake wants that day. The deliverable count collapses to platform x generation, and G-101's +container decision stops being a fork in the road and becomes a default. + +The detection half is already partly built and about to be extended: `punk_main.tcl` +branches on zipfs, `::tcl::kitpath` (starkit) and a cookfs volume today, and G-129 replaces +the zipfs branch's compiled-in `//zipfs:/app` assumption with a derivation from the +runtime's own mount list. What is missing is (a) runtimes that carry both capabilities at +once and (b) a detection step that treats container type as a question rather than three +independently-written branches. + +## Approach + +- Extend, do not rewrite. G-129 lands the zipfs derivation and unblocks third-party kit + baking on its own; this goal generalises the same step to metakit and turns the branch + set into one detection decision. +- PREINIT-SHAPED FROM THE START. G-101's Tcl 9 survey found that the `proc tclInit` + override - which is what the tclkit lineage's boot.tcl IS - is dead in 9.1 + (tclInterp.c:619-622) and `tcl_libPath` is gone. A design that works on both generations + therefore has to hang off `Tcl_SetPreInitScript` rather than a tclInit override, and that + shapes the whole thing; discovering it late would mean rewriting the boot twice. + (UNVERIFIED there and still to check: whether 9.0.5, what suite_tcl90 actually builds, + exports `Tcl_SetPreInitScript`.) +- Detection reads evidence, not configuration: what is mounted, what the runtime can mount, + and what is actually attached to this executable. A bake-time or compile-time constant + naming the container defeats the purpose. +- Keep the blast radius honest: cookfs and other exotics stay out of scope and their + existing branches are left alone. Two containers is the whole win - it is what collapses + the deliverable count - and each further one costs capability weight in every runtime. + +## Alternatives considered + +- Compile-time container selection, as KitCreator does it (`KIT_STORAGE_MK4` / `_ZIP` / + `_CVFS` driving `$::tclKitStorage`, with boot.tcl branching on the result - see G-101's + Reference material) - rejected, and it is the closest prior art so the rejection matters: + a compile-time switch still produces one binary per container, which is exactly the + deliverable multiplication this goal exists to remove. It remains the right reference for + WHICH steps are container-specific. +- Keep one container per runtime and simply choose well once (the G-101 decision) - + rejected as the status quo: it leaves punkshell unable to bake a starkit and a zipfs kit + from one acquired runtime, and it keeps every future container question expensive. +- Detect at bake time and stamp the answer into the kit's main.tcl - rejected for the same + reason G-129 rejected stamping the mount point: it bakes a build-host assumption into the + artifact, and the running kit is in a strictly better position to look. + +## Notes + +- Depends on: G-129 - the zipfs mount-point derivation this generalises. That goal is + deliberately NOT superseded: it is small, it is the last blocker for baking a working kit + on a third-party runtime (see its Notes for the measured chain), and it delivers that on + its own well before a both-capabilities runtime exists. +- Depends on: G-101 - for the metakit arm to exist at all on a suite-built runtime. An 8.6 + binary carrying both needs metakit AND the androwish zipfs backport compiled in (both + surveyed there and both additive); a 9 binary carrying metakit needs the mk4tcl port that + goal costed at ~23 `Tcl_Size` call sites plus one `Tcl_ChannelType` and five + `Tcl_ObjType` initialisers. This goal runs ALONGSIDE G-101's decision - it does not wait + on it to be designed - but it cannot complete before that port work. +- Related: G-031 - the thin-main/shared-boot-core restructure. Container detection belongs + in the boot core, not in a per-project main; whichever lands second adopts the other's + shape. `project_main.tcl` carries the same code as `punk_main.tcl` today, which is why + both are in Scope until G-031 collapses them. +- Related: G-130 - its 32-bit runtimes are candidates to carry both capabilities, and the + deliverable-count argument is sharpest there: a new platform x two generations x two + containers is four artifacts to keep current, or two if this goal lands. +- Related: G-123 - fewer runtime deliverables is fewer artifacts to publish, checksum and + keep current in the punkbin tiers; the reduction this goal records is directly a + reduction in that goal's ongoing work. +- Related: G-024 - if the container becomes a bake-time choice, a mapvfs entry naming a kit + type stops implying a particular runtime, which is a simplification that lands on the toml + format work. +- The `make.tcl` `kit_type_mismatch` check is named in Scope because a runtime that + legitimately supports more than one container makes its premise false. It currently + probes `runtime_caps` for has_zipfs/has_starkit/has_cookfs and treats disagreement with + the target kit type as an error condition; with both-capability runtimes that disagreement + becomes the normal case.