punkshell/goals/archive/G-122-host-target-platform-split.md

G-122 make.tcl host/target platform split: bake by target, not host personality

Status: achieved 2026-07-26 Scope: src/make.tcl (target-platform derivation; store folder, exe suffixing, kit naming, presence checks and process-sweep tooling keyed by target; platform-canon inline copy; zipfs-less zip-type assembly fallback); src/modules/punk/platform-999999.0a1.0.tm (stable canon tags for cygwin-family hosts, help platforms doc); src/runtime/mapvfs.config (per-entry target platform via minimal compatible extension - format ownership coordinated with G-024); bin/runtime// store tiers (as consumed; fixture tier for tests); src/tests/shell/testsuites/punkexe/ (characterization) Goal: make.tcl's kit surfaces (bake/bakelist/bin) operate on an explicit target platform rather than the driving tclsh's personality: any capable host (native windows tclsh, msys/cygwin-runtime tclsh, a FreeBSD host running linux binaries) addresses the same per-target runtime store, artifact naming and process handling for the kits it reports/bakes; non-native-personality runtime tiers (e.g third-party mingw64 builds served from punkbin or an alternative artifact server) are representable as first-class mapping entries with their own store tier; and zip-type kit assembly no longer requires zipfs in the driving tcl. Acceptance: an msys/cygwin-runtime tclsh driving bakelist/bake on windows reports - and for a changed kit, bakes and deploys - the same win32-x86_64 kit set with identical names and store addressing as a native tclsh, with the pre-deploy process sweep using windows tooling for win32 targets regardless of host personality; punk::platform and the make.tcl inline canon map MSYS_NT/MINGW64_NT/CYGWIN_NT/UCRT64-class hosts to stable documented tags (MSYSTEM-variance documented, no windows build number in the tag) and 'help platforms' lists them; a mapvfs entry declaring a non-default target platform lists in bakelist with its own tier's store presence and bakes to a correctly suffixed artifact, verified against a fixture store tier (a real third-party runtime is not required); a zipfs-less driving tcl (8.6) assembles a zip-type kit via runtime split + punk::zip::mkzip + concatenation (archive-start-relative offsets) and the artifact boots on a zipfs-capable runtime; the linuxulator expectation (a linux-personality tclsh on FreeBSD 14/15 addresses the linux-x86_64 store unchanged) is recorded here with verification deferred to the planned linux-kit-on-FreeBSD trial; native-windows behaviour stays characterization-stable (existing punkexe maketcl tests pass unchanged).

Context

Field session 2026-07-26: the scoop msys2 tclsh8.6 driving src/make.tcl bakelist on windows showed every configured kit as runtime=missing (and deployed absent) from two compounding causes: the store folder derives from the HOST's platform canon (observed mingw64-x86_64 under a MINGW64_NT os string - note the tag follows the MSYSTEM environment, so the same binary can canonize as msys/ucrt64/mingw64 in different shells), and the platform=unix branch probes suffixless filenames - an assumption that is false for the entire cygwin family, whose executables are .exe files (the msys tclsh itself is tclsh8.6.exe). The interim self-diagnosis shipped in punkshell 0.24.1 (bakelist store-folder WARNING with the canon derivation + (FOLDER MISSING) header mark; bake's no-runtimes exit note) tells the user what happened; this goal makes the tooling actually cater for it.

The underlying defect is a conflation: make.tcl's scattered $::tcl_platform(platform) eq "windows" tests mix three separable concerns - host process semantics (cp vs cmd /c copy, ps/kill vs tasklist/taskkill, path personality), target artifact naming (.exe suffixing, kit/output names), and store addressing (bin/runtime/<canon>). An msys host pulls them apart. The process sweep is the sharpest edge: msys ps sees only msys-descendant processes, so a natively-launched kit exe is invisible to the pre-deploy sweep (taskkill remains exec-able from msys).

Direction (user, 2026-07-26): even if msys never becomes a zig buildsuite target, third-party-built runtime tiers (e.g an msys2-packaged tclsh) served from punkbin or an alternative artifact server must be supportable as kit runtimes - REPRESENTATION (mapping entry + store tier) belongs here; publication/fetch is the flagged follow-on goal.

FreeBSD linuxulator parallel (user): a linux-personality tclsh running on FreeBSD via the linux compat layer is expected to self-identify as linux and address bin/runtime/linux-x86_64 correctly - the planned linux-kit-on-FreeBSD-14/15 trial doubles as verification that target-keyed stores follow runtime personality, not host OS identity.

Zip offsets (user correction, 2026-07-26): tcl zipfs kits mount with archive-start-relative offsets - cat-style construction (split the original runtime first, then append a freshly written zip) is valid; zipfs mkimg's file-relative offset adjustment is one convention, not a mounting requirement. The existing zipcat kit type is the in-repo proof, and its machinery already has zipfs-less fallbacks.

Approach

• One target_platform derivation: default from the host canon, with cygwin-family hosts on windows defaulting to target win32-x86_64; per-entry mapping override for non-default tiers (a minimal compatible extension of the current vfsconfig entry - e.g a fourth element - whose schema home is G-024's toml conversion; the parsed-model seam (G-121, achieved - see goals/archive/G-121-bakelist-selective-bake.md) keeps every consumer format-agnostic while the format moves).
• Key store folder, suffixing, kit/artifact naming, presence checks and the process-sweep tool selection by TARGET; keep copy commands, path handling and prompt behaviour host-driven.
• punk::platform canon: explicit MSYS_NT/MINGW64_NT/CYGWIN_NT/UCRT64 mapping to stable documented tags with the MSYSTEM-variance caveat; the make.tcl inline canon copy stays in sync per its existing comment contract.
• zip-type fallback for zipfs-less driving tcls: reuse the zipcat machinery (extract_preamble split + punk::zip::mkzip + concatenation).
• Characterization against a fixture store tier so no real third-party runtime is required; the msys-hosted acceptance run uses the dev machine's msys tclsh where present (else recorded env-gated, per punkexe suite conventions).

Alternatives considered

• Per-site host patches (special-case each windows test) - rejected: the three-concern conflation just re-accumulates with each new host class.
• Full self-hosted msys platform lineage (msys-personality kits, accelerator builds, own test baselines) - out of scope: the third-party tier direction covers the need without a new first-party build lineage.
• Status quo (require a native tclsh to drive make.tcl) - rejected: field friction (the all-missing bakelist), and upcoming linux/FreeBSD hosts make host-personality assumptions untenable regardless of msys.

Notes

• Related: G-024 - owns the mapping format; the per-entry target platform is drafted as a minimal compatible extension and its schema home is that goal's toml conversion (make.tcl's own comments already pair the platform column with the toml change).
• Related: G-105 - cross-target zig builds populate first-party non-native store tiers; this goal is the consuming/addressing side.
• Related: G-114 - same punk::platform canon vocabulary, applied to per-platform tm module roots.
• Related: G-101 - 8.6 kit containers ride the same mapping/store surfaces; lister/bake stay kit-type-agnostic (per the archived G-121 decision).
• Related: G-057 - per-entry mapping extension precedent (per-vfs icon override rides the same entry shape).
• Related: G-023 - kit naming schemes; target-keyed naming must not collide with its version-named binaries direction.
• Follow-on: G-123 (drafted 2026-07-26, user-approved) - third-party runtime tier publication/fetch via punkbin or alternative servers, with schema-v2 provenance labeling and server-trust-keyed consent.
• Related: G-028 - the pre-deploy process sweep is the shared surface: this goal picks the sweep TOOLING by target (windows tooling for win32 targets from any host personality), that goal names the process actually holding a target the build cannot replace. A locker-report helper must make the same host-vs-target tooling choice. (activation-freshness survey, 2026-07-26)
• Related: G-060 - the eventual vehicle for exercising non-native target output: this goal makes cross-target kits addressable/bakeable from one host, a guest matrix is what verifies the artifacts actually boot (and is the natural home for the deferred linuxulator trial below). (activation-freshness survey, 2026-07-26)
• Out of scope: msys pty raw-mode console/repl behaviour - the cursor-row-1 field finding of 2026-07-26 is recorded in G-013's Notes.
• FreeBSD linuxulator expectation (recorded per the acceptance; verification deferred to the planned linux-kit-on-FreeBSD-14/15 trial, G-060 being the natural harness). A linux-personality tclsh running on FreeBSD via the linux compat layer self-identifies as linux, so platform_punk yields linux-x86_64, platform_target_default returns it unchanged (the cygwin-family exception is keyed on the os token being msys/cygwin, which a linuxulator tclsh never reports), and the bake addresses bin/runtime/linux-x86_64 with suffixless artifact names and posix (ps/kill) process tooling. Target-keyed stores follow RUNTIME personality, not host OS identity: that is the same rule that makes an msys host bake win32 kits, applied in the other direction. What the trial must actually check is that tcl_platform(os) reports Linux (not FreeBSD) under the compat layer - if it reported FreeBSD, the store addressed would be bin/runtime/freebsd-x86_64 and the expectation would need revisiting.
• Finding (2026-07-26, kept for whoever meets it next): a cygwin-family tclsh is a POSIX Tcl - C:/repo/.../src/make.tcl is not an absolute path to it, so file normalize prefixes the cwd and the run dies in an unrecognisable cascade (missing bootsupport, then no project root). make.tcl now diagnoses exactly that shape and prints the /c/repo/... spelling to use. This is caller-side spelling, deliberately not rewritten by make.tcl.
• Finding (2026-07-26): the zipfs-less EXTRACTION half depends on tcllib's zipfile::decode, a SYSTEM package that punkshell does not vendor. msys2's /usr/bin/tclsh8.6 has no tcllib, so a bake there cannot lift the runtime's own zip (its tcl_library) into the kit and the artifact will not boot; the bake now emits a recapped BUILD-WARNING saying so, and still deploys. With a tcllib on the path the same msys tclsh splits and decodes byte-identically to a native 8.6 host (verified: baseoffset 5267968, 1429659 zip bytes, 1784 entries from both). Candidate follow-ons flagged to the user rather than written as goals: (a) vendor or write a pure-Tcl zip decoder so the zipfs-less path is self-contained, (b) refuse to DEPLOY a kit whose extraction produced no tcl_library instead of warning past it.
• Finding (2026-07-26): the bin/punkshell902 name is now producible on windows (cross-target bake), and several punkexe suites use it as the second-choice PUNK_SHELL_TEST_EXE candidate after bin/punk902z.exe. On a windows machine with no windows kit but a cross-baked linux one, that fallback would select an ELF binary. Left as-is (pre-existing candidate-list logic, narrow state); PUNK_SHELL_TEST_EXE overrides it.

Progress

Achieved 2026-07-26 in one work unit. What landed and how each acceptance clause was verified:

• Target-platform derivation (::punkboot::lib::platform_* in src/make.tcl): platform_target_default (cygwin-family host -> win32), platform_store_tier (macosx-* -> macosx), platform_exe_suffix, platform_process_family, platform_name_problem, plus platform_normalize_os mirroring the module. Globals ::punkboot::host_platform / target_platform / host_windows separate the two axes; this_platform_generic stays the HOST key for bootsupport and vendorlib platform dirs.
• Mapping model keyed by target: mapvfs_parse takes the store ROOT plus a default target and resolves each entry's own tier/suffix (4th element; conflicting or malformed targets are config errors); mapvfs_kit_outputs records carry target/store_tier/runtime_dir. The bake path parses the mapping BEFORE runtime discovery, since the mapping is what says which tiers to look in; cross-target runtimes are picked up only when named.
• msys-host acceptance (msys2 /usr/bin/tclsh8.6, canon msys-x86_64, tcl_platform(platform) unix): bakelist output is identical to the native Tcl 9.0.3 run (line endings aside) - same 10 kits, same names, same runtimes: bin/runtime/win32-x86_64 addressing, punk91 detail resolving bin/runtime/win32-x86_64/tclsfe-x64.exe (present), target: win32-x86_64 (host default). Before the split every row read runtime=missing against a nonexistent mingw64-x86_64 store. A changed punk91 then baked and deployed from that host with a hidden punk91.exe (pid 10120) running: the sweep chose tasklist, FOUND it - msys ps reports 0 matches for the same process, measured side by side - killed it with taskkill and replaced bin/punk91.exe.
• msys2 argument mangling (found by that run): msys2 rewrites arguments that look like absolute posix paths when spawning a native windows program, so taskkill /PID 10120 arrived as taskkill C:/<msysroot>/PID 10120 and the kill failed the kit. Native windows command lines now go through ::punkboot::exec_nativeargs (MSYS2_ARG_CONV_EXCL=*, inert on other hosts); the re-run killed and deployed cleanly.
• Cross-target entry: the linux tclkit entry declares linux-x86_64 and is read from bin/runtime/linux-x86_64/ with no .exe. bakelist lists it with a target=linux-x86_64 note and (present) store state; make.tcl bake punkshell902 from a windows host produced a 24,200,032-byte ELF bin/punkshell902 (sdx wrap over the linux runtime), with the process sweep correctly skipped as not-visible-from-this-host. A real cross-target runtime was used - the acceptance's fixture-tier allowance was not needed.
• Zipfs-less zip assembly: native Tcl 8.6.17 (no zipfs) baked punk91 via raw-runtime split + punk::zip::mkzip + concatenation (5,267,968 + 44,242,534 = 49,510,502 bytes) and the artifact BOOTS - //zipfs:/app/tcl_library mounted, 4 zipfs mounts, punk::lib 0.5.1 loaded. The zipcat kit type now shares the same ::punkboot::assemble_zipcat_image helper.
• Latent defects the split exposed and fixed: the runtime capability probe tested a suffixless filename and so reported find-fail for every runtime on windows (it now probes the target's own tier and suffix, and skips cross-target runtimes as unrunnable); with real capabilities arriving, the extraction try-list was found to test capability PRESENCE rather than value, which would send a plain zipfs runtime down the kit path where a failed sdx mksplit replaces raw_runtime with the un-split original - a silently unbootable kit; BUILDCOPY naming now follows the runtime's own filename; and runtime map keys strip only .exe (file rootname would eat the last dotted segment of tclsh9.0.5).
• Characterization: new src/tests/shell/testsuites/punkexe/maketclplatform.test (5 tests: the check derivation line, default-target detail, the cross-target entry's tier/target/suffixless naming, and two cygwin-host tests that self-gate on discovering and probing a real msys/cygwin tclsh - the derivation line and full bakelist row-set parity against the native run). Native behaviour stayed characterization-stable: full suite 1112 tests, 1093 passed, 18 skipped, 1 failed = the documented exec-14.3 baseline; the pre-existing maketclbakelist/maketclcolour tests pass unchanged.
• Reported surface: make.tcl check prints platform (G-122): host=... target=... store=... exe-suffix=... process-tooling=...; bakelist rows carry target=<platform> for non-default targets and the detail block names the tier and whether the target is the host default or declared; the workflow text gained key note [K9] and a target-keyed store line. punkshell 0.24.1 -> 0.25.0.