punkshell/goals/G-096-zig-buildsuite-piperepl.md

G-096 Tracked zig tcl buildsuite (suite_tcl90) with recovered TCLSH_PIPEREPL kit-repl capability

Status: active Scope: src/buildsuites/ (suite_tcl90 new; samplesuite1 sketch reconciliation; VCS ignore flip), src/buildsuites/_build/ (untracked build/source area via existing _build globs), .gitignore + .fossil-settings/ignore-glob, src/scriptapps/bin/ (getzig scriptset rename), bin/ (regenerated punk-getzig wrapper twins, old names retired, AGENTS.md naming policy), bin/tools/ (untracked zig toolchains, existing), TEMP_REFERENCE/2024zig (read-only recovery origin) Goal: src/buildsuites gains its first real, VCS-tracked suite (suite_tcl90) that reproducibly builds the Tcl 9.0 windows runtime from clean sources using only project-managed tooling - a punk-getzig wrapper fetching a pinned zig into untracked bin/tools, with all transient checkouts and outputs under untracked src/buildsuites/_build - and the 2024 TCLSH_PIPEREPL tclMain patch is recovered into the suite in tracked form and rebased onto current Tcl 9 sources, so a suite-built runtime can return to a standard interactive repl after a kit-style main.tcl completes with unconsumed piped input buffered rather than eval'd, while stock behaviour is preserved whenever the env gate is unset. Acceptance: from a clean checkout plus a punk-getzig-fetched pinned zig, the suite builds tclsh90s/tclsh90szip from a fresh core-9-0-branch checkout under src/buildsuites/_build with recipe-generated tclUuid.h and tclsh.exe.manifest (no hand-placed files) and the result reports the expected patchlevel; the recovered patch set (piperepl plus the companion tclZipfs/tclCmdIL patches and a notes doc) is tracked in the suite; a piperepl-patched build with TCLSH_PIPEREPL unset behaves identically to the unpatched build on a tty/piped/script-arg matrix, and with TCLSH_PIPEREPL=1 a script setting ::tclsh(dorepl) 1 lands in a live interactive console repl after piped-input eof with the unconsumed input available in ::tclsh(inputbuffer); src/buildsuites is tracked in both VCSes with only _build content ignored and the samplesuite1 vendorbuild-prefix residue reconciled (vendorbuild concept retired); bin/AGENTS.md documents the utility naming policy (punk- prefix for punkshell-own utilities; exceptions: external-tool wrappers such as dtplite/sdx/kettle, and getpunk as the intended future cross-platform entry point) and the getzig scriptset and its bin wrapper twins (getzig.cmd/getzig.ps1) are renamed to punk-getzig with the old names removed and references updated.

Context

• src/buildsuites (2026-07-10 sketch: samplesuite1 with a Tcl-script-style download_and_build.config carrying SOURCEDOWNLOAD/CONFIGSTART records) is blanket-ignored in BOTH VCSes today (.gitignore /src/buildsuites, .fossil-settings/ignore-glob src/buildsuites) - deliberately untracked while the sketch was non-final. User decision 2026-07-20: buildsuites is the home for build source-pull/recipe declarations ("it was essentially for this purpose"); this goal makes suite content tracked.
• Untracked build/source area: src/buildsuites/_build (user 2026-07-20: "seems neater to keep it all under src/buildsuites/_build"). Already covered by the existing _build ignore globs in both VCSes (git **/_build/; fossil _build + */_build) - no new ignore rules needed for it. The sketch's <PROJECTDIR>/vendorbuild/<suite> buildprefix and the root /vendorbuilds/ ignore entries are retired in its favour.
• The TCLSH_PIPEREPL patch (Aug 2024; c:/buildtcl/2024zig/build_tcl90/patch/ tclMain.c.piperepl.patch + tclMain_piperepl.c, snapshot under git-ignored TEMP_REFERENCE/2024zig) enables returning to a standard C-level tcl repl after a kit main.tcl - not possible with unmodified sources (user 2026-07-20). Semantics: gated by the TCLSH_PIPEREPL env var (unset/0 = stock behaviour); publishes ::tclsh(istty) plus linked ::tclsh(dorepl)/::tclsh(evalinput); unconsumed piped input accumulates in ::tclsh(inputbuffer) rather than being eval'd (unless evalinput is set - protects arbitrary piped data from accidental eval); on stdin eof with dorepl set, stdin reopens from the console (CONIN$ on windows, /dev/tty elsewhere) giving a live standard repl. Companion patches in the same folder: tclZipfs.patch, tclCmdIL.patch (unexamined). None of these files are VCS-tracked anywhere - recovery into the suite is the durability fix.
• Recipe baseline: the validated build provenance in goals/archive/G-076-tcl9-deadconsole-fix-adoption.md (2026-07-20 entries; G-076 achieved 2026-07-20) - build90.zig / build905.zig with zig 0.14.0-dev.2074, the fresh-checkout generated-file preconditions (tclUuid.h from manifest.uuid; tclsh.exe.manifest from its .in with @TCL_WIN_VERSION@/@MACHINE@), and the 9.0.5+ mingw_unicode_entry_point requirement (upstream removed TCL_BROKEN_MAINARGS from win/tclAppInit.c).
• getzig today: scriptset src/scriptapps/bin/getzig.{bash,ps1} + getzig_wrap.toml, wrapped by punk::mix::commandset::scriptwrap::multishell into bin/getzig.cmd (tracked) with a bin/getzig.ps1 twin. The twin is a deliberate polyglot-launchability artifact - windows users may start from cmd.exe or powershell - not residue. Fetches zig into bin/tools (untracked via the existing bin/* globs) with minisign signature material, sourcing ziglang.org, community-mirrors.txt and the punkbin gitea mirror. bin/tools currently holds zig 0.15.1/0.15.2/0.16.0-dev toolchains; the known-good recipe used a manual C:/zig install (0.14.0-dev.2074).
• Toolchain policy (user 2026-07-20, recorded in G-076 - achieved, archived - and agent memory): zig is the only build mechanism approved to become part of the punkshell project; msys/mingw and MS toolchains are test/validation only (MS last resort).

Approach

• suite_tcl90 tracked layout: the zig recipe (build905.zig lineage, parameterized for suite paths), patches/ (piperepl + tclZipfs + tclCmdIL + a notes doc carrying the semantics summary and rebase history), suite README. Transient content under src/buildsuites/_build// (source checkouts, build prefixes/outputs); fossil clones stay in the shared ~/.fossils.
• The recipe generates tclUuid.h and tclsh.exe.manifest itself (wrapfiletofile-style steps) so fresh checkouts build with no hand-placed files.
• Pin an exact zig version retrievable via punk-getzig (punkbin can host the pinned zip); migrating the recipe from 0.14.0-dev.2074 to the pinned version is in-scope (zig build API churn), with the validated exact-combo build as fallback per the duplicate-first policy.
• Naming policy: punk- (hyphenated) prefix approved by user 2026-07-20 over bare punk concatenation - readable, groups in listings, distinct from the punk*.exe shell-kit namespace. Applied here to getzig (touched anyway); the sweep of the remaining punkshell-own utilities is G-097.
• Suite config format stays pragmatic (zig/script files); toml-based suite declarations remain a G-005-era decision - buildsuites structure beyond what this goal claims is still non-final.

Alternatives considered

• <PROJECTDIR>/vendorbuild/<suite> as the untracked build area (samplesuite1 sketch + /vendorbuilds/ ignores) - retired 2026-07-20 in favour of src/buildsuites/_build: one area adjacent to the suites, already ignore-covered in both VCSes.
• Upstreaming the piperepl capability instead of carrying a patch - not precluded as a future path; tracked in-suite recovery is the durability baseline either way and is required regardless while nothing upstream provides it.
• Folding the bin-wide utility rename sweep into this goal - split to G-097 to keep this acceptance verifiable and let the sweep wait for a suitable time (user 2026-07-20).

Notes

• Related goals: G-005 (zig build infrastructure - this suite is its first concrete increment), G-018 (zig plain tclsh kits), G-006 (prebuilt artifact download consent; punkbin mirror overlap), G-004 (no committed binaries - untracked bin/tools and _build align), G-076 (achieved 2026-07-20 - verification-runtime provenance this recipe formalizes; goals/archive/G-076-tcl9-deadconsole-fix-adoption.md), G-038 (punkshell-level piped session continuity - the punk-repl analogue of piperepl), G-060 (qemu cross-platform matrix - see G-097 notes on the windows-sandbox experiments), G-089 (scriptlib in kits; the scriptapps/bin deployment surface).
• getpunk (naming-policy exception): intended future single-download cross-platform starting point - run in a new folder, the folder becomes the repo automatically; needs smarts to pull the repo and then update itself safely without snagging on the now-dirty repo (user 2026-07-20). The polyglot .cmd/.ps1 twin mechanism serves that cmd.exe-or-powershell first-contact story and may need review in that light; candidate goal when picked up.
• The G-076 kill-procedure runtime (zig-out905 under c:/buildtcl/2024zig) predates this suite; once suite_tcl90 reproduces it, the ad-hoc c:/buildtcl/2024zig builds become historical reference only.

Progress

2026-07-20 increment 1: activated; buildsuites tracked; suite_tcl90 builds 9.0.5 end-to-end

• Goal set active (user 2026-07-20). VCS tracking flip: the blanket /src/buildsuites ignores removed from .gitignore and .fossil-settings/ignore-glob; the retired vendorbuilds ignores removed with them; src/buildsuites/_build confirmed ignored via the existing **/_build/ (git) and */_build (fossil) globs.
• samplesuite1 sketch reconciled: marked SKETCH ONLY with the toolchain-policy note; buildprefix moved from /vendorbuild to src/buildsuites/_build/samplesuite1.
• suite_tcl90 created (tracked): build905.zig verbatim from the G-076-validated recipe, helper build modules, src/main.zig stub, tools (wrapfiletofile.zig, zipfs_mkzip/mkimg.tcl), patches/ (piperepl pair + tclZipfs + tclCmdIL + README with the piperepl semantics), suite README, and the suite.tcl driver.
• Driver design: the tracked recipe is STAGED into _build/suite_tcl90/build with sibling sources arranged around it (tcl905 = fossil core-9-0-branch; tclthread and tclvfs fossil trunk; zlib copied from the tcl checkout's own compat/zlib 1.3.2; vqtcl from a local dir - no public upstream confirmed), so the recipe needs no path surgery and cache/outputs land under _build naturally. Fossil checkouts open --nested (the stage is inside the punkshell checkout) with FOSSIL_HOME= so the global fossil config db records nothing; clones reuse ~/.fossils (tcl, tclthread, tclvfs all already present).
• Fresh-checkout generation lives in the driver (G-076 findings productized): win/tclUuid.h from manifest.uuid; win/tclsh.exe.manifest from its .in (TCL_WIN_VERSION per the tree's own formula, parsed from tcl.h - note the defines are indented '# define'); thread.rc gains the tcl.h include for its version resource (scripted idempotent edit of the staged checkout, replacing the historical uncommitted tweak in the 2024 tclthread checkout).
• Recipe fixes made in the suite copies: the four tcl-dependent helper modules' hardcoded ../tcl90/generic include (author's "todo - fix") now use their tcldir param; build_tclthread additionally puts the generated threadUuid.h's output dir on the include path - current thread trunk #includes threadUuid.h (upstream makefiles generate it into their build dir; the recipe's copy-to-source went to win/, which is not an include dir). build_libtommath.zig keeps its hardcoded path but is not called by the recipe.
• End-to-end run PASS (native tclsh 9.0.3 driving; zig 0.14.0-dev.2074 at C:/zig): fetch -> stage -> generate -> build -> smoke. Static and zip shells both report 9.0.5 (checkout 1a9c3b9d96 = core-9-0-branch tip carrying the f10d91c2 console fix); zip shell tzdata working; tcl9thread301.dll builds from CURRENT thread trunk (3.0.1) and loads. vfs/vlerq dlls also built.

2026-07-20 increment 2: punk-getzig rename + naming policy; 0.15 migration attempted then reverted

• getzig -> punk-getzig: scriptset renamed (bash/ps1/_wrap.toml; getzig_original.polyglot kept as historical reference), payload self-references updated, and the ziglang.org release URL now derives its version from the single $releasearchive pin (it carried a separately hardcoded 0.15.1 against the 0.15.2 archive). Re-wrapped via scriptwrap multishell (checkfile ERROR-free; the two boundary-scan warnings are the documented-normal polyglot ones). bin: getzig.cmd removed, punk-getzig.cmd tracked, punk-getzig.ps1 twin (byte-copy) refreshed; bin/AGENTS.md gains the punk- naming policy section (exceptions dtplite/sdx/kettle wrappers + getpunk; sweep = G-097); src/scriptapps/AGENTS.md reference updated.
• zig 0.15.2 migration attempted (bin/tools/zig = the getzig-managed 0.15.2): a scripted API transform (addExecutable/addObject root_module + createModule, addShared/StaticLibrary -> addLibrary with .linkage, std.ArrayList -> std.array_list.Managed, b.host -> b.graph.host) got the recipe to compile up to a strict-format print issue, but the user flagged mid-work that the 0.14->0.16 churn is significant and the 2024 zig target comes first - REVERTED to the committed 0.14-validated recipe and re-verified PASS (9.0.5 shells, zig 0.14.0-dev.2074). The transformer script and findings are preserved (scratchpad zig015_migrate.tcl; known remaining 0.15 items: slice-of-strings debug.print format, plus whatever the next compile reveals). Pin-path decision put to the user: official 0.14.1 release as-is, punkbin-hosted 0.14.0-dev.2074, or completing the 0.15.2 migration.

2026-07-20 increment 3: pinned zig 0.14.1 (user-chosen path) validated, hosted on punkbin, wired as suite default

• User decision: pin the first OFFICIAL release in the recipe's line - zig 0.14.1 - rather than migrating to 0.15.x now or hosting the unreleased dev.2074. Fetched from a ziglang community mirror (pkg.hexops.org; ziglang.org direct timed out), minisign-verified against the zig release key (the key punk-getzig embeds), and extracted to bin/tools/zig-x86_64-windows-0.14.1/.
• Recipe compat deltas for 0.14.1, validated to keep dev.2074 working (regression PASS on both zigs): b.host -> b.graph.host; the redundant second UNICODE tclMain add into tcl_objs_static removed (0.14.1 links what dev.2074's archive semantics left unpulled); the exe's ansi tclMain copy now compiles with -UUNICODE -U_UNICODE -DTCL_ASCII_MAIN (the zig cc driver can supply windows-default defines - under 0.14.1 the "ansi" copy came out as a duplicate UNICODE compile: lld duplicate Tcl_MainExW/commons; upstream's TCL_ASCII_MAIN is the intended ascii-variant guard).
• Cache discipline (user insight: zig caches have needed clearing across version changes): the driver now uses per-zig-version local caches (.zig-cache- in the staged build dir) and stages the recipe by per-item sync instead of wholesale delete (windows can hold transient locks on cache trees). During diagnosis both a fresh local AND isolated global cache were tested - the 0.14.1 duplicate-symbol failure was real recipe/driver behaviour, not stale cache, but the segregation guards the class going forward.
• punkbin publish (user-directed): zig-x86_64-windows-0.14.1.zip + .minisig added to punkbin win32-x86_64/tools with sha1sums regenerated per that repo's workflow (NEW x2, no CHANGED), committed with provenance and PUSHED upstream (abcd87c) - the suite pin is now fetchable from punk-getzig's primary mirror.
• suite.tcl zig resolution is now: -zig > PUNK_ZIG > pinned bin/tools/zig-x86_64-windows-0.14.1/zig.exe > PATH; verified end-to-end with no options (pinned default -> PASS, static+zip shells 9.0.5). README documents the pin, the compat deltas, and the per-version cache rationale.
• The 0.15.2 migration remains parked (transformer script + findings preserved; increment-2 record).

Remaining for acceptance:

• punk-getzig per-version fetch argument, so the wrapper itself can materialize the suite's pinned 0.14.1 (currently it installs only its own single pin, 0.15.2, to bin/tools/zig; the suite pin was materialized by the same fetch+minisign flow manually and is hosted on punkbin). Payload growth needs the 512-byte spacer check on re-wrap.
• Rebase the piperepl patch onto current core-9-0-branch sources; build a piperepl-enabled variant; behaviour matrix (TCLSH_PIPEREPL unset == stock on tty/piped/script-arg forms; =1 with a script setting ::tclsh(dorepl) 1 -> live interactive console repl after piped eof with unconsumed input in ::tclsh(inputbuffer)).
• Final clean-checkout re-verification of the whole flow once the above land.