Browse Source

make.tcl tool subcommand: build vendored zig tools into bin/ (G-126); 0.28.3

New surface for the vendored first-party build tools under src/tools:

  make.tcl tool list|info|build|test ?<toolname> ...? ?-test 0|1?

- discovery is convention-based (a src/tools directory carrying
  build.zig is a tool), so the G-128 tool will appear with zero
  make.tcl edits; per-tool records are read tolerantly from
  build.zig.zon (version, minimum_zig_version floor) and PROVENANCE.md
  (upstream, vendored commit)
- list/info report installed-binary state (current|stale|absent vs
  bin/<name><exe-suffix>) and the resolved toolchain
- toolchain resolution: PUNK_ZIG=<path> overrides; otherwise
  bin/tools/zig* is scanned and the LOWEST release satisfying the floor
  wins - deterministic as newer toolchains get unpacked beside the
  pinned one; dev builds count only when their base version exceeds
  the floor
- build runs the tree's own 'zig build test' as an EXIT-CODE gate
  (expected stderr warnings do not fail it), then builds ReleaseSafe
  and installs to bin/; nothing installs when the gate or build fails;
  -test 0 skips the gate; build caches are deliberately left in place
  between runs for rebuild speed (git- and fossil-ignored)
- zig stays OPTIONAL: packages/bake never require the step; without a
  suitable toolchain, list reports the state and build/test exit
  nonzero with bin/punk-getzig.cmd guidance
- full subcommand accompaniments per src/AGENTS.md: punk::args argdoc
  (SUMMARIES/HELPTEXTS/bespoke passthrough definition), SUBGROUPS
  ("vendored tools"), known_commands, degraded-mode dispatch, plain
  help text, bootsupport-staleness exemption, workflow text (new
  OUT-OF-BAND SUBSYSTEMS section also covering buildsuite), project
  version 0.28.3 + CHANGELOG entry
- piped characterization (zig-independent):
  src/tests/shell/testsuites/punkexe/maketcltool.test - 4/4 green
- verified live: 'tool build punkzip' gated on the vendored tree's
  130/130 suite and installed bin/punkzip.exe (ReleaseSafe, v2.3.1,
  1.1MB) over the stale Nov-2024 v2.1.0 binary; list state flipped
  stale -> current; PUNKBOOT_PLAIN degraded dispatch and tabled help
  verified

G-126 Progress updated: the acceptance's build-step clause is
satisfied. Remaining: punk::zip fast path, parity suite, benchmark.

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 4 days ago
parent
commit
d76a7907ea
  1. 13
      CHANGELOG.md
  2. 33
      goals/G-126-punkzip-accelerator.md
  3. 2
      punkproject.toml
  4. 411
      src/make.tcl
  5. 154
      src/tests/shell/testsuites/punkexe/maketcltool.test

13
CHANGELOG.md

@ -5,6 +5,19 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to
Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` Entries are newest-first; one bullet per notable change. See the root `AGENTS.md`
"Project Versioning" section for the bump policy. "Project Versioning" section for the bump policy.
## [0.28.3] - 2026-07-27
- New `make.tcl tool` subcommand (G-126): surface for the vendored first-party zig
build tools under src/tools (punkzip first). `tool list|info` report each tool's
version, declared zig floor, installed-binary state and vendored upstream commit
(from its PROVENANCE.md); `tool build ?name ...? ?-test 0|1?` resolves a pinned
toolchain (PUNK_ZIG override, else the lowest satisfying release under
bin/tools/zig*), runs the tree's own 'zig build test' as an exit-code gate, builds
ReleaseSafe and installs to bin/. zig stays optional: packages/bake never require
the step, and without a suitable toolchain the state is reported (build/test exit
nonzero with bin/punk-getzig.cmd guidance). Piped characterization:
src/tests/shell/testsuites/punkexe/maketcltool.test.
## [0.28.2] - 2026-07-27 ## [0.28.2] - 2026-07-27
- Bakes verify payload/target consistency (G-133, both checks advisory - recapped - Bakes verify payload/target consistency (G-133, both checks advisory - recapped

33
goals/G-126-punkzip-accelerator.md

@ -256,10 +256,29 @@ working tree; one now exists.
extended for the vendored fixtures (hamlet.txt is CRLF byte-exact test data). The extended for the vendored fixtures (hamlet.txt is CRLF byte-exact test data). The
acceptance's vendoring clause (origin, state, re-vendor procedure, licensing) is acceptance's vendoring clause (origin, state, re-vendor procedure, licensing) is
satisfied. satisfied.
- Remaining for acceptance: make.tcl tool build step producing `bin/punkzip.exe` - 2026-07-27 make.tcl TOOL STEP landed (user-directed; punkshell 0.28.3): new
reproducibly on a clean checkout (zig OPTIONAL - absence reported, not fatal; test `make.tcl tool list|info|build|test ?<name> ...? ?-test 0|1?` subcommand.
gate keys on exit code, not clean stderr - the zip suite's negative tests print Convention-based discovery (a directory under src/tools carrying build.zig is a
EOCD warnings while passing; step design shared with the G-128 tool); punk::zip tool - the G-128 tool will appear with zero make.tcl edits); per-tool records read
accelerator detection + fast path behind the G-124 floor; parity suite in tolerantly from build.zig.zon (version, zig floor) and PROVENANCE.md (upstream,
`src/tests/modules/punk/zip/`; recorded benchmark (speedup + entry count where it vendored commit); installed-state reporting (current|stale|absent vs
becomes material). bin/<name><exe-suffix>); toolchain resolution = PUNK_ZIG override, else a scan of
bin/tools/zig* choosing the LOWEST release satisfying the floor (deterministic as
newer toolchains appear beside the pin; dev builds count only above the floor);
'zig build test' EXIT-CODE gate before a ReleaseSafe build and install to bin/
(nothing installs on gate/build failure). zig-OPTIONAL honoured: packages/bake
never touch the step; without a suitable toolchain list reports the state and
build/test exit nonzero with punk-getzig guidance. Verified live: gate + build
installed bin/punkzip.exe (ReleaseSafe, 1.1MB, v2.3.1) replacing the stale
Nov-2024 v2.1.0 binary, state flipped stale->current; degraded PUNKBOOT_PLAIN
dispatch and tabled help verified. Full subcommand accompaniments: argdoc +
SUBGROUPS + known_commands + plain help + workflow-text OUT-OF-BAND SUBSYSTEMS
section + piped characterization
src/tests/shell/testsuites/punkexe/maketcltool.test (4/4, zig-independent).
Build caches (.zig-cache/zig-out) left in place between runs for rebuild speed
(user decision 2026-07-27). The acceptance's build-step clause is satisfied:
bin/punkzip.exe is reproducible on a clean checkout from vendored source + pinned
toolchain + this step.
- Remaining for acceptance: punk::zip accelerator detection + fast path behind the
G-124 floor; parity suite in `src/tests/modules/punk/zip/`; recorded benchmark
(speedup + entry count where it becomes material).

2
punkproject.toml

@ -1,4 +1,4 @@
[project] [project]
name = "punkshell" name = "punkshell"
version = "0.28.2" version = "0.28.3"
license = "BSD-2-Clause" license = "BSD-2-Clause"

411
src/make.tcl

@ -187,7 +187,7 @@ namespace eval ::punkboot {
variable foldername [file tail $scriptfolder] variable foldername [file tail $scriptfolder]
variable pkg_requirements [list]; variable pkg_missing [list];variable pkg_loaded [list] variable pkg_requirements [list]; variable pkg_missing [list];variable pkg_loaded [list]
variable help_flags [list -help --help /? -h] variable help_flags [list -help --help /? -h]
variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate bootsupport vfscommonupdate projectversion workflow buildsuite project vfs] variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate bootsupport vfscommonupdate projectversion workflow buildsuite tool project vfs]
} }
@ -2152,6 +2152,11 @@ proc ::punkboot::punkboot_gethelp {args} {
append h " $scriptname buildsuite list|info|build ?<suitename>? ?driver-args ...?" \n append h " $scriptname buildsuite list|info|build ?<suitename>? ?driver-args ...?" \n
append h " - list the defined buildsuites under src/buildsuites (zig runtime factory), show a suite's configured" \n append h " - list the defined buildsuites under src/buildsuites (zig runtime factory), show a suite's configured" \n
append h " detail (sources/zig pin/products from its sources.config), or run its driver forwarding the args" \n \n append h " detail (sources/zig pin/products from its sources.config), or run its driver forwarding the args" \n \n
append h " $scriptname tool list|info|build|test ?<toolname> ...? ?-test 0|1?" \n
append h " - vendored first-party build tools under src/tools (zig source; G-126): list/detail them, run their" \n
append h " test suites, or build (test-gated) and install to <projectdir>/bin" \n
append h " - zig is OPTIONAL: packages/bake never require this step; without a suitable toolchain the state is" \n
append h " reported and build/test exit nonzero with fetch guidance (bin/punk-getzig.cmd)" \n \n
append h " $scriptname help ?subcommand?" \n append h " $scriptname help ?subcommand?" \n
append h " - show usage for $scriptname or one of its subcommands (equivalent: $scriptname <subcommand> -help)" \n \n append h " - show usage for $scriptname or one of its subcommands (equivalent: $scriptname <subcommand> -help)" \n \n
append h " Deprecated aliases (transition window):" \n append h " Deprecated aliases (transition window):" \n
@ -2402,6 +2407,14 @@ KEY / NOTES
with binary dependencies, and version-preference outcomes are visible only to with binary dependencies, and version-preference outcomes are visible only to
the smoke probe - and only for declared packages. 'make.tcl check' reports both. the smoke probe - and only for declared packages. 'make.tcl check' reports both.
OUT-OF-BAND SUBSYSTEMS (not part of the packages/bake data flow above)
'make.tcl tool' builds the vendored first-party zig tools under src/tools
(G-126; punkzip first) into <projectdir>/bin, gated on each tree's own
'zig build test' exit code. zig is OPTIONAL: packages/bake never require it.
'make.tcl buildsuite' is the analogous arm's-length factory for whole Tcl
runtimes built from EXTERNAL sources under src/buildsuites (fetch + test-gate
contract per suite; products land in the bin/runtime store tiers).
MAINTENANCE (agents take note) MAINTENANCE (agents take note)
---------------------------------------------------------------------------- ----------------------------------------------------------------------------
This text is embedded in make.tcl (::punkboot::workflow_text). When the build This text is embedded in make.tcl (::punkboot::workflow_text). When the build
@ -2514,6 +2527,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
shell "Run the punk shell using bootsupport libraries" shell "Run the punk shell using bootsupport libraries"
help "Show usage for make.tcl or one of its subcommands" help "Show usage for make.tcl or one of its subcommands"
buildsuite "List, describe or run the defined buildsuites under src/buildsuites (zig runtime factory)" buildsuite "List, describe or run the defined buildsuites under src/buildsuites (zig runtime factory)"
tool "List, test or build the vendored tools under src/tools (zig) and install them to <projectdir>/bin"
project "DEPRECATED alias for bakehouse (one-line notice, then runs bakehouse)" project "DEPRECATED alias for bakehouse (one-line notice, then runs bakehouse)"
vfs "DEPRECATED alias for bake (one-line notice, then runs bake)" vfs "DEPRECATED alias for bake (one-line notice, then runs bake)"
} }
@ -2683,6 +2697,45 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
(copy-and-tweak workflow) appears here automatically with no (copy-and-tweak workflow) appears here automatically with no
make.tcl edits. The contract is documented in make.tcl edits. The contract is documented in
src/buildsuites/README.md." src/buildsuites/README.md."
tool
"
Surface for the vendored first-party build tools under src/tools
(G-126: zig source vendored in full; boundary and editing policy in
src/tools/AGENTS.md, per-tree origin/licence records in
<tool>/PROVENANCE.md). Discovery is convention-based: a directory
under src/tools carrying build.zig is a tool; its installed
artifact is <projectdir>/bin/<name> with the host executable
suffix.
Actions:
tool list
Discover the tools with version (build.zig.zon), declared zig
floor, installed-binary state (current|stale|absent) and the
vendored upstream commit; report the resolved toolchain.
tool info <name> ...
Per-tool detail: paths, provenance records, toolchain
resolution.
tool build ?<name> ...? ?-test 0|1?
Resolve a zig satisfying the tool's declared floor, run the
'zig build test' gate (default on; the gate is the EXIT CODE -
expected warnings on stderr do not fail it), build ReleaseSafe
and install to <projectdir>/bin. Nothing is installed when the
gate or the build fails. With no name, all tools are
processed.
tool test ?<name> ...?
Run each tool's 'zig build test' only.
Toolchain resolution: PUNK_ZIG=<path-to-zig-or-its-folder>
overrides; otherwise <projectdir>/bin/tools/zig* is scanned and
the LOWEST release satisfying the floor wins (deterministic as
newer toolchains appear beside the pinned one; dev builds count
only when their base version exceeds the floor). zig is OPTIONAL
for punkshell builds: packages/bake never require this step -
without a suitable toolchain, list reports the state and
build/test exit nonzero with fetch guidance
(bin/punk-getzig.cmd). Build caches (.zig-cache/zig-out) are
deliberately left in place between runs for rebuild speed (both
git- and fossil-ignored)."
project project
" "
DEPRECATED alias for 'bakehouse' (G-112 stage-true rename) - prints a DEPRECATED alias for 'bakehouse' (G-112 stage-true rename) - prints a
@ -2855,6 +2908,15 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
arg -type any -optional 1 -multiple 1 -help\ arg -type any -optional 1 -multiple 1 -help\
"Action (list|info|build), suite name, and - for build - driver "Action (list|info|build), suite name, and - for build - driver
arguments forwarded to the suite's suite.tcl untouched."} arguments forwarded to the suite's suite.tcl untouched."}
punk::args::define\
"@id -id (script)::punkboot::tool"\
"@normalize"\
"@cmd -name [list "make.tcl tool"] -summary [list [dict get $SUMMARIES tool]] -help [list [dict get $HELPTEXTS tool]]"\
{@form -synopsis "make.tcl tool list|info|build|test ?<toolname> ...? ?-test 0|1?"}\
{@values -min 0 -max -1
arg -type any -optional 1 -multiple 1 -help\
"Action (list|info|build|test), tool name(s), and - for build -
the -test 0|1 gate flag. Parsed by the handler."}
punk::args::define\ punk::args::define\
"@id -id (script)::punkboot::help"\ "@id -id (script)::punkboot::help"\
"@normalize"\ "@normalize"\
@ -2871,6 +2933,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
"promotion gates" {bootsupport vfscommonupdate} "promotion gates" {bootsupport vfscommonupdate}
"source maintenance" {vendorupdate} "source maintenance" {vendorupdate}
"buildsuites" {buildsuite} "buildsuites" {buildsuite}
"vendored tools" {tool}
"informational" {info check bakelist projectversion workflow help} "informational" {info check bakelist projectversion workflow help}
"interactive" {shell} "interactive" {shell}
"deprecated aliases" {project vfs} "deprecated aliases" {project vfs}
@ -2963,6 +3026,10 @@ if {$::punkboot::punkargs_ok} {
#args (arbitrary suite flags) are forwarded untouched (G-104) #args (arbitrary suite flags) are forwarded untouched (G-104)
set ::punkboot::command buildsuite set ::punkboot::command buildsuite
set ::punkboot::bs_args $subargs set ::punkboot::bs_args $subargs
} elseif {$subcommand eq "tool"} {
#declared passthrough: the handler parses action/names/-test itself (G-126)
set ::punkboot::command tool
set ::punkboot::tool_args $subargs
} else { } else {
if {[catch {punk::args::parse $subargs -errorstyle $::punkboot::errstyle withid (script)::punkboot::$subcommand} argd]} { if {[catch {punk::args::parse $subargs -errorstyle $::punkboot::errstyle withid (script)::punkboot::$subcommand} argd]} {
puts stderr $argd puts stderr $argd
@ -2993,6 +3060,12 @@ if {$::punkboot::punkargs_ok} {
set commands_found [list buildsuite] set commands_found [list buildsuite]
set argsleft [list] set argsleft [list]
set ::punkboot::bs_args [lrange $scriptargs 1 end] set ::punkboot::bs_args [lrange $scriptargs 1 end]
} elseif {[lindex $scriptargs 0] eq "tool"} {
#declared passthrough (as in the punk::args path) - the handler parses
#action/names/-test itself (G-126)
set commands_found [list tool]
set argsleft [list]
set ::punkboot::tool_args [lrange $scriptargs 1 end]
} }
while {[llength $argsleft]} { while {[llength $argsleft]} {
set a [lindex $argsleft 0] set a [lindex $argsleft 0]
@ -3116,7 +3189,8 @@ set forcekill $::punkboot::opt_forcekill
# - patch bump -> warn and proceed (non-breaking) # - patch bump -> warn and proceed (non-breaking)
# 'check' is exempt in all cases the warning is shown at the end of its output instead. # 'check' is exempt in all cases the warning is shown at the end of its output instead.
# 'projectversion', 'workflow' and 'bakelist' are informational/read-only and are likewise exempt. # 'projectversion', 'workflow' and 'bakelist' are informational/read-only and are likewise exempt.
if {[info exists ::punkboot::stale_bootsupport] && $::punkboot::command ni {check projectversion workflow bakelist buildsuite}} { # 'buildsuite' and 'tool' are arm's-length subsystems that do not consume bootsupport modules.
if {[info exists ::punkboot::stale_bootsupport] && $::punkboot::command ni {check projectversion workflow bakelist buildsuite tool}} {
set _stale $::punkboot::stale_bootsupport set _stale $::punkboot::stale_bootsupport
set _have_major 0 set _have_major 0
set _have_minor 0 set _have_minor 0
@ -4242,6 +4316,339 @@ if {$::punkboot::command eq "buildsuite"} {
} }
} }
if {$::punkboot::command eq "tool"} {
#G-126: vendored first-party build tools under src/tools (zig source vendored in
#full - boundary and editing policy in src/tools/AGENTS.md; per-tree origin,
#re-vendor procedure and licence records in <tool>/PROVENANCE.md). Discovery is
#convention-based: a directory under src/tools carrying build.zig is a tool, its
#name is the directory name, its build artifact is <tooldir>/zig-out/bin/<name>
#and its installed copy is <projectdir>/bin/<name> (host executable suffix).
#zig stays OPTIONAL for punkshell builds: nothing in packages/bake depends on
#this step; explicit build/test without a suitable toolchain exits nonzero with
#fetch guidance. Build caches (.zig-cache/zig-out) are deliberately left in
#place between builds for rebuild speed (both git- and fossil-ignored).
set tool_root [file join $sourcefolder tools]
if {![info exists ::punkboot::tool_args]} {set ::punkboot::tool_args [list]}
set targs $::punkboot::tool_args
set tool_action [lindex $targs 0]
if {$tool_action eq ""} {set tool_action list}
#args after the action: tool names plus an optional '-test 0|1' flag (build)
set tool_dotest 1
set tool_names [list]
set _i 1
while {$_i < [llength $targs]} {
set a [lindex $targs $_i]
if {$a eq "-test"} {
incr _i
set v [lindex $targs $_i]
if {![string is boolean -strict $v]} {
puts stderr "make.tcl tool: -test requires a boolean value (got '$v')"
exit 2
}
set tool_dotest [expr {bool($v)}]
} else {
lappend tool_names $a
}
incr _i
}
unset -nocomplain _i a v
proc ::punkboot::tool_dirs {tool_root} {
set dirs [list]
foreach dir [lsort [glob -nocomplain -dir $tool_root -type d *]] {
if {![file exists [file join $dir build.zig]]} {continue} ;#not a tool
lappend dirs $dir
}
return $dirs
}
#tolerant build.zig.zon reader: tool version + declared minimum zig version
proc ::punkboot::tool_zoninfo {tooldir} {
set m [dict create version "" zigfloor ""]
if {![catch {
set f [open [file join $tooldir build.zig.zon] r]
set data [read $f]
close $f
}]} {
if {[regexp {\.version\s*=\s*"([^"]+)"} $data -> _v]} {dict set m version $_v}
if {[regexp {\.minimum_zig_version\s*=\s*"([^"]+)"} $data -> _fl]} {dict set m zigfloor $_fl}
}
return $m
}
#tolerant PROVENANCE.md reader: vendored upstream commit + upstream location
proc ::punkboot::tool_provenance {tooldir} {
set m [dict create commit "" upstream ""]
if {![catch {
set f [open [file join $tooldir PROVENANCE.md] r]
set data [read $f]
close $f
}]} {
if {[regexp {Vendored state: commit\s+`?([0-9a-fA-F]{7,40})} $data -> _c]} {dict set m commit $_c}
if {[regexp {Upstream repository:\s+`?([^`\n]+)} $data -> _u]} {dict set m upstream [string trim $_u]}
}
return $m
}
proc ::punkboot::tool_newest_src_mtime {dir} {
set newest 0
foreach f [glob -nocomplain -dir $dir -type f *] {
if {[file mtime $f] > $newest} {set newest [file mtime $f]}
}
foreach d [glob -nocomplain -dir $dir -type d *] {
if {[file tail $d] in {zig-out .zig-cache zig-cache}} {continue}
set sub [::punkboot::tool_newest_src_mtime $d]
if {$sub > $newest} {set newest $sub}
}
return $newest
}
proc ::punkboot::tool_zig_version {zigexe} {
if {[catch {exec $zigexe version 2>@1} v]} {return ""}
return [string trim [lindex [split $v \n] 0]]
}
#a dev build's base version must EXCEED the floor (e.g 0.16.0-dev < 0.16.0)
proc ::punkboot::tool_zig_satisfies {ver floor} {
if {$floor eq ""} {return 1}
set base $ver ; set isdev 0
if {[regexp {^([0-9][0-9.]*)[-+]} $ver -> b]} {set base $b ; set isdev 1}
if {[catch {package vcompare $base $floor} cmp]} {return 0}
if {$cmp > 0} {return 1}
if {$cmp == 0 && !$isdev} {return 1}
return 0
}
#resolve a zig for the given floor: PUNK_ZIG env override (path to the zig
#executable or its folder), else scan <projectdir>/bin/tools/zig* preferring
#releases over dev builds and the LOWEST satisfying version - a deterministic
#choice that stays stable as newer toolchains get unpacked beside the pinned one
proc ::punkboot::tool_resolve_zig {projectroot floor} {
if {[info exists ::env(PUNK_ZIG)] && $::env(PUNK_ZIG) ne ""} {
set cand $::env(PUNK_ZIG)
if {[file isdirectory $cand]} {
foreach n {zig.exe zig} {
if {[file exists [file join $cand $n]]} {set cand [file join $cand $n] ; break}
}
}
set v [::punkboot::tool_zig_version $cand]
if {$v eq ""} {
return [dict create note "PUNK_ZIG='$::env(PUNK_ZIG)' did not answer 'zig version'"]
}
if {![::punkboot::tool_zig_satisfies $v $floor]} {
return [dict create note "PUNK_ZIG zig is $v which does not satisfy this tool's floor (>= $floor)"]
}
return [dict create zig $cand version $v via PUNK_ZIG]
}
set best "" ; set bestv "" ; set bestbase "" ; set bestdev 1
foreach dir [lsort [glob -nocomplain -dir [file join $projectroot bin tools] -type d zig*]] {
set p ""
foreach n {zig.exe zig} {
if {[file exists [file join $dir $n]]} {set p [file join $dir $n] ; break}
}
if {$p eq ""} {continue}
set v [::punkboot::tool_zig_version $p]
if {$v eq "" || ![::punkboot::tool_zig_satisfies $v $floor]} {continue}
set base $v ; set isdev 0
if {[regexp {^([0-9][0-9.]*)[-+]} $v -> b]} {set base $b ; set isdev 1}
set take 0
if {$best eq ""} {
set take 1
} elseif {$bestdev && !$isdev} {
set take 1
} elseif {$bestdev == $isdev && [package vcompare $base $bestbase] < 0} {
set take 1
}
if {$take} {set best $p ; set bestv $v ; set bestbase $base ; set bestdev $isdev}
}
if {$best eq ""} {
set want [expr {$floor eq "" ? "any version" : ">= $floor"}]
return [dict create note "no suitable zig toolchain ($want) under [file join $projectroot bin tools] - 'bin/punk-getzig.cmd <version>' fetches one; PUNK_ZIG=<path-to-zig> overrides"]
}
return [dict create zig $best version $bestv via scan]
}
proc ::punkboot::tool_installed_exe {projectroot name} {
set suffix [punkboot::lib::platform_exe_suffix $::punkboot::target_platform]
return [file join $projectroot bin ${name}$suffix]
}
proc ::punkboot::tool_install_state {projectroot tooldir} {
set exe [::punkboot::tool_installed_exe $projectroot [file tail $tooldir]]
if {![file exists $exe]} {return absent}
if {[file mtime $exe] >= [::punkboot::tool_newest_src_mtime $tooldir]} {return current}
return stale
}
#run 'zig <argl>' with cwd at the tool tree (the tree's CLI tests resolve
#fixtures and zig-out relative to the build root). The gate is the EXIT CODE:
#stderr legitimately carries expected warnings (punkzip's zip suite negative
#tests print EOCD warnings while passing). Returns the child's exit status.
proc ::punkboot::tool_zig_run {zig tooldir argl} {
set saved [pwd]
cd $tooldir
set rc 0
if {[catch {exec $zig {*}$argl >@stdout 2>@stderr} _e _o]} {
set ecode [dict get $_o -errorcode]
if {[lindex $ecode 0] eq "CHILDSTATUS"} {
set rc [lindex $ecode 2]
} else {
puts stderr "tool: failed to run '$zig $argl' in $tooldir: $_e"
set rc 1
}
}
cd $saved
return $rc
}
set alldirs [::punkboot::tool_dirs $tool_root]
set byname [dict create]
foreach d $alldirs {dict set byname [file tail $d] $d}
if {[llength $tool_names]} {
set selected [list]
foreach n $tool_names {
if {![dict exists $byname $n]} {
puts stderr "tool '$n' not found under $tool_root - configured tools: [dict keys $byname]"
exit 2
}
lappend selected [dict get $byname $n]
}
} else {
set selected $alldirs
}
if {![llength $alldirs] && $tool_action ne "list"} {
puts stderr "no vendored tools found under $tool_root (a tool = a directory carrying build.zig)"
exit 2
}
switch -- $tool_action {
list {
if {![llength $alldirs]} {
puts stdout "no vendored tools found under $tool_root (a tool = a directory carrying build.zig)"
exit 0
}
puts stdout "vendored tools under $tool_root (boundary: src/tools/AGENTS.md; per-tree PROVENANCE.md):"
puts stdout [format " %-10s %-9s %-10s %-8s %s" name version zig-floor bin vendored-commit]
set maxfloor ""
foreach dir $alldirs {
set name [file tail $dir]
set zon [::punkboot::tool_zoninfo $dir]
set prov [::punkboot::tool_provenance $dir]
set floor [dict get $zon zigfloor]
if {$floor ne "" && ($maxfloor eq "" || [package vcompare $floor $maxfloor] > 0)} {
set maxfloor $floor
}
set commit [dict get $prov commit]
set commit [expr {$commit eq "" ? "-" : [string range $commit 0 11]}]
set floorshow [expr {$floor eq "" ? "-" : ">=$floor"}]
puts stdout [format " %-10s %-9s %-10s %-8s %s" $name [dict get $zon version] $floorshow [::punkboot::tool_install_state $projectroot $dir] $commit]
}
set zres [::punkboot::tool_resolve_zig $projectroot $maxfloor]
if {[dict exists $zres zig]} {
puts stdout "toolchain: zig [dict get $zres version] via [dict get $zres via] ([dict get $zres zig])"
} else {
puts stdout "toolchain: [dict get $zres note]"
}
puts stdout ""
puts stdout " - 'make.tcl tool info <name>' shows a tool's detail"
puts stdout " - 'make.tcl tool build ?<name> ...? ?-test 0|1?' builds (test-gated) and installs to bin/"
puts stdout " - zig is OPTIONAL for punkshell builds: packages/bake never require this step"
exit 0
}
info {
if {![llength $tool_names]} {
puts stderr "usage: make.tcl tool info <toolname> ('make.tcl tool list' shows the names)"
exit 2
}
foreach dir $selected {
set name [file tail $dir]
set zon [::punkboot::tool_zoninfo $dir]
set prov [::punkboot::tool_provenance $dir]
set floor [dict get $zon zigfloor]
puts stdout "tool: $name"
puts stdout " path: $dir"
puts stdout " version: [dict get $zon version] (build.zig.zon)"
puts stdout " zig floor: [expr {$floor eq "" ? "(none declared)" : ">= $floor"}]"
puts stdout " upstream: [expr {[dict get $prov upstream] eq "" ? "(no PROVENANCE.md record)" : [dict get $prov upstream]}]"
puts stdout " vendored commit: [expr {[dict get $prov commit] eq "" ? "(no PROVENANCE.md record)" : [dict get $prov commit]}]"
puts stdout " provenance: [file join $dir PROVENANCE.md]"
set exe [::punkboot::tool_installed_exe $projectroot $name]
puts stdout " installed: $exe ([::punkboot::tool_install_state $projectroot $dir])"
set zres [::punkboot::tool_resolve_zig $projectroot $floor]
if {[dict exists $zres zig]} {
puts stdout " toolchain: zig [dict get $zres version] via [dict get $zres via] ([dict get $zres zig])"
} else {
puts stdout " toolchain: [dict get $zres note]"
}
}
exit 0
}
build {
foreach dir $selected {
set name [file tail $dir]
set floor [dict get [::punkboot::tool_zoninfo $dir] zigfloor]
set zres [::punkboot::tool_resolve_zig $projectroot $floor]
if {![dict exists $zres zig]} {
puts stderr "tool build $name: [dict get $zres note]"
exit 1
}
set zig [dict get $zres zig]
puts stdout "tool build $name: zig [dict get $zres version] via [dict get $zres via] ($zig)"
flush stdout
if {$tool_dotest} {
puts stdout "tool build $name: test gate ('zig build test') ..."
flush stdout
set rc [::punkboot::tool_zig_run $zig $dir [list build test]]
if {$rc != 0} {
puts stderr "tool build $name: test gate FAILED (exit $rc) - nothing installed"
exit 1
}
puts stdout "tool build $name: test gate passed"
} else {
puts stdout "tool build $name: test gate SKIPPED (-test 0)"
}
puts stdout "tool build $name: building (ReleaseSafe) ..."
flush stdout
set rc [::punkboot::tool_zig_run $zig $dir [list build -Doptimize=ReleaseSafe]]
if {$rc != 0} {
puts stderr "tool build $name: build FAILED (exit $rc) - nothing installed"
exit 1
}
set suffix [punkboot::lib::platform_exe_suffix $::punkboot::target_platform]
set built [file join $dir zig-out bin ${name}$suffix]
if {![file exists $built]} {
puts stderr "tool build $name: build reported success but $built is missing"
exit 1
}
set exe [::punkboot::tool_installed_exe $projectroot $name]
if {[catch {file copy -force $built $exe} cperr]} {
puts stderr "tool build $name: could not install $exe ($cperr) - is a copy currently running?"
exit 1
}
puts stdout "tool build $name: installed $exe ([file size $exe] bytes)"
}
exit 0
}
test {
foreach dir $selected {
set name [file tail $dir]
set floor [dict get [::punkboot::tool_zoninfo $dir] zigfloor]
set zres [::punkboot::tool_resolve_zig $projectroot $floor]
if {![dict exists $zres zig]} {
puts stderr "tool test $name: [dict get $zres note]"
exit 1
}
set zig [dict get $zres zig]
puts stdout "tool test $name: zig [dict get $zres version] via [dict get $zres via] - running 'zig build test' ..."
flush stdout
set rc [::punkboot::tool_zig_run $zig $dir [list build test]]
if {$rc != 0} {
puts stderr "tool test $name: FAILED (exit $rc)"
exit 1
}
puts stdout "tool test $name: passed"
}
exit 0
}
default {
puts stderr "unknown tool action '$tool_action' - expected list|info|build|test"
puts stderr "usage: make.tcl tool list|info|build|test ?<toolname> ...? ?-test 0|1?"
exit 2
}
}
}
if {$::punkboot::command eq "shell"} { if {$::punkboot::command eq "shell"} {
#G-113: the ansistrip transform serves make.tcl's own output only - the repl's #G-113: the ansistrip transform serves make.tcl's own output only - the repl's
#colour behaviour is the shell's own concern (colour on/off + NO_COLOR there). #colour behaviour is the shell's own concern (colour on/off + NO_COLOR there).

154
src/tests/shell/testsuites/punkexe/maketcltool.test

@ -0,0 +1,154 @@
package require tcltest
#Piped characterization of the make.tcl tool subcommand (goal G-126): the surface
#for the vendored first-party zig build tools under src/tools. Runs the WORKING
#TREE's src/make.tcl under the built punk executable's 'script' subcommand with
#output captured through a pipe, and pins:
# - 'tool' (bare = list) exits 0, output is ESC-free (G-113 piped policy), the
# stable header row is present, the punkzip row carries the declared zig floor
# (>=0.16.0), an installed-binary state from the documented vocabulary and a
# 12-hex vendored-commit column, and the toolchain line reports either a
# resolved zig or the documented no-suitable-toolchain guidance (the test must
# pass on machines without a 0.16 toolchain)
# - 'tool info punkzip' exits 0 and reports the PROVENANCE.md-sourced records
# (upstream, full 40-hex vendored commit) and the provenance file path
# - 'tool <unknown-action>' exits 2 naming the action and the expected set
# - 'tool info <unknown-name>' exits 2 naming the tool and the configured names
#The build/test actions are deliberately NOT exercised here (they need a zig
#toolchain and minutes of compile time); the tree's own 'zig build test' is the
#gate for those, driven by 'make.tcl tool build/test'.
#The punkzip row pins are characterization of the current vendored tree - a
#re-vendor with a raised floor legitimately updates them.
#
#Target executable resolved from env(PUNK_SHELL_TEST_EXE), else <projectroot>/bin/punk902z.exe
#then <projectroot>/bin/punkshell902. Skipped (constraint punkexeavailable) if none found.
namespace eval ::testspace {
namespace import ::tcltest::*
variable testdir [file dirname [file normalize [info script]]]
#<projectroot>/src/tests/shell/testsuites/punkexe -> 5 levels up to <projectroot>
variable projectroot [file normalize [file join $testdir .. .. .. .. ..]]
variable maketcl [file join $projectroot src make.tcl]
variable punkexe ""
if {[info exists ::env(PUNK_SHELL_TEST_EXE)] && $::env(PUNK_SHELL_TEST_EXE) ne ""} {
set punkexe [file normalize $::env(PUNK_SHELL_TEST_EXE)]
} else {
foreach candidate [list [file join $projectroot bin punk902z.exe] [file join $projectroot bin punkshell902]] {
if {[file exists $candidate]} {
set punkexe $candidate
break
}
}
}
testConstraint punkexeavailable [expr {$punkexe ne "" && [file exists $punkexe]}]
variable maketcl_run_timeout_ms 60000
variable runstate
array set runstate {}
proc maketcl_run_read {chan} {
variable runstate
append runstate(output) [read $chan]
if {[chan eof $chan]} {
chan event $chan readable {}
set runstate(done) eof
}
}
#Run <punkexe> script src/make.tcl <subcommand...> with output captured through a
#pipe (stdin half-closed for immediate EOF - make.tcl must never wait on stdin for
#these subcommands). Returns dict: timedout 0|1, exitcode <int|"">, output
#<combined stdout+stderr>.
proc maketcl_run {cmdargs} {
variable runstate
variable maketcl_run_timeout_ms
variable punkexe
variable maketcl
array unset runstate
set runstate(output) ""
set runstate(done) ""
set chan [open |[list $punkexe script $maketcl {*}$cmdargs 2>@1] r+]
chan configure $chan -blocking 0 -translation binary
catch {chan close $chan write} ;#no stdin for the child - immediate EOF
set timerid [after $maketcl_run_timeout_ms [list set [namespace current]::runstate(done) timeout]]
chan event $chan readable [list [namespace current]::maketcl_run_read $chan]
while {$runstate(done) eq ""} {
vwait [namespace current]::runstate(done)
}
after cancel $timerid
set timedout [expr {$runstate(done) eq "timeout"}]
set exitcode ""
if {$timedout} {
catch {exec {*}[auto_execok taskkill] /F /T /PID [lindex [pid $chan] 0]}
catch {chan close $chan}
} else {
chan configure $chan -blocking 1
if {[catch {chan close $chan} errdata errdict]} {
set exitcode [lindex [dict get $errdict -errorcode] end]
} else {
set exitcode 0
}
}
return [dict create timedout $timedout exitcode $exitcode output $runstate(output)]
}
proc esc_count {text} {
return [regexp -all {\x1b} $text]
}
#added 2026-07-27 (agent, G-126)
test maketcl_tool_list {piped make.tcl tool (bare = list): exit 0, ESC-free, header + punkzip row + toolchain line} -constraints {punkexeavailable} -body {
set r [maketcl_run {tool}]
set out [dict get $r output]
set result [list]
lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode]
lappend result esc [esc_count $out]
lappend result header [regexp {(?n)^ name\s+version\s+zig-floor\s+bin\s+vendored-commit\s*$} $out]
lappend result punkziprow [regexp {(?n)^ punkzip\s+\S+\s+>=0\.16\.0\s+(current|stale|absent)\s+[0-9a-f]{12}\s*$} $out]
lappend result toolchain [regexp {(?n)^toolchain: (zig \S+ via |no suitable zig toolchain)} $out]
lappend result optionalnote [regexp {zig is OPTIONAL for punkshell builds} $out]
set result
} -result {timedout 0 exitcode 0 esc 0 header 1 punkziprow 1 toolchain 1 optionalnote 1}
#added 2026-07-27 (agent, G-126)
test maketcl_tool_info_punkzip {tool info punkzip reports the PROVENANCE.md records and paths} -constraints {punkexeavailable} -body {
set r [maketcl_run {tool info punkzip}]
set out [dict get $r output]
set result [list]
lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode]
lappend result toolline [regexp {(?n)^tool: punkzip\s*$} $out]
lappend result upstream [regexp {(?n)^ upstream:\s+\S+} $out]
lappend result commit [regexp {(?n)^ vendored commit: [0-9a-f]{40}\s*$} $out]
lappend result provfile [regexp {(?n)^ provenance:\s+\S+PROVENANCE\.md\s*$} $out]
lappend result floor [regexp {(?n)^ zig floor:\s+>= 0\.16\.0\s*$} $out]
set result
} -result {timedout 0 exitcode 0 toolline 1 upstream 1 commit 1 provfile 1 floor 1}
#added 2026-07-27 (agent, G-126)
test maketcl_tool_unknown_action {tool with an unknown action exits 2 naming it and the expected actions} -constraints {punkexeavailable} -body {
set r [maketcl_run {tool frobnicate}]
set out [dict get $r output]
set result [list]
lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode]
lappend result named [regexp {unknown tool action 'frobnicate' - expected list\|info\|build\|test} $out]
set result
} -result {timedout 0 exitcode 2 named 1}
#added 2026-07-27 (agent, G-126)
test maketcl_tool_unknown_name {tool info with an unknown tool name exits 2 listing the configured tools} -constraints {punkexeavailable} -body {
set r [maketcl_run {tool info nosuchtoolxyz}]
set out [dict get $r output]
set result [list]
lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode]
lappend result named [regexp {tool 'nosuchtoolxyz' not found under } $out]
lappend result confignames [regexp {configured tools: .*punkzip} $out]
set result
} -result {timedout 0 exitcode 2 named 1 confignames 1}
cleanupTests
}
namespace delete ::testspace
Loading…
Cancel
Save