From d76a7907ea5875a36c9f025c9178326862f20b99 Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Mon, 27 Jul 2026 18:20:07 +1000 Subject: [PATCH] make.tcl tool subcommand: build vendored zig tools into bin/ (G-126); 0.28.3 New surface for the vendored first-party build tools under src/tools: make.tcl tool list|info|build|test ? ...? ?-test 0|1? - discovery is convention-based (a src/tools directory carrying build.zig is a tool), so the G-128 tool will appear with zero make.tcl edits; per-tool records are read tolerantly from build.zig.zon (version, minimum_zig_version floor) and PROVENANCE.md (upstream, vendored commit) - list/info report installed-binary state (current|stale|absent vs bin/) and the resolved toolchain - toolchain resolution: PUNK_ZIG= overrides; otherwise bin/tools/zig* is scanned and the LOWEST release satisfying the floor wins - deterministic as newer toolchains get unpacked beside the pinned one; dev builds count only when their base version exceeds the floor - build runs the tree's own 'zig build test' as an EXIT-CODE gate (expected stderr warnings do not fail it), then builds ReleaseSafe and installs to bin/; nothing installs when the gate or build fails; -test 0 skips the gate; build caches are deliberately left in place between runs for rebuild speed (git- and fossil-ignored) - zig stays OPTIONAL: packages/bake never require the step; without a suitable toolchain, list reports the state and build/test exit nonzero with bin/punk-getzig.cmd guidance - full subcommand accompaniments per src/AGENTS.md: punk::args argdoc (SUMMARIES/HELPTEXTS/bespoke passthrough definition), SUBGROUPS ("vendored tools"), known_commands, degraded-mode dispatch, plain help text, bootsupport-staleness exemption, workflow text (new OUT-OF-BAND SUBSYSTEMS section also covering buildsuite), project version 0.28.3 + CHANGELOG entry - piped characterization (zig-independent): src/tests/shell/testsuites/punkexe/maketcltool.test - 4/4 green - verified live: 'tool build punkzip' gated on the vendored tree's 130/130 suite and installed bin/punkzip.exe (ReleaseSafe, v2.3.1, 1.1MB) over the stale Nov-2024 v2.1.0 binary; list state flipped stale -> current; PUNKBOOT_PLAIN degraded dispatch and tabled help verified G-126 Progress updated: the acceptance's build-step clause is satisfied. Remaining: punk::zip fast path, parity suite, benchmark. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- CHANGELOG.md | 13 + goals/G-126-punkzip-accelerator.md | 33 +- punkproject.toml | 2 +- src/make.tcl | 411 +++++++++++++++++- .../shell/testsuites/punkexe/maketcltool.test | 154 +++++++ 5 files changed, 603 insertions(+), 10 deletions(-) create mode 100644 src/tests/shell/testsuites/punkexe/maketcltool.test diff --git a/CHANGELOG.md b/CHANGELOG.md index ddc89d2f..46bfa459 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,19 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` "Project Versioning" section for the bump policy. +## [0.28.3] - 2026-07-27 + +- New `make.tcl tool` subcommand (G-126): surface for the vendored first-party zig + build tools under src/tools (punkzip first). `tool list|info` report each tool's + version, declared zig floor, installed-binary state and vendored upstream commit + (from its PROVENANCE.md); `tool build ?name ...? ?-test 0|1?` resolves a pinned + toolchain (PUNK_ZIG override, else the lowest satisfying release under + bin/tools/zig*), runs the tree's own 'zig build test' as an exit-code gate, builds + ReleaseSafe and installs to bin/. zig stays optional: packages/bake never require + the step, and without a suitable toolchain the state is reported (build/test exit + nonzero with bin/punk-getzig.cmd guidance). Piped characterization: + src/tests/shell/testsuites/punkexe/maketcltool.test. + ## [0.28.2] - 2026-07-27 - Bakes verify payload/target consistency (G-133, both checks advisory - recapped diff --git a/goals/G-126-punkzip-accelerator.md b/goals/G-126-punkzip-accelerator.md index 88689520..3fb94cb9 100644 --- a/goals/G-126-punkzip-accelerator.md +++ b/goals/G-126-punkzip-accelerator.md @@ -256,10 +256,29 @@ working tree; one now exists. extended for the vendored fixtures (hamlet.txt is CRLF byte-exact test data). The acceptance's vendoring clause (origin, state, re-vendor procedure, licensing) is satisfied. -- Remaining for acceptance: make.tcl tool build step producing `bin/punkzip.exe` - reproducibly on a clean checkout (zig OPTIONAL - absence reported, not fatal; test - gate keys on exit code, not clean stderr - the zip suite's negative tests print - EOCD warnings while passing; step design shared with the G-128 tool); punk::zip - accelerator detection + fast path behind the G-124 floor; parity suite in - `src/tests/modules/punk/zip/`; recorded benchmark (speedup + entry count where it - becomes material). +- 2026-07-27 make.tcl TOOL STEP landed (user-directed; punkshell 0.28.3): new + `make.tcl tool list|info|build|test ? ...? ?-test 0|1?` subcommand. + Convention-based discovery (a directory under src/tools carrying build.zig is a + tool - the G-128 tool will appear with zero make.tcl edits); per-tool records read + tolerantly from build.zig.zon (version, zig floor) and PROVENANCE.md (upstream, + vendored commit); installed-state reporting (current|stale|absent vs + bin/); toolchain resolution = PUNK_ZIG override, else a scan of + bin/tools/zig* choosing the LOWEST release satisfying the floor (deterministic as + newer toolchains appear beside the pin; dev builds count only above the floor); + 'zig build test' EXIT-CODE gate before a ReleaseSafe build and install to bin/ + (nothing installs on gate/build failure). zig-OPTIONAL honoured: packages/bake + never touch the step; without a suitable toolchain list reports the state and + build/test exit nonzero with punk-getzig guidance. Verified live: gate + build + installed bin/punkzip.exe (ReleaseSafe, 1.1MB, v2.3.1) replacing the stale + Nov-2024 v2.1.0 binary, state flipped stale->current; degraded PUNKBOOT_PLAIN + dispatch and tabled help verified. Full subcommand accompaniments: argdoc + + SUBGROUPS + known_commands + plain help + workflow-text OUT-OF-BAND SUBSYSTEMS + section + piped characterization + src/tests/shell/testsuites/punkexe/maketcltool.test (4/4, zig-independent). + Build caches (.zig-cache/zig-out) left in place between runs for rebuild speed + (user decision 2026-07-27). The acceptance's build-step clause is satisfied: + bin/punkzip.exe is reproducible on a clean checkout from vendored source + pinned + toolchain + this step. +- Remaining for acceptance: punk::zip accelerator detection + fast path behind the + G-124 floor; parity suite in `src/tests/modules/punk/zip/`; recorded benchmark + (speedup + entry count where it becomes material). diff --git a/punkproject.toml b/punkproject.toml index 2bab8e25..e80e9765 100644 --- a/punkproject.toml +++ b/punkproject.toml @@ -1,4 +1,4 @@ [project] name = "punkshell" -version = "0.28.2" +version = "0.28.3" license = "BSD-2-Clause" diff --git a/src/make.tcl b/src/make.tcl index b3263dba..0658f3c4 100644 --- a/src/make.tcl +++ b/src/make.tcl @@ -187,7 +187,7 @@ namespace eval ::punkboot { variable foldername [file tail $scriptfolder] variable pkg_requirements [list]; variable pkg_missing [list];variable pkg_loaded [list] variable help_flags [list -help --help /? -h] - variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate bootsupport vfscommonupdate projectversion workflow buildsuite project vfs] + variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate bootsupport vfscommonupdate projectversion workflow buildsuite tool project vfs] } @@ -2152,6 +2152,11 @@ proc ::punkboot::punkboot_gethelp {args} { append h " $scriptname buildsuite list|info|build ?? ?driver-args ...?" \n append h " - list the defined buildsuites under src/buildsuites (zig runtime factory), show a suite's configured" \n append h " detail (sources/zig pin/products from its sources.config), or run its driver forwarding the args" \n \n + append h " $scriptname tool list|info|build|test ? ...? ?-test 0|1?" \n + append h " - vendored first-party build tools under src/tools (zig source; G-126): list/detail them, run their" \n + append h " test suites, or build (test-gated) and install to /bin" \n + append h " - zig is OPTIONAL: packages/bake never require this step; without a suitable toolchain the state is" \n + append h " reported and build/test exit nonzero with fetch guidance (bin/punk-getzig.cmd)" \n \n append h " $scriptname help ?subcommand?" \n append h " - show usage for $scriptname or one of its subcommands (equivalent: $scriptname -help)" \n \n append h " Deprecated aliases (transition window):" \n @@ -2402,6 +2407,14 @@ KEY / NOTES with binary dependencies, and version-preference outcomes are visible only to the smoke probe - and only for declared packages. 'make.tcl check' reports both. +OUT-OF-BAND SUBSYSTEMS (not part of the packages/bake data flow above) + 'make.tcl tool' builds the vendored first-party zig tools under src/tools + (G-126; punkzip first) into /bin, gated on each tree's own + 'zig build test' exit code. zig is OPTIONAL: packages/bake never require it. + 'make.tcl buildsuite' is the analogous arm's-length factory for whole Tcl + runtimes built from EXTERNAL sources under src/buildsuites (fetch + test-gate + contract per suite; products land in the bin/runtime store tiers). + MAINTENANCE (agents take note) ---------------------------------------------------------------------------- This text is embedded in make.tcl (::punkboot::workflow_text). When the build @@ -2514,6 +2527,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO shell "Run the punk shell using bootsupport libraries" help "Show usage for make.tcl or one of its subcommands" buildsuite "List, describe or run the defined buildsuites under src/buildsuites (zig runtime factory)" + tool "List, test or build the vendored tools under src/tools (zig) and install them to /bin" project "DEPRECATED alias for bakehouse (one-line notice, then runs bakehouse)" vfs "DEPRECATED alias for bake (one-line notice, then runs bake)" } @@ -2683,6 +2697,45 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO (copy-and-tweak workflow) appears here automatically with no make.tcl edits. The contract is documented in src/buildsuites/README.md." + tool + " + Surface for the vendored first-party build tools under src/tools + (G-126: zig source vendored in full; boundary and editing policy in + src/tools/AGENTS.md, per-tree origin/licence records in + /PROVENANCE.md). Discovery is convention-based: a directory + under src/tools carrying build.zig is a tool; its installed + artifact is /bin/ with the host executable + suffix. + + Actions: + tool list + Discover the tools with version (build.zig.zon), declared zig + floor, installed-binary state (current|stale|absent) and the + vendored upstream commit; report the resolved toolchain. + tool info ... + Per-tool detail: paths, provenance records, toolchain + resolution. + tool build ? ...? ?-test 0|1? + Resolve a zig satisfying the tool's declared floor, run the + 'zig build test' gate (default on; the gate is the EXIT CODE - + expected warnings on stderr do not fail it), build ReleaseSafe + and install to /bin. Nothing is installed when the + gate or the build fails. With no name, all tools are + processed. + tool test ? ...? + Run each tool's 'zig build test' only. + + Toolchain resolution: PUNK_ZIG= + overrides; otherwise /bin/tools/zig* is scanned and + the LOWEST release satisfying the floor wins (deterministic as + newer toolchains appear beside the pinned one; dev builds count + only when their base version exceeds the floor). zig is OPTIONAL + for punkshell builds: packages/bake never require this step - + without a suitable toolchain, list reports the state and + build/test exit nonzero with fetch guidance + (bin/punk-getzig.cmd). Build caches (.zig-cache/zig-out) are + deliberately left in place between runs for rebuild speed (both + git- and fossil-ignored)." project " DEPRECATED alias for 'bakehouse' (G-112 stage-true rename) - prints a @@ -2855,6 +2908,15 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO arg -type any -optional 1 -multiple 1 -help\ "Action (list|info|build), suite name, and - for build - driver arguments forwarded to the suite's suite.tcl untouched."} + punk::args::define\ + "@id -id (script)::punkboot::tool"\ + "@normalize"\ + "@cmd -name [list "make.tcl tool"] -summary [list [dict get $SUMMARIES tool]] -help [list [dict get $HELPTEXTS tool]]"\ + {@form -synopsis "make.tcl tool list|info|build|test ? ...? ?-test 0|1?"}\ + {@values -min 0 -max -1 + arg -type any -optional 1 -multiple 1 -help\ + "Action (list|info|build|test), tool name(s), and - for build - + the -test 0|1 gate flag. Parsed by the handler."} punk::args::define\ "@id -id (script)::punkboot::help"\ "@normalize"\ @@ -2871,6 +2933,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "promotion gates" {bootsupport vfscommonupdate} "source maintenance" {vendorupdate} "buildsuites" {buildsuite} + "vendored tools" {tool} "informational" {info check bakelist projectversion workflow help} "interactive" {shell} "deprecated aliases" {project vfs} @@ -2963,6 +3026,10 @@ if {$::punkboot::punkargs_ok} { #args (arbitrary suite flags) are forwarded untouched (G-104) set ::punkboot::command buildsuite set ::punkboot::bs_args $subargs + } elseif {$subcommand eq "tool"} { + #declared passthrough: the handler parses action/names/-test itself (G-126) + set ::punkboot::command tool + set ::punkboot::tool_args $subargs } else { if {[catch {punk::args::parse $subargs -errorstyle $::punkboot::errstyle withid (script)::punkboot::$subcommand} argd]} { puts stderr $argd @@ -2993,6 +3060,12 @@ if {$::punkboot::punkargs_ok} { set commands_found [list buildsuite] set argsleft [list] set ::punkboot::bs_args [lrange $scriptargs 1 end] + } elseif {[lindex $scriptargs 0] eq "tool"} { + #declared passthrough (as in the punk::args path) - the handler parses + #action/names/-test itself (G-126) + set commands_found [list tool] + set argsleft [list] + set ::punkboot::tool_args [lrange $scriptargs 1 end] } while {[llength $argsleft]} { set a [lindex $argsleft 0] @@ -3116,7 +3189,8 @@ set forcekill $::punkboot::opt_forcekill # - patch bump -> warn and proceed (non-breaking) # 'check' is exempt in all cases — the warning is shown at the end of its output instead. # 'projectversion', 'workflow' and 'bakelist' are informational/read-only and are likewise exempt. -if {[info exists ::punkboot::stale_bootsupport] && $::punkboot::command ni {check projectversion workflow bakelist buildsuite}} { +# 'buildsuite' and 'tool' are arm's-length subsystems that do not consume bootsupport modules. +if {[info exists ::punkboot::stale_bootsupport] && $::punkboot::command ni {check projectversion workflow bakelist buildsuite tool}} { set _stale $::punkboot::stale_bootsupport set _have_major 0 set _have_minor 0 @@ -4242,6 +4316,339 @@ if {$::punkboot::command eq "buildsuite"} { } } +if {$::punkboot::command eq "tool"} { + #G-126: vendored first-party build tools under src/tools (zig source vendored in + #full - boundary and editing policy in src/tools/AGENTS.md; per-tree origin, + #re-vendor procedure and licence records in /PROVENANCE.md). Discovery is + #convention-based: a directory under src/tools carrying build.zig is a tool, its + #name is the directory name, its build artifact is /zig-out/bin/ + #and its installed copy is /bin/ (host executable suffix). + #zig stays OPTIONAL for punkshell builds: nothing in packages/bake depends on + #this step; explicit build/test without a suitable toolchain exits nonzero with + #fetch guidance. Build caches (.zig-cache/zig-out) are deliberately left in + #place between builds for rebuild speed (both git- and fossil-ignored). + set tool_root [file join $sourcefolder tools] + if {![info exists ::punkboot::tool_args]} {set ::punkboot::tool_args [list]} + set targs $::punkboot::tool_args + set tool_action [lindex $targs 0] + if {$tool_action eq ""} {set tool_action list} + #args after the action: tool names plus an optional '-test 0|1' flag (build) + set tool_dotest 1 + set tool_names [list] + set _i 1 + while {$_i < [llength $targs]} { + set a [lindex $targs $_i] + if {$a eq "-test"} { + incr _i + set v [lindex $targs $_i] + if {![string is boolean -strict $v]} { + puts stderr "make.tcl tool: -test requires a boolean value (got '$v')" + exit 2 + } + set tool_dotest [expr {bool($v)}] + } else { + lappend tool_names $a + } + incr _i + } + unset -nocomplain _i a v + + proc ::punkboot::tool_dirs {tool_root} { + set dirs [list] + foreach dir [lsort [glob -nocomplain -dir $tool_root -type d *]] { + if {![file exists [file join $dir build.zig]]} {continue} ;#not a tool + lappend dirs $dir + } + return $dirs + } + #tolerant build.zig.zon reader: tool version + declared minimum zig version + proc ::punkboot::tool_zoninfo {tooldir} { + set m [dict create version "" zigfloor ""] + if {![catch { + set f [open [file join $tooldir build.zig.zon] r] + set data [read $f] + close $f + }]} { + if {[regexp {\.version\s*=\s*"([^"]+)"} $data -> _v]} {dict set m version $_v} + if {[regexp {\.minimum_zig_version\s*=\s*"([^"]+)"} $data -> _fl]} {dict set m zigfloor $_fl} + } + return $m + } + #tolerant PROVENANCE.md reader: vendored upstream commit + upstream location + proc ::punkboot::tool_provenance {tooldir} { + set m [dict create commit "" upstream ""] + if {![catch { + set f [open [file join $tooldir PROVENANCE.md] r] + set data [read $f] + close $f + }]} { + if {[regexp {Vendored state: commit\s+`?([0-9a-fA-F]{7,40})} $data -> _c]} {dict set m commit $_c} + if {[regexp {Upstream repository:\s+`?([^`\n]+)} $data -> _u]} {dict set m upstream [string trim $_u]} + } + return $m + } + proc ::punkboot::tool_newest_src_mtime {dir} { + set newest 0 + foreach f [glob -nocomplain -dir $dir -type f *] { + if {[file mtime $f] > $newest} {set newest [file mtime $f]} + } + foreach d [glob -nocomplain -dir $dir -type d *] { + if {[file tail $d] in {zig-out .zig-cache zig-cache}} {continue} + set sub [::punkboot::tool_newest_src_mtime $d] + if {$sub > $newest} {set newest $sub} + } + return $newest + } + proc ::punkboot::tool_zig_version {zigexe} { + if {[catch {exec $zigexe version 2>@1} v]} {return ""} + return [string trim [lindex [split $v \n] 0]] + } + #a dev build's base version must EXCEED the floor (e.g 0.16.0-dev < 0.16.0) + proc ::punkboot::tool_zig_satisfies {ver floor} { + if {$floor eq ""} {return 1} + set base $ver ; set isdev 0 + if {[regexp {^([0-9][0-9.]*)[-+]} $ver -> b]} {set base $b ; set isdev 1} + if {[catch {package vcompare $base $floor} cmp]} {return 0} + if {$cmp > 0} {return 1} + if {$cmp == 0 && !$isdev} {return 1} + return 0 + } + #resolve a zig for the given floor: PUNK_ZIG env override (path to the zig + #executable or its folder), else scan /bin/tools/zig* preferring + #releases over dev builds and the LOWEST satisfying version - a deterministic + #choice that stays stable as newer toolchains get unpacked beside the pinned one + proc ::punkboot::tool_resolve_zig {projectroot floor} { + if {[info exists ::env(PUNK_ZIG)] && $::env(PUNK_ZIG) ne ""} { + set cand $::env(PUNK_ZIG) + if {[file isdirectory $cand]} { + foreach n {zig.exe zig} { + if {[file exists [file join $cand $n]]} {set cand [file join $cand $n] ; break} + } + } + set v [::punkboot::tool_zig_version $cand] + if {$v eq ""} { + return [dict create note "PUNK_ZIG='$::env(PUNK_ZIG)' did not answer 'zig version'"] + } + if {![::punkboot::tool_zig_satisfies $v $floor]} { + return [dict create note "PUNK_ZIG zig is $v which does not satisfy this tool's floor (>= $floor)"] + } + return [dict create zig $cand version $v via PUNK_ZIG] + } + set best "" ; set bestv "" ; set bestbase "" ; set bestdev 1 + foreach dir [lsort [glob -nocomplain -dir [file join $projectroot bin tools] -type d zig*]] { + set p "" + foreach n {zig.exe zig} { + if {[file exists [file join $dir $n]]} {set p [file join $dir $n] ; break} + } + if {$p eq ""} {continue} + set v [::punkboot::tool_zig_version $p] + if {$v eq "" || ![::punkboot::tool_zig_satisfies $v $floor]} {continue} + set base $v ; set isdev 0 + if {[regexp {^([0-9][0-9.]*)[-+]} $v -> b]} {set base $b ; set isdev 1} + set take 0 + if {$best eq ""} { + set take 1 + } elseif {$bestdev && !$isdev} { + set take 1 + } elseif {$bestdev == $isdev && [package vcompare $base $bestbase] < 0} { + set take 1 + } + if {$take} {set best $p ; set bestv $v ; set bestbase $base ; set bestdev $isdev} + } + if {$best eq ""} { + set want [expr {$floor eq "" ? "any version" : ">= $floor"}] + return [dict create note "no suitable zig toolchain ($want) under [file join $projectroot bin tools] - 'bin/punk-getzig.cmd ' fetches one; PUNK_ZIG= overrides"] + } + return [dict create zig $best version $bestv via scan] + } + proc ::punkboot::tool_installed_exe {projectroot name} { + set suffix [punkboot::lib::platform_exe_suffix $::punkboot::target_platform] + return [file join $projectroot bin ${name}$suffix] + } + proc ::punkboot::tool_install_state {projectroot tooldir} { + set exe [::punkboot::tool_installed_exe $projectroot [file tail $tooldir]] + if {![file exists $exe]} {return absent} + if {[file mtime $exe] >= [::punkboot::tool_newest_src_mtime $tooldir]} {return current} + return stale + } + #run 'zig ' with cwd at the tool tree (the tree's CLI tests resolve + #fixtures and zig-out relative to the build root). The gate is the EXIT CODE: + #stderr legitimately carries expected warnings (punkzip's zip suite negative + #tests print EOCD warnings while passing). Returns the child's exit status. + proc ::punkboot::tool_zig_run {zig tooldir argl} { + set saved [pwd] + cd $tooldir + set rc 0 + if {[catch {exec $zig {*}$argl >@stdout 2>@stderr} _e _o]} { + set ecode [dict get $_o -errorcode] + if {[lindex $ecode 0] eq "CHILDSTATUS"} { + set rc [lindex $ecode 2] + } else { + puts stderr "tool: failed to run '$zig $argl' in $tooldir: $_e" + set rc 1 + } + } + cd $saved + return $rc + } + + set alldirs [::punkboot::tool_dirs $tool_root] + set byname [dict create] + foreach d $alldirs {dict set byname [file tail $d] $d} + if {[llength $tool_names]} { + set selected [list] + foreach n $tool_names { + if {![dict exists $byname $n]} { + puts stderr "tool '$n' not found under $tool_root - configured tools: [dict keys $byname]" + exit 2 + } + lappend selected [dict get $byname $n] + } + } else { + set selected $alldirs + } + if {![llength $alldirs] && $tool_action ne "list"} { + puts stderr "no vendored tools found under $tool_root (a tool = a directory carrying build.zig)" + exit 2 + } + + switch -- $tool_action { + list { + if {![llength $alldirs]} { + puts stdout "no vendored tools found under $tool_root (a tool = a directory carrying build.zig)" + exit 0 + } + puts stdout "vendored tools under $tool_root (boundary: src/tools/AGENTS.md; per-tree PROVENANCE.md):" + puts stdout [format " %-10s %-9s %-10s %-8s %s" name version zig-floor bin vendored-commit] + set maxfloor "" + foreach dir $alldirs { + set name [file tail $dir] + set zon [::punkboot::tool_zoninfo $dir] + set prov [::punkboot::tool_provenance $dir] + set floor [dict get $zon zigfloor] + if {$floor ne "" && ($maxfloor eq "" || [package vcompare $floor $maxfloor] > 0)} { + set maxfloor $floor + } + set commit [dict get $prov commit] + set commit [expr {$commit eq "" ? "-" : [string range $commit 0 11]}] + set floorshow [expr {$floor eq "" ? "-" : ">=$floor"}] + puts stdout [format " %-10s %-9s %-10s %-8s %s" $name [dict get $zon version] $floorshow [::punkboot::tool_install_state $projectroot $dir] $commit] + } + set zres [::punkboot::tool_resolve_zig $projectroot $maxfloor] + if {[dict exists $zres zig]} { + puts stdout "toolchain: zig [dict get $zres version] via [dict get $zres via] ([dict get $zres zig])" + } else { + puts stdout "toolchain: [dict get $zres note]" + } + puts stdout "" + puts stdout " - 'make.tcl tool info ' shows a tool's detail" + puts stdout " - 'make.tcl tool build ? ...? ?-test 0|1?' builds (test-gated) and installs to bin/" + puts stdout " - zig is OPTIONAL for punkshell builds: packages/bake never require this step" + exit 0 + } + info { + if {![llength $tool_names]} { + puts stderr "usage: make.tcl tool info ('make.tcl tool list' shows the names)" + exit 2 + } + foreach dir $selected { + set name [file tail $dir] + set zon [::punkboot::tool_zoninfo $dir] + set prov [::punkboot::tool_provenance $dir] + set floor [dict get $zon zigfloor] + puts stdout "tool: $name" + puts stdout " path: $dir" + puts stdout " version: [dict get $zon version] (build.zig.zon)" + puts stdout " zig floor: [expr {$floor eq "" ? "(none declared)" : ">= $floor"}]" + puts stdout " upstream: [expr {[dict get $prov upstream] eq "" ? "(no PROVENANCE.md record)" : [dict get $prov upstream]}]" + puts stdout " vendored commit: [expr {[dict get $prov commit] eq "" ? "(no PROVENANCE.md record)" : [dict get $prov commit]}]" + puts stdout " provenance: [file join $dir PROVENANCE.md]" + set exe [::punkboot::tool_installed_exe $projectroot $name] + puts stdout " installed: $exe ([::punkboot::tool_install_state $projectroot $dir])" + set zres [::punkboot::tool_resolve_zig $projectroot $floor] + if {[dict exists $zres zig]} { + puts stdout " toolchain: zig [dict get $zres version] via [dict get $zres via] ([dict get $zres zig])" + } else { + puts stdout " toolchain: [dict get $zres note]" + } + } + exit 0 + } + build { + foreach dir $selected { + set name [file tail $dir] + set floor [dict get [::punkboot::tool_zoninfo $dir] zigfloor] + set zres [::punkboot::tool_resolve_zig $projectroot $floor] + if {![dict exists $zres zig]} { + puts stderr "tool build $name: [dict get $zres note]" + exit 1 + } + set zig [dict get $zres zig] + puts stdout "tool build $name: zig [dict get $zres version] via [dict get $zres via] ($zig)" + flush stdout + if {$tool_dotest} { + puts stdout "tool build $name: test gate ('zig build test') ..." + flush stdout + set rc [::punkboot::tool_zig_run $zig $dir [list build test]] + if {$rc != 0} { + puts stderr "tool build $name: test gate FAILED (exit $rc) - nothing installed" + exit 1 + } + puts stdout "tool build $name: test gate passed" + } else { + puts stdout "tool build $name: test gate SKIPPED (-test 0)" + } + puts stdout "tool build $name: building (ReleaseSafe) ..." + flush stdout + set rc [::punkboot::tool_zig_run $zig $dir [list build -Doptimize=ReleaseSafe]] + if {$rc != 0} { + puts stderr "tool build $name: build FAILED (exit $rc) - nothing installed" + exit 1 + } + set suffix [punkboot::lib::platform_exe_suffix $::punkboot::target_platform] + set built [file join $dir zig-out bin ${name}$suffix] + if {![file exists $built]} { + puts stderr "tool build $name: build reported success but $built is missing" + exit 1 + } + set exe [::punkboot::tool_installed_exe $projectroot $name] + if {[catch {file copy -force $built $exe} cperr]} { + puts stderr "tool build $name: could not install $exe ($cperr) - is a copy currently running?" + exit 1 + } + puts stdout "tool build $name: installed $exe ([file size $exe] bytes)" + } + exit 0 + } + test { + foreach dir $selected { + set name [file tail $dir] + set floor [dict get [::punkboot::tool_zoninfo $dir] zigfloor] + set zres [::punkboot::tool_resolve_zig $projectroot $floor] + if {![dict exists $zres zig]} { + puts stderr "tool test $name: [dict get $zres note]" + exit 1 + } + set zig [dict get $zres zig] + puts stdout "tool test $name: zig [dict get $zres version] via [dict get $zres via] - running 'zig build test' ..." + flush stdout + set rc [::punkboot::tool_zig_run $zig $dir [list build test]] + if {$rc != 0} { + puts stderr "tool test $name: FAILED (exit $rc)" + exit 1 + } + puts stdout "tool test $name: passed" + } + exit 0 + } + default { + puts stderr "unknown tool action '$tool_action' - expected list|info|build|test" + puts stderr "usage: make.tcl tool list|info|build|test ? ...? ?-test 0|1?" + exit 2 + } + } +} + if {$::punkboot::command eq "shell"} { #G-113: the ansistrip transform serves make.tcl's own output only - the repl's #colour behaviour is the shell's own concern (colour on/off + NO_COLOR there). diff --git a/src/tests/shell/testsuites/punkexe/maketcltool.test b/src/tests/shell/testsuites/punkexe/maketcltool.test new file mode 100644 index 00000000..2283b291 --- /dev/null +++ b/src/tests/shell/testsuites/punkexe/maketcltool.test @@ -0,0 +1,154 @@ +package require tcltest + +#Piped characterization of the make.tcl tool subcommand (goal G-126): the surface +#for the vendored first-party zig build tools under src/tools. Runs the WORKING +#TREE's src/make.tcl under the built punk executable's 'script' subcommand with +#output captured through a pipe, and pins: +# - 'tool' (bare = list) exits 0, output is ESC-free (G-113 piped policy), the +# stable header row is present, the punkzip row carries the declared zig floor +# (>=0.16.0), an installed-binary state from the documented vocabulary and a +# 12-hex vendored-commit column, and the toolchain line reports either a +# resolved zig or the documented no-suitable-toolchain guidance (the test must +# pass on machines without a 0.16 toolchain) +# - 'tool info punkzip' exits 0 and reports the PROVENANCE.md-sourced records +# (upstream, full 40-hex vendored commit) and the provenance file path +# - 'tool ' exits 2 naming the action and the expected set +# - 'tool info ' exits 2 naming the tool and the configured names +#The build/test actions are deliberately NOT exercised here (they need a zig +#toolchain and minutes of compile time); the tree's own 'zig build test' is the +#gate for those, driven by 'make.tcl tool build/test'. +#The punkzip row pins are characterization of the current vendored tree - a +#re-vendor with a raised floor legitimately updates them. +# +#Target executable resolved from env(PUNK_SHELL_TEST_EXE), else /bin/punk902z.exe +#then /bin/punkshell902. Skipped (constraint punkexeavailable) if none found. + +namespace eval ::testspace { + namespace import ::tcltest::* + + variable testdir [file dirname [file normalize [info script]]] + #/src/tests/shell/testsuites/punkexe -> 5 levels up to + variable projectroot [file normalize [file join $testdir .. .. .. .. ..]] + variable maketcl [file join $projectroot src make.tcl] + + variable punkexe "" + if {[info exists ::env(PUNK_SHELL_TEST_EXE)] && $::env(PUNK_SHELL_TEST_EXE) ne ""} { + set punkexe [file normalize $::env(PUNK_SHELL_TEST_EXE)] + } else { + foreach candidate [list [file join $projectroot bin punk902z.exe] [file join $projectroot bin punkshell902]] { + if {[file exists $candidate]} { + set punkexe $candidate + break + } + } + } + testConstraint punkexeavailable [expr {$punkexe ne "" && [file exists $punkexe]}] + + variable maketcl_run_timeout_ms 60000 + + variable runstate + array set runstate {} + + proc maketcl_run_read {chan} { + variable runstate + append runstate(output) [read $chan] + if {[chan eof $chan]} { + chan event $chan readable {} + set runstate(done) eof + } + } + + #Run script src/make.tcl with output captured through a + #pipe (stdin half-closed for immediate EOF - make.tcl must never wait on stdin for + #these subcommands). Returns dict: timedout 0|1, exitcode , output + #. + proc maketcl_run {cmdargs} { + variable runstate + variable maketcl_run_timeout_ms + variable punkexe + variable maketcl + array unset runstate + set runstate(output) "" + set runstate(done) "" + + set chan [open |[list $punkexe script $maketcl {*}$cmdargs 2>@1] r+] + chan configure $chan -blocking 0 -translation binary + catch {chan close $chan write} ;#no stdin for the child - immediate EOF + set timerid [after $maketcl_run_timeout_ms [list set [namespace current]::runstate(done) timeout]] + chan event $chan readable [list [namespace current]::maketcl_run_read $chan] + while {$runstate(done) eq ""} { + vwait [namespace current]::runstate(done) + } + after cancel $timerid + set timedout [expr {$runstate(done) eq "timeout"}] + set exitcode "" + if {$timedout} { + catch {exec {*}[auto_execok taskkill] /F /T /PID [lindex [pid $chan] 0]} + catch {chan close $chan} + } else { + chan configure $chan -blocking 1 + if {[catch {chan close $chan} errdata errdict]} { + set exitcode [lindex [dict get $errdict -errorcode] end] + } else { + set exitcode 0 + } + } + return [dict create timedout $timedout exitcode $exitcode output $runstate(output)] + } + + proc esc_count {text} { + return [regexp -all {\x1b} $text] + } + + #added 2026-07-27 (agent, G-126) + test maketcl_tool_list {piped make.tcl tool (bare = list): exit 0, ESC-free, header + punkzip row + toolchain line} -constraints {punkexeavailable} -body { + set r [maketcl_run {tool}] + set out [dict get $r output] + set result [list] + lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode] + lappend result esc [esc_count $out] + lappend result header [regexp {(?n)^ name\s+version\s+zig-floor\s+bin\s+vendored-commit\s*$} $out] + lappend result punkziprow [regexp {(?n)^ punkzip\s+\S+\s+>=0\.16\.0\s+(current|stale|absent)\s+[0-9a-f]{12}\s*$} $out] + lappend result toolchain [regexp {(?n)^toolchain: (zig \S+ via |no suitable zig toolchain)} $out] + lappend result optionalnote [regexp {zig is OPTIONAL for punkshell builds} $out] + set result + } -result {timedout 0 exitcode 0 esc 0 header 1 punkziprow 1 toolchain 1 optionalnote 1} + + #added 2026-07-27 (agent, G-126) + test maketcl_tool_info_punkzip {tool info punkzip reports the PROVENANCE.md records and paths} -constraints {punkexeavailable} -body { + set r [maketcl_run {tool info punkzip}] + set out [dict get $r output] + set result [list] + lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode] + lappend result toolline [regexp {(?n)^tool: punkzip\s*$} $out] + lappend result upstream [regexp {(?n)^ upstream:\s+\S+} $out] + lappend result commit [regexp {(?n)^ vendored commit: [0-9a-f]{40}\s*$} $out] + lappend result provfile [regexp {(?n)^ provenance:\s+\S+PROVENANCE\.md\s*$} $out] + lappend result floor [regexp {(?n)^ zig floor:\s+>= 0\.16\.0\s*$} $out] + set result + } -result {timedout 0 exitcode 0 toolline 1 upstream 1 commit 1 provfile 1 floor 1} + + #added 2026-07-27 (agent, G-126) + test maketcl_tool_unknown_action {tool with an unknown action exits 2 naming it and the expected actions} -constraints {punkexeavailable} -body { + set r [maketcl_run {tool frobnicate}] + set out [dict get $r output] + set result [list] + lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode] + lappend result named [regexp {unknown tool action 'frobnicate' - expected list\|info\|build\|test} $out] + set result + } -result {timedout 0 exitcode 2 named 1} + + #added 2026-07-27 (agent, G-126) + test maketcl_tool_unknown_name {tool info with an unknown tool name exits 2 listing the configured tools} -constraints {punkexeavailable} -body { + set r [maketcl_run {tool info nosuchtoolxyz}] + set out [dict get $r output] + set result [list] + lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode] + lappend result named [regexp {tool 'nosuchtoolxyz' not found under } $out] + lappend result confignames [regexp {configured tools: .*punkzip} $out] + set result + } -result {timedout 0 exitcode 2 named 1 confignames 1} + + cleanupTests +} +namespace delete ::testspace