From 0abd07ecd7fd950b3f1692928ecd76f8d6c1eb76 Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Sun, 26 Jul 2026 00:55:20 +1000 Subject: [PATCH] G-121: make.tcl bakelist + selective bake (punkshell 0.24.0) One parsed-mapping model (::punkboot::lib::mapvfs_parse / mapvfs_kit_outputs / mapvfs_match_outputs) now backs the kit surfaces - the bake machinery, the new bakelist report and the bake/bakelist argdoc kit-name choices all consume the model, never the file format, so the G-024 toml conversion can swap the reader underneath. - bakelist ?kitname ...? (informational SUBGROUP): kit name, type, runtime with presence, vfs folder, deployed state of bin/ vs src/_build/ (current|stale|absent|nobuild; size/mtime fast path, byte-compare when in doubt), anomalies as trailing key=value notes (runtime=missing, vfs=missing, rtrev=r via the shared runtime_materialization_check core). Name filtering doubles as per-kit detail (resolved paths/sizes/mtimes). - bake ?kitname ...?: selective kit assembly - unknown names error before any build listing the configured names (flag-order hint included); selected-but- unbuildable kits (missing runtime/vfs) fail fast; runtime BUILDCOPY/caps probe/vfs cksums/vfslibs phase all narrow to the selected kits so other kits' _build/bin artifacts and punkcheck records stay untouched. Bare bake unchanged (verified full-set run). Deprecated alias vfs mirrors the values. - argdocs: SUBVALUES/VALUES_SYNOPSES mechanism, kitname choices enumerated from the mapping at define time (-choicerestricted 0, degrade-safe), block-form help values; SUMMARIES/HELPTEXTS/SUBOPTS/SUBGROUPS/known_commands/plain gethelp/workflow text updated per the update contract; degraded-mode (PUNKBOOT_PLAIN) dispatch collects trailing kit names. - mapvfs.config todo comment retired to point at G-121. - piped characterization: src/tests/shell/testsuites/punkexe/maketclbakelist.test (full-report row/vocabulary + ESC-free, filtered detail, unknown-name errors on both surfaces with the no-build guarantee) - 4 PASS; maketclcolour.test still 3 PASS. - selective-refresh verification (recorded in the goal file): forced punk91 rebuild + deploy via 'bake -confirm 0 punk91'; stat diffs over src/_build and bin show only punk91.exe, punk91.exe.vfs, raw_tclsfe-x64.exe and bin/punk91.exe changed; .punkcheck diffs mention only punk91's records (negative grep for all other kit/vfs names clean). - docs: src/AGENTS.md work-guidance bullet; ARCHITECTURE.md build-entry subcommand list refreshed (bake/bakehouse naming + bakelist, stale in-flux G-112 mention dropped); CHANGELOG 0.24.0 + punkproject.toml minor bump. Claude-Session: https://claude.ai/code/session_01Jz7wkUsknJzyuJ3tgMaL2t Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- ARCHITECTURE.md | 4 +- CHANGELOG.md | 15 + punkproject.toml | 2 +- src/AGENTS.md | 1 + src/make.tcl | 839 +++++++++++++++--- src/runtime/mapvfs.config | 6 +- .../testsuites/punkexe/maketclbakelist.test | 150 ++++ 7 files changed, 905 insertions(+), 112 deletions(-) create mode 100644 src/tests/shell/testsuites/punkexe/maketclbakelist.test diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 7ae81fcf..687b4810 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -80,7 +80,7 @@ Two layers with a deliberate dependency direction (the class never depends on th ## Build, provenance and kits -- **`src/make.tcl` is the single build entry.** Key subcommands: `modules`/`libs`/`packages` (repo-root build outputs), `vfscommonupdate` (regenerate `_vfscommon.vfs`), `vfs`, `bin`, `project` (full kit build into `bin/`), `bootsupport` (snapshot refresh), `vendorupdate`, `check`, `workflow`, `projectversion`. Dispatch and help dogfood `punk::args` (G-030, achieved) and degrade to plain scan/help when bootsupport `punk::args` is stale (`PUNKBOOT_PLAIN=1` forces the degraded mode). Source: `src/AGENTS.md`. +- **`src/make.tcl` is the single build entry.** Key subcommands: `modules`/`libs`/`packages` (repo-root build outputs), `vfscommonupdate` (regenerate `_vfscommon.vfs`), `bake` (kit assembly into `bin/`; optional kit names confine the run to those kits - G-121), `bakelist` (report the configured kit matrix with runtime/vfs presence and deployed state), `bin`, `bakehouse` (packages + bake for a clean checkout; `project`/`vfs` remain as deprecated aliases - G-112 rename, achieved), `bootsupport` (snapshot refresh), `vendorupdate`, `check`, `workflow`, `projectversion`. Dispatch and help dogfood `punk::args` (G-030, achieved) and degrade to plain scan/help when bootsupport `punk::args` is stale (`PUNKBOOT_PLAIN=1` forces the degraded mode). Source: `src/AGENTS.md`. - **punkcheck** records every install/delete as events in per-folder `.punkcheck` directories - the basis for skip/copy change detection, superseded-module pruning and provenance. Single OO record lifecycle (G-094, achieved) with atomic saves and advisory event-scoped locking for concurrent writers (G-095, achieved). - **Provenance gates.** Build/promotion commands warn on uncommitted `src/` changes (column-0 `PROVENANCE-WARNING:` token, `-dirty-abort` for strict mode); `vendorupdate` warns for dirty source-project checkouts. - **Buildsuites and the kit family.** `src/buildsuites/suite_tcl90/` builds Tcl/Tk/tcllib from source with a pinned zig toolchain and produces the runtime kit family (plain / punk / bi) plus artifact metadata (G-096-G-117 era: see archived goals G-096, G-098, G-102, G-103, G-107); artifacts publish to the punkbin repo that `bin/punk-runtime.cmd` fetches from. The 8.6 family and kit container strategy are in-flux - see "In-flux areas". @@ -102,7 +102,7 @@ Mechanics here are documented as pointers only - detailed prose would be obsolet - Input mode and completion: G-013 (raw-mode default), G-044 (punk::args-driven completion). - Stored configuration and help: G-014 (punk::config toml), G-042 (subshell help topics), G-043 (subshell definition plugins). - Console err channel: G-011. -- Launch surface: G-023 (version-named binaries), G-033 (`proj:` package mode), G-077 (`-e` one-liners), G-089 (scriptlib in kits), G-112 (make.tcl subcommand rename). +- Launch surface: G-023 (version-named binaries), G-033 (`proj:` package mode), G-077 (`-e` one-liners), G-089 (scriptlib in kits). - 8.6 buildsuite family: G-099, G-100, G-101. ## Doc map - which lane holds what diff --git a/CHANGELOG.md b/CHANGELOG.md index 7f13e065..94ef18d1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,21 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` "Project Versioning" section for the bump policy. +## [0.24.0] - 2026-07-26 + +- make.tcl bakelist + selective bake (G-121): new informational `bakelist ?kitname ...?` + subcommand reports the kit outputs configured in src/runtime/mapvfs.config - kit name, + kit type, runtime (with presence in the runtime store), vfs folder, and the deployed + state of bin/ vs the src/_build build product (current|stale|absent|nobuild), with + anomalies (runtime=missing, vfs=missing, rtrev=r runtime-materialization + staleness) in a trailing notes column; name filtering doubles as per-kit detail. + `bake ?kitname ...?` now bakes and deploys only the named kits - other kits' _build/bin + artifacts and punkcheck records are untouched, the vfslibs phase narrows to the named + kits' vfs folders, and an unknown name errors before any build, listing the configured + names. Bare `bake` is unchanged (all configured kits). Both surfaces (and the argdoc + kit-name choices) consume one parsed mapping model (::punkboot::lib::mapvfs_*), so the + config-format work (G-024 toml direction) can swap the reader underneath. + ## [0.23.3] - 2026-07-25 - make.tcl colour policy: unix-class Tcl 8.6 terminals auto-detected via the tty channel diff --git a/punkproject.toml b/punkproject.toml index 816fb136..697efe07 100644 --- a/punkproject.toml +++ b/punkproject.toml @@ -1,4 +1,4 @@ [project] name = "punkshell" -version = "0.23.3" +version = "0.24.0" license = "BSD-2-Clause" diff --git a/src/AGENTS.md b/src/AGENTS.md index b7f1ea6f..7077c8eb 100644 --- a/src/AGENTS.md +++ b/src/AGENTS.md @@ -72,6 +72,7 @@ Recovery after a wrong path guess: - make.tcl subcommands and flags are punk::args-declared (G-030): `tclsh src/make.tcl help ?subcommand?` or ` -help` shows tabled usage; invalid arguments produce punk::args usage errors. The dispatch degrades to plain scan/help when the bootsupport punk::args (or its rendering stack) is stale or unavailable; `PUNKBOOT_PLAIN=1` forces the degraded mode for troubleshooting. The `::punkboot::argdoc` definitions are constructed (string-built) and use the G-045 authoring mechanisms — block-form help values + `@normalize`, explicit `@form -synopsis` — see src/modules/AGENTS.md "punk::args definition authoring ergonomics" before editing them or authoring similar definitions. - Uncommitted `make.tcl`-generated outputs may be batched into one commit, regardless of how many `make.tcl` invocations produced them. This covers punkcheck-managed build outputs that are VCS-tracked: `src/bootsupport/`, `_vfscommon.vfs/modules` + `/lib`, declared per-kit `*.vfs/lib_tcl/` subfolders, the thin-layout sync copies (`src/project_layouts/vendor/punk//src/{make.tcl,build.tcl}` + bootsupport manifest copies), and the module-shipped layout payload under `src/modules/punk/mix/#modpod-templates-*/templates/project_layouts/` (G-087: synced from `src/project_layouts` - never hand-edit the modpod copy). (Root `modules/`, `lib/`, `modules_tcl/`, `lib_tcl/` are gitignored and never committed.) Each module's old→new version rename is atomic; a regeneration reflects one build-output refresh, not independent hand-edits. This is a carve-out from generic "split aggressively" commit guidance for punkcheck-managed build outputs only — it does not cover user-curated VFS content (`_config/`, per-kit `*.vfs/` outside declared `lib_tcl` subfolders, `_aside/`, `mkzipfix.vfs`, `_vfscommon.vfs/doc`), which commits separately per its own concerns. - Kit bakes consume the punk-runtime WORKING COPIES under `bin/runtime//`: `make.tcl bake`/`bakehouse` emit a `BUILD-WARNING:` (recapped at end of run) when a wrapped runtime's beside-toml revision is older than an `-r` artifact present in the same folder - the forgot-to-switch-back guard for deliberate `punk-runtime use ` testing excursions. Heed it before trusting freshly baked family kits; `bin/punk-runtime.cmd use >` (`.exe` suffix optional) re-materializes the working copy. +- `tclsh src/make.tcl bakelist ?kitname ...?` (G-121) reports the kit outputs configured in `src/runtime/mapvfs.config`: kit name, kit type, runtime (with presence in the runtime store), vfs folder, and the deployed state of `bin/` vs the `src/_build` build product (`current|stale|absent|nobuild`), with anomalies (`runtime=missing`, `vfs=missing`, `rtrev=r` materialization staleness) in a trailing notes column. Name arguments filter the report and add a per-kit detail block. `make.tcl bake ?kitname ...?` bakes and deploys only the named kits - other kits' `_build`/`bin` artifacts and punkcheck records are untouched and the vfslibs phase narrows to the named kits' vfs folders; an unknown name errors before any build, listing the configured names; flags go before kit names (`make.tcl bake -confirm 0 punk91`). Bare `bake` processes all configured kits as before. Both surfaces consume the shared parsed-mapping helpers (`::punkboot::lib::mapvfs_*`) rather than the file format, so the G-024 config-format work swaps the reader underneath. Piped characterization: `src/tests/shell/testsuites/punkexe/maketclbakelist.test`. - Use `punk make.tcl bakehouse` or `punk902z make.tcl bakehouse` inside Punk shell when building binaries through Punk (the deprecated `project` alias still maps). Driving make.tcl from a built punk executable is supported for informational/update subcommands and for kit builds of *other* kits — the kit whose deployed executable is running the build is skipped with a warning (it cannot be replaced while running, and the pre-deploy process sweep must not kill the build itself; the sweep also excludes the build's own pid in all cases). Rebuild that kit from tclsh or a different kit. - **Punk-exe-hosted make.tcl runs in a pre-loaded interp, not a virgin one** — the kit's script-mode boot has already loaded much of the punk stack (punk, punk::lib, punk::repl, punk::console, punk::du, flagfilter, struct::set ...) and set process state (app-punkscript forces `::tcl_interactive 0` for script semantics; libunknown/packagepreference are active). Consequences make.tcl must (and now does) handle explicitly: `package require` of an already-provided package is a no-op, so anything make.tcl breaks in the interp (the accelerator-reload block forgets+destroys sha1/md5/struct::* — it re-requires what was loaded), and anything computed at package-load time (punk::repl's `::tcl_interactive` probe — the shell branch recomputes it before `repl::start`), must be restored deliberately rather than relying on later loads to re-fire. Also note the copies that run in this mode are the *kit's* pre-loaded modules, not the bootsupport snapshots make.tcl's paths would otherwise prefer — silent provenance mixing when versions diverge. When adding interp-surgery or load-time-state assumptions to make.tcl, test under both `tclsh src/make.tcl ...` and ` src/make.tcl ...`. - Binary images are platform-specific; build on each target platform rather than expecting a cross-platform flag. diff --git a/src/make.tcl b/src/make.tcl index d4662a07..83930762 100644 --- a/src/make.tcl +++ b/src/make.tcl @@ -187,7 +187,7 @@ namespace eval ::punkboot { variable foldername [file tail $scriptfolder] variable pkg_requirements [list]; variable pkg_missing [list];variable pkg_loaded [list] variable help_flags [list -help --help /? -h] - variable known_commands [list bakehouse packages modules libs bake vfslibs bin info check shell vendorupdate bootsupport vfscommonupdate projectversion workflow buildsuite project vfs] + variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate bootsupport vfscommonupdate projectversion workflow buildsuite project vfs] } @@ -526,6 +526,370 @@ proc ::punkboot::lib::bootsupport_prompt_yesno {prompt} { return [expr {$answer eq "y" || $answer eq "yes"}] } +# -- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- +# Parsed kit-mapping model (G-121). One reader for src/runtime/mapvfs.config - +# the 'bake' kit machinery, the 'bakelist' report and the bake/bakelist argdoc +# choices all consume the model these helpers return, never the file format +# directly, so a config-format change (G-024 toml direction) swaps in underneath +# by reimplementing mapvfs_parse alone. +# +# mapvfs_parse: pure parse - no output, no exits. Returns a dict: +# exists 0|1 (mapfile present) +# mapfile path as supplied +# runtime_vfs_map runtime -> raw vfs_specs as listed (appname/type may be "") +# vfs_runtime_map vfstail -> list of {runtime appname type} (defaults applied; +# only vfs folders that exist on disk - matching the historical +# inline parse the kit loop was built around) +# missing names of referenced-but-absent runtimes/vfs folders +# warnings warning lines in encounter order, ready to print verbatim +# (missing-item lines carry their historical WARNING: prefix) +# badentries subset of warnings: malformed entry lines (these have no kit +# record to carry them, so bakelist prints them; missing-item +# warnings are instead surfaced as per-row state there) +# configerrors fatal config problems (duplicate runtime line; caller decides +# whether to abort - the bake path keeps its historical exit 3) +proc ::punkboot::lib::mapvfs_parse {mapfile rtfolder sourcefolder} { + set model [dict create\ + exists 0\ + mapfile $mapfile\ + runtime_vfs_map [dict create]\ + vfs_runtime_map [dict create]\ + missing [list]\ + warnings [list]\ + badentries [list]\ + configerrors [list]\ + ] + if {![file exists $mapfile]} { + return $model + } + dict set model exists 1 + set fdmap [open $mapfile r] + fconfigure $fdmap -translation binary + set mapdata [read $fdmap] + close $fdmap + set mapdata [string map [list \r\n \n] $mapdata] + set runtime_vfs_map [dict create] + set vfs_runtime_map [dict create] + set missing [list] + set warnings [list] + set badentries [list] + set configerrors [list] + foreach ln [split $mapdata \n] { + set ln [string trim $ln] + if {$ln eq "" || [string match #* $ln]} { + continue + } + set vfs_specs [lassign $ln runtime] + if {[string match *.exe $runtime]} { + #.exe is superfluous but allowed + #drop windows .exe suffix so same config can work cross platform - extension will be re-added if necessary later + set runtime [string range $runtime 0 end-4] + } + if {$runtime ne "-"} { + set runtime_test $runtime + if {"windows" eq $::tcl_platform(platform)} { + set runtime_test $runtime.exe + } + if {![file exists [file join $rtfolder $runtime_test]]} { + lappend warnings "WARNING: Missing runtime file $rtfolder/$runtime_test (line in mapvfs.config: $ln)" + lappend missing $runtime + } + } + foreach vfsconfig $vfs_specs { + switch -- [llength $vfsconfig] { + 1 - 2 - 3 { + lassign $vfsconfig vfstail appname target_kit_type + if {![file isdirectory [file join $sourcefolder vfs $vfstail]]} { + lappend warnings "WARNING: Missing vfs folder [file join $sourcefolder vfs $vfstail] specified in mapvfs.config for runtime $runtime" + lappend missing $vfstail + } else { + if {$appname eq ""} { + set appname [file rootname $vfstail] + } + dict lappend vfs_runtime_map $vfstail [list $runtime $appname $target_kit_type] + } + } + default { + set badline "bad entry in mapvfs.config - expected each entry after the runtime name to be of length 1 or length 2. got: $vfsconfig ([llength $vfsconfig])" + lappend warnings $badline + lappend badentries $badline + } + } + } + if {[dict exists $runtime_vfs_map $runtime]} { + lappend configerrors "CONFIG FILE ERROR. runtime: $runtime was specified more than once in $mapfile." + } else { + dict set runtime_vfs_map $runtime $vfs_specs + } + } + dict set model runtime_vfs_map $runtime_vfs_map + dict set model vfs_runtime_map $vfs_runtime_map + dict set model missing $missing + dict set model warnings $warnings + dict set model badentries $badentries + dict set model configerrors $configerrors + return $model +} + +# Ordered kit-output enumeration over a mapvfs_parse model. +# Replicates the kit loop's iteration + naming (vfs_tails glob order with map +# extras appended, per-vfs runtime order from the mapping, appname defaulting to +# the vfs rootname, kit_type defaulting to 'kit', '-' runtime -> .kit, +# windows .exe suffix, duplicate appname -> _) so listed names +# match what a bake produces. Entries whose vfs folder is missing on disk never +# reach the kit loop - they are appended after the buildable set so 'bakelist' +# can surface them as broken config. Duplicate-name disambiguation assumes the +# config keeps appnames unique (as the kit loop itself effectively does). +# Returns a list of record dicts: +# kitname base output name (selection key, e.g punk91) +# targetkit platform artifact name (e.g punk91.exe, or .kit for '-') +# kit_type kit|zip|zipcat|cookfs... as configured (defaulted to kit) +# runtime runtime name as configured (unsuffixed), or "-" +# runtime_file platform runtime filename ("" for "-") +# runtime_present 1|0 (1 for "-": no runtime needed) +# vfs vfs folder tail (e.g punk9wintk903.vfs) +# vfs_present 1|0 +proc ::punkboot::lib::mapvfs_kit_outputs {model rtfolder sourcefolder} { + set runtime_vfs_map [dict get $model runtime_vfs_map] + set vfs_runtime_map [dict get $model vfs_runtime_map] + set on_windows [expr {"windows" eq $::tcl_platform(platform)}] + set vfs_tails [glob -nocomplain -dir $sourcefolder/vfs -types d -tail *.vfs] + dict for {vfstail -} $vfs_runtime_map { + if {$vfstail ni $vfs_tails} { + lappend vfs_tails $vfstail + } + } + set records [list] + set exe_names_seen [list] + set record_pair {{rtname vfstail appname target_kit_type} { + upvar 1 records records exe_names_seen exe_names_seen rtfolder rtfolder on_windows on_windows sourcefolder sourcefolder + if {$appname eq ""} { + set appname [file rootname $vfstail] + } + if {$target_kit_type eq ""} { + set target_kit_type "kit" + } + if {$rtname eq "-"} { + set targetkit $appname.kit + set kitname $appname + set runtime_file "" + set runtime_present 1 ;#no runtime needed + } else { + if {$on_windows} { + set runtime_file $rtname.exe + set targetkit ${appname}.exe + } else { + set runtime_file $rtname + set targetkit $appname + } + set kitname $appname + if {$targetkit in $exe_names_seen} { + #duplicate appname configured - kit loop disambiguates the same way + set targetkit ${appname}_$rtname + set kitname ${appname}_$rtname + } + set runtime_present [file exists [file join $rtfolder $runtime_file]] + } + lappend exe_names_seen $targetkit + lappend records [dict create\ + kitname $kitname\ + targetkit $targetkit\ + kit_type $target_kit_type\ + runtime $rtname\ + runtime_file $runtime_file\ + runtime_present $runtime_present\ + vfs $vfstail\ + vfs_present [file isdirectory [file join $sourcefolder vfs $vfstail]]\ + ] + }} + foreach vfstail $vfs_tails { + if {![dict exists $vfs_runtime_map $vfstail]} { + continue ;#unmapped vfs folder - the kit loop derives no output names from it + } + foreach rt_app [dict get $vfs_runtime_map $vfstail] { + set rtname [lindex $rt_app 0] + if {![dict exists $runtime_vfs_map $rtname]} { + continue + } + foreach vfs_app [dict get $runtime_vfs_map $rtname] { + lassign $vfs_app configured_vfs appname target_kit_type + if {$configured_vfs ne $vfstail} { + continue + } + apply $record_pair $rtname $vfstail $appname $target_kit_type + } + } + } + #entries whose vfs folder is missing (excluded from vfs_runtime_map by the parse) + dict for {rtname vfs_specs} $runtime_vfs_map { + foreach vfsconfig $vfs_specs { + if {[llength $vfsconfig] < 1 || [llength $vfsconfig] > 3} { + continue + } + lassign $vfsconfig vfstail appname target_kit_type + if {[dict exists $vfs_runtime_map $vfstail]} { + continue ;#already enumerated above + } + apply $record_pair $rtname $vfstail $appname $target_kit_type + } + } + return $records +} + +# Match user-supplied kit names against mapvfs_kit_outputs records. +# Accepts the base kit name (punk91) or the platform artifact name (punk91.exe, +# name.kit); matching is case-insensitive on windows. Returns a dict: +# selected matched records (kit-output order, deduplicated) +# unknown supplied names that matched nothing +proc ::punkboot::lib::mapvfs_match_outputs {kit_outputs names} { + set on_windows [expr {"windows" eq $::tcl_platform(platform)}] + set selected_idx [list] + set unknown [list] + foreach name $names { + set base $name + if {[string match -nocase *.exe $base] || [string match -nocase *.kit $base]} { + set base [string range $base 0 end-4] + } + set found 0 + set i 0 + foreach rec $kit_outputs { + foreach candidate [list [dict get $rec kitname] [dict get $rec targetkit]] { + if {$candidate eq ""} {continue} + if {($on_windows && [string equal -nocase $candidate $base]) || (!$on_windows && $candidate eq $base)\ + || ($on_windows && [string equal -nocase $candidate $name]) || (!$on_windows && $candidate eq $name)} { + set found 1 + break + } + } + if {$found} { + if {$i ni $selected_idx} { + lappend selected_idx $i + } + break + } + incr i + } + if {!$found} { + lappend unknown $name + } + } + set selected [list] + foreach i [lsort -integer $selected_idx] { + lappend selected [lindex $kit_outputs $i] + } + return [dict create selected $selected unknown $unknown] +} + +# Local runtime-materialization staleness core (shared by the bake-path +# BUILD-WARNING and the bakelist report - the G-121 reuse of the G-103/G-117 +# toml-revision metadata). Compares a punk-runtime WORKING COPY's beside-toml +# revision against the highest -r artifact revision present in the same +# folder. Returns an empty dict when there is nothing to report (pre-family +# runtime without a toml, direct -r artifact reference, or no newer artifact +# beside the working copy); otherwise a dict {current_rev N maxrev N maxname S}. +proc ::punkboot::lib::runtime_materialization_check {rtfolder runtime_fullname} { + set root $runtime_fullname + if {[string match -nocase *.exe $root]} { + set root [file rootname $root] + } + if {[regexp -- {-r\d+$} $root]} { + return [dict create] ;#an immutable artifact referenced directly - not a working copy + } + set toml [file join $rtfolder ${root}.toml] + if {![file isfile $toml]} { + return [dict create] + } + if {[catch { + set f [open $toml r] + set tomldata [read $f] + close $f + }]} { + return [dict create] + } + if {![regexp -line {^\s*revision\s*=\s*(\d+)} $tomldata -> current_rev]} { + return [dict create] + } + set maxrev -1 + set maxname "" + foreach cand [glob -nocomplain -directory $rtfolder -tails -- "${root}-r*"] { + if {[regexp -- {^.+-r(\d+)(?:\.[Ee][Xx][Ee])?$} $cand -> rev]} { + if {$rev > $maxrev} { + set maxrev $rev + set maxname $cand + } + } + } + if {$maxrev > $current_rev} { + return [dict create current_rev $current_rev maxrev $maxrev maxname $maxname] + } + return [dict create] +} + +# Byte-equality of two files (chunked compare, early exit on first difference). +# Used by kit_deploy_state - avoids hashing dependencies in the boot path. +proc ::punkboot::lib::files_content_identical {patha pathb} { + if {[file size $patha] != [file size $pathb]} { + return 0 + } + set fa [open $patha r] + set fb [open $pathb r] + set same 1 + try { + chan configure $fa -translation binary -buffersize 262144 + chan configure $fb -translation binary -buffersize 262144 + while {1} { + set da [read $fa 262144] + set db [read $fb 262144] + if {$da ne $db} { + set same 0 + break + } + if {[chan eof $fa] || [chan eof $fb]} { + if {[chan eof $fa] != [chan eof $fb]} { + set same 0 + } + break + } + } + } finally { + close $fa + close $fb + } + return $same +} + +# Deployed-state classification for a configured kit output (G-121 bakelist): +# the deployed copy (/bin/) vs the build product +# (src/_build/). +# absent - no deployed copy in the bin folder +# nobuild - deployed copy exists but there is no build product to compare against +# current - deployed copy is byte-identical to the build product +# stale - deployed copy differs from the build product +# Size mismatch decides 'stale' cheaply; equal size + equal mtime short-circuits +# to 'current' (the deploy step is a plain 'file copy'); otherwise a full content +# compare decides. +proc ::punkboot::lib::kit_deploy_state {buildfolder deployfolder targetkit} { + set built [file join $buildfolder $targetkit] + set deployed [file join $deployfolder $targetkit] + if {![file isfile $deployed]} { + return absent + } + if {![file isfile $built]} { + return nobuild + } + if {[file size $built] != [file size $deployed]} { + return stale + } + if {[file mtime $built] == [file mtime $deployed]} { + return current + } + if {[punkboot::lib::files_content_identical $built $deployed]} { + return current + } + return stale +} +# -- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- + if {"::try" ni [info commands ::try]} { puts stderr "Tcl interpreter possibly too old - need at least tcl 8.6 - 'try' command not found - aborting" exit 1 @@ -1558,9 +1922,14 @@ proc ::punkboot::punkboot_gethelp {args} { append h " - refuses uncommitted src by default (-dirty-abort defaults ON: the bakehouse bakes from the committed recipe; pass -dirty-abort 0 to override)" \n append h " - the optional -k flag will terminate running processes matching the executable being built (if applicable)" \n append h " - does NOT run the promotion gates (bootsupport, vfscommonupdate) - on a clean checkout they are already satisfied by the committed tree" \n \n - append h " $scriptname bake ?-k?" \n + append h " $scriptname bake ?-k? ?kitname ...?" \n append h " - assemble kit/zipkit executables from the promoted payload (src/vfs) and src/runtime runtimes into /bin" \n - append h " - includes the vfslibs phase; does not rebuild modules/libs and does not run the promotion gates" \n \n + append h " - includes the vfslibs phase; does not rebuild modules/libs and does not run the promotion gates" \n + append h " - with kitname arguments, bakes and deploys only the named configured kits (unknown names error before any build)" \n \n + append h " $scriptname bakelist ?kitname ...?" \n + append h " - list the kit outputs configured in src/runtime/mapvfs.config: name, kit type, runtime (with presence)," \n + append h " vfs folder and deployed state (bin copy absent/current/stale/nobuild vs the src/_build product)" \n + append h " - kitname arguments filter to the named entries (per-kit detail)" \n \n append h " $scriptname modules" \n append h " - build (or copy if build not required) .tm modules from src/modules src/vendormodules etc to their corresponding locations under " \n append h " This does not scan src/runtime and src/vfs folders to build kit/zipkit/cookfs executables" \n \n @@ -1718,6 +2087,10 @@ DIAGRAM 2 - KIT ASSEMBLY DETAIL (the 'make.tcl bake' stage; incl. vfslibs phase) ---------------------------------------------------------------------------- src/runtime/mapvfs.config which vfs folder pairs with which runtime -> kit name + 'make.tcl bakelist ?kitname ...?' reports the configured + kit outputs (runtime/vfs presence + deployed state); + 'make.tcl bake ?kitname ...?' bakes only the named kits + (bare bake = all; unknown names error before any build) src/runtime/.exe bare Tcl runtime (tclkit / tclsfe build) src/vfs/_vfscommon.vfs/ common payload (built modules + libs, main boot support) src/vfs/.vfs/ per-kit overlay (main.tcl, kit-only modules/config) @@ -1767,7 +2140,9 @@ KEY / NOTES [K5] The deploy step cannot replace a kit exe that is currently executing. Close running kit shells before step 8, or rerun 'make.tcl bake -confirm 0' afterwards - the freshly built kits wait in src/_build. punkcheck records - mean the rerun only redoes the failed deploys. + mean the rerun only redoes the failed deploys. When iterating on one kit, + 'make.tcl bake ' rebuilds/deploys just that kit; 'make.tcl + bakelist' shows each configured kit's deployed state (bin vs src/_build). [K6] All confirmation prompts follow -confirm: unattended/agent runs must pass -confirm 0 (non-interactive stdin aborts fast at prompts; piping 'y' is @@ -1884,6 +2259,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO modules "Build .tm modules from src/modules, src/vendormodules etc into /modules etc" libs "Build pkgIndex.tcl libraries from src/lib, src/vendorlib etc into /lib etc" bake "Assemble kit/zipkit executables from promoted payload (src/vfs) and runtimes into /bin" + bakelist "List the kit outputs configured in src/runtime/mapvfs.config: type, runtime/vfs presence, deployed state" vfslibs "Propagate declared vendored platform-library packages into kit vfs lib_tcl trees" bin "Install executables from src/bin into /bin, then build kits as for bake" vendorupdate "Update src/vendormodules based on src/vendormodules/include_modules.config" @@ -1945,13 +2321,37 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO build each configured kit from the promoted payload (src/vfs/_vfscommon.vfs + custom vfs folders) and its runtime, and install to /bin. + With kitname arguments, bake and deploy ONLY the named configured + kits (see 'make.tcl bakelist' for the configured names) - other + kits' build products, bin copies and punkcheck records are left + untouched. An unknown name errors before any build, listing the + configured names. Bare 'bake' processes all configured kits. Includes the vfslibs phase (declared vendored platform-library propagation into kit vfs lib_tcl trees) so a bake cannot ship - stale binary libs. + stale binary libs; a selective bake narrows that phase to the + named kits' vfs folders. Does not rebuild modules/libs and does not run the promotion gates - built modules reach the kit payload only via 'make.tcl vfscommonupdate' (committed for provenance). Consumer full path from a clean checkout: 'make.tcl bakehouse'." + bakelist + " + List the kit outputs configured in src/runtime/mapvfs.config (the + same parsed mapping the bake subcommand consumes): kit name, kit + type (kit|zip|zipcat...), runtime with presence in the runtime + store (/bin/runtime/), vfs folder, and the + deployed state of /bin/ vs the src/_build build + product: + current - deployed copy is identical to the build product + stale - deployed copy differs from the build product + absent - no deployed copy in /bin + nobuild - deployed copy exists but no build product to compare + A trailing notes column flags anomalies (runtime=missing, + vfs=missing, rtrev=r when the runtime working copy is + materialized from an older revision than a -r artifact beside + it). + kitname arguments filter the report to the named entries (per-kit + detail, with resolved paths). Reporting only - never builds." vfslibs " Propagate declared vendored platform-library packages into kit vfs @@ -2097,6 +2497,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO modules {OPT_DIRTYABORT OPT_CONFIRM} libs {OPT_DIRTYABORT OPT_CONFIRM} bake {OPT_FORCEKILL OPT_DIRTYABORT OPT_CONFIRM} + bakelist {} vfslibs {OPT_DIRTYABORT OPT_CONFIRM} bin {OPT_FORCEKILL OPT_DIRTYABORT OPT_CONFIRM} vendorupdate {OPT_CONFIRM} @@ -2118,11 +2519,54 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO OPT_DIRTYABORT_ON "?-dirty-abort 1|0?" OPT_CONFIRM "?-confirm 0|1?" } + #per-subcommand positional values (default: none). bake (and its deprecated + #alias vfs) and bakelist take optional kit names (G-121 selective bake). + #Discoverability nicety: the configured kit names are offered as choices when + #the mapping parses at definition time - informational only (-choicerestricted + #0), so the handlers' own validation (which lists the configured names, and + #still works when the mapping cannot be parsed here) stays authoritative. + variable KITNAME_CHOICES [list] + catch { + set _mapmodel [punkboot::lib::mapvfs_parse [file join $::punkboot::scriptfolder runtime mapvfs.config] "" $::punkboot::scriptfolder] + foreach _rec [punkboot::lib::mapvfs_kit_outputs $_mapmodel "" $::punkboot::scriptfolder] { + lappend KITNAME_CHOICES [dict get $_rec kitname] + } + } + set _kitname_choicepart "" + if {[llength $KITNAME_CHOICES]} { + set _kitname_choicepart " -choices [list $KITNAME_CHOICES] -choicerestricted 0" + } + variable SUBVALUES [dict create] + dict set SUBVALUES bake [list\ + "@values -min 0 -max -1"\ + "kitname -type string -optional 1 -multiple 1${_kitname_choicepart} -help\ + { + Configured kit output(s) to bake and deploy - see 'make.tcl bakelist' + for the configured names. With no kitname, all configured kits are + processed. An unknown name errors before any build.}"\ + ] + dict set SUBVALUES vfs [dict get $SUBVALUES bake] + dict set SUBVALUES bakelist [list\ + "@values -min 0 -max -1"\ + "kitname -type string -optional 1 -multiple 1${_kitname_choicepart} -help\ + { + Configured kit output(s) to report on (filters the listing - doubles + as per-kit detail). With no kitname, all configured entries are + listed.}"\ + ] + variable VALUES_SYNOPSES { + bake "?kitname ...?" + vfs "?kitname ...?" + bakelist "?kitname ...?" + } foreach {sub optvars} $SUBOPTS { set synparts [list "make.tcl $sub"] foreach ov $optvars { lappend synparts [dict get $OPT_SYNOPSES $ov] } + if {[dict exists $VALUES_SYNOPSES $sub]} { + lappend synparts [dict get $VALUES_SYNOPSES $sub] + } set defparts [list] lappend defparts "@id -id (script)::punkboot::$sub" lappend defparts "@normalize" ;#G-045 block-form value re-basing for constructed definitions @@ -2132,7 +2576,13 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO foreach ov $optvars { lappend defparts [set $ov] } - lappend defparts "@values -min 0 -max 0" + if {[dict exists $SUBVALUES $sub]} { + foreach vp [dict get $SUBVALUES $sub] { + lappend defparts $vp + } + } else { + lappend defparts "@values -min 0 -max 0" + } punk::args::define {*}$defparts } punk::args::define\ @@ -2172,7 +2622,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "promotion gates" {bootsupport vfscommonupdate} "source maintenance" {vendorupdate} "buildsuites" {buildsuite} - "informational" {info check projectversion workflow help} + "informational" {info check bakelist projectversion workflow help} "interactive" {shell} "deprecated aliases" {project vfs} } @@ -2200,9 +2650,10 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO #plain fallback immediately instead of erroring at first parse/usage. punk::args::get_spec (script)::punkboot punk::args::get_spec (script)::punkboot::bakehouse + punk::args::get_spec (script)::punkboot::bake ;#exercises the G-121 kitname values spec (-choicerestricted mechanism) punk::args::get_spec (script)::punkboot::help punk::args::get_spec (script)::punkboot::shell - unset -nocomplain sub optvars defparts ov + unset -nocomplain sub optvars defparts ov vp _mapmodel _rec _kitname_choicepart } } _defserr]} { set ::punkboot::punkargs_ok 1 @@ -2224,6 +2675,7 @@ set help_exitcode 0 set ::punkboot::opt_forcekill 0 set ::punkboot::opt_dirty_abort 0 set ::punkboot::opt_confirm 1 +set ::punkboot::opt_kitnames [list] ;#G-121: requested kit names for bake/bakelist (empty = all configured) if {$::punkboot::punkargs_ok} { set first [lindex $scriptargs 0] @@ -2274,6 +2726,10 @@ if {$::punkboot::punkargs_ok} { set ::punkboot::$_var [dict get $_opts $_optname] } } + if {[dict exists $argd values kitname]} { + #declared -multiple: a list of the supplied kit names (bake/vfs/bakelist - G-121) + set ::punkboot::opt_kitnames [dict get $argd values kitname] + } unset -nocomplain _opts _optname _var } } @@ -2319,6 +2775,13 @@ if {$::punkboot::punkargs_ok} { lappend commands_found $a } } + #G-121: bake/bakelist (and the deprecated alias vfs) take optional trailing kit + #names - collect them here so the single-command check below still holds in + #degraded mode (the handlers do their own name validation). + if {[lindex $commands_found 0] in {bake bakelist vfs}} { + set ::punkboot::opt_kitnames [lrange $commands_found 1 end] + set commands_found [lrange $commands_found 0 0] + } if {![llength $scriptargs] || [lindex $commands_found 0] eq "help"} { set do_help 1 } elseif {!$do_help} { @@ -2403,8 +2866,8 @@ set forcekill $::punkboot::opt_forcekill # - minor bump -> prompt user; backward-compat aliases *should* keep old bootsupport functional # - patch bump -> warn and proceed (non-breaking) # 'check' is exempt in all cases — the warning is shown at the end of its output instead. -# 'projectversion' and 'workflow' are informational/read-only and are likewise exempt. -if {[info exists ::punkboot::stale_bootsupport] && $::punkboot::command ni {check projectversion workflow buildsuite}} { +# 'projectversion', 'workflow' and 'bakelist' are informational/read-only and are likewise exempt. +if {[info exists ::punkboot::stale_bootsupport] && $::punkboot::command ni {check projectversion workflow bakelist buildsuite}} { set _stale $::punkboot::stale_bootsupport set _have_major 0 set _have_minor 0 @@ -2479,6 +2942,77 @@ if {![string length [set projectroot [punk::repo::find_project $scriptfolder]]]} set sourcefolder $projectroot/src set binfolder $projectroot/bin +# ---------------------------------------- +# G-121 selective bake: resolve requested kit names against the parsed kit-mapping +# model before ANY build machinery runs - an unknown name must error without building, +# and a selected-but-unbuildable kit (missing runtime/vfs) fails fast rather than +# silently doing nothing (bare 'bake' keeps the historical warn-and-continue +# behaviour for the full configured set). The kit machinery and the vfslibs phase +# consult the bake_selected_* results to confine processing (and punkcheck events) +# to the requested kits. +set ::punkboot::bake_selected_kitnames [list] ;#empty = all configured (bare bake/bakehouse/bin) +set ::punkboot::bake_selected_targetkits [list] ;#platform artifact names (e.g punk91.exe) +set ::punkboot::bake_selected_vfs [list] ;#vfs folder tails the selected kits use +set ::punkboot::bake_selected_runtimes [list] ;#unsuffixed runtime names the selected kits wrap +if {$::punkboot::command eq "bake" && [llength $::punkboot::opt_kitnames]} { + switch -glob -- $this_platform_generic { + macosx-* { + set _rt_os_arch macosx + } + default { + set _rt_os_arch $this_platform_generic + } + } + set _rtfolder $binfolder/runtime/$_rt_os_arch + set _mapmodel [punkboot::lib::mapvfs_parse $sourcefolder/runtime/mapvfs.config $_rtfolder $sourcefolder] + if {[llength [dict get $_mapmodel configerrors]]} { + foreach _errline [dict get $_mapmodel configerrors] { + puts stderr $_errline + } + exit 3 + } + set _kit_outputs [punkboot::lib::mapvfs_kit_outputs $_mapmodel $_rtfolder $sourcefolder] + set _matchinfo [punkboot::lib::mapvfs_match_outputs $_kit_outputs $::punkboot::opt_kitnames] + if {[llength [dict get $_matchinfo unknown]]} { + puts stderr "make.tcl bake: unknown kit name(s): [join [dict get $_matchinfo unknown] {, }] - nothing built" + if {[llength $_kit_outputs]} { + puts stderr " configured kit outputs: [join [lmap _r $_kit_outputs {dict get $_r kitname}] {, }]" + } else { + puts stderr " no kit outputs are configured (src/runtime/mapvfs.config)" + } + foreach _n [dict get $_matchinfo unknown] { + if {[string match -* $_n]} { + puts stderr " note: flags go before kit names, e.g 'make.tcl bake -confirm 0 punk91'" + break + } + } + puts stderr " ('make.tcl bakelist' shows the configured kit matrix)" + exit 1 + } + foreach _rec [dict get $_matchinfo selected] { + if {![dict get $_rec vfs_present]} { + puts stderr "make.tcl bake: kit '[dict get $_rec kitname]' cannot be built - vfs folder src/vfs/[dict get $_rec vfs] is missing - nothing built" + exit 1 + } + if {[dict get $_rec runtime] ne "-" && ![dict get $_rec runtime_present]} { + puts stderr "make.tcl bake: kit '[dict get $_rec kitname]' cannot be built - runtime [dict get $_rec runtime_file] not present in bin/runtime/$_rt_os_arch - nothing built" + exit 1 + } + } + foreach _rec [dict get $_matchinfo selected] { + lappend ::punkboot::bake_selected_kitnames [dict get $_rec kitname] + lappend ::punkboot::bake_selected_targetkits [dict get $_rec targetkit] + if {[dict get $_rec vfs] ni $::punkboot::bake_selected_vfs} { + lappend ::punkboot::bake_selected_vfs [dict get $_rec vfs] + } + if {[dict get $_rec runtime] ni $::punkboot::bake_selected_runtimes} { + lappend ::punkboot::bake_selected_runtimes [dict get $_rec runtime] + } + } + puts stdout "selective bake (G-121): [join $::punkboot::bake_selected_kitnames {, }]" + unset -nocomplain _rt_os_arch _rtfolder _mapmodel _kit_outputs _matchinfo _rec _r _errline +} + # ---------------------------------------- # Provenance-warning presentation + dirty-src check for build/promotion commands (goal G-026 direction). # Build/promotion commands stamp versions onto src content and propagate it into trees other @@ -2952,6 +3486,125 @@ if {$::punkboot::command eq "workflow"} { exit 0 } +if {$::punkboot::command eq "bakelist"} { + #G-121: report the kit outputs configured in src/runtime/mapvfs.config, consuming + #the same parsed mapping model as the bake kit machinery (::punkboot::lib::mapvfs_*). + #Reporting only - never builds. Data rows are single-line and column-stable for + #grepping; expected state sits in the columns, anomalies surface in a trailing + #sparse notes column (key=value tags). Kit name arguments filter the report and + #add a per-kit detail block (resolved paths/sizes/mtimes). + switch -glob -- $this_platform_generic { + macosx-* { + set rt_os_arch macosx + } + default { + set rt_os_arch $this_platform_generic + } + } + set rtfolder $binfolder/runtime/$rt_os_arch + set buildfolder $sourcefolder/_build + set mapfile $sourcefolder/runtime/mapvfs.config + set mapmodel [punkboot::lib::mapvfs_parse $mapfile $rtfolder $sourcefolder] + foreach errline [dict get $mapmodel configerrors] { + puts stderr $errline + } + foreach badline [dict get $mapmodel badentries] { + puts stderr $badline + } + if {![dict get $mapmodel exists]} { + puts stdout "no kit mapping config at $mapfile - no kit outputs configured" + exit 0 + } + set kit_outputs [punkboot::lib::mapvfs_kit_outputs $mapmodel $rtfolder $sourcefolder] + if {![llength $kit_outputs]} { + puts stdout "no kit outputs configured in $mapfile" + exit 0 + } + set reportset $kit_outputs + if {[llength $::punkboot::opt_kitnames]} { + set matchinfo [punkboot::lib::mapvfs_match_outputs $kit_outputs $::punkboot::opt_kitnames] + if {[llength [dict get $matchinfo unknown]]} { + puts stderr "bakelist: unknown kit name(s): [join [dict get $matchinfo unknown] {, }]" + puts stderr " configured kit outputs: [join [lmap r $kit_outputs {dict get $r kitname}] {, }]" + exit 1 + } + set reportset [dict get $matchinfo selected] + } + #assemble rows before printing so columns can be sized to content + set rows [list] + foreach rec $reportset { + set kitname [dict get $rec kitname] + set kit_type [dict get $rec kit_type] + set runtime [dict get $rec runtime] + set vfstail [dict get $rec vfs] + set deployed [punkboot::lib::kit_deploy_state $buildfolder $binfolder [dict get $rec targetkit]] + set notes [list] + if {$runtime ne "-" && ![dict get $rec runtime_present]} { + lappend notes runtime=missing + } + if {![dict get $rec vfs_present]} { + lappend notes vfs=missing + } + if {$runtime ne "-" && [dict get $rec runtime_present]} { + set rtstale [punkboot::lib::runtime_materialization_check $rtfolder [dict get $rec runtime_file]] + if {[dict size $rtstale]} { + lappend notes rtrev=r[dict get $rtstale current_rev] vs src/_build/" + set headline "" + foreach h $headings w $widths { + append headline [format "%-*s " $w $h] + } + puts stdout [string trimright $headline] + foreach row $rows { + set line "" + foreach cell [lrange $row 0 4] w $widths { + append line [format "%-*s " $w $cell] + } + append line [lindex $row 5] + puts stdout [string trimright $line] + } + if {[llength $::punkboot::opt_kitnames]} { + #name filtering doubles as per-kit detail (punk-runtime list precedent) + proc ::punkboot::lib::bakelist_file_detail {path} { + if {![file isfile $path]} { + return "(not present)" + } + return "([file size $path] bytes, [clock format [file mtime $path] -format {%Y-%m-%d %H:%M:%S}])" + } + foreach rec $reportset row $rows { + puts stdout "" + puts stdout "detail: [dict get $rec kitname]" + set runtime [dict get $rec runtime] + if {$runtime eq "-"} { + puts stdout " runtime: (none - unwrapped .kit output)" + } else { + puts stdout " runtime file: bin/runtime/$rt_os_arch/[dict get $rec runtime_file] [expr {[dict get $rec runtime_present] ? {(present)} : {(MISSING)}}]" + } + puts stdout " vfs folder: src/vfs/[dict get $rec vfs] [expr {[dict get $rec vfs_present] ? {(present)} : {(MISSING)}}]" + puts stdout " kit type: [dict get $rec kit_type]" + puts stdout " build product: src/_build/[dict get $rec targetkit] [punkboot::lib::bakelist_file_detail $buildfolder/[dict get $rec targetkit]]" + puts stdout " deployed: bin/[dict get $rec targetkit] [punkboot::lib::bakelist_file_detail $binfolder/[dict get $rec targetkit]] state=[lindex $row 4]" + } + } + exit 0 +} + if {$::punkboot::command eq "buildsuite"} { #G-104: thin front for the buildsuites (zig runtime factory - arm's-length #subsystem building Tcl runtimes from external sources). Discovery is a @@ -3707,6 +4360,13 @@ if {$::punkboot::command in {bakehouse bake vfslibs}} { set pkgtail [file tail $source_pkg_folder] foreach t $itargets { set target_vfs_dir [lindex [file split $t] 0] + if {[llength $::punkboot::bake_selected_kitnames] && $target_vfs_dir ni $::punkboot::bake_selected_vfs} { + #G-121 selective bake: narrow the vfslibs phase to the selected kits' + #vfs folders (skip precedes the existence check so an unselected + #entry's problem cannot abort a selective run) + puts stdout "VFSLIBS install.$iname: skipping vfs/$t (selective bake - not a selected kit's vfs)" + continue + } if {![file isdirectory $sourcefolder/vfs/$target_vfs_dir]} { puts stderr "$A(BAD)VFSLIBS: entry install.$iname target '$t' - no vfs folder at $sourcefolder/vfs/$target_vfs_dir$A(RST)" exit 3 @@ -4079,44 +4739,19 @@ proc ::punkboot::lib::runtime_buildcopyname {runtime} { #working copy's beside-toml revision (G-103/G-117 metadata) against the highest #-r artifact revision present in the same folder. Pre-family runtimes (no #toml) and direct -r references stay silent; one warning per runtime per run. +#G-121: the comparison core lives in ::punkboot::lib::runtime_materialization_check, +#shared with the bakelist report's rtrev= note. set ::punkboot::runtime_rev_warned [dict create] proc ::punkboot::runtime_materialization_warning {rtfolder runtime_fullname} { if {[dict exists $::punkboot::runtime_rev_warned $runtime_fullname]} { return } dict set ::punkboot::runtime_rev_warned $runtime_fullname 1 - set root $runtime_fullname - if {[string match -nocase *.exe $root]} { - set root [file rootname $root] - } - if {[regexp -- {-r\d+$} $root]} { - return ;#an immutable artifact referenced directly - not a working copy - } - set toml [file join $rtfolder ${root}.toml] - if {![file isfile $toml]} { - return - } - if {[catch { - set f [open $toml r] - set tomldata [read $f] - close $f - }]} { - return - } - if {![regexp -line {^\s*revision\s*=\s*(\d+)} $tomldata -> current_rev]} { - return - } - set maxrev -1 - set maxname "" - foreach cand [glob -nocomplain -directory $rtfolder -tails -- "${root}-r*"] { - if {[regexp -- {^.+-r(\d+)(?:\.[Ee][Xx][Ee])?$} $cand -> rev]} { - if {$rev > $maxrev} { - set maxrev $rev - set maxname $cand - } - } - } - if {$maxrev > $current_rev} { + set stale [punkboot::lib::runtime_materialization_check $rtfolder $runtime_fullname] + if {[dict size $stale]} { + set current_rev [dict get $stale current_rev] + set maxrev [dict get $stale maxrev] + set maxname [dict get $stale maxname] ::punkboot::print_build_warnings [list "runtime $runtime_fullname working copy is materialized from revision r$current_rev but $maxname (r$maxrev) is present beside it - kits wrapping this runtime get r$current_rev. To bake with r$maxrev: bin/punk-runtime.cmd use $maxname then re-run the bake"] } return @@ -4162,78 +4797,50 @@ if {![llength $runtimes]} { #todo - don't exit - it is valid to use runtime of - to just build a .kit/.zipkit ? exit 0 } +#G-121 selective bake: restrict processing to the runtimes the selected kits wrap - +#other runtimes get no BUILDCOPY refresh, no capability probe and no punkcheck events. +if {[llength $::punkboot::bake_selected_kitnames]} { + set runtimes [lmap _rtfile $runtimes { + set _rtkey $_rtfile + if {[string match *.exe $_rtkey]} { + set _rtkey [string range $_rtkey 0 end-4] + } + if {$_rtkey ni $::punkboot::bake_selected_runtimes} { + continue + } + set _rtfile + }] + unset -nocomplain _rtfile _rtkey +} #previous have_sdx test location # -- --- --- --- --- --- --- --- --- --- #load mapvfs.config file (if any) in runtime folder to map runtimes to vfs folders. -#build a dict keyed on runtime executable name. +#build a dict keyed on runtime executable name. #If no mapfile (or no mapfile entry for that runtime) - the runtime will be paired with a matching .vfs folder in src folder. e.g punk.exe to src/punk.vfs #If vfs folders or runtime executables which are explicitly listed in the mapfile don't exist - warn on stderr - but continue. if such nonexistants found; prompt user for whether to continue or abort. +#G-121: parsing lives in ::punkboot::lib::mapvfs_parse - the shared model consumed here, +#by 'bakelist' and by the selective-bake name resolution (never the file format directly). set mapfile $rt_sourcefolder/mapvfs.config -set runtime_vfs_map [dict create] -set vfs_runtime_map [dict create] -if {[file exists $mapfile]} { - set fdmap [open $mapfile r] - fconfigure $fdmap -translation binary - set mapdata [read $fdmap] - close $fdmap - set mapdata [string map [list \r\n \n] $mapdata] - set missing [list] - foreach ln [split $mapdata \n] { - set ln [string trim $ln] - if {$ln eq "" || [string match #* $ln]} { - continue - } - set vfs_specs [lassign $ln runtime] - if {[string match *.exe $runtime]} { - #.exe is superfluous but allowed - #drop windows .exe suffix so same config can work cross platform - extension will be re-added if necessary later - set runtime [string range $runtime 0 end-4] - } - if {$runtime ne "-"} { - set runtime_test $runtime - if {"windows" eq $::tcl_platform(platform)} { - set runtime_test $runtime.exe - } - if {![file exists [file join $rtfolder $runtime_test]]} { - puts stderr "WARNING: Missing runtime file $rtfolder/$runtime_test (line in mapvfs.config: $ln)" - lappend missing $runtime - } - } - foreach vfsconfig $vfs_specs { - switch -- [llength $vfsconfig] { - 1 - 2 - 3 { - lassign $vfsconfig vfstail appname target_kit_type - if {![file isdirectory [file join $sourcefolder vfs $vfstail]]} { - puts stderr "WARNING: Missing vfs folder [file join $sourcefolder vfs $vfstail] specified in mapvfs.config for runtime $runtime" - lappend missing $vfstail - } else { - if {$appname eq ""} { - set appname [file rootname $vfstail] - } - dict lappend vfs_runtime_map $vfstail [list $runtime $appname $target_kit_type] - } - } - default { - puts stderr "bad entry in mapvfs.config - expected each entry after the runtime name to be of length 1 or length 2. got: $vfsconfig ([llength $vfsconfig])" - } - } - - - } - if {[dict exists $runtime_vfs_map $runtime]} { - puts stderr "CONFIG FILE ERROR. runtime: $runtime was specified more than once in $mapfile." - exit 3 - } - dict set runtime_vfs_map $runtime $vfs_specs - } - if {[llength $missing]} { - puts stderr "WARNING [llength $missing] missing items from $mapfile. (TODO - prompt user to continue/abort)" - foreach m $missing { - puts stderr " $m" - } - puts stderr "continuing..." +set mapmodel [punkboot::lib::mapvfs_parse $mapfile $rtfolder $sourcefolder] +foreach warnline [dict get $mapmodel warnings] { + puts stderr $warnline +} +if {[llength [dict get $mapmodel configerrors]]} { + foreach errline [dict get $mapmodel configerrors] { + puts stderr $errline } + exit 3 +} +set runtime_vfs_map [dict get $mapmodel runtime_vfs_map] +set vfs_runtime_map [dict get $mapmodel vfs_runtime_map] +set missing [dict get $mapmodel missing] +if {[llength $missing]} { + puts stderr "WARNING [llength $missing] missing items from $mapfile. (TODO - prompt user to continue/abort)" + foreach m $missing { + puts stderr " $m" + } + puts stderr "continuing..." } # -- --- --- --- --- --- --- --- --- --- puts "-- runtime_vfs_map --" @@ -4251,6 +4858,9 @@ puts "---------------------" #mthod3 qemu? set runtime_caps [dict create] foreach runtime [dict keys $runtime_vfs_map] { + if {[llength $::punkboot::bake_selected_kitnames] && $runtime ni $::punkboot::bake_selected_runtimes} { + continue ;#G-121 selective bake - probe only the runtimes being wrapped + } set capscript { set caps [dict create] dict set caps patchlevel [info patchlevel] @@ -4327,6 +4937,17 @@ dict for {vfstail -} $vfs_runtime_map { lappend vfs_tails $vfstail } } +#G-121 selective bake: only the selected kits' vfs folders get checked/checksummed - +#other vfs folders' punkcheck build events never fire. +if {[llength $::punkboot::bake_selected_kitnames]} { + set vfs_tails [lmap _vt $vfs_tails { + if {$_vt ni $::punkboot::bake_selected_vfs} { + continue + } + set _vt + }] + unset -nocomplain _vt +} if {![llength $vfs_tails]} { puts stdout "No .vfs folders found at '$sourcefolder/vfs' - no kits to build" puts stdout " -done- " @@ -4706,6 +5327,9 @@ foreach vfstail $vfs_tails { # $runtimes may now include a dash entry "-" (from mapvfs.config file) foreach runtime_fullname $runtimes { set rtname [file rootname $runtime_fullname] + if {[llength $::punkboot::bake_selected_kitnames] && $rtname ni $::punkboot::bake_selected_runtimes} { + continue ;#G-121 selective bake - no requested kit wraps this runtime + } #rtname of "-" indicates build a kit without a runtime if {$runtime_fullname ne "-"} { ::punkboot::runtime_materialization_warning $rtfolder $runtime_fullname @@ -4750,6 +5374,9 @@ foreach vfstail $vfs_tails { } puts stdout " vfs: $vfstail runtime: $rtname targetkits: $targetkits" foreach targetkit_info $targetkits { + if {[llength $::punkboot::bake_selected_kitnames] && [lindex $targetkit_info 0] ni $::punkboot::bake_selected_targetkits} { + continue ;#G-121 selective bake - not a requested kit output + } puts stdout " processing targetkit: $targetkit_info" lassign $targetkit_info targetkit target_kit_type #Self-build guard: never process the kit whose deployed executable is running this diff --git a/src/runtime/mapvfs.config b/src/runtime/mapvfs.config index 458a23c1..33a6a19e 100644 --- a/src/runtime/mapvfs.config +++ b/src/runtime/mapvfs.config @@ -65,9 +65,9 @@ tclsfe-x64.exe {punk9wintk903.vfs punk91 zip} #_beta convention: wrap freshly suite-built runtimes as *_beta kits even BEFORE they have passed the # punk test suites - so new-runtime behaviour can be trialled interactively. Drop _beta on acceptance. # (punk905_beta trialled and PROMOTED to punk905 2026-07-21.) -#todo: 'make.tcl bake' enhancement - build a single configured kit by name + list configured kits -# (sibling-ergonomics wish recorded in G-104, achieved 2026-07-25 WITHOUT this item - -# see goals/archive/G-104-maketcl-buildsuite-surface.md; still open) +#'make.tcl bakelist ?kitname ...?' lists the kit outputs configured here (runtime/vfs presence + +# deployed state); 'make.tcl bake ?kitname ...?' bakes/deploys only the named kits (G-121 - +# see goals/G-121-bakelist-selective-bake.md; wish originally recorded in archived G-104). #5 tclsh905.exe {punk9wintk905.vfs punk905 zip} diff --git a/src/tests/shell/testsuites/punkexe/maketclbakelist.test b/src/tests/shell/testsuites/punkexe/maketclbakelist.test new file mode 100644 index 00000000..2a94919b --- /dev/null +++ b/src/tests/shell/testsuites/punkexe/maketclbakelist.test @@ -0,0 +1,150 @@ +package require tcltest + +#Piped characterization of the make.tcl bakelist report and the bake-side selective +#kit-name validation (goal G-121). Runs the WORKING TREE's src/make.tcl under the +#built punk executable's 'script' subcommand with output captured through a pipe, +#and pins: +# - 'bakelist' exits 0, output is ESC-free (G-113 piped policy), the stable header +# row is present and the punk91 row carries the expected type/runtime/vfs columns +# with a deployed state from the documented vocabulary +# - 'bakelist ' filters to the named entry and appends the per-kit detail +# block; unfiltered rows do not appear +# - 'bakelist ' exits 1 naming the unknown and listing the configured names +# - 'bake ' exits 1 BEFORE any build ("nothing built"), listing the +# configured names - the no-build guarantee of selective bake +#The row pins are characterization of the current src/runtime/mapvfs.config kit +#matrix (punk91 = tclsfe-x64 + punk9wintk903.vfs, zip) - a deliberate config change +#legitimately updates them. +# +#Target executable resolved from env(PUNK_SHELL_TEST_EXE), else /bin/punk902z.exe +#then /bin/punkshell902. Skipped (constraint punkexeavailable) if none found. + +namespace eval ::testspace { + namespace import ::tcltest::* + + variable testdir [file dirname [file normalize [info script]]] + #/src/tests/shell/testsuites/punkexe -> 5 levels up to + variable projectroot [file normalize [file join $testdir .. .. .. .. ..]] + variable maketcl [file join $projectroot src make.tcl] + + variable punkexe "" + if {[info exists ::env(PUNK_SHELL_TEST_EXE)] && $::env(PUNK_SHELL_TEST_EXE) ne ""} { + set punkexe [file normalize $::env(PUNK_SHELL_TEST_EXE)] + } else { + foreach candidate [list [file join $projectroot bin punk902z.exe] [file join $projectroot bin punkshell902]] { + if {[file exists $candidate]} { + set punkexe $candidate + break + } + } + } + testConstraint punkexeavailable [expr {$punkexe ne "" && [file exists $punkexe]}] + + variable maketcl_run_timeout_ms 60000 + + variable runstate + array set runstate {} + + proc maketcl_run_read {chan} { + variable runstate + append runstate(output) [read $chan] + if {[chan eof $chan]} { + chan event $chan readable {} + set runstate(done) eof + } + } + + #Run script src/make.tcl with output captured through a + #pipe (stdin half-closed for immediate EOF - make.tcl must never wait on stdin for + #these subcommands). Returns dict: timedout 0|1, exitcode , output + #. + proc maketcl_run {cmdargs} { + variable runstate + variable maketcl_run_timeout_ms + variable punkexe + variable maketcl + array unset runstate + set runstate(output) "" + set runstate(done) "" + + set chan [open |[list $punkexe script $maketcl {*}$cmdargs 2>@1] r+] + chan configure $chan -blocking 0 -translation binary + catch {chan close $chan write} ;#no stdin for the child - immediate EOF + set timerid [after $maketcl_run_timeout_ms [list set [namespace current]::runstate(done) timeout]] + chan event $chan readable [list [namespace current]::maketcl_run_read $chan] + while {$runstate(done) eq ""} { + vwait [namespace current]::runstate(done) + } + after cancel $timerid + set timedout [expr {$runstate(done) eq "timeout"}] + set exitcode "" + if {$timedout} { + catch {exec {*}[auto_execok taskkill] /F /T /PID [lindex [pid $chan] 0]} + catch {chan close $chan} + } else { + chan configure $chan -blocking 1 + if {[catch {chan close $chan} errdata errdict]} { + set exitcode [lindex [dict get $errdict -errorcode] end] + } else { + set exitcode 0 + } + } + return [dict create timedout $timedout exitcode $exitcode output $runstate(output)] + } + + proc esc_count {text} { + return [regexp -all {\x1b} $text] + } + + #added 2026-07-26 (agent, G-121) + test maketcl_bakelist_full_report {piped make.tcl bakelist: exit 0, ESC-free, header row + punk91 row with documented deployed vocabulary} -constraints {punkexeavailable} -body { + set r [maketcl_run {bakelist}] + set out [dict get $r output] + set result [list] + lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode] + lappend result esc [esc_count $out] + lappend result header [regexp {(?n)^kit\s+type\s+runtime\s+vfs\s+deployed\s*$} $out] + lappend result punk91row [regexp {(?n)^punk91\s+zip\s+tclsfe-x64\s+punk9wintk903\.vfs\s+(current|stale|absent|nobuild)\y} $out] + set result + } -result {timedout 0 exitcode 0 esc 0 header 1 punk91row 1} + + #added 2026-07-26 (agent, G-121) + test maketcl_bakelist_filtered_detail {bakelist punk91 filters to the named entry and appends the per-kit detail block} -constraints {punkexeavailable} -body { + set r [maketcl_run {bakelist punk91}] + set out [dict get $r output] + set result [list] + lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode] + lappend result punk91row [regexp {(?n)^punk91\s+zip\s+} $out] + lappend result otherrows [regexp {(?n)^(punk905|punkbi|punksys|punk86)\s} $out] + lappend result detailblock [regexp {(?n)^detail: punk91\s*$} $out] + lappend result detailpaths [regexp {(?n)^\s+build product: src/_build/punk91} $out] + set result + } -result {timedout 0 exitcode 0 punk91row 1 otherrows 0 detailblock 1 detailpaths 1} + + #added 2026-07-26 (agent, G-121) + test maketcl_bakelist_unknown_name {bakelist with an unknown kit name exits 1 naming it and listing the configured names} -constraints {punkexeavailable} -body { + set r [maketcl_run {bakelist nosuchkitxyz}] + set out [dict get $r output] + set result [list] + lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode] + lappend result named [regexp {bakelist: unknown kit name\(s\): nosuchkitxyz} $out] + lappend result confignames [regexp {configured kit outputs: .*punk91} $out] + set result + } -result {timedout 0 exitcode 1 named 1 confignames 1} + + #added 2026-07-26 (agent, G-121) + test maketcl_bake_unknown_name_no_build {bake with an unknown kit name exits 1 before any build, listing the configured names} -constraints {punkexeavailable} -body { + set r [maketcl_run {bake nosuchkitxyz}] + set out [dict get $r output] + set result [list] + lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode] + lappend result nothingbuilt [regexp {make\.tcl bake: unknown kit name\(s\): nosuchkitxyz - nothing built} $out] + lappend result confignames [regexp {configured kit outputs: .*punk91} $out] + #the run must never reach the kit machinery (no vfs scan / kit processing output) + lappend result nomachinery [expr {![regexp {processing targetkit:} $out]}] + set result + } -result {timedout 0 exitcode 1 nothingbuilt 1 confignames 1 nomachinery 1} + + cleanupTests +} +namespace delete ::testspace