Browse Source

G-139 phase 1: make.tcl libfetch + bin/packages input tier (project 0.31.0)

New 'make.tcl libfetch' subcommand (argdoc'd; workflow text + plain help
updated): fetches the punkbin LIB-TIER artifacts declared in the new
src/runtime/libpackages.toml ([artifact.<name>] name/target; the four G-138
r1 artifacts, both generations) from <origin>/lib/<target>/ into the
untracked bin/packages/<target>/ input tier - zip + .toml sidecar, each
sha1-verified against the server's per-target sha1sums.txt (local tier copy
kept, the punk-runtime store idiom) - then materializes each verified zip
via punk::zip::unzip (crc-verified, mtimes restored, accelerator-eligible)
into bin/packages/<target>/tcl<N>/<pkgfolder>/ (generation subdirs: both
generations install identically-named folders). Idempotent: present
artifacts re-verify and skip; a materialized tree is current while its
embedded record names the declared artifact (the G-138 self-description
payoff); -force overrides.

Consent keyed to server trust (the G-123 posture): the canonical punkbin
origin (punk-runtime raw-base convention) gates nothing; any other origin
(-serverurl or env PUNKBIN_URL) refuses before network access without the
explicit -trust-server flag - never an interactive prompt. file:// origins
served natively (mirrors/fixtures); http(s) transport: Tcl http (+tls) when
loadable, else curl, else PowerShell. The dispatch block sits before the
build-command whitelist gate per the standalone-command pattern and exits
itself.

Emission-side amendment (both suites' library_artifacts.tcl): sidecar tomls
now ride the per-tier staging sha1sums.txt, matching the punkbin repo's
build_sha1sums.tcl coverage so staging-derived mirrors serve verifiable
sidecars.

Verified against a file:// mirror assembled from both suites' staging:
trust-gate refusal, consented fetch of all 4 artifacts (sha1-verified),
materialization of all 4 trees, fully idempotent re-run, and a package-load
proof from the tier (md5 2.0.9 + tcllibc accel=1 under the plain family
kit, ifneeded paths in bin/packages). bin/packages is already ignored in
both VCS via the /bin/* rules. Docs: bin/AGENTS.md tier section,
src/runtime/AGENTS.md libpackages.toml bullet. Project version 0.31.0 +
CHANGELOG (new make.tcl subcommand = product surface). Remaining G-139 work
recorded in the goal's Progress section.

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 2 days ago
parent
commit
11200dbf8a
  1. 15
      CHANGELOG.md
  2. 20
      bin/AGENTS.md
  3. 60
      goals/G-139-punkbin-library-consumption.md
  4. 2
      punkproject.toml
  5. 3
      src/buildsuites/suite_tcl86/tools/library_artifacts.tcl
  6. 3
      src/buildsuites/suite_tcl90/tools/library_artifacts.tcl
  7. 384
      src/make.tcl
  8. 1
      src/runtime/AGENTS.md
  9. 47
      src/runtime/libpackages.toml

15
CHANGELOG.md

@ -5,6 +5,21 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to
Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` Entries are newest-first; one bullet per notable change. See the root `AGENTS.md`
"Project Versioning" section for the bump policy. "Project Versioning" section for the bump policy.
## [0.31.0] - 2026-07-30
- New `make.tcl libfetch` subcommand (G-139 phase 1): fetches the punkbin
lib-tier library artifacts declared in `src/runtime/libpackages.toml`
(suite-built tcllib/tcllibc zips, both Tcl generations - G-138) into the
untracked `bin/packages/<target>/` input tier with sha1 verification
against the server's per-target sha1sums.txt, and materializes the
installed-shape package trees under `bin/packages/<target>/tcl<N>/`
(self-describing via their embedded punkbin-artifact.toml records).
Consent is keyed to server trust: the canonical punkbin origin needs no
flag; any other origin (`-serverurl` / env `PUNKBIN_URL`) requires the
explicit `-trust-server` flag. `file://` origins serve local mirrors and
fixtures. Idempotent: present artifacts that verify are skipped and
current trees are left untouched (`-force` overrides).
## [0.30.7] - 2026-07-29 ## [0.30.7] - 2026-07-29
- Baked kits pinned archive-relative (G-134): every assembled kit image is - Baked kits pinned archive-relative (G-134): every assembled kit image is

20
bin/AGENTS.md

@ -361,6 +361,26 @@ punkshell platform-dir names defined by the `punk::platform` module and surfaced
aarch64->arm64; the bash payload's local-platform prongs emit canonical names - the aarch64->arm64; the bash payload's local-platform prongs emit canonical names - the
FreeBSD dir was realigned from `freebsd-amd64` to `freebsd-x86_64` accordingly). FreeBSD dir was realigned from `freebsd-amd64` to `freebsd-x86_64` accordingly).
### Library package input tier (`bin/packages/`, G-139)
`bin/packages/<target>/` is the untracked input tier for punkbin LIB-TIER library
artifacts (tcllib/tcllibc as immutable generation-tagged zips - shellspy G-138),
mirroring the `bin/runtime/<platform>/` store idiom: per-target folders holding
the sha1-verified `-r<N>` zips, their `.toml` sidecars and a local copy of the
server's `sha1sums.txt`, plus the materialized installed-shape package trees
under `bin/packages/<target>/tcl<N>/<pkgfolder>/` (generation subfolders because
both Tcl generations install identically-named folders). Materialized trees are
self-describing via their embedded `punkbin-artifact.toml` records (the G-138
schema) - `make.tcl libfetch` uses that record to skip re-materializing a
current tree. Populated by `tclsh src/make.tcl libfetch` from the declarations
in `src/runtime/libpackages.toml` (consent keyed to server trust: the canonical
punkbin origin needs no flag, any other origin - `-serverurl` or env
`PUNKBIN_URL` - requires the explicit `-trust-server` flag; `file://` origins
serve local mirrors/fixtures). Everything under `bin/packages/` is
fetch-regenerable and ignored by both VCS (`/bin/*` rules) - never commit from
it; downstream consumption into `lib_tcl<N>/` and kit vfs trees is the
remainder of G-139.
### Interactive verification shells ### Interactive verification shells
- Interactive console/repl verification should cover both Tcl generations - behaviour can differ materially (e.g. the Tcl 8.6 windows console channel driver vs the Tcl 9 rewrite). Use a Tcl 8.6-based punk shell (`punksys.exe`) and a current Tcl 9-based punk shell (named for the Tcl release it embeds, e.g. `punk902z.exe` at the time of writing - ask the user which is current rather than assuming). `info patchlevel` in-session confirms the runtime. - Interactive console/repl verification should cover both Tcl generations - behaviour can differ materially (e.g. the Tcl 8.6 windows console channel driver vs the Tcl 9 rewrite). Use a Tcl 8.6-based punk shell (`punksys.exe`) and a current Tcl 9-based punk shell (named for the Tcl release it embeds, e.g. `punk902z.exe` at the time of writing - ask the user which is current rather than assuming). `info patchlevel` in-session confirms the runtime.

60
goals/G-139-punkbin-library-consumption.md

@ -119,3 +119,63 @@ State at drafting worth recording:
- Activation survey 2026-07-30: goals_xref score G-139 re-run at activation - Activation survey 2026-07-30: goals_xref score G-139 re-run at activation
(user-directed, same day as drafting - no goals drafted in the interval); (user-directed, same day as drafting - no goals drafted in the interval);
no overlaps beyond the drafting survey. no overlaps beyond the drafting survey.
## Progress
- 2026-07-30 phase 1 landed: the fetch/materialize input tier and its
make.tcl entrypoint (project 0.31.0).
- `make.tcl libfetch` (argdoc'd subcommand, workflow-text + plain-help
updated): reads `src/runtime/libpackages.toml` declarations
([artifact.<name>] name/target), fetches from <origin>/lib/<target>/
into `bin/packages/<target>/` - zip + .toml sidecar, each sha1-verified
against the server's fetched per-target sha1sums.txt (kept as the local
tier copy, the punk-runtime store idiom) - then materializes each
verified zip via punk::zip::unzip (defaults: crc-verified, mtimes
restored, punkzip-accelerator-eligible) into
`bin/packages/<target>/tcl<N>/<pkgfolder>/` (generation subdirs: both
generations install identically-named folders). Idempotent: present
artifacts re-verify and are skipped; a materialized tree is skipped
while its embedded record's name matches the declared artifact (the
G-138 self-description payoff); -force re-does both.
- Consent (per this goal's acceptance + the G-123 posture): canonical
punkbin origin (the punk-runtime raw-base convention) prompts nothing;
ANY other origin - -serverurl flag or PUNKBIN_URL env - refuses before
network access without the explicit -trust-server flag. Never an
interactive prompt. file:// origins supported natively (local mirrors,
fixtures). Transport for http(s): Tcl http (+tls for https) when
loadable, else curl, else PowerShell.
- Verified 2026-07-30 against a file:// mirror assembled from BOTH
suites' out/library/punkbin staging (the shape publication will
produce): trust-gate refusal without the flag; consented fetch of all
4 declared r1 artifacts with sha1 verification; materialization of all
4 trees; full idempotent re-run (verified-present + tree-current);
package-load proof from the tier (md5 2.0.9 + tcllibc from
bin/packages/*/tcl9 under the plain family kit, accelerator engaged,
ifneeded source path in the tier).
- Emission-side amendment (both suites' library_artifacts.tcl): sidecar
tomls now ride the per-tier staging sha1sums.txt (the punkbin repo's
build_sha1sums.tcl already sums them; the staging mirror now matches),
so sidecars are verifiable from a staging-derived mirror too.
- bin/packages is ignored by both VCS via the existing /bin/* rules (no
ignore edits needed). Docs: bin/AGENTS.md "Library package input tier",
src/runtime/AGENTS.md libpackages.toml bullet, workflow diagram block.
- Session note: the suite_tcl86 staging r1 zips changed bytes when its
installer/critcl Run steps re-ran under this session's different PATH
(zig Run cache keys include the env) - immaterial pre-publication, and
exactly the deliberate-publish revision discipline's point; the tcl9
zips stayed byte-identical across three independent emissions.
- Remaining for acceptance: libs-step materialization of
lib_tcl<N>/allplatforms + lib_tcl<N>/<platform> from the tier with
punkcheck provenance; tcllib/tcllibc vendorlib_vfs.toml (or successor)
declarations with supersedes removing tcllib1.21 (incl. nested tcllibc
0.4 + md5c) from 8.6-family kits; punk86-family tcllib 2.0 upgrade
validated by kit-hosted runtests vs a same-day baseline (differences
dispositioned); vendored src/vendorlib_tcl8|9 tcllib-tree removal in its
own commit after a bake fed from the artifact path reproduces working
kits; a bake on a checkout without those trees shipping kits with
verifiable embedded provenance; the derived-project decision recorded
here (fetch default; in-project suite build per G-104 as the documented
alternative - the stale samplesuite1 download_and_build.config stub to be
replaced/removed when recording it). Also pending: publication of the r1
artifacts to the real punkbin origin (user step) so libfetch's canonical
path goes live end-to-end.

2
punkproject.toml

@ -1,4 +1,4 @@
[project] [project]
name = "punkshell" name = "punkshell"
version = "0.30.7" version = "0.31.0"
license = "BSD-2-Clause" license = "BSD-2-Clause"

3
src/buildsuites/suite_tcl86/tools/library_artifacts.tcl

@ -383,6 +383,9 @@ foreach {folder tier license testlibs pkgpath} $opt(-packages) {
fconfigure $f -translation lf fconfigure $f -translation lf
puts $f [compose_record $ident [dict create sha1 $sha1 size $size built $built] $tests] puts $f [compose_record $ident [dict create sha1 $sha1 size $size built $built] $tests]
close $f close $f
#sidecars ride sha1sums too (punkbin repo convention: build_sha1sums.tcl sums
#every tracked file) - so a staging mirror serves verifiable sidecars as well
dict lappend sha1lines $tier "[sha1::sha1 -hex -file $mf] *[file tail $mf]"
lappend emitted "$folder -> lib/$tier/$artifact" lappend emitted "$folder -> lib/$tier/$artifact"
note "emitted lib/$tier/$artifact (sha1 $sha1, [expr {$size/1024}] KB, $members members) + [file tail $mf]" note "emitted lib/$tier/$artifact (sha1 $sha1, [expr {$size/1024}] KB, $members members) + [file tail $mf]"
} }

3
src/buildsuites/suite_tcl90/tools/library_artifacts.tcl

@ -383,6 +383,9 @@ foreach {folder tier license testlibs pkgpath} $opt(-packages) {
fconfigure $f -translation lf fconfigure $f -translation lf
puts $f [compose_record $ident [dict create sha1 $sha1 size $size built $built] $tests] puts $f [compose_record $ident [dict create sha1 $sha1 size $size built $built] $tests]
close $f close $f
#sidecars ride sha1sums too (punkbin repo convention: build_sha1sums.tcl sums
#every tracked file) - so a staging mirror serves verifiable sidecars as well
dict lappend sha1lines $tier "[sha1::sha1 -hex -file $mf] *[file tail $mf]"
lappend emitted "$folder -> lib/$tier/$artifact" lappend emitted "$folder -> lib/$tier/$artifact"
note "emitted lib/$tier/$artifact (sha1 $sha1, [expr {$size/1024}] KB, $members members) + [file tail $mf]" note "emitted lib/$tier/$artifact (sha1 $sha1, [expr {$size/1024}] KB, $members members) + [file tail $mf]"
} }

384
src/make.tcl

@ -187,7 +187,7 @@ namespace eval ::punkboot {
variable foldername [file tail $scriptfolder] variable foldername [file tail $scriptfolder]
variable pkg_requirements [list]; variable pkg_missing [list];variable pkg_loaded [list] variable pkg_requirements [list]; variable pkg_missing [list];variable pkg_loaded [list]
variable help_flags [list -help --help /? -h] variable help_flags [list -help --help /? -h]
variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate bootsupport vfscommonupdate projectversion workflow buildsuite tool project vfs] variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate libfetch bootsupport vfscommonupdate projectversion workflow buildsuite tool project vfs]
} }
@ -2135,6 +2135,10 @@ proc ::punkboot::punkboot_gethelp {args} {
append h " - propagate declared vendored platform-library packages into kit vfs lib_tcl<N> trees" \n append h " - propagate declared vendored platform-library packages into kit vfs lib_tcl<N> trees" \n
append h " - declarations (per-package per-kit, with superseded-version removal) live in src/runtime/vendorlib_vfs.toml" \n append h " - declarations (per-package per-kit, with superseded-version removal) live in src/runtime/vendorlib_vfs.toml" \n
append h " - also runs automatically as a phase of '$scriptname bake' (and therefore of '$scriptname bakehouse')" \n \n append h " - also runs automatically as a phase of '$scriptname bake' (and therefore of '$scriptname bakehouse')" \n \n
append h " $scriptname libfetch ?-serverurl <url>? ?-trust-server? ?-force?" \n
append h " - fetch declared punkbin lib-tier library artifacts (src/runtime/libpackages.toml) into bin/packages/<target>" \n
append h " - sha1-verified against the server's per-target sha1sums.txt; materializes package trees under bin/packages/<target>/tcl<N>" \n
append h " - canonical origin trusted by default; other servers (incl. PUNKBIN_URL overrides) require -trust-server" \n \n
append h " $scriptname vfscommonupdate" \n append h " $scriptname vfscommonupdate" \n
append h " - promotion gate: update the src/vfs/_vfscommon.vfs from compiled src/modules and src/lib etc" \n append h " - promotion gate: update the src/vfs/_vfscommon.vfs from compiled src/modules and src/lib etc" \n
append h " - before calling this (followed by '$scriptname bake') - you can test using '<builtexe>(.exe) dev'" \n append h " - before calling this (followed by '$scriptname bake') - you can test using '<builtexe>(.exe) dev'" \n
@ -2323,6 +2327,26 @@ DIAGRAM 2 - KIT ASSEMBLY DETAIL (the 'make.tcl bake' stage; incl. vfslibs phase)
running shell still holds the old exe [K5] running shell still holds the old exe [K5]
LIBRARY-ARTIFACT FETCH (G-139; input tier so far - lib_tcl<N>/kit consumption follows)
----------------------------------------------------------------------------
src/runtime/libpackages.toml which punkbin LIB-TIER artifacts this checkout
consumes (declared artifact names + targets)
|
| 'make.tcl libfetch' - sha1-verified download of the declared
| <pkg>-<tcl8|tcl9>-r<N>.zip artifacts + .toml sidecars from
| <origin>/lib/<target>/ (canonical punkbin origin trusted by
| default; -serverurl/PUNKBIN_URL override needs -trust-server;
| file:// origins serve local mirrors/fixtures), then
| materialization of each verified zip's package tree
v
bin/packages/<target>/ verified -r<N> zips + toml sidecars (input tier,
untracked - mirrors bin/runtime/<target>)
bin/packages/<target>/tcl<N>/<pkgfolder>/
materialized installed-shape trees, self-describing
via their embedded punkbin-artifact.toml records
KEY / NOTES KEY / NOTES
---------------------------------------------------------------------------- ----------------------------------------------------------------------------
[K1] Source modules carry the literal magic version 999999.0a1.0 in the filename. [K1] Source modules carry the literal magic version 999999.0a1.0 in the filename.
@ -2527,6 +2551,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
vfslibs "Propagate declared vendored platform-library packages into kit vfs lib_tcl<N> trees" vfslibs "Propagate declared vendored platform-library packages into kit vfs lib_tcl<N> trees"
bin "Install executables from src/bin into <projectdir>/bin, then build kits as for bake" bin "Install executables from src/bin into <projectdir>/bin, then build kits as for bake"
vendorupdate "Update src/vendormodules based on src/vendormodules/include_modules.config" vendorupdate "Update src/vendormodules based on src/vendormodules/include_modules.config"
libfetch "Fetch declared punkbin lib-tier library artifacts into bin/packages/<target> (sha1-verified) and materialize the package trees"
bootsupport "Promotion gate: update src/bootsupport modules from built project modules" bootsupport "Promotion gate: update src/bootsupport modules from built project modules"
vfscommonupdate "Promotion gate: replace src/vfs/_vfscommon.vfs modules and libs with the project's built set" vfscommonupdate "Promotion gate: replace src/vfs/_vfscommon.vfs modules and libs with the project's built set"
info "Show the name and base folder of the project to be built" info "Show the name and base folder of the project to be built"
@ -2630,6 +2655,24 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
Also runs automatically as a phase of 'make.tcl bake' (and therefore Also runs automatically as a phase of 'make.tcl bake' (and therefore
of 'make.tcl bakehouse'); this standalone narrowing serves quick of 'make.tcl bakehouse'); this standalone narrowing serves quick
iteration on the declarations." iteration on the declarations."
libfetch
"
Fetch the punkbin LIB-TIER library artifacts declared in
src/runtime/libpackages.toml into the untracked input tier
bin/packages/<target>/ - immutable <pkg>-<tcl8|tcl9>-r<N>.zip
artifacts plus their .toml sidecars, each verified against the
server's per-target sha1sums.txt - then materialize each verified
zip's installed-shape package tree under
bin/packages/<target>/tcl<N>/ (self-describing via the embedded
punkbin-artifact.toml record; G-138/G-139).
Present artifacts that already verify are skipped (-force
re-downloads). The canonical punkbin origin is trusted by
default; -serverurl (or env PUNKBIN_URL) overrides it, and any
NON-canonical origin requires the explicit -trust-server flag
(never an interactive prompt - unattended runs stay unattended).
A file:// origin is supported for local mirrors/fixtures.
Transport for http(s): Tcl http+tls when loadable, else curl,
else PowerShell (windows)."
bin bin
" "
Install executables from src/bin to <projectdir>/bin, then check vfs Install executables from src/bin to <projectdir>/bin, then check vfs
@ -2799,6 +2842,30 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
(packages -> test -> bootsupport/vfscommonupdate -> commit -> bake). (packages -> test -> bootsupport/vfscommonupdate -> commit -> bake).
Pass -dirty-abort 0 to build anyway with PROVENANCE-WARNINGs." Pass -dirty-abort 0 to build anyway with PROVENANCE-WARNINGs."
} }
#libfetch fragments (G-139): consent is keyed to SERVER TRUST (the G-123
#posture) - the canonical punkbin origin needs no acknowledgement, any
#other origin needs the explicit flag. Never an interactive prompt.
variable OPT_SERVERURL {
-serverurl -type string -default "" -help\
"Artifact server base url override (default: the canonical punkbin
origin; env PUNKBIN_URL overrides the default, this flag overrides
both). The lib tier is addressed as <serverurl>/lib/<target>/.
file:// urls are supported for local mirrors and fixtures.
A NON-canonical origin additionally requires -trust-server."
}
variable OPT_TRUSTSERVER {
-trust-server -type none -default 0 -help\
"Acknowledge fetching from a NON-canonical artifact server
(consent keyed to server trust - the interim G-123/G-006
mechanism). Without this flag, a non-canonical -serverurl or
PUNKBIN_URL aborts before any network access. Fetches from the
canonical origin never require it."
}
variable OPT_FORCE {
-force -type none -default 0 -help\
"Re-download and re-materialize even when a present artifact
already verifies against the server sha1sums."
}
#which shared option fragments each subcommand declares #which shared option fragments each subcommand declares
#(shell and help have bespoke definitions below: shell passes trailing args #(shell and help have bespoke definitions below: shell passes trailing args
# through to the repl untouched, help takes a subject value) # through to the repl untouched, help takes a subject value)
@ -2812,6 +2879,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
vfslibs {OPT_DIRTYABORT OPT_CONFIRM} vfslibs {OPT_DIRTYABORT OPT_CONFIRM}
bin {OPT_FORCEKILL OPT_DIRTYABORT OPT_CONFIRM} bin {OPT_FORCEKILL OPT_DIRTYABORT OPT_CONFIRM}
vendorupdate {OPT_CONFIRM} vendorupdate {OPT_CONFIRM}
libfetch {OPT_SERVERURL OPT_TRUSTSERVER OPT_FORCE}
bootsupport {OPT_DIRTYABORT OPT_CONFIRM} bootsupport {OPT_DIRTYABORT OPT_CONFIRM}
vfscommonupdate {OPT_DIRTYABORT OPT_CONFIRM} vfscommonupdate {OPT_DIRTYABORT OPT_CONFIRM}
info {OPT_CONFIRM} info {OPT_CONFIRM}
@ -2829,6 +2897,9 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
OPT_DIRTYABORT "?-dirty-abort?" OPT_DIRTYABORT "?-dirty-abort?"
OPT_DIRTYABORT_ON "?-dirty-abort 1|0?" OPT_DIRTYABORT_ON "?-dirty-abort 1|0?"
OPT_CONFIRM "?-confirm 0|1?" OPT_CONFIRM "?-confirm 0|1?"
OPT_SERVERURL "?-serverurl <url>?"
OPT_TRUSTSERVER "?-trust-server?"
OPT_FORCE "?-force?"
} }
#per-subcommand positional values (default: none). bake (and its deprecated #per-subcommand positional values (default: none). bake (and its deprecated
#alias vfs) and bakelist take optional kit names (G-121 selective bake). #alias vfs) and bakelist take optional kit names (G-121 selective bake).
@ -2940,7 +3011,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
variable SUBGROUPS { variable SUBGROUPS {
"build & bake" {bakehouse packages modules libs bake vfslibs bin} "build & bake" {bakehouse packages modules libs bake vfslibs bin}
"promotion gates" {bootsupport vfscommonupdate} "promotion gates" {bootsupport vfscommonupdate}
"source maintenance" {vendorupdate} "source maintenance" {vendorupdate libfetch}
"buildsuites" {buildsuite} "buildsuites" {buildsuite}
"vendored tools" {tool} "vendored tools" {tool}
"informational" {info check bakelist projectversion workflow help} "informational" {info check bakelist projectversion workflow help}
@ -2996,6 +3067,9 @@ set help_exitcode 0
set ::punkboot::opt_forcekill 0 set ::punkboot::opt_forcekill 0
set ::punkboot::opt_dirty_abort 0 set ::punkboot::opt_dirty_abort 0
set ::punkboot::opt_confirm 1 set ::punkboot::opt_confirm 1
set ::punkboot::opt_serverurl ""
set ::punkboot::opt_trust_server 0
set ::punkboot::opt_force 0
set ::punkboot::opt_kitnames [list] ;#G-121: requested kit names for bake/bakelist (empty = all configured) set ::punkboot::opt_kitnames [list] ;#G-121: requested kit names for bake/bakelist (empty = all configured)
if {$::punkboot::punkargs_ok} { if {$::punkboot::punkargs_ok} {
@ -3046,7 +3120,7 @@ if {$::punkboot::punkargs_ok} {
} }
set ::punkboot::command $subcommand set ::punkboot::command $subcommand
set _opts [dict get $argd opts] set _opts [dict get $argd opts]
foreach {_optname _var} {-k opt_forcekill -dirty-abort opt_dirty_abort -confirm opt_confirm} { foreach {_optname _var} {-k opt_forcekill -dirty-abort opt_dirty_abort -confirm opt_confirm -serverurl opt_serverurl -trust-server opt_trust_server -force opt_force} {
if {[dict exists $_opts $_optname]} { if {[dict exists $_opts $_optname]} {
set ::punkboot::$_var [dict get $_opts $_optname] set ::punkboot::$_var [dict get $_opts $_optname]
} }
@ -3083,6 +3157,12 @@ if {$::punkboot::punkargs_ok} {
switch -- $a { switch -- $a {
-k {set ::punkboot::opt_forcekill 1} -k {set ::punkboot::opt_forcekill 1}
-dirty-abort {set ::punkboot::opt_dirty_abort 1} -dirty-abort {set ::punkboot::opt_dirty_abort 1}
-trust-server {set ::punkboot::opt_trust_server 1}
-force {set ::punkboot::opt_force 1}
-serverurl {
set ::punkboot::opt_serverurl [lindex $argsleft 0]
set argsleft [lrange $argsleft 1 end]
}
-confirm { -confirm {
set v [lindex $argsleft 0] set v [lindex $argsleft 0]
set argsleft [lrange $argsleft 1 end] set argsleft [lrange $argsleft 1 end]
@ -5121,6 +5201,303 @@ if {$::punkboot::command eq "bootsupport"} {
if {$::punkboot::command eq "libfetch"} {
#G-139 phase 1: fetch the punkbin LIB-TIER artifacts declared in
#src/runtime/libpackages.toml into the untracked bin/packages/<target> input
#tier (mirroring bin/runtime/<target>) with sha1 verification against the
#server's per-target sha1sums.txt, then materialize each verified zip's
#installed-shape package tree under bin/packages/<target>/tcl<N>/ (the tree
#is self-describing via its embedded punkbin-artifact.toml - the G-138
#record). Consumption into lib_tcl<N>/ and kit vfs trees is the remainder
#of G-139 - this entrypoint owns fetch + verify + materialize only.
proc ::punkboot::libfetch_transfer {url outfile} {
#one url -> local file. file:// is served natively (local mirrors,
#fixtures - no network, no external tools). http(s) transports in
#order: Tcl http (+tls for https) when loadable, curl (ubiquitous,
#including stock windows 10+), powershell Invoke-WebRequest (windows).
#Returns the transport used; errors when all fail.
file mkdir [file dirname $outfile]
if {[string match -nocase "file://*" $url]} {
set path [string range $url 7 end]
if {[regexp {^/+([A-Za-z]:/.*)$} $path -> winpath]} {
set path $winpath ;#file:///C:/x -> C:/x
}
if {![file isfile $path]} {
error "file origin has no such file: $path"
}
file copy -force $path $outfile
return "file-copy"
}
set is_https [string match -nocase "https://*" $url]
set tcl_http_ok 0
if {![catch {package require http}]} {
if {!$is_https} {
set tcl_http_ok 1
} elseif {![catch {package require tls}]} {
catch {http::register https 443 [list ::tls::socket -autoservername true]}
set tcl_http_ok 1
}
}
if {$tcl_http_ok} {
set fd [open $outfile wb]
set ok 0
if {![catch {http::geturl $url -channel $fd -binary 1} tok]} {
if {[http::status $tok] eq "ok" && [http::ncode $tok] == 200} {set ok 1}
http::cleanup $tok
}
close $fd
if {$ok} {return "tcl-http"}
file delete -force $outfile
}
set curl [auto_execok curl]
if {$curl ne ""} {
if {![catch {exec {*}$curl --fail -sSL -o $outfile $url 2>@1} curlerr]} {
return "curl"
}
file delete -force $outfile
}
if {$::tcl_platform(platform) eq "windows"} {
set ps [auto_execok powershell]
if {$ps ne ""} {
set pscmd "\$ProgressPreference='SilentlyContinue'; Invoke-WebRequest -Uri '$url' -OutFile '$outfile' -ErrorAction Stop"
if {![catch {exec {*}$ps -NoProfile -NonInteractive -Command $pscmd 2>@1} pserr]} {
return "powershell"
}
file delete -force $outfile
}
}
set detail ""
if {[info exists curlerr]} {set detail " (curl: [string range $curlerr 0 200])"}
error "all transports failed for $url$detail"
}
set libfetch_config $sourcefolder/runtime/libpackages.toml
#Canonical origin: the punkbin raw-file base (the punk-runtime.cmd
#convention; lib tier addressed as <origin>/lib/<target>/...).
#Server-trust consent (G-139, the G-123 posture): the canonical origin
#never gates; ANY other origin - flag or PUNKBIN_URL env - requires the
#explicit -trust-server flag. Never an interactive prompt: unattended
#runs stay unattended, and refusal happens before any network access.
set libfetch_canonical "https://www.gitea1.intx.com.au/jn/punkbin/raw/branch/master"
set libfetch_origin $libfetch_canonical
if {[info exists ::env(PUNKBIN_URL)] && $::env(PUNKBIN_URL) ne ""} {
set libfetch_origin $::env(PUNKBIN_URL)
}
if {$::punkboot::opt_serverurl ne ""} {
set libfetch_origin $::punkboot::opt_serverurl
}
set libfetch_origin [string trimright $libfetch_origin /]
if {$libfetch_origin ne $libfetch_canonical && !$::punkboot::opt_trust_server} {
puts stderr "$A(BAD)LIBFETCH: origin '$libfetch_origin' is not the canonical punkbin origin$A(RST)"
puts stderr "LIBFETCH: canonical: $libfetch_canonical"
puts stderr "LIBFETCH: fetching from a non-canonical server requires the explicit -trust-server flag (consent keyed to server trust - G-139/G-123). Nothing was fetched."
exit 3
}
if {![file exists $libfetch_config]} {
puts stdout "LIBFETCH: no config at $libfetch_config - nothing declared, nothing to fetch"
} else {
if {[catch {package require tomlish} tomlish_err]} {
puts stderr "$A(BAD)LIBFETCH: cannot process $libfetch_config - tomlish package unavailable ($tomlish_err)$A(RST)"
exit 3
}
if {[catch {package require sha1} sha1_err]} {
puts stderr "$A(BAD)LIBFETCH: sha1 package unavailable ($sha1_err) - cannot verify artifacts$A(RST)"
exit 3
}
if {[catch {package require punk::zip} punkzip_err]} {
puts stderr "$A(BAD)LIBFETCH: punk::zip package unavailable ($punkzip_err) - cannot materialize package trees$A(RST)"
exit 3
}
set fd [open $libfetch_config r]
fconfigure $fd -encoding utf-8
set libfetch_tomldata [read $fd]
close $fd
if {[catch {tomlish::to_dict [tomlish::from_toml $libfetch_tomldata]} libfetch_dict]} {
puts stderr "$A(BAD)LIBFETCH: failed to parse $libfetch_config\n$libfetch_dict$A(RST)"
exit 3
}
if {![llength [info commands ::punkboot::lib::toml_untag]]} {
#tomlish::to_dict values are type-tagged (also defined by the vfslibs
#phase - that block does not run for the libfetch command)
proc ::punkboot::lib::toml_untag {tagged} {
if {[dict get $tagged type] eq "ARRAY"} {
set plain [list]
foreach el [dict get $tagged value] {
lappend plain [::punkboot::lib::toml_untag $el]
}
return $plain
}
return [dict get $tagged value]
}
}
set libfetch_artifacts [dict create]
if {[dict exists $libfetch_dict artifact]} {
set libfetch_artifacts [dict get $libfetch_dict artifact]
}
if {![dict size $libfetch_artifacts]} {
puts stdout "LIBFETCH: no \[artifact.<name>\] entries found in $libfetch_config"
}
puts stdout "LIBFETCH: origin $libfetch_origin ([dict size $libfetch_artifacts] declared artifact(s))"
set libfetch_sums [dict create] ;#target -> dict(filename -> sha1) from the server's sha1sums.txt
set libfetch_failed 0
set libfetch_actions [list]
dict for {aname aentry} $libfetch_artifacts {
foreach reqkey {name target} {
if {![dict exists $aentry $reqkey]} {
puts stderr "$A(BAD)LIBFETCH: entry artifact.$aname in $libfetch_config is missing required key '$reqkey'$A(RST)"
exit 3
}
}
set artifact [punkboot::lib::toml_untag [dict get $aentry name]]
set target [punkboot::lib::toml_untag [dict get $aentry target]]
if {![regexp {^(.+)-(tcl[0-9]+)-r([0-9]+)\.zip$} $artifact -> pkgfolder generation rev]} {
puts stderr "$A(BAD)LIBFETCH: artifact.$aname name '$artifact' does not match <pkgfolder>-tcl<N>-r<N>.zip$A(RST)"
exit 3
}
set tierdir $projectroot/bin/packages/$target
file mkdir $tierdir
#server truth per target: lib/<target>/sha1sums.txt (fetched once per
#run per target; the local copy in the tier mirrors the punk-runtime
#store idiom)
if {![dict exists $libfetch_sums $target]} {
set sumsurl "$libfetch_origin/lib/$target/sha1sums.txt"
set sumslocal $tierdir/sha1sums.txt
if {[catch {::punkboot::libfetch_transfer $sumsurl $sumslocal} terr]} {
puts stderr "$A(BAD)LIBFETCH: cannot fetch $sumsurl - $terr$A(RST)"
exit 3
}
set sfd [open $sumslocal r]
set sumsdata [read $sfd]
close $sfd
set sums [dict create]
foreach ln [split $sumsdata \n] {
set ln [string trim $ln]
if {$ln eq ""} {continue}
set sp [string first " " $ln]
if {$sp < 1} {continue}
set h [string range $ln 0 $sp-1]
set tag [string range $ln $sp+1 end]
dict set sums [string range $tag 1 end] [string tolower $h]
}
dict set libfetch_sums $target $sums
}
set sums [dict get $libfetch_sums $target]
if {![dict exists $sums $artifact]} {
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: '$artifact' is not listed in the server's lib/$target/sha1sums.txt$A(RST)"
set libfetch_failed 1
continue
}
set want_sha1 [dict get $sums $artifact]
set ziplocal $tierdir/$artifact
set have_ok 0
if {[file isfile $ziplocal] && !$::punkboot::opt_force} {
if {[string tolower [sha1::sha1 -hex -file $ziplocal]] eq $want_sha1} {
set have_ok 1
}
}
if {!$have_ok} {
set zipurl "$libfetch_origin/lib/$target/$artifact"
set partfile $ziplocal.part
if {[catch {::punkboot::libfetch_transfer $zipurl $partfile} terr]} {
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: fetch failed - $terr$A(RST)"
set libfetch_failed 1
continue
}
set got_sha1 [string tolower [sha1::sha1 -hex -file $partfile]]
if {$got_sha1 ne $want_sha1} {
file delete -force $partfile
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: sha1 MISMATCH for $artifact (got $got_sha1, server sha1sums says $want_sha1) - deleted$A(RST)"
set libfetch_failed 1
continue
}
file rename -force $partfile $ziplocal
lappend libfetch_actions "fetched lib/$target/$artifact"
} else {
lappend libfetch_actions "verified-present lib/$target/$artifact"
}
#beside-toml sidecar (also sha1sums-listed by emission)
set sidecar "[string range $artifact 0 end-4].toml"
set sidecarlocal $tierdir/$sidecar
set sidecar_ok 0
if {[dict exists $sums $sidecar]} {
set want_side [dict get $sums $sidecar]
if {[file isfile $sidecarlocal] && !$::punkboot::opt_force} {
if {[string tolower [sha1::sha1 -hex -file $sidecarlocal]] eq $want_side} {
set sidecar_ok 1
}
}
if {!$sidecar_ok} {
if {[catch {::punkboot::libfetch_transfer "$libfetch_origin/lib/$target/$sidecar" $sidecarlocal.part} terr]} {
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: sidecar fetch failed - $terr$A(RST)"
set libfetch_failed 1
continue
}
if {[string tolower [sha1::sha1 -hex -file $sidecarlocal.part]] ne $want_side} {
file delete -force $sidecarlocal.part
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: sidecar sha1 MISMATCH for $sidecar - deleted$A(RST)"
set libfetch_failed 1
continue
}
file rename -force $sidecarlocal.part $sidecarlocal
}
} else {
puts stderr "LIBFETCH: note - $sidecar not listed in server sha1sums (artifact accepted; sidecar skipped)"
}
#materialize: installed-shape tree under <target>/tcl<N>/<pkgfolder>
#(generation subdir because both generations install identically-named
#folders). Fresh tree per materialization; -mtime restores member
#mtimes so the tree mirrors what was zipped.
set gendir $tierdir/$generation
file mkdir $gendir
set treedir $gendir/$pkgfolder
if {[file isdirectory $treedir] && !$::punkboot::opt_force} {
#re-materialize only when the tree's embedded record no longer
#matches this artifact name (self-describing tier - G-138)
set embrec $treedir/punkbin-artifact.toml
set tree_current 0
if {[file isfile $embrec]} {
set efd [open $embrec r]
set edata [read $efd]
close $efd
if {[regexp -line "^name = \"$artifact\"$" $edata]} {
set tree_current 1
}
}
if {$tree_current} {
lappend libfetch_actions "tree-current $target/$generation/$pkgfolder"
continue
}
}
file delete -force $treedir
#defaults: whole archive, -overwrite 1 -mtime 1 (member mtimes restored)
#-verify 1 (per-member crc32) - and accelerator-eligible (punkzip exe)
if {[catch {punk::zip::unzip $ziplocal $gendir} unzip_err]} {
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: materialization failed - $unzip_err$A(RST)"
file delete -force $treedir
set libfetch_failed 1
continue
}
if {![file isfile $treedir/pkgIndex.tcl] || ![file isfile $treedir/punkbin-artifact.toml]} {
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: materialized tree at $treedir lacks pkgIndex.tcl and/or punkbin-artifact.toml$A(RST)"
set libfetch_failed 1
continue
}
lappend libfetch_actions "materialized $target/$generation/$pkgfolder"
}
foreach act $libfetch_actions {
puts stdout "LIBFETCH: $act"
}
if {$libfetch_failed} {
puts stderr "$A(BAD)LIBFETCH: FAILED - one or more declared artifacts could not be fetched/verified/materialized (see above)$A(RST)"
exit 3
}
puts stdout "LIBFETCH: done ([llength $libfetch_actions] action(s)) -> $projectroot/bin/packages"
}
flush stdout
::exit 0
}
if {$::punkboot::command ni {bakehouse packages modules libs bake vfslibs bin}} { if {$::punkboot::command ni {bakehouse packages modules libs bake vfslibs bin}} {
puts stderr "Command $::punkboot::command not implemented - aborting." puts stderr "Command $::punkboot::command not implemented - aborting."
flush stderr flush stderr
@ -5341,6 +5718,7 @@ if {$::punkboot::command in {bakehouse bake vfslibs}} {
} }
} }
if {$::punkboot::command in {bakehouse packages libs}} { if {$::punkboot::command in {bakehouse packages libs}} {
######################################################## ########################################################
lappend projectlibfolders lib lappend projectlibfolders lib

1
src/runtime/AGENTS.md

@ -17,6 +17,7 @@ Houses the `mapvfs.config` that maps VFS payloads to platform runtimes, plus the
- **Target platform (4th element, G-122).** A kit config entry may name the canonical punkshell platform its runtime is for (`help platforms` lists the names). It decides which `bin/runtime/<platform>/` tier the runtime is read from (macosx-* collapsing to the universal `macosx` folder), whether the runtime file and the built kit carry `.exe`, and whether the pre-deploy process sweep applies. It is a property of the RUNTIME: entries on one runtime line may repeat it but must not disagree (a conflict, or a name that is not an `<os>-<cpu>` platform-dir name, is a fatal config error). Omitted means the build host's default target - which for an msys2/cygwin-runtime tclsh is `win32-x86_64`, not that host's own canon. The live example is the `tclkit-902-Linux64-intel-dyn` entry (target `linux-x86_64`, suffixless `punkshell902` artifact). The schema's eventual home is the G-024 toml conversion; consumers read the parsed model (`::punkboot::lib::mapvfs_*`), never the file format. - **Target platform (4th element, G-122).** A kit config entry may name the canonical punkshell platform its runtime is for (`help platforms` lists the names). It decides which `bin/runtime/<platform>/` tier the runtime is read from (macosx-* collapsing to the universal `macosx` folder), whether the runtime file and the built kit carry `.exe`, and whether the pre-deploy process sweep applies. It is a property of the RUNTIME: entries on one runtime line may repeat it but must not disagree (a conflict, or a name that is not an `<os>-<cpu>` platform-dir name, is a fatal config error). Omitted means the build host's default target - which for an msys2/cygwin-runtime tclsh is `win32-x86_64`, not that host's own canon. The live example is the `tclkit-902-Linux64-intel-dyn` entry (target `linux-x86_64`, suffixless `punkshell902` artifact). The schema's eventual home is the G-024 toml conversion; consumers read the parsed model (`::punkboot::lib::mapvfs_*`), never the file format.
- Runtime executables are placed in `bin/runtime/<platform>/` by the `bin/punk-runtime.cmd` helper, manually, or (G-103 family runtimes) copied from the suite_tcl90 `kit-family` build products under `src/buildsuites/_build/suite_tcl90/out/family/`. - Runtime executables are placed in `bin/runtime/<platform>/` by the `bin/punk-runtime.cmd` helper, manually, or (G-103 family runtimes) copied from the suite_tcl90 `kit-family` build products under `src/buildsuites/_build/suite_tcl90/out/family/`.
- `punkshell.ico` here is the project-DEFAULT kit icon: the bake icon step (G-057) records it in every kit's `<kitname>.resources.toml` sidecar and embeds it into win32-target kits (per-kit override + format + skip semantics: bin/AGENTS.md). It is a derived copy of `src/assets/logo/punk-mark.ico` - its `.assetorigin.toml` sidecar records the derivation; regenerate via the logo pipeline, never edit the `.ico` here. - `punkshell.ico` here is the project-DEFAULT kit icon: the bake icon step (G-057) records it in every kit's `<kitname>.resources.toml` sidecar and embeds it into win32-target kits (per-kit override + format + skip semantics: bin/AGENTS.md). It is a derived copy of `src/assets/logo/punk-mark.ico` - its `.assetorigin.toml` sidecar records the derivation; regenerate via the logo pipeline, never edit the `.ico` here.
- `libpackages.toml` (G-139) declares the punkbin LIB-TIER library artifacts this checkout consumes: `[artifact.<name>]` entries with `name` (immutable `<pkgfolder>-<tcl8|tcl9>-r<N>.zip` artifact filename) and `target` (`allplatforms` or a platform-dir name). Processed by `tclsh src/make.tcl libfetch` into the untracked `bin/packages/<target>/` input tier (sha1-verified against the server's per-target sha1sums.txt; trees materialized under `bin/packages/<target>/tcl<N>/`). Consuming a new revision = editing the declared name here (deliberate, reviewable - mirroring the emission side's deliberate-publish discipline). The file's own header comment is the user-facing spec; keep the two in step.
- The `_build/` subdirectory holds build intermediates; it can be safely deleted. - The `_build/` subdirectory holds build intermediates; it can be safely deleted.
## Work Guidance ## Work Guidance

47
src/runtime/libpackages.toml

@ -0,0 +1,47 @@
# punkbin lib-tier library artifacts this checkout consumes (goal G-139)
#
# Processed by 'tclsh src/make.tcl libfetch': each declared artifact is fetched
# from <origin>/lib/<target>/ into the untracked input tier
# bin/packages/<target>/ (zip + .toml sidecar, sha1-verified against the
# server's per-target sha1sums.txt) and its installed-shape package tree is
# materialized under bin/packages/<target>/tcl<N>/ - self-describing via the
# embedded punkbin-artifact.toml record (G-138). The origin defaults to the
# canonical punkbin raw base; PUNKBIN_URL / -serverurl override it (any
# non-canonical origin requires the explicit -trust-server flag).
#
# Artifact names are punkbin-immutable: <installed folder>-<tcl8|tcl9>-r<N>.zip.
# The generation tag is identity (both generations install identically-named
# folders); consuming a new revision means editing the declared name here -
# a deliberate, reviewable act, mirroring the deliberate-publish discipline on
# the emission side (zig-built dlls are not bit-reproducible).
#
# Keys per [artifact.<name>] entry:
# name - artifact filename in the punkbin lib tier
# target - lib-tier subfolder: 'allplatforms' (platform-neutral, pure-tcl)
# or a canonical platform-dir name (binary packages)
#
# NOTE: the r1 artifacts below are emitted by the suite_tcl90/suite_tcl86
# 'library-artifacts' steps (G-138, achieved 2026-07-30) but PUBLICATION to the
# canonical punkbin origin is a deliberate user step - until it happens,
# libfetch against the canonical origin reports them missing from the server
# sha1sums. A local mirror (file:// origin + -trust-server) or a pre-populated
# bin/packages tier serves in the interim.
#
# Downstream consumption (lib_tcl<N> deployment, kit vfs declarations) is the
# remainder of G-139 - see goals/G-139-punkbin-library-consumption.md.
[artifact.tcllib-tcl9]
name = "tcllib2.0-tcl9-r1.zip"
target = "allplatforms"
[artifact.tcllibc-tcl9]
name = "tcllibc-tcl9-r1.zip"
target = "win32-x86_64"
[artifact.tcllib-tcl8]
name = "tcllib2.0-tcl8-r1.zip"
target = "allplatforms"
[artifact.tcllibc-tcl8]
name = "tcllibc-tcl8-r1.zip"
target = "win32-x86_64"
Loading…
Cancel
Save