Browse Source

G-139 phase 1: make.tcl libfetch + bin/packages input tier (project 0.31.0)

New 'make.tcl libfetch' subcommand (argdoc'd; workflow text + plain help
updated): fetches the punkbin LIB-TIER artifacts declared in the new
src/runtime/libpackages.toml ([artifact.<name>] name/target; the four G-138
r1 artifacts, both generations) from <origin>/lib/<target>/ into the
untracked bin/packages/<target>/ input tier - zip + .toml sidecar, each
sha1-verified against the server's per-target sha1sums.txt (local tier copy
kept, the punk-runtime store idiom) - then materializes each verified zip
via punk::zip::unzip (crc-verified, mtimes restored, accelerator-eligible)
into bin/packages/<target>/tcl<N>/<pkgfolder>/ (generation subdirs: both
generations install identically-named folders). Idempotent: present
artifacts re-verify and skip; a materialized tree is current while its
embedded record names the declared artifact (the G-138 self-description
payoff); -force overrides.

Consent keyed to server trust (the G-123 posture): the canonical punkbin
origin (punk-runtime raw-base convention) gates nothing; any other origin
(-serverurl or env PUNKBIN_URL) refuses before network access without the
explicit -trust-server flag - never an interactive prompt. file:// origins
served natively (mirrors/fixtures); http(s) transport: Tcl http (+tls) when
loadable, else curl, else PowerShell. The dispatch block sits before the
build-command whitelist gate per the standalone-command pattern and exits
itself.

Emission-side amendment (both suites' library_artifacts.tcl): sidecar tomls
now ride the per-tier staging sha1sums.txt, matching the punkbin repo's
build_sha1sums.tcl coverage so staging-derived mirrors serve verifiable
sidecars.

Verified against a file:// mirror assembled from both suites' staging:
trust-gate refusal, consented fetch of all 4 artifacts (sha1-verified),
materialization of all 4 trees, fully idempotent re-run, and a package-load
proof from the tier (md5 2.0.9 + tcllibc accel=1 under the plain family
kit, ifneeded paths in bin/packages). bin/packages is already ignored in
both VCS via the /bin/* rules. Docs: bin/AGENTS.md tier section,
src/runtime/AGENTS.md libpackages.toml bullet. Project version 0.31.0 +
CHANGELOG (new make.tcl subcommand = product surface). Remaining G-139 work
recorded in the goal's Progress section.

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 1 day ago
parent
commit
11200dbf8a
  1. 15
      CHANGELOG.md
  2. 20
      bin/AGENTS.md
  3. 60
      goals/G-139-punkbin-library-consumption.md
  4. 2
      punkproject.toml
  5. 3
      src/buildsuites/suite_tcl86/tools/library_artifacts.tcl
  6. 3
      src/buildsuites/suite_tcl90/tools/library_artifacts.tcl
  7. 384
      src/make.tcl
  8. 1
      src/runtime/AGENTS.md
  9. 47
      src/runtime/libpackages.toml

15
CHANGELOG.md

@ -5,6 +5,21 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to
Entries are newest-first; one bullet per notable change. See the root `AGENTS.md`
"Project Versioning" section for the bump policy.
## [0.31.0] - 2026-07-30
- New `make.tcl libfetch` subcommand (G-139 phase 1): fetches the punkbin
lib-tier library artifacts declared in `src/runtime/libpackages.toml`
(suite-built tcllib/tcllibc zips, both Tcl generations - G-138) into the
untracked `bin/packages/<target>/` input tier with sha1 verification
against the server's per-target sha1sums.txt, and materializes the
installed-shape package trees under `bin/packages/<target>/tcl<N>/`
(self-describing via their embedded punkbin-artifact.toml records).
Consent is keyed to server trust: the canonical punkbin origin needs no
flag; any other origin (`-serverurl` / env `PUNKBIN_URL`) requires the
explicit `-trust-server` flag. `file://` origins serve local mirrors and
fixtures. Idempotent: present artifacts that verify are skipped and
current trees are left untouched (`-force` overrides).
## [0.30.7] - 2026-07-29
- Baked kits pinned archive-relative (G-134): every assembled kit image is

20
bin/AGENTS.md

@ -361,6 +361,26 @@ punkshell platform-dir names defined by the `punk::platform` module and surfaced
aarch64->arm64; the bash payload's local-platform prongs emit canonical names - the
FreeBSD dir was realigned from `freebsd-amd64` to `freebsd-x86_64` accordingly).
### Library package input tier (`bin/packages/`, G-139)
`bin/packages/<target>/` is the untracked input tier for punkbin LIB-TIER library
artifacts (tcllib/tcllibc as immutable generation-tagged zips - shellspy G-138),
mirroring the `bin/runtime/<platform>/` store idiom: per-target folders holding
the sha1-verified `-r<N>` zips, their `.toml` sidecars and a local copy of the
server's `sha1sums.txt`, plus the materialized installed-shape package trees
under `bin/packages/<target>/tcl<N>/<pkgfolder>/` (generation subfolders because
both Tcl generations install identically-named folders). Materialized trees are
self-describing via their embedded `punkbin-artifact.toml` records (the G-138
schema) - `make.tcl libfetch` uses that record to skip re-materializing a
current tree. Populated by `tclsh src/make.tcl libfetch` from the declarations
in `src/runtime/libpackages.toml` (consent keyed to server trust: the canonical
punkbin origin needs no flag, any other origin - `-serverurl` or env
`PUNKBIN_URL` - requires the explicit `-trust-server` flag; `file://` origins
serve local mirrors/fixtures). Everything under `bin/packages/` is
fetch-regenerable and ignored by both VCS (`/bin/*` rules) - never commit from
it; downstream consumption into `lib_tcl<N>/` and kit vfs trees is the
remainder of G-139.
### Interactive verification shells
- Interactive console/repl verification should cover both Tcl generations - behaviour can differ materially (e.g. the Tcl 8.6 windows console channel driver vs the Tcl 9 rewrite). Use a Tcl 8.6-based punk shell (`punksys.exe`) and a current Tcl 9-based punk shell (named for the Tcl release it embeds, e.g. `punk902z.exe` at the time of writing - ask the user which is current rather than assuming). `info patchlevel` in-session confirms the runtime.

60
goals/G-139-punkbin-library-consumption.md

@ -119,3 +119,63 @@ State at drafting worth recording:
- Activation survey 2026-07-30: goals_xref score G-139 re-run at activation
(user-directed, same day as drafting - no goals drafted in the interval);
no overlaps beyond the drafting survey.
## Progress
- 2026-07-30 phase 1 landed: the fetch/materialize input tier and its
make.tcl entrypoint (project 0.31.0).
- `make.tcl libfetch` (argdoc'd subcommand, workflow-text + plain-help
updated): reads `src/runtime/libpackages.toml` declarations
([artifact.<name>] name/target), fetches from <origin>/lib/<target>/
into `bin/packages/<target>/` - zip + .toml sidecar, each sha1-verified
against the server's fetched per-target sha1sums.txt (kept as the local
tier copy, the punk-runtime store idiom) - then materializes each
verified zip via punk::zip::unzip (defaults: crc-verified, mtimes
restored, punkzip-accelerator-eligible) into
`bin/packages/<target>/tcl<N>/<pkgfolder>/` (generation subdirs: both
generations install identically-named folders). Idempotent: present
artifacts re-verify and are skipped; a materialized tree is skipped
while its embedded record's name matches the declared artifact (the
G-138 self-description payoff); -force re-does both.
- Consent (per this goal's acceptance + the G-123 posture): canonical
punkbin origin (the punk-runtime raw-base convention) prompts nothing;
ANY other origin - -serverurl flag or PUNKBIN_URL env - refuses before
network access without the explicit -trust-server flag. Never an
interactive prompt. file:// origins supported natively (local mirrors,
fixtures). Transport for http(s): Tcl http (+tls for https) when
loadable, else curl, else PowerShell.
- Verified 2026-07-30 against a file:// mirror assembled from BOTH
suites' out/library/punkbin staging (the shape publication will
produce): trust-gate refusal without the flag; consented fetch of all
4 declared r1 artifacts with sha1 verification; materialization of all
4 trees; full idempotent re-run (verified-present + tree-current);
package-load proof from the tier (md5 2.0.9 + tcllibc from
bin/packages/*/tcl9 under the plain family kit, accelerator engaged,
ifneeded source path in the tier).
- Emission-side amendment (both suites' library_artifacts.tcl): sidecar
tomls now ride the per-tier staging sha1sums.txt (the punkbin repo's
build_sha1sums.tcl already sums them; the staging mirror now matches),
so sidecars are verifiable from a staging-derived mirror too.
- bin/packages is ignored by both VCS via the existing /bin/* rules (no
ignore edits needed). Docs: bin/AGENTS.md "Library package input tier",
src/runtime/AGENTS.md libpackages.toml bullet, workflow diagram block.
- Session note: the suite_tcl86 staging r1 zips changed bytes when its
installer/critcl Run steps re-ran under this session's different PATH
(zig Run cache keys include the env) - immaterial pre-publication, and
exactly the deliberate-publish revision discipline's point; the tcl9
zips stayed byte-identical across three independent emissions.
- Remaining for acceptance: libs-step materialization of
lib_tcl<N>/allplatforms + lib_tcl<N>/<platform> from the tier with
punkcheck provenance; tcllib/tcllibc vendorlib_vfs.toml (or successor)
declarations with supersedes removing tcllib1.21 (incl. nested tcllibc
0.4 + md5c) from 8.6-family kits; punk86-family tcllib 2.0 upgrade
validated by kit-hosted runtests vs a same-day baseline (differences
dispositioned); vendored src/vendorlib_tcl8|9 tcllib-tree removal in its
own commit after a bake fed from the artifact path reproduces working
kits; a bake on a checkout without those trees shipping kits with
verifiable embedded provenance; the derived-project decision recorded
here (fetch default; in-project suite build per G-104 as the documented
alternative - the stale samplesuite1 download_and_build.config stub to be
replaced/removed when recording it). Also pending: publication of the r1
artifacts to the real punkbin origin (user step) so libfetch's canonical
path goes live end-to-end.

2
punkproject.toml

@ -1,4 +1,4 @@
[project]
name = "punkshell"
version = "0.30.7"
version = "0.31.0"
license = "BSD-2-Clause"

3
src/buildsuites/suite_tcl86/tools/library_artifacts.tcl

@ -383,6 +383,9 @@ foreach {folder tier license testlibs pkgpath} $opt(-packages) {
fconfigure $f -translation lf
puts $f [compose_record $ident [dict create sha1 $sha1 size $size built $built] $tests]
close $f
#sidecars ride sha1sums too (punkbin repo convention: build_sha1sums.tcl sums
#every tracked file) - so a staging mirror serves verifiable sidecars as well
dict lappend sha1lines $tier "[sha1::sha1 -hex -file $mf] *[file tail $mf]"
lappend emitted "$folder -> lib/$tier/$artifact"
note "emitted lib/$tier/$artifact (sha1 $sha1, [expr {$size/1024}] KB, $members members) + [file tail $mf]"
}

3
src/buildsuites/suite_tcl90/tools/library_artifacts.tcl

@ -383,6 +383,9 @@ foreach {folder tier license testlibs pkgpath} $opt(-packages) {
fconfigure $f -translation lf
puts $f [compose_record $ident [dict create sha1 $sha1 size $size built $built] $tests]
close $f
#sidecars ride sha1sums too (punkbin repo convention: build_sha1sums.tcl sums
#every tracked file) - so a staging mirror serves verifiable sidecars as well
dict lappend sha1lines $tier "[sha1::sha1 -hex -file $mf] *[file tail $mf]"
lappend emitted "$folder -> lib/$tier/$artifact"
note "emitted lib/$tier/$artifact (sha1 $sha1, [expr {$size/1024}] KB, $members members) + [file tail $mf]"
}

384
src/make.tcl

@ -187,7 +187,7 @@ namespace eval ::punkboot {
variable foldername [file tail $scriptfolder]
variable pkg_requirements [list]; variable pkg_missing [list];variable pkg_loaded [list]
variable help_flags [list -help --help /? -h]
variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate bootsupport vfscommonupdate projectversion workflow buildsuite tool project vfs]
variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate libfetch bootsupport vfscommonupdate projectversion workflow buildsuite tool project vfs]
}
@ -2135,6 +2135,10 @@ proc ::punkboot::punkboot_gethelp {args} {
append h " - propagate declared vendored platform-library packages into kit vfs lib_tcl<N> trees" \n
append h " - declarations (per-package per-kit, with superseded-version removal) live in src/runtime/vendorlib_vfs.toml" \n
append h " - also runs automatically as a phase of '$scriptname bake' (and therefore of '$scriptname bakehouse')" \n \n
append h " $scriptname libfetch ?-serverurl <url>? ?-trust-server? ?-force?" \n
append h " - fetch declared punkbin lib-tier library artifacts (src/runtime/libpackages.toml) into bin/packages/<target>" \n
append h " - sha1-verified against the server's per-target sha1sums.txt; materializes package trees under bin/packages/<target>/tcl<N>" \n
append h " - canonical origin trusted by default; other servers (incl. PUNKBIN_URL overrides) require -trust-server" \n \n
append h " $scriptname vfscommonupdate" \n
append h " - promotion gate: update the src/vfs/_vfscommon.vfs from compiled src/modules and src/lib etc" \n
append h " - before calling this (followed by '$scriptname bake') - you can test using '<builtexe>(.exe) dev'" \n
@ -2323,6 +2327,26 @@ DIAGRAM 2 - KIT ASSEMBLY DETAIL (the 'make.tcl bake' stage; incl. vfslibs phase)
running shell still holds the old exe [K5]
LIBRARY-ARTIFACT FETCH (G-139; input tier so far - lib_tcl<N>/kit consumption follows)
----------------------------------------------------------------------------
src/runtime/libpackages.toml which punkbin LIB-TIER artifacts this checkout
consumes (declared artifact names + targets)
|
| 'make.tcl libfetch' - sha1-verified download of the declared
| <pkg>-<tcl8|tcl9>-r<N>.zip artifacts + .toml sidecars from
| <origin>/lib/<target>/ (canonical punkbin origin trusted by
| default; -serverurl/PUNKBIN_URL override needs -trust-server;
| file:// origins serve local mirrors/fixtures), then
| materialization of each verified zip's package tree
v
bin/packages/<target>/ verified -r<N> zips + toml sidecars (input tier,
untracked - mirrors bin/runtime/<target>)
bin/packages/<target>/tcl<N>/<pkgfolder>/
materialized installed-shape trees, self-describing
via their embedded punkbin-artifact.toml records
KEY / NOTES
----------------------------------------------------------------------------
[K1] Source modules carry the literal magic version 999999.0a1.0 in the filename.
@ -2527,6 +2551,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
vfslibs "Propagate declared vendored platform-library packages into kit vfs lib_tcl<N> trees"
bin "Install executables from src/bin into <projectdir>/bin, then build kits as for bake"
vendorupdate "Update src/vendormodules based on src/vendormodules/include_modules.config"
libfetch "Fetch declared punkbin lib-tier library artifacts into bin/packages/<target> (sha1-verified) and materialize the package trees"
bootsupport "Promotion gate: update src/bootsupport modules from built project modules"
vfscommonupdate "Promotion gate: replace src/vfs/_vfscommon.vfs modules and libs with the project's built set"
info "Show the name and base folder of the project to be built"
@ -2630,6 +2655,24 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
Also runs automatically as a phase of 'make.tcl bake' (and therefore
of 'make.tcl bakehouse'); this standalone narrowing serves quick
iteration on the declarations."
libfetch
"
Fetch the punkbin LIB-TIER library artifacts declared in
src/runtime/libpackages.toml into the untracked input tier
bin/packages/<target>/ - immutable <pkg>-<tcl8|tcl9>-r<N>.zip
artifacts plus their .toml sidecars, each verified against the
server's per-target sha1sums.txt - then materialize each verified
zip's installed-shape package tree under
bin/packages/<target>/tcl<N>/ (self-describing via the embedded
punkbin-artifact.toml record; G-138/G-139).
Present artifacts that already verify are skipped (-force
re-downloads). The canonical punkbin origin is trusted by
default; -serverurl (or env PUNKBIN_URL) overrides it, and any
NON-canonical origin requires the explicit -trust-server flag
(never an interactive prompt - unattended runs stay unattended).
A file:// origin is supported for local mirrors/fixtures.
Transport for http(s): Tcl http+tls when loadable, else curl,
else PowerShell (windows)."
bin
"
Install executables from src/bin to <projectdir>/bin, then check vfs
@ -2799,6 +2842,30 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
(packages -> test -> bootsupport/vfscommonupdate -> commit -> bake).
Pass -dirty-abort 0 to build anyway with PROVENANCE-WARNINGs."
}
#libfetch fragments (G-139): consent is keyed to SERVER TRUST (the G-123
#posture) - the canonical punkbin origin needs no acknowledgement, any
#other origin needs the explicit flag. Never an interactive prompt.
variable OPT_SERVERURL {
-serverurl -type string -default "" -help\
"Artifact server base url override (default: the canonical punkbin
origin; env PUNKBIN_URL overrides the default, this flag overrides
both). The lib tier is addressed as <serverurl>/lib/<target>/.
file:// urls are supported for local mirrors and fixtures.
A NON-canonical origin additionally requires -trust-server."
}
variable OPT_TRUSTSERVER {
-trust-server -type none -default 0 -help\
"Acknowledge fetching from a NON-canonical artifact server
(consent keyed to server trust - the interim G-123/G-006
mechanism). Without this flag, a non-canonical -serverurl or
PUNKBIN_URL aborts before any network access. Fetches from the
canonical origin never require it."
}
variable OPT_FORCE {
-force -type none -default 0 -help\
"Re-download and re-materialize even when a present artifact
already verifies against the server sha1sums."
}
#which shared option fragments each subcommand declares
#(shell and help have bespoke definitions below: shell passes trailing args
# through to the repl untouched, help takes a subject value)
@ -2812,6 +2879,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
vfslibs {OPT_DIRTYABORT OPT_CONFIRM}
bin {OPT_FORCEKILL OPT_DIRTYABORT OPT_CONFIRM}
vendorupdate {OPT_CONFIRM}
libfetch {OPT_SERVERURL OPT_TRUSTSERVER OPT_FORCE}
bootsupport {OPT_DIRTYABORT OPT_CONFIRM}
vfscommonupdate {OPT_DIRTYABORT OPT_CONFIRM}
info {OPT_CONFIRM}
@ -2829,6 +2897,9 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
OPT_DIRTYABORT "?-dirty-abort?"
OPT_DIRTYABORT_ON "?-dirty-abort 1|0?"
OPT_CONFIRM "?-confirm 0|1?"
OPT_SERVERURL "?-serverurl <url>?"
OPT_TRUSTSERVER "?-trust-server?"
OPT_FORCE "?-force?"
}
#per-subcommand positional values (default: none). bake (and its deprecated
#alias vfs) and bakelist take optional kit names (G-121 selective bake).
@ -2940,7 +3011,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
variable SUBGROUPS {
"build & bake" {bakehouse packages modules libs bake vfslibs bin}
"promotion gates" {bootsupport vfscommonupdate}
"source maintenance" {vendorupdate}
"source maintenance" {vendorupdate libfetch}
"buildsuites" {buildsuite}
"vendored tools" {tool}
"informational" {info check bakelist projectversion workflow help}
@ -2996,6 +3067,9 @@ set help_exitcode 0
set ::punkboot::opt_forcekill 0
set ::punkboot::opt_dirty_abort 0
set ::punkboot::opt_confirm 1
set ::punkboot::opt_serverurl ""
set ::punkboot::opt_trust_server 0
set ::punkboot::opt_force 0
set ::punkboot::opt_kitnames [list] ;#G-121: requested kit names for bake/bakelist (empty = all configured)
if {$::punkboot::punkargs_ok} {
@ -3046,7 +3120,7 @@ if {$::punkboot::punkargs_ok} {
}
set ::punkboot::command $subcommand
set _opts [dict get $argd opts]
foreach {_optname _var} {-k opt_forcekill -dirty-abort opt_dirty_abort -confirm opt_confirm} {
foreach {_optname _var} {-k opt_forcekill -dirty-abort opt_dirty_abort -confirm opt_confirm -serverurl opt_serverurl -trust-server opt_trust_server -force opt_force} {
if {[dict exists $_opts $_optname]} {
set ::punkboot::$_var [dict get $_opts $_optname]
}
@ -3083,6 +3157,12 @@ if {$::punkboot::punkargs_ok} {
switch -- $a {
-k {set ::punkboot::opt_forcekill 1}
-dirty-abort {set ::punkboot::opt_dirty_abort 1}
-trust-server {set ::punkboot::opt_trust_server 1}
-force {set ::punkboot::opt_force 1}
-serverurl {
set ::punkboot::opt_serverurl [lindex $argsleft 0]
set argsleft [lrange $argsleft 1 end]
}
-confirm {
set v [lindex $argsleft 0]
set argsleft [lrange $argsleft 1 end]
@ -5121,6 +5201,303 @@ if {$::punkboot::command eq "bootsupport"} {
if {$::punkboot::command eq "libfetch"} {
#G-139 phase 1: fetch the punkbin LIB-TIER artifacts declared in
#src/runtime/libpackages.toml into the untracked bin/packages/<target> input
#tier (mirroring bin/runtime/<target>) with sha1 verification against the
#server's per-target sha1sums.txt, then materialize each verified zip's
#installed-shape package tree under bin/packages/<target>/tcl<N>/ (the tree
#is self-describing via its embedded punkbin-artifact.toml - the G-138
#record). Consumption into lib_tcl<N>/ and kit vfs trees is the remainder
#of G-139 - this entrypoint owns fetch + verify + materialize only.
proc ::punkboot::libfetch_transfer {url outfile} {
#one url -> local file. file:// is served natively (local mirrors,
#fixtures - no network, no external tools). http(s) transports in
#order: Tcl http (+tls for https) when loadable, curl (ubiquitous,
#including stock windows 10+), powershell Invoke-WebRequest (windows).
#Returns the transport used; errors when all fail.
file mkdir [file dirname $outfile]
if {[string match -nocase "file://*" $url]} {
set path [string range $url 7 end]
if {[regexp {^/+([A-Za-z]:/.*)$} $path -> winpath]} {
set path $winpath ;#file:///C:/x -> C:/x
}
if {![file isfile $path]} {
error "file origin has no such file: $path"
}
file copy -force $path $outfile
return "file-copy"
}
set is_https [string match -nocase "https://*" $url]
set tcl_http_ok 0
if {![catch {package require http}]} {
if {!$is_https} {
set tcl_http_ok 1
} elseif {![catch {package require tls}]} {
catch {http::register https 443 [list ::tls::socket -autoservername true]}
set tcl_http_ok 1
}
}
if {$tcl_http_ok} {
set fd [open $outfile wb]
set ok 0
if {![catch {http::geturl $url -channel $fd -binary 1} tok]} {
if {[http::status $tok] eq "ok" && [http::ncode $tok] == 200} {set ok 1}
http::cleanup $tok
}
close $fd
if {$ok} {return "tcl-http"}
file delete -force $outfile
}
set curl [auto_execok curl]
if {$curl ne ""} {
if {![catch {exec {*}$curl --fail -sSL -o $outfile $url 2>@1} curlerr]} {
return "curl"
}
file delete -force $outfile
}
if {$::tcl_platform(platform) eq "windows"} {
set ps [auto_execok powershell]
if {$ps ne ""} {
set pscmd "\$ProgressPreference='SilentlyContinue'; Invoke-WebRequest -Uri '$url' -OutFile '$outfile' -ErrorAction Stop"
if {![catch {exec {*}$ps -NoProfile -NonInteractive -Command $pscmd 2>@1} pserr]} {
return "powershell"
}
file delete -force $outfile
}
}
set detail ""
if {[info exists curlerr]} {set detail " (curl: [string range $curlerr 0 200])"}
error "all transports failed for $url$detail"
}
set libfetch_config $sourcefolder/runtime/libpackages.toml
#Canonical origin: the punkbin raw-file base (the punk-runtime.cmd
#convention; lib tier addressed as <origin>/lib/<target>/...).
#Server-trust consent (G-139, the G-123 posture): the canonical origin
#never gates; ANY other origin - flag or PUNKBIN_URL env - requires the
#explicit -trust-server flag. Never an interactive prompt: unattended
#runs stay unattended, and refusal happens before any network access.
set libfetch_canonical "https://www.gitea1.intx.com.au/jn/punkbin/raw/branch/master"
set libfetch_origin $libfetch_canonical
if {[info exists ::env(PUNKBIN_URL)] && $::env(PUNKBIN_URL) ne ""} {
set libfetch_origin $::env(PUNKBIN_URL)
}
if {$::punkboot::opt_serverurl ne ""} {
set libfetch_origin $::punkboot::opt_serverurl
}
set libfetch_origin [string trimright $libfetch_origin /]
if {$libfetch_origin ne $libfetch_canonical && !$::punkboot::opt_trust_server} {
puts stderr "$A(BAD)LIBFETCH: origin '$libfetch_origin' is not the canonical punkbin origin$A(RST)"
puts stderr "LIBFETCH: canonical: $libfetch_canonical"
puts stderr "LIBFETCH: fetching from a non-canonical server requires the explicit -trust-server flag (consent keyed to server trust - G-139/G-123). Nothing was fetched."
exit 3
}
if {![file exists $libfetch_config]} {
puts stdout "LIBFETCH: no config at $libfetch_config - nothing declared, nothing to fetch"
} else {
if {[catch {package require tomlish} tomlish_err]} {
puts stderr "$A(BAD)LIBFETCH: cannot process $libfetch_config - tomlish package unavailable ($tomlish_err)$A(RST)"
exit 3
}
if {[catch {package require sha1} sha1_err]} {
puts stderr "$A(BAD)LIBFETCH: sha1 package unavailable ($sha1_err) - cannot verify artifacts$A(RST)"
exit 3
}
if {[catch {package require punk::zip} punkzip_err]} {
puts stderr "$A(BAD)LIBFETCH: punk::zip package unavailable ($punkzip_err) - cannot materialize package trees$A(RST)"
exit 3
}
set fd [open $libfetch_config r]
fconfigure $fd -encoding utf-8
set libfetch_tomldata [read $fd]
close $fd
if {[catch {tomlish::to_dict [tomlish::from_toml $libfetch_tomldata]} libfetch_dict]} {
puts stderr "$A(BAD)LIBFETCH: failed to parse $libfetch_config\n$libfetch_dict$A(RST)"
exit 3
}
if {![llength [info commands ::punkboot::lib::toml_untag]]} {
#tomlish::to_dict values are type-tagged (also defined by the vfslibs
#phase - that block does not run for the libfetch command)
proc ::punkboot::lib::toml_untag {tagged} {
if {[dict get $tagged type] eq "ARRAY"} {
set plain [list]
foreach el [dict get $tagged value] {
lappend plain [::punkboot::lib::toml_untag $el]
}
return $plain
}
return [dict get $tagged value]
}
}
set libfetch_artifacts [dict create]
if {[dict exists $libfetch_dict artifact]} {
set libfetch_artifacts [dict get $libfetch_dict artifact]
}
if {![dict size $libfetch_artifacts]} {
puts stdout "LIBFETCH: no \[artifact.<name>\] entries found in $libfetch_config"
}
puts stdout "LIBFETCH: origin $libfetch_origin ([dict size $libfetch_artifacts] declared artifact(s))"
set libfetch_sums [dict create] ;#target -> dict(filename -> sha1) from the server's sha1sums.txt
set libfetch_failed 0
set libfetch_actions [list]
dict for {aname aentry} $libfetch_artifacts {
foreach reqkey {name target} {
if {![dict exists $aentry $reqkey]} {
puts stderr "$A(BAD)LIBFETCH: entry artifact.$aname in $libfetch_config is missing required key '$reqkey'$A(RST)"
exit 3
}
}
set artifact [punkboot::lib::toml_untag [dict get $aentry name]]
set target [punkboot::lib::toml_untag [dict get $aentry target]]
if {![regexp {^(.+)-(tcl[0-9]+)-r([0-9]+)\.zip$} $artifact -> pkgfolder generation rev]} {
puts stderr "$A(BAD)LIBFETCH: artifact.$aname name '$artifact' does not match <pkgfolder>-tcl<N>-r<N>.zip$A(RST)"
exit 3
}
set tierdir $projectroot/bin/packages/$target
file mkdir $tierdir
#server truth per target: lib/<target>/sha1sums.txt (fetched once per
#run per target; the local copy in the tier mirrors the punk-runtime
#store idiom)
if {![dict exists $libfetch_sums $target]} {
set sumsurl "$libfetch_origin/lib/$target/sha1sums.txt"
set sumslocal $tierdir/sha1sums.txt
if {[catch {::punkboot::libfetch_transfer $sumsurl $sumslocal} terr]} {
puts stderr "$A(BAD)LIBFETCH: cannot fetch $sumsurl - $terr$A(RST)"
exit 3
}
set sfd [open $sumslocal r]
set sumsdata [read $sfd]
close $sfd
set sums [dict create]
foreach ln [split $sumsdata \n] {
set ln [string trim $ln]
if {$ln eq ""} {continue}
set sp [string first " " $ln]
if {$sp < 1} {continue}
set h [string range $ln 0 $sp-1]
set tag [string range $ln $sp+1 end]
dict set sums [string range $tag 1 end] [string tolower $h]
}
dict set libfetch_sums $target $sums
}
set sums [dict get $libfetch_sums $target]
if {![dict exists $sums $artifact]} {
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: '$artifact' is not listed in the server's lib/$target/sha1sums.txt$A(RST)"
set libfetch_failed 1
continue
}
set want_sha1 [dict get $sums $artifact]
set ziplocal $tierdir/$artifact
set have_ok 0
if {[file isfile $ziplocal] && !$::punkboot::opt_force} {
if {[string tolower [sha1::sha1 -hex -file $ziplocal]] eq $want_sha1} {
set have_ok 1
}
}
if {!$have_ok} {
set zipurl "$libfetch_origin/lib/$target/$artifact"
set partfile $ziplocal.part
if {[catch {::punkboot::libfetch_transfer $zipurl $partfile} terr]} {
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: fetch failed - $terr$A(RST)"
set libfetch_failed 1
continue
}
set got_sha1 [string tolower [sha1::sha1 -hex -file $partfile]]
if {$got_sha1 ne $want_sha1} {
file delete -force $partfile
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: sha1 MISMATCH for $artifact (got $got_sha1, server sha1sums says $want_sha1) - deleted$A(RST)"
set libfetch_failed 1
continue
}
file rename -force $partfile $ziplocal
lappend libfetch_actions "fetched lib/$target/$artifact"
} else {
lappend libfetch_actions "verified-present lib/$target/$artifact"
}
#beside-toml sidecar (also sha1sums-listed by emission)
set sidecar "[string range $artifact 0 end-4].toml"
set sidecarlocal $tierdir/$sidecar
set sidecar_ok 0
if {[dict exists $sums $sidecar]} {
set want_side [dict get $sums $sidecar]
if {[file isfile $sidecarlocal] && !$::punkboot::opt_force} {
if {[string tolower [sha1::sha1 -hex -file $sidecarlocal]] eq $want_side} {
set sidecar_ok 1
}
}
if {!$sidecar_ok} {
if {[catch {::punkboot::libfetch_transfer "$libfetch_origin/lib/$target/$sidecar" $sidecarlocal.part} terr]} {
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: sidecar fetch failed - $terr$A(RST)"
set libfetch_failed 1
continue
}
if {[string tolower [sha1::sha1 -hex -file $sidecarlocal.part]] ne $want_side} {
file delete -force $sidecarlocal.part
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: sidecar sha1 MISMATCH for $sidecar - deleted$A(RST)"
set libfetch_failed 1
continue
}
file rename -force $sidecarlocal.part $sidecarlocal
}
} else {
puts stderr "LIBFETCH: note - $sidecar not listed in server sha1sums (artifact accepted; sidecar skipped)"
}
#materialize: installed-shape tree under <target>/tcl<N>/<pkgfolder>
#(generation subdir because both generations install identically-named
#folders). Fresh tree per materialization; -mtime restores member
#mtimes so the tree mirrors what was zipped.
set gendir $tierdir/$generation
file mkdir $gendir
set treedir $gendir/$pkgfolder
if {[file isdirectory $treedir] && !$::punkboot::opt_force} {
#re-materialize only when the tree's embedded record no longer
#matches this artifact name (self-describing tier - G-138)
set embrec $treedir/punkbin-artifact.toml
set tree_current 0
if {[file isfile $embrec]} {
set efd [open $embrec r]
set edata [read $efd]
close $efd
if {[regexp -line "^name = \"$artifact\"$" $edata]} {
set tree_current 1
}
}
if {$tree_current} {
lappend libfetch_actions "tree-current $target/$generation/$pkgfolder"
continue
}
}
file delete -force $treedir
#defaults: whole archive, -overwrite 1 -mtime 1 (member mtimes restored)
#-verify 1 (per-member crc32) - and accelerator-eligible (punkzip exe)
if {[catch {punk::zip::unzip $ziplocal $gendir} unzip_err]} {
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: materialization failed - $unzip_err$A(RST)"
file delete -force $treedir
set libfetch_failed 1
continue
}
if {![file isfile $treedir/pkgIndex.tcl] || ![file isfile $treedir/punkbin-artifact.toml]} {
puts stderr "$A(BAD)LIBFETCH: artifact.$aname: materialized tree at $treedir lacks pkgIndex.tcl and/or punkbin-artifact.toml$A(RST)"
set libfetch_failed 1
continue
}
lappend libfetch_actions "materialized $target/$generation/$pkgfolder"
}
foreach act $libfetch_actions {
puts stdout "LIBFETCH: $act"
}
if {$libfetch_failed} {
puts stderr "$A(BAD)LIBFETCH: FAILED - one or more declared artifacts could not be fetched/verified/materialized (see above)$A(RST)"
exit 3
}
puts stdout "LIBFETCH: done ([llength $libfetch_actions] action(s)) -> $projectroot/bin/packages"
}
flush stdout
::exit 0
}
if {$::punkboot::command ni {bakehouse packages modules libs bake vfslibs bin}} {
puts stderr "Command $::punkboot::command not implemented - aborting."
flush stderr
@ -5341,6 +5718,7 @@ if {$::punkboot::command in {bakehouse bake vfslibs}} {
}
}
if {$::punkboot::command in {bakehouse packages libs}} {
########################################################
lappend projectlibfolders lib

1
src/runtime/AGENTS.md

@ -17,6 +17,7 @@ Houses the `mapvfs.config` that maps VFS payloads to platform runtimes, plus the
- **Target platform (4th element, G-122).** A kit config entry may name the canonical punkshell platform its runtime is for (`help platforms` lists the names). It decides which `bin/runtime/<platform>/` tier the runtime is read from (macosx-* collapsing to the universal `macosx` folder), whether the runtime file and the built kit carry `.exe`, and whether the pre-deploy process sweep applies. It is a property of the RUNTIME: entries on one runtime line may repeat it but must not disagree (a conflict, or a name that is not an `<os>-<cpu>` platform-dir name, is a fatal config error). Omitted means the build host's default target - which for an msys2/cygwin-runtime tclsh is `win32-x86_64`, not that host's own canon. The live example is the `tclkit-902-Linux64-intel-dyn` entry (target `linux-x86_64`, suffixless `punkshell902` artifact). The schema's eventual home is the G-024 toml conversion; consumers read the parsed model (`::punkboot::lib::mapvfs_*`), never the file format.
- Runtime executables are placed in `bin/runtime/<platform>/` by the `bin/punk-runtime.cmd` helper, manually, or (G-103 family runtimes) copied from the suite_tcl90 `kit-family` build products under `src/buildsuites/_build/suite_tcl90/out/family/`.
- `punkshell.ico` here is the project-DEFAULT kit icon: the bake icon step (G-057) records it in every kit's `<kitname>.resources.toml` sidecar and embeds it into win32-target kits (per-kit override + format + skip semantics: bin/AGENTS.md). It is a derived copy of `src/assets/logo/punk-mark.ico` - its `.assetorigin.toml` sidecar records the derivation; regenerate via the logo pipeline, never edit the `.ico` here.
- `libpackages.toml` (G-139) declares the punkbin LIB-TIER library artifacts this checkout consumes: `[artifact.<name>]` entries with `name` (immutable `<pkgfolder>-<tcl8|tcl9>-r<N>.zip` artifact filename) and `target` (`allplatforms` or a platform-dir name). Processed by `tclsh src/make.tcl libfetch` into the untracked `bin/packages/<target>/` input tier (sha1-verified against the server's per-target sha1sums.txt; trees materialized under `bin/packages/<target>/tcl<N>/`). Consuming a new revision = editing the declared name here (deliberate, reviewable - mirroring the emission side's deliberate-publish discipline). The file's own header comment is the user-facing spec; keep the two in step.
- The `_build/` subdirectory holds build intermediates; it can be safely deleted.
## Work Guidance

47
src/runtime/libpackages.toml

@ -0,0 +1,47 @@
# punkbin lib-tier library artifacts this checkout consumes (goal G-139)
#
# Processed by 'tclsh src/make.tcl libfetch': each declared artifact is fetched
# from <origin>/lib/<target>/ into the untracked input tier
# bin/packages/<target>/ (zip + .toml sidecar, sha1-verified against the
# server's per-target sha1sums.txt) and its installed-shape package tree is
# materialized under bin/packages/<target>/tcl<N>/ - self-describing via the
# embedded punkbin-artifact.toml record (G-138). The origin defaults to the
# canonical punkbin raw base; PUNKBIN_URL / -serverurl override it (any
# non-canonical origin requires the explicit -trust-server flag).
#
# Artifact names are punkbin-immutable: <installed folder>-<tcl8|tcl9>-r<N>.zip.
# The generation tag is identity (both generations install identically-named
# folders); consuming a new revision means editing the declared name here -
# a deliberate, reviewable act, mirroring the deliberate-publish discipline on
# the emission side (zig-built dlls are not bit-reproducible).
#
# Keys per [artifact.<name>] entry:
# name - artifact filename in the punkbin lib tier
# target - lib-tier subfolder: 'allplatforms' (platform-neutral, pure-tcl)
# or a canonical platform-dir name (binary packages)
#
# NOTE: the r1 artifacts below are emitted by the suite_tcl90/suite_tcl86
# 'library-artifacts' steps (G-138, achieved 2026-07-30) but PUBLICATION to the
# canonical punkbin origin is a deliberate user step - until it happens,
# libfetch against the canonical origin reports them missing from the server
# sha1sums. A local mirror (file:// origin + -trust-server) or a pre-populated
# bin/packages tier serves in the interim.
#
# Downstream consumption (lib_tcl<N> deployment, kit vfs declarations) is the
# remainder of G-139 - see goals/G-139-punkbin-library-consumption.md.
[artifact.tcllib-tcl9]
name = "tcllib2.0-tcl9-r1.zip"
target = "allplatforms"
[artifact.tcllibc-tcl9]
name = "tcllibc-tcl9-r1.zip"
target = "win32-x86_64"
[artifact.tcllib-tcl8]
name = "tcllib2.0-tcl8-r1.zip"
target = "allplatforms"
[artifact.tcllibc-tcl8]
name = "tcllibc-tcl8-r1.zip"
target = "win32-x86_64"
Loading…
Cancel
Save