punkshell/goals/archive/G-138-punkbin-library-artifact-tier.md

G-138 Punkbin library artifact tier: suite-built tcllib/tcllibc published as versioned package zips

Status: achieved 2026-07-30 Scope: src/buildsuites/suite_tcl90/ (library-artifact emission step + tools - beside family_artifacts.tcl, schema emission coordinated with G-123); src/buildsuites/suite_tcl86/ (same emission for the tcl8-generation build); punkbin artifact repo (external, c:/repo/jn/punkbin - new lib/ tier: lib/allplatforms + lib/, layout docs, sha1sums, minisign posture); src/tools/punkzip (as consumed - deterministic zip assembly) Goal: the library packages the buildsuites already build and verify - tcllib (pure-Tcl, platform-neutral) and tcllibc (per-platform critcl accelerators built by critcl driving zig cc), both Tcl generations - are published as immutable <package>-r<N>.zip artifacts in a distinct library tier of the punkbin artifact repository (platform-neutral packages under lib/allplatforms, per-platform binary packages under lib/), each zip carrying the package folder in installed pkgIndex.tcl shape plus an embedded artifact record (schema class "library", the G-123 schema-v2 lineage) with checkout/toolchain provenance, so library binaries get the same punkbin integrity story as runtimes (immutable -r names, toml sidecars, sha1sums, minisign posture) and the repo can stop carrying hand-dropped built copies (the G-004 direction; consumption is G-139). Acceptance: an emission step produces punkbin-layout library artifacts from verified suite builds - suite_tcl90 demonstrated end-to-end (a platform-neutral tcllib zip under lib/allplatforms and a win32-x86_64 tcllibc zip under lib/win32-x86_64), with suite_tcl86's tcl8-generation emission demonstrated or recorded here as an explicit follow-through; zips are assembled with deterministic member ordering (punkzip) so identical input trees reproduce identical zips, and the zig non-bit-reproducibility caveat is recorded with the publish discipline it forces (revisions bump only on deliberate publish, never per rebuild); each zip contains the package folder in installed shape with an embedded punkbin-artifact.toml (class library; provenance covering the source checkout uuids, critcl ref, zig version, optimize mode and driving-shell patchlevel); beside-artifact toml sidecars and sha1sums.txt entries are emitted in the same step; publication to the real punkbin repo remains a deliberate user step and the punkbin layout docs record the lib tier; the md5c package question is dispositioned (published as its own zip or dropped, with the tcllibc-provides-the-md5-accelerator rationale recorded either way).

Context

Provenance investigation 2026-07-30: the vendored tcllib trees (src/vendorlib_tcl9/win32-x86_64/tcllib2.0, src/vendorlib_tcl8/win32-x86_64/tcllib1.21) turned out to be hand-dropped copies whose only machine-readable provenance is critcl's teapot.txt - the tcl9 set a 2024-10-02 local critcl 3.3.1 build, the tcl8 set a 2023-09-05 critcl 3.2 build of tcllibc 0.4 against tcllib 1.21. Meanwhile both buildsuites build tcllib 2.0 + tcllibc with zig in their DEFAULT step lists (build905.zig tcllib/tcllibc steps; build86.zig equivalents, G-100 achieved 2026-07-26 with runtests validation), with full provenance available at build time (manifestUuid, critcl 3.3.1 pin, zig version). The build capability exists; only the delivery channel is missing.

Direction (user, 2026-07-30): headed toward G-004 (no committed binaries), so vendoring suite output back into the repo is rejected. The long-game idea - binary-carrying .tm files as the universal punkbin unit - depends on the pkgIndex-to-.tm conversion being feasible for all libs (G-066) plus a distribution-package grouping unit that does not yet exist; rather than block on that, an interim tier of <package>.zip artifacts in a distinct folder of the artifact repository is the chosen carrier. Decisions recorded at drafting: lib/allplatforms + lib/ layout (fixing, in passing, the current misfiling of pure-Tcl tcllib under a platform dir); consumption materializes via an untracked bin/packages/ tier (G-139).

Approach

• Extend, don't fork: the -r immutable-name + toml sidecar + sha1sums pattern is the family_artifacts.tcl machinery (G-103/G-117, achieved); the schema evolution rides G-123's schema v2 (class/builder/source_url fields) - this goal adds class = "library" emission, not a parallel schema.
• Tier layout: punkbin lib/allplatforms/-r.zip for platform-neutral packages (pure-Tcl tcllib); lib//-r.zip for binary packages (tcllibc per platform). Package name/version from the installed folder name (tcllib2.0 -> tcllib2.0-r1.zip).
• Zip assembly with the vendored punkzip (G-126): deterministic sorted member order, so re-emission over unchanged content is byte-stable; the dlls inside are NOT bit-reproducible across zig rebuilds (recorded suite finding), hence the deliberate-publish revision discipline.
• Embedded record inside the zip at package root (punkbin-artifact.toml beside pkgIndex.tcl - inert to package loading): a materialized lib tree becomes self-describing exactly where today's vendored trees are silent. Finished-zip facts (sha1, size) stay sidecar-only per the G-117 embed-then-hash rule.
• Generation split: the same package name exists per generation (tcl9 tcllib2.0 vs a tcl8-generation build); the artifact record carries the driving-shell patchlevel and the sidecar/naming keeps the generations distinct - exact naming settled in the work and recorded in the punkbin layout docs.
• The mechanism is package-generic by construction (any installed-shape package folder + record can ride it - thread, tclvfs, future critcl packages); tcllib/tcllibc are this goal's committed set.

Alternatives considered

• Vendor suite output back into src/vendorlib (punkcheck vendor-back step) - rejected: opposite of the G-004 direction, and zig non-reproducibility would churn committed dll bytes on every re-vendor.
• Binary-carrying .tm modules as the artifact unit - deferred, not superseded: needs G-066's pkgIndex-to-.tm conversion across multi-file packages plus a grouping/distribution unit; the zip tier is the interim carrier and both consume the same suite output, so the .tm route stays open (G-066/G-067).
• Publishing the whole lib/ prefix as one zip - rejected: per-package zips give independent versioning/revision cadence and per-package provenance records.

Notes

• Related: G-139 - the consumption half (fetch/materialize tier, kit-vfs declarations, retirement of the vendored trees).
• Related: G-123 - runtime-tier sibling; shares the punkbin layout docs, schema-v2 provenance direction and the server-trust consent posture. Library emission lands beside its family_artifacts.tcl schema-v2 work, not as a second mechanism.
• Related: G-004 - this tier is what lets built library binaries leave the tracked tree.
• Related: G-126 (achieved) - punkzip is the deterministic zip assembler.
• Related: G-066/G-067 - the .tm long game this tier is interim to.
• Anchors (archived): G-117 (embedded artifact record schema v1 and the embed-then-hash sidecar rule), G-103 (the -r artifact emission pattern), G-100 (the 8.6 tcllib/tcllibc build this publishes), G-102 (critcl -target pin finding; the zig-cc critcl_zig.config - goals/archive/G-102-buildsuite-zig-bootstrap.md).
• Overlap survey 2026-07-30 (drafting): goals_xref paths src/buildsuites src/vendorlib_tcl8|9 src/runtime/vendorlib_vfs.toml bin/ src/make.tcl - strongest live intersections G-123 (punkbin layout + schema), G-115/G-127 (vendorlib_vfs.toml as a surface), G-004/G-006 (posture), all carried as Related here or in G-139; no live goal owns src/vendorlib_tcl8|9.
• Activation survey 2026-07-30: goals_xref score G-138 re-run at activation (user-directed, same day as drafting - no goals drafted in the interval); no overlaps beyond the drafting survey. Remaining unlinked pairs (G-130/G-101/G-105/G-116 et al) share only the buildsuites tree, no contract-level relationship.
• md5c DISPOSITION (2026-07-30): NOT published as its own zip. tcllib's md5 accelerator resolution prefers tcllibc (tcllib modules/md5/md5x.tcl KnownImplementations probe: package require tcllibc FIRST, md5c the fallback), tcllibc bundles the md5 accelerator (every suite build's pkg_smoke -accel md5 tcllibc asserts ::md5::accel(critcl); the materialized-tier load tests below show accel=1 with no md5c present), so a standalone md5c adds nothing the published tcllibc does not provide. The vendored trees' nested md5c copies retire with those trees under G-139.
• Naming decision (settled in the work, 2026-07-30): artifact names carry the Tcl GENERATION tag - <installed folder>-<tcl8|tcl9>-r<N>.zip (tcllib2.0-tcl9-r1.zip, tcllibc-tcl8-r1.zip). Both generations install identically-named folders (tcllib2.0, tcllibc), and the tcl8/tcl9 builds are genuinely different bytes (verified: same member count, different sha1s), so the tag is identity, not decoration. Recorded in the punkbin layout docs.

Progress

• 2026-07-30 emission landed, BOTH generations demonstrated end-to-end:
  • tools/library_artifacts.tcl (suite_tcl90 + byte-identical suite_tcl86 copy): mtime-synced staging copy, embedded schema-2 class=library record (embed-then-hash; record mtime pinned to the package's pkgIndex.tcl), punkzip assembly, porcelain re-verification of the embedded member, sidecar + per-tier sha1sums emission. Records carry [provenance] class="suite-built", tcllib/critcl checkout uuids, critcl version from the package's own teapot.txt, driving_tcl_patchlevel.
  • build905.zig/build86.zig 'library-artifacts' steps (in the default step lists, suite.tcl + bootstrap parity): -Dlibrev revision option (default 1 - nothing published yet), punkzip compiled host-native from the staged upstream checkout (sources.config git record + build.zig.zon pin, commit 0882f0373e = v2.3.1, generation-neutral). suite_tcl90 runs the emission under the plain family kit (proof-doubling); suite_tcl86 under the installed static shell.
  • suite_tcl90 demo: tcllib2.0-tcl9-r1.zip (lib/allplatforms, 871 members, sha1 aabda9db...) + tcllibc-tcl9-r1.zip (lib/win32-x86_64, sha1 f9300c02...); dependency chain re-verified tcllib/tcllibc smokes + plain family check on every emission.
  • suite_tcl86 demo: tcllib2.0-tcl8-r1.zip + tcllibc-tcl8-r1.zip (same layout, sha1s 5f385fab.../40bf00da...).
  • Determinism verified BOTH suites: immediate re-emission reproduced byte-identical zips (identical sha1s across independent runs). punkzip field study: zip members carry file AND dir mtimes - hence the tool's mtime-sync + record-mtime pinning design.
  • Materialized-load verification: zips extracted with punkzip, then md5/sha1/tcllibc package-required FROM the extracted trees (auto_path pinned to the tier; attached/machine copies excluded) under the plain family kit (tcl9) and the installed static 8.6 shell (tcl8) - all loads from the tier confirmed via package ifneeded paths, critcl accelerator engaged (accel=1) both generations. 8.6 test note: the first run tripped the recorded tcl86 tm-path shadowing trap (machine module tree via default tm paths) - hermetic re-run (tm paths cleared) proved the tier.
  • zig-0.16 'failed command'-on-stderr-success cosmetic trap reconfirmed on critcl re-runs (Run cache keys include the inherited PATH, so a different shell session re-runs critcl; the step succeeds, zig still prints the command under a 'failed command:' banner when stderr had warnings).
• 2026-07-30 punkbin layout docs landed (the last acceptance item): punkbin AGENTS.md gains the lib-tier layout contract line + a "Library artifacts - lib tier (schema v2)" section (naming, embedded-record shape and the v2-superset/ordering caveats, determinism + deliberate-publish discipline, md5c disposition, publication flow); README.md mentions the tier; build_sha1sums.tcl processes lib/ subfolders (per-target sha1sums, lib never enters platforms.txt) - verified against the repo (no lib/ yet: clean no-op; existing artifacts SAME). Committed in the punkbin repo (git only there - not pushed; pushing is the maintainer's call).
• ACHIEVED FLIP 2026-07-30. Acceptance walk: emission step from verified builds (dependency chain re-runs the smokes + family check per emission) - BOTH suites demonstrated, exceeding the demonstrate-90/record-86 minimum; deterministic punkzip assembly with byte-identical re-emission verified in both; non-reproducibility caveat + deliberate-publish revision discipline recorded (recipes, suite READMEs, punkbin AGENTS.md); embedded class=library records carry source checkout uuids, critcl ref + version, zig version, optimize mode, driving-shell patchlevel (inspected in the emitted sidecars); sidecars + per-tier sha1sums emitted in the same step; publication remains a deliberate user step (nothing was published; artifacts sit under each suite's out/library/punkbin/lib, revisions unbumped at default 1); punkbin layout docs record the tier; md5c dispositioned (dropped, rationale above). Honesty note: sidecar [tests] lines are the suite's recorded G-107 evidence for the source/toolchain combination (the established runtime-tier semantic), not per-byte proofs of a given rebuild's dll.