From 9065232e82657c0f12b304ba4500b47bbb97ffb2 Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Thu, 30 Jul 2026 17:02:54 +1000 Subject: [PATCH] G-138 + G-139 drafted (user-approved): punkbin library artifact tier and its consumption Motivated by the 2026-07-30 provenance investigation: the vendored tcllib trees are hand-dropped 2023/2024 local builds whose only machine-readable provenance is critcl's teapot.txt, while both buildsuites already build tcllib 2.0 + tcllibc (zig cc) in their default step lists with full provenance available at build time. G-138: publish suite-built tcllib/tcllibc as immutable -r .zip artifacts in a new punkbin lib/ tier - lib/allplatforms for platform-neutral (user decision 2026-07-30), lib/ for binary packages - deterministic punkzip assembly, embedded class=library artifact record (G-123 schema-v2 lineage), sidecars + sha1sums, deliberate-publish revision discipline (zig non-reproducibility), md5c disposition required. G-139: consume the tier - consent-gated make.tcl fetch into an untracked bin/packages/ input tier mirroring bin/runtime/ (user decision 2026-07-30), punkcheck-installed lib_tcl materialization, tcllib/tcllibc brought under vendorlib_vfs.toml declarations with supersedes removing tcllib1.21/tcllibc0.4 from 8.6-family kits (kit-hosted runtests validation), vendored-tree retirement sequenced after a proven artifact-path bake (G-004 payoff), derived-project story: fetch default / in-project suite build alternative. Interim-carrier framing recorded: the zip tier does not supersede the G-066/G-067 .tm direction. Overlap surveys run (goals_xref); strongest live relation G-123 (runtime-tier sibling) carried in both. Also recorded in G-139 Context: lib_tcl8|9/test_old orphans moved to TEMP_REFERENCE/retired_lib_tcl_test_old 2026-07-30 (were untracked). goals_lint clean. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- GOALS.md | 8 ++ goals/G-138-punkbin-library-artifact-tier.md | 92 +++++++++++++++++ goals/G-139-punkbin-library-consumption.md | 103 +++++++++++++++++++ 3 files changed, 203 insertions(+) create mode 100644 goals/G-138-punkbin-library-artifact-tier.md create mode 100644 goals/G-139-punkbin-library-consumption.md diff --git a/GOALS.md b/GOALS.md index c14903fa..bdd05084 100644 --- a/GOALS.md +++ b/GOALS.md @@ -406,3 +406,11 @@ Detail: goals/G-136-generated-project-icon-seeding.md ### G-137 [proposed] Kit version resources: RT_VERSION stamping from punkproject.toml via punkres Scope: src/tools/punkres/ (VS_VERSIONINFO construction, set-version + field-level read-back - upstream at c:/repo/jn/zig/punkres); src/make.tcl (version stamping at the G-057/G-128 kit-icon seam, punkres arm); bin/*.resources.toml sidecar format + bin/AGENTS.md (version fields and status vocabulary); punkproject.toml (field source - configurable overrides); src/tests/ (characterization) Detail: goals/G-137-kit-version-resources.md + +### G-138 [proposed] Punkbin library artifact tier: suite-built tcllib/tcllibc published as versioned package zips +Scope: src/buildsuites/suite_tcl90/ (library-artifact emission step + tools - beside family_artifacts.tcl, schema emission coordinated with G-123); src/buildsuites/suite_tcl86/ (same emission for the tcl8-generation build); punkbin artifact repo (external, c:/repo/jn/punkbin - new lib/ tier: lib/allplatforms + lib/, layout docs, sha1sums, minisign posture); src/tools/punkzip (as consumed - deterministic zip assembly) +Detail: goals/G-138-punkbin-library-artifact-tier.md + +### G-139 [proposed] Consume punkbin library artifacts: fetch/materialize tier replacing vendored tcllib/tcllibc +Scope: src/make.tcl (consent-gated library fetch entrypoint; libs step consuming the packages tier; vfslibs as touched); bin/packages// (new untracked materialization tier mirroring bin/runtime//); src/runtime/vendorlib_vfs.toml (tcllib/tcllibc kit-vfs declarations with supersedes removal of the legacy versions); src/vendorlib_tcl8/win32-x86_64/tcllib1.21 + src/vendorlib_tcl9/win32-x86_64/tcllib2.0 (retirement targets); src/vfs/*.vfs lib_tcl trees (kit payload switchover incl. the punk86-family tcllib 2.0 upgrade); lib_tcl8/ + lib_tcl9/ (deployed-tree materialization targets); user-config consent (server-trust pattern shared with G-123/G-006) +Detail: goals/G-139-punkbin-library-consumption.md diff --git a/goals/G-138-punkbin-library-artifact-tier.md b/goals/G-138-punkbin-library-artifact-tier.md new file mode 100644 index 00000000..ea2c8e49 --- /dev/null +++ b/goals/G-138-punkbin-library-artifact-tier.md @@ -0,0 +1,92 @@ +# G-138 Punkbin library artifact tier: suite-built tcllib/tcllibc published as versioned package zips + +Status: proposed +Scope: src/buildsuites/suite_tcl90/ (library-artifact emission step + tools - beside family_artifacts.tcl, schema emission coordinated with G-123); src/buildsuites/suite_tcl86/ (same emission for the tcl8-generation build); punkbin artifact repo (external, c:/repo/jn/punkbin - new lib/ tier: lib/allplatforms + lib/, layout docs, sha1sums, minisign posture); src/tools/punkzip (as consumed - deterministic zip assembly) +Goal: the library packages the buildsuites already build and verify - tcllib (pure-Tcl, platform-neutral) and tcllibc (per-platform critcl accelerators built by critcl driving zig cc), both Tcl generations - are published as immutable `-r.zip` artifacts in a distinct library tier of the punkbin artifact repository (platform-neutral packages under lib/allplatforms, per-platform binary packages under lib/), each zip carrying the package folder in installed pkgIndex.tcl shape plus an embedded artifact record (schema class "library", the G-123 schema-v2 lineage) with checkout/toolchain provenance, so library binaries get the same punkbin integrity story as runtimes (immutable -r names, toml sidecars, sha1sums, minisign posture) and the repo can stop carrying hand-dropped built copies (the G-004 direction; consumption is G-139). +Acceptance: an emission step produces punkbin-layout library artifacts from verified suite builds - suite_tcl90 demonstrated end-to-end (a platform-neutral tcllib zip under lib/allplatforms and a win32-x86_64 tcllibc zip under lib/win32-x86_64), with suite_tcl86's tcl8-generation emission demonstrated or recorded here as an explicit follow-through; zips are assembled with deterministic member ordering (punkzip) so identical input trees reproduce identical zips, and the zig non-bit-reproducibility caveat is recorded with the publish discipline it forces (revisions bump only on deliberate publish, never per rebuild); each zip contains the package folder in installed shape with an embedded punkbin-artifact.toml (class library; provenance covering the source checkout uuids, critcl ref, zig version, optimize mode and driving-shell patchlevel); beside-artifact toml sidecars and sha1sums.txt entries are emitted in the same step; publication to the real punkbin repo remains a deliberate user step and the punkbin layout docs record the lib tier; the md5c package question is dispositioned (published as its own zip or dropped, with the tcllibc-provides-the-md5-accelerator rationale recorded either way). + +## Context + +Provenance investigation 2026-07-30: the vendored tcllib trees +(src/vendorlib_tcl9/win32-x86_64/tcllib2.0, src/vendorlib_tcl8/win32-x86_64/tcllib1.21) +turned out to be hand-dropped copies whose only machine-readable provenance is +critcl's teapot.txt - the tcl9 set a 2024-10-02 local critcl 3.3.1 build, the +tcl8 set a 2023-09-05 critcl 3.2 build of tcllibc 0.4 against tcllib 1.21. +Meanwhile both buildsuites build tcllib 2.0 + tcllibc with zig in their DEFAULT +step lists (build905.zig tcllib/tcllibc steps; build86.zig equivalents, G-100 +achieved 2026-07-26 with runtests validation), with full provenance available +at build time (manifestUuid, critcl 3.3.1 pin, zig version). The build +capability exists; only the delivery channel is missing. + +Direction (user, 2026-07-30): headed toward G-004 (no committed binaries), so +vendoring suite output back into the repo is rejected. The long-game idea - +binary-carrying .tm files as the universal punkbin unit - depends on the +pkgIndex-to-.tm conversion being feasible for all libs (G-066) plus a +distribution-package grouping unit that does not yet exist; rather than block +on that, an interim tier of `.zip` artifacts in a distinct folder of +the artifact repository is the chosen carrier. Decisions recorded at drafting: +lib/allplatforms + lib/ layout (fixing, in passing, the current +misfiling of pure-Tcl tcllib under a platform dir); consumption materializes +via an untracked bin/packages/ tier (G-139). + +## Approach + +- Extend, don't fork: the -r immutable-name + toml sidecar + sha1sums + pattern is the family_artifacts.tcl machinery (G-103/G-117, achieved); the + schema evolution rides G-123's schema v2 (class/builder/source_url fields) - + this goal adds class = "library" emission, not a parallel schema. +- Tier layout: punkbin lib/allplatforms/-r.zip for platform-neutral + packages (pure-Tcl tcllib); lib//-r.zip for binary + packages (tcllibc per platform). Package name/version from the installed + folder name (tcllib2.0 -> tcllib2.0-r1.zip). +- Zip assembly with the vendored punkzip (G-126): deterministic sorted member + order, so re-emission over unchanged content is byte-stable; the dlls inside + are NOT bit-reproducible across zig rebuilds (recorded suite finding), hence + the deliberate-publish revision discipline. +- Embedded record inside the zip at package root (punkbin-artifact.toml beside + pkgIndex.tcl - inert to package loading): a materialized lib tree becomes + self-describing exactly where today's vendored trees are silent. Finished-zip + facts (sha1, size) stay sidecar-only per the G-117 embed-then-hash rule. +- Generation split: the same package name exists per generation (tcl9 tcllib2.0 + vs a tcl8-generation build); the artifact record carries the driving-shell + patchlevel and the sidecar/naming keeps the generations distinct - exact + naming settled in the work and recorded in the punkbin layout docs. +- The mechanism is package-generic by construction (any installed-shape + package folder + record can ride it - thread, tclvfs, future critcl + packages); tcllib/tcllibc are this goal's committed set. + +## Alternatives considered + +- Vendor suite output back into src/vendorlib (punkcheck vendor-back step) - + rejected: opposite of the G-004 direction, and zig non-reproducibility would + churn committed dll bytes on every re-vendor. +- Binary-carrying .tm modules as the artifact unit - deferred, not superseded: + needs G-066's pkgIndex-to-.tm conversion across multi-file packages plus a + grouping/distribution unit; the zip tier is the interim carrier and both + consume the same suite output, so the .tm route stays open (G-066/G-067). +- Publishing the whole lib/ prefix as one zip - rejected: per-package zips give + independent versioning/revision cadence and per-package provenance records. + +## Notes + +- Related: G-139 - the consumption half (fetch/materialize tier, kit-vfs + declarations, retirement of the vendored trees). +- Related: G-123 - runtime-tier sibling; shares the punkbin layout docs, + schema-v2 provenance direction and the server-trust consent posture. Library + emission lands beside its family_artifacts.tcl schema-v2 work, not as a + second mechanism. +- Related: G-004 - this tier is what lets built library binaries leave the + tracked tree. +- Related: G-126 (achieved) - punkzip is the deterministic zip assembler. +- Related: G-066/G-067 - the .tm long game this tier is interim to. +- Anchors (archived): G-117 (embedded artifact record schema v1 and the + embed-then-hash sidecar rule), G-103 (the -r artifact emission pattern), + G-100 (the 8.6 tcllib/tcllibc build this publishes), G-102 (critcl + -target pin finding; the zig-cc critcl_zig.config - + goals/archive/G-102-buildsuite-zig-bootstrap.md). +- Overlap survey 2026-07-30 (drafting): goals_xref paths src/buildsuites + src/vendorlib_tcl8|9 src/runtime/vendorlib_vfs.toml bin/ src/make.tcl - + strongest live intersections G-123 (punkbin layout + schema), + G-115/G-127 (vendorlib_vfs.toml as a surface), G-004/G-006 (posture), + all carried as Related here or in G-139; no live goal owns + src/vendorlib_tcl8|9. diff --git a/goals/G-139-punkbin-library-consumption.md b/goals/G-139-punkbin-library-consumption.md new file mode 100644 index 00000000..d094b034 --- /dev/null +++ b/goals/G-139-punkbin-library-consumption.md @@ -0,0 +1,103 @@ +# G-139 Consume punkbin library artifacts: fetch/materialize tier replacing vendored tcllib/tcllibc + +Status: proposed +Scope: src/make.tcl (consent-gated library fetch entrypoint; libs step consuming the packages tier; vfslibs as touched); bin/packages// (new untracked materialization tier mirroring bin/runtime//); src/runtime/vendorlib_vfs.toml (tcllib/tcllibc kit-vfs declarations with supersedes removal of the legacy versions); src/vendorlib_tcl8/win32-x86_64/tcllib1.21 + src/vendorlib_tcl9/win32-x86_64/tcllib2.0 (retirement targets); src/vfs/*.vfs lib_tcl trees (kit payload switchover incl. the punk86-family tcllib 2.0 upgrade); lib_tcl8/ + lib_tcl9/ (deployed-tree materialization targets); user-config consent (server-trust pattern shared with G-123/G-006) +Goal: punkshell obtains tcllib and tcllibc from the punkbin library tier (G-138) instead of committed vendored copies - a consent-gated fetch materializes verified library zips into an untracked bin/packages/ input tier from which make.tcl installs the deployed lib_tcl trees and the declared kit vfs payloads - retiring the hand-dropped src/vendorlib tcllib trees in the G-004 direction, with every materialized package self-describing via its embedded artifact record, and the derived-project story recorded (fetch as the default path; in-project suite build as the alternative). +Acceptance: a make.tcl entrypoint fetches declared library artifacts from the punkbin lib tier into bin/packages// with sha1 verification and beside-toml metadata (consent keyed to server trust per the G-123 posture - fetches from the canonical origin prompt nothing new); the libs step materializes lib_tcl/allplatforms and lib_tcl/ from that tier with punkcheck-tracked provenance; tcllib and tcllibc enter src/runtime/vendorlib_vfs.toml (or its G-115/G-024 successor) declarations so the declared kit vfs trees receive them, with supersedes removing tcllib1.21 (including its nested tcllibc 0.4 and md5c) from the 8.6-family kits; the punk86-family upgrade to tcllib 2.0 + a current tcllibc is validated by kit-hosted runtests against a same-day baseline with every difference dispositioned; the vendored src/vendorlib_tcl8/win32-x86_64/tcllib1.21 and src/vendorlib_tcl9/win32-x86_64/tcllib2.0 trees are removed from the tracked tree once a bake fed from the artifact path reproduces working kits; a bake on a checkout without those trees (fetch consented or bin/packages pre-populated) ships kits whose tcllib/tcllibc provenance is verifiable from the embedded records; the derived-project decision is recorded in this file (fetch via the same entrypoint as the default - the copy-and-tweak in-project suite build per G-104's self-description contract remains the documented alternative). + +## Context + +The publish half is G-138 (see its Context for the 2026-07-30 provenance +investigation that motivated both). This goal owns the consuming side and the +repo-changing consequences: what src/vendorlib_tcl means afterwards (it +shrinks to genuinely hand-managed exceptions), how kit vfs trees stop drifting +independently, and how derived projects get the same packages. + +State at drafting worth recording: + +- The deployed lib_tcl8|9 trees are pure punkcheck mirrors of + src/vendorlib_tcl8|9 (make.tcl VENDORLIB_tcl step); src/lib_tcl8|9 are + empty. So switching the vendorlib source of truth switches the deployed + trees with no second mechanism. +- Kit vfs copies of tcllib (punk86/punk8win/punk8min/punk86bawt/ + punk8_statictwapi lib_tcl8/tcllib1.21; mkzipfix/punk9cook-family + lib_tcl9/tcllib2.0) are INDEPENDENT hand-drops from the same 2024 import - + vendorlib_vfs.toml declares only tcludp today. Bringing tcllib/tcllibc under + the G-037 declaration mechanism is part of this goal, not an aside. +- The 8.6 side is a version upgrade, not a swap: vendored is tcllib 1.21 + + tcllibc 0.4 (2023, critcl 3.2); the suite builds tcllib 2.0 + tcllibc + (critcl 3.3.1, zig cc). G-100 (achieved 2026-07-26) already ran punkshell + runtests on the suite-built 8.6 runtime - this goal's validation burden is + the KIT payload upgrade, using the established kit-hosted runtests method + (vfscommonupdate-current kits, per the recorded traps). +- Orphaned stray copies were cleared 2026-07-30 ahead of this work: + lib_tcl8/test_old + lib_tcl9/test_old moved (untracked, never on auto_path) + to TEMP_REFERENCE/retired_lib_tcl_test_old/. Remaining known strays to + sweep during the switchover: bin/test2/lib_tcl9/tcllib2.0, + scratch/tclkit86bi.vfs/lib/tcllibc. + +## Approach + +- bin/packages// mirrors the bin/runtime// input-tier idiom + (untracked working copies + beside-toml metadata, sha1-verified on + materialization); allplatforms is a target value here exactly as in the + punkbin lib tier. +- Unzip via the punk::zip/punkzip machinery already in the tree; installs out + of bin/packages into lib_tcl ride the existing punkcheck::install path so + provenance/skip-unchanged behaviour matches the current vendorlib step. +- vendorlib_vfs.toml gains install entries for tcllib + tcllibc per + generation, source paths pointing at the materialized tier; supersedes + handles the legacy-version removal (the explicit no-silent-mixed-version + mechanism it was built for). If G-115/G-024 land their toml consolidation + first, the declarations ride the successor surface. +- Consent: no new mechanism - the server-trust posture of G-123 (canonical + punkbin origin trusted by default; G-006's mechanism when landed, else the + interim documented flag). Unattended bakes stay unattended. +- Retirement sequencing: artifact path proven (bake + kit checks green) BEFORE + the vendored trees are removed; removal is its own commit so the fallback is + a revert. +- Derived projects: default is the fetch path (their make.tcl copy gains the + same entrypoint via the layout/G-027 update channels); projects wanting + source-level control copy a suite (G-104 self-description contract) and + build in-project - a tcllib/tcllibc-only suite needs no Tcl source tree + (critcl bundles its own headers; any punk runtime can drive it), which keeps + that alternative genuinely light. The stale samplesuite1 + download_and_build.config stub in the project-0.1 layout should be replaced + or removed as part of recording this story. + +## Alternatives considered + +- Keep vendoring (suite output committed into src/vendorlib) - rejected: + G-004 direction; see G-138 Alternatives. +- Fetch directly into lib_tcl with no bin/packages tier - rejected: the + input-tier split (immutable verified artifact copy vs punkcheck-installed + deployment) is what makes re-bakes, freshness checks and multi-kit + propagation coherent; it mirrors the proven runtime-tier design. +- Blanket-copy fetched packages into every kit vfs - rejected: per-package + per-kit declaration is the settled G-037 design (kit package sets + intentionally differ). + +## Notes + +- Related: G-138 - the publish half; this goal consumes its tier. +- Related: G-123 - shared consent posture and materialization idiom + (bin/runtime/ : runtimes :: bin/packages/ : libraries). +- Related: G-004 - the vendored-tree retirement is its first major payoff. +- Related: G-006 - consent surface, as coordinated through G-123. +- Related: G-065 - declarative vendoring generally; this goal implements the + artifact-fetch shape for the tcllib case specifically, and G-065 should + absorb or reference the resulting pattern rather than duplicate it. +- Related: G-115/G-024 - vendorlib_vfs.toml consolidation surfaces; the + declarations here ride whichever surface is current. +- Related: G-027 - how derived projects receive the updated make.tcl + entrypoint. +- Anchors (archived): G-037 (vendorlib -> kit vfs propagation mechanism and + its supersedes design), G-100 (8.6 suite runtime validation this builds on), + G-104 (buildsuite self-description contract the in-project-build alternative + leans on - goals/archive/G-104-maketcl-buildsuite-surface.md), G-119 + (freshness/row vocabulary if library listing surfaces are added - + goals/archive/G-119-runtime-active-freshness.md). +- Overlap survey 2026-07-30 (drafting): shared with G-138 (see its Notes); + additionally src/vfs intersects G-115/G-127/G-065 (payload declaration + surfaces - carried as Related) and lib_tcl/make.tcl show no live-goal + intersections.