From 9b587aed59ddaf727efe39a94ea13bd6b78141ab Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Fri, 31 Jul 2026 14:23:52 +1000 Subject: [PATCH] G-123 achieved: third-party runtime tiers - flip, archive record, reference sweep Status active -> achieved 2026-07-31 (acceptance walk in the detail file); index entry moved to GOALS-archive.md; live-tier sweep: achieved markers on the G-127/G-130/G-131/G-141 Related lines and the G-006 absorption note (interim trust flags libfetch + punk-runtime named); actionable-note pushes to G-105 (server surfaces ready for cross-target artifacts), G-101 (8.6 tier riding the mechanism; tclkit86bi sidecars), G-067 (provenance/consent patterns to inherit), G-127 (the no-alternate-mapvfs-seam finding from the e2e demonstration). xref report: no unlinked pair had G-123 as its only bridge. Detail-file rename to goals/archive/ follows as its own pure-rename commit. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- GOALS-archive.md | 4 ++++ GOALS.md | 5 ----- goals/G-006-prebuilt-artifact-download.md | 12 ++++++++++-- goals/G-067-module-artifact-channel.md | 6 ++++++ goals/G-101-tcl86-kit-container-strategy.md | 6 ++++++ goals/G-105-buildsuite-cross-target.md | 7 +++++++ goals/G-123-thirdparty-runtime-tiers.md | 2 +- goals/G-127-crosstarget-vfs-bake.md | 13 ++++++++++--- goals/G-130-32bit-windows-buildsuites.md | 4 +++- goals/G-131-boot-payload-autodetection.md | 4 +++- goals/G-141-libfetch-target-selection.md | 12 +++++++++--- 11 files changed, 59 insertions(+), 16 deletions(-) diff --git a/GOALS-archive.md b/GOALS-archive.md index a2aad9f3..0ad76013 100644 --- a/GOALS-archive.md +++ b/GOALS-archive.md @@ -207,6 +207,10 @@ Acceptance: from the pinned zig and fetched sources the suite produces a Tk 8.6 Scope: src/make.tcl (target-platform derivation; store folder, exe suffixing, kit naming, presence checks and process-sweep tooling keyed by target; platform-canon inline copy; zipfs-less zip-type assembly fallback); src/modules/punk/platform-999999.0a1.0.tm (stable canon tags for cygwin-family hosts, help platforms doc); src/runtime/mapvfs.config (per-entry target platform via minimal compatible extension - format ownership coordinated with G-024); bin/runtime// store tiers (as consumed; fixture tier for tests); src/tests/shell/testsuites/punkexe/ (characterization) Acceptance: an msys/cygwin-runtime tclsh driving bakelist/bake on windows reports - and for a changed kit, bakes and deploys - the same win32-x86_64 kit set with identical names and store addressing as a native tclsh, with the pre-deploy process sweep using windows tooling for win32 targets regardless of host personality; punk::platform and the make.tcl inline canon map MSYS_NT/MINGW64_NT/CYGWIN_NT/UCRT64-class hosts to stable documented tags (MSYSTEM-variance documented, no windows build number in the tag) and 'help platforms' lists them; a mapvfs entry declaring a non-default target platform lists in bakelist with its own tier's store presence and bakes to a correctly suffixed artifact, verified against a fixture store tier (a real third-party runtime is not required); a zipfs-less driving tcl (8.6) assembles a zip-type kit via runtime split + punk::zip::mkzip + concatenation (archive-start-relative offsets) and the artifact boots on a zipfs-capable runtime; the linuxulator expectation (a linux-personality tclsh on FreeBSD 14/15 addresses the linux-x86_64 store unchanged) is recorded here with verification deferred to the planned linux-kit-on-FreeBSD trial; native-windows behaviour stays characterization-stable (existing punkexe maketcl tests pass unchanged). +### G-123 [achieved 2026-07-31] Third-party runtime tiers via artifact servers: per-target publish, list, fetch, materialize → detail: goals/archive/G-123-thirdparty-runtime-tiers.md +Scope: src/scriptapps/punk-runtime.ps1 + src/scriptapps/bin/punk-runtime.bash + punk-runtime_wrap.toml (named-tier operations in both payloads; bin/punk-runtime.cmd via the established re-wrap channel); punkbin artifact repo (external, c:/repo/jn/punkbin - per-tier layout, defaults, sha1sums, schema-v2 provenance sidecars incl. retroactive backfill, layout docs); src/buildsuites/suite_tcl90/ (family_artifacts.tcl schema-v2 emission for suite-built artifacts); alternative-server override + trusted-server set (PUNKBIN_URL pattern; consent surface coordinated with G-006); src/tests/shell/testsuites/binscripts/ (fixture-server characterization); bin/runtime// stores (as materialization targets) +Acceptance: against a fixture artifact server (httpfixture precedent) carrying a non-native tier, punk-runtime supports an explicit target-tier argument on list/use (and the fetch path) in BOTH payloads - ps1 driven as the committed bin/punk-runtime.cmd under cmd.exe, bash driven under msys bash - with row/verdict parity and the established single-line row vocabulary scoped to the named tier; 'use' materializes the tier's working copy under bin/runtime// with sha1 verification and beside-toml metadata, and freshness verdicts compare against that tier's server default; artifact-toml schema v2 provenance distinguishes suite-built from third-party (class/builder/source_url/upstream_ref/retrieved), with third-party rows carrying a compact origin tag in list -remote's sparse notes column and full fields in the per-name detail view while suite-built rows are unchanged; existing punkbin third-party artifacts (e.g win32-x86_64/tclsfe-x64.exe, tclkit86bi.exe) gain retroactive schema-v2 sidecars additively (artifact bytes, bare pre-family names and sha1sums.txt authority unchanged; toml-less entries continue to degrade as no-basis rows); consent is keyed to SERVER TRUST - fetches from the trusted set (default: the canonical punkbin origin) prompt nothing, leaving existing unattended flows unchanged, while the first fetch from an untrusted/alternative server gates on an explicit acknowledgement (G-006's mechanism when landed, else an interim documented flag) that surfaces the artifact's provenance class; an end-to-end fixture demonstration wires a fetched fixture-tier runtime through a G-122 non-default-target mapping entry to a bakelist row showing the runtime present; the punkbin layout docs record the per-tier + schema-v2 extension and existing native-tier clients are unaffected (existing binscripts tests pass unchanged). + ### G-124 [achieved 2026-07-26] Dependency-free zip reading in punk::zip: archives, runtime-prefixed executables, and a scripter-facing introspection surface → detail: goals/archive/G-124-punkzip-reader.md Scope: src/modules/punk/zip-999999.0a1.0.tm (reader + introspection procs, base-offset derivation factored out of extract_preamble for shared use, buildversion); src/make.tcl (zipfs-less kit extraction path as first consumer - the zipfile::decode requirement is removed, not made optional); src/tests/modules/punk/zip/ (new testsuite - mkzip round-trip, prefixed-archive cases, introspection, unsupported-input behaviour); bootsupport + src/vfs/_vfscommon.vfs copies via established sync channels Acceptance: punk::zip extracts to a target directory from (a) a plain zip, (b) an executable-prefixed archive with archive-relative offsets and (c) one with file-relative offsets - bin/runtime/win32-x86_64/tclsh90b4_piperepl.exe is the recorded (c) case, on which the current split-then-tcllib path fails with "Bad zip file. Bad closure." - producing CRC-verified byte-identical members with directory entries materialized as directories, on a tclsh with no zipfs, no vfs::zip and no tcllib reachable (the interpreter used is recorded here); an introspection call lists members without extracting, returning per-entry name, size, compressed size, method, mtime, crc and directory-vs-file, over the same three input shapes, and both surfaces carry punk::args definitions with worked examples so 'i punk::zip::' documents them; unsupported inputs (zip64, encrypted, unknown compression method) fail naming the reason rather than emitting partial or garbled output; make.tcl's zipfs-less kit extraction consumes the new surface and NO zipfile::decode requirement remains anywhere in the shipped tree, verified by a tree grep, and demonstrated by an 8.6 zipfs-less bake of a zip kit on a tcllib-less tclsh (msys2 /usr/bin/tclsh8.6 as reference host) producing an artifact that boots with its tcl_library present; a new src/tests/modules/punk/zip/ suite covers a mkzip -> read round trip asserting names, content, directory-vs-file classification and stored attributes - recording the answer to the standing zipfs dir-misidentification question noted in punk::mix::cli and src/vfs/mkzipfix.vfs - plus the three prefixed-archive cases, the introspection output, and the unsupported-input failures; punk::zip buildversion minor-bumped with its changelog line. diff --git a/GOALS.md b/GOALS.md index ea5ebab5..fa01af81 100644 --- a/GOALS.md +++ b/GOALS.md @@ -377,11 +377,6 @@ Detail: goals/G-115-declarative-vfs-composition.md Scope: src/buildsuites/suite_tcl90/ (build_tcltls module + crypto-backend build, sources.config + build.zig.zon source records, kit-family bi payload + metadata extension, test_gate record step); src/vfs kit payloads carrying vendored tcltls binaries (current-state reference + recorded disposition only - removal stays G-004-era work); punkbin (as eventual artifact destination via the G-067 library class); consumers punk::imap4 / punk::netbox (package require tls) as verification context Detail: goals/G-116-suite-built-tcltls.md -### G-123 [active] Third-party runtime tiers via artifact servers: per-target publish, list, fetch, materialize -Scope: src/scriptapps/punk-runtime.ps1 + src/scriptapps/bin/punk-runtime.bash + punk-runtime_wrap.toml (named-tier operations in both payloads; bin/punk-runtime.cmd via the established re-wrap channel); punkbin artifact repo (external, c:/repo/jn/punkbin - per-tier layout, defaults, sha1sums, schema-v2 provenance sidecars incl. retroactive backfill, layout docs); src/buildsuites/suite_tcl90/ (family_artifacts.tcl schema-v2 emission for suite-built artifacts); alternative-server override + trusted-server set (PUNKBIN_URL pattern; consent surface coordinated with G-006); src/tests/shell/testsuites/binscripts/ (fixture-server characterization); bin/runtime// stores (as materialization targets) -Detail: goals/G-123-thirdparty-runtime-tiers.md - - ### G-127 [proposed] Cross-target bake of a custom .vfs: per-target payload and per-target output location Scope: src/make.tcl (target-keyed output location and deploy folder, kit-name disambiguation by target, payload/target mismatch reporting, retirement of the kit-loop cross-platform TODO); src/runtime/vendorlib_vfs.toml (per-target payload declaration - format coordinated with G-115 and G-024); src/runtime/mapvfs.config (one vfs definition paired with several targets); src/vfs/ (per-target payload convention with punk9linux.vfs as the migration case); bin/kits// (non-native kit output tier, beside the existing bin/runtime// input tier); src/tests/shell/testsuites/punkexe/ (characterization - two-target bake, no-collision, mismatch report); bin/AGENTS.md + src/AGENTS.md (where cross-target kits land) diff --git a/goals/G-006-prebuilt-artifact-download.md b/goals/G-006-prebuilt-artifact-download.md index 39ee02d7..bf916862 100644 --- a/goals/G-006-prebuilt-artifact-download.md +++ b/goals/G-006-prebuilt-artifact-download.md @@ -57,10 +57,18 @@ This goal is parallel to and independent of G-005. A user with zig uses G-005; a make.tcl-integrated downloader this file's Approach already leans toward is the indicated shape - e.g. a 'make.tcl tool fetch ' surface reusing the tool step's vendored-version/installed-state knowledge, with this goal's - consent flag and G-123's server-trust keying; extending punk-runtime remains - G-123's parallel lane for runtime tiers. The first concrete artifact this + consent flag and G-123's server-trust keying (G-123 achieved 2026-07-31 - + see goals/archive/G-123-thirdparty-runtime-tiers.md; punk-runtime now + carries the live runtime-tier instance). The first concrete artifact this channel would serve exists: punkres on the punkbin tools tier (G-128, achieved - route in src/tools/punkres/PROVENANCE.md "Distribution"). +- G-123 (archived) left this goal the interim consent flags to ABSORB into + one mechanism when it lands: make.tcl libfetch's -serverurl/-trust-server + (G-139) and punk-runtime's -trust-server / PUNKBIN_TRUST_SERVER=1 + (fetch-only gate; canonical origin trusted by default; never an + interactive prompt) - one vocabulary, deliberately, so this goal replaces + both rather than adding a third. See + goals/archive/G-123-thirdparty-runtime-tiers.md. - G-139 (archived 2026-07-30) landed a WORKING instance of this goal's consent shape for the library tier: 'make.tcl libfetch' with server-trust keying - canonical punkbin origin fetches gate nothing, any other origin diff --git a/goals/G-067-module-artifact-channel.md b/goals/G-067-module-artifact-channel.md index 1a1275f8..5978f11b 100644 --- a/goals/G-067-module-artifact-channel.md +++ b/goals/G-067-module-artifact-channel.md @@ -73,3 +73,9 @@ and pull (G-065) instead of re-vendoring by hand. goals/archive/G-138-punkbin-library-artifact-tier.md); this goal's .tm artifact channel remains the long game and would replace that tier's carrier format while inheriting its record/provenance pattern. +- G-123 (achieved 2026-07-31, archived) completed the runtime-tier kin of + this channel: schema-v2 build-origin provenance + (suite-built|third-party|local + builder/source_url/upstream_ref/ + retrieved) and the server-trust consent gate are now live patterns to + inherit for module artifacts. See + goals/archive/G-123-thirdparty-runtime-tiers.md. diff --git a/goals/G-101-tcl86-kit-container-strategy.md b/goals/G-101-tcl86-kit-container-strategy.md index b1db5149..3c616108 100644 --- a/goals/G-101-tcl86-kit-container-strategy.md +++ b/goals/G-101-tcl86-kit-container-strategy.md @@ -402,3 +402,9 @@ it over cookfs/zipvfs, since Tcl 9 already has zipfs and needs no help. `Tcl_FSRegister` / TCL_FILESYSTEM_VERSION_1 unchanged in 9.1. Core `zlib push` + `chan create` on 9 would also let mk4vfs use real streaming, which the 8.6 note above says it cannot. +- G-123 (achieved 2026-07-31, archived) landed the runtime-tier serving + mechanism a published 8.6-container tier would ride (per-tier + list/fetch/materialize + schema-v2 provenance + server-trust consent); + the existing punkbin 8.6-era third-party kits (tclkit86bi etc) already + carry its retroactive provenance sidecars. See + goals/archive/G-123-thirdparty-runtime-tiers.md. diff --git a/goals/G-105-buildsuite-cross-target.md b/goals/G-105-buildsuite-cross-target.md index 0846fbb8..3ffccd4f 100644 --- a/goals/G-105-buildsuite-cross-target.md +++ b/goals/G-105-buildsuite-cross-target.md @@ -153,3 +153,10 @@ goals/archive/G-100-suite-tcl86-tk-tcllib.md), G-102 (achieved 2026-07-21 - driv platform::generic-shaped (linux-x86_64) for critcl's MapPlatform loader, with the glibc floor riding only the triple - see the G-140 detail file's mechanism notes. +- G-123 (achieved 2026-07-31, archived) landed the server surfaces this + goal's cross-target artifacts will ride: explicit target-tier + list/fetch/use/materialize in both punk-runtime payloads, schema-v2 + provenance (class suite-built|third-party|local), and the server-trust + consent gate - publish a suite-built cross-target runtime into its + punkbin tier and the whole client path already works. See + goals/archive/G-123-thirdparty-runtime-tiers.md. diff --git a/goals/G-123-thirdparty-runtime-tiers.md b/goals/G-123-thirdparty-runtime-tiers.md index f03b1711..a3c6246e 100644 --- a/goals/G-123-thirdparty-runtime-tiers.md +++ b/goals/G-123-thirdparty-runtime-tiers.md @@ -1,6 +1,6 @@ # G-123 Third-party runtime tiers via artifact servers: per-target publish, list, fetch, materialize -Status: active +Status: achieved 2026-07-31 Scope: src/scriptapps/punk-runtime.ps1 + src/scriptapps/bin/punk-runtime.bash + punk-runtime_wrap.toml (named-tier operations in both payloads; bin/punk-runtime.cmd via the established re-wrap channel); punkbin artifact repo (external, c:/repo/jn/punkbin - per-tier layout, defaults, sha1sums, schema-v2 provenance sidecars incl. retroactive backfill, layout docs); src/buildsuites/suite_tcl90/ (family_artifacts.tcl schema-v2 emission for suite-built artifacts); alternative-server override + trusted-server set (PUNKBIN_URL pattern; consent surface coordinated with G-006); src/tests/shell/testsuites/binscripts/ (fixture-server characterization); bin/runtime// stores (as materialization targets) Goal: runtime artifacts for tiers beyond the host-native platform - third-party-built personalities (e.g an msys2-packaged mingw64-x86_64 tclsh) and cross-personality consumers (a linux tier used from FreeBSD) - are first-class on punkbin or an alternative artifact server: punk-runtime lists/fetches/uses artifacts for an explicitly named target tier from both payloads, materializing sha1-verified -r artifacts + toml metadata into bin/runtime//, with build-origin provenance (suite-built vs third-party) labeled on every artifact, consent keyed to server trust rather than artifact origin, and the G-122 chain completed from server artifact to a bakeable mapping entry. Acceptance: against a fixture artifact server (httpfixture precedent) carrying a non-native tier, punk-runtime supports an explicit target-tier argument on list/use (and the fetch path) in BOTH payloads - ps1 driven as the committed bin/punk-runtime.cmd under cmd.exe, bash driven under msys bash - with row/verdict parity and the established single-line row vocabulary scoped to the named tier; 'use' materializes the tier's working copy under bin/runtime// with sha1 verification and beside-toml metadata, and freshness verdicts compare against that tier's server default; artifact-toml schema v2 provenance distinguishes suite-built from third-party (class/builder/source_url/upstream_ref/retrieved), with third-party rows carrying a compact origin tag in list -remote's sparse notes column and full fields in the per-name detail view while suite-built rows are unchanged; existing punkbin third-party artifacts (e.g win32-x86_64/tclsfe-x64.exe, tclkit86bi.exe) gain retroactive schema-v2 sidecars additively (artifact bytes, bare pre-family names and sha1sums.txt authority unchanged; toml-less entries continue to degrade as no-basis rows); consent is keyed to SERVER TRUST - fetches from the trusted set (default: the canonical punkbin origin) prompt nothing, leaving existing unattended flows unchanged, while the first fetch from an untrusted/alternative server gates on an explicit acknowledgement (G-006's mechanism when landed, else an interim documented flag) that surfaces the artifact's provenance class; an end-to-end fixture demonstration wires a fetched fixture-tier runtime through a G-122 non-default-target mapping entry to a bakelist row showing the runtime present; the punkbin layout docs record the per-tier + schema-v2 extension and existing native-tier clients are unaffected (existing binscripts tests pass unchanged). diff --git a/goals/G-127-crosstarget-vfs-bake.md b/goals/G-127-crosstarget-vfs-bake.md index 9acfc4b6..e4618077 100644 --- a/goals/G-127-crosstarget-vfs-bake.md +++ b/goals/G-127-crosstarget-vfs-bake.md @@ -157,6 +157,13 @@ exists locally (empty but for sha1sums.txt) and punkbin upstream carries - Related: G-131 - sibling surface in the same kit loop (its kit_type_mismatch rework and container autodetection read the same mapvfs/runtime_caps machinery whose outputs this goal relocates); their previously recorded bridge was G-133, now archived. -- Related: G-123 - a published third-party runtime tier is a consumer of the cross-target - artifacts this goal gives a home to; recorded directly now that the shared G-122/G-133 - references are both archive-tier. +- Related: G-123 (achieved 2026-07-31 - see + goals/archive/G-123-thirdparty-runtime-tiers.md) - a published third-party runtime + tier is a consumer of the cross-target artifacts this goal gives a home to; recorded + directly now that the shared G-122/G-133 references are both archive-tier. +- G-123 (archived) recorded a finding this goal's make.tcl surface should note: its + end-to-end demonstration (fixture-tier runtime -> non-default-target mapvfs entry -> + bakelist row runtime-present) had to be LIVE-REVERSIBLE because make.tcl has no + alternate-mapvfs-config seam for characterization tests - adding one (or a scratch + bakehouse mode) would let cross-target bake behaviour be pinned without editing the + real config. See its increment-5 Progress notes. diff --git a/goals/G-130-32bit-windows-buildsuites.md b/goals/G-130-32bit-windows-buildsuites.md index f5dffdd7..89a51256 100644 --- a/goals/G-130-32bit-windows-buildsuites.md +++ b/goals/G-130-32bit-windows-buildsuites.md @@ -68,7 +68,9 @@ the other's generation. - G-103 (achieved 2026-07-22 - see goals/archive/G-103-runtime-kit-family.md) defines the artifact-metadata contract the Scope and Acceptance above name: a published runtime's record carries its target, which for this goal must name the 32-bit platform. -- Related: G-123 - the artifacts publish through the runtime-tier mechanism, and +- Related: G-123 (achieved 2026-07-31 - see + goals/archive/G-123-thirdparty-runtime-tiers.md) - the artifacts publish through + the runtime-tier mechanism, and `punk-runtime.cmd list -platform win32-ix86` is where a user would see suite-built runtimes appear beside the third-party ones. - Related: G-129 (achieved 2026-07-27 - see goals/archive/G-129-kitboot-mountpoint-derivation.md) diff --git a/goals/G-131-boot-payload-autodetection.md b/goals/G-131-boot-payload-autodetection.md index 75ce796c..bb7b1568 100644 --- a/goals/G-131-boot-payload-autodetection.md +++ b/goals/G-131-boot-payload-autodetection.md @@ -85,7 +85,9 @@ independently-written branches. - Related: G-130 - its 32-bit runtimes are candidates to carry both capabilities, and the deliverable-count argument is sharpest there: a new platform x two generations x two containers is four artifacts to keep current, or two if this goal lands. -- Related: G-123 - fewer runtime deliverables is fewer artifacts to publish, checksum and +- Related: G-123 (achieved 2026-07-31 - see + goals/archive/G-123-thirdparty-runtime-tiers.md) - fewer runtime deliverables is + fewer artifacts to publish, checksum and keep current in the punkbin tiers; the reduction this goal records is directly a reduction in that goal's ongoing work. - Related: G-024 - if the container becomes a bake-time choice, a mapvfs entry naming a kit diff --git a/goals/G-141-libfetch-target-selection.md b/goals/G-141-libfetch-target-selection.md index 758627ac..06ca4e8e 100644 --- a/goals/G-141-libfetch-target-selection.md +++ b/goals/G-141-libfetch-target-selection.md @@ -34,7 +34,8 @@ The growth vector making this worth a goal: G-004 names the remaining vendored trees (Img, itcl, sqlite, tdbc, twapi, ...) for migration through the G-139 template - declarations multiply across libraries x platforms x generations, and all-or-nothing fetch scales as every-host x every-artifact. -The runtime tier - the design twin per G-123 - is already selective +The runtime tier - the design twin per G-123 (achieved 2026-07-31 - see +goals/archive/G-123-thirdparty-runtime-tiers.md) - is already selective (punk-runtime fetches/materializes named runtimes on demand); the lib tier has no equivalent of "materialize what this checkout needs". @@ -81,8 +82,13 @@ has no equivalent of "materialize what this checkout needs". target selection must survive that absorption. - Related: G-065 - declarative vendoring absorbs/references the G-139 fetch pattern; selection becomes part of the inherited pattern. -- Related: G-123 - runtime-tier twin whose per-artifact selectivity this - brings to the lib tier; keep vocabulary parity where sensible. +- Related: G-123 (achieved 2026-07-31 - see + goals/archive/G-123-thirdparty-runtime-tiers.md) - runtime-tier twin whose + per-artifact selectivity this brings to the lib tier; keep vocabulary + parity where sensible. Its consent surface is now LIVE in punk-runtime + (-trust-server / PUNKBIN_TRUST_SERVER=1, fetch-only gate) alongside + libfetch's -trust-server - this goal's selection options join that + vocabulary, and G-006 absorbs all of it when it lands. - Related: G-127 - owns per-target kit payload declaration and bake output targeting; whichever lands second adopts the other (its per-target declarations become this derivation's input).