diff --git a/goals/G-103-runtime-kit-family.md b/goals/G-103-runtime-kit-family.md index 4eccb4c6..456d3d77 100644 --- a/goals/G-103-runtime-kit-family.md +++ b/goals/G-103-runtime-kit-family.md @@ -236,3 +236,101 @@ contract the sequencing note promises G-101. NAMING DECISION (user-approved): it adopts the kit-artifact tier punk-.exe verbatim and layers its -dev / release-gated-plain-name policy on the working-name tier. + +## Progress + +### 2026-07-22 increment: kit-family + kit-family-artifacts steps landed, all three members verified + +ATTACHED-IMAGE LAYOUT DECISION (resolves the Context "open at activation" +items - staging-tree convention and payload layout): the image mirrors the +installed prefix, with two boot-anchored placements and one punkshell-authored +hook file: + +- `tcl_library/` at the app root (the C-level zipfs boot looks only for + `/app/main.tcl` and `/app/tcl_library`; NO main.tcl is included - stock boot + falls through to the ordinary interactive shell, per the Context requirement + that payload sit where the STOCK boot searches). +- tm modules at `tcl9//` beside it (tm.tcl Defaults anchors at + `[file dirname [info library]]` = `/app`). +- batteries under `lib//` with their INSTALLED-SHAPE pkgIndexes unchanged + (`$dir/../../bin` dll references resolve to `/app/bin`), dlls under `bin/` - + the same shape the suite's out/ prefix and the existing punk .vfs payloads + use, so pkgIndex generation stays single-source (build_tclthread/build_tk + return their generated pkgIndex + version facts; tclvfs's configured package + generation is shared between prefix install and family staging). +- `lib/pkgIndex.tcl`: a one-line stock hook (`lappend ::auto_path $dir`) + joining lib/ to the package search - auto_path starts as + `[tcl_library, /app]`, the /app scan sources `/app/*/pkgIndex.tcl`, and + tclPkgUnknown re-scans auto_path growth mid-scan (the same stock mechanism + tcllib's own top-level index uses). No C changes, no boot script. +- Tk script library resolution: tcl_findLibrary iterates auto_path entries + joined with `tk9.0`, so `/app/lib/tk9.0` is found once the hook has run + (package require Tk goes through the pkgIndex scan first by construction). + +Landed (suite_tcl90, all in the default build pipeline): + +- `kit-family` step: two WriteFiles staging trees (core, bi) + three + zipfs_mkimg wraps emit the working-name products into `out/family/`: + tclsh9.0.5.exe (prefix exe tclsh90s), tclsh9.0.5-punk.exe (tclsh90spr), + tclsh9.0.5-punk-bi.exe (tclsh90spr + Tk/tklib payload). +- `tools/family_check.tcl`: per-member self-containment verification - the kit + is copied ALONE into a scratch dir and probed from there (defeats the + `/../lib` tm root and exe-relative tcl_findLibrary entries) with + TCL_LIBRARY/TK_LIBRARY/TCLLIBPATH/VFS_LIBRARY/TCL*_TM_PATH scrubbed and + TCLSH_PIPEREPL controlled per probe. The piperepl discriminator is + `[info exists ::tclsh(istty)]` in a script-arg run (G-096 matrix: machinery + published iff patched AND gate open; script-arg runs never set dorepl, so no + console-reopen/hang risk). +- `kit-family-artifacts` step + `tools/family_artifacts.tcl`: punkbin-layout + emission into `out/family/punkbin/win32-x86_64/` - immutable -r artifact + copies (`-Dfamilyrev=N`, default 1), per-artifact toml metadata (variant, + working name, revision, target, sha1, size, built, tcl patchlevel, piperepl + policy incl the TCLSH_PIPEREPL=0 opt-out, attached battery versions, + suite/toolchain/optimize, per-source checkout uuids where materialized - + manifest.uuid is a per-repo fossil setting: tcl/tk/thread carry it, + tclvfs/tcllib/tklib record "unrecorded" - and G-107 test-evidence result + lines from out/testreports/*.summary), plus punkbin-format sha1sums.txt. + Deliberately runs UNDER the plain family kit itself (sha1 via its attached + tcllib) - each emission re-proves the runtime executes real tooling + self-contained. Depends on the checks: only verified kits get records. +- Consumers: working names copied to bin/runtime/win32-x86_64/ (machine-local, + gitignored) and mapvfs.config entries added - tclsh9.0.5-punk.exe + + punk9wintk905.vfs -> punk9_beta, tclsh9.0.5-punk-bi.exe + + punk9win_for_tkruntime.vfs -> punk9bi_beta (*_beta trial convention; kit + working names per the naming decision). + +VERIFICATION 2026-07-22 (acceptance self-containment checks; suite build on the +9.0.5 checkout 1a9c3b9d96, zig 0.16.0, ReleaseFast; family_check probes run +from a scratch dir with scrubbed env - full pipeline +`suite.tcl build` PASS end-to-end including both new steps): + +- ALL THREE members: info patchlevel 9.0.5; `info library` = + //zipfs:/app/tcl_library (attached image); tzdata + encodings reachable; + `package require platform` served from the attached tm tree; Thread 3.0.7 + loads from the attached zip AND executes (thread::create + cross-thread eval + = 42); vfs 1.4.2 + vfs::zip 1.0.4 + vfs::urltype 1.0, with a functional + vfs::zip mount round-trip (zipfs mkzip a scratch payload, mount via tclvfs, + read back, unmount); tcllib md5 2.0.9 with the tcllibc critcl accelerator + ENGAGED (md5::accel(critcl)=1). +- plain tclsh9.0.5.exe: stock behaviour proven - no ::tclsh machinery with env + unset AND with TCLSH_PIPEREPL=1 (unpatched binary ignores the enable). +- punk tclsh9.0.5-punk.exe / punk-bi tclsh9.0.5-punk-bi.exe: piperepl ACTIVE BY + DEFAULT (::tclsh machinery published with env unset), disabled via the + documented TCLSH_PIPEREPL=0 opt-out. +- punk-bi additionally: Tk 9.0.2 loads from the attached zip - button + create/destroy + root destroy clean; tklib tooltip 2.0.4. +- Artifacts emitted: tclsh9.0.5-r1.exe (sha1 0f469ecd..., 6455 KB), + tclsh9.0.5-punk-r1.exe (54ee10ba..., 6458 KB), tclsh9.0.5-punk-bi-r1.exe + (9805f035..., 9165 KB) + per-artifact tomls + sha1sums.txt, generated by the + plain family kit itself. + +Remaining for acceptance: + +- runtime scriptset (punk-runtime) list/use surfacing of the artifact metadata + (list variant/version facts from the tomls; 'use' materializes a chosen -r + artifact into its working name). +- exercise the new mapvfs entries with an actual kit wrap (punk9_beta / + punk9bi_beta) - entries land this increment, the wrap trial is the next one + (kit builds touch user-facing bin/ kits; beta-trial convention applies). +- record the 8.6-family deferral note against G-101 when closing (the + acceptance's explicit deferral - no work item here). diff --git a/src/buildsuites/suite_tcl90/README.md b/src/buildsuites/suite_tcl90/README.md index 19bc4464..29d3371d 100644 --- a/src/buildsuites/suite_tcl90/README.md +++ b/src/buildsuites/suite_tcl90/README.md @@ -1,11 +1,13 @@ -# suite_tcl90 - zig build of the Tcl 9.0 windows runtime (G-096, G-102) +# suite_tcl90 - zig build of the Tcl 9.0 windows runtime (G-096, G-102, G-103) The first real, tracked buildsuite: reproducibly builds `tclsh90s.exe` (static) and `tclsh90szip.exe` (self-contained zipfs) plus the TCLSH_PIPEREPL variants and the thread/vfs/tk extension dlls, tklib/tcllib installs and critcl-built tcllibc, from Tcl core-9-0-branch sources, using zig only (no MS toolchain, per the project -toolchain policy). (vqtcl/vlerq was removed 2026-07-20 - an old-tclkit experiment; -future tclkit investigation has TEMP_REFERENCE/metakit and TEMP_REFERENCE/KitCreator +toolchain policy) - and assembles them into the G-103 RUNTIME KIT FAMILY: +verified self-contained batteries-attached runtimes (see "Runtime kit family" +below). (vqtcl/vlerq was removed 2026-07-20 - an old-tclkit experiment; future +tclkit investigation has TEMP_REFERENCE/metakit and TEMP_REFERENCE/KitCreator as reference material.) Everything transient lives under `src/buildsuites/_build/suite_tcl90/` (VCS-ignored via @@ -27,7 +29,8 @@ run from THIS directory with the pinned zig. Sources come from `build.zig.zon` - content-hashed per-checkin tarball pins ('refresh' = bump a pin; `zig fetch --save= ` edits the manifest). Steps for the nested staged build: `-Dsteps=...` (repeat the flag), default install install-libraries make-zipfs smoke -tklib tcllib tcllibc. The test gate: `zig build bootstrap -Dsteps=test-gate` (or +tklib tcllib tcllibc kit-family kit-family-artifacts. The test gate: +`zig build bootstrap -Dsteps=test-gate` (or invoke the staged build file directly, as suite.tcl does). zig extracts fetched packages into `zig-pkg/` beside the manifest and `.zig-cache/` for the outer invocation - both VCS-ignored. @@ -107,6 +110,62 @@ Timing-sensitive suites (thread) must not run in parallel with heavy ones: `suite.tcl test` passes `-j1` so combined step invocations serialize (zig otherwise runs independent steps concurrently). +## Runtime kit family (G-103) + +The `kit-family` step (in the default build pipeline) assembles the project's +runtime kit family - runnable, self-contained executables whose info library AND +core batteries live in the initially attached zip, depending on no external +filesystem tree: + +- **plain** `tclsh.exe` (e.g `tclsh9.0.5.exe`) - stock shell (no + piperepl patch) + batteries: loadable Thread, tclvfs with the vfs::* packages, + tcllib + tcllibc (critcl accelerators). +- **punk** `tclsh-punk.exe` - the same batteries on the + TCLSH_PIPEREPL-patched shell, gate ENABLED BY DEFAULT (opt out with + `TCLSH_PIPEREPL=0`; users wanting stock semantics take the plain kit). +- **punk-bi** `tclsh-punk-bi.exe` - punk + the batteries-included + tier we build: Tk (dll + script library) and tklib. (tcltls joins when its + zig-built crypto backend lands - see the G-103 goal notes.) + +Attached-image layout (also the make.tcl kit contract - a future kit build +extracts this image and merges its .vfs payload over it): `tcl_library/` at the +app root (the C-level zipfs boot looks only for `/app/main.tcl` and +`/app/tcl_library`; no main.tcl is included, so the stock boot falls through to +the ordinary interactive shell), tm modules at `tcl9//` beside it (tm.tcl +anchors at the dirname of tcl_library), batteries under `lib//` with their +installed-shape pkgIndexes (`$dir/../../bin` dll references resolve to +`/app/bin`), dlls under `bin/`, and a one-line stock pkgIndex hook at +`lib/pkgIndex.tcl` joining `lib/` to the package search (auto_path is +`[tcl_library, /app]`, and tclPkgUnknown re-scans auto_path growth mid-scan - +the same mechanism tcllib's own top-level index uses). Battery dlls load out of +zipfs via the core's copy-to-temp fallback - the TIP-741 temp-dir accumulation +wart is characterized in the G-103 goal notes. + +Every member is verified by `tools/family_check.tcl` before the step passes: +the kit is copied ALONE into a scratch dir and probed from there with a +scrubbed environment (no external Tcl visible) - patchlevel, zipfs tcl_library, +tzdata/encodings, tm modules, functional Thread (cross-thread eval), tclvfs +(vfs::zip mount round-trip), tcllib md5 with the tcllibc accelerator engaged, +piperepl default/opt-out behaviour per variant, and Tk create/destroy + tklib +for bi. Products land in `out/family/`. + +The `kit-family-artifacts` step (also default; depends on the checks) emits the +punkbin ARTIFACT tier into `out/family/punkbin/win32-x86_64/`: immutable +`-r`-named copies (revision via `-Dfamilyrev=N`, suite.tcl +`-zigargs {-Dfamilyrev=N}`; default 1) plus per-artifact `.toml` metadata +(variant, working name, sha1/size, tcl patchlevel, piperepl policy, attached +battery versions, source-checkout provenance, toolchain, G-107 test-evidence +result lines) and a punkbin-format `sha1sums.txt`. The emission deliberately +runs UNDER the plain family kit itself (sha1 via its attached tcllib) - every +run re-proves the family runtime executes real tooling self-contained. +Publication to the real punkbin repo remains a deliberate user step, deferred +per the goal notes until the family shape is accepted. + +Consumers: working-name runtimes are copied into `bin/runtime/win32-x86_64/` +and referenced by `src/runtime/mapvfs.config` (punk -> punk9wintk905.vfs as +`punk9_beta`, punk-bi -> punk9win_for_tkruntime.vfs as `punk9bi_beta`, per the +*_beta trial convention). + ## Pinned zig The suite pin is **zig 0.16.0** (current official release; adopted 2026-07-20 - @@ -140,10 +199,12 @@ producing stale/duplicate artifacts at link time). CRT provides the exe entry. - `tools/` - `wrapfiletofile.zig` (uuid-header overlay generation), `stagetree.zig` (bootstrap-mode source/recipe materializer), - `zipfs_mkzip.tcl` / `zipfs_mkimg.tcl` (make-zipfs drivers), and the - suite-built-shell step scripts `test_gate.tcl` / `suite_smoke.tcl` / - `pkg_smoke.tcl` (G-102). `test_gate.tcl` is the shared testsuite engine - (G-107): gate/record modes, driver selection, evidence summaries. + `zipfs_mkzip.tcl` / `zipfs_mkimg.tcl` (make-zipfs + kit-family wrap drivers), + and the suite-built-shell step scripts `test_gate.tcl` / `suite_smoke.tcl` / + `pkg_smoke.tcl` (G-102) plus `family_check.tcl` / `family_artifacts.tcl` + (G-103: kit-family self-containment verification; punkbin-layout artifact + + metadata emission). `test_gate.tcl` is the shared testsuite engine (G-107): + gate/record modes, driver selection, evidence summaries. - `expected_test_failures.txt` (tcl core) / `expected_test_failures_thread.txt` / `expected_test_failures_tclvfs.txt` - tracked dispositioned gate baselines. - `patches/` - recovered 2024 experiment patches, including TCLSH_PIPEREPL diff --git a/src/buildsuites/suite_tcl90/build905.zig b/src/buildsuites/suite_tcl90/build905.zig index 707fe0c6..c8dc930c 100644 --- a/src/buildsuites/suite_tcl90/build905.zig +++ b/src/buildsuites/suite_tcl90/build905.zig @@ -16,6 +16,7 @@ const tclvfs_shared = @import("build_tclvfs/build_tclvfs_shared.zig"); const build_tclthread = @import("build_tclthread/build_tclthread.zig").build_tclthread; const build_tk = @import("build_tk/build_tk.zig").build_tk; //G-098 +const TkBuild = @import("build_tk/build_tk.zig").TkBuild; //G-103 const tcl_major_version = "9"; const tcl_nodot_version = "90"; //VER in makefile @@ -1953,12 +1954,25 @@ pub fn build(b: *std.Build) !void { }); install_libraries.dependOn(&cookiejar_install.step); - const tm_http_install = b.addInstallFileWithDir(b.path(tcl_source_folder ++ "/library/http/http.tcl"), .prefix, b.pathJoin(&.{ module_install_dir_rel, tcl_dot_version, "http-2.10.2.tm" })); //(G-102: the former decorative 'echo --Installing package X--' banner steps were //removed - 'echo' is a cmd builtin, and spawning it as a program only worked when //a coreutils echo.exe happened to be on PATH; in a scrubbed environment the five //banner spawns failed and took install-libraries down transitively.) - install_libraries.dependOn(&tm_http_install.step); + //tm modules under the module path (lib/tcl9/9.0). Version-stamped names follow + //the checkout's library tree; single source for the prefix install AND the + //G-103 family staging (a kit's tm root is /app/tcl9/ - tm.tcl anchors at + //the dirname of the attached tcl_library). + const tm_installs = [_]struct { src: []const u8, tm: []const u8 }{ + .{ .src = "/library/http/http.tcl", .tm = "http-2.10.2.tm" }, + .{ .src = "/library/msgcat/msgcat.tcl", .tm = "msgcat-1.7.1.tm" }, + .{ .src = "/library/tcltest/tcltest.tcl", .tm = "tcltest-2.5.11.tm" }, + .{ .src = "/library/platform/platform.tcl", .tm = "platform-1.1.1.tm" }, + .{ .src = "/library/platform/shell.tcl", .tm = "platform/shell-1.1.4.tm" }, + }; + inline for (tm_installs) |tmi| { + const tm_inst = b.addInstallFileWithDir(b.path(tcl_source_folder ++ tmi.src), .prefix, b.pathJoin(&.{ module_install_dir_rel, tcl_dot_version, tmi.tm })); + install_libraries.dependOn(&tm_inst.step); + } //opt/*.tcl const opt_tcl_install = b.addInstallDirectory(.{ @@ -1980,18 +1994,6 @@ pub fn build(b: *std.Build) !void { const pkgindex_install = b.addInstallFile(b.path(tcl_source_folder ++ "/library/manifest.txt"), b.pathJoin(&.{ script_install_dir_rel, "pkgIndex.tcl" })); install_libraries.dependOn(&pkgindex_install.step); - const tm_msgcat_install = b.addInstallFileWithDir(b.path(tcl_source_folder ++ "/library/msgcat/msgcat.tcl"), .prefix, b.pathJoin(&.{ module_install_dir_rel, tcl_dot_version, "msgcat-1.7.1.tm" })); - install_libraries.dependOn(&tm_msgcat_install.step); - - const tm_tcltest_install = b.addInstallFileWithDir(b.path(tcl_source_folder ++ "/library/tcltest/tcltest.tcl"), .prefix, b.pathJoin(&.{ module_install_dir_rel, tcl_dot_version, "tcltest-2.5.11.tm" })); - install_libraries.dependOn(&tm_tcltest_install.step); - - const tm_platform_install = b.addInstallFileWithDir(b.path(tcl_source_folder ++ "/library/platform/platform.tcl"), .prefix, b.pathJoin(&.{ module_install_dir_rel, tcl_dot_version, "platform-1.1.1.tm" })); - install_libraries.dependOn(&tm_platform_install.step); - - const tm_shell_install = b.addInstallFileWithDir(b.path(tcl_source_folder ++ "/library/platform/shell.tcl"), .prefix, b.pathJoin(&.{ module_install_dir_rel, tcl_dot_version, "platform", "shell-1.1.4.tm" })); - install_libraries.dependOn(&tm_shell_install.step); - const encodings_install = b.addInstallDirectory(.{ .source_dir = b.path(tcl_source_folder ++ "/library/encoding"), .include_extensions = &.{".enc"}, @@ -2175,16 +2177,18 @@ pub fn build(b: *std.Build) !void { //**************************** - const thread_lib_compile = try build_tclthread(tcl_source_folder, "../tclthread", b, target, optimize, finalstublib); + const thread_build = try build_tclthread(tcl_source_folder, "../tclthread", b, target, optimize, finalstublib); const build_tclthread_step = b.step("build-tclthread", "build tcl thread library"); - build_tclthread_step.dependOn(&thread_lib_compile.step); + build_tclthread_step.dependOn(&thread_build.lib.step); b.getInstallStep().dependOn(build_tclthread_step); - //G-098: Tk loadable extension (windows target only for now) + //G-098: Tk loadable extension (windows target only for now). The build facts + //are hoisted (optional) so the G-103 bi-family staging can consume them. + var tk_build: ?TkBuild = null; if (target.result.os.tag == .windows) { - const tk_lib_compile = try build_tk(tcl_source_folder, "../tk9", b, target, optimize, finalstublib); + tk_build = try build_tk(tcl_source_folder, "../tk9", b, target, optimize, finalstublib); const build_tk_step = b.step("build-tk", "build tk shared library (tcl9tk90.dll) + tk script library install"); - build_tk_step.dependOn(&tk_lib_compile.step); + build_tk_step.dependOn(&tk_build.?.lib.step); } //if !ZIPFS_BUILD @@ -2327,9 +2331,153 @@ pub fn build(b: *std.Build) !void { tcllibc_smoke.step.dependOn(b.getInstallStep()); tcllibc_step.dependOn(&tcllibc_smoke.step); + const testreports_dir = common.replaceAll(b, b.pathJoin(&.{ b.install_path, "testreports" }), "\\", "/"); + + // ================== G-103 runtime kit family ================== + //PLAIN (stock shell + batteries), PUNK (piperepl-patched shell, gate + //default-ON with TCLSH_PIPEREPL=0 opt-out, same batteries), PUNK-BI + //(punk + the libraries we build: Tk dll + script lib, tklib). Working + //names carry the dotted tcl patchlevel with punk-marked patched shells + //(tclsh9.0.5.exe / tclsh9.0.5-punk.exe / tclsh9.0.5-punk-bi.exe - G-103 + //naming decision); -r artifact copies are kit-family-artifacts' job. + // + //Attached-image layout (the make.tcl kit contract: what a future kit + //build extracts and merges under its .vfs payload): + // tcl_library/ core script library (the C-level zipfs boot looks only + // for /app/main.tcl and /app/tcl_library; no main.tcl - + // stock fallthrough to the interactive shell) + // tcl9// tm modules (tm.tcl anchors at dirname of tcl_library) + // lib// batteries with their installed-shape pkgIndexes + // ($dir/../../bin dll references resolve to /app/bin) + // bin/ battery dlls (loaded from zipfs via copy-to-temp - + // the characterized TIP-741 cleanup wart, goal Notes) + // lib/pkgIndex.tcl one-line stock hook joining lib/ to the package + // search: auto_path is [tcl_library, /app] and + // tclPkgUnknown re-scans auto_path growth mid-scan (the + // same mechanism tcllib's own top-level index uses) + if (target.result.os.tag == .windows) { + const family_step = b.step("kit-family", "assemble + verify the G-103 runtime kit family (plain/punk/punk-bi): self-contained batteries-attached shells -> /family"); + const family_libext_pkgindex = + \\#punkshell runtime kit family (G-103): stock package-index hook extending + \\#the package search into the attached image's lib/ tree (tclPkgUnknown + \\#re-scans auto_path growth mid-scan - the same mechanism tcllib's own + \\#top-level pkgIndex uses). The batteries keep their installed-shape + \\#indexes ($dir/../../bin dll references resolve to /app/bin). + \\if {$dir ni $::auto_path} {lappend ::auto_path $dir} + \\ + ; + const tkb = tk_build.?; //windows target (checked just above) + var family_tree_base: [2]std.Build.LazyPath = undefined; + for (0..2) |ti| { //0 = core payload (plain+punk), 1 = bi payload + const famwf = b.addWriteFiles(); + //core script library at the boot position (tzdata/encoding ride + //inside the source library tree), dde/registry dlls inside + const fam_tcl_library = famwf.addCopyDirectory(b.path(tcl_source_folder ++ "/library"), "base/tcl_library", .{}); + _ = famwf.addCopyFile(b.path(tcl_source_folder ++ "/library/manifest.txt"), "base/tcl_library/pkgIndex.tcl"); + _ = famwf.addCopyFile(install_dde_dll.artifact.getEmittedBin(), "base/tcl_library/dde/" ++ dde_dll_file); + _ = famwf.addCopyFile(reg_dll.getEmittedBin(), "base/tcl_library/registry/" ++ reg_dll_file); + //tm modules (same set the prefix install carries) + inline for (tm_installs) |tmi| { + _ = famwf.addCopyFile(b.path(tcl_source_folder ++ tmi.src), "base/tcl9/" ++ tcl_dot_version ++ "/" ++ tmi.tm); + } + //package-search hook for the lib/ tree + _ = famwf.add("base/lib/pkgIndex.tcl", family_libext_pkgindex); + //batteries (installed shape) + try tclvfs_shared.familyadd_tclvfs_package("../tclvfs", b, famwf, "base/lib/vfs" ++ tclvfs_shared.dotversion, tclvfs_compile); + _ = famwf.addCopyFile(thread_build.pkgidx, b.fmt("base/lib/thread{s}/pkgIndex.tcl", .{thread_build.version})); + _ = famwf.addCopyFile(thread_build.lib.getEmittedBin(), b.fmt("base/bin/{s}", .{thread_build.dll_file})); + _ = famwf.addCopyDirectory(tcllib_out, b.fmt("base/lib/tcllib{s}", .{tcllib_ver}), .{}); + _ = famwf.addCopyDirectory(critcl_libout, "base/lib", .{}); //carries tcllibc/ + if (ti == 1) { + _ = famwf.addCopyDirectory(b.path("../tk9/library"), b.fmt("base/lib/{s}", .{tkb.libdir}), .{}); + _ = famwf.addCopyFile(tkb.pkgidx, b.fmt("base/lib/{s}/pkgIndex.tcl", .{tkb.libdir})); + _ = famwf.addCopyFile(tkb.lib.getEmittedBin(), b.fmt("base/bin/{s}", .{tkb.dll_file})); + _ = famwf.addCopyDirectory(tklib_out, b.fmt("base/lib/tklib{s}", .{tklib_ver}), .{}); + } + family_tree_base[ti] = fam_tcl_library.dirname(); + } + + const FamilyKit = struct { + variant: []const u8, + working_name: []const u8, + prefix_exe: *std.Build.Step.Compile, + tree: usize, + }; + const family_kits = [_]FamilyKit{ + .{ .variant = "plain", .working_name = b.fmt("tclsh{s}.exe", .{tcl_h_patchlevel}), .prefix_exe = tclsh_exe, .tree = 0 }, + .{ .variant = "punk", .working_name = b.fmt("tclsh{s}-punk.exe", .{tcl_h_patchlevel}), .prefix_exe = tclsh_pr_exe, .tree = 0 }, + .{ .variant = "punk-bi", .working_name = b.fmt("tclsh{s}-punk-bi.exe", .{tcl_h_patchlevel}), .prefix_exe = tclsh_pr_exe, .tree = 1 }, + }; + var family_checks: [family_kits.len]*std.Build.Step = undefined; + var family_installed: [family_kits.len][]const u8 = undefined; + for (family_kits, 0..) |fk, ki| { + const wrap = b.addRunArtifact(tclsh_exe); + common.scrubTclEnv(wrap); + wrap.setEnvironmentVariable("TCL_LIBRARY", tcl_library_src); + wrap.addFileArg(b.path("tools/zipfs_mkimg.tcl")); + wrap.addArg("-outfile"); + const wrapped = wrap.addOutputFileArg(fk.working_name); + wrap.addArg("-indir"); + wrap.addDirectoryArg(family_tree_base[fk.tree]); + wrap.addArg("-strip"); + wrap.addDirectoryArg(family_tree_base[fk.tree]); + wrap.addArg("-infile"); + wrap.addArtifactArg(fk.prefix_exe); + const inst = b.addInstallFileWithDir(wrapped, .prefix, b.fmt("family/{s}", .{fk.working_name})); + inst.step.dependOn(&wrap.step); + const installed_path = common.replaceAll(b, b.pathJoin(&.{ b.install_path, "family", fk.working_name }), "\\", "/"); + family_installed[ki] = installed_path; + //self-containment verification: the tool copies the kit alone into + //a scratch dir and probes from there with a scrubbed environment + //(no external Tcl visible - see tools/family_check.tcl) + const check = b.addRunArtifact(tclsh_exe); + common.scrubTclEnv(check); + check.setEnvironmentVariable("TCL_LIBRARY", tcl_library_src); + check.has_side_effects = true; + check.addFileArg(b.path("tools/family_check.tcl")); + check.addArgs(&.{ "-exe", installed_path, "-variant", fk.variant, "-expectpatch", tcl_h_patchlevel }); + check.addArgs(&.{ "-thread", thread_build.version, "-vfs", tclvfs_shared.dotversion, "-tcllib", tcllib_ver }); + if (std.mem.eql(u8, fk.variant, "punk-bi")) { + check.addArgs(&.{ "-tk", tkb.patchlevel, "-tklib", tklib_ver }); + } + check.step.dependOn(&inst.step); + family_checks[ki] = &check.step; + family_step.dependOn(&check.step); + } + + //-- kit-family-artifacts: punkbin-layout emission (-r immutable + //artifact names + per-artifact toml metadata + sha1sums), run UNDER + //THE PLAIN FAMILY KIT itself - every emission doubles as a proof the + //family runtime executes real tooling from its attached batteries. + //Depends on the checks: only verified kits get artifact records. + //Publication to the real punkbin repo stays a deliberate user step. + const familyrev = b.option(u32, "familyrev", "assembly revision N for the -r family artifact names (kit-family-artifacts; default 1)") orelse 1; + const family_artifacts_step = b.step("kit-family-artifacts", "emit punkbin-layout artifact copies (-r) + toml metadata + sha1sums for the verified family kits -> /family/punkbin/"); + const artifacts_outdir = common.replaceAll(b, b.pathJoin(&.{ b.install_path, "family", "punkbin", "win32-x86_64" }), "\\", "/"); + const uuid_tcl = common.manifestUuid(b, tcl_source_folder); + const uuid_thread = common.manifestUuid(b, "../tclthread"); + const uuid_tclvfs = common.manifestUuid(b, "../tclvfs"); + const uuid_tk = common.manifestUuid(b, "../tk9"); + const uuid_tcllib = common.manifestUuid(b, "../tcllib"); + const uuid_tklib = common.manifestUuid(b, "../tklib"); + const emit = b.addSystemCommand(&.{family_installed[0]}); + common.scrubTclEnv(emit); + emit.has_side_effects = true; + emit.addFileArg(b.path("tools/family_artifacts.tcl")); + emit.addArgs(&.{ "-outdir", artifacts_outdir, "-rev", b.fmt("{d}", .{familyrev}), "-target", "win32-x86_64" }); + emit.addArgs(&.{ "-suite", "suite_tcl90", "-tclpatch", tcl_h_patchlevel, "-zig", builtin.zig_version_string, "-optimize", @tagName(optimize) }); + emit.addArgs(&.{ "-components", b.fmt("Thread {s} vfs {s} tcllib {s} tcllibc {s}", .{ thread_build.version, tclvfs_shared.dotversion, tcllib_ver, tcllib_ver }) }); + emit.addArgs(&.{ "-bicomponents", b.fmt("Tk {s} tklib {s}", .{ tkb.patchlevel, tklib_ver }) }); + emit.addArgs(&.{ "-provenance", b.fmt("tcl {s} thread {s} tclvfs {s} tk {s} tcllib {s} tklib {s}", .{ uuid_tcl, uuid_thread, uuid_tclvfs, uuid_tk, uuid_tcllib, uuid_tklib }) }); + emit.addArgs(&.{ "-testreports", testreports_dir }); + emit.addArgs(&.{ "-kits", b.fmt("plain {{{s}}} punk {{{s}}} punk-bi {{{s}}}", .{ family_installed[0], family_installed[1], family_installed[2] }) }); + for (family_checks) |cs| emit.step.dependOn(cs); + family_artifacts_step.dependOn(&emit.step); + } + // ================== + //-- test-gate: the core testsuite under the built shell, gated on parsed //totals vs the tracked dispositioned baseline (all.tcl's exit code lies) - const testreports_dir = common.replaceAll(b, b.pathJoin(&.{ b.install_path, "testreports" }), "\\", "/"); const gate_step = b.step("test-gate", "tcl core testsuite under the built shell, gated on parsed totals vs expected_test_failures.txt"); const testargs_opt = b.option([]const u8, "testargs", "extra tcltest args for test-gate (e.g -file http.test)") orelse ""; const gate_run = b.addRunArtifact(tclsh_exe); @@ -2547,7 +2695,7 @@ fn bootstrapMode(b: *std.Build) !void { //suite.tcl performs after its fossil staging. Per-zig-version cache dir (object //caches must never be shared across zig versions - user-confirmed hazard), same //name suite.tcl derives so both flows share the staged cache. - const steps_opt = b.option([]const []const u8, "steps", "steps for the staged build (repeat the flag; default: install install-libraries make-zipfs smoke tklib tcllib tcllibc - the full pipeline short of test-gate)") orelse @as([]const []const u8, &.{ "install", "install-libraries", "make-zipfs", "smoke", "tklib", "tcllib", "tcllibc" }); + const steps_opt = b.option([]const []const u8, "steps", "steps for the staged build (repeat the flag; default: install install-libraries make-zipfs smoke tklib tcllib tcllibc kit-family kit-family-artifacts - the full pipeline short of test-gate)") orelse @as([]const []const u8, &.{ "install", "install-libraries", "make-zipfs", "smoke", "tklib", "tcllib", "tcllibc", "kit-family", "kit-family-artifacts" }); var cachever: []const u8 = builtin.zig_version_string; cachever = common.replaceAll(b, cachever, "+", "_"); cachever = common.replaceAll(b, cachever, "/", "_"); diff --git a/src/buildsuites/suite_tcl90/build_common.zig b/src/buildsuites/suite_tcl90/build_common.zig index 8ee66331..b7cd6044 100644 --- a/src/buildsuites/suite_tcl90/build_common.zig +++ b/src/buildsuites/suite_tcl90/build_common.zig @@ -53,6 +53,16 @@ pub fn scrubTclEnv(run: *std.Build.Step.Run) void { } } +//checkout uuid from a tree's manifest.uuid when the repo materializes it (a +//per-repo fossil 'manifest' setting: tcl/tk/thread enable it, tclvfs/tcllib/ +//tklib do not - true for live checkouts AND fossil tarball exports alike), else +//"unrecorded" so provenance emission stays total (G-103 artifact metadata). +pub fn manifestUuid(b: *std.Build, tree_from_root: []const u8) []const u8 { + const abs = b.pathFromRoot(b.fmt("{s}/manifest.uuid", .{tree_from_root})); + const data = std.Io.Dir.cwd().readFileAlloc(b.graph.io, abs, b.allocator, .limited(4096)) catch return "unrecorded"; + return std.mem.trim(u8, data, " \t\r\n"); +} + //package version from a TEA tree's configure.ac 'AC_INIT([name],[version])' //line (thread, tclvfs, ...). G-107: version-derived artifact names must follow //the checkout - hardcoded versions stamp mismatched sources (the thread recipe diff --git a/src/buildsuites/suite_tcl90/build_tclthread/build_tclthread.zig b/src/buildsuites/suite_tcl90/build_tclthread/build_tclthread.zig index 50324660..99fc82e0 100644 --- a/src/buildsuites/suite_tcl90/build_tclthread/build_tclthread.zig +++ b/src/buildsuites/suite_tcl90/build_tclthread/build_tclthread.zig @@ -7,7 +7,17 @@ const common = @import("../build_common.zig"); // declaratively construct a build graph that will be executed by an external // runner. //pub fn build(b: *std.Build) !void {} -pub fn build_tclthread(comptime tcldir: []const u8, comptime subdir: []const u8, b: *std.Build, target: std.Build.ResolvedTarget, optimize: std.builtin.OptimizeMode, stublib: *std.Build.Step.Compile) !*std.Build.Step.Compile { + +//G-103: the recipe's kit-family staging consumes the same generated pkgIndex and +//version-derived names as the prefix install - returned alongside the compile. +pub const TclThreadBuild = struct { + lib: *std.Build.Step.Compile, + pkgidx: std.Build.LazyPath, + version: []const u8, //e.g "3.0.7" (from the checkout's AC_INIT) + dll_file: []const u8, //e.g "tcl9thread307.dll" +}; + +pub fn build_tclthread(comptime tcldir: []const u8, comptime subdir: []const u8, b: *std.Build, target: std.Build.ResolvedTarget, optimize: std.builtin.OptimizeMode, stublib: *std.Build.Step.Compile) !TclThreadBuild { //Version derived from the checkout's AC_INIT (G-107): the recipe formerly //hardcoded 3.0.1, compiling newer trunk sources with a stale PACKAGE_VERSION - //the dll then self-reported the wrong version and the thread testsuite's @@ -186,5 +196,5 @@ pub fn build_tclthread(comptime tcldir: []const u8, comptime subdir: []const u8, const pkgidx_install = b.addInstallFileWithDir(pkgidx, .prefix, b.fmt("lib/thread{s}/pkgIndex.tcl", .{thread_version})); b.getInstallStep().dependOn(&pkgidx_install.step); - return lib; + return .{ .lib = lib, .pkgidx = pkgidx, .version = thread_version, .dll_file = pkg_lib_file }; } diff --git a/src/buildsuites/suite_tcl90/build_tclvfs/build_tclvfs_shared.zig b/src/buildsuites/suite_tcl90/build_tclvfs/build_tclvfs_shared.zig index 2fd799d5..2cbc52ba 100644 --- a/src/buildsuites/suite_tcl90/build_tclvfs/build_tclvfs_shared.zig +++ b/src/buildsuites/suite_tcl90/build_tclvfs/build_tclvfs_shared.zig @@ -4,12 +4,24 @@ const common = @import("../build_common.zig"); const tclvfs_dotversion = "1.4.2"; const tclvfs_nodotversion = "142"; +pub const dotversion = tclvfs_dotversion; +pub const dll_file = "tcl9vfs" ++ tclvfs_nodotversion ++ ".dll"; -//G-102: install the tclvfs script package (lib/vfs) into the prefix - the two //configure-products (vfs.tcl, pkgIndex.tcl) generated from their .in templates at -//configure time, the library scripts + template dir copied, and the dll alongside -//(vfs.tcl loads it from its own dir via ::vfs::self). Formerly a suite.tcl -//post-build block. Windows dll naming - the cross-target story is G-105's. +//configure time. Shared by the prefix install and the G-103 family staging so the +//package is generated from one derivation. +fn configuredContent(b: *std.Build, comptime subdir: []const u8, comptime template: []const u8) []const u8 { + var t: []const u8 = common.readSourceFile(b, subdir ++ "/" ++ template); + t = common.replaceAll(b, t, "@PACKAGE_VERSION@", tclvfs_dotversion); + t = common.replaceAll(b, t, "@PKG_LIB_FILE9@", dll_file); + t = common.replaceAll(b, t, "@PKG_LIB_FILE8@", "tclvfs" ++ tclvfs_nodotversion ++ ".dll"); + return t; +} + +//G-102: install the tclvfs script package (lib/vfs) into the prefix - the two +//generated configure-products, the library scripts + template dir copied, and the +//dll alongside (vfs.tcl loads it from its own dir via ::vfs::self). Formerly a +//suite.tcl post-build block. Windows dll naming - the cross-target story is G-105's. pub fn install_tclvfs_package(comptime subdir: []const u8, b: *std.Build, tclvfs_compile: *std.Build.Step.Compile) !void { const pkgsub = "lib/vfs" ++ tclvfs_dotversion; const install_scripts = b.addInstallDirectory(.{ @@ -31,18 +43,25 @@ pub fn install_tclvfs_package(comptime subdir: []const u8, b: *std.Build, tclvfs .{ "library/vfs.tcl.in", "vfs.tcl" }, .{ "pkgIndex.tcl.in", "pkgIndex.tcl" }, }) |pair| { - var t: []const u8 = common.readSourceFile(b, subdir ++ "/" ++ pair[0]); - t = common.replaceAll(b, t, "@PACKAGE_VERSION@", tclvfs_dotversion); - t = common.replaceAll(b, t, "@PKG_LIB_FILE9@", "tcl9vfs" ++ tclvfs_nodotversion ++ ".dll"); - t = common.replaceAll(b, t, "@PKG_LIB_FILE8@", "tclvfs" ++ tclvfs_nodotversion ++ ".dll"); - const gen = wf.add(pair[1], t); + const gen = wf.add(pair[1], configuredContent(b, subdir, pair[0])); const inst = b.addInstallFileWithDir(gen, .prefix, pkgsub ++ "/" ++ pair[1]); b.getInstallStep().dependOn(&inst.step); } - const dll_inst = b.addInstallFileWithDir(tclvfs_compile.getEmittedBin(), .prefix, pkgsub ++ "/tcl9vfs" ++ tclvfs_nodotversion ++ ".dll"); + const dll_inst = b.addInstallFileWithDir(tclvfs_compile.getEmittedBin(), .prefix, pkgsub ++ "/" ++ dll_file); b.getInstallStep().dependOn(&dll_inst.step); } +//G-103: write the SAME installed-shape vfs package into a family staging tree +//(WriteFiles) at destsub (e.g "base/lib/vfs1.4.2") - scripts, template dir, +//generated vfs.tcl/pkgIndex.tcl, dll inside the package dir. +pub fn familyadd_tclvfs_package(comptime subdir: []const u8, b: *std.Build, wf: *std.Build.Step.WriteFile, comptime destsub: []const u8, tclvfs_compile: *std.Build.Step.Compile) !void { + _ = wf.addCopyDirectory(b.path(subdir ++ "/library"), destsub, .{ .include_extensions = &.{".tcl"} }); + _ = wf.addCopyDirectory(b.path(subdir ++ "/library/template"), destsub ++ "/template", .{}); + _ = wf.add(destsub ++ "/vfs.tcl", configuredContent(b, subdir, "library/vfs.tcl.in")); + _ = wf.add(destsub ++ "/pkgIndex.tcl", configuredContent(b, subdir, "pkgIndex.tcl.in")); + _ = wf.addCopyFile(tclvfs_compile.getEmittedBin(), destsub ++ "/" ++ dll_file); +} + pub fn vfsadd_tclvfs_files(comptime tcldir: []const u8, comptime subdir: []const u8, b: *std.Build, vfswrite: *std.Build.Step.WriteFile) !*std.Build.Step.WriteFile { //_ = tcldir; //const version = "1.4.2"; diff --git a/src/buildsuites/suite_tcl90/build_tk/build_tk.zig b/src/buildsuites/suite_tcl90/build_tk/build_tk.zig index 4c573a62..6670d8f7 100644 --- a/src/buildsuites/suite_tcl90/build_tk/build_tk.zig +++ b/src/buildsuites/suite_tcl90/build_tk/build_tk.zig @@ -48,7 +48,17 @@ const ttk_names = [_][]const u8{ "ttkTreeview", "ttkWidget", "ttkStubInit", }; -pub fn build_tk(comptime tcldir: []const u8, comptime subdir: []const u8, b: *std.Build, target: std.Build.ResolvedTarget, optimize: std.builtin.OptimizeMode, stublib: *std.Build.Step.Compile) !*std.Build.Step.Compile { +//G-103: the recipe's kit-family staging consumes the same generated pkgIndex, +//script-library source and version facts as the prefix install. +pub const TkBuild = struct { + lib: *std.Build.Step.Compile, + pkgidx: std.Build.LazyPath, + patchlevel: []const u8, //e.g "9.0.2" (tk.h TK_PATCH_LEVEL) + libdir: []const u8, //install dir name, e.g "tk9.0" + dll_file: []const u8, //e.g "tcl9tk90.dll" +}; + +pub fn build_tk(comptime tcldir: []const u8, comptime subdir: []const u8, b: *std.Build, target: std.Build.ResolvedTarget, optimize: std.builtin.OptimizeMode, stublib: *std.Build.Step.Compile) !TkBuild { const lib = b.addLibrary(.{ .linkage = .dynamic, .name = "tcl9tk90", @@ -200,5 +210,5 @@ pub fn build_tk(comptime tcldir: []const u8, comptime subdir: []const u8, b: *st const pkgidx_install = b.addInstallFileWithDir(pkgidx, .prefix, "lib/tk9.0/pkgIndex.tcl"); b.getInstallStep().dependOn(&pkgidx_install.step); - return lib; + return .{ .lib = lib, .pkgidx = pkgidx, .patchlevel = tk_h_patchlevel, .libdir = "tk9.0", .dll_file = "tcl9tk90.dll" }; } diff --git a/src/buildsuites/suite_tcl90/suite.tcl b/src/buildsuites/suite_tcl90/suite.tcl index e6144e81..7a86e45c 100644 --- a/src/buildsuites/suite_tcl90/suite.tcl +++ b/src/buildsuites/suite_tcl90/suite.tcl @@ -22,8 +22,13 @@ # -seedfossils read-only seed collection for missing clones (default ~/.fossils; "" disables) # -all <0|1> with 'clean': reserved (the clone store lives outside the stage) # -steps zig build steps. 'build' default: {install install-libraries -# make-zipfs smoke tklib tcllib tcllibc} - the full pipeline -# short of the test steps. 'test' default: {test-gate} (the +# make-zipfs smoke tklib tcllib tcllibc kit-family +# kit-family-artifacts} - the full pipeline short of the test +# steps, ending in the G-103 runtime kit family (verified +# self-contained plain/punk/punk-bi shells -> out/family plus +# punkbin-layout -r artifacts + toml metadata; artifact +# revision via -zigargs {-Dfamilyrev=N}). +# 'test' default: {test-gate} (the # core suite gate); other test steps (G-107 library runs): # test-libraries (thread+tclvfs gated, tcllib+tklib recorded), # test-thread test-tclvfs test-tcllib test-tklib, and the @@ -76,7 +81,7 @@ array set opt { -optimize ReleaseFast -tclbranch {} -refresh 0 - -steps {install install-libraries make-zipfs smoke tklib tcllib tcllibc} + -steps {install install-libraries make-zipfs smoke tklib tcllib tcllibc kit-family kit-family-artifacts} -testargs {} -zigargs {} -repofolder {} diff --git a/src/buildsuites/suite_tcl90/tools/family_artifacts.tcl b/src/buildsuites/suite_tcl90/tools/family_artifacts.tcl new file mode 100644 index 00000000..1b7d94c2 --- /dev/null +++ b/src/buildsuites/suite_tcl90/tools/family_artifacts.tcl @@ -0,0 +1,145 @@ +#family_artifacts.tcl (G-103): emit punkbin-layout artifact copies + per-artifact +#toml metadata + sha1sums for the verified runtime kit family members. +# +#Deliberately run UNDER THE PLAIN FAMILY KIT itself (not the builder shell): the +#sha1 digests come from the kit's attached tcllib/tcllibc, so every emission run +#doubles as a proof that the family runtime executes real tooling self-contained. +# +#Artifact tier naming (G-103 naming decision): -r.exe - +#immutable punkbin names; the -r assembly revision comes from the invocation +#(-rev, suite option -Dfamilyrev). The emitted tree mirrors a punkbin platform +#folder: /.exe + .toml + sha1sums.txt (punkbin +#format: ' *'). Publication to the real punkbin repo is a +#deliberate user step (copy + build_sha1sums.tcl there); it is DEFERRED per the +#goal notes until the family shape is accepted. +# +#args: -outdir -rev -target +# -suite -tclpatch -zig -optimize +# -components { ...} (attached battery versions, all variants) +# -bicomponents { ...} (additional bi-only batteries) +# -provenance { ...} (source checkout uuids) +# -testreports (G-107 evidence summaries; optional) +# -kits { ...} + +proc fail {msg} {puts stderr "family_artifacts FAIL: $msg"; flush stderr; exit 1} +proc note {msg} {puts stdout "family_artifacts: $msg"; flush stdout} + +array set opt { + -outdir {} -rev 1 -target {} -suite {} -tclpatch {} -zig {} -optimize {} + -components {} -bicomponents {} -provenance {} -testreports {} -kits {} +} +foreach {k v} $argv { + if {![info exists opt($k)]} {fail "unknown option '$k'"} + set opt($k) $v +} +foreach req {-outdir -target -suite -tclpatch -zig -optimize -components -kits} { + if {$opt($req) eq ""} {fail "missing required option $req"} +} +if {![string is integer -strict $opt(-rev)] || $opt(-rev) < 1} {fail "-rev must be a positive integer"} + +if {[catch {package require sha1} sha1ver]} { + fail "package require sha1 failed under [info nameofexecutable] - the family kit must carry tcllib: $sha1ver" +} + +proc toml_str {s} { + #basic toml string: escape backslash and double-quote (values here are names, + #versions, uuids, iso dates - no control chars expected) + return "\"[string map {\\ \\\\ \" \\\"} $s]\"" +} + +file mkdir $opt(-outdir) +set built [clock format [clock seconds] -format %Y-%m-%dT%H:%M:%SZ -timezone :UTC] +set sha1lines {} +set emitted {} + +foreach {variant kitpath} $opt(-kits) { + set kitpath [file normalize $kitpath] + if {![file exists $kitpath]} {fail "kit exe not found: $kitpath"} + set working [file tail $kitpath] + set artifact "[file rootname $working]-r$opt(-rev)[file extension $working]" + set dest [file join $opt(-outdir) $artifact] + file delete -force $dest + file copy $kitpath $dest + set sha1 [sha1::sha1 -hex -file $dest] + set size [file size $dest] + lappend sha1lines "$sha1 *$artifact" + + set components $opt(-components) + if {$variant eq "punk-bi"} {lappend components {*}$opt(-bicomponents)} + set batteries {} + foreach {n v} $components {lappend batteries [toml_str "$n $v"]} + + set m {} + lappend m "#punkshell runtime artifact metadata (G-103) - generated by family_artifacts.tcl" + lappend m "\[artifact\]" + lappend m "name = [toml_str $artifact]" + lappend m "class = \"runtime\"" + lappend m "variant = [toml_str $variant]" + lappend m "working_name = [toml_str $working]" + lappend m "revision = $opt(-rev)" + lappend m "target = [toml_str $opt(-target)]" + lappend m "sha1 = [toml_str $sha1]" + lappend m "size = $size" + lappend m "built = [toml_str $built]" + lappend m "" + lappend m "\[runtime\]" + lappend m "tcl_patchlevel = [toml_str $opt(-tclpatch)]" + set pr [expr {$variant ne "plain"}] + lappend m "piperepl = [expr {$pr ? "true" : "false"}]" + if {$pr} { + lappend m "piperepl_default = \"on\"" + lappend m "piperepl_opt_out = \"TCLSH_PIPEREPL=0\"" + } + lappend m "attached_batteries = \[[join $batteries {, }]\]" + lappend m "" + lappend m "\[provenance\]" + lappend m "suite = [toml_str $opt(-suite)]" + lappend m "toolchain = [toml_str "zig $opt(-zig)"]" + lappend m "optimize = [toml_str $opt(-optimize)]" + foreach {n uuid} $opt(-provenance) { + lappend m "${n}_checkout = [toml_str $uuid]" + } + #G-107 evidence summaries available at emission time (result lines only; the + #full line-record summaries stay the canonical evidence artifacts) + if {$opt(-testreports) ne "" && [file isdirectory $opt(-testreports)]} { + set tlines {} + foreach sf [lsort [glob -nocomplain -directory $opt(-testreports) *.summary]] { + set rec [dict create] + set f [open $sf r] + foreach line [split [read $f] \n] { + set line [string trim $line] + if {$line eq "" || [string index $line 0] eq "#"} continue + if {[catch {llength $line} n] || $n < 2} continue + dict set rec [lindex $line 0] [lrange $line 1 end] + } + close $f + if {![dict exists $rec library] || ![dict exists $rec result]} continue + set lib [dict get $rec library] + set parts [list "result=[dict get $rec result]"] + foreach fkey {mode total passed skipped failed} { + if {[dict exists $rec $fkey]} {lappend parts "$fkey=[dict get $rec $fkey]"} + } + lappend tlines "$lib = [toml_str [join $parts { }]]" + } + if {[llength $tlines]} { + lappend m "" + lappend m "\[tests\]" + lappend m {*}$tlines + } + } + set mf [file join $opt(-outdir) "[file rootname $artifact].toml"] + set f [open $mf w] + fconfigure $f -translation lf + puts $f [join $m \n] + close $f + lappend emitted "$variant -> $artifact" + note "emitted $artifact (sha1 $sha1, [expr {$size/1024}] KB) + [file tail $mf]" +} + +set f [open [file join $opt(-outdir) sha1sums.txt] w] +fconfigure $f -translation lf +puts $f [join $sha1lines \n] +close $f +note "sha1sums.txt written ([llength $sha1lines] artifacts)" +puts "family_artifacts OK: [join $emitted {; }] -> $opt(-outdir)" +exit 0 diff --git a/src/buildsuites/suite_tcl90/tools/family_check.tcl b/src/buildsuites/suite_tcl90/tools/family_check.tcl new file mode 100644 index 00000000..34f0b657 --- /dev/null +++ b/src/buildsuites/suite_tcl90/tools/family_check.tcl @@ -0,0 +1,201 @@ +#family_check.tcl (G-103): self-containment verification for one runtime kit +#family member (plain / punk / punk-bi). Run under the suite-built BUILDER shell; +#the KIT exe under test is exec'd as probe children. +# +#Why the copy-to-scratch: beside the installed out/family location sits ../lib +#(the suite prefix), which the stock tm-path setup (/../lib) and +#tcl_findLibrary's exe-relative entries would silently satisfy. The kit is copied +#alone into a fresh scratch dir and probed FROM THERE with cwd=scratch, so only +#the attached image can serve the probes ("no external Tcl visible"). +# +#The probe environment is scrubbed of every tcl redirection var (TCL_LIBRARY, +#TK_LIBRARY, TCLLIBPATH, VFS_LIBRARY, TCL*_TM_PATH); TCLSH_PIPEREPL is controlled +#per probe - it is the punk-vs-plain discriminator (G-096 behaviour matrix: with +#the gate open the patched shell publishes ::tclsh(istty) before the startup +#script runs; with TCLSH_PIPEREPL=0 or unpatched sources no ::tclsh machinery +#exists). Script-arg probes never set ::tclsh(dorepl), so no probe can land in +#the console-reopen repl (no hang risk). +# +#args: -exe -variant plain|punk|punk-bi -expectpatch +# -thread -vfs -tcllib ?-tk ? ?-tklib ? + +proc fail {msg} {puts stderr "family_check FAIL: $msg"; flush stderr; exit 1} +proc note {msg} {puts stdout "family_check: $msg"; flush stdout} + +array set opt {-exe {} -variant {} -expectpatch {} -thread {} -vfs {} -tcllib {} -tk {} -tklib {}} +foreach {k v} $argv { + if {![info exists opt($k)]} {fail "unknown option '$k'"} + set opt($k) $v +} +foreach req {-exe -variant -expectpatch -thread -vfs -tcllib} { + if {$opt($req) eq ""} {fail "missing required option $req"} +} +if {$opt(-variant) ni {plain punk punk-bi}} {fail "-variant must be plain|punk|punk-bi"} +if {$opt(-variant) eq "punk-bi" && ($opt(-tk) eq "" || $opt(-tklib) eq "")} { + fail "punk-bi variant requires -tk and -tklib expected versions" +} +set exe [file normalize $opt(-exe)] +if {![file exists $exe]} {fail "kit exe not found: $exe"} + +#-- scrub the probe environment (children inherit ::env at exec time) --------- +foreach ev [array names ::env] { + if {$ev in {TCL_LIBRARY TK_LIBRARY TCLLIBPATH VFS_LIBRARY TCLSH_PIPEREPL} + || [string match TCL*_TM_PATH $ev]} { + unset ::env($ev) + } +} + +#-- scratch dir with the kit copied in alone --------------------------------- +set scratch [file join [file tempdir] "punkfamilycheck_[pid]_[clock clicks -microseconds]"] +file mkdir $scratch +set kitname [file tail $exe] +set kit [file join $scratch $kitname] +file copy $exe $kit + +proc probe {name script args} { + #write the probe script into the scratch dir and run the kit on it with + #cwd=scratch; env pairs in $args are set for the child and removed after. + #returns the child output; fails hard on nonzero exit. + global scratch kit + set sf [file join $scratch probe_$name.tcl] + set f [open $sf w] + puts $f $script + close $f + foreach {ev val} $args {set ::env($ev) $val} + set rc [catch {exec $kit $sf 2>@1} out] + foreach {ev val} $args {unset ::env($ev)} + if {$rc} {fail "probe '$name' failed (kit $kit):\n$out"} + return $out +} +proc checks {output} { + #parse 'CHECK ' lines into a dict + set d [dict create] + foreach line [split $output \n] { + set line [string trim $line] + if {[string match "CHECK *" $line]} { + dict set d [lindex $line 1] [lindex $line 2] + } + } + return $d +} +proc asserteq {d key expect what} { + if {![dict exists $d $key]} {fail "$what: no CHECK line for '$key' (got: $d)"} + set got [dict get $d $key] + if {$got ne $expect} {fail "$what: $key = '$got', expected '$expect'"} +} + +#-- probe: core batteries (all variants) ------------------------------------- +#Also proves cwd independence: the probe chdirs into a subdir it creates, so +#relative-path leakage from the scratch dir itself would surface. +set core_script { + proc out {k v} {puts [list CHECK $k $v]} + out patchlevel [info patchlevel] + out tcl_library $::tcl_library + out library_zipfs [string match //zipfs:/app/* $::tcl_library] + #attached-library facts (suite_smoke parity) + out tz [expr {[catch {clock format 0 -gmt 0 -format %Z}] ? "FAIL" : "ok"}] + out encoding [expr {[catch {encoding convertto cp1250 test}] ? "FAIL" : "ok"}] + #tm path serves modules from the attached image (tcl9/ beside tcl_library) + out platform_tm [expr {[catch {package require platform} v] ? "FAIL:$v" : "ok"}] + #Thread: version + functional cross-thread eval + if {[catch {package require Thread} tver]} {out thread "FAIL:$tver"} else { + out thread $tver + set tid [thread::create] + out thread_eval [thread::send $tid {expr {6*7}}] + thread::release $tid + } + #tclvfs: version + representative vfs::* + functional zip mount round-trip + #(the kit's own zipfs mkzip makes the test archive - no external tools) + if {[catch {package require vfs} vver]} {out vfs "FAIL:$vver"} else { + out vfs $vver + out vfs_zip [expr {[catch {package require vfs::zip} zv] ? "FAIL:$zv" : $zv}] + out vfs_urltype [expr {[catch {package require vfs::urltype} uv] ? "FAIL:$uv" : $uv}] + set d [file join [pwd] vfstest] + file mkdir $d/payload + set f [open $d/payload/hello.txt w]; puts -nonewline $f "family-vfs-roundtrip"; close $f + cd $d + tcl::zipfs::mkzip probe.zip payload payload + set mnt [vfs::zip::Mount [file join $d probe.zip] zipmnt] + set f [open zipmnt/hello.txt r]; set data [read $f]; close $f + vfs::zip::Unmount $mnt zipmnt + out vfs_roundtrip [expr {$data eq "family-vfs-roundtrip" ? "ok" : "FAIL:$data"}] + } + #tcllib module + tcllibc acceleration engaged (pkg_smoke -accel parity) + if {[catch {package require md5} mver]} {out md5 "FAIL:$mver"} else { + out md5 $mver + out tcllibc [expr {[catch {package require tcllibc} cv] ? "FAIL:$cv" : $cv}] + upvar #0 ::md5::accel accelarr + out md5_accel [expr {[info exists accelarr(critcl)] && $accelarr(critcl) ? 1 : 0}] + } + exit 0 +} +set d [checks [probe core $core_script]] +asserteq $d patchlevel $opt(-expectpatch) "core" +asserteq $d library_zipfs 1 "core (tcl_library=[dict get $d tcl_library])" +asserteq $d tz ok "core" +asserteq $d encoding ok "core" +asserteq $d platform_tm ok "core" +asserteq $d thread $opt(-thread) "core" +asserteq $d thread_eval 42 "core" +asserteq $d vfs $opt(-vfs) "core" +asserteq $d vfs_roundtrip ok "core" +asserteq $d md5_accel 1 "core (tcllibc=[expr {[dict exists $d tcllibc]?[dict get $d tcllibc]:"?"}])" +foreach k {vfs_zip vfs_urltype tcllibc} { + if {[string match FAIL* [dict get $d $k]]} {fail "core: $k [dict get $d $k]"} +} +note "core OK ($opt(-variant)): tcl [dict get $d patchlevel] library [dict get $d tcl_library] thread [dict get $d thread] vfs [dict get $d vfs] (zip [dict get $d vfs_zip], urltype [dict get $d vfs_urltype]) md5 [dict get $d md5] accel=1" + +#-- probe: Tk create/destroy (bi only) --------------------------------------- +if {$opt(-variant) eq "punk-bi"} { + set tk_script { + proc out {k v} {puts [list CHECK $k $v]} + if {[catch {package require Tk} tkver]} {out tk "FAIL:$tkver"; exit 0} + out tk $tkver + wm withdraw . + button .b -text family + out tk_widget [winfo exists .b] + destroy .b + out tk_widget_destroyed [expr {![winfo exists .b]}] + destroy . + out tk_done ok + exit 0 + } + set d [checks [probe tk $tk_script]] + asserteq $d tk $opt(-tk) "tk" + asserteq $d tk_widget 1 "tk" + asserteq $d tk_widget_destroyed 1 "tk" + asserteq $d tk_done ok "tk" + #tklib rides only in the bi payload + set tklib_script { + proc out {k v} {puts [list CHECK $k $v]} + package require Tk + wm withdraw . + out tooltip [expr {[catch {package require tooltip} v] ? "FAIL:$v" : $v}] + destroy . + exit 0 + } + set d [checks [probe tklib $tklib_script]] + if {[string match FAIL* [dict get $d tooltip]]} {fail "tklib: tooltip [dict get $d tooltip]"} + note "tk OK: Tk $opt(-tk) create/destroy + tklib tooltip [dict get $d tooltip]" +} + +#-- probe: piperepl gate (variant discriminator) ------------------------------ +set gate_script {puts [list CHECK tclshmachinery [info exists ::tclsh(istty)]]; exit 0} +set d_default [checks [probe gate_default $gate_script]] +set d_optout [checks [probe gate_optout $gate_script TCLSH_PIPEREPL 0]] +if {$opt(-variant) eq "plain"} { + #unpatched: no machinery regardless of the env var (even explicitly enabled) + asserteq $d_default tclshmachinery 0 "piperepl (plain, env unset)" + set d_on [checks [probe gate_on $gate_script TCLSH_PIPEREPL 1]] + asserteq $d_on tclshmachinery 0 "piperepl (plain, TCLSH_PIPEREPL=1)" + note "piperepl OK (plain): stock behaviour - no ::tclsh machinery with env unset or =1" +} else { + #punk kits: ACTIVE BY DEFAULT (machinery published), documented opt-out =0 + asserteq $d_default tclshmachinery 1 "piperepl ($opt(-variant), env unset - default must be ON)" + asserteq $d_optout tclshmachinery 0 "piperepl ($opt(-variant), TCLSH_PIPEREPL=0 opt-out)" + note "piperepl OK ($opt(-variant)): active by default, disabled via TCLSH_PIPEREPL=0" +} + +file delete -force $scratch +puts "family_check OK: $opt(-variant) $kitname self-contained (scratch-dir probes, scrubbed env)" +exit 0 diff --git a/src/runtime/AGENTS.md b/src/runtime/AGENTS.md index 4cb29467..e882790c 100644 --- a/src/runtime/AGENTS.md +++ b/src/runtime/AGENTS.md @@ -7,12 +7,12 @@ Houses the `mapvfs.config` that maps VFS payloads to platform runtimes, plus the ## Ownership - Agents must not modify runtime executables or `mapvfs.config` unless explicitly asked. -- Runtime `.exe` files are platform binaries installed via `bin/runtime.cmd` and must not be edited. +- Runtime `.exe` files are platform binaries installed via `bin/punk-runtime.cmd` (renamed from `runtime.cmd` under G-097) and must not be edited. ## Local Contracts -- `mapvfs.config` defines which `src/vfs/*.vfs` folders combine with which runtime binaries. -- Runtime executables are placed here by the `bin/runtime.cmd` helper or manually. +- `mapvfs.config` defines which `src/vfs/*.vfs` folders combine with which runtime binaries (stored under `bin/runtime//`). +- Runtime executables are placed in `bin/runtime//` by the `bin/punk-runtime.cmd` helper, manually, or (G-103 family runtimes) copied from the suite_tcl90 `kit-family` build products under `src/buildsuites/_build/suite_tcl90/out/family/`. - The `_build/` subdirectory holds build intermediates; it can be safely deleted. ## Work Guidance diff --git a/src/runtime/mapvfs.config b/src/runtime/mapvfs.config index 10742289..c11b6923 100644 --- a/src/runtime/mapvfs.config +++ b/src/runtime/mapvfs.config @@ -60,9 +60,8 @@ tclsfe-x64.exe {punk9wintk903.vfs punk91 zip} #suite_tcl90 zig build (G-096/G-098, both achieved 2026-07-20) - runtime copied manually from # src/buildsuites/_build/suite_tcl90/out/bin/tclsh90szip.exe (static tclsh 9.0.5 with attached zip -# carrying the tcl library). tk/thread dlls + script libs ride in the .vfs for now - future: attach -# at least tclvfs & thread to the runtime's own zip (G-103 kit family; a piperepl-runtime punk kit -# based on tclsh90sprzip is also G-103 territory). +# carrying the tcl library). tk/thread dlls + script libs ride in the .vfs for this entry; the +# G-103 family runtimes below supersede that arrangement. #_beta convention: wrap freshly suite-built runtimes as *_beta kits even BEFORE they have passed the # punk test suites - so new-runtime behaviour can be trialled interactively. Drop _beta on acceptance. # (punk905_beta trialled and PROMOTED to punk905 2026-07-21.) @@ -70,6 +69,18 @@ tclsfe-x64.exe {punk9wintk903.vfs punk91 zip} #5 tclsh905.exe {punk9wintk905.vfs punk905 zip} +#G-103 runtime kit family (suite_tcl90 'kit-family' step): self-contained runtimes whose attached +# zip already carries the tcl library PLUS Thread/tclvfs/tcllib+tcllibc (bi adds Tk+tklib). WORKING +# names per the G-103 naming decision (dotted tcl patchlevel; piperepl-patched runtimes carry +# 'punk'; -r stays on the immutable punkbin artifact tier - 'runtime use' materializes an +# artifact into these names). Copied from src/buildsuites/_build/suite_tcl90/out/family/. +#punk runtime + the full punk vfs (which still carries its own tk/thread/etc payload - duplicated +# batteries resolve to the highest version, i.e. the runtime's; the vfs payload slims as G-103 +# progresses): +tclsh9.0.5-punk.exe {punk9wintk905.vfs punk9_beta zip} +#bi runtime already carries Tk - pair with the for-tkruntime vfs (no tk payload in the vfs): +tclsh9.0.5-punk-bi.exe {punk9win_for_tkruntime.vfs punk9bi_beta zip} + #---------------------------------------------- #experiment - what happens when we run against a 'wish' runtime? Will we have stdin stdout problems? #tksfe-twapi-x64.exe {punk9wintk903.vfs punkwish91 zip}