From 0d80cf6771ada264d143131af9182ee1700db88b Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Fri, 31 Jul 2026 17:45:19 +1000 Subject: [PATCH] G-024 + G-115 achieved: mapvfs.toml kit mapping + declarative per-.vfs payloads (punkshell 0.38.0) G-024 - kit mapping converted to toml (tomlish-parsed), same-day activation->achieved: - src/runtime/mapvfs.toml: [kit.] / [group.] / [scheme.] tables (header comment = user-facing spec). All 12 active line-format mappings migrated (order preserved - identical bakelist rows vs the legacy parse); historical comments carried over; mapvfs.config deleted here AND in the project-0.1 layout (converted to mapvfs.toml there). - make.tcl reader family under the unchanged G-121 model surface: mapvfs_locate (toml > legacy; PUNK_MAPVFS_CONFIG env override = characterization seam), mapvfs_parse dispatching on extension, strict mapvfs_parse_toml, deprecated mapvfs_parse_config fallback (un-migrated generated projects keep baking, with a NOTE; toml wins when both files exist). Shared toml helpers hoisted to punkboot::lib (inline toml_untag copies removed). - Strict entry-named validation: unknown keys/types/schemes, conflicting targets, duplicate outputs, parse failures fatal (exit 3 pre-build). Full-bake unresolvables split by nature: missing vfs folder (repo content) fatal per entry; missing runtime file (environment - part-populated stores are legitimate) entry-named recapped BUILD-WARNING + skip. - Groups: @ selection in bake/bakelist, group/default/scheme notes columns + detail lines, groups listed in errors and argdoc choices. verify-ix86 wraps the G-129 kits; bake_default=false excludes entries from full bakes. - G-023 versioned scheme declared live ([scheme.punk8]/[scheme.punk9], group versioned, bake_default=false pending G-023 activation): parse-time expansion from punkproject.toml to punk- / punk-dev / release-gated punk; kit loop creates the plain name when absent and never overwrites it on a normal bake. punk9-dev baked end-to-end; punk9 gate skip demonstrated. - Consumer dedupe fix: several entries may share one runtime on one vfs - kit-output enumeration and the kit-loop runtimes list process each distinct runtime once (latent multiplication in the legacy format, unreachable there). G-115 - declarative .vfs composition with drop-in preservation, same-day activation->achieved: - Sibling declarations src/vfs/.vfs.toml ([payload.]: source, source_root src|packages, target, supersedes, replace) materialized by the new VFSPAYLOAD phase (bakehouse/bake/vfslibs; selective bakes narrow; strict entry-named errors). Drop-in-wins precedence via punkcheck -overwrite synced-targets: undeclared or modified-since-install files preserved and reported (changed-but-preserved NOTE = files_skipped minus sources_unchanged); supersedes/replace are the explicit exceptions; an undeclared .vfs is untouched. - vendorlib_vfs.toml relationship settled: folded/superseded - all six install entries migrated into per-.vfs files for the 13 participating folders (first materialization run reproduced every payload with zero tracked-content change); file deleted; a leftover copy warns and is ignored. - Demonstration kit punkdeclare (group declare-demo, bake_default=false, smokerequire udp+tcllibc): VCS carries only the boot fauxlink; the whole lib_tcl9 payload (tcludp vendor tree + tcllibc consent-gated packages tier) rematerializes from the declaration - clean-tree rebuild smokes pass in-kit, and drop-in survival was proven live (a foreign pkgIndex.tcl and an undeclared note both survived; the foreign file genuinely won until removed, caught by the G-133 smoke probe as designed). Payload ignored in both VCS (.gitignore + derived ignore-glob; ignore-sync verification clean). - Docs: src/vfs/README.md (format + precedence spec), src/vfs + src/runtime + src + bin AGENTS.md, ARCHITECTURE.md (+lint clean), workflow text and summaries/helptexts/argdoc; layout store seeds the convention (vfs README + mapvfs.toml; machinery travels via the established make.tcl sync). Verification: legacy-vs-toml identical kit rows; punkluck86 force-rebuilt under both formats content-identical (identical 1.73MB runtime+stamp prefix, identical 2993-member payloads; whole-artifact byte identity is not a pipeline property - consecutive same-config rebuilds differ in zip timestamp metadata, pre-existing); punkexe suite 98 tests 0 failures incl. 5 new G-024 characterizations (maketclbakelist.test: @group filter + PUNK_MAPVFS_CONFIG fixture tests) and updated pins (payloadcheck declared list + punkdeclare; platform detail wording). Goals: both flipped achieved 2026-07-31 with evidence in the detail files; GOALS.md entries moved to GOALS-archive.md; live-tier reference sweep applied (G-004/G-019/G-023/G-025/G-026/G-035/G-065/G-127/G-131/G-137/G-141) including the G-127 mapvfs-seam retirement note (PUNK_MAPVFS_CONFIG) and a direct G-137<->G-127 bridge replacing the archived G-024 one. Detail files move to goals/archive/ in the follow-up pure-rename commit. Also included as found: the developer's concurrent G-142 goal addition (GOALS.md entry + goals/G-142-punkbin-listing-manifests.md), committed unmodified for git/fossil coherence - not agent-authored. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- .fossil-settings/ignore-glob | 3 + .gitignore | 5 + ARCHITECTURE.md | 10 +- CHANGELOG.md | 37 +- GOALS-archive.md | 8 + GOALS.md | 8 - bin/AGENTS.md | 6 +- goals/G-004-no-committed-binaries.md | 11 +- .../G-019-dependency-scan-module-trimming.md | 9 +- goals/G-023-version-named-binaries.md | 34 +- goals/G-024-mapvfs-toml.md | 73 +- goals/G-025-exe-selfreport.md | 8 +- goals/G-026-vendor-provenance-policy.md | 9 +- goals/G-035-mixed-tm-pkgindex-provision.md | 3 +- goals/G-065-declarative-vendoring.md | 11 +- goals/G-115-declarative-vfs-composition.md | 75 +- goals/G-127-crosstarget-vfs-bake.md | 23 +- goals/G-131-boot-payload-autodetection.md | 7 +- goals/G-137-kit-version-resources.md | 10 +- goals/G-141-libfetch-target-selection.md | 17 +- punkproject.toml | 2 +- src/AGENTS.md | 6 +- src/buildsuites/suite_tcl90/README.md | 2 +- src/make.tcl | 984 ++++++++++++++---- .../project-0.1/src/runtime/mapvfs.config | 16 - .../punk/project-0.1/src/runtime/mapvfs.toml | 34 + .../vendor/punk/project-0.1/src/vfs/README.md | 31 + src/runtime/AGENTS.md | 18 +- src/runtime/mapvfs.config | 150 --- src/runtime/mapvfs.toml | 220 ++++ src/runtime/vendorlib_vfs.toml | 122 --- .../testsuites/punkexe/maketclbakelist.test | 107 +- .../punkexe/maketclpayloadcheck.test | 8 +- .../testsuites/punkexe/maketclplatform.test | 6 +- src/vfs/AGENTS.md | 11 +- src/vfs/README.md | 75 ++ src/vfs/mkzipfix.vfs.toml | 21 + src/vfs/punk86.vfs.toml | 20 + src/vfs/punk86bawt.vfs.toml | 12 + src/vfs/punk8_statictwapi.vfs.toml | 20 + src/vfs/punk8win.vfs.toml | 27 + src/vfs/punk9cook.vfs.toml | 21 + src/vfs/punk9cookbasic.vfs.toml | 21 + src/vfs/punk9linux.vfs.toml | 20 + src/vfs/punk9magicsplat.vfs.toml | 21 + src/vfs/punk9win_for_tkruntime.vfs.toml | 28 + src/vfs/punk9wintk903.vfs.toml | 28 + src/vfs/punk9wintk905.vfs.toml | 28 + src/vfs/punk9wintk90b2.vfs.toml | 28 + src/vfs/punkdeclare.vfs.toml | 20 + ...main.tcl#@punk%3a%3aboot,merge_over#.fxlnk | 0 51 files changed, 1893 insertions(+), 581 deletions(-) delete mode 100644 src/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.config create mode 100644 src/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.toml create mode 100644 src/project_layouts/vendor/punk/project-0.1/src/vfs/README.md delete mode 100644 src/runtime/mapvfs.config create mode 100644 src/runtime/mapvfs.toml delete mode 100644 src/runtime/vendorlib_vfs.toml create mode 100644 src/vfs/README.md create mode 100644 src/vfs/mkzipfix.vfs.toml create mode 100644 src/vfs/punk86.vfs.toml create mode 100644 src/vfs/punk86bawt.vfs.toml create mode 100644 src/vfs/punk8_statictwapi.vfs.toml create mode 100644 src/vfs/punk8win.vfs.toml create mode 100644 src/vfs/punk9cook.vfs.toml create mode 100644 src/vfs/punk9cookbasic.vfs.toml create mode 100644 src/vfs/punk9linux.vfs.toml create mode 100644 src/vfs/punk9magicsplat.vfs.toml create mode 100644 src/vfs/punk9win_for_tkruntime.vfs.toml create mode 100644 src/vfs/punk9wintk903.vfs.toml create mode 100644 src/vfs/punk9wintk905.vfs.toml create mode 100644 src/vfs/punk9wintk90b2.vfs.toml create mode 100644 src/vfs/punkdeclare.vfs.toml create mode 100644 src/vfs/punkdeclare.vfs/main.tcl#..+_config+punk_main.tcl#@punk%3a%3aboot,merge_over#.fxlnk diff --git a/.fossil-settings/ignore-glob b/.fossil-settings/ignore-glob index 17866a7d..cd0c2940 100644 --- a/.fossil-settings/ignore-glob +++ b/.fossil-settings/ignore-glob @@ -45,6 +45,9 @@ test* src/testdata src/scriptapps/test_* +#G-115 demonstration kit payload - materialized from src/vfs/punkdeclare.vfs.toml, never committed +src/vfs/punkdeclare.vfs/lib_tcl9 + #review src/docgen diff --git a/.gitignore b/.gitignore index e217e799..2e39ee5a 100644 --- a/.gitignore +++ b/.gitignore @@ -44,6 +44,11 @@ scratch* /src/testdata /src/scriptapps/test_* +#G-115 demonstration kit payload - materialized from the src/vfs/punkdeclare.vfs.toml +#declaration (vendor tree + consent-gated bin/packages tier); reproducing it from the +#declaration is the point, so it is never committed +/src/vfs/punkdeclare.vfs/lib_tcl9/ + #review /src/docgen diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 2a72681a..0ee975eb 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -41,7 +41,7 @@ app package from src/lib/ (app-punkshell, app-punkscript, app-repl, app-shellsp ## Boot and launch chain -- **Kit anatomy.** Each `src/vfs/*.vfs` folder is a runtime payload; `src/runtime/mapvfs.config` maps payload folders to platform runtimes under `bin/runtime//`; `src/vfs/_vfscommon.vfs/` is a generated merge of common libraries (never hand-edited); `src/vfs/_config/` holds the entry scripts. Sources: `src/vfs/AGENTS.md`, `src/runtime/AGENTS.md`. +- **Kit anatomy.** Each `src/vfs/*.vfs` folder is a runtime payload; `src/runtime/mapvfs.toml` (G-024, tomlish-parsed; deprecated `mapvfs.config` line format still readable) maps payload folders to platform runtimes under `bin/runtime//` as named kit outputs, with named groups and generative version-named schemes; an optional sibling `src/vfs/.vfs.toml` declares payload packages materialized INTO the folder with drop-in-wins precedence (G-115 - spec: `src/vfs/README.md`); `src/vfs/_vfscommon.vfs/` is a generated merge of common libraries (never hand-edited); `src/vfs/_config/` holds the entry scripts. Sources: `src/vfs/AGENTS.md`, `src/runtime/AGENTS.md`. - **Entry point.** `src/vfs/_config/punk_main.tcl`. An optional first argument is a dash-delimited package mode composed of tokens from `dev`, `os`, `src`, `internal` (`internal` is always appended when absent), optionally scoped with the `proj:` prefix (G-033: `dev`/`src` resolve against the project containing the cwd - walk-up to the nearest git/fossil repo root with a punkshell-style src tree - instead of the executable's own project; discovery is always reported, never a silent rebind). The next argument is the subcommand: `shell`, `script`, `punk`, `tclsh`, `shellspy`. Any other first argument is treated as a script invocation; no arguments at all means interactive shell. Subcommands map to app packages under `src/lib/` (`app-punkshell`, `app-punkscript`, `app-repl`, `app-shellspy`). - **Package modes pick module provenance.** `internal` uses module snapshots baked into the kit at build time; `dev` loads build outputs from repo-root `modules/` and `lib/`; `src` loads the unbuilt working tree (`src/modules`, `src/lib`, `src/bootsupport`, `src/vendormodules`) with `package prefer latest` so magic-version dev modules (`999999.0a1.0`) beat stamped snapshots. `src` mode is the standard way to verify working-tree changes without a rebuild. Source: `bin/AGENTS.md` "Launch package modes". - **Payload mount derivation (G-129, achieved).** The boot keys zipfs presence on `tcl::zipfs::mount` and derives where the executable's attached archive actually mounted from the mount table itself (the entry whose archive file is the executable; fallback: the mount containing the boot script) instead of assuming `tcl::zipfs::root` + `//zipfs:/app`. Modern runtimes still answer `//zipfs:/app`; the androwish/undroidwish 8.6 backport has no `root` command, mounts at the executable's own path, and boots the same payload through its `app/main.tcl` in-archive hook (`src/vfs/punk8win.vfs` carries fauxlinks for both conventions). A mounted archive the boot cannot attribute is reported on stderr rather than silently yielding no internal paths. The derivation procs stay defined post-boot (`::punkboot::zipfs_kit_mountbase`). Sources: `src/vfs/AGENTS.md`, `bin/AGENTS.md` "Kit-wrappable runtime requirements", `src/tests/shell/testsuites/punkexe/kitmountpoint.test`. @@ -87,14 +87,14 @@ Two layers with a deliberate dependency direction (the class never depends on th ## Build, provenance and kits -- **`src/make.tcl` is the single build entry.** Key subcommands: `modules`/`libs`/`packages` (repo-root build outputs), `vfscommonupdate` (regenerate `_vfscommon.vfs`), `bake` (kit assembly into `bin/`; optional kit names confine the run to those kits - G-121), `bakelist` (report the configured kit matrix with runtime/vfs presence and deployed state), `bin`, `bakehouse` (packages + bake for a clean checkout; `project`/`vfs` remain as deprecated aliases - G-112 rename, achieved), `bootsupport` (snapshot refresh), `vendorupdate`, `check`, `workflow`, `projectversion`. Dispatch and help dogfood `punk::args` (G-030, achieved) and degrade to plain scan/help when bootsupport `punk::args` is stale (`PUNKBOOT_PLAIN=1` forces the degraded mode). Source: `src/AGENTS.md`. -- **Host vs target platform (G-122).** The kit surfaces separate what the driving tclsh IS from what a bake is FOR. Target-keyed: the `bin/runtime/` store a runtime is read from, `.exe` suffixing of runtimes and kit outputs, presence checks, and the pre-deploy process sweep's tooling (`tasklist`/`taskkill` vs `ps`/`kill`, skipped when the target's processes cannot exist on this host). Host-keyed: copy commands, path handling, filesystem case rules, prompts. The default target is the host canon except for cygwin-family hosts (msys2/cygwin-runtime tclsh - `tcl_platform(platform)` `unix` on windows), which target `win32-x86_64`; a `mapvfs.config` entry may declare its own target and is then addressed in that platform's tier. `make.tcl check` prints the derivation. Zip-type kits assemble without zipfs in the driving tcl (raw-runtime split + `punk::zip::mkzip` + concatenation), and since G-124 they EXTRACT without it too: `punk::zip` reads a zip - plain, or attached to an executable under either offset convention - with stock Tcl only, so a bake needs neither zipfs nor tcllib to carry a runtime's `tcl_library` into the kit. Sources: `src/AGENTS.md`, `src/runtime/AGENTS.md`, `src/modules/punk/platform-999999.0a1.0.tm`, `src/modules/punk/zip-999999.0a1.0.tm`. +- **`src/make.tcl` is the single build entry.** Key subcommands: `modules`/`libs`/`packages` (repo-root build outputs), `vfscommonupdate` (regenerate `_vfscommon.vfs`), `bake` (kit assembly into `bin/`; optional kit names or `@group` selectors confine the run to those kits - G-121/G-024; `bake_default=false` entries build only when selected), `bakelist` (report the configured kit matrix with runtime/vfs presence, deployed state, groups and scheme roles), `vfslibs` (materialize the per-.vfs payload declarations - G-115), `bin`, `bakehouse` (packages + bake for a clean checkout; `project`/`vfs` remain as deprecated aliases - G-112 rename, achieved), `bootsupport` (snapshot refresh), `vendorupdate`, `check`, `workflow`, `projectversion`. Dispatch and help dogfood `punk::args` (G-030, achieved) and degrade to plain scan/help when bootsupport `punk::args` is stale (`PUNKBOOT_PLAIN=1` forces the degraded mode). Source: `src/AGENTS.md`. +- **Host vs target platform (G-122).** The kit surfaces separate what the driving tclsh IS from what a bake is FOR. Target-keyed: the `bin/runtime/` store a runtime is read from, `.exe` suffixing of runtimes and kit outputs, presence checks, and the pre-deploy process sweep's tooling (`tasklist`/`taskkill` vs `ps`/`kill`, skipped when the target's processes cannot exist on this host). Host-keyed: copy commands, path handling, filesystem case rules, prompts. The default target is the host canon except for cygwin-family hosts (msys2/cygwin-runtime tclsh - `tcl_platform(platform)` `unix` on windows), which target `win32-x86_64`; a `mapvfs.toml` entry may declare its own target and is then addressed in that platform's tier. `make.tcl check` prints the derivation. Zip-type kits assemble without zipfs in the driving tcl (raw-runtime split + `punk::zip::mkzip` + concatenation), and since G-124 they EXTRACT without it too: `punk::zip` reads a zip - plain, or attached to an executable under either offset convention - with stock Tcl only, so a bake needs neither zipfs nor tcllib to carry a runtime's `tcl_library` into the kit. Sources: `src/AGENTS.md`, `src/runtime/AGENTS.md`, `src/modules/punk/platform-999999.0a1.0.tm`, `src/modules/punk/zip-999999.0a1.0.tm`. - **punkcheck** records every install/delete as events in per-folder `.punkcheck` directories - the basis for skip/copy change detection, superseded-module pruning and provenance. Single OO record lifecycle (G-094, achieved) with atomic saves and advisory event-scoped locking for concurrent writers (G-095, achieved). - **Provenance gates.** Build/promotion commands warn on uncommitted `src/` changes (column-0 `PROVENANCE-WARNING:` token, `-dirty-abort` for strict mode); `vendorupdate` warns for dirty source-project checkouts. - **Boot-precondition gate (G-125).** A bake refuses a kit whose merged `.vfs` supplies no tcl library rather than deploying an artifact that cannot initialise: the kit lands in `FAILED KITS` and nothing is written, so the previously deployed `bin/` survives. Structural and non-executing (so cross-target kits are covered), reading the merged tree rather than the extraction outcome. The predicate is `punkboot::utils::vfs_boot_library_report`, called through the same guarded require as the provenance check so a stale bootsupport snapshot degrades it to a notice; `make.tcl check` reports ACTIVE/UNAVAILABLE. Sources: `src/AGENTS.md`, `src/modules/punkboot/utils-999999.0a1.0.tm`. -- **Payload/target consistency checks (G-133 + G-134), all advisory.** At the same post-merge seam, a bake classifies each binary library in the merged tree by header (`punkboot::utils::binary_arch_classify` - PE/ELF/Mach-O, honest unknowns) against the kit's target platform; wrong-arch libraries outside platform-discriminated subdirs (canonical `-` names, vendor spellings like `win-x64`) earn recapped `BUILD-WARNING`s. After assembly, a kit declaring smoke-require packages (`mapvfs.config` 5th entry element) has each one plain-`package require`d inside the freshly built artifact via its tclsh subcommand - the only check that sees resolution-order defects (wrong-arch version shadowing); cross-target kits skip with a stated reason. The assembled image is also probed for its zip offset convention (G-134, `punkboot::utils::kit_offsetstyle_report` over `punk::zip::archive_info`): a FILE-relative attached payload warns (the pipeline emits archive-relative; the G-128 stamper refuses file-relative by default), no-zip/plain results stay silent. Same guarded-require degradation; `make.tcl check` reports all three. None guarantees statics, pure-tcl packages with binary deps, or version preference beyond the declared smoke set. Sources: `src/AGENTS.md`, `src/runtime/AGENTS.md`, `src/modules/punkboot/utils-999999.0a1.0.tm`. +- **Payload/target consistency checks (G-133 + G-134), all advisory.** At the same post-merge seam, a bake classifies each binary library in the merged tree by header (`punkboot::utils::binary_arch_classify` - PE/ELF/Mach-O, honest unknowns) against the kit's target platform; wrong-arch libraries outside platform-discriminated subdirs (canonical `-` names, vendor spellings like `win-x64`) earn recapped `BUILD-WARNING`s. After assembly, a kit declaring smoke-require packages (`mapvfs.toml` `smokerequire` key) has each one plain-`package require`d inside the freshly built artifact via its tclsh subcommand - the only check that sees resolution-order defects (wrong-arch version shadowing); cross-target kits skip with a stated reason. The assembled image is also probed for its zip offset convention (G-134, `punkboot::utils::kit_offsetstyle_report` over `punk::zip::archive_info`): a FILE-relative attached payload warns (the pipeline emits archive-relative; the G-128 stamper refuses file-relative by default), no-zip/plain results stay silent. Same guarded-require degradation; `make.tcl check` reports all three. None guarantees statics, pure-tcl packages with binary deps, or version preference beyond the declared smoke set. Sources: `src/AGENTS.md`, `src/runtime/AGENTS.md`, `src/modules/punkboot/utils-999999.0a1.0.tm`. - **Kit icon step (G-057 + G-128).** Every kit a bake builds gets a `.resources.toml` sidecar (deployed beside `bin/`) recording the build-time icon choice - default `src/runtime/punkshell.ico`, overridable by a root `punkshell.ico` in the kit's own custom `.vfs` folder (pre-merge) - with sha256 identity and provenance lifted from the icon's G-135 assetorigin record. win32-target kits get the icon embedded as PE `RT_ICON`/`RT_GROUP_ICON` behind a single seam (`::punkboot::kit_icon_process`): the vendored punkres portable stamper (`src/tools/punkres`, built to `bin/punkres(.exe)` by the make.tcl tool step) is selected when present and works from any build host; otherwise the twapi arm (tcl-sfe mechanism) serves windows hosts with nothing built. Both arms share one semantic (delete all icon/group entries, write ids 1..N; the group is NAMED from the icon file's uppercased rootname - `PUNKSHELL` - with language adopted from the replaced group) and stamp a per-kit copy of the payload-free raw runtime prefix BEFORE payload attach; punkres can also stamp a FINISHED kit post-hoc, preserving an appended zip payload (archive-relative moved verbatim; file-relative refused by default, shifted with a consent flag). Non-PE targets skip as not applicable; a host where neither mechanism serves skips with a combined notice naming the punkres build remedy; the sidecar is written in every case. Sources: `src/AGENTS.md`, `bin/AGENTS.md`, `src/vfs/AGENTS.md`, `src/tools/AGENTS.md`. -- **Buildsuites and the kit family.** `src/buildsuites/suite_tcl90/` builds Tcl/Tk/tcllib from source with a pinned zig toolchain and produces the runtime kit family (plain / punk / bi) plus artifact metadata (G-096-G-117 era: see archived goals G-096, G-098, G-102, G-103, G-107); artifacts publish to the punkbin repo that `bin/punk-runtime.cmd` fetches from. Both suites also emit the punkbin LIBRARY tier (G-138, achieved): verified tcllib/tcllibc installs as generation-tagged immutable zips (`lib/allplatforms` + `lib/`, embedded schema-v2 class=library records, punkzip-deterministic) - the `library-artifacts` step in each recipe. The tier's `` axis includes cross-built lanes: suite_tcl90's `tcllibc-linux` step (G-140, achieved) cross-builds the critcl accelerators for linux-x86_64 via critcl driving `zig cc -target` (stubs linkage - no linux host involved), structurally ELF-gated on the build host and consumed by the punk9linux kit payload. Consumption (G-139, achieved): `make.tcl libfetch` materializes declared artifacts into the untracked `bin/packages/` tier with server-trust consent, and the PACKAGES_tcl libs phase + vfslibs packages-root declarations feed the deployed `lib_tcl` trees and kit vfs payloads - the vendored tcllib trees are retired, and embedded records ride into baked kits. `src/buildsuites/suite_tcl86/` is the 8.6 sibling (G-099 + G-100, both achieved): a Tcl 8.6 windows runtime (static + dynamic shells, `tcl86t.dll`, on-disk lib tree - no zipfs, so no self-contained kit) with thread 2.8 + tclvfs + Tk 8.6 + tklib + tcllib(+tcllibc critcl accelerators) companions, gated against the 8.6 core/thread/tclvfs testsuites with tcllib/tklib on a record tier and an opt-in `test-tk`; punkshell's own runtests is censused on that runtime at parity with a same-day native-8.6 baseline. Suite child shells scrub `TCL__TM_PATH` as well as TCLLIBPATH/TCL_LIBRARY/TK_LIBRARY - without it a census measures the machine's module trees. The 8.6 kit container strategy remains in-flux - see "In-flux areas". +- **Buildsuites and the kit family.** `src/buildsuites/suite_tcl90/` builds Tcl/Tk/tcllib from source with a pinned zig toolchain and produces the runtime kit family (plain / punk / bi) plus artifact metadata (G-096-G-117 era: see archived goals G-096, G-098, G-102, G-103, G-107); artifacts publish to the punkbin repo that `bin/punk-runtime.cmd` fetches from. Both suites also emit the punkbin LIBRARY tier (G-138, achieved): verified tcllib/tcllibc installs as generation-tagged immutable zips (`lib/allplatforms` + `lib/`, embedded schema-v2 class=library records, punkzip-deterministic) - the `library-artifacts` step in each recipe. The tier's `` axis includes cross-built lanes: suite_tcl90's `tcllibc-linux` step (G-140, achieved) cross-builds the critcl accelerators for linux-x86_64 via critcl driving `zig cc -target` (stubs linkage - no linux host involved), structurally ELF-gated on the build host and consumed by the punk9linux kit payload. Consumption (G-139, achieved): `make.tcl libfetch` materializes declared artifacts into the untracked `bin/packages/` tier with server-trust consent, and the PACKAGES_tcl libs phase + per-.vfs payload declarations with `source_root = "packages"` (G-115) feed the deployed `lib_tcl` trees and kit vfs payloads - the vendored tcllib trees are retired, and embedded records ride into baked kits. `src/buildsuites/suite_tcl86/` is the 8.6 sibling (G-099 + G-100, both achieved): a Tcl 8.6 windows runtime (static + dynamic shells, `tcl86t.dll`, on-disk lib tree - no zipfs, so no self-contained kit) with thread 2.8 + tclvfs + Tk 8.6 + tklib + tcllib(+tcllibc critcl accelerators) companions, gated against the 8.6 core/thread/tclvfs testsuites with tcllib/tklib on a record tier and an opt-in `test-tk`; punkshell's own runtests is censused on that runtime at parity with a same-day native-8.6 baseline. Suite child shells scrub `TCL__TM_PATH` as well as TCLLIBPATH/TCL_LIBRARY/TK_LIBRARY - without it a census measures the machine's module trees. The 8.6 kit container strategy remains in-flux - see "In-flux areas". - **Project generation.** `dev project.new` composes thin layouts from `src/project_layouts/` (overlay chain with `.anti` deletion markers and `name@base` derivation - G-087, achieved) and injects bootsupport modules from the generating shell at generation time. - **Workflow overview.** `tclsh src/make.tcl workflow` prints the embedded ASCII data-flow overview of the build/release pipeline, with its own update contract in `src/AGENTS.md`. This section deliberately summarises rather than copies it. diff --git a/CHANGELOG.md b/CHANGELOG.md index 8980c6d5..63df75c5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,7 +5,42 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` "Project Versioning" section for the bump policy. -## [0.37.0] - 2026-07-31 +## [0.38.0] - 2026-07-31 + +- Kit mapping converted to toml (G-024, achieved): `src/runtime/mapvfs.toml` + (tomlish-parsed `[kit.]` / `[group.]` / `[scheme.]` tables) + replaces the mapvfs.config line format, which remains readable as a + deprecated fallback for un-migrated trees (toml wins when both exist). + Strict entry-named validation (unknown keys/types/schemes, conflicting + targets and parse failures are fatal; a full bake refuses a default entry + whose vfs folder is missing and warn-skips entries whose runtime file is + absent from its store tier). Named groups select in `bake`/`bakelist` as + `@`; `bake_default=false` entries are excluded from full bakes but + bakeable by name/group. The G-023 `versioned` scheme expands one declaration + into `punk-` + `punk-dev` + the release-gated plain + `punk` (created when absent; a normal bake never overwrites it) from + the punkproject.toml version - no per-version config edits. New + `PUNK_MAPVFS_CONFIG` env seam points bakelist/bake at an alternate mapping + for characterization. Verified: legacy-vs-toml identical kit rows, and a + forced punkluck86 rebuild content-identical across formats (identical + runtime prefix, identical 2993-member payload). +- Declarative .vfs payload composition (G-115, achieved): a sibling + `src/vfs/.vfs.toml` declares package folders materialized INTO the + `.vfs` folder (`[payload.]`: source, source_root src|packages, target, + supersedes, replace) by `make.tcl vfslibs` and the bake phase - + punkcheck-tracked with drop-in-wins precedence (`-overwrite synced-targets`: + undeclared or hand-modified files are preserved and reported, never + overwritten; supersedes/replace are the explicit exceptions). A `.vfs` + with no declaration is untouched. The per-package + `src/runtime/vendorlib_vfs.toml` surface is migrated into per-.vfs files + and retired (a leftover copy is ignored with a warning). Format + + precedence spec: `src/vfs/README.md`; the project-0.1 layout seeds the + convention (mapvfs.toml + vfs README) for derived projects. New + demonstration kit `punkdeclare` (group `declare-demo`, bake_default=false): + VCS carries only its boot fauxlink and the whole binary payload (tcludp + vendor tree + tcllibc packages tier) rematerializes from the declaration - + proven by clean-tree rebuilds with passing in-kit smoke requires and by + drop-in survival runs. - Buildsuite artifact identity fields (`project`, `project_url`, `packager`) now default from the enclosing project's `punkproject.toml` `[project]` diff --git a/GOALS-archive.md b/GOALS-archive.md index 4336e9a2..a5a3302b 100644 --- a/GOALS-archive.md +++ b/GOALS-archive.md @@ -254,3 +254,11 @@ Acceptance: a make.tcl entrypoint fetches declared library artifacts from the pu ### G-140 [achieved 2026-07-30] Linux lib-tier emission: cross-built tcllibc (zig cc -target) for the punk9linux payload → detail: goals/archive/G-140-linux-lib-tier-emission.md Scope: src/buildsuites/suite_tcl90/ (critcl_zig.config linux target section; build905.zig cross-target library-artifacts lane; tools as touched); src/runtime/libpackages.toml + src/runtime/vendorlib_vfs.toml (linux-x86_64 artifact + kit-vfs declarations); src/vfs/punk9linux.vfs/lib_tcl9 (hand-dropped tcllibc + stray tcllibc2.0 - retirement targets); punkbin artifact repo (external, c:/repo/jn/punkbin - new lib/linux-x86_64 target folder); bin/packages/linux-x86_64/ (tier target as materialized); WSL verification (testsupport/wslprobe.tcl patterns + a punkbin linux-x86_64 Tcl 9 runtime as the smoke interpreter) Acceptance: a documented suite_tcl90 invocation produces lib/linux-x86_64/tcllibc-tcl9-r.zip from a critcl build driving zig cc via an explicit linux target section in critcl_zig.config (the win32-x86_64-zig pin pattern; no linux host in the build path), with the embedded record carrying target = linux-x86_64, build_host_platform = the actual build host, and the G-138 provenance set (checkout uuids, critcl version, zig version, optimize mode, driving-shell patchlevel); the contained shared library classifies as ELF x86-64 by a host-side structural check (the G-133 classifier or equivalent, recorded here); re-emission over an unchanged build is byte-identical; the libc/triple decision (dynamic glibc with a pinned version floor vs musl-dynamic - the .so must dlopen into existing glibc-dynamic linux tclsh builds without mixing libcs) is made and recorded here with rationale, explicitly feeding the G-105 decision item rather than re-deciding it later; a WSL (or native linux) smoke on the MATERIALIZED artifact tree under a linux-x86_64 Tcl 9 runtime (e.g. the punkbin tclkit-902 artifact) proves package require md5 engages the critcl accelerator from the tree (evidence recorded here); consumption lands via libpackages.toml (target linux-x86_64) and a vendorlib_vfs.toml entry switching punk9linux.vfs/lib_tcl9 with supersedes/replace retiring the hand-dropped tcllibc and tcllibc2.0; a punkshell902 bake ships the artifact-fed tree with its embedded record present in the kit payload, and a WSL run of the baked kit loading md5 with the accelerator engaged is recorded; publication to the real punkbin repo remains a deliberate user step (build_sha1sums.tcl already handles arbitrary lib/ folders); the tcl8-generation question is dispositioned (no linux tcl8 kit exists - recorded as out of the committed set, with the mechanism staying package-generic and generation-generic by construction). + +### G-024 [achieved 2026-07-31] mapvfs.config converted to toml (tomlish-parsed) with succinct scheme declarations → detail: goals/archive/G-024-mapvfs-toml.md +Scope: src/runtime/mapvfs.config (replaced/deprecated); src/make.tcl (parsing); src/bootsupport/modules/tomlish-*.tm (parser dependency) +Acceptance: a mapvfs toml file parsed via tomlish (no ad-hoc toml parsing) drives the build: every mapping currently active in mapvfs.config is expressible and at least one existing target builds identically from the toml; the G-023 versioned/dev/release-gated output scheme is declared in one entry that expands to its outputs without enumerating versions; malformed or unresolvable entries fail the build with a clear message naming the entry; the legacy .config format is either fully migrated (old parser removed) or explicitly deprecated with documented precedence between the two files. + +### G-115 [achieved 2026-07-31] Declarative .vfs composition: toml-defined kit payloads with drop-in preservation → detail: goals/archive/G-115-declarative-vfs-composition.md +Scope: src/make.tcl (vfs assembly); src/runtime/vendorlib_vfs.toml (existing per-package declaration surface - fold/supersede settled in the work); src/vfs/ (per-.vfs declaration files + README); punk::mix machinery as touched; src/project_layouts (seeding for derived projects); coordinates with G-067 (artifact sources); G-006 (consent); G-004 (binary-free committed tree) +Acceptance: a punkshell kit .vfs (or demonstration .vfs) builds from a toml declaration reproducing its payload on a clean tree, with declared binary content arriving via consented retrieval or local build products; an undeclared dropped-in file survives re-materialization per the documented precedence; a .vfs with NO declaration builds exactly as today (pure drop-in mode unchanged); the declaration format and precedence rules are documented; the vendorlib_vfs.toml relationship is settled with rationale; the layout store seeds the convention for derived projects. diff --git a/GOALS.md b/GOALS.md index 5d8d8ba9..b80ef8bd 100644 --- a/GOALS.md +++ b/GOALS.md @@ -157,10 +157,6 @@ Detail: goals/G-022-fossil-rename-punkshell.md Scope: src/make.tcl; src/runtime/ (mapping config - see G-024); bin/ (build outputs) Detail: goals/G-023-version-named-binaries.md -### G-024 [active] mapvfs.config converted to toml (tomlish-parsed) with succinct scheme declarations -Scope: src/runtime/mapvfs.config (replaced/deprecated); src/make.tcl (parsing); src/bootsupport/modules/tomlish-*.tm (parser dependency) -Detail: goals/G-024-mapvfs-toml.md - ### G-025 [proposed] Punk executables self-report project version and build provenance Scope: src/vfs/_config/punk_main.tcl (subcommand dispatch); src/make.tcl (stamping build info into the vfs); src/vfs/ (stamp payload location); src/modules/punk/ (in-shell command - the single implementation) Detail: goals/G-025-exe-selfreport.md @@ -369,10 +365,6 @@ Detail: goals/G-111-modpod-tidy-tests.md Scope: src/vfs/_config/punk_main.tcl + src/vfs/_config/project_main.tcl + src/make.tcl (boot tm-path wiring); src/modules/punk/platform-999999.0a1.0.tm (canon names, as consumer); new per-platform tm root trees (layout/naming decision - sibling roots beside src/vendormodules_tclX / src/modules_tclX); src/project_layouts (layout seeding); modpod demonstration artifact; tree READMEs; coordinates with G-109 (manifest target-platform field stays that goal's item) Detail: goals/G-114-per-platform-tm-roots.md -### G-115 [active] Declarative .vfs composition: toml-defined kit payloads with drop-in preservation -Scope: src/make.tcl (vfs assembly); src/runtime/vendorlib_vfs.toml (existing per-package declaration surface - fold/supersede settled in the work); src/vfs/ (per-.vfs declaration files + README); punk::mix machinery as touched; src/project_layouts (seeding for derived projects); coordinates with G-067 (artifact sources); G-006 (consent); G-004 (binary-free committed tree) -Detail: goals/G-115-declarative-vfs-composition.md - ### G-116 [proposed] Suite-built tcltls with a zig-built crypto backend: bi-family battery, prebuilt replacement path Scope: src/buildsuites/suite_tcl90/ (build_tcltls module + crypto-backend build, sources.config + build.zig.zon source records, kit-family bi payload + metadata extension, test_gate record step); src/vfs kit payloads carrying vendored tcltls binaries (current-state reference + recorded disposition only - removal stays G-004-era work); punkbin (as eventual artifact destination via the G-067 library class); consumers punk::imap4 / punk::netbox (package require tls) as verification context Detail: goals/G-116-suite-built-tcltls.md diff --git a/bin/AGENTS.md b/bin/AGENTS.md index ba5f04ba..99dd1c79 100644 --- a/bin/AGENTS.md +++ b/bin/AGENTS.md @@ -146,8 +146,8 @@ The `runtime/win32-ix86/` tier holds the two third-party 32-bit verification run bracketing the mount conventions: `tclsh8.6.10-luck-zip.exe` (exe-path mount; reproducible via `LUCK.url` + the base64 build request in `tclsh8.6.10-luck-zip.cfg`) and `tclsh9.1b0-tclsfe.exe` (modern `//zipfs:/app`; apnadkarni/tcl-sfe releases), -wrapped as `punkluck86` / `punk91ix86` in `src/runtime/mapvfs.config`. Characterization: -`src/tests/shell/testsuites/punkexe/kitmountpoint.test`. +wrapped as `punkluck86` / `punk91ix86` in `src/runtime/mapvfs.toml` (group `verify-ix86`). +Characterization: `src/tests/shell/testsuites/punkexe/kitmountpoint.test`. ### What a bake verifies about a deployed kit (G-125, G-133, G-134) @@ -156,7 +156,7 @@ merged vfs supplies a tcl library - a kit failing that is never deployed) and th G-133 advisory checks have run: a structural binary-arch scan of its payload (a wrong-arch `.dll`/`.so`/`.dylib` outside a platform-discriminated subdir earned a recapped `BUILD-WARNING` - advisory, so heed the bake recap before trusting a warned -kit) and, when its `src/runtime/mapvfs.config` entry declares smoke-require packages +kit) and, when its `src/runtime/mapvfs.toml` entry declares smoke-require packages and the kit is host-runnable, a plain `package require` of each inside the built artifact via its tclsh subcommand. The assembled image was also probed for its zip offset convention (G-134, advisory): a FILE-relative attached payload earned a diff --git a/goals/G-004-no-committed-binaries.md b/goals/G-004-no-committed-binaries.md index a6f4748a..63ea5a98 100644 --- a/goals/G-004-no-committed-binaries.md +++ b/goals/G-004-no-committed-binaries.md @@ -79,9 +79,14 @@ A standing repo-wide rule in root `AGENTS.md` (User Preferences) already directs couple unrelated components and churn on any payload change, so (iii) is recorded as an OPEN option for release-snapshot/derived-project convenience rather than the primary mechanism - decide when removal actually forces the - payloads out. A declarative .vfs composition surface (toml-defined payload - pulling from (i)/(ii), preserving drop-in simplicity) is the candidate - glue - see the G-115 proposal (2026-07-22) if adopted. + payloads out. The declarative .vfs composition surface (toml-defined payload + pulling from (i)/(ii), preserving drop-in simplicity) is now LIVE (G-115, + achieved 2026-07-31 - see goals/archive/G-115-declarative-vfs-composition.md): + per-.vfs declarations (src/vfs/.vfs.toml) materialize vendor-tree and + consent-gated packages-tier content with drop-in-wins preservation, and the + punkdeclare demonstration kit shows a fully-declared binary payload with + nothing committed (its lib_tcl9 is ignored in both VCS) - the exit-path glue + this note anticipated. - Depends on G-005 and G-006. Sequence G-005 ∥ G-006, then G-004. - The scan in step 4 should distinguish "executable binary" from "zip archive" and from "zip archive containing an executable." A naive extension-based check is insufficient; content inspection (file magic, zip listing) is required. - The AGENTS.md user-preference rule is the transition guard: it stops agents from adding new binaries during the G-005/G-006 development period while the developer's existing vendor/vfs binary commits are known and intentional. diff --git a/goals/G-019-dependency-scan-module-trimming.md b/goals/G-019-dependency-scan-module-trimming.md index d3c2b16b..951a245b 100644 --- a/goals/G-019-dependency-scan-module-trimming.md +++ b/goals/G-019-dependency-scan-module-trimming.md @@ -64,9 +64,10 @@ here. - Interacts with G-005/G-006 (how the c-only tclparser binary is provisioned if chosen) and G-018 (the spectrum of executable flavours). -- G-115 relationship (recorded 2026-07-24 after overlap review; neither file - previously referenced the other): the scan closure plus declared dynamic - extras is a payload-determination input G-115's toml materialization can - consume - a trimmed kit is then a declared kit with a generated declaration. +- G-115 relationship (recorded 2026-07-24; G-115 achieved 2026-07-31 - see + goals/archive/G-115-declarative-vfs-composition.md): the scan closure plus + declared dynamic extras is a payload-determination input the now-live per-.vfs + declaration surface (src/vfs/.vfs.toml) can consume - a trimmed kit is + then a declared kit with a generated declaration. - punk::lib sites already noting tclparser use/fallback: tclword_to_scriptlist and related parsing helpers (~lib 3322-3580). diff --git a/goals/G-023-version-named-binaries.md b/goals/G-023-version-named-binaries.md index c4625388..22c29bab 100644 --- a/goals/G-023-version-named-binaries.md +++ b/goals/G-023-version-named-binaries.md @@ -8,12 +8,12 @@ Acceptance: a project build at the current punkproject.toml version produces pun ## Context The project version is now tracked and change-driven-bumped in punkproject.toml -(root AGENTS.md "Project Versioning"), so builds can be named after it. Today's -build outputs are fixed names declared line-by-line in src/runtime/mapvfs.config -(punksys, punk902z, ...) that encode the runtime generation informally and say -nothing about the project version they contain - which build a given exe -represents is unknowable without launching it (G-025 addresses the launching -side; this goal addresses the naming side). +(root AGENTS.md "Project Versioning"), so builds can be named after it. The +fixed-name outputs (punksys, punk902z, ...) declared in the kit mapping (now +src/runtime/mapvfs.toml - G-024, achieved 2026-07-31) encode the runtime +generation informally and say nothing about the project version they contain - +which build a given exe represents is unknowable without launching it (G-025 +addresses the launching side; this goal addresses the naming side). Naming scheme: @@ -44,14 +44,27 @@ practice (bin/AGENTS.md: a Tcl 8.6 punk shell and a current Tcl 9 punk shell). - Names derive from punkproject.toml at build time - no hand-maintained per-version config entries. The scheme is declared once, succinctly, in the G-024 toml mapping; the expansion to concrete output names happens in - make.tcl. + make.tcl. STATUS 2026-07-31 (G-024 achieved - see + goals/archive/G-024-mapvfs-toml.md): this groundwork is LIVE - [scheme.punk8] + + [scheme.punk9] versioned-scheme entries ship in src/runtime/mapvfs.toml + (group 'versioned', bake_default=false pending THIS goal's activation), + expansion produces punk- / punk-dev / release-gated + punk records in the parsed model (bakelist shows them; punk9-dev was + baked end-to-end as verification), and the kit loop already implements the + release gate interim policy: plain punk is created when absent and a + normal bake never overwrites it (skip message points here; workaround: delete + bin/punk to force). Remaining for this goal: flip the versioned group's + bake_default (activation), settle/implement the explicit release step that + REPLACES the plain names, confirm per-version refresh semantics across + version bumps, and the space/archival notes below. - The release step that updates the plain punk8.exe/punk9.exe names should be an explicit make.tcl action tied to release tagging, not a side effect of a normal build. Its precise trigger (fossil/git tag presence, or a manual subcommand) is an implementation decision to record here. - Existing specifically-named outputs (punksys, punk902z, ...) keep working via - their explicit mappings (G-024 preserves them); the versioned scheme is - additive. + their explicit mappings (G-024 preserved them - verified: identical kit rows + and a content-identical punkluck86 rebuild across the format switch); the + versioned scheme is additive. ## Space and archival @@ -72,7 +85,8 @@ binary-artifacts repository). Revisit when the accumulation actually bites. ## Notes -- Related: G-024 (scheme declared in toml mapping), G-025 (exe self-reports its +- Related: G-024 (scheme declared in toml mapping; achieved 2026-07-31 - + goals/archive/G-024-mapvfs-toml.md), G-025 (exe self-reports its version/provenance - the complement of name-encoded versioning once files are renamed/copied), G-006 (future artifact-store home for archived versions), G-018/G-019 (other points on the executable-flavour spectrum). diff --git a/goals/G-024-mapvfs-toml.md b/goals/G-024-mapvfs-toml.md index 847efb6d..bcacb08d 100644 --- a/goals/G-024-mapvfs-toml.md +++ b/goals/G-024-mapvfs-toml.md @@ -1,6 +1,6 @@ # G-024 mapvfs.config converted to toml (tomlish-parsed) with succinct scheme declarations -Status: active +Status: achieved 2026-07-31 Scope: src/runtime/mapvfs.config (replaced/deprecated); src/make.tcl (parsing); src/bootsupport/modules/tomlish-*.tm (parser dependency) Goal: the runtime-to-vfs-to-executable build mapping moves from the custom line format of src/runtime/mapvfs.config to a toml file parsed with the tomlish package - still supporting explicit per-executable mappings (runtime, vfs folder, output name, kit type) while also expressing generative schemes like the G-023 versioned naming in a single succinct declaration. Acceptance: a mapvfs toml file parsed via tomlish (no ad-hoc toml parsing) drives the build: every mapping currently active in mapvfs.config is expressible and at least one existing target builds identically from the toml; the G-023 versioned/dev/release-gated output scheme is declared in one entry that expands to its outputs without enumerating versions; malformed or unresolvable entries fail the build with a clear message naming the entry; the legacy .config format is either fully migrated (old parser removed) or explicitly deprecated with documented precedence between the two files. @@ -92,6 +92,77 @@ with the G-014 direction of toml + tomlish for declared configuration. kit type stops implying a particular runtime (simplification landing on this format); G-137 - reads punkproject.toml fields at the same bake seam the versioned scheme reads the project version from. + +## Progress + +ACHIEVED 2026-07-31 (activated and implemented same day; punkshell 0.38.0). Delivered: + +- src/runtime/mapvfs.toml (tomlish-parsed; file header = user-facing spec) with + \[kit.\] / \[group.\] / \[scheme.\] tables. All 12 active mapvfs.config + mappings migrated (entry order preserved so per-vfs runtime order and bakelist rows are + unchanged); useful historical comments carried into the toml; mapvfs.config deleted from + punkshell AND the project-0.1 layout template (converted to mapvfs.toml there). +- Reader family in make.tcl under the unchanged G-121 model surface: mapvfs_locate + (toml > legacy > toml-name default; PUNK_MAPVFS_CONFIG env override - the + characterization seam G-123's increment-5 notes asked for), mapvfs_parse dispatching on + extension, mapvfs_parse_toml (strict), mapvfs_parse_config (DEPRECATED fallback kept for + un-migrated trees, parses with a NOTE; documented precedence: toml wins when both exist - + the acceptance's deprecation arm, chosen because layout-generated projects out in the + world still carry mapvfs.config, exactly the complication the Approach recorded). + Shared toml helpers (toml_untag, toml_file_to_dict) hoisted to punkboot::lib (the two + inline toml_untag definitions in the vfslibs/libfetch phases removed). +- Model additions: format, groups, missing_detail, per-entry attrs (entry label, group, + bake_default, scheme, scheme_role) riding as a 6th vfs_spec element + (mapvfs_spec_attrs accessor; legacy tuples unchanged). Consumer dedupe fix: several toml + entries may share one runtime on one vfs - mapvfs_kit_outputs and the kit-loop runtimes + list now enumerate each distinct runtime's spec set once (the multiplication was latent + in the legacy format too, unreachable with one-line-per-runtime configs). +- Validation: fatal entry-named configerrors (unknown key/type/scheme, missing required + keys, bad/conflicting targets, duplicate output names, tomlish/parse failures, unreadable + project version for a scheme) - bake exits 3 before any machinery. Unresolvable-on-disk + split DECIDED: missing vfs folder = fatal per entry (vfs folders are repo content, so + absence means a wrong declaration); missing runtime FILE = entry-named recapped + BUILD-WARNING + skip (runtime stores are ENVIRONMENT - legitimately part-populated on + fresh clones, runtimes arrive via the consent-gated fetch tiers; hard-failing would brick + clean-checkout bakes, and the live tree already carries a tolerated instance: punkmagic's + tclsh90magic absent from the win32-x86_64 store). +- Groups (the Notes' named-groupings direction): verify-ix86 (G-129 kits), declare-demo + (G-115 demo kit), versioned; @ selection in bake/bakelist (mapvfs_match_outputs), + group/default/scheme notes columns + per-kit detail lines in bakelist, groups listed in + unknown-name errors and the argdoc choices. +- G-023 versioned scheme: \[scheme.punk8\] + \[scheme.punk9\] declared in the shipped file + (group versioned, bake_default=false pending G-023 activation - full bakes unchanged); + expansion at parse time from punkproject.toml \[project\] version to -, + -dev and release-gated records that join the model (bakelist/bake + selection per the archived G-121 note). Release gating implemented in the kit loop: plain + name created when absent, otherwise skipped with a message naming G-023's explicit + release step (bin/punk9.exe, a 2024-11 artifact, correctly survives as the + initially-created copy). + +Verification (win32-x86_64 host, tclsh90 9.0.3 runner + Tcl 8.6 tclsh driving make.tcl): +- Model parity: bakelist rows from the legacy file (via PUNK_MAPVFS_CONFIG) vs mapvfs.toml + IDENTICAL for the whole default set (kit/type/runtime/vfs/deployed columns). +- Existing-target build identity: punkluck86 force-rebuilt under BOTH formats - + byte-identical 1.73MB runtime+stamp prefix, identical 2993-member extracted payloads + (zero content diffs). Byte-identity of the whole artifact is not a pipeline property + (consecutive same-config rebuilds differ in zip timestamp metadata at the same offset - + pre-existing, format-independent). Smoke Thread OK in the rebuilt kit. +- Scheme end-to-end: punk9-dev baked and deployed by name from the expansion; punk9 + release-gated skip demonstrated; after the 0.37.0->0.38.0 project bump the expanded rows + track the new version with no config edit (the no-enumeration property live). +- make.tcl workflow (both modes, width-checked), help bake (@group choices render), check + (declared smoke list incl. punkdeclare) all verified; architecture_lint clean. +- Tests: src/tests/shell/testsuites/punkexe suite 98 tests, 0 failures (runtests + -include-paths shell/testsuites/punkexe/***) including 5 new G-024 characterizations in + maketclbakelist.test (@group filter; via the PUNK_MAPVFS_CONFIG seam: scheme expansion + shape, entry-named unknown-key error, bake exit-3 on unknown scheme, legacy deprecation + NOTE) and updated pins in maketclplatform.test / maketclpayloadcheck.test. + +Remaining manual items: none for this goal. Follow-through pointers: G-023 activation +flips the versioned group's bake_default (and owns the explicit release step + plain-name +refresh policy); other live goals' Scope lines naming src/runtime/mapvfs.config or +vendorlib_vfs.toml (G-101, G-127, G-131, G-141) are contract text left untouched - +proposed rewordings flagged in the completion report for user approval. - G-133 (archived) landed the interim per-kit smoke-require declaration as a 5th mapvfs.config entry element (list of packages the built kit must plain-require); the toml conversion is its designated schema home, alongside the verification-set diff --git a/goals/G-025-exe-selfreport.md b/goals/G-025-exe-selfreport.md index a3b93871..2cba0287 100644 --- a/goals/G-025-exe-selfreport.md +++ b/goals/G-025-exe-selfreport.md @@ -78,9 +78,11 @@ Edge cases driving the acceptance clauses: ## Notes -- Related: G-023 (version-named binaries this complements), G-024 (build - mapping that will drive stamping inputs), G-015 (piped invocation of the - subcommand), G-014 (punkproject.toml reading may share tomlish plumbing). +- Related: G-023 (version-named binaries this complements), G-024 (achieved + 2026-07-31 - see goals/archive/G-024-mapvfs-toml.md; the mapvfs.toml mapping + that will drive stamping inputs, incl. its parse-time punkproject.toml + version read), G-015 (piped invocation of the subcommand), G-014 + (punkproject.toml reading may share tomlish plumbing). - Adjacent need surfaced 2026-07-14 (G-030 follow-up, recorded in src/AGENTS.md "pre-loaded interp" note): when make.tcl runs under a built punk executable, the module copies actually executing are the kit's diff --git a/goals/G-026-vendor-provenance-policy.md b/goals/G-026-vendor-provenance-policy.md index 2fdd45cf..69b3ba41 100644 --- a/goals/G-026-vendor-provenance-policy.md +++ b/goals/G-026-vendor-provenance-policy.md @@ -70,8 +70,9 @@ be recorded here: - include_modules.config files carry their own "#todo - change to include_modules.toml" - a natural companion change under the G-024 - toml-conversion pattern, but deliberately not part of this goal's - acceptance. + toml-conversion pattern (achieved 2026-07-31 for the mapvfs surface - see + goals/archive/G-024-mapvfs-toml.md; its strict entry-named tomlish reader is + the template), but deliberately not part of this goal's acceptance. - Related: G-014 (tomlish provenance incident motivated this), G-024 (toml - config direction), G-025 (build stamping machinery a provenance stamp could - ride on). + config direction; achieved - archive path above), G-025 (build stamping + machinery a provenance stamp could ride on). diff --git a/goals/G-035-mixed-tm-pkgindex-provision.md b/goals/G-035-mixed-tm-pkgindex-provision.md index f0321efe..af6ecabf 100644 --- a/goals/G-035-mixed-tm-pkgindex-provision.md +++ b/goals/G-035-mixed-tm-pkgindex-provision.md @@ -65,7 +65,8 @@ is a realistic hazard - and same-version-different-content drift (the extend them), G-033 (achieved 2026-07-28, goals/archive/G-033-proj-mode-cwd-project.md - visitor-mode path mixing is now live, making cross-form collisions more reachable: a kit's internal libs vs a visited project's .tm trees), - G-026/G-024 (same-version drift provenance), punk::packagepreference 0.1.1 + G-026/G-024 (same-version drift provenance; G-024 achieved 2026-07-31 - + goals/archive/G-024-mapvfs-toml.md), punk::packagepreference 0.1.1 (its moduledoc auto-load hook lives in exactly this machinery). - G-112 relationship (recorded 2026-07-24; G-112 achieved 2026-07-25 - see goals/archive/G-112-maketcl-subcommand-rename.md): its rename sweep updated diff --git a/goals/G-065-declarative-vendoring.md b/goals/G-065-declarative-vendoring.md index b4f44aea..e41e6391 100644 --- a/goals/G-065-declarative-vendoring.md +++ b/goals/G-065-declarative-vendoring.md @@ -49,13 +49,18 @@ remains the punkshell format (G-024 direction). ## Notes -- Related: G-004 (binary policy this enforces at vendor time), G-024 (toml direction), +- Related: G-004 (binary policy this enforces at vendor time), G-024 (toml direction; + achieved 2026-07-31 - goals/archive/G-024-mapvfs-toml.md - its strict entry-named + tomlish reader + PUNK_MAPVFS_CONFIG-style test seam are the pattern to reuse), G-026 (local-pull provenance sibling; include_modules.config -> toml note), G-027 (derived-project travel), G-063 (license recording hook at vendor time), G-066 (repackaging consumes declared packages), G-067 (artifact retrieval as an alternative upstream kind), G-068 (moduledoc status tracked against the manifest), - G-115 (sibling toml declaration surface for .vfs payloads - keep the schemas aligned, - per the 2026-07-24 overlap review), G-047 (punkproject.toml [workflow] schema + G-115 (sibling toml declaration surface for .vfs payloads - ACHIEVED 2026-07-31, + goals/archive/G-115-declarative-vfs-composition.md; its per-.vfs [payload.*] key + vocabulary - source/source_root/target/supersedes/replace - is now the live schema + to stay aligned with; this goal's vendor-sync manifest feeds the src-root sources + those declarations consume), G-047 (punkproject.toml [workflow] schema co-tenant - if the manifest lives in punkproject.toml, the two schemas must not collide). - Proving case: tcl_oauth2_library. A hand-trimmed copy (keeping diff --git a/goals/G-115-declarative-vfs-composition.md b/goals/G-115-declarative-vfs-composition.md index 99e72412..a43cd111 100644 --- a/goals/G-115-declarative-vfs-composition.md +++ b/goals/G-115-declarative-vfs-composition.md @@ -1,6 +1,6 @@ # G-115 Declarative .vfs composition: toml-defined kit payloads with drop-in preservation -Status: active +Status: achieved 2026-07-31 Scope: src/make.tcl (vfs assembly); src/runtime/vendorlib_vfs.toml (existing per-package declaration surface - fold/supersede settled in the work); src/vfs/ (per-.vfs declaration files + README); punk::mix machinery as touched; src/project_layouts (seeding for derived projects); coordinates with G-067 (artifact sources); G-006 (consent); G-004 (binary-free committed tree) Goal: a .vfs folder's payload can be DECLARED in a toml definition (sources: suite build products, punkbin runtime/library artifacts via the consent-gated channels, vendor trees) and materialized into the folder by the build - while the folder remains the operative assembly area: undeclared dropped-in files (binary libs/modules included) are preserved with documented precedence, so derived projects with permissive binary policies and gitignored test .vfs folders keep drop-in simplicity with no declaration required. Acceptance: a punkshell kit .vfs (or demonstration .vfs) builds from a toml declaration reproducing its payload on a clean tree, with declared binary content arriving via consented retrieval or local build products; an undeclared dropped-in file survives re-materialization per the documented precedence; a .vfs with NO declaration builds exactly as today (pure drop-in mode unchanged); the declaration format and precedence rules are documented; the vendorlib_vfs.toml relationship is settled with rationale; the layout store seeds the convention for derived projects. @@ -66,3 +66,76 @@ for telling the two apart). libfetch target derivation consumes whichever declaration surface is current; if this goal supersedes vendorlib_vfs.toml the derivation reads the successor surface (recorded in its own Notes). + +## Progress + +ACHIEVED 2026-07-31 (activated and implemented same day, alongside G-024; punkshell +0.38.0). Delivered: + +- Per-.vfs declaration files src/vfs/.vfs.toml (SIBLING of the folder, so never part + of the payload) with \[payload.\] entries: source, source_root (src|packages), + target (relative, inside-the-folder validated), supersedes, replace - the key vocabulary + carried over from the proven vendorlib_vfs.toml surface (sibling grammar to mapvfs.toml + per the G-024 coordination note). Processed by the make.tcl VFSPAYLOAD phase (commands + bakehouse/bake/vfslibs, replacing the old VFSLIBS block; G-121 selective bakes narrow to + the selected kits' folders; strict entry-named errors for unknown keys, missing sources, + bad roots/targets, parse failures). +- PRECEDENCE (the core design work) - drop-in wins, via punkcheck -overwrite + synced-targets (records in src/vfs/.punkcheck, outside payloads): undeclared files are + never touched by construction; a file at a declared path is overwritten only when + punkcheck's records show this mechanism installed it AND it is unmodified since - a + foreign pre-existing copy (no record) or hand-modified install is preserved, with + punkcheck per-file skip lines plus a phase NOTE counting changed-but-preserved files + (computed as files_skipped minus sources_unchanged, so routine unchanged-source skips + stay quiet). supersedes/replace are the explicit declared exceptions (replace semantics + preserved from the old surface: unconditional per-run clean-slate). A .vfs with NO + declaration file is untouched - the phase only globs *.vfs.toml. +- vendorlib_vfs.toml relationship SETTLED: fold/supersede. All six \[install.*\] entries + migrated into per-.vfs files for the 13 participating folders (identical + source/supersedes/replace semantics; per-target vfs lists inverted into per-kit files); + src/runtime/vendorlib_vfs.toml deleted; a leftover copy in any tree is ignored with a + migration warning. Rationale: per-kit files keep each kit's composition readable in one + place and give new kits a copy-a-sibling starting point; per-package cross-kit lists + made adding a kit an every-entry edit; G-141's derivation reads the successor surface + (its Notes anticipated exactly this) and G-127 gets one place to add the platform axis. + The DRY cost (shared packages declared per kit) is accepted - entries are 4-6 lines. +- Consent linkage: source_root="packages" resolves against the untracked bin/packages tier + populated by the consent-gated 'make.tcl libfetch' (declared + sha1-verified; + G-139/G-006 lineage) - declared binary content arrives on a clean tree without being + committed. source_root="src" covers vendor trees and suite build products. +- Demonstration kit punkdeclare (mapvfs.toml \[kit.punkdeclare\], group declare-demo, + bake_default=false, smokerequire udp+tcllibc): VCS carries ONLY the boot fauxlink; the + entire lib_tcl9 binary payload (tcludp1.0.13 from vendorlib_tcl9, tcllibc from + win32-x86_64/tcl9 packages tier) is declaration-materialized and ignored in both VCS + (/src/vfs/punkdeclare.vfs/lib_tcl9/ in .gitignore + derived ignore-glob; ignore-sync + verification run clean). +- Documentation: src/vfs/README.md (format + precedence spec), src/vfs/AGENTS.md, + src/runtime/AGENTS.md, src/AGENTS.md, ARCHITECTURE.md (+lint clean), make.tcl workflow + text + vfslibs summary/helptext. Layout seeding: project-0.1 layout gains + src/vfs/README.md (generic convention doc) + src/runtime/mapvfs.toml; the MACHINERY + reaches derived projects automatically via the established make.tcl thin-layout sync, + and the modpod templates copy follows via the LAYOUT->MODPOD sync on the next + modules-class build. + +Verification (win32-x86_64 host): +- Migration parity: first full VFSPAYLOAD run over all 13 migrated declarations produced + ZERO tracked-content changes under src/vfs (the per-.vfs files reproduce the + vendorlib_vfs.toml-installed payloads exactly). +- Clean-tree reproduction: punkdeclare.vfs/lib_tcl9 deleted, 'bake -confirm 0 punkdeclare' + rematerializes the payload from both source classes; smoke-require OK for udp 1.0.13 AND + tcllibc 2.0 inside the built artifact; the deployed kit boots and plain-requires both + (piped tclsh probe). +- Drop-in survival: an undeclared DROPIN-note.txt AND a foreign hand-placed pkgIndex.tcl + at a declared path both survived re-materialization; the foreign pkgIndex genuinely won + (the kit's udp smoke failed with a recapped BUILD-WARNING until the drop-in was removed + - the G-133 probe catching exactly the payload defect a preserved drop-in can cause). +- No-declaration parity: punk8min.vfs/punk9min.vfs (undeclared) untouched by the phase; + punkluck86 (their consumer) force-rebuilt content-identical (see the G-024 record: + identical prefix + 2993-member payload). +- Suite: punkexe 98 tests 0 failures (includes the updated smoke-declared pin carrying + punkdeclare). + +Remaining manual items: none. Extension points recorded for successors: G-127 adds the +platform axis to these declarations (one file per kit, ready for per-target entries); +G-141's derivation input is now the per-.vfs files' source_root="packages" entries; +per-kit payload detail in bakelist (archived G-121 note) can now read the declaration. diff --git a/goals/G-127-crosstarget-vfs-bake.md b/goals/G-127-crosstarget-vfs-bake.md index e4618077..f2eed621 100644 --- a/goals/G-127-crosstarget-vfs-bake.md +++ b/goals/G-127-crosstarget-vfs-bake.md @@ -99,9 +99,12 @@ exists locally (empty but for sha1sums.txt) and punkbin upstream carries goal adds should read as one vocabulary with the gate's refusal rather than a second, differently-worded failure surface. The gate is deliberately non-executing precisely so it covers cross-target kits this host could never run. -- Related: G-115 - owns the payload DECLARATION FORMAT and is currently written without - any platform axis (zero mentions of platform or target). Whichever of the two lands - second adopts the other: this goal supplies the axis, that goal supplies the format. +- Related: G-115 (achieved 2026-07-31 - see + goals/archive/G-115-declarative-vfs-composition.md) - owned the payload DECLARATION + FORMAT, which is now live as per-.vfs files (src/vfs/.vfs.toml, [payload.*] + entries with source/source_root/target/supersedes/replace) deliberately WITHOUT a + platform axis. This goal lands second and adopts that format: it supplies the axis + (per-target payload entries in those files), the format supplies the surface. - Related: G-023 - owns output NAMES and its Acceptance hardcodes `punk9-0.5.0.exe`. This goal owns output LOCATION and the suffix; the two must compose (`bin/kits//punk9-0.5.0`) rather than compete. @@ -110,7 +113,10 @@ exists locally (empty but for sha1sums.txt) and punkbin upstream carries the TARGET's. Note the xref shows G-114 and G-115 unlinked despite overlapping: the platform axis and the payload axis have never been connected, and this goal is that connection. -- Related: G-024 - format home for expressing one vfs definition against several targets. +- Related: G-024 (achieved 2026-07-31 - see goals/archive/G-024-mapvfs-toml.md) - the + mapvfs.toml format home for expressing one vfs definition against several targets is + live (per-kit [kit.] tables each carrying their own target already allow one vfs + under several targets; the output-collision half stays this goal's). - Related: G-057 (achieved 2026-07-29, archived - see goals/archive/G-057-kit-icon-embedding.md) - icon embedding is a windows-target-only wrap step keyed off a per-vfs override, so "which kits @@ -163,7 +169,8 @@ exists locally (empty but for sha1sums.txt) and punkbin upstream carries directly now that the shared G-122/G-133 references are both archive-tier. - G-123 (archived) recorded a finding this goal's make.tcl surface should note: its end-to-end demonstration (fixture-tier runtime -> non-default-target mapvfs entry -> - bakelist row runtime-present) had to be LIVE-REVERSIBLE because make.tcl has no - alternate-mapvfs-config seam for characterization tests - adding one (or a scratch - bakehouse mode) would let cross-target bake behaviour be pinned without editing the - real config. See its increment-5 Progress notes. + bakelist row runtime-present) had to be LIVE-REVERSIBLE because make.tcl had no + alternate-mapvfs-config seam for characterization tests. RETIRED by G-024 (achieved + 2026-07-31): the PUNK_MAPVFS_CONFIG env var now points bakelist/bake at an alternate + mapping file (used by the G-024 fixture tests in maketclbakelist.test) - this goal's + cross-target characterizations can pin behaviour without editing the real config. diff --git a/goals/G-131-boot-payload-autodetection.md b/goals/G-131-boot-payload-autodetection.md index bb7b1568..9bb981c4 100644 --- a/goals/G-131-boot-payload-autodetection.md +++ b/goals/G-131-boot-payload-autodetection.md @@ -90,9 +90,10 @@ independently-written branches. fewer artifacts to publish, checksum and keep current in the punkbin tiers; the reduction this goal records is directly a reduction in that goal's ongoing work. -- Related: G-024 - if the container becomes a bake-time choice, a mapvfs entry naming a kit - type stops implying a particular runtime, which is a simplification that lands on the toml - format work. +- Related: G-024 (achieved 2026-07-31 - see goals/archive/G-024-mapvfs-toml.md) - if the + container becomes a bake-time choice, a mapvfs entry naming a kit type stops implying a + particular runtime; the toml format ([kit.] tables with a free per-entry type key) + is live and accommodates that simplification when this goal lands. - The `make.tcl` `kit_type_mismatch` check is named in Scope because a runtime that legitimately supports more than one container makes its premise false. It currently probes `runtime_caps` for has_zipfs/has_starkit/has_cookfs and treats disagreement with diff --git a/goals/G-137-kit-version-resources.md b/goals/G-137-kit-version-resources.md index c235dff8..467683c2 100644 --- a/goals/G-137-kit-version-resources.md +++ b/goals/G-137-kit-version-resources.md @@ -74,8 +74,14 @@ sidecar. - Related: G-023 - version-named binaries draw on the same punkproject.toml source; its versioned/dev outputs are rewritten every build, the cadence the seam already handles. -- Related: G-024 - if per-kit version overrides are ever wanted, they land in - its mapvfs toml grammar. +- Related: G-024 (achieved 2026-07-31 - see goals/archive/G-024-mapvfs-toml.md; + mapvfs.toml is live) - if per-kit version overrides are ever wanted, they land + in its mapvfs toml grammar. +- Related: G-127 - its bin/kits// output relocation moves the artifacts + this stamping seam emits; stamp + sidecar emission must follow the artifact's + location (the same per-target wrap-seam composition question G-127's G-057 icon + note records). Bridge recorded directly 2026-07-31 when G-024 - previously the + pair's only shared live reference - was archived. - Related: G-028 - a locked kit exe blocks a resource update exactly as it blocks deploy; its locker-report direction applies to version stamping failures too. diff --git a/goals/G-141-libfetch-target-selection.md b/goals/G-141-libfetch-target-selection.md index ebc377c1..8d79ec16 100644 --- a/goals/G-141-libfetch-target-selection.md +++ b/goals/G-141-libfetch-target-selection.md @@ -56,7 +56,9 @@ has no equivalent of "materialize what this checkout needs". the failing bake needs. - Coordinate surfaces, not formats: G-127 owns per-target kit payload declaration and G-115 the declaration format - the derivation consumes - whichever declaration surface is current (vendorlib_vfs.toml today). + whichever declaration surface is current (the per-.vfs + src/vfs/.vfs.toml files since G-115's 2026-07-31 supersession of + vendorlib_vfs.toml - see the Notes update). ## Alternatives considered @@ -92,8 +94,14 @@ has no equivalent of "materialize what this checkout needs". - Related: G-127 - owns per-target kit payload declaration and bake output targeting; whichever lands second adopts the other (its per-target declarations become this derivation's input). -- Related: G-115 - owns the payload declaration format; if it supersedes - vendorlib_vfs.toml the derivation reads the successor surface. +- Related: G-115 (achieved 2026-07-31 - see + goals/archive/G-115-declarative-vfs-composition.md) - it DID supersede + vendorlib_vfs.toml: the kit-consumption input for this goal's derivation is + now the per-.vfs declaration files (src/vfs/.vfs.toml), whose + source_root="packages" entries carry the same -prefixed source paths + the derivation unions (punk9linux.vfs.toml declares the linux-x86_64 tcllibc, + preserving the cross-target-consumption example in Context). The Scope's + vendorlib_vfs.toml reference reads as that successor surface. - Related: G-004 - the growth driver: each vendored tree migrating to the tier multiplies declarations; selection keeps per-host cost proportional to consumption. @@ -108,5 +116,6 @@ has no equivalent of "materialize what this checkout needs". src/tests/shell/testsuites/punkexe - strongest live intersections G-127 and G-115 (recorded above); G-005/G-023/G-024/G-101/G-131/G-077/G-061 surfaced as sibling-path adjacency only (G-024's format interest arrives via the - G-127/G-115 coordination). Non-path relations G-006/G-065/G-004/G-123/G-027 + G-127/G-115 coordination; G-024 achieved 2026-07-31 - + goals/archive/G-024-mapvfs-toml.md). Non-path relations G-006/G-065/G-004/G-123/G-027 from the libfetch reference sweep. diff --git a/punkproject.toml b/punkproject.toml index e020ff64..45fc3911 100644 --- a/punkproject.toml +++ b/punkproject.toml @@ -1,6 +1,6 @@ [project] name = "punkshell" -version = "0.37.0" +version = "0.38.0" license = "BSD-2-Clause" url = "https://www.gitea1.intx.com.au/jn/punkshell" #packager: declared identity for published artifacts (declarative, not proof - diff --git a/src/AGENTS.md b/src/AGENTS.md index baee91f7..4a7c0fe4 100644 --- a/src/AGENTS.md +++ b/src/AGENTS.md @@ -72,11 +72,11 @@ Recovery after a wrong path guess: - make.tcl subcommands and flags are punk::args-declared (G-030): `tclsh src/make.tcl help ?subcommand?` or ` -help` shows tabled usage; invalid arguments produce punk::args usage errors. The dispatch degrades to plain scan/help when the bootsupport punk::args (or its rendering stack) is stale or unavailable; `PUNKBOOT_PLAIN=1` forces the degraded mode for troubleshooting. The `::punkboot::argdoc` definitions are constructed (string-built) and use the G-045 authoring mechanisms — block-form help values + `@normalize`, explicit `@form -synopsis` — see src/modules/AGENTS.md "punk::args definition authoring ergonomics" before editing them or authoring similar definitions. - Uncommitted `make.tcl`-generated outputs may be batched into one commit, regardless of how many `make.tcl` invocations produced them. This covers punkcheck-managed build outputs that are VCS-tracked: `src/bootsupport/`, `_vfscommon.vfs/modules` + `/lib`, declared per-kit `*.vfs/lib_tcl/` subfolders, the thin-layout sync copies (`src/project_layouts/vendor/punk//src/{make.tcl,build.tcl}` + bootsupport manifest copies), and the module-shipped layout payload under `src/modules/punk/mix/#modpod-templates-*/templates/project_layouts/` (G-087: synced from `src/project_layouts` - never hand-edit the modpod copy). (Root `modules/`, `lib/`, `modules_tcl/`, `lib_tcl/` are gitignored and never committed.) Each module's old→new version rename is atomic; a regeneration reflects one build-output refresh, not independent hand-edits. This is a carve-out from generic "split aggressively" commit guidance for punkcheck-managed build outputs only — it does not cover user-curated VFS content (`_config/`, per-kit `*.vfs/` outside declared `lib_tcl` subfolders, `_aside/`, `mkzipfix.vfs`, `_vfscommon.vfs/doc`), which commits separately per its own concerns. - **Boot-precondition gate (G-125).** A bake refuses any kit whose merged `.vfs` supplies no tcl library: the kit is listed under `FAILED KITS` with a reason naming the cause, and NOTHING is written - no `src/_build/`, no deploy, and the previously deployed `bin/` is left byte-identical. This is deliberately a failure rather than a warning: the deploy step deletes the old kit before copying the new one, so the pre-gate behaviour replaced a working shell with an artifact that died at startup with `application-specific initialization failed: Cannot find a usable init.tcl`. The check is structural and non-executing (so it covers cross-target kits too): `init.tcl` in `tcl_library/`, `lib/tcl./` or `tcl./` - the third for runtimes whose archive mounts at the executable's own path rather than `//zipfs:/app`, so `[info library]` is `/tcl8.6` (the androwish/undroidwish zipfs backport for 8.6 does this) - plus one companion file a real Tcl library carries beside it (`tm.tcl`, `package.tcl`, `auto.tcl`, `clock.tcl`, `history.tcl`, `word.tcl`) or an `encoding/` directory - without that second test the `lib/BWidget1.10.1/init.tcl` every punkshell kit carries would answer for a tcl library it cannot provide. It reads the MERGED tree rather than asking whether extraction ran, so a `.vfs` that legitimately supplies its own tcl library (`src/vfs/punk8_statictwapi.vfs`, `src/vfs/punk9test.vfs`) builds and deploys unchanged. **Remedy when a kit is refused:** the usual cause is that the runtime's own payload could not be extracted - read the recapped `BUILD-WARNING` naming what was tried, then either fix extraction (a driving tclsh with no zipfs needs punk::zip >= 0.2.0 in bootsupport - see the extraction bullet above), point the kit at a runtime that carries a payload, or have the kit's `.vfs` supply a tcl library itself. The predicate is `punkboot::utils::vfs_boot_library_report` called through a guarded require, exactly as the dirty-src provenance check is: a stale or missing bootsupport snapshot degrades the gate to a `NOTE` rather than failing every kit. `tclsh src/make.tcl check` reports whether the gate is `ACTIVE` or `UNAVAILABLE`. -- **Payload/target consistency checks (G-133), both ADVISORY** - warnings are recapped `BUILD-WARNING`s and the kit still builds and deploys (unlike the G-125 gate). (a) **Binary-arch scan**, at the same post-merge seam as the gate: every binary library (`*.dll`/`*.so`/`*.dylib`) in the merged tree is classified by header bytes (`punkboot::utils::binary_arch_classify` - PE Machine field, ELF e_machine, Mach-O incl. universal; unclassifiable content is honestly `unknown` and never warned about) against the kit's target platform. Structural and non-executing, so it runs for cross-target kits too (measured ~30-140ms per kit, avg ~93ms across the 12 assembled trees). Libraries under a platform-discriminated subdir - canonical `-` platform-dir names, the universal `macosx` folder, or a recognised vendor spelling (`win-x64` etc; the lists are namespace variables in `punkboot::utils`) - are exempt: multi-arch payloads are legitimate there. First real sweep (2026-07-27) found a true latent instance: `zint.dll` 2.13.0 in `src/vfs/punk8win.vfs/lib_tcl8/` is 32-bit (i386) and can never load in the x64 tcl8 kits (punk86/punkbi/punksys) that carry it - those bakes warn until the payload is fixed. (b) **Smoke-require probe**: a kit declaring packages in `src/runtime/mapvfs.config` (5th entry element - see `src/runtime/AGENTS.md`) has each one plain-`package require`d INSIDE the freshly built artifact via its tclsh subcommand, which exits cleanly even on runtimes whose full repl teardown is fragile. This is the only check that observes actual package RESOLUTION - the punkluck86 2026-07-27 incident class (a higher-versioned wrong-arch Thread shadowing the runtime's working copy on plain require) is invisible to structural checks because the working file exists too. Failures are recapped naming kit, package and the actual error; cross-target kits skip with a stated reason; undeclared kits run nothing new; the probe only runs when the kit actually rebuilds. **What the checks do NOT guarantee:** statically linked packages, pure-tcl packages whose binary dependency lives elsewhere, and version-preference outcomes are visible only to the smoke probe - and only for the declared package set; the arch scan proves nothing about loadability beyond architecture, and an `unknown` classification is silence, not a pass. Both degrade to a NOTE when the bootsupport `punkboot::utils` snapshot is stale (same guarded require as the G-125 gate); `tclsh src/make.tcl check` reports the scan's ACTIVE/UNAVAILABLE state and the currently declared smoke-require kits. +- **Payload/target consistency checks (G-133), both ADVISORY** - warnings are recapped `BUILD-WARNING`s and the kit still builds and deploys (unlike the G-125 gate). (a) **Binary-arch scan**, at the same post-merge seam as the gate: every binary library (`*.dll`/`*.so`/`*.dylib`) in the merged tree is classified by header bytes (`punkboot::utils::binary_arch_classify` - PE Machine field, ELF e_machine, Mach-O incl. universal; unclassifiable content is honestly `unknown` and never warned about) against the kit's target platform. Structural and non-executing, so it runs for cross-target kits too (measured ~30-140ms per kit, avg ~93ms across the 12 assembled trees). Libraries under a platform-discriminated subdir - canonical `-` platform-dir names, the universal `macosx` folder, or a recognised vendor spelling (`win-x64` etc; the lists are namespace variables in `punkboot::utils`) - are exempt: multi-arch payloads are legitimate there. First real sweep (2026-07-27) found a true latent instance: `zint.dll` 2.13.0 in `src/vfs/punk8win.vfs/lib_tcl8/` is 32-bit (i386) and can never load in the x64 tcl8 kits (punk86/punkbi/punksys) that carry it - those bakes warn until the payload is fixed. (b) **Smoke-require probe**: a kit declaring packages in `src/runtime/mapvfs.toml` (`smokerequire` key - see `src/runtime/AGENTS.md`) has each one plain-`package require`d INSIDE the freshly built artifact via its tclsh subcommand, which exits cleanly even on runtimes whose full repl teardown is fragile. This is the only check that observes actual package RESOLUTION - the punkluck86 2026-07-27 incident class (a higher-versioned wrong-arch Thread shadowing the runtime's working copy on plain require) is invisible to structural checks because the working file exists too. Failures are recapped naming kit, package and the actual error; cross-target kits skip with a stated reason; undeclared kits run nothing new; the probe only runs when the kit actually rebuilds. **What the checks do NOT guarantee:** statically linked packages, pure-tcl packages whose binary dependency lives elsewhere, and version-preference outcomes are visible only to the smoke probe - and only for the declared package set; the arch scan proves nothing about loadability beyond architecture, and an `unknown` classification is silence, not a pass. Both degrade to a NOTE when the bootsupport `punkboot::utils` snapshot is stale (same guarded require as the G-125 gate); `tclsh src/make.tcl check` reports the scan's ACTIVE/UNAVAILABLE state and the currently declared smoke-require kits. - **Kit offset-style pin (G-134), ADVISORY** - at the same seam family, every ASSEMBLED kit image is probed with `punk::zip::archive_info` (via `punkboot::utils::kit_offsetstyle_report`, same guarded-require degradation to a NOTE; `make.tcl check` reports ACTIVE/UNAVAILABLE): a FILE-relative attached zip payload earns a recapped `BUILD-WARNING` naming the kit, because the pipeline emits ARCHIVE-relative by construction and the G-128 stamper refuses file-relative by default - a warning here means an assembly path regressed the output convention. **What the pin does NOT cover:** `plain` and `none` results (a bare zip, or no attached zip at all - the metakit kit shape, and any unreadable input) are silence, never warnings - the pin asserts the offset CONVENTION of an attached zip, not the presence of one; and it asserts only what make.tcl's own assembly paths EMIT - `punk::zip::mkzip -offsettype file` (and modpod's equivalent) remain available for deliberate manual creation of file-relative artifacts, and file-relative INPUT artifacts remain tolerated by the G-124 reader and normalized by any re-bake. - Kit bakes consume the punk-runtime WORKING COPIES under `bin/runtime//`: `make.tcl bake`/`bakehouse` emit a `BUILD-WARNING:` (recapped at end of run) when a wrapped runtime's beside-toml revision is older than an `-r` artifact present in the same folder - the forgot-to-switch-back guard for deliberate `punk-runtime use ` testing excursions. Heed it before trusting freshly baked family kits; `bin/punk-runtime.cmd use >` (`.exe` suffix optional) re-materializes the working copy. -- `tclsh src/make.tcl bakelist ?kitname ...?` (G-121) reports the kit outputs configured in `src/runtime/mapvfs.config`: kit name, kit type, runtime (with presence in the runtime store), vfs folder, and the deployed state of `bin/` vs the `src/_build` build product (`current|stale|absent|nobuild`), with anomalies (`runtime=missing`, `vfs=missing`, `rtrev=r` materialization staleness) and the cross-target marker `target=` in a trailing notes column. A nonexistent store tier is flagged loudly per tier actually referenced (header `(FOLDER MISSING)` for the default tier + stderr derivation; bake's no-runtimes exit self-diagnoses the same way). Name arguments filter the report and add a per-kit detail block (resolved store tier, target and provenance of the target, paths/sizes/mtimes). `make.tcl bake ?kitname ...?` bakes and deploys only the named kits - other kits' `_build`/`bin` artifacts and punkcheck records are untouched and the vfslibs phase narrows to the named kits' vfs folders; an unknown name errors before any build, listing the configured names; flags go before kit names (`make.tcl bake -confirm 0 punk91`). Bare `bake` processes all configured kits as before. Both surfaces consume the shared parsed-mapping helpers (`::punkboot::lib::mapvfs_*`) rather than the file format, so the G-024 config-format work swaps the reader underneath. Piped characterization: `src/tests/shell/testsuites/punkexe/maketclbakelist.test`. -- **Host vs target platform (G-122).** Everything a bake EMITS is keyed by the artifact's TARGET platform, never by the driving tclsh's personality: the `bin/runtime/` store the runtime is read from, `.exe` suffixing of runtime files and kit outputs, the presence checks behind `runtime=missing`, and whether the pre-deploy process sweep uses `tasklist`/`taskkill` or `ps`/`kill`. Host semantics - copy commands, path handling, filesystem case rules, prompts - stay keyed to the host. The default target is the host's own platform canon, EXCEPT a cygwin-family host (an msys2/cygwin-runtime tclsh, which reports `tcl_platform(platform)` `unix` on windows and canonizes as `msys-x86_64`/`cygwin-x86_64`), which targets `win32-x86_64`: such a host now drives the identical kit set, names and store addressing as a native tclsh. `tclsh src/make.tcl check` prints the derivation on one `platform (G-122): host=... target=... store=... exe-suffix=... process-tooling=...` line. A mapvfs entry may declare its own target platform (4th element - see `src/runtime/AGENTS.md`), which is then read from that platform's tier, named with that platform's executable convention, and skipped by the process sweep (its processes are not visible to this host). Traps this closes, all field-verified 2026-07-26: msys `ps` cannot see a natively-launched kit (only `tasklist` can), and msys2 rewrites arguments that look like absolute posix paths when spawning a native windows program - so `taskkill /PID ` arrived as `taskkill C://PID `; native-windows command lines therefore go through `::punkboot::exec_nativeargs` (sets `MSYS2_ARG_CONV_EXCL=*`, inert elsewhere). Also note a cygwin-family tclsh is a POSIX Tcl: name the script the way that shell spells paths (`tclsh /c/repo/.../src/make.tcl`, not `C:/repo/...`, which it resolves relative to the cwd) - make.tcl diagnoses the windows-spelling case. Piped characterization: `src/tests/shell/testsuites/punkexe/maketclplatform.test` (the cygwin-host tests self-gate on finding and probing a real msys/cygwin tclsh; `PUNK_MSYS_TCLSH` names one explicitly). +- `tclsh src/make.tcl bakelist ?kitname|@group ...?` (G-121/G-024) reports the kit outputs configured in `src/runtime/mapvfs.toml`: kit name, kit type, runtime (with presence in the runtime store), vfs folder, and the deployed state of `bin/` vs the `src/_build` build product (`current|stale|absent|nobuild`), with anomalies (`runtime=missing`, `vfs=missing`, `rtrev=r` materialization staleness), the cross-target marker `target=`, groups (`group=`), full-bake exclusion (`default=no`) and scheme roles (`scheme=versioned|dev|release`) in a trailing notes column. A nonexistent store tier is flagged loudly per tier actually referenced (header `(FOLDER MISSING)` for the default tier + stderr derivation; bake's no-runtimes exit self-diagnoses the same way). Name/@group arguments filter the report and add a per-kit detail block (resolved store tier, target and provenance of the target, config entry, group, paths/sizes/mtimes). `make.tcl bake ?kitname|@group ...?` bakes and deploys only the named kits - other kits' `_build`/`bin` artifacts and punkcheck records are untouched and the vfslibs phase narrows to the named kits' vfs folders; an unknown name errors before any build, listing the configured names and groups; flags go before kit names (`make.tcl bake -confirm 0 punk91`). Bare `bake` processes all configured kits except `bake_default=false` entries; a `versioned`-scheme release output (plain `punk` name) is created when absent and otherwise skipped - a normal bake never overwrites it (the explicit release step is G-023's). Both surfaces consume the shared parsed-mapping helpers (`::punkboot::lib::mapvfs_*`) rather than the file format (G-024: toml reader canonical, deprecated `mapvfs.config` line reader retained as fallback; `PUNK_MAPVFS_CONFIG` env points at an alternate config for characterization). Piped characterization: `src/tests/shell/testsuites/punkexe/maketclbakelist.test`. +- **Host vs target platform (G-122).** Everything a bake EMITS is keyed by the artifact's TARGET platform, never by the driving tclsh's personality: the `bin/runtime/` store the runtime is read from, `.exe` suffixing of runtime files and kit outputs, the presence checks behind `runtime=missing`, and whether the pre-deploy process sweep uses `tasklist`/`taskkill` or `ps`/`kill`. Host semantics - copy commands, path handling, filesystem case rules, prompts - stay keyed to the host. The default target is the host's own platform canon, EXCEPT a cygwin-family host (an msys2/cygwin-runtime tclsh, which reports `tcl_platform(platform)` `unix` on windows and canonizes as `msys-x86_64`/`cygwin-x86_64`), which targets `win32-x86_64`: such a host now drives the identical kit set, names and store addressing as a native tclsh. `tclsh src/make.tcl check` prints the derivation on one `platform (G-122): host=... target=... store=... exe-suffix=... process-tooling=...` line. A mapvfs entry may declare its own target platform (`target` key - see `src/runtime/AGENTS.md`), which is then read from that platform's tier, named with that platform's executable convention, and skipped by the process sweep (its processes are not visible to this host). Traps this closes, all field-verified 2026-07-26: msys `ps` cannot see a natively-launched kit (only `tasklist` can), and msys2 rewrites arguments that look like absolute posix paths when spawning a native windows program - so `taskkill /PID ` arrived as `taskkill C://PID `; native-windows command lines therefore go through `::punkboot::exec_nativeargs` (sets `MSYS2_ARG_CONV_EXCL=*`, inert elsewhere). Also note a cygwin-family tclsh is a POSIX Tcl: name the script the way that shell spells paths (`tclsh /c/repo/.../src/make.tcl`, not `C:/repo/...`, which it resolves relative to the cwd) - make.tcl diagnoses the windows-spelling case. Piped characterization: `src/tests/shell/testsuites/punkexe/maketclplatform.test` (the cygwin-host tests self-gate on finding and probing a real msys/cygwin tclsh; `PUNK_MSYS_TCLSH` names one explicitly). - Zip-type kit assembly does not require zipfs in the driving tcl (G-122): with `tcl::zipfs::mkimg` present it is used as before, otherwise the image is assembled by concatenation (raw-runtime split + `punk::zip::mkzip` + append, the same `::punkboot::assemble_zipcat_image` helper the `zipcat` kit type uses). Both mount identically - tcl zipfs reads the archive with archive-start-relative offsets. - Neither is EXTRACTING the runtime's own attached zip (needed to carry its `tcl_library` into the kit): `punk::zip` >= 0.2.0 reads a zip with stock Tcl only - no zipfs, no vfs::zip, no tcllib (G-124, the former tcllib `zipfile::decode` dependency is gone). The zipfs-less bake path splits off the executable prefix with `punk::zip::extract_preamble` and then reads the MEMBERS from the original runtime with `punk::zip::unzip`, never from the split-off intermediate: a runtime whose zip offsets are file-relative (the historical `zipfs mkimg` convention - `tclsh90b4_piperepl.exe` in the store is one) splits into a .zip whose offsets still count from the removed prefix, which plain zip readers reject. Reading the whole file at the derived base offset makes the two offset conventions indistinguishable to callers; `punk::zip::archive_info ` reports which one a given artifact uses. Verified 2026-07-26 by baking a zip kit from msys2's `/usr/bin/tclsh8.6` (no zipfs, no vfs::zip, no tcllib) and booting the result with its `tcl_library` present. - What the RUNTIME must provide to be zip-kit-wrappable - which zipfs commands, mount conventions and startup hooks are supported - is documented in `bin/AGENTS.md` "Kit-wrappable runtime requirements (G-129)". Short form: `tcl::zipfs::mount` with pairwise no-arg output (`tcl::zipfs::root` NOT required), any attached-archive mount point (the boot derives it from the runtime's own mount table - the androwish/undroidwish 8.6 backport mounts at the executable's own path), a `main.tcl`/`app/main.tcl` startup hook, and a liftable library payload for the G-125 gate. The bake's runtime capability probe keys `has_zipfs` on `tcl::zipfs::mount` accordingly. diff --git a/src/buildsuites/suite_tcl90/README.md b/src/buildsuites/suite_tcl90/README.md index 49e82cb1..027fe1d4 100644 --- a/src/buildsuites/suite_tcl90/README.md +++ b/src/buildsuites/suite_tcl90/README.md @@ -178,7 +178,7 @@ ITS project's manifest): suite.tcl passes the options through `-zigargs` (e.g `-zigargs {-Dprojecturl=https://...}`). Consumers: working-name runtimes are copied into `bin/runtime/win32-x86_64/` -and referenced by `src/runtime/mapvfs.config` (punk -> punk9wintk905.vfs as +and referenced by `src/runtime/mapvfs.toml` (punk -> punk9wintk905.vfs as `punk9_beta`, punk-bi -> punk9win_for_tkruntime.vfs as `punk9bi_beta`, per the *_beta trial convention). diff --git a/src/make.tcl b/src/make.tcl index a55eda23..25e97c3b 100644 --- a/src/make.tcl +++ b/src/make.tcl @@ -627,11 +627,15 @@ proc ::punkboot::lib::bootsupport_prompt_yesno {prompt} { } # -- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- -# Parsed kit-mapping model (G-121). One reader for src/runtime/mapvfs.config - -# the 'bake' kit machinery, the 'bakelist' report and the bake/bakelist argdoc -# choices all consume the model these helpers return, never the file format -# directly, so a config-format change (G-024 toml direction) swaps in underneath -# by reimplementing mapvfs_parse alone. +# Parsed kit-mapping model (G-121). One reader family for the kit-output mapping +# config - the 'bake' kit machinery, the 'bakelist' report and the bake/bakelist +# argdoc choices all consume the model these helpers return, never the file +# format directly. G-024: the canonical format is src/runtime/mapvfs.toml +# (tomlish-parsed [kit.] / [group.] / [scheme.] tables - the +# file's own header comment is the user-facing spec); the pre-G-024 line format +# src/runtime/mapvfs.config remains readable as a DEPRECATED fallback so +# un-migrated trees (older generated projects) keep baking. mapvfs_locate picks +# the file (toml wins when both exist); mapvfs_parse dispatches on extension. # # mapvfs_parse: pure parse - no output, no exits. Every store address, presence # probe and artifact suffix it computes is keyed by the entry's TARGET platform @@ -639,39 +643,423 @@ proc ::punkboot::lib::bootsupport_prompt_yesno {prompt} { # (bin/runtime) and each entry addresses $rtbase/. Returns a dict: # exists 0|1 (mapfile present) # mapfile path as supplied +# format toml|legacy (which reader produced the model) # default_target target platform for entries that declare none -# runtime_vfs_map runtime -> raw vfs_specs as listed (appname/type/target may be "") -# vfs_runtime_map vfstail -> list of {runtime appname type target smokerequires} -# (defaults applied; only vfs folders that exist on disk - matching -# the historical inline parse the kit loop was built around; -# smokerequires is the G-133 smoke-require package list, may be "") +# runtime_vfs_map runtime -> list of vfs_specs. legacy: raw specs as listed +# (appname/type/target may be ""). toml: normalized 6-element +# specs {vfstail appname type target smokerequires attrs} +# vfs_runtime_map vfstail -> list of {runtime appname type target smokerequires +# ?attrs?} (defaults applied; only vfs folders that exist on +# disk - matching the historical inline parse the kit loop was +# built around; smokerequires is the G-133 smoke-require package +# list, may be ""). attrs (6th element, toml models only - read +# via mapvfs_spec_attrs) is a dict: entry (config table label +# e.g kit.punk91), group, bake_default, scheme, scheme_role. # runtime_target runtime -> resolved target platform (all runtimes, mapped order) +# groups group name -> {description bake_default 0|1} (toml only) # missing names of referenced-but-absent runtimes/vfs folders +# missing_detail structured missing items (toml only): list of dicts +# {kind runtime|vfs label name target } +# - the bake path's strict per-entry failure reporting (G-024) # warnings warning lines in encounter order, ready to print verbatim # (missing-item lines carry their historical WARNING: prefix) # badentries subset of warnings: malformed entry lines (these have no kit # record to carry them, so bakelist prints them; missing-item # warnings are instead surfaced as per-row state there) -# configerrors fatal config problems (duplicate runtime line; unusable or +# configerrors fatal config problems (malformed/unresolvable declarations, +# each naming its entry; duplicate runtime line; unusable or # self-contradicting target platform; caller decides whether to # abort - the bake path keeps its historical exit 3) -proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} { - set model [dict create\ + +# Locate the kit-mapping config for a src/runtime folder (G-024). +# Precedence: PUNK_MAPVFS_CONFIG env override (test/characterization seam - an +# alternate config exercised without editing the real one; relative paths +# resolve against the runtime folder), then mapvfs.toml, then the deprecated +# mapvfs.config. When neither exists the toml name is returned so no-mapping +# messages name the current format. +proc ::punkboot::lib::mapvfs_locate {rt_sourcefolder} { + if {[info exists ::env(PUNK_MAPVFS_CONFIG)] && $::env(PUNK_MAPVFS_CONFIG) ne ""} { + set override $::env(PUNK_MAPVFS_CONFIG) + if {[file pathtype $override] ne "absolute"} { + set override [file join $rt_sourcefolder $override] + } + return $override + } + if {[file exists $rt_sourcefolder/mapvfs.toml]} { + return $rt_sourcefolder/mapvfs.toml + } + if {[file exists $rt_sourcefolder/mapvfs.config]} { + return $rt_sourcefolder/mapvfs.config + } + return $rt_sourcefolder/mapvfs.toml +} + +#tomlish::to_dict values are type-tagged (e.g {type STRING value x}, {type ARRAY value {...}}); +#tables are plain nested dicts. Shared by the mapvfs toml reader, the per-.vfs +#payload declarations (G-115) and any other build-side toml consumption. +proc ::punkboot::lib::toml_untag {tagged} { + if {[dict get $tagged type] eq "ARRAY"} { + set plain [list] + foreach el [dict get $tagged value] { + lappend plain [::punkboot::lib::toml_untag $el] + } + return $plain + } + return [dict get $tagged value] +} + +# Read + parse a toml file via tomlish. Returns {ok } or {error }. +proc ::punkboot::lib::toml_file_to_dict {path} { + if {[catch {package require tomlish} tomlish_err]} { + return [list error "tomlish package unavailable ($tomlish_err)"] + } + if {[catch { + set fd [open $path r] + fconfigure $fd -encoding utf-8 + set tomldata [read $fd] + close $fd + } readerr]} { + catch {close $fd} + return [list error "cannot read $path ($readerr)"] + } + if {[catch {tomlish::to_dict [tomlish::from_toml $tomldata]} parsed]} { + return [list error "toml parse failed for $path\n$parsed"] + } + return [list ok $parsed] +} + +# attrs element of a parsed vfs_spec tuple (6th element, toml models). Legacy +# models carry no attrs - callers get an empty dict and defaults apply +# (group "", bake_default 1, no scheme role). +proc ::punkboot::lib::mapvfs_spec_attrs {vfs_spec} { + if {[llength $vfs_spec] >= 6} { + return [lindex $vfs_spec 5] + } + return [dict create] +} + +proc ::punkboot::lib::mapvfs_model_new {mapfile default_target format} { + return [dict create\ exists 0\ mapfile $mapfile\ + format $format\ default_target $default_target\ runtime_vfs_map [dict create]\ vfs_runtime_map [dict create]\ runtime_target [dict create]\ + groups [dict create]\ missing [list]\ + missing_detail [list]\ warnings [list]\ badentries [list]\ configerrors [list]\ ] +} + +proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} { + if {[string tolower [file extension $mapfile]] eq ".toml"} { + return [mapvfs_parse_toml $mapfile $rtbase $sourcefolder $default_target] + } + return [mapvfs_parse_config $mapfile $rtbase $sourcefolder $default_target] +} + +# G-024 canonical reader: mapvfs.toml. Strict per-entry validation - a malformed +# or unresolvable declaration lands in configerrors naming its table (the bake +# path aborts on configerrors; missing-on-disk runtime/vfs items land in +# warnings + missing_detail and the bake path decides fatality per entry). +proc ::punkboot::lib::mapvfs_parse_toml {mapfile rtbase sourcefolder default_target} { + set model [mapvfs_model_new $mapfile $default_target toml] if {![file exists $mapfile]} { return $model } dict set model exists 1 + set legacy_sibling [file join [file dirname $mapfile] mapvfs.config] + if {[file exists $legacy_sibling] && [file tail $mapfile] eq "mapvfs.toml"} { + dict lappend model warnings "NOTE: both mapvfs.toml and mapvfs.config present in [file dirname $mapfile] - mapvfs.toml wins and the deprecated mapvfs.config is ignored (remove it, or keep it only as reference)." + } + lassign [toml_file_to_dict $mapfile] tstatus tpayload + if {$tstatus ne "ok"} { + dict lappend model configerrors "CONFIG FILE ERROR. $mapfile: $tpayload" + return $model + } + set kit_types [list kit zip zipcat cookit cookfs] + set groups [dict create] + set configerrors [list] + set warnings [list] + set missing [list] + set missing_detail [list] + #top-level tables: kit / group / scheme only + dict for {topkey -} $tpayload { + if {$topkey ni {kit group scheme}} { + lappend configerrors "CONFIG FILE ERROR. $mapfile: unknown top-level table or key '$topkey' (expected \[kit.\], \[group.\], \[scheme.\])" + } + } + #-- groups + if {[dict exists $tpayload group]} { + dict for {gname gentry} [dict get $tpayload group] { + set label "group.$gname" + set gdesc "" + set gdefault 1 + dict for {k v} $gentry { + switch -- $k { + description {set gdesc [toml_untag $v]} + bake_default { + set gdefault [toml_untag $v] + if {![string is boolean -strict $gdefault]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: 'bake_default' must be boolean (got '$gdefault')" + set gdefault 1 + } + } + default { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown key '$k' (expected description, bake_default)" + } + } + } + dict set groups $gname [dict create description $gdesc bake_default [expr {bool($gdefault)}]] + } + } + #-- normalized candidate entries: {label rtname vfstail appname kittype declared_target smokes attrs} + set entries [list] + set appnames_seen [dict create] ;#appname -> label + set entry_reader {{model_varname label entry required_type} { + #shared [kit.*]/[scheme.*] field reader; returns a dict or empty on fatal field errors + upvar 1 configerrors configerrors kit_types kit_types groups groups mapfile mapfile + set fields [dict create runtime "" vfs "" type "" target "" smokerequire [list] group "" bake_default "" scheme "" prefix ""] + set badentry 0 + dict for {k v} $entry { + switch -- $k { + runtime - vfs - type - target - group - scheme - prefix { + dict set fields $k [::punkboot::lib::toml_untag $v] + } + smokerequire { + if {[dict get $v type] eq "ARRAY"} { + dict set fields smokerequire [::punkboot::lib::toml_untag $v] + } else { + dict set fields smokerequire [list [::punkboot::lib::toml_untag $v]] + } + } + bake_default { + set bd [::punkboot::lib::toml_untag $v] + if {![string is boolean -strict $bd]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: 'bake_default' must be boolean (got '$bd')" + set badentry 1 + } else { + dict set fields bake_default [expr {bool($bd)}] + } + } + default { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown key '$k'" + set badentry 1 + } + } + } + foreach reqkey {runtime vfs} { + if {[dict get $fields $reqkey] eq ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile is missing required key '$reqkey'" + set badentry 1 + } + } + set ktype [dict get $fields type] + if {$ktype eq ""} { + if {$required_type} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile is missing required key 'type' (scheme entries declare their kit type explicitly; one of: [join $kit_types {, }])" + set badentry 1 + } else { + dict set fields type kit + } + } elseif {$ktype ni $kit_types} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown kit type '$ktype' (expected one of: [join $kit_types {, }])" + set badentry 1 + } + set declared [dict get $fields target] + if {$declared ne ""} { + set problem [::punkboot::lib::platform_name_problem $declared] + if {$problem ne ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile declares target platform '$declared' - $problem." + set badentry 1 + } + } + #strip a superfluous windows .exe from the runtime name (legacy-format parity) + set rtname [dict get $fields runtime] + if {[string match -nocase *.exe $rtname]} { + dict set fields runtime [string range $rtname 0 end-4] + } + set g [dict get $fields group] + if {$g ne "" && ![dict exists $groups $g]} { + #a group referenced without its own [group.] table is implicitly plain + dict set groups $g [dict create description "" bake_default 1] + } + if {$badentry} { + return [dict create] + } + return $fields + }} + if {[dict exists $tpayload kit]} { + dict for {kname kentry} [dict get $tpayload kit] { + set label "kit.$kname" + set fields [apply $entry_reader model $label $kentry 0] + if {![dict size $fields]} { + continue + } + foreach badkey {scheme prefix} { + if {[dict get $fields $badkey] ne ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: key '$badkey' is only valid in \[scheme.\] entries" + } + } + set g [dict get $fields group] + set bd [dict get $fields bake_default] + if {$bd eq ""} { + set bd [expr {$g ne "" ? [dict get $groups $g bake_default] : 1}] + } + if {[dict exists $appnames_seen $kname]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: output name '$kname' already produced by [dict get $appnames_seen $kname]" + continue + } + dict set appnames_seen $kname $label + lappend entries [list $label [dict get $fields runtime] [dict get $fields vfs] $kname\ + [dict get $fields type] [dict get $fields target] [dict get $fields smokerequire]\ + [dict create entry $label group $g bake_default $bd scheme "" scheme_role ""]] + } + } + #-- schemes (G-023/G-024: generative entries expanded at parse time - no version enumeration) + set project_version "" + set project_version_err "" + if {[dict exists $tpayload scheme]} { + dict for {sname sentry} [dict get $tpayload scheme] { + set label "scheme.$sname" + set fields [apply $entry_reader model $label $sentry 1] + if {![dict size $fields]} { + continue + } + set skind [dict get $fields scheme] + if {$skind eq ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile is missing required key 'scheme' (known schemes: versioned)" + continue + } + if {$skind ne "versioned"} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown scheme '$skind' (known schemes: versioned)" + continue + } + #versioned scheme (G-023): output names derive from the punkproject.toml + #project version at parse time - punk-, punk-dev and the + #release-gated plain punk - so one declaration tracks the version with + #no per-version config edits. + if {$project_version eq "" && $project_version_err eq ""} { + set pptoml [file join [file dirname $sourcefolder] punkproject.toml] + lassign [toml_file_to_dict $pptoml] pstatus ppayload + if {$pstatus ne "ok"} { + set project_version_err $ppayload + } elseif {[catch {toml_untag [dict get $ppayload project version]} pver]} { + set project_version_err "no \[project\] version field readable in $pptoml ($pver)" + } elseif {$pver eq ""} { + set project_version_err "empty \[project\] version field in $pptoml" + } else { + set project_version $pver + } + } + if {$project_version eq ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: cannot expand 'versioned' scheme - $project_version_err" + continue + } + set prefix [dict get $fields prefix] + if {$prefix eq ""} { + set prefix $sname + } + set g [dict get $fields group] + set bd [dict get $fields bake_default] + if {$bd eq ""} { + set bd [expr {$g ne "" ? [dict get $groups $g bake_default] : 1}] + } + foreach {appname role} [list ${prefix}-$project_version versioned ${prefix}-dev dev $prefix release] { + if {[dict exists $appnames_seen $appname]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: expanded output name '$appname' already produced by [dict get $appnames_seen $appname]" + continue + } + dict set appnames_seen $appname $label + lappend entries [list "$label ($role)" [dict get $fields runtime] [dict get $fields vfs] $appname\ + [dict get $fields type] [dict get $fields target] [dict get $fields smokerequire]\ + [dict create entry $label group $g bake_default $bd scheme $sname scheme_role $role]] + } + } + } + #-- per-runtime target resolution (target is a property of the RUNTIME - entries + #may repeat the declaration but must not disagree; legacy-format parity) + set runtime_target [dict create] + set runtime_first_label [dict create] + foreach e $entries { + lassign $e label rtname - - - declared - - + if {![dict exists $runtime_first_label $rtname]} { + dict set runtime_first_label $rtname $label + } + if {$declared eq ""} { + continue + } + if {![dict exists $runtime_target $rtname] || [dict get $runtime_target $rtname] eq $default_target} { + dict set runtime_target $rtname $declared + } elseif {[dict get $runtime_target $rtname] ne $declared} { + lappend configerrors "CONFIG FILE ERROR. runtime: $rtname declares conflicting target platforms ('[dict get $runtime_target $rtname]' and '$declared') in $mapfile - a runtime has one target." + } + } + foreach rtname [dict keys $runtime_first_label] { + if {![dict exists $runtime_target $rtname]} { + dict set runtime_target $rtname $default_target + } + } + #-- runtime presence (one check per runtime, in first-reference order) + foreach rtname [dict keys $runtime_first_label] { + if {$rtname eq "-"} { + continue + } + set rt_target [dict get $runtime_target $rtname] + set rtdir [file join $rtbase [platform_store_tier $rt_target]] + set runtime_test $rtname[platform_exe_suffix $rt_target] + if {![file exists [file join $rtdir $runtime_test]]} { + lappend warnings "WARNING: Missing runtime file $rtdir/$runtime_test (entry [dict get $runtime_first_label $rtname] in $mapfile)" + lappend missing $rtname + lappend missing_detail [dict create kind runtime label [dict get $runtime_first_label $rtname] name $rtname target $rt_target] + } + } + #-- build the maps (entry order preserved; vfs existence filtering matches the + #historical inline parse the kit loop was built around) + set runtime_vfs_map [dict create] + set vfs_runtime_map [dict create] + foreach e $entries { + lassign $e label rtname vfstail appname kittype - smokes attrs + set rt_target [dict get $runtime_target $rtname] + set spec [list $vfstail $appname $kittype $rt_target $smokes $attrs] + dict lappend runtime_vfs_map $rtname $spec + if {![file isdirectory [file join $sourcefolder vfs $vfstail]]} { + lappend warnings "WARNING: Missing vfs folder [file join $sourcefolder vfs $vfstail] specified in $mapfile for entry $label" + if {$vfstail ni $missing} { + lappend missing $vfstail + } + lappend missing_detail [dict create kind vfs label $label name $vfstail target $rt_target] + } else { + dict lappend vfs_runtime_map $vfstail [list $rtname $appname $kittype $rt_target $smokes $attrs] + } + } + dict set model runtime_vfs_map $runtime_vfs_map + dict set model vfs_runtime_map $vfs_runtime_map + dict set model runtime_target $runtime_target + dict set model groups $groups + dict set model missing $missing + dict set model missing_detail $missing_detail + dict set model warnings [concat [dict get $model warnings] $warnings] + dict set model configerrors $configerrors + return $model +} + +# DEPRECATED legacy reader: the pre-G-024 mapvfs.config line format +# ( {vfsfolder ?kitname? ?kittype? ?targetplatform? ?smokerequires?} ...). +# Kept so un-migrated trees (older generated projects) keep baking; behaviour is +# unchanged from before the toml conversion apart from the deprecation NOTE. +proc ::punkboot::lib::mapvfs_parse_config {mapfile rtbase sourcefolder default_target} { + set model [mapvfs_model_new $mapfile $default_target legacy] + if {![file exists $mapfile]} { + return $model + } + dict set model exists 1 + dict lappend model warnings "NOTE: $mapfile uses the deprecated mapvfs.config line format - convert to mapvfs.toml (the file header of src/runtime/mapvfs.toml in the punkshell tree is the format spec; when both files exist the toml file wins)." set fdmap [open $mapfile r] fconfigure $fdmap -translation binary set mapdata [read $fdmap] @@ -765,7 +1153,7 @@ proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} dict set model vfs_runtime_map $vfs_runtime_map dict set model runtime_target $runtime_target dict set model missing $missing - dict set model warnings $warnings + dict set model warnings [concat [dict get $model warnings] $warnings] dict set model badentries $badentries dict set model configerrors $configerrors return $model @@ -795,6 +1183,12 @@ proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} # vfs vfs folder tail (e.g punk9wintk903.vfs) # vfs_present 1|0 # smokerequire G-133 smoke-require package list as configured ("" when undeclared) +# entry config table label (toml models, e.g kit.punk91 / "scheme.punk9 (dev)"; "" legacy) +# group G-024 named grouping ("" when ungrouped) +# bake_default 1|0 - 0 means excluded from full (no-name) bakes; still bakeable +# by name or @group selection +# scheme / scheme_role scheme-expanded outputs: source scheme name and role +# (versioned|dev|release); "" for explicit entries proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { set runtime_vfs_map [dict get $model runtime_vfs_map] set vfs_runtime_map [dict get $model vfs_runtime_map] @@ -808,7 +1202,7 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { } set records [list] set exe_names_seen [list] - set record_pair {{rtname vfstail appname target_kit_type smokerequires} { + set record_pair {{rtname vfstail appname target_kit_type smokerequires attrs} { upvar 1 records records exe_names_seen exe_names_seen rtbase rtbase sourcefolder sourcefolder\ runtime_target runtime_target default_target default_target if {$appname eq ""} { @@ -855,14 +1249,25 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { vfs $vfstail\ vfs_present [file isdirectory [file join $sourcefolder vfs $vfstail]]\ smokerequire $smokerequires\ + entry [expr {[dict exists $attrs entry] ? [dict get $attrs entry] : ""}]\ + group [expr {[dict exists $attrs group] ? [dict get $attrs group] : ""}]\ + bake_default [expr {[dict exists $attrs bake_default] ? [dict get $attrs bake_default] : 1}]\ + scheme [expr {[dict exists $attrs scheme] ? [dict get $attrs scheme] : ""}]\ + scheme_role [expr {[dict exists $attrs scheme_role] ? [dict get $attrs scheme_role] : ""}]\ ] }} foreach vfstail $vfs_tails { if {![dict exists $vfs_runtime_map $vfstail]} { continue ;#unmapped vfs folder - the kit loop derives no output names from it } + set rtnames_done [list] ;#several entries may share a runtime on one vfs (toml models) - + #enumerate each distinct runtime's spec set exactly once foreach rt_app [dict get $vfs_runtime_map $vfstail] { set rtname [lindex $rt_app 0] + if {$rtname in $rtnames_done} { + continue + } + lappend rtnames_done $rtname if {![dict exists $runtime_vfs_map $rtname]} { continue } @@ -871,49 +1276,70 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { if {$configured_vfs ne $vfstail} { continue } - apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires + apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires [::punkboot::lib::mapvfs_spec_attrs $vfs_app] } } } #entries whose vfs folder is missing (excluded from vfs_runtime_map by the parse) dict for {rtname vfs_specs} $runtime_vfs_map { foreach vfsconfig $vfs_specs { - if {[llength $vfsconfig] < 1 || [llength $vfsconfig] > 5} { + if {[llength $vfsconfig] < 1 || [llength $vfsconfig] > 6} { continue } lassign $vfsconfig vfstail appname target_kit_type - smokerequires if {[dict exists $vfs_runtime_map $vfstail]} { continue ;#already enumerated above } - apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires + apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires [::punkboot::lib::mapvfs_spec_attrs $vfsconfig] } } return $records } # Match user-supplied kit names against mapvfs_kit_outputs records. -# Accepts the base kit name (punk91) or the platform artifact name (punk91.exe, -# name.kit); matching is case-insensitive when the HOST filesystem is (a name -# the user types is a host-side convenience - a cygwin-family host is as +# Accepts the base kit name (punk91), the platform artifact name (punk91.exe, +# name.kit), or a G-024 group selector @ (selects every record in the +# group); matching is case-insensitive when the HOST filesystem is (a name the +# user types is a host-side convenience - a cygwin-family host is as # case-insensitive as a native windows one, whatever it targets). Returns a dict: # selected matched records (kit-output order, deduplicated) # unknown supplied names that matched nothing proc ::punkboot::lib::mapvfs_match_outputs {kit_outputs names} { set on_windows [expr {[info exists ::punkboot::host_windows] ? $::punkboot::host_windows : ("windows" eq $::tcl_platform(platform))}] + set name_eq {{on_windows a b} { + expr {$on_windows ? [string equal -nocase $a $b] : [string equal $a $b]} + }} set selected_idx [list] set unknown [list] foreach name $names { + set found 0 + if {[string match @?* $name]} { + #group selector: every record carrying the group (G-024) + set gname [string range $name 1 end] + set i 0 + foreach rec $kit_outputs { + if {[dict get $rec group] ne "" && [apply $name_eq $on_windows [dict get $rec group] $gname]} { + set found 1 + if {$i ni $selected_idx} { + lappend selected_idx $i + } + } + incr i + } + if {!$found} { + lappend unknown $name + } + continue + } set base $name if {[string match -nocase *.exe $base] || [string match -nocase *.kit $base]} { set base [string range $base 0 end-4] } - set found 0 set i 0 foreach rec $kit_outputs { foreach candidate [list [dict get $rec kitname] [dict get $rec targetkit]] { if {$candidate eq ""} {continue} - if {($on_windows && [string equal -nocase $candidate $base]) || (!$on_windows && $candidate eq $base)\ - || ($on_windows && [string equal -nocase $candidate $name]) || (!$on_windows && $candidate eq $name)} { + if {[apply $name_eq $on_windows $candidate $base] || [apply $name_eq $on_windows $candidate $name]} { set found 1 break } @@ -2110,7 +2536,7 @@ proc ::punkboot::punkboot_gethelp {args} { append h " - includes the vfslibs phase; does not rebuild modules/libs and does not run the promotion gates" \n append h " - with kitname arguments, bakes and deploys only the named configured kits (unknown names error before any build)" \n \n append h " $scriptname bakelist ?kitname ...?" \n - append h " - list the kit outputs configured in src/runtime/mapvfs.config: name, kit type, runtime (with presence)," \n + append h " - list the kit outputs configured in src/runtime/mapvfs.toml: name, kit type, runtime (with presence)," \n append h " vfs folder and deployed state (bin copy absent/current/stale/nobuild vs the src/_build product)" \n append h " - kitname arguments filter to the named entries (per-kit detail)" \n \n append h " $scriptname modules" \n @@ -2278,19 +2704,28 @@ DIAGRAM 1b - THIN-LAYOUT SYNC (part of modules/libs/packages/bakehouse runs) DIAGRAM 2 - KIT ASSEMBLY DETAIL (the 'make.tcl bake' stage; incl. vfslibs phase) ---------------------------------------------------------------------------- - src/runtime/mapvfs.config which vfs folder pairs with which runtime -> kit name, - and optionally the TARGET PLATFORM of that runtime [K9] - 'make.tcl bakelist ?kitname ...?' reports the configured - kit outputs (runtime/vfs presence + deployed state); - 'make.tcl bake ?kitname ...?' bakes only the named kits - (bare bake = all; unknown names error before any build) + src/runtime/mapvfs.toml which vfs folder pairs with which runtime -> kit name + ([kit.] tables; tomlish-parsed - G-024), incl. the + runtime's TARGET PLATFORM [K9], smoke-require packages, + named groups and generative version-named schemes + (deprecated mapvfs.config line format still readable) + 'make.tcl bakelist ?name|@group ...?' reports the + configured kit outputs (presence + deployed state); + 'make.tcl bake ?name|@group ...?' bakes only the named + kits (bare bake = all entries except bake_default=false + ones; unknown names error before any build) bin/runtime// bare Tcl runtime (tclkit / tclsfe / suite build), in the store tier of the TARGET platform it builds kits for [K9] src/vfs/_vfscommon.vfs/ common payload (built modules + libs, main boot support) src/vfs/.vfs/ per-kit overlay (main.tcl, kit-only modules/config) - - _vfscommon.vfs .vfs - | | + src/vfs/.vfs.toml optional per-.vfs payload declaration (G-115): [payload.*] + entries materialized INTO the .vfs folder by the vfslibs + phase (vendor trees / bin/packages tier), punkcheck- + tracked with drop-in-wins precedence - undeclared or + hand-modified files are preserved, never overwritten + + _vfscommon.vfs .vfs <-- payload declarations (if any) already + | | materialized by the vfslibs phase +-----------+----------+ | copy common, then merge overlay over it v @@ -2349,7 +2784,8 @@ LIBRARY-ARTIFACT FETCH + CONSUMPTION (G-139) | consumption (two lanes, both punkcheck-tracked): | PACKAGES_tcl phase of modules/libs/packages/bakehouse | -> lib_tcl/allplatforms + lib_tcl/; - | vfslibs entries with source_root = "packages" + | per-.vfs payload declarations (src/vfs/.vfs.toml, + | G-115) with source_root = "packages" | -> declared kit vfs lib_tcl trees (see DIAGRAM 2) v lib_tcl/... + src/vfs/.vfs/lib_tcl/... (embedded records ride along @@ -2410,9 +2846,10 @@ KEY / NOTES whether the pre-deploy sweep uses tasklist/taskkill or ps/kill. The default target is this host's own platform - except a cygwin-family host (an msys2/cygwin-runtime tclsh, which reports platform 'unix' on windows), which - targets win32. A mapvfs.config entry may declare its own target as a 4th - element and is then read from that platform's tier, named with its executable - convention, and (being unrunnable here) skipped by the process sweep. + targets win32. A mapvfs.toml entry may declare its own target (target key; + legacy config: 4th element) and is then read from that platform's tier, named + with its executable convention, and (being unrunnable here) skipped by the + process sweep. 'make.tcl check' prints the derivation; 'make.tcl bakelist' shows per-kit targets, with a target= note on rows that are not on the default target. @@ -2440,8 +2877,8 @@ KEY / NOTES cross-target kits are covered) against the kit's target platform; wrong-arch libraries outside platform-discriminated subdirs (win32-ix86/, win-x64/ etc - multi-arch payloads are legitimate there) earn recapped BUILD-WARNINGs. - (b) Smoke-require probe: a kit declaring packages (mapvfs.config 5th entry - element) has each one plain-'package require'd INSIDE the freshly built + (b) Smoke-require probe: a kit declaring packages (mapvfs.toml smokerequire + key; legacy config: 5th element) has each one plain-'package require'd INSIDE the freshly built artifact via its tclsh subcommand - the only check that sees resolution-order defects (a wrong-arch higher-versioned package shadowing a working copy); cross-target kits skip with a stated reason, undeclared kits run nothing new. @@ -2556,8 +2993,8 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO modules "Build .tm modules from src/modules, src/vendormodules etc into /modules etc" libs "Build pkgIndex.tcl libraries from src/lib, src/vendorlib etc into /lib etc" bake "Assemble kit/zipkit executables from promoted payload (src/vfs) and runtimes into /bin" - bakelist "List the kit outputs configured in src/runtime/mapvfs.config: type, runtime/vfs presence, deployed state" - vfslibs "Propagate declared platform-library packages (vendored or bin/packages tier) into kit vfs lib_tcl trees" + bakelist "List the kit outputs configured in src/runtime/mapvfs.toml: type, runtime/vfs presence, deployed state, groups" + vfslibs "Materialize per-.vfs payload declarations (src/vfs/.vfs.toml - G-115) into kit vfs folders; drop-ins preserved" bin "Install executables from src/bin into /bin, then build kits as for bake" vendorupdate "Update src/vendormodules based on src/vendormodules/include_modules.config" libfetch "Fetch declared punkbin lib-tier library artifacts into bin/packages/ (sha1-verified) and materialize the package trees" @@ -2620,25 +3057,28 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO build each configured kit from the promoted payload (src/vfs/_vfscommon.vfs + custom vfs folders) and its runtime, and install to /bin. - With kitname arguments, bake and deploy ONLY the named configured - kits (see 'make.tcl bakelist' for the configured names) - other - kits' build products, bin copies and punkcheck records are left - untouched. An unknown name errors before any build, listing the - configured names. Bare 'bake' processes all configured kits. - Includes the vfslibs phase (declared vendored platform-library - propagation into kit vfs lib_tcl trees) so a bake cannot ship - stale binary libs; a selective bake narrows that phase to the - named kits' vfs folders. + With kitname (or @groupname - G-024 groups) arguments, bake and + deploy ONLY the named configured kits (see 'make.tcl bakelist' + for the configured names and groups) - other kits' build + products, bin copies and punkcheck records are left untouched. + An unknown name errors before any build, listing the configured + names. Bare 'bake' processes all configured kits except + bake_default=false entries (those bake only by name/@group). + Includes the vfslibs phase (per-.vfs payload declarations + src/vfs/.vfs.toml materialized into the kit vfs folders - + G-115) so a bake cannot ship stale binary libs; a selective + bake narrows that phase to the named kits' vfs folders. Does not rebuild modules/libs and does not run the promotion gates - built modules reach the kit payload only via 'make.tcl vfscommonupdate' (committed for provenance). Consumer full path from a clean checkout: 'make.tcl bakehouse'." bakelist " - List the kit outputs configured in src/runtime/mapvfs.config (the - same parsed mapping the bake subcommand consumes): kit name, kit - type (kit|zip|zipcat...), runtime with presence in the runtime - store of that kit's TARGET platform + List the kit outputs configured in src/runtime/mapvfs.toml (the + same parsed mapping the bake subcommand consumes; the deprecated + mapvfs.config line format is honoured when no toml file exists): + kit name, kit type (kit|zip|zipcat...), runtime with presence in + the runtime store of that kit's TARGET platform (/bin/runtime/), vfs folder, and the deployed state of /bin/ vs the src/_build build product: @@ -2651,20 +3091,26 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO materialized from an older revision than a -r artifact beside it) and marks cross-target entries (target= on any kit whose mapvfs entry declares a target other than this host's - default - 'make.tcl check' shows that default). - kitname arguments filter the report to the named entries (per-kit - detail, with resolved store tier, target and paths). Reporting - only - never builds." + default - 'make.tcl check' shows that default), G-024 groups + (group=), entries excluded from full bakes (default=no) + and scheme-expanded outputs (scheme=versioned|dev|release). + kitname (or @groupname) arguments filter the report to the named + entries (per-kit detail, with resolved store tier, target, + config entry and paths). Reporting only - never builds." vfslibs " - Propagate declared platform-library packages into kit vfs - lib_tcl trees - from vendored src/ folders (source_root - 'src', the default) or from the bin/packages punkbin lib-tier - input (source_root 'packages', populated by 'make.tcl - libfetch' - G-139). - Declarations (per-package per-kit, with superseded-version - removal and clean-slate same-name 'replace') live in - src/runtime/vendorlib_vfs.toml. + Materialize the per-.vfs payload declarations (G-115): each + src/vfs/.vfs.toml declares package folders to install + INTO src/vfs/.vfs - from vendored src/ folders + (source_root 'src', the default) or from the bin/packages + punkbin lib-tier input (source_root 'packages', populated by + 'make.tcl libfetch' - G-139) - with explicit superseded-version + removal and clean-slate same-name 'replace'. + punkcheck-tracked with drop-in-wins precedence: files the + mechanism did not install (or that changed since it installed + them) are preserved and reported, never overwritten; a .vfs + with no declaration file is untouched. Format + precedence: + src/vfs/README.md. Also runs automatically as a phase of 'make.tcl bake' (and therefore of 'make.tcl bakehouse'); this standalone narrowing serves quick iteration on the declarations." @@ -2927,10 +3373,14 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO #still works when the mapping cannot be parsed here) stays authoritative. variable KITNAME_CHOICES [list] catch { - set _mapmodel [punkboot::lib::mapvfs_parse [file join $::punkboot::scriptfolder runtime mapvfs.config] "" $::punkboot::scriptfolder $::punkboot::target_platform] + set _mapmodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate [file join $::punkboot::scriptfolder runtime]] "" $::punkboot::scriptfolder $::punkboot::target_platform] foreach _rec [punkboot::lib::mapvfs_kit_outputs $_mapmodel "" $::punkboot::scriptfolder] { lappend KITNAME_CHOICES [dict get $_rec kitname] } + #G-024 group selectors join the informational choices (@) + foreach _g [dict keys [dict get $_mapmodel groups]] { + lappend KITNAME_CHOICES @$_g + } } set _kitname_choicepart "" if {[llength $KITNAME_CHOICES]} { @@ -2941,16 +3391,19 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "@values -min 0 -max -1"\ "kitname -type string -optional 1 -multiple 1${_kitname_choicepart} -help\ { - Configured kit output(s) to bake and deploy - see 'make.tcl bakelist' - for the configured names. With no kitname, all configured kits are - processed. An unknown name errors before any build.}"\ + Configured kit output(s) to bake and deploy - a kit name, or + @ for every kit in a configured group (G-024) - see + 'make.tcl bakelist' for the configured names and groups. With no + kitname, all configured kits are processed except entries with + bake_default=false. An unknown name errors before any build.}"\ ] dict set SUBVALUES vfs [dict get $SUBVALUES bake] dict set SUBVALUES bakelist [list\ "@values -min 0 -max -1"\ "kitname -type string -optional 1 -multiple 1${_kitname_choicepart} -help\ { - Configured kit output(s) to report on (filters the listing - doubles + Configured kit output(s) to report on - a kit name, or @ + for every kit in a configured group (filters the listing - doubles as per-kit detail). With no kitname, all configured entries are listed.}"\ ] @@ -3386,7 +3839,7 @@ set ::punkboot::bake_selected_vfs [list] ;#vfs folder tails the selected set ::punkboot::bake_selected_runtimes [list] ;#unsuffixed runtime names the selected kits wrap if {$::punkboot::command eq "bake" && [llength $::punkboot::opt_kitnames]} { set _rtbase $binfolder/runtime - set _mapmodel [punkboot::lib::mapvfs_parse $sourcefolder/runtime/mapvfs.config $_rtbase $sourcefolder $::punkboot::target_platform] + set _mapmodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate $sourcefolder/runtime] $_rtbase $sourcefolder $::punkboot::target_platform] if {[llength [dict get $_mapmodel configerrors]]} { foreach _errline [dict get $_mapmodel configerrors] { puts stderr $_errline @@ -3399,8 +3852,11 @@ if {$::punkboot::command eq "bake" && [llength $::punkboot::opt_kitnames]} { puts stderr "make.tcl bake: unknown kit name(s): [join [dict get $_matchinfo unknown] {, }] - nothing built" if {[llength $_kit_outputs]} { puts stderr " configured kit outputs: [join [lmap _r $_kit_outputs {dict get $_r kitname}] {, }]" + if {[dict size [dict get $_mapmodel groups]]} { + puts stderr " configured groups (select with @): [join [dict keys [dict get $_mapmodel groups]] {, }]" + } } else { - puts stderr " no kit outputs are configured (src/runtime/mapvfs.config)" + puts stderr " no kit outputs are configured (src/runtime/mapvfs.toml)" } foreach _n [dict get $_matchinfo unknown] { if {[string match -* $_n]} { @@ -3740,7 +4196,7 @@ if {$::punkboot::command eq "check"} { if {$_chk_suffix eq ""} {set _chk_suffix "(none)"} puts stdout "platform (G-122): host=$::punkboot::host_platform target=$::punkboot::target_platform store=bin/runtime/[punkboot::lib::platform_store_tier $::punkboot::target_platform] exe-suffix=$_chk_suffix process-tooling=[punkboot::lib::platform_process_family $::punkboot::target_platform]" puts stdout " (host = this tclsh's punk::platform canon, from tcl_platform(os)='$::tcl_platform(os)' platform='$::tcl_platform(platform)'; target = default kit target - a cygwin-family host (msys2/cygwin runtime on windows) bakes win32 kits)" - puts stdout " (mapvfs.config entries may declare their own target platform as a 4th element - 'make.tcl bakelist' shows each kit's target, store tier and artifact name)" + puts stdout " (mapvfs.toml entries may declare their own target platform (target key) - 'make.tcl bakelist' shows each kit's target, store tier and artifact name)" unset -nocomplain _chk_suffix puts stdout $sep puts stdout "module/library checks - paths from bootsupport only" @@ -3950,8 +4406,8 @@ if {$::punkboot::command eq "check"} { puts stdout " bake will proceed with a NOTE and cannot flag a file-relative kit payload - run 'make.tcl modules' then 'make.tcl bootsupport'." } set _smoke_declared [list] - if {[file exists $sourcefolder/runtime/mapvfs.config]} { - set _smokemodel [punkboot::lib::mapvfs_parse $sourcefolder/runtime/mapvfs.config [file dirname $sourcefolder]/bin/runtime $sourcefolder $::punkboot::target_platform] + if {[file exists [punkboot::lib::mapvfs_locate $sourcefolder/runtime]]} { + set _smokemodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate $sourcefolder/runtime] [file dirname $sourcefolder]/bin/runtime $sourcefolder $::punkboot::target_platform] foreach _rec [punkboot::lib::mapvfs_kit_outputs $_smokemodel [file dirname $sourcefolder]/bin/runtime $sourcefolder] { if {[llength [dict get $_rec smokerequire]]} { lappend _smoke_declared "[dict get $_rec kitname] ([join [dict get $_rec smokerequire] {, }])" @@ -3960,7 +4416,7 @@ if {$::punkboot::command eq "check"} { unset -nocomplain _smokemodel _rec } puts stdout "smoke-require probe (G-133, advisory): freshly built host-runnable kits declaring packages" - puts stdout " (mapvfs.config 5th entry element) have each one plain-'package require'd inside the built" + puts stdout " (mapvfs.toml smokerequire key; legacy mapvfs.config 5th entry element) have each one plain-'package require'd inside the built" puts stdout " artifact via its tclsh subcommand - the only check that sees resolution-order defects such as" puts stdout " a wrong-arch higher-versioned package shadowing a working copy. Failures are recapped" puts stdout " BUILD-WARNINGs; cross-target kits skip with a stated reason; undeclared kits run nothing new." @@ -4167,7 +4623,7 @@ if {$::punkboot::command eq "bakelist"} { set rtbase $binfolder/runtime set rtfolder $rtbase/$rt_os_arch set buildfolder $sourcefolder/_build - set mapfile $sourcefolder/runtime/mapvfs.config + set mapfile [punkboot::lib::mapvfs_locate $sourcefolder/runtime] set mapmodel [punkboot::lib::mapvfs_parse $mapfile $rtbase $sourcefolder $rt_default_target] foreach errline [dict get $mapmodel configerrors] { puts stderr $errline @@ -4175,6 +4631,15 @@ if {$::punkboot::command eq "bakelist"} { foreach badline [dict get $mapmodel badentries] { puts stderr $badline } + if {[dict get $mapmodel format] eq "legacy"} { + #deprecation NOTE (first warnings line) - bakelist is reporting-only, so + #surface it here rather than relying on the bake path's warning printout + foreach warnline [dict get $mapmodel warnings] { + if {[string match NOTE:* $warnline]} { + puts stderr $warnline + } + } + } if {![dict get $mapmodel exists]} { puts stdout "no kit mapping config at $mapfile - no kit outputs configured" exit 0 @@ -4200,7 +4665,7 @@ if {$::punkboot::command eq "bakelist"} { if {$t eq $rt_os_arch} { puts stderr "WARNING: runtime store folder bin/runtime/$t does not exist - every configured runtime on the default target will read runtime=missing." puts stderr " Default target '$rt_default_target' derives from THIS tclsh: tcl_platform(os)='$::tcl_platform(os)' tcl_platform(platform)='$::tcl_platform(platform)' -> host canon '$::punkboot::host_platform' -> target '$rt_default_target'." - puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.config for runtimes stored elsewhere." + puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.toml for runtimes stored elsewhere." } else { puts stderr "WARNING: runtime store folder bin/runtime/$t does not exist - configured entries targeting $t will read runtime=missing." } @@ -4211,6 +4676,9 @@ if {$::punkboot::command eq "bakelist"} { if {[llength [dict get $matchinfo unknown]]} { puts stderr "bakelist: unknown kit name(s): [join [dict get $matchinfo unknown] {, }]" puts stderr " configured kit outputs: [join [lmap r $kit_outputs {dict get $r kitname}] {, }]" + if {[dict size [dict get $mapmodel groups]]} { + puts stderr " configured groups (select with @): [join [dict keys [dict get $mapmodel groups]] {, }]" + } exit 1 } set reportset [dict get $matchinfo selected] @@ -4235,6 +4703,16 @@ if {$::punkboot::command eq "bakelist"} { #its own platform, not this host's default lappend notes target=[dict get $rec target] } + if {[dict get $rec group] ne ""} { + lappend notes group=[dict get $rec group] + } + if {![dict get $rec bake_default]} { + #excluded from full bakes - bake by name or @group (G-024) + lappend notes default=no + } + if {[dict get $rec scheme_role] ne ""} { + lappend notes scheme=[dict get $rec scheme_role] + } if {$runtime ne "-" && [dict get $rec runtime_present]} { set rtstale [punkboot::lib::runtime_materialization_check [dict get $rec runtime_dir] [dict get $rec runtime_file]] if {[dict size $rtstale]} { @@ -4256,7 +4734,15 @@ if {$::punkboot::command eq "bakelist"} { set widths $newwidths } set rt_store_note [expr {$rt_store_missing ? " (FOLDER MISSING)" : ""}] - puts stdout "# mapvfs: src/runtime/mapvfs.config runtimes: bin/runtime/$rt_os_arch$rt_store_note deployed: bin/ vs src/_build/" + if {[info exists ::env(PUNK_MAPVFS_CONFIG)] && $::env(PUNK_MAPVFS_CONFIG) ne ""} { + set map_display "$mapfile (PUNK_MAPVFS_CONFIG override)" + } else { + set map_display "src/runtime/[file tail $mapfile]" + if {[dict get $mapmodel format] eq "legacy"} { + append map_display " (deprecated format)" + } + } + puts stdout "# mapvfs: $map_display runtimes: bin/runtime/$rt_os_arch$rt_store_note deployed: bin/ vs src/_build/" set headline "" foreach h $headings w $widths { append headline [format "%-*s " $w $h] @@ -4289,7 +4775,15 @@ if {$::punkboot::command eq "bakelist"} { } puts stdout " vfs folder: src/vfs/[dict get $rec vfs] [expr {[dict get $rec vfs_present] ? {(present)} : {(MISSING)}}]" puts stdout " kit type: [dict get $rec kit_type]" - puts stdout " target: [dict get $rec target][expr {[dict get $rec target] eq $rt_default_target ? " (host default)" : " (declared in mapvfs.config)"}]" + puts stdout " target: [dict get $rec target][expr {[dict get $rec target] eq $rt_default_target ? " (host default)" : " (declared in the kit mapping)"}]" + if {[dict get $rec entry] ne ""} { + puts stdout " config entry: \[[dict get $rec entry]\]" + } + if {[dict get $rec group] ne ""} { + puts stdout " group: [dict get $rec group][expr {[dict get $rec bake_default] ? "" : " (bake_default=false - bake by name or @[dict get $rec group])"}]" + } elseif {![dict get $rec bake_default]} { + puts stdout " bake_default: false (excluded from full bakes - bake by name to build)" + } puts stdout " build product: src/_build/[dict get $rec targetkit] [punkboot::lib::bakelist_file_detail $buildfolder/[dict get $rec targetkit]]" puts stdout " deployed: bin/[dict get $rec targetkit] [punkboot::lib::bakelist_file_detail $binfolder/[dict get $rec targetkit]] state=[lindex $row 4]" } @@ -5349,20 +5843,8 @@ if {$::punkboot::command eq "libfetch"} { puts stderr "$A(BAD)LIBFETCH: failed to parse $libfetch_config\n$libfetch_dict$A(RST)" exit 3 } - if {![llength [info commands ::punkboot::lib::toml_untag]]} { - #tomlish::to_dict values are type-tagged (also defined by the vfslibs - #phase - that block does not run for the libfetch command) - proc ::punkboot::lib::toml_untag {tagged} { - if {[dict get $tagged type] eq "ARRAY"} { - set plain [list] - foreach el [dict get $tagged value] { - lappend plain [::punkboot::lib::toml_untag $el] - } - return $plain - } - return [dict get $tagged value] - } - } + #tomlish::to_dict values are type-tagged - ::punkboot::lib::toml_untag (defined + #with the shared toml helpers near the top of this script) resolves them set libfetch_artifacts [dict create] if {[dict exists $libfetch_dict artifact]} { set libfetch_artifacts [dict get $libfetch_dict artifact] @@ -5683,77 +6165,97 @@ if {$::punkboot::command in {bakehouse packages libs}} { } if {$::punkboot::command in {bakehouse bake vfslibs}} { - #G-037: propagate declared vendored platform-library packages into kit vfs lib_tcl trees. - #Participation is per-package per-kit (kit vfs package sets intentionally differ - e.g. twapi - #versions vary per kit) - declared in src/runtime/vendorlib_vfs.toml, never blanket-copied. - #punkcheck records live at src/vfs/.punkcheck so they are not baked into kit payloads. - set vfslibs_config $sourcefolder/runtime/vendorlib_vfs.toml - if {![file exists $vfslibs_config]} { - puts stdout "VFSLIBS: no config at $vfslibs_config - skipping vendorlib -> kit vfs propagation" - } else { - if {[catch {package require tomlish} tomlish_err]} { - puts stderr "$A(BAD)VFSLIBS: cannot process $vfslibs_config - tomlish package unavailable ($tomlish_err)$A(RST)" + #G-115: declarative .vfs payload composition. Each src/vfs/.vfs may have a + #SIBLING declaration file src/vfs/.vfs.toml declaring payload package folders + #to materialize INTO it - while the folder remains the operative assembly area: + #drop-ins need no declaration and are never clobbered. The declaration file's own + #header comment (and src/vfs/README.md) is the user-facing spec. Entry keys per + #[payload.]: source (folder path, resolved per source_root), source_root + #('src' default - vendor trees / suite build products under src/; or 'packages' - + #the consent-gated punkbin lib tier at /bin/packages populated by + #'make.tcl libfetch'), target (folder path INSIDE the .vfs, e.g "lib_tcl9"; + #default the .vfs root), supersedes (explicit legacy sibling folder removals), + #replace (boolean clean-slate of the SAME-NAMED target folder before install). + #Materialization is punkcheck-tracked (records in src/vfs/.punkcheck, outside kit + #payloads) with -overwrite synced-targets: a target file punkcheck did not install, + #or that changed since it was installed (a drop-in or hand edit), is PRESERVED and + #reported, never overwritten - the G-115 drop-in-wins precedence; supersedes and + #replace are the explicit declared exceptions. A .vfs with no declaration file is + #untouched (pure drop-in mode, exactly as before G-115). + #This phase supersedes the per-package src/runtime/vendorlib_vfs.toml surface + #(G-037/G-139): its [install.] entries were migrated into per-.vfs files + #2026-07-31 - rationale in goals/G-115-declarative-vfs-composition.md. + set legacy_vfslibs $sourcefolder/runtime/vendorlib_vfs.toml + if {[file exists $legacy_vfslibs]} { + puts stderr "WARNING: $legacy_vfslibs is superseded by per-.vfs payload declarations (src/vfs/.vfs.toml - G-115) and is IGNORED - migrate its \[install.*\] entries into per-.vfs files and remove it." + } + set payload_files [lsort [glob -nocomplain -dir $sourcefolder/vfs -types f -tail *.vfs.toml]] + if {![llength $payload_files]} { + puts stdout "VFSPAYLOAD: no per-.vfs payload declarations (src/vfs/.vfs.toml) - nothing to materialize" + } + foreach ptail $payload_files { + set pfile $sourcefolder/vfs/$ptail + set vfstail [file rootname $ptail] ;#punkdeclare.vfs.toml -> punkdeclare.vfs + if {[llength $::punkboot::bake_selected_kitnames] && $vfstail ni $::punkboot::bake_selected_vfs} { + #G-121 selective bake: narrow to the selected kits' vfs folders (skip + #precedes the existence checks so an unselected declaration's problem + #cannot abort a selective run) + puts stdout "VFSPAYLOAD $vfstail: skipping (selective bake - not a selected kit's vfs)" + continue + } + if {![file isdirectory $sourcefolder/vfs/$vfstail]} { + puts stderr "$A(BAD)VFSPAYLOAD: declaration $pfile has no matching vfs folder at $sourcefolder/vfs/$vfstail$A(RST)" exit 3 } - set fd [open $vfslibs_config r] - fconfigure $fd -encoding utf-8 - set vfslibs_tomldata [read $fd] - close $fd - if {[catch {tomlish::to_dict [tomlish::from_toml $vfslibs_tomldata]} vfslibs_dict]} { - puts stderr "$A(BAD)VFSLIBS: failed to parse $vfslibs_config\n$vfslibs_dict$A(RST)" + lassign [punkboot::lib::toml_file_to_dict $pfile] pstatus ppayload + if {$pstatus ne "ok"} { + puts stderr "$A(BAD)VFSPAYLOAD: $ppayload$A(RST)" exit 3 } - #tomlish::to_dict values are type-tagged (e.g {type STRING value x}, {type ARRAY value {...}}) - proc ::punkboot::lib::toml_untag {tagged} { - if {[dict get $tagged type] eq "ARRAY"} { - set plain [list] - foreach el [dict get $tagged value] { - lappend plain [::punkboot::lib::toml_untag $el] - } - return $plain + dict for {topkey -} $ppayload { + if {$topkey ne "payload"} { + puts stderr "$A(BAD)VFSPAYLOAD: $pfile: unknown top-level table or key '$topkey' (expected \[payload.\] entries)$A(RST)" + exit 3 } - return [dict get $tagged value] } - set vfslibs_installs [dict create] - if {[dict exists $vfslibs_dict install]} { - set vfslibs_installs [dict get $vfslibs_dict install] + set payload_entries [dict create] + if {[dict exists $ppayload payload]} { + set payload_entries [dict get $ppayload payload] } - if {![dict size $vfslibs_installs]} { - puts stdout "VFSLIBS: no \[install.\] entries found in $vfslibs_config" + if {![dict size $payload_entries]} { + puts stdout "VFSPAYLOAD $vfstail: no \[payload.\] entries in $ptail" + continue } - dict for {iname ientry} $vfslibs_installs { - foreach reqkey {source targets} { - if {![dict exists $ientry $reqkey]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname in $vfslibs_config is missing required key '$reqkey'$A(RST)" - exit 3 - } - } - set isource [punkboot::lib::toml_untag [dict get $ientry source]] - set itargets [punkboot::lib::toml_untag [dict get $ientry targets]] - set isupersedes [list] - if {[dict exists $ientry supersedes]} { - set isupersedes [punkboot::lib::toml_untag [dict get $ientry supersedes]] - } - #source_root (G-139): 'src' (default - the original vendorlib - #declarations) or 'packages' - resolve against the untracked - #bin/packages lib-tier input (populated by 'make.tcl libfetch'). + dict for {iname ientry} $payload_entries { + set entrylabel "payload.$iname ($ptail)" + set isource "" set isource_root "src" - if {[dict exists $ientry source_root]} { - set isource_root [punkboot::lib::toml_untag [dict get $ientry source_root]] - } - #replace (G-139): clean-slate a SAME-NAMED existing target folder - #before install. supersedes deliberately cannot name the folder being - #installed; replace covers same-name refreshes where the old copy may - #carry strays the new tree lacks (e.g. hand-drops nesting md5c/tcllibc - #inside tcllib2.0). + set itarget "" + set isupersedes [list] set ireplace 0 - if {[dict exists $ientry replace]} { - set ireplace [punkboot::lib::toml_untag [dict get $ientry replace]] - if {![string is boolean -strict $ireplace]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname 'replace' must be boolean (got '$ireplace')$A(RST)" - exit 3 + dict for {k v} $ientry { + switch -- $k { + source {set isource [punkboot::lib::toml_untag $v]} + source_root {set isource_root [punkboot::lib::toml_untag $v]} + target {set itarget [punkboot::lib::toml_untag $v]} + supersedes {set isupersedes [punkboot::lib::toml_untag $v]} + replace { + set ireplace [punkboot::lib::toml_untag $v] + if {![string is boolean -strict $ireplace]} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel: 'replace' must be boolean (got '$ireplace')$A(RST)" + exit 3 + } + } + default { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel: unknown key '$k' (expected source, source_root, target, supersedes, replace)$A(RST)" + exit 3 + } } } + if {$isource eq ""} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel is missing required key 'source'$A(RST)" + exit 3 + } switch -- $isource_root { src { set source_pkg_folder $sourcefolder/$isource @@ -5764,55 +6266,64 @@ if {$::punkboot::command in {bakehouse bake vfslibs}} { set source_missing_hint " - populate the bin/packages tier first: 'tclsh src/make.tcl libfetch' (declarations in src/runtime/libpackages.toml)" } default { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname has unknown source_root '$isource_root' (expected src|packages)$A(RST)" + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel has unknown source_root '$isource_root' (expected src|packages)$A(RST)" exit 3 } } if {![file isdirectory $source_pkg_folder]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname source '$isource' not found at $source_pkg_folder$source_missing_hint$A(RST)" + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel source '$isource' not found at $source_pkg_folder$source_missing_hint$A(RST)" + exit 3 + } + if {$itarget ne "" && ([file pathtype $itarget] ne "relative" || ".." in [file split $itarget])} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel target '$itarget' must be a relative path inside the .vfs folder (no '..')$A(RST)" exit 3 } set pkgtail [file tail $source_pkg_folder] - foreach t $itargets { - set target_vfs_dir [lindex [file split $t] 0] - if {[llength $::punkboot::bake_selected_kitnames] && $target_vfs_dir ni $::punkboot::bake_selected_vfs} { - #G-121 selective bake: narrow the vfslibs phase to the selected kits' - #vfs folders (skip precedes the existence check so an unselected - #entry's problem cannot abort a selective run) - puts stdout "VFSLIBS install.$iname: skipping vfs/$t (selective bake - not a selected kit's vfs)" - continue - } - if {![file isdirectory $sourcefolder/vfs/$target_vfs_dir]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname target '$t' - no vfs folder at $sourcefolder/vfs/$target_vfs_dir$A(RST)" - exit 3 + if {$itarget eq ""} { + set target_lib_folder $sourcefolder/vfs/$vfstail + set target_display vfs/$vfstail + } else { + set target_lib_folder $sourcefolder/vfs/$vfstail/$itarget + set target_display vfs/$vfstail/$itarget + } + file mkdir $target_lib_folder + #supersession first - no silent mixed-version provision (G-035 concerns) + foreach sup $isupersedes { + if {$sup eq $pkgtail} { + continue ;#never remove what we are about to install - use replace for that } - set target_lib_folder $sourcefolder/vfs/$t - file mkdir $target_lib_folder - #supersession first - no silent mixed-version provision (G-035 concerns) - foreach sup $isupersedes { - if {$sup eq $pkgtail} { - continue ;#never remove what we are about to install - } - set suppath $target_lib_folder/$sup - if {[file exists $suppath]} { - puts stdout "VFSLIBS install.$iname: removing superseded '$sup' from vfs/$t" - file delete -force $suppath - } + set suppath $target_lib_folder/$sup + if {[file exists $suppath]} { + puts stdout "VFSPAYLOAD $entrylabel: removing superseded '$sup' from $target_display" + file delete -force $suppath } - if {$ireplace && [file isdirectory $target_lib_folder/$pkgtail]} { - puts stdout "VFSLIBS install.$iname: replace=true - removing existing '$pkgtail' from vfs/$t (clean-slate install)" - file delete -force $target_lib_folder/$pkgtail + } + if {$ireplace && [file isdirectory $target_lib_folder/$pkgtail]} { + puts stdout "VFSPAYLOAD $entrylabel: replace=true - removing existing '$pkgtail' from $target_display (clean-slate install)" + file delete -force $target_lib_folder/$pkgtail + } + puts stdout "VFSPAYLOAD $entrylabel: $isource_root:$isource -> $target_display/$pkgtail (if source changed; drop-ins/hand-edits preserved)" + set resultdict [punkcheck::install $source_pkg_folder $target_lib_folder/$pkgtail\ + -installer make.tcl\ + -createdir 1\ + -overwrite synced-targets\ + -punkcheck-folder $sourcefolder/vfs\ + -progresschannel stdout\ + ] + puts stdout [punkcheck::summarize_install_resultdict $resultdict] + #preserved-collision report: a file skipped although its SOURCE changed was + #withheld by synced-targets (a drop-in, or hand-modified since install) - + #distinct from the routine source-unchanged skips. + set _preserved [list] + foreach _sk [dict get $resultdict files_skipped] { + if {$_sk ni [dict get $resultdict sources_unchanged]} { + lappend _preserved $_sk } - puts stdout "VFSLIBS install.$iname: $isource -> vfs/$t/$pkgtail (if source file changed)" - set resultdict [punkcheck::install $source_pkg_folder $target_lib_folder/$pkgtail\ - -installer make.tcl\ - -createdir 1\ - -overwrite installedsourcechanged-targets\ - -punkcheck-folder $sourcefolder/vfs\ - -progresschannel stdout\ - ] - puts stdout [punkcheck::summarize_install_resultdict $resultdict] } + if {[llength $_preserved]} { + puts stdout "VFSPAYLOAD $entrylabel: NOTE - [llength $_preserved] changed file(s) NOT overwritten (existing target content was not installed by this mechanism, or was modified since) - the G-115 drop-in-wins precedence. Declare 'replace = true' (whole package folder) or remove the target files to hand control to the declaration." + } + unset -nocomplain _preserved _sk } } } @@ -6224,7 +6735,7 @@ proc ::punkboot::runtime_materialization_warning {rtfolder runtime_fullname} { #G-122: parsed BEFORE runtime discovery, because the mapping is what says which #store tiers this bake must look in - an entry declaring a target platform other #than the host default addresses its own bin/runtime/. -set mapfile $rt_sourcefolder/mapvfs.config +set mapfile [punkboot::lib::mapvfs_locate $rt_sourcefolder] set mapmodel [punkboot::lib::mapvfs_parse $mapfile $rtbase $sourcefolder $rt_default_target] foreach warnline [dict get $mapmodel warnings] { puts stderr $warnline @@ -6240,11 +6751,53 @@ set vfs_runtime_map [dict get $mapmodel vfs_runtime_map] set runtime_target [dict get $mapmodel runtime_target] set missing [dict get $mapmodel missing] if {[llength $missing]} { - puts stderr "WARNING [llength $missing] missing items from $mapfile. (TODO - prompt user to continue/abort)" - foreach m $missing { - puts stderr " $m" + if {[dict get $mapmodel format] eq "toml" && ![llength $::punkboot::bake_selected_kitnames]} { + #G-024 strictness (toml models): a full bake with unresolvable DEFAULT entries + #reports each one BY ENTRY rather than as a bare item list. A missing vfs + #folder is fatal - vfs folders are repo content, so its absence means a wrong + #declaration. A missing runtime FILE is environment, not declaration: store + #tiers are legitimately part-populated (fresh clones; runtimes arrive via the + #consent-gated fetch tiers), so those entries skip with a recapped + #BUILD-WARNING naming the entry (cross-target wording states the tier reason). + #bake_default=false entries are not in the full-bake set; selective bakes + #fail fast at selection instead. + set _strict_fatal [list] + set _strict_warn [list] + foreach _rec [punkboot::lib::mapvfs_kit_outputs $mapmodel $rtbase $sourcefolder] { + if {![dict get $_rec bake_default]} { + continue + } + set _entrylabel [expr {[dict get $_rec entry] ne "" ? [dict get $_rec entry] : [dict get $_rec kitname]}] + if {![dict get $_rec vfs_present]} { + lappend _strict_fatal "entry $_entrylabel: vfs folder src/vfs/[dict get $_rec vfs] is missing" + continue + } + if {[dict get $_rec runtime] ne "-" && ![dict get $_rec runtime_present]} { + if {[dict get $_rec target] eq $rt_default_target} { + lappend _strict_warn "kit [dict get $_rec kitname] (entry $_entrylabel) skipped - runtime [dict get $_rec runtime_file] not present in bin/runtime/[dict get $_rec store_tier]" + } else { + lappend _strict_warn "kit [dict get $_rec kitname] (entry $_entrylabel) skipped - cross-target runtime [dict get $_rec runtime_file] not present in bin/runtime/[dict get $_rec store_tier] (target [dict get $_rec target] tier not populated on this host)" + } + } + } + if {[llength $_strict_fatal]} { + puts stderr "CONFIG ERROR: unresolvable kit mapping entries in $mapfile - nothing built:" + foreach _line $_strict_fatal { + puts stderr " $_line" + } + exit 3 + } + if {[llength $_strict_warn]} { + ::punkboot::print_build_warnings $_strict_warn + } + unset -nocomplain _rec _entrylabel _strict_fatal _strict_warn _line + } else { + puts stderr "WARNING [llength $missing] missing items from $mapfile. (TODO - prompt user to continue/abort)" + foreach m $missing { + puts stderr " $m" + } + puts stderr "continuing..." } - puts stderr "continuing..." } # -- --- --- --- --- --- --- --- --- --- puts "-- runtime_vfs_map --" @@ -6333,7 +6886,7 @@ dict for {_rtname _rttarget} $runtime_target { dict set rtfolder_of $_file $_dir } if {[llength $cross_target_tiers]} { - puts stdout "cross-target runtime tiers in use (mapvfs.config target platforms): [join $cross_target_tiers {, }]" + puts stdout "cross-target runtime tiers in use (kit-mapping target platforms): [join $cross_target_tiers {, }]" } unset -nocomplain _rtname _rttarget _tier _dir _file if {![llength $runtimes]} { @@ -6342,7 +6895,7 @@ if {![llength $runtimes]} { if {![file isdirectory $rtfolder]} { #same store-tier self-diagnosis as bakelist puts stderr "NOTE: that runtime store folder does not exist. Default target '$rt_default_target' derives from THIS tclsh: tcl_platform(os)='$::tcl_platform(os)' tcl_platform(platform)='$::tcl_platform(platform)' -> host canon '$::punkboot::host_platform' -> target '$rt_default_target'." - puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.config for runtimes stored elsewhere." + puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.toml for runtimes stored elsewhere." } #todo - don't exit - it is valid to use runtime of - to just build a .kit/.zipkit ? exit 0 @@ -7323,6 +7876,10 @@ foreach vfstail $vfs_tails { } set rt_target [punkboot::lib::runtime_target_of $rt] set rt_file $rt[punkboot::lib::platform_exe_suffix $rt_target] + if {$rt_file in $runtimes} { + continue ;#several entries may share a runtime on one vfs (toml models) - + #process each runtime once (its spec set enumerates all its kits) + } if {[file exists $rtbase/[punkboot::lib::platform_store_tier $rt_target]/$rt_file]} { lappend runtimes $rt_file } @@ -7376,7 +7933,9 @@ foreach vfstail $vfs_tails { set applist [dict get $runtime_vfs_map $rtname] foreach vfs_app $applist { #5th element = G-133 smoke-require package list ("" when undeclared) + #6th element (toml models) = G-024 entry attrs (group/bake_default/scheme_role) lassign $vfs_app configured_vfs appname target_kit_type - kit_smokerequires + set kit_attrs [punkboot::lib::mapvfs_spec_attrs $vfs_app] if {$configured_vfs ne $vfstail} { continue } @@ -7398,7 +7957,7 @@ foreach vfstail $vfs_tails { } } lappend exe_names_seen $targetkit - lappend targetkits [list $targetkit $target_kit_type $kit_smokerequires] + lappend targetkits [list $targetkit $target_kit_type $kit_smokerequires $kit_attrs] } } puts stdout " vfs: $vfstail runtime: $rtname targetkits: $targetkits" @@ -7407,7 +7966,24 @@ foreach vfstail $vfs_tails { continue ;#G-121 selective bake - not a requested kit output } puts stdout " processing targetkit: $targetkit_info" - lassign $targetkit_info targetkit target_kit_type kit_smokerequires + lassign $targetkit_info targetkit target_kit_type kit_smokerequires kit_attrs + #G-024: bake_default=false entries (declared per entry or via their group) + #are excluded from full bakes - explicit name/@group selection builds them. + if {![llength $::punkboot::bake_selected_kitnames] + && [dict exists $kit_attrs bake_default] && ![dict get $kit_attrs bake_default]} { + puts stdout " skipping $targetkit (bake_default=false - bake by name or @group to build it)" + lappend skipped_kits [list kit $targetkit reason "bake_default=false (not part of full bakes)"] + continue + } + #G-023/G-024 release gating: a 'versioned' scheme's plain-name output is + #created when absent, then replaced only by an explicit release step + #(G-023 owns that step) - a normal bake never overwrites it. + if {[dict exists $kit_attrs scheme_role] && [dict get $kit_attrs scheme_role] eq "release" + && [file exists $binfolder/$targetkit]} { + puts stdout " skipping $targetkit (release-gated scheme output: bin/$targetkit exists and a normal bake never overwrites it - the explicit release step is G-023; to force a rebuild now, delete bin/$targetkit first)" + lappend skipped_kits [list kit $targetkit reason "release-gated (bin/$targetkit exists)"] + continue + } #Self-build guard: never process the kit whose deployed executable is running this #build - the process sweep below would kill this very process, and the deploy step #cannot replace a running image anyway. Informational/update subcommands are @@ -8299,7 +8875,7 @@ if {[llength $failed_kits]} { } set had_kits [expr {[llength $installed_kits] || [llength $failed_kits] || [llength $skipped_kits]}] if {$had_kits} { - puts stdout " module builds and kit/zipkit builds processed (vfs config: src/runtime/mapvfs.config)" + puts stdout " module builds and kit/zipkit builds processed (kit mapping: src/runtime/mapvfs.toml)" puts stdout " - use 'make.tcl modules' to build modules without baking the vfs folders into executable kits/zipkits" puts stdout " - use 'make.tcl vfscommonupdate' to promote built modules into the base vfs folder /src/vfs/_vfscommon.vfs (promotion gate)" puts stdout " - Note that without the vfscommonupdate step, 'make.tcl bake' (the kit-assembly stage of 'make.tcl bakehouse') will build vfs based executables" @@ -8309,7 +8885,7 @@ if {$had_kits} { puts stdout " module builds processed" puts stdout "" puts stdout " If kit/zipkit based executables required - create src/vfs/.vfs folders containing lib,modules,modules_tcl9 etc folders" - puts stdout " Also ensure appropriate executables exist in src/runtime along with src/runtime/mapvfs.config" + puts stdout " Also ensure appropriate runtime executables exist in bin/runtime/ along with the kit mapping src/runtime/mapvfs.toml" } puts stdout "-done-" exit 0 diff --git a/src/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.config b/src/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.config deleted file mode 100644 index c4d82bf4..00000000 --- a/src/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.config +++ /dev/null @@ -1,16 +0,0 @@ -#single line per runtime executable. Name of runtime followed by list of kit configuration lists. -#Each kit config list is from 1 to 3 elements -#1) .vfs folder with path relative to src folder. -#2) name of wrapped kit binary to create (default to rootname of .vfs folder e.g myapp.vfs will create a kit named myapp (or myapp.exe) on windows) -#3) Type of kit to create (zip or kit) - default kit. kit for metakit or vlerq based kit, zip for more modern zipkit. -# -# e.g The following creates 2 zipkits from the same runtime and 2 different .vfs folder -# mytcl90kit.exe {mainapp.vfs myapp.exe zip} {test.vfs testapp.exe zip} -# -#if runtime has no entry - it will only match a .vfs folder with a matching filename e.g runtime1.exe runtime1.vfs -#Use a runtime with a name of dash (-) to build a .kit file from the .vfs folder using no runtime -#e.g -#- myproject.vfs -#- punk86.vfs -# -tclsh902z.exe {project.vfs project zip} diff --git a/src/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.toml b/src/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.toml new file mode 100644 index 00000000..8cab3328 --- /dev/null +++ b/src/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.toml @@ -0,0 +1,34 @@ +# mapvfs.toml - kit output mapping (parsed with tomlish via make.tcl's ::punkboot::lib::mapvfs_parse). +# Maps runtime executables (/bin/runtime//) and .vfs payload folders +# (src/vfs/) to the named kit executables a bake produces. +# +# [kit.] - one table per kit output; is the output executable's base name. +# runtime (required) runtime executable name, unsuffixed (a superfluous .exe is tolerated). +# Use "-" to build a runtime-less .kit file. A runtime with NO entry anywhere +# still auto-pairs with a same-named .vfs folder. +# vfs (required) .vfs folder name under src/vfs/ (e.g "project.vfs"). +# type kit|zip|zipcat|cookit|cookfs (default kit). kit = metakit/starkit wrap, +# zip = zipfs image, zipcat = raw exe+zip concatenation. +# target (optional) target platform-dir name ('help platforms' lists them); defaults to +# the build host's own target. Decides the bin/runtime/ store, .exe +# suffixing and process tooling. A property of the RUNTIME - entries sharing a +# runtime must not disagree. +# smokerequire (optional) packages the freshly built kit must plain-'package require' +# (advisory probe; failures are recapped BUILD-WARNINGs). +# group (optional) named grouping - 'make.tcl bake @' / 'bakelist @'. +# bake_default (optional boolean, default true; a group's bake_default is the members' +# default) false = excluded from full bakes, still bakeable by name/@group. +# +# [group.] - optional group attributes: description, bake_default. +# [scheme.] - generative version-named outputs (scheme = "versioned"): expands to +# - + -dev + release-gated plain from the punkproject.toml +# [project] version. Keys: scheme, prefix (default: table name), runtime, vfs, type +# (required), plus target/smokerequire/group/bake_default as for kits. +# +# The punkshell tree's src/runtime/mapvfs.toml header carries the full format spec. +# (The pre-toml mapvfs.config line format remains readable as a deprecated fallback.) + +[kit.project] +runtime = "tclsh902z" +vfs = "project.vfs" +type = "zip" diff --git a/src/project_layouts/vendor/punk/project-0.1/src/vfs/README.md b/src/project_layouts/vendor/punk/project-0.1/src/vfs/README.md new file mode 100644 index 00000000..8397d864 --- /dev/null +++ b/src/project_layouts/vendor/punk/project-0.1/src/vfs/README.md @@ -0,0 +1,31 @@ +# src/vfs - kit payload folders and per-.vfs payload declarations + +Each `.vfs` folder is the payload of a configured kit output (kit mapping: +`src/runtime/mapvfs.toml`), merged with the `_vfscommon.vfs` overlay at bake time. + +## Per-.vfs payload declarations (optional) + +A `.vfs` folder's payload can be DECLARED in a sibling toml file and materialized by the +build - while the folder remains the operative assembly area: dropping files in by hand +needs no declaration, and materialization never clobbers what it did not itself install. +A `.vfs` with NO declaration file is untouched - pure drop-in mode. + +- Declaration file: `src/vfs/.vfs.toml` (sibling of the folder, never baked into it). +- Processed by `tclsh src/make.tcl vfslibs`, and automatically as a phase of + `make.tcl bake` / `bakehouse`. + +```toml +[payload.] +source = "vendorlib/mylib1.0" # folder path, resolved per source_root +source_root = "src" # optional: 'src' (default) or 'packages' + # (/bin/packages - 'make.tcl libfetch' tier) +target = "lib" # optional: folder INSIDE the .vfs (default: the .vfs root) +supersedes = ["mylib0.9"] # optional: legacy sibling folders removed from target first +replace = true # optional: clean-slate the SAME-NAMED target folder first +``` + +Precedence (drop-in wins): materialization is punkcheck-tracked with +`-overwrite synced-targets` - undeclared files are never touched, and a file at a declared +path is only overwritten when the mechanism itself installed it and it is unmodified +since; a foreign or hand-modified copy is preserved and reported. `supersedes` and +`replace` are the explicit declared exceptions. diff --git a/src/runtime/AGENTS.md b/src/runtime/AGENTS.md index 815cadce..ea0d9f6d 100644 --- a/src/runtime/AGENTS.md +++ b/src/runtime/AGENTS.md @@ -2,19 +2,19 @@ ## Purpose -Houses the `mapvfs.config` that maps VFS payloads to platform runtimes, plus the Tcl runtime executables and support files consumed during binary builds. +Houses the `mapvfs.toml` kit mapping (VFS payloads paired with platform runtimes into named kit outputs), the `libpackages.toml` lib-tier artifact declarations, plus support files consumed during binary builds. ## Ownership -- Agents must not modify runtime executables or `mapvfs.config` unless explicitly asked. +- Agents must not modify runtime executables or `mapvfs.toml` unless explicitly asked. - Runtime `.exe` files are platform binaries installed via `bin/punk-runtime.cmd` (renamed from `runtime.cmd` under G-097) and must not be edited. ## Local Contracts -- `mapvfs.config` defines which `src/vfs/*.vfs` folders combine with which runtime binaries (stored under `bin/runtime//`). -- Entry grammar: one line per runtime, ` {vfs ?kitname? ?kittype? ?targetplatform? ?smokerequires?} ...`. The file's own header comment is the user-facing spec; keep the two in step. -- **Smoke-require packages (5th element, G-133).** A kit entry may name packages the freshly built artifact must be able to plain-`package require`; the bake executes host-runnable kits via their tclsh subcommand and requires each one inside the real artifact (the only check that sees package resolution order - e.g a higher-versioned wrong-arch copy shadowing a working one). Advisory: failures are recapped `BUILD-WARNING`s; cross-target kits skip with a stated reason; undeclared kits run nothing new. Interim line format - the schema's eventual home is the G-024 toml conversion. Element positions are fixed, so declaring smoke-requires means also spelling out the target platform. -- **Target platform (4th element, G-122).** A kit config entry may name the canonical punkshell platform its runtime is for (`help platforms` lists the names). It decides which `bin/runtime//` tier the runtime is read from (macosx-* collapsing to the universal `macosx` folder), whether the runtime file and the built kit carry `.exe`, and whether the pre-deploy process sweep applies. It is a property of the RUNTIME: entries on one runtime line may repeat it but must not disagree (a conflict, or a name that is not an `-` platform-dir name, is a fatal config error). Omitted means the build host's default target - which for an msys2/cygwin-runtime tclsh is `win32-x86_64`, not that host's own canon. The live example is the `tclkit-902-Linux64-intel-dyn` entry (target `linux-x86_64`, suffixless `punkshell902` artifact). The schema's eventual home is the G-024 toml conversion; consumers read the parsed model (`::punkboot::lib::mapvfs_*`), never the file format. +- `mapvfs.toml` (G-024, tomlish-parsed) defines which `src/vfs/*.vfs` folders combine with which runtime binaries (stored under `bin/runtime//`) into which named kit outputs. The file's own header comment is the user-facing format spec; keep the two in step. `[kit.]` tables carry `runtime`, `vfs`, `type` (kit|zip|zipcat|cookit|cookfs), optional `target`, `smokerequire`, `group`, `bake_default`; `[group.]` tables name kit groupings (selectable in `bake`/`bakelist` as `@`; `bake_default = false` excludes members from full bakes while leaving them bakeable by name/group); `[scheme.]` tables are generative (G-023 `versioned` scheme: `-` + `-dev` + release-gated plain `` derived from `punkproject.toml` at parse time - the plain name is created when absent and never overwritten by a normal bake). Validation is strict and entry-named: unknown keys/types/schemes, conflicting targets and parse failures are fatal; a full bake refuses a default entry whose vfs folder is missing, and skips (recapped `BUILD-WARNING`) entries whose runtime file is absent from its store tier. Consumers read the parsed model (`::punkboot::lib::mapvfs_*`), never the file format. The pre-G-024 `mapvfs.config` line format remains readable as a DEPRECATED fallback (un-migrated generated projects; toml wins when both exist, legacy parses with a NOTE); the `PUNK_MAPVFS_CONFIG` env var points the reader at an alternate file (characterization-test seam). +- **Smoke-require packages (`smokerequire` key, G-133).** A kit entry may name packages the freshly built artifact must be able to plain-`package require`; the bake executes host-runnable kits via their tclsh subcommand and requires each one inside the real artifact (the only check that sees package resolution order - e.g a higher-versioned wrong-arch copy shadowing a working one). Advisory: failures are recapped `BUILD-WARNING`s; cross-target kits skip with a stated reason; undeclared kits run nothing new. +- **Target platform (`target` key, G-122).** A kit entry may name the canonical punkshell platform its runtime is for (`help platforms` lists the names). It decides which `bin/runtime//` tier the runtime is read from (macosx-* collapsing to the universal `macosx` folder), whether the runtime file and the built kit carry `.exe`, and whether the pre-deploy process sweep applies. It is a property of the RUNTIME: entries sharing a runtime may repeat it but must not disagree (a conflict, or a name that is not an `-` platform-dir name, is a fatal config error). Omitted means the build host's default target - which for an msys2/cygwin-runtime tclsh is `win32-x86_64`, not that host's own canon. The live example is the `[kit.punkshell902]` entry (target `linux-x86_64`, suffixless artifact). +- Per-.vfs payload declarations (`src/vfs/.vfs.toml`, G-115) are the sibling declaration surface for what goes INSIDE a `.vfs` folder - format and precedence in `src/vfs/README.md`. The former per-package `vendorlib_vfs.toml` here was migrated into those files 2026-07-31 and retired (a leftover copy is ignored with a warning). - Runtime executables are placed in `bin/runtime//` by the `bin/punk-runtime.cmd` helper, manually, or (G-103 family runtimes) copied from the suite_tcl90 `kit-family` build products under `src/buildsuites/_build/suite_tcl90/out/family/`. - `punkshell.ico` here is the project-DEFAULT kit icon: the bake icon step (G-057) records it in every kit's `.resources.toml` sidecar and embeds it into win32-target kits (per-kit override + format + skip semantics: bin/AGENTS.md). It is a derived copy of `src/assets/logo/punk-mark.ico` - its `.assetorigin.toml` sidecar records the derivation; regenerate via the logo pipeline, never edit the `.ico` here. - `libpackages.toml` (G-139) declares the punkbin LIB-TIER library artifacts this checkout consumes: `[artifact.]` entries with `name` (immutable `--r.zip` artifact filename) and `target` (`allplatforms` or a platform-dir name). Processed by `tclsh src/make.tcl libfetch` into the untracked `bin/packages//` input tier (sha1-verified against the server's per-target sha1sums.txt; trees materialized under `bin/packages//tcl/`). Consuming a new revision = editing the declared name here (deliberate, reviewable - mirroring the emission side's deliberate-publish discipline). The file's own header comment is the user-facing spec; keep the two in step. @@ -22,13 +22,13 @@ Houses the `mapvfs.config` that maps VFS payloads to platform runtimes, plus the ## Work Guidance -- When adding a new platform: create the runtime directory under `bin/runtime//`, add a VFS under `src/vfs/`, and update `mapvfs.config` - declaring the target platform on the entry when it is not the build host's default. +- When adding a new platform: create the runtime directory under `bin/runtime//`, add a VFS under `src/vfs/`, and add a `[kit.]` entry to `mapvfs.toml` - declaring the target platform on the entry when it is not the build host's default. - Do not commit large binary runtimes to version control unless specifically required. ## Verification -- `tclsh src/make.tcl bakelist` resolves every entry: each row's runtime presence, target and store tier (add a kit name for the per-kit detail block). -- `tclsh src/make.tcl bake` uses `mapvfs.config` correctly. +- `tclsh src/make.tcl bakelist` resolves every entry: each row's runtime presence, target, store tier, group and scheme notes (add a kit name or `@group` for the per-kit detail block). +- `tclsh src/make.tcl bake` uses `mapvfs.toml` correctly. - Built binaries launch and boot the Punk REPL. ## Child DOX Index diff --git a/src/runtime/mapvfs.config b/src/runtime/mapvfs.config deleted file mode 100644 index 39c89831..00000000 --- a/src/runtime/mapvfs.config +++ /dev/null @@ -1,150 +0,0 @@ -#single line per runtime executable. Name of runtime followed by list of kit configuration lists. -#Each kit config list is from 1 to 5 elements -#1) .vfs folder with path relative to src folder. -#2) name of wrapped kit binary to create (default to rootname of .vfs folder e.g myapp.vfs will create a kit named myapp (or myapp.exe) for a windows target) -#3) Type of kit to create (zip or kit) - default kit. kit for metakit or vlerq based kit, zip for more modern zipkit. -#4) TARGET PLATFORM (G-122) - canonical punkshell platform-dir name ('help platforms' lists them). -# Defaults to the build host's own target: win32-x86_64 on windows INCLUDING from an -# msys2/cygwin-runtime tclsh, whose own canon (msys-x86_64) is a host identity, not a kit target. -# The declared target decides everything the entry emits: which store tier holds the runtime -# (bin/runtime//, macosx-* collapsing to the universal macosx folder), whether the runtime -# file and the built kit carry .exe, and which process tooling the pre-deploy sweep uses. -# It is a property of the RUNTIME: entries on one runtime line may repeat it but must not disagree. -#5) SMOKE-REQUIRE packages (G-133) - a list of package names the freshly built kit must be able to -# plain-'package require'. After building, a host-runnable kit is executed via its tclsh -# subcommand and each named package is required inside the real artifact - the only check that -# sees package RESOLUTION (e.g a higher-versioned wrong-arch copy shadowing a working one). -# ADVISORY: failures become recapped BUILD-WARNINGs naming kit, package and error; the kit still -# builds and deploys. Cross-target kits skip with a stated reason; kits with no declaration run -# nothing new. Interim line format - the schema's eventual home is the G-024 toml conversion. -# e.g myruntime.exe {myapp.vfs myapp zip win32-x86_64 {Thread sqlite3}} -# (element positions are fixed: declaring smoke-requires means also spelling out the target) -# -# e.g The following creates 2 zipkits from the same runtime and 2 different .vfs folder -# mytcl90kit.exe {mainapp.vfs myapp.exe zip} {test.vfs testapp.exe zip} -# -# e.g a runtime kept in bin/runtime/linux-x86_64/ producing a suffixless linux kit: -# mylinuxruntime {myapp.vfs myapp kit linux-x86_64} -# -#if runtime has no entry - it will only match a .vfs folder with a matching filename e.g runtime1.exe runtime1.vfs -#Use a runtime with a name of dash (-) to build a .kit file from the .vfs folder using no runtime -#e.g -#- myproject.vfs -#- punk86.vfs -# - -#0 -tclkit86bi.exe {punk8win.vfs punkbi kit} - -#tclkit-win64-dyn.exe {punk86bawt.vfs punkbawt kit} - -#------------------------------------------------------------------------ -#broken 'lreplace' - but runtime beforehand is ok - thread library issue? -#tclkit-win64-dyn.exe {punk86bawt.vfs punksys kit} -#------------------------------------------------------------------------ -#same kit with different .vfs is ok -#1 -tclkit-win64-dyn.exe {punk8win.vfs punksys kit} - -#punk86: the Tcl 8.6 reference punk kit (8.6-behaviour checks - see src/tests/AGENTS.md and the -#test-interpreter notes). Remapped 2026-07-21: the previous unmapped tclkit8613-era build had -#fossilized (kit-stamped punk::args 0.1.0 winning unversioned requires while versioned requires -#pulled modern modules - the runtests punk::args::define skew). Rebuilt from the 8.6.17 dyn -#tclkit, matching the native Tcl86 test interpreter patchlevel (punksys stays on the 8.6.13 dyn). -tclkit-win64-dyn-8617.exe {punk8win.vfs punk86 kit} - -#tclkit87a5.exe {punk86.vfs punk87} {punk.vfs punkmain} - -#TCL9 -#tclsh902z.exe {project.vfs project zip} -#2 -#tclsh90b2 {punk9wintk90b2.vfs punk90b2 zip} -#3 -#tclsh90b4_piperepl.exe {punk9wintk90b2.vfs punk90b4 zip} -#tclsh901t.exe {punk92in.vfs punk901t zip} -tclsh90magic.exe {punk9magicsplat.vfs punkmagic zipcat} - -#made with Bawt (2025-08) -#tclkit -#tclkit902.exe {punk9win_for_tkruntime.vfs punk902kit kit} -#static build - with tk dll and tk lib added to zip -#4 -tclsh902z.exe {punk9win_for_tkruntime.vfs punk902z zip} - -#apn's tcl-sfe -#https://github.com/apnadkarni/tcl-sfe/releases -#tclsfe-x64.exe is not a tk runtime. The versions with tk are 'wish' based so unsuitable. -#punk9win_for_tkruntime.vfs is intended for kits which already have Tk -tclsfe-x64.exe {punk9wintk903.vfs punk91 zip} - -#suite_tcl90 zig build (G-096/G-098, both achieved 2026-07-20) - runtime copied manually from -# src/buildsuites/_build/suite_tcl90/out/bin/tclsh90szip.exe (static tclsh 9.0.5 with attached zip -# carrying the tcl library). tk/thread dlls + script libs ride in the .vfs for this entry; the -# G-103 family runtimes below supersede that arrangement. -#_beta convention: wrap freshly suite-built runtimes as *_beta kits even BEFORE they have passed the -# punk test suites - so new-runtime behaviour can be trialled interactively. Drop _beta on acceptance. -# (punk905_beta trialled and PROMOTED to punk905 2026-07-21.) -#'make.tcl bakelist ?kitname ...?' lists the kit outputs configured here (runtime/vfs presence + -# deployed state); 'make.tcl bake ?kitname ...?' bakes/deploys only the named kits (G-121, achieved - -# see goals/archive/G-121-bakelist-selective-bake.md; wish originally recorded in archived G-104). -#5 -tclsh905.exe {punk9wintk905.vfs punk905 zip} - -#G-103 runtime kit family (suite_tcl90 'kit-family' step): self-contained runtimes whose attached -# zip already carries the tcl library PLUS Thread/tclvfs/tcllib+tcllibc (bi adds Tk+tklib). WORKING -# names per the G-103 naming decision (dotted tcl patchlevel; piperepl-patched runtimes carry -# 'punk'; -r stays on the immutable punkbin artifact tier - 'runtime use' materializes an -# artifact into these names). Copied from src/buildsuites/_build/suite_tcl90/out/family/. -#punk runtime + the full punk vfs (which still carries its own tk/thread/etc payload - duplicated -# batteries resolve to the highest version, i.e. the runtime's; the vfs payload slims as G-103 -# progresses): -tclsh9.0.5-punk.exe {punk9wintk905.vfs punk9_beta zip} -#bi runtime already carries Tk - pair with the for-tkruntime vfs (no tk payload in the vfs): -tclsh9.0.5-punk-bi.exe {punk9win_for_tkruntime.vfs punk9bi_beta zip} - -#---------------------------------------------- -#experiment - what happens when we run against a 'wish' runtime? Will we have stdin stdout problems? -#tksfe-twapi-x64.exe {punk9wintk903.vfs punkwish91 zip} -# result so far: the kit builds - but the tk console doesn't display. -# we can run tk apps e.g punkwish91 lib:tkhello -# - todo - consider if we can first get the built in tk console showing for initial exploration, although it is unsuitable for a punk console as it doesn't handle any ansi. -# - see if we can develop a textwidget based virtual console that we can get punk shell running in. -#---------------------------------------------- - -#---------------------------------------------- -#testing wrong target kit type for runtime -#tclkit902.exe {punk9win_for_tkruntime.vfs punkwrongruntime zip} - -#only tclkit based runtime avail for linux for now - todo make zip-based runtime and install as artifact on gitea1.intx.com.au -#Cross-target entry (G-122): declares linux-x86_64 so the runtime is read from -#bin/runtime/linux-x86_64/ with no .exe suffix and the kit deploys as bin/punkshell902. -#Before the target declaration this row read runtime=missing from a windows host, -#because the runtime was looked for as tclkit-902-Linux64-intel-dyn.exe in the windows store. -#Smoke-require Thread (G-133): on a non-linux host this SKIPS with a stated reason (the -# artifact is not executable here) - a linux-hosted bake would probe the tclkit's Thread. -tclkit-902-Linux64-intel-dyn {punk9linux.vfs punkshell902 kit linux-x86_64 Thread} - -#G-129 verification runtimes: third-party 32-bit windows, bin/runtime/win32-ix86/ tier -# (see goals/archive/G-129-kitboot-mountpoint-derivation.md). Kept active so both zipfs -# mount conventions stay covered by real kits. -#These wrap the MINIMAL payloads punk8min.vfs / punk9min.vfs: boot fauxlinks plus -# multi-arch binary packages only (win32-ix86 subdir present: iocp-2.0.2 both majors, -# twapi-5.0b1 for tcl8), with the pure-tcl module/lib set arriving via the _vfscommon -# overlay. The original full punk8win/punk9win_for_tkruntime pairings shipped x64-only -# binary packages into 32-bit kits: a higher-versioned x64 Thread shadowed the runtime's -# working 32-bit copy (broke the repl), and x64-only dlls (tcludp etc) could never load -# (repointed 2026-07-27). -#androwish/undroidwish 8.6 zipfs backport (LUCK build): mounts its attached archive at the -# EXECUTABLE'S OWN PATH and has no tcl::zipfs::root - the boot derives the mount point -# from the runtime's own mount table. -#Smoke-require Thread (G-133): the 2026-07-27 incident kit - a payload x64 Thread 2.8.12 -# outversioned the runtime's working 32-bit 2.8.5, so plain 'package require Thread' -# selected the unloadable copy and the repl's tsv/codethread machinery died. The probe -# plain-requires Thread in the built artifact, so that class is caught by construction. -tclsh8.6.10-luck-zip.exe {punk8min.vfs punkluck86 zip win32-ix86 Thread} -#apn tcl-sfe 9.1b0 32-bit: modern //zipfs:/app convention - proves the 32-bit target -# itself needs nothing special. Runtime carries no Tk; Thread/twapi/sqlite3/tdbc are -# STATIC in this runtime (32-bit builtins - G-058 capture resolves them). -#Smoke-require (G-133): Thread resolves via the static builtin; iocp exercises the -# payload's platform-subdir (win32-ix86) dll selection inside the 32-bit artifact. -tclsh9.1b0-tclsfe.exe {punk9min.vfs punk91ix86 zip win32-ix86 {Thread iocp}} diff --git a/src/runtime/mapvfs.toml b/src/runtime/mapvfs.toml new file mode 100644 index 00000000..527984a2 --- /dev/null +++ b/src/runtime/mapvfs.toml @@ -0,0 +1,220 @@ +# mapvfs.toml - kit output mapping (G-024; parsed with tomlish via ::punkboot::lib::mapvfs_parse). +# Maps runtime executables (bin/runtime//) and .vfs payload folders (src/vfs/) to the +# named kit executables a bake produces. This header is the user-facing format spec - keep it in +# step with the reader ('make.tcl bakelist' shows how the file resolves; 'make.tcl bake ?name|@group ...?' +# builds it). The pre-G-024 line format mapvfs.config remains readable as a deprecated fallback in +# un-migrated trees; when both files exist this one wins. +# +# [kit.] - one table per kit output; is the output executable's base name. +# runtime (required) runtime executable name, unsuffixed (a superfluous .exe is tolerated +# and stripped so one config works cross-platform). Use "-" to build a runtime-less +# .kit file from the vfs folder. A runtime with NO entry anywhere still auto-pairs +# with a same-named .vfs folder, exactly as before. +# vfs (required) .vfs folder name under src/vfs/ (e.g "punk8win.vfs"). +# type kit|zip|zipcat|cookit|cookfs (default kit). kit = metakit/starkit wrap, +# zip = zipfs image, zipcat = raw exe+zip concatenation. +# target (optional) TARGET PLATFORM (G-122) - canonical punkshell platform-dir name +# ('help platforms' lists them). Defaults to the build host's own target +# (win32-x86_64 on windows INCLUDING from an msys2/cygwin-runtime tclsh). Decides +# which bin/runtime/ holds the runtime, .exe suffixing of runtime and kit, +# and which process tooling the pre-deploy sweep uses. It is a property of the +# RUNTIME: entries sharing a runtime may repeat it but must not disagree. +# smokerequire (optional) list of package names the freshly built kit must be able to +# plain-'package require' (G-133). Advisory: failures are recapped BUILD-WARNINGs; +# cross-target kits skip with a stated reason. +# group (optional) named grouping (G-024): purely organisational unless the group (or +# entry) sets bake_default=false. 'make.tcl bake @' / 'bakelist @' +# select by group. +# bake_default (optional boolean, default true; a group's bake_default is the members' default) +# false = excluded from full (no-name) bakes - still listed by bakelist (default=no) +# and bakeable by name or @group. +# +# [group.] - optional group attributes: description, bake_default. +# +# [scheme.] - generative entries (G-023/G-024) expanded at parse time; no per-version edits. +# scheme = "versioned" is the only kind: reads the punkproject.toml [project] version and expands +# to THREE outputs - - (refreshed while that version is current), -dev +# (always the latest build) and the release-gated plain (created when absent, then +# replaced only by an explicit release step - G-023 owns that step; a normal bake never +# overwrites it). Keys: scheme, prefix (default: table name), runtime, vfs, type (required), +# plus target/smokerequire/group/bake_default as for kits. +# +# Validation is strict and entry-named: unknown keys/types/schemes, conflicting targets and +# unparseable toml are fatal config errors; a full bake also refuses when a default entry's vfs +# folder (always) or host-target runtime (missing from its tier) cannot be resolved - +# cross-target entries whose tier is unpopulated on this host skip with a stated reason. +# +# PUNK_MAPVFS_CONFIG env var: points bakelist/bake at an alternate mapping file +# (characterization-test seam - exercise config behaviour without editing this file). + +#-- Tcl 8.6 kits ------------------------------------------------------------------------------ + +[kit.punkbi] +runtime = "tclkit86bi" +vfs = "punk8win.vfs" +type = "kit" + +# historical note (pre-toml): tclkit-win64-dyn paired with punk86bawt.vfs produced a kit with a +# broken 'lreplace' (thread library issue?) although the runtime itself was ok - the same runtime +# with punk8win.vfs (punksys) is fine. +[kit.punksys] +runtime = "tclkit-win64-dyn" +vfs = "punk8win.vfs" +type = "kit" + +# punk86: the Tcl 8.6 reference punk kit (8.6-behaviour checks - see src/tests/AGENTS.md and the +# test-interpreter notes). Remapped 2026-07-21: the previous unmapped tclkit8613-era build had +# fossilized (kit-stamped punk::args 0.1.0 winning unversioned requires while versioned requires +# pulled modern modules - the runtests punk::args::define skew). Rebuilt from the 8.6.17 dyn +# tclkit, matching the native Tcl86 test interpreter patchlevel (punksys stays on the 8.6.13 dyn). +[kit.punk86] +runtime = "tclkit-win64-dyn-8617" +vfs = "punk8win.vfs" +type = "kit" + +#-- Tcl 9 kits -------------------------------------------------------------------------------- + +[kit.punkmagic] +runtime = "tclsh90magic" +vfs = "punk9magicsplat.vfs" +type = "zipcat" + +# static build - with tk dll and tk lib added to zip +[kit.punk902z] +runtime = "tclsh902z" +vfs = "punk9win_for_tkruntime.vfs" +type = "zip" + +# apn's tcl-sfe https://github.com/apnadkarni/tcl-sfe/releases +# tclsfe-x64.exe is not a tk runtime. The versions with tk are 'wish' based so unsuitable. +# punk9win_for_tkruntime.vfs is intended for kits which already have Tk. +[kit.punk91] +runtime = "tclsfe-x64" +vfs = "punk9wintk903.vfs" +type = "zip" + +# suite_tcl90 zig build (G-096/G-098, both achieved 2026-07-20) - runtime copied manually from +# src/buildsuites/_build/suite_tcl90/out/bin/tclsh90szip.exe (static tclsh 9.0.5 with attached zip +# carrying the tcl library). tk/thread dlls + script libs ride in the .vfs for this entry; the +# G-103 family runtimes below supersede that arrangement. +# _beta convention: wrap freshly suite-built runtimes as *_beta kits even BEFORE they have passed +# the punk test suites - so new-runtime behaviour can be trialled interactively. Drop _beta on +# acceptance. (punk905_beta trialled and PROMOTED to punk905 2026-07-21.) +[kit.punk905] +runtime = "tclsh905" +vfs = "punk9wintk905.vfs" +type = "zip" + +# G-103 runtime kit family (suite_tcl90 'kit-family' step): self-contained runtimes whose attached +# zip already carries the tcl library PLUS Thread/tclvfs/tcllib+tcllibc (bi adds Tk+tklib). WORKING +# names per the G-103 naming decision (dotted tcl patchlevel; piperepl-patched runtimes carry +# 'punk'; -r stays on the immutable punkbin artifact tier - 'runtime use' materializes an +# artifact into these names). Copied from src/buildsuites/_build/suite_tcl90/out/family/. +# punk runtime + the full punk vfs (which still carries its own tk/thread/etc payload - duplicated +# batteries resolve to the highest version, i.e. the runtime's; the vfs payload slims as G-103 +# progresses): +[kit.punk9_beta] +runtime = "tclsh9.0.5-punk" +vfs = "punk9wintk905.vfs" +type = "zip" + +# bi runtime already carries Tk - pair with the for-tkruntime vfs (no tk payload in the vfs): +[kit.punk9bi_beta] +runtime = "tclsh9.0.5-punk-bi" +vfs = "punk9win_for_tkruntime.vfs" +type = "zip" + +# historical notes (pre-toml experiments): +# - 'wish' runtime experiment: tksfe-twapi-x64 {punk9wintk903.vfs punkwish91 zip} - the kit builds +# but the tk console doesn't display; tk apps run (punkwish91 lib:tkhello). Revisit with a +# textwidget-based virtual console if pursued. +# - wrong-target kit-type test row: tclkit902 {punk9win_for_tkruntime.vfs punkwrongruntime zip} + +#-- cross-target ------------------------------------------------------------------------------ + +# only tclkit based runtime avail for linux for now - todo make zip-based runtime and install as +# artifact on gitea1.intx.com.au +# Cross-target entry (G-122): target linux-x86_64 so the runtime is read from +# bin/runtime/linux-x86_64/ with no .exe suffix and the kit deploys as bin/punkshell902. +# smokerequire Thread (G-133): on a non-linux host this SKIPS with a stated reason (the artifact +# is not executable here) - a linux-hosted bake would probe the tclkit's Thread. +[kit.punkshell902] +runtime = "tclkit-902-Linux64-intel-dyn" +vfs = "punk9linux.vfs" +type = "kit" +target = "linux-x86_64" +smokerequire = ["Thread"] + +#-- G-129 verification kits (32-bit windows, third-party runtimes) ---------------------------- + +[group.verify-ix86] +description = "G-129 verification runtimes: third-party 32-bit windows, bin/runtime/win32-ix86/ tier - kept active so both zipfs mount conventions stay covered by real kits (see goals/archive/G-129-kitboot-mountpoint-derivation.md)" + +# These wrap the MINIMAL payloads punk8min.vfs / punk9min.vfs: boot fauxlinks plus multi-arch +# binary packages only (win32-ix86 subdir present: iocp-2.0.2 both majors, twapi-5.0b1 for tcl8), +# with the pure-tcl module/lib set arriving via the _vfscommon overlay. An x64-only binary package +# must NOT be added to these payloads - a higher-versioned wrong-arch copy silently shadows a +# runtime's own working 32-bit package on plain 'package require' (measured 2026-07-27: payload +# Thread 2.8.12 x64 beat the LUCK runtime's 2.8.5 and broke the repl). +# androwish/undroidwish 8.6 zipfs backport (LUCK build): mounts its attached archive at the +# EXECUTABLE'S OWN PATH and has no tcl::zipfs::root - the boot derives the mount point from the +# runtime's own mount table. +# smokerequire Thread (G-133): the 2026-07-27 incident kit - the probe plain-requires Thread in +# the built artifact, so that class is caught by construction. +[kit.punkluck86] +runtime = "tclsh8.6.10-luck-zip" +vfs = "punk8min.vfs" +type = "zip" +target = "win32-ix86" +smokerequire = ["Thread"] +group = "verify-ix86" + +# apn tcl-sfe 9.1b0 32-bit: modern //zipfs:/app convention - proves the 32-bit target itself needs +# nothing special. Runtime carries no Tk; Thread/twapi/sqlite3/tdbc are STATIC in this runtime +# (32-bit builtins - G-058 capture resolves them). smokerequire: Thread resolves via the static +# builtin; iocp exercises the payload's platform-subdir (win32-ix86) dll selection inside the +# 32-bit artifact. +[kit.punk91ix86] +runtime = "tclsh9.1b0-tclsfe" +vfs = "punk9min.vfs" +type = "zip" +target = "win32-ix86" +smokerequire = ["Thread", "iocp"] +group = "verify-ix86" + +#-- G-115 declarative-payload demonstration kit ----------------------------------------------- + +[group.declare-demo] +description = "G-115 demonstration: kit whose binary payload materializes from the punkdeclare.vfs.toml declaration (vendor tree + consent-gated packages tier) - bake with 'make.tcl bake punkdeclare'" +bake_default = false + +# punkdeclare.vfs is near-empty in VCS (boot fauxlink only): lib_tcl9 payload (tcludp from the +# vendor tree, tcllibc from the bin/packages tier) arrives via the per-.vfs declaration at +# src/vfs/punkdeclare.vfs.toml, and the smokerequire proves both binary packages resolve inside +# the built artifact. +[kit.punkdeclare] +runtime = "tclsh9.0.5-punk" +vfs = "punkdeclare.vfs" +type = "zip" +smokerequire = ["udp", "tcllibc"] +group = "declare-demo" + +#-- G-023 versioned naming schemes (declared, pending G-023 activation) ----------------------- + +[group.versioned] +description = "G-023 versioned/dev/release-gated outputs: punk- + punk-dev + release-gated punk, names derived from punkproject.toml at bake time. bake_default=false until G-023 activates - bake with '@versioned' or by name (bakelist shows the expanded names)" +bake_default = false + +[scheme.punk8] +scheme = "versioned" +runtime = "tclkit-win64-dyn-8617" +vfs = "punk8win.vfs" +type = "kit" +group = "versioned" + +[scheme.punk9] +scheme = "versioned" +runtime = "tclsh9.0.5-punk" +vfs = "punk9wintk905.vfs" +type = "zip" +group = "versioned" diff --git a/src/runtime/vendorlib_vfs.toml b/src/runtime/vendorlib_vfs.toml deleted file mode 100644 index f5b3ec29..00000000 --- a/src/runtime/vendorlib_vfs.toml +++ /dev/null @@ -1,122 +0,0 @@ -# vendorlib -> kit vfs platform-library propagation (goal G-037) -# -# Declares which vendored platform-library package folders (paths relative to src/) are -# installed into which kit vfs lib trees (paths relative to src/vfs/). Processed by -# 'tclsh src/make.tcl vfslibs' - and automatically as a phase of 'tclsh src/make.tcl bake' -# (and therefore of 'bakehouse'; G-112 folded the phase in so a bake cannot ship stale -# binary libs) - with punkcheck-tracked provenance (records in src/vfs/.punkcheck, -# outside kit payloads). -# -# Participation is per-package per-kit by design: kit vfs package sets intentionally differ -# (e.g. twapi versions vary per kit), so nothing is blanket-copied and a vendorlib drop does -# not force a package into every kit. -# -# Cross-generation installs are allowed when the package supports both Tcl major versions -# (e.g. tcludp 1.0.13 ships tcl9udp1013.dll + udp1013t.dll with a pkgIndex.tcl that selects -# by generation - so the same source folder serves lib_tcl9 and lib_tcl8 targets). -# -# Keys per [install.] entry: -# source - package folder path, resolved per source_root -# source_root - optional: 'src' (default; source is relative to src/ - the -# vendored-tree declarations) or 'packages' (G-139; source is -# relative to /bin/packages - the untracked punkbin -# lib-tier input populated by 'tclsh src/make.tcl libfetch') -# targets - list of kit vfs lib folders relative to src/vfs/ -# supersedes - optional list of legacy/older package folder names to remove from -# each target before install (explicit - no silent mixed-version -# provision, per the G-035 concerns). Cannot name the folder being -# installed (same-name entries are ignored) - use 'replace' for that. -# replace - optional boolean (G-139): clean-slate a SAME-NAMED existing target -# folder before install. For same-name refreshes where the old copy -# may carry strays the new tree lacks (the pre-G-139 hand-drops -# nested md5c/tcllibc INSIDE their tcllib2.0 folder). -# -# This file is a candidate for absorption into the mapvfs toml conversion (goal G-024). - -[install.tcludp] -source = "vendorlib_tcl9/win32-x86_64/tcludp1.0.13" -targets = [ - "punk9wintk90b2.vfs/lib_tcl9", - "punk9wintk903.vfs/lib_tcl9", - "punk9wintk905.vfs/lib_tcl9", - "punk9win_for_tkruntime.vfs/lib_tcl9", - "punk8win.vfs/lib_tcl8", -] -supersedes = ["udp1.0.12", "tcludp1.0.12"] - -# -- G-139: suite-built tcllib/tcllibc from the punkbin lib tier --------------- -# Sourced from the bin/packages input tier ('make.tcl libfetch'; artifacts -# declared in libpackages.toml). Targets = the kit vfs trees that carried -# hand-dropped copies at switchover (2026-07-30) - per-kit participation, never -# blanket. replace=true because the old tcl9 hand-drops nested md5c + tcllibc -# INSIDE tcllib2.0 (same folder name as the new install - supersedes cannot -# touch it). The tcl8 kits upgrade tcllib1.21 (+nested tcllibc 0.4 + md5c) -> -# tcllib 2.0 via supersedes. tcllibc_tcl9 is the win32-x86_64 build for the -# windows kits; punk9linux (linux target) gets the platform-neutral tcllib2.0 -# plus its OWN tcllibc from the linux-x86_64 tier (G-140: the suite_tcl90 -# tcllibc-linux cross build - replace retires the provenance-less hand-dropped -# tcllibc, supersedes the stray-named tcllibc2.0). punk86bawt carried a bare -# tcllibc only - it gets the tcllibc refresh and no tcllib entry. - -[install.tcllib_tcl9] -source_root = "packages" -source = "allplatforms/tcl9/tcllib2.0" -replace = true -targets = [ - "mkzipfix.vfs/lib_tcl9", - "punk9cook.vfs/lib_tcl9", - "punk9cookbasic.vfs/lib_tcl9", - "punk9linux.vfs/lib_tcl9", - "punk9magicsplat.vfs/lib_tcl9", - "punk9win_for_tkruntime.vfs/lib_tcl9", - "punk9wintk903.vfs/lib_tcl9", - "punk9wintk905.vfs/lib_tcl9", - "punk9wintk90b2.vfs/lib_tcl9", -] - -[install.tcllibc_tcl9] -source_root = "packages" -source = "win32-x86_64/tcl9/tcllibc" -replace = true -targets = [ - "mkzipfix.vfs/lib_tcl9", - "punk9cook.vfs/lib_tcl9", - "punk9cookbasic.vfs/lib_tcl9", - "punk9magicsplat.vfs/lib_tcl9", - "punk9win_for_tkruntime.vfs/lib_tcl9", - "punk9wintk903.vfs/lib_tcl9", - "punk9wintk905.vfs/lib_tcl9", - "punk9wintk90b2.vfs/lib_tcl9", -] -supersedes = ["tcllibc2.0"] - -[install.tcllibc_tcl9_linux] -source_root = "packages" -source = "linux-x86_64/tcl9/tcllibc" -replace = true -targets = [ - "punk9linux.vfs/lib_tcl9", -] -supersedes = ["tcllibc2.0"] - -[install.tcllib_tcl8] -source_root = "packages" -source = "allplatforms/tcl8/tcllib2.0" -replace = true -targets = [ - "punk86.vfs/lib_tcl8", - "punk8win.vfs/lib_tcl8", - "punk8_statictwapi.vfs/lib_tcl8", -] -supersedes = ["tcllib1.21"] - -[install.tcllibc_tcl8] -source_root = "packages" -source = "win32-x86_64/tcl8/tcllibc" -replace = true -targets = [ - "punk86.vfs/lib_tcl8", - "punk8win.vfs/lib_tcl8", - "punk8_statictwapi.vfs/lib_tcl8", - "punk86bawt.vfs/lib_tcl8", -] diff --git a/src/tests/shell/testsuites/punkexe/maketclbakelist.test b/src/tests/shell/testsuites/punkexe/maketclbakelist.test index 2a94919b..d46b8fb3 100644 --- a/src/tests/shell/testsuites/punkexe/maketclbakelist.test +++ b/src/tests/shell/testsuites/punkexe/maketclbakelist.test @@ -12,9 +12,13 @@ package require tcltest # - 'bakelist ' exits 1 naming the unknown and listing the configured names # - 'bake ' exits 1 BEFORE any build ("nothing built"), listing the # configured names - the no-build guarantee of selective bake -#The row pins are characterization of the current src/runtime/mapvfs.config kit +#The row pins are characterization of the current src/runtime/mapvfs.toml kit #matrix (punk91 = tclsfe-x64 + punk9wintk903.vfs, zip) - a deliberate config change #legitimately updates them. +#G-024 additions: @group selection, toml parsing exercised through the +#PUNK_MAPVFS_CONFIG env seam (fixture configs - the real mapping is never edited): +#versioned-scheme expansion, strict entry-named config errors, and the legacy +#mapvfs.config deprecation NOTE. # #Target executable resolved from env(PUNK_SHELL_TEST_EXE), else /bin/punk902z.exe #then /bin/punkshell902. Skipped (constraint punkexeavailable) if none found. @@ -145,6 +149,107 @@ namespace eval ::testspace { set result } -result {timedout 0 exitcode 1 nothingbuilt 1 confignames 1 nomachinery 1} + #added 2026-07-31 (agent, G-024) + test maketcl_bakelist_group_filter {bakelist @verify-ix86 selects the group's rows and no others} -constraints {punkexeavailable} -body { + set r [maketcl_run {bakelist @verify-ix86}] + set out [dict get $r output] + set result [list] + lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode] + lappend result luckrow [regexp {(?n)^punkluck86\s+zip\s+} $out] + lappend result sferow [regexp {(?n)^punk91ix86\s+zip\s+} $out] + lappend result groupnote [regexp {group=verify-ix86} $out] + lappend result otherrows [regexp {(?n)^(punk91|punk905|punkbi|punksys|punk86)\s} $out] + set result + } -result {timedout 0 exitcode 0 luckrow 1 sferow 1 groupnote 1 otherrows 0} + + #G-024 toml characterization runs through the PUNK_MAPVFS_CONFIG env seam so the + #REAL kit mapping is never edited (fixture files in the tcltest temp dir). + proc maketcl_run_mapseam {fixturename fixturelines cmdargs} { + set fixture [tcltest::makeFile [join $fixturelines \n] $fixturename] + set ::env(PUNK_MAPVFS_CONFIG) [file normalize $fixture] + try { + return [maketcl_run $cmdargs] + } finally { + unset -nocomplain ::env(PUNK_MAPVFS_CONFIG) + tcltest::removeFile $fixturename + } + } + + #added 2026-07-31 (agent, G-024) + test maketcl_bakelist_toml_scheme_expansion {fixture toml via PUNK_MAPVFS_CONFIG: one versioned scheme entry expands to - + -dev + release-gated rows} -constraints {punkexeavailable} -body { + set r [maketcl_run_mapseam fixture_scheme.toml { + {[kit.fixkit]} + {runtime = "tclsfe-x64"} + {vfs = "punk9wintk903.vfs"} + {type = "zip"} + {group = "fixg"} + {} + {[scheme.fixscheme]} + {scheme = "versioned"} + {prefix = "fixv"} + {runtime = "tclsfe-x64"} + {vfs = "punk9wintk903.vfs"} + {type = "zip"} + } {bakelist}] + set out [dict get $r output] + set result [list] + lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode] + lappend result fixkitrow [regexp {(?n)^fixkit\s+zip\s+tclsfe-x64\s+punk9wintk903\.vfs\s} $out] + lappend result groupnote [regexp {(?n)^fixkit\s.*group=fixg} $out] + #version derives from punkproject.toml at parse time - pin the shape, not the number + lappend result versionedrow [regexp {(?n)^fixv-\d+[^ ]*\s+zip\s+.*scheme=versioned} $out] + lappend result devrow [regexp {(?n)^fixv-dev\s+zip\s+.*scheme=dev} $out] + lappend result releaserow [regexp {(?n)^fixv\s+zip\s+.*scheme=release} $out] + #the real config's rows must not appear - the seam fully replaces the mapping + lappend result realrows [regexp {(?n)^punk91\s} $out] + set result + } -result {timedout 0 exitcode 0 fixkitrow 1 groupnote 1 versionedrow 1 devrow 1 releaserow 1 realrows 0} + + #added 2026-07-31 (agent, G-024) + test maketcl_bakelist_toml_configerror_names_entry {fixture toml with an unknown key: CONFIG FILE ERROR names the offending [kit.] entry and the key} -constraints {punkexeavailable} -body { + set r [maketcl_run_mapseam fixture_badkey.toml { + {[kit.badkit]} + {runtime = "tclsfe-x64"} + {vfs = "punk9wintk903.vfs"} + {typo_key = "zip"} + } {bakelist}] + set out [dict get $r output] + set result [list] + lappend result timedout [dict get $r timedout] + lappend result entrynamed [regexp {CONFIG FILE ERROR\. entry kit\.badkit .*unknown key 'typo_key'} $out] + set result + } -result {timedout 0 entrynamed 1} + + #added 2026-07-31 (agent, G-024) + test maketcl_bake_toml_configerror_aborts {bake against a fixture toml with an unknown scheme kind exits 3 before any build, naming the entry} -constraints {punkexeavailable} -body { + set r [maketcl_run_mapseam fixture_badscheme.toml { + {[scheme.badscheme]} + {scheme = "nosuchscheme"} + {runtime = "tclsfe-x64"} + {vfs = "punk9wintk903.vfs"} + {type = "zip"} + } {bake badname}] + set out [dict get $r output] + set result [list] + lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode] + lappend result entrynamed [regexp {CONFIG FILE ERROR\. entry scheme\.badscheme .*unknown scheme 'nosuchscheme'} $out] + lappend result nomachinery [expr {![regexp {processing targetkit:} $out]}] + set result + } -result {timedout 0 exitcode 3 entrynamed 1 nomachinery 1} + + #added 2026-07-31 (agent, G-024) + test maketcl_bakelist_legacy_deprecation_note {fixture legacy line-format config via the seam: parsed with the deprecation NOTE, rows still resolve} -constraints {punkexeavailable} -body { + set r [maketcl_run_mapseam fixture_legacy.config { + {tclsfe-x64.exe {punk9wintk903.vfs fixlegacy zip}} + } {bakelist}] + set out [dict get $r output] + set result [list] + lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode] + lappend result depnote [regexp {NOTE: .*deprecated mapvfs\.config line format} $out] + lappend result legacyrow [regexp {(?n)^fixlegacy\s+zip\s+tclsfe-x64\s+punk9wintk903\.vfs\s} $out] + set result + } -result {timedout 0 exitcode 0 depnote 1 legacyrow 1} + cleanupTests } namespace delete ::testspace diff --git a/src/tests/shell/testsuites/punkexe/maketclpayloadcheck.test b/src/tests/shell/testsuites/punkexe/maketclpayloadcheck.test index 6532f3d4..5e3de937 100644 --- a/src/tests/shell/testsuites/punkexe/maketclpayloadcheck.test +++ b/src/tests/shell/testsuites/punkexe/maketclpayloadcheck.test @@ -11,7 +11,7 @@ package require tcltest # - 'check' describes the smoke-require probe contract (host-runnable kits only, # cross-target kits skip with a stated reason, undeclared kits run nothing new) and # lists the kits declaring smoke-require packages. The declared list is -# characterization of the current src/runtime/mapvfs.config - a deliberate config +# characterization of the current src/runtime/mapvfs.toml - a deliberate config # change legitimately updates it (same stance as maketclbakelist.test row pins) # - the report stays ESC-free under the G-113 piped colour policy #The scan's VERDICT - what classifies and what mismatches - is characterized against @@ -151,13 +151,15 @@ namespace eval ::testspace { -result {timedout 0 exitcode 0 pinline 1 probe 1 warnshape 1 silence 1} #added 2026-07-27 (agent, G-133) - the declared list characterizes the current - #src/runtime/mapvfs.config smoke-require matrix (deliberate config changes update it) + #src/runtime/mapvfs.toml smoke-require matrix (deliberate config changes update it) + #updated 2026-07-31 (agent, G-024/G-115): + punkdeclare (udp, tcllibc) - the G-115 + #declarative-payload demonstration kit test maketcl_check_payloadchecks_declared_kits {check lists the kits currently declaring smoke-require packages}\ -constraints {punkexeavailable} -body { set r [maketcl_run {check}] set out [dict get $r output] set result [list] - lappend result declared [regexp -line {^\s+declared: punkluck86 \(Thread\); punkshell902 \(Thread\); punk91ix86 \(Thread, iocp\)\s*$} $out] + lappend result declared [regexp -line {^\s+declared: punkluck86 \(Thread\); punkshell902 \(Thread\); punk91ix86 \(Thread, iocp\); punkdeclare \(udp, tcllibc\)\s*$} $out] set result }\ -result {declared 1} diff --git a/src/tests/shell/testsuites/punkexe/maketclplatform.test b/src/tests/shell/testsuites/punkexe/maketclplatform.test index e08b286e..7487f0a3 100644 --- a/src/tests/shell/testsuites/punkexe/maketclplatform.test +++ b/src/tests/shell/testsuites/punkexe/maketclplatform.test @@ -8,14 +8,14 @@ package require tcltest # (host / target / store tier / exe suffix / process tooling) # - 'bakelist ' detail names the store tier and target of each kit, and a # default-target kit reports "(host default)" -# - a mapvfs.config entry declaring its own target platform resolves its runtime in +# - a mapvfs.toml entry declaring its own target platform resolves its runtime in # THAT platform's store tier, names its artifact with that platform's executable # convention (no .exe for linux-x86_64), and carries a target= note in the report # - (env-gated) an msys2/cygwin-runtime tclsh - which reports tcl_platform(platform) # 'unix' on windows and canonizes as msys-x86_64 - drives the SAME win32-x86_64 # kit set, with identical names and store addressing, as the native run # -#The per-kit row pins are characterization of the current src/runtime/mapvfs.config +#The per-kit row pins are characterization of the current src/runtime/mapvfs.toml #matrix (punk91 = tclsfe-x64 + punk9wintk903.vfs zip, default target; punkshell902 = #tclkit-902-Linux64-intel-dyn + punk9linux.vfs kit, target linux-x86_64) - a deliberate #config change legitimately updates them. @@ -201,7 +201,7 @@ namespace eval ::testspace { lappend result timedout [dict get $r timedout] exitcode [dict get $r exitcode] lappend result row [regexp {(?n)^punkshell902\s+kit\s+tclkit-902-Linux64-intel-dyn\s+punk9linux\.vfs\s+\S+\s+target=linux-x86_64} $out] lappend result store [regexp {(?n)^\s+runtime file:\s+bin/runtime/linux-x86_64/tclkit-902-Linux64-intel-dyn \(present\)} $out] - lappend result target [regexp {(?n)^\s+target:\s+linux-x86_64 \(declared in mapvfs\.config\)} $out] + lappend result target [regexp {(?n)^\s+target:\s+linux-x86_64 \(declared in the kit mapping\)} $out] #linux target -> no .exe on the build product or the deployed artifact lappend result nosuffix [regexp {(?n)^\s+build product: src/_build/punkshell902 } $out] lappend result nosuffix2 [regexp {(?n)^\s+deployed:\s+bin/punkshell902 } $out] diff --git a/src/vfs/AGENTS.md b/src/vfs/AGENTS.md index 9b571051..63aafe1c 100644 --- a/src/vfs/AGENTS.md +++ b/src/vfs/AGENTS.md @@ -13,6 +13,8 @@ VFS (Virtual File System) folders define the runtime payloads that get wrapped i ## Local Contracts - `*.vfs` folders are build artifacts consumed by the kit-assembly stage `tclsh src/make.tcl bake` (and its consumer umbrella `bakehouse`). +- **Per-.vfs payload declarations (G-115).** A sibling `src/vfs/.vfs.toml` may declare package folders to materialize INTO `.vfs` (sources: vendor trees under `src/`, or the consent-gated `bin/packages` lib tier) - processed by `make.tcl vfslibs` and as a bake phase, punkcheck-tracked (records in `src/vfs/.punkcheck`) with drop-in-wins precedence: undeclared or hand-modified files are preserved, never overwritten (`supersedes`/`replace` are the explicit declared exceptions). A `.vfs` with no declaration file is untouched (pure drop-in mode). Format + precedence spec: `src/vfs/README.md`. This surface supersedes the retired per-package `src/runtime/vendorlib_vfs.toml` (migrated 2026-07-31). +- `punkdeclare.vfs` + `punkdeclare.vfs.toml` are the G-115 demonstration kit: VCS carries only the boot fauxlink; the whole `lib_tcl9/` binary payload (tcludp vendor tree + tcllibc packages tier) materializes from the declaration and is deliberately ignored in both VCS. Bake with `make.tcl bake -confirm 0 punkdeclare` (bake_default=false keeps it out of full bakes); its smoke-require (`udp`, `tcllibc`) proves the declared payload resolves inside the built artifact. - A `*.vfs` folder built against a runtime is expected to carry a root-level startup script: an actual `main.tcl`, or a root fauxlink resolving to the name `main.tcl` whose target exists and is a `.tcl` file. `make.tcl` warns when neither is present (column-0 `BUILD-WARNING:` token, ANSI-highlighted, recapped at end of run like provenance warnings) but still builds - a kit without a startup script is legal. Overlay-only folders with no runtime mapping (e.g `_vfscommon.vfs`) are not checked. - VFS content must be compatible with the target platform runtime. - A root-level `punkshell.ico` in a kit's own custom `.vfs` folder overrides @@ -26,13 +28,13 @@ VFS (Virtual File System) folders define the runtime payloads that get wrapped i - `src` mode loads unbuilt modules and libraries from the project's `src/` tree (`src/modules`, `src/lib`, `src/bootsupport`, `src/vendormodules`) instead of built output. It also runs an inline `#modpod` package-ifneeded scanner (using Tcl builtins only, since no punk modules are loaded at boot time) and sets `package prefer latest` so `999999.0a1.0` dev modules are preferred over stable bootsupport/vendored copies. - `_config/punk_main.tcl` and `_config/project_main.tcl` derive the kit's payload location from the runtime's own zipfs mount table (G-129): zipfs presence keys on `tcl::zipfs::mount` (not `tcl::zipfs::root`, which the androwish/undroidwish 8.6 backport lacks), the mount base is the table entry whose archive file is `[info nameofexecutable]` (fallback: the mount containing the boot script), `//zipfs:/app` stays the expected modern answer rather than a compiled-in assumption, and a mounted archive that cannot be attributed to the executable is reported on stderr. The derivation lives in `::punkboot` procs (`zipfs_kit_mountbase`, `zipfs_mount_pairs`, `path_within`) that remain defined after boot for probing and tests. Both files carry identical copies of this logic (G-031 is the planned consolidation). Characterization: `src/tests/shell/testsuites/punkexe/kitmountpoint.test`. - `punk8win.vfs` carries TWO startup-script fauxlinks resolving to `_config/punk_main.tcl`: root `main.tcl` (modern zipfs and metakit convention) and `app/main.tcl` (the undroidwish family sources `app/main.tcl` INSIDE the attached archive; the file is inert in kits whose runtime does not look for it). Fauxlink targets are link-location-relative (symlink semantics): `make.tcl`'s `merge_over` resolves them against the link file's own directory, which is indistinguishable from the old vfs-root-relative resolution for root-level links. -- `punk8min.vfs` / `punk9min.vfs` are MINIMAL payloads for the win32-ix86 verification kits (punkluck86 / punk91ix86 in `src/runtime/mapvfs.config`): both startup fauxlinks plus directory fauxlinks to MULTI-ARCH binary packages only - packages whose own tree carries a `win32-ix86` platform subdir (iocp-2.0.2 for both majors, twapi-5.0b1 for tcl8), linked from their existing homes in sibling `.vfs` folders so no binary is duplicated in VCS. Everything pure-tcl arrives via the `_vfscommon.vfs` overlay (which deliberately contains no dlls). Composition rule: an x64-only binary package must NOT be added to these payloads - a higher-versioned wrong-arch copy silently shadows a runtime's own working 32-bit package on plain `package require` (measured 2026-07-27: payload Thread 2.8.12 x64 beat the LUCK runtime's 2.8.5 and broke the repl), and wrong-arch-only packages surface as "couldn't load from current filesystem" at require time. +- `punk8min.vfs` / `punk9min.vfs` are MINIMAL payloads for the win32-ix86 verification kits (punkluck86 / punk91ix86 in `src/runtime/mapvfs.toml`, group `verify-ix86`): both startup fauxlinks plus directory fauxlinks to MULTI-ARCH binary packages only - packages whose own tree carries a `win32-ix86` platform subdir (iocp-2.0.2 for both majors, twapi-5.0b1 for tcl8), linked from their existing homes in sibling `.vfs` folders so no binary is duplicated in VCS. Everything pure-tcl arrives via the `_vfscommon.vfs` overlay (which deliberately contains no dlls). Composition rule: an x64-only binary package must NOT be added to these payloads - a higher-versioned wrong-arch copy silently shadows a runtime's own working 32-bit package on plain `package require` (measured 2026-07-27: payload Thread 2.8.12 x64 beat the LUCK runtime's 2.8.5 and broke the repl), and wrong-arch-only packages surface as "couldn't load from current filesystem" at require time. - `dev` mode loads built output from `/modules` and `/lib`; `src` mode loads unbuilt source from `/src/modules` etc. Use `src-dev-internal` to combine both with src priority. ## Work Guidance -- To add a module to a VFS build, update the VFS folder's module set, then run `tclsh src/make.tcl bake`. -- The `src/runtime/mapvfs.config` file maps VFS folders to platform runtimes. +- To add a module to a VFS build, update the VFS folder's module set (drop-in, or a per-.vfs declaration entry), then run `tclsh src/make.tcl bake`. +- The `src/runtime/mapvfs.toml` file maps VFS folders to platform runtimes and kit outputs. - Describe regeneration steps in commit messages when touching VFS payloads. ## Verification @@ -44,4 +46,5 @@ VFS (Virtual File System) folders define the runtime payloads that get wrapped i - `_config/` — Entry-point .tcl files for built executables - `_vfscommon.vfs/` — Auto-generated common VFS libraries (do not edit) -- Various `*.vfs/` — Platform-specific VFS images \ No newline at end of file +- Various `*.vfs/` — Platform-specific VFS images +- `README.md` — Per-.vfs payload declaration format + drop-in precedence spec (G-115) \ No newline at end of file diff --git a/src/vfs/README.md b/src/vfs/README.md new file mode 100644 index 00000000..d590715a --- /dev/null +++ b/src/vfs/README.md @@ -0,0 +1,75 @@ +# src/vfs - kit payload folders and per-.vfs payload declarations + +Each `.vfs` folder is the payload of a configured kit output (kit mapping: +`src/runtime/mapvfs.toml`), merged with the auto-generated `_vfscommon.vfs` overlay at bake +time. Folder ownership and payload rules: `src/vfs/AGENTS.md`. + +## Per-.vfs payload declarations (G-115) + +A `.vfs` folder's payload can be DECLARED in a sibling toml file and materialized by the +build - while the folder remains the operative assembly area: dropping files in by hand +needs no declaration, and materialization never clobbers what it did not itself install. + +- Declaration file: `src/vfs/.vfs.toml` (sibling of `src/vfs/.vfs/`, so the + declaration itself is never part of the kit payload). +- Processed by `tclsh src/make.tcl vfslibs`, and automatically as a phase of + `make.tcl bake` / `bakehouse` (a selective bake narrows to the selected kits' folders). +- A `.vfs` with NO declaration file is untouched - pure drop-in mode, exactly as before. + +### Entry format + +```toml +[payload.] +source = "vendorlib_tcl9/win32-x86_64/tcludp1.0.13" # folder path, resolved per source_root +source_root = "src" # optional: 'src' (default) or 'packages' +target = "lib_tcl9" # optional: folder INSIDE the .vfs (default: the .vfs root) +supersedes = ["udp1.0.12"] # optional: legacy sibling folders removed from target first +replace = true # optional: clean-slate the SAME-NAMED target folder first +``` + +- `source_root = "src"` resolves against `src/` - vendor trees (`vendorlib*`, + `vendormodules*`), suite build products, any folder under src. +- `source_root = "packages"` resolves against `/bin/packages` - the untracked, + consent-gated punkbin lib tier populated by `tclsh src/make.tcl libfetch` (artifact + declarations + sha1 verification: `src/runtime/libpackages.toml`). This is how declared + BINARY content arrives on a clean tree without being committed (G-004 direction). +- The source folder is installed AS `///`. +- `target` must be a relative path inside the `.vfs` folder (no `..`). +- Unknown keys, missing sources, or an unparseable file fail the build naming the entry. + +### Precedence (drop-in wins) + +Materialization is punkcheck-tracked (records in `src/vfs/.punkcheck`, outside kit +payloads) using `-overwrite synced-targets`: + +1. Files the mechanism never declared are never touched (undeclared drop-ins survive + re-materialization by construction). +2. A file at a declared path is only overwritten when punkcheck's records show the + mechanism itself installed it AND it is unmodified since. A pre-existing foreign copy + (no install record) or a hand-modified installed copy is PRESERVED and reported + ("changed file(s) NOT overwritten" note; per-file punkcheck skip lines). +3. The declared, explicit exceptions: `supersedes` removes the named legacy sibling + folders, and `replace = true` clean-slates the same-named package folder before + install. Use `replace`, or remove the files, to hand a collided path back to the + declaration. + +### Relationship to the former vendorlib_vfs.toml + +The per-package `src/runtime/vendorlib_vfs.toml` surface (G-037/G-139: `[install.]` +entries with cross-kit target lists) was migrated into these per-.vfs files 2026-07-31 and +retired: per-kit files keep each kit's composition readable in one place, give new kits a +copy-a-sibling starting point, and are the surface later goals consume (G-141 derivation, +G-127 per-target payloads). A leftover `vendorlib_vfs.toml` is ignored with a warning. +Rationale and decision record: `goals/G-115-declarative-vfs-composition.md` (archived to +`goals/archive/` once the goal flips). + +### Demonstration kit + +`punkdeclare.vfs` + `punkdeclare.vfs.toml` demonstrate the mechanism end-to-end: the +folder in VCS carries only the boot fauxlink; its whole binary payload (tcludp from the +vendor tree, tcllibc from the packages tier) materializes from the declaration, and the +kit's smoke-require (`udp`, `tcllibc` - see `[kit.punkdeclare]` in mapvfs.toml) proves the +payload resolves inside the built artifact. `tclsh src/make.tcl bake -confirm 0 punkdeclare` +builds it (bake_default=false keeps it out of full bakes). The materialized +`punkdeclare.vfs/lib_tcl9/` tree is deliberately NOT committed (ignored in both VCS) - +reproducing it from the declaration is the point. diff --git a/src/vfs/mkzipfix.vfs.toml b/src/vfs/mkzipfix.vfs.toml new file mode 100644 index 00000000..23208789 --- /dev/null +++ b/src/vfs/mkzipfix.vfs.toml @@ -0,0 +1,21 @@ +# Declarative payload for src/vfs/mkzipfix.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# G-139: suite-built tcllib/tcllibc from the punkbin lib tier (bin/packages input tier populated +# by 'tclsh src/make.tcl libfetch'; artifacts declared in src/runtime/libpackages.toml). +# replace=true because the old tcl9 hand-drops nested md5c + tcllibc INSIDE tcllib2.0 (same +# folder name as the new install - supersedes cannot touch it). +[payload.tcllib] +source_root = "packages" +source = "allplatforms/tcl9/tcllib2.0" +target = "lib_tcl9" +replace = true + +[payload.tcllibc] +source_root = "packages" +source = "win32-x86_64/tcl9/tcllibc" +target = "lib_tcl9" +replace = true +supersedes = ["tcllibc2.0"] diff --git a/src/vfs/punk86.vfs.toml b/src/vfs/punk86.vfs.toml new file mode 100644 index 00000000..3552cb96 --- /dev/null +++ b/src/vfs/punk86.vfs.toml @@ -0,0 +1,20 @@ +# Declarative payload for src/vfs/punk86.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# G-139: suite-built tcllib/tcllibc from the punkbin lib tier (bin/packages input tier populated +# by 'tclsh src/make.tcl libfetch'; artifacts declared in src/runtime/libpackages.toml). The tcl8 +# kits upgrade tcllib1.21 (+nested tcllibc 0.4 + md5c) -> tcllib 2.0 via supersedes. +[payload.tcllib] +source_root = "packages" +source = "allplatforms/tcl8/tcllib2.0" +target = "lib_tcl8" +replace = true +supersedes = ["tcllib1.21"] + +[payload.tcllibc] +source_root = "packages" +source = "win32-x86_64/tcl8/tcllibc" +target = "lib_tcl8" +replace = true diff --git a/src/vfs/punk86bawt.vfs.toml b/src/vfs/punk86bawt.vfs.toml new file mode 100644 index 00000000..215a2afe --- /dev/null +++ b/src/vfs/punk86bawt.vfs.toml @@ -0,0 +1,12 @@ +# Declarative payload for src/vfs/punk86bawt.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# punk86bawt carried a bare tcllibc only - it gets the tcllibc refresh and no tcllib entry +# (G-139 migration note). +[payload.tcllibc] +source_root = "packages" +source = "win32-x86_64/tcl8/tcllibc" +target = "lib_tcl8" +replace = true diff --git a/src/vfs/punk8_statictwapi.vfs.toml b/src/vfs/punk8_statictwapi.vfs.toml new file mode 100644 index 00000000..260915ec --- /dev/null +++ b/src/vfs/punk8_statictwapi.vfs.toml @@ -0,0 +1,20 @@ +# Declarative payload for src/vfs/punk8_statictwapi.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# G-139: suite-built tcllib/tcllibc from the punkbin lib tier (bin/packages input tier populated +# by 'tclsh src/make.tcl libfetch'; artifacts declared in src/runtime/libpackages.toml). The tcl8 +# kits upgrade tcllib1.21 (+nested tcllibc 0.4 + md5c) -> tcllib 2.0 via supersedes. +[payload.tcllib] +source_root = "packages" +source = "allplatforms/tcl8/tcllib2.0" +target = "lib_tcl8" +replace = true +supersedes = ["tcllib1.21"] + +[payload.tcllibc] +source_root = "packages" +source = "win32-x86_64/tcl8/tcllibc" +target = "lib_tcl8" +replace = true diff --git a/src/vfs/punk8win.vfs.toml b/src/vfs/punk8win.vfs.toml new file mode 100644 index 00000000..1da7dc50 --- /dev/null +++ b/src/vfs/punk8win.vfs.toml @@ -0,0 +1,27 @@ +# Declarative payload for src/vfs/punk8win.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# tcludp 1.0.13 (vendor tree; G-037 lineage). Cross-generation install: the tcl9-tree source +# folder ships udp1013t.dll + a pkgIndex.tcl selecting by generation, so it serves lib_tcl8 too. +[payload.tcludp] +source = "vendorlib_tcl9/win32-x86_64/tcludp1.0.13" +target = "lib_tcl8" +supersedes = ["udp1.0.12", "tcludp1.0.12"] + +# G-139: suite-built tcllib/tcllibc from the punkbin lib tier (bin/packages input tier populated +# by 'tclsh src/make.tcl libfetch'; artifacts declared in src/runtime/libpackages.toml). The tcl8 +# kits upgrade tcllib1.21 (+nested tcllibc 0.4 + md5c) -> tcllib 2.0 via supersedes. +[payload.tcllib] +source_root = "packages" +source = "allplatforms/tcl8/tcllib2.0" +target = "lib_tcl8" +replace = true +supersedes = ["tcllib1.21"] + +[payload.tcllibc] +source_root = "packages" +source = "win32-x86_64/tcl8/tcllibc" +target = "lib_tcl8" +replace = true diff --git a/src/vfs/punk9cook.vfs.toml b/src/vfs/punk9cook.vfs.toml new file mode 100644 index 00000000..dda4053c --- /dev/null +++ b/src/vfs/punk9cook.vfs.toml @@ -0,0 +1,21 @@ +# Declarative payload for src/vfs/punk9cook.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# G-139: suite-built tcllib/tcllibc from the punkbin lib tier (bin/packages input tier populated +# by 'tclsh src/make.tcl libfetch'; artifacts declared in src/runtime/libpackages.toml). +# replace=true because the old tcl9 hand-drops nested md5c + tcllibc INSIDE tcllib2.0 (same +# folder name as the new install - supersedes cannot touch it). +[payload.tcllib] +source_root = "packages" +source = "allplatforms/tcl9/tcllib2.0" +target = "lib_tcl9" +replace = true + +[payload.tcllibc] +source_root = "packages" +source = "win32-x86_64/tcl9/tcllibc" +target = "lib_tcl9" +replace = true +supersedes = ["tcllibc2.0"] diff --git a/src/vfs/punk9cookbasic.vfs.toml b/src/vfs/punk9cookbasic.vfs.toml new file mode 100644 index 00000000..27c3a42e --- /dev/null +++ b/src/vfs/punk9cookbasic.vfs.toml @@ -0,0 +1,21 @@ +# Declarative payload for src/vfs/punk9cookbasic.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# G-139: suite-built tcllib/tcllibc from the punkbin lib tier (bin/packages input tier populated +# by 'tclsh src/make.tcl libfetch'; artifacts declared in src/runtime/libpackages.toml). +# replace=true because the old tcl9 hand-drops nested md5c + tcllibc INSIDE tcllib2.0 (same +# folder name as the new install - supersedes cannot touch it). +[payload.tcllib] +source_root = "packages" +source = "allplatforms/tcl9/tcllib2.0" +target = "lib_tcl9" +replace = true + +[payload.tcllibc] +source_root = "packages" +source = "win32-x86_64/tcl9/tcllibc" +target = "lib_tcl9" +replace = true +supersedes = ["tcllibc2.0"] diff --git a/src/vfs/punk9linux.vfs.toml b/src/vfs/punk9linux.vfs.toml new file mode 100644 index 00000000..4cfac05c --- /dev/null +++ b/src/vfs/punk9linux.vfs.toml @@ -0,0 +1,20 @@ +# Declarative payload for src/vfs/punk9linux.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# G-139/G-140: platform-neutral tcllib plus this linux-target kit's OWN tcllibc from the +# linux-x86_64 tier (the suite_tcl90 tcllibc-linux cross build). replace retires the +# provenance-less hand-dropped tcllibc; supersedes removes the stray-named tcllibc2.0. +[payload.tcllib] +source_root = "packages" +source = "allplatforms/tcl9/tcllib2.0" +target = "lib_tcl9" +replace = true + +[payload.tcllibc] +source_root = "packages" +source = "linux-x86_64/tcl9/tcllibc" +target = "lib_tcl9" +replace = true +supersedes = ["tcllibc2.0"] diff --git a/src/vfs/punk9magicsplat.vfs.toml b/src/vfs/punk9magicsplat.vfs.toml new file mode 100644 index 00000000..10e441cb --- /dev/null +++ b/src/vfs/punk9magicsplat.vfs.toml @@ -0,0 +1,21 @@ +# Declarative payload for src/vfs/punk9magicsplat.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# G-139: suite-built tcllib/tcllibc from the punkbin lib tier (bin/packages input tier populated +# by 'tclsh src/make.tcl libfetch'; artifacts declared in src/runtime/libpackages.toml). +# replace=true because the old tcl9 hand-drops nested md5c + tcllibc INSIDE tcllib2.0 (same +# folder name as the new install - supersedes cannot touch it). +[payload.tcllib] +source_root = "packages" +source = "allplatforms/tcl9/tcllib2.0" +target = "lib_tcl9" +replace = true + +[payload.tcllibc] +source_root = "packages" +source = "win32-x86_64/tcl9/tcllibc" +target = "lib_tcl9" +replace = true +supersedes = ["tcllibc2.0"] diff --git a/src/vfs/punk9win_for_tkruntime.vfs.toml b/src/vfs/punk9win_for_tkruntime.vfs.toml new file mode 100644 index 00000000..590d32b6 --- /dev/null +++ b/src/vfs/punk9win_for_tkruntime.vfs.toml @@ -0,0 +1,28 @@ +# Declarative payload for src/vfs/punk9win_for_tkruntime.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# tcludp 1.0.13 (vendor tree; G-037 lineage). Ships tcl9udp1013.dll + udp1013t.dll with a +# pkgIndex.tcl that selects by generation - the same source folder serves tcl9 and tcl8 kits. +[payload.tcludp] +source = "vendorlib_tcl9/win32-x86_64/tcludp1.0.13" +target = "lib_tcl9" +supersedes = ["udp1.0.12", "tcludp1.0.12"] + +# G-139: suite-built tcllib/tcllibc from the punkbin lib tier (bin/packages input tier populated +# by 'tclsh src/make.tcl libfetch'; artifacts declared in src/runtime/libpackages.toml). +# replace=true because the old tcl9 hand-drops nested md5c + tcllibc INSIDE tcllib2.0 (same +# folder name as the new install - supersedes cannot touch it). +[payload.tcllib] +source_root = "packages" +source = "allplatforms/tcl9/tcllib2.0" +target = "lib_tcl9" +replace = true + +[payload.tcllibc] +source_root = "packages" +source = "win32-x86_64/tcl9/tcllibc" +target = "lib_tcl9" +replace = true +supersedes = ["tcllibc2.0"] diff --git a/src/vfs/punk9wintk903.vfs.toml b/src/vfs/punk9wintk903.vfs.toml new file mode 100644 index 00000000..af88c381 --- /dev/null +++ b/src/vfs/punk9wintk903.vfs.toml @@ -0,0 +1,28 @@ +# Declarative payload for src/vfs/punk9wintk903.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# tcludp 1.0.13 (vendor tree; G-037 lineage). Ships tcl9udp1013.dll + udp1013t.dll with a +# pkgIndex.tcl that selects by generation - the same source folder serves tcl9 and tcl8 kits. +[payload.tcludp] +source = "vendorlib_tcl9/win32-x86_64/tcludp1.0.13" +target = "lib_tcl9" +supersedes = ["udp1.0.12", "tcludp1.0.12"] + +# G-139: suite-built tcllib/tcllibc from the punkbin lib tier (bin/packages input tier populated +# by 'tclsh src/make.tcl libfetch'; artifacts declared in src/runtime/libpackages.toml). +# replace=true because the old tcl9 hand-drops nested md5c + tcllibc INSIDE tcllib2.0 (same +# folder name as the new install - supersedes cannot touch it). +[payload.tcllib] +source_root = "packages" +source = "allplatforms/tcl9/tcllib2.0" +target = "lib_tcl9" +replace = true + +[payload.tcllibc] +source_root = "packages" +source = "win32-x86_64/tcl9/tcllibc" +target = "lib_tcl9" +replace = true +supersedes = ["tcllibc2.0"] diff --git a/src/vfs/punk9wintk905.vfs.toml b/src/vfs/punk9wintk905.vfs.toml new file mode 100644 index 00000000..d9647e28 --- /dev/null +++ b/src/vfs/punk9wintk905.vfs.toml @@ -0,0 +1,28 @@ +# Declarative payload for src/vfs/punk9wintk905.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# tcludp 1.0.13 (vendor tree; G-037 lineage). Ships tcl9udp1013.dll + udp1013t.dll with a +# pkgIndex.tcl that selects by generation - the same source folder serves tcl9 and tcl8 kits. +[payload.tcludp] +source = "vendorlib_tcl9/win32-x86_64/tcludp1.0.13" +target = "lib_tcl9" +supersedes = ["udp1.0.12", "tcludp1.0.12"] + +# G-139: suite-built tcllib/tcllibc from the punkbin lib tier (bin/packages input tier populated +# by 'tclsh src/make.tcl libfetch'; artifacts declared in src/runtime/libpackages.toml). +# replace=true because the old tcl9 hand-drops nested md5c + tcllibc INSIDE tcllib2.0 (same +# folder name as the new install - supersedes cannot touch it). +[payload.tcllib] +source_root = "packages" +source = "allplatforms/tcl9/tcllib2.0" +target = "lib_tcl9" +replace = true + +[payload.tcllibc] +source_root = "packages" +source = "win32-x86_64/tcl9/tcllibc" +target = "lib_tcl9" +replace = true +supersedes = ["tcllibc2.0"] diff --git a/src/vfs/punk9wintk90b2.vfs.toml b/src/vfs/punk9wintk90b2.vfs.toml new file mode 100644 index 00000000..55a6c056 --- /dev/null +++ b/src/vfs/punk9wintk90b2.vfs.toml @@ -0,0 +1,28 @@ +# Declarative payload for src/vfs/punk9wintk90b2.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# tcludp 1.0.13 (vendor tree; G-037 lineage). Ships tcl9udp1013.dll + udp1013t.dll with a +# pkgIndex.tcl that selects by generation - the same source folder serves tcl9 and tcl8 kits. +[payload.tcludp] +source = "vendorlib_tcl9/win32-x86_64/tcludp1.0.13" +target = "lib_tcl9" +supersedes = ["udp1.0.12", "tcludp1.0.12"] + +# G-139: suite-built tcllib/tcllibc from the punkbin lib tier (bin/packages input tier populated +# by 'tclsh src/make.tcl libfetch'; artifacts declared in src/runtime/libpackages.toml). +# replace=true because the old tcl9 hand-drops nested md5c + tcllibc INSIDE tcllib2.0 (same +# folder name as the new install - supersedes cannot touch it). +[payload.tcllib] +source_root = "packages" +source = "allplatforms/tcl9/tcllib2.0" +target = "lib_tcl9" +replace = true + +[payload.tcllibc] +source_root = "packages" +source = "win32-x86_64/tcl9/tcllibc" +target = "lib_tcl9" +replace = true +supersedes = ["tcllibc2.0"] diff --git a/src/vfs/punkdeclare.vfs.toml b/src/vfs/punkdeclare.vfs.toml new file mode 100644 index 00000000..778c05be --- /dev/null +++ b/src/vfs/punkdeclare.vfs.toml @@ -0,0 +1,20 @@ +# Declarative payload for src/vfs/punkdeclare.vfs (G-115) - materialized by 'tclsh src/make.tcl vfslibs' +# (and automatically as a phase of 'make.tcl bake'/'bakehouse'). Entry format + the drop-in-wins +# precedence rules: src/vfs/README.md. punkcheck-tracked (records in src/vfs/.punkcheck). +# Migrated 2026-07-31 from the per-package src/runtime/vendorlib_vfs.toml surface (G-037/G-139). + +# G-115 DEMONSTRATION: the entire binary payload of punkdeclare.vfs is DECLARED - the folder in +# VCS carries only the boot fauxlink (main.tcl -> _config/punk_main.tcl). Clean-tree +# reproduction: with lib_tcl9 absent (fresh clone, or deleted), 'tclsh src/make.tcl vfslibs' +# (or 'bake punkdeclare') rematerializes the payload from the declared sources; the kit's +# smokerequire (udp + tcllibc in mapvfs.toml) proves both binary packages resolve inside the +# built artifact. Undeclared files dropped into the folder are preserved across +# re-materialization (drop-in-wins precedence - src/vfs/README.md). +[payload.tcludp] +source = "vendorlib_tcl9/win32-x86_64/tcludp1.0.13" +target = "lib_tcl9" + +[payload.tcllibc] +source_root = "packages" +source = "win32-x86_64/tcl9/tcllibc" +target = "lib_tcl9" diff --git a/src/vfs/punkdeclare.vfs/main.tcl#..+_config+punk_main.tcl#@punk%3a%3aboot,merge_over#.fxlnk b/src/vfs/punkdeclare.vfs/main.tcl#..+_config+punk_main.tcl#@punk%3a%3aboot,merge_over#.fxlnk new file mode 100644 index 00000000..e69de29b