diff --git a/AGENTS.md b/AGENTS.md index e5bae1a3..3044307e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -211,7 +211,7 @@ The project version is fully independent of module versions. A module bump (even - `lib/` (root) — Build output target for `tclsh src/make.tcl libs` - `modules_tcl8/`, `modules_tcl9/` — Tcl version-specific build output targets - `lib_tcl8/`, `lib_tcl9/` — Tcl version-specific build output targets - - `src/project_layouts/` — Thin project layout payloads (structure, scripts, docs, manifests - no stored bootsupport/vfs module snapshots; G-087, achieved - see goals/archive/G-087-thin-project-layouts.md). The build's thin-layout sync step (`make.tcl modules`/`libs`/`packages`/`project`) keeps each layout's `src/make.tcl`, `src/build.tcl` and bootsupport `include_modules.config` copies current, and syncs the punk.project payload into the `punk::mix::templates` modpod source tree; `dev project.new` composes each layout's overlay chain (vendor then custom tier, `@` derived folders, `.anti` deletion markers - see the store README) via `layout_materialize`, injects bootsupport modules/libs from the generating shell at generation time, and seeds generated-project identity metadata (punkproject.toml, CHANGELOG.md, punkorigin.toml provenance baseline) programmatically. Agents must not directly edit files under `src/project_layouts/` unless the user explicitly names a path there or specifically requests layout changes; never hand-edit the synced copies (they are punkcheck-managed build outputs). + - `src/project_layouts/` — Thin project layout payloads (structure, scripts, docs, manifests - no stored bootsupport/vfs module snapshots; G-087, achieved - see goals/archive/G-087-thin-project-layouts.md). The build's thin-layout sync step (`make.tcl modules`/`libs`/`packages`/`bakehouse`) keeps each layout's `src/make.tcl`, `src/build.tcl` and bootsupport `include_modules.config` copies current, and syncs the punk.project payload into the `punk::mix::templates` modpod source tree; `dev project.new` composes each layout's overlay chain (vendor then custom tier, `@` derived folders, `.anti` deletion markers - see the store README) via `layout_materialize`, injects bootsupport modules/libs from the generating shell at generation time, and seeds generated-project identity metadata (punkproject.toml, CHANGELOG.md, punkorigin.toml provenance baseline) programmatically. Agents must not directly edit files under `src/project_layouts/` unless the user explicitly names a path there or specifically requests layout changes; never hand-edit the synced copies (they are punkcheck-managed build outputs). ## Repo-wide Notes diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 0ee975eb..61fff089 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -87,7 +87,7 @@ Two layers with a deliberate dependency direction (the class never depends on th ## Build, provenance and kits -- **`src/make.tcl` is the single build entry.** Key subcommands: `modules`/`libs`/`packages` (repo-root build outputs), `vfscommonupdate` (regenerate `_vfscommon.vfs`), `bake` (kit assembly into `bin/`; optional kit names or `@group` selectors confine the run to those kits - G-121/G-024; `bake_default=false` entries build only when selected), `bakelist` (report the configured kit matrix with runtime/vfs presence, deployed state, groups and scheme roles), `vfslibs` (materialize the per-.vfs payload declarations - G-115), `bin`, `bakehouse` (packages + bake for a clean checkout; `project`/`vfs` remain as deprecated aliases - G-112 rename, achieved), `bootsupport` (snapshot refresh), `vendorupdate`, `check`, `workflow`, `projectversion`. Dispatch and help dogfood `punk::args` (G-030, achieved) and degrade to plain scan/help when bootsupport `punk::args` is stale (`PUNKBOOT_PLAIN=1` forces the degraded mode). Source: `src/AGENTS.md`. +- **`src/make.tcl` is the single build entry.** Key subcommands: `modules`/`libs`/`packages` (repo-root build outputs), `vfscommonupdate` (regenerate `_vfscommon.vfs`), `bake` (kit assembly into `bin/`; optional kit names or `@group` selectors confine the run to those kits - G-121/G-024; `bake_default=false` entries build only when selected), `bakelist` (report the configured kit matrix with runtime/vfs presence, deployed state, groups and scheme roles), `vfslibs` (materialize the per-.vfs payload declarations - G-115), `bin`, `bakehouse` (packages + bake for a clean checkout - G-112 rename, achieved; the transitional `project`/`vfs` aliases were removed 2026-08-01), `bootsupport` (snapshot refresh), `vendorupdate`, `check`, `workflow`, `projectversion`. Dispatch and help dogfood `punk::args` (G-030, achieved) and degrade to plain scan/help when bootsupport `punk::args` is stale (`PUNKBOOT_PLAIN=1` forces the degraded mode). Source: `src/AGENTS.md`. - **Host vs target platform (G-122).** The kit surfaces separate what the driving tclsh IS from what a bake is FOR. Target-keyed: the `bin/runtime/` store a runtime is read from, `.exe` suffixing of runtimes and kit outputs, presence checks, and the pre-deploy process sweep's tooling (`tasklist`/`taskkill` vs `ps`/`kill`, skipped when the target's processes cannot exist on this host). Host-keyed: copy commands, path handling, filesystem case rules, prompts. The default target is the host canon except for cygwin-family hosts (msys2/cygwin-runtime tclsh - `tcl_platform(platform)` `unix` on windows), which target `win32-x86_64`; a `mapvfs.toml` entry may declare its own target and is then addressed in that platform's tier. `make.tcl check` prints the derivation. Zip-type kits assemble without zipfs in the driving tcl (raw-runtime split + `punk::zip::mkzip` + concatenation), and since G-124 they EXTRACT without it too: `punk::zip` reads a zip - plain, or attached to an executable under either offset convention - with stock Tcl only, so a bake needs neither zipfs nor tcllib to carry a runtime's `tcl_library` into the kit. Sources: `src/AGENTS.md`, `src/runtime/AGENTS.md`, `src/modules/punk/platform-999999.0a1.0.tm`, `src/modules/punk/zip-999999.0a1.0.tm`. - **punkcheck** records every install/delete as events in per-folder `.punkcheck` directories - the basis for skip/copy change detection, superseded-module pruning and provenance. Single OO record lifecycle (G-094, achieved) with atomic saves and advisory event-scoped locking for concurrent writers (G-095, achieved). - **Provenance gates.** Build/promotion commands warn on uncommitted `src/` changes (column-0 `PROVENANCE-WARNING:` token, `-dirty-abort` for strict mode); `vendorupdate` warns for dirty source-project checkouts. diff --git a/CHANGELOG.md b/CHANGELOG.md index af8109b7..23c7f08e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,13 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` "Project Versioning" section for the bump policy. +## [0.39.1] - 2026-08-01 + +- make.tcl deprecated aliases removed (G-112 transition window closed): `project` and + `vfs` no longer map to `bakehouse`/`bake` - they now produce an unknown-subcommand + usage error. All alias plumbing is gone (dispatch mapping, punk::args definitions, + the "deprecated aliases" help group, plain-help and `workflow` mentions). + ## [0.39.0] - 2026-07-31 - Target-keyed kit output locations (G-127): a kit whose mapvfs entry declares diff --git a/punkproject.toml b/punkproject.toml index 7d6b07d2..8279bdcf 100644 --- a/punkproject.toml +++ b/punkproject.toml @@ -1,6 +1,6 @@ [project] name = "punkshell" -version = "0.39.0" +version = "0.39.1" license = "BSD-2-Clause" url = "https://www.gitea1.intx.com.au/jn/punkshell" #packager: declared identity for published artifacts (declarative, not proof - diff --git a/src/AGENTS.md b/src/AGENTS.md index 0282968a..b40b7c5e 100644 --- a/src/AGENTS.md +++ b/src/AGENTS.md @@ -59,7 +59,7 @@ Recovery after a wrong path guess: ## Work Guidance - Run `tclsh src/make.tcl packages` once after cloning to populate generated module and library assets. -- Use `tclsh src/make.tcl bakehouse` for full builds from a clean checkout (packages + bake; refuses uncommitted src by default - `-dirty-abort 0` overrides). The deprecated `project` alias maps to it with a notice. +- Use `tclsh src/make.tcl bakehouse` for full builds from a clean checkout (packages + bake; refuses uncommitted src by default - `-dirty-abort 0` overrides). - Use `tclsh src/make.tcl modules` to build just the module packages. - Use `tclsh src/make.tcl libs` to build just the library packages. - Use `tclsh src/make.tcl packages` to build both modules and libraries. @@ -68,7 +68,7 @@ Recovery after a wrong path guess: - All build/promotion commands (`bakehouse`, `packages`, `modules`, `libs`, `bake`, `vfslibs`, `bin`, `bootsupport`, `vfscommonupdate`) warn similarly when this project's own `src/` has uncommitted fossil/git changes (dirt outside `src/` is ignored). Warn-only by default EXCEPT `bakehouse` (aborts by default; `-dirty-abort 0` overrides); pass `-dirty-abort` elsewhere to make the check aborting. For evaluating uncommitted source without a build, prefer ` src` / ` src shell`. The check is `punkboot::utils::vcs_dirty_warnings` (optional scope argument) loaded guardedly from bootsupport - if the snapshot is stale/missing the check degrades to a skip notice (but `-dirty-abort` then aborts rather than silently losing the requested strictness). - Provenance warnings (dirty-src gate + vendorupdate source-project check) print with a plain column-0 `PROVENANCE-WARNING:` token (grep for it in captured build output) and are recapped at the end of the run via make.tcl's wrapped `::exit`. Interactive terminal runs get a 3-second ctrl-c grace countdown before a dirty build proceeds; piped/agent runs skip it. `tclsh src/make.tcl check` reports the current src provenance status and what the build commands would do. - Use `tclsh src/make.tcl vfscommonupdate` to rebuild `_vfscommon.vfs`. The REPLACE confirmation prompts on an interactive terminal; unattended/agent runs must pass `-confirm 0` (with the default `-confirm 1` and a piped/closed stdin, make.tcl aborts fast with guidance instead of reading stdin - do not pipe `y`). -- `tclsh src/make.tcl workflow` prints an embedded ASCII data-flow overview of the build/release workflow (release-ready command sequence, source→outputs folder flow, kit assembly, keyed notes). The text lives in `::punkboot::workflow_text` in `src/make.tcl` - embedded rather than a data file so it travels with the make.tcl copies seeded into generated projects. **Update contract (agents):** whenever build data flow changes - a make.tcl subcommand added/removed/repurposed, a source or output folder added or rerouted, a propagation target added, a gate (staleness/provenance/confirm) or deploy behaviour changed - update the workflow text in the same change-set. Keep it plain ASCII, max line width 100, and preserve the RELEASE SEQUENCE / DIAGRAM / KEY-NOTES / MAINTENANCE structure; verify by running `tclsh src/make.tcl workflow` (and once under `PUNKBOOT_PLAIN=1`). Adding/changing a subcommand also means: SUMMARIES + HELPTEXTS + SUBOPTS (+ SUBGROUPS) in `::punkboot::argdoc`, `known_commands`, the plain `punkboot_gethelp` text, and a project-version bump (make.tcl interface is product surface per root AGENTS.md). Layout make.tcl copies pick the change up via `make.tcl bootsupport` - do not hand-sync them. +- `tclsh src/make.tcl workflow` prints an embedded ASCII data-flow overview of the build/release workflow (release-ready command sequence, source→outputs folder flow, kit assembly, keyed notes). The text lives in `::punkboot::workflow_text` in `src/make.tcl` - embedded rather than a data file so it travels with the make.tcl copies seeded into generated projects. **Update contract (agents):** whenever build data flow changes - a make.tcl subcommand added/removed/repurposed, a source or output folder added or rerouted, a propagation target added, a gate (staleness/provenance/confirm) or deploy behaviour changed - update the workflow text in the same change-set. Keep it plain ASCII, max line width 100, and preserve the RELEASE SEQUENCE / DIAGRAM / KEY-NOTES / MAINTENANCE structure; verify by running `tclsh src/make.tcl workflow` (and once under `PUNKBOOT_PLAIN=1`). Adding/changing a subcommand also means: SUMMARIES + HELPTEXTS + SUBOPTS (+ SUBGROUPS) in `::punkboot::argdoc`, `known_commands`, the plain `punkboot_gethelp` text, and a project-version bump (make.tcl interface is product surface per root AGENTS.md). Layout make.tcl copies pick the change up via the thin-layout sync step in any `make.tcl modules`/`libs`/`packages`/`bakehouse` run - do not hand-sync them. - make.tcl subcommands and flags are punk::args-declared (G-030): `tclsh src/make.tcl help ?subcommand?` or ` -help` shows tabled usage; invalid arguments produce punk::args usage errors. The dispatch degrades to plain scan/help when the bootsupport punk::args (or its rendering stack) is stale or unavailable; `PUNKBOOT_PLAIN=1` forces the degraded mode for troubleshooting. The `::punkboot::argdoc` definitions are constructed (string-built) and use the G-045 authoring mechanisms — block-form help values + `@normalize`, explicit `@form -synopsis` — see src/modules/AGENTS.md "punk::args definition authoring ergonomics" before editing them or authoring similar definitions. - Uncommitted `make.tcl`-generated outputs may be batched into one commit, regardless of how many `make.tcl` invocations produced them. This covers punkcheck-managed build outputs that are VCS-tracked: `src/bootsupport/`, `_vfscommon.vfs/modules` + `/lib`, declared per-kit `*.vfs/lib_tcl/` subfolders, the thin-layout sync copies (`src/project_layouts/vendor/punk//src/{make.tcl,build.tcl}` + bootsupport manifest copies), and the module-shipped layout payload under `src/modules/punk/mix/#modpod-templates-*/templates/project_layouts/` (G-087: synced from `src/project_layouts` - never hand-edit the modpod copy). (Root `modules/`, `lib/`, `modules_tcl/`, `lib_tcl/` are gitignored and never committed.) Each module's old→new version rename is atomic; a regeneration reflects one build-output refresh, not independent hand-edits. This is a carve-out from generic "split aggressively" commit guidance for punkcheck-managed build outputs only — it does not cover user-curated VFS content (`_config/`, per-kit `*.vfs/` outside declared `lib_tcl` subfolders, `_aside/`, `mkzipfix.vfs`, `_vfscommon.vfs/doc`), which commits separately per its own concerns. - **Boot-precondition gate (G-125).** A bake refuses any kit whose merged `.vfs` supplies no tcl library: the kit is listed under `FAILED KITS` with a reason naming the cause, and NOTHING is written - no `src/_build/`, no deploy, and the previously deployed `bin/` is left byte-identical. This is deliberately a failure rather than a warning: the deploy step deletes the old kit before copying the new one, so the pre-gate behaviour replaced a working shell with an artifact that died at startup with `application-specific initialization failed: Cannot find a usable init.tcl`. The check is structural and non-executing (so it covers cross-target kits too): `init.tcl` in `tcl_library/`, `lib/tcl./` or `tcl./` - the third for runtimes whose archive mounts at the executable's own path rather than `//zipfs:/app`, so `[info library]` is `/tcl8.6` (the androwish/undroidwish zipfs backport for 8.6 does this) - plus one companion file a real Tcl library carries beside it (`tm.tcl`, `package.tcl`, `auto.tcl`, `clock.tcl`, `history.tcl`, `word.tcl`) or an `encoding/` directory - without that second test the `lib/BWidget1.10.1/init.tcl` every punkshell kit carries would answer for a tcl library it cannot provide. It reads the MERGED tree rather than asking whether extraction ran, so a `.vfs` that legitimately supplies its own tcl library (`src/vfs/punk8_statictwapi.vfs`, `src/vfs/punk9test.vfs`) builds and deploys unchanged. **Remedy when a kit is refused:** the usual cause is that the runtime's own payload could not be extracted - read the recapped `BUILD-WARNING` naming what was tried, then either fix extraction (a driving tclsh with no zipfs needs punk::zip >= 0.2.0 in bootsupport - see the extraction bullet above), point the kit at a runtime that carries a payload, or have the kit's `.vfs` supply a tcl library itself. The predicate is `punkboot::utils::vfs_boot_library_report` called through a guarded require, exactly as the dirty-src provenance check is: a stale or missing bootsupport snapshot degrades the gate to a `NOTE` rather than failing every kit. `tclsh src/make.tcl check` reports whether the gate is `ACTIVE` or `UNAVAILABLE`. @@ -81,7 +81,7 @@ Recovery after a wrong path guess: - Zip-type kit assembly does not require zipfs in the driving tcl (G-122): with `tcl::zipfs::mkimg` present it is used as before, otherwise the image is assembled by concatenation (raw-runtime split + `punk::zip::mkzip` + append, the same `::punkboot::assemble_zipcat_image` helper the `zipcat` kit type uses). Both mount identically - tcl zipfs reads the archive with archive-start-relative offsets. - Neither is EXTRACTING the runtime's own attached zip (needed to carry its `tcl_library` into the kit): `punk::zip` >= 0.2.0 reads a zip with stock Tcl only - no zipfs, no vfs::zip, no tcllib (G-124, the former tcllib `zipfile::decode` dependency is gone). The zipfs-less bake path splits off the executable prefix with `punk::zip::extract_preamble` and then reads the MEMBERS from the original runtime with `punk::zip::unzip`, never from the split-off intermediate: a runtime whose zip offsets are file-relative (the historical `zipfs mkimg` convention - `tclsh90b4_piperepl.exe` in the store is one) splits into a .zip whose offsets still count from the removed prefix, which plain zip readers reject. Reading the whole file at the derived base offset makes the two offset conventions indistinguishable to callers; `punk::zip::archive_info ` reports which one a given artifact uses. Verified 2026-07-26 by baking a zip kit from msys2's `/usr/bin/tclsh8.6` (no zipfs, no vfs::zip, no tcllib) and booting the result with its `tcl_library` present. - What the RUNTIME must provide to be zip-kit-wrappable - which zipfs commands, mount conventions and startup hooks are supported - is documented in `bin/AGENTS.md` "Kit-wrappable runtime requirements (G-129)". Short form: `tcl::zipfs::mount` with pairwise no-arg output (`tcl::zipfs::root` NOT required), any attached-archive mount point (the boot derives it from the runtime's own mount table - the androwish/undroidwish 8.6 backport mounts at the executable's own path), a `main.tcl`/`app/main.tcl` startup hook, and a liftable library payload for the G-125 gate. The bake's runtime capability probe keys `has_zipfs` on `tcl::zipfs::mount` accordingly. -- Use `punk make.tcl bakehouse` or `punk902z make.tcl bakehouse` inside Punk shell when building binaries through Punk (the deprecated `project` alias still maps). Driving make.tcl from a built punk executable is supported for informational/update subcommands and for kit builds of *other* kits — the kit whose deployed executable is running the build is skipped with a warning (it cannot be replaced while running, and the pre-deploy process sweep must not kill the build itself; the sweep also excludes the build's own pid in all cases). Rebuild that kit from tclsh or a different kit. +- Use `punk make.tcl bakehouse` or `punk902z make.tcl bakehouse` inside Punk shell when building binaries through Punk. Driving make.tcl from a built punk executable is supported for informational/update subcommands and for kit builds of *other* kits — the kit whose deployed executable is running the build is skipped with a warning (it cannot be replaced while running, and the pre-deploy process sweep must not kill the build itself; the sweep also excludes the build's own pid in all cases). Rebuild that kit from tclsh or a different kit. - **Punk-exe-hosted make.tcl runs in a pre-loaded interp, not a virgin one** — the kit's script-mode boot has already loaded much of the punk stack (punk, punk::lib, punk::repl, punk::console, punk::du, flagfilter, struct::set ...) and set process state (app-punkscript forces `::tcl_interactive 0` for script semantics; libunknown/packagepreference are active). Consequences make.tcl must (and now does) handle explicitly: `package require` of an already-provided package is a no-op, so anything make.tcl breaks in the interp (the accelerator-reload block forgets+destroys sha1/md5/struct::* — it re-requires what was loaded), and anything computed at package-load time (punk::repl's `::tcl_interactive` probe — the shell branch recomputes it before `repl::start`), must be restored deliberately rather than relying on later loads to re-fire. Also note the copies that run in this mode are the *kit's* pre-loaded modules, not the bootsupport snapshots make.tcl's paths would otherwise prefer — silent provenance mixing when versions diverge. When adding interp-surgery or load-time-state assumptions to make.tcl, test under both `tclsh src/make.tcl ...` and ` src/make.tcl ...`. - Binary images are platform-specific; build on each target platform rather than expecting a cross-platform flag. - Remove `_build/` artifacts only when a clean/resync is needed, then rerun the relevant `make.tcl` command. Avoid partial cleans that break boot modules. diff --git a/src/README.md b/src/README.md index 8c87d9f4..8598545d 100644 --- a/src/README.md +++ b/src/README.md @@ -38,7 +38,7 @@ e.g tclsh src/make.tcl bake ;# assemble kit/zipkit executables from the promoted vfs payload + runtimes # (consumer path from a CLEAN checkout: 'tclsh src/make.tcl bakehouse' = packages + bake in one run; - # 'make.tcl workflow' prints the full data-flow overview. 'project'/'vfs' remain as deprecated aliases.) + # 'make.tcl workflow' prints the full data-flow overview.) ``` diff --git a/src/make.tcl b/src/make.tcl index 08d72dcf..69267f8e 100644 --- a/src/make.tcl +++ b/src/make.tcl @@ -187,7 +187,7 @@ namespace eval ::punkboot { variable foldername [file tail $scriptfolder] variable pkg_requirements [list]; variable pkg_missing [list];variable pkg_loaded [list] variable help_flags [list -help --help /? -h] - variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate libfetch bootsupport vfscommonupdate projectversion workflow buildsuite tool project vfs] + variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate libfetch bootsupport vfscommonupdate projectversion workflow buildsuite tool] } @@ -2603,9 +2603,6 @@ proc ::punkboot::punkboot_gethelp {args} { append h " reported and build/test exit nonzero with fetch guidance (bin/punk-getzig.cmd)" \n \n append h " $scriptname help ?subcommand?" \n append h " - show usage for $scriptname or one of its subcommands (equivalent: $scriptname -help)" \n \n - append h " Deprecated aliases (transition window):" \n - append h " $scriptname project -> runs bakehouse (with a one-line notice)" \n - append h " $scriptname vfs -> runs bake (with a one-line notice)" \n \n append h " Flags:" \n append h " -confirm 0|1" \n append h " - interactive y/n confirmation policy (default 1: prompt when stdin is a terminal, abort fast when it is not)." \n @@ -2649,7 +2646,6 @@ gates, one bake) recipe; -dirty-abort 0 overrides). DEVELOPER (changing payload): pass the gates explicitly - the commit is the publishing act, not a build step. That is the RELEASE SEQUENCE below. - Deprecated aliases (transition window): 'project' -> bakehouse, 'vfs' -> bake. RELEASE SEQUENCE (the developer order that satisfies the staleness + provenance gates) ---------------------------------------------------------------------------- @@ -3034,8 +3030,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO help "Show usage for make.tcl or one of its subcommands" buildsuite "List, describe or run the defined buildsuites under src/buildsuites (zig runtime factory)" tool "List, test or build the vendored tools under src/tools (zig) and install them to /bin" - project "DEPRECATED alias for bakehouse (one-line notice, then runs bakehouse)" - vfs "DEPRECATED alias for bake (one-line notice, then runs bake)" } #@cmd -help bodies for the constructed definitions below. Authored BLOCK-FORM #(first value line whitespace-only) so the @normalize directive in each @@ -3283,16 +3277,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO (bin/punk-getzig.cmd). Build caches (.zig-cache/zig-out) are deliberately left in place between runs for rebuild speed (both git- and fossil-ignored)." - project - " - DEPRECATED alias for 'bakehouse' (G-112 stage-true rename) - prints a - one-line notice and runs bakehouse (packages + bake, -dirty-abort - default ON). Retained for a transition window." - vfs - " - DEPRECATED alias for 'bake' (G-112 stage-true rename) - prints a - one-line notice and runs bake (kit assembly including the vfslibs - phase). Retained for a transition window." } #shared option fragments composed into the per-subcommand definitions below variable OPT_CONFIRM { @@ -3326,7 +3310,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "Terminate running processes matching the executable being built (if applicable) so the build can install over it." } - #bakehouse (and its deprecated alias project) defaults the dirty gate ON: + #bakehouse defaults the dirty gate ON: #the bakehouse bakes from the committed recipe (G-112 two-persona model). variable OPT_DIRTYABORT_ON { -dirty-abort -type boolean -default 1 -help\ @@ -3381,8 +3365,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO check {} projectversion {} workflow {} - project {OPT_FORCEKILL OPT_DIRTYABORT_ON OPT_CONFIRM} - vfs {OPT_FORCEKILL OPT_DIRTYABORT OPT_CONFIRM} } #synopsis fragments matching the shared option fragments (explicit @form -synopsis #so usage shows the real invocation 'make.tcl ...' rather than the @@ -3396,8 +3378,8 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO OPT_TRUSTSERVER "?-trust-server?" OPT_FORCE "?-force?" } - #per-subcommand positional values (default: none). bake (and its deprecated - #alias vfs) and bakelist take optional kit names (G-121 selective bake). + #per-subcommand positional values (default: none). bake and bakelist take + #optional kit names (G-121 selective bake). #Discoverability nicety: the configured kit names are offered as choices when #the mapping parses at definition time - informational only (-choicerestricted #0), so the handlers' own validation (which lists the configured names, and @@ -3428,7 +3410,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO kitname, all configured kits are processed except entries with bake_default=false. An unknown name errors before any build.}"\ ] - dict set SUBVALUES vfs [dict get $SUBVALUES bake] dict set SUBVALUES bakelist [list\ "@values -min 0 -max -1"\ "kitname -type string -optional 1 -multiple 1${_kitname_choicepart} -help\ @@ -3440,7 +3421,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO ] variable VALUES_SYNOPSES { bake "?kitname ...?" - vfs "?kitname ...?" bakelist "?kitname ...?" } foreach {sub optvars} $SUBOPTS { @@ -3518,7 +3498,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "vendored tools" {tool} "informational" {info check bakelist projectversion workflow help} "interactive" {shell} - "deprecated aliases" {project vfs} } punk::args::define\ "@id -id (script)::punkboot"\ @@ -3628,7 +3607,7 @@ if {$::punkboot::punkargs_ok} { } } if {[dict exists $argd values kitname]} { - #declared -multiple: a list of the supplied kit names (bake/vfs/bakelist - G-121) + #declared -multiple: a list of the supplied kit names (bake/bakelist - G-121) set ::punkboot::opt_kitnames [dict get $argd values kitname] } unset -nocomplain _opts _optname _var @@ -3688,10 +3667,10 @@ if {$::punkboot::punkargs_ok} { lappend commands_found $a } } - #G-121: bake/bakelist (and the deprecated alias vfs) take optional trailing kit - #names - collect them here so the single-command check below still holds in - #degraded mode (the handlers do their own name validation). - if {[lindex $commands_found 0] in {bake bakelist vfs}} { + #G-121: bake/bakelist take optional trailing kit names - collect them here so + #the single-command check below still holds in degraded mode (the handlers do + #their own name validation). + if {[lindex $commands_found 0] in {bake bakelist}} { set ::punkboot::opt_kitnames [lrange $commands_found 1 end] set commands_found [lrange $commands_found 0 0] } @@ -3744,23 +3723,7 @@ if {$do_help} { exit $help_exitcode } -#G-112 deprecated aliases - one-line mapping before any dispatch: -# project -> bakehouse (packages + bake, -dirty-abort default ON) -# vfs -> bake (kit assembly including the vfslibs phase) -#Kept for a transition window (until the 8.6 arc completes - see the G-112 -#dispositions). The alias definitions above mirror their targets' options so -#flags parse identically; only the command word maps here. if {[info exists ::punkboot::command]} { - switch -- $::punkboot::command { - project { - puts stderr "make.tcl: 'project' is DEPRECATED - use 'bakehouse' (packages + bake; refuses uncommitted src by default). Running bakehouse." - set ::punkboot::command bakehouse - } - vfs { - puts stderr "make.tcl: 'vfs' is DEPRECATED - use 'bake' (kit assembly incl. the vfslibs phase). Running bake." - set ::punkboot::command bake - } - } if {!$::punkboot::punkargs_ok && $::punkboot::command eq "bakehouse"} { #degraded fallback scan has flag-only -dirty-abort (no way to pass 0): #bakehouse's documented default is ON, so force it here. @@ -9024,7 +8987,7 @@ if {$had_kits} { puts stdout " module builds processed" puts stdout "" puts stdout " If kit/zipkit based executables required - create src/vfs/.vfs folders containing lib,modules,modules_tcl9 etc folders" - puts stdout " Also ensure appropriate runtime executables exist in bin/runtime/ along with the kit mapping src/runtime/mapvfs.toml" + puts stdout " Also ensure appropriate runtime executables exist in bin/runtime/ along with the kit mapping src/runtime/mapvfs.toml" } puts stdout "-done-" exit 0 diff --git a/src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/make.tcl b/src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/make.tcl index a55eda23..69267f8e 100644 --- a/src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/make.tcl +++ b/src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/make.tcl @@ -187,7 +187,7 @@ namespace eval ::punkboot { variable foldername [file tail $scriptfolder] variable pkg_requirements [list]; variable pkg_missing [list];variable pkg_loaded [list] variable help_flags [list -help --help /? -h] - variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate libfetch bootsupport vfscommonupdate projectversion workflow buildsuite tool project vfs] + variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate libfetch bootsupport vfscommonupdate projectversion workflow buildsuite tool] } @@ -627,11 +627,15 @@ proc ::punkboot::lib::bootsupport_prompt_yesno {prompt} { } # -- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- -# Parsed kit-mapping model (G-121). One reader for src/runtime/mapvfs.config - -# the 'bake' kit machinery, the 'bakelist' report and the bake/bakelist argdoc -# choices all consume the model these helpers return, never the file format -# directly, so a config-format change (G-024 toml direction) swaps in underneath -# by reimplementing mapvfs_parse alone. +# Parsed kit-mapping model (G-121). One reader family for the kit-output mapping +# config - the 'bake' kit machinery, the 'bakelist' report and the bake/bakelist +# argdoc choices all consume the model these helpers return, never the file +# format directly. G-024: the canonical format is src/runtime/mapvfs.toml +# (tomlish-parsed [kit.] / [group.] / [scheme.] tables - the +# file's own header comment is the user-facing spec); the pre-G-024 line format +# src/runtime/mapvfs.config remains readable as a DEPRECATED fallback so +# un-migrated trees (older generated projects) keep baking. mapvfs_locate picks +# the file (toml wins when both exist); mapvfs_parse dispatches on extension. # # mapvfs_parse: pure parse - no output, no exits. Every store address, presence # probe and artifact suffix it computes is keyed by the entry's TARGET platform @@ -639,39 +643,423 @@ proc ::punkboot::lib::bootsupport_prompt_yesno {prompt} { # (bin/runtime) and each entry addresses $rtbase/. Returns a dict: # exists 0|1 (mapfile present) # mapfile path as supplied +# format toml|legacy (which reader produced the model) # default_target target platform for entries that declare none -# runtime_vfs_map runtime -> raw vfs_specs as listed (appname/type/target may be "") -# vfs_runtime_map vfstail -> list of {runtime appname type target smokerequires} -# (defaults applied; only vfs folders that exist on disk - matching -# the historical inline parse the kit loop was built around; -# smokerequires is the G-133 smoke-require package list, may be "") +# runtime_vfs_map runtime -> list of vfs_specs. legacy: raw specs as listed +# (appname/type/target may be ""). toml: normalized 6-element +# specs {vfstail appname type target smokerequires attrs} +# vfs_runtime_map vfstail -> list of {runtime appname type target smokerequires +# ?attrs?} (defaults applied; only vfs folders that exist on +# disk - matching the historical inline parse the kit loop was +# built around; smokerequires is the G-133 smoke-require package +# list, may be ""). attrs (6th element, toml models only - read +# via mapvfs_spec_attrs) is a dict: entry (config table label +# e.g kit.punk91), group, bake_default, scheme, scheme_role. # runtime_target runtime -> resolved target platform (all runtimes, mapped order) +# groups group name -> {description bake_default 0|1} (toml only) # missing names of referenced-but-absent runtimes/vfs folders +# missing_detail structured missing items (toml only): list of dicts +# {kind runtime|vfs label name target } +# - the bake path's strict per-entry failure reporting (G-024) # warnings warning lines in encounter order, ready to print verbatim # (missing-item lines carry their historical WARNING: prefix) # badentries subset of warnings: malformed entry lines (these have no kit # record to carry them, so bakelist prints them; missing-item # warnings are instead surfaced as per-row state there) -# configerrors fatal config problems (duplicate runtime line; unusable or +# configerrors fatal config problems (malformed/unresolvable declarations, +# each naming its entry; duplicate runtime line; unusable or # self-contradicting target platform; caller decides whether to # abort - the bake path keeps its historical exit 3) -proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} { - set model [dict create\ + +# Locate the kit-mapping config for a src/runtime folder (G-024). +# Precedence: PUNK_MAPVFS_CONFIG env override (test/characterization seam - an +# alternate config exercised without editing the real one; relative paths +# resolve against the runtime folder), then mapvfs.toml, then the deprecated +# mapvfs.config. When neither exists the toml name is returned so no-mapping +# messages name the current format. +proc ::punkboot::lib::mapvfs_locate {rt_sourcefolder} { + if {[info exists ::env(PUNK_MAPVFS_CONFIG)] && $::env(PUNK_MAPVFS_CONFIG) ne ""} { + set override $::env(PUNK_MAPVFS_CONFIG) + if {[file pathtype $override] ne "absolute"} { + set override [file join $rt_sourcefolder $override] + } + return $override + } + if {[file exists $rt_sourcefolder/mapvfs.toml]} { + return $rt_sourcefolder/mapvfs.toml + } + if {[file exists $rt_sourcefolder/mapvfs.config]} { + return $rt_sourcefolder/mapvfs.config + } + return $rt_sourcefolder/mapvfs.toml +} + +#tomlish::to_dict values are type-tagged (e.g {type STRING value x}, {type ARRAY value {...}}); +#tables are plain nested dicts. Shared by the mapvfs toml reader, the per-.vfs +#payload declarations (G-115) and any other build-side toml consumption. +proc ::punkboot::lib::toml_untag {tagged} { + if {[dict get $tagged type] eq "ARRAY"} { + set plain [list] + foreach el [dict get $tagged value] { + lappend plain [::punkboot::lib::toml_untag $el] + } + return $plain + } + return [dict get $tagged value] +} + +# Read + parse a toml file via tomlish. Returns {ok } or {error }. +proc ::punkboot::lib::toml_file_to_dict {path} { + if {[catch {package require tomlish} tomlish_err]} { + return [list error "tomlish package unavailable ($tomlish_err)"] + } + if {[catch { + set fd [open $path r] + fconfigure $fd -encoding utf-8 + set tomldata [read $fd] + close $fd + } readerr]} { + catch {close $fd} + return [list error "cannot read $path ($readerr)"] + } + if {[catch {tomlish::to_dict [tomlish::from_toml $tomldata]} parsed]} { + return [list error "toml parse failed for $path\n$parsed"] + } + return [list ok $parsed] +} + +# attrs element of a parsed vfs_spec tuple (6th element, toml models). Legacy +# models carry no attrs - callers get an empty dict and defaults apply +# (group "", bake_default 1, no scheme role). +proc ::punkboot::lib::mapvfs_spec_attrs {vfs_spec} { + if {[llength $vfs_spec] >= 6} { + return [lindex $vfs_spec 5] + } + return [dict create] +} + +proc ::punkboot::lib::mapvfs_model_new {mapfile default_target format} { + return [dict create\ exists 0\ mapfile $mapfile\ + format $format\ default_target $default_target\ runtime_vfs_map [dict create]\ vfs_runtime_map [dict create]\ runtime_target [dict create]\ + groups [dict create]\ missing [list]\ + missing_detail [list]\ warnings [list]\ badentries [list]\ configerrors [list]\ ] +} + +proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} { + if {[string tolower [file extension $mapfile]] eq ".toml"} { + return [mapvfs_parse_toml $mapfile $rtbase $sourcefolder $default_target] + } + return [mapvfs_parse_config $mapfile $rtbase $sourcefolder $default_target] +} + +# G-024 canonical reader: mapvfs.toml. Strict per-entry validation - a malformed +# or unresolvable declaration lands in configerrors naming its table (the bake +# path aborts on configerrors; missing-on-disk runtime/vfs items land in +# warnings + missing_detail and the bake path decides fatality per entry). +proc ::punkboot::lib::mapvfs_parse_toml {mapfile rtbase sourcefolder default_target} { + set model [mapvfs_model_new $mapfile $default_target toml] + if {![file exists $mapfile]} { + return $model + } + dict set model exists 1 + set legacy_sibling [file join [file dirname $mapfile] mapvfs.config] + if {[file exists $legacy_sibling] && [file tail $mapfile] eq "mapvfs.toml"} { + dict lappend model warnings "NOTE: both mapvfs.toml and mapvfs.config present in [file dirname $mapfile] - mapvfs.toml wins and the deprecated mapvfs.config is ignored (remove it, or keep it only as reference)." + } + lassign [toml_file_to_dict $mapfile] tstatus tpayload + if {$tstatus ne "ok"} { + dict lappend model configerrors "CONFIG FILE ERROR. $mapfile: $tpayload" + return $model + } + set kit_types [list kit zip zipcat cookit cookfs] + set groups [dict create] + set configerrors [list] + set warnings [list] + set missing [list] + set missing_detail [list] + #top-level tables: kit / group / scheme only + dict for {topkey -} $tpayload { + if {$topkey ni {kit group scheme}} { + lappend configerrors "CONFIG FILE ERROR. $mapfile: unknown top-level table or key '$topkey' (expected \[kit.\], \[group.\], \[scheme.\])" + } + } + #-- groups + if {[dict exists $tpayload group]} { + dict for {gname gentry} [dict get $tpayload group] { + set label "group.$gname" + set gdesc "" + set gdefault 1 + dict for {k v} $gentry { + switch -- $k { + description {set gdesc [toml_untag $v]} + bake_default { + set gdefault [toml_untag $v] + if {![string is boolean -strict $gdefault]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: 'bake_default' must be boolean (got '$gdefault')" + set gdefault 1 + } + } + default { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown key '$k' (expected description, bake_default)" + } + } + } + dict set groups $gname [dict create description $gdesc bake_default [expr {bool($gdefault)}]] + } + } + #-- normalized candidate entries: {label rtname vfstail appname kittype declared_target smokes attrs} + set entries [list] + set appnames_seen [dict create] ;#appname -> label + set entry_reader {{model_varname label entry required_type} { + #shared [kit.*]/[scheme.*] field reader; returns a dict or empty on fatal field errors + upvar 1 configerrors configerrors kit_types kit_types groups groups mapfile mapfile + set fields [dict create runtime "" vfs "" type "" target "" smokerequire [list] group "" bake_default "" scheme "" prefix ""] + set badentry 0 + dict for {k v} $entry { + switch -- $k { + runtime - vfs - type - target - group - scheme - prefix { + dict set fields $k [::punkboot::lib::toml_untag $v] + } + smokerequire { + if {[dict get $v type] eq "ARRAY"} { + dict set fields smokerequire [::punkboot::lib::toml_untag $v] + } else { + dict set fields smokerequire [list [::punkboot::lib::toml_untag $v]] + } + } + bake_default { + set bd [::punkboot::lib::toml_untag $v] + if {![string is boolean -strict $bd]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: 'bake_default' must be boolean (got '$bd')" + set badentry 1 + } else { + dict set fields bake_default [expr {bool($bd)}] + } + } + default { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown key '$k'" + set badentry 1 + } + } + } + foreach reqkey {runtime vfs} { + if {[dict get $fields $reqkey] eq ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile is missing required key '$reqkey'" + set badentry 1 + } + } + set ktype [dict get $fields type] + if {$ktype eq ""} { + if {$required_type} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile is missing required key 'type' (scheme entries declare their kit type explicitly; one of: [join $kit_types {, }])" + set badentry 1 + } else { + dict set fields type kit + } + } elseif {$ktype ni $kit_types} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown kit type '$ktype' (expected one of: [join $kit_types {, }])" + set badentry 1 + } + set declared [dict get $fields target] + if {$declared ne ""} { + set problem [::punkboot::lib::platform_name_problem $declared] + if {$problem ne ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile declares target platform '$declared' - $problem." + set badentry 1 + } + } + #strip a superfluous windows .exe from the runtime name (legacy-format parity) + set rtname [dict get $fields runtime] + if {[string match -nocase *.exe $rtname]} { + dict set fields runtime [string range $rtname 0 end-4] + } + set g [dict get $fields group] + if {$g ne "" && ![dict exists $groups $g]} { + #a group referenced without its own [group.] table is implicitly plain + dict set groups $g [dict create description "" bake_default 1] + } + if {$badentry} { + return [dict create] + } + return $fields + }} + if {[dict exists $tpayload kit]} { + dict for {kname kentry} [dict get $tpayload kit] { + set label "kit.$kname" + set fields [apply $entry_reader model $label $kentry 0] + if {![dict size $fields]} { + continue + } + foreach badkey {scheme prefix} { + if {[dict get $fields $badkey] ne ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: key '$badkey' is only valid in \[scheme.\] entries" + } + } + set g [dict get $fields group] + set bd [dict get $fields bake_default] + if {$bd eq ""} { + set bd [expr {$g ne "" ? [dict get $groups $g bake_default] : 1}] + } + if {[dict exists $appnames_seen $kname]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: output name '$kname' already produced by [dict get $appnames_seen $kname]" + continue + } + dict set appnames_seen $kname $label + lappend entries [list $label [dict get $fields runtime] [dict get $fields vfs] $kname\ + [dict get $fields type] [dict get $fields target] [dict get $fields smokerequire]\ + [dict create entry $label group $g bake_default $bd scheme "" scheme_role ""]] + } + } + #-- schemes (G-023/G-024: generative entries expanded at parse time - no version enumeration) + set project_version "" + set project_version_err "" + if {[dict exists $tpayload scheme]} { + dict for {sname sentry} [dict get $tpayload scheme] { + set label "scheme.$sname" + set fields [apply $entry_reader model $label $sentry 1] + if {![dict size $fields]} { + continue + } + set skind [dict get $fields scheme] + if {$skind eq ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile is missing required key 'scheme' (known schemes: versioned)" + continue + } + if {$skind ne "versioned"} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown scheme '$skind' (known schemes: versioned)" + continue + } + #versioned scheme (G-023): output names derive from the punkproject.toml + #project version at parse time - punk-, punk-dev and the + #release-gated plain punk - so one declaration tracks the version with + #no per-version config edits. + if {$project_version eq "" && $project_version_err eq ""} { + set pptoml [file join [file dirname $sourcefolder] punkproject.toml] + lassign [toml_file_to_dict $pptoml] pstatus ppayload + if {$pstatus ne "ok"} { + set project_version_err $ppayload + } elseif {[catch {toml_untag [dict get $ppayload project version]} pver]} { + set project_version_err "no \[project\] version field readable in $pptoml ($pver)" + } elseif {$pver eq ""} { + set project_version_err "empty \[project\] version field in $pptoml" + } else { + set project_version $pver + } + } + if {$project_version eq ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: cannot expand 'versioned' scheme - $project_version_err" + continue + } + set prefix [dict get $fields prefix] + if {$prefix eq ""} { + set prefix $sname + } + set g [dict get $fields group] + set bd [dict get $fields bake_default] + if {$bd eq ""} { + set bd [expr {$g ne "" ? [dict get $groups $g bake_default] : 1}] + } + foreach {appname role} [list ${prefix}-$project_version versioned ${prefix}-dev dev $prefix release] { + if {[dict exists $appnames_seen $appname]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: expanded output name '$appname' already produced by [dict get $appnames_seen $appname]" + continue + } + dict set appnames_seen $appname $label + lappend entries [list "$label ($role)" [dict get $fields runtime] [dict get $fields vfs] $appname\ + [dict get $fields type] [dict get $fields target] [dict get $fields smokerequire]\ + [dict create entry $label group $g bake_default $bd scheme $sname scheme_role $role]] + } + } + } + #-- per-runtime target resolution (target is a property of the RUNTIME - entries + #may repeat the declaration but must not disagree; legacy-format parity) + set runtime_target [dict create] + set runtime_first_label [dict create] + foreach e $entries { + lassign $e label rtname - - - declared - - + if {![dict exists $runtime_first_label $rtname]} { + dict set runtime_first_label $rtname $label + } + if {$declared eq ""} { + continue + } + if {![dict exists $runtime_target $rtname] || [dict get $runtime_target $rtname] eq $default_target} { + dict set runtime_target $rtname $declared + } elseif {[dict get $runtime_target $rtname] ne $declared} { + lappend configerrors "CONFIG FILE ERROR. runtime: $rtname declares conflicting target platforms ('[dict get $runtime_target $rtname]' and '$declared') in $mapfile - a runtime has one target." + } + } + foreach rtname [dict keys $runtime_first_label] { + if {![dict exists $runtime_target $rtname]} { + dict set runtime_target $rtname $default_target + } + } + #-- runtime presence (one check per runtime, in first-reference order) + foreach rtname [dict keys $runtime_first_label] { + if {$rtname eq "-"} { + continue + } + set rt_target [dict get $runtime_target $rtname] + set rtdir [file join $rtbase [platform_store_tier $rt_target]] + set runtime_test $rtname[platform_exe_suffix $rt_target] + if {![file exists [file join $rtdir $runtime_test]]} { + lappend warnings "WARNING: Missing runtime file $rtdir/$runtime_test (entry [dict get $runtime_first_label $rtname] in $mapfile)" + lappend missing $rtname + lappend missing_detail [dict create kind runtime label [dict get $runtime_first_label $rtname] name $rtname target $rt_target] + } + } + #-- build the maps (entry order preserved; vfs existence filtering matches the + #historical inline parse the kit loop was built around) + set runtime_vfs_map [dict create] + set vfs_runtime_map [dict create] + foreach e $entries { + lassign $e label rtname vfstail appname kittype - smokes attrs + set rt_target [dict get $runtime_target $rtname] + set spec [list $vfstail $appname $kittype $rt_target $smokes $attrs] + dict lappend runtime_vfs_map $rtname $spec + if {![file isdirectory [file join $sourcefolder vfs $vfstail]]} { + lappend warnings "WARNING: Missing vfs folder [file join $sourcefolder vfs $vfstail] specified in $mapfile for entry $label" + if {$vfstail ni $missing} { + lappend missing $vfstail + } + lappend missing_detail [dict create kind vfs label $label name $vfstail target $rt_target] + } else { + dict lappend vfs_runtime_map $vfstail [list $rtname $appname $kittype $rt_target $smokes $attrs] + } + } + dict set model runtime_vfs_map $runtime_vfs_map + dict set model vfs_runtime_map $vfs_runtime_map + dict set model runtime_target $runtime_target + dict set model groups $groups + dict set model missing $missing + dict set model missing_detail $missing_detail + dict set model warnings [concat [dict get $model warnings] $warnings] + dict set model configerrors $configerrors + return $model +} + +# DEPRECATED legacy reader: the pre-G-024 mapvfs.config line format +# ( {vfsfolder ?kitname? ?kittype? ?targetplatform? ?smokerequires?} ...). +# Kept so un-migrated trees (older generated projects) keep baking; behaviour is +# unchanged from before the toml conversion apart from the deprecation NOTE. +proc ::punkboot::lib::mapvfs_parse_config {mapfile rtbase sourcefolder default_target} { + set model [mapvfs_model_new $mapfile $default_target legacy] if {![file exists $mapfile]} { return $model } dict set model exists 1 + dict lappend model warnings "NOTE: $mapfile uses the deprecated mapvfs.config line format - convert to mapvfs.toml (the file header of src/runtime/mapvfs.toml in the punkshell tree is the format spec; when both files exist the toml file wins)." set fdmap [open $mapfile r] fconfigure $fdmap -translation binary set mapdata [read $fdmap] @@ -765,7 +1153,7 @@ proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} dict set model vfs_runtime_map $vfs_runtime_map dict set model runtime_target $runtime_target dict set model missing $missing - dict set model warnings $warnings + dict set model warnings [concat [dict get $model warnings] $warnings] dict set model badentries $badentries dict set model configerrors $configerrors return $model @@ -778,8 +1166,10 @@ proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} # TARGET .exe suffix, duplicate appname -> _) so listed names # match what a bake produces. Entries whose vfs folder is missing on disk never # reach the kit loop - they are appended after the buildable set so 'bakelist' -# can surface them as broken config. Duplicate-name disambiguation assumes the -# config keeps appnames unique (as the kit loop itself effectively does). +# can surface them as broken config. Duplicate-name disambiguation is per TARGET +# (G-127): same-named kits for different targets coexist under their own output +# tiers; a same-target duplicate assumes the config keeps appnames unique (as +# the kit loop itself effectively does). # $rtbase is the store ROOT (bin/runtime): each record resolves its own tier # under it, so entries targeting different platforms report their own store # presence (G-122). Returns a list of record dicts: @@ -792,9 +1182,18 @@ proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} # runtime_dir store tier folder holding the runtime ("" for "-") # target target platform (e.g win32-x86_64) # store_tier bin/runtime tier folder name for the target +# out_tier kit OUTPUT tier relative to bin/ and src/_build/ - "" for the +# default (host) target's flat locations, "kits/" for any +# other target (G-127) # vfs vfs folder tail (e.g punk9wintk903.vfs) # vfs_present 1|0 # smokerequire G-133 smoke-require package list as configured ("" when undeclared) +# entry config table label (toml models, e.g kit.punk91 / "scheme.punk9 (dev)"; "" legacy) +# group G-024 named grouping ("" when ungrouped) +# bake_default 1|0 - 0 means excluded from full (no-name) bakes; still bakeable +# by name or @group selection +# scheme / scheme_role scheme-expanded outputs: source scheme name and role +# (versioned|dev|release); "" for explicit entries proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { set runtime_vfs_map [dict get $model runtime_vfs_map] set vfs_runtime_map [dict get $model vfs_runtime_map] @@ -808,7 +1207,7 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { } set records [list] set exe_names_seen [list] - set record_pair {{rtname vfstail appname target_kit_type smokerequires} { + set record_pair {{rtname vfstail appname target_kit_type smokerequires attrs} { upvar 1 records records exe_names_seen exe_names_seen rtbase rtbase sourcefolder sourcefolder\ runtime_target runtime_target default_target default_target if {$appname eq ""} { @@ -834,14 +1233,16 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { set runtime_file $rtname[::punkboot::lib::platform_exe_suffix $target] set targetkit ${appname}[::punkboot::lib::platform_exe_suffix $target] set kitname $appname - if {$targetkit in $exe_names_seen} { - #duplicate appname configured - kit loop disambiguates the same way + if {[list $target $targetkit] in $exe_names_seen} { + #duplicate appname configured for the same target - kit loop + #disambiguates the same way (G-127: different-target namesakes + #coexist under their own kits// output tiers instead) set targetkit ${appname}_$rtname set kitname ${appname}_$rtname } set runtime_present [file exists [file join $runtime_dir $runtime_file]] } - lappend exe_names_seen $targetkit + lappend exe_names_seen [list $target $targetkit] lappend records [dict create\ kitname $kitname\ targetkit $targetkit\ @@ -852,17 +1253,29 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { runtime_dir $runtime_dir\ target $target\ store_tier $store_tier\ + out_tier [expr {$target eq $default_target ? {} : "kits/$target"}]\ vfs $vfstail\ vfs_present [file isdirectory [file join $sourcefolder vfs $vfstail]]\ smokerequire $smokerequires\ + entry [expr {[dict exists $attrs entry] ? [dict get $attrs entry] : ""}]\ + group [expr {[dict exists $attrs group] ? [dict get $attrs group] : ""}]\ + bake_default [expr {[dict exists $attrs bake_default] ? [dict get $attrs bake_default] : 1}]\ + scheme [expr {[dict exists $attrs scheme] ? [dict get $attrs scheme] : ""}]\ + scheme_role [expr {[dict exists $attrs scheme_role] ? [dict get $attrs scheme_role] : ""}]\ ] }} foreach vfstail $vfs_tails { if {![dict exists $vfs_runtime_map $vfstail]} { continue ;#unmapped vfs folder - the kit loop derives no output names from it } + set rtnames_done [list] ;#several entries may share a runtime on one vfs (toml models) - + #enumerate each distinct runtime's spec set exactly once foreach rt_app [dict get $vfs_runtime_map $vfstail] { set rtname [lindex $rt_app 0] + if {$rtname in $rtnames_done} { + continue + } + lappend rtnames_done $rtname if {![dict exists $runtime_vfs_map $rtname]} { continue } @@ -871,58 +1284,85 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { if {$configured_vfs ne $vfstail} { continue } - apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires + apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires [::punkboot::lib::mapvfs_spec_attrs $vfs_app] } } } #entries whose vfs folder is missing (excluded from vfs_runtime_map by the parse) dict for {rtname vfs_specs} $runtime_vfs_map { foreach vfsconfig $vfs_specs { - if {[llength $vfsconfig] < 1 || [llength $vfsconfig] > 5} { + if {[llength $vfsconfig] < 1 || [llength $vfsconfig] > 6} { continue } lassign $vfsconfig vfstail appname target_kit_type - smokerequires if {[dict exists $vfs_runtime_map $vfstail]} { continue ;#already enumerated above } - apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires + apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires [::punkboot::lib::mapvfs_spec_attrs $vfsconfig] } } return $records } # Match user-supplied kit names against mapvfs_kit_outputs records. -# Accepts the base kit name (punk91) or the platform artifact name (punk91.exe, -# name.kit); matching is case-insensitive when the HOST filesystem is (a name -# the user types is a host-side convenience - a cygwin-family host is as +# Accepts the base kit name (punk91), the platform artifact name (punk91.exe, +# name.kit), or a G-024 group selector @ (selects every record in the +# group). A base name matching several records - one per target, coexisting under +# their own output tiers (G-127) - selects them all. +# Matching is case-insensitive when the HOST filesystem is (a name the +# user types is a host-side convenience - a cygwin-family host is as # case-insensitive as a native windows one, whatever it targets). Returns a dict: # selected matched records (kit-output order, deduplicated) # unknown supplied names that matched nothing proc ::punkboot::lib::mapvfs_match_outputs {kit_outputs names} { set on_windows [expr {[info exists ::punkboot::host_windows] ? $::punkboot::host_windows : ("windows" eq $::tcl_platform(platform))}] + set name_eq {{on_windows a b} { + expr {$on_windows ? [string equal -nocase $a $b] : [string equal $a $b]} + }} set selected_idx [list] set unknown [list] foreach name $names { + set found 0 + if {[string match @?* $name]} { + #group selector: every record carrying the group (G-024) + set gname [string range $name 1 end] + set i 0 + foreach rec $kit_outputs { + if {[dict get $rec group] ne "" && [apply $name_eq $on_windows [dict get $rec group] $gname]} { + set found 1 + if {$i ni $selected_idx} { + lappend selected_idx $i + } + } + incr i + } + if {!$found} { + lappend unknown $name + } + continue + } set base $name if {[string match -nocase *.exe $base] || [string match -nocase *.kit $base]} { set base [string range $base 0 end-4] } - set found 0 set i 0 foreach rec $kit_outputs { + set rec_match 0 foreach candidate [list [dict get $rec kitname] [dict get $rec targetkit]] { if {$candidate eq ""} {continue} - if {($on_windows && [string equal -nocase $candidate $base]) || (!$on_windows && $candidate eq $base)\ - || ($on_windows && [string equal -nocase $candidate $name]) || (!$on_windows && $candidate eq $name)} { - set found 1 + if {[apply $name_eq $on_windows $candidate $base] || [apply $name_eq $on_windows $candidate $name]} { + set rec_match 1 break } } - if {$found} { + if {$rec_match} { + #G-127: one kit NAME may legitimately match several records - one per + #target, coexisting under their own output tiers - so a name selects + #them ALL (like the @group branch), not just the first hit + set found 1 if {$i ni $selected_idx} { lappend selected_idx $i } - break } incr i } @@ -2110,7 +2550,7 @@ proc ::punkboot::punkboot_gethelp {args} { append h " - includes the vfslibs phase; does not rebuild modules/libs and does not run the promotion gates" \n append h " - with kitname arguments, bakes and deploys only the named configured kits (unknown names error before any build)" \n \n append h " $scriptname bakelist ?kitname ...?" \n - append h " - list the kit outputs configured in src/runtime/mapvfs.config: name, kit type, runtime (with presence)," \n + append h " - list the kit outputs configured in src/runtime/mapvfs.toml: name, kit type, runtime (with presence)," \n append h " vfs folder and deployed state (bin copy absent/current/stale/nobuild vs the src/_build product)" \n append h " - kitname arguments filter to the named entries (per-kit detail)" \n \n append h " $scriptname modules" \n @@ -2163,9 +2603,6 @@ proc ::punkboot::punkboot_gethelp {args} { append h " reported and build/test exit nonzero with fetch guidance (bin/punk-getzig.cmd)" \n \n append h " $scriptname help ?subcommand?" \n append h " - show usage for $scriptname or one of its subcommands (equivalent: $scriptname -help)" \n \n - append h " Deprecated aliases (transition window):" \n - append h " $scriptname project -> runs bakehouse (with a one-line notice)" \n - append h " $scriptname vfs -> runs bake (with a one-line notice)" \n \n append h " Flags:" \n append h " -confirm 0|1" \n append h " - interactive y/n confirmation policy (default 1: prompt when stdin is a terminal, abort fast when it is not)." \n @@ -2209,7 +2646,6 @@ gates, one bake) recipe; -dirty-abort 0 overrides). DEVELOPER (changing payload): pass the gates explicitly - the commit is the publishing act, not a build step. That is the RELEASE SEQUENCE below. - Deprecated aliases (transition window): 'project' -> bakehouse, 'vfs' -> bake. RELEASE SEQUENCE (the developer order that satisfies the staleness + provenance gates) ---------------------------------------------------------------------------- @@ -2278,19 +2714,32 @@ DIAGRAM 1b - THIN-LAYOUT SYNC (part of modules/libs/packages/bakehouse runs) DIAGRAM 2 - KIT ASSEMBLY DETAIL (the 'make.tcl bake' stage; incl. vfslibs phase) ---------------------------------------------------------------------------- - src/runtime/mapvfs.config which vfs folder pairs with which runtime -> kit name, - and optionally the TARGET PLATFORM of that runtime [K9] - 'make.tcl bakelist ?kitname ...?' reports the configured - kit outputs (runtime/vfs presence + deployed state); - 'make.tcl bake ?kitname ...?' bakes only the named kits - (bare bake = all; unknown names error before any build) + src/runtime/mapvfs.toml which vfs folder pairs with which runtime -> kit name + ([kit.] tables; tomlish-parsed - G-024), incl. the + runtime's TARGET PLATFORM [K9], smoke-require packages, + named groups and generative version-named schemes + (deprecated mapvfs.config line format still readable) + 'make.tcl bakelist ?name|@group ...?' reports the + configured kit outputs (presence + deployed state); + 'make.tcl bake ?name|@group ...?' bakes only the named + kits (bare bake = all entries except bake_default=false + ones; unknown names error before any build) bin/runtime// bare Tcl runtime (tclkit / tclsfe / suite build), in the store tier of the TARGET platform it builds kits for [K9] src/vfs/_vfscommon.vfs/ common payload (built modules + libs, main boot support) src/vfs/.vfs/ per-kit overlay (main.tcl, kit-only modules/config) - - _vfscommon.vfs .vfs - | | + src/vfs/.vfs.toml optional per-.vfs payload declaration (G-115): [payload.*] + entries materialized INTO the .vfs folder by the vfslibs + phase (vendor trees / bin/packages tier), punkcheck- + tracked with drop-in-wins precedence - undeclared or + hand-modified files are preserved, never overwritten. + A source containing %platform% materializes once per + CONSUMING KIT TARGET (set derived from mapvfs) into + _targets// staging; each kit's merged image + then gets only its own target's subtree (G-127) + + _vfscommon.vfs .vfs <-- payload declarations (if any) already + | | materialized by the vfslibs phase +-----------+----------+ | copy common, then merge overlay over it v @@ -2349,7 +2798,8 @@ LIBRARY-ARTIFACT FETCH + CONSUMPTION (G-139) | consumption (two lanes, both punkcheck-tracked): | PACKAGES_tcl phase of modules/libs/packages/bakehouse | -> lib_tcl/allplatforms + lib_tcl/; - | vfslibs entries with source_root = "packages" + | per-.vfs payload declarations (src/vfs/.vfs.toml, + | G-115) with source_root = "packages" | -> declared kit vfs lib_tcl trees (see DIAGRAM 2) v lib_tcl/... + src/vfs/.vfs/lib_tcl/... (embedded records ride along @@ -2401,7 +2851,9 @@ KEY / NOTES [K8] What is VCS-tracked where (checkin targets after a build): tracked: src/** (sources, src/bootsupport, src/project_layouts copies, src/vfs/_vfscommon.vfs), punkproject.toml, CHANGELOG.md - untracked: modules*/ lib*/ at projectroot, src/_build/, bin/.exe + untracked: modules*/ lib*/ at projectroot, src/_build/, bin/.exe, + bin/kits// (cross-target kit outputs - G-127), + src/vfs/*.vfs/_targets/ (per-platform payload staging) (bin/ scripts/tools are tracked; kit executables are not) [K9] HOST vs TARGET platform (G-122). What a bake EMITS is keyed by the target @@ -2410,11 +2862,18 @@ KEY / NOTES whether the pre-deploy sweep uses tasklist/taskkill or ps/kill. The default target is this host's own platform - except a cygwin-family host (an msys2/cygwin-runtime tclsh, which reports platform 'unix' on windows), which - targets win32. A mapvfs.config entry may declare its own target as a 4th - element and is then read from that platform's tier, named with its executable - convention, and (being unrunnable here) skipped by the process sweep. + targets win32. A mapvfs.toml entry may declare its own target (target key; + legacy config: 4th element) and is then read from that platform's tier, named + with its executable convention, and (being unrunnable here) skipped by the + process sweep. 'make.tcl check' prints the derivation; 'make.tcl bakelist' shows per-kit targets, with a target= note on rows that are not on the default target. + OUTPUT locations are target-keyed too (G-127): default-target kits keep the + flat src/_build/ + bin/ locations, any other target's kit builds + and deploys under the kits// tier of both (bin/kits/ = outputs, + beside bin/runtime/ = inputs), so same-named kits for different targets + coexist instead of overwriting each other; bakelist marks such rows with + out=kits// and its detail block prints the tiered paths. [K10] BOOT-PRECONDITION gate (G-125). A kit whose merged vfs has no tcl library cannot initialise at all ('Cannot find a usable init.tcl'), so the bake @@ -2440,8 +2899,8 @@ KEY / NOTES cross-target kits are covered) against the kit's target platform; wrong-arch libraries outside platform-discriminated subdirs (win32-ix86/, win-x64/ etc - multi-arch payloads are legitimate there) earn recapped BUILD-WARNINGs. - (b) Smoke-require probe: a kit declaring packages (mapvfs.config 5th entry - element) has each one plain-'package require'd INSIDE the freshly built + (b) Smoke-require probe: a kit declaring packages (mapvfs.toml smokerequire + key; legacy config: 5th element) has each one plain-'package require'd INSIDE the freshly built artifact via its tclsh subcommand - the only check that sees resolution-order defects (a wrong-arch higher-versioned package shadowing a working copy); cross-target kits skip with a stated reason, undeclared kits run nothing new. @@ -2556,8 +3015,8 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO modules "Build .tm modules from src/modules, src/vendormodules etc into /modules etc" libs "Build pkgIndex.tcl libraries from src/lib, src/vendorlib etc into /lib etc" bake "Assemble kit/zipkit executables from promoted payload (src/vfs) and runtimes into /bin" - bakelist "List the kit outputs configured in src/runtime/mapvfs.config: type, runtime/vfs presence, deployed state" - vfslibs "Propagate declared platform-library packages (vendored or bin/packages tier) into kit vfs lib_tcl trees" + bakelist "List the kit outputs configured in src/runtime/mapvfs.toml: type, runtime/vfs presence, deployed state, groups" + vfslibs "Materialize per-.vfs payload declarations (src/vfs/.vfs.toml - G-115) into kit vfs folders; drop-ins preserved" bin "Install executables from src/bin into /bin, then build kits as for bake" vendorupdate "Update src/vendormodules based on src/vendormodules/include_modules.config" libfetch "Fetch declared punkbin lib-tier library artifacts into bin/packages/ (sha1-verified) and materialize the package trees" @@ -2571,8 +3030,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO help "Show usage for make.tcl or one of its subcommands" buildsuite "List, describe or run the defined buildsuites under src/buildsuites (zig runtime factory)" tool "List, test or build the vendored tools under src/tools (zig) and install them to /bin" - project "DEPRECATED alias for bakehouse (one-line notice, then runs bakehouse)" - vfs "DEPRECATED alias for bake (one-line notice, then runs bake)" } #@cmd -help bodies for the constructed definitions below. Authored BLOCK-FORM #(first value line whitespace-only) so the @normalize directive in each @@ -2620,25 +3077,28 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO build each configured kit from the promoted payload (src/vfs/_vfscommon.vfs + custom vfs folders) and its runtime, and install to /bin. - With kitname arguments, bake and deploy ONLY the named configured - kits (see 'make.tcl bakelist' for the configured names) - other - kits' build products, bin copies and punkcheck records are left - untouched. An unknown name errors before any build, listing the - configured names. Bare 'bake' processes all configured kits. - Includes the vfslibs phase (declared vendored platform-library - propagation into kit vfs lib_tcl trees) so a bake cannot ship - stale binary libs; a selective bake narrows that phase to the - named kits' vfs folders. + With kitname (or @groupname - G-024 groups) arguments, bake and + deploy ONLY the named configured kits (see 'make.tcl bakelist' + for the configured names and groups) - other kits' build + products, bin copies and punkcheck records are left untouched. + An unknown name errors before any build, listing the configured + names. Bare 'bake' processes all configured kits except + bake_default=false entries (those bake only by name/@group). + Includes the vfslibs phase (per-.vfs payload declarations + src/vfs/.vfs.toml materialized into the kit vfs folders - + G-115) so a bake cannot ship stale binary libs; a selective + bake narrows that phase to the named kits' vfs folders. Does not rebuild modules/libs and does not run the promotion gates - built modules reach the kit payload only via 'make.tcl vfscommonupdate' (committed for provenance). Consumer full path from a clean checkout: 'make.tcl bakehouse'." bakelist " - List the kit outputs configured in src/runtime/mapvfs.config (the - same parsed mapping the bake subcommand consumes): kit name, kit - type (kit|zip|zipcat...), runtime with presence in the runtime - store of that kit's TARGET platform + List the kit outputs configured in src/runtime/mapvfs.toml (the + same parsed mapping the bake subcommand consumes; the deprecated + mapvfs.config line format is honoured when no toml file exists): + kit name, kit type (kit|zip|zipcat...), runtime with presence in + the runtime store of that kit's TARGET platform (/bin/runtime/), vfs folder, and the deployed state of /bin/ vs the src/_build build product: @@ -2651,20 +3111,31 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO materialized from an older revision than a -r artifact beside it) and marks cross-target entries (target= on any kit whose mapvfs entry declares a target other than this host's - default - 'make.tcl check' shows that default). - kitname arguments filter the report to the named entries (per-kit - detail, with resolved store tier, target and paths). Reporting - only - never builds." + default - 'make.tcl check' shows that default), G-024 groups + (group=), entries excluded from full bakes (default=no) + and scheme-expanded outputs (scheme=versioned|dev|release). + Non-default-target kits build and deploy under the + kits// tier of src/_build and bin (G-127) - their rows + carry out=kits// and the deployed state compares the + tiered paths; a kit NAME shared by entries for several targets + selects (and lists) every one of them. + kitname (or @groupname) arguments filter the report to the named + entries (per-kit detail, with resolved store tier, target, + config entry and paths). Reporting only - never builds." vfslibs " - Propagate declared platform-library packages into kit vfs - lib_tcl trees - from vendored src/ folders (source_root - 'src', the default) or from the bin/packages punkbin lib-tier - input (source_root 'packages', populated by 'make.tcl - libfetch' - G-139). - Declarations (per-package per-kit, with superseded-version - removal and clean-slate same-name 'replace') live in - src/runtime/vendorlib_vfs.toml. + Materialize the per-.vfs payload declarations (G-115): each + src/vfs/.vfs.toml declares package folders to install + INTO src/vfs/.vfs - from vendored src/ folders + (source_root 'src', the default) or from the bin/packages + punkbin lib-tier input (source_root 'packages', populated by + 'make.tcl libfetch' - G-139) - with explicit superseded-version + removal and clean-slate same-name 'replace'. + punkcheck-tracked with drop-in-wins precedence: files the + mechanism did not install (or that changed since it installed + them) are preserved and reported, never overwritten; a .vfs + with no declaration file is untouched. Format + precedence: + src/vfs/README.md. Also runs automatically as a phase of 'make.tcl bake' (and therefore of 'make.tcl bakehouse'); this standalone narrowing serves quick iteration on the declarations." @@ -2806,16 +3277,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO (bin/punk-getzig.cmd). Build caches (.zig-cache/zig-out) are deliberately left in place between runs for rebuild speed (both git- and fossil-ignored)." - project - " - DEPRECATED alias for 'bakehouse' (G-112 stage-true rename) - prints a - one-line notice and runs bakehouse (packages + bake, -dirty-abort - default ON). Retained for a transition window." - vfs - " - DEPRECATED alias for 'bake' (G-112 stage-true rename) - prints a - one-line notice and runs bake (kit assembly including the vfslibs - phase). Retained for a transition window." } #shared option fragments composed into the per-subcommand definitions below variable OPT_CONFIRM { @@ -2849,7 +3310,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "Terminate running processes matching the executable being built (if applicable) so the build can install over it." } - #bakehouse (and its deprecated alias project) defaults the dirty gate ON: + #bakehouse defaults the dirty gate ON: #the bakehouse bakes from the committed recipe (G-112 two-persona model). variable OPT_DIRTYABORT_ON { -dirty-abort -type boolean -default 1 -help\ @@ -2904,8 +3365,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO check {} projectversion {} workflow {} - project {OPT_FORCEKILL OPT_DIRTYABORT_ON OPT_CONFIRM} - vfs {OPT_FORCEKILL OPT_DIRTYABORT OPT_CONFIRM} } #synopsis fragments matching the shared option fragments (explicit @form -synopsis #so usage shows the real invocation 'make.tcl ...' rather than the @@ -2919,18 +3378,22 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO OPT_TRUSTSERVER "?-trust-server?" OPT_FORCE "?-force?" } - #per-subcommand positional values (default: none). bake (and its deprecated - #alias vfs) and bakelist take optional kit names (G-121 selective bake). + #per-subcommand positional values (default: none). bake and bakelist take + #optional kit names (G-121 selective bake). #Discoverability nicety: the configured kit names are offered as choices when #the mapping parses at definition time - informational only (-choicerestricted #0), so the handlers' own validation (which lists the configured names, and #still works when the mapping cannot be parsed here) stays authoritative. variable KITNAME_CHOICES [list] catch { - set _mapmodel [punkboot::lib::mapvfs_parse [file join $::punkboot::scriptfolder runtime mapvfs.config] "" $::punkboot::scriptfolder $::punkboot::target_platform] + set _mapmodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate [file join $::punkboot::scriptfolder runtime]] "" $::punkboot::scriptfolder $::punkboot::target_platform] foreach _rec [punkboot::lib::mapvfs_kit_outputs $_mapmodel "" $::punkboot::scriptfolder] { lappend KITNAME_CHOICES [dict get $_rec kitname] } + #G-024 group selectors join the informational choices (@) + foreach _g [dict keys [dict get $_mapmodel groups]] { + lappend KITNAME_CHOICES @$_g + } } set _kitname_choicepart "" if {[llength $KITNAME_CHOICES]} { @@ -2941,22 +3404,23 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "@values -min 0 -max -1"\ "kitname -type string -optional 1 -multiple 1${_kitname_choicepart} -help\ { - Configured kit output(s) to bake and deploy - see 'make.tcl bakelist' - for the configured names. With no kitname, all configured kits are - processed. An unknown name errors before any build.}"\ + Configured kit output(s) to bake and deploy - a kit name, or + @ for every kit in a configured group (G-024) - see + 'make.tcl bakelist' for the configured names and groups. With no + kitname, all configured kits are processed except entries with + bake_default=false. An unknown name errors before any build.}"\ ] - dict set SUBVALUES vfs [dict get $SUBVALUES bake] dict set SUBVALUES bakelist [list\ "@values -min 0 -max -1"\ "kitname -type string -optional 1 -multiple 1${_kitname_choicepart} -help\ { - Configured kit output(s) to report on (filters the listing - doubles + Configured kit output(s) to report on - a kit name, or @ + for every kit in a configured group (filters the listing - doubles as per-kit detail). With no kitname, all configured entries are listed.}"\ ] variable VALUES_SYNOPSES { bake "?kitname ...?" - vfs "?kitname ...?" bakelist "?kitname ...?" } foreach {sub optvars} $SUBOPTS { @@ -3034,7 +3498,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "vendored tools" {tool} "informational" {info check bakelist projectversion workflow help} "interactive" {shell} - "deprecated aliases" {project vfs} } punk::args::define\ "@id -id (script)::punkboot"\ @@ -3144,7 +3607,7 @@ if {$::punkboot::punkargs_ok} { } } if {[dict exists $argd values kitname]} { - #declared -multiple: a list of the supplied kit names (bake/vfs/bakelist - G-121) + #declared -multiple: a list of the supplied kit names (bake/bakelist - G-121) set ::punkboot::opt_kitnames [dict get $argd values kitname] } unset -nocomplain _opts _optname _var @@ -3204,10 +3667,10 @@ if {$::punkboot::punkargs_ok} { lappend commands_found $a } } - #G-121: bake/bakelist (and the deprecated alias vfs) take optional trailing kit - #names - collect them here so the single-command check below still holds in - #degraded mode (the handlers do their own name validation). - if {[lindex $commands_found 0] in {bake bakelist vfs}} { + #G-121: bake/bakelist take optional trailing kit names - collect them here so + #the single-command check below still holds in degraded mode (the handlers do + #their own name validation). + if {[lindex $commands_found 0] in {bake bakelist}} { set ::punkboot::opt_kitnames [lrange $commands_found 1 end] set commands_found [lrange $commands_found 0 0] } @@ -3260,23 +3723,7 @@ if {$do_help} { exit $help_exitcode } -#G-112 deprecated aliases - one-line mapping before any dispatch: -# project -> bakehouse (packages + bake, -dirty-abort default ON) -# vfs -> bake (kit assembly including the vfslibs phase) -#Kept for a transition window (until the 8.6 arc completes - see the G-112 -#dispositions). The alias definitions above mirror their targets' options so -#flags parse identically; only the command word maps here. if {[info exists ::punkboot::command]} { - switch -- $::punkboot::command { - project { - puts stderr "make.tcl: 'project' is DEPRECATED - use 'bakehouse' (packages + bake; refuses uncommitted src by default). Running bakehouse." - set ::punkboot::command bakehouse - } - vfs { - puts stderr "make.tcl: 'vfs' is DEPRECATED - use 'bake' (kit assembly incl. the vfslibs phase). Running bake." - set ::punkboot::command bake - } - } if {!$::punkboot::punkargs_ok && $::punkboot::command eq "bakehouse"} { #degraded fallback scan has flag-only -dirty-abort (no way to pass 0): #bakehouse's documented default is ON, so force it here. @@ -3386,7 +3833,7 @@ set ::punkboot::bake_selected_vfs [list] ;#vfs folder tails the selected set ::punkboot::bake_selected_runtimes [list] ;#unsuffixed runtime names the selected kits wrap if {$::punkboot::command eq "bake" && [llength $::punkboot::opt_kitnames]} { set _rtbase $binfolder/runtime - set _mapmodel [punkboot::lib::mapvfs_parse $sourcefolder/runtime/mapvfs.config $_rtbase $sourcefolder $::punkboot::target_platform] + set _mapmodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate $sourcefolder/runtime] $_rtbase $sourcefolder $::punkboot::target_platform] if {[llength [dict get $_mapmodel configerrors]]} { foreach _errline [dict get $_mapmodel configerrors] { puts stderr $_errline @@ -3399,8 +3846,11 @@ if {$::punkboot::command eq "bake" && [llength $::punkboot::opt_kitnames]} { puts stderr "make.tcl bake: unknown kit name(s): [join [dict get $_matchinfo unknown] {, }] - nothing built" if {[llength $_kit_outputs]} { puts stderr " configured kit outputs: [join [lmap _r $_kit_outputs {dict get $_r kitname}] {, }]" + if {[dict size [dict get $_mapmodel groups]]} { + puts stderr " configured groups (select with @): [join [dict keys [dict get $_mapmodel groups]] {, }]" + } } else { - puts stderr " no kit outputs are configured (src/runtime/mapvfs.config)" + puts stderr " no kit outputs are configured (src/runtime/mapvfs.toml)" } foreach _n [dict get $_matchinfo unknown] { if {[string match -* $_n]} { @@ -3740,7 +4190,7 @@ if {$::punkboot::command eq "check"} { if {$_chk_suffix eq ""} {set _chk_suffix "(none)"} puts stdout "platform (G-122): host=$::punkboot::host_platform target=$::punkboot::target_platform store=bin/runtime/[punkboot::lib::platform_store_tier $::punkboot::target_platform] exe-suffix=$_chk_suffix process-tooling=[punkboot::lib::platform_process_family $::punkboot::target_platform]" puts stdout " (host = this tclsh's punk::platform canon, from tcl_platform(os)='$::tcl_platform(os)' platform='$::tcl_platform(platform)'; target = default kit target - a cygwin-family host (msys2/cygwin runtime on windows) bakes win32 kits)" - puts stdout " (mapvfs.config entries may declare their own target platform as a 4th element - 'make.tcl bakelist' shows each kit's target, store tier and artifact name)" + puts stdout " (mapvfs.toml entries may declare their own target platform (target key) - 'make.tcl bakelist' shows each kit's target, store tier and artifact name)" unset -nocomplain _chk_suffix puts stdout $sep puts stdout "module/library checks - paths from bootsupport only" @@ -3950,8 +4400,8 @@ if {$::punkboot::command eq "check"} { puts stdout " bake will proceed with a NOTE and cannot flag a file-relative kit payload - run 'make.tcl modules' then 'make.tcl bootsupport'." } set _smoke_declared [list] - if {[file exists $sourcefolder/runtime/mapvfs.config]} { - set _smokemodel [punkboot::lib::mapvfs_parse $sourcefolder/runtime/mapvfs.config [file dirname $sourcefolder]/bin/runtime $sourcefolder $::punkboot::target_platform] + if {[file exists [punkboot::lib::mapvfs_locate $sourcefolder/runtime]]} { + set _smokemodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate $sourcefolder/runtime] [file dirname $sourcefolder]/bin/runtime $sourcefolder $::punkboot::target_platform] foreach _rec [punkboot::lib::mapvfs_kit_outputs $_smokemodel [file dirname $sourcefolder]/bin/runtime $sourcefolder] { if {[llength [dict get $_rec smokerequire]]} { lappend _smoke_declared "[dict get $_rec kitname] ([join [dict get $_rec smokerequire] {, }])" @@ -3960,7 +4410,7 @@ if {$::punkboot::command eq "check"} { unset -nocomplain _smokemodel _rec } puts stdout "smoke-require probe (G-133, advisory): freshly built host-runnable kits declaring packages" - puts stdout " (mapvfs.config 5th entry element) have each one plain-'package require'd inside the built" + puts stdout " (mapvfs.toml smokerequire key; legacy mapvfs.config 5th entry element) have each one plain-'package require'd inside the built" puts stdout " artifact via its tclsh subcommand - the only check that sees resolution-order defects such as" puts stdout " a wrong-arch higher-versioned package shadowing a working copy. Failures are recapped" puts stdout " BUILD-WARNINGs; cross-target kits skip with a stated reason; undeclared kits run nothing new." @@ -4167,7 +4617,7 @@ if {$::punkboot::command eq "bakelist"} { set rtbase $binfolder/runtime set rtfolder $rtbase/$rt_os_arch set buildfolder $sourcefolder/_build - set mapfile $sourcefolder/runtime/mapvfs.config + set mapfile [punkboot::lib::mapvfs_locate $sourcefolder/runtime] set mapmodel [punkboot::lib::mapvfs_parse $mapfile $rtbase $sourcefolder $rt_default_target] foreach errline [dict get $mapmodel configerrors] { puts stderr $errline @@ -4175,6 +4625,15 @@ if {$::punkboot::command eq "bakelist"} { foreach badline [dict get $mapmodel badentries] { puts stderr $badline } + if {[dict get $mapmodel format] eq "legacy"} { + #deprecation NOTE (first warnings line) - bakelist is reporting-only, so + #surface it here rather than relying on the bake path's warning printout + foreach warnline [dict get $mapmodel warnings] { + if {[string match NOTE:* $warnline]} { + puts stderr $warnline + } + } + } if {![dict get $mapmodel exists]} { puts stdout "no kit mapping config at $mapfile - no kit outputs configured" exit 0 @@ -4200,7 +4659,7 @@ if {$::punkboot::command eq "bakelist"} { if {$t eq $rt_os_arch} { puts stderr "WARNING: runtime store folder bin/runtime/$t does not exist - every configured runtime on the default target will read runtime=missing." puts stderr " Default target '$rt_default_target' derives from THIS tclsh: tcl_platform(os)='$::tcl_platform(os)' tcl_platform(platform)='$::tcl_platform(platform)' -> host canon '$::punkboot::host_platform' -> target '$rt_default_target'." - puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.config for runtimes stored elsewhere." + puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.toml for runtimes stored elsewhere." } else { puts stderr "WARNING: runtime store folder bin/runtime/$t does not exist - configured entries targeting $t will read runtime=missing." } @@ -4211,6 +4670,9 @@ if {$::punkboot::command eq "bakelist"} { if {[llength [dict get $matchinfo unknown]]} { puts stderr "bakelist: unknown kit name(s): [join [dict get $matchinfo unknown] {, }]" puts stderr " configured kit outputs: [join [lmap r $kit_outputs {dict get $r kitname}] {, }]" + if {[dict size [dict get $mapmodel groups]]} { + puts stderr " configured groups (select with @): [join [dict keys [dict get $mapmodel groups]] {, }]" + } exit 1 } set reportset [dict get $matchinfo selected] @@ -4222,7 +4684,14 @@ if {$::punkboot::command eq "bakelist"} { set kit_type [dict get $rec kit_type] set runtime [dict get $rec runtime] set vfstail [dict get $rec vfs] - set deployed [punkboot::lib::kit_deploy_state $buildfolder $binfolder [dict get $rec targetkit]] + #G-127: deploy-state roots follow the record's output tier (non-default + #targets live under kits// in both bin/ and src/_build/) + set out_tier [dict get $rec out_tier] + if {$out_tier eq ""} { + set deployed [punkboot::lib::kit_deploy_state $buildfolder $binfolder [dict get $rec targetkit]] + } else { + set deployed [punkboot::lib::kit_deploy_state $buildfolder/$out_tier $binfolder/$out_tier [dict get $rec targetkit]] + } set notes [list] if {$runtime ne "-" && ![dict get $rec runtime_present]} { lappend notes runtime=missing @@ -4234,6 +4703,18 @@ if {$::punkboot::command eq "bakelist"} { #cross-target entry: its runtime, store tier and artifact naming follow #its own platform, not this host's default lappend notes target=[dict get $rec target] + #G-127: and its build/deploy locations sit under the kits// tier + lappend notes out=$out_tier/ + } + if {[dict get $rec group] ne ""} { + lappend notes group=[dict get $rec group] + } + if {![dict get $rec bake_default]} { + #excluded from full bakes - bake by name or @group (G-024) + lappend notes default=no + } + if {[dict get $rec scheme_role] ne ""} { + lappend notes scheme=[dict get $rec scheme_role] } if {$runtime ne "-" && [dict get $rec runtime_present]} { set rtstale [punkboot::lib::runtime_materialization_check [dict get $rec runtime_dir] [dict get $rec runtime_file]] @@ -4256,7 +4737,15 @@ if {$::punkboot::command eq "bakelist"} { set widths $newwidths } set rt_store_note [expr {$rt_store_missing ? " (FOLDER MISSING)" : ""}] - puts stdout "# mapvfs: src/runtime/mapvfs.config runtimes: bin/runtime/$rt_os_arch$rt_store_note deployed: bin/ vs src/_build/" + if {[info exists ::env(PUNK_MAPVFS_CONFIG)] && $::env(PUNK_MAPVFS_CONFIG) ne ""} { + set map_display "$mapfile (PUNK_MAPVFS_CONFIG override)" + } else { + set map_display "src/runtime/[file tail $mapfile]" + if {[dict get $mapmodel format] eq "legacy"} { + append map_display " (deprecated format)" + } + } + puts stdout "# mapvfs: $map_display runtimes: bin/runtime/$rt_os_arch$rt_store_note deployed: bin/ vs src/_build/ (out= rows: under kits//)" set headline "" foreach h $headings w $widths { append headline [format "%-*s " $w $h] @@ -4289,9 +4778,18 @@ if {$::punkboot::command eq "bakelist"} { } puts stdout " vfs folder: src/vfs/[dict get $rec vfs] [expr {[dict get $rec vfs_present] ? {(present)} : {(MISSING)}}]" puts stdout " kit type: [dict get $rec kit_type]" - puts stdout " target: [dict get $rec target][expr {[dict get $rec target] eq $rt_default_target ? " (host default)" : " (declared in mapvfs.config)"}]" - puts stdout " build product: src/_build/[dict get $rec targetkit] [punkboot::lib::bakelist_file_detail $buildfolder/[dict get $rec targetkit]]" - puts stdout " deployed: bin/[dict get $rec targetkit] [punkboot::lib::bakelist_file_detail $binfolder/[dict get $rec targetkit]] state=[lindex $row 4]" + puts stdout " target: [dict get $rec target][expr {[dict get $rec target] eq $rt_default_target ? " (host default)" : " (declared in the kit mapping)"}]" + if {[dict get $rec entry] ne ""} { + puts stdout " config entry: \[[dict get $rec entry]\]" + } + if {[dict get $rec group] ne ""} { + puts stdout " group: [dict get $rec group][expr {[dict get $rec bake_default] ? "" : " (bake_default=false - bake by name or @[dict get $rec group])"}]" + } elseif {![dict get $rec bake_default]} { + puts stdout " bake_default: false (excluded from full bakes - bake by name to build)" + } + set relkit [expr {[dict get $rec out_tier] eq "" ? [dict get $rec targetkit] : "[dict get $rec out_tier]/[dict get $rec targetkit]"}] + puts stdout " build product: src/_build/$relkit [punkboot::lib::bakelist_file_detail $buildfolder/$relkit]" + puts stdout " deployed: bin/$relkit [punkboot::lib::bakelist_file_detail $binfolder/$relkit] state=[lindex $row 4]" } } exit 0 @@ -5349,20 +5847,8 @@ if {$::punkboot::command eq "libfetch"} { puts stderr "$A(BAD)LIBFETCH: failed to parse $libfetch_config\n$libfetch_dict$A(RST)" exit 3 } - if {![llength [info commands ::punkboot::lib::toml_untag]]} { - #tomlish::to_dict values are type-tagged (also defined by the vfslibs - #phase - that block does not run for the libfetch command) - proc ::punkboot::lib::toml_untag {tagged} { - if {[dict get $tagged type] eq "ARRAY"} { - set plain [list] - foreach el [dict get $tagged value] { - lappend plain [::punkboot::lib::toml_untag $el] - } - return $plain - } - return [dict get $tagged value] - } - } + #tomlish::to_dict values are type-tagged - ::punkboot::lib::toml_untag (defined + #with the shared toml helpers near the top of this script) resolves them set libfetch_artifacts [dict create] if {[dict exists $libfetch_dict artifact]} { set libfetch_artifacts [dict get $libfetch_dict artifact] @@ -5683,135 +6169,222 @@ if {$::punkboot::command in {bakehouse packages libs}} { } if {$::punkboot::command in {bakehouse bake vfslibs}} { - #G-037: propagate declared vendored platform-library packages into kit vfs lib_tcl trees. - #Participation is per-package per-kit (kit vfs package sets intentionally differ - e.g. twapi - #versions vary per kit) - declared in src/runtime/vendorlib_vfs.toml, never blanket-copied. - #punkcheck records live at src/vfs/.punkcheck so they are not baked into kit payloads. - set vfslibs_config $sourcefolder/runtime/vendorlib_vfs.toml - if {![file exists $vfslibs_config]} { - puts stdout "VFSLIBS: no config at $vfslibs_config - skipping vendorlib -> kit vfs propagation" - } else { - if {[catch {package require tomlish} tomlish_err]} { - puts stderr "$A(BAD)VFSLIBS: cannot process $vfslibs_config - tomlish package unavailable ($tomlish_err)$A(RST)" + #G-115: declarative .vfs payload composition. Each src/vfs/.vfs may have a + #SIBLING declaration file src/vfs/.vfs.toml declaring payload package folders + #to materialize INTO it - while the folder remains the operative assembly area: + #drop-ins need no declaration and are never clobbered. The declaration file's own + #header comment (and src/vfs/README.md) is the user-facing spec. Entry keys per + #[payload.]: source (folder path, resolved per source_root), source_root + #('src' default - vendor trees / suite build products under src/; or 'packages' - + #the consent-gated punkbin lib tier at /bin/packages populated by + #'make.tcl libfetch'), target (folder path INSIDE the .vfs, e.g "lib_tcl9"; + #default the .vfs root), supersedes (explicit legacy sibling folder removals), + #replace (boolean clean-slate of the SAME-NAMED target folder before install). + #A source containing %platform% is a PER-PLATFORM entry (G-127): one copy per + #consuming kit target (derived from the kit mapping below), staged under + #_targets// and selected into each kit's merged image at bake time. + #Materialization is punkcheck-tracked (records in src/vfs/.punkcheck, outside kit + #payloads) with -overwrite synced-targets: a target file punkcheck did not install, + #or that changed since it was installed (a drop-in or hand edit), is PRESERVED and + #reported, never overwritten - the G-115 drop-in-wins precedence; supersedes and + #replace are the explicit declared exceptions. A .vfs with no declaration file is + #untouched (pure drop-in mode, exactly as before G-115). + #This phase supersedes the per-package src/runtime/vendorlib_vfs.toml surface + #(G-037/G-139): its [install.] entries were migrated into per-.vfs files + #2026-07-31 - rationale in goals/G-115-declarative-vfs-composition.md. + set legacy_vfslibs $sourcefolder/runtime/vendorlib_vfs.toml + if {[file exists $legacy_vfslibs]} { + puts stderr "WARNING: $legacy_vfslibs is superseded by per-.vfs payload declarations (src/vfs/.vfs.toml - G-115) and is IGNORED - migrate its \[install.*\] entries into per-.vfs files and remove it." + } + set payload_files [lsort [glob -nocomplain -dir $sourcefolder/vfs -types f -tail *.vfs.toml]] + if {![llength $payload_files]} { + puts stdout "VFSPAYLOAD: no per-.vfs payload declarations (src/vfs/.vfs.toml) - nothing to materialize" + } + #G-127 per-platform payload derivation: a payload source containing %platform% + #declares ONE payload materialized per CONSUMING KIT TARGET - the platform set + #is derived from the kit mapping (every mapvfs kit output pairing this vfs, + #whatever its bake_default), never declared a second time in the payload file. + #Each platform's copy lands under _targets/// inside the .vfs + #folder (VCS-ignored staging); the bake selects the consuming kit's subtree + #into its merged image and drops the staging tree, so an assembled kit carries + #only its own target's payload. + set vfspayload_kit_targets [dict create] + if {[llength $payload_files]} { + set _pmapmodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate $sourcefolder/runtime] $binfolder/runtime $sourcefolder $::punkboot::target_platform] + if {[llength [dict get $_pmapmodel configerrors]]} { + puts stderr "WARNING: VFSPAYLOAD: kit-mapping config errors - per-platform (%platform%) payload derivation unavailable this run (the mapping's own gates report the errors)" + } else { + foreach _prec [punkboot::lib::mapvfs_kit_outputs $_pmapmodel $binfolder/runtime $sourcefolder] { + if {![dict exists $vfspayload_kit_targets [dict get $_prec vfs]] + || [dict get $_prec target] ni [dict get $vfspayload_kit_targets [dict get $_prec vfs]]} { + dict lappend vfspayload_kit_targets [dict get $_prec vfs] [dict get $_prec target] + } + } + } + unset -nocomplain _pmapmodel _prec + } + foreach ptail $payload_files { + set pfile $sourcefolder/vfs/$ptail + set vfstail [file rootname $ptail] ;#punkdeclare.vfs.toml -> punkdeclare.vfs + if {[llength $::punkboot::bake_selected_kitnames] && $vfstail ni $::punkboot::bake_selected_vfs} { + #G-121 selective bake: narrow to the selected kits' vfs folders (skip + #precedes the existence checks so an unselected declaration's problem + #cannot abort a selective run) + puts stdout "VFSPAYLOAD $vfstail: skipping (selective bake - not a selected kit's vfs)" + continue + } + if {![file isdirectory $sourcefolder/vfs/$vfstail]} { + puts stderr "$A(BAD)VFSPAYLOAD: declaration $pfile has no matching vfs folder at $sourcefolder/vfs/$vfstail$A(RST)" exit 3 } - set fd [open $vfslibs_config r] - fconfigure $fd -encoding utf-8 - set vfslibs_tomldata [read $fd] - close $fd - if {[catch {tomlish::to_dict [tomlish::from_toml $vfslibs_tomldata]} vfslibs_dict]} { - puts stderr "$A(BAD)VFSLIBS: failed to parse $vfslibs_config\n$vfslibs_dict$A(RST)" + lassign [punkboot::lib::toml_file_to_dict $pfile] pstatus ppayload + if {$pstatus ne "ok"} { + puts stderr "$A(BAD)VFSPAYLOAD: $ppayload$A(RST)" exit 3 } - #tomlish::to_dict values are type-tagged (e.g {type STRING value x}, {type ARRAY value {...}}) - proc ::punkboot::lib::toml_untag {tagged} { - if {[dict get $tagged type] eq "ARRAY"} { - set plain [list] - foreach el [dict get $tagged value] { - lappend plain [::punkboot::lib::toml_untag $el] - } - return $plain + dict for {topkey -} $ppayload { + if {$topkey ne "payload"} { + puts stderr "$A(BAD)VFSPAYLOAD: $pfile: unknown top-level table or key '$topkey' (expected \[payload.\] entries)$A(RST)" + exit 3 } - return [dict get $tagged value] } - set vfslibs_installs [dict create] - if {[dict exists $vfslibs_dict install]} { - set vfslibs_installs [dict get $vfslibs_dict install] + set payload_entries [dict create] + if {[dict exists $ppayload payload]} { + set payload_entries [dict get $ppayload payload] } - if {![dict size $vfslibs_installs]} { - puts stdout "VFSLIBS: no \[install.\] entries found in $vfslibs_config" + if {![dict size $payload_entries]} { + puts stdout "VFSPAYLOAD $vfstail: no \[payload.\] entries in $ptail" + continue } - dict for {iname ientry} $vfslibs_installs { - foreach reqkey {source targets} { - if {![dict exists $ientry $reqkey]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname in $vfslibs_config is missing required key '$reqkey'$A(RST)" - exit 3 - } - } - set isource [punkboot::lib::toml_untag [dict get $ientry source]] - set itargets [punkboot::lib::toml_untag [dict get $ientry targets]] - set isupersedes [list] - if {[dict exists $ientry supersedes]} { - set isupersedes [punkboot::lib::toml_untag [dict get $ientry supersedes]] - } - #source_root (G-139): 'src' (default - the original vendorlib - #declarations) or 'packages' - resolve against the untracked - #bin/packages lib-tier input (populated by 'make.tcl libfetch'). + dict for {iname ientry} $payload_entries { + set entrylabel "payload.$iname ($ptail)" + set isource "" set isource_root "src" - if {[dict exists $ientry source_root]} { - set isource_root [punkboot::lib::toml_untag [dict get $ientry source_root]] - } - #replace (G-139): clean-slate a SAME-NAMED existing target folder - #before install. supersedes deliberately cannot name the folder being - #installed; replace covers same-name refreshes where the old copy may - #carry strays the new tree lacks (e.g. hand-drops nesting md5c/tcllibc - #inside tcllib2.0). + set itarget "" + set isupersedes [list] set ireplace 0 - if {[dict exists $ientry replace]} { - set ireplace [punkboot::lib::toml_untag [dict get $ientry replace]] - if {![string is boolean -strict $ireplace]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname 'replace' must be boolean (got '$ireplace')$A(RST)" - exit 3 + dict for {k v} $ientry { + switch -- $k { + source {set isource [punkboot::lib::toml_untag $v]} + source_root {set isource_root [punkboot::lib::toml_untag $v]} + target {set itarget [punkboot::lib::toml_untag $v]} + supersedes {set isupersedes [punkboot::lib::toml_untag $v]} + replace { + set ireplace [punkboot::lib::toml_untag $v] + if {![string is boolean -strict $ireplace]} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel: 'replace' must be boolean (got '$ireplace')$A(RST)" + exit 3 + } + } + default { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel: unknown key '$k' (expected source, source_root, target, supersedes, replace)$A(RST)" + exit 3 + } } } - switch -- $isource_root { - src { - set source_pkg_folder $sourcefolder/$isource - set source_missing_hint "" - } - packages { - set source_pkg_folder $projectroot/bin/packages/$isource - set source_missing_hint " - populate the bin/packages tier first: 'tclsh src/make.tcl libfetch' (declarations in src/runtime/libpackages.toml)" - } - default { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname has unknown source_root '$isource_root' (expected src|packages)$A(RST)" - exit 3 - } + if {$isource eq ""} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel is missing required key 'source'$A(RST)" + exit 3 + } + if {$itarget ne "" && ([file pathtype $itarget] ne "relative" || ".." in [file split $itarget])} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel target '$itarget' must be a relative path inside the .vfs folder (no '..')$A(RST)" + exit 3 } - if {![file isdirectory $source_pkg_folder]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname source '$isource' not found at $source_pkg_folder$source_missing_hint$A(RST)" + #G-127 per-platform axis: %platform% in 'source' selects the payload per + #consuming kit target (set derived above); 'target' stays literal - its + #per-platform staging prefix (_targets//) is implicit. + if {[string match *%platform%* $itarget]} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel: 'target' must not contain %platform% - the per-platform staging prefix (_targets//) is implicit for a %platform% source$A(RST)" exit 3 } - set pkgtail [file tail $source_pkg_folder] - foreach t $itargets { - set target_vfs_dir [lindex [file split $t] 0] - if {[llength $::punkboot::bake_selected_kitnames] && $target_vfs_dir ni $::punkboot::bake_selected_vfs} { - #G-121 selective bake: narrow the vfslibs phase to the selected kits' - #vfs folders (skip precedes the existence check so an unselected - #entry's problem cannot abort a selective run) - puts stdout "VFSLIBS install.$iname: skipping vfs/$t (selective bake - not a selected kit's vfs)" + if {[string match *%platform%* $isource]} { + if {![dict exists $vfspayload_kit_targets $vfstail] + || ![llength [dict get $vfspayload_kit_targets $vfstail]]} { + puts stderr "WARNING: VFSPAYLOAD entry $entrylabel declares a per-platform source (%platform%) but no kit in the mapping consumes $vfstail - no platform set to derive, entry skipped" continue } - if {![file isdirectory $sourcefolder/vfs/$target_vfs_dir]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname target '$t' - no vfs folder at $sourcefolder/vfs/$target_vfs_dir$A(RST)" + set entry_platforms [dict get $vfspayload_kit_targets $vfstail] + } else { + set entry_platforms [list ""] ;#platform-neutral entry - materialize once, directly in the folder + } + foreach eplatform $entry_platforms { + if {$eplatform eq ""} { + set esource $isource + set elabel $entrylabel + set evfsprefix "" + } else { + set esource [string map [list %platform% $eplatform] $isource] + set elabel "$entrylabel \[$eplatform\]" + set evfsprefix _targets/$eplatform + } + switch -- $isource_root { + src { + set source_pkg_folder $sourcefolder/$esource + set source_missing_hint "" + } + packages { + set source_pkg_folder $projectroot/bin/packages/$esource + set source_missing_hint " - populate the bin/packages tier first: 'tclsh src/make.tcl libfetch' (declarations in src/runtime/libpackages.toml)" + } + default { + puts stderr "$A(BAD)VFSPAYLOAD: entry $elabel has unknown source_root '$isource_root' (expected src|packages)$A(RST)" + exit 3 + } + } + if {![file isdirectory $source_pkg_folder]} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $elabel source '$esource' not found at $source_pkg_folder$source_missing_hint$A(RST)" exit 3 } - set target_lib_folder $sourcefolder/vfs/$t + set pkgtail [file tail $source_pkg_folder] + set etarget_subpath $itarget + if {$evfsprefix ne ""} { + set etarget_subpath [expr {$itarget eq "" ? $evfsprefix : "$evfsprefix/$itarget"}] + } + if {$etarget_subpath eq ""} { + set target_lib_folder $sourcefolder/vfs/$vfstail + set target_display vfs/$vfstail + } else { + set target_lib_folder $sourcefolder/vfs/$vfstail/$etarget_subpath + set target_display vfs/$vfstail/$etarget_subpath + } file mkdir $target_lib_folder #supersession first - no silent mixed-version provision (G-035 concerns) foreach sup $isupersedes { if {$sup eq $pkgtail} { - continue ;#never remove what we are about to install + continue ;#never remove what we are about to install - use replace for that } set suppath $target_lib_folder/$sup if {[file exists $suppath]} { - puts stdout "VFSLIBS install.$iname: removing superseded '$sup' from vfs/$t" + puts stdout "VFSPAYLOAD $elabel: removing superseded '$sup' from $target_display" file delete -force $suppath } } if {$ireplace && [file isdirectory $target_lib_folder/$pkgtail]} { - puts stdout "VFSLIBS install.$iname: replace=true - removing existing '$pkgtail' from vfs/$t (clean-slate install)" + puts stdout "VFSPAYLOAD $elabel: replace=true - removing existing '$pkgtail' from $target_display (clean-slate install)" file delete -force $target_lib_folder/$pkgtail } - puts stdout "VFSLIBS install.$iname: $isource -> vfs/$t/$pkgtail (if source file changed)" + puts stdout "VFSPAYLOAD $elabel: $isource_root:$esource -> $target_display/$pkgtail (if source changed; drop-ins/hand-edits preserved)" set resultdict [punkcheck::install $source_pkg_folder $target_lib_folder/$pkgtail\ -installer make.tcl\ -createdir 1\ - -overwrite installedsourcechanged-targets\ + -overwrite synced-targets\ -punkcheck-folder $sourcefolder/vfs\ -progresschannel stdout\ ] puts stdout [punkcheck::summarize_install_resultdict $resultdict] + #preserved-collision report: a file skipped although its SOURCE changed was + #withheld by synced-targets (a drop-in, or hand-modified since install) - + #distinct from the routine source-unchanged skips. + set _preserved [list] + foreach _sk [dict get $resultdict files_skipped] { + if {$_sk ni [dict get $resultdict sources_unchanged]} { + lappend _preserved $_sk + } + } + if {[llength $_preserved]} { + puts stdout "VFSPAYLOAD $elabel: NOTE - [llength $_preserved] changed file(s) NOT overwritten (existing target content was not installed by this mechanism, or was modified since) - the G-115 drop-in-wins precedence. Declare 'replace = true' (whole package folder) or remove the target files to hand control to the declaration." + } + unset -nocomplain _preserved _sk } } } @@ -6224,7 +6797,7 @@ proc ::punkboot::runtime_materialization_warning {rtfolder runtime_fullname} { #G-122: parsed BEFORE runtime discovery, because the mapping is what says which #store tiers this bake must look in - an entry declaring a target platform other #than the host default addresses its own bin/runtime/. -set mapfile $rt_sourcefolder/mapvfs.config +set mapfile [punkboot::lib::mapvfs_locate $rt_sourcefolder] set mapmodel [punkboot::lib::mapvfs_parse $mapfile $rtbase $sourcefolder $rt_default_target] foreach warnline [dict get $mapmodel warnings] { puts stderr $warnline @@ -6240,11 +6813,53 @@ set vfs_runtime_map [dict get $mapmodel vfs_runtime_map] set runtime_target [dict get $mapmodel runtime_target] set missing [dict get $mapmodel missing] if {[llength $missing]} { - puts stderr "WARNING [llength $missing] missing items from $mapfile. (TODO - prompt user to continue/abort)" - foreach m $missing { - puts stderr " $m" + if {[dict get $mapmodel format] eq "toml" && ![llength $::punkboot::bake_selected_kitnames]} { + #G-024 strictness (toml models): a full bake with unresolvable DEFAULT entries + #reports each one BY ENTRY rather than as a bare item list. A missing vfs + #folder is fatal - vfs folders are repo content, so its absence means a wrong + #declaration. A missing runtime FILE is environment, not declaration: store + #tiers are legitimately part-populated (fresh clones; runtimes arrive via the + #consent-gated fetch tiers), so those entries skip with a recapped + #BUILD-WARNING naming the entry (cross-target wording states the tier reason). + #bake_default=false entries are not in the full-bake set; selective bakes + #fail fast at selection instead. + set _strict_fatal [list] + set _strict_warn [list] + foreach _rec [punkboot::lib::mapvfs_kit_outputs $mapmodel $rtbase $sourcefolder] { + if {![dict get $_rec bake_default]} { + continue + } + set _entrylabel [expr {[dict get $_rec entry] ne "" ? [dict get $_rec entry] : [dict get $_rec kitname]}] + if {![dict get $_rec vfs_present]} { + lappend _strict_fatal "entry $_entrylabel: vfs folder src/vfs/[dict get $_rec vfs] is missing" + continue + } + if {[dict get $_rec runtime] ne "-" && ![dict get $_rec runtime_present]} { + if {[dict get $_rec target] eq $rt_default_target} { + lappend _strict_warn "kit [dict get $_rec kitname] (entry $_entrylabel) skipped - runtime [dict get $_rec runtime_file] not present in bin/runtime/[dict get $_rec store_tier]" + } else { + lappend _strict_warn "kit [dict get $_rec kitname] (entry $_entrylabel) skipped - cross-target runtime [dict get $_rec runtime_file] not present in bin/runtime/[dict get $_rec store_tier] (target [dict get $_rec target] tier not populated on this host)" + } + } + } + if {[llength $_strict_fatal]} { + puts stderr "CONFIG ERROR: unresolvable kit mapping entries in $mapfile - nothing built:" + foreach _line $_strict_fatal { + puts stderr " $_line" + } + exit 3 + } + if {[llength $_strict_warn]} { + ::punkboot::print_build_warnings $_strict_warn + } + unset -nocomplain _rec _entrylabel _strict_fatal _strict_warn _line + } else { + puts stderr "WARNING [llength $missing] missing items from $mapfile. (TODO - prompt user to continue/abort)" + foreach m $missing { + puts stderr " $m" + } + puts stderr "continuing..." } - puts stderr "continuing..." } # -- --- --- --- --- --- --- --- --- --- puts "-- runtime_vfs_map --" @@ -6333,7 +6948,7 @@ dict for {_rtname _rttarget} $runtime_target { dict set rtfolder_of $_file $_dir } if {[llength $cross_target_tiers]} { - puts stdout "cross-target runtime tiers in use (mapvfs.config target platforms): [join $cross_target_tiers {, }]" + puts stdout "cross-target runtime tiers in use (kit-mapping target platforms): [join $cross_target_tiers {, }]" } unset -nocomplain _rtname _rttarget _tier _dir _file if {![llength $runtimes]} { @@ -6342,7 +6957,7 @@ if {![llength $runtimes]} { if {![file isdirectory $rtfolder]} { #same store-tier self-diagnosis as bakelist puts stderr "NOTE: that runtime store folder does not exist. Default target '$rt_default_target' derives from THIS tclsh: tcl_platform(os)='$::tcl_platform(os)' tcl_platform(platform)='$::tcl_platform(platform)' -> host canon '$::punkboot::host_platform' -> target '$rt_default_target'." - puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.config for runtimes stored elsewhere." + puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.toml for runtimes stored elsewhere." } #todo - don't exit - it is valid to use runtime of - to just build a .kit/.zipkit ? exit 0 @@ -7323,6 +7938,10 @@ foreach vfstail $vfs_tails { } set rt_target [punkboot::lib::runtime_target_of $rt] set rt_file $rt[punkboot::lib::platform_exe_suffix $rt_target] + if {$rt_file in $runtimes} { + continue ;#several entries may share a runtime on one vfs (toml models) - + #process each runtime once (its spec set enumerates all its kits) + } if {[file exists $rtbase/[punkboot::lib::platform_store_tier $rt_target]/$rt_file]} { lappend runtimes $rt_file } @@ -7376,7 +7995,9 @@ foreach vfstail $vfs_tails { set applist [dict get $runtime_vfs_map $rtname] foreach vfs_app $applist { #5th element = G-133 smoke-require package list ("" when undeclared) + #6th element (toml models) = G-024 entry attrs (group/bake_default/scheme_role) lassign $vfs_app configured_vfs appname target_kit_type - kit_smokerequires + set kit_attrs [punkboot::lib::mapvfs_spec_attrs $vfs_app] if {$configured_vfs ne $vfstail} { continue } @@ -7390,15 +8011,18 @@ foreach vfstail $vfs_tails { set targetkit $appname.kit } else { set targetkit ${appname}$rt_suffix - if {$targetkit in $exe_names_seen} { - #duplicate appname configured? + if {[list $rt_target $targetkit] in $exe_names_seen} { + #duplicate appname configured for the SAME target? + #(G-127: same-named kits for DIFFERENT targets coexist under + #their own kits// output tiers - only a same-target + #duplicate needs a disambiguating name) #todo - consider creating as ${appname}(2) etc? - puts stderr "targetkit: $targetkit already seen - using name ${appname}_$rtname" + puts stderr "targetkit: $targetkit already seen for target $rt_target - using name ${appname}_$rtname" set targetkit ${appname}_$rtname } } - lappend exe_names_seen $targetkit - lappend targetkits [list $targetkit $target_kit_type $kit_smokerequires] + lappend exe_names_seen [list $rt_target $targetkit] + lappend targetkits [list $targetkit $target_kit_type $kit_smokerequires $kit_attrs] } } puts stdout " vfs: $vfstail runtime: $rtname targetkits: $targetkits" @@ -7407,7 +8031,41 @@ foreach vfstail $vfs_tails { continue ;#G-121 selective bake - not a requested kit output } puts stdout " processing targetkit: $targetkit_info" - lassign $targetkit_info targetkit target_kit_type kit_smokerequires + lassign $targetkit_info targetkit target_kit_type kit_smokerequires kit_attrs + #G-127: output locations are keyed by the kit's TARGET. Kits for this + #host's own default target keep the flat locations every launcher, test + #and habit expects (src/_build/, bin/); any other target's kit + #builds and deploys under a kits// tier so same-named kits for + #different targets coexist instead of overwriting each other across + #selective runs. bin/kits/ rather than bin// because + #bin/runtime// already means build INPUTS. + if {$rt_target eq $rt_default_target} { + set kit_builddir $buildfolder + set kit_deploydir $binfolder + set kit_relpath $targetkit ;#kit path relative to bin/ and src/_build/ + } else { + set kit_builddir $buildfolder/kits/$rt_target + set kit_deploydir $binfolder/kits/$rt_target + set kit_relpath kits/$rt_target/$targetkit + } + file mkdir $kit_builddir + #G-024: bake_default=false entries (declared per entry or via their group) + #are excluded from full bakes - explicit name/@group selection builds them. + if {![llength $::punkboot::bake_selected_kitnames] + && [dict exists $kit_attrs bake_default] && ![dict get $kit_attrs bake_default]} { + puts stdout " skipping $targetkit (bake_default=false - bake by name or @group to build it)" + lappend skipped_kits [list kit $targetkit reason "bake_default=false (not part of full bakes)"] + continue + } + #G-023/G-024 release gating: a 'versioned' scheme's plain-name output is + #created when absent, then replaced only by an explicit release step + #(G-023 owns that step) - a normal bake never overwrites it. + if {[dict exists $kit_attrs scheme_role] && [dict get $kit_attrs scheme_role] eq "release" + && [file exists $kit_deploydir/$targetkit]} { + puts stdout " skipping $targetkit (release-gated scheme output: bin/$kit_relpath exists and a normal bake never overwrites it - the explicit release step is G-023; to force a rebuild now, delete bin/$kit_relpath first)" + lappend skipped_kits [list kit $targetkit reason "release-gated (bin/$kit_relpath exists)"] + continue + } #Self-build guard: never process the kit whose deployed executable is running this #build - the process sweep below would kill this very process, and the deploy step #cannot replace a running image anyway. Informational/update subcommands are @@ -7415,7 +8073,7 @@ foreach vfstail $vfs_tails { #punk executable remains supported for everything except rebuilding that executable. #(a HOST filesystem comparison - case rules follow the host, not the target) set self_exe [file normalize [info nameofexecutable]] - set self_deploy_target [file normalize $binfolder/$targetkit] + set self_deploy_target [file normalize $kit_deploydir/$targetkit] if {$::punkboot::host_windows} { set is_self_build [string equal -nocase $self_exe $self_deploy_target] } else { @@ -7431,7 +8089,7 @@ foreach vfstail $vfs_tails { set vfs_installer [punkcheck::installtrack new $installername $basedir/.punkcheck] $vfs_installer set_source_target $sourcefolder $buildfolder set vfs_event [$vfs_installer start_event {-make-step build_vfs}] - $vfs_event targetset_init INSTALL $buildfolder/$targetkit + $vfs_event targetset_init INSTALL $kit_builddir/$targetkit set relvfs [punkcheck::lib::path_relative $basedir $sourcefolder/vfs/$vfstail] if {![dict exists $path_cksum_cache $relvfs]} { #e.g ../vfs/punk87.vfs {cksum xxxx cksum_all_opts {-cksum_content 1 ... -cksum_algorithm sha1}} @@ -7475,7 +8133,8 @@ foreach vfstail $vfs_tails { package require control #keep this a simple name - bin/punk script calls into src/_build/exename.vfs/main.tcl - set targetvfs $buildfolder/$targetkit.vfs + #(G-127: non-default-target kits keep the simple name under their kits// tier) + set targetvfs $kit_builddir/$targetkit.vfs file delete -force $targetvfs set extraction_done 0 @@ -7781,6 +8440,26 @@ foreach vfstail $vfs_tails { set sourcevfs [file join $sourcefolder vfs $vfstail] merge_over $sourcevfs $targetvfs + #G-127 per-platform payload selection: the VFSPAYLOAD phase stages + #%platform% payload entries under _targets// in the .vfs + #folder (one subtree per consuming kit target), and the merge above + #carried that staging tree into the image. Select THIS kit's target + #subtree into the image root and drop the staging tree, so the + #assembled kit carries only its own target's payload - upstream of + #the arch scan and the G-125 gate, which must judge the final tree. + if {[file isdirectory $targetvfs/_targets]} { + set _pstaging ${targetvfs}._targets_staging + file delete -force $_pstaging + file rename $targetvfs/_targets $_pstaging + if {[file isdirectory $_pstaging/$rt_target]} { + puts stdout " per-platform payload (G-127): selecting _targets/$rt_target into the merged image for kit $targetkit" + merge_over $_pstaging/$rt_target $targetvfs + } else { + ::punkboot::print_build_warnings [list "per-platform payload (G-127): kit $targetkit (target $rt_target) - the merged vfs has a _targets/ staging tree but no _targets/$rt_target subtree, so this kit gets NO per-platform payload (check the %platform% declarations in src/vfs/$vfstail.toml against the kit's mapvfs target)"] + } + file delete -force $_pstaging + } + #G-133 advisory payload/target binary-arch scan, at the same post-merge #seam as the G-125 gate below. Structural (header bytes only, nothing #executed), so it runs for every kit including cross-target ones. A @@ -7847,7 +8526,7 @@ foreach vfstail $vfs_tails { #the assemblers below consume wrap_runtime - the stamped per-kit copy #of the payload-free raw prefix when embedding happened, the untouched #raw prefix otherwise. Runtime store originals are never modified. - lassign [::punkboot::kit_icon_process $targetkit $rt_target $rtname $target_kit_type $sourcefolder $vfstail $raw_runtime $buildfolder] wrap_runtime kiticon_sidecar + lassign [::punkboot::kit_icon_process $targetkit $rt_target $rtname $target_kit_type $sourcefolder $vfstail $raw_runtime $kit_builddir] wrap_runtime kiticon_sidecar set wrapvfs $targetvfs switch -- $target_kit_type { @@ -8147,13 +8826,13 @@ foreach vfstail $vfs_tails { } } - if {[file exists $buildfolder/$targetkit]} { - puts stderr "deleting existing $buildfolder/$targetkit" + if {[file exists $kit_builddir/$targetkit]} { + puts stderr "deleting existing $kit_builddir/$targetkit" if {[catch { - file delete $buildfolder/$targetkit + file delete $kit_builddir/$targetkit } msg]} { - puts stderr "Failed to delete $buildfolder/$targetkit" - lappend failed_kits [list kit $targetkit reason "could not delete buildfolder kit at $buildfolder/$targetkit"] + puts stderr "Failed to delete $kit_builddir/$targetkit" + lappend failed_kits [list kit $targetkit reason "could not delete buildfolder kit at $kit_builddir/$targetkit"] $vfs_event targetset_end FAILED $vfs_event destroy $vfs_installer destroy @@ -8163,7 +8842,7 @@ foreach vfstail $vfs_tails { #WINDOWS filesystem 'tunnelling' (file replacement within 15secs) could cause targetkit to copy ctime & shortname metadata from previous file! #This is probably harmless - but worth being aware of. - file rename $buildfolder/$vfsname.new $buildfolder/$targetkit + file rename $buildfolder/$vfsname.new $kit_builddir/$targetkit # -- --- --- --- --- --- $vfs_event targetset_end OK @@ -8175,7 +8854,7 @@ foreach vfstail $vfs_tails { #deploys; plain/none (no attached zip - the metakit kit shape) and #unreadable results are silence. One EOCD scan + CD walk per #REBUILT kit only. - lassign [::punkboot::get_kit_offsetstyle_report $buildfolder/$targetkit] offprobe_available offprobe + lassign [::punkboot::get_kit_offsetstyle_report $kit_builddir/$targetkit] offprobe_available offprobe if {!$offprobe_available} { puts stderr "NOTE: kit offset-style pin unavailable (punkboot::utils kit_offsetstyle_report not loadable from bootsupport) - continuing without it" } elseif {[dict get $offprobe offsetstyle] eq "file"} { @@ -8195,7 +8874,7 @@ foreach vfstail $vfs_tails { puts stdout " smoke-require skipped for kit $targetkit: cross-target ($rt_target) - the artifact is not executable on this $::punkboot::host_platform host" } else { puts stdout " smoke-require probe for kit $targetkit (packages: $kit_smokerequires) - executing freshly built artifact via its tclsh subcommand" - ::punkboot::kit_smoke_require_probe $buildfolder/$targetkit $targetkit $kit_smokerequires + ::punkboot::kit_smoke_require_probe $kit_builddir/$targetkit $targetkit $kit_smokerequires } } @@ -8209,14 +8888,13 @@ foreach vfstail $vfs_tails { $vfs_event destroy $vfs_installer destroy - #TODO - allow building of kits for other platforms - # - we need to use /bin for only kits targetting current platform, - # and use /bin/ for others - # that we we can have same target executable for multiple platforms e.g executable just named 'punkshell' - # - to do this we need to change runtime/mapvfs.config to have platform names not just standalone target executable names - # (todo: change to toml format at same time) + #Cross-platform kit outputs: RETIRED TODO (G-127 - see + #goals/G-127-crosstarget-vfs-bake.md). Kits for this host's default + #target deploy flat to /bin exactly as always; any other + #target's kit deploys under /bin/kits// (the + #mapping side gained platform names + toml format under G-122/G-024). after 200 - set deployment_folder [file dirname $sourcefolder]/bin + set deployment_folder $kit_deploydir file mkdir $deployment_folder # -- ---------- @@ -8228,7 +8906,7 @@ foreach vfstail $vfs_tails { #set last_completion [$bin_event targetset_last_complete] $bin_event targetset_addsource $deployment_folder/$targetkit ;#add target as a source of metadata for change detection - $bin_event targetset_addsource $buildfolder/$targetkit + $bin_event targetset_addsource $kit_builddir/$targetkit $bin_event targetset_started # -- ---------- @@ -8251,16 +8929,16 @@ foreach vfstail $vfs_tails { } puts stdout "copying.." - puts stdout "$buildfolder/$targetkit" + puts stdout "$kit_builddir/$targetkit" puts stdout "to:" puts stdout "$deployment_folder/$targetkit" after 300 - file copy $buildfolder/$targetkit $deployment_folder/$targetkit + file copy $kit_builddir/$targetkit $deployment_folder/$targetkit #G-057: the kit resource sidecar ships with the kit - if {[file isfile $buildfolder/$targetkit.resources.toml]} { - file copy -force $buildfolder/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml + if {[file isfile $kit_builddir/$targetkit.resources.toml]} { + file copy -force $kit_builddir/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml } - lappend installed_kits $targetkit + lappend installed_kits $kit_relpath # -- ---------- $bin_event targetset_end OK # -- ---------- @@ -8271,7 +8949,7 @@ foreach vfstail $vfs_tails { #G-057: keep the deployed resource sidecar current even when the kit #itself is unchanged (the record can be newer than the kit rebuild - #e.g the sidecar feature arriving, or a host-capability change) - if {[::punkboot::kiticon::copy_ifchanged $buildfolder/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml] eq "copied"} { + if {[::punkboot::kiticon::copy_ifchanged $kit_builddir/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml] eq "copied"} { puts stdout " kit icon (G-057): refreshed deployed sidecar $deployment_folder/$targetkit.resources.toml" } lappend skipped_kit_installs [list kit $targetkit reason "no change detected"] @@ -8299,7 +8977,7 @@ if {[llength $failed_kits]} { } set had_kits [expr {[llength $installed_kits] || [llength $failed_kits] || [llength $skipped_kits]}] if {$had_kits} { - puts stdout " module builds and kit/zipkit builds processed (vfs config: src/runtime/mapvfs.config)" + puts stdout " module builds and kit/zipkit builds processed (kit mapping: src/runtime/mapvfs.toml)" puts stdout " - use 'make.tcl modules' to build modules without baking the vfs folders into executable kits/zipkits" puts stdout " - use 'make.tcl vfscommonupdate' to promote built modules into the base vfs folder /src/vfs/_vfscommon.vfs (promotion gate)" puts stdout " - Note that without the vfscommonupdate step, 'make.tcl bake' (the kit-assembly stage of 'make.tcl bakehouse') will build vfs based executables" @@ -8309,7 +8987,7 @@ if {$had_kits} { puts stdout " module builds processed" puts stdout "" puts stdout " If kit/zipkit based executables required - create src/vfs/.vfs folders containing lib,modules,modules_tcl9 etc folders" - puts stdout " Also ensure appropriate executables exist in src/runtime along with src/runtime/mapvfs.config" + puts stdout " Also ensure appropriate runtime executables exist in bin/runtime/ along with the kit mapping src/runtime/mapvfs.toml" } puts stdout "-done-" exit 0 diff --git a/src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.config b/src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.config deleted file mode 100644 index c4d82bf4..00000000 --- a/src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.config +++ /dev/null @@ -1,16 +0,0 @@ -#single line per runtime executable. Name of runtime followed by list of kit configuration lists. -#Each kit config list is from 1 to 3 elements -#1) .vfs folder with path relative to src folder. -#2) name of wrapped kit binary to create (default to rootname of .vfs folder e.g myapp.vfs will create a kit named myapp (or myapp.exe) on windows) -#3) Type of kit to create (zip or kit) - default kit. kit for metakit or vlerq based kit, zip for more modern zipkit. -# -# e.g The following creates 2 zipkits from the same runtime and 2 different .vfs folder -# mytcl90kit.exe {mainapp.vfs myapp.exe zip} {test.vfs testapp.exe zip} -# -#if runtime has no entry - it will only match a .vfs folder with a matching filename e.g runtime1.exe runtime1.vfs -#Use a runtime with a name of dash (-) to build a .kit file from the .vfs folder using no runtime -#e.g -#- myproject.vfs -#- punk86.vfs -# -tclsh902z.exe {project.vfs project zip} diff --git a/src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.toml b/src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.toml new file mode 100644 index 00000000..8cab3328 --- /dev/null +++ b/src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/runtime/mapvfs.toml @@ -0,0 +1,34 @@ +# mapvfs.toml - kit output mapping (parsed with tomlish via make.tcl's ::punkboot::lib::mapvfs_parse). +# Maps runtime executables (/bin/runtime//) and .vfs payload folders +# (src/vfs/) to the named kit executables a bake produces. +# +# [kit.] - one table per kit output; is the output executable's base name. +# runtime (required) runtime executable name, unsuffixed (a superfluous .exe is tolerated). +# Use "-" to build a runtime-less .kit file. A runtime with NO entry anywhere +# still auto-pairs with a same-named .vfs folder. +# vfs (required) .vfs folder name under src/vfs/ (e.g "project.vfs"). +# type kit|zip|zipcat|cookit|cookfs (default kit). kit = metakit/starkit wrap, +# zip = zipfs image, zipcat = raw exe+zip concatenation. +# target (optional) target platform-dir name ('help platforms' lists them); defaults to +# the build host's own target. Decides the bin/runtime/ store, .exe +# suffixing and process tooling. A property of the RUNTIME - entries sharing a +# runtime must not disagree. +# smokerequire (optional) packages the freshly built kit must plain-'package require' +# (advisory probe; failures are recapped BUILD-WARNINGs). +# group (optional) named grouping - 'make.tcl bake @' / 'bakelist @'. +# bake_default (optional boolean, default true; a group's bake_default is the members' +# default) false = excluded from full bakes, still bakeable by name/@group. +# +# [group.] - optional group attributes: description, bake_default. +# [scheme.] - generative version-named outputs (scheme = "versioned"): expands to +# - + -dev + release-gated plain from the punkproject.toml +# [project] version. Keys: scheme, prefix (default: table name), runtime, vfs, type +# (required), plus target/smokerequire/group/bake_default as for kits. +# +# The punkshell tree's src/runtime/mapvfs.toml header carries the full format spec. +# (The pre-toml mapvfs.config line format remains readable as a deprecated fallback.) + +[kit.project] +runtime = "tclsh902z" +vfs = "project.vfs" +type = "zip" diff --git a/src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/vfs/README.md b/src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/vfs/README.md new file mode 100644 index 00000000..8397d864 --- /dev/null +++ b/src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/vfs/README.md @@ -0,0 +1,31 @@ +# src/vfs - kit payload folders and per-.vfs payload declarations + +Each `.vfs` folder is the payload of a configured kit output (kit mapping: +`src/runtime/mapvfs.toml`), merged with the `_vfscommon.vfs` overlay at bake time. + +## Per-.vfs payload declarations (optional) + +A `.vfs` folder's payload can be DECLARED in a sibling toml file and materialized by the +build - while the folder remains the operative assembly area: dropping files in by hand +needs no declaration, and materialization never clobbers what it did not itself install. +A `.vfs` with NO declaration file is untouched - pure drop-in mode. + +- Declaration file: `src/vfs/.vfs.toml` (sibling of the folder, never baked into it). +- Processed by `tclsh src/make.tcl vfslibs`, and automatically as a phase of + `make.tcl bake` / `bakehouse`. + +```toml +[payload.] +source = "vendorlib/mylib1.0" # folder path, resolved per source_root +source_root = "src" # optional: 'src' (default) or 'packages' + # (/bin/packages - 'make.tcl libfetch' tier) +target = "lib" # optional: folder INSIDE the .vfs (default: the .vfs root) +supersedes = ["mylib0.9"] # optional: legacy sibling folders removed from target first +replace = true # optional: clean-slate the SAME-NAMED target folder first +``` + +Precedence (drop-in wins): materialization is punkcheck-tracked with +`-overwrite synced-targets` - undeclared files are never touched, and a file at a declared +path is only overwritten when the mechanism itself installed it and it is unmodified +since; a foreign or hand-modified copy is preserved and reported. `supersedes` and +`replace` are the explicit declared exceptions. diff --git a/src/project_layouts/vendor/punk/basic/src/make.tcl b/src/project_layouts/vendor/punk/basic/src/make.tcl index a55eda23..69267f8e 100644 --- a/src/project_layouts/vendor/punk/basic/src/make.tcl +++ b/src/project_layouts/vendor/punk/basic/src/make.tcl @@ -187,7 +187,7 @@ namespace eval ::punkboot { variable foldername [file tail $scriptfolder] variable pkg_requirements [list]; variable pkg_missing [list];variable pkg_loaded [list] variable help_flags [list -help --help /? -h] - variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate libfetch bootsupport vfscommonupdate projectversion workflow buildsuite tool project vfs] + variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate libfetch bootsupport vfscommonupdate projectversion workflow buildsuite tool] } @@ -627,11 +627,15 @@ proc ::punkboot::lib::bootsupport_prompt_yesno {prompt} { } # -- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- -# Parsed kit-mapping model (G-121). One reader for src/runtime/mapvfs.config - -# the 'bake' kit machinery, the 'bakelist' report and the bake/bakelist argdoc -# choices all consume the model these helpers return, never the file format -# directly, so a config-format change (G-024 toml direction) swaps in underneath -# by reimplementing mapvfs_parse alone. +# Parsed kit-mapping model (G-121). One reader family for the kit-output mapping +# config - the 'bake' kit machinery, the 'bakelist' report and the bake/bakelist +# argdoc choices all consume the model these helpers return, never the file +# format directly. G-024: the canonical format is src/runtime/mapvfs.toml +# (tomlish-parsed [kit.] / [group.] / [scheme.] tables - the +# file's own header comment is the user-facing spec); the pre-G-024 line format +# src/runtime/mapvfs.config remains readable as a DEPRECATED fallback so +# un-migrated trees (older generated projects) keep baking. mapvfs_locate picks +# the file (toml wins when both exist); mapvfs_parse dispatches on extension. # # mapvfs_parse: pure parse - no output, no exits. Every store address, presence # probe and artifact suffix it computes is keyed by the entry's TARGET platform @@ -639,39 +643,423 @@ proc ::punkboot::lib::bootsupport_prompt_yesno {prompt} { # (bin/runtime) and each entry addresses $rtbase/. Returns a dict: # exists 0|1 (mapfile present) # mapfile path as supplied +# format toml|legacy (which reader produced the model) # default_target target platform for entries that declare none -# runtime_vfs_map runtime -> raw vfs_specs as listed (appname/type/target may be "") -# vfs_runtime_map vfstail -> list of {runtime appname type target smokerequires} -# (defaults applied; only vfs folders that exist on disk - matching -# the historical inline parse the kit loop was built around; -# smokerequires is the G-133 smoke-require package list, may be "") +# runtime_vfs_map runtime -> list of vfs_specs. legacy: raw specs as listed +# (appname/type/target may be ""). toml: normalized 6-element +# specs {vfstail appname type target smokerequires attrs} +# vfs_runtime_map vfstail -> list of {runtime appname type target smokerequires +# ?attrs?} (defaults applied; only vfs folders that exist on +# disk - matching the historical inline parse the kit loop was +# built around; smokerequires is the G-133 smoke-require package +# list, may be ""). attrs (6th element, toml models only - read +# via mapvfs_spec_attrs) is a dict: entry (config table label +# e.g kit.punk91), group, bake_default, scheme, scheme_role. # runtime_target runtime -> resolved target platform (all runtimes, mapped order) +# groups group name -> {description bake_default 0|1} (toml only) # missing names of referenced-but-absent runtimes/vfs folders +# missing_detail structured missing items (toml only): list of dicts +# {kind runtime|vfs label name target } +# - the bake path's strict per-entry failure reporting (G-024) # warnings warning lines in encounter order, ready to print verbatim # (missing-item lines carry their historical WARNING: prefix) # badentries subset of warnings: malformed entry lines (these have no kit # record to carry them, so bakelist prints them; missing-item # warnings are instead surfaced as per-row state there) -# configerrors fatal config problems (duplicate runtime line; unusable or +# configerrors fatal config problems (malformed/unresolvable declarations, +# each naming its entry; duplicate runtime line; unusable or # self-contradicting target platform; caller decides whether to # abort - the bake path keeps its historical exit 3) -proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} { - set model [dict create\ + +# Locate the kit-mapping config for a src/runtime folder (G-024). +# Precedence: PUNK_MAPVFS_CONFIG env override (test/characterization seam - an +# alternate config exercised without editing the real one; relative paths +# resolve against the runtime folder), then mapvfs.toml, then the deprecated +# mapvfs.config. When neither exists the toml name is returned so no-mapping +# messages name the current format. +proc ::punkboot::lib::mapvfs_locate {rt_sourcefolder} { + if {[info exists ::env(PUNK_MAPVFS_CONFIG)] && $::env(PUNK_MAPVFS_CONFIG) ne ""} { + set override $::env(PUNK_MAPVFS_CONFIG) + if {[file pathtype $override] ne "absolute"} { + set override [file join $rt_sourcefolder $override] + } + return $override + } + if {[file exists $rt_sourcefolder/mapvfs.toml]} { + return $rt_sourcefolder/mapvfs.toml + } + if {[file exists $rt_sourcefolder/mapvfs.config]} { + return $rt_sourcefolder/mapvfs.config + } + return $rt_sourcefolder/mapvfs.toml +} + +#tomlish::to_dict values are type-tagged (e.g {type STRING value x}, {type ARRAY value {...}}); +#tables are plain nested dicts. Shared by the mapvfs toml reader, the per-.vfs +#payload declarations (G-115) and any other build-side toml consumption. +proc ::punkboot::lib::toml_untag {tagged} { + if {[dict get $tagged type] eq "ARRAY"} { + set plain [list] + foreach el [dict get $tagged value] { + lappend plain [::punkboot::lib::toml_untag $el] + } + return $plain + } + return [dict get $tagged value] +} + +# Read + parse a toml file via tomlish. Returns {ok } or {error }. +proc ::punkboot::lib::toml_file_to_dict {path} { + if {[catch {package require tomlish} tomlish_err]} { + return [list error "tomlish package unavailable ($tomlish_err)"] + } + if {[catch { + set fd [open $path r] + fconfigure $fd -encoding utf-8 + set tomldata [read $fd] + close $fd + } readerr]} { + catch {close $fd} + return [list error "cannot read $path ($readerr)"] + } + if {[catch {tomlish::to_dict [tomlish::from_toml $tomldata]} parsed]} { + return [list error "toml parse failed for $path\n$parsed"] + } + return [list ok $parsed] +} + +# attrs element of a parsed vfs_spec tuple (6th element, toml models). Legacy +# models carry no attrs - callers get an empty dict and defaults apply +# (group "", bake_default 1, no scheme role). +proc ::punkboot::lib::mapvfs_spec_attrs {vfs_spec} { + if {[llength $vfs_spec] >= 6} { + return [lindex $vfs_spec 5] + } + return [dict create] +} + +proc ::punkboot::lib::mapvfs_model_new {mapfile default_target format} { + return [dict create\ exists 0\ mapfile $mapfile\ + format $format\ default_target $default_target\ runtime_vfs_map [dict create]\ vfs_runtime_map [dict create]\ runtime_target [dict create]\ + groups [dict create]\ missing [list]\ + missing_detail [list]\ warnings [list]\ badentries [list]\ configerrors [list]\ ] +} + +proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} { + if {[string tolower [file extension $mapfile]] eq ".toml"} { + return [mapvfs_parse_toml $mapfile $rtbase $sourcefolder $default_target] + } + return [mapvfs_parse_config $mapfile $rtbase $sourcefolder $default_target] +} + +# G-024 canonical reader: mapvfs.toml. Strict per-entry validation - a malformed +# or unresolvable declaration lands in configerrors naming its table (the bake +# path aborts on configerrors; missing-on-disk runtime/vfs items land in +# warnings + missing_detail and the bake path decides fatality per entry). +proc ::punkboot::lib::mapvfs_parse_toml {mapfile rtbase sourcefolder default_target} { + set model [mapvfs_model_new $mapfile $default_target toml] + if {![file exists $mapfile]} { + return $model + } + dict set model exists 1 + set legacy_sibling [file join [file dirname $mapfile] mapvfs.config] + if {[file exists $legacy_sibling] && [file tail $mapfile] eq "mapvfs.toml"} { + dict lappend model warnings "NOTE: both mapvfs.toml and mapvfs.config present in [file dirname $mapfile] - mapvfs.toml wins and the deprecated mapvfs.config is ignored (remove it, or keep it only as reference)." + } + lassign [toml_file_to_dict $mapfile] tstatus tpayload + if {$tstatus ne "ok"} { + dict lappend model configerrors "CONFIG FILE ERROR. $mapfile: $tpayload" + return $model + } + set kit_types [list kit zip zipcat cookit cookfs] + set groups [dict create] + set configerrors [list] + set warnings [list] + set missing [list] + set missing_detail [list] + #top-level tables: kit / group / scheme only + dict for {topkey -} $tpayload { + if {$topkey ni {kit group scheme}} { + lappend configerrors "CONFIG FILE ERROR. $mapfile: unknown top-level table or key '$topkey' (expected \[kit.\], \[group.\], \[scheme.\])" + } + } + #-- groups + if {[dict exists $tpayload group]} { + dict for {gname gentry} [dict get $tpayload group] { + set label "group.$gname" + set gdesc "" + set gdefault 1 + dict for {k v} $gentry { + switch -- $k { + description {set gdesc [toml_untag $v]} + bake_default { + set gdefault [toml_untag $v] + if {![string is boolean -strict $gdefault]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: 'bake_default' must be boolean (got '$gdefault')" + set gdefault 1 + } + } + default { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown key '$k' (expected description, bake_default)" + } + } + } + dict set groups $gname [dict create description $gdesc bake_default [expr {bool($gdefault)}]] + } + } + #-- normalized candidate entries: {label rtname vfstail appname kittype declared_target smokes attrs} + set entries [list] + set appnames_seen [dict create] ;#appname -> label + set entry_reader {{model_varname label entry required_type} { + #shared [kit.*]/[scheme.*] field reader; returns a dict or empty on fatal field errors + upvar 1 configerrors configerrors kit_types kit_types groups groups mapfile mapfile + set fields [dict create runtime "" vfs "" type "" target "" smokerequire [list] group "" bake_default "" scheme "" prefix ""] + set badentry 0 + dict for {k v} $entry { + switch -- $k { + runtime - vfs - type - target - group - scheme - prefix { + dict set fields $k [::punkboot::lib::toml_untag $v] + } + smokerequire { + if {[dict get $v type] eq "ARRAY"} { + dict set fields smokerequire [::punkboot::lib::toml_untag $v] + } else { + dict set fields smokerequire [list [::punkboot::lib::toml_untag $v]] + } + } + bake_default { + set bd [::punkboot::lib::toml_untag $v] + if {![string is boolean -strict $bd]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: 'bake_default' must be boolean (got '$bd')" + set badentry 1 + } else { + dict set fields bake_default [expr {bool($bd)}] + } + } + default { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown key '$k'" + set badentry 1 + } + } + } + foreach reqkey {runtime vfs} { + if {[dict get $fields $reqkey] eq ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile is missing required key '$reqkey'" + set badentry 1 + } + } + set ktype [dict get $fields type] + if {$ktype eq ""} { + if {$required_type} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile is missing required key 'type' (scheme entries declare their kit type explicitly; one of: [join $kit_types {, }])" + set badentry 1 + } else { + dict set fields type kit + } + } elseif {$ktype ni $kit_types} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown kit type '$ktype' (expected one of: [join $kit_types {, }])" + set badentry 1 + } + set declared [dict get $fields target] + if {$declared ne ""} { + set problem [::punkboot::lib::platform_name_problem $declared] + if {$problem ne ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile declares target platform '$declared' - $problem." + set badentry 1 + } + } + #strip a superfluous windows .exe from the runtime name (legacy-format parity) + set rtname [dict get $fields runtime] + if {[string match -nocase *.exe $rtname]} { + dict set fields runtime [string range $rtname 0 end-4] + } + set g [dict get $fields group] + if {$g ne "" && ![dict exists $groups $g]} { + #a group referenced without its own [group.] table is implicitly plain + dict set groups $g [dict create description "" bake_default 1] + } + if {$badentry} { + return [dict create] + } + return $fields + }} + if {[dict exists $tpayload kit]} { + dict for {kname kentry} [dict get $tpayload kit] { + set label "kit.$kname" + set fields [apply $entry_reader model $label $kentry 0] + if {![dict size $fields]} { + continue + } + foreach badkey {scheme prefix} { + if {[dict get $fields $badkey] ne ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: key '$badkey' is only valid in \[scheme.\] entries" + } + } + set g [dict get $fields group] + set bd [dict get $fields bake_default] + if {$bd eq ""} { + set bd [expr {$g ne "" ? [dict get $groups $g bake_default] : 1}] + } + if {[dict exists $appnames_seen $kname]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: output name '$kname' already produced by [dict get $appnames_seen $kname]" + continue + } + dict set appnames_seen $kname $label + lappend entries [list $label [dict get $fields runtime] [dict get $fields vfs] $kname\ + [dict get $fields type] [dict get $fields target] [dict get $fields smokerequire]\ + [dict create entry $label group $g bake_default $bd scheme "" scheme_role ""]] + } + } + #-- schemes (G-023/G-024: generative entries expanded at parse time - no version enumeration) + set project_version "" + set project_version_err "" + if {[dict exists $tpayload scheme]} { + dict for {sname sentry} [dict get $tpayload scheme] { + set label "scheme.$sname" + set fields [apply $entry_reader model $label $sentry 1] + if {![dict size $fields]} { + continue + } + set skind [dict get $fields scheme] + if {$skind eq ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile is missing required key 'scheme' (known schemes: versioned)" + continue + } + if {$skind ne "versioned"} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown scheme '$skind' (known schemes: versioned)" + continue + } + #versioned scheme (G-023): output names derive from the punkproject.toml + #project version at parse time - punk-, punk-dev and the + #release-gated plain punk - so one declaration tracks the version with + #no per-version config edits. + if {$project_version eq "" && $project_version_err eq ""} { + set pptoml [file join [file dirname $sourcefolder] punkproject.toml] + lassign [toml_file_to_dict $pptoml] pstatus ppayload + if {$pstatus ne "ok"} { + set project_version_err $ppayload + } elseif {[catch {toml_untag [dict get $ppayload project version]} pver]} { + set project_version_err "no \[project\] version field readable in $pptoml ($pver)" + } elseif {$pver eq ""} { + set project_version_err "empty \[project\] version field in $pptoml" + } else { + set project_version $pver + } + } + if {$project_version eq ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: cannot expand 'versioned' scheme - $project_version_err" + continue + } + set prefix [dict get $fields prefix] + if {$prefix eq ""} { + set prefix $sname + } + set g [dict get $fields group] + set bd [dict get $fields bake_default] + if {$bd eq ""} { + set bd [expr {$g ne "" ? [dict get $groups $g bake_default] : 1}] + } + foreach {appname role} [list ${prefix}-$project_version versioned ${prefix}-dev dev $prefix release] { + if {[dict exists $appnames_seen $appname]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: expanded output name '$appname' already produced by [dict get $appnames_seen $appname]" + continue + } + dict set appnames_seen $appname $label + lappend entries [list "$label ($role)" [dict get $fields runtime] [dict get $fields vfs] $appname\ + [dict get $fields type] [dict get $fields target] [dict get $fields smokerequire]\ + [dict create entry $label group $g bake_default $bd scheme $sname scheme_role $role]] + } + } + } + #-- per-runtime target resolution (target is a property of the RUNTIME - entries + #may repeat the declaration but must not disagree; legacy-format parity) + set runtime_target [dict create] + set runtime_first_label [dict create] + foreach e $entries { + lassign $e label rtname - - - declared - - + if {![dict exists $runtime_first_label $rtname]} { + dict set runtime_first_label $rtname $label + } + if {$declared eq ""} { + continue + } + if {![dict exists $runtime_target $rtname] || [dict get $runtime_target $rtname] eq $default_target} { + dict set runtime_target $rtname $declared + } elseif {[dict get $runtime_target $rtname] ne $declared} { + lappend configerrors "CONFIG FILE ERROR. runtime: $rtname declares conflicting target platforms ('[dict get $runtime_target $rtname]' and '$declared') in $mapfile - a runtime has one target." + } + } + foreach rtname [dict keys $runtime_first_label] { + if {![dict exists $runtime_target $rtname]} { + dict set runtime_target $rtname $default_target + } + } + #-- runtime presence (one check per runtime, in first-reference order) + foreach rtname [dict keys $runtime_first_label] { + if {$rtname eq "-"} { + continue + } + set rt_target [dict get $runtime_target $rtname] + set rtdir [file join $rtbase [platform_store_tier $rt_target]] + set runtime_test $rtname[platform_exe_suffix $rt_target] + if {![file exists [file join $rtdir $runtime_test]]} { + lappend warnings "WARNING: Missing runtime file $rtdir/$runtime_test (entry [dict get $runtime_first_label $rtname] in $mapfile)" + lappend missing $rtname + lappend missing_detail [dict create kind runtime label [dict get $runtime_first_label $rtname] name $rtname target $rt_target] + } + } + #-- build the maps (entry order preserved; vfs existence filtering matches the + #historical inline parse the kit loop was built around) + set runtime_vfs_map [dict create] + set vfs_runtime_map [dict create] + foreach e $entries { + lassign $e label rtname vfstail appname kittype - smokes attrs + set rt_target [dict get $runtime_target $rtname] + set spec [list $vfstail $appname $kittype $rt_target $smokes $attrs] + dict lappend runtime_vfs_map $rtname $spec + if {![file isdirectory [file join $sourcefolder vfs $vfstail]]} { + lappend warnings "WARNING: Missing vfs folder [file join $sourcefolder vfs $vfstail] specified in $mapfile for entry $label" + if {$vfstail ni $missing} { + lappend missing $vfstail + } + lappend missing_detail [dict create kind vfs label $label name $vfstail target $rt_target] + } else { + dict lappend vfs_runtime_map $vfstail [list $rtname $appname $kittype $rt_target $smokes $attrs] + } + } + dict set model runtime_vfs_map $runtime_vfs_map + dict set model vfs_runtime_map $vfs_runtime_map + dict set model runtime_target $runtime_target + dict set model groups $groups + dict set model missing $missing + dict set model missing_detail $missing_detail + dict set model warnings [concat [dict get $model warnings] $warnings] + dict set model configerrors $configerrors + return $model +} + +# DEPRECATED legacy reader: the pre-G-024 mapvfs.config line format +# ( {vfsfolder ?kitname? ?kittype? ?targetplatform? ?smokerequires?} ...). +# Kept so un-migrated trees (older generated projects) keep baking; behaviour is +# unchanged from before the toml conversion apart from the deprecation NOTE. +proc ::punkboot::lib::mapvfs_parse_config {mapfile rtbase sourcefolder default_target} { + set model [mapvfs_model_new $mapfile $default_target legacy] if {![file exists $mapfile]} { return $model } dict set model exists 1 + dict lappend model warnings "NOTE: $mapfile uses the deprecated mapvfs.config line format - convert to mapvfs.toml (the file header of src/runtime/mapvfs.toml in the punkshell tree is the format spec; when both files exist the toml file wins)." set fdmap [open $mapfile r] fconfigure $fdmap -translation binary set mapdata [read $fdmap] @@ -765,7 +1153,7 @@ proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} dict set model vfs_runtime_map $vfs_runtime_map dict set model runtime_target $runtime_target dict set model missing $missing - dict set model warnings $warnings + dict set model warnings [concat [dict get $model warnings] $warnings] dict set model badentries $badentries dict set model configerrors $configerrors return $model @@ -778,8 +1166,10 @@ proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} # TARGET .exe suffix, duplicate appname -> _) so listed names # match what a bake produces. Entries whose vfs folder is missing on disk never # reach the kit loop - they are appended after the buildable set so 'bakelist' -# can surface them as broken config. Duplicate-name disambiguation assumes the -# config keeps appnames unique (as the kit loop itself effectively does). +# can surface them as broken config. Duplicate-name disambiguation is per TARGET +# (G-127): same-named kits for different targets coexist under their own output +# tiers; a same-target duplicate assumes the config keeps appnames unique (as +# the kit loop itself effectively does). # $rtbase is the store ROOT (bin/runtime): each record resolves its own tier # under it, so entries targeting different platforms report their own store # presence (G-122). Returns a list of record dicts: @@ -792,9 +1182,18 @@ proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} # runtime_dir store tier folder holding the runtime ("" for "-") # target target platform (e.g win32-x86_64) # store_tier bin/runtime tier folder name for the target +# out_tier kit OUTPUT tier relative to bin/ and src/_build/ - "" for the +# default (host) target's flat locations, "kits/" for any +# other target (G-127) # vfs vfs folder tail (e.g punk9wintk903.vfs) # vfs_present 1|0 # smokerequire G-133 smoke-require package list as configured ("" when undeclared) +# entry config table label (toml models, e.g kit.punk91 / "scheme.punk9 (dev)"; "" legacy) +# group G-024 named grouping ("" when ungrouped) +# bake_default 1|0 - 0 means excluded from full (no-name) bakes; still bakeable +# by name or @group selection +# scheme / scheme_role scheme-expanded outputs: source scheme name and role +# (versioned|dev|release); "" for explicit entries proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { set runtime_vfs_map [dict get $model runtime_vfs_map] set vfs_runtime_map [dict get $model vfs_runtime_map] @@ -808,7 +1207,7 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { } set records [list] set exe_names_seen [list] - set record_pair {{rtname vfstail appname target_kit_type smokerequires} { + set record_pair {{rtname vfstail appname target_kit_type smokerequires attrs} { upvar 1 records records exe_names_seen exe_names_seen rtbase rtbase sourcefolder sourcefolder\ runtime_target runtime_target default_target default_target if {$appname eq ""} { @@ -834,14 +1233,16 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { set runtime_file $rtname[::punkboot::lib::platform_exe_suffix $target] set targetkit ${appname}[::punkboot::lib::platform_exe_suffix $target] set kitname $appname - if {$targetkit in $exe_names_seen} { - #duplicate appname configured - kit loop disambiguates the same way + if {[list $target $targetkit] in $exe_names_seen} { + #duplicate appname configured for the same target - kit loop + #disambiguates the same way (G-127: different-target namesakes + #coexist under their own kits// output tiers instead) set targetkit ${appname}_$rtname set kitname ${appname}_$rtname } set runtime_present [file exists [file join $runtime_dir $runtime_file]] } - lappend exe_names_seen $targetkit + lappend exe_names_seen [list $target $targetkit] lappend records [dict create\ kitname $kitname\ targetkit $targetkit\ @@ -852,17 +1253,29 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { runtime_dir $runtime_dir\ target $target\ store_tier $store_tier\ + out_tier [expr {$target eq $default_target ? {} : "kits/$target"}]\ vfs $vfstail\ vfs_present [file isdirectory [file join $sourcefolder vfs $vfstail]]\ smokerequire $smokerequires\ + entry [expr {[dict exists $attrs entry] ? [dict get $attrs entry] : ""}]\ + group [expr {[dict exists $attrs group] ? [dict get $attrs group] : ""}]\ + bake_default [expr {[dict exists $attrs bake_default] ? [dict get $attrs bake_default] : 1}]\ + scheme [expr {[dict exists $attrs scheme] ? [dict get $attrs scheme] : ""}]\ + scheme_role [expr {[dict exists $attrs scheme_role] ? [dict get $attrs scheme_role] : ""}]\ ] }} foreach vfstail $vfs_tails { if {![dict exists $vfs_runtime_map $vfstail]} { continue ;#unmapped vfs folder - the kit loop derives no output names from it } + set rtnames_done [list] ;#several entries may share a runtime on one vfs (toml models) - + #enumerate each distinct runtime's spec set exactly once foreach rt_app [dict get $vfs_runtime_map $vfstail] { set rtname [lindex $rt_app 0] + if {$rtname in $rtnames_done} { + continue + } + lappend rtnames_done $rtname if {![dict exists $runtime_vfs_map $rtname]} { continue } @@ -871,58 +1284,85 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { if {$configured_vfs ne $vfstail} { continue } - apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires + apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires [::punkboot::lib::mapvfs_spec_attrs $vfs_app] } } } #entries whose vfs folder is missing (excluded from vfs_runtime_map by the parse) dict for {rtname vfs_specs} $runtime_vfs_map { foreach vfsconfig $vfs_specs { - if {[llength $vfsconfig] < 1 || [llength $vfsconfig] > 5} { + if {[llength $vfsconfig] < 1 || [llength $vfsconfig] > 6} { continue } lassign $vfsconfig vfstail appname target_kit_type - smokerequires if {[dict exists $vfs_runtime_map $vfstail]} { continue ;#already enumerated above } - apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires + apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires [::punkboot::lib::mapvfs_spec_attrs $vfsconfig] } } return $records } # Match user-supplied kit names against mapvfs_kit_outputs records. -# Accepts the base kit name (punk91) or the platform artifact name (punk91.exe, -# name.kit); matching is case-insensitive when the HOST filesystem is (a name -# the user types is a host-side convenience - a cygwin-family host is as +# Accepts the base kit name (punk91), the platform artifact name (punk91.exe, +# name.kit), or a G-024 group selector @ (selects every record in the +# group). A base name matching several records - one per target, coexisting under +# their own output tiers (G-127) - selects them all. +# Matching is case-insensitive when the HOST filesystem is (a name the +# user types is a host-side convenience - a cygwin-family host is as # case-insensitive as a native windows one, whatever it targets). Returns a dict: # selected matched records (kit-output order, deduplicated) # unknown supplied names that matched nothing proc ::punkboot::lib::mapvfs_match_outputs {kit_outputs names} { set on_windows [expr {[info exists ::punkboot::host_windows] ? $::punkboot::host_windows : ("windows" eq $::tcl_platform(platform))}] + set name_eq {{on_windows a b} { + expr {$on_windows ? [string equal -nocase $a $b] : [string equal $a $b]} + }} set selected_idx [list] set unknown [list] foreach name $names { + set found 0 + if {[string match @?* $name]} { + #group selector: every record carrying the group (G-024) + set gname [string range $name 1 end] + set i 0 + foreach rec $kit_outputs { + if {[dict get $rec group] ne "" && [apply $name_eq $on_windows [dict get $rec group] $gname]} { + set found 1 + if {$i ni $selected_idx} { + lappend selected_idx $i + } + } + incr i + } + if {!$found} { + lappend unknown $name + } + continue + } set base $name if {[string match -nocase *.exe $base] || [string match -nocase *.kit $base]} { set base [string range $base 0 end-4] } - set found 0 set i 0 foreach rec $kit_outputs { + set rec_match 0 foreach candidate [list [dict get $rec kitname] [dict get $rec targetkit]] { if {$candidate eq ""} {continue} - if {($on_windows && [string equal -nocase $candidate $base]) || (!$on_windows && $candidate eq $base)\ - || ($on_windows && [string equal -nocase $candidate $name]) || (!$on_windows && $candidate eq $name)} { - set found 1 + if {[apply $name_eq $on_windows $candidate $base] || [apply $name_eq $on_windows $candidate $name]} { + set rec_match 1 break } } - if {$found} { + if {$rec_match} { + #G-127: one kit NAME may legitimately match several records - one per + #target, coexisting under their own output tiers - so a name selects + #them ALL (like the @group branch), not just the first hit + set found 1 if {$i ni $selected_idx} { lappend selected_idx $i } - break } incr i } @@ -2110,7 +2550,7 @@ proc ::punkboot::punkboot_gethelp {args} { append h " - includes the vfslibs phase; does not rebuild modules/libs and does not run the promotion gates" \n append h " - with kitname arguments, bakes and deploys only the named configured kits (unknown names error before any build)" \n \n append h " $scriptname bakelist ?kitname ...?" \n - append h " - list the kit outputs configured in src/runtime/mapvfs.config: name, kit type, runtime (with presence)," \n + append h " - list the kit outputs configured in src/runtime/mapvfs.toml: name, kit type, runtime (with presence)," \n append h " vfs folder and deployed state (bin copy absent/current/stale/nobuild vs the src/_build product)" \n append h " - kitname arguments filter to the named entries (per-kit detail)" \n \n append h " $scriptname modules" \n @@ -2163,9 +2603,6 @@ proc ::punkboot::punkboot_gethelp {args} { append h " reported and build/test exit nonzero with fetch guidance (bin/punk-getzig.cmd)" \n \n append h " $scriptname help ?subcommand?" \n append h " - show usage for $scriptname or one of its subcommands (equivalent: $scriptname -help)" \n \n - append h " Deprecated aliases (transition window):" \n - append h " $scriptname project -> runs bakehouse (with a one-line notice)" \n - append h " $scriptname vfs -> runs bake (with a one-line notice)" \n \n append h " Flags:" \n append h " -confirm 0|1" \n append h " - interactive y/n confirmation policy (default 1: prompt when stdin is a terminal, abort fast when it is not)." \n @@ -2209,7 +2646,6 @@ gates, one bake) recipe; -dirty-abort 0 overrides). DEVELOPER (changing payload): pass the gates explicitly - the commit is the publishing act, not a build step. That is the RELEASE SEQUENCE below. - Deprecated aliases (transition window): 'project' -> bakehouse, 'vfs' -> bake. RELEASE SEQUENCE (the developer order that satisfies the staleness + provenance gates) ---------------------------------------------------------------------------- @@ -2278,19 +2714,32 @@ DIAGRAM 1b - THIN-LAYOUT SYNC (part of modules/libs/packages/bakehouse runs) DIAGRAM 2 - KIT ASSEMBLY DETAIL (the 'make.tcl bake' stage; incl. vfslibs phase) ---------------------------------------------------------------------------- - src/runtime/mapvfs.config which vfs folder pairs with which runtime -> kit name, - and optionally the TARGET PLATFORM of that runtime [K9] - 'make.tcl bakelist ?kitname ...?' reports the configured - kit outputs (runtime/vfs presence + deployed state); - 'make.tcl bake ?kitname ...?' bakes only the named kits - (bare bake = all; unknown names error before any build) + src/runtime/mapvfs.toml which vfs folder pairs with which runtime -> kit name + ([kit.] tables; tomlish-parsed - G-024), incl. the + runtime's TARGET PLATFORM [K9], smoke-require packages, + named groups and generative version-named schemes + (deprecated mapvfs.config line format still readable) + 'make.tcl bakelist ?name|@group ...?' reports the + configured kit outputs (presence + deployed state); + 'make.tcl bake ?name|@group ...?' bakes only the named + kits (bare bake = all entries except bake_default=false + ones; unknown names error before any build) bin/runtime// bare Tcl runtime (tclkit / tclsfe / suite build), in the store tier of the TARGET platform it builds kits for [K9] src/vfs/_vfscommon.vfs/ common payload (built modules + libs, main boot support) src/vfs/.vfs/ per-kit overlay (main.tcl, kit-only modules/config) - - _vfscommon.vfs .vfs - | | + src/vfs/.vfs.toml optional per-.vfs payload declaration (G-115): [payload.*] + entries materialized INTO the .vfs folder by the vfslibs + phase (vendor trees / bin/packages tier), punkcheck- + tracked with drop-in-wins precedence - undeclared or + hand-modified files are preserved, never overwritten. + A source containing %platform% materializes once per + CONSUMING KIT TARGET (set derived from mapvfs) into + _targets// staging; each kit's merged image + then gets only its own target's subtree (G-127) + + _vfscommon.vfs .vfs <-- payload declarations (if any) already + | | materialized by the vfslibs phase +-----------+----------+ | copy common, then merge overlay over it v @@ -2349,7 +2798,8 @@ LIBRARY-ARTIFACT FETCH + CONSUMPTION (G-139) | consumption (two lanes, both punkcheck-tracked): | PACKAGES_tcl phase of modules/libs/packages/bakehouse | -> lib_tcl/allplatforms + lib_tcl/; - | vfslibs entries with source_root = "packages" + | per-.vfs payload declarations (src/vfs/.vfs.toml, + | G-115) with source_root = "packages" | -> declared kit vfs lib_tcl trees (see DIAGRAM 2) v lib_tcl/... + src/vfs/.vfs/lib_tcl/... (embedded records ride along @@ -2401,7 +2851,9 @@ KEY / NOTES [K8] What is VCS-tracked where (checkin targets after a build): tracked: src/** (sources, src/bootsupport, src/project_layouts copies, src/vfs/_vfscommon.vfs), punkproject.toml, CHANGELOG.md - untracked: modules*/ lib*/ at projectroot, src/_build/, bin/.exe + untracked: modules*/ lib*/ at projectroot, src/_build/, bin/.exe, + bin/kits// (cross-target kit outputs - G-127), + src/vfs/*.vfs/_targets/ (per-platform payload staging) (bin/ scripts/tools are tracked; kit executables are not) [K9] HOST vs TARGET platform (G-122). What a bake EMITS is keyed by the target @@ -2410,11 +2862,18 @@ KEY / NOTES whether the pre-deploy sweep uses tasklist/taskkill or ps/kill. The default target is this host's own platform - except a cygwin-family host (an msys2/cygwin-runtime tclsh, which reports platform 'unix' on windows), which - targets win32. A mapvfs.config entry may declare its own target as a 4th - element and is then read from that platform's tier, named with its executable - convention, and (being unrunnable here) skipped by the process sweep. + targets win32. A mapvfs.toml entry may declare its own target (target key; + legacy config: 4th element) and is then read from that platform's tier, named + with its executable convention, and (being unrunnable here) skipped by the + process sweep. 'make.tcl check' prints the derivation; 'make.tcl bakelist' shows per-kit targets, with a target= note on rows that are not on the default target. + OUTPUT locations are target-keyed too (G-127): default-target kits keep the + flat src/_build/ + bin/ locations, any other target's kit builds + and deploys under the kits// tier of both (bin/kits/ = outputs, + beside bin/runtime/ = inputs), so same-named kits for different targets + coexist instead of overwriting each other; bakelist marks such rows with + out=kits// and its detail block prints the tiered paths. [K10] BOOT-PRECONDITION gate (G-125). A kit whose merged vfs has no tcl library cannot initialise at all ('Cannot find a usable init.tcl'), so the bake @@ -2440,8 +2899,8 @@ KEY / NOTES cross-target kits are covered) against the kit's target platform; wrong-arch libraries outside platform-discriminated subdirs (win32-ix86/, win-x64/ etc - multi-arch payloads are legitimate there) earn recapped BUILD-WARNINGs. - (b) Smoke-require probe: a kit declaring packages (mapvfs.config 5th entry - element) has each one plain-'package require'd INSIDE the freshly built + (b) Smoke-require probe: a kit declaring packages (mapvfs.toml smokerequire + key; legacy config: 5th element) has each one plain-'package require'd INSIDE the freshly built artifact via its tclsh subcommand - the only check that sees resolution-order defects (a wrong-arch higher-versioned package shadowing a working copy); cross-target kits skip with a stated reason, undeclared kits run nothing new. @@ -2556,8 +3015,8 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO modules "Build .tm modules from src/modules, src/vendormodules etc into /modules etc" libs "Build pkgIndex.tcl libraries from src/lib, src/vendorlib etc into /lib etc" bake "Assemble kit/zipkit executables from promoted payload (src/vfs) and runtimes into /bin" - bakelist "List the kit outputs configured in src/runtime/mapvfs.config: type, runtime/vfs presence, deployed state" - vfslibs "Propagate declared platform-library packages (vendored or bin/packages tier) into kit vfs lib_tcl trees" + bakelist "List the kit outputs configured in src/runtime/mapvfs.toml: type, runtime/vfs presence, deployed state, groups" + vfslibs "Materialize per-.vfs payload declarations (src/vfs/.vfs.toml - G-115) into kit vfs folders; drop-ins preserved" bin "Install executables from src/bin into /bin, then build kits as for bake" vendorupdate "Update src/vendormodules based on src/vendormodules/include_modules.config" libfetch "Fetch declared punkbin lib-tier library artifacts into bin/packages/ (sha1-verified) and materialize the package trees" @@ -2571,8 +3030,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO help "Show usage for make.tcl or one of its subcommands" buildsuite "List, describe or run the defined buildsuites under src/buildsuites (zig runtime factory)" tool "List, test or build the vendored tools under src/tools (zig) and install them to /bin" - project "DEPRECATED alias for bakehouse (one-line notice, then runs bakehouse)" - vfs "DEPRECATED alias for bake (one-line notice, then runs bake)" } #@cmd -help bodies for the constructed definitions below. Authored BLOCK-FORM #(first value line whitespace-only) so the @normalize directive in each @@ -2620,25 +3077,28 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO build each configured kit from the promoted payload (src/vfs/_vfscommon.vfs + custom vfs folders) and its runtime, and install to /bin. - With kitname arguments, bake and deploy ONLY the named configured - kits (see 'make.tcl bakelist' for the configured names) - other - kits' build products, bin copies and punkcheck records are left - untouched. An unknown name errors before any build, listing the - configured names. Bare 'bake' processes all configured kits. - Includes the vfslibs phase (declared vendored platform-library - propagation into kit vfs lib_tcl trees) so a bake cannot ship - stale binary libs; a selective bake narrows that phase to the - named kits' vfs folders. + With kitname (or @groupname - G-024 groups) arguments, bake and + deploy ONLY the named configured kits (see 'make.tcl bakelist' + for the configured names and groups) - other kits' build + products, bin copies and punkcheck records are left untouched. + An unknown name errors before any build, listing the configured + names. Bare 'bake' processes all configured kits except + bake_default=false entries (those bake only by name/@group). + Includes the vfslibs phase (per-.vfs payload declarations + src/vfs/.vfs.toml materialized into the kit vfs folders - + G-115) so a bake cannot ship stale binary libs; a selective + bake narrows that phase to the named kits' vfs folders. Does not rebuild modules/libs and does not run the promotion gates - built modules reach the kit payload only via 'make.tcl vfscommonupdate' (committed for provenance). Consumer full path from a clean checkout: 'make.tcl bakehouse'." bakelist " - List the kit outputs configured in src/runtime/mapvfs.config (the - same parsed mapping the bake subcommand consumes): kit name, kit - type (kit|zip|zipcat...), runtime with presence in the runtime - store of that kit's TARGET platform + List the kit outputs configured in src/runtime/mapvfs.toml (the + same parsed mapping the bake subcommand consumes; the deprecated + mapvfs.config line format is honoured when no toml file exists): + kit name, kit type (kit|zip|zipcat...), runtime with presence in + the runtime store of that kit's TARGET platform (/bin/runtime/), vfs folder, and the deployed state of /bin/ vs the src/_build build product: @@ -2651,20 +3111,31 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO materialized from an older revision than a -r artifact beside it) and marks cross-target entries (target= on any kit whose mapvfs entry declares a target other than this host's - default - 'make.tcl check' shows that default). - kitname arguments filter the report to the named entries (per-kit - detail, with resolved store tier, target and paths). Reporting - only - never builds." + default - 'make.tcl check' shows that default), G-024 groups + (group=), entries excluded from full bakes (default=no) + and scheme-expanded outputs (scheme=versioned|dev|release). + Non-default-target kits build and deploy under the + kits// tier of src/_build and bin (G-127) - their rows + carry out=kits// and the deployed state compares the + tiered paths; a kit NAME shared by entries for several targets + selects (and lists) every one of them. + kitname (or @groupname) arguments filter the report to the named + entries (per-kit detail, with resolved store tier, target, + config entry and paths). Reporting only - never builds." vfslibs " - Propagate declared platform-library packages into kit vfs - lib_tcl trees - from vendored src/ folders (source_root - 'src', the default) or from the bin/packages punkbin lib-tier - input (source_root 'packages', populated by 'make.tcl - libfetch' - G-139). - Declarations (per-package per-kit, with superseded-version - removal and clean-slate same-name 'replace') live in - src/runtime/vendorlib_vfs.toml. + Materialize the per-.vfs payload declarations (G-115): each + src/vfs/.vfs.toml declares package folders to install + INTO src/vfs/.vfs - from vendored src/ folders + (source_root 'src', the default) or from the bin/packages + punkbin lib-tier input (source_root 'packages', populated by + 'make.tcl libfetch' - G-139) - with explicit superseded-version + removal and clean-slate same-name 'replace'. + punkcheck-tracked with drop-in-wins precedence: files the + mechanism did not install (or that changed since it installed + them) are preserved and reported, never overwritten; a .vfs + with no declaration file is untouched. Format + precedence: + src/vfs/README.md. Also runs automatically as a phase of 'make.tcl bake' (and therefore of 'make.tcl bakehouse'); this standalone narrowing serves quick iteration on the declarations." @@ -2806,16 +3277,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO (bin/punk-getzig.cmd). Build caches (.zig-cache/zig-out) are deliberately left in place between runs for rebuild speed (both git- and fossil-ignored)." - project - " - DEPRECATED alias for 'bakehouse' (G-112 stage-true rename) - prints a - one-line notice and runs bakehouse (packages + bake, -dirty-abort - default ON). Retained for a transition window." - vfs - " - DEPRECATED alias for 'bake' (G-112 stage-true rename) - prints a - one-line notice and runs bake (kit assembly including the vfslibs - phase). Retained for a transition window." } #shared option fragments composed into the per-subcommand definitions below variable OPT_CONFIRM { @@ -2849,7 +3310,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "Terminate running processes matching the executable being built (if applicable) so the build can install over it." } - #bakehouse (and its deprecated alias project) defaults the dirty gate ON: + #bakehouse defaults the dirty gate ON: #the bakehouse bakes from the committed recipe (G-112 two-persona model). variable OPT_DIRTYABORT_ON { -dirty-abort -type boolean -default 1 -help\ @@ -2904,8 +3365,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO check {} projectversion {} workflow {} - project {OPT_FORCEKILL OPT_DIRTYABORT_ON OPT_CONFIRM} - vfs {OPT_FORCEKILL OPT_DIRTYABORT OPT_CONFIRM} } #synopsis fragments matching the shared option fragments (explicit @form -synopsis #so usage shows the real invocation 'make.tcl ...' rather than the @@ -2919,18 +3378,22 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO OPT_TRUSTSERVER "?-trust-server?" OPT_FORCE "?-force?" } - #per-subcommand positional values (default: none). bake (and its deprecated - #alias vfs) and bakelist take optional kit names (G-121 selective bake). + #per-subcommand positional values (default: none). bake and bakelist take + #optional kit names (G-121 selective bake). #Discoverability nicety: the configured kit names are offered as choices when #the mapping parses at definition time - informational only (-choicerestricted #0), so the handlers' own validation (which lists the configured names, and #still works when the mapping cannot be parsed here) stays authoritative. variable KITNAME_CHOICES [list] catch { - set _mapmodel [punkboot::lib::mapvfs_parse [file join $::punkboot::scriptfolder runtime mapvfs.config] "" $::punkboot::scriptfolder $::punkboot::target_platform] + set _mapmodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate [file join $::punkboot::scriptfolder runtime]] "" $::punkboot::scriptfolder $::punkboot::target_platform] foreach _rec [punkboot::lib::mapvfs_kit_outputs $_mapmodel "" $::punkboot::scriptfolder] { lappend KITNAME_CHOICES [dict get $_rec kitname] } + #G-024 group selectors join the informational choices (@) + foreach _g [dict keys [dict get $_mapmodel groups]] { + lappend KITNAME_CHOICES @$_g + } } set _kitname_choicepart "" if {[llength $KITNAME_CHOICES]} { @@ -2941,22 +3404,23 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "@values -min 0 -max -1"\ "kitname -type string -optional 1 -multiple 1${_kitname_choicepart} -help\ { - Configured kit output(s) to bake and deploy - see 'make.tcl bakelist' - for the configured names. With no kitname, all configured kits are - processed. An unknown name errors before any build.}"\ + Configured kit output(s) to bake and deploy - a kit name, or + @ for every kit in a configured group (G-024) - see + 'make.tcl bakelist' for the configured names and groups. With no + kitname, all configured kits are processed except entries with + bake_default=false. An unknown name errors before any build.}"\ ] - dict set SUBVALUES vfs [dict get $SUBVALUES bake] dict set SUBVALUES bakelist [list\ "@values -min 0 -max -1"\ "kitname -type string -optional 1 -multiple 1${_kitname_choicepart} -help\ { - Configured kit output(s) to report on (filters the listing - doubles + Configured kit output(s) to report on - a kit name, or @ + for every kit in a configured group (filters the listing - doubles as per-kit detail). With no kitname, all configured entries are listed.}"\ ] variable VALUES_SYNOPSES { bake "?kitname ...?" - vfs "?kitname ...?" bakelist "?kitname ...?" } foreach {sub optvars} $SUBOPTS { @@ -3034,7 +3498,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "vendored tools" {tool} "informational" {info check bakelist projectversion workflow help} "interactive" {shell} - "deprecated aliases" {project vfs} } punk::args::define\ "@id -id (script)::punkboot"\ @@ -3144,7 +3607,7 @@ if {$::punkboot::punkargs_ok} { } } if {[dict exists $argd values kitname]} { - #declared -multiple: a list of the supplied kit names (bake/vfs/bakelist - G-121) + #declared -multiple: a list of the supplied kit names (bake/bakelist - G-121) set ::punkboot::opt_kitnames [dict get $argd values kitname] } unset -nocomplain _opts _optname _var @@ -3204,10 +3667,10 @@ if {$::punkboot::punkargs_ok} { lappend commands_found $a } } - #G-121: bake/bakelist (and the deprecated alias vfs) take optional trailing kit - #names - collect them here so the single-command check below still holds in - #degraded mode (the handlers do their own name validation). - if {[lindex $commands_found 0] in {bake bakelist vfs}} { + #G-121: bake/bakelist take optional trailing kit names - collect them here so + #the single-command check below still holds in degraded mode (the handlers do + #their own name validation). + if {[lindex $commands_found 0] in {bake bakelist}} { set ::punkboot::opt_kitnames [lrange $commands_found 1 end] set commands_found [lrange $commands_found 0 0] } @@ -3260,23 +3723,7 @@ if {$do_help} { exit $help_exitcode } -#G-112 deprecated aliases - one-line mapping before any dispatch: -# project -> bakehouse (packages + bake, -dirty-abort default ON) -# vfs -> bake (kit assembly including the vfslibs phase) -#Kept for a transition window (until the 8.6 arc completes - see the G-112 -#dispositions). The alias definitions above mirror their targets' options so -#flags parse identically; only the command word maps here. if {[info exists ::punkboot::command]} { - switch -- $::punkboot::command { - project { - puts stderr "make.tcl: 'project' is DEPRECATED - use 'bakehouse' (packages + bake; refuses uncommitted src by default). Running bakehouse." - set ::punkboot::command bakehouse - } - vfs { - puts stderr "make.tcl: 'vfs' is DEPRECATED - use 'bake' (kit assembly incl. the vfslibs phase). Running bake." - set ::punkboot::command bake - } - } if {!$::punkboot::punkargs_ok && $::punkboot::command eq "bakehouse"} { #degraded fallback scan has flag-only -dirty-abort (no way to pass 0): #bakehouse's documented default is ON, so force it here. @@ -3386,7 +3833,7 @@ set ::punkboot::bake_selected_vfs [list] ;#vfs folder tails the selected set ::punkboot::bake_selected_runtimes [list] ;#unsuffixed runtime names the selected kits wrap if {$::punkboot::command eq "bake" && [llength $::punkboot::opt_kitnames]} { set _rtbase $binfolder/runtime - set _mapmodel [punkboot::lib::mapvfs_parse $sourcefolder/runtime/mapvfs.config $_rtbase $sourcefolder $::punkboot::target_platform] + set _mapmodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate $sourcefolder/runtime] $_rtbase $sourcefolder $::punkboot::target_platform] if {[llength [dict get $_mapmodel configerrors]]} { foreach _errline [dict get $_mapmodel configerrors] { puts stderr $_errline @@ -3399,8 +3846,11 @@ if {$::punkboot::command eq "bake" && [llength $::punkboot::opt_kitnames]} { puts stderr "make.tcl bake: unknown kit name(s): [join [dict get $_matchinfo unknown] {, }] - nothing built" if {[llength $_kit_outputs]} { puts stderr " configured kit outputs: [join [lmap _r $_kit_outputs {dict get $_r kitname}] {, }]" + if {[dict size [dict get $_mapmodel groups]]} { + puts stderr " configured groups (select with @): [join [dict keys [dict get $_mapmodel groups]] {, }]" + } } else { - puts stderr " no kit outputs are configured (src/runtime/mapvfs.config)" + puts stderr " no kit outputs are configured (src/runtime/mapvfs.toml)" } foreach _n [dict get $_matchinfo unknown] { if {[string match -* $_n]} { @@ -3740,7 +4190,7 @@ if {$::punkboot::command eq "check"} { if {$_chk_suffix eq ""} {set _chk_suffix "(none)"} puts stdout "platform (G-122): host=$::punkboot::host_platform target=$::punkboot::target_platform store=bin/runtime/[punkboot::lib::platform_store_tier $::punkboot::target_platform] exe-suffix=$_chk_suffix process-tooling=[punkboot::lib::platform_process_family $::punkboot::target_platform]" puts stdout " (host = this tclsh's punk::platform canon, from tcl_platform(os)='$::tcl_platform(os)' platform='$::tcl_platform(platform)'; target = default kit target - a cygwin-family host (msys2/cygwin runtime on windows) bakes win32 kits)" - puts stdout " (mapvfs.config entries may declare their own target platform as a 4th element - 'make.tcl bakelist' shows each kit's target, store tier and artifact name)" + puts stdout " (mapvfs.toml entries may declare their own target platform (target key) - 'make.tcl bakelist' shows each kit's target, store tier and artifact name)" unset -nocomplain _chk_suffix puts stdout $sep puts stdout "module/library checks - paths from bootsupport only" @@ -3950,8 +4400,8 @@ if {$::punkboot::command eq "check"} { puts stdout " bake will proceed with a NOTE and cannot flag a file-relative kit payload - run 'make.tcl modules' then 'make.tcl bootsupport'." } set _smoke_declared [list] - if {[file exists $sourcefolder/runtime/mapvfs.config]} { - set _smokemodel [punkboot::lib::mapvfs_parse $sourcefolder/runtime/mapvfs.config [file dirname $sourcefolder]/bin/runtime $sourcefolder $::punkboot::target_platform] + if {[file exists [punkboot::lib::mapvfs_locate $sourcefolder/runtime]]} { + set _smokemodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate $sourcefolder/runtime] [file dirname $sourcefolder]/bin/runtime $sourcefolder $::punkboot::target_platform] foreach _rec [punkboot::lib::mapvfs_kit_outputs $_smokemodel [file dirname $sourcefolder]/bin/runtime $sourcefolder] { if {[llength [dict get $_rec smokerequire]]} { lappend _smoke_declared "[dict get $_rec kitname] ([join [dict get $_rec smokerequire] {, }])" @@ -3960,7 +4410,7 @@ if {$::punkboot::command eq "check"} { unset -nocomplain _smokemodel _rec } puts stdout "smoke-require probe (G-133, advisory): freshly built host-runnable kits declaring packages" - puts stdout " (mapvfs.config 5th entry element) have each one plain-'package require'd inside the built" + puts stdout " (mapvfs.toml smokerequire key; legacy mapvfs.config 5th entry element) have each one plain-'package require'd inside the built" puts stdout " artifact via its tclsh subcommand - the only check that sees resolution-order defects such as" puts stdout " a wrong-arch higher-versioned package shadowing a working copy. Failures are recapped" puts stdout " BUILD-WARNINGs; cross-target kits skip with a stated reason; undeclared kits run nothing new." @@ -4167,7 +4617,7 @@ if {$::punkboot::command eq "bakelist"} { set rtbase $binfolder/runtime set rtfolder $rtbase/$rt_os_arch set buildfolder $sourcefolder/_build - set mapfile $sourcefolder/runtime/mapvfs.config + set mapfile [punkboot::lib::mapvfs_locate $sourcefolder/runtime] set mapmodel [punkboot::lib::mapvfs_parse $mapfile $rtbase $sourcefolder $rt_default_target] foreach errline [dict get $mapmodel configerrors] { puts stderr $errline @@ -4175,6 +4625,15 @@ if {$::punkboot::command eq "bakelist"} { foreach badline [dict get $mapmodel badentries] { puts stderr $badline } + if {[dict get $mapmodel format] eq "legacy"} { + #deprecation NOTE (first warnings line) - bakelist is reporting-only, so + #surface it here rather than relying on the bake path's warning printout + foreach warnline [dict get $mapmodel warnings] { + if {[string match NOTE:* $warnline]} { + puts stderr $warnline + } + } + } if {![dict get $mapmodel exists]} { puts stdout "no kit mapping config at $mapfile - no kit outputs configured" exit 0 @@ -4200,7 +4659,7 @@ if {$::punkboot::command eq "bakelist"} { if {$t eq $rt_os_arch} { puts stderr "WARNING: runtime store folder bin/runtime/$t does not exist - every configured runtime on the default target will read runtime=missing." puts stderr " Default target '$rt_default_target' derives from THIS tclsh: tcl_platform(os)='$::tcl_platform(os)' tcl_platform(platform)='$::tcl_platform(platform)' -> host canon '$::punkboot::host_platform' -> target '$rt_default_target'." - puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.config for runtimes stored elsewhere." + puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.toml for runtimes stored elsewhere." } else { puts stderr "WARNING: runtime store folder bin/runtime/$t does not exist - configured entries targeting $t will read runtime=missing." } @@ -4211,6 +4670,9 @@ if {$::punkboot::command eq "bakelist"} { if {[llength [dict get $matchinfo unknown]]} { puts stderr "bakelist: unknown kit name(s): [join [dict get $matchinfo unknown] {, }]" puts stderr " configured kit outputs: [join [lmap r $kit_outputs {dict get $r kitname}] {, }]" + if {[dict size [dict get $mapmodel groups]]} { + puts stderr " configured groups (select with @): [join [dict keys [dict get $mapmodel groups]] {, }]" + } exit 1 } set reportset [dict get $matchinfo selected] @@ -4222,7 +4684,14 @@ if {$::punkboot::command eq "bakelist"} { set kit_type [dict get $rec kit_type] set runtime [dict get $rec runtime] set vfstail [dict get $rec vfs] - set deployed [punkboot::lib::kit_deploy_state $buildfolder $binfolder [dict get $rec targetkit]] + #G-127: deploy-state roots follow the record's output tier (non-default + #targets live under kits// in both bin/ and src/_build/) + set out_tier [dict get $rec out_tier] + if {$out_tier eq ""} { + set deployed [punkboot::lib::kit_deploy_state $buildfolder $binfolder [dict get $rec targetkit]] + } else { + set deployed [punkboot::lib::kit_deploy_state $buildfolder/$out_tier $binfolder/$out_tier [dict get $rec targetkit]] + } set notes [list] if {$runtime ne "-" && ![dict get $rec runtime_present]} { lappend notes runtime=missing @@ -4234,6 +4703,18 @@ if {$::punkboot::command eq "bakelist"} { #cross-target entry: its runtime, store tier and artifact naming follow #its own platform, not this host's default lappend notes target=[dict get $rec target] + #G-127: and its build/deploy locations sit under the kits// tier + lappend notes out=$out_tier/ + } + if {[dict get $rec group] ne ""} { + lappend notes group=[dict get $rec group] + } + if {![dict get $rec bake_default]} { + #excluded from full bakes - bake by name or @group (G-024) + lappend notes default=no + } + if {[dict get $rec scheme_role] ne ""} { + lappend notes scheme=[dict get $rec scheme_role] } if {$runtime ne "-" && [dict get $rec runtime_present]} { set rtstale [punkboot::lib::runtime_materialization_check [dict get $rec runtime_dir] [dict get $rec runtime_file]] @@ -4256,7 +4737,15 @@ if {$::punkboot::command eq "bakelist"} { set widths $newwidths } set rt_store_note [expr {$rt_store_missing ? " (FOLDER MISSING)" : ""}] - puts stdout "# mapvfs: src/runtime/mapvfs.config runtimes: bin/runtime/$rt_os_arch$rt_store_note deployed: bin/ vs src/_build/" + if {[info exists ::env(PUNK_MAPVFS_CONFIG)] && $::env(PUNK_MAPVFS_CONFIG) ne ""} { + set map_display "$mapfile (PUNK_MAPVFS_CONFIG override)" + } else { + set map_display "src/runtime/[file tail $mapfile]" + if {[dict get $mapmodel format] eq "legacy"} { + append map_display " (deprecated format)" + } + } + puts stdout "# mapvfs: $map_display runtimes: bin/runtime/$rt_os_arch$rt_store_note deployed: bin/ vs src/_build/ (out= rows: under kits//)" set headline "" foreach h $headings w $widths { append headline [format "%-*s " $w $h] @@ -4289,9 +4778,18 @@ if {$::punkboot::command eq "bakelist"} { } puts stdout " vfs folder: src/vfs/[dict get $rec vfs] [expr {[dict get $rec vfs_present] ? {(present)} : {(MISSING)}}]" puts stdout " kit type: [dict get $rec kit_type]" - puts stdout " target: [dict get $rec target][expr {[dict get $rec target] eq $rt_default_target ? " (host default)" : " (declared in mapvfs.config)"}]" - puts stdout " build product: src/_build/[dict get $rec targetkit] [punkboot::lib::bakelist_file_detail $buildfolder/[dict get $rec targetkit]]" - puts stdout " deployed: bin/[dict get $rec targetkit] [punkboot::lib::bakelist_file_detail $binfolder/[dict get $rec targetkit]] state=[lindex $row 4]" + puts stdout " target: [dict get $rec target][expr {[dict get $rec target] eq $rt_default_target ? " (host default)" : " (declared in the kit mapping)"}]" + if {[dict get $rec entry] ne ""} { + puts stdout " config entry: \[[dict get $rec entry]\]" + } + if {[dict get $rec group] ne ""} { + puts stdout " group: [dict get $rec group][expr {[dict get $rec bake_default] ? "" : " (bake_default=false - bake by name or @[dict get $rec group])"}]" + } elseif {![dict get $rec bake_default]} { + puts stdout " bake_default: false (excluded from full bakes - bake by name to build)" + } + set relkit [expr {[dict get $rec out_tier] eq "" ? [dict get $rec targetkit] : "[dict get $rec out_tier]/[dict get $rec targetkit]"}] + puts stdout " build product: src/_build/$relkit [punkboot::lib::bakelist_file_detail $buildfolder/$relkit]" + puts stdout " deployed: bin/$relkit [punkboot::lib::bakelist_file_detail $binfolder/$relkit] state=[lindex $row 4]" } } exit 0 @@ -5349,20 +5847,8 @@ if {$::punkboot::command eq "libfetch"} { puts stderr "$A(BAD)LIBFETCH: failed to parse $libfetch_config\n$libfetch_dict$A(RST)" exit 3 } - if {![llength [info commands ::punkboot::lib::toml_untag]]} { - #tomlish::to_dict values are type-tagged (also defined by the vfslibs - #phase - that block does not run for the libfetch command) - proc ::punkboot::lib::toml_untag {tagged} { - if {[dict get $tagged type] eq "ARRAY"} { - set plain [list] - foreach el [dict get $tagged value] { - lappend plain [::punkboot::lib::toml_untag $el] - } - return $plain - } - return [dict get $tagged value] - } - } + #tomlish::to_dict values are type-tagged - ::punkboot::lib::toml_untag (defined + #with the shared toml helpers near the top of this script) resolves them set libfetch_artifacts [dict create] if {[dict exists $libfetch_dict artifact]} { set libfetch_artifacts [dict get $libfetch_dict artifact] @@ -5683,135 +6169,222 @@ if {$::punkboot::command in {bakehouse packages libs}} { } if {$::punkboot::command in {bakehouse bake vfslibs}} { - #G-037: propagate declared vendored platform-library packages into kit vfs lib_tcl trees. - #Participation is per-package per-kit (kit vfs package sets intentionally differ - e.g. twapi - #versions vary per kit) - declared in src/runtime/vendorlib_vfs.toml, never blanket-copied. - #punkcheck records live at src/vfs/.punkcheck so they are not baked into kit payloads. - set vfslibs_config $sourcefolder/runtime/vendorlib_vfs.toml - if {![file exists $vfslibs_config]} { - puts stdout "VFSLIBS: no config at $vfslibs_config - skipping vendorlib -> kit vfs propagation" - } else { - if {[catch {package require tomlish} tomlish_err]} { - puts stderr "$A(BAD)VFSLIBS: cannot process $vfslibs_config - tomlish package unavailable ($tomlish_err)$A(RST)" + #G-115: declarative .vfs payload composition. Each src/vfs/.vfs may have a + #SIBLING declaration file src/vfs/.vfs.toml declaring payload package folders + #to materialize INTO it - while the folder remains the operative assembly area: + #drop-ins need no declaration and are never clobbered. The declaration file's own + #header comment (and src/vfs/README.md) is the user-facing spec. Entry keys per + #[payload.]: source (folder path, resolved per source_root), source_root + #('src' default - vendor trees / suite build products under src/; or 'packages' - + #the consent-gated punkbin lib tier at /bin/packages populated by + #'make.tcl libfetch'), target (folder path INSIDE the .vfs, e.g "lib_tcl9"; + #default the .vfs root), supersedes (explicit legacy sibling folder removals), + #replace (boolean clean-slate of the SAME-NAMED target folder before install). + #A source containing %platform% is a PER-PLATFORM entry (G-127): one copy per + #consuming kit target (derived from the kit mapping below), staged under + #_targets// and selected into each kit's merged image at bake time. + #Materialization is punkcheck-tracked (records in src/vfs/.punkcheck, outside kit + #payloads) with -overwrite synced-targets: a target file punkcheck did not install, + #or that changed since it was installed (a drop-in or hand edit), is PRESERVED and + #reported, never overwritten - the G-115 drop-in-wins precedence; supersedes and + #replace are the explicit declared exceptions. A .vfs with no declaration file is + #untouched (pure drop-in mode, exactly as before G-115). + #This phase supersedes the per-package src/runtime/vendorlib_vfs.toml surface + #(G-037/G-139): its [install.] entries were migrated into per-.vfs files + #2026-07-31 - rationale in goals/G-115-declarative-vfs-composition.md. + set legacy_vfslibs $sourcefolder/runtime/vendorlib_vfs.toml + if {[file exists $legacy_vfslibs]} { + puts stderr "WARNING: $legacy_vfslibs is superseded by per-.vfs payload declarations (src/vfs/.vfs.toml - G-115) and is IGNORED - migrate its \[install.*\] entries into per-.vfs files and remove it." + } + set payload_files [lsort [glob -nocomplain -dir $sourcefolder/vfs -types f -tail *.vfs.toml]] + if {![llength $payload_files]} { + puts stdout "VFSPAYLOAD: no per-.vfs payload declarations (src/vfs/.vfs.toml) - nothing to materialize" + } + #G-127 per-platform payload derivation: a payload source containing %platform% + #declares ONE payload materialized per CONSUMING KIT TARGET - the platform set + #is derived from the kit mapping (every mapvfs kit output pairing this vfs, + #whatever its bake_default), never declared a second time in the payload file. + #Each platform's copy lands under _targets/// inside the .vfs + #folder (VCS-ignored staging); the bake selects the consuming kit's subtree + #into its merged image and drops the staging tree, so an assembled kit carries + #only its own target's payload. + set vfspayload_kit_targets [dict create] + if {[llength $payload_files]} { + set _pmapmodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate $sourcefolder/runtime] $binfolder/runtime $sourcefolder $::punkboot::target_platform] + if {[llength [dict get $_pmapmodel configerrors]]} { + puts stderr "WARNING: VFSPAYLOAD: kit-mapping config errors - per-platform (%platform%) payload derivation unavailable this run (the mapping's own gates report the errors)" + } else { + foreach _prec [punkboot::lib::mapvfs_kit_outputs $_pmapmodel $binfolder/runtime $sourcefolder] { + if {![dict exists $vfspayload_kit_targets [dict get $_prec vfs]] + || [dict get $_prec target] ni [dict get $vfspayload_kit_targets [dict get $_prec vfs]]} { + dict lappend vfspayload_kit_targets [dict get $_prec vfs] [dict get $_prec target] + } + } + } + unset -nocomplain _pmapmodel _prec + } + foreach ptail $payload_files { + set pfile $sourcefolder/vfs/$ptail + set vfstail [file rootname $ptail] ;#punkdeclare.vfs.toml -> punkdeclare.vfs + if {[llength $::punkboot::bake_selected_kitnames] && $vfstail ni $::punkboot::bake_selected_vfs} { + #G-121 selective bake: narrow to the selected kits' vfs folders (skip + #precedes the existence checks so an unselected declaration's problem + #cannot abort a selective run) + puts stdout "VFSPAYLOAD $vfstail: skipping (selective bake - not a selected kit's vfs)" + continue + } + if {![file isdirectory $sourcefolder/vfs/$vfstail]} { + puts stderr "$A(BAD)VFSPAYLOAD: declaration $pfile has no matching vfs folder at $sourcefolder/vfs/$vfstail$A(RST)" exit 3 } - set fd [open $vfslibs_config r] - fconfigure $fd -encoding utf-8 - set vfslibs_tomldata [read $fd] - close $fd - if {[catch {tomlish::to_dict [tomlish::from_toml $vfslibs_tomldata]} vfslibs_dict]} { - puts stderr "$A(BAD)VFSLIBS: failed to parse $vfslibs_config\n$vfslibs_dict$A(RST)" + lassign [punkboot::lib::toml_file_to_dict $pfile] pstatus ppayload + if {$pstatus ne "ok"} { + puts stderr "$A(BAD)VFSPAYLOAD: $ppayload$A(RST)" exit 3 } - #tomlish::to_dict values are type-tagged (e.g {type STRING value x}, {type ARRAY value {...}}) - proc ::punkboot::lib::toml_untag {tagged} { - if {[dict get $tagged type] eq "ARRAY"} { - set plain [list] - foreach el [dict get $tagged value] { - lappend plain [::punkboot::lib::toml_untag $el] - } - return $plain + dict for {topkey -} $ppayload { + if {$topkey ne "payload"} { + puts stderr "$A(BAD)VFSPAYLOAD: $pfile: unknown top-level table or key '$topkey' (expected \[payload.\] entries)$A(RST)" + exit 3 } - return [dict get $tagged value] } - set vfslibs_installs [dict create] - if {[dict exists $vfslibs_dict install]} { - set vfslibs_installs [dict get $vfslibs_dict install] + set payload_entries [dict create] + if {[dict exists $ppayload payload]} { + set payload_entries [dict get $ppayload payload] } - if {![dict size $vfslibs_installs]} { - puts stdout "VFSLIBS: no \[install.\] entries found in $vfslibs_config" + if {![dict size $payload_entries]} { + puts stdout "VFSPAYLOAD $vfstail: no \[payload.\] entries in $ptail" + continue } - dict for {iname ientry} $vfslibs_installs { - foreach reqkey {source targets} { - if {![dict exists $ientry $reqkey]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname in $vfslibs_config is missing required key '$reqkey'$A(RST)" - exit 3 - } - } - set isource [punkboot::lib::toml_untag [dict get $ientry source]] - set itargets [punkboot::lib::toml_untag [dict get $ientry targets]] - set isupersedes [list] - if {[dict exists $ientry supersedes]} { - set isupersedes [punkboot::lib::toml_untag [dict get $ientry supersedes]] - } - #source_root (G-139): 'src' (default - the original vendorlib - #declarations) or 'packages' - resolve against the untracked - #bin/packages lib-tier input (populated by 'make.tcl libfetch'). + dict for {iname ientry} $payload_entries { + set entrylabel "payload.$iname ($ptail)" + set isource "" set isource_root "src" - if {[dict exists $ientry source_root]} { - set isource_root [punkboot::lib::toml_untag [dict get $ientry source_root]] - } - #replace (G-139): clean-slate a SAME-NAMED existing target folder - #before install. supersedes deliberately cannot name the folder being - #installed; replace covers same-name refreshes where the old copy may - #carry strays the new tree lacks (e.g. hand-drops nesting md5c/tcllibc - #inside tcllib2.0). + set itarget "" + set isupersedes [list] set ireplace 0 - if {[dict exists $ientry replace]} { - set ireplace [punkboot::lib::toml_untag [dict get $ientry replace]] - if {![string is boolean -strict $ireplace]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname 'replace' must be boolean (got '$ireplace')$A(RST)" - exit 3 + dict for {k v} $ientry { + switch -- $k { + source {set isource [punkboot::lib::toml_untag $v]} + source_root {set isource_root [punkboot::lib::toml_untag $v]} + target {set itarget [punkboot::lib::toml_untag $v]} + supersedes {set isupersedes [punkboot::lib::toml_untag $v]} + replace { + set ireplace [punkboot::lib::toml_untag $v] + if {![string is boolean -strict $ireplace]} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel: 'replace' must be boolean (got '$ireplace')$A(RST)" + exit 3 + } + } + default { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel: unknown key '$k' (expected source, source_root, target, supersedes, replace)$A(RST)" + exit 3 + } } } - switch -- $isource_root { - src { - set source_pkg_folder $sourcefolder/$isource - set source_missing_hint "" - } - packages { - set source_pkg_folder $projectroot/bin/packages/$isource - set source_missing_hint " - populate the bin/packages tier first: 'tclsh src/make.tcl libfetch' (declarations in src/runtime/libpackages.toml)" - } - default { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname has unknown source_root '$isource_root' (expected src|packages)$A(RST)" - exit 3 - } + if {$isource eq ""} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel is missing required key 'source'$A(RST)" + exit 3 + } + if {$itarget ne "" && ([file pathtype $itarget] ne "relative" || ".." in [file split $itarget])} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel target '$itarget' must be a relative path inside the .vfs folder (no '..')$A(RST)" + exit 3 } - if {![file isdirectory $source_pkg_folder]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname source '$isource' not found at $source_pkg_folder$source_missing_hint$A(RST)" + #G-127 per-platform axis: %platform% in 'source' selects the payload per + #consuming kit target (set derived above); 'target' stays literal - its + #per-platform staging prefix (_targets//) is implicit. + if {[string match *%platform%* $itarget]} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel: 'target' must not contain %platform% - the per-platform staging prefix (_targets//) is implicit for a %platform% source$A(RST)" exit 3 } - set pkgtail [file tail $source_pkg_folder] - foreach t $itargets { - set target_vfs_dir [lindex [file split $t] 0] - if {[llength $::punkboot::bake_selected_kitnames] && $target_vfs_dir ni $::punkboot::bake_selected_vfs} { - #G-121 selective bake: narrow the vfslibs phase to the selected kits' - #vfs folders (skip precedes the existence check so an unselected - #entry's problem cannot abort a selective run) - puts stdout "VFSLIBS install.$iname: skipping vfs/$t (selective bake - not a selected kit's vfs)" + if {[string match *%platform%* $isource]} { + if {![dict exists $vfspayload_kit_targets $vfstail] + || ![llength [dict get $vfspayload_kit_targets $vfstail]]} { + puts stderr "WARNING: VFSPAYLOAD entry $entrylabel declares a per-platform source (%platform%) but no kit in the mapping consumes $vfstail - no platform set to derive, entry skipped" continue } - if {![file isdirectory $sourcefolder/vfs/$target_vfs_dir]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname target '$t' - no vfs folder at $sourcefolder/vfs/$target_vfs_dir$A(RST)" + set entry_platforms [dict get $vfspayload_kit_targets $vfstail] + } else { + set entry_platforms [list ""] ;#platform-neutral entry - materialize once, directly in the folder + } + foreach eplatform $entry_platforms { + if {$eplatform eq ""} { + set esource $isource + set elabel $entrylabel + set evfsprefix "" + } else { + set esource [string map [list %platform% $eplatform] $isource] + set elabel "$entrylabel \[$eplatform\]" + set evfsprefix _targets/$eplatform + } + switch -- $isource_root { + src { + set source_pkg_folder $sourcefolder/$esource + set source_missing_hint "" + } + packages { + set source_pkg_folder $projectroot/bin/packages/$esource + set source_missing_hint " - populate the bin/packages tier first: 'tclsh src/make.tcl libfetch' (declarations in src/runtime/libpackages.toml)" + } + default { + puts stderr "$A(BAD)VFSPAYLOAD: entry $elabel has unknown source_root '$isource_root' (expected src|packages)$A(RST)" + exit 3 + } + } + if {![file isdirectory $source_pkg_folder]} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $elabel source '$esource' not found at $source_pkg_folder$source_missing_hint$A(RST)" exit 3 } - set target_lib_folder $sourcefolder/vfs/$t + set pkgtail [file tail $source_pkg_folder] + set etarget_subpath $itarget + if {$evfsprefix ne ""} { + set etarget_subpath [expr {$itarget eq "" ? $evfsprefix : "$evfsprefix/$itarget"}] + } + if {$etarget_subpath eq ""} { + set target_lib_folder $sourcefolder/vfs/$vfstail + set target_display vfs/$vfstail + } else { + set target_lib_folder $sourcefolder/vfs/$vfstail/$etarget_subpath + set target_display vfs/$vfstail/$etarget_subpath + } file mkdir $target_lib_folder #supersession first - no silent mixed-version provision (G-035 concerns) foreach sup $isupersedes { if {$sup eq $pkgtail} { - continue ;#never remove what we are about to install + continue ;#never remove what we are about to install - use replace for that } set suppath $target_lib_folder/$sup if {[file exists $suppath]} { - puts stdout "VFSLIBS install.$iname: removing superseded '$sup' from vfs/$t" + puts stdout "VFSPAYLOAD $elabel: removing superseded '$sup' from $target_display" file delete -force $suppath } } if {$ireplace && [file isdirectory $target_lib_folder/$pkgtail]} { - puts stdout "VFSLIBS install.$iname: replace=true - removing existing '$pkgtail' from vfs/$t (clean-slate install)" + puts stdout "VFSPAYLOAD $elabel: replace=true - removing existing '$pkgtail' from $target_display (clean-slate install)" file delete -force $target_lib_folder/$pkgtail } - puts stdout "VFSLIBS install.$iname: $isource -> vfs/$t/$pkgtail (if source file changed)" + puts stdout "VFSPAYLOAD $elabel: $isource_root:$esource -> $target_display/$pkgtail (if source changed; drop-ins/hand-edits preserved)" set resultdict [punkcheck::install $source_pkg_folder $target_lib_folder/$pkgtail\ -installer make.tcl\ -createdir 1\ - -overwrite installedsourcechanged-targets\ + -overwrite synced-targets\ -punkcheck-folder $sourcefolder/vfs\ -progresschannel stdout\ ] puts stdout [punkcheck::summarize_install_resultdict $resultdict] + #preserved-collision report: a file skipped although its SOURCE changed was + #withheld by synced-targets (a drop-in, or hand-modified since install) - + #distinct from the routine source-unchanged skips. + set _preserved [list] + foreach _sk [dict get $resultdict files_skipped] { + if {$_sk ni [dict get $resultdict sources_unchanged]} { + lappend _preserved $_sk + } + } + if {[llength $_preserved]} { + puts stdout "VFSPAYLOAD $elabel: NOTE - [llength $_preserved] changed file(s) NOT overwritten (existing target content was not installed by this mechanism, or was modified since) - the G-115 drop-in-wins precedence. Declare 'replace = true' (whole package folder) or remove the target files to hand control to the declaration." + } + unset -nocomplain _preserved _sk } } } @@ -6224,7 +6797,7 @@ proc ::punkboot::runtime_materialization_warning {rtfolder runtime_fullname} { #G-122: parsed BEFORE runtime discovery, because the mapping is what says which #store tiers this bake must look in - an entry declaring a target platform other #than the host default addresses its own bin/runtime/. -set mapfile $rt_sourcefolder/mapvfs.config +set mapfile [punkboot::lib::mapvfs_locate $rt_sourcefolder] set mapmodel [punkboot::lib::mapvfs_parse $mapfile $rtbase $sourcefolder $rt_default_target] foreach warnline [dict get $mapmodel warnings] { puts stderr $warnline @@ -6240,11 +6813,53 @@ set vfs_runtime_map [dict get $mapmodel vfs_runtime_map] set runtime_target [dict get $mapmodel runtime_target] set missing [dict get $mapmodel missing] if {[llength $missing]} { - puts stderr "WARNING [llength $missing] missing items from $mapfile. (TODO - prompt user to continue/abort)" - foreach m $missing { - puts stderr " $m" + if {[dict get $mapmodel format] eq "toml" && ![llength $::punkboot::bake_selected_kitnames]} { + #G-024 strictness (toml models): a full bake with unresolvable DEFAULT entries + #reports each one BY ENTRY rather than as a bare item list. A missing vfs + #folder is fatal - vfs folders are repo content, so its absence means a wrong + #declaration. A missing runtime FILE is environment, not declaration: store + #tiers are legitimately part-populated (fresh clones; runtimes arrive via the + #consent-gated fetch tiers), so those entries skip with a recapped + #BUILD-WARNING naming the entry (cross-target wording states the tier reason). + #bake_default=false entries are not in the full-bake set; selective bakes + #fail fast at selection instead. + set _strict_fatal [list] + set _strict_warn [list] + foreach _rec [punkboot::lib::mapvfs_kit_outputs $mapmodel $rtbase $sourcefolder] { + if {![dict get $_rec bake_default]} { + continue + } + set _entrylabel [expr {[dict get $_rec entry] ne "" ? [dict get $_rec entry] : [dict get $_rec kitname]}] + if {![dict get $_rec vfs_present]} { + lappend _strict_fatal "entry $_entrylabel: vfs folder src/vfs/[dict get $_rec vfs] is missing" + continue + } + if {[dict get $_rec runtime] ne "-" && ![dict get $_rec runtime_present]} { + if {[dict get $_rec target] eq $rt_default_target} { + lappend _strict_warn "kit [dict get $_rec kitname] (entry $_entrylabel) skipped - runtime [dict get $_rec runtime_file] not present in bin/runtime/[dict get $_rec store_tier]" + } else { + lappend _strict_warn "kit [dict get $_rec kitname] (entry $_entrylabel) skipped - cross-target runtime [dict get $_rec runtime_file] not present in bin/runtime/[dict get $_rec store_tier] (target [dict get $_rec target] tier not populated on this host)" + } + } + } + if {[llength $_strict_fatal]} { + puts stderr "CONFIG ERROR: unresolvable kit mapping entries in $mapfile - nothing built:" + foreach _line $_strict_fatal { + puts stderr " $_line" + } + exit 3 + } + if {[llength $_strict_warn]} { + ::punkboot::print_build_warnings $_strict_warn + } + unset -nocomplain _rec _entrylabel _strict_fatal _strict_warn _line + } else { + puts stderr "WARNING [llength $missing] missing items from $mapfile. (TODO - prompt user to continue/abort)" + foreach m $missing { + puts stderr " $m" + } + puts stderr "continuing..." } - puts stderr "continuing..." } # -- --- --- --- --- --- --- --- --- --- puts "-- runtime_vfs_map --" @@ -6333,7 +6948,7 @@ dict for {_rtname _rttarget} $runtime_target { dict set rtfolder_of $_file $_dir } if {[llength $cross_target_tiers]} { - puts stdout "cross-target runtime tiers in use (mapvfs.config target platforms): [join $cross_target_tiers {, }]" + puts stdout "cross-target runtime tiers in use (kit-mapping target platforms): [join $cross_target_tiers {, }]" } unset -nocomplain _rtname _rttarget _tier _dir _file if {![llength $runtimes]} { @@ -6342,7 +6957,7 @@ if {![llength $runtimes]} { if {![file isdirectory $rtfolder]} { #same store-tier self-diagnosis as bakelist puts stderr "NOTE: that runtime store folder does not exist. Default target '$rt_default_target' derives from THIS tclsh: tcl_platform(os)='$::tcl_platform(os)' tcl_platform(platform)='$::tcl_platform(platform)' -> host canon '$::punkboot::host_platform' -> target '$rt_default_target'." - puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.config for runtimes stored elsewhere." + puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.toml for runtimes stored elsewhere." } #todo - don't exit - it is valid to use runtime of - to just build a .kit/.zipkit ? exit 0 @@ -7323,6 +7938,10 @@ foreach vfstail $vfs_tails { } set rt_target [punkboot::lib::runtime_target_of $rt] set rt_file $rt[punkboot::lib::platform_exe_suffix $rt_target] + if {$rt_file in $runtimes} { + continue ;#several entries may share a runtime on one vfs (toml models) - + #process each runtime once (its spec set enumerates all its kits) + } if {[file exists $rtbase/[punkboot::lib::platform_store_tier $rt_target]/$rt_file]} { lappend runtimes $rt_file } @@ -7376,7 +7995,9 @@ foreach vfstail $vfs_tails { set applist [dict get $runtime_vfs_map $rtname] foreach vfs_app $applist { #5th element = G-133 smoke-require package list ("" when undeclared) + #6th element (toml models) = G-024 entry attrs (group/bake_default/scheme_role) lassign $vfs_app configured_vfs appname target_kit_type - kit_smokerequires + set kit_attrs [punkboot::lib::mapvfs_spec_attrs $vfs_app] if {$configured_vfs ne $vfstail} { continue } @@ -7390,15 +8011,18 @@ foreach vfstail $vfs_tails { set targetkit $appname.kit } else { set targetkit ${appname}$rt_suffix - if {$targetkit in $exe_names_seen} { - #duplicate appname configured? + if {[list $rt_target $targetkit] in $exe_names_seen} { + #duplicate appname configured for the SAME target? + #(G-127: same-named kits for DIFFERENT targets coexist under + #their own kits// output tiers - only a same-target + #duplicate needs a disambiguating name) #todo - consider creating as ${appname}(2) etc? - puts stderr "targetkit: $targetkit already seen - using name ${appname}_$rtname" + puts stderr "targetkit: $targetkit already seen for target $rt_target - using name ${appname}_$rtname" set targetkit ${appname}_$rtname } } - lappend exe_names_seen $targetkit - lappend targetkits [list $targetkit $target_kit_type $kit_smokerequires] + lappend exe_names_seen [list $rt_target $targetkit] + lappend targetkits [list $targetkit $target_kit_type $kit_smokerequires $kit_attrs] } } puts stdout " vfs: $vfstail runtime: $rtname targetkits: $targetkits" @@ -7407,7 +8031,41 @@ foreach vfstail $vfs_tails { continue ;#G-121 selective bake - not a requested kit output } puts stdout " processing targetkit: $targetkit_info" - lassign $targetkit_info targetkit target_kit_type kit_smokerequires + lassign $targetkit_info targetkit target_kit_type kit_smokerequires kit_attrs + #G-127: output locations are keyed by the kit's TARGET. Kits for this + #host's own default target keep the flat locations every launcher, test + #and habit expects (src/_build/, bin/); any other target's kit + #builds and deploys under a kits// tier so same-named kits for + #different targets coexist instead of overwriting each other across + #selective runs. bin/kits/ rather than bin// because + #bin/runtime// already means build INPUTS. + if {$rt_target eq $rt_default_target} { + set kit_builddir $buildfolder + set kit_deploydir $binfolder + set kit_relpath $targetkit ;#kit path relative to bin/ and src/_build/ + } else { + set kit_builddir $buildfolder/kits/$rt_target + set kit_deploydir $binfolder/kits/$rt_target + set kit_relpath kits/$rt_target/$targetkit + } + file mkdir $kit_builddir + #G-024: bake_default=false entries (declared per entry or via their group) + #are excluded from full bakes - explicit name/@group selection builds them. + if {![llength $::punkboot::bake_selected_kitnames] + && [dict exists $kit_attrs bake_default] && ![dict get $kit_attrs bake_default]} { + puts stdout " skipping $targetkit (bake_default=false - bake by name or @group to build it)" + lappend skipped_kits [list kit $targetkit reason "bake_default=false (not part of full bakes)"] + continue + } + #G-023/G-024 release gating: a 'versioned' scheme's plain-name output is + #created when absent, then replaced only by an explicit release step + #(G-023 owns that step) - a normal bake never overwrites it. + if {[dict exists $kit_attrs scheme_role] && [dict get $kit_attrs scheme_role] eq "release" + && [file exists $kit_deploydir/$targetkit]} { + puts stdout " skipping $targetkit (release-gated scheme output: bin/$kit_relpath exists and a normal bake never overwrites it - the explicit release step is G-023; to force a rebuild now, delete bin/$kit_relpath first)" + lappend skipped_kits [list kit $targetkit reason "release-gated (bin/$kit_relpath exists)"] + continue + } #Self-build guard: never process the kit whose deployed executable is running this #build - the process sweep below would kill this very process, and the deploy step #cannot replace a running image anyway. Informational/update subcommands are @@ -7415,7 +8073,7 @@ foreach vfstail $vfs_tails { #punk executable remains supported for everything except rebuilding that executable. #(a HOST filesystem comparison - case rules follow the host, not the target) set self_exe [file normalize [info nameofexecutable]] - set self_deploy_target [file normalize $binfolder/$targetkit] + set self_deploy_target [file normalize $kit_deploydir/$targetkit] if {$::punkboot::host_windows} { set is_self_build [string equal -nocase $self_exe $self_deploy_target] } else { @@ -7431,7 +8089,7 @@ foreach vfstail $vfs_tails { set vfs_installer [punkcheck::installtrack new $installername $basedir/.punkcheck] $vfs_installer set_source_target $sourcefolder $buildfolder set vfs_event [$vfs_installer start_event {-make-step build_vfs}] - $vfs_event targetset_init INSTALL $buildfolder/$targetkit + $vfs_event targetset_init INSTALL $kit_builddir/$targetkit set relvfs [punkcheck::lib::path_relative $basedir $sourcefolder/vfs/$vfstail] if {![dict exists $path_cksum_cache $relvfs]} { #e.g ../vfs/punk87.vfs {cksum xxxx cksum_all_opts {-cksum_content 1 ... -cksum_algorithm sha1}} @@ -7475,7 +8133,8 @@ foreach vfstail $vfs_tails { package require control #keep this a simple name - bin/punk script calls into src/_build/exename.vfs/main.tcl - set targetvfs $buildfolder/$targetkit.vfs + #(G-127: non-default-target kits keep the simple name under their kits// tier) + set targetvfs $kit_builddir/$targetkit.vfs file delete -force $targetvfs set extraction_done 0 @@ -7781,6 +8440,26 @@ foreach vfstail $vfs_tails { set sourcevfs [file join $sourcefolder vfs $vfstail] merge_over $sourcevfs $targetvfs + #G-127 per-platform payload selection: the VFSPAYLOAD phase stages + #%platform% payload entries under _targets// in the .vfs + #folder (one subtree per consuming kit target), and the merge above + #carried that staging tree into the image. Select THIS kit's target + #subtree into the image root and drop the staging tree, so the + #assembled kit carries only its own target's payload - upstream of + #the arch scan and the G-125 gate, which must judge the final tree. + if {[file isdirectory $targetvfs/_targets]} { + set _pstaging ${targetvfs}._targets_staging + file delete -force $_pstaging + file rename $targetvfs/_targets $_pstaging + if {[file isdirectory $_pstaging/$rt_target]} { + puts stdout " per-platform payload (G-127): selecting _targets/$rt_target into the merged image for kit $targetkit" + merge_over $_pstaging/$rt_target $targetvfs + } else { + ::punkboot::print_build_warnings [list "per-platform payload (G-127): kit $targetkit (target $rt_target) - the merged vfs has a _targets/ staging tree but no _targets/$rt_target subtree, so this kit gets NO per-platform payload (check the %platform% declarations in src/vfs/$vfstail.toml against the kit's mapvfs target)"] + } + file delete -force $_pstaging + } + #G-133 advisory payload/target binary-arch scan, at the same post-merge #seam as the G-125 gate below. Structural (header bytes only, nothing #executed), so it runs for every kit including cross-target ones. A @@ -7847,7 +8526,7 @@ foreach vfstail $vfs_tails { #the assemblers below consume wrap_runtime - the stamped per-kit copy #of the payload-free raw prefix when embedding happened, the untouched #raw prefix otherwise. Runtime store originals are never modified. - lassign [::punkboot::kit_icon_process $targetkit $rt_target $rtname $target_kit_type $sourcefolder $vfstail $raw_runtime $buildfolder] wrap_runtime kiticon_sidecar + lassign [::punkboot::kit_icon_process $targetkit $rt_target $rtname $target_kit_type $sourcefolder $vfstail $raw_runtime $kit_builddir] wrap_runtime kiticon_sidecar set wrapvfs $targetvfs switch -- $target_kit_type { @@ -8147,13 +8826,13 @@ foreach vfstail $vfs_tails { } } - if {[file exists $buildfolder/$targetkit]} { - puts stderr "deleting existing $buildfolder/$targetkit" + if {[file exists $kit_builddir/$targetkit]} { + puts stderr "deleting existing $kit_builddir/$targetkit" if {[catch { - file delete $buildfolder/$targetkit + file delete $kit_builddir/$targetkit } msg]} { - puts stderr "Failed to delete $buildfolder/$targetkit" - lappend failed_kits [list kit $targetkit reason "could not delete buildfolder kit at $buildfolder/$targetkit"] + puts stderr "Failed to delete $kit_builddir/$targetkit" + lappend failed_kits [list kit $targetkit reason "could not delete buildfolder kit at $kit_builddir/$targetkit"] $vfs_event targetset_end FAILED $vfs_event destroy $vfs_installer destroy @@ -8163,7 +8842,7 @@ foreach vfstail $vfs_tails { #WINDOWS filesystem 'tunnelling' (file replacement within 15secs) could cause targetkit to copy ctime & shortname metadata from previous file! #This is probably harmless - but worth being aware of. - file rename $buildfolder/$vfsname.new $buildfolder/$targetkit + file rename $buildfolder/$vfsname.new $kit_builddir/$targetkit # -- --- --- --- --- --- $vfs_event targetset_end OK @@ -8175,7 +8854,7 @@ foreach vfstail $vfs_tails { #deploys; plain/none (no attached zip - the metakit kit shape) and #unreadable results are silence. One EOCD scan + CD walk per #REBUILT kit only. - lassign [::punkboot::get_kit_offsetstyle_report $buildfolder/$targetkit] offprobe_available offprobe + lassign [::punkboot::get_kit_offsetstyle_report $kit_builddir/$targetkit] offprobe_available offprobe if {!$offprobe_available} { puts stderr "NOTE: kit offset-style pin unavailable (punkboot::utils kit_offsetstyle_report not loadable from bootsupport) - continuing without it" } elseif {[dict get $offprobe offsetstyle] eq "file"} { @@ -8195,7 +8874,7 @@ foreach vfstail $vfs_tails { puts stdout " smoke-require skipped for kit $targetkit: cross-target ($rt_target) - the artifact is not executable on this $::punkboot::host_platform host" } else { puts stdout " smoke-require probe for kit $targetkit (packages: $kit_smokerequires) - executing freshly built artifact via its tclsh subcommand" - ::punkboot::kit_smoke_require_probe $buildfolder/$targetkit $targetkit $kit_smokerequires + ::punkboot::kit_smoke_require_probe $kit_builddir/$targetkit $targetkit $kit_smokerequires } } @@ -8209,14 +8888,13 @@ foreach vfstail $vfs_tails { $vfs_event destroy $vfs_installer destroy - #TODO - allow building of kits for other platforms - # - we need to use /bin for only kits targetting current platform, - # and use /bin/ for others - # that we we can have same target executable for multiple platforms e.g executable just named 'punkshell' - # - to do this we need to change runtime/mapvfs.config to have platform names not just standalone target executable names - # (todo: change to toml format at same time) + #Cross-platform kit outputs: RETIRED TODO (G-127 - see + #goals/G-127-crosstarget-vfs-bake.md). Kits for this host's default + #target deploy flat to /bin exactly as always; any other + #target's kit deploys under /bin/kits// (the + #mapping side gained platform names + toml format under G-122/G-024). after 200 - set deployment_folder [file dirname $sourcefolder]/bin + set deployment_folder $kit_deploydir file mkdir $deployment_folder # -- ---------- @@ -8228,7 +8906,7 @@ foreach vfstail $vfs_tails { #set last_completion [$bin_event targetset_last_complete] $bin_event targetset_addsource $deployment_folder/$targetkit ;#add target as a source of metadata for change detection - $bin_event targetset_addsource $buildfolder/$targetkit + $bin_event targetset_addsource $kit_builddir/$targetkit $bin_event targetset_started # -- ---------- @@ -8251,16 +8929,16 @@ foreach vfstail $vfs_tails { } puts stdout "copying.." - puts stdout "$buildfolder/$targetkit" + puts stdout "$kit_builddir/$targetkit" puts stdout "to:" puts stdout "$deployment_folder/$targetkit" after 300 - file copy $buildfolder/$targetkit $deployment_folder/$targetkit + file copy $kit_builddir/$targetkit $deployment_folder/$targetkit #G-057: the kit resource sidecar ships with the kit - if {[file isfile $buildfolder/$targetkit.resources.toml]} { - file copy -force $buildfolder/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml + if {[file isfile $kit_builddir/$targetkit.resources.toml]} { + file copy -force $kit_builddir/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml } - lappend installed_kits $targetkit + lappend installed_kits $kit_relpath # -- ---------- $bin_event targetset_end OK # -- ---------- @@ -8271,7 +8949,7 @@ foreach vfstail $vfs_tails { #G-057: keep the deployed resource sidecar current even when the kit #itself is unchanged (the record can be newer than the kit rebuild - #e.g the sidecar feature arriving, or a host-capability change) - if {[::punkboot::kiticon::copy_ifchanged $buildfolder/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml] eq "copied"} { + if {[::punkboot::kiticon::copy_ifchanged $kit_builddir/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml] eq "copied"} { puts stdout " kit icon (G-057): refreshed deployed sidecar $deployment_folder/$targetkit.resources.toml" } lappend skipped_kit_installs [list kit $targetkit reason "no change detected"] @@ -8299,7 +8977,7 @@ if {[llength $failed_kits]} { } set had_kits [expr {[llength $installed_kits] || [llength $failed_kits] || [llength $skipped_kits]}] if {$had_kits} { - puts stdout " module builds and kit/zipkit builds processed (vfs config: src/runtime/mapvfs.config)" + puts stdout " module builds and kit/zipkit builds processed (kit mapping: src/runtime/mapvfs.toml)" puts stdout " - use 'make.tcl modules' to build modules without baking the vfs folders into executable kits/zipkits" puts stdout " - use 'make.tcl vfscommonupdate' to promote built modules into the base vfs folder /src/vfs/_vfscommon.vfs (promotion gate)" puts stdout " - Note that without the vfscommonupdate step, 'make.tcl bake' (the kit-assembly stage of 'make.tcl bakehouse') will build vfs based executables" @@ -8309,7 +8987,7 @@ if {$had_kits} { puts stdout " module builds processed" puts stdout "" puts stdout " If kit/zipkit based executables required - create src/vfs/.vfs folders containing lib,modules,modules_tcl9 etc folders" - puts stdout " Also ensure appropriate executables exist in src/runtime along with src/runtime/mapvfs.config" + puts stdout " Also ensure appropriate runtime executables exist in bin/runtime/ along with the kit mapping src/runtime/mapvfs.toml" } puts stdout "-done-" exit 0 diff --git a/src/project_layouts/vendor/punk/project-0.1/src/make.tcl b/src/project_layouts/vendor/punk/project-0.1/src/make.tcl index a55eda23..69267f8e 100644 --- a/src/project_layouts/vendor/punk/project-0.1/src/make.tcl +++ b/src/project_layouts/vendor/punk/project-0.1/src/make.tcl @@ -187,7 +187,7 @@ namespace eval ::punkboot { variable foldername [file tail $scriptfolder] variable pkg_requirements [list]; variable pkg_missing [list];variable pkg_loaded [list] variable help_flags [list -help --help /? -h] - variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate libfetch bootsupport vfscommonupdate projectversion workflow buildsuite tool project vfs] + variable known_commands [list bakehouse packages modules libs bake bakelist vfslibs bin info check shell vendorupdate libfetch bootsupport vfscommonupdate projectversion workflow buildsuite tool] } @@ -627,11 +627,15 @@ proc ::punkboot::lib::bootsupport_prompt_yesno {prompt} { } # -- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- -# Parsed kit-mapping model (G-121). One reader for src/runtime/mapvfs.config - -# the 'bake' kit machinery, the 'bakelist' report and the bake/bakelist argdoc -# choices all consume the model these helpers return, never the file format -# directly, so a config-format change (G-024 toml direction) swaps in underneath -# by reimplementing mapvfs_parse alone. +# Parsed kit-mapping model (G-121). One reader family for the kit-output mapping +# config - the 'bake' kit machinery, the 'bakelist' report and the bake/bakelist +# argdoc choices all consume the model these helpers return, never the file +# format directly. G-024: the canonical format is src/runtime/mapvfs.toml +# (tomlish-parsed [kit.] / [group.] / [scheme.] tables - the +# file's own header comment is the user-facing spec); the pre-G-024 line format +# src/runtime/mapvfs.config remains readable as a DEPRECATED fallback so +# un-migrated trees (older generated projects) keep baking. mapvfs_locate picks +# the file (toml wins when both exist); mapvfs_parse dispatches on extension. # # mapvfs_parse: pure parse - no output, no exits. Every store address, presence # probe and artifact suffix it computes is keyed by the entry's TARGET platform @@ -639,39 +643,423 @@ proc ::punkboot::lib::bootsupport_prompt_yesno {prompt} { # (bin/runtime) and each entry addresses $rtbase/. Returns a dict: # exists 0|1 (mapfile present) # mapfile path as supplied +# format toml|legacy (which reader produced the model) # default_target target platform for entries that declare none -# runtime_vfs_map runtime -> raw vfs_specs as listed (appname/type/target may be "") -# vfs_runtime_map vfstail -> list of {runtime appname type target smokerequires} -# (defaults applied; only vfs folders that exist on disk - matching -# the historical inline parse the kit loop was built around; -# smokerequires is the G-133 smoke-require package list, may be "") +# runtime_vfs_map runtime -> list of vfs_specs. legacy: raw specs as listed +# (appname/type/target may be ""). toml: normalized 6-element +# specs {vfstail appname type target smokerequires attrs} +# vfs_runtime_map vfstail -> list of {runtime appname type target smokerequires +# ?attrs?} (defaults applied; only vfs folders that exist on +# disk - matching the historical inline parse the kit loop was +# built around; smokerequires is the G-133 smoke-require package +# list, may be ""). attrs (6th element, toml models only - read +# via mapvfs_spec_attrs) is a dict: entry (config table label +# e.g kit.punk91), group, bake_default, scheme, scheme_role. # runtime_target runtime -> resolved target platform (all runtimes, mapped order) +# groups group name -> {description bake_default 0|1} (toml only) # missing names of referenced-but-absent runtimes/vfs folders +# missing_detail structured missing items (toml only): list of dicts +# {kind runtime|vfs label name target } +# - the bake path's strict per-entry failure reporting (G-024) # warnings warning lines in encounter order, ready to print verbatim # (missing-item lines carry their historical WARNING: prefix) # badentries subset of warnings: malformed entry lines (these have no kit # record to carry them, so bakelist prints them; missing-item # warnings are instead surfaced as per-row state there) -# configerrors fatal config problems (duplicate runtime line; unusable or +# configerrors fatal config problems (malformed/unresolvable declarations, +# each naming its entry; duplicate runtime line; unusable or # self-contradicting target platform; caller decides whether to # abort - the bake path keeps its historical exit 3) -proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} { - set model [dict create\ + +# Locate the kit-mapping config for a src/runtime folder (G-024). +# Precedence: PUNK_MAPVFS_CONFIG env override (test/characterization seam - an +# alternate config exercised without editing the real one; relative paths +# resolve against the runtime folder), then mapvfs.toml, then the deprecated +# mapvfs.config. When neither exists the toml name is returned so no-mapping +# messages name the current format. +proc ::punkboot::lib::mapvfs_locate {rt_sourcefolder} { + if {[info exists ::env(PUNK_MAPVFS_CONFIG)] && $::env(PUNK_MAPVFS_CONFIG) ne ""} { + set override $::env(PUNK_MAPVFS_CONFIG) + if {[file pathtype $override] ne "absolute"} { + set override [file join $rt_sourcefolder $override] + } + return $override + } + if {[file exists $rt_sourcefolder/mapvfs.toml]} { + return $rt_sourcefolder/mapvfs.toml + } + if {[file exists $rt_sourcefolder/mapvfs.config]} { + return $rt_sourcefolder/mapvfs.config + } + return $rt_sourcefolder/mapvfs.toml +} + +#tomlish::to_dict values are type-tagged (e.g {type STRING value x}, {type ARRAY value {...}}); +#tables are plain nested dicts. Shared by the mapvfs toml reader, the per-.vfs +#payload declarations (G-115) and any other build-side toml consumption. +proc ::punkboot::lib::toml_untag {tagged} { + if {[dict get $tagged type] eq "ARRAY"} { + set plain [list] + foreach el [dict get $tagged value] { + lappend plain [::punkboot::lib::toml_untag $el] + } + return $plain + } + return [dict get $tagged value] +} + +# Read + parse a toml file via tomlish. Returns {ok } or {error }. +proc ::punkboot::lib::toml_file_to_dict {path} { + if {[catch {package require tomlish} tomlish_err]} { + return [list error "tomlish package unavailable ($tomlish_err)"] + } + if {[catch { + set fd [open $path r] + fconfigure $fd -encoding utf-8 + set tomldata [read $fd] + close $fd + } readerr]} { + catch {close $fd} + return [list error "cannot read $path ($readerr)"] + } + if {[catch {tomlish::to_dict [tomlish::from_toml $tomldata]} parsed]} { + return [list error "toml parse failed for $path\n$parsed"] + } + return [list ok $parsed] +} + +# attrs element of a parsed vfs_spec tuple (6th element, toml models). Legacy +# models carry no attrs - callers get an empty dict and defaults apply +# (group "", bake_default 1, no scheme role). +proc ::punkboot::lib::mapvfs_spec_attrs {vfs_spec} { + if {[llength $vfs_spec] >= 6} { + return [lindex $vfs_spec 5] + } + return [dict create] +} + +proc ::punkboot::lib::mapvfs_model_new {mapfile default_target format} { + return [dict create\ exists 0\ mapfile $mapfile\ + format $format\ default_target $default_target\ runtime_vfs_map [dict create]\ vfs_runtime_map [dict create]\ runtime_target [dict create]\ + groups [dict create]\ missing [list]\ + missing_detail [list]\ warnings [list]\ badentries [list]\ configerrors [list]\ ] +} + +proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} { + if {[string tolower [file extension $mapfile]] eq ".toml"} { + return [mapvfs_parse_toml $mapfile $rtbase $sourcefolder $default_target] + } + return [mapvfs_parse_config $mapfile $rtbase $sourcefolder $default_target] +} + +# G-024 canonical reader: mapvfs.toml. Strict per-entry validation - a malformed +# or unresolvable declaration lands in configerrors naming its table (the bake +# path aborts on configerrors; missing-on-disk runtime/vfs items land in +# warnings + missing_detail and the bake path decides fatality per entry). +proc ::punkboot::lib::mapvfs_parse_toml {mapfile rtbase sourcefolder default_target} { + set model [mapvfs_model_new $mapfile $default_target toml] + if {![file exists $mapfile]} { + return $model + } + dict set model exists 1 + set legacy_sibling [file join [file dirname $mapfile] mapvfs.config] + if {[file exists $legacy_sibling] && [file tail $mapfile] eq "mapvfs.toml"} { + dict lappend model warnings "NOTE: both mapvfs.toml and mapvfs.config present in [file dirname $mapfile] - mapvfs.toml wins and the deprecated mapvfs.config is ignored (remove it, or keep it only as reference)." + } + lassign [toml_file_to_dict $mapfile] tstatus tpayload + if {$tstatus ne "ok"} { + dict lappend model configerrors "CONFIG FILE ERROR. $mapfile: $tpayload" + return $model + } + set kit_types [list kit zip zipcat cookit cookfs] + set groups [dict create] + set configerrors [list] + set warnings [list] + set missing [list] + set missing_detail [list] + #top-level tables: kit / group / scheme only + dict for {topkey -} $tpayload { + if {$topkey ni {kit group scheme}} { + lappend configerrors "CONFIG FILE ERROR. $mapfile: unknown top-level table or key '$topkey' (expected \[kit.\], \[group.\], \[scheme.\])" + } + } + #-- groups + if {[dict exists $tpayload group]} { + dict for {gname gentry} [dict get $tpayload group] { + set label "group.$gname" + set gdesc "" + set gdefault 1 + dict for {k v} $gentry { + switch -- $k { + description {set gdesc [toml_untag $v]} + bake_default { + set gdefault [toml_untag $v] + if {![string is boolean -strict $gdefault]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: 'bake_default' must be boolean (got '$gdefault')" + set gdefault 1 + } + } + default { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown key '$k' (expected description, bake_default)" + } + } + } + dict set groups $gname [dict create description $gdesc bake_default [expr {bool($gdefault)}]] + } + } + #-- normalized candidate entries: {label rtname vfstail appname kittype declared_target smokes attrs} + set entries [list] + set appnames_seen [dict create] ;#appname -> label + set entry_reader {{model_varname label entry required_type} { + #shared [kit.*]/[scheme.*] field reader; returns a dict or empty on fatal field errors + upvar 1 configerrors configerrors kit_types kit_types groups groups mapfile mapfile + set fields [dict create runtime "" vfs "" type "" target "" smokerequire [list] group "" bake_default "" scheme "" prefix ""] + set badentry 0 + dict for {k v} $entry { + switch -- $k { + runtime - vfs - type - target - group - scheme - prefix { + dict set fields $k [::punkboot::lib::toml_untag $v] + } + smokerequire { + if {[dict get $v type] eq "ARRAY"} { + dict set fields smokerequire [::punkboot::lib::toml_untag $v] + } else { + dict set fields smokerequire [list [::punkboot::lib::toml_untag $v]] + } + } + bake_default { + set bd [::punkboot::lib::toml_untag $v] + if {![string is boolean -strict $bd]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: 'bake_default' must be boolean (got '$bd')" + set badentry 1 + } else { + dict set fields bake_default [expr {bool($bd)}] + } + } + default { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown key '$k'" + set badentry 1 + } + } + } + foreach reqkey {runtime vfs} { + if {[dict get $fields $reqkey] eq ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile is missing required key '$reqkey'" + set badentry 1 + } + } + set ktype [dict get $fields type] + if {$ktype eq ""} { + if {$required_type} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile is missing required key 'type' (scheme entries declare their kit type explicitly; one of: [join $kit_types {, }])" + set badentry 1 + } else { + dict set fields type kit + } + } elseif {$ktype ni $kit_types} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown kit type '$ktype' (expected one of: [join $kit_types {, }])" + set badentry 1 + } + set declared [dict get $fields target] + if {$declared ne ""} { + set problem [::punkboot::lib::platform_name_problem $declared] + if {$problem ne ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile declares target platform '$declared' - $problem." + set badentry 1 + } + } + #strip a superfluous windows .exe from the runtime name (legacy-format parity) + set rtname [dict get $fields runtime] + if {[string match -nocase *.exe $rtname]} { + dict set fields runtime [string range $rtname 0 end-4] + } + set g [dict get $fields group] + if {$g ne "" && ![dict exists $groups $g]} { + #a group referenced without its own [group.] table is implicitly plain + dict set groups $g [dict create description "" bake_default 1] + } + if {$badentry} { + return [dict create] + } + return $fields + }} + if {[dict exists $tpayload kit]} { + dict for {kname kentry} [dict get $tpayload kit] { + set label "kit.$kname" + set fields [apply $entry_reader model $label $kentry 0] + if {![dict size $fields]} { + continue + } + foreach badkey {scheme prefix} { + if {[dict get $fields $badkey] ne ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: key '$badkey' is only valid in \[scheme.\] entries" + } + } + set g [dict get $fields group] + set bd [dict get $fields bake_default] + if {$bd eq ""} { + set bd [expr {$g ne "" ? [dict get $groups $g bake_default] : 1}] + } + if {[dict exists $appnames_seen $kname]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: output name '$kname' already produced by [dict get $appnames_seen $kname]" + continue + } + dict set appnames_seen $kname $label + lappend entries [list $label [dict get $fields runtime] [dict get $fields vfs] $kname\ + [dict get $fields type] [dict get $fields target] [dict get $fields smokerequire]\ + [dict create entry $label group $g bake_default $bd scheme "" scheme_role ""]] + } + } + #-- schemes (G-023/G-024: generative entries expanded at parse time - no version enumeration) + set project_version "" + set project_version_err "" + if {[dict exists $tpayload scheme]} { + dict for {sname sentry} [dict get $tpayload scheme] { + set label "scheme.$sname" + set fields [apply $entry_reader model $label $sentry 1] + if {![dict size $fields]} { + continue + } + set skind [dict get $fields scheme] + if {$skind eq ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile is missing required key 'scheme' (known schemes: versioned)" + continue + } + if {$skind ne "versioned"} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: unknown scheme '$skind' (known schemes: versioned)" + continue + } + #versioned scheme (G-023): output names derive from the punkproject.toml + #project version at parse time - punk-, punk-dev and the + #release-gated plain punk - so one declaration tracks the version with + #no per-version config edits. + if {$project_version eq "" && $project_version_err eq ""} { + set pptoml [file join [file dirname $sourcefolder] punkproject.toml] + lassign [toml_file_to_dict $pptoml] pstatus ppayload + if {$pstatus ne "ok"} { + set project_version_err $ppayload + } elseif {[catch {toml_untag [dict get $ppayload project version]} pver]} { + set project_version_err "no \[project\] version field readable in $pptoml ($pver)" + } elseif {$pver eq ""} { + set project_version_err "empty \[project\] version field in $pptoml" + } else { + set project_version $pver + } + } + if {$project_version eq ""} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: cannot expand 'versioned' scheme - $project_version_err" + continue + } + set prefix [dict get $fields prefix] + if {$prefix eq ""} { + set prefix $sname + } + set g [dict get $fields group] + set bd [dict get $fields bake_default] + if {$bd eq ""} { + set bd [expr {$g ne "" ? [dict get $groups $g bake_default] : 1}] + } + foreach {appname role} [list ${prefix}-$project_version versioned ${prefix}-dev dev $prefix release] { + if {[dict exists $appnames_seen $appname]} { + lappend configerrors "CONFIG FILE ERROR. entry $label in $mapfile: expanded output name '$appname' already produced by [dict get $appnames_seen $appname]" + continue + } + dict set appnames_seen $appname $label + lappend entries [list "$label ($role)" [dict get $fields runtime] [dict get $fields vfs] $appname\ + [dict get $fields type] [dict get $fields target] [dict get $fields smokerequire]\ + [dict create entry $label group $g bake_default $bd scheme $sname scheme_role $role]] + } + } + } + #-- per-runtime target resolution (target is a property of the RUNTIME - entries + #may repeat the declaration but must not disagree; legacy-format parity) + set runtime_target [dict create] + set runtime_first_label [dict create] + foreach e $entries { + lassign $e label rtname - - - declared - - + if {![dict exists $runtime_first_label $rtname]} { + dict set runtime_first_label $rtname $label + } + if {$declared eq ""} { + continue + } + if {![dict exists $runtime_target $rtname] || [dict get $runtime_target $rtname] eq $default_target} { + dict set runtime_target $rtname $declared + } elseif {[dict get $runtime_target $rtname] ne $declared} { + lappend configerrors "CONFIG FILE ERROR. runtime: $rtname declares conflicting target platforms ('[dict get $runtime_target $rtname]' and '$declared') in $mapfile - a runtime has one target." + } + } + foreach rtname [dict keys $runtime_first_label] { + if {![dict exists $runtime_target $rtname]} { + dict set runtime_target $rtname $default_target + } + } + #-- runtime presence (one check per runtime, in first-reference order) + foreach rtname [dict keys $runtime_first_label] { + if {$rtname eq "-"} { + continue + } + set rt_target [dict get $runtime_target $rtname] + set rtdir [file join $rtbase [platform_store_tier $rt_target]] + set runtime_test $rtname[platform_exe_suffix $rt_target] + if {![file exists [file join $rtdir $runtime_test]]} { + lappend warnings "WARNING: Missing runtime file $rtdir/$runtime_test (entry [dict get $runtime_first_label $rtname] in $mapfile)" + lappend missing $rtname + lappend missing_detail [dict create kind runtime label [dict get $runtime_first_label $rtname] name $rtname target $rt_target] + } + } + #-- build the maps (entry order preserved; vfs existence filtering matches the + #historical inline parse the kit loop was built around) + set runtime_vfs_map [dict create] + set vfs_runtime_map [dict create] + foreach e $entries { + lassign $e label rtname vfstail appname kittype - smokes attrs + set rt_target [dict get $runtime_target $rtname] + set spec [list $vfstail $appname $kittype $rt_target $smokes $attrs] + dict lappend runtime_vfs_map $rtname $spec + if {![file isdirectory [file join $sourcefolder vfs $vfstail]]} { + lappend warnings "WARNING: Missing vfs folder [file join $sourcefolder vfs $vfstail] specified in $mapfile for entry $label" + if {$vfstail ni $missing} { + lappend missing $vfstail + } + lappend missing_detail [dict create kind vfs label $label name $vfstail target $rt_target] + } else { + dict lappend vfs_runtime_map $vfstail [list $rtname $appname $kittype $rt_target $smokes $attrs] + } + } + dict set model runtime_vfs_map $runtime_vfs_map + dict set model vfs_runtime_map $vfs_runtime_map + dict set model runtime_target $runtime_target + dict set model groups $groups + dict set model missing $missing + dict set model missing_detail $missing_detail + dict set model warnings [concat [dict get $model warnings] $warnings] + dict set model configerrors $configerrors + return $model +} + +# DEPRECATED legacy reader: the pre-G-024 mapvfs.config line format +# ( {vfsfolder ?kitname? ?kittype? ?targetplatform? ?smokerequires?} ...). +# Kept so un-migrated trees (older generated projects) keep baking; behaviour is +# unchanged from before the toml conversion apart from the deprecation NOTE. +proc ::punkboot::lib::mapvfs_parse_config {mapfile rtbase sourcefolder default_target} { + set model [mapvfs_model_new $mapfile $default_target legacy] if {![file exists $mapfile]} { return $model } dict set model exists 1 + dict lappend model warnings "NOTE: $mapfile uses the deprecated mapvfs.config line format - convert to mapvfs.toml (the file header of src/runtime/mapvfs.toml in the punkshell tree is the format spec; when both files exist the toml file wins)." set fdmap [open $mapfile r] fconfigure $fdmap -translation binary set mapdata [read $fdmap] @@ -765,7 +1153,7 @@ proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} dict set model vfs_runtime_map $vfs_runtime_map dict set model runtime_target $runtime_target dict set model missing $missing - dict set model warnings $warnings + dict set model warnings [concat [dict get $model warnings] $warnings] dict set model badentries $badentries dict set model configerrors $configerrors return $model @@ -778,8 +1166,10 @@ proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} # TARGET .exe suffix, duplicate appname -> _) so listed names # match what a bake produces. Entries whose vfs folder is missing on disk never # reach the kit loop - they are appended after the buildable set so 'bakelist' -# can surface them as broken config. Duplicate-name disambiguation assumes the -# config keeps appnames unique (as the kit loop itself effectively does). +# can surface them as broken config. Duplicate-name disambiguation is per TARGET +# (G-127): same-named kits for different targets coexist under their own output +# tiers; a same-target duplicate assumes the config keeps appnames unique (as +# the kit loop itself effectively does). # $rtbase is the store ROOT (bin/runtime): each record resolves its own tier # under it, so entries targeting different platforms report their own store # presence (G-122). Returns a list of record dicts: @@ -792,9 +1182,18 @@ proc ::punkboot::lib::mapvfs_parse {mapfile rtbase sourcefolder default_target} # runtime_dir store tier folder holding the runtime ("" for "-") # target target platform (e.g win32-x86_64) # store_tier bin/runtime tier folder name for the target +# out_tier kit OUTPUT tier relative to bin/ and src/_build/ - "" for the +# default (host) target's flat locations, "kits/" for any +# other target (G-127) # vfs vfs folder tail (e.g punk9wintk903.vfs) # vfs_present 1|0 # smokerequire G-133 smoke-require package list as configured ("" when undeclared) +# entry config table label (toml models, e.g kit.punk91 / "scheme.punk9 (dev)"; "" legacy) +# group G-024 named grouping ("" when ungrouped) +# bake_default 1|0 - 0 means excluded from full (no-name) bakes; still bakeable +# by name or @group selection +# scheme / scheme_role scheme-expanded outputs: source scheme name and role +# (versioned|dev|release); "" for explicit entries proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { set runtime_vfs_map [dict get $model runtime_vfs_map] set vfs_runtime_map [dict get $model vfs_runtime_map] @@ -808,7 +1207,7 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { } set records [list] set exe_names_seen [list] - set record_pair {{rtname vfstail appname target_kit_type smokerequires} { + set record_pair {{rtname vfstail appname target_kit_type smokerequires attrs} { upvar 1 records records exe_names_seen exe_names_seen rtbase rtbase sourcefolder sourcefolder\ runtime_target runtime_target default_target default_target if {$appname eq ""} { @@ -834,14 +1233,16 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { set runtime_file $rtname[::punkboot::lib::platform_exe_suffix $target] set targetkit ${appname}[::punkboot::lib::platform_exe_suffix $target] set kitname $appname - if {$targetkit in $exe_names_seen} { - #duplicate appname configured - kit loop disambiguates the same way + if {[list $target $targetkit] in $exe_names_seen} { + #duplicate appname configured for the same target - kit loop + #disambiguates the same way (G-127: different-target namesakes + #coexist under their own kits// output tiers instead) set targetkit ${appname}_$rtname set kitname ${appname}_$rtname } set runtime_present [file exists [file join $runtime_dir $runtime_file]] } - lappend exe_names_seen $targetkit + lappend exe_names_seen [list $target $targetkit] lappend records [dict create\ kitname $kitname\ targetkit $targetkit\ @@ -852,17 +1253,29 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { runtime_dir $runtime_dir\ target $target\ store_tier $store_tier\ + out_tier [expr {$target eq $default_target ? {} : "kits/$target"}]\ vfs $vfstail\ vfs_present [file isdirectory [file join $sourcefolder vfs $vfstail]]\ smokerequire $smokerequires\ + entry [expr {[dict exists $attrs entry] ? [dict get $attrs entry] : ""}]\ + group [expr {[dict exists $attrs group] ? [dict get $attrs group] : ""}]\ + bake_default [expr {[dict exists $attrs bake_default] ? [dict get $attrs bake_default] : 1}]\ + scheme [expr {[dict exists $attrs scheme] ? [dict get $attrs scheme] : ""}]\ + scheme_role [expr {[dict exists $attrs scheme_role] ? [dict get $attrs scheme_role] : ""}]\ ] }} foreach vfstail $vfs_tails { if {![dict exists $vfs_runtime_map $vfstail]} { continue ;#unmapped vfs folder - the kit loop derives no output names from it } + set rtnames_done [list] ;#several entries may share a runtime on one vfs (toml models) - + #enumerate each distinct runtime's spec set exactly once foreach rt_app [dict get $vfs_runtime_map $vfstail] { set rtname [lindex $rt_app 0] + if {$rtname in $rtnames_done} { + continue + } + lappend rtnames_done $rtname if {![dict exists $runtime_vfs_map $rtname]} { continue } @@ -871,58 +1284,85 @@ proc ::punkboot::lib::mapvfs_kit_outputs {model rtbase sourcefolder} { if {$configured_vfs ne $vfstail} { continue } - apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires + apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires [::punkboot::lib::mapvfs_spec_attrs $vfs_app] } } } #entries whose vfs folder is missing (excluded from vfs_runtime_map by the parse) dict for {rtname vfs_specs} $runtime_vfs_map { foreach vfsconfig $vfs_specs { - if {[llength $vfsconfig] < 1 || [llength $vfsconfig] > 5} { + if {[llength $vfsconfig] < 1 || [llength $vfsconfig] > 6} { continue } lassign $vfsconfig vfstail appname target_kit_type - smokerequires if {[dict exists $vfs_runtime_map $vfstail]} { continue ;#already enumerated above } - apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires + apply $record_pair $rtname $vfstail $appname $target_kit_type $smokerequires [::punkboot::lib::mapvfs_spec_attrs $vfsconfig] } } return $records } # Match user-supplied kit names against mapvfs_kit_outputs records. -# Accepts the base kit name (punk91) or the platform artifact name (punk91.exe, -# name.kit); matching is case-insensitive when the HOST filesystem is (a name -# the user types is a host-side convenience - a cygwin-family host is as +# Accepts the base kit name (punk91), the platform artifact name (punk91.exe, +# name.kit), or a G-024 group selector @ (selects every record in the +# group). A base name matching several records - one per target, coexisting under +# their own output tiers (G-127) - selects them all. +# Matching is case-insensitive when the HOST filesystem is (a name the +# user types is a host-side convenience - a cygwin-family host is as # case-insensitive as a native windows one, whatever it targets). Returns a dict: # selected matched records (kit-output order, deduplicated) # unknown supplied names that matched nothing proc ::punkboot::lib::mapvfs_match_outputs {kit_outputs names} { set on_windows [expr {[info exists ::punkboot::host_windows] ? $::punkboot::host_windows : ("windows" eq $::tcl_platform(platform))}] + set name_eq {{on_windows a b} { + expr {$on_windows ? [string equal -nocase $a $b] : [string equal $a $b]} + }} set selected_idx [list] set unknown [list] foreach name $names { + set found 0 + if {[string match @?* $name]} { + #group selector: every record carrying the group (G-024) + set gname [string range $name 1 end] + set i 0 + foreach rec $kit_outputs { + if {[dict get $rec group] ne "" && [apply $name_eq $on_windows [dict get $rec group] $gname]} { + set found 1 + if {$i ni $selected_idx} { + lappend selected_idx $i + } + } + incr i + } + if {!$found} { + lappend unknown $name + } + continue + } set base $name if {[string match -nocase *.exe $base] || [string match -nocase *.kit $base]} { set base [string range $base 0 end-4] } - set found 0 set i 0 foreach rec $kit_outputs { + set rec_match 0 foreach candidate [list [dict get $rec kitname] [dict get $rec targetkit]] { if {$candidate eq ""} {continue} - if {($on_windows && [string equal -nocase $candidate $base]) || (!$on_windows && $candidate eq $base)\ - || ($on_windows && [string equal -nocase $candidate $name]) || (!$on_windows && $candidate eq $name)} { - set found 1 + if {[apply $name_eq $on_windows $candidate $base] || [apply $name_eq $on_windows $candidate $name]} { + set rec_match 1 break } } - if {$found} { + if {$rec_match} { + #G-127: one kit NAME may legitimately match several records - one per + #target, coexisting under their own output tiers - so a name selects + #them ALL (like the @group branch), not just the first hit + set found 1 if {$i ni $selected_idx} { lappend selected_idx $i } - break } incr i } @@ -2110,7 +2550,7 @@ proc ::punkboot::punkboot_gethelp {args} { append h " - includes the vfslibs phase; does not rebuild modules/libs and does not run the promotion gates" \n append h " - with kitname arguments, bakes and deploys only the named configured kits (unknown names error before any build)" \n \n append h " $scriptname bakelist ?kitname ...?" \n - append h " - list the kit outputs configured in src/runtime/mapvfs.config: name, kit type, runtime (with presence)," \n + append h " - list the kit outputs configured in src/runtime/mapvfs.toml: name, kit type, runtime (with presence)," \n append h " vfs folder and deployed state (bin copy absent/current/stale/nobuild vs the src/_build product)" \n append h " - kitname arguments filter to the named entries (per-kit detail)" \n \n append h " $scriptname modules" \n @@ -2163,9 +2603,6 @@ proc ::punkboot::punkboot_gethelp {args} { append h " reported and build/test exit nonzero with fetch guidance (bin/punk-getzig.cmd)" \n \n append h " $scriptname help ?subcommand?" \n append h " - show usage for $scriptname or one of its subcommands (equivalent: $scriptname -help)" \n \n - append h " Deprecated aliases (transition window):" \n - append h " $scriptname project -> runs bakehouse (with a one-line notice)" \n - append h " $scriptname vfs -> runs bake (with a one-line notice)" \n \n append h " Flags:" \n append h " -confirm 0|1" \n append h " - interactive y/n confirmation policy (default 1: prompt when stdin is a terminal, abort fast when it is not)." \n @@ -2209,7 +2646,6 @@ gates, one bake) recipe; -dirty-abort 0 overrides). DEVELOPER (changing payload): pass the gates explicitly - the commit is the publishing act, not a build step. That is the RELEASE SEQUENCE below. - Deprecated aliases (transition window): 'project' -> bakehouse, 'vfs' -> bake. RELEASE SEQUENCE (the developer order that satisfies the staleness + provenance gates) ---------------------------------------------------------------------------- @@ -2278,19 +2714,32 @@ DIAGRAM 1b - THIN-LAYOUT SYNC (part of modules/libs/packages/bakehouse runs) DIAGRAM 2 - KIT ASSEMBLY DETAIL (the 'make.tcl bake' stage; incl. vfslibs phase) ---------------------------------------------------------------------------- - src/runtime/mapvfs.config which vfs folder pairs with which runtime -> kit name, - and optionally the TARGET PLATFORM of that runtime [K9] - 'make.tcl bakelist ?kitname ...?' reports the configured - kit outputs (runtime/vfs presence + deployed state); - 'make.tcl bake ?kitname ...?' bakes only the named kits - (bare bake = all; unknown names error before any build) + src/runtime/mapvfs.toml which vfs folder pairs with which runtime -> kit name + ([kit.] tables; tomlish-parsed - G-024), incl. the + runtime's TARGET PLATFORM [K9], smoke-require packages, + named groups and generative version-named schemes + (deprecated mapvfs.config line format still readable) + 'make.tcl bakelist ?name|@group ...?' reports the + configured kit outputs (presence + deployed state); + 'make.tcl bake ?name|@group ...?' bakes only the named + kits (bare bake = all entries except bake_default=false + ones; unknown names error before any build) bin/runtime// bare Tcl runtime (tclkit / tclsfe / suite build), in the store tier of the TARGET platform it builds kits for [K9] src/vfs/_vfscommon.vfs/ common payload (built modules + libs, main boot support) src/vfs/.vfs/ per-kit overlay (main.tcl, kit-only modules/config) - - _vfscommon.vfs .vfs - | | + src/vfs/.vfs.toml optional per-.vfs payload declaration (G-115): [payload.*] + entries materialized INTO the .vfs folder by the vfslibs + phase (vendor trees / bin/packages tier), punkcheck- + tracked with drop-in-wins precedence - undeclared or + hand-modified files are preserved, never overwritten. + A source containing %platform% materializes once per + CONSUMING KIT TARGET (set derived from mapvfs) into + _targets// staging; each kit's merged image + then gets only its own target's subtree (G-127) + + _vfscommon.vfs .vfs <-- payload declarations (if any) already + | | materialized by the vfslibs phase +-----------+----------+ | copy common, then merge overlay over it v @@ -2349,7 +2798,8 @@ LIBRARY-ARTIFACT FETCH + CONSUMPTION (G-139) | consumption (two lanes, both punkcheck-tracked): | PACKAGES_tcl phase of modules/libs/packages/bakehouse | -> lib_tcl/allplatforms + lib_tcl/; - | vfslibs entries with source_root = "packages" + | per-.vfs payload declarations (src/vfs/.vfs.toml, + | G-115) with source_root = "packages" | -> declared kit vfs lib_tcl trees (see DIAGRAM 2) v lib_tcl/... + src/vfs/.vfs/lib_tcl/... (embedded records ride along @@ -2401,7 +2851,9 @@ KEY / NOTES [K8] What is VCS-tracked where (checkin targets after a build): tracked: src/** (sources, src/bootsupport, src/project_layouts copies, src/vfs/_vfscommon.vfs), punkproject.toml, CHANGELOG.md - untracked: modules*/ lib*/ at projectroot, src/_build/, bin/.exe + untracked: modules*/ lib*/ at projectroot, src/_build/, bin/.exe, + bin/kits// (cross-target kit outputs - G-127), + src/vfs/*.vfs/_targets/ (per-platform payload staging) (bin/ scripts/tools are tracked; kit executables are not) [K9] HOST vs TARGET platform (G-122). What a bake EMITS is keyed by the target @@ -2410,11 +2862,18 @@ KEY / NOTES whether the pre-deploy sweep uses tasklist/taskkill or ps/kill. The default target is this host's own platform - except a cygwin-family host (an msys2/cygwin-runtime tclsh, which reports platform 'unix' on windows), which - targets win32. A mapvfs.config entry may declare its own target as a 4th - element and is then read from that platform's tier, named with its executable - convention, and (being unrunnable here) skipped by the process sweep. + targets win32. A mapvfs.toml entry may declare its own target (target key; + legacy config: 4th element) and is then read from that platform's tier, named + with its executable convention, and (being unrunnable here) skipped by the + process sweep. 'make.tcl check' prints the derivation; 'make.tcl bakelist' shows per-kit targets, with a target= note on rows that are not on the default target. + OUTPUT locations are target-keyed too (G-127): default-target kits keep the + flat src/_build/ + bin/ locations, any other target's kit builds + and deploys under the kits// tier of both (bin/kits/ = outputs, + beside bin/runtime/ = inputs), so same-named kits for different targets + coexist instead of overwriting each other; bakelist marks such rows with + out=kits// and its detail block prints the tiered paths. [K10] BOOT-PRECONDITION gate (G-125). A kit whose merged vfs has no tcl library cannot initialise at all ('Cannot find a usable init.tcl'), so the bake @@ -2440,8 +2899,8 @@ KEY / NOTES cross-target kits are covered) against the kit's target platform; wrong-arch libraries outside platform-discriminated subdirs (win32-ix86/, win-x64/ etc - multi-arch payloads are legitimate there) earn recapped BUILD-WARNINGs. - (b) Smoke-require probe: a kit declaring packages (mapvfs.config 5th entry - element) has each one plain-'package require'd INSIDE the freshly built + (b) Smoke-require probe: a kit declaring packages (mapvfs.toml smokerequire + key; legacy config: 5th element) has each one plain-'package require'd INSIDE the freshly built artifact via its tclsh subcommand - the only check that sees resolution-order defects (a wrong-arch higher-versioned package shadowing a working copy); cross-target kits skip with a stated reason, undeclared kits run nothing new. @@ -2556,8 +3015,8 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO modules "Build .tm modules from src/modules, src/vendormodules etc into /modules etc" libs "Build pkgIndex.tcl libraries from src/lib, src/vendorlib etc into /lib etc" bake "Assemble kit/zipkit executables from promoted payload (src/vfs) and runtimes into /bin" - bakelist "List the kit outputs configured in src/runtime/mapvfs.config: type, runtime/vfs presence, deployed state" - vfslibs "Propagate declared platform-library packages (vendored or bin/packages tier) into kit vfs lib_tcl trees" + bakelist "List the kit outputs configured in src/runtime/mapvfs.toml: type, runtime/vfs presence, deployed state, groups" + vfslibs "Materialize per-.vfs payload declarations (src/vfs/.vfs.toml - G-115) into kit vfs folders; drop-ins preserved" bin "Install executables from src/bin into /bin, then build kits as for bake" vendorupdate "Update src/vendormodules based on src/vendormodules/include_modules.config" libfetch "Fetch declared punkbin lib-tier library artifacts into bin/packages/ (sha1-verified) and materialize the package trees" @@ -2571,8 +3030,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO help "Show usage for make.tcl or one of its subcommands" buildsuite "List, describe or run the defined buildsuites under src/buildsuites (zig runtime factory)" tool "List, test or build the vendored tools under src/tools (zig) and install them to /bin" - project "DEPRECATED alias for bakehouse (one-line notice, then runs bakehouse)" - vfs "DEPRECATED alias for bake (one-line notice, then runs bake)" } #@cmd -help bodies for the constructed definitions below. Authored BLOCK-FORM #(first value line whitespace-only) so the @normalize directive in each @@ -2620,25 +3077,28 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO build each configured kit from the promoted payload (src/vfs/_vfscommon.vfs + custom vfs folders) and its runtime, and install to /bin. - With kitname arguments, bake and deploy ONLY the named configured - kits (see 'make.tcl bakelist' for the configured names) - other - kits' build products, bin copies and punkcheck records are left - untouched. An unknown name errors before any build, listing the - configured names. Bare 'bake' processes all configured kits. - Includes the vfslibs phase (declared vendored platform-library - propagation into kit vfs lib_tcl trees) so a bake cannot ship - stale binary libs; a selective bake narrows that phase to the - named kits' vfs folders. + With kitname (or @groupname - G-024 groups) arguments, bake and + deploy ONLY the named configured kits (see 'make.tcl bakelist' + for the configured names and groups) - other kits' build + products, bin copies and punkcheck records are left untouched. + An unknown name errors before any build, listing the configured + names. Bare 'bake' processes all configured kits except + bake_default=false entries (those bake only by name/@group). + Includes the vfslibs phase (per-.vfs payload declarations + src/vfs/.vfs.toml materialized into the kit vfs folders - + G-115) so a bake cannot ship stale binary libs; a selective + bake narrows that phase to the named kits' vfs folders. Does not rebuild modules/libs and does not run the promotion gates - built modules reach the kit payload only via 'make.tcl vfscommonupdate' (committed for provenance). Consumer full path from a clean checkout: 'make.tcl bakehouse'." bakelist " - List the kit outputs configured in src/runtime/mapvfs.config (the - same parsed mapping the bake subcommand consumes): kit name, kit - type (kit|zip|zipcat...), runtime with presence in the runtime - store of that kit's TARGET platform + List the kit outputs configured in src/runtime/mapvfs.toml (the + same parsed mapping the bake subcommand consumes; the deprecated + mapvfs.config line format is honoured when no toml file exists): + kit name, kit type (kit|zip|zipcat...), runtime with presence in + the runtime store of that kit's TARGET platform (/bin/runtime/), vfs folder, and the deployed state of /bin/ vs the src/_build build product: @@ -2651,20 +3111,31 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO materialized from an older revision than a -r artifact beside it) and marks cross-target entries (target= on any kit whose mapvfs entry declares a target other than this host's - default - 'make.tcl check' shows that default). - kitname arguments filter the report to the named entries (per-kit - detail, with resolved store tier, target and paths). Reporting - only - never builds." + default - 'make.tcl check' shows that default), G-024 groups + (group=), entries excluded from full bakes (default=no) + and scheme-expanded outputs (scheme=versioned|dev|release). + Non-default-target kits build and deploy under the + kits// tier of src/_build and bin (G-127) - their rows + carry out=kits// and the deployed state compares the + tiered paths; a kit NAME shared by entries for several targets + selects (and lists) every one of them. + kitname (or @groupname) arguments filter the report to the named + entries (per-kit detail, with resolved store tier, target, + config entry and paths). Reporting only - never builds." vfslibs " - Propagate declared platform-library packages into kit vfs - lib_tcl trees - from vendored src/ folders (source_root - 'src', the default) or from the bin/packages punkbin lib-tier - input (source_root 'packages', populated by 'make.tcl - libfetch' - G-139). - Declarations (per-package per-kit, with superseded-version - removal and clean-slate same-name 'replace') live in - src/runtime/vendorlib_vfs.toml. + Materialize the per-.vfs payload declarations (G-115): each + src/vfs/.vfs.toml declares package folders to install + INTO src/vfs/.vfs - from vendored src/ folders + (source_root 'src', the default) or from the bin/packages + punkbin lib-tier input (source_root 'packages', populated by + 'make.tcl libfetch' - G-139) - with explicit superseded-version + removal and clean-slate same-name 'replace'. + punkcheck-tracked with drop-in-wins precedence: files the + mechanism did not install (or that changed since it installed + them) are preserved and reported, never overwritten; a .vfs + with no declaration file is untouched. Format + precedence: + src/vfs/README.md. Also runs automatically as a phase of 'make.tcl bake' (and therefore of 'make.tcl bakehouse'); this standalone narrowing serves quick iteration on the declarations." @@ -2806,16 +3277,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO (bin/punk-getzig.cmd). Build caches (.zig-cache/zig-out) are deliberately left in place between runs for rebuild speed (both git- and fossil-ignored)." - project - " - DEPRECATED alias for 'bakehouse' (G-112 stage-true rename) - prints a - one-line notice and runs bakehouse (packages + bake, -dirty-abort - default ON). Retained for a transition window." - vfs - " - DEPRECATED alias for 'bake' (G-112 stage-true rename) - prints a - one-line notice and runs bake (kit assembly including the vfslibs - phase). Retained for a transition window." } #shared option fragments composed into the per-subcommand definitions below variable OPT_CONFIRM { @@ -2849,7 +3310,7 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "Terminate running processes matching the executable being built (if applicable) so the build can install over it." } - #bakehouse (and its deprecated alias project) defaults the dirty gate ON: + #bakehouse defaults the dirty gate ON: #the bakehouse bakes from the committed recipe (G-112 two-persona model). variable OPT_DIRTYABORT_ON { -dirty-abort -type boolean -default 1 -help\ @@ -2904,8 +3365,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO check {} projectversion {} workflow {} - project {OPT_FORCEKILL OPT_DIRTYABORT_ON OPT_CONFIRM} - vfs {OPT_FORCEKILL OPT_DIRTYABORT OPT_CONFIRM} } #synopsis fragments matching the shared option fragments (explicit @form -synopsis #so usage shows the real invocation 'make.tcl ...' rather than the @@ -2919,18 +3378,22 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO OPT_TRUSTSERVER "?-trust-server?" OPT_FORCE "?-force?" } - #per-subcommand positional values (default: none). bake (and its deprecated - #alias vfs) and bakelist take optional kit names (G-121 selective bake). + #per-subcommand positional values (default: none). bake and bakelist take + #optional kit names (G-121 selective bake). #Discoverability nicety: the configured kit names are offered as choices when #the mapping parses at definition time - informational only (-choicerestricted #0), so the handlers' own validation (which lists the configured names, and #still works when the mapping cannot be parsed here) stays authoritative. variable KITNAME_CHOICES [list] catch { - set _mapmodel [punkboot::lib::mapvfs_parse [file join $::punkboot::scriptfolder runtime mapvfs.config] "" $::punkboot::scriptfolder $::punkboot::target_platform] + set _mapmodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate [file join $::punkboot::scriptfolder runtime]] "" $::punkboot::scriptfolder $::punkboot::target_platform] foreach _rec [punkboot::lib::mapvfs_kit_outputs $_mapmodel "" $::punkboot::scriptfolder] { lappend KITNAME_CHOICES [dict get $_rec kitname] } + #G-024 group selectors join the informational choices (@) + foreach _g [dict keys [dict get $_mapmodel groups]] { + lappend KITNAME_CHOICES @$_g + } } set _kitname_choicepart "" if {[llength $KITNAME_CHOICES]} { @@ -2941,22 +3404,23 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "@values -min 0 -max -1"\ "kitname -type string -optional 1 -multiple 1${_kitname_choicepart} -help\ { - Configured kit output(s) to bake and deploy - see 'make.tcl bakelist' - for the configured names. With no kitname, all configured kits are - processed. An unknown name errors before any build.}"\ + Configured kit output(s) to bake and deploy - a kit name, or + @ for every kit in a configured group (G-024) - see + 'make.tcl bakelist' for the configured names and groups. With no + kitname, all configured kits are processed except entries with + bake_default=false. An unknown name errors before any build.}"\ ] - dict set SUBVALUES vfs [dict get $SUBVALUES bake] dict set SUBVALUES bakelist [list\ "@values -min 0 -max -1"\ "kitname -type string -optional 1 -multiple 1${_kitname_choicepart} -help\ { - Configured kit output(s) to report on (filters the listing - doubles + Configured kit output(s) to report on - a kit name, or @ + for every kit in a configured group (filters the listing - doubles as per-kit detail). With no kitname, all configured entries are listed.}"\ ] variable VALUES_SYNOPSES { bake "?kitname ...?" - vfs "?kitname ...?" bakelist "?kitname ...?" } foreach {sub optvars} $SUBOPTS { @@ -3034,7 +3498,6 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO "vendored tools" {tool} "informational" {info check bakelist projectversion workflow help} "interactive" {shell} - "deprecated aliases" {project vfs} } punk::args::define\ "@id -id (script)::punkboot"\ @@ -3144,7 +3607,7 @@ if {$::punkboot::punkargs_ok} { } } if {[dict exists $argd values kitname]} { - #declared -multiple: a list of the supplied kit names (bake/vfs/bakelist - G-121) + #declared -multiple: a list of the supplied kit names (bake/bakelist - G-121) set ::punkboot::opt_kitnames [dict get $argd values kitname] } unset -nocomplain _opts _optname _var @@ -3204,10 +3667,10 @@ if {$::punkboot::punkargs_ok} { lappend commands_found $a } } - #G-121: bake/bakelist (and the deprecated alias vfs) take optional trailing kit - #names - collect them here so the single-command check below still holds in - #degraded mode (the handlers do their own name validation). - if {[lindex $commands_found 0] in {bake bakelist vfs}} { + #G-121: bake/bakelist take optional trailing kit names - collect them here so + #the single-command check below still holds in degraded mode (the handlers do + #their own name validation). + if {[lindex $commands_found 0] in {bake bakelist}} { set ::punkboot::opt_kitnames [lrange $commands_found 1 end] set commands_found [lrange $commands_found 0 0] } @@ -3260,23 +3723,7 @@ if {$do_help} { exit $help_exitcode } -#G-112 deprecated aliases - one-line mapping before any dispatch: -# project -> bakehouse (packages + bake, -dirty-abort default ON) -# vfs -> bake (kit assembly including the vfslibs phase) -#Kept for a transition window (until the 8.6 arc completes - see the G-112 -#dispositions). The alias definitions above mirror their targets' options so -#flags parse identically; only the command word maps here. if {[info exists ::punkboot::command]} { - switch -- $::punkboot::command { - project { - puts stderr "make.tcl: 'project' is DEPRECATED - use 'bakehouse' (packages + bake; refuses uncommitted src by default). Running bakehouse." - set ::punkboot::command bakehouse - } - vfs { - puts stderr "make.tcl: 'vfs' is DEPRECATED - use 'bake' (kit assembly incl. the vfslibs phase). Running bake." - set ::punkboot::command bake - } - } if {!$::punkboot::punkargs_ok && $::punkboot::command eq "bakehouse"} { #degraded fallback scan has flag-only -dirty-abort (no way to pass 0): #bakehouse's documented default is ON, so force it here. @@ -3386,7 +3833,7 @@ set ::punkboot::bake_selected_vfs [list] ;#vfs folder tails the selected set ::punkboot::bake_selected_runtimes [list] ;#unsuffixed runtime names the selected kits wrap if {$::punkboot::command eq "bake" && [llength $::punkboot::opt_kitnames]} { set _rtbase $binfolder/runtime - set _mapmodel [punkboot::lib::mapvfs_parse $sourcefolder/runtime/mapvfs.config $_rtbase $sourcefolder $::punkboot::target_platform] + set _mapmodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate $sourcefolder/runtime] $_rtbase $sourcefolder $::punkboot::target_platform] if {[llength [dict get $_mapmodel configerrors]]} { foreach _errline [dict get $_mapmodel configerrors] { puts stderr $_errline @@ -3399,8 +3846,11 @@ if {$::punkboot::command eq "bake" && [llength $::punkboot::opt_kitnames]} { puts stderr "make.tcl bake: unknown kit name(s): [join [dict get $_matchinfo unknown] {, }] - nothing built" if {[llength $_kit_outputs]} { puts stderr " configured kit outputs: [join [lmap _r $_kit_outputs {dict get $_r kitname}] {, }]" + if {[dict size [dict get $_mapmodel groups]]} { + puts stderr " configured groups (select with @): [join [dict keys [dict get $_mapmodel groups]] {, }]" + } } else { - puts stderr " no kit outputs are configured (src/runtime/mapvfs.config)" + puts stderr " no kit outputs are configured (src/runtime/mapvfs.toml)" } foreach _n [dict get $_matchinfo unknown] { if {[string match -* $_n]} { @@ -3740,7 +4190,7 @@ if {$::punkboot::command eq "check"} { if {$_chk_suffix eq ""} {set _chk_suffix "(none)"} puts stdout "platform (G-122): host=$::punkboot::host_platform target=$::punkboot::target_platform store=bin/runtime/[punkboot::lib::platform_store_tier $::punkboot::target_platform] exe-suffix=$_chk_suffix process-tooling=[punkboot::lib::platform_process_family $::punkboot::target_platform]" puts stdout " (host = this tclsh's punk::platform canon, from tcl_platform(os)='$::tcl_platform(os)' platform='$::tcl_platform(platform)'; target = default kit target - a cygwin-family host (msys2/cygwin runtime on windows) bakes win32 kits)" - puts stdout " (mapvfs.config entries may declare their own target platform as a 4th element - 'make.tcl bakelist' shows each kit's target, store tier and artifact name)" + puts stdout " (mapvfs.toml entries may declare their own target platform (target key) - 'make.tcl bakelist' shows each kit's target, store tier and artifact name)" unset -nocomplain _chk_suffix puts stdout $sep puts stdout "module/library checks - paths from bootsupport only" @@ -3950,8 +4400,8 @@ if {$::punkboot::command eq "check"} { puts stdout " bake will proceed with a NOTE and cannot flag a file-relative kit payload - run 'make.tcl modules' then 'make.tcl bootsupport'." } set _smoke_declared [list] - if {[file exists $sourcefolder/runtime/mapvfs.config]} { - set _smokemodel [punkboot::lib::mapvfs_parse $sourcefolder/runtime/mapvfs.config [file dirname $sourcefolder]/bin/runtime $sourcefolder $::punkboot::target_platform] + if {[file exists [punkboot::lib::mapvfs_locate $sourcefolder/runtime]]} { + set _smokemodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate $sourcefolder/runtime] [file dirname $sourcefolder]/bin/runtime $sourcefolder $::punkboot::target_platform] foreach _rec [punkboot::lib::mapvfs_kit_outputs $_smokemodel [file dirname $sourcefolder]/bin/runtime $sourcefolder] { if {[llength [dict get $_rec smokerequire]]} { lappend _smoke_declared "[dict get $_rec kitname] ([join [dict get $_rec smokerequire] {, }])" @@ -3960,7 +4410,7 @@ if {$::punkboot::command eq "check"} { unset -nocomplain _smokemodel _rec } puts stdout "smoke-require probe (G-133, advisory): freshly built host-runnable kits declaring packages" - puts stdout " (mapvfs.config 5th entry element) have each one plain-'package require'd inside the built" + puts stdout " (mapvfs.toml smokerequire key; legacy mapvfs.config 5th entry element) have each one plain-'package require'd inside the built" puts stdout " artifact via its tclsh subcommand - the only check that sees resolution-order defects such as" puts stdout " a wrong-arch higher-versioned package shadowing a working copy. Failures are recapped" puts stdout " BUILD-WARNINGs; cross-target kits skip with a stated reason; undeclared kits run nothing new." @@ -4167,7 +4617,7 @@ if {$::punkboot::command eq "bakelist"} { set rtbase $binfolder/runtime set rtfolder $rtbase/$rt_os_arch set buildfolder $sourcefolder/_build - set mapfile $sourcefolder/runtime/mapvfs.config + set mapfile [punkboot::lib::mapvfs_locate $sourcefolder/runtime] set mapmodel [punkboot::lib::mapvfs_parse $mapfile $rtbase $sourcefolder $rt_default_target] foreach errline [dict get $mapmodel configerrors] { puts stderr $errline @@ -4175,6 +4625,15 @@ if {$::punkboot::command eq "bakelist"} { foreach badline [dict get $mapmodel badentries] { puts stderr $badline } + if {[dict get $mapmodel format] eq "legacy"} { + #deprecation NOTE (first warnings line) - bakelist is reporting-only, so + #surface it here rather than relying on the bake path's warning printout + foreach warnline [dict get $mapmodel warnings] { + if {[string match NOTE:* $warnline]} { + puts stderr $warnline + } + } + } if {![dict get $mapmodel exists]} { puts stdout "no kit mapping config at $mapfile - no kit outputs configured" exit 0 @@ -4200,7 +4659,7 @@ if {$::punkboot::command eq "bakelist"} { if {$t eq $rt_os_arch} { puts stderr "WARNING: runtime store folder bin/runtime/$t does not exist - every configured runtime on the default target will read runtime=missing." puts stderr " Default target '$rt_default_target' derives from THIS tclsh: tcl_platform(os)='$::tcl_platform(os)' tcl_platform(platform)='$::tcl_platform(platform)' -> host canon '$::punkboot::host_platform' -> target '$rt_default_target'." - puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.config for runtimes stored elsewhere." + puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.toml for runtimes stored elsewhere." } else { puts stderr "WARNING: runtime store folder bin/runtime/$t does not exist - configured entries targeting $t will read runtime=missing." } @@ -4211,6 +4670,9 @@ if {$::punkboot::command eq "bakelist"} { if {[llength [dict get $matchinfo unknown]]} { puts stderr "bakelist: unknown kit name(s): [join [dict get $matchinfo unknown] {, }]" puts stderr " configured kit outputs: [join [lmap r $kit_outputs {dict get $r kitname}] {, }]" + if {[dict size [dict get $mapmodel groups]]} { + puts stderr " configured groups (select with @): [join [dict keys [dict get $mapmodel groups]] {, }]" + } exit 1 } set reportset [dict get $matchinfo selected] @@ -4222,7 +4684,14 @@ if {$::punkboot::command eq "bakelist"} { set kit_type [dict get $rec kit_type] set runtime [dict get $rec runtime] set vfstail [dict get $rec vfs] - set deployed [punkboot::lib::kit_deploy_state $buildfolder $binfolder [dict get $rec targetkit]] + #G-127: deploy-state roots follow the record's output tier (non-default + #targets live under kits// in both bin/ and src/_build/) + set out_tier [dict get $rec out_tier] + if {$out_tier eq ""} { + set deployed [punkboot::lib::kit_deploy_state $buildfolder $binfolder [dict get $rec targetkit]] + } else { + set deployed [punkboot::lib::kit_deploy_state $buildfolder/$out_tier $binfolder/$out_tier [dict get $rec targetkit]] + } set notes [list] if {$runtime ne "-" && ![dict get $rec runtime_present]} { lappend notes runtime=missing @@ -4234,6 +4703,18 @@ if {$::punkboot::command eq "bakelist"} { #cross-target entry: its runtime, store tier and artifact naming follow #its own platform, not this host's default lappend notes target=[dict get $rec target] + #G-127: and its build/deploy locations sit under the kits// tier + lappend notes out=$out_tier/ + } + if {[dict get $rec group] ne ""} { + lappend notes group=[dict get $rec group] + } + if {![dict get $rec bake_default]} { + #excluded from full bakes - bake by name or @group (G-024) + lappend notes default=no + } + if {[dict get $rec scheme_role] ne ""} { + lappend notes scheme=[dict get $rec scheme_role] } if {$runtime ne "-" && [dict get $rec runtime_present]} { set rtstale [punkboot::lib::runtime_materialization_check [dict get $rec runtime_dir] [dict get $rec runtime_file]] @@ -4256,7 +4737,15 @@ if {$::punkboot::command eq "bakelist"} { set widths $newwidths } set rt_store_note [expr {$rt_store_missing ? " (FOLDER MISSING)" : ""}] - puts stdout "# mapvfs: src/runtime/mapvfs.config runtimes: bin/runtime/$rt_os_arch$rt_store_note deployed: bin/ vs src/_build/" + if {[info exists ::env(PUNK_MAPVFS_CONFIG)] && $::env(PUNK_MAPVFS_CONFIG) ne ""} { + set map_display "$mapfile (PUNK_MAPVFS_CONFIG override)" + } else { + set map_display "src/runtime/[file tail $mapfile]" + if {[dict get $mapmodel format] eq "legacy"} { + append map_display " (deprecated format)" + } + } + puts stdout "# mapvfs: $map_display runtimes: bin/runtime/$rt_os_arch$rt_store_note deployed: bin/ vs src/_build/ (out= rows: under kits//)" set headline "" foreach h $headings w $widths { append headline [format "%-*s " $w $h] @@ -4289,9 +4778,18 @@ if {$::punkboot::command eq "bakelist"} { } puts stdout " vfs folder: src/vfs/[dict get $rec vfs] [expr {[dict get $rec vfs_present] ? {(present)} : {(MISSING)}}]" puts stdout " kit type: [dict get $rec kit_type]" - puts stdout " target: [dict get $rec target][expr {[dict get $rec target] eq $rt_default_target ? " (host default)" : " (declared in mapvfs.config)"}]" - puts stdout " build product: src/_build/[dict get $rec targetkit] [punkboot::lib::bakelist_file_detail $buildfolder/[dict get $rec targetkit]]" - puts stdout " deployed: bin/[dict get $rec targetkit] [punkboot::lib::bakelist_file_detail $binfolder/[dict get $rec targetkit]] state=[lindex $row 4]" + puts stdout " target: [dict get $rec target][expr {[dict get $rec target] eq $rt_default_target ? " (host default)" : " (declared in the kit mapping)"}]" + if {[dict get $rec entry] ne ""} { + puts stdout " config entry: \[[dict get $rec entry]\]" + } + if {[dict get $rec group] ne ""} { + puts stdout " group: [dict get $rec group][expr {[dict get $rec bake_default] ? "" : " (bake_default=false - bake by name or @[dict get $rec group])"}]" + } elseif {![dict get $rec bake_default]} { + puts stdout " bake_default: false (excluded from full bakes - bake by name to build)" + } + set relkit [expr {[dict get $rec out_tier] eq "" ? [dict get $rec targetkit] : "[dict get $rec out_tier]/[dict get $rec targetkit]"}] + puts stdout " build product: src/_build/$relkit [punkboot::lib::bakelist_file_detail $buildfolder/$relkit]" + puts stdout " deployed: bin/$relkit [punkboot::lib::bakelist_file_detail $binfolder/$relkit] state=[lindex $row 4]" } } exit 0 @@ -5349,20 +5847,8 @@ if {$::punkboot::command eq "libfetch"} { puts stderr "$A(BAD)LIBFETCH: failed to parse $libfetch_config\n$libfetch_dict$A(RST)" exit 3 } - if {![llength [info commands ::punkboot::lib::toml_untag]]} { - #tomlish::to_dict values are type-tagged (also defined by the vfslibs - #phase - that block does not run for the libfetch command) - proc ::punkboot::lib::toml_untag {tagged} { - if {[dict get $tagged type] eq "ARRAY"} { - set plain [list] - foreach el [dict get $tagged value] { - lappend plain [::punkboot::lib::toml_untag $el] - } - return $plain - } - return [dict get $tagged value] - } - } + #tomlish::to_dict values are type-tagged - ::punkboot::lib::toml_untag (defined + #with the shared toml helpers near the top of this script) resolves them set libfetch_artifacts [dict create] if {[dict exists $libfetch_dict artifact]} { set libfetch_artifacts [dict get $libfetch_dict artifact] @@ -5683,135 +6169,222 @@ if {$::punkboot::command in {bakehouse packages libs}} { } if {$::punkboot::command in {bakehouse bake vfslibs}} { - #G-037: propagate declared vendored platform-library packages into kit vfs lib_tcl trees. - #Participation is per-package per-kit (kit vfs package sets intentionally differ - e.g. twapi - #versions vary per kit) - declared in src/runtime/vendorlib_vfs.toml, never blanket-copied. - #punkcheck records live at src/vfs/.punkcheck so they are not baked into kit payloads. - set vfslibs_config $sourcefolder/runtime/vendorlib_vfs.toml - if {![file exists $vfslibs_config]} { - puts stdout "VFSLIBS: no config at $vfslibs_config - skipping vendorlib -> kit vfs propagation" - } else { - if {[catch {package require tomlish} tomlish_err]} { - puts stderr "$A(BAD)VFSLIBS: cannot process $vfslibs_config - tomlish package unavailable ($tomlish_err)$A(RST)" + #G-115: declarative .vfs payload composition. Each src/vfs/.vfs may have a + #SIBLING declaration file src/vfs/.vfs.toml declaring payload package folders + #to materialize INTO it - while the folder remains the operative assembly area: + #drop-ins need no declaration and are never clobbered. The declaration file's own + #header comment (and src/vfs/README.md) is the user-facing spec. Entry keys per + #[payload.]: source (folder path, resolved per source_root), source_root + #('src' default - vendor trees / suite build products under src/; or 'packages' - + #the consent-gated punkbin lib tier at /bin/packages populated by + #'make.tcl libfetch'), target (folder path INSIDE the .vfs, e.g "lib_tcl9"; + #default the .vfs root), supersedes (explicit legacy sibling folder removals), + #replace (boolean clean-slate of the SAME-NAMED target folder before install). + #A source containing %platform% is a PER-PLATFORM entry (G-127): one copy per + #consuming kit target (derived from the kit mapping below), staged under + #_targets// and selected into each kit's merged image at bake time. + #Materialization is punkcheck-tracked (records in src/vfs/.punkcheck, outside kit + #payloads) with -overwrite synced-targets: a target file punkcheck did not install, + #or that changed since it was installed (a drop-in or hand edit), is PRESERVED and + #reported, never overwritten - the G-115 drop-in-wins precedence; supersedes and + #replace are the explicit declared exceptions. A .vfs with no declaration file is + #untouched (pure drop-in mode, exactly as before G-115). + #This phase supersedes the per-package src/runtime/vendorlib_vfs.toml surface + #(G-037/G-139): its [install.] entries were migrated into per-.vfs files + #2026-07-31 - rationale in goals/G-115-declarative-vfs-composition.md. + set legacy_vfslibs $sourcefolder/runtime/vendorlib_vfs.toml + if {[file exists $legacy_vfslibs]} { + puts stderr "WARNING: $legacy_vfslibs is superseded by per-.vfs payload declarations (src/vfs/.vfs.toml - G-115) and is IGNORED - migrate its \[install.*\] entries into per-.vfs files and remove it." + } + set payload_files [lsort [glob -nocomplain -dir $sourcefolder/vfs -types f -tail *.vfs.toml]] + if {![llength $payload_files]} { + puts stdout "VFSPAYLOAD: no per-.vfs payload declarations (src/vfs/.vfs.toml) - nothing to materialize" + } + #G-127 per-platform payload derivation: a payload source containing %platform% + #declares ONE payload materialized per CONSUMING KIT TARGET - the platform set + #is derived from the kit mapping (every mapvfs kit output pairing this vfs, + #whatever its bake_default), never declared a second time in the payload file. + #Each platform's copy lands under _targets/// inside the .vfs + #folder (VCS-ignored staging); the bake selects the consuming kit's subtree + #into its merged image and drops the staging tree, so an assembled kit carries + #only its own target's payload. + set vfspayload_kit_targets [dict create] + if {[llength $payload_files]} { + set _pmapmodel [punkboot::lib::mapvfs_parse [punkboot::lib::mapvfs_locate $sourcefolder/runtime] $binfolder/runtime $sourcefolder $::punkboot::target_platform] + if {[llength [dict get $_pmapmodel configerrors]]} { + puts stderr "WARNING: VFSPAYLOAD: kit-mapping config errors - per-platform (%platform%) payload derivation unavailable this run (the mapping's own gates report the errors)" + } else { + foreach _prec [punkboot::lib::mapvfs_kit_outputs $_pmapmodel $binfolder/runtime $sourcefolder] { + if {![dict exists $vfspayload_kit_targets [dict get $_prec vfs]] + || [dict get $_prec target] ni [dict get $vfspayload_kit_targets [dict get $_prec vfs]]} { + dict lappend vfspayload_kit_targets [dict get $_prec vfs] [dict get $_prec target] + } + } + } + unset -nocomplain _pmapmodel _prec + } + foreach ptail $payload_files { + set pfile $sourcefolder/vfs/$ptail + set vfstail [file rootname $ptail] ;#punkdeclare.vfs.toml -> punkdeclare.vfs + if {[llength $::punkboot::bake_selected_kitnames] && $vfstail ni $::punkboot::bake_selected_vfs} { + #G-121 selective bake: narrow to the selected kits' vfs folders (skip + #precedes the existence checks so an unselected declaration's problem + #cannot abort a selective run) + puts stdout "VFSPAYLOAD $vfstail: skipping (selective bake - not a selected kit's vfs)" + continue + } + if {![file isdirectory $sourcefolder/vfs/$vfstail]} { + puts stderr "$A(BAD)VFSPAYLOAD: declaration $pfile has no matching vfs folder at $sourcefolder/vfs/$vfstail$A(RST)" exit 3 } - set fd [open $vfslibs_config r] - fconfigure $fd -encoding utf-8 - set vfslibs_tomldata [read $fd] - close $fd - if {[catch {tomlish::to_dict [tomlish::from_toml $vfslibs_tomldata]} vfslibs_dict]} { - puts stderr "$A(BAD)VFSLIBS: failed to parse $vfslibs_config\n$vfslibs_dict$A(RST)" + lassign [punkboot::lib::toml_file_to_dict $pfile] pstatus ppayload + if {$pstatus ne "ok"} { + puts stderr "$A(BAD)VFSPAYLOAD: $ppayload$A(RST)" exit 3 } - #tomlish::to_dict values are type-tagged (e.g {type STRING value x}, {type ARRAY value {...}}) - proc ::punkboot::lib::toml_untag {tagged} { - if {[dict get $tagged type] eq "ARRAY"} { - set plain [list] - foreach el [dict get $tagged value] { - lappend plain [::punkboot::lib::toml_untag $el] - } - return $plain + dict for {topkey -} $ppayload { + if {$topkey ne "payload"} { + puts stderr "$A(BAD)VFSPAYLOAD: $pfile: unknown top-level table or key '$topkey' (expected \[payload.\] entries)$A(RST)" + exit 3 } - return [dict get $tagged value] } - set vfslibs_installs [dict create] - if {[dict exists $vfslibs_dict install]} { - set vfslibs_installs [dict get $vfslibs_dict install] + set payload_entries [dict create] + if {[dict exists $ppayload payload]} { + set payload_entries [dict get $ppayload payload] } - if {![dict size $vfslibs_installs]} { - puts stdout "VFSLIBS: no \[install.\] entries found in $vfslibs_config" + if {![dict size $payload_entries]} { + puts stdout "VFSPAYLOAD $vfstail: no \[payload.\] entries in $ptail" + continue } - dict for {iname ientry} $vfslibs_installs { - foreach reqkey {source targets} { - if {![dict exists $ientry $reqkey]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname in $vfslibs_config is missing required key '$reqkey'$A(RST)" - exit 3 - } - } - set isource [punkboot::lib::toml_untag [dict get $ientry source]] - set itargets [punkboot::lib::toml_untag [dict get $ientry targets]] - set isupersedes [list] - if {[dict exists $ientry supersedes]} { - set isupersedes [punkboot::lib::toml_untag [dict get $ientry supersedes]] - } - #source_root (G-139): 'src' (default - the original vendorlib - #declarations) or 'packages' - resolve against the untracked - #bin/packages lib-tier input (populated by 'make.tcl libfetch'). + dict for {iname ientry} $payload_entries { + set entrylabel "payload.$iname ($ptail)" + set isource "" set isource_root "src" - if {[dict exists $ientry source_root]} { - set isource_root [punkboot::lib::toml_untag [dict get $ientry source_root]] - } - #replace (G-139): clean-slate a SAME-NAMED existing target folder - #before install. supersedes deliberately cannot name the folder being - #installed; replace covers same-name refreshes where the old copy may - #carry strays the new tree lacks (e.g. hand-drops nesting md5c/tcllibc - #inside tcllib2.0). + set itarget "" + set isupersedes [list] set ireplace 0 - if {[dict exists $ientry replace]} { - set ireplace [punkboot::lib::toml_untag [dict get $ientry replace]] - if {![string is boolean -strict $ireplace]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname 'replace' must be boolean (got '$ireplace')$A(RST)" - exit 3 + dict for {k v} $ientry { + switch -- $k { + source {set isource [punkboot::lib::toml_untag $v]} + source_root {set isource_root [punkboot::lib::toml_untag $v]} + target {set itarget [punkboot::lib::toml_untag $v]} + supersedes {set isupersedes [punkboot::lib::toml_untag $v]} + replace { + set ireplace [punkboot::lib::toml_untag $v] + if {![string is boolean -strict $ireplace]} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel: 'replace' must be boolean (got '$ireplace')$A(RST)" + exit 3 + } + } + default { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel: unknown key '$k' (expected source, source_root, target, supersedes, replace)$A(RST)" + exit 3 + } } } - switch -- $isource_root { - src { - set source_pkg_folder $sourcefolder/$isource - set source_missing_hint "" - } - packages { - set source_pkg_folder $projectroot/bin/packages/$isource - set source_missing_hint " - populate the bin/packages tier first: 'tclsh src/make.tcl libfetch' (declarations in src/runtime/libpackages.toml)" - } - default { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname has unknown source_root '$isource_root' (expected src|packages)$A(RST)" - exit 3 - } + if {$isource eq ""} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel is missing required key 'source'$A(RST)" + exit 3 + } + if {$itarget ne "" && ([file pathtype $itarget] ne "relative" || ".." in [file split $itarget])} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel target '$itarget' must be a relative path inside the .vfs folder (no '..')$A(RST)" + exit 3 } - if {![file isdirectory $source_pkg_folder]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname source '$isource' not found at $source_pkg_folder$source_missing_hint$A(RST)" + #G-127 per-platform axis: %platform% in 'source' selects the payload per + #consuming kit target (set derived above); 'target' stays literal - its + #per-platform staging prefix (_targets//) is implicit. + if {[string match *%platform%* $itarget]} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $entrylabel: 'target' must not contain %platform% - the per-platform staging prefix (_targets//) is implicit for a %platform% source$A(RST)" exit 3 } - set pkgtail [file tail $source_pkg_folder] - foreach t $itargets { - set target_vfs_dir [lindex [file split $t] 0] - if {[llength $::punkboot::bake_selected_kitnames] && $target_vfs_dir ni $::punkboot::bake_selected_vfs} { - #G-121 selective bake: narrow the vfslibs phase to the selected kits' - #vfs folders (skip precedes the existence check so an unselected - #entry's problem cannot abort a selective run) - puts stdout "VFSLIBS install.$iname: skipping vfs/$t (selective bake - not a selected kit's vfs)" + if {[string match *%platform%* $isource]} { + if {![dict exists $vfspayload_kit_targets $vfstail] + || ![llength [dict get $vfspayload_kit_targets $vfstail]]} { + puts stderr "WARNING: VFSPAYLOAD entry $entrylabel declares a per-platform source (%platform%) but no kit in the mapping consumes $vfstail - no platform set to derive, entry skipped" continue } - if {![file isdirectory $sourcefolder/vfs/$target_vfs_dir]} { - puts stderr "$A(BAD)VFSLIBS: entry install.$iname target '$t' - no vfs folder at $sourcefolder/vfs/$target_vfs_dir$A(RST)" + set entry_platforms [dict get $vfspayload_kit_targets $vfstail] + } else { + set entry_platforms [list ""] ;#platform-neutral entry - materialize once, directly in the folder + } + foreach eplatform $entry_platforms { + if {$eplatform eq ""} { + set esource $isource + set elabel $entrylabel + set evfsprefix "" + } else { + set esource [string map [list %platform% $eplatform] $isource] + set elabel "$entrylabel \[$eplatform\]" + set evfsprefix _targets/$eplatform + } + switch -- $isource_root { + src { + set source_pkg_folder $sourcefolder/$esource + set source_missing_hint "" + } + packages { + set source_pkg_folder $projectroot/bin/packages/$esource + set source_missing_hint " - populate the bin/packages tier first: 'tclsh src/make.tcl libfetch' (declarations in src/runtime/libpackages.toml)" + } + default { + puts stderr "$A(BAD)VFSPAYLOAD: entry $elabel has unknown source_root '$isource_root' (expected src|packages)$A(RST)" + exit 3 + } + } + if {![file isdirectory $source_pkg_folder]} { + puts stderr "$A(BAD)VFSPAYLOAD: entry $elabel source '$esource' not found at $source_pkg_folder$source_missing_hint$A(RST)" exit 3 } - set target_lib_folder $sourcefolder/vfs/$t + set pkgtail [file tail $source_pkg_folder] + set etarget_subpath $itarget + if {$evfsprefix ne ""} { + set etarget_subpath [expr {$itarget eq "" ? $evfsprefix : "$evfsprefix/$itarget"}] + } + if {$etarget_subpath eq ""} { + set target_lib_folder $sourcefolder/vfs/$vfstail + set target_display vfs/$vfstail + } else { + set target_lib_folder $sourcefolder/vfs/$vfstail/$etarget_subpath + set target_display vfs/$vfstail/$etarget_subpath + } file mkdir $target_lib_folder #supersession first - no silent mixed-version provision (G-035 concerns) foreach sup $isupersedes { if {$sup eq $pkgtail} { - continue ;#never remove what we are about to install + continue ;#never remove what we are about to install - use replace for that } set suppath $target_lib_folder/$sup if {[file exists $suppath]} { - puts stdout "VFSLIBS install.$iname: removing superseded '$sup' from vfs/$t" + puts stdout "VFSPAYLOAD $elabel: removing superseded '$sup' from $target_display" file delete -force $suppath } } if {$ireplace && [file isdirectory $target_lib_folder/$pkgtail]} { - puts stdout "VFSLIBS install.$iname: replace=true - removing existing '$pkgtail' from vfs/$t (clean-slate install)" + puts stdout "VFSPAYLOAD $elabel: replace=true - removing existing '$pkgtail' from $target_display (clean-slate install)" file delete -force $target_lib_folder/$pkgtail } - puts stdout "VFSLIBS install.$iname: $isource -> vfs/$t/$pkgtail (if source file changed)" + puts stdout "VFSPAYLOAD $elabel: $isource_root:$esource -> $target_display/$pkgtail (if source changed; drop-ins/hand-edits preserved)" set resultdict [punkcheck::install $source_pkg_folder $target_lib_folder/$pkgtail\ -installer make.tcl\ -createdir 1\ - -overwrite installedsourcechanged-targets\ + -overwrite synced-targets\ -punkcheck-folder $sourcefolder/vfs\ -progresschannel stdout\ ] puts stdout [punkcheck::summarize_install_resultdict $resultdict] + #preserved-collision report: a file skipped although its SOURCE changed was + #withheld by synced-targets (a drop-in, or hand-modified since install) - + #distinct from the routine source-unchanged skips. + set _preserved [list] + foreach _sk [dict get $resultdict files_skipped] { + if {$_sk ni [dict get $resultdict sources_unchanged]} { + lappend _preserved $_sk + } + } + if {[llength $_preserved]} { + puts stdout "VFSPAYLOAD $elabel: NOTE - [llength $_preserved] changed file(s) NOT overwritten (existing target content was not installed by this mechanism, or was modified since) - the G-115 drop-in-wins precedence. Declare 'replace = true' (whole package folder) or remove the target files to hand control to the declaration." + } + unset -nocomplain _preserved _sk } } } @@ -6224,7 +6797,7 @@ proc ::punkboot::runtime_materialization_warning {rtfolder runtime_fullname} { #G-122: parsed BEFORE runtime discovery, because the mapping is what says which #store tiers this bake must look in - an entry declaring a target platform other #than the host default addresses its own bin/runtime/. -set mapfile $rt_sourcefolder/mapvfs.config +set mapfile [punkboot::lib::mapvfs_locate $rt_sourcefolder] set mapmodel [punkboot::lib::mapvfs_parse $mapfile $rtbase $sourcefolder $rt_default_target] foreach warnline [dict get $mapmodel warnings] { puts stderr $warnline @@ -6240,11 +6813,53 @@ set vfs_runtime_map [dict get $mapmodel vfs_runtime_map] set runtime_target [dict get $mapmodel runtime_target] set missing [dict get $mapmodel missing] if {[llength $missing]} { - puts stderr "WARNING [llength $missing] missing items from $mapfile. (TODO - prompt user to continue/abort)" - foreach m $missing { - puts stderr " $m" + if {[dict get $mapmodel format] eq "toml" && ![llength $::punkboot::bake_selected_kitnames]} { + #G-024 strictness (toml models): a full bake with unresolvable DEFAULT entries + #reports each one BY ENTRY rather than as a bare item list. A missing vfs + #folder is fatal - vfs folders are repo content, so its absence means a wrong + #declaration. A missing runtime FILE is environment, not declaration: store + #tiers are legitimately part-populated (fresh clones; runtimes arrive via the + #consent-gated fetch tiers), so those entries skip with a recapped + #BUILD-WARNING naming the entry (cross-target wording states the tier reason). + #bake_default=false entries are not in the full-bake set; selective bakes + #fail fast at selection instead. + set _strict_fatal [list] + set _strict_warn [list] + foreach _rec [punkboot::lib::mapvfs_kit_outputs $mapmodel $rtbase $sourcefolder] { + if {![dict get $_rec bake_default]} { + continue + } + set _entrylabel [expr {[dict get $_rec entry] ne "" ? [dict get $_rec entry] : [dict get $_rec kitname]}] + if {![dict get $_rec vfs_present]} { + lappend _strict_fatal "entry $_entrylabel: vfs folder src/vfs/[dict get $_rec vfs] is missing" + continue + } + if {[dict get $_rec runtime] ne "-" && ![dict get $_rec runtime_present]} { + if {[dict get $_rec target] eq $rt_default_target} { + lappend _strict_warn "kit [dict get $_rec kitname] (entry $_entrylabel) skipped - runtime [dict get $_rec runtime_file] not present in bin/runtime/[dict get $_rec store_tier]" + } else { + lappend _strict_warn "kit [dict get $_rec kitname] (entry $_entrylabel) skipped - cross-target runtime [dict get $_rec runtime_file] not present in bin/runtime/[dict get $_rec store_tier] (target [dict get $_rec target] tier not populated on this host)" + } + } + } + if {[llength $_strict_fatal]} { + puts stderr "CONFIG ERROR: unresolvable kit mapping entries in $mapfile - nothing built:" + foreach _line $_strict_fatal { + puts stderr " $_line" + } + exit 3 + } + if {[llength $_strict_warn]} { + ::punkboot::print_build_warnings $_strict_warn + } + unset -nocomplain _rec _entrylabel _strict_fatal _strict_warn _line + } else { + puts stderr "WARNING [llength $missing] missing items from $mapfile. (TODO - prompt user to continue/abort)" + foreach m $missing { + puts stderr " $m" + } + puts stderr "continuing..." } - puts stderr "continuing..." } # -- --- --- --- --- --- --- --- --- --- puts "-- runtime_vfs_map --" @@ -6333,7 +6948,7 @@ dict for {_rtname _rttarget} $runtime_target { dict set rtfolder_of $_file $_dir } if {[llength $cross_target_tiers]} { - puts stdout "cross-target runtime tiers in use (mapvfs.config target platforms): [join $cross_target_tiers {, }]" + puts stdout "cross-target runtime tiers in use (kit-mapping target platforms): [join $cross_target_tiers {, }]" } unset -nocomplain _rtname _rttarget _tier _dir _file if {![llength $runtimes]} { @@ -6342,7 +6957,7 @@ if {![llength $runtimes]} { if {![file isdirectory $rtfolder]} { #same store-tier self-diagnosis as bakelist puts stderr "NOTE: that runtime store folder does not exist. Default target '$rt_default_target' derives from THIS tclsh: tcl_platform(os)='$::tcl_platform(os)' tcl_platform(platform)='$::tcl_platform(platform)' -> host canon '$::punkboot::host_platform' -> target '$rt_default_target'." - puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.config for runtimes stored elsewhere." + puts stderr " Populate it with 'bin/punk-runtime.cmd' (or by hand), or declare a per-entry target platform in mapvfs.toml for runtimes stored elsewhere." } #todo - don't exit - it is valid to use runtime of - to just build a .kit/.zipkit ? exit 0 @@ -7323,6 +7938,10 @@ foreach vfstail $vfs_tails { } set rt_target [punkboot::lib::runtime_target_of $rt] set rt_file $rt[punkboot::lib::platform_exe_suffix $rt_target] + if {$rt_file in $runtimes} { + continue ;#several entries may share a runtime on one vfs (toml models) - + #process each runtime once (its spec set enumerates all its kits) + } if {[file exists $rtbase/[punkboot::lib::platform_store_tier $rt_target]/$rt_file]} { lappend runtimes $rt_file } @@ -7376,7 +7995,9 @@ foreach vfstail $vfs_tails { set applist [dict get $runtime_vfs_map $rtname] foreach vfs_app $applist { #5th element = G-133 smoke-require package list ("" when undeclared) + #6th element (toml models) = G-024 entry attrs (group/bake_default/scheme_role) lassign $vfs_app configured_vfs appname target_kit_type - kit_smokerequires + set kit_attrs [punkboot::lib::mapvfs_spec_attrs $vfs_app] if {$configured_vfs ne $vfstail} { continue } @@ -7390,15 +8011,18 @@ foreach vfstail $vfs_tails { set targetkit $appname.kit } else { set targetkit ${appname}$rt_suffix - if {$targetkit in $exe_names_seen} { - #duplicate appname configured? + if {[list $rt_target $targetkit] in $exe_names_seen} { + #duplicate appname configured for the SAME target? + #(G-127: same-named kits for DIFFERENT targets coexist under + #their own kits// output tiers - only a same-target + #duplicate needs a disambiguating name) #todo - consider creating as ${appname}(2) etc? - puts stderr "targetkit: $targetkit already seen - using name ${appname}_$rtname" + puts stderr "targetkit: $targetkit already seen for target $rt_target - using name ${appname}_$rtname" set targetkit ${appname}_$rtname } } - lappend exe_names_seen $targetkit - lappend targetkits [list $targetkit $target_kit_type $kit_smokerequires] + lappend exe_names_seen [list $rt_target $targetkit] + lappend targetkits [list $targetkit $target_kit_type $kit_smokerequires $kit_attrs] } } puts stdout " vfs: $vfstail runtime: $rtname targetkits: $targetkits" @@ -7407,7 +8031,41 @@ foreach vfstail $vfs_tails { continue ;#G-121 selective bake - not a requested kit output } puts stdout " processing targetkit: $targetkit_info" - lassign $targetkit_info targetkit target_kit_type kit_smokerequires + lassign $targetkit_info targetkit target_kit_type kit_smokerequires kit_attrs + #G-127: output locations are keyed by the kit's TARGET. Kits for this + #host's own default target keep the flat locations every launcher, test + #and habit expects (src/_build/, bin/); any other target's kit + #builds and deploys under a kits// tier so same-named kits for + #different targets coexist instead of overwriting each other across + #selective runs. bin/kits/ rather than bin// because + #bin/runtime// already means build INPUTS. + if {$rt_target eq $rt_default_target} { + set kit_builddir $buildfolder + set kit_deploydir $binfolder + set kit_relpath $targetkit ;#kit path relative to bin/ and src/_build/ + } else { + set kit_builddir $buildfolder/kits/$rt_target + set kit_deploydir $binfolder/kits/$rt_target + set kit_relpath kits/$rt_target/$targetkit + } + file mkdir $kit_builddir + #G-024: bake_default=false entries (declared per entry or via their group) + #are excluded from full bakes - explicit name/@group selection builds them. + if {![llength $::punkboot::bake_selected_kitnames] + && [dict exists $kit_attrs bake_default] && ![dict get $kit_attrs bake_default]} { + puts stdout " skipping $targetkit (bake_default=false - bake by name or @group to build it)" + lappend skipped_kits [list kit $targetkit reason "bake_default=false (not part of full bakes)"] + continue + } + #G-023/G-024 release gating: a 'versioned' scheme's plain-name output is + #created when absent, then replaced only by an explicit release step + #(G-023 owns that step) - a normal bake never overwrites it. + if {[dict exists $kit_attrs scheme_role] && [dict get $kit_attrs scheme_role] eq "release" + && [file exists $kit_deploydir/$targetkit]} { + puts stdout " skipping $targetkit (release-gated scheme output: bin/$kit_relpath exists and a normal bake never overwrites it - the explicit release step is G-023; to force a rebuild now, delete bin/$kit_relpath first)" + lappend skipped_kits [list kit $targetkit reason "release-gated (bin/$kit_relpath exists)"] + continue + } #Self-build guard: never process the kit whose deployed executable is running this #build - the process sweep below would kill this very process, and the deploy step #cannot replace a running image anyway. Informational/update subcommands are @@ -7415,7 +8073,7 @@ foreach vfstail $vfs_tails { #punk executable remains supported for everything except rebuilding that executable. #(a HOST filesystem comparison - case rules follow the host, not the target) set self_exe [file normalize [info nameofexecutable]] - set self_deploy_target [file normalize $binfolder/$targetkit] + set self_deploy_target [file normalize $kit_deploydir/$targetkit] if {$::punkboot::host_windows} { set is_self_build [string equal -nocase $self_exe $self_deploy_target] } else { @@ -7431,7 +8089,7 @@ foreach vfstail $vfs_tails { set vfs_installer [punkcheck::installtrack new $installername $basedir/.punkcheck] $vfs_installer set_source_target $sourcefolder $buildfolder set vfs_event [$vfs_installer start_event {-make-step build_vfs}] - $vfs_event targetset_init INSTALL $buildfolder/$targetkit + $vfs_event targetset_init INSTALL $kit_builddir/$targetkit set relvfs [punkcheck::lib::path_relative $basedir $sourcefolder/vfs/$vfstail] if {![dict exists $path_cksum_cache $relvfs]} { #e.g ../vfs/punk87.vfs {cksum xxxx cksum_all_opts {-cksum_content 1 ... -cksum_algorithm sha1}} @@ -7475,7 +8133,8 @@ foreach vfstail $vfs_tails { package require control #keep this a simple name - bin/punk script calls into src/_build/exename.vfs/main.tcl - set targetvfs $buildfolder/$targetkit.vfs + #(G-127: non-default-target kits keep the simple name under their kits// tier) + set targetvfs $kit_builddir/$targetkit.vfs file delete -force $targetvfs set extraction_done 0 @@ -7781,6 +8440,26 @@ foreach vfstail $vfs_tails { set sourcevfs [file join $sourcefolder vfs $vfstail] merge_over $sourcevfs $targetvfs + #G-127 per-platform payload selection: the VFSPAYLOAD phase stages + #%platform% payload entries under _targets// in the .vfs + #folder (one subtree per consuming kit target), and the merge above + #carried that staging tree into the image. Select THIS kit's target + #subtree into the image root and drop the staging tree, so the + #assembled kit carries only its own target's payload - upstream of + #the arch scan and the G-125 gate, which must judge the final tree. + if {[file isdirectory $targetvfs/_targets]} { + set _pstaging ${targetvfs}._targets_staging + file delete -force $_pstaging + file rename $targetvfs/_targets $_pstaging + if {[file isdirectory $_pstaging/$rt_target]} { + puts stdout " per-platform payload (G-127): selecting _targets/$rt_target into the merged image for kit $targetkit" + merge_over $_pstaging/$rt_target $targetvfs + } else { + ::punkboot::print_build_warnings [list "per-platform payload (G-127): kit $targetkit (target $rt_target) - the merged vfs has a _targets/ staging tree but no _targets/$rt_target subtree, so this kit gets NO per-platform payload (check the %platform% declarations in src/vfs/$vfstail.toml against the kit's mapvfs target)"] + } + file delete -force $_pstaging + } + #G-133 advisory payload/target binary-arch scan, at the same post-merge #seam as the G-125 gate below. Structural (header bytes only, nothing #executed), so it runs for every kit including cross-target ones. A @@ -7847,7 +8526,7 @@ foreach vfstail $vfs_tails { #the assemblers below consume wrap_runtime - the stamped per-kit copy #of the payload-free raw prefix when embedding happened, the untouched #raw prefix otherwise. Runtime store originals are never modified. - lassign [::punkboot::kit_icon_process $targetkit $rt_target $rtname $target_kit_type $sourcefolder $vfstail $raw_runtime $buildfolder] wrap_runtime kiticon_sidecar + lassign [::punkboot::kit_icon_process $targetkit $rt_target $rtname $target_kit_type $sourcefolder $vfstail $raw_runtime $kit_builddir] wrap_runtime kiticon_sidecar set wrapvfs $targetvfs switch -- $target_kit_type { @@ -8147,13 +8826,13 @@ foreach vfstail $vfs_tails { } } - if {[file exists $buildfolder/$targetkit]} { - puts stderr "deleting existing $buildfolder/$targetkit" + if {[file exists $kit_builddir/$targetkit]} { + puts stderr "deleting existing $kit_builddir/$targetkit" if {[catch { - file delete $buildfolder/$targetkit + file delete $kit_builddir/$targetkit } msg]} { - puts stderr "Failed to delete $buildfolder/$targetkit" - lappend failed_kits [list kit $targetkit reason "could not delete buildfolder kit at $buildfolder/$targetkit"] + puts stderr "Failed to delete $kit_builddir/$targetkit" + lappend failed_kits [list kit $targetkit reason "could not delete buildfolder kit at $kit_builddir/$targetkit"] $vfs_event targetset_end FAILED $vfs_event destroy $vfs_installer destroy @@ -8163,7 +8842,7 @@ foreach vfstail $vfs_tails { #WINDOWS filesystem 'tunnelling' (file replacement within 15secs) could cause targetkit to copy ctime & shortname metadata from previous file! #This is probably harmless - but worth being aware of. - file rename $buildfolder/$vfsname.new $buildfolder/$targetkit + file rename $buildfolder/$vfsname.new $kit_builddir/$targetkit # -- --- --- --- --- --- $vfs_event targetset_end OK @@ -8175,7 +8854,7 @@ foreach vfstail $vfs_tails { #deploys; plain/none (no attached zip - the metakit kit shape) and #unreadable results are silence. One EOCD scan + CD walk per #REBUILT kit only. - lassign [::punkboot::get_kit_offsetstyle_report $buildfolder/$targetkit] offprobe_available offprobe + lassign [::punkboot::get_kit_offsetstyle_report $kit_builddir/$targetkit] offprobe_available offprobe if {!$offprobe_available} { puts stderr "NOTE: kit offset-style pin unavailable (punkboot::utils kit_offsetstyle_report not loadable from bootsupport) - continuing without it" } elseif {[dict get $offprobe offsetstyle] eq "file"} { @@ -8195,7 +8874,7 @@ foreach vfstail $vfs_tails { puts stdout " smoke-require skipped for kit $targetkit: cross-target ($rt_target) - the artifact is not executable on this $::punkboot::host_platform host" } else { puts stdout " smoke-require probe for kit $targetkit (packages: $kit_smokerequires) - executing freshly built artifact via its tclsh subcommand" - ::punkboot::kit_smoke_require_probe $buildfolder/$targetkit $targetkit $kit_smokerequires + ::punkboot::kit_smoke_require_probe $kit_builddir/$targetkit $targetkit $kit_smokerequires } } @@ -8209,14 +8888,13 @@ foreach vfstail $vfs_tails { $vfs_event destroy $vfs_installer destroy - #TODO - allow building of kits for other platforms - # - we need to use /bin for only kits targetting current platform, - # and use /bin/ for others - # that we we can have same target executable for multiple platforms e.g executable just named 'punkshell' - # - to do this we need to change runtime/mapvfs.config to have platform names not just standalone target executable names - # (todo: change to toml format at same time) + #Cross-platform kit outputs: RETIRED TODO (G-127 - see + #goals/G-127-crosstarget-vfs-bake.md). Kits for this host's default + #target deploy flat to /bin exactly as always; any other + #target's kit deploys under /bin/kits// (the + #mapping side gained platform names + toml format under G-122/G-024). after 200 - set deployment_folder [file dirname $sourcefolder]/bin + set deployment_folder $kit_deploydir file mkdir $deployment_folder # -- ---------- @@ -8228,7 +8906,7 @@ foreach vfstail $vfs_tails { #set last_completion [$bin_event targetset_last_complete] $bin_event targetset_addsource $deployment_folder/$targetkit ;#add target as a source of metadata for change detection - $bin_event targetset_addsource $buildfolder/$targetkit + $bin_event targetset_addsource $kit_builddir/$targetkit $bin_event targetset_started # -- ---------- @@ -8251,16 +8929,16 @@ foreach vfstail $vfs_tails { } puts stdout "copying.." - puts stdout "$buildfolder/$targetkit" + puts stdout "$kit_builddir/$targetkit" puts stdout "to:" puts stdout "$deployment_folder/$targetkit" after 300 - file copy $buildfolder/$targetkit $deployment_folder/$targetkit + file copy $kit_builddir/$targetkit $deployment_folder/$targetkit #G-057: the kit resource sidecar ships with the kit - if {[file isfile $buildfolder/$targetkit.resources.toml]} { - file copy -force $buildfolder/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml + if {[file isfile $kit_builddir/$targetkit.resources.toml]} { + file copy -force $kit_builddir/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml } - lappend installed_kits $targetkit + lappend installed_kits $kit_relpath # -- ---------- $bin_event targetset_end OK # -- ---------- @@ -8271,7 +8949,7 @@ foreach vfstail $vfs_tails { #G-057: keep the deployed resource sidecar current even when the kit #itself is unchanged (the record can be newer than the kit rebuild - #e.g the sidecar feature arriving, or a host-capability change) - if {[::punkboot::kiticon::copy_ifchanged $buildfolder/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml] eq "copied"} { + if {[::punkboot::kiticon::copy_ifchanged $kit_builddir/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml] eq "copied"} { puts stdout " kit icon (G-057): refreshed deployed sidecar $deployment_folder/$targetkit.resources.toml" } lappend skipped_kit_installs [list kit $targetkit reason "no change detected"] @@ -8299,7 +8977,7 @@ if {[llength $failed_kits]} { } set had_kits [expr {[llength $installed_kits] || [llength $failed_kits] || [llength $skipped_kits]}] if {$had_kits} { - puts stdout " module builds and kit/zipkit builds processed (vfs config: src/runtime/mapvfs.config)" + puts stdout " module builds and kit/zipkit builds processed (kit mapping: src/runtime/mapvfs.toml)" puts stdout " - use 'make.tcl modules' to build modules without baking the vfs folders into executable kits/zipkits" puts stdout " - use 'make.tcl vfscommonupdate' to promote built modules into the base vfs folder /src/vfs/_vfscommon.vfs (promotion gate)" puts stdout " - Note that without the vfscommonupdate step, 'make.tcl bake' (the kit-assembly stage of 'make.tcl bakehouse') will build vfs based executables" @@ -8309,7 +8987,7 @@ if {$had_kits} { puts stdout " module builds processed" puts stdout "" puts stdout " If kit/zipkit based executables required - create src/vfs/.vfs folders containing lib,modules,modules_tcl9 etc folders" - puts stdout " Also ensure appropriate executables exist in src/runtime along with src/runtime/mapvfs.config" + puts stdout " Also ensure appropriate runtime executables exist in bin/runtime/ along with the kit mapping src/runtime/mapvfs.toml" } puts stdout "-done-" exit 0