diff --git a/AGENTS.md b/AGENTS.md index c4ac90e1..7761f55f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -205,7 +205,7 @@ The project version is fully independent of module versions. A module bump (even - `lib/` (root) — Build output target for `tclsh src/make.tcl libs` - `modules_tcl8/`, `modules_tcl9/` — Tcl version-specific build output targets - `lib_tcl8/`, `lib_tcl9/` — Tcl version-specific build output targets - - `src/project_layouts/` — Thin project layout payloads (structure, scripts, docs, manifests - no stored bootsupport/vfs module snapshots; G-087). The build's thin-layout sync step (`make.tcl modules`/`libs`/`packages`/`project`) keeps each layout's `src/make.tcl`, `src/build.tcl` and bootsupport `include_modules.config` copies current, and syncs the punk.project payload into the `punk::mix::templates` modpod source tree; `dev project.new` injects bootsupport modules/libs into generated projects from the generating shell at generation time. Agents must not directly edit files under `src/project_layouts/` unless the user explicitly names a path there or specifically requests layout changes; never hand-edit the synced copies (they are punkcheck-managed build outputs). + - `src/project_layouts/` — Thin project layout payloads (structure, scripts, docs, manifests - no stored bootsupport/vfs module snapshots; G-087, achieved - see goals/archive/G-087-thin-project-layouts.md). The build's thin-layout sync step (`make.tcl modules`/`libs`/`packages`/`project`) keeps each layout's `src/make.tcl`, `src/build.tcl` and bootsupport `include_modules.config` copies current, and syncs the punk.project payload into the `punk::mix::templates` modpod source tree; `dev project.new` composes each layout's overlay chain (vendor then custom tier, `@` derived folders, `.anti` deletion markers - see the store README) via `layout_materialize`, injects bootsupport modules/libs from the generating shell at generation time, and seeds generated-project identity metadata (punkproject.toml, CHANGELOG.md, punkorigin.toml provenance baseline) programmatically. Agents must not directly edit files under `src/project_layouts/` unless the user explicitly names a path there or specifically requests layout changes; never hand-edit the synced copies (they are punkcheck-managed build outputs). ## Repo-wide Notes diff --git a/CHANGELOG.md b/CHANGELOG.md index 81ec29f7..e2d7f386 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,12 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` "Project Versioning" section for the bump policy. +## [0.17.0] - 2026-07-19 + +- G-087 stages 4+5 (goal achieved) - overlay layout composition: `dev project.new` now materializes a layout's overlay chain before copying - custom-tier files override same-name vendor-tier files (folders merge recursively), zero-byte `.anti` marker files delete lower-layer items, and a `@` folder presents as a derived layout starting from its base layout's chain (new `punk::mix::commandset::layout::lib::layout_materialize` staging primitive; single-folder layouts such as punk.project are used in place, unchanged). The sample fixtures demonstrate all three (`punk.othersample` is a working derived layout with a `.anti` deletion). `layout.files`/`layout.templatefiles` list the effective merged view. +- Generated projects are version-seeded at creation (each only-if-missing): `punkproject.toml` (`[project]` name + version 0.1.0), a matching `CHANGELOG.md` `## [0.1.0]` seed, and `punkorigin.toml` - a generation provenance baseline (layout name + resolved store target, punk::mix::templates version, punkshell version when derivable, origin path) as the interim carrier for the planned G-027 pull updates. +- project.new fixes/robustness: module names (including the projectname-derived default) are validated before any folder or fossil-repo creation (a dashed project name previously failed late with the tree and repo already created; use -modules {} for dashed names wanting no initial module); generation now also works programmatically/script-mode (explicit shellrun dependency, fully-qualified punk::mix::util calls). The retired unimplemented `src/PROJECT_LAYOUTS_*` marker folder and its install/sync exclusions are gone; the `_project` pseudo-vendor store level is retired for layouts (custom/ is the one override level). + ## [0.16.0] - 2026-07-19 - G-087 stage 3 - thin project layouts with generation-time injection: `dev project.new` now injects bootsupport into the generated project from the generating shell's current trees (modules per the layout's `include_modules.config` manifests plus adjacent support files; bootsupport lib trees from a shell project; standalone kits resolve modules from their own module paths and emit a notice for the manifest-less lib set). The ~40MB of stored bootsupport/vfs module snapshots left `src/project_layouts` (fat `custom/_project` fragments and their `_overrides` refs retired; vendor layouts thinned to structure/scripts/docs/manifests). diff --git a/goals/G-012-template-payload-safety.md b/goals/G-012-template-payload-safety.md index d3051392..3c95745e 100644 --- a/goals/G-012-template-payload-safety.md +++ b/goals/G-012-template-payload-safety.md @@ -90,3 +90,10 @@ payload drift is unmanaged. - G-062 (archived) recorded a follow-on in this territory: seeding LICENSE.txt into generated project layouts (deliberately excluded from G-062's acceptance because src/project_layouts sync is restricted) - see goals/archive/G-062-project-license-file.md. +- G-087 (archived - goals/archive/G-087-thin-project-layouts.md) reconfirmed at + each generation verification (stages 3 and 4, 2026-07-19): the trailing + fossil initial commit of a punk.project-generated project aborts on binary + payload content (bin/sdx.kit from the layout plus injected zip-based modpod + .tm modules) because thin layouts ship no binary-glob settings - this goal's + inert VCS-config payload treatment is the designated fix. Layouts without + binaries (sample/derived fixtures) commit cleanly. diff --git a/goals/G-025-exe-selfreport.md b/goals/G-025-exe-selfreport.md index 5eb474e4..38f407a5 100644 --- a/goals/G-025-exe-selfreport.md +++ b/goals/G-025-exe-selfreport.md @@ -91,3 +91,9 @@ Edge cases driving the acceptance clauses: dev paths) would make that mixing observable instead of silent. (G-030 achieved 2026-07-14 - goals/archive/G-030-maketcl-punkargs.md) - Archived-goal references in this file: G-007 achieved 2026-07-05 (goals/archive/G-007-console-location-transparency.md);G-015 achieved 2026-07-07 (goals/archive/G-015-script-subcommand-piped-stdin.md). +- G-087 (archived - goals/archive/G-087-thin-project-layouts.md) added a + consumer for the kit version stamp this goal would provide: generated + projects' punkorigin.toml records punkshell_version only when the + generating shell belongs to a checkout (read via punkboot::utils from + punkproject.toml); bare kits currently record an empty version - a stamped + kit-level punkshell version would close that gap. diff --git a/goals/G-027-derived-project-pull-updates.md b/goals/G-027-derived-project-pull-updates.md index 413872cd..51519f8f 100644 --- a/goals/G-027-derived-project-pull-updates.md +++ b/goals/G-027-derived-project-pull-updates.md @@ -134,3 +134,13 @@ What a pull may touch splits by ownership, not by file type: legacy installfile_* pipeline G-094 retires; G-095 adds concurrent-writer safety - a pull writing a derived project's .punkcheck while builds run there is a future instance of exactly its scenario. +- G-087 (archived - goals/archive/G-087-thin-project-layouts.md) seeded the + interim provenance carrier this goal's baseline record needs: generated + projects now carry a root punkorigin.toml (format = 1; [generation] utc, + layout name + resolved store target, punk::mix::templates version, + punkshell version when derivable, origin projectroot path hint), written + programmatically by project.new, only-if-missing. This goal should adopt + or migrate that record (its header text marks it as G-027-interim) and + maintain it on each pull (origin identity + pulled version). G-087 also + retired the child-side PROJECT_LAYOUTS_CREATE_FROM_... marker on the + grounds that post-generation refresh belongs to this goal's pull. diff --git a/goals/G-087-thin-project-layouts.md b/goals/G-087-thin-project-layouts.md index fe7338cf..41416ce7 100644 --- a/goals/G-087-thin-project-layouts.md +++ b/goals/G-087-thin-project-layouts.md @@ -1,6 +1,6 @@ # G-087 Thin project layouts: injected infrastructure, fauxlink layout refs, working overlay merge -Status: active +Status: achieved 2026-07-19 Scope: src/project_layouts/ (payload store restructure), src/decktemplates/ (layout_refs migration to fauxlink), src/modules/punk/mix/commandset/project-999999.0a1.0.tm (project.new generation path), src/modules/punk/mix/commandset/layout-999999.0a1.0.tm, src/modules/punk/cap/handlers/templates-999999.0a1.0.tm (ref resolution), src/modules/punk/mix/#modpod-templates-999999.0a1.0/ (module-shipped refs/layouts), src/make.tcl (layout sync steps) Goal: project.new generates projects from thin layouts (structure, scripts, docs - no stored bootsupport/module/vfs snapshot trees) whose heavy infrastructure is injected at generation time from the generating shell's own current trees; layout references are fauxlink files (one ref concept repo-wide, no bespoke .ref grammar, no real symlinks); and the documented overlay semantics (custom-over-vendor, .anti deletions, name@base derived layouts, _project staging) are implemented by a single staging-directory materialization primitive - so a bare built kit can generate a current project, and the committed multi-megabyte stale layout snapshots are retired. Acceptance: hygiene stage - in a punkshell checkout `dev project.new` with defaulted -layout resolves an exact layout name (no tcl::prefix ambiguity); the no-handler warning paths execute without error (put/puts typos fixed) and an absolute-pathtype provider registration does not raise an expr syntax error; test/junk refs (test2, sample-0.1 duplicate, the a.txt-only punk.shell-0.1) no longer present as selectable layouts; src/project_layouts/README.md describes the implemented punk.templates+refs mechanism including the ref filename grammar. Pivot stage - a project generated from the thin punk.project layout carries bootsupport modules matching the generating shell's current src/bootsupport (verified by comparing a marker module version such as punk::mix::cli, not a stored snapshot vintage); src/project_layouts contains no embedded bootsupport/vfs module trees and make.tcl's layout bootsupport-sync step is retired; layout refs are .fauxlink files resolved via the fauxlink module with the bespoke .ref parser removed, every layout ref carrying a non-empty nominal name (leading-# punkcheck-exclusion hazard documented in this file); a bare built kit run outside any project lists at least the module-shipped punk.project layout and generates a working project from it; and one overlay case exercising custom-over-vendor override, a .anti deletion, and a name@base derived layout (the existing sample-0.1/othersample fixtures) materializes correctly through the staging primitive into a generated project. @@ -107,7 +107,12 @@ Staged; hygiene first since it is independently shippable. refs + thin payloads; heavy parts arrive by the same injection and refresh via G-027); decide whether the vendor/custom x `_project` matrix at both levels still earns its complexity once overlays are cheap (vendor/_project and custom/_project - may collapse into one override level). + may collapse into one override level); and generated-project version seeding + (2026-07-19 assessment): project.new stamps a minimal punkproject.toml + (`[project]` name + starting version 0.1.0) plus a matching CHANGELOG.md seed + into the generated project programmatically post-materialization, and records + generation provenance in the G-027-designated carrier - constraints and + rationale in Notes ("Version/provenance decisions"). ## Alternatives considered @@ -164,6 +169,32 @@ Staged; hygiene first since it is independently shippable. - The `#n` duplicate-suffix mechanism in `_get_itemdict` and the priority ordering (module < shellproject < currentproject < absolute < adhoc) are sound and carry over unchanged. +- Version/provenance decisions (2026-07-19 naming+version-tracking assessment, + separate session, read-only): the default layout's name/folder must NOT track + punkproject.toml's `[project]` version - version series are independent per the + root AGENTS.md; punkshell's version is unavailable in bare kits (no VFS + punkproject.toml or stamped literal; G-025 proposed would add one); post-thin + layouts the vintage lives in the injected module versions (this goal's + marker-module principle), so a punkshell-tracking layout name would misstate it; + and per-release renames would reset punkcheck path-keyed install history (both + generation and -update of existing projects), orphan `name@base` derived layouts + (base is matched by folder name), and re-create the tcl::prefix ambiguity the + hygiene stage fixed. Stamping constraints for the stage-5 item: write + punkproject.toml programmatically post-materialization, never ship one inside a + payload (punk::repo::is_candidate_root existence check - the repo.tm comment and + G-033 hazard); seed CHANGELOG.md `## [0.1.0]` alongside or the injected make.tcl + projectversion advisory warns forever in every generated project; confirm + punkboot::utils reaches generated projects (make.tcl's check requires it). + Provenance (layout name + resolved target, punk::mix::templates version, + punkshell version when derivable - checkout contexts only until a kit stamp + exists) belongs in the G-027-designated carrier (fauxlink TOML body / provenance + record), not in `[project]`, whose schema is contested by G-047/G-065/G-057. A + template tag for the new project's version, if wanted, must be %projectversion% - + %version% already means module version in module.new. Any versionless->highest + layout-version selection must compare with `package vcompare` (the handler's + within-base ordering is lexical lsort) and must never mix the layout + format-version series (0.x) with a project-version series (vcompare makes + 0.15.0 > 0.2 win permanently). ## Progress @@ -266,6 +297,15 @@ Notes from the work: spun out as G-094 (single OO lifecycle) and G-095 (concurrent-writer safety); the MAX_PATH normalization candidate above is flagged in G-094's notes as a natural rider on its load/save consolidation. +- FIXED 2026-07-19 (commandset::project 0.3.1, shipped within the 0.4.0 + stage-4 bump - the 0.3.1 changelog line is retained in buildversion.txt): + project.new with a dashed project name (e.g infra-man) failed late inside + module.new - after the project tree and fossil repo already existed - + because the projectname-derived default module name carried the dash + (invalid in Tcl package names). Module names (defaulted and explicit + -modules entries, modulename-version forms accepted) are now validated + before any folder or fossil-repo creation; -modules {} remains the escape + hatch for dashed project names wanting no initial module. ### 2026-07-19 stage 3 complete: thin layouts + generation-time injection (increment 3) @@ -369,9 +409,96 @@ Notes / residue from stage 3: - sdx.kit remains in the store layout (grandfathered committed binary, excluded from the modpod) - G-004/G-006 decide its future. -Remaining for acceptance (Approach stages 4-5): - -- staging-directory overlay merge primitive exercising the sample-0.1/othersample - fixtures (stage 4). -- child-side PROJECT_LAYOUTS_CREATE_FROM_... marker retirement and the - vendor/custom x _project matrix decision (stage 5). +### 2026-07-19 stages 4+5 complete: overlay merge primitive, marker/matrix retirement, version seeding (increment 4) + +Landed (punk::mix::commandset::layout 0.2.0, punk::mix::commandset::project +0.4.0 (also carrying the separately-recorded 0.3.1 module-name validation +fix), make.tcl sync exclusion trimmed, punkshell 0.17.0): + +- Overlay merge primitive in commandset::layout::lib: layout_materialize + (entry point; returns the effective content folder), layout_resolve_chain + (vendor-then-custom tier chain per store, @ derived-layout + expansion, recursive + cycle-guarded), layout_stage_chain (staging-dir + composition; per layer .anti deletions apply before the layer's own copy; + #layout_staging marker file guards against clobbering non-staging dirs), + private helpers _dir_entries/_find_anti_relpaths/_merge_copy_into. + Design decisions: .anti markers are zero-byte FILES by convention (the + documented empty-folder form is also accepted but cannot be committed to + git/fossil); marker files stay a plain filename convention rather than + tag-bearing fauxlinks - a fauxlink is a reference every fauxlink-aware + tool may follow (fcat auto-follows .fauxlink), while an anti-marker is a + tombstone with no target; the "one ref concept" principle covers things + that point. Chains of one folder with no markers are used in place + (staged 0) - punk.project (store and zipfs modpod) is byte-identical in + behaviour to stage 3. Fixtures: othersample gained its ref + (decktemplates/custom/punk/layout_refs/othersample#custom+punk+othersample@sample-0.1.fauxlink, + link_as round-trip verified) and a .gitignore.anti demo marker; extra.txt + rewritten to the implemented semantics; store README documents the + composition. +- project.new copies from the materialized path (staging deleted after the + template scan), and the misnamed antipaths exclusion list folded into + exclude_paths. layout.files/layout.templatefiles display the effective + overlay-merged view (new path-based layoutfolder_scan_for_template_files + core). +- Stage 5 retirements/decisions: the empty (untracked) child-side + src/PROJECT_LAYOUTS_CREATE_FROM_DECKTEMPLATES_LAYOUT_REFERENCES marker + folder deleted with its project.new + make.tcl sync exclusions (thin + layouts + injection made it moot; refresh belongs to G-027 pull). The + vendor/custom x _project matrix is COLLAPSED for layouts: custom/ + is the one override level; _project store folders take no part in overlay + chains (materialize treats them as opaque single-folder layouts) - the + _project pseudo-vendor remains only for decktemplates module/utility + templates. +- Stage 5 version seeding (per the 2026-07-19 assessment constraints): + project.new stamps generated projects post-materialization, each + only-if-missing and never from payload - punkproject.toml ([project] name + + version 0.1.0), CHANGELOG.md with matching ## [0.1.0] header (keeps the + injected make.tcl projectversion advisory quiet), and punkorigin.toml: an + INTERIM generation-provenance baseline (format = 1; [generation] utc, + layout name, resolved store target, punk::mix::templates version, + punkshell version when derivable via punkboot::utils from the generating + shell's checkout, origin projectroot path hint) for G-027 to adopt or + migrate (noted in G-027's detail file). +- Robustness fixes en route (script-mode/programmatic project.new - found + driving generation via `punk902z src script`): explicit `package require + shellrun` (runx/run aliases exist only in shell contexts otherwise), + punk::mix::util::askuser/do_in_path fully qualified (the util:: shorthand + is wired only when a cli imports the commandset), commandset::layout + requires punk::mix::util and calls foreach-file fully qualified. + +Verification (2026-07-19, native tclsh 9.0.3 + punk902z src): + +- New suite src/tests/modules/punk/mix/testsuites/layout/materialize.test: + 20/20 (chain shapes incl. recursion + cycle, override/add, file+dir .anti + forms, folder replace, type conflicts, dotfiles, empty dirs, marker + stripping, refuse-clobber, restage, fastpath/opaque/_project materialize + via -layoutinfo). Full punk/mix subtree 66 pass / 1 known skip; + punkcheck install+installtrack 64/64. +- End-to-end (punk902z src script, FOSSIL_HOME+HOME diverted, fresh + .fossils): punk.othersample listed via the new custom-deck ref and + generated a project through the 3-folder staged chain - derived README, + extra.txt present, .gitignore anti-deleted, src/modules/README.md from + the vendor base, no *.anti in output, module created, staging dir + removed, trailing fossil initial commit OK (sample layouts ship no + binaries). punk.sample-0.1 generated with the custom README override and + .gitignore retained. punk.project generated fast-path (staged 0, no + staging output) with bootsupport injection intact (113 modules + 3 lib + folders); punk.project#2 (zipfs modpod store) materializes fast-path with + the store-shape decomposition working on //zipfs:/ paths. All three + seeded punkproject.toml/CHANGELOG.md/punkorigin.toml; `make.tcl + projectversion` inside the generated punk.project project reports + consistency OK (punkboot::utils reaches generated projects). The + punk.project trailing fossil commit still aborts on binary payload + content (bin/sdx.kit + zip-based .tm modules) - pre-existing G-012 + residue, unchanged by this work. +- Known limitation reconfirmed en route (pre-existing, G-094 rider): deep + target dirs make punkcheck's relative source paths exceed Windows + MAX_PATH ('could not read ... no such file' with a long ../ chain) - + generation into a shallow directory works. + +Acceptance review at flip: hygiene criteria verified 2026-07-17 (increment +1); fauxlink-ref criteria verified 2026-07-17 (increment 2); thin-layout + +injection + bare-kit criteria verified 2026-07-19 (increment 3); the +overlay case (custom-over-vendor override + .anti deletion + name@base +derived layout via the sample-0.1/othersample fixtures materializing into a +generated project) verified this increment. No remaining items. diff --git a/goals/G-094-punkcheck-single-lifecycle.md b/goals/G-094-punkcheck-single-lifecycle.md index a404cd7b..ef832f1d 100644 --- a/goals/G-094-punkcheck-single-lifecycle.md +++ b/goals/G-094-punkcheck-single-lifecycle.md @@ -80,9 +80,12 @@ lifecycle, not an ad-hoc bypass with its own record code. bootsupport's copy; a dev shell runs the built one) - re-vendor bootsupport promptly after landing, and sequence this refactor before any on-disk protocol change from G-095. -- G-087 recorded a punkcheck MAX_PATH candidate (deep relative source chains - exceed Windows path limits, unnormalized) - natural rider on this goal's - load/save consolidation. +- G-087 (archived - goals/archive/G-087-thin-project-layouts.md) recorded a + punkcheck MAX_PATH candidate (deep relative source chains exceed Windows + path limits, unnormalized) - natural rider on this goal's load/save + consolidation. Reconfirmed 2026-07-19 during the stage-4 verification: + project generation into a deep directory fails reading the layout store + manifests through a long ../ chain. - The characterization suites were written expressly to lock behaviour "before any refactoring" - this is that refactoring. - punkcheck::cli (status/paths) is a read-only consumer and stays untouched. diff --git a/punkproject.toml b/punkproject.toml index f45f0efa..9e97ba28 100644 --- a/punkproject.toml +++ b/punkproject.toml @@ -1,4 +1,4 @@ [project] name = "punkshell" -version = "0.16.0" +version = "0.17.0" license = "BSD-2-Clause" diff --git a/src/decktemplates/custom/punk/layout_refs/othersample#custom+punk+othersample@sample-0.1.fauxlink b/src/decktemplates/custom/punk/layout_refs/othersample#custom+punk+othersample@sample-0.1.fauxlink new file mode 100644 index 00000000..e69de29b diff --git a/src/make.tcl b/src/make.tcl index 9b923485..fd4bf051 100644 --- a/src/make.tcl +++ b/src/make.tcl @@ -3383,7 +3383,8 @@ if {$::punkboot::command in {project packages modules libs}} { #store -> modpod payload sync: the punk::mix::templates module ships the thin punk.project layout #so module-pathtype layout refs resolve and a bare kit outside any project can generate from it. #The modpod copy is build-managed - do not hand-edit it. Excludes: bin/sdx.kit (no new binaries - #inside zip-based .tm modules) and the unimplemented child-side src/PROJECT_LAYOUTS_* marker. + #inside zip-based .tm modules). (The unimplemented child-side src/PROJECT_LAYOUTS_* marker was + #retired by G-087 stage 5 - thin layouts + generation-time injection made it moot.) set store_layout $sourcefolder/project_layouts/vendor/punk/project-0.1 set modpod_layout "$sourcefolder/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1" if {[file isdirectory $store_layout]} { @@ -3398,7 +3399,7 @@ if {$::punkboot::command in {project packages modules libs}} { -overwrite installedsourcechanged-targets -createempty 1 -exclude-paths-core {#* **/#* _aside **/_aside _build **/_build .git **/.git} - -exclude-paths {bin/sdx.kit src/PROJECT_LAYOUTS_*} + -exclude-paths {bin/sdx.kit} }] puts stdout [punkcheck::summarize_install_resultdict $resultdict] if {![catch {package require punk::mix::cli}] diff --git a/src/modules/punk/mix/commandset/layout-999999.0a1.0.tm b/src/modules/punk/mix/commandset/layout-999999.0a1.0.tm index a8c7a448..e30f7fc9 100644 --- a/src/modules/punk/mix/commandset/layout-999999.0a1.0.tm +++ b/src/modules/punk/mix/commandset/layout-999999.0a1.0.tm @@ -22,6 +22,7 @@ package require punk::args #sort of a circular dependency when commandset loaded by punk::mix::cli - that's ok, but this could theoretically be loaded by another cli and with another base package require punk::mix package require punk::mix::base +package require punk::mix::util package require punk::lib package require textblock @@ -85,12 +86,12 @@ namespace eval punk::mix::commandset::layout { puts stderr "layout '$layout' not found." return } - set layoutinfo [dict get $layoutdict $layout] - set layoutfolder [dict get $layoutinfo path] - + #G-087 stage 4: display against the effective (overlay-materialized) content + set materialized [lib::layout_materialize $layout -layoutinfo [dict get $layoutdict $layout]] + set layoutfolder [dict get $materialized path] set stripprefix [file normalize $layoutfolder] - set templatefiles_and_tags [lib::layout_scan_for_template_files $layout] + set templatefiles_and_tags [lib::layoutfolder_scan_for_template_files $layoutfolder] set flatlist [list] foreach entry $templatefiles_and_tags { lassign $entry templatefullpath tags @@ -222,7 +223,358 @@ namespace eval punk::mix::commandset::layout { } return $layout_dict } - + + # -- --- --- --- --- --- --- --- --- --- --- + # G-087 stage 4: overlay merge primitive + # A layout's effective content can be composed from an ordered chain of store folders + # (vendor tier, then custom tier, with @ folders deriving from another + # layout's chain) with .anti markers deleting lower-layer items. + # layout_materialize is the entry point; project.new copies from its result path. + # -- --- --- --- --- --- --- --- --- --- --- + + punk::args::define { + @id -id ::punk::mix::commandset::layout::lib::layout_resolve_chain + @cmd -name punk::mix::commandset::layout::lib::layout_resolve_chain\ + -summary\ + "Resolve a layout store folder name to its ordered overlay chain"\ + -help\ + "Returns the ordered list (lowest priority first) of existing store folders + that contribute to the named layout within a single project_layouts store: + + @> + /vendor// (and @ siblings) + /custom// (and @ siblings) + + A folder named @ declares a derived layout: names another + layout in the same store+vendor whose chain is expanded first (recursively, + cycle-guarded with a warning). Layout names must not themselves contain @ - + the first @ in a folder name is the derivation separator. + + The result may be empty if none of the candidate folders exist." + @values -min 3 -max 4 + storeroot -type string -help\ + "Store root folder (a src/project_layouts folder, or a module's + templates/project_layouts folder)" + vendor -type string -help\ + "Vendor folder name within the store's vendor/ and custom/ tiers" + layoutfoldername -type string -help\ + "Layout folder name - either or the derived form @" + seen -type list -optional 1 -help\ + "Internal recursion guard - layout names already being expanded" + } + proc layout_resolve_chain {storeroot vendor layoutfoldername {seen {}}} { + #manual args - see PUNKARGS ::punk::mix::commandset::layout::lib::layout_resolve_chain + set atpos [string first @ $layoutfoldername] + if {$atpos > 0} { + set layoutname [string range $layoutfoldername 0 $atpos-1] + } else { + set layoutname $layoutfoldername + } + if {$layoutname in $seen} { + puts stderr "layout_resolve_chain WARNING - layout derivation cycle detected at '$layoutname' (expansion path: $seen) - stopping expansion" + return [list] + } + lappend seen $layoutname + #glob-escape the name so metachars in a folder name can't broaden the derived-folder search + set globname [string map [list \\ \\\\ \[ \\\[ \] \\\] \{ \\\{ \} \\\} * \\* ? \\?] $layoutname] + set bases [list] + set tierfolders [list] + foreach tier {vendor custom} { + set tdir [file join $storeroot $tier $vendor] + set plain [file join $tdir $layoutname] + if {[file isdirectory $plain]} { + lappend tierfolders $plain + } + foreach d [lsort [glob -nocomplain -dir $tdir -type d ${globname}@*]] { + lappend tierfolders $d + set b [string range [file tail $d] [string length ${layoutname}@] end] + if {$b ne "" && $b ni $bases} { + lappend bases $b + } + } + } + set chain [list] + foreach b $bases { + foreach p [layout_resolve_chain $storeroot $vendor $b $seen] { + if {$p ni $chain} { + lappend chain $p + } + } + } + foreach p $tierfolders { + if {$p ni $chain} { + lappend chain $p + } + } + return $chain + } + + punk::args::define { + @id -id ::punk::mix::commandset::layout::lib::layout_stage_chain + @cmd -name punk::mix::commandset::layout::lib::layout_stage_chain\ + -summary\ + "Compose an overlay chain of layout folders into a staging directory"\ + -help\ + "Deletes and recreates stagingdir, then merges each chain folder into it in + order (lowest priority first). For each layer, .anti deletion markers are + applied before the layer's own content is copied: + + - an entry named .anti (zero-byte file recommended; a marker folder + is also accepted) deletes - file or folder - as contributed by + lower layers, and is itself never copied. + - a same-name file in a higher layer replaces the lower layer's file; + folders merge recursively. To fully replace a folder rather than merge + into it, pair a .anti marker with a replacement in the + same layer. + + A #layout_staging marker file is written into stagingdir so later calls can + recognise the folder as disposable staging output (the leading # also keeps + the marker out of punkcheck-driven installs). If stagingdir already exists, + is non-empty and carries no such marker, the call errors rather than + deleting it. + + Returns a dict: path (stagingdir), chain, deletions (relative paths removed + by .anti markers), copied (number of files copied)." + @values -min 2 -max 2 + chain -type list -help\ + "Ordered list of layout source folders, lowest priority first + (as returned by layout_resolve_chain)" + stagingdir -type string -help\ + "Directory to compose into - deleted and recreated by this call" + } + proc layout_stage_chain {chain stagingdir} { + #manual args - see PUNKARGS ::punk::mix::commandset::layout::lib::layout_stage_chain + if {![llength $chain]} { + error "layout_stage_chain: empty chain" + } + foreach sourcefolder $chain { + if {![file isdirectory $sourcefolder]} { + error "layout_stage_chain: chain member '$sourcefolder' is not a directory" + } + } + set stagingdir [file normalize $stagingdir] + set marker [file join $stagingdir #layout_staging] + if {[file exists $stagingdir]} { + if {[llength [_dir_entries $stagingdir]] && ![file exists $marker]} { + error "layout_stage_chain: staging dir '$stagingdir' exists, is not empty, and has no #layout_staging marker - refusing to delete it. Supply an empty/nonexistent dir or a previous staging dir." + } + file delete -force $stagingdir + } + file mkdir $stagingdir + #write the marker first so an interrupted composition is still recognisable as staging output + set fd [open $marker w] + close $fd + set deletions [list] + set copied 0 + foreach sourcefolder $chain { + foreach relanti [_find_anti_relpaths $sourcefolder] { + set victim [file join $stagingdir $relanti] + if {[file exists $victim]} { + file delete -force $victim + lappend deletions $relanti + } + } + incr copied [_merge_copy_into $sourcefolder $stagingdir] + } + return [dict create path $stagingdir chain $chain deletions $deletions copied $copied] + } + + punk::args::define { + @id -id ::punk::mix::commandset::layout::lib::layout_materialize + @cmd -name punk::mix::commandset::layout::lib::layout_materialize\ + -summary\ + "Materialize a layout's overlay chain into a single source folder"\ + -help\ + "Resolves the named layout's overlay chain (layout_resolve_chain) within its + store and returns a folder holding the effective layout content. + + When the chain is a single folder containing no .anti markers (the common + case - e.g the thin punk.project layout, or any module-carried layout) that + folder is returned as-is (staged 0) and no copying occurs. Otherwise the + chain is composed into a staging directory (layout_stage_chain) and the + staged folder is returned (staged 1) - the caller may delete it when done + (the default location is deterministic, so a later call recomposes it). + + Layout folders whose store position does not follow the + /{vendor,custom}// shape (including any + under the retired _project pseudo-vendor) are treated as opaque + single-folder chains. + + Returns a dict: + path - folder holding the effective layout content + staged - 1 if path is a disposable staging composition, + 0 if it is the store folder itself + chain - ordered contributing folders, lowest priority first + deletions - relative paths removed by .anti markers (staged only) + copied - number of files copied into staging (staged only)" + @leaders -min 1 -max 1 + layoutname -type string -optional 0 -help\ + "Visible layout name as shown by 'dev project.layouts' (e.g punk.project)" + @opts + -layoutinfo -default "" -help\ + "Layout info dict as returned by layouts_dict (keys path, basefolder, ...). + When empty, layoutname is resolved via layouts_dict - which requires the + punk.templates capability context of the calling shell." + -stagingdir -default "" -help\ + "Directory to compose an overlay chain into when staging is needed. + Defaults to a deterministic per-layout-name folder under the system + temp directory." + @values -min 0 -max 0 + } + proc layout_materialize {args} { + set argd [punk::args::parse $args withid ::punk::mix::commandset::layout::lib::layout_materialize] + set layoutname [dict get $argd leaders layoutname] + set layoutinfo [dict get $argd opts -layoutinfo] + set opt_stagingdir [dict get $argd opts -stagingdir] + if {$layoutinfo eq ""} { + set layoutdict [layouts_dict] + if {![dict exists $layoutdict $layoutname]} { + error "layout_materialize: layout '$layoutname' not found. Known layouts: [dict keys $layoutdict]" + } + set layoutinfo [dict get $layoutdict $layoutname] + } + set layoutfolder [dict get $layoutinfo path] + set chain [list $layoutfolder] + if {[dict exists $layoutinfo basefolder]} { + #decompose /// + #anything else (unexpected depth, unknown tier, retired _project pseudo-vendor) stays an opaque single-folder chain + set storeroot [dict get $layoutinfo basefolder] + set rootparts [file split $storeroot] + set folderparts [file split $layoutfolder] + set rootlen [llength $rootparts] + if {[llength $folderparts] == $rootlen + 3 && [lrange $folderparts 0 $rootlen-1] eq $rootparts} { + lassign [lrange $folderparts end-2 end] tier vendor layoutfoldername + if {$tier in {vendor custom} && $vendor ne "_project"} { + set resolved [layout_resolve_chain $storeroot $vendor $layoutfoldername] + if {$layoutfolder in $resolved} { + set chain $resolved + } + } + } + } + #fast path - a single-source chain with no .anti markers is usable in place + if {[llength $chain] == 1 && ![llength [_find_anti_relpaths [lindex $chain 0]]]} { + return [dict create path [lindex $chain 0] staged 0 chain $chain deletions [list] copied 0] + } + if {$opt_stagingdir eq ""} { + set sanitized [regsub -all {[^A-Za-z0-9_.-]} $layoutname _] + set opt_stagingdir [file join [punk::lib::tempdir] punk_layout_staging $sanitized] + } + set stageinfo [layout_stage_chain $chain $opt_stagingdir] + return [dict create path [dict get $stageinfo path] staged 1 chain $chain deletions [dict get $stageinfo deletions] copied [dict get $stageinfo copied]] + } + + punk::args::define { + @id -id ::punk::mix::commandset::layout::lib::_dir_entries + @cmd -name punk::mix::commandset::layout::lib::_dir_entries\ + -summary\ + "List folder entry tails including dotfiles, excluding . and .."\ + -help\ + "Internal helper for the overlay merge primitive. glob * misses leading-dot + entries on unix while matching them on windows - this returns the sorted + union of * and .* minus the . and .. entries so tree walks behave + identically on both." + @values -min 1 -max 1 + folder -type string + } + proc _dir_entries {folder} { + #manual args - see PUNKARGS ::punk::mix::commandset::layout::lib::_dir_entries + set entries [list] + foreach e [lsort -unique [glob -nocomplain -dir $folder -tails * .*]] { + if {$e eq "." || $e eq ".."} { + continue + } + lappend entries $e + } + return $entries + } + + punk::args::define { + @id -id ::punk::mix::commandset::layout::lib::_find_anti_relpaths + @cmd -name punk::mix::commandset::layout::lib::_find_anti_relpaths\ + -summary\ + "Recursively collect .anti marker targets below a layout folder"\ + -help\ + "Internal helper for the overlay merge primitive. Returns the relative paths + (marker name minus the .anti suffix) marked for deletion by .anti + entries anywhere below basefolder. Marker entries may be files (zero-byte + recommended) or folders; marker folders are not walked. A bare '.anti' + entry has no target and is ignored with a warning." + @values -min 1 -max 2 + basefolder -type string + prefix -type string -optional 1 + } + proc _find_anti_relpaths {basefolder {prefix ""}} { + #manual args - see PUNKARGS ::punk::mix::commandset::layout::lib::_find_anti_relpaths + if {$prefix eq ""} { + set here $basefolder + } else { + set here [file join $basefolder $prefix] + } + set antis [list] + foreach entry [_dir_entries $here] { + if {$prefix eq ""} { + set rel $entry + } else { + set rel [file join $prefix $entry] + } + if {[string match *.anti $entry]} { + if {[string length $entry] == 5} { + puts stderr "layout overlay WARNING - ignoring degenerate '.anti' marker (no target name) at '[file join $here $entry]'" + continue + } + lappend antis [string range $rel 0 end-5] + } elseif {[file isdirectory [file join $here $entry]]} { + lappend antis {*}[_find_anti_relpaths $basefolder $rel] + } + } + return $antis + } + + punk::args::define { + @id -id ::punk::mix::commandset::layout::lib::_merge_copy_into + @cmd -name punk::mix::commandset::layout::lib::_merge_copy_into\ + -summary\ + "Recursively merge-copy a folder over a target, skipping .anti markers"\ + -help\ + "Internal helper for the overlay merge primitive. Files overwrite same-name + targets; folders merge recursively; on a file/folder type conflict the + source entry wins (the conflicting target is deleted first). .anti marker + entries are never copied. Returns the number of files copied." + @values -min 2 -max 2 + srcdir -type string + tgtdir -type string + } + proc _merge_copy_into {srcdir tgtdir} { + #manual args - see PUNKARGS ::punk::mix::commandset::layout::lib::_merge_copy_into + set copied 0 + foreach entry [_dir_entries $srcdir] { + if {[string match *.anti $entry]} { + continue + } + set s [file join $srcdir $entry] + set t [file join $tgtdir $entry] + if {[file isdirectory $s]} { + if {[file exists $t] && ![file isdirectory $t]} { + #a lower layer contributed a file where this layer has a folder - the folder wins + file delete -force $t + } + file mkdir $t + incr copied [_merge_copy_into $s $t] + } else { + if {[file isdirectory $t]} { + #a lower layer contributed a folder where this layer has a file - the file wins + file delete -force $t + } + file copy -force $s $t + incr copied + } + } + return $copied + } + + # -- --- --- --- --- --- --- --- --- --- --- + proc layout_all_files {layout} { #todo - allow versionless layout name to pick highest version found set layoutdict [layouts_dict] @@ -230,31 +582,53 @@ namespace eval punk::mix::commandset::layout { puts stderr "layout '$layout' not found." return } - set layoutinfo [dict get $layoutdict $layout] - set layoutfolder [dict get $layoutinfo path] + #G-087 stage 4: list the effective (overlay-materialized) content, not just the primary store folder + set materialized [layout_materialize $layout -layoutinfo [dict get $layoutdict $layout]] + set layoutfolder [dict get $materialized path] if {![file isdirectory $layoutfolder]} { puts stderr "layout '$layout' points to path $layoutfolder - but it doesn't seem to exist" } - set file_list [list] - util::foreach-file $layoutfolder path { - lappend file_list $path + set file_list [list] + #fully qualified - the util:: shorthand is only wired when a cli has imported this commandset + punk::mix::util::foreach-file $layoutfolder path { + if {[file tail $path] ne "#layout_staging"} { + lappend file_list $path + } } return $file_list } - # proc layout_scan_for_template_files {layout {tags {}}} { - #todo JMN set layoutdict [layouts_dict] if {![dict exists $layoutdict $layout]} { puts stderr "layout '$layout' not found." return } - set layoutinfo [dict get $layoutdict $layout] - set layoutfolder [dict get $layoutinfo path] + #G-087 stage 4: scan the effective (overlay-materialized) content + set materialized [layout_materialize $layout -layoutinfo [dict get $layoutdict $layout]] + return [layoutfolder_scan_for_template_files [dict get $materialized path] $tags] + } - #use last matching layout found. review silent if multiple? + punk::args::define { + @id -id ::punk::mix::commandset::layout::lib::layoutfolder_scan_for_template_files + @cmd -name punk::mix::commandset::layout::lib::layoutfolder_scan_for_template_files\ + -summary\ + "Scan a layout content folder for files containing template tags"\ + -help\ + "Path-based core of layout_scan_for_template_files: scans the supplied + folder (typically the path returned by layout_materialize) and returns a + list of {fullpath found_tagnames} entries for files containing any of the + tag placeholders. With no tags supplied, the standard tag set (%project%) + is used." + @values -min 1 -max 2 + layoutfolder -type string -help\ + "Folder holding layout content to scan" + tags -type list -optional 1 -help\ + "Tag placeholder literals to search for (defaults to the standard set)" + } + proc layoutfolder_scan_for_template_files {layoutfolder {tags {}}} { + #manual args - see PUNKARGS ::punk::mix::commandset::layout::lib::layoutfolder_scan_for_template_files if {![llength $tags]} { #todo - get standard tags from somewhere set tagnames [list project] @@ -262,8 +636,12 @@ namespace eval punk::mix::commandset::layout { lappend tags [string cat % $tn %] ;#make sure actual tag literal doesn't appear in this source file } } - set file_list [list] - util::foreach-file $layoutfolder path { + set file_list [list] + #fully qualified - the util:: shorthand is only wired when a cli has imported this commandset + punk::mix::util::foreach-file $layoutfolder path { + if {[file tail $path] eq "#layout_staging"} { + continue + } set fd [open $path r] fconfigure $fd -translation binary set data [read $fd] diff --git a/src/modules/punk/mix/commandset/layout-buildversion.txt b/src/modules/punk/mix/commandset/layout-buildversion.txt index d97a9d8a..91577416 100644 --- a/src/modules/punk/mix/commandset/layout-buildversion.txt +++ b/src/modules/punk/mix/commandset/layout-buildversion.txt @@ -1,4 +1,6 @@ -0.1.1 +0.2.0 #First line must be a semantic version number #all other lines are ignored. +#0.2.0 - G-087 stage 4: overlay merge primitive in lib - layout_materialize (entry point returning the effective content folder; fast path returns the store folder untouched for single-source chains with no .anti markers), layout_resolve_chain (vendor-then-custom tier chain with @ derived-layout expansion, cycle-guarded), layout_stage_chain (staging-dir composition applying .anti deletion markers; #layout_staging marker guards against clobbering non-staging dirs), plus private helpers _dir_entries/_find_anti_relpaths/_merge_copy_into. The _project pseudo-vendor participates in no chain (retired for layouts - G-087 stage 5 decision). +#0.2.0 - new lib::layoutfolder_scan_for_template_files (path-based core of layout_scan_for_template_files); layout_scan_for_template_files, layout_all_files and templatefiles.relative now operate on the overlay-materialized content rather than the primary store folder only (the staging marker file is excluded from listings/scans). #0.1.1 - fix: no-handler warning paths in collection::references_as_dict and lib::layouts_dict used 'put' instead of 'puts' and then errored on an unset dict variable; they now warn correctly and return an empty dict. references_as_dict warning message no longer misattributes itself to layouts_dict. diff --git a/src/modules/punk/mix/commandset/project-999999.0a1.0.tm b/src/modules/punk/mix/commandset/project-999999.0a1.0.tm index e14d626f..3434eafe 100644 --- a/src/modules/punk/mix/commandset/project-999999.0a1.0.tm +++ b/src/modules/punk/mix/commandset/project-999999.0a1.0.tm @@ -183,6 +183,7 @@ namespace eval punk::mix::commandset::project { set opt_update [dict get $opts -update] # -- --- --- --- --- --- --- --- --- --- --- --- --- set opt_modules [dict get $opts -modules] + set modules_defaulted 0 if {[llength $opt_modules] == 1 && [lindex $opt_modules 0] eq "\uFFFF"} { #if not specified - add a single module matching project name if {$opt_force || $opt_update} { @@ -191,6 +192,21 @@ namespace eval punk::mix::commandset::project { set opt_modules [list] } else { set opt_modules [list [string tolower $projectname]] ;#default modules to lowercase as is the modern (tip 590) recommendation for Tcl + set modules_defaulted 1 + } + } + #validate module names now - before any folder or fossil repo creation - rather than + #failing partway through generation when module.new runs. e.g a dash in the project + #name flows into the default module name, and dashes are invalid in Tcl package names + #except as the modulename-version separator. + foreach m $opt_modules { + lassign [punk::mix::util::split_modulename_version [string trimleft $m :]] mbase mversion + if {[catch {punk::mix::cli::lib::validate_modulename $mbase -errorprefix "module name"} vErr]} { + if {$modules_defaulted} { + error "punk mix project.new error: project name '$projectname' is not usable as the default module name ($vErr). Use a project name that is a valid Tcl package name (no dashes), or pass -modules {} for no initial module, or supply valid names via -modules." + } else { + error "punk mix project.new error: invalid -modules entry '$m' ($vErr)" + } } } # -- --- --- --- --- --- --- --- --- --- --- --- --- @@ -198,12 +214,13 @@ namespace eval punk::mix::commandset::project { #todo - install support binaries on a per-project basis in a way that doesn't impact machine (e.g not added to path) - cache in user config dir if possible, supply mechanism to clear cache # + package require shellrun ;#run/runx global aliases - present in shell contexts, but not e.g punk script mode set fossil_prog [auto_execok fossil] if {![string length $fossil_prog]} { puts stderr "The fossil program was not found. A fossil executable is required to use most deck features." if {[string length [set scoop_prog [auto_execok scoop]]]} { #restrict to windows? - set answer [util::askuser "scoop detected. Would you like deck to install fossil now using scoop? Y|N"] + set answer [punk::mix::util::askuser "scoop detected. Would you like deck to install fossil now using scoop? Y|N"] if {[string tolower $answer] ne "y"} { puts stderr "mix new aborting due to user response '$answer' (required Y|y to proceed) use -confirm 0 to avoid prompts." return @@ -224,7 +241,7 @@ namespace eval punk::mix::commandset::project { return } #todo - ask user if they want to configure fosssil first.. - set answer [util::askuser "Fossil command now appears to be available. You may wish to answer N to exit and customize it - but default config may be ok. Type the word 'continue' to proceed with default configuration."] + set answer [punk::mix::util::askuser "Fossil command now appears to be available. You may wish to answer N to exit and customize it - but default config may be ok. Type the word 'continue' to proceed with default configuration."] if {[string tolower $answer] ne "continue"} { return } @@ -281,7 +298,22 @@ namespace eval punk::mix::commandset::project { #set layout_path ... } - + #G-087 stage 4: materialize the layout's overlay chain (custom-over-vendor merging, + #.anti deletion markers, name@base derived layouts) into a single effective source folder. + #Single-folder chains with no .anti markers are used in place (staged 0) - e.g the thin + #punk.project layout and module-carried layouts - so the common case copies nothing here. + package require punk::mix::commandset::layout + set materialized [punk::mix::commandset::layout::lib::layout_materialize $layout_name -layoutinfo $layout_info] + set layout_sourcepath [dict get $materialized path] + if {[dict get $materialized staged]} { + puts stdout "layout '$layout_name' composed from [llength [dict get $materialized chain]]-folder overlay chain into staging: $layout_sourcepath" + foreach chainfolder [dict get $materialized chain] { + puts stdout " overlay source: $chainfolder" + } + if {[llength [dict get $materialized deletions]]} { + puts stdout " .anti deletions applied: [dict get $materialized deletions]" + } + } #todo - detect whether inside cwd-project or inside a different project set projectdir $projectparentdir/$projectname @@ -296,7 +328,7 @@ namespace eval punk::mix::commandset::project { puts stderr "mix new WARNING: target project location is within a git repo based at [punk::repo::find_git $projectparentdir]" puts stderr "The new project will create a fossil repository (which you are free to ignore - but but will be used to confirm project base)" puts stderr "If you intend to use both git and fossil in the same project space - you should research and understand the details and any possible interactions/issues" - set answer [util::askuser "Do you want to proceed to create a project based at: $projectdir? Y|N"] + set answer [punk::mix::util::askuser "Do you want to proceed to create a project based at: $projectdir? Y|N"] if {[string tolower $answer] ne "y"} { puts stderr "mix new aborting due to user response '$answer' (required Y|y to proceed) use -confirm 0 to avoid prompts." return @@ -307,7 +339,7 @@ namespace eval punk::mix::commandset::project { puts stderr "mix new WARNING: target project location is within an open fossil repo based at [punk::repo::find_fossil $projectparentdir] NESTED fossil repository" if {$opt_confirm ni [list 0 no false]} { puts stderr "If you proceed - the new project's fossil repo will be created using the --nested flag" - set answer [util::askuser "Do you want to proceed to create a NESTED project based at: $projectdir? Y|N"] + set answer [punk::mix::util::askuser "Do you want to proceed to create a NESTED project based at: $projectdir? Y|N"] if {[string tolower $answer] ne "y"} { puts stderr "mix new aborting due to user response '$answer' (required Y|y to proceed) use -confirm 0 to avoid prompts." return @@ -324,7 +356,7 @@ namespace eval punk::mix::commandset::project { } elseif {$project_dir_exists && $opt_force} { puts stderr "mix new WARNING: -force 1 was supplied. Will copy layout $layout_path using -force option to overwrite from template" if {$opt_confirm ni [list 0 no false]} { - set answer [util::askuser "Do you want to proceed to possibly overwrite existing files in $projectdir? Y|N"] + set answer [punk::mix::util::askuser "Do you want to proceed to possibly overwrite existing files in $projectdir? Y|N"] if {[string tolower $answer] ne "y"} { puts stderr "mix new aborting due to user response '$answer' (required Y|y to proceed) use -confirm 0 to avoid prompts." return @@ -335,7 +367,7 @@ namespace eval punk::mix::commandset::project { if {$opt_confirm} { puts stderr $warnmsg set msg "Do you want to proceed to possibly overwrite some existing files in $projectdir? Y|N" - set answer [util::askuser $msg] + set answer [punk::mix::util::askuser $msg] if {[string tolower $answer] ne "y"} { puts stderr "mix new aborting due to user response '$answer' (required Y|y to proceed) use -confirm 0 to avoid prompt." return @@ -390,34 +422,28 @@ namespace eval punk::mix::commandset::project { file mkdir $projectdir - puts stdout ">>> about to call punkcheck::install $layout_path $projectdir" + puts stdout ">>> about to call punkcheck::install $layout_sourcepath $projectdir" set resultdict [dict create] - set antipaths [list {*}{ + #src/doc handled separately below with SYNCED-TARGETS semantics. + #(.anti overlay deletion markers are applied at materialization time and never reach $layout_sourcepath) + set exclude_paths [list {*}{ src/doc/* src/doc/include/* - src/PROJECT_LAYOUTS_* - }] - - #set exclude_paths [list {*}{ - # _ignore_* - # **/_ignore_* - #}] - set exclude_paths [list {*}{ }] #default -exclude-paths-core will stop .fossil* from being updated - which is generally desirable as these are likely to be customized if {$opt_force} { puts stdout "copying layout files - with force applied - overwrite all-targets" - set resultdict [punkcheck::install $layout_path $projectdir -installer project.new -createempty 1 -overwrite ALL-TARGETS -exclude-paths [concat $antipaths $exclude_paths]] + set resultdict [punkcheck::install $layout_sourcepath $projectdir -installer project.new -createempty 1 -overwrite ALL-TARGETS -exclude-paths $exclude_paths] } else { puts stdout "copying layout files - (if source file changed)" - set resultdict [punkcheck::install $layout_path $projectdir -installer project.new -createempty 1 -overwrite installedsourcechanged-targets -exclude-paths [concat $antipaths $exclude_paths]] + set resultdict [punkcheck::install $layout_sourcepath $projectdir -installer project.new -createempty 1 -overwrite installedsourcechanged-targets -exclude-paths $exclude_paths] } puts stdout [punkcheck::summarize_install_resultdict $resultdict] - if {[file exists $layout_path/src/doc]} { + if {[file exists $layout_sourcepath/src/doc]} { puts stdout "copying layout src/doc files (if target missing)" - set resultdict [punkcheck::install $layout_path/src/doc $projectdir/src/doc -createdir 1 -punkcheck-folder $projectdir -installer project.new -overwrite SYNCED-TARGETS] + set resultdict [punkcheck::install $layout_sourcepath/src/doc $projectdir/src/doc -createdir 1 -punkcheck-folder $projectdir -installer project.new -overwrite SYNCED-TARGETS] puts stdout [punkcheck::summarize_install_resultdict $resultdict] } else { puts stdout "no src/doc in source template - update not required" @@ -427,17 +453,17 @@ namespace eval punk::mix::commandset::project { #In this case we need to override the default exclude-paths_core - as .fossil-xxx folders need to be installed from template if missing, or if target is uncustomized. ## default_excludepaths_core [list "#*" "**/#*" "_aside" "**/_aside" "_build" "**/_build" ".git" "**/.git" ".fossil*" "**/.fossil*"] set override_excludepaths_core [list #* **/#* _aside **/_aside .git **/.git] - if {[file exists $layout_path/.fossil-custom]} { + if {[file exists $layout_sourcepath/.fossil-custom]} { puts stdout "copying layout src/.fossil-custom files (if target missing or uncustomised)" - set resultdict [punkcheck::install $layout_path/.fossil-custom $projectdir/.fossil-custom -createdir 1 -createempty 1 -punkcheck-folder $projectdir -installer project.new -exclude-paths-core $override_excludepaths_core -overwrite SYNCED-TARGETS] + set resultdict [punkcheck::install $layout_sourcepath/.fossil-custom $projectdir/.fossil-custom -createdir 1 -createempty 1 -punkcheck-folder $projectdir -installer project.new -exclude-paths-core $override_excludepaths_core -overwrite SYNCED-TARGETS] puts stdout [punkcheck::summarize_install_resultdict $resultdict] } else { puts stdout "no .fossil-custom in source template - update not required" } - if {[file exists $layout_path/.fossil-settings]} { + if {[file exists $layout_sourcepath/.fossil-settings]} { puts stdout "copying layout src/.fossil-settings files (if target missing or uncustomised)" - set resultdict [punkcheck::install $layout_path/.fossil-settings $projectdir/.fossil-settings -createdir 1 -createempty 1 -punkcheck-folder $projectdir -installer project.new -exclude-paths-core $override_excludepaths_core -overwrite SYNCED-TARGETS] + set resultdict [punkcheck::install $layout_sourcepath/.fossil-settings $projectdir/.fossil-settings -createdir 1 -createempty 1 -punkcheck-folder $projectdir -installer project.new -exclude-paths-core $override_excludepaths_core -overwrite SYNCED-TARGETS] puts stdout [punkcheck::summarize_install_resultdict $resultdict] } else { puts stdout "no .fossil-settings in source template - update not required" @@ -453,8 +479,8 @@ namespace eval punk::mix::commandset::project { #scan all files in template # #TODO - deck command to substitute templates? - set templateinfo_list [punk::mix::commandset::layout::lib::layout_scan_for_template_files $opt_layout] - set stripprefix [file normalize $layout_path] + set templateinfo_list [punk::mix::commandset::layout::lib::layoutfolder_scan_for_template_files $layout_sourcepath] + set stripprefix [file normalize $layout_sourcepath] set tagmap [list [lib::template_tag project] $projectname] if {[llength $templateinfo_list]} { @@ -485,6 +511,91 @@ namespace eval punk::mix::commandset::project { } } + if {[dict get $materialized staged]} { + #layout copying and template scanning are done with the staging composition - + #remove it (the deterministic path under the temp dir is recomposed on demand) + catch {file delete -force $layout_sourcepath} + } + + #G-087 stage 5: stamp the generated project's own identity/version metadata plus a + #generation provenance baseline. Written programmatically - layout payloads must never + #ship a punkproject.toml (a payload copy would make the store folder itself present as + #a candidate project root - punk::repo hazard, G-033). Only-if-missing so -update/-force + #reruns never clobber a project's own version history or original generation record. + set pp_file [file join $projectdir punkproject.toml] + if {![file exists $pp_file]} { + set fd [open $pp_file w] + fconfigure $fd -translation lf + puts $fd "\[project\]" + puts $fd "name = \"$projectname\"" + puts $fd "version = \"0.1.0\"" + close $fd + puts stdout "seeded punkproject.toml (\[project\] name '$projectname' version 0.1.0)" + } + set cl_file [file join $projectdir CHANGELOG.md] + if {![file exists $cl_file]} { + #seed the matching changelog header - without it the injected make.tcl's + #'projectversion' advisory check would warn in every generated project forever + set gendate [clock format [clock seconds] -format %Y-%m-%d] + set fd [open $cl_file w] + fconfigure $fd -translation lf + puts $fd "# Changelog" + puts $fd "" + puts $fd "All notable changes to the $projectname project version are documented here." + puts $fd "The latest `## \[X.Y.Z\]` header must match the `version` field in `punkproject.toml`." + puts $fd "Entries are newest-first; one bullet per notable change." + puts $fd "" + puts $fd "## \[0.1.0\] - $gendate" + puts $fd "" + puts $fd "- project generated from punkshell layout '$layout_name' (dev project.new)" + close $fd + puts stdout "seeded CHANGELOG.md (\[0.1.0\] - $gendate)" + } + set origin_file [file join $projectdir punkorigin.toml] + if {![file exists $origin_file]} { + #generation provenance baseline - INTERIM carrier pending the G-027 pull-update + #design (G-027's pull maintains a committed baseline of origin identity + pulled + #version; it should adopt or migrate this record - format marked for that purpose). + #Provenance deliberately does not live in punkproject.toml's [project] table + #(schema contested - see goals G-047/G-065/G-057). + set shellbase [file dirname [file dirname [file normalize [info nameofexecutable]/___]]] + set origin_projectroot [punk::repo::find_project $shellbase] + set origin_version "" + if {$origin_projectroot ne "" && [file exists [file join $origin_projectroot punkproject.toml]]} { + #punkshell project version is only derivable when the generating shell belongs + #to a checkout - bare kits carry no version stamp (G-025 proposed would add one) + if {![catch {package require punkboot::utils}]} { + catch {set origin_version [punkboot::utils::read_punkproject_version [file join $origin_projectroot punkproject.toml]]} + } + } + set templates_version [package provide punk::mix::templates] + set layout_target [file normalize [dict get $layout_info path]] + if {[dict exists $layout_info basefolder]} { + set storeroot [file normalize [dict get $layout_info basefolder]] + if {[string equal -length [string length $storeroot/] $storeroot/ $layout_target]} { + set layout_target [string range $layout_target [string length $storeroot/] end] + } + } + set genutc [clock format [clock seconds] -format %Y-%m-%dT%H:%M:%SZ -timezone :UTC] + set fd [open $origin_file w] + fconfigure $fd -translation lf + puts $fd "# punkorigin.toml - generation/pull provenance baseline for this project's" + puts $fd "# punkshell-derived infrastructure. Seeded by punkshell 'dev project.new'." + puts $fd "# INTERIM record format pending the punkshell G-027 pull-update design, which" + puts $fd "# maintains this baseline (origin identity + pulled version) on each pull." + puts $fd "format = 1" + puts $fd "" + puts $fd "\[generation\]" + puts $fd "utc = \"$genutc\"" + puts $fd "layout = \"$layout_name\"" + puts $fd "layout_target = \"$layout_target\"" + puts $fd "punk_mix_templates_version = \"$templates_version\"" + puts $fd "punkshell_version = \"$origin_version\"" + puts $fd "origin_projectroot = \"$origin_projectroot\"" + close $fd + puts stdout "seeded punkorigin.toml (generation provenance: layout '$layout_name', origin version '[expr {$origin_version ne "" ? $origin_version : "unknown"}]')" + } + ::cd $projectdir if {[file exists $projectdir/src/modules]} { @@ -503,10 +614,10 @@ namespace eval punk::mix::commandset::project { #we should rarely if ever want to force any src/modules to be overwritten if {$opt_force} { if {$has_pod} { - set answer [util::askuser "OVERWRITE the src/modules file $podfile ?? (generally not desirable) Y|N"] + set answer [punk::mix::util::askuser "OVERWRITE the src/modules file $podfile ?? (generally not desirable) Y|N"] set overwrite_type zip } else { - set answer [util::askuser "OVERWRITE the src/modules file $tmfile ?? (generally not desirable) Y|N"] + set answer [punk::mix::util::askuser "OVERWRITE the src/modules file $tmfile ?? (generally not desirable) Y|N"] set overwrite_type $opt_type } if {[string tolower $answer] eq "y"} { @@ -591,7 +702,7 @@ namespace eval punk::mix::commandset::project { } if {$first_fossil} { #fossil commit may prompt user for input.. runx runout etc will pause with no prompts - util::do_in_path $projectdir { + punk::mix::util::do_in_path $projectdir { set fossilcommit [run -n {*}$fossil_prog commit -m "initial project commit"] } if {[dict get $fossilcommit exitcode] != 0} { @@ -1064,7 +1175,7 @@ namespace eval punk::mix::commandset::project { return [pwd] } } else { - set answer [util::askuser "Change directory to working folder - select a number from 1 to [llength $col_rowids] or any other key to cancel."] + set answer [punk::mix::util::askuser "Change directory to working folder - select a number from 1 to [llength $col_rowids] or any other key to cancel."] if {[string trim $answer] in $col_rowids} { set index [expr {$answer - 1}] set workingdir [lindex $workdirs $index] diff --git a/src/modules/punk/mix/commandset/project-buildversion.txt b/src/modules/punk/mix/commandset/project-buildversion.txt index 848bf240..8651ceb8 100644 --- a/src/modules/punk/mix/commandset/project-buildversion.txt +++ b/src/modules/punk/mix/commandset/project-buildversion.txt @@ -1,6 +1,9 @@ -0.3.0 +0.4.0 #First line must be a semantic version number #all other lines are ignored. +#0.4.0 - G-087 stage 4: project.new copies the layout from punk::mix::commandset::layout::lib::layout_materialize's effective content folder - overlay chains (custom-over-vendor merging, .anti deletion markers, name@base derived layouts) are composed into a disposable staging dir (deleted after use); single-folder chains with no .anti markers keep the direct store-folder path (no behaviour change for punk.project / module-carried layouts). Template scanning uses the new path-based layoutfolder_scan_for_template_files against the effective folder. The src/PROJECT_LAYOUTS_* install exclusion was retired with the marker folder (G-087 stage 5) and the misnamed antipaths variable folded into exclude_paths. +#0.4.0 - G-087 stage 5: project.new seeds generated-project version metadata post-materialization (all only-if-missing, never from payload): punkproject.toml ([project] name + version 0.1.0), CHANGELOG.md with a matching ## [0.1.0] header (keeps the injected make.tcl projectversion advisory quiet), and punkorigin.toml - an interim generation-provenance baseline (utc, layout name + resolved store target, punk::mix::templates version, punkshell version when derivable from the generating shell's checkout via punkboot::utils, origin projectroot path hint) for the G-027 pull-update design to adopt. +#0.3.1 - fix: project.new validates module names (projectname-derived default and explicit -modules entries, modulename-version forms accepted) before any folder/fossil-repo creation - a dashed project name (e.g infra-man) previously failed late inside module.new after the project tree and fossil repo already existed #0.3.0 - G-087 stage 3: new lib::bootsupport_inject (+ lib::_tm_version_sort) - project.new now injects bootsupport modules into the generated project from the generating shell's current trees, driven by the thin layout's include_modules.config manifests (shell-project bootsupport tree preferred; standalone kits resolve via tcl::tm::list; 999999.* dev versions never injected; punkcheck-recorded with plain-copy degradation) #0.3.0 - bootsupport_inject also tree-copies the shell project's src/bootsupport/lib* pkgIndex-based lib trees (e.g fileutil - required by the generated make.tcl under plain tclsh); standalone kits emit a notice (lib set has no manifest) - generated make.tcl remains runnable via the kit executable #0.3.0 - bootsupport_inject carries adjacent non-.tm support files beside matched modules (struct sets_*.tcl loader companions, textutil .tex hyphenation data) via new private helper _bootsupport_inject_copyfile; bootsupport include_modules.config gained the previously-missing cmdline and struct::set entries (files existed recordless in bootsupport; fileutil requires cmdline, make.tcl uses struct::set) diff --git a/src/project_layouts/README.md b/src/project_layouts/README.md index eb205f3c..89fe24e7 100644 --- a/src/project_layouts/README.md +++ b/src/project_layouts/README.md @@ -83,28 +83,66 @@ module snapshot trees. `packages`/`project`) keeps each layout's src/make.tcl, src/build.tcl and bootsupport manifest copies current from the canonical src/ files, then syncs the punk.project payload into the punk::mix::templates modpod source - tree (excluding bin/sdx.kit and the src/PROJECT_LAYOUTS_* marker). Records - live in src/project_layouts/.punkcheck. Never hand-edit the synced copies - or the modpod copy - edit the canonical files/store layout and rebuild. + tree (excluding bin/sdx.kit). Records live in + src/project_layouts/.punkcheck. Never hand-edit the synced copies or the + modpod copy - edit the canonical files/store layout and rebuild. - A manifest edit (src/bootsupport/modules*/include_modules.config) reaches the layouts and the modpod only on the next such build - `make.tcl bootsupport` alone updates the checkout's own bootsupport, not the layouts. +- Alongside the layout copy and injection, `dev project.new` seeds the + generated project's own identity metadata programmatically (each + only-if-missing, never shipped inside a payload): punkproject.toml + (`[project]` name + starting version 0.1.0), a matching CHANGELOG.md + `## [0.1.0]` header, and punkorigin.toml - a generation provenance + baseline (layout name + resolved store target, punk::mix::templates + version, punkshell version when derivable) that is the interim carrier + for the G-027 pull-update design. + +Overlay composition (G-087 stage 4) +----------------------------------- + +A layout's effective content is composed from an ordered chain of store +folders by punk::mix::commandset::layout::lib::layout_materialize (used by +`dev project.new` and the layout listing/scanning commands): + + @> + vendor// + custom// + +- Same-name files in a higher layer replace lower-layer files; folders merge + recursively. +- A marker named .anti (zero-byte file recommended; a marker folder is + also accepted) deletes as contributed by lower layers, and never + appears in the composed output. A marker paired with a same-name folder in + the same layer replaces the folder instead of merging. +- A folder named @ is a derived layout: it presents + under its own name (via its own layout ref) and starts from the base + layout's full chain. See custom/punk/othersample@sample-0.1 (with its ref + in src/decktemplates/custom/punk/layout_refs/) for a working example of + custom-over-vendor override, a .anti deletion and derivation; semantics + detail in custom/punk/sample-0.1/extra.txt. +- Chains of one folder with no .anti markers (e.g the thin punk.project + layout, module-carried layouts) are used in place - no copying. Anything + else is composed into a disposable staging directory (deterministic + per-layout path under the system temp dir, marked with a #layout_staging + file) which project.new removes after use. Store structure --------------- - src/project_layouts/{vendor,custom}/{,_project}/ + src/project_layouts/{vendor,custom}// - vendor/ trees should not be customised directly; place overrides under - custom/. The pseudo-vendor _project holds project-local staging copies - (currently empty - the former custom/_project fat bootsupport-fragment - copies were retired by G-087 stage 3, injection having made them - redundant). -- Overlay semantics (custom-over-vendor merging, .anti deletion markers, - newname@base derived layouts, _project staging) are documented design - - see custom/punk/sample-0.1/extra.txt - but are NOT yet implemented: - project.new currently copies exactly one resolved layout folder. - -Goal G-087 (goals/G-087-thin-project-layouts.md) tracks the remaining work: -the staging-directory overlay merge primitive (stage 4) and the -marker/vendor-matrix decisions (stage 5). + custom/ - the one override level. +- The former _project pseudo-vendor store level is retired for layouts + (G-087 stage 5 decision): its store content was deleted as redundant by + stage 3 (generation-time injection replaced the fat staging fragments), + and _project folders take no part in overlay chains. The _project concept + remains in use for decktemplates module/utility templates only. +- The unimplemented child-side src/PROJECT_LAYOUTS_CREATE_FROM_... marker + folder was likewise retired (G-087 stage 5): with thin layouts, generated + projects need only refs + thin payloads, and post-generation refresh + belongs to the G-027 pull design. + +G-087 (thin layouts, fauxlink refs, overlay merge - achieved 2026-07-19) is +recorded in goals/archive/G-087-thin-project-layouts.md. diff --git a/src/project_layouts/custom/punk/othersample@sample-0.1/.gitignore.anti b/src/project_layouts/custom/punk/othersample@sample-0.1/.gitignore.anti new file mode 100644 index 00000000..e69de29b diff --git a/src/project_layouts/custom/punk/othersample@sample-0.1/README.md b/src/project_layouts/custom/punk/othersample@sample-0.1/README.md index 01b4fce5..59c326f7 100644 --- a/src/project_layouts/custom/punk/othersample@sample-0.1/README.md +++ b/src/project_layouts/custom/punk/othersample@sample-0.1/README.md @@ -1,14 +1,23 @@ %project% ============================== -Another example of customisation of a file from a vendor layout -Due to the foldername othersample@sample-0.1, this layout presents under the name punk.othersample even though there is no 'othersample' template structure within vendor/punk - -The original file exists at project_layouts/vendor/punk/sample-0.1/README.md - -Due to this file's location at project_layouts/custom/punk/sample-0.1/README.md it will be used instead of the original. - -see also extra.txt for notes regarding customisation of vendor layouts. +Example of a DERIVED layout. + +Due to the folder name othersample@sample-0.1, this layout presents under its +own name (visible as punk.othersample via the layout ref in +src/decktemplates/custom/punk/layout_refs/) even though there is no +'othersample' folder within vendor/punk. The part after the @ names the base +layout: this layout's content starts from the full overlay chain of +sample-0.1 (vendor/punk/sample-0.1 merged with custom/punk/sample-0.1), with +this folder's own content applied on top. + +This README overrides the one inherited from the sample-0.1 chain, and the +zero-byte .gitignore.anti marker beside it deletes the .gitignore inherited +from vendor/punk/sample-0.1 - so a project generated from punk.othersample +contains this README, extra.txt from custom/punk/sample-0.1, and no +.gitignore. + +See custom/punk/sample-0.1/extra.txt for the full overlay/merge semantics. + diff --git a/src/project_layouts/custom/punk/sample-0.1/extra.txt b/src/project_layouts/custom/punk/sample-0.1/extra.txt index 67231ae1..0ab7a25e 100644 --- a/src/project_layouts/custom/punk/sample-0.1/extra.txt +++ b/src/project_layouts/custom/punk/sample-0.1/extra.txt @@ -1,11 +1,33 @@ -example of an additional file added to a vendor layout - -All files/folders from the vendor/layoutname tree are included unless they are over-ridden by a file of the same name in custom/vendor/layoutname or unless there is a file/folder of the same name except suffixed with .anti - -For example, to completely replace src/modules in a vendor layout; -create an empty custom/vendor/layoutname/src/modules.anti folder -Then create a custom/vendor/layoutname/src/modules folder with your custom contents. - -To just override a single module in vendor/layoutname/src/modules - just put a replacement module with the same name (or named with a later version) in custom/vendor/layoutname/src/modules. -All the existing modules from vendor/layoutname/src/modules will then be present in the resulting layout along with your modified/new module. - +example of an additional file added to a vendor layout + +Overlay/merge semantics (implemented by punk::mix::commandset::layout::lib::layout_materialize, +used by dev project.new - G-087 stage 4): + +All files/folders from the vendor// tree are included unless they are +over-ridden by a file of the same name in custom// or deleted by a +marker of the same name suffixed with .anti. Same-name folders merge recursively; the layer +order is vendor then custom, so custom content wins. + +.anti markers: +- A zero-byte FILE named .anti is the recommended marker form (empty folders cannot be + committed to git/fossil). A marker FOLDER named .anti is also accepted, matching the + originally documented form. Either deletes - file or folder - as contributed by + lower layers, and the marker itself never appears in the composed layout. +- To completely replace src/modules from a vendor layout: put a zero-byte + custom///src/modules.anti marker file alongside a + custom///src/modules folder with your custom contents. The marker + is applied before the same layer's own content is copied. +- To just override a single module in vendor//src/modules - put a replacement + module with the same name in custom///src/modules. All the existing + modules from the vendor layer will then be present in the resulting layout along with your + modified/new module. + +Derived layouts: +- A folder named @ presents as its own layout (given a layout ref + naming it). Its content starts from the base layout's full overlay chain with the derived + folder's own content (and .anti markers) applied on top. See ../othersample@sample-0.1 for + a working example (it also demonstrates a .anti deletion). + +Composition happens in a disposable staging directory at generation time - the store folders +are never modified, and a layout whose chain is a single folder with no .anti markers is used +in place without any copying. diff --git a/src/tests/modules/punk/mix/testsuites/layout/materialize.test b/src/tests/modules/punk/mix/testsuites/layout/materialize.test new file mode 100644 index 00000000..238198aa --- /dev/null +++ b/src/tests/modules/punk/mix/testsuites/layout/materialize.test @@ -0,0 +1,370 @@ +package require tcltest + +package require punk::mix::commandset::layout + +#added 2026-07-19 (agent, G-087) - stage 4 overlay merge primitive: layout_resolve_chain / +#layout_stage_chain / layout_materialize in punk::mix::commandset::layout::lib, exercised +#against synthetic stores (no punk.templates capability context needed - layout_materialize +#is fed -layoutinfo directly). The repo fixtures (vendor/custom punk/sample-0.1 + +#othersample@sample-0.1) are exercised end-to-end via dev project.new - see the G-087 detail +#file verification records. + +namespace eval ::testspace { + namespace import ::tcltest::* + + variable BASE [makeDirectory g087_materialize] + + proc fwrite {path content} { + file mkdir [file dirname $path] + set fd [open $path w] + fconfigure $fd -translation lf + puts -nonewline $fd $content + close $fd + } + proc fread {path} { + set fd [open $path r] + set data [read $fd] + close $fd + return $data + } + #relative sorted listing of all files (not dirs) below a folder - forward slashes + proc treefiles {folder} { + set files [list] + set queue [list ""] + while {[llength $queue]} { + set rel [lindex $queue 0] + set queue [lrange $queue 1 end] + set here [expr {$rel eq "" ? $folder : [file join $folder $rel]}] + foreach entry [lsort -unique [glob -nocomplain -dir $here -tails * .*]] { + if {$entry in {. ..}} {continue} + set relentry [expr {$rel eq "" ? $entry : "$rel/$entry"}] + if {[file isdirectory [file join $here $entry]]} { + lappend queue $relentry + } else { + lappend files $relentry + } + } + } + return [lsort $files] + } + #build a fresh synthetic store under $BASE/ and return its path + proc make_store {name} { + variable BASE + set store [file join $BASE $name] + file delete -force $store + file mkdir $store + return $store + } + + variable common { + set result "" + } + + # -- --- --- layout_resolve_chain --- --- -- + + test resolve_chain_vendor_only {single vendor folder yields single-entry chain}\ + -setup $common -body { + set store [make_store s1] + file mkdir $store/vendor/punk/basic + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk basic] + lappend result [llength $chain] + lappend result [expr {[lindex $chain 0] eq [file join $store vendor punk basic]}] + }\ + -cleanup {}\ + -result [list 1 1] + + test resolve_chain_custom_over_vendor {vendor tier precedes custom tier in chain order}\ + -setup $common -body { + set store [make_store s2] + file mkdir $store/vendor/punk/sample $store/custom/punk/sample + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample] + lappend result [llength $chain] + lappend result [expr {[lindex $chain 0] eq [file join $store vendor punk sample]}] + lappend result [expr {[lindex $chain 1] eq [file join $store custom punk sample]}] + }\ + -cleanup {}\ + -result [list 2 1 1] + + test resolve_chain_derived {name@base folder expands base chain first, derived folder last}\ + -setup $common -body { + set store [make_store s3] + file mkdir $store/vendor/punk/sample $store/custom/punk/sample "$store/custom/punk/other@sample" + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk other@sample] + lappend result [llength $chain] + lappend result [expr {[lindex $chain 0] eq [file join $store vendor punk sample]}] + lappend result [expr {[lindex $chain 1] eq [file join $store custom punk sample]}] + lappend result [expr {[lindex $chain 2] eq [file join $store custom punk other@sample]}] + }\ + -cleanup {}\ + -result [list 3 1 1 1] + + test resolve_chain_recursive_base {derivation chains recurse (a@b where b is b@c)}\ + -setup $common -body { + set store [make_store s4] + file mkdir $store/vendor/punk/c "$store/custom/punk/b@c" "$store/custom/punk/a@b" + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk a@b] + lappend result [llength $chain] + lappend result [expr {[lindex $chain 0] eq [file join $store vendor punk c]}] + lappend result [expr {[lindex $chain 1] eq [file join $store custom punk b@c]}] + lappend result [expr {[lindex $chain 2] eq [file join $store custom punk a@b]}] + }\ + -cleanup {}\ + -result [list 3 1 1 1] + + test resolve_chain_cycle {derivation cycle terminates with each folder listed once}\ + -setup $common -body { + set store [make_store s5] + file mkdir "$store/custom/punk/a@b" "$store/custom/punk/b@a" + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk a@b] + #cycle guard stops the b->a expansion; both folders appear exactly once + lappend result [llength $chain] + lappend result [llength [lsort -unique $chain]] + }\ + -cleanup {}\ + -result [list 2 2] + + test resolve_chain_missing {nonexistent layout name yields empty chain}\ + -setup $common -body { + set store [make_store s6] + file mkdir $store/vendor/punk + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk nosuch] + llength $chain + }\ + -cleanup {}\ + -result 0 + + # -- --- --- layout_stage_chain --- --- -- + + test stage_override_and_add {custom same-name file wins, extra file added, folders merge}\ + -setup $common -body { + set store [make_store s7] + fwrite $store/vendor/punk/sample/README.md "vendor-readme" + fwrite $store/vendor/punk/sample/src/modules/mod.txt "vendor-mod" + fwrite $store/custom/punk/sample/README.md "custom-readme" + fwrite $store/custom/punk/sample/extra.txt "extra" + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample] + set staging [file join $store staging] + set sres [punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging] + lappend result [fread $staging/README.md] + lappend result [fread $staging/extra.txt] + lappend result [fread $staging/src/modules/mod.txt] + lappend result [dict get $sres copied] + }\ + -cleanup {}\ + -result [list custom-readme extra vendor-mod 4] + + test stage_anti_file_deletes_file {zero-byte .anti file marker deletes lower-layer file}\ + -setup $common -body { + set store [make_store s8] + fwrite $store/vendor/punk/sample/.gitignore "ignore-content" + fwrite $store/vendor/punk/sample/keep.txt "keep" + file mkdir $store/custom/punk/sample + fwrite $store/custom/punk/sample/.gitignore.anti "" + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample] + set staging [file join $store staging] + set sres [punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging] + lappend result [file exists $staging/.gitignore] + lappend result [file exists $staging/keep.txt] + lappend result [dict get $sres deletions] + }\ + -cleanup {}\ + -result [list 0 1 .gitignore] + + test stage_anti_markers_never_copied {no .anti entries appear in the composed staging}\ + -setup $common -body { + set store [make_store s9] + fwrite $store/vendor/punk/sample/a.txt "a" + fwrite $store/custom/punk/sample/a.txt.anti "" + fwrite $store/custom/punk/sample/sub/b.txt.anti "" + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample] + set staging [file join $store staging] + punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging + set antis [list] + foreach f [treefiles $staging] { + if {[string match *.anti $f]} {lappend antis $f} + } + lappend result $antis + lappend result [file exists $staging/a.txt] + }\ + -cleanup {}\ + -result [list {} 0] + + test stage_anti_folder_replace {folder.anti marker plus replacement folder in same layer fully replaces}\ + -setup $common -body { + set store [make_store s10] + fwrite $store/vendor/punk/sample/src/modules/old1.txt "old1" + fwrite $store/vendor/punk/sample/src/modules/old2.txt "old2" + #marker as zero-byte FILE (recommended - committable), replacement folder alongside + fwrite $store/custom/punk/sample/src/modules.anti "" + fwrite $store/custom/punk/sample/src/modules/new.txt "new" + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample] + set staging [file join $store staging] + punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging + lappend result [treefiles $staging/src/modules] + }\ + -cleanup {}\ + -result [list new.txt] + + test stage_anti_dir_marker_form {a directory named .anti also acts as a deletion marker}\ + -setup $common -body { + set store [make_store s11] + fwrite $store/vendor/punk/sample/src/modules/old.txt "old" + file mkdir $store/custom/punk/sample/src/modules.anti + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample] + set staging [file join $store staging] + punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging + lappend result [file exists $staging/src/modules] + lappend result [file exists $staging/src/modules.anti] + }\ + -cleanup {}\ + -result [list 0 0] + + test stage_type_conflicts {higher-layer entry wins on file/folder type conflicts}\ + -setup $common -body { + set store [make_store s12] + fwrite $store/vendor/punk/sample/thing "vendor-file" + fwrite $store/vendor/punk/sample/other/nested.txt "vendor-nested" + fwrite $store/custom/punk/sample/thing/inner.txt "custom-inner" + fwrite $store/custom/punk/sample/other "custom-file" + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample] + set staging [file join $store staging] + punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging + lappend result [file isdirectory $staging/thing] + lappend result [fread $staging/thing/inner.txt] + lappend result [expr {![file isdirectory $staging/other]}] + lappend result [fread $staging/other] + }\ + -cleanup {}\ + -result [list 1 custom-inner 1 custom-file] + + test stage_empty_dir_preserved {empty folders in a layer are created in staging}\ + -setup $common -body { + set store [make_store s13] + file mkdir $store/vendor/punk/sample/emptydir + fwrite $store/vendor/punk/sample/a.txt "a" + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample] + set staging [file join $store staging] + punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging + file isdirectory $staging/emptydir + }\ + -cleanup {}\ + -result 1 + + test stage_refuse_clobber {existing non-empty dir without staging marker is refused}\ + -setup $common -body { + set store [make_store s14] + fwrite $store/vendor/punk/sample/a.txt "a" + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample] + set staging [file join $store precious] + fwrite $staging/important.txt "do-not-delete" + set code [catch {punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging} emsg] + lappend result $code + lappend result [file exists $staging/important.txt] + lappend result [string match "*refusing to delete*" $emsg] + }\ + -cleanup {}\ + -result [list 1 1 1] + + test stage_restage_over_marker {a previous staging composition (marker present) is replaced}\ + -setup $common -body { + set store [make_store s15] + fwrite $store/vendor/punk/sample/a.txt "one" + set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample] + set staging [file join $store staging] + punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging + fwrite $store/vendor/punk/sample/b.txt "two" + set code [catch {punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging} sres] + lappend result $code + lappend result [file exists $staging/a.txt] + lappend result [file exists $staging/b.txt] + }\ + -cleanup {}\ + -result [list 0 1 1] + + # -- --- --- layout_materialize --- --- -- + + test materialize_fastpath {single-folder chain with no .anti markers returns the store folder unstaged}\ + -setup $common -body { + set store [make_store s16] + fwrite $store/vendor/punk/solo/a.txt "a" + set linfo [list path [file join $store vendor punk solo] basefolder $store sourceinfo {}] + set mat [punk::mix::commandset::layout::lib::layout_materialize punk.solo -layoutinfo $linfo] + lappend result [dict get $mat staged] + lappend result [expr {[dict get $mat path] eq [file join $store vendor punk solo]}] + lappend result [llength [dict get $mat chain]] + }\ + -cleanup {}\ + -result [list 0 1 1] + + test materialize_single_with_antis {a single folder containing .anti markers is staged with markers stripped}\ + -setup $common -body { + set store [make_store s17] + fwrite $store/vendor/punk/solo/a.txt "a" + fwrite $store/vendor/punk/solo/ghost.txt.anti "" + set linfo [list path [file join $store vendor punk solo] basefolder $store sourceinfo {}] + set staging [file join $store staging] + set mat [punk::mix::commandset::layout::lib::layout_materialize punk.solo -layoutinfo $linfo -stagingdir $staging] + lappend result [dict get $mat staged] + lappend result [file exists [dict get $mat path]/a.txt] + lappend result [file exists [dict get $mat path]/ghost.txt.anti] + }\ + -cleanup {}\ + -result [list 1 1 0] + + test materialize_opaque_path {a layout folder outside the tier/vendor store shape stays an opaque single-folder chain}\ + -setup $common -body { + set store [make_store s18] + fwrite $store/somewhere/deep/layoutx/a.txt "a" + set linfo [list path [file join $store somewhere deep layoutx] basefolder $store sourceinfo {}] + set mat [punk::mix::commandset::layout::lib::layout_materialize punk.layoutx -layoutinfo $linfo] + lappend result [dict get $mat staged] + lappend result [llength [dict get $mat chain]] + lappend result [expr {[dict get $mat path] eq [file join $store somewhere deep layoutx]}] + }\ + -cleanup {}\ + -result [list 0 1 1] + + test materialize_project_pseudovendor_opaque {_project pseudo-vendor folders take no part in overlay chains}\ + -setup $common -body { + set store [make_store s19] + fwrite $store/vendor/_project/sample/a.txt "proj-a" + fwrite $store/custom/_project/sample/a.txt "proj-custom-a" + set linfo [list path [file join $store vendor _project sample] basefolder $store sourceinfo {}] + set mat [punk::mix::commandset::layout::lib::layout_materialize sample -layoutinfo $linfo] + lappend result [dict get $mat staged] + lappend result [llength [dict get $mat chain]] + }\ + -cleanup {}\ + -result [list 0 1] + + test materialize_derived_full {derived layout materializes base chain content with overrides and .anti deletions}\ + -setup $common -body { + set store [make_store s20] + fwrite $store/vendor/punk/sample/README.md "vendor-readme" + fwrite $store/vendor/punk/sample/.gitignore "gitignore" + fwrite $store/vendor/punk/sample/src/modules/README.md "modules-readme" + fwrite $store/custom/punk/sample/README.md "custom-readme" + fwrite $store/custom/punk/sample/extra.txt "extra" + fwrite "$store/custom/punk/other@sample/README.md" "derived-readme" + fwrite "$store/custom/punk/other@sample/.gitignore.anti" "" + set linfo [list path [file join $store custom punk other@sample] basefolder $store sourceinfo {}] + set staging [file join $store staging] + set mat [punk::mix::commandset::layout::lib::layout_materialize punk.other -layoutinfo $linfo -stagingdir $staging] + set effective [dict get $mat path] + lappend result [dict get $mat staged] + lappend result [llength [dict get $mat chain]] + lappend result [fread $effective/README.md] + lappend result [fread $effective/extra.txt] + lappend result [fread $effective/src/modules/README.md] + lappend result [file exists $effective/.gitignore] + lappend result [dict get $mat deletions] + }\ + -cleanup {}\ + -result [list 1 3 derived-readme extra modules-readme 0 .gitignore] + + variable BASE + file delete -force $BASE + + cleanupTests +} +namespace delete ::testspace