Browse Source

G-087 stages 4+5: overlay merge primitive, marker/matrix retirement, version seeding

Stage 4 - documented overlay semantics implemented (commandset::layout 0.2.0):
layout_materialize resolves a layout to its overlay chain (vendor tier then
custom tier per store+vendor, <name>@<base> derived-layout folders expanding
their base chain first, cycle-guarded) and composes multi-folder chains into a
disposable staging dir applying <item>.anti deletion markers (zero-byte marker
files recommended - committable, unlike the originally documented empty marker
folders, which are also accepted). Single-folder chains with no markers are
used in place, so punk.project (store and zipfs modpod) behaviour is unchanged.
project.new (commandset::project 0.4.0) copies from the materialized path and
the layout listing/scanning commands show the effective merged view.
Fixtures: punk.othersample is now a live derived layout (new custom-deck
fauxlink ref + .gitignore.anti demo marker); extra.txt/store README rewritten
to the implemented semantics. New tcltest suite
src/tests/modules/punk/mix/testsuites/layout/materialize.test (20 tests).
Anti markers deliberately stay a plain filename convention rather than
tag-bearing fauxlinks: a fauxlink is a reference tooling may follow (fcat
auto-follows), a marker is a tombstone with no target.

Stage 5 - retirements and generated-project version seeding:
- The empty unimplemented child-side src/PROJECT_LAYOUTS_CREATE_FROM_... marker
  folder is deleted along with its project.new and make.tcl sync exclusions.
- The vendor/custom x _project matrix collapses for layouts: custom/<vendor> is
  the one override level; _project folders join no overlay chain (the
  pseudo-vendor remains for decktemplates module/utility templates only).
- project.new seeds generated projects post-materialization (only-if-missing,
  never from payload): punkproject.toml ([project] name + 0.1.0), a matching
  CHANGELOG.md ## [0.1.0] header, and punkorigin.toml - an interim generation
  provenance baseline (utc, layout name + resolved store target,
  punk::mix::templates version, punkshell version when derivable, origin path)
  for the G-027 pull design to adopt (noted in G-027).

Also: script-mode/programmatic project.new robustness (explicit shellrun
require; punk::mix::util askuser/do_in_path/foreach-file fully qualified -
the util:: shorthand exists only after cli commandset import). Includes the
concurrently-landed (other session, previously uncommitted) 0.3.1 early
module-name validation fix in project.new, shipped within the 0.4.0 bump.

G-087 flipped achieved 2026-07-19 (verification in the detail file; archive
move follows in the next commit). punkshell 0.16.0 -> 0.17.0.

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 1 week ago
parent
commit
e2fc7332fe
  1. 2
      AGENTS.md
  2. 6
      CHANGELOG.md
  3. 7
      goals/G-012-template-payload-safety.md
  4. 6
      goals/G-025-exe-selfreport.md
  5. 10
      goals/G-027-derived-project-pull-updates.md
  6. 143
      goals/G-087-thin-project-layouts.md
  7. 9
      goals/G-094-punkcheck-single-lifecycle.md
  8. 2
      punkproject.toml
  9. 0
      src/decktemplates/custom/punk/layout_refs/othersample#custom+punk+othersample@sample-0.1.fauxlink
  10. 5
      src/make.tcl
  11. 412
      src/modules/punk/mix/commandset/layout-999999.0a1.0.tm
  12. 4
      src/modules/punk/mix/commandset/layout-buildversion.txt
  13. 173
      src/modules/punk/mix/commandset/project-999999.0a1.0.tm
  14. 5
      src/modules/punk/mix/commandset/project-buildversion.txt
  15. 70
      src/project_layouts/README.md
  16. 0
      src/project_layouts/custom/punk/othersample@sample-0.1/.gitignore.anti
  17. 25
      src/project_layouts/custom/punk/othersample@sample-0.1/README.md
  18. 44
      src/project_layouts/custom/punk/sample-0.1/extra.txt
  19. 370
      src/tests/modules/punk/mix/testsuites/layout/materialize.test

2
AGENTS.md

@ -205,7 +205,7 @@ The project version is fully independent of module versions. A module bump (even
- `lib/` (root) — Build output target for `tclsh src/make.tcl libs`
- `modules_tcl8/`, `modules_tcl9/` — Tcl version-specific build output targets
- `lib_tcl8/`, `lib_tcl9/` — Tcl version-specific build output targets
- `src/project_layouts/` — Thin project layout payloads (structure, scripts, docs, manifests - no stored bootsupport/vfs module snapshots; G-087). The build's thin-layout sync step (`make.tcl modules`/`libs`/`packages`/`project`) keeps each layout's `src/make.tcl`, `src/build.tcl` and bootsupport `include_modules.config` copies current, and syncs the punk.project payload into the `punk::mix::templates` modpod source tree; `dev project.new` injects bootsupport modules/libs into generated projects from the generating shell at generation time. Agents must not directly edit files under `src/project_layouts/` unless the user explicitly names a path there or specifically requests layout changes; never hand-edit the synced copies (they are punkcheck-managed build outputs).
- `src/project_layouts/` — Thin project layout payloads (structure, scripts, docs, manifests - no stored bootsupport/vfs module snapshots; G-087, achieved - see goals/archive/G-087-thin-project-layouts.md). The build's thin-layout sync step (`make.tcl modules`/`libs`/`packages`/`project`) keeps each layout's `src/make.tcl`, `src/build.tcl` and bootsupport `include_modules.config` copies current, and syncs the punk.project payload into the `punk::mix::templates` modpod source tree; `dev project.new` composes each layout's overlay chain (vendor then custom tier, `<name>@<base>` derived folders, `.anti` deletion markers - see the store README) via `layout_materialize`, injects bootsupport modules/libs from the generating shell at generation time, and seeds generated-project identity metadata (punkproject.toml, CHANGELOG.md, punkorigin.toml provenance baseline) programmatically. Agents must not directly edit files under `src/project_layouts/` unless the user explicitly names a path there or specifically requests layout changes; never hand-edit the synced copies (they are punkcheck-managed build outputs).
## Repo-wide Notes

6
CHANGELOG.md

@ -5,6 +5,12 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to
Entries are newest-first; one bullet per notable change. See the root `AGENTS.md`
"Project Versioning" section for the bump policy.
## [0.17.0] - 2026-07-19
- G-087 stages 4+5 (goal achieved) - overlay layout composition: `dev project.new` now materializes a layout's overlay chain before copying - custom-tier files override same-name vendor-tier files (folders merge recursively), zero-byte `<item>.anti` marker files delete lower-layer items, and a `<newname>@<base>` folder presents as a derived layout starting from its base layout's chain (new `punk::mix::commandset::layout::lib::layout_materialize` staging primitive; single-folder layouts such as punk.project are used in place, unchanged). The sample fixtures demonstrate all three (`punk.othersample` is a working derived layout with a `.anti` deletion). `layout.files`/`layout.templatefiles` list the effective merged view.
- Generated projects are version-seeded at creation (each only-if-missing): `punkproject.toml` (`[project]` name + version 0.1.0), a matching `CHANGELOG.md` `## [0.1.0]` seed, and `punkorigin.toml` - a generation provenance baseline (layout name + resolved store target, punk::mix::templates version, punkshell version when derivable, origin path) as the interim carrier for the planned G-027 pull updates.
- project.new fixes/robustness: module names (including the projectname-derived default) are validated before any folder or fossil-repo creation (a dashed project name previously failed late with the tree and repo already created; use -modules {} for dashed names wanting no initial module); generation now also works programmatically/script-mode (explicit shellrun dependency, fully-qualified punk::mix::util calls). The retired unimplemented `src/PROJECT_LAYOUTS_*` marker folder and its install/sync exclusions are gone; the `_project` pseudo-vendor store level is retired for layouts (custom/<vendor> is the one override level).
## [0.16.0] - 2026-07-19
- G-087 stage 3 - thin project layouts with generation-time injection: `dev project.new` now injects bootsupport into the generated project from the generating shell's current trees (modules per the layout's `include_modules.config` manifests plus adjacent support files; bootsupport lib trees from a shell project; standalone kits resolve modules from their own module paths and emit a notice for the manifest-less lib set). The ~40MB of stored bootsupport/vfs module snapshots left `src/project_layouts` (fat `custom/_project` fragments and their `_overrides` refs retired; vendor layouts thinned to structure/scripts/docs/manifests).

7
goals/G-012-template-payload-safety.md

@ -90,3 +90,10 @@ payload drift is unmanaged.
- G-062 (archived) recorded a follow-on in this territory: seeding LICENSE.txt into generated
project layouts (deliberately excluded from G-062's acceptance because src/project_layouts
sync is restricted) - see goals/archive/G-062-project-license-file.md.
- G-087 (archived - goals/archive/G-087-thin-project-layouts.md) reconfirmed at
each generation verification (stages 3 and 4, 2026-07-19): the trailing
fossil initial commit of a punk.project-generated project aborts on binary
payload content (bin/sdx.kit from the layout plus injected zip-based modpod
.tm modules) because thin layouts ship no binary-glob settings - this goal's
inert VCS-config payload treatment is the designated fix. Layouts without
binaries (sample/derived fixtures) commit cleanly.

6
goals/G-025-exe-selfreport.md

@ -91,3 +91,9 @@ Edge cases driving the acceptance clauses:
dev paths) would make that mixing observable instead of silent.
(G-030 achieved 2026-07-14 - goals/archive/G-030-maketcl-punkargs.md)
- Archived-goal references in this file: G-007 achieved 2026-07-05 (goals/archive/G-007-console-location-transparency.md);G-015 achieved 2026-07-07 (goals/archive/G-015-script-subcommand-piped-stdin.md).
- G-087 (archived - goals/archive/G-087-thin-project-layouts.md) added a
consumer for the kit version stamp this goal would provide: generated
projects' punkorigin.toml records punkshell_version only when the
generating shell belongs to a checkout (read via punkboot::utils from
punkproject.toml); bare kits currently record an empty version - a stamped
kit-level punkshell version would close that gap.

10
goals/G-027-derived-project-pull-updates.md

@ -134,3 +134,13 @@ What a pull may touch splits by ownership, not by file type:
legacy installfile_* pipeline G-094 retires; G-095 adds concurrent-writer
safety - a pull writing a derived project's .punkcheck while builds run there
is a future instance of exactly its scenario.
- G-087 (archived - goals/archive/G-087-thin-project-layouts.md) seeded the
interim provenance carrier this goal's baseline record needs: generated
projects now carry a root punkorigin.toml (format = 1; [generation] utc,
layout name + resolved store target, punk::mix::templates version,
punkshell version when derivable, origin projectroot path hint), written
programmatically by project.new, only-if-missing. This goal should adopt
or migrate that record (its header text marks it as G-027-interim) and
maintain it on each pull (origin identity + pulled version). G-087 also
retired the child-side PROJECT_LAYOUTS_CREATE_FROM_... marker on the
grounds that post-generation refresh belongs to this goal's pull.

143
goals/G-087-thin-project-layouts.md

@ -1,6 +1,6 @@
# G-087 Thin project layouts: injected infrastructure, fauxlink layout refs, working overlay merge
Status: active
Status: achieved 2026-07-19
Scope: src/project_layouts/ (payload store restructure), src/decktemplates/ (layout_refs migration to fauxlink), src/modules/punk/mix/commandset/project-999999.0a1.0.tm (project.new generation path), src/modules/punk/mix/commandset/layout-999999.0a1.0.tm, src/modules/punk/cap/handlers/templates-999999.0a1.0.tm (ref resolution), src/modules/punk/mix/#modpod-templates-999999.0a1.0/ (module-shipped refs/layouts), src/make.tcl (layout sync steps)
Goal: project.new generates projects from thin layouts (structure, scripts, docs - no stored bootsupport/module/vfs snapshot trees) whose heavy infrastructure is injected at generation time from the generating shell's own current trees; layout references are fauxlink files (one ref concept repo-wide, no bespoke .ref grammar, no real symlinks); and the documented overlay semantics (custom-over-vendor, .anti deletions, name@base derived layouts, _project staging) are implemented by a single staging-directory materialization primitive - so a bare built kit can generate a current project, and the committed multi-megabyte stale layout snapshots are retired.
Acceptance: hygiene stage - in a punkshell checkout `dev project.new` with defaulted -layout resolves an exact layout name (no tcl::prefix ambiguity); the no-handler warning paths execute without error (put/puts typos fixed) and an absolute-pathtype provider registration does not raise an expr syntax error; test/junk refs (test2, sample-0.1 duplicate, the a.txt-only punk.shell-0.1) no longer present as selectable layouts; src/project_layouts/README.md describes the implemented punk.templates+refs mechanism including the ref filename grammar. Pivot stage - a project generated from the thin punk.project layout carries bootsupport modules matching the generating shell's current src/bootsupport (verified by comparing a marker module version such as punk::mix::cli, not a stored snapshot vintage); src/project_layouts contains no embedded bootsupport/vfs module trees and make.tcl's layout bootsupport-sync step is retired; layout refs are .fauxlink files resolved via the fauxlink module with the bespoke .ref parser removed, every layout ref carrying a non-empty nominal name (leading-# punkcheck-exclusion hazard documented in this file); a bare built kit run outside any project lists at least the module-shipped punk.project layout and generates a working project from it; and one overlay case exercising custom-over-vendor override, a .anti deletion, and a name@base derived layout (the existing sample-0.1/othersample fixtures) materializes correctly through the staging primitive into a generated project.
@ -107,7 +107,12 @@ Staged; hygiene first since it is independently shippable.
refs + thin payloads; heavy parts arrive by the same injection and refresh via
G-027); decide whether the vendor/custom x `_project` matrix at both levels still
earns its complexity once overlays are cheap (vendor/_project and custom/_project
may collapse into one override level).
may collapse into one override level); and generated-project version seeding
(2026-07-19 assessment): project.new stamps a minimal punkproject.toml
(`[project]` name + starting version 0.1.0) plus a matching CHANGELOG.md seed
into the generated project programmatically post-materialization, and records
generation provenance in the G-027-designated carrier - constraints and
rationale in Notes ("Version/provenance decisions").
## Alternatives considered
@ -164,6 +169,32 @@ Staged; hygiene first since it is independently shippable.
- The `#n` duplicate-suffix mechanism in `_get_itemdict` and the priority ordering
(module < shellproject < currentproject < absolute < adhoc) are sound and carry over
unchanged.
- Version/provenance decisions (2026-07-19 naming+version-tracking assessment,
separate session, read-only): the default layout's name/folder must NOT track
punkproject.toml's `[project]` version - version series are independent per the
root AGENTS.md; punkshell's version is unavailable in bare kits (no VFS
punkproject.toml or stamped literal; G-025 proposed would add one); post-thin
layouts the vintage lives in the injected module versions (this goal's
marker-module principle), so a punkshell-tracking layout name would misstate it;
and per-release renames would reset punkcheck path-keyed install history (both
generation and -update of existing projects), orphan `name@base` derived layouts
(base is matched by folder name), and re-create the tcl::prefix ambiguity the
hygiene stage fixed. Stamping constraints for the stage-5 item: write
punkproject.toml programmatically post-materialization, never ship one inside a
payload (punk::repo::is_candidate_root existence check - the repo.tm comment and
G-033 hazard); seed CHANGELOG.md `## [0.1.0]` alongside or the injected make.tcl
projectversion advisory warns forever in every generated project; confirm
punkboot::utils reaches generated projects (make.tcl's check requires it).
Provenance (layout name + resolved target, punk::mix::templates version,
punkshell version when derivable - checkout contexts only until a kit stamp
exists) belongs in the G-027-designated carrier (fauxlink TOML body / provenance
record), not in `[project]`, whose schema is contested by G-047/G-065/G-057. A
template tag for the new project's version, if wanted, must be %projectversion% -
%version% already means module version in module.new. Any versionless->highest
layout-version selection must compare with `package vcompare` (the handler's
within-base ordering is lexical lsort) and must never mix the layout
format-version series (0.x) with a project-version series (vcompare makes
0.15.0 > 0.2 win permanently).
## Progress
@ -266,6 +297,15 @@ Notes from the work:
spun out as G-094 (single OO lifecycle) and G-095 (concurrent-writer
safety); the MAX_PATH normalization candidate above is flagged in G-094's
notes as a natural rider on its load/save consolidation.
- FIXED 2026-07-19 (commandset::project 0.3.1, shipped within the 0.4.0
stage-4 bump - the 0.3.1 changelog line is retained in buildversion.txt):
project.new with a dashed project name (e.g infra-man) failed late inside
module.new - after the project tree and fossil repo already existed -
because the projectname-derived default module name carried the dash
(invalid in Tcl package names). Module names (defaulted and explicit
-modules entries, modulename-version forms accepted) are now validated
before any folder or fossil-repo creation; -modules {} remains the escape
hatch for dashed project names wanting no initial module.
### 2026-07-19 stage 3 complete: thin layouts + generation-time injection (increment 3)
@ -369,9 +409,96 @@ Notes / residue from stage 3:
- sdx.kit remains in the store layout (grandfathered committed binary,
excluded from the modpod) - G-004/G-006 decide its future.
Remaining for acceptance (Approach stages 4-5):
- staging-directory overlay merge primitive exercising the sample-0.1/othersample
fixtures (stage 4).
- child-side PROJECT_LAYOUTS_CREATE_FROM_... marker retirement and the
vendor/custom x _project matrix decision (stage 5).
### 2026-07-19 stages 4+5 complete: overlay merge primitive, marker/matrix retirement, version seeding (increment 4)
Landed (punk::mix::commandset::layout 0.2.0, punk::mix::commandset::project
0.4.0 (also carrying the separately-recorded 0.3.1 module-name validation
fix), make.tcl sync exclusion trimmed, punkshell 0.17.0):
- Overlay merge primitive in commandset::layout::lib: layout_materialize
(entry point; returns the effective content folder), layout_resolve_chain
(vendor-then-custom tier chain per store, <name>@<base> derived-layout
expansion, recursive + cycle-guarded), layout_stage_chain (staging-dir
composition; per layer .anti deletions apply before the layer's own copy;
#layout_staging marker file guards against clobbering non-staging dirs),
private helpers _dir_entries/_find_anti_relpaths/_merge_copy_into.
Design decisions: .anti markers are zero-byte FILES by convention (the
documented empty-folder form is also accepted but cannot be committed to
git/fossil); marker files stay a plain filename convention rather than
tag-bearing fauxlinks - a fauxlink is a reference every fauxlink-aware
tool may follow (fcat auto-follows .fauxlink), while an anti-marker is a
tombstone with no target; the "one ref concept" principle covers things
that point. Chains of one folder with no markers are used in place
(staged 0) - punk.project (store and zipfs modpod) is byte-identical in
behaviour to stage 3. Fixtures: othersample gained its ref
(decktemplates/custom/punk/layout_refs/othersample#custom+punk+othersample@sample-0.1.fauxlink,
link_as round-trip verified) and a .gitignore.anti demo marker; extra.txt
rewritten to the implemented semantics; store README documents the
composition.
- project.new copies from the materialized path (staging deleted after the
template scan), and the misnamed antipaths exclusion list folded into
exclude_paths. layout.files/layout.templatefiles display the effective
overlay-merged view (new path-based layoutfolder_scan_for_template_files
core).
- Stage 5 retirements/decisions: the empty (untracked) child-side
src/PROJECT_LAYOUTS_CREATE_FROM_DECKTEMPLATES_LAYOUT_REFERENCES marker
folder deleted with its project.new + make.tcl sync exclusions (thin
layouts + injection made it moot; refresh belongs to G-027 pull). The
vendor/custom x _project matrix is COLLAPSED for layouts: custom/<vendor>
is the one override level; _project store folders take no part in overlay
chains (materialize treats them as opaque single-folder layouts) - the
_project pseudo-vendor remains only for decktemplates module/utility
templates.
- Stage 5 version seeding (per the 2026-07-19 assessment constraints):
project.new stamps generated projects post-materialization, each
only-if-missing and never from payload - punkproject.toml ([project] name
+ version 0.1.0), CHANGELOG.md with matching ## [0.1.0] header (keeps the
injected make.tcl projectversion advisory quiet), and punkorigin.toml: an
INTERIM generation-provenance baseline (format = 1; [generation] utc,
layout name, resolved store target, punk::mix::templates version,
punkshell version when derivable via punkboot::utils from the generating
shell's checkout, origin projectroot path hint) for G-027 to adopt or
migrate (noted in G-027's detail file).
- Robustness fixes en route (script-mode/programmatic project.new - found
driving generation via `punk902z src script`): explicit `package require
shellrun` (runx/run aliases exist only in shell contexts otherwise),
punk::mix::util::askuser/do_in_path fully qualified (the util:: shorthand
is wired only when a cli imports the commandset), commandset::layout
requires punk::mix::util and calls foreach-file fully qualified.
Verification (2026-07-19, native tclsh 9.0.3 + punk902z src):
- New suite src/tests/modules/punk/mix/testsuites/layout/materialize.test:
20/20 (chain shapes incl. recursion + cycle, override/add, file+dir .anti
forms, folder replace, type conflicts, dotfiles, empty dirs, marker
stripping, refuse-clobber, restage, fastpath/opaque/_project materialize
via -layoutinfo). Full punk/mix subtree 66 pass / 1 known skip;
punkcheck install+installtrack 64/64.
- End-to-end (punk902z src script, FOSSIL_HOME+HOME diverted, fresh
.fossils): punk.othersample listed via the new custom-deck ref and
generated a project through the 3-folder staged chain - derived README,
extra.txt present, .gitignore anti-deleted, src/modules/README.md from
the vendor base, no *.anti in output, module created, staging dir
removed, trailing fossil initial commit OK (sample layouts ship no
binaries). punk.sample-0.1 generated with the custom README override and
.gitignore retained. punk.project generated fast-path (staged 0, no
staging output) with bootsupport injection intact (113 modules + 3 lib
folders); punk.project#2 (zipfs modpod store) materializes fast-path with
the store-shape decomposition working on //zipfs:/ paths. All three
seeded punkproject.toml/CHANGELOG.md/punkorigin.toml; `make.tcl
projectversion` inside the generated punk.project project reports
consistency OK (punkboot::utils reaches generated projects). The
punk.project trailing fossil commit still aborts on binary payload
content (bin/sdx.kit + zip-based .tm modules) - pre-existing G-012
residue, unchanged by this work.
- Known limitation reconfirmed en route (pre-existing, G-094 rider): deep
target dirs make punkcheck's relative source paths exceed Windows
MAX_PATH ('could not read ... no such file' with a long ../ chain) -
generation into a shallow directory works.
Acceptance review at flip: hygiene criteria verified 2026-07-17 (increment
1); fauxlink-ref criteria verified 2026-07-17 (increment 2); thin-layout +
injection + bare-kit criteria verified 2026-07-19 (increment 3); the
overlay case (custom-over-vendor override + .anti deletion + name@base
derived layout via the sample-0.1/othersample fixtures materializing into a
generated project) verified this increment. No remaining items.

9
goals/G-094-punkcheck-single-lifecycle.md

@ -80,9 +80,12 @@ lifecycle, not an ad-hoc bypass with its own record code.
bootsupport's copy; a dev shell runs the built one) - re-vendor bootsupport
promptly after landing, and sequence this refactor before any on-disk
protocol change from G-095.
- G-087 recorded a punkcheck MAX_PATH candidate (deep relative source chains
exceed Windows path limits, unnormalized) - natural rider on this goal's
load/save consolidation.
- G-087 (archived - goals/archive/G-087-thin-project-layouts.md) recorded a
punkcheck MAX_PATH candidate (deep relative source chains exceed Windows
path limits, unnormalized) - natural rider on this goal's load/save
consolidation. Reconfirmed 2026-07-19 during the stage-4 verification:
project generation into a deep directory fails reading the layout store
manifests through a long ../ chain.
- The characterization suites were written expressly to lock behaviour "before
any refactoring" - this is that refactoring.
- punkcheck::cli (status/paths) is a read-only consumer and stays untouched.

2
punkproject.toml

@ -1,4 +1,4 @@
[project]
name = "punkshell"
version = "0.16.0"
version = "0.17.0"
license = "BSD-2-Clause"

0
src/decktemplates/custom/punk/layout_refs/othersample#custom+punk+othersample@sample-0.1.fauxlink

5
src/make.tcl

@ -3383,7 +3383,8 @@ if {$::punkboot::command in {project packages modules libs}} {
#store -> modpod payload sync: the punk::mix::templates module ships the thin punk.project layout
#so module-pathtype layout refs resolve and a bare kit outside any project can generate from it.
#The modpod copy is build-managed - do not hand-edit it. Excludes: bin/sdx.kit (no new binaries
#inside zip-based .tm modules) and the unimplemented child-side src/PROJECT_LAYOUTS_* marker.
#inside zip-based .tm modules). (The unimplemented child-side src/PROJECT_LAYOUTS_* marker was
#retired by G-087 stage 5 - thin layouts + generation-time injection made it moot.)
set store_layout $sourcefolder/project_layouts/vendor/punk/project-0.1
set modpod_layout "$sourcefolder/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1"
if {[file isdirectory $store_layout]} {
@ -3398,7 +3399,7 @@ if {$::punkboot::command in {project packages modules libs}} {
-overwrite installedsourcechanged-targets
-createempty 1
-exclude-paths-core {#* **/#* _aside **/_aside _build **/_build .git **/.git}
-exclude-paths {bin/sdx.kit src/PROJECT_LAYOUTS_*}
-exclude-paths {bin/sdx.kit}
}]
puts stdout [punkcheck::summarize_install_resultdict $resultdict]
if {![catch {package require punk::mix::cli}]

412
src/modules/punk/mix/commandset/layout-999999.0a1.0.tm

@ -22,6 +22,7 @@ package require punk::args
#sort of a circular dependency when commandset loaded by punk::mix::cli - that's ok, but this could theoretically be loaded by another cli and with another base
package require punk::mix
package require punk::mix::base
package require punk::mix::util
package require punk::lib
package require textblock
@ -85,12 +86,12 @@ namespace eval punk::mix::commandset::layout {
puts stderr "layout '$layout' not found."
return
}
set layoutinfo [dict get $layoutdict $layout]
set layoutfolder [dict get $layoutinfo path]
#G-087 stage 4: display against the effective (overlay-materialized) content
set materialized [lib::layout_materialize $layout -layoutinfo [dict get $layoutdict $layout]]
set layoutfolder [dict get $materialized path]
set stripprefix [file normalize $layoutfolder]
set templatefiles_and_tags [lib::layout_scan_for_template_files $layout]
set templatefiles_and_tags [lib::layoutfolder_scan_for_template_files $layoutfolder]
set flatlist [list]
foreach entry $templatefiles_and_tags {
lassign $entry templatefullpath tags
@ -222,7 +223,358 @@ namespace eval punk::mix::commandset::layout {
}
return $layout_dict
}
# -- --- --- --- --- --- --- --- --- --- ---
# G-087 stage 4: overlay merge primitive
# A layout's effective content can be composed from an ordered chain of store folders
# (vendor tier, then custom tier, with <name>@<base> folders deriving from another
# layout's chain) with .anti markers deleting lower-layer items.
# layout_materialize is the entry point; project.new copies from its result path.
# -- --- --- --- --- --- --- --- --- --- ---
punk::args::define {
@id -id ::punk::mix::commandset::layout::lib::layout_resolve_chain
@cmd -name punk::mix::commandset::layout::lib::layout_resolve_chain\
-summary\
"Resolve a layout store folder name to its ordered overlay chain"\
-help\
"Returns the ordered list (lowest priority first) of existing store folders
that contribute to the named layout within a single project_layouts store:
<base chain - when a contributing folder is named <name>@<base>>
<storeroot>/vendor/<vendor>/<name> (and <name>@<base> siblings)
<storeroot>/custom/<vendor>/<name> (and <name>@<base> siblings)
A folder named <name>@<base> declares a derived layout: <base> names another
layout in the same store+vendor whose chain is expanded first (recursively,
cycle-guarded with a warning). Layout names must not themselves contain @ -
the first @ in a folder name is the derivation separator.
The result may be empty if none of the candidate folders exist."
@values -min 3 -max 4
storeroot -type string -help\
"Store root folder (a src/project_layouts folder, or a module's
templates/project_layouts folder)"
vendor -type string -help\
"Vendor folder name within the store's vendor/ and custom/ tiers"
layoutfoldername -type string -help\
"Layout folder name - either <name> or the derived form <name>@<base>"
seen -type list -optional 1 -help\
"Internal recursion guard - layout names already being expanded"
}
proc layout_resolve_chain {storeroot vendor layoutfoldername {seen {}}} {
#manual args - see PUNKARGS ::punk::mix::commandset::layout::lib::layout_resolve_chain
set atpos [string first @ $layoutfoldername]
if {$atpos > 0} {
set layoutname [string range $layoutfoldername 0 $atpos-1]
} else {
set layoutname $layoutfoldername
}
if {$layoutname in $seen} {
puts stderr "layout_resolve_chain WARNING - layout derivation cycle detected at '$layoutname' (expansion path: $seen) - stopping expansion"
return [list]
}
lappend seen $layoutname
#glob-escape the name so metachars in a folder name can't broaden the derived-folder search
set globname [string map [list \\ \\\\ \[ \\\[ \] \\\] \{ \\\{ \} \\\} * \\* ? \\?] $layoutname]
set bases [list]
set tierfolders [list]
foreach tier {vendor custom} {
set tdir [file join $storeroot $tier $vendor]
set plain [file join $tdir $layoutname]
if {[file isdirectory $plain]} {
lappend tierfolders $plain
}
foreach d [lsort [glob -nocomplain -dir $tdir -type d ${globname}@*]] {
lappend tierfolders $d
set b [string range [file tail $d] [string length ${layoutname}@] end]
if {$b ne "" && $b ni $bases} {
lappend bases $b
}
}
}
set chain [list]
foreach b $bases {
foreach p [layout_resolve_chain $storeroot $vendor $b $seen] {
if {$p ni $chain} {
lappend chain $p
}
}
}
foreach p $tierfolders {
if {$p ni $chain} {
lappend chain $p
}
}
return $chain
}
punk::args::define {
@id -id ::punk::mix::commandset::layout::lib::layout_stage_chain
@cmd -name punk::mix::commandset::layout::lib::layout_stage_chain\
-summary\
"Compose an overlay chain of layout folders into a staging directory"\
-help\
"Deletes and recreates stagingdir, then merges each chain folder into it in
order (lowest priority first). For each layer, .anti deletion markers are
applied before the layer's own content is copied:
- an entry named <item>.anti (zero-byte file recommended; a marker folder
is also accepted) deletes <item> - file or folder - as contributed by
lower layers, and is itself never copied.
- a same-name file in a higher layer replaces the lower layer's file;
folders merge recursively. To fully replace a folder rather than merge
into it, pair a <folder>.anti marker with a replacement <folder> in the
same layer.
A #layout_staging marker file is written into stagingdir so later calls can
recognise the folder as disposable staging output (the leading # also keeps
the marker out of punkcheck-driven installs). If stagingdir already exists,
is non-empty and carries no such marker, the call errors rather than
deleting it.
Returns a dict: path (stagingdir), chain, deletions (relative paths removed
by .anti markers), copied (number of files copied)."
@values -min 2 -max 2
chain -type list -help\
"Ordered list of layout source folders, lowest priority first
(as returned by layout_resolve_chain)"
stagingdir -type string -help\
"Directory to compose into - deleted and recreated by this call"
}
proc layout_stage_chain {chain stagingdir} {
#manual args - see PUNKARGS ::punk::mix::commandset::layout::lib::layout_stage_chain
if {![llength $chain]} {
error "layout_stage_chain: empty chain"
}
foreach sourcefolder $chain {
if {![file isdirectory $sourcefolder]} {
error "layout_stage_chain: chain member '$sourcefolder' is not a directory"
}
}
set stagingdir [file normalize $stagingdir]
set marker [file join $stagingdir #layout_staging]
if {[file exists $stagingdir]} {
if {[llength [_dir_entries $stagingdir]] && ![file exists $marker]} {
error "layout_stage_chain: staging dir '$stagingdir' exists, is not empty, and has no #layout_staging marker - refusing to delete it. Supply an empty/nonexistent dir or a previous staging dir."
}
file delete -force $stagingdir
}
file mkdir $stagingdir
#write the marker first so an interrupted composition is still recognisable as staging output
set fd [open $marker w]
close $fd
set deletions [list]
set copied 0
foreach sourcefolder $chain {
foreach relanti [_find_anti_relpaths $sourcefolder] {
set victim [file join $stagingdir $relanti]
if {[file exists $victim]} {
file delete -force $victim
lappend deletions $relanti
}
}
incr copied [_merge_copy_into $sourcefolder $stagingdir]
}
return [dict create path $stagingdir chain $chain deletions $deletions copied $copied]
}
punk::args::define {
@id -id ::punk::mix::commandset::layout::lib::layout_materialize
@cmd -name punk::mix::commandset::layout::lib::layout_materialize\
-summary\
"Materialize a layout's overlay chain into a single source folder"\
-help\
"Resolves the named layout's overlay chain (layout_resolve_chain) within its
store and returns a folder holding the effective layout content.
When the chain is a single folder containing no .anti markers (the common
case - e.g the thin punk.project layout, or any module-carried layout) that
folder is returned as-is (staged 0) and no copying occurs. Otherwise the
chain is composed into a staging directory (layout_stage_chain) and the
staged folder is returned (staged 1) - the caller may delete it when done
(the default location is deterministic, so a later call recomposes it).
Layout folders whose store position does not follow the
<storeroot>/{vendor,custom}/<vendor>/<layoutfolder> shape (including any
under the retired _project pseudo-vendor) are treated as opaque
single-folder chains.
Returns a dict:
path - folder holding the effective layout content
staged - 1 if path is a disposable staging composition,
0 if it is the store folder itself
chain - ordered contributing folders, lowest priority first
deletions - relative paths removed by .anti markers (staged only)
copied - number of files copied into staging (staged only)"
@leaders -min 1 -max 1
layoutname -type string -optional 0 -help\
"Visible layout name as shown by 'dev project.layouts' (e.g punk.project)"
@opts
-layoutinfo -default "" -help\
"Layout info dict as returned by layouts_dict (keys path, basefolder, ...).
When empty, layoutname is resolved via layouts_dict - which requires the
punk.templates capability context of the calling shell."
-stagingdir -default "" -help\
"Directory to compose an overlay chain into when staging is needed.
Defaults to a deterministic per-layout-name folder under the system
temp directory."
@values -min 0 -max 0
}
proc layout_materialize {args} {
set argd [punk::args::parse $args withid ::punk::mix::commandset::layout::lib::layout_materialize]
set layoutname [dict get $argd leaders layoutname]
set layoutinfo [dict get $argd opts -layoutinfo]
set opt_stagingdir [dict get $argd opts -stagingdir]
if {$layoutinfo eq ""} {
set layoutdict [layouts_dict]
if {![dict exists $layoutdict $layoutname]} {
error "layout_materialize: layout '$layoutname' not found. Known layouts: [dict keys $layoutdict]"
}
set layoutinfo [dict get $layoutdict $layoutname]
}
set layoutfolder [dict get $layoutinfo path]
set chain [list $layoutfolder]
if {[dict exists $layoutinfo basefolder]} {
#decompose <storeroot>/<tier>/<vendor>/<layoutfoldername>
#anything else (unexpected depth, unknown tier, retired _project pseudo-vendor) stays an opaque single-folder chain
set storeroot [dict get $layoutinfo basefolder]
set rootparts [file split $storeroot]
set folderparts [file split $layoutfolder]
set rootlen [llength $rootparts]
if {[llength $folderparts] == $rootlen + 3 && [lrange $folderparts 0 $rootlen-1] eq $rootparts} {
lassign [lrange $folderparts end-2 end] tier vendor layoutfoldername
if {$tier in {vendor custom} && $vendor ne "_project"} {
set resolved [layout_resolve_chain $storeroot $vendor $layoutfoldername]
if {$layoutfolder in $resolved} {
set chain $resolved
}
}
}
}
#fast path - a single-source chain with no .anti markers is usable in place
if {[llength $chain] == 1 && ![llength [_find_anti_relpaths [lindex $chain 0]]]} {
return [dict create path [lindex $chain 0] staged 0 chain $chain deletions [list] copied 0]
}
if {$opt_stagingdir eq ""} {
set sanitized [regsub -all {[^A-Za-z0-9_.-]} $layoutname _]
set opt_stagingdir [file join [punk::lib::tempdir] punk_layout_staging $sanitized]
}
set stageinfo [layout_stage_chain $chain $opt_stagingdir]
return [dict create path [dict get $stageinfo path] staged 1 chain $chain deletions [dict get $stageinfo deletions] copied [dict get $stageinfo copied]]
}
punk::args::define {
@id -id ::punk::mix::commandset::layout::lib::_dir_entries
@cmd -name punk::mix::commandset::layout::lib::_dir_entries\
-summary\
"List folder entry tails including dotfiles, excluding . and .."\
-help\
"Internal helper for the overlay merge primitive. glob * misses leading-dot
entries on unix while matching them on windows - this returns the sorted
union of * and .* minus the . and .. entries so tree walks behave
identically on both."
@values -min 1 -max 1
folder -type string
}
proc _dir_entries {folder} {
#manual args - see PUNKARGS ::punk::mix::commandset::layout::lib::_dir_entries
set entries [list]
foreach e [lsort -unique [glob -nocomplain -dir $folder -tails * .*]] {
if {$e eq "." || $e eq ".."} {
continue
}
lappend entries $e
}
return $entries
}
punk::args::define {
@id -id ::punk::mix::commandset::layout::lib::_find_anti_relpaths
@cmd -name punk::mix::commandset::layout::lib::_find_anti_relpaths\
-summary\
"Recursively collect .anti marker targets below a layout folder"\
-help\
"Internal helper for the overlay merge primitive. Returns the relative paths
(marker name minus the .anti suffix) marked for deletion by <item>.anti
entries anywhere below basefolder. Marker entries may be files (zero-byte
recommended) or folders; marker folders are not walked. A bare '.anti'
entry has no target and is ignored with a warning."
@values -min 1 -max 2
basefolder -type string
prefix -type string -optional 1
}
proc _find_anti_relpaths {basefolder {prefix ""}} {
#manual args - see PUNKARGS ::punk::mix::commandset::layout::lib::_find_anti_relpaths
if {$prefix eq ""} {
set here $basefolder
} else {
set here [file join $basefolder $prefix]
}
set antis [list]
foreach entry [_dir_entries $here] {
if {$prefix eq ""} {
set rel $entry
} else {
set rel [file join $prefix $entry]
}
if {[string match *.anti $entry]} {
if {[string length $entry] == 5} {
puts stderr "layout overlay WARNING - ignoring degenerate '.anti' marker (no target name) at '[file join $here $entry]'"
continue
}
lappend antis [string range $rel 0 end-5]
} elseif {[file isdirectory [file join $here $entry]]} {
lappend antis {*}[_find_anti_relpaths $basefolder $rel]
}
}
return $antis
}
punk::args::define {
@id -id ::punk::mix::commandset::layout::lib::_merge_copy_into
@cmd -name punk::mix::commandset::layout::lib::_merge_copy_into\
-summary\
"Recursively merge-copy a folder over a target, skipping .anti markers"\
-help\
"Internal helper for the overlay merge primitive. Files overwrite same-name
targets; folders merge recursively; on a file/folder type conflict the
source entry wins (the conflicting target is deleted first). .anti marker
entries are never copied. Returns the number of files copied."
@values -min 2 -max 2
srcdir -type string
tgtdir -type string
}
proc _merge_copy_into {srcdir tgtdir} {
#manual args - see PUNKARGS ::punk::mix::commandset::layout::lib::_merge_copy_into
set copied 0
foreach entry [_dir_entries $srcdir] {
if {[string match *.anti $entry]} {
continue
}
set s [file join $srcdir $entry]
set t [file join $tgtdir $entry]
if {[file isdirectory $s]} {
if {[file exists $t] && ![file isdirectory $t]} {
#a lower layer contributed a file where this layer has a folder - the folder wins
file delete -force $t
}
file mkdir $t
incr copied [_merge_copy_into $s $t]
} else {
if {[file isdirectory $t]} {
#a lower layer contributed a folder where this layer has a file - the file wins
file delete -force $t
}
file copy -force $s $t
incr copied
}
}
return $copied
}
# -- --- --- --- --- --- --- --- --- --- ---
proc layout_all_files {layout} {
#todo - allow versionless layout name to pick highest version found
set layoutdict [layouts_dict]
@ -230,31 +582,53 @@ namespace eval punk::mix::commandset::layout {
puts stderr "layout '$layout' not found."
return
}
set layoutinfo [dict get $layoutdict $layout]
set layoutfolder [dict get $layoutinfo path]
#G-087 stage 4: list the effective (overlay-materialized) content, not just the primary store folder
set materialized [layout_materialize $layout -layoutinfo [dict get $layoutdict $layout]]
set layoutfolder [dict get $materialized path]
if {![file isdirectory $layoutfolder]} {
puts stderr "layout '$layout' points to path $layoutfolder - but it doesn't seem to exist"
}
set file_list [list]
util::foreach-file $layoutfolder path {
lappend file_list $path
set file_list [list]
#fully qualified - the util:: shorthand is only wired when a cli has imported this commandset
punk::mix::util::foreach-file $layoutfolder path {
if {[file tail $path] ne "#layout_staging"} {
lappend file_list $path
}
}
return $file_list
}
#
proc layout_scan_for_template_files {layout {tags {}}} {
#todo JMN
set layoutdict [layouts_dict]
if {![dict exists $layoutdict $layout]} {
puts stderr "layout '$layout' not found."
return
}
set layoutinfo [dict get $layoutdict $layout]
set layoutfolder [dict get $layoutinfo path]
#G-087 stage 4: scan the effective (overlay-materialized) content
set materialized [layout_materialize $layout -layoutinfo [dict get $layoutdict $layout]]
return [layoutfolder_scan_for_template_files [dict get $materialized path] $tags]
}
#use last matching layout found. review silent if multiple?
punk::args::define {
@id -id ::punk::mix::commandset::layout::lib::layoutfolder_scan_for_template_files
@cmd -name punk::mix::commandset::layout::lib::layoutfolder_scan_for_template_files\
-summary\
"Scan a layout content folder for files containing template tags"\
-help\
"Path-based core of layout_scan_for_template_files: scans the supplied
folder (typically the path returned by layout_materialize) and returns a
list of {fullpath found_tagnames} entries for files containing any of the
tag placeholders. With no tags supplied, the standard tag set (%project%)
is used."
@values -min 1 -max 2
layoutfolder -type string -help\
"Folder holding layout content to scan"
tags -type list -optional 1 -help\
"Tag placeholder literals to search for (defaults to the standard set)"
}
proc layoutfolder_scan_for_template_files {layoutfolder {tags {}}} {
#manual args - see PUNKARGS ::punk::mix::commandset::layout::lib::layoutfolder_scan_for_template_files
if {![llength $tags]} {
#todo - get standard tags from somewhere
set tagnames [list project]
@ -262,8 +636,12 @@ namespace eval punk::mix::commandset::layout {
lappend tags [string cat % $tn %] ;#make sure actual tag literal doesn't appear in this source file
}
}
set file_list [list]
util::foreach-file $layoutfolder path {
set file_list [list]
#fully qualified - the util:: shorthand is only wired when a cli has imported this commandset
punk::mix::util::foreach-file $layoutfolder path {
if {[file tail $path] eq "#layout_staging"} {
continue
}
set fd [open $path r]
fconfigure $fd -translation binary
set data [read $fd]

4
src/modules/punk/mix/commandset/layout-buildversion.txt

@ -1,4 +1,6 @@
0.1.1
0.2.0
#First line must be a semantic version number
#all other lines are ignored.
#0.2.0 - G-087 stage 4: overlay merge primitive in lib - layout_materialize (entry point returning the effective content folder; fast path returns the store folder untouched for single-source chains with no .anti markers), layout_resolve_chain (vendor-then-custom tier chain with <name>@<base> derived-layout expansion, cycle-guarded), layout_stage_chain (staging-dir composition applying .anti deletion markers; #layout_staging marker guards against clobbering non-staging dirs), plus private helpers _dir_entries/_find_anti_relpaths/_merge_copy_into. The _project pseudo-vendor participates in no chain (retired for layouts - G-087 stage 5 decision).
#0.2.0 - new lib::layoutfolder_scan_for_template_files (path-based core of layout_scan_for_template_files); layout_scan_for_template_files, layout_all_files and templatefiles.relative now operate on the overlay-materialized content rather than the primary store folder only (the staging marker file is excluded from listings/scans).
#0.1.1 - fix: no-handler warning paths in collection::references_as_dict and lib::layouts_dict used 'put' instead of 'puts' and then errored on an unset dict variable; they now warn correctly and return an empty dict. references_as_dict warning message no longer misattributes itself to layouts_dict.

173
src/modules/punk/mix/commandset/project-999999.0a1.0.tm

@ -183,6 +183,7 @@ namespace eval punk::mix::commandset::project {
set opt_update [dict get $opts -update]
# -- --- --- --- --- --- --- --- --- --- --- --- ---
set opt_modules [dict get $opts -modules]
set modules_defaulted 0
if {[llength $opt_modules] == 1 && [lindex $opt_modules 0] eq "\uFFFF"} {
#if not specified - add a single module matching project name
if {$opt_force || $opt_update} {
@ -191,6 +192,21 @@ namespace eval punk::mix::commandset::project {
set opt_modules [list]
} else {
set opt_modules [list [string tolower $projectname]] ;#default modules to lowercase as is the modern (tip 590) recommendation for Tcl
set modules_defaulted 1
}
}
#validate module names now - before any folder or fossil repo creation - rather than
#failing partway through generation when module.new runs. e.g a dash in the project
#name flows into the default module name, and dashes are invalid in Tcl package names
#except as the modulename-version separator.
foreach m $opt_modules {
lassign [punk::mix::util::split_modulename_version [string trimleft $m :]] mbase mversion
if {[catch {punk::mix::cli::lib::validate_modulename $mbase -errorprefix "module name"} vErr]} {
if {$modules_defaulted} {
error "punk mix project.new error: project name '$projectname' is not usable as the default module name ($vErr). Use a project name that is a valid Tcl package name (no dashes), or pass -modules {} for no initial module, or supply valid names via -modules."
} else {
error "punk mix project.new error: invalid -modules entry '$m' ($vErr)"
}
}
}
# -- --- --- --- --- --- --- --- --- --- --- --- ---
@ -198,12 +214,13 @@ namespace eval punk::mix::commandset::project {
#todo - install support binaries on a per-project basis in a way that doesn't impact machine (e.g not added to path) - cache in user config dir if possible, supply mechanism to clear cache
#
package require shellrun ;#run/runx global aliases - present in shell contexts, but not e.g punk script mode
set fossil_prog [auto_execok fossil]
if {![string length $fossil_prog]} {
puts stderr "The fossil program was not found. A fossil executable is required to use most deck features."
if {[string length [set scoop_prog [auto_execok scoop]]]} {
#restrict to windows?
set answer [util::askuser "scoop detected. Would you like deck to install fossil now using scoop? Y|N"]
set answer [punk::mix::util::askuser "scoop detected. Would you like deck to install fossil now using scoop? Y|N"]
if {[string tolower $answer] ne "y"} {
puts stderr "mix new aborting due to user response '$answer' (required Y|y to proceed) use -confirm 0 to avoid prompts."
return
@ -224,7 +241,7 @@ namespace eval punk::mix::commandset::project {
return
}
#todo - ask user if they want to configure fosssil first..
set answer [util::askuser "Fossil command now appears to be available. You may wish to answer N to exit and customize it - but default config may be ok. Type the word 'continue' to proceed with default configuration."]
set answer [punk::mix::util::askuser "Fossil command now appears to be available. You may wish to answer N to exit and customize it - but default config may be ok. Type the word 'continue' to proceed with default configuration."]
if {[string tolower $answer] ne "continue"} {
return
}
@ -281,7 +298,22 @@ namespace eval punk::mix::commandset::project {
#set layout_path ...
}
#G-087 stage 4: materialize the layout's overlay chain (custom-over-vendor merging,
#.anti deletion markers, name@base derived layouts) into a single effective source folder.
#Single-folder chains with no .anti markers are used in place (staged 0) - e.g the thin
#punk.project layout and module-carried layouts - so the common case copies nothing here.
package require punk::mix::commandset::layout
set materialized [punk::mix::commandset::layout::lib::layout_materialize $layout_name -layoutinfo $layout_info]
set layout_sourcepath [dict get $materialized path]
if {[dict get $materialized staged]} {
puts stdout "layout '$layout_name' composed from [llength [dict get $materialized chain]]-folder overlay chain into staging: $layout_sourcepath"
foreach chainfolder [dict get $materialized chain] {
puts stdout " overlay source: $chainfolder"
}
if {[llength [dict get $materialized deletions]]} {
puts stdout " .anti deletions applied: [dict get $materialized deletions]"
}
}
#todo - detect whether inside cwd-project or inside a different project
set projectdir $projectparentdir/$projectname
@ -296,7 +328,7 @@ namespace eval punk::mix::commandset::project {
puts stderr "mix new WARNING: target project location is within a git repo based at [punk::repo::find_git $projectparentdir]"
puts stderr "The new project will create a fossil repository (which you are free to ignore - but but will be used to confirm project base)"
puts stderr "If you intend to use both git and fossil in the same project space - you should research and understand the details and any possible interactions/issues"
set answer [util::askuser "Do you want to proceed to create a project based at: $projectdir? Y|N"]
set answer [punk::mix::util::askuser "Do you want to proceed to create a project based at: $projectdir? Y|N"]
if {[string tolower $answer] ne "y"} {
puts stderr "mix new aborting due to user response '$answer' (required Y|y to proceed) use -confirm 0 to avoid prompts."
return
@ -307,7 +339,7 @@ namespace eval punk::mix::commandset::project {
puts stderr "mix new WARNING: target project location is within an open fossil repo based at [punk::repo::find_fossil $projectparentdir] NESTED fossil repository"
if {$opt_confirm ni [list 0 no false]} {
puts stderr "If you proceed - the new project's fossil repo will be created using the --nested flag"
set answer [util::askuser "Do you want to proceed to create a NESTED project based at: $projectdir? Y|N"]
set answer [punk::mix::util::askuser "Do you want to proceed to create a NESTED project based at: $projectdir? Y|N"]
if {[string tolower $answer] ne "y"} {
puts stderr "mix new aborting due to user response '$answer' (required Y|y to proceed) use -confirm 0 to avoid prompts."
return
@ -324,7 +356,7 @@ namespace eval punk::mix::commandset::project {
} elseif {$project_dir_exists && $opt_force} {
puts stderr "mix new WARNING: -force 1 was supplied. Will copy layout $layout_path using -force option to overwrite from template"
if {$opt_confirm ni [list 0 no false]} {
set answer [util::askuser "Do you want to proceed to possibly overwrite existing files in $projectdir? Y|N"]
set answer [punk::mix::util::askuser "Do you want to proceed to possibly overwrite existing files in $projectdir? Y|N"]
if {[string tolower $answer] ne "y"} {
puts stderr "mix new aborting due to user response '$answer' (required Y|y to proceed) use -confirm 0 to avoid prompts."
return
@ -335,7 +367,7 @@ namespace eval punk::mix::commandset::project {
if {$opt_confirm} {
puts stderr $warnmsg
set msg "Do you want to proceed to possibly overwrite some existing files in $projectdir? Y|N"
set answer [util::askuser $msg]
set answer [punk::mix::util::askuser $msg]
if {[string tolower $answer] ne "y"} {
puts stderr "mix new aborting due to user response '$answer' (required Y|y to proceed) use -confirm 0 to avoid prompt."
return
@ -390,34 +422,28 @@ namespace eval punk::mix::commandset::project {
file mkdir $projectdir
puts stdout ">>> about to call punkcheck::install $layout_path $projectdir"
puts stdout ">>> about to call punkcheck::install $layout_sourcepath $projectdir"
set resultdict [dict create]
set antipaths [list {*}{
#src/doc handled separately below with SYNCED-TARGETS semantics.
#(.anti overlay deletion markers are applied at materialization time and never reach $layout_sourcepath)
set exclude_paths [list {*}{
src/doc/*
src/doc/include/*
src/PROJECT_LAYOUTS_*
}]
#set exclude_paths [list {*}{
# _ignore_*
# **/_ignore_*
#}]
set exclude_paths [list {*}{
}]
#default -exclude-paths-core will stop .fossil* from being updated - which is generally desirable as these are likely to be customized
if {$opt_force} {
puts stdout "copying layout files - with force applied - overwrite all-targets"
set resultdict [punkcheck::install $layout_path $projectdir -installer project.new -createempty 1 -overwrite ALL-TARGETS -exclude-paths [concat $antipaths $exclude_paths]]
set resultdict [punkcheck::install $layout_sourcepath $projectdir -installer project.new -createempty 1 -overwrite ALL-TARGETS -exclude-paths $exclude_paths]
} else {
puts stdout "copying layout files - (if source file changed)"
set resultdict [punkcheck::install $layout_path $projectdir -installer project.new -createempty 1 -overwrite installedsourcechanged-targets -exclude-paths [concat $antipaths $exclude_paths]]
set resultdict [punkcheck::install $layout_sourcepath $projectdir -installer project.new -createempty 1 -overwrite installedsourcechanged-targets -exclude-paths $exclude_paths]
}
puts stdout [punkcheck::summarize_install_resultdict $resultdict]
if {[file exists $layout_path/src/doc]} {
if {[file exists $layout_sourcepath/src/doc]} {
puts stdout "copying layout src/doc files (if target missing)"
set resultdict [punkcheck::install $layout_path/src/doc $projectdir/src/doc -createdir 1 -punkcheck-folder $projectdir -installer project.new -overwrite SYNCED-TARGETS]
set resultdict [punkcheck::install $layout_sourcepath/src/doc $projectdir/src/doc -createdir 1 -punkcheck-folder $projectdir -installer project.new -overwrite SYNCED-TARGETS]
puts stdout [punkcheck::summarize_install_resultdict $resultdict]
} else {
puts stdout "no src/doc in source template - update not required"
@ -427,17 +453,17 @@ namespace eval punk::mix::commandset::project {
#In this case we need to override the default exclude-paths_core - as .fossil-xxx folders need to be installed from template if missing, or if target is uncustomized.
## default_excludepaths_core [list "#*" "**/#*" "_aside" "**/_aside" "_build" "**/_build" ".git" "**/.git" ".fossil*" "**/.fossil*"]
set override_excludepaths_core [list #* **/#* _aside **/_aside .git **/.git]
if {[file exists $layout_path/.fossil-custom]} {
if {[file exists $layout_sourcepath/.fossil-custom]} {
puts stdout "copying layout src/.fossil-custom files (if target missing or uncustomised)"
set resultdict [punkcheck::install $layout_path/.fossil-custom $projectdir/.fossil-custom -createdir 1 -createempty 1 -punkcheck-folder $projectdir -installer project.new -exclude-paths-core $override_excludepaths_core -overwrite SYNCED-TARGETS]
set resultdict [punkcheck::install $layout_sourcepath/.fossil-custom $projectdir/.fossil-custom -createdir 1 -createempty 1 -punkcheck-folder $projectdir -installer project.new -exclude-paths-core $override_excludepaths_core -overwrite SYNCED-TARGETS]
puts stdout [punkcheck::summarize_install_resultdict $resultdict]
} else {
puts stdout "no .fossil-custom in source template - update not required"
}
if {[file exists $layout_path/.fossil-settings]} {
if {[file exists $layout_sourcepath/.fossil-settings]} {
puts stdout "copying layout src/.fossil-settings files (if target missing or uncustomised)"
set resultdict [punkcheck::install $layout_path/.fossil-settings $projectdir/.fossil-settings -createdir 1 -createempty 1 -punkcheck-folder $projectdir -installer project.new -exclude-paths-core $override_excludepaths_core -overwrite SYNCED-TARGETS]
set resultdict [punkcheck::install $layout_sourcepath/.fossil-settings $projectdir/.fossil-settings -createdir 1 -createempty 1 -punkcheck-folder $projectdir -installer project.new -exclude-paths-core $override_excludepaths_core -overwrite SYNCED-TARGETS]
puts stdout [punkcheck::summarize_install_resultdict $resultdict]
} else {
puts stdout "no .fossil-settings in source template - update not required"
@ -453,8 +479,8 @@ namespace eval punk::mix::commandset::project {
#scan all files in template
#
#TODO - deck command to substitute templates?
set templateinfo_list [punk::mix::commandset::layout::lib::layout_scan_for_template_files $opt_layout]
set stripprefix [file normalize $layout_path]
set templateinfo_list [punk::mix::commandset::layout::lib::layoutfolder_scan_for_template_files $layout_sourcepath]
set stripprefix [file normalize $layout_sourcepath]
set tagmap [list [lib::template_tag project] $projectname]
if {[llength $templateinfo_list]} {
@ -485,6 +511,91 @@ namespace eval punk::mix::commandset::project {
}
}
if {[dict get $materialized staged]} {
#layout copying and template scanning are done with the staging composition -
#remove it (the deterministic path under the temp dir is recomposed on demand)
catch {file delete -force $layout_sourcepath}
}
#G-087 stage 5: stamp the generated project's own identity/version metadata plus a
#generation provenance baseline. Written programmatically - layout payloads must never
#ship a punkproject.toml (a payload copy would make the store folder itself present as
#a candidate project root - punk::repo hazard, G-033). Only-if-missing so -update/-force
#reruns never clobber a project's own version history or original generation record.
set pp_file [file join $projectdir punkproject.toml]
if {![file exists $pp_file]} {
set fd [open $pp_file w]
fconfigure $fd -translation lf
puts $fd "\[project\]"
puts $fd "name = \"$projectname\""
puts $fd "version = \"0.1.0\""
close $fd
puts stdout "seeded punkproject.toml (\[project\] name '$projectname' version 0.1.0)"
}
set cl_file [file join $projectdir CHANGELOG.md]
if {![file exists $cl_file]} {
#seed the matching changelog header - without it the injected make.tcl's
#'projectversion' advisory check would warn in every generated project forever
set gendate [clock format [clock seconds] -format %Y-%m-%d]
set fd [open $cl_file w]
fconfigure $fd -translation lf
puts $fd "# Changelog"
puts $fd ""
puts $fd "All notable changes to the $projectname project version are documented here."
puts $fd "The latest `## \[X.Y.Z\]` header must match the `version` field in `punkproject.toml`."
puts $fd "Entries are newest-first; one bullet per notable change."
puts $fd ""
puts $fd "## \[0.1.0\] - $gendate"
puts $fd ""
puts $fd "- project generated from punkshell layout '$layout_name' (dev project.new)"
close $fd
puts stdout "seeded CHANGELOG.md (\[0.1.0\] - $gendate)"
}
set origin_file [file join $projectdir punkorigin.toml]
if {![file exists $origin_file]} {
#generation provenance baseline - INTERIM carrier pending the G-027 pull-update
#design (G-027's pull maintains a committed baseline of origin identity + pulled
#version; it should adopt or migrate this record - format marked for that purpose).
#Provenance deliberately does not live in punkproject.toml's [project] table
#(schema contested - see goals G-047/G-065/G-057).
set shellbase [file dirname [file dirname [file normalize [info nameofexecutable]/___]]]
set origin_projectroot [punk::repo::find_project $shellbase]
set origin_version ""
if {$origin_projectroot ne "" && [file exists [file join $origin_projectroot punkproject.toml]]} {
#punkshell project version is only derivable when the generating shell belongs
#to a checkout - bare kits carry no version stamp (G-025 proposed would add one)
if {![catch {package require punkboot::utils}]} {
catch {set origin_version [punkboot::utils::read_punkproject_version [file join $origin_projectroot punkproject.toml]]}
}
}
set templates_version [package provide punk::mix::templates]
set layout_target [file normalize [dict get $layout_info path]]
if {[dict exists $layout_info basefolder]} {
set storeroot [file normalize [dict get $layout_info basefolder]]
if {[string equal -length [string length $storeroot/] $storeroot/ $layout_target]} {
set layout_target [string range $layout_target [string length $storeroot/] end]
}
}
set genutc [clock format [clock seconds] -format %Y-%m-%dT%H:%M:%SZ -timezone :UTC]
set fd [open $origin_file w]
fconfigure $fd -translation lf
puts $fd "# punkorigin.toml - generation/pull provenance baseline for this project's"
puts $fd "# punkshell-derived infrastructure. Seeded by punkshell 'dev project.new'."
puts $fd "# INTERIM record format pending the punkshell G-027 pull-update design, which"
puts $fd "# maintains this baseline (origin identity + pulled version) on each pull."
puts $fd "format = 1"
puts $fd ""
puts $fd "\[generation\]"
puts $fd "utc = \"$genutc\""
puts $fd "layout = \"$layout_name\""
puts $fd "layout_target = \"$layout_target\""
puts $fd "punk_mix_templates_version = \"$templates_version\""
puts $fd "punkshell_version = \"$origin_version\""
puts $fd "origin_projectroot = \"$origin_projectroot\""
close $fd
puts stdout "seeded punkorigin.toml (generation provenance: layout '$layout_name', origin version '[expr {$origin_version ne "" ? $origin_version : "unknown"}]')"
}
::cd $projectdir
if {[file exists $projectdir/src/modules]} {
@ -503,10 +614,10 @@ namespace eval punk::mix::commandset::project {
#we should rarely if ever want to force any src/modules to be overwritten
if {$opt_force} {
if {$has_pod} {
set answer [util::askuser "OVERWRITE the src/modules file $podfile ?? (generally not desirable) Y|N"]
set answer [punk::mix::util::askuser "OVERWRITE the src/modules file $podfile ?? (generally not desirable) Y|N"]
set overwrite_type zip
} else {
set answer [util::askuser "OVERWRITE the src/modules file $tmfile ?? (generally not desirable) Y|N"]
set answer [punk::mix::util::askuser "OVERWRITE the src/modules file $tmfile ?? (generally not desirable) Y|N"]
set overwrite_type $opt_type
}
if {[string tolower $answer] eq "y"} {
@ -591,7 +702,7 @@ namespace eval punk::mix::commandset::project {
}
if {$first_fossil} {
#fossil commit may prompt user for input.. runx runout etc will pause with no prompts
util::do_in_path $projectdir {
punk::mix::util::do_in_path $projectdir {
set fossilcommit [run -n {*}$fossil_prog commit -m "initial project commit"]
}
if {[dict get $fossilcommit exitcode] != 0} {
@ -1064,7 +1175,7 @@ namespace eval punk::mix::commandset::project {
return [pwd]
}
} else {
set answer [util::askuser "Change directory to working folder - select a number from 1 to [llength $col_rowids] or any other key to cancel."]
set answer [punk::mix::util::askuser "Change directory to working folder - select a number from 1 to [llength $col_rowids] or any other key to cancel."]
if {[string trim $answer] in $col_rowids} {
set index [expr {$answer - 1}]
set workingdir [lindex $workdirs $index]

5
src/modules/punk/mix/commandset/project-buildversion.txt

@ -1,6 +1,9 @@
0.3.0
0.4.0
#First line must be a semantic version number
#all other lines are ignored.
#0.4.0 - G-087 stage 4: project.new copies the layout from punk::mix::commandset::layout::lib::layout_materialize's effective content folder - overlay chains (custom-over-vendor merging, .anti deletion markers, name@base derived layouts) are composed into a disposable staging dir (deleted after use); single-folder chains with no .anti markers keep the direct store-folder path (no behaviour change for punk.project / module-carried layouts). Template scanning uses the new path-based layoutfolder_scan_for_template_files against the effective folder. The src/PROJECT_LAYOUTS_* install exclusion was retired with the marker folder (G-087 stage 5) and the misnamed antipaths variable folded into exclude_paths.
#0.4.0 - G-087 stage 5: project.new seeds generated-project version metadata post-materialization (all only-if-missing, never from payload): punkproject.toml ([project] name + version 0.1.0), CHANGELOG.md with a matching ## [0.1.0] header (keeps the injected make.tcl projectversion advisory quiet), and punkorigin.toml - an interim generation-provenance baseline (utc, layout name + resolved store target, punk::mix::templates version, punkshell version when derivable from the generating shell's checkout via punkboot::utils, origin projectroot path hint) for the G-027 pull-update design to adopt.
#0.3.1 - fix: project.new validates module names (projectname-derived default and explicit -modules entries, modulename-version forms accepted) before any folder/fossil-repo creation - a dashed project name (e.g infra-man) previously failed late inside module.new after the project tree and fossil repo already existed
#0.3.0 - G-087 stage 3: new lib::bootsupport_inject (+ lib::_tm_version_sort) - project.new now injects bootsupport modules into the generated project from the generating shell's current trees, driven by the thin layout's include_modules.config manifests (shell-project bootsupport tree preferred; standalone kits resolve via tcl::tm::list; 999999.* dev versions never injected; punkcheck-recorded with plain-copy degradation)
#0.3.0 - bootsupport_inject also tree-copies the shell project's src/bootsupport/lib* pkgIndex-based lib trees (e.g fileutil - required by the generated make.tcl under plain tclsh); standalone kits emit a notice (lib set has no manifest) - generated make.tcl remains runnable via the kit executable
#0.3.0 - bootsupport_inject carries adjacent non-.tm support files beside matched modules (struct sets_*.tcl loader companions, textutil .tex hyphenation data) via new private helper _bootsupport_inject_copyfile; bootsupport include_modules.config gained the previously-missing cmdline and struct::set entries (files existed recordless in bootsupport; fileutil requires cmdline, make.tcl uses struct::set)

70
src/project_layouts/README.md

@ -83,28 +83,66 @@ module snapshot trees.
`packages`/`project`) keeps each layout's src/make.tcl, src/build.tcl and
bootsupport manifest copies current from the canonical src/ files, then
syncs the punk.project payload into the punk::mix::templates modpod source
tree (excluding bin/sdx.kit and the src/PROJECT_LAYOUTS_* marker). Records
live in src/project_layouts/.punkcheck. Never hand-edit the synced copies
or the modpod copy - edit the canonical files/store layout and rebuild.
tree (excluding bin/sdx.kit). Records live in
src/project_layouts/.punkcheck. Never hand-edit the synced copies or the
modpod copy - edit the canonical files/store layout and rebuild.
- A manifest edit (src/bootsupport/modules*/include_modules.config) reaches
the layouts and the modpod only on the next such build - `make.tcl
bootsupport` alone updates the checkout's own bootsupport, not the layouts.
- Alongside the layout copy and injection, `dev project.new` seeds the
generated project's own identity metadata programmatically (each
only-if-missing, never shipped inside a payload): punkproject.toml
(`[project]` name + starting version 0.1.0), a matching CHANGELOG.md
`## [0.1.0]` header, and punkorigin.toml - a generation provenance
baseline (layout name + resolved store target, punk::mix::templates
version, punkshell version when derivable) that is the interim carrier
for the G-027 pull-update design.
Overlay composition (G-087 stage 4)
-----------------------------------
A layout's effective content is composed from an ordered chain of store
folders by punk::mix::commandset::layout::lib::layout_materialize (used by
`dev project.new` and the layout listing/scanning commands):
<base chain - when the folder is named <name>@<base>>
vendor/<vendorname>/<name>
custom/<vendorname>/<name>
- Same-name files in a higher layer replace lower-layer files; folders merge
recursively.
- A marker named <item>.anti (zero-byte file recommended; a marker folder is
also accepted) deletes <item> as contributed by lower layers, and never
appears in the composed output. A marker paired with a same-name folder in
the same layer replaces the folder instead of merging.
- A folder named <newname>@<baselayoutname> is a derived layout: it presents
under its own name (via its own layout ref) and starts from the base
layout's full chain. See custom/punk/othersample@sample-0.1 (with its ref
in src/decktemplates/custom/punk/layout_refs/) for a working example of
custom-over-vendor override, a .anti deletion and derivation; semantics
detail in custom/punk/sample-0.1/extra.txt.
- Chains of one folder with no .anti markers (e.g the thin punk.project
layout, module-carried layouts) are used in place - no copying. Anything
else is composed into a disposable staging directory (deterministic
per-layout path under the system temp dir, marked with a #layout_staging
file) which project.new removes after use.
Store structure
---------------
src/project_layouts/{vendor,custom}/{<vendorname>,_project}/<layoutfolder>
src/project_layouts/{vendor,custom}/<vendorname>/<layoutfolder>
- vendor/ trees should not be customised directly; place overrides under
custom/. The pseudo-vendor _project holds project-local staging copies
(currently empty - the former custom/_project fat bootsupport-fragment
copies were retired by G-087 stage 3, injection having made them
redundant).
- Overlay semantics (custom-over-vendor merging, .anti deletion markers,
newname@base derived layouts, _project staging) are documented design -
see custom/punk/sample-0.1/extra.txt - but are NOT yet implemented:
project.new currently copies exactly one resolved layout folder.
Goal G-087 (goals/G-087-thin-project-layouts.md) tracks the remaining work:
the staging-directory overlay merge primitive (stage 4) and the
marker/vendor-matrix decisions (stage 5).
custom/ - the one override level.
- The former _project pseudo-vendor store level is retired for layouts
(G-087 stage 5 decision): its store content was deleted as redundant by
stage 3 (generation-time injection replaced the fat staging fragments),
and _project folders take no part in overlay chains. The _project concept
remains in use for decktemplates module/utility templates only.
- The unimplemented child-side src/PROJECT_LAYOUTS_CREATE_FROM_... marker
folder was likewise retired (G-087 stage 5): with thin layouts, generated
projects need only refs + thin payloads, and post-generation refresh
belongs to the G-027 pull design.
G-087 (thin layouts, fauxlink refs, overlay merge - achieved 2026-07-19) is
recorded in goals/archive/G-087-thin-project-layouts.md.

0
src/project_layouts/custom/punk/othersample@sample-0.1/.gitignore.anti

25
src/project_layouts/custom/punk/othersample@sample-0.1/README.md

@ -1,14 +1,23 @@
%project%
==============================
Another example of customisation of a file from a vendor layout
Due to the foldername othersample@sample-0.1, this layout presents under the name punk.othersample even though there is no 'othersample' template structure within vendor/punk
The original file exists at project_layouts/vendor/punk/sample-0.1/README.md
Due to this file's location at project_layouts/custom/punk/sample-0.1/README.md it will be used instead of the original.
see also extra.txt for notes regarding customisation of vendor layouts.
Example of a DERIVED layout.
Due to the folder name othersample@sample-0.1, this layout presents under its
own name (visible as punk.othersample via the layout ref in
src/decktemplates/custom/punk/layout_refs/) even though there is no
'othersample' folder within vendor/punk. The part after the @ names the base
layout: this layout's content starts from the full overlay chain of
sample-0.1 (vendor/punk/sample-0.1 merged with custom/punk/sample-0.1), with
this folder's own content applied on top.
This README overrides the one inherited from the sample-0.1 chain, and the
zero-byte .gitignore.anti marker beside it deletes the .gitignore inherited
from vendor/punk/sample-0.1 - so a project generated from punk.othersample
contains this README, extra.txt from custom/punk/sample-0.1, and no
.gitignore.
See custom/punk/sample-0.1/extra.txt for the full overlay/merge semantics.
+

44
src/project_layouts/custom/punk/sample-0.1/extra.txt

@ -1,11 +1,33 @@
example of an additional file added to a vendor layout
All files/folders from the vendor/layoutname tree are included unless they are over-ridden by a file of the same name in custom/vendor/layoutname or unless there is a file/folder of the same name except suffixed with .anti
For example, to completely replace src/modules in a vendor layout;
create an empty custom/vendor/layoutname/src/modules.anti folder
Then create a custom/vendor/layoutname/src/modules folder with your custom contents.
To just override a single module in vendor/layoutname/src/modules - just put a replacement module with the same name (or named with a later version) in custom/vendor/layoutname/src/modules.
All the existing modules from vendor/layoutname/src/modules will then be present in the resulting layout along with your modified/new module.
example of an additional file added to a vendor layout
Overlay/merge semantics (implemented by punk::mix::commandset::layout::lib::layout_materialize,
used by dev project.new - G-087 stage 4):
All files/folders from the vendor/<vendorname>/<layoutname> tree are included unless they are
over-ridden by a file of the same name in custom/<vendorname>/<layoutname> or deleted by a
marker of the same name suffixed with .anti. Same-name folders merge recursively; the layer
order is vendor then custom, so custom content wins.
.anti markers:
- A zero-byte FILE named <item>.anti is the recommended marker form (empty folders cannot be
committed to git/fossil). A marker FOLDER named <item>.anti is also accepted, matching the
originally documented form. Either deletes <item> - file or folder - as contributed by
lower layers, and the marker itself never appears in the composed layout.
- To completely replace src/modules from a vendor layout: put a zero-byte
custom/<vendorname>/<layoutname>/src/modules.anti marker file alongside a
custom/<vendorname>/<layoutname>/src/modules folder with your custom contents. The marker
is applied before the same layer's own content is copied.
- To just override a single module in vendor/<layoutname>/src/modules - put a replacement
module with the same name in custom/<vendorname>/<layoutname>/src/modules. All the existing
modules from the vendor layer will then be present in the resulting layout along with your
modified/new module.
Derived layouts:
- A folder named <newname>@<baselayoutname> presents as its own layout (given a layout ref
naming it). Its content starts from the base layout's full overlay chain with the derived
folder's own content (and .anti markers) applied on top. See ../othersample@sample-0.1 for
a working example (it also demonstrates a .anti deletion).
Composition happens in a disposable staging directory at generation time - the store folders
are never modified, and a layout whose chain is a single folder with no .anti markers is used
in place without any copying.

370
src/tests/modules/punk/mix/testsuites/layout/materialize.test

@ -0,0 +1,370 @@
package require tcltest
package require punk::mix::commandset::layout
#added 2026-07-19 (agent, G-087) - stage 4 overlay merge primitive: layout_resolve_chain /
#layout_stage_chain / layout_materialize in punk::mix::commandset::layout::lib, exercised
#against synthetic stores (no punk.templates capability context needed - layout_materialize
#is fed -layoutinfo directly). The repo fixtures (vendor/custom punk/sample-0.1 +
#othersample@sample-0.1) are exercised end-to-end via dev project.new - see the G-087 detail
#file verification records.
namespace eval ::testspace {
namespace import ::tcltest::*
variable BASE [makeDirectory g087_materialize]
proc fwrite {path content} {
file mkdir [file dirname $path]
set fd [open $path w]
fconfigure $fd -translation lf
puts -nonewline $fd $content
close $fd
}
proc fread {path} {
set fd [open $path r]
set data [read $fd]
close $fd
return $data
}
#relative sorted listing of all files (not dirs) below a folder - forward slashes
proc treefiles {folder} {
set files [list]
set queue [list ""]
while {[llength $queue]} {
set rel [lindex $queue 0]
set queue [lrange $queue 1 end]
set here [expr {$rel eq "" ? $folder : [file join $folder $rel]}]
foreach entry [lsort -unique [glob -nocomplain -dir $here -tails * .*]] {
if {$entry in {. ..}} {continue}
set relentry [expr {$rel eq "" ? $entry : "$rel/$entry"}]
if {[file isdirectory [file join $here $entry]]} {
lappend queue $relentry
} else {
lappend files $relentry
}
}
}
return [lsort $files]
}
#build a fresh synthetic store under $BASE/<name> and return its path
proc make_store {name} {
variable BASE
set store [file join $BASE $name]
file delete -force $store
file mkdir $store
return $store
}
variable common {
set result ""
}
# -- --- --- layout_resolve_chain --- --- --
test resolve_chain_vendor_only {single vendor folder yields single-entry chain}\
-setup $common -body {
set store [make_store s1]
file mkdir $store/vendor/punk/basic
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk basic]
lappend result [llength $chain]
lappend result [expr {[lindex $chain 0] eq [file join $store vendor punk basic]}]
}\
-cleanup {}\
-result [list 1 1]
test resolve_chain_custom_over_vendor {vendor tier precedes custom tier in chain order}\
-setup $common -body {
set store [make_store s2]
file mkdir $store/vendor/punk/sample $store/custom/punk/sample
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample]
lappend result [llength $chain]
lappend result [expr {[lindex $chain 0] eq [file join $store vendor punk sample]}]
lappend result [expr {[lindex $chain 1] eq [file join $store custom punk sample]}]
}\
-cleanup {}\
-result [list 2 1 1]
test resolve_chain_derived {name@base folder expands base chain first, derived folder last}\
-setup $common -body {
set store [make_store s3]
file mkdir $store/vendor/punk/sample $store/custom/punk/sample "$store/custom/punk/other@sample"
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk other@sample]
lappend result [llength $chain]
lappend result [expr {[lindex $chain 0] eq [file join $store vendor punk sample]}]
lappend result [expr {[lindex $chain 1] eq [file join $store custom punk sample]}]
lappend result [expr {[lindex $chain 2] eq [file join $store custom punk other@sample]}]
}\
-cleanup {}\
-result [list 3 1 1 1]
test resolve_chain_recursive_base {derivation chains recurse (a@b where b is b@c)}\
-setup $common -body {
set store [make_store s4]
file mkdir $store/vendor/punk/c "$store/custom/punk/b@c" "$store/custom/punk/a@b"
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk a@b]
lappend result [llength $chain]
lappend result [expr {[lindex $chain 0] eq [file join $store vendor punk c]}]
lappend result [expr {[lindex $chain 1] eq [file join $store custom punk b@c]}]
lappend result [expr {[lindex $chain 2] eq [file join $store custom punk a@b]}]
}\
-cleanup {}\
-result [list 3 1 1 1]
test resolve_chain_cycle {derivation cycle terminates with each folder listed once}\
-setup $common -body {
set store [make_store s5]
file mkdir "$store/custom/punk/a@b" "$store/custom/punk/b@a"
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk a@b]
#cycle guard stops the b->a expansion; both folders appear exactly once
lappend result [llength $chain]
lappend result [llength [lsort -unique $chain]]
}\
-cleanup {}\
-result [list 2 2]
test resolve_chain_missing {nonexistent layout name yields empty chain}\
-setup $common -body {
set store [make_store s6]
file mkdir $store/vendor/punk
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk nosuch]
llength $chain
}\
-cleanup {}\
-result 0
# -- --- --- layout_stage_chain --- --- --
test stage_override_and_add {custom same-name file wins, extra file added, folders merge}\
-setup $common -body {
set store [make_store s7]
fwrite $store/vendor/punk/sample/README.md "vendor-readme"
fwrite $store/vendor/punk/sample/src/modules/mod.txt "vendor-mod"
fwrite $store/custom/punk/sample/README.md "custom-readme"
fwrite $store/custom/punk/sample/extra.txt "extra"
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample]
set staging [file join $store staging]
set sres [punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging]
lappend result [fread $staging/README.md]
lappend result [fread $staging/extra.txt]
lappend result [fread $staging/src/modules/mod.txt]
lappend result [dict get $sres copied]
}\
-cleanup {}\
-result [list custom-readme extra vendor-mod 4]
test stage_anti_file_deletes_file {zero-byte <name>.anti file marker deletes lower-layer file}\
-setup $common -body {
set store [make_store s8]
fwrite $store/vendor/punk/sample/.gitignore "ignore-content"
fwrite $store/vendor/punk/sample/keep.txt "keep"
file mkdir $store/custom/punk/sample
fwrite $store/custom/punk/sample/.gitignore.anti ""
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample]
set staging [file join $store staging]
set sres [punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging]
lappend result [file exists $staging/.gitignore]
lappend result [file exists $staging/keep.txt]
lappend result [dict get $sres deletions]
}\
-cleanup {}\
-result [list 0 1 .gitignore]
test stage_anti_markers_never_copied {no .anti entries appear in the composed staging}\
-setup $common -body {
set store [make_store s9]
fwrite $store/vendor/punk/sample/a.txt "a"
fwrite $store/custom/punk/sample/a.txt.anti ""
fwrite $store/custom/punk/sample/sub/b.txt.anti ""
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample]
set staging [file join $store staging]
punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging
set antis [list]
foreach f [treefiles $staging] {
if {[string match *.anti $f]} {lappend antis $f}
}
lappend result $antis
lappend result [file exists $staging/a.txt]
}\
-cleanup {}\
-result [list {} 0]
test stage_anti_folder_replace {folder.anti marker plus replacement folder in same layer fully replaces}\
-setup $common -body {
set store [make_store s10]
fwrite $store/vendor/punk/sample/src/modules/old1.txt "old1"
fwrite $store/vendor/punk/sample/src/modules/old2.txt "old2"
#marker as zero-byte FILE (recommended - committable), replacement folder alongside
fwrite $store/custom/punk/sample/src/modules.anti ""
fwrite $store/custom/punk/sample/src/modules/new.txt "new"
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample]
set staging [file join $store staging]
punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging
lappend result [treefiles $staging/src/modules]
}\
-cleanup {}\
-result [list new.txt]
test stage_anti_dir_marker_form {a directory named <name>.anti also acts as a deletion marker}\
-setup $common -body {
set store [make_store s11]
fwrite $store/vendor/punk/sample/src/modules/old.txt "old"
file mkdir $store/custom/punk/sample/src/modules.anti
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample]
set staging [file join $store staging]
punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging
lappend result [file exists $staging/src/modules]
lappend result [file exists $staging/src/modules.anti]
}\
-cleanup {}\
-result [list 0 0]
test stage_type_conflicts {higher-layer entry wins on file/folder type conflicts}\
-setup $common -body {
set store [make_store s12]
fwrite $store/vendor/punk/sample/thing "vendor-file"
fwrite $store/vendor/punk/sample/other/nested.txt "vendor-nested"
fwrite $store/custom/punk/sample/thing/inner.txt "custom-inner"
fwrite $store/custom/punk/sample/other "custom-file"
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample]
set staging [file join $store staging]
punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging
lappend result [file isdirectory $staging/thing]
lappend result [fread $staging/thing/inner.txt]
lappend result [expr {![file isdirectory $staging/other]}]
lappend result [fread $staging/other]
}\
-cleanup {}\
-result [list 1 custom-inner 1 custom-file]
test stage_empty_dir_preserved {empty folders in a layer are created in staging}\
-setup $common -body {
set store [make_store s13]
file mkdir $store/vendor/punk/sample/emptydir
fwrite $store/vendor/punk/sample/a.txt "a"
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample]
set staging [file join $store staging]
punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging
file isdirectory $staging/emptydir
}\
-cleanup {}\
-result 1
test stage_refuse_clobber {existing non-empty dir without staging marker is refused}\
-setup $common -body {
set store [make_store s14]
fwrite $store/vendor/punk/sample/a.txt "a"
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample]
set staging [file join $store precious]
fwrite $staging/important.txt "do-not-delete"
set code [catch {punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging} emsg]
lappend result $code
lappend result [file exists $staging/important.txt]
lappend result [string match "*refusing to delete*" $emsg]
}\
-cleanup {}\
-result [list 1 1 1]
test stage_restage_over_marker {a previous staging composition (marker present) is replaced}\
-setup $common -body {
set store [make_store s15]
fwrite $store/vendor/punk/sample/a.txt "one"
set chain [punk::mix::commandset::layout::lib::layout_resolve_chain $store punk sample]
set staging [file join $store staging]
punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging
fwrite $store/vendor/punk/sample/b.txt "two"
set code [catch {punk::mix::commandset::layout::lib::layout_stage_chain $chain $staging} sres]
lappend result $code
lappend result [file exists $staging/a.txt]
lappend result [file exists $staging/b.txt]
}\
-cleanup {}\
-result [list 0 1 1]
# -- --- --- layout_materialize --- --- --
test materialize_fastpath {single-folder chain with no .anti markers returns the store folder unstaged}\
-setup $common -body {
set store [make_store s16]
fwrite $store/vendor/punk/solo/a.txt "a"
set linfo [list path [file join $store vendor punk solo] basefolder $store sourceinfo {}]
set mat [punk::mix::commandset::layout::lib::layout_materialize punk.solo -layoutinfo $linfo]
lappend result [dict get $mat staged]
lappend result [expr {[dict get $mat path] eq [file join $store vendor punk solo]}]
lappend result [llength [dict get $mat chain]]
}\
-cleanup {}\
-result [list 0 1 1]
test materialize_single_with_antis {a single folder containing .anti markers is staged with markers stripped}\
-setup $common -body {
set store [make_store s17]
fwrite $store/vendor/punk/solo/a.txt "a"
fwrite $store/vendor/punk/solo/ghost.txt.anti ""
set linfo [list path [file join $store vendor punk solo] basefolder $store sourceinfo {}]
set staging [file join $store staging]
set mat [punk::mix::commandset::layout::lib::layout_materialize punk.solo -layoutinfo $linfo -stagingdir $staging]
lappend result [dict get $mat staged]
lappend result [file exists [dict get $mat path]/a.txt]
lappend result [file exists [dict get $mat path]/ghost.txt.anti]
}\
-cleanup {}\
-result [list 1 1 0]
test materialize_opaque_path {a layout folder outside the tier/vendor store shape stays an opaque single-folder chain}\
-setup $common -body {
set store [make_store s18]
fwrite $store/somewhere/deep/layoutx/a.txt "a"
set linfo [list path [file join $store somewhere deep layoutx] basefolder $store sourceinfo {}]
set mat [punk::mix::commandset::layout::lib::layout_materialize punk.layoutx -layoutinfo $linfo]
lappend result [dict get $mat staged]
lappend result [llength [dict get $mat chain]]
lappend result [expr {[dict get $mat path] eq [file join $store somewhere deep layoutx]}]
}\
-cleanup {}\
-result [list 0 1 1]
test materialize_project_pseudovendor_opaque {_project pseudo-vendor folders take no part in overlay chains}\
-setup $common -body {
set store [make_store s19]
fwrite $store/vendor/_project/sample/a.txt "proj-a"
fwrite $store/custom/_project/sample/a.txt "proj-custom-a"
set linfo [list path [file join $store vendor _project sample] basefolder $store sourceinfo {}]
set mat [punk::mix::commandset::layout::lib::layout_materialize sample -layoutinfo $linfo]
lappend result [dict get $mat staged]
lappend result [llength [dict get $mat chain]]
}\
-cleanup {}\
-result [list 0 1]
test materialize_derived_full {derived layout materializes base chain content with overrides and .anti deletions}\
-setup $common -body {
set store [make_store s20]
fwrite $store/vendor/punk/sample/README.md "vendor-readme"
fwrite $store/vendor/punk/sample/.gitignore "gitignore"
fwrite $store/vendor/punk/sample/src/modules/README.md "modules-readme"
fwrite $store/custom/punk/sample/README.md "custom-readme"
fwrite $store/custom/punk/sample/extra.txt "extra"
fwrite "$store/custom/punk/other@sample/README.md" "derived-readme"
fwrite "$store/custom/punk/other@sample/.gitignore.anti" ""
set linfo [list path [file join $store custom punk other@sample] basefolder $store sourceinfo {}]
set staging [file join $store staging]
set mat [punk::mix::commandset::layout::lib::layout_materialize punk.other -layoutinfo $linfo -stagingdir $staging]
set effective [dict get $mat path]
lappend result [dict get $mat staged]
lappend result [llength [dict get $mat chain]]
lappend result [fread $effective/README.md]
lappend result [fread $effective/extra.txt]
lappend result [fread $effective/src/modules/README.md]
lappend result [file exists $effective/.gitignore]
lappend result [dict get $mat deletions]
}\
-cleanup {}\
-result [list 1 3 derived-readme extra modules-readme 0 .gitignore]
variable BASE
file delete -force $BASE
cleanupTests
}
namespace delete ::testspace
Loading…
Cancel
Save