# G-087 Thin project layouts: injected infrastructure, fauxlink layout refs, working overlay merge

Status: active
Scope: src/project_layouts/ (payload store restructure), src/decktemplates/ (layout_refs migration to fauxlink), src/modules/punk/mix/commandset/project-999999.0a1.0.tm (project.new generation path), src/modules/punk/mix/commandset/layout-999999.0a1.0.tm, src/modules/punk/cap/handlers/templates-999999.0a1.0.tm (ref resolution), src/modules/punk/mix/#modpod-templates-999999.0a1.0/ (module-shipped refs/layouts), src/make.tcl (layout sync steps)
Goal: project.new generates projects from thin layouts (structure, scripts, docs - no stored bootsupport/module/vfs snapshot trees) whose heavy infrastructure is injected at generation time from the generating shell's own current trees; layout references are fauxlink files (one ref concept repo-wide, no bespoke .ref grammar, no real symlinks); and the documented overlay semantics (custom-over-vendor, .anti deletions, name@base derived layouts, _project staging) are implemented by a single staging-directory materialization primitive - so a bare built kit can generate a current project, and the committed multi-megabyte stale layout snapshots are retired.
Acceptance: hygiene stage - in a punkshell checkout `dev project.new` with defaulted -layout resolves an exact layout name (no tcl::prefix ambiguity); the no-handler warning paths execute without error (put/puts typos fixed) and an absolute-pathtype provider registration does not raise an expr syntax error; test/junk refs (test2, sample-0.1 duplicate, the a.txt-only punk.shell-0.1) no longer present as selectable layouts; src/project_layouts/README.md describes the implemented punk.templates+refs mechanism including the ref filename grammar. Pivot stage - a project generated from the thin punk.project layout carries bootsupport modules matching the generating shell's current src/bootsupport (verified by comparing a marker module version such as punk::mix::cli, not a stored snapshot vintage); src/project_layouts contains no embedded bootsupport/vfs module trees and make.tcl's layout bootsupport-sync step is retired; layout refs are .fauxlink files resolved via the fauxlink module with the bespoke .ref parser removed, every layout ref carrying a non-empty nominal name (leading-# punkcheck-exclusion hazard documented in this file); a bare built kit run outside any project lists at least the module-shipped punk.project layout and generates a working project from it; and one overlay case exercising custom-over-vendor override, a .anti deletion, and a name@base derived layout (the existing sample-0.1/othersample fixtures) materializes correctly through the staging primitive into a generated project.

## Context

Review of the layout system (2026-07-16) found a coherent three-layer design whose
centerpiece was never implemented:

1. **punk.templates capability** (punk::mix::templates declarations resolved by
   punk::cap::handlers::templates) locates template base folders in priority order
   (module < shellproject < currentproject < absolute < adhoc).
2. **Layout references** - zero-byte `.ref` files under `decktemplates/*/layout_refs/`
   whose filename encodes alias + path into `<projectroot>/src/project_layouts`
   (`alias@vendor+punk+project-0.1.ref`). This indirection is the recursion firewall:
   modules ship only refs; layout payloads (which embed module trees, including
   punk::mix::templates itself) live once, outside any module.
3. **Payload store** `src/project_layouts/{vendor,custom}/{<vendor>,_project}/` with
   overlay semantics documented in `custom/punk/sample-0.1/extra.txt` (custom over
   vendor, `.anti` deletion markers, `newname@base` derived layouts, `_project`
   staging pseudo-vendor).

Unimplemented/broken as found:

- **No merging at all** - project.new copies exactly one resolved directory
  (punkcheck::install); `.anti`, custom overlays, and `name@base` folders
  (e.g. `custom/punk/othersample@sample-0.1`) are documented but inert.
- **module-pathtype refs can never resolve** - their `base` is the module directory,
  and `get_itemdict_projectlayouts` looks for `<base>/src/project_layouts`, which
  cannot exist inside a module. Consequence: the only source of the exact name
  `punk.project` is dead, and a bare kit outside any project has an empty layout list
  (project.new unusable standalone).
- **Default -layout is prefix-ambiguous** in-checkout: no exact `punk.project` exists;
  `tcl::prefix::match` sees `punk.project-0.1` and `punk.project-0.1_overrides`.
- **Stale fat snapshots**: `vendor/punk/project-0.1` is 559 files / ~19MB frozen at the
  punk::mix::cli 0.3/0.3.1 era (both versions side by side); `custom/_project/*`
  duplicates ~23MB more. make.tcl refreshes only the `custom/_project` copies
  (bootsupport + make/build sync) - treating drift symptomatically.
- **Names lie**: `@vendor+_project+punk.shell-0.1.ref` presents as `punk.shell-0.1`
  but targets a folder containing a single junk `a.txt`; the real 12MB shell layout is
  exposed only as `punk.shell-0.1_overrides`. Test refs (`test2`, `sample-0.1`) show as
  selectable layouts.
- **Child materialization missing**: the vendored layout carries a marker folder
  `src/PROJECT_LAYOUTS_CREATE_FROM_DECKTEMPLATES_LAYOUT_REFERENCES/` (and project.new
  excludes `src/PROJECT_LAYOUTS_*`); the step it names was never written, so generated
  projects hold refs pointing at a folder that never exists.
- **Design-generation residue**: README describes a `punk.projectlayout` capability
  that exists nowhere in code (punk.templates + refs superseded it); the modpod's
  `templates/layouts/project` subtree has no reader; make.tcl still carries
  `old_layout_update_list` referencing `src/mixtemplates`.
- **Latent bugs**: `put stderr` (missing s) in no-handler warning paths
  (commandset/layout.tm ~lines 208/219, commandset/project.tm ~line 255); non-braced
  `if {!file exists $normpath}` in the templates handler's `absolute` branch
  (~line 175) errors on any absolute-pathtype registration.

The pivot insight: the recursion problem and the snapshot weight have the same cause -
layouts *store* module trees. Stop storing them (inject at generation time from the
generating shell, which always has current bootsupport) and: layouts become small
enough to ship inside the punk::mix::templates module without recursion (a bare kit
becomes self-sufficient); ~40MB of committed snapshots leaves the repo; overlay merging
becomes cheap; and generated projects start current instead of at snapshot vintage,
with later refresh belonging to G-027's pull.

## Approach

Staged; hygiene first since it is independently shippable.

1. **Hygiene** (no design changes): fix the `put`/`!file exists` bugs; restore an exact
   default layout name (rename `@vendor+punk+project-0.1.ref` to
   `project@vendor+punk+project-0.1.ref`, yielding exactly `punk.project`); quarantine
   or ignore-prefix the test/junk refs; rewrite `src/project_layouts/README.md` to
   describe the implemented system and the ref filename grammar; remove the dead modpod
   `templates/layouts` subtree and make.tcl `old_layout_update_list` residue.
2. **Fauxlink refs** (user decision 2026-07-16: one ref concept repo-wide, no real
   symlinks): migrate `layout_refs/*.ref` to `.fauxlink` files resolved by the vendored
   `fauxlink` module - `alias@vendor+punk+project-0.1.ref` becomes
   `alias#vendor+punk+project-0.1.fauxlink` (same `+` path encoding; `@` in target
   segments like `othersample@sample-0.1` is legal literal fauxlink content). Target
   paths stay application-resolved against `<projectroot>/src/project_layouts` (fauxlink
   is app-driven by design). Zero-byte property preserved (target fully in the name).
   Clean cutover preferred over dual-grammar support - all refs are in-repo.
   Constraint: layout refs MUST carry a non-empty nominal name - an alias-less ref
   (`#vendor+...fauxlink`) starts with `#`, which this project's punkcheck core
   excludes (`#*`, `**/#*`) treat as aside/hidden, so it would silently vanish from
   punkcheck-driven installs. Opportunities: the fauxlink tags segment can carry layout
   metadata (e.g. staging markers); the reserved TOML body can carry per-ref update
   policy (the `allowupdates`/`repo`/`reposubdir` keys currently parked in the
   punk::mix::templates declarations) - feeding G-027 provenance.
3. **Thin layouts + injection**: strip stored bootsupport/vfs/module trees from layout
   payloads; project.new copies the thin layout then injects bootsupport from the
   generating shell's own current trees (same mechanism make.tcl already uses to sync
   `custom/_project` layouts, relocated to generation time). Retire make.tcl's
   layout bootsupport-sync step and the fat `custom/_project` copies. Ship the thin
   `punk.project` layout (plus its ref) inside the punk::mix::templates module so
   module-pathtype refs resolve to module-carried payloads and a bare kit works.
4. **Overlay merge primitive**: a `layout_materialize` lib proc resolving a layout name
   to an ordered chain of source dirs (base from `name@base` folder naming if present,
   then vendor/<v>/<name>, then custom/<v>/<name>, then `_project` overlays), composed
   into a staging directory applying `.anti` deletions; project.new keeps its
   single-path contract and receives the staged path. Brings every documented-but-dead
   concept to life at once.
5. **Retirement/decision**: drop or implement the child-side
   `PROJECT_LAYOUTS_CREATE_FROM_...` marker (with thin layouts, children need only
   refs + thin payloads; heavy parts arrive by the same injection and refresh via
   G-027); decide whether the vendor/custom x `_project` matrix at both levels still
   earns its complexity once overlays are cheap (vendor/_project and custom/_project
   may collapse into one override level).

## Alternatives considered

- Real filesystem symlinks for refs - rejected: platform-finicky (Windows privilege
  and VCS behaviour), and fauxlink exists precisely to avoid them (user preference,
  2026-07-16).
- Keeping the bespoke `.ref` grammar - rejected: it is a hand-rolled subset of
  fauxlink's naming scheme; two ref concepts in one repo with near-identical encodings
  invites drift.
- Teaching punkcheck::install multi-source layered installs - rejected in favour of
  staging-directory composition: simpler, independently testable, preserves
  project.new's single-path contract.
- Keeping fat vendor snapshots and fixing drift via pull updates alone - rejected as
  end state: the snapshots would still be redundant third/fourth copies of bootsupport
  content, still 40MB in-repo, and the recursion constraint on module-shipped layouts
  would remain.

## Notes

- fauxlink upstream source is directly accessible at C:/repo/jn/tclmodules/fauxlink
  (fossil checkout, punkshell-style layout; user confirmed 2026-07-17): fauxlink tests
  or module changes needed for the stage-2 ref migration can be made upstream there and
  re-vendored, rather than patching the vendored copy.
- fauxlink 0.1.2 upstream work (2026-07-17, user-directed): complete test suite
  (src/tests/fauxlink.test, 62 tests), punk::args documentation blocks (inert PUNKARGS +
  registration, no punk::args dependency) including a (package)fauxlink id - the id
  registers and resolves once its namespace is scanned; automatic id-derived scan and
  user-facing rendering await G-075. Fixes: decode_unicode_escapes first-char drop on
  hexless %U, 3-segment comment validation/decoding parity, link_as stub errors instead
  of silent no-op. link_as (the encoder counterpart of resolve) remains unimplemented -
  stage 2 migration tooling will want it (candidate follow-on).
- punkshell fauxlink consumer survey (2026-07-17): punk::mix::util fcat follows
  .fauxlink (and .lnk via punk::winlnk) unless -noredirect - note fcat matches only the
  .fauxlink extension, not .fxlnk; make.tcl merge_over does fauxlink-aware recursive
  copy (follows 'punk::boot,merge_over'-tagged links, copies others as files) and the
  startup-script check resolves root main.tcl fauxlinks; src/scriptapps/punk.tcl kit
  boot fallback follows a main.tcl#*.fauxlink/.fxlnk with the same tag; vendored via
  src/vendormodules/include_modules.config from c:/repo/jn/tclmodules/fauxlink/modules.
- Intention (user, 2026-07-17): punk::nav::fs should gain sensible fauxlink handling in
  future - analogous to its optional punk::winlnk treatment of windows .lnk shortcuts
  (resolve for target/type classification and display) - so fauxlinks present usefully
  in directory listings/navigation. Candidate goal when picked up.
- Related goals: G-086 (light project layout - becomes just another thin layout with
  less injected; this goal generalizes its premise to all layouts), G-027
  (derived-project pull updates - takes over post-generation refresh; fauxlink TOML
  body is a candidate carrier for its provenance/update-policy records), G-012
  (template payload safety - inert VCS-config payload rules apply to thin layouts
  unchanged), G-004 (no committed binaries - snapshot retirement aligns), G-026
  (vendor provenance policy - source-side cleanliness for injection).
- Naming rules to enforce with the merge primitive: visible name = vendor.name derived
  from the ref; report colliding visible names in `dev project.layouts`; versionless
  request picks highest version (existing todo in commandset/project.tm); default
  -layout must be an exact name.
- The `#n` duplicate-suffix mechanism in `_get_itemdict` and the priority ordering
  (module < shellproject < currentproject < absolute < adhoc) are sound and carry over
  unchanged.

## Progress

### 2026-07-17 hygiene stage complete (increment 1)

Landed (punk::mix::commandset::layout 0.1.1, project 0.2.2, module 0.1.1,
scriptwrap 0.1.1; punk::mix::base 0.1.1; punk::cap::handlers::templates 0.1.1;
punk::mix::templates 0.1.4; punkshell 0.12.38):

- put -> puts fixed at all six no-handler warning sites. The closeout sweep found
  three sites beyond the review's list (mix/base get_template_basefolders,
  commandset::module templates_dict, commandset::scriptwrap templates_dict). Three
  procs also dereferenced an unset result variable after the warning - the
  no-handler paths now warn and return an empty dict; layout/module warning
  messages no longer misattribute their owning proc.
- templates handler: the non-braced 'if {!file exists $normpath}' expr bug fixed.
  Verification exposed an adjacent latent bug, also fixed: pkg_register's
  duplicate-registration guard did 'dict get $provider_info $pkg' without a
  dict-exists check, erroring for any second provider package registering the
  same capability.
- @vendor+punk+project-0.1.ref renamed to project@vendor+punk+project-0.1.ref:
  exact layout name 'punk.project' now exists and the defaulted -layout
  prefix-resolves unambiguously.
- Junk refs deleted: test2, the custom/_project duplicate sample-0.1 ref, and the
  alias-less punk.shell-0.1 ref together with its a.txt-only target folder
  (project_layouts/vendor/_project/punk.shell-0.1).
- src/project_layouts/README.md rewritten: implemented punk.templates + layout_refs
  mechanism, ref filename grammar, unimplemented-overlay status, G-087 pointer;
  dead punk.projectlayout capability description removed.
- Dead modpod templates/layouts/project payload removed (punk::mix::templates
  0.1.4); make.tcl old_layout_update_list and stale mixtemplates help text removed.

Verification (2026-07-17, native tclsh 8.7/9 runtests + rebuilt root modules):

- Scratch script (g087_hygiene_verify.tcl) 13/13: exact punk.project present and
  tcl::prefix::match resolves it; test2 / bare sample-0.1 / punk.shell-0.1 absent
  from layouts_dict; absolute-pathtype registration returns 1 for an existing path
  and warn+0 for a missing one; with the handler entry blanked, all five
  no-handler paths warn and return empty without error.
- runtests: prune 19/19, fossilmove 10/10, multishell 9 pass + 1 skip,
  libsearch 8/8. make.tcl modules builds clean; rebuilt templates modpod 0.1.4
  mounts and its module-shipped ref lists as punk.project#2 (module-pathtype
  resolution still dead pending stage 3).
- Resulting layout names: punk.sample-0.1 punk.project punk.minimal punk.basic
  punk.shell-0.1_overrides punk.project-0.1_overrides.

### 2026-07-17 stage 2 complete: fauxlink layout refs (increment 2)

Landed (punk::cap::handlers::templates 0.2.0, punk::mix::templates 0.1.5;
fauxlink 0.2.0 vendored from upstream):

- fauxlink::link_as implemented upstream (fauxlink 0.2.0, checkin 1e6b4ee8):
  name + forward-slash target (+ optional -tags/-comments/-extension) -> fauxlink
  filename with a resolve round-trip guarantee. 83-test upstream suite.
- Handler cutover: get_itemdict_projectlayoutrefs selects *#*.fauxlink/.fxlnk
  and derives names via fauxlink::resolve; get_itemdict_projectlayouts derives
  the layout folder from the resolved targetpath; the bespoke <alias>@<target>.ref
  parser is gone and .ref files are no longer recognised. Alias-less (leading-#)
  refs and unresolvable refs are skipped with a warning (punkcheck hazard per
  Approach stage 2). Bonus fix: the ignore* skip previously matched against the
  full glob path so it never applied.
- All 7 live-tier refs migrated via a link_as-driven script (each new name
  round-trip-verified before rename): decktemplates vendor/punk x4 (alias-less
  refs gained their target-tail aliases: basic, minimal, sample-0.1),
  custom/_project x2, module-shipped modpod ref x1. Zero-byte property kept.
- src/project_layouts/README.md ref-grammar section rewritten for fauxlink.

Verification (2026-07-17): stage-2 scratch script 7/7 - layout names identical
to the pre-migration set with exact punk.project resolution; refs dict paths all
.fauxlink; planted alias-less decoy warned + skipped; planted old-grammar .ref
decoy ignored; module-shipped ref lists as punk.project#2 (resolution still dead
pending stage 3). runtests prune/fossilmove/multishell/libsearch all pass.
End-to-end `dev project.new <abs-path> -confirm 0` (punk902z src, piped,
FOSSIL_HOME diverted) resolved the default punk.project layout through fauxlink
refs and materialized the full project tree; the run's trailing in-project
fossil commit failed on a binary-data prompt (layout payload ships no
binary-glob - G-012 territory, unrelated to refs).

Notes from the work:

- Stale-snapshot payload copies of the old .ref grammar remain under
  src/project_layouts/*/src/bootsupport/... and the fat vendor snapshot's inner
  trees - stale-by-design, retired with the snapshots in stage 3. Generated
  projects' decktemplates payloads currently carry no layout_refs at all.
- project.new from a cwd outside any project has an empty layout list even with
  a shell built from a project: the punk::mix::templates shellproject_multivendor
  declaration points at <shellproject>/decktemplates (repo root - nonexistent),
  not src/decktemplates. Pre-existing; natural stage-3 fix alongside module-ref
  resolution.
- punkcheck::install source reads can exceed Windows MAX_PATH when the relative
  source chain is long (deep target dir + many ../ segments, unnormalized path);
  surfaced as 'could not read ... no such file' during a deep-scratchpad
  generation. Pre-existing, candidate for a punkcheck normalization fix.
- make.tcl bootsupport prune of punk/mix/templates-<v>.tm fails with permission
  denied when the running make.tcl process itself has that modpod mounted
  (self-lock); the stale copy must be deleted manually afterwards. Candidate
  small fix: unmount or defer-prune modpods the process mounted.

Remaining for acceptance (pivot stage - Approach stages 3-5):

- thin layouts + generation-time bootsupport injection, retire make.tcl layout
  bootsupport-sync, module-shipped thin punk.project layout for bare kits (stage 3).
- staging-directory overlay merge primitive exercising the sample-0.1/othersample
  fixtures (stage 4).
- child-side PROJECT_LAYOUTS_CREATE_FROM_... marker retirement and the
  vendor/custom x _project matrix decision (stage 5).