punkshell/goals/archive/G-012-template-payload-safety.md

G-012 Template system: inert VCS-config payloads and explicit layout refresh

Status: achieved 2026-08-01 Scope: src/project_layouts/; src/make.tcl; src/modules/punk/mix/ (layout instantiation); fauxlink module (bootsupport 0.2.0 - promoted if chosen as mechanism) Goal: project layouts carry no live nested VCS-config files - template .gitignore payloads are stored inert (renamed, or fauxlink-encoded) and materialized at project generation - and src/make.tcl has an explicit punkcheck-tracked step that refreshes layout payloads from their canonical sources. Acceptance: a scan of src/project_layouts finds no file named .gitignore, and git check-ignore --no-index over every layout file matches only root-.gitignore rules (nested rules provably inert); a project generated from each affected layout receives a working .gitignore whose content matches the canonical payload/target; after editing a canonical source (e.g. root .gitignore), the make.tcl template-refresh step updates the derived layout payloads (punkcheck-tracked), covering vendor/punk layouts as well as custom/_project; the previously hidden template files (layout READMEs, vendored TODO-class files) remain git-tracked without per-file force-add exceptions.

Context

Project layouts under src/project_layouts/ carry .gitignore files as payload - the ignore file a generated project is supposed to receive (essentially a copy of shellspy's own root .gitignore). But git has no concept of payload: any .gitignore anywhere in the working tree is live configuration for its subtree, with leading-/ patterns anchored to the nested file's own directory. The result is a template that self-censors - the layout ignores, in the shellspy repo, exactly what a generated project would ignore in itself.

Discovered 2026-07-06 during git/fossil tracked-set verification (fossil reads only the checkout-root .fossil-settings/ignore-glob, so it managed template files git silently dropped): four vendor/punk/* layouts each carry a live .gitignore, and .../project-0.1/{bin,lib,modules}/README.md plus bin/runtime/README.md were untracked in git. The bin/runtime case shows the nastier variant - git does not descend into an excluded directory, so negation patterns cannot resurrect deeper paths. The immediate stopgap was per-file git add -f (commit fc1c474c), which protects only existing files: any new file placed in a template's ignored spots drops silently again. The dual-VCS contract (.fossil-settings/AGENTS.md) records the git/fossil asymmetry and points here for the fix.

A secondary weakness: template payloads that mirror living files (the root .gitignore, build.tcl/make.tcl, bootsupport modules) have only partial refresh machinery. make.tcl's sync_layouts step (punkcheck-tracked) syncs just build.tcl and make.tcl, and only into the custom/_project layout base - the vendor/punk/* layouts have no refresh mechanism, so payload drift is unmanaged.

Approach

Mechanism decision (recorded 2026-08-01, implementation start): neutral rename. Payloads are stored under an inert name (gitignore.in) and materialized to their live name (.gitignore) at generation. Chosen over fauxlink because: (a) fauxlink's model in this codebase is zero-byte filename-as-pointer (layout refs) - real authored payload content does not fit it; (b) a fauxlink target flattens every layout's payload to one canonical file, removing the possibility of per-layout canonical payloads, and pointing at the out-of-store repo-root .gitignore would need a new resolution convention; (c) rename is dependency-free and the layout system already has a single materialization funnel (layout_materialize) where a small explicit rename map (extensible to future live-config payloads such as .gitattributes) hooks in cleanly. The four vendor/punk .gitignore payloads are treated as drifted mirrors of the canonical root .gitignore (the goal context's "essentially a copy") - the make.tcl template-refresh step rederives them from root, so the vintage variants are eliminated rather than preserved.

1. Store VCS-config payloads inert. Two candidate mechanisms; the choice is an implementation decision to be recorded here when made:
   • Neutral rename: payload stored as e.g. _gitignore or gitignore.in; layout instantiation renames it to .gitignore in the generated project. Simple, no new dependency; templates keep a literal copy (drift handled by the refresh step).
   • Fauxlink: payload stored as <nominal>#<encoded-target>.fauxlink (see fauxlink module, currently bootsupport 0.2.0) and resolved at generation. Inert for git by construction, and the target can be the canonical root .gitignore itself - eliminating copy drift rather than managing it. Requires generation-time resolution in punk::mix and possibly promoting fauxlink to a first-class module dependency.
2. Materialize at generation. The punk::mix layout-instantiation path produces a real .gitignore in the generated project regardless of mechanism, with content equal to the canonical payload/target.
3. Explicit template refresh in make.tcl. Generalise the existing sync_layouts step into a defined template-refresh covering the vendor/punk/* layout base as well as custom/_project, and derived payloads beyond build.tcl/make.tcl (the gitignore payload, and prospectively other mirrored content). punkcheck-tracked like the existing step, so refreshes are recorded and skippable when sources are unchanged.
4. Retire the stopgap. With inert payloads, the per-file force-adds from fc1c474c become ordinary tracked files (no ignore rule matches them); the accidental-ignore class is structurally impossible rather than patched per file.

Alternatives considered

• Negation entries inside the nested template .gitignore files (e.g. !README.md) - rejected: pollutes the payload that generated projects receive, and cannot fix the excluded-directory case (bin/runtime/) because git does not descend into excluded dirs to evaluate negations.
• Keep per-file git add -f only - rejected as the durable answer: silent drop recurs for every new file placed under a template's ignored paths; nothing structural changes.
• .git/info/exclude or repo-local config tricks - rejected: not versioned/portable to other clones and contributors.
• Do nothing on the git side (fossil sees the files) - rejected: git is the day-to-day VCS; a mirror having the truer view is a symptom, not a solution.

Progress

Landed 2026-08-01 (activated and implemented in one session at user request):

• Inert storage - the four vendor/punk .gitignore payloads renamed to gitignore.in; the custom/punk/othersample@sample-0.1/.gitignore.anti marker renamed to gitignore.in.anti (.anti markers target store names). No file named .gitignore remains under src/project_layouts or in the modpod payload source folder.
• Materialization - punk::mix::commandset::layout 0.3.0: new lib::layout_materialize_renames map ({gitignore.in .gitignore}) with private helpers _find_inert_payload_relpaths/_apply_materialize_renames; layout_stage_chain renames inert payloads to live names after composing all layers (new materialized return key); layout_materialize withholds the in-place fast path from inert-payload folders so the rename only ever happens in staging. Both store-name and materialized-name presence at one location errors. Characterization: src/tests/modules/punk/mix/testsuites/layout/ materialize.test (+5 tests, 25/25; full punk/mix subtree 71 pass / 1 known skip).
• Template refresh - make.tcl's thin-layout sync (the sync_layouts make-step recorded in src/project_layouts/.punkcheck) now syncs the canonical REPO ROOT .gitignore into every vendor/punk layout as gitignore.in alongside the existing boot-script/manifest pairs (which remain gated on the layout carrying src/make.tcl); workflow_text DIAGRAM 1b + release step 9 updated. The four payloads are byte-identical mirrors of root - the vintage variants are eliminated as drift per this goal's "essentially a copy" framing.
• Modpod path - the store->modpod sync carries gitignore.in and its prune removed the stale modpod .gitignore; the rebuilt templates-0.2.0.tm was verified to carry the byte-identical payload. (The kit-baked copy inside punk91.exe etc is stale until the routine vfscommonupdate + bake refresh - noted, not a source defect.)

Verification evidence (acceptance clause by clause):

1. "scan of src/project_layouts finds no file named .gitignore" - VERIFIED (recursive force listing; also none in the modpod payload source folder).
2. "git check-ignore --no-index over every layout file matches only root-.gitignore rules" - VERIFIED: full sweep matched exactly one file, src/project_layouts/.punkcheck, by the ROOT rule *.punkcheck; no nested rule exists to fire.
3. "project generated from each affected layout receives a working .gitignore whose content matches the canonical payload" - VERIFIED end-to-end (punk91 src script driver, FOSSIL_HOME diverted): generated projects from punk.basic, punk.minimal, punk.project and punk.sample-0.1 each received .gitignore byte-identical to root; punk.othersample (the .anti deletion fixture) received none, by design. Generation from the module-carried layout (punk.project#2) returns old-variant content only from the stale kit-baked templates module; the freshly built templates-0.2.0.tm carries the byte-identical payload (verified by direct member extraction).
4. "editing a canonical source updates the derived payloads (punkcheck-tracked), covering vendor/punk" - VERIFIED both directions: a transient marker line appended to root .gitignore propagated to all four payloads on make.tcl libs (punkcheck-recorded "PROJECT LAYOUT update" events) and was removed again after reverting root. Stale clause: "...as well as custom/_project" - the _project pseudo-vendor store level was retired for layouts by G-087 stage 5 (2026-07-19, after this goal was drafted 2026-07-06): no custom/_project layout exists to cover, and custom-tier payloads are authored overrides (not derived mirrors) by design. Flagged for the user at completion - the refresh covers every layout under vendor/punk, which is the live reading of the clause.
5. "previously hidden template files remain git-tracked without per-file force-add exceptions" - VERIFIED: the fc1c474c README files (project-0.1/{bin,lib,modules}/ README.md, bin/runtime/README.md) are tracked and match no ignore rule; the previously invisible modpod-copy README trees now appear as ordinary (trackable) files. The accidental-ignore class is structurally impossible: no nested .gitignore exists to fire.

Remaining manual items: none for the goal itself. The stale-kit refresh (vfscommonupdate + bake) is routine build maintenance. The binary-glob/fossil-commit note from G-087 (generated projects' trailing fossil commit aborts on binary payload - layouts ship no binary-glob) is a different problem class (missing fossil-side setting, not a live-config hazard) and stays out of this goal's acceptance - candidate follow-on goal.

Notes

• Affected today: src/project_layouts/vendor/punk/{basic,minimal,project-0.1,sample-0.1}/.gitignore. The same inert-payload treatment should apply to any future live-config payloads in layouts (e.g. .gitattributes); .fossil-settings/ payloads in layouts are inert by nature (fossil reads them only at a checkout root) and need no change.
• AGENTS.md payloads are the same hazard class (identified 2026-07-07 while designing template DOX for derived projects, see G-027 detail): an AGENTS.md stored as template payload under src/project_layouts/ is live DOX for any agent whose work touches layout paths in the punkshell repo itself - the DOX walking rule makes it binding contract, though its instructions are written for a generated project (e.g. its build/test commands would operate on template internals). When layouts gain AGENTS.md payloads (per the G-027 documentation-ownership design), they need the same inert storage + materialize-at-generation treatment as the .gitignore payloads.
• fauxlink background: target encoded in the filename (+ for /, url-style escapes), no filesystem support required, application-driven resolution - see src/bootsupport/modules/fauxlink-0.2.0.tm doctools header.
• Verification tie-in: the dual-VCS sync checks in .fossil-settings/AGENTS.md double as a regression detector for this class - after this goal, the "git-tracked invisible to fossil" comparison should show no template-payload entries.
• G-062 (archived) recorded a follow-on in this territory: seeding LICENSE.txt into generated project layouts (deliberately excluded from G-062's acceptance because src/project_layouts sync is restricted) - see goals/archive/G-062-project-license-file.md.
• G-087 (archived - goals/archive/G-087-thin-project-layouts.md) reconfirmed at each generation verification (stages 3 and 4, 2026-07-19): the trailing fossil initial commit of a punk.project-generated project aborts on binary payload content (bin/sdx.kit from the layout plus injected zip-based modpod .tm modules) because thin layouts ship no binary-glob settings - this goal's inert VCS-config payload treatment is the designated fix. Layouts without binaries (sample/derived fixtures) commit cleanly.
• Related (activation survey 2026-08-01, goals_xref score G-012): G-086 (light layout's VCS-config payloads get the same inert treatment), G-031 (thin-main skeleton propagates via this goal's refresh step), G-136 (icon seeding's overwrite-on-regeneration rides the same refresh semantics) - all three already point here; back-pointers recorded at activation.
• G-047 relationship (recorded 2026-07-24 after overlap review): G-047 seeds punkproject.toml workflow defaults plus ignore rules into layout payload - a new payload class for this goal's inert-storage plus refresh machinery to carry; the seeding rides this goal's refresh step.