# G-037 Propagate platform vendor libraries into kit vfs lib_tcl trees via make.tcl

Status: achieved 2026-07-08
Scope: src/make.tcl (new or extended step), src/vendorlib_tcl8 + src/vendorlib_tcl9 (sources), src/vfs/<kit>.vfs/lib_tcl8 + lib_tcl9 (targets), punkcheck tracking
Goal: platform-specific vendored binary packages under src/vendorlib_tcl<N>/<platform> reach the kit vfs lib_tcl<N> trees through a make.tcl step instead of hand-copying - updating a vendored package becomes a vendorlib drop plus standard build invocations (motivating case 2026-07-08: tcludp 1.0.12 -> 1.0.13 for the G-036 wedge fix - `libs`, `vfscommonupdate` and `project` all completed while every kit vfs still bundled udp 1.0.12, requiring a manual copy into each vfs lib_tcl9 folder).
Acceptance: with a newer package version placed under src/vendorlib_tcl9/<platform>, one documented make.tcl invocation updates the participating src/vfs/*/lib_tcl9 trees - installing the new package and removing or explicitly retiring the superseded version (no silent mixed-version provision, per the G-035 concerns) - with punkcheck-tracked provenance; which vfs folders participate is explicitly declared per kit rather than blanket-copied (kit vfs package sets may intentionally differ), with the declaration mechanism recorded (candidate home: the G-024 mapvfs toml); a subsequent `make.tcl project` yields kits loading the new version (provable via the tcludp case: built punk902z reports `package require udp` == 1.0.13 with no udp1.0.12 folder remaining in its vfs); the lib_tcl8 tree gets the same treatment or an explicit exclusion rationale in the goal record.

## Context

Motivating case (2026-07-08, the G-036 wedge remedy): tcludp 1.0.13 was placed in
`src/vendorlib_tcl9/win32-x86_64` and `make.tcl libs`, `make.tcl vfscommonupdate`, and
`make.tcl project` were all run - yet the rebuilt punk902z still loaded udp 1.0.12, because
every kit vfs still carried `lib_tcl9/udp1.0.12`. The upgrade had to be hand-copied into
each kit vfs folder (done for punk9win.vfs and punk9win_for_tkruntime.vfs in commit
a1b232dc).

What the existing targets actually cover (verified in make.tcl at the time of writing):

- `libs`/`project` (VENDORLIB steps, make.tcl ~lines 2420-2475): copy `src/vendorlib` ->
  `<projectroot>/lib`, and `src/vendorlib_tcl<N>/{allplatforms,<platform>}` ->
  `<projectroot>/lib_tcl<N>/{allplatforms,<platform>}` (punkcheck-tracked). These feed
  tclsh-based use of the project tree - never the kit vfs folders.
- `vfscommonupdate` (~lines 1934-1963): wholesale-replaces `src/vfs/_vfscommon.vfs/modules`
  and `.../lib` from `<projectroot>/modules` and `<projectroot>/lib`. `_vfscommon.vfs`
  carries no `lib_tcl<N>` platform tree at all.
- Kit vfs `lib_tcl<N>` folders (`punk9win.vfs`, `punk9win_for_tkruntime.vfs`,
  `punk8win.vfs`, etc.) are hand-maintained; `project` merges `_vfscommon.vfs` over each
  kit vfs but nothing sources the platform-lib payloads from vendorlib.

## Approach (design points to settle)

- **Per-kit participation must be declarative, not blanket.** Kit vfs package sets
  intentionally differ (e.g. punk9win vs punk9win_for_tkruntime vs slim kits), and a
  vendorlib drop must not force a package into every kit. Candidate home for the
  declaration: the mapvfs config, especially once G-024 moves it to toml (a per-kit
  package/include list or a `vendorlib` participation flag).
- **Supersession semantics**: installing `foo<newver>` alongside `foo<oldver>` creates
  mixed-version provision (G-035 territory) - the step should remove or explicitly retire
  the superseded folder, or refuse with a clear message. Folder-name conventions currently
  differ (`udp1.0.12` in vfs vs `tcludp1.0.13` in vendorlib) - the sync should make the
  target name match the source, and tolerate legacy names when detecting supersession
  (pkgIndex-declared package name/version, not folder name, is the truth).
- **Platform axis**: vendorlib platform folders are per-`<platform_generic>`
  (win32-x86_64 today); kit vfs folders are platform-specific by kit. The step should map
  vendorlib platform -> the kits built for that platform (mapvfs already knows runtime ->
  vfs -> output mappings).
- **punkcheck provenance** as with the other install steps (skip unchanged, record source).
- **lib_tcl8**: same mechanism should serve the tcl8 kits (punk8win.vfs still bundles udp
  1.0.12 as of 0.4.3 - see G-036 loose ends), or the exclusion is recorded here.

## Implementation (2026-07-08)

Mechanism as landed (design points above settled as follows):

- **Declaration**: `src/runtime/vendorlib_vfs.toml`, parsed with the bootsupport tomlish
  package (1.1.10) - no ad-hoc parsing. Named tables `[install.<name>]` with keys:
  `source` (package folder relative to src/), `targets` (list of kit vfs lib folders
  relative to src/vfs/), optional `supersedes` (folder names removed from each target
  before install). G-024's mapvfs toml conversion was NOT required first - this file is
  recorded as a candidate for absorption into it. (make.tcl's own help already carried a
  matching intent: "update the src/vendorlib based on src/vendorlib/config.toml (todo)".)
- **Step**: `make.tcl vfslibs` (new subcommand), also run automatically inside
  `make.tcl project` before kit vfs assembly. Malformed/unresolvable entries fail loudly
  naming the entry (exit 3).
- **Supersession**: explicit `supersedes` folder-name list per entry (the simpler option;
  pkgIndex-declared package identity detection remains a possible refinement). The step
  never removes the folder it is about to install, and removal happens before install.
- **Direction/authority**: declaration-driven only - nothing syncs unless declared, so the
  observed twapi drift (vendorlib 5.0b1 older than kit copies) is untouched until someone
  declares a twapi entry. This resolves the "explicit direction/authority semantics"
  question: authority is the declaration, not file timestamps/versions.
- **Cross-generation sources**: allowed when the package supports it - the motivating
  tcludp 1.0.13 folder is dual-generation (tcl9udp1013.dll + udp1013t.dll, pkgIndex
  selects by [package vsatisfies]), so the single vendorlib_tcl9 source serves the
  punk8win.vfs/lib_tcl8 target (no duplicate binary committed under vendorlib_tcl8).
- **punkcheck provenance**: records at `src/vfs/.punkcheck` via -punkcheck-folder (kept
  outside the .vfs payload folders so no records are baked into kits); skip-if-unchanged
  verified (second run: 0 copied, 9 skipped per target).
- **lib_tcl8**: same mechanism, exercised immediately - punk8win.vfs/lib_tcl8 had
  udp1.0.12 removed and tcludp1.0.13 installed via the declaration (the G-036 loose-end
  decision, made 2026-07-08).

Verification (2026-07-08, acceptance met - goal flipped to achieved):

- `make.tcl vfslibs`: installed tcludp1.0.13 into punk8win.vfs/lib_tcl8 (9 files),
  removed superseded udp1.0.12; re-recorded the two tcl9 kit targets (previously
  hand-copied). Second run: 0 copied / 9 skipped per target (punkcheck skip-unchanged).
- `make.tcl project` (with the vfslibs step now inside it): INSTALLED KITS punkbi.exe,
  punksys.exe, punk902z.exe. punk9win_for_tkruntime.vfs correctly detected as unchanged
  (its 1.0.13 payload predated the step) - cached kit redeployed.
- Built kits: punksys 8.6.13 `package require udp` = 1.0.13 loading
  lib_tcl8/tcludp1.0.13/udp1013t.dll; punkbi likewise; punk902z 9.0.2 = 1.0.13 loading
  zipfs lib_tcl9/tcludp1.0.13/tcl9udp1013.dll. No udp1.0.12 folder remains anywhere
  under src/vfs. Piped-shell exit smoke on punksys clean (PUNK_PIPE_EOF=exit).

## Alternatives considered

- Extending `vfscommonupdate` to also sync platform libs into `_vfscommon.vfs` - rejected
  as-is: `_vfscommon.vfs` is deliberately platform-neutral common payload; platform libs
  are per-kit and per-platform, so the declaration belongs at the kit/mapvfs level.
- Documented manual copy as the permanent process - rejected: silent staleness (the
  motivating case shipped kits with a known-buggy tcludp through three make invocations
  with no warning).

## Notes

- Related goals: G-024 (mapvfs toml - natural home for per-kit declarations), G-026 (vendor
  provenance policy), G-035 (mixed-version provision characterisation), G-004/G-005 (the
  no-committed-binaries endgame would reshape where these payloads come from), G-036 (the
  motivating tcludp upgrade).
- Interim state: tcl9 kits carry tcludp1.0.13 (hand-copied, commit a1b232dc); punk8win.vfs
  still carries udp1.0.12.
- Further drift evidence (2026-07-08): src/vendorlib_tcl9/win32-x86_64 carries twapi-5.0b1
  while the punk9win kit vfs bundles twapi 5.2.0 - vendorlib and kit vfs trees have already
  diverged in BOTH directions (vendorlib newer for udp, older for twapi), so the sync step
  needs explicit direction/authority semantics, not just "copy newer over older".