punkshell/goals/G-141-libfetch-target-selection.md

G-141 libfetch consumption-derived target selection

Status: proposed Scope: src/make.tcl (libfetch target-set derivation + narrow/widen options; VFSLIBS missing-source hint; PACKAGES phase as the host-need input); src/runtime/libpackages.toml (declaration surface as consumed); src/runtime/vendorlib_vfs.toml (source_root="packages" entries as the kit-consumption input - format coordinated with G-115/G-127); src/tests/shell/testsuites/punkexe/maketcllibfetch.test (characterization: derivation, narrowing, widening); bin/AGENTS.md + src/runtime/AGENTS.md (derivation rule documentation) Goal: 'make.tcl libfetch' fetches what the checkout will consume rather than everything declared - the default target set is derived from the consumption declarations already in the tree (the host's lib_tcl deploy needs plus the lib-tier targets consumed by the kit set in play), with explicit options to narrow (a selective bake's kit set) and widen (the full declared set) - so libpackages.toml can grow across platforms, architectures and libraries without every host downloading every artifact. Acceptance: a linux host preparing the linux kit set obtains exactly its consumed set - allplatforms + linux-x86_64 fetched, win32-x86_64 NOT fetched, observable in the libfetch action report - via the default derivation or a single derived invocation libfetch/the bake names; the canonical win32 host's default stays bake-sufficient (linux-x86_64 derived for the punk9linux.vfs cross-target need; a default fetch then a full bake hits no missing-packages-source error); an explicit widening option fetches the full declared set (mirror priming / prepopulation); libfetch reports the derived targets and which declaration pulled each in; the VFSLIBS missing-source message names an invocation that fetches what the failing bake actually needs; maketcllibfetch.test characterizes derivation, narrowing and widening through the existing PUNK_LIBFETCH_CONFIG / PUNK_LIBFETCH_PACKAGES seams; the derivation rule is documented in the tier docs.

Context

libfetch was born target-generic in G-139 (achieved 2026-07-30 - see goals/archive/G-139-punkbin-library-consumption.md): it processes every [artifact.] entry in src/runtime/libpackages.toml unconditionally. At birth the declared set (allplatforms + win32-x86_64, both generations) was exactly the canonical win32 host's full consumption, so all-or-nothing was invisible. No alternatives weighing selective fetch were recorded.

G-140 (achieved 2026-07-30 - see goals/archive/G-140-linux-lib-tier-emission.md) added linux-x86_64 and already contains the evidence the gap pinches: its consumption phase needed a narrowed fetch and got one by pointing the PUNK_LIBFETCH_CONFIG test seam at a linux-only declarations file - a workaround, not a workflow. Meanwhile the CONSUMING phases are selective where the fetch is not: the PACKAGES deploy walks only allplatforms + the host platform (src/make.tcl), and VFSLIBS is per-kit declared, narrowed by selective bake (G-121, archived - see goals/archive/G-121-bakelist-selective-bake.md). Observed 2026-07-31: a WSL checkout preparing the linux bake downloads win32-x86_64 zips nothing there consumes.

Cross-target consumption is real and rules out naive host filtering: the win32 host's bake NEEDS linux-x86_64 (punk9linux.vfs declares linux-x86_64/tcl9/tcllibc in vendorlib_vfs.toml).

The growth vector making this worth a goal: G-004 names the remaining vendored trees (Img, itcl, sqlite, tdbc, twapi, ...) for migration through the G-139 template - declarations multiply across libraries x platforms x generations, and all-or-nothing fetch scales as every-host x every-artifact. The runtime tier - the design twin per G-123 - is already selective (punk-runtime fetches/materializes named runtimes on demand); the lib tier has no equivalent of "materialize what this checkout needs".

Approach

• Derive, don't declare twice: the needed set is computable from existing declarations - {allplatforms, host platform} for the lib_tcl deploy, union the prefixes of source_root="packages" vendorlib_vfs.toml entries for the kit set in play. libpackages.toml stays identity/consent focused (no host-conditional keys).
• Default kit set = all declared trees (canonical-host behaviour unchanged); narrowing reuses the selective-bake selection vocabulary (G-121) or a direct -targets override; a widening option (-targets all or equivalent) keeps fetch-everything reachable. The exact default composition for a standalone (non-bake) invocation is settled in the work.
• Report the derivation (target -> pulling declaration) in libfetch output; make the VFSLIBS missing-source hint name the invocation that fetches what the failing bake needs.
• Coordinate surfaces, not formats: G-127 owns per-target kit payload declaration and G-115 the declaration format - the derivation consumes whichever declaration surface is current (vendorlib_vfs.toml today).

Alternatives considered

• Host-platform filtering alone - rejected: cross-target kit consumption is real (linux-x86_64 needed on the win32 host for punk9linux.vfs).
• Host-conditional keys in libpackages.toml - rejected: duplicates consumption facts vendorlib_vfs.toml already declares and erodes the declaration file's deliberate identity/consent focus (G-139 design).
• Status quo with documented PUNK_LIBFETCH_CONFIG per-host files - rejected: that seam exists for test isolation; parallel declaration files are drift by design (G-140 used it exactly once, as a workaround).

Notes

• Related: G-139 (archived - goals/archive/G-139-punkbin-library-consumption.md) - created libfetch target-generic; this goal adds the selection dimension.
• Related: G-140 (archived - goals/archive/G-140-linux-lib-tier-emission.md) - the linux tier whose WSL scenario motivates this; its config-seam workaround is the recorded evidence of the gap.
• Related: G-121 (archived - goals/archive/G-121-bakelist-selective-bake.md) - the kit-selection vocabulary the narrowing reuses.
• Related: G-006 - will absorb libfetch's interim consent/fetch mechanism; target selection must survive that absorption.
• Related: G-065 - declarative vendoring absorbs/references the G-139 fetch pattern; selection becomes part of the inherited pattern.
• Related: G-123 - runtime-tier twin whose per-artifact selectivity this brings to the lib tier; keep vocabulary parity where sensible.
• Related: G-127 - owns per-target kit payload declaration and bake output targeting; whichever lands second adopts the other (its per-target declarations become this derivation's input).
• Related: G-115 - owns the payload declaration format; if it supersedes vendorlib_vfs.toml the derivation reads the successor surface.
• Related: G-004 - the growth driver: each vendored tree migrating to the tier multiplies declarations; selection keeps per-host cost proportional to consumption.
• Related: G-027 - derived projects receive libfetch via layout sync and inherit selection unchanged.
• Overlap survey 2026-07-31 (drafting): goals_xref paths over src/make.tcl, src/runtime/libpackages.toml, src/runtime/vendorlib_vfs.toml, bin/packages, src/tests/shell/testsuites/punkexe - strongest live intersections G-127 and G-115 (recorded above); G-005/G-023/G-024/G-101/G-131/G-077/G-061 surfaced as sibling-path adjacency only (G-024's format interest arrives via the G-127/G-115 coordination). Non-path relations G-006/G-065/G-004/G-123/G-027 from the libfetch reference sweep.