diff --git a/GOALS.md b/GOALS.md index 35827801..57f14cb9 100644 --- a/GOALS.md +++ b/GOALS.md @@ -251,3 +251,9 @@ Acceptance: ` proj:internal-src shell` (the documented canoni Scope: src/modules/punk/mix/ (modpod mount path), vfs::zip availability in the repl code interp, src/modules/punk/cap/ (templates capability handler) Goal: zip-based `#modpod` modules (e.g. `punk::mix::templates`) mount and their punk::cap handlers register in the `shell` subcommand's code interp on Tcl 8.6, matching the `script`/main-interp context - so `dev module.templates` and other `punk.templates`-capability consumers work in an interactive 8.6 shell instead of failing with `invalid command name vfs::RegisterMount`. Acceptance: `dev module.templates` in an interactive 8.6 punk shell (`shell` subcommand) lists the template providers with no `failed to load ZIP archive-based module` / `invalid command name vfs::RegisterMount` / `Unable to register any template providers` / `invalid command name ::punk::cap::handlers::templates::api_punk.templates` errors, matching the `script`-subcommand output on the same binary (verified 2026-07-07: script works, shell fails); root cause fixed (the code interp lacks the vfs::zip library that provides vfs::RegisterMount for pre-zipfs Tcl, present in the main interp - restore it to the code interp or use an alternative modpod mount path there); Tcl 9 (built-in zipfs) behaviour is unchanged. + +### G-035 [proposed] Characterise mixed .tm / pkgIndex.tcl provision of the same package +Scope: src/tests/modules/punk/libunknown/testsuites/ (characterization suite), src/modules/punk/libunknown-0.1.tm and src/modules/punk/packagepreference-999999.0a1.0.tm (as characterised, fixed only if outright bugs surface), src/modules/AGENTS.md + src/lib/AGENTS.md (resulting guidance) +Detail: goals/G-035-mixed-tm-pkgindex-provision.md +Goal: the behaviour when the same package is provided both as a .tm module and as a pkgIndex.tcl-based library - same or differing versions, under the standard package unknown, punk::libunknown and punk::packagepreference - is characterised by committed tests, and the currently informal working rule ("avoid mixing provision forms for one package - unexpected behaviour even with libunknown's improvements") is either substantiated with the specific failure modes named in AGENTS.md guidance, or retired if the characterisation shows the machinery now handles mixing predictably. +Acceptance: a committed test suite (extending src/tests/modules/punk/libunknown/testsuites/) characterises at least: same name+version provided via .tm and via pkgIndex.tcl (which registration wins, and whether it is deterministic across scan-trigger orderings) under the standard scanner, under punk::libunknown, and with punk::packagepreference active; differing versions across the two forms (version selection integrity including package prefer latest, and whether the losing form's registration lingers); re-registration effects (package forget then re-require crossing forms); surprising-but-accepted behaviours are pinned with GAP/known-quirk comments (the fossilmove characterization pattern), outright bugs fixed or filed as goals; the resulting do/don't guidance lands in src/modules/AGENTS.md and src/lib/AGENTS.md naming the characterised failure modes (or explicitly lifting the avoid-mixing rule if unwarranted). diff --git a/goals/G-035-mixed-tm-pkgindex-provision.md b/goals/G-035-mixed-tm-pkgindex-provision.md new file mode 100644 index 00000000..06156923 --- /dev/null +++ b/goals/G-035-mixed-tm-pkgindex-provision.md @@ -0,0 +1,69 @@ +# G-035 Characterise mixed .tm / pkgIndex.tcl provision of the same package + +Status: proposed +Scope: src/tests/modules/punk/libunknown/testsuites/ (characterization suite), src/modules/punk/libunknown-0.1.tm and src/modules/punk/packagepreference-999999.0a1.0.tm (as characterised, fixed only if outright bugs surface), src/modules/AGENTS.md + src/lib/AGENTS.md (resulting guidance) +Acceptance: as in root GOALS.md index (canonical). + +## Context + +Tcl packages arrive by two registration routes with different machinery: + +- **.tm modules**: `tcl::tm` path scan during `package unknown`; registration + keyed on name-version from the filename; the same-version tie-break rules + are now pinned by src/tests/modules/punk/libunknown/testsuites/shadowing/ + (2026-07-07): tcl::tm::add prepends, head of tcl::tm::list wins exact-version + ties, version beats order, punk::libunknown preserves all of it. +- **pkgIndex.tcl libraries**: auto_path directory scan (tclPkgUnknown) + evaluating index scripts; different scan timing, different overwrite + semantics, `auto_path` ordering rather than tm-list ordering. + +When the SAME package is provided by both routes - same version or different +versions - the interaction is not well characterised. The user's standing +informal rule (recalled 2026-07-07): **avoid mixing .tm and pkgIndex.tcl +provision for one package** - unexpected behaviour was observed even after +punk::libunknown's improvements to the unknown-handler chain. That rule is +currently folklore: undocumented, unproven, and untested. Meanwhile the punk +ecosystem genuinely straddles both forms (src/modules .tm trees vs src/lib +pkgIndex libraries; kits carry both; punk::packagepreference overloads +`package require` on top of the unknown-handler chain), so accidental mixing +is a realistic hazard - and same-version-different-content drift (the +2026-07-06 textblock incident class) would interact with it unpredictably. + +## Questions the characterisation must answer + +- Same name+version via .tm and via pkgIndex.tcl: which registration wins? + Is it deterministic, or dependent on which scan ran first (tm scan vs + auto_path scan can be triggered in either order by unrelated requires)? +- Different versions across forms: does version selection stay correct + (including under `package prefer latest` and with alpha/dev versions)? + Does the losing form's `package ifneeded` registration linger, and can it + resurface via `package forget` + re-require? +- How do punk::libunknown and punk::packagepreference each change the + answers? (libunknown replaces/wraps the unknown chain; packagepreference + overloads `package require` itself - three layers that can each reorder + scans.) +- Are there differences across Tcl 8.6 and 9 (tm.tcl and tclPkgUnknown have + evolved)? + +## Approach + +- Extend the shadowing suite's child-interp probe pattern: scratch dirs + providing the same test package as a .tm and as a pkgIndex.tcl library, all + combinations of {same version, tm newer, pkgIndex newer} x {standard + scanner, libunknown, packagepreference} x scan-trigger orderings. +- Pin observed behaviour; mark surprising-but-tolerable outcomes with + GAP/known-quirk comments (the fossilmove characterization pattern) rather + than encoding wishes; anything outright broken is fixed or filed. +- Convert the folklore into documentation: AGENTS.md guidance in src/modules + and src/lib stating either the substantiated avoid-mixing rule with its + named failure modes, or the conditions under which mixing is safe. + +## Notes + +- Related: the shadowing pin-tests (same suite family - this goal's tests + extend them), G-033 (visitor-mode path mixing makes cross-form collisions + more reachable: a kit's internal libs vs a visited project's .tm trees), + G-026/G-024 (same-version drift provenance), punk::packagepreference 0.1.1 + (its moduledoc auto-load hook lives in exactly this machinery). +- Out of scope: redesigning the loading machinery. This goal is + characterise-document-and-pin; behavioural changes only for outright bugs. diff --git a/src/tests/modules/AGENTS.md b/src/tests/modules/AGENTS.md index a21d27f7..211f266f 100644 --- a/src/tests/modules/AGENTS.md +++ b/src/tests/modules/AGENTS.md @@ -39,4 +39,5 @@ Unit tests for editable source modules under `src/modules/`, `src/modules_tcl8/` - `punkcheck/` — punkcheck module tests (install, summarize_install_resultdict, installtrack) - `punk/mix/` — punk::mix::cli tests (prune helpers, punkcheck virtual sources) and punk::mix::commandset::repo fossil move/rename characterization tests (`testsuites/repo/`, FOSSIL_HOME-isolated; GAP-marked tests pin behaviour G-022 will change) +- `punk/libunknown/` — .tm same-version shadowing pin-tests (`testsuites/shadowing/`): tcl::tm::add prepend rule, head-of-tm-list wins exact-version ties, version beats order, punk::libunknown parity — shipped behaviour depends on these (runtests tm ordering, punk_main package-mode precedence, G-033); mixed .tm/pkgIndex.tcl characterization is goal G-035 diff --git a/src/tests/modules/punk/libunknown/testsuites/shadowing/shadowing.test b/src/tests/modules/punk/libunknown/testsuites/shadowing/shadowing.test new file mode 100644 index 00000000..1ef79f04 --- /dev/null +++ b/src/tests/modules/punk/libunknown/testsuites/shadowing/shadowing.test @@ -0,0 +1,178 @@ +# -*- tcl -*- +# Pin-tests for Tcl module (.tm) same-version shadowing rules, under both the +# standard 'package unknown' scanner and punk::libunknown. +# +# Rules under test (verified experimentally 2026-07-06 on Tcl 9.0.3): +# 1. tcl::tm::add PREPENDS each of its arguments in turn - a single call +# 'tcl::tm::add A B' yields tcl::tm::list beginning {B A ...} (the LAST +# argument ends up at the head). +# 2. For the SAME module name-version found under multiple tm paths, the copy +# under the path nearest the HEAD of tcl::tm::list wins (first-scanned +# registration is kept, not overwritten). +# 3. Order only arbitrates exact-version ties: a higher version wins from any +# position. +# 4. punk::libunknown preserves all of the above (parity with the standard +# scanner). +# +# Shipped behaviour DEPENDS on these rules - if they change, more than this +# file breaks: +# - src/tests/runtests.tcl orders its testinterp tm list so bootsupport wins +# same-version ties over vendormodules (see its ORDERING NOTE; the +# stale-textblock incident of 2026-07-06). +# - punk_main.tcl's package-mode assembly uses block order as the deliberate +# same-version preference mechanism, and goal G-033's proj:/kit-first design +# builds on it. +# +# Run: tclsh src/tests/runtests.tcl -report compact -show-passes 0 -include-paths modules/punk/libunknown/** shadowing.test + +package require tcltest +package require punk::lib + +namespace eval ::testspace { + namespace import ::tcltest::* + + # ------------------------------------------------------------------------- + # Fixture: two tm dirs each providing shadowtest-1.0 with different impls, + # dirB additionally provides shadowver-1.0, dirA provides shadowver-1.1 + # (for the version-beats-order pin). + # ------------------------------------------------------------------------- + variable fixture_error "" + variable dirA "" + variable dirB "" + try { + set base [punk::lib::tempdir_newfolder -prefix pkshadowtest] + set dirA [file join $base tmA] + set dirB [file join $base tmB] + file mkdir $dirA $dirB + set fd [open [file join $dirA shadowtest-1.0.tm] w] + puts $fd {package provide shadowtest 1.0; proc ::shadow_whoami {} {return A}} + close $fd + set fd [open [file join $dirB shadowtest-1.0.tm] w] + puts $fd {package provide shadowtest 1.0; proc ::shadow_whoami {} {return B}} + close $fd + #version-beats-order fixture: higher version deliberately placed in the + #tie-LOSING dir for the A-first arrangement + set fd [open [file join $dirA shadowver-1.1.tm] w] + puts $fd {package provide shadowver 1.1; proc ::shadowver_whoami {} {return A11}} + close $fd + set fd [open [file join $dirB shadowver-1.0.tm] w] + puts $fd {package provide shadowver 1.0; proc ::shadowver_whoami {} {return B10}} + close $fd + variable tempbase $base + } on error {result} { + set fixture_error $result + } + testConstraint shadowfixture [expr {$fixture_error eq ""}] + if {$fixture_error ne ""} { + puts stderr "shadowing.test fixture setup failed (tests will be skipped): $fixture_error" + } + + #punk::libunknown must be resolvable from the testinterp's module paths for + #the parity tests (bootsupport provides it). + testConstraint libunknownavailable [expr {![catch {package require punk::libunknown}]}] + + # ------------------------------------------------------------------------- + # Probe: fresh child interp; optionally installs punk::libunknown; sets its + # tm paths to the scratch dirs (plus the parent's paths so punk modules + # resolve); triggers a scan; returns dict {head winner verwinner} + # head - first element of the child's tcl::tm::list (tail component) + # winner - which impl of shadowtest-1.0 loaded (A|B) + # verwinner- which impl of shadowver loaded (A11|B10) + # addmode: single -> one call: tcl::tm::add d1 d2 + # separate-> tcl::tm::add d1; tcl::tm::add d2 + # ------------------------------------------------------------------------- + proc shadow_probe {d1 d2 addmode libunknown} { + set i [interp create] + try { + interp eval $i {tcl::tm::remove {*}[tcl::tm::list]} + if {$libunknown} { + #make punk::libunknown loadable, then activate it + interp eval $i [list tcl::tm::add {*}[tcl::tm::list]] + interp eval $i {package require punk::libunknown; punk::libunknown::init} + interp eval $i {tcl::tm::remove {*}[tcl::tm::list]} + } + switch -- $addmode { + single { interp eval $i [list tcl::tm::add $d1 $d2] } + separate { + interp eval $i [list tcl::tm::add $d1] + interp eval $i [list tcl::tm::add $d2] + } + default { error "unknown addmode '$addmode'" } + } + set head [file tail [lindex [interp eval $i tcl::tm::list] 0]] + interp eval $i {package require shadowtest} + set winner [interp eval $i shadow_whoami] + interp eval $i {package require shadowver} + set verwinner [interp eval $i shadowver_whoami] + return [dict create head $head winner $winner verwinner $verwinner] + } finally { + interp delete $i + } + } + + # -- rule 1+2: standard scanner ------------------------------------------------ + + test shadow_tm_prepend_and_headwins {single-call add A B: list head is B (prepend rule) and B's impl wins the tie}\ + -constraints shadowfixture -body { + variable dirA; variable dirB + set r [shadow_probe $dirA $dirB single 0] + list [dict get $r head] [dict get $r winner] + } -result {tmB B} + + test shadow_tm_reversed {single-call add B A: list head is A and A's impl wins the tie}\ + -constraints shadowfixture -body { + variable dirA; variable dirB + set r [shadow_probe $dirB $dirA single 0] + list [dict get $r head] [dict get $r winner] + } -result {tmA A} + + test shadow_tm_separate_calls {separate add calls: later-added dir is at head and wins the tie}\ + -constraints shadowfixture -body { + variable dirA; variable dirB + set r [shadow_probe $dirA $dirB separate 0] + list [dict get $r head] [dict get $r winner] + } -result {tmB B} + + # -- rule 3: version beats order ------------------------------------------------ + + test shadow_tm_version_beats_order {higher version wins even from the tie-losing position}\ + -constraints shadowfixture -body { + variable dirA; variable dirB + #B is at head (wins ties) but shadowver-1.1 lives in A: version selection + #must pick 1.1 regardless of path order + set r [shadow_probe $dirA $dirB single 0] + dict get $r verwinner + } -result A11 + + # -- rule 4: punk::libunknown parity -------------------------------------------- + + test shadow_libunknown_parity {libunknown scanner: same prepend/head-wins outcome as standard}\ + -constraints {shadowfixture libunknownavailable} -body { + variable dirA; variable dirB + set r [shadow_probe $dirA $dirB single 1] + list [dict get $r head] [dict get $r winner] + } -result {tmB B} + + test shadow_libunknown_parity_reversed {libunknown scanner: reversed order parity}\ + -constraints {shadowfixture libunknownavailable} -body { + variable dirA; variable dirB + set r [shadow_probe $dirB $dirA single 1] + list [dict get $r head] [dict get $r winner] + } -result {tmA A} + + test shadow_libunknown_version_beats_order {libunknown scanner: version still beats order}\ + -constraints {shadowfixture libunknownavailable} -body { + variable dirA; variable dirB + set r [shadow_probe $dirA $dirB single 1] + dict get $r verwinner + } -result A11 + + # cleanup fixture + variable tempbase + if {[info exists tempbase] && $tempbase ne "" && [file isdirectory $tempbase]} { + catch {file delete -force $tempbase} + } +} + +tcltest::cleanupTests +namespace delete ::testspace