G-035: update goal detail to reflect resolved mechanism investigation

The avoid-mixing rule is no longer folklore: both co-causes are now
identified and pinned. Co-cause 1 is stock Tcl's $satisfied early return
suppressing the library scan (silent older-version selection under
prefer latest, order-dependent indexing). Co-cause 2 is the libunknown
0.2.2 namespace-context bug (fixed 0.2.3, pinned by
pkgindex_relative_namespace_is_global). The rule is substantiated for
stock Tcl and neutralised under the punkshell chain.

Updates:
- Goal: replaced open-ended 'substantiate or retire' framing with the
  resolved finding; remaining work is the crossform characterization
  suite and AGENTS.md guidance
- Context: removed 'folklore' characterization; presented the mechanism
  as established knowledge with both co-causes
- Restructured Questions into Resolved (identified-mechanism findings
  with reproduction table) and Still Open (same-version collisions,
  package forget crossing forms, packagepreference interaction)
- Approach: references the new crossform suite complementing existing
  shadowing/ and discovery/ suites; guidance deliverables named
- Notes: promoted the 0.2.3 bug fix from baseline note to co-cause
  alongside the $satisfied mechanism; both explain the original folklore
- Identified mechanism section: restructured as two co-causes with
  explicit consequences replacing the either/or framing
- Acceptance: tightened to enumerate the five characterization cases and
  name the specific guidance deliverables
- Fixed typo: lllength -> llength in quoted stock Tcl source

Assisted-by: harness=pi; primary-model=anthropic/claude-sonnet-4-20250514; api-location=anthropic.com

goals/G-035-mixed-tm-pkgindex-provision.md

@@ -2,8 +2,8 @@
 
 Status: proposed
 Scope: src/tests/modules/punk/libunknown/testsuites/ (characterization suite); src/modules/punk/libunknown-*.tm and src/modules/punk/packagepreference-999999.0a1.0.tm (as characterised, fixed only if outright bugs surface); src/modules/AGENTS.md + src/lib/AGENTS.md (resulting guidance)
-Goal: the behaviour when the same package is provided both as a .tm module and as a pkgIndex.tcl-based library - same or differing versions, under the standard package unknown, punk::libunknown and punk::packagepreference - is characterised by committed tests, and the currently informal working rule ("avoid mixing provision forms for one package - unexpected behaviour even with libunknown's improvements") is either substantiated with the specific failure modes named in AGENTS.md guidance, or retired if the characterisation shows the machinery now handles mixing predictably.
-Acceptance: a committed test suite (extending src/tests/modules/punk/libunknown/testsuites/) characterises at least: same name+version provided via .tm and via pkgIndex.tcl (which registration wins, and whether it is deterministic across scan-trigger orderings) under the standard scanner, under punk::libunknown, and with punk::packagepreference active; differing versions across the two forms (version selection integrity including package prefer latest, and whether the losing form's registration lingers); re-registration effects (package forget then re-require crossing forms); surprising-but-accepted behaviours are pinned with GAP/known-quirk comments (the fossilmove characterization pattern), outright bugs fixed or filed as goals; the resulting do/don't guidance lands in src/modules/AGENTS.md and src/lib/AGENTS.md naming the characterised failure modes (or explicitly lifting the avoid-mixing rule if unwarranted).
+Goal: the per-chain behaviour when the same package is provided both as a .tm module and as a pkgIndex.tcl-based library is characterised by committed crossform tests. The avoid-mixing rule is already substantiated for stock Tcl (named mechanism: `$satisfied`-suppressed chain fallthrough producing silent older-version selection and order-dependent indexing; see the Identified Mechanism section) and neutralised under the punkshell chain (libunknown always falls through; the 0.2.3 namespace-context bug is fixed). The remaining work is the crossform characterization suite covering the still-open cases (same-version collisions, `package forget` crossing forms, packagepreference interaction) and AGENTS.md guidance naming the failure mode for stock Tcl and the safe condition for the punkshell chain.
+Acceptance: a committed crossform test suite under src/tests/modules/punk/libunknown/testsuites/ (new suite, complementing the existing shadowing/ and discovery/ suites which cover .tm-vs-.tm tie-breaking and pkgIndex.tcl sourcing scope respectively) characterises the identified-mechanism cases and the still-open cases: (1) the identified mechanism (tm `$satisfied` suppressing the library scan) reproduced under the standard scanner and confirmed absent under the punkshell chain - a .tm providing an older version silently winning over a newer pkgIndex.tcl version under `prefer latest`; (2) same name+version via both forms (which registration wins and whether it is deterministic across trigger orderings) under each chain; (3) differing versions with the newer in pkgIndex.tcl (version selection integrity under `prefer latest`, and whether the losing form's ifneeded registration lingers and can resurface after `package forget then re-require`); (4) interaction with punk::packagepreference's `package require` overload; (5) the pkgIndex.tcl side-effect and sibling-registration consequences of suppressed vs unsuppressed library scans; all three chains (standard scanner, libunknown, packagepreference) are covered by the same fixture matrix; surprising-but-accepted behaviours are pinned with GAP/known-quirk comments; outright bugs are fixed or filed; the resulting guidance lands in src/modules/AGENTS.md and src/lib/AGENTS.md naming the stock-Tcl failure mode (silent older-version selection + order-dependent indexing suppression) and the punkshell safe condition (libunknown chain always falls through; uninit'd child interps and plain tclsh still see stock behaviour).
 
 ## Context
 
@@ -19,45 +19,72 @@ Tcl packages arrive by two registration routes with different machinery:
   semantics, `auto_path` ordering rather than tm-list ordering.
 
 When the SAME package is provided by both routes - same version or different
-versions - the interaction is not well characterised. The user's standing
-informal rule (recalled 2026-07-07): **avoid mixing .tm and pkgIndex.tcl
-provision for one package** - unexpected behaviour was observed even after
-punk::libunknown's improvements to the unknown-handler chain. That rule is
-currently folklore: undocumented, unproven, and untested. Meanwhile the punk
-ecosystem genuinely straddles both forms (src/modules .tm trees vs src/lib
-pkgIndex libraries; kits carry both; punk::packagepreference overloads
+versions - the interaction has a now-identified mechanism (see below). The
+avoid-mixing rule is substantiated for stock Tcl with a named failure mode
+(silent older-version selection via the `$satisfied` early return; see the
+Identified Mechanism section) and neutralised under the punkshell chain
+(libunknown always falls through). Two co-causes of the original "unexpected
+behaviour" reports are now pinned: (1) the `$satisfied` suppression, and
+(2) the libunknown 0.2.2 namespace-context bug (pkgIndex.tcl's relative
+`namespace eval` creating `::punk::libunknown::foo` instead of `::foo`), fixed
+in 0.2.3 and pinned by `pkgindex_relative_namespace_is_global` in the
+discovery suite. What remains uncharacterised is the per-chain crossform
+matrix (same-version collisions, `package forget` crossing forms,
+packagepreference interaction) and its AGENTS.md guidance. Meanwhile the
+punk ecosystem genuinely straddles both forms (src/modules .tm trees vs
+src/lib pkgIndex libraries; kits carry both; punk::packagepreference overloads
 `package require` on top of the unknown-handler chain), so accidental mixing
 is a realistic hazard - and same-version-different-content drift (the
 2026-07-06 textblock incident class) would interact with it unpredictably.
 
-## Questions the characterisation must answer
-
-- Same name+version via .tm and via pkgIndex.tcl: which registration wins?
-  Is it deterministic, or dependent on which scan ran first (tm scan vs
-  auto_path scan can be triggered in either order by unrelated requires)?
-- Different versions across forms: does version selection stay correct
-  (including under `package prefer latest` and with alpha/dev versions)?
-  Does the losing form's `package ifneeded` registration linger, and can it
-  resurface via `package forget` + re-require?
-- How do punk::libunknown and punk::packagepreference each change the
-  answers? (libunknown replaces/wraps the unknown chain; packagepreference
-  overloads `package require` itself - three layers that can each reorder
-  scans.)
-- Are there differences across Tcl 8.6 and 9 (tm.tcl and tclPkgUnknown have
-  evolved)?
+## Resolved questions (identified mechanism, 2026-07-27)
+
+- **Stock Tcl silently selects the older .tm version over a newer pkgIndex.tcl
+  version.** Reproduced under `prefer latest` with mixpkg 1.0 (.tm) vs 2.0
+  (pkgIndex.tcl) on both 8.6.10 and 9.0.3: the standard chain returns 1.0,
+  `package versions` shows only 1.0, and the auto_path scan never runs for that
+  require (suppressed by `$satisfied`). Under the punkshell chain, both
+  versions register and `prefer latest` correctly selects 2.0.
+- **The suppression is order-dependent at inter-package scope.** Whether a
+  given auto_path directory gets indexed at all depends on whether some
+  earlier, unrelated require happened to be satisfied by a .tm - so sibling
+  packages and side effects in the same pkgIndex.tcl may or may not run
+  depending on the order of unrelated `package require` calls.
+- **The libunknown chain always falls through.** `zipfs_tm_UnknownHandler`
+  computes `$satisfied` but its early return is commented out; the chain
+  always proceeds to `zipfs_tclPkgUnknown`. This is affordable because the
+  epoch cache avoids re-scanning already-indexed directories.
+
+## Questions the characterisation suite must still answer
+
+- Same name+version via both forms: which registration wins, and is it
+  deterministic across trigger orderings? (The `$satisfied` mechanism does
+  not apply here - both scans must run since neither form satisfies alone
+  when versions match.)
+- `package forget` + re-require crossing forms: does the losing form's
+  `ifneeded` registration linger, and can it resurface?
+- How does punk::packagepreference's `package require` overload interact
+  with the crossform matrix? (It wraps `package require` itself, adding a
+  third layer on top of the two unknown-handler chains.)
+- Are there differences across Tcl 8.6 and 9 in the still-open cases?
 
 ## Approach
 
-- Extend the shadowing suite's child-interp probe pattern: scratch dirs
-  providing the same test package as a .tm and as a pkgIndex.tcl library, all
-  combinations of {same version, tm newer, pkgIndex newer} x {standard
-  scanner, libunknown, packagepreference} x scan-trigger orderings.
+- New crossform test suite under src/tests/modules/punk/libunknown/testsuites/
+  (complementing shadowing/ and discovery/), using the child-interp probe pattern
+  from both existing suites: scratch dirs providing the same test package as a
+  .tm and as a pkgIndex.tcl library, all combinations of {same version, tm
+  newer, pkgIndex newer} x {standard scanner, libunknown, packagepreference}
+  x scan-trigger orderings. The existing suites cover .tm-vs-.tm shadowing and
+  pkgIndex.tcl sourcing scope respectively; crossform is the missing matrix.
 - Pin observed behaviour; mark surprising-but-tolerable outcomes with
   GAP/known-quirk comments (the fossilmove characterization pattern) rather
   than encoding wishes; anything outright broken is fixed or filed.
-- Convert the folklore into documentation: AGENTS.md guidance in src/modules
-  and src/lib stating either the substantiated avoid-mixing rule with its
-  named failure modes, or the conditions under which mixing is safe.
+- Land the guidance: src/modules/AGENTS.md and src/lib/AGENTS.md name the
+  stock-Tcl failure mode (silent older-version selection + order-dependent
+  indexing suppression) and the punkshell safe condition (libunknown chain
+  always falls through; epoch cache makes this affordable; caveat: uninit'd
+  child interps and plain tclsh still see stock behaviour).
 
 ## Notes
 
@@ -84,25 +111,26 @@ is a realistic hazard - and same-version-different-content drift (the
   installguard.test sources the source-tree commandstack+packagepreference
   pair into child interps by path (resolution-proof) - see
   goals/archive/G-160-commandstack-hygiene.md.
-- Baseline moved 2026-07-27 (before activation): one such outright bug was found and
-  fixed under this Scope's "fixed only if outright bugs surface" clause - punk::libunknown
-  0.2.2 -> 0.2.3. Index scripts were being sourced in a proc of the `punk::libunknown`
-  namespace, so a pkgIndex.tcl's relative `namespace eval foo` created
-  `::punk::libunknown::foo` rather than `::foo`; indexes that define commands at source
-  time for their own ifneeded scripts to call then failed at require time with `invalid
-  command name` (found on twapi 4.7.2 in a third-party 8.6 kit). Pinned by
+- Co-cause of the original "unexpected behaviour" reports (identified alongside the
+  `$satisfied` mechanism): punk::libunknown 0.2.2 -> 0.2.3 fixed a namespace-context
+  bug where pkgIndex.tcl scripts were sourced in a proc of the `punk::libunknown`
+  namespace, so a relative `namespace eval foo` created `::punk::libunknown::foo`
+  rather than `::foo` (found on twapi 4.7.2 in a third-party 8.6 kit). Pinned by
   `pkgindex_relative_namespace_is_global` in
-  `src/tests/modules/punk/libunknown/testsuites/discovery/discovery.test`. Relevant to
-  this goal's characterization in two ways: the *namespace context* of index sourcing is
-  now a pinned property to preserve alongside the frame isolation, and one of the
-  behaviours the "avoid mixing provision forms" folklore may have been describing has
-  turned out to be a plain bug rather than a mixing hazard.
+  `src/tests/modules/punk/libunknown/testsuites/discovery/discovery.test`. The two
+  co-causes together - the stock-Tcl `$satisfied` suppression and the libunknown
+  namespace-context bug - explain the original folklore: (1) stock Tcl silently
+  suppressing the library scan made mixing unreliable, and (2) even when the library
+  scan did run under libunknown, the wrong namespace context broke pkgIndex.tcl
+  scripts that defined helper commands. Both are now fixed in the punkshell chain;
+  neither is a mixing-hazard inherent to having both forms present simultaneously.
 
-### The folklore's mechanism, identified 2026-07-27
+### Identified mechanism (2026-07-27 investigation)
 
-The developer's recollection (the rule came from `$satisfied` in the default
-`::tcl::tm::UnknownHandler` aborting the search too early) is correct, and the mechanism
-is now pinned down. Stock `tm.tcl` ends its scan with:
+Two co-causes explain the original "avoid mixing" rule, both now pinned:
+
+**Co-cause 1: stock Tcl's `$satisfied` early return suppresses the library scan.**
+Stock `tm.tcl` ends its scan with:
 
     if {$satisfied} {
         return
@@ -121,6 +149,11 @@ still have to fallback to the regular package search"), i.e. a performance optim
 but it couples an unrelated question (did the tm scan happen to satisfy this request?) to
 whether the library half of the search runs at all.
 
+punk::libunknown neutralises this: `zipfs_tm_UnknownHandler` still computes `$satisfied`
+but its early return is commented out, so the chain ALWAYS falls through to
+`zipfs_tclPkgUnknown`. This is what the epoch cache buys: unconditional fallthrough is
+only affordable because already-scanned directories are not re-globbed within an epoch.
+
 Reproduced with a synthetic fixture - one `.tm` providing `mixpkg 1.0`, one auto_path
 library providing `mixpkg 2.0` plus a sibling package and an index side effect, under
 `package prefer latest`. Identical results on Tcl 8.6.10 and 9.0.3:
@@ -138,25 +171,25 @@ no error, no warning. The other rows explain the spooky-action reports: whether
 auto_path directory has been indexed at all depends on whether some earlier, unrelated
 require happened to be satisfied by a .tm, so behaviour varies with require ORDER.
 
-punk::libunknown already neutralises this: `zipfs_tm_UnknownHandler` still computes
-`satisfied` but its early return is commented out, so the chain ALWAYS falls through to
-`zipfs_tclPkgUnknown`. That is the other half of the developer's recollection - and it is
-what the epoch cache buys: unconditional fallthrough is only affordable because
-already-scanned directories are not re-globbed within an epoch. The two mechanisms are
-one design.
+**Co-cause 2: libunknown 0.2.2 namespace-context bug (fixed 0.2.3).** Even when the
+library scan ran under libunknown, pkgIndex.tcl scripts were sourced in a proc of the
+`punk::libunknown` namespace, so a relative `namespace eval foo` created
+`::punk::libunknown::foo` rather than `::foo`. Indexes that defined helper commands for
+their ifneeded scripts then failed at require time with `invalid command name` (found on
+twapi 4.7.2). Pinned by `pkgindex_relative_namespace_is_global` in the discovery suite.
+This was a plain bug, not a mixing hazard inherent to having both forms present.
 
-Consequences for this goal's framing (the Goal line contemplates either substantiating the
-avoid-mixing rule or retiring it):
+**Consequences for the avoid-mixing rule:**
 
 - The rule is SUBSTANTIATED for stock Tcl, with a named failure mode (silent older-version
-  selection) rather than vague "unexpected behaviour".
-- It is NEUTRALISED under the punkshell chain, so the characterization should be
-  per-chain rather than a single verdict.
+  selection + order-dependent indexing suppression) rather than vague "unexpected behaviour".
+- It is NEUTRALISED under the punkshell chain (libunknown always falls through; the
+  namespace-context bug is fixed).
 - Stock behaviour stays reachable inside punkshell's own world - any interp that has not
   run `punk::libunknown::init` (child interps created without it, including deliberate
   test probes; a plain tclsh consuming punk .tm trees). So guidance cannot simply say
   "libunknown handles it".
-- Still open for the characterization proper: same-version (rather than newer-library)
+- Still open for the characterization suite: same-version (rather than newer-library)
   collisions, whether the losing form's registration lingers and can resurface after
   `package forget`, and the interaction with punk::packagepreference's `package require`
   overload.