punkshell/goals/G-093-runtests-includepaths-discovery.md

G-093 runtests path-glob discovery: fix -include-paths multi-pattern/repeated-flag matching + expose -exclude-paths

Status: active Scope: src/tests/runtests.tcl, src/modules/punk/path-999999.0a1.0.tm (treefilenames/globmatchpath as implicated), src/modules/punk/args-999999.0a1.0.tm (only if -multiple implicated), src/tests/ (runner-targeting regression suite), src/tests/AGENTS.md Goal: runtests.tcl -include-paths discovery honours its documented contract - a single value carrying multiple space-separated glob patterns includes every pattern's matches, repeated -include-paths flags accumulate, and deep ** patterns match regardless of whether explicit file-tail globs are supplied - and a new -exclude-paths option (treefilenames' existing capability exposed, or an equivalent post-discovery globmatchpath filter over the relative paths) gives broad development runs a middle ground between full and targeted by excluding glob-matched files, with both directions' semantics pinned by the same regression suite and the include-side root cause recorded. Acceptance: the two 2026-07-19 failures reproduce as characterization tests before the fix and pass after it - (a) a single -include-paths value listing four deep patterns (the G-092 measurement set) discovers files from all four subtrees under the default *.test tail glob, and (b) two repeated -include-paths flags naming distinct subtrees discover both (observed matching zero files); the root cause is identified and recorded in this file; -exclude-paths is exposed on runtests (argdoc + src/tests/AGENTS.md documented, accumulate semantics matching -include-paths, exclusion winning over inclusion for the same file) and a full run excluding a named subtree discovers exactly the default-discovery file list minus that subtree; a regression suite under src/tests covers both options across single-value multi-pattern, repeated-flag, deep-pattern and tail-glob-present/absent combinations plus include/exclude interaction; a full-suite default-discovery run before and after the changes compares PARITY ok via scriptlib/developer/runtests_parity.tcl (default '**' discovery with no exclusions must be unaffected); src/tests/AGENTS.md path-targeting wording is reconciled with the verified semantics.

Context

Observed 2026-07-19 during the G-092 child-wall measurements (also recorded in goals/archive/G-092-split-longrunning-testfiles.md):

• FAILED (one of four matched): a single value with four deep patterns and no tail globs - -include-paths 'modules/punk/mix/testsuites/scriptwrap/** shell/testsuites/binscripts/** modules/argparsingtest/** shell/testsuites/punkexe/**' - discovered only the modules/argparsingtest file (1 file instead of ~10).
• FAILED (zero matched): the same subtrees supplied as repeated -include-paths flags, no tail globs - discovered 0 files, despite src/tests/AGENTS.md documenting accumulate semantics for repeated flags ("-multiple 1 as of 2026-07-10; previously the last flag silently won").
• WORKED in the same session: a multi-pattern single value WITH explicit file-tail globs (three patterns + six tails matched files across three subtrees); a single deep pattern with no tails ('modules/punk/args/' matched 28 files); the default ''.
• WORKED standalone: punk::path::globmatchpath matched the same deep patterns against full relative paths in isolation (e.g shell/testsuites/stdin/** vs shell/testsuites/stdin/runstdin.test -> 1), pointing suspicion at the treefilenames walk/prune path or the option-value shape handed to it rather than the matcher itself.
• Failure correlation so far: multi-pattern or repeated-flag forms combined with the DEFAULT tail glob; the same include-path forms with explicit tails worked. Not yet minimized - the first investigation step is a reduction against a small fixture tree.
• Workaround in use since: file-tail glob targeting (multiple tails are independent matches - documented in src/tests/AGENTS.md Work Guidance).

Candidate root causes to check (from the runtests side inward):

• runtests.tcl flattens the -multiple option value with 'concat {*}' - verify the shape it hands treefilenames for each invocation form (one list value vs list-of-occurrence-lists).
• punk::path::treefilenames' recursive walk/prune: whether directory descent decisions consult every include pattern or effectively only one when tail globs are defaulted.
• punk::args -multiple accumulation for -type list options (only if the received value shape contradicts its documentation).

Root cause (identified 2026-07-19)

A single cause explains both observed failures - a semantics mismatch between the layers, not multi-pattern interaction and not punk::args:

• punk::path::treefilenames matches -include-paths patterns against the DIRECTORY containing candidate files, and its documented X/** boundary semantics ("include files in subfolders based at /usr but not files within /usr itself") exclude files sitting directly in X: walk_treefilenames' directory_state sets recurse_below/next_allbelow but NOT include_files when the current directory equals the pattern's boundary (the pattern minus its trailing **).
• Three of the four G-092 measurement subtrees (scriptwrap, binscripts, punkexe) keep their .test files directly at the pattern root, so their X/** patterns matched nothing; only modules/argparsingtest (files one level deeper, under testsuites/argparsingtest/) matched - exactly the observed "one of four" (1 file pre-split, 2 post-split).
• Mechanical reduction (fixture tree, tclsh 9.0.3): treefilenames -include-paths {sub1/} with sub1/x.test present returns EMPTY; {sub2/} with sub2/deep/y.test returns the deep file. Real-tree reduction with the exact four-pattern value returned only the 2 argparsingtest files.
• punk::args -multiple accumulation is CORRECT and was probed explicitly: repeated -include-paths flags and a single space-separated value both reach the runner as the same flattened pattern list (runtests' 'concat {*}' flatten included). The repeated-flag zero-file observation is fully explained by the boundary semantics applied to direct-file subtrees; no punk::args change was needed.
• The with-tails invocation that "worked" during G-092 worked because its subtrees kept files deeper than the pattern roots - tail globs were never a factor in the failure.

Fix shape: runtests' contract (matching each test file's RELATIVE PATH, consistent with its pre-existing -serial-paths matching at the same layer) is implemented as a post-walk filter in src/tests/testsupport/discovery.tcl - treefilenames walks the whole tree (match-everything include, tailglobs as before) and runtests_discovery::filter_paths applies include/exclude patterns to the tailbase-relative file paths via globmatchpath (include on any match, exclusion wins). punk::path is deliberately untouched.

Alternatives considered

• Passing translated patterns down to treefilenames (X/** expanded to {X X/**} to pull in boundary-direct files) - rejected: cannot express exact-file patterns or per-pattern tail constraints, and leaves runtests coupled to directory-oriented semantics.
• Changing treefilenames' own X/** semantics to include boundary-direct files - rejected: its documented directory-oriented contract is self-consistent and other callers depend on the module; a file-path matching mode for punk::path remains a possible future enhancement (out of scope here).
• Walk pruning is lost for targeted runs (the walk is now always full-tree) - accepted: the default '**' run already walked the whole tree, and src/tests is small.

Notes

• The -exclude-paths clause was bundled into this goal at user direction 2026-07-19 (it was initially a Notes-level suggestion). treefilenames already implements -exclude-paths (its argdoc documents tailbase-relative matching) but runtests does not expose it - the exposure is expected to be a thin option declaration + passthrough, OR a post-discovery filter over the returned relative paths via globmatchpath (shown reliable standalone), whichever the include-side root cause makes the safer home. Middle-ground motivation examples: excluding shell/** when no built kit is present, runtimebash_wsl without WSL, or a heavy suite family unrelated to the change - most valuable for sequential/tcl8.6 contexts where full runs still cost minutes (-jobs 8 full runs are ~1m26s post G-092). RESOLVED: the include-side root cause made the post-discovery globmatchpath filter the home for both directions (see Root cause above).
• Behaviour change pinned deliberately: a bare directory path without a glob tail (e.g -include-paths modules/punk/args) now matches NOTHING (append /** for a subtree). Under the old directory-oriented semantics it matched files directly in that directory. In exchange, an exact relative FILE path is now a valid single-file targeting pattern. Neither form was documented at the runtests level before; all documented usage is X/**.
• runtests.tcl -discover-only 1 (added with the fix): prints the discovered list and exits before any run machinery - the observation channel used by the regression suite and for cheap human/agent targeting checks. Emits a well-formed tcl-list line 'RUNTESTS_FILES ...' (marker first element, emitted last).
• Two-stage boot (added for -discover-only usability): discovery-phase loads only punk::args + punk::path (via testsupport/discovery.tcl); punk/Thread/shellrun/ punk::tcltestrun and the testinterp/child path assembly load after the -discover-only exit. Discover-only child processes dropped ~11.2s -> ~0.5s, and the regression suite's 8 end-to-end children dropped its file wall 93s -> ~7s (parallel-safe, well under the post-G-092 ~31s max-child bound).
• Verification context note: run verification through a context whose children have no attached Windows console (e.g the agent Bash tool; Git Bash/mintty). Under the agent PowerShell tool the children inherit an attached console with redirected stdio and the known console-context colour class fires (grepstr/nslist failures - the unfixed class observed during the G-091/G-092 console-run work, both achieved 2026-07-19 - see goals/archive/G-091-runtests-parallel-jobs.md), polluting baselines.

Progress

2026-07-19 root cause + fix + -exclude-paths + regression suite

Landed:

• src/tests/testsupport/discovery.tcl (new): runtests_discovery::filter_paths (pure include/exclude filter over relative file paths via punk::path::globmatchpath - any-include-match, then exclusion wins) and discover_testfiles (full treefilenames walk + filter, discovery order preserved). punk::path deliberately untouched.
• src/tests/runtests.tcl: -include-paths argdoc rewritten to the file-relpath contract; -exclude-paths exposed (-type list -multiple 1, same flatten, exclusion wins); -discover-only added (prints the discovered list plus a RUNTESTS_FILES tcl-list line and exits before any run machinery, removing the already-created tcltest tmpdir); discovery routed through discover_testfiles; two-stage boot so the discovery phase loads only punk::args + punk::path (discover-only child ~11.2s -> ~0.5s).
• src/tests/runner/testsuites/discovery/pathdiscovery.test (new, 23 tests): pure-filter cases, fixture-tree composed discovery (including the boundary-direct-file pin), and end-to-end -discover-only children against the real tree with both G-092 failure forms as characterization cases (file wall ~7s, parallel-safe).
• src/tests/runner/AGENTS.md (new); src/tests/AGENTS.md discovery bullets, testsupport bullet, Work Guidance (targeting, -exclude-paths, -discover-only) and Child DOX Index reconciled with the verified semantics.

Verification (2026-07-19, native tclsh 9.0.3, agent Bash context - see the Notes entry on the console-context requirement):

• Acceptance (a): the four-pattern G-092 value discovers 13 files across all four subtrees (was 2 - argparsingtest only). Pre-fix reproduction recorded mechanically at the discovery-engine layer (treefilenames with the runner's call shape: fixture sub1/** with a boundary-direct file -> empty; real-tree four-pattern value -> 2/13 files); the pre-fix runner had no discovery observation channel (-discover-only landed with the fix). Pinned ongoing by pathdiscovery-3.0.
• Acceptance (b): repeated -include-paths flags accumulate and equal the single-value form, both named subtrees discovered (pathdiscovery-3.1); the historical zero-file observation is explained under Root cause (punk::args -multiple probed healthy).
• Acceptance (-exclude-paths): a full run excluding a named subtree equals the default discovery list minus that subtree, order preserved - pinned structurally by pathdiscovery-3.3 and exercised at full-suite scale below.
• Acceptance (parity): pre-change baseline (default discovery, -jobs 16: 93 files / 1010 tests / failed=1, exec-14.3 only) vs post-change -jobs 16 with -exclude-paths runner/: PARITY ok - default '' discovery of the pre-existing set is unaffected. Post-change default -jobs 16 vs baseline differs ONLY by the added suite file (total 1010 -> 1033, passed 994 -> 1017, failed unchanged).
• Mode parity (src/tests/AGENTS.md verification bullet): post-change -jobs 16 vs singleproc sequential full runs: PARITY ok (94 files / 1033 tests / failed=1).
• Suite: pathdiscovery.test 23/23 in both modes; runtests -help renders the new options under the stage-1 boot (textblock soft-require resolves from bootsupport, ~0.9s); goals_lint clean; no runtests_tmp leak from -discover-only.

Remaining for acceptance: none known - all acceptance clauses verified above; the achieved flip is the user's call.