20 KiB

Raw Blame History

G-093 runtests path-glob discovery: fix -include-paths multi-pattern/repeated-flag matching + expose -exclude-paths

Status: achieved 2026-07-20 Scope: src/tests/runtests.tcl, src/modules/punk/path-999999.0a1.0.tm (treefilenames/globmatchpath as implicated), src/modules/punk/args-999999.0a1.0.tm (only if -multiple implicated), src/tests/ (runner-targeting regression suite), src/tests/AGENTS.md Goal: runtests.tcl path targeting shares one directory-oriented pathglob contract with punk::path::treefilenames - patterns match the directory containing a test file, a single -include-paths value carrying multiple space-separated patterns includes every pattern's matches, repeated flags accumulate, the pathglob syntax gains *** (zero or more segments) so X/*** expresses a base-inclusive subtree, and a new -exclude-paths option (same syntax and accumulation, exclusion winning) gives broad development runs a middle ground between full and targeted - with the include and exclude lattices (X, X/, X/, X/) separable, pinned by regression suites at both the punk::path and runner levels, and the root cause of the 2026-07-19 failures recorded in this file. Acceptance: the G-092 four-subtree measurement set spelled X/*** discovers files from all four subtrees under the default .test tail glob, and the X/* spelling of a base-direct suite matches zero files while emitting a stderr advisory (both pinned as characterization tests); two repeated -include-paths flags naming distinct subtrees in X/*** form discover both and equal the single-value form; the root cause of the original failures is identified and recorded in this file; punk::path 0.4.0 implements *** across pathglob_as_re/globmatchpath/treefilenames with the include and exclude lattice forms separable and pinned in the punk::path suite; -exclude-paths is exposed on runtests (argdoc + src/tests/AGENTS.md documented, accumulate semantics matching -include-paths, exclusion winning over inclusion for the same file) and a full run excluding a named subtree discovers exactly the default-discovery file list minus that subtree; -serial-paths classifies by the same directory-of-file reading; a regression suite under src/tests covers both options across single-value multi-pattern, repeated-flag, lattice-form and tail-glob-present/absent combinations plus include/exclude interaction; a full-suite default-discovery run before and after the changes compares PARITY ok via scriptlib/developer/runtests_parity.tcl (default '**' discovery with no exclusions must be unaffected); src/tests/AGENTS.md path-targeting wording is reconciled with the verified semantics.

Context

Observed 2026-07-19 during the G-092 child-wall measurements (also recorded in goals/archive/G-092-split-longrunning-testfiles.md):

FAILED (one of four matched): a single value with four deep patterns and no tail globs - -include-paths 'modules/punk/mix/testsuites/scriptwrap/** shell/testsuites/binscripts/** modules/argparsingtest/** shell/testsuites/punkexe/**' - discovered only the modules/argparsingtest file (1 file instead of ~10).
FAILED (zero matched): the same subtrees supplied as repeated -include-paths flags, no tail globs - discovered 0 files, despite src/tests/AGENTS.md documenting accumulate semantics for repeated flags ("-multiple 1 as of 2026-07-10; previously the last flag silently won").
WORKED in the same session: a multi-pattern single value WITH explicit file-tail globs (three patterns + six tails matched files across three subtrees); a single deep pattern with no tails ('modules/punk/args/' matched 28 files); the default ''.
WORKED standalone: punk::path::globmatchpath matched the same deep patterns against full relative paths in isolation (e.g shell/testsuites/stdin/** vs shell/testsuites/stdin/runstdin.test -> 1), pointing suspicion at the treefilenames walk/prune path or the option-value shape handed to it rather than the matcher itself.
Failure correlation so far: multi-pattern or repeated-flag forms combined with the DEFAULT tail glob; the same include-path forms with explicit tails worked. Not yet minimized - the first investigation step is a reduction against a small fixture tree.
Workaround in use since: file-tail glob targeting (multiple tails are independent matches - documented in src/tests/AGENTS.md Work Guidance).

Candidate root causes to check (from the runtests side inward):

runtests.tcl flattens the -multiple option value with 'concat {*}' - verify the shape it hands treefilenames for each invocation form (one list value vs list-of-occurrence-lists).
punk::path::treefilenames' recursive walk/prune: whether directory descent decisions consult every include pattern or effectively only one when tail globs are defaulted.
punk::args -multiple accumulation for -type list options (only if the received value shape contradicts its documentation).

Root cause (identified 2026-07-19)

A single cause explains both observed failures - a semantics mismatch between the layers, not multi-pattern interaction and not punk::args:

punk::path::treefilenames matches -include-paths patterns against the DIRECTORY containing candidate files, and its documented X/** boundary semantics ("include files in subfolders based at /usr but not files within /usr itself") exclude files sitting directly in X: walk_treefilenames' directory_state sets recurse_below/next_allbelow but NOT include_files when the current directory equals the pattern's boundary (the pattern minus its trailing **).
Three of the four G-092 measurement subtrees (scriptwrap, binscripts, punkexe) keep their .test files directly at the pattern root, so their X/** patterns matched nothing; only modules/argparsingtest (files one level deeper, under testsuites/argparsingtest/) matched - exactly the observed "one of four" (1 file pre-split, 2 post-split).
Mechanical reduction (fixture tree, tclsh 9.0.3): treefilenames -include-paths {sub1/} with sub1/x.test present returns EMPTY; {sub2/} with sub2/deep/y.test returns the deep file. Real-tree reduction with the exact four-pattern value returned only the 2 argparsingtest files.
punk::args -multiple accumulation is CORRECT and was probed explicitly: repeated -include-paths flags and a single space-separated value both reach the runner as the same flattened pattern list (runtests' 'concat {*}' flatten included). The repeated-flag zero-file observation is fully explained by the boundary semantics applied to direct-file subtrees; no punk::args change was needed.
The with-tails invocation that "worked" during G-092 worked because its subtrees kept files deeper than the pattern roots - tail globs were never a factor in the failure.

Fix shape (final, 2026-07-20): the ergonomics gap is closed at the pathglob-syntax layer - punk::path 0.4.0 adds *** (zero or more segments as a whole segment), so X/*** is the base-inclusive subtree spelling and runtests passes its patterns through to treefilenames unchanged: one directory-oriented semantics at every layer, with the file-name axis staying on the trailing tail globs. Two engine separability defects found en route are fixed in the same punk::path release: an exact or single-segment-glob include match no longer drags the whole subtree in (the deep walk previously set allbelow on ANY full match; the zipfs walk was already separable), and non-spanning exclude matches now drop only the matched directory's files instead of pruning (making the argdoc's **/_aside vs /_aside/ distinction real). The 2026-07-19 interim fix (file-relpath post-filtering at the runner layer) was reworked away at user direction - see Alternatives considered.

Alternatives considered

File-relpath post-filtering at the runner layer (SHIPPED 2026-07-19, reworked away 2026-07-20 at user direction): treefilenames walked everything and runtests_discovery::filter_paths matched include/exclude patterns against the tailbase-relative FILE paths. It fixed the boundary miss and made exact file paths valid patterns, but gave -include-paths a different meaning at the runtests layer than in treefilenames (same name, same syntax, different referent), made the trailing tail-glob axis partially redundant (a pattern's final segment also constrained the filename), and lost walk pruning. The user's review framed the root problem as syntax ergonomics (no zero-or-more-segments form) rather than the matching target - hence the *** rework. Single-file targeting is now a directory pattern plus a file tail.
Passing translated patterns down to treefilenames (X/** expanded to {X X/**} to pull in boundary-direct files) - rejected: two spellings for one intent, and unable to express per-pattern filename constraints.
Changing treefilenames' X/** semantics to include boundary-direct files - rejected: the in-dir / one-below / one-or-more-below separability is a deliberate design; *** completes the lattice instead of collapsing it.

Notes

The -exclude-paths clause was bundled into this goal at user direction 2026-07-19 (it was initially a Notes-level suggestion). treefilenames already implements -exclude-paths (its argdoc documents tailbase-relative matching) but runtests does not expose it - the exposure is expected to be a thin option declaration + passthrough, OR a post-discovery filter over the returned relative paths via globmatchpath (shown reliable standalone), whichever the include-side root cause makes the safer home. Middle-ground motivation examples: excluding shell/** when no built kit is present, runtimebash_wsl without WSL, or a heavy suite family unrelated to the change - most valuable for sequential/tcl8.6 contexts where full runs still cost minutes (-jobs 8 full runs are ~1m26s post G-092). RESOLVED (final form 2026-07-20): a thin passthrough to treefilenames' own -exclude-paths, which gained the exclude-side separability fix (|*-tail prunes, other matches drop only that directory's files).
Final semantics notes: a bare directory path X includes exactly the files directly in X (the lattice in-dir form); X/** deliberately excludes files directly in X, and a received include pattern contributing no files earns a stderr advisory pointing at X/***; single-file targeting is an exact directory pattern plus a file tail (-include-paths modules/punk/args/testsuites/args dynamic.test). The interim 2026-07-19 file-relpath semantics (exact FILE paths as patterns; bare dir matching nothing) lasted one day and are superseded - see Alternatives considered.
punk::path 0.4.0 ride-alongs (same release, see path-buildversion.txt): the legacy bare-* include collapse is removed (a * in treefilenames -include-paths no longer means match-everything - it is the one-below lattice form; no repo caller used it), and the globmatchpath default-nocase driveletter exception is now a direct regex-head rewrite (the old file-split of the regex text broke on c:/***'s glued group and accepted non-alpha drives). subfolders/subfolders1 rule-compiler machinery untouched (its own inadequacy TODO stands in the module).
-serial-paths reconciled to the same directory-of-file reading (default switched to modules/punk/console/*** + modules/opunk/console/*** - same file set as the old file-path-oriented console/** default): the pre-existing serial-vs-discovery divergence was the codebase's other same-option-different-referent case; both gone.
runtests.tcl -discover-only 1 (added with the fix): prints the discovered list and exits before any run machinery - the observation channel used by the regression suite and for cheap human/agent targeting checks. Emits a well-formed tcl-list line 'RUNTESTS_FILES ...' (marker first element, emitted last).
Two-stage boot (added for -discover-only usability): discovery-phase loads only punk::args + punk::path (via testsupport/discovery.tcl, which requires punk::path 0.4.0- - the runner parent resolves it from the refreshed bootsupport snapshot); punk/Thread/shellrun/punk::tcltestrun and the testinterp/child path assembly load after the -discover-only exit. Discover-only child processes dropped ~11.2s -> ~0.5s, and the regression suite's end-to-end children dropped its file wall 93s -> ~7s (parallel-safe, well under the post-G-092 ~31s max-child bound).
Verification context note: run verification through a context whose children have no attached Windows console (e.g the agent Bash tool; Git Bash/mintty). Under the agent PowerShell tool the children inherit an attached console with redirected stdio and the known console-context colour class fires (grepstr/nslist failures - the unfixed class observed during the G-091/G-092 console-run work, both achieved 2026-07-19 - see goals/archive/G-091-runtests-parallel-jobs.md), polluting baselines.

Progress

2026-07-19 root cause + fix + -exclude-paths + regression suite

Landed:

src/tests/testsupport/discovery.tcl (new): runtests_discovery::filter_paths (pure include/exclude filter over relative file paths via punk::path::globmatchpath - any-include-match, then exclusion wins) and discover_testfiles (full treefilenames walk + filter, discovery order preserved). punk::path deliberately untouched.
src/tests/runtests.tcl: -include-paths argdoc rewritten to the file-relpath contract; -exclude-paths exposed (-type list -multiple 1, same flatten, exclusion wins); -discover-only added (prints the discovered list plus a RUNTESTS_FILES tcl-list line and exits before any run machinery, removing the already-created tcltest tmpdir); discovery routed through discover_testfiles; two-stage boot so the discovery phase loads only punk::args + punk::path (discover-only child ~11.2s -> ~0.5s).
src/tests/runner/testsuites/discovery/pathdiscovery.test (new, 23 tests): pure-filter cases, fixture-tree composed discovery (including the boundary-direct-file pin), and end-to-end -discover-only children against the real tree with both G-092 failure forms as characterization cases (file wall ~7s, parallel-safe).
src/tests/runner/AGENTS.md (new); src/tests/AGENTS.md discovery bullets, testsupport bullet, Work Guidance (targeting, -exclude-paths, -discover-only) and Child DOX Index reconciled with the verified semantics.

Verification (2026-07-19, native tclsh 9.0.3, agent Bash context - see the Notes entry on the console-context requirement):

Acceptance (a): the four-pattern G-092 value discovers 13 files across all four subtrees (was 2 - argparsingtest only). Pre-fix reproduction recorded mechanically at the discovery-engine layer (treefilenames with the runner's call shape: fixture sub1/** with a boundary-direct file -> empty; real-tree four-pattern value -> 2/13 files); the pre-fix runner had no discovery observation channel (-discover-only landed with the fix). Pinned ongoing by pathdiscovery-3.0.
Acceptance (b): repeated -include-paths flags accumulate and equal the single-value form, both named subtrees discovered (pathdiscovery-3.1); the historical zero-file observation is explained under Root cause (punk::args -multiple probed healthy).
Acceptance (-exclude-paths): a full run excluding a named subtree equals the default discovery list minus that subtree, order preserved - pinned structurally by pathdiscovery-3.3 and exercised at full-suite scale below.
Acceptance (parity): pre-change baseline (default discovery, -jobs 16: 93 files / 1010 tests / failed=1, exec-14.3 only) vs post-change -jobs 16 with -exclude-paths runner/: PARITY ok - default '' discovery of the pre-existing set is unaffected. Post-change default -jobs 16 vs baseline differs ONLY by the added suite file (total 1010 -> 1033, passed 994 -> 1017, failed unchanged).
Mode parity (src/tests/AGENTS.md verification bullet): post-change -jobs 16 vs singleproc sequential full runs: PARITY ok (94 files / 1033 tests / failed=1).
Suite: pathdiscovery.test 23/23 in both modes; runtests -help renders the new options under the stage-1 boot (textblock soft-require resolves from bootsupport, ~0.9s); goals_lint clean; no runtests_tmp leak from -discover-only.

Status at the time: all original acceptance clauses verified. Superseded the next day by the user-approved contract amendment and *** rework below.

2026-07-20 rework to *** directory-oriented pass-through (user-approved contract amendment)

User review of the 2026-07-19 increment identified the divergence cost (-include-paths meaning different things at the runner than in treefilenames) and proposed the *** syntax; the Goal/Acceptance contract was amended to the *** spelling with user approval and the increment reworked the same day.