Browse Source

runtests.tcl: fix multiple trailing file tails collapsing into one glob

Two independent list-flattening bugs made `runtests.tcl foo.test bar.test`
match zero files, forcing agents to run files individually:

- the trailing glob value is -multiple 1, so its parsed value is a list;
  plain lappend nested it as a single space-containing element
- punk::path::treefilenames declares tailglobs as a -multiple 1 positional
  (one glob per argument), but the whole file_globs list was passed as one
  argument, re-collapsing it (this also broke multiple globs supplied via
  -tcltestoptions {-file {...}})

Single names always worked because a one-element Tcl list flattens to
itself, which masked both bugs. Verified: two file tails now discover and
run both suites (files=2, 40/40 pass).

Also note multi-file-tail support in src/tests/AGENTS.md Work Guidance.

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 6 days ago
parent
commit
079caa9ae2
  1. 1
      src/tests/AGENTS.md
  2. 7
      src/tests/runtests.tcl

1
src/tests/AGENTS.md

@ -37,6 +37,7 @@ Top-level test harness and source-tree tests for ShellSpy/Punk. Tests here exerc
- Use `-include-paths` with paths relative to `src/tests/`, using forward slashes. The flag accepts a space-separated list of glob patterns and may also be repeated, with all occurrences accumulating (`-multiple 1` as of 2026-07-10; previously the last flag silently won).
- Use `-tcltestoptions {-match <test_name>}` for focused single-test runs.
- For agent-efficient focused runs, prefer `<tcl_interpreter> src/tests/runtests.tcl -report compact -show-passes 0 -include-paths <relative/path/**> <file-tail.test>`.
- Multiple trailing file tails are supported and match independently: `runtests.tcl foo.test bar.test` runs both in one invocation (fixed 2026-07-17; previously multiple names collapsed into one glob matching zero files).
- Treat `RUNTESTS_RESULT status=warn` and compact warning reasons such as `missing-cleanupTests` as incomplete test results, even if observed pass events are listed.
- Add `-slowest <n>` when timing outliers are relevant.
- Add `-strict-exit 1` when a nonzero shell exit code is needed for failures or parser warnings.

7
src/tests/runtests.tcl

@ -439,7 +439,9 @@ if {[dict exists $tcltestoptions -file]} {
}
if {[dict exists $received glob]} {
lappend file_globs [dict get $values glob]
#glob is -multiple 1: $values glob is a list of the supplied names - expand,
#or multiple file tails collapse into one space-containing glob matching nothing
lappend file_globs {*}[dict get $values glob]
}
#fall back to default if still empty
if {[llength $file_globs] == 0} {
@ -498,7 +500,8 @@ dict set tallydict files_with_warnings [list]
set file_summaries [list]
set all_passes [list]
set testfiles [punk::path::treefilenames -dir $test_base -tailbase $test_base -exclude-files $exclude_files -include-paths $include_paths $file_globs]
#tailglobs is a -multiple 1 positional: each glob must be a separate argument, or multiple globs collapse into one
set testfiles [punk::path::treefilenames -dir $test_base -tailbase $test_base -exclude-files $exclude_files -include-paths $include_paths {*}$file_globs]
if {[dict exists $tcltestoptions -singleproc] && [dict get $tcltestoptions -singleproc]} {
if {!$report_json_only} {

Loading…
Cancel
Save