diff --git a/GOALS.md b/GOALS.md index 971de23f..34411429 100644 --- a/GOALS.md +++ b/GOALS.md @@ -338,7 +338,7 @@ Detail: goals/G-089-scriptlib-kits-and-modes.md Scope: src/modules/shellfilter-999999.0a1.0.tm (stack placement, insert_transform, junctions), src/modules/shellthread-999999.0a1.0.tm (worker park/adoption/termination), src/tests/modules/ (new characterization suites), README.md ('very unripe parts' note) Detail: goals/G-090-shellfilter-shellthread-audit.md -### G-093 [proposed] runtests path-glob discovery: fix -include-paths multi-pattern/repeated-flag matching + expose -exclude-paths +### G-093 [active] runtests path-glob discovery: fix -include-paths multi-pattern/repeated-flag matching + expose -exclude-paths Scope: src/tests/runtests.tcl, src/modules/punk/path-999999.0a1.0.tm (treefilenames/globmatchpath as implicated), src/modules/punk/args-999999.0a1.0.tm (only if -multiple implicated), src/tests/ (runner-targeting regression suite), src/tests/AGENTS.md Detail: goals/G-093-runtests-includepaths-discovery.md diff --git a/goals/G-093-runtests-includepaths-discovery.md b/goals/G-093-runtests-includepaths-discovery.md index f6601dea..9bf3990b 100644 --- a/goals/G-093-runtests-includepaths-discovery.md +++ b/goals/G-093-runtests-includepaths-discovery.md @@ -1,6 +1,6 @@ # G-093 runtests path-glob discovery: fix -include-paths multi-pattern/repeated-flag matching + expose -exclude-paths -Status: proposed +Status: active Scope: src/tests/runtests.tcl, src/modules/punk/path-999999.0a1.0.tm (treefilenames/globmatchpath as implicated), src/modules/punk/args-999999.0a1.0.tm (only if -multiple implicated), src/tests/ (runner-targeting regression suite), src/tests/AGENTS.md Goal: runtests.tcl -include-paths discovery honours its documented contract - a single value carrying multiple space-separated glob patterns includes every pattern's matches, repeated -include-paths flags accumulate, and deep ** patterns match regardless of whether explicit file-tail globs are supplied - and a new -exclude-paths option (treefilenames' existing capability exposed, or an equivalent post-discovery globmatchpath filter over the relative paths) gives broad development runs a middle ground between full and targeted by excluding glob-matched files, with both directions' semantics pinned by the same regression suite and the include-side root cause recorded. Acceptance: the two 2026-07-19 failures reproduce as characterization tests before the fix and pass after it - (a) a single -include-paths value listing four deep patterns (the G-092 measurement set) discovers files from all four subtrees under the default *.test tail glob, and (b) two repeated -include-paths flags naming distinct subtrees discover both (observed matching zero files); the root cause is identified and recorded in this file; -exclude-paths is exposed on runtests (argdoc + src/tests/AGENTS.md documented, accumulate semantics matching -include-paths, exclusion winning over inclusion for the same file) and a full run excluding a named subtree discovers exactly the default-discovery file list minus that subtree; a regression suite under src/tests covers both options across single-value multi-pattern, repeated-flag, deep-pattern and tail-glob-present/absent combinations plus include/exclude interaction; a full-suite default-discovery run before and after the changes compares PARITY ok via scriptlib/developer/runtests_parity.tcl (default '**' discovery with no exclusions must be unaffected); src/tests/AGENTS.md path-targeting wording is reconciled with the verified semantics. @@ -39,6 +39,52 @@ Candidate root causes to check (from the runtests side inward): - punk::args -multiple accumulation for -type list options (only if the received value shape contradicts its documentation). +## Root cause (identified 2026-07-19) + +A single cause explains both observed failures - a semantics mismatch between the layers, +not multi-pattern interaction and not punk::args: + +- punk::path::treefilenames matches -include-paths patterns against the DIRECTORY + containing candidate files, and its documented X/** boundary semantics ("include files + in subfolders based at /usr but not files within /usr itself") exclude files sitting + directly in X: walk_treefilenames' directory_state sets recurse_below/next_allbelow but + NOT include_files when the current directory equals the pattern's boundary (the pattern + minus its trailing **). +- Three of the four G-092 measurement subtrees (scriptwrap, binscripts, punkexe) keep + their .test files directly at the pattern root, so their X/** patterns matched nothing; + only modules/argparsingtest (files one level deeper, under testsuites/argparsingtest/) + matched - exactly the observed "one of four" (1 file pre-split, 2 post-split). +- Mechanical reduction (fixture tree, tclsh 9.0.3): treefilenames -include-paths {sub1/**} + with sub1/x.test present returns EMPTY; {sub2/**} with sub2/deep/y.test returns the + deep file. Real-tree reduction with the exact four-pattern value returned only the 2 + argparsingtest files. +- punk::args -multiple accumulation is CORRECT and was probed explicitly: repeated + -include-paths flags and a single space-separated value both reach the runner as the + same flattened pattern list (runtests' 'concat {*}' flatten included). The repeated-flag + zero-file observation is fully explained by the boundary semantics applied to + direct-file subtrees; no punk::args change was needed. +- The with-tails invocation that "worked" during G-092 worked because its subtrees kept + files deeper than the pattern roots - tail globs were never a factor in the failure. + +Fix shape: runtests' contract (matching each test file's RELATIVE PATH, consistent with +its pre-existing -serial-paths matching at the same layer) is implemented as a post-walk +filter in src/tests/testsupport/discovery.tcl - treefilenames walks the whole tree +(match-everything include, tailglobs as before) and runtests_discovery::filter_paths +applies include/exclude patterns to the tailbase-relative file paths via globmatchpath +(include on any match, exclusion wins). punk::path is deliberately untouched. + +## Alternatives considered + +- Passing translated patterns down to treefilenames (X/** expanded to {X X/**} to pull in + boundary-direct files) - rejected: cannot express exact-file patterns or per-pattern + tail constraints, and leaves runtests coupled to directory-oriented semantics. +- Changing treefilenames' own X/** semantics to include boundary-direct files - rejected: + its documented directory-oriented contract is self-consistent and other callers depend + on the module; a file-path matching mode for punk::path remains a possible future + enhancement (out of scope here). +- Walk pruning is lost for targeted runs (the walk is now always full-tree) - accepted: + the default '**' run already walked the whole tree, and src/tests is small. + ## Notes - The -exclude-paths clause was bundled into this goal at user direction 2026-07-19 (it @@ -50,4 +96,79 @@ Candidate root causes to check (from the runtests side inward): motivation examples: excluding shell/** when no built kit is present, runtimebash_wsl without WSL, or a heavy suite family unrelated to the change - most valuable for sequential/tcl8.6 contexts where full runs still cost minutes (-jobs 8 full runs are - ~1m26s post G-092). + ~1m26s post G-092). RESOLVED: the include-side root cause made the post-discovery + globmatchpath filter the home for both directions (see Root cause above). +- Behaviour change pinned deliberately: a bare directory path without a glob tail + (e.g -include-paths modules/punk/args) now matches NOTHING (append /** for a subtree). + Under the old directory-oriented semantics it matched files directly in that directory. + In exchange, an exact relative FILE path is now a valid single-file targeting pattern. + Neither form was documented at the runtests level before; all documented usage is X/**. +- runtests.tcl -discover-only 1 (added with the fix): prints the discovered list and + exits before any run machinery - the observation channel used by the regression suite + and for cheap human/agent targeting checks. Emits a well-formed tcl-list line + 'RUNTESTS_FILES ...' (marker first element, emitted last). +- Two-stage boot (added for -discover-only usability): discovery-phase loads only + punk::args + punk::path (via testsupport/discovery.tcl); punk/Thread/shellrun/ + punk::tcltestrun and the testinterp/child path assembly load after the -discover-only + exit. Discover-only child processes dropped ~11.2s -> ~0.5s, and the regression suite's + 8 end-to-end children dropped its file wall 93s -> ~7s (parallel-safe, well under the + post-G-092 ~31s max-child bound). +- Verification context note: run verification through a context whose children have no + attached Windows console (e.g the agent Bash tool; Git Bash/mintty). Under the agent + PowerShell tool the children inherit an attached console with redirected stdio and the + known console-context colour class fires (grepstr/nslist failures - the unfixed class + observed during the G-091/G-092 console-run work, both achieved 2026-07-19 - see + goals/archive/G-091-runtests-parallel-jobs.md), polluting baselines. + +## Progress + +### 2026-07-19 root cause + fix + -exclude-paths + regression suite + +Landed: + +- src/tests/testsupport/discovery.tcl (new): runtests_discovery::filter_paths (pure + include/exclude filter over relative file paths via punk::path::globmatchpath - + any-include-match, then exclusion wins) and discover_testfiles (full treefilenames + walk + filter, discovery order preserved). punk::path deliberately untouched. +- src/tests/runtests.tcl: -include-paths argdoc rewritten to the file-relpath contract; + -exclude-paths exposed (-type list -multiple 1, same flatten, exclusion wins); + -discover-only added (prints the discovered list plus a RUNTESTS_FILES tcl-list line + and exits before any run machinery, removing the already-created tcltest tmpdir); + discovery routed through discover_testfiles; two-stage boot so the discovery phase + loads only punk::args + punk::path (discover-only child ~11.2s -> ~0.5s). +- src/tests/runner/testsuites/discovery/pathdiscovery.test (new, 23 tests): pure-filter + cases, fixture-tree composed discovery (including the boundary-direct-file pin), and + end-to-end -discover-only children against the real tree with both G-092 failure + forms as characterization cases (file wall ~7s, parallel-safe). +- src/tests/runner/AGENTS.md (new); src/tests/AGENTS.md discovery bullets, testsupport + bullet, Work Guidance (targeting, -exclude-paths, -discover-only) and Child DOX Index + reconciled with the verified semantics. + +Verification (2026-07-19, native tclsh 9.0.3, agent Bash context - see the Notes entry +on the console-context requirement): + +- Acceptance (a): the four-pattern G-092 value discovers 13 files across all four + subtrees (was 2 - argparsingtest only). Pre-fix reproduction recorded mechanically at + the discovery-engine layer (treefilenames with the runner's call shape: fixture + sub1/** with a boundary-direct file -> empty; real-tree four-pattern value -> 2/13 + files); the pre-fix runner had no discovery observation channel (-discover-only + landed with the fix). Pinned ongoing by pathdiscovery-3.0. +- Acceptance (b): repeated -include-paths flags accumulate and equal the single-value + form, both named subtrees discovered (pathdiscovery-3.1); the historical zero-file + observation is explained under Root cause (punk::args -multiple probed healthy). +- Acceptance (-exclude-paths): a full run excluding a named subtree equals the default + discovery list minus that subtree, order preserved - pinned structurally by + pathdiscovery-3.3 and exercised at full-suite scale below. +- Acceptance (parity): pre-change baseline (default discovery, -jobs 16: 93 files / + 1010 tests / failed=1, exec-14.3 only) vs post-change -jobs 16 with -exclude-paths + runner/**: PARITY ok - default '**' discovery of the pre-existing set is unaffected. + Post-change default -jobs 16 vs baseline differs ONLY by the added suite file + (total 1010 -> 1033, passed 994 -> 1017, failed unchanged). +- Mode parity (src/tests/AGENTS.md verification bullet): post-change -jobs 16 vs + singleproc sequential full runs: PARITY ok (94 files / 1033 tests / failed=1). +- Suite: pathdiscovery.test 23/23 in both modes; runtests -help renders the new options + under the stage-1 boot (textblock soft-require resolves from bootsupport, ~0.9s); + goals_lint clean; no runtests_tmp leak from -discover-only. + +Remaining for acceptance: none known - all acceptance clauses verified above; the +achieved flip is the user's call. diff --git a/src/tests/AGENTS.md b/src/tests/AGENTS.md index 01626e8d..029bb187 100644 --- a/src/tests/AGENTS.md +++ b/src/tests/AGENTS.md @@ -15,6 +15,8 @@ Top-level test harness and source-tree tests for ShellSpy/Punk. Tests here exerc - `runtests.tcl` is the primary source-tree test entry point; sublevel `all.tcl` files are a legacy pattern and are not required. - Tests use `tcltest` unless a child AGENTS.md documents a different local harness. - `runtests.tcl` excludes `AGENTS.md` and `*.tcl` helper files when discovering `.test` files. +- Discovery path targeting (G-093, 2026-07-19): `-include-paths`/`-exclude-paths` patterns match each test file's path relative to `src/tests/` (forward slashes) via `punk::path::globmatchpath` — `*` within a segment, `?` a single character, `**` spanning segments. `X/**` includes files directly in `X` as well as deeper ones; an exact relative path is a valid single-file pattern; a bare directory path without a glob tail matches nothing (append `/**` for a subtree). Both options accept a space-separated pattern list and accumulate across repeated flags; exclusion wins over inclusion for the same file. Implemented in `testsupport/discovery.tcl` as a post-walk filter over the discovered relative FILE paths — deliberately not punk::path::treefilenames' own directory-oriented `-include-paths`, whose documented `X/**` boundary semantics exclude files directly in `X` and silently missed suites keeping `.test` files at the subtree root (the 2026-07-19 G-092 measurement failures). `-serial-paths` matching was already file-path-oriented and is unchanged. Pinned by `runner/testsuites/discovery/pathdiscovery.test`. +- `-discover-only 1` prints the discovered test-file list and exits before any run machinery starts — subsecond, via the G-093 two-stage boot (only punk::args + punk::path load before the exit; punk/Thread/shellrun/punk::tcltestrun and the testinterp/child path assembly load after). Always emits one machine-readable `RUNTESTS_FILES ...` line (a well-formed tcl list with the marker as first element, emitted last), plus one path per line first in non-json report modes. Use it to check targeting before an expensive run. - `runtests.tcl` defaults tcltest `-tmpdir` (the `makeFile`/`makeDirectory` location) to a fresh directory under the system temp area, deleted before exit, so test helper files never land in the source tree (aborted core exec.test runs previously littered `src/tests`). A `-tmpdir` supplied via `-tcltestoptions` overrides this. Applies to both modes: multi-process mode forwards `-tmpdir` (and all other tcltest options) to child processes via the generated environment payload. - The testinterp auto_path (replaced wholesale) also includes the running kit's internal `lib`/`lib_tcl` trees (zipfs app mount, tclkit `::tcl::kitpath`, cookfs `//cookit:/` — mirroring `src/vfs/_config/punk_main.tcl`), so kit-bundled packages (e.g. tcllib's `tcl::chan::fifo2`, needed by the shellrun harness) resolve when the project tree doesn't supply them. `tclPkgUnknown` scans only an entry plus immediate children, so the kit lib dirs must be explicit entries. No-op under a native tclsh. - The single-process testinterp runs `package prefer latest` so alpha-versioned dev modules (`999999.0a1.0`) are preferred over stable bootsupport/vendored copies on unversioned `package require`. @@ -33,14 +35,16 @@ Top-level test harness and source-tree tests for ShellSpy/Punk. Tests here exerc - Human-facing pass/fail/warning indicators in markdown/compact reports are ANSI-coloured only when stdout is a real windows console (`-colour auto` default: twapi `GetConsoleMode` on the STD_OUTPUT handle as the isatty-equivalent - `get_console_handle` is unsuitable, it succeeds for piped children of console shells; `NO_COLOR` honoured; `-colour on|off` overrides). Machine-facing output (the `RUNTESTS_RESULT` line, json reports) is never coloured, and piped/redirected output stays plain for agents. The runner uses raw literal SGR rather than `a+`/`a` so its indicators are independent of punk::console's process-global colour state. - ERROR-status failures now surface `errorInfo` (full Tcl error message/stack trace) in markdown, compact, and json reports. Compact mode truncates to a single line. - FAILED-status failures (result mismatch, not error) now surface `result_was` (actual) and `result_expected` (expected) in markdown, compact, and json reports. Compact mode truncates each to a single line. -- `testsupport/` holds helper `.tcl` files sourced or exec'd by `.test` files or the runner (not discovered as suites). `testsupport/child_test_runner.tcl` is the multi-process-mode bootstrap `runtests.tcl` execs per test file (see the multi-process bullets above). `testsupport/repl_console_driver.tcl` (G-001) is exec'd in a child tclsh by `modules/punk/repl/testsuites/repl/consolebackends.test`: an interactive repl cannot run inside the shared testinterp (the codethread's quit/exit callbacks thread::send to the thread's MAIN interp, bypassing a child testinterp), so repl-through-console verification always drives a child process. `testsupport/wslprobe.tcl` (G-059) provides `::punktest::wsl`: a memoized capability probe yielding the `wsllinux` constraint (default distro launches, answers uname/tool probes, and round-trips a file through a native tempdir - NOT mere wsl.exe existence) plus native-filesystem staging helpers (`staging_create`/`staging_copy_in`/`run_in`/`staging_cleanup`). WSL-gated tests must execute from a staging dir on the distro's native filesystem with the shared `/mnt` path used only for one-way copy-in/out - never operate on the Windows checkout from inside WSL (DrvFs is slow and cross-boundary stat differences make git re-hash its index and fossil see phantom changes). Probe invocations use `wsl -e ` only (`wsl --status`/`-l` emit UTF-16). Known limitation: a broken-but-present WSL that hangs (rather than errors) on `wsl -e` can stall the probe. +- `testsupport/` holds helper `.tcl` files sourced or exec'd by `.test` files or the runner (not discovered as suites). `testsupport/child_test_runner.tcl` is the multi-process-mode bootstrap `runtests.tcl` execs per test file (see the multi-process bullets above). `testsupport/discovery.tcl` is the discovery walk + path-targeting filter sourced by `runtests.tcl` (parent side, before children exist) and by the runner-targeting regression suite (see the discovery bullets above). `testsupport/repl_console_driver.tcl` (G-001) is exec'd in a child tclsh by `modules/punk/repl/testsuites/repl/consolebackends.test`: an interactive repl cannot run inside the shared testinterp (the codethread's quit/exit callbacks thread::send to the thread's MAIN interp, bypassing a child testinterp), so repl-through-console verification always drives a child process. `testsupport/wslprobe.tcl` (G-059) provides `::punktest::wsl`: a memoized capability probe yielding the `wsllinux` constraint (default distro launches, answers uname/tool probes, and round-trips a file through a native tempdir - NOT mere wsl.exe existence) plus native-filesystem staging helpers (`staging_create`/`staging_copy_in`/`run_in`/`staging_cleanup`). WSL-gated tests must execute from a staging dir on the distro's native filesystem with the shared `/mnt` path used only for one-way copy-in/out - never operate on the Windows checkout from inside WSL (DrvFs is slow and cross-boundary stat differences make git re-hash its index and fossil see phantom changes). Probe invocations use `wsl -e ` only (`wsl --status`/`-l` emit UTF-16). Known limitation: a broken-but-present WSL that hangs (rather than errors) on `wsl -e` can stall the probe. - Test provenance comments: an agent adding a test places one comment line directly above it — or one line above a contiguous group added in the same piece of work — of the form `#added (agent[, G-])[ - ]`. Date and the `agent` marker are mandatory; the goal reference is mandatory when a goal motivated the test; the note is optional (omit it when the test name/description already says why the test exists). Provenance lines record immutable facts only — never expectations such as "G-NNN will flip this" (that state lives in `_GAP_`-style test names and the owning goal's files). User-added tests may use the same form with `(user)` but are not required to. Do not retrofit existing suites in bulk; add provenance opportunistically when editing a file. Rationale: provenance must survive where VCS history does not travel — packaged test modules (G-029), the git+fossil dual history, and file regeneration/moves. ## Work Guidance - Select an appropriate interpreter, such as `tclsh`, `tclsh90`, or a built Punk executable such as `punk902z`. - Run the full source-tree suite with ` src/tests/runtests.tcl`. -- Use `-include-paths` with paths relative to `src/tests/`, using forward slashes. The flag accepts a space-separated list of glob patterns and may also be repeated, with all occurrences accumulating (`-multiple 1` as of 2026-07-10; previously the last flag silently won). +- Use `-include-paths` with glob patterns matched against test-file paths relative to `src/tests/`, forward slashes — e.g `modules/punk/args/**`; an exact relative path targets a single file, and as of G-093 `X/**` also matches files sitting directly in `X`. The flag accepts a space-separated list of patterns and may also be repeated, with all occurrences accumulating (`-multiple 1` as of 2026-07-10; previously the last flag silently won). +- Use `-exclude-paths` (same syntax and accumulation) to remove files from the included set — exclusion wins for the same file. It gives broad runs a middle ground between full and targeted: e.g `-exclude-paths shell/**` when no built kit is present, or skipping a heavy suite family unrelated to the change. +- Verify targeting cheaply before an expensive run with `-discover-only 1` (prints the discovered file list and exits, subsecond). - Use `-tcltestoptions {-match }` for focused single-test runs. - For agent-efficient focused runs, prefer ` src/tests/runtests.tcl -report compact -show-passes 0 -include-paths `. - Multiple trailing file tails are supported and match independently: `runtests.tcl foo.test bar.test` runs both in one invocation (fixed 2026-07-17; previously multiple names collapsed into one glob matching zero files). @@ -66,3 +70,4 @@ Top-level test harness and source-tree tests for ShellSpy/Punk. Tests here exerc - `modules/` — Source module test suites mirroring editable module namespaces (see modules/AGENTS.md) - `core/` — Selected Tcl core compatibility tests for standard Tcl and built Punk executables (see core/AGENTS.md) - `shell/` — Shell behavior tests and command-line interaction suites (see shell/AGENTS.md) +- `runner/` — Suites for the runtests.tcl harness itself: discovery/path-targeting (see runner/AGENTS.md) diff --git a/src/tests/runner/AGENTS.md b/src/tests/runner/AGENTS.md new file mode 100644 index 00000000..3536efa3 --- /dev/null +++ b/src/tests/runner/AGENTS.md @@ -0,0 +1,25 @@ +# src/tests/runner — Runner Harness Suites + +## Purpose + +Test suites for the `src/tests/runtests.tcl` harness itself (discovery/path-targeting, runner options) — as opposed to suites for source modules (`modules/`), shell behaviour (`shell/`) or Tcl core compatibility (`core/`). + +## Ownership + +- Same rules as the parent `src/tests/AGENTS.md`: maintainer-owned; agents may update tests here when changing runner behaviour; do not delete, skip, or weaken existing tests without explicit user direction. + +## Local Contracts + +- Suites here may exec `runtests.tcl -discover-only 1` child processes of `[info nameofexecutable]` (subsecond since the G-093 two-stage boot — discovery loads only punk::args + punk::path before the early exit). Keep such children cheap and few: a suite file's child-process cost lands in its `-jobs` child wall. +- `testsuites/discovery/pathdiscovery.test` is the G-093 runner-targeting regression suite: it pins the `-include-paths`/`-exclude-paths` file-relpath matching semantics at three levels — pure `runtests_discovery::filter_paths`, composed `discover_testfiles` over a fixture tree, and end-to-end `-discover-only` runs against the real test tree, including the two 2026-07-19 G-092 measurement failures as characterization cases. +- Fixture trees go under tcltest's `temporaryDirectory` (the runner's run-scoped `-tmpdir`), never into the source tree. +- Suites here must stay parallel-safe (no console interaction) — they are not in `-serial-paths`. + +## Work Guidance + +- Run: ` src/tests/runtests.tcl -report compact -show-passes 0 -include-paths runner/**` +- LF line endings (per root AGENTS.md user preferences). + +## Verification + +- The run above passes. For changes to runner discovery/targeting behaviour, also apply the parent `src/tests/AGENTS.md` verification bullet (full suite in both modes plus `scriptlib/developer/runtests_parity.tcl`). diff --git a/src/tests/runner/testsuites/discovery/pathdiscovery.test b/src/tests/runner/testsuites/discovery/pathdiscovery.test new file mode 100644 index 00000000..bd895993 --- /dev/null +++ b/src/tests/runner/testsuites/discovery/pathdiscovery.test @@ -0,0 +1,247 @@ +# -*- tcl -*- +# Regression suite for runtests.tcl path-glob discovery targeting (G-093): +# the -include-paths/-exclude-paths semantics implemented by +# testsupport/discovery.tcl (runtests_discovery::filter_paths and +# runtests_discovery::discover_testfiles), plus end-to-end +# 'runtests.tcl -discover-only' child runs against the real test tree, +# including the two 2026-07-19 G-092 measurement failures as characterization +# cases (a four-pattern single value that matched only one subtree, and +# repeated -include-paths flags that matched zero files). +# +# Contract pinned here (G-093 root cause: punk::path::treefilenames' +# directory-oriented -include-paths excluded files directly at an X/** +# boundary, so suites keeping .test files at the subtree root were silently +# missed; runtests now filters the tailbase-relative FILE paths instead): +# 1. Patterns match each test file's path relative to the test base +# (forward slashes): * within a segment, ? single char, ** spans segments. +# 2. X/** includes files directly in X as well as deeper ones. +# 3. An exact relative path is a valid single-file pattern; a bare directory +# path without a glob tail matches nothing. +# 4. A single -include-paths value may carry multiple space-separated +# patterns; repeated flags accumulate; both forms are equivalent. +# 5. -exclude-paths uses the same syntax and accumulation semantics; +# exclusion wins over inclusion for the same file. +# 6. File-tail globs (trailing positional args) compose with path patterns. +# +# Run: tclsh src/tests/runtests.tcl -report compact -show-passes 0 -include-paths runner/** pathdiscovery.test + +package require tcltest +package require punk::path + +namespace eval ::testspace { + namespace import ::tcltest::* + + #src/tests, located relative to this test file (four levels up), so the suite + #exercises the SOURCE-TREE runner and discovery helper regardless of cwd. + variable testbase [file dirname [file dirname [file dirname [file dirname [file normalize [info script]]]]]] + source [file join $testbase testsupport discovery.tcl] + + #the runner's own discovery exclude-files list (mirrored from runtests.tcl) + variable runner_exclude_files {AGENTS.md *.tcl} + + proc runtests_discovered {args} { + #run runtests.tcl -discover-only in a child process of this interpreter's + #executable and return the discovered relative-path list parsed from the + #RUNTESTS_FILES line (a well-formed tcl list with the marker as its first + #element). stderr is merged - parent boot noise lines are tolerated. + variable testbase + set out [exec [info nameofexecutable] [file join $testbase runtests.tcl] -discover-only 1 {*}$args 2>@1] + foreach ln [split $out \n] { + set ln [string trimright $ln \r] + if {[string match {RUNTESTS_FILES *} $ln]} { + return [lrange $ln 1 end] + } + } + error "no RUNTESTS_FILES line in runtests.tcl -discover-only output:\n$out" + } + + proc with_prefix {prefix paths} { + #the subset of paths lying below prefix (prefix without trailing slash) + set matched [list] + foreach p $paths { + if {[string match "$prefix/*" $p]} { + lappend matched $p + } + } + return $matched + } + + # ------------------------------------------------------------------------- + # Fixture tree (under tcltest's temporaryDirectory - the runner points that + # at a run-scoped system temp dir): + # fixbase/direct.test (file directly at the base) + # fixbase/helper.tcl (must be dropped by the *.tcl exclude-files rule) + # fixbase/AGENTS.md (must be dropped by the AGENTS.md exclude-files rule) + # fixbase/sub1/x.test (file directly at an X/** pattern boundary) + # fixbase/sub1/x.other (tail-glob discrimination) + # fixbase/sub2/deep/y.test (file below a deeper boundary) + # fixbase/sub2/other.test + # ------------------------------------------------------------------------- + variable fixture_error "" + variable fixbase "" + try { + set fixbase [file join [temporaryDirectory] g093_pathdiscovery_fixture] + file delete -force $fixbase + file mkdir [file join $fixbase sub1] [file join $fixbase sub2 deep] + foreach relpath { + direct.test + helper.tcl + AGENTS.md + sub1/x.test + sub1/x.other + sub2/deep/y.test + sub2/other.test + } { + set fd [open [file join $fixbase {*}[file split $relpath]] w] + puts $fd "#g093 pathdiscovery fixture file" + close $fd + } + } on error {result} { + set fixture_error $result + } + testConstraint g093fixture [expr {$fixture_error eq ""}] + if {$fixture_error ne ""} { + puts stderr "pathdiscovery.test fixture setup failed (fixture tests will be skipped): $fixture_error" + } + + proc fixdiscover {includes excludes {tails {*.test}}} { + #composed walk+filter over the fixture tree, result sorted for + #order-independent comparison + variable fixbase + variable runner_exclude_files + lsort [runtests_discovery::discover_testfiles $fixbase $includes $excludes $tails $runner_exclude_files] + } + + #added 2026-07-19 (agent, G-093) + test pathdiscovery-1.0 {filter_paths: single value carrying multiple patterns includes every pattern's matches} -body { + runtests_discovery::filter_paths {a/f.test b/g.test c/h.test} {a/** b/**} {} + } -result {a/f.test b/g.test} + + test pathdiscovery-1.1 {filter_paths: a literal ** include pattern matches every path} -body { + runtests_discovery::filter_paths {direct.test a/b/c.test} {**} {} + } -result {direct.test a/b/c.test} + + test pathdiscovery-1.2 {filter_paths: exclusion wins over inclusion for the same file} -body { + runtests_discovery::filter_paths {a/f.test a/g.test} {a/**} {a/g.test} + } -result {a/f.test} + + test pathdiscovery-1.3 {filter_paths: a literal ** exclude pattern removes everything} -body { + runtests_discovery::filter_paths {a/f.test b/g.test} {**} {**} + } -result {} + + test pathdiscovery-1.4 {filter_paths: a bare directory-like pattern matches no file paths} -body { + runtests_discovery::filter_paths {a/f.test} {a} {} + } -result {} + + test pathdiscovery-2.0 {fixture: default ** discovers every .test file at all depths} -constraints g093fixture -body { + fixdiscover {**} {} + } -result {direct.test sub1/x.test sub2/deep/y.test sub2/other.test} + + test pathdiscovery-2.1 {fixture: X/** includes a file directly at the pattern boundary (G-093 regression)} -constraints g093fixture -body { + fixdiscover {sub1/**} {} + } -result {sub1/x.test} + + test pathdiscovery-2.2 {fixture: deeper X/** boundary} -constraints g093fixture -body { + fixdiscover {sub2/deep/**} {} + } -result {sub2/deep/y.test} + + test pathdiscovery-2.3 {fixture: single value carrying two patterns includes both subtrees} -constraints g093fixture -body { + fixdiscover {sub1/** sub2/**} {} + } -result {sub1/x.test sub2/deep/y.test sub2/other.test} + + test pathdiscovery-2.4 {fixture: a bare directory path matches nothing (append /** for a subtree)} -constraints g093fixture -body { + fixdiscover {sub1} {} + } -result {} + + test pathdiscovery-2.5 {fixture: an exact relative path targets a single file} -constraints g093fixture -body { + fixdiscover {sub1/x.test} {} + } -result {sub1/x.test} + + test pathdiscovery-2.6 {fixture: explicit file-tail glob composes with a path pattern} -constraints g093fixture -body { + fixdiscover {sub1/**} {} {x.other} + } -result {sub1/x.other} + + test pathdiscovery-2.7 {fixture: multiple file-tail globs match independently under a path pattern} -constraints g093fixture -body { + fixdiscover {sub1/**} {} {*.test *.other} + } -result {sub1/x.other sub1/x.test} + + test pathdiscovery-2.8 {fixture: exclude a subtree from a full run} -constraints g093fixture -body { + fixdiscover {**} {sub2/**} + } -result {direct.test sub1/x.test} + + test pathdiscovery-2.9 {fixture: exclusion wins inside an included subtree} -constraints g093fixture -body { + fixdiscover {sub2/**} {sub2/deep/**} + } -result {sub2/other.test} + + test pathdiscovery-2.10 {fixture: exact-path exclusion beats an including pattern} -constraints g093fixture -body { + fixdiscover {sub1/**} {sub1/x.test} + } -result {} + + test pathdiscovery-2.11 {fixture: exclude-files rules drop AGENTS.md and *.tcl helpers regardless of tails} -constraints g093fixture -body { + fixdiscover {**} {} {*} + } -result {direct.test sub1/x.other sub1/x.test sub2/deep/y.test sub2/other.test} + + test pathdiscovery-3.0 {end-to-end: the G-092 four-pattern single value discovers files from all four subtrees (2026-07-19 failure (a))} -body { + set files [runtests_discovered -include-paths {modules/punk/mix/testsuites/scriptwrap/** shell/testsuites/binscripts/** modules/argparsingtest/** shell/testsuites/punkexe/**}] + set result [list] + foreach prefix { + modules/punk/mix/testsuites/scriptwrap + shell/testsuites/binscripts + modules/argparsingtest + shell/testsuites/punkexe + } { + lappend result [expr {[llength [with_prefix $prefix $files]] > 0}] + } + set result + } -result {1 1 1 1} + + test pathdiscovery-3.1 {end-to-end: repeated -include-paths flags accumulate and equal the single-value form (2026-07-19 failure (b))} -body { + set a [runtests_discovered -include-paths modules/punk/mix/testsuites/scriptwrap/** -include-paths shell/testsuites/binscripts/**] + set b [runtests_discovered -include-paths {modules/punk/mix/testsuites/scriptwrap/** shell/testsuites/binscripts/**}] + list [expr {$a eq $b}]\ + [expr {[llength [with_prefix modules/punk/mix/testsuites/scriptwrap $a]] > 0}]\ + [expr {[llength [with_prefix shell/testsuites/binscripts $a]] > 0}] + } -result {1 1 1} + + test pathdiscovery-3.2 {end-to-end: default discovery covers boundary-direct, deep and this suite's own file} -body { + set files [runtests_discovered] + set result [list] + foreach path { + modules/punk/mix/testsuites/scriptwrap/multishell.test + modules/argparsingtest/testsuites/argparsingtest/argparsingtest.test + runner/testsuites/discovery/pathdiscovery.test + } { + lappend result [expr {$path in $files}] + } + set result + } -result {1 1 1} + + test pathdiscovery-3.3 {end-to-end: -exclude-paths yields exactly the default list minus the excluded subtree, order preserved} -body { + set full [runtests_discovered] + set excl [runtests_discovered -exclude-paths modules/punk/mix/testsuites/scriptwrap/**] + set manual [list] + foreach p $full { + if {![string match modules/punk/mix/testsuites/scriptwrap/* $p]} { + lappend manual $p + } + } + list [expr {$excl eq $manual}] [expr {[llength $full] > [llength $excl]}] + } -result {1 1} + + test pathdiscovery-3.4 {end-to-end: path patterns compose with an explicit file tail (json report mode)} -body { + runtests_discovered -report json -include-paths {modules/punk/mix/testsuites/scriptwrap/** shell/testsuites/binscripts/** modules/argparsingtest/** shell/testsuites/punkexe/**} multishell.test + } -result {modules/punk/mix/testsuites/scriptwrap/multishell.test} + + test pathdiscovery-3.5 {end-to-end: an exact relative path discovers exactly that file} -body { + runtests_discovered -include-paths runner/testsuites/discovery/pathdiscovery.test + } -result {runner/testsuites/discovery/pathdiscovery.test} + + # cleanup fixture + variable fixbase + if {$fixbase ne "" && [file isdirectory $fixbase]} { + catch {file delete -force $fixbase} + } +} + +tcltest::cleanupTests +namespace delete ::testspace diff --git a/src/tests/runtests.tcl b/src/tests/runtests.tcl index c3bd99d1..7968b4c0 100644 --- a/src/tests/runtests.tcl +++ b/src/tests/runtests.tcl @@ -32,11 +32,17 @@ if {$libdir_arch ni $::auto_path} { lappend ::auto_path $libdir_arch } #------------------------------------ -package require punk +#Boot in two stages (G-093): discovery-time code needs only punk::args (argdoc/parse) and +#punk::path (required by testsupport/discovery.tcl) - the heavy run-phase packages (punk, +#Thread, shellrun, punk::tcltestrun) and the testinterp/child path assembly load after the +#-discover-only early exit below, keeping 'runtests.tcl -discover-only 1' a fast targeting +#inspection tool (boot ~1s instead of ~11s) for humans, agents and the runner-targeting +#regression suite (runner/testsuites/discovery/pathdiscovery.test). package require punk::args -package require Thread -package require shellrun -package require punk::tcltestrun + +#test-file discovery + -include-paths/-exclude-paths filtering (G-093) - shared with the +#runner-targeting regression suite +source [file join $test_base testsupport discovery.tcl] proc runtests_json_string {text} { set text [string map [list \\ \\\\ \" \\\" \n \\n \r \\r \t \\t] $text] @@ -488,76 +494,8 @@ proc runtests_collect_child_result {testfile_relative jobresult} { -#------------------------------------ -#for the tests running in child processes, -#use the unbuilt modules/libraries under development in preference to the installed versions. -#ORDERING NOTE (rules verified experimentally on Tcl 9.0.3, 2026-07-06 - identical results -#under Tcl's standard 'package unknown' scanner and punk::libunknown): -# - tcl::tm::add PREPENDS each of its arguments in turn, so a single call -# 'tcl::tm::add {*}$test_tmlist' produces tcl::tm::list in REVERSED test_tmlist order -# (the last element of test_tmlist ends up at the head of tcl::tm::list). -# - For the SAME module name-version found under multiple tm paths, the copy under the -# path nearest the HEAD of tcl::tm::list wins (first-scanned registration is kept). -# - Net effect: the LAST element of test_tmlist wins same-version ties. -#Different-version selection is unaffected by order: dev modules (999999.0a1.0) win via -#'package prefer latest' wherever they sit. But bootsupport and vendormodules snapshots can -#carry the SAME version of a module with different code (a version-bump-discipline failure in -#the source project, but it happens - observed 2026-07-06 with a stale vendormodules -#textblock 0.1.3 shadowing the fresh bootsupport copy and breaking shellfilter::run). -#bootsupport is deliberately the LAST test_tmlist element so the punkshell-managed bootsupport -#snapshot wins any same-version tie. -set test_tmlist [list] -set ifneeded_script "" -lappend test_tmlist [file normalize $test_base/../modules] ;#ie /src/modules -append ifneeded_script [punk::tcltestrun::tm_path_additional_ifneeded [file normalize $test_base/../modules] $project_root] -lappend test_tmlist [file normalize $test_base/../modules_tcl$tcl_major] -append ifneeded_script [punk::tcltestrun::tm_path_additional_ifneeded [file normalize $test_base/../modules_tcl$tcl_major] $project_root] -#vendored module dependencies (e.g voo) must be resolvable by modules/tests using them -lappend test_tmlist [file normalize $test_base/../vendormodules] -if {[file isdirectory $test_base/../vendormodules_tcl$tcl_major]} { - lappend test_tmlist [file normalize $test_base/../vendormodules_tcl$tcl_major] -} -lappend test_tmlist [file normalize $test_base/../bootsupport/modules] ;#after vendormodules - wins same-version ties -set test_auto_path [list] -lappend test_auto_path [file normalize $test_base/../lib] -lappend test_auto_path [file normalize $test_base/../lib/tcl$tcl_major] -lappend test_auto_path [file normalize $test_base/../../lib_tcl$tcl_major/$arch] -lappend test_auto_path [info library] -#the parent of [info library] is where an installation's binary packages usually live -#(e.g Thread, needed by shellrun in the testinterp) - the default tclsh auto_path has it, -#but we replace auto_path wholesale in the testinterp, and not every project carries a -#root lib_tcl/ runtime payload that would otherwise supply Thread. -lappend test_auto_path [file dirname [info library]] -#kit-internal library trees: when running under a kit/zipkit executable the wholesale -#auto_path replacement above would otherwise hide packages bundled in the kit's lib trees -#(e.g tcllib's tcl::chan::fifo2, needed by the shellrun harness) - tclPkgUnknown only scans -#an auto_path entry and its immediate children, so [file dirname [info library]] (e.g -#//zipfs:/app) cannot reach //zipfs:/app/lib_tcl/. Mirrors the internal-path -#classification in src/vfs/_config/punk_main.tcl: zipfs app mount, tclkit ::tcl::kitpath -#(exe path dir-shadow), cookfs //cookit:/ volume (mount name is compile-configurable; -#only the known default is supported - as per punk_main.tcl). -set kit_lib_bases [list] -if {[info commands tcl::zipfs::root] ne "" && [llength [tcl::zipfs::mount]]} { - set zipbase [file join [tcl::zipfs::root] app] - if {"$zipbase" in [tcl::zipfs::mount]} { - lappend kit_lib_bases $zipbase - } -} -if {[info exists ::tcl::kitpath] && $::tcl::kitpath ne ""} { - lappend kit_lib_bases [file normalize $::tcl::kitpath] -} -if {"//cookit:/" in [file volumes]} { - lappend kit_lib_bases //cookit:/ -} -foreach kit_base $kit_lib_bases { - foreach p [list lib lib_tcl$tcl_major] { - set kit_libdir [file join $kit_base $p] - if {[file isdirectory $kit_libdir] && $kit_libdir ni $test_auto_path} { - lappend test_auto_path $kit_libdir - } - } -} -#------------------------------------ +#(the testinterp/child module path assembly - test_tmlist/test_auto_path/ifneeded_script - +# runs below the -discover-only early exit as stage 2 of the G-093 two-stage boot) punk::args::define { @@ -598,16 +536,37 @@ punk::args::define { stderr from " -include-paths -type list -default {**} -multiple 1 -help\ - "List of glob patterns for paths. - Paths are relative to the test base and should use forward slashes as separators. - - Only test files under paths matching these patterns will be included. - For example, to only include test files under /src/modules/punk - -include-paths {modules/punk/**} - The flag may also be repeated, with all occurrences accumulating: + "List of glob patterns matched against each test file's path relative to the + test base, using forward slashes as separators. + + Only test files whose relative path matches at least one pattern are included. + * matches within a single path segment, ? matches a single character and + ** spans path segments - so modules/punk/** matches every test file below + modules/punk, including files sitting directly in modules/punk. + An exact relative path is also a valid pattern (single-file targeting): + -include-paths modules/punk/args/testsuites/args/dynamic.test + A bare directory path without a glob tail matches nothing - append /** to + include a subtree. + The value is a space-separated list of patterns, and the flag may also be + repeated, with all occurrences accumulating: -include-paths modules/punk/args/** -include-paths modules/punk/ns/** is equivalent to - -include-paths {modules/punk/args/** modules/punk/ns/**}" + -include-paths {modules/punk/args/** modules/punk/ns/**} + Files matching -exclude-paths are removed from the included set." + -exclude-paths -type list -default {} -multiple 1 -help\ + "List of glob patterns matched against each test file's path relative to the + test base - same syntax and accumulation semantics as -include-paths. + Test files matching any pattern are excluded; exclusion wins over inclusion + for the same file. + Gives broad runs a middle ground between full and targeted - e.g + -exclude-paths shell/** when no built kit is present, or excluding a heavy + suite family unrelated to the change being tested." + -discover-only -type boolean -default 0 -help\ + "If true, print the discovered test file list and exit without running any + tests. Always emits one machine-readable line 'RUNTESTS_FILES ...' + (a well-formed tcl list with RUNTESTS_FILES as its first element), plus one + path per line first in non-json report modes. No test report is produced. + For inspecting -include-paths/-exclude-paths targeting." -report -type string -default markdown -choices {markdown compact json markdown+json} -help\ "Output report style. markdown preserves the detailed human-readable report. compact prints a short Markdown report with failures, warnings, and optional slow tests. @@ -685,9 +644,11 @@ set report_json_only [expr {$opt_report eq "json"}] set report_emit_json [expr {$opt_report in {json markdown+json}}] -#-include-paths is -multiple 1: each occurrence contributes a list of patterns - flatten -#(the unreceived default {**} and single-occurrence usage flatten to themselves) +#-include-paths/-exclude-paths are -multiple 1: each occurrence contributes a list of +#patterns - flatten (unreceived defaults and single-occurrence usage flatten to themselves) set include_paths [concat {*}[dict get $opts -include-paths]] +set exclude_paths [concat {*}[dict get $opts -exclude-paths]] +set opt_discover_only [dict get $opts -discover-only] set tcltestoptions [dict merge $default_tcltestoptions $opt_tcltestoptions] @@ -765,7 +726,110 @@ set file_summaries [list] set all_passes [list] #tailglobs is a -multiple 1 positional: each glob must be a separate argument, or multiple globs collapse into one -set testfiles [punk::path::treefilenames -dir $test_base -tailbase $test_base -exclude-files $exclude_files -include-paths $include_paths {*}$file_globs] +#Discovery walks the whole tree below test_base (tailglobs applied per directory), then +#-include-paths/-exclude-paths filter the tailbase-relative FILE paths (globmatchpath; +#exclusion wins) - see testsupport/discovery.tcl for the contract and the G-093 rationale +#(treefilenames' own -include-paths matches the directory containing files and excludes +#files directly at an X/** boundary, silently missing suites keeping .test files at the +#subtree root). +set testfiles [runtests_discovery::discover_testfiles $test_base $include_paths $exclude_paths $file_globs $exclude_files] + +if {$opt_discover_only} { + #print the discovered relative paths and exit before any run machinery starts + #(before the multi-process workdir and the UDP tee log workers are created; the + #already-created default tcltest tmpdir is removed as at normal run end). + #The RUNTESTS_FILES line is a well-formed tcl list (marker first element) so paths + #survive machine parsing regardless of content; it is emitted last, mirroring the + #RUNTESTS_RESULT end-of-output convention. + if {!$report_json_only} { + foreach tf $testfiles { + puts $tf + } + } + puts [list RUNTESTS_FILES {*}$testfiles] + if {$runtests_tmpdir ne "" && [file isdirectory $runtests_tmpdir]} { + catch {file delete -force $runtests_tmpdir} + } + exit 0 +} + +#run-phase boot (stage 2 of the G-093 two-stage boot): the heavy run-phase packages and the +#testinterp/child path assembly, deferred past the -discover-only early exit above. +package require punk +package require Thread +package require shellrun +package require punk::tcltestrun + +#------------------------------------ +#for the tests running in child processes, +#use the unbuilt modules/libraries under development in preference to the installed versions. +#ORDERING NOTE (rules verified experimentally on Tcl 9.0.3, 2026-07-06 - identical results +#under Tcl's standard 'package unknown' scanner and punk::libunknown): +# - tcl::tm::add PREPENDS each of its arguments in turn, so a single call +# 'tcl::tm::add {*}$test_tmlist' produces tcl::tm::list in REVERSED test_tmlist order +# (the last element of test_tmlist ends up at the head of tcl::tm::list). +# - For the SAME module name-version found under multiple tm paths, the copy under the +# path nearest the HEAD of tcl::tm::list wins (first-scanned registration is kept). +# - Net effect: the LAST element of test_tmlist wins same-version ties. +#Different-version selection is unaffected by order: dev modules (999999.0a1.0) win via +#'package prefer latest' wherever they sit. But bootsupport and vendormodules snapshots can +#carry the SAME version of a module with different code (a version-bump-discipline failure in +#the source project, but it happens - observed 2026-07-06 with a stale vendormodules +#textblock 0.1.3 shadowing the fresh bootsupport copy and breaking shellfilter::run). +#bootsupport is deliberately the LAST test_tmlist element so the punkshell-managed bootsupport +#snapshot wins any same-version tie. +set test_tmlist [list] +set ifneeded_script "" +lappend test_tmlist [file normalize $test_base/../modules] ;#ie /src/modules +append ifneeded_script [punk::tcltestrun::tm_path_additional_ifneeded [file normalize $test_base/../modules] $project_root] +lappend test_tmlist [file normalize $test_base/../modules_tcl$tcl_major] +append ifneeded_script [punk::tcltestrun::tm_path_additional_ifneeded [file normalize $test_base/../modules_tcl$tcl_major] $project_root] +#vendored module dependencies (e.g voo) must be resolvable by modules/tests using them +lappend test_tmlist [file normalize $test_base/../vendormodules] +if {[file isdirectory $test_base/../vendormodules_tcl$tcl_major]} { + lappend test_tmlist [file normalize $test_base/../vendormodules_tcl$tcl_major] +} +lappend test_tmlist [file normalize $test_base/../bootsupport/modules] ;#after vendormodules - wins same-version ties +set test_auto_path [list] +lappend test_auto_path [file normalize $test_base/../lib] +lappend test_auto_path [file normalize $test_base/../lib/tcl$tcl_major] +lappend test_auto_path [file normalize $test_base/../../lib_tcl$tcl_major/$arch] +lappend test_auto_path [info library] +#the parent of [info library] is where an installation's binary packages usually live +#(e.g Thread, needed by shellrun in the testinterp) - the default tclsh auto_path has it, +#but we replace auto_path wholesale in the testinterp, and not every project carries a +#root lib_tcl/ runtime payload that would otherwise supply Thread. +lappend test_auto_path [file dirname [info library]] +#kit-internal library trees: when running under a kit/zipkit executable the wholesale +#auto_path replacement above would otherwise hide packages bundled in the kit's lib trees +#(e.g tcllib's tcl::chan::fifo2, needed by the shellrun harness) - tclPkgUnknown only scans +#an auto_path entry and its immediate children, so [file dirname [info library]] (e.g +#//zipfs:/app) cannot reach //zipfs:/app/lib_tcl/. Mirrors the internal-path +#classification in src/vfs/_config/punk_main.tcl: zipfs app mount, tclkit ::tcl::kitpath +#(exe path dir-shadow), cookfs //cookit:/ volume (mount name is compile-configurable; +#only the known default is supported - as per punk_main.tcl). +set kit_lib_bases [list] +if {[info commands tcl::zipfs::root] ne "" && [llength [tcl::zipfs::mount]]} { + set zipbase [file join [tcl::zipfs::root] app] + if {"$zipbase" in [tcl::zipfs::mount]} { + lappend kit_lib_bases $zipbase + } +} +if {[info exists ::tcl::kitpath] && $::tcl::kitpath ne ""} { + lappend kit_lib_bases [file normalize $::tcl::kitpath] +} +if {"//cookit:/" in [file volumes]} { + lappend kit_lib_bases //cookit:/ +} +foreach kit_base $kit_lib_bases { + foreach p [list lib lib_tcl$tcl_major] { + set kit_libdir [file join $kit_base $p] + if {[file isdirectory $kit_libdir] && $kit_libdir ni $test_auto_path} { + lappend test_auto_path $kit_libdir + } + } +} +#------------------------------------ if {[dict exists $tcltestoptions -singleproc] && [dict get $tcltestoptions -singleproc]} { if {!$report_json_only} { diff --git a/src/tests/testsupport/discovery.tcl b/src/tests/testsupport/discovery.tcl new file mode 100644 index 00000000..1b6e13ab --- /dev/null +++ b/src/tests/testsupport/discovery.tcl @@ -0,0 +1,71 @@ +#testsupport/discovery.tcl - test-file discovery and path-targeting filter for runtests.tcl +#added 2026-07-19 (agent, G-093) +# +#Discovery contract (the runner's -include-paths/-exclude-paths semantics): +# - the tree below the test base is walked unrestricted (punk::path::treefilenames with its +# default match-everything include, tailglobs applied per directory as before), then the +# tailbase-RELATIVE FILE PATHS are filtered against the include/exclude patterns using +# punk::path::globmatchpath (* within a segment, ? single char, ** spans segments). +# - a file is included when ANY include pattern matches its relative path, then dropped when +# ANY exclude pattern matches it - exclusion wins over inclusion for the same file. +# - matching FILE paths (not the directory containing them) means modules/punk/mix/** includes +# files sitting directly in modules/punk/mix as well as deeper ones, and an exact relative +# path is a valid single-file pattern. A bare directory path without a glob tail matches +# nothing (append /** for a subtree). +# +#Why the filter exists here rather than passing the patterns to treefilenames (G-093 root +#cause, 2026-07-19): treefilenames' own -include-paths matches the DIRECTORY containing +#files, and its documented X/** boundary semantics exclude files directly in X - so suites +#keeping .test files at the subtree root (e.g modules/punk/mix/testsuites/scriptwrap) were +#silently missed by X/** targeting. runtests' contract - like its -serial-paths matching - +#is file-path oriented, so the patterns are applied to the discovered relative paths instead. +#The walk is unpruned, but the default full run already walked the whole tree; the test tree +#is small. +# +#Sourced by runtests.tcl (parent side - discovery happens once, before any child processes) +#and by the runner-targeting regression suite (runner/testsuites/discovery/pathdiscovery.test). +#Caller must have module paths established before sourcing (bootsupport for the runner, +#the generated payload for suite child processes). + +package require punk::path + +namespace eval ::runtests_discovery {} + +proc ::runtests_discovery::filter_paths {relpaths include_patterns exclude_patterns} { + #pure filter over relative file paths - include when any include pattern matches, + #then exclusion wins. A literal ** include pattern matches every path (fast path). + set include_all [expr {"**" in $include_patterns}] + set kept [list] + foreach relpath $relpaths { + set include $include_all + if {!$include} { + foreach pat $include_patterns { + if {[punk::path::globmatchpath $pat $relpath]} { + set include 1 + break + } + } + } + if {!$include} { + continue + } + foreach pat $exclude_patterns { + if {[punk::path::globmatchpath $pat $relpath]} { + set include 0 + break + } + } + if {$include} { + lappend kept $relpath + } + } + return $kept +} + +proc ::runtests_discovery::discover_testfiles {base include_patterns exclude_patterns tailglobs exclude_files} { + #walk everything below base (treefilenames' own include filtering deliberately unused - + #see header), then filter the relative paths. Result keeps treefilenames' natural sort + #order, so runner discovery order is unchanged from pre-G-093 default discovery. + set relpaths [punk::path::treefilenames -directory $base -tailbase $base -include-paths {**} -exclude-files $exclude_files {*}$tailglobs] + return [filter_paths $relpaths $include_patterns $exclude_patterns] +}