punkshell/goals/G-029-testmodules-from-srctests.md

G-029 Build packaged test:: #modpod modules from src/tests

Status: proposed Scope: src/make.tcl (test-module packaging step), src/tests/ (source of truth), src/modules/test/ (generated #modpod targets), src/modules/punk/mix/ (modpod tooling as needed) Goal: src/tests is the single source of truth for module test suites - a punkcheck-tracked make.tcl step generates the packaged test:: #modpod modules from src/tests/modules//testsuites content, ending hand-maintenance of parallel copies under src/modules/test/; the packaged form is a distributable in its own right - a user who downloads a built module can optionally download the matching test:: and verify the module's behaviour on their own system (package require + RUN, or an executable's -app test) with no source tree or test harness required. Acceptance: a make.tcl step generates a test:: #modpod under src/modules/test/ from the corresponding src/tests testsuites (punkcheck-tracked, skipped when sources unchanged); the generated package works through the packaged path - loadable via package require test:: and runnable via its SUITE/RUN interface (e.g a built executable's -app test) - reporting the same test names and pass counts as running the same suites directly via src/tests/runtests.tcl; the consumer scenario is proven standalone: the generated package (plus the module under test and their dependencies) runs in an environment without the project source tree present; suite data files (e.g roundtrip toml files with deliberate crlf/mixed line endings) survive packaging byte-for-byte; documentation states src/tests is the source of truth and the generated modpods are build artifacts not to be hand-edited; proven end-to-end for at least one real module (tomlish, whose src/tests port and still-live modpod created the dual-copy situation, is the natural first).

Context

Module tests currently exist in two forms:

1. src/tests/modules//testsuites/ - the source-tree harness form: self-contained tcltest files (own package requires, final cleanupTests, self-locating data) run via src/tests/runtests.tcl. This is where test development happens and is the intended structure going forward.
2. src/modules/test/#modpod--/ - the packaged form: a test:: module (wrapper .tm providing a SUITE/RUN interface and locating its bundled *_testsuites) that ships in kits and is run in built executables, e.g tomlish.exe -app test.

The packaged form is not just kit plumbing - it is a distributable for module consumers: a user who downloads a built module can optionally download the matching test:: package and verify the module's behaviour on their own system/platform, with no source tree or test harness - just package require test::<modulename> + its RUN interface, or a wrapped executable's -app test. That end-user scenario is what fixes the packaged form's requirements: fully self-sufficient (suites + data bundled), locatable via ordinary module paths, and runnable in a bare interp.

Nothing generates one form from the other, so they drift as parallel hand-maintained copies. The tomlish project (2026-07-07) is the live example: its modpod testsuites were ported to src/tests (with fixes - self-containment, single cleanupTests, self-located data), while the modpod remains the source the built test::tomlish module actually comes from. Both are now fossil-tracked and must be kept in sync by hand until this goal lands.

Approach

• A make.tcl step (natural companion to the existing module build steps) packages src/tests/modules//testsuites/** into the test:: #modpod - punkcheck-tracked so unchanged sources skip.
• The wrapper .tm (SUITE/RUN interface, suitesdir resolution) becomes part of the generation: either templated per module or a shared runtime the modpod references. Its interface must keep serving the -app test dispatch used by wrapped executables.
• Because src/tests files are self-contained, the packaged copies can be byte-identical file copies; the wrapper supplies discovery (suite names from directory structure) rather than environment variables the files depend on. Legacy wrapper variables (e.g ::test::tomlish::suitesdir) should not be reintroduced into test files - self-containment is the contract.
• Data fidelity: suite data (e.g roundtrip toml files with deliberate crlf/mixed line endings) must be copied byte-for-byte; the #modpod directory form is plain files, so no archive-level transformation should apply.
• Aggregate suites in the old modpods (fauxlink-based tests/ and dev/ collections) are a generation-time decision: reproduce via fauxlink, replace with wrapper-level suite selection, or drop - record the choice here.
• Naming: test:: derives from the src/tests path (modules/tomlish -> test::tomlish, modules/punk/args -> test::punk::args). Only modules with testsuites content get a package generated.

Alternatives considered

• Making the packaged modpod the source of truth and generating src/tests from it - rejected: src/tests is where development happens (runtests.tcl ergonomics, agent workflows, per-suite layout) and its self-contained-file contract is the healthier base; the wrapper form is derivable, the reverse requires stripping wrapper dependencies (as the tomlish port demonstrated).
• Keeping dual hand-maintained copies - rejected: that is the current drift the tomlish incident demonstrated; sync-by-discipline does not hold.

Notes

• Related: the tomlish project's src/tests port (its fossil checkins 20bd00dc, feedb6f0 record the dual-copy interim and name this goal); punkshell's own src/modules/test/ packages are further targets once the mechanism exists.
• The packaging step's output is a build artifact by policy even though it lives under src/ - the punkcheck records make regeneration cheap and hand-edits detectable.