Browse Source

add goal G-029: build packaged test::<modulename> modpods from src/tests

src/tests becomes the single source of truth for module test suites: a
punkcheck-tracked make.tcl step generates the packaged test::<modulename>
#modpod modules from src/tests/modules/<namespacepath>/testsuites,
ending hand-maintained parallel copies under src/modules/test/.

The packaged form is a distributable in its own right: a user who
downloads a built module can optionally download the matching
test::<modulename> and verify the module''s behaviour on their own
system - package require + RUN interface, or a wrapped executable''s
-app test - with no source tree or harness. Acceptance proves the
standalone consumer scenario (runs without the project source tree),
suite-result parity with src/tests/runtests.tcl, and byte-for-byte
data fidelity (crlf roundtrip fixtures).

Motivated by the tomlish dual-copy interim (its modpod testsuites were
ported to src/tests while the modpod remains the live test::tomlish
build source - tomlish checkins 20bd00dc, feedb6f0).

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 1 week ago
parent
commit
bfcbf705be
  1. 6
      GOALS.md
  2. 77
      goals/G-029-testmodules-from-srctests.md

6
GOALS.md

@ -216,3 +216,9 @@ Scope: src/modules/punkboot/utils-999999.0a1.0.tm (locker-report helper), src/ma
Detail: goals/G-028-file-locker-identification.md
Goal: when a build cannot delete/overwrite a target file (typically a built executable held open by another program), the failure message names the locking process(es) - via a punkboot::utils helper using the Windows Restart Manager API through optionally-available twapi or cffi, degrading cleanly to the current message when the API or bindings are unavailable or on other platforms.
Acceptance: on Windows with twapi or cffi loadable, a punkboot::utils proc given a file path returns the locking processes (at least pid and process name; empty list when unlocked); punkboot::utils itself stays pure Tcl - the binary binding is required lazily at call time and its absence yields a clean 'unavailable' result, not an error (bootsupport must not gain a compiled-extension dependency); make.tcl kit deploy failures include the locker report when determinable (e.g. "could not delete target binary ... in use by: 7zFM.exe (pid 1234)"); non-Windows platforms and binding-less environments produce the existing message unchanged; verified against a deliberately held handle (documented manual verification acceptable).
### G-029 [proposed] Build packaged test::<modulename> #modpod modules from src/tests
Scope: src/make.tcl (test-module packaging step), src/tests/ (source of truth), src/modules/test/ (generated #modpod targets), src/modules/punk/mix/ (modpod tooling as needed)
Detail: goals/G-029-testmodules-from-srctests.md
Goal: src/tests is the single source of truth for module test suites - a punkcheck-tracked make.tcl step generates the packaged test::<modulename> #modpod modules from src/tests/modules/<namespacepath>/testsuites content, ending hand-maintenance of parallel copies under src/modules/test/; the packaged form is a distributable in its own right - a user who downloads a built module can optionally download the matching test::<modulename> and verify the module's behaviour on their own system (package require + RUN, or an executable's -app test) with no source tree or test harness required.
Acceptance: a make.tcl step generates a test::<modulename> #modpod under src/modules/test/ from the corresponding src/tests testsuites (punkcheck-tracked, skipped when sources unchanged); the generated package works through the packaged path - loadable via package require test::<modulename> and runnable via its SUITE/RUN interface (e.g a built executable's -app test) - reporting the same test names and pass counts as running the same suites directly via src/tests/runtests.tcl; the consumer scenario is proven standalone: the generated package (plus the module under test and their dependencies) runs in an environment without the project source tree present; suite data files (e.g roundtrip toml files with deliberate crlf/mixed line endings) survive packaging byte-for-byte; documentation states src/tests is the source of truth and the generated modpods are build artifacts not to be hand-edited; proven end-to-end for at least one real module (tomlish, whose src/tests port and still-live modpod created the dual-copy situation, is the natural first).

77
goals/G-029-testmodules-from-srctests.md

@ -0,0 +1,77 @@
# G-029 Build packaged test::<modulename> #modpod modules from src/tests
Status: proposed
Scope: src/make.tcl (test-module packaging step), src/tests/ (source of truth), src/modules/test/ (generated #modpod targets), src/modules/punk/mix/ (modpod tooling as needed)
Acceptance: as in root GOALS.md index (canonical).
## Context
Module tests currently exist in two forms:
1. **src/tests/modules/<namespacepath>/testsuites/** - the source-tree harness
form: self-contained tcltest files (own package requires, final
cleanupTests, self-locating data) run via src/tests/runtests.tcl. This is
where test development happens and is the intended structure going forward.
2. **src/modules/test/#modpod-<module>-<ver>/** - the packaged form: a
test::<modulename> module (wrapper .tm providing a SUITE/RUN interface and
locating its bundled *_testsuites) that ships in kits and is run in built
executables, e.g `tomlish.exe -app test`.
The packaged form is not just kit plumbing - it is a **distributable for
module consumers**: a user who downloads a built module can optionally
download the matching test::<modulename> package and verify the module's
behaviour on their own system/platform, with no source tree or test harness -
just `package require test::<modulename>` + its RUN interface, or a wrapped
executable's `-app test`. That end-user scenario is what fixes the packaged
form's requirements: fully self-sufficient (suites + data bundled), locatable
via ordinary module paths, and runnable in a bare interp.
Nothing generates one form from the other, so they drift as parallel
hand-maintained copies. The tomlish project (2026-07-07) is the live example:
its modpod testsuites were ported to src/tests (with fixes -
self-containment, single cleanupTests, self-located data), while the modpod
remains the source the built test::tomlish module actually comes from. Both
are now fossil-tracked and must be kept in sync by hand until this goal lands.
## Approach
- A make.tcl step (natural companion to the existing module build steps)
packages src/tests/modules/<namespacepath>/testsuites/** into the
test::<modulename> #modpod - punkcheck-tracked so unchanged sources skip.
- The wrapper .tm (SUITE/RUN interface, suitesdir resolution) becomes part of
the generation: either templated per module or a shared runtime the modpod
references. Its interface must keep serving the -app test dispatch used by
wrapped executables.
- Because src/tests files are self-contained, the packaged copies can be
byte-identical file copies; the wrapper supplies discovery (suite names from
directory structure) rather than environment variables the files depend on.
Legacy wrapper variables (e.g ::test::tomlish::suitesdir) should not be
reintroduced into test files - self-containment is the contract.
- Data fidelity: suite data (e.g roundtrip toml files with deliberate
crlf/mixed line endings) must be copied byte-for-byte; the #modpod directory
form is plain files, so no archive-level transformation should apply.
- Aggregate suites in the old modpods (fauxlink-based tests/ and dev/
collections) are a generation-time decision: reproduce via fauxlink, replace
with wrapper-level suite selection, or drop - record the choice here.
- Naming: test::<modulename> derives from the src/tests path
(modules/tomlish -> test::tomlish, modules/punk/args -> test::punk::args).
Only modules with testsuites content get a package generated.
## Alternatives considered
- Making the packaged modpod the source of truth and generating src/tests
from it - rejected: src/tests is where development happens (runtests.tcl
ergonomics, agent workflows, per-suite layout) and its self-contained-file
contract is the healthier base; the wrapper form is derivable, the reverse
requires stripping wrapper dependencies (as the tomlish port demonstrated).
- Keeping dual hand-maintained copies - rejected: that is the current drift
the tomlish incident demonstrated; sync-by-discipline does not hold.
## Notes
- Related: the tomlish project's src/tests port (its fossil checkins 20bd00dc,
feedb6f0 record the dual-copy interim and name this goal); punkshell's own
src/modules/test/ packages are further targets once the mechanism exists.
- The packaging step's output is a build artifact by policy even though it
lives under src/ - the punkcheck records make regeneration cheap and
hand-edits detectable.
Loading…
Cancel
Save