From bfcbf705be8db7c5de2ffafb544f01bb934bdabd Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Tue, 7 Jul 2026 01:01:56 +1000 Subject: [PATCH] add goal G-029: build packaged test:: modpods from src/tests src/tests becomes the single source of truth for module test suites: a punkcheck-tracked make.tcl step generates the packaged test:: #modpod modules from src/tests/modules//testsuites, ending hand-maintained parallel copies under src/modules/test/. The packaged form is a distributable in its own right: a user who downloads a built module can optionally download the matching test:: and verify the module''s behaviour on their own system - package require + RUN interface, or a wrapped executable''s -app test - with no source tree or harness. Acceptance proves the standalone consumer scenario (runs without the project source tree), suite-result parity with src/tests/runtests.tcl, and byte-for-byte data fidelity (crlf roundtrip fixtures). Motivated by the tomlish dual-copy interim (its modpod testsuites were ported to src/tests while the modpod remains the live test::tomlish build source - tomlish checkins 20bd00dc, feedb6f0). Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- GOALS.md | 6 ++ goals/G-029-testmodules-from-srctests.md | 77 ++++++++++++++++++++++++ 2 files changed, 83 insertions(+) create mode 100644 goals/G-029-testmodules-from-srctests.md diff --git a/GOALS.md b/GOALS.md index 32dfab57..b62a61db 100644 --- a/GOALS.md +++ b/GOALS.md @@ -216,3 +216,9 @@ Scope: src/modules/punkboot/utils-999999.0a1.0.tm (locker-report helper), src/ma Detail: goals/G-028-file-locker-identification.md Goal: when a build cannot delete/overwrite a target file (typically a built executable held open by another program), the failure message names the locking process(es) - via a punkboot::utils helper using the Windows Restart Manager API through optionally-available twapi or cffi, degrading cleanly to the current message when the API or bindings are unavailable or on other platforms. Acceptance: on Windows with twapi or cffi loadable, a punkboot::utils proc given a file path returns the locking processes (at least pid and process name; empty list when unlocked); punkboot::utils itself stays pure Tcl - the binary binding is required lazily at call time and its absence yields a clean 'unavailable' result, not an error (bootsupport must not gain a compiled-extension dependency); make.tcl kit deploy failures include the locker report when determinable (e.g. "could not delete target binary ... in use by: 7zFM.exe (pid 1234)"); non-Windows platforms and binding-less environments produce the existing message unchanged; verified against a deliberately held handle (documented manual verification acceptable). + +### G-029 [proposed] Build packaged test:: #modpod modules from src/tests +Scope: src/make.tcl (test-module packaging step), src/tests/ (source of truth), src/modules/test/ (generated #modpod targets), src/modules/punk/mix/ (modpod tooling as needed) +Detail: goals/G-029-testmodules-from-srctests.md +Goal: src/tests is the single source of truth for module test suites - a punkcheck-tracked make.tcl step generates the packaged test:: #modpod modules from src/tests/modules//testsuites content, ending hand-maintenance of parallel copies under src/modules/test/; the packaged form is a distributable in its own right - a user who downloads a built module can optionally download the matching test:: and verify the module's behaviour on their own system (package require + RUN, or an executable's -app test) with no source tree or test harness required. +Acceptance: a make.tcl step generates a test:: #modpod under src/modules/test/ from the corresponding src/tests testsuites (punkcheck-tracked, skipped when sources unchanged); the generated package works through the packaged path - loadable via package require test:: and runnable via its SUITE/RUN interface (e.g a built executable's -app test) - reporting the same test names and pass counts as running the same suites directly via src/tests/runtests.tcl; the consumer scenario is proven standalone: the generated package (plus the module under test and their dependencies) runs in an environment without the project source tree present; suite data files (e.g roundtrip toml files with deliberate crlf/mixed line endings) survive packaging byte-for-byte; documentation states src/tests is the source of truth and the generated modpods are build artifacts not to be hand-edited; proven end-to-end for at least one real module (tomlish, whose src/tests port and still-live modpod created the dual-copy situation, is the natural first). diff --git a/goals/G-029-testmodules-from-srctests.md b/goals/G-029-testmodules-from-srctests.md new file mode 100644 index 00000000..1ea767f2 --- /dev/null +++ b/goals/G-029-testmodules-from-srctests.md @@ -0,0 +1,77 @@ +# G-029 Build packaged test:: #modpod modules from src/tests + +Status: proposed +Scope: src/make.tcl (test-module packaging step), src/tests/ (source of truth), src/modules/test/ (generated #modpod targets), src/modules/punk/mix/ (modpod tooling as needed) +Acceptance: as in root GOALS.md index (canonical). + +## Context + +Module tests currently exist in two forms: + +1. **src/tests/modules//testsuites/** - the source-tree harness + form: self-contained tcltest files (own package requires, final + cleanupTests, self-locating data) run via src/tests/runtests.tcl. This is + where test development happens and is the intended structure going forward. +2. **src/modules/test/#modpod--/** - the packaged form: a + test:: module (wrapper .tm providing a SUITE/RUN interface and + locating its bundled *_testsuites) that ships in kits and is run in built + executables, e.g `tomlish.exe -app test`. + +The packaged form is not just kit plumbing - it is a **distributable for +module consumers**: a user who downloads a built module can optionally +download the matching test:: package and verify the module's +behaviour on their own system/platform, with no source tree or test harness - +just `package require test::` + its RUN interface, or a wrapped +executable's `-app test`. That end-user scenario is what fixes the packaged +form's requirements: fully self-sufficient (suites + data bundled), locatable +via ordinary module paths, and runnable in a bare interp. + +Nothing generates one form from the other, so they drift as parallel +hand-maintained copies. The tomlish project (2026-07-07) is the live example: +its modpod testsuites were ported to src/tests (with fixes - +self-containment, single cleanupTests, self-located data), while the modpod +remains the source the built test::tomlish module actually comes from. Both +are now fossil-tracked and must be kept in sync by hand until this goal lands. + +## Approach + +- A make.tcl step (natural companion to the existing module build steps) + packages src/tests/modules//testsuites/** into the + test:: #modpod - punkcheck-tracked so unchanged sources skip. +- The wrapper .tm (SUITE/RUN interface, suitesdir resolution) becomes part of + the generation: either templated per module or a shared runtime the modpod + references. Its interface must keep serving the -app test dispatch used by + wrapped executables. +- Because src/tests files are self-contained, the packaged copies can be + byte-identical file copies; the wrapper supplies discovery (suite names from + directory structure) rather than environment variables the files depend on. + Legacy wrapper variables (e.g ::test::tomlish::suitesdir) should not be + reintroduced into test files - self-containment is the contract. +- Data fidelity: suite data (e.g roundtrip toml files with deliberate + crlf/mixed line endings) must be copied byte-for-byte; the #modpod directory + form is plain files, so no archive-level transformation should apply. +- Aggregate suites in the old modpods (fauxlink-based tests/ and dev/ + collections) are a generation-time decision: reproduce via fauxlink, replace + with wrapper-level suite selection, or drop - record the choice here. +- Naming: test:: derives from the src/tests path + (modules/tomlish -> test::tomlish, modules/punk/args -> test::punk::args). + Only modules with testsuites content get a package generated. + +## Alternatives considered + +- Making the packaged modpod the source of truth and generating src/tests + from it - rejected: src/tests is where development happens (runtests.tcl + ergonomics, agent workflows, per-suite layout) and its self-contained-file + contract is the healthier base; the wrapper form is derivable, the reverse + requires stripping wrapper dependencies (as the tomlish port demonstrated). +- Keeping dual hand-maintained copies - rejected: that is the current drift + the tomlish incident demonstrated; sync-by-discipline does not hold. + +## Notes + +- Related: the tomlish project's src/tests port (its fossil checkins 20bd00dc, + feedb6f0 record the dual-copy interim and name this goal); punkshell's own + src/modules/test/ packages are further targets once the mechanism exists. +- The packaging step's output is a build artifact by policy even though it + lives under src/ - the punkcheck records make regeneration cheap and + hand-edits detectable.