Browse Source
User-approved draft from the G-091 verification data: multishell.test's ~2m child wall is the -jobs floor (full-suite -jobs 8 = 2m16s); splitting the long suites (multishell, dtplite, argparsingtest, scriptexec if worthwhile; exec.test excluded as imported core material) along shared-fixture boundaries targets no child wall above 45s and at least a 25% full-suite improvement, with name-level test conservation, provenance lines travelling with moved tests, weight-table updates, and parity gating. Context records the fixture-duplication tradeoff (total CPU up, wall down) and the complementary lean-children candidate. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.commaster
2 changed files with 56 additions and 0 deletions
@ -0,0 +1,52 @@
|
||||
# G-092 Split long-running test files to lower the parallel floor of broad runs |
||||
|
||||
Status: proposed |
||||
Scope: src/tests/modules/punk/mix/testsuites/scriptwrap/ (multishell.test), src/tests/shell/testsuites/binscripts/ (dtplite.test), src/tests/modules/argparsingtest/testsuites/argparsingtest/ (argparsingtest.test), src/tests/shell/testsuites/punkexe/ (scriptexec.test - if assessed worthwhile), src/tests/runtests.tcl (longest-first weight table), src/tests/AGENTS.md + affected child AGENTS.md |
||||
Goal: no single monolithic test file dominates the -jobs parallel critical path: the long-running suites are split into cohesive smaller files (grouped along shared-fixture boundaries, every test preserved verbatim) so a broad -jobs 8 run's slowest child process is bounded by its slowest cohesive test group rather than a whole legacy file. |
||||
Acceptance: after the split, repeated -jobs 8 full-suite runs on the reference machine report no child process wall above 45s in the runner's slowest-child listing and improve full-suite wall time by at least 25% versus the G-091 verification baseline (2m16s); test conservation is proven and recorded in this file - global tallies identical to a pre-split sequential baseline and the full set of test names unchanged (name-level comparison, mechanism decided in the work - e.g. a names mode for scriptlib/developer/runtests_parity.tcl); no test is deleted, skipped, weakened, or renamed, and provenance comment lines travel with moved tests; the runtests longest-first weight table covers the new files; scriptlib/developer/runtests_parity.tcl reports PARITY ok between -jobs 8 and -jobs 1 on the split tree; affected AGENTS.md contracts (src/tests + shell/binscripts and scriptwrap references) are updated. |
||||
|
||||
## Context |
||||
|
||||
G-091's verification (2026-07-19, archived - goals/archive/G-091-runtests-parallel-jobs.md) |
||||
established the floor anatomy with per-child WALL measurements (the runner now prints them; |
||||
child wall includes boot and file-level fixture cost that per-test usec misses): |
||||
|
||||
- multishell.test child: ~2m wall (85s summed test time - the two ~30s roundtrip/checkfile |
||||
tests plus two ~11s wrap tests - the remainder boot + shared scriptwrap fixtures). This |
||||
single child IS the current -jobs floor: full-suite -jobs 8 wall 2m16s. |
||||
- dtplite.test ~25s (two ~12s tests), argparsingtest.test ~27-30s (largest single test |
||||
7.8s), scriptexec.test ~10-11s. Everything else is small. |
||||
- Post-split ceiling estimate: total child work ~330s / 8 workers ~41s, serial tail ~16s, |
||||
slowest single test ~32s (scriptwrap_runtime_cmd_roundtrip_no_drift) + child boot ~3s -> |
||||
slowest child ~35-40s, full-suite wall ~65-80s expected (acceptance leaves headroom for |
||||
the ~30s run-to-run machine variance observed during G-091). |
||||
|
||||
Split considerations gathered in the same session: |
||||
|
||||
- Fixture duplication tradeoff: multishell's namespace-level fixtures are shared by all its |
||||
tests; each new file re-pays fixture setup and child boot, so total CPU rises as wall |
||||
falls. Split along fixture boundaries (tests sharing a fixture stay together) to bound |
||||
the duplication; the tradeoff is accepted by this goal's premise (wall time of broad |
||||
parallel runs is the optimisation target). |
||||
- exec.test is excluded: it is imported/adapted Tcl core material (src/tests/core contract |
||||
preserves upstream compatibility intent) and its child is only ~5s. |
||||
- src/tests contracts that bind the work: do not delete, skip, or weaken tests; provenance |
||||
comment lines are immutable facts that must move with their tests; child AGENTS.md files |
||||
name multishell.test and dtplite.test in their contracts and need corresponding updates. |
||||
- The runtests longest-first weight table (static globs in runtests.tcl) references the |
||||
pre-split filenames and must be updated with measured weights for the new files. |
||||
- None of the split candidates are in the -serial-paths default, so the serial tail is |
||||
unaffected. |
||||
|
||||
## Notes |
||||
|
||||
- Complementary, independent: the lean-children candidate (explicit package requires for |
||||
the 17 preload-dependent files so children can drop the shellrun preload + runx warmup) |
||||
reduces the per-file boot cost that splitting multiplies - doing it first or alongside |
||||
lowers the split's total-CPU cost. Recorded in goals/archive/G-091 notes. |
||||
- Related: G-029 (packaged test::<modulename> modules built from src/tests) consumes this |
||||
tree as its source of truth - file granularity changes flow into that packaging design. |
||||
- The 45s child-wall criterion is the noise-robust metric (child walls were stable across |
||||
runs while full-suite wall varied up to ~30s); the 25%-vs-2m16s wall criterion is |
||||
relative to the same-machine G-091 baseline rather than an absolute number for the same |
||||
reason. |
||||
Loading…
Reference in new issue