From 469ecf67d9abcc7921c9cd00d08fd8eadaf7b639 Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Sun, 19 Jul 2026 04:50:55 +1000 Subject: [PATCH] GOALS: add G-092 - split long-running test files (parallel-floor reduction) User-approved draft from the G-091 verification data: multishell.test's ~2m child wall is the -jobs floor (full-suite -jobs 8 = 2m16s); splitting the long suites (multishell, dtplite, argparsingtest, scriptexec if worthwhile; exec.test excluded as imported core material) along shared-fixture boundaries targets no child wall above 45s and at least a 25% full-suite improvement, with name-level test conservation, provenance lines travelling with moved tests, weight-table updates, and parity gating. Context records the fixture-duplication tradeoff (total CPU up, wall down) and the complementary lean-children candidate. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- GOALS.md | 4 ++ goals/G-092-split-longrunning-testfiles.md | 52 ++++++++++++++++++++++ 2 files changed, 56 insertions(+) create mode 100644 goals/G-092-split-longrunning-testfiles.md diff --git a/GOALS.md b/GOALS.md index f1178197..29985c1c 100644 --- a/GOALS.md +++ b/GOALS.md @@ -341,3 +341,7 @@ Detail: goals/G-089-scriptlib-kits-and-modes.md ### G-090 [proposed] shellfilter/shellthread stack and log-worker lifecycle audit Scope: src/modules/shellfilter-999999.0a1.0.tm (stack placement, insert_transform, junctions), src/modules/shellthread-999999.0a1.0.tm (worker park/adoption/termination), src/tests/modules/ (new characterization suites), README.md ('very unripe parts' note) Detail: goals/G-090-shellfilter-shellthread-audit.md + +### G-092 [proposed] Split long-running test files to lower the parallel floor of broad runs +Scope: src/tests/modules/punk/mix/testsuites/scriptwrap/ (multishell.test), src/tests/shell/testsuites/binscripts/ (dtplite.test), src/tests/modules/argparsingtest/testsuites/argparsingtest/ (argparsingtest.test), src/tests/shell/testsuites/punkexe/ (scriptexec.test - if assessed worthwhile), src/tests/runtests.tcl (longest-first weight table), src/tests/AGENTS.md + affected child AGENTS.md +Detail: goals/G-092-split-longrunning-testfiles.md diff --git a/goals/G-092-split-longrunning-testfiles.md b/goals/G-092-split-longrunning-testfiles.md new file mode 100644 index 00000000..79bd96cc --- /dev/null +++ b/goals/G-092-split-longrunning-testfiles.md @@ -0,0 +1,52 @@ +# G-092 Split long-running test files to lower the parallel floor of broad runs + +Status: proposed +Scope: src/tests/modules/punk/mix/testsuites/scriptwrap/ (multishell.test), src/tests/shell/testsuites/binscripts/ (dtplite.test), src/tests/modules/argparsingtest/testsuites/argparsingtest/ (argparsingtest.test), src/tests/shell/testsuites/punkexe/ (scriptexec.test - if assessed worthwhile), src/tests/runtests.tcl (longest-first weight table), src/tests/AGENTS.md + affected child AGENTS.md +Goal: no single monolithic test file dominates the -jobs parallel critical path: the long-running suites are split into cohesive smaller files (grouped along shared-fixture boundaries, every test preserved verbatim) so a broad -jobs 8 run's slowest child process is bounded by its slowest cohesive test group rather than a whole legacy file. +Acceptance: after the split, repeated -jobs 8 full-suite runs on the reference machine report no child process wall above 45s in the runner's slowest-child listing and improve full-suite wall time by at least 25% versus the G-091 verification baseline (2m16s); test conservation is proven and recorded in this file - global tallies identical to a pre-split sequential baseline and the full set of test names unchanged (name-level comparison, mechanism decided in the work - e.g. a names mode for scriptlib/developer/runtests_parity.tcl); no test is deleted, skipped, weakened, or renamed, and provenance comment lines travel with moved tests; the runtests longest-first weight table covers the new files; scriptlib/developer/runtests_parity.tcl reports PARITY ok between -jobs 8 and -jobs 1 on the split tree; affected AGENTS.md contracts (src/tests + shell/binscripts and scriptwrap references) are updated. + +## Context + +G-091's verification (2026-07-19, archived - goals/archive/G-091-runtests-parallel-jobs.md) +established the floor anatomy with per-child WALL measurements (the runner now prints them; +child wall includes boot and file-level fixture cost that per-test usec misses): + +- multishell.test child: ~2m wall (85s summed test time - the two ~30s roundtrip/checkfile + tests plus two ~11s wrap tests - the remainder boot + shared scriptwrap fixtures). This + single child IS the current -jobs floor: full-suite -jobs 8 wall 2m16s. +- dtplite.test ~25s (two ~12s tests), argparsingtest.test ~27-30s (largest single test + 7.8s), scriptexec.test ~10-11s. Everything else is small. +- Post-split ceiling estimate: total child work ~330s / 8 workers ~41s, serial tail ~16s, + slowest single test ~32s (scriptwrap_runtime_cmd_roundtrip_no_drift) + child boot ~3s -> + slowest child ~35-40s, full-suite wall ~65-80s expected (acceptance leaves headroom for + the ~30s run-to-run machine variance observed during G-091). + +Split considerations gathered in the same session: + +- Fixture duplication tradeoff: multishell's namespace-level fixtures are shared by all its + tests; each new file re-pays fixture setup and child boot, so total CPU rises as wall + falls. Split along fixture boundaries (tests sharing a fixture stay together) to bound + the duplication; the tradeoff is accepted by this goal's premise (wall time of broad + parallel runs is the optimisation target). +- exec.test is excluded: it is imported/adapted Tcl core material (src/tests/core contract + preserves upstream compatibility intent) and its child is only ~5s. +- src/tests contracts that bind the work: do not delete, skip, or weaken tests; provenance + comment lines are immutable facts that must move with their tests; child AGENTS.md files + name multishell.test and dtplite.test in their contracts and need corresponding updates. +- The runtests longest-first weight table (static globs in runtests.tcl) references the + pre-split filenames and must be updated with measured weights for the new files. +- None of the split candidates are in the -serial-paths default, so the serial tail is + unaffected. + +## Notes + +- Complementary, independent: the lean-children candidate (explicit package requires for + the 17 preload-dependent files so children can drop the shellrun preload + runx warmup) + reduces the per-file boot cost that splitting multiplies - doing it first or alongside + lowers the split's total-CPU cost. Recorded in goals/archive/G-091 notes. +- Related: G-029 (packaged test:: modules built from src/tests) consumes this + tree as its source of truth - file granularity changes flow into that packaging design. +- The 45s child-wall criterion is the noise-robust metric (child walls were stable across + runs while full-suite wall varied up to ~30s); the 25%-vs-2m16s wall criterion is + relative to the same-machine G-091 baseline rather than an absolute number for the same + reason.