3.7 KiB
G-091 runtests multiprocess parallelism (-jobs N)
Status: proposed Scope: src/tests/runtests.tcl, src/tests/testsupport/, src/tests/AGENTS.md Goal: src/tests/runtests.tcl runs multi-process mode's per-file child processes through a -jobs N worker pool (tpool; per-child tmpdirs and output files; longest-first scheduling; a serial group for console-sensitive suites; per-file report sections emitted in discovery order) so the full source-tree suite completes in a fraction of its sequential wall time with unchanged results. Acceptance: -jobs 8 produces aggregate results identical to -jobs 1 (scriptlib/developer/runtests_parity.tcl reports PARITY ok) across repeated full-suite runs on a native tclsh 9.0.x, reduces full-suite wall time by at least 2.5x versus the sequential baseline on the reference machine, leaves the repo clean after runs (git status unchanged), and src/tests/AGENTS.md documents the -jobs flag and the serial-group policy.
Context
Groundwork shipped 2026-07-18 (commit 218fbb1c): -singleproc 0 multi-process mode with a
per-file child bootstrap (testsupport/child_test_runner.tcl + generated environment payload),
plain-exec file-captured child spawn designed for pool reuse, honest nonzero-exit
classification, and the parity harness (scriptlib/developer/runtests_parity.tcl). Full-suite
parity between singleproc and multiproc modes verified same day (87 files / 989 tests;
exec-14.3 the sole expected failure on native tclsh 9.0.3).
Why processes are the parallel unit: test files mutate process-global state (cd in multishell/dtplite/path suites, ::env save/mutate/restore in console suites, thread creation and tsv use, shellrun/shellfilter global channel transforms) - safe across processes, unsafe within one. Suites are already well-behaved across processes (FOSSIL_HOME isolation in fossilmove.test, port-0 ephemeral sockets, tmpdir-scoped fixtures).
Measured shape (native tclsh90, piped context, 2026-07-18): ~350s sequential wall; slowest files multishell.test ~85s, argparsingtest ~27s, dtplite ~25s, scriptexec ~10s. Parallel floor is the slowest single file (~90s incl. overhead) -> expect roughly 3-3.5x at 8 jobs with diminishing returns beyond.
Notes
- Per-child tmpdir subdirs are mandatory under parallelism (shared tcltest -tmpdir would collide on same-named makeFile helpers across concurrent files); phase-1's shared default matches singleproc and must become per-child in pool mode. Same for the per-child stdout/stderr capture file names.
- Serial-group default candidates (relaxable after parity evidence under a real console): modules/punk/console/, modules/opunk/console/, modules/punk/repl/, shell/. In piped contexts these mostly skip anyway.
- Kit-executable caveat carries over from phase 1: children of a kit exe boot with kit-stamped punk packages preloaded (runner warns; native tclsh preferred).
- Candidate independent win (either order): make the 17 preload-dependent files' punk::* package requires explicit so children can drop the shellrun preload + runx warmup and go leaner/faster - the parity tool gates the change. See child_test_runner.tcl comments for the two-stage preload detail (shellrun require + runx execution pulling punk::lib).
- Raising the ceiling beyond the multishell bound means splitting multishell.test (two 30s+ tests) and dtplite.test - suite restructure, optional, floor then ~40s (slowest single test ~32s).
- PUNK_TEST_UDPTEE watch mode (commit
a65f9e5b): under a pool, the parent's per-file relay and lifecycle events still work; the cooked log format's source column plus file-start/ file-end events let one viewer demux interleaved children. - Timing noise: -slowest per-test microseconds get noisier under CPU oversubscription; document rather than fix.