src/tests/AGENTS.md: -jobs guidance updated with measured scaling curve
User-measured reference-machine numbers 2026-07-19: -jobs 8 ~79-86s,
-jobs 16 ~61s (~5.6x vs ~5m40s sequential), plateau by -jobs 24 (~60s).
Guidance now recommends -jobs 16 and records the plateau anatomy
(slowest single child ~31s runtimecmd pair, serial console tail ~16s
piped / larger under a real console, ~10s runner overhead).
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
@ -47,7 +47,7 @@ Top-level test harness and source-tree tests for ShellSpy/Punk. Tests here exerc
- Add `-slowest <n>` when timing outliers are relevant.
- Multi-process runs use `-tcltestoptions {-singleproc 0}` with otherwise identical flags. Check result parity between modes by capturing `-report json` stdout from each and comparing with `tclsh scriptlib/developer/runtests_parity.tcl <a.json> <b.json>` (ignores timings; exit 0 on parity).
- To watch runs live (including runs launched by agents or other processes, which inherit the environment): set `PUNK_TEST_UDPTEE=41197` and point a UDP listener/viewer at that port. No runner flags needed; results are unaffected.
- For broad runs, `-jobs 8` cuts full-suite wall time roughly 4x (~1m26s vs ~5m40s sequential on the reference machine, 2026-07-19 post G-092 split; result parity verified) - e.g `<tcl_interpreter> src/tests/runtests.tcl -jobs 8 -report compact -show-passes 0`. No child process exceeds ~31s; the wall is now work-distribution bound rather than dominated by any single file. Focused runs rarely need `-jobs`.
- For broad runs use `-jobs` - e.g `<tcl_interpreter> src/tests/runtests.tcl -jobs 16 -report compact -show-passes 0`. Reference-machine full-suite scaling (2026-07-19, post G-092 split, result parity verified at each level): sequential ~5m40s, `-jobs 8` ~79-86s, `-jobs 16` ~61s (~5.6x), plateau by `-jobs 24` (~60s) - beyond 16 there is nothing left to overlap: the wall decomposes into the slowest single child (~31s, the runtimecmd pair - single tests, not further splittable), the serial console tail (~16s piped; larger under a real console where those suites actually run), and runner overhead (~10s). `-jobs 16` is the reference-machine sweet spot; focused runs rarely need `-jobs`.
- Add `-strict-exit 1` when a nonzero shell exit code is needed for failures or parser warnings.
- Capture enough stderr or failure context to identify the failing command or assertion.
- For ERROR-status failures, the markdown report's `errorInfo` block and compact `message=` field carry the full Tcl error message; use `-report markdown` for untruncated context.