punkshell/goals/G-161-tcltestrun-multiline-banner.md

G-161 tcltestrun failure parsing robust to multi-line test descriptions

Status: active Scope: src/modules/punk/tcltestrun-999999.0a1.0.tm (parse_testrun failure-banner state machine); src/tests/runner/ (parser characterization suite - synthetic captured-output + end-to-end failing-fixture coverage); src/tests/AGENTS.md (single-line-description contract note relaxed to style guidance once tolerated) Goal: a failing tcltest whose description contains embedded newlines is reported with full fidelity - failure event with name and description, result_was/result_expected, and every subsequent event including the summary line - instead of the parser missing the multi-line opening banner, misreading the closing banner as an opener, and wedging in failure-capture state so the file degrades to warn/missing-cleanupTests with no failure detail. Acceptance: a committed runner suite covers both a synthetic captured-output fixture (multi-line-description failure followed by passes and a summary line, parsed to the correct event set) and an end-to-end fixture .test file whose deliberately failing multi-line-description test is reported by runtests.tcl as status=FAILED with actual/expected populated and a trusted summary (summaryline_detected=1, correct totals) in BOTH single-process and multi-process modes; single-line-description failure reports keep their existing shape, verified by scriptlib/developer/runtests_parity.tcl comparing -report json output of an identical full-suite run before and after the change; the src/tests/AGENTS.md single-line-description bullet is updated to style guidance reflecting the new tolerance.

Context

Discovered 2026-08-03 while authoring the commandstack characterisation suite. tcltest prints a failing test's opening banner as ONE puts of "==== FAILED" and only trims the description's ENDS, so a description with embedded newlines makes the banner span lines. punk::tcltestrun's per-line scanner recognises an opening banner only as a line both starting "==== " and ending " FAILED": the multi-line opener never matches, the later single-line closing banner "==== FAILED" is then treated as an OPENER, and the parser waits forever for a "==== Contents of test case:" line that already went past - swallowing every subsequent event, including the pass events of later tests and the cleanupTests summary line. The file then reports warn/missing-cleanupTests with observed passes stopping at the failure and no failure detail at all. The trap is recorded as a hard contract in src/tests/AGENTS.md (descriptions must be single content line); this goal makes the parser tolerant so the contract can stand down to a style preference.

Approach

Two candidate mechanisms, settled in the work:

• Banner buffering: a line starting "==== " that does not end " FAILED" opens a bounded lookahead buffer joined until a line ending " FAILED" completes the banner (or the buffer is abandoned as ordinary output).
• Start-event anchoring: the runner always passes -verbose start, so "---- start" precedes every test; carrying the current test name makes the closing-banner match exact and the opening banner recognisable by prefix alone. Either way the existing state machine's confounding-output heuristics (emulated banner lines inside test output) must keep their current behaviour for single-line cases.

Notes

• Overlap survey 2026-08-03: goals_xref paths over src/modules/punk/tcltestrun-999999.0a1.0.tm, src/tests/runner and src/tests/runtests.tcl found no intersecting live goals.
• Activation-freshness survey 2026-08-03 (goals_xref score G-161, run at the user-directed activation): no related live goals - every overlap pair is unlinked with zero path intersection, lexical noise only (top scores: G-085 0.55 cmdtrace, G-162 0.43 bakelist freshness, G-014 0.42 config toml; the interval-drafted G-163 tty drain 0.39 is likewise unrelated). Reconfirms the drafting-time survey; no Related: lines warranted.
• Reproduction record: the commandstack suite's pre-rewrite failure mode (24-of-29 observed passes, no failure event, no summary) in both runner modes, 2026-08-03; zipper.test's single-line-description failure reported correctly under the same conditions - the discriminating experiment.
• punk::tcltestrun is also consumed by kit-baked copies; a mint (and later promotion) carries the fix to kits per the normal channels.