Browse Source

runtests: punk-kit hosts launch children in 'src' package_mode; tally shows Testfiles-failed

Under punk91/punk902z/punk905_beta, '-include-paths modules/punk/console/***
-jobs 16' failed all 7 files with zero tests run: child processes boot via
plain kit script dispatch, preloading kit-stamped punk packages
(punk::console 0.7.2) before the payload's src paths apply - the suites'
pinned 'require punk::console 999999.0a1.0-' then version-conflicts and
the child exits 1 (the known shadowing runtests already warned about).

Fix: when the host is a PUNK kit (kit container heuristics AND the
::punkboot namespace marker from punk_main.tcl boot) children launch as
'kitexe src child_test_runner.tcl ...' so they boot the project's src dev
modules - matching what the payload selects. Non-punk kit containers (e.g
tclsh90szip.exe: zipfs app mount, no punk boot) keep the old warning and
get no extra arg. runtests_run_child_process gains a required exemodeargs
param (tpool info args/body replication unaffected).

Tally: file-level failures are not test results, so a FAIL run could sit
beside 'Failed 0' unexplained (user report). The tally line now appends a
fail-coloured 'Testfiles-failed N' whenever testfiles failed.

Verification: console suites 71/71 under punk905_beta, punk91, punk902z
(previously 0 run / 7 files failed); native Tcl903 + suite tclsh90s
unchanged 71/71 with no modeargs; tclsh90szip warning path passes; tally
demo shows 'Total 0 ... Failed 0 Testfiles-failed 1'.

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 5 days ago
parent
commit
8dde10bbdf
  1. 29
      goals/G-098-suite-tk-tcllib-critcl.md
  2. 37
      src/tests/runtests.tcl

29
goals/G-098-suite-tk-tcllib-critcl.md

@ -233,6 +233,35 @@ user direction 2026-07-20:
built from _vfscommon before this increment) - unaffected for default-console
interactive use (collect path is foreign-console only); picks the fix up at the
next vfscommonupdate + project cycle.
### 2026-07-20 increment 7: kit-hosted runtests fixed ('src' package_mode children) + tally line surfaces file-level failures
- User datapoint: 'runtests.tcl -include-paths modules/punk/console/*** -jobs 16'
FAILED under punk91/punk902z/punk905_beta while the tally said 'Failed 0' beside a
red FAIL. Reproduced: ALL 7 console files failed at file level with ZERO tests run.
Cause: child test processes ('kitexe child_test_runner.tcl ...') boot via plain kit
script dispatch, which preloads kit-stamped punk packages (punk::console 0.7.2)
before the payload's src paths apply - the suites' pinned 'require punk::console
999999.0a1.0-' then version-conflicts and the child exits 1. runtests already
self-warned about exactly this ('prefer a native tclsh').
- FIX: runtests detects a PUNK kit host (kit container heuristics AND ::punkboot
namespace - the punk_main.tcl boot marker) and launches children with the kit's
'src' package_mode ('kitexe src child_test_runner.tcl ...'), so children boot the
project's src dev modules (probe: punk::console 999999.0a1.0 provided, dev require
satisfied, argv/argv0 forwarding intact). Non-punk kit containers (e.g the suite's
tclsh90szip.exe - zipfs app mount, no punk boot) keep the old warning and get NO
extra arg (a plain tclsh would treat 'src' as the script to run).
runtests_run_child_process gains a required exemodeargs param (tpool replication
keeps working via info args/body).
- TALLY: file-level failures (load/setup errors, child process failures) are not
test results, so the human tally could show 'Failed 0' on a FAIL run. The tally
line now appends 'Testfiles-failed N' (fail-coloured) whenever files failed:
'Total 0 Passed 0 Skipped 0 Failed 0 Testfiles-failed 1'. RUNTESTS_RESULT already
carried files_failed - unchanged.
- VERIFICATION: console suites 71/71 PASS under punk905_beta, punk91, punk902z (all
previously 7 files failed / 0 run), native Tcl903 and suite tclsh90s unchanged
71/71 (no modeargs, no notes), tclsh90szip takes the non-punk warning path and
passes; tally demo via throwaway error-at-load file shows Testfiles-failed 1.
- (superseded note kept for history below: the clone store subsequently moved from
the suite-owned _build location to the machine-level ~/.punkshell/fossils - see
increment 1 and the suite README for current truth)

37
src/tests/runtests.tcl

@ -321,12 +321,14 @@ proc runtests_write_child_payload {payloadfile test_tmlist test_auto_path ifneed
close $fd
}
proc runtests_run_child_process {exe bootstrap payloadfile testfile outfile errfile childtmpdir} {
proc runtests_run_child_process {exe exemodeargs bootstrap payloadfile testfile outfile errfile childtmpdir} {
#Run one test file in a child process, capturing stdout/stderr to files. Deliberately no
#shellrun/shellfilter capture on the parent side: file capture avoids the parent-stdin/pipe
#machinery, and -jobs parallel scheduling reuses the same mechanism (this proc is
#replicated verbatim into tpool worker threads via info args/info body - keep it free of
#package dependencies and namespace/global state, and keep every argument required).
#exemodeargs: extra argv between exe and bootstrap ({} normally; {src} under a punk kit
#so the child boots the project's src dev modules instead of kit-stamped packages).
#childtmpdir overrides the child's tcltest -tmpdir when nonempty (mandatory per child
#under -jobs so concurrent makeFile/makeDirectory helpers cannot collide); "" keeps the
#shared -tmpdir carried in the payload options.
@ -360,7 +362,7 @@ proc runtests_run_child_process {exe bootstrap payloadfile testfile outfile errf
#errors in early -jobs runs, 2026-07-19). With autopurge off each pid is purged
#explicitly after its status is read.
tcl::process autopurge false
if {[catch {exec -- $exe $bootstrap $payloadfile $testfile $childtmpdir < $nulldev > $outfile 2> $errfile &} spawnresult errdict]} {
if {[catch {exec -- $exe {*}$exemodeargs $bootstrap $payloadfile $testfile $childtmpdir < $nulldev > $outfile 2> $errfile &} spawnresult errdict]} {
#launch failure - no usable test run
return [dict create error $spawnresult errorCode [dict get $errdict -errorcode] errorInfo [dict get $errdict -errorinfo]]
}
@ -392,7 +394,7 @@ proc runtests_run_child_process {exe bootstrap payloadfile testfile outfile errf
#blocking fallback for runtimes without tcl::process (tcl 8.6): correct sequentially;
#concurrent use from -jobs workers re-convoys on Windows (documented in
#src/tests/AGENTS.md)
if {[catch {exec -- $exe $bootstrap $payloadfile $testfile $childtmpdir < $nulldev > $outfile 2> $errfile} emsg errdict]} {
if {[catch {exec -- $exe {*}$exemodeargs $bootstrap $payloadfile $testfile $childtmpdir < $nulldev > $outfile 2> $errfile} emsg errdict]} {
set ecode [dict get $errdict -errorcode]
if {[lindex $ecode 0] eq "CHILDSTATUS"} {
set exitcode [lindex $ecode 2]
@ -887,8 +889,19 @@ if {!$report_json_only} {
#tcltest options) so children select the same unbuilt dev modules as the singleproc testinterp.
set runtests_workdir ""
if {!$singleproc} {
set child_exe_modeargs [list]
if {[llength $kit_lib_bases]} {
puts stderr "WARNING: -singleproc 0 under a kit executable: child processes boot via the kit's script dispatch with kit-stamped punk packages already loaded, which can shadow the src dev modules under test. Prefer a native tclsh for multi-process runs."
if {[namespace exists ::punkboot]} {
#punk kit (punk_main.tcl boot): plain script dispatch boots kit-stamped punk
#packages into the child (e.g punk::console 0.7.2) BEFORE the payload's src
#paths apply, so suite requires pinned 999999.0a1.0- hit version conflicts.
#The kit's 'src' package_mode boots children on the project's src dev modules
#instead - matching what the payload selects (G-098 kit-hosted runs 2026-07-20).
set child_exe_modeargs [list src]
puts stderr "NOTE: punk kit executable - child test processes launch with the kit's 'src' package_mode so src dev modules boot (not kit-stamped packages)."
} else {
puts stderr "WARNING: -singleproc 0 under a non-punk kit executable: child processes boot with the kit's bundled packages which can shadow the src dev modules under test. Prefer a native tclsh for multi-process runs."
}
}
set child_bootstrap [file join $test_base testsupport child_test_runner.tcl]
if {![file exists $child_bootstrap]} {
@ -1045,7 +1058,7 @@ if {$jobs_mode} {
set c_err [file join $runtests_workdir child_stderr_$i.txt]
set c_tmp [file join $childtmp_base childtmp_$i]
file mkdir $c_tmp
set jid [tpool::post $pool [list runtests_run_child_process $thisexecutable $child_bootstrap $child_payloadfile $tfabs $c_out $c_err $c_tmp]]
set jid [tpool::post $pool [list runtests_run_child_process $thisexecutable $child_exe_modeargs $child_bootstrap $child_payloadfile $tfabs $c_out $c_err $c_tmp]]
dict set jobmap $jid $tf
}
set pending [dict keys $jobmap]
@ -1077,7 +1090,7 @@ if {$jobs_mode} {
set c_err [file join $runtests_workdir child_stderr_$i.txt]
set c_tmp [file join $childtmp_base childtmp_$i]
file mkdir $c_tmp
set jobresult [runtests_run_child_process $thisexecutable $child_bootstrap $child_payloadfile $tfabs $c_out $c_err $c_tmp]
set jobresult [runtests_run_child_process $thisexecutable $child_exe_modeargs $child_bootstrap $child_payloadfile $tfabs $c_out $c_err $c_tmp]
if {$report_detailed_markdown} {
puts stdout "serial completed: $tf"
flush stdout
@ -1183,7 +1196,7 @@ foreach testfile_relative $testfiles {
#file-collected event happened at collection time
set result $runtests_job_results($testfile_relative)
} else {
set result [runtests_run_child_process $thisexecutable $child_bootstrap $child_payloadfile $testfile $child_outfile $child_errfile ""]
set result [runtests_run_child_process $thisexecutable $child_exe_modeargs $child_bootstrap $child_payloadfile $testfile $child_outfile $child_errfile ""]
if {$runtests_udptee_target ne ""} {
#relay the child's captured output to the watch target (file granularity -
#children do not stream live; see src/tests/AGENTS.md)
@ -1196,7 +1209,7 @@ foreach testfile_relative $testfiles {
}
}
if {$report_detailed_markdown} {
puts stdout "executed $thisexecutable testsupport/child_test_runner.tcl $testfile_relative "
puts stdout "executed $thisexecutable[expr {[llength $child_exe_modeargs] ? " $child_exe_modeargs" : ""}] testsupport/child_test_runner.tcl $testfile_relative "
}
flush stdout
}
@ -1508,7 +1521,13 @@ if {!$report_json_only} {
} else {
set tally_passed [runtests_clr pass $tally_passed]
}
puts stdout "Total [dict get $tallydict total] $tally_passed Skipped [dict get $tallydict skipped] $tally_failed "
set tally_line "Total [dict get $tallydict total] $tally_passed Skipped [dict get $tallydict skipped] $tally_failed"
if {[llength [dict get $tallydict files_with_fails]]} {
#file-level failures (load/setup errors, child process failures) are not test
#results - surface them here so a red FAIL can't sit beside 'Failed 0' unexplained
append tally_line " " [runtests_clr fail "Testfiles-failed [llength [dict get $tallydict files_with_fails]]"]
}
puts stdout "$tally_line "
puts stdout ""
}

Loading…
Cancel
Save