From a65f9e5b67188b7aa5cec7031fb62ffcab6477ed Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Sat, 18 Jul 2026 23:19:51 +1000 Subject: [PATCH] shellrun 0.2.0 runx -teelog + runtests UDP watch mode (PUNK_TEST_UDPTEE) runx -teelog {-tag/-syslog/-file/-raw settings}: live tee of the captured stdout/stderr to shellfilter::log workers (log sources stdout/stderr) while captured/returned data is unchanged. Implemented as write traces on the capture variables rather than extra channel transforms: probing showed a tee_to_log float combined with the float-locked capture junction is starved or loses its worker settings through add/remove churn (float-placement logic is unaudited - README 'very unripe parts'), and shellthread's parked-worker adoption does not re-apply settings (writes silently misroute to a previous target). Both findings deliberately not addressed here - recorded for a proposed shellfilter/shellthread audit goal. Tag workers stay open for reuse; conflicting reconfiguration errors explicitly. runtests.tcl watch mode: env PUNK_TEST_UDPTEE=|: streams runner lifecycle events (tag 'runtests', tcl-list RUNTESTS-EVENT messages: run-start/file-start/file-end/run-end with tallies) and test output (tags teststdout/teststderr) to a local UDP listener via the same mechanism - live per chunk in singleproc mode (runx -teelog inside the testinterp), relayed per completed file by the parent in multiproc mode. Watch mode terminates the per-testinterp log worker threads after each file; unwatched runs are unchanged (including the pre-existing teehandle-worker leak, also noted for the audit). The parent extends auto_path with /lib_tcl/ so workers load tcludp 1.0.13+ (1.0.12 UDP_ExitProc = G-036 exit-hang class). Verified on native tclsh 9.0.3: full-suite result parity (87 files, 989 tests, exec-14.3 sole failure) for watch-off and watch-on singleproc runs vs the pre-change baseline; focused watch runs in both modes captured over UDP (events + live/relayed output). Viewer-side caveat documented: tcludp's receive fileevent under Tcl 9.0.3/Windows delivered only the first datagram (send side unaffected; .NET UdpClient received all). Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- CHANGELOG.md | 5 ++ punkproject.toml | 2 +- src/modules/shellrun-999999.0a1.0.tm | 63 +++++++++++++++ src/modules/shellrun-buildversion.txt | 3 +- src/tests/AGENTS.md | 3 + src/tests/runtests.tcl | 109 +++++++++++++++++++++++++- 6 files changed, 182 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 28ca23d4..717e1007 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,11 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` "Project Versioning" section for the bump policy. +## [0.15.0] - 2026-07-18 + +- shellrun 0.2.0: `runx` gains `-teelog` - a live tee of the captured stdout/stderr to the shellfilter::log mechanism (shellthread worker threads; `-tag`/`-syslog host:port`/`-file`/`-raw` settings; log sources `stdout`/`stderr`) while the captured/returned data stays unchanged. Implemented as write traces on the capture variables rather than additional channel transforms: probing showed the unaudited float-placement logic can starve a tee_to_log float combined with the float-locked capture junction, and shellthread's parked-worker adoption does not re-apply settings (logs silently misrouted to a previous target) - both findings recorded for a future shellfilter/shellthread audit. Tag workers are left open for reuse across calls; a conflicting reconfiguration errors explicitly. +- runtests.tcl UDP watch mode (test tooling, rides this release): env `PUNK_TEST_UDPTEE=|:` streams runner lifecycle events (tag `runtests`, tcl-list `RUNTESTS-EVENT` messages) and test output (tags `teststdout`/`teststderr`) to a local UDP listener - live per output chunk in single-process mode via `runx -teelog`, per completed file in multi-process mode. In watch mode the singleproc loop also terminates the shellfilter::log worker threads spawned inside each per-file testinterp (unwatched runs keep the pre-existing small teehandle-worker leak rather than pay ~0.6s/file terminate handshakes - noted for the shellfilter/shellthread audit). Full-suite result parity with the tee on and off verified via `scriptlib/developer/runtests_parity.tcl`. + ## [0.14.0] - 2026-07-18 - `lib:` scriptlib scripts may now be extensionless: an extensionless (or unknown-extension) `lib:` call also matches a file named exactly `` when its first lines identify it as tcl - a leading `# tcl` comment, a shebang naming tclsh/tclkit/wish, or the portable sh-trampoline (`#!/bin/sh` with a commented `exec tclsh` continuation). When both `` and `.tcl` exist, the spelling matching the call wins, checked per scriptlib location so kit-internal precedence is unaffected. Implemented in punk::path 0.3.0 (`scriptlib_resolve` + new internal `scriptfile_is_tcl`; result dict gains `scripttype`/`notes`, not-found errors list skip notes); `.kit` is now a known extension so `lib:.kit` resolves as itself (previously it wrongly searched `.kit.tcl`, making the shell subcommand's kit dispatch unreachable - now restored). The `shell` subcommand's `lib:` handling (app-punkshell, app_shellrun) is refactored onto the shared resolver, fixing two silent-failure bugs in its former inline copy (no message when a scriptlib dir existed but lacked the script; wrong kit-internal path in the error listing). Test fixtures + coverage added under `scriptlib/_punktest/` and `src/tests/shell/testsuites/punkexe/scriptexec.test`. diff --git a/punkproject.toml b/punkproject.toml index 8d55d47f..12618870 100644 --- a/punkproject.toml +++ b/punkproject.toml @@ -1,4 +1,4 @@ [project] name = "punkshell" -version = "0.14.0" +version = "0.15.0" license = "BSD-2-Clause" diff --git a/src/modules/shellrun-999999.0a1.0.tm b/src/modules/shellrun-999999.0a1.0.tm index 6bf54b0e..c3bdf89b 100644 --- a/src/modules/shellrun-999999.0a1.0.tm +++ b/src/modules/shellrun-999999.0a1.0.tm @@ -624,6 +624,19 @@ namespace eval shellrun { "Enable debug diagnostics from shellfilter::run." --timeout= -type integer -help\ "Timeout in milliseconds for the external process." + -teelog -type dict -help\ + "Log-target settings dict enabling a live tee of the captured stdout and + stderr to the shellfilter::log mechanism (shellthread workers) while the + command runs. Keys: -tag (base tag, default 'shellrun' - the log sources + become stdout and stderr), -syslog host:port (UDP target, e.g + 127.0.0.1:514), -file path, -raw 0|1. With neither -syslog nor -file the + workers are noops. Streaming observes the capture variables via write + traces, so captured/returned data is unchanged. The tag workers are left + open for reuse by later calls: a later -teelog with the same tag and + settings shares them, a conflicting configuration errors explicitly. + Release with shellfilter::log::close ; short-lived interps should + then also run shellthread::manager::shutdown_free_threads before + deletion or the parked worker threads are orphaned." @values -min 1 -max -1 cmdname -type string -help\ "Name of the external command (or Tcl command when -tcl is @@ -631,6 +644,22 @@ namespace eval shellrun { cmdarg -type any -multiple 1 -optional 1 -help\ "Additional arguments passed through to the command." }] + #write-trace callback for runx -teelog: stream the newly appended portion of a capture + #variable to the tag's shellfilter::log worker. The capture transforms (var/tee_to_var) + #only ever append to the variable during a run, so the delta since the last callback is + #exactly the new data. Never let an error escape - a trace error would surface as a + #write error on the transformed channel. + proc teelog_stream {tag varname args} { + variable teelog_sent + catch { + set data [set $varname] + set new [string range $data $teelog_sent($tag) end] + set teelog_sent($tag) [string length $data] + if {$new ne ""} { + ::shellfilter::log::write $tag $new + } + } + } proc runx {args} { #set_last_run_display [list] variable runout @@ -664,6 +693,34 @@ namespace eval shellrun { set stdout_stackid [shellfilter::stack::add stdout var -action float-locked -junction 1 -settings {-varname ::shellrun::runout}] } + #-teelog: live-stream the capture variables to shellfilter::log workers via write + #traces. Deliberately NOT implemented as extra tee_to_log stack transforms: the + #float-placement logic is unaudited for junction interactions (README 'very unripe + #parts') and probing (2026-07-18) showed a tee_to_log float combined with the + #float-locked capture junction is either starved (added second - a new float sits + #below existing floats, under the junction) or loses its worker settings through + #add/remove re-instantiation churn (added first) - shellthread's parked-worker + #adoption does not re-apply settings. The capture vars grow per-chunk (var and + #tee_to_var transforms append on every write), so variable traces stream the same + #data live without touching the stack. + set teelog_taglist [list] + if {[dict exists $received "-teelog"]} { + variable teelog_sent + set teelog_defaults [dict create -tag shellrun -syslog "" -file "" -raw 0] + set teelog_settings [dict merge $teelog_defaults [dict get $opts -teelog]] + set teelog_base [dict get $teelog_settings -tag] + foreach {std capvar} [list stdout ::shellrun::runout stderr ::shellrun::runerr] { + set teelog_tag ${teelog_base}$std + set worker_settings $teelog_settings + dict set worker_settings -tag $teelog_tag + #open (or share) the tag's log worker - left open at end of call (see argdoc) + ::shellfilter::log::open $teelog_tag $worker_settings + set teelog_sent($teelog_tag) 0 + trace add variable $capvar write [list ::shellrun::teelog_stream $teelog_tag $capvar] + lappend teelog_taglist $teelog_tag $capvar + } + } + set callopts [dict create] if {[dict exists $received "-tcl"]} { dict set callopts -tclscript 1 @@ -691,6 +748,12 @@ namespace eval shellrun { flush stderr flush stdout + #stack removal flushed any remaining transform buffers into the capture vars, so the + #traces have already streamed the final data - safe to remove them now. + foreach {teelog_tag capvar} $teelog_taglist { + trace remove variable $capvar write [list ::shellrun::teelog_stream $teelog_tag $capvar] + } + if {[dict exists $exitinfo error]} { if {[dict exists $received "-tcl"]} { diff --git a/src/modules/shellrun-buildversion.txt b/src/modules/shellrun-buildversion.txt index 1a2866ce..5c3da589 100644 --- a/src/modules/shellrun-buildversion.txt +++ b/src/modules/shellrun-buildversion.txt @@ -1,6 +1,7 @@ -0.1.4 +0.2.0 #First line must be a semantic version number #all other lines are ignored. +#0.2.0 - runx: new -teelog option - live tee of captured stdout/stderr to shellfilter::log workers (-tag/-syslog/-file/-raw settings; sources stdout/stderr). Implemented as write traces on the capture variables (not stack transforms - float-placement/junction interaction unaudited, and shellthread parked-worker adoption does not re-apply settings). Tag workers left open for reuse; conflicting reconfiguration errors explicitly. #0.1.4 - added missing 'package require punk::args' (0.1.3 call sites use punk::args::parse; absence broke shellrun in bare interps e.g. the runtests testinterp) #0.1.3 - migrated runx from get_run_opts to punk::args::parse (added PUNKARGS spec with @cmd + -help) #0.1.3 - migrated sh_run/sh_runout/sh_runerr/sh_runx from get_run_opts to punk::args::parse (added PUNKARGS specs) diff --git a/src/tests/AGENTS.md b/src/tests/AGENTS.md index 89022d34..a7bbc6a4 100644 --- a/src/tests/AGENTS.md +++ b/src/tests/AGENTS.md @@ -21,6 +21,8 @@ Top-level test harness and source-tree tests for ShellSpy/Punk. Tests here exerc - The testinterp module path includes `src/vendormodules` (and `src/vendormodules_tcl` when present) so vendored dependencies such as `voo` resolve in tests. - Multi-process mode (`-tcltestoptions {-singleproc 0}`, completed 2026-07-18): each test file runs in a child process of the same executable via `testsupport/child_test_runner.tcl`, driven by a per-run environment payload `runtests.tcl` generates (package prefer latest, test tm paths, auto_path, modpod ifneeded definitions, tcltest options; per-file `-testdir` computed child-side). The child then mirrors the single-process testinterp preload: package require shellrun plus one no-op `shellrun::runx -tcl` call (the testinterp sources test files via runx, whose execution pulls further runtime deps - currently punk::lib). Several existing suites use punk::* commands without a package require of their own and depend on this preload (17 files / 97 tests error without shellrun, a further 7 files / 31 tests without the runx warmup; making suite dependencies explicit so children can go leaner is a candidate cleanup, verifiable with the parity tool below). The bootstrap warms `clock format` before the module-path wipe: first script-level clock use loads msgcat from the runtime's default module paths, which the test module paths do not supply (the single-process testinterp is shielded only by the runtests parent's earlier clock use - a latent gap for any test explicitly requiring msgcat in either mode). - Multi-process failure classification: a nonzero child exit is reported as a file-level failure (compact/markdown/json carry an ERROR entry with errorcode `CHILDPROCESS exit ` and the child's stderr tail); exit-0 with no tcltest summary line remains the `missing-cleanupTests` warning. Prefer a native tclsh for multi-process runs: children of a kit executable boot via the kit's script dispatch with kit-stamped punk packages preloaded, which can shadow the src dev modules under test (the runner prints a warning). +- UDP watch mode (added 2026-07-18): `env(PUNK_TEST_UDPTEE)` = `` or `:` (e.g `41197`) makes runtests emit the run over UDP via the shellfilter::log mechanism (shellthread worker threads -> tcludp, one datagram per line, cooked format whose source column carries the tag). Tag `runtests` carries lifecycle events; each event message is a well-formed tcl list `RUNTESTS-EVENT ?key value ...?` (`run-start`, `file-start`, `file-end` with per-file tallies, `run-end`). Tags `teststdout`/`teststderr` carry test file output: streamed live per output chunk in single-process mode (`shellrun::runx -teelog` 0.2.0 - write traces on the capture variables, so captured results and reports are unchanged) and relayed per completed file by the parent in multi-process mode. No listener is required (sends to an unbound local port are discarded). The parent extends its auto_path with `/lib_tcl/` so log workers resolve the project's tcludp 1.0.13+ (1.0.12's UDP_ExitProc is the G-036 exit-hang class). Viewer-side caveat: tcludp's RECEIVE fileevent under Tcl 9.0.3/Windows delivered only the first datagram in testing (send side unaffected; a .NET UdpClient received everything) - a viewer should use timer-driven non-blocking reads or a non-tcludp receiver until that is resolved. +- In watch mode the singleproc loop closes and terminates the shellfilter::log worker threads opened inside each per-file testinterp before deleting it (the `-teelog` tag workers plus shellrun's punkshout/punksherr teehandle workers). The terminate handshakes cost ~0.5-0.7s per file, so unwatched runs skip this and keep the pre-existing small leak of the two teehandle worker threads per file (noted for the shellfilter/shellthread audit; the process exits at run end regardless). - Tests should run against source modules and libraries from `src/`, not installed packages or root-level build outputs. - Test files must `package require` any extra packages explicitly. - Tcltest files must finish with `tcltest::cleanupTests`; missing cleanup produces a `missing-cleanupTests` runner warning and only untrusted observed testcase events. @@ -43,6 +45,7 @@ Top-level test harness and source-tree tests for ShellSpy/Punk. Tests here exerc - Treat `RUNTESTS_RESULT status=warn` and compact warning reasons such as `missing-cleanupTests` as incomplete test results, even if observed pass events are listed. - Add `-slowest ` when timing outliers are relevant. - Multi-process runs use `-tcltestoptions {-singleproc 0}` with otherwise identical flags. Check result parity between modes by capturing `-report json` stdout from each and comparing with `tclsh scriptlib/developer/runtests_parity.tcl ` (ignores timings; exit 0 on parity). +- To watch runs live (including runs launched by agents or other processes, which inherit the environment): set `PUNK_TEST_UDPTEE=41197` and point a UDP listener/viewer at that port. No runner flags needed; results are unaffected. - Add `-strict-exit 1` when a nonzero shell exit code is needed for failures or parser warnings. - Capture enough stderr or failure context to identify the failing command or assertion. - For ERROR-status failures, the markdown report's `errorInfo` block and compact `message=` field carry the full Tcl error message; use `-report markdown` for untruncated context. diff --git a/src/tests/runtests.tcl b/src/tests/runtests.tcl index 8331f41a..c271f2c8 100644 --- a/src/tests/runtests.tcl +++ b/src/tests/runtests.tcl @@ -349,6 +349,15 @@ proc runtests_run_child_process {exe bootstrap payloadfile testfile outfile errf return [dict create exitcode $exitcode stdout $out stderr $err] } +proc runtests_udptee_event {args} { + #emit a runner lifecycle event to the 'runtests' log tag when the UDP tee is active. + #Each event message is a well-formed tcl list: RUNTESTS-EVENT ?key value ...? + if {$::runtests_udptee_target eq ""} { + return + } + catch {::shellfilter::log::write runtests [list RUNTESTS-EVENT {*}$args]} +} + @@ -636,8 +645,55 @@ if {!$singleproc} { runtests_write_child_payload $child_payloadfile $test_tmlist $test_auto_path $ifneeded_script $child_tcltestoptions } +#Optional live watch of test runs over UDP (e.g a local tk viewer app): set env(PUNK_TEST_UDPTEE) +#to or : (e.g 41197 for udp://127.0.0.1:41197). Uses the shellfilter::log +#mechanism (shellthread worker threads -> tcludp datagrams, one per line, cooked format with a +#source column). Runner lifecycle events go to tag 'runtests' (each message is a tcl list: +#RUNTESTS-EVENT ?key value ...?). Test file output goes to tags 'teststdout' and +#'teststderr': streamed live via shellrun::runx -teelog inside the testinterp in single-process +#mode; written per completed file by this parent in multi-process mode (children do not stream). +#No listener is required - datagram sends to an unbound local port are discarded harmlessly. +set runtests_udptee_target "" +if {[info exists ::env(PUNK_TEST_UDPTEE)] && [string trim $::env(PUNK_TEST_UDPTEE)] ne ""} { + set udptee_raw [string trim $::env(PUNK_TEST_UDPTEE)] + if {[string is integer -strict $udptee_raw]} { + set runtests_udptee_target 127.0.0.1:$udptee_raw + } else { + set runtests_udptee_target $udptee_raw + } + lassign [split $runtests_udptee_target :] udptee_host udptee_port + if {$udptee_host eq "" || ![string is integer -strict $udptee_port] || $udptee_port < 1 || $udptee_port > 65535} { + puts stderr "runtests: ignoring invalid PUNK_TEST_UDPTEE value '$::env(PUNK_TEST_UDPTEE)' (expected or :)" + set runtests_udptee_target "" + } +} +if {$runtests_udptee_target ne ""} { + puts stderr "runtests: UDP tee enabled -> $runtests_udptee_target (PUNK_TEST_UDPTEE)" + #ensure log workers resolve the project's tcludp (1.0.13+) rather than an older copy on + #the runtime's default auto_path - 1.0.12's UDP_ExitProc closes process-global event + #sources at exit (the G-036 piped-stdin exit-hang class). Workers inherit ::auto_path at + #creation, so this must precede the opens. (The testinterp/child test env already gets + #this path via test_auto_path.) + set runtests_udp_libdir [file normalize $test_base/../../lib_tcl$tcl_major/$arch] + if {[file isdirectory $runtests_udp_libdir] && $runtests_udp_libdir ni $::auto_path} { + lappend ::auto_path $runtests_udp_libdir + } + if {[catch {::shellfilter::log::open runtests [list -syslog $runtests_udptee_target]} errM]} { + puts stderr "runtests: could not open UDP tee log worker ($errM) - disabling tee" + set runtests_udptee_target "" + } elseif {!$singleproc} { + #multi-process mode: the parent relays each child's captured output after the file runs + catch {::shellfilter::log::open teststdout [list -syslog $runtests_udptee_target]} + catch {::shellfilter::log::open teststderr [list -syslog $runtests_udptee_target]} + } +} +runtests_udptee_event run-start mode [expr {$singleproc ? "singleproc" : "multiproc"}] executable $thisexecutable tcl [info patchlevel] files [llength $testfiles] argv $::argv + +set runtests_file_number 0 foreach testfile_relative $testfiles { set testfile [file normalize [file join $test_base $testfile_relative]] + incr runtests_file_number + runtests_udptee_event file-start path $testfile_relative index $runtests_file_number of [llength $testfiles] if {$report_detailed_markdown} { puts stdout "" #puts stdout "running test file $testfile" @@ -671,7 +727,33 @@ foreach testfile_relative $testfiles { # } # return $result # }] <<<<" - set result [testinterp eval [list shellrun::runx -tcl source $testfile]] + if {$runtests_udptee_target ne ""} { + #live-stream this file's output to the UDP watch target while capturing + #(log sources teststdout/teststderr - same tags the multiproc parent relay uses) + set result [testinterp eval [list shellrun::runx -teelog [list -tag test -syslog $runtests_udptee_target] -tcl source $testfile]] + } else { + set result [testinterp eval [list shellrun::runx -tcl source $testfile]] + } + if {$runtests_udptee_target ne ""} { + #Watch mode: close the shellfilter::log worker threads opened inside the testinterp + #(the -teelog tag workers plus shellrun's punkshout/punksherr teehandle workers) and + #terminate them - they are process-level threads whose manager state lives in this + #short-lived interp, so without this every watched file would leak 4 idle worker + #threads for the remainder of the run. Costs ~0.5-0.7s per file (terminate + #handshakes), so it is not done for unwatched runs - those spawn only the two + #teehandle workers per file, a small pre-existing leak noted for the + #shellfilter/shellthread audit. + testinterp eval { + catch { + if {[info exists ::shellthread::manager::workers]} { + foreach _rt_tag [dict keys $::shellthread::manager::workers] { + catch {::shellfilter::log::close $_rt_tag} + } + } + catch {shellthread::manager::shutdown_free_threads 1000} + } + } + } interp delete testinterp if {$report_detailed_markdown} { puts stdout "sourced $testfile " @@ -679,6 +761,16 @@ foreach testfile_relative $testfiles { flush stdout } else { set result [runtests_run_child_process $thisexecutable $child_bootstrap $child_payloadfile $testfile $child_outfile $child_errfile] + if {$runtests_udptee_target ne ""} { + #relay the child's captured output to the watch target (file granularity - + #children do not stream live; see src/tests/AGENTS.md) + if {[dict exists $result stdout] && [dict get $result stdout] ne ""} { + catch {::shellfilter::log::write teststdout [dict get $result stdout]} + } + if {[dict exists $result stderr] && [dict get $result stderr] ne ""} { + catch {::shellfilter::log::write teststderr [dict get $result stderr]} + } + } if {$report_detailed_markdown} { puts stdout "executed $thisexecutable testsupport/child_test_runner.tcl $testfile_relative " } @@ -868,10 +960,15 @@ foreach testfile_relative $testfiles { flush stdout } + if {$runtests_udptee_target ne ""} { + set last_summary [lindex $file_summaries end] + runtests_udptee_event file-end path $testfile_relative status [dict get $last_summary status] total [dict get $last_summary total] passed [dict get $last_summary passed] skipped [dict get $last_summary skipped] failed [dict get $last_summary failed] + } set i 0 } set status [runtests_status $tallydict] set slowest_tests [runtests_slowest_tests $all_passes $opt_slowest] +runtests_udptee_event run-end status $status total [dict get $tallydict total] passed [dict get $tallydict passed] skipped [dict get $tallydict skipped] failed [dict get $tallydict failed] files [llength $file_summaries] elapsed_seconds [expr {[clock seconds] - $ts_start}] if {!$report_json_only} { puts stdout "## runtests summary" @@ -1003,6 +1100,16 @@ if {$runtests_workdir ne "" && [file isdirectory $runtests_workdir]} { catch {file delete -force $runtests_workdir} } +if {$runtests_udptee_target ne ""} { + #orderly shutdown of the parent's tee log workers before exit - worker threads and their + #tcludp sockets must not be left running into process teardown (see the G-036 exit-wedge + #history around udp workers at exit/quit) + catch {::shellfilter::log::close runtests} + catch {::shellfilter::log::close teststdout} + catch {::shellfilter::log::close teststderr} + catch {shellthread::manager::shutdown_free_threads 2500} +} + if {$opt_strict_exit} { if {[dict get $tallydict failed] > 0 || [llength [dict get $tallydict files_with_fails]] > 0} { exit 1