- scriptlib/developer/testbody_lint.tcl (new): tcltest .test lint per
src/tests/AGENTS.md Verification. E-level: a -body whose final command
always returns empty vs a -result that cannot match empty under its
-match mode; test invocations with no -body at all (vacuous pass);
unbalanced-brace parse kills. W-level: uninterpretable shapes.
Hand-rolled command-word splitter (the corpus '{desc}\' style is
command-legal but list-illegal), namespace-eval recursion, literal
{*}{...} splices, -selftest (8 fixtures) + -stats. 142 files /
1674 tests parse clean.
- src/tests/runtests.tcl: FAILED reports with an empty actual and a
non-empty expected append a hint= (compact) / hint : (markdown) line
naming the trailing 'set result'/'return $result' convention, so the
failure carries its own diagnosis. JSON reports unchanged.
- src/tests/modules/punk/args/testsuites/args/args.test: the
literalprefix sibling of parse_withdef_value_leading_multiple_not_greedy
was passing VACUOUSLY - a missing description made the braced
description word swallow -setup/-body/-cleanup/-result, so tcltest ran
an empty body against the default -result "". Description restored,
orphan closing brace removed; the de-vacuized test executes and passes
(file 34/34).
- AGENTS.md (root), src/tests/AGENTS.md, .agents/.claude tcl-runtests
SKILL.md (byte-identical copies): the body-ender convention now
sanctions 'return $result' alongside 'set result' (verified on 8.6 and
9.0: default -returnCodes {ok return} accepts the return form,
-cleanup still runs, an explicit -returnCodes list omitting 'return'
rejects it loudly); linter and hint documented.
Verification: testbody_lint -selftest PASS on tclsh90s and tclsh86ts;
corpus lint clean; args.test 34/34; full suite in both runner modes with
runtests_parity.tcl PARITY: ok (1677 tests, 1631 pass). Residual 23
failures are pre-existing/environment, not this change: 12 maketclhelp
(auto_execok tclsh Bash-env trap; pass under PowerShell), 8
maketcllibfetch + 2 maketclbakelist (reproduce identically with this
runtests.tcl change stashed), core exec-14.3 (pins the pre-'-encoding'
exec error message vs tcl 9.0.5).
Claude-Session: https://claude.ai/code/session_016Bk571eG3prsJWYnewboW2
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
The hand-rolled -report json emitter in runtests.tcl classified values as
JSON numbers via `string is entier -strict`, which (a) is deprecated in Tcl 9.0
per TIP 514 (string is integer became the canonical unlimited-range form) and
(b) accepted several forms that are NOT valid JSON numbers, emitting them bare:
007 (leading zeros), +5 (leading plus), 0x10/0o17/0b101 (hex/octal/binary).
The G-093 work already patched one symptom (a "0\n" result embedded a raw
newline) but left the rest of the class open. The string escaper
(runtests_json_string) also left most C0 control chars (0x00-0x1F) raw, which
RFC 8259 forbids in strings - a test whose result_was carries a bell or NUL
corrupted the report the same way.
Hardening (src/tests/testsupport/json_emit.tcl, extracted from runtests.tcl so
the new suite can unit-test it without a child run):
- runtests_json_int? replaces `string is entier -strict` with an RFC 8259
integer regex (^-?(0|[1-9][0-9]*)$) - version-independent, eliminates entier
entirely (no Tcl-9 deprecation exposure), subsumes the G-093 whitespace guard
via ^...$ anchoring.
- runtests_json_string now escapes the whole C0 range: 0x08 -> \b, 0x0C -> \f,
others -> \u00XX. The existing \n \r \t \ \" escapes are preserved
byte-for-byte (the multilinebanner.test end-to-end suite probes exact
substrings such as "result_was":"x\n").
Pinning suite (src/tests/runner/testsuites/parser/jsonemit.test, 8 tests):
RFC 8259 number syntax (007/+5/0x10/0o17/0b101 emitted as quoted strings; valid
integers bare), C0 control-char escaping (no raw 0x00-0x1F survives; \b \f
short escapes; \u00XX for the rest), entier-deprecation hygiene (asserts the
loaded predicate proc body uses regexp and mentions no entier), named-escape
byte regression, and tcllib-json round-trip of a representative testdict array.
DOX: src/tests/AGENTS.md testsupport bullet records json_emit.tcl;
src/tests/runner/AGENTS.md parser description records jsonemit.test.
Verification: jsonemit.test 8/8 on Tcl 9.0.5 and Tcl 8.6; multilinebanner.test
2/2 (no regression to the existing emitter output contract); full runner/
subtree 40/40; mode parity runtests_parity.tcl PARITY: ok on a broad real-data
subset (modules/punk/args, 35 files / 341 tests, singleproc vs multiproc).
Tests-only change - no punkproject.toml bump per the versioning policy.
Assisted-by: harness=pi; primary-model=zai-org/GLM-5.2; api-location=huggingface.co
- runner/testsuites/parser/multilinebanner.test + fixtures/
multilinedesc.testfixture: the deliberately failing fixture (multi-line
description; non-.test extension so default *.test discovery never sees
it) is run through real child runtests.tcl invocations in BOTH modes and
the json report asserted as exact-text probes: failure entry FAILED with
result_was/result_expected populated, trusted summary
(summaryline_detected 1, totals 2/1/0/1). 2/2 green on tclsh90 and
native tclsh 8.6; runner subtree 30/30 under -jobs 4.
- runtests.tcl: punk::tcltestrun require floored at 0.4.0- (tolerant
parser + result_expected capture; kit-fossil rejection floor).
- parsetestrun.test: explicit punk::args + punk::lib requires - the lean
-jobs child exposed tcltestrun's load-time punk::args dependency (about
section calls punk::args::lib::tstr/resolved_def at source time; every
other context preloads punk::args). Declared at test level per the
established lean-children convention.
- src/tests/AGENTS.md: single-line-description rule relaxed to STYLE
guidance (multi-line tolerated since punk::tcltestrun 0.4.0, boundary
documented); floors bullet gains punk::tcltestrun 0.4.0-; runner index
line mentions output parsing.
- runner/AGENTS.md: parser characterization pair documented (fixture
extension convention, two-children cost note).
- modules/AGENTS.md commandstack bullet + commandstack.test NOTE comment:
hard contract wording updated to style guidance (G-161).
- CHANGELOG.md + punkproject.toml: 0.49.6.
Full-suite parity before/after the parser change verified in BOTH modes
with scriptlib/developer/runtests_parity.tcl (new parser dir excluded from
the after-runs to keep the file set identical): PARITY ok, 126 files,
1422/1400/20/2 both sides. Inclusive full run: 1430 total, same two
pre-existing failing files (core exec.test, punkboot offsetstyle.test),
0 warnings.
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
Root cause of the punk86-kit runtests failure: a punk-kit interpreter's
boot pre-registers its kit-stamped package versions, and an unversioned
require satisfiable by such a registration never fires package unknown -
the freshly-added bootsupport tm path is never scanned. The old kit's
punk::args 0.1.0 won the require, then the versioned punk::path 0.4.0-
pulled a modern module whose load-time argdoc needs punk::args::define:
invalid command name.
Floors added: punk::args 0.12- (runtests.tcl toplevel), punk::lib
0.5.1- (both 8.6 lpop guards; punk::path 0.4.0- already floored in
discovery.tcl). Feature-era minimums, not latest pins - their purpose is
rejecting fossils so package unknown resolves the bootsupport copy.
Verified: the floors alone resurrect even the OLD fossilized punk86 kit
(discover-only + a 35/35 suite run); native 9 and 8.6 unaffected.
concurrency.test child preamble floors its punkcheck require (0.6-):
kit-hosted children boot with kit-stamped punkcheck PRELOADED, and a
stale kit (pre-vfscommonupdate _vfscommon) made children fail on
'invalid command name punkcheck:🔒:acquire' - the floor turns that
into a loud version-conflict in the child log.
src/tests/AGENTS.md records the floor rationale and the kit-hosted
child-suite requirement (vfscommonupdate-current kit content).
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
The validated compat family is brought across for the harness: guarded
'package require punk::lib' when ::lpop is absent, at both boot levels -
the toplevel two-stage boot (right after punk::args, whose parse needs
lpop at runtests.tcl's own argument parsing) and the singleproc
testinterp (before the shellrun preload, whose runx parses via
punk::args::parse). No-ops on Tcl 8.7+/9: -discover-only stays ~0.35s.
Native 8.6.17 verification: -discover-only works; punkcheck subtree
84/84, punk/lib subtree 43/43+1 gated skip, runner discovery suite
22/22, and a multiproc (-singleproc 0) probe 49/49.
Tcl 9.0.3 regression per the src/tests/AGENTS.md runner-change
contract: full suite in both modes with -report json,
runtests_parity.tcl reports PARITY: ok (files=96 total=1064
passed=1047 skipped=16 failed=1 - the sole failure is the documented
pre-existing exec-14.3 baseline).
src/tests/AGENTS.md records the supported-interpreter change and the
still-broken punk86-kit runtests route (kit/dev module skew - use a
native 8.6 tclsh for 8.6 legs).
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
Under punk91/punk902z/punk905_beta, '-include-paths modules/punk/console/***
-jobs 16' failed all 7 files with zero tests run: child processes boot via
plain kit script dispatch, preloading kit-stamped punk packages
(punk::console 0.7.2) before the payload's src paths apply - the suites'
pinned 'require punk::console 999999.0a1.0-' then version-conflicts and
the child exits 1 (the known shadowing runtests already warned about).
Fix: when the host is a PUNK kit (kit container heuristics AND the
::punkboot namespace marker from punk_main.tcl boot) children launch as
'kitexe src child_test_runner.tcl ...' so they boot the project's src dev
modules - matching what the payload selects. Non-punk kit containers (e.g
tclsh90szip.exe: zipfs app mount, no punk boot) keep the old warning and
get no extra arg. runtests_run_child_process gains a required exemodeargs
param (tpool info args/body replication unaffected).
Tally: file-level failures are not test results, so a FAIL run could sit
beside 'Failed 0' unexplained (user report). The tally line now appends a
fail-coloured 'Testfiles-failed N' whenever testfiles failed.
Verification: console suites 71/71 under punk905_beta, punk91, punk902z
(previously 0 run / 7 files failed); native Tcl903 + suite tclsh90s
unchanged 71/71 with no modeargs; tclsh90szip warning path passes; tally
demo shows 'Total 0 ... Failed 0 Testfiles-failed 1'.
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
User-approved contract amendment replacing the day-1 file-relpath filter: one
directory-oriented pathglob semantics at every layer, with the ergonomics gap
closed in the syntax itself.
punk::path 0.4.0:
- *** as a whole segment matches ZERO or more segments (join-aware regex in
pathglob_as_re: an adjacent separator is absorbed into an optional group),
so X/*** = X and everything below - the subtree spelling; ***/f.txt matches
at any depth including the root.
- include separability fix: an exact or single-segment-glob full match no
longer sets allbelow (the deep walk previously dragged the whole subtree in
for X and X/* patterns; the zipfs walk was already separable).
- exclude separability fix (new exclude_state classifier, both walks): only
**|***-tail patterns prune the matched subtree; other matches exclude just
that directory's files - the **/_aside vs **/_aside/** argdoc distinction is
now real. Dead _path_matches_any removed.
- legacy bare-* include collapse removed (a * in -include-paths is the
one-below lattice form, not a match-everything alias; no caller used it).
- globmatchpath default-nocase driveletter exception rewritten as a direct
regex-head rewrite (the regex-text file-split broke on c:/***'s glued group
and accepted non-alpha drives).
- path.test +5 lattice/*** pins (40 total).
runtests.tcl:
- -include-paths/-exclude-paths pass through to treefilenames unchanged;
argdocs teach the separable X | X/* | X/** | X/*** lattice; directory and
file-name axes independent (tails); single-file targeting = exact dir
pattern + file tail.
- -serial-paths reconciled to the same directory-of-file reading (default now
modules/punk/console/*** modules/opunk/console/*** via
runtests_discovery::dir_matches_any) - the pre-existing file-path serial
matching was the other same-option-different-referent divergence.
- per-pattern zero-match stderr advisory (the X/**-for-subtree foot-gun points
at X/***).
- json report emitter: whitespace-padded entier-looking values now take the
quoted-escaped path ('string is entier -strict' accepts surrounding
whitespace - a failed test's "0\n" result_was embedded a raw newline and
truncated the parseable report line).
- testsupport/discovery.tcl now a thin passthrough (requires punk::path
0.4.0-); pathdiscovery.test rewritten to the directory-semantics contract
(22 tests incl. the X/** zero-match advisory characterization); AGENTS.md
targeting guidance swept to /*** across src/tests and src/modules/opunk.
Verification (native tclsh 9.0.3, piped, serialized): 45-probe matrix, path
suite 40/40, runner suite 22/22; vs the pre-G-093 baseline the default
discovery FILE SET differs by exactly the added runner suite and result
parity only by the five new path tests (+ that suite); mode parity -jobs 16
vs singleproc: PARITY ok (94/1037/failed=1 exec-14.3). Goal contract amended
in the detail file; full record there.
Claude-Session: https://claude.ai/code/session_01BNUVVkYq9vHa6G3S9a3XTZ
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
Root cause (recorded in the goal file): punk::path::treefilenames matches
-include-paths against the DIRECTORY containing files, and its documented
X/** boundary semantics exclude files directly in X - so suites keeping
.test files at the subtree root (scriptwrap, binscripts, punkexe) were
silently missed by X/** targeting: the G-092 four-pattern (2/13 files) and
repeated-flag (zero files) failures. punk::args -multiple accumulation
probed healthy - repeated flags and a single space-separated value reach
discovery identically; no punk change was needed or made.
runtests.tcl now filters the tailbase-relative FILE paths after an
unrestricted walk (new testsupport/discovery.tcl: globmatchpath,
any-include-match, exclusion wins): X/** includes files directly in X, an
exact relative path targets a single file, a bare directory path matches
nothing (append /**). -exclude-paths exposed (same syntax and accumulation,
exclusion wins over inclusion). -discover-only prints the discovered list
plus a RUNTESTS_FILES tcl-list line and exits before any run machinery -
subsecond via a two-stage boot (discovery phase loads only punk::args +
punk::path; punk/Thread/shellrun/punk::tcltestrun and the testinterp/child
path assembly load after the exit).
New regression suite runner/testsuites/discovery/pathdiscovery.test
(23 tests: pure filter, fixture-tree walk+filter, end-to-end -discover-only
children including both G-092 characterization forms; ~7s wall,
parallel-safe). src/tests/AGENTS.md targeting wording reconciled; new
src/tests/runner/AGENTS.md. G-093 index status proposed -> active
(user-directed).
Verification (native tclsh 9.0.3, piped Bash context): pre-change baseline
default -jobs 16 (93 files / 1010 tests / failed=1 exec-14.3 only) vs
post-change -jobs 16 with -exclude-paths runner/**: PARITY ok; post-change
default run differs from baseline only by the added suite (+23 passing);
mode parity -jobs 16 vs singleproc sequential: PARITY ok (94/1033/1).
Claude-Session: https://claude.ai/code/session_01BNUVVkYq9vHa6G3S9a3XTZ
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
User request after console-context failures went unnoticed: compact
per-file status words, failure/warning list entries, the per-file
had-failures/passed/child-failed notices, the markdown
failures/warnings file lists, the tally Passed/Failed segments and a
coloured overall STATUS on the DONE line are now SGR-coloured (green
pass, bold red fail/error, bold yellow warning).
Gating: -colour auto|on|off (default auto) colours only when stdout is
a real windows console, detected via twapi GetConsoleMode on the
STD_OUTPUT handle - the isatty-equivalent; get_console_handle proved
unsuitable (it succeeds for piped children of console shells, verified
both ways). NO_COLOR honoured in auto. Machine output (RUNTESTS_RESULT,
json) is never coloured; piped output stays plain for agents (verified
zero escape bytes piped, codes present with -colour on). Raw literal
SGR is used rather than a+/a so the runner's indicators are independent
of punk::console's process-global colour state (the same 2026-07-19
console runs showed that state shifting under mixed tty-ness). The
previously UNCONDITIONAL raw-red errorInfo block in the file-error
branch now goes through the same gate, so piped error output is cleaner
than before.
Also sharpens the -jobs guidance wording: subtree runs (e.g 28-file
modules/punk/args/**: ~50s sequential vs ~11s at -jobs 16) benefit from
-jobs - only single-file/few-file runs do not.
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
The '### runtests DONE at <ts>' human line gains '(elapsed <N>s)', the
RUNTESTS_RESULT summary line gains a trailing elapsed_seconds=<N>
key=value, and the json report gains a top-level "elapsed_seconds"
field - matching the name already used by the watch-mode run-end event,
which now shares the same computed value. Additive only; the parity
tool reads named keys and is unaffected (verified on all three report
styles).
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
multishell.test (10 tests, ~105s solo child - the -jobs floor) split five
ways along shared-fixture boundaries, every test moved verbatim with its
provenance lines: multishell.test keeps wrap + structure/LF-only + exec
smoke (incl WSL); multishell_wrapverify.test (fresh-wrap checkfile) and
multishell_wrapdeterminism.test (byte-identical re-wrap) each re-pay the
wrap fixture; runtimecmd_checkfile.test and runtimecmd_roundtrip.test
guard the committed bin/runtime.cmd and its source round-trip with no
wrap fixture at all. argparsingtest.test (17 tests, ~32s) split into
timeit/discover/contract/first_call (11) + argparsingtest_compare.test
(compare_*, 6). Assessed and left unsplit: dtplite.test (24.6s solo),
scriptexec.test (12s); exec.test excluded as imported core material.
File headers point at the src/tests/modules/AGENTS.md punk/mix bullet
for the sibling map; the runtests longest-first weight table carries
measured post-split weights.
runtests_parity.tcl gains a -names mode: multiset comparison of
'- testname:' lines from '-report markdown -show-passes 1' captures,
file attribution ignored - verbatim moves compare identical, lost/
duplicated/renamed tests are reported.
Verified (native tclsh 9.0.3): repeated -jobs 8 full-suite runs 1m26.3s
/ 1m26.8s (was 2m16s at G-091, ~5m40s sequential - 4.07x vs the
pre-parallel baseline, 36.6% vs the G-091 reference); slowest child
30.7s / 31.3s (bound 45s); NAMES identical (989 occurrences) vs the
pre-split sequential capture; tallies identical in every run (989/973/
15/1, exec-14.3 sole failure, files 87 -> 92); PARITY ok -jobs 8 vs
-jobs 1 and between repeats; repo clean after runs. Notable finding
recorded in the goal file: child-wall contention inflation (~45% with
50s+ dominant children present) collapsed to ~15% once the work
distribution was flattened - splitting reduced both the critical path
and the contention itself.
G-092 flipped to achieved 2026-07-19 and archived (GOALS-archive.md
record, detail file with full verification evidence to goals/archive/).
goals_lint clean. Also noted in the goal file, out of scope: runtests
-include-paths multi-pattern/repeated-flag matching misbehaved during
measurement (single value with four deep patterns matched one; repeated
flags matched zero) - contradicts documented semantics, needs its own
investigation.
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
-jobs N (implies -singleproc 0; explicit -singleproc 1 conflicts) runs
test files through a tpool of N concurrent child processes with
per-child capture files and tcltest -tmpdir subdirs, longest-first
static-weight submission, and a -serial-paths serial tail (default: the
console-state suites modules/punk/console/** + modules/opunk/console/**;
piped-child-driven suites proved parallel-safe and each serial file
costs its full child boot strictly after the parallel phase). Cached
results feed the unchanged processing loop in discovery order, so
reports are identical in shape and order to sequential runs. Watch mode
emits file-collected events (completion order, output relayed there);
file-end (tallies) stays discovery-order. The runner now reports jobs
phase times and the ten slowest child WALL times (child wall includes
boot + file-level fixture cost that per-test usec misses).
Two Tcl-core-level behaviours found and worked around in the child
spawn (see the verification record in the archived goal file):
- concurrent BLOCKING execs from worker threads convoy on Windows (all
return at the longest concurrent child's exit; plain sleepers 2-12s
from 8 workers all walled at 12.0s) -> spawn via 'exec ... &' and
poll per-pid tcl::process status; blocking fallback retained for
runtimes without tcl::process (tcl 8.6).
- Tcl_ReapDetachedProcs runs on every exec and purges finished
background children process-wide, racing the poll (nondeterministic
'key <pid> not known in dictionary') -> tcl::process autopurge false
+ explicit per-pid purge after the status read.
Verified on native tclsh 9.0.3: repeated -jobs 8 full-suite runs
2m16.2s / 2m15.6s with PARITY ok vs -jobs 1, between repeats, and vs
the pre-parallel baseline (987-test identity, exec-14.3 sole expected
failure); 2.58x vs the ~350s sequential baseline (2.41x vs one
cooler-machine 5m28.8s sequential run - recorded in the goal file);
repo clean after every run.
G-091 flipped to achieved 2026-07-19 and archived (entry to
GOALS-archive.md, detail file to goals/archive/ with the verification
record; reference sweep updated runtests.tcl comments and the
src/tests/AGENTS.md marker). goals_lint clean.
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
runx -teelog {-tag/-syslog/-file/-raw settings}: live tee of the
captured stdout/stderr to shellfilter::log workers (log sources
<tag>stdout/<tag>stderr) while captured/returned data is unchanged.
Implemented as write traces on the capture variables rather than extra
channel transforms: probing showed a tee_to_log float combined with the
float-locked capture junction is starved or loses its worker settings
through add/remove churn (float-placement logic is unaudited - README
'very unripe parts'), and shellthread's parked-worker adoption does not
re-apply settings (writes silently misroute to a previous target). Both
findings deliberately not addressed here - recorded for a proposed
shellfilter/shellthread audit goal. Tag workers stay open for reuse;
conflicting reconfiguration errors explicitly.
runtests.tcl watch mode: env PUNK_TEST_UDPTEE=<port>|<host>:<port>
streams runner lifecycle events (tag 'runtests', tcl-list
RUNTESTS-EVENT messages: run-start/file-start/file-end/run-end with
tallies) and test output (tags teststdout/teststderr) to a local UDP
listener via the same mechanism - live per chunk in singleproc mode
(runx -teelog inside the testinterp), relayed per completed file by the
parent in multiproc mode. Watch mode terminates the per-testinterp log
worker threads after each file; unwatched runs are unchanged (including
the pre-existing teehandle-worker leak, also noted for the audit). The
parent extends auto_path with <projectroot>/lib_tcl<N>/<arch> so
workers load tcludp 1.0.13+ (1.0.12 UDP_ExitProc = G-036 exit-hang
class).
Verified on native tclsh 9.0.3: full-suite result parity (87 files, 989
tests, exec-14.3 sole failure) for watch-off and watch-on singleproc
runs vs the pre-change baseline; focused watch runs in both modes
captured over UDP (events + live/relayed output). Viewer-side caveat
documented: tcludp's receive fileevent under Tcl 9.0.3/Windows
delivered only the first datagram (send side unaffected; .NET UdpClient
received all).
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
Multi-process mode previously execd bare test files with no environment
(children died on their first package require) and misreported the
crashes as missing-cleanupTests warnings with the stderr discarded.
- testsupport/child_test_runner.tcl (new): per-file child bootstrap.
Applies a runtests-generated environment payload (prefer latest, test
tm paths, auto_path, modpod ifneeded defs, tcltest options; per-file
-testdir computed child-side), warms 'clock format' before the
module-path wipe (first script-level clock use loads msgcat from the
runtime default paths, which the test paths do not supply), and
mirrors the testinterp preload: package require shellrun plus one
no-op runx -tcl call (runx execution pulls punk::lib; several suites
depend on preloaded punk::* commands).
- runtests.tcl: generate the payload per run; spawn children via plain
exec with file-captured stdout/stderr and immediate-EOF stdin
(mechanism reusable for future parallel -jobs scheduling); classify
nonzero child exits as file-level failures with the stderr tail
surfaced in all report styles (exit-0-no-summary remains the
missing-cleanupTests warning); warn under kit executables (children
boot with kit-stamped punk packages preloaded, shadowing src dev
modules - prefer a native tclsh).
- scriptlib/developer/runtests_parity.tcl (new): compares two
'-report json' captures on result identity (per-file counts,
failure/skip/warning identities; timings ignored), tolerating
preamble noise and the ANSI SGR reset emitted on the JSON line.
Verified on native tclsh 9.0.3: full suite singleproc vs multiproc
PARITY ok (87 files, 989 tests, 973/15/1 with the documented exec-14.3
baseline as the sole failure); crash and warning classification
exercised. Wall time ~5m50s in both modes.
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
Two independent list-flattening bugs made `runtests.tcl foo.test bar.test`
match zero files, forcing agents to run files individually:
- the trailing glob value is -multiple 1, so its parsed value is a list;
plain lappend nested it as a single space-containing element
- punk::path::treefilenames declares tailglobs as a -multiple 1 positional
(one glob per argument), but the whole file_globs list was passed as one
argument, re-collapsing it (this also broke multiple globs supplied via
-tcltestoptions {-file {...}})
Single names always worked because a one-element Tcl list flattens to
itself, which masked both bugs. Verified: two file tails now discover and
run both suites (files=2, 40/40 pass).
Also note multi-file-tail support in src/tests/AGENTS.md Work Guidance.
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
- Repeated -include-paths flags previously overrode each other silently (standard
non-multiple punk::args opt semantics), quietly narrowing the test selection
while reporting green - a recurring misuse driven by Unix-CLI accumulate-flag
muscle memory (gcc -I / curl -H style).
- The option is now -multiple 1: repeated flags accumulate, the single
space-separated quoted list form is unchanged, and the unreceived default {**}
passes through (the consumer flattens occurrences with concat). Help text
documents both forms as equivalent.
- src/tests/AGENTS.md work-guidance updated. Known remaining sibling issue:
multiple trailing file-tail globs still silently match 0 files.
- Verified: repeated-flag and single-list invocations select identical file sets
(20 files / 189 tests) on Tcl 9.0.3, repeated-flag accumulation on 8.7, and a
no-flag default run with a file-tail glob unchanged. The general strict-arity
mechanism is proposed as G-053.
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
The testinterp replaces auto_path wholesale (project src/lib dirs, root
lib_tcl<N>/<arch>, [info library] + parent). Under a kit executable that
parent is e.g //zipfs:/app, and tclPkgUnknown scans only an entry plus its
immediate children - so kit-bundled packages under //zipfs:/app/lib_tcl<N>/
<pkg> (e.g tcllib's tcl::chan::fifo2, required by the shellrun harness at
testinterp setup) were unreachable. Unnoticed in punkshell because the
project's own root lib_tcl9/win32-x86_64 carries tcllib2.0; projects
without that payload (e.g tomlish) failed every file at
'package require shellrun' when run under a punk kit exe.
Now the kit's internal lib/lib_tcl<N> dirs are appended when present,
mirroring the internal-path classification in src/vfs/_config/punk_main.tcl:
zipfs app mount, tclkit ::tcl::kitpath exe dir-shadow, cookfs //cookit:/
volume (default mount name only, as per punk_main.tcl). No-op under native
tclsh. Verified: punk902z + tclsh902z suites pass; classification probed OK
under tclkit 8.6.17, punk9cook (cookit), punk91 (zipfs 9.1), native Tcl903;
tomlish suite under punk902z/punk91 goes 0 -> 149/149 with the same edit.
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
makeFile/makeDirectory output previously landed in the working directory -
aborted core exec.test runs littered src/tests with untracked helper files
(cat, echo, gorp.file ...). Unless -tcltestoptions supplies -tmpdir, a
fresh directory from 'file tempdir' (TMPDIR/TEMP/TMP + pid fallback for
tcl 8.6) is configured and deleted before exit; a hard-aborted run now
leaves litter in the OS temp area rather than the source tree.
Takes effect in single-process mode; multi-process mode still does not
forward tcltest options to child processes (pre-existing todo).
Contract documented in src/tests/AGENTS.md.
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
shellfilter::run now wraps everything between tee-stack add and removal in
try/finally so the callers stdout/stderr are always restored, and the
decorative stack-status table rendering in log::critical blocks is
catch-guarded. Previously an error there aborted the run after diverting
the channels and before restoring them, leaving process stdout/stderr
broken with all subsequent output silently lost - observed when a stale
same-version textblock snapshot (old option-value syntax) called the new
punk::ansi sgr_merge_singles inside the stack-status render during tomlish
test runs.
runtests.tcl:
- testinterp tm path order: bootsupport moved to last test_tmlist element
so it wins same-version module ties over vendormodules. Rules verified
experimentally on Tcl 9.0.3 (identical under standard package unknown
and punk::libunknown): tcl:™️:add prepends each argument, head of
tcl:™️:list wins same-version ties, so the last element of a
single-call add list wins.
- testinterp auto_path gains the parent of [info library] so binary
packages such as Thread (needed by shellrun) resolve in projects
without a root lib_tcl<N>/<arch> payload.
Project version 0.2.6. Verified: full shellspy suite at baseline (517
pass / exec-14.3 known); tomlish ported suite 184/184; reconstructed
stale-module scenario now completes with captured results and live
stdout instead of silent loss.
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
Both tm_path_additional_ifneeded call sites now pass \ as required by the 0.3.0 API. The second call site also passed the wrong path (../modules instead of ../modules_tcl\) due to a copy-paste bug, so Tcl-major-specific #modpod modules were never scanned. Added a zipper #modpod verification test that confirms package require zipper resolves to the 999999.0a1.0 #modpod version via the ifneeded mechanism and that initialize/addentry/finalize work correctly. All 4 tests pass under tclsh90.
runtests_failure_summaries now includes errorinfo, result_was, and result_expected when present in the parsed testdict, so the json report (which iterates keys) and downstream consumers get the full failure context. runtests_print_failure_details prints an errorInfo fenced block for ERROR-status failures and result_was/result_expected fenced blocks for FAILED-status mismatches, with full multi-line text. The compact report appends one-line truncated message=, actual=, and expected= fields (120/80/80 chars) so compact mode stays compact while still being actionable for agents. src/tests/AGENTS.md Local Contracts and Work Guidance updated to document the new failure-detail surfacing.
Julian Noble