From e68cc1c223a3c44f7404b0fd23b987815d5f36d0 Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Tue, 14 Jul 2026 16:39:24 +1000 Subject: [PATCH] punk::lib 0.5.0: has_tclbug_nestedswitch_tracelines check (behavioural probe of tktview 5d5b1052280c976ea3d4) New check pair in punk::lib::check, in the established style (G-076 console_deadspin exemplar: pure facts-in/verdict-out classifier tclbug_nestedswitch_tracelines_signature + has_tclbug_* proc returning the standard buginfo dict {bug bugref description level}, auto-surfaced through the help tcl warning report via the info procs enumeration). Unlike the version-gated checks, this one is a LIVE behavioural probe (safe and cheap - no console interaction): a dynamically-defined proc with a nested 4-word switch is enterstep-traced for inner arm 1 (affected split- list index 1 - reports line 3) and the default arm (index 5, beyond the command word count - reports line 2 correctly), matched against the characterized wrong/right signature. Detection therefore flips automatically on a Tcl runtime containing the upstream fix - no fixed_in version variable needed. Probe layout note: a 2-arm probe would put the default body at affected index 3 and hide the signature (both arms mismark) - the middle arm exists to push default to safe index 5 (this initially bit the probe itself, confirming the law once more). Followup-when-fixed documented in the proc: re-verify with scriptlib/developer/tcl_switch_traceline_repro.tcl, flip the ns/cmdtrace.test GAP pins and fixed-canary, update the punk::ns::cmdtrace argdoc caveat, note the fixed-in version. New canary test cmdtrace_upstream_bug_check_canary asserts the check still reports the bug - on a fixed Tcl it fails first and points at that workflow. Verified: punk/ns 69/69 and punk/lib 37/37 under Tcl 9.0.3; cmdtrace.test 6/6 under 8.7; all has_tclbug_*/has_libbug_* procs exercised help-tcl style without error; probe self-cleans (no temp proc/var residue). punk::lib buildversion 0.4.3 -> 0.5.0 (new API procs = minor); project 0.12.32 + CHANGELOG; tests AGENTS.md index updated. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- CHANGELOG.md | 4 + punkproject.toml | 2 +- src/modules/punk/lib-999999.0a1.0.tm | 91 +++++++++++++++++++ src/modules/punk/lib-buildversion.txt | 3 +- src/tests/modules/AGENTS.md | 2 +- .../punk/ns/testsuites/ns/cmdtrace.test | 18 ++++ 6 files changed, 117 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1d8d2f1f..7d40f71d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` "Project Versioning" section for the bump policy. +## [0.12.32] - 2026-07-14 + +- punk::lib 0.5.0: new punk::lib::check::has_tclbug_nestedswitch_tracelines (+ pure signature classifier) - live behavioural probe of the upstream nested-switch trace-line mis-attribution (tcl tktview 5d5b1052280c976ea3d4), surfaced in the 'help tcl' warning report (level medium) alongside the existing has_tclbug_* checks. No version gate: the probe matches the characterized wrong/right arm signature, so it flips automatically on a fixed Tcl runtime; a fixed-canary test in ns/cmdtrace.test then fails first, triggering the documented followup (flip GAP pins, update cmdtrace argdoc caveat, record fixed-in version). + ## [0.12.31] - 2026-07-14 - punk::ns 0.6.0: cmdtrace gains -pause 0 for non-interactive use (bypasses the enter-key pause so the marked-up traced-body report can be captured by scripts/tests). The nested-switch line-mismark was characterized with a punk-free minimal repro (identical on Tcl 8.6.17/8.7a6/9.0.3) confirming the bug upstream (core.tcl-lang.org tktview 5d5b1052280c976ea3d4): arm bodies whose split-list index lands on a literal word of the switch command report container-relative lines; new ns/cmdtrace.test pins correct flat/2-word-form marking and GAP-pins the upstream mismarks; cmdtrace argdoc caveat updated with the ticket and pattern. diff --git a/punkproject.toml b/punkproject.toml index 99921f99..4d03dc4e 100644 --- a/punkproject.toml +++ b/punkproject.toml @@ -1,4 +1,4 @@ [project] name = "punkshell" -version = "0.12.31" +version = "0.12.32" license = "BSD-2-Clause" diff --git a/src/modules/punk/lib-999999.0a1.0.tm b/src/modules/punk/lib-999999.0a1.0.tm index 36ab200c..f3ee037f 100644 --- a/src/modules/punk/lib-999999.0a1.0.tm +++ b/src/modules/punk/lib-999999.0a1.0.tm @@ -307,6 +307,97 @@ tcl::namespace::eval punk::lib::check { return [dict create bug $bug bugref f10d91c2d3 description $description level major mitigated $mitigated mitigation $mitigation] } + #pure classifier, separated for testability - facts in, verdict out. + #The bug's signature is a MIXTURE within one switch: the default arm reports its + #single command arm-relative (line 2 of the arm script) while arm 1 reports it + #shifted by (line of the switch command within its containing script - 1), which + #is 3 in the probe layout below. A self-consistent upstream fix of either flavour + #(uniform arm-relative, or uniform container-relative) breaks the signature and + #reports no bug - if upstream instead changes attribution semantics some other + #way, revisit this classifier. + proc tclbug_nestedswitch_tracelines_signature {arm1line defaultline} { + return [expr {$defaultline == 2 && $arm1line == 3}] + } + proc has_tclbug_nestedswitch_tracelines {} { + #Execution-trace line mis-attribution for nested single-block switch arm bodies. + #https://core.tcl-lang.org/tcl/tktview/5d5b1052280c976ea3d4 (reported by punkshell author 2025-11-10) + #For a proc without source-line correlation (e.g dynamically defined - also the regime + #module-loaded punkshell procs trace in), an arm body whose index into the split + #pattern/body list lands on a LITERAL word of the switch command (an option word, --, + #or the block itself) reports enterstep/info-frame lines shifted by (line of the switch + #command within its containing script - 1) - attributed as if the arm body began at the + #switch command's line. Arms whose index lands on a dynamic word or beyond the command's + #word count report correctly arm-relative - so WHICH arms mismark varies with the option + #words used. Verified identical on Tcl 8.6.17 / 8.7a6 / 9.0.3 (2026-07-14). + #Standalone repro/write-up: scriptlib/developer/tcl_switch_traceline_repro.tcl + # + #Behavioural probe (no version gate needed - detection flips automatically when a fixed + #Tcl runtime is in use): a dynamically-defined proc with a nested 'switch -- $c {...}' + #is enterstep-traced once for arm 1 and once for the default arm, and the reported lines + #are matched against the characterized wrong/right signature (see the classifier above). + # + #Followup when upstream fixed (this check starts reporting bug 0 on a new Tcl release): + # - re-verify with scriptlib/developer/tcl_switch_traceline_repro.tcl on that runtime + # - flip the GAP pins in test suite ns/cmdtrace.test + # (cmdtrace_marks_nested_upstream_mismark_GAP and the fixed-canary test there) + # - update the punk::ns::cmdtrace argdoc caveat that cites the ticket + # - note the fixed-in version here + #Consumers: punk::ns::cmdtrace switch-arm line marking accuracy; 'help tcl' warning report. + set pname [namespace current]::Nestedswitch_traceline_probeproc + set lvar [namespace current]::Nestedswitch_traceline_probelines + #the body must be passed via a variable so the probe proc has no source-line + #correlation (the absolute-source regime does not exhibit the mis-attribution) + #inner switch is the 4-word shape (switch -- $string ): per the law, split-list + #body indices 1 and 3 are affected (they land on the literal -- and block words), so the + #middle arm exists purely to push the default arm's body to index 5 - beyond the switch + #command's word count and therefore correctly arm-relative. (A 2-arm probe would put the + #default body at index 3 and BOTH arms would mismark - hiding the signature.) + set probebody { + switch -- [string index $s 0] { + a { + switch -- [string index $s 1] { + 1 { + llength probemark + } + 2 { + llength probeunused + } + default { + llength probemark + } + } + } + default {} + } +} + proc $pname {s} $probebody + set $lvar [list] + set handler [list ::apply [list {pname lvar args} { + set f [::tcl::info::frame -2] + if {[dict exists $f proc] && [dict get $f proc] eq $pname && [dict exists $f cmd]} { + if {[string match "llength probemark*" [dict get $f cmd]]} { + catch {lappend $lvar [dict get $f line]} + } + } + }] $pname $lvar] + set bug 0 + try { + trace add execution $pname enterstep $handler + $pname a1 ;#inner arm 1 + $pname a9 ;#inner default arm + lassign [set $lvar] arm1line defaultline + if {$arm1line ne "" && $defaultline ne ""} { + set bug [tclbug_nestedswitch_tracelines_signature $arm1line $defaultline] + } + } finally { + catch {trace remove execution $pname enterstep $handler} + catch {rename $pname ""} + catch {unset $lvar} + } + set description "Execution traces on a nested single-block switch mis-attribute lines for the arm bodies\nwhose split-list index lands on a literal word of the switch command (options/--/block):\nthose arms report lines as if their body began at the switch command's line, while other\narms report correctly arm-relative. Debug/trace tooling line markers (e.g punk::ns::cmdtrace)\nare wrong for the affected arms - which arms are affected varies with the switch options used." + return [dict create bug $bug bugref 5d5b1052280c976ea3d4 description $description level medium] + } + #has_libbug_* procs report bugs in bundled/vendored library packages rather than the Tcl core. #They are surfaced through the same 'help tcl' warning report as the has_tclbug_* checks. diff --git a/src/modules/punk/lib-buildversion.txt b/src/modules/punk/lib-buildversion.txt index 33df3fab..1655f981 100644 --- a/src/modules/punk/lib-buildversion.txt +++ b/src/modules/punk/lib-buildversion.txt @@ -1,6 +1,7 @@ -0.4.3 +0.5.0 #First line must be a semantic version number #all other lines are ignored. +#0.5.0 - new check pair punk::lib::check::has_tclbug_nestedswitch_tracelines + tclbug_nestedswitch_tracelines_signature (pure facts-in/verdict-out classifier, G-076 console_deadspin style): live behavioural probe of upstream tcl tktview 5d5b1052280c976ea3d4 (execution-trace line mis-attribution for nested single-block switch arm bodies whose split-list index lands on a literal word of the switch command; punkshell-author-reported; punk-free repro scriptlib/developer/tcl_switch_traceline_repro.tcl, identical 8.6.17/8.7a6/9.0.3). The probe defines a dynamic-body proc, enterstep-traces inner arm 1 (affected index 1 -> reports 3) vs the default arm (index 5, beyond the 4-word command -> reports 2) and matches the characterized wrong/right signature - so detection flips automatically on a fixed Tcl (no version gate; followup steps for that day documented in the proc: re-verify with the repro script, flip the ns/cmdtrace.test GAP pins + canary, update the cmdtrace argdoc caveat, note fixed-in version). Surfaced via 'help tcl' warnings (level medium) alongside the other has_tclbug_* checks; probe is self-cleaning (temp proc/var removed in finally). #0.4.3 - G-076: buginfo dicts gain an optional mitigated/mitigation axis (mitigated boolean + mitigation text) orthogonal to level - a check whose defect is covered by a shipped punkshell mitigation reports it so 'help tcl' can render the warning subdued while keeping the severity classification. has_tclbug_console_deadspin populates it: mitigated when punk::repl >= 0.5.0 (console liveness watchdog) is available to the runtime (udp-check-style version discovery without loading), with the watchdog scope described in the mitigation text (non-repl console reads remain exposed). #0.4.2 - G-076: new punk::lib::check::has_tclbug_console_deadspin + tclbug_console_deadspin_applies classifier - version-based detection of the tcl9 windows dead-console defect (f10d91c2d3, root-caused in G-039: dead console never delivered as a fileevent + core reader thread busy-loop). Gate variable check::tclbug_console_deadspin_fixed_in (empty = no fixed release known - all Tcl 9 windows affected); shared by the 'help tcl' warning and repl::start's watchdog arming. #0.4.1 - G-051 call-site adaptation: tclscript analysis accepts cmdinfo's new 'doconly' cmdtype alongside 'notfound' (subcommand-walk stop condition and the dispatchwords bucketing switch) - bucketed identically to notfound to preserve prior analysis behaviour; a future refinement may count doconly (a documented pseudo-command such as 'string is xdigit') as a valid command word. diff --git a/src/tests/modules/AGENTS.md b/src/tests/modules/AGENTS.md index bbc8966d..f64557f3 100644 --- a/src/tests/modules/AGENTS.md +++ b/src/tests/modules/AGENTS.md @@ -41,7 +41,7 @@ Unit tests for editable source modules under `src/modules/`, `src/modules_tcl8/` - `punkcheck/` — punkcheck module tests (install, summarize_install_resultdict, installtrack) - `punk/ansi/` — punk::ansi tests (`testsuites/ansi/`): ansistrip/ansimerge, plus characterization of the ANSI-at-position mechanisms (`ansistring.test`: INDEX/INDEXCODE/INDEXCHAR/RANGE/INSERT grapheme indexing with SGR-prefix merging, INDEXCOLUMNS/COLUMNINDEX double-wide column mapping, trim/VIEW), code splitting invariants (`ta.test`: detect/detectcode distinction, split_codes/split_codes_single/split_at_codes shapes and round-trip) and single-code/effective-state semantics (`codetype.test`: is_sgr_reset/has_sgr_leadingreset, has_any/all_effective, sgr_merge, sequence_type classify), grepstr characterization (`grepstr.test`: return modes incl summarydict (linemap pinned as always-present - the -help says -n-only, reconciliation deferred to the planned hygiene pass), exact highlight SGR wrapping, -n line numbering, invert + empty-highlight strip, -C context/breaks, capture groups, and the tab deficiency: warns once per call on stderr, single-pass tab line survives - the multi-pass mangling is pinned at consumer level in punk/ns corp.test), and untabify characterization (`untabify.test`: -stops int/list/terminal, -with spaces/unicode/custom-pair, multiline, errors, plus the EXPERIMENTAL -plastic elastic-tabstop mode deliberately pinned-as-interim and retained for possible repl editbuf use). Console queries (get_tabstops/get_size + punk::console::tabwidth) are mocked per the overtype renderline.test pattern - they emit live terminal queries that block/error headless. ANSI codes in these tests are literal escape strings so results are colour-state independent - `punk/args/` — punk::args tests (`testsuites/args/`): parsing, choices/choicegroups, forms, rendering/indentation characterization, synopsis display characterization (`synopsis.test`: basic italic argname/`` styling, longopt `--x=` alias forms, literal/literalprefix/stringstartswith/stringendswith type-alternates rendering unitalicised, option alternate parenthesization, multi-element clause display incl `?type?` members and argname tail-word hints, `-typesynopsis` value-element lists and option passthrough incl documenter ANSI, and the small-restricted-choice-set literal rule: 1-3 restricted choices render as unitalicised `|`-joined literals in leader/option/value positions with choicegroups counted, >3 or `-choicerestricted 0` falling back to italics, `-typesynopsis` taking precedence), usage-marking characterization (`usagemarking.test`: -parsedargs/-badarg/-parsestatus/-scheme marking primitives plus goodchoice highlighting of selected/default-in-effect choice words, asserted by SGR-parameter subset against the live colour arrays; the G-049 nocolour/colour-leak GAP pins flipped 2026-07-10 to scheme-statelessness assertions), the G-049 parse-status structure (`parsestatus.test`: punk::args::parse_status overall/per-argument statuses, badarg for type/allocation failures, -caller attribution, errorcode -argspecs stripping), -parsekey characterization (`parsekey.test`: result/received/solos/multis keying, shared-key required satisfaction and defaults, mash-path and prefix-abbreviation keying, plus GAP pins for last-defined-member default precedence, cross-member -multiple value loss, parsekey/optname collision conflation, and values/leaders parsekey breakage - desired-behaviour pins disabled behind punkargsKnownBug in `testsuites/dev/parsekey-knownbugs.test`), and tclcore doc/interpreter behavioural parity (`tclcoreparity.test`, G-054, gated on have_tclcoredocs: 'string is' class choices equal the live-harvested set, per-class docids exist, error-vs-ok agreement across the probe matrix, version-note labels conditional on class presence - expectations derived from the running interpreter, green on 8.6/8.7/9.0; under 8.6 run the file directly via a plain tclkit + tcltest driver since runtests' harness needs newer infrastructure) -- `punk/ns/` — punk::ns tests (`testsuites/ns/`): cmdwhich/cmdinfo/cmd_traverse doc-lookup flow (`cmdflow.test`, G-040 parity), corp proc-retrieval and syntax/untabify interplay (`corp.test`: name edge cases, -ranges/-n line handling, basic-highlight ansistrip equivalence, -untabify spaces/unicode tab-free output, the KNOWN-DEFICIENCY pin for default -untabify none on tabbed bodies - grepstr warns per pass and brace overlays mangle tabbed lines, deterministic under mocked console tabstops - and a ::tcl::CopyDirectory -untabify spaces smoke test; precursor coverage for the planned punk::ns hygiene pass), cmdtrace characterization (`cmdtrace.test`: -pause 0 non-interactive runs, linedict line-mark keys for flat and 2-word-form nested switches as correct-mark guards, and GAP pins for the upstream nested-switch mismark - core.tcl-lang.org tktview 5d5b1052280c976ea3d4, arm bodies whose split-list index lands on a literal switch-command word report container-relative lines; mark tests gated on have_tclcoredocs because cmdtrace's arm-offset correction parses against the ::switch argdoc), and cmdhelp usage-rendering integration (`cmdhelp.test`: scheme selection, goodarg/badarg marking incl type/allocation failures, goodchoice highlighting of supplied/default choice words, alias path, cmdinfo result shape, queried-command failure attribution, and `-return dict` parse-status returns (G-049 - its GAP pins flipped 2026-07-10); remaining GAP pins for pseudo-command cmdtype + space-form docid prefixes (G-051, real `string is` pins behind the have_tclcoredocs constraint), TclOO undocumented-method fallback (G-052), and synopsis marking absence (G-050)) +- `punk/ns/` — punk::ns tests (`testsuites/ns/`): cmdwhich/cmdinfo/cmd_traverse doc-lookup flow (`cmdflow.test`, G-040 parity), corp proc-retrieval and syntax/untabify interplay (`corp.test`: name edge cases, -ranges/-n line handling, basic-highlight ansistrip equivalence, -untabify spaces/unicode tab-free output, the KNOWN-DEFICIENCY pin for default -untabify none on tabbed bodies - grepstr warns per pass and brace overlays mangle tabbed lines, deterministic under mocked console tabstops - and a ::tcl::CopyDirectory -untabify spaces smoke test; precursor coverage for the planned punk::ns hygiene pass), cmdtrace characterization (`cmdtrace.test`: -pause 0 non-interactive runs, linedict line-mark keys for flat and 2-word-form nested switches as correct-mark guards, and GAP pins for the upstream nested-switch mismark - core.tcl-lang.org tktview 5d5b1052280c976ea3d4, arm bodies whose split-list index lands on a literal switch-command word report container-relative lines; mark tests gated on have_tclcoredocs because cmdtrace's arm-offset correction parses against the ::switch argdoc; plus the fixed-canary asserting punk::lib::check::has_tclbug_nestedswitch_tracelines still reports the bug - a live behavioural probe, so a fixed Tcl release fails the canary first and triggers the documented flip workflow), and cmdhelp usage-rendering integration (`cmdhelp.test`: scheme selection, goodarg/badarg marking incl type/allocation failures, goodchoice highlighting of supplied/default choice words, alias path, cmdinfo result shape, queried-command failure attribution, and `-return dict` parse-status returns (G-049 - its GAP pins flipped 2026-07-10); remaining GAP pins for pseudo-command cmdtype + space-form docid prefixes (G-051, real `string is` pins behind the have_tclcoredocs constraint), TclOO undocumented-method fallback (G-052), and synopsis marking absence (G-050)) - `punk/mix/` — punk::mix::cli tests (prune helpers, punkcheck virtual sources), punk::mix::commandset::repo fossil move/rename characterization tests (`testsuites/repo/`, FOSSIL_HOME-isolated; GAP-marked tests pin behaviour G-022 will change), punk::mix::commandset::loadedlib tests (`testsuites/loadedlib/libsearch.test`: 'dev lib.search' match semantics via -return list — wrap-glob default, =exact prefix, case rules, explicit globs, version aggregation — plus the loadedlib 0.2.0 contract: deep discovery by default (deep .tm modules found without -refresh, registration persists), -refresh = genuine re-scan (epoch incr + rediscovery picks up .tm files added to already-scanned dirs), and highlight working without the shell-global a+ alias; shared provisioned child interp sourcing the source-tree libunknown directly — see the file's ORDERING NOTE), and the MULTISHELL polyglot build machinery (`testsuites/scriptwrap/multishell.test`: scriptset wrap via the punk.multishell.cmd template - structure/LF-only/determinism, checkfile 512-byte label validation of fresh wraps AND the committed bin/runtime.cmd, the runtime scriptset round-trip byte-identity pin, and platform-gated execution smoke: cmd.exe→powershell payload on windows, sh payload on unix or via the `wsllinux` capability constraint from `src/tests/testsupport/wslprobe.tcl` - staged to the WSL distro's native filesystem, G-059) - `punk/lib/` — punk::lib tests (`testsuites/lib/`): range/index/parse/compat/interp_sync utilities, G-058 static-baseline seeding (`staticseed.test`: interp_sync_package_paths/snapshot_package_paths propagate a simulated ::punkboot static baseline and seed `load {} ` ifneeded mappings; no-op without a baseline), and the repl command-completeness engine (`commandcomplete.test`: punk::lib::system::incomplete pending-opener stacks - the info-complete quoting quirk progression (`set x "{*}{"` standalone vs in-proc-body), single openers, tabs, escapes, incomplete<->info-complete parity property; pre-repl-refactor characterization, see goals/G-044 detail preserve-list) - `punk/packagepreference/` — punk::packagepreference tests (`testsuites/packagepreference/`): G-058 static-vs-bundled policy (`staticpolicy.test`: require of a baseline package triggers the index scan before resolution so a newer bundled copy wins, static beats older bundled, exact requires of bundled versions stay reachable, missing static mappings get seeded) diff --git a/src/tests/modules/punk/ns/testsuites/ns/cmdtrace.test b/src/tests/modules/punk/ns/testsuites/ns/cmdtrace.test index 6f4a6e50..673880a2 100644 --- a/src/tests/modules/punk/ns/testsuites/ns/cmdtrace.test +++ b/src/tests/modules/punk/ns/testsuites/ns/cmdtrace.test @@ -148,6 +148,24 @@ namespace eval ::testspace { {2 4 15 16 24} ] + test cmdtrace_upstream_bug_check_canary {punk::lib::check::has_tclbug_nestedswitch_tracelines reports the upstream mismark present}\ + -setup { + set result "" + }\ + -body { + #Fixed-canary (added 2026-07-14): the check performs a live behavioural probe of + #tcl tktview 5d5b1052280c976ea3d4, so on a Tcl release containing the upstream fix + #this test fails FIRST. Followup then: flip + #cmdtrace_marks_nested_upstream_mismark_GAP above, update the punk::ns::cmdtrace + #argdoc caveat, and record the fixed-in version in the check proc + #(punk::lib check namespace - see its followup comment). + set d [punk::lib::check::has_tclbug_nestedswitch_tracelines] + lappend result [dict get $d bug] [dict get $d bugref] [dict get $d level] + }\ + -result [list\ + 1 5d5b1052280c976ea3d4 medium + ] + test cmdtrace_marks_error_call {a traced call that raises still reports and counts the error}\ -setup $csetup -body { proc ::testspace::boom {x} {