Browse Source

G-071 increment 1: allocation characterization suite + retreat-path debug silencing (punk::args 0.8.3, project 0.12.10)

G-071 flipped proposed -> active (user direction 2026-07-12).

New allocation.test (7 tests) with reduced fixtures isolating the failing
shape from the tclcore moduledoc: the lseq-range shape (start ?sep? end
?by-step clause?) reproduces the moduledoc failures exactly. Current
mis-allocation pinned as GAP tests ("1 2 3" -> invalid blaming sep,
"1 2 by 3" -> incomplete blaming end, "1 2 3 4" correctly invalid but
misblaming sep), noise-word variants and the if-shape noise-word clauses
pinned as regression guards, parse_status -form absence pinned as GAP.

Also silenced four more unconditional debug puts on get_dict allocation
retreat paths ((111)/(222)/(333)/(444)) - these printed to stderr during
NORMAL successful parses at every optional-skip retreat, polluting
interactive help/parse output for noise-word-modelled commands. Behaviour
unchanged; the retreat logic they marked is exactly this goal fix target.

Full punk::args suite 193 pass / 1 pre-existing skip / 0 fail (tclsh
9.0.3).

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 2 days ago
parent
commit
17b8a6dbbd
  1. 4
      CHANGELOG.md
  2. 2
      GOALS.md
  3. 35
      goals/G-071-punkargs-optional-allocation.md
  4. 2
      punkproject.toml
  5. 8
      src/modules/punk/args-999999.0a1.0.tm
  6. 3
      src/modules/punk/args-buildversion.txt
  7. 144
      src/tests/modules/punk/args/testsuites/args/allocation.test

4
CHANGELOG.md

@ -5,6 +5,10 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to
Entries are newest-first; one bullet per notable change. See the root `AGENTS.md`
"Project Versioning" section for the bump policy.
## [0.12.10] - 2026-07-12
- punk::args 0.8.3 (G-071 increment): silenced four more unconditional debug lines that printed to stderr during normal argument parsing whenever an optional element was skipped (visible when using 'i'/help display against commands modelled with noise-word arguments such as if and lseq). New allocation characterization testsuite pins the optional-element mis-allocation ('lseq 0 10 2'-class failures) as GAP tests ahead of the allocator fix.
## [0.12.9] - 2026-07-12
- punk::args 0.8.2: fixed stray debug output ("checking tp ... against value ...") printed to the console when parsing multi-element clause arguments (e.g. 'i try ...' style usage against the tclcore moduledoc). Found during G-041 prework probing of if/switch/try/lseq real-vs-model divergence; findings recorded in the G-041 and G-055 goal detail files.

2
GOALS.md

@ -286,7 +286,7 @@ Detail: goals/G-069-splitter-tclparser-lint.md
Scope: src/modules/punk/lib-999999.0a1.0.tm (tclparser_tcl stub + dispatch; new module if size warrants - decided in the work), src/tests/modules/punk/lib/ (parity + fallback suites), TEMP_REFERENCE/ (tclparser reference source, user-provided, read-only)
Detail: goals/G-070-pure-tcl-tclparser.md
### G-071 [proposed] punk::args value-allocation correctness for optional elements (lseq-class arglists) + parse_status -form
### G-071 [active] punk::args value-allocation correctness for optional elements (lseq-class arglists) + parse_status -form
Scope: src/modules/punk/args-999999.0a1.0.tm (get_dict value allocation, private::get_dict_can_assign_value, parse_status), src/tests/modules/punk/args/testsuites/args/ (new allocation characterization suite), src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (::lseq as the proving consumer)
Detail: goals/G-071-punkargs-optional-allocation.md

35
goals/G-071-punkargs-optional-allocation.md

@ -1,6 +1,6 @@
# G-071 punk::args value-allocation correctness for optional elements (lseq-class arglists) + parse_status -form
Status: proposed
Status: active
Scope: src/modules/punk/args-999999.0a1.0.tm (get_dict value allocation, private::get_dict_can_assign_value, parse_status), src/tests/modules/punk/args/testsuites/args/ (new allocation characterization suite), src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (::lseq as the proving consumer)
Goal: argument lists that are valid for a single form parse correctly when optional standalone values and optional-member clauses are skipped or filled in any documented combination - specifically the lseq range shape (an optional single-word choice value between required values, followed by a trailing optional-member clause), where today the allocator force-feeds a word to the skippable optional and fails ('lseq 0 10 2', '0 10 by 2', '1 5 by 0' all fail in-form while '0 to 10 2' parses) - and parse failures blame the genuinely failing element rather than an unrelated optional; parse_status gains -form so per-form status probing works.
Acceptance: a new allocation characterization suite drives the lseq range matrix under explicit -form range - '0 10', '0 10 2', '0 10 by 2', '0 to 10 2', '0 .. 10 by 2', '1 5 by 0' parse per the lseq.n grammar (the three currently-failing cases fixed) and '0 10 2 4' still fails - plus reduced fixtures isolating the shape (optional choice value between required values + trailing optional-member clause) independent of the moduledoc; the ::if noise-word cases (mid-clause ?literal(then)?, clause-leading ?literal(else)?) keep passing - no regression to optional clause members generally; a genuinely invalid arglist's error names the failing element (the '..|to' misblame case pinned fixed); punk::args::parse_status accepts -form (single form name/index at minimum, consistent with parse) with its status structure reporting the form used; full punk::args and punk::ns suites pass with no expectations weakened; before/after results for the probe matrix recorded in this file.
@ -43,6 +43,39 @@ needs it first.
- The G-041 detail file notes the form-attempt logic will exercise these paths
heavily; landing this first keeps that goal's failures meaningful.
## Progress
### 2026-07-12 increment 1: characterization suite + retreat-path debug silencing (punk::args 0.8.3)
- New allocation.test (7 tests): reduced lseq-range fixture (start ?sep? end
?by-step clause? - no moduledoc dependency) with the current mis-allocation
pinned as GAPs (allocation_range_optskip_bare_step_GAP '1 2 3' -> invalid/sep,
allocation_range_optskip_by_clause_GAP '1 2 by 3' -> incomplete/end,
allocation_range_excess_invalid_misblame_GAP '1 2 3 4' -> invalid but blaming
'sep'), noise-word variants pinned as regression guards (all parse with
expected receivednames), the if-shape noise-word guards (mid-clause
?literal(then)?, clause-leading ?literal(else)? - 7 valid + 1 invalid case),
and parsestatus_form_option_GAP pinning that -form raises today.
- Fixture probing confirmed the reduced shape reproduces the moduledoc failures
exactly (same badarg, same status classes), so the fix can iterate against the
fixtures alone.
- Silenced four more unconditional debug puts on get_dict's allocation retreat
paths ((111)/(222)/(333)/(444)) - they fired on stderr during NORMAL
successful parses at every optional-skip retreat (visible when the fixture
probes ran; also polluting interactive shell parses of if/lseq-modelled
commands). Behaviour unchanged - the retreat logic they marked is this goal's
fix target.
- Verified (tclsh 9.0.3): full punk::args suite 193 pass / 1 pre-existing skip /
0 fail.
Remaining for acceptance:
- allocator fix: optional-element rejection yields the word to subsequent
elements/clauses when a consistent allocation exists (the three GAPs flip)
- blame quality: misblame GAP flips to naming the genuinely failing element
- parse_status -form support (its GAP flips)
- lseq moduledoc matrix verified under -form range post-fix; before/after
results recorded here; punk::ns suite regression pass
## Notes
- Related: G-041 (this is its prerequisite; probe evidence in its Notes), G-053

2
punkproject.toml

@ -1,4 +1,4 @@
[project]
name = "punkshell"
version = "0.12.9"
version = "0.12.10"
license = "BSD-2-Clause"

8
src/modules/punk/args-999999.0a1.0.tm

@ -10761,7 +10761,7 @@ tcl::namespace::eval punk::args {
set newtypelist [dict get $assign_d typelist]
if {$consumed == 0} {
if {[tcl::dict::get $argstate $leadername -optional]} {
puts stderr "get_dict cannot assign val:$ldr to leadername:$leadername leaders:$leaders (111)"
#puts stderr "get_dict cannot assign val:$ldr to leadername:$leadername leaders:$leaders (111)"
#return -options [list -code error -errorcode [list PUNKARGS UNCONSUMED -argspecs $argspecs]] "private::get_dict_can_assign_value consumed 0 unexpected 1?"
incr ldridx -1
set leadername_multiple ""
@ -10774,7 +10774,7 @@ tcl::namespace::eval punk::args {
set msg "Bad number of leaders for %caller%. Not enough remaining values to assign to required arguments (fail on $leadername)."
return -options [list -code error -errorcode [list PUNKARGS VALIDATION [list missingrequiredleader $leadername ] -badarg $leadername -argspecs $argspecs]] $msg
} else {
puts stderr "get_dict cannot assign val:$ldr to leadername:$leadername (222)"
#puts stderr "get_dict cannot assign val:$ldr to leadername:$leadername (222)"
#return -options [list -code error -errorcode [list PUNKARGS UNCONSUMED -argspecs $argspecs]] "private::get_dict_can_assign_value consumed 0 unexpected 2?"
incr ldridx -1
set leadername_multiple ""
@ -10973,7 +10973,7 @@ tcl::namespace::eval punk::args {
if {$consumed == 0} {
if {[tcl::dict::get $argstate $valname -optional]} {
#error 333
puts stderr "get_dict cannot assign val:$val to valname:$valname (333)"
#puts stderr "get_dict cannot assign val:$val to valname:$valname (333)"
incr validx -1
set valname_multiple ""
incr nameidx
@ -10986,7 +10986,7 @@ tcl::namespace::eval punk::args {
return -options [list -code error -errorcode [list PUNKARGS VALIDATION [list missingrequiredvalue $valname ] -badarg $valname -argspecs $argspecs]] $msg
} else {
#error 444
puts stderr "get_dict cannot assign val:$val to valname:$valname (444)"
#puts stderr "get_dict cannot assign val:$val to valname:$valname (444)"
incr validx -1
set valname_multiple ""
incr nameidx

3
src/modules/punk/args-buildversion.txt

@ -1,6 +1,7 @@
0.8.2
0.8.3
#First line must be a semantic version number
#all other lines are ignored.
#0.8.3 - G-071: silenced four more unconditional debug puts on get_dict's allocation retreat paths ("get_dict cannot assign val:... (111)/(222)/(333)/(444)") - these fired on stderr during NORMAL successful parses whenever an optional element's candidate word was rejected and allocation retreated (e.g. every noise-word skip when parsing if/lseq-modelled commands), polluting interactive output. Same class as the 0.2.3/0.8.2 finds. The retreat logic they marked is the target of the G-071 allocator-correctness work; behaviour unchanged, output only. New allocation.test characterization suite (reduced lseq-range/if-shape fixtures): current mis-allocation and misblame pinned as GAP tests (flip with G-071), noise-word variants and if-shape guards pinned as regression guards, parse_status -form absence pinned as GAP.
#0.8.2 - fixed stray debug output: an unconditional 'puts' in private::get_dict_can_assign_value ("checking tp '<type>' against value '<value>'") fired on multi-element clause type checks (e.g. parsing 'try ... trap ... on ...' style arglists against the tclcore moduledoc), polluting interactive output - now commented like its 0.2.3 companion. Found during G-041 prework probing (real-vs-model divergence sweep of if/switch/try/lseq).
#0.8.1 - G-045 documentation-only: the container quoting rules pinned by defquoting.test promoted into the define -help documentation (-help key section): braced values fully literal (backslash sequences survive as typed) with only tstr placeholders and their backslash escape special; double-quoted values get Tcl backslash semantics at record parse (\n -> newline, doubled backslash collapses) while $ and [] stay literal with no variable/command substitution outside tstr placeholders; the backslash-escaped placeholder idiom renders a literal placeholder.
#0.8.0 - G-045: new bare @normalize directive - opts a definition into indent normalization of BLOCK-FORM multi-line field values (first line whitespace-only, as authored by opening a braced literal with a newline): the structural first newline and a whitespace-only trailing line are dropped, the content lines' common leading whitespace is the block's base indent, the first content line is unindented fully and subsequent lines re-based to the standard 4-space continuation convention (deeper relative indents preserved; whitespace-only inner lines become empty). Head-form values are never altered - their base indent is ambiguous (uniform continuation indent may be the deliberate +2 relative convention over base 4) - which makes @normalize a no-op on conforming file-style definitions (idempotence pinned). Fields in a record's -unindentedfields are exempt. Implemented as a resolve pre-pass over split records (private::normalize_records / rebase_multiline_value); @normalize with options is an error. Intended for constructed/string-built definitions (no whole-block indent treatment otherwise); ::punk::helptopic::define_docs converts to it as the consumer proof (punk 0.2.5). New testsuite normalize.test; define doc documents the directive; rendering.test P4 notes stay pinned as the unopted default.

144
src/tests/modules/punk/args/testsuites/args/allocation.test

@ -0,0 +1,144 @@
package require tcltest
package require punk::args
#Value-allocation characterization for optional standalone values and
#optional-member clauses (G-071) - added 2026-07-12 before any allocator changes
#(tests-first per the punk::args convention). Reduced fixtures isolate the
#lseq-range shape (start ?sep? end ?by-step clause?) and the if shape (noise-word
#clauses) from the tclcore moduledoc, so the pins do not depend on moduledoc
#availability or vintage.
#
#GAP-labelled tests pin the CURRENT broken allocation and flip when G-071 lands:
#with the optional 'sep' noise word absent, the allocator force-assigns the word
#after start to 'sep' (whose -choices then reject it) instead of yielding it to
#later elements/clauses - so '1 2 3' and '1 2 by 3' fail in-form while every
#variant carrying the noise word parses. '1 2 3 4' is genuinely invalid but the
#blame lands on 'sep' rather than the failing clause (misblame pin).
namespace eval ::testspace {
namespace import ::tcltest::*
variable common {
set result ""
punk::args::define {
@id -id ::testspace::alloc_range
@values -min 2 -max 5
start -type int
sep -type string -choices {.. to} -optional 1
end -type int
"by step" -type {?literalprefix(by)? int} -optional 1
}
punk::args::define {
@id -id ::testspace::alloc_if
@values -min 2 -max -1
expr1 -type int -optional 0
then -type literal(then) -optional 1
body1 -type int -optional 0
"elseif_clause" -type {literal(elseif) int ?literal(then)? int} -optional 1 -multiple 1
"else_clause" -type {?literal(else)? int} -optional 1 -multiple 0
}
}
variable cleanup {
punk::args::undefine ::testspace::alloc_range 1
punk::args::undefine ::testspace::alloc_if 1
}
#ok/status plus badarg (failed) or receivednames (ok) - compact per-case signature
proc pstat {arglist id} {
set st [punk::args::parse_status $arglist withid $id]
if {[dict get $st ok]} {
return [list 1 [dict get $st status] [dict get $st receivednames]]
}
return [list 0 [dict get $st status] [dict get $st badarg]]
}
#added 2026-07-12 (agent, G-071) - regression guards: allocation with the
#optional noise word present (or no step) works today and must keep working
test allocation_range_noiseword_variants {lseq-range shape: variants with the sep noise word present (or only start end) allocate correctly}\
-setup $common -body {
lappend result [pstat {1 2} ::testspace::alloc_range]
lappend result [pstat {1 to 2} ::testspace::alloc_range]
lappend result [pstat {1 to 2 3} ::testspace::alloc_range]
lappend result [pstat {1 .. 2 by 3} ::testspace::alloc_range]
lappend result [pstat {1 to 2 by 3} ::testspace::alloc_range]
}\
-cleanup $cleanup\
-result [list\
{1 valid {start end}}\
{1 valid {start sep end}}\
{1 valid {start sep end {by step}}}\
{1 valid {start sep end {by step}}}\
{1 valid {start sep end {by step}}}\
]
#added 2026-07-12 (agent, G-071)
test allocation_range_optskip_bare_step_GAP {GAP: sep absent + bare step ('1 2 3') fails - word 2 force-fed to the skippable optional 'sep' instead of yielding to end + clause}\
-setup $common -body {
pstat {1 2 3} ::testspace::alloc_range
}\
-cleanup $cleanup\
-result {0 invalid sep}
#added 2026-07-12 (agent, G-071)
test allocation_range_optskip_by_clause_GAP {GAP: sep absent + by-clause ('1 2 by 3') fails as incomplete blaming 'end'}\
-setup $common -body {
pstat {1 2 by 3} ::testspace::alloc_range
}\
-cleanup $cleanup\
-result {0 incomplete end}
#added 2026-07-12 (agent, G-071) - '1 2 3 4' must STAY invalid when the
#allocator is fixed (per lseq.n no form accepts it) but the blame should move
#off the unrelated optional 'sep' to the genuinely failing element
test allocation_range_excess_invalid_misblame_GAP {GAP: genuinely invalid '1 2 3 4' is rejected (correct) but blames the unrelated optional 'sep' (misblame)}\
-setup $common -body {
pstat {1 2 3 4} ::testspace::alloc_range
}\
-cleanup $cleanup\
-result {0 invalid sep}
#added 2026-07-12 (agent, G-071) - guard: with the noise word present the
#equivalent excess arglist is rejected without touching 'sep'
test allocation_range_excess_with_noiseword_invalid {lseq-range shape: '1 to 2 3 4' rejected (excess values)}\
-setup $common -body {
set st [punk::args::parse_status {1 to 2 3 4} withid ::testspace::alloc_range]
list [dict get $st ok] [dict get $st status]
}\
-cleanup $cleanup\
-result {0 invalid}
#added 2026-07-12 (agent, G-071) - regression guards: the ::if noise-word
#shapes (mid-clause ?literal(then)?, clause-leading ?literal(else)?) allocate
#correctly today and must not regress with the allocator fix
test allocation_if_noiseword_guards {if shape: mid-clause and clause-leading optional literals allocate correctly}\
-setup $common -body {
foreach c {
{1 2}
{1 then 2}
{1 2 else 3}
{1 2 3}
{1 2 elseif 3 4}
{1 2 elseif 3 then 4 else 5}
{1 then 2 elseif 3 then 4 elseif 5 6 else 7}
} {
set st [punk::args::parse_status $c withid ::testspace::alloc_if]
lappend result [list [dict get $st ok] [dict get $st status]]
}
#incomplete elseif clause is rejected
set st [punk::args::parse_status {1 2 elseif 3} withid ::testspace::alloc_if]
lappend result [list [dict get $st ok] [dict get $st status]]
}\
-cleanup $cleanup\
-result [list {1 valid} {1 valid} {1 valid} {1 valid} {1 valid} {1 valid} {1 valid} {0 invalid}]
#added 2026-07-12 (agent, G-071)
test parsestatus_form_option_GAP {GAP: parse_status does not accept -form - per-form status probing raises instead of parsing}\
-setup $common -body {
set rc [catch {punk::args::parse_status {1 to 2} withid ::testspace::alloc_range -form 0} r]
list $rc [string match "Bad number of values*" $r]
}\
-cleanup $cleanup\
-result {1 1}
cleanupTests
}
namespace delete ::testspace
Loading…
Cancel
Save