# G-071 punk::args value-allocation correctness for optional elements (lseq-class arglists) + parse_status -form Status: achieved 2026-07-12 Scope: src/modules/punk/args-999999.0a1.0.tm (get_dict value allocation, private::get_dict_can_assign_value, parse_status), src/tests/modules/punk/args/testsuites/args/ (new allocation characterization suite), src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (::lseq as the proving consumer) Goal: argument lists that are valid for a single form parse correctly when optional standalone values and optional-member clauses are skipped or filled in any documented combination - specifically the lseq range shape (an optional single-word choice value between required values, followed by a trailing optional-member clause), where today the allocator force-feeds a word to the skippable optional and fails ('lseq 0 10 2', '0 10 by 2', '1 5 by 0' all fail in-form while '0 to 10 2' parses) - and parse failures blame the genuinely failing element rather than an unrelated optional; parse_status gains -form so per-form status probing works. Acceptance: a new allocation characterization suite drives the lseq range matrix under explicit -form range - '0 10', '0 10 2', '0 10 by 2', '0 to 10 2', '0 .. 10 by 2', '1 5 by 0' parse per the lseq.n grammar (the three currently-failing cases fixed) and '0 10 2 4' still fails - plus reduced fixtures isolating the shape (optional choice value between required values + trailing optional-member clause) independent of the moduledoc; the ::if noise-word cases (mid-clause ?literal(then)?, clause-leading ?literal(else)?) keep passing - no regression to optional clause members generally; a genuinely invalid arglist's error names the failing element (the '..|to' misblame case pinned fixed); punk::args::parse_status accepts -form (single form name/index at minimum, consistent with parse) with its status structure reporting the form used; full punk::args and punk::ns suites pass with no expectations weakened; before/after results for the probe matrix recorded in this file. ## Context Found 2026-07-12 during G-041 activation prework (user concern: variable-length clauses with optional elements). Real-vs-model probing of the tclcore moduledoc showed the per-form models are doc-faithful (lseq.n's three synopsis lines map 1:1 to the modelled range/start_count/count forms; if/switch/try likewise verified), but the range form fails IN-FORM whenever the ../to noise word is absent and a step is present - the allocator insists on assigning the word after start to the optional '..|to' choice value (whose choices then reject it) instead of skipping the optional and letting the trailing "by step" clause (-type {?literalprefix(by)? number|expr}) absorb the remainder. The start_count form proves the optional-member clause itself can absorb a lone word ('1 count 5 2' parses as {by step} {{} 2}) when preceding words are pinned by a literal - the failure is the combination with a skippable standalone optional value. No other lseq form accepts '0 10 2', so G-041's form auto-selection cannot fix 'i lseq 0 10 2' without this - making this goal a G-041 prerequisite. Probe results are recorded in goals/G-041-punkargs-form-matching.md Notes (2026-07-12). parse_status -form: per-form probing during the same session had to fall back to punk::args::parse + catch because parse_status does not accept -form. The form-candidacy API sketched in G-041 (and the G-044 partial-arglist consumer) wants exactly this surface, so it lands here where the characterization suite needs it first. ## Approach - Characterize first: reduced fixtures pinning current allocation behaviour for the failing shape (and the working ::if shapes as regression guards), then fix the allocator - likely in the lookahead/backtrack decision where get_dict_can_assign_value rejects an optional's candidate word: rejection of an optional element should yield the word to subsequent elements/clauses rather than failing the parse when a consistent allocation exists. - Blame quality: when no consistent allocation exists, prefer reporting the element whose constraint actually failed over the first optional tried. - The G-041 detail file notes the form-attempt logic will exercise these paths heavily; landing this first keeps that goal's failures meaningful. ## Progress ### 2026-07-12 increment 1: characterization suite + retreat-path debug silencing (punk::args 0.8.3) - New allocation.test (7 tests): reduced lseq-range fixture (start ?sep? end ?by-step clause? - no moduledoc dependency) with the current mis-allocation pinned as GAPs (allocation_range_optskip_bare_step_GAP '1 2 3' -> invalid/sep, allocation_range_optskip_by_clause_GAP '1 2 by 3' -> incomplete/end, allocation_range_excess_invalid_misblame_GAP '1 2 3 4' -> invalid but blaming 'sep'), noise-word variants pinned as regression guards (all parse with expected receivednames), the if-shape noise-word guards (mid-clause ?literal(then)?, clause-leading ?literal(else)? - 7 valid + 1 invalid case), and parsestatus_form_option_GAP pinning that -form raises today. - Fixture probing confirmed the reduced shape reproduces the moduledoc failures exactly (same badarg, same status classes), so the fix can iterate against the fixtures alone. - Silenced four more unconditional debug puts on get_dict's allocation retreat paths ((111)/(222)/(333)/(444)) - they fired on stderr during NORMAL successful parses at every optional-skip retreat (visible when the fixture probes ran; also polluting interactive shell parses of if/lseq-modelled commands). Behaviour unchanged - the retreat logic they marked is this goal's fix target. - Verified (tclsh 9.0.3): full punk::args suite 193 pass / 1 pre-existing skip / 0 fail. ### 2026-07-12 increment 2: allocation choice screen (punk::args 0.9.0) - acceptance complete - Root cause: get_dict_can_assign_value blanket-satisfied the member check for any argument with choices ("each tp in the clause is just for validating a value outside the choice-list when -choicerestricted 0" - applied to restricted sets too), so an optional choice value consumed any word whenever arity permitted. Arity always permitted here because (a) the tail-reservation scan cannot see the by-step clause (its lsearch for literal* misses the optional-wrapped ?literalprefix(by)?), and (b) tail_needs counts only required tail elements. - Fix: a choiceword_match allocation screen (the shared G-040 implementation, so allocation acceptance cannot diverge from parse acceptance - exact/alias/prefix/ nocase; -choicemultiple words screened per list member within min/max), applied ONLY where allocation has an alternative: -optional arguments and further occurrences of -multiple arguments. Two refinements forced by the existing suites during implementation, both kept as design decisions: - REQUIRED arguments are not screened - the word must fill them regardless, and screening only masked validation's informative choiceviolation as a missingrequired* (caught by parsestatus.test/cmdhelp.test pins). - -choicemultiple list-valued words are screened member-wise (caught by choices.test choicemultiple pins). The screen compares raw words (no ansistrip): an ansi-wrapped choice word on an optional argument would be skipped where validation would accept it - accepted edge, recorded in the code comment. - Correction of an increment-1 mis-premise: parse_status ALREADY accepts -form - positioned before the withid/withdef tail per its documented synopsis; the probing failure was a trailing -form (argument-order mistake). The GAP was replaced by parsestatus_form_option_order pinning correct-order acceptance (with the status structure's form key) and trailing-order rejection. The acceptance clause "parse_status accepts -form ... reporting the form used" was therefore already satisfied; verified with a multiform fixture (form=beta reported). The Goal line's "parse_status gains -form" framing was inaccurate - no code change was needed or made for it. - Before/after (reduced fixture, tclsh 9.0.3): '1 2 3' {0 invalid sep} -> {1 valid {start end {by step}}} with clause {{} 3}; '1 2 by 3' {0 incomplete end} -> {1 valid ...} with clause {by 3}; '1 2 3 4' {0 invalid sep} -> {0 invalid {}} (same excess-values rejection as the noise-word twin); prefix semantics preserved ('1 2 b 3' -> clause {by 3}, '1 t 2 3' -> sep normalized to 'to'); all noise-word variants and all if-shape guards unchanged. - lseq moduledoc matrix (punk902z src, Tcl 9.0.2): explicit -form range now parses '0 10 2', '0 10 by 2', '1 5 by 0' per lseq.n (previously in-form failures); count and start_count forms unchanged; default-form parsing of '0 10 2' now valid end-to-end (the original 'i lseq 0 10 2' complaint). Remaining real-vs-model divergences all trace to G-041 form selection or the -type expr non-validation recorded in G-055's notes - not allocation. - Verified (tclsh 9.0.3): punk::args suite 193 pass / 1 pre-existing skip / 0 fail (allocation GAPs flipped; the two required-choice-arg pins and the choicemultiple pins unweakened); punk::ns suite 53/53; full source-tree suite 827 total, 813 pass, 13 skip, 1 fail = exec-14.3 only (known pre-existing core-test baseline) - zero regressions; make.tcl modules builds clean. ## Notes - Related: G-041 (this is its prerequisite; probe evidence in its Notes), G-053 (occurrence-arity work touches adjacent allocation machinery), G-055 (lseq parity pins; TIP 746 note there makes lseq operand types version-conditional - this goal's matrix should derive expectations from the live interpreter). - Incidental fixes already landed during the probing session: unconditional debug puts on the clause type-check path (punk::args 0.8.2), and an earlier companion (0.2.3, noted in G-041).