punk::ns 0.11.1: G-174 cmd_traverse word accounting - doc-lookup walk never drops argument words (project 0.66.3)
The walk behind cmdinfo dropped all but the last trailing word for commands
whose definition's first form is zero-arg with no leaders (the punk::ansi::a?
class): the empty-leader position fell through to the outer per-word loop,
whose exhaustion return reported only the last tail snapshot - dropped words
landed in neither args_resolved nor args_remaining, so cmdhelp's advisory
parse cleanly mis-narrowed 'i a? web green' to the loose sample form with no
failure signal anywhere.
- cmd_traverse: the empty-leader position returns new lane 11 with the
unconsumed tail (remaining shape mirrors the code-3 lane); word-accounting
contract stated at the record comment (conservation; sanctioned exceptions:
ensemble-parameter re-presentation, alias/script currying; first-form-only
inspection recorded as a limit); stale code-7 witness comment corrected
('i namespace which -v x' actually terminates in the code-8 lane).
- cmdflow.test: traverse_final/word_accounting helpers, fixtures
flowmultiform (a?-shaped) / flowmultilead / flownomap / flowplainlead,
lane-code + conservation battery across return lanes 0a,2,X,3,4,6,7,8,9,10,11
(code-7 and code-10 witnessed for the first time), multi-word no-drop pins,
the real-world a? witness, and the first-form-only recorded-limit pin.
- cmdhelp.test: 'a? web green' -return dict (multipleformmatches, formstatus
keys {web sample}) and -return text (web's argument section before sample's)
presentation pins, plus the punk::ns::synopsis underline-marking pin.
- Witness: pre-fix, 8 of the 9 new tests fail (5 cmdflow + 3 cmdhelp);
post-fix punk/ns subtree green on canonical tclsh90s (138 pass, 2
pre-existing gated skips), cmdflow+cmdhelp green on tclsh86ts (86/86).
- G-174 activated on user direction, flipped achieved 2026-08-09 and archived
(entry to GOALS-archive.md, detail to goals/archive/ with Follow-ons);
suite-index DOX updated; project 0.66.3 + CHANGELOG entry.
Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
@ -20,6 +20,10 @@ Do not edit archived records here except to fix a broken path. Archived detail f
## Archived goals
### G-174 [achieved 2026-08-09] cmd_traverse word accounting - the doc-lookup walk never drops argument words → detail: goals/archive/G-174-cmdtraverse-word-accounting.md
Acceptance: characterisation tests in cmdflow.test assert the conservation invariant (args_resolved plus args_remaining account for every supplied trailing word, each word in exactly one) across the walk's return lanes - including an a?-shaped fixture (plain proc, multiform definition, zero-arg first form) with multi-word tails, plus witnesses for the previously unwitnessed code-7 and code-10 return lanes; punk::ns::cmdhelp for 'a? web green' runs its advisory parse on the full tail and presents the web/sample multipleformmatches ambiguity (web's argument table rendered first, both forms marked in the synopsis) rather than a clean sample-only render, and punk::ns::synopsis marks the same forms for the same words; the first-form-only inspection (fid = first of form_names) is either extended or pinned as a recorded limit with its own witness, with the decision recorded in this file; the punk::ns suite passes under the canonical runtests interpreter with no regressions in cmdflow/cmdhelp.
Acceptance: characterisation tests of the CURRENT libunknown override landed before any behaviour change and stayed green byte-identically through the migration (epoch single-arg/indexed forms + error arms, the forget arm, abbreviations, delegation, re-init guard, packagepreference interop in both install orders) with only marked install-shape pins flipping at the migration; libunknown installs its ::package override via commandstack::rename_command with a stack-query guard and no ::package:: residue; rename_command -punkargs docs are queryable while the record lives (incl renames performed BEFORE punk::args loads - pinned in a bare child) and are removed by remove_rename/pop_rename/remove_renamer/restore_original alike; cmdhelp shows an override-in-place notice (cmdinfo 'overridden' second-axis key, no hard commandstack dependency) enumerating record-attached doc ids; 'i package epoch' resolves the libunknown-attached doc and the how-to is described in commandstack::help + the rename_command argdoc; punk/libunknown, commandstack, punk/ns and punk/packagepreference suites pass under the canonical runtests interpreter.
Goal: the doc-lookup walk behind punk::ns::cmdinfo has a stated, characterised word-accounting contract - every supplied trailing word lands in exactly one of args_resolved or args_remaining - so args_remaining-sensitive consumers (cmdhelp's advisory parse and G-041 form narrowing, synopsis form marking) always operate on the full unconsumed tail, and a command whose definition's first form is zero-arg (the punk::ansi::a? class) no longer has its literal discriminator words silently dropped before form narrowing runs.
Acceptance: characterisation tests in cmdflow.test assert the conservation invariant (args_resolved plus args_remaining account for every supplied trailing word, each word in exactly one) across the walk's return lanes - including an a?-shaped fixture (plain proc, multiform definition, zero-arg first form) with multi-word tails, plus witnesses for the previously unwitnessed code-7 and code-10 return lanes; punk::ns::cmdhelp for 'a? web green' runs its advisory parse on the full tail and presents the web/sample multipleformmatches ambiguity (web's argument table rendered first, both forms marked in the synopsis) rather than a clean sample-only render, and punk::ns::synopsis marks the same forms for the same words; the first-form-only inspection (fid = first of form_names) is either extended or pinned as a recorded limit with its own witness, with the decision recorded in this file; the punk::ns suite passes under the canonical runtests interpreter with no regressions in cmdflow/cmdhelp.
@ -116,3 +116,90 @@ defect lives precisely in the uncovered lanes.
multipleformmatches with per-form formstatus (all-valid keys) and noformmatch
with ranked -formerrors; cmdhelp consumes them correctly. No punk::args change
is expected under this goal.
- Related (activation overlap survey 2026-08-09, goals_xref score G-174):
G-050 (punk::ns::synopsis argument-validity marking) - the synopsis marking
consumer this goal feeds the full tail; G-174 pins WHICH forms are marked,
regressions in cmdflow/cmdhelp); cmdflow+cmdhelp green under tclsh86ts
(86/86, the 8.6 leg); testbody_lint clean over the ns testsuite dir (7 files,
140 tests, 0 findings).
Acceptance satisfied in full; flipped achieved 2026-08-09.
## Follow-ons
Follow-on: cross-form traversal discrimination - extend cmd_traverse's first-form-only inspection (fid = first of form_names) so choices-bearing subcommand leaders on later forms are consulted during the walk; flip cmdflow.test cmdtraverse_firstformonly_recorded_limit when it lands => open
Follow-on: tighten punk::ansi::a?'s loose 'sample' form so discriminator words do not ambiguate => declined 2026-08-09 (the marked-ambiguity presentation is the G-041 multipleformmatches contract - drafting-time default confirmed at landing)
Follow-on: eparam double-listing corner - with nonempty eparams the code-3/code-11 remaining shape prepends already-consumed -ensembleparameter words beyond the sanctioned descent re-presentation (hand-authored -ensembleparameter-then-choiceless-leader first forms only; no shipped definition has the shape) => open
#0.11.1 - G-174 cmd_traverse word accounting: the doc-lookup walk behind cmdinfo no longer silently drops trailing words for commands whose definition's FIRST form is zero-arg with no leaders (the punk::ansi::a? class). The empty-leader position in the subcommand walk (the annotated '#todo - return?' break) now returns the unconsumed tail as new lane code 11 (remaining shape mirroring the code-3 lane) instead of falling back to the outer per-word loop, whose exhaustion return reported only the LAST tail snapshot - with N trailing words, N-1 landed in NEITHER args_resolved NOR args_remaining. Consumers operating on args_remaining now see the full unconsumed tail: cmdhelp's advisory parse for 'i a? web green' reports the genuine web/sample multipleformmatches ambiguity (web's argument table first, both forms marked) instead of a clean sample-only render, and punk::ns::synopsis marks the same forms. Word accounting contract stated at the cmd_traverse record comment: every supplied trailing word lands in exactly one of resolvedargs/remainingargs (sanctioned exceptions: -ensembleparameter re-presentation, alias/script currying). First-form-only inspection (fid = first of form_names) pinned as a recorded limit - cross-form traversal discrimination is a possible follow-on. Pinned by cmdflow.test's G-174 section (lane-code + conservation battery incl the previously unwitnessed code-7/code-10 lanes, a?-shaped fixtures) and cmdhelp.test's G-174 section (dict/text ambiguity presentation, synopsis form marking).
#0.11.0 - G-176 increment 4: cmdinfo gains an 'overridden' result key - a second axis beside cmdtype/unavailable (G-166 style) reporting the LIVE commandstack override state of the base command: one {renamer <string> docids <list>} dict per stack record (bottom-up), docids being the punk::args ids of doc blocks the record attached via 'rename_command -punkargs' (derived with punk::args::rawdef_id). Guarded live query (info commands on commandstack::get_stack) - empty list when commandstack is not loaded, never cached so it is correct as stacks change. cmdhelp surfaces it: the table/string renders lead with a plain "note: '<cmd>' is currently overridden/extended via commandstack by: <renamers> (override-attached docs: '<ids>')" line, -return text leads with an OVERRIDDEN: line (below UNAVAILABLE when both apply - the G-166 leading-line convention), -return dict carries the 'overridden' key on all its return shapes, and the undocumented-command fallback carries the note too. The cmdhelp.test cmdinfo/dict key-list pins flipped deliberately with this key (as they did for G-166's 'unavailable'); new pins cover the documented-command note incl the docs clause + dict axis, and the doc-less-proc autodef render leading with the note, the -return text OVERRIDDEN: line and the note clearing when the override unwinds.
#0.10.0 - G-166 recognised-but-unavailable landings are surfaced distinctly. (a) cmdinfo gains an 'unavailable' result key on EVERY result - empty, or the canonical -choiceunavailable name (punk::args 0.16.0) the resolution addressed. It is a SECOND axis beside cmdtype, never a cmdtype value: cmdtype still answers what kind of executable thing was found ('string is dict' and 'string is true' on Tcl 8.6 both stay 'doconly'), availability answers whether the resolved word is selectable in this runtime/context. cmd_traverse's yielded/returned records carry a sixth element for it; cmdinfo keeps the final landing's value. (b) DECIDED (enacting the G-073 doc-walk follow-on): a unique PREFIX landing on an unavailable name now resolves that name's virtual docid, attributed - previously it conservatively resolved nothing, which made addressing inconsistent ('i string is tr' resolved but 'i string is dic' did not, while exact 'dict' and 'true' both did). The choice traverse's parse-parity claim is untouched: an unavailable word still identifies no SUBCOMMAND there, it is only attributed (the path taken when the name has no space-form virtual docid). (c) cmdhelp surfaces it in every return form: -return dict carries the key, table/string render the marking below the usage, -return text leads with an 'UNAVAILABLE:' line - and an unavailable subject is never presented as cleanly usable, so 'i string is dict 5' no longer renders info-scheme-clean just because the advisory parse of '5' succeeds against the virtual id. Wording (punk::ns::_cmdhelp_unavailable_marking) mirrors punk::args' tailored parse rejection; the runtime-specific why stays in the definition. Definitions without -choiceunavailable behave unchanged. Tests: cmdflow.test gains the flowunavail/flowunavail_nodoc fixtures + 5 pins, cmdhelp.test gains the helpstrua fixture + 5 pins (3 local, 2 real-world tclcore-gated) and the two deliberate key-list flips (cmdhelp_cmdinfo_result_shape, cmdhelp_return_dict_valid).
#0.9.3 - G-073 doc-walk parity: both cmd_traverse choiceword_match call sites (the G-051 space-form prefix-parity branch and the choice-traverse subcommand resolution) now pass the argument's -choiceunavailable list (punk::args 0.16.0 8th resolver argument), so the doc walk honours recognised-but-unavailable names identically to parsing: a prefix shadowed into ambiguity by an unavailable name resolves no documentation, and an unavailable word is not accepted as a choice (its documentation remains reachable via the exact space-form virtual docid, checked before the choice traverse - e.g 'i string is dict' on Tcl 8.6 via the tclcore moduledoc 0.4.0 per-class ids). No behaviour change for definitions without the key.
zeroarg-firstform {::testspace::flowmultiform val green} 11
} {
set code [lindex [traverse_final {*}$probe] 0]
lappend result [list $label $code {*}[word_accounting $probe]]
}
set result
}\
-cleanup {
}\
-result [list\
{docid-notail 0a 1 1}\
{single-notfound 2 1 1}\
{spaceform-exhaust X 1 1}\
{choiceless-leader 3 1 1}\
{unknown-word 4 1 1}\
{subhelp-landing 6 1 1}\
{unmapped-choice 7 1 1}\
{optsvals-form 8 1 1}\
{undoc-tail 9 1 1}\
{eparam-exhaust 10 1 1}\
{zeroarg-firstform 11 1 1}\
]
#added 2026-08-09 (agent, G-174) - the defect class: multi-word tails on a zero-arg-first-form
#command. Before the fix args_remaining held only the LAST word (earlier words in neither
#list); the walk now returns the whole unconsumed tail.
test cmdtraverse_zeroargfirst_multiword_no_drop {multi-word tails on a zero-arg-first-form command land whole in args_remaining}\
-setup $common -body {
set cinfo [punk::ns::cmdinfo ::testspace::flowmultiform val green]
lappend result [dict get $cinfo args_remaining] [dict get $cinfo docid]
set cinfo [punk::ns::cmdinfo ::testspace::flowmultiform lead red]
lappend result [dict get $cinfo args_remaining]
set cinfo [punk::ns::cmdinfo ::testspace::flowmultiform val green blue]
lappend result [dict get $cinfo args_remaining]
}\
-cleanup {
}\
-result [list {val green} ::testspace::flowmultiform {lead red} {val green blue}]
test cmdtraverse_zeroargfirst_singleword {a single trailing word still lands in args_remaining - via the leader-exhausted return, no longer by last-snapshot luck}\
-setup $common -body {
set rec [traverse_final ::testspace::flowmultiform val]
lappend result [lindex $rec 0] [lindex $rec 3]
}\
-cleanup {
}\
-result [list 11 val]
#added 2026-08-09 (agent, G-174) - the real-world witness from the goal: the walk no longer
#narrows 'i a? web green' to the loose sample form by dropping 'web' before the advisory parse
test cmdinfo_realworld_ansi_aq_no_drop {punk::ansi::a? discriminator words survive to args_remaining for form narrowing}\
-setup $common -body {
set cinfo [punk::ns::cmdinfo punk::ansi::a? web green]
lappend result [dict get $cinfo args_remaining]
set cinfo [punk::ns::cmdinfo punk::ansi::a? term pastel]
lappend result [dict get $cinfo args_remaining]
}\
-cleanup {
}\
-result [list {web green} {term pastel}]
#added 2026-08-09 (agent, G-174) - code-7 lane detail: the choice resolver matches the word
#but no docid serves it (no choiceinfo mapping, no space-form id) - the walk stops at the
#parent's docid with the matched word unconsumed; a mapped sibling still descends
test cmdtraverse_code7_witness_unmapped_choice {a matched choice with no mapping returns lane 7 with the word unconsumed and the parent docid}\
-setup $common -body {
set rec [traverse_final ::testspace::flownomap nomap]
#added 2026-08-09 (agent, G-174) - first-form-only inspection pinned as a RECORDED LIMIT
#(G-174 decision): the walk inspects only the definition's FIRST form, so a subcommand
#leader on a LATER form is never consulted - the word stays unconsumed at the parent
#docid (conserved) instead of descending to the later form's subhelp docid. Cross-form
#traversal discrimination is a possible follow-on goal; flipping this pin is the signal
#that it landed.
test cmdtraverse_firstformonly_recorded_limit {a choices-bearing leader on a later form is not consulted - the walk stays at the parent with the word conserved in args_remaining}\
-setup $common -body {
set rec [traverse_final ::testspace::flowmultilead alpha]
lappend result [lindex $rec 0]
set cinfo [punk::ns::cmdinfo ::testspace::flowmultilead alpha]
lappend result [dict get $cinfo docid] [dict get $cinfo args_remaining]
test cmdhelp_zeroargfirst_ambiguity_dict {cmdhelp -return dict for 'a? web green' carries the full tail and the web/sample multipleformmatches parse status}\
-setup $common -body {
set d [punk::ns::cmdhelp -return dict ::punk::ansi::a? web green]
lappend result [dict get $d args_remaining]
set ps [dict get $d parsestatus]
lappend result [dict get $ps ok] [dict get $ps failureclass]
lappend result [dict keys [dict get $ps formstatus]]
test cmdhelp_zeroargfirst_ambiguity_text {cmdhelp -return text for 'a? web green' renders web's argument table first with sample's following - both matching forms, no others}\
-setup $common -body {
set txt [punk::ns::cmdhelp -return text ::punk::ansi::a? web green]
set positions {}
foreach f {sgr_overview term tk web x11 sample} {
set pos [string first "--ARGUMENTS (form '$f')--" $txt]
if {$pos >= 0} {lappend positions $f $pos}
}
lappend result [dict keys $positions]
lappend result [expr {[dict get $positions web] < [dict get $positions sample]}]
}\
-cleanup {
}\
-result [list {web sample} 1]
#added 2026-08-09 (agent, G-174) - punk::ns::synopsis marks the same forms for the same
#words (the args_remaining-sensitive marking consumer beside cmdhelp's advisory parse).
#Marking is underline-wrapping of the matching forms' synopsis lines; non-comment
#synopsis lines map to form names in declaration order.
test synopsis_zeroargfirst_marks_matching_forms {punk::ns::synopsis for 'a? web green' underlines exactly the web and sample form lines}\
-setup $common -body {
set syn [punk::ns::synopsis -return full ::punk::ansi::a? web green]
set UL [punk::ansi::a+ underline]
set formnames [punk::args::forms ::punk::ansi::a?]
lappend result $formnames
set marked {}
set fidx 0
foreach line [split $syn \n] {
if {[string range $line 0 1] in {"# " "##"}} {continue}
if {[string first $UL $line] == 0} {lappend marked [lindex $formnames $fidx]}
incr fidx
}
lappend result $marked [expr {$fidx == [llength $formnames]}]
}\
-cleanup {
}\
-result [list {sgr_overview term tk web x11 sample} {web sample} 1]
}
tcltest::cleanupTests ;#needed to produce test summary line.
