punkshell/goals/G-174-cmdtraverse-word-accounting.md

G-174 cmd_traverse word accounting - the doc-lookup walk never drops argument words

Status: proposed Scope: src/modules/punk/ns-999999.0a1.0.tm (cmd_traverse/cmdinfo doc-lookup walk, cmdhelp advisory parse + synopsis consumers); src/tests/modules/punk/ns/testsuites/ns/{cmdflow,cmdhelp}.test (conservation characterisation + consumer pins); src/modules/punk/ansi-999999.0a1.0.tm (punk::ansi::a? argdoc - real-world witness, edits only if the sample-form tightening decision lands) Goal: the doc-lookup walk behind punk::ns::cmdinfo has a stated, characterised word-accounting contract - every supplied trailing word lands in exactly one of args_resolved or args_remaining - so args_remaining-sensitive consumers (cmdhelp's advisory parse and G-041 form narrowing, synopsis form marking) always operate on the full unconsumed tail, and a command whose definition's first form is zero-arg (the punk::ansi::a? class) no longer has its literal discriminator words silently dropped before form narrowing runs. Acceptance: characterisation tests in cmdflow.test assert the conservation invariant (args_resolved plus args_remaining account for every supplied trailing word, each word in exactly one) across the walk's return lanes - including an a?-shaped fixture (plain proc, multiform definition, zero-arg first form) with multi-word tails, plus witnesses for the previously unwitnessed code-7 and code-10 return lanes; punk::ns::cmdhelp for 'a? web green' runs its advisory parse on the full tail and presents the web/sample multipleformmatches ambiguity (web's argument table rendered first, both forms marked in the synopsis) rather than a clean sample-only render, and punk::ns::synopsis marks the same forms for the same words; the first-form-only inspection (fid = first of form_names) is either extended or pinned as a recorded limit with its own witness, with the decision recorded in this file; the punk::ns suite passes under the canonical runtests interpreter with no regressions in cmdflow/cmdhelp.

Context

Root-caused 2026-08-07 from the user observation that 'i a? web green' presents the help for the loose 'sample' form (ordinal 5, first value typed string|stringstartswith(...)) instead of the 'web' form (ordinal 3, first value literal(web)) - "why don't the literals win out?".

The form-narrowing machinery is NOT at fault. Parsed directly, punk::args::parse_status {web green} withid ::punk::ansi::a? correctly reports BOTH 'web' and 'sample' as clean matches (multipleformmatches, no silent preference - the G-041 decision; achieved 2026-07-13, see goals/archive/G-041-punkargs-form-matching.md), web first in declaration order, so cmdhelp would render web's argument table and mark both forms. The literal-affinity ranking (form_literal_affinity) only orders candidates in the no-form-matches failure display and plays no role here.

The defect is one layer up: punk::ns::cmd_traverse (sole caller: cmdinfo's coroutine spawn) drops all but the LAST trailing word for commands whose definition's first form is zero-arg. Mechanism (line anchors are point-in-time, ns-999999.0a1.0.tm as at 2026-08-07):

1. Each outer-loop iteration re-snapshots queryargs/queryargs_untested to [lrange $args $i end] (~ns:5013) - a fresh tail per word.
2. The walk inspects only the spec's FIRST form - set fid [lindex $form_names 0] (~ns:5036); the multiform-discrimination block beside it is commented out ("multiform discrimination not yet supported").
3. a?'s first form is sgr_overview (zero-arg overview form): no opts, no values, so the ensemble-shape test ![llength $optnames] && ![llength $valnames] (~ns:5063) routes into the subcommand walk - but the form has no leaders either, so leadernames_matched is empty and the foreach immediately hits if {$lname eq ""} break (~ns:5073, annotated #todo - return? - the author already suspected the spot). Nothing is consumed, returned, or restored.
4. Control falls back to the outer for, which just advances i and re-snapshots. At loop exhaustion the code-10 fallthrough return (~ns:5309, annotated #REVIEW!!!) reports the LAST snapshot: with N trailing words, N-1 are silently discarded - present in NEITHER args_resolved nor args_remaining.

Probe evidence (punk91 src, 2026-08-07):

cmdinfo punk::ansi::a? web        -> args_remaining={web}
cmdinfo punk::ansi::a? web green  -> args_remaining={green}   ("web" vanished)
cmdinfo punk::ansi::a? term pastel-> args_remaining={pastel}  ("term" vanished)

cmdhelp's advisory parse then legitimately single-matches 'sample' on the remaining word ('green' fails web's required literal(web) first value, matches sample's string) - a CLEAN ok=1 narrow, so the mispresentation carries no failure signal anywhere.

Consumer sensitivity split (all cmdinfo consumers in the dev tree):

• docid-only, immune to the drop: punk::ns::forms, punk::ns::eg, punk::args::eg, punk::lib::tclscript_info, punk::args::ensemble_subcommands_definition.
• args_remaining-sensitive, inherit the corruption: punk::ns::cmdhelp (the 'i' advisory parse / G-041 narrowing - the observed defect), punk::ns::synopsis and synopsis_raw ('s' form marking from unresolved_args), cmdtrace/cmdtracebasic, and cmdinfo's own curried-notfound recursion.

Coverage determination (2026-08-07 review of both suites): cmdflow.test (26 tests) pins the choice-walk arcs - parse-parity per G-040 (achieved 2026-07-08, goals/archive/G-040-punkargs-choicealiases.md) covering denylist/reservelist/aliases/prefix/nocase, space-form docid jumps per G-051 (achieved 2026-07-13, goals/archive/G-051-cmdinfo-pseudo-and-prefix.md), unavailable attribution per G-166 (achieved 2026-08-06, goals/archive/G-166-cmdinfo-unavailable-surfacing.md), ensemble -parameters traversal. cmdhelp.test (~50 tests) pins rendering and POST-resolution multiform narrowing (the G-041/G-143/G-164 arcs; G-143 achieved 2026-08-01, goals/archive/G-143-maketcl-multiform-help.md; G-164 achieved 2026-08-04, goals/archive/G-164-viability-alternative-allocation.md), doconly/'string is' real-world pins, oo methods. Many tests assert args_remaining values, but always through subcommand-walk or single-trailing-word scenarios. Nothing asserts the conservation invariant; no fixture has the zero-arg-first-form shape; the code-7 lane's own source comment names its test ('i namespace which -v x') but no such test exists; code-10 has none; the first-form-only inspection is unpinned. The defect lives precisely in the uncovered lanes.

Approach

1. Characterise first, fix second (defect witnesses included): add cmdflow.test coverage asserting the conservation invariant across the walk's return lanes, an a?-shaped fixture (plain proc, multiform definition, zero-arg first form) exercised with multi-word tails, and witnesses for the code-7 and code-10 lanes. Pin the CURRENT dropping behaviour as the defect witness, flipped by the fix commit.
2. Fix minimally at the annotated sites: return the remaining tail from the empty-leader break (code-8-like return honouring #todo - return?), and/or tighten subcommand-walk eligibility so a level is only walkable when the inspected form actually has a choices-bearing leader. Choose the smallest change that restores conservation without disturbing the characterised parity/ensemble/unavailable lanes.
3. Consumer verification: cmdhelp ('i a? web green' ambiguity presentation, -return text parity) and synopsis form marking on the same words.
4. Decision item - first-form-only inspection (fid = form_names[0]): default is pin-as-recorded-limit with a witness; cross-form discrimination during traversal is its own follow-on goal if wanted.
5. Decision item (optional, ansi-side): whether a?'s 'sample' form stays string-loose so discriminator words genuinely ambiguate (default: accept - the marked-ambiguity presentation IS the G-041 contract), or the argdoc is tightened. No ansi edit unless this lands the other way.

Notes

• The a? argdoc (ansi-999999.0a1.0.tm ~3213-3279) is the trigger and real-world witness, not an edit target: forms sgr_overview/term/tk/web/x11/sample, with term/tk carrying their literals as @leaders while web/x11 carry them as @values - both shapes should appear in the fixture.
• Single-word tails survive today only because the last word is always the final snapshot - 'i a? web' working is luck, not contract.
• args-side channels are already adequate: parse_status reports multipleformmatches with per-form formstatus (all-valid keys) and noformmatch with ranked -formerrors; cmdhelp consumes them correctly. No punk::args change is expected under this goal.