punkshell/goals/archive/G-041-punkargs-form-matching.md

G-041 punk::args multi-form matching: automated form selection for parsing and documentation

Status: achieved 2026-07-13 Scope: src/modules/punk/args-999999.0a1.0.tm (parse form selection, arg_error/usage form marking), src/modules/punk/ns-999999.0a1.0.tm (cmdhelp/synopsis closest-form indication), src/tests/modules/punk/args/testsuites/ Goal: for multi-form definitions punk::args determines which form(s) an argument list matches - parse without -form attempts all permitted forms instead of effectively form 0, -form accepts the documented list-of-forms restriction, and the documentation surface indicates the match ('i after cancel ' presents the cancel form; 's after cancel someid' marks the closest synopsis) - with explicit single-form restriction retained for callers that require it. Acceptance: parsing a multiform definition (after-like fixture) without -form succeeds when the args match exactly one form (the pinned GAP tests in forms.test flip to auto-selected results); an argument list matching no form (or several) produces an error naming the candidate forms rather than a form-0 type error; -form with a list of form names/indices restricts parsing to that subset (currently an 'Expected int 0-N or one of ...' error); 'i <args...>' and synopsis output indicate the best-matching form(s) for supplied args; explicit -form behaviour is unchanged; the full punk::args and punk::ns suites pass.

Context

Multi-form definitions (@form directive) describe commands whose argument sets are orthogonal - the motivating example is the Tcl builtin after, documented with several forms (after ms ?script...?, after cancel id, after idle script..., ...). The definition and display side works: form_names are recorded, punk::ns::synopsis renders every form (## FORM n headers), and punk::args::parse -form <name-or-index> parses against an explicitly chosen form.

What is missing (characterized 2026-07-08 with an after-like fixture, probe results pinned as GAP tests in src/tests/modules/punk/args/testsuites/args/forms.test):

• punk::args::parse without -form (default *) does NOT attempt the other forms: arguments that match only a non-zero form (e.g. cancel after#1) fail with form 0's type/arity error ("Bad number of values ... Not enough remaining values", the leading word rejected by form 0's int type). Effectively * parses form 0 only.
• -form is documented as "Restrict parsing to the set of forms listed" and typed as a list, but a list of two form names errors with "invalid -form value ... Expected int 0-N or one of ''" - only a single name or index is accepted.
• Documentation surface: i after cancel <id> renders help for the default form rather than determining that the cancel form is the closest match, and s after cancel someid lists all synopses without marking the matching one.
• Internal note: punk::args::parse's own definition is multiform (withid/withdef, discriminated by literal types) but the proc hand-parses its arguments on the happy path, so the multiform selection machinery is not exercised internally either.

User direction (2026-07-08): there are times when restricting to a particular form is appropriate and times when automated selection is better - both must remain expressible. This is an advanced improvement, deferred; the immediate prerequisite work was making the punk::args test suite comprehensive so the current behaviour is pinned before any form-selection logic lands.

Approach (sketch - to be refined when activated)

1. Form candidacy: for -form * (or a list), attempt each permitted form's parse; classify outcomes (clean match / match-with-defaults / type-or-arity failure). Exactly one clean match -> auto-select. None or several -> error naming candidate forms (and for 'several', possibly a deterministic preference rule - e.g. first-declared - recorded when decided).
2. Literal/leading-word discrimination as a fast path: many multiform commands (after, parse itself) discriminate on a leading literal - a pre-pass on literal/literalprefix leader types can pick the form without full parse attempts.
3. -form list support: accept a subset of names/indices as documented; single value behaviour unchanged.
4. Documentation surface: cmdhelp/arg_error accept the supplied trailing args (cmdhelp already parses them for goodargs marking) and use the same candidacy logic to pick or rank forms; synopsis display marks the matching form(s).
5. Error-message quality: a no-form-matches error should show per-form failure reasons compactly rather than only form 0's.

Downstream consumer note (2026-07-08, G-044): the interactive command-completion/hinting goal consumes form candidacy on PARTIAL argument lists - as the user types, the hinting display wants "which forms are still compatible with the words so far" - so the candidacy mechanism should be exposed as an API that accepts an incomplete argument list and returns per-form compatibility (not only an all-or-nothing full parse). Design for that consumer when shaping step 1.

Progress

• 2026-07-13 increment 1 (engine - punk::args 0.10.0, project 0.12.14): multi-form candidacy implemented in get_dict. The single-form parse body was extracted verbatim to private::get_dict_form (no caller-frame use inside - verified; resolve still runs in the caller's context before selection). -form * / multi-form selections attempt each permitted form: exactly one clean match auto-selects; none raises 'noformmatch' naming each candidate form's first-line failure; several raise 'multipleformmatches' naming the forms. -form list support landed across get_dict/parse/parse_status/ arg_error via the shared private::form_selection resolver (supplied order preserved; arg_error renders the argument table for the first listed form and marks all listed forms' synopsis entries). Results gain 'form' (+'formstatus' when candidacy ran); parse_status 'form' is the matched/best-candidate form and its new 'formstatus' key is the per-form compatibility surface designed for the G-044 hinting consumer. DECISIONS RECORDED:
  • Several clean matches ERROR rather than silently preferring first-declared (the acceptance wording; a silent pick could validate against an unintended form and return a wrong values shape - callers with a preference pass -form).
  • Candidate ranking (display/status only - candidacy always from real parse attempts, no literal fast-path skipping): leading-literal affinity of each form with the supplied words (agreeing literal(x)/literalprefix(x) leading run scores up, first disagreement disqualifies), then incomplete before invalid, then declaration order. Known limits noted in form_literal_affinity: no alignment through received options; choice-discriminated forms score 0 (refinable).
  • Literal-discrimination pre-pass as a parse fast path (approach step 2) deferred as a perf refinement: correctness must come from real parse attempts, and the affinity heuristic's word alignment is not exact through options. Tests: forms.test GAPs flipped (forms_parse_autoselect, forms_parse_formlist_restriction) + noformmatch ranking/errorcode, multipleformmatches, shared-prologue arity discrimination, formstatus coverage. punk::args 198/199 (1 pre-existing skip), punk::ns 53/53, punk::lib 35/35. Remaining after increment 1: doc-surface increment (cmdhelp best-form presentation via parse_status form key - cmdhelp's -form default was still 0; synopsis closest-form marking), live tclcore verification ('lseq 3', switch block form), @form -synopsis override GAP (adjacent - decide in doc-surface increment).
• 2026-07-13 increment 2 (doc surface - punk::args 0.11.0, punk::ns 0.5.0, project 0.12.15): cmdhelp -form defaults to * and renders the advisory parse's matched/ best-candidate form at both render sites ('i after cancel ' presents the cancel form's table; noformmatch passes the ranked candidates and multipleformmatches the matching forms to arg_error so all are marked in the synopsis block). punk::ns::synopsis underlines the form(s) trailing words match ('s after cancel someid' marks both cancel forms; 's lseq 0 10 2' marks the range form) - ordinal line-to-form mapping, skipped under alias-currying excess or an explicit -form. The adjacent @form -synopsis override GAP was fixed and flipped (forms_form_synopsis_override_rendered): punk::args::synopsis now renders the documented override in full and summary modes. Ranking refinement: form_literal_affinity extended to RESTRICTED-choice discriminators via choiceword_match (shared G-040 resolver) - the tclcore models express subcommand words as -choices (after's cancel/idle/info), so 'after cancel' ranks the cancel forms first instead of declaration-order fallback. REAL-MODEL FINDINGS (live tclcore verification, punk902z + tclsh probes):
  • 'lseq 3' -> count form, 'lseq 0 10 2' -> range form, switch separate/block both auto-select, 'i lseq 0 10 2' renders the range form info-scheme with its synopsis line underlined.
  • 'after cancel someid' is GENUINELY ambiguous in the doc-faithful model (cancelid 'after cancel id' vs cancelscript 'after cancel script ?script?...') - real Tcl disambiguates semantically by the id's shape, a textual model cannot. Behaviour: multipleformmatches naming both, first candidate's table, both synopsis lines marked. Accepted as correct for a doc-faithful model.
  • 'lseq 1 count 5' is ambiguous (range + start_count) ONLY because the model's expr-typed operands accept the word 'count' as an end value - the -type expr non-validation already recorded for G-055; noted there as a model-tightening candidate (TIP 746 removes expr operand behaviour in 9.1 anyway). Suites: punk::args 198/199 (1 pre-existing skip), punk::ns 57/57, full source tree 836 total / 822 passed / 13 skipped / 1 failed = exec-14.3 (the known baseline).
• 2026-07-13 ACHIEVED - acceptance review against each clause:
  • Multiform parse without -form succeeds on a unique clean match: the forms.test GAP pins flipped to auto-selected results (forms_parse_autoselect - ms/cancel/ idle each selected from the words alone; forms_parse_autoselect_shared_prologue covers arity-discriminated forms sharing a leader block).
  • No-match/several-match errors name the candidate forms instead of a form-0 type error: noformmatch carries every candidate's first-line failure ranked best-first (forms_parse_noformmatch) and multipleformmatches names the matching forms (forms_parse_multipleformmatches) - the 'several' outcome is an error by recorded decision (no silent first-declared preference).
  • -form list restriction: a list of names/indices restricts candidacy to the subset across get_dict/parse/parse_status/arg_error (forms_parse_formlist_restriction; unrecognised elements keep the invalid -form error).
  • Doc surface indicates the best-matching form(s): 'i after cancel ' presents the cancel form's argument table (cmdhelp_multiform_autoselected_form_presented, cmdhelp_multiform_noformmatch_best_candidate) and 's after cancel someid' underlines the matching synopsis line(s) (synopsis_multiform_marks_matching_form/_unmarked_without_args); live-verified against the tclcore models on punk902z ('i lseq 0 10 2' range form underlined info-scheme; 'i after cancel someid' ambiguity named with both cancel forms marked).
  • Explicit single -form behaviour unchanged: the pre-existing explicit-form tests (forms_parse_explicit_form_by_name/_by_index) pass unmodified.
  • Full punk::args and punk::ns suites pass (198/199 with 1 pre-existing skip; 57/57); full source tree 822/836 with the exec-14.3 baseline as the only failure. Adjacent extra: the @form -synopsis override GAP flipped (forms_form_synopsis_override_rendered). The G-044 candidacy-API design note is satisfied by parse_status's formstatus key (per-form compatibility on partial arglists). Deferred as refinements (not acceptance): literal-discrimination fast path for parse performance; affinity alignment through received options; punk::args::synopsis -form list support (single-form restriction retained there).

Alternatives considered

• Documenting alternate forms as separate subhelp choiceinfo entries (the trace-style pattern) - works only when a literal subcommand word exists and fragments a command's documentation into pseudo-subcommands; rejected as the general answer.
• Requiring callers to always pass -form - status quo for parsing, but unusable for the interactive doc surface ('i after cancel ') where the user supplies args, not form names.

Notes

• Probe script: scratchpad formprobe.tcl (2026-07-08); findings encoded in src/tests/modules/punk/args/testsuites/args/forms.test (GAP-labelled tests reference this goal and flip when it is implemented).
• Related: G-040 (choice aliasing/doc-lookup parity - same "parser and doc surface must agree" principle); punk::args::parse -cache interaction with per-form attempts will need review (cache key already includes selected form).
• Incidental find during the same probing: an uncommented debug puts stderr ">>>_get_dict_can_assign_value NOT alloc_ok..." fired on every failed clause type assignment (punk/args ~l.6186; its happy-path twin was already commented). Fixed 2026-07-08 (punk::args 0.2.3) - noted here because form-attempt logic will exercise that failure path heavily.
• 2026-07-12 prework probes (user concerns raised before activation - real-builtin evidence beyond the after fixture; results from the punk902z kit, Tcl 9.0.2):
  • Auto-selection gaps confirmed on real commands: 'lseq 3' (count form), 'lseq 1 count 5 ?by 2?' (start_count form) and 'switch abc {a {} default {}}' including options+block ('switch -regexp -matchvar m abc {...}') all fail under default form-0 parsing but parse correctly with an explicit -form. The per-form models are doc-faithful: lseq.n's three synopsis lines map 1:1 to the modelled range/start_count/count forms; switch separate/block both verified per-form.
  • PREREQUISITE independent of form selection: the value allocator mishandles an optional single-word choice value followed by a required value plus a trailing optional-member clause. lseq range-form arglists WITHOUT the ../to noise word but WITH a step ('0 10 2', '0 10 by 2', '1 5 by 0') fail IN the range form (the error blames '..|to'), while '0 to 10 2' and '0 .. 10 by 2' parse fine. No other lseq form accepts those arglists, so form auto-selection alone could not make 'i lseq 0 10 2' work. The allocator prerequisite became G-071 and was FIXED the same day (achieved 2026-07-12, punk::args 0.9.0 allocation choice screen - see goals/archive/G-071-punkargs-optional-allocation.md): the lseq matrix now parses in-form, leaving form auto-selection as this goal's remaining gap for the 'lseq 3' / switch-block-form cases. ::if's noise-word clauses ('?literal(then)?' mid-clause, '?literal(else)?' clause-leading) parse correctly in the same probes, so the failure is specific to the optional standalone value
    • trailing optional-member clause combination, not optional clause members generally.
  • CORRECTED (G-071 closeout): parse_status DOES accept -form - positioned before the withid/withdef tail per its documented synopsis; the probe had passed it trailing. Behaviour pinned in allocation.test (parsestatus_form_option_order). The candidacy API and the G-044 consumer can build on the existing surface (list-of-forms support remains this goal's -form list item, as for parse).
  • Over-acceptance divergences (model valid where the real command errors) recorded for the modelability list in G-055's detail file - not this goal's scope.
  • Incidental fix during probing: another unconditional debug puts on the clause type-check path (private::get_dict_can_assign_value "checking tp ...", fixed as punk::args 0.8.2) - companion to the 0.2.3 find noted above; form-attempt logic will exercise these paths heavily.
• Archived-goal references in this file: G-040 achieved 2026-07-08 (goals/archive/G-040-punkargs-choicealiases.md).