punkshell/goals/G-044-repl-command-completion.md

G-044 punk::args-driven interactive command completion and hinting in the repl (raw mode primary)

Status: proposed Scope: src/modules/punk/repl-999999.0a1.0.tm (editbuf/reader integration, provider seam), src/modules/punk/args-999999.0a1.0.tm + src/modules/punk/ns-999999.0a1.0.tm (introspection surfaces as consumed), src/modules/punk/console-999999.0a1.0.tm (rendering) Goal: the interactive repl (raw mode primary) offers command completion and hinting driven by punk::args documentation - the command word resolved through the same doc-lookup flow as 'i' (ensembles, subcommands, ensemble parameters) and the argument position through the definition (options, choices with parse-consistent matching, literals, form awareness per G-041) - via an activation scheme that preserves the editbuf's literal-tab support (not exclusively plain Tab), behind a per-subshell completion-provider seam so alternative-language subshells (e.g. an interactive xtal session) can replace, augment or cleanly disable the Tcl-centric completer. Acceptance: in a raw-mode interactive session a documented trigger (recorded in the detail file; a literal tab remains enterable into the editbuf) presents completions for command words, ensemble subcommands (resolved via the punk::ns doc-lookup flow without executing candidate commands), option flags and choice values (matching consistent with parse semantics: prefix/deny/reserve, and aliases once G-040 lands); a hint display shows the synopsis of the form(s) matching the partial input (consumes G-041's candidacy API when available - until then all-forms or form-0 display with the limitation noted is acceptable); no per-keystroke terminal queries are added when no completion display is active (G-013 approach note); the completer is a provider interface with the punk::args-driven implementation as the default - a subshell can declare an alternative provider or none, proven at minimum by a subshell with completion disabled showing no Tcl-centric interference (the xtal minimum bar; full xtal completion out of scope but not precluded by the interface); line-mode behaviour is unchanged unless a documented subset is added; G-013's raw editor essentials are a prerequisite and this goal does not weaken G-013's acceptance.

Context

User direction (2026-07-08): the raw mode repl system will use punk::args based documentation to retrieve enough information about incomplete commands - taking note of different forms - to provide interactive command-completion and hinting.

Constraints stated up front:

• The editbuf must continue to support literal tabs, so completion activation is not purely Tab-based. (Tab is data in this shell, not only a UI gesture.)
• The Tcl/punk::args-centric completer must be able to get out of the way: the motivating non-Tcl scenario is an interactive session for an alternative syntax such as the xtal Tcl package (https://tarray.magicsplat.com/xtal_lang.html) running as a subshell, perhaps with plugin-defined repl behaviour for completion/syntax. The minimum bar for this goal is a clean disable/replace seam; actual xtal completion or an xtal session itself is not this goal's deliverable.

Why punk::args is the right substrate: the definitions already carry per-position knowledge - leaders (including ensemble parameters), option names, choice sets with prefix/deny/reserve semantics (choices.test), literal/literalprefix types, and multiple forms - and the punk::ns doc-lookup flow (cmdwhich/cmdinfo/cmd_traverse, characterized in cmdflow.test 2026-07-08) already resolves command words through aliases, ensembles, subhelp choiceinfo and ensemble parameters WITHOUT executing candidate commands (the arginfo principle: introspection must not run commands to elicit usage).

Approach (sketch - to be refined when activated)

1. Introspection layer (no repl dependency, testable headless): given a partial word-list, return a completion/hint dataset - candidate next words (subcommands from choiceinfo, option flags, choice values, literals from -type), plus the synopsis/form information for the current position. Sources: punk::ns::cmdinfo for the command-word path, punk::args::get_spec for the argument position. Choice-word matching must reuse the parser's resolution rules (the shared choice-resolution helper from G-040 (achieved 2026-07-08) is the natural common code; prefix highlighting from the usage renderer already computes minimal prefixes). Form handling: before G-041, enumerate all forms' next-position candidates (union) and show all/form-0 synopses with the limitation noted; when G-041's candidacy API exists, rank/filter forms by the partial input (G-041's detail file notes the API should accept partial argument lists for exactly this consumer).
2. Repl integration (raw mode): trigger scheme preserving literal tab - candidates to evaluate at implementation time (decision recorded here): double-Tab, a dedicated chord (e.g. ctrl-space), Tab-completes-only-after-non-whitespace with an escape to insert a literal tab, or fully configurable binding via G-014 config. Hinting may be passive (status/ghost line) independent of the completion trigger. Rendering budget: no added per-keystroke terminal queries when idle (G-013 approach note - cache console size with invalidation); the G-013 editbuf live-view debug gate is the development aid for this work (G-013 debug inventory item 2).
3. Provider seam: the completion engine behind an interface (provider per subshell); the punk::args-driven implementation is the default provider. A subshell declares an alternative provider or none - the declaration rides the subshell-definition composition of G-042/G-043 (a punk.subshell provider or stored-config entry may name a completion provider alongside commandsets and help topics). Language subshells (xtal-style: alternative syntax translated to Tcl) are the motivating consumer; an actual interactive xtal session would be its own goal when pursued, composing a G-043 subshell definition (commandset/evaluator, help topics, completion provider).

Alternatives considered

• Command-name-only completion (readline-style over info commands) - insufficient: the value is in argument-position awareness (subcommands, flags, choices, forms) which the punk::args corpus already encodes.
• Executing candidate commands to discover usage - rejected: the established arginfo principle forbids running commands for introspection (risky side effects); everything needed is available declaratively.
• Hardcoding Tab as the only trigger - rejected per the literal-tab constraint; the editbuf accepts tab characters as content.

Notes

• Prerequisites/relations: G-013 (raw mode default + editor essentials - prerequisite; its editbuf live view is the dev aid; its no-per-keystroke-query note is adopted), G-041 (form candidacy on partial input - consumed when available), G-040 (choice alias resolution shared with the parser), G-042/G-043 (per-subshell provider declaration), G-014 (configurable trigger/bindings and per-subshell defaults).
• xtal reference: https://tarray.magicsplat.com/xtal_lang.html - an alternative syntax for Tcl (tarray project); an interactive xtal subshell is the concrete language-subshell scenario the provider seam must not preclude.
• The doc-lookup flow this consumes is characterized by src/tests/modules/punk/ns/testsuites/ns/cmdflow.test (2026-07-08), including ensemble -parameters handling; the choice semantics by src/tests/modules/punk/args/testsuites/args/choices.test; forms by forms.test.
• Archived-goal references in this file: G-001 achieved 2026-07-11 (goals/archive/G-001-pluggable-console-backends.md); G-040 achieved 2026-07-08 (goals/archive/G-040-punkargs-choicealiases.md).

Repl behaviour preserve-list + testability findings (2026-07-11, user-directed - pre-refactor ordering)

Current interactive behaviour that this goal (and ANY repl refactor) must preserve and ultimately expand upon (user-specified 2026-07-11):

1. info-complete-parity continuation behaviour incl the Tcl quoting-rules quirk: a quoted word with unbalanced braces (e.g set x "{*}{") is complete standalone but inside a braced context (proc body) needs extra closing characters.
2. Closing-prompt hints in line and raw mode driven by the pending-opener stack (punk::lib::system::incomplete). Accepted limitation, pinned not fixed: only one close candidate is expressed when several exist.
3. Raw-mode ANSI colour staging: in-progress editbuf colour, different colour once submitted.
4. Literal tab acceptance preserved in the submitted string; raw-mode tab markers with navigation/backspace/delete smarts over them.
5. Raw-mode dim grey dots for spaces - display-only; the submitted string carries real spaces/tabs (dots/markers must never leak into history recall or evaluation).
6. History entries are editbuf objects (repl editbuf_list): up/down arrow navigates MULTILINE editbufs and recalled entries are editable.

Testability findings (spikes 2026-07-11):

• punk::lib::system::incomplete is a pure function - characterized NOW in src/tests/modules/punk/lib/testsuites/lib/commandcomplete.test (the quirk scenario progression, single openers, tabs, escapes, and an incomplete<->info-complete parity property).
• class_editbuf is console-coupled at its core: add_chunk renders via overtype::renderline against live terminal metrics (columns via cursor-position probing, tabstops via DECRQPSR) - headless instantiation works but content operations emit terminal queries (or fail/spam when ANSI facts are suppressed). Items 3-6 above are therefore NOT unit-characterizable until a console seam exists: an ::opunk::Console test double (G-001) or injectable metrics is the missing piece, and G-001 should be understood as the enabling refactor for repl characterization, not just a feature goal.
• Interactive end-to-end verification tiers: G-020 (screencap+input injection) as the near-term windows-only harness; a pseudoconsole (ConPTY/pty) expect-alternative as the durable cross-platform mechanism (candidate goal drafted 2026-07-11).