Browse Source

repl pre-refactor: characterize command-completeness engine + record preserve-list and testability findings

- new commandcomplete.test (6 tests, green 9.0.3 + 8.7): punk::lib::system::incomplete pure-function characterization - the info-complete quoting quirk (a quoted word with unbalanced braces e.g set x "{*}{" is complete standalone but needs extra closers inside a proc body; the pending-opener stack { " { " shrinks per typed closer), single openers, tabs in open braces, escapes, and an incomplete<->info-complete parity property over a case battery
- goals/G-044 detail: the user-specified preserve-list any repl refactor must honour (info-complete parity + quirk, closing-prompt hints incl the accepted single-candidate limitation pinned-not-fixed, raw-mode colour staging in-progress vs submitted, literal tab acceptance with raw-mode marker edit smarts, dim space dots display-only never leaking into submitted strings/history, up/down navigation of MULTILINE editbuf history with recalled entries editable) + testability findings: class_editbuf is console-coupled at its core (add_chunk renders via overtype::renderline against live terminal metrics - cursor-position size probing, DECRQPSR tabstops) so items 3-6 are not unit-characterizable until a console seam exists - G-001 (pluggable console backends / ::opunk::Console test double) is the enabling refactor, not just a feature goal
- goals/G-020 detail: repl interactive-behaviour verification recorded as a driving use case (near-term windows harness: keystroke injection + capture) pending the durable pseudoconsole expect-alternative

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 5 days ago
parent
commit
8c65fe9d6f
  1. 13
      goals/G-020-screencap-input-module.md
  2. 37
      goals/G-044-repl-command-completion.md
  3. 2
      src/tests/modules/AGENTS.md
  4. 124
      src/tests/modules/punk/lib/testsuites/lib/commandcomplete.test

13
goals/G-020-screencap-input-module.md

@ -91,3 +91,16 @@ the module is a separate development, not a refactor of it.
- Related: G-018 (kit composition; Tk as loadable package, no wish binaries),
G-019 (a trimmed capture-capable executable is a plausible scan target),
G-021 (agent-facing surface over this module), G-001 (backend plugin pattern).
## Additional driving use case: repl interactive-behaviour verification (2026-07-11)
Characterizing punkshell's interactive repl behaviour (raw-mode colour staging, tab
markers and dim space dots in the editbuf, closing-prompt hints, multiline history
navigation - see the preserve-list in goals/G-044 detail) currently has no automated
harness: the underlying editbuf is console-coupled and there is no expect-like system.
This module is the practical near-term bridge on windows: inject keystrokes at a live
punkshell window and capture/compare the rendered region. Coarser than a pseudoconsole
expect-alternative (the durable successor - candidate goal drafted 2026-07-11) but
available sooner, and it can verify exactly the rendered-behaviour tier that unit tests
cannot reach. Worth weighting this goal's priority accordingly when sequencing repl
refactor work.

37
goals/G-044-repl-command-completion.md

@ -85,3 +85,40 @@ arginfo principle: introspection must not run commands to elicit usage).
src/tests/modules/punk/ns/testsuites/ns/cmdflow.test (2026-07-08), including ensemble
-parameters handling; the choice semantics by
src/tests/modules/punk/args/testsuites/args/choices.test; forms by forms.test.
## Repl behaviour preserve-list + testability findings (2026-07-11, user-directed - pre-refactor ordering)
Current interactive behaviour that this goal (and ANY repl refactor) must preserve and
ultimately expand upon (user-specified 2026-07-11):
1. info-complete-parity continuation behaviour incl the Tcl quoting-rules quirk: a
quoted word with unbalanced braces (e.g `set x "{*}{"`) is complete standalone but
inside a braced context (proc body) needs extra closing characters.
2. Closing-prompt hints in line and raw mode driven by the pending-opener stack
(punk::lib::system::incomplete). Accepted limitation, pinned not fixed: only one
close candidate is expressed when several exist.
3. Raw-mode ANSI colour staging: in-progress editbuf colour, different colour once
submitted.
4. Literal tab acceptance preserved in the submitted string; raw-mode tab markers
with navigation/backspace/delete smarts over them.
5. Raw-mode dim grey dots for spaces - display-only; the submitted string carries real
spaces/tabs (dots/markers must never leak into history recall or evaluation).
6. History entries are editbuf objects (repl editbuf_list): up/down arrow navigates
MULTILINE editbufs and recalled entries are editable.
Testability findings (spikes 2026-07-11):
- punk::lib::system::incomplete is a pure function - characterized NOW in
src/tests/modules/punk/lib/testsuites/lib/commandcomplete.test (the quirk scenario
progression, single openers, tabs, escapes, and an incomplete<->info-complete parity
property).
- class_editbuf is console-coupled at its core: add_chunk renders via
overtype::renderline against live terminal metrics (columns via cursor-position
probing, tabstops via DECRQPSR) - headless instantiation works but content operations
emit terminal queries (or fail/spam when ANSI facts are suppressed). Items 3-6 above
are therefore NOT unit-characterizable until a console seam exists: an
::opunk::Console test double (G-001) or injectable metrics is the missing piece, and
G-001 should be understood as the enabling refactor for repl characterization, not
just a feature goal.
- Interactive end-to-end verification tiers: G-020 (screencap+input injection) as the
near-term windows-only harness; a pseudoconsole (ConPTY/pty) expect-alternative as
the durable cross-platform mechanism (candidate goal drafted 2026-07-11).

2
src/tests/modules/AGENTS.md

@ -42,7 +42,7 @@ Unit tests for editable source modules under `src/modules/`, `src/modules_tcl8/`
- `punk/args/` — punk::args tests (`testsuites/args/`): parsing, choices/choicegroups, forms, rendering/indentation characterization, usage-marking characterization (`usagemarking.test`: -parsedargs/-badarg/-parsestatus/-scheme marking primitives plus goodchoice highlighting of selected/default-in-effect choice words, asserted by SGR-parameter subset against the live colour arrays; the G-049 nocolour/colour-leak GAP pins flipped 2026-07-10 to scheme-statelessness assertions), and the G-049 parse-status structure (`parsestatus.test`: punk::args::parse_status overall/per-argument statuses, badarg for type/allocation failures, -caller attribution, errorcode -argspecs stripping)
- `punk/ns/` — punk::ns tests (`testsuites/ns/`): cmdwhich/cmdinfo/cmd_traverse doc-lookup flow (`cmdflow.test`, G-040 parity) and cmdhelp usage-rendering integration (`cmdhelp.test`: scheme selection, goodarg/badarg marking incl type/allocation failures, goodchoice highlighting of supplied/default choice words, alias path, cmdinfo result shape, queried-command failure attribution, and `-return dict` parse-status returns (G-049 - its GAP pins flipped 2026-07-10); remaining GAP pins for pseudo-command cmdtype + space-form docid prefixes (G-051, real `string is` pins behind the have_tclcoredocs constraint), TclOO undocumented-method fallback (G-052), and synopsis marking absence (G-050))
- `punk/mix/` — punk::mix::cli tests (prune helpers, punkcheck virtual sources), punk::mix::commandset::repo fossil move/rename characterization tests (`testsuites/repo/`, FOSSIL_HOME-isolated; GAP-marked tests pin behaviour G-022 will change), and the MULTISHELL polyglot build machinery (`testsuites/scriptwrap/multishell.test`: scriptset wrap via the punk.multishell.cmd template - structure/LF-only/determinism, checkfile 512-byte label validation of fresh wraps AND the committed bin/runtime.cmd, the runtime scriptset round-trip byte-identity pin, and platform-gated execution smoke: cmd.exe→powershell payload on windows, sh payload on unix or via the `wsllinux` capability constraint from `src/tests/testsupport/wslprobe.tcl` - staged to the WSL distro's native filesystem, G-059)
- `punk/lib/` — punk::lib tests (`testsuites/lib/`): range/index/parse/compat/interp_sync utilities, and G-058 static-baseline seeding (`staticseed.test`: interp_sync_package_paths/snapshot_package_paths propagate a simulated ::punkboot static baseline and seed `load {} <prefix>` ifneeded mappings; no-op without a baseline)
- `punk/lib/` — punk::lib tests (`testsuites/lib/`): range/index/parse/compat/interp_sync utilities, G-058 static-baseline seeding (`staticseed.test`: interp_sync_package_paths/snapshot_package_paths propagate a simulated ::punkboot static baseline and seed `load {} <prefix>` ifneeded mappings; no-op without a baseline), and the repl command-completeness engine (`commandcomplete.test`: punk::lib::system::incomplete pending-opener stacks - the info-complete quoting quirk progression (`set x "{*}{"` standalone vs in-proc-body), single openers, tabs, escapes, incomplete<->info-complete parity property; pre-repl-refactor characterization, see goals/G-044 detail preserve-list)
- `punk/packagepreference/` — punk::packagepreference tests (`testsuites/packagepreference/`): G-058 static-vs-bundled policy (`staticpolicy.test`: require of a baseline package triggers the index scan before resolution so a newer bundled copy wins, static beats older bundled, exact requires of bundled versions stay reachable, missing static mappings get seeded)
- `punk/libunknown/` — .tm same-version shadowing pin-tests (`testsuites/shadowing/`): tcl::tm::add prepend rule, head-of-tm-list wins exact-version ties, version beats order, punk::libunknown parity — shipped behaviour depends on these (runtests tm ordering, punk_main package-mode precedence, G-033); mixed .tm/pkgIndex.tcl characterization is goal G-035

124
src/tests/modules/punk/lib/testsuites/lib/commandcomplete.test

@ -0,0 +1,124 @@
package require tcltest
package require punk::lib
#Characterization of punk::lib::system::incomplete - the repl's command-completeness /
#continuation engine (added 2026-07-11 at the user's direction, ahead of any repl
#refactor goals). Given a partial command string it returns the stack of pending
#"openers" awaiting closure ({} when the command is complete per [info complete]).
#The repl uses this for:
# - deciding whether to evaluate or wait for continuation lines (info complete parity,
# including Tcl's quoting-rules quirk: a quoted word containing unbalanced braces is
# complete STANDALONE but inside a braced context (e.g a proc body) brace counting
# applies, so extra closing characters must be typed - surprising but correct Tcl)
# - the closing-prompt hints in line and raw mode (the stack top is what the >{ / >"
# continuation hints render). Known, accepted limitation: when more than one closer
# candidate exists the repl expresses only one - pinned here as current behaviour,
# not a fix target.
#
#These are PURE function tests runnable under plain tclsh. The raw-mode editbuf side
#(tab ⇥ markers, dim space dots, colour staging, history navigation) is currently
#console-coupled (class_editbuf add_chunk renders via overtype::renderline against live
#terminal metrics) and needs a console seam (G-001) or pseudoconsole harness before it
#can be characterized - see goals/G-044 detail notes.
#
#NOTE: the deliberately-unbalanced case strings cannot appear as braced literals in
#this (or any) tcl file - they are built from escaped fragments.
namespace eval ::testspace {
namespace import ::tcltest::*
variable common {
set result ""
}
variable Q "\""
variable OB "\{"
variable CB "\}"
#the user-reported scenario: works standalone, needs extra closers inside a proc body
variable standalone "set x ${Q}${OB}*${CB}${OB}${Q}"
variable inproc "proc j ${OB}${CB} ${OB}\n set x ${Q}${OB}*${CB}${OB}${Q}"
test commandcomplete_standalone_quoted_braces {a quoted word containing unbalanced braces is a complete command standalone}\
-setup $common -body {
variable standalone
lappend result [info complete $standalone]
lappend result [punk::lib::system::incomplete $standalone]
}\
-cleanup {
}\
-result [list 1 {}]
test commandcomplete_inproc_progression {the same line inside a proc body needs extra closers - the pending-opener stack shrinks as each is typed}\
-setup $common -body {
variable inproc
variable Q
variable CB
set s $inproc
lappend result [info complete $s] [punk::lib::system::incomplete $s]
append s "\n${CB}"
lappend result [info complete $s] [punk::lib::system::incomplete $s]
append s "\n${Q}"
lappend result [info complete $s] [punk::lib::system::incomplete $s]
append s "\n${CB}"
lappend result [info complete $s] [punk::lib::system::incomplete $s]
}\
-cleanup {
}\
-result [list 0 [list \{ \" \{ \"] 0 [list \{ \"] 0 [list \{] 1 {}]
test commandcomplete_single_openers {unterminated quote, brace and bracket each report their single pending opener}\
-setup $common -body {
variable Q
variable OB
lappend result [punk::lib::system::incomplete "puts ${Q}hello"]
lappend result [punk::lib::system::incomplete "puts ${OB}hello"]
lappend result [punk::lib::system::incomplete "puts \[clock seconds"]
}\
-cleanup {
}\
-result [list [list \"] [list \{] [list \[]]
test commandcomplete_tab_in_open_brace {literal tabs inside an open braced context do not disturb the pending-opener stack}\
-setup $common -body {
variable OB
lappend result [punk::lib::system::incomplete "set z ${OB}\n line1\ttabbed"]
}\
-cleanup {
}\
-result [list [list \{]]
test commandcomplete_escapes {backslash-escaped braces and quotes do not open/close contexts}\
-setup $common -body {
variable Q
lappend result [info complete "puts \\${Q}"] [punk::lib::system::incomplete "puts \\${Q}"]
lappend result [info complete "puts \\\{"] [punk::lib::system::incomplete "puts \\\{"]
#an escaped quote inside an open quoted word leaves the quote pending
lappend result [punk::lib::system::incomplete "puts ${Q}a\\${Q}b"]
}\
-cleanup {
}\
-result [list 1 {} 1 {} [list \"]]
test commandcomplete_parity_with_info_complete {incomplete returns the empty list exactly when info complete reports true (property over the case battery)}\
-setup $common -body {
variable Q
variable OB
variable CB
variable standalone
variable inproc
set battery [list {puts hello} $standalone $inproc "$inproc\n${CB}" "$inproc\n${CB}\n${Q}" "$inproc\n${CB}\n${Q}\n${CB}" "puts ${Q}hello" "puts ${OB}hello" "puts \[clock seconds" "set z ${OB}\n line1\ttabbed" "puts \\${Q}" "if ${OB}1${CB} ${OB}\n puts ok\n${CB}"]
set mismatches [list]
foreach c $battery {
set emptystack [expr {[llength [punk::lib::system::incomplete $c]] == 0}]
if {$emptystack != [info complete $c]} {
lappend mismatches $c
}
}
lappend result $mismatches
}\
-cleanup {
}\
-result [list {}]
}
tcltest::cleanupTests ;#needed to produce test summary line.
Loading…
Cancel
Save