punkshell/goals/G-061-pseudoconsole-expect.md

G-061 Pseudoconsole expect-alternative for interactive shell testing (ConPTY + unix pty)

Status: proposed Scope: test-harness support (location TBD during the work: src/tests/testsupport/ and/or a small punk module), src/tests/shell/ (capability-gated interactive suites), goals/G-061-pseudoconsole-expect.md Goal: interactive punkshell behaviour is testable headlessly and byte-accurately by driving a real shell under a pseudoconsole - ConPTY on windows, pty on unix - through an expect-like harness (spawn, send literal keys and control/arrow sequences, await patterns in the rendered ANSI byte stream with timeouts, collect transcript), giving the goals/G-044-detail preserve-list behaviours (continuation hints, raw-mode colour staging, tab markers and space dots, multiline history navigation) their durable cross-platform verification tier - complementary to G-020 capture testing (which verifies what reaches the screen; this verifies the byte stream) and to the G-001 console-seam unit tier. Acceptance: a capability-gated harness spawns a built punkshell under ConPTY on windows (and under a unix pty when run on unix or via the G-059 WSL staging pattern), sends keys including control/arrow sequences, and awaits expected patterns with timeouts and forced teardown on expiry (hang-proof per the shell-test conventions); at least three preserve-list behaviours are verified through it (e.g the in-proc set x "{*}{" continuation-hint sequence, raw-mode tab-marker rendering and deletion, up-arrow recall and edit of a multiline history entry); tests skip cleanly where no pseudoconsole is available; the harness API is platform-agnostic with per-OS backends; candidate synchronisation/communication mechanisms between harness and shell (including the inter-subshell beep-protocol idea) are evaluated and the chosen approach documented in the detail file.

Context

punkshell's interactive behaviour has no automated verification. The preserve-list recorded in goals/G-044-repl-command-completion.md (2026-07-11, user-specified) covers what any repl refactor must not break: info-complete-parity continuation with the Tcl quoting quirk, closing-prompt hints, raw-mode colour staging (in-progress vs submitted), literal tab acceptance with raw-mode tab markers and edit smarts, dim grey space dots (display-only - never in the submitted string), and up/down navigation + editing of multiline editbuf history entries.

The verification tier model established 2026-07-11:

1. PURE layer - testable now: punk::lib::system::incomplete characterized in src/tests/modules/punk/lib/testsuites/lib/commandcomplete.test.
2. Console-seam UNIT tier - blocked on a seam: class_editbuf is console-coupled at its core (add_chunk renders via overtype::renderline against live terminal metrics - cursor-position size probing, DECRQPSR tabstops). An ::opunk::Console test double (G-001) or injectable metrics is the enabling refactor.
3. RENDERED tier: G-020 screencap+input injection is the near-term windows-only bridge (verifies what reaches the screen); THIS goal is the durable, headless, cross-platform, byte-accurate mechanism (verifies the byte stream the shell emits).

Approach

Harness contract (platform-agnostic; per-OS backends)

expect-like primitives over a REAL pseudo-terminal:

• spawn: launch a built punkshell attached to a pseudoconsole (ConPTY on windows, openpty/forkpty on unix) with controlled size (cols x rows) - size control matters because raw-mode rendering is width-dependent.
• send: literal keys plus control/arrow/escape sequences (up-arrow history recall, backspace over a tab marker, etc).
• expect: await patterns in the rendered ANSI byte stream with timeouts; forced teardown on expiry (hang-proof, per src/tests/shell/AGENTS.md conventions - the punk_run pattern generalized to a live conversation).
• transcript: full byte capture for post-hoc assertions (colour staging = SGR sequences around the in-progress vs submitted line; tab markers/space dots = the substituted glyph bytes).

Open implementation questions (deliberately left to the work)

• Driving ConPTY from Tcl: candidates include a twapi surface (if CreatePseudoConsole is reachable), a small helper executable owning the ConPTY pair and relaying bytes over stdio/sockets, or an existing tool at arm's length. The unix side is simpler (openpty via a small C/critcl shim, or driving through an existing pty-capable tool).
• Synchronisation between harness and shell: pattern-awaiting alone can be racy for timing-sensitive raw-mode behaviour. Candidate: the inter-subshell communication / beep-protocol idea (user, 2026-07-11) - an in-band signalling channel the shell can emit for the harness to key off. Evaluate against simpler prompt-sentinel approaches; document the choice here.
• Unix execution from the windows dev machine: the G-059 WSL staging pattern (native-filesystem staging, capability constraint) extends naturally - a pty harness run inside the distro against a linux punkbin runtime/kit.

Sequencing

• G-001 console-seam unit tier and G-020 capture tier are complementary, not prerequisites - but preserve-list items verified at a cheaper tier need not be re-verified here beyond smoke coverage.
• First verification targets (per acceptance): the in-proc set x "{*}{" continuation-hint sequence, raw-mode tab-marker rendering + deletion, up-arrow recall and edit of a multiline history entry.

Notes

• Reference material (added 2026-07-21): TEMP_REFERENCE/expect-6.0a1 is a readonly checkout of the community Tcl 9 port of classic Expect (contributions from Steve Shaw, Paul Obermeier, OpenMandriva, Ashok P. Nadkarni; per its README: builds on Tcl 8.6 and 9.x, work-in-progress alpha, only Linux tested). It is unix-pty only - it contains NO ConPTY/windows backend (the _WIN32 occurrences are autoconf/tcl.m4 boilerplate) - so it does not solve this goal's windows side. Its value here: (a) the authoritative reference for expect primitive semantics this harness mirrors - spawn/send/expect/interact, timeout and eof handling, pattern matching over the pty byte stream (exp_command.c, exp_inter.c, exp_event.c, exp_chan.c); (b) a concrete candidate for the "existing tool at arm's length" option for the unix backend (see open implementation questions above) - a Tcl 9-compatible expect proper may serve as the unix pty driver, with the harness contract layered over it; (c) its pty allocation code (pty_termios.c etc) as reference if we build our own openpty shim instead.
• Related: G-044 (completion/hinting - consumes this for its own verification), G-001 (fake-console unit tier), G-020 (screen-capture tier), G-059 (WSL execution pattern), G-060 (guest-driving contract - a pty harness inside a QEMU guest is the eventual cross-platform matrix combination).
• ConPTY availability gates the windows side to Win10 1809+ (fine for punkshell's supported baseline); the capability probe should verify actual ConPTY function, not windows version.
• Console-context test-matrix findings 2026-07-19 (user console-attached sequential runtests runs; a concrete workload for this goal's harness): three tests fail ONLY under a HALF-attached console state (stderr = real console, stdout = redirected file) - punk/ansi grepstr.test grepstr_return_all_highlight_exact and grepstr_tab_warning_and_passthrough emit EMPTY SGR (ESC[m) where the expected term-teal codes should be, and punk/ns nslist.test nslist_marking_exported_imported loses its markers (0 0 0). Class: punk::console PROCESS-GLOBAL colour state reacting to mixed tty-ness in singleproc runs (fully-piped and -jobs child-process runs are green; fully-interactive stdout+stderr-console behaviour is UNVERIFIED - exactly the context matrix a pseudoconsole harness makes testable deterministically). Interim repro technique without ConPTY: Start-Process a .cmd wrapper with selective stdio redirection (stderr left on the spawned console) - used 2026-07-19 to root-cause the related-but-distinct capture_stderr defect (a chan-push transform on console-backed stderr receives the driver's utf-16le wide-char bytes; FIXED in the three capture_stderr helpers, commit 913ea749). The empty-SGR/marker findings remain open; a pseudoconsole-driven context matrix (fully-attached / half-attached / detached) over these suites is a natural early acceptance workload alongside the preserve-list behaviours.
• G-093 (archived) recorded a further cell of the console-context matrix while verifying runtests targeting: children spawned by the agent PowerShell tool run with an ATTACHED console while BOTH stdio streams are redirected, and the same colour class fires (grepstr/nslist failures; agent Bash tool children are console-free and green) - see goals/archive/G-093-runtests-includepaths-discovery.md.
• Archived-goal references in this file: G-001 achieved 2026-07-11 (goals/archive/G-001-pluggable-console-backends.md);G-059 achieved 2026-07-11 (goals/archive/G-059-wsl-test-driving.md);G-093 achieved 2026-07-20 (goals/archive/G-093-runtests-includepaths-discovery.md).