From bac122da9f306a03f67407fd1063072c6b012ee5 Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Sun, 12 Jul 2026 18:50:08 +1000 Subject: [PATCH] G-039 achieved: punk::repl 0.5.0 dead-console watchdog stops orphaned-shell CPU spin (project 0.12.2) An interactive shell orphaned by its hosting console dying (killed conhost/terminal) previously spun ~2 cores forever: the Tcl 9 windows console driver never delivers the dead-console state to the script as a fileevent (tclWinConsole.c ConsoleEventProc only notifies on buffered data) while its reader thread busy-loops on the persistent channel error, so the script level is completely blind and nothing ever closes the channel. Fix: repl::console_watchdog - a self-rescheduling liveness poll (default 5s, repl::console_watchdog_ms; read-only GetConsoleMode probe via chan configure -inputmode) armed by repl::start only for a tcl9 console input channel serving the process-default console on windows. On probe failure it closes the input channel (stopping the driver's reader thread) and finishes the repl via the normal eof done-path; app-punkshell then finds no console reopenable and exits cleanly. Piped, foreign-console and tcl 8.6 inputs never arm it. repl::start's post-vwait reader deregistration now tolerates a watchdog-closed channel. Verification (2026-07-12): kill procedure post-fix exits the orphan in 1.5-2.0s with no traceback (src mode and rebuilt kits punkbi/punk91/ punk902z); 25s live-console soak shows no spurious trigger, CPU delta 0; piped stdin unaffected; repl consolebackends 3/3 and punk::console suites 88 pass / 1 skip; root cause verified against plain tclsh 9.0.3. Upstream ticket filed: https://core.tcl-lang.org/tcl/tktview/f10d91c2d3 Included: - goals: G-039 flipped achieved 2026-07-12 and archived (index entry to GOALS-archive.md, detail file to goals/archive/ with full root-cause, repro and verification record); reference sweep adds the eof done-value producer note to G-038 - src/modules/punk/AGENTS.md: watchdog contract documented - project version 0.12.1 -> 0.12.2 + CHANGELOG entry - vfs payload sync: _vfscommon.vfs punk/repl 0.4.0 -> 0.5.0 Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- CHANGELOG.md | 4 + GOALS-archive.md | 4 + GOALS.md | 4 - goals/G-038-piped-session-continuity.md | 10 +- goals/G-039-orphan-console-spin.md | 49 ------ goals/archive/G-039-orphan-console-spin.md | 157 ++++++++++++++++++ punkproject.toml | 2 +- src/modules/punk/AGENTS.md | 1 + src/modules/punk/repl-999999.0a1.0.tm | 83 ++++++++- src/modules/punk/repl-buildversion.txt | 3 +- .../punk/{repl-0.4.0.tm => repl-0.5.0.tm} | 85 +++++++++- 11 files changed, 343 insertions(+), 59 deletions(-) delete mode 100644 goals/G-039-orphan-console-spin.md create mode 100644 goals/archive/G-039-orphan-console-spin.md rename src/vfs/_vfscommon.vfs/modules/punk/{repl-0.4.0.tm => repl-0.5.0.tm} (98%) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ce03af6..0c2442de 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` "Project Versioning" section for the bump policy. +## [0.12.2] - 2026-07-12 + +- G-039 fix: an interactive shell orphaned by its hosting console dying (killed conhost/terminal) previously spun ~2 CPU cores forever — the Tcl 9 windows console driver never delivers the dead-console state to the script level as a fileevent (tclWinConsole.c `ConsoleEventProc` only notifies on buffered data) while its reader thread busy-loops on the persistent channel error. punk::repl 0.5.0 adds `repl::console_watchdog`, a 5s liveness poll (read-only GetConsoleMode via `chan configure -inputmode`) armed only for a tcl9 console input channel on the process-default console on windows: on a dead console it closes the input channel (stopping the driver's reader thread) and finishes the repl via the normal eof path, so the orphan exits cleanly within seconds. Piped, foreign-console and tcl 8.6 inputs are unaffected. + ## [0.12.1] - 2026-07-11 - G-062 achieved: canonical project license declared as BSD-2-Clause. `LICENSE.txt` added at the repo root with the standard BSD-2-Clause text (copyright Julian Marcel Noble, 2023-2026); `README.md` names BSD-2-Clause and points at `LICENSE.txt`; root `AGENTS.md` Repo-wide Notes names the license precisely; `punkproject.toml` `[project]` carries `license = "BSD-2-Clause"`. Repo-sweep found no top-level license claims contradicting it (vendored library licenses and module-level generated docs are not project-level claims). diff --git a/GOALS-archive.md b/GOALS-archive.md index 1b76124a..e880a091 100644 --- a/GOALS-archive.md +++ b/GOALS-archive.md @@ -38,6 +38,10 @@ Acceptance: (reworked 2026-07-08 after the root cause was found) the wedge mecha Scope: src/make.tcl (new or extended step), src/vendorlib_tcl8 + src/vendorlib_tcl9 (sources), src/vfs/.vfs/lib_tcl8 + lib_tcl9 (targets), punkcheck tracking Acceptance: with a newer package version placed under src/vendorlib_tcl9/, one documented make.tcl invocation updates the participating src/vfs/*/lib_tcl9 trees - installing the new package and removing or explicitly retiring the superseded version (no silent mixed-version provision, per the G-035 concerns) - with punkcheck-tracked provenance; which vfs folders participate is explicitly declared per kit rather than blanket-copied (kit vfs package sets may intentionally differ), with the declaration mechanism recorded (candidate home: the G-024 mapvfs toml); a subsequent `make.tcl project` yields kits loading the new version (provable via the tcludp case: built punk902z reports `package require udp` == 1.0.13 with no udp1.0.12 folder remaining in its vfs); the lib_tcl8 tree gets the same treatment or an explicit exclusion rationale in the goal record. +### G-039 [achieved 2026-07-12] Investigate the orphaned-shell one-core spin on a dead console → detail: goals/archive/G-039-orphan-console-spin.md +Scope: src/modules/punk/repl-999999.0a1.0.tm (console reader/event loop and EOF/error paths), src/modules/punk/console-999999.0a1.0.tm; investigation-first +Acceptance: a documented procedure reproduces the spin on the current kit (e.g. launch an interactive shell in a terminal, then kill/close the hosting terminal or conhost), or the investigation records the attempts made and what evidence would reopen it; the spinning code path is identified (prime suspect: a console read/event loop treating a dead console's immediate EOF/error as retryable without backoff or termination - adjacent to the console-EOF restart path G-038 takes ownership of); after fix/mitigation, the same procedure shows the orphaned process exiting or settling at effectively zero CPU within a short grace period, with live-console interactive behaviour unchanged; the wedge-scoring hazard note (orphans polluting process-liveness checks in test harnesses) is updated to match the outcome. + ### G-040 [achieved 2026-07-08] punk::args choice aliasing (-choicealiases) with parse normalization, display folding, and doc-lookup parity → detail: goals/archive/G-040-punkargs-choicealiases.md Scope: src/modules/punk/args-999999.0a1.0.tm (parse + usage rendering), src/modules/punk/ns-999999.0a1.0.tm (cmdinfo/cmd_traverse choice resolution parity), src/modules/punk-999999.0a1.0.tm (punk::help topic argdoc as first consumer), src/tests/modules/punk/args/testsuites/, src/tests/modules/punk/ns/testsuites/ Acceptance: a definition using -choicealiases parses an alias (and an alias prefix where -choiceprefix allows) to its canonical choice in the parse result, with -choicerestricted 0 passthrough and the deny/reserve lists honoured unchanged; usage display shows one entry per canonical choice with aliases folded (no duplicate rows; -choicelabels attach to the canonical); punk::ns::cmdinfo/cmd_traverse resolve subcommand words to docids with the same outcome as the parser for alias, prefix, denied, reserved and unknown words (the pre-goal characterization tests updated from pinned-GAP to fixed); punk::help's topic definition adopts the feature so `i help` lists one entry per registered topic while `help h`/`help e` still fall through to command lookup; definitions without -choicealiases behave unchanged (existing punk::args and punk::ns suites pass). diff --git a/GOALS.md b/GOALS.md index fd3d86bd..b5ca7a7d 100644 --- a/GOALS.md +++ b/GOALS.md @@ -194,10 +194,6 @@ Detail: goals/G-035-mixed-tm-pkgindex-provision.md Scope: src/lib/app-punkshell/punkshell.tcl (eof-restart handover), src/modules/punk/repl-999999.0a1.0.tm (eof-restart done-mode that skips codethread teardown), src/modules/punk/repl/codethread-999999.0a1.0.tm (as touched) Detail: goals/G-038-piped-session-continuity.md -### G-039 [proposed] Investigate the orphaned-shell one-core spin on a dead console -Scope: src/modules/punk/repl-999999.0a1.0.tm (console reader/event loop and EOF/error paths), src/modules/punk/console-999999.0a1.0.tm; investigation-first -Detail: goals/G-039-orphan-console-spin.md - ### G-041 [proposed] punk::args multi-form matching: automated form selection for parsing and documentation Scope: src/modules/punk/args-999999.0a1.0.tm (parse form selection, arg_error/usage form marking), src/modules/punk/ns-999999.0a1.0.tm (cmdhelp/synopsis closest-form indication), src/tests/modules/punk/args/testsuites/ Detail: goals/G-041-punkargs-form-matching.md diff --git a/goals/G-038-piped-session-continuity.md b/goals/G-038-piped-session-continuity.md index fccf785d..a17a6840 100644 --- a/goals/G-038-piped-session-continuity.md +++ b/goals/G-038-piped-session-continuity.md @@ -151,6 +151,14 @@ Risks / interactions: G-002 (non-nested subshell architecture - same family of repl lifecycle decoupling); G-031 (componentized boot); G-001/G-011 (console object rebinding on reopen); G-015 (script subcommand unaffected - it never drops to interactive). +- G-039 (archived) root-caused the dead-console orphan spin (Tcl 9 tclWinConsole.c never + delivers a dead console as a fileevent; reader thread busy-loops) and added + repl::console_watchdog (punk::repl 0.5.0), which closes a dead process-console input + and finishes the repl with done={eof } - a third producer of the eof done-value + this goal's caller-driven restart must handle (its console-reopen failure branch shares + the same decision point: CONIN$ unopenable => exit cleanly, never retry-loop). The + watchdog's arming is per repl::start frame; the restart variant must re-arm it for the + renewed channel. See goals/archive/G-039-orphan-console-spin.md. ### Post-restart console-query harness (2026-07-08) @@ -181,4 +189,4 @@ object's channels). State observed pre-fix, for reference: repl thread post-rest `stdin` eof=1 (the exhausted pipe) alongside the new `file...` CONIN$ channel; the code interp's stdin is a live console channel (utf-16, -inputmode present); anchored instances known to the repl thread: `default` (anchored to the dead pair). -- Archived-goal references in this file: G-001 achieved 2026-07-11 (goals/archive/G-001-pluggable-console-backends.md);G-007 achieved 2026-07-05 (goals/archive/G-007-console-location-transparency.md);G-015 achieved 2026-07-07 (goals/archive/G-015-script-subcommand-piped-stdin.md);G-036 achieved 2026-07-08 (goals/archive/G-036-tcl9-udp-console-worker-wedge.md). +- Archived-goal references in this file: G-001 achieved 2026-07-11 (goals/archive/G-001-pluggable-console-backends.md);G-007 achieved 2026-07-05 (goals/archive/G-007-console-location-transparency.md);G-015 achieved 2026-07-07 (goals/archive/G-015-script-subcommand-piped-stdin.md);G-036 achieved 2026-07-08 (goals/archive/G-036-tcl9-udp-console-worker-wedge.md);G-039 achieved 2026-07-12 (goals/archive/G-039-orphan-console-spin.md). diff --git a/goals/G-039-orphan-console-spin.md b/goals/G-039-orphan-console-spin.md deleted file mode 100644 index da78b19d..00000000 --- a/goals/G-039-orphan-console-spin.md +++ /dev/null @@ -1,49 +0,0 @@ -# G-039 Investigate the orphaned-shell one-core spin on a dead console - -Status: proposed -Scope: src/modules/punk/repl-999999.0a1.0.tm (console reader/event loop and EOF/error paths), src/modules/punk/console-999999.0a1.0.tm; investigation-first -Goal: the observed failure mode - an interactive punk902z left running after its hosting terminal/console went away spins roughly a full core indefinitely (observed 2026-07-08: a 37-minute orphan with a single hard-looping thread) - is reliably reproduced and root-caused, then fixed or mitigated so a shell whose console dies exits or reaches zero-CPU idle cleanly. -Acceptance: a documented procedure reproduces the spin on the current kit (e.g. launch an interactive shell in a terminal, then kill/close the hosting terminal or conhost), or the investigation records the attempts made and what evidence would reopen it; the spinning code path is identified (prime suspect: a console read/event loop treating a dead console's immediate EOF/error as retryable without backoff or termination - adjacent to the console-EOF restart path G-038 takes ownership of); after fix/mitigation, the same procedure shows the orphaned process exiting or settling at effectively zero CPU within a short grace period, with live-console interactive behaviour unchanged; the wedge-scoring hazard note (orphans polluting process-liveness checks in test harnesses) is updated to match the outcome. - -## Context - -Observation (2026-07-08, during the G-036 investigation): a punk902z process from an earlier -interactive session (started ~37 minutes prior; its hosting terminal presumed closed) was -found consuming CPU continuously - ~1550 CPU-seconds accumulating at roughly +0.5-1 -core-seconds per wall second, with exactly ONE thread in Running state (1546s of the total) -and every other thread idle in normal waits. The process had to be killed manually. It also -polluted the wedge-scoring checks of the day (process-liveness was being used as a hang -signal), which is how it was noticed. - -Not diagnosed at the time (killed to unblock the G-036 work); no dump was taken. The -suspicion: when the hosting console goes away, a console channel read/event path returns -immediately (EOF or error) and the surrounding loop retries without backoff or a -give-up-and-exit decision. Candidate sites: the repl reader loop, the EOF branch behaviour -when reopen/restart fails or loops (note tcl_interactive would be true for a real console, -enabling the `after 1 reopen_stdin` path - see the race notes in the G-038 detail file), or -punk::console query/read helpers. - -## Approach - -1. Reproduce: launch an interactive shell in a disposable terminal and kill the host - (close the tab/window; `Stop-Process` on the terminal; for classic conhost, killing - conhost.exe; also try Windows Terminal vs conhost - behaviour may differ). Watch the - orphan's CPU and thread states (`(Get-Process punk902z).Threads` sorted by - TotalProcessorTime). Try both idle-at-prompt and mid-command states at kill time. -2. If reproduced: procdump + WinDbgX scripted stack capture of the spinning thread (the - G-036 detail file documents the working tooling pipeline and pitfalls - WinDbgX process - name is DbgX.Shell, cdb is ACL-buried, `-z/-p -c -logo` scripting works). With the spin - thread's stack, map to the retry loop and decide fix: treat dead-console EOF/error as - terminal (exit per PUNK_PIPE_EOF-like policy), or add backoff + detection. -3. Coordinate with G-038: its caller-driven restart owns the console-EOF path; the dead - console case is the failure branch of the same decision point (reopen CONIN$ fails or - the console is gone entirely) and should be designed together. - -## Notes - -- Related: G-038 (console-EOF restart ownership, reopen_stdin race notes), G-036 (dump/ - debugging tooling pipeline), tcl86-console-parked-read prior art (different mechanism, - same neighbourhood). -- Harness hygiene until fixed: verify no stray punk902z processes before scoring any - liveness-based test run. -- Archived-goal references in this file: G-036 achieved 2026-07-08 (goals/archive/G-036-tcl9-udp-console-worker-wedge.md). diff --git a/goals/archive/G-039-orphan-console-spin.md b/goals/archive/G-039-orphan-console-spin.md new file mode 100644 index 00000000..92ee9c12 --- /dev/null +++ b/goals/archive/G-039-orphan-console-spin.md @@ -0,0 +1,157 @@ +# G-039 Investigate the orphaned-shell one-core spin on a dead console + +Status: achieved 2026-07-12 +Scope: src/modules/punk/repl-999999.0a1.0.tm (console reader/event loop and EOF/error paths), src/modules/punk/console-999999.0a1.0.tm; investigation-first +Goal: the observed failure mode - an interactive punk902z left running after its hosting terminal/console went away spins roughly a full core indefinitely (observed 2026-07-08: a 37-minute orphan with a single hard-looping thread) - is reliably reproduced and root-caused, then fixed or mitigated so a shell whose console dies exits or reaches zero-CPU idle cleanly. +Acceptance: a documented procedure reproduces the spin on the current kit (e.g. launch an interactive shell in a terminal, then kill/close the hosting terminal or conhost), or the investigation records the attempts made and what evidence would reopen it; the spinning code path is identified (prime suspect: a console read/event loop treating a dead console's immediate EOF/error as retryable without backoff or termination - adjacent to the console-EOF restart path G-038 takes ownership of); after fix/mitigation, the same procedure shows the orphaned process exiting or settling at effectively zero CPU within a short grace period, with live-console interactive behaviour unchanged; the wedge-scoring hazard note (orphans polluting process-liveness checks in test harnesses) is updated to match the outcome. + +## Context + +Observation (2026-07-08, during the G-036 investigation): a punk902z process from an earlier +interactive session (started ~37 minutes prior; its hosting terminal presumed closed) was +found consuming CPU continuously - ~1550 CPU-seconds accumulating at roughly +0.5-1 +core-seconds per wall second, with exactly ONE thread in Running state (1546s of the total) +and every other thread idle in normal waits. The process had to be killed manually. It also +polluted the wedge-scoring checks of the day (process-liveness was being used as a hang +signal), which is how it was noticed. + +Not diagnosed at the time (killed to unblock the G-036 work); no dump was taken. The +suspicion: when the hosting console goes away, a console channel read/event path returns +immediately (EOF or error) and the surrounding loop retries without backoff or a +give-up-and-exit decision. Candidate sites: the repl reader loop, the EOF branch behaviour +when reopen/restart fails or loops (note tcl_interactive would be true for a real console, +enabling the `after 1 reopen_stdin` path - see the race notes in the G-038 detail file), or +punk::console query/read helpers. + +## Approach + +1. Reproduce: launch an interactive shell in a disposable terminal and kill the host + (close the tab/window; `Stop-Process` on the terminal; for classic conhost, killing + conhost.exe; also try Windows Terminal vs conhost - behaviour may differ). Watch the + orphan's CPU and thread states (`(Get-Process punk902z).Threads` sorted by + TotalProcessorTime). Try both idle-at-prompt and mid-command states at kill time. +2. If reproduced: procdump + WinDbgX scripted stack capture of the spinning thread (the + G-036 detail file documents the working tooling pipeline and pitfalls - WinDbgX process + name is DbgX.Shell, cdb is ACL-buried, `-z/-p -c -logo` scripting works). With the spin + thread's stack, map to the retry loop and decide fix: treat dead-console EOF/error as + terminal (exit per PUNK_PIPE_EOF-like policy), or add backoff + detection. +3. Coordinate with G-038: its caller-driven restart owns the console-EOF path; the dead + console case is the failure branch of the same decision point (reopen CONIN$ fails or + the console is gone entirely) and should be designed together. + +## Notes + +- Related: G-038 (console-EOF restart ownership, reopen_stdin race notes), G-036 (dump/ + debugging tooling pipeline), tcl86-console-parked-read prior art (different mechanism, + same neighbourhood). +- Harness hygiene (resolved 2026-07-12): kits were rebuilt with punk::repl 0.5.0 the same + day (plain-kit kill test re-verified: exit within 2.0s), so the stray-orphan precaution + applies only when deliberately testing pre-0.5.0 kits. +- Upstream: the root cause is a Tcl 9 core defect pair in win/tclWinConsole.c (see + Progress below): (a) ConsoleEventProc drops the error/EOF notification that + ConsoleSetupProc/ConsoleCheckProc generate for lastError != 0 (only ring-buffer data + produces a Tcl_NotifyChannel), so a script can never see a dead console via fileevent; + (b) ConsoleReaderThread's persistent-error branch (inputLen==0 && lastError!=0) wakes + CVs + NudgeWatchers and continues with no wait, busy-looping one core (and via the + wakeups a second) until the channel is closed. Both defects verified against a plain + stock tclsh 9.0.3 (repro script with a stdin fileevent + vwait: ~1.85 cores after + conhost kill, readable handler never fires) and the code is unchanged in the 9.1b1 + reference sources. A ready-to-file ticket draft with the standalone repro lives at + `TEMP_REFERENCE/tcl9-dead-console-spin-TICKET-DRAFT.md`. A kit-side Tcl source patch + is an alternative once G-005/G-006 build infrastructure exists; the script-level + watchdog stays valid regardless. +- Archived-goal references in this file: G-036 achieved 2026-07-08 (goals/archive/G-036-tcl9-udp-console-worker-wedge.md). + +## Progress + +### 2026-07-12 reproduced, root-caused, fixed in source (punk::repl 0.5.0) + +Reproduction procedure (reliable, first-try, current punk902z kit 2026-07-11 build): + +1. Launch the shell interactively under a dedicated classic conhost (isolates the kill + from the user's terminal): + `Start-Process -WindowStyle Hidden conhost.exe -ArgumentList 'C:\repo\jn\shellspy\bin\punk902z.exe'` + (optionally via cmd.exe with `> out.txt 2> err.txt` redirection to capture the trail; + `punk902z src` for dev modules). +2. Confirm the tree (`punk902z` is a child of the new conhost) and let it settle at the + idle prompt (CPU delta 0 over several seconds). +3. `Stop-Process -Force` the conhost pid. punk902z survives and immediately spins: + ~1.7-1.9 cores, matching the 2026-07-08 observation (that one showed ~1 core; both + hot threads here are part of the same mechanism). Both idle-at-prompt kill state and + the redirected-stdio variant reproduce. + +Spin mechanism (dump: procdump64 -ma + `WinDbgX -z -c "~*k 30; .logclose; q" -logo`, +per the G-036 tooling notes; mapped against TEMP_REFERENCE/tcl9/win/tclWinConsole.c): + +- Hot thread 1 = the Tcl console reader thread (stack: thread start -> ConsoleReaderThread + -> SetEvent storm). After the console dies, ConsoleDataAvailable returns -1 + (PeekConsoleInputW fails) which the caller treats as truthy, ReadConsoleChars fails + (observed error: broken pipe - ERROR_BROKEN_PIPE, not ERROR_INVALID_HANDLE) and + handleInfoPtr->lastError is set. From then on the reader thread's top-of-loop branch + (inputLen>0 || lastError!=0) runs every iteration - WakeAllConditionVariable + + NudgeWatchers(SetEvent/Tcl_ThreadAlert) + continue - with no sleep on that path, until + channel close drops numRefs to 1. lastError is never cleared because the script never + reads again (see next point). +- Hot thread 2 = the repl/main interp thread: each nudge alerts its notifier; + ConsoleSetupProc counts lastError as "readable" and sets max block time 0, and + ConsoleCheckProc queues a console event for it - but ConsoleEventProc only calls + Tcl_NotifyChannel when the ring buffer has DATA and ignores lastError entirely, so the + queued event is discarded and the armed readable fileevent (repl_handler) NEVER fires. + The event loop spins at zero block time queueing and discarding events. +- Script level is therefore completely blind: instrumented src-mode run confirmed + repl_handler never runs after console death (no entry, no EOF branch, no reopen_stdin, + no bgerror, vwait never returns), so nothing ever closes the channel and both spins are + permanent. The reopen_stdin restart loop suspected in the original goal statement is NOT + the mechanism - it never gets the chance to run. + +Fix (src/modules/punk/repl-999999.0a1.0.tm, punk::repl 0.5.0): + +- New `repl::console_watchdog` self-rescheduling liveness poll (default 5000ms, + `repl::console_watchdog_ms`), armed by repl::start before its vwait, only when platform + is windows AND the repl serves the process-default console AND the input channel's + `chan configure` dict has `-inputmode` (i.e. a tcl9 console channel; piped stdin, + foreign consoles and tcl 8.6 console channels never arm it). The probe is a read-only + `chan configure $inchan -inputmode` (live GetConsoleMode). On probe failure: disarm, + close the input channel (this is what stops the reader-thread spin - numRefs drops and + the thread exits), and set `::repl::done {eof }` so the repl finishes via the + normal eof path; app-punkshell's eof handling then finds CONIN$ unopenable ("broken + pipe") and exits cleanly (heuristic policy, exit 0). repl::start's post-vwait + `chan event $inchan readable {}` deregistration is now guarded for the + watchdog-closed-channel case (previously an unguarded call raised a traceback on this + path). Scheduling state per channel name in `repl::console_watchdog_afterids`; + cancelled after vwait by the arming frame. + +Verification (2026-07-12, punk902z kit 2026-07-11 in `src` mode - dev modules; Tcl 9.0.2 +kit runtime): + +- Same kill procedure post-fix: watchdog message on stderr, orphan EXITED 1.5s after the + conhost kill (worst case is ~watchdog interval + teardown), no traceback. Remaining + teardown noise (disableRaw warning, codethread tsv warning) is cosmetic, printed to an + already-dead console in the real scenario. +- Live-console soak: 25s at the idle prompt (>=4 watchdog probes) - process alive, CPU + delta 0, prompt intact; no spurious trigger. +- Piped stdin (`'set ::x 1' | punk902z src` with PUNK_PIPE_EOF=exit): unaffected, exit 0 + (watchdog not armed for pipes). +- runtests (native tclsh 9.0.3): repl consolebackends.test 3/3 pass; punk::console suites + (console/*.test) 88 pass / 1 skip / 0 fail. +- `tclsh src/make.tcl modules` builds clean (provenance warnings = expected dirty tree). + +### 2026-07-12 acceptance closed (kits rebuilt, user verification) + +All remaining items resolved: + +- Kits rebuilt 2026-07-12 with punk::repl 0.5.0 (punkbi, punk91, punk902z). Plain-kit + kill procedure re-verified on the rebuilt punk902z: orphan exited 2.0s after the + conhost kill via the clean eof path (watchdog message, app-punkshell "no console + available", exit 0), no traceback. +- User verified interactive sessions on the rebuilt kits (punkbi, punk91, punk902z, + punk::repl confirmed at 0.5.0): behaviour unchanged with the 5s probe - "all seems + well". +- Tcl 8.6 (punksys) confirmed OUT OF SCOPE by the user 2026-07-12: different console + driver, watchdog deliberately not armed there; reopen as a new goal if an 8.6 orphan + spin is ever observed. +- Wedge-scoring hazard note updated (see Notes - precaution now applies only to + pre-0.5.0 kits). +- Root cause verified independent of punkshell with plain tclsh 9.0.3; upstream ticket + draft written to `TEMP_REFERENCE/tcl9-dead-console-spin-TICKET-DRAFT.md` (filing is a + user decision, not gating this goal). diff --git a/punkproject.toml b/punkproject.toml index dfbf51e0..20445dae 100644 --- a/punkproject.toml +++ b/punkproject.toml @@ -1,4 +1,4 @@ [project] name = "punkshell" -version = "0.12.1" +version = "0.12.2" license = "BSD-2-Clause" diff --git a/src/modules/punk/AGENTS.md b/src/modules/punk/AGENTS.md index 6545eb33..315c0bf5 100644 --- a/src/modules/punk/AGENTS.md +++ b/src/modules/punk/AGENTS.md @@ -29,6 +29,7 @@ Source of truth for all modules under the `punk::*` namespace. This is the prima - New modules under `punk::*` should be created as `/-999999.0a1.0.tm` following the namespace-to-path convention. - punk::repl supports launch-time console selection (G-001): `repl::init -console ` (any `punk::console::console_spec_resolve` spec form) selects the console the repl reads/writes; `repl::start`'s input channel argument is optional and defaults to the selected console's input. Repl output flows through per-repl channel state (`repl::conin/conout/conerr` - rputs maps stdout/stderr per-repl, conerr==conout for a foreign console pending G-011), the code interp's stdout/stderr are diverted via shellfilter `var` junction stacks and emitted to the console after each run (repltype punk/0 only so far), and eof/size/capability questions go through `repl::console_at_eof`/`repl::console_get_size` which dispatch to the selected `::opunk::Console` object's (possibly overridden) methods. Process-console behaviours (tcl_interactive prompt gating, stdin reopen on eof, raw-mode re-enable) apply only when no foreign console is selected. Tests: `src/tests/modules/punk/repl/testsuites/repl/consolebackends.test` (child-process drivers - a repl cannot run inside the shared testinterp; see the suite header). +- punk::repl has a dead-console watchdog (G-039): `repl::start` arms `repl::console_watchdog` (default 5s, read-only `chan configure -inputmode` probe) only for a tcl9 console input channel (-inputmode present) serving the process-default console on windows - the Tcl 9 console driver never delivers a dead console (killed conhost/terminal) to the script as a fileevent and its reader thread busy-loops on the persistent error, so without the watchdog an orphaned shell spins CPU forever. On probe failure the watchdog closes the input channel (stopping the driver's reader thread) and finishes the repl via the normal eof path. Piped, foreign-console and tcl 8.6 inputs never arm it. Root-cause and verification detail: `goals/archive/G-039-orphan-console-spin.md` (achieved 2026-07-12); upstream ticket FILED 2026-07-12: https://core.tcl-lang.org/tcl/tktview/f10d91c2d3 (full text with repro scripts in `TEMP_REFERENCE/tcl9-dead-console-spin-TICKET-DRAFT.md` - scripts elided from the web submission). If a future Tcl release fixes the driver, the watchdog can become version-conditional. - punk::console uses the documented `-console` convention throughout: a `-console` value may be a 2-element {in out} channel list, an anchored `opunk::console` instance name, or an `::opunk::Console` object value (resolved via `punk::console::console_spec_resolve`). Query functions use the hybrid pattern (legacy trailing positional spec also accepted, parsed by `punk::console::internal::hybrid_console_spec`) - new query procs must follow it, with tests (see `src/tests/modules/punk/console/testsuites/console/queryprocs.test`). PUNKARGS definitions include the `-console` option via the shared fragments `::punk::console::argdoc::console_opts` (query/set functions) or `::punk::console::argdoc::console_emit_opts` (emit functions) rather than duplicating the option text; never re-add a `-minsize 2` constraint to `-console` (it rejects instance-name specs). The internal `get_size_using_*` size mechanisms deliberately remain canonical-pair positional (always fed by `get_size`). - punk::console emit-side functions (the `punk::console::ansi::*` emit wrappers, mouse/paste toggles, `vt52`, `set_tabstop_width`, `titleset`, top-level `move`, and the width-test probes) accept an optional trailing `-console ` pair, parsed manually for performance by `punk::console::internal::opt_console_out`/`opt_console_channels` (`_var` variants for procs whose args-tail also carries row/col/data triples). Each carries a documentation-only PUNKARGS definition that includes the shared `::punk::console::argdoc::console_emit_opts` fragment via `punk::args::resolved_def`; keep manual parsing and PUNKARGS synchronized. New emit procs must follow this pattern. Tests live in `src/tests/modules/punk/console/testsuites/console/emitconsole.test`. - punk::console terminal-property facts (is_vt52, tabwidth, cell_size, last_da1_result, grapheme_cluster_support, check::has_bug_*) are per-console: read/write them via `punk::console::console_fact_get`/`console_fact_set`, keyed by canonical {in out} channel pair. The store is tsv-backed (G-007) so all threads read the same values: the process-default console `{stdin stdout}` keeps the legacy namespace variables (`::punk::console::is_vt52`, `tabwidth`, ...) as its authoritative local storage with write traces mirroring into tsv `punk_console_facts` (so existing external readers and direct writers keep working); non-default consoles store facts only in tsv with an owner-qualified key. Do not bypass the helpers for non-default consoles; use `console_fact_clear` (not direct store manipulation) to reset facts in tests. `ansi_wanted`/`colour_disabled` (string-generation gates), `ansi_available` and raw-mode state are deliberately process-global (rationale documented at the fact store in the module). Tests live in `src/tests/modules/punk/console/testsuites/console/consolefacts.test`. diff --git a/src/modules/punk/repl-999999.0a1.0.tm b/src/modules/punk/repl-999999.0a1.0.tm index da1cf18a..2cdd0867 100644 --- a/src/modules/punk/repl-999999.0a1.0.tm +++ b/src/modules/punk/repl-999999.0a1.0.tm @@ -629,12 +629,36 @@ proc repl::start {args} { #catch { # set punk::console::tabwidth [punk::console::get_tabstop_apparent_width] #} + #dead-console watchdog (G-039): the Tcl 9 windows console driver never delivers a dead + #console (killed conhost/terminal) to the script level as a fileevent, and its reader + #thread busy-loops on the persistent error - an orphaned shell would spin CPU forever. + #Poll liveness on the process console so the repl can finish via the normal eof path. + #Armed only for a tcl9 console channel (-inputmode present) on the process-default + #console; piped/foreign/8.6 inputs are unaffected. + variable console_watchdog_afterids + variable console_watchdog_ms + set watchdog_chan "" + if {"windows" eq $::tcl_platform(platform) && [console_is_default] + && ![info exists console_watchdog_afterids($inchan)]} { + if {![catch {chan configure $inchan} wdconf] && [dict exists $wdconf -inputmode]} { + set watchdog_chan $inchan + set console_watchdog_afterids($inchan) [after $console_watchdog_ms [list [namespace current]::console_watchdog $inchan]] + } + } + vwait [namespace current]::done + if {$watchdog_chan ne "" && [info exists console_watchdog_afterids($watchdog_chan)]} { + after cancel $console_watchdog_afterids($watchdog_chan) + array unset console_watchdog_afterids $watchdog_chan + } #done can be set before the deferred reader registration above has run (e.g exit/quit message from codethread arriving first) #in which case the pending idle callback would re-register a readable handler on a repl that has finished # - it must be cancelled as well as clearing any active registration. after cancel $reader_registration_id - chan event $inchan readable {} + if {$inchan in [chan names]} { + #the console_watchdog closes a dead-console inchan before setting done + chan event $inchan readable {} + } #puts stderr "-->start done = $::repl::done" @@ -933,6 +957,63 @@ proc repl::console_at_eof {inputchan} { return [chan eof $inputchan] } +namespace eval repl { + #dead-console watchdog state (G-039) - see repl::console_watchdog + variable console_watchdog_ms 5000 + variable console_watchdog_afterids + array set console_watchdog_afterids {} +} +namespace eval repl::argdoc { + lappend PUNKARGS [list { + @id -id ::repl::console_watchdog + @cmd -name "repl::console_watchdog"\ + -summary\ + "Self-rescheduling liveness poll for the repl's process-console input channel."\ + -help\ + "Probes the repl input channel's console with 'chan configure -inputmode' + (a live GetConsoleMode call on windows) every repl::console_watchdog_ms + milliseconds. If the probe fails the hosting console is gone (e.g the + terminal or conhost was killed): the Tcl 9 windows console driver never + delivers that state to the script level as a fileevent (its ConsoleEventProc + only notifies on buffered data) and its reader thread busy-loops on the + persistent channel error, so an orphaned shell would otherwise spin CPU + indefinitely (G-039). On a failed probe the watchdog closes the input + channel (which lets the driver's reader thread exit, stopping the spin) + and finishes the repl via the normal eof done-path. + + Armed by repl::start only for a tcl9 console channel (-inputmode present + in the chan configure dict) serving the process-default console on + windows. Scheduling state is kept per channel name in + repl::console_watchdog_afterids; a watchdog whose channel has + disappeared disarms itself silently." + @values -min 1 -max 1 + inputchan -type string -help\ + "the repl input channel being watched" + }] +} +proc repl::console_watchdog {inputchan} { + variable console_watchdog_afterids + variable console_watchdog_ms + if {$inputchan ni [chan names]} { + array unset console_watchdog_afterids $inputchan + return + } + if {[catch {chan configure $inputchan -inputmode}]} { + #GetConsoleMode failed - the hosting console is gone. + #Close the channel (stops the tclWinConsole.c reader thread's error busy-loop) + #and finish the repl via the normal eof path. Do not attempt a console reopen: + #with the console dead, CONIN$ cannot be opened either (app-punkshell's eof + #handling makes the same discovery and exits cleanly). + array unset console_watchdog_afterids $inputchan + catch {chan event $inputchan readable {}} + catch {chan close $inputchan} + catch {puts stderr "|repl> console_watchdog: console unavailable for '$inputchan' - closing input and finishing repl (eof)"} + set ::repl::done [list eof $inputchan] + return + } + set console_watchdog_afterids($inputchan) [after $console_watchdog_ms [list [namespace current]::console_watchdog $inputchan]] +} + namespace eval repl::argdoc { lappend PUNKARGS [list { @id -id ::repl::console_get_size diff --git a/src/modules/punk/repl-buildversion.txt b/src/modules/punk/repl-buildversion.txt index e85782a8..43596ba0 100644 --- a/src/modules/punk/repl-buildversion.txt +++ b/src/modules/punk/repl-buildversion.txt @@ -1,6 +1,7 @@ -0.4.0 +0.5.0 #First line must be a semantic version number #all other lines are ignored. +#0.5.0 - G-039: new repl::console_watchdog - a self-rescheduling liveness poll (default 5s, repl::console_watchdog_ms) armed by repl::start for a tcl9 console input channel (-inputmode present) serving the process-default console on windows. The Tcl 9 windows console driver never delivers a dead console (killed conhost/terminal) to the script level as a fileevent (tclWinConsole.c ConsoleEventProc only notifies on buffered data) and its reader thread busy-loops on the persistent channel error, so an orphaned shell previously spun ~2 cores indefinitely. On a failed probe (chan configure -inputmode = live GetConsoleMode) the watchdog closes the input channel (stopping the driver reader thread) and finishes the repl via the normal eof done-path; app-punkshell's eof handling then finds no console reopenable and exits cleanly. repl::start's post-vwait reader-deregistration now tolerates an inchan closed by the watchdog. Piped/foreign-console/tcl8.6 inputs are unaffected (watchdog not armed). #0.4.0 - G-001: repl::init -console selects the console the repl reads/writes ({in out} pair, anchored opunk::console instance name, or ::opunk::Console object value, resolved via punk::console::console_spec_resolve). repl::start's inchan is now optional (defaults to the selected console's input). New repl-level channel state (conin/conout/conerr) is routed through rputs (stdout/stderr mapped per-repl) and doprompt; for a selected foreign console the code interp's stdout/stderr are diverted via shellfilter 'var' junction stacks and emitted to the console after each run. New helpers repl::console_is_default / console_at_eof / console_get_size - eof and size are answered by the selected console object's (possibly overridden) methods. Process-console behaviours (tcl_interactive prompt gating, stdin reopen on eof, raw-mode re-enable, utf-16be windows line re-decode experiment, mode-line on exit) now apply only to the default console. Default-console (stdin/stdout) behaviour unchanged. Also fixes rputs pseudo-channel mapping in the 3-arg -nonewline form (mapped value previously written over the -nonewline flag). #0.3.0 - G-058: codethread init script receives the runtime static/builtin package baseline (::punkboot::static_prefixes/static_packages via new %staticprefixes%/%staticpackages% scriptmap entries) and seeds ifneeded mappings in the codethread interp, so the code interp (seeded in turn via punk::lib::interp_sync_package_paths) can package-require statically-linked runtime packages - fixes 'can't find package Thread' booting a punk9win.vfs kit on the tclsfe-x64 static runtime (punk91) #0.2.2 - repl_handler line-mode waiting-chunks path no longer performs its opportunistic unsized read on tcl 8.6 windows console channels (no -inputmode key, real twapi console handle): the path is entered via 'after idle' with the channel usually drained, and on the 8.6 console driver a drained read parks a blocking cooked-mode ReadConsole that a later raw flip cannot cancel - after typed-ahead was stashed during a terminal-query raw window, that parked read swallowed every subsequent query response in the session until Enter (companion to the punk::console 0.7.1 three-site console-misdetection fix). On such consoles the path now consumes only data already in the Tcl channel buffer (chan pending input + sized read - no driver probe); with nothing buffered it processes the stashed complete lines directly and arms the readable handler for any remaining partial line (replacing an after-idle reinvoke that could never progress). Behaviour on tcl 9/8.7 (-inputmode consoles) and in raw mode is unchanged. diff --git a/src/vfs/_vfscommon.vfs/modules/punk/repl-0.4.0.tm b/src/vfs/_vfscommon.vfs/modules/punk/repl-0.5.0.tm similarity index 98% rename from src/vfs/_vfscommon.vfs/modules/punk/repl-0.4.0.tm rename to src/vfs/_vfscommon.vfs/modules/punk/repl-0.5.0.tm index 46156fed..61e028d1 100644 --- a/src/vfs/_vfscommon.vfs/modules/punk/repl-0.4.0.tm +++ b/src/vfs/_vfscommon.vfs/modules/punk/repl-0.5.0.tm @@ -629,12 +629,36 @@ proc repl::start {args} { #catch { # set punk::console::tabwidth [punk::console::get_tabstop_apparent_width] #} + #dead-console watchdog (G-039): the Tcl 9 windows console driver never delivers a dead + #console (killed conhost/terminal) to the script level as a fileevent, and its reader + #thread busy-loops on the persistent error - an orphaned shell would spin CPU forever. + #Poll liveness on the process console so the repl can finish via the normal eof path. + #Armed only for a tcl9 console channel (-inputmode present) on the process-default + #console; piped/foreign/8.6 inputs are unaffected. + variable console_watchdog_afterids + variable console_watchdog_ms + set watchdog_chan "" + if {"windows" eq $::tcl_platform(platform) && [console_is_default] + && ![info exists console_watchdog_afterids($inchan)]} { + if {![catch {chan configure $inchan} wdconf] && [dict exists $wdconf -inputmode]} { + set watchdog_chan $inchan + set console_watchdog_afterids($inchan) [after $console_watchdog_ms [list [namespace current]::console_watchdog $inchan]] + } + } + vwait [namespace current]::done + if {$watchdog_chan ne "" && [info exists console_watchdog_afterids($watchdog_chan)]} { + after cancel $console_watchdog_afterids($watchdog_chan) + array unset console_watchdog_afterids $watchdog_chan + } #done can be set before the deferred reader registration above has run (e.g exit/quit message from codethread arriving first) #in which case the pending idle callback would re-register a readable handler on a repl that has finished # - it must be cancelled as well as clearing any active registration. after cancel $reader_registration_id - chan event $inchan readable {} + if {$inchan in [chan names]} { + #the console_watchdog closes a dead-console inchan before setting done + chan event $inchan readable {} + } #puts stderr "-->start done = $::repl::done" @@ -933,6 +957,63 @@ proc repl::console_at_eof {inputchan} { return [chan eof $inputchan] } +namespace eval repl { + #dead-console watchdog state (G-039) - see repl::console_watchdog + variable console_watchdog_ms 5000 + variable console_watchdog_afterids + array set console_watchdog_afterids {} +} +namespace eval repl::argdoc { + lappend PUNKARGS [list { + @id -id ::repl::console_watchdog + @cmd -name "repl::console_watchdog"\ + -summary\ + "Self-rescheduling liveness poll for the repl's process-console input channel."\ + -help\ + "Probes the repl input channel's console with 'chan configure -inputmode' + (a live GetConsoleMode call on windows) every repl::console_watchdog_ms + milliseconds. If the probe fails the hosting console is gone (e.g the + terminal or conhost was killed): the Tcl 9 windows console driver never + delivers that state to the script level as a fileevent (its ConsoleEventProc + only notifies on buffered data) and its reader thread busy-loops on the + persistent channel error, so an orphaned shell would otherwise spin CPU + indefinitely (G-039). On a failed probe the watchdog closes the input + channel (which lets the driver's reader thread exit, stopping the spin) + and finishes the repl via the normal eof done-path. + + Armed by repl::start only for a tcl9 console channel (-inputmode present + in the chan configure dict) serving the process-default console on + windows. Scheduling state is kept per channel name in + repl::console_watchdog_afterids; a watchdog whose channel has + disappeared disarms itself silently." + @values -min 1 -max 1 + inputchan -type string -help\ + "the repl input channel being watched" + }] +} +proc repl::console_watchdog {inputchan} { + variable console_watchdog_afterids + variable console_watchdog_ms + if {$inputchan ni [chan names]} { + array unset console_watchdog_afterids $inputchan + return + } + if {[catch {chan configure $inputchan -inputmode}]} { + #GetConsoleMode failed - the hosting console is gone. + #Close the channel (stops the tclWinConsole.c reader thread's error busy-loop) + #and finish the repl via the normal eof path. Do not attempt a console reopen: + #with the console dead, CONIN$ cannot be opened either (app-punkshell's eof + #handling makes the same discovery and exits cleanly). + array unset console_watchdog_afterids $inputchan + catch {chan event $inputchan readable {}} + catch {chan close $inputchan} + catch {puts stderr "|repl> console_watchdog: console unavailable for '$inputchan' - closing input and finishing repl (eof)"} + set ::repl::done [list eof $inputchan] + return + } + set console_watchdog_afterids($inputchan) [after $console_watchdog_ms [list [namespace current]::console_watchdog $inputchan]] +} + namespace eval repl::argdoc { lappend PUNKARGS [list { @id -id ::repl::console_get_size @@ -4437,7 +4518,7 @@ namespace eval ::punk::args::register { package provide punk::repl [namespace eval punk::repl { variable version - set version 0.4.0 + set version 0.5.0 }] #repl::start $program_read_stdin_pipe