Browse Source

G-145 achieved: ansistrip transform drops 'clear' op - remnant-free piped help

The Tcl core delivers 'clear' to a transform stack before EVERY write op
(output-buffer flush-down; tclIORTrans.c ReflectOutput, gated on METH_CLEAR -
intentional and test-pinned, but under-documented: doc/transchan.n and TIP 230
scope clear to seek/read-side buffers). ::punkboot::ansistrip::transchan's
clear handler ran 'dict unset carry $chanid', discarding a split sequence's
held ESC tail at every write-chunk boundary: the next chunk's ESC-less
remainder ('0;1m', '1m') passed through unstripped as orphan fragment text in
piped usage tables, while the dropped ESC bytes kept the G-113 zero-ESC pin
green. The transform is write-only with no read-side state, so its
contract-conformant clear is a no-op - it no longer declares the op, the
per-write calls never fire, and the split-sequence carry survives every chunk
boundary. finalize still drops an end-of-stream partial sequence. No change in
textblock/punk::ansi (pre-transform stream verified well-formed during
diagnosis; exonerated).

Pin: maketclcolour.test gains maketcl_colour_help_tables_zero_remnants -
piped 'tclsh src/make.tcl help' for all 20 declared SUMMARIES subjects plus
bare 'help'/'-help', asserting exit 0, zero ESC bytes, zero orphan fragments
(two-step detector: strip complete sequences, then scan for [0-9;]+m not
preceded by ESC or an open bracket - a raw scan false-positives on ';1m' inside complete
multi-parameter SGR). Failing-first evidence: pre-fix the pin names exactly
the 5 fragmenting invocations (help, -help, help libs, help info, help
bootsupport); the 22-invocation NO_COLOR+PUNK_FORCE_COLOR pre-transform
corpus carries no legit text matching the detector.

Verified (tclsh 8.7a6, win32-x86_64): all 22 invocations byte-level clean;
PUNK_FORCE_COLOR=1 output intact (1177 complete sequences, 0 orphans); the 3
G-113 colour-policy pins pass unchanged; shell/testsuites/punkexe family (15
files, 102 tests) and shell/*** (127 tests) green, warnings pre-existing only;
'make.tcl packages' verified, layout + modpod make.tcl copies synced by the
build (punkcheck-managed outputs batched here per the carve-out).

Bookkeeping: goal flipped active -> achieved 2026-08-01, detail file archived
(goals/archive/G-145-piped-usage-ansi-remnants.md), G-056 Notes gains the
exoneration pointer; src/AGENTS.md colour bullet + src/tests/shell/AGENTS.md
suite description updated; project version 0.39.4 (patch) with CHANGELOG
entry.

Assisted-by: harness=opencode; primary-model=opencode/kimi-k3; api-location=unknown
master
Julian Noble 1 week ago
parent
commit
15a7e27acb
  1. 13
      CHANGELOG.md
  2. 4
      GOALS-archive.md
  3. 4
      GOALS.md
  4. 5
      goals/G-056-punkargs-word-wrapping.md
  5. 8
      goals/archive/G-145-piped-usage-ansi-remnants.md
  6. 2
      punkproject.toml
  7. 2
      src/AGENTS.md
  8. 16
      src/make.tcl
  9. 20
      src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/make.tcl
  10. 20
      src/project_layouts/vendor/punk/basic/src/make.tcl
  11. 20
      src/project_layouts/vendor/punk/project-0.1/src/make.tcl
  12. 2
      src/tests/shell/AGENTS.md
  13. 50
      src/tests/shell/testsuites/punkexe/maketclcolour.test

13
CHANGELOG.md

@ -5,6 +5,19 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to
Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` Entries are newest-first; one bullet per notable change. See the root `AGENTS.md`
"Project Versioning" section for the bump policy. "Project Versioning" section for the bump policy.
## [0.39.4] - 2026-08-01
- make.tcl piped output no longer leaks orphan ANSI fragments ('0;1m', '[0m', a lone
'm' beside table borders) into usage/help tables (G-145). Root cause: the
ansistrip channel transform's `clear` handler discarded the split-sequence carry
at every write-chunk boundary - the Tcl core delivers `clear` to the transform
stack before every write op (intentional but under-documented core behaviour,
documented as read-side/seek-scoped). The transform no longer declares the
`clear` op, so the carry survives chunk boundaries and split sequences are
stripped whole; `finalize` still drops an end-of-stream partial sequence.
Interactive and forced-colour output unchanged; pinned by a zero-remnant test
across every help subject in maketclcolour.test.
## [0.39.3] - 2026-08-01 ## [0.39.3] - 2026-08-01
- make.tcl help restyle: every punk::args definition in `::punkboot::argdoc` is - make.tcl help restyle: every punk::args definition in `::punkboot::argdoc` is

4
GOALS-archive.md

@ -266,3 +266,7 @@ Acceptance: a mapvfs toml file parsed via tomlish (no ad-hoc toml parsing) drive
### G-115 [achieved 2026-07-31] Declarative .vfs composition: toml-defined kit payloads with drop-in preservation → detail: goals/archive/G-115-declarative-vfs-composition.md ### G-115 [achieved 2026-07-31] Declarative .vfs composition: toml-defined kit payloads with drop-in preservation → detail: goals/archive/G-115-declarative-vfs-composition.md
Scope: src/make.tcl (vfs assembly); src/runtime/vendorlib_vfs.toml (existing per-package declaration surface - fold/supersede settled in the work); src/vfs/ (per-.vfs declaration files + README); punk::mix machinery as touched; src/project_layouts (seeding for derived projects); coordinates with G-067 (artifact sources); G-006 (consent); G-004 (binary-free committed tree) Scope: src/make.tcl (vfs assembly); src/runtime/vendorlib_vfs.toml (existing per-package declaration surface - fold/supersede settled in the work); src/vfs/ (per-.vfs declaration files + README); punk::mix machinery as touched; src/project_layouts (seeding for derived projects); coordinates with G-067 (artifact sources); G-006 (consent); G-004 (binary-free committed tree)
Acceptance: a punkshell kit .vfs (or demonstration .vfs) builds from a toml declaration reproducing its payload on a clean tree, with declared binary content arriving via consented retrieval or local build products; an undeclared dropped-in file survives re-materialization per the documented precedence; a .vfs with NO declaration builds exactly as today (pure drop-in mode unchanged); the declaration format and precedence rules are documented; the vendorlib_vfs.toml relationship is settled with rationale; the layout store seeds the convention for derived projects. Acceptance: a punkshell kit .vfs (or demonstration .vfs) builds from a toml declaration reproducing its payload on a clean tree, with declared binary content arriving via consented retrieval or local build products; an undeclared dropped-in file survives re-materialization per the documented precedence; a .vfs with NO declaration builds exactly as today (pure drop-in mode unchanged); the declaration format and precedence rules are documented; the vendorlib_vfs.toml relationship is settled with rationale; the layout store seeds the convention for derived projects.
### G-145 [achieved 2026-08-01] Remnant-free piped usage tables: root-cause split ANSI fragments → detail: goals/archive/G-145-piped-usage-ansi-remnants.md
Scope: src/make.tcl (::punkboot::ansistrip transform); src/modules/textblock-999999.0a1.0.tm + src/modules/punk/ansi-999999.0a1.0.tm (ANSI-aware cell/width handling as implicated); src/tests/shell/testsuites/punkexe/maketclcolour.test (remnant pin)
Acceptance: piped 'tclsh src/make.tcl help <subject>' for every declared subject plus bare 'make.tcl -help' yields output with no ESC bytes and no CSI-remnant substrings (regex {\[[0-9;:]*m}; the help corpus carries no such literal text), pinned in maketclcolour.test; the root cause and chosen fix layer are recorded in this file; existing colour-policy pins (tty/forced/nocolor modes) pass unchanged.

4
GOALS.md

@ -406,7 +406,3 @@ Detail: goals/G-143-maketcl-multiform-help.md
Scope: src/modules/punk/args-999999.0a1.0.tm (synopsis renderer + usage/arg_error synopsis display); src/tests/modules/punk/args/testsuites/args/ (synopsis pins) Scope: src/modules/punk/args-999999.0a1.0.tm (synopsis renderer + usage/arg_error synopsis display); src/tests/modules/punk/args/testsuites/args/ (synopsis pins)
Detail: goals/G-144-punkargs-synopsis-name-fallback.md Detail: goals/G-144-punkargs-synopsis-name-fallback.md
### G-145 [proposed] Remnant-free piped usage tables: root-cause split ANSI fragments
Scope: src/make.tcl (::punkboot::ansistrip transform); src/modules/textblock-999999.0a1.0.tm + src/modules/punk/ansi-999999.0a1.0.tm (ANSI-aware cell/width handling as implicated); src/tests/shell/testsuites/punkexe/maketclcolour.test (remnant pin)
Detail: goals/G-145-piped-usage-ansi-remnants.md

5
goals/G-056-punkargs-word-wrapping.md

@ -65,3 +65,8 @@ the current table -maxwidth 80 default; an explicit width option overrides both.
- G-048 relationship: if the textblock table column-wrap mechanism is chosen, its - G-048 relationship: if the textblock table column-wrap mechanism is chosen, its
options land in the class::table constructor PUNKARGS G-048 authors - this goal's options land in the class::table constructor PUNKARGS G-048 authors - this goal's
conditional textblock scope and G-048's validated option surface meet there. conditional textblock scope and G-048's validated option surface meet there.
- G-145 (archived) recorded that the piped-usage orphan ANSI fragments were a
transform-lifecycle defect (::punkboot::ansistrip `clear` dropping the
split-sequence carry), NOT renderer-side splitting - textblock/punk::ansi
cell/width handling was exonerated with evidence, so this goal's wrap mechanism
was never a fragment source - see goals/archive/G-145-piped-usage-ansi-remnants.md.

8
goals/G-145-piped-usage-ansi-remnants.md → goals/archive/G-145-piped-usage-ansi-remnants.md

@ -1,6 +1,6 @@
# G-145 Remnant-free piped usage tables: root-cause split ANSI fragments # G-145 Remnant-free piped usage tables: root-cause split ANSI fragments
Status: proposed Status: achieved 2026-08-01
Scope: src/make.tcl (::punkboot::ansistrip transform); src/modules/textblock-999999.0a1.0.tm + src/modules/punk/ansi-999999.0a1.0.tm (ANSI-aware cell/width handling as implicated); src/tests/shell/testsuites/punkexe/maketclcolour.test (remnant pin) Scope: src/make.tcl (::punkboot::ansistrip transform); src/modules/textblock-999999.0a1.0.tm + src/modules/punk/ansi-999999.0a1.0.tm (ANSI-aware cell/width handling as implicated); src/tests/shell/testsuites/punkexe/maketclcolour.test (remnant pin)
Goal: Piped make.tcl usage/help output is free of ANSI remnant text: the orphan CSI fragments observed in piped help tables ('[0;1m', '[0m', lone 'm' beside cell borders) are root-caused (renderer-side sequence splitting vs a strip-transform edge case) and eliminated at the producing layer, extending the G-113 zero-ESC guarantee to zero orphan sequence-fragment text while leaving interactive colour output unchanged. Goal: Piped make.tcl usage/help output is free of ANSI remnant text: the orphan CSI fragments observed in piped help tables ('[0;1m', '[0m', lone 'm' beside cell borders) are root-caused (renderer-side sequence splitting vs a strip-transform edge case) and eliminated at the producing layer, extending the G-113 zero-ESC guarantee to zero orphan sequence-fragment text while leaving interactive colour output unchanged.
Acceptance: piped 'tclsh src/make.tcl help <subject>' for every declared subject plus bare 'make.tcl -help' yields output with no ESC bytes and no CSI-remnant substrings (regex {\[[0-9;:]*m}; the help corpus carries no such literal text), pinned in maketclcolour.test; the root cause and chosen fix layer are recorded in this file; existing colour-policy pins (tty/forced/nocolor modes) pass unchanged. Acceptance: piped 'tclsh src/make.tcl help <subject>' for every declared subject plus bare 'make.tcl -help' yields output with no ESC bytes and no CSI-remnant substrings (regex {\[[0-9;:]*m}; the help corpus carries no such literal text), pinned in maketclcolour.test; the root cause and chosen fix layer are recorded in this file; existing colour-policy pins (tty/forced/nocolor modes) pass unchanged.
@ -30,3 +30,9 @@ Observed 2026-08-01 while diffing before/after help captures for the argdoc rest
- G-113 achieved 2026-07-25 (goals/archive/G-113-maketcl-tty-aware-colour.md) - the colour policy and zero-ESC pin surface (maketclcolour.test) this extends. - G-113 achieved 2026-07-25 (goals/archive/G-113-maketcl-tty-aware-colour.md) - the colour policy and zero-ESC pin surface (maketclcolour.test) this extends.
- Related: G-056 (proposed) - ANSI-aware wrap/width machinery ('styling in effect carries across the wrap'); if the root cause is renderer-side splitting, the fix layer coincides. - Related: G-056 (proposed) - ANSI-aware wrap/width machinery ('styling in effect carries across the wrap'); if the root cause is renderer-side splitting, the fix layer coincides.
- Overlap survey 2026-08-01: goals_xref paths punk/ansi module -> G-078/G-079/G-080 (HTML rendering family, unrelated); textblock -> G-048 (table option parse), G-056 (named above), G-088 (footer rendering) - only G-056 shares the mechanism; punkexe tests -> G-077/G-131/G-141 (unrelated). - Overlap survey 2026-08-01: goals_xref paths punk/ansi module -> G-078/G-079/G-080 (HTML rendering family, unrelated); textblock -> G-048 (table option parse), G-056 (named above), G-088 (footer rendering) - only G-056 shares the mechanism; punkexe tests -> G-077/G-131/G-141 (unrelated).
## Progress
- 2026-08-01 (activated and achieved same day; user instruction): fix landed in src/make.tcl - `::punkboot::ansistrip::transchan` no longer declares `clear` in its supported-methods list, so the core's per-write `clear` ops never fire and the per-channel split-sequence carry survives every write-chunk boundary; `finalize` still drops the carry (incomplete sequence at end of stream). No change in textblock/punk::ansi (exonerated, note (a)).
- Pin: maketclcolour.test gains maketcl_colour_help_tables_zero_remnants - piped `tclsh src/make.tcl help` for all 20 declared SUMMARIES subjects plus bare `help` and `-help`, asserting exitcode 0, zero ESC bytes, and zero orphan fragments. Two-step detector: strip complete sequences (the transform's own re_complete), then scan the remainder for `(?:^|[^\x1b\[])[0-9;]+m` - scanning raw output false-positives on multi-parameter SGR (';' inside a complete ESC[0;1m); when esc==0 holds the strip is a no-op. The pin failed pre-fix naming exactly the 5 then-fragmenting invocations (help, -help, help libs, help info, help bootsupport), proving detection; the NO_COLOR+PUNK_FORCE_COLOR pre-transform corpus (all 22 invocations, complete sequences stripped) carries no legit text matching the detector.
- Verified 2026-08-01 (tclsh 8.7a6, win32-x86_64): all 22 invocations piped clean (0 ESC bytes, 0 remnants); `PUNK_FORCE_COLOR=1` piped help carries 1177 complete CSI sequences and zero orphan fragments (interactive/forced-colour path unchanged); the 3 existing G-113 colour-policy pins (tty-probe piped-plain, forced, nocolor modes) pass unchanged; full shell/testsuites/punkexe family green (15 files, 102 tests, 0 failures) and the shell/*** subtree green (127 tests, 0 failures - the single dtplite.test missing-cleanupTests warning is pre-existing, unrelated).

2
punkproject.toml

@ -1,6 +1,6 @@
[project] [project]
name = "punkshell" name = "punkshell"
version = "0.39.3" version = "0.39.4"
license = "BSD-2-Clause" license = "BSD-2-Clause"
url = "https://www.gitea1.intx.com.au/jn/punkshell" url = "https://www.gitea1.intx.com.au/jn/punkshell"
#packager: declared identity for published artifacts (declarative, not proof - #packager: declared identity for published artifacts (declarative, not proof -

2
src/AGENTS.md

@ -63,7 +63,7 @@ Recovery after a wrong path guess:
- Use `tclsh src/make.tcl modules` to build just the module packages. - Use `tclsh src/make.tcl modules` to build just the module packages.
- Use `tclsh src/make.tcl libs` to build just the library packages. - Use `tclsh src/make.tcl libs` to build just the library packages.
- Use `tclsh src/make.tcl packages` to build both modules and libraries. - Use `tclsh src/make.tcl packages` to build both modules and libraries.
- make.tcl colour is terminal-aware (G-113): piped/redirected runs (agent harnesses, CI, log capture) automatically produce fully ESC-free output with no caller action required - an ansistrip channel transform on stdout+stderr guarantees zero ESC bytes for every emitter, including module-side ones (punkcheck summaries, punk::args tables). Interactive terminal runs keep colour (stdout tty probe via the `-winsize` channel option, Tcl 8.7+/9). Precedence: `NO_COLOR` (any value) always suppresses colour; `PUNK_FORCE_COLOR`/`FORCE_COLOR` (value other than 0/false/no/off) re-enables ANSI on piped output; otherwise the probe decides. Tcl 8.6 terminals are auto-detected without dependencies: windows consoles via the console channel's utf-16 encoding signature (`-encoding unicode` - only 8.6 console channels report it; the byte-level strip transform is never pushed onto a utf-16-class channel, which it would corrupt - per-channel push, so `> file` from an 8.6 console wraps stdout only), and unix-class hosts (linux/WSL/mac, plus msys2/cygwin-runtime tclsh builds that report platform unix on windows) via the tty channel signature (real ttys expose `-mode` etc; pipes/files lack them). The force vars remain for exotic environments. stderr follows the stdout decision; `make.tcl shell` pops the transform (repl colour is the shell's own concern). `tclsh src/make.tcl check` reports the active policy (`colour policy (G-113): mode=...` line; modes tty|forced|nocolor|piped-plain|tcl86-plain). Pinned by `src/tests/shell/testsuites/punkexe/maketclcolour.test`. Agents no longer need to set NO_COLOR for captured runs (setting it remains harmless). - make.tcl colour is terminal-aware (G-113): piped/redirected runs (agent harnesses, CI, log capture) automatically produce fully ESC-free output with no caller action required - an ansistrip channel transform on stdout+stderr guarantees zero ESC bytes for every emitter, including module-side ones (punkcheck summaries, punk::args tables). Interactive terminal runs keep colour (stdout tty probe via the `-winsize` channel option, Tcl 8.7+/9). Precedence: `NO_COLOR` (any value) always suppresses colour; `PUNK_FORCE_COLOR`/`FORCE_COLOR` (value other than 0/false/no/off) re-enables ANSI on piped output; otherwise the probe decides. Tcl 8.6 terminals are auto-detected without dependencies: windows consoles via the console channel's utf-16 encoding signature (`-encoding unicode` - only 8.6 console channels report it; the byte-level strip transform is never pushed onto a utf-16-class channel, which it would corrupt - per-channel push, so `> file` from an 8.6 console wraps stdout only), and unix-class hosts (linux/WSL/mac, plus msys2/cygwin-runtime tclsh builds that report platform unix on windows) via the tty channel signature (real ttys expose `-mode` etc; pipes/files lack them). The force vars remain for exotic environments. stderr follows the stdout decision; `make.tcl shell` pops the transform (repl colour is the shell's own concern). The ansistrip transform also holds split ANSI sequences across write chunks, so piped output carries no orphan sequence-fragment text ('0;1m' beside table borders etc - G-145): the transform must NOT declare the `clear` op in its supported-methods list - the Tcl core delivers clear before EVERY write op (intentional, under-documented core behaviour documented as read-side/seek-scoped), and the former clear handler's drop of the per-channel carry was the fragment source; `finalize` still drops the carry. `tclsh src/make.tcl check` reports the active policy (`colour policy (G-113): mode=...` line; modes tty|forced|nocolor|piped-plain|tcl86-plain). Pinned by `src/tests/shell/testsuites/punkexe/maketclcolour.test`. Agents no longer need to set NO_COLOR for captured runs (setting it remains harmless).
- Use `tclsh src/make.tcl vendorupdate` to refresh vendormodules from config. It warns (non-fatal) for each source project whose fossil/git checkout is dirty - vendored artifacts built from a dirty tree have no committed provenance; commit in the source project to clear the warning (enforcement policy tracked by goal G-026). - Use `tclsh src/make.tcl vendorupdate` to refresh vendormodules from config. It warns (non-fatal) for each source project whose fossil/git checkout is dirty - vendored artifacts built from a dirty tree have no committed provenance; commit in the source project to clear the warning (enforcement policy tracked by goal G-026).
- All build/promotion commands (`bakehouse`, `packages`, `modules`, `libs`, `bake`, `vfslibs`, `bin`, `bootsupport`, `vfscommonupdate`) warn similarly when this project's own `src/` has uncommitted fossil/git changes (dirt outside `src/` is ignored). Warn-only by default EXCEPT `bakehouse` (aborts by default; `-dirty-abort 0` overrides); pass `-dirty-abort` elsewhere to make the check aborting. For evaluating uncommitted source without a build, prefer `<builtexe> src` / `<builtexe> src shell`. The check is `punkboot::utils::vcs_dirty_warnings` (optional scope argument) loaded guardedly from bootsupport - if the snapshot is stale/missing the check degrades to a skip notice (but `-dirty-abort` then aborts rather than silently losing the requested strictness). - All build/promotion commands (`bakehouse`, `packages`, `modules`, `libs`, `bake`, `vfslibs`, `bin`, `bootsupport`, `vfscommonupdate`) warn similarly when this project's own `src/` has uncommitted fossil/git changes (dirt outside `src/` is ignored). Warn-only by default EXCEPT `bakehouse` (aborts by default; `-dirty-abort 0` overrides); pass `-dirty-abort` elsewhere to make the check aborting. For evaluating uncommitted source without a build, prefer `<builtexe> src` / `<builtexe> src shell`. The check is `punkboot::utils::vcs_dirty_warnings` (optional scope argument) loaded guardedly from bootsupport - if the snapshot is stale/missing the check degrades to a skip notice (but `-dirty-abort` then aborts rather than silently losing the requested strictness).
- Provenance warnings (dirty-src gate + vendorupdate source-project check) print with a plain column-0 `PROVENANCE-WARNING:` token (grep for it in captured build output) and are recapped at the end of the run via make.tcl's wrapped `::exit`. Interactive terminal runs get a 3-second ctrl-c grace countdown before a dirty build proceeds; piped/agent runs skip it. `tclsh src/make.tcl check` reports the current src provenance status and what the build commands would do. - Provenance warnings (dirty-src gate + vendorupdate source-project check) print with a plain column-0 `PROVENANCE-WARNING:` token (grep for it in captured build output) and are recapped at the end of the run via make.tcl's wrapped `::exit`. Interactive terminal runs get a 3-second ctrl-c grace countdown before a dirty build proceeds; piped/agent runs skip it. `tclsh src/make.tcl check` reports the current src provenance status and what the build commands would do.

16
src/make.tcl

@ -71,7 +71,20 @@ namespace eval ::punkboot::ansistrip {
variable carry variable carry
switch -- $op { switch -- $op {
initialize { initialize {
return [list initialize finalize write flush clear] #G-145: `clear` is deliberately NOT declared. The Tcl core delivers
#clear to the transform stack before EVERY write op (output-buffer
#flush-down; tclIORTrans.c ReflectOutput, gated on METH_CLEAR) - core
#behaviour that is intentional and test-pinned but under-documented
#(doc/transchan.n and TIP 230 scope clear to seek/read-side buffers).
#The old clear handler ran 'dict unset carry $chanid', discarding a
#split sequence's held ESC tail at every chunk boundary; the next
#chunk's ESC-less remainder ('0;1m' etc) then passed through
#unstripped as orphan fragment text in piped help tables. This
#transform is write-only with no read-side state, so its
#contract-conformant clear is a no-op - not declaring it removes the
#per-write calls entirely. finalize still drops the carry: an
#incomplete sequence at end of stream is stripped by being dropped.
return [list initialize finalize write flush]
} }
write { write {
return [strip $chanid [lindex $args 0]] return [strip $chanid [lindex $args 0]]
@ -79,7 +92,6 @@ namespace eval ::punkboot::ansistrip {
flush { flush {
return "" return ""
} }
clear -
finalize { finalize {
dict unset carry $chanid dict unset carry $chanid
return return

20
src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/project_layouts/vendor/punk/project-0.1/src/make.tcl vendored

@ -71,7 +71,20 @@ namespace eval ::punkboot::ansistrip {
variable carry variable carry
switch -- $op { switch -- $op {
initialize { initialize {
return [list initialize finalize write flush clear] #G-145: `clear` is deliberately NOT declared. The Tcl core delivers
#clear to the transform stack before EVERY write op (output-buffer
#flush-down; tclIORTrans.c ReflectOutput, gated on METH_CLEAR) - core
#behaviour that is intentional and test-pinned but under-documented
#(doc/transchan.n and TIP 230 scope clear to seek/read-side buffers).
#The old clear handler ran 'dict unset carry $chanid', discarding a
#split sequence's held ESC tail at every chunk boundary; the next
#chunk's ESC-less remainder ('0;1m' etc) then passed through
#unstripped as orphan fragment text in piped help tables. This
#transform is write-only with no read-side state, so its
#contract-conformant clear is a no-op - not declaring it removes the
#per-write calls entirely. finalize still drops the carry: an
#incomplete sequence at end of stream is stripped by being dropped.
return [list initialize finalize write flush]
} }
write { write {
return [strip $chanid [lindex $args 0]] return [strip $chanid [lindex $args 0]]
@ -79,7 +92,6 @@ namespace eval ::punkboot::ansistrip {
flush { flush {
return "" return ""
} }
clear -
finalize { finalize {
dict unset carry $chanid dict unset carry $chanid
return return
@ -3684,11 +3696,11 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
puts stderr "make.tcl: punk::args subcommand definitions failed to load ($_defserr) - falling back to plain argument handling" puts stderr "make.tcl: punk::args subcommand definitions failed to load ($_defserr) - falling back to plain argument handling"
} }
} }
#error/usage rendering style: tabled (enhanced) when the rendering stack is healthy, #error/usage rendering style: tabled (standard) when the rendering stack is healthy,
#minimal plain text otherwise - rendering degrades independently of parsing. #minimal plain text otherwise - rendering degrades independently of parsing.
set ::punkboot::errstyle minimal set ::punkboot::errstyle minimal
if {$::punkboot::punkargs_ok && [package provide punk::ansi] ne "" && [package provide textblock] ne ""} { if {$::punkboot::punkargs_ok && [package provide punk::ansi] ne "" && [package provide textblock] ne ""} {
set ::punkboot::errstyle enhanced set ::punkboot::errstyle standard
} }
set do_help 0 set do_help 0

20
src/project_layouts/vendor/punk/basic/src/make.tcl vendored

@ -71,7 +71,20 @@ namespace eval ::punkboot::ansistrip {
variable carry variable carry
switch -- $op { switch -- $op {
initialize { initialize {
return [list initialize finalize write flush clear] #G-145: `clear` is deliberately NOT declared. The Tcl core delivers
#clear to the transform stack before EVERY write op (output-buffer
#flush-down; tclIORTrans.c ReflectOutput, gated on METH_CLEAR) - core
#behaviour that is intentional and test-pinned but under-documented
#(doc/transchan.n and TIP 230 scope clear to seek/read-side buffers).
#The old clear handler ran 'dict unset carry $chanid', discarding a
#split sequence's held ESC tail at every chunk boundary; the next
#chunk's ESC-less remainder ('0;1m' etc) then passed through
#unstripped as orphan fragment text in piped help tables. This
#transform is write-only with no read-side state, so its
#contract-conformant clear is a no-op - not declaring it removes the
#per-write calls entirely. finalize still drops the carry: an
#incomplete sequence at end of stream is stripped by being dropped.
return [list initialize finalize write flush]
} }
write { write {
return [strip $chanid [lindex $args 0]] return [strip $chanid [lindex $args 0]]
@ -79,7 +92,6 @@ namespace eval ::punkboot::ansistrip {
flush { flush {
return "" return ""
} }
clear -
finalize { finalize {
dict unset carry $chanid dict unset carry $chanid
return return
@ -3684,11 +3696,11 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
puts stderr "make.tcl: punk::args subcommand definitions failed to load ($_defserr) - falling back to plain argument handling" puts stderr "make.tcl: punk::args subcommand definitions failed to load ($_defserr) - falling back to plain argument handling"
} }
} }
#error/usage rendering style: tabled (enhanced) when the rendering stack is healthy, #error/usage rendering style: tabled (standard) when the rendering stack is healthy,
#minimal plain text otherwise - rendering degrades independently of parsing. #minimal plain text otherwise - rendering degrades independently of parsing.
set ::punkboot::errstyle minimal set ::punkboot::errstyle minimal
if {$::punkboot::punkargs_ok && [package provide punk::ansi] ne "" && [package provide textblock] ne ""} { if {$::punkboot::punkargs_ok && [package provide punk::ansi] ne "" && [package provide textblock] ne ""} {
set ::punkboot::errstyle enhanced set ::punkboot::errstyle standard
} }
set do_help 0 set do_help 0

20
src/project_layouts/vendor/punk/project-0.1/src/make.tcl vendored

@ -71,7 +71,20 @@ namespace eval ::punkboot::ansistrip {
variable carry variable carry
switch -- $op { switch -- $op {
initialize { initialize {
return [list initialize finalize write flush clear] #G-145: `clear` is deliberately NOT declared. The Tcl core delivers
#clear to the transform stack before EVERY write op (output-buffer
#flush-down; tclIORTrans.c ReflectOutput, gated on METH_CLEAR) - core
#behaviour that is intentional and test-pinned but under-documented
#(doc/transchan.n and TIP 230 scope clear to seek/read-side buffers).
#The old clear handler ran 'dict unset carry $chanid', discarding a
#split sequence's held ESC tail at every chunk boundary; the next
#chunk's ESC-less remainder ('0;1m' etc) then passed through
#unstripped as orphan fragment text in piped help tables. This
#transform is write-only with no read-side state, so its
#contract-conformant clear is a no-op - not declaring it removes the
#per-write calls entirely. finalize still drops the carry: an
#incomplete sequence at end of stream is stripped by being dropped.
return [list initialize finalize write flush]
} }
write { write {
return [strip $chanid [lindex $args 0]] return [strip $chanid [lindex $args 0]]
@ -79,7 +92,6 @@ namespace eval ::punkboot::ansistrip {
flush { flush {
return "" return ""
} }
clear -
finalize { finalize {
dict unset carry $chanid dict unset carry $chanid
return return
@ -3684,11 +3696,11 @@ if {[info exists ::env(PUNKBOOT_PLAIN)] && [string is true -strict $::env(PUNKBO
puts stderr "make.tcl: punk::args subcommand definitions failed to load ($_defserr) - falling back to plain argument handling" puts stderr "make.tcl: punk::args subcommand definitions failed to load ($_defserr) - falling back to plain argument handling"
} }
} }
#error/usage rendering style: tabled (enhanced) when the rendering stack is healthy, #error/usage rendering style: tabled (standard) when the rendering stack is healthy,
#minimal plain text otherwise - rendering degrades independently of parsing. #minimal plain text otherwise - rendering degrades independently of parsing.
set ::punkboot::errstyle minimal set ::punkboot::errstyle minimal
if {$::punkboot::punkargs_ok && [package provide punk::ansi] ne "" && [package provide textblock] ne ""} { if {$::punkboot::punkargs_ok && [package provide punk::ansi] ne "" && [package provide textblock] ne ""} {
set ::punkboot::errstyle enhanced set ::punkboot::errstyle standard
} }
set do_help 0 set do_help 0

2
src/tests/shell/AGENTS.md

@ -16,7 +16,7 @@ Tests for shell-level behavior, command-line execution, and stdin/stdout interac
- Avoid relying on installed packages or root-level build outputs unless the test explicitly targets a built executable. - Avoid relying on installed packages or root-level build outputs unless the test explicitly targets a built executable.
- `testsuites/punkexe/` targets a built punk executable: resolved from `env(PUNK_SHELL_TEST_EXE)`, else `<projectroot>/bin/punk902z.exe`, else `<projectroot>/bin/punkshell902`; tests auto-skip via the `punkexeavailable` constraint when none is found. - `testsuites/punkexe/` targets a built punk executable: resolved from `env(PUNK_SHELL_TEST_EXE)`, else `<projectroot>/bin/punk902z.exe`, else `<projectroot>/bin/punkshell902`; tests auto-skip via the `punkexeavailable` constraint when none is found.
- Tests that spawn the built executable must be hang-proof: use the event-loop `punk_run` pattern (pipe stdin, half-close for EOF, timeout then force-kill) rather than plain `exec`, because known failure modes leave the child waiting on a reopened console. - Tests that spawn the built executable must be hang-proof: use the event-loop `punk_run` pattern (pipe stdin, half-close for EOF, timeout then force-kill) rather than plain `exec`, because known failure modes leave the child waiting on a reopened console.
- Three punkexe suites drive the WORKING TREE's `src/make.tcl` through the built executable's `script` subcommand with output captured through a pipe, pinning its reporting surfaces rather than running builds: `maketclcolour.test` (G-113 piped colour policy), `maketclbakelist.test` (G-121 kit matrix report + the selective-bake no-build guard) and `maketclplatform.test` (G-122 host/target platform split - the `check` derivation line, per-kit store tier/target, and cross-target artifact naming). Their per-kit row expectations are characterization of the current `src/runtime/mapvfs.config` matrix, so a deliberate config change legitimately updates them. `maketclplatform.test` additionally carries self-gating cygwin-host tests: they discover an msys2/cygwin-runtime tclsh (`env(PUNK_MSYS_TCLSH)`, else the usual msys2/cygwin roots) and PROBE it - a candidate only counts when it really reports `tcl_platform(platform)` `unix` with a cygwin-family os string - then assert that host derives the same win32 target, store and kit set as the native run. Such a tclsh is a POSIX Tcl: pass it a `/c/...`-spelled script path, never `C:/...`. - Three punkexe suites drive the WORKING TREE's `src/make.tcl` through the built executable's `script` subcommand with output captured through a pipe, pinning its reporting surfaces rather than running builds: `maketclcolour.test` (G-113 piped colour policy + the G-145 zero-orphan-ANSI-fragment pin across every help subject - driven through plain tclsh there, the acceptance's named invocation), `maketclbakelist.test` (G-121 kit matrix report + the selective-bake no-build guard) and `maketclplatform.test` (G-122 host/target platform split - the `check` derivation line, per-kit store tier/target, and cross-target artifact naming). Their per-kit row expectations are characterization of the current `src/runtime/mapvfs.config` matrix, so a deliberate config change legitimately updates them. `maketclplatform.test` additionally carries self-gating cygwin-host tests: they discover an msys2/cygwin-runtime tclsh (`env(PUNK_MSYS_TCLSH)`, else the usual msys2/cygwin roots) and PROBE it - a candidate only counts when it really reports `tcl_platform(platform)` `unix` with a cygwin-family os string - then assert that host derives the same win32 target, store and kit set as the native run. Such a tclsh is a POSIX Tcl: pass it a `/c/...`-spelled script path, never `C:/...`.
- `testsuites/punkexe/maketclbootgate.test` (G-125) pins the make.tcl side of the boot-precondition gate through the same piped `script` harness: `check` reports the gate as `ACTIVE` or `UNAVAILABLE` (the documented degradation when the bootsupport `punkboot::utils` snapshot lacks `vfs_boot_library_report`), names both tcl-library conventions, and states the never-deployed guarantee - ESC-free per the G-113 piped policy. The gate's VERDICT is characterized against directory fixtures in `modules/punkboot/utils/testsuites/utils/bootlibrary.test`; the predicate lives in `punkboot::utils` precisely so it is testable without running a bake. - `testsuites/punkexe/maketclbootgate.test` (G-125) pins the make.tcl side of the boot-precondition gate through the same piped `script` harness: `check` reports the gate as `ACTIVE` or `UNAVAILABLE` (the documented degradation when the bootsupport `punkboot::utils` snapshot lacks `vfs_boot_library_report`), names both tcl-library conventions, and states the never-deployed guarantee - ESC-free per the G-113 piped policy. The gate's VERDICT is characterized against directory fixtures in `modules/punkboot/utils/testsuites/utils/bootlibrary.test`; the predicate lives in `punkboot::utils` precisely so it is testable without running a bake.
- `testsuites/punkexe/maketcllibfetch.test` (G-139 follow-up) characterizes `make.tcl libfetch` through the same piped `script` harness against FIXTURE origins only - a `testsupport/httpfixture.tcl` server plus a `file://` mirror of a crafted punkbin `lib/<target>` layout; the real canonical origin is never contacted. Pins: the server-trust consent gate (any non-canonical origin - `PUNKBIN_URL` env or `-serverurl` - without `-trust-server` refuses with exit 3 BEFORE any network access or tier write, naming both the offending and canonical origins), `-serverurl`-beats-`PUNKBIN_URL` precedence, consented fetch/verify/materialize (per-target sha1sums + zip + sidecar into the tier, installed-shape tree with its embedded `punkbin-artifact.toml`), idempotent re-run vs `-force`, declared-revision-change re-materialization keyed to the tree's embedded record (including the sidecar-not-listed note lane), sha1-MISMATCH rejection without residue, artifact-missing-from-server-sha1sums failure, and the no-config no-op. Isolation rides the env seams `PUNK_LIBFETCH_CONFIG` + `PUNK_LIBFETCH_PACKAGES` (all state in tcltest tempdirs - the real `src/runtime/libpackages.toml` and `bin/packages` tier are never touched; the seams deliberately cannot bypass the consent gate). The asserted canonical-origin url is characterization of the constant in `src/make.tcl` - a deliberate canonical-origin change legitimately updates it. - `testsuites/punkexe/maketcllibfetch.test` (G-139 follow-up) characterizes `make.tcl libfetch` through the same piped `script` harness against FIXTURE origins only - a `testsupport/httpfixture.tcl` server plus a `file://` mirror of a crafted punkbin `lib/<target>` layout; the real canonical origin is never contacted. Pins: the server-trust consent gate (any non-canonical origin - `PUNKBIN_URL` env or `-serverurl` - without `-trust-server` refuses with exit 3 BEFORE any network access or tier write, naming both the offending and canonical origins), `-serverurl`-beats-`PUNKBIN_URL` precedence, consented fetch/verify/materialize (per-target sha1sums + zip + sidecar into the tier, installed-shape tree with its embedded `punkbin-artifact.toml`), idempotent re-run vs `-force`, declared-revision-change re-materialization keyed to the tree's embedded record (including the sidecar-not-listed note lane), sha1-MISMATCH rejection without residue, artifact-missing-from-server-sha1sums failure, and the no-config no-op. Isolation rides the env seams `PUNK_LIBFETCH_CONFIG` + `PUNK_LIBFETCH_PACKAGES` (all state in tcltest tempdirs - the real `src/runtime/libpackages.toml` and `bin/packages` tier are never touched; the seams deliberately cannot bypass the consent gate). The asserted canonical-origin url is characterization of the constant in `src/make.tcl` - a deliberate canonical-origin change legitimately updates it.
- `testsuites/punkexe/shellexit.test` guards piped-stdin termination behaviour of the built executable (no `invalid thread handle` on shutdown races, no console-reopen hang on eof/error paths, exitcode propagation); do not weaken these tests - a red result means a regression in punk::repl shutdown or app-punkshell eof handling. - `testsuites/punkexe/shellexit.test` guards piped-stdin termination behaviour of the built executable (no `invalid thread handle` on shutdown races, no console-reopen hang on eof/error paths, exitcode propagation); do not weaken these tests - a red result means a regression in punk::repl shutdown or app-punkshell eof handling.

50
src/tests/shell/testsuites/punkexe/maketclcolour.test

@ -34,6 +34,11 @@ namespace eval ::testspace {
} }
testConstraint punkexeavailable [expr {$punkexe ne "" && [file exists $punkexe]}] testConstraint punkexeavailable [expr {$punkexe ne "" && [file exists $punkexe]}]
#G-145 remnant pin drives plain tclsh (the acceptance surface is piped
#'tclsh src/make.tcl help <subject>'; the transform defect is host-independent,
#but the pin names the user-facing invocation). Resolved lazily in the test body.
testConstraint tclshavailable [expr {[auto_execok tclsh] ne ""}]
variable maketcl_run_timeout_ms 60000 variable maketcl_run_timeout_ms 60000
variable runstate variable runstate
@ -52,7 +57,10 @@ namespace eval ::testspace {
#envoverrides is a dict: name value pairs applied to ::env for the child (a value #envoverrides is a dict: name value pairs applied to ::env for the child (a value
#of "" means unset). Saved values are restored after the run. Returns dict: #of "" means unset). Saved values are restored after the run. Returns dict:
#timedout 0|1, exitcode <int|"">, output <combined stdout+stderr>. #timedout 0|1, exitcode <int|"">, output <combined stdout+stderr>.
proc maketcl_run {cmdargs envoverrides} { #driverexe (G-145): when non-empty, an alternative executable driven directly as
#'<driverexe> <maketcl> <subcommand...>' (e.g a plain tclsh) instead of the
#punkexe 'script' subcommand form.
proc maketcl_run {cmdargs envoverrides {driverexe ""}} {
variable runstate variable runstate
variable maketcl_run_timeout_ms variable maketcl_run_timeout_ms
variable punkexe variable punkexe
@ -75,7 +83,12 @@ namespace eval ::testspace {
} }
} }
try { try {
set chan [open |[list $punkexe script $maketcl {*}$cmdargs 2>@1] r+] if {$driverexe eq ""} {
set runcmd [list $punkexe script $maketcl {*}$cmdargs 2>@1]
} else {
set runcmd [list {*}$driverexe $maketcl {*}$cmdargs 2>@1]
}
set chan [open |$runcmd r+]
chan configure $chan -blocking 0 -translation binary chan configure $chan -blocking 0 -translation binary
catch {chan close $chan write} ;#no stdin for the child - immediate EOF catch {chan close $chan write} ;#no stdin for the child - immediate EOF
set timerid [after $maketcl_run_timeout_ms [list set [namespace current]::runstate(done) timeout]] set timerid [after $maketcl_run_timeout_ms [list set [namespace current]::runstate(done) timeout]]
@ -145,6 +158,39 @@ namespace eval ::testspace {
set result set result
} -result {timedout 0 exitcode 0 esc 0 policyline 1} } -result {timedout 0 exitcode 0 esc 0 policyline 1}
#added 2026-08-01 (agent, G-145) - remnant-free piped usage tables
test maketcl_colour_help_tables_zero_remnants {piped tclsh src/make.tcl help for every declared subject plus bare help/-help: zero ESC bytes and zero orphan CSI-fragment text} -constraints {tclshavailable} -body {
#Declared subjects: the dict keys of ::punkboot::argdoc SUMMARIES in src/make.tcl,
#pulled verbatim by the help subject arg's -choices. Characterization of the
#declared set as of 2026-08-01 - a new subcommand lands in SUMMARIES and here.
set subjects {bakehouse packages modules libs bake bakelist vfslibs bin vendorupdate libfetch bootsupport vfscommonupdate info check projectversion workflow shell help buildsuite tool}
set invocations [list help -help]
foreach subject $subjects {lappend invocations [list help $subject]}
set tclshexe [auto_execok tclsh]
set offenders [list]
#Complete-ANSI-sequence pattern, mirroring ::punkboot::ansistrip::re_complete.
set re_complete {\x1b(?:\[[0-9;:<=>?]*[ -/]*[@-~]|\][^\x07\x1b]*(?:\x07|\x1b\\)|[@-Z\\^_])}
foreach cargs $invocations {
set r [maketcl_run $cargs {NO_COLOR "" FORCE_COLOR "" PUNK_FORCE_COLOR ""} $tclshexe]
set output [dict get $r output]
set esc [esc_count $output]
#Orphan CSI-fragment detector: strip every COMPLETE sequence first, then scan
#the remainder for digit/semicolon runs terminated by 'm' not preceded by ESC
#or '['. Scanning raw output would false-positive on multi-parameter SGR
#(';' inside ESC[0;1m); scanning stripped output is exact - and when esc==0
#holds, the strip is a no-op anyway. The strip eats '[' with the ESC, so the
#G-113-era {\[[0-9;:]*m} pattern alone would miss these remnants. Verified
#2026-08-01 against the NO_COLOR+PUNK_FORCE_COLOR pre-transform corpus (all
#22 invocations, complete sequences stripped): no legit help text matches.
set stripped [regsub -all -- $re_complete $output ""]
set remnants [regexp -all -inline -- {(?:^|[^\x1b\[])[0-9;]+m} $stripped]
if {[dict get $r timedout] || [dict get $r exitcode] ne "0" || $esc != 0 || [llength $remnants] != 0} {
lappend offenders $cargs [list timedout [dict get $r timedout] exitcode [dict get $r exitcode] esc $esc remnants $remnants]
}
}
set offenders
} -result {}
cleanupTests cleanupTests
} }
namespace delete ::testspace namespace delete ::testspace

Loading…
Cancel
Save