G-145 root cause: ansistrip 'clear' op drops the split-sequence carry (record + shellfilter warning)
Investigation record only - no fix yet (goal remains proposed).
Root cause: the Tcl core delivers the transform 'clear' op before every
write (output-buffer flush-down; tclsh 8.7 and 9.0.3), and
::punkboot::ansistrip's transchan handles 'clear' with 'dict unset carry'
- discarding a split sequence's held tail at every chunk boundary. The
tail's remainder (e.g. 0;1m) passes through unstripped while the held
ESC[ bytes are never emitted, so the zero-ESC pin (maketclcolour.test)
stays green while fragments leak. Evidence: pre-transform stream captured
via NO_COLOR+PUNK_FORCE_COLOR is fully well-formed (textblock/punk::ansi
exonerated; G-056 not the mechanism); the verbatim strip proc is correct
for every possible 2-chunk split and 512..8192 block sizes; an identity
logging transform under ansistrip captured the boundary byte-exactly;
minimal isolated repro on a bare file channel reproduces both the
per-write 'clear' op sequence and a fragment. Deterministic per table
geometry, matching the observed intermittency.
Upstream (TEMP_REFERENCE/tcl9 survey): per-write 'clear' is intentional,
test-pinned core behavior (ioTrans.test iortrans-7.1 "chan write, write
clears read buffers") but transchan.n and TIP 230 document it as
seek-only and read-side-scoped - an upstream documentation gap, not an
implementation bug; no existing Tcl ticket found. A write-only
transform's contract-conformant 'clear' is a no-op; fix layer recorded
in the goal file (drop 'clear' from the supported-methods list or make
it a no-op; 'finalize' keeps dropping the carry).
- goals/G-145-piped-usage-ansi-remnants.md: root cause, evidence chain,
upstream documentation status, fix layer, encoding note, repro recipes
- shellfilter: G-145 warning comment over the commented-out 'clear'
method so future refactors don't discard o_encbuf/stream state there
goals_lint clean (83 active-index goals, 62 archived).
Assisted-by: harness=opencode; primary-model=opencode/kimi-k3; api-location=unknown
@ -11,12 +11,22 @@ Observed 2026-08-01 while diffing before/after help captures for the argdoc rest
## Approach
- Evidence so far: the G-113 write-side strip (::punkboot::ansistrip) already carries a trailing incomplete sequence across write chunks (per-channel carry; flush preserves it, finalize drops it), so a clean ESC-boundary split at the channel would be handled. First suspect is therefore upstream: styled content split by ANSI-aware width/cell handling before write, leaving ESC-less fragment text a byte-level strip correctly ignores.
- Evidence so far: the G-113 write-side strip (::punkboot::ansistrip) already carries a trailing incomplete sequence across write chunks (per-channel carry; flush preserves it, finalize drops it), so a clean ESC-boundary split at the channel would be handled. First suspect is therefore upstream: styled content split by ANSI-aware width/cell handling before write, leaving ESC-less fragment text a byte-level strip correctly ignores. (First suspect REFUTED 2026-08-01 - root cause is transform-side, see Notes.)
- Piped runs do legitimately receive ANSI from module emitters (the strip is load-bearing by design - G-113); punk::args resolve caching also carries a colour-state staleness todo (cached specs can embed colour from resolve time), a possible fragment source to check.
- Diagnosis route: capture the pre-transform stream for an affected subject (transform popped, or PUNK_FORCE_COLOR run) and determine whether fragments exist pre-strip; fix in textblock/punk::ansi width handling if so, else harden the transform.
## Notes
- ROOT CAUSE (determined 2026-08-01): transform-lifecycle defect in `::punkboot::ansistrip` (src/make.tcl), NOT renderer-side splitting. The Tcl core delivers a `clear` op to the transform stack before EVERY write op (output-buffer flush-down; observed on tclsh 8.7 and 9.0.3, on the real stdout stack and on a bare file channel with no make.tcl code involved). `ansistrip::transchan`'s `clear` handler runs `dict unset carry $chanid`, discarding a split sequence's held tail at every chunk boundary: write #k ends mid-sequence (e.g `ESC[` held in carry), `clear` drops it, write #k+1 begins with `0;1m` which passes through unstripped, and the held `ESC[` bytes are never emitted. The dropped ESC bytes are why the G-113 zero-ESC pin (maketclcolour.test) stays green while fragments leak. Evidence chain:
(a) pre-transform stream captured via `NO_COLOR=1` + `PUNK_FORCE_COLOR=1` (NO_COLOR wins colour, force skips the transform push) is fully well-formed - every `ESC[...m` complete; textblock and punk::ansi cell/width handling are exonerated, and G-056 is not the mechanism.
(b) the verbatim `strip` proc fed the captured stream under every possible 2-chunk split and 512..8192 block sizes yields zero fragments - the carry logic itself is sound for arbitrary chunking.
(c) an identity logging transform pushed on stdout before sourcing make.tcl (ansistrip stacks on top) recorded `clear` before every `write`; the output chunk boundary landed exactly at the observed `0;1m` remnant (prior chunk ended with the Default-cell spaces, the following `ESC[` held then dropped).
(d) minimal isolated repro (verbatim transform on a file channel, repetitive styled payload, no make.tcl): op log shows `clear` per flush-down, output carries a fragment (`;1m`).
Determinism: fragments are deterministic per table geometry (buffer-boundary positions are fixed; a remnant appears only where a boundary lands inside a sequence), matching the observed intermittency and position shifts across subjects/restyles.
- Upstream documentation status (2026-08-01, TEMP_REFERENCE/tcl9 survey): the per-write `clear` is INTENTIONAL Tcl core behavior carrying an upstream documentation gap. `generic/tclIORTrans.c``ReflectOutput` (~line 1280) invokes the transform's `clear` before every write when METH_CLEAR is declared, with the comment "Discard partial data in the input buffers, i.e. on the read side. Like we do when explicitly seeking as well." The core test suite pins it: `tests/ioTrans.test` iortrans-7.1 "chan write, write clears read buffers" expects `{clear rt*} {write rt* snarf}` from a plain puts+flush (7.2 seek, 7.3 result-ignored, 7.4 lifecycle bug 2921116). But `doc/transchan.n` (lines 47-54) documents `clear` as "called when a chan seek is performed on the channel being transformed", and TIP 230's text scopes it to "internal input buffers" that "happens only when the user seeks the channel the transformation is attached to" - both silent on the write-path call. Verdict: upstream DOCUMENTATION bug (behavior is deliberate and test-pinned; man page and TIP text are stale), not an implementation bug; no existing Tcl ticket found covering the discrepancy (one tangential refchan write-buffering ticket, de232b49f2, unrelated to transforms). Two consequences for the fix: (a) `clear` only fires at all when the transform DECLARES it (the core gates on METH_CLEAR), so dropping it from the supported-methods list eliminates the per-write calls entirely; (b) `clear`'s documented scope is INPUT/read-side buffers - a write-only transform like ::punkboot::ansistrip has no read-side state, so its contract-conformant `clear` is a no-op; dropping the write-side carry was wrong under both the documented and the actual contract.
- Fix layer: the transform's `clear` handling in src/make.tcl (`::punkboot::ansistrip::transchan`) - `clear` must not discard the per-channel carry (drop the op from the supported-methods list or make it a no-op); `finalize` dropping the carry remains correct (incomplete sequence at end of stream). No change needed in textblock/punk::ansi.
- Encoding note (raised during investigation): the byte-level transform is encoding-safe for its approach - 0x1B never occurs inside UTF-8 multibyte chars (a box char split `e2 94 | 81` across writes was observed and is harmless), and utf-16-class channels are guarded at push time (ESC would arrive as `1B 00`). Compare `shellfilter::chan::ansistrip` (shellfilter-999999.0a1.0.tm): it decodes via `encoding convertfrom` and carries `o_encbuf` for split multibyte chars ("bytes can break at arbitrary points making encoding conversions invalid"), but explicitly assumes line buffering and does not handle split ANSI codes at all. The punkboot defect is lifecycle (`clear`), not encoding.
- Repro (2026-08-01, current geometry, tclsh 9.0.3, all deterministic): bare `tclsh src/make.tcl help` piped -> 1 fragment (`0;1m`); `help libs` / `help info` -> `;1m` x1 each; `help bootsupport` -> `1m` x1. Detector regex: `(?<![\x1b\[])[0-9;]+m` (bracket is eaten with the ESC, so the G-113-era `{\[[0-9;:]*m}` pattern alone would miss these). In-shell: `set out [runout -n tclsh src/make.tcl help]` then `regexp -all -inline -- {(?:^|[^\x1b\[])[0-9;]+m} $out` (exec/runout pipe stdout, engaging the transform).
- G-113 achieved 2026-07-25 (goals/archive/G-113-maketcl-tty-aware-colour.md) - the colour policy and zero-ESC pin surface (maketclcolour.test) this extends.
- Related: G-056 (proposed) - ANSI-aware wrap/width machinery ('styling in effect carries across the wrap'); if the root cause is renderer-side splitting, the fix layer coincides.
