# G-107 buildsuite library test runs: record/gate external library testsuites Status: proposed Scope: src/buildsuites/suite_tcl90/ (per-library test steps in recipe, tools/ runners, tracked dispositioned baselines), src/buildsuites/_build/ (report outputs via existing globs); pattern inherited by later suites (suite_tcl86) Goal: the external libraries the buildsuites compile and ship (thread, tclvfs, tcllib+tcllibc, tklib, tk; future bi members as they join) have their own testsuites run under the suite-built shell as named recipe steps with per-library policy - gate (parsed totals vs tracked dispositioned baseline, the core test-gate mechanism), record (complete + emit report artifacts the suite summary points to), or opt-in - so suite artifacts carry test evidence for the exact zig-built combinations nobody upstream tests. Acceptance: thread and tclvfs run gated with tracked dispositioned baselines, deterministic across two consecutive full runs; tcllib (with tcllibc accelerators engaged) and tklib run in record mode emitting report artifacts whose paths are recorded in this file; a Tk step exists opt-in with a recorded first census; per-library policy is overridable at invocation without recipe edits; a testsuite that fails to complete (no totals) fails its step in every mode; the evidence-summary shape G-103 metadata consumes is documented here. ## Context Drafted 2026-07-21 from user question: we run and gate Tcl's core testsuite (G-098 mechanism, G-102 step shape) but never run or record the testsuites of the libraries we build and ship. Survey of the pinned sources (_build/suite_tcl90 checkouts, zon pins as of G-102): - thread: tests/ with all.tcl + 5 test files (thread, tpool, tsv, ttrace, one tkt regression). Standard tcltest; timing-sensitive entries expected -> dispositions. - tclvfs: tests/ with all.tcl + 6 files; vfsFtp/vfsUrl are network-reaching - an offline census will skip or disposition those. - tcllib: no top-level tests dir - 305 .test files across modules/*/, driven by sak.tcl (its own runner and log format, NOT all.tcl totals). - tklib: same sak shape, 8 test files, Tk-dependent. - tk: 128-file tests/all.tcl suite - needs an interactive desktop, steals focus, maps real windows, runs tens of minutes, own per-platform census. - critcl: has test/ but is a build tool - tcllibc building + its accelerators passing tcllib's tests validates it transitively; its own suite is out of scope here. MACHINERY: tools/test_gate.tcl is already the generic engine (-testsdir -baseline -logfile -testargs; parses all.tcl totals rather than trusting exit codes; extracts FAILED names; diffs vs dispositioned baseline; reports stale entries). thread/tclvfs should run nearly as-is (verify each all.tcl vintage emits standard runAllTests totals; supply a thin standard driver where not). tcllib/tklib need a wrapper - drive sak and parse its log, or bypass sak with our own tcltest runner emitting standard totals; the latter also forces the mode that matters most: tcllib WITH tcllibc accelerators engaged. That is the single highest-value run in this space - zig-compiled critcl accelerators on windows against a static 9.0.5 tclsh is a combination nobody upstream tests. Same argument, milder, for thread/tclvfs: load-bearing for every kit; a miscompile surfaces as heisenbugs later. KNOWN COSTS (all characterized during G-102): initial failure census per library per platform (one-time, then stable - zon pins mean baselines move only on deliberate repin, not with a moving trunk); base-env needs (cmd.exe/ COMSPEC/PATHEXT/SystemDrive for process-spawning tests, TCL_LIBRARY for cached-exe runs, captureStdOut on cached Run steps); firewall consent when socket-using tests first listen (tcllib comm will trigger it); wall-clock (thread+tclvfs trivial next to the ~70k-test core gate; tcllib adds real minutes; tk is the outlier). POLICY TIERS (per-library defaults in the recipe, -D overridable at invocation; no new config file until suite_tcl86 forces a shared shape): - gate (after census): thread, tclvfs - small, load-bearing; fail only on UNEXPLAINED failures vs the tracked dispositioned baseline (core-gate semantics, including stale-entry reporting). - record: tcllib+tcllibc, tklib - step completes regardless, emits parsed summary + full log under _build//out/testreports/, and the suite summary points at them; promote to gate per-library once the census is dispositioned and two-run deterministic. - opt-in: tk full suite - named step outside the default composite; intended as mandatory before PUBLISHING a bi artifact, not on every build. Every mode: a run that does not complete (no totals line) FAILS the step - infrastructure breakage must never masquerade as recorded results. Gate runs stay under the plain built shell (as the core gate today); piperepl-variant divergence remains the behaviour-matrix territory of G-096 (achieved 2026-07-20; its residual patch-characterization item). Relationships: G-098 (achieved 2026-07-20) - gate mechanism precedent. G-102 (achieved 2026-07-21) - the post-tclsh step shape these runs slot into. G-103 - artifact metadata is the consumer of the evidence summaries (a family runtime published to punkbin carries per-component test evidence; aligns with G-103's note that punkbin publication follows its artifacts). G-099/G-100/ G-101 - the 8.6 arc inherits the mechanism (thread 2.8 / tclvfs baselines on 8.6); same born-onto-the-mechanism fork-timing logic as G-102-before-G-099. ## Notes - 2026-07-21: drafted and approved from the library-test discussion (survey + policy tiers above). Sequencing recommendation (agent, unconfirmed): activation fits alongside or just after G-103, before the suite_tcl86 fork; the user has not yet confirmed a slot in the G-103 -> 8.6-arc order.