punkshell/goals/archive/G-154-tclparser-punk-zig-build.md

G-154 tclparser_punk zig build: fork-as-upstream parser dll for kits and the G-070 oracle

Status: achieved 2026-08-02 Scope: external fork repo c:/repo/jn/tclparser_punk (gitea jn/tclparser - the build lane lands there); src/buildsuites/suite_tcl90/sources.config and src/buildsuites/suite_tcl90/build.zig.zon (ready-to-land tclparser records drafted in the detail file; landing them is suite-consumption era); src/buildsuites/suite_tcl86/sources.config and src/buildsuites/suite_tcl86/build.zig.zon (same - the 8.6 records); src/vendorlib_tcl8/win32-x86_64/tclparser1.8 and src/vfs/*/lib_tcl9/tclparser1.9 (vendored third-party parser packages - replacement targets at consumption, not touched by this goal); goals/G-070-pure-tcl-tclparser.md (consumer - parity oracle and corpus seed) Goal: The punkshell-maintained tclparser fork (c:/repo/jn/tclparser_punk - gitea jn/tclparser - tcltk-depot parser 1.9 porting baseline with the aspect fossil lineage preserved as an in-repo reference snapshot) builds standalone under the pinned zig toolchain as stubs-linked parser dlls for Tcl 8.6 and Tcl 9, for win32-x86_64 and win32-ix86 alike, gated by the ported upstream testsuite - so kit parser packages (today unprovenanced third-party dlls) and the G-070 parity oracle rest on a first-party, commit-pinned, provenance-carrying source referenced by the buildsuites the punkzip/metakit way (commit pin in sources.config plus zon tarball record) rather than on dormant third-party upstreams or opaque vendored binaries. Acceptance: In c:/repo/jn/tclparser_punk under the pinned zig 0.16.0, 'zig build' (lazy zon tcl pins, no local source trees) and 'zig build dual' pass, producing tcl9tclparser19.dll and tclparser19t.dll for both windows arches; 'zig build test -Dtclsh9= -Dtclsh86=' passes the ported 114-case upstream suite for both generations under suite-built shells with the transcript recorded in this file; both upstream snapshots are landed byte-frozen with the provenance table and root LICENSE (license.terms verbatim); a punk kit-class shell for each generation loads the built dll and completes a parse-command round trip recorded in this file; and the fork commit is pinned with ready-to-land sources.config and build.zig.zon record text for both suites captured in this file.

Context

punkshell kits already ship the tclparser C extension (Tcl package parser: the TclPro parser exposing Tcl_ParseCommand and friends as the parse command) as third-party binaries: tclparser1.9 dirs (tcl9tclparser19.dll, magicsplat-shaped) in several vfs lib_tcl9 trees and tclparser1.8 (tclparser18.dll) in src/vendorlib_tcl8 and bawt-era vfs trees. Those dlls arrived in bulk commits (ae837492, 332d6732) with no provenance records - exactly the gap the provenance/trust direction closes for runtimes (G-123, achieved - see goals/archive/G-123-thirdparty-runtime-tiers.md). punk::lib's script-analysis machinery consumes the package (G-019 direction), G-069 wants it as a lint oracle, and G-070 needs it as the behavioural reference for a pure-Tcl implementation.

Upstream landscape (survey 2026-08-02, detail in the fork's src/reference/AGENTS.md): the aspect chiselapp fossil repo is a TclPro CVS mirror with conservative fixes, version 1.4.1, dormant since 2015-06-18 (trunk 53f5b44b14); tcltk-depot/tcl-parser on github is parser 1.9 - the same TclPro component extracted from the tclprodebug lineage and ported to Tcl 9 (July 2025, 4 commits, tip c6c841d6). The two lineages converged: the whole tclParser.c delta is prototype/type modernisation including a Tcl_Size compat shim that builds 8.6 AND 9 from one source; parse logic (including the TclFindElement-based parse list body) is identical. Both upstreams are dormant-to-minimal, so the buildsuites' live-upstream referencing pattern does not fit; the applicable precedent is punkzip (G-126, achieved) / metakit_punk (G-153, active): a punkshell-maintained fork that IS the upstream, self-hosted on gitea, commit-pinned by the suites, with its own zig build system.

The fork repo was seeded 2026-08-02 (user-created git repo, agent-built skeleton per the metakit_punk shape): DOX/AGENTS.md governance, byte-frozen snapshots of BOTH upstreams under src/reference/ with a provenance table and fork-difference analysis, port baseline (tcltk-depot tclParser.c + 114-case testsuite + parse docs, verbatim), root build.zig/build.zig.zon, and the testsuite gate. This goal is the plan of record for the lane; the port is driven from punkshell sessions with access to this goals tree.

Approach

• Source-identity pins: the fork's lazy zon tcl dependencies are the SAME checkouts the suites pin (core-9-0-branch 1a9c3b9d96 for tcl9, core-8-6-branch 1d1d5cbd91 for tcl86), with -Dtcl9-src/-Dtcl86-src escape hatches, so a bare clone builds with no punkshell present and source identity matches the suites.
• Single source, per-generation compile: the depot baseline's Tcl_Size shim means one tclParser.c compiled twice - against tcl9 headers producing tcl9tclparser19.dll and against tcl86 headers producing tclparser19t.dll (drop-in artifact names matching the packages kits already carry; install layout lib///parser1.9/ with generated pkgIndex.tcl, explicit Tclparser load prefix; platform dirs per punkshell canon win32-x86_64 / win32-ix86 - G-105).
• Stub library compiled from generic/tclStubLib.c of the pinned tree, never a prebuilt archive: a foreign-toolchain libtclstub .a was measured to leave Tcl_InitStubs unresolved under zig's lld (2026-08-02, suite out/lib archives) - compiling the one-file stub lib sidesteps archive-format coupling entirely.
• Testsuite as the gate: tools/testsuite_check.zig materialises a scratch dir, wraps the dll load + source all.tcl under a caller-provided tclsh, and gates on the tcltest totals line (Failed 0, Total > 0). Dual-generation green is the port-correctness signal; the same 114-case suite seeds G-070's parity corpus.
• win32-ix86: the dual step builds both arches for both generations (compile/link proof). The ix86 RUNTIME check needs a 32-bit shell and lands when G-130's runtimes exist; risk direction is benign (the extension is a thin wrapper over the core parser API).
• Suite consumption (later, not this goal): sources.config commit pin + zon tarball record per suite, replacing the vendored parser package dirs at the same time; delivery-tier shape (punkbin lib-tier precedent: G-138, achieved
  • see goals/archive/G-138-punkbin-library-artifact-tier.md) decided at consumption. Draft records below become concrete once the fork is pushed and pinned.

Notes

• Related: G-070 (proposed) - primary consumer: the fork is its behavioural reference (snapshots + provenance) and the zig-built dll its parity oracle; the ported testsuite is its corpus seed. G-070's Scope/Acceptance were reworded 2026-08-02 to point here (formerly TEMP_REFERENCE, which never received a tclparser tree).
• Related: G-069 (proposed) - consumer: the splitter cross-check lint runs where the parser binary is available; first-party dlls widen that gate (and G-070 widens it further).
• Related: G-153 (active) - the pattern precedent (fork-as-upstream, dual-arch acceptance, suite pin-consumption drafts); this goal shares the sources.config/build.zig.zon record surfaces with its metakit records.
• Related: G-101 (proposed) - owns the 8.6 suite consumption era in which the suite_tcl86 records land (as G-101-era consumption, like metakit's).
• Related: G-105 (proposed) - platform-name canon the install layout follows.
• Related: G-130 (proposed) - the win32-ix86 runtime lane; the ix86 dll runtime check runs under its 32-bit shells when they exist.
• Related: G-004/G-005/G-006 (proposed) - binary-free direction: consumption replaces the unprovenanced vendored parser dlls (bulk commits ae837492, 332d6732 carry no provenance records) with first-party builds.
• Related: G-019 (proposed) - downstream analysis consumer via punk::lib.
• Suite-dir neighbours G-131/G-108/G-110/G-116 have no parser coupling beyond the shared suite record surfaces.
• Overlap survey 2026-08-02: goals_xref paths src/buildsuites/suite_tcl86, src/buildsuites/suite_tcl90, src/vendorlib_tcl8, src/modules/punk/lib-999999.0a1.0.tm -> G-101 G-153 G-131 G-105 G-130 G-108 G-110 G-116 G-019 G-070 (dispositions above; src/vendorlib_tcl8 intersects no live goal).
• Ready-to-land sources.config record (both suites; name/dir chosen to match the gitea repo name; the ssh origin ssh://git@pcm-gitea1.corp.intx.com.au:2222/jn/tclparser is the push remote, the https web URL is the fetch/pin form; pinned 2026-08-02): source tclparser git https://www.gitea1.intx.com.au/jn/tclparser 16d2f460b89dfd819aa08ceb97dfebf37b9fa55a tclparser
• Ready-to-land build.zig.zon record (both suites; hash computed with the pinned zig 0.16.0 via 'zig fetch' from the suite_tcl86 dir per that file's TRAP note, 2026-08-02): .tclparser = .{ .url = "https://www.gitea1.intx.com.au/jn/tclparser/archive/16d2f460b89dfd819aa08ceb97dfebf37b9fa55a.tar.gz", .hash = "tclparser_punk-0.1.0-aKPixmovFABEMo61xgv1YjUqdgqe41ZiB-EucAgmM-We", .lazy = true },
• Package identity facts: the dll provides parser 1.9 for BOTH generations (PACKAGE_VERSION define; upstream-shaped for drop-in compatibility). The fork repo's own version is its zon .version (0.1.0), independent - punkzip/metakit precedent.
• Testsuite quirk worth knowing for G-070 corpus work: tests/all.tcl checks info commands parser (not parse) before package require parser, so the package-require path always runs unless a command named parser exists; a loaded dll satisfies it via Tcl_PkgProvide. The 2 skips per generation are the OTHER generation's constraint-gated cases (tcl8/tcl9 tcltest constraints).
• Build accommodation facts (rationale in the fork's build.zig header): -fno-sanitize=undefined/-fno-sanitize-trap=undefined (2000-era C under zig's default trapping UBSan on Debug-class builds; suite tcl/tk/zlib precedent), -Wno-inconsistent-dllimport (EXTERN storage-class quirk after tclInt.h resets TCL_STORAGE_CLASS; symbol still exports). The single tclInt.h dependency is TclFindElement (parse list), reached through the internal stub table; going public-API-only is a recorded fork option, not assumed.

Progress

Seed increment landed 2026-08-02 (fork commits 7c18555 governance+snapshots, 16d2f46 port+build, on local master; push to the gitea origin pending - user decision): DOX framework (root/src/src-reference AGENTS.md), byte-frozen snapshots aspect@53f5b44b14 (21 files) + tcltk-depot@c6c841d6 (28 files) with provenance table, port baseline verbatim (tclParser.c byte-identical to the depot snapshot; tests; parse.man/parse.n), root LICENSE, build.zig/zon (lazy suite-matching tcl pins + -D*-src escapes), tools/testsuite_check.zig, dual step, generated pkgIndex per package dir.

VERIFIED 2026-08-02 under the pinned zig 0.16.0 (punkshell bin/tools/zig-x86_64-windows-0.16.0/zig.exe):

• 'zig build' green via BOTH source paths: -D*-src suite checkouts AND the lazy zon fetches standalone (no local tcl trees).
• 'zig build dual' green: 4 dlls (tcl9+tcl86 x win32-x86_64+win32-ix86).
• 'zig build test' green both generations; totals transcript: tcl9 (suite tclsh90szip, Tcl 9.0.5): all.tcl: Total 114 Passed 112 Skipped 2 Failed 0 (skipped constraint: 2 tcl8) tcl86 (suite tclsh86t, Tcl 8.6.18): all.tcl: Total 114 Passed 112 Skipped 2 Failed 0 (skipped constraint: 2 tcl9)

Completion increment 2026-08-02 (same day): the user pushed the fork to the gitea origin; acceptance completed in full:

• Pin: origin/master = 16d2f460b89dfd819aa08ceb97dfebf37b9fa55a (the port+build seed commit). The ready-to-land records in Notes are concrete (zig-fetch hash computed from the suite_tcl86 dir under the pinned zig).
• 'zig build dual' re-verified via the lazy zon tcl pins alone (no local source trees): 4 dlls green.
• Kit-class shell load + parse round trip (x64 dlls from the standalone zon build; scratch kit copies per the locked-exe discipline): punk902z.exe (Tcl 9.0.2 kit) + tcl9tclparser19.dll -> parser 1.9; parse command 4-element result; parse list + getstring round trip OK punksys.exe (Tcl 8.6.13 kit) + tclparser19t.dll -> parser 1.9; same round trip OK Built against 9.0.5/8.6.18 headers, loading in 9.0.2/8.6.13 kits - the stubs contract covering the version spread as intended.
• BEYOND-ACCEPTANCE BONUS: the win32-ix86 dlls verified at RUNTIME too (the Approach's G-130 deferral dissolved - the third-party 32-bit verification kits already on hand cover it, WoW64): punkluck86.exe (Tcl 8.6.10 ix86 kit) + win32-ix86 tclparser19t.dll -> parser 1.9, round trip OK (the kit's punk::mix template-provider warnings are its known vfs quirk, unrelated) punk91ix86.exe (Tcl 9.1b0 ix86 kit) + win32-ix86 tcl9tclparser19.dll -> parser 1.9, round trip OK Combined coverage: Tcl 8.6.10/8.6.13/8.6.18 and 9.0.2/9.0.5/9.1b0, x64 and ix86 alike, from two dll builds per arch. All Remaining-for-acceptance items are resolved; every acceptance clause is verified (builds standalone + dual, testsuite gate transcript above, snapshots + provenance + LICENSE landed, kit-shell transcripts, pinned records captured).