punkshell/goals/G-153-metakit-punk-zig-port.md

G-153 metakit_punk zig port: fork-as-upstream Mk4tcl for the 8.6 kit container

Status: active Scope: external fork repo c:/repo/jn/metakit_punk (gitea jn/metakit - the port itself lands there); src/buildsuites/suite_tcl86/sources.config and src/buildsuites/suite_tcl86/build.zig.zon (ready-to-land metakit records drafted in the detail file; landing them is G-101-era consumption); TEMP_REFERENCE/{metakit_jnorthrup,metakit_pooryorick,KitCreator} (read-only patch/boot-chain reference - machine-local); goals/G-101-tcl86-kit-container-strategy.md (consumer decision record) Goal: the punkshell-maintained metakit fork (c:/repo/jn/metakit_punk - gitea jn/metakit - pooryorick 2.4.9.8 porting baseline with the dead upstreams preserved as in-repo reference snapshots) builds standalone under the pinned zig toolchain as a Tcl-agnostic C++ core library plus the Mk4tcl binding in both loadable-dll and static-embed shapes, for win32-x86_64 and win32-ix86 alike, and a suite-built 8.6 shell loads and exercises the dll - so the G-101 mk4 container candidate rests on a first-party, commit-pinned, provenance-carrying source referenced by the buildsuites the punkzip way (gitea commit pin in sources.config plus zon tarball record) rather than on dead third-party upstreams or a wholesale vendor copy. Acceptance: in c:/repo/jn/metakit_punk under the pinned zig 0.16.0, 'zig build' and 'zig build test' pass for both x86_64-windows and x86-windows with the 32-bit tests executed natively on the x64 host (WoW64); the KitCreator mk4tcl patch set (eight patches vs 2.4.9.7) is audited against the 2.4.9.8 baseline with a per-patch disposition recorded (ported as fork commits citing origin / already present in baseline / rejected with reason); the Mk4tcl binding builds as a stubs-linked dll against the same tcl core-8-6-branch pin suite_tcl86 carries and suite-built tclsh86t loads it - package require Mk4tcl plus an mk::file open/set/select/close round trip - with the transcript recorded in this file; an mk4 datafile written by the x86_64 build reads back identically under the ix86 build and vice versa; a static-embed shape (archive registerable via Tcl_StaticPackage) links into a copy of the kit-class shell target or its blocker is recorded here; the fork commit is pinned with ready-to-land sources.config and build.zig.zon record text captured in this file; and metakit_punk's root LICENSE lands with the first ported sources (MIT/X11 retained).

Context

Metakit's upstreams are dead or dormant: the official jcw lineage froze at 2.4.9.7 (jnorthrup GitHub mirror, last commit 2011) and the pooryorick community fork at 2.4.9.8 (2018). The buildsuites' referencing pattern for LIVE upstreams (fossil branch records in sources.config + per-checkin tarball pins in build.zig.zon, suite-owned recipes over raw trees) therefore does not fit; the applicable precedent is punkzip (G-126, achieved 2026-07-27 - see goals/archive/G-126-punkzip-accelerator.md): a punkshell-maintained fork that IS the upstream, self-hosted on gitea, commit-pinned by the suites, with its own zig build system.

The fork repo was seeded 2026-07-30 (user-built skeleton): DOX/AGENTS.md governance, read-only reference snapshots of both upstreams under src/reference/ with a provenance table, and a port-relevant fork-difference analysis (33 files differ 2.4.9.7 -> 2.4.9.8; classified noise / warning-hygiene / type modernization / Tcl-binding behavior). Its own contracts already fix key decisions: port from the pooryorick snapshot only; Python binding out of scope; upstream build systems (autoconf/MSVC/CMake) not carried; snapshots byte-frozen. Direction confirmed by the user 2026-08-02: the port is driven from punkshell sessions with access to this goals tree, so this file is the plan of record.

Boundary (deliberate, keeps the fork reusable): metakit_punk owns the storage engine (upstream src/ + include/) and the mk::* Tcl binding (tcl/mk4tcl.cpp/.h + tcl/tests). Kit boot machinery - kitInit.c preinit, boot.tcl, setupvfs-style kit writers, rechan - stays G-101 suite-side territory (kbskit/KitCreator kitsh are the references, per G-101's reading order), and mk4vfs.tcl ships with tclvfs which the suite already builds. tequila/mkshow/mk4too stay reference-only unless a need appears.

Sequencing value: milestones 1-2 (standalone build; dll smoke under tclsh86t) need no shellspy changes at all, and the dll milestone doubles as G-101's cheap first probe for the sdx bootstrap circularity (run sdx under the suite-built dynamic shell with Mk4tcl loaded as a package - option (a) in G-101's sdx section) and as the proof that a zig-built Mk4tcl works at all.

Approach

• Target axis from day one: build.zig uses b.standardTargetOptions like punkzip and build86.zig; test steps run for x86_64-windows AND x86-windows (WoW64 executes 32-bit natively on the x64 host - force the run steps if the builder classifies the target foreign). Resolved triples map to punk canonical names (x86-windows-gnu -> win32-ix86) for any artifact naming.
• Tcl headers without losing standalone buildability: a -Dtcl-src= build option whose default is a lazy build.zig.zon dependency on the SAME tcl core-8-6-branch tarball pin suite_tcl86 carries (checkout 1d1d5cbd91), so source identity matches the suite and a bare clone still builds with no punkshell present. The dll shape links tclstub86; the static shape compiles for Tcl_StaticPackage registration.
• Suite consumption shape (when G-101 lands it): unlike tcl/thread/tclvfs (raw trees + suite-authored recipes), metakit_punk brings its own build.zig - the suite consumes it as a zig package dependency (b.dependency().artifact()) rather than growing a forked build_mk4tcl86 recipe. Expose installed headers + named static libs cleanly for that.
• Referencing is pin-only: no wholesale vendor of the fork into shellspy while the port churns. If a make.tcl-level tool step later needs to rebuild metakit from a bare shellspy checkout, the punkzip vendoring pattern (src/tools// + PROVENANCE.md, edit-upstream-first, re-vendor procedure, G-063-style licence records) transfers directly.
• Restraint rules from the fork's own docs carry extra force during the port: never re-simplify the pooryorick paren/brace hygiene; prefer per-file warning flags in build.zig over source edits so diffs against the reference snapshot stay meaningful.
• Early C++ spike before porting all ~47 files: link a trivial C++ static lib into a copy of the kit-class shell target at both arches (zig's libc++ is static on windows-gnu targets - measured 2026-08-02, no new dll imports beyond the UCRT set the shells already carry, recorded in G-130 Notes) and check whether metakit's THROW layer permits -fno-exceptions, which simplifies the i386 EH story.

Notes

• Related: G-101 (proposed) - the primary consumer: this goal is its mk4 candidate's leadup, separately landable; the container decision, boot chain, sdx provenance/circularity, and the metakit-under-Tcl-9 port-cost survey all live there. G-101 flags "deciding which patch lineage to carry is a required step" - this goal's patch audit IS that step, landing as fork commits.
• Related: G-131 (proposed) - the strongest argument for the mk4 candidate is one container format across both generations; G-131's metakit arm cannot complete before the port work this goal starts. The 8.6-side shapes land here; the Tcl 9 mk4tcl port stays future work at the cost G-101 surveyed (~23 Tcl_Size call sites, one Tcl_ChannelType, five Tcl_ObjType initialisers) - keep version-sensitive spots localized as they are touched, do not port to 9 under this goal.
• Related: G-130 (proposed) - the win32-ix86 axis is baked into this goal's acceptance (dual-arch tests + mk4 interchange check) so the fork never grows an x64-only assumption; G-130 owns the 32-bit runtimes themselves. The interchange check matters because punkshell's cross-baking path writes kits on the x64 host for ix86 runtimes (the G-127 target-keyed bake lineage - achieved 2026-07-31, see goals/archive/G-127-crosstarget-vfs-bake.md).
• Related: G-105 (proposed) - platform-name canon and the zig-triple <-> platform-dir mapping this goal's artifact naming follows.
• Related: G-026 (proposed) and G-063 (proposed) - fork-boundary provenance and per-package licence records; metakit is MIT/X11 (license.terms retained in the snapshots and in ported sources; root LICENSE lands with first port).
• Patch-audit inventory (from G-101's reference survey, KitCreator/mk4tcl/ patches/all/ vs metakit 2.4.9.7): 64-bit filespace, 32/64 fix, unregisterchans, fixeventchannels, shared-obj linker, ranlib/ar (eight files total). The pooryorick 2.4.9.8 baseline already absorbed at least the channel unregistration and stdint fixed-width types (per the fork-difference analysis in metakit_punk src/reference/AGENTS.md); the 64-bit filespace and fixeventchannels lineage are the likely live ones. Audit each; x86_64 correctness hinges on the 64-bit set, and the linker/ranlib ones are expected autoconf-era rejects.
• Draft sources.config record for suite_tcl86 (name/dir chosen to match the gitea repo name; dir names are recipe-coupled per that file's header): source metakit git https://www.gitea1.intx.com.au/jn/metakit <commit> metakit (commit pin like punkzip - the fork carries no tags yet; the ssh origin ssh://git@pcm-gitea1.corp.intx.com.au:2222/jn/metakit is the push remote, the https web URL is the fetch/pin form).
• Draft build.zig.zon record for suite_tcl86 (hash computed with the pinned zig at landing time - 'zig fetch' from the suite dir per that file's TRAP note): .metakit = .{ .url = "https://www.gitea1.intx.com.au/jn/metakit/archive/<commit>.tar.gz", .hash = "<zig-fetch>", .lazy = true },
• UCRT/deployment: adding the C++ engine to shells changes exe size, not the redistribution story - zig links libc++ statically on windows-gnu and the CRT import surface stays the UCRT set measured in G-130 Notes (2026-08-02).
• 32-bit specifics: metakit's native era was 32-bit, so risk direction is benign; mmap address space caps very large datafiles on ix86 (irrelevant at kit scale - worth one doc line in the fork); the mk4 FILE FORMAT is portable across arches by design, which is exactly what the interchange acceptance pins.
• Version/package identity: keep package provide Mk4tcl at an upstream-shaped version initially (2.4.9.8-era) for boot-script compatibility; fork releases take their own repo-level versioning like punkzip did (v2.x.y). Decide the exact package-version policy at first dll landing and record it here.
• Overlap survey 2026-08-02: goals_xref paths src/buildsuites/suite_tcl86, src/buildsuites, bin/runtime/win32-ix86 -> G-101 G-131 G-105 G-130 (all named above); suite_tcl90-side neighbours G-108/G-110/G-116 are target/debug/tls lanes with no metakit coupling beyond shared suite dirs.

Progress

Increment 1 landed 2026-08-02 (metakit_punk commit b9670b3, local master - push to the gitea origin pending): pooryorick core + regress suite ported byte-identical (src/include 6 headers, src/core 40 files + config.h fork addition, src/tests 18 sources + 248-file ok/ set + fixtures/reversed from upstream unix/); root build.zig + build.zig.zon (package .metakit_punk 0.1.0, min zig 0.16.0) + tools/regress_check.zig host driver; root LICENSE (MIT/X11 license.terms verbatim). VERIFIED: 'zig build' plus 'zig build test -Dtarget=x86_64-windows' and -Dtarget=x86-windows all green under the pinned zig 0.16.0 (6/6 build steps each; 32-bit regress executed natively via WoW64; independent rerun of each arch binary + 'diff -r --strip-trailing-cr src/tests/ok' clean at 248 files per arch) - the acceptance's first clause is met by the current tree and re-verifies at the final pinned commit.

Port findings worth keeping (details in the fork's src/AGENTS.md + build.zig):

• The Approach's C++ spike question is answered: the core is exception-free (zero throw/try/catch sites) and RTTI-free; -fno-exceptions -fno-rtti are live fork flags, so the ix86 EH story is a non-issue. The regress harness's try/catch compiles away under GNUC (q4_NOTHROW).
• zig cc's default trapping UBSan fires on 2007-era formal UB (derived.cpp:308 c4_FilterSeq::PreChange binds a reference to a null c4_Cursor on non-insert notifications) -> -fno-sanitize=undefined -fno-sanitize-trap=undefined, the suite's tcl/tk/zlib accommodation. A source-level UB-hygiene pass would be a fork improvement project, not port scope.
• config.h (autoconf/CMake product upstream) is a tracked fork file: HAVE_MEMMOVE is its only live setting on clang/mingw targets; q4_I86/M68K CPU flags are defined-but-unused, format portability is byte-level by design.
• Container layer q4_UNIV default (std.cpp excluded) per upstream CMake; library installs as lib/mk4.lib (0.16 name) + include/{mk4*.h,*.inl} - the linkLibrary(dep.artifact("mk4")) consumer shape the suite arm needs.

Increment 2 landed 2026-08-02 (metakit_punk commit 9b1befa): the interchange acceptance clause is MET. Upstream demo/dump/struct ported byte-identical as the check vehicles (user-approved fold-in; myio/bigdemo stay reference-only); 'zig build demos' (selected target) and 'zig build interchange' (fixed dual windows targets + host driver tools/interchange_check.zig) added. Demos compile WITHOUT -fno-exceptions (dump.cpp carries real try/catch on windows); the core lib stays exception-free. VERIFIED under zig 0.16.0: full step matrix (build / test x64 / test x86 / demos / interchange) PASS. Interchange transcript (also saved as evidence files in the step scratch): OK demo stdout x64 vs x86 OK READ-BACK dump: x64-written file, x64 vs x86 reader OK READ-BACK dump: x86-written file, x86 vs x64 reader OK WRITER-PARITY dump: x64-written vs x86-written OK READ-BACK struct: x64-written file, x64 vs x86 reader OK READ-BACK struct: x86-written file, x86 vs x64 reader OK WRITER-PARITY struct: x64-written vs x86-written INFO datafiles differ physically (sizes 437/437, first diff at offset 169) - logical identity is what the format guarantees interchange_check: PASS - read-back identity and writer parity hold in both directions (dump + struct) Patch-audit observation carried forward: the two writers' files differ by ONE byte (offset 169, 0x9C vs 0x0C - a continuation bit in the commit-walk varint region) at equal sizes; read-back identity holds bidirectionally, so this is bounded, but the 64-bit-filespace patch lineage review should explain it. Binding-milestone fact recorded in the fork DOX: upstream CMake compiles mk4too.cpp INTO the Mk4tcl shared lib (metakit_tcl_SOURCES) - include-or-not is a decision to record when the dll lands.

Increment 3 landed 2026-08-02 (metakit_punk commit 07ca657): the PATCH-AUDIT acceptance clause is MET. All KitCreator mk4tcl patches audited per-patch against the 2.4.9.8 baseline with the disposition table recorded in the fork's src/reference/AGENTS.md (beside the fork-difference analysis it completes) - outcome: ZERO patches to port. The four live lineages (unregisterchans, 64-bit filespace wideint returns, cursor int->long widening incl. the misleadingly-named fix64bitbigendianfff484, event-channel source machinery) are all verifiably present in the baseline (grep-verified sites recorded in the table); compilewithcvs is superseded by the baseline's CONST86 declarations + version-guarded Tcl_GetErrorLine shim (which keeps the line number the patch dropped); dontrequireautoconf / fixsharedobjlinker / supportranlibandar are rejected as autoconf-era build-system fixes the zig build supersedes. This confirms pooryorick 2.4.9.8 as the correct porting baseline. COUNT CORRECTION: the set is NINE patches, not the eight this file's Notes inventory and the Acceptance parenthetical say - the audit covered all nine (a superset of the contracted eight; amending the Acceptance wording is a user call). Second mk4too data point: KitCreator's Mk4tcl link rule bundles mk4too.o exactly as upstream CMake does - the binding milestone's include-or-not decision now has both upstream builds on the include side; the fork default (exclude per the reference-only boundary, kit boot uses mk::* not mk4too) stands until recorded otherwise at dll landing.

Remaining for acceptance:

• Mk4tcl binding as stubs-linked dll against the suite tcl core-8-6-branch pin (1d1d5cbd91; lazy zon dep + -Dtcl-src option per Approach) + tclsh86t load and mk::file round-trip transcript recorded here.
• Static-embed shape (Tcl_StaticPackage) into a kit-class shell copy, or its blocker recorded here.
• Fork commit pinned + ready-to-land sources.config and build.zig.zon record text captured here (drafts above become concrete once the port is pushed).