punkshell/goals/G-126-punkzip-accelerator.md

G-126 punkzip as a vendored zip accelerator: zig 0.16 port, reproducible build into bin/, punk::zip fast path

Status: proposed Scope: src/tools/punkzip/ (vendored punkzip source tree - root name settled in the work - with provenance and licence records); src/make.tcl (tool build step); bin/tools/zig* (pinned toolchain as consumed); bin/punkzip.exe (untracked build output); src/modules/punk/zip-999999.0a1.0.tm (accelerator detection + fast path behind the G-124 pure-Tcl floor); src/tests/modules/punk/zip/ (parity suite - accelerated and pure-Tcl paths must agree); upstream maintained checkout c:/repo/jn/zig/punkzip (re-vendor source, read-only from this repo) Goal: punkshell carries punkzip as vendored source, builds it from the repo's pinned zig toolchain, and uses the resulting binary as an optional accelerator for zip reading - with the pure-Tcl path as the always-available floor - so a machine with zig gets fast archive listing and extraction (materially so for bakes with large library payloads) while a clean checkout with only tclsh still builds and bakes exactly as before. Acceptance: the vendored source builds under the repo's pinned zig 0.16 toolchain with 'zig build test' green (the port's own 10 test modules), driven by a make.tcl step that produces bin/punkzip.exe reproducibly on a clean checkout - and zig stays OPTIONAL: a tclsh-only clean checkout completes packages + bake unchanged, with the step's absence reported rather than fatal; punkzip gains an explicit extract-to-directory argument and a machine-readable listing mode whose format is pinned by a test (the current fixed-width table stays the human default); punk::zip detects the accelerator and uses it, with a parity suite proving accelerated and pure-Tcl paths return identical listings and produce byte-identical extraction trees over the G-124 fixture set including the file-relative prefixed executable, and proving the pure-Tcl path is still taken when the binary is absent; a recorded benchmark on a representative kit payload states the speedup and the entry count at which it becomes material; the vendored tree records its upstream origin, the commit or state it was taken from, the re-vendor procedure, and its licensing (public-domain hwzip lineage plus MIT musl time.zig) per G-063 and the G-026 provenance direction.

Context

punkzip is a fork of hwzip.zig (Hadrien Dorio's zig port of Hans Wennborg's hwzip 2.1), maintained by the developer at c:/repo/jn/zig/punkzip and adapted for the case standard zip tools handle inconsistently: an executable or script concatenated with a zip whose offsets were never adjusted. Its README records the motivation - 7z and peazip can read and even edit catenated exezips with unadjusted offsets, but not when offsets have been rewritten file-relative, and behaviour varies with the file extension used. That is precisely the axis punkshell cares about (see G-124's Context and the archived G-122).

Measured 2026-07-26 with the existing bin/punkzip.exe (v2.1.0, Nov 2024, untracked - bin/*.exe is gitignored - and already stale against the 2.1.1 source):

• lists bin/runtime/win32-x86_64/tclsh90b4_piperepl.exe, the FILE-relative prefixed executable on which tcllib's decoder fails with "Bad zip file. Bad closure." - 846 entries, no error
• lists a 49MB kit (punk91.exe, 3629 entries) in 96ms whole-process, against 46ms for a native Tcl 9 zipfs mount plus recursive walk in-process
• extracts 892 files from the 1.4MB tclsfe tcl_library payload in 493ms from a split zip, and 490ms STRAIGHT FROM THE PREFIXED EXECUTABLE with no split step - against tcllib's 52ms parse plus 830ms unzip for the same set
• honest caveat: much of that 490ms is filesystem writes, which a pure-Tcl reader also pays. The tcllib comparison is inflated by its byte-oriented fileutil::decode framework, so the pure-Tcl-vs-punkzip gap will be smaller. The accelerator's value grows with entry count - the large-library-payload bake is the case that motivates it.

Zig-version feasibility, measured the same day: the source builds CLEAN under its declared zig 0.15.2 (zig build, exit 0, 2.0MB exe), which means the hard part - the 0.15.1 Writer/Reader redesign - is already absorbed. Under zig 0.16.0 it fails on one surfaced error, std.process.argsAlloc having been removed; 0.16 continues the same architectural move, threading an explicit Io through std.process/std.fs (Args+Iterator, currentPath(io,...), spawn(io,...), Io.Dir) and handing main an Init carrying args/environ/io. The blast radius is bounded: 17 std.process., 24 std.fs., 1 std.io., 1 std.os., 4 ArrayList and 8 writer/reader call sites, confined to 3 of roughly 20 files (punkzip.zig 715 lines, zipper.zig, punkzip_test.zig). The ~10k-line compression core (deflate, huffman, lz77, shrink, reduce, implode, bits, bitstream, tables, zip) is pure computation and untouched by the Io redesign. There are no external dependencies (.dependencies = .{}), and zig build test already runs ten test modules.

Toolchain is on hand: bin/tools/ already carries unpacked zig 0.16.0 alongside 0.15.1, 0.15.2 and 0.14.1 (bin/tools/zig is currently 0.15.2), with bin/punk-getzig.cmd for per-version fetch.

Upstream state at drafting: the 0.15.2 migration is IN PROGRESS and uncommitted (build.zig, build.zig.zon, src/punkzip.zig, src/punkzip_test.zig, src/zipper.zig modified; AGENTS.md, ZIG_IO.md, ZIG_ARRAYLIST.md, refactor_2026-01-19.txt untracked). Vendoring must take a deliberate, recorded upstream state rather than a working tree.

Approach

• VENDOR the source into punkshell (developer decision, 2026-07-26). It is small, dependency-free and public domain, so vendoring makes clean-checkout reproducibility trivial and removes any build-time reach outside the repo. The cost - a maintenance fork from c:/repo/jn/zig/punkzip - is accepted, and mitigated by recording the upstream origin and a re-vendor procedure beside the tree.
• Keep zig OPTIONAL. The tool build is its own make.tcl step, not part of bakehouse: a clean checkout with only tclsh must still build and bake, since making zig a prerequisite for an ordinary build would be a significant regression in what punkshell needs to bootstrap.
• punk::zip decides per call whether the accelerator is usable and silently falls back; the pure-Tcl floor from G-124 is never optional, and parity between the two paths is a test obligation rather than an assumption.
• Two small upstream-side changes are prerequisites for use as a build tool: extract currently writes to the current directory only (no target-directory argument), and there is no explicit machine-readable listing mode - the human table is parseable (banner lines prefixed # , fixed columns, name last) but nothing pins it.
• Port order is a work decision: porting upstream first and then vendoring the ported state keeps the fork boundary clean; vendoring first and porting in-tree gets the in-tree build gate working sooner. Record which was done and why.

Alternatives considered

• Build from the external checkout at c:/repo/jn/zig/punkzip (buildsuite-style fetch or a path reference) - rejected by developer decision: a clean checkout would depend on a machine-local path or a network fetch for a tool that is a few hundred KB of dependency-free public-domain source.
• Put it under src/buildsuites/ - not chosen: the buildsuites are the arm's-length factory for building EXTERNAL sources (Tcl runtimes) fetched per sources.config, with a fetch and test-gate contract that a vendored first-party tool does not need. Revisit only if the tool build grows a fetch step.
• Keep hand-building bin/punkzip.exe as today - rejected: the current binary is already a year stale against its own source, nothing in the repo can rebuild it, and its provenance is unrecorded.
• Build it as part of the default bakehouse path - rejected: makes zig a prerequisite for an ordinary build (see Approach).
• Use punkzip as the only zip reader, with no pure-Tcl path - rejected: it would make a build tool a hard dependency of module-level functionality, and would not help a kit running on a platform we have no punkzip build for. G-124 is the floor.

Notes

• Depends on: G-124 - the pure-Tcl floor and the fixture set this goal's parity suite reuses. This goal is additive to it and must not weaken it.
• Related: G-005 - zig-based build infrastructure for binary dependencies from source; this is a small, self-contained instance of that direction and the first zig build in the tree that is not a Tcl runtime.
• Related: G-004 - bin/*.exe is gitignored, so the built accelerator is an untracked local artifact, consistent with the no-committed-binaries direction: the source is committed and the binary is built.
• Related: G-063 - licence tracking for the vendored tree (public-domain hwzip lineage plus the MIT musl time.zig).
• Related: G-026 - provenance for the vendored copy (which upstream state, when, and how to re-vendor).
• Related: G-123 / G-006 - the other way to put a punkzip binary on a machine is to PUBLISH one and fetch it, riding the artifact-server and consent machinery those goals build, rather than requiring zig locally. Not chosen here (the source is tiny and dependency-free, and a build keeps bin/ binaries reproducible from committed source per G-004) but it is the natural extension if a no-zig fast path is ever wanted. Recorded from the drafting overlap survey.
• Related: G-105 - if cross-target punkzip builds are ever wanted, the toolchain and target-naming work lives there rather than here.
• Related: G-101 - an 8.6 container decision may make archive reading hot enough for the accelerator to matter to it.
• punkzip CLI as at drafting: list <zipfile>, extract <zipfile>, create [-m <method>] [-c <comment>] <zipfile> <files...>, plus an undocumented build subcommand using zipper.add_pathinfo_items. Compression methods store, shrink, reduce, implode, deflate - the legacy methods are implemented in-tree, so it reads archives zlib-based readers cannot.