# G-126 punkzip as a vendored zip accelerator: zig 0.16 port, reproducible build into bin/, punk::zip fast path

Status: active
Scope: src/tools/punkzip/ (vendored punkzip source tree - root name settled in the work - with provenance and licence records); src/make.tcl (tool build step); bin/tools/zig* (pinned toolchain as consumed); bin/punkzip.exe (untracked build output); src/modules/punk/zip-999999.0a1.0.tm (accelerator detection + fast path behind the G-124 pure-Tcl floor); src/tests/modules/punk/zip/ (parity suite - accelerated and pure-Tcl paths must agree); upstream maintained checkout c:/repo/jn/zig/punkzip (re-vendor source, read-only from this repo)
Goal: punkshell carries punkzip as vendored source, builds it from the repo's pinned zig toolchain, and uses the resulting binary as an optional accelerator for zip reading - with the pure-Tcl path as the always-available floor - so a machine with zig gets fast archive listing and extraction (materially so for bakes with large library payloads) while a clean checkout with only tclsh still builds and bakes exactly as before.
Acceptance: the vendored source builds under the repo's pinned zig 0.16 toolchain with 'zig build test' green (the port's own 10 test modules), driven by a make.tcl step that produces bin/punkzip.exe reproducibly on a clean checkout - and zig stays OPTIONAL: a tclsh-only clean checkout completes packages + bake unchanged, with the step's absence reported rather than fatal; punkzip gains an explicit extract-to-directory argument and a machine-readable listing mode whose format is pinned by a test (the current fixed-width table stays the human default); punk::zip detects the accelerator and uses it, with a parity suite proving accelerated and pure-Tcl paths return identical listings and produce byte-identical extraction trees over the G-124 fixture set including the file-relative prefixed executable, and proving the pure-Tcl path is still taken when the binary is absent; a recorded benchmark on a representative kit payload states the speedup and the entry count at which it becomes material; the vendored tree records its upstream origin, the commit or state it was taken from, the re-vendor procedure, and its licensing (public-domain hwzip lineage plus MIT musl time.zig) per G-063 and the G-026 provenance direction.

## Context

`punkzip` is a fork of hwzip.zig (Hadrien Dorio's zig port of Hans Wennborg's hwzip 2.1),
maintained by the developer at `c:/repo/jn/zig/punkzip` and adapted for the case standard
zip tools handle inconsistently: an executable or script concatenated with a zip whose
offsets were never adjusted. Its README records the motivation - 7z and peazip can read
and even edit catenated exezips with unadjusted offsets, but not when offsets have been
rewritten file-relative, and behaviour varies with the file extension used. That is
precisely the axis punkshell cares about (see G-124's Context and the archived G-122).

Measured 2026-07-26 with the existing `bin/punkzip.exe` (v2.1.0, Nov 2024, untracked -
`bin/*.exe` is gitignored - and already stale against the 2.1.1 source):

- lists `bin/runtime/win32-x86_64/tclsh90b4_piperepl.exe`, the FILE-relative prefixed
  executable on which tcllib's decoder fails with "Bad zip file. Bad closure." - 846
  entries, no error
- lists a 49MB kit (`punk91.exe`, 3629 entries) in 96ms whole-process, against 46ms for a
  native Tcl 9 zipfs mount plus recursive walk in-process
- extracts 892 files from the 1.4MB tclsfe tcl_library payload in 493ms from a split zip,
  and 490ms STRAIGHT FROM THE PREFIXED EXECUTABLE with no split step - against tcllib's
  52ms parse plus 830ms unzip for the same set
- honest caveat: much of that 490ms is filesystem writes, which a pure-Tcl reader also
  pays. The tcllib comparison is inflated by its byte-oriented `fileutil::decode`
  framework, so the pure-Tcl-vs-punkzip gap will be smaller. The accelerator's value
  grows with entry count - the large-library-payload bake is the case that motivates it.

Zig-version feasibility, measured the same day: the source builds CLEAN under its
declared zig 0.15.2 (`zig build`, exit 0, 2.0MB exe), which means the hard part - the
0.15.1 Writer/Reader redesign - is already absorbed. Under zig 0.16.0 it fails on one
surfaced error, `std.process.argsAlloc` having been removed; 0.16 continues the same
architectural move, threading an explicit `Io` through `std.process`/`std.fs`
(`Args`+`Iterator`, `currentPath(io,...)`, `spawn(io,...)`, `Io.Dir`) and handing `main`
an `Init` carrying args/environ/io. The blast radius is bounded: 17 `std.process.`, 24
`std.fs.`, 1 `std.io.`, 1 `std.os.`, 4 ArrayList and 8 writer/reader call sites, confined
to 3 of roughly 20 files (`punkzip.zig` 715 lines, `zipper.zig`, `punkzip_test.zig`). The
~10k-line compression core (deflate, huffman, lz77, shrink, reduce, implode, bits,
bitstream, tables, zip) is pure computation and untouched by the Io redesign. There are
no external dependencies (`.dependencies = .{}`), and `zig build test` already runs ten
test modules.

Toolchain is on hand: `bin/tools/` already carries unpacked zig 0.16.0 alongside 0.15.1,
0.15.2 and 0.14.1 (`bin/tools/zig` is currently 0.15.2), with `bin/punk-getzig.cmd` for
per-version fetch.

Upstream state: the drafting-time 0.15.2 migration (then in progress and uncommitted) was
completed, made fully green and committed 2026-07-27 as upstream e9b5b553, with the
localtime banner restored dependency-free in 78453fc0 - see Notes for the pre-activation
baseline record. Vendoring must take a deliberate, recorded upstream state rather than a
working tree; one now exists.

## Approach

- VENDOR the source into punkshell (developer decision, 2026-07-26). It is small,
  dependency-free and public domain, so vendoring makes clean-checkout reproducibility
  trivial and removes any build-time reach outside the repo. The cost - a maintenance
  fork from `c:/repo/jn/zig/punkzip` - is accepted, and mitigated by recording the
  upstream origin and a re-vendor procedure beside the tree.
- Keep zig OPTIONAL. The tool build is its own make.tcl step, not part of `bakehouse`:
  a clean checkout with only tclsh must still build and bake, since making zig a
  prerequisite for an ordinary build would be a significant regression in what punkshell
  needs to bootstrap.
- punk::zip decides per call whether the accelerator is usable and silently falls back;
  the pure-Tcl floor from G-124 is never optional, and parity between the two paths is a
  test obligation rather than an assumption.
- Two small upstream-side changes are prerequisites for use as a build tool: `extract`
  currently writes to the current directory only (no target-directory argument), and
  there is no explicit machine-readable listing mode - the human table is parseable
  (banner lines prefixed `# `, fixed columns, name last) but nothing pins it.
- Port order is a work decision: porting upstream first and then vendoring the ported
  state keeps the fork boundary clean; vendoring first and porting in-tree gets the
  in-tree build gate working sooner. Record which was done and why.

## Alternatives considered

- Build from the external checkout at `c:/repo/jn/zig/punkzip` (buildsuite-style fetch or
  a path reference) - rejected by developer decision: a clean checkout would depend on a
  machine-local path or a network fetch for a tool that is a few hundred KB of
  dependency-free public-domain source.
- Put it under `src/buildsuites/` - not chosen: the buildsuites are the arm's-length
  factory for building EXTERNAL sources (Tcl runtimes) fetched per `sources.config`, with
  a fetch and test-gate contract that a vendored first-party tool does not need. Revisit
  only if the tool build grows a fetch step.
- Keep hand-building `bin/punkzip.exe` as today - rejected: the current binary is already
  a year stale against its own source, nothing in the repo can rebuild it, and its
  provenance is unrecorded.
- Build it as part of the default `bakehouse` path - rejected: makes zig a prerequisite
  for an ordinary build (see Approach).
- Use punkzip as the only zip reader, with no pure-Tcl path - rejected: it would make a
  build tool a hard dependency of module-level functionality, and would not help a kit
  running on a platform we have no punkzip build for. G-124 is the floor.

## Notes

- Depends on: G-124 (achieved 2026-07-26 - see goals/archive/G-124-punkzip-reader.md) - the
  pure-Tcl floor and the fixture set this goal's parity suite reuses. This goal is additive
  to it and must not weaken it. What landed and what this goal must match:
  `punk::zip::archive_info` / `punk::zip::members` / `punk::zip::unzip` in punk::zip 0.2.0,
  with the member-dict keys the parity suite has to reproduce (name, isdirectory, size,
  csize, method, methodname, mtime, dostime, crc, offset, fileoffset, flags, encrypted,
  attributes, iattributes, madeby, hostsystem, version, comment) and the fixture set now
  built in `src/tests/modules/punk/zip/testsuites/zip/zipreader.test` (the three archive
  shapes from one source tree, plus hand-built raw zips for the refusal cases via that
  file's `rawzip` helper). Recorded pure-Tcl timing to beat: 841 members of the
  file-relative `tclsh90b4_piperepl.exe` extracted crc-verified in 735ms on Tcl 9.0.3.
- Related: G-128 - back-pointer added at the G-124 archive sweep (2026-07-26), where G-124
  was the pair's only recorded bridge. The overlap is direct and named in G-128's own
  Scope: it is the second vendored zig tool and shares THIS goal's make.tcl tool build step
  and pinned `bin/tools/zig*` toolchain, so the build step designed here has to serve both.
- Related: G-005 - zig-based build infrastructure for binary dependencies from source;
  this is a small, self-contained instance of that direction and the first zig build in
  the tree that is not a Tcl runtime.
- Related: G-004 - `bin/*.exe` is gitignored, so the built accelerator is an untracked
  local artifact, consistent with the no-committed-binaries direction: the source is
  committed and the binary is built.
- Related: G-063 - licence tracking for the vendored tree (public-domain hwzip lineage
  plus the MIT musl `time.zig`).
- Related: G-026 - provenance for the vendored copy (which upstream state, when, and how
  to re-vendor).
- Related: G-123 / G-006 - the other way to put a punkzip binary on a machine is to
  PUBLISH one and fetch it, riding the artifact-server and consent machinery those goals
  build, rather than requiring zig locally. Not chosen here (the source is tiny and
  dependency-free, and a build keeps bin/ binaries reproducible from committed source per
  G-004) but it is the natural extension if a no-zig fast path is ever wanted. Recorded
  from the drafting overlap survey.
- Related: G-105 - if cross-target punkzip builds are ever wanted, the toolchain and
  target-naming work lives there rather than here.
- Related: G-101 - an 8.6 container decision may make archive reading hot enough for the
  accelerator to matter to it.
- punkzip CLI as at drafting: `list <zipfile>`, `extract <zipfile>`,
  `create [-m <method>] [-c <comment>] <zipfile> <files...>`, plus an undocumented
  `build` subcommand using `zipper.add_pathinfo_items`. Compression methods store,
  shrink, reduce, implode, deflate - the legacy methods are implemented in-tree, so it
  reads archives zlib-based readers cannot.
- Pre-activation upstream baseline (user-directed, 2026-07-27): the maintained checkout
  `c:/repo/jn/zig/punkzip` now carries commit e9b5b553 "zig 0.15.2 migration complete:
  build + test green (122/122)" and 78453fc0 "restore localtime diagnostic banner,
  dependency-free, on stderr". `zig build test` under the pinned 0.15.2 toolchain runs
  all ten test modules - 122/122 tests green on Windows, including the previously
  mis-wired `bitstream_test.zig` (its 4 tests had never run anywhere: build.zig pointed
  test_bitstream at bits_test.zig) and the 13 exact-stdout CLI tests, whose expected
  outputs now PIN the current human table listing and creating:/extracting: wording.
  Port-order decision per Approach: UPSTREAM-FIRST, adopted in practice - port and
  stabilise upstream, then vendor the ported state.
- zeit decision (user, 2026-07-27): the zeit timezone dependency stays out - the
  localtime diagnostic returns dependency-free instead (Windows kernel32
  GetLocalTime/GetTimeZoneInformation with numeric offset + zone name; honest UTC line
  elsewhere), emitted on STDERR. Contract worth keeping: stdout is the parseable
  product, diagnostics go to stderr - the machine-readable listing mode this goal adds
  must preserve that split. Fallback if real tz fidelity is ever wanted: zeit v0.9.0 is
  zig-0.16-compatible, MIT, zero transitive deps - vendor it as a path dep (never a
  build-time URL fetch, which conflicts with this goal's no-network-build premise).
- Known upstream gaps carried into the goal work: the undocumented `build` subcommand
  remains unfinished scaffolding (wrong arg indexing, writes only an EOCD; its finalize
  call was restored at the checkpoint) - finish or remove it before vendoring. (The
  base_offset test gap noted here at activation was closed 2026-07-27 - see Progress.)
- TRAP for any zig-0.16 stdout-writing tool (applies to the G-128 tool too): msys/cygwin
  shells hand children OVERLAPPED (asynchronous) pipe handles, but zig 0.16's
  `Io.File.stdout()` hardcodes blocking flags and the `Io.Threaded` blocking write path
  hits `unreachable` on NtWriteFile PENDING once the 64KB pipe buffer fills - output
  truncates at ~64KB and the process dies mid-listing (exit code masked in pipelines).
  Fix pattern (landed upstream 2026-07-27): probe the real handle mode once via
  `NtQueryInformationFile(.Mode)` and set `flags.nonblocking` honestly so the Io
  implementation takes its correct alertable-wait path; console handles fail the probe
  and stay blocking. Anonymous CreatePipe pipes (Tcl exec, zig build test) are
  synchronous and unaffected.

## Progress

- 2026-07-27 upstream zig 0.16.0 port COMPLETE (commit 3b87087 on top of the e9b5b553
  checkpoint): floor moved 0.15.2 -> 0.16.0 (comptime gate + zon minimum_zig_version),
  `main(init: std.process.Init)` with Io threaded through all fs access in punkzip.zig +
  zipper.zig + punkzip_test.zig, version 2.1.3. Two load-bearing findings fixed en
  route: a build-graph race (run steps did not depend on the install step, so the CLI
  suite tested a stale zig-out binary under 0.16's scheduler) and the msys overlapped
  stdout trap recorded in Notes. Verified on Windows: `zig build test` green under
  0.16.0; 841 members CRC-verified extracted from the file-relative
  `tclsh90b4_piperepl.exe` in 0.5s; listing byte-identical through msys pipe and file
  redirect, and byte-identical to the old 2.1.0 binary's listing of the same kit.
- 2026-07-27 base_offset read path pinned upstream (commit 2e49c75): three container
  tests (`zip_prefixed_unadjusted_offsets`, `zip_prefixed_file_relative_offsets`,
  `zip_prefixed_empty`) build archives in memory, prepend a fake executable prefix,
  and verify both offset conventions land on the right base_offset with members
  extracting intact. Suite now 125/125.
- Remaining for acceptance: extract-to-directory argument (upstream); machine-readable
  listing mode + zig-side format pin test (upstream; must keep the stdout=product /
  stderr=diagnostics split per Notes); `build` subcommand finish-or-remove decision;
  vendor into `src/tools/punkzip/` with provenance (upstream commit id), re-vendor
  procedure and licence records; make.tcl tool build step (zig OPTIONAL - absence
  reported, not fatal; test gate keys on exit code, not clean stderr - the zip suite's
  negative tests print EOCD warnings while passing); punk::zip accelerator detection +
  fast path behind the G-124 floor; parity suite in `src/tests/modules/punk/zip/`;
  recorded benchmark (speedup + entry count where it becomes material).