punkshell/goals/G-124-punkzip-reader.md

G-124 Dependency-free zip reading in punk::zip

Status: proposed Scope: src/modules/punk/zip-999999.0a1.0.tm (reader + introspection procs, base-offset derivation factored out of extract_preamble for shared use, buildversion); src/make.tcl (zipfs-less kit extraction path as first consumer - the zipfile::decode requirement is removed, not made optional); src/tests/modules/punk/zip/ (new testsuite - mkzip round-trip, prefixed-archive cases, introspection, unsupported-input behaviour); bootsupport + src/vfs/_vfscommon.vfs copies via established sync channels Goal: punkshell can read a zip archive - including one prefixed by an executable, with either archive-relative or file-relative offsets - using only stock Tcl, and can do so as a documented utility a Tcl script programmer would reach for: list what is inside without extracting, extract all or part of it, on any tclsh with no zipfs, no vfs::zip and no tcllib. make.tcl's zipfs-less kit extraction stops depending on tcllib's zipfile::decode (absent from e.g msys2's /usr/bin/tclsh8.6) and stops depending on a split intermediate whose offsets can be wrong, and punk::zip's writer gains the round-trip verification it has never had. Acceptance: punk::zip extracts to a target directory from (a) a plain zip, (b) an executable-prefixed archive with archive-relative offsets and (c) one with file-relative offsets - bin/runtime/win32-x86_64/tclsh90b4_piperepl.exe is the recorded (c) case, on which the current split-then-tcllib path fails with "Bad zip file. Bad closure." - producing CRC-verified byte-identical members with directory entries materialized as directories, on a tclsh with no zipfs, no vfs::zip and no tcllib reachable (the interpreter used is recorded here); an introspection call lists members without extracting, returning per-entry name, size, compressed size, method, mtime, crc and directory-vs-file, over the same three input shapes, and both surfaces carry punk::args definitions with worked examples so 'i punk::zip::' documents them; unsupported inputs (zip64, encrypted, unknown compression method) fail naming the reason rather than emitting partial or garbled output; make.tcl's zipfs-less kit extraction consumes the new surface and NO zipfile::decode requirement remains anywhere in the shipped tree, verified by a tree grep, and demonstrated by an 8.6 zipfs-less bake of a zip kit on a tcllib-less tclsh (msys2 /usr/bin/tclsh8.6 as reference host) producing an artifact that boots with its tcl_library present; a new src/tests/modules/punk/zip/ suite covers a mkzip -> read round trip asserting names, content, directory-vs-file classification and stored attributes - recording the answer to the standing zipfs dir-misidentification question noted in punk::mix::cli and src/vfs/mkzipfix.vfs - plus the three prefixed-archive cases, the introspection output, and the unsupported-input failures; punk::zip buildversion minor-bumped with its changelog line.

Context

punk::zip is a write-plus-split module: walk, mkzip/Addentry, extract_preamble. Nothing in the tree can READ a zip without zipfs (Tcl 8.7+/9), vfs::zip, or tcllib's zipfile::decode. That last one is a SYSTEM package punkshell does not vendor, and it has exactly one consumer: make.tcl's zipfs-less kit-extraction fallback (shipped under G-122, achieved - see goals/archive/G-122-host-target-platform-split.md).

Measured 2026-07-26: msys2's /usr/bin/tclsh8.6 has no tcllib, so a bake driven from that host cannot lift the runtime's own zip (its tcl_library) into the kit and the artifact does not boot. The same tclsh, given a tcllib on its path, splits and decodes byte-identically to a native 8.6 host (baseoffset 5267968, 1429659 zip bytes, 1784 entries from both) - so the gap is purely the absent package, not an msys defect.

The dependency is not the only defect. extract_preamble carries #todo - if it was internal preamble - need to adjust offsets to fix the split off zipfile. That todo is live: an archive whose offsets are FILE-relative (the historical zipfs mkimg convention) splits into a .zip whose offsets still assume the removed preamble. bin/runtime/win32-x86_64/tclsh90b4_piperepl.exe is such a file (filerel_cdirstart == recorded_offset == 4085188); the current pipeline splits it correctly and tcllib then fails on the result with "Bad zip file. Bad closure.". A reader that works on the ORIGINAL file at the derived base offset never produces the broken intermediate - the failure mode disappears rather than needing offset-rewriting code. (Survey of the current store: every other runtime and every built kit is archive-relative, so this is a real but presently-unexercised shape.)

Reading has value well beyond the build. punk::zip is in the bootsupport manifest and ships in every kit (_vfscommon.vfs/modules/punk/zip-*.tm), so a reader there is available to any script the shell runs - and punkshell today has no answer at all for "what is inside this .tm modpod / kit / zip" on a runtime without zipfs.

Approach

• Factor the base-offset derivation out of extract_preamble into a shared private helper so the reader accepts a plain zip or a prefixed executable through one code path, and extract_preamble stops being the only thing that knows how to find the archive within a file.
• Walk ALL central-directory records. extract_preamble binary-scans only the first, with its own #todo! loop through all cdr file headers note. Sizes and offsets taken from the CD sidestep data descriptors entirely.
• Inflate with the stock primitives the writer already uses: zlib inflate / zlib stream inflate mirror Addentry's zlib deflate / zlib stream deflate, and zlib crc32 gives verification tcllib never performed (its CopyFile carries # FUTURE: Run crc checksum on created file). Mirror Addentry's existing 2MB threshold for choosing whole-member vs streaming.
• Support store and deflate; fail by name on zip64, encryption (GPBF bit 0) and unknown methods rather than emitting partial output.
• The introspection surface is the same CD walk without extraction, so listing costs no decompression - which is what makes it usable as a general scripting utility.

Alternatives considered

• Vendor tcllib's zipfile::decode (plus its fileutil::decode dependency) through G-065 - rejected: it imports a byte-oriented decode framework to obtain one capability, does not verify CRCs, cannot read a prefixed executable at all (the caller must split first, which is exactly where the file-relative bug lives), and leaves punk::zip's writer with no round-trip coverage. Cost for comparison, measured 2026-07-26: 830ms to extract 892 files that punkzip does in 490ms.
• Require zipfs, i.e. declare 8.6-without-tcllib unsupported - rejected: 8.6 is a real target (src/AGENTS.md), G-101's 8.6 container work has no zipfs by definition, and G-034 is an existing 8.6 read failure in the shell itself.
• Shell out to an external zip tool as the ONLY path - rejected as a floor, since it would make a build tool a hard dependency of module-level functionality. Adopted as an optional accelerator instead - see G-126.

Notes

• Related: G-034 - its Acceptance permits "an alternative modpod mount path" for the 8.6 code interp, which currently fails for want of vfs::RegisterMount. A dependency-free reader is that alternative: extract rather than mount.
• Related: G-101 - an 8.6 kit container has no zipfs by definition, so whatever container is chosen needs a reader that works without it. This goal is that primitive.
• Related: G-109 - manifest-declared discovery reads members out of zip-based .tm modules; the introspection surface here is the natural mechanism.
• Related: G-111 - modpod IS the zip .tm format and that goal establishes its first test baseline. The round-trip suite here is adjacent coverage of the writer, not a substitute for modpod's own.
• Related: G-065 - declarative vendoring is the mechanism the rejected alternative above would have used.
• Related: G-066 - lib.copyasmodule emits zip .tm modules; its converter testsuite could verify output with this reader.
• Related: G-110 - adjacent, not a dependency: it studies caching of EXTRACTED shared libs, where extraction today is done by zipfs/vfs::zip.
• Related: G-126 - the optional zig accelerator that sits behind this pure-Tcl floor.
• Related: G-125 - the deploy gate that turns an extraction failure into a failed kit rather than a deployed-but-unbootable one. Independent of this goal, which removes the most common cause of that failure.
• Related: G-122 (achieved) - shipped the zipfs-less assembly half and recorded the tcllib gap that motivates this goal; see goals/archive/G-122-host-target-platform-split.md.
• Test baseline as at drafting: punk::zip has NO tests (grep -rn "punk::zip" src/tests/ finds only log noise). The zipper modpod - a separate low-level writer - has four write-only smoke tests in src/tests/modules/zipper/testsuites/zipper/zipper.test (version, initialize, addentry, finalize-produces->100-bytes) with no content assertions. So nothing today verifies that anything punkshell WRITES is readable.
• Standing question the round-trip suite should settle: punk::mix::cli records that "punk::zip::mkzip stores permissions - (unix style) - which zipfs mkzip doesn't" and src/vfs/mkzipfix.vfs's copy records that this "confuses zipfs when reading - it misidentifies dirs as files". The reader makes that assertable in-tree for the first time; record the finding here.