# G-165 Driver-invariant kit assembly: single zip writer path with punkzip acceleration, file-relative by explicit opt-in only

Status: achieved 2026-08-04
Scope: src/make.tcl (zip-kit assembly branch); src/modules/punk/zip-999999.0a1.0.tm (mkzip write-path acceleration seam); src/tools/punkzip/ (re-vendor after upstream enabling changes); src/runtime/mapvfs.toml (per-kit offsetstyle opt-in); src/tests/ (accelerated/floor parity + driver-vintage characterization); src/AGENTS.md; bin/AGENTS.md
Goal: every make.tcl zip-kit bake emits an ARCHIVE-relative kit regardless of the driving interpreter (legacy file-relative mkimg in 8.7/9.0.0, fixed mkimg in 9.0.1+, or no mkimg at all) and produces equivalent kit zip payloads for a given merged tree across drivers, platforms and architectures - supporting bake servers running sizeable platform/content matrices: zip-kit assembly uses a single writer layer, punkzip (zig) when its binary is present with punk::zip::mkzip (pure Tcl) as the always-available floor, semantically equivalent, with tcl::zipfs::mkimg retired from the pipeline; file-relative output is available only as an explicit per-kit opt-in recorded in mapvfs.toml - the G-134 deliberate-creation carve-out expressed as configuration, never ambient driver behaviour.
Acceptance: a kit baked from the same source tree under all three driver classes on the reference machine - tclsh 8.7a6 (legacy file-relative mkimg, the reported failure case), tclsh90 9.0.x (fixed mkimg), and a mkimg-less tclsh (the zipcat-only class) - probes offsetstyle=archive via punk::zip::archive_info with identical member sets and member CRCs across the three runs; with bin/punkzip present the bake assembles via punkzip (visible in bake output) and a write-path parity suite proves member-set, CRC and offsetstyle equivalence between the accelerated and pure-Tcl paths over the G-124 fixture set plus a real .vfs payload, while a no-punkzip bake of the same tree produces an equivalent archive; a kit declaring offsetstyle="file" in its mapvfs.toml entry probes file - the opt-in recorded, and the offsetstyle.test baseline exempts declared kits - while G-134's BAKE-WARNING never fires on default (non-opt-in) bakes under any driver class; upstream punkzip gains the root-strip and repeatable-exclude options the bake's call shape requires (zig build test green, entries rooted at the payload tree with mkzip's default exclusion set honoured identically), re-vendored with PROVENANCE.md updated and tools-tier publication recorded or explicitly user-deferred; make.tcl's kit assembly path contains no tcl::zipfs::mkimg call; full punkexe + punkboot utils suites green in both runner modes; and src/AGENTS.md plus bin/AGENTS.md state the single-writer contract (consistent kits regardless of driving interpreter; file-relative by explicit opt-in only).

## Context

The zip-kit assembly branch in make.tcl selects its writer on the PRESENCE of
`tcl::zipfs::mkimg` in the driving interpreter (~line 9135), so the kit's zip
offset convention is a property of which tclsh drives the bake, not of the
pipeline. mkimg's convention changed upstream: pre-2021 archive-relative, then
file-relative (internal preamble) from 2021 until 2024-12-05, when the fix
argued for by the developer landed (Tcl ticket aaa84fbbc5, submitted
juliannoble2 with patch; fixed in 3d62ee540d by jan.nijtmans; the same change
retired the build system's `zip -A` step). Drivers therefore split by vintage:
8.7 alphas and 9.0.0 emit the legacy file-relative convention (measured on the
reference machine 2026-08-04: prefixed-mkimg probe under tclsh 8.7a6 ->
offsetstyle=file), 9.0.1+ and 9.1 emit archive-relative (same probe under
tclsh90 9.0.3 -> offsetstyle=archive), and mkimg-less drivers take the zipcat
concatenation path (archive-relative by mkzip default). An 8.7-driven bake
today produces file-relative kits: G-134's advisory BAKE-WARNING fires (by
design, but signalling "pipeline regression" when the cause is driver vintage),
and offsetstyle.test's _bake baseline goes permanently red for such users.
The kits themselves mount and boot on either convention (G-129 measured); the
material consumer is G-128's punkres stamper, which refuses file-relative
payloads by default because appending a PE section shifts the overlay.

User intent (stated 2026-08-04): punkshell produces consistent kits regardless
of the driving tclsh; file-relative kits are by explicit user
configuration/opt-in, not a default; legacy file-offset mkimg runtimes will
exist in the wild for many years. Server-driven bakes across a
platform/architecture/content matrix are in scope, so assembly throughput
matters beyond the single-bake case.

Write-path performance measured 2026-08-04 (reference machine; whole-tree
assembly of src/_bake/punk91.exe.vfs and a 3x duplication; all writers
C-zlib deflate with store-if-larger; output sizes within 1.5%):

| payload                    | mkimg (9.0.3) | punk::zip::mkzip (Tcl) | punkzip build (zig) |
|----------------------------|---------------|------------------------|---------------------|
| 132MB / 3,645 files        | 11.4s         | 8.1s                   | 6.7s                |
| 396MB / 10,935 files       | 35.9s         | 24.8s                  | 20.0s               |

Linear scaling in all three (mkimg's figure includes its unavoidable
per-member progress printing, also what a real bake emits). No wall short of
multi-GB payloads. Conclusion: the pure-Tcl floor is viable as the sole path;
punkzip acceleration is a matrix-scale dividend, not a correctness
prerequisite. (Scratch benchmark artifacts: temp zipbench dir, regenerable.)

punkzip (v2.3.1) gaps for kit-payload writing, measured the same day:
`build` carries the input path into entry names (no root-strip option -
entries must root at the payload tree) and has no exclude-pattern option
(mkzip's default exclusion set has no equivalent). Non-gaps already present:
sorted deterministic entries, directory entries (zipfs mounts by trailing
slash), comment flag, deflate + store-if-larger equivalence.

## Approach

Two workstreams, ordered by user decision 2026-08-04 (upstream first as the
enabling step):

1. Upstream punkzip (developer's repo c:/repo/jn/zig/punkzip, re-vendored per
   src/tools/AGENTS.md): add root-strip (entries rooted at a named base, the
   mkzip -base semantics) and repeatable exclude patterns; verify
   store-if-larger and determinism unchanged; version bump; zig build test
   green; tools-tier publication per the punkres route (G-123, achieved - see
   goals/archive/G-123-thirdparty-runtime-tiers.md) or explicit user
   deferral.
2. punkshell: a write-path acceleration seam inside punk::zip::mkzip mirroring
   G-126's read-path pattern (detect bin/punkzip -> shell out with the
   equivalent strip/exclude arguments; pure-Tcl path unchanged as the floor,
   with parity as the contract), so all mkzip consumers accelerate, not just
   the bake; make.tcl's kit assembly retires the tcl::zipfs::mkimg branch in
   favour of the single zipcat compose point (runtime prefix concatenated
   outside the writer, as assemble_zipcat_image already does); per-kit
   offsetstyle key in mapvfs.toml (exact key name settled at implementation;
   opt-in kits always take the floor with -offsettype file, which mkzip
   already supports - punkzip needs no offset-adjustment scope);
   offsetstyle.test's baseline reads the mapping and exempts declared kits;
   G-134's pin and advisory posture stay, its warning text accurate again
   (file means opt-in or genuine regression, never driver vintage); docs in
   src/AGENTS.md (G-122/G-134 bullets) and bin/AGENTS.md.

## Alternatives considered

- Normalize old-mkimg output to archive-relative post-assembly (conversion) -
  rejected: keeps two assembly paths and requires the parked in-place
  conversion machinery (G-128's open follow-on); manages the sensitivity
  rather than removing it.
- Probe the driver's mkimg convention once per bake and specialize
  warnings/test gating - rejected for the same reason: driver-invariant
  output is the intent, not driver-aware messaging.
- Retire the mkimg branch WITHOUT the punkzip acceleration layer - viable on
  the measured numbers (the floor beats mkimg at both measured sizes); the
  acceleration layer was user-directed for the bake-matrix case and adopted.
- Acceleration seam in make.tcl (bake-only) rather than inside punk::zip::mkzip -
  rejected: the mkzip seam accelerates every consumer with one parity
  contract and mirrors G-126 exactly.
- Bake CLI flag for the file-relative opt-in - rejected: session-scoped;
  the per-kit mapvfs declaration is durable configuration (user decision,
  opt-in shape (a), 2026-08-04).

## Progress

- 2026-08-04 (workstream 1, upstream punkzip - LANDED): root-strip
  `build -b <base>` (mkzip -base semantics: entry names rooted at base,
  base's own entry not stored, fileless directories pruned) and repeatable
  `build -x <pattern>` (Tcl string match against the entry name; '*'
  crosses '/', '?', '[...]' ranges, no class negation - verified empirically
  against tclsh) landed upstream as v2.4.0, committed 2026-08-04 as
  c:/repo/jn/zig/punkzip 2599127fa98dae2df8e9ccf3647f51c08dc8f3c0 (user's
  commit word: "Commit upstream punkzip as necessary to do this work").
  Entry name and disk path split (MemberPathInfo.diskpath); add_entry opens
  by disk path. Tests: new zipper_test.zig (5 unit: glob matcher,
  rel_to_base, prune) plus 5 new CLI tests and the usage pin update;
  `zig build test` exit 0, all suites green standalone (test_punkzip 23/23,
  test_zipper 5/5; 140/140 total). Differential probe against the pure-Tcl
  floor: mkzip (default exclusion set) vs punkzip -b/-x produced identical
  member names, CRCs, directory flags and method choices over a fixture
  covering all five default patterns at two depths plus empty dirs
  (compressed bytes within 1 byte - legitimate deflate implementation
  difference). Diagnosed and documented in upstream AGENTS.md a pre-existing
  zig 0.16.0 windows test-runner teardown artifact ("failed command ...
  --listen=-" with overall exit 0; gate is the exit code per
  src/tools/AGENTS.md). Re-vendored into src/tools/punkzip/ the same day per
  PROVENANCE.md (vendored-state line, lineage, and the src/tools/AGENTS.md
  140/140 count updated); vendored tree `zig build test` exit 0;
  `make.tcl tool build punkzip` test gate passed and installed
  bin/punkzip.exe v2.4.0 (ReleaseSafe; smoke-verified: usage shows -b/-x and
  the deployed binary reproduces the floor's parity member set).
- 2026-08-04 (workstream 2, punkshell - LANDED, all acceptance evidence in):
  (a) write-path acceleration seam inside punk::zip::mkzip mirroring G-126
  (walk stays authoritative for member set and returned names; exec punkzip
  build -b <base> -x <pattern>...; produced archive read back and member
  names compared against the walk result; any failure/unreadable/mismatch
  deletes the file and falls back silently; -offsettype file and
  -runtime/-zipkit prefixes always floor; diagnostics
  last_write_engine/last_write_note) - punk::zip 0.4.0, bootsupport-promoted.
  (b) make.tcl: the tcl::zipfs::mkimg branch retired (zip kits always
  assemble via assemble_zipcat_image, itself routed through punk::zip::mkzip
  with the project's bin/punkzip.exe pointed at the write seam - the driving
  tclsh is not beside it); G-134 warning now fires only on UNDECLARED
  file-relative output and exempts declared kits; workflow text updated;
  bake/bakehouse -force flag added (user-endorsed 2026-08-04: bypasses the
  'no change detected' skip - used for the per-driver forced re-assembly);
  punkkit-stamp.toml now written with pinned LF translation (the default
  text translation followed the driver - crlf vs lf - making the stamp's
  bytes driver-dependent; caught by the four-driver parity run's only
  initial CRC diff). (c) mapvfs.toml: offsetstyle key documented in the
  header spec and validated in entry_reader ("file" only, zip-type only,
  entry-named errors); [kit.punkfiledemo] declared specimen
  (bake_default=false). (d) offsetstyle.test baseline reads the declared set
  from mapvfs.toml and exempts declared kits (asserting they DO probe file)
  - 7/7. (e) docs: src/AGENTS.md (G-122 single-writer bullet, G-134
  opt-in-or-regression bullet), bin/AGENTS.md (G-129 emit-vs-mount note,
  G-134 opt-in exemption), src/runtime/AGENTS.md (offsetstyle key).
  (f) zipwriteaccel.test (14/14; #added 2026-08-04 (agent, G-165)):
  accelerated-vs-floor parity over the G-124 tree shape, an exclusion-heavy
  tree (default set honoured identically), custom excludes, comment,
  floor-when-absent, ineligible-call floors, failure fallback, return
  contract, and the committed _vfscommon.vfs payload (65.8MB); method choice
  on already-compressed members documented engine-dependent (crc/name/isdir
  are the contract).
  ACCEPTANCE RUNS (all on the reference machine): punkluck86 baked under
  tclsh 8.7a6 (legacy mkimg), tclsh90 9.0.3 (fixed mkimg), tclsh86 8.6.17
  (native, mkimg-less) and msys2 /usr/bin/tclsh8.6 8.6.12 (unix platform,
  mkimg-less - the G-122 class): all four probe offsetstyle=archive with
  IDENTICAL member sets and CRCs (3319 members), each visibly assembled via
  the accelerated seam, zero offsetstyle BAKE-WARNINGs; a no-punkzip bake
  produced an equivalent archive (floor path); punkfiledemo probes file with
  no warning (declared); stale file-relative punk9_beta (pre-G-165 8.7
  artifact) rebaked to archive, clearing the offsetstyle baseline.
  Suites: zip 53/53 (tclsh90), punkboot utils 81/81 both runner modes,
  punkexe 146/150 (4 constraint skips, 0 failures) both runner modes,
  modpod roundtrip 7/7, maketclhelp 12/12 (bakehouse synopsis pin updated
  for -force). punkproject 0.52.0 + CHANGELOG; projectversion check OK.
- 2026-08-04 (tools-tier publication - user's word "publish"): punkzip
  2.4.0 published to the punkbin tools tier as
  win32-x86_64/tools/punkzip-x86_64-windows-2.4.0.exe (sha1
  2008b51ffa26e31faa0e76ea1e229ee1cec16395, recorded in the tier's
  sha1sums.txt via src/build_sha1sums.tcl - NEW, no CHANGED), punkbin git
  a8303c4, pushed to the canonical origin. Non-windows cross-builds remain
  a follow-on (see Follow-ons). This satisfies the acceptance's final item
  (publication recorded - not deferred).

## Notes

- Related: G-122 (achieved - see goals/archive/G-122-host-target-platform-split.md) -
  introduced the zipcat concatenation fallback (assemble_zipcat_image) this
  goal makes the only assembly path.
- Related: G-124 (achieved - see goals/archive/G-124-punkzip-reader.md) - the
  punk::zip reader/archive_info classification instrument used by the G-134
  pin and this goal's acceptance; its bootsupport >= 0.2.0 extraction
  requirement is long satisfied.
- Related: G-126 (achieved - see goals/archive/G-126-punkzip-accelerator.md) -
  the read-path acceleration pattern (detect binary -> fast path, pure-Tcl
  floor, parity suite) mirrored for writes; the vendored-tool build step and
  provenance/publication route this goal reuses.
- Related: G-128 (achieved - see goals/archive/G-128-portable-pe-resource-stamping.md) -
  the stamper whose default refusal makes archive-relative the load-bearing
  convention. Its open in-place-conversion follow-on is NOT enacted here
  (mkzip -offsettype file serves the opt-in) and stays open.
- Related: G-129 (achieved - see goals/archive/G-129-kitboot-mountpoint-derivation.md) -
  runtime zipfs requirements; both offset conventions mount on both verified
  runtime families, which is why driver-vintage kits booted despite being
  off-convention.
- Related: G-134 (achieved - see goals/archive/G-134-baked-kits-pinned-archive-relative.md) -
  the pin this goal makes driver-independent by construction; its
  deliberate-creation carve-out (user decision 2026-07-27) becomes the
  per-kit mapvfs opt-in.
- Related: G-131 (live) - deliberate non-overlap: container autodetection
  (metakit or zipfs) is orthogonal to zip offset convention; the single-writer
  change touches only the zipfs kit path and must not disturb G-131's metakit
  branches or its make.tcl check reconciliation.
- Related: G-158 (live) - sibling surface: the 8.6 'kit' (metakit/sdx) lane is
  a different container family, untouched here.
- Related: G-005/G-006 (live) - binary build/retrieval infrastructure; the
  bake-server punkzip distribution rides the tools-tier route (punkres
  precedent), G-105-family cross-builds for non-windows bake servers.
- Related: G-162 (live) - sibling bake-surface goal: its content-digest
  freshness stamps meet this goal's writer swap inside make.tcl's
  bake/bakelist path. Classified orthogonal at activation: the digests run
  over bake inputs, which this goal does not change (the writer layer is not
  an input) - re-check if G-162 ever digests bake OUTPUTS, since the
  accelerated and floor writers are member-set/CRC-equivalent, not byte-pinned.
- Related: G-142 (live) - punkbin curated listing manifests: the tools-tier
  publication this goal's workstream 1 rides (punkzip cross-builds for
  non-windows bake servers, a follow-on candidate) would surface artifacts in
  punkbin, whose per-tier listing manifests are G-142's subject.
- Overlap survey 2026-08-04 (drafting): `goals_xref.tcl paths src/make.tcl
  src/modules/punk src/tools/punkzip src/runtime/mapvfs.toml` - no live goals
  intersect the first three; mapvfs.toml surfaces path-mention hits (G-131,
  G-158, G-005, G-006, G-023, G-141), all read and classified orthogonal to
  offset convention (container type, kit naming, libfetch target selection,
  8.6 kit parity). `goals_xref.tcl report` unlinked-pair list reviewed: no
  pair intersects this goal's scope.
- Activation survey 2026-08-04 (proposed -> active flip, user-confirmed):
  `goals_xref.tcl score G-165` re-run at activation; new Related lines
  recorded for G-162 and G-142 (above). One-directional pairs (G-158, G-131,
  G-141, G-105, G-023, G-005, G-006) already carry this goal's pointer from
  drafting - back-pointers optional, not added. Remaining top unlinked pairs
  (G-137, G-073, G-147, G-157, G-034) reviewed and classified lexical noise
  (bin/AGENTS.md adjacency, generic term overlap). goals_lint clean after
  the flip (82 active-index, 83 archived).
- The offsetstyle.test baseline's premise ("today's bakes produce no warning")
  was environment-dependent when written (the reference driver has fixed
  mkimg); this goal makes it driver-independent again - the baseline stays a
  true tripwire once opt-in kits are exempted by declaration.
- mkzip's per-member punk::args::parse in Addentry was profiled as a suspected
  hot spot before the benchmark; measured all-in per-member cost ~2.2ms at
  3.6k members - not a blowup, left as-is (floor stays simple and correct).
- The same vintage sensitivity exists one level down in the buildsuites'
  runtime wrapper (src/buildsuites/suite_tcl90/tools/zipfs_mkimg.tcl): a
  runtime built with a legacy-mkimg driver comes out file-relative (the
  store's tclsh90b4_piperepl.exe is exactly such an artifact). Tolerated on
  input by the G-124 reader; recorded here as hygiene, not in this goal's
  scope - a candidate follow-on if runtime-side consistency is later wanted.

## Follow-ons

Follow-on: tools-tier cross-builds of punkzip for non-windows bake servers (linux/macosx/freebsd and other arches; the G-105-family cross-build route, punkres publication precedent) => open
Follow-on: buildsuites runtime-side zipfs_mkimg.tcl wrapper carries the same mkimg vintage sensitivity (a runtime built with a legacy-mkimg driver comes out file-relative; tolerated on input by the G-124 reader) - hygiene, runtime-side consistency if ever wanted => landed 2026-08-05 (src/buildsuites/suite_tcl90/tools/zipfs_wrap.tcl: payload written by the driving shell's builtin tcl::zipfs::mkzip - convention-stable across mkimg vintages, probed 8.7a6 vs 9.0.3 - with the prefix executable concatenated outside the writer, plus an EOCD archive-relative self-check per wrap; build905.zig wrap lane renamed zipwrap_*, suite README updated; suite make-zipfs/smoke/kit-family green, all 5 products probe archive-relative and boot)