punkshell/goals/G-128-portable-pe-resource-stamping.md

G-128 Portable PE resource stamping

Status: active Scope: src/tools/punkres/ (vendored zig source - tool name settled in the work - with provenance and licence records); src/make.tcl (mechanism selection at the G-057 seam, sharing G-126's tool build step); bin/tools/zig* (pinned toolchain as consumed); bin/punkres.exe (untracked build output); src/tests/ (characterization - icon replaced, payload intact, idempotent, overlay-unsafe refusal); TEMP_REFERENCE/tcl-sfe (read-only reference for the resource structures) Goal: punkshell can stamp windows resources into a FINISHED kit executable from any build host - no twapi, no windows host, no disturbing the appended vfs payload - so a cross-target win32 kit is not condemned to its runtime's icon, and resource stamping becomes a post-hoc operation on a built artifact rather than something wired into the middle of the kit wrap. G-057's mechanism seam selects this implementation when it is present and keeps its twapi path otherwise. Acceptance: the vendored tool, built by the same make.tcl tool step G-126 establishes, replaces RT_ICON/RT_GROUP_ICON in a pre-built PE that already carries an appended zip overlay, for the kit shapes we build (kit, zip, zipcat per mapvfs.config): the new icon is confirmed by reading the resources back with the tool ITSELF (so verification does not require a windows host or twapi) and cross-checked once on windows with twapi::extract_resources; the stamped kit still boots to a working punk shell reading its payload; re-stamping converges with no resource accumulation; it works where .rsrc is NOT the last section - the shape our own runtimes have (tclsfe-x64/punk91: .text .rdata .data .pdata .rsrc .reloc; suite-built tclsh9.0.5-punk: .text .rdata .buildid .data .pdata .tls .rsrc .reloc) - by appending a section and repointing the resource data directory rather than growing .rsrc in place; the appended payload survives the file-offset shift this causes, which is safe for an ARCHIVE-relative zip and not for a file-relative one; a file-relative payload is REFUSED BY DEFAULT with an error naming the reason and the rewrap remedy, but an explicit consent flag (tool CLI and the G-057 seam, never an interactive prompt in non-interactive runs) stamps it by shifting the overlay AND rewriting every central-directory local-header offset plus the EOCD directory offset by the shift delta - deliberately PRESERVING the file-relative convention, with a notification that the format is non-standard - verified by re-reading the stamped artifact (offsetstyle still file, members crc-verified); zip64 and multi-disk archives are refused even with consent; one end-to-end run from a NON-WINDOWS host produces a stamped win32 kit whose icon a windows machine confirms (recorded here: which host); G-057's seam selects the tool when present and falls back to its twapi path otherwise, with a parity check showing both mechanisms yield equivalent resource content from the same input, and zig stays optional - with no tool built, G-057 behaves exactly as it does without this goal; the tool's distribution is settled and recorded - buildable from the vendored source with the pinned toolchain AND publishable as a punkbin artifact through the G-123/G-006 channels, so a no-toolchain user has a route that does not depend on the committed twapi; the tool's structure handling is written so the parked RT_VERSION stamping needs no second tool; licensing and upstream provenance of the vendored tree are recorded per G-063 and G-026.

Context

G-057 embeds icons with twapi, which is the right mechanism to ship first (demonstrated prior art in tcl-sfe, no new build infrastructure, and the vendored twapi makes it work on a fresh windows checkout with nothing built). It has two limits that are structural rather than incidental:

• Host-bound. twapi is a Windows extension, so a windows-target kit cross-baked from a linux or FreeBSD host cannot be stamped at all. G-122 made cross-target bakes routine and G-127 makes them first-class, so this stops being hypothetical.
• Ordering-bound. A Windows resource update rewrites the PE image and corrupts an already-appended vfs payload, which is why G-057 must either icon the stub BEFORE appending or do sfe's split/update/reattach dance. Stamping cannot be applied to a finished kit.

Feasibility measured 2026-07-26. The zig toolchain we already vendor carries both halves of the problem's prior art: zig rc is resinator, a full Windows RC compiler bundled at lib/compiler/resinator/, which is the reference for building correct RT_GROUP_ICON / RT_ICON structures from an .ico image directory; and std.coff parses PE. Note what resinator is NOT: a compiler produces a .res blob for a LINKER to embed into an executable it is building. Our runtimes are pre-built third-party binaries we never link, so the patch-an-existing-image half is the work this goal owns.

Section layout of our actual runtimes, via llvm-objdump:

tclsfe-x64.exe / punk91.exe    .text .rdata .data .pdata .rsrc .reloc
tclsh9.0.5-punk.exe            .text .rdata .buildid .data .pdata .tls .rsrc .reloc

.rsrc is not last - .reloc follows it - so growing it in place is unavailable and the implementation must append a section and repoint IMAGE_DIRECTORY_ENTRY_RESOURCE (the approach rcedit takes). That shifts the overlay's file offset, which is exactly the archive-relative vs file-relative distinction that G-122 and G-124 keep surfacing: an archive-relative zip survives being moved because its internal offsets are relative to its own start, a file-relative one does not. punkshell's own kits are archive-relative (measured: punk91.exe, preamble 5267968), so post-hoc stamping is viable for them - and the unsafe case must be detected and refused, not attempted.

The payoff is therefore larger than portability: a tool that treats the overlay as a first-class thing to preserve removes G-057's ordering constraint entirely, which is what the parked RT_VERSION stamping wants, since the project version is known late.

Approach

• Vendor and build through the pattern G-126 establishes - same tool build step, same optional-zig posture, same provenance and licence recording. This is the second instance of that pattern, not a new one.
• Read-back is part of the tool, not an afterthought: without it the acceptance cannot be checked from a non-windows host, which is the whole point.
• Refuse rather than risk: a payload whose offsets would break if the archive moves is a hard error naming the reason.
• Build the resource structures once, general enough that the parked RT_VERSION work extends the same tool rather than starting another.

Alternatives considered

• Shell out to rcedit - rejected: a third-party Node/C++ binary, against the vendored-source direction, and it would be another committed or fetched executable with no in-tree provenance.
• Keep twapi as the only mechanism - rejected as the END state: it leaves cross-host kits icon-less and keeps stamping wired into the middle of the wrap. Note it is NOT rejected as a path - G-057's twapi arm is explicitly permanent, and this goal must not retire it.
• Build resources at link time - impossible here: we do not link the runtimes, we receive them as finished binaries.
• Write it in Tcl instead - rejected: PE section surgery on multi-megabyte images is exactly the file-format work the zig tooling direction exists for, and unlike the punk::zip reader (G-124) there is no module-level need that would justify a pure-Tcl floor.

Notes

• Depends on: G-057 (achieved 2026-07-29, archived - see goals/archive/G-057-kit-icon-embedding.md) - supplies the mechanism seam this plugs into, and stays the no-toolchain windows path. This goal adds a mechanism; it does not remove one. The seam now EXISTS: ::punkboot::kit_icon_process in src/make.tcl is the single entry point with mechanism selection inside it (::punkboot::kit_icon_embed_twapi is the twapi arm), the per-kit <kitname>.resources.toml sidecar records embedding/status/reason (bin/AGENTS.md documents the format), and the cross-host skip notice names this goal as the remedy - the WSL linux cross-bake of a win32 kit (2026-07-29) is the live demonstration of the gap this goal closes. Note the bake seam stamps stub-first (pre-payload); this goal's post-hoc split/update/reattach operates on FINISHED kits, so its integration point can be either the same seam (replacing the twapi call for cross-host bakes) or a standalone post-hoc invocation - both stay substitutions behind kit_icon_process.
• Depends on: G-126 (achieved 2026-07-27 - see goals/archive/G-126-punkzip-accelerator.md) - the vendored-zig-tool build step, optional-zig posture and provenance pattern are shared rather than duplicated. Everything this goal needs from it now EXISTS: vendor under src/tools// with a PROVENANCE.md (record format + re-vendor procedure per src/tools/AGENTS.md), declare the zig floor in build.zig.zon minimum_zig_version, and make.tcl tool discovers/builds/tests/installs it with zero make.tcl edits. Traps recorded in the archived detail worth reading before the port: zig build run-steps need their own install-step dependency, msys pipes hand OVERLAPPED stdout handles (probe pattern in punkzip's stdout_file), and 0.16 Io openFile defaults allow_directory=true (breaks file-vs-dir probes) and cannot stamp directory timestamps (direct SetFileTime route).
• Related: G-123 / G-006 - publishing the built tool as a punkbin artifact is the route that gives a no-toolchain user portable stamping without depending on the committed twapi, which is what makes the G-004 tension resolvable rather than permanent.
• Related: G-004 - the tool is built or fetched, never committed; and it is the mechanism by which the vendored twapi could eventually stop being load-bearing.
• Related: G-127 - a win32 kit cross-baked from a non-windows host is precisely the case twapi cannot serve; this is what makes those kits complete rather than icon-less.
• Related: G-124 (achieved 2026-07-26 - see goals/archive/G-124-punkzip-reader.md) - shares the archive-relative vs file-relative offset distinction: that goal reads a zip at a derived base offset, this one moves one and must not break it. It leaves this goal a ready-made instrument for both halves of that requirement: punk::zip::archive_info <file> answers "is this artifact safe to move the overlay of?" in one call (offsetstyle archive vs file, plus dataoffset), and re-reading the stamped artifact with punk::zip::members/unzip proves the payload survived. Survey recorded there 2026-07-26 over the whole runtime store: 15 archive-relative, 1 file-relative (tclsh90b4_piperepl.exe), 4 with no attached zip.
• Related: G-023 / G-025 / G-117 (achieved) - the parked RT_VERSION stamping this tool is structured for would carry the project version and become a further provenance surface; G-057's Notes already require those to be reconciled into one story.
• Related: G-028 - a locked kit exe blocks a resource update exactly as it blocks deploy.
• Related: G-063 / G-026 - licence and provenance recording for the vendored tree.
• Recorded 2026-07-27 (user decision): a file-relative payload is no longer a straight refusal - default refuse with a consent-flag preserve, per the amended Acceptance. Rationale: the format works and serves naive zip-tooling/SFX-repair interop and external zip pipelines that treat recorded offsets as file positions. The consented path costs one central-directory offset fix-up pass (the shift delta added to every local-header offset and to the EOCD directory offset) - no decompression, member data untouched; a zip digital-signature record carries no offsets and survives. In-place CONVERSION between conventions falls out of the same machinery with the delta direction reversed - parked, not part of this goal.
• Stamping invalidates any Authenticode signature regardless of zip convention: the certificate table is located via the PE security directory (a file offset) and the image hash covers the appended payload - sign AFTER stamping; the sign-last rule is orthogonal to the archive/file-relative question.
• Related: G-134 - pins make.tcl's own kit outputs archive-relative (advisory probe), so a default-refusal case reaching the stamper reads as a provenance signal (the artifact did not come through the bake pipeline), not a pipeline regression.
• Codepage finding (2026-07-29): the IMAGE_RESOURCE_DATA_ENTRY CodePage field is host-locale NOISE under the twapi arm - Windows EndUpdateResource rebuilds the whole resource section stamping the host ANSI codepage (1252 here) on every entry INCLUDING untouched types (observed: RT_VERSION/RT_MANIFEST of the zig-built luck stub flipped 0->1252 under twapi). punkres instead inherits the codepage of the entries it replaces (deterministic, input-derived; 0 for a stub with no prior icons) and preserves untouched types byte-exactly. Parity is therefore full listing identity on MSVC-lineage stubs (tclsfe family, 1252) and a codepage-only metadata divergence on zig-built stubs (punkres 0 vs twapi host-ACP) - image bytes identical in all cases.
• Environment finding (2026-07-29): the twapi arm is environmentally fragile on this tree - the ROOT vendorlib_tcl9/ tier has no win32-x86_64/ subfolder, so neither the Git tclsh nor the native tclsh9.0.5-punk could load the vendored twapi during a bake (it loads fine when TCLLIBPATH points at src/vendorlib_tcl9/win32-x86_64). punkres embeds regardless of the driving tclsh, which is precisely the host-independence this goal exists for.
• Nested-payload finding (2026-07-29): raw tclsfe stubs carry their OWN tcl_library zip payload, so a zipcat construction over a raw stub yields zip-after-non-zip-overlay - the derived archive base sits after the overlay start and moves by exactly the shift delta. punkres handles this (verify invariant base2 == base1 + delta) and reports the leading-byte distance via an overlay_note line.
• Related: G-105 - recorded at activation (2026-07-29 overlap survey): the end-to-end non-windows-host acceptance clause needs a host-native punkres build; cross-target zig builds and target naming live in G-105 rather than here (the same division G-126 recorded for punkzip), and its WSL-first verification environment is the natural host for this goal's e2e run.
• Related: G-101 - recorded at activation (2026-07-29 overlap survey): the 'kit' shape named in this goal's acceptance is today the sdx/metakit container whose Tcl 8.6 future G-101 decides. A metakit payload is not a zip, so POST-HOC stamping of that shape falls to the opaque-overlay taxonomy; stub-first stamping at the G-057 seam is shape-independent either way. MEASURED 2026-07-29 (see Progress): the metakit attachment is MOVE-SAFE - punksys.exe stamped post-hoc via -allow-opaque-overlay boots and loads punk::lib from the moved payload (metakit self-locates from the file end), so the consent flag serves the kit shape today.

Progress

• 2026-07-29 increment 1 (activation day): the tool EXISTS, is vendored, and the seam selects it. Landed:
  • UPSTREAM punkres v0.1.0 at c:/repo/jn/zig/punkres (git; gitea remote recorded in the vendored PROVENANCE.md; commits 8e47767 initial, 09d5ec5 codepage-inheritance + verify fix). zig 0.16.0 floor (comptime gate + zon). Modules: pe.zig (PE32/PE32+ parse, overlay boundary, MS checksum, append-vs-rebuild placement with convergence), rsrc.zig (generic 3-level resource tree parse + deterministic sorted serialize - RT_VERSION needs no second tool), ico.zig (.ico <-> group/icon resources, reassembly), overlay.zig (EOCD/CD analysis, derived-base archive/file detection, zip64/multi-disk refusal, file-relative offset fix-up), synthpe.zig (all test fixtures synthesized in memory - no committed binaries). CLI: info/list (pinned porcelain v1), extract-ico, set-icon with BUILT-IN read-back verification; exit taxonomy 0 ok / 1 error / 2 hard refusal (zip64, multi-disk, odd signature) / 3 consent-liftable refusal (-allow-file-relative-shift, -allow-opaque-overlay, -strip-signature). twapi-arm-equivalent semantics (all icon/group entries deleted, ids 1..N under first pre-existing group's name+lang, fallback 1/1033). msys overlapped-stdout probe carried from punkzip preemptively. zig build test 31/31 across 5 suites.
  • VENDORED src/tools/punkres @ upstream 09d5ec5 with PROVENANCE.md (origin, re-vendor procedure, G-063 licence records: BSD-2-Clause first-party); src/tools/AGENTS.md indexed. make.tcl tool step discovered it with ZERO make.tcl edits: test gate + ReleaseSafe build installed bin/punkres.exe.
  • SEAM (G-057) wired in src/make.tcl: kiticon::punkres_available (memoized; binary presence = selection signal) + kit_icon_embed_punkres; selection = punkres when built (any host), else twapi, else skip with a COMBINED notice naming both gaps and the build remedy; sidecar embedding value punkres|twapi|none (bin/AGENTS.md + ARCHITECTURE.md updated). Live-bake fallback matrix on punkluck86 (win32-ix86 - the PE32 arm live): punkres present -> "via punkres" embedded/sidecar punkres/kit boots + smoke OK; punkres absent + twapi loadable (TCLLIBPATH) -> "via twapi"; both absent -> unavailable + combined reason + embedding none (exact pre-G-128 degradation). Live-bake parity: twapi-baked vs punkres-baked punkluck86 listings identical except the codepage-noise finding (Notes); extracted icons byte-identical.
  • VERIFIED on real artifacts (this windows host):
    • punk91.exe copy (49MB, 6-section shape, 3634-member archive-relative zip): post-hoc stamp, overlay byte-identical at new offset, kit BOOTS and package-requires punk::lib from the payload, re-stamp byte-identical (convergence), FULL twapi parity - listing identity, extract-ico byte-identical between mechanisms AND to the source punkshell.ico, twapi::extract_resources cross-check identical (8 resources, run inside the stamped kit itself).
    • tclsh9.0.5-punk.exe copy (8-section .buildid/.tls suite shape, 1615-member attached image): stamped, boots.
    • tclsh90b4_piperepl.exe copy (FILE-relative, 841 members): refused by default exit 3 naming reason + both remedies; with consent flag 841 CD offsets + EOCD rewritten, punk::zip::archive_info offsetstyle STILL file, punkzip extracted 841/841 members crc-verified.
    • punksys.exe copy (sdx/metakit kit shape, 17 sections): classified opaque data, refused by default, consent-moved verbatim, BOOTS loading punk::lib 0.5.1 - metakit is move-safe (G-101 pointer updated).
    • zipcat shape (raw tclsfe stub + zipfs-mkzip append): stamped with the nested-payload note, appended payload mounts and sources, icon byte-identical.
  • CHARACTERIZATION src/tests/shell/testsuites/punkexe/punkres.test (7/7 under the tclsh9.0.5-punk runner, self-gating on bin/punkres + the store fixture): file-relative refusal exit 3 + untouched input, consent shift + convention preserved, payload intact (punk::zip members + crc-verified extraction), icon replaced (extract-ico == punkshell.ico), idempotent re-stamp byte-identical, make.tcl tool info punkres provenance records.
  • punkshell 0.30.1 (CHANGELOG entry).
• REMAINS for acceptance:
  • End-to-end run from a NON-WINDOWS host (WSL linux per G-105's direction): build punkres on that host, stamp a win32 kit there, confirm the icon on a windows machine, and record which host here.
  • Distribution second leg: publication of the built tool as a punkbin artifact through the G-123/G-006 channels (the no-toolchain route). The buildable-from-vendored-source leg is done.
  • FLAG for user (acceptance wording, not editable by agent): the acceptance reads "a pre-built PE that already carries an appended ZIP overlay, for the kit shapes we build (kit, zip, zipcat)" - the 'kit' shape's payload is metakit, not zip. Measured behaviour: it stamps post-hoc via the consented opaque-overlay path and survives (boots). Confirm whether that satisfies the clause as written or the clause should name the metakit case explicitly.