Browse Source
Follow-on goals from the G-122 host/target split, each drafted with its overlap survey
and approved before writing. The G-057 contract edit (title, Scope, Goal, Acceptance) was
shown clause-by-clause and approved per the proposal-first rule.
G-057 amended - the goal was drafted before G-122 separated host from target, so its skip
condition read as one host-shaped test ("non-Windows platform skips") while its Notes used
target language. Split in two: a target with no PE resources is NOT APPLICABLE, a windows
target on a host that cannot run twapi is a distinct shortcoming. Both still emit a text
sidecar, now produced for every kit on every target and defined so the parked RT_VERSION
follow-on extends the same file rather than adding a second. Embedding sits behind a
single mechanism seam, and the twapi arm is explicitly PERMANENT - the seam selects
between mechanisms, it does not replace one: twapi is vendored and git-tracked
(src/vendorlib_tcl8|9/win32-x86_64/twapi-5.0b1) and reachable from make.tcl's auto_path,
so a fresh windows checkout that fetches a runtime and bakes gets icons with no build
toolchain (probed under a stock tclsh 9.0.3 and a fetched plain family runtime). The
per-vfs override is settled as shared across a kit definition's targets with no target
dimension.
G-127 - cross-target bake of a custom .vfs. G-122 made the build READ the right runtime
for a target; this makes it WRITE the right kit for one. Three measured gaps: output is
flat, so two POSIX targets of one kit collide (within a run the duplicate guard renames by
RUNTIME not platform; across selective runs exe_names_seen resets and the second
overwrites the first); the payload model is one hand-curated folder per target
(punk9linux.vfs, not a vfslibs participant, with vendorlib_vfs.toml hardcoding the
platform into its source path - a shape inherited from G-037 when there was one target);
and nothing verifies payload against target. Non-native kits go to bin/kits/<platform>/
with native staying at bin/<name>; /bin/* already ignores it so no dual-track ignore work.
Records that G-114 and G-115 overlap yet never reference each other - the platform axis
and the payload axis have never been connected.
G-128 - portable PE resource stamping via a vendored zig tool, plugging into G-057's seam
without retiring its twapi arm. Feasibility measured: zig rc (resinator, bundled in the
0.16 toolchain) is a resource COMPILER supplying the RT_GROUP_ICON/RT_ICON structure
reference, not a patcher - we never link these runtimes, so patching an existing PE is the
work. In our runtimes .rsrc is not the last section (.reloc follows), so the
implementation appends a section and repoints the resource data directory, which shifts
the overlay: safe for an archive-relative zip, not for a file-relative one, so the unsafe
case is refused rather than corrupted. The payoff beyond portability is post-hoc stamping
of a finished kit, which removes G-057's icon-before-append ordering constraint. Its
distribution is settled as buildable from vendored source AND publishable as a punkbin
artifact, so the no-toolchain route need not depend on the committed twapi (the G-004
tension).
Also adds non-contract Related back-pointers in G-057 to G-122/G-127/G-128.
Assisted-by: harness=claude; primary-model=claude-opus-5[1m]; api-location=anthropic.com
master
4 changed files with 271 additions and 8 deletions
@ -0,0 +1,119 @@ |
|||||||
|
# G-127 Cross-target bake of a custom .vfs |
||||||
|
|
||||||
|
Status: proposed |
||||||
|
Scope: src/make.tcl (target-keyed output location and deploy folder, kit-name disambiguation by target, payload/target mismatch reporting, retirement of the kit-loop cross-platform TODO); src/runtime/vendorlib_vfs.toml (per-target payload declaration - format coordinated with G-115 and G-024); src/runtime/mapvfs.config (one vfs definition paired with several targets); src/vfs/ (per-target payload convention with punk9linux.vfs as the migration case); bin/kits/<platform>/ (non-native kit output tier, beside the existing bin/runtime/<platform>/ input tier); src/tests/shell/testsuites/punkexe/ (characterization - two-target bake, no-collision, mismatch report); bin/AGENTS.md + src/AGENTS.md (where cross-target kits land) |
||||||
|
Goal: one kit definition bakes for several declared target platforms from a single source of truth - each target getting ITS binary payload and an output location that cannot be overwritten by another target's kit of the same name - so building a custom .vfs for this machine and for FreeBSD is a declaration, not a duplicated .vfs folder and a naming convention held in the developer's head. G-122 made the build read the right runtime for a target; this makes it write the right kit for one. |
||||||
|
Acceptance: two mapvfs entries pairing the SAME vfs definition with two different targets bake in one run to distinct artifacts - a native-target kit stays at bin/<name> exactly where it lands today, a non-native one at bin/kits/<platform>/<name> with src/_build separated the same way - and neither is overwritten when the two are baked as separate selective single-kit runs (the current defect: exe_names_seen resets per run, so the second write lands on the first); two NON-NATIVE targets sharing a kit name coexist as distinct artifacts, where today they collide or are disambiguated by runtime name rather than by platform; the binary payload installed into each target's kit is selected for that target from src/vendorlib_tcl<N>/<platform> through one declaration rather than by duplicating a .vfs folder per target, and a kit declared for a single target builds byte-comparably to today; a payload binary that would be LOADED on the kit's target but was built for another platform is reported naming the file and both platforms, while per-platform subdirectory layouts that the loader resolves correctly (e.g tcllib's <pkg>/<platform>/<lib>) are explicitly not flagged; bakelist shows each row's output location; the existing punk9linux.vfs is either migrated to the declaration or recorded here as deliberately staying hand-curated, and the linux punkshell902 kit's move from bin/ to bin/kits/linux-x86_64/ is recorded as an intended relocation; the G-023 reconciliation is stated (version-named outputs compose with the platform directory rather than competing with it); native win32 kit outputs are unchanged in name and location and the existing punkexe suites pass unchanged; the kit-loop TODO at src/make.tcl "allow building of kits for other platforms" is retired pointing here. |
||||||
|
|
||||||
|
## Context |
||||||
|
|
||||||
|
G-122 (achieved - see goals/archive/G-122-host-target-platform-split.md) made the build |
||||||
|
READ correctly for a target: which `bin/runtime/<tier>` a runtime comes from, its |
||||||
|
filename suffix, the artifact's suffix, presence checks, process-sweep applicability. It |
||||||
|
deliberately did not touch the output side. make.tcl still carries the standing TODO in |
||||||
|
the kit loop, of which G-122 satisfied only the second half: |
||||||
|
|
||||||
|
#TODO - allow building of kits for other platforms |
||||||
|
# - we need to use <project>/bin for only kits targetting current platform, |
||||||
|
# and use <project>/bin/<platform> for others |
||||||
|
# that we we can have same target executable for multiple platforms |
||||||
|
# - to do this we need to change runtime/mapvfs.config to have platform names <-- done |
||||||
|
|
||||||
|
Three gaps remain, all measured 2026-07-26: |
||||||
|
|
||||||
|
1. **Output is flat.** `src/_build/<name>` and `bin/<name>`, no platform dimension. |
||||||
|
`targetkit = <appname><suffix>` and both linux-x86_64 and freebsd-x86_64 yield an |
||||||
|
empty suffix, so two POSIX targets of the same kit collide. Within one run the |
||||||
|
duplicate guard renames the second to `<appname>_<runtimename>` - RUNTIME-keyed, not |
||||||
|
platform-keyed. Across separate SELECTIVE runs `exe_names_seen` starts empty, so the |
||||||
|
second simply overwrites the first in both `src/_build` and `bin`. win32-vs-posix is |
||||||
|
safe today only because of the `.exe` suffix - naming luck, not design, and the reason |
||||||
|
the linux `punkshell902` currently coexists with `punk91.exe` without trouble. |
||||||
|
|
||||||
|
2. **The payload model is one hand-curated folder per target.** `punk9linux.vfs` IS the |
||||||
|
convention: a separate folder carrying linux `.so` files (tcllibc, tcltls, tdom, |
||||||
|
thread). It is not a vfslibs participant, so its lib tree is maintained by hand. |
||||||
|
`vendorlib_vfs.toml` hardcodes the platform into the source path |
||||||
|
(`source = "vendorlib_tcl9/win32-x86_64/tcludp1.0.13"`) with explicit per-vfs target |
||||||
|
lists - a shape inherited from G-037 (achieved 2026-07-08), which established |
||||||
|
"which vfs folders participate is explicitly declared per kit" back when there was |
||||||
|
only one target. So "bake the same custom .vfs here and on FreeBSD" is not |
||||||
|
expressible; you duplicate the folder and re-copy the pure-Tcl payload. |
||||||
|
Two mitigating facts: `_vfscommon.vfs` (merged into every kit) carries no platform |
||||||
|
binaries, so the common payload is already target-agnostic - the problem is confined |
||||||
|
to the kit-specific vfs. And leakage runs both ways: `punk9linux.vfs` currently |
||||||
|
contains `tcllib2.0/md5c/win32-x86_64/md5c.dll` and |
||||||
|
`tcllib2.0/tcllibc/win32-x86_64/tcllibc.dll`, harmless (the loader picks by platform |
||||||
|
dir) but dead weight from a wholesale tree copy. |
||||||
|
|
||||||
|
3. **Nothing verifies payload against target.** The cross-target `punkshell902` bake |
||||||
|
produced a valid ELF, but the build would not have complained had that vfs been full |
||||||
|
of DLLs. |
||||||
|
|
||||||
|
The runtime side is already in place for a FreeBSD experiment: `bin/runtime/freebsd-x86_64/` |
||||||
|
exists locally (empty but for sha1sums.txt) and punkbin upstream carries |
||||||
|
`freebsd-x86_64/tclkit-851-freebsd7-x86_64`, fetchable via |
||||||
|
`punk-runtime.cmd fetch -platform freebsd-x86_64`. |
||||||
|
|
||||||
|
## Approach |
||||||
|
|
||||||
|
- Adopt the developer's recorded split, with the directory named explicitly: |
||||||
|
native-target kits stay at `bin/<name>` (every launcher, test and habit expects |
||||||
|
`bin/punk91.exe` there), non-native kits go to `bin/kits/<platform>/<name>`. |
||||||
|
`bin/kits/` rather than `bin/<platform>/` because `bin/runtime/<platform>/` already |
||||||
|
means build INPUTS - having outputs in a sibling shape one level up would read as the |
||||||
|
same thing. No ignore-rule work: `/bin/*` already covers `bin/kits/` (verified), so |
||||||
|
neither `.gitignore` nor the fossil ignore-glob needs a change. |
||||||
|
- Supply the PLATFORM AXIS to whatever payload-declaration mechanism exists at the time |
||||||
|
(today `vendorlib_vfs.toml`'s explicit per-kit lists; G-115's toml if it lands first). |
||||||
|
This goal does not own the declaration FORMAT - it owns the requirement that a payload |
||||||
|
can be declared per target and selected by the kit's target. |
||||||
|
- Keep the mismatch check load-path-relevant: report a binary that would actually be |
||||||
|
LOADED on the target, and stay silent on per-platform subdirectory layouts the loader |
||||||
|
resolves correctly. A check that fires on every existing tree is noise, and the current |
||||||
|
trees have benign leakage. |
||||||
|
|
||||||
|
## Alternatives considered |
||||||
|
|
||||||
|
- Put ALL kits under `bin/kits/<platform>/`, including native - rejected: breaks every |
||||||
|
launcher, test, doc and habit that expects `bin/punk91.exe`, for uniformity's sake. |
||||||
|
- Keep one .vfs folder per target and fix only the output collision - rejected: leaves in |
||||||
|
place exactly the duplication that motivates the goal. |
||||||
|
- Make a payload/target mismatch a hard build failure - rejected as the DEFAULT: the |
||||||
|
current tree would fail immediately on known-benign leakage. Offer it as a strict flag. |
||||||
|
- Disambiguate colliding names by runtime rather than by platform (extending today's |
||||||
|
`<appname>_<runtimename>` behaviour) - rejected: it encodes the wrong axis, and it does |
||||||
|
nothing for the cross-run overwrite, which is the sharper half of the defect. |
||||||
|
|
||||||
|
## Notes |
||||||
|
|
||||||
|
- Depends on: G-122 (achieved) - the input half; this is the output half of the same |
||||||
|
split, and retires the TODO that goal left standing. |
||||||
|
- Related: G-115 - owns the payload DECLARATION FORMAT and is currently written without |
||||||
|
any platform axis (zero mentions of platform or target). Whichever of the two lands |
||||||
|
second adopts the other: this goal supplies the axis, that goal supplies the format. |
||||||
|
- Related: G-023 - owns output NAMES and its Acceptance hardcodes `punk9-0.5.0.exe`. |
||||||
|
This goal owns output LOCATION and the suffix; the two must compose |
||||||
|
(`bin/kits/<platform>/punk9-0.5.0`) rather than compete. |
||||||
|
- Related: G-114 - the same platform axis one layer down (per-platform tm module roots |
||||||
|
registered by the boot for the running platform). A cross-target kit's tm roots must be |
||||||
|
the TARGET's. Note the xref shows G-114 and G-115 unlinked despite overlapping: the |
||||||
|
platform axis and the payload axis have never been connected, and this goal is that |
||||||
|
connection. |
||||||
|
- Related: G-024 - format home for expressing one vfs definition against several targets. |
||||||
|
- Related: G-057 - icon embedding is a windows-target-only wrap step keyed off a per-vfs |
||||||
|
override, so "which kits get an icon" becomes a target question once one vfs definition |
||||||
|
serves several targets. Settled there 2026-07-26: the override is shared across a kit |
||||||
|
definition's targets and carries no target dimension - used where embedding applies, |
||||||
|
ignored elsewhere. |
||||||
|
- Related: G-105 - produces the cross-target RUNTIMES this consumes for custom kits. |
||||||
|
- Related: G-005 / G-006 - where a target's binary payload comes from: built from source |
||||||
|
or downloaded with consent. |
||||||
|
- Related: G-116 / G-020 - concrete per-platform binary payloads inside kit vfs lib trees |
||||||
|
(suite-built tcltls; the screen-capture backends under punk9wintk903.vfs/lib_tcl9). |
||||||
|
- Related: G-101 - 8.6 containers ride the same mapping and output surfaces. |
||||||
|
- Related: G-037 (achieved 2026-07-08) - established the per-kit declared vfslibs |
||||||
|
propagation whose single-target assumption this goal generalizes; see |
||||||
|
goals/archive/G-037-vendorlib-vfs-propagation.md. |
||||||
|
- Tension to hold in view: G-004 (no committed binaries) - per-target payloads multiply |
||||||
|
the binary count in the tree unless they arrive by build or fetch. |
||||||
@ -0,0 +1,95 @@ |
|||||||
|
# G-128 Portable PE resource stamping |
||||||
|
|
||||||
|
Status: proposed |
||||||
|
Scope: src/tools/punkres/ (vendored zig source - tool name settled in the work - with provenance and licence records); src/make.tcl (mechanism selection at the G-057 seam, sharing G-126's tool build step); bin/tools/zig* (pinned toolchain as consumed); bin/punkres.exe (untracked build output); src/tests/ (characterization - icon replaced, payload intact, idempotent, overlay-unsafe refusal); TEMP_REFERENCE/tcl-sfe (read-only reference for the resource structures) |
||||||
|
Goal: punkshell can stamp windows resources into a FINISHED kit executable from any build host - no twapi, no windows host, no disturbing the appended vfs payload - so a cross-target win32 kit is not condemned to its runtime's icon, and resource stamping becomes a post-hoc operation on a built artifact rather than something wired into the middle of the kit wrap. G-057's mechanism seam selects this implementation when it is present and keeps its twapi path otherwise. |
||||||
|
Acceptance: the vendored tool, built by the same make.tcl tool step G-126 establishes, replaces RT_ICON/RT_GROUP_ICON in a pre-built PE that already carries an appended zip overlay, for the kit shapes we build (kit, zip, zipcat per mapvfs.config): the new icon is confirmed by reading the resources back with the tool ITSELF (so verification does not require a windows host or twapi) and cross-checked once on windows with twapi::extract_resources; the stamped kit still boots to a working punk shell reading its payload; re-stamping converges with no resource accumulation; it works where .rsrc is NOT the last section - the shape our own runtimes have (tclsfe-x64/punk91: .text .rdata .data .pdata .rsrc .reloc; suite-built tclsh9.0.5-punk: .text .rdata .buildid .data .pdata .tls .rsrc .reloc) - by appending a section and repointing the resource data directory rather than growing .rsrc in place; the appended payload survives the file-offset shift this causes, which is safe for an ARCHIVE-relative zip and not for a file-relative one, so a file-relative payload is REFUSED naming that reason rather than silently corrupted; one end-to-end run from a NON-WINDOWS host produces a stamped win32 kit whose icon a windows machine confirms (recorded here: which host); G-057's seam selects the tool when present and falls back to its twapi path otherwise, with a parity check showing both mechanisms yield equivalent resource content from the same input, and zig stays optional - with no tool built, G-057 behaves exactly as it does without this goal; the tool's distribution is settled and recorded - buildable from the vendored source with the pinned toolchain AND publishable as a punkbin artifact through the G-123/G-006 channels, so a no-toolchain user has a route that does not depend on the committed twapi; the tool's structure handling is written so the parked RT_VERSION stamping needs no second tool; licensing and upstream provenance of the vendored tree are recorded per G-063 and G-026. |
||||||
|
|
||||||
|
## Context |
||||||
|
|
||||||
|
G-057 embeds icons with twapi, which is the right mechanism to ship first (demonstrated |
||||||
|
prior art in tcl-sfe, no new build infrastructure, and the vendored twapi makes it work |
||||||
|
on a fresh windows checkout with nothing built). It has two limits that are structural |
||||||
|
rather than incidental: |
||||||
|
|
||||||
|
- **Host-bound.** twapi is a Windows extension, so a windows-target kit cross-baked from |
||||||
|
a linux or FreeBSD host cannot be stamped at all. G-122 made cross-target bakes |
||||||
|
routine and G-127 makes them first-class, so this stops being hypothetical. |
||||||
|
- **Ordering-bound.** A Windows resource update rewrites the PE image and corrupts an |
||||||
|
already-appended vfs payload, which is why G-057 must either icon the stub BEFORE |
||||||
|
appending or do sfe's split/update/reattach dance. Stamping cannot be applied to a |
||||||
|
finished kit. |
||||||
|
|
||||||
|
Feasibility measured 2026-07-26. The zig toolchain we already vendor carries both halves |
||||||
|
of the problem's prior art: `zig rc` is resinator, a full Windows RC compiler bundled at |
||||||
|
`lib/compiler/resinator/`, which is the reference for building correct RT_GROUP_ICON / |
||||||
|
RT_ICON structures from an .ico image directory; and `std.coff` parses PE. Note what |
||||||
|
resinator is NOT: a compiler produces a .res blob for a LINKER to embed into an |
||||||
|
executable it is building. Our runtimes are pre-built third-party binaries we never link, |
||||||
|
so the patch-an-existing-image half is the work this goal owns. |
||||||
|
|
||||||
|
Section layout of our actual runtimes, via llvm-objdump: |
||||||
|
|
||||||
|
tclsfe-x64.exe / punk91.exe .text .rdata .data .pdata .rsrc .reloc |
||||||
|
tclsh9.0.5-punk.exe .text .rdata .buildid .data .pdata .tls .rsrc .reloc |
||||||
|
|
||||||
|
`.rsrc` is not last - `.reloc` follows it - so growing it in place is unavailable and the |
||||||
|
implementation must append a section and repoint IMAGE_DIRECTORY_ENTRY_RESOURCE (the |
||||||
|
approach rcedit takes). That shifts the overlay's file offset, which is exactly the |
||||||
|
archive-relative vs file-relative distinction that G-122 and G-124 keep surfacing: an |
||||||
|
archive-relative zip survives being moved because its internal offsets are relative to |
||||||
|
its own start, a file-relative one does not. punkshell's own kits are archive-relative |
||||||
|
(measured: punk91.exe, preamble 5267968), so post-hoc stamping is viable for them - and |
||||||
|
the unsafe case must be detected and refused, not attempted. |
||||||
|
|
||||||
|
The payoff is therefore larger than portability: a tool that treats the overlay as a |
||||||
|
first-class thing to preserve removes G-057's ordering constraint entirely, which is what |
||||||
|
the parked RT_VERSION stamping wants, since the project version is known late. |
||||||
|
|
||||||
|
## Approach |
||||||
|
|
||||||
|
- Vendor and build through the pattern G-126 establishes - same tool build step, same |
||||||
|
optional-zig posture, same provenance and licence recording. This is the second |
||||||
|
instance of that pattern, not a new one. |
||||||
|
- Read-back is part of the tool, not an afterthought: without it the acceptance cannot be |
||||||
|
checked from a non-windows host, which is the whole point. |
||||||
|
- Refuse rather than risk: a payload whose offsets would break if the archive moves is a |
||||||
|
hard error naming the reason. |
||||||
|
- Build the resource structures once, general enough that the parked RT_VERSION work |
||||||
|
extends the same tool rather than starting another. |
||||||
|
|
||||||
|
## Alternatives considered |
||||||
|
|
||||||
|
- Shell out to rcedit - rejected: a third-party Node/C++ binary, against the |
||||||
|
vendored-source direction, and it would be another committed or fetched executable with |
||||||
|
no in-tree provenance. |
||||||
|
- Keep twapi as the only mechanism - rejected as the END state: it leaves cross-host kits |
||||||
|
icon-less and keeps stamping wired into the middle of the wrap. Note it is NOT rejected |
||||||
|
as a path - G-057's twapi arm is explicitly permanent, and this goal must not retire it. |
||||||
|
- Build resources at link time - impossible here: we do not link the runtimes, we receive |
||||||
|
them as finished binaries. |
||||||
|
- Write it in Tcl instead - rejected: PE section surgery on multi-megabyte images is |
||||||
|
exactly the file-format work the zig tooling direction exists for, and unlike the |
||||||
|
punk::zip reader (G-124) there is no module-level need that would justify a pure-Tcl |
||||||
|
floor. |
||||||
|
|
||||||
|
## Notes |
||||||
|
|
||||||
|
- Depends on: G-057 - supplies the mechanism seam this plugs into, and stays the |
||||||
|
no-toolchain windows path. This goal adds a mechanism; it does not remove one. |
||||||
|
- Depends on: G-126 - the vendored-zig-tool build step, optional-zig posture and |
||||||
|
provenance pattern are shared rather than duplicated. |
||||||
|
- Related: G-123 / G-006 - publishing the built tool as a punkbin artifact is the route |
||||||
|
that gives a no-toolchain user portable stamping without depending on the committed |
||||||
|
twapi, which is what makes the G-004 tension resolvable rather than permanent. |
||||||
|
- Related: G-004 - the tool is built or fetched, never committed; and it is the mechanism |
||||||
|
by which the vendored twapi could eventually stop being load-bearing. |
||||||
|
- Related: G-127 - a win32 kit cross-baked from a non-windows host is precisely the case |
||||||
|
twapi cannot serve; this is what makes those kits complete rather than icon-less. |
||||||
|
- Related: G-124 - shares the archive-relative vs file-relative offset distinction: that |
||||||
|
goal reads a zip at a derived base offset, this one moves one and must not break it. |
||||||
|
- Related: G-023 / G-025 / G-117 (achieved) - the parked RT_VERSION stamping this tool is |
||||||
|
structured for would carry the project version and become a further provenance surface; |
||||||
|
G-057's Notes already require those to be reconciled into one story. |
||||||
|
- Related: G-028 - a locked kit exe blocks a resource update exactly as it blocks deploy. |
||||||
|
- Related: G-063 / G-026 - licence and provenance recording for the vendored tree. |
||||||
Loading…
Reference in new issue