diff --git a/CHANGELOG.md b/CHANGELOG.md index aa31070f..bd309e52 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,22 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` "Project Versioning" section for the bump policy. +## [0.30.0] - 2026-07-29 + +- Kit builds carry a configurable icon (G-057): every baked kit ships a + `bin/.resources.toml` sidecar recording the build-time icon choice + (default `src/runtime/punkshell.ico`; per-kit override = root `punkshell.ico` + in the kit's own custom `.vfs` folder), with icon identity and provenance + hashes reaching back through the G-135 assetorigin records to the SVG master. + On win32 targets baked by a twapi-capable host the icon is embedded as PE + RT_ICON/RT_GROUP_ICON resources - the payload-free runtime stub copy is + stamped before the vfs payload is attached (tcl-sfe mechanism, Ashok P. + Nadkarni), so built kits now wear the punkshell mark in Explorer and the + taskbar instead of their runtime's icon. Non-PE targets skip as not + applicable; incapable hosts (no twapi / non-windows build host) skip with a + distinct notice; the sidecar is written in every case and runtime store + originals are never modified. + ## [0.29.0] - 2026-07-28 - `proj:` package-mode scope prefix (G-033): an installed or standalone punk shell diff --git a/bin/AGENTS.md b/bin/AGENTS.md index 72872dcd..d92eb998 100644 --- a/bin/AGENTS.md +++ b/bin/AGENTS.md @@ -156,6 +156,42 @@ set; and an unclassifiable binary is silence, not a pass. A kit whose bake recap showed no `BUILD-WARNING`s still only proves what those checks measure - not that every payload package loads. +### Kit resource record sidecar + embedded icon (G-057) + +Every kit a bake builds ships with a `bin/.resources.toml` text +sidecar (deployed beside the kit; the build copy sits in `src/_build/`). +It records the build-time icon choice for EVERY target, and on win32 targets +baked by a capable host the same icon is embedded into the kit executable as +PE RT_ICON/RT_GROUP_ICON resources (twapi mechanism, stub stamped before the +payload is appended so the vfs payload is never at risk; runtime store +originals are never modified). Named "resources", not "icon", so the parked +RT_VERSION stamping extends the same file. + +- Icon choice: a root-level `punkshell.ico` in the kit's own custom `.vfs` + folder (consulted pre-merge, so the `_vfscommon.vfs` copy never reads as an + override) overrides the project default `src/runtime/punkshell.ico`. The + override is shared across a kit definition's targets: used where embedding + applies, ignored elsewhere. +- Format: schema 1, one flat TOML root table in the tolerant family of the + G-135 assetorigin records (unknown keys skipped, unrecognised schema + ignored). Fields: `schema = 1`; `kit`, `target`, `kit_type` (as configured); + `icon_origin` = `default` | `override`; `icon_source` (repo-root-relative + path of the chosen icon); `icon_hash` (`sha256:`-prefixed); + `icon_provenance_source` / `icon_provenance_source_hash` (lifted from the + chosen icon's `.assetorigin.toml` when one exists - the record then reaches + back to the SVG-derived master); `embedding` = `twapi` | `none`; + `embedding_status` = `embedded` | `not-applicable` | `unavailable` | + `failed`; `embedding_reason` (empty when embedded). +- Skip semantics: a target with no PE resources (linux, freebsd, macosx + targets, runtimeless `.kit` artifacts) is NOT APPLICABLE - not degradation; + a win32 target on a host that cannot write resources (twapi unloadable, or + a non-windows build host) records `unavailable` with the distinct reason, + and a portable post-hoc stamper is the G-128 remedy. In every case the + sidecar is still written and the build completes. +- Byte-stable at unchanged input and host capability: no timestamps, fixed + field order, rewritten only when bytes change. The default icon is part of + the kit's punkcheck source set, so changing it rebuilds kits. + ### Runtime fetch/selection (`punk-runtime.cmd`) `bin/punk-runtime.cmd {fetch|list|use|run}` manages the plain runtimes under diff --git a/goals/G-057-kit-icon-embedding.md b/goals/G-057-kit-icon-embedding.md index 710e9067..b33a4ed6 100644 --- a/goals/G-057-kit-icon-embedding.md +++ b/goals/G-057-kit-icon-embedding.md @@ -49,13 +49,33 @@ image directory. Either icon the stub copy BEFORE appending the zip/metakit payload, or use the sfe approach of splitting payload off, updating resources, and reattaching (sfe-0.2.tm does the split/update/reattach dance for exactly this reason). - Chosen ordering to be recorded here. + CHOSEN (2026-07-29): stub-first. The bake's extraction phase already emits a + payload-free raw runtime prefix (`_build/raw_`) for every PE-relevant + kit type, and all three assemblers consume it (`tcl::zipfs::mkimg` and + `assemble_zipcat_image` take the raw exe; `sdx wrap -runtime` takes the + mksplit .head) - so the seam stamps a PER-KIT COPY of that raw prefix before + assembly and hands the stamped copy to the assembler. No overlay exists at + UpdateResource time, making the corruption class structurally impossible, and + a per-kit copy (not an in-place stamp of the shared raw_runtime) keeps kits + of the same runtime independently overridable. sfe's split/update/reattach + exists because sfe retrofits FINISHED sfx files; the bake owns the + pre-assembly moment, so it uses it. Post-hoc stamping of finished kits stays + G-128 territory (with the tcl-sfe attribution: the split/update/reattach + prior art is Ashok P. Nadkarni's sfe-0.2.tm). - Default icon: src/runtime/punkshell.ico. Per-kit override supplied by the kit's .vfs. IMPORTANT wrinkle for the override convention: _vfscommon.vfs already merges a punkshell.ico into every built vfs tree, so override detection must consult the kit's own CUSTOM .vfs folder (pre-merge, under src/vfs/) - or use an explicit mapvfs.config element - otherwise every kit would appear to - "override" with the same common file. Chosen convention to be recorded here. + "override" with the same common file. CHOSEN (2026-07-29): a root-level + `punkshell.ico` in the kit's own custom folder (`src/vfs/.vfs/punkshell.ico`, + consulted pre-merge) is the override; no mapvfs element (if a config-driven + override is ever wanted it belongs in G-024's toml grammar). This buys a + coherence property for free: the overlay merge order (_vfscommon then custom) + puts the same file into the shipped payload that the embedding stamps into + the PE, so a kit always wears the icon it carries. The override is shared + across a kit definition's targets (2026-07-26 decision, in the Acceptance): + used where embedding applies, ignored elsewhere. - Sidecar first, embedding second. The icon CHOICE is recorded in a plain-text record emitted for every kit on every target; PE embedding is one platform-specific projection of that record. Non-PE targets therefore carry the same information, and @@ -71,6 +91,66 @@ image directory. - Idempotence: re-running a wrap on an already-processed build copy must converge (sfe deletes existing icon/group resources before writing - follow that). +## Decisions (2026-07-29, implementation) + +- Sidecar name and format: `.resources.toml` beside the artifact + (emitted into `src/_build/`, deployed beside `bin/`). Named for + windows RESOURCES, not icons, so the parked RT_VERSION follow-on extends the + same file. Schema 1, one flat TOML root table in the tolerant format family + G-135 established (unknown keys skipped, unrecognised schema ignored): + `schema = 1`, `kit`, `target`, `kit_type`, `icon_origin` (default|override), + `icon_source` (path relative to the punkshell repo root), `icon_hash` + (sha256:-prefixed), `icon_provenance_source`/`icon_provenance_source_hash` + (lifted from the chosen icon's `.assetorigin.toml` when one exists - the + G-135 chain, so the record reaches back to the SVG-derived master), + `embedding` (twapi|none), `embedding_status` + (embedded|not-applicable|unavailable|failed), `embedding_reason` (empty when + embedded). Byte-stability: no timestamps, deterministic field order, written + only when bytes differ; stability is per unchanged input AND host capability + (a cross-host bake legitimately records a different embedding_status than a + windows-host bake of the same kit). +- Seam home: make.tcl inline (`::punkboot::kit_icon_process` is the single + internal entry point; `::punkboot::kit_icon_embed_twapi` is the mechanism it + selects today). Rationale: bake-orchestration glue with one consumer, and + inline keeps the fresh-checkout no-toolchain persona free of + bootsupport-staleness coupling (the guarded-require degradation the G-125 + and G-133 checks accept - both achieved, archived at + goals/archive/G-125-unbootable-kit-deploy-gate.md and + goals/archive/G-133-bake-payload-consistency-checks.md - is wrong for a step + the acceptance requires on a fresh checkout). + Revisit to a punk:: module if a second consumer appears (the G-128 parity + harness is the likely one). +- Mechanism (generalized from sfe-0.2.tm ReplaceIconInStub): enumerate existing + RT_GROUP_ICON (type 14) and RT_ICON (type 3) via + `twapi::load_library -datafile` + `twapi::extract_resources` (tolerating + TWAPI_WIN32 1812/1814 = no resource section / no such resource); then in one + `begin_resource_update` transaction delete ALL type-3 and type-14 entries and + write the new icons as ids 1..N plus one group entry - the .ico's 16-byte + ICONDIRENTRYs converted to 14-byte GRPICONDIRENTRYs (icon id replaces the + image offset; 0 width/height read as 256) - under the first pre-existing + group's name+lang (fallback name 1, lang 1033), committing with + `end_resource_update` (-discard on error). Delete-all-then-write converges + (idempotence), and every bake re-splits the raw prefix from the pristine + runtime anyway. Differences from sfe recorded deliberately: sfe preserves its + known single group entry in place and reuses ids (and its new-id branch has a + latent `maxExistinIconId` typo that errors when the new .ico carries more + images than the old group - not copied); we replace all groups so a + multi-group stub cannot present a stale first group to Explorer's + lowest-entry pick. +- Skip taxonomy (each with its own notice, sidecar always written): non-PE + target -> not applicable; runtimeless `-` .kit artifact -> not applicable + (no PE container); cookfs/cookit kit type -> unavailable (payload is edited + in place inside the mounted runtime - no payload-free stub moment exists); + win32 target with host process-family != windows -> unavailable, cross-host + (remedy: G-128); twapi/twapi_resource not loadable -> unavailable, named; + raw runtime prefix missing (extraction did not produce one) -> unavailable, + named. A twapi error during embedding is a recapped BUILD-WARNING and + `embedding_status = "failed"` - advisory in the G-133 manner, the kit still + builds and deploys with the runtime's own icon. +- Change detection: `src/runtime/punkshell.ico` is added to the kit's punkcheck + source set, so a default-icon change rebuilds kits; an override icon lives + inside the kit's `.vfs` folder, which is already a source. + ## Follow-on candidate goal (flagged, not yet a GOALS.md entry) Version-information resource stamping (RT_VERSION, type 16) - sfe-0.2.tm already @@ -152,3 +232,71 @@ delete-then-write). Requirements sketch agreed 2026-07-10: file already flags (G-025's kit stamp, G-117's embedded artifact record, the parked RT_VERSION follow-on) - one text record per kit, on every platform. - G-121 (archived) parameterized the wrap steps this goal's acceptance runs through: 'make.tcl bake ' now rebuilds a single kit for cheap icon-override verification, and a per-kit icon override would slot into bakelist's filtered per-kit detail - see goals/archive/G-121-bakelist-selective-bake.md +- KNOWN SHORTCOMING (cross-host, per Acceptance): a win32-target kit baked on a + host that cannot run the twapi resource APIs deploys UNSTAMPED (with its + runtime's own icon), records `embedding_status = "unavailable"` with the + distinct reason in its sidecar, and the build otherwise completes unchanged. + Two conditions produce it, each with its own notice text: a non-windows + build host (process family != windows - e.g a WSL/linux cross-bake), and a + windows host whose driving tclsh cannot load twapi (observed live 2026-07-29: + the msys2 ucrt64 mingw tclsh86 - vendored twapi 5.0b1 does not load there). + REMEDY: G-128's portable post-hoc stamper - it operates on the finished kit + from any host, and G-057's seam selects it when present. Until then, rebaking + the affected kit from a capable windows host (or stamping later from one) + closes the gap. + +## Progress + +Implementation landed 2026-07-29 (activation b757d183). Verification evidence +so far, windows host win32-x86_64 (native Tcl 9.0.3 driving unless stated), +recorded as gathered: + +- Full bake: 10 of 11 buildable kits INSTALLED with the icon step live - + embedded via twapi for every win32 kit (zip: punk91, punk905, punk902z, + punk9bi_beta; zip win32-ix86: punkluck86, punk91ix86; kit/sdx-metakit: + punkbi, punksys, punk86; punk9_beta embedded + built but failed DEPLOY on + the pre-existing bin file lock - its stamped build product waits in + src/_build, sidecar deliberately not deployed with it). punkmagic (the one + zipcat-configured kit) is unbuildable for the pre-existing reason that its + runtime tclsh90magic.exe was never present in the store - the zipcat + ASSEMBLER is exercised via the zipfs-less concat path below, which is the + same ::punkboot::assemble_zipcat_image call. +- punkshell902 (kit-type, target linux-x86_64): NOT APPLICABLE notice, sidecar + written with the full icon choice + provenance and + embedding_status not-applicable. +- Resource inspection (twapi::extract_resources + read_resource, not + eyeballing): punk91.exe, punk86.exe (metakit), punk91ix86.exe (32-bit PE) all + PASS - exactly one RT_GROUP_ICON (the stub's original group name "SFE" + preserved, lang 1033), 7 entries, every RT_ICON byte-identical to + src/runtime/punkshell.ico's images including the 256px PNG entry. +- Boot (scratch copies, no user shells touched): punk91 (zip, zipfs mount, + tcl_library from payload, punk::args + punk::zip load), punk86 (metakit, + library inside mounted exe path), punk905 (zip) - all BOOT-OK reading their + payloads. +- Override round-trip: alternative/punk-mark-solid.ico placed as + src/vfs/punk9wintk903.vfs/punkshell.ico -> bake punk91 -> sidecar + icon_origin "override" with the override's path + hash (distinct artwork + hash 2e91ae70...), embedded resources match the SOLID ico exactly; + provenance fields correctly absent (override carries no assetorigin + sidecar). Override file removed, full bake restored the default - deployed + punk91 verified back on punkshell.ico. +- Idempotence: kit_icon_embed_twapi run twice more on an already-stamped stub + copy - identical file size both times, resource inspection still exactly + 7+1 entries byte-matching the ico (no accumulation, converged). Every bake + additionally restarts from a pristine re-split raw prefix by construction. +- Sidecar byte-stability: independent rebuilds at unchanged input emit + byte-identical records (the punk91 selective bake and the later full bake + produced the same sidecar bytes; no timestamps by design, and the emitter + rewrites the file only when bytes change). +- No-toolchain windows path: embedding driven end-to-end under the fetched + family runtime tclsh9.0.5-punk.exe with ONLY the vendored twapi 5.0b1 on + auto_path (no system twapi present in that runtime) - 7 images embedded, + converged on restamp. The system tclsh 9.0.3 run used its own twapi 5.2.0 - + both arms of the 2026-07-26 probe now exercised for real. +- Host-capability skip, twapi-unloadable condition: punkluck86 baked under the + msys2 ucrt64 mingw tclsh86 (windows process family, 8.6, no zipfs) - + distinct notice "embedding UNAVAILABLE ... twapi resource APIs not + loadable", sidecar written, kit assembled via the zipfs-less concat path + and deployed unstamped (graceful completion). +- Runtime store originals untouched throughout: stamping operates on + per-kit copies (src/_build/iconed_) of the extraction-phase raw prefix. diff --git a/punkproject.toml b/punkproject.toml index 688971bc..79c84b5a 100644 --- a/punkproject.toml +++ b/punkproject.toml @@ -1,4 +1,4 @@ [project] name = "punkshell" -version = "0.29.0" +version = "0.30.0" license = "BSD-2-Clause" diff --git a/src/make.tcl b/src/make.tcl index 0658f3c4..7366093d 100644 --- a/src/make.tcl +++ b/src/make.tcl @@ -2300,15 +2300,24 @@ DIAGRAM 2 - KIT ASSEMBLY DETAIL (the 'make.tcl bake' stage; incl. vfslibs phase) (gate) --> refused: kit listed under FAILED KITS, nothing | built, nothing deployed, bin/ untouched | - | zip image appended to the runtime exe: tcl::zipfs::mkimg, - | or (driving tcl without zipfs, e.g 8.6) raw-runtime split - | + punk::zip::mkzip + concatenation - both mount the same + | kit icon step (G-057): .resources.toml sidecar records + | the icon choice for every target (default src/runtime/ + | punkshell.ico; override = punkshell.ico in the kit's own .vfs + | folder, pre-merge); win32 targets on a twapi-capable host get + | the icon stamped into a per-kit copy of the raw runtime stub + | BEFORE the payload attach (non-PE target: not applicable; + | incapable host: distinct notice - sidecar written either way) + | + | zip image appended to the (stamped) runtime exe: tcl::zipfs:: + | mkimg, or (driving tcl without zipfs, e.g 8.6) raw-runtime + | split + punk::zip::mkzip + concatenation - both mount the same v - src/_build/.exe + src/_build/.exe (+ .exe.resources.toml) | | smoke-require probe (host-runnable kits with declared | packages): plain 'package require' inside the artifact [K11] | deploy step: delete old bin/.exe, copy new one in + | (the resource sidecar deploys beside the kit) v bin/.exe <-- FAILS (kit listed under FAILED KITS) if a running shell still holds the old exe [K5] @@ -6296,6 +6305,378 @@ proc ::punkboot::assemble_zipcat_image {raw_runtime wrapvfs outfile zipfile} { return $zipbytes } +# -- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- +#G-057 kit icon step. Every built kit gets a .resources.toml sidecar +#recording the build-time icon choice (default src/runtime/punkshell.ico, +#overridable by a root-level punkshell.ico in the kit's own custom .vfs folder, +#consulted PRE-merge so the _vfscommon copy never reads as an override), and +#win32-target kits get that icon embedded as PE RT_ICON/RT_GROUP_ICON resources +#when this host can write them. The mechanism follows tcl-sfe's sfe-0.2.tm +#ReplaceIconInStub (twapi resource-update APIs; Ashok P. Nadkarni) generalized +#to arbitrary stubs: enumerate, delete ALL icon/group entries, write fresh - +#idempotent by construction. Ordering is stub-first: a per-kit copy of the +#payload-free raw runtime prefix is stamped BEFORE any payload is appended, so +#the resource rewrite can never corrupt an overlay. kit_icon_process is the +#single internal entry point - mechanism selection lives inside it (and +#kit_icon_embed_twapi) so a portable stamper (G-128) is a substitution, not a +#caller rewrite. Non-PE targets skip as NOT APPLICABLE; a win32 target on an +#incapable host skips with a distinct notice; both still write the sidecar. +#Decisions and skip taxonomy: goals G-057 detail file. Sidecar format doc: +#bin/AGENTS.md. +#sha256 is self-contained (same algorithm as src/assets/logo/make-ico.tcl and +#scriptlib/developer/assetorigin_check.tcl - each stays standalone: make.tcl +#must not depend on scriptlib, and this step must not depend on bootsupport +#freshness; bootsupport carries only sha1/md5). Memoized per path per run. +namespace eval ::punkboot::kiticon { + variable sha256_cache + if {![info exists sha256_cache]} {set sha256_cache [dict create]} + variable twapi_state "" + variable K { + 0x428a2f98 0x71374491 0xb5c0fbcf 0xe9b5dba5 0x3956c25b 0x59f111f1 0x923f82a4 0xab1c5ed5 + 0xd807aa98 0x12835b01 0x243185be 0x550c7dc3 0x72be5d74 0x80deb1fe 0x9bdc06a7 0xc19bf174 + 0xe49b69c1 0xefbe4786 0x0fc19dc6 0x240ca1cc 0x2de92c6f 0x4a7484aa 0x5cb0a9dc 0x76f988da + 0x983e5152 0xa831c66d 0xb00327c8 0xbf597fc7 0xc6e00bf3 0xd5a79147 0x06ca6351 0x14292967 + 0x27b70a85 0x2e1b2138 0x4d2c6dfc 0x53380d13 0x650a7354 0x766a0abb 0x81c2c92e 0x92722c85 + 0xa2bfe8a1 0xa81a664b 0xc24b8b70 0xc76c51a3 0xd192e819 0xd6990624 0xf40e3585 0x106aa070 + 0x19a4c116 0x1e376c08 0x2748774c 0x34b0bcb5 0x391c0cb3 0x4ed8aa4a 0x5b9cca4f 0x682e6ff3 + 0x748f82ee 0x78a5636f 0x84c87814 0x8cc70208 0x90befffa 0xa4506ceb 0xbef9a3f7 0xc67178f2 + } +} +proc ::punkboot::kiticon::sha256_data {data} { + variable K + set bitlen [expr {wide([string length $data]) * 8}] + append data \x80 + set pad [expr {(56 - [string length $data] % 64 + 64) % 64}] + append data [string repeat \x00 $pad] [binary format W $bitlen] + set h0 0x6a09e667; set h1 0xbb67ae85; set h2 0x3c6ef372; set h3 0xa54ff53a + set h4 0x510e527f; set h5 0x9b05688c; set h6 0x1f83d9ab; set h7 0x5be0cd19 + set n [string length $data] + for {set off 0} {$off < $n} {incr off 64} { + binary scan [string range $data $off [expr {$off + 63}]] Iu16 w + for {set t 16} {$t < 64} {incr t} { + set x [lindex $w [expr {$t - 15}]] + set s0 [expr {((($x >> 7) | ($x << 25)) ^ (($x >> 18) | ($x << 14)) ^ ($x >> 3)) & 0xffffffff}] + set x [lindex $w [expr {$t - 2}]] + set s1 [expr {((($x >> 17) | ($x << 15)) ^ (($x >> 19) | ($x << 13)) ^ ($x >> 10)) & 0xffffffff}] + lappend w [expr {([lindex $w [expr {$t - 16}]] + $s0 + [lindex $w [expr {$t - 7}]] + $s1) & 0xffffffff}] + } + set a $h0; set b $h1; set c $h2; set d $h3 + set e $h4; set f $h5; set g $h6; set h $h7 + for {set t 0} {$t < 64} {incr t} { + set S1 [expr {((($e >> 6) | ($e << 26)) ^ (($e >> 11) | ($e << 21)) ^ (($e >> 25) | ($e << 7))) & 0xffffffff}] + set ch [expr {(($e & $f) ^ (~$e & $g)) & 0xffffffff}] + set T1 [expr {($h + $S1 + $ch + [lindex $K $t] + [lindex $w $t]) & 0xffffffff}] + set S0 [expr {((($a >> 2) | ($a << 30)) ^ (($a >> 13) | ($a << 19)) ^ (($a >> 22) | ($a << 10))) & 0xffffffff}] + set maj [expr {($a & $b) ^ ($a & $c) ^ ($b & $c)}] + set T2 [expr {($S0 + $maj) & 0xffffffff}] + set h $g; set g $f; set f $e; set e [expr {($d + $T1) & 0xffffffff}] + set d $c; set c $b; set b $a; set a [expr {($T1 + $T2) & 0xffffffff}] + } + set h0 [expr {($h0 + $a) & 0xffffffff}]; set h1 [expr {($h1 + $b) & 0xffffffff}] + set h2 [expr {($h2 + $c) & 0xffffffff}]; set h3 [expr {($h3 + $d) & 0xffffffff}] + set h4 [expr {($h4 + $e) & 0xffffffff}]; set h5 [expr {($h5 + $f) & 0xffffffff}] + set h6 [expr {($h6 + $g) & 0xffffffff}]; set h7 [expr {($h7 + $h) & 0xffffffff}] + } + return [format %08x%08x%08x%08x%08x%08x%08x%08x $h0 $h1 $h2 $h3 $h4 $h5 $h6 $h7] +} +proc ::punkboot::kiticon::sha256_file {path} { + variable sha256_cache + set key [file normalize $path] + if {[dict exists $sha256_cache $key]} { + return [dict get $sha256_cache $key] + } + set f [open $path rb] + set data [read $f] + close $f + set hex [sha256_data $data] + dict set sha256_cache $key $hex + return $hex +} +proc ::punkboot::kiticon::toml_escape {s} { + return [string map {\\ \\\\ \" \\\"} $s] +} +#Tolerant subset reader for the chosen icon's G-135 assetorigin sidecar - lifts +#source/source_hash so the kit record reaches back to the SVG-derived master. +#Missing sidecar, unknown schema or malformed content simply yields no +#provenance fields (never an error). +proc ::punkboot::kiticon::assetorigin_read {assetpath} { + set sc $assetpath.assetorigin.toml + set out [dict create] + if {![file isfile $sc]} { + return $out + } + if {[catch { + set f [open $sc rb] + set raw [read $f] + close $f + }]} { + return $out + } + foreach line [split [encoding convertfrom utf-8 $raw] \n] { + set line [string trim [string map [list \r {}] $line]] + if {$line eq "" || [string index $line 0] eq "#"} continue + if {[string index $line 0] eq "\["} break + set eq [string first = $line] + if {$eq < 1} continue + set key [string trim [string range $line 0 [expr {$eq - 1}]]] + if {$key ni {schema source source_hash}} continue + set val [string trim [string range $line [expr {$eq + 1}] end]] + if {[string index $val 0] eq "\""} { + set parsed "" + set ok 0 + set len [string length $val] + for {set i 1} {$i < $len} {incr i} { + set chr [string index $val $i] + if {$chr eq "\\"} { + incr i + append parsed [string index $val $i] + } elseif {$chr eq "\""} { + set ok 1 + break + } else { + append parsed $chr + } + } + if {!$ok} continue + set val $parsed + } else { + set cmt [string first # $val] + if {$cmt >= 0} { + set val [string trim [string range $val 0 [expr {$cmt - 1}]]] + } + } + dict set out $key $val + } + if {![dict exists $out schema] || [dict get $out schema] ne "1"} { + return [dict create] + } + return $out +} +proc ::punkboot::kiticon::writefile_ifchanged {path content} { + if {[file exists $path]} { + set f [open $path rb] + set existing [read $f] + close $f + if {$existing eq $content} { + return unchanged + } + } + set f [open $path wb] + puts -nonewline $f $content + close $f + return written +} +proc ::punkboot::kiticon::copy_ifchanged {from to} { + if {![file isfile $from]} { + return absent + } + set f [open $from rb] + set fromdata [read $f] + close $f + if {[file exists $to]} { + set f [open $to rb] + set todata [read $f] + close $f + if {$todata eq $fromdata} { + return unchanged + } + } + file copy -force $from $to + return copied +} +#Memoized once per run: can this host's tclsh write PE resources? Returns +#{1 {}} or {0 reason} with the reason distinguishing cross-host from +#twapi-unavailable (the acceptance requires distinct notices). +proc ::punkboot::kiticon::twapi_available {} { + variable twapi_state + if {$twapi_state eq "ok"} { + return [list 1 {}] + } + if {$twapi_state ne ""} { + return [list 0 $twapi_state] + } + if {[punkboot::lib::platform_process_family $::punkboot::host_platform] ne "windows"} { + set twapi_state "cross-host: win32 target baked on a $::punkboot::host_platform host cannot run the twapi resource APIs (portable post-hoc stamping is the G-128 remedy)" + return [list 0 $twapi_state] + } + if {[catch {package require twapi_resource}] && [catch {package require twapi}]} { + set twapi_state "twapi resource APIs not loadable in this tclsh ([info nameofexecutable]) - the vendored twapi under vendorlib_tcl/win32-x86_64 serves native windows tclsh builds" + return [list 0 $twapi_state] + } + if {![llength [info commands ::twapi::begin_resource_update]]} { + set twapi_state "twapi loaded but its resource-update APIs are absent" + return [list 0 $twapi_state] + } + set twapi_state ok + return [list 1 {}] +} +#Replace RT_ICON (type 3) and RT_GROUP_ICON (type 14) in the PE at exepath with +#the images of icopath. Follows sfe-0.2.tm ReplaceIconInStub generalized: all +#existing icon/group entries (any name, any lang) are deleted and the new set +#written as ids 1..N under the first pre-existing group's name+lang (fallback +#name 1, lang 1033), in one resource-update transaction. Raises an error on +#failure (after -discard); returns the number of images embedded. The file at +#exepath must carry NO appended payload - a resource update rewrites the image. +proc ::punkboot::kit_icon_embed_twapi {exepath icopath} { + set f [open $icopath rb] + set icondata [read $f] + close $f + binary scan $icondata ttt ico_reserved ico_type ico_count + if {$ico_reserved != 0 || $ico_type != 1} { + error "$icopath is not recognisable as a .ico file" + } + if {$ico_count == 0} { + error "no images in $icopath" + } + set old_icons [list] + set old_groups [list] + set libh [twapi::load_library $exepath -datafile] + try { + set resources [twapi::extract_resources $libh] + foreach {restype var} {3 old_icons 14 old_groups} { + if {[dict exists $resources $restype]} { + foreach rname [dict keys [dict get $resources $restype]] { + foreach rlang [dict keys [dict get $resources $restype $rname]] { + lappend $var [list $rname $rlang] + } + } + } + } + } trap {TWAPI_WIN32 1812} {} { + #no resource section at all - nothing to enumerate + } finally { + twapi::free_library $libh + } + set group_name 1 + set group_lang 1033 + if {[llength $old_groups]} { + lassign [lindex $old_groups 0] group_name group_lang + } + set update_h [twapi::begin_resource_update $exepath] + try { + foreach entry $old_icons { + lassign $entry rname rlang + twapi::delete_resource $update_h 3 $rname $rlang + } + foreach entry $old_groups { + lassign $entry rname rlang + twapi::delete_resource $update_h 14 $rname $rlang + } + set grouphdr [binary format ttt 0 1 $ico_count] + for {set i 0} {$i < $ico_count} {incr i} { + #.ico ICONDIRENTRY is 16 bytes; GRPICONDIRENTRY is 14 - the trailing + #4-byte image offset becomes a 2-byte icon resource id. Raw width and + #height bytes are carried as-is (0 means 256 in both formats). + set off [expr {6 + 16 * $i}] + binary scan $icondata "@${off} cu cu cu cu tu tu nu nu" w h cc rsv planes bits bytesinres imgoff + set iconid [expr {$i + 1}] + append grouphdr [binary format "cu cu cu cu tu tu nu tu" $w $h $cc $rsv $planes $bits $bytesinres $iconid] + twapi::update_resource $update_h 3 $iconid $group_lang [string range $icondata $imgoff [expr {$imgoff + $bytesinres - 1}]] + } + twapi::update_resource $update_h 14 $group_name $group_lang $grouphdr + } on error {msg ropts} { + catch {twapi::end_resource_update $update_h -discard} + return -options $ropts $msg + } + twapi::end_resource_update $update_h + return $ico_count +} +#G-057 single internal entry point. Decides the kit's icon (override: root +#punkshell.ico in the kit's own custom .vfs folder, pre-merge; else the project +#default src/runtime/punkshell.ico), writes/refreshes the kit's +#.resources.toml sidecar in the build folder, and - when the target +#has PE resources and this host can write them - stamps a per-kit copy of the +#payload-free raw runtime prefix. Returns {wrap_runtime sidecarpath}: +#wrap_runtime is the runtime prefix the assembly step must consume (the stamped +#copy when embedding happened, the untouched raw prefix otherwise). +proc ::punkboot::kit_icon_process {targetkit rt_target rtname kit_type sourcefolder vfstail raw_runtime buildfolder} { + set repo_root [file dirname $sourcefolder] + set override_icon $sourcefolder/vfs/$vfstail/punkshell.ico + if {[file isfile $override_icon]} { + set icon_origin override + set icon_path $override_icon + } else { + set icon_origin default + set icon_path $sourcefolder/runtime/punkshell.ico + } + set have_icon [file isfile $icon_path] + set embedding none + set status "" + set reason "" + if {![string match "win32-*" $rt_target]} { + set status not-applicable + set reason "target $rt_target has no PE resources" + } elseif {$rtname eq "-"} { + set status not-applicable + set reason "runtimeless .kit artifact has no PE container" + } elseif {$kit_type in {cookit cookfs}} { + set status unavailable + set reason "kit type $kit_type edits the payload in place inside the mounted runtime - no payload-free stub moment to stamp" + } elseif {!$have_icon} { + set status unavailable + set reason "icon file not found at $icon_path" + } elseif {$raw_runtime in {{} -} || ![file isfile $raw_runtime]} { + set status unavailable + set reason "extraction produced no raw runtime prefix to stamp" + } else { + lassign [::punkboot::kiticon::twapi_available] twapi_ok twapi_reason + if {!$twapi_ok} { + set status unavailable + set reason $twapi_reason + } + } + set wrap_runtime $raw_runtime + if {$status eq ""} { + set stamped $buildfolder/iconed_$targetkit + file copy -force $raw_runtime $stamped + catch {exec chmod +w $stamped} + if {[catch {::punkboot::kit_icon_embed_twapi $stamped $icon_path} embed_result]} { + ::punkboot::print_build_warnings [list "kit icon embedding FAILED for $targetkit ($icon_origin icon $icon_path): $embed_result - kit keeps its runtime's own icon"] + set embedding twapi + set status failed + set reason $embed_result + catch {file delete $stamped} + } else { + set embedding twapi + set status embedded + set wrap_runtime $stamped + puts stdout " kit icon (G-057): embedded $icon_origin icon into $targetkit stub via twapi ($embed_result images from [file tail $icon_path])" + } + } elseif {$status eq "not-applicable"} { + puts stdout " kit icon (G-057): embedding NOT APPLICABLE for $targetkit - $reason (sidecar still written)" + } else { + puts stdout " kit icon (G-057): embedding UNAVAILABLE for $targetkit - $reason (sidecar still written)" + } + set esc ::punkboot::kiticon::toml_escape + set lines [list] + lappend lines "# kit resource record (schema 1) for the kit artifact named by this file" + lappend lines "# with the .resources.toml suffix stripped. Emitted by the make.tcl bake" + lappend lines "# icon step (G-057). Optional and advisory; format: bin/AGENTS.md." + lappend lines "schema = 1" + lappend lines "kit = \"[$esc $targetkit]\"" + lappend lines "target = \"[$esc $rt_target]\"" + lappend lines "kit_type = \"[$esc $kit_type]\"" + lappend lines "icon_origin = \"$icon_origin\"" + lappend lines "icon_source = \"[$esc [punkcheck::lib::path_relative $repo_root $icon_path]]\"" + if {$have_icon} { + lappend lines "icon_hash = \"sha256:[::punkboot::kiticon::sha256_file $icon_path]\"" + set ao [::punkboot::kiticon::assetorigin_read $icon_path] + if {[dict exists $ao source]} { + lappend lines "icon_provenance_source = \"[$esc [dict get $ao source]]\"" + } + if {[dict exists $ao source_hash]} { + lappend lines "icon_provenance_source_hash = \"[$esc [dict get $ao source_hash]]\"" + } + } + lappend lines "embedding = \"$embedding\"" + lappend lines "embedding_status = \"$status\"" + lappend lines "embedding_reason = \"[$esc $reason]\"" + set sidecar $buildfolder/$targetkit.resources.toml + ::punkboot::kiticon::writefile_ifchanged $sidecar "[join $lines \n]\n" + return [list $wrap_runtime $sidecar] +} +# -- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- + set startdir [pwd] puts stdout "Found [llength $vfs_tails] .vfs folders - checking each for executables that may need to be built" cd [file dirname $buildfolder] @@ -6461,6 +6842,11 @@ foreach vfstail $vfs_tails { $vfs_event targetset_addsource $sourcefolder/vfs/_config ;#some files linked via fauxlink - need to detect change $vfs_event targetset_addsource $sourcefolder/vfs/_vfscommon.vfs $vfs_event targetset_addsource $sourcefolder/vfs/$vfstail + if {[file isfile $sourcefolder/runtime/punkshell.ico]} { + #G-057: the default kit icon is a build input - a changed icon rebuilds + #kits (an override icon lives inside the .vfs folder, already a source) + $vfs_event targetset_addsource $sourcefolder/runtime/punkshell.ico + } if {$rtname ne "-"} { set building_runtime $buildfolder/build_$runtime_fullname ;#working copy of runtime executable - (possibly with kit/zipfs/cookfs etc attached!) $vfs_event targetset_addsource $building_runtime @@ -6857,6 +7243,13 @@ foreach vfstail $vfs_tails { continue } + #G-057 kit icon step: sidecar for every kit; PE icon embedding where + #the target has PE resources and this host can write them. Stub-first: + #the assemblers below consume wrap_runtime - the stamped per-kit copy + #of the payload-free raw prefix when embedding happened, the untouched + #raw prefix otherwise. Runtime store originals are never modified. + lassign [::punkboot::kit_icon_process $targetkit $rt_target $rtname $target_kit_type $sourcefolder $vfstail $raw_runtime $buildfolder] wrap_runtime kiticon_sidecar + set wrapvfs $targetvfs switch -- $target_kit_type { zip { @@ -6886,14 +7279,14 @@ foreach vfstail $vfs_tails { } if {[info commands ::tcl::zipfs::mkimg] ne ""} { #note - as at 2024-08 - there is some discussion about the interface to mkimg - it is considered unstable (may change to -option value syntax) - puts stderr "calling: tcl::zipfs::mkimg $buildfolder/$vfsname.new $wrapvfs $wrapvfs \"\" $raw_runtime" - tcl::zipfs::mkimg $buildfolder/$vfsname.new $wrapvfs $wrapvfs "" $raw_runtime + puts stderr "calling: tcl::zipfs::mkimg $buildfolder/$vfsname.new $wrapvfs $wrapvfs \"\" $wrap_runtime" + tcl::zipfs::mkimg $buildfolder/$vfsname.new $wrapvfs $wrapvfs "" $wrap_runtime } else { #G-122: the driving tcl has no zipfs (8.6) - assemble the same #kind of image by concatenation. The result mounts on any #zipfs-capable runtime (archive-start-relative offsets). puts stderr "WARNING: tcl shell '[info nameofexecutable]' has no zipfs - assembling zip kit $targetkit by concatenation (raw runtime + mkzip)" - ::punkboot::assemble_zipcat_image $raw_runtime $wrapvfs $buildfolder/$vfsname.new $buildfolder/$vfsname.zip + ::punkboot::assemble_zipcat_image $wrap_runtime $wrapvfs $buildfolder/$vfsname.new $buildfolder/$vfsname.zip } } result ]} { set failmsg "zipfs mkimg failed with msg: $result" @@ -6930,7 +7323,7 @@ foreach vfstail $vfs_tails { } #'archive' based zip offsets - editable in 7z,peazip - ::punkboot::assemble_zipcat_image $raw_runtime $wrapvfs $buildfolder/$vfsname.new $buildfolder/$vfsname.zip + ::punkboot::assemble_zipcat_image $wrap_runtime $wrapvfs $buildfolder/$vfsname.new $buildfolder/$vfsname.zip } result ]} { set failmsg "creating zipcat image failed with msg: $result" puts stderr "creating image (zipcat) $targetkit failed" @@ -7005,13 +7398,13 @@ foreach vfstail $vfs_tails { cd [file dirname $building_runtime] ;#make sure SDX is called with the same working dir as the files we're working on. if {[catch { if {$rtname ne "-"} { - exec {*}$::sdxpath wrap $buildfolder/$vfsname.new -vfs $wrapvfs -runtime $raw_runtime {*}$verbose + exec {*}$::sdxpath wrap $buildfolder/$vfsname.new -vfs $wrapvfs -runtime $wrap_runtime {*}$verbose } else { exec {*}$::sdxpath wrap $buildfolder/$vfsname.new -vfs $wrapvfs {*}$verbose } } result]} { if {$rtname ne "-"} { - set sdxmsg "$::sdxpath wrap $buildfolder/$vfsname.new -vfs $wrapvfs -runtime $raw_runtime {*}$verbose failed with msg: $result" + set sdxmsg "$::sdxpath wrap $buildfolder/$vfsname.new -vfs $wrapvfs -runtime $wrap_runtime {*}$verbose failed with msg: $result" } else { set sdxmsg "$::sdxpath wrap $buildfolder/$vfsname.new -vfs $wrapvfs {*}$verbose failed with msg: $result" } @@ -7249,6 +7642,10 @@ foreach vfstail $vfs_tails { puts stdout "$deployment_folder/$targetkit" after 300 file copy $buildfolder/$targetkit $deployment_folder/$targetkit + #G-057: the kit resource sidecar ships with the kit + if {[file isfile $buildfolder/$targetkit.resources.toml]} { + file copy -force $buildfolder/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml + } lappend installed_kits $targetkit # -- ---------- $bin_event targetset_end OK @@ -7257,6 +7654,12 @@ foreach vfstail $vfs_tails { set skipped_kit_install 1 puts stderr "." puts stdout "Skipping kit install for $targetkit with vfs $vfstail runtime $rtname - no change detected" + #G-057: keep the deployed resource sidecar current even when the kit + #itself is unchanged (the record can be newer than the kit rebuild - + #e.g the sidecar feature arriving, or a host-capability change) + if {[::punkboot::kiticon::copy_ifchanged $buildfolder/$targetkit.resources.toml $deployment_folder/$targetkit.resources.toml] eq "copied"} { + puts stdout " kit icon (G-057): refreshed deployed sidecar $deployment_folder/$targetkit.resources.toml" + } lappend skipped_kit_installs [list kit $targetkit reason "no change detected"] $bin_event targetset_end SKIPPED }