diff --git a/goals/G-101-tcl86-kit-container-strategy.md b/goals/G-101-tcl86-kit-container-strategy.md index 2ab2350e..7f2be7c5 100644 --- a/goals/G-101-tcl86-kit-container-strategy.md +++ b/goals/G-101-tcl86-kit-container-strategy.md @@ -1,7 +1,7 @@ # G-101 kit container strategy for zig-built Tcl 8.6 runtimes (no zipfs) Status: proposed -Scope: investigation + decision record under src/buildsuites/suite_tcl86/ (or successor mechanism); TEMP_REFERENCE/metakit + TEMP_REFERENCE/KitCreator (read-only guidance); src/runtime/mapvfs.config ('kit' type consumers); src/make.tcl kit-wrap path (as consumer) +Scope: investigation + decision record under src/buildsuites/suite_tcl86/ (or successor mechanism); TEMP_REFERENCE/{metakit,KitCreator,kbs,sdx,tcl9} (read-only guidance - inventory in 'Reference material' below); src/runtime/mapvfs.config ('kit' type consumers); src/make.tcl kit-wrap path (as consumer); bin/sdx.kit (the tracked wrapping tool - provenance is part of this goal, see 'sdx: provenance' below) Goal: a decided, demonstrated strategy for wrapping zig-built 8.6 runtimes into self-contained punk86-class kit executables in the absence of zipfs - candidate containers: the metakit/mk4vfs tclkit lineage, cookfs, and zipvfs-over-tclvfs boot - so the prebuilt third-party kit runtimes (tclkit86bi.exe, tclkit-win64-dyn.exe) can eventually be replaced by suite-built, provenance-carrying ones. Acceptance: the candidate containers are assessed and recorded here against punkshell's kit expectations (compatibility with make.tcl's 'kit' wrap path or a documented replacement path; boot-time vfs mount without an installed Tcl; loading binary dlls from or via the container; buildable from source under the zig-only policy); one strategy is demonstrated end-to-end - a kit wrapped from the suite-built 8.6 runtime boots on a path without any Tcl installation and runs a script from inside its vfs; the decision, rationale, and the follow-on work needed to reach punk86-kit parity are recorded in this file. @@ -112,3 +112,199 @@ unblocked; activation is the user's call. future zero-flash launcher takes the w-suffix convention (punkw) rather than -gui/-cli (which in kit lineage encode Tk linkage, not subsystem). - G-121 (archived) built bakelist/selective-bake over the parsed mapping without assuming zip-only types ('kit'-type entries list and bake by name), so 8.6 container entries join that surface automatically - see goals/archive/G-121-bakelist-selective-bake.md +- 2026-07-26: pre-activation survey added four sections below - reference-material + inventory (with the boot-chain reading order), sdx provenance + the + characterize-before-rebuild ordering, the self-mount shadowing gotcha (verified + live, container-agnostic), and metakit-under-Tcl-9 feasibility. Two of them + carry proposed acceptance riders. The first - capture the sdx wrap.tcl patch + somewhere versioned before any sdx rebuild - was DONE the same day + (`src/buildsuites/suite_tcl86/patches/`, with a back-pointer added to + TEMP_REFERENCE/AGENTS.md so the survey is not repeated). The second stands: + make the end-to-end demonstration wrap a SEPARATE COPY of the runtime so the + self-mount shadowing path is actually exercised rather than accidentally + avoided. + +## Reference material (TEMP_REFERENCE, machine-local) + +Recorded 2026-07-26 from an agent survey. CAVEAT: TEMP_REFERENCE is ignored by +BOTH git and fossil (root .gitignore:59, .fossil-settings/ignore-glob:58-59), as +is scratch* (.gitignore:35). Everything named here is unversioned and machine-local +- it can vanish to a `git clean -xdf`. Findings derived from it belong in THIS +file, not in notes left under that tree. + +Metakit boot chain, in reading order (this is the mechanism the mk4 candidate +would reproduce for a suite-built 8.6 runtime): + +- `kbs/kbskit/sources/kbskit0.4/generic/kitInit.c` - the clearest single + description, and the closest to an 8.6 target. `TclKit_AppInit` registers + Mk4tcl/rechan/vfs/tclkitpath via `Tcl_StaticPackage` (:170-177), then installs + a pre-init script (:95-135) that opens the exe AS a metakit datafile + (`mk::file open exe $::tcl::kitpath -readonly`), selects `boot.tcl` out of + `exe.dirs!0.files`, zlib-inflates it when stored size != content length, and + evals it. Comment at :88-93 explains why it must be re-entrant: it runs for + every interp, including child and thread interps. +- `kbs/kbskit/sources/kbskit0.4/library/boot.tcl` - the chicken-and-egg step. + VFS is not mounted yet, so it hand-evaluates `vfsUtils/vfslib/mk4vfs` out of + `lib/vfs1*` with raw `mk::select` (:24-38), mounts + `::vfs::mk4::handler exe`, then UNMOUNTS AND REMOUNTS after fixing the system + encoding (:109-116). Note :40-44 - "8.6 core zlib does not support this for + mk4vfs" forces `mk4vfs::zstreamed 0`, i.e. whole-blob decompression via + `vfs::memchan` rather than streaming (the two branches are at + `kbs/kbskit/sources/vfs1.4/library/mk4vfs.tcl:167-180`). Directly relevant to + this goal's memory/latency profile for a battery-carrying 8.6 kit. +- `kbs/kbskit/sources/kbskit0.4/library/setupvfs.tcl:143-200` - how the kit gets + WRITTEN (the counterpart to sdx wrap). +- `metakit/tcl/mk4tcl.cpp` (2621 lines) - `Mk4tcl_Init` -> `Mktcl_Cmds` + (:2559-2597): stubs at "8.1", registers the mkProperty/mkCursor obj types, one + `MkWorkspace` per interp with assoc-data delproc + exit handler + event source, + defines the nine `mk::*` commands. 8.6 compat is a one-line shim at :27-29; + `CHANGES` dates that work to 2007-06-25. The storage engine (`metakit/src/`) + is Tcl-agnostic - only `tcl/mk4tcl.cpp` + `mk4tcl.h` are Tcl-coupled. +- `KitCreator/kitsh/buildsrc/kitsh-0.0/{kitInit.c,boot.tcl}` - same shape, + generalised: storage backend is a compile-time switch + (`KIT_STORAGE_MK4`/`_ZIP`/`_CVFS`) driving `$::tclKitStorage`, and boot.tcl + branches on it (:18-90). Best reference for separating metakit-specific steps + from generic starkit boot - i.e. for scoping how much of the boot survives if + this goal picks cookfs or zipvfs instead. It also re-roots `tcl::tm` + (:158-161), which kbskit's does not and which THIS goal's payload needs. +- `KitCreator/mk4tcl/build.sh` + `KitCreator/mk4tcl/patches/all/` - the build + recipe and EIGHT patches against metakit 2.4.9.7 (64-bit filespace, 32/64 fix, + `unregisterchans`, `fixeventchannels`, shared-obj linker, ranlib/ar). The + `TEMP_REFERENCE/metakit` checkout is UNPATCHED upstream + (`github.com/jnorthrup/metakit`, last commit "URL fix"), and kbs.tcl:2813-2827 + pulls that same unpatched master zip. If mk4 is chosen, deciding which patch + lineage to carry is a required step, not a detail. + +Other trees in scope: `tcl9/` is the fossil Tcl core checkout (currently 9.1b1 +per generic/tcl.h - NOT the 9.0.5 the suites build, so init-path claims must be +re-verified against 9.0.5); `tcl-sfe/` is the cookfs-lineage prior art named in +Context; `sdx/` is covered below. + +## sdx: provenance and the wrapping-tool question + +Recorded 2026-07-26. The user raised this before activation; findings are +concrete enough to shape acceptance. + +- THE PROVENANCE GAP, precisely. There are two distinct sdx binaries on this + machine, differing by 121 bytes: + - `bin/sdx.kit` == `TEMP_REFERENCE/sdx/sdx.kit`, sha256 `5d4aa629...`, + 119191 bytes. TRACKED IN GIT. This is the one make.tcl resolves. + - `scratch/sdx.kit` == `scratch/sdx-20110317.kit`, sha256 `04dc85d4...`, + 119312 bytes. UNTRACKED AND GITIGNORED. + Unwrapping both (the `.vfs` trees sit beside them) shows the trees are + identical at 64 files each EXCEPT ONE: `lib/app-sdx/wrap.tcl`. The whole delta + is a two-line local patch in the Windows version-resource path: + `open [file join $idir tclkit.inf]` becomes `open ... "r"` plus + `fconfigure $fd -translation binary`, immediately before + `::stringfileinfo::writeStringInfo` (wrap.tcl:232-239 in the patched copy). + Plausibly a real fix for CRLF corruption of `tclkit.inf` on Windows. + So the tracked tool is a locally-patched sdx whose patch is recorded NOWHERE + in version control, and the stock baseline that proves what the patch IS + exists only in two ignored directories on this one machine. RESOLVED + 2026-07-26: both sides are now captured in tracked form at + `src/buildsuites/suite_tcl86/patches/` (stock baseline, patched file, diff, and + a README with the hashes and the reconstruction rules) - a rebuild can no + longer silently drop or silently re-invent the fix. Note the patched path is + DORMANT for present builds: it fires only when `-runtime` is given AND the + runtime is a `.exe` AND `tclkit.inf` exists in the source `.vfs`, and no + `tclkit.inf` exists anywhere in this repo. Whether the fix is CORRECT is + unverified - no reproducer was recorded with it; confirming or retiring it is + part of this goal's characterization work. +- CHARACTERIZATION PRIMITIVE ALREADY EXISTS. sdx ships `version` + (`lib/app-sdx/version.tcl`), which walks the mounted kit and prints + ` -`. Verified live under + `bin/runtime/win32-x86_64/tclkit86bi.exe`: the tracked kit reports + `2025/06/04 22:56:36 69170-33997`, the stock one `2011/03/17 23:39:41 + 50768-16003` - and the former's timestamp is exactly the patched wrap.tcl's + mtime, independently confirming the patch is inside the tracked kit. TRAP for + any rebuild-and-compare: `traverse` hashes file NAME + MTIME + SIZE, not + content, so the signature is not reproducible across a re-wrap unless mtimes + are normalised (sdx has `version -fixtime`). A content-level characterization + battery needs its own comparison (unwrap both, diff trees; plus behavioural + wrap/unwrap/mksplit round-trips), not just this signature. +- BUNDLED LIBRARIES ARE 2011-ERA, and mostly irrelevant to wrapping: base64 + 2.4.1, md5 1.4.2, uri 1.2.1, autoproxy 1.5.1, ftpd 0.4, plus an entire Wikit + GUI + httpd/ftpd/rexecd server stack. Only `app-sdx` (wrap/unwrap/mksplit/ + mkpack/mkinfo/lsk/qwrap) and `stringfileinfo` are on this goal's path. That + argues a rebuild should be a REDUCTION (drop the server/wiki payload) rather + than a library refresh - smaller attack surface, smaller characterization + battery, and the dropped packages have no bearing on kit wrapping. ChangeLog + ends 2011-03-17 (Pat Thoyts); upstream was `svn://svn.equi4.com/sdx/trunk`. +- BOOTSTRAP CIRCULARITY. sdx.kit is itself a metakit starkit (mk4 schema string + `dirs[name:S,parent:I,files[...]]` visible in its tail bytes; contents stored + uncompressed - plaintext greps hit inside the kit). Running it needs an + existing tclkit, so the tool that would wrap a suite-built 8.6 runtime + currently depends on a prebuilt third-party runtime of exactly the kind this + goal exists to replace. make.tcl:5048-5097 encodes that fallback ladder and + ends at `$binfolder/sdx.kit` + a tclkit. Breaking the circularity is a real + acceptance question: either (a) run sdx under the suite-built dynamic + `tclsh86t` with Mk4tcl loaded as a package rather than under a tclkit, or + (b) replace sdx's wrap step with a suite-owned writer (setupvfs.tcl above is + the reference for what writing a kit actually requires). Option (a) is the + cheaper first probe and also tests whether a suite-built Mk4tcl works at all. +- STALE WRAPPER: `bin/sdx1.bat` (and its `TEMP_REFERENCE/sdx/sdx.bat` twin) + invokes `%~dp0..\src\runtime\tclkit86bi.exe`, a path that no longer exists - + runtimes moved to `bin/runtime//`. bin/ has no `sdx` wrapper under + that exact name either, so make.tcl's `auto_execok $binfolder/sdx` probe + cannot hit it and resolution falls through to the sdx.kit + tclkit last + resort. Cosmetic, but it means the documented `sdx` entry point in + bin/AGENTS.md:54 (external tools keep their own name) is currently unmet. + +## Self-mount shadowing (the same-path gotcha) + +Recorded 2026-07-26, verified live rather than inferred. A running tclkit mounts +its VFS AT ITS OWN EXECUTABLE PATH, so from inside itself the exe is a DIRECTORY: +probing `bin/runtime/win32-x86_64/tclkit86bi.exe` gives +`[file type [info nameofexecutable]] == directory`, `[file isfile]` 0, +`[vfs::filesystem info]` listing that same exe path, and +`glob $exe/*` yielding `bin boot.tcl lib tclkit.ico`. + +Consequences this goal inherits whichever container it picks (zipfs has the same +shape via its /app anchor, so this is not an mk4-specific wart): + +- A kit cannot read or copy ITSELF as a file - so it cannot wrap itself, and + cannot be handed to sdx as `-runtime`. sdx's `-runtime` path does + `LoadHeader $pfile` (wrap.tcl:109-115), a plain file read that fails on a + shadowed exe. +- make.tcl already works around this and the workaround should be preserved as a + requirement, not rediscovered: :4995-5005 detects `[file type $runtimefile] eq + "directory"` and substitutes a `_BUILDCOPY` (comment at :4996 states the rule), + and :4980-4981 records that sdx objects when the wrapping shell and the + `-runtime` argument are the same file. +- Acceptance rider: whichever container lands, the demonstration should include + a runtime wrapping a DIFFERENT copy of itself, so the shadowing path is + exercised rather than accidentally avoided. + +## Metakit under Tcl 9 (bearing on container choice) + +Recorded 2026-07-26 from an API survey against TEMP_REFERENCE/tcl9 (9.1b1). +Relevant because it decides whether mk4 is an 8.6-only stopgap or ONE container +format across both runtime lines - which is the strongest argument for choosing +it over cookfs/zipvfs, since Tcl 9 already has zipfs and needs no help. + +- The BOOT hook is easier on 9 than on 8.6: `Tcl_SetPreInitScript` is public API + in 9.1 (generic/tcl.h:2306, run at generic/tclInterp.c:837), where 8.6 forced + kbskit to hand-declare the internal `TclSetPreInitScript` (kitInit.c:50). And + `LocatePreInitScript` honours `$tcl_library` first and exclusively + (tclInterp.c:487-490), so a preinit that mounts the container and sets + tcl_library gets init.tcl sourced from inside it. +- BUT the `proc tclInit {}` override - which is what boot.tcl IS - is dead in + 9.1: tclInterp.c:619-622 says so explicitly, and `tcl_libPath` is gone + (:441-444). A dual-generation design must be preinit-shaped from the start. + UNVERIFIED: whether 9.0.5 (what suite_tcl90 builds) exports + `Tcl_SetPreInitScript`; the reference tree is 9.1b1 and its comments describe + 9.0 as still tclInit-driven. Check before relying on this. +- Mk4tcl PORT COST is bounded but real: ~23 `Tcl_GetStringFromObj` + + list/bytearray call sites need `Tcl_Size*` not `int*`; stubs bump to "9.0"; + one `Tcl_ChannelType` (`mk::channel`) against TCL_CHANNEL_VERSION_5 is the + fiddliest piece; five `Tcl_ObjType` initialisers gain a version + abstract-list + tail (compile clean under C++ aggregate init, worth making explicit). + `Tcl_ObjCmdProc` still takes `int objc` (tcl.h:566) unless built + `-DTCL_NO_DEPRECATED`, so the dispatcher signatures survive. + `Tcl_StaticPackage` -> `Tcl_StaticLibrary` (tcl.h:2309). +- tclvfs is largely FREE on 9: the `forTcl9` branch (1.5.0) already exists + upstream and is already recorded as the tcl9 pin in + src/buildsuites/suite_tcl86/sources.config:28-29 (user decision 2026-07-22). + `Tcl_FSRegister` / TCL_FILESYSTEM_VERSION_1 unchanged in 9.1. Core `zlib push` + + `chan create` on 9 would also let mk4vfs use real streaming, which the 8.6 + note above says it cannot. diff --git a/src/buildsuites/suite_tcl86/patches/README.md b/src/buildsuites/suite_tcl86/patches/README.md new file mode 100644 index 00000000..bc1b1c32 --- /dev/null +++ b/src/buildsuites/suite_tcl86/patches/README.md @@ -0,0 +1,109 @@ +# suite_tcl86 patches + +## sdx wrap.tcl - tclkit.inf binary-read fix (G-101) + +Recovered into tracked form 2026-07-26. Same rationale as the suite_tcl90 patch +recovery: these files previously existed ONLY in VCS-ignored on-disk copies +(`scratch/` and `TEMP_REFERENCE/` are ignored by both git and fossil - root +`.gitignore:35,59` and `.fossil-settings/ignore-glob:35-36,58-59`), so a +`git clean -xdf` would have destroyed the only evidence of what the patch is. + +### What this documents + +`bin/sdx.kit` is TRACKED and is the tool `src/make.tcl` resolves for the 'kit' +wrap path (discovery ladder at src/make.tcl:5048-5097). It is NOT stock sdx: it +is a locally-patched build whose patch was recorded nowhere in version control. + +Two distinct sdx binaries were found on the author's machine, differing by 121 +bytes: + +| copy | sha256 | size | VCS | +| --- | --- | --- | --- | +| `bin/sdx.kit` (== `TEMP_REFERENCE/sdx/sdx.kit`) | `5d4aa629495bb68b737dfe7280363ceb0a6653938380e8c9ccc9063a3864f531` | 119191 | tracked | +| `scratch/sdx.kit` (== `scratch/sdx-20110317.kit`) | `04dc85d4503c037f26c2b8719e45e4e20385674fbc7404481f3c171fce0a26c3` | 119312 | ignored | + +Unwrapped, both are 64 files and are IDENTICAL except `lib/app-sdx/wrap.tcl`. +That one file is the entire provenance delta, and it is the two-line change +captured here. + +### The change + +In the Windows version-resource customization path (wrap.tcl:232-239 in the +patched copy), reading `tclkit.inf`: + +``` +- set fd [open [file join $idir tclkit.inf]] ++ set fd [open [file join $idir tclkit.inf] "r"] ++ fconfigure $fd -translation binary +``` + +The array read from that file is handed straight to +`::stringfileinfo::writeStringInfo`, which patches the PE version resource of the +output executable. Without binary translation the read is line-ending-translated, +so this reads as a fix for CRLF corruption of `tclkit.inf` content on Windows. +NOT VERIFIED against a failing case - the fix is plausible and small, but no +reproducer was recorded with it. Confirming (or retiring) it is part of the +G-101 sdx characterization work. + +Note the patch only fires when ALL of: `-runtime` was given, the runtime has a +`.exe` extension, and `tclkit.inf` exists in the source `.vfs`. No `tclkit.inf` +currently exists anywhere in this repo, so the patched path is DORMANT for +present builds - it matters only if/when version-stamping of wrapped executables +is turned on. + +### Files + +- `sdx-2011.03.17-wrap.tcl` - the STOCK baseline, byte-exact + (sha256 `41e669dce6da5b102f4924a4c177501b0ab8f3f62bf0657d88bdea73089c0a75`). + Kept in full because upstream is GONE: sdx's home was + `svn://svn.equi4.com/sdx/trunk` (per its ChangeLog, last entry 2011-03-17 by + Pat Thoyts) and it cannot be re-fetched. This is the irreplaceable artifact + here - the patched version is independently recoverable by unwrapping the + tracked `bin/sdx.kit`. +- `sdx-wrap_tclkitinf_binary.tcl` - the PATCHED file as it exists inside + `bin/sdx.kit`, byte-exact + (sha256 `7a77009be04e0da5fb750d86a6c1e7141bf712422434d0908456e40766b847be`). + Kept as a drop-in per the suite_tcl90 convention (full patched file alongside + the diff), so a rebuild does not depend on a working `patch` tool. +- `sdx-wrap.tcl.tclkitinf-binary.patch` - the diff between the two. Round-trip + verified 2026-07-26: applying it to the stock baseline reproduces the patched + file byte for byte. Apply with `patch --binary` on Windows - without it the + CRLF context lines will not match. + +LINE ENDINGS: the two `.tcl` files are CRLF and MUST STAY CRLF. They are vendored +fidelity artifacts whose bytes must match the kit contents; the repo's usual +LF-on-edit preference does not apply to them. Do not run a line-ending normalizer +over this directory. This README is LF. + +### Verifying the capture + +`sdx version` walks a mounted kit and prints ` -` +(`lib/app-sdx/version.tcl`). Observed under +`bin/runtime/win32-x86_64/tclkit86bi.exe`: + +``` +bin/sdx.kit -> 2025/06/04 22:56:36 69170-33997 +scratch/sdx.kit -> 2011/03/17 23:39:41 50768-16003 +``` + +The tracked kit's timestamp is exactly the patched `wrap.tcl` mtime, which is +what confirms the patch is inside the tracked kit rather than only on disk. + +TRAP for any rebuild-and-compare: `traverse` hashes file NAME + MTIME + SIZE, not +content, so this signature is NOT reproducible across a re-wrap unless mtimes are +normalised (`sdx version -fixtime`). Content-level characterization needs its own +comparison - unwrap and diff trees, plus behavioural wrap/unwrap/mksplit +round-trips - not this signature alone. + +### Why this lives here + +Placed under the suite that owns the question rather than beside the binary: +`bin/` is documented as build outputs and tooling (bin/AGENTS.md), not source +material. G-101 (kit container strategy for zig-built 8.6 runtimes) owns the sdx +provenance and rebuild decision, and sdx is the kit-wrapping tool that goal must +either keep, rebuild, or replace. If G-101 replaces sdx with a suite-owned kit +writer, this capture stays as the record of what the retired tool did. + +Full findings - including the bootstrap circularity (sdx.kit is itself a metakit +starkit and needs a tclkit to run), the 2011-era bundled library set, and the +stale `bin/sdx1.bat` wrapper - are in `goals/G-101-tcl86-kit-container-strategy.md`. diff --git a/src/buildsuites/suite_tcl86/patches/sdx-2011.03.17-wrap.tcl b/src/buildsuites/suite_tcl86/patches/sdx-2011.03.17-wrap.tcl new file mode 100644 index 00000000..7c1cdb08 --- /dev/null +++ b/src/buildsuites/suite_tcl86/patches/sdx-2011.03.17-wrap.tcl @@ -0,0 +1,344 @@ +# wrap - convert file system tree to a starkit +# by Matt Newman and Jean-Claude Wippler +# +# 20020607 jcw added -writeable, switched to starkit package +# the default is now to creates a read-only starkit + +set header \ +{#!/bin/sh +# % +exec @PROG@ "$0" ${1+"$@"} +package require starkit +starkit::header @TYPE@ @OPTS@ +} +append header \32 +regsub % $header \\ header + +if {[llength $argv] < 1} { + puts stderr "Usage: $argv0 outfile ?-option ...?" + exit 1 +} + +set origtime [clock seconds] + +proc readfile {name} { + set fd [open $name] + fconfigure $fd -translation binary + set data [read $fd] + close $fd + return $data +} + +proc writefile {name data} { + set fd [open $name w] + fconfigure $fd -translation binary + puts -nonewline $fd $data + close $fd +} + +# decode Windows .ICO file contents into the individual bit maps +# Each icon header has: +# byte : width in pixels (0 for 256 px) +# byte : height in pixels (0 for 256px) +# byte : number of colors in palette (0 for truecolor) +# byte : reserved (0) +# short: number of color planes (0 or 1) +# short: number of bits per pixel +# long : size of the bitmap data in bytes +# long : offset to the start of bitmap data +proc decICO {dat} { + set result {} + binary scan $dat sss - type count + for {set pos 6} {[incr count -1] >= 0} {incr pos 16} { + binary scan $dat @${pos}ccccssii w h cc - p bc bir io + if {$cc == 0} { set cc 256 } + #puts "pos $pos w $w h $h cc $cc p $p bc $bc bir $bir io $io" + binary scan $dat @${io}a$bir image + lappend result ${w}x${h}/$bc $image + } + return $result +} + +proc LoadHeader {filename} { + if {[file normalize $filename] == [info nameofexe]} { + puts stderr "file in use, cannot be prefix: [file normalize $filename]" + exit 1 + } + set size [file size $filename] + catch { + package require vfs::mk4 + vfs::mk4::Mount $filename hdr -readonly + # we only look for an icon if the runtime is called *.exe (!) + if {[string tolower [file extension $filename]] == ".exe"} { + catch { set ::origicon [readfile hdr/tclkit.ico] } + } + } + catch { vfs::unmount $filename } + return [readfile $filename] +} + +set out [lindex $argv 0] +set base [file root [file tail $out]] +set idir $base.vfs +if {![file isdirectory $idir]} { + if {[file isdirectory $base] && [file exists $base/main.tcl]} { + set idir $base + if {[lindex [split $out .] 0] eq $out} { + # append kit if user hasn't specified an extension + set out $base.kit + } + } +} +set compress 1 +set verbose 0 +set ropts -readonly +set prefix 0 +set reusefile 0 +set prog tclkit +set type mk4 +set explist {} +set syncopts {} + +set a [lindex $argv 1] +while {[string match -* $a]} { + switch -- $a { + -interp { + set prog [lindex $argv 2] + set argv [lreplace $argv 1 2] + } + -runtime { + set pfile [lindex $argv 2] + if {$pfile == $out} { + set reusefile 1 + } else { + set header [LoadHeader $pfile] + } + set argv [lreplace $argv 1 2] + set prefix 1 + } + -writable - + -writeable { + #set ropts "-nocommit" + set ropts "" + set argv [lreplace $argv 1 1] + } + -nocomp - + -nocompress { + set compress 0 + set argv [lreplace $argv 1 1] + } + -verbose { + set verbose 1 + set argv [lreplace $argv 1 1] + } + -uncomp { + lappend explist [lindex $argv 2] + set argv [lreplace $argv 1 2] + } + -vfs { + set idir [lindex $argv 2] + set argv [lreplace $argv 1 2] + } + default { + lappend syncopts [lindex $argv 1] [lindex $argv 2] + set argv [lreplace $argv 1 2] + } + } + set a [lindex $argv 1] +} + +if {![file isdir $idir]} { + puts stderr "Input directory not found: $idir" + exit 1 +} + +if {!$prefix} { + regsub @PROG@ $header $prog header + regsub @OPTS@ $header $ropts header + regsub @TYPE@ $header $type header + + set n [string length $header] + while {$n <= 240} { + append header ################ + incr n 16 + } + + set slop [expr { 15 - (($n + 15) % 16) }] + for {set i 0} {$i < $slop} {incr i} { + append header # + } + set n [string length $header] + if {$n % 16 != 0} { + error "Header size is $n, should be a multiple of 16" + } +} + +if {!$reusefile} { + writefile $out $header +} + +set origsize [file size $out] + +switch $tcl_platform(platform) { + unix { + catch {file attributes $out -permissions +x} + } + windows { + set batfile [file root $out].bat + # 2005-03-18 don't create a batfile if "-runtime" is specified + if {![file exists $batfile] && ![info exists pfile]} { + set fd [open $batfile w] + puts -nonewline $fd \ + "@$prog [file tail $out] %1 %2 %3 %4 %5 %6 %7 %8 %9" + close $fd + } + } + macintosh { + catch {file attributes $out -creator TKd4} + } +} + +# 2003-02-08: added code to patch icon in windows executable +# triggered by existence of tclkit.ico in vfs dir *and* tclkit.ico in orig + +# careful: this applies only to windows executables, but the +# icon replacement can in fact take place on any platform... + +if {[info exists origicon] && [file exists [file join $idir tclkit.ico]]} { + puts " customizing tclkit.ico in executable" + set custicon [readfile [file join $idir tclkit.ico]] + array set newimg [decICO $custicon] + foreach {k v} [decICO $origicon] { + if {[info exists newimg($k)]} { + set len [string length $v] + set pos [string first $v $header] + if {$pos < 0} { + puts " icon $k: NOT FOUND" + } elseif {[string length $newimg($k)] != $len} { + puts " icon $k: NOT SAME SIZE" + } else { + binary scan $header a${pos}a${len}a* prefix - suffix + set header "$prefix$newimg($k)$suffix" + puts " icon $k: replaced" + } + } + } + writefile $out $header +} + +# 2005-03-15 added AF's code to customize version/description strings in exe's +if {[info exists pfile] && + [string tolower [file extension $pfile]] == ".exe" && + [file exists [file join $idir tclkit.inf]]} { + puts " customizing strings in executable" + package require stringfileinfo + set fd [open [file join $idir tclkit.inf]] + array set strinfo [read $fd] + close $fd + ::stringfileinfo::writeStringInfo $out strinfo +} + +proc tclLog {msg} { puts $msg } + +proc mkFileStart {filename} { + set end [file size $filename] + if {$end < 27} { + fail "file too small, cannot be a datafile" + } + + set fd [open $filename] + fconfigure $fd -translation binary + seek $fd -16 end + binary scan [read $fd 16] IIII a b c d + close $fd + + #puts [format %x-%d-%x-%d $a $b $c $d] + + if {($c >> 24) != -128} { + error "this is not a Metakit datafile" + } + + # avoid negative sign / overflow issues + if {[format %x [expr {$a & 0xffffffff}]] eq "80000000"} { + set start [expr {$end - 16 - $b}] + } else { + # if the file is in commit-progress state, we need to do more + error "this code needs to be finished..." + } + + return $start +} + +if {![catch { package require Mk4tcl }]} { + vfs::mk4::Mount $out $out + set argv $syncopts + lappend argv -compress $compress -verbose $verbose -noerror 0 $idir $out + + source [file join [file dirname [info script]] sync.tcl] + + # 2003-06-19: new "-uncomp name" option to store specific file(s) + # in uncompressed form, even if the rest is compressed + set o $vfs::mk4::compress + set vfs::mk4::compress 0 + foreach f $explist { + file delete -force [file join $out $f] + file copy [file join $idir $f] [file join $out $f] + } + set vfs::mk4::compress $o + + vfs::unmount $out +} elseif {![catch { package require vlerq }]} { + package require vfs::m2m 1.8 + + if {$verbose} { + puts "Mk4tcl not found, switching to the Vlerq driver." + } + + # 2006-12-06, must do a bit more work for starpacks when there is no Mk4tcl: + # - split the tail off + # - truncate the file so only the head remains + # - copy the original vfs contents to m2m, to be overwritten/extended later + + set outsize [file size $out] + + if {![catch { mkFileStart $out } mkpos]} { + set fd [open $out] + fconfigure $fd -translation binary + set outhead [read $fd $mkpos] + set origvfs [read $fd] + close $fd + + set fd [open $out w] + fconfigure $fd -translation binary + puts -nonewline $fd $outhead + close $fd + } + + vfs::m2m::Mount $out $out + + if {[info exists origvfs]} { + set fd [open $out.tmp w] + fconfigure $fd -translation binary + puts -nonewline $fd $origvfs + close $fd + + package require vfs::mkcl + vfs::mkcl::Mount $out.tmp $out.tmp + + set argv [list -verbose 0 $out.tmp $out] + source [file join [file dirname [info script]] sync.tcl] + + vfs::unmount $out.tmp + file delete $out.tmp + } + + set argv $syncopts + lappend argv -verbose $verbose -noerror 0 $idir $out + + source [file join [file dirname [info script]] sync.tcl] + + vfs::unmount $out +} else { + puts stderr "cannot find required packages (Mk4tcl or Vlerq)" + exit 1 +} diff --git a/src/buildsuites/suite_tcl86/patches/sdx-wrap.tcl.tclkitinf-binary.patch b/src/buildsuites/suite_tcl86/patches/sdx-wrap.tcl.tclkitinf-binary.patch new file mode 100644 index 00000000..e62e79b2 --- /dev/null +++ b/src/buildsuites/suite_tcl86/patches/sdx-wrap.tcl.tclkitinf-binary.patch @@ -0,0 +1,12 @@ +--- a/lib/app-sdx/wrap.tcl (sdx 2011-03-17 stock) ++++ b/lib/app-sdx/wrap.tcl (bin/sdx.kit patched) +@@ -232,7 +232,8 @@ + [file exists [file join $idir tclkit.inf]]} { + puts " customizing strings in executable" + package require stringfileinfo +- set fd [open [file join $idir tclkit.inf]] ++ set fd [open [file join $idir tclkit.inf] "r"] ++ fconfigure $fd -translation binary + array set strinfo [read $fd] + close $fd + ::stringfileinfo::writeStringInfo $out strinfo diff --git a/src/buildsuites/suite_tcl86/patches/sdx-wrap_tclkitinf_binary.tcl b/src/buildsuites/suite_tcl86/patches/sdx-wrap_tclkitinf_binary.tcl new file mode 100644 index 00000000..a13463d5 --- /dev/null +++ b/src/buildsuites/suite_tcl86/patches/sdx-wrap_tclkitinf_binary.tcl @@ -0,0 +1,345 @@ +# wrap - convert file system tree to a starkit +# by Matt Newman and Jean-Claude Wippler +# +# 20020607 jcw added -writeable, switched to starkit package +# the default is now to creates a read-only starkit + +set header \ +{#!/bin/sh +# % +exec @PROG@ "$0" ${1+"$@"} +package require starkit +starkit::header @TYPE@ @OPTS@ +} +append header \32 +regsub % $header \\ header + +if {[llength $argv] < 1} { + puts stderr "Usage: $argv0 outfile ?-option ...?" + exit 1 +} + +set origtime [clock seconds] + +proc readfile {name} { + set fd [open $name] + fconfigure $fd -translation binary + set data [read $fd] + close $fd + return $data +} + +proc writefile {name data} { + set fd [open $name w] + fconfigure $fd -translation binary + puts -nonewline $fd $data + close $fd +} + +# decode Windows .ICO file contents into the individual bit maps +# Each icon header has: +# byte : width in pixels (0 for 256 px) +# byte : height in pixels (0 for 256px) +# byte : number of colors in palette (0 for truecolor) +# byte : reserved (0) +# short: number of color planes (0 or 1) +# short: number of bits per pixel +# long : size of the bitmap data in bytes +# long : offset to the start of bitmap data +proc decICO {dat} { + set result {} + binary scan $dat sss - type count + for {set pos 6} {[incr count -1] >= 0} {incr pos 16} { + binary scan $dat @${pos}ccccssii w h cc - p bc bir io + if {$cc == 0} { set cc 256 } + #puts "pos $pos w $w h $h cc $cc p $p bc $bc bir $bir io $io" + binary scan $dat @${io}a$bir image + lappend result ${w}x${h}/$bc $image + } + return $result +} + +proc LoadHeader {filename} { + if {[file normalize $filename] == [info nameofexe]} { + puts stderr "file in use, cannot be prefix: [file normalize $filename]" + exit 1 + } + set size [file size $filename] + catch { + package require vfs::mk4 + vfs::mk4::Mount $filename hdr -readonly + # we only look for an icon if the runtime is called *.exe (!) + if {[string tolower [file extension $filename]] == ".exe"} { + catch { set ::origicon [readfile hdr/tclkit.ico] } + } + } + catch { vfs::unmount $filename } + return [readfile $filename] +} + +set out [lindex $argv 0] +set base [file root [file tail $out]] +set idir $base.vfs +if {![file isdirectory $idir]} { + if {[file isdirectory $base] && [file exists $base/main.tcl]} { + set idir $base + if {[lindex [split $out .] 0] eq $out} { + # append kit if user hasn't specified an extension + set out $base.kit + } + } +} +set compress 1 +set verbose 0 +set ropts -readonly +set prefix 0 +set reusefile 0 +set prog tclkit +set type mk4 +set explist {} +set syncopts {} + +set a [lindex $argv 1] +while {[string match -* $a]} { + switch -- $a { + -interp { + set prog [lindex $argv 2] + set argv [lreplace $argv 1 2] + } + -runtime { + set pfile [lindex $argv 2] + if {$pfile == $out} { + set reusefile 1 + } else { + set header [LoadHeader $pfile] + } + set argv [lreplace $argv 1 2] + set prefix 1 + } + -writable - + -writeable { + #set ropts "-nocommit" + set ropts "" + set argv [lreplace $argv 1 1] + } + -nocomp - + -nocompress { + set compress 0 + set argv [lreplace $argv 1 1] + } + -verbose { + set verbose 1 + set argv [lreplace $argv 1 1] + } + -uncomp { + lappend explist [lindex $argv 2] + set argv [lreplace $argv 1 2] + } + -vfs { + set idir [lindex $argv 2] + set argv [lreplace $argv 1 2] + } + default { + lappend syncopts [lindex $argv 1] [lindex $argv 2] + set argv [lreplace $argv 1 2] + } + } + set a [lindex $argv 1] +} + +if {![file isdir $idir]} { + puts stderr "Input directory not found: $idir" + exit 1 +} + +if {!$prefix} { + regsub @PROG@ $header $prog header + regsub @OPTS@ $header $ropts header + regsub @TYPE@ $header $type header + + set n [string length $header] + while {$n <= 240} { + append header ################ + incr n 16 + } + + set slop [expr { 15 - (($n + 15) % 16) }] + for {set i 0} {$i < $slop} {incr i} { + append header # + } + set n [string length $header] + if {$n % 16 != 0} { + error "Header size is $n, should be a multiple of 16" + } +} + +if {!$reusefile} { + writefile $out $header +} + +set origsize [file size $out] + +switch $tcl_platform(platform) { + unix { + catch {file attributes $out -permissions +x} + } + windows { + set batfile [file root $out].bat + # 2005-03-18 don't create a batfile if "-runtime" is specified + if {![file exists $batfile] && ![info exists pfile]} { + set fd [open $batfile w] + puts -nonewline $fd \ + "@$prog [file tail $out] %1 %2 %3 %4 %5 %6 %7 %8 %9" + close $fd + } + } + macintosh { + catch {file attributes $out -creator TKd4} + } +} + +# 2003-02-08: added code to patch icon in windows executable +# triggered by existence of tclkit.ico in vfs dir *and* tclkit.ico in orig + +# careful: this applies only to windows executables, but the +# icon replacement can in fact take place on any platform... + +if {[info exists origicon] && [file exists [file join $idir tclkit.ico]]} { + puts " customizing tclkit.ico in executable" + set custicon [readfile [file join $idir tclkit.ico]] + array set newimg [decICO $custicon] + foreach {k v} [decICO $origicon] { + if {[info exists newimg($k)]} { + set len [string length $v] + set pos [string first $v $header] + if {$pos < 0} { + puts " icon $k: NOT FOUND" + } elseif {[string length $newimg($k)] != $len} { + puts " icon $k: NOT SAME SIZE" + } else { + binary scan $header a${pos}a${len}a* prefix - suffix + set header "$prefix$newimg($k)$suffix" + puts " icon $k: replaced" + } + } + } + writefile $out $header +} + +# 2005-03-15 added AF's code to customize version/description strings in exe's +if {[info exists pfile] && + [string tolower [file extension $pfile]] == ".exe" && + [file exists [file join $idir tclkit.inf]]} { + puts " customizing strings in executable" + package require stringfileinfo + set fd [open [file join $idir tclkit.inf] "r"] + fconfigure $fd -translation binary + array set strinfo [read $fd] + close $fd + ::stringfileinfo::writeStringInfo $out strinfo +} + +proc tclLog {msg} { puts $msg } + +proc mkFileStart {filename} { + set end [file size $filename] + if {$end < 27} { + fail "file too small, cannot be a datafile" + } + + set fd [open $filename] + fconfigure $fd -translation binary + seek $fd -16 end + binary scan [read $fd 16] IIII a b c d + close $fd + + #puts [format %x-%d-%x-%d $a $b $c $d] + + if {($c >> 24) != -128} { + error "this is not a Metakit datafile" + } + + # avoid negative sign / overflow issues + if {[format %x [expr {$a & 0xffffffff}]] eq "80000000"} { + set start [expr {$end - 16 - $b}] + } else { + # if the file is in commit-progress state, we need to do more + error "this code needs to be finished..." + } + + return $start +} + +if {![catch { package require Mk4tcl }]} { + vfs::mk4::Mount $out $out + set argv $syncopts + lappend argv -compress $compress -verbose $verbose -noerror 0 $idir $out + + source [file join [file dirname [info script]] sync.tcl] + + # 2003-06-19: new "-uncomp name" option to store specific file(s) + # in uncompressed form, even if the rest is compressed + set o $vfs::mk4::compress + set vfs::mk4::compress 0 + foreach f $explist { + file delete -force [file join $out $f] + file copy [file join $idir $f] [file join $out $f] + } + set vfs::mk4::compress $o + + vfs::unmount $out +} elseif {![catch { package require vlerq }]} { + package require vfs::m2m 1.8 + + if {$verbose} { + puts "Mk4tcl not found, switching to the Vlerq driver." + } + + # 2006-12-06, must do a bit more work for starpacks when there is no Mk4tcl: + # - split the tail off + # - truncate the file so only the head remains + # - copy the original vfs contents to m2m, to be overwritten/extended later + + set outsize [file size $out] + + if {![catch { mkFileStart $out } mkpos]} { + set fd [open $out] + fconfigure $fd -translation binary + set outhead [read $fd $mkpos] + set origvfs [read $fd] + close $fd + + set fd [open $out w] + fconfigure $fd -translation binary + puts -nonewline $fd $outhead + close $fd + } + + vfs::m2m::Mount $out $out + + if {[info exists origvfs]} { + set fd [open $out.tmp w] + fconfigure $fd -translation binary + puts -nonewline $fd $origvfs + close $fd + + package require vfs::mkcl + vfs::mkcl::Mount $out.tmp $out.tmp + + set argv [list -verbose 0 $out.tmp $out] + source [file join [file dirname [info script]] sync.tcl] + + vfs::unmount $out.tmp + file delete $out.tmp + } + + set argv $syncopts + lappend argv -verbose $verbose -noerror 0 $idir $out + + source [file join [file dirname [info script]] sync.tcl] + + vfs::unmount $out +} else { + puts stderr "cannot find required packages (Mk4tcl or Vlerq)" + exit 1 +}