jn
/
punkshell
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
1211 Commits
1 Branch
1 Tag
192 MiB
 
 
 
 
 
 
Tree: 768dab399f
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '768dab399f'
${ noResults }
punkshell/bin/AGENTS.md

23 KiB
Raw Blame History

bin/

Purpose

Built punk shell executables (kits with the punk boot layer), assorted build/experiment tooling, and plain runtime kits under runtime/. Executables here are build outputs - agents do not hand-edit binaries.

Generated polyglot .cmd scripts - never edit in place

The .cmd scripts here (e.g punk-runtime.cmd) are punk MULTISHELL polyglots GENERATED by punk::mix::commandset::scriptwrap::multishell from scriptset sources under src/scriptapps/bin/ (the home for bin-deployed scriptsets, e.g punk-runtime.* alongside punk-getzig.*) or src/scriptapps/ itself (e.g punk-tclargs.*, dtplite.*): payload scripts (<name>.ps1, <name>.bash, <name>.tcl, ...) plus a <name>_wrap.toml config, spliced into the punk.multishell.cmd template. The polyglot structure is deliberately fragile (mutual shell-hiding tricks, LF-only endings, cmd.exe's 512-byte label-scanner constraints) - a hand-edit can silently break one of the participating shells.

When asked to "fix bin/.cmd":

  1. Edit the payload/config sources under src/scriptapps/bin/ (never the output file).
  2. Re-wrap from the scriptset's folder: cd src/scriptapps/bin then punk::mix::commandset::scriptwrap::multishell <name> -askme 0 (output defaults to <projectroot>/bin; requires Thread + the punk::mix modules, e.g a punk shell or a tclsh with project module paths).
  3. The wrapper runs checkfile (the 512-byte label/boundary validator) automatically - heed ERROR output; "possibly bogus target" warnings are normal for polyglots. Payload growth can push a TEMPLATE label beyond the payloads (e.g :exit_multishell) across a 512-byte boundary: fix by resizing the byte-alignment spacer comment at the end of the affected payload (see the (512B spacer) comment in punk-runtime.bash) and re-wrapping until checkfile is ERROR-free.
  4. Commit source AND regenerated output together: the scriptwrap test suite pins that a re-wrap of the punk-runtime scriptset reproduces bin/punk-runtime.cmd byte for byte (src/tests/modules/punk/mix/testsuites/scriptwrap/runtimecmd_roundtrip.test scriptwrap_runtime_cmd_roundtrip_no_drift), so drift in either direction fails tests. bin/dtplite.cmd is pinned the same way (src/tests/shell/testsuites/binscripts/dtplite.test dtplite_cmd_roundtrip_no_drift).

dtplite (dtplite.cmd)

bin/dtplite.cmd wraps tcllib's dtplite doctools processor (payload src/scriptapps/dtplite.tcl, tclsh nextshell on all platforms). It exists so the punk repl's unknown-handler can resolve the bare dtplite command via PATH - dev doc.validate (punk::mix::commandset::doc) depends on it. The payload falls back to the project-vendored tcllib (src/vendorlib_tcl9/<arch>/tcllib*) when the invoking tclsh has no dtplite package installed.

Local Contracts

Utility naming policy: punk- prefix (G-096)

Punkshell-own utilities deployed to bin/ take a punk- name prefix (e.g punk-getzig) so a user who adds <project>/bin to their PATH gets collision-resistant names. Exceptions: wrappers whose purpose is to present an external tool under its own name (dtplite, sdx, kettle), and getpunk (already punkshell-specific; intended future single-download cross-platform entry point). Policy decided by the user 2026-07-20 (G-096); the remaining pre-policy names were swept under G-097 (2026-07-21): punk-bits, punk-runtime, punk-tclargs and the punk- prefixed selfsign experiment scripts.

.ps1 twin (corrected 2026-07-22 - the twin is SELF-MATERIALIZED, not manually refreshed): the generated .cmd's batch layer creates <name>.ps1 beside itself when missing AND fc-compares/re-copies it whenever content differs (powershell needs a .ps1 extension for -File), so the twin regenerates on any .cmd launch and self-heals after a re-wrap - no manual copy step. The twin is a byte-copy of the polyglot and is VCS-ignored (a runtime artifact, unlike the committed .cmd). Caveats: a user who only ever launches the .ps1 directly can ride a stale twin until the next .cmd-path launch heals it; and the two launch paths run DIFFERENT powershells - ./bin/<name> in a powershell session resolves the .ps1 and runs it under the INVOKING shell (pwsh 7 or powershell 5), while the .cmd route is pinned by the wrap toml (cmd.exe /c powershell = Windows PowerShell) - payload code must be edition-portable and avoid implementation-defined behaviour (e.g. Hashtable enumeration order differs between the editions; sort explicitly).

INVOCATION GUIDELINE for user-facing documentation (user policy 2026-07-22; no user-facing guidelines existed yet when recorded - apply to all future README/help/ doc examples involving the polyglot bin/*.cmd utilities): show invocations WITH the .cmd extension, e.g. bin/punk-runtime.cmd list, even though extensionless invocation happens to work on windows in some cases. Rationale: extensionless resolution in powershell prefers the .ps1 twin, whose direct execution is ExecutionPolicy-gated (Restricted/AllSigned hosts fail) and runs under the invoking edition, whereas the .cmd always executes and follows the wrap-pinned, policy- bypassing tested path. The SAME .cmd file also runs on unix shells (the polyglot needs no extension stripping - ./bin/punk-runtime.cmd from bash/zsh, or bash bin/punk-runtime.cmd), so one form serves all platforms and the only platform-specific difference left in examples is path separator style (bin/punk-runtime.cmd vs bin\punk-runtime.cmd).

Launch package modes (built punk shells)

The first argument to a built punk shell may be a dash-delimited package mode composed of tokens from dev, os, src, internal (e.g. punksys src, punk902z dev-os), optionally scoped with the proj: prefix (e.g. punk91 proj:internal-src). internal is the default and is always appended when absent. A first argument that is not a valid mode list is treated as a subcommand instead (a proj: prefix on a non-mode string is not consumed either). Implementation: src/vfs/_config/punk_main.tcl (search all_package_modes). The full packagemode contract (block meanings, ordering, proj: examples) is the punk::args documentation at id (script)::punkexe (module punk::args::moduledoc::punkexe).

src mode is the one that matters for verifying working-tree changes:

  • Discovers the project root from the executable's location (exe in bin/ -> parent directory), not from the cwd - it works from any working directory as long as the executable resides in the project's bin/.
  • Prepends src/modules, src/modules_tcl<N>, src/bootsupport/modules{,_tcl<N>} and src/vendormodules{,_tcl<N>} to the module path, sets package prefer latest so dev-numbered 999999.0a1.0 source modules beat kit-stamped snapshots on unversioned package require, and registers #modpod modules from src/modules (startup notice: src mode: registered N #modpod modules from .../src/modules).
  • Consequence: punksys src / punk902z src exercises current working-tree source with no kit rebuild; a plain launch (punksys) exercises the module snapshots baked into the kit at build time. package present <pkg> reporting 999999.0a1.0 (vs a release version like 0.7.1) tells you which set a session is running - runbooks with version expectations must state the intended launch mode.

proj: scope (G-033, achieved - see goals/archive/G-033-proj-mode-cwd-project.md) makes dev/src resolve against the project containing the CURRENT DIRECTORY instead of the executable's own project - visitor mode for an installed/standalone punk binary exploring a project that builds no shell of its own (the project may be modules-only, e.g. tomlish):

  • Root discovery walks up from the cwd to the nearest git/fossil repo root that looks like a punkshell-style project (has ./src plus one of src/modules, src/vfs, src/scriptapps, or punkproject.toml - a lean boot mirror of punk::repo::is_project_root).
  • The launch always reports the detected root and effective mode precedence on stderr; when no project is found it warns and proceeds with dev/src unbound (never a silent or false rebind, no exe-relative fallback). proj: with no dev/src block warns that it has no effect.
  • Canonical visitor invocation: <punkexe> proj:internal-src shell - kit copies win same-version ties, protecting the visiting shell's own infrastructure. proj:src is the faithful-vintage variant where the visited project's copies win ties. The project's genuinely-own code (dev-versioned or absent from the kit) loads from the project in either order.

Kit-wrappable runtime requirements (G-129)

What a runtime must provide for make.tcl bake to wrap it as a punk ZIP kit and for that kit to boot. (Starkit/tclkit and cookfs runtimes are separate kit types with their own conventions - this section is the zipfs family.)

  • The core zipfs command set including tcl::zipfs::mount, whose no-argument form reports the mount table as {mountpoint archivefile} pairs. tcl::zipfs::root is NOT required: both the kit boot and the bake's runtime capability probe key zipfs presence on tcl::zipfs::mount. Verified generations: Tcl 8.7/9 core zipfs, and the androwish/undroidwish Tcl 8.6 zipfs backport (no root; its ::zipfs ensemble also lacks a root subcommand).
  • Any attached-archive mount convention, because the boot derives the payload base from the mount table itself (the entry whose archive file is [info nameofexecutable], fallback the mount containing the boot script): modern runtimes mount at the compiled-in //zipfs:/app; the 8.6 backport family mounts at the EXECUTABLE'S OWN PATH and reports the archive file without its windows drive prefix (both handled). A mounted archive the boot cannot attribute to the executable is reported on stderr rather than silently configuring no internal paths.
  • A startup hook that sources a script from the attached archive: modern zipfs runtimes run main.tcl at the app-mount root; the undroidwish family looks for app/main.tcl INSIDE the archive. src/vfs/punk8win.vfs carries root main.tcl AND app/main.tcl fauxlinks so one payload boots either family.
  • Either zip offset convention in the assembled kit: zipfs mkimg (file-relative offsets) and raw concatenation (archive-relative) both mount on both verified families (measured 2026-07-27 with the LUCK backport runtime).
  • A payload the bake can lift: the runtime's own library must be readable from its attached archive (the G-124 punk::zip reader handles either offset convention) and the merged kit must pass the G-125 boot-precondition gate (tcl_library/, lib/tcl<M>.<m>/ or root tcl<M>.<m>/ library conventions).

The runtime/win32-ix86/ tier holds the two third-party 32-bit verification runtimes bracketing the mount conventions: tclsh8.6.10-luck-zip.exe (exe-path mount; reproducible via LUCK.url + the base64 build request in tclsh8.6.10-luck-zip.cfg) and tclsh9.1b0-tclsfe.exe (modern //zipfs:/app; apnadkarni/tcl-sfe releases), wrapped as punkluck86 / punk91ix86 in src/runtime/mapvfs.config. Characterization: src/tests/shell/testsuites/punkexe/kitmountpoint.test.

What a bake verifies about a deployed kit (G-125, G-133)

A bin/<kit> that a bake deployed has passed the G-125 boot-precondition gate (its merged vfs supplies a tcl library - a kit failing that is never deployed) and the G-133 advisory checks have run: a structural binary-arch scan of its payload (a wrong-arch .dll/.so/.dylib outside a platform-discriminated subdir earned a recapped BUILD-WARNING - advisory, so heed the bake recap before trusting a warned kit) and, when its src/runtime/mapvfs.config entry declares smoke-require packages and the kit is host-runnable, a plain package require of each inside the built artifact via its tclsh subcommand. What that does NOT guarantee: statically linked packages and pure-tcl packages with binary dependencies are invisible to the arch scan; package-resolution outcomes (a wrong-arch higher-versioned copy shadowing a working one) are visible only to the smoke probe and only for the declared package set; and an unclassifiable binary is silence, not a pass. A kit whose bake recap showed no BUILD-WARNINGs still only proves what those checks measure - not that every payload package loads.

Runtime fetch/selection (punk-runtime.cmd)

bin/punk-runtime.cmd {fetch|list|use|run} manages the plain runtimes under bin/runtime/<platform>/ (retrieved from the punkbin artifact repo with sha1 verification against its sha1sums.txt - both the powershell payload on windows and the bash payload on unix verify; a fetch whose checksum fails leaves only a .tmp). The <platform> folders are store TIERS keyed by the platform a runtime is FOR, and the kit build reads them the same way: src/make.tcl picks a tier per kit from that kit's target platform, not from the driving tclsh's own personality (G-122 - see src/AGENTS.md and src/runtime/AGENTS.md), so a tier holding runtimes for another platform is a first-class thing to populate here. Hashing is deliberately dependency-free in both payloads: the powershell payload computes sha1 via .NET (Get-PunkFileSha1) rather than Get-FileHash, because Get-FileHash is a script-defined function in Windows PowerShell 5.1 resolved through PSModulePath and vanishes on machines with a damaged PSModulePath while compiled cmdlets still work (field-observed; a one-time note flags the condition); the bash payload probes sha1sum/shasum/sha1/openssl. Keep new payload code free of PS 5.1 script-module-defined cmdlets (Get-FileHash, New-TemporaryFile, New-Guid, Import-PowerShellDataFile, ...) for the same reason. list -remote compares local runtimes against the server's sha1sums (Same version / UPDATE AVAILABLE / not-listed, plus remote-only entries; cached sha1sums.txt fallback with a warning when the server is unreachable). It also surfaces the selection state: summary lines show the platform's server default (from defaults.txt, best-effort) and the locally active runtime (annotated (= server default) when they match), the active runtime's row carries the same * marker the local listing uses, and the default's row - local or remote-only - is annotated (server default). The artifact server base url is overridable via PUNKBIN_URL (mirrors/testing) in both payloads. Which runtime run launches is the per-machine "active" selection in bin/runtime/<platform>/active.toml (constrained single-key toml active = "<name>", written by punk-runtime.cmd use <name>, marked in list, VCS-ignored via the existing bin/* globs). Resolution order for run: PUNK_ACTIVE_RUNTIME env override, then active.toml, then a sole installed candidate - with multiple runtimes and no selection it errors listing candidates rather than guessing. The first fetch sets the active runtime only when none is recorded.

G-103 runtime-family artifact metadata (both payloads): a runtime may carry a <rootname>.toml metadata record beside it (emitted by the suite_tcl90 kit-family-artifacts step; fetched from punkbin alongside -r<N>-named artifacts - absence tolerated for pre-family runtimes). list shows a per-runtime summary from it (variant, tcl patchlevel, revision, piperepl policy, and - on a materialized working copy - which immutable artifact it came from). use <artifact-r<N>-name> MATERIALIZES the immutable artifact into its WORKING name (the name minus -r<N> - what mapvfs and projects reference), copies the metadata toml alongside, and selects the working name; use <workingname> selects as before. use accepts names with or without the .exe suffix (2026-07-25; exact names - incl. extensionless unix artifacts in foreign -platform folders - match first, then the .exe-suffixed spelling). Root-name handling strips only a .exe suffix (dotted tcl patchlevels make generic last-dot stripping wrong for extensionless unix names). Candidate listing excludes directories, *_BUILDCOPY* names, and SUPPORT FILES recognised by extension - docs/text (.txt .md .html .htm .pdf .url), config/metadata (.toml .json .xml .yml .yaml .ini .cfg), dev leftovers (.tm .tmp .log .bak .old), checksums/signatures (.sha1 .sha256 .asc .sig) and archives (.zip .tar .gz .bz2 .xz .7z). The same set filters use and the sha1 value-matching against the server's sha1sums (support rows ride in that file too). It is defined ONCE per payload - $PunkRuntimeSupportExt/Test-PunkRuntimeSupportName in the ps1, punk_support_ext_re/is_punk_support_name in the bash - and the two must stay in step because the listings are required to agree byte-for-byte; extend both and re-wrap. What a tier is expected to contain is documented for users in bin/runtime/README.md. Forgot-to-switch-back guard (2026-07-25): make.tcl bake/bakehouse emit a BUILD-WARNING (recapped at end of run) when a working runtime they wrap is materialized from an older revision than an -r<N> artifact present beside it - a deliberate use <old-rN> (e.g to exercise list -remote row marking) that was never switched back no longer bakes stale kits silently (see src/AGENTS.md Work Guidance).

G-119 freshness verdict (both payloads): whenever the ACTIVE runtime and the SERVER DEFAULT are both known, list -remote states their relationship explicitly instead of only displaying the two facts. freshness: lines report AHEAD/BEHIND naming both revisions (BEHIND adds a fetch+use update guidance line), an equivalent current-verdict line where the (= server default) identity annotation did not already fire, a different-series incomparable note, or a stated no-revision-basis note. Revision basis: -r<N> parsed from an artifact-named active, else the beside-toml revision field (materialized working copies); the default's revision parses from its defaults.txt name. Actives with no revision basis (pre-family: no toml, no -r<N> name) fall back to sha1 VALUE-matching against the server sha1sums - a byte-copy of a server artifact is identified by content and gains that artifact's revision verdict (support-file rows .txt/.toml/... are excluded from the match; the first ordinal/C-collation match wins in both payloads). A NO-NAME fetch (the "give me the default" form) prints the same verdict as a one-line note when the active diverges from the resolved default - silent when current, and BEHIND carries an inline use <default> hint since the artifact was just fetched. Verdict lines are byte-identical across the payloads; only the guidance lines carry each payload's program-name convention. Identity middle column (2026-07-25 follow-on, user-decided shape): list -remote rows the server listing does not name carry an UNTITLED middle column - from=<artifact> for a materialized working copy (its beside-toml source artifact; content is then verified against that artifact's server sha1, so the Remote cell shows a real status: Same version, CONTENT DIFFERS for the immutable-artifact anomaly, or (artifact not listed on server)), and sha1=<name> for a tomlless byte-copy of a server artifact (default-first-then- ordinal pick - the same rule as the freshness fallback, so row identity and verdict always name the same artifact). (not listed on server) remains only for genuinely unidentified files, and the (server default) row annotation follows artifact identity (a working copy of the default carries it). Rationale: the starred active row is the salient element - it must change with every use switch and teach the working-copy mechanics, and column position carries meaning (identity phrasing inside the Remote column would read as a server fact). Pinned by src/tests/shell/testsuites/binscripts/runtimecmd_freshness.test against a fixture punkbin server (PUNKBIN_URL override + src/tests/testsupport/httpfixture.tcl).

Cross-platform surface (G-105 groundwork; both payloads): fetch/list/use accept -platform <p> where <p> is a punkbin platform-DIR name (win32-x86_64, linux-x86_64, macosx, ... - never zig triples; the buildsuite maps triples to platform dirs at build time). Resolution: -platform arg > PUNK_RUNTIME_PLATFORM env > the detected local platform; validation is shape-only, lowercase (the artifact server's URL paths are case-sensitive - the server is the truth for what exists). Foreign-platform folders serve cross-build staging/provisioning: a foreign fetch requires an explicit runtime name (no foreign defaults), use -platform manages that folder's active.toml/materialization (the marker travels with the folder when deployed; unix exec bits are restored at deploy time), and list flags a runtime whose metadata target disagrees with the folder it sits in (!TARGET-MISMATCH). run is LOCAL ONLY - it rejects a leading -platform and ignores the env override (later run args pass through to the runtime untouched). A no-args invocation prints a short usage block; the help action gives the full operator reference (actions, options, env vars incl PUNKBIN_URL, examples). platforms ?-remote? enumerates platform folders: local bin/runtime/* dirs (local platform marked), or - with -remote - the platforms the artifact server serves, read from the server's root-level platforms.txt discovery manifest (part of the punkbin layout contract, generated by punkbin's src/build_sha1sums.tcl; third-party mirrors using the layout carry the same file - raw-file servers have no directory listing, so the manifest IS the discovery mechanism), with local presence marked and cached-copy fallback; servers without the manifest get an actionable message.

The default runtime a no-name fetch retrieves is NOT baked into the payloads: it comes from the server's root-level curated defaults.txt (<platform> <runtimename> per line - a punkbin RELEASE DECISION, hand-edited there as part of each publication change-set and validated by punkbin's src/build_sha1sums.tcl; mirrors may curate their own). The lookup keys on the resolved platform, so a no-name fetch -platform <p> works too; platforms without a recorded default, and servers without the file, produce an actionable name-it-explicitly message (cached-copy fallback on network failure). Platform names follow the CANONICAL punkshell platform-dir names defined by the punk::platform module and surfaced as help platforms in the punk shell (cpu tokens normalized: amd64->x86_64, aarch64->arm64; the bash payload's local-platform prongs emit canonical names - the FreeBSD dir was realigned from freebsd-amd64 to freebsd-x86_64 accordingly).

Interactive verification shells

  • Interactive console/repl verification should cover both Tcl generations - behaviour can differ materially (e.g. the Tcl 8.6 windows console channel driver vs the Tcl 9 rewrite). Use a Tcl 8.6-based punk shell (punksys.exe) and a current Tcl 9-based punk shell (named for the Tcl release it embeds, e.g. punk902z.exe at the time of writing - ask the user which is current rather than assuming). info patchlevel in-session confirms the runtime.
  • runtime/win32-x86_64/ holds plain tclkits/tclsh runtimes without the punk boot layer - use these for clean-environment probes isolating Tcl-level behaviour from punk (they may lack extensions such as twapi; add an external lib dir to auto_path when a probe needs one).

Work Guidance

Verification

None - build outputs; behaviour is verified via src/tests/ and interactive runbooks.

Child DOX Index

  • runtime/win32-x86_64/ - plain runtime kits (no child AGENTS.md needed; covered by this file's contracts)