From cf669c85d756a5074e24962b7906c359f303a3d7 Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Sat, 8 Aug 2026 20:34:42 +1000 Subject: [PATCH] src/AGENTS.md: make.tcl driving-interpreter guidance (tcllibc accelerators) Agent-agnostic record of the 2026-08-08 pitfall: digest-heavy make.tcl commands (packages/modules/libs, libfetch, vfscommonupdate, bake) hash via tcllib sha1 and run ~460x slower without the tcllibc critcl accelerator - measured: a packages mint burned 458+ CPU-seconds unfinished under the bare zig suite tclsh, then completed both promotion phases in minutes under an accelerated interpreter. Guidance: drive make.tcl with a machine tclsh carrying tcllib+tcllibc or a punk family runtime; the bare suite tclsh remains canonical for TEST runs only. Includes the punkcheck dead-pid lock recovery for mints killed on this account. Also corrects the dirty-run grace countdown doc to 4 seconds (changed cc92078c). Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- src/AGENTS.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/AGENTS.md b/src/AGENTS.md index ddaffc14..dad3863d 100644 --- a/src/AGENTS.md +++ b/src/AGENTS.md @@ -63,10 +63,11 @@ Recovery after a wrong path guess: - Use `tclsh src/make.tcl modules` to mint just the module packages. - Use `tclsh src/make.tcl libs` to mint just the library packages. - Use `tclsh src/make.tcl packages` to mint both modules and libraries. +- The driving interpreter matters for digest-heavy commands (`packages`/`modules`/`libs`, `libfetch`, `vfscommonupdate`, `bake`): punkcheck installs and artifact verification hash large trees via tcllib's sha1, which is ~460x slower without the tcllibc critcl accelerator (measured 2026-08-08: a `packages` mint burned 458+ CPU-seconds without finishing under the bare zig-built suite tclsh, then completed both `packages` AND `vfscommonupdate` in a few minutes under an accelerated interpreter). Drive make.tcl with a tclsh that can load tcllibc: a machine tclsh with tcllib+tcllibc installed, or a punk family runtime (`bin/runtime//tclsh*-punk*.exe` - these also exercise the punk-exe-hosted make.tcl path). The bare suite-built tclsh (`src/buildsuites/_build/*/out/bin/`) is the canonical runner for the TEST suites (src/tests/AGENTS.md) but carries no tcllibc - right for tests, wrong for make.tcl's hash-dominated commands. A mint killed mid-run for this reason is punkcheck-recoverable: check any `.punkcheck.lock` for a dead pid, delete it, rerun. - make.tcl colour is terminal-aware (G-113): piped/redirected runs (agent harnesses, CI, log capture) automatically produce fully ESC-free output with no caller action required - an ansistrip channel transform on stdout+stderr guarantees zero ESC bytes for every emitter, including module-side ones (punkcheck summaries, punk::args tables). Interactive terminal runs keep colour (stdout tty probe via the `-winsize` channel option, Tcl 8.7+/9). Precedence: `NO_COLOR` (any value) always suppresses colour; `PUNK_FORCE_COLOR`/`FORCE_COLOR` (value other than 0/false/no/off) re-enables ANSI on piped output; otherwise the probe decides. Tcl 8.6 terminals are auto-detected without dependencies: windows consoles via the console channel's utf-16 encoding signature (`-encoding unicode` - only 8.6 console channels report it; the byte-level strip transform is never pushed onto a utf-16-class channel, which it would corrupt - per-channel push, so `> file` from an 8.6 console wraps stdout only), and unix-class hosts (linux/WSL/mac, plus msys2/cygwin-runtime tclsh builds that report platform unix on windows) via the tty channel signature (real ttys expose `-mode` etc; pipes/files lack them). The force vars remain for exotic environments. stderr follows the stdout decision; `make.tcl shell` pops the transform (repl colour is the shell's own concern). The ansistrip transform also holds split ANSI sequences across write chunks, so piped output carries no orphan sequence-fragment text ('0;1m' beside table borders etc - G-145): the transform must NOT declare the `clear` op in its supported-methods list - the Tcl core delivers clear before EVERY write op (intentional, under-documented core behaviour documented as read-side/seek-scoped), and the former clear handler's drop of the per-channel carry was the fragment source; `finalize` still drops the carry. `tclsh src/make.tcl check` reports the active policy (`colour policy (G-113): mode=...` line; modes tty|forced|nocolor|piped-plain|tcl86-plain). Pinned by `src/tests/shell/testsuites/punkexe/maketclcolour.test`. Agents no longer need to set NO_COLOR for captured runs (setting it remains harmless). - Use `tclsh src/make.tcl vendorupdate` to refresh vendormodules from config. It warns (non-fatal) for each source project whose fossil/git checkout is dirty - vendored artifacts built from a dirty tree have no committed provenance; commit in the source project to clear the warning (enforcement policy tracked by goal G-026). - All producing commands (mint/promote/bake: `bakehouse`, `packages`, `modules`, `libs`, `bake`, `vfslibs`, `bin`, `bootsupport`, `vfscommonupdate`) warn similarly when this project's own `src/` has uncommitted fossil/git changes (dirt outside `src/` is ignored). Warn-only by default EXCEPT `bakehouse` (aborts by default; `-dirty-abort 0` overrides); pass `-dirty-abort` elsewhere to make the check aborting. For evaluating uncommitted source without minting or baking, prefer ` src` / ` src shell`. The check is `punkboot::utils::vcs_dirty_warnings` (optional scope argument) loaded guardedly from bootsupport - if the snapshot is stale/missing the check degrades to a skip notice (but `-dirty-abort` then aborts rather than silently losing the requested strictness). -- Provenance warnings (dirty-src gate + vendorupdate source-project check) print with a plain column-0 `PROVENANCE-WARNING:` token (grep for it in captured make.tcl output) and are recapped at the end of the run via make.tcl's wrapped `::exit`. Interactive terminal runs get a 3-second ctrl-c grace countdown before a dirty run proceeds; piped/agent runs skip it. `tclsh src/make.tcl check` reports the current src provenance status and what the producing commands would do. +- Provenance warnings (dirty-src gate + vendorupdate source-project check) print with a plain column-0 `PROVENANCE-WARNING:` token (grep for it in captured make.tcl output) and are recapped at the end of the run via make.tcl's wrapped `::exit`. Interactive terminal runs get a 4-second ctrl-c grace countdown before a dirty run proceeds; piped/agent runs skip it. `tclsh src/make.tcl check` reports the current src provenance status and what the producing commands would do. - Use `tclsh src/make.tcl vfscommonupdate` to rebuild `_vfscommon.vfs`. The REPLACE confirmation prompts on an interactive terminal; unattended/agent runs must pass `-confirm 0` (with the default `-confirm 1` and a piped/closed stdin, make.tcl aborts fast with guidance instead of reading stdin - do not pipe `y`). - `tclsh src/make.tcl workflow` prints an embedded ASCII data-flow overview of the release workflow (the TERMINOLOGY stage-verb key, release-ready command sequence, source→outputs folder flow, kit assembly, keyed notes). The text lives in `::punkboot::workflow_text` in `src/make.tcl` - embedded rather than a data file so it travels with the make.tcl copies seeded into generated projects. **Update contract (agents):** whenever make.tcl data flow changes - a make.tcl subcommand added/removed/repurposed, a source or output folder added or rerouted, a propagation target added, a gate (staleness/provenance/confirm) or deploy behaviour changed - update the workflow text in the same change-set. Keep it plain ASCII, max line width 100, and preserve the TERMINOLOGY / RELEASE SEQUENCE / DIAGRAM / KEY-NOTES / MAINTENANCE structure; verify by running `tclsh src/make.tcl workflow` (and once under `PUNKBOOT_PLAIN=1`). Adding/changing a subcommand also means: SUMMARIES + HELPTEXTS + its braced `punk::args::define` block (+ SUBGROUPS) in `::punkboot::argdoc`, `known_commands`, the plain `punkboot_gethelp` text, and a project-version bump (make.tcl interface is product surface per root AGENTS.md). Layout make.tcl copies pick the change up via the thin-layout sync step in any `make.tcl modules`/`libs`/`packages`/`bakehouse` run - do not hand-sync them. - make.tcl subcommands and flags are punk::args-declared (G-030; help depth G-143, achieved; single-mechanism revision 2026-08-05, user direction): `tclsh src/make.tcl help ?subcommand? ?arg ...?` shows tabular usage - the leading `help` word is the ONLY help entry; help FLAGS (-help/--help/-h//?) are ordinary arguments that fail whatever gate they meet (top level: the subcommand choice violation; tool option position: noformmatch; a value position with restricted choices: the choice gate; degraded scan: Unknown flag + help exit 1), with the usage error itself carrying the documentation. Invalid arguments produce punk::args usage errors. Bare `make.tcl help` (and bare `make.tcl`) renders a rich top-level overview (2026-08-02, the `i info` style): per-subcommand cells carrying the one-line summary plus that subcommand's own auto-generated synopsis line(s) - per-form lines for the multi-form subcommands - grouped by SUBGROUPS (rich `-choicelabels` built by `::punkboot::argdoc::overview_labels` from `punk::args::synopsis` per id, hung on the separate `(script)::punkboot.overview` id so dispatch never pays the label cost; the lean `(script)::punkboot` id remains the dispatch surface). `-choicecolumns` is 1 for now - the long one-line summaries make a 2-column layout too wide without table CELL WRAPPING; revisit `-choicecolumns 2` when that lands (user decision 2026-08-02; the `## FORM` header lines are likewise deliberately dropped from the cells). tool and buildsuite are multi-form definitions (one @form per action with a literal action leader), so whole-subcommand help renders one synopsis line per action, and help accepts the subcommand's own COMMAND LINE after the subject and DRY-RUNS it through that subcommand's declaration: `make.tcl help tool build -test 0 punkzip` parses via punk::args form auto-selection and renders the build form's usage, while a line the subcommand would reject (unknown action word, option-first line, unknown flag in option position) gets the same pointed punk::args noformmatch diagnosis dispatch gives - synopsis plus per-form reasons - on stderr with exit 1, never a plain-help fallback. There is deliberately no bare-action carve-out: an action word alone is dry-run too, so a form with required values reports them via the diagnosis (`help buildsuite build` exits 1 naming the missing suitename - the error table carries the form's argument rows, so it IS the form documentation). An accepted line is confirmed with a one-line received-args report after the table (`dry-run: line accepted (form build) - action = build | -test = 0 | toolname = punkzip`); punk::args' positional model consumes flag-like words at/after the first value position as VALUES where the value arg is unconstrained (tool's toolname), and the report makes such swallows visible - value args with restricted choices (bake/bakelist kitnames as of 2026-08-02) instead reject the flag-like word at the choice gate, exactly as dispatch does. Declaration-level passthroughs (shell args, buildsuite driver args) always parse clean; numeric form indexes are deliberately not an interface. tool DISPATCH parses through its definition: unknown actions/flags are punk::args usage errors (exit 1), and the declared positional model puts options before the tool names (`make.tcl tool build -test 0 ...`, matching bake's flags-before-names convention; the historic flag-anywhere order is deliberately not accepted - a flag-shaped tool name earns a stderr hint). buildsuite keeps its passthrough driver-args contract - its forms serve help/synopsis accuracy only. The dispatch degrades to plain scan/help when the bootsupport punk::args (or its rendering stack) is stale or unavailable; `PUNKBOOT_PLAIN=1` forces the degraded mode for troubleshooting (the degraded tool handler keeps the historic manual tail parse and exit-2 surface). Piped characterization: `src/tests/shell/testsuites/punkexe/maketclhelp.test`. The `::punkboot::argdoc` definitions are braced file-style blocks using the G-045 authoring mechanisms — `-&` record continuations, tstr placeholders pulling SUMMARIES/HELPTEXTS and the shared `OPT_*` option fragments — with `-help` bodies expanding as deferred display fields so the HELPTEXTS block indentation deliberately renders as a centred Description (no `@normalize` re-basing). Synopses are the automatic punk::args bracket notation via the G-144 `@cmd -name` fallback; explicit `@form -synopsis` overrides were retired 2026-08-01 except the top-level `make.tcl ?subcommand? ?flags?` line (it states the bare-invocation/flags reality the auto render cannot) - do not reintroduce per-subcommand overrides without cause. See src/modules/AGENTS.md "punk::args definition authoring ergonomics" before editing them or authoring similar definitions.