From 9d92c11bd720097679f6a711217ef8d4aead6f58 Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Thu, 6 Aug 2026 15:57:48 +1000 Subject: [PATCH] agent skills: tcl-runtests added; whatis/nslist gain the script -e quick-probe line tcl-runtests (new): agent quickstart for src/tests/runtests.tcl. Runner-interpreter policy: the zig-built suite tclsh (src/buildsuites/_build/suite_tcl90/out/bin/tclsh90s; tclsh86ts for 8.6 parity) or a bin/runtime store tclsh; a fresh checkout bootstraps via the no-tclsh 'zig build bootstrap' flow (pinned zig only, bin/punk-getzig) or by fetching prebuilt runtimes from punkbin via bin/punk-runtime - which path is the user's choice, not an agent default. Never bare 'tclsh' from an agent shell (commonly the MSYS/Git-for-Windows one - spurious failures); native tclsh preferred over punk kit exes (kit-stamped module shadowing). Plus canonical invocation/targeting forms, result-reading rules (status=warn = incomplete), agent-environment traps (NO_COLOR=1 breaks exact-SGR pins agent-side only; PowerShell-tool attached consoles; no mid-run .test edits; PUNK_SHELL_TEST_EXE for locked/stale kit exes) and .test authoring micro-traps (trailing 'set result', brace-in-comment parse death, \y-not-\b). Pointer-first: src/tests/AGENTS.md remains the authoritative harness contract. tcl-whatis/tcl-nslist: one sentence added to the Command section - the launcher one-liner form `bin/punk91 src script -e ''` for RUNNING a snippet rather than introspecting it (behaviour checks, cross-version probes via the documented punk905/punksys kits). Facts verified live before writing: suite tclsh90s = Tcl 9.0.5 + Thread 3.0.7 and drove a focused runtests run (args dynamic.test 2/2 PASS, 5s); runtime-store tclsh9.0.5-punk/plain = 9.0.5 + Thread 3.0.7; -e forms on punk91/punk905/punksys; punk-runtime list output; PUNK_SHELL_TEST_EXE resolution in the punkexe suites. .claude/skills copies are byte-identical re-copies of the .agents canonicals (diff-verified) per the root AGENTS.md skill sync convention. Claude-Session: https://claude.ai/code/session_01Y1diJnhjUxKgEG6EwYAzxj Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- .agents/skills/tcl-nslist/SKILL.md | 6 +- .agents/skills/tcl-runtests/SKILL.md | 113 +++++++++++++++++++++++++++ .agents/skills/tcl-whatis/SKILL.md | 6 +- .claude/skills/tcl-nslist/SKILL.md | 6 +- .claude/skills/tcl-runtests/SKILL.md | 113 +++++++++++++++++++++++++++ .claude/skills/tcl-whatis/SKILL.md | 6 +- 6 files changed, 242 insertions(+), 8 deletions(-) create mode 100644 .agents/skills/tcl-runtests/SKILL.md create mode 100644 .claude/skills/tcl-runtests/SKILL.md diff --git a/.agents/skills/tcl-nslist/SKILL.md b/.agents/skills/tcl-nslist/SKILL.md index a59f1fe9..162437d1 100644 --- a/.agents/skills/tcl-nslist/SKILL.md +++ b/.agents/skills/tcl-nslist/SKILL.md @@ -17,8 +17,10 @@ auto-loads the providing package when the namespace is not yet populated. Run from the repo root (`punk91` lives in `bin/`, so `bin/punk91` from the Bash tool; any punkshell kit works). `punk91` embeds Tcl 9.1; for Tcl-version-sensitive questions, or if punk91 is unavailable, use `punk905` -(Tcl 9.0) or `punksys` (Tcl 8.6) with the same command line. Cost: sub-1s, -no side effects. +(Tcl 9.0) or `punksys` (Tcl 8.6) with the same command line. To RUN a +snippet rather than introspect it (behaviour checks, cross-version probes) +the same launchers take a one-liner: `bin/punk91 src script -e ''`. +Cost: sub-1s, no side effects. punk91 src script lib:developer/nslist ?-synopsis? ?-pathcommands? ... diff --git a/.agents/skills/tcl-runtests/SKILL.md b/.agents/skills/tcl-runtests/SKILL.md new file mode 100644 index 00000000..24c63944 --- /dev/null +++ b/.agents/skills/tcl-runtests/SKILL.md @@ -0,0 +1,113 @@ +--- +name: tcl-runtests +description: "Use BEFORE running, adding or editing any tests in this repo, and BEFORE choosing an interpreter to run them with. Canonical runner: the repo's zig-built tclsh under src/buildsuites/_build or a runtime-store tclsh under bin/runtime; a fresh checkout bootstraps either via one-time 'zig build bootstrap' (only the pinned zig required) or by fetching prebuilt runtimes from the punkbin artifact repo via bin/punk-runtime (user's choice). Bare 'tclsh' from an agent shell is often the MSYS/Git-for-Windows one and yields spurious failures. Quickstart invocation and targeting forms, result-reading rules, and the agent-environment traps that waste runs: NO_COLOR=1 breaking exact-SGR pins agent-side only, PowerShell-tool attached consoles, editing .test files mid-run, locked/stale kit exes (PUNK_SHELL_TEST_EXE), warn-status results that look like passes, plus .test authoring micro-traps. Cost: focused runs seconds; full suite ~50s at -jobs 16. Full harness contract: src/tests/AGENTS.md." +--- + +# tcl-runtests + +Running the source-tree test suite (`src/tests/runtests.tcl`) as an agent: +which interpreter to drive it with, the invocation forms worth using, and +the environment traps that produce spurious failures or wasted runs. This +is the quickstart + traps layer only - the full harness contract +(targeting grammar, multi-process/jobs internals, watch mode, provenance +rules) lives in `src/tests/AGENTS.md` and stays authoritative. + +## Runner interpreter + +Preferred: the repo's own zig-built runtime - portable across machines, no +reliance on whatever tclsh a host happens to have: + + src/buildsuites/_build/suite_tcl90/out/bin/tclsh90s + +(`.exe` on windows; Tcl 9.0.5 with Thread included, so `-jobs` works. +`_build/suite_tcl86/out/bin/tclsh86ts` is the Tcl 8.6 counterpart - +runtests supports an 8.6 runner.) The runtime store +`bin/runtime//` is an equally good runner source when populated +(e.g `tclsh9.0.5-punk.exe` - same 9.0.5+Thread family); check both before +bootstrapping anything. + +On a fresh checkout (neither present) there are two sanctioned no-tclsh +bootstrap paths - which one is the USER's choice, not an agent default: + +- BUILD: obtain the pinned zig with `bin/punk-getzig.cmd` (windows + helper; on other hosts supply zig on PATH or via `PUNK_ZIG`), then from + `src/buildsuites/suite_tcl90/` run `zig build bootstrap`. Builds and + smokes the full runtime family - a substantial first build; zig caching + makes reruns cheap. The suite README covers `-Dsteps` narrowing; + `tclsh src/make.tcl buildsuite build suite_tcl90` is equivalent once + some tclsh exists. +- DOWNLOAD: fetch prebuilt runtimes from the punkbin artifact repo with + the VCS-tracked polyglot `bin/punk-runtime.cmd` (`list`, `list -remote`, + `use `; the same file runs from bash on unix). Fetched runtimes + land in `bin/runtime//`. This pulls prebuilt binaries over + the network - confirm with the user before fetching. + +- NEVER bare `tclsh` from the Bash tool: it commonly resolves to the + MSYS/Git-for-Windows tclsh (8.6, msys path semantics) and yields + spurious failures. A machine-local NATIVE tclsh90/tclsh87 is also fine + as a runner when present. +- Prefer a native tclsh over a punk kit exe: kit children boot with + kit-stamped punk modules preloaded, which can shadow the src dev + modules under test (the runner warns when this applies). + +## Command forms + +Run from the repo root; options come BEFORE any trailing file-tail globs. +`$RUNNER` below is the interpreter chosen above. + + $RUNNER src/tests/runtests.tcl -discover-only 1 -include-paths ?? + $RUNNER src/tests/runtests.tcl -report compact -show-passes 0 -include-paths modules/punk/args/testsuites/args dynamic.test + $RUNNER src/tests/runtests.tcl -report compact -show-passes 0 -include-paths "modules/punk/args/***" -jobs 16 + $RUNNER src/tests/runtests.tcl -jobs 16 -report compact -show-passes 0 + +Top to bottom: subsecond targeting pre-check (prints the discovered file +list and exits); single-file run; subtree run; full suite (~50s at +`-jobs 16` on the reference machine - use `-jobs` for anything beyond a +handful of files). + +- `-include-paths` patterns are directory globs relative to `src/tests/`, + forward slashes: bare `X` = files directly in X, `X/***` = X and + everything below. Trailing bare words are independent file-tail globs. +- Single test within a file: add `-tcltestoptions {-match }`. +- Failure detail: `-report markdown` shows untruncated errorInfo (ERROR + status) and result_was/result_expected (FAILED status). +- Add `-strict-exit 1` when the shell exit code must reflect failures. + +## Reading results + +- Trust the final tally and `RUNTESTS_RESULT` line. `status=warn` or a + `missing-cleanupTests` reason means INCOMPLETE results, even when + observed pass events are listed. +- ERROR = the test raised an error; FAILED = result mismatch. Both carry + detail fields in compact/markdown/json reports. + +## Traps (agent environment) + +- Agent harnesses commonly export `NO_COLOR=1`. Suites that pin exact SGR + sequences (punk/ansi and friends) then fail agent-side only - unset it + in the same command (`env -u NO_COLOR $RUNNER ...` in bash) for those + runs. +- Use the Bash tool, not the PowerShell tool: PowerShell-tool children + get an attached console, which can flip punkshell's colour/terminal + detection and change output classification. +- Never edit `.test` files or the modules under test while a run is in + flight - multi-process children source them mid-run. +- `shell/`ish punkexe suites exec a punk kit resolved from + `env(PUNK_SHELL_TEST_EXE)`, else `/bin/punk902z.exe`. On + windows, running punkshells keep `bin/` exes locked (deploys leave them + stale) - point `PUNK_SHELL_TEST_EXE` at a fresh copy rather than + killing shells. + +## Traps (authoring .test files) + +- tcltest compares the body's RETURN VALUE with `-result`; a body ending + in a loop returns the empty string - end such bodies with an explicit + `set result`. +- A `.test` file is parsed as one script: an unbalanced brace ANYWHERE, + including inside a `#` comment, kills the whole file's parse. +- Tcl ARE regexp: `\y` is the word boundary; `\b` is a BACKSPACE. +- `package require` every extra package explicitly; finish files with + `tcltest::cleanupTests` (its absence is the `missing-cleanupTests` + warning above). +- Agent-added tests take a `#added (agent...)` provenance comment + line - format and rules in `src/tests/AGENTS.md`. diff --git a/.agents/skills/tcl-whatis/SKILL.md b/.agents/skills/tcl-whatis/SKILL.md index 9d1ca979..19671aa5 100644 --- a/.agents/skills/tcl-whatis/SKILL.md +++ b/.agents/skills/tcl-whatis/SKILL.md @@ -16,8 +16,10 @@ zipfs) is actually loaded. Run from the repo root (`punk91` lives in `bin/`, so `bin/punk91` from the Bash tool; any punkshell kit works). `punk91` embeds Tcl 9.1; for Tcl-version-sensitive questions, or if punk91 is unavailable, use `punk905` -(Tcl 9.0) or `punksys` (Tcl 8.6) with the same command line. Cost: sub-1s, -no side effects. +(Tcl 9.0) or `punksys` (Tcl 8.6) with the same command line. To RUN a +snippet rather than introspect it (behaviour checks, cross-version probes) +the same launchers take a one-liner: `bin/punk91 src script -e ''`. +Cost: sub-1s, no side effects. punk91 src script lib:developer/whatis ?-body? ?-doc? ?subcommand?... diff --git a/.claude/skills/tcl-nslist/SKILL.md b/.claude/skills/tcl-nslist/SKILL.md index a59f1fe9..162437d1 100644 --- a/.claude/skills/tcl-nslist/SKILL.md +++ b/.claude/skills/tcl-nslist/SKILL.md @@ -17,8 +17,10 @@ auto-loads the providing package when the namespace is not yet populated. Run from the repo root (`punk91` lives in `bin/`, so `bin/punk91` from the Bash tool; any punkshell kit works). `punk91` embeds Tcl 9.1; for Tcl-version-sensitive questions, or if punk91 is unavailable, use `punk905` -(Tcl 9.0) or `punksys` (Tcl 8.6) with the same command line. Cost: sub-1s, -no side effects. +(Tcl 9.0) or `punksys` (Tcl 8.6) with the same command line. To RUN a +snippet rather than introspect it (behaviour checks, cross-version probes) +the same launchers take a one-liner: `bin/punk91 src script -e ''`. +Cost: sub-1s, no side effects. punk91 src script lib:developer/nslist ?-synopsis? ?-pathcommands? ... diff --git a/.claude/skills/tcl-runtests/SKILL.md b/.claude/skills/tcl-runtests/SKILL.md new file mode 100644 index 00000000..24c63944 --- /dev/null +++ b/.claude/skills/tcl-runtests/SKILL.md @@ -0,0 +1,113 @@ +--- +name: tcl-runtests +description: "Use BEFORE running, adding or editing any tests in this repo, and BEFORE choosing an interpreter to run them with. Canonical runner: the repo's zig-built tclsh under src/buildsuites/_build or a runtime-store tclsh under bin/runtime; a fresh checkout bootstraps either via one-time 'zig build bootstrap' (only the pinned zig required) or by fetching prebuilt runtimes from the punkbin artifact repo via bin/punk-runtime (user's choice). Bare 'tclsh' from an agent shell is often the MSYS/Git-for-Windows one and yields spurious failures. Quickstart invocation and targeting forms, result-reading rules, and the agent-environment traps that waste runs: NO_COLOR=1 breaking exact-SGR pins agent-side only, PowerShell-tool attached consoles, editing .test files mid-run, locked/stale kit exes (PUNK_SHELL_TEST_EXE), warn-status results that look like passes, plus .test authoring micro-traps. Cost: focused runs seconds; full suite ~50s at -jobs 16. Full harness contract: src/tests/AGENTS.md." +--- + +# tcl-runtests + +Running the source-tree test suite (`src/tests/runtests.tcl`) as an agent: +which interpreter to drive it with, the invocation forms worth using, and +the environment traps that produce spurious failures or wasted runs. This +is the quickstart + traps layer only - the full harness contract +(targeting grammar, multi-process/jobs internals, watch mode, provenance +rules) lives in `src/tests/AGENTS.md` and stays authoritative. + +## Runner interpreter + +Preferred: the repo's own zig-built runtime - portable across machines, no +reliance on whatever tclsh a host happens to have: + + src/buildsuites/_build/suite_tcl90/out/bin/tclsh90s + +(`.exe` on windows; Tcl 9.0.5 with Thread included, so `-jobs` works. +`_build/suite_tcl86/out/bin/tclsh86ts` is the Tcl 8.6 counterpart - +runtests supports an 8.6 runner.) The runtime store +`bin/runtime//` is an equally good runner source when populated +(e.g `tclsh9.0.5-punk.exe` - same 9.0.5+Thread family); check both before +bootstrapping anything. + +On a fresh checkout (neither present) there are two sanctioned no-tclsh +bootstrap paths - which one is the USER's choice, not an agent default: + +- BUILD: obtain the pinned zig with `bin/punk-getzig.cmd` (windows + helper; on other hosts supply zig on PATH or via `PUNK_ZIG`), then from + `src/buildsuites/suite_tcl90/` run `zig build bootstrap`. Builds and + smokes the full runtime family - a substantial first build; zig caching + makes reruns cheap. The suite README covers `-Dsteps` narrowing; + `tclsh src/make.tcl buildsuite build suite_tcl90` is equivalent once + some tclsh exists. +- DOWNLOAD: fetch prebuilt runtimes from the punkbin artifact repo with + the VCS-tracked polyglot `bin/punk-runtime.cmd` (`list`, `list -remote`, + `use `; the same file runs from bash on unix). Fetched runtimes + land in `bin/runtime//`. This pulls prebuilt binaries over + the network - confirm with the user before fetching. + +- NEVER bare `tclsh` from the Bash tool: it commonly resolves to the + MSYS/Git-for-Windows tclsh (8.6, msys path semantics) and yields + spurious failures. A machine-local NATIVE tclsh90/tclsh87 is also fine + as a runner when present. +- Prefer a native tclsh over a punk kit exe: kit children boot with + kit-stamped punk modules preloaded, which can shadow the src dev + modules under test (the runner warns when this applies). + +## Command forms + +Run from the repo root; options come BEFORE any trailing file-tail globs. +`$RUNNER` below is the interpreter chosen above. + + $RUNNER src/tests/runtests.tcl -discover-only 1 -include-paths ?? + $RUNNER src/tests/runtests.tcl -report compact -show-passes 0 -include-paths modules/punk/args/testsuites/args dynamic.test + $RUNNER src/tests/runtests.tcl -report compact -show-passes 0 -include-paths "modules/punk/args/***" -jobs 16 + $RUNNER src/tests/runtests.tcl -jobs 16 -report compact -show-passes 0 + +Top to bottom: subsecond targeting pre-check (prints the discovered file +list and exits); single-file run; subtree run; full suite (~50s at +`-jobs 16` on the reference machine - use `-jobs` for anything beyond a +handful of files). + +- `-include-paths` patterns are directory globs relative to `src/tests/`, + forward slashes: bare `X` = files directly in X, `X/***` = X and + everything below. Trailing bare words are independent file-tail globs. +- Single test within a file: add `-tcltestoptions {-match }`. +- Failure detail: `-report markdown` shows untruncated errorInfo (ERROR + status) and result_was/result_expected (FAILED status). +- Add `-strict-exit 1` when the shell exit code must reflect failures. + +## Reading results + +- Trust the final tally and `RUNTESTS_RESULT` line. `status=warn` or a + `missing-cleanupTests` reason means INCOMPLETE results, even when + observed pass events are listed. +- ERROR = the test raised an error; FAILED = result mismatch. Both carry + detail fields in compact/markdown/json reports. + +## Traps (agent environment) + +- Agent harnesses commonly export `NO_COLOR=1`. Suites that pin exact SGR + sequences (punk/ansi and friends) then fail agent-side only - unset it + in the same command (`env -u NO_COLOR $RUNNER ...` in bash) for those + runs. +- Use the Bash tool, not the PowerShell tool: PowerShell-tool children + get an attached console, which can flip punkshell's colour/terminal + detection and change output classification. +- Never edit `.test` files or the modules under test while a run is in + flight - multi-process children source them mid-run. +- `shell/`ish punkexe suites exec a punk kit resolved from + `env(PUNK_SHELL_TEST_EXE)`, else `/bin/punk902z.exe`. On + windows, running punkshells keep `bin/` exes locked (deploys leave them + stale) - point `PUNK_SHELL_TEST_EXE` at a fresh copy rather than + killing shells. + +## Traps (authoring .test files) + +- tcltest compares the body's RETURN VALUE with `-result`; a body ending + in a loop returns the empty string - end such bodies with an explicit + `set result`. +- A `.test` file is parsed as one script: an unbalanced brace ANYWHERE, + including inside a `#` comment, kills the whole file's parse. +- Tcl ARE regexp: `\y` is the word boundary; `\b` is a BACKSPACE. +- `package require` every extra package explicitly; finish files with + `tcltest::cleanupTests` (its absence is the `missing-cleanupTests` + warning above). +- Agent-added tests take a `#added (agent...)` provenance comment + line - format and rules in `src/tests/AGENTS.md`. diff --git a/.claude/skills/tcl-whatis/SKILL.md b/.claude/skills/tcl-whatis/SKILL.md index 9d1ca979..19671aa5 100644 --- a/.claude/skills/tcl-whatis/SKILL.md +++ b/.claude/skills/tcl-whatis/SKILL.md @@ -16,8 +16,10 @@ zipfs) is actually loaded. Run from the repo root (`punk91` lives in `bin/`, so `bin/punk91` from the Bash tool; any punkshell kit works). `punk91` embeds Tcl 9.1; for Tcl-version-sensitive questions, or if punk91 is unavailable, use `punk905` -(Tcl 9.0) or `punksys` (Tcl 8.6) with the same command line. Cost: sub-1s, -no side effects. +(Tcl 9.0) or `punksys` (Tcl 8.6) with the same command line. To RUN a +snippet rather than introspect it (behaviour checks, cross-version probes) +the same launchers take a one-liner: `bin/punk91 src script -e ''`. +Cost: sub-1s, no side effects. punk91 src script lib:developer/whatis ?-body? ?-doc? ?subcommand?...