Compare commits

...

26 Commits

Author SHA1 Message Date
Julian Noble 04aa034aa3 README: refresh to match current project state — add build/getting-started section, supported platforms, shipped features (script subcommand, pluggable consoles, runtime.cmd, static runtime support, help system), update missing/unripe sections, add documentation pointers, rename n/new->newns and d/new->newdir 3 weeks ago
Julian Noble b3d42ca406 bootsupport/vfs: sync punk::repl 0.4.0 and opunk::console::tk 0.2.0 (G-001) 3 weeks ago
Julian Noble 2a25ecedf5 scriptlib/developer: tkconsole_demo showcase for the G-001 tk console backend 3 weeks ago
Julian Noble 3d14721d42 goals: G-001 achieved 2026-07-11 - archive entry + detail; project 0.11.0 3 weeks ago
Julian Noble 83c1e3e82b tests: repl-through-console-backend verification suite (G-001 acceptance) 3 weeks ago
Julian Noble 7e8d819663 opunk::console::tk 0.2.0: widget field, -in/-out channels, live console wiring, resize-aware size (G-001) 3 weeks ago
Julian Noble ab75a78b06 punk::repl 0.4.0: launch-time console selection (G-001) 3 weeks ago
Julian Noble 331e894c03 goals: add G-062/G-063/G-064 licensing and lib.search machine-output goals 3 weeks ago
Julian Noble 9cfeaaadb2 vfscommonupdate: sync punkboot::utils 0.2.0 into _vfscommon.vfs 3 weeks ago
Julian Noble 803f3ca5ff bootsupport + layout sync: punkboot::utils 0.2.0, pending 0.2.0 module snapshots 3 weeks ago
Julian Noble 1fa2988a3f make.tcl: dirty-src provenance gate for build/promotion commands (G-026 direction) 3 weeks ago
Julian Noble 3970b8e695 punk::libunknown 0.1 -> 0.2.0: adopt major.minor.patch + manual-versioning guidance 3 weeks ago
Julian Noble d35e055efc punk::libunknown: isolate pkgIndex.tcl sourcing - stop clobbering global 'dir' 3 weeks ago
Julian Noble 0c243a9ba9 dev lib.search: deep module discovery by default; -refresh = genuine re-scan 3 weeks ago
Julian Noble 4924fca21b vfscommonupdate: sync pending built modules into _vfscommon.vfs 3 weeks ago
Julian Noble 1d6a24f642 AGENTS.md + G-016 notes: fossil config-db pollution from throwaway repos 3 weeks ago
Julian Noble ce31d8623c update punk::sshrun with wiki documentation 3 weeks ago
Julian Noble 3bafc2f195 GOALS + src/tests: test provenance comments and superseded/abandoned test-sweep rules 3 weeks ago
Julian Noble ae04b3f8c9 GOALS: two-tier index restructure - summary-only GOALS.md, canonical Goal/Acceptance in detail files, archive-on-flip 3 weeks ago
Julian Noble aec77ca0d1 G-001 increment 1: pluggable ::opunk::Console backends - test double, ssh-channel, tk-widget (punkshell 0.9.0) 3 weeks ago
Julian Noble 4bb7b3d68f G-054 achieved: tclcore string is class choices harvested from the running interpreter (tclcore moduledoc 0.2.0, punkshell 0.8.2) 3 weeks ago
Julian Noble bd1baa39f6 update bootsupport + layout bootsupport: punk::lib 0.4.0, punk::repl 0.3.0, punk::packagepreference 0.2.0, shellthread 1.7.0 3 weeks ago
Julian Noble e8ef639793 G-016 detail: record the -return design decisions (completes 222886c0) 3 weeks ago
Julian Noble 222886c082 GOALS: G-016/G-017 amended - projects.work gains -return table|dict|json in contract 3 weeks ago
Julian Noble 89daa99269 GOALS: add G-061 [proposed] pseudoconsole expect-alternative for interactive shell testing 3 weeks ago
Julian Noble 8c65fe9d6f repl pre-refactor: characterize command-completeness engine + record preserve-list and testability findings 3 weeks ago
  1. 11
      AGENTS.md
  2. 35
      CHANGELOG.md
  3. 52
      GOALS-archive.md
  4. 232
      GOALS.md
  5. 58
      README.md
  6. 45
      goals/AGENTS.md
  7. 1
      goals/G-002-non-nested-subshell.md
  8. 1
      goals/G-003-subshell-resource-limits.md
  9. 1
      goals/G-004-no-committed-binaries.md
  10. 1
      goals/G-005-zig-build-infrastructure.md
  11. 1
      goals/G-006-prebuilt-artifact-download.md
  12. 1
      goals/G-008-scoped-console-state.md
  13. 1
      goals/G-009-themed-subshell-profiles.md
  14. 1
      goals/G-010-subshell-tree-navigation.md
  15. 1
      goals/G-011-console-stderr-semantics.md
  16. 1
      goals/G-012-template-payload-safety.md
  17. 3
      goals/G-013-raw-mode-default.md
  18. 3
      goals/G-014-punk-config-toml.md
  19. 33
      goals/G-016-projects-work-git-discovery.md
  20. 6
      goals/G-017-agent-project-discovery.md
  21. 3
      goals/G-018-zig-plain-tclsh-kits.md
  22. 3
      goals/G-019-dependency-scan-module-trimming.md
  23. 16
      goals/G-020-screencap-input-module.md
  24. 3
      goals/G-021-agent-visual-verification.md
  25. 3
      goals/G-022-fossil-rename-punkshell.md
  26. 3
      goals/G-023-version-named-binaries.md
  27. 3
      goals/G-024-mapvfs-toml.md
  28. 3
      goals/G-025-exe-selfreport.md
  29. 3
      goals/G-026-vendor-provenance-policy.md
  30. 3
      goals/G-027-derived-project-pull-updates.md
  31. 3
      goals/G-028-file-locker-identification.md
  32. 3
      goals/G-029-testmodules-from-srctests.md
  33. 3
      goals/G-030-maketcl-punkargs.md
  34. 3
      goals/G-031-componentized-kit-boot.md
  35. 3
      goals/G-032-launcher-punkargs.md
  36. 3
      goals/G-033-proj-mode-cwd-project.md
  37. 6
      goals/G-034-modpod-codeinterp-tcl86.md
  38. 3
      goals/G-035-mixed-tm-pkgindex-provision.md
  39. 1
      goals/G-038-piped-session-continuity.md
  40. 1
      goals/G-039-orphan-console-spin.md
  41. 1
      goals/G-041-punkargs-form-matching.md
  42. 3
      goals/G-042-subshell-help-topics.md
  43. 1
      goals/G-043-subshell-definition-plugins.md
  44. 38
      goals/G-044-repl-command-completion.md
  45. 1
      goals/G-045-punkargs-authoring-ergonomics.md
  46. 1
      goals/G-047-declared-primary-vcs.md
  47. 1
      goals/G-048-textblock-table-punkargs.md
  48. 3
      goals/G-050-synopsis-validity-marking.md
  49. 3
      goals/G-051-cmdinfo-pseudo-and-prefix.md
  50. 3
      goals/G-052-oo-method-autodef.md
  51. 3
      goals/G-053-punkargs-multiple-ranges.md
  52. 3
      goals/G-055-tclcore-regen-workflow.md
  53. 3
      goals/G-056-punkargs-word-wrapping.md
  54. 3
      goals/G-057-kit-icon-embedding.md
  55. 3
      goals/G-060-qemu-test-matrix.md
  56. 79
      goals/G-061-pseudoconsole-expect.md
  57. 21
      goals/G-062-project-license-file.md
  58. 34
      goals/G-063-package-license-tracking.md
  59. 23
      goals/G-064-libsearch-machine-returns.md
  60. 106
      goals/archive/G-001-pluggable-console-backends.md
  61. 1
      goals/archive/G-007-console-location-transparency.md
  62. 5
      goals/archive/G-015-script-subcommand-piped-stdin.md
  63. 3
      goals/archive/G-036-tcl9-udp-console-worker-wedge.md
  64. 1
      goals/archive/G-037-vendorlib-vfs-propagation.md
  65. 1
      goals/archive/G-040-punkargs-choicealiases.md
  66. 1
      goals/archive/G-046-punkargs-deferred-help-and-fixes.md
  67. 3
      goals/archive/G-049-punkargs-parse-status-model.md
  68. 6
      goals/archive/G-054-tclcore-stringis-harvest.md
  69. 3
      goals/archive/G-058-static-runtime-packages.md
  70. 3
      goals/archive/G-059-wsl-test-driving.md
  71. 2
      punkproject.toml
  72. 373
      scriptlib/developer/tkconsole_demo.tcl
  73. 2
      src/AGENTS.md
  74. 105
      src/bootsupport/modules/punk/args/moduledoc/tclcore-0.2.0.tm
  75. 44
      src/bootsupport/modules/punk/lib-0.4.0.tm
  76. 210
      src/bootsupport/modules/punk/libunknown-0.2.0.tm
  77. 93
      src/bootsupport/modules/punk/mix/commandset/loadedlib-0.2.0.tm
  78. 31
      src/bootsupport/modules/punk/packagepreference-0.2.0.tm
  79. 288
      src/bootsupport/modules/punk/repl-0.4.0.tm
  80. 54
      src/bootsupport/modules/punkboot/utils-0.2.0.tm
  81. 19
      src/bootsupport/modules/shellthread-1.7.0.tm
  82. 2
      src/lib/app-punkscript/punkscript.tcl
  83. 120
      src/make.tcl
  84. 4
      src/modules/AGENTS.md
  85. 2
      src/modules/opunk/AGENTS.md
  86. 111
      src/modules/opunk/console/ssh-999999.0a1.0.tm
  87. 4
      src/modules/opunk/console/ssh-buildversion.txt
  88. 120
      src/modules/opunk/console/test-999999.0a1.0.tm
  89. 4
      src/modules/opunk/console/test-buildversion.txt
  90. 370
      src/modules/opunk/console/tk-999999.0a1.0.tm
  91. 6
      src/modules/opunk/console/tk-buildversion.txt
  92. 1
      src/modules/punk/AGENTS.md
  93. 99
      src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm
  94. 3
      src/modules/punk/args/moduledoc/tclcore-buildversion.txt
  95. 210
      src/modules/punk/libunknown-0.2.0.tm
  96. 89
      src/modules/punk/mix/commandset/loadedlib-999999.0a1.0.tm
  97. 6
      src/modules/punk/mix/commandset/loadedlib-buildversion.txt
  98. 286
      src/modules/punk/repl-999999.0a1.0.tm
  99. 3
      src/modules/punk/repl-buildversion.txt
  100. 86
      src/modules/punk/sshrun-999999.0a1.0.tm
  101. Some files were not shown because too many files have changed in this diff Show More

11
AGENTS.md

@ -86,6 +86,7 @@ When the user requests a durable behavior change, record it here or in the relev
- LF line endings are strongly preferred for all files in this repository. Converting a CRLF text file to LF when an edit touches it is correct and welcome - do not preserve CRLF for diff-minimisation. Preserve existing line endings only for files with deliberately mixed/CRLF endings (e.g. line-ending round-trip test data) or when explicitly instructed for a file.
- If the active editor is on a source-derived snapshot, bootstrap copy, or build output path such as `src/bootsupport/`, root `modules/`, root `lib/`, `modules_tcl8/`, `modules_tcl9/`, `lib_tcl8/`, or `lib_tcl9/`, confirm the intended target before editing unless the user explicitly named that path.
- cmd.exe PATH truncation (this machine, and any Windows machine with a heavily populated PATH): cmd.exe truncates a long PATH, so a tool that resolves fine in PowerShell may be "not found" when invoked via `cmd.exe /c`. Use absolute executable paths inside any `cmd /c` command line, and prefer PowerShell-native invocation unless a console host is specifically required (e.g. hidden-console test harnesses). If a tool is missing only under cmd.exe, suspect truncation before absence.
- Throwaway fossil repositories (test/experiment repos an agent creates, e.g. in a session scratchpad) must not register in the user's real global fossil config-db: `fossil init`/`fossil open` write persistent `repo:`/`ckout:` rows into `%LOCALAPPDATA%\_fossil`, which is the enumeration source for `dev projects.work` project discovery (G-016/G-017). Set `FOSSIL_HOME` to a disposable scratch directory for the duration of such fossil commands (both fossil and `punk::repo::fossil_get_configdb` honour it first). If pollution has already occurred: `fossil all ignore <repo-path>`, delete the directory, then any `fossil all` command prunes the orphaned `ckout:` row.
- Do not commit new executable binaries (shared libs, .exe, native .so/.dll/.dylib, bare ELF/Mach-O, or zip-based .tm modules embedding executables) to the repository. Existing binaries in `bin/`, `src/vfs/`, `src/vendorlib/`, `src/vendormodules/`, and `src/bootsupport/` are there intentionally pending the build/retrieval infrastructure tracked by goals G-004/G-005/G-006; do not flag, "fix", or hassle the developer about these — they are known and will be removed once G-005 (zig build) or G-006 (pre-built download) provides an alternative. This rule stops agents from adding new binaries; it does not block the developer's interim commits of existing vendor/vfs binaries. It is workflow policy only - deliberately NOT enforced at the local VCS layer: fossil `binary-glob` is `*` (versioned in `.fossil-settings/`) so binary checkins proceed without warnings/prompts, here and in sub-projects like tomlish.
## Commit Conventions (any VCS)
@ -178,13 +179,13 @@ The project version is fully independent of module versions. A module bump (even
- `src/testansi/` — Sample ANSI art files (do not modify)
- `bin/` — Built punk shell executables, launch package modes (incl. `src` mode for working-tree verification), plain runtime kits (see bin/AGENTS.md)
- `.fossil-settings/` — Versioned fossil settings and the git+fossil dual-VCS coexistence contract: ignore-sync rules, files neither system may track (see .fossil-settings/AGENTS.md)
- `GOALS.md` — Top-level technical goal index; required read for non-trivial work (no child AGENTS.md; the file documents its own format and the agent goal-authoring workflow)
- `GOALS-archive.md`One-line records of achieved goals moved out of the active index (historical context only)
- `goals/`Optional detail prose for goals needing more than a one-line summary (see goals/AGENTS.md)
- `GOALS.md` — Top-level technical goal index, summary-only (ID, status, title, scope, detail pointer per goal); required read for non-trivial work (no child AGENTS.md; the file documents its own format, read workflow, and the agent goal-authoring workflow)
- `GOALS-archive.md`Summary records of achieved goals moved out of the active index (historical context only)
- `goals/`Per-goal detail files carrying the canonical Goal/Acceptance contract prose plus supporting detail (see goals/AGENTS.md)
- `goals/archive/` — Detail files for achieved/archived goals
- Directories agents should not directly modify (no child DOX needed):
- `callbacks/` — Experimental shellspy features, user-only
- `scriptlib/` — Shared utilities + manual tests, user-only. EXCEPTION: `scriptlib/_punktest/` is test-owned (fixtures for `src/tests/shell/testsuites/punkexe/scriptexec.test`, resolved via `lib:_punktest/<name>`); agents may manage that subfolder as part of test work. The rest of `scriptlib/` stays user-only.
- `scriptlib/` — Shared utilities + manual tests, user-only. EXCEPTIONS: `scriptlib/_punktest/` is test-owned (fixtures for `src/tests/shell/testsuites/punkexe/scriptexec.test`, resolved via `lib:_punktest/<name>`); agents may manage that subfolder as part of test work. `scriptlib/developer/` holds agent-authored developer showcase/demo apps (created at user request 2026-07-11, e.g `tkconsole_demo.tcl` for the G-001 tk console backend); agents may add or update demos there when the user asks for one. The rest of `scriptlib/` stays user-only.
- `bin/` — Built binaries and helpers, build output target. This includes the polyglot `.cmd` launcher/utility scripts (e.g `bin/runtime.cmd`): they are GENERATED by the punk::mix scriptwrap machinery from sources under `src/scriptapps/` — a request to "fix bin/<name>.cmd" means editing `src/scriptapps/<name>.*` + `<name>_wrap.toml` and re-wrapping (see bin/AGENTS.md), never editing the output
- `modules/` (root) — Build output target for `tclsh src/make.tcl modules`
- `lib/` (root) — Build output target for `tclsh src/make.tcl libs`
@ -200,4 +201,4 @@ The project version is fully independent of module versions. A module bump (even
- Source-tree build, testing, linting, and file-resolution workflow lives in `src/AGENTS.md`.
- Tcl module authoring conventions live in `src/modules/AGENTS.md` and closer module child docs.
- If AGENTS.md conflicts with CLAUDE.md, AGENTS.md wins.
- Technical project goals live in root `GOALS.md` (index) with optional detail prose in `goals/G-<id>-<slug>.md`. Goals are user-owned: agents add or edit goal entries only at the user's request, proposal-first (show the proposed wording, get explicit approval before writing - see the `GOALS.md` maintenance rules); never on their own initiative. Suggesting candidate goals when work surfaces something goal-worthy is welcome and encouraged - as a flagged proposal, not a file edit. Detail-file updates from user-directed work need only be reported in the completion summary. Agents auto-flip a goal to `achieved <date>` when its stated acceptance criterion is met as part of the DOX closeout pass, and must flag `proposed`→`active` transitions in their completion report for user confirmation. See `GOALS.md` for the full workflow, including how to author a new goal when asked.
- Technical project goals live in root `GOALS.md` (summary index: ID, status, title, scope, detail pointer per goal) with the canonical Goal/Acceptance contract prose in `goals/G-<id>-<slug>.md` detail files - read the detail file of every goal whose scope intersects paths being edited. Goals are user-owned: agents add or edit goal contract elements (in either tier) only at the user's request, proposal-first (show the proposed wording, get explicit approval before writing - see the `GOALS.md` maintenance rules); never on their own initiative. Suggesting candidate goals when work surfaces something goal-worthy is welcome and encouraged - as a flagged proposal, not a file edit. Non-contract detail-file updates from user-directed work need only be reported in the completion summary. Agents auto-flip a goal to `achieved <date>` when its detail-file acceptance criterion is met as part of the DOX closeout pass - the flip includes archiving the entry to `GOALS-archive.md` and the detail file to `goals/archive/` - and must flag `proposed`→`active` transitions in their completion report for user confirmation. See `GOALS.md` for the full workflow, including how to author a new goal when asked.

35
CHANGELOG.md

@ -5,6 +5,41 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to
Entries are newest-first; one bullet per notable change. See the root `AGENTS.md`
"Project Versioning" section for the bump policy.
## [0.11.0] - 2026-07-11
- G-001 achieved: an interactive REPL can be launched against a non-detectable terminal-like device via an `::opunk::Console` subclass — `repl::init -console <spec>` selects the console (channel pair, anchored instance name, or object value), `repl::start`'s input channel defaults to it, and the repl's output (prompts, results, and the code interp's stdout/stderr — diverted via shellfilter junction stacks and emitted per run) routes through the selected console's channels, with eof/size/capability answered by the subclass overrides (punk::repl 0.4.0; base `::opunk::Console` and `punk::console` untouched). Verified end-to-end by child-process driver tests: an `::opunk::SshConsole` socket session whose scripted remote terminal answers `CSI 6n` (size resolved over the socket) and a `::opunk::TkConsole` text widget wired as a live console by new `opunk::console::tk::console` (reflected output channel rendering into the widget + Return-binding input pipe, opunk::console::tk 0.2.0). Process-console behaviours (tcl_interactive prompt gating, stdin reopen on eof, raw-mode re-enable, the windows line re-decode experiment) now apply only when no foreign console is selected; default stdin/stdout repl behaviour is unchanged.
## [0.10.3] - 2026-07-11
- make.tcl build/promotion commands (`project`, `packages`, `modules`, `libs`, `vfs`, `vfslibs`, `bin`, `bootsupport`, `vfscommonupdate`) now warn when `src/` has uncommitted fossil/git changes — artifacts built from dirty src have no committed provenance (G-026 direction). Warn-only by default; a new `-dirty-abort` flag makes the check aborting. Dirt outside `src/` is ignored (`punkboot::utils::vcs_dirty_warnings` gained an optional scope argument, utils 0.2.0). With `<builtexe> src` / `<builtexe> src shell` available for evaluating uncommitted source directly, building is the promotion step the warning treats it as.
- Provenance warnings (dirty-src gate and vendorupdate's dirty source-project check) are presented with a plain column-0 `PROVENANCE-WARNING:` token for automated discovery in redirected output plus ANSI colour for humans, and are recapped at the very end of the run (via a wrapped `::exit`) so they survive scrolling chatty build output. Interactive terminal runs (Tcl 8.7+/9 stdin `-inputmode` probe) additionally get a 3-second ctrl-c grace countdown before a dirty build proceeds; piped/agent/CI runs pay no delay. `make.tcl check` now reports src provenance status and what the build/promotion commands would do.
## [0.10.2] - 2026-07-11
- punk::libunknown bumped 0.1 → 0.2.0 (file renamed `libunknown-0.2.0.tm`), retroactively versioning the register_all_tm and source_pkgindex additions and adopting full `major.minor.patch` form. Verified nothing requires it by exact version or filename: punk_main.tcl and punk::repl glob `libunknown-*.tm` picking the highest by vcompare (the dev copy now also outversions the stale bootsupport/project.vfs `0.1` copies deterministically), bootsupport's include_modules.config lists it by name only. The module header now carries a version-history block in lieu of a buildversion.txt, and `src/modules/AGENTS.md` documents the manual-versioning bump mechanics for agents (same Patch/Minor/Major rules as buildversion-tracked modules).
## [0.10.1] - 2026-07-11
- punk::libunknown: pkgIndex.tcl scripts are now executed in an isolated `source_pkgindex` frame providing the documented `$dir` variable plus `auto_path`/`env` global links (matching stock tclPkgUnknown's contract), instead of at `::` scope with a `global dir`. Fixes any user global named `dir` being silently overwritten whenever a `package require` fell through to the pkg unknown handler, and stops index scripts' helper variables (`ver`, `pkg`, `script`, `_CawtSubDirs`, ...) leaking into the global namespace. Verified against tcllib 2.0's index behaviours (auto_path extension, apply-scoped subindex sweep, critcl-style loaders); regression tests pin the `$dir`/auto_path/no-leak contract.
## [0.10.0] - 2026-07-11
- `dev lib.search` now performs deep module discovery by default: new `punk::libunknown::register_all_tm` registers `.tm` modules at every namespace depth across all tm paths (reusing/populating the per-epoch directory index cache, at most one scan per `package epoch` per interp), so namespaced modules in never-requested subfolders (e.g. `test::*`) appear in search results without `-refresh` (punk::mix::commandset::loadedlib 0.2.0).
- `dev lib.search -refresh` repurposed to mean a genuine filesystem re-scan: it increments the package epoch (invalidating punk::libunknown's scan caches) and re-runs discovery, picking up `.tm` files added/removed on disk and re-sourcing `pkgIndex.tcl` files. Without punk::libunknown active, `-refresh` retains the previous dummy-require deep-walk behaviour.
- lib.search dependency fixes: works in bare interps now — highlight ANSI codes computed only when highlighting (fully-qualified `punk::ansi` with inline require; previously errored on the shell-global `a+` alias even with `-highlight 0`), inline requires for `punk::path` (fallback walk) and `textblock` (table output).
## [0.9.1] - 2026-07-11
- `dev lib.search` `-refresh` help rewritten to document actual semantics (punk::mix::commandset::loadedlib 0.1.1, doc-only): the flag performs a deep tm discovery pass (registering `.tm` modules in never-requested namespace subfolders), it does not re-scan directories already indexed in the current `package epoch` (run `package epoch incr` first for a genuine filesystem re-scan), and tm/auto_path list changes are epoch-invalidated automatically. New characterization test suites pin the underlying punk::libunknown discovery/epoch-cache behaviour (`src/tests/modules/punk/libunknown/testsuites/discovery/`) and the lib.search match + `-refresh` contract (`src/tests/modules/punk/mix/testsuites/loadedlib/`), including a GAP pin of the pkg-unknown handler clobbering a user global `dir` variable (stock Tcl keeps it proc-local).
## [0.9.0] - 2026-07-11
- G-001 (in progress) increment 1: three pluggable `::opunk::Console` backend modules, with the base class and punk::console untouched — `opunk::console::test` (`::opunk::TestConsole`: deterministic channel-pair test double with fixed size and probe-free eof — the console seam the repl/editbuf characterization work needs), `opunk::console::ssh` (`::opunk::SshConsole`: socket-carried terminal sessions — construction-time capability, chan-eof without byte-consuming probes, and size resolved by punk::console's ANSI cursor-report provider *querying over the connection*, proven against a scripted remote terminal answering `CSI 6n` over a socket pair), and `opunk::console::tk` (`::opunk::TkConsole`: a Tk text widget as terminal — widget char dimensions as size, backend eof marker via `opunk::console::tk::set_eof`, verified live under the tk-capable punk91 kit). Subclass values dispatch through existing base-class holders and `console_spec_resolve` virtually. Remaining for the goal: repl launch-time console selection and output-channel parameterization.
## [0.8.2] - 2026-07-11
- G-054 achieved: tclcore moduledoc 0.2.0 — the `string is` class choices shown by `i string is` (and the per-class docids like `i string is digit`) are now harvested from the running interpreter at define time instead of a hand-maintained list, so the documented/parsed class set always matches what the interpreter accepts (previously the static Tcl 9.0 list wrongly accepted `dict` under 8.6 and rejected 8.7's `unicode`). Version notes render on classes that differ across releases; unrecognized future classes get a generic label instead of vanishing. Behavioural parity pinned by `tclcoreparity.test` with expectations derived from the live interpreter — green on Tcl 8.6.13, 8.7a6 and 9.0.3.
## [0.8.1] - 2026-07-11
- `runtime.cmd` powershell payload: cached `sha1sums.txt` fallback backported from the bash payload - `list -remote` against an unreachable server now warns and compares using the previously fetched copy instead of dying on an unhandled download error, and the `fetch` path's pre-existing silent fall-through to a cached copy now announces itself. Also fixed a latent undefined-variable bug creating the runtime folder in the `list -remote` branch.

52
GOALS-archive.md

@ -1,21 +1,63 @@
# Achieved Goals Archive
This file holds one-line records of goals that have been achieved and moved out of the active `GOALS.md` index to keep that file lean. Records here are historical context only — they explain why code exists in its current shape and are useful when future agents refactor or revisit the same area.
This file holds summary records of goals that have been achieved and moved out of the active `GOALS.md` index to keep that file lean. Goals are archived as part of the achieved flip (see the `GOALS.md` maintenance rules). Records here are historical context only — they explain why code exists in its current shape and are useful when future agents refactor or revisit the same area.
## Format
Each archived goal is one line, preserving its original ID and acceptance criterion so it remains traceable:
Each archived goal is one compact record, preserving its original ID and acceptance criterion so it remains traceable:
```
### G-<id> [achieved <YYYY-MM-DD>] <short title> → detail: goals/archive/G-<id>-<slug>.md
Scope: <as in original index>
Acceptance: <as in original index>
Acceptance: <as achieved>
```
If a goal had no detail file, omit the `→ detail:` clause.
The full record (including the Goal statement) lives in the archived detail file.
Do not edit archived entries except to fix a broken path. If an archived goal is reopened, move it back to `GOALS.md` with a new ID and mark the old entry `superseded by G-<new id>`.
## Archived goals
_None yet._
### G-001 [achieved 2026-07-11] Pluggable console backends for non-detectable terminals → detail: goals/archive/G-001-pluggable-console-backends.md
Scope: src/modules/opunk/console-999999.0a1.0.tm, src/modules/punk/console-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm, src/lib/app-punkshell/punkshell.tcl
Acceptance: a subshell started with an ssh-channel-backed and a tk-widget-backed ::opunk::Console subclass runs an interactive REPL that reads/writes through that console; size, at_eof, and can_respond are answered by the subclass overrides; the base ::opunk::Console and punk::console module are unchanged.
### G-007 [achieved 2026-07-05] Location-transparent punk::console across repl and code interps → detail: goals/archive/G-007-console-location-transparency.md
Scope: src/modules/punk/console-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm
Acceptance: from a running punk session's code interp, without `repl eval`: `console_fact_get` returns the same values the parent sees and a fact set in the parent is immediately visible; a terminal query (e.g. `get_cursor_pos` or `dec_get_mode`) against the default console succeeds and cooperates with the repl reader (no lost or garbled input); a console constructed and owned by code-interp code is operated on locally (no round-trip to the parent); the existing console test suites pass and single-interp (non-repl) usage is unchanged.
### G-015 [achieved 2026-07-07] Punk executable `script` subcommand: reliable non-interactive piped/script execution → detail: goals/archive/G-015-script-subcommand-piped-stdin.md
Scope: src/vfs/_config/punk_main.tcl, src/lib/app-punkshell/punkshell.tcl (script path or a leaner dedicated app package)
Acceptance: piping commands to `<punkexe> script` runs them and terminates at stdin EOF with no trailing `exit` required, exit code 0 on success; a failing piped command terminates the process with a nonzero exit code and the error on stderr, never landing in an interactive shell regardless of console availability or PUNK_PIPE_EOF; `<punkexe> script <file> [<args>...]` executes the file with conventional ::argv0/::argv and propagates its error status the same way; the script path installs none of the `shell` subcommand's shellfilter stacks/transforms and the launch plumbing itself emits nothing on stdout/stderr (the current stub's stderr diagnostics removed) so exec-style callers see only the script's own output; the motivating example works with no package require boilerplate: piping `dev projects.work *<name>*` to `<punkexe> script` emits the matching-project table and exits 0, because the script interp carries the default punk shell module/alias environment.
### G-036 [achieved 2026-07-08] Root-cause the Tcl 9 console+udp worker-thread event-loop wedge; minimal repro for possible upstream reporting → detail: goals/archive/G-036-tcl9-udp-console-worker-wedge.md
Scope: src/modules/shellthread-999999.0a1.0.tm, src/bootsupport/modules/shellthread-1.6.2.tm, src/modules/shellfilter-999999.0a1.0.tm (as characterised - no product-code changes required by this goal; the punkshell mitigations are separate fixes)
Acceptance: (reworked 2026-07-08 after the root cause was found) the wedge mechanism is identified and written up in the detail file - DONE: bundled tcludp 1.0.12's Windows per-thread UDP_ExitProc closes the process-global synchronization events, proven by dump handle-table analysis plus a live CloseHandle breakpoint and confirmed by the upstream 1.0.12->1.0.13 diff, which already fixes it (so no upstream report is required; the standalone minimal repro originally required here is waived as moot by the user); the tcl9 kits bundle tcludp >= 1.0.13 with the in-context batch harness baseline resolved - DONE 2026-07-08 (run-2 syslog workers alive vs the 4/4-wedged 1.0.12 baseline); DONE 2026-07-08 (punk::lib 0.3.0 has_libbug_udp_threadexit, surfaced via 'help tcl' in punk 0.2.1): a has_bug-style detection in the punkshell check machinery (in the vein of punk::lib::check::has_tclbug_* / punk::console::check::has_bug_*, surfaced through the same reporting as 'help tcl'/'help console') reports the vulnerable combination - simple version-based detection (loaded/bundled tcludp < 1.0.13 on Tcl 9 Windows) is sufficient, no behavioural probe needed; loose-end decisions (punk8win 8.6 kit's udp 1.0.12 swap; optional upstream tickets for residual tcludp trunk weaknesses) recorded in the detail file when made (open as of 2026-07-08 - non-gating).
### G-037 [achieved 2026-07-08] Propagate platform vendor libraries into kit vfs lib_tcl trees via make.tcl → detail: goals/archive/G-037-vendorlib-vfs-propagation.md
Scope: src/make.tcl (new or extended step), src/vendorlib_tcl8 + src/vendorlib_tcl9 (sources), src/vfs/<kit>.vfs/lib_tcl8 + lib_tcl9 (targets), punkcheck tracking
Acceptance: with a newer package version placed under src/vendorlib_tcl9/<platform>, one documented make.tcl invocation updates the participating src/vfs/*/lib_tcl9 trees - installing the new package and removing or explicitly retiring the superseded version (no silent mixed-version provision, per the G-035 concerns) - with punkcheck-tracked provenance; which vfs folders participate is explicitly declared per kit rather than blanket-copied (kit vfs package sets may intentionally differ), with the declaration mechanism recorded (candidate home: the G-024 mapvfs toml); a subsequent `make.tcl project` yields kits loading the new version (provable via the tcludp case: built punk902z reports `package require udp` == 1.0.13 with no udp1.0.12 folder remaining in its vfs); the lib_tcl8 tree gets the same treatment or an explicit exclusion rationale in the goal record.
### G-040 [achieved 2026-07-08] punk::args choice aliasing (-choicealiases) with parse normalization, display folding, and doc-lookup parity → detail: goals/archive/G-040-punkargs-choicealiases.md
Scope: src/modules/punk/args-999999.0a1.0.tm (parse + usage rendering), src/modules/punk/ns-999999.0a1.0.tm (cmdinfo/cmd_traverse choice resolution parity), src/modules/punk-999999.0a1.0.tm (punk::help topic argdoc as first consumer), src/tests/modules/punk/args/testsuites/, src/tests/modules/punk/ns/testsuites/
Acceptance: a definition using -choicealiases parses an alias (and an alias prefix where -choiceprefix allows) to its canonical choice in the parse result, with -choicerestricted 0 passthrough and the deny/reserve lists honoured unchanged; usage display shows one entry per canonical choice with aliases folded (no duplicate rows; -choicelabels attach to the canonical); punk::ns::cmdinfo/cmd_traverse resolve subcommand words to docids with the same outcome as the parser for alias, prefix, denied, reserved and unknown words (the pre-goal characterization tests updated from pinned-GAP to fixed); punk::help's topic definition adopts the feature so `i help` lists one entry per registered topic while `help h`/`help e` still fall through to command lookup; definitions without -choicealiases behave unchanged (existing punk::args and punk::ns suites pass).
### G-046 [achieved 2026-07-10] punk::args deferred -help resolution (parse-time performance + reentrancy) and rendering/value-shape fixes → detail: goals/archive/G-046-punkargs-deferred-help-and-fixes.md
Scope: src/modules/punk/args-999999.0a1.0.tm (resolve/get_dict: display-field deferral, dynamic-cache subst path, prefix writeback, string renderer, cmdhelp-facing messages), src/modules/punk/ansi-999999.0a1.0.tm (mark_columns argdoc as the reentrancy/perf testbed), src/tests/modules/punk/args/testsuites/ (GAP tests flip; perf verification)
Acceptance: parsing/argument resolution provably skips -help expansion (a definition whose -help contains a ${[...]} that would error or record its invocation shows the substitution did NOT run during a parse-only path, only for help display); first parse of punk::ansi::mark_columns drops from ~4s to well under a second with 'i punk::ansi::mark_columns' still rendering the embedded example, and a -help that parses its OWN definition id resolves or errors cleanly rather than looping; first-parse timing improves for at least one other heavily documented command (recorded in the detail file); rendering_atdynamic_multiline_help_insertion_GAP flips to all-aligned; choicegroups_imap_prefix_listwrap_GAP flips to shape-identical (prefix input yields the same plain string as exact input); the -return string renderer's cmd-help continuations align under the first line with relative indents preserved (rendering_string_renderer_characterization updated); the 'Bad number of leading values...' prefix shown by goodargs parsing in 'i string is'-style output is reworded or suppressed for the usage-display path; full punk::args and punk::ns suites pass with no non-GAP expectations weakened.
### G-049 [achieved 2026-07-10] punk::args parse-status data model with machine-parsable cmdhelp returns → detail: goals/archive/G-049-punkargs-parse-status-model.md
Scope: src/modules/punk/args-999999.0a1.0.tm (arg_error, parse error dispatch, colour-scheme handling), src/modules/punk/ns-999999.0a1.0.tm (cmdhelp), src/tests/modules/punk/args/testsuites/args/usagemarking.test, src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test
Acceptance: cmdhelp -return dict distinguishes an incomplete, a fully-valid and an invalid argument set via per-argument statuses (received/ok/bad + overall scheme/message/form) with the structure documented; the table and string renderers derive their marking from that same structure, with rendered output unchanged except where the pinned GAP tests flip: badarg marking covers type/allocation failures not just choice violations (cmdhelp_GAP_no_badarg_marking_for_failed_typed_value), an explicit -scheme is honoured on the parse-failure path (cmdhelp_GAP_explicit_scheme_ignored_on_failure), the failure message names the queried command instead of cmdhelp's internal parse source line (cmdhelp_GAP_errormsg_leaks_internal_source), and scheme rendering no longer depends on or mutates shared colour state - the documented -scheme choice value 'nocolour' takes effect and repeated renders of the same call are identical regardless of prior scheme renders (usagemarking_GAP_scheme_nocolour_renders_with_leftover_colours, usagemarking_GAP_dash_nocolour_leaks_into_shared_array, usagemarking_GAP_dash_nocolour_leak_affects_later_info_render); all non-GAP characterization tests in usagemarking.test and cmdhelp.test pass unchanged.
### G-054 [achieved 2026-07-11] tclcore moduledoc: runtime-harvested 'string is' class choices with cross-version behavioural parity pins → detail: goals/archive/G-054-tclcore-stringis-harvest.md
Scope: src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (+ tclcore-buildversion.txt), src/tests/modules/punk/args/testsuites/args/ (new parity test), TEMP_REFERENCE/tcl9 (read-only reference)
Acceptance: parse/parse_status against ::tcl::string::is and its per-class virtual ids agrees with the real interpreter's error-vs-ok outcome for a pinned probe matrix (missing args, trailing flag-like str word, option/class unique-prefix acceptance and ambiguity rejection, unknown option/class, -failindex var consumption leaving no str, per-version class presence: dict, unicode) on Tcl 9.0.x and 8.6; the rendered choices show only classes the running interp accepts; the parity test derives expectations from the live interpreter (not version arithmetic) and passes under both; existing args/tclcore suites pass; tclcore buildversion bumped with changelog.
### G-058 [achieved 2026-07-10] Boot honours statically-linked runtime packages (static baseline seeding + packagepreference static-awareness) → detail: goals/archive/G-058-static-runtime-packages.md
Scope: src/vfs/_config/punk_main.tcl (boot auto_path/tm path filtering), src/modules/punk/packagepreference-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm + src/modules/punk/repl/codethread-999999.0a1.0.tm (code interp / codethread bootstrap), src/modules/shellthread-999999.0a1.0.tm (punkshell-created worker threads) as applicable, src/tests/ (un-gated unit tests + constraint-gated shell/kit integration tests), punkbin artifact repo (separate git repo, local checkout c:/repo/jn/punkbin - pinned runtime additions)
Acceptance: a punk91-style kit (tclsfe-x64 + punk9win.vfs, no thread dll in the vfs) boots to a working repl with no "can't find package Thread" - punk::console loads in the code interp, and package require Thread succeeds there and in a punkshell-created worker thread, resolving to the static version; in the same kit, twapi resolves per the documented policy (no repeat of the observed static-Twapi-masked-by-older-vfs-twapi-5.0.2 double load; a genuinely newer bundled copy remains reachable by that policy); dll-based kits (punk902z) boot and pass their existing shell test baseline unchanged, as does a plain tclsh dev launch; the seeding mechanism is generic - driven by the captured baseline, no runtime-specific package naming - and the boot-time static baseline is introspectable at the repl; the seeding/preference logic is covered by un-gated unit tests against simulated baselines (runnable under plain tclsh), while kit-boot integration tests are gated behind a capability-probed tcltest constraint (a built kit whose baseline shows static entries including Thread) that skips cleanly when no such kit is present; the runtimes used for verification (tclsfe-x64.exe at minimum) are added to the punkbin artifact repository under win32-x86_64 with sha1sums.txt updated, so the constraint is satisfiable on other machines via the existing runtime-retrieval path; the punk91 code-interp vfs/vfs::zip load failure is re-diagnosed after the fix and either resolved or recorded as a distinct issue/candidate goal.
### G-059 [achieved 2026-07-11] WSL detection and suitability probing for driving unix-side tests from Windows → detail: goals/archive/G-059-wsl-test-driving.md
Scope: src/tests/ (capability probe helpers + constraint-gated cases in existing suites, e.g the unix sh-payload execution test in modules/punk/mix/testsuites/scriptwrap/multishell.test and a runtime.bash behaviour test), src/tests/AGENTS.md (enablement notes)
Acceptance: a documented probe helper yields a wsl_linux_available constraint whose checks are capability-based (distro launches and answers uname/tool probes; staging into a native tempdir works) and which cannot misfire on wsl.exe-present-but-unusable installs (no distro, WSL1 limitations, broken interop); the currently unix-gated multishell sh-payload execution test runs green via WSL on a suitable machine and still skips cleanly elsewhere; at least one runtime.bash behaviour test (active/use/run resolution against a fixture runtime folder) runs inside WSL - all such tests executing from a native-filesystem staging dir with the shared path used only for one-way copy-in/out; the Windows checkout's git and fossil state is untouched by a WSL-gated run (verifiable: git status/fossil changes identical before and after); suite results on a WSL-less machine are unchanged (skips, not failures); enablement/limitations and the staging pattern recorded in src/tests/AGENTS.md.

232
GOALS.md

@ -1,411 +1,275 @@
# Project Goals
This file is the canonical, harness-agnostic index of technical project goals for ShellSpy. It is referenced from the root `AGENTS.md` Child DOX Index and is a required read for any non-trivial work, so that agents can discover goals whose scope intersects their work.
This file is the canonical, harness-agnostic index of technical project goals for punkshell. It is referenced from the root `AGENTS.md` Child DOX Index and is a required read for any non-trivial work, so that agents can discover goals whose scope intersects their work. It is deliberately summary-only so that reading it in full stays cheap: each entry carries ID, status, title and Scope plus a detail-file pointer, and the goal's full contract prose (Goal statement and Acceptance criterion) lives in its detail file under `goals/` (see `goals/AGENTS.md`).
Detail prose for goals that need it lives in `goals/G-<id>-<slug>.md` (see `goals/AGENTS.md`). The index entry is canonical; a detail file only elaborates and never contradicts its index entry.
## How agents use this index
1. Read this file in full (it is small by design).
2. Identify every entry whose Scope intersects the paths or module areas being inspected or edited. Scope lines name real repo paths, so path search works (e.g. searching this file for `punk/repl` finds the goals touching that module).
3. Before editing those paths, read the detail file of each intersecting goal.
## Canonicality
- The index entry (this file) is canonical for ID, status, title and Scope.
- The detail file is canonical for the Goal statement and the Acceptance criterion.
- The tiers must not contradict: `Status:` and `Scope:` repeated in a detail file mirror the index. On disagreement the index wins for status/title/Scope and the detail file wins for Goal/Acceptance; fix the stale copy.
## Format
Each goal is one block:
Each goal is one compact entry:
```
### G-<id> [<status>] <short title>
Scope: <repo paths or module areas this goal touches>
Goal: <one line what done looks like, self-contained>
Detail: goals/G-<id>-<slug>.md <- optional, omit if absent
Acceptance: <measurable, verifiable pass/fail criterion>
Detail: goals/G-<id>-<slug>.md
```
Every goal has a detail file; it holds the canonical Goal and Acceptance plus any supporting prose (structure in `goals/AGENTS.md`). The index entry must stay safe standalone: title plus Scope alone must be enough that an agent who reads no detail file still does no harm.
### Status tags
- `proposed` — not yet started; awaiting user confirmation to go `active`
- `active` — in progress
- `achieved <YYYY-MM-DD>` — done; kept as a one-line record
- `achieved <YYYY-MM-DD>` — done. The flip includes archiving: the entry moves to `GOALS-archive.md` as a summary record and its detail file to `goals/archive/`. Achieved entries do not accumulate in this file.
- `abandoned` — dropped; one line on why stays in the entry
- `superseded by G-<id>` — replaced; do not delete the old entry
### Maintenance rules
- Goals are user-owned. Agents add or edit goal entries only at the user's request or with the user's explicit approval - never on their own initiative, and never in bulk from an agent's own survey of what "should" be goals.
- Suggesting is always allowed and encouraged when grounded in the work at hand: a discovered gap, a recurring manual step, a deferred design decision, or a natural follow-on that fits what is currently being worked on. Flag it as a candidate goal in conversation or the completion report, optionally with a drafted block ready for approval. A suggestion is not an edit - nothing is written to this file or `goals/` until the user approves per the proposal-first rule.
- Proposal-first: before writing a new goal entry or changing an existing entry's contract (title, status tag, Scope, Goal, Acceptance), show the user the proposed wording - the full block for a new goal, the changed clause(s) for an edit - and get explicit approval. If the user already supplied or approved the exact wording this session, apply it and report what was written.
- Exception (sanctioned autonomous edit): an agent whose work satisfies a goal's `Acceptance:` must flip that goal to `achieved <date>` as part of its DOX closeout pass, and report the flip in its completion summary.
- Suggesting is always allowed and encouraged when grounded in the work at hand: a discovered gap, a recurring manual step, a deferred design decision, or a natural follow-on that fits what is currently being worked on. Flag it as a candidate goal in conversation or the completion report, optionally with a drafted entry ready for approval. A suggestion is not an edit - nothing is written to this file or `goals/` until the user approves per the proposal-first rule.
- The goal contract spans both tiers: the index entry (title, status tag, Scope) and the detail file's `Goal:` and `Acceptance:` lines.
- Proposal-first: before writing a new goal or changing any contract element in either tier, show the user the proposed wording - the full entry plus detail-file header for a new goal, the changed clause(s) for an edit - and get explicit approval. If the user already supplied or approved the exact wording this session, apply it and report what was written.
- Exception (sanctioned autonomous edit): an agent whose work satisfies a goal's `Acceptance:` (judged against the detail file's criterion, never the index entry alone) must flip that goal to `achieved <date>` as part of its DOX closeout pass, archive it (entry to `GOALS-archive.md` per that file's format, detail file to `goals/archive/`), and report the flip in its completion summary. If the detail file carries a `## Progress` section, the flip additionally requires its remaining-work list to be resolved — empty, or each item verified satisfied; a partial increment never flips a goal.
- Agents must not flip `proposed``active`. They flag it in their completion report for the user to confirm.
- Detail files under `goals/` may be updated without pre-approval when recording findings, decisions, or verification artifacts from work the user directed on that goal or its subject matter; report such updates in the completion summary. The index entry is canonical - a detail edit must never contradict it.
- The `Goal:` line must stay self-contained enough that an agent who skips the detail file still does no harm. Detail files are enrichment, not load-bearing for safety.
- When the achieved section grows past ~30 entries, the oldest are moved to `GOALS-archive.md` and their detail files to `goals/archive/`.
- If a goal cannot be safely summarized in one line, that is a signal it is really two goals — split it.
- Index entries carry no progress: the status tag is the only state the index records. Incremental progress on an `active` goal (what landed, what remains) is recorded in the detail file's `## Progress` section, never as annotations on the index entry.
- Marking a goal `superseded by G-<id>` or `abandoned` includes a test sweep: search the tree (at minimum `src/tests`) for `G-<old id>` references and for the tests named in the goal's detail-file Acceptance, and record each affected test's disposition in the superseding goal's detail file (or the abandoned goal's own): pinned expectations that transfer to the new goal, pins that stand down to plain behaviour characterization, and any that lapse. The sweep never deletes, skips, or weakens a test on its own — that still requires explicit user direction per `src/tests/AGENTS.md`.
- Detail files may be updated without pre-approval when recording findings, decisions, or verification artifacts from work the user directed on that goal or its subject matter; report such updates in the completion summary. Edits to a detail file's `Goal:` or `Acceptance:` lines are contract changes and follow proposal-first.
- If a goal cannot be summarized safely by its title and Scope alone, that is a signal it is really two goals — split it.
## Authoring a new goal (for agents)
When the user asks to "write a goal for X" or "help me draft a goal for Y", do the following:
1. Read this file in full so you know the format and can pick the next free `G-<id>`.
1. Read this file in full and pick the next free `G-<id>`. IDs are never reused: the next free ID is one past the highest ID across `GOALS.md` **and** `GOALS-archive.md` (achieved goals leave this file, so ID gaps here are normal).
2. Ask the user only the questions below. Do not invent answers; ask them one at a time or batched if the user prefers. Stop asking once every required field has a real answer.
- **Scope:** Which repo paths or module areas does this goal touch? (paths are preferred; module names are acceptable if paths are not yet known)
- **Goal:** In one sentence, what does done look like? Push for an outcome, not an activity ("X compiles to bytecode ≤ 1.10× cost of Y", not "improve compiler performance").
- **Goal:** In one or a few sentences, what does done look like? Push for an outcome, not an activity ("X compiles to bytecode ≤ 1.10× cost of Y", not "improve compiler performance").
- **Acceptance:** What is the measurable, verifiable pass/fail criterion an agent can check against? If the user cannot state one, propose 2-3 candidate criteria and ask them to pick or refine.
- **Status:** Default to `proposed` unless the user says it is already in progress (`active`).
- **Detail file?** Only if the goal has non-obvious rationale, a multi-phase plan, alternatives worth recording, or needs more than ~3 lines of prose to state properly. If yes, propose a slug and offer to draft the detail file too. If no, omit the `Detail:` line.
3. Draft the goal block in this file's format and show it to the user for review, applying it only after approval (per the maintenance rules). Do not set it `active` unless the user confirms.
4. If a detail file is warranted and the user approves, create `goals/G-<id>-<slug>.md` using the structure in `goals/AGENTS.md`.
5. Do not delete or rewrite existing goals to make room for a new one. Append with the next free ID.
3. Draft the index entry (this file's format) and the detail file (header block per `goals/AGENTS.md`, plus Context/Approach/etc. sections only when there is real content for them) and show both to the user for review, applying them only after approval (per the maintenance rules). Do not set it `active` unless the user confirms.
4. Do not delete or rewrite existing goals to make room for a new one. Append with the next free ID.
## Goals
<!-- Append new goals below using the format above. Keep the list ordered by G-<id>. -->
### G-001 [proposed] Pluggable console backends for non-detectable terminals
Scope: src/modules/opunk/console-999999.0a1.0.tm, src/modules/punk/console-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm, src/lib/app-punkshell/punkshell.tcl
Detail: goals/G-001-pluggable-console-backends.md
Goal: an interactive REPL can be launched against a non-detectable terminal-like device (ssh channel, tk text widget) via an ::opunk::Console subclass, with no edits to the base class or punk::console.
Acceptance: a subshell started with an ssh-channel-backed and a tk-widget-backed ::opunk::Console subclass runs an interactive REPL that reads/writes through that console; size, at_eof, and can_respond are answered by the subclass overrides; the base ::opunk::Console and punk::console module are unchanged.
<!-- Append new goals below using the format above. Keep the list ordered by G-<id>. IDs are never reused; achieved goals move to GOALS-archive.md, so ID gaps are normal. -->
### G-002 [proposed] Non-nested subshell with console targeting and inter-subshell comms
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm
Detail: goals/G-002-non-nested-subshell.md
Goal: a subshell can target a named console (default or non-default) and run without blocking the parent, replacing the synchronous nested interp-eval model.
Acceptance: a parent REPL launches a subshell against a named console and continues processing its own input while the subshell runs; the parent can signal/query the running subshell; thread::send -async dispatched from within the subshell's code interp arrives at that interp (so packages like promise work when thread features aren't disabled); the "first subshell asymmetry" TODO at repl-999999.0a1.0.tm:3130 is resolved; existing synchronous `subshell punk`/`safe`/`safebase`/`punksafe` behaviour is preserved as a default mode.
### G-003 [proposed] Configurable resource limits and sandboxing on subshell interps
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm
Detail: goals/G-003-subshell-resource-limits.md
Goal: a subshell's code interp can be launched with configurable resource limits (command-count, time) and sandboxing features, building on the resolved first-subshell asymmetry from G-002.
Acceptance: a subshell can be launched with at least one resource limit (command-count via `interp limit -command`, or time via `interp limit -time`) and one sandboxing feature (e.g. `interp hide` of a command, or full safe-interp restrictions) applied to its code interp, enforceable regardless of subshell nesting depth; a subshell can be configured anywhere on the spectrum from unrestricted to fully safe via expose/hide of commands; the existing default subshell behaviour (no limits, no extra sandbox beyond the existing safe/safebase/punksafe types) is preserved when no limits are configured.
### G-004 [proposed] No executable binaries committed to the repository
Scope: repo-wide (bin/, src/vfs/, src/vendorlib/, src/vendormodules/, src/bootsupport/)
Detail: goals/G-004-no-committed-binaries.md
Goal: the committed repository contains no executable binaries; zip-based .tm modules are allowed but not if they embed executables.
Acceptance: a scan of the committed tree finds no executable binaries (shared libs, .exe, native .so/.dll/.dylib, bare ELF/Mach-O); any zip-based .tm modules present contain no embedded executables; the binary artifacts previously committed are retrievable via G-005 (build from source) or G-006 (pre-built download) so their removal does not break builds.
### G-005 [proposed] Zig-based build infrastructure for binary dependencies from source
Scope: src/runtime/, build.zig / build.zig.zon (new), src/make.tcl integration
Detail: goals/G-005-zig-build-infrastructure.md
Goal: a zig-based build system retrieves and builds binary dependencies (including Tcl9) from source, replacing the committed-binary approach for the vendored/native components.
Acceptance: running the zig build produces the binary artifacts the repo previously committed (at minimum: Tcl9 library for one target platform); existing Tcl9-zig experiments brought into the project; `tclsh src/make.tcl` integrates with the zig build so a normal project build retrieves/builds binaries via zig when not present; no binary artifacts need to be committed for the build to succeed on a clean checkout with the zig toolchain available.
### G-006 [proposed] Optional pre-built binary artifact download with consent gating
Scope: src/runtime/, src/make.tcl integration, user-config (consent flags)
Detail: goals/G-006-prebuilt-artifact-download.md
Goal: pre-built binary artifacts can be downloaded from a separate related binary-artifacts repository or user-configured sources, gated by explicit user consent/configuration by default.
Acceptance: a download mechanism fetches binary artifacts (the same set the zig build produces) from a configured source on demand; by default the download is gated behind explicit user consent (a config flag or interactive prompt) and does not occur silently; a user-configured source URL overrides the default binary-artifacts repo; downloaded artifacts satisfy the same build requirements as zig-built artifacts so `tclsh src/make.tcl project` succeeds with downloaded artifacts in place of built ones.
### G-007 [achieved 2026-07-05] Location-transparent punk::console across repl and code interps
Scope: src/modules/punk/console-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm
Detail: goals/G-007-console-location-transparency.md
Goal: punk::console presents one API in every interp/thread of a punk session - code-interp callers see the same console facts and can perform the same queries/operations as the parent, routed to the console-owning context via a punk-side ownership registry, with no `repl eval` required.
Acceptance: from a running punk session's code interp, without `repl eval`: `console_fact_get` returns the same values the parent sees and a fact set in the parent is immediately visible; a terminal query (e.g. `get_cursor_pos` or `dec_get_mode`) against the default console succeeds and cooperates with the repl reader (no lost or garbled input); a console constructed and owned by code-interp code is operated on locally (no round-trip to the parent); the existing console test suites pass and single-interp (non-repl) usage is unchanged.
### G-008 [proposed] Scoped console state for same-console subshells (activatable state sets)
Scope: src/modules/punk/console-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm
Detail: goals/G-008-scoped-console-state.md
Goal: a subshell on the shared console can opt into scoped console state - its terminal mutations (tabstops, modes, cursor style, palette, title) are captured as an activatable per-subshell state set and the prior state is re-established on quit or switch-away, with today's shared/persistent behaviour remaining the default.
Acceptance: with a scoped subshell: tabstops, at least one DEC/ANSI mode and at least one palette entry changed inside the subshell are restored for the parent on quit (verified by terminal query where queryable) and the facts store matches the terminal; the default (unscoped) launch behaviour is unchanged; activation is set-based, proven by applying state set A, activating set B, then re-activating A and observing A's state; irreversible outputs (cleared screen/scrollback, emitted text) are documented as out of scope.
### G-009 [proposed] Themed subshell profiles binding poshinfo schemes to behavioural configuration
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/poshinfo-999999.0a1.0.tm, src/modules/punk/console-999999.0a1.0.tm, src/modules/punk/island-999999.0a1.0.tm
Detail: goals/G-009-themed-subshell-profiles.md
Goal: a subshell can be launched from a named profile binding a poshinfo-enumerated scheme to behavioural aspects (e.g. G-003 interp limits/hidden commands, punk::island filesystem access) so that restricted subshells are visually distinct, with the theme's terminal effects and associated profile data riding the G-008 state set.
Acceptance: a named profile associating a poshinfo scheme with at least one restriction aspect (an interp limit, a hidden command, or island-restricted filesystem access) launches by name; the scheme's visual state applies on entry (at minimum prompt styling plus one underlying terminal aspect such as a palette change) and is removed/restored on quit via the G-008 mechanism; profile-associated non-terminal data is scoped to the subshell's lifetime; an unthemed launch is unchanged.
### G-010 [proposed] Subshell suspend/resume and tree navigation
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm
Detail: goals/G-010-subshell-tree-navigation.md
Goal: subshells form a navigable tree - a subshell can be suspended rather than quit, listed, resumed, and the console switched to any live subshell in the tree (e.g. grandchild to grandparent, or across branches) with each subshell's console state re-applied via its G-008 state set, building on the non-nested subshell model of G-002.
Acceptance: from a grandchild subshell a single switch command reaches the grandparent without unwinding through the intermediate parent; switching between subshells on different branches preserves each subshell's session state and re-applies its console state set on activation; suspended subshells can be listed and resumed; `quit` still unwinds to the launching parent as today; a subshell whose switch commands are hidden/restricted cannot initiate switches.
### G-011 [proposed] Optional per-console err channel with defined stderr semantics
Scope: src/modules/punk/console-999999.0a1.0.tm, src/modules/opunk/console-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm
Detail: goals/G-011-console-stderr-semantics.md
Goal: a console optionally carries an err channel as an attribute of its canonical {in out} identity - {in out err} specs accepted everywhere -console is, err resolving to process stderr for the default console and to the console's out channel elsewhere - so diagnostics and emit-to-err are first-class per-console operations instead of raw puts stderr.
Acceptance: console_spec_resolve and every -console site accept an {in out err} spec (err optional; existing pair/instance-name/object spec forms unchanged); opunk::Console exposes the err channel (nullable, additive base-class change); an unset err resolves to stderr for the default console and to the console's own out channel otherwise; punk::console's own warnings/diagnostics emitted while operating on a resolvable console go to that console's err (raw puts stderr remains only where no console is in play); an emit-to-err path exists and is exercised by at least one real consumer (e.g punk::repl); the effective err is discoverable from any thread/interp via console_fact_get (fact key err, returning the effective err channel name); ownership/fact/mode-cache keys remain canonical {in out}; the existing console test suites pass unchanged.
### G-012 [proposed] Template system: inert VCS-config payloads and explicit layout refresh
Scope: src/project_layouts/, src/make.tcl, src/modules/punk/mix/ (layout instantiation), fauxlink module (bootsupport 0.1.1 - promoted if chosen as mechanism)
Detail: goals/G-012-template-payload-safety.md
Goal: project layouts carry no live nested VCS-config files - template .gitignore payloads are stored inert (renamed, or fauxlink-encoded) and materialized at project generation - and src/make.tcl has an explicit punkcheck-tracked step that refreshes layout payloads from their canonical sources.
Acceptance: a scan of src/project_layouts finds no file named .gitignore, and git check-ignore --no-index over every layout file matches only root-.gitignore rules (nested rules provably inert); a project generated from each affected layout receives a working .gitignore whose content matches the canonical payload/target; after editing a canonical source (e.g. root .gitignore), the make.tcl template-refresh step updates the derived layout payloads (punkcheck-tracked), covering vendor/punk layouts as well as custom/_project; the previously hidden template files (layout READMEs, vendored TODO-class files) remain git-tracked without per-file force-add exceptions.
### G-013 [proposed] Raw mode as the repl's default input mode
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/console-999999.0a1.0.tm
Detail: goals/G-013-raw-mode-default.md
Goal: a repl launched without explicit mode configuration starts in raw input mode with a clean display - every per-keystroke debug emission behind its own repl-operable toggle - and raw editing covers the line-mode essentials (history navigation, cursor movement).
Acceptance: a default launch lands in raw mode with line mode still selectable; with debug toggles at their defaults (off unless stored configuration via ::punk::config says otherwise - see G-014), typing/editing/submitting a command emits no cursor-positioned debug output; the per-keystroke add_chunk frame and the right-hand live editbuf view are gated separately, each toggleable from within a running repl; arrow-key history navigation and left/right cursor movement work in raw mode (current stubs replaced); the marked-line debugrepl output form is retained and works on terminals without cursor addressing (e.g. vt52); the debugrepl first-word activation mechanism is reviewed and the keep/replace outcome (e.g. a proper Tcl command that interp hide can restrict in subshells) is recorded in the detail file; on tcl 8.6 a background-initiated terminal query at an idle raw-mode prompt succeeds (the residue scenario fail-fast-guarded in punk::console 0.7.1).
### G-014 [proposed] ::punk::config stored configuration: toml files with named-subshell scoping
Scope: src/modules/punk/config-0.1.tm, src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm, src/vendormodules/tomlish-*.tm (vendored; canonical source in the external tomlish project space)
Detail: goals/G-014-punk-config-toml.md
Goal: ::punk::config loads stored configuration from toml files in the XDG-located config dir - parsed via the vendored tomlish module, never an ad-hoc parser - and consumers resolve settings with per-named-subshell overrides, so features like the G-013 debug-view startup defaults read declared user configuration instead of hardcoded fallbacks.
Acceptance: a setting declared in a toml file under the XDG-located config dir is visible through the punk::config API at repl startup, and with no config files present built-in defaults apply with no errors beyond the existing missing-dir notice; a named subshell resolves its own overriding value for a key also defined at the parent/default scope, and a subshell with no override inherits the outer value (proven with at least one real key); at least one shipped feature (the G-013 editbuf-view startup default is the natural first) reads its default through this path rather than a hardcoded value; all toml reading/writing in punk::config goes through the tomlish module, and the tomlish API procs punk::config consumes carry punk::args (PUNKARGS) documentation - added upstream in the tomlish project and re-vendored here before punk::config implementation proceeds.
### G-015 [achieved 2026-07-07] Punk executable `script` subcommand: reliable non-interactive piped/script execution
Scope: src/vfs/_config/punk_main.tcl, src/lib/app-punkshell/punkshell.tcl (script path or a leaner dedicated app package)
Detail: goals/G-015-script-subcommand-piped-stdin.md
Goal: `<punkexe> script [<scriptname>] [<args>...]` executes a script file or piped stdin content (scriptname optional when input is piped) in an interp preloaded with the basic punk modules and aliases a punk shell provides by default, and always terminates with the script's success/failure as its exit code - without the `shell` subcommand's shellfilter channel transforms/logging stacks and without ever dropping into an interactive shell - so agents can reliably make piped script calls to punk executables.
Acceptance: piping commands to `<punkexe> script` runs them and terminates at stdin EOF with no trailing `exit` required, exit code 0 on success; a failing piped command terminates the process with a nonzero exit code and the error on stderr, never landing in an interactive shell regardless of console availability or PUNK_PIPE_EOF; `<punkexe> script <file> [<args>...]` executes the file with conventional ::argv0/::argv and propagates its error status the same way; the script path installs none of the `shell` subcommand's shellfilter stacks/transforms and the launch plumbing itself emits nothing on stdout/stderr (the current stub's stderr diagnostics removed) so exec-style callers see only the script's own output; the motivating example works with no package require boilerplate: piping `dev projects.work *<name>*` to `<punkexe> script` emits the matching-project table and exits 0, because the script interp carries the default punk shell module/alias environment.
### G-016 [proposed] `projects.work` discovers git-based projects alongside fossil
Scope: src/modules/punk/mix/commandset/project-999999.0a1.0.tm, src/modules/punk/repo-999999.0a1.0.tm
Detail: goals/G-016-projects-work-git-discovery.md
Goal: `dev projects.work <glob>` lists git-based project checkouts as well as fossil-based ones, each result identifying its VCS - fossil discovery stays central-config-db based, and git discovery uses a defined enumeration source (git has no central registry; the chosen mechanism is recorded in the detail file).
Acceptance: with a git-only project on disk registered in the chosen enumeration source and matching the glob, `projects.work` lists its working directory and identifies it as git; existing fossil results are unchanged apart from any added VCS-identifying column; a project that is both git and fossil (e.g. this repo) appears with both indicated rather than duplicated; glob matching remains case-insensitive.
### G-017 [proposed] Agents locate local projects via piped `projects.work` calls, not filesystem scanning
Scope: AGENTS.md (root) or a child doc it indexes (guidance content only - no code)
Goal: once G-015 makes piped script calls reliable, repository guidance directs agents asked to locate another local project to query it via a piped `projects.work` call to a punk executable instead of grepping/globbing the wider filesystem.
Acceptance: root AGENTS.md (or a child doc indexed from it) records the exact recommended invocation - executable, subcommand, glob usage, expected output shape - and states when filesystem scanning remains appropriate (projects not registered in any discovery source); the guidance is added only after G-015 is achieved (and notes the fossil-only limitation until G-016); following the documented pattern, an agent locates a named sibling project's checkout dir with a single piped call.
Detail: goals/G-017-agent-project-discovery.md
### G-018 [proposed] Zig-built plain tclsh kits: self-contained zip-based executables without punk infrastructure
Scope: build.zig / build.zig.zon (per G-005), src/runtime/, src/make.tcl integration
Detail: goals/G-018-zig-plain-tclsh-kits.md
Goal: developers can use the G-005 zig build system to produce self-contained zip-based tclsh executables that carry no punk-specific infrastructure (no punk boot layer, punk modules, or punk apps) - plain tclsh kits usable independently of the punkshell product.
Acceptance: a documented zig invocation on a clean checkout (zig toolchain available) produces a zip-based tclsh executable for at least one target platform that runs conventional tclsh invocations (`<exe> script.tcl args`, piped stdin) on a machine with no Tcl installation; a listing of the kit's mounted/zip contents shows stock Tcl (plus any declared stock runtime deps) and no punk namespaces, punk boot files, or punkshell apps; the punk-flavoured executables remain producible alongside.
### G-019 [proposed] Dependency-scan-driven module trimming for punk-based executables
Scope: src/make.tcl, src/modules/punk/lib-999999.0a1.0.tm (tclparser use), src/vfs/ (kit assembly), scanning module (new or existing punk module - to be determined)
Detail: goals/G-019-dependency-scan-module-trimming.md
Goal: a package-dependency scan from an executable's entrypoint (candidate basis: the tclparser parse API - currently satisfied only by the c-only tclparser library, with punk::lib's pure-Tcl fallback an unimplemented stub) determines the module closure the executable actually requires, so a build can ship only those modules - while 'batteries included' builds remain a supported alternative, not a casualty.
Acceptance: for at least one punk-based executable target, the build can run a dependency scan from its entrypoint producing the closure of required packages/modules plus a mechanism to declare dynamically-loaded extras the scan cannot see; a trimmed kit assembled from that closure starts and passes its basic function check (e.g. repl launch or the app's smoke test) with no missing-package errors; the trimmed kit's module listing is a strict subset of the batteries-included equivalent (demonstrating real exclusion); batteries-included builds remain producible unchanged.
### G-020 [proposed] Screen capture and input injection module with per-platform backends (Windows first)
Scope: src/modules/punk/ (new module - name TBD), src/vfs/punk9win.vfs/lib_tcl9/ (existing treectrl/Img/twapi payloads); scriptlib/aloupe.tcl stays untouched as a standalone app
Detail: goals/G-020-screencap-input-module.md
Goal: a punk module drives screen/window capture and mouse/keyboard injection from scripts via per-platform backends - Windows (treectrl loupe capture + twapi input/window-location) is the initial complete target, with the backend contract designed so X11 (Linux/FreeBSD) and macOS backends can be added without changing callers, and Wayland-native sessions explicitly out of scope.
Acceptance: on Windows from a punk shell or script: a screen region and a window located by title/class pattern are each captured to a Tk photo and written as a valid PNG; mouse movement/click and key events injected into a located test window produce their observable effect (typed text arrives, click acts); window location returns the handle and geometry for a pattern; a capability-introspection call reports per-feature support and an unsupported platform/backend yields a clean capability-based refusal, not a crash; the backend interface (capture / input / window-locate) is documented well enough that a non-Windows backend can be added without modifying callers; the aloupe script remains functional and unmodified.
### G-021 [proposed] Agent-drivable visual/UI verification via piped snapshot and interaction calls
Scope: src/modules/punk/ (G-020 module's agent-facing surface), AGENTS.md guidance (post G-015 pattern), src/tests/ (visual-verification test hooks)
Detail: goals/G-021-agent-visual-verification.md
Goal: a tool-calling agent can, during a session, use piped script calls (G-015) to a punk executable to locate the applicable UI window, snapshot it to a PNG file and/or base64 output suitable for AI image analysis, and drive mouse/keyboard interactions - enabling tests whose verification is visual-only and/or input-driven.
Acceptance: on Windows, single piped script calls (no interactive session) can: list/match windows for a pattern with machine-parseable output; save a located window's snapshot to a caller-specified path and optionally emit it base64 on stdout; run a scripted interaction sequence (focus, click at offset, type text, snapshot) end-to-end; failures exit nonzero with the error on stderr per G-015 semantics; the invocation patterns are documented for agents alongside the G-017 guidance; at least one real visual-or-input-driven verification (e.g. a Tk app smoke test) is exercised through this path.
### G-022 [proposed] Scriptable safe fossil move/rename in `dev repo`; rename this project's fossil repo to punkshell
Scope: src/modules/punk/mix/commandset/repo-999999.0a1.0.tm, src/modules/punk/repo-999999.0a1.0.tm, src/tests/modules/punk/mix/testsuites/repo/
Detail: goals/G-022-fossil-rename-punkshell.md
Goal: the `dev repo` commandset can move and rename fossil repositories non-interactively and safely - all checkouts repointed, no phantom central config-db entries, no dangling old repo db, fossil project-name renamable with project-code unchanged - and this project's fossil repo (currently project-name 'shellspy') is renamed to 'punkshell' through that mechanism via a G-015 piped script call, not by hand.
Acceptance: the commandset provides a flag-driven (no stdin prompts) move/rename operation which on a scratch repo with an open checkout: repoints every registered checkout, leaves the central config-db listing only the new path, removes or archives the old repo db file (per option), clears stale ckout: back-references, and applies a requested project-name change while preserving project-code; the GAP characterization tests in src/tests/modules/punk/mix/testsuites/repo/fossilmove.test are updated to assert the clean behaviour and pass; after G-015 is achieved, this repo's fossil db (shellspy.fossil / project-name shellspy) is renamed to punkshell via the new operation invoked through a piped `script` call, with `fossil info` in this checkout showing the new repository path and project-name and `fossil all ls` free of the old path.
### G-023 [proposed] Version-named punk binaries per Tcl generation (versioned / dev / release-gated plain names)
Scope: src/make.tcl, src/runtime/ (mapping config - see G-024), bin/ (build outputs)
Detail: goals/G-023-version-named-binaries.md
Goal: project builds produce version-named punk executables for tcl 8.6 and tcl 9 as the project version advances - punk8-<major>-<minor>-<patch>.exe / punk9-<major>-<minor>-<patch>.exe per version, punk8-dev.exe / punk9-dev.exe tracking the latest build, and plain punk8.exe / punk9.exe created initially then replaced only when an actual release is tagged - tolerating the disk growth for now.
Acceptance: a project build at the current punkproject.toml version produces punk8-<M>-<m>-<p>.exe and punk9-<M>-<m>-<p>.exe (names derived from the version, not hand-maintained) plus punk8-dev.exe / punk9-dev.exe updated to that same build; rebuilding at an unchanged version refreshes that version's binaries and -dev without touching other versions' outputs; plain punk8.exe / punk9.exe exist and are replaced only by an explicit release step - a normal build never overwrites them; the scheme is declared succinctly via the G-024 toml mapping (no per-version config edits); archival/deletion of accumulated versioned binaries is out of scope with the trigger question recorded in the detail file.
### G-024 [proposed] mapvfs.config converted to toml (tomlish-parsed) with succinct scheme declarations
Scope: src/runtime/mapvfs.config (replaced/deprecated), src/make.tcl (parsing), src/bootsupport/modules/tomlish-*.tm (parser dependency)
Detail: goals/G-024-mapvfs-toml.md
Goal: the runtime-to-vfs-to-executable build mapping moves from the custom line format of src/runtime/mapvfs.config to a toml file parsed with the tomlish package - still supporting explicit per-executable mappings (runtime, vfs folder, output name, kit type) while also expressing generative schemes like the G-023 versioned naming in a single succinct declaration.
Acceptance: a mapvfs toml file parsed via tomlish (no ad-hoc toml parsing) drives the build: every mapping currently active in mapvfs.config is expressible and at least one existing target builds identically from the toml; the G-023 versioned/dev/release-gated output scheme is declared in one entry that expands to its outputs without enumerating versions; malformed or unresolvable entries fail the build with a clear message naming the entry; the legacy .config format is either fully migrated (old parser removed) or explicitly deprecated with documented precedence between the two files.
### G-025 [proposed] Punk executables self-report project version and build provenance
Scope: src/vfs/_config/punk_main.tcl (subcommand dispatch), src/make.tcl (stamping build info into the vfs), src/vfs/ (stamp payload location), src/modules/punk/ (in-shell command - the single implementation)
Detail: goals/G-025-exe-selfreport.md
Goal: a punk executable reports its identity from embedded data rather than its filename - a documented subcommand prints the punkproject.toml project version it was built from plus the input runtime binary name and vfs folder name used to assemble it - with the same-named command available in the punk module so scripts running in any punk shell (including tclsh-hosted ones like `tclsh src/make.tcl shell`) get the same report in-process without exec, stamp fields reported as absent rather than fabricated when there is no stamp.
Acceptance: the build stamps project version, runtime binary name, and vfs folder name into the kit; the built executable invoked with the version-report subcommand prints those fields machine-parseably on stdout and exits 0 with no other output (G-015-compatible; no repl fallthrough); a same-named command in the punk module returns the same fields in-process (subcommand implemented as a wrapper over it - one implementation) and works from the code interp; the report distinguishes stamped provenance from live facts: a stamped kit reports its stamp, a `src`-mode or source-tree session additionally reports the live punkproject.toml version as a distinct field when it differs, and unstamped contexts (`tclsh src/make.tcl shell`, plain tclsh with punk modules) report stamp fields explicitly absent with live runtime facts (actual `info nameofexecutable`) still provided; the report is correct when the executable file has been renamed or copied; executables built before stamping existed fail gracefully with a clear message rather than fabricating values.
### G-026 [proposed] Enforceable clean-checkout provenance policy for vendor and bootsupport pulls
Scope: src/make.tcl (vendorupdate and bootsupport steps), src/vendormodules/include_modules.config, src/bootsupport/modules*/include_modules.config
Detail: goals/G-026-vendor-provenance-policy.md
Goal: pulling vendored or bootsupport artifacts from local source projects enforces committed provenance - the warn-only dirty-checkout check added to vendorupdate in project 0.2.5 becomes a policy that can abort with an explicit override, covers the bootsupport update path as well, and the residual staleness question (built modules that predate or postdate the committed source even in a clean checkout) has a recorded design decision.
Acceptance: vendorupdate and the bootsupport update refuse to pull from a source project whose fossil/git checkout is dirty unless an explicit documented override is given (warn-only selectable as a configured mode); the check reports each VCS root once per run and does not fire for unversioned source locations; bootsupport_localupdate is covered by the same shared check (no second divergent implementation); the staleness gap - a clean checkout whose built modules/ artifacts do not correspond to the committed source - is either detected (mechanism chosen and implemented) or explicitly recorded in the detail file as accepted risk with the considered mechanisms; behaviour is exercised by a test or documented manual verification against a scratch dirty checkout.
### G-027 [proposed] Pull-based infrastructure updates for punkshell-derived projects
Scope: src/modules/punk/mix/commandset/project-999999.0a1.0.tm (project.new push path), src/modules/punkcheck-999999.0a1.0.tm (install provenance records), src/make.tcl (derived-project pull entrypoint)
Detail: goals/G-027-derived-project-pull-updates.md
Goal: a project generated from a punkshell layout can pull infrastructure updates (make.tcl/build.tcl, bootsupport modules and libs, layout template payloads) from its originating punkshell project by running one command inside the derived project - replacing the current push model (`dev project.new -force 1 -update 1` run from punkshell) - with install provenance robust to derived-project workdir moves (not local relative paths alone), VCS-state awareness on both ends, and a recorded decision on pulling from remote sources.
Acceptance: one documented command run inside a derived project updates its punkshell-derived infrastructure from the source punkshell project; the update still works after the derived project's working directory has been moved (proven by moving a scratch derived project and pulling); VCS integration on both ends: the pull applies the G-026 clean-checkout policy to the punkshell source, and reports (or refuses per option) when target files it would overwrite carry uncommitted local modifications in the derived project's git/fossil checkout; .punkcheck records remain the provenance basis (updates are recorded and unchanged targets skipped, as with existing punkcheck-tracked installs); the push flow keeps working until explicitly retired; the remote-pull question (updating from a remote punkshell repository rather than a local checkout) has a recorded design decision - implemented, or deferred with rationale in the detail file.
### G-028 [proposed] Name the process locking a file when builds cannot replace a target
Scope: src/modules/punkboot/utils-999999.0a1.0.tm (locker-report helper), src/make.tcl (kit deploy failure reporting)
Detail: goals/G-028-file-locker-identification.md
Goal: when a build cannot delete/overwrite a target file (typically a built executable held open by another program), the failure message names the locking process(es) - via a punkboot::utils helper using the Windows Restart Manager API through optionally-available twapi or cffi, degrading cleanly to the current message when the API or bindings are unavailable or on other platforms.
Acceptance: on Windows with twapi or cffi loadable, a punkboot::utils proc given a file path returns the locking processes (at least pid and process name; empty list when unlocked); punkboot::utils itself stays pure Tcl - the binary binding is required lazily at call time and its absence yields a clean 'unavailable' result, not an error (bootsupport must not gain a compiled-extension dependency); make.tcl kit deploy failures include the locker report when determinable (e.g. "could not delete target binary ... in use by: 7zFM.exe (pid 1234)"); non-Windows platforms and binding-less environments produce the existing message unchanged; verified against a deliberately held handle (documented manual verification acceptable).
### G-029 [proposed] Build packaged test::<modulename> #modpod modules from src/tests
Scope: src/make.tcl (test-module packaging step), src/tests/ (source of truth), src/modules/test/ (generated #modpod targets), src/modules/punk/mix/ (modpod tooling as needed)
Detail: goals/G-029-testmodules-from-srctests.md
Goal: src/tests is the single source of truth for module test suites - a punkcheck-tracked make.tcl step generates the packaged test::<modulename> #modpod modules from src/tests/modules/<namespacepath>/testsuites content, ending hand-maintenance of parallel copies under src/modules/test/; the packaged form is a distributable in its own right - a user who downloads a built module can optionally download the matching test::<modulename> and verify the module's behaviour on their own system (package require + RUN, or an executable's -app test) with no source tree or test harness required.
Acceptance: a make.tcl step generates a test::<modulename> #modpod under src/modules/test/ from the corresponding src/tests testsuites (punkcheck-tracked, skipped when sources unchanged); the generated package works through the packaged path - loadable via package require test::<modulename> and runnable via its SUITE/RUN interface (e.g a built executable's -app test) - reporting the same test names and pass counts as running the same suites directly via src/tests/runtests.tcl; the consumer scenario is proven standalone: the generated package (plus the module under test and their dependencies) runs in an environment without the project source tree present; suite data files (e.g roundtrip toml files with deliberate crlf/mixed line endings) survive packaging byte-for-byte; documentation states src/tests is the source of truth and the generated modpods are build artifacts not to be hand-edited; proven end-to-end for at least one real module (tomlish, whose src/tests port and still-live modpod created the dual-copy situation, is the natural first).
### G-030 [proposed] make.tcl dogfoods punk::args: tabled usage, declared subcommands, prompt-free flags
Scope: src/make.tcl (dispatch, help, prompts), src/bootsupport/AGENTS.md + src/modules/AGENTS.md (bootstrap-tracked staleness contract), src/modules/punk/args-999999.0a1.0.tm (only as consumed)
Detail: goals/G-030-maketcl-punkargs.md
Goal: make.tcl - the first surface a developer sees - parses its subcommands and options via punk::args and showcases the tabled usage output for help and argument errors, every interactive y/n prompt gains a declared flag equivalent so agents can drive make.tcl with arguments instead of piped input, punk::args joins the bootstrap-tracked staleness set, and the boot phase plus the environment-repair commands keep working with degraded plain help when the bootsupport punk::args (or the table-rendering stack) is stale or unavailable.
Acceptance: `tclsh src/make.tcl` and `-help` render punk::args tabled usage listing every subcommand with a summary, and `make.tcl help <subcommand>` (or `<subcommand> -help`) shows that subcommand's definition; invalid arguments produce a punk::args usage error rather than ad-hoc messages; every y/n prompt has a documented flag equivalent (proven at least for vfscommonupdate and the project-build confirmations: a run with the flag completes non-interactively with stdin closed) and a non-interactive stdin without the flag fails fast with usage rather than hanging or half-aborting; punk::args is added to the bootstrap-tracked buildversion set with the doc contract updated (src/bootsupport/AGENTS.md, src/modules/AGENTS.md); with bootsupport punk::args unavailable or unloadable, make.tcl still boots and `check`, `bootsupport` and `modules` remain usable with plain-text fallback help (the guarded-require degrade rule); layout make.tcl copies follow via the established sync/G-027 channels (noted, not hand-synced).
### G-031 [proposed] Componentized kit boot: thin project-owned main + shared layout-owned boot core
Scope: src/vfs/_config/ (punk_main.tcl, project_main.tcl restructure), src/vfs/_vfscommon.vfs (boot core delivery), src/project_layouts/ (thin-main skeleton, via established sync channels)
Detail: goals/G-031-componentized-kit-boot.md
Goal: the per-project vfs main script becomes a thin project-owned file - declare the application's subcommands and launch defaults at clearly commented customization points, then hand over to a shared layout-owned boot core (vfs mounts, package modes and paths, libunknown, src-mode modpod registration) and default dispatch pulled in from within the kit - so project developers add app-specific subcommands without wading through or forking ~1000 lines of boot boilerplate, and boot improvements reach derived projects as pull-updatable payload instead of dying in vintage forks (tomlish_main.tcl: ~20 custom lines carrying a stale 500-line 2025 copy of the rest).
Acceptance: punkshell's own kits boot through a thin main plus shared boot core with behaviour parity - package modes including src mode, existing tclsh/shellspy/punk/shell/script dispatch semantics, and supported vfs types (zipfs/metakit/cookfs) all unchanged; a project-specific subcommand is added by editing only the thin main at a commented customization point (proven end-to-end in a derived project - tomlish replacing its forked main is the natural first); the boot core ships as layout-owned payload (via _vfscommon/layout channels) and the thin main as a project-owned skeleton, per the G-027 ownership classification; the boot core is versioned/identifiable so a kit can report which boot-core vintage it carries (ties to G-025 stamping).
### G-032 [proposed] Kit launcher dogfoods punk::args: tabled help and parsed subcommands
Scope: src/vfs/_config/ (default dispatch), src/lib/app-punkshell and sibling app packages as touched
Detail: goals/G-032-launcher-punkargs.md
Goal: the default launch dispatch defines its subcommands via punk::args - `<punkexe> -help` and argument errors render the tabled usage enumerating built-in and project-registered subcommands with summaries, and subcommand options parse through punk::args so projects can declare complex arguments - with the G-030 degradation rules (boot never fails and help degrades to plain text when punk::args or the ANSI rendering stack is unavailable).
Acceptance: `<punkexe> -help` renders tabled usage listing all subcommands including project-registered ones, each with a summary; at least one built-in subcommand's options are declared and parsed via punk::args with tabled usage errors on invalid input (the G-015 script subcommand or the G-025 version-report subcommand are the natural candidates); a project-registered subcommand's help appears by registration alone - no edits to the shared dispatch; with punk::args or the rendering stack unloadable, boot proceeds and help degrades to a plain subcommand list; verified on both a zipfs-based and a non-zipfs kit where both remain supported.
### G-033 [proposed] `proj:` package-mode scope prefix: visitor binary resolves dev/src against the cwd's project
Scope: src/vfs/_config/punk_main.tcl (package-mode dispatch and boot-time root discovery), src/modules/punk/repo-999999.0a1.0.tm (find_project / is_project_root - reuse or lean boot mirror), bin/AGENTS.md (mode docs)
Detail: goals/G-033-proj-mode-cwd-project.md
Goal: a `proj:` prefix on the package-mode string (e.g. `punkshell proj:internal-src shell`) makes the dev/src path blocks resolve against the punk project containing the current working directory - walking up to the nearest VCS repo root, the marker punk::repo::is_project_root already uses - instead of the executable's own project, so an installed ("visitor") punkshell binary, including one downloaded standalone with no source tree around it, can interactively explore a project that builds no shell-capable binary or no binary at all; the prefix scopes WHICH project, staying outside the ordered dash-list whose block order remains the same-version tie-break dial, and the explicit prefix gates the behaviour so no `cd` silently rebinds a normal launch.
Acceptance: `<installed-punkexe> proj:internal-src shell` (the documented canonical visitor invocation - kit copies win same-version ties, protecting the visiting shell's infrastructure; `proj:src` is the documented faithful-vintage variant where the project's copies win) launched from within a punk project's tree discovers the project root by walking up from cwd to the nearest VCS repo root and feeds it to the existing src-mode path machinery (src/modules, src/bootsupport/modules, src/vendormodules on the module path, src/lib on auto_path), so the session loads the project's dev-versioned modules while ordinary version resolution still lets the project supply anything the kit lacks or exceeds; the launch reports the detected project root and effective path precedence (never a silent rebind); with no project found walking up from cwd, or a `proj:` string containing no root-using block (dev/src), it warns and proceeds without false rebind; proven for a standalone binary outside any source tree (internal = kit contents regardless of binary location) against a project that builds no executable (tomlish is the natural first); exe-relative `src`/`dev` discovery is unchanged for a binary that IS in a project's bin/; the packagemode help text drafted in the detail file becomes the live punk::args documentation when implemented (rendered via G-032).
### G-034 [proposed] Zip-based #modpod modules mount in the shell code-interp on Tcl 8.6
Scope: src/modules/punk/mix/ (modpod mount path), vfs::zip availability in the repl code interp, src/modules/punk/cap/ (templates capability handler)
Goal: zip-based `#modpod` modules (e.g. `punk::mix::templates`) mount and their punk::cap handlers register in the `shell` subcommand's code interp on Tcl 8.6, matching the `script`/main-interp context - so `dev module.templates` and other `punk.templates`-capability consumers work in an interactive 8.6 shell instead of failing with `invalid command name vfs::RegisterMount`.
Acceptance: `dev module.templates` in an interactive 8.6 punk shell (`shell` subcommand) lists the template providers with no `failed to load ZIP archive-based module` / `invalid command name vfs::RegisterMount` / `Unable to register any template providers` / `invalid command name ::punk::cap::handlers::templates::api_punk.templates` errors, matching the `script`-subcommand output on the same binary (verified 2026-07-07: script works, shell fails); root cause fixed (the code interp lacks the vfs::zip library that provides vfs::RegisterMount for pre-zipfs Tcl, present in the main interp - restore it to the code interp or use an alternative modpod mount path there); Tcl 9 (built-in zipfs) behaviour is unchanged.
Detail: goals/G-034-modpod-codeinterp-tcl86.md
### G-035 [proposed] Characterise mixed .tm / pkgIndex.tcl provision of the same package
Scope: src/tests/modules/punk/libunknown/testsuites/ (characterization suite), src/modules/punk/libunknown-0.1.tm and src/modules/punk/packagepreference-999999.0a1.0.tm (as characterised, fixed only if outright bugs surface), src/modules/AGENTS.md + src/lib/AGENTS.md (resulting guidance)
Detail: goals/G-035-mixed-tm-pkgindex-provision.md
Goal: the behaviour when the same package is provided both as a .tm module and as a pkgIndex.tcl-based library - same or differing versions, under the standard package unknown, punk::libunknown and punk::packagepreference - is characterised by committed tests, and the currently informal working rule ("avoid mixing provision forms for one package - unexpected behaviour even with libunknown's improvements") is either substantiated with the specific failure modes named in AGENTS.md guidance, or retired if the characterisation shows the machinery now handles mixing predictably.
Acceptance: a committed test suite (extending src/tests/modules/punk/libunknown/testsuites/) characterises at least: same name+version provided via .tm and via pkgIndex.tcl (which registration wins, and whether it is deterministic across scan-trigger orderings) under the standard scanner, under punk::libunknown, and with punk::packagepreference active; differing versions across the two forms (version selection integrity including package prefer latest, and whether the losing form's registration lingers); re-registration effects (package forget then re-require crossing forms); surprising-but-accepted behaviours are pinned with GAP/known-quirk comments (the fossilmove characterization pattern), outright bugs fixed or filed as goals; the resulting do/don't guidance lands in src/modules/AGENTS.md and src/lib/AGENTS.md naming the characterised failure modes (or explicitly lifting the avoid-mixing rule if unwarranted).
### G-036 [achieved 2026-07-08] Root-cause the Tcl 9 console+udp worker-thread event-loop wedge; minimal repro for possible upstream reporting
Scope: src/modules/shellthread-999999.0a1.0.tm, src/bootsupport/modules/shellthread-1.6.2.tm, src/modules/shellfilter-999999.0a1.0.tm (as characterised - no product-code changes required by this goal; the punkshell mitigations are separate fixes)
Detail: goals/G-036-tcl9-udp-console-worker-wedge.md
Goal: the Tcl 9-only wedge - a worker thread that has used a tcludp syslog socket stops servicing its event queue (timers, thread::send) when the punkshell process is console-attached, the proximate trigger of the piped-stdin exit/quit hang - is root-caused to a named component (Tcl 9 Windows console driver, tcludp, Thread extension, or a specific interaction) with the smallest demonstrating repro, so the user can decide on and manually file an upstream report.
Acceptance: (reworked 2026-07-08 after the root cause was found) the wedge mechanism is identified and written up in the detail file - DONE: bundled tcludp 1.0.12's Windows per-thread UDP_ExitProc closes the process-global synchronization events, proven by dump handle-table analysis plus a live CloseHandle breakpoint and confirmed by the upstream 1.0.12->1.0.13 diff, which already fixes it (so no upstream report is required; the standalone minimal repro originally required here is waived as moot by the user); the tcl9 kits bundle tcludp >= 1.0.13 with the in-context batch harness baseline resolved - DONE 2026-07-08 (run-2 syslog workers alive vs the 4/4-wedged 1.0.12 baseline); DONE 2026-07-08 (punk::lib 0.3.0 has_libbug_udp_threadexit, surfaced via 'help tcl' in punk 0.2.1): a has_bug-style detection in the punkshell check machinery (in the vein of punk::lib::check::has_tclbug_* / punk::console::check::has_bug_*, surfaced through the same reporting as 'help tcl'/'help console') reports the vulnerable combination - simple version-based detection (loaded/bundled tcludp < 1.0.13 on Tcl 9 Windows) is sufficient, no behavioural probe needed; loose-end decisions (punk8win 8.6 kit's udp 1.0.12 swap; optional upstream tickets for residual tcludp trunk weaknesses) recorded in the detail file when made (open as of 2026-07-08 - non-gating).
### G-037 [achieved 2026-07-08] Propagate platform vendor libraries into kit vfs lib_tcl trees via make.tcl
Scope: src/make.tcl (new or extended step), src/vendorlib_tcl8 + src/vendorlib_tcl9 (sources), src/vfs/<kit>.vfs/lib_tcl8 + lib_tcl9 (targets), punkcheck tracking
Detail: goals/G-037-vendorlib-vfs-propagation.md
Goal: platform-specific vendored binary packages under src/vendorlib_tcl<N>/<platform> reach the kit vfs lib_tcl<N> trees through a make.tcl step instead of hand-copying - updating a vendored package becomes a vendorlib drop plus standard build invocations (motivating case 2026-07-08: tcludp 1.0.12 -> 1.0.13 for the G-036 wedge fix - `libs`, `vfscommonupdate` and `project` all completed while every kit vfs still bundled udp 1.0.12, requiring a manual copy into each vfs lib_tcl9 folder).
Acceptance: with a newer package version placed under src/vendorlib_tcl9/<platform>, one documented make.tcl invocation updates the participating src/vfs/*/lib_tcl9 trees - installing the new package and removing or explicitly retiring the superseded version (no silent mixed-version provision, per the G-035 concerns) - with punkcheck-tracked provenance; which vfs folders participate is explicitly declared per kit rather than blanket-copied (kit vfs package sets may intentionally differ), with the declaration mechanism recorded (candidate home: the G-024 mapvfs toml); a subsequent `make.tcl project` yields kits loading the new version (provable via the tcludp case: built punk902z reports `package require udp` == 1.0.13 with no udp1.0.12 folder remaining in its vfs); the lib_tcl8 tree gets the same treatment or an explicit exclusion rationale in the goal record.
### G-038 [proposed] Piped-to-interactive restart continues the same session (context preserved)
Scope: src/lib/app-punkshell/punkshell.tcl (eof-restart handover), src/modules/punk/repl-999999.0a1.0.tm (eof-restart done-mode that skips codethread teardown), src/modules/punk/repl/codethread-999999.0a1.0.tm (as touched)
Detail: goals/G-038-piped-session-continuity.md
Goal: when piped stdin ends and app-punkshell opens the interactive console shell, the session continues rather than restarts - the piped phase's codethread/code interp survives (variables, procs, namespaces, loaded packages, cwd, ::errorInfo/::errorCode) and only the input channel and repl reader are renewed - so `'set ::jjj blah' | <punkexe> shell` leaves ::jjj inspectable at the prompt, replacing today's silent fresh-session swap (least-surprise violation; motivating transcripts in the detail file) and obsoleting the standalone piped-error-record mechanism this goal described before its 2026-07-08 rework.
Acceptance: after `'set ::jjj blah;error xxxx' | <punkexe> shell` reaches the interactive prompt: `set ::jjj` returns blah; the xxxx message, errorInfo traceback and errorCode are inspectable (via preserved ::errorInfo if nothing in the handover overwrites it, else a documented record - the chosen mechanism noted in the detail file); a proc defined and a package required during the piped phase remain available; a cwd change from the piped phase persists; a one-line notice at the restart says the session continued from piped input (mentioning the error when the last piped command errored, quiet otherwise); after the restart a terminal query from the code interp succeeds (e.g. `help console` runs its cursor-position test instead of being refused by the stale settled can_respond=0 anchor); interactive exit/quit teardown afterwards is clean (the G-036 regression harness still passes); PUNK_PIPE_EOF policy semantics and the script subcommand are unchanged; verified on both Tcl generations.
### G-039 [proposed] Investigate the orphaned-shell one-core spin on a dead console
Scope: src/modules/punk/repl-999999.0a1.0.tm (console reader/event loop and EOF/error paths), src/modules/punk/console-999999.0a1.0.tm; investigation-first
Detail: goals/G-039-orphan-console-spin.md
Goal: the observed failure mode - an interactive punk902z left running after its hosting terminal/console went away spins roughly a full core indefinitely (observed 2026-07-08: a 37-minute orphan with a single hard-looping thread) - is reliably reproduced and root-caused, then fixed or mitigated so a shell whose console dies exits or reaches zero-CPU idle cleanly.
Acceptance: a documented procedure reproduces the spin on the current kit (e.g. launch an interactive shell in a terminal, then kill/close the hosting terminal or conhost), or the investigation records the attempts made and what evidence would reopen it; the spinning code path is identified (prime suspect: a console read/event loop treating a dead console's immediate EOF/error as retryable without backoff or termination - adjacent to the console-EOF restart path G-038 takes ownership of); after fix/mitigation, the same procedure shows the orphaned process exiting or settling at effectively zero CPU within a short grace period, with live-console interactive behaviour unchanged; the wedge-scoring hazard note (orphans polluting process-liveness checks in test harnesses) is updated to match the outcome.
### G-040 [achieved 2026-07-08] punk::args choice aliasing (-choicealiases) with parse normalization, display folding, and doc-lookup parity
Scope: src/modules/punk/args-999999.0a1.0.tm (parse + usage rendering), src/modules/punk/ns-999999.0a1.0.tm (cmdinfo/cmd_traverse choice resolution parity), src/modules/punk-999999.0a1.0.tm (punk::help topic argdoc as first consumer), src/tests/modules/punk/args/testsuites/, src/tests/modules/punk/ns/testsuites/
Detail: goals/G-040-punkargs-choicealiases.md
Goal: punk::args supports choice aliases (-choicealiases {alias canonical ...}) accepted at parse and normalized to the canonical choice in results, folded into the canonical entry in usage display - and the punk::ns doc-lookup walk resolves choice words by the same rules as the parser (aliases, -choiceprefix, -choiceprefixdenylist, -choiceprefixreservelist) - so alias sets like punk::help's topics|help and console|term|terminal collapse to one displayed entry per topic with `help X` and `i help X` agreeing.
Acceptance: a definition using -choicealiases parses an alias (and an alias prefix where -choiceprefix allows) to its canonical choice in the parse result, with -choicerestricted 0 passthrough and the deny/reserve lists honoured unchanged; usage display shows one entry per canonical choice with aliases folded (no duplicate rows; -choicelabels attach to the canonical); punk::ns::cmdinfo/cmd_traverse resolve subcommand words to docids with the same outcome as the parser for alias, prefix, denied, reserved and unknown words (the pre-goal characterization tests updated from pinned-GAP to fixed); punk::help's topic definition adopts the feature so `i help` lists one entry per registered topic while `help h`/`help e` still fall through to command lookup; definitions without -choicealiases behave unchanged (existing punk::args and punk::ns suites pass).
### G-041 [proposed] punk::args multi-form matching: automated form selection for parsing and documentation
Scope: src/modules/punk/args-999999.0a1.0.tm (parse form selection, arg_error/usage form marking), src/modules/punk/ns-999999.0a1.0.tm (cmdhelp/synopsis closest-form indication), src/tests/modules/punk/args/testsuites/
Detail: goals/G-041-punkargs-form-matching.md
Goal: for multi-form definitions punk::args determines which form(s) an argument list matches - parse without -form attempts all permitted forms instead of effectively form 0, -form accepts the documented list-of-forms restriction, and the documentation surface indicates the match ('i after cancel <id>' presents the cancel form; 's after cancel someid' marks the closest synopsis) - with explicit single-form restriction retained for callers that require it.
Acceptance: parsing a multiform definition (after-like fixture) without -form succeeds when the args match exactly one form (the pinned GAP tests in forms.test flip to auto-selected results); an argument list matching no form (or several) produces an error naming the candidate forms rather than a form-0 type error; -form with a list of form names/indices restricts parsing to that subset (currently an 'Expected int 0-N or one of ...' error); 'i <cmd> <args...>' and synopsis output indicate the best-matching form(s) for supplied args; explicit -form <single-name-or-index> behaviour is unchanged; the full punk::args and punk::ns suites pass.
### G-042 [proposed] Subshell-declared help topics via punk::config with defined shadowing policy
Scope: src/modules/punk-999999.0a1.0.tm (::punk::helptopic registry), src/modules/punk/config-0.1.tm (stored declaration source), src/modules/punk/repl-999999.0a1.0.tm (subshell entry/exit hooks)
Detail: goals/G-042-subshell-help-topics.md
Goal: a named subshell can declare its own help topics (name, aliases, summary, text content) in punk::config stored configuration - registered into the ::punk::helptopic registry on subshell entry and removed/restored on exit - so 'help' inside a customised subshell presents that subshell's topics with 'help topics' and 'i help' staying accurate (the registry already regenerates the punk::args definitions), under a defined precedence policy between topics and command-name fallthrough.
Acceptance: a subshell whose configuration declares at least one custom topic shows it in 'help topics' and renders its configured content via 'help <topic>' inside that subshell, and 'i help' lists it as a documented choice; on quit/switch-away the parent's topic set is restored (no leakage, proven by declaring a topic in a subshell and checking the parent after exit); configuration is read through punk::config (toml per G-014 - no ad-hoc parsing) and a shell with no topic configuration behaves unchanged; config-declared topics are text-content only - they cannot name code to execute (or a recorded design decision to the contrary with its sandboxing rationale in the detail file); the shadowing policy is implemented and documented: a declared topic colliding with a built-in topic is rejected (or explicitly overrides per a documented rule), a topic name shadowing a command name wins over command fallthrough as today with 'i <cmd>'/'s <cmd>' documented as the command-help escape hatch; depends on G-014 for the stored-config substrate - the registry-side registration/restore mechanism may land earlier behind a programmatic API, with the config binding completing when G-014 lands.
### G-043 [proposed] Subshell definition plugins: punk.subshell capability composing commandsets, help topics and config defaults
Scope: src/modules/punk/cap-999999.0a1.0.tm (+ new punk.subshell handler), src/modules/punk/overlay-999999.0a1.0.tm (commandset composition), src/modules/punk/pluginmgr-0.5.1.tm (discovery/trust layer), src/modules/punk/mix-999999.0a1.0.tm (init wiring precedent), src/modules/punk/repl-999999.0a1.0.tm (subshell entry/exit), src/modules/punk/config-0.1.tm (data-declaration overlap with G-014/G-042)
Detail: goals/G-043-subshell-definition-plugins.md
Goal: a named subshell's definition - commandset imports (punk::overlay), help topics (the G-042 registry API) and config defaults - can be supplied by provider packages registered through a punk.subshell capability (punk::cap handler validating declarations) and/or declared in stored config (toml per G-014, naming already-installed commandset packages - absorbing punk::overlay's "toml configuration files for defining CLI configurations" todo), with punk::pluginmgr as the discovery/safe-interp trust layer for providers not already trusted - so entering a subshell composes its command surface and help from declarations instead of hardcoded init code.
Acceptance: a provider package declaring the punk.subshell capability supplies at least a commandset binding (namespace + prefix) and a help-topic set for a named subshell, and entering that subshell composes them (prefixed commands callable, topics in 'help topics'/'i help') with quit/switch-away restoring the parent surface (no leakage); the punk.subshell punk::cap handler validates declarations (malformed declarations vetoed with a useful message); a subshell's commandset composition can equivalently be declared in stored config without a provider package (G-014 substrate), and the existing hardcoded 'dev' CLI composition keeps working unchanged; declarations for capabilities with no registered handler are discoverable (query or report - closing the silent punk.isbogus gap); punk::cap pkg_unregister leaves no stale handler state (the 'destroy api objects?' review resolved); punk::pluginmgr-based discovery/loading of a provider is either demonstrated end-to-end (including .tm module loading in the safe interp) or explicitly deferred with the remaining pluginmgr gaps recorded in the detail file; G-042's registry mechanism is consumed, not duplicated.
### G-044 [proposed] punk::args-driven interactive command completion and hinting in the repl (raw mode primary)
Scope: src/modules/punk/repl-999999.0a1.0.tm (editbuf/reader integration, provider seam), src/modules/punk/args-999999.0a1.0.tm + src/modules/punk/ns-999999.0a1.0.tm (introspection surfaces as consumed), src/modules/punk/console-999999.0a1.0.tm (rendering)
Detail: goals/G-044-repl-command-completion.md
Goal: the interactive repl (raw mode primary) offers command completion and hinting driven by punk::args documentation - the command word resolved through the same doc-lookup flow as 'i' (ensembles, subcommands, ensemble parameters) and the argument position through the definition (options, choices with parse-consistent matching, literals, form awareness per G-041) - via an activation scheme that preserves the editbuf's literal-tab support (not exclusively plain Tab), behind a per-subshell completion-provider seam so alternative-language subshells (e.g. an interactive xtal session) can replace, augment or cleanly disable the Tcl-centric completer.
Acceptance: in a raw-mode interactive session a documented trigger (recorded in the detail file; a literal tab remains enterable into the editbuf) presents completions for command words, ensemble subcommands (resolved via the punk::ns doc-lookup flow without executing candidate commands), option flags and choice values (matching consistent with parse semantics: prefix/deny/reserve, and aliases once G-040 lands); a hint display shows the synopsis of the form(s) matching the partial input (consumes G-041's candidacy API when available - until then all-forms or form-0 display with the limitation noted is acceptable); no per-keystroke terminal queries are added when no completion display is active (G-013 approach note); the completer is a provider interface with the punk::args-driven implementation as the default - a subshell can declare an alternative provider or none, proven at minimum by a subshell with completion disabled showing no Tcl-centric interference (the xtal minimum bar; full xtal completion out of scope but not precluded by the interface); line-mode behaviour is unchanged unless a documented subset is added; G-013's raw editor essentials are a prerequisite and this goal does not weaken G-013's acceptance.
### G-045 [proposed] punk::args definition authoring ergonomics: record continuation, @cmd unindented fields, constructed-definition normalization
Scope: src/modules/punk/args-999999.0a1.0.tm (record parsing in resolve, tstr interplay, arg_error @cmd rendering), src/tests/modules/punk/args/testsuites/ (rendering.test/defquoting.test as the safety net), src/modules/punk-999999.0a1.0.tm (::punk::helptopic::define_docs de-hacked as the consumer proof)
Detail: goals/G-045-punkargs-authoring-ergonomics.md
Goal: authoring punk::args definitions no longer requires backslash line-continuations or ad-hoc workarounds for multi-line records and constructed definitions - a parser-recognised record-continuation mechanism (candidate: an unquoted trailing -& token, with the detail file recording the collision analysis and the element-count disambiguation alternative), -unindentedfields honoured for @cmd fields, and constructed (string-built) definitions able to opt into the same whole-block indent normalization file-style definitions get - with the container quoting rules (braced=literal, quoted=Tcl backslash semantics, \$\{ escape) promoted from defquoting.test into the define documentation.
Acceptance: a definition using the chosen record-continuation mechanism parses identically to its backslash-continuation equivalent (existing definitions unchanged - continuation is additive), with the token's collision rules documented and an escape/rejection story for values that legitimately match it; @cmd -help/-summary honour -unindentedfields (the rendering.test GAP rendering_unindentedfields_cmd_help_GAP flips to aligned); a constructed definition can request whole-block normalization so embedded continuation indentation behaves as in file-style definitions (the rendering_constructed_def_indent_characterization expectations updated to the chosen semantics), and ::punk::helptopic::define_docs drops its manual pre-normalization to prove it; the quoting rules from defquoting.test appear in the punk::args::define -help documentation; the full punk::args suite (128 tests incl. the rendering invariants: nesting independence, relative-indent preservation) passes with GAP tests flipped, none weakened.
### G-046 [achieved 2026-07-10] punk::args deferred -help resolution (parse-time performance + reentrancy) and rendering/value-shape fixes
Scope: src/modules/punk/args-999999.0a1.0.tm (resolve/get_dict: display-field deferral, dynamic-cache subst path, prefix writeback, string renderer, cmdhelp-facing messages), src/modules/punk/ansi-999999.0a1.0.tm (mark_columns argdoc as the reentrancy/perf testbed), src/tests/modules/punk/args/testsuites/ (GAP tests flip; perf verification)
Detail: goals/G-046-punkargs-deferred-help-and-fixes.md
Goal: argument resolution no longer processes -help and other display-only fields - their tstr expansion is deferred to display time (separately cached, per the existing in-source review notes) - so first parse of heavily documented commands gets measurably faster and definitions whose -help calls punk::args-parsing commands (the punk::ansi::mark_columns class) neither loop nor stall; alongside, the mechanical defects pinned by the characterization suites are fixed: @dynamic double-substituted multiline values align at their insertion column, prefix-normalized choice values keep the same shape as exact input, the -return string renderer aligns cmd-help continuations under the first line, and the misleading goodargs parse-error prefix in 'i <cmd> <args>' output is fixed.
Acceptance: parsing/argument resolution provably skips -help expansion (a definition whose -help contains a ${[...]} that would error or record its invocation shows the substitution did NOT run during a parse-only path, only for help display); first parse of punk::ansi::mark_columns drops from ~4s to well under a second with 'i punk::ansi::mark_columns' still rendering the embedded example, and a -help that parses its OWN definition id resolves or errors cleanly rather than looping; first-parse timing improves for at least one other heavily documented command (recorded in the detail file); rendering_atdynamic_multiline_help_insertion_GAP flips to all-aligned; choicegroups_imap_prefix_listwrap_GAP flips to shape-identical (prefix input yields the same plain string as exact input); the -return string renderer's cmd-help continuations align under the first line with relative indents preserved (rendering_string_renderer_characterization updated); the 'Bad number of leading values...' prefix shown by goodargs parsing in 'i string is'-style output is reworded or suppressed for the usage-display path; full punk::args and punk::ns suites pass with no non-GAP expectations weakened.
### G-047 [proposed] Declared primary VCS in punkproject.toml with per-developer commit-target override
Scope: punkproject.toml (schema), punkproject.local.toml (new, uncommitted per-checkout override), root AGENTS.md (Commit Conventions (any VCS) section), .gitignore + .fossil-settings/ignore-glob (ignore rules for the local file, per the coexistence contract), src/project_layouts/ (layout template payload - default values and ignore seeding only; project.new validation is follow-on work)
Detail: goals/G-047-declared-primary-vcs.md
Goal: a `[workflow] vcs = "<system>"` field in punkproject.toml declares the team's primary upstream VCS - the authoritative interchange history and the default target for unqualified "commit"/"checkin" requests - while the same field in an uncommitted per-checkout punkproject.local.toml lets an individual developer redirect their own unqualified commit instructions to their preferred system (mixed git/fossil-preferring teams), resolution order local override > project field > filesystem detection; the declaration governs developer commit workflow only - punk internal machinery (punkcheck tracking of related projects, upstream infrastructure pull per G-027, central project discovery per G-016) remains standardized on fossil regardless of the declared field; punkshell itself carries `[workflow] vcs = "git"` and derived-project layout templates default to fossil.
Acceptance: punkshell's punkproject.toml contains `[workflow] vcs = "git"`, and root AGENTS.md "Commit Conventions (any VCS)" documents the resolution order (punkproject.local.toml field, then punkproject.toml field, then filesystem detection, with the existing prose as final fallback) as the source agents consult for unqualified commit/checkin requests; punkproject.local.toml is ignored by both VCS per the .fossil-settings coexistence contract (git check-ignore matches it, ignore-glob covers it, the contract's verification comparisons stay clean); a reader resolving the primary VCS anchors at the project root via the punk::repo::is_project_root marker and ignores any nested punkproject.toml/punkproject.local.toml `[workflow] vcs`; the fossil-machinery carve-out is recorded in root AGENTS.md alongside the field documentation (a git-primary ecosystem project still maintains its fossil repo for punkcheck/pull/discovery machinery), and the field stays advisory to G-016 discovery with detection as fallback; the mixed-team sync semantics are documented (team primary = authoritative interchange; a developer committing granularly to the secondary owns batching their work up to the primary); project version patch-bumped with a CHANGELOG entry for the schema addition; make.tcl/project.new validation of the field against detected VCS systems stays out of scope (follow-on work).
### G-048 [proposed] textblock::table: parse via punk::args with shared passthrough documentation for table constructor options
Scope: src/modules/textblock-999999.0a1.0.tm (textblock::table proc + PUNKARGS, textblock::class::table class - constructor, opts_table_defaults, methods)
Detail: goals/G-048-textblock-table-punkargs.md
Goal: textblock::table parses its arguments via punk::args::parse (replacing the unvalidated dict merge at L6397), with its PUNKARGS definition covering both table-wrapper-specific options (-return, -rows, -headers) and the constructor passthrough options - the latter sourced by referencing punk::args definition blocks authored inline on the textblock::class::table class methods (the pattern used for render_to_input_line and rendertest in punk::ansi::class::class_ansi: lappend PUNKARGS [list { @id ... }] immediately before the method, punk::args::parse $args withid "..." inside the method body), so the constructor itself parses via punk::args and its documented option set is the single source of truth that textblock::table's PUNKARGS references rather than a parallel hand-typed list - retiring the "more options available - argument definition is incomplete" caveat and closing the -return -choiceprefix documentation/parsing mismatch (item #8 from the punk::args -choices audit) as a side effect.
Acceptance: textblock::class::table's constructor carries an inline punk::args define block (lappend PUNKARGS, @id naming the class+method) and parses its args via punk::args::parse withid, replacing the current manual switch at L466; textblock::table's PUNKARGS references the constructor's documented options (via @id reference, a shared fragment, or a documented include mechanism - the chosen mechanism recorded in the detail file) so table's definition is complete without hand-duplicating the constructor's option list; an invalid -return value (e.g. -return tab) produces a punk::args usage error instead of silently passing through; valid inputs reach textblock::class::table new with behaviour parity to the current dict merge; the -choiceprefix 0 on -return is honoured by construction (punk::args::parse enforces it); the "NOTE: more options available - argument definition is incomplete" comment is removed; existing textblock test suites pass.
### G-049 [achieved 2026-07-10] punk::args parse-status data model with machine-parsable cmdhelp returns
Scope: src/modules/punk/args-999999.0a1.0.tm (arg_error, parse error dispatch, colour-scheme handling), src/modules/punk/ns-999999.0a1.0.tm (cmdhelp), src/tests/modules/punk/args/testsuites/args/usagemarking.test, src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test
Detail: goals/G-049-punkargs-parse-status-model.md
Goal: the information behind usage-display argument marking (which supplied arguments validated, which argument failed and why, which scheme applies) exists as a documented parse-status structure produced from a parse attempt and consumed by both arg_error renderers, and punk::ns::cmdhelp can return it machine-parsably via -return dict - replacing the transient goodargs/badarg locals, the badarg gaps in non-choice validation failures, and the stateful shared colour-array scheme handling.
Acceptance: cmdhelp -return dict distinguishes an incomplete, a fully-valid and an invalid argument set via per-argument statuses (received/ok/bad + overall scheme/message/form) with the structure documented; the table and string renderers derive their marking from that same structure, with rendered output unchanged except where the pinned GAP tests flip: badarg marking covers type/allocation failures not just choice violations (cmdhelp_GAP_no_badarg_marking_for_failed_typed_value), an explicit -scheme is honoured on the parse-failure path (cmdhelp_GAP_explicit_scheme_ignored_on_failure), the failure message names the queried command instead of cmdhelp's internal parse source line (cmdhelp_GAP_errormsg_leaks_internal_source), and scheme rendering no longer depends on or mutates shared colour state - the documented -scheme choice value 'nocolour' takes effect and repeated renders of the same call are identical regardless of prior scheme renders (usagemarking_GAP_scheme_nocolour_renders_with_leftover_colours, usagemarking_GAP_dash_nocolour_leaks_into_shared_array, usagemarking_GAP_dash_nocolour_leak_affects_later_info_render); all non-GAP characterization tests in usagemarking.test and cmdhelp.test pass unchanged.
### G-050 [proposed] punk::ns::synopsis argument-validity marking and status-aware returns
Scope: src/modules/punk/ns-999999.0a1.0.tm (synopsis), src/modules/punk/args-999999.0a1.0.tm (synopsis renderer), src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test (synopsis pins)
Detail: goals/G-050-synopsis-validity-marking.md
Goal: synopsis ('s') marks supplied argument words for validity the way cmdhelp's usage display does (goodarg/badarg styles via the G-049 parse-status structure), offers the same status machine-parsably in its dict return, and replaces the curried-alias excess-args length arithmetic (the 's pse' REVIEW note) with a parse against the resolved definition.
Acceptance: 's <cmd> <valid args>' renders the supplied words marked as validated and 's <cmd> <invalid args>' marks the offending word, while output with no argument words supplied is unchanged (the synopsis_GAP_no_argument_validity_marking pin flips); the dict return carries per-argument status; the curried-alias substitution behaviour (synopsis_curried_alias_shows_braced_target pin) is resolved to a documented presentation derived from parsing rather than list-length arithmetic; existing synopsis output tests (args testsuite synopsis.test and test::punk::args) pass; depends on G-049 for the status structure; G-044's completion/hint display is a consumer of the same status API (cross-reference, not a dependency).
### G-051 [proposed] cmdinfo truthful cmdtype for doc-only pseudo-commands and space-form docid prefix parity
Scope: src/modules/punk/ns-999999.0a1.0.tm (cmdinfo, cmd_traverse), src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test, src/tests/modules/punk/ns/testsuites/ns/cmdflow.test
Detail: goals/G-051-cmdinfo-pseudo-and-prefix.md
Goal: cmdinfo reports a distinct cmdtype (e.g 'doconly') when resolution lands on a punk::args id with no corresponding real command instead of today's 'notfound', and the space-delimited docid jump accepts the same word prefixes the parser accepts - via the shared punk::args::choiceword_match resolver, not a second matching rule - so 'i string is tr' documents what 'string is tr' actually executes.
Acceptance: the pinned GAP tests flip: cmdhelp_GAP_pseudo_command_cmdtype_notfound and cmdhelp_GAP_string_is_true_pseudo report the new cmdtype with the docid unchanged, cmdhelp_GAP_spaceform_docid_prefix_not_honoured and cmdhelp_GAP_string_is_prefix_not_honoured resolve the child docid from a prefix exactly when parse accepts that prefix (honouring -choiceprefix, -choiceprefixdenylist, -choiceprefixreservelist and -choicealiases per G-040 parity); consumers of cmdinfo's cmdtype (cmdhelp, synopsis, eg) handle the new value with no behaviour change for real commands; cmdflow.test and the non-GAP cmdhelp.test tests pass unchanged.
### G-052 [proposed] TclOO method-level autodef documentation
Scope: src/modules/punk/ns-999999.0a1.0.tm (generate_autodef oo branches, cmd_traverse), src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test
Detail: goals/G-052-oo-method-autodef.md
Goal: an undocumented method on a tcl::oo object or class gets an (autodef) punk::args definition generated from its introspected parameter list (info object call + method definition arglists - the machinery generate_autodef already uses for its class summary), so 'i $obj <method> <args...>' shows method-level usage with good/bad argument marking instead of only the class summary with the method word highlighted - explicitly documented methods continue to win.
Acceptance: the cmdhelp_GAP_oo_undocumented_method_class_summary_only pin flips: cmdinfo on an undocumented method resolves a method-level (autodef) docid with the trailing words in args_remaining, and bogus trailing arguments render error-scheme output instead of an info-scheme class summary; documented-method behaviour (cmdhelp_oo_documented_method) is unchanged; instance methods, class-defined methods and mixin/superclass-inherited methods resolve (constructor/'new' signatures at minimum characterized, in-scope or explicitly deferred in the detail file); the "-choiceprefix 0 ... methods must be specified in full always? - review" question on the class-summary method choicelist is resolved and documented either way; cmdflow.test and cmdhelp.test pass.
### G-053 [proposed] punk::args range-valued -multiple: occurrence arity with strict duplicate handling
Scope: src/modules/punk/args-999999.0a1.0.tm (spec compiler, parse, arg_error/synopsis renderers), src/tests/modules/punk/args/testsuites/args/
Detail: goals/G-053-punkargs-multiple-ranges.md
Goal: -multiple accepts a {min max} occurrence range (mirroring -choicemultiple; max -1 unbounded) alongside the legacy booleans - so a definition can declare "at most once, repeat is an error" ({0 1}) or bounded repetition ({2 4}) instead of choosing between silent last-wins (0) and unbounded collection (1) - with boolean semantics preserved exactly, including the prepend-defaults/last-wins override idiom.
Acceptance: parse raises a usage-style arity error naming the argument for occurrences outside a declared range; boolean -multiple 0/1 behaviour is unchanged (full existing punk::args suite passes untouched); the -optional/range-min reconciliation rule is documented and enforced at define time; the usage table Multi column and synopsis reflect declared ranges; -multipleunique/-multipleuniqueset compose with max>1 ranges unchanged; characterization tests cover the new forms and the value-shape rule.
### G-054 [proposed] tclcore moduledoc: runtime-harvested 'string is' class choices with cross-version behavioural parity pins
Scope: src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (+ tclcore-buildversion.txt), src/tests/modules/punk/args/testsuites/args/ (new parity test), TEMP_REFERENCE/tcl9 (read-only reference)
Goal: the ::tcl::string::is definition's class choices (and the generated per-class virtual docids) are harvested from the running interpreter at define time - static choicelabels applied to classes present, generic label for unrecognized classes, a version note for dict - so accept/reject parity with the running Tcl holds on 8.6 and 9.x without hand-maintained per-version lists.
Acceptance: parse/parse_status against ::tcl::string::is and its per-class virtual ids agrees with the real interpreter's error-vs-ok outcome for a pinned probe matrix (missing args, trailing flag-like str word, option/class unique-prefix acceptance and ambiguity rejection, unknown option/class, -failindex var consumption leaving no str, per-version class presence: dict, unicode) on Tcl 9.0.x and 8.6; the rendered choices show only classes the running interp accepts; the parity test derives expectations from the live interpreter (not version arithmetic) and passes under both; existing args/tclcore suites pass; tclcore buildversion bumped with changelog.
### G-055 [proposed] Agent-driven tclcore moduledoc regeneration workflow with behavioural parity verification
Scope: goals/G-055-tclcore-regen-workflow.md (workflow doc), src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (+ buildversion), src/tests/modules/punk/args/testsuites/args/ (parity pins), TEMP_REFERENCE/tcl9 (read-only source input; source retrieval mechanism deferred to the buildsuites toml configs / G-005 era)
Detail: goals/G-055-tclcore-regen-workflow.md
Goal: refreshing punk::args::moduledoc::tclcore is a documented, repeatable agent-driven workflow taking a Tcl source tree + version as input - man-page/source text carried verbatim (rearranged into @cmd -help/@examples/choicelabels, not reimagined; exceptions: synopsis notation is translated into punkshell's own more generic synopsis syntax rather than copied from the source docs, and text may be re-folded/line-wrapped to keep help display-width friendly while punk::args lacks word wrapping), one adaptive package with version-conditional definitions where released behaviour differs (8.6 vs 9.x), each regenerated command verified by a real-vs-model behavioural probe before acceptance, and source-checkin provenance recorded - designed so the same workflow extends later to tkcore (which loads on 'package require tk' and relies heavily on shared documentation sections, e.g the (default)::punk::args::moduledoc::tkcore::tk_standardoptions id) and to moduledocs for other core.tcl-lang.org projects (e.g tcludp).
Acceptance: the workflow is documented in the detail file (inputs, verbatim-text fidelity policy, probe-verification gate, provenance recording, shared-section reuse guidance for the tkcore pattern); the whole ::string ensemble plus a small selection of other commands including the multi-form ::after have been regenerated or verified through the workflow against the reference Tcl 9 sources, with parity pins added and 8.6 released-behaviour differences honoured (G-054's harvest/version-conditional techniques); an initial scan across tclcore commands identifying constructs not adequately modelable in the current punk::args system is recorded in the detail file, with each gap flagged as a candidate goal rather than worked around silently; tkcore and other-project moduledocs remain out of scope beyond the workflow being demonstrably reusable for them.
### G-056 [proposed] punk::args display-time word wrapping for help content
Scope: src/modules/punk/args-999999.0a1.0.tm (arg_error table and string renderers, helpers), src/modules/textblock-999999.0a1.0.tm (only if cell/column-level wrap is the chosen mechanism), src/tests/modules/punk/args/testsuites/args/ (new wrapping characterization + existing rendering suites)
Detail: goals/G-056-punkargs-word-wrapping.md
Goal: punk::args help display (@cmd -help, argument -help, choicelabels) word-wraps over-width lines to the effective display width at render time - ANSI/grapheme aware, splitting long logical lines but never joining existing ones, so deliberately structured content (hand-folded blocks, choice tables, art choicelabels, indented continuations) is preserved by construction - letting definitions store unfolded verbatim text (retiring the G-055 re-folding exception for new work) while usage output stays terminal-width friendly.
Acceptance: a definition whose -help/choicelabel contains a single long unfolded line renders fully within the effective display width in both the table and -return string renderers (no truncation, no overflow, wrapped continuations indented to match the field's existing paramindent alignment); wrap-point calculation is ANSI-aware (SGR sequences measure zero width; styling in effect carries across the wrap) and grapheme/double-width aware to the same standard as existing punk::ansi/textblock width handling; existing hand-folded and structured help renders byte-identical (full existing args rendering/usagemarking/deferredhelp suites pass unchanged - never-join semantics verified by characterization tests); effective width derives from the terminal when available with the current -maxwidth 80 table default as fallback, and an explicit width option overrides; the chosen mechanism (punk::args-side pre-wrap of field text vs textblock table column wrap support) and its rationale are recorded in the detail file; G-055's folding exception is marked lapsed for new work once this ships.
### G-057 [proposed] Windows kit builds embed a configurable icon (twapi resource replacement, per-vfs override)
Scope: src/make.tcl (kit/zipkit wrap steps), src/runtime/punk1.ico (project default, existing), src/vfs/*.vfs (override placement convention), src/runtime/mapvfs.config (only if an explicit config element is the chosen override mechanism), TEMP_REFERENCE/tcl-sfe (read-only reference), helper proc location decided in the work (make.tcl inline vs punk::mix lib)
Detail: goals/G-057-kit-icon-embedding.md
Goal: Windows kit/zipkit builds produce executables carrying an embedded icon chosen at build time - defaulting to the project icon src/runtime/punk1.ico, overridable per kit by its .vfs folder - by replacing the icon resources in the built executable using the twapi-based mechanism demonstrated in tcl-sfe (TEMP_REFERENCE/tcl-sfe, by twapi author and Tcl core member Ashok P. Nadkarni): RT_ICON/RT_GROUP_ICON replacement via twapi resource-update APIs, applied so the appended vfs payload stays intact (icon the stub before appending, or sfe-style split/update/reattach).
Acceptance: a Windows `make.tcl project` build produces kit executables whose embedded icon resources are the project default punk1.ico, and a kit whose .vfs supplies an override icon gets that icon instead (verified by resource inspection, e.g twapi::extract_resources, not just Explorer eyeballing); the icon-replaced executables still boot to a working punk shell reading their vfs payload for the kit types we build (kit, zip, zipcat per mapvfs.config); runtimes under src/runtime are never modified - replacement applies to the built copies only; twapi unavailable or non-Windows platform skips the icon step with a notice and the build otherwise completes unchanged; rebuilds are idempotent (re-wrapping an already-iconed build copy converges, no resource accumulation); the override convention (filename/location in the kit's custom .vfs folder vs a mapvfs.config element) and the stub-vs-split ordering decision are recorded in the detail file with the tcl-sfe attribution.
### G-058 [achieved 2026-07-10] Boot honours statically-linked runtime packages (static baseline seeding + packagepreference static-awareness)
Scope: src/vfs/_config/punk_main.tcl (boot auto_path/tm path filtering), src/modules/punk/packagepreference-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm + src/modules/punk/repl/codethread-999999.0a1.0.tm (code interp / codethread bootstrap), src/modules/shellthread-999999.0a1.0.tm (punkshell-created worker threads) as applicable, src/tests/ (un-gated unit tests + constraint-gated shell/kit integration tests), punkbin artifact repo (separate git repo, local checkout c:/repo/jn/punkbin - pinned runtime additions)
Detail: goals/G-058-static-runtime-packages.md
Goal: punkshell running on a runtime with statically-linked/builtin packages (tclsfe-x64: Thread/twapi/sqlite3/tdbc; punkbin runtimes' builtin Thread/tcllibc/vfs/vlerq; the expected shape of future zig-built static runtimes, G-005) keeps those packages resolvable in every interp and thread punkshell fabricates: boot captures the static baseline (info loaded entries with empty filename, plus their provided versions) before replacing package search paths, code interps and punkshell-created threads are seeded with ifneeded scripts mapping each static package to 'load {} <pkg>', and punk::packagepreference resolves static-vs-bundled by a documented version-aware policy instead of blindly loading a bundled dll over an already-provided static package.
Acceptance: a punk91-style kit (tclsfe-x64 + punk9win.vfs, no thread dll in the vfs) boots to a working repl with no "can't find package Thread" - punk::console loads in the code interp, and package require Thread succeeds there and in a punkshell-created worker thread, resolving to the static version; in the same kit, twapi resolves per the documented policy (no repeat of the observed static-Twapi-masked-by-older-vfs-twapi-5.0.2 double load; a genuinely newer bundled copy remains reachable by that policy); dll-based kits (punk902z) boot and pass their existing shell test baseline unchanged, as does a plain tclsh dev launch; the seeding mechanism is generic - driven by the captured baseline, no runtime-specific package naming - and the boot-time static baseline is introspectable at the repl; the seeding/preference logic is covered by un-gated unit tests against simulated baselines (runnable under plain tclsh), while kit-boot integration tests are gated behind a capability-probed tcltest constraint (a built kit whose baseline shows static entries including Thread) that skips cleanly when no such kit is present; the runtimes used for verification (tclsfe-x64.exe at minimum) are added to the punkbin artifact repository under win32-x86_64 with sha1sums.txt updated, so the constraint is satisfiable on other machines via the existing runtime-retrieval path; the punk91 code-interp vfs/vfs::zip load failure is re-diagnosed after the fix and either resolved or recorded as a distinct issue/candidate goal.
### G-059 [achieved 2026-07-11] WSL detection and suitability probing for driving unix-side tests from Windows
Scope: src/tests/ (capability probe helpers + constraint-gated cases in existing suites, e.g the unix sh-payload execution test in modules/punk/mix/testsuites/scriptwrap/multishell.test and a runtime.bash behaviour test), src/tests/AGENTS.md (enablement notes)
Detail: goals/G-059-wsl-test-driving.md
Goal: test runs on a Windows dev machine can exercise selected unix-side behaviour through WSL when it is present AND suitable - a capability-probed tcltest constraint (not mere wsl.exe existence) verifies a launchable distro, required tools (bash, coreutils/sha1 tooling), and working one-way staging into the guest's NATIVE filesystem - with all guest-side execution happening on that native filesystem: per-test artifacts are staged into a WSL-native tempdir and results collected back, the Windows checkout is never operated on via the shared /mnt path (DrvFs is slow and cross-boundary stat differences make git re-hash its index and fossil see phantom changes), and any future full-suite mode uses a separate native clone rather than the shared working tree.
Acceptance: a documented probe helper yields a wsl_linux_available constraint whose checks are capability-based (distro launches and answers uname/tool probes; staging into a native tempdir works) and which cannot misfire on wsl.exe-present-but-unusable installs (no distro, WSL1 limitations, broken interop); the currently unix-gated multishell sh-payload execution test runs green via WSL on a suitable machine and still skips cleanly elsewhere; at least one runtime.bash behaviour test (active/use/run resolution against a fixture runtime folder) runs inside WSL - all such tests executing from a native-filesystem staging dir with the shared path used only for one-way copy-in/out; the Windows checkout's git and fossil state is untouched by a WSL-gated run (verifiable: git status/fossil changes identical before and after); suite results on a WSL-less machine are unchanged (skips, not failures); enablement/limitations and the staging pattern recorded in src/tests/AGENTS.md.
### G-060 [proposed] QEMU-based cross-platform test matrix (arm's-length integration, GPL-safe posture)
Scope: build/test orchestration config and scripts (location to be settled with the buildsuites toml work - see G-005/buildsuites direction), goals/G-060-qemu-test-matrix.md (workflow + license posture), src/tests/ (any guest-driving hooks)
Detail: goals/G-060-qemu-test-matrix.md
Goal: comprehensive cross-platform verification (linux variants, FreeBSD/NetBSD/OpenBSD, arm architectures) is achievable from a single dev machine by driving QEMU guests as strictly external tooling - punkshell invokes qemu as a separate process with declarative per-guest config, and nothing in punkshell or its artifact repos bundles, links against, derives from or otherwise couples to QEMU (GPLv2) - so the project's BSD licensing posture is unaffected and QEMU remains a swappable convenience: the guest-driving contract is push-based artifact staging and result collection (the same pattern G-059 establishes for WSL - guests never share a working tree with the host) and must be satisfiable by real hardware, WSL or another hypervisor.
Acceptance: a documented, repeatable workflow provisions at least one non-Windows QEMU guest (e.g FreeBSD x86_64) that fetches a punkbin runtime and runs the source-tree suite inside the guest, driven from the Windows dev machine with results collected back; the guest-driving interface is hypervisor-agnostic (documented contract; QEMU is one provider); the license posture is recorded in the detail file (external-process invocation only, no QEMU binaries or derived code committed to punkshell or punkbin, guest OS images not redistributed by the project - fetched/built per machine); suite runs on machines without QEMU are unaffected.
### G-061 [proposed] Pseudoconsole expect-alternative for interactive shell testing (ConPTY + unix pty)
Scope: test-harness support (location TBD during the work: src/tests/testsupport/ and/or a small punk module), src/tests/shell/ (capability-gated interactive suites), goals/G-061-pseudoconsole-expect.md
Detail: goals/G-061-pseudoconsole-expect.md
### G-062 [proposed] Canonical project license: BSD-2-Clause LICENSE file with SPDX-identified references
Scope: LICENSE.txt (new, repo root), README.md, punkproject.toml ([project] license field), AGENTS.md (Repo-wide Notes license mention)
Detail: goals/G-062-project-license-file.md
### G-063 [proposed] Per-package license tracking: SPDX-normalized indications with copyleft audit
Scope: src/modules (Meta license headers), src/vendormodules/ + src/vendorlib/ (vendored license recording), punk::mix module templates (%license% seeding), mapping module (new, name TBD), audit surface (src/make.tcl or dev commandset - TBD)
Detail: goals/G-063-package-license-tracking.md
### G-064 [proposed] lib.search machine-parsable returns (dict/json) and license surfacing option
Scope: src/modules/punk/mix/commandset/loadedlib-999999.0a1.0.tm, src/tests/modules/punk/mix/ (new testsuite), G-063 mapping module (as consumed)
Detail: goals/G-064-libsearch-machine-returns.md

58
README.md

@ -2,8 +2,35 @@
BSD license
2023-08 Note: this is **alpha** level software and still highly experimental.
Version 0.11.0 (2026-07) — this is **alpha** level software and still highly experimental. See `CHANGELOG.md` for recent changes.
### Supported Platforms
- Tcl 8.6+ required; Tcl 9.0 and 9.1 supported.
- Primary target: Windows (win32-x86_64). Linux, macOS, and FreeBSD are secondary targets.
### Getting Started
The project uses a Tcl-based build system (`src/make.tcl`). From the project root:
```
# Fetch a suitable Tcl runtime (cross-platform polyglot wrapper — runs from bash, powershell, or cmd.exe)
./bin/runtime.cmd fetch
# Build modules and libraries into the project root
tclsh src/make.tcl packages
# Full build (modules + libraries + VFS + binaries)
tclsh src/make.tcl project
```
See `src/README.md` for detailed build instructions and `bin/AGENTS.md` for the runtime manager.
For evaluating uncommitted source without a full build, a built executable can run directly against the source tree:
```
<punkexe> src shell
```
### Features
- default ansi color output - toggle with 'colour on' and 'colour off' (or set NO_COLOR environment variable)
@ -70,32 +97,49 @@ BSD license
with colourised indication of type such as proc,alias,ensemble,oo object,oo class,imported,exported where possible.
(renamed aliases and builtins and commands loaded from binaries will appear unmarked)
- `nn/` - move up one namespace towards root namespace '::' analogous to `cd ..` (alias `::/`)
- `n/new <somename>` - create a child namespace called 'somename' and switch to it in one operation. (alias `:/new`)
- `newns <somename>` - create a child namespace called 'somename' and switch to it in one operation. (alias `:/new`)
- cross-platform alternative to cd & ls/dir without invoking child processes. Display colourised listing of dirs and folders - with vfs indication.
- `d/` - list current directory (alias `./`)
also `d/ <globpattern>` to restrict output
- `d/ <subdir>` - switch to subdir and list contents in one operation
- `dd/` - move up one directory and output listing. Roughly equivalent to `cd ..` followed by dir or ls (alias `../`)
- `d/new <folder>` - create a child directory and switch to it in one operation. (alias `./new <folder>`)
- `newdir <folder>` - create a child directory and switch to it in one operation.
- `<punkexe> script` subcommand for reliable non-interactive script execution:
- `punkexe script <file.tcl> ?args?` — runs a script file with conventional `::argv0`/`::argv`
- `commands | punkexe script` — runs piped stdin commands and exits at EOF (no trailing `exit` needed)
- Honest exit codes (0 success, 1 error with errorInfo on stderr); no shellfilter transforms or logging side effects.
- `lib:<name>` resolution for scriptlib scripts: `punkexe script lib:hello` or bare `punkexe lib:hello`
- pluggable console backends: the REPL can run against non-detectable terminal-like devices via `::opunk::Console` subclasses (ssh-channel, tk-widget, test-double), with size, eof, and capability answered by subclass overrides.
- cross-platform runtime manager (`bin/runtime.cmd`): fetch, list, use, and run Tcl runtimes — works from bash, powershell, or cmd.exe. `list -remote` compares local vs server runtimes with sha1 verification.
- Ability to create zipfs wrapped applications with all required libraries built in.
- Additional libraries
- improved fork of tcllib's imap4 (punk::imap4)
- netbox client library (punk::netbox)
- telnet client with support for cp437
e.g can render mapscii.me with mouse support, and can view the ANSI max-headroom movie at 1984.ws
- A feature-rich argument processor that can be used purely to generate function documentation, or
- A feature-rich argument processor (punk::args) that can be used purely to generate function documentation, or
also to process and validate flags and values (punk::args)
The argument processor/documentor can display command synopses, grids of applicable argument choices,
and indications of when short-forms (prefixes for flags/arguments/values) are applicable.
Commands documented with punk::args carry inline usage tables accessible via `i <command>` at the repl.
- In-shell help system: `i <command>` shows documented usage tables; `i help` lists registered help topics
(tcl, env, console, etc.) with per-topic documented usage.
#### missing
- raw mode REPL (read-eval-print-loop) to allow commandline completion etc (undergoing development).
- documentation is incomplete.
- tests coverage is low.
- documentation is incomplete, though the punk::args (PUNKARGS) system now provides inline usage tables and command documentation for many commands via `i <command>` at the repl.
- tests coverage is growing but not yet comprehensive — characterization suites exist for punk::args, punk::ns, punk::lib, punk::libunknown, punk::mix, shellthread, shellfilter, and console, among others.
- signal handling on unix-like platforms (ctrl-c implemented on windows only).
#### very unripe parts:
- commandline options - in need of work to document and lock down specifics - in particular: punkshell somescript.tcl needs a fix to emit errors.
- shellfilter - api is clumsy
- scriptlib - will likely be reorganised/pruned significantly
- theming
### Documentation
- `src/README.md` — detailed build instructions
- `CHANGELOG.md` — version history and recent changes
- `AGENTS.md` (root) + child AGENTS.md files — contributor guidance and project structure (DOX hierarchy)
- `GOALS.md` — technical goal index with per-goal detail files under `goals/`

45
goals/AGENTS.md

@ -2,40 +2,41 @@
## Purpose
Holds optional detail prose for goals in the root `GOALS.md` index that need more than a one-line summary to state properly. The index entry is canonical; a detail file only elaborates and never contradicts its index entry.
Holds the per-goal detail files for the root `GOALS.md` index. Every goal has a detail file: it carries the goal's canonical contract prose (`Goal:` and `Acceptance:`) plus any supporting detail (context, approach, decisions, verification records). The index entry in root `GOALS.md` stays summary-only (ID, status, title, Scope, detail pointer).
## Ownership
- Files here are owned by the goal authoring workflow described in root `GOALS.md`.
- Goals are user-owned: agents add or edit goals only at the user's request or with explicit approval, proposal-first for index-entry changes (see the root `GOALS.md` maintenance rules). Agents must not write goals on their own initiative - suggesting candidate goals for user approval is welcome and encouraged.
- Detail files may be updated without pre-approval when recording findings, decisions, or verification artifacts from user-directed work on the goal or its subject matter; report such updates in the completion summary.
- The root `GOALS.md` index entry is the source of truth. If index and detail disagree, the index wins; fix the detail.
- Goals are user-owned: agents add or edit goals only at the user's request or with explicit approval, proposal-first for any contract element (see the root `GOALS.md` maintenance rules). The contract spans both tiers: the index entry (title, status, Scope) and this folder's `Goal:`/`Acceptance:` lines. Agents must not write goals on their own initiative - suggesting candidate goals for user approval is welcome and encouraged.
- Non-contract content (Context, Approach, Alternatives, Notes, Progress, findings, decisions, verification artifacts) may be updated without pre-approval during user-directed work on the goal or its subject matter; report such updates in the completion summary.
- Canonicality split: a detail file is canonical for its `Goal:` and `Acceptance:`; the root index is canonical for ID, status, title and Scope. The `Status:`/`Scope:` lines here mirror the index - if they disagree, the index wins and the mirror here is fixed.
## Local Contracts
### When a detail file is warranted
### One detail file per goal
A goal earns a detail file when it has any of:
- Non-obvious rationale (rejected alternatives, constraints discovered, why it is not done the obvious way)
- Multi-phase plan with sub-acceptance criteria
- External references or prior art worth citing
- More than ~3 lines of prose to state properly
Simple goals with a clear one-liner and measurable acceptance stay index-only. Do not create empty detail files for completeness.
Every entry in root `GOALS.md` has exactly one detail file here. A minimal detail file is just the header block below - do not pad it with empty sections; add Context/Approach/etc. only when there is real content.
### Naming
`G-<id>-<slug>.md` — e.g. `G-007-bytecompiler.md`. The `<id>` is the stable reference (taken from the `G-<id>` in the root index); the slug is human-readable and may change without breaking links as long as the ID prefix is preserved. Sortable by `ls goals/`.
`G-<id>-<slug>.md` — e.g. `G-013-raw-mode-default.md`. The `<id>` is the stable reference (taken from the `G-<id>` in the root index); the slug is human-readable and may change without breaking links as long as the ID prefix is preserved. Sortable by `ls goals/`.
### Detail file structure
### Detail file structure (suggested)
Required header (the first lines of every detail file):
```
# G-<id> <short title>
Status: <as in index>
Scope: <as in index>
Acceptance: <as in index>
Goal: <canonical - outcome-focused statement of what done looks like>
Acceptance: <canonical measurable, verifiable pass/fail criterion>
```
Optional sections after the header, only when they have real content:
```
## Context
<why this goal exists, what problem it solves>
@ -47,14 +48,20 @@ Acceptance: <as in index>
- <alt B> — deferred, see G-NNN
## Notes
<implementation notes, references, links>
<implementation notes, references, links, findings, verification records>
## Progress
<active goals worked incrementally: what landed (dated), what remains for acceptance>
```
### Progress tracking (active goals)
When user-directed work on an `active` goal lands without satisfying its `Acceptance:`, record it in the detail file's `## Progress` section: what landed (dated) and what remains for acceptance. Progress is non-contract content — updatable without pre-approval, reported in the completion summary. The achieved flip requires the remaining-work list to be resolved (empty, or each item verified satisfied); a partial increment never flips a goal. Increment numbering in commit messages is free-form and carries no contract meaning.
### Archive
- `goals/archive/` holds detail files for goals that have been achieved and moved to `GOALS-archive.md`.
- On archive: move `goals/G-<id>-<slug>.md``goals/archive/G-<id>-<slug>.md`. Do not rename the ID prefix.
- No orphan detail files: every file under `goals/` (excluding `archive/` and this `AGENTS.md`) must correspond to an `active` or `proposed` entry in the root `GOALS.md` index.
- `goals/archive/` holds detail files for achieved goals. Archiving happens as part of the achieved flip (see the root `GOALS.md` maintenance rules): the detail file moves `goals/G-<id>-<slug>.md``goals/archive/G-<id>-<slug>.md` and the index entry moves to `GOALS-archive.md`. Do not rename the ID prefix.
- No orphan detail files: every file here (excluding `archive/` and this `AGENTS.md`) corresponds to a `proposed`, `active`, `abandoned` or `superseded` entry in root `GOALS.md`; every file under `archive/` corresponds to a record in `GOALS-archive.md`.
## Work Guidance

1
goals/G-002-non-nested-subshell.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm
Goal: a subshell can target a named console (default or non-default) and run without blocking the parent, replacing the synchronous nested interp-eval model.
Acceptance: a parent REPL launches a subshell against a named console and continues processing its own input while the subshell runs; the parent can signal/query the running subshell; thread::send -async dispatched from within the subshell's code interp arrives at that interp (so packages like promise work when thread features aren't disabled); the "first subshell asymmetry" TODO at repl-999999.0a1.0.tm:3130 is resolved; existing synchronous `subshell punk`/`safe`/`safebase`/`punksafe` behaviour is preserved as a default mode.
## Context

1
goals/G-003-subshell-resource-limits.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm
Goal: a subshell's code interp can be launched with configurable resource limits (command-count, time) and sandboxing features, building on the resolved first-subshell asymmetry from G-002.
Acceptance: a subshell can be launched with at least one resource limit (command-count via `interp limit -command`, or time via `interp limit -time`) and one sandboxing feature (e.g. `interp hide` of a command, or full safe-interp restrictions) applied to its code interp, enforceable regardless of subshell nesting depth; a subshell can be configured anywhere on the spectrum from unrestricted to fully safe via expose/hide of commands; the existing default subshell behaviour (no limits, no extra sandbox beyond the existing safe/safebase/punksafe types) is preserved when no limits are configured.
## Context

1
goals/G-004-no-committed-binaries.md

@ -2,6 +2,7 @@
Status: proposed
Scope: repo-wide (bin/, src/vfs/, src/vendorlib/, src/vendormodules/, src/bootsupport/)
Goal: the committed repository contains no executable binaries; zip-based .tm modules are allowed but not if they embed executables.
Acceptance: a scan of the committed tree finds no executable binaries (shared libs, .exe, native .so/.dll/.dylib, bare ELF/Mach-O); any zip-based .tm modules present contain no embedded executables; the binary artifacts previously committed are retrievable via G-005 (build from source) or G-006 (pre-built download) so their removal does not break builds.
## Context

1
goals/G-005-zig-build-infrastructure.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/runtime/, build.zig / build.zig.zon (new), src/make.tcl integration
Goal: a zig-based build system retrieves and builds binary dependencies (including Tcl9) from source, replacing the committed-binary approach for the vendored/native components.
Acceptance: running the zig build produces the binary artifacts the repo previously committed (at minimum: Tcl9 library for one target platform); existing Tcl9-zig experiments brought into the project; `tclsh src/make.tcl` integrates with the zig build so a normal project build retrieves/builds binaries via zig when not present; no binary artifacts need to be committed for the build to succeed on a clean checkout with the zig toolchain available.
## Context

1
goals/G-006-prebuilt-artifact-download.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/runtime/, src/make.tcl integration, user-config (consent flags)
Goal: pre-built binary artifacts can be downloaded from a separate related binary-artifacts repository or user-configured sources, gated by explicit user consent/configuration by default.
Acceptance: a download mechanism fetches binary artifacts (the same set the zig build produces) from a configured source on demand; by default the download is gated behind explicit user consent (a config flag or interactive prompt) and does not occur silently; a user-configured source URL overrides the default binary-artifacts repo; downloaded artifacts satisfy the same build requirements as zig-built artifacts so `tclsh src/make.tcl project` succeeds with downloaded artifacts in place of built ones.
## Context

1
goals/G-008-scoped-console-state.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/modules/punk/console-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm
Goal: a subshell on the shared console can opt into scoped console state - its terminal mutations (tabstops, modes, cursor style, palette, title) are captured as an activatable per-subshell state set and the prior state is re-established on quit or switch-away, with today's shared/persistent behaviour remaining the default.
Acceptance: with a scoped subshell: tabstops, at least one DEC/ANSI mode and at least one palette entry changed inside the subshell are restored for the parent on quit (verified by terminal query where queryable) and the facts store matches the terminal; the default (unscoped) launch behaviour is unchanged; activation is set-based, proven by applying state set A, activating set B, then re-activating A and observing A's state; irreversible outputs (cleared screen/scrollback, emitted text) are documented as out of scope.
## Context

1
goals/G-009-themed-subshell-profiles.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/poshinfo-999999.0a1.0.tm, src/modules/punk/console-999999.0a1.0.tm, src/modules/punk/island-999999.0a1.0.tm
Goal: a subshell can be launched from a named profile binding a poshinfo-enumerated scheme to behavioural aspects (e.g. G-003 interp limits/hidden commands, punk::island filesystem access) so that restricted subshells are visually distinct, with the theme's terminal effects and associated profile data riding the G-008 state set.
Acceptance: a named profile associating a poshinfo scheme with at least one restriction aspect (an interp limit, a hidden command, or island-restricted filesystem access) launches by name; the scheme's visual state applies on entry (at minimum prompt styling plus one underlying terminal aspect such as a palette change) and is removed/restored on quit via the G-008 mechanism; profile-associated non-terminal data is scoped to the subshell's lifetime; an unthemed launch is unchanged.
## Context

1
goals/G-010-subshell-tree-navigation.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm
Goal: subshells form a navigable tree - a subshell can be suspended rather than quit, listed, resumed, and the console switched to any live subshell in the tree (e.g. grandchild to grandparent, or across branches) with each subshell's console state re-applied via its G-008 state set, building on the non-nested subshell model of G-002.
Acceptance: from a grandchild subshell a single switch command reaches the grandparent without unwinding through the intermediate parent; switching between subshells on different branches preserves each subshell's session state and re-applies its console state set on activation; suspended subshells can be listed and resumed; `quit` still unwinds to the launching parent as today; a subshell whose switch commands are hidden/restricted cannot initiate switches.
## Context

1
goals/G-011-console-stderr-semantics.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/modules/punk/console-999999.0a1.0.tm, src/modules/opunk/console-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm
Goal: a console optionally carries an err channel as an attribute of its canonical {in out} identity - {in out err} specs accepted everywhere -console is, err resolving to process stderr for the default console and to the console's out channel elsewhere - so diagnostics and emit-to-err are first-class per-console operations instead of raw puts stderr.
Acceptance: console_spec_resolve and every -console site accept an {in out err} spec (err optional; existing pair/instance-name/object spec forms unchanged); opunk::Console exposes the err channel (nullable, additive base-class change); an unset err resolves to stderr for the default console and to the console's own out channel otherwise; punk::console's own warnings/diagnostics emitted while operating on a resolvable console go to that console's err (raw puts stderr remains only where no console is in play); an emit-to-err path exists and is exercised by at least one real consumer (e.g punk::repl); the effective err is discoverable from any thread/interp via console_fact_get (fact key err, returning the effective err channel name); ownership/fact/mode-cache keys remain canonical {in out}; the existing console test suites pass unchanged.
## Context

1
goals/G-012-template-payload-safety.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/project_layouts/, src/make.tcl, src/modules/punk/mix/ (layout instantiation), fauxlink module (bootsupport 0.1.1 - promoted if chosen as mechanism)
Goal: project layouts carry no live nested VCS-config files - template .gitignore payloads are stored inert (renamed, or fauxlink-encoded) and materialized at project generation - and src/make.tcl has an explicit punkcheck-tracked step that refreshes layout payloads from their canonical sources.
Acceptance: a scan of src/project_layouts finds no file named .gitignore, and git check-ignore --no-index over every layout file matches only root-.gitignore rules (nested rules provably inert); a project generated from each affected layout receives a working .gitignore whose content matches the canonical payload/target; after editing a canonical source (e.g. root .gitignore), the make.tcl template-refresh step updates the derived layout payloads (punkcheck-tracked), covering vendor/punk layouts as well as custom/_project; the previously hidden template files (layout READMEs, vendored TODO-class files) remain git-tracked without per-file force-add exceptions.
## Context

3
goals/G-013-raw-mode-default.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/console-999999.0a1.0.tm
Acceptance: as in root GOALS.md index (canonical).
Goal: a repl launched without explicit mode configuration starts in raw input mode with a clean display - every per-keystroke debug emission behind its own repl-operable toggle - and raw editing covers the line-mode essentials (history navigation, cursor movement).
Acceptance: a default launch lands in raw mode with line mode still selectable; with debug toggles at their defaults (off unless stored configuration via ::punk::config says otherwise - see G-014), typing/editing/submitting a command emits no cursor-positioned debug output; the per-keystroke add_chunk frame and the right-hand live editbuf view are gated separately, each toggleable from within a running repl; arrow-key history navigation and left/right cursor movement work in raw mode (current stubs replaced); the marked-line debugrepl output form is retained and works on terminals without cursor addressing (e.g. vt52); the debugrepl first-word activation mechanism is reviewed and the keep/replace outcome (e.g. a proper Tcl command that interp hide can restrict in subshells) is recorded in the detail file; on tcl 8.6 a background-initiated terminal query at an idle raw-mode prompt succeeds (the residue scenario fail-fast-guarded in punk::console 0.7.1).
## Context

3
goals/G-014-punk-config-toml.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/modules/punk/config-0.1.tm, src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm, src/vendormodules/tomlish-*.tm (vendored; canonical source in the external tomlish project space)
Acceptance: as in root GOALS.md index (canonical).
Goal: ::punk::config loads stored configuration from toml files in the XDG-located config dir - parsed via the vendored tomlish module, never an ad-hoc parser - and consumers resolve settings with per-named-subshell overrides, so features like the G-013 debug-view startup defaults read declared user configuration instead of hardcoded fallbacks.
Acceptance: a setting declared in a toml file under the XDG-located config dir is visible through the punk::config API at repl startup, and with no config files present built-in defaults apply with no errors beyond the existing missing-dir notice; a named subshell resolves its own overriding value for a key also defined at the parent/default scope, and a subshell with no override inherits the outer value (proven with at least one real key); at least one shipped feature (the G-013 editbuf-view startup default is the natural first) reads its default through this path rather than a hardcoded value; all toml reading/writing in punk::config goes through the tomlish module, and the tomlish API procs punk::config consumes carry punk::args (PUNKARGS) documentation - added upstream in the tomlish project and re-vendored here before punk::config implementation proceeds.
## Context

33
goals/G-016-projects-work-git-discovery.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/modules/punk/mix/commandset/project-999999.0a1.0.tm, src/modules/punk/repo-999999.0a1.0.tm
Acceptance: as in root GOALS.md index (canonical).
Goal: `dev projects.work <glob>` lists git-based project checkouts as well as fossil-based ones, each result identifying its VCS - fossil discovery stays central-config-db based, and git discovery uses a defined enumeration source (git has no central registry; the chosen mechanism is recorded in the detail file); output gains a `-return table|dict|json` option (default table, unchanged): dict is the canonical machine form (per-checkout records incl the VCS field, documented in the argdoc), json a faithful rendering of it for out-of-process consumers.
Acceptance: with a git-only project on disk registered in the chosen enumeration source and matching the glob, `projects.work` lists its working directory and identifies it as git; existing fossil results are unchanged apart from any added VCS-identifying column; a project that is both git and fossil (e.g. this repo) appears with both indicated rather than duplicated; glob matching remains case-insensitive; `-return dict` yields the documented record structure containing everything the table shows (incl VCS identification), `-return json` round-trips the same data, and the default table output is unchanged for existing users.
## Context
@ -50,8 +51,38 @@ Result-shape considerations:
- Relying on external tools' state (e.g. IDE/zoxide/gh caches) - rejected:
non-portable, not present on all machines, opaque formats.
## Machine-parsable returns (added to contract 2026-07-11)
- `-return table|dict|json` (default table, existing display unchanged), following the
house convention (cmdhelp `-return dict`, parse_status - G-049): the dict is the
canonical machine form and the single source the other renderings derive from -
build it FIRST, render the table from it (avoids maintaining parallel assembly).
- Suggested dict shape: list of per-checkout records - workdir, projectname,
projectcode, vcs (fossil|git|both - this goal's addition), repo db/enumeration
source path, multi-checkout dup info, optional -detail file-state - plus overall
keys (globs, counts). Document the structure in the PUNKARGS argdoc like
::punk::args::parse_status does.
- json = faithful rendering of the dict (tcllib json::write is available in kits via
tcllibc) - the parse target G-017's agent guidance standardizes on: self-delimiting
and language-neutral over a pipe.
- Markdown deliberately NOT offered as a machine form (2026-07-11 decision): agents
parsing should use json; the human table remains for eyeballing - a markdown
rendering would be a third sync burden, lossier than json.
## Notes
- Staleness of the fossil config-db source itself (observed 2026-07-11): a
throwaway test repo fossil-init/opened in an agent session scratchpad (temp
path) persisted for days as row 1 of the listing - `repo:` rows live until an
explicit `fossil all ignore`, and `ckout:` rows are pruned only by `fossil all`
commands run after the checkout dir is deleted. So the "registry can go stale"
concern listed against the punk-maintained-registry option applies equally to
fossil's own central config-db. Whatever enumeration design this goal ships
needs a staleness story for BOTH VCS sources - e.g. flag or filter rows whose
checkout dirs are missing or under known temp locations - especially for the
`-return dict|json` machine forms agents will consume unfiltered. The
prevention side (scratch `FOSSIL_HOME` for throwaway repos) is recorded in
root AGENTS.md User Preferences.
- Depends on nothing, but its value to agents is realised through G-015
(reliable piped invocation) and G-017 (agent guidance documenting the call).
- The `-cd` / `-detail` options and case-insensitive glob behaviour of the

6
goals/G-017-agent-project-discovery.md

@ -0,0 +1,6 @@
# G-017 Agents locate local projects via piped `projects.work` calls, not filesystem scanning
Status: proposed
Scope: AGENTS.md (root) or a child doc it indexes (guidance content only - no code)
Goal: once G-015 makes piped script calls reliable, repository guidance directs agents asked to locate another local project to query it via a piped `projects.work` call to a punk executable instead of grepping/globbing the wider filesystem.
Acceptance: root AGENTS.md (or a child doc indexed from it) records the exact recommended invocation - executable, subcommand, glob usage, and `-return json` as the parse target with its record fields - and states when filesystem scanning remains appropriate (projects not registered in any discovery source); the guidance is added only after G-015 is achieved (and notes the fossil-only limitation until G-016); following the documented pattern, an agent locates a named sibling project's checkout dir with a single piped call.

3
goals/G-018-zig-plain-tclsh-kits.md

@ -2,7 +2,8 @@
Status: proposed
Scope: build.zig / build.zig.zon (per G-005), src/runtime/, src/make.tcl integration
Acceptance: as in root GOALS.md index (canonical).
Goal: developers can use the G-005 zig build system to produce self-contained zip-based tclsh executables that carry no punk-specific infrastructure (no punk boot layer, punk modules, or punk apps) - plain tclsh kits usable independently of the punkshell product.
Acceptance: a documented zig invocation on a clean checkout (zig toolchain available) produces a zip-based tclsh executable for at least one target platform that runs conventional tclsh invocations (`<exe> script.tcl args`, piped stdin) on a machine with no Tcl installation; a listing of the kit's mounted/zip contents shows stock Tcl (plus any declared stock runtime deps) and no punk namespaces, punk boot files, or punkshell apps; the punk-flavoured executables remain producible alongside.
## Context

3
goals/G-019-dependency-scan-module-trimming.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/make.tcl, src/modules/punk/lib-999999.0a1.0.tm (tclparser use), src/vfs/ (kit assembly), scanning module (new or existing punk module - to be determined)
Acceptance: as in root GOALS.md index (canonical).
Goal: a package-dependency scan from an executable's entrypoint (candidate basis: the tclparser parse API - currently satisfied only by the c-only tclparser library, with punk::lib's pure-Tcl fallback an unimplemented stub) determines the module closure the executable actually requires, so a build can ship only those modules - while 'batteries included' builds remain a supported alternative, not a casualty.
Acceptance: for at least one punk-based executable target, the build can run a dependency scan from its entrypoint producing the closure of required packages/modules plus a mechanism to declare dynamically-loaded extras the scan cannot see; a trimmed kit assembled from that closure starts and passes its basic function check (e.g. repl launch or the app's smoke test) with no missing-package errors; the trimmed kit's module listing is a strict subset of the batteries-included equivalent (demonstrating real exclusion); batteries-included builds remain producible unchanged.
## Context

16
goals/G-020-screencap-input-module.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/modules/punk/ (new module - name TBD), src/vfs/punk9win.vfs/lib_tcl9/ (existing treectrl/Img/twapi payloads); scriptlib/aloupe.tcl stays untouched as a standalone app
Acceptance: as in root GOALS.md index (canonical).
Goal: a punk module drives screen/window capture and mouse/keyboard injection from scripts via per-platform backends - Windows (treectrl loupe capture + twapi input/window-location) is the initial complete target, with the backend contract designed so X11 (Linux/FreeBSD) and macOS backends can be added without changing callers, and Wayland-native sessions explicitly out of scope.
Acceptance: on Windows from a punk shell or script: a screen region and a window located by title/class pattern are each captured to a Tk photo and written as a valid PNG; mouse movement/click and key events injected into a located test window produce their observable effect (typed text arrives, click acts); window location returns the handle and geometry for a pattern; a capability-introspection call reports per-feature support and an unsupported platform/backend yields a clean capability-based refusal, not a crash; the backend interface (capture / input / window-locate) is documented well enough that a non-Windows backend can be added without modifying callers; the aloupe script remains functional and unmodified.
## Context
@ -91,3 +92,16 @@ the module is a separate development, not a refactor of it.
- Related: G-018 (kit composition; Tk as loadable package, no wish binaries),
G-019 (a trimmed capture-capable executable is a plausible scan target),
G-021 (agent-facing surface over this module), G-001 (backend plugin pattern).
## Additional driving use case: repl interactive-behaviour verification (2026-07-11)
Characterizing punkshell's interactive repl behaviour (raw-mode colour staging, tab
markers and dim space dots in the editbuf, closing-prompt hints, multiline history
navigation - see the preserve-list in goals/G-044 detail) currently has no automated
harness: the underlying editbuf is console-coupled and there is no expect-like system.
This module is the practical near-term bridge on windows: inject keystrokes at a live
punkshell window and capture/compare the rendered region. Coarser than a pseudoconsole
expect-alternative (the durable successor - candidate goal drafted 2026-07-11) but
available sooner, and it can verify exactly the rendered-behaviour tier that unit tests
cannot reach. Worth weighting this goal's priority accordingly when sequencing repl
refactor work.

3
goals/G-021-agent-visual-verification.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/modules/punk/ (G-020 module's agent-facing surface), AGENTS.md guidance (post G-015 pattern), src/tests/ (visual-verification test hooks)
Acceptance: as in root GOALS.md index (canonical).
Goal: a tool-calling agent can, during a session, use piped script calls (G-015) to a punk executable to locate the applicable UI window, snapshot it to a PNG file and/or base64 output suitable for AI image analysis, and drive mouse/keyboard interactions - enabling tests whose verification is visual-only and/or input-driven.
Acceptance: on Windows, single piped script calls (no interactive session) can: list/match windows for a pattern with machine-parseable output; save a located window's snapshot to a caller-specified path and optionally emit it base64 on stdout; run a scripted interaction sequence (focus, click at offset, type text, snapshot) end-to-end; failures exit nonzero with the error on stderr per G-015 semantics; the invocation patterns are documented for agents alongside the G-017 guidance; at least one real visual-or-input-driven verification (e.g. a Tk app smoke test) is exercised through this path.
## Context

3
goals/G-022-fossil-rename-punkshell.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/modules/punk/mix/commandset/repo-999999.0a1.0.tm, src/modules/punk/repo-999999.0a1.0.tm, src/tests/modules/punk/mix/testsuites/repo/
Acceptance: as in root GOALS.md index (canonical).
Goal: the `dev repo` commandset can move and rename fossil repositories non-interactively and safely - all checkouts repointed, no phantom central config-db entries, no dangling old repo db, fossil project-name renamable with project-code unchanged - and this project's fossil repo (currently project-name 'shellspy') is renamed to 'punkshell' through that mechanism via a G-015 piped script call, not by hand.
Acceptance: the commandset provides a flag-driven (no stdin prompts) move/rename operation which on a scratch repo with an open checkout: repoints every registered checkout, leaves the central config-db listing only the new path, removes or archives the old repo db file (per option), clears stale ckout: back-references, and applies a requested project-name change while preserving project-code; the GAP characterization tests in src/tests/modules/punk/mix/testsuites/repo/fossilmove.test are updated to assert the clean behaviour and pass; after G-015 is achieved, this repo's fossil db (shellspy.fossil / project-name shellspy) is renamed to punkshell via the new operation invoked through a piped `script` call, with `fossil info` in this checkout showing the new repository path and project-name and `fossil all ls` free of the old path.
## Context

3
goals/G-023-version-named-binaries.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/make.tcl, src/runtime/ (mapping config - see G-024), bin/ (build outputs)
Acceptance: as in root GOALS.md index (canonical).
Goal: project builds produce version-named punk executables for tcl 8.6 and tcl 9 as the project version advances - punk8-<major>-<minor>-<patch>.exe / punk9-<major>-<minor>-<patch>.exe per version, punk8-dev.exe / punk9-dev.exe tracking the latest build, and plain punk8.exe / punk9.exe created initially then replaced only when an actual release is tagged - tolerating the disk growth for now.
Acceptance: a project build at the current punkproject.toml version produces punk8-<M>-<m>-<p>.exe and punk9-<M>-<m>-<p>.exe (names derived from the version, not hand-maintained) plus punk8-dev.exe / punk9-dev.exe updated to that same build; rebuilding at an unchanged version refreshes that version's binaries and -dev without touching other versions' outputs; plain punk8.exe / punk9.exe exist and are replaced only by an explicit release step - a normal build never overwrites them; the scheme is declared succinctly via the G-024 toml mapping (no per-version config edits); archival/deletion of accumulated versioned binaries is out of scope with the trigger question recorded in the detail file.
## Context

3
goals/G-024-mapvfs-toml.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/runtime/mapvfs.config (replaced/deprecated), src/make.tcl (parsing), src/bootsupport/modules/tomlish-*.tm (parser dependency)
Acceptance: as in root GOALS.md index (canonical).
Goal: the runtime-to-vfs-to-executable build mapping moves from the custom line format of src/runtime/mapvfs.config to a toml file parsed with the tomlish package - still supporting explicit per-executable mappings (runtime, vfs folder, output name, kit type) while also expressing generative schemes like the G-023 versioned naming in a single succinct declaration.
Acceptance: a mapvfs toml file parsed via tomlish (no ad-hoc toml parsing) drives the build: every mapping currently active in mapvfs.config is expressible and at least one existing target builds identically from the toml; the G-023 versioned/dev/release-gated output scheme is declared in one entry that expands to its outputs without enumerating versions; malformed or unresolvable entries fail the build with a clear message naming the entry; the legacy .config format is either fully migrated (old parser removed) or explicitly deprecated with documented precedence between the two files.
## Context

3
goals/G-025-exe-selfreport.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/vfs/_config/punk_main.tcl (subcommand dispatch), src/make.tcl (stamping build info into the vfs), src/vfs/ (stamp payload location), src/modules/punk/ (in-shell command - the single implementation)
Acceptance: as in root GOALS.md index (canonical).
Goal: a punk executable reports its identity from embedded data rather than its filename - a documented subcommand prints the punkproject.toml project version it was built from plus the input runtime binary name and vfs folder name used to assemble it - with the same-named command available in the punk module so scripts running in any punk shell (including tclsh-hosted ones like `tclsh src/make.tcl shell`) get the same report in-process without exec, stamp fields reported as absent rather than fabricated when there is no stamp.
Acceptance: the build stamps project version, runtime binary name, and vfs folder name into the kit; the built executable invoked with the version-report subcommand prints those fields machine-parseably on stdout and exits 0 with no other output (G-015-compatible; no repl fallthrough); a same-named command in the punk module returns the same fields in-process (subcommand implemented as a wrapper over it - one implementation) and works from the code interp; the report distinguishes stamped provenance from live facts: a stamped kit reports its stamp, a `src`-mode or source-tree session additionally reports the live punkproject.toml version as a distinct field when it differs, and unstamped contexts (`tclsh src/make.tcl shell`, plain tclsh with punk modules) report stamp fields explicitly absent with live runtime facts (actual `info nameofexecutable`) still provided; the report is correct when the executable file has been renamed or copied; executables built before stamping existed fail gracefully with a clear message rather than fabricating values.
## Context

3
goals/G-026-vendor-provenance-policy.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/make.tcl (vendorupdate and bootsupport steps), src/vendormodules/include_modules.config, src/bootsupport/modules*/include_modules.config
Acceptance: as in root GOALS.md index (canonical).
Goal: pulling vendored or bootsupport artifacts from local source projects enforces committed provenance - the warn-only dirty-checkout check added to vendorupdate in project 0.2.5 becomes a policy that can abort with an explicit override, covers the bootsupport update path as well, and the residual staleness question (built modules that predate or postdate the committed source even in a clean checkout) has a recorded design decision.
Acceptance: vendorupdate and the bootsupport update refuse to pull from a source project whose fossil/git checkout is dirty unless an explicit documented override is given (warn-only selectable as a configured mode); the check reports each VCS root once per run and does not fire for unversioned source locations; bootsupport_localupdate is covered by the same shared check (no second divergent implementation); the staleness gap - a clean checkout whose built modules/ artifacts do not correspond to the committed source - is either detected (mechanism chosen and implemented) or explicitly recorded in the detail file as accepted risk with the considered mechanisms; behaviour is exercised by a test or documented manual verification against a scratch dirty checkout.
## Context

3
goals/G-027-derived-project-pull-updates.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/modules/punk/mix/commandset/project-999999.0a1.0.tm (project.new push path), src/modules/punkcheck-999999.0a1.0.tm (install provenance records), src/make.tcl (derived-project pull entrypoint)
Acceptance: as in root GOALS.md index (canonical).
Goal: a project generated from a punkshell layout can pull infrastructure updates (make.tcl/build.tcl, bootsupport modules and libs, layout template payloads) from its originating punkshell project by running one command inside the derived project - replacing the current push model (`dev project.new -force 1 -update 1` run from punkshell) - with install provenance robust to derived-project workdir moves (not local relative paths alone), VCS-state awareness on both ends, and a recorded decision on pulling from remote sources.
Acceptance: one documented command run inside a derived project updates its punkshell-derived infrastructure from the source punkshell project; the update still works after the derived project's working directory has been moved (proven by moving a scratch derived project and pulling); VCS integration on both ends: the pull applies the G-026 clean-checkout policy to the punkshell source, and reports (or refuses per option) when target files it would overwrite carry uncommitted local modifications in the derived project's git/fossil checkout; .punkcheck records remain the provenance basis (updates are recorded and unchanged targets skipped, as with existing punkcheck-tracked installs); the push flow keeps working until explicitly retired; the remote-pull question (updating from a remote punkshell repository rather than a local checkout) has a recorded design decision - implemented, or deferred with rationale in the detail file.
## Context

3
goals/G-028-file-locker-identification.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/modules/punkboot/utils-999999.0a1.0.tm (locker-report helper), src/make.tcl (kit deploy failure reporting)
Acceptance: as in root GOALS.md index (canonical).
Goal: when a build cannot delete/overwrite a target file (typically a built executable held open by another program), the failure message names the locking process(es) - via a punkboot::utils helper using the Windows Restart Manager API through optionally-available twapi or cffi, degrading cleanly to the current message when the API or bindings are unavailable or on other platforms.
Acceptance: on Windows with twapi or cffi loadable, a punkboot::utils proc given a file path returns the locking processes (at least pid and process name; empty list when unlocked); punkboot::utils itself stays pure Tcl - the binary binding is required lazily at call time and its absence yields a clean 'unavailable' result, not an error (bootsupport must not gain a compiled-extension dependency); make.tcl kit deploy failures include the locker report when determinable (e.g. "could not delete target binary ... in use by: 7zFM.exe (pid 1234)"); non-Windows platforms and binding-less environments produce the existing message unchanged; verified against a deliberately held handle (documented manual verification acceptable).
## Context

3
goals/G-029-testmodules-from-srctests.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/make.tcl (test-module packaging step), src/tests/ (source of truth), src/modules/test/ (generated #modpod targets), src/modules/punk/mix/ (modpod tooling as needed)
Acceptance: as in root GOALS.md index (canonical).
Goal: src/tests is the single source of truth for module test suites - a punkcheck-tracked make.tcl step generates the packaged test::<modulename> #modpod modules from src/tests/modules/<namespacepath>/testsuites content, ending hand-maintenance of parallel copies under src/modules/test/; the packaged form is a distributable in its own right - a user who downloads a built module can optionally download the matching test::<modulename> and verify the module's behaviour on their own system (package require + RUN, or an executable's -app test) with no source tree or test harness required.
Acceptance: a make.tcl step generates a test::<modulename> #modpod under src/modules/test/ from the corresponding src/tests testsuites (punkcheck-tracked, skipped when sources unchanged); the generated package works through the packaged path - loadable via package require test::<modulename> and runnable via its SUITE/RUN interface (e.g a built executable's -app test) - reporting the same test names and pass counts as running the same suites directly via src/tests/runtests.tcl; the consumer scenario is proven standalone: the generated package (plus the module under test and their dependencies) runs in an environment without the project source tree present; suite data files (e.g roundtrip toml files with deliberate crlf/mixed line endings) survive packaging byte-for-byte; documentation states src/tests is the source of truth and the generated modpods are build artifacts not to be hand-edited; proven end-to-end for at least one real module (tomlish, whose src/tests port and still-live modpod created the dual-copy situation, is the natural first).
## Context

3
goals/G-030-maketcl-punkargs.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/make.tcl (dispatch, help, prompts), src/bootsupport/AGENTS.md + src/modules/AGENTS.md (bootstrap-tracked staleness contract), src/modules/punk/args-999999.0a1.0.tm (only as consumed)
Acceptance: as in root GOALS.md index (canonical).
Goal: make.tcl - the first surface a developer sees - parses its subcommands and options via punk::args and showcases the tabled usage output for help and argument errors, every interactive y/n prompt gains a declared flag equivalent so agents can drive make.tcl with arguments instead of piped input, punk::args joins the bootstrap-tracked staleness set, and the boot phase plus the environment-repair commands keep working with degraded plain help when the bootsupport punk::args (or the table-rendering stack) is stale or unavailable.
Acceptance: `tclsh src/make.tcl` and `-help` render punk::args tabled usage listing every subcommand with a summary, and `make.tcl help <subcommand>` (or `<subcommand> -help`) shows that subcommand's definition; invalid arguments produce a punk::args usage error rather than ad-hoc messages; every y/n prompt has a documented flag equivalent (proven at least for vfscommonupdate and the project-build confirmations: a run with the flag completes non-interactively with stdin closed) and a non-interactive stdin without the flag fails fast with usage rather than hanging or half-aborting; punk::args is added to the bootstrap-tracked buildversion set with the doc contract updated (src/bootsupport/AGENTS.md, src/modules/AGENTS.md); with bootsupport punk::args unavailable or unloadable, make.tcl still boots and `check`, `bootsupport` and `modules` remain usable with plain-text fallback help (the guarded-require degrade rule); layout make.tcl copies follow via the established sync/G-027 channels (noted, not hand-synced).
## Context

3
goals/G-031-componentized-kit-boot.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/vfs/_config/ (punk_main.tcl, project_main.tcl restructure), src/vfs/_vfscommon.vfs (boot core delivery), src/project_layouts/ (thin-main skeleton, via established sync channels)
Acceptance: as in root GOALS.md index (canonical).
Goal: the per-project vfs main script becomes a thin project-owned file - declare the application's subcommands and launch defaults at clearly commented customization points, then hand over to a shared layout-owned boot core (vfs mounts, package modes and paths, libunknown, src-mode modpod registration) and default dispatch pulled in from within the kit - so project developers add app-specific subcommands without wading through or forking ~1000 lines of boot boilerplate, and boot improvements reach derived projects as pull-updatable payload instead of dying in vintage forks (tomlish_main.tcl: ~20 custom lines carrying a stale 500-line 2025 copy of the rest).
Acceptance: punkshell's own kits boot through a thin main plus shared boot core with behaviour parity - package modes including src mode, existing tclsh/shellspy/punk/shell/script dispatch semantics, and supported vfs types (zipfs/metakit/cookfs) all unchanged; a project-specific subcommand is added by editing only the thin main at a commented customization point (proven end-to-end in a derived project - tomlish replacing its forked main is the natural first); the boot core ships as layout-owned payload (via _vfscommon/layout channels) and the thin main as a project-owned skeleton, per the G-027 ownership classification; the boot core is versioned/identifiable so a kit can report which boot-core vintage it carries (ties to G-025 stamping).
## Context

3
goals/G-032-launcher-punkargs.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/vfs/_config/ (default dispatch), src/lib/app-punkshell and sibling app packages as touched
Acceptance: as in root GOALS.md index (canonical).
Goal: the default launch dispatch defines its subcommands via punk::args - `<punkexe> -help` and argument errors render the tabled usage enumerating built-in and project-registered subcommands with summaries, and subcommand options parse through punk::args so projects can declare complex arguments - with the G-030 degradation rules (boot never fails and help degrades to plain text when punk::args or the ANSI rendering stack is unavailable).
Acceptance: `<punkexe> -help` renders tabled usage listing all subcommands including project-registered ones, each with a summary; at least one built-in subcommand's options are declared and parsed via punk::args with tabled usage errors on invalid input (the G-015 script subcommand or the G-025 version-report subcommand are the natural candidates); a project-registered subcommand's help appears by registration alone - no edits to the shared dispatch; with punk::args or the rendering stack unloadable, boot proceeds and help degrades to a plain subcommand list; verified on both a zipfs-based and a non-zipfs kit where both remain supported.
## Context

3
goals/G-033-proj-mode-cwd-project.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/vfs/_config/punk_main.tcl (package-mode dispatch and boot-time root discovery), src/modules/punk/repo-999999.0a1.0.tm (find_project / is_project_root - reuse or lean boot mirror), bin/AGENTS.md (mode docs)
Acceptance: as in root GOALS.md index (canonical).
Goal: a `proj:` prefix on the package-mode string (e.g. `punkshell proj:internal-src shell`) makes the dev/src path blocks resolve against the punk project containing the current working directory - walking up to the nearest VCS repo root, the marker punk::repo::is_project_root already uses - instead of the executable's own project, so an installed ("visitor") punkshell binary, including one downloaded standalone with no source tree around it, can interactively explore a project that builds no shell-capable binary or no binary at all; the prefix scopes WHICH project, staying outside the ordered dash-list whose block order remains the same-version tie-break dial, and the explicit prefix gates the behaviour so no `cd` silently rebinds a normal launch.
Acceptance: `<installed-punkexe> proj:internal-src shell` (the documented canonical visitor invocation - kit copies win same-version ties, protecting the visiting shell's infrastructure; `proj:src` is the documented faithful-vintage variant where the project's copies win) launched from within a punk project's tree discovers the project root by walking up from cwd to the nearest VCS repo root and feeds it to the existing src-mode path machinery (src/modules, src/bootsupport/modules, src/vendormodules on the module path, src/lib on auto_path), so the session loads the project's dev-versioned modules while ordinary version resolution still lets the project supply anything the kit lacks or exceeds; the launch reports the detected project root and effective path precedence (never a silent rebind); with no project found walking up from cwd, or a `proj:` string containing no root-using block (dev/src), it warns and proceeds without false rebind; proven for a standalone binary outside any source tree (internal = kit contents regardless of binary location) against a project that builds no executable (tomlish is the natural first); exe-relative `src`/`dev` discovery is unchanged for a binary that IS in a project's bin/; the packagemode help text drafted in the detail file becomes the live punk::args documentation when implemented (rendered via G-032).
## Context

6
goals/G-034-modpod-codeinterp-tcl86.md

@ -0,0 +1,6 @@
# G-034 Zip-based #modpod modules mount in the shell code-interp on Tcl 8.6
Status: proposed
Scope: src/modules/punk/mix/ (modpod mount path), vfs::zip availability in the repl code interp, src/modules/punk/cap/ (templates capability handler)
Goal: zip-based `#modpod` modules (e.g. `punk::mix::templates`) mount and their punk::cap handlers register in the `shell` subcommand's code interp on Tcl 8.6, matching the `script`/main-interp context - so `dev module.templates` and other `punk.templates`-capability consumers work in an interactive 8.6 shell instead of failing with `invalid command name vfs::RegisterMount`.
Acceptance: `dev module.templates` in an interactive 8.6 punk shell (`shell` subcommand) lists the template providers with no `failed to load ZIP archive-based module` / `invalid command name vfs::RegisterMount` / `Unable to register any template providers` / `invalid command name ::punk::cap::handlers::templates::api_punk.templates` errors, matching the `script`-subcommand output on the same binary (verified 2026-07-07: script works, shell fails); root cause fixed (the code interp lacks the vfs::zip library that provides vfs::RegisterMount for pre-zipfs Tcl, present in the main interp - restore it to the code interp or use an alternative modpod mount path there); Tcl 9 (built-in zipfs) behaviour is unchanged.

3
goals/G-035-mixed-tm-pkgindex-provision.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/tests/modules/punk/libunknown/testsuites/ (characterization suite), src/modules/punk/libunknown-0.1.tm and src/modules/punk/packagepreference-999999.0a1.0.tm (as characterised, fixed only if outright bugs surface), src/modules/AGENTS.md + src/lib/AGENTS.md (resulting guidance)
Acceptance: as in root GOALS.md index (canonical).
Goal: the behaviour when the same package is provided both as a .tm module and as a pkgIndex.tcl-based library - same or differing versions, under the standard package unknown, punk::libunknown and punk::packagepreference - is characterised by committed tests, and the currently informal working rule ("avoid mixing provision forms for one package - unexpected behaviour even with libunknown's improvements") is either substantiated with the specific failure modes named in AGENTS.md guidance, or retired if the characterisation shows the machinery now handles mixing predictably.
Acceptance: a committed test suite (extending src/tests/modules/punk/libunknown/testsuites/) characterises at least: same name+version provided via .tm and via pkgIndex.tcl (which registration wins, and whether it is deterministic across scan-trigger orderings) under the standard scanner, under punk::libunknown, and with punk::packagepreference active; differing versions across the two forms (version selection integrity including package prefer latest, and whether the losing form's registration lingers); re-registration effects (package forget then re-require crossing forms); surprising-but-accepted behaviours are pinned with GAP/known-quirk comments (the fossilmove characterization pattern), outright bugs fixed or filed as goals; the resulting do/don't guidance lands in src/modules/AGENTS.md and src/lib/AGENTS.md naming the characterised failure modes (or explicitly lifting the avoid-mixing rule if unwarranted).
## Context

1
goals/G-038-piped-session-continuity.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/lib/app-punkshell/punkshell.tcl (eof-restart handover), src/modules/punk/repl-999999.0a1.0.tm (eof-restart done-mode that skips codethread teardown), src/modules/punk/repl/codethread-999999.0a1.0.tm (as touched)
Goal: when piped stdin ends and app-punkshell opens the interactive console shell, the session continues rather than restarts - the piped phase's codethread/code interp survives (variables, procs, namespaces, loaded packages, cwd, ::errorInfo/::errorCode) and only the input channel and repl reader are renewed - so `'set ::jjj blah' | <punkexe> shell` leaves ::jjj inspectable at the prompt, replacing today's silent fresh-session swap (least-surprise violation; motivating transcripts in the detail file) and obsoleting the standalone piped-error-record mechanism this goal described before its 2026-07-08 rework.
Acceptance: after `'set ::jjj blah;error xxxx' | <punkexe> shell` reaches the interactive prompt: `set ::jjj` returns blah; the xxxx message, errorInfo traceback and errorCode are inspectable (via preserved ::errorInfo if nothing in the handover overwrites it, else a documented record - the chosen mechanism noted in the detail file); a proc defined and a package required during the piped phase remain available; a cwd change from the piped phase persists; a one-line notice at the restart says the session continued from piped input (mentioning the error when the last piped command errored, quiet otherwise); after the restart a terminal query from the code interp succeeds (e.g. `help console` runs its cursor-position test instead of being refused by the stale settled can_respond=0 anchor); interactive exit/quit teardown afterwards is clean (the G-036 regression harness still passes); PUNK_PIPE_EOF policy semantics and the script subcommand are unchanged; verified on both Tcl generations.
## Context

1
goals/G-039-orphan-console-spin.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/modules/punk/repl-999999.0a1.0.tm (console reader/event loop and EOF/error paths), src/modules/punk/console-999999.0a1.0.tm; investigation-first
Goal: the observed failure mode - an interactive punk902z left running after its hosting terminal/console went away spins roughly a full core indefinitely (observed 2026-07-08: a 37-minute orphan with a single hard-looping thread) - is reliably reproduced and root-caused, then fixed or mitigated so a shell whose console dies exits or reaches zero-CPU idle cleanly.
Acceptance: a documented procedure reproduces the spin on the current kit (e.g. launch an interactive shell in a terminal, then kill/close the hosting terminal or conhost), or the investigation records the attempts made and what evidence would reopen it; the spinning code path is identified (prime suspect: a console read/event loop treating a dead console's immediate EOF/error as retryable without backoff or termination - adjacent to the console-EOF restart path G-038 takes ownership of); after fix/mitigation, the same procedure shows the orphaned process exiting or settling at effectively zero CPU within a short grace period, with live-console interactive behaviour unchanged; the wedge-scoring hazard note (orphans polluting process-liveness checks in test harnesses) is updated to match the outcome.
## Context

1
goals/G-041-punkargs-form-matching.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/modules/punk/args-999999.0a1.0.tm (parse form selection, arg_error/usage form marking), src/modules/punk/ns-999999.0a1.0.tm (cmdhelp/synopsis closest-form indication), src/tests/modules/punk/args/testsuites/
Goal: for multi-form definitions punk::args determines which form(s) an argument list matches - parse without -form attempts all permitted forms instead of effectively form 0, -form accepts the documented list-of-forms restriction, and the documentation surface indicates the match ('i after cancel <id>' presents the cancel form; 's after cancel someid' marks the closest synopsis) - with explicit single-form restriction retained for callers that require it.
Acceptance: parsing a multiform definition (after-like fixture) without -form succeeds when the args match exactly one form (the pinned GAP tests in forms.test flip to auto-selected results); an argument list matching no form (or several) produces an error naming the candidate forms rather than a form-0 type error; -form with a list of form names/indices restricts parsing to that subset (currently an 'Expected int 0-N or one of ...' error); 'i <cmd> <args...>' and synopsis output indicate the best-matching form(s) for supplied args; explicit -form <single-name-or-index> behaviour is unchanged; the full punk::args and punk::ns suites pass.
## Context

3
goals/G-042-subshell-help-topics.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/modules/punk-999999.0a1.0.tm (::punk::helptopic registry), src/modules/punk/config-0.1.tm (stored declaration source), src/modules/punk/repl-999999.0a1.0.tm (subshell entry/exit hooks)
Goal: a named subshell can declare its own help topics (name, aliases, summary, text content) in punk::config stored configuration - registered into the ::punk::helptopic registry on subshell entry and removed/restored on exit - so 'help' inside a customised subshell presents that subshell's topics with 'help topics' and 'i help' staying accurate (the registry already regenerates the punk::args definitions), under a defined precedence policy between topics and command-name fallthrough.
Acceptance: a subshell whose configuration declares at least one custom topic shows it in 'help topics' and renders its configured content via 'help <topic>' inside that subshell, and 'i help' lists it as a documented choice; on quit/switch-away the parent's topic set is restored (no leakage, proven by declaring a topic in a subshell and checking the parent after exit); configuration is read through punk::config (toml per G-014 - no ad-hoc parsing) and a shell with no topic configuration behaves unchanged; config-declared topics are text-content only - they cannot name code to execute (or a recorded design decision to the contrary with its sandboxing rationale in the detail file); the shadowing policy is implemented and documented: a declared topic colliding with a built-in topic is rejected (or explicitly overrides per a documented rule), a topic name shadowing a command name wins over command fallthrough as today with 'i <cmd>'/'s <cmd>' documented as the command-help escape hatch; depends on G-014 for the stored-config substrate - the registry-side registration/restore mechanism may land earlier behind a programmatic API, with the config binding completing when G-014 lands.
## Context
@ -115,6 +116,6 @@ Implications adopted for this goal:
- Seam already in place (punk 0.2.0): ::punk::helptopic::register / resolve / define_docs;
'help topics' derives from the registry; per-topic punk::args ids follow the
::punk::helptopic::<topic> convention consumed by 'i help <topic>' subhelp choiceinfo.
- Related session records: goals/G-040-punkargs-choicealiases.md (choice display folding),
- Related session records: goals/archive/G-040-punkargs-choicealiases.md (choice display folding),
the 2026-07-08 CHANGELOG entries for punk 0.2.0 (registry) and punk::ns 0.1.1
(doc-lookup trace cleanup).

1
goals/G-043-subshell-definition-plugins.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/modules/punk/cap-999999.0a1.0.tm (+ new punk.subshell handler), src/modules/punk/overlay-999999.0a1.0.tm (commandset composition), src/modules/punk/pluginmgr-0.5.1.tm (discovery/trust layer), src/modules/punk/mix-999999.0a1.0.tm (init wiring precedent), src/modules/punk/repl-999999.0a1.0.tm (subshell entry/exit), src/modules/punk/config-0.1.tm (data-declaration overlap with G-014/G-042)
Goal: a named subshell's definition - commandset imports (punk::overlay), help topics (the G-042 registry API) and config defaults - can be supplied by provider packages registered through a punk.subshell capability (punk::cap handler validating declarations) and/or declared in stored config (toml per G-014, naming already-installed commandset packages - absorbing punk::overlay's "toml configuration files for defining CLI configurations" todo), with punk::pluginmgr as the discovery/safe-interp trust layer for providers not already trusted - so entering a subshell composes its command surface and help from declarations instead of hardcoded init code.
Acceptance: a provider package declaring the punk.subshell capability supplies at least a commandset binding (namespace + prefix) and a help-topic set for a named subshell, and entering that subshell composes them (prefixed commands callable, topics in 'help topics'/'i help') with quit/switch-away restoring the parent surface (no leakage); the punk.subshell punk::cap handler validates declarations (malformed declarations vetoed with a useful message); a subshell's commandset composition can equivalently be declared in stored config without a provider package (G-014 substrate), and the existing hardcoded 'dev' CLI composition keeps working unchanged; declarations for capabilities with no registered handler are discoverable (query or report - closing the silent punk.isbogus gap); punk::cap pkg_unregister leaves no stale handler state (the 'destroy api objects?' review resolved); punk::pluginmgr-based discovery/loading of a provider is either demonstrated end-to-end (including .tm module loading in the safe interp) or explicitly deferred with the remaining pluginmgr gaps recorded in the detail file; G-042's registry mechanism is consumed, not duplicated.
## Context

38
goals/G-044-repl-command-completion.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/modules/punk/repl-999999.0a1.0.tm (editbuf/reader integration, provider seam), src/modules/punk/args-999999.0a1.0.tm + src/modules/punk/ns-999999.0a1.0.tm (introspection surfaces as consumed), src/modules/punk/console-999999.0a1.0.tm (rendering)
Goal: the interactive repl (raw mode primary) offers command completion and hinting driven by punk::args documentation - the command word resolved through the same doc-lookup flow as 'i' (ensembles, subcommands, ensemble parameters) and the argument position through the definition (options, choices with parse-consistent matching, literals, form awareness per G-041) - via an activation scheme that preserves the editbuf's literal-tab support (not exclusively plain Tab), behind a per-subshell completion-provider seam so alternative-language subshells (e.g. an interactive xtal session) can replace, augment or cleanly disable the Tcl-centric completer.
Acceptance: in a raw-mode interactive session a documented trigger (recorded in the detail file; a literal tab remains enterable into the editbuf) presents completions for command words, ensemble subcommands (resolved via the punk::ns doc-lookup flow without executing candidate commands), option flags and choice values (matching consistent with parse semantics: prefix/deny/reserve, and aliases once G-040 lands); a hint display shows the synopsis of the form(s) matching the partial input (consumes G-041's candidacy API when available - until then all-forms or form-0 display with the limitation noted is acceptable); no per-keystroke terminal queries are added when no completion display is active (G-013 approach note); the completer is a provider interface with the punk::args-driven implementation as the default - a subshell can declare an alternative provider or none, proven at minimum by a subshell with completion disabled showing no Tcl-centric interference (the xtal minimum bar; full xtal completion out of scope but not precluded by the interface); line-mode behaviour is unchanged unless a documented subset is added; G-013's raw editor essentials are a prerequisite and this goal does not weaken G-013's acceptance.
## Context
@ -85,3 +86,40 @@ arginfo principle: introspection must not run commands to elicit usage).
src/tests/modules/punk/ns/testsuites/ns/cmdflow.test (2026-07-08), including ensemble
-parameters handling; the choice semantics by
src/tests/modules/punk/args/testsuites/args/choices.test; forms by forms.test.
## Repl behaviour preserve-list + testability findings (2026-07-11, user-directed - pre-refactor ordering)
Current interactive behaviour that this goal (and ANY repl refactor) must preserve and
ultimately expand upon (user-specified 2026-07-11):
1. info-complete-parity continuation behaviour incl the Tcl quoting-rules quirk: a
quoted word with unbalanced braces (e.g `set x "{*}{"`) is complete standalone but
inside a braced context (proc body) needs extra closing characters.
2. Closing-prompt hints in line and raw mode driven by the pending-opener stack
(punk::lib::system::incomplete). Accepted limitation, pinned not fixed: only one
close candidate is expressed when several exist.
3. Raw-mode ANSI colour staging: in-progress editbuf colour, different colour once
submitted.
4. Literal tab acceptance preserved in the submitted string; raw-mode tab markers
with navigation/backspace/delete smarts over them.
5. Raw-mode dim grey dots for spaces - display-only; the submitted string carries real
spaces/tabs (dots/markers must never leak into history recall or evaluation).
6. History entries are editbuf objects (repl editbuf_list): up/down arrow navigates
MULTILINE editbufs and recalled entries are editable.
Testability findings (spikes 2026-07-11):
- punk::lib::system::incomplete is a pure function - characterized NOW in
src/tests/modules/punk/lib/testsuites/lib/commandcomplete.test (the quirk scenario
progression, single openers, tabs, escapes, and an incomplete<->info-complete parity
property).
- class_editbuf is console-coupled at its core: add_chunk renders via
overtype::renderline against live terminal metrics (columns via cursor-position
probing, tabstops via DECRQPSR) - headless instantiation works but content operations
emit terminal queries (or fail/spam when ANSI facts are suppressed). Items 3-6 above
are therefore NOT unit-characterizable until a console seam exists: an
::opunk::Console test double (G-001) or injectable metrics is the missing piece, and
G-001 should be understood as the enabling refactor for repl characterization, not
just a feature goal.
- Interactive end-to-end verification tiers: G-020 (screencap+input injection) as the
near-term windows-only harness; a pseudoconsole (ConPTY/pty) expect-alternative as
the durable cross-platform mechanism (candidate goal drafted 2026-07-11).

1
goals/G-045-punkargs-authoring-ergonomics.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/modules/punk/args-999999.0a1.0.tm (record parsing in resolve, tstr interplay, arg_error @cmd rendering), src/tests/modules/punk/args/testsuites/ (rendering.test/defquoting.test as the safety net), src/modules/punk-999999.0a1.0.tm (::punk::helptopic::define_docs de-hacked as the consumer proof)
Goal: authoring punk::args definitions no longer requires backslash line-continuations or ad-hoc workarounds for multi-line records and constructed definitions - a parser-recognised record-continuation mechanism (candidate: an unquoted trailing -& token, with the detail file recording the collision analysis and the element-count disambiguation alternative), -unindentedfields honoured for @cmd fields, and constructed (string-built) definitions able to opt into the same whole-block indent normalization file-style definitions get - with the container quoting rules (braced=literal, quoted=Tcl backslash semantics, \$\{ escape) promoted from defquoting.test into the define documentation.
Acceptance: a definition using the chosen record-continuation mechanism parses identically to its backslash-continuation equivalent (existing definitions unchanged - continuation is additive), with the token's collision rules documented and an escape/rejection story for values that legitimately match it; @cmd -help/-summary honour -unindentedfields (the rendering.test GAP rendering_unindentedfields_cmd_help_GAP flips to aligned); a constructed definition can request whole-block normalization so embedded continuation indentation behaves as in file-style definitions (the rendering_constructed_def_indent_characterization expectations updated to the chosen semantics), and ::punk::helptopic::define_docs drops its manual pre-normalization to prove it; the quoting rules from defquoting.test appear in the punk::args::define -help documentation; the full punk::args suite (128 tests incl. the rendering invariants: nesting independence, relative-indent preservation) passes with GAP tests flipped, none weakened.
## Context

1
goals/G-047-declared-primary-vcs.md

@ -2,6 +2,7 @@
Status: proposed
Scope: punkproject.toml (schema), punkproject.local.toml (new, uncommitted per-checkout override), root AGENTS.md (Commit Conventions (any VCS) section), .gitignore + .fossil-settings/ignore-glob (ignore rules for the local file, per the coexistence contract), src/project_layouts/ (layout template payload - default values and ignore seeding only; project.new validation is follow-on work)
Goal: a `[workflow] vcs = "<system>"` field in punkproject.toml declares the team's primary upstream VCS - the authoritative interchange history and the default target for unqualified "commit"/"checkin" requests - while the same field in an uncommitted per-checkout punkproject.local.toml lets an individual developer redirect their own unqualified commit instructions to their preferred system (mixed git/fossil-preferring teams), resolution order local override > project field > filesystem detection; the declaration governs developer commit workflow only - punk internal machinery (punkcheck tracking of related projects, upstream infrastructure pull per G-027, central project discovery per G-016) remains standardized on fossil regardless of the declared field; punkshell itself carries `[workflow] vcs = "git"` and derived-project layout templates default to fossil.
Acceptance: punkshell's punkproject.toml contains `[workflow] vcs = "git"`, and root AGENTS.md "Commit Conventions (any VCS)" documents the resolution order (punkproject.local.toml field, then punkproject.toml field, then filesystem detection, with the existing prose as final fallback) as the source agents consult for unqualified commit/checkin requests; punkproject.local.toml is ignored by both VCS per the .fossil-settings coexistence contract (git check-ignore matches it, ignore-glob covers it, the contract's verification comparisons stay clean); a reader resolving the primary VCS anchors at the project root via the punk::repo::is_project_root marker and ignores any nested punkproject.toml/punkproject.local.toml `[workflow] vcs`; the fossil-machinery carve-out is recorded in root AGENTS.md alongside the field documentation (a git-primary ecosystem project still maintains its fossil repo for punkcheck/pull/discovery machinery), and the field stays advisory to G-016 discovery with detection as fallback; the mixed-team sync semantics are documented (team primary = authoritative interchange; a developer committing granularly to the secondary owns batching their work up to the primary); project version patch-bumped with a CHANGELOG entry for the schema addition; make.tcl/project.new validation of the field against detected VCS systems stays out of scope (follow-on work).
## Context

1
goals/G-048-textblock-table-punkargs.md

@ -2,6 +2,7 @@
Status: proposed
Scope: src/modules/textblock-999999.0a1.0.tm (textblock::table proc + PUNKARGS, textblock::class::table class - constructor, opts_table_defaults, methods)
Goal: textblock::table parses its arguments via punk::args::parse (replacing the unvalidated dict merge at L6397), with its PUNKARGS definition covering both table-wrapper-specific options (-return, -rows, -headers) and the constructor passthrough options - the latter sourced by referencing punk::args definition blocks authored inline on the textblock::class::table class methods (the pattern used for render_to_input_line and rendertest in punk::ansi::class::class_ansi: lappend PUNKARGS [list { @id ... }] immediately before the method, punk::args::parse $args withid "..." inside the method body), so the constructor itself parses via punk::args and its documented option set is the single source of truth that textblock::table's PUNKARGS references rather than a parallel hand-typed list - retiring the "more options available - argument definition is incomplete" caveat and closing the -return -choiceprefix documentation/parsing mismatch (item #8 from the punk::args -choices audit) as a side effect.
Acceptance: textblock::class::table's constructor carries an inline punk::args define block (lappend PUNKARGS, @id naming the class+method) and parses its args via punk::args::parse withid, replacing the current manual switch at L466; textblock::table's PUNKARGS references the constructor's documented options (via @id reference, a shared fragment, or a documented include mechanism - the chosen mechanism recorded in the detail file) so table's definition is complete without hand-duplicating the constructor's option list; an invalid -return value (e.g. -return tab) produces a punk::args usage error instead of silently passing through; valid inputs reach textblock::class::table new with behaviour parity to the current dict merge; the -choiceprefix 0 on -return is honoured by construction (punk::args::parse enforces it); the "NOTE: more options available - argument definition is incomplete" comment is removed; existing textblock test suites pass.
## Context

3
goals/G-050-synopsis-validity-marking.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/modules/punk/ns-999999.0a1.0.tm (synopsis), src/modules/punk/args-999999.0a1.0.tm (synopsis renderer), src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test (synopsis pins)
Acceptance: see GOALS.md index entry (canonical).
Goal: synopsis ('s') marks supplied argument words for validity the way cmdhelp's usage display does (goodarg/badarg styles via the G-049 parse-status structure), offers the same status machine-parsably in its dict return, and replaces the curried-alias excess-args length arithmetic (the 's pse' REVIEW note) with a parse against the resolved definition.
Acceptance: 's <cmd> <valid args>' renders the supplied words marked as validated and 's <cmd> <invalid args>' marks the offending word, while output with no argument words supplied is unchanged (the synopsis_GAP_no_argument_validity_marking pin flips); the dict return carries per-argument status; the curried-alias substitution behaviour (synopsis_curried_alias_shows_braced_target pin) is resolved to a documented presentation derived from parsing rather than list-length arithmetic; existing synopsis output tests (args testsuite synopsis.test and test::punk::args) pass; depends on G-049 for the status structure; G-044's completion/hint display is a consumer of the same status API (cross-reference, not a dependency).
## Context

3
goals/G-051-cmdinfo-pseudo-and-prefix.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/modules/punk/ns-999999.0a1.0.tm (cmdinfo, cmd_traverse), src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test, src/tests/modules/punk/ns/testsuites/ns/cmdflow.test
Acceptance: see GOALS.md index entry (canonical).
Goal: cmdinfo reports a distinct cmdtype (e.g 'doconly') when resolution lands on a punk::args id with no corresponding real command instead of today's 'notfound', and the space-delimited docid jump accepts the same word prefixes the parser accepts - via the shared punk::args::choiceword_match resolver, not a second matching rule - so 'i string is tr' documents what 'string is tr' actually executes.
Acceptance: the pinned GAP tests flip: cmdhelp_GAP_pseudo_command_cmdtype_notfound and cmdhelp_GAP_string_is_true_pseudo report the new cmdtype with the docid unchanged, cmdhelp_GAP_spaceform_docid_prefix_not_honoured and cmdhelp_GAP_string_is_prefix_not_honoured resolve the child docid from a prefix exactly when parse accepts that prefix (honouring -choiceprefix, -choiceprefixdenylist, -choiceprefixreservelist and -choicealiases per G-040 parity); consumers of cmdinfo's cmdtype (cmdhelp, synopsis, eg) handle the new value with no behaviour change for real commands; cmdflow.test and the non-GAP cmdhelp.test tests pass unchanged.
## Context

3
goals/G-052-oo-method-autodef.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/modules/punk/ns-999999.0a1.0.tm (generate_autodef oo branches, cmd_traverse), src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test
Acceptance: see GOALS.md index entry (canonical).
Goal: an undocumented method on a tcl::oo object or class gets an (autodef) punk::args definition generated from its introspected parameter list (info object call + method definition arglists - the machinery generate_autodef already uses for its class summary), so 'i $obj <method> <args...>' shows method-level usage with good/bad argument marking instead of only the class summary with the method word highlighted - explicitly documented methods continue to win.
Acceptance: the cmdhelp_GAP_oo_undocumented_method_class_summary_only pin flips: cmdinfo on an undocumented method resolves a method-level (autodef) docid with the trailing words in args_remaining, and bogus trailing arguments render error-scheme output instead of an info-scheme class summary; documented-method behaviour (cmdhelp_oo_documented_method) is unchanged; instance methods, class-defined methods and mixin/superclass-inherited methods resolve (constructor/'new' signatures at minimum characterized, in-scope or explicitly deferred in the detail file); the "-choiceprefix 0 ... methods must be specified in full always? - review" question on the class-summary method choicelist is resolved and documented either way; cmdflow.test and cmdhelp.test pass.
## Context

3
goals/G-053-punkargs-multiple-ranges.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/modules/punk/args-999999.0a1.0.tm (spec compiler, parse, arg_error/synopsis renderers), src/tests/modules/punk/args/testsuites/args/
Acceptance: see GOALS.md index entry (canonical).
Goal: -multiple accepts a {min max} occurrence range (mirroring -choicemultiple; max -1 unbounded) alongside the legacy booleans - so a definition can declare "at most once, repeat is an error" ({0 1}) or bounded repetition ({2 4}) instead of choosing between silent last-wins (0) and unbounded collection (1) - with boolean semantics preserved exactly, including the prepend-defaults/last-wins override idiom.
Acceptance: parse raises a usage-style arity error naming the argument for occurrences outside a declared range; boolean -multiple 0/1 behaviour is unchanged (full existing punk::args suite passes untouched); the -optional/range-min reconciliation rule is documented and enforced at define time; the usage table Multi column and synopsis reflect declared ranges; -multipleunique/-multipleuniqueset compose with max>1 ranges unchanged; characterization tests cover the new forms and the value-shape rule.
## Context

3
goals/G-055-tclcore-regen-workflow.md

@ -2,7 +2,8 @@
Status: proposed
Scope: goals/G-055-tclcore-regen-workflow.md (workflow doc), src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (+ buildversion), src/tests/modules/punk/args/testsuites/args/ (parity pins), TEMP_REFERENCE/tcl9 (read-only source input; source retrieval mechanism deferred to the buildsuites toml configs / G-005 era)
Acceptance: see GOALS.md index entry (canonical).
Goal: refreshing punk::args::moduledoc::tclcore is a documented, repeatable agent-driven workflow taking a Tcl source tree + version as input - man-page/source text carried verbatim (rearranged into @cmd -help/@examples/choicelabels, not reimagined; exceptions: synopsis notation is translated into punkshell's own more generic synopsis syntax rather than copied from the source docs, and text may be re-folded/line-wrapped to keep help display-width friendly while punk::args lacks word wrapping), one adaptive package with version-conditional definitions where released behaviour differs (8.6 vs 9.x), each regenerated command verified by a real-vs-model behavioural probe before acceptance, and source-checkin provenance recorded - designed so the same workflow extends later to tkcore (which loads on 'package require tk' and relies heavily on shared documentation sections, e.g the (default)::punk::args::moduledoc::tkcore::tk_standardoptions id) and to moduledocs for other core.tcl-lang.org projects (e.g tcludp).
Acceptance: the workflow is documented in the detail file (inputs, verbatim-text fidelity policy, probe-verification gate, provenance recording, shared-section reuse guidance for the tkcore pattern); the whole ::string ensemble plus a small selection of other commands including the multi-form ::after have been regenerated or verified through the workflow against the reference Tcl 9 sources, with parity pins added and 8.6 released-behaviour differences honoured (G-054's harvest/version-conditional techniques); an initial scan across tclcore commands identifying constructs not adequately modelable in the current punk::args system is recorded in the detail file, with each gap flagged as a candidate goal rather than worked around silently; tkcore and other-project moduledocs remain out of scope beyond the workflow being demonstrably reusable for them.
## Context

3
goals/G-056-punkargs-word-wrapping.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/modules/punk/args-999999.0a1.0.tm (arg_error table and string renderers, helpers), src/modules/textblock-999999.0a1.0.tm (only if cell/column-level wrap is the chosen mechanism), src/tests/modules/punk/args/testsuites/args/ (new wrapping characterization + existing rendering suites)
Acceptance: see GOALS.md index entry (canonical).
Goal: punk::args help display (@cmd -help, argument -help, choicelabels) word-wraps over-width lines to the effective display width at render time - ANSI/grapheme aware, splitting long logical lines but never joining existing ones, so deliberately structured content (hand-folded blocks, choice tables, art choicelabels, indented continuations) is preserved by construction - letting definitions store unfolded verbatim text (retiring the G-055 re-folding exception for new work) while usage output stays terminal-width friendly.
Acceptance: a definition whose -help/choicelabel contains a single long unfolded line renders fully within the effective display width in both the table and -return string renderers (no truncation, no overflow, wrapped continuations indented to match the field's existing paramindent alignment); wrap-point calculation is ANSI-aware (SGR sequences measure zero width; styling in effect carries across the wrap) and grapheme/double-width aware to the same standard as existing punk::ansi/textblock width handling; existing hand-folded and structured help renders byte-identical (full existing args rendering/usagemarking/deferredhelp suites pass unchanged - never-join semantics verified by characterization tests); effective width derives from the terminal when available with the current -maxwidth 80 table default as fallback, and an explicit width option overrides; the chosen mechanism (punk::args-side pre-wrap of field text vs textblock table column wrap support) and its rationale are recorded in the detail file; G-055's folding exception is marked lapsed for new work once this ships.
## Context

3
goals/G-057-kit-icon-embedding.md

@ -2,7 +2,8 @@
Status: proposed
Scope: src/make.tcl (kit/zipkit wrap steps), src/runtime/punk1.ico (project default, existing), src/vfs/*.vfs (override placement convention), src/runtime/mapvfs.config (only if an explicit config element is the chosen override mechanism), TEMP_REFERENCE/tcl-sfe (read-only reference), helper proc location decided in the work (make.tcl inline vs punk::mix lib)
Acceptance: see GOALS.md index entry (canonical).
Goal: Windows kit/zipkit builds produce executables carrying an embedded icon chosen at build time - defaulting to the project icon src/runtime/punk1.ico, overridable per kit by its .vfs folder - by replacing the icon resources in the built executable using the twapi-based mechanism demonstrated in tcl-sfe (TEMP_REFERENCE/tcl-sfe, by twapi author and Tcl core member Ashok P. Nadkarni): RT_ICON/RT_GROUP_ICON replacement via twapi resource-update APIs, applied so the appended vfs payload stays intact (icon the stub before appending, or sfe-style split/update/reattach).
Acceptance: a Windows `make.tcl project` build produces kit executables whose embedded icon resources are the project default punk1.ico, and a kit whose .vfs supplies an override icon gets that icon instead (verified by resource inspection, e.g twapi::extract_resources, not just Explorer eyeballing); the icon-replaced executables still boot to a working punk shell reading their vfs payload for the kit types we build (kit, zip, zipcat per mapvfs.config); runtimes under src/runtime are never modified - replacement applies to the built copies only; twapi unavailable or non-Windows platform skips the icon step with a notice and the build otherwise completes unchanged; rebuilds are idempotent (re-wrapping an already-iconed build copy converges, no resource accumulation); the override convention (filename/location in the kit's custom .vfs folder vs a mapvfs.config element) and the stub-vs-split ordering decision are recorded in the detail file with the tcl-sfe attribution.
## Context

3
goals/G-060-qemu-test-matrix.md

@ -2,7 +2,8 @@
Status: proposed
Scope: build/test orchestration config and scripts (location to be settled with the buildsuites toml work - see G-005/buildsuites direction), goals/G-060-qemu-test-matrix.md (workflow + license posture), src/tests/ (any guest-driving hooks)
Acceptance: see GOALS.md index entry (canonical).
Goal: comprehensive cross-platform verification (linux variants, FreeBSD/NetBSD/OpenBSD, arm architectures) is achievable from a single dev machine by driving QEMU guests as strictly external tooling - punkshell invokes qemu as a separate process with declarative per-guest config, and nothing in punkshell or its artifact repos bundles, links against, derives from or otherwise couples to QEMU (GPLv2) - so the project's BSD licensing posture is unaffected and QEMU remains a swappable convenience: the guest-driving contract is push-based artifact staging and result collection (the same pattern G-059 establishes for WSL - guests never share a working tree with the host) and must be satisfiable by real hardware, WSL or another hypervisor.
Acceptance: a documented, repeatable workflow provisions at least one non-Windows QEMU guest (e.g FreeBSD x86_64) that fetches a punkbin runtime and runs the source-tree suite inside the guest, driven from the Windows dev machine with results collected back; the guest-driving interface is hypervisor-agnostic (documented contract; QEMU is one provider); the license posture is recorded in the detail file (external-process invocation only, no QEMU binaries or derived code committed to punkshell or punkbin, guest OS images not redistributed by the project - fetched/built per machine); suite runs on machines without QEMU are unaffected.
## Context

79
goals/G-061-pseudoconsole-expect.md

@ -0,0 +1,79 @@
# G-061 Pseudoconsole expect-alternative for interactive shell testing (ConPTY + unix pty)
Status: proposed
Scope: test-harness support (location TBD during the work: src/tests/testsupport/ and/or a small punk module), src/tests/shell/ (capability-gated interactive suites), goals/G-061-pseudoconsole-expect.md
Goal: interactive punkshell behaviour is testable headlessly and byte-accurately by driving a real shell under a pseudoconsole - ConPTY on windows, pty on unix - through an expect-like harness (spawn, send literal keys and control/arrow sequences, await patterns in the rendered ANSI byte stream with timeouts, collect transcript), giving the goals/G-044-detail preserve-list behaviours (continuation hints, raw-mode colour staging, tab markers and space dots, multiline history navigation) their durable cross-platform verification tier - complementary to G-020 capture testing (which verifies what reaches the screen; this verifies the byte stream) and to the G-001 console-seam unit tier.
Acceptance: a capability-gated harness spawns a built punkshell under ConPTY on windows (and under a unix pty when run on unix or via the G-059 WSL staging pattern), sends keys including control/arrow sequences, and awaits expected patterns with timeouts and forced teardown on expiry (hang-proof per the shell-test conventions); at least three preserve-list behaviours are verified through it (e.g the in-proc `set x "{*}{"` continuation-hint sequence, raw-mode tab-marker rendering and deletion, up-arrow recall and edit of a multiline history entry); tests skip cleanly where no pseudoconsole is available; the harness API is platform-agnostic with per-OS backends; candidate synchronisation/communication mechanisms between harness and shell (including the inter-subshell beep-protocol idea) are evaluated and the chosen approach documented in the detail file.
## Context
punkshell's interactive behaviour has no automated verification. The preserve-list
recorded in goals/G-044-repl-command-completion.md (2026-07-11, user-specified) covers
what any repl refactor must not break: info-complete-parity continuation with the Tcl
quoting quirk, closing-prompt hints, raw-mode colour staging (in-progress vs submitted),
literal tab acceptance with raw-mode tab markers and edit smarts, dim grey space dots
(display-only - never in the submitted string), and up/down navigation + editing of
multiline editbuf history entries.
The verification tier model established 2026-07-11:
1. PURE layer - testable now: punk::lib::system::incomplete characterized in
src/tests/modules/punk/lib/testsuites/lib/commandcomplete.test.
2. Console-seam UNIT tier - blocked on a seam: class_editbuf is console-coupled at its
core (add_chunk renders via overtype::renderline against live terminal metrics -
cursor-position size probing, DECRQPSR tabstops). An ::opunk::Console test double
([[G-001]]) or injectable metrics is the enabling refactor.
3. RENDERED tier: [[G-020]] screencap+input injection is the near-term windows-only
bridge (verifies what reaches the screen); THIS goal is the durable, headless,
cross-platform, byte-accurate mechanism (verifies the byte stream the shell emits).
## Approach
### Harness contract (platform-agnostic; per-OS backends)
expect-like primitives over a REAL pseudo-terminal:
- spawn: launch a built punkshell attached to a pseudoconsole (ConPTY on windows,
openpty/forkpty on unix) with controlled size (cols x rows) - size control matters
because raw-mode rendering is width-dependent.
- send: literal keys plus control/arrow/escape sequences (up-arrow history recall,
backspace over a tab marker, etc).
- expect: await patterns in the rendered ANSI byte stream with timeouts; forced
teardown on expiry (hang-proof, per src/tests/shell/AGENTS.md conventions - the
punk_run pattern generalized to a live conversation).
- transcript: full byte capture for post-hoc assertions (colour staging = SGR
sequences around the in-progress vs submitted line; tab markers/space dots = the
substituted glyph bytes).
### Open implementation questions (deliberately left to the work)
- Driving ConPTY from Tcl: candidates include a twapi surface (if CreatePseudoConsole
is reachable), a small helper executable owning the ConPTY pair and relaying bytes
over stdio/sockets, or an existing tool at arm's length. The unix side is simpler
(openpty via a small C/critcl shim, or driving through an existing pty-capable tool).
- Synchronisation between harness and shell: pattern-awaiting alone can be racy for
timing-sensitive raw-mode behaviour. Candidate: the inter-subshell communication /
beep-protocol idea (user, 2026-07-11) - an in-band signalling channel the shell can
emit for the harness to key off. Evaluate against simpler prompt-sentinel approaches;
document the choice here.
- Unix execution from the windows dev machine: the [[G-059]] WSL staging pattern
(native-filesystem staging, capability constraint) extends naturally - a pty harness
run inside the distro against a linux punkbin runtime/kit.
### Sequencing
- [[G-001]] console-seam unit tier and [[G-020]] capture tier are complementary, not
prerequisites - but preserve-list items verified at a cheaper tier need not be
re-verified here beyond smoke coverage.
- First verification targets (per acceptance): the in-proc `set x "{*}{"`
continuation-hint sequence, raw-mode tab-marker rendering + deletion, up-arrow recall
and edit of a multiline history entry.
## Notes
- Related: [[G-044]] (completion/hinting - consumes this for its own verification),
[[G-001]] (fake-console unit tier), [[G-020]] (screen-capture tier), [[G-059]] (WSL
execution pattern), [[G-060]] (guest-driving contract - a pty harness inside a QEMU
guest is the eventual cross-platform matrix combination).
- ConPTY availability gates the windows side to Win10 1809+ (fine for punkshell's
supported baseline); the capability probe should verify actual ConPTY function, not
windows version.

21
goals/G-062-project-license-file.md

@ -0,0 +1,21 @@
# G-062 Canonical project license: BSD-2-Clause LICENSE file with SPDX-identified references
Status: proposed
Scope: LICENSE.txt (new, repo root), README.md, punkproject.toml ([project] license field), AGENTS.md (Repo-wide Notes license mention)
Goal: the repository declares exactly one canonical license: a root LICENSE.txt containing the verbatim BSD-2-Clause text, with README.md and root AGENTS.md naming the license by its SPDX identifier and pointing at LICENSE.txt, and punkproject.toml carrying the identifier machine-readably so tooling (the G-063 audit, derived-project seeding) can read the project license without parsing prose.
Acceptance: LICENSE.txt exists at the repo root containing the standard BSD-2-Clause text with a real copyright line; README.md names BSD-2-Clause and points at LICENSE.txt; root AGENTS.md Repo-wide Notes names the license precisely; punkproject.toml [project] carries license = "BSD-2-Clause"; a repo-sweep for top-level license claims finds none contradicting it (sweep result recorded here).
## Context
README has said only "BSD license" since 2023 with no LICENSE file anywhere at the
repo root - every LICENSE file in-tree belongs to a vendored library. "BSD" without
a variant is not a specific license; BSD-2-Clause is the chosen variant
(decision 2026-07-11).
## Notes
- Copyright line content (holder name, year range) to be confirmed with the user at
write time.
- Seeding LICENSE.txt into generated project layouts is a natural follow-on but is
deliberately not part of this goal's acceptance (src/project_layouts sync is
restricted; see G-012/G-027 territory).

34
goals/G-063-package-license-tracking.md

@ -0,0 +1,34 @@
# G-063 Per-package license tracking: SPDX-normalized indications with copyleft audit
Status: proposed
Scope: src/modules (Meta license headers), src/vendormodules/ + src/vendorlib/ (vendored license recording), punk::mix module templates (%license% seeding), mapping module (new, name TBD), audit surface (src/make.tcl or dev commandset - TBD)
Goal: every package punkshell ships - first-party modules, vendored modules, vendored libs - carries a license indication resolvable to an SPDX identifier, without requiring module authors to know SPDX: authors write familiar names ("BSD", "MIT", "Tcl license") in the existing Meta license header slot and a mapping layer normalizes them; an audit surface reports per-package license posture and loudly distinguishes copyleft/viral licenses and unresolved/unspecified entries, so GPL-family code cannot enter the tree unnoticed.
Acceptance: a mapping facility (punk module, name decided in the work) resolves friendly license names to SPDX ids - tolerant of case/spacing variants, covering at least every value currently present in Meta license headers, and returning a distinct "unresolved" result for unknown strings rather than guessing; an audit command (make.tcl subcommand or dev commandset command, decided in the work) enumerates packages under src/modules, src/vendormodules and src/vendorlib and reports each one's SPDX id, unresolved raw value, or unspecified; a documented copyleft policy list (GPL, AGPL and LGPL families at minimum) is flagged distinctly in audit output; module templates no longer emit the literal %license% placeholder; first-party <unspecified> headers are populated or the remainder listed here as pending with reasons.
## Context
Module files already carry a teapot-style "Meta license" header slot, but it is
mostly unpopulated: a 2026-07-11 tally across src/modules + src/vendormodules found
69 <unspecified>, 19 literal %license% template placeholders, 31 MIT, ~36 BSD,
1 ISC, 1 <unknown>. Vendored libraries under src/vendorlib carry their upstream
LICENSE files in their directories but nothing aggregates them. The project's
copyleft caution is already on record (G-060's GPL-safe posture); this goal makes
it auditable per package.
## Approach
- Reuse the existing Meta license slot rather than inventing new header syntax;
normalization happens at read time.
- Mapping data lives in a punk module so the shell (lib.search per G-064) and
make.tcl share one implementation. A vendored subset of the SPDX license list
plus pass-through of already-valid SPDX ids is expected to suffice; the full
list is large and mostly irrelevant to the Tcl ecosystem.
- Audit starts read-only (report/warn). Enforcement (abort on copyleft match)
follows the G-026 pattern: policy with explicit override, only after a cleanup
pass proves it won't block routine work.
## Notes
- Related: G-026 (vendorupdate is the natural hook for capturing vendored license
provenance), G-060 (recorded GPL-safe posture), G-062 (project's own license id),
G-064 (lib.search as the interactive surface for these indications).

23
goals/G-064-libsearch-machine-returns.md

@ -0,0 +1,23 @@
# G-064 lib.search machine-parsable returns (dict/json) and license surfacing option
Status: proposed
Scope: src/modules/punk/mix/commandset/loadedlib-999999.0a1.0.tm, src/tests/modules/punk/mix/ (new testsuite), G-063 mapping module (as consumed)
Goal: dev lib.search results are consumable by programs as well as humans: ansi-free -return options producing a Tcl dict and json with a documented structure (package name, available versions, present version as data fields), plus an option that surfaces each package's license indication in both the human table and the machine returns - so an agent piping 'dev lib.search -return json', or Tcl code using -return dict, needs no ansi stripping or list-position guessing.
Acceptance: -return gains dict and json choices whose output contains no ansi escapes regardless of -highlight; the returned structure is documented in the command's punk::args definition; the present version is conveyed as a data field, not highlight markup; a license option adds a per-package license field (SPDX id, unresolved raw value, or unspecified - per the G-063 mapping) to table, dict and json returns; new tests pin dict/json shape, ansi-freeness, and the license option's unspecified fallback.
## Context
Current -return choices are {table tableobject list lines}; list/lines embed ansi
highlight codes unless -highlight 0 is passed, and the list structure is
undocumented positional {name versions} pairs. G-049 (achieved 2026-07-10)
established the machine-parsable -return dict pattern on punk::ns::cmdhelp; this
goal applies it to lib.search and adds json for agent consumers (the G-017
piped-call direction).
## Notes
- The dict/json returns do not depend on G-063 and can land first; the license
field consumes G-063's mapping when present (progress tracked here if split).
- json serialization mechanism decided in the work: a quick sweep found no obvious
json writer in-tree (options: minimal local serializer, vendored tcllib
json::write, or a punk-native helper).

106
goals/G-001-pluggable-console-backends.md → goals/archive/G-001-pluggable-console-backends.md

@ -1,7 +1,8 @@
# G-001 Pluggable console backends for non-detectable terminals
Status: proposed
Status: achieved 2026-07-11
Scope: src/modules/opunk/console-999999.0a1.0.tm, src/modules/punk/console-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm, src/lib/app-punkshell/punkshell.tcl
Goal: an interactive REPL can be launched against a non-detectable terminal-like device (ssh channel, tk text widget) via an ::opunk::Console subclass, with no edits to the base class or punk::console.
Acceptance: a subshell started with an ssh-channel-backed and a tk-widget-backed ::opunk::Console subclass runs an interactive REPL that reads/writes through that console; size, at_eof, and can_respond are answered by the subclass overrides; the base ::opunk::Console and punk::console module are unchanged.
## Context
@ -44,3 +45,106 @@ Then add launch-time console selection so a REPL or subshell can be started agai
- The base class `at_eof` does a non-blocking 1-byte probe on pipe-like channels and parks the byte in `waiting_chunks_waiting`. The ssh subclass should override `at_eof` to use `chan eof` on the ssh channel directly — probing a socket would consume a byte the protocol layer may need.
- The "first subshell asymmetry" TODO at `repl-999999.0a1.0.tm:3130-3132` is G-002's concern, not G-001's, but G-001's launch-time console selection is a prerequisite for G-002's "target a named console" acceptance criterion. Sequence G-001 before G-002.
- No persisted prior chat on this topic was found in project sessions; the motivation comes from the existing class design comments and the user's stated intent.
## Progress (activated 2026-07-11)
Additional context since authoring: the G-044-detail repl characterization work (2026-07-11)
identified this goal as the ENABLING refactor for repl/editbuf unit testing - class_editbuf
is console-coupled and needs a deterministic console double. A third reference subclass
(the test double) was therefore added ahead of ssh/tk in the build order.
### Landed (increment 1)
Three backend modules under src/modules/opunk/console/, base ::opunk::Console and
punk::console UNCHANGED (verified - no diffs):
- `opunk::console::test` / ::opunk::TestConsole - deterministic channel-pair double:
fixed size (-columns/-rows at construction, stored in the inherited default-size
field), is_console_or_tty/can_respond 1 (settled wins), at_eof = plain chan eof with
NO probe (a pending byte is never consumed - pinned by test).
- `opunk::console::ssh` / ::opunk::SshConsole - per the Approach: construction-time
capability, chan-eof without probe, size via the registered size_query_provider.
Flagship verification: with a scripted "remote terminal" answering CSI 6n over a
socket pair, ::opunk::Console::size on the subclass value resolves 100x30 through
punk::console's ANSI cursor-report provider QUERYING OVER THE SOCKET - the entire
value proposition proven without a real ssh connection.
- `opunk::console::tk` / ::opunk::TkConsole - widget path in the in/out slots,
terminal_class tk-text, size from widget char dims, at_eof via backend marker
(::opunk::console::tk::set_eof) or widget destruction; no Tk require at module load.
Verified live under punk91 src (the tk-loading experiment kit): size/eof
flag/clear/destroy all correct.
voo -extends findings recorded in src/modules/opunk/AGENTS.md: child classes inherit
public accessors + field INDEX variables but not parent-private my.* accessors -
subclass constructors initialise inherited private fields via the index variables;
method bodies use the parent's public accessor methods.
Tests: src/tests/modules/opunk/console/testsuites/console/backends.test (8 tests;
7 green + tk env-gated on tcl 9.0.3 and 8.7; tk case verified under punk91).
### Landed (increment 2, 2026-07-11) - acceptance met
Repl launch wiring (punk::repl 0.4.0, base ::opunk::Console and punk::console still
UNCHANGED - verified via VCS status at the flip):
- `repl::init -console <spec>` resolves any console_spec_resolve form and stores per-repl
channel state (repl::conin/conout/conerr; conerr==conout for a foreign console - per-repl
err channels are G-011). `repl::start`'s inchan is now optional, defaulting to the
selected console's input. All existing callers (always explicit stdin) unchanged.
- Output routing: rputs maps stdout/stderr (and the debug pseudo-channels) per-repl;
doprompt writes conerr and prompts unconditionally for a foreign console (a selected
console is a terminal by declaration - process tcl_interactive no longer gates it).
- Code-interp output: for a foreign console the init_script (repltype punk/0) installs
shellfilter 'var' JUNCTION stacks on the code interp's stdout/stderr (no pass-through
to the process channels); the repl collects the pending vars after each runscript and
emits them to the console via rputs. Interleaving between the two streams within a
run is not preserved; emission is per-run (streaming is future work, see residue).
- Console-object dispatch: new repl::console_at_eof / console_get_size /
console_is_default helpers - eof checks at the reader loop and the error traps, and
size for the raw-mode width path, dispatch to the selected object's (possibly
overridden) at_eof/size; can_respond gates the ansi_wanted==2 settle without probing.
Process-console behaviours (stdin reopen on eof, raw-mode re-enable, mode line at
exit, the windows utf-16be line re-decode experiment) now apply only to the default
console.
- opunk::console::tk 0.2.0: TkConsole holds the widget in a subclass o_widget field
(always the authority for size/eof/capability; new 'widget' accessor) and accepts
-in/-out channels; new minimal wiring ::opunk::console::tk::console builds a
reflected output channel rendering into the widget plus a Return-binding input pipe
(submit/feed/teardown), so a channel-driven repl runs against the widget. Default
widget-path construction unchanged (backends.test pins re-verified).
Acceptance verification (src/tests/modules/punk/repl/testsuites/repl/consolebackends.test,
exec-driven child processes via src/tests/testsupport/repl_console_driver.tcl - a repl
cannot run inside the shared runtests testinterp because codethread quit/exit callbacks
thread::send to the thread's MAIN interp):
- ssh: ::opunk::SshConsole over a socket pair; the scripted remote terminal answers
CSI 6n (repl::console_get_size resolves 100x30 THROUGH THE SOCKET via punk::console's
provider, exercising the subclass size + can_respond overrides) and converses
prompt-by-prompt: results (7, 42), diverted code-interp stdout+stderr routed to the
socket, clean `exit 0` completion.
- tk: ::opunk::TkConsole over a wired text widget; lines "typed" via feed echo in the
widget, results and diverted output render into it, size from widget char dims
(100x30), at_eof from the widget marker/existence, clean `exit 0`. (Runs un-gated:
Tk loads only in the child process; self-gates when Tk is unavailable.)
- test: ::opunk::TestConsole chan-pipe double - same conversation, the repl seam for
future editbuf/repl characterization (G-044).
### Residue / follow-on notes (non-gating)
- Foreign-console code-interp output is emitted per-run (batch): a long-running command's
incremental output and between-run background `after` writes appear at the next run
boundary. A streaming bridge (junction-to-pipe + fileevent forwarding) is the natural
refinement - fits the G-002 subshell-comms work.
- Junction stacks are wired for repltype punk/0 only; safe/safebase/punksafe subshell
types on a foreign console would still write code-interp output to the process std
channels.
- Line mode is the verified path; raw-mode/editbuf behaviour on a foreign console is
untested (G-013 raw-default and G-044 completion work will exercise it; the width
path already routes via console_get_size).
- ansi_wanted/colour_disabled and the raw-state tsv remain process-global (documented
punk::console design); a foreign-console repl settling ansi_wanted affects the whole
process - scoped console state is G-008.
- get_prompt_config still keys result/info prompt decoration on process tcl_interactive,
so a foreign-console repl in a non-interactive process shows undecorated results
(prompts themselves are emitted).

1
goals/G-007-console-location-transparency.md → goals/archive/G-007-console-location-transparency.md

@ -2,6 +2,7 @@
Status: achieved 2026-07-05
Scope: src/modules/punk/console-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm
Goal: punk::console presents one API in every interp/thread of a punk session - code-interp callers see the same console facts and can perform the same queries/operations as the parent, routed to the console-owning context via a punk-side ownership registry, with no `repl eval` required.
Acceptance: from a running punk session's code interp, without `repl eval`: `console_fact_get` returns the same values the parent sees and a fact set in the parent is immediately visible; a terminal query (e.g. `get_cursor_pos` or `dec_get_mode`) against the default console succeeds and cooperates with the repl reader (no lost or garbled input); a console constructed and owned by code-interp code is operated on locally (no round-trip to the parent); the existing console test suites pass and single-interp (non-repl) usage is unchanged.
## Context

5
goals/G-015-script-subcommand-piped-stdin.md → goals/archive/G-015-script-subcommand-piped-stdin.md

@ -1,8 +1,9 @@
# G-015 Punk executable `script` subcommand: reliable non-interactive piped/script execution
Status: achieved 2026-07-07 (implemented as src/lib/app-punkscript; verified on punk902z/tcl9 and punksys/tcl8.6 kits: piped success/error/exit-code cases, file form with args and stdin passthrough, and the motivating `dev projects.work *tomlish*` one-liner - table emitted, exit 0, no boilerplate. Stdin form additionally echoes the script's final result when non-empty, giving return-value commands one-liner ergonomics; file form keeps pure script semantics. Terminal-stdin-with-no-args yields a usage error by design - spot-check from a real console typed session remains a manual item.)
Status: achieved 2026-07-07
Scope: src/vfs/_config/punk_main.tcl, src/lib/app-punkshell/punkshell.tcl (script path or a leaner dedicated app package)
Acceptance: as in root GOALS.md index (canonical).
Goal: `<punkexe> script [<scriptname>] [<args>...]` executes a script file or piped stdin content (scriptname optional when input is piped) in an interp preloaded with the basic punk modules and aliases a punk shell provides by default, and always terminates with the script's success/failure as its exit code - without the `shell` subcommand's shellfilter channel transforms/logging stacks and without ever dropping into an interactive shell - so agents can reliably make piped script calls to punk executables.
Acceptance: piping commands to `<punkexe> script` runs them and terminates at stdin EOF with no trailing `exit` required, exit code 0 on success; a failing piped command terminates the process with a nonzero exit code and the error on stderr, never landing in an interactive shell regardless of console availability or PUNK_PIPE_EOF; `<punkexe> script <file> [<args>...]` executes the file with conventional ::argv0/::argv and propagates its error status the same way; the script path installs none of the `shell` subcommand's shellfilter stacks/transforms and the launch plumbing itself emits nothing on stdout/stderr (the current stub's stderr diagnostics removed) so exec-style callers see only the script's own output; the motivating example works with no package require boilerplate: piping `dev projects.work *<name>*` to `<punkexe> script` emits the matching-project table and exits 0, because the script interp carries the default punk shell module/alias environment.
## Context

3
goals/G-036-tcl9-udp-console-worker-wedge.md → goals/archive/G-036-tcl9-udp-console-worker-wedge.md

@ -2,7 +2,8 @@
Status: achieved 2026-07-08
Scope: src/modules/shellthread-999999.0a1.0.tm, src/bootsupport/modules/shellthread-1.6.2.tm, src/modules/shellfilter-999999.0a1.0.tm (as characterised - no product-code changes required by this goal; the punkshell mitigations are separate fixes)
Acceptance: (reworked 2026-07-08 after the root cause was found) the wedge mechanism is identified and written up in the detail file - DONE: bundled tcludp 1.0.12's Windows per-thread UDP_ExitProc closes the process-global synchronization events, proven by dump handle-table analysis plus a live CloseHandle breakpoint and confirmed by the upstream 1.0.12->1.0.13 diff, which already fixes it (so no upstream report is required; the standalone minimal repro originally required here is waived as moot by the user); the tcl9 kits bundle tcludp >= 1.0.13 with the in-context batch harness baseline resolved - DONE 2026-07-08 (run-2 syslog workers alive vs the 4/4-wedged 1.0.12 baseline); REMAINING: a has_bug-style detection in the punkshell check machinery (in the vein of punk::lib::check::has_tclbug_* / punk::console::check::has_bug_*, surfaced through the same reporting as 'help tcl'/'help console') reports the vulnerable combination - simple version-based detection (loaded/bundled tcludp < 1.0.13 on Tcl 9 Windows) is sufficient, no behavioural probe needed; loose-end decisions (punk8win 8.6 kit's udp 1.0.12 swap; optional upstream tickets for residual tcludp trunk weaknesses) recorded in the detail file when made.
Goal: the Tcl 9-only wedge - a worker thread that has used a tcludp syslog socket stops servicing its event queue (timers, thread::send) when the punkshell process is console-attached, the proximate trigger of the piped-stdin exit/quit hang - is root-caused to a named component (Tcl 9 Windows console driver, tcludp, Thread extension, or a specific interaction) with the smallest demonstrating repro, so the user can decide on and manually file an upstream report.
Acceptance: (reworked 2026-07-08 after the root cause was found) the wedge mechanism is identified and written up in the detail file - DONE: bundled tcludp 1.0.12's Windows per-thread UDP_ExitProc closes the process-global synchronization events, proven by dump handle-table analysis plus a live CloseHandle breakpoint and confirmed by the upstream 1.0.12->1.0.13 diff, which already fixes it (so no upstream report is required; the standalone minimal repro originally required here is waived as moot by the user); the tcl9 kits bundle tcludp >= 1.0.13 with the in-context batch harness baseline resolved - DONE 2026-07-08 (run-2 syslog workers alive vs the 4/4-wedged 1.0.12 baseline); DONE 2026-07-08 (punk::lib 0.3.0 has_libbug_udp_threadexit, surfaced via 'help tcl' in punk 0.2.1): a has_bug-style detection in the punkshell check machinery (in the vein of punk::lib::check::has_tclbug_* / punk::console::check::has_bug_*, surfaced through the same reporting as 'help tcl'/'help console') reports the vulnerable combination - simple version-based detection (loaded/bundled tcludp < 1.0.13 on Tcl 9 Windows) is sufficient, no behavioural probe needed; loose-end decisions (punk8win 8.6 kit's udp 1.0.12 swap; optional upstream tickets for residual tcludp trunk weaknesses) recorded in the detail file when made (open as of 2026-07-08 - non-gating).
## Context

1
goals/G-037-vendorlib-vfs-propagation.md → goals/archive/G-037-vendorlib-vfs-propagation.md

@ -2,6 +2,7 @@
Status: achieved 2026-07-08
Scope: src/make.tcl (new or extended step), src/vendorlib_tcl8 + src/vendorlib_tcl9 (sources), src/vfs/<kit>.vfs/lib_tcl8 + lib_tcl9 (targets), punkcheck tracking
Goal: platform-specific vendored binary packages under src/vendorlib_tcl<N>/<platform> reach the kit vfs lib_tcl<N> trees through a make.tcl step instead of hand-copying - updating a vendored package becomes a vendorlib drop plus standard build invocations (motivating case 2026-07-08: tcludp 1.0.12 -> 1.0.13 for the G-036 wedge fix - `libs`, `vfscommonupdate` and `project` all completed while every kit vfs still bundled udp 1.0.12, requiring a manual copy into each vfs lib_tcl9 folder).
Acceptance: with a newer package version placed under src/vendorlib_tcl9/<platform>, one documented make.tcl invocation updates the participating src/vfs/*/lib_tcl9 trees - installing the new package and removing or explicitly retiring the superseded version (no silent mixed-version provision, per the G-035 concerns) - with punkcheck-tracked provenance; which vfs folders participate is explicitly declared per kit rather than blanket-copied (kit vfs package sets may intentionally differ), with the declaration mechanism recorded (candidate home: the G-024 mapvfs toml); a subsequent `make.tcl project` yields kits loading the new version (provable via the tcludp case: built punk902z reports `package require udp` == 1.0.13 with no udp1.0.12 folder remaining in its vfs); the lib_tcl8 tree gets the same treatment or an explicit exclusion rationale in the goal record.
## Context

1
goals/G-040-punkargs-choicealiases.md → goals/archive/G-040-punkargs-choicealiases.md

@ -2,6 +2,7 @@
Status: achieved 2026-07-08
Scope: src/modules/punk/args-999999.0a1.0.tm (parse + usage rendering), src/modules/punk/ns-999999.0a1.0.tm (cmdinfo/cmd_traverse choice resolution parity), src/modules/punk-999999.0a1.0.tm (punk::help topic argdoc as first consumer), src/tests/modules/punk/args/testsuites/, src/tests/modules/punk/ns/testsuites/
Goal: punk::args supports choice aliases (-choicealiases {alias canonical ...}) accepted at parse and normalized to the canonical choice in results, folded into the canonical entry in usage display - and the punk::ns doc-lookup walk resolves choice words by the same rules as the parser (aliases, -choiceprefix, -choiceprefixdenylist, -choiceprefixreservelist) - so alias sets like punk::help's topics|help and console|term|terminal collapse to one displayed entry per topic with `help X` and `i help X` agreeing.
Acceptance: a definition using -choicealiases parses an alias (and an alias prefix where -choiceprefix allows) to its canonical choice in the parse result, with -choicerestricted 0 passthrough and the deny/reserve lists honoured unchanged; usage display shows one entry per canonical choice with aliases folded (no duplicate rows; -choicelabels attach to the canonical); punk::ns::cmdinfo/cmd_traverse resolve subcommand words to docids with the same outcome as the parser for alias, prefix, denied, reserved and unknown words (the pre-goal characterization tests updated from pinned-GAP to fixed); punk::help's topic definition adopts the feature so `i help` lists one entry per registered topic while `help h`/`help e` still fall through to command lookup; definitions without -choicealiases behave unchanged (existing punk::args and punk::ns suites pass).
## Context

1
goals/G-046-punkargs-deferred-help-and-fixes.md → goals/archive/G-046-punkargs-deferred-help-and-fixes.md

@ -2,6 +2,7 @@
Status: achieved 2026-07-10
Scope: src/modules/punk/args-999999.0a1.0.tm (resolve/get_dict: display-field deferral, dynamic-cache subst path, prefix writeback, string renderer, cmdhelp-facing messages), src/modules/punk/ansi-999999.0a1.0.tm (mark_columns argdoc as the reentrancy/perf testbed), src/tests/modules/punk/args/testsuites/ (GAP tests flip; perf verification)
Goal: argument resolution no longer processes -help and other display-only fields - their tstr expansion is deferred to display time (separately cached, per the existing in-source review notes) - so first parse of heavily documented commands gets measurably faster and definitions whose -help calls punk::args-parsing commands (the punk::ansi::mark_columns class) neither loop nor stall; alongside, the mechanical defects pinned by the characterization suites are fixed: @dynamic double-substituted multiline values align at their insertion column, prefix-normalized choice values keep the same shape as exact input, the -return string renderer aligns cmd-help continuations under the first line, and the misleading goodargs parse-error prefix in 'i <cmd> <args>' output is fixed.
Acceptance: parsing/argument resolution provably skips -help expansion (a definition whose -help contains a ${[...]} that would error or record its invocation shows the substitution did NOT run during a parse-only path, only for help display); first parse of punk::ansi::mark_columns drops from ~4s to well under a second with 'i punk::ansi::mark_columns' still rendering the embedded example, and a -help that parses its OWN definition id resolves or errors cleanly rather than looping; first-parse timing improves for at least one other heavily documented command (recorded in the detail file); rendering_atdynamic_multiline_help_insertion_GAP flips to all-aligned; choicegroups_imap_prefix_listwrap_GAP flips to shape-identical (prefix input yields the same plain string as exact input); the -return string renderer's cmd-help continuations align under the first line with relative indents preserved (rendering_string_renderer_characterization updated); the 'Bad number of leading values...' prefix shown by goodargs parsing in 'i string is'-style output is reworded or suppressed for the usage-display path; full punk::args and punk::ns suites pass with no non-GAP expectations weakened.
## Context

3
goals/G-049-punkargs-parse-status-model.md → goals/archive/G-049-punkargs-parse-status-model.md

@ -2,7 +2,8 @@
Status: achieved 2026-07-10
Scope: src/modules/punk/args-999999.0a1.0.tm (arg_error, parse error dispatch, colour-scheme handling), src/modules/punk/ns-999999.0a1.0.tm (cmdhelp), src/tests/modules/punk/args/testsuites/args/usagemarking.test, src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test
Acceptance: see GOALS.md index entry (canonical).
Goal: the information behind usage-display argument marking (which supplied arguments validated, which argument failed and why, which scheme applies) exists as a documented parse-status structure produced from a parse attempt and consumed by both arg_error renderers, and punk::ns::cmdhelp can return it machine-parsably via -return dict - replacing the transient goodargs/badarg locals, the badarg gaps in non-choice validation failures, and the stateful shared colour-array scheme handling.
Acceptance: cmdhelp -return dict distinguishes an incomplete, a fully-valid and an invalid argument set via per-argument statuses (received/ok/bad + overall scheme/message/form) with the structure documented; the table and string renderers derive their marking from that same structure, with rendered output unchanged except where the pinned GAP tests flip: badarg marking covers type/allocation failures not just choice violations (cmdhelp_GAP_no_badarg_marking_for_failed_typed_value), an explicit -scheme is honoured on the parse-failure path (cmdhelp_GAP_explicit_scheme_ignored_on_failure), the failure message names the queried command instead of cmdhelp's internal parse source line (cmdhelp_GAP_errormsg_leaks_internal_source), and scheme rendering no longer depends on or mutates shared colour state - the documented -scheme choice value 'nocolour' takes effect and repeated renders of the same call are identical regardless of prior scheme renders (usagemarking_GAP_scheme_nocolour_renders_with_leftover_colours, usagemarking_GAP_dash_nocolour_leaks_into_shared_array, usagemarking_GAP_dash_nocolour_leak_affects_later_info_render); all non-GAP characterization tests in usagemarking.test and cmdhelp.test pass unchanged.
## Context

6
goals/archive/G-054-tclcore-stringis-harvest.md

@ -0,0 +1,6 @@
# G-054 tclcore moduledoc: runtime-harvested 'string is' class choices with cross-version behavioural parity pins
Status: achieved 2026-07-11
Scope: src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (+ tclcore-buildversion.txt), src/tests/modules/punk/args/testsuites/args/ (new parity test), TEMP_REFERENCE/tcl9 (read-only reference)
Goal: the ::tcl::string::is definition's class choices (and the generated per-class virtual docids) are harvested from the running interpreter at define time - static choicelabels applied to classes present, generic label for unrecognized classes, a version note for dict - so accept/reject parity with the running Tcl holds on 8.6 and 9.x without hand-maintained per-version lists.
Acceptance: parse/parse_status against ::tcl::string::is and its per-class virtual ids agrees with the real interpreter's error-vs-ok outcome for a pinned probe matrix (missing args, trailing flag-like str word, option/class unique-prefix acceptance and ambiguity rejection, unknown option/class, -failindex var consumption leaving no str, per-version class presence: dict, unicode) on Tcl 9.0.x and 8.6; the rendered choices show only classes the running interp accepts; the parity test derives expectations from the live interpreter (not version arithmetic) and passes under both; existing args/tclcore suites pass; tclcore buildversion bumped with changelog.

3
goals/G-058-static-runtime-packages.md → goals/archive/G-058-static-runtime-packages.md

@ -2,7 +2,8 @@
Status: achieved 2026-07-10
Scope: src/vfs/_config/punk_main.tcl (boot auto_path/tm path filtering), src/modules/punk/packagepreference-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm + src/modules/punk/repl/codethread-999999.0a1.0.tm (code interp / codethread bootstrap), src/modules/shellthread-999999.0a1.0.tm (punkshell-created worker threads) as applicable, src/tests/ (un-gated unit tests + constraint-gated shell/kit integration tests), punkbin artifact repo (separate git repo, local checkout c:/repo/jn/punkbin - pinned runtime additions)
Acceptance: see GOALS.md index entry (canonical).
Goal: punkshell running on a runtime with statically-linked/builtin packages (tclsfe-x64: Thread/twapi/sqlite3/tdbc; punkbin runtimes' builtin Thread/tcllibc/vfs/vlerq; the expected shape of future zig-built static runtimes, G-005) keeps those packages resolvable in every interp and thread punkshell fabricates: boot captures the static baseline (info loaded entries with empty filename, plus their provided versions) before replacing package search paths, code interps and punkshell-created threads are seeded with ifneeded scripts mapping each static package to 'load {} <pkg>', and punk::packagepreference resolves static-vs-bundled by a documented version-aware policy instead of blindly loading a bundled dll over an already-provided static package.
Acceptance: a punk91-style kit (tclsfe-x64 + punk9win.vfs, no thread dll in the vfs) boots to a working repl with no "can't find package Thread" - punk::console loads in the code interp, and package require Thread succeeds there and in a punkshell-created worker thread, resolving to the static version; in the same kit, twapi resolves per the documented policy (no repeat of the observed static-Twapi-masked-by-older-vfs-twapi-5.0.2 double load; a genuinely newer bundled copy remains reachable by that policy); dll-based kits (punk902z) boot and pass their existing shell test baseline unchanged, as does a plain tclsh dev launch; the seeding mechanism is generic - driven by the captured baseline, no runtime-specific package naming - and the boot-time static baseline is introspectable at the repl; the seeding/preference logic is covered by un-gated unit tests against simulated baselines (runnable under plain tclsh), while kit-boot integration tests are gated behind a capability-probed tcltest constraint (a built kit whose baseline shows static entries including Thread) that skips cleanly when no such kit is present; the runtimes used for verification (tclsfe-x64.exe at minimum) are added to the punkbin artifact repository under win32-x86_64 with sha1sums.txt updated, so the constraint is satisfiable on other machines via the existing runtime-retrieval path; the punk91 code-interp vfs/vfs::zip load failure is re-diagnosed after the fix and either resolved or recorded as a distinct issue/candidate goal.
## Context

3
goals/G-059-wsl-test-driving.md → goals/archive/G-059-wsl-test-driving.md

@ -2,7 +2,8 @@
Status: achieved 2026-07-11
Scope: src/tests/ (capability probe helpers + constraint-gated cases in existing suites, e.g the unix sh-payload execution test in modules/punk/mix/testsuites/scriptwrap/multishell.test and a runtime.bash behaviour test), src/tests/AGENTS.md (enablement notes)
Acceptance: see GOALS.md index entry (canonical).
Goal: test runs on a Windows dev machine can exercise selected unix-side behaviour through WSL when it is present AND suitable - a capability-probed tcltest constraint (not mere wsl.exe existence) verifies a launchable distro, required tools (bash, coreutils/sha1 tooling), and working one-way staging into the guest's NATIVE filesystem - with all guest-side execution happening on that native filesystem: per-test artifacts are staged into a WSL-native tempdir and results collected back, the Windows checkout is never operated on via the shared /mnt path (DrvFs is slow and cross-boundary stat differences make git re-hash its index and fossil see phantom changes), and any future full-suite mode uses a separate native clone rather than the shared working tree.
Acceptance: a documented probe helper yields a wsl_linux_available constraint whose checks are capability-based (distro launches and answers uname/tool probes; staging into a native tempdir works) and which cannot misfire on wsl.exe-present-but-unusable installs (no distro, WSL1 limitations, broken interop); the currently unix-gated multishell sh-payload execution test runs green via WSL on a suitable machine and still skips cleanly elsewhere; at least one runtime.bash behaviour test (active/use/run resolution against a fixture runtime folder) runs inside WSL - all such tests executing from a native-filesystem staging dir with the shared path used only for one-way copy-in/out; the Windows checkout's git and fossil state is untouched by a WSL-gated run (verifiable: git status/fossil changes identical before and after); suite results on a WSL-less machine are unchanged (skips, not failures); enablement/limitations and the staging pattern recorded in src/tests/AGENTS.md.
## Context

2
punkproject.toml

@ -1,3 +1,3 @@
[project]
name = "punkshell"
version = "0.8.1"
version = "0.11.0"

373
scriptlib/developer/tkconsole_demo.tcl

@ -0,0 +1,373 @@
#! /usr/bin/env tclsh
# =============================================================================
# tkconsole_demo.tcl - developer showcase for the G-001 Tk console backend
# =============================================================================
#
# WHAT THIS DEMONSTRATES
#
# An interactive punk repl running against a Tk text widget instead of the
# process console - the tk-widget case of goal G-001 (see
# goals/archive/G-001-pluggable-console-backends.md). You type commands into
# the text widget; results, prompts and the code interp's stdout/stderr all
# render back into the same widget.
#
# HOW TO RUN
#
# from a punk shell checkout (any of):
# <punkexe> script lib:developer/tkconsole_demo.tcl
# tclsh scriptlib/developer/tkconsole_demo.tcl
# (when run under plain tclsh from inside a punkshell checkout, the script
# adds the checkout's src module/lib paths itself - see BOOTSTRAP below)
#
# options (punk::args-parsed - see the definition below; try --help):
# -columns/-rows widget character dimensions (what the console's 'size'
# method reports to the repl)
# -font text widget font (default TkFixedFont)
# -title window title
# -demo auto-type a short scripted session via
# ::opunk::console::tk::feed before handing you the keys
# -autoclose <ms> destroy the window after <ms> (automation/testing aid -
# exercises the <Destroy> -> teardown -> eof path)
#
# THE MECHANICS, LAYER BY LAYER (follow the numbered comments in the code)
#
# [1] ::opunk::TkConsole (module opunk::console::tk) is a voo value-based
# subclass of the -virtual base class ::opunk::Console. A console value
# is a plain Tcl list whose slot 0 carries the concrete class namespace
# tag, so existing holders calling BASE-class methods
# (::opunk::Console::size $obj, ::at_eof, ::can_respond) dispatch to the
# subclass overrides - no edits to the base class or punk::console were
# needed (that was G-001's acceptance constraint). TkConsole's overrides
# answer from the WIDGET: size = the text widget's ACTUAL character
# dimensions while mapped (current pixel size / font metrics - resize
# the window and watch the status bar and size queries follow; the
# requested -width/-height only answer for an unmapped widget),
# at_eof = a backend marker (::opunk::console::tk::set_eof) or widget
# destruction, can_respond/is_console_or_tty = 1 by construction.
#
# [2] ::opunk::console::tk::console $widget wires the widget as a LIVE
# console and returns a TkConsole whose in/out slots carry CHANNELS:
# - out: a reflected channel (chan create write); everything written
# to it is rendered into the widget (ansi-stripped when punk::ansi
# is present, \r\n normalized)
# - in: the read end of a chan pipe; the <Return> binding (proc
# 'submit') takes the text typed after the 'conin_start' mark (i.e
# since the last output) and writes it as one line to the pipe.
# ::opunk::console::tk::feed does the same programmatically.
# - <Destroy> on the widget runs 'teardown': flags backend eof and
# closes the pipe's write end, so the repl's reader sees eof.
# Because in/out are real channels, the channel-driven repl core needs
# no special casing - the repl reads/writes channels, and consults the
# console OBJECT for the capability questions.
#
# [3] repl::init -console $con resolves the spec via
# punk::console::console_spec_resolve and stores per-repl channel state
# (repl::conin/conout/conerr - conerr==conout for a foreign console;
# separate err channels are goal G-011). Because a foreign console is
# selected:
# - prompts/results are written to the console channels (rputs maps
# stdout/stderr per-repl; doprompt no longer needs tcl_interactive)
# - the codethread's CODE INTERP gets shellfilter 'var' JUNCTION
# stacks on its stdout/stderr: writes are diverted (no pass-through
# to the process std channels) and the repl emits the collected
# output to the console after each command run. Note the caveat:
# output of a run appears when the run completes - incremental
# output of a long-running command is not streamed (yet).
# - eof and size questions dispatch through repl::console_at_eof /
# repl::console_get_size to the TkConsole overrides ([1]).
#
# [4] repl::start (input channel omitted - it defaults to the selected
# console's input) blocks in a vwait servicing the event loop, which is
# what keeps Tk alive: key events fire the <Return> binding, the repl's
# readable handler fires on the input pipe, and the reflected output
# channel renders. Typing 'exit' (or 'quit') in the console - or
# destroying the window - completes repl::start and this script exits.
#
# [5] in-session size queries: 'punk::console::get_size' typed at the P%
# prompt is bridged (by this demo - see the commented block before
# repl::start) from the code interp back to this thread's
# repl::console_get_size, so it reports the tk console's CURRENT size,
# resize-aware, instead of the codethread's process console. This is a
# hand-rolled preview of G-008 scoped console state.
#
# CAVEATS WORTH KNOWING (also recorded in the archived G-001 detail file)
# - line mode only: raw-mode/editbuf interaction with foreign consoles is
# future work (G-013/G-044); there is no history/line-editing beyond what
# the text widget itself gives you.
# - code-interp output is emitted per-run (see [3]).
# - colour/raw-mode/ansi_wanted state is process-global (G-008): the repl
# may settle ansi capability for the whole process.
#
# =============================================================================
# ---------------------------------------------------------------------------
# BOOTSTRAP - make the punkshell dev modules reachable under plain tclsh.
# Under '<punkexe> script' (app-punkscript) the module environment is already
# set up and these requires just work.
# ---------------------------------------------------------------------------
if {[catch {package require punk::repl}]} {
#assume we are scriptlib/developer/<this file> inside a punkshell checkout
set checkout [file dirname [file dirname [file dirname [file normalize [info script]]]]]
if {[file isdirectory [file join $checkout src modules]]} {
package prefer latest ;#dev modules use alpha magic version 999999.0a1.0
tcl::tm::add [file join $checkout src modules] [file join $checkout src vendormodules]
lappend ::auto_path [file join $checkout src lib] [file join $checkout src vendorlib]
}
package require punk::repl ;#errors usefully if we still can't find it
}
#repl::init -console arrived in punk::repl 0.4.0 (G-001). A punk kit built
#before that provides an older repl (and won't take the dev-path fallback
#above since punk::repl IS loadable) - fail with directions instead of an
#'unknown option' error later.
if {[package vcompare [package provide punk::repl] 0.4.0] < 0} {
puts stderr "tkconsole_demo: punk::repl [package provide punk::repl] is too old (need >= 0.4.0 for 'repl::init -console')."
puts stderr "Run from the source checkout (tclsh scriptlib/developer/tkconsole_demo.tcl), via '<punkexe> src script ...', or rebuild the kits."
exit 5
}
package require punk::args
package require opunk::console::tk ;#loads WITHOUT Tk (class def only) ...
package require Tk ;#... Tk is needed for the live wiring below
# ---------------------------------------------------------------------------
# Argument definition & parsing - a punk::args usage example in its own right:
# the definition is both the parser and the documentation ('i' inspectable in
# a punk shell once the namespace is registered, and rendered by --help here).
# ---------------------------------------------------------------------------
namespace eval ::developer::tkconsole_demo {
variable PUNKARGS
lappend PUNKARGS [list {
@id -id ::developer::tkconsole_demo
@cmd -name "developer::tkconsole_demo"\
-summary\
"Interactive showcase: a punk repl on a Tk text widget console (G-001 tk backend)."\
-help\
"Wires a Tk text widget as a live ::opunk::TkConsole via
::opunk::console::tk::console and starts an interactive repl
against it with 'repl::init -console'. Type commands at the
P% prompt inside the widget; 'exit' (or closing the window)
ends the session. See the header comments of
scriptlib/developer/tkconsole_demo.tcl for a layer-by-layer
walkthrough of the mechanics."
@opts
-columns -type integer -default 100 -help\
"text widget width in characters.
Also what the console's size method - and therefore
repl::console_get_size inside the running repl - reports."
-rows -type integer -default 30 -help\
"text widget height in characters (see -columns)"
-font -type string -default TkFixedFont -help\
"font for the console text widget"
-title -type string -default "punk repl on a Tk text widget (G-001)" -help\
"window title"
-demo -type none -help\
"auto-type a short scripted session first, using
::opunk::console::tk::feed (the programmatic input path the
verification tests use), then leave the session interactive"
-autoclose -type integer -default 0 -help\
"destroy the window after this many milliseconds (0 = never).
Automation/testing aid: exercises the <Destroy> binding ->
::opunk::console::tk::teardown -> input-pipe eof path, after
which the repl finishes as if the terminal disconnected."
@values -min 0 -max 0
}]
#register so 'i developer::tkconsole_demo' can find the definition in a punk shell
namespace eval ::punk::args::register {
lappend ::punk::args::register::NAMESPACES ::developer::tkconsole_demo
}
}
apply {{} {
foreach block $::developer::tkconsole_demo::PUNKARGS {
punk::args::define {*}$block
}
}}
if {"--help" in $::argv || "-help" in $::argv} {
puts stdout [punk::args::usage ::developer::tkconsole_demo]
exit 0
}
set argd [punk::args::parse $::argv withid ::developer::tkconsole_demo]
set opts [dict get $argd opts]
set opt_columns [dict get $opts -columns]
set opt_rows [dict get $opts -rows]
set opt_font [dict get $opts -font]
set opt_title [dict get $opts -title]
set opt_demo [dict exists [dict get $argd received] -demo]
set opt_autoclose [dict get $opts -autoclose]
# ---------------------------------------------------------------------------
# UI - a text widget playing the terminal role, plus a small toolbar whose
# buttons poke the console OBJECT directly so you can watch the base-class
# methods dispatch to the TkConsole overrides ([1] in the header).
# ---------------------------------------------------------------------------
wm title . $opt_title
set txt [text .console -width $opt_columns -height $opt_rows -font $opt_font\
-wrap char -background black -foreground green -insertbackground green\
-yscrollcommand {.scroll set}]
scrollbar .scroll -command [list $txt yview]
frame .bar
label .bar.status -anchor w -text "console object: (not wired yet)"
button .bar.size -text "Query size" -command {
#Base-class call, subclass answer: ::opunk::Console::size dispatches on the
#value's slot-0 tag to TkConsole's override, which reads the WIDGET's
#character dimensions - no channel or terminal query involved.
.bar.status configure -text "size: [::opunk::Console::size $::con] at_eof: [::opunk::Console::at_eof $::con] can_respond: [::opunk::Console::can_respond $::con]"
}
button .bar.eof -text "End session (set_eof)" -command {
#Flag backend eof (the marker TkConsole's at_eof consults), then submit an
#empty line: at_eof is CONSULTED by the repl's reader when input arrives,
#so the nudge is what makes the repl notice and finish - a deliberate
#teaching point about where eof checks happen in the loop.
::opunk::console::tk::set_eof $::txt
::opunk::console::tk::feed $::txt ""
}
pack .bar.size .bar.eof -side left -padx 2 -pady 2
pack .bar.status -side left -padx 8
pack .bar -side bottom -fill x
pack .scroll -side right -fill y
pack $txt -side left -fill both -expand 1
focus $txt
# ---------------------------------------------------------------------------
# [2] Wire the widget as a live console. From here on:
# - $con is an ::opunk::TkConsole VALUE (try: lindex $con 0 -> class tag)
# - [::opunk::Console::in $con] is the input pipe's read end
# - [::opunk::Console::out $con] is the reflected channel -> widget
# - <Return> on the widget submits the current input line
# - destroying the widget tears the wiring down (eof to the repl)
# ---------------------------------------------------------------------------
set con [::opunk::console::tk::console $txt]
.bar.status configure -text "console object tag: [lindex $con 0] channels: [::opunk::Console::channels $con]"
#Live resize feedback: TkConsole's size override computes the ACTUAL character
#dimensions from the widget's current pixel size and font metrics whenever the
#widget is mapped (the requested -width/-height only describe the initial
#size). <Configure> fires on every resize, so the status bar tracks reality -
#and the same override is what the repl's repl::console_get_size (and the
#in-session punk::console::get_size bridge below) report.
bind $txt <Configure> {+after idle {catch {
.bar.status configure -text "resized - console size now: [::opunk::Console::size $::con]"
}}}
#Keep a transcript snapshot for the end-of-session report: the widget may be
#gone by then (autoclose or user close), and a <Destroy> binding is too late -
#when destruction cascades from the toplevel the widget command is already
#dead by the time its binding fires. Instead snapshot on every content change
#via the text widget's <<Modified>> virtual event (re-armed by resetting the
#modified flag). The '+' prefix APPENDS, preserving any existing bindings.
set ::final_transcript ""
bind $txt <<Modified>> {+
catch {set ::final_transcript [%W get 1.0 end-1c]}
catch {%W edit modified 0}
}
#Anything a holder writes to the console's out channel renders in the widget -
#the repl does exactly this internally (rputs/doprompt write repl::conout).
set outch [::opunk::Console::out $con]
puts $outch "=== tkconsole_demo: this banner was written to the console's out channel ==="
puts $outch "=== type Tcl at the P% prompt; 'exit' or closing the window ends it ==="
# ---------------------------------------------------------------------------
# Optional scripted session (-demo): programmatic typing via feed - each line
# is inserted into the widget's input area and submitted exactly as the
# <Return> binding would. Staggered with 'after' so you can watch each
# command run; the repl services these timer events from inside repl::start.
# ---------------------------------------------------------------------------
#helper for the -demo resize step: grow the toplevel by a fixed pixel amount -
#the <Configure> binding updates the status bar and the next size query shows
#the console's reported dimensions following the window
proc ::developer::tkconsole_demo::grow_window {} {
catch {wm geometry . [expr {[winfo width .] + 240}]x[expr {[winfo height .] + 96}]}
}
if {$opt_demo} {
set delay 1500
foreach step {
{feed {set demo_x 7}}
{feed {expr {$demo_x * 6}}}
{feed {puts stdout "hello from the code interp (diverted to the widget)"}}
{feed {puts stderr "stderr lands here too (conerr==conout pending G-011)"}}
{feed {punk::console::get_size}}
{grow}
{feed {punk::console::get_size}}
} {
lassign $step kind line
switch -- $kind {
feed {
after $delay [list ::opunk::console::tk::feed $txt $line]
}
grow {
#resize happens in THIS (repl/Tk) thread; the two surrounding
#get_size queries run in the code interp and report the size
#before and after - proving the whole chain tracks the window
after $delay ::developer::tkconsole_demo::grow_window
}
}
incr delay 1200
}
}
if {$opt_autoclose > 0} {
after $opt_autoclose {catch {destroy .}}
}
# ---------------------------------------------------------------------------
# [3]+[4] Select the console and run the repl. init spins up the codethread
# and its code interp (with the output-diverting junction stacks, because a
# foreign console is selected); start blocks servicing events until 'exit',
# 'quit', or console eof. NOTE: -type 0 (the plain 'punk' code interp) is the
# repltype wired for foreign-console output diversion.
# ---------------------------------------------------------------------------
repl::init -type 0 -console $con
# ---------------------------------------------------------------------------
# In-session size queries: make 'punk::console::get_size' typed at the P%
# prompt answer with THIS console's current size.
#
# Why this needs wiring at all: commands you type run in the CODE INTERP,
# which lives in the codethread - a different thread with no Tk and no access
# to this thread's channels or widget. A plain punk::console::get_size there
# would report the codethread's own default console (the process console -
# actively misleading inside a tk-console session).
#
# The bridge reuses the pattern the repl itself uses for its 'colour'/'mode'/
# 'vt52' code-interp aliases: an alias in the code interp targets a proc in
# the codethread's MAIN interp, which thread::sends the query back to this
# (repl) thread, where repl::console_get_size dispatches to the TkConsole
# size override - so it tracks live resizing just like the Query size button.
# The synchronous send is serviced because the repl runs 'update' while it
# waits for a command to complete (the same property the G-007 owner-routed
# queries rely on).
#
# NOTE this is a hand-rolled, single-proc preview of "scoped console state
# for subshells" - goal G-008 owns the general answer (ALL punk::console
# state/queries scoped to the selected console, not just get_size). It is
# installed by the DEMO, not by repl::init, precisely because the general
# design is still G-008's to make. (Without the bridge you could still reach
# the repl thread explicitly with: repl eval repl::console_get_size)
#
# Ordering: repl::init queued its codethread init_script asynchronously; this
# synchronous thread::send lands AFTER it in the codethread's event queue, so
# the code interp (and its punk::console package) exist by the time it runs.
# ---------------------------------------------------------------------------
thread::send $::repl::codethread [string map [list %replthread% [thread::id]] {
namespace eval ::tkconsole_demo_helpers {
proc console_get_size {args} {
#query the repl thread's selected console (the tk widget console)
thread::send %replthread% {repl::console_get_size}
}
}
interp alias code ::punk::console::get_size {} ::tkconsole_demo_helpers::console_get_size
}]
set done [repl::start]
# repl::start returned: report how the session ended. The window may already
# be gone (autoclose / user closed it - even the Tk application itself may be
# destroyed); the process stdout may or may not be visible depending on how we
# were launched - all reports are best-effort.
catch {puts stdout "tkconsole_demo: repl finished with: $done"}
if {![catch {$txt get 1.0 end-1c} live_transcript]} {
set ::final_transcript $live_transcript ;#widget still alive - freshest copy
}
catch {puts stdout "tkconsole_demo: final transcript:\n$::final_transcript"}
catch {destroy .}
exit 0

2
src/AGENTS.md

@ -64,6 +64,8 @@ Recovery after a wrong path guess:
- Use `tclsh src/make.tcl libs` to build just the library packages.
- Use `tclsh src/make.tcl packages` to build both modules and libraries.
- Use `tclsh src/make.tcl vendorupdate` to refresh vendormodules from config. It warns (non-fatal) for each source project whose fossil/git checkout is dirty - vendored artifacts built from a dirty tree have no committed provenance; commit in the source project to clear the warning (enforcement policy tracked by goal G-026).
- All build/promotion commands (`project`, `packages`, `modules`, `libs`, `vfs`, `vfslibs`, `bin`, `bootsupport`, `vfscommonupdate`) warn similarly when this project's own `src/` has uncommitted fossil/git changes (dirt outside `src/` is ignored). Warn-only by default; pass `-dirty-abort` to make the check aborting. For evaluating uncommitted source without a build, prefer `<builtexe> src` / `<builtexe> src shell`. The check is `punkboot::utils::vcs_dirty_warnings` (optional scope argument) loaded guardedly from bootsupport - if the snapshot is stale/missing the check degrades to a skip notice (but `-dirty-abort` then aborts rather than silently losing the requested strictness).
- Provenance warnings (dirty-src gate + vendorupdate source-project check) print with a plain column-0 `PROVENANCE-WARNING:` token (grep for it in captured build output) and are recapped at the end of the run via make.tcl's wrapped `::exit`. Interactive terminal runs get a 3-second ctrl-c grace countdown before a dirty build proceeds; piped/agent runs skip it. `tclsh src/make.tcl check` reports the current src provenance status and what the build commands would do.
- Use `tclsh src/make.tcl vfscommonupdate` to rebuild `_vfscommon.vfs`.
- Uncommitted `make.tcl`-generated outputs may be batched into one commit, regardless of how many `make.tcl` invocations produced them. This covers punkcheck-managed build outputs that are VCS-tracked: `src/bootsupport/`, `_vfscommon.vfs/modules` + `/lib`, and declared per-kit `*.vfs/lib_tcl<N>/<pkg>` subfolders. (Root `modules/`, `lib/`, `modules_tcl<N>/`, `lib_tcl<N>/` are gitignored and never committed.) Each module's old→new version rename is atomic; a regeneration reflects one build-output refresh, not independent hand-edits. This is a carve-out from generic "split aggressively" commit guidance for punkcheck-managed build outputs only — it does not cover user-curated VFS content (`_config/`, per-kit `*.vfs/` outside declared `lib_tcl<N>` subfolders, `_aside/`, `mkzipfix.vfs`, `_vfscommon.vfs/doc`), which commits separately per its own concerns.
- Use `punk make.tcl project` or `punk902z make.tcl project` inside Punk shell when building binaries through Punk.

105
src/project_layouts/custom/_project/punk.project-0.1/src/bootsupport/modules/punk/args/moduledoc/tclcore-0.1.0.tm → src/bootsupport/modules/punk/args/moduledoc/tclcore-0.2.0.tm

@ -8,7 +8,7 @@
# (C) 2025
#
# @@ Meta Begin
# Application punk::args::moduledoc::tclcore 0.1.0
# Application punk::args::moduledoc::tclcore 0.2.0
# Meta platform tcl
# Meta license MIT
# @@ Meta End
@ -18,7 +18,7 @@
# doctools header
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
#*** !doctools
#[manpage_begin punkshell_module_punk::args::moduledoc::tclcore 0 0.1.0]
#[manpage_begin punkshell_module_punk::args::moduledoc::tclcore 0 0.2.0]
#[copyright "2025"]
#[titledesc {punk::args definitions for tcl core commands}] [comment {-- Name section and table of contents description --}]
#[moddesc {tcl core argument definitions}] [comment {-- Description at end of page heading --}]
@ -9817,41 +9817,37 @@ tcl::namespace::eval punk::args::moduledoc::tclcore {
# ==============================================================================================================
punk::args::define [punk::args::lib::tstr -return string {
@id -id ::tcl::string::is
@cmd -name "Built-in: tcl::string::is"\
-summary\
"Test character class of string."\
-help\
"Returns 1 if string is a valid member of the specified character class, otherwise returns 0.
"
@leaders -min 1 -max 1
class -type string\
-choices {
alnum
alpha
ascii
boolean
control
dict
digit
double
entier
false
graph
integer
list
lower
print
punct
space
true
upper
wideinteger
wordchar
xdigit
}\
-choicelabels {
# -- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- ---
#G-054: the 'string is' class set is harvested from the RUNNING interpreter rather
#than hard-coded. It varies by tcl version (8.6 has no dict class; the unreleased
#8.7 series added unicode, which tcl 9 removed) and a static list breaks
#accept/reject parity between these docs and the interpreter they load into.
#A deliberately invalid probe of the pure builtin (safe, side-effect free) yields
#the authoritative list from its error message:
# bad class "zzz": must be alnum, alpha, ..., or xdigit
set string_is_classes [list]
if {[catch {string is __punk_argdoc_probe__ x} _sis_msg]} {
if {[regexp {must be (.+)$} $_sis_msg -> _sis_csv]} {
foreach _sis_c [split $_sis_csv ,] {
set _sis_c [string trim $_sis_c]
if {[string match "or *" $_sis_c]} {
set _sis_c [string range $_sis_c 3 end]
}
if {$_sis_c ne ""} {
lappend string_is_classes $_sis_c
}
}
}
}
if {![llength $string_is_classes]} {
#harvest failed (unexpected error message format) - fall back to the tcl 9.0 set
set string_is_classes {alnum alpha ascii boolean control dict digit double entier false graph integer list lower print punct space true upper wideinteger wordchar xdigit}
}
set string_is_classes [lsort $string_is_classes] ;#display order (as the previous hand-written list)
#hand-written class descriptions (man-page derived, verbatim) - applied below only for
#classes the running interpreter accepts; accepted classes without an entry get a
#generic label. tstr here resolves the ${$A_WARN}/${$A_RST} highlights as before.
set string_is_class_descriptions [punk::args::lib::tstr -return string {
alnum
" Any Unicode alphabet
or digit character"
@ -9877,7 +9873,9 @@ tcl::namespace::eval punk::args::moduledoc::tclcore {
will contain the index of
the \"element\" where the
dict parsing fails or -1 if
this cannot be determined."
this cannot be determined.
(class not present in
Tcl 8.6)"
digit
" Any Unicode digit char.
Note that this includes
@ -9937,6 +9935,11 @@ tcl::namespace::eval punk::args::moduledoc::tclcore {
" Any of the forms allowed
for Tcl_GetBoolean where the
value is true"
unicode
" Any Unicode character.
(class exists only in the
unreleased Tcl 8.7 series -
removed in Tcl 9)"
upper
" Any upper case alphabet
character in the Unicode
@ -9959,7 +9962,29 @@ tcl::namespace::eval punk::args::moduledoc::tclcore {
xdigit
" Any hexadecimal digit
character ([0-9A-Fa-f])."
}\
}]
set string_is_choicelabels ""
foreach _sis_c $string_is_classes {
if {[dict exists $string_is_class_descriptions $_sis_c]} {
append string_is_choicelabels [list $_sis_c] " " [list [dict get $string_is_class_descriptions $_sis_c]] \n
} else {
append string_is_choicelabels [list $_sis_c] " " [list " (class accepted by this Tcl\n runtime - not yet described\n in the punk tclcore docs)"] \n
}
}
unset -nocomplain _sis_msg _sis_csv _sis_c
punk::args::define [punk::args::lib::tstr -return string {
@id -id ::tcl::string::is
@cmd -name "Built-in: tcl::string::is"\
-summary\
"Test character class of string."\
-help\
"Returns 1 if string is a valid member of the specified character class, otherwise returns 0.
"
@leaders -min 1 -max 1
class -type string\
-choices {${$string_is_classes}}\
-choicelabels {${$string_is_choicelabels}}\
-help\
"character class
In the case of boolean, true and false, if the function will return 0, then the
@ -12518,7 +12543,7 @@ namespace eval ::punk::args::register {
package provide punk::args::moduledoc::tclcore [tcl::namespace::eval punk::args::moduledoc::tclcore {
variable pkg punk::args::moduledoc::tclcore
variable version
set version 0.1.0
set version 0.2.0
}]
return

44
src/project_layouts/custom/_project/punk.shell-0.1/src/bootsupport/modules/punk/lib-0.3.1.tm → src/bootsupport/modules/punk/lib-0.4.0.tm

@ -8,7 +8,7 @@
# (C) 2024
#
# @@ Meta Begin
# Application punk::lib 0.3.1
# Application punk::lib 0.4.0
# Meta platform tcl
# Meta license BSD
# @@ Meta End
@ -18,7 +18,7 @@
# doctools header
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
#*** !doctools
#[manpage_begin punkshell_module_punk::lib 0 0.3.1]
#[manpage_begin punkshell_module_punk::lib 0 0.4.0]
#[copyright "2024"]
#[titledesc {punk general utility functions}] [comment {-- Name section and table of contents description --}]
#[moddesc {punk library}] [comment {-- Description at end of page heading --}]
@ -8197,6 +8197,12 @@ namespace eval punk::lib {
is used, in which case punk::libunknown is sourced from the tm paths that were
already set up by the earlier part of the script.
When the kit boot captured runtime static/builtin packages
(::punkboot::static_packages - see punk_main.tcl), the script also copies
that baseline and seeds 'package ifneeded <name> <ver> {load {} <prefix>}'
entries so statically-linked packages (e.g Thread/twapi on a tcl-sfe
runtime) resolve in the target.
See also: punk::lib::interp_sync_package_paths"
@opts
-libunknown -type boolean -default 0 -help\
@ -8235,6 +8241,18 @@ namespace eval punk::lib {
append script "tcl::tm::add \{*\}\[list [lreverse $tmlist]\]\n"
append script "set ::auto_path \[list $ap\]\n"
append script "package prefer $prefer\n"
#G-058: propagate runtime static/builtin package mappings captured at kit boot
#(see punk_main.tcl) so 'package require' of a statically-linked package (e.g
#Thread/twapi on a tcl-sfe runtime) resolves in the new thread/interp even though
#no pkgIndex for it exists on the (replaced) package search paths.
if {[info exists ::punkboot::static_packages] && [dict size $::punkboot::static_packages]} {
append script "namespace eval ::punkboot {}\n"
append script [list set ::punkboot::static_prefixes $::punkboot::static_prefixes] \n
append script [list set ::punkboot::static_packages $::punkboot::static_packages] \n
dict for {pkgname pinfo} $::punkboot::static_packages {
append script "if \{\[package provide [list $pkgname]\] eq \"\"\} \{[list package ifneeded $pkgname [dict get $pinfo version] [list load {} [dict get $pinfo prefix]]]\}\n"
}
}
if {$do_libunknown} {
if {[info exists ::punk::libunknown::epoch]} {
append script "namespace eval ::punk::libunknown {}\n"
@ -8299,6 +8317,12 @@ namespace eval punk::lib {
package unknown handler and scan cache as the parent. This is recommended
when the child will do significant package loading.
When the kit boot captured runtime static/builtin packages
(::punkboot::static_packages - see punk_main.tcl), the child also receives
that baseline plus 'package ifneeded <name> <ver> {load {} <prefix>}'
seeding so statically-linked packages (e.g Thread/twapi on a tcl-sfe
runtime) resolve in the child.
This is an opt-in helper — call sites that don't use it keep their current
behavior. It does not load packages, install punk::packagepreference, share
channels, or configure safe-interp restrictions — those remain caller
@ -8340,6 +8364,20 @@ namespace eval punk::lib {
interp eval $interp {tcl::tm::remove {*}[tcl::tm::list]}
interp eval $interp [list tcl::tm::add {*}[lreverse [tcl::tm::list]]]
interp eval $interp [list package prefer [package prefer]]
#G-058: propagate runtime static/builtin package mappings captured at kit boot
#(see punk_main.tcl) so 'package require' of a statically-linked package (e.g
#Thread/twapi on a tcl-sfe runtime) resolves in the child interp even though no
#pkgIndex for it exists on the (replaced) package search paths.
if {[info exists ::punkboot::static_packages] && [dict size $::punkboot::static_packages]} {
interp eval $interp [list namespace eval ::punkboot {}]
interp eval $interp [list set ::punkboot::static_prefixes $::punkboot::static_prefixes]
interp eval $interp [list set ::punkboot::static_packages $::punkboot::static_packages]
dict for {pkgname pinfo} $::punkboot::static_packages {
if {[interp eval $interp [list package provide $pkgname]] eq ""} {
interp eval $interp [list package ifneeded $pkgname [dict get $pinfo version] [list load {} [dict get $pinfo prefix]]]
}
}
}
#extended: copy epoch + source libunknown + call init
if {$do_libunknown} {
if {[info exists ::punk::libunknown::epoch]} {
@ -9135,7 +9173,7 @@ namespace eval ::punk::args::register {
package provide punk::lib [tcl::namespace::eval punk::lib {
variable pkg punk::lib
variable version
set version 0.3.1
set version 0.4.0
}]
return

210
src/project_layouts/custom/_project/punk.shell-0.1/src/bootsupport/modules/punk/libunknown-0.1.tm → src/bootsupport/modules/punk/libunknown-0.2.0.tm

@ -7,17 +7,34 @@
# (C) 2025
#
# @@ Meta Begin
# Application punk::libunknown 0.1
# Application punk::libunknown 0.2.0
# Meta platform tcl
# Meta license MIT
# @@ Meta End
#
# Version history (manually versioned module - the real version lives in the
# filename and the 'package provide' block at the bottom; there is no
# <name>-buildversion.txt. Apply the standard Patch/Minor/Major bump rules
# from src/modules/AGENTS.md "Versioning And Releases" - bumping means
# renaming the file AND updating the Meta line above, the manpage_begin line
# below and the provide-block version, then appending a line here):
#0.2.0 - register_all_tm: deep discovery proc registering ifneeded scripts for
# .tm modules at every namespace depth (once per tm epoch, interp-local
# tm_fullscan guard); used by 'dev lib.search' by default
#0.2.0 - source_pkgindex: pkgIndex.tcl scripts execute in an isolated frame
# ($dir formal + auto_path/env global links) instead of at :: scope -
# user global 'dir' no longer clobbered, index helper vars no longer
# leak into the global namespace ('global dir' removed from
# zipfs_tclPkgUnknown)
#0.1 - initial: epoch-based zipfs_tm_UnknownHandler/zipfs_tclPkgUnknown
# package unknown chain, 'package epoch' command, controlled forget
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
# doctools header
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
#*** !doctools
#[manpage_begin punkshell_module_punk::libunknown 0 0.1]
#[manpage_begin punkshell_module_punk::libunknown 0 0.2.0]
#[copyright "2025"]
#[titledesc {Module API}] [comment {-- Name section and table of contents description --}]
#[moddesc {-}] [comment {-- Description at end of page heading --}]
@ -81,6 +98,10 @@ tcl::namespace::eval ::punk::libunknown {
}]
variable epoch ;#don't set - can be pre-set cooperatively
variable tm_fullscan [dict create] ;#tm epochs fully scanned by register_all_tm.
#Deliberately interp-local (NOT part of the shareable epoch dict): 'package ifneeded'
#registrations are interp-local, so an interp receiving a shared epoch must still run
#its own registration pass (cheap - directory listings come from the shared index cache).
variable has_package_files
if {[catch {package files foobaz}]} {
@ -100,6 +121,21 @@ tcl::namespace::eval ::punk::libunknown {
}
}
#Execute a pkgIndex.tcl script in this proc's frame (tcl_Pkg_source uplevels
#the actual 'source' into its caller).
#The pkgIndex.tcl contract is that $dir holds the index file's directory, and
#stock tclPkgUnknown additionally exposes the auto_path and env globals (some
#indexes, e.g tcllib's, extend auto_path with an unqualified lappend).
#Providing exactly that environment here means anything ELSE the script sets
#stays local to this frame and is discarded - previously indexes were sourced
#via 'namespace eval ::', which clobbered any user global named 'dir' (via the
#handler's since-removed 'global dir') and leaked each index's helper
#variables (ver, pkg, script, ...) into the global namespace.
proc source_pkgindex {dir indexfile} {
global auto_path env
tcl_Pkg_source $indexfile
}
namespace eval argdoc {
variable PUNKARGS
lappend PUNKARGS [list {
@ -492,6 +528,12 @@ tcl::namespace::eval ::punk::libunknown {
Key differences from Tcl's standard tclPkgUnknown:
- Uses an epoch-based cache to avoid re-sourcing pkgIndex.tcl files that have
already been processed in the current epoch.
- pkgIndex.tcl scripts execute in an isolated frame (see source_pkgindex)
providing the documented \$dir variable plus auto_path/env global links.
Stray unqualified variables set by index scripts stay local to that frame
instead of leaking into the global namespace, and user globals (notably
'dir') are not clobbered. Stock tclPkgUnknown instead exposes all of its
own proc locals to the index scripts.
- Processes auto_path entries from end to front (same as tclPkgUnknown), but
tracks which packages and versions were added or changed by each pkgIndex.tcl
so that ifneeded scripts from earlier (higher-priority) paths can be reverted
@ -514,7 +556,10 @@ tcl::namespace::eval ::punk::libunknown {
proc zipfs_tclPkgUnknown {name args} {
#puts "-> zipfs_tclPkgUnknown $name $args EXPERIMENTAL"
global dir
#Note: no 'global dir' here (an earlier revision had one so that pkgIndex.tcl
#scripts sourced at :: scope could read $dir - at the cost of clobbering any
#user global named 'dir'). Index scripts now execute in a source_pkgindex
#frame which provides $dir locally - see source_pkgindex.
variable epoch
set pkg_epoch [dict get $epoch pkg current]
@ -679,9 +724,8 @@ tcl::namespace::eval ::punk::libunknown {
# puts stderr "----->0 sourcing zipfs file $file"
#}
incr sourced ;#count as sourced even if source fails; keep before actual source action
#::tcl::Pkg::source $file
#lappend sourced_files $file
namespace eval :: [list ::punk::libunknown::tcl_Pkg_source $file]
source_pkgindex $dir $file
} trap {POSIX EACCES} {} {
# $file was not readable; silently ignore
puts stderr "zipfs_tclPkgUnknown file unreadable '$file' while trying to load $name (1)"
@ -711,8 +755,7 @@ tcl::namespace::eval ::punk::libunknown {
#puts "----->2 sourcing $file"
incr sourced
#lappend sourced_files $file
#::tcl::Pkg::source $file
namespace eval :: [list punk::libunknown::tcl_Pkg_source $file]
source_pkgindex $dir $file
} trap {POSIX EACCES} {} {
# $file was not readable; silently ignore
puts stderr "zipfs_tclPkgUnknown file unreadable '$file' while trying to load $name (2)"
@ -1195,6 +1238,157 @@ tcl::namespace::eval ::punk::libunknown {
}
}
namespace eval argdoc {
variable PUNKARGS
lappend PUNKARGS [list {
@id -id ::punk::libunknown::register_all_tm
@cmd -name punk::libunknown::register_all_tm\
-summary\
"Deep discovery: register ifneeded scripts for all .tm modules at every namespace depth."\
-help\
"Walks every path in tcl::tm::list recursively and registers a 'package ifneeded'
script for each .tm module found - including modules in namespace subfolders
that have never been requested. (The tm package unknown handler registers
sibling .tm files only at the namespace depth of the package being required,
so such modules are otherwise absent from 'package names' until first
requested.)
Existing ifneeded scripts are never overridden - first registration wins,
matching the tm unknown handler, so head-of-tm-list precedence for
same-version modules is preserved.
Uses and populates the same per-epoch directory index cache as the unknown
handlers: for zipfs (static) paths the whole tree listing is gathered in one
call; for filesystem paths each directory's *.tm glob is cached per
'package epoch'. Directories named #modpod-*, #tarjar-* and _build are
skipped.
The scan runs at most once per tm epoch per interp (tracked in the
interp-local tm_fullscan variable - deliberately not part of the shareable
epoch dict, because ifneeded registrations are interp-local; an interp
receiving a shared epoch re-registers cheaply from the cached indexes).
Path-list changes and 'package epoch incr' start a new tm epoch,
re-enabling the scan - a call after 'package epoch incr' re-reads the
filesystem and picks up modules added or removed on disk.
Returns a stats dict: epoch, roots, dirs, tmfiles, registered - plus
'cached 1' when the call was a per-epoch no-op.
punk::libunknown::init must have been called first."
@opts
-force -type none -help\
"Run the registration pass even if already performed in the current tm epoch.
Directory listings still come from the epoch index cache where present -
use 'package epoch incr' first to force re-reading the filesystem."
}]
}
proc register_all_tm {args} {
variable epoch
if {![info exists epoch]} {
error "punk::libunknown::register_all_tm - punk::libunknown::init has not been called in this interp"
}
set opt_force [expr {"-force" in $args}]
variable tm_fullscan
set tm_epoch [dict get $epoch tm current]
if {!$opt_force && [dict exists $tm_fullscan $tm_epoch]} {
return [dict merge [dict get $tm_fullscan $tm_epoch] [dict create cached 1]]
}
upvar ::tcl::tm::paths paths
upvar ::tcl::tm::pkgpattern pkgpattern
if {[info commands ::tcl::zipfs::root] ne ""} {
set zipfsroot [tcl::zipfs::root]
set has_zipfs 1
} else {
set zipfsroot "//zipfs:/" ;#doesn't matter much what we use here - don't expect in tm list if no zipfs commands
set has_zipfs 0
}
set stat_roots 0
set stat_dirs 0
set stat_files 0
set stat_registered 0
foreach path $paths {
if {![interp issafe] && ![file exists $path]} {
continue
}
incr stat_roots
set tmfiles [list]
if {$has_zipfs && [string match $zipfsroot* $path]} {
#static filesystem - the whole tm tree is available in one quick call
#(as the tm unknown handler's zipfs branch does)
set tmfiles [::tcl::zipfs::list $path/*.tm]
set seen_dirs [dict create]
foreach tm_path $tmfiles {
set d [file dirname $tm_path]
dict set seen_dirs $d 1
dict set epoch tm epochs $tm_epoch indexes $d $tm_path $tm_epoch
}
incr stat_dirs [dict size $seen_dirs]
} else {
#plain filesystem - breadth-first walk, reusing/populating the per-epoch
#directory index cache so the unknown handlers can short-circuit later
set pending [list $path]
while {[llength $pending]} {
set current [lindex $pending 0]
set pending [lrange $pending 1 end]
incr stat_dirs
if {[dict exists $epoch tm epochs $tm_epoch indexes $current]} {
set dirfiles [dict keys [dict get $epoch tm epochs $tm_epoch indexes $current]]
} else {
set dirfiles [glob -nocomplain -directory $current -types f *.tm]
dict set epoch tm epochs $tm_epoch indexes $current [dict create]
foreach f $dirfiles {
dict set epoch tm epochs $tm_epoch indexes $current $f $tm_epoch
}
}
lappend tmfiles {*}$dirfiles
foreach sub [glob -nocomplain -directory $current -types d *] {
set tail [file tail $sub]
if {[string match "#modpod-*" $tail] || [string match "#tarjar-*" $tail] || $tail eq "_build"} {
continue
}
lappend pending $sub
}
}
}
#registration - same rules as the tm unknown handler (don't override existing
#ifneeded scripts: for tm modules the first encountered 'wins')
set strip [llength [file split $path]]
foreach file $tmfiles {
if {[string match "*/_build/*" $file]} {
continue
}
incr stat_files
set pkgfilename [join [lrange [file split $file] $strip end] ::]
if {![regexp -- $pkgpattern $pkgfilename --> pkgname pkgversion]} {
# Ignore everything not matching our pattern for package names.
continue
}
try {
package vcompare $pkgversion 0
} on error {} {
# Ignore everything where the version part is not acceptable to
# "package vcompare".
continue
}
if {([package ifneeded $pkgname $pkgversion] ne {}) && (![interp issafe])} {
#already registered - possibly by an earlier (higher precedence) path
dict set epoch tm epochs $tm_epoch added $path $pkgname $pkgversion e$tm_epoch
dict unset epoch tm untracked $pkgname
continue
}
package ifneeded $pkgname $pkgversion \
"[::list package provide $pkgname $pkgversion];[::list source $file]"
incr stat_registered
dict set epoch tm epochs $tm_epoch added $path $pkgname $pkgversion e$tm_epoch
dict unset epoch tm untracked $pkgname
}
}
set stats [dict create epoch $tm_epoch roots $stat_roots dirs $stat_dirs tmfiles $stat_files registered $stat_registered]
dict set tm_fullscan $tm_epoch $stats
return $stats
}
#see what basic info we can gather *quickly* about the indexes for each version of a pkg that the package db knows about.
#we want no calls out to the actual filesystem - but we can use some 'file' calls such as 'file dirname', 'file split' (review -safe interp problem)
#in practice the info is only available for tm modules
@ -1925,7 +2119,7 @@ namespace eval ::punk::args::register {
package provide punk::libunknown [tcl::namespace::eval ::punk::libunknown {
variable pkg punk::libunknown
variable version
set version 0.1
set version 0.2.0
}]
return

93
src/bootsupport/modules/punk/mix/commandset/loadedlib-0.1.0.tm → src/bootsupport/modules/punk/mix/commandset/loadedlib-0.2.0.tm

@ -7,7 +7,7 @@
# (C) 2023
#
# @@ Meta Begin
# Application punk::mix::commandset::loadedlib 0.1.0
# Application punk::mix::commandset::loadedlib 0.2.0
# Meta platform tcl
# Meta license <unspecified>
# @@ Meta End
@ -28,13 +28,37 @@ namespace eval punk::mix::commandset::loadedlib {
#search automatically wrapped in * * - can contain inner * ? globs
punk::args::define {
@id -id ::punk::mix::commandset::loadedlib::search
@cmd -name "punk::mix::commandset::loadedlib search" -help "search all Tcl libraries available to your local interpreter"
@cmd -name "punk::mix::commandset::loadedlib search" -help\
"search all Tcl libraries available to your local interpreter.
When punk::libunknown is active (the punkshell default) a deep module
discovery pass runs first, so .tm modules at every namespace depth are
included - cached per 'package epoch', repeat searches are cheap.
See -refresh for cache/re-scan details."
-return -type string -default table -choices {table tableobject list lines}
-present -type integer -default 2 -choices {0 1 2} -choicelabels {absent present both} -help\
"(unimplemented) Display only those that are 0:absent 1:present 2:either"
-highlight -type boolean -default 1 -help\
"Highlight which version is present with ansi underline and colour"
-refresh -type none -help "Re-scan the tm and library folders"
-refresh -type none -help\
"Force a genuine filesystem re-scan of the module and library folders.
When punk::libunknown is active (the punkshell default), every search
already performs a deep module discovery pass - registering .tm modules
at every namespace depth across all tcl::tm::list paths (see
punk::libunknown::register_all_tm) - cached per 'package epoch' so
repeat searches are cheap. Because directories already indexed in the
current epoch are not re-globbed, that default pass does not notice .tm
files added to (or removed from) already-scanned folders.
-refresh increments the package epoch ('package epoch incr'),
invalidating the scan caches, then re-runs discovery - picking up
on-disk changes and re-sourcing the pkgIndex.tcl files on ::auto_path.
Changes to tcl::tm::list or ::auto_path increment the epoch
automatically (via variable traces) - this flag is not needed for
those cases.
When punk::libunknown is not active there is no epoch cache: the
default search reflects only packages already registered in the
package database, and -refresh performs the (comparatively expensive)
deep discovery walk directly using dummy package require calls at
each namespace depth."
searchstring -default * -multiple 1 -help\
"Names to search for, may contain glob chars (* ?) e.g *lib*
If no glob chars are explicitly specified, the searchstring will be wrapped with star globs.
@ -51,19 +75,47 @@ namespace eval punk::mix::commandset::loadedlib {
set opt_highlight [dict get $opts -highlight]
set opt_refresh [dict exists $received -refresh]
if {$opt_refresh} {
catch {package require frobznodule666} ;#ensure pkg system has loaded/searched for everything REVIEW - this doesn't result in full scans
foreach tm_path [tcl::tm::list] {
set paths_below [punk::path::subfolders -recursive $tm_path]
foreach folder $paths_below {
set tail [file tail $folder]
if {[string match #modpod-* $tail] || [string match #tarjar-* $tail]} {
continue
#Deep discovery of available packages.
#The package unknown handlers register .tm siblings only at the namespace depth
#being requested - modules in never-requested subfolders are absent from
#'package names' until a deep pass registers them.
#Note: ::info must be fully qualified here - this namespace defines its own 'info' proc
if {[::info commands ::punk::libunknown::register_all_tm] ne "" && [::info exists ::punk::libunknown::epoch]} {
if {$opt_refresh} {
#genuine filesystem re-scan: start a new package epoch so the scan
#caches are invalidated (pkgIndex.tcl files on ::auto_path will also be
#re-sourced by the dummy require below)
package epoch incr
}
#register .tm modules at every namespace depth (cached per tm epoch -
#repeat searches are cheap)
punk::libunknown::register_all_tm
#sweep the pkgIndex.tcl files of ::auto_path for library-style packages
catch {package require frobznodule666}
} else {
#punk::libunknown deep registration unavailable - without its epoch cache a
#recursive walk on every search would be expensive, so deep discovery runs
#only on explicit request.
if {$opt_refresh} {
if {[::info exists ::punk::libunknown::epoch]} {
#older punk::libunknown without register_all_tm: invalidate its scan
#caches so the dummy requires below re-read the filesystem
catch {package epoch incr}
}
catch {package require frobznodule666} ;#top-level tm scan + pkgIndex sweep of ::auto_path
package require punk::path
foreach tm_path [tcl::tm::list] {
set paths_below [punk::path::subfolders -recursive $tm_path]
foreach folder $paths_below {
set tail [file tail $folder]
if {[string match #modpod-* $tail] || [string match #tarjar-* $tail]} {
continue
}
if {[string match */_build/* $folder]} {continue}
set relpath [string tolower [punk::path::relative $tm_path $folder]]
set modpath [string map {/ ::} $relpath]
catch {package require ${modpath}::flobrudder99}
}
if {[string match */_build/* $folder]} {continue}
set relpath [string tolower [punk::path::relative $tm_path $folder]]
set modpath [string map {/ ::} $relpath]
catch {package require ${modpath}::flobrudder99}
}
}
}
@ -85,8 +137,12 @@ namespace eval punk::mix::commandset::loadedlib {
}
set matches [lsort -unique $matches]
set matchinfo [list]
set highlight_ansi [a+ web-limegreen underline]
set RST [a]
if {$opt_highlight} {
#not a module-top require: punk::ansi is only needed for highlighting
package require punk::ansi
set highlight_ansi [punk::ansi::a+ web-limegreen underline]
set RST [punk::ansi::a]
}
foreach m $matches {
set versions [package versions $m]
if {![llength $versions]} {
@ -122,6 +178,7 @@ namespace eval punk::mix::commandset::loadedlib {
return [join $matchinfo \n]
}
table - tableobject {
package require textblock
set t [textblock::class::table new]
$t add_column -headers "Package"
$t add_column -headers "Version"
@ -601,6 +658,6 @@ namespace eval punk::mix::commandset::loadedlib {
## Ready
package provide punk::mix::commandset::loadedlib [namespace eval punk::mix::commandset::loadedlib {
variable version
set version 0.1.0
set version 0.2.0
}]
return

31
src/project_layouts/custom/_project/punk.shell-0.1/src/bootsupport/modules/punk/packagepreference-0.1.1.tm → src/bootsupport/modules/punk/packagepreference-0.2.0.tm

@ -8,7 +8,7 @@
# (C) 2024
#
# @@ Meta Begin
# Application punk::packagepreference 0.1.1
# Application punk::packagepreference 0.2.0
# Meta platform tcl
# Meta license <unspecified>
# @@ Meta End
@ -18,7 +18,7 @@
# doctools header
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
#*** !doctools
#[manpage_begin punkshell_module_punk::packagepreference 0 0.1.1]
#[manpage_begin punkshell_module_punk::packagepreference 0 0.2.0]
#[copyright "2024"]
#[titledesc {punkshell package/module loading}] [comment {-- Name section and table of contents description --}]
#[moddesc {package/module load}] [comment {-- Description at end of page heading --}]
@ -171,6 +171,24 @@ tcl::namespace::eval punk::packagepreference {
return [uplevel 1 [list $COMMANDSTACKNEXT {*}$args]]
}
#G-058 static-vs-bundled policy for runtime static/builtin packages
#(baseline captured at kit boot into ::punkboot::static_packages - see punk_main.tcl).
#Ensure the static mapping exists in this interp (seeding is normally done at
#interp/thread creation - this covers stragglers), then trigger the package
#unknown index scan so any bundled vfs/module copies are ALSO registered before
#resolution. With all candidates registered, standard resolution compares
#versions (highest acceptable wins) - a static copy is not blindly masked by an
#older bundled dll, and a genuinely newer bundled copy remains reachable.
#(On identical versions the bundled copy's ifneeded entry, registered by the
# later scan, wins - same version either way.)
if {[info exists ::punkboot::static_packages] && [dict exists $::punkboot::static_packages $pkg]} {
set spinfo [dict get $::punkboot::static_packages $pkg]
if {[dict get $spinfo version] ni [$COMMANDSTACKNEXT_ORIGINAL versions $pkg]} {
$COMMANDSTACKNEXT_ORIGINAL ifneeded $pkg [dict get $spinfo version] [list load {} [dict get $spinfo prefix]]
}
catch {uplevel 1 [list {*}[$COMMANDSTACKNEXT_ORIGINAL unknown] $pkg]}
}
if {!$is_exact && [llength $vwant] <= 1 } {
#required version unspecified - or specified singularly
set available_versions [$COMMANDSTACKNEXT_ORIGINAL versions $pkg]
@ -179,7 +197,12 @@ tcl::namespace::eval punk::packagepreference {
#An attempt to detect dll/so loaded and try to load same version
#dll/so files are often named with version numbers that don't contain dots or a version number at all
#e.g sqlite3400.dll Thread288.dll
set pkgloadedinfo [lsearch -nocase -inline -index 1 [info loaded] $pkg]
#G-058: exclude empty-filename entries - those are static REGISTRATIONS
#(process-global, not necessarily loaded anywhere) and forcing their
#version here would defeat the version-aware static-vs-bundled policy
#above (e.g pinning to the static version even when a bundled copy is
#genuinely newer)
set pkgloadedinfo [lsearch -nocase -inline -index 1 [lsearch -all -inline -not -exact -index 0 [info loaded] {}] $pkg]
if {[llength $pkgloadedinfo]} {
if {[llength $available_versions] > 1} {
@ -489,7 +512,7 @@ namespace eval ::punk::args::register {
package provide punk::packagepreference [tcl::namespace::eval punk::packagepreference {
variable pkg punk::packagepreference
variable version
set version 0.1.1
set version 0.2.0
}]
return

288
src/vfs/_vfscommon.vfs/modules/punk/repl-0.3.0.tm → src/bootsupport/modules/punk/repl-0.4.0.tm

@ -157,6 +157,29 @@ namespace eval repl {
}
variable codethread_cond
# -- G-001 selected console --
#repl_console: the ::opunk::Console (subclass) object value selected via 'repl::init -console <spec>',
#or "" when the repl runs on the process-default console (legacy stdin/stdout/stderr behaviour).
#conin/conout/conerr: the channels this repl reads from and writes to. For a selected foreign console,
#conin/conout come from the console's in/out slots and conerr is the same channel as conout
#(a single terminal stream - optional per-console err channels are G-011's concern, not G-001's).
variable repl_console
if {![info exists repl_console]} {
set repl_console ""
}
variable conin
if {![info exists conin]} {
set conin stdin
}
variable conout
if {![info exists conout]} {
set conout stdout
}
variable conerr
if {![info exists conerr]} {
set conerr stderr
}
variable screen_last_chars "" ;#a small sliding append buffer for last char of any screen output to detect \n vs string
variable screen_last_char_list [list]
@ -473,11 +496,20 @@ proc punk::repl::get_prompt_config {} {
return [list resultprompt $resultprompt nlprompt $nlprompt infoprompt $infoprompt debugprompt $debugprompt]
}
proc repl::start {inchan args} {
proc repl::start {args} {
#debug only - we need clean output for 'exec' to succeed without forcing use of -ignorestderr.
#puts stderr "-->repl::start $inchan $args"
#puts stderr "-->repl::start $args"
#flush stderr
#G-001: inchan is optional - when omitted (or the args begin with an option) the input
#defaults to the console selected at repl::init (conin - process stdin when no -console given).
variable conin
if {[llength $args] && ![string match -* [lindex $args 0]]} {
set args [lassign $args inchan]
} else {
set inchan $conin
}
upvar ::punk::console::input_chunks_waiting input_chunks_waiting
if {![info exists input_chunks_waiting($inchan)]} {
set input_chunks_waiting($inchan) [list]
@ -564,7 +596,13 @@ proc repl::start {inchan args} {
# ---
if {$::punk::console::ansi_wanted == 2} {
if {[::punk::console::test_can_ansi]} {
variable repl_console
if {![console_is_default] && $repl_console ne ""} {
#foreign console: never emit a probe to the process console - settle from the
#console object's (possibly overridden) can_respond.
#(ansi_wanted is deliberately process-global - scoped console state is G-008)
set ::punk::console::ansi_wanted [expr {[::opunk::Console::can_respond $repl_console] ? 1 : -1}]
} elseif {[::punk::console::test_can_ansi]} {
set ::punk::console::ansi_wanted 1
} else {
set ::punk::console::ansi_wanted -1
@ -629,7 +667,9 @@ proc repl::start {inchan args} {
thread::cond destroy $codethread_cond ;#race if we destroy cond before child thread has exited - as it can send a -async quit
set codethread ""
set codethread_cond ""
punk::console::mode line ;#review - revert to line mode on final exit - but we may be exiting a nested repl
if {[console_is_default]} {
punk::console::mode line ;#review - revert to line mode on final exit - but we may be exiting a nested repl
}
set donevalue [set [namespace current]::done]
#e.g
# "eof stdin"
@ -845,13 +885,86 @@ proc repl::newout2 {} {
}
#--------------------------------------
# -- G-001 selected-console helpers --
namespace eval repl::argdoc {
lappend PUNKARGS [list {
@id -id ::repl::console_is_default
@cmd -name "repl::console_is_default"\
-summary\
"Whether this repl runs on the process-default console."\
-help\
"Returns 1 when no foreign console was selected at repl::init
(conin/conout are the process stdin/stdout), 0 when the repl
reads/writes a selected ::opunk::Console backend's channels.
Legacy process-console behaviours (prompt gating on
tcl_interactive, stdin reopen on eof, raw-mode re-enable)
apply only when this returns 1."
@values -min 0 -max 0
}]
}
proc repl::console_is_default {} {
variable conin
variable conout
expr {$conin eq "stdin" && $conout eq "stdout"}
}
namespace eval repl::argdoc {
lappend PUNKARGS [list {
@id -id ::repl::console_at_eof
@cmd -name "repl::console_at_eof"\
-summary\
"eof state of the repl input, answered by the selected console object when present."\
-help\
"For a repl started with a selected ::opunk::Console backend this dispatches
to the console object's (possibly overridden) at_eof method - e.g a tk-widget
backend answers from its backend eof marker rather than any channel state.
Otherwise (or for a non-console input channel) plain chan eof is used."
@values -min 1 -max 1
inputchan -type string -help\
"the repl input channel being read"
}]
}
proc repl::console_at_eof {inputchan} {
variable repl_console
variable conin
if {$repl_console ne "" && $inputchan eq $conin} {
return [::opunk::Console::at_eof $repl_console]
}
return [chan eof $inputchan]
}
namespace eval repl::argdoc {
lappend PUNKARGS [list {
@id -id ::repl::console_get_size
@cmd -name "repl::console_get_size"\
-summary\
"Console size for this repl - selected console object when present, else punk::console::get_size."\
-help\
"Returns a dict with keys columns and rows. A selected ::opunk::Console
backend answers via its (possibly overridden) size method; the default
console uses punk::console::get_size."
@values -min 0 -max 0
}]
}
proc repl::console_get_size {} {
variable repl_console
if {$repl_console ne ""} {
return [::opunk::Console::size $repl_console]
}
return [punk::console::get_size]
}
proc repl::doprompt {prompt {col {green bold}}} {
#prompt to stderr.
#prompt to stderr (conerr - same channel as conout for a selected foreign console).
#We can pipe commands into repl's stdin without the prompt interfering with the output.
#Although all command output for each line goes to stdout - not just what is emitted with puts
if {$::tcl_interactive} {
flush stdout; #we are writing this prompt on stderr, but stdout could still be writing to screen
variable conout
variable conerr
#G-001: a selected foreign console is a terminal by declaration (constructed knowing its role),
#so prompts are wanted regardless of the process-level tcl_interactive state.
if {$::tcl_interactive || ![console_is_default]} {
flush $conout; #we are writing this prompt on conerr, but conout could still be writing to screen
#our first char on stderr is based on the 'lastchar' of stdout which we have recorded but may not have arrived on screen.
#The issue we're trying to avoid is the (stderr)prompt arriving midway through a large stdout chunk
#REVIEW - this basic attempt to get stderr/stdout to cooperate is experimental and unlikely to achieve the desired effect in all situations
@ -895,9 +1008,9 @@ proc repl::doprompt {prompt {col {green bold}}} {
set o [a {*}$col]
set r [a]
puts -nonewline stderr $c$pre$o$prompt$r
puts -nonewline $conerr $c$pre$o$prompt$r
screen_last_char_add " " "prompt-stderr" prompt
flush stderr
flush $conerr
}
}
@ -910,11 +1023,13 @@ proc repl::rputs {args} {
variable screen_last_chars
variable last_out_was_newline
variable last_repl_char
variable conout
variable conerr
set pseudo_map [dict create {*}{
debug stderr
debugreport stderr
}]
#map pseudo-channels - and, for a selected foreign console (G-001), the std channel
#names - to this repl's console channels. Identity mapping for stdout/stderr when
#running on the process-default console.
set pseudo_map [dict create debug $conerr debugreport $conerr stdout $conout stderr $conerr]
if {[::tcl::mathop::<= 1 [llength $args] 3]} {
set out [lindex $args end]
@ -922,20 +1037,28 @@ proc repl::rputs {args} {
if {([llength $args] > 1) && [lindex $args 0] ne "-nonewline"} {
set this_tail \n
set rputschan [lindex $args 0]
#map pseudo-channels to real
if {$rputschan in [dict keys $pseudo_map]} {
#map pseudo/std channels to this repl's console channels
if {[dict exists $pseudo_map $rputschan]} {
lset args 0 [dict get $pseudo_map $rputschan]
}
} elseif {[llength $args] == 1} {
set this_tail \n
set rputschan "stdout"
#implicit stdout - make the channel explicit so a selected console receives it
set args [list $conout $out]
} else {
#>1 arg with -nonewline
#first arg is -nonewline
set this_tail [string index $out end]
set rputschan [lindex $args 1]
#map pseudo-channels to real
if {$rputschan in [dict keys $pseudo_map]} {
lset args 0 [dict get $pseudo_map $rputschan]
if {[llength $args] == 3} {
set rputschan [lindex $args 1]
#map pseudo/std channels to this repl's console channels
if {[dict exists $pseudo_map $rputschan]} {
lset args 1 [dict get $pseudo_map $rputschan]
}
} else {
#2 args: -nonewline <data> - implicit stdout
set rputschan "stdout"
set args [list -nonewline $conout $out]
}
}
set last_char_info_width 60
@ -961,7 +1084,7 @@ proc repl::rputs {args} {
#set x \ud83c\udf1e
#(2 surrogate pairs - treated as single char in tcl8 - fixed in 9 but won't/can't be backported) -
#see also: https://core.tcl-lang.org/tips/doc/trunk/tip/619.md
puts stderr "$repl_error"
puts $conerr "$repl_error"
}
} else {
#looks like an invalid puts call - use the normal error produced by the puts command
@ -980,7 +1103,7 @@ proc repl::rputs {args} {
} else {
set clear "\n"
}
puts -nonewline stderr "$clear[a red bold]! REPL ERROR IN rputs $c$err$n\n"
puts -nonewline $conerr "$clear[a red bold]! REPL ERROR IN rputs $c$err$n\n"
screen_last_char_add "\n" replerror "rputs err: '$err'"
return
} else {
@ -1898,7 +2021,7 @@ proc repl::repl_handler {inputchan readmore prompt_config} {
}
if {$readmore} {
if {![chan eof $inputchan]} {
if {![console_at_eof $inputchan]} {
##################################################################################
#Re-enable channel read handler only if no waiting chunks - must process in order
##################################################################################
@ -1941,7 +2064,9 @@ proc repl::repl_handler {inputchan readmore prompt_config} {
}
}
if {$::tcl_interactive} {
if {$::tcl_interactive && [console_is_default]} {
#process-console repl only: a foreign console's eof (e.g remote disconnect,
#widget teardown) must finish the repl, not reopen the process stdin.
rputs stderr "\nrepl_handler EOF inputchannel: [chan conf $inputchan]"
#rputs stderr "\n|repl> ctrl-c EOF on $inputchan."
after 1 [list repl::reopen_stdin]
@ -2135,6 +2260,8 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
# ---
variable reading
variable id_outstack
variable conout
variable conerr
#upvar ::punk::config::configdata configd
#set running_config [dict get $configd running]
@ -2296,10 +2423,10 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
set do_checkwidth 1 ;#make configurable if performance hit is too severe? TODO
set consolewidth 132
if {$do_checkwidth} {
if {[catch {set consolewidth [dict get [punk::console::get_size] columns]} errM]} {
if {[catch {set consolewidth [dict get [repl::console_get_size] columns]} errM]} {
#review
if {!$is_vt52} {
puts stderr "repl_process_data failed on call to punk::console::get_size :$errM"
puts stderr "repl_process_data failed on call to repl::console_get_size :$errM"
}
}
#if chan conf stdout doesn't give dimensions and console doesn't respond to queries - we can get empty results in get_size dict
@ -2420,7 +2547,7 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
} trap {POSIX} {e eopts} {
rputs stderr "trap1 POSIX '$e' eopts:'$eopts"
flush stderr
flush $conerr
} on error {repl_error erropts} {
rputs stderr "error1 in repl_process_data: $repl_error"
rputs stderr "-------------"
@ -2432,14 +2559,14 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
} else {
rputs stderr "*> $inputchan reader active"
}
if {[chan eof $inputchan]} {
if {[console_at_eof $inputchan]} {
rputs stderr "todo - attempt restart of repl on input channel: $inputchan in next loop"
catch {set ::punk::nav::ns::ns_current "::"}
#todo set flag to restart repl ?
} else {
rputs stderr "continuing.."
}
flush stderr
flush $conerr
}
@ -2499,7 +2626,9 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
}
set stdinconf [fconfigure $inputchan]
if {$::tcl_platform(platform) eq "windows" && [dict get $stdinconf -encoding] ni [list unicode utf-16 utf-8]} {
if {[console_is_default] && $::tcl_platform(platform) eq "windows" && [dict get $stdinconf -encoding] ni [list unicode utf-16 utf-8]} {
#(default process console only: a selected foreign console's channel encoding is the
# transport's business - re-decoding its lines as utf-16be would mangle them)
#some long console inputs are split weirdly when -encoding and -translation are left at defaults - requiring extra enter-key to get repl to process.
#experiment to see if using iso8859-1 (raw bytes) and handling line endings manually gives insight.
# - do: chan conf stdin -encoding iso859-1 -translation lf
@ -2695,6 +2824,29 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
set raw_result [tsv::get codethread_$codethread result]
lassign [tsv::get codethread_$codethread info] _o lastoutchar_codethread _e lasterrchar_codethread
if {![console_is_default]} {
#G-001 foreign-console repl: code-interp std-channel writes were diverted into
#pending vars by the junction stacks installed at init - collect and emit them
#to the selected console's channels (rputs maps stdout/stderr to conout/conerr).
#Interleaving between the two streams within a run is not preserved (single
#terminal stream; per-console err semantics are G-011).
set pending [thread::send $codethread {
interp eval code {
set _pendinglist [list $::codeinterp::console_pending_out $::codeinterp::console_pending_err]
set ::codeinterp::console_pending_out ""
set ::codeinterp::console_pending_err ""
set _pendinglist
}
}]
lassign $pending pending_out pending_err
if {$pending_out ne ""} {
rputs -nonewline stdout $pending_out
}
if {$pending_err ne ""} {
rputs -nonewline stderr $pending_err
}
}
#set status [catch {
# thread::send $
# uplevel 1 {namespace inscope $::punk::nav::ns::ns_current $run_command_string}
@ -2711,8 +2863,8 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
#puts stderr "-->>$result<--"
#===============================================================================
flush stdout
flush stderr
flush $conout
flush $conerr
#foreach s [lreverse $outstack] {
# shellfilter::stack::remove stdout $s
#}
@ -3022,8 +3174,10 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
#----------------------------------------------------------------------------
#after any external command - raw mode as the console sees it can be disabled
#set it to match current state of the tsv
#(process-default console only: the tsv raw state and enableRaw target the
# process console, not a selected foreign console - scoped state is G-008)
#----------------------------------------------------------------------------
if {[tsv::get punk_console is_raw]} {
if {[console_is_default] && [tsv::get punk_console is_raw]} {
if {$::tcl_platform(platform) eq "windows"} {
#review
#we are in parent process - twapi might not be loaded here - even if it is in the code interp
@ -3057,14 +3211,14 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
set ::punk::repl::signal_control_c 0
chan event $inputchan readable {}
rputs stderr "* console_control: control-c"
flush stderr
flush $conerr
set c [a yellow bold]
set n [a]
rputs stderr "${c}repl interrupted$n"
#set commandstr [list error "repl interrupted"]
set commandstr ""
doprompt ">_ "
flush stdout
flush $conout
} else {
# parse and determine outermost unclosed quote/bracket and include in prompt
@ -3108,7 +3262,7 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
} trap {POSIX} {e eopts} {
rputs stderr "trap POSIX '$e' eopts:'$eopts"
flush stderr
flush $conerr
} on error {repl_error erropts} {
rputs stderr "error2 in repl_process_data: $repl_error"
rputs stderr "-------------"
@ -3120,14 +3274,14 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
} else {
rputs stderr "*> $inputchan reader active"
}
if {[chan eof $inputchan]} {
if {[console_at_eof $inputchan]} {
rputs stderr "todo - attempt restart of repl on input channel: $inputchan in next loop"
catch {set ::punk::nav::ns::ns_current "::"}
#todo set flag to restart repl ?
} else {
rputs stderr "continuing.."
}
flush stderr
flush $conerr
}
}
@ -3194,10 +3348,10 @@ namespace eval repl {
variable codethread_cond
variable codethread_mutex
set opts [list -force 0 -type 0 -safelog 0 -paths {} -callback_interp $default_callback_interp]
set opts [list -force 0 -type 0 -safelog 0 -paths {} -callback_interp $default_callback_interp -console {}]
foreach {k v} $args {
switch -- $k {
-force - -type - -safelog - -paths - -callback_interp {
-force - -type - -safelog - -paths - -callback_interp - -console {
dict set opts $k $v
}
default {
@ -3224,6 +3378,37 @@ namespace eval repl {
if {$codethread ne "" && !$opt_force && [thread::exists $codethread] } {
error "repl:init codethread: $codethread already exists. use -force 1 to override"
}
#G-001 console selection: a -console spec ({in out} pair, anchored opunk::console instance
#name, or ::opunk::Console object value) selects the console this repl reads/writes.
#The resolved object (when present) answers at_eof/size/can_respond via its (possibly
#overridden) methods - see repl::console_at_eof / repl::console_get_size.
set opt_console [dict get $opts -console]
variable repl_console
variable conin
variable conout
variable conerr
if {$opt_console ne ""} {
set cinfo [punk::console::console_spec_resolve $opt_console]
set repl_console [dict get $cinfo object]
set conin [dict get $cinfo in]
set conout [dict get $cinfo out]
if {$conin eq "stdin" && $conout eq "stdout"} {
#explicit selection of the process-default console - full legacy behaviour incl. stderr
set conerr stderr
} else {
set conerr $conout
}
} else {
set repl_console ""
set conin stdin
set conout stdout
set conerr stderr
}
#whether code-interp std-channel writes must be diverted and routed to the selected console
#instead of reaching the process std channels
set conredirect [expr {![console_is_default]}]
set codethread [thread::create -preserved]
#review - naming of the possibly 2 cond variables parent and child thread
set codethread_cond [thread::cond create] ;#repl::codethread_cond held by parent(repl) vs punk::repl::codethread::replthread_cond held by child(codethread)
@ -3243,6 +3428,7 @@ namespace eval repl {
} %packageprefer% [list [package prefer]] {*}{
} %staticprefixes% [list [expr {[info exists ::punkboot::static_prefixes] ? $::punkboot::static_prefixes : ""}]] {*}{
} %staticpackages% [list [expr {[info exists ::punkboot::static_packages] ? $::punkboot::static_packages : ""}]] {*}{
} %conredirect% [list $conredirect] {*}{
}
]
#scriptmap applied at end to satisfy silly editor highlighting.
@ -4125,6 +4311,21 @@ namespace eval repl {
package require punk
package require shellrun
package require shellfilter
if {%conredirect%} {
#G-001 foreign-console repl: code-interp writes to stdout/stderr must not
#reach the process std channels. The shellfilter 'var' transform is a
#junction (no pass-through): writes divert into pending vars which the
#parent repl thread collects after each runscript and emits to the
#selected console's channels. Added before the colour ansiwrap stacks so
#those still apply to the diverted data (writes traverse newest transform
#first, so a junction added earlier captures last).
namespace eval ::codeinterp {
variable console_pending_out ""
variable console_pending_err ""
}
shellfilter::stack::add stdout var -settings {-varname ::codeinterp::console_pending_out}
shellfilter::stack::add stderr var -settings {-varname ::codeinterp::console_pending_err}
}
#set running_config $::punk::config::running
apply {running_config {
if {[string length [dict get $running_config color_stderr]] && [punk::console::colour]} {
@ -4229,9 +4430,14 @@ namespace eval repl {
}
#init - don't auto init - require init with possible options e.g -type
}
namespace eval ::punk::args::register {
#use fully qualified so 8.6 doesn't find existing var in global namespace
lappend ::punk::args::register::NAMESPACES ::repl
}
package provide punk::repl [namespace eval punk::repl {
variable version
set version 0.3.0
set version 0.4.0
}]
#repl::start $program_read_stdin_pipe

54
src/vfs/_vfscommon.vfs/modules/punkboot/utils-0.1.1.tm → src/bootsupport/modules/punkboot/utils-0.2.0.tm

@ -7,7 +7,7 @@
# (C) 2023
#
# @@ Meta Begin
# Application punkboot::utils 0.1.1
# Application punkboot::utils 0.2.0
# Meta platform tcl
# Meta license BSD
# @@ Meta End
@ -134,6 +134,10 @@ namespace eval punkboot::utils {
used to report each VCS root at most once across calls.
label, if supplied, is prefixed into each warning to name
the calling operation (e.g. 'vendorupdate').
scope, if supplied, is a subpath relative to path; only
uncommitted changes under that subpath are counted (e.g.
scope 'src' warns about dirty src/ while ignoring dirt
elsewhere in the checkout).
Returns an empty list if the path is clean, unversioned,
missing, already reported, or state cannot be determined."
@ -144,9 +148,11 @@ namespace eval punkboot::utils {
"Name of a dict variable in the caller's scope for dedupe across calls"
label -type string -optional 1 -default "" -help\
"Operation name prefixed into warnings"
scope -type string -optional 1 -default "" -help\
"Subpath relative to path limiting which changes count; empty = whole checkout"
}]
}
proc vcs_dirty_warnings {path checkedrootsvar {label ""}} {
proc vcs_dirty_warnings {path checkedrootsvar {label ""} {scope ""}} {
upvar 1 $checkedrootsvar checkedroots
if {![info exists checkedroots]} {set checkedroots [dict create]}
if {$label ne ""} {set label "$label "}
@ -155,6 +161,12 @@ namespace eval punkboot::utils {
if {![file isdirectory $dir]} {
return $warnings ;#missing path - not this proc's business to report
}
set scopeabs ""
set scopedesc "" ;#included in warning text when scoped
if {$scope ne ""} {
set scopeabs [file normalize [file join $dir $scope]]
set scopedesc " under [string trimright $scope /]/"
}
set fossilroot ""
set gitroot ""
while {1} {
@ -171,8 +183,8 @@ namespace eval punkboot::utils {
if {$parent eq $dir} { break }
set dir $parent
}
if {$fossilroot ne "" && ![dict exists $checkedroots fossil,$fossilroot] && [llength [auto_execok fossil]]} {
dict set checkedroots fossil,$fossilroot 1
if {$fossilroot ne "" && ![dict exists $checkedroots fossil,$fossilroot,$scope] && [llength [auto_execok fossil]]} {
dict set checkedroots fossil,$fossilroot,$scope 1
#fossil changes must run from within the checkout
set original_cwd [pwd]
if {[catch {
@ -180,19 +192,37 @@ namespace eval punkboot::utils {
set fchanges [string trim [exec {*}[auto_execok fossil] changes]]
} errM]} {
lappend warnings "WARNING: ${label}could not determine fossil state of source project at $fossilroot ($errM)"
} elseif {$fchanges ne ""} {
lappend warnings "WARNING: ${label}source project at $fossilroot has uncommitted fossil changes ([llength [split $fchanges \n]] file(s)) - artifacts built from a dirty tree have no committed provenance"
} else {
set flines [list]
foreach line [split $fchanges \n] {
set line [string trim $line]
if {$line eq ""} {continue}
if {$scopeabs ne ""} {
#fossil changes lines are '<STATUS> <path-relative-to-checkout-root>'
if {![regexp {^\S+\s+(.*)$} $line _ relfile]} {continue}
set normfile [file normalize [file join $fossilroot $relfile]]
if {$normfile ne $scopeabs && ![string match "${scopeabs}/*" $normfile]} {continue}
}
lappend flines $line
}
if {[llength $flines]} {
lappend warnings "WARNING: ${label}source project at $fossilroot has uncommitted fossil changes${scopedesc} ([llength $flines] file(s)) - artifacts built from a dirty tree have no committed provenance"
}
}
cd $original_cwd
}
if {$gitroot ne "" && ![dict exists $checkedroots git,$gitroot] && [llength [auto_execok git]]} {
dict set checkedroots git,$gitroot 1
if {$gitroot ne "" && ![dict exists $checkedroots git,$gitroot,$scope] && [llength [auto_execok git]]} {
dict set checkedroots git,$gitroot,$scope 1
set gitpathargs [list]
if {$scopeabs ne ""} {
set gitpathargs [list -- $scopeabs]
}
if {[catch {
set gchanges [string trim [exec {*}[auto_execok git] -C $gitroot status --porcelain]]
set gchanges [string trim [exec {*}[auto_execok git] -C $gitroot status --porcelain {*}$gitpathargs]]
} errM]} {
lappend warnings "WARNING: ${label}could not determine git state of source project at $gitroot ($errM)"
} elseif {$gchanges ne ""} {
lappend warnings "WARNING: ${label}source project at $gitroot has uncommitted git changes ([llength [split $gchanges \n]] file(s)) - artifacts built from a dirty tree have no committed provenance"
lappend warnings "WARNING: ${label}source project at $gitroot has uncommitted git changes${scopedesc} ([llength [split $gchanges \n]] file(s)) - artifacts built from a dirty tree have no committed provenance"
}
}
return $warnings
@ -207,8 +237,8 @@ namespace eval ::punk::args::register {
## Ready
package provide punkboot::utils [tcl::namespace::eval punkboot::utils {
variable version
#- this version number, exactly 0.1.1, is a literal used in src module folders
#- this version number, exactly 0.2.0, is a literal used in src module folders
#- we refer to this sometimes as the magic version number
set version 0.1.1
set version 0.2.0
}]
return

19
src/project_layouts/custom/_project/punk.shell-0.1/src/bootsupport/modules/shellthread-1.6.3.tm → src/bootsupport/modules/shellthread-1.7.0.tm

@ -617,7 +617,9 @@ namespace eval shellthread::manager {
#set ts_start [::shellthread::iso8601]
set tidworker [thread::create -preserved]
set init_script [string map [list %ts_start% $ts_start %mp% [tcl::tm::list] %ap% $::auto_path %tidcli% $tidclient %sd% $settingsdict] {
set static_prefixes [expr {[info exists ::punkboot::static_prefixes] ? $::punkboot::static_prefixes : ""}]
set static_packages [expr {[info exists ::punkboot::static_packages] ? $::punkboot::static_packages : ""}]
set init_script [string map [list %ts_start% $ts_start %mp% [tcl::tm::list] %ap% $::auto_path %tidcli% $tidclient %sd% $settingsdict %spfx% [list $static_prefixes] %spkg% [list $static_packages]] {
#set tclbase [file dirname [file dirname [info nameofexecutable]]]
#set tcllib $tclbase/lib
#if {$tcllib ni $::auto_path} {
@ -642,6 +644,19 @@ namespace eval shellthread::manager {
set ::auto_path [dict get $::settingsinfo auto_path]
}
#G-058: runtime static/builtin package baseline captured at kit boot (punk_main.tcl)
#seed ifneeded mappings so statically-linked packages resolve in this worker
namespace eval ::punkboot {}
set ::punkboot::static_prefixes %spfx%
set ::punkboot::static_packages %spkg%
if {[dict size $::punkboot::static_packages]} {
dict for {pkgname pinfo} $::punkboot::static_packages {
if {[package provide $pkgname] eq ""} {
package ifneeded $pkgname [dict get $pinfo version] [list load {} [dict get $pinfo prefix]]
}
}
}
package require punk::packagepreference
punk::packagepreference::install
@ -973,7 +988,7 @@ namespace eval shellthread::manager {
package provide shellthread [namespace eval shellthread {
variable version
set version 1.6.3
set version 1.7.0
}]

2
src/lib/app-punkscript/punkscript.tcl

@ -1,7 +1,7 @@
package provide app-punkscript 1.0
#Lean one-shot script runner for the punk executable 'script' subcommand (goal G-015).
#
#Contract (G-015 - see goals/G-015-script-subcommand-piped-stdin.md):
#Contract (G-015 - see goals/archive/G-015-script-subcommand-piped-stdin.md):
# - runs a script FILE (first argument, remaining arguments become the script's ::argv)
# or, with no arguments, the whole of piped/redirected stdin as the script.
# - the script interp carries the default punk shell module/alias environment

120
src/make.tcl

@ -20,7 +20,7 @@ namespace eval ::punkboot {
variable scriptfolder [file normalize [file dirname [info script]]]
variable foldername [file tail $scriptfolder]
variable pkg_requirements [list]; variable pkg_missing [list];variable pkg_loaded [list]
variable non_help_flags [list -k]
variable non_help_flags [list -k -dirty-abort]
variable help_flags [list -help --help /? -h]
variable known_commands [list project modules libs packages vfs vfslibs bin info check shell vendorupdate bootsupport vfscommonupdate projectversion]
}
@ -1364,6 +1364,13 @@ proc ::punkboot::punkboot_gethelp {args} {
append h " - run the punk shell using bootsupport libraries." \n
append h " $scriptname projectversion" \n
append h " - advisory check: verify CHANGELOG.md matches punkproject.toml and warn if src/ has changes since the last project-version bump." \n \n
append h " Flags:" \n
append h " -dirty-abort" \n
append h " - abort build/promotion commands (project packages modules libs vfs vfslibs bin bootsupport vfscommonupdate) when src/ has" \n
append h " uncommitted VCS changes. Default is warn-only: artifacts built from dirty src have no committed provenance." \n
append h " Warnings carry a plain PROVENANCE-WARNING: prefix (greppable in redirected output) and are recapped at the end of the run." \n
append h " Use '$scriptname check' to see the current provenance status. To evaluate uncommitted source without building," \n
append h " use '<builtexe> src' or '<builtexe> src shell'." \n \n
append h "" \n
if {[llength [dict get $pkg_availability missing]] || [llength [dict get $pkg_availability broken]]} {
set has_recommended 0
@ -1604,6 +1611,98 @@ if {![string length [set projectroot [punk::repo::find_project $scriptfolder]]]}
set sourcefolder $projectroot/src
set binfolder $projectroot/bin
# ----------------------------------------
# Provenance-warning presentation + dirty-src check for build/promotion commands (goal G-026 direction).
# Build/promotion commands stamp versions onto src content and propagate it into trees other
# things consume (root modules/, bootsupport snapshots, vfs payloads, kits/zipkits).
# Artifacts built from uncommitted src have no committed provenance - warn by default,
# abort if the -dirty-abort flag was given. To evaluate uncommitted source without
# building, use '<builtexe> src' / '<builtexe> src shell' instead.
# Guarded require: a stale or missing punkboot::utils bootsupport snapshot must never
# brick the make.tcl commands used to repair it - the check degrades to a skip notice
# (but -dirty-abort still aborts rather than silently skipping the requested strictness).
#Emit provenance warnings with a plain column-0 token for automated discovery (grep PROVENANCE-WARNING
#in redirected output) and ANSI colour for humans. Lines are also accumulated in
#::punkboot::provenance_warnings_pending unless -norecap is given, so the wrapped ::exit below can
#recap them at the tail of the output - build progress output is chatty and the original warning
#position may scroll out of sight or even out of scrollback.
set ::punkboot::provenance_warnings_pending [list]
proc ::punkboot::print_provenance_warnings {warninglines args} {
global A
if {![array size A]} {punkboot::define_global_ansi}
foreach w $warninglines {
regsub {^WARNING: } $w "" w
puts stderr "PROVENANCE-WARNING: $A(BAD)$w$A(RST)"
if {"-norecap" ni $args} {
lappend ::punkboot::provenance_warnings_pending $w
}
}
flush stderr
}
#Availability probe + scoped dirty-src check, shared by the build/promotion gate below and the
#'check' command report. Returns {available warninglist}.
proc ::punkboot::get_src_provenance_warnings {projectroot label} {
set available [expr {\
![catch {package require punkboot::utils}]\
&& [llength [info commands ::punkboot::utils::vcs_dirty_warnings]]\
&& "scope" in [info args ::punkboot::utils::vcs_dirty_warnings]\
}]
if {!$available} {
return [list 0 [list]]
}
set checked_vcs_roots [dict create]
return [list 1 [::punkboot::utils::vcs_dirty_warnings $projectroot checked_vcs_roots $label src]]
}
#Wrap ::exit so accumulated provenance warnings are recapped at the very end of output no matter
#which of make.tcl's many exit points a command takes.
if {![llength [info commands ::punkboot::exit_original]]} {
rename ::exit ::punkboot::exit_original
proc ::exit {{returnCode 0}} {
if {[info exists ::punkboot::provenance_warnings_pending] && [llength $::punkboot::provenance_warnings_pending]} {
puts stderr ""
puts stderr "PROVENANCE-WARNING: recap of warning(s) emitted earlier in this run:"
::punkboot::print_provenance_warnings $::punkboot::provenance_warnings_pending -norecap
}
::punkboot::exit_original $returnCode
}
}
if {$::punkboot::command in {project packages modules libs vfs vfslibs bin bootsupport vfscommonupdate}} {
set dirty_abort [expr {[lsearch $::argv -dirty-abort] >= 0}]
lassign [::punkboot::get_src_provenance_warnings $projectroot "make.tcl $::punkboot::command"] have_scoped_dirty_check dirty_warnings
if {$have_scoped_dirty_check} {
if {[llength $dirty_warnings]} {
::punkboot::print_provenance_warnings $dirty_warnings
puts stderr " To evaluate uncommitted source without building, use '<builtexe> src' or '<builtexe> src shell'."
if {$dirty_abort} {
set ::punkboot::provenance_warnings_pending [list] ;#no recap needed - aborting adjacent to the warning
puts stderr "-aborted- (-dirty-abort given and src has uncommitted changes - commit first, or rerun without -dirty-abort to build with a warning)"
exit 1
}
#Grace period for an attentive human to ctrl-c - only when stdin is a terminal
#(-inputmode is only supported on terminal channels, Tcl 8.7+/9; pipes/redirects/8.6 skip the delay
# so agent/CI runs pay nothing)
if {![catch {chan configure stdin -inputmode}]} {
puts -nonewline stderr " proceeding despite dirty src - ctrl-c now to abort "
flush stderr
foreach tick {3 2 1} {
puts -nonewline stderr "..$tick"
flush stderr
after 1000
}
puts stderr ""
}
}
} else {
if {$dirty_abort} {
puts stderr "-aborted- (-dirty-abort given but the dirty-src provenance check is unavailable: punkboot::utils vcs_dirty_warnings with scope support not loadable from bootsupport)"
exit 1
}
puts stderr "NOTE: dirty-src provenance check unavailable (punkboot::utils vcs_dirty_warnings with scope support not loadable from bootsupport) - continuing without it"
}
}
if {$::punkboot::command eq "check"} {
set sep [string repeat - 75]
puts stdout $sep
@ -1738,6 +1837,21 @@ if {$::punkboot::command eq "check"} {
puts stderr " cd \[projectroot\]/src && tclsh make.tcl modules && tclsh make.tcl bootsupport"
puts stderr "=============================================================================="
}
# Dirty-src provenance status - what the build/promotion commands would do
puts stdout $sep
lassign [::punkboot::get_src_provenance_warnings $projectroot "make.tcl check"] _prov_available _prov_warnings
if {!$_prov_available} {
puts stdout "src provenance: check unavailable (punkboot::utils vcs_dirty_warnings with scope support not loadable from bootsupport)"
puts stdout " build/promotion commands will proceed with a NOTE; -dirty-abort would abort as unverifiable."
} elseif {![llength $_prov_warnings]} {
puts stdout "src provenance: OK (no uncommitted fossil/git changes under src/)"
puts stdout " build/promotion commands (project packages modules libs vfs vfslibs bin bootsupport vfscommonupdate) will proceed without provenance warnings."
} else {
::punkboot::print_provenance_warnings $_prov_warnings -norecap
puts stdout " build/promotion commands (project packages modules libs vfs vfslibs bin bootsupport vfscommonupdate) will WARN as above and proceed."
puts stdout " With -dirty-abort they would abort. To evaluate uncommitted source without building, use '<builtexe> src' or '<builtexe> src shell'."
}
puts stdout $sep
exit 0
}
@ -2040,9 +2154,7 @@ if {$::punkboot::command eq "vendorupdate"} {
set normsrclocation [file normalize $srclocation]
if {![dict exists $checked_source_paths $normsrclocation]} {
dict set checked_source_paths $normsrclocation 1
foreach w [::punkboot::utils::vcs_dirty_warnings $normsrclocation checked_vcs_roots vendorupdate] {
puts stderr $w
}
::punkboot::print_provenance_warnings [::punkboot::utils::vcs_dirty_warnings $normsrclocation checked_vcs_roots vendorupdate]
}
}
#puts stdout "$relpath $module $module_subpath $srclocation"

4
src/modules/AGENTS.md

@ -15,7 +15,7 @@ Source of truth for all editable Punk project modules. This is where agents shou
- Module filenames use the literal suffix `-999999.0a1.0.tm`.
- Corresponding `<modulename>-buildversion.txt` files hold the real version number.
- The exception is `punk::libunknown`, which uses its own version `0.1`.
- The exception is `punk::libunknown`, which is manually versioned: its real `major.minor.patch` version lives in the filename (`libunknown-<version>.tm`) and there is no buildversion.txt. The same bump rules apply as for buildversion-tracked modules (see "Versioning And Releases"); the mechanics differ — see the manual-versioning bullet there.
- `#modpod-*` directories contain internal files packed into `.tm` archives during build; do not flatten or edit them without understanding the modpod format.
- `_build/` directory holds build intermediates and should not be manually edited.
- Always declare dependencies explicitly using `package require <name>` near file tops.
@ -321,7 +321,7 @@ return
This keeps the changelog discoverable alongside the version number without requiring a separate CHANGES file.
- **Bootstrap-tracked files only**`punkcheck-buildversion.txt`, `punk/repo-buildversion.txt`, `punk/mix-buildversion.txt` (the top-level `punk::mix` file only; `punk::mix::util`, `punk::mix::cli`, and `punk/mix/commandset/*` version independently and are not covered by this check), and `punk/tdl-buildversion.txt`: bump at least **minor** whenever a call site inside that file is updated to use a new API, even if the file's own interface is unchanged. `src/make.tcl` reads only these four files to classify bootsupport staleness (major=abort, minor=prompt, patch=silent-proceed); under-bumping one of them to patch when the call-site change is more significant causes `make.tcl` to mis-classify and either wrongly proceed without prompting or wrongly abort. See `src/bootsupport/AGENTS.md` "Bootsupport Staleness Handling" for the full contract.
- **All other modules** (including `punk::mix` submodules): call-site updates follow ordinary judgement from the Patch/Minor/Major rules above — a behavior-preserving call-site change is a patch (or no bump if genuinely a no-op); reserve minor for changes that add capability to the module's *own* API. `src/make.tcl` does not read these versions, so there is no automated consequence, but keep the changelog accurate since it is the only record of the module's real semantic version.
- There may be occasional exceptions such as `punk::libunknown-0.1.tm`, which is deliberately manually versioned; new modules should use the magic version mechanism.
- **Manually versioned modules** (currently only `punk::libunknown`): the real `major.minor.patch` version is the filename suffix and the `package provide` block value; there is no `<name>-buildversion.txt` and the build does not stamp a version. The Patch/Minor/Major bump rules above apply identically — an agent changing such a module bumps by (1) renaming the file (`git mv`) to the new version, (2) updating the `# Application <name> <version>` Meta line, the doctools `manpage_begin` version and the provide-block `set version`, and (3) appending a changelog comment line to the version-history block in the module header (it substitutes for the buildversion.txt changelog). Before the first bump of such a module, verify nothing requires it by exact version or hardcoded filename (for punk::libunknown: punk_main.tcl and punk::repl glob `libunknown-*.tm` and pick the highest by vcompare, bootsupport's include_modules.config lists it by name only, and all requires are unversioned - verified 2026-07-11). New modules should use the magic version mechanism instead.
- Modules with the magic version number must not appear in output paths such as `<projectdir>/modules`.
- When referencing ranges, use bounded specs such as `1.2.3-2.0.0`.
- Convert loose versions to bounded form in module metadata; helper utilities exist in boot modules for this purpose.

2
src/modules/opunk/AGENTS.md

@ -20,6 +20,8 @@ Modules under the `opunk::*` namespace explore value-based class implementations
- Query-based mechanisms enter the class the same way: `::opunk::console::size_query_provider` is a command prefix (invoked with the console object) the base `size` method consults after the -winsize fast path and capability gating; punk::console registers `console_size_provider` via `ensure_object_integration`. Non-channel subclasses override the relevant `-virtual` methods and never touch providers.
- Anchor lifecycle is observable the same way: `::opunk::console::lifecycle_callback` is a command prefix invoked as `{*}$cb created|forgotten <name> {in out}` after `create` anchors / `forget` releases an instance; punk::console registers its handler via `ensure_object_integration` to maintain the G-007 console ownership registry. Empty by default; callback errors propagate to the create/forget caller.
- Method-entry validity guards use `[llength $this] < N` (not `!= N`) so field-adding subclasses keep working.
- Subclassing (`voo::class <Child> -extends ::opunk::Console {...}`): the child inherits public accessors and the field INDEX variables, but not the parent's private `my.*` accessors - a subclass CONSTRUCTOR initialises inherited private fields via the index variables (`variable o_in` then `lset obj $o_in $value` - the sanctioned exception to the avoid-`variable o_<field>` rule, which still applies in method bodies where the parent's public accessor METHODS should be used instead, e.g `[::opunk::Console::in $this]`). Overridden `-virtual` methods are plain `method` declarations in the child; base-class calls dispatch to them via the slot-0 tag.
- G-001 backend subclasses (console/ subfolder): `opunk::console::test` (`::opunk::TestConsole` - deterministic channel-pair test double: fixed size, no probes; the console seam for repl/editbuf characterization), `opunk::console::ssh` (`::opunk::SshConsole` - socket-carried terminal sessions: construction-time capability, chan-eof without probe, size via the ANSI size-query provider over the connection), `opunk::console::tk` (`::opunk::TkConsole` - Tk text widget as terminal: widget held in a subclass `o_widget` field which always answers size/eof/capability; in/out slots carry the widget path by default or channels via `-in`/`-out`; backend eof marker via `::opunk::console::tk::set_eof`; `::opunk::console::tk::console <widget>` wires a reflected output channel + Return-binding input pipe so a channel-driven repl runs against the widget; no Tk package require at load - only the wiring proc requires Tk). Tests: `src/tests/modules/opunk/console/testsuites/console/backends.test` (tk case gated behind env `PUNK_TEST_TK=1` - loading Tk into the shared runtests testinterp has event-loop side effects) and the repl-integration suite `src/tests/modules/punk/repl/testsuites/repl/consolebackends.test` (child-process drivers; tk case self-gates on Tk availability in the child).
- Module filenames use the `-999999.0a1.0.tm` magic suffix with `<modulename>-buildversion.txt` per parent conventions.
- `opunk::console` (with its dependency `voo`) is included in bootsupport via `src/bootsupport/modules/include_modules.config` for eventual boot/repl-startup console objects; after releasing a new version, rebuild with `make.tcl modules` then refresh the snapshot with `make.tcl bootsupport`.

111
src/modules/opunk/console/ssh-999999.0a1.0.tm

@ -0,0 +1,111 @@
# -*- tcl -*-
# Maintenance Instruction: leave the 999999.xxx.x as is and use punkshell 'dev make' or bin/punkmake to update from <pkg>-buildversion.txt
#
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
# (C) 2026
#
# @@ Meta Begin
# Application opunk::console::ssh 999999.0a1.0
# Meta platform tcl
# Meta license BSD
# @@ Meta End
package require Tcl 8.6-
package require voo
package require opunk::console
#opunk::SshConsole - an ::opunk::Console backend for terminal sessions carried over a
#socket-like channel pair (G-001) - the ssh-channel case: a remote user's terminal is on
#the other end of the connection, but the local channel is a plain socket exposing none
#of the signals terminal detection relies on (-inputmode/-mode, twapi handles, env hints).
#
#The backend is CONSTRUCTED knowing the far end is a terminal, so:
# - is_console_or_tty: always 1 (explicit construction-time knowledge, not detection)
# - can_respond: settled value if set, else 1 (the remote terminal answers ANSI
# queries carried over the connection)
# - at_eof: plain [chan eof] on the channel - deliberately NO base-class
# pipe-probe: probing a socket would consume a byte the protocol
# layer/reader needs (see goals/G-001 detail)
# - size: the registered ::opunk::console::size_query_provider when
# available (punk::console's ANSI cursor-report mechanisms operate
# over the channel pair like any other console), else the
# configured default size. No -winsize attempt (sockets have none).
#
#No edits to the base ::opunk::Console or punk::console are needed (G-001 acceptance).
voo::class ::opunk::SshConsole -extends ::opunk::Console {
public {
method is_console_or_tty {} {
return 1
}
method can_respond {} {
set settled [get.o_can_respond $this]
if {$settled != -1} {
return $settled
}
return 1
}
method at_eof {} {
set in [::opunk::Console::in $this]
if {[catch {chan eof $in} is_eof]} {
return 1 ;#closed/invalid channel - unusable
}
return $is_eof
}
method size {} {
if {![can_respond $this]} {
return [::opunk::Console::default_size $this]
}
if {$::opunk::console::size_query_provider ne ""} {
if {![catch {{*}$::opunk::console::size_query_provider $this} sized]} {
if {[dict size $sized] && [string is integer -strict [dict get $sized columns]] && [string is integer -strict [dict get $sized rows]]} {
return $sized
}
}
}
return [::opunk::Console::default_size $this]
}
}
constructor {in out} {
#in/out: the channel pair of the connection (a single socket channel may be
#passed for both). Parent fields are private - set via imported index variables.
variable o_in
variable o_out
set obj [new()]
lset obj $o_in $in
lset obj $o_out $out
return $obj
}
}
namespace eval ::opunk::SshConsole {
namespace ensemble create
}
tcl::namespace::eval ::opunk::console::ssh {
tcl::namespace::export {[a-z]*}
variable PUNKARGS
lappend PUNKARGS [list {
@id -id "(package)opunk::console::ssh"
@package -name "opunk::console::ssh" -help\
"voo class ::opunk::SshConsole - an ::opunk::Console backend for terminal sessions
carried over socket-like channels (e.g an ssh connection): constructed knowing the
far end is a terminal, eof via chan eof (no byte-consuming probe), size via the
registered ANSI size-query provider over the connection (G-001)."
}]
}
namespace eval ::punk::args::register {
#use fully qualified so 8.6 doesn't find existing var in global namespace
lappend ::punk::args::register::NAMESPACES ::opunk::console::ssh ::opunk::SshConsole
}
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
## Ready
package provide opunk::console::ssh [tcl::namespace::eval ::opunk::console::ssh {
variable pkg opunk::console::ssh
variable version
set version 999999.0a1.0
}]
return

4
src/modules/opunk/console/ssh-buildversion.txt

@ -0,0 +1,4 @@
0.1.0
#First line must be a semantic version number
#all other lines are ignored.
#0.1.0 - G-001: initial ::opunk::Console backend subclass

120
src/modules/opunk/console/test-999999.0a1.0.tm

@ -0,0 +1,120 @@
# -*- tcl -*-
# Maintenance Instruction: leave the 999999.xxx.x as is and use punkshell 'dev make' or bin/punkmake to update from <pkg>-buildversion.txt
#
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
# (C) 2026
#
# @@ Meta Begin
# Application opunk::console::test 999999.0a1.0
# Meta platform tcl
# Meta license BSD
# @@ Meta End
package require Tcl 8.6-
package require voo
package require opunk::console
#opunk::TestConsole - a scripted/deterministic ::opunk::Console backend (G-001).
#
#The first client of the pluggable-backend seam and the console TEST DOUBLE the repl
#characterization work needs (see goals/G-044 detail - class_editbuf and raw-mode
#behaviours are console-coupled and cannot be unit-tested against a live terminal).
#
#Wraps a caller-supplied channel pair (typically [chan pipe] ends or reflected channels
#owned by a test harness) and answers the capability/size questions DETERMINISTICALLY:
# - is_console_or_tty: always 1 (constructed knowing it plays the terminal role)
# - can_respond: settled value if set, else 1
# - at_eof: plain [chan eof] on the input channel - NEVER probes (a probe
# would consume a byte the test harness scripted)
# - size: the configured size - no -winsize, no query emission, no
# size_query_provider consultation. Set at construction
# (-columns/-rows) and stored in the inherited o_default_size.
#
#No edits to the base ::opunk::Console or punk::console are needed (G-001 acceptance):
#values carry this class's namespace tag at slot 0 and existing holders (e.g
#punk::console::console_spec_resolve callers) dispatch to these overrides virtually.
voo::class ::opunk::TestConsole -extends ::opunk::Console {
public {
method is_console_or_tty {} {
return 1
}
method can_respond {} {
set settled [get.o_can_respond $this]
if {$settled != -1} {
return $settled
}
return 1
}
method at_eof {} {
set in [::opunk::Console::in $this]
if {[catch {chan eof $in} is_eof]} {
return 1 ;#closed/invalid channel - unusable
}
return $is_eof
}
method size {} {
return [::opunk::Console::default_size $this]
}
}
constructor {in out args} {
#args: optional -columns <int> -rows <int> (default 80x24 from the base field).
#Parent fields are private, so their imported INDEX variables are used to set
#slots on the child-tagged default value (the voo -extends pattern for
#initialising inherited private fields).
variable o_in
variable o_out
variable o_default_size
set obj [new()]
lset obj $o_in $in
lset obj $o_out $out
set size [lindex $obj $o_default_size]
foreach {k v} $args {
switch -- $k {
-columns {
dict set size columns $v
}
-rows {
dict set size rows $v
}
default {
error "opunk::TestConsole::new unknown option '$k'. Known options: -columns -rows"
}
}
}
lset obj $o_default_size $size
return $obj
}
}
namespace eval ::opunk::TestConsole {
namespace ensemble create
}
tcl::namespace::eval ::opunk::console::test {
tcl::namespace::export {[a-z]*}
variable PUNKARGS
lappend PUNKARGS [list {
@id -id "(package)opunk::console::test"
@package -name "opunk::console::test" -help\
"voo class ::opunk::TestConsole - a scripted/deterministic ::opunk::Console backend
wrapping a caller-supplied channel pair (e.g chan pipe ends) with fixed capability
answers and a configured size. The console test double for harness-driven repl and
editbuf testing (G-001)."
}]
}
namespace eval ::punk::args::register {
#use fully qualified so 8.6 doesn't find existing var in global namespace
lappend ::punk::args::register::NAMESPACES ::opunk::console::test ::opunk::TestConsole
}
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
## Ready
package provide opunk::console::test [tcl::namespace::eval ::opunk::console::test {
variable pkg opunk::console::test
variable version
set version 999999.0a1.0
}]
return

4
src/modules/opunk/console/test-buildversion.txt

@ -0,0 +1,4 @@
0.1.0
#First line must be a semantic version number
#all other lines are ignored.
#0.1.0 - G-001: initial ::opunk::Console backend subclass

370
src/modules/opunk/console/tk-999999.0a1.0.tm

@ -0,0 +1,370 @@
# -*- tcl -*-
# Maintenance Instruction: leave the 999999.xxx.x as is and use punkshell 'dev make' or bin/punkmake to update from <pkg>-buildversion.txt
#
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
# (C) 2026
#
# @@ Meta Begin
# Application opunk::console::tk 999999.0a1.0
# Meta platform tcl
# Meta license BSD
# @@ Meta End
package require Tcl 8.6-
package require voo
package require opunk::console
#NOTE: deliberately NO 'package require Tk' here - the class definition is loadable
#without Tk; construction errors usefully if the widget doesn't exist. This keeps the
#backend requirable in non-GUI processes (introspection, docs) and leaves Tk loading
#policy to the application (punk runtimes load Tk as an extension - punkbin convention).
#Only ::opunk::console::tk::console (which wires live channels to a widget) requires Tk.
#opunk::TkConsole - an ::opunk::Console backend for a Tk text widget acting as a
#terminal (G-001). A widget is not a channel at all, so every channel-shaped base
#method is overridden - none of the channel/provider logic applies (the class comment
#in opunk/console anticipates exactly this subclass).
#
# - o_widget (subclass field) carries the WIDGET PATH; capability/size/eof questions
# are always answered from the widget
# - o_in/o_out carry CHANNELS when constructed with -in/-out (e.g the reflected/pipe
# pair made by ::opunk::console::tk::console - this is what lets a channel-driven
# repl read/write through the console); without them they default to the widget
# path (documented non-channel reuse of the inherited slots - holders calling
# in/out/channels then get the widget path and must know their backend, which is
# what is_console_or_tty/terminal_class are for)
# - is_console_or_tty: always 1 (constructed as a terminal)
# - can_respond: settled value if set, else 1 (the widget side can answer -
# whatever integrating layer renders to the widget also responds)
# - at_eof: a backend eof MARKER, not a channel eof: 1 when the widget no
# longer exists (winfo exists - requires Tk loaded) or when the
# integrating layer has flagged eof via ::opunk::console::tk::set_eof
# (e.g the user closed the console frame but the widget remains)
# - size: the widget's ACTUAL character dimensions when mapped (pixel
# size / font metrics - tracks live window resizing), else the
# requested -width/-height, falling back to the inherited
# default size if the widget can't be queried.
#
#No edits to the base ::opunk::Console or punk::console are needed (G-001 acceptance).
tcl::namespace::eval ::opunk::console::tk {
#backend eof markers keyed by widget path - set via set_eof, consulted by at_eof
variable eof_flags
if {![array exists eof_flags]} {
array set eof_flags {}
}
#widget-console wiring state keyed by widget path (see proc console)
variable wiring
if {![array exists wiring]} {
array set wiring {}
}
}
voo::class ::opunk::TkConsole -extends ::opunk::Console {
private {
string_t o_widget ""
}
public {
method widget {} {
my.get.o_widget $this
}
method is_console_or_tty {} {
return 1
}
method can_respond {} {
set settled [get.o_can_respond $this]
if {$settled != -1} {
return $settled
}
return 1
}
method at_eof {} {
set w [my.get.o_widget $this]
if {[info exists ::opunk::console::tk::eof_flags($w)] && $::opunk::console::tk::eof_flags($w)} {
return 1
}
if {[catch {winfo exists $w} exists]} {
#Tk not loaded (or not usable) in this interp - the widget cannot be
#driven from here; report eof rather than pretending liveness
return 1
}
return [expr {!$exists}]
}
method size {} {
#Size of the console the widget presents RIGHT NOW:
# 1. mapped widget: ACTUAL character dimensions computed from the current
# pixel size and font metrics - tracks live window resizing.
# ([$w cget -width/-height] only report the REQUESTED size, which does
# not change when the user resizes the window.)
# 2. unmapped (constructed but not yet displayed, or withdrawn): the
# requested -width/-height
# 3. the inherited default size when the widget can't be queried at all
set w [my.get.o_widget $this]
if {![catch {winfo ismapped $w} mapped] && $mapped} {
if {![catch {
set fnt [$w cget -font]
set charw [font measure $fnt 0]
set charh [font metrics $fnt -linespace]
set extra [expr {[$w cget -borderwidth] + [$w cget -highlightthickness]}]
set cols [expr {([winfo width $w] - 2*($extra + [$w cget -padx])) / $charw}]
set rows [expr {([winfo height $w] - 2*($extra + [$w cget -pady])) / $charh}]
}]} {
if {$cols > 0 && $rows > 0} {
return [dict create columns $cols rows $rows]
}
}
}
if {![catch {list [$w cget -width] [$w cget -height]} wh]} {
lassign $wh cols rows
if {[string is integer -strict $cols] && [string is integer -strict $rows] && $cols > 0 && $rows > 0} {
return [dict create columns $cols rows $rows]
}
}
return [::opunk::Console::default_size $this]
}
}
constructor {widget args} {
#widget: path of a Tk text widget playing the terminal role - stored in the
#subclass o_widget field (always the authority for size/eof/capability).
#args: optional -in <chan> -out <chan> channels for the inherited in/out slots
#(e.g from ::opunk::console::tk::console); default is the widget path in both
#slots (non-channel reuse). Parent fields are private - set via imported index
#variables (the voo -extends pattern for initialising inherited private fields).
variable o_in
variable o_out
variable o_terminal_class
variable o_widget
set obj [new()]
set in $widget
set out $widget
foreach {k v} $args {
switch -- $k {
-in {
set in $v
}
-out {
set out $v
}
default {
error "opunk::TkConsole::new unknown option '$k'. Known options: -in -out"
}
}
}
lset obj $o_in $in
lset obj $o_out $out
lset obj $o_terminal_class tk-text
lset obj $o_widget $widget
return $obj
}
}
namespace eval ::opunk::TkConsole {
namespace ensemble create
}
tcl::namespace::eval ::opunk::console::tk {
tcl::namespace::export {[a-z]*}
variable PUNKARGS
#flag (or clear) backend eof for a widget-backed console - e.g when the hosting
#frame/toplevel is being torn down but holders may still consult the console value
proc set_eof {widget {value 1}} {
variable eof_flags
set eof_flags($widget) [expr {bool($value)}]
return
}
#Minimal widget-console wiring (G-001): turn a Tk text widget into a live console a
#channel-driven repl can be launched against. Deliberately basic - a fuller virtual
#terminal (ansi rendering, line editing, scrollback control) is the punkwish
#textwidget exploration's territory, not this module's.
# - output: a reflected write channel; written text is rendered into the widget
# (ansi-stripped when punk::ansi is present, \r\n normalized to \n)
# - input: a chan pipe; the <Return> binding (or programmatic 'feed') submits the
# text typed after the last output position
# - <Destroy> on the widget flags backend eof and closes the input feed end, so a
# repl reading the input channel sees eof and finishes
proc console {widget} {
variable wiring
package require Tk
if {![winfo exists $widget]} {
error "opunk::console::tk::console - widget '$widget' does not exist"
}
if {[info exists wiring($widget)]} {
error "opunk::console::tk::console - widget '$widget' is already wired as a console"
}
lassign [chan pipe] in_rd in_wr
chan configure $in_wr -translation lf -encoding utf-8 -buffering none
chan configure $in_rd -translation lf -encoding utf-8
set out_ch [chan create write [list [namespace current]::out_handler $widget]]
chan configure $out_ch -translation lf -encoding utf-8 -buffering none
set wiring($widget) [dict create in_rd $in_rd in_wr $in_wr out_ch $out_ch]
$widget mark set conin_start end-1c
$widget mark gravity conin_start left
bind $widget <Return> "[list [namespace current]::submit $widget]; break"
bind $widget <Destroy> [list [namespace current]::teardown $widget]
return [::opunk::TkConsole::new $widget -in $in_rd -out $out_ch]
}
proc submit {widget} {
#submit the current input area (text after conin_start) as one line on the
#console's input channel - the <Return> binding and 'feed' both land here
variable wiring
if {![info exists wiring($widget)]} {
return
}
set line [$widget get conin_start end-1c]
$widget insert end \n
$widget mark set conin_start end-1c
$widget see end
catch {puts [dict get $wiring($widget) in_wr] $line}
return
}
proc feed {widget line} {
#programmatic typing: append the line to the widget's input area and submit it
#(automation/testing convenience - the interactive path is the <Return> binding)
$widget insert end $line
submit $widget
}
proc teardown {widget} {
#widget destroyed: flag backend eof and close the input feed end so a reader of
#the console's input channel sees eof. The output channel is left to its holder
#(out_handler no-ops once the widget is gone).
variable wiring
set_eof $widget
if {![info exists wiring($widget)]} {
return
}
set w $wiring($widget)
unset wiring($widget)
catch {close [dict get $w in_wr]}
return
}
proc out_handler {widget cmd args} {
#reflected-channel handler (chan create write) rendering console output into
#the text widget
switch -- $cmd {
initialize {
return {initialize finalize watch write}
}
finalize {
return
}
watch {
return
}
write {
lassign $args ch bytes
set text [encoding convertfrom utf-8 $bytes]
set text [string map [list \r\n \n] $text]
if {![catch {package present punk::ansi}]} {
set text [punk::ansi::ansistrip $text]
}
if {![catch {winfo exists $widget} exists] && $exists} {
$widget insert end $text
$widget mark set conin_start end-1c
$widget see end
}
return [string length $bytes]
}
}
}
lappend PUNKARGS [list {
@id -id "(package)opunk::console::tk"
@package -name "opunk::console::tk" -help\
"voo class ::opunk::TkConsole - an ::opunk::Console backend wrapping a Tk text
widget acting as a terminal: size from the widget's character dimensions, eof via
a backend marker (::opunk::console::tk::set_eof) or widget destruction - a
non-channel subclass overriding all channel-shaped base methods (G-001).
::opunk::console::tk::console wires a widget with a reflected output channel and
an input pipe (Return-binding driven), so a channel-driven repl can be launched
against the widget (repl::init -console)."
}]
lappend PUNKARGS [list {
@id -id ::opunk::console::tk::set_eof
@cmd -name opunk::console::tk::set_eof\
-summary\
"Flag or clear backend eof for a widget-backed console."\
-help\
"Sets the backend eof marker consulted by ::opunk::TkConsole at_eof.
Use when the hosting frame/toplevel is being torn down (or reopened) while
holders may still consult console values wrapping the widget."
@leaders -min 1 -max 1
widget -type string -help\
"Tk widget path the console wraps"
@values -min 0 -max 1
value -type boolean -default 1 -optional 1 -help\
"1 to flag eof (default), 0 to clear"
}]
lappend PUNKARGS [list {
@id -id ::opunk::console::tk::console
@cmd -name opunk::console::tk::console\
-summary\
"Wire a Tk text widget as a live console and return a ::opunk::TkConsole for it."\
-help\
"Creates a reflected output channel rendering into the widget and an input
pipe fed by the widget's <Return> binding (or programmatic 'feed'), then
returns a ::opunk::TkConsole constructed with those channels (-in/-out).
The returned console value can be passed to 'repl::init -console' to run an
interactive repl through the widget. <Destroy> on the widget flags backend
eof and closes the input feed end. Requires Tk."
@leaders -min 1 -max 1
widget -type string -help\
"path of an existing Tk text widget"
}]
lappend PUNKARGS [list {
@id -id ::opunk::console::tk::submit
@cmd -name opunk::console::tk::submit\
-summary\
"Submit the widget's current input area as one line on the console input channel."\
-help\
"The text after the conin_start mark (i.e typed since the last output) is
written as one line to the console's input pipe. Bound to <Return> by
'console'; call directly (or via 'feed') for programmatic input."
@leaders -min 1 -max 1
widget -type string -help\
"Tk widget path wired by ::opunk::console::tk::console"
}]
lappend PUNKARGS [list {
@id -id ::opunk::console::tk::feed
@cmd -name opunk::console::tk::feed\
-summary\
"Programmatically type a line into a wired widget console and submit it."
@leaders -min 1 -max 1
widget -type string -help\
"Tk widget path wired by ::opunk::console::tk::console"
@values -min 1 -max 1
line -type string -help\
"the line to append to the input area and submit"
}]
lappend PUNKARGS [list {
@id -id ::opunk::console::tk::teardown
@cmd -name opunk::console::tk::teardown\
-summary\
"Release a widget's console wiring: flag eof and close the input feed end."\
-help\
"Bound to <Destroy> by 'console'; may be called directly to end a widget
console session while the widget lives on."
@leaders -min 1 -max 1
widget -type string -help\
"Tk widget path wired by ::opunk::console::tk::console"
}]
}
namespace eval ::punk::args::register {
#use fully qualified so 8.6 doesn't find existing var in global namespace
lappend ::punk::args::register::NAMESPACES ::opunk::console::tk ::opunk::TkConsole
}
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
## Ready
package provide opunk::console::tk [tcl::namespace::eval ::opunk::console::tk {
variable pkg opunk::console::tk
variable version
set version 999999.0a1.0
}]
return

6
src/modules/opunk/console/tk-buildversion.txt

@ -0,0 +1,6 @@
0.2.0
#First line must be a semantic version number
#all other lines are ignored.
#0.2.0 - size now reports the widget's ACTUAL character dimensions when mapped (current pixel size / font metrics - tracks live window resizing); the requested -width/-height remain the answer for unmapped widgets (construction-time/tests unchanged)
#0.2.0 - G-001: TkConsole gains a subclass o_widget field (widget is always the authority for size/eof/capability; new 'widget' accessor) and optional -in/-out constructor channels; new minimal widget-console wiring ::opunk::console::tk::console (reflected output channel rendering into the widget, input pipe fed by the <Return> binding or programmatic submit/feed, <Destroy> teardown) so a channel-driven repl can be launched against the widget via repl::init -console. Default construction (widget path in in/out slots) unchanged.
#0.1.0 - G-001: initial ::opunk::Console backend subclass

1
src/modules/punk/AGENTS.md

@ -28,6 +28,7 @@ Source of truth for all modules under the `punk::*` namespace. This is the prima
## Work Guidance
- New modules under `punk::*` should be created as `<subpath>/<modulename>-999999.0a1.0.tm` following the namespace-to-path convention.
- punk::repl supports launch-time console selection (G-001): `repl::init -console <spec>` (any `punk::console::console_spec_resolve` spec form) selects the console the repl reads/writes; `repl::start`'s input channel argument is optional and defaults to the selected console's input. Repl output flows through per-repl channel state (`repl::conin/conout/conerr` - rputs maps stdout/stderr per-repl, conerr==conout for a foreign console pending G-011), the code interp's stdout/stderr are diverted via shellfilter `var` junction stacks and emitted to the console after each run (repltype punk/0 only so far), and eof/size/capability questions go through `repl::console_at_eof`/`repl::console_get_size` which dispatch to the selected `::opunk::Console` object's (possibly overridden) methods. Process-console behaviours (tcl_interactive prompt gating, stdin reopen on eof, raw-mode re-enable) apply only when no foreign console is selected. Tests: `src/tests/modules/punk/repl/testsuites/repl/consolebackends.test` (child-process drivers - a repl cannot run inside the shared testinterp; see the suite header).
- punk::console uses the documented `-console` convention throughout: a `-console` value may be a 2-element {in out} channel list, an anchored `opunk::console` instance name, or an `::opunk::Console` object value (resolved via `punk::console::console_spec_resolve`). Query functions use the hybrid pattern (legacy trailing positional spec also accepted, parsed by `punk::console::internal::hybrid_console_spec`) - new query procs must follow it, with tests (see `src/tests/modules/punk/console/testsuites/console/queryprocs.test`). PUNKARGS definitions include the `-console` option via the shared fragments `::punk::console::argdoc::console_opts` (query/set functions) or `::punk::console::argdoc::console_emit_opts` (emit functions) rather than duplicating the option text; never re-add a `-minsize 2` constraint to `-console` (it rejects instance-name specs). The internal `get_size_using_*` size mechanisms deliberately remain canonical-pair positional (always fed by `get_size`).
- punk::console emit-side functions (the `punk::console::ansi::*` emit wrappers, mouse/paste toggles, `vt52`, `set_tabstop_width`, `titleset`, top-level `move`, and the width-test probes) accept an optional trailing `-console <consolespec>` pair, parsed manually for performance by `punk::console::internal::opt_console_out`/`opt_console_channels` (`_var` variants for procs whose args-tail also carries row/col/data triples). Each carries a documentation-only PUNKARGS definition that includes the shared `::punk::console::argdoc::console_emit_opts` fragment via `punk::args::resolved_def`; keep manual parsing and PUNKARGS synchronized. New emit procs must follow this pattern. Tests live in `src/tests/modules/punk/console/testsuites/console/emitconsole.test`.
- punk::console terminal-property facts (is_vt52, tabwidth, cell_size, last_da1_result, grapheme_cluster_support, check::has_bug_*) are per-console: read/write them via `punk::console::console_fact_get`/`console_fact_set`, keyed by canonical {in out} channel pair. The store is tsv-backed (G-007) so all threads read the same values: the process-default console `{stdin stdout}` keeps the legacy namespace variables (`::punk::console::is_vt52`, `tabwidth`, ...) as its authoritative local storage with write traces mirroring into tsv `punk_console_facts` (so existing external readers and direct writers keep working); non-default consoles store facts only in tsv with an owner-qualified key. Do not bypass the helpers for non-default consoles; use `console_fact_clear` (not direct store manipulation) to reset facts in tests. `ansi_wanted`/`colour_disabled` (string-generation gates), `ansi_available` and raw-mode state are deliberately process-global (rationale documented at the fact store in the module). Tests live in `src/tests/modules/punk/console/testsuites/console/consolefacts.test`.

99
src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm

@ -9817,41 +9817,37 @@ tcl::namespace::eval punk::args::moduledoc::tclcore {
# ==============================================================================================================
punk::args::define [punk::args::lib::tstr -return string {
@id -id ::tcl::string::is
@cmd -name "Built-in: tcl::string::is"\
-summary\
"Test character class of string."\
-help\
"Returns 1 if string is a valid member of the specified character class, otherwise returns 0.
"
@leaders -min 1 -max 1
class -type string\
-choices {
alnum
alpha
ascii
boolean
control
dict
digit
double
entier
false
graph
integer
list
lower
print
punct
space
true
upper
wideinteger
wordchar
xdigit
}\
-choicelabels {
# -- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- --- ---
#G-054: the 'string is' class set is harvested from the RUNNING interpreter rather
#than hard-coded. It varies by tcl version (8.6 has no dict class; the unreleased
#8.7 series added unicode, which tcl 9 removed) and a static list breaks
#accept/reject parity between these docs and the interpreter they load into.
#A deliberately invalid probe of the pure builtin (safe, side-effect free) yields
#the authoritative list from its error message:
# bad class "zzz": must be alnum, alpha, ..., or xdigit
set string_is_classes [list]
if {[catch {string is __punk_argdoc_probe__ x} _sis_msg]} {
if {[regexp {must be (.+)$} $_sis_msg -> _sis_csv]} {
foreach _sis_c [split $_sis_csv ,] {
set _sis_c [string trim $_sis_c]
if {[string match "or *" $_sis_c]} {
set _sis_c [string range $_sis_c 3 end]
}
if {$_sis_c ne ""} {
lappend string_is_classes $_sis_c
}
}
}
}
if {![llength $string_is_classes]} {
#harvest failed (unexpected error message format) - fall back to the tcl 9.0 set
set string_is_classes {alnum alpha ascii boolean control dict digit double entier false graph integer list lower print punct space true upper wideinteger wordchar xdigit}
}
set string_is_classes [lsort $string_is_classes] ;#display order (as the previous hand-written list)
#hand-written class descriptions (man-page derived, verbatim) - applied below only for
#classes the running interpreter accepts; accepted classes without an entry get a
#generic label. tstr here resolves the ${$A_WARN}/${$A_RST} highlights as before.
set string_is_class_descriptions [punk::args::lib::tstr -return string {
alnum
" Any Unicode alphabet
or digit character"
@ -9877,7 +9873,9 @@ tcl::namespace::eval punk::args::moduledoc::tclcore {
will contain the index of
the \"element\" where the
dict parsing fails or -1 if
this cannot be determined."
this cannot be determined.
(class not present in
Tcl 8.6)"
digit
" Any Unicode digit char.
Note that this includes
@ -9937,6 +9935,11 @@ tcl::namespace::eval punk::args::moduledoc::tclcore {
" Any of the forms allowed
for Tcl_GetBoolean where the
value is true"
unicode
" Any Unicode character.
(class exists only in the
unreleased Tcl 8.7 series -
removed in Tcl 9)"
upper
" Any upper case alphabet
character in the Unicode
@ -9959,7 +9962,29 @@ tcl::namespace::eval punk::args::moduledoc::tclcore {
xdigit
" Any hexadecimal digit
character ([0-9A-Fa-f])."
}\
}]
set string_is_choicelabels ""
foreach _sis_c $string_is_classes {
if {[dict exists $string_is_class_descriptions $_sis_c]} {
append string_is_choicelabels [list $_sis_c] " " [list [dict get $string_is_class_descriptions $_sis_c]] \n
} else {
append string_is_choicelabels [list $_sis_c] " " [list " (class accepted by this Tcl\n runtime - not yet described\n in the punk tclcore docs)"] \n
}
}
unset -nocomplain _sis_msg _sis_csv _sis_c
punk::args::define [punk::args::lib::tstr -return string {
@id -id ::tcl::string::is
@cmd -name "Built-in: tcl::string::is"\
-summary\
"Test character class of string."\
-help\
"Returns 1 if string is a valid member of the specified character class, otherwise returns 0.
"
@leaders -min 1 -max 1
class -type string\
-choices {${$string_is_classes}}\
-choicelabels {${$string_is_choicelabels}}\
-help\
"character class
In the case of boolean, true and false, if the function will return 0, then the

3
src/modules/punk/args/moduledoc/tclcore-buildversion.txt

@ -1,3 +1,4 @@
0.1.0
0.2.0
#First line must be a semantic version number
#all other lines are ignored.
#0.2.0 - G-054: 'string is' class choices (and the generated per-class virtual docids) are harvested from the RUNNING interpreter at define time instead of a hand-maintained list - a deliberately invalid probe of the builtin yields the authoritative class set from its error message (8.6: 21 classes, no dict; unreleased 8.7: +dict +unicode; 9.0: +dict, unicode removed), fixing accept/reject drift such as the doc wrongly accepting 'string is dict' under 8.6. Hand-written man-page descriptions apply only to classes the runtime accepts (generic label for unrecognized future classes); static version notes on dict (not in 8.6) and unicode (unreleased 8.7 only). Parity pinned by tclcoreparity.test with expectations derived from the live interpreter (green on 8.6.13, 8.7a6, 9.0.3)

210
src/modules/punk/libunknown-0.1.tm → src/modules/punk/libunknown-0.2.0.tm

@ -7,17 +7,34 @@
# (C) 2025
#
# @@ Meta Begin
# Application punk::libunknown 0.1
# Application punk::libunknown 0.2.0
# Meta platform tcl
# Meta license MIT
# @@ Meta End
#
# Version history (manually versioned module - the real version lives in the
# filename and the 'package provide' block at the bottom; there is no
# <name>-buildversion.txt. Apply the standard Patch/Minor/Major bump rules
# from src/modules/AGENTS.md "Versioning And Releases" - bumping means
# renaming the file AND updating the Meta line above, the manpage_begin line
# below and the provide-block version, then appending a line here):
#0.2.0 - register_all_tm: deep discovery proc registering ifneeded scripts for
# .tm modules at every namespace depth (once per tm epoch, interp-local
# tm_fullscan guard); used by 'dev lib.search' by default
#0.2.0 - source_pkgindex: pkgIndex.tcl scripts execute in an isolated frame
# ($dir formal + auto_path/env global links) instead of at :: scope -
# user global 'dir' no longer clobbered, index helper vars no longer
# leak into the global namespace ('global dir' removed from
# zipfs_tclPkgUnknown)
#0.1 - initial: epoch-based zipfs_tm_UnknownHandler/zipfs_tclPkgUnknown
# package unknown chain, 'package epoch' command, controlled forget
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
# doctools header
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
#*** !doctools
#[manpage_begin punkshell_module_punk::libunknown 0 0.1]
#[manpage_begin punkshell_module_punk::libunknown 0 0.2.0]
#[copyright "2025"]
#[titledesc {Module API}] [comment {-- Name section and table of contents description --}]
#[moddesc {-}] [comment {-- Description at end of page heading --}]
@ -81,6 +98,10 @@ tcl::namespace::eval ::punk::libunknown {
}]
variable epoch ;#don't set - can be pre-set cooperatively
variable tm_fullscan [dict create] ;#tm epochs fully scanned by register_all_tm.
#Deliberately interp-local (NOT part of the shareable epoch dict): 'package ifneeded'
#registrations are interp-local, so an interp receiving a shared epoch must still run
#its own registration pass (cheap - directory listings come from the shared index cache).
variable has_package_files
if {[catch {package files foobaz}]} {
@ -100,6 +121,21 @@ tcl::namespace::eval ::punk::libunknown {
}
}
#Execute a pkgIndex.tcl script in this proc's frame (tcl_Pkg_source uplevels
#the actual 'source' into its caller).
#The pkgIndex.tcl contract is that $dir holds the index file's directory, and
#stock tclPkgUnknown additionally exposes the auto_path and env globals (some
#indexes, e.g tcllib's, extend auto_path with an unqualified lappend).
#Providing exactly that environment here means anything ELSE the script sets
#stays local to this frame and is discarded - previously indexes were sourced
#via 'namespace eval ::', which clobbered any user global named 'dir' (via the
#handler's since-removed 'global dir') and leaked each index's helper
#variables (ver, pkg, script, ...) into the global namespace.
proc source_pkgindex {dir indexfile} {
global auto_path env
tcl_Pkg_source $indexfile
}
namespace eval argdoc {
variable PUNKARGS
lappend PUNKARGS [list {
@ -492,6 +528,12 @@ tcl::namespace::eval ::punk::libunknown {
Key differences from Tcl's standard tclPkgUnknown:
- Uses an epoch-based cache to avoid re-sourcing pkgIndex.tcl files that have
already been processed in the current epoch.
- pkgIndex.tcl scripts execute in an isolated frame (see source_pkgindex)
providing the documented \$dir variable plus auto_path/env global links.
Stray unqualified variables set by index scripts stay local to that frame
instead of leaking into the global namespace, and user globals (notably
'dir') are not clobbered. Stock tclPkgUnknown instead exposes all of its
own proc locals to the index scripts.
- Processes auto_path entries from end to front (same as tclPkgUnknown), but
tracks which packages and versions were added or changed by each pkgIndex.tcl
so that ifneeded scripts from earlier (higher-priority) paths can be reverted
@ -514,7 +556,10 @@ tcl::namespace::eval ::punk::libunknown {
proc zipfs_tclPkgUnknown {name args} {
#puts "-> zipfs_tclPkgUnknown $name $args EXPERIMENTAL"
global dir
#Note: no 'global dir' here (an earlier revision had one so that pkgIndex.tcl
#scripts sourced at :: scope could read $dir - at the cost of clobbering any
#user global named 'dir'). Index scripts now execute in a source_pkgindex
#frame which provides $dir locally - see source_pkgindex.
variable epoch
set pkg_epoch [dict get $epoch pkg current]
@ -679,9 +724,8 @@ tcl::namespace::eval ::punk::libunknown {
# puts stderr "----->0 sourcing zipfs file $file"
#}
incr sourced ;#count as sourced even if source fails; keep before actual source action
#::tcl::Pkg::source $file
#lappend sourced_files $file
namespace eval :: [list ::punk::libunknown::tcl_Pkg_source $file]
source_pkgindex $dir $file
} trap {POSIX EACCES} {} {
# $file was not readable; silently ignore
puts stderr "zipfs_tclPkgUnknown file unreadable '$file' while trying to load $name (1)"
@ -711,8 +755,7 @@ tcl::namespace::eval ::punk::libunknown {
#puts "----->2 sourcing $file"
incr sourced
#lappend sourced_files $file
#::tcl::Pkg::source $file
namespace eval :: [list punk::libunknown::tcl_Pkg_source $file]
source_pkgindex $dir $file
} trap {POSIX EACCES} {} {
# $file was not readable; silently ignore
puts stderr "zipfs_tclPkgUnknown file unreadable '$file' while trying to load $name (2)"
@ -1195,6 +1238,157 @@ tcl::namespace::eval ::punk::libunknown {
}
}
namespace eval argdoc {
variable PUNKARGS
lappend PUNKARGS [list {
@id -id ::punk::libunknown::register_all_tm
@cmd -name punk::libunknown::register_all_tm\
-summary\
"Deep discovery: register ifneeded scripts for all .tm modules at every namespace depth."\
-help\
"Walks every path in tcl::tm::list recursively and registers a 'package ifneeded'
script for each .tm module found - including modules in namespace subfolders
that have never been requested. (The tm package unknown handler registers
sibling .tm files only at the namespace depth of the package being required,
so such modules are otherwise absent from 'package names' until first
requested.)
Existing ifneeded scripts are never overridden - first registration wins,
matching the tm unknown handler, so head-of-tm-list precedence for
same-version modules is preserved.
Uses and populates the same per-epoch directory index cache as the unknown
handlers: for zipfs (static) paths the whole tree listing is gathered in one
call; for filesystem paths each directory's *.tm glob is cached per
'package epoch'. Directories named #modpod-*, #tarjar-* and _build are
skipped.
The scan runs at most once per tm epoch per interp (tracked in the
interp-local tm_fullscan variable - deliberately not part of the shareable
epoch dict, because ifneeded registrations are interp-local; an interp
receiving a shared epoch re-registers cheaply from the cached indexes).
Path-list changes and 'package epoch incr' start a new tm epoch,
re-enabling the scan - a call after 'package epoch incr' re-reads the
filesystem and picks up modules added or removed on disk.
Returns a stats dict: epoch, roots, dirs, tmfiles, registered - plus
'cached 1' when the call was a per-epoch no-op.
punk::libunknown::init must have been called first."
@opts
-force -type none -help\
"Run the registration pass even if already performed in the current tm epoch.
Directory listings still come from the epoch index cache where present -
use 'package epoch incr' first to force re-reading the filesystem."
}]
}
proc register_all_tm {args} {
variable epoch
if {![info exists epoch]} {
error "punk::libunknown::register_all_tm - punk::libunknown::init has not been called in this interp"
}
set opt_force [expr {"-force" in $args}]
variable tm_fullscan
set tm_epoch [dict get $epoch tm current]
if {!$opt_force && [dict exists $tm_fullscan $tm_epoch]} {
return [dict merge [dict get $tm_fullscan $tm_epoch] [dict create cached 1]]
}
upvar ::tcl::tm::paths paths
upvar ::tcl::tm::pkgpattern pkgpattern
if {[info commands ::tcl::zipfs::root] ne ""} {
set zipfsroot [tcl::zipfs::root]
set has_zipfs 1
} else {
set zipfsroot "//zipfs:/" ;#doesn't matter much what we use here - don't expect in tm list if no zipfs commands
set has_zipfs 0
}
set stat_roots 0
set stat_dirs 0
set stat_files 0
set stat_registered 0
foreach path $paths {
if {![interp issafe] && ![file exists $path]} {
continue
}
incr stat_roots
set tmfiles [list]
if {$has_zipfs && [string match $zipfsroot* $path]} {
#static filesystem - the whole tm tree is available in one quick call
#(as the tm unknown handler's zipfs branch does)
set tmfiles [::tcl::zipfs::list $path/*.tm]
set seen_dirs [dict create]
foreach tm_path $tmfiles {
set d [file dirname $tm_path]
dict set seen_dirs $d 1
dict set epoch tm epochs $tm_epoch indexes $d $tm_path $tm_epoch
}
incr stat_dirs [dict size $seen_dirs]
} else {
#plain filesystem - breadth-first walk, reusing/populating the per-epoch
#directory index cache so the unknown handlers can short-circuit later
set pending [list $path]
while {[llength $pending]} {
set current [lindex $pending 0]
set pending [lrange $pending 1 end]
incr stat_dirs
if {[dict exists $epoch tm epochs $tm_epoch indexes $current]} {
set dirfiles [dict keys [dict get $epoch tm epochs $tm_epoch indexes $current]]
} else {
set dirfiles [glob -nocomplain -directory $current -types f *.tm]
dict set epoch tm epochs $tm_epoch indexes $current [dict create]
foreach f $dirfiles {
dict set epoch tm epochs $tm_epoch indexes $current $f $tm_epoch
}
}
lappend tmfiles {*}$dirfiles
foreach sub [glob -nocomplain -directory $current -types d *] {
set tail [file tail $sub]
if {[string match "#modpod-*" $tail] || [string match "#tarjar-*" $tail] || $tail eq "_build"} {
continue
}
lappend pending $sub
}
}
}
#registration - same rules as the tm unknown handler (don't override existing
#ifneeded scripts: for tm modules the first encountered 'wins')
set strip [llength [file split $path]]
foreach file $tmfiles {
if {[string match "*/_build/*" $file]} {
continue
}
incr stat_files
set pkgfilename [join [lrange [file split $file] $strip end] ::]
if {![regexp -- $pkgpattern $pkgfilename --> pkgname pkgversion]} {
# Ignore everything not matching our pattern for package names.
continue
}
try {
package vcompare $pkgversion 0
} on error {} {
# Ignore everything where the version part is not acceptable to
# "package vcompare".
continue
}
if {([package ifneeded $pkgname $pkgversion] ne {}) && (![interp issafe])} {
#already registered - possibly by an earlier (higher precedence) path
dict set epoch tm epochs $tm_epoch added $path $pkgname $pkgversion e$tm_epoch
dict unset epoch tm untracked $pkgname
continue
}
package ifneeded $pkgname $pkgversion \
"[::list package provide $pkgname $pkgversion];[::list source $file]"
incr stat_registered
dict set epoch tm epochs $tm_epoch added $path $pkgname $pkgversion e$tm_epoch
dict unset epoch tm untracked $pkgname
}
}
set stats [dict create epoch $tm_epoch roots $stat_roots dirs $stat_dirs tmfiles $stat_files registered $stat_registered]
dict set tm_fullscan $tm_epoch $stats
return $stats
}
#see what basic info we can gather *quickly* about the indexes for each version of a pkg that the package db knows about.
#we want no calls out to the actual filesystem - but we can use some 'file' calls such as 'file dirname', 'file split' (review -safe interp problem)
#in practice the info is only available for tm modules
@ -1925,7 +2119,7 @@ namespace eval ::punk::args::register {
package provide punk::libunknown [tcl::namespace::eval ::punk::libunknown {
variable pkg punk::libunknown
variable version
set version 0.1
set version 0.2.0
}]
return

89
src/modules/punk/mix/commandset/loadedlib-999999.0a1.0.tm

@ -28,13 +28,37 @@ namespace eval punk::mix::commandset::loadedlib {
#search automatically wrapped in * * - can contain inner * ? globs
punk::args::define {
@id -id ::punk::mix::commandset::loadedlib::search
@cmd -name "punk::mix::commandset::loadedlib search" -help "search all Tcl libraries available to your local interpreter"
@cmd -name "punk::mix::commandset::loadedlib search" -help\
"search all Tcl libraries available to your local interpreter.
When punk::libunknown is active (the punkshell default) a deep module
discovery pass runs first, so .tm modules at every namespace depth are
included - cached per 'package epoch', repeat searches are cheap.
See -refresh for cache/re-scan details."
-return -type string -default table -choices {table tableobject list lines}
-present -type integer -default 2 -choices {0 1 2} -choicelabels {absent present both} -help\
"(unimplemented) Display only those that are 0:absent 1:present 2:either"
-highlight -type boolean -default 1 -help\
"Highlight which version is present with ansi underline and colour"
-refresh -type none -help "Re-scan the tm and library folders"
-refresh -type none -help\
"Force a genuine filesystem re-scan of the module and library folders.
When punk::libunknown is active (the punkshell default), every search
already performs a deep module discovery pass - registering .tm modules
at every namespace depth across all tcl::tm::list paths (see
punk::libunknown::register_all_tm) - cached per 'package epoch' so
repeat searches are cheap. Because directories already indexed in the
current epoch are not re-globbed, that default pass does not notice .tm
files added to (or removed from) already-scanned folders.
-refresh increments the package epoch ('package epoch incr'),
invalidating the scan caches, then re-runs discovery - picking up
on-disk changes and re-sourcing the pkgIndex.tcl files on ::auto_path.
Changes to tcl::tm::list or ::auto_path increment the epoch
automatically (via variable traces) - this flag is not needed for
those cases.
When punk::libunknown is not active there is no epoch cache: the
default search reflects only packages already registered in the
package database, and -refresh performs the (comparatively expensive)
deep discovery walk directly using dummy package require calls at
each namespace depth."
searchstring -default * -multiple 1 -help\
"Names to search for, may contain glob chars (* ?) e.g *lib*
If no glob chars are explicitly specified, the searchstring will be wrapped with star globs.
@ -51,19 +75,47 @@ namespace eval punk::mix::commandset::loadedlib {
set opt_highlight [dict get $opts -highlight]
set opt_refresh [dict exists $received -refresh]
if {$opt_refresh} {
catch {package require frobznodule666} ;#ensure pkg system has loaded/searched for everything REVIEW - this doesn't result in full scans
foreach tm_path [tcl::tm::list] {
set paths_below [punk::path::subfolders -recursive $tm_path]
foreach folder $paths_below {
set tail [file tail $folder]
if {[string match #modpod-* $tail] || [string match #tarjar-* $tail]} {
continue
#Deep discovery of available packages.
#The package unknown handlers register .tm siblings only at the namespace depth
#being requested - modules in never-requested subfolders are absent from
#'package names' until a deep pass registers them.
#Note: ::info must be fully qualified here - this namespace defines its own 'info' proc
if {[::info commands ::punk::libunknown::register_all_tm] ne "" && [::info exists ::punk::libunknown::epoch]} {
if {$opt_refresh} {
#genuine filesystem re-scan: start a new package epoch so the scan
#caches are invalidated (pkgIndex.tcl files on ::auto_path will also be
#re-sourced by the dummy require below)
package epoch incr
}
#register .tm modules at every namespace depth (cached per tm epoch -
#repeat searches are cheap)
punk::libunknown::register_all_tm
#sweep the pkgIndex.tcl files of ::auto_path for library-style packages
catch {package require frobznodule666}
} else {
#punk::libunknown deep registration unavailable - without its epoch cache a
#recursive walk on every search would be expensive, so deep discovery runs
#only on explicit request.
if {$opt_refresh} {
if {[::info exists ::punk::libunknown::epoch]} {
#older punk::libunknown without register_all_tm: invalidate its scan
#caches so the dummy requires below re-read the filesystem
catch {package epoch incr}
}
catch {package require frobznodule666} ;#top-level tm scan + pkgIndex sweep of ::auto_path
package require punk::path
foreach tm_path [tcl::tm::list] {
set paths_below [punk::path::subfolders -recursive $tm_path]
foreach folder $paths_below {
set tail [file tail $folder]
if {[string match #modpod-* $tail] || [string match #tarjar-* $tail]} {
continue
}
if {[string match */_build/* $folder]} {continue}
set relpath [string tolower [punk::path::relative $tm_path $folder]]
set modpath [string map {/ ::} $relpath]
catch {package require ${modpath}::flobrudder99}
}
if {[string match */_build/* $folder]} {continue}
set relpath [string tolower [punk::path::relative $tm_path $folder]]
set modpath [string map {/ ::} $relpath]
catch {package require ${modpath}::flobrudder99}
}
}
}
@ -85,8 +137,12 @@ namespace eval punk::mix::commandset::loadedlib {
}
set matches [lsort -unique $matches]
set matchinfo [list]
set highlight_ansi [a+ web-limegreen underline]
set RST [a]
if {$opt_highlight} {
#not a module-top require: punk::ansi is only needed for highlighting
package require punk::ansi
set highlight_ansi [punk::ansi::a+ web-limegreen underline]
set RST [punk::ansi::a]
}
foreach m $matches {
set versions [package versions $m]
if {![llength $versions]} {
@ -122,6 +178,7 @@ namespace eval punk::mix::commandset::loadedlib {
return [join $matchinfo \n]
}
table - tableobject {
package require textblock
set t [textblock::class::table new]
$t add_column -headers "Package"
$t add_column -headers "Version"

6
src/modules/punk/mix/commandset/loadedlib-buildversion.txt

@ -1,3 +1,7 @@
0.1.0
0.2.0
#First line must be a semantic version number
#all other lines are ignored.
#0.2.0 - search now performs deep module discovery by default when punk::libunknown is active: calls new punk::libunknown::register_all_tm (registers .tm modules at every namespace depth, cached per 'package epoch') plus the pkgIndex.tcl sweep of ::auto_path - so namespaced modules in never-requested subfolders (e.g test::*) appear without -refresh
#0.2.0 - -refresh repurposed to mean a genuine filesystem re-scan: increments the package epoch first (invalidating punk::libunknown's scan caches) then re-runs discovery, picking up .tm files added/removed on disk and re-sourcing pkgIndex.tcl files; without punk::libunknown (or with an older copy lacking register_all_tm) -refresh falls back to the previous behaviour (deep discovery walk via dummy package require calls at each namespace depth)
#0.2.0 - dependency fixes: highlight ansi codes now computed only when -highlight 1 using fully-qualified punk::ansi::a+/a with an inline package require (previously relied on shell-global a+ alias unconditionally and errored in bare interps even with -highlight 0); inline package require punk::path in the fallback -refresh walk and package require textblock in the table/tableobject return branch (previously assumed the shell environment had loaded them)
#0.1.1 - doc-only: rewrote 'search -refresh' PUNKARGS help to document actual semantics - a deep tm discovery pass (dummy requires at each namespace depth so ifneeded scripts get registered for .tm modules in never-requested subfolders), not a filesystem re-scan; within an epoch the punk::libunknown index cache short-circuits re-globbing of already-scanned dirs (run 'package epoch incr' first for a genuine re-scan); tm/auto_path list changes are handled automatically by epoch traces and don't need the flag

286
src/modules/punk/repl-999999.0a1.0.tm

@ -157,6 +157,29 @@ namespace eval repl {
}
variable codethread_cond
# -- G-001 selected console --
#repl_console: the ::opunk::Console (subclass) object value selected via 'repl::init -console <spec>',
#or "" when the repl runs on the process-default console (legacy stdin/stdout/stderr behaviour).
#conin/conout/conerr: the channels this repl reads from and writes to. For a selected foreign console,
#conin/conout come from the console's in/out slots and conerr is the same channel as conout
#(a single terminal stream - optional per-console err channels are G-011's concern, not G-001's).
variable repl_console
if {![info exists repl_console]} {
set repl_console ""
}
variable conin
if {![info exists conin]} {
set conin stdin
}
variable conout
if {![info exists conout]} {
set conout stdout
}
variable conerr
if {![info exists conerr]} {
set conerr stderr
}
variable screen_last_chars "" ;#a small sliding append buffer for last char of any screen output to detect \n vs string
variable screen_last_char_list [list]
@ -473,11 +496,20 @@ proc punk::repl::get_prompt_config {} {
return [list resultprompt $resultprompt nlprompt $nlprompt infoprompt $infoprompt debugprompt $debugprompt]
}
proc repl::start {inchan args} {
proc repl::start {args} {
#debug only - we need clean output for 'exec' to succeed without forcing use of -ignorestderr.
#puts stderr "-->repl::start $inchan $args"
#puts stderr "-->repl::start $args"
#flush stderr
#G-001: inchan is optional - when omitted (or the args begin with an option) the input
#defaults to the console selected at repl::init (conin - process stdin when no -console given).
variable conin
if {[llength $args] && ![string match -* [lindex $args 0]]} {
set args [lassign $args inchan]
} else {
set inchan $conin
}
upvar ::punk::console::input_chunks_waiting input_chunks_waiting
if {![info exists input_chunks_waiting($inchan)]} {
set input_chunks_waiting($inchan) [list]
@ -564,7 +596,13 @@ proc repl::start {inchan args} {
# ---
if {$::punk::console::ansi_wanted == 2} {
if {[::punk::console::test_can_ansi]} {
variable repl_console
if {![console_is_default] && $repl_console ne ""} {
#foreign console: never emit a probe to the process console - settle from the
#console object's (possibly overridden) can_respond.
#(ansi_wanted is deliberately process-global - scoped console state is G-008)
set ::punk::console::ansi_wanted [expr {[::opunk::Console::can_respond $repl_console] ? 1 : -1}]
} elseif {[::punk::console::test_can_ansi]} {
set ::punk::console::ansi_wanted 1
} else {
set ::punk::console::ansi_wanted -1
@ -629,7 +667,9 @@ proc repl::start {inchan args} {
thread::cond destroy $codethread_cond ;#race if we destroy cond before child thread has exited - as it can send a -async quit
set codethread ""
set codethread_cond ""
punk::console::mode line ;#review - revert to line mode on final exit - but we may be exiting a nested repl
if {[console_is_default]} {
punk::console::mode line ;#review - revert to line mode on final exit - but we may be exiting a nested repl
}
set donevalue [set [namespace current]::done]
#e.g
# "eof stdin"
@ -845,13 +885,86 @@ proc repl::newout2 {} {
}
#--------------------------------------
# -- G-001 selected-console helpers --
namespace eval repl::argdoc {
lappend PUNKARGS [list {
@id -id ::repl::console_is_default
@cmd -name "repl::console_is_default"\
-summary\
"Whether this repl runs on the process-default console."\
-help\
"Returns 1 when no foreign console was selected at repl::init
(conin/conout are the process stdin/stdout), 0 when the repl
reads/writes a selected ::opunk::Console backend's channels.
Legacy process-console behaviours (prompt gating on
tcl_interactive, stdin reopen on eof, raw-mode re-enable)
apply only when this returns 1."
@values -min 0 -max 0
}]
}
proc repl::console_is_default {} {
variable conin
variable conout
expr {$conin eq "stdin" && $conout eq "stdout"}
}
namespace eval repl::argdoc {
lappend PUNKARGS [list {
@id -id ::repl::console_at_eof
@cmd -name "repl::console_at_eof"\
-summary\
"eof state of the repl input, answered by the selected console object when present."\
-help\
"For a repl started with a selected ::opunk::Console backend this dispatches
to the console object's (possibly overridden) at_eof method - e.g a tk-widget
backend answers from its backend eof marker rather than any channel state.
Otherwise (or for a non-console input channel) plain chan eof is used."
@values -min 1 -max 1
inputchan -type string -help\
"the repl input channel being read"
}]
}
proc repl::console_at_eof {inputchan} {
variable repl_console
variable conin
if {$repl_console ne "" && $inputchan eq $conin} {
return [::opunk::Console::at_eof $repl_console]
}
return [chan eof $inputchan]
}
namespace eval repl::argdoc {
lappend PUNKARGS [list {
@id -id ::repl::console_get_size
@cmd -name "repl::console_get_size"\
-summary\
"Console size for this repl - selected console object when present, else punk::console::get_size."\
-help\
"Returns a dict with keys columns and rows. A selected ::opunk::Console
backend answers via its (possibly overridden) size method; the default
console uses punk::console::get_size."
@values -min 0 -max 0
}]
}
proc repl::console_get_size {} {
variable repl_console
if {$repl_console ne ""} {
return [::opunk::Console::size $repl_console]
}
return [punk::console::get_size]
}
proc repl::doprompt {prompt {col {green bold}}} {
#prompt to stderr.
#prompt to stderr (conerr - same channel as conout for a selected foreign console).
#We can pipe commands into repl's stdin without the prompt interfering with the output.
#Although all command output for each line goes to stdout - not just what is emitted with puts
if {$::tcl_interactive} {
flush stdout; #we are writing this prompt on stderr, but stdout could still be writing to screen
variable conout
variable conerr
#G-001: a selected foreign console is a terminal by declaration (constructed knowing its role),
#so prompts are wanted regardless of the process-level tcl_interactive state.
if {$::tcl_interactive || ![console_is_default]} {
flush $conout; #we are writing this prompt on conerr, but conout could still be writing to screen
#our first char on stderr is based on the 'lastchar' of stdout which we have recorded but may not have arrived on screen.
#The issue we're trying to avoid is the (stderr)prompt arriving midway through a large stdout chunk
#REVIEW - this basic attempt to get stderr/stdout to cooperate is experimental and unlikely to achieve the desired effect in all situations
@ -895,9 +1008,9 @@ proc repl::doprompt {prompt {col {green bold}}} {
set o [a {*}$col]
set r [a]
puts -nonewline stderr $c$pre$o$prompt$r
puts -nonewline $conerr $c$pre$o$prompt$r
screen_last_char_add " " "prompt-stderr" prompt
flush stderr
flush $conerr
}
}
@ -910,11 +1023,13 @@ proc repl::rputs {args} {
variable screen_last_chars
variable last_out_was_newline
variable last_repl_char
variable conout
variable conerr
set pseudo_map [dict create {*}{
debug stderr
debugreport stderr
}]
#map pseudo-channels - and, for a selected foreign console (G-001), the std channel
#names - to this repl's console channels. Identity mapping for stdout/stderr when
#running on the process-default console.
set pseudo_map [dict create debug $conerr debugreport $conerr stdout $conout stderr $conerr]
if {[::tcl::mathop::<= 1 [llength $args] 3]} {
set out [lindex $args end]
@ -922,20 +1037,28 @@ proc repl::rputs {args} {
if {([llength $args] > 1) && [lindex $args 0] ne "-nonewline"} {
set this_tail \n
set rputschan [lindex $args 0]
#map pseudo-channels to real
if {$rputschan in [dict keys $pseudo_map]} {
#map pseudo/std channels to this repl's console channels
if {[dict exists $pseudo_map $rputschan]} {
lset args 0 [dict get $pseudo_map $rputschan]
}
} elseif {[llength $args] == 1} {
set this_tail \n
set rputschan "stdout"
#implicit stdout - make the channel explicit so a selected console receives it
set args [list $conout $out]
} else {
#>1 arg with -nonewline
#first arg is -nonewline
set this_tail [string index $out end]
set rputschan [lindex $args 1]
#map pseudo-channels to real
if {$rputschan in [dict keys $pseudo_map]} {
lset args 0 [dict get $pseudo_map $rputschan]
if {[llength $args] == 3} {
set rputschan [lindex $args 1]
#map pseudo/std channels to this repl's console channels
if {[dict exists $pseudo_map $rputschan]} {
lset args 1 [dict get $pseudo_map $rputschan]
}
} else {
#2 args: -nonewline <data> - implicit stdout
set rputschan "stdout"
set args [list -nonewline $conout $out]
}
}
set last_char_info_width 60
@ -961,7 +1084,7 @@ proc repl::rputs {args} {
#set x \ud83c\udf1e
#(2 surrogate pairs - treated as single char in tcl8 - fixed in 9 but won't/can't be backported) -
#see also: https://core.tcl-lang.org/tips/doc/trunk/tip/619.md
puts stderr "$repl_error"
puts $conerr "$repl_error"
}
} else {
#looks like an invalid puts call - use the normal error produced by the puts command
@ -980,7 +1103,7 @@ proc repl::rputs {args} {
} else {
set clear "\n"
}
puts -nonewline stderr "$clear[a red bold]! REPL ERROR IN rputs $c$err$n\n"
puts -nonewline $conerr "$clear[a red bold]! REPL ERROR IN rputs $c$err$n\n"
screen_last_char_add "\n" replerror "rputs err: '$err'"
return
} else {
@ -1898,7 +2021,7 @@ proc repl::repl_handler {inputchan readmore prompt_config} {
}
if {$readmore} {
if {![chan eof $inputchan]} {
if {![console_at_eof $inputchan]} {
##################################################################################
#Re-enable channel read handler only if no waiting chunks - must process in order
##################################################################################
@ -1941,7 +2064,9 @@ proc repl::repl_handler {inputchan readmore prompt_config} {
}
}
if {$::tcl_interactive} {
if {$::tcl_interactive && [console_is_default]} {
#process-console repl only: a foreign console's eof (e.g remote disconnect,
#widget teardown) must finish the repl, not reopen the process stdin.
rputs stderr "\nrepl_handler EOF inputchannel: [chan conf $inputchan]"
#rputs stderr "\n|repl> ctrl-c EOF on $inputchan."
after 1 [list repl::reopen_stdin]
@ -2135,6 +2260,8 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
# ---
variable reading
variable id_outstack
variable conout
variable conerr
#upvar ::punk::config::configdata configd
#set running_config [dict get $configd running]
@ -2296,10 +2423,10 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
set do_checkwidth 1 ;#make configurable if performance hit is too severe? TODO
set consolewidth 132
if {$do_checkwidth} {
if {[catch {set consolewidth [dict get [punk::console::get_size] columns]} errM]} {
if {[catch {set consolewidth [dict get [repl::console_get_size] columns]} errM]} {
#review
if {!$is_vt52} {
puts stderr "repl_process_data failed on call to punk::console::get_size :$errM"
puts stderr "repl_process_data failed on call to repl::console_get_size :$errM"
}
}
#if chan conf stdout doesn't give dimensions and console doesn't respond to queries - we can get empty results in get_size dict
@ -2420,7 +2547,7 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
} trap {POSIX} {e eopts} {
rputs stderr "trap1 POSIX '$e' eopts:'$eopts"
flush stderr
flush $conerr
} on error {repl_error erropts} {
rputs stderr "error1 in repl_process_data: $repl_error"
rputs stderr "-------------"
@ -2432,14 +2559,14 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
} else {
rputs stderr "*> $inputchan reader active"
}
if {[chan eof $inputchan]} {
if {[console_at_eof $inputchan]} {
rputs stderr "todo - attempt restart of repl on input channel: $inputchan in next loop"
catch {set ::punk::nav::ns::ns_current "::"}
#todo set flag to restart repl ?
} else {
rputs stderr "continuing.."
}
flush stderr
flush $conerr
}
@ -2499,7 +2626,9 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
}
set stdinconf [fconfigure $inputchan]
if {$::tcl_platform(platform) eq "windows" && [dict get $stdinconf -encoding] ni [list unicode utf-16 utf-8]} {
if {[console_is_default] && $::tcl_platform(platform) eq "windows" && [dict get $stdinconf -encoding] ni [list unicode utf-16 utf-8]} {
#(default process console only: a selected foreign console's channel encoding is the
# transport's business - re-decoding its lines as utf-16be would mangle them)
#some long console inputs are split weirdly when -encoding and -translation are left at defaults - requiring extra enter-key to get repl to process.
#experiment to see if using iso8859-1 (raw bytes) and handling line endings manually gives insight.
# - do: chan conf stdin -encoding iso859-1 -translation lf
@ -2695,6 +2824,29 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
set raw_result [tsv::get codethread_$codethread result]
lassign [tsv::get codethread_$codethread info] _o lastoutchar_codethread _e lasterrchar_codethread
if {![console_is_default]} {
#G-001 foreign-console repl: code-interp std-channel writes were diverted into
#pending vars by the junction stacks installed at init - collect and emit them
#to the selected console's channels (rputs maps stdout/stderr to conout/conerr).
#Interleaving between the two streams within a run is not preserved (single
#terminal stream; per-console err semantics are G-011).
set pending [thread::send $codethread {
interp eval code {
set _pendinglist [list $::codeinterp::console_pending_out $::codeinterp::console_pending_err]
set ::codeinterp::console_pending_out ""
set ::codeinterp::console_pending_err ""
set _pendinglist
}
}]
lassign $pending pending_out pending_err
if {$pending_out ne ""} {
rputs -nonewline stdout $pending_out
}
if {$pending_err ne ""} {
rputs -nonewline stderr $pending_err
}
}
#set status [catch {
# thread::send $
# uplevel 1 {namespace inscope $::punk::nav::ns::ns_current $run_command_string}
@ -2711,8 +2863,8 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
#puts stderr "-->>$result<--"
#===============================================================================
flush stdout
flush stderr
flush $conout
flush $conerr
#foreach s [lreverse $outstack] {
# shellfilter::stack::remove stdout $s
#}
@ -3022,8 +3174,10 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
#----------------------------------------------------------------------------
#after any external command - raw mode as the console sees it can be disabled
#set it to match current state of the tsv
#(process-default console only: the tsv raw state and enableRaw target the
# process console, not a selected foreign console - scoped state is G-008)
#----------------------------------------------------------------------------
if {[tsv::get punk_console is_raw]} {
if {[console_is_default] && [tsv::get punk_console is_raw]} {
if {$::tcl_platform(platform) eq "windows"} {
#review
#we are in parent process - twapi might not be loaded here - even if it is in the code interp
@ -3057,14 +3211,14 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
set ::punk::repl::signal_control_c 0
chan event $inputchan readable {}
rputs stderr "* console_control: control-c"
flush stderr
flush $conerr
set c [a yellow bold]
set n [a]
rputs stderr "${c}repl interrupted$n"
#set commandstr [list error "repl interrupted"]
set commandstr ""
doprompt ">_ "
flush stdout
flush $conout
} else {
# parse and determine outermost unclosed quote/bracket and include in prompt
@ -3108,7 +3262,7 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
} trap {POSIX} {e eopts} {
rputs stderr "trap POSIX '$e' eopts:'$eopts"
flush stderr
flush $conerr
} on error {repl_error erropts} {
rputs stderr "error2 in repl_process_data: $repl_error"
rputs stderr "-------------"
@ -3120,14 +3274,14 @@ proc repl::repl_process_data {inputchan chunktype chunk stdinlines prompt_config
} else {
rputs stderr "*> $inputchan reader active"
}
if {[chan eof $inputchan]} {
if {[console_at_eof $inputchan]} {
rputs stderr "todo - attempt restart of repl on input channel: $inputchan in next loop"
catch {set ::punk::nav::ns::ns_current "::"}
#todo set flag to restart repl ?
} else {
rputs stderr "continuing.."
}
flush stderr
flush $conerr
}
}
@ -3194,10 +3348,10 @@ namespace eval repl {
variable codethread_cond
variable codethread_mutex
set opts [list -force 0 -type 0 -safelog 0 -paths {} -callback_interp $default_callback_interp]
set opts [list -force 0 -type 0 -safelog 0 -paths {} -callback_interp $default_callback_interp -console {}]
foreach {k v} $args {
switch -- $k {
-force - -type - -safelog - -paths - -callback_interp {
-force - -type - -safelog - -paths - -callback_interp - -console {
dict set opts $k $v
}
default {
@ -3224,6 +3378,37 @@ namespace eval repl {
if {$codethread ne "" && !$opt_force && [thread::exists $codethread] } {
error "repl:init codethread: $codethread already exists. use -force 1 to override"
}
#G-001 console selection: a -console spec ({in out} pair, anchored opunk::console instance
#name, or ::opunk::Console object value) selects the console this repl reads/writes.
#The resolved object (when present) answers at_eof/size/can_respond via its (possibly
#overridden) methods - see repl::console_at_eof / repl::console_get_size.
set opt_console [dict get $opts -console]
variable repl_console
variable conin
variable conout
variable conerr
if {$opt_console ne ""} {
set cinfo [punk::console::console_spec_resolve $opt_console]
set repl_console [dict get $cinfo object]
set conin [dict get $cinfo in]
set conout [dict get $cinfo out]
if {$conin eq "stdin" && $conout eq "stdout"} {
#explicit selection of the process-default console - full legacy behaviour incl. stderr
set conerr stderr
} else {
set conerr $conout
}
} else {
set repl_console ""
set conin stdin
set conout stdout
set conerr stderr
}
#whether code-interp std-channel writes must be diverted and routed to the selected console
#instead of reaching the process std channels
set conredirect [expr {![console_is_default]}]
set codethread [thread::create -preserved]
#review - naming of the possibly 2 cond variables parent and child thread
set codethread_cond [thread::cond create] ;#repl::codethread_cond held by parent(repl) vs punk::repl::codethread::replthread_cond held by child(codethread)
@ -3243,6 +3428,7 @@ namespace eval repl {
} %packageprefer% [list [package prefer]] {*}{
} %staticprefixes% [list [expr {[info exists ::punkboot::static_prefixes] ? $::punkboot::static_prefixes : ""}]] {*}{
} %staticpackages% [list [expr {[info exists ::punkboot::static_packages] ? $::punkboot::static_packages : ""}]] {*}{
} %conredirect% [list $conredirect] {*}{
}
]
#scriptmap applied at end to satisfy silly editor highlighting.
@ -4125,6 +4311,21 @@ namespace eval repl {
package require punk
package require shellrun
package require shellfilter
if {%conredirect%} {
#G-001 foreign-console repl: code-interp writes to stdout/stderr must not
#reach the process std channels. The shellfilter 'var' transform is a
#junction (no pass-through): writes divert into pending vars which the
#parent repl thread collects after each runscript and emits to the
#selected console's channels. Added before the colour ansiwrap stacks so
#those still apply to the diverted data (writes traverse newest transform
#first, so a junction added earlier captures last).
namespace eval ::codeinterp {
variable console_pending_out ""
variable console_pending_err ""
}
shellfilter::stack::add stdout var -settings {-varname ::codeinterp::console_pending_out}
shellfilter::stack::add stderr var -settings {-varname ::codeinterp::console_pending_err}
}
#set running_config $::punk::config::running
apply {running_config {
if {[string length [dict get $running_config color_stderr]] && [punk::console::colour]} {
@ -4229,6 +4430,11 @@ namespace eval repl {
}
#init - don't auto init - require init with possible options e.g -type
}
namespace eval ::punk::args::register {
#use fully qualified so 8.6 doesn't find existing var in global namespace
lappend ::punk::args::register::NAMESPACES ::repl
}
package provide punk::repl [namespace eval punk::repl {
variable version
set version 999999.0a1.0

3
src/modules/punk/repl-buildversion.txt

@ -1,6 +1,7 @@
0.3.0
0.4.0
#First line must be a semantic version number
#all other lines are ignored.
#0.4.0 - G-001: repl::init -console <spec> selects the console the repl reads/writes ({in out} pair, anchored opunk::console instance name, or ::opunk::Console object value, resolved via punk::console::console_spec_resolve). repl::start's inchan is now optional (defaults to the selected console's input). New repl-level channel state (conin/conout/conerr) is routed through rputs (stdout/stderr mapped per-repl) and doprompt; for a selected foreign console the code interp's stdout/stderr are diverted via shellfilter 'var' junction stacks and emitted to the console after each run. New helpers repl::console_is_default / console_at_eof / console_get_size - eof and size are answered by the selected console object's (possibly overridden) methods. Process-console behaviours (tcl_interactive prompt gating, stdin reopen on eof, raw-mode re-enable, utf-16be windows line re-decode experiment, mode-line on exit) now apply only to the default console. Default-console (stdin/stdout) behaviour unchanged. Also fixes rputs pseudo-channel mapping in the 3-arg -nonewline form (mapped value previously written over the -nonewline flag).
#0.3.0 - G-058: codethread init script receives the runtime static/builtin package baseline (::punkboot::static_prefixes/static_packages via new %staticprefixes%/%staticpackages% scriptmap entries) and seeds ifneeded mappings in the codethread interp, so the code interp (seeded in turn via punk::lib::interp_sync_package_paths) can package-require statically-linked runtime packages - fixes 'can't find package Thread' booting a punk9win.vfs kit on the tclsfe-x64 static runtime (punk91)
#0.2.2 - repl_handler line-mode waiting-chunks path no longer performs its opportunistic unsized read on tcl 8.6 windows console channels (no -inputmode key, real twapi console handle): the path is entered via 'after idle' with the channel usually drained, and on the 8.6 console driver a drained read parks a blocking cooked-mode ReadConsole that a later raw flip cannot cancel - after typed-ahead was stashed during a terminal-query raw window, that parked read swallowed every subsequent query response in the session until Enter (companion to the punk::console 0.7.1 three-site console-misdetection fix). On such consoles the path now consumes only data already in the Tcl channel buffer (chan pending input + sized read - no driver probe); with nothing buffered it processes the stashed complete lines directly and arms the readable handler for any remaining partial line (replacing an after-idle reinvoke that could never progress). Behaviour on tcl 9/8.7 (-inputmode consoles) and in raw mode is unchanged.
#0.2.1 - call-site update for punk::console 0.6.0 tsv array rename: console -> punk_console (is_raw); no behaviour change

86
src/modules/punk/sshrun-999999.0a1.0.tm

@ -55,6 +55,92 @@
#[subsection Concepts]
#[para] -
####### original wiki documentation
## Overview
## Tclssh is a package of pure Tcl functions for executing Tcl scripts in a remote host. It is used by Nbsp and the WRFPak to offload some long-time running jobs in parallel to other computers.
##
## The remote host must be accessible via ssh keys without a password; that is, for example, the contents of your ./ssh/id_rsa.pub must be added to the .ssh/authorized_keys file in the remote host.
##
## The directory
##
## <prefix>/share/doc/examples
## has several examples that can be used for testing and documentation.
##
## Example
## The best way to explain how to use is by looking at one of them, ex-1.tcl.
##
## #!/usr/bin/tclsh
##
## package require ssh
##
## #
## # Collect in the string "script" the script that will be sent to the
## # remote (slave) host.
## #
## set script {
## puts "I am a remote host named [info hostname] executing a script
## that my master sent me. I am going to sent him back the results of some
## commands. Here they are:
##
## The date is: [exec date]
## My info is: [exec uname -a]
##
## Now I need to inform my master that I have finished.
##
## DONE 0"
## };
##
## set slave "is.1-loop.net"; # This is the remote host
##
## #
## # If the tcl shell in the remote host is just "tclsh" (e.g. a linux system)
## # the -t option (and its argment) can be omitted; otherwise it can be
## # used to specify the name of the remote tcl shell.
## #
## ::ssh::connect -t tclsh8.6 -- $slave
## ::ssh::push $slave $script
## ::ssh::send $slave;
## while {[::ssh::pop_line $slave line] >= 0} {
## if {[regexp {^DONE\s+(\d+)} $line match code]} {
## break;
## }
## puts $line;
## }
## ::ssh::disconnect $slave;
##
## puts "Slave finished with code: $code";
## Below is the output of executing ex-1.tcl from a machine named elite:
##
## [nieves@elite examples]$ ./ex-1.tcl
## I am a remote host named is.1-loop.net executing a script
## that my master sent me. I am going to sent him back the results of some
## commands. Here they are:
##
## The date is: Sun Dec 9 17:35:43 AST 2012
## My info is: Linux is 2.6.32-308.8.2.el5.028stab101.1 #1 SMP Sun Jun 24 20:25:35 MSD 2012 i686 GNU/Linux
##
## Now I need to inform my master that I have finished.
## Slave finished with code: 0
## [nieves@elite examples]$
## Several other examples in the
##
## <prefix>/share/doc/examples
## directory illustrate more capabilities of this package.
# ++ +++ +++ +++ +++ +++ +++ +++ +++ +++ +++
## Requirements

Some files were not shown because too many files have changed in this diff Show More

Loading…
Cancel
Save