punkshell/goals/archive/G-176-commandstack-doc-integration.md

G-176 commandstack doc integration - stack-tied overrider argdocs, override hints, and the libunknown package-override migration

Status: achieved 2026-08-08 Scope: src/modules/commandstack-999999.0a1.0.tm (rename_command -punkargs option, stackdocs pending-docs namespace, remove_rename detach hook, help/argdoc how-to); src/modules/punk/libunknown-0.3.1.tm (package-override migration to commandstack + flagship epoch doc - fixed-version file, renamed 0.2.4 -> 0.3.0 -> 0.3.1 across increments 2 and 4); src/modules/punk/ns-999999.0a1.0.tm (cmdhelp/cmdinfo override-in-place hint); src/tests/modules/punk/libunknown/testsuites/discovery/ (override characterisation arms - discovery.test extensions or a sibling packageoverride.test); src/tests/modules/commandstack/testsuites/commandstack/commandstack.test (-punkargs lifecycle pins); src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test (override-hint pins) Goal: a command overrider using commandstack can attach punk::args doc blocks that live and die with its stack entry (supplied at rename_command time, active whether or not punk::args is loaded yet, removed by every removal path), cmdhelp renders an override-in-place hint on any commandstack-managed command's existing documentation (naming the renamers and any doc ids their records added), and the punk::libunknown ::package override - the flagship consumer, whose 'package epoch' subcommand gains real documentation this way - is migrated onto commandstack so both live ::package overrides share one managed stack. Acceptance: characterisation tests of the CURRENT libunknown override land before any behaviour change and stay green byte-identically through the migration (the epoch tm/pkg single-arg forms, indexed ?keys...? queries and their error arms, the forget arm incl the tcl/Tcl/tcl::oo/tk denial and epoch-scripts pruning, abbreviation acceptance (ep/epo/epoc, fo/for/forge), a delegation pass-through pin, the re-init guard, and a packagepreference interop child covering both install orders) with only explicitly marked install-shape pins (::package import link, :📦: parked name) flipping when the migration lands; libunknown installs its ::package override via commandstack::rename_command with a stack-query install guard mirroring punk::packagepreference and no :📦: residue; commandstack::rename_command accepts a -punkargs list of definitionlists whose docs are queryable while the record lives - including renames performed BEFORE punk::args loads (pinned in a bare child that loads punk::args afterwards) - and are removed by remove_rename, pop_rename, remove_renamer and restore_original alike (record key appended per the key-order contract, reload-contract guard on new state); punk::ns::cmdhelp shows an override-in-place notice for a commandstack-managed command (cmdinfo carrying it as a second-axis key in the G-166 (achieved, see goals/archive/G-166-cmdinfo-unavailable-surfacing.md) style) without a hard commandstack dependency, enumerating record-attached doc ids; 'i package epoch' resolves the libunknown-attached doc in a punk shell and the mechanism how-to is described in commandstack::help and the rename_command argdoc; punk/libunknown, commandstack, punk/ns and punk/packagepreference suites pass under the canonical runtests interpreter.

Context

Feasibility investigated 2026-08-08 (user-directed; probe evidence below from tclsh90s with src modules + tclcore moduledoc). Motivating observation: punk::libunknown extends ::package with an 'epoch' subcommand that has no punk::args documentation, and an overrider currently has no way to attach docs whose lifetime matches the override.

Probe-verified doc mechanics (the punk::args side needs NO changes):

• A space-form id {::package epoch} defines cleanly and punk::args::define RETURNS the id. i package epoch resolves the new doc as a focused render EVEN THOUGH 'epoch' is absent from the parent ::package definition's restricted subcommand choices - the G-051 space-form docid jump (achieved, see goals/archive/G-051-cmdinfo-pseudo-and-prefix.md) bypasses the choice screen. i package is unaffected. undefine (quiet flag available) removes cleanly and i package epoch falls back to the parent render; re-define cycles are clean.
• punk::args::undefine_deflist is deflist-keyed - commandstack never parses ids.
• Recorded limitation: the parent def's choice TABLE never lists 'epoch' and parsing 'epoch' against ::package fails its choice restriction - mitigated by the hint mechanism enumerating record-attached doc ids on the parent render.

commandstack architecture facts (as at 2026-08-08, commandstack 0.7.1):

• Record keys are APPEND-only extensible by documented contract (token/renamer at lsearch -index 1/-index 3; 'additive keys must be APPENDED').
• EVERY removal path funnels through remove_rename (pop_rename :884, remove_renamer :936, restore_original :996 all delegate) - one detach choke point at the doomed_record site; one attach point in rename_command's success path.
• Early boot: commandstack renames happen BEFORE punk::args loads. The inert registration pattern (namespace eval ::punk::args::register {lappend NAMESPACES ...}
  • used by libunknown itself at :2175) covers this: a ::commandstack::stackdocs namespace PUNKARGS variable mirrors live records' deflists; punk::args consumes it lazily on first doc query. When punk::args is ALREADY loaded the attach must also define immediately (a namespace already consumed into loaded_packages never lazy-loads later appends; redefinition is idempotent).
• The reload contract (0.7.1) preserves module state across re-source via info-exists guards - new stackdocs state needs the same guard.

libunknown override facts (libunknown-0.2.4.tm, line anchors point-in-time):

• Current install (:1692-1822): proc package in ::punk::libunknown + rename ::package ::package:: + NAMESPACE IMPORT into :: - the import is load-bearing (an imported proc executes in its origin namespace, resolving the body's bare epoch_incr_pkg/epoch_incr_tm calls and variable statement). A commandstack::rename_command migration was started and commented out (:1689).
• Migration deltas are enumerable: qualify 4 bare epoch_incr_pkg/tm calls (a commandstack-installed proc runs in ::), swap the 2 ::package:: delegation sites (:1734 forget, :1816 default) for COMMANDSTACKNEXT, drop the DEAD variable has_package_files declaration (:1695 - its only use :1709 is commented out; the :143-147 module-load probe stays), replace rename+import with rename_command + a stack-query install guard (punk::packagepreference::install :110-137 is the exact in-tree precedent incl the G-160 (achieved, see goals/archive/G-160-commandstack-hygiene.md) silent re-install return), keep the :1574 info-commands re-init guard semantics.
• NO external :📦: callers exist anywhere in the tree (grepped src/modules + tests) - the parked-name change is libunknown-internal.
• Latent hazard the migration cures: in a punkshell boot packagepreference installs its ::package wrapper via commandstack FIRST (app entrypoints, e.g punkshell.tcl:9) and libunknown's plain rename lands ON TOP (repl-triggered init) - commandstack's ::package stack then believes the wrapper is live at ::package while libunknown's import actually is; restore_original or a third-party rename_command would operate on the wrong command and strand :📦:. make.tcl (:1748) runs libunknown with NO packagepreference - both orders occur, and one managed stack handles either (multi-renamer any-order removal is commandstack.test-pinned).
• Hot-path note: rename_command's injected header resolves COMMANDSTACKNEXT via get_next_command on every invocation - packagepreference already pays exactly this on the same hot command (every package require); precedent accepted.

Coverage audit (2026-08-08): discovery.test pins six epoch arms (0-arg dict shape, tm-paths/auto_path trace increments, epoch incr, cache-blocks-until-incr, register_all_tm per-epoch no-op) plus libsearch.test indirect use. UNTESTED: the epoch tm/epoch pkg single-arg forms, indexed queries + error arms, the entire forget arm, abbreviation acceptance (the override accepts prefixes like 'ep' that the builtin would reject - an uncharacterised behaviour delta), delegation as a contract, the install shape, the re-init guard, and any packagepreference interop.

Approach

Increments land as separate commits, each leaving all suites green:

1. Characterisation: pin every untested arm of the CURRENT override (list in Acceptance) in punk/libunknown testsuites (discovery.test extensions or a sibling packageoverride.test - child-interp pattern, children source libunknown AND commandstack by path). Install-shape pins (::package import link, :📦: parked name) carry explicit marked-to-flip comments.
2. Migration: libunknown installs via commandstack::rename_command (deltas per Context). Only the marked install-shape pins flip; every behaviour pin stays green unchanged. Decision recorded in this file: hard package require commandstack (lean, dependency-free, present in bootsupport + kits) - no plain-rename fallback path (two install behaviours would double the test surface). Fixed-version file rename on bump: git mv + fossil mv.
3. commandstack -punkargs mechanism: leading option on rename_command (beside -renamer) taking a list of definitionlists; record key punkargs appended; dual activation (stackdocs PUNKARGS variable + immediate define when punk::args present); detach in remove_rename via undefine_deflist (quiet) + variable removal; reload guard; help/argdoc how-to; commandstack.test pins incl the pre-punk::args pending path (bare child renames with -punkargs, then loads punk::args and queries).
4. Override hint + flagship docs: cmdinfo gains a second-axis override key sourced from a guarded live commandstack::get_stack query (no hard dependency); cmdhelp renders the notice naming renamers + record-attached doc ids; libunknown attaches the {::package epoch} doc block via -punkargs (retiring the doc gap that motivated the arc); cmdhelp.test pins; i package epoch verified in a punk shell.

Progress

• Increment 1 (characterisation) landed 2026-08-08: src/tests/modules/punk/libunknown/testsuites/discovery/packageoverride.test - 11 pins, all green first run against the current implementation. BEHAVIOUR pins: epoch single-arg (tm/pkg) and indexed query forms, the four error arms, the filtered forget arm (real-ifneeded forgotten, no-ifneeded filtered, tcl/Tcl/tcl::oo/tk denied, mixed lists partially applied), abbreviation acceptance (ep/epo/epoc + fo routing; the parked builtin rejects 'ep'), delegation pass-through, and the re-init no-op contract. MARKED-TO-FLIP pins (comment-fenced to the increment-2 commit): install shape (::package origin = ::punk::libunknown::package import, :📦: parked and functional) and the two-override interop children (both real-world orders: pkgpref-first stack=1 + import origin; libunknown-first stack=1 + ::package proc origin). Children source the source-tree libunknown/commandstack/packagepreference by path via a shared locator (discovery.test pattern). Tests AGENTS.md index updated.

• Increment 2 (migration) landed 2026-08-08: punk::libunknown 0.3.0 (file renamed from libunknown-0.2.4.tm - git mv + fossil mv). The ::package override installs via commandstack::rename_command (renamer punk::libunknown): init does 'package require commandstack' (a no-op when already provided - resolvable via tm paths in every real init context; bootsupport and the common vfs both carry commandstack-0.7.1) and re-entry is guarded by a stack query for our record (same 'already done' notice). Body deltas exactly as enumerated in Context: four epoch_incr_* calls qualified, the dead 'variable has_package_files' declaration dropped, the two delegation sites moved to COMMANDSTACKNEXT, the rename+import install replaced, the init argdoc reworded. All increment-1 BEHAVIOUR pins stayed green unchanged; only the marked install-shape pins flipped (origin ::package, no :📦: residue, managed stack records - 1 solo, 2 in both interop orders). discovery.test and packageoverride.test children now pre-source the source-tree commandstack by path (their tm paths are cleared); libsearch/shadowing children resolve commandstack via inherited parent paths and needed no changes. Suites: libunknown tree + loadedlib 42/42; full modules tree green. Project 0.63.0.

• Increment 3 (-punkargs stack-tied docs) landed 2026-08-08: commandstack 0.8.0. rename_command takes a leading '-punkargs ' (each element one definitionlist as given to punk::args::define - the same shape as a PUNKARGS variable element; the leading-option parse is generalised, -renamer and -punkargs in either order, both leading-only with pointed misplacement errors). Docs attach on a landed rename only: record gains a TRAILING 'punkargs' key (key-order contract intact), deflists are mirrored in the new reload-guarded ::commandstack::stackdocs namespace (registered inert into ::punk::args::register::NAMESPACES at module load - punk::args need not exist) and are ALSO defined immediately when punk::args is present (a namespace already consumed into loaded_packages never lazy-loads later appends; redefinition idempotent - a define error surfaces to the caller with the rename installed). Detach lives in remove_rename's doomed-record site - pop_rename/remove_renamer/restore_original all funnel through it - removing one mirror occurrence per deflist and undefining via punk::args::undefine_deflist (deflist-keyed; never-defined deflists skip silently). One live record per doc id is the documented shape. commandstack::help + the rename_command argdoc carry the how-to; In-tree users line gains punk::libunknown. 5 new commandstack.test pins (docs live with record incl key order, all four removal paths detach, the pre-punk::args pending path in a bare child loading punk::args afterwards, reload contract on the mirror + single registration, option positions) + 2 G-160 message pins updated to the generalised usage/misplacement wording. Suite 46/46. Project 0.64.0.

• Increment 4 (override hint + flagship docs) landed 2026-08-08: punk::ns 0.11.0

  • punk::libunknown 0.3.1 (file renamed from 0.3.0). cmdinfo gains the 'overridden' second-axis key (G-166 style): live guarded commandstack::get_stack query on the base command - one {renamer docids} dict per record, docids derived with punk::args::rawdef_id from the record's -punkargs deflists; empty when commandstack absent; never cached. cmdhelp leads table/string renders with the plain "note: '' is currently overridden/extended via commandstack by: (override-attached docs: ...)" line, -return text leads with OVERRIDDEN: (below UNAVAILABLE when both apply), -return dict carries the key on all return shapes, and the undocumented fallback carries the note. libunknown attaches the {::package epoch} doc block via -punkargs (substitution-free deflist; init's require tightened to 'commandstack 0.8-' so a pre-mechanism stable copy fails pointedly at init rather than mid-call - found via shadowing.test, whose require-based children needed 'package prefer latest': stable preference picks a stale stable bootsupport commandstack over the alpha dev copy). Verified end-to-end: 'i package epoch' renders the real epoch usage (the arc's motivating gap, retired), 'i package' leads with the note naming punk::libunknown and '::package epoch', restore_original removes doc and note. cmdhelp.test: 2 new hint pins + the two key-list pins flipped again (as with G-166); packageoverride.test: flagship pending-path pin (doc mirrored pre-punk::args in path-cleared children, lazily defined, removed on unwind). Suites: libunknown+ns+commandstack+loadedlib 216/216. Project 0.65.0.

Follow-ons

• Boot-site commandstack pre-sourcing (out of the current Scope - a Scope amendment is needed if pursued): punkboot core.tcl, make.tcl, punk::repl's two blocks and punk::lib's interp_sync child-seeding all pre-source LIBUNKNOWN by path expressly to avoid triggering the old package unknown pre-init. init's new 'package require commandstack' resolves in all of them, but when commandstack is not yet provided it performs one ordinary unknown-path resolution pre-init (a root-depth sibling scan lands in 'untracked' - functional, marginally looser epoch bookkeeping; in punkshell app boot packagepreference::install provides commandstack first so the require is a no-op there). Extending the same glob+source technique to commandstack-*.tm at those sites would preserve the minimal-untracked intent exactly. => open
• make.tcl guard modernization (out of the current Scope): the 'info commands ::punk::libunknown::package' check that skips re-load when make.tcl is driven by a built punk shell tests a proc the 0.3.0 install no longer creates. Old kits (0.2.4) still satisfy it so nothing breaks today, but once kits carry 0.3.0 the check never fires and make.tcl re-sources libunknown into the driving shell (harmless - init's guard no-ops - but noisy and wasteful). Switch it to a "[package provide punk::libunknown] ne {}" check before or with the bootsupport/kit promotion of 0.3.0. => open

Notes

• Same-id shadowing: one live record per doc id is the documented constraint (natural usage - each overrider documents its OWN added subcommands); no re-instatement machinery unless a real consumer needs it.
• Rename_stack parking: default decision is detach-on-park (docs describe a command that no longer answers); revisit if an unpark consumer appears.
• The abbreviation-acceptance delta (override accepts ep/fo prefixes the builtin rejects) is characterised as CURRENT behaviour in increment 1 - whether to keep it is a separate question, deliberately out of scope here (the migration must not silently change it).
• Mechanism B's optional heuristic for UNMANAGED overrides (doc @cmd -name says 'Built-in:' while the live command is a proc/import) is out of scope - after the increment-2 migration both live ::package overrides are managed.
• The tclcore parent ::package definition is untouched: 'epoch' stays outside its restricted subcommand choices; the hint line plus the space-form doc are the contract. Extending parent choice tables from override records is a possible follow-on, not part of this goal.
• G-058 static-package interplay (achieved, see goals/archive/G-058-static-runtime-packages.md): the packagepreference wrapper body consulted in Context is the same one the interop child exercises - its COMMANDSTACKNEXT_ORIGINAL fast-path is behaviour the interop pins must not disturb.