diff --git a/CHANGELOG.md b/CHANGELOG.md index 4f9d29d3..c4f2db4b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,11 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` "Project Versioning" section for the bump policy. +## [0.4.15] - 2026-07-10 + +- punk::args 0.4.2 (G-046): argument resolution no longer expands display-only `${...}` content (`-help` on `@cmd`/`@examples`/argument records, `@formdisplay` bodies) - expansion is deferred to help-display time with its own cache. First use of heavily documented commands is dramatically faster (`punk::ansi::mark_columns` first call ~4.3s → ~12ms; tclcore `::lseq` resolve ~184ms → ~2ms) and `-help` content that calls punk::args-parsing commands (including against its own definition id) resolves cleanly instead of stalling or looping. Also fixed: `@dynamic` multiline `-help` substitutions now align at their insertion column; prefix/alias-normalized choice values keep the same plain-string shape as exact input (`\Deleted`-style values); the `-return string` renderer aligns cmd-help continuation lines under the `Description:` label and its `Example:` line shows the example reference instead of the doc url. +- punk::ns 0.1.4 (G-046): `i ` for a command whose definition requires arguments (e.g `i string is`, `i punk::args::define`) shows plain usage instead of an internal-looking "Bad number of leading values for punk::args::parse ..." error prefix - and builds the usage table once, not twice (the discarded advisory-parse error no longer renders its own table; display timing for large argdocs stays at pre-change levels). + ## [0.4.14] - 2026-07-10 - punk::args 0.4.1: first `i ` in a fresh shell now shows subcommand-help markers and synopsis choicelabels for documented subcommands whose argdocs are lazily registered (e.g `i ansistring`); previously the autogenerated ensemble help omitted them until another path loaded the ensemble's registered namespace (`punk::args::update_definitions`). diff --git a/GOALS.md b/GOALS.md index ad02748a..1756d300 100644 --- a/GOALS.md +++ b/GOALS.md @@ -321,7 +321,7 @@ Detail: goals/G-045-punkargs-authoring-ergonomics.md Goal: authoring punk::args definitions no longer requires backslash line-continuations or ad-hoc workarounds for multi-line records and constructed definitions - a parser-recognised record-continuation mechanism (candidate: an unquoted trailing -& token, with the detail file recording the collision analysis and the element-count disambiguation alternative), -unindentedfields honoured for @cmd fields, and constructed (string-built) definitions able to opt into the same whole-block indent normalization file-style definitions get - with the container quoting rules (braced=literal, quoted=Tcl backslash semantics, \$\{ escape) promoted from defquoting.test into the define documentation. Acceptance: a definition using the chosen record-continuation mechanism parses identically to its backslash-continuation equivalent (existing definitions unchanged - continuation is additive), with the token's collision rules documented and an escape/rejection story for values that legitimately match it; @cmd -help/-summary honour -unindentedfields (the rendering.test GAP rendering_unindentedfields_cmd_help_GAP flips to aligned); a constructed definition can request whole-block normalization so embedded continuation indentation behaves as in file-style definitions (the rendering_constructed_def_indent_characterization expectations updated to the chosen semantics), and ::punk::helptopic::define_docs drops its manual pre-normalization to prove it; the quoting rules from defquoting.test appear in the punk::args::define -help documentation; the full punk::args suite (128 tests incl. the rendering invariants: nesting independence, relative-indent preservation) passes with GAP tests flipped, none weakened. -### G-046 [proposed] punk::args deferred -help resolution (parse-time performance + reentrancy) and rendering/value-shape fixes +### G-046 [achieved 2026-07-10] punk::args deferred -help resolution (parse-time performance + reentrancy) and rendering/value-shape fixes Scope: src/modules/punk/args-999999.0a1.0.tm (resolve/get_dict: display-field deferral, dynamic-cache subst path, prefix writeback, string renderer, cmdhelp-facing messages), src/modules/punk/ansi-999999.0a1.0.tm (mark_columns argdoc as the reentrancy/perf testbed), src/tests/modules/punk/args/testsuites/ (GAP tests flip; perf verification) Detail: goals/G-046-punkargs-deferred-help-and-fixes.md Goal: argument resolution no longer processes -help and other display-only fields - their tstr expansion is deferred to display time (separately cached, per the existing in-source review notes) - so first parse of heavily documented commands gets measurably faster and definitions whose -help calls punk::args-parsing commands (the punk::ansi::mark_columns class) neither loop nor stall; alongside, the mechanical defects pinned by the characterization suites are fixed: @dynamic double-substituted multiline values align at their insertion column, prefix-normalized choice values keep the same shape as exact input, the -return string renderer aligns cmd-help continuations under the first line, and the misleading goodargs parse-error prefix in 'i ' output is fixed. diff --git a/goals/G-046-punkargs-deferred-help-and-fixes.md b/goals/G-046-punkargs-deferred-help-and-fixes.md index 628ba516..e236e5ff 100644 --- a/goals/G-046-punkargs-deferred-help-and-fixes.md +++ b/goals/G-046-punkargs-deferred-help-and-fixes.md @@ -1,6 +1,6 @@ # G-046 punk::args deferred -help resolution (parse-time performance + reentrancy) and rendering/value-shape fixes -Status: proposed +Status: achieved 2026-07-10 Scope: src/modules/punk/args-999999.0a1.0.tm (resolve/get_dict: display-field deferral, dynamic-cache subst path, prefix writeback, string renderer, cmdhelp-facing messages), src/modules/punk/ansi-999999.0a1.0.tm (mark_columns argdoc as the reentrancy/perf testbed), src/tests/modules/punk/args/testsuites/ (GAP tests flip; perf verification) Acceptance: parsing/argument resolution provably skips -help expansion (a definition whose -help contains a ${[...]} that would error or record its invocation shows the substitution did NOT run during a parse-only path, only for help display); first parse of punk::ansi::mark_columns drops from ~4s to well under a second with 'i punk::ansi::mark_columns' still rendering the embedded example, and a -help that parses its OWN definition id resolves or errors cleanly rather than looping; first-parse timing improves for at least one other heavily documented command (recorded in the detail file); rendering_atdynamic_multiline_help_insertion_GAP flips to all-aligned; choicegroups_imap_prefix_listwrap_GAP flips to shape-identical (prefix input yields the same plain string as exact input); the -return string renderer's cmd-help continuations align under the first line with relative indents preserved (rendering_string_renderer_characterization updated); the 'Bad number of leading values...' prefix shown by goodargs parsing in 'i string is'-style output is reworded or suppressed for the usage-display path; full punk::args and punk::ns suites pass with no non-GAP expectations weakened. @@ -76,6 +76,89 @@ the (empty) trailing args. Reword for the usage-display path or suppress (the pa only advisory for highlighting supplied args). User: "the message is not accurate and could do with a fix up at some stage." +## Implementation design (item 1, decided 2026-07-10) + +Masking approach - the existing tstr pipeline stays authoritative for everything eager: + +1. Phase 0 (per block containing `${`): `tstr -return list -eval 0` extracts + (textparts, params) without evaluation. Assign each param a globally numbered inert + placeholder token and build the joined placeholder text. +2. Phase 1 (classification): split the placeholder text into records (the record splitter + factored out of resolve into a shared private proc) and classify each placeholder by + the field it lands in. Display-only fields: `-help` on @cmd/@examples and on argument + records, `-choicelabels` values, @formdisplay bodies. Everything else (including @cmd + -summary, -choices, -choicealiases, -typesynopsis, @form -synopsis) stays eager so + synopsis/parse never need the display spec. +3. Phase 2 (masked rebuild): for blocks carrying display params, rebuild the block with + display `${...}` sources replaced by their placeholder tokens, then run the EXISTING + `tstr -return string -eval 1 -allowcommands -undent 1` call unchanged - eager params + get bit-identical current behaviour (alignment, undent, errors); placeholders pass + through as literal text into the parsed argdata (so -help values in the parse spec + contain tokens, never expanded content). +4. argdata gains a deferred-params map (token -> {param source, defspace, dynamic flag}). + get_dict never consults display fields, so parsing works from this spec directly. +5. Display expansion (new private helper): walks the known display slots of a spec dict, + evaluates each token's param (`namespace eval $defspace [list ::subst $param]`; + for @dynamic definitions a second subst round runs on the result - this is where the + round-2 paramindent re-alignment (item 2) is implemented), inserts multiline results + aligned to the leading whitespace of the token's line within the stored field value + (mirroring tstr -paramindents line). Non-dynamic display specs are cached; @dynamic + display expansion re-runs per render (preserving provider-refresh semantics). +6. Hooks: get_spec returns the display-expanded spec (public consumers see today's + shape); arg_error expands any spec dict still carrying a deferred map (parse-error + -argspecs travel unexpanded). synopsis stays on eager fields only. +7. Reentrancy guard (defence in depth): an in-progress set keyed by definition id; a + display expansion re-entered for the same id substitutes the raw `${...}` source + instead of recursing, so a -help that renders help for its own id resolves cleanly. + +@dynamic interaction: round-1 display params are masked (so their `${[provider]}` +payloads never run at parse time); the round-2 unresolved-cache (ptlist/paramlist of the +round-1 OUTPUT) carries the tokens inertly in its text parts, while round-2 parse-relevant +params keep refreshing every resolve exactly as today. + +## Verification (2026-07-10, punk::args 0.4.2 / punk::ns 0.1.4) + +Perf (punk902z src, this machine; OLD numbers measured same-day with the pre-deferral +module stashed back in): +- punk::ansi::mark_columns first call: OLD 4317.6ms -> NEW 12.4ms (subsequent 0.5ms); + first 'i'-equivalent help render 52ms with the embedded num_columns_example ruler + rendering correctly and no deferral tokens in output. +- tclcore ::lseq (helpers::example blocks): first parse-spec resolve OLD 184.4ms -> + NEW 1.8ms; the display cost (581ms incl. table layout) is now paid only on first help + render. + +Behaviour (all in src/tests/modules/punk/args/testsuites/args/deferredhelp.test): +- parse provably skips -help ${[...]} (invocation-recording probe: 0 calls across + parses, 1 after first display, cached for non-dynamic thereafter). +- -help substitution parsing against its OWN id resolves cleanly at display time + (parse inside the helper uses the already-cached parse spec). +- -help substitution rendering usage for its OWN id completes via the reentrancy + guard (nested expansion substitutes raw ${...} sources). +- @dynamic deferred -help re-expands per display (provider refresh preserved). + +GAP flips: rendering_atdynamic_multiline_help_insertion (was ..._GAP) now all-aligned +[0 0 2 0]; choicegroups_imap_prefix_shape (was ..._listwrap_GAP) shape-identical; +rendering_string_renderer_characterization updated to aligned cmd-help continuations. +cmdhelp goodargs message suppressed for the no-args usage-display path +(punk/ns cmdhelp.test cmdhelp_leader_required_no_args_plain_usage; live 'i string is' +verified clean). + +Suites: punk/args 148 (147 pass/1 pre-existing skip) + punk/ns 49 pass on tcl903 and +tcl87; full source-tree suite 707 tests with only the documented exec-14.3 baseline +failure. No non-GAP expectations weakened (characterization updates assert the fixed +behaviour per acceptance). + +Post-achievement follow-up (2026-07-10, user-reported): 'i punk::args::define' timing +roughly doubled (~5.2/4.0s -> ~9.9/8.1s). Cause was the first cut of the item-5 fix, +not the deferral: for a definition with required args and no supplied words, the +advisory parse (at -errorstyle standard/basic) rendered the full usage table inside the +raised error, which the new plain-usage branch then discarded and re-rendered - two +table builds per 'i'. Fixed by running the no-supplied-words advisory parse with +-errorstyle minimal (cheap failure, single render). Like-for-like on punk91 src: +pre-change 5299/4083ms, post-fix 5360/4176ms (parity within noise; residual cost is +table/ANSI construction, explicitly out of G-046 scope). The define argdoc's single +deferred ${[helpers::example ...]} expands once into the display cache as designed. + ## Alternatives considered - Reentrancy guard alone (without deferral) for item 1 - rejected as primary: it fixes @@ -93,6 +176,22 @@ could do with a fix up at some stage." - Perf evidence baseline (2026-07-09): mark_columns first parse ~3.94s / subsequent 60us (punk902z dev, this machine). tclcore moduledoc commands (e.g. ::lseq with helpers::example blocks) are candidates for the second timing datapoint. +- Pre-existing adjacent defect found 2026-07-10 (assessed, not in this goal's scope; candidate + goal flagged): 'i ' from a namespace that hasn't loaded the ensemble's argdocs + renders the autogenerated ensemble help without subcommand-help markers/choicelabels. + Root cause: punk::ns::arginfo (cmd_traverse entry, ns-999999.0a1.0.tm ~4886) calls + punk::args::update_definitions with [namespace qualifiers $origin] (e.g. ::punk::ansi for + ensemble ::punk::ansi::ansistring), but the subcommand argdocs live in the separately + registered pkgns ::punk::ansi::ansistring (its ::argdoc child), which never gets loaded on + that path; punk::args::ensemble_subcommands_definition (and the older inline path in + punk::ns::arginfo ~4610) then run their punk::args::id_exists checks against unloaded ids. + Self-heals once anything loads the ns (autodef regenerates per 'i' call). No existing test + pins the ordering. G-046 relevance: (a) 'i ' output/timing used during G-046 + verification depends on session load history - pin update_definitions state explicitly in + any such test/timing; (b) strengthens the item-1 classification note: autodef generation + computes choicelabels eagerly via punk::args::synopsis -return summary per subcommand, so + @cmd -summary and synopsis inputs must resolve from the cheap parse-spec path, never force + full -help expansion per subcommand. - Related: G-045 (authoring ergonomics - same resolve neighbourhood; free sequencing), G-041 (form matching will exercise resolve paths more heavily - deferral lightens its per-form attempts), G-044 (completion wants fast parse-spec resolution). diff --git a/punkproject.toml b/punkproject.toml index f325b1d6..13f09baf 100644 --- a/punkproject.toml +++ b/punkproject.toml @@ -1,3 +1,3 @@ [project] name = "punkshell" -version = "0.4.14" +version = "0.4.15" diff --git a/src/modules/punk/args-999999.0a1.0.tm b/src/modules/punk/args-999999.0a1.0.tm index 2259a37a..b826d4dc 100644 --- a/src/modules/punk/args-999999.0a1.0.tm +++ b/src/modules/punk/args-999999.0a1.0.tm @@ -1266,6 +1266,7 @@ tcl::namespace::eval punk::args { variable rawdef_cache_about variable id_cache_rawdef variable rawdef_cache_argdata + variable argdefcache_display if {[id_exists $id]} { if {!$quiet} { puts stderr "punk::args::undefine clearing existing data for id:$id" @@ -1274,11 +1275,13 @@ tcl::namespace::eval punk::args { set deflist [dict get $id_cache_rawdef $id] dict unset rawdef_cache_about $deflist dict unset rawdef_cache_argdata $deflist + dict unset argdefcache_display $deflist dict unset id_cache_rawdef $id } else { dict for {k v} $rawdef_cache_argdata { if {[dict get $v id] eq $id} { dict unset rawdef_cache_argdata $k + dict unset argdefcache_display $k } } dict for {k v} $rawdef_cache_about { @@ -1314,16 +1317,390 @@ tcl::namespace::eval punk::args { proc undefine_deflist {deflist} { variable rawdef_cache_about variable id_cache_rawdef - variable rawdef_cache_argdata + variable rawdef_cache_argdata + variable argdefcache_display if {[dict exists $rawdef_cache_about $deflist -id]} { set id [dict get $rawdef_cache_about $deflist -id] dict unset rawdef_cache_about $deflist dict unset rawdef_cache_argdata $deflist + dict unset argdefcache_display $deflist dict unset id_cache_rawdef $id } } + # -- --- --- --- --- --- --- --- --- --- --- + # G-046 display-field deferral + # Expansion of ${...} content landing in display-only fields (-help on @cmd/@examples + # and argument records, @formdisplay -header/-body) is deferred out of resolve: the + # parse spec stores inert tokens plus a DISPLAY_DEFERRED map, and display consumers + # (arg_error, eg, resolved_def) expand on demand via private::expand_display_fields. + # -choicelabels stays eager - punk::ns reads it directly from parse specs during the + # subcommand walk. + variable argdefcache_display [tcl::dict::create] ;#display-expanded specs (non-dynamic definitions), keyed by raw deflist + variable display_expanding [list] ;#definition ids currently being display-expanded (reentrancy guard) + + proc private::split_definition_records {optionspecs} { + #definition text -> list of records (factored out of resolve - G-046) + #records are delimited by newlines, but multiline values are allowed if properly + #quoted/braced - 'info complete' (on ansi-stripped accumulation) finds record ends. + set records [list] + set linebuild "" + set linelist [split $optionspecs \n] + set record_base_indent "" ;#indent of first line in the record e.g a parameter or @directive record which will often have subsequent lines further indented. + #find the first record's base indent + foreach ln $linelist { + if {[tcl::string::trim $ln] eq ""} {continue} + regexp {(\s*).*} $ln _all record_base_indent + break ;#break at first non-empty + } + set in_record_continuation 0 + if {[catch {package require punk::ansi} errM]} { + set has_punkansi 0 + } else { + set has_punkansi 1 + } + set record_id 0 + set record_line 0 ;#incremented at each incomplete record, set to zero after processing a complete record + foreach rawline $linelist { + set record_so_far [tcl::string::cat $linebuild $rawline] + #ansi colours can stop info complete from working (contain square brackets) + #review - when exactly are ansi codes allowed/expected in record lines. + # - we might reasonably expect them in default values or choices or help strings + # - square brackets in ansi aren't and can't be escaped if they're to work as literals in the data. + # - eg set line "set x \"a[a+ red]red[a]\"" + # - 'info complete' will report 0, and subst would require -nocommand option or it will complain of missing close-bracket + if {$has_punkansi} { + set test_record [punk::ansi::ansistrip $record_so_far] + } else { + #review + #we only need to strip enough to stop interference with 'info complete' + set test_record [string map [list \x1b\[ ""] $record_so_far] + } + if {![tcl::info::complete $test_record]} { + if {$in_record_continuation} { + #trim only the whitespace corresponding to the record indent - not all whitespace on left + #this allows alignment of multiline help strings to left margin whilst maintaining a visual indent in source form. + #(note string first "" $str is fast and returns -1) + if {[tcl::string::first $record_base_indent $rawline] == 0} { + set trimmedline [tcl::string::range $rawline [tcl::string::length $record_base_indent] end] + append linebuild $trimmedline \n + } else { + append linebuild $rawline \n + } + } else { + assert {$record_line == 0} punk::args::private::split_definition_records record_line + regexp {(\s*).*} $rawline _all record_base_indent + append linebuild $rawline \n + set in_record_continuation 1 + } + incr record_line + } else { + #either we're on a single line record, or last line of multiline record + if {$record_line != 0} { + if {[tcl::string::first $record_base_indent $rawline] == 0} { + set trimmedline [tcl::string::range $rawline [tcl::string::length $record_base_indent] end] + append linebuild $trimmedline + } else { + append linebuild $rawline + } + } else { + append linebuild $rawline + } + lappend records $linebuild + set linebuild "" + #prep for next record + set in_record_continuation 0 + incr record_id + set record_line 0 + } + } + if {$in_record_continuation} { + puts stderr "punk::args::resolve incomplete record:" + puts stderr "$linebuild" + } + return $records + } + + proc private::classify_display_tokens {placeholdertext tokenlist} { + #G-046: return the subset of tokenlist whose tokens land in display-only field + #values. Display-only: -help on @cmd/@examples and on argument records, and + #@formdisplay -header/-body. Everything else (record names, -choices, -default, + #-choicelabels, @form -synopsis, directive settings) is parse/synopsis-relevant + #and stays eagerly expanded. + set display_tokens [list] + if {![llength $tokenlist]} { + return $display_tokens + } + set records [split_definition_records $placeholdertext] + foreach rec $records { + set trimrec [tcl::string::trim $rec] + switch -- [tcl::string::index $trimrec 0] { + "" - # {continue} + } + if {[catch {set record_values [lassign $trimrec firstword]}]} { + continue ;#unparseable here - real errors surface in resolve's own record loop + } + if {[llength $record_values] % 2 != 0} { + continue + } + switch -glob -- $firstword { + @cmd - @examples { + set displaykeys {-help} + } + @formdisplay { + set displaykeys {-header -body} + } + @* { + set displaykeys {} + } + default { + #argument record + set displaykeys {-help} + } + } + if {![llength $displaykeys]} {continue} + foreach {k v} $record_values { + if {$k in $displaykeys && [string first \x01PADEFER $v] >= 0} { + foreach tk $tokenlist { + if {$tk ni $display_tokens && [string first $tk $v] >= 0} { + lappend display_tokens $tk + } + } + } + } + } + return $display_tokens + } + + proc private::mask_display_params {normargs defspace is_dynamic} { + #G-046 phases 0-2: extract ${...} params without evaluation (tstr -eval 0), + #classify which land in display-only fields, and return + # [list ] + #Blocks without display params are returned untouched (original text) so the + #normal tstr path processes them bit-identically. + set blockparts [list] + set tokenlist [list] + set n 0 + set phtext "" + foreach block $normargs { + if {[string first \$\{ $block] >= 0} { + lassign [punk::args::lib::tstr -return list -eval 0 -undent 0 $block] ptlist paramlist + set btokens [list] + set phblock "" + set i 0 + foreach param $paramlist { + set tk [tcl::string::cat \x01 PADEFER [incr n] \x02] + append phblock [lindex $ptlist $i] $tk + lappend btokens $tk + incr i + } + append phblock [lindex $ptlist $i] + lappend blockparts [list 1 $ptlist $paramlist $btokens] + lappend tokenlist {*}$btokens + append phtext $phblock \n + } else { + lappend blockparts [list 0 {} {} {}] + append phtext $block \n + } + } + if {![llength $tokenlist]} { + return [list $normargs {}] + } + set display_tokens [classify_display_tokens $phtext $tokenlist] + if {![llength $display_tokens]} { + return [list $normargs {}] + } + set deferred [tcl::dict::create] + set outblocks [list] + foreach block $normargs bp $blockparts { + lassign $bp hasparams ptlist paramlist btokens + if {!$hasparams} { + lappend outblocks $block + continue + } + set block_display 0 + foreach tk $btokens { + if {$tk in $display_tokens} { + set block_display 1 + break + } + } + if {!$block_display} { + lappend outblocks $block + continue + } + set masked "" + set i 0 + foreach param $paramlist tk $btokens { + append masked [lindex $ptlist $i] + if {$tk in $display_tokens} { + append masked $tk + tcl::dict::set deferred $tk [tcl::dict::create source $param defspace $defspace dynamic $is_dynamic] + } else { + #parse-relevant param - reconstruct for normal tstr evaluation + append masked [tcl::string::cat \$\{ $param \}] + } + incr i + } + append masked [lindex $ptlist $i] + lappend outblocks $masked + } + return [list $outblocks $deferred] + } + + proc private::expand_display_fields {spec} { + #G-046: expand deferred display-field content of a parse spec on demand. + #Returns the spec with display fields expanded and DISPLAY_DEFERRED emptied. + #Non-dynamic expansions are cached (keyed by raw deflist); @dynamic definitions + #re-expand each call so provider changes refresh (matching resolve semantics). + if {![tcl::dict::exists $spec DISPLAY_DEFERRED]} { + return $spec + } + set map [tcl::dict::get $spec DISPLAY_DEFERRED] + if {![tcl::dict::size $map]} { + return $spec + } + set id [tcl::dict::get $spec id] + upvar ::punk::args::id_cache_rawdef id_cache_rawdef + upvar ::punk::args::rawdef_cache_about rawdef_cache_about + upvar ::punk::args::argdefcache_display argdefcache_display + set deflist "" + set spec_dynamic 0 + if {[tcl::dict::exists $id_cache_rawdef $id]} { + set deflist [tcl::dict::get $id_cache_rawdef $id] + if {[tcl::dict::exists $rawdef_cache_about $deflist -dynamic]} { + set spec_dynamic [tcl::dict::get $rawdef_cache_about $deflist -dynamic] + } + } + if {!$spec_dynamic && $deflist ne "" && [tcl::dict::exists $argdefcache_display $deflist]} { + return [tcl::dict::get $argdefcache_display $deflist] + } + upvar ::punk::args::display_expanding display_expanding + if {$id in $display_expanding} { + #reentrancy: deferred content is (indirectly) rendering help for an id whose + #display expansion is already in progress. Substitute the raw ${...} sources + #instead of evaluating - resolves cleanly, never loops. Not cached. + return [expand_display_slots $spec $map raw] + } + lappend display_expanding $id + try { + set spec [expand_display_slots $spec $map eval] + } finally { + set posn [lsearch -exact $display_expanding $id] + set display_expanding [lreplace $display_expanding $posn $posn] + } + tcl::dict::set spec DISPLAY_DEFERRED {} + if {!$spec_dynamic && $deflist ne ""} { + tcl::dict::set argdefcache_display $deflist $spec + } + return $spec + } + + proc private::expand_display_slots {spec map mode} { + #walk the display-only slots of a spec and expand deferred tokens in place + foreach infokey {cmd_info examples_info} { + if {[tcl::dict::exists $spec $infokey -help]} { + set v [tcl::dict::get $spec $infokey -help] + if {[string first \x01PADEFER $v] >= 0} { + tcl::dict::set spec $infokey -help [expand_deferred_text $v $map $mode] + } + } + } + tcl::dict::for {fid fdict} [tcl::dict::get $spec FORMS] { + if {[tcl::dict::exists $fdict FORMDISPLAY]} { + tcl::dict::for {k v} [tcl::dict::get $fdict FORMDISPLAY] { + if {[string first \x01PADEFER $v] >= 0} { + tcl::dict::set spec FORMS $fid FORMDISPLAY $k [expand_deferred_text $v $map $mode] + } + } + } + tcl::dict::for {argname arginfo} [tcl::dict::get $fdict ARG_INFO] { + if {[tcl::dict::exists $arginfo -help]} { + set v [tcl::dict::get $arginfo -help] + if {[string first \x01PADEFER $v] >= 0} { + tcl::dict::set spec FORMS $fid ARG_INFO $argname -help [expand_deferred_text $v $map $mode] + } + } + } + } + return $spec + } + + proc private::expand_deferred_text {text map mode} { + #replace deferred display tokens in a field value. + #mode 'eval': substitute each token's ${...} source in its recorded definition + #namespace; @dynamic sources get the second substitution round (the ${$DYN_X} + #idiom). mode 'raw': substitute the raw ${...} source text (reentrancy path). + #Multiline results align continuation lines with the leading whitespace of the + #token's line - the same 'line' paramindents treatment punk::args::lib::tstr + #applies to eagerly expanded params (G-046 item 2 for the @dynamic path). + set out "" + set firstline 1 + foreach ln [split $text \n] { + if {!$firstline} { + append out \n + } + set firstline 0 + if {[string first \x01PADEFER $ln] < 0} { + append out $ln + continue + } + regexp {^(\s*)} $ln _all lineindent + while {[regexp -indices {\x01PADEFER[0-9]+\x02} $ln tokenposns]} { + lassign $tokenposns t0 t1 + set token [string range $ln $t0 $t1] + set replacement "" + if {[tcl::dict::exists $map $token]} { + set param [tcl::dict::get $map $token source] + set defspace [tcl::dict::get $map $token defspace] + set dynamic [tcl::dict::get $map $token dynamic] + if {$defspace eq ""} { + set defspace :: + } + if {$mode eq "raw"} { + set replacement [tcl::string::cat \$\{ $param \}] + } else { + if {[catch {tcl::namespace::eval $defspace [list ::subst $param]} presult]} { + #tstr behaviour on evaluation error: substitute the raw placeholder source + set replacement [tcl::string::cat \$\{ $param \}] + } else { + if {$dynamic && [string first \$\{ $presult] >= 0} { + #@dynamic second-round substitution + lassign [punk::args::lib::tstr -return list -eval 0 -undent 0 $presult] ptlist2 paramlist2 + set r2 "" + set i 0 + foreach param2 $paramlist2 { + append r2 [lindex $ptlist2 $i] + if {[catch {tcl::namespace::eval $defspace [list ::subst $param2]} p2result]} { + append r2 [tcl::string::cat \$\{ $param2 \}] + } else { + append r2 $p2result + } + incr i + } + append r2 [lindex $ptlist2 $i] + set presult $r2 + } + set replacement $presult + } + } + } + if {$lineindent ne "" && [string first \n $replacement] >= 0} { + set rlines [split $replacement \n] + set replacement [lindex $rlines 0] + foreach rl [lrange $rlines 1 end] { + append replacement \n $lineindent $rl + } + } + set ln [string replace $ln $t0 $t1 $replacement] + } + append out $ln + } + return $out + } + # -- --- --- --- --- --- --- --- --- --- --- + lappend PUNKARGS [list { @id -id ::punk::args::resolve @cmd -name punk::args::resolve\ @@ -1403,12 +1780,14 @@ tcl::namespace::eval punk::args { lappend normargs [tcl::string::map {\r\n \n} $a] } + #G-046: -help section processing is sometimes expensive (e.g tcl syntax + #highlighted examples) and isn't required for parsing of arguments. Display-only + #${...} params are masked with inert tokens here and expanded on demand at + #display time (private::expand_display_fields) with a separate cache. + set deferred_display {} + lassign [private::mask_display_params $normargs $defspace 0] normargs deferred_display + set optionspecs [list] - #REVIEW - whilst this is only done once for each command definition, the -help section processing is sometimes expensive, - #and isn't required for parsing of arguments, so it unnecessarily slows first use of a command that uses punk::args and is heavily documented, - #especially if it has tcl syntax highlighted examples. - #- ideally we would delay expansion of -help sections until needed for display, - #and use a different cache key for the parsing vs display versions of the resolved definition. foreach block $normargs { if {[string first \$\{ $block] >= 0} { if {$defspace ne ""} { @@ -1443,10 +1822,11 @@ tcl::namespace::eval punk::args { #} } else { + set deferred_display {} if {[tcl::dict::exists $argdefcache_unresolved $cache_key]} { #cached - so first round of substitution already done set pt_params [tcl::dict::get $argdefcache_unresolved $cache_key] - lassign $pt_params ptlist paramlist + lassign $pt_params ptlist paramlist deferred_display set optionspecs "" #subst is only being called on the parameters (contents of ${..}) foreach pt $ptlist param $paramlist { @@ -1463,6 +1843,11 @@ tcl::namespace::eval punk::args { lappend normargs [tcl::string::map {\r\n \n} $a] } + #G-046: mask display-only ${...} params (see non-dynamic branch note). + #Masked at round 1, so display payloads (including their round-2 + #${$DYN_X} providers) never run during argument resolution. + lassign [private::mask_display_params $normargs $defspace 1] normargs deferred_display + set optionspecs [list] foreach block $normargs { if {[string first \$\{ $block] >= 0} { @@ -1498,8 +1883,8 @@ tcl::namespace::eval punk::args { append optionspecs $pt [uplevel $LVL [list ::subst $param]] } } - #key is the raw def, value is the 2 element list of textparts, paramparts - tcl::dict::set argdefcache_unresolved $cache_key $pt_params + #key is the raw def, value is the list of textparts, paramparts (and the G-046 deferred display map) + tcl::dict::set argdefcache_unresolved $cache_key [list {*}$pt_params $deferred_display] } else { #wasn't really a 'dynamic' definition - no 2nd round parameter substitution in definition puts stderr "punk::args::resolve - bad @dynamic tag for id:$id - no 2nd round substitution required" @@ -1557,141 +1942,7 @@ tcl::namespace::eval punk::args { #set opt_solos [list] #first process dashed and non-dashed record names without regard to whether non-dashed are at the beginning or end - set records [list] - set linebuild "" - - set linelist [split $optionspecs \n] - set record_base_indent "" ;#indent of first line in the record e.g a parameter or @directive record which will often have subsequent lines further indented. - #find the first record's base indent - foreach ln $linelist { - if {[tcl::string::trim $ln] eq ""} {continue} - regexp {(\s*).*} $ln _all record_base_indent - break ;#break at first non-empty - } - #puts "indent1:[ansistring VIEW $record_base_indent]" - set in_record_continuation 0 - if {[catch {package require punk::ansi} errM]} { - set has_punkansi 0 - } else { - set has_punkansi 1 - } - set record_id 0 - set record_line 0 ;#incremented at each incomplete record, set to zero after processing a complete record - foreach rawline $linelist { - #puts stderr "$record_line $rawline" - #XXX - #set rawtrimmed [string trim $rawline] - #if {$in_record_continuation && $rawtrimmed ne "" && [string index $rawtrimmed 0] ni [list "\}" {"} "#"]} { - # regexp {(\s*).*} $rawline _ rawline_indent - # if {[string length $rawline_indent] <= [string length $record_base_indent]} { - # lappend records $linebuild - # set linebuild "" - # #prep for next record - # set in_record_continuation 0 - # incr record_id - # set record_line 0 - # } - #} - set record_so_far [tcl::string::cat $linebuild $rawline] - #ansi colours can stop info complete from working (contain square brackets) - #review - when exactly are ansi codes allowed/expected in record lines. - # - we might reasonably expect them in default values or choices or help strings - # - square brackets in ansi aren't and can't be escaped if they're to work as literals in the data. - # - eg set line "set x \"a[a+ red]red[a]\"" - # - 'info complete' will report 0, and subst would require -nocommand option or it will complain of missing close-bracket - if {$has_punkansi} { - set test_record [punk::ansi::ansistrip $record_so_far] - } else { - #review - #we only need to strip enough to stop interference with 'info complete' - set test_record [string map [list \x1b\[ ""] $record_so_far] - } - if {![tcl::info::complete $test_record]} { - #append linebuild [string trimleft $rawline] \n - if {$in_record_continuation} { - #incr record_line - #if {$record_line == 1} { - # #first continuation line sets the indent - #} - - #/// - #if {[tcl::string::first "$record_base_indent " $rawline] == 0} { - # set trimmedline [tcl::string::range $rawline [tcl::string::length "$record_base_indent "] end] - # append linebuild $trimmedline \n - #} else { - # append linebuild $rawline \n - #} - - if {[tcl::string::first $record_base_indent $rawline] == 0} { - set trimmedline [tcl::string::range $rawline [tcl::string::length $record_base_indent] end] - append linebuild $trimmedline \n - } else { - append linebuild $rawline \n - } - - #trim only the whitespace corresponding to last record indent or record_base_indent + 4 spaces - not all whitespace on left - #this allows alignment of multiline help strings to left margin whilst maintaining a visual indent in source form. - #Aligning subsequent lines with the record, or aligning 4 spaces in are equivalent. - #ie to indent lines further - whitespace should be added 4+ columns in from the record-line start position. - #(this leaves an oddity if indenting is only 1 2 or 3 spaces mixed with longer indents as we don't check for it.. REVIEW) - #(note string first "" $str is fast and returns -1) - #if {[tcl::string::first "$record_base_indent " $rawline] == 0} { - # set trimmedline [tcl::string::range $rawline [tcl::string::length "$record_base_indent "] end] - # append linebuild $trimmedline \n - #} elseif {[tcl::string::first $record_base_indent $rawline] == 0} { - # set trimmedline [tcl::string::range $rawline [tcl::string::length $record_base_indent] end] - # append linebuild $trimmedline \n - #} else { - # append linebuild $rawline \n - #} - } else { - assert {$record_line == 0} punk::args::resolve record_line - regexp {(\s*).*} $rawline _all record_base_indent - #puts "indent: [ansistring VIEW -lf 1 $record_base_indent]" - #puts "indent from rawline:$rawline " - append linebuild $rawline \n - set in_record_continuation 1 - } - incr record_line - } else { - #either we're on a single line record, or last line of multiline record - - if {$record_line != 0} { - #trim only the whitespace corresponding to record_base_indent or record_base_indent + 4 spaces - not all whitespace on left - #if {[tcl::string::first "$record_base_indent " $rawline] == 0} { - # set trimmedline [tcl::string::range $rawline [tcl::string::length "$record_base_indent "] end] - # append linebuild $trimmedline - #} elseif {[tcl::string::first $record_base_indent $rawline] == 0} { - # set trimmedline [tcl::string::range $rawline [tcl::string::length $record_base_indent] end] - # append linebuild $trimmedline - #} else { - # append linebuild $rawline - #} - if {[tcl::string::first $record_base_indent $rawline] == 0} { - set trimmedline [tcl::string::range $rawline [tcl::string::length $record_base_indent] end] - append linebuild $trimmedline - } else { - append linebuild $rawline - } - } else { - append linebuild $rawline - } - lappend records $linebuild - set linebuild "" - #prep for next record - set in_record_continuation 0 - incr record_id - set record_line 0 - } - } - if {$in_record_continuation} { - puts stderr "punk::args::resolve incomplete record:" - puts stderr "$linebuild" - } - #puts stderr 1[lindex $records 1] - #puts stderr 4[lindex $records 4] - #puts stderr 5[lindex $records 5] - #puts stderr 6[lindex $records 6] + set records [private::split_definition_records $optionspecs] set cmd_info {} set package_info {} @@ -1854,7 +2105,9 @@ tcl::namespace::eval punk::args { #however.. as it stands we have define @dynamic making *immediate* resolutions .. is that really desirable? if {[dict exists $at_specs -id]} { - set copyfrom [get_spec [dict get $at_specs -id]] + #G-046: expand any deferred display content of the source spec - + #its deferred map doesn't travel with the copied fields. + set copyfrom [private::expand_display_fields [get_spec [dict get $at_specs -id]]] #we don't copy the @id info from the source #for now we only copy across if nothing set.. #todo - bring across defaults for empty keys at targets? @@ -3169,7 +3422,11 @@ tcl::namespace::eval punk::args { } id_info $id_info {*}{ } FORMS $F {*}{ } form_names [dict keys $F] {*}{ - } form_info $form_info {*}{ + } form_info $form_info {*}[ + # G-046: DISPLAY_DEFERRED maps inert display tokens -> {source <${...}-content> defspace dynamic } + # for display-only field content whose expansion was deferred out of parsing. + # Expanded on demand by private::expand_display_fields (empty when nothing deferred). + ] DISPLAY_DEFERRED $deferred_display {*}{ } ] @@ -3404,6 +3661,9 @@ tcl::namespace::eval punk::args { set deflist [tcl::dict::get $id_cache_rawdef $realid] set specdict [uplevel 1 [list ::punk::args::resolve {*}$deflist]] + #G-046: resolved_def re-emits definition text from the spec - deferred display + #tokens must never leak into consumer definitions, so expand first. + set specdict [private::expand_display_fields $specdict] set opt_form [dict get $opts -form] if {[string is integer -strict $opt_form]} { @@ -4467,6 +4727,11 @@ tcl::namespace::eval punk::args { #bas ic recursion blocker variable arg_error_isrunning 0 proc arg_error {msg spec_dict args} { + #G-046: specs may arrive with display-only field content deferred (parse specs + #travel in error options as -argspecs). Expand for display before rendering. + if {[tcl::dict::exists $spec_dict DISPLAY_DEFERRED] && [tcl::dict::size [tcl::dict::get $spec_dict DISPLAY_DEFERRED]]} { + set spec_dict [private::expand_display_fields $spec_dict] + } #todo - test a configurable flag (in the CALLER) for whether to do a faster return on the unhappy path. #accept an option here so that we can still use full output for usage requests. #This may be desired for codebases where tests based on 'catch' are used on procs that parse with punk::args @@ -4800,7 +5065,19 @@ tcl::namespace::eval punk::args { if {$use_table} { $t configure_header $h -colspans $arg_colspans -values [list Description: $cmdhelp_display] } else { - lappend errlines "Description: $cmdhelp_display" + #align continuation lines under the first line as the table renderer's + #cell does - relative indents among continuations preserved (G-046 item 4) + set desclines [split $cmdhelp_display \n] + set labelpad [string repeat " " [string length "Description: "]] + set descjoined [lindex $desclines 0] + foreach dline [lrange $desclines 1 end] { + if {$dline eq ""} { + append descjoined \n + } else { + append descjoined \n $labelpad $dline + } + } + lappend errlines "Description: $descjoined" } incr h } @@ -4819,7 +5096,7 @@ tcl::namespace::eval punk::args { if {$use_table} { $t configure_header $h -colspans $arg_colspans -values [list Example: $example_display] } else { - lappend errlines "Example: $docurl_display" + lappend errlines "Example: $example_display" } incr h } @@ -10653,9 +10930,17 @@ tcl::namespace::eval punk::args { } tcl::dict::set $dname $argname_or_ident $existing } else { - #test: choice_multielement_clause - lset existing $element_index $chosen - tcl::dict::set $dname $argname_or_ident $existing + if {$clause_size == 1} { + #single-element clause - value in $dname is the plain (non list-wrapped) string. + #overwrite wholesale: lset would treat the scalar as a list and list-quote + #values needing quoting (backslashes/spaces) e.g {\Deleted}, giving a different + #value shape than exact input (G-046 item 3) + tcl::dict::set $dname $argname_or_ident $chosen + } else { + #test: choice_multielement_clause + lset existing $element_index $chosen + tcl::dict::set $dname $argname_or_ident $existing + } } } else { if {$is_multiple} { @@ -10965,6 +11250,7 @@ tcl::namespace::eval punk::args { if {$spec eq ""} { return } + set spec [private::expand_display_fields $spec] ;#G-046 @examples -help may be deferred if {[dict exists $spec examples_info -help]} { set egdata [dict get $spec examples_info -help] return [punk::args::helpers::strip_nodisplay_lines $egdata] diff --git a/src/modules/punk/args-buildversion.txt b/src/modules/punk/args-buildversion.txt index 84bd13eb..ba887786 100644 --- a/src/modules/punk/args-buildversion.txt +++ b/src/modules/punk/args-buildversion.txt @@ -1,6 +1,7 @@ -0.4.1 +0.4.2 #First line must be a semantic version number #all other lines are ignored. +#0.4.2 - G-046: display-field deferral - resolve no longer tstr-expands display-only content during argument resolution: ${...} in -help (@cmd/@examples/argument records) and @formdisplay -header/-body is masked with inert tokens (spec key DISPLAY_DEFERRED) and expanded on demand at display time (arg_error/eg/resolved_def/@default-copyfrom hooks; separate argdefcache_display cache for non-dynamic defs; @dynamic display content re-expands per render preserving provider refresh). First parse of heavily documented commands drops accordingly (punk::ansi::mark_columns ~4.3s -> ~12ms; tclcore ::lseq resolve ~184ms -> ~2ms) and -help content that calls punk::args-parsing commands (including against its own id) no longer stalls or loops - plus a display-time reentrancy guard (raw ${...} source substituted on nested expansion of the same id). Record splitter factored to private::split_definition_records. Also: @dynamic second-round multiline substitutions into deferred fields now get the 'line' paramindents alignment (rendering_atdynamic_multiline_help_insertion GAP flipped); prefix/alias choice normalization writeback no longer list-quotes single-element-clause values ({\Deleted} shape bug, choicegroups_imap_prefix_shape GAP flipped); -return string renderer aligns cmd-help continuations under the Description label and its Example line shows the example instead of the doc url. Tests: deferredhelp.test (new), rendering.test/choicegroups.test updated per G-046 acceptance #0.4.1 - fixed ensemble_subcommands_definition building the definition against unloaded argdocs: subhelp/doctype-punkargs choiceinfo entries and synopsis choicelabels were omitted for documented subcommands whose registered namespace (::punk::args::register::NAMESPACES) had not yet been lazily loaded - e.g first 'i ansistring' in a fresh shell rendered the autogenerated ensemble help without subcommand-help markers until something else loaded ::punk::ansi::ansistring. The generator now runs update_definitions for the namespaces its id_exists checks could resolve in before testing them (tests: punk/args ensembledef.test, punk/ns cmdhelp.test cmdhelp_ensemble_lazy_registered_argdocs) #0.4.0 - tidy-up (pre G-046): removed dead developer-test procs define2 and parseXXX and the unused ::punk::args::TEST definition; renamed the interactive Get_caller diagnostics test_get_dict/test_get_by_id/test_callers (+ test1 id) and lib::tstr_test_one with leading double underscores and excluded them from exports (punk::args::lib now exports {[a-z]*} instead of *). new punk::args::private namespace (namespace path to punk::args/lib/system) for internal helpers - _get_dict_can_assign_value, _split_type_expression, _check_clausecolumn, _synopsis_form_arg_display, _argerror_load_colours and lib::_parse_tstr_parts moved there with underscore prefixes dropped. documentation: PUNKARGS definitions added for every remaining exported command (errorstyle, undefine, undefine_deflist, raw_def, resolved_def_values, get_spec, is_dynamic, rawdef_id, rawdef_is_dynamic, id_exists, idaliases, set_idalias, unset_idalias, get_idalias, id_query, real_id, status, choiceword_match, get_dict; lib: string_is_dict, flatzip, zero_based_posns, choiceinfo_marks, indent, undent, undentleader, longestCommonPrefix, lunique; helpers and argdocbase B/N/I/NI); resolve's empty @cmd -help filled in; punk::args::argdocbase added to the registered definition namespaces #0.3.2 - Tcl 9.1 compatibility (TIP 746 removed expr behaviour from lseq operands): zero_based_posns now wraps its count-1 operand in expr diff --git a/src/modules/punk/ns-999999.0a1.0.tm b/src/modules/punk/ns-999999.0a1.0.tm index 2bd39952..d8cd7bed 100644 --- a/src/modules/punk/ns-999999.0a1.0.tm +++ b/src/modules/punk/ns-999999.0a1.0.tm @@ -5597,8 +5597,25 @@ y" {return quirkykeyscript} if {[llength $testresolved] == 1} { #only the command itself is in the args_resolved list - so we can't resolve to a deeper subcommand ledit queryargs -1 -1 {*}$args_remaining ;#prepend - if {[catch {punk::args::parse $queryargs -form $opt_form -errorstyle $estyle withid $rootdoc} parseresult]} { - if {$opt_return eq "tableobject"} { + #advisory goodargs parse (marking of supplied words). With NO supplied args + #a failure only reflects missing required leaders/values - nothing to mark and + #nothing wrong with the user's (absent) input, so we show plain usage in the + #info scheme instead of the internal-looking parse error (G-046 item 5). + #Use -errorstyle minimal for that case: the $estyle rendering would build the + #full usage table INSIDE the raised error only for it to be discarded here - + #doubling display time for large argdocs (e.g 'i punk::args::define'). + if {![llength $queryargs]} { + set advisory_estyle minimal + } else { + set advisory_estyle $estyle + } + if {[catch {punk::args::parse $queryargs -form $opt_form -errorstyle $advisory_estyle withid $rootdoc} parseresult]} { + if {![llength $queryargs]} { + if {!$scheme_received} { + dict set nextopts -scheme info + } + set result [punk::args::arg_error "" [punk::args::get_spec $rootdoc] {*}$nextopts -aserror 0] + } elseif {$opt_return eq "tableobject"} { set result [punk::args::arg_error "$parseresult" [punk::args::get_spec $rootdoc] {*}$nextopts -aserror 0] } else { set result $parseresult @@ -5679,8 +5696,25 @@ y" {return quirkykeyscript} } if {$origindoc ne ""} { #important not to use "-cache 1" for this parse - need to reflect dynamically updated ensembles etc - if {[catch {punk::args::parse $args_remaining -cache 0 -form $opt_form -errorstyle $estyle withid $origindoc} parseresult]} { - if {$opt_return eq "tableobject"} { + #advisory goodargs parse (marking of supplied words). With NO supplied trailing args + #a failure only reflects missing required leaders/values (e.g 'i string is') - nothing + #to mark and nothing wrong with the user's (absent) input, so we show plain usage in + #the info scheme instead of the internal-looking parse error (G-046 item 5). + #Use -errorstyle minimal for that case: the $estyle rendering would build the full + #usage table INSIDE the raised error only for it to be discarded here - doubling + #display time for large argdocs (e.g 'i punk::args::define'). + if {![llength $args_remaining]} { + set advisory_estyle minimal + } else { + set advisory_estyle $estyle + } + if {[catch {punk::args::parse $args_remaining -cache 0 -form $opt_form -errorstyle $advisory_estyle withid $origindoc} parseresult]} { + if {![llength $args_remaining]} { + if {!$scheme_received} { + dict set nextopts -scheme info + } + set result [punk::args::arg_error "" [punk::args::get_spec $origindoc] {*}$nextopts -aserror 0] + } elseif {$opt_return eq "tableobject"} { set result [punk::args::arg_error "$parseresult" [punk::args::get_spec $origindoc] {*}$nextopts -aserror 0] } else { set result $parseresult diff --git a/src/modules/punk/ns-buildversion.txt b/src/modules/punk/ns-buildversion.txt index c02867ef..c87e704f 100644 --- a/src/modules/punk/ns-buildversion.txt +++ b/src/modules/punk/ns-buildversion.txt @@ -1,6 +1,7 @@ -0.1.3 +0.1.4 #First line must be a semantic version number #all other lines are ignored. +#0.1.4 - G-046 item 5: cmdhelp's advisory goodargs parse failing with NO supplied argument words (e.g 'i string is' where the definition requires leaders) now shows plain info-scheme usage instead of the internal-looking "Bad number of leading values for punk::args::parse ..." error output (both the alias path and the main path; error display for supplied-but-invalid words unchanged). The no-supplied-words advisory parse runs with -errorstyle minimal so its failure doesn't render the full usage table inside the discarded error - large argdocs (e.g 'i punk::args::define') render the table once, not twice (verified parity with pre-G-046 timings: ~5.3s first/~4.1s repeat on punk91 src, table construction dominant). Test: cmdhelp.test cmdhelp_leader_required_no_args_plain_usage #0.1.3 - documentation-only: cmdhelp 'subcommand' argument help rewritten to match actual behaviour (was described as ensemble-subcommands-only; also covers tcl::oo methods and argument words, whose validity drives the info/error scheme and received-argument marking of the usage display) #0.1.2 - cmd_traverse subcommand walk resolves choice words via the shared punk::args::choiceword_match resolver (G-040 parity): -choiceprefixdenylist and -choiceprefixreservelist are now honoured in doc lookup (previously ignored - 'i ' could accept words parsing rejects), -choicealiases normalize to their canonical before choiceinfo lookup, and -nocase is honoured in the walk #0.1.1 - commented out five development trace puts in the doc-lookup machinery: "PROC auto def"/"ENSEMBLE auto def" (generate_autodef - emitted on STDOUT, polluting 'i'/'s' output in script/exec contexts), "cmd_traverse - skipping to documented subcommand" (space-form id path), "---> cmd_traverse ensembleparam" (ensemble -parameters traversal), and "cmd_traverse 10 ... - review" (fallthrough return). No functional change. diff --git a/src/tests/modules/punk/args/testsuites/args/choicegroups.test b/src/tests/modules/punk/args/testsuites/args/choicegroups.test index f7faa9dc..c640377e 100644 --- a/src/tests/modules/punk/args/testsuites/args/choicegroups.test +++ b/src/tests/modules/punk/args/testsuites/args/choicegroups.test @@ -114,17 +114,17 @@ namespace eval ::testspace { }\ -result [list 1 1 1 1 1 1] - test choicegroups_imap_prefix_listwrap_GAP {GAP: prefix-normalizing a choice containing a backslash stores a list-quoted value ({\Deleted}) where exact input stores the plain string - value shape depends on how the user typed it}\ + test choicegroups_imap_prefix_shape {prefix-normalizing a choice containing a backslash yields the same plain string as exact input (GAP flipped by the G-046 single-value writeback fix)}\ -setup $common -body { - #the prefix-normalization writeback (lset into the stored value) list-quotes - #values needing quoting (backslashes, spaces). Exact input bypasses the - #writeback and stays a plain string. Pinned as characterization pending a fix. + #was choicegroups_imap_prefix_listwrap_GAP: the single-value prefix-normalization + #writeback lset into the stored scalar, list-quoting values needing quoting + #(backslashes, spaces) - consumers got a one-element list ({\Deleted}) where + #exact input stored the plain string. set exact [dict get [dict get [punk::args::parse [list "\\Deleted"] withid ::testspace::cgimap] values] flagname] set viapfx [dict get [dict get [punk::args::parse [list "\\De"] withid ::testspace::cgimap] values] flagname] lappend result [expr {$exact eq "\\Deleted"}] - #current (GAP) behaviour: the prefix path yields a one-element list - lappend result [expr {$viapfx ne "\\Deleted"}] - lappend result [expr {[lindex $viapfx 0] eq "\\Deleted"}] + lappend result [expr {$viapfx eq "\\Deleted"}] + lappend result [expr {$viapfx eq $exact}] }\ -cleanup { }\ diff --git a/src/tests/modules/punk/args/testsuites/args/deferredhelp.test b/src/tests/modules/punk/args/testsuites/args/deferredhelp.test new file mode 100644 index 00000000..6ab4a127 --- /dev/null +++ b/src/tests/modules/punk/args/testsuites/args/deferredhelp.test @@ -0,0 +1,164 @@ +package require tcltest + +package require punk::args +package require punk::ansi + +#G-046 item 1: display-field deferral. Argument resolution must not expand ${...} +#content in display-only fields (-help on @cmd/@examples/argument records, +#@formdisplay bodies) - expansion happens at display time, separately cached. +#This removes the parse-time cost of heavily documented commands (punk::ansi's +#mark_columns argdoc: ~4s first parse) and the reentrancy class where -help content +#calls punk::args-parsing commands (including commands that parse against the very +#definition being resolved). + +namespace eval ::testspace { + namespace import ::tcltest::* + variable common { + set result "" + } + + variable DEFHELP_CALLS 0 + proc defhelp_probe {} { + variable DEFHELP_CALLS + incr DEFHELP_CALLS + return "PROBE_L1 probe text\nPROBE_L2 probe more" + } + + test deferredhelp_parse_skips_help_expansion {parsing never runs -help ${[...]} substitutions; display runs them once (non-dynamic display cache); parsing after display still doesn't}\ + -setup $common -body { + variable DEFHELP_CALLS + set DEFHELP_CALLS 0 + punk::args::define { + @id -id ::testspace::dhelp + @cmd -name testspace::dhelp -summary\ + "Deferred help fixture."\ + -help\ + "DFIRST line. + ${[::testspace::defhelp_probe]}" + @values -min 1 -max 1 + v1 -type string -help\ + "v1 help." + } + #define does not resolve - force resolution via a parse + punk::args::parse {avalue} withid ::testspace::dhelp + lappend result $DEFHELP_CALLS ;#0 - parse-time resolution skipped the -help substitution + #display renders the substituted content + set r [punk::ansi::ansistrip [punk::args::usage ::testspace::dhelp]] + lappend result $DEFHELP_CALLS ;#1 - expanded for display + lappend result [string match "*PROBE_L1*" $r] + #no internal deferral tokens leak into rendered output + lappend result [string match "*\x01*" $r] + #multiline substitution aligns at the insertion column (as eager tstr does) + set c0 -1 + set c1 -2 + foreach ln [split $r \n] { + if {[set ix [string first PROBE_L1 $ln]] >= 0} {set c0 $ix} + if {[set ix [string first PROBE_L2 $ln]] >= 0} {set c1 $ix} + } + lappend result [expr {$c1 - $c0}] + #display expansion is cached for non-dynamic definitions + punk::args::usage ::testspace::dhelp + lappend result $DEFHELP_CALLS ;#still 1 + #parsing again still doesn't touch it + punk::args::parse {bvalue} withid ::testspace::dhelp + lappend result $DEFHELP_CALLS ;#still 1 + }\ + -cleanup { + punk::args::undefine ::testspace::dhelp 1 + }\ + -result [list 0 1 1 0 0 1 1] + + variable DHSELF_CALLS 0 + proc dhself_helper {args} { + #parses against the definition whose -help invokes this proc + #(the punk::ansi::mark_columns reentrancy class) + variable DHSELF_CALLS + incr DHSELF_CALLS + set argd [punk::args::parse [list "helper-example"] withid ::testspace::dhself] + return "HELPER_RESULT [dict get [dict get $argd values] v1]" + } + + test deferredhelp_selfparsing_help_resolves {a -help substitution that parses against its OWN definition id resolves cleanly: never runs at parse time, and at display time the parse inside the helper uses the (already cached) parse spec}\ + -setup $common -body { + variable DHSELF_CALLS + set DHSELF_CALLS 0 + punk::args::define { + @id -id ::testspace::dhself + @cmd -name testspace::dhself -summary\ + "Self-parsing help fixture."\ + -help\ + "example output: + ${[::testspace::dhself_helper]}" + @values -min 1 -max 1 + v1 -type string -help\ + "v1 help." + } + punk::args::parse {plainvalue} withid ::testspace::dhself + lappend result $DHSELF_CALLS ;#0 + set r [punk::ansi::ansistrip [punk::args::usage ::testspace::dhself]] + lappend result $DHSELF_CALLS ;#1 + lappend result [string match "*HELPER_RESULT helper-example*" $r] + }\ + -cleanup { + punk::args::undefine ::testspace::dhself 1 + }\ + -result [list 0 1 1] + + test deferredhelp_self_display_reentrancy_guard {a -help substitution that renders usage for its OWN id resolves cleanly (raw ${...} sources substituted on the nested render) rather than looping}\ + -setup $common -body { + punk::args::define { + @id -id ::testspace::dhloop + @cmd -name testspace::dhloop -summary\ + "Self-rendering help fixture."\ + -help\ + "own usage follows: + ${[punk::args::usage ::testspace::dhloop]}" + @values -min 0 -max 0 + } + set r [punk::ansi::ansistrip [punk::args::usage ::testspace::dhloop]] + #completed without looping; nested render shows the raw source form + lappend result [expr {[string length $r] > 0}] + lappend result [string match "*own usage follows:*" $r] + lappend result [string match "*\x01*" $r] + }\ + -cleanup { + punk::args::undefine ::testspace::dhloop 1 + }\ + -result [list 1 1 0] + + variable DHDYN_CALLS 0 + proc dhdyn_provider {} { + variable DHDYN_CALLS + incr DHDYN_CALLS + return "DYNPROBE call$DHDYN_CALLS" + } + variable DYN_DHHELP {${[::testspace::dhdyn_provider]}} + + test deferredhelp_dynamic_display_refreshes {@dynamic deferred -help content is skipped at parse time and re-expanded on each display (provider refresh semantics preserved)}\ + -setup $common -body { + variable DHDYN_CALLS + set DHDYN_CALLS 0 + punk::args::define { + @dynamic + @id -id ::testspace::dhdyn + @cmd -name testspace::dhdyn -summary\ + "Dynamic deferred help fixture."\ + -help\ + "${$DYN_DHHELP}" + @values -min 0 -max 0 + } + punk::args::parse {} withid ::testspace::dhdyn + lappend result $DHDYN_CALLS ;#0 - provider untouched by parsing + set r1 [punk::ansi::ansistrip [punk::args::usage ::testspace::dhdyn]] + lappend result [string match "*DYNPROBE call1*" $r1] + set r2 [punk::ansi::ansistrip [punk::args::usage ::testspace::dhdyn]] + lappend result [string match "*DYNPROBE call2*" $r2] ;#re-expanded per display + punk::args::parse {} withid ::testspace::dhdyn + lappend result $DHDYN_CALLS ;#2 - parsing still doesn't run it + }\ + -cleanup { + punk::args::undefine ::testspace::dhdyn 1 + }\ + -result [list 0 1 1 2] +} +tcltest::cleanupTests ;#needed to produce test summary line. diff --git a/src/tests/modules/punk/args/testsuites/args/rendering.test b/src/tests/modules/punk/args/testsuites/args/rendering.test index 2fc24230..ac7aa6b1 100644 --- a/src/tests/modules/punk/args/testsuites/args/rendering.test +++ b/src/tests/modules/punk/args/testsuites/args/rendering.test @@ -191,8 +191,10 @@ namespace eval ::testspace { }\ -result [list 0 2 0 2] - test rendering_string_renderer_characterization {string renderer: arg-help aligned with +2 preserved; cmd-help first line joins the Description label while continuations sit flush-left (first-vs-rest offset is a known wart)}\ + test rendering_string_renderer_characterization {string renderer: arg-help aligned with +2 preserved; cmd-help continuations align under the first line on the Description label with relative indents preserved}\ -setup $common -body { + #cmd-help continuations previously landed flush-left (pinned as a wart); + #aligned under the first line by the G-046 item 4 fix. punk::args::define { @id -id ::testspace::rstring @cmd -name testspace::rstring -summary\ @@ -216,10 +218,8 @@ namespace eval ::testspace { if {[string match "Description:*" $ln]} {set descline $ln} } lappend result [string match "Description:*SFIRST*" $descline] - #... while continuations are flush-left with relative indent preserved - #(characterization: the string renderer does not re-indent cmd-help - # continuations under the label - first-vs-rest misalignment by design/wart) - lappend result [markercol $r SFLUSH] + #... and continuations align under the first line, +2 preserved between them + lappend result [expr {[markercol $r SFLUSH] - [markercol $r SFIRST]}] lappend result [expr {[markercol $r SPLUS2] - [markercol $r SFLUSH]}] }\ -cleanup { @@ -541,13 +541,13 @@ GFLUSH line" }\ -result [list 1 1 0 bluex-rejected-at-counter0 1 1 {mode bluex}] - test rendering_atdynamic_multiline_help_insertion_GAP {GAP: @dynamic double substitution of a MULTILINE result into -help loses insertion-column alignment for lines 2+ (they land 4 left; internal relative indents preserved)}\ + test rendering_atdynamic_multiline_help_insertion {@dynamic double substitution of a MULTILINE result into -help aligns all lines at the insertion column with internal relative indents preserved (GAP flipped by G-046 display deferral)}\ -setup $common -body { - #contrast with rendering_tstr_multiline_command_insertion where single-level - #${[cmd]} insertion aligns all lines at the insertion column. The @dynamic - #second-round substitution (plain subst of the cached parameters) has no - #paramindent re-alignment - a known fragility of the double-substitution - #path, pinned here pending a fix. + #was rendering_atdynamic_multiline_help_insertion_GAP: the second-round + #plain-subst path had no paramindent re-alignment (lines 2+ landed 4 left). + #-help content now defers to display time where the expansion applies the + #same 'line' paramindents treatment as single-level ${[cmd]} insertion + #(rendering_tstr_multiline_command_insertion). proc rdyn_multi {} { return "YINS_L1 first\nYINS_L2 second\n YINS_L3 plus2" } @@ -574,7 +574,7 @@ GFLUSH line" punk::args::undefine ::testspace::rdynmulti 1 rename ::testspace::rdyn_multi "" }\ - -result [list 0 -4 -2 0] + -result [list 0 0 2 0] test rendering_atdynamic_art_choicelabels {the textblock::frame pattern: @dynamic multiline art choicelabels via ${$DYN_...} with -unindentedfields render byte-aligned}\ -setup $common -body { diff --git a/src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test b/src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test index 0af2b394..845c6071 100644 --- a/src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test +++ b/src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test @@ -484,6 +484,33 @@ namespace eval ::testspace { }\ -result [list {{::testspace::helpfix v1} [-flag] firstval lastval}] + #--- leader-requiring definition, no supplied args (G-046 item 5) -------------------------- + + namespace eval ::testspace::leaderhelp {} + punk::args::define { + @id -id ::testspace::leaderhelp::classy + @cmd -name testspace::leaderhelp::classy -summary\ + "Leader-requiring fixture."\ + -help\ + "classy help." + @leaders -min 1 -max 1 + zzclass -type string -help\ + "mandatory leading class argument." + @values -min 0 -max 0 + } + proc ::testspace::leaderhelp::classy {zzclass} {return $zzclass} + + test cmdhelp_leader_required_no_args_plain_usage {cmdhelp with no argument words for a leader-requiring definition shows plain usage - the advisory goodargs parse failure ('Bad number of leading values for punk::args::parse ...') is suppressed (G-046 item 5, the 'i string is' shape)}\ + -setup $common -body { + set out [punk::ansi::ansistrip [punk::ns::cmdhelp -return string ::testspace::leaderhelp::classy]] + lappend result [string match "*Bad number of leading values*" $out] + lappend result [string match "*punk::args::parse*" $out] + lappend result [string match "*zzclass*" $out] + }\ + -cleanup { + }\ + -result [list 0 0 1] + #--- ensemble autodef with lazily-registered subcommand argdocs ---------------------------- #Integration surface of the defect pinned unit-level in #src/tests/modules/punk/args/testsuites/args/ensembledef.test ('i ansistring' in a