Browse Source
Both user-approved, drafted from the 2026-07-09 characterization findings with their acceptance criteria anchored to the pinned GAP tests. - G-045 (authoring ergonomics): parser-recognised record continuation (candidate: unquoted trailing -& with collision rules and an element-count disambiguation alternative recorded), @cmd honouring -unindentedfields, constructed (string-built) definitions able to opt into the whole-block indent normalization file-style definitions get (de-hacking ::punk::helptopic::define_docs as the consumer proof), and the braced-vs-quoted container quoting rules promoted from defquoting.test into the define documentation. - G-046 (deferred -help + fixes): headline item per user emphasis at approval - argument resolution stops processing -help/display-only fields, deferring their tstr expansion to display time with separate parse-vs-display caching (per the existing in-source review note), for measurable first-parse gains and fixing the reentrancy class reproduced with punk::ansi::mark_columns (~3.94s first call: its -help calls a proc that calls mark_columns, whose parse resolves the same still-resolving definition). Plus: @dynamic second-round multiline paramindent alignment, prefix-normalization value shape (list-wrap of backslash-containing choices), -return string renderer cmd-help continuation alignment, and the misleading goodargs 'Bad number of leading values...' prefix in 'i <cmd> <args>' output. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.commaster
3 changed files with 205 additions and 0 deletions
@ -0,0 +1,95 @@ |
|||||||
|
# G-045 punk::args definition authoring ergonomics: record continuation, @cmd unindented fields, constructed-definition normalization |
||||||
|
|
||||||
|
Status: proposed |
||||||
|
Scope: src/modules/punk/args-999999.0a1.0.tm (record parsing in resolve, tstr interplay, arg_error @cmd rendering), src/tests/modules/punk/args/testsuites/ (rendering.test/defquoting.test as the safety net), src/modules/punk-999999.0a1.0.tm (::punk::helptopic::define_docs de-hacked as the consumer proof) |
||||||
|
Acceptance: a definition using the chosen record-continuation mechanism parses identically to its backslash-continuation equivalent (existing definitions unchanged - continuation is additive), with the token's collision rules documented and an escape/rejection story for values that legitimately match it; @cmd -help/-summary honour -unindentedfields (the rendering.test GAP rendering_unindentedfields_cmd_help_GAP flips to aligned); a constructed definition can request whole-block normalization so embedded continuation indentation behaves as in file-style definitions (the rendering_constructed_def_indent_characterization expectations updated to the chosen semantics), and ::punk::helptopic::define_docs drops its manual pre-normalization to prove it; the quoting rules from defquoting.test appear in the punk::args::define -help documentation; the full punk::args suite (128 tests incl. the rendering invariants: nesting independence, relative-indent preservation) passes with GAP tests flipped, none weakened. |
||||||
|
|
||||||
|
## Context |
||||||
|
|
||||||
|
Drafted 2026-07-09 after the tests-first characterization pass (user direction: implement |
||||||
|
tests before changes in this area). The pain points, in the user's framing: |
||||||
|
|
||||||
|
- The define-block syntax deliberately avoids Tcl dict syntax for human readability, which |
||||||
|
led to backslash line-continuations and a complex interplay between record parsing in |
||||||
|
punk::args::resolve and tstr (especially ${...} expansions keeping whitespace aligned |
||||||
|
during multiline substitutions). |
||||||
|
- Constructed (string-built) definitions - ::punk::helptopic::define_docs and |
||||||
|
punk::args::ensemble_subcommands_definition are the in-tree examples - have poor |
||||||
|
ergonomics for getting indentation right: unlike file-style definitions they receive no |
||||||
|
whole-block indent normalization (proven: rendering.test |
||||||
|
rendering_constructed_def_indent_characterization - continuations at exactly 4 spaces |
||||||
|
align, other depths leak), so builders pre-normalize by hand (undent/trim hacks, |
||||||
|
\n-relative authoring). |
||||||
|
- A key property of normal definitions that must be preserved: they do not depend on the |
||||||
|
indentation of the code block they appear in (proven and pinned: |
||||||
|
rendering_nesting_independence_plain / _tstr). |
||||||
|
- The user's experiment used an ad-hoc `--` flag followed by a braced line-spanning empty |
||||||
|
string as a de-facto record continuation - functional but not designed-for, and |
||||||
|
unsatisfying. |
||||||
|
|
||||||
|
## The -& record-continuation candidate |
||||||
|
|
||||||
|
The user's proposal: a special token the record parser recognises as record continuation, |
||||||
|
e.g. an unquoted `-&` at end of line. Design considerations recorded for implementation: |
||||||
|
|
||||||
|
- Collision risk is low but not zero (a value or flag legitimately spelled -&). Candidate |
||||||
|
rules: only an UNQUOTED trailing `-&` at end of a record line counts; and/or use the |
||||||
|
element count of the assembled record to disambiguate (an odd element count where pairs |
||||||
|
are expected implies the trailing token is a continuation marker, not a value). The |
||||||
|
detail decision must include an escape story for a literal trailing -& value (bracing it |
||||||
|
suffices if 'unquoted' is part of the rule). |
||||||
|
- Detection must happen in the record parser (resolve) before/alongside line splitting - |
||||||
|
the same place backslash-continuation assembly effectively happens via Tcl's own parse |
||||||
|
of braced blocks today. Unlike backslash-newline, -& survives inside braced blocks of |
||||||
|
CONSTRUCTED definitions too - which is precisely why it helps: string-built defs cannot |
||||||
|
use backslash-newline ergonomically (it is consumed by the building code's own quoting). |
||||||
|
|
||||||
|
## @cmd -unindentedfields |
||||||
|
|
||||||
|
Accepted but ignored today (arg_error's cmd_info rendering carries an '#unindentedfields ?' |
||||||
|
todo; pinned by rendering_unindentedfields_cmd_help_GAP: left-margin-authored @cmd help |
||||||
|
renders its first line +4 vs continuations). Honouring it makes left-margin authoring |
||||||
|
available for @cmd -help/-summary as it already is for argument -help and -choicelabels. |
||||||
|
|
||||||
|
## Constructed-definition normalization |
||||||
|
|
||||||
|
Options to decide at implementation (record the choice): |
||||||
|
- an explicit define option/directive requesting whole-block undent of the supplied text |
||||||
|
- or automatic common-prefix normalization when a definition arrives as a single text |
||||||
|
block (risk: changing existing constructed defs' rendering - the characterization test |
||||||
|
documents current expectations to update deliberately) |
||||||
|
::punk::helptopic::define_docs currently pre-normalizes by hand (undent/trim + \n-relative |
||||||
|
help strings + -unindentedfields) - it converts to the new mechanism as the proof, and the |
||||||
|
user's uncommitted experimental hack in punk.tm is superseded. |
||||||
|
|
||||||
|
## Quoting documentation |
||||||
|
|
||||||
|
defquoting.test pinned the container rules (braced values fully literal - $, [], two-char |
||||||
|
\n, bare backslashes; quoted values get Tcl backslash semantics - \n newline, \\ -> \ - |
||||||
|
with $/[] still literal; \$\{...\} renders a literal ${...} in tstr-processed blocks). |
||||||
|
Promote these rules into the punk::args::define -help documentation so documenters (e.g. |
||||||
|
punk::imap4's {\Deleted}/{$MDNSent} choice values) have them stated, not just tested. |
||||||
|
|
||||||
|
## Alternatives considered |
||||||
|
|
||||||
|
- Keeping backslash continuations + manual pre-normalization as the documented way - |
||||||
|
rejected: the friction is proven by two in-tree constructed-def workarounds and the |
||||||
|
user's punk.tm experiments. |
||||||
|
- Tcl dict syntax for definitions - rejected long ago by design (readability). |
||||||
|
- The ad-hoc `--` + braced-gap trick as the blessed continuation - rejected: exploits |
||||||
|
parser behaviour that was never a contract. |
||||||
|
|
||||||
|
## Notes |
||||||
|
|
||||||
|
- Safety net: src/tests/modules/punk/args/testsuites/args/rendering.test (15 tests: |
||||||
|
nesting independence, relative-indent preservation, unindentedfields, constructed-def |
||||||
|
characterization, multiline tstr insertions, @dynamic) and defquoting.test (container |
||||||
|
quoting rules). GAPs to flip here: rendering_unindentedfields_cmd_help_GAP, |
||||||
|
rendering_constructed_def_indent_characterization (expectations updated to chosen |
||||||
|
semantics). |
||||||
|
- Related: G-046 (deferred -help resolution and rendering fixes - same code |
||||||
|
neighbourhood; sequencing between the two goals is free but they touch resolve |
||||||
|
together), G-044 (completion consumes the documentation corpus these ergonomics grow). |
||||||
|
- Session records 2026-07-09: probe scripts renderprobe*.tcl (scratchpad), the user's |
||||||
|
experimental define_docs hack (uncommitted punk.tm working-tree diff at the time of |
||||||
|
drafting - superseded by this goal when implemented). |
||||||
@ -0,0 +1,98 @@ |
|||||||
|
# G-046 punk::args deferred -help resolution (parse-time performance + reentrancy) and rendering/value-shape fixes |
||||||
|
|
||||||
|
Status: proposed |
||||||
|
Scope: src/modules/punk/args-999999.0a1.0.tm (resolve/get_dict: display-field deferral, dynamic-cache subst path, prefix writeback, string renderer, cmdhelp-facing messages), src/modules/punk/ansi-999999.0a1.0.tm (mark_columns argdoc as the reentrancy/perf testbed), src/tests/modules/punk/args/testsuites/ (GAP tests flip; perf verification) |
||||||
|
Acceptance: parsing/argument resolution provably skips -help expansion (a definition whose -help contains a ${[...]} that would error or record its invocation shows the substitution did NOT run during a parse-only path, only for help display); first parse of punk::ansi::mark_columns drops from ~4s to well under a second with 'i punk::ansi::mark_columns' still rendering the embedded example, and a -help that parses its OWN definition id resolves or errors cleanly rather than looping; first-parse timing improves for at least one other heavily documented command (recorded in the detail file); rendering_atdynamic_multiline_help_insertion_GAP flips to all-aligned; choicegroups_imap_prefix_listwrap_GAP flips to shape-identical (prefix input yields the same plain string as exact input); the -return string renderer's cmd-help continuations align under the first line with relative indents preserved (rendering_string_renderer_characterization updated); the 'Bad number of leading values...' prefix shown by goodargs parsing in 'i string is'-style output is reworded or suppressed for the usage-display path; full punk::args and punk::ns suites pass with no non-GAP expectations weakened. |
||||||
|
|
||||||
|
## Context |
||||||
|
|
||||||
|
Drafted 2026-07-09 from defects found (and pinned as GAP tests) during the tests-first |
||||||
|
characterization pass. User emphasis at approval: "Any performance gain we can gain from |
||||||
|
deferring -help processing out of argument resolution would be worthwhile" - deferral is |
||||||
|
the headline item, not merely a reentrancy patch. |
||||||
|
|
||||||
|
### 1. Deferred -help resolution (performance + reentrancy) |
||||||
|
|
||||||
|
punk::args::resolve currently tstr-processes entire definition blocks, including -help |
||||||
|
sections. The in-source review note (resolve, near the block-processing loop) already |
||||||
|
observes: "-help section processing is sometimes expensive ... isn't required for parsing |
||||||
|
of arguments ... ideally we would delay expansion of -help sections until needed for |
||||||
|
display, and use a different cache key for the parsing vs display versions of the resolved |
||||||
|
definition." |
||||||
|
|
||||||
|
The reentrancy class this also fixes, reproduced 2026-07-09 with punk::ansi::mark_columns: |
||||||
|
its argdoc -help contains ${[punk::ansi::argdoc::num_columns_example]}, and that example |
||||||
|
proc itself calls mark_columns, whose punk::args::parse resolves the same still-resolving |
||||||
|
definition. First call measured at ~3.94 SECONDS (60us thereafter, once cached). The |
||||||
|
mark_columns source (punk/ansi ~6443-6470) documents the hazard - including the observed |
||||||
|
"too many nested evaluations (infinite loop?)" when [a red] etc. are used in that -help - |
||||||
|
and lists the candidate fixes: (a) avoid parsing functions in argdocs (limiting, not DRY), |
||||||
|
(b) reentrancy guard, (c) don't process -help during argument resolution. This goal takes |
||||||
|
(c) as primary (with the perf benefit), with a reentrancy guard as defence-in-depth for |
||||||
|
the display path (a -help that renders help for its OWN id must error cleanly or resolve, |
||||||
|
never loop). |
||||||
|
|
||||||
|
Design notes: |
||||||
|
- separate cache keys for parse-spec vs display-spec (per the in-source note); the |
||||||
|
@dynamic unresolved-cache (ptlist/paramlist subst mechanism) splits accordingly. |
||||||
|
- fields in display-only scope: -help on @cmd/@examples/args, -choicelabels content, |
||||||
|
@formdisplay bodies - anything not consulted by get_dict for matching/validation. |
||||||
|
(-choices/-choicealiases/-typesynopsis ARE parse-relevant or synopsis-relevant - |
||||||
|
classify explicitly in implementation.) |
||||||
|
|
||||||
|
### 2. @dynamic multiline paramindent gap |
||||||
|
|
||||||
|
rendering_atdynamic_multiline_help_insertion_GAP: a ${$DYN_X} double substitution whose |
||||||
|
second-round result is multiline lands lines 2+ at insertion-column minus 4 (internal |
||||||
|
relative indents preserved), because the cached-path plain `subst` has no paramindent |
||||||
|
re-alignment - contrast single-level ${[cmd]} insertion which aligns fully |
||||||
|
(rendering_tstr_multiline_command_insertion). Fix: apply the same paramindent treatment |
||||||
|
to second-round substitution results. Note interaction with item 1: if display fields |
||||||
|
defer, the fix belongs in the display-time expansion path. |
||||||
|
|
||||||
|
### 3. Prefix-normalization value shape (list-wrap) |
||||||
|
|
||||||
|
choicegroups_imap_prefix_listwrap_GAP: for choices containing backslashes (punk::imap4's |
||||||
|
{\Deleted} etc. - spaces would behave likewise), exact input stores the plain string but |
||||||
|
prefix-normalized input stores a one-element LIST ({\Deleted}) - the get_dict overwrite |
||||||
|
writeback lsets into the stored value treating a scalar as a list. Consumers get a |
||||||
|
different value shape depending on how the user typed the choice. Fix the writeback for |
||||||
|
the single-value case (the -multiple/clause paths already index deliberately). |
||||||
|
|
||||||
|
### 4. -return string renderer cmd-help alignment |
||||||
|
|
||||||
|
rendering_string_renderer_characterization: the string renderer joins the -help first |
||||||
|
line onto the "Description: " label but drops continuations flush-left (relative indents |
||||||
|
preserved among themselves). The table renderer aligns properly. Align continuations |
||||||
|
under the first line. (The characterization test's pinned flush-left expectations get |
||||||
|
updated to the aligned form.) |
||||||
|
|
||||||
|
### 5. cmdhelp goodargs message accuracy |
||||||
|
|
||||||
|
'i string is' (and any command whose definition requires leaders) prefixes its usage |
||||||
|
output with "Bad number of leading values for punk::args::parse $args_remaining ..." - |
||||||
|
an internal-looking, inaccurate message produced by cmdhelp's goodargs-marking parse of |
||||||
|
the (empty) trailing args. Reword for the usage-display path or suppress (the parse is |
||||||
|
only advisory for highlighting supplied args). User: "the message is not accurate and |
||||||
|
could do with a fix up at some stage." |
||||||
|
|
||||||
|
## Alternatives considered |
||||||
|
|
||||||
|
- Reentrancy guard alone (without deferral) for item 1 - rejected as primary: it fixes |
||||||
|
the loop but forfeits the first-parse performance win the user explicitly wants, and |
||||||
|
heavy -help processing would still run on every first parse. |
||||||
|
- Documenting "don't call parsing functions from argdocs" - rejected (the mark_columns |
||||||
|
source itself calls this limiting and non-DRY). |
||||||
|
|
||||||
|
## Notes |
||||||
|
|
||||||
|
- GAP tests to flip: rendering_atdynamic_multiline_help_insertion_GAP, |
||||||
|
choicegroups_imap_prefix_listwrap_GAP; characterization tests to update: |
||||||
|
rendering_string_renderer_characterization. Safety net: the full 128-test punk/args |
||||||
|
suite (esp. rendering.test invariants and dev/dynamic-cache.test) and punk/ns cmdflow. |
||||||
|
- Perf evidence baseline (2026-07-09): mark_columns first parse ~3.94s / subsequent 60us |
||||||
|
(punk902z dev, this machine). tclcore moduledoc commands (e.g. ::lseq with |
||||||
|
helpers::example blocks) are candidates for the second timing datapoint. |
||||||
|
- Related: G-045 (authoring ergonomics - same resolve neighbourhood; free sequencing), |
||||||
|
G-041 (form matching will exercise resolve paths more heavily - deferral lightens its |
||||||
|
per-form attempts), G-044 (completion wants fast parse-spec resolution). |
||||||
Loading…
Reference in new issue