Browse Source
Both user-approved, drafted from the 2026-07-09 characterization findings with their acceptance criteria anchored to the pinned GAP tests. - G-045 (authoring ergonomics): parser-recognised record continuation (candidate: unquoted trailing -& with collision rules and an element-count disambiguation alternative recorded), @cmd honouring -unindentedfields, constructed (string-built) definitions able to opt into the whole-block indent normalization file-style definitions get (de-hacking ::punk::helptopic::define_docs as the consumer proof), and the braced-vs-quoted container quoting rules promoted from defquoting.test into the define documentation. - G-046 (deferred -help + fixes): headline item per user emphasis at approval - argument resolution stops processing -help/display-only fields, deferring their tstr expansion to display time with separate parse-vs-display caching (per the existing in-source review note), for measurable first-parse gains and fixing the reentrancy class reproduced with punk::ansi::mark_columns (~3.94s first call: its -help calls a proc that calls mark_columns, whose parse resolves the same still-resolving definition). Plus: @dynamic second-round multiline paramindent alignment, prefix-normalization value shape (list-wrap of backslash-containing choices), -return string renderer cmd-help continuation alignment, and the misleading goodargs 'Bad number of leading values...' prefix in 'i <cmd> <args>' output. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.commaster
3 changed files with 205 additions and 0 deletions
@ -0,0 +1,95 @@
|
||||
# G-045 punk::args definition authoring ergonomics: record continuation, @cmd unindented fields, constructed-definition normalization |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/args-999999.0a1.0.tm (record parsing in resolve, tstr interplay, arg_error @cmd rendering), src/tests/modules/punk/args/testsuites/ (rendering.test/defquoting.test as the safety net), src/modules/punk-999999.0a1.0.tm (::punk::helptopic::define_docs de-hacked as the consumer proof) |
||||
Acceptance: a definition using the chosen record-continuation mechanism parses identically to its backslash-continuation equivalent (existing definitions unchanged - continuation is additive), with the token's collision rules documented and an escape/rejection story for values that legitimately match it; @cmd -help/-summary honour -unindentedfields (the rendering.test GAP rendering_unindentedfields_cmd_help_GAP flips to aligned); a constructed definition can request whole-block normalization so embedded continuation indentation behaves as in file-style definitions (the rendering_constructed_def_indent_characterization expectations updated to the chosen semantics), and ::punk::helptopic::define_docs drops its manual pre-normalization to prove it; the quoting rules from defquoting.test appear in the punk::args::define -help documentation; the full punk::args suite (128 tests incl. the rendering invariants: nesting independence, relative-indent preservation) passes with GAP tests flipped, none weakened. |
||||
|
||||
## Context |
||||
|
||||
Drafted 2026-07-09 after the tests-first characterization pass (user direction: implement |
||||
tests before changes in this area). The pain points, in the user's framing: |
||||
|
||||
- The define-block syntax deliberately avoids Tcl dict syntax for human readability, which |
||||
led to backslash line-continuations and a complex interplay between record parsing in |
||||
punk::args::resolve and tstr (especially ${...} expansions keeping whitespace aligned |
||||
during multiline substitutions). |
||||
- Constructed (string-built) definitions - ::punk::helptopic::define_docs and |
||||
punk::args::ensemble_subcommands_definition are the in-tree examples - have poor |
||||
ergonomics for getting indentation right: unlike file-style definitions they receive no |
||||
whole-block indent normalization (proven: rendering.test |
||||
rendering_constructed_def_indent_characterization - continuations at exactly 4 spaces |
||||
align, other depths leak), so builders pre-normalize by hand (undent/trim hacks, |
||||
\n-relative authoring). |
||||
- A key property of normal definitions that must be preserved: they do not depend on the |
||||
indentation of the code block they appear in (proven and pinned: |
||||
rendering_nesting_independence_plain / _tstr). |
||||
- The user's experiment used an ad-hoc `--` flag followed by a braced line-spanning empty |
||||
string as a de-facto record continuation - functional but not designed-for, and |
||||
unsatisfying. |
||||
|
||||
## The -& record-continuation candidate |
||||
|
||||
The user's proposal: a special token the record parser recognises as record continuation, |
||||
e.g. an unquoted `-&` at end of line. Design considerations recorded for implementation: |
||||
|
||||
- Collision risk is low but not zero (a value or flag legitimately spelled -&). Candidate |
||||
rules: only an UNQUOTED trailing `-&` at end of a record line counts; and/or use the |
||||
element count of the assembled record to disambiguate (an odd element count where pairs |
||||
are expected implies the trailing token is a continuation marker, not a value). The |
||||
detail decision must include an escape story for a literal trailing -& value (bracing it |
||||
suffices if 'unquoted' is part of the rule). |
||||
- Detection must happen in the record parser (resolve) before/alongside line splitting - |
||||
the same place backslash-continuation assembly effectively happens via Tcl's own parse |
||||
of braced blocks today. Unlike backslash-newline, -& survives inside braced blocks of |
||||
CONSTRUCTED definitions too - which is precisely why it helps: string-built defs cannot |
||||
use backslash-newline ergonomically (it is consumed by the building code's own quoting). |
||||
|
||||
## @cmd -unindentedfields |
||||
|
||||
Accepted but ignored today (arg_error's cmd_info rendering carries an '#unindentedfields ?' |
||||
todo; pinned by rendering_unindentedfields_cmd_help_GAP: left-margin-authored @cmd help |
||||
renders its first line +4 vs continuations). Honouring it makes left-margin authoring |
||||
available for @cmd -help/-summary as it already is for argument -help and -choicelabels. |
||||
|
||||
## Constructed-definition normalization |
||||
|
||||
Options to decide at implementation (record the choice): |
||||
- an explicit define option/directive requesting whole-block undent of the supplied text |
||||
- or automatic common-prefix normalization when a definition arrives as a single text |
||||
block (risk: changing existing constructed defs' rendering - the characterization test |
||||
documents current expectations to update deliberately) |
||||
::punk::helptopic::define_docs currently pre-normalizes by hand (undent/trim + \n-relative |
||||
help strings + -unindentedfields) - it converts to the new mechanism as the proof, and the |
||||
user's uncommitted experimental hack in punk.tm is superseded. |
||||
|
||||
## Quoting documentation |
||||
|
||||
defquoting.test pinned the container rules (braced values fully literal - $, [], two-char |
||||
\n, bare backslashes; quoted values get Tcl backslash semantics - \n newline, \\ -> \ - |
||||
with $/[] still literal; \$\{...\} renders a literal ${...} in tstr-processed blocks). |
||||
Promote these rules into the punk::args::define -help documentation so documenters (e.g. |
||||
punk::imap4's {\Deleted}/{$MDNSent} choice values) have them stated, not just tested. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- Keeping backslash continuations + manual pre-normalization as the documented way - |
||||
rejected: the friction is proven by two in-tree constructed-def workarounds and the |
||||
user's punk.tm experiments. |
||||
- Tcl dict syntax for definitions - rejected long ago by design (readability). |
||||
- The ad-hoc `--` + braced-gap trick as the blessed continuation - rejected: exploits |
||||
parser behaviour that was never a contract. |
||||
|
||||
## Notes |
||||
|
||||
- Safety net: src/tests/modules/punk/args/testsuites/args/rendering.test (15 tests: |
||||
nesting independence, relative-indent preservation, unindentedfields, constructed-def |
||||
characterization, multiline tstr insertions, @dynamic) and defquoting.test (container |
||||
quoting rules). GAPs to flip here: rendering_unindentedfields_cmd_help_GAP, |
||||
rendering_constructed_def_indent_characterization (expectations updated to chosen |
||||
semantics). |
||||
- Related: G-046 (deferred -help resolution and rendering fixes - same code |
||||
neighbourhood; sequencing between the two goals is free but they touch resolve |
||||
together), G-044 (completion consumes the documentation corpus these ergonomics grow). |
||||
- Session records 2026-07-09: probe scripts renderprobe*.tcl (scratchpad), the user's |
||||
experimental define_docs hack (uncommitted punk.tm working-tree diff at the time of |
||||
drafting - superseded by this goal when implemented). |
||||
@ -0,0 +1,98 @@
|
||||
# G-046 punk::args deferred -help resolution (parse-time performance + reentrancy) and rendering/value-shape fixes |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/args-999999.0a1.0.tm (resolve/get_dict: display-field deferral, dynamic-cache subst path, prefix writeback, string renderer, cmdhelp-facing messages), src/modules/punk/ansi-999999.0a1.0.tm (mark_columns argdoc as the reentrancy/perf testbed), src/tests/modules/punk/args/testsuites/ (GAP tests flip; perf verification) |
||||
Acceptance: parsing/argument resolution provably skips -help expansion (a definition whose -help contains a ${[...]} that would error or record its invocation shows the substitution did NOT run during a parse-only path, only for help display); first parse of punk::ansi::mark_columns drops from ~4s to well under a second with 'i punk::ansi::mark_columns' still rendering the embedded example, and a -help that parses its OWN definition id resolves or errors cleanly rather than looping; first-parse timing improves for at least one other heavily documented command (recorded in the detail file); rendering_atdynamic_multiline_help_insertion_GAP flips to all-aligned; choicegroups_imap_prefix_listwrap_GAP flips to shape-identical (prefix input yields the same plain string as exact input); the -return string renderer's cmd-help continuations align under the first line with relative indents preserved (rendering_string_renderer_characterization updated); the 'Bad number of leading values...' prefix shown by goodargs parsing in 'i string is'-style output is reworded or suppressed for the usage-display path; full punk::args and punk::ns suites pass with no non-GAP expectations weakened. |
||||
|
||||
## Context |
||||
|
||||
Drafted 2026-07-09 from defects found (and pinned as GAP tests) during the tests-first |
||||
characterization pass. User emphasis at approval: "Any performance gain we can gain from |
||||
deferring -help processing out of argument resolution would be worthwhile" - deferral is |
||||
the headline item, not merely a reentrancy patch. |
||||
|
||||
### 1. Deferred -help resolution (performance + reentrancy) |
||||
|
||||
punk::args::resolve currently tstr-processes entire definition blocks, including -help |
||||
sections. The in-source review note (resolve, near the block-processing loop) already |
||||
observes: "-help section processing is sometimes expensive ... isn't required for parsing |
||||
of arguments ... ideally we would delay expansion of -help sections until needed for |
||||
display, and use a different cache key for the parsing vs display versions of the resolved |
||||
definition." |
||||
|
||||
The reentrancy class this also fixes, reproduced 2026-07-09 with punk::ansi::mark_columns: |
||||
its argdoc -help contains ${[punk::ansi::argdoc::num_columns_example]}, and that example |
||||
proc itself calls mark_columns, whose punk::args::parse resolves the same still-resolving |
||||
definition. First call measured at ~3.94 SECONDS (60us thereafter, once cached). The |
||||
mark_columns source (punk/ansi ~6443-6470) documents the hazard - including the observed |
||||
"too many nested evaluations (infinite loop?)" when [a red] etc. are used in that -help - |
||||
and lists the candidate fixes: (a) avoid parsing functions in argdocs (limiting, not DRY), |
||||
(b) reentrancy guard, (c) don't process -help during argument resolution. This goal takes |
||||
(c) as primary (with the perf benefit), with a reentrancy guard as defence-in-depth for |
||||
the display path (a -help that renders help for its OWN id must error cleanly or resolve, |
||||
never loop). |
||||
|
||||
Design notes: |
||||
- separate cache keys for parse-spec vs display-spec (per the in-source note); the |
||||
@dynamic unresolved-cache (ptlist/paramlist subst mechanism) splits accordingly. |
||||
- fields in display-only scope: -help on @cmd/@examples/args, -choicelabels content, |
||||
@formdisplay bodies - anything not consulted by get_dict for matching/validation. |
||||
(-choices/-choicealiases/-typesynopsis ARE parse-relevant or synopsis-relevant - |
||||
classify explicitly in implementation.) |
||||
|
||||
### 2. @dynamic multiline paramindent gap |
||||
|
||||
rendering_atdynamic_multiline_help_insertion_GAP: a ${$DYN_X} double substitution whose |
||||
second-round result is multiline lands lines 2+ at insertion-column minus 4 (internal |
||||
relative indents preserved), because the cached-path plain `subst` has no paramindent |
||||
re-alignment - contrast single-level ${[cmd]} insertion which aligns fully |
||||
(rendering_tstr_multiline_command_insertion). Fix: apply the same paramindent treatment |
||||
to second-round substitution results. Note interaction with item 1: if display fields |
||||
defer, the fix belongs in the display-time expansion path. |
||||
|
||||
### 3. Prefix-normalization value shape (list-wrap) |
||||
|
||||
choicegroups_imap_prefix_listwrap_GAP: for choices containing backslashes (punk::imap4's |
||||
{\Deleted} etc. - spaces would behave likewise), exact input stores the plain string but |
||||
prefix-normalized input stores a one-element LIST ({\Deleted}) - the get_dict overwrite |
||||
writeback lsets into the stored value treating a scalar as a list. Consumers get a |
||||
different value shape depending on how the user typed the choice. Fix the writeback for |
||||
the single-value case (the -multiple/clause paths already index deliberately). |
||||
|
||||
### 4. -return string renderer cmd-help alignment |
||||
|
||||
rendering_string_renderer_characterization: the string renderer joins the -help first |
||||
line onto the "Description: " label but drops continuations flush-left (relative indents |
||||
preserved among themselves). The table renderer aligns properly. Align continuations |
||||
under the first line. (The characterization test's pinned flush-left expectations get |
||||
updated to the aligned form.) |
||||
|
||||
### 5. cmdhelp goodargs message accuracy |
||||
|
||||
'i string is' (and any command whose definition requires leaders) prefixes its usage |
||||
output with "Bad number of leading values for punk::args::parse $args_remaining ..." - |
||||
an internal-looking, inaccurate message produced by cmdhelp's goodargs-marking parse of |
||||
the (empty) trailing args. Reword for the usage-display path or suppress (the parse is |
||||
only advisory for highlighting supplied args). User: "the message is not accurate and |
||||
could do with a fix up at some stage." |
||||
|
||||
## Alternatives considered |
||||
|
||||
- Reentrancy guard alone (without deferral) for item 1 - rejected as primary: it fixes |
||||
the loop but forfeits the first-parse performance win the user explicitly wants, and |
||||
heavy -help processing would still run on every first parse. |
||||
- Documenting "don't call parsing functions from argdocs" - rejected (the mark_columns |
||||
source itself calls this limiting and non-DRY). |
||||
|
||||
## Notes |
||||
|
||||
- GAP tests to flip: rendering_atdynamic_multiline_help_insertion_GAP, |
||||
choicegroups_imap_prefix_listwrap_GAP; characterization tests to update: |
||||
rendering_string_renderer_characterization. Safety net: the full 128-test punk/args |
||||
suite (esp. rendering.test invariants and dev/dynamic-cache.test) and punk/ns cmdflow. |
||||
- Perf evidence baseline (2026-07-09): mark_columns first parse ~3.94s / subsequent 60us |
||||
(punk902z dev, this machine). tclcore moduledoc commands (e.g. ::lseq with |
||||
helpers::example blocks) are candidates for the second timing datapoint. |
||||
- Related: G-045 (authoring ergonomics - same resolve neighbourhood; free sequencing), |
||||
G-041 (form matching will exercise resolve paths more heavily - deferral lightens its |
||||
per-form attempts), G-044 (completion wants fast parse-spec resolution). |
||||
Loading…
Reference in new issue