Browse Source

G-045 increment 4: punk::args 0.8.0 @normalize directive; define_docs converts as consumer proof (project 0.12.7)

New bare @normalize directive: opts a definition into indent normalization
of BLOCK-FORM multi-line field values (first line whitespace-only, as
authored by opening a braced literal with a newline). The structural first
newline and a whitespace-only trailing line are dropped, the content
lines' common leading whitespace is the block's base indent, the first
content line is unindented fully and subsequent lines are re-based to the
standard 4-space continuation convention (deeper relative indents
preserved; whitespace-only inner lines become empty). Fields in a record's
-unindentedfields are exempt. Implemented as a resolve pre-pass over the
split records (private::normalize_records / rebase_multiline_value);
@normalize with options is an error.

Head-form values are never altered - their base indent is unknowable
(continuations uniformly at 6 may be base-4 with the deliberate +2
relative convention, or base-6 flush; re-basing would flatten the former).
The idempotence test caught exactly that on a file-style definition during
implementation, narrowing the user-confirmed re-base semantics to block
form only - which also makes @normalize a proven no-op on conforming
file-style definitions.

Consumer proof (punk 0.2.5, per the recorded user decision):
::punk::helptopic::define_docs converts from interim left-margin authoring
to indented block-form values under @normalize, dropping its
-unindentedfields declarations; 'i help' and 'i help_chunks' verified
aligned in punk902z src, including the blank-line separator in the
combined basehelp+extra block.

Tests: new normalize.test (5 tests: block re-base, block left-margin,
head-form boundary, exemption byte-exactness, file-style idempotence);
rendering.test P4 characterization stays pinned as the deliberate unopted
default with its text updated to reference @normalize. punk::args suite
186 pass / 1 pre-existing skip / 0 fail; punk::ns 53/53; full source-tree
suite 806 pass / 13 skip / 1 fail = exec-14.3 only (known baseline) - zero
regressions (tclsh 9.0.3). define doc documents the directive.

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 3 days ago
parent
commit
22606faebe
  1. 4
      CHANGELOG.md
  2. 42
      goals/G-045-punkargs-authoring-ergonomics.md
  3. 2
      punkproject.toml
  4. 44
      src/modules/punk-999999.0a1.0.tm
  5. 3
      src/modules/punk-buildversion.txt
  6. 159
      src/modules/punk/args-999999.0a1.0.tm
  7. 3
      src/modules/punk/args-buildversion.txt
  8. 192
      src/tests/modules/punk/args/testsuites/args/normalize.test
  9. 8
      src/tests/modules/punk/args/testsuites/args/rendering.test

4
CHANGELOG.md

@ -5,6 +5,10 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to
Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` Entries are newest-first; one bullet per notable change. See the root `AGENTS.md`
"Project Versioning" section for the bump policy. "Project Versioning" section for the bump policy.
## [0.12.7] - 2026-07-12
- G-045 (increment): punk::args 0.8.0 adds the bare @normalize directive - opts a definition into indent normalization of block-form multi-line field values (structural leading newline dropped, content re-based to the file-style 4-space continuation convention, relative indents preserved, -unindentedfields exempt, no-op on conforming file-style definitions). Intended for constructed (string-built) definitions. punk 0.2.5: the help-system definitions (::punk::help / help_chunks) convert to indented block authoring under @normalize; 'i help' rendering unchanged.
## [0.12.6] - 2026-07-12 ## [0.12.6] - 2026-07-12
- G-045 (increment): punk::args 0.7.0 adds the -& record-continuation token - an unquoted trailing -& on a definition record line continues the record on the next line, assembling byte-identically to the equivalent backslash continuation. Intended for constructed (string-built) definitions, where backslash-newline is consumed by the building code's own quoting. Brace a literal trailing -& value ({-&}); -& elsewhere on a line or inside braced/quoted multi-line values is ordinary data. Backslash-continuation authoring is unchanged. - G-045 (increment): punk::args 0.7.0 adds the -& record-continuation token - an unquoted trailing -& on a definition record line continues the record on the next line, assembling byte-identically to the equivalent backslash continuation. Intended for constructed (string-built) definitions, where backslash-newline is consumed by the building code's own quoting. Brace a literal trailing -& value ({-&}); -& elsewhere on a line or inside braced/quoted multi-line values is ordinary data. Backslash-continuation authoring is unchanged.

42
goals/G-045-punkargs-authoring-ergonomics.md

@ -191,11 +191,43 @@ punk::imap4's {\Deleted}/{$MDNSent} choice values) have them stated, not just te
dev-time lint cross-checking splitter boundaries against parse where the dev-time lint cross-checking splitter boundaries against parse where the
binary is available. binary is available.
### 2026-07-12 increment 4: @normalize constructed-def normalization (punk::args 0.8.0, punk 0.2.5)
- Mechanism (user-approved 2026-07-12): bare @normalize directive (like @dynamic;
options are an error), detected as a resolve pre-pass over the split records.
Semantics user-confirmed as re-base-to-+4; narrowed during implementation to
BLOCK-FORM values only (first line whitespace-only): the structural first
newline and a whitespace-only trailing line are dropped (the user's
leading-newline ergonomics question - resolved as automatic), the content
lines' common leading whitespace is the base, first content line unindented
fully, subsequent lines re-based to 4 spaces with deeper relative indents
preserved. -unindentedfields fields exempt.
- Why block-form only: head-form values have an unknowable base - continuations
uniformly at 6 may be base-4 with the deliberate +2 relative convention (P2)
or base-6 flush; common-prefix re-basing flattens the former. The idempotence
test caught exactly this on a file-style def, forcing the narrowing. Block
form is unambiguous (the first content line carries the full base), and
head-form untouchedness makes @normalize a proven no-op on conforming
file-style definitions.
- Consumer proof: define_docs converted from interim left-margin authoring to
indented block-form values + @normalize (per the increment 2 decision);
-unindentedfields declarations dropped; 'i help' and 'i help_chunks'
verified aligned in punk902z src (including the blank-line separator in the
combined basehelp+extra block).
- rendering_constructed_def_indent_characterization stays pinned as the
deliberate unopted default, its description/header updated to reference
@normalize as the opt-in remedy (the acceptance's "expectations updated to
the chosen semantics" - the chosen semantics being opt-in, unopted
behaviour is contract, not gap).
- New testsuite normalize.test (5 tests): block-form re-base, block-form
left-margin, head-form untouched boundary, -unindentedfields exemption
(structural newline kept byte-exact), idempotence on file-style defs.
- Verified (tclsh 9.0.3): punk::args suite 186 pass / 1 pre-existing skip / 0
fail; punk::ns suite 53/53; full source-tree suite 820 total, 806 pass, 13
skip, 1 fail = exec-14.3 only (known pre-existing core-test baseline) - zero
regressions; make.tcl modules builds clean; 'i help'/'i help_chunks' verified
aligned in punk902z src.
Remaining for acceptance: Remaining for acceptance:
- constructed-definition whole-block normalization opt-in, with
rendering_constructed_def_indent_characterization expectations updated and
::punk::helptopic::define_docs converted from interim left-margin authoring
to indented-plus-normalized as the consumer proof (user decision 2026-07-12,
see increment 2 notes)
- defquoting.test container quoting rules promoted into the punk::args::define - defquoting.test container quoting rules promoted into the punk::args::define
-help documentation -help documentation

2
punkproject.toml

@ -1,4 +1,4 @@
[project] [project]
name = "punkshell" name = "punkshell"
version = "0.12.6" version = "0.12.7"
license = "BSD-2-Clause" license = "BSD-2-Clause"

44
src/modules/punk-999999.0a1.0.tm

@ -8494,30 +8494,38 @@ namespace eval punk {
} }
#keep help-text lines manually folded (~70 cols) - the usage tables don't yet #keep help-text lines manually folded (~70 cols) - the usage tables don't yet
#wrap to terminal width, so line lengths here directly set the table width #wrap to terminal width, so line lengths here directly set the table width
#help text is authored at the left margin and the @cmd/topic lines declare #help text is authored as indented blocks (structural leading newline) and
#-unindentedfields {-help} (G-045): constructed definitions get no #the generated definition declares @normalize (G-045): block-form
#whole-block indent normalization, so source-indented authoring here would #multi-line values are re-based to the file-style convention at resolve
#leak extra indent into the rendered help #time - no manual undent/trim and no left-margin authoring in this builder
set basehelp {Help system for the punk shell. set basehelp {
With no arguments - an overview of some key shell Help system for the punk shell.
commands is displayed. With no arguments - an overview of some key shell
When the first argument is a recognised topic - help commands is displayed.
for that topic is displayed. When the first argument is a recognised topic - help
('help topics' lists the available topics) for that topic is displayed.
Anything else is treated as a command name - basic ('help topics' lists the available topics)
command info (type and synopsis) is shown for a Anything else is treated as a command name - basic
resolvable command, or the resolved path for an command info (type and synopsis) is shown for a
external executable.} resolvable command, or the resolved path for an
set topichelp "Help topic, or command words for basic command info.\nTopics accept their aliases and unique prefixes\n(some short words deliberately fall through to command lookup)." external executable.}
set topichelp {
Help topic, or command words for basic command info.
Topics accept their aliases and unique prefixes
(some short words deliberately fall through to command lookup).}
set specs [list] set specs [list]
lappend specs ::punk::help help "Punk shell help system." "" lappend specs ::punk::help help "Punk shell help system." ""
lappend specs ::punk::help_chunks punk::help_chunks "Punk shell help system - content as {channel text} chunks." "\n\nhelp_chunks returns the help content as a list of\n{channel text} chunks rather than emitting it." lappend specs ::punk::help_chunks punk::help_chunks "Punk shell help system - content as {channel text} chunks." {
help_chunks returns the help content as a list of
{channel text} chunks rather than emitting it.}
foreach {id name summary extra} $specs { foreach {id name summary extra} $specs {
set def "" set def ""
append def "@normalize" \n
append def "@id -id $id" \n append def "@id -id $id" \n
append def "@cmd -name $name -summary \"$summary\" -unindentedfields {-help} -help \"$basehelp$extra\"" \n append def "@cmd -name $name -summary \"$summary\" -help \"$basehelp$extra\"" \n
append def "@leaders -min 0 -max -1" \n append def "@leaders -min 0 -max -1" \n
append def "topic -optional 1 -multiple 1 -type string -choiceprefix 1 -choicerestricted 0 -choicecolumns 2 -choices {$choices} -choicealiases {$choicealiases} -choiceprefixdenylist {$topic_prefixdenylist} -choiceprefixreservelist {$topic_prefixreservelist} -choiceinfo {$choiceinfo} -choicelabels {$choicelabels} -unindentedfields {-help} -help \"$topichelp\"" \n append def "topic -optional 1 -multiple 1 -type string -choiceprefix 1 -choicerestricted 0 -choicecolumns 2 -choices {$choices} -choicealiases {$choicealiases} -choiceprefixdenylist {$topic_prefixdenylist} -choiceprefixreservelist {$topic_prefixreservelist} -choiceinfo {$choiceinfo} -choicelabels {$choicelabels} -help \"$topichelp\"" \n
append def "@values -min 0 -max 0" append def "@values -min 0 -max 0"
if {[punk::args::id_exists $id]} { if {[punk::args::id_exists $id]} {
#quiet undefine - redefinition on registry change is expected, not noteworthy #quiet undefine - redefinition on registry change is expected, not noteworthy

3
src/modules/punk-buildversion.txt

@ -1,6 +1,7 @@
0.2.4 0.2.5
#First line must be a semantic version number #First line must be a semantic version number
#all other lines are ignored. #all other lines are ignored.
#0.2.5 - G-045: ::punk::helptopic::define_docs converts from interim left-margin authoring to indented block-form values under the new punk::args @normalize directive (the constructed-definition normalization consumer proof): basehelp/topichelp/help_chunks-extra are braced indented blocks with a structural leading newline, the generated definitions declare @normalize, and the -unindentedfields declarations from punk 0.2.4 are dropped. Rendered 'i help'/'i help_chunks' output unchanged (verified aligned incl. the blank-line separator in help_chunks).
#0.2.4 - G-045: 'i help' usage table alignment - ::punk::helptopic::define_docs authors its help text at the left margin and declares -unindentedfields {-help} on both the generated @cmd line (honoured as of punk::args 0.6.1) and the topic argument line. Previously the @cmd -help braced literal carried ~16 spaces of source indent into the constructed definition (no whole-block normalization), rendering Description continuations +12 right of the first line, and the \n-relative topic -help rendered its first line +4 (the injected display prefix). Both blocks now render flush. Text content unchanged (manual ~70-col folding retained). #0.2.4 - G-045: 'i help' usage table alignment - ::punk::helptopic::define_docs authors its help text at the left margin and declares -unindentedfields {-help} on both the generated @cmd line (honoured as of punk::args 0.6.1) and the topic argument line. Previously the @cmd -help braced literal carried ~16 spaces of source indent into the constructed definition (no whole-block normalization), rendering Description continuations +12 right of the first line, and the \n-relative topic -help rendered its first line +4 (the injected display prefix). Both blocks now render flush. Text content unchanged (manual ~70-col folding retained).
#0.2.3 - punk::help topic definition adopts punk::args -choicealiases (G-040): choices are the four canonical topics with registry aliases folded ('i help' shows one entry per topic with an (alias:...) note), unique prefixes of topics and aliases accepted, minimum-prefix policy per user decision recorded in ::punk::helptopic (denylist {help}: h/he/hel stay command words; reservelist {c to tc}: fall through to command lookup); unrecognised words still fall through to basic command info; argless 'help' overview unchanged #0.2.3 - punk::help topic definition adopts punk::args -choicealiases (G-040): choices are the four canonical topics with registry aliases folded ('i help' shows one entry per topic with an (alias:...) note), unique prefixes of topics and aliases accepted, minimum-prefix policy per user decision recorded in ::punk::helptopic (denylist {help}: h/he/hel stay command words; reservelist {c to tc}: fall through to command lookup); unrecognised words still fall through to basic command info; argless 'help' overview unchanged
#0.2.2 - documentation-only: helptopic argdoc help texts manually folded (~70 cols) and the generated ::punk::help topic choices grid set to -choicecolumns 2 - 'i help'/'i help <topic>' tables now render at reasonable widths (61-68 cols, was ~160); usage tables don't yet wrap to terminal width so source line lengths set table width; argless 'help' overview output unchanged (strict 80-col layout preserved) #0.2.2 - documentation-only: helptopic argdoc help texts manually folded (~70 cols) and the generated ::punk::help topic choices grid set to -choicecolumns 2 - 'i help'/'i help <topic>' tables now render at reasonable widths (61-68 cols, was ~160); usage tables don't yet wrap to terminal width so source line lengths set table width; argless 'help' overview output unchanged (strict 80-col layout preserved)

159
src/modules/punk/args-999999.0a1.0.tm

@ -714,6 +714,27 @@ tcl::namespace::eval punk::args {
fields depend on values that can change after definition time. fields depend on values that can change after definition time.
Dynamic definitions have a small performance cost and can interact Dynamic definitions have a small performance cost and can interact
with punk::args::parse -cache 1; see the parse -cache option. with punk::args::parse -cache 1; see the parse -cache option.
%B%@normalize%N%
Bare directive (no options), conventionally placed near the top.
Opts the definition into indent normalization of BLOCK-FORM
multi-line field values - values whose first line is
whitespace-only, as authored by opening a braced literal with a
newline and indenting the content as a block. For such values the
structural first newline (and a whitespace-only trailing line)
are dropped, the content lines' common leading whitespace is
taken as the block's base indent, the first content line is
unindented fully and subsequent lines are re-based to the
standard 4-space continuation convention with deeper relative
indents preserved.
Head-form values (content starting on the first line) are never
altered: their base indent is ambiguous (uniform indent may be
deliberate relative convention), and they already follow the
absolute 4-space convention - so the directive is harmless on
conforming file-style definitions.
Fields named in a record's -unindentedfields are exempt.
Intended for constructed (string-built) definitions, which do not
get the whole-block indent treatment braced file-style sources
give records.
%B%@id%N% ?opt val...? %B%@id%N% ?opt val...?
directive-options: -id <str> directive-options: -id <str>
%B%@cmd%N% ?opt val...? %B%@cmd%N% ?opt val...?
@ -1477,6 +1498,124 @@ tcl::namespace::eval punk::args {
return $records return $records
} }
proc private::rebase_multiline_value {value} {
#G-045 @normalize: re-base a BLOCK-FORM multi-line field value to the
#file-style continuation convention. Block form = the first line is
#whitespace-only (the value was authored as an indented block, e.g. a braced
#literal opening with a newline). The structural first newline (and a
#whitespace-only trailing line) are dropped, the common leading whitespace of
#the content lines is the block's base indent: the first content line is
#unindented fully and subsequent lines have the base replaced with 4 spaces
#(deeper relative indents preserved). Whitespace-only inner lines become
#empty lines.
#Head-form values (content on the first line) are returned UNCHANGED: their
#base indent is unknowable - a value whose continuations sit uniformly at 6
#may be base-4 with the deliberate +2 relative convention, or base-6 flush;
#re-basing would flatten the former. Head-form values follow the standard
#absolute convention (continuations at 4 + relative extras) as always, which
#also makes @normalize a no-op on conforming file-style definitions.
set lines [split $value \n]
if {[llength $lines] < 2} {
return $value
}
if {[tcl::string::trim [lindex $lines 0]] ne ""} {
#head form - unchanged
return $value
}
set lines [lrange $lines 1 end]
if {[llength $lines] > 1 && [tcl::string::trim [lindex $lines end]] eq ""} {
set lines [lrange $lines 0 end-1]
}
set prefix ""
set have_prefix 0
foreach ln $lines {
if {[tcl::string::trim $ln] eq ""} {continue}
regexp {^[ \t]*} $ln thisprefix
if {!$have_prefix} {
set prefix $thisprefix
set have_prefix 1
} else {
while {$prefix ne "" && [tcl::string::first $prefix $ln] != 0} {
set prefix [tcl::string::range $prefix 0 end-1]
}
}
if {$have_prefix && $prefix eq ""} {break}
}
if {!$have_prefix} {
#no content lines - degenerate block, structural lines dropped
return [join $lines \n]
}
set plen [tcl::string::length $prefix]
set out [list]
set done_first 0
foreach ln $lines {
if {[tcl::string::trim $ln] eq ""} {
lappend out ""
} elseif {!$done_first} {
#first content line - unindent fully (it starts with the common prefix)
lappend out [tcl::string::range $ln $plen end]
set done_first 1
} else {
lappend out " [tcl::string::range $ln $plen end]"
}
}
return [join $out \n]
}
proc private::normalize_records {records} {
#G-045 @normalize pre-pass (value semantics in rebase_multiline_value).
#Records are list-shaped (resolve consumes them via lassign), so a record is
#rewritten as a canonical list only when one of its values changed; fields
#named in the record's own -unindentedfields are exempt. Malformed records
#pass through untouched - resolve's record loop raises the real errors.
set out [list]
foreach rec $records {
set trimrec [tcl::string::trim $rec]
switch -- [tcl::string::index $trimrec 0] {
"" - # {
lappend out $rec
continue
}
}
if {[tcl::string::first \n $rec] < 0} {
lappend out $rec
continue
}
if {[catch {set recvalues [lassign $trimrec firstword]}]} {
lappend out $rec
continue
}
if {[llength $recvalues] % 2 != 0} {
lappend out $rec
continue
}
if {[dict exists $recvalues -unindentedfields]} {
set unindented [dict get $recvalues -unindentedfields]
} else {
set unindented {}
}
set newrec [list $firstword]
set changed 0
foreach {k v} $recvalues {
if {$k ni $unindented && [tcl::string::first \n $v] >= 0} {
set nv [rebase_multiline_value $v]
if {$nv ne $v} {
set changed 1
}
lappend newrec $k $nv
} else {
lappend newrec $k $v
}
}
if {$changed} {
lappend out $newrec
} else {
lappend out $rec
}
}
return $out
}
proc private::classify_display_tokens {placeholdertext tokenlist} { proc private::classify_display_tokens {placeholdertext tokenlist} {
#G-046: return the subset of tokenlist whose tokens land in display-only field #G-046: return the subset of tokenlist whose tokens land in display-only field
#values. Display-only: -help on @cmd/@examples and on argument records, and #values. Display-only: -help on @cmd/@examples and on argument records, and
@ -1999,6 +2138,17 @@ tcl::namespace::eval punk::args {
set records [private::split_definition_records $optionspecs] set records [private::split_definition_records $optionspecs]
#G-045: a bare @normalize directive record opts the whole definition into
#indent normalization of multi-line field values - the treatment constructed
#(string-built) definitions don't otherwise get. Pre-pass, so values are
#already re-based when the record loop below processes them.
foreach rec $records {
if {[tcl::string::trim $rec] eq "@normalize"} {
set records [private::normalize_records $records]
break
}
}
set cmd_info {} set cmd_info {}
set package_info {} set package_info {}
set id_info {} ;#e.g -children <list> ?? set id_info {} ;#e.g -children <list> ??
@ -2125,6 +2275,13 @@ tcl::namespace::eval punk::args {
dynamic { dynamic {
set is_dynamic 1 set is_dynamic 1
} }
normalize {
#G-045 - applied as a pre-pass over the split records (values
#are already re-based by the time this arm is reached)
if {[llength $at_specs]} {
error "punk::args::resolve - @normalize takes no options @id:$DEF_definition_id"
}
}
id { id {
#disallow duplicate @id line ? #disallow duplicate @id line ?
#review - nothing to stop multiple @id lines - or redefining as auto (which is ignored?) #review - nothing to stop multiple @id lines - or redefining as auto (which is ignored?)
@ -2724,7 +2881,7 @@ tcl::namespace::eval punk::args {
set examples_info [dict merge $examples_info $at_specs] set examples_info [dict merge $examples_info $at_specs]
} }
default { default {
error "punk::args::resolve - unrecognised @ line in '$rec'. Expected @id @cmd @form... @leaders @opts @values @doc @examples @formdisplay - use @@name if paramname needs to be @name @id:$DEF_definition_id" error "punk::args::resolve - unrecognised @ line in '$rec'. Expected @id @cmd @form... @leaders @opts @values @doc @examples @formdisplay @normalize - use @@name if paramname needs to be @name @id:$DEF_definition_id"
} }
} }
#record_type directive #record_type directive

3
src/modules/punk/args-buildversion.txt

@ -1,6 +1,7 @@
0.7.0 0.8.0
#First line must be a semantic version number #First line must be a semantic version number
#all other lines are ignored. #all other lines are ignored.
#0.8.0 - G-045: new bare @normalize directive - opts a definition into indent normalization of BLOCK-FORM multi-line field values (first line whitespace-only, as authored by opening a braced literal with a newline): the structural first newline and a whitespace-only trailing line are dropped, the content lines' common leading whitespace is the block's base indent, the first content line is unindented fully and subsequent lines re-based to the standard 4-space continuation convention (deeper relative indents preserved; whitespace-only inner lines become empty). Head-form values are never altered - their base indent is ambiguous (uniform continuation indent may be the deliberate +2 relative convention over base 4) - which makes @normalize a no-op on conforming file-style definitions (idempotence pinned). Fields in a record's -unindentedfields are exempt. Implemented as a resolve pre-pass over split records (private::normalize_records / rebase_multiline_value); @normalize with options is an error. Intended for constructed/string-built definitions (no whole-block indent treatment otherwise); ::punk::helptopic::define_docs converts to it as the consumer proof (punk 0.2.5). New testsuite normalize.test; define doc documents the directive; rendering.test P4 notes stay pinned as the unopted default.
#0.7.0 - G-045: record-continuation token -& - an unquoted trailing -& element on a definition record line continues the record on the next line. Implemented in private::split_definition_records: the token is dropped and the next line joins after a single space with its leading whitespace collapsed, exactly how the Tcl parser joins backslash-newline continuations before a braced definition reaches the splitter - so a -& record assembles byte-identical to its backslash-continued equivalent (proven by parse+render equality test). Motivation: constructed/string-built definitions cannot author backslash-newline ergonomically (the building code's own quoting consumes it); -& is plain text and survives any construction. Collision rules: the token must be a bare word preceded by whitespace (or the whole line), trailing whitespace after it is tolerated (more forgiving than raw backslash-newline); a braced/quoted -& is data ({-&} is the escape for a literal trailing value); -& mid-line, -& as a word suffix (abc-&), and -& on lines inside still-open braced/quoted values are all data. Backslash continuation authoring is unchanged (continuation is additive). New testsuite recordcontinuation.test; define doc documents the token alongside backslash continuation. #0.7.0 - G-045: record-continuation token -& - an unquoted trailing -& element on a definition record line continues the record on the next line. Implemented in private::split_definition_records: the token is dropped and the next line joins after a single space with its leading whitespace collapsed, exactly how the Tcl parser joins backslash-newline continuations before a braced definition reaches the splitter - so a -& record assembles byte-identical to its backslash-continued equivalent (proven by parse+render equality test). Motivation: constructed/string-built definitions cannot author backslash-newline ergonomically (the building code's own quoting consumes it); -& is plain text and survives any construction. Collision rules: the token must be a bare word preceded by whitespace (or the whole line), trailing whitespace after it is tolerated (more forgiving than raw backslash-newline); a braced/quoted -& is data ({-&} is the escape for a literal trailing value); -& mid-line, -& as a word suffix (abc-&), and -& on lines inside still-open braced/quoted values are all data. Backslash continuation authoring is unchanged (continuation is additive). New testsuite recordcontinuation.test; define doc documents the token alongside backslash continuation.
#0.6.1 - G-045: @cmd honours -unindentedfields for -help - arg_error's display-time indent transform (undent " "+help, max 4) is now gated by "-help" membership in the @cmd line's -unindentedfields list (same gate argument -help already had), so left-margin-authored cmd help renders its first line flush with continuations in both the table and string renderers. Previously the option was accepted on @cmd but ignored (rendering.test rendering_unindentedfields_cmd_help_GAP - flipped to rendering_unindentedfields_cmd_help). No in-tree definitions set @cmd -unindentedfields, so existing rendering is unchanged. Note: @cmd -summary has no indent transform in any renderer, so -unindentedfields membership for -summary is accepted and vacuously honoured. define doc for -unindentedfields now states where the option is valid. #0.6.1 - G-045: @cmd honours -unindentedfields for -help - arg_error's display-time indent transform (undent " "+help, max 4) is now gated by "-help" membership in the @cmd line's -unindentedfields list (same gate argument -help already had), so left-margin-authored cmd help renders its first line flush with continuations in both the table and string renderers. Previously the option was accepted on @cmd but ignored (rendering.test rendering_unindentedfields_cmd_help_GAP - flipped to rendering_unindentedfields_cmd_help). No in-tree definitions set @cmd -unindentedfields, so existing rendering is unchanged. Note: @cmd -summary has no indent transform in any renderer, so -unindentedfields membership for -summary is accepted and vacuously honoured. define doc for -unindentedfields now states where the option is valid.
#0.6.0 - synopsis display: small restricted choice sets now render as literal alternates - an argument whose choice pool (-choices plus -choicegroups members, deduplicated) has 1-3 members and -choicerestricted true (the default) displays those words unitalicised joined by | in synopses (e.g 'after cancel' shows literal cancel; a 3-choice option shows (left|centre|right)), matching the display style of literal()/literalprefix() type-alternatives. Larger or unrestricted (-choicerestricted 0) choice sets keep the italicised argname/<type> display, and an explicit -typesynopsis always takes precedence. New private helper punk::args::private::synopsis_choice_literals shared by both synopsis render paths (leaders/values via synopsis_form_arg_display, options inline in synopsis); applies only to single-element -type lists (multi-element clause display unchanged). define doc for -choices documents the rule. Tests: synopsis.test - new characterization coverage for literal/literalprefix/stringstartswith/stringendswith type-alternates, option alternate parenthesization, multi-element clause display, -typesynopsis (value element lists, option passthrough incl documenter ANSI), plus the new choice-literal rule (small sets in leader/option/value positions, choicegroups counting, >3 and unrestricted fallbacks, -typesynopsis precedence) #0.6.0 - synopsis display: small restricted choice sets now render as literal alternates - an argument whose choice pool (-choices plus -choicegroups members, deduplicated) has 1-3 members and -choicerestricted true (the default) displays those words unitalicised joined by | in synopses (e.g 'after cancel' shows literal cancel; a 3-choice option shows (left|centre|right)), matching the display style of literal()/literalprefix() type-alternatives. Larger or unrestricted (-choicerestricted 0) choice sets keep the italicised argname/<type> display, and an explicit -typesynopsis always takes precedence. New private helper punk::args::private::synopsis_choice_literals shared by both synopsis render paths (leaders/values via synopsis_form_arg_display, options inline in synopsis); applies only to single-element -type lists (multi-element clause display unchanged). define doc for -choices documents the rule. Tests: synopsis.test - new characterization coverage for literal/literalprefix/stringstartswith/stringendswith type-alternates, option alternate parenthesization, multi-element clause display, -typesynopsis (value element lists, option passthrough incl documenter ANSI), plus the new choice-literal rule (small sets in leader/option/value positions, choicegroups counting, >3 and unrestricted fallbacks, -typesynopsis precedence)

192
src/tests/modules/punk/args/testsuites/args/normalize.test

@ -0,0 +1,192 @@
package require tcltest
package require punk::args
package require punk::ansi
#@normalize directive (G-045): opt-in indent normalization of BLOCK-FORM multi-line
#field values for constructed (string-built) definitions. Semantics (user-confirmed
#re-base 2026-07-12; narrowed to block form during implementation): a value whose
#first line is whitespace-only is a block - the structural first newline and a
#whitespace-only trailing line are dropped, the content lines' common leading
#whitespace is the block's base indent, the first content line is unindented fully
#and subsequent lines are re-based to the 4-space convention (deeper relative
#indents preserved). Head-form values are NEVER altered: their base is ambiguous
#(uniform continuation indent may be the deliberate +2 relative convention over
#base 4, or a deeper flush base) - which also makes @normalize a no-op on
#conforming file-style definitions. Fields in a record's -unindentedfields are
#exempt. Without @normalize, constructed-def behaviour is unchanged - see the P4
#characterization in rendering.test.
namespace eval ::testspace {
namespace import ::tcltest::*
variable common {
set result ""
}
proc render_table {id} {
return [punk::ansi::ansistrip [punk::args::arg_error "" [punk::args::get_spec $id] -aserror 0]]
}
#column (0-based) at which a unique marker string appears in the rendered text, -1 if absent
proc markercol {rendered marker} {
foreach ln [split $rendered \n] {
set ix [string first $marker $ln]
if {$ix >= 0} {
return $ix
}
}
return -1
}
#added 2026-07-12 (agent, G-045) - the head-form boundary: base indent is
#ambiguous for head-form values so @normalize leaves them alone (deep indents
#leak exactly as in the unopted P4 characterization: display strips only 4)
test normalize_headform_untouched {@normalize: head-form values (content on the first line) are not altered - deep continuation indents render as authored}\
-setup $common -body {
set help "NFIRST line\n NFLUSH line\n NPLUS2 line"
set def ""
append def "@normalize" \n
append def "@id -id ::testspace::nz_head" \n
append def "@cmd -name testspace::nz_head -summary \"Head.\" -help \"$help\"" \n
append def "@values -min 0 -max 0"
punk::args::define $def
set r [render_table ::testspace::nz_head]
set n0 [markercol $r NFIRST]
lappend result [expr {[markercol $r NFLUSH] - $n0}]
lappend result [expr {[markercol $r NPLUS2] - $n0}]
}\
-cleanup {
punk::args::undefine ::testspace::nz_head 1
}\
-result [list 12 14]
#added 2026-07-12 (agent, G-045) - the block-form ergonomics: a braced value
#authored as a clean indented block (leading newline, closing-brace line) needs
#no manual string trim
test normalize_blockform_leading_newline {@normalize: block-form value (whitespace-only first line) drops the structural newline and renders flush}\
-setup $common -body {
set help {
BFIRST line
BFLUSH line
BPLUS2 line
}
set def ""
append def "@normalize" \n
append def "@id -id ::testspace::nz_block" \n
append def "@cmd -name testspace::nz_block -summary \"Block.\" -help \"$help\"" \n
append def "@values -min 0 -max 0"
punk::args::define $def
set r [render_table ::testspace::nz_block]
set b0 [markercol $r BFIRST]
lappend result [expr {[markercol $r BFLUSH] - $b0}]
lappend result [expr {[markercol $r BPLUS2] - $b0}]
#the structural leading newline is gone: BFIRST appears on the same
#rendered line as the Description: label
set desc_line ""
foreach ln [split $r \n] {
if {[string first "Description:" $ln] >= 0} {
set desc_line $ln
break
}
}
lappend result [expr {[string first "BFIRST" $desc_line] >= 0}]
}\
-cleanup {
punk::args::undefine ::testspace::nz_block 1
}\
-result [list 0 2 1]
#added 2026-07-12 (agent, G-045) - block form with zero base indent: left-margin
#content behind a structural leading newline re-bases to the convention (no
#-unindentedfields declaration needed)
test normalize_blockform_leftmargin_gains_base {@normalize: a block-form value with left-margin content is re-based and renders flush}\
-setup $common -body {
set help "\nLFIRST line\nLFLUSH line"
set def ""
append def "@normalize" \n
append def "@id -id ::testspace::nz_left" \n
append def "@cmd -name testspace::nz_left -summary \"Left.\" -help \"$help\"" \n
append def "@values -min 0 -max 0"
punk::args::define $def
set r [render_table ::testspace::nz_left]
lappend result [expr {[markercol $r LFLUSH] - [markercol $r LFIRST]}]
}\
-cleanup {
punk::args::undefine ::testspace::nz_left 1
}\
-result [list 0]
#added 2026-07-12 (agent, G-045) - exemption: an -unindentedfields field keeps
#its block-form value byte-exact (structural newline preserved, no re-base, no
#display transform) where a non-exempt block would lose the leading blank line
test normalize_unindentedfields_exempt {@normalize: fields in a record's -unindentedfields are exempt - block-form value kept literally}\
-setup $common -body {
set help "\n EFIRST line\n ESIX line"
set def ""
append def "@normalize" \n
append def "@id -id ::testspace::nz_exempt" \n
append def "@cmd -name testspace::nz_exempt -summary \"Exempt.\" -unindentedfields {-help} -help \"$help\"" \n
append def "@values -min 0 -max 0"
punk::args::define $def
set r [render_table ::testspace::nz_exempt]
#leading structural newline preserved: EFIRST is NOT on the Description: line
set desc_line ""
foreach ln [split $r \n] {
if {[string first "Description:" $ln] >= 0} {
set desc_line $ln
break
}
}
lappend result [expr {[string first "EFIRST" $desc_line] < 0}]
#6-space indents kept literally - both lines at the same column
lappend result [expr {[markercol $r ESIX] - [markercol $r EFIRST]}]
}\
-cleanup {
punk::args::undefine ::testspace::nz_exempt 1
}\
-result [list 1 0]
#added 2026-07-12 (agent, G-045) - idempotence: @normalize on a conforming
#file-style braced definition changes nothing
test normalize_idempotent_on_filestyle {@normalize: a braced file-style definition renders identically with and without the directive}\
-setup $common -body {
punk::args::define {
@id -id ::testspace::nz_fplain
@cmd -name testspace::nz_fstyle -summary\
"Filestyle."\
-help\
"FFIRST line.
FFLUSH line.
FPLUS2 line."
@values -min 1 -max 1
v1 -type string -help\
"val help line1
val help line2"
}
punk::args::define {
@normalize
@id -id ::testspace::nz_fnorm2
@cmd -name testspace::nz_fstyle -summary\
"Filestyle."\
-help\
"FFIRST line.
FFLUSH line.
FPLUS2 line."
@values -min 1 -max 1
v1 -type string -help\
"val help line1
val help line2"
}
set r_plain [render_table ::testspace::nz_fplain]
set r_norm [render_table ::testspace::nz_fnorm2]
#ids deliberately same length (nz_fplain/nz_fnorm2) so table geometry matches
lappend result [expr {[string map {nz_fnorm2 nz_fplain} $r_norm] eq $r_plain}]
}\
-cleanup {
punk::args::undefine ::testspace::nz_fplain 1
punk::args::undefine ::testspace::nz_fnorm2 1
}\
-result [list 1]
cleanupTests
}
namespace delete ::testspace

8
src/tests/modules/punk/args/testsuites/args/rendering.test

@ -21,8 +21,10 @@ package require punk::ansi
# normalization ('constructed' - not to be confused with the @dynamic directive): # normalization ('constructed' - not to be confused with the @dynamic directive):
# embedded continuation indentation is interpreted via the display-time # embedded continuation indentation is interpreted via the display-time
# undent(prefix4,max4) transform, so continuations written at exactly 4 spaces # undent(prefix4,max4) transform, so continuations written at exactly 4 spaces
# align flush with the first line, and any other depth misaligns (pinned as # align flush with the first line, and any other depth misaligns. This remains
# characterization - this is why ::punk::helptopic::define_docs pre-normalizes) # the (deliberate) default: G-045 added the opt-in @normalize directive for
# block-form values (see normalize.test) and this characterization pins the
# unopted behaviour builders rely on
# P5 tstr ${...} substitutions with MULTILINE results (command or variable) insert # P5 tstr ${...} substitutions with MULTILINE results (command or variable) insert
# aligned at the insertion column with the inserted text's own relative indents # aligned at the insertion column with the inserted text's own relative indents
# preserved (the -paramindents machinery) - including whole-definition insertion # preserved (the -paramindents machinery) - including whole-definition insertion
@ -281,7 +283,7 @@ GFLUSH line"
#--- P4 constructed (string-built) definitions ----------------------------------------- #--- P4 constructed (string-built) definitions -----------------------------------------
test rendering_constructed_def_indent_characterization {constructed (string-built) definitions: continuation indentation is absolute - exactly 4 spaces aligns flush; other depths shift (why constructed-def builders must pre-normalize)}\ test rendering_constructed_def_indent_characterization {constructed (string-built) definitions: continuation indentation is absolute - exactly 4 spaces aligns flush; other depths shift (opt-in remedy: the G-045 @normalize directive, normalize.test)}\
-setup $common -body { -setup $common -body {
#continuations at exactly 4 -> aligned flush with first line (the current #continuations at exactly 4 -> aligned flush with first line (the current
#correct authoring convention for constructed definitions) #correct authoring convention for constructed definitions)

Loading…
Cancel
Save