Browse Source

G-045 increment 3: punk::args 0.7.0 - record-continuation token -& (project 0.12.6)

An unquoted trailing -& element on a definition record line continues the
record on the next line. Implemented in
private::split_definition_records: the token is dropped and the next line
joins after a single space with its leading whitespace collapsed - exactly
how the Tcl parser joins backslash-newline continuations before a braced
definition ever reaches the splitter - so a -& record assembles
byte-identical to its backslash-continued equivalent. Motivation:
constructed (string-built) definitions cannot author backslash-newline
ergonomically (the building code's own quoting consumes it); -& is plain
text and survives any construction.

Collision rules (analysis and decision recorded in the goal detail file):
bare word preceded by whitespace (or whole line), trailing whitespace
tolerated; braced/quoted -& is data ({-&} escapes a literal trailing
value); mid-line -&, word-suffix abc-&, and -& inside still-open
braced/quoted values are data. Element-count disambiguation rejected
(action-at-a-distance; positional rule is locally decidable). Backslash
authoring unchanged - continuation is additive.

Tests: new recordcontinuation.test (6 tests) incl. backslash-twin
parse+render byte-equality and the constructed-def chaining case. Full
punk::args suite 181 pass / 1 pre-existing skip / 0 fail; full source-tree
suite 801 pass / 13 skip / 1 fail = exec-14.3 only (known baseline) - zero
regressions (tclsh 9.0.3). define doc documents -& alongside backslash
continuation.

Also recorded: tclparser considered and rejected for the splitter (ANSI
unbalanced-bracket data, dialect semantics outside Tcl grammar, binary
dependency vs G-004); dev-time parse cross-check lint flagged as candidate.

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 2 days ago
parent
commit
52ad1211a2
  1. 4
      CHANGELOG.md
  2. 60
      goals/G-045-punkargs-authoring-ergonomics.md
  3. 2
      punkproject.toml
  4. 58
      src/modules/punk/args-999999.0a1.0.tm
  5. 3
      src/modules/punk/args-buildversion.txt
  6. 182
      src/tests/modules/punk/args/testsuites/args/recordcontinuation.test

4
CHANGELOG.md

@ -5,6 +5,10 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to
Entries are newest-first; one bullet per notable change. See the root `AGENTS.md`
"Project Versioning" section for the bump policy.
## [0.12.6] - 2026-07-12
- G-045 (increment): punk::args 0.7.0 adds the -& record-continuation token - an unquoted trailing -& on a definition record line continues the record on the next line, assembling byte-identically to the equivalent backslash continuation. Intended for constructed (string-built) definitions, where backslash-newline is consumed by the building code's own quoting. Brace a literal trailing -& value ({-&}); -& elsewhere on a line or inside braced/quoted multi-line values is ordinary data. Backslash-continuation authoring is unchanged.
## [0.12.5] - 2026-07-12
- G-045 (increment): 'i help' usage table alignment fixed (punk 0.2.4) - the Description block's continuations no longer render indented +12 relative to the first line, and the topic argument's help first line no longer renders +4 relative to its continuations. ::punk::helptopic::define_docs authors help text at the left margin with -unindentedfields {-help} on the @cmd and topic lines. Help text content unchanged.

60
goals/G-045-punkargs-authoring-ergonomics.md

@ -28,7 +28,7 @@ tests before changes in this area). The pain points, in the user's framing:
string as a de-facto record continuation - functional but not designed-for, and
unsatisfying.
## The -& record-continuation candidate
## The -& record-continuation candidate (implemented 2026-07-12 - see Progress increment 3)
The user's proposal: a special token the record parser recognises as record continuation,
e.g. an unquoted `-&` at end of line. Design considerations recorded for implementation:
@ -45,6 +45,32 @@ e.g. an unquoted `-&` at end of line. Design considerations recorded for impleme
CONSTRUCTED definitions too - which is precisely why it helps: string-built defs cannot
use backslash-newline ergonomically (it is consumed by the building code's own quoting).
Decision as implemented (punk::args 0.7.0):
- Rule: unquoted trailing `-&` only - the token must be a bare word preceded by
whitespace (or be the whole line) and be the last element on the line; trailing
whitespace after it is tolerated (deliberately more forgiving than raw
backslash-newline, whose invisible-trailing-whitespace failure is a classic trap).
- Escape story: brace or double-quote a literal trailing -& value ({-&}) - the raw line
then ends with the closing delimiter and never matches. A -& mid-line, as a word
suffix (abc-&), or on a line inside a still-open braced/quoted value is data.
- Element-count disambiguation: NOT implemented. The positional rule is deterministic
and locally decidable per line; element counting would require assembling and
list-parsing the record first (fragile against in-progress records) and gives
confusing action-at-a-distance when a distant key/value slips the count. Rejected
rather than deferred - the escape story covers the residual collision (a literal
trailing -default -& must be braced).
- Assembly semantics: byte-identity with the backslash equivalent. Key finding: for
braced (file-style) definitions Tcl itself collapses backslash-newline plus following
whitespace to a single space BEFORE the text reaches the record splitter - so -& is
implemented the same way (token dropped, single-space join, next line's leading
whitespace collapsed), making the assembled record byte-identical to its
backslash-continued twin. Proven by parse-result and rendered-table equality in
recordcontinuation.test.
- Reserved-word consequence: an argument cannot be NAMED -& via a record line ending in
the bare token (e.g. a lone '-& ' line would read as a continuation). Any such need is
met by bracing. Considered acceptable and documented here rather than guarded in code.
## @cmd -unindentedfields
Accepted but ignored today (arg_error's cmd_info rendering carries an '#unindentedfields ?'
@ -135,9 +161,37 @@ punk::imap4's {\Deleted}/{$MDNSent} choice values) have them stated, not just te
rendering_unindentedfields_cmd_help), but it is not the preferred style for
define_docs itself.
### 2026-07-12 increment 3: -& record continuation (punk::args 0.7.0)
- Implemented in private::split_definition_records per the decision recorded in
"The -& record-continuation candidate" section above (unquoted trailing token,
brace escape, element-count alternative rejected, byte-identical assembly to
the backslash equivalent).
- First implementation attempt kept a literal backslash-newline in the assembled
record and failed: braced definitions never contain backslash-newline by the
time they reach the splitter (Tcl pre-collapses it), and downstream record
handling assumes the collapsed shape. The collapse-to-single-space rewrite is
the correct equivalence and is what landed.
- New testsuite recordcontinuation.test (6 tests): backslash-twin parse+render
equality (same-length ids so table geometry matches), constructed-def chaining
incl. into a multi-line quoted value, braced {-&} escape, mid-line -&,
word-suffix abc-&, -& inside still-open braces.
- define doc documents -& alongside the backslash continuation paragraph.
- Verified (tclsh 9.0.3): full punk::args suite 181 pass / 1 pre-existing skip /
0 fail; full source-tree suite (splitter regression sweep) 815 total, 801
pass, 13 skip, 1 fail = exec-14.3 only (the known pre-existing core-test
baseline) - zero regressions.
- tclparser considered and rejected for the splitter (user question 2026-07-12):
parse command mis-tokenizes the dialect's legitimate unbalanced-bracket ANSI
data inside quoted values (info complete gets a throwaway ansistripped copy;
parse would need strip+range-remapping), the punk semantics (base-indent line
trimming, -&) sit outside Tcl's grammar either way, and a binary dependency in
punk::args inverts G-004 for the module that must load everywhere. tclparser
remains right for real-script analysis (punk::lib/G-019). Candidate flagged:
dev-time lint cross-checking splitter boundaries against parse where the
binary is available.
Remaining for acceptance:
- record-continuation mechanism (the -& candidate): collision rules, escape
story, parses identically to backslash-continuation equivalent, documented
- constructed-definition whole-block normalization opt-in, with
rendering_constructed_def_indent_characterization expectations updated and
::punk::helptopic::define_docs converted from interim left-margin authoring

2
punkproject.toml

@ -1,4 +1,4 @@
[project]
name = "punkshell"
version = "0.12.5"
version = "0.12.6"
license = "BSD-2-Clause"

58
src/modules/punk/args-999999.0a1.0.tm

@ -694,6 +694,14 @@ tcl::namespace::eval punk::args {
The line continuation character
(\\ at the end of the line) can be used to continue the set of arguments for
a leading word.
The record-continuation token -& does the same job when it appears unquoted
as the last element on a line (trailing whitespace after it is tolerated):
the record continues on the next line, exactly as with a trailing \\.
Unlike a backslash, -& survives inside constructed (string-built)
definitions, where the building code's own quoting would consume a
backslash-newline. To pass a literal -& as a trailing value, brace or
quote it (e.g -default {-&}); a -& elsewhere on a line, or inside a braced
or quoted multi-line value, is ordinary data.
Leading words beginning with the @ character are directives controlling argument
parsing, defaults for subsequent arguments, and help display.
directives include:
@ -1371,7 +1379,16 @@ tcl::namespace::eval punk::args {
}
set record_id 0
set record_line 0 ;#incremented at each incomplete record, set to zero after processing a complete record
set amp_trimnext 0 ;#previous line ended with the -& record-continuation token (G-045)
foreach rawline $linelist {
if {$amp_trimnext} {
#the previous line ended with the -& record-continuation token and
#linebuild already ends with the joining space: collapse this line's
#leading whitespace, as the Tcl parser does for a backslash-newline
#continuation
set rawline [tcl::string::trimleft $rawline]
set amp_trimnext 0
}
set record_so_far [tcl::string::cat $linebuild $rawline]
#ansi colours can stop info complete from working (contain square brackets)
#review - when exactly are ansi codes allowed/expected in record lines.
@ -1406,6 +1423,24 @@ tcl::namespace::eval punk::args {
incr record_line
} else {
#either we're on a single line record, or last line of multiline record
#G-045 record continuation: an unquoted trailing -& element continues
#the record on the next line. The token is dropped and the next line
#joins after a single space with its leading whitespace collapsed -
#exactly how the Tcl parser joins a backslash-newline continuation
#before a braced definition ever reaches this splitter - so a -& record
#assembles byte-identical to its backslash-continued equivalent.
#(Constructed/string-built definitions cannot author backslash-newline
#ergonomically - the building code's own quoting consumes it - which is
#what -& is for.) Rules: the token must be a bare word preceded by
#whitespace, or be the whole line; trailing whitespace after it is
#tolerated. A braced or double-quoted -& never matches (the raw line
#then ends with the closing delimiter) - brace a literal trailing -&
#value: {-&}. A -& inside a still-open braced/quoted value is data
#(info complete is false for those lines - handled in the branch above).
set is_ampcontinuation [regexp {(?:^|\s)-&\s*$} $rawline]
if {$is_ampcontinuation} {
regsub {\s*-&\s*$} $rawline "" rawline
}
if {$record_line != 0} {
if {[tcl::string::first $record_base_indent $rawline] == 0} {
set trimmedline [tcl::string::range $rawline [tcl::string::length $record_base_indent] end]
@ -1416,12 +1451,23 @@ tcl::namespace::eval punk::args {
} else {
append linebuild $rawline
}
lappend records $linebuild
set linebuild ""
#prep for next record
set in_record_continuation 0
incr record_id
set record_line 0
if {$is_ampcontinuation} {
#record continues on the next line
append linebuild " "
set amp_trimnext 1
if {$record_line == 0} {
regexp {(\s*).*} $rawline _all record_base_indent
set in_record_continuation 1
}
incr record_line
} else {
lappend records $linebuild
set linebuild ""
#prep for next record
set in_record_continuation 0
incr record_id
set record_line 0
}
}
}
if {$in_record_continuation} {

3
src/modules/punk/args-buildversion.txt

@ -1,6 +1,7 @@
0.6.1
0.7.0
#First line must be a semantic version number
#all other lines are ignored.
#0.7.0 - G-045: record-continuation token -& - an unquoted trailing -& element on a definition record line continues the record on the next line. Implemented in private::split_definition_records: the token is dropped and the next line joins after a single space with its leading whitespace collapsed, exactly how the Tcl parser joins backslash-newline continuations before a braced definition reaches the splitter - so a -& record assembles byte-identical to its backslash-continued equivalent (proven by parse+render equality test). Motivation: constructed/string-built definitions cannot author backslash-newline ergonomically (the building code's own quoting consumes it); -& is plain text and survives any construction. Collision rules: the token must be a bare word preceded by whitespace (or the whole line), trailing whitespace after it is tolerated (more forgiving than raw backslash-newline); a braced/quoted -& is data ({-&} is the escape for a literal trailing value); -& mid-line, -& as a word suffix (abc-&), and -& on lines inside still-open braced/quoted values are all data. Backslash continuation authoring is unchanged (continuation is additive). New testsuite recordcontinuation.test; define doc documents the token alongside backslash continuation.
#0.6.1 - G-045: @cmd honours -unindentedfields for -help - arg_error's display-time indent transform (undent " "+help, max 4) is now gated by "-help" membership in the @cmd line's -unindentedfields list (same gate argument -help already had), so left-margin-authored cmd help renders its first line flush with continuations in both the table and string renderers. Previously the option was accepted on @cmd but ignored (rendering.test rendering_unindentedfields_cmd_help_GAP - flipped to rendering_unindentedfields_cmd_help). No in-tree definitions set @cmd -unindentedfields, so existing rendering is unchanged. Note: @cmd -summary has no indent transform in any renderer, so -unindentedfields membership for -summary is accepted and vacuously honoured. define doc for -unindentedfields now states where the option is valid.
#0.6.0 - synopsis display: small restricted choice sets now render as literal alternates - an argument whose choice pool (-choices plus -choicegroups members, deduplicated) has 1-3 members and -choicerestricted true (the default) displays those words unitalicised joined by | in synopses (e.g 'after cancel' shows literal cancel; a 3-choice option shows (left|centre|right)), matching the display style of literal()/literalprefix() type-alternatives. Larger or unrestricted (-choicerestricted 0) choice sets keep the italicised argname/<type> display, and an explicit -typesynopsis always takes precedence. New private helper punk::args::private::synopsis_choice_literals shared by both synopsis render paths (leaders/values via synopsis_form_arg_display, options inline in synopsis); applies only to single-element -type lists (multi-element clause display unchanged). define doc for -choices documents the rule. Tests: synopsis.test - new characterization coverage for literal/literalprefix/stringstartswith/stringendswith type-alternates, option alternate parenthesization, multi-element clause display, -typesynopsis (value element lists, option passthrough incl documenter ANSI), plus the new choice-literal rule (small sets in leader/option/value positions, choicegroups counting, >3 and unrestricted fallbacks, -typesynopsis precedence)
#0.5.0 - G-049 parse-status data model: new punk::args::parse_status - runs a parse attempt (withid/withdef) and returns a documented status structure instead of raising on validation failure (overall ok/status valid|invalid|incomplete/scheme/message/errorcode-minus-argspecs/failureclass/badarg/id/form/receivednames + per-argument argstatus with class/status ok|bad|unparsed/received/positions/hasvalue/value-in-effect incl -default fill). arg_error: new -parsestatus option - both renderers (table and string) now derive goodarg/badarg row marking and choice value-in-effect highlighting from the structure (built internally from -badarg/-parsedargs when not supplied), replacing the transient goodargs/badarg locals; scheme colours resolve per-render into a local array (scheme renders no longer mutate the shared arg_error_CLR array - the -nocolour leak) and the DOCUMENTED -scheme choice value 'nocolour' (and 'nocolor') now takes effect instead of falling through to leftover colours. parse: new -caller option overriding the %caller% frame-walk substitution in validation failure messages (parse_status defaults it to the definition's @cmd -name). get_dict: missingrequiredvalue/missingrequiredleader allocation failures now carry -badarg (type-failed words get badarg marking, not just choice violations). Tests: parsestatus.test (new), usagemarking.test nocolour/leak GAP pins flipped + -parsestatus render parity

182
src/tests/modules/punk/args/testsuites/args/recordcontinuation.test

@ -0,0 +1,182 @@
package require tcltest
package require punk::args
package require punk::ansi
#Record-continuation token -& (G-045): an unquoted trailing -& element continues a
#definition record on the next line, rewritten internally to a backslash
#line-continuation so the assembled record is byte-identical to its
#backslash-continued equivalent. Key properties pinned here:
# - equivalence: a -& definition parses AND renders identically to its
# backslash-continuation twin (continuation is additive - backslash authoring
# unchanged)
# - the motivating case: constructed (string-built) definitions can use -& where
# a backslash-newline would be consumed by the building code's own quoting
# - collision/escape rules: a braced trailing {-&} is data; -& mid-line is data;
# a word merely ending in the characters -& is data; -& on a line inside a
# still-open braced value is data
namespace eval ::testspace {
namespace import ::tcltest::*
variable common {
set result ""
}
proc render_table {id} {
return [punk::ansi::ansistrip [punk::args::arg_error "" [punk::args::get_spec $id] -aserror 0]]
}
#added 2026-07-12 (agent, G-045)
#the two ids are deliberately the same length: the id appears in the rendered
#synopsis, so differing lengths would change table geometry and defeat the
#render-equality comparison after the id string map
test recordcontinuation_amp_equivalent_to_backslash {a definition using trailing -& parses and renders identically to its backslash-continuation equivalent}\
-setup $common -body {
punk::args::define {
@id -id ::testspace::rc_bsl
@cmd -name testspace::rc_fixture -summary\
"Fixture."\
-help\
"CFIRST line.
CFLUSH line."
@opts
-o1 -type string -default od1 -help\
"opt help line1
opt help line2"
@values -min 1 -max 1
v1 -type string -help\
"val help."
}
punk::args::define {
@id -id ::testspace::rc_amp
@cmd -name testspace::rc_fixture -summary -&
"Fixture." -&
-help -&
"CFIRST line.
CFLUSH line."
@opts
-o1 -type string -default od1 -help -&
"opt help line1
opt help line2"
@values -min 1 -max 1
v1 -type string -help -&
"val help."
}
set r_b [punk::args::parse {-o1 X VAL1} withid ::testspace::rc_bsl]
set r_a [punk::args::parse {-o1 X VAL1} withid ::testspace::rc_amp]
lappend result [expr {[string map {rc_amp rc_bsl} $r_a] eq $r_b}]
set t_b [render_table ::testspace::rc_bsl]
set t_a [render_table ::testspace::rc_amp]
lappend result [expr {[string map {rc_amp rc_bsl} $t_a] eq $t_b}]
}\
-cleanup {
punk::args::undefine ::testspace::rc_bsl 1
punk::args::undefine ::testspace::rc_amp 1
}\
-result [list 1 1]
#added 2026-07-12 (agent, G-045) - the motivating case: string-built definitions
#cannot use backslash-newline (consumed by the builder's own quoting) but can use -&
test recordcontinuation_amp_constructed_definition {a constructed (string-built) definition using -& chains record lines, including into a multi-line quoted value}\
-setup $common -body {
set def ""
append def "@id -id ::testspace::rc_chain" \n
append def "@cmd -name testspace::rc_chain -summary \"Chain.\" -help \"chain help\"" \n
append def "@opts" \n
append def "-o1 -type string -&" \n
append def "-default HDEF -& " \n
append def "-help \"h line1" \n
append def "h line2\"" \n
append def "@values -min 0 -max 0"
punk::args::define $def
set argd [punk::args::parse {} withid ::testspace::rc_chain]
lappend result [dict get [dict get $argd opts] -o1]
set t [render_table ::testspace::rc_chain]
lappend result [expr {[string first "h line1" $t] >= 0}]
lappend result [expr {[string first "h line2" $t] >= 0}]
}\
-cleanup {
punk::args::undefine ::testspace::rc_chain 1
}\
-result [list HDEF 1 1]
#added 2026-07-12 (agent, G-045)
test recordcontinuation_amp_braced_escape_is_data {a braced trailing {-&} is a literal value - the record ends and the next line is a separate record}\
-setup $common -body {
punk::args::define {
@id -id ::testspace::rc_braced
@cmd -name testspace::rc_braced -summary "Braced." -help "braced help"
@opts
-o1 -type string -default {-&}
-o2 -type string -default d2
@values -min 0 -max 0
}
set argd [punk::args::parse {} withid ::testspace::rc_braced]
lappend result [dict get [dict get $argd opts] -o1]
lappend result [dict get [dict get $argd opts] -o2]
}\
-cleanup {
punk::args::undefine ::testspace::rc_braced 1
}\
-result [list -& d2]
#added 2026-07-12 (agent, G-045)
test recordcontinuation_amp_midline_is_data {an unquoted -& that is not the last element on the line is an ordinary value}\
-setup $common -body {
punk::args::define {
@id -id ::testspace::rc_midline
@cmd -name testspace::rc_midline -summary "Midline." -help "midline help"
@opts
-o1 -default -& -type string
@values -min 0 -max 0
}
set argd [punk::args::parse {} withid ::testspace::rc_midline]
lappend result [dict get [dict get $argd opts] -o1]
}\
-cleanup {
punk::args::undefine ::testspace::rc_midline 1
}\
-result [list -&]
#added 2026-07-12 (agent, G-045)
test recordcontinuation_amp_wordend_is_data {a word merely ending in the characters -& is not a continuation token}\
-setup $common -body {
punk::args::define {
@id -id ::testspace::rc_wordend
@cmd -name testspace::rc_wordend -summary "Wordend." -help "wordend help"
@opts
-o1 -type string -default abc-&
-o2 -type string -default d2
@values -min 0 -max 0
}
set argd [punk::args::parse {} withid ::testspace::rc_wordend]
lappend result [dict get [dict get $argd opts] -o1]
lappend result [dict get [dict get $argd opts] -o2]
}\
-cleanup {
punk::args::undefine ::testspace::rc_wordend 1
}\
-result [list abc-& d2]
#added 2026-07-12 (agent, G-045)
test recordcontinuation_amp_inside_braces_is_data {a line ending in -& inside a still-open braced value stays literal data}\
-setup $common -body {
punk::args::define {
@id -id ::testspace::rc_inbrace
@cmd -name testspace::rc_inbrace -summary "Inbrace." -help "inbrace help"
@values -min 1 -max 1
v1 -type string -help {AFIRST -&
ASECOND line}
}
set t [render_table ::testspace::rc_inbrace]
lappend result [expr {[string first "AFIRST -&" $t] >= 0}]
lappend result [expr {[string first "ASECOND line" $t] >= 0}]
}\
-cleanup {
punk::args::undefine ::testspace::rc_inbrace 1
}\
-result [list 1 1]
cleanupTests
}
namespace delete ::testspace
Loading…
Cancel
Save