diff --git a/CHANGELOG.md b/CHANGELOG.md index fe5527ab..538320aa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ The latest `## [X.Y.Z]` header must match the `version` field in `punkproject.to Entries are newest-first; one bullet per notable change. See the root `AGENTS.md` "Project Versioning" section for the bump policy. +## [0.12.6] - 2026-07-12 + +- G-045 (increment): punk::args 0.7.0 adds the -& record-continuation token - an unquoted trailing -& on a definition record line continues the record on the next line, assembling byte-identically to the equivalent backslash continuation. Intended for constructed (string-built) definitions, where backslash-newline is consumed by the building code's own quoting. Brace a literal trailing -& value ({-&}); -& elsewhere on a line or inside braced/quoted multi-line values is ordinary data. Backslash-continuation authoring is unchanged. + ## [0.12.5] - 2026-07-12 - G-045 (increment): 'i help' usage table alignment fixed (punk 0.2.4) - the Description block's continuations no longer render indented +12 relative to the first line, and the topic argument's help first line no longer renders +4 relative to its continuations. ::punk::helptopic::define_docs authors help text at the left margin with -unindentedfields {-help} on the @cmd and topic lines. Help text content unchanged. diff --git a/goals/G-045-punkargs-authoring-ergonomics.md b/goals/G-045-punkargs-authoring-ergonomics.md index 01a861a2..b9a2aac6 100644 --- a/goals/G-045-punkargs-authoring-ergonomics.md +++ b/goals/G-045-punkargs-authoring-ergonomics.md @@ -28,7 +28,7 @@ tests before changes in this area). The pain points, in the user's framing: string as a de-facto record continuation - functional but not designed-for, and unsatisfying. -## The -& record-continuation candidate +## The -& record-continuation candidate (implemented 2026-07-12 - see Progress increment 3) The user's proposal: a special token the record parser recognises as record continuation, e.g. an unquoted `-&` at end of line. Design considerations recorded for implementation: @@ -45,6 +45,32 @@ e.g. an unquoted `-&` at end of line. Design considerations recorded for impleme CONSTRUCTED definitions too - which is precisely why it helps: string-built defs cannot use backslash-newline ergonomically (it is consumed by the building code's own quoting). +Decision as implemented (punk::args 0.7.0): + +- Rule: unquoted trailing `-&` only - the token must be a bare word preceded by + whitespace (or be the whole line) and be the last element on the line; trailing + whitespace after it is tolerated (deliberately more forgiving than raw + backslash-newline, whose invisible-trailing-whitespace failure is a classic trap). +- Escape story: brace or double-quote a literal trailing -& value ({-&}) - the raw line + then ends with the closing delimiter and never matches. A -& mid-line, as a word + suffix (abc-&), or on a line inside a still-open braced/quoted value is data. +- Element-count disambiguation: NOT implemented. The positional rule is deterministic + and locally decidable per line; element counting would require assembling and + list-parsing the record first (fragile against in-progress records) and gives + confusing action-at-a-distance when a distant key/value slips the count. Rejected + rather than deferred - the escape story covers the residual collision (a literal + trailing -default -& must be braced). +- Assembly semantics: byte-identity with the backslash equivalent. Key finding: for + braced (file-style) definitions Tcl itself collapses backslash-newline plus following + whitespace to a single space BEFORE the text reaches the record splitter - so -& is + implemented the same way (token dropped, single-space join, next line's leading + whitespace collapsed), making the assembled record byte-identical to its + backslash-continued twin. Proven by parse-result and rendered-table equality in + recordcontinuation.test. +- Reserved-word consequence: an argument cannot be NAMED -& via a record line ending in + the bare token (e.g. a lone '-& ' line would read as a continuation). Any such need is + met by bracing. Considered acceptable and documented here rather than guarded in code. + ## @cmd -unindentedfields Accepted but ignored today (arg_error's cmd_info rendering carries an '#unindentedfields ?' @@ -135,9 +161,37 @@ punk::imap4's {\Deleted}/{$MDNSent} choice values) have them stated, not just te rendering_unindentedfields_cmd_help), but it is not the preferred style for define_docs itself. +### 2026-07-12 increment 3: -& record continuation (punk::args 0.7.0) + +- Implemented in private::split_definition_records per the decision recorded in + "The -& record-continuation candidate" section above (unquoted trailing token, + brace escape, element-count alternative rejected, byte-identical assembly to + the backslash equivalent). +- First implementation attempt kept a literal backslash-newline in the assembled + record and failed: braced definitions never contain backslash-newline by the + time they reach the splitter (Tcl pre-collapses it), and downstream record + handling assumes the collapsed shape. The collapse-to-single-space rewrite is + the correct equivalence and is what landed. +- New testsuite recordcontinuation.test (6 tests): backslash-twin parse+render + equality (same-length ids so table geometry matches), constructed-def chaining + incl. into a multi-line quoted value, braced {-&} escape, mid-line -&, + word-suffix abc-&, -& inside still-open braces. +- define doc documents -& alongside the backslash continuation paragraph. +- Verified (tclsh 9.0.3): full punk::args suite 181 pass / 1 pre-existing skip / + 0 fail; full source-tree suite (splitter regression sweep) 815 total, 801 + pass, 13 skip, 1 fail = exec-14.3 only (the known pre-existing core-test + baseline) - zero regressions. +- tclparser considered and rejected for the splitter (user question 2026-07-12): + parse command mis-tokenizes the dialect's legitimate unbalanced-bracket ANSI + data inside quoted values (info complete gets a throwaway ansistripped copy; + parse would need strip+range-remapping), the punk semantics (base-indent line + trimming, -&) sit outside Tcl's grammar either way, and a binary dependency in + punk::args inverts G-004 for the module that must load everywhere. tclparser + remains right for real-script analysis (punk::lib/G-019). Candidate flagged: + dev-time lint cross-checking splitter boundaries against parse where the + binary is available. + Remaining for acceptance: -- record-continuation mechanism (the -& candidate): collision rules, escape - story, parses identically to backslash-continuation equivalent, documented - constructed-definition whole-block normalization opt-in, with rendering_constructed_def_indent_characterization expectations updated and ::punk::helptopic::define_docs converted from interim left-margin authoring diff --git a/punkproject.toml b/punkproject.toml index 397b5f24..73de5355 100644 --- a/punkproject.toml +++ b/punkproject.toml @@ -1,4 +1,4 @@ [project] name = "punkshell" -version = "0.12.5" +version = "0.12.6" license = "BSD-2-Clause" diff --git a/src/modules/punk/args-999999.0a1.0.tm b/src/modules/punk/args-999999.0a1.0.tm index b3b2aa48..9be3abe2 100644 --- a/src/modules/punk/args-999999.0a1.0.tm +++ b/src/modules/punk/args-999999.0a1.0.tm @@ -694,6 +694,14 @@ tcl::namespace::eval punk::args { The line continuation character (\\ at the end of the line) can be used to continue the set of arguments for a leading word. + The record-continuation token -& does the same job when it appears unquoted + as the last element on a line (trailing whitespace after it is tolerated): + the record continues on the next line, exactly as with a trailing \\. + Unlike a backslash, -& survives inside constructed (string-built) + definitions, where the building code's own quoting would consume a + backslash-newline. To pass a literal -& as a trailing value, brace or + quote it (e.g -default {-&}); a -& elsewhere on a line, or inside a braced + or quoted multi-line value, is ordinary data. Leading words beginning with the @ character are directives controlling argument parsing, defaults for subsequent arguments, and help display. directives include: @@ -1371,7 +1379,16 @@ tcl::namespace::eval punk::args { } set record_id 0 set record_line 0 ;#incremented at each incomplete record, set to zero after processing a complete record + set amp_trimnext 0 ;#previous line ended with the -& record-continuation token (G-045) foreach rawline $linelist { + if {$amp_trimnext} { + #the previous line ended with the -& record-continuation token and + #linebuild already ends with the joining space: collapse this line's + #leading whitespace, as the Tcl parser does for a backslash-newline + #continuation + set rawline [tcl::string::trimleft $rawline] + set amp_trimnext 0 + } set record_so_far [tcl::string::cat $linebuild $rawline] #ansi colours can stop info complete from working (contain square brackets) #review - when exactly are ansi codes allowed/expected in record lines. @@ -1406,6 +1423,24 @@ tcl::namespace::eval punk::args { incr record_line } else { #either we're on a single line record, or last line of multiline record + #G-045 record continuation: an unquoted trailing -& element continues + #the record on the next line. The token is dropped and the next line + #joins after a single space with its leading whitespace collapsed - + #exactly how the Tcl parser joins a backslash-newline continuation + #before a braced definition ever reaches this splitter - so a -& record + #assembles byte-identical to its backslash-continued equivalent. + #(Constructed/string-built definitions cannot author backslash-newline + #ergonomically - the building code's own quoting consumes it - which is + #what -& is for.) Rules: the token must be a bare word preceded by + #whitespace, or be the whole line; trailing whitespace after it is + #tolerated. A braced or double-quoted -& never matches (the raw line + #then ends with the closing delimiter) - brace a literal trailing -& + #value: {-&}. A -& inside a still-open braced/quoted value is data + #(info complete is false for those lines - handled in the branch above). + set is_ampcontinuation [regexp {(?:^|\s)-&\s*$} $rawline] + if {$is_ampcontinuation} { + regsub {\s*-&\s*$} $rawline "" rawline + } if {$record_line != 0} { if {[tcl::string::first $record_base_indent $rawline] == 0} { set trimmedline [tcl::string::range $rawline [tcl::string::length $record_base_indent] end] @@ -1416,12 +1451,23 @@ tcl::namespace::eval punk::args { } else { append linebuild $rawline } - lappend records $linebuild - set linebuild "" - #prep for next record - set in_record_continuation 0 - incr record_id - set record_line 0 + if {$is_ampcontinuation} { + #record continues on the next line + append linebuild " " + set amp_trimnext 1 + if {$record_line == 0} { + regexp {(\s*).*} $rawline _all record_base_indent + set in_record_continuation 1 + } + incr record_line + } else { + lappend records $linebuild + set linebuild "" + #prep for next record + set in_record_continuation 0 + incr record_id + set record_line 0 + } } } if {$in_record_continuation} { diff --git a/src/modules/punk/args-buildversion.txt b/src/modules/punk/args-buildversion.txt index 0705b740..dad3a580 100644 --- a/src/modules/punk/args-buildversion.txt +++ b/src/modules/punk/args-buildversion.txt @@ -1,6 +1,7 @@ -0.6.1 +0.7.0 #First line must be a semantic version number #all other lines are ignored. +#0.7.0 - G-045: record-continuation token -& - an unquoted trailing -& element on a definition record line continues the record on the next line. Implemented in private::split_definition_records: the token is dropped and the next line joins after a single space with its leading whitespace collapsed, exactly how the Tcl parser joins backslash-newline continuations before a braced definition reaches the splitter - so a -& record assembles byte-identical to its backslash-continued equivalent (proven by parse+render equality test). Motivation: constructed/string-built definitions cannot author backslash-newline ergonomically (the building code's own quoting consumes it); -& is plain text and survives any construction. Collision rules: the token must be a bare word preceded by whitespace (or the whole line), trailing whitespace after it is tolerated (more forgiving than raw backslash-newline); a braced/quoted -& is data ({-&} is the escape for a literal trailing value); -& mid-line, -& as a word suffix (abc-&), and -& on lines inside still-open braced/quoted values are all data. Backslash continuation authoring is unchanged (continuation is additive). New testsuite recordcontinuation.test; define doc documents the token alongside backslash continuation. #0.6.1 - G-045: @cmd honours -unindentedfields for -help - arg_error's display-time indent transform (undent " "+help, max 4) is now gated by "-help" membership in the @cmd line's -unindentedfields list (same gate argument -help already had), so left-margin-authored cmd help renders its first line flush with continuations in both the table and string renderers. Previously the option was accepted on @cmd but ignored (rendering.test rendering_unindentedfields_cmd_help_GAP - flipped to rendering_unindentedfields_cmd_help). No in-tree definitions set @cmd -unindentedfields, so existing rendering is unchanged. Note: @cmd -summary has no indent transform in any renderer, so -unindentedfields membership for -summary is accepted and vacuously honoured. define doc for -unindentedfields now states where the option is valid. #0.6.0 - synopsis display: small restricted choice sets now render as literal alternates - an argument whose choice pool (-choices plus -choicegroups members, deduplicated) has 1-3 members and -choicerestricted true (the default) displays those words unitalicised joined by | in synopses (e.g 'after cancel' shows literal cancel; a 3-choice option shows (left|centre|right)), matching the display style of literal()/literalprefix() type-alternatives. Larger or unrestricted (-choicerestricted 0) choice sets keep the italicised argname/ display, and an explicit -typesynopsis always takes precedence. New private helper punk::args::private::synopsis_choice_literals shared by both synopsis render paths (leaders/values via synopsis_form_arg_display, options inline in synopsis); applies only to single-element -type lists (multi-element clause display unchanged). define doc for -choices documents the rule. Tests: synopsis.test - new characterization coverage for literal/literalprefix/stringstartswith/stringendswith type-alternates, option alternate parenthesization, multi-element clause display, -typesynopsis (value element lists, option passthrough incl documenter ANSI), plus the new choice-literal rule (small sets in leader/option/value positions, choicegroups counting, >3 and unrestricted fallbacks, -typesynopsis precedence) #0.5.0 - G-049 parse-status data model: new punk::args::parse_status - runs a parse attempt (withid/withdef) and returns a documented status structure instead of raising on validation failure (overall ok/status valid|invalid|incomplete/scheme/message/errorcode-minus-argspecs/failureclass/badarg/id/form/receivednames + per-argument argstatus with class/status ok|bad|unparsed/received/positions/hasvalue/value-in-effect incl -default fill). arg_error: new -parsestatus option - both renderers (table and string) now derive goodarg/badarg row marking and choice value-in-effect highlighting from the structure (built internally from -badarg/-parsedargs when not supplied), replacing the transient goodargs/badarg locals; scheme colours resolve per-render into a local array (scheme renders no longer mutate the shared arg_error_CLR array - the -nocolour leak) and the DOCUMENTED -scheme choice value 'nocolour' (and 'nocolor') now takes effect instead of falling through to leftover colours. parse: new -caller option overriding the %caller% frame-walk substitution in validation failure messages (parse_status defaults it to the definition's @cmd -name). get_dict: missingrequiredvalue/missingrequiredleader allocation failures now carry -badarg (type-failed words get badarg marking, not just choice violations). Tests: parsestatus.test (new), usagemarking.test nocolour/leak GAP pins flipped + -parsestatus render parity diff --git a/src/tests/modules/punk/args/testsuites/args/recordcontinuation.test b/src/tests/modules/punk/args/testsuites/args/recordcontinuation.test new file mode 100644 index 00000000..bea60a16 --- /dev/null +++ b/src/tests/modules/punk/args/testsuites/args/recordcontinuation.test @@ -0,0 +1,182 @@ +package require tcltest + +package require punk::args +package require punk::ansi + +#Record-continuation token -& (G-045): an unquoted trailing -& element continues a +#definition record on the next line, rewritten internally to a backslash +#line-continuation so the assembled record is byte-identical to its +#backslash-continued equivalent. Key properties pinned here: +# - equivalence: a -& definition parses AND renders identically to its +# backslash-continuation twin (continuation is additive - backslash authoring +# unchanged) +# - the motivating case: constructed (string-built) definitions can use -& where +# a backslash-newline would be consumed by the building code's own quoting +# - collision/escape rules: a braced trailing {-&} is data; -& mid-line is data; +# a word merely ending in the characters -& is data; -& on a line inside a +# still-open braced value is data + +namespace eval ::testspace { + namespace import ::tcltest::* + variable common { + set result "" + } + + proc render_table {id} { + return [punk::ansi::ansistrip [punk::args::arg_error "" [punk::args::get_spec $id] -aserror 0]] + } + + #added 2026-07-12 (agent, G-045) + #the two ids are deliberately the same length: the id appears in the rendered + #synopsis, so differing lengths would change table geometry and defeat the + #render-equality comparison after the id string map + test recordcontinuation_amp_equivalent_to_backslash {a definition using trailing -& parses and renders identically to its backslash-continuation equivalent}\ + -setup $common -body { + punk::args::define { + @id -id ::testspace::rc_bsl + @cmd -name testspace::rc_fixture -summary\ + "Fixture."\ + -help\ + "CFIRST line. + CFLUSH line." + @opts + -o1 -type string -default od1 -help\ + "opt help line1 + opt help line2" + @values -min 1 -max 1 + v1 -type string -help\ + "val help." + } + punk::args::define { + @id -id ::testspace::rc_amp + @cmd -name testspace::rc_fixture -summary -& + "Fixture." -& + -help -& + "CFIRST line. + CFLUSH line." + @opts + -o1 -type string -default od1 -help -& + "opt help line1 + opt help line2" + @values -min 1 -max 1 + v1 -type string -help -& + "val help." + } + set r_b [punk::args::parse {-o1 X VAL1} withid ::testspace::rc_bsl] + set r_a [punk::args::parse {-o1 X VAL1} withid ::testspace::rc_amp] + lappend result [expr {[string map {rc_amp rc_bsl} $r_a] eq $r_b}] + set t_b [render_table ::testspace::rc_bsl] + set t_a [render_table ::testspace::rc_amp] + lappend result [expr {[string map {rc_amp rc_bsl} $t_a] eq $t_b}] + }\ + -cleanup { + punk::args::undefine ::testspace::rc_bsl 1 + punk::args::undefine ::testspace::rc_amp 1 + }\ + -result [list 1 1] + + #added 2026-07-12 (agent, G-045) - the motivating case: string-built definitions + #cannot use backslash-newline (consumed by the builder's own quoting) but can use -& + test recordcontinuation_amp_constructed_definition {a constructed (string-built) definition using -& chains record lines, including into a multi-line quoted value}\ + -setup $common -body { + set def "" + append def "@id -id ::testspace::rc_chain" \n + append def "@cmd -name testspace::rc_chain -summary \"Chain.\" -help \"chain help\"" \n + append def "@opts" \n + append def "-o1 -type string -&" \n + append def "-default HDEF -& " \n + append def "-help \"h line1" \n + append def "h line2\"" \n + append def "@values -min 0 -max 0" + punk::args::define $def + set argd [punk::args::parse {} withid ::testspace::rc_chain] + lappend result [dict get [dict get $argd opts] -o1] + set t [render_table ::testspace::rc_chain] + lappend result [expr {[string first "h line1" $t] >= 0}] + lappend result [expr {[string first "h line2" $t] >= 0}] + }\ + -cleanup { + punk::args::undefine ::testspace::rc_chain 1 + }\ + -result [list HDEF 1 1] + + #added 2026-07-12 (agent, G-045) + test recordcontinuation_amp_braced_escape_is_data {a braced trailing {-&} is a literal value - the record ends and the next line is a separate record}\ + -setup $common -body { + punk::args::define { + @id -id ::testspace::rc_braced + @cmd -name testspace::rc_braced -summary "Braced." -help "braced help" + @opts + -o1 -type string -default {-&} + -o2 -type string -default d2 + @values -min 0 -max 0 + } + set argd [punk::args::parse {} withid ::testspace::rc_braced] + lappend result [dict get [dict get $argd opts] -o1] + lappend result [dict get [dict get $argd opts] -o2] + }\ + -cleanup { + punk::args::undefine ::testspace::rc_braced 1 + }\ + -result [list -& d2] + + #added 2026-07-12 (agent, G-045) + test recordcontinuation_amp_midline_is_data {an unquoted -& that is not the last element on the line is an ordinary value}\ + -setup $common -body { + punk::args::define { + @id -id ::testspace::rc_midline + @cmd -name testspace::rc_midline -summary "Midline." -help "midline help" + @opts + -o1 -default -& -type string + @values -min 0 -max 0 + } + set argd [punk::args::parse {} withid ::testspace::rc_midline] + lappend result [dict get [dict get $argd opts] -o1] + }\ + -cleanup { + punk::args::undefine ::testspace::rc_midline 1 + }\ + -result [list -&] + + #added 2026-07-12 (agent, G-045) + test recordcontinuation_amp_wordend_is_data {a word merely ending in the characters -& is not a continuation token}\ + -setup $common -body { + punk::args::define { + @id -id ::testspace::rc_wordend + @cmd -name testspace::rc_wordend -summary "Wordend." -help "wordend help" + @opts + -o1 -type string -default abc-& + -o2 -type string -default d2 + @values -min 0 -max 0 + } + set argd [punk::args::parse {} withid ::testspace::rc_wordend] + lappend result [dict get [dict get $argd opts] -o1] + lappend result [dict get [dict get $argd opts] -o2] + }\ + -cleanup { + punk::args::undefine ::testspace::rc_wordend 1 + }\ + -result [list abc-& d2] + + #added 2026-07-12 (agent, G-045) + test recordcontinuation_amp_inside_braces_is_data {a line ending in -& inside a still-open braced value stays literal data}\ + -setup $common -body { + punk::args::define { + @id -id ::testspace::rc_inbrace + @cmd -name testspace::rc_inbrace -summary "Inbrace." -help "inbrace help" + @values -min 1 -max 1 + v1 -type string -help {AFIRST -& + ASECOND line} + } + set t [render_table ::testspace::rc_inbrace] + lappend result [expr {[string first "AFIRST -&" $t] >= 0}] + lappend result [expr {[string first "ASECOND line" $t] >= 0}] + }\ + -cleanup { + punk::args::undefine ::testspace::rc_inbrace 1 + }\ + -result [list 1 1] + + cleanupTests +} +namespace delete ::testspace