jn
/
punkshell
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Browse Source

argparsingtest 0.2.0: parser timing comparison with cold+warm first-call metrics 

New procs timeit/compare/discover_parsers/first_call for rough cross-parser
performance comparison (punk::args variants vs hand-rolled dict/switch/prefix,
tcllib opt/cmdline/tepam, argp), with PUNKARGS argdoc coverage throughout.

first_call times each parser twice in fresh child interps:
- cold: first call after package load (includes byte-compilation of the parser
  proc body and any backing-library procs it invokes)
- warm: the backing library is pre-exercised via per-library sacrificial
  warm-up scripts (library_warmups dict) so the timed call isolates per-proc
  first-use cost. Replaces an earlier punk::args-only
  'punk::args::parse {} withdef {}' warm hack that was unfair to the other
  libraries under test and only partially warmed punk::args itself.

Child interps pin package requires to the parent's loaded versions
(punk::args, punk::lib when present, argparsingtest) - plain package require
prefers stable versions, so children otherwise timed older stable copies
visible on the module paths instead of the dev module under test; under
Tcl 8.6 the punk::lib pin also supplies the lpop compat shim punk::args::parse
needs, unbreaking punk::args timing in bare child interps there.

compare renders first_cold/first_warm columns (table), nested cold/warm
sub-dicts (dict/json). New test suite under src/tests/modules/argparsingtest.

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 6 days ago
parent
2b0296c07b
commit
7a170a6a46
3 changed files with 1078 additions and 1 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 929
      src/modules/argparsingtest-999999.0a1.0.tm
  2. 5
      src/modules/argparsingtest-buildversion.txt
  3. 145
      src/tests/modules/argparsingtest/testsuites/argparsingtest/argparsingtest.test

929
src/modules/argparsingtest-999999.0a1.0.tm
View File

File diff suppressed because it is too large Load Diff

5
src/modules/argparsingtest-buildversion.txt
Unescape View File

@ -1,3 +1,6 @@
0.1.0
0.2.0
#First line must be a semantic version number
#all other lines are ignored.
#0.2.0 - added timeit/compare/discover_parsers procs for rough parser timing comparison; PUNKARGS argdoc blocks added to all procs
#0.2.0 - fixed timeit to call parsers with namespace-qualified name (workaround for upstream argp 0.2 caller-normalisation bug); noted the bug at the test1_argp registration site
#0.2.0 - first_call now times cold AND warm first calls per parser (warm = backing library pre-compiled via per-library sacrificial warm-up scripts in library_warmups); replaces the punk::args-only 'punk::args::parse {} withdef {}' warm hack which was unfair to other libraries; return shape now {parser {cold <us> warm <us>}}; compare table gains first_cold/first_warm columns

145
src/tests/modules/argparsingtest/testsuites/argparsingtest/argparsingtest.test
Unescape View File

@ -0,0 +1,145 @@
package require tcltest
package require argparsingtest
package require json
namespace eval ::testspace {
namespace import ::tcltest::*
#added 2026-07-17 (agent) - argp is an optional package; its load-time require is guarded by catch in the module
testConstraint argp_loaded [expr {![catch {package present argp}]}]
#added 2026-07-17 (agent)
test timeit_basic {Test that timeit -iterations 50 returns a dict with expected keys} -body {
set r [argparsingtest::timeit -iterations 50 -parsers {test1_switch}]
set ok 1
if {![dict exists $r test1_switch]} { set ok 0 }
if {$ok} {
set sub [dict get $r test1_switch]
if {![dict exists $sub us_per_call]} { set ok 0 }
if {$ok && ![string is double -strict [dict get $sub us_per_call]]} { set ok 0 }
if {$ok && [dict get $sub iterations] != 50} { set ok 0 }
}
set ok
} -result 1
#added 2026-07-17 (agent)
test compare_table {Test that compare -iterations 20 -format table returns string with Parser header and not loaded footer} -body {
set r [argparsingtest::compare -iterations 20 -format table]
set ok 1
if {[string first "Parser" $r] < 0} { set ok 0 }
if {$ok && [string first "not loaded:" $r] < 0} { set ok 0 }
set ok
} -result 1
#added 2026-07-17 (agent)
test compare_dict {Test that compare -iterations 20 -format dict returns a dict with not_loaded and results keys} -body {
set r [argparsingtest::compare -iterations 20 -format dict]
set ok 1
if {![dict exists $r not_loaded]} { set ok 0 }
if {$ok && ![dict exists $r results]} { set ok 0 }
set ok
} -result 1
#added 2026-07-17 (agent)
test compare_json {Test that compare -iterations 20 -format json returns valid JSON with results and not_loaded keys} -body {
set r [argparsingtest::compare -iterations 20 -format json]
set ok 1
if {[catch {set d [json::json2dict $r]} _err]} { set ok 0 }
if {$ok && ![dict exists $d results]} { set ok 0 }
if {$ok && ![dict exists $d not_loaded]} { set ok 0 }
set ok
} -result 1
#added 2026-07-17 (agent)
test compare_badformat {Test that compare -format bogus raises an error containing choiceviolation} -body {
set caught [catch {argparsingtest::compare -format bogus} err]
set ok 0
if {$caught && ([string first "choiceviolation" $err] >= 0 || [string first "must be one of" $err] >= 0)} {
set ok 1
}
set ok
} -result 1
#added 2026-07-17 (agent)
test discover_parsers_basic {Test that discover_parsers returns a dict with test1_switch loaded and 21 entries} -body {
set r [argparsingtest::discover_parsers]
set ok 1
if {![dict exists $r test1_switch]} { set ok 0 }
if {$ok && [dict get $r test1_switch] ne "loaded"} { set ok 0 }
if {$ok && [dict size $r] != 21} { set ok 0 }
set ok
} -result 1
#added 2026-07-17 (agent)
test first_call_handrolled {Test that first_call returns cold and warm float timings for a hand-rolled parser} -body {
set r [argparsingtest::first_call -parsers {test1_switch}]
set ok 1
if {![dict exists $r test1_switch]} { set ok 0 }
if {$ok} {
set sub [dict get $r test1_switch]
foreach mode {cold warm} {
if {![dict exists $sub $mode]} { set ok 0 ; break }
if {![string is double -strict [dict get $sub $mode]]} { set ok 0 ; break }
}
}
set ok
} -result 1
#added 2026-07-17 (agent)
test first_call_library_backed {Test that first_call returns cold and warm float timings for library-backed parsers (warm-up scripts must run cleanly)} -body {
set r [argparsingtest::first_call -parsers {test1_punkargs test1_punkargs_by_id test1_opt test1_cmdline_untyped test1_cmdline_typed test1_tepam}]
set ok 1
foreach {pname sub} $r {
foreach mode {cold warm} {
if {![dict exists $sub $mode]} { set ok "missing $mode for $pname" ; break }
if {![string is double -strict [dict get $sub $mode]]} { set ok "$pname $mode: [dict get $sub $mode]" ; break }
}
if {$ok ne 1} { break }
}
set ok
} -result 1
#added 2026-07-17 (agent)
test first_call_argp {Test first_call cold and warm for test1_argp when argp is loaded} -constraints argp_loaded -body {
set r [argparsingtest::first_call -parsers {test1_argp}]
set sub [dict get $r test1_argp]
set ok 1
foreach mode {cold warm} {
if {![string is double -strict [dict get $sub $mode]]} { set ok "$mode: [dict get $sub $mode]" ; break }
}
set ok
} -result 1
#added 2026-07-17 (agent)
test compare_table_first_columns {Test that compare table output has first_cold and first_warm columns} -body {
set r [argparsingtest::compare -iterations 20 -format table -parsers {test1_switch}]
set ok 1
if {[string first "first_cold" $r] < 0} { set ok 0 }
if {$ok && [string first "first_warm" $r] < 0} { set ok 0 }
set ok
} -result 1
#added 2026-07-17 (agent)
test compare_json_first_call_structure {Test that compare json first_call entries are objects with cold and warm keys} -body {
set r [argparsingtest::compare -iterations 20 -format json -parsers {test1_switch}]
set d [json::json2dict $r]
set ok 1
if {![dict exists $d first_call test1_switch cold]} { set ok 0 }
if {$ok && ![dict exists $d first_call test1_switch warm]} { set ok 0 }
set ok
} -result 1
#added 2026-07-17 (agent)
test timeit_nonexistent {Test that a nonexistent parser is recorded as error not re-raised} -body {
set r [argparsingtest::timeit -parsers {nonexistent_proc}]
set ok 1
if {![dict exists $r nonexistent_proc]} { set ok 0 }
if {$ok} {
set sub [dict get $r nonexistent_proc]
if {![dict exists $sub error]} { set ok 0 }
}
set ok
} -result 1
}
tcltest::cleanupTests
Write Preview
Loading…
Cancel
Save